2024-04-23T09:59:30.148 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-23T09:59:30.148 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/fs/test_o_trunc}} --owner scheduled_leonidus@teuthology --name leonidus-2024-04-23_09:49:14-fs-wip-lusov-quiescer-distro-default-smithi --archive /home/teuthworker/archive/leonidus-2024-04-23_09:49:14-fs-wip-lusov-quiescer-distro-default-smithi/7669527 --verbose -- /home/teuthworker/archive/leonidus-2024-04-23_09:49:14-fs-wip-lusov-quiescer-distro-default-smithi/7669527/orig.config.yaml 2024-04-23T09:59:30.208 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-23T09:59:30.284 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/leonidus-2024-04-23_09:49:14-fs-wip-lusov-quiescer-distro-default-smithi/7669527 branch: wip-lusov-quiescer description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/fs/test_o_trunc}} email: null first_in_suite: false job_id: '7669527' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true name: leonidus-2024-04-23_09:49:14-fs-wip-lusov-quiescer-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-lusov-quiescer ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount dry_run: true install: ceph: flavor: default sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - nowsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lusov-quiescer sha1: 0a6c3ed699031b80a2b419e7e795368719871394 owner: scheduled_leonidus@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 5306 sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f sleep_before_teardown: 0 subset: 200/111111 suite: fs suite_branch: wip-lusov-quiescer suite_path: /home/teuthworker/src/github.com_ceph_ceph_0a6c3ed699031b80a2b419e7e795368719871394/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph suite_sha1: 0a6c3ed699031b80a2b419e7e795368719871394 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi097.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM0WNRFBYBYss3yIKq2XyClIgavrv0TL6DyKwIJW2Kbt8xpIAqy+v+8evrwUOsVhzAB/bgh1WZtUJhOGBHgQNuQ= smithi183.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHmNbOyVA8FPXfGB64H1JUZvi1svb3mEk6C/0mNsuv7AoB8FybU9XiRlJcAJl0faz5nzSX6uRazc7nOGGJfQGmc= smithi196.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOqnwH0W7khzvQ8zQBerMGwNueAzp7eAbE9RrHlRMKs3G4nAcPK6F9S9fnEbRTvsLKWIL4/sCjSwYdhTGHrZXx4= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - fs/test_o_trunc.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-23_09:49:14 tube: smithi user: leonidus verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-23T09:59:30.284 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph_0a6c3ed699031b80a2b419e7e795368719871394/qa; will attempt to use it 2024-04-23T09:59:30.285 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph_0a6c3ed699031b80a2b419e7e795368719871394/qa/tasks 2024-04-23T09:59:30.285 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-23T09:59:30.287 INFO:teuthology.task.internal:Checking packages... 2024-04-23T09:59:30.310 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd69f13d827952bb07bb9b7973bca14f97d764d6f' 2024-04-23T09:59:30.311 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-23T09:59:30.311 INFO:teuthology.packaging:ref: None 2024-04-23T09:59:30.311 INFO:teuthology.packaging:tag: None 2024-04-23T09:59:30.311 INFO:teuthology.packaging:branch: wip-lusov-quiescer 2024-04-23T09:59:30.311 INFO:teuthology.packaging:sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-23T09:59:30.311 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-lusov-quiescer 2024-04-23T09:59:30.546 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3156.gd69f13d8 2024-04-23T09:59:30.547 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-23T09:59:30.555 INFO:teuthology.task.internal:no buildpackages task found 2024-04-23T09:59:30.555 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-23T09:59:30.561 INFO:teuthology.task.internal:Saving configuration 2024-04-23T09:59:30.583 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-23T09:59:30.589 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-23T09:59:30.614 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi097.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-23_09:49:14-fs-wip-lusov-quiescer-distro-default-smithi/7669527', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-23 09:51:17.821151', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM0WNRFBYBYss3yIKq2XyClIgavrv0TL6DyKwIJW2Kbt8xpIAqy+v+8evrwUOsVhzAB/bgh1WZtUJhOGBHgQNuQ='} 2024-04-23T09:59:30.633 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi183.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-23_09:49:14-fs-wip-lusov-quiescer-distro-default-smithi/7669527', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-23 09:51:17.822841', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHmNbOyVA8FPXfGB64H1JUZvi1svb3mEk6C/0mNsuv7AoB8FybU9XiRlJcAJl0faz5nzSX6uRazc7nOGGJfQGmc='} 2024-04-23T09:59:30.651 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi196.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-23_09:49:14-fs-wip-lusov-quiescer-distro-default-smithi/7669527', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-23 09:51:17.819808', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOqnwH0W7khzvQ8zQBerMGwNueAzp7eAbE9RrHlRMKs3G4nAcPK6F9S9fnEbRTvsLKWIL4/sCjSwYdhTGHrZXx4='} 2024-04-23T09:59:30.652 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-23T09:59:30.658 INFO:teuthology.task.internal:roles: ubuntu@smithi097.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-23T09:59:30.658 INFO:teuthology.task.internal:roles: ubuntu@smithi183.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-23T09:59:30.658 INFO:teuthology.task.internal:roles: ubuntu@smithi196.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-23T09:59:30.658 INFO:teuthology.run_tasks:Running task console_log... 2024-04-23T09:59:30.753 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f6a63c140d0>, signals=[15]) 2024-04-23T09:59:30.753 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-23T09:59:30.759 INFO:teuthology.task.internal:Opening connections... 2024-04-23T09:59:30.760 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi097.front.sepia.ceph.com 2024-04-23T09:59:30.761 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T09:59:30.852 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi183.front.sepia.ceph.com 2024-04-23T09:59:30.853 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T09:59:30.937 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi196.front.sepia.ceph.com 2024-04-23T09:59:30.938 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T09:59:31.018 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-23T09:59:31.024 DEBUG:teuthology.orchestra.run.smithi097:> uname -m 2024-04-23T09:59:31.043 INFO:teuthology.orchestra.run.smithi097.stdout:x86_64 2024-04-23T09:59:31.043 DEBUG:teuthology.orchestra.run.smithi097:> cat /etc/os-release 2024-04-23T09:59:31.102 INFO:teuthology.orchestra.run.smithi097.stdout:NAME="CentOS Stream" 2024-04-23T09:59:31.102 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION="9" 2024-04-23T09:59:31.102 INFO:teuthology.orchestra.run.smithi097.stdout:ID="centos" 2024-04-23T09:59:31.102 INFO:teuthology.orchestra.run.smithi097.stdout:ID_LIKE="rhel fedora" 2024-04-23T09:59:31.102 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION_ID="9" 2024-04-23T09:59:31.102 INFO:teuthology.orchestra.run.smithi097.stdout:PLATFORM_ID="platform:el9" 2024-04-23T09:59:31.102 INFO:teuthology.orchestra.run.smithi097.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-23T09:59:31.102 INFO:teuthology.orchestra.run.smithi097.stdout:ANSI_COLOR="0;31" 2024-04-23T09:59:31.102 INFO:teuthology.orchestra.run.smithi097.stdout:LOGO="fedora-logo-icon" 2024-04-23T09:59:31.102 INFO:teuthology.orchestra.run.smithi097.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-23T09:59:31.102 INFO:teuthology.orchestra.run.smithi097.stdout:HOME_URL="https://centos.org/" 2024-04-23T09:59:31.103 INFO:teuthology.orchestra.run.smithi097.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-23T09:59:31.103 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-23T09:59:31.103 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-23T09:59:31.103 INFO:teuthology.lock.ops:Updating smithi097.front.sepia.ceph.com on lock server 2024-04-23T09:59:31.127 DEBUG:teuthology.orchestra.run.smithi183:> uname -m 2024-04-23T09:59:31.146 INFO:teuthology.orchestra.run.smithi183.stdout:x86_64 2024-04-23T09:59:31.146 DEBUG:teuthology.orchestra.run.smithi183:> cat /etc/os-release 2024-04-23T09:59:31.204 INFO:teuthology.orchestra.run.smithi183.stdout:NAME="CentOS Stream" 2024-04-23T09:59:31.204 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION="9" 2024-04-23T09:59:31.204 INFO:teuthology.orchestra.run.smithi183.stdout:ID="centos" 2024-04-23T09:59:31.204 INFO:teuthology.orchestra.run.smithi183.stdout:ID_LIKE="rhel fedora" 2024-04-23T09:59:31.204 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION_ID="9" 2024-04-23T09:59:31.204 INFO:teuthology.orchestra.run.smithi183.stdout:PLATFORM_ID="platform:el9" 2024-04-23T09:59:31.204 INFO:teuthology.orchestra.run.smithi183.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-23T09:59:31.204 INFO:teuthology.orchestra.run.smithi183.stdout:ANSI_COLOR="0;31" 2024-04-23T09:59:31.204 INFO:teuthology.orchestra.run.smithi183.stdout:LOGO="fedora-logo-icon" 2024-04-23T09:59:31.205 INFO:teuthology.orchestra.run.smithi183.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-23T09:59:31.205 INFO:teuthology.orchestra.run.smithi183.stdout:HOME_URL="https://centos.org/" 2024-04-23T09:59:31.205 INFO:teuthology.orchestra.run.smithi183.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-23T09:59:31.205 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-23T09:59:31.205 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-23T09:59:31.205 INFO:teuthology.lock.ops:Updating smithi183.front.sepia.ceph.com on lock server 2024-04-23T09:59:31.227 DEBUG:teuthology.orchestra.run.smithi196:> uname -m 2024-04-23T09:59:31.246 INFO:teuthology.orchestra.run.smithi196.stdout:x86_64 2024-04-23T09:59:31.246 DEBUG:teuthology.orchestra.run.smithi196:> cat /etc/os-release 2024-04-23T09:59:31.305 INFO:teuthology.orchestra.run.smithi196.stdout:NAME="CentOS Stream" 2024-04-23T09:59:31.305 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION="9" 2024-04-23T09:59:31.306 INFO:teuthology.orchestra.run.smithi196.stdout:ID="centos" 2024-04-23T09:59:31.306 INFO:teuthology.orchestra.run.smithi196.stdout:ID_LIKE="rhel fedora" 2024-04-23T09:59:31.306 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION_ID="9" 2024-04-23T09:59:31.306 INFO:teuthology.orchestra.run.smithi196.stdout:PLATFORM_ID="platform:el9" 2024-04-23T09:59:31.306 INFO:teuthology.orchestra.run.smithi196.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-23T09:59:31.306 INFO:teuthology.orchestra.run.smithi196.stdout:ANSI_COLOR="0;31" 2024-04-23T09:59:31.306 INFO:teuthology.orchestra.run.smithi196.stdout:LOGO="fedora-logo-icon" 2024-04-23T09:59:31.306 INFO:teuthology.orchestra.run.smithi196.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-23T09:59:31.306 INFO:teuthology.orchestra.run.smithi196.stdout:HOME_URL="https://centos.org/" 2024-04-23T09:59:31.306 INFO:teuthology.orchestra.run.smithi196.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-23T09:59:31.306 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-23T09:59:31.306 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-23T09:59:31.307 INFO:teuthology.lock.ops:Updating smithi196.front.sepia.ceph.com on lock server 2024-04-23T09:59:31.327 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-23T09:59:31.337 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-23T09:59:31.343 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-23T09:59:31.343 DEBUG:teuthology.orchestra.run.smithi097:> test '!' -e /home/ubuntu/cephtest 2024-04-23T09:59:31.346 DEBUG:teuthology.orchestra.run.smithi183:> test '!' -e /home/ubuntu/cephtest 2024-04-23T09:59:31.348 DEBUG:teuthology.orchestra.run.smithi196:> test '!' -e /home/ubuntu/cephtest 2024-04-23T09:59:31.364 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-23T09:59:31.370 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-23T09:59:31.370 DEBUG:teuthology.orchestra.run.smithi097:> test -z $(ls -A /var/lib/ceph) 2024-04-23T09:59:31.403 DEBUG:teuthology.orchestra.run.smithi183:> test -z $(ls -A /var/lib/ceph) 2024-04-23T09:59:31.406 DEBUG:teuthology.orchestra.run.smithi196:> test -z $(ls -A /var/lib/ceph) 2024-04-23T09:59:31.554 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-23T09:59:31.629 INFO:teuthology.run_tasks:Running task kernel... 2024-04-23T09:59:31.644 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-04-23T09:59:31.644 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-04-23T09:59:31.645 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-04-23T09:59:31.645 DEBUG:teuthology.orchestra.run.smithi097:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-23T09:59:31.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T09:59:31.662 DEBUG:teuthology.orchestra.run.smithi097:> uname -r 2024-04-23T09:59:31.720 INFO:teuthology.orchestra.run.smithi097.stdout:5.14.0-437.el9.x86_64 2024-04-23T09:59:31.720 INFO:teuthology.task.kernel:Running kernel on smithi097: 5.14.0-437.el9.x86_64 2024-04-23T09:59:31.720 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum install -y kernel 2024-04-23T09:59:35.044 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - BaseOS 5.0 MB/s | 8.0 MB 00:01 2024-04-23T09:59:39.222 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - AppStream 6.1 MB/s | 19 MB 00:03 2024-04-23T09:59:43.995 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - CRB 5.3 MB/s | 6.2 MB 00:01 2024-04-23T09:59:46.697 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - Extras packages 12 kB/s | 16 kB 00:01 2024-04-23T09:59:47.965 INFO:teuthology.orchestra.run.smithi097.stdout:Extra Packages for Enterprise Linux 18 MB/s | 21 MB 00:01 2024-04-23T09:59:53.106 INFO:teuthology.orchestra.run.smithi097.stdout:lab-extras 11 kB/s | 1.7 kB 00:00 2024-04-23T09:59:54.962 INFO:teuthology.orchestra.run.smithi097.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-23T09:59:54.962 INFO:teuthology.orchestra.run.smithi097.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-23T09:59:55.051 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-04-23T09:59:55.052 INFO:teuthology.orchestra.run.smithi097.stdout:Nothing to do. 2024-04-23T09:59:55.053 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-04-23T09:59:55.149 DEBUG:teuthology.orchestra.run.smithi097:> echo no | sudo yum reinstall kernel || true 2024-04-23T09:59:55.646 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:00:02 ago on Tue 23 Apr 2024 09:59:53 AM UTC. 2024-04-23T09:59:55.764 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-04-23T09:59:55.777 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-04-23T09:59:55.778 INFO:teuthology.orchestra.run.smithi097.stdout: Package Architecture Version Repository Size 2024-04-23T09:59:55.778 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-04-23T09:59:55.778 INFO:teuthology.orchestra.run.smithi097.stdout:Reinstalling: 2024-04-23T09:59:55.778 INFO:teuthology.orchestra.run.smithi097.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-23T09:59:55.778 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T09:59:55.778 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-04-23T09:59:55.778 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-04-23T09:59:55.778 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T09:59:55.778 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 255 k 2024-04-23T09:59:55.778 INFO:teuthology.orchestra.run.smithi097.stdout:Installed size: 0 2024-04-23T09:59:55.778 INFO:teuthology.orchestra.run.smithi097.stderr:Operation aborted. 2024-04-23T09:59:55.840 INFO:teuthology.orchestra.run.smithi097.stdout:Is this ok [y/N]: 2024-04-23T09:59:55.840 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum reinstall -y kernel || true 2024-04-23T09:59:56.335 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:00:03 ago on Tue 23 Apr 2024 09:59:53 AM UTC. 2024-04-23T09:59:56.450 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-04-23T09:59:56.451 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-04-23T09:59:56.451 INFO:teuthology.orchestra.run.smithi097.stdout: Package Architecture Version Repository Size 2024-04-23T09:59:56.451 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-04-23T09:59:56.451 INFO:teuthology.orchestra.run.smithi097.stdout:Reinstalling: 2024-04-23T09:59:56.451 INFO:teuthology.orchestra.run.smithi097.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-23T09:59:56.452 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T09:59:56.452 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-04-23T09:59:56.452 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-04-23T09:59:56.452 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T09:59:56.452 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 255 k 2024-04-23T09:59:56.452 INFO:teuthology.orchestra.run.smithi097.stdout:Installed size: 0 2024-04-23T09:59:56.452 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-04-23T09:59:56.808 INFO:teuthology.orchestra.run.smithi097.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.1 MB/s | 255 kB 00:00 2024-04-23T09:59:56.808 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-04-23T09:59:56.809 INFO:teuthology.orchestra.run.smithi097.stdout:Total 714 kB/s | 255 kB 00:00 2024-04-23T09:59:56.829 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-04-23T09:59:57.035 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-04-23T09:59:57.035 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-04-23T09:59:57.040 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-04-23T09:59:57.041 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-04-23T09:59:57.182 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-04-23T09:59:57.276 INFO:teuthology.orchestra.run.smithi097.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-23T09:59:57.276 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-23T09:59:57.929 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-23T09:59:57.930 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-23T09:59:58.631 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-23T09:59:58.632 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T09:59:58.632 INFO:teuthology.orchestra.run.smithi097.stdout:Reinstalled: 2024-04-23T09:59:58.632 INFO:teuthology.orchestra.run.smithi097.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-23T09:59:58.632 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T09:59:58.632 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-04-23T09:59:58.799 DEBUG:teuthology.orchestra.run.smithi097:> rpm -q kernel | sort -rV | head -n 1 2024-04-23T09:59:58.846 INFO:teuthology.orchestra.run.smithi097.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-23T09:59:58.846 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-23T09:59:58.846 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-23T09:59:58.846 DEBUG:teuthology.parallel:result is None 2024-04-23T09:59:58.846 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-23T09:59:58.854 INFO:teuthology.task.internal:Creating test directory... 2024-04-23T09:59:58.854 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-23T09:59:58.856 DEBUG:teuthology.orchestra.run.smithi183:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-23T09:59:58.858 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-23T09:59:58.876 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-23T09:59:58.896 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-23T09:59:58.903 INFO:teuthology.task.internal:Creating archive directory... 2024-04-23T09:59:58.903 DEBUG:teuthology.orchestra.run.smithi097:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-23T09:59:58.912 DEBUG:teuthology.orchestra.run.smithi183:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-23T09:59:58.917 DEBUG:teuthology.orchestra.run.smithi196:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-23T09:59:59.172 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-23T09:59:59.180 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-23T09:59:59.180 DEBUG:teuthology.orchestra.run.smithi183:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-23T09:59:59.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T09:59:59.196 DEBUG:teuthology.orchestra.run.smithi196:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-23T09:59:59.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T09:59:59.213 DEBUG:teuthology.orchestra.run.smithi097:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-23T09:59:59.215 DEBUG:teuthology.orchestra.run.smithi183:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-23T09:59:59.241 DEBUG:teuthology.orchestra.run.smithi196:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-23T09:59:59.248 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-23T09:59:59.260 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-23T09:59:59.281 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-23T09:59:59.294 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-23T09:59:59.311 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-23T09:59:59.312 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-23T09:59:59.314 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-23T09:59:59.321 INFO:teuthology.task.internal:Configuring sudo... 2024-04-23T09:59:59.321 DEBUG:teuthology.orchestra.run.smithi097:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-23T09:59:59.323 DEBUG:teuthology.orchestra.run.smithi183:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-23T09:59:59.357 DEBUG:teuthology.orchestra.run.smithi196:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-23T09:59:59.386 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-23T09:59:59.395 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-23T09:59:59.395 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-23T09:59:59.398 DEBUG:teuthology.orchestra.run.smithi183:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-23T09:59:59.425 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-23T09:59:59.443 DEBUG:teuthology.orchestra.run.smithi097:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-23T09:59:59.495 DEBUG:teuthology.orchestra.run.smithi097:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-23T09:59:59.536 DEBUG:teuthology.orchestra.run.smithi097:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-23T09:59:59.619 DEBUG:teuthology.orchestra.run.smithi097:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-23T09:59:59.656 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-23T09:59:59.657 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-23T09:59:59.719 DEBUG:teuthology.orchestra.run.smithi183:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-23T09:59:59.756 DEBUG:teuthology.orchestra.run.smithi183:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-23T09:59:59.811 DEBUG:teuthology.orchestra.run.smithi183:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-23T09:59:59.898 DEBUG:teuthology.orchestra.run.smithi183:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-23T09:59:59.934 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-23T09:59:59.935 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-23T10:00:00.001 DEBUG:teuthology.orchestra.run.smithi196:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-23T10:00:00.041 DEBUG:teuthology.orchestra.run.smithi196:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-23T10:00:00.105 DEBUG:teuthology.orchestra.run.smithi196:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-23T10:00:00.185 DEBUG:teuthology.orchestra.run.smithi196:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-23T10:00:00.231 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-23T10:00:00.231 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-23T10:00:00.298 DEBUG:teuthology.orchestra.run.smithi097:> sudo service rsyslog restart 2024-04-23T10:00:00.300 DEBUG:teuthology.orchestra.run.smithi183:> sudo service rsyslog restart 2024-04-23T10:00:00.302 DEBUG:teuthology.orchestra.run.smithi196:> sudo service rsyslog restart 2024-04-23T10:00:00.337 INFO:teuthology.orchestra.run.smithi097.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-23T10:00:00.338 INFO:teuthology.orchestra.run.smithi183.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-23T10:00:00.375 INFO:teuthology.orchestra.run.smithi196.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-23T10:00:00.879 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-23T10:00:00.887 INFO:teuthology.task.internal:Starting timer... 2024-04-23T10:00:00.887 INFO:teuthology.run_tasks:Running task pcp... 2024-04-23T10:00:00.982 INFO:teuthology.run_tasks:Running task selinux... 2024-04-23T10:00:00.991 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-23T10:00:00.991 DEBUG:teuthology.orchestra.run.smithi097:> sudo service auditd rotate 2024-04-23T10:00:01.061 INFO:teuthology.orchestra.run.smithi097.stdout:Rotating logs: 2024-04-23T10:00:01.063 DEBUG:teuthology.orchestra.run.smithi183:> sudo service auditd rotate 2024-04-23T10:00:01.126 INFO:teuthology.orchestra.run.smithi183.stdout:Rotating logs: 2024-04-23T10:00:01.128 DEBUG:teuthology.orchestra.run.smithi196:> sudo service auditd rotate 2024-04-23T10:00:01.187 INFO:teuthology.orchestra.run.smithi196.stdout:Rotating logs: 2024-04-23T10:00:01.189 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-23T10:00:01.189 DEBUG:teuthology.orchestra.run.smithi097:> /usr/sbin/getenforce 2024-04-23T10:00:01.214 INFO:teuthology.orchestra.run.smithi097.stdout:Permissive 2024-04-23T10:00:01.214 DEBUG:teuthology.orchestra.run.smithi183:> /usr/sbin/getenforce 2024-04-23T10:00:01.241 INFO:teuthology.orchestra.run.smithi183.stdout:Permissive 2024-04-23T10:00:01.241 DEBUG:teuthology.orchestra.run.smithi196:> /usr/sbin/getenforce 2024-04-23T10:00:01.263 INFO:teuthology.orchestra.run.smithi196.stdout:Permissive 2024-04-23T10:00:01.263 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi097.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi183.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi196.front.sepia.ceph.com': 'permissive'} 2024-04-23T10:00:01.264 DEBUG:teuthology.orchestra.run.smithi097:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-23T10:00:01.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T10:00:01.289 DEBUG:teuthology.orchestra.run.smithi183:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-23T10:00:01.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T10:00:01.315 DEBUG:teuthology.orchestra.run.smithi196:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-23T10:00:01.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T10:00:01.343 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-23T10:00:01.343 DEBUG:teuthology.orchestra.run.smithi097:> sudo /usr/sbin/setenforce permissive 2024-04-23T10:00:01.392 DEBUG:teuthology.orchestra.run.smithi183:> sudo /usr/sbin/setenforce permissive 2024-04-23T10:00:01.419 DEBUG:teuthology.orchestra.run.smithi196:> sudo /usr/sbin/setenforce permissive 2024-04-23T10:00:01.446 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-23T10:00:01.456 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-23T10:00:01.478 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-23T10:00:01.507 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi097.front.sepia.ceph.com,smithi183.front.sepia.ceph.com,smithi196.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-23T10:04:23.013 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi097.front.sepia.ceph.com'), Remote(name='ubuntu@smithi183.front.sepia.ceph.com'), Remote(name='ubuntu@smithi196.front.sepia.ceph.com')] 2024-04-23T10:04:23.015 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-04-23T10:04:23.016 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T10:04:23.091 DEBUG:teuthology.orchestra.run.smithi097:> true 2024-04-23T10:04:23.165 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-04-23T10:04:23.166 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi183.front.sepia.ceph.com' 2024-04-23T10:04:23.166 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T10:04:23.245 DEBUG:teuthology.orchestra.run.smithi183:> true 2024-04-23T10:04:23.329 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi183.front.sepia.ceph.com' 2024-04-23T10:04:23.330 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-04-23T10:04:23.331 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T10:04:23.452 DEBUG:teuthology.orchestra.run.smithi196:> true 2024-04-23T10:04:23.531 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-04-23T10:04:23.532 INFO:teuthology.run_tasks:Running task clock... 2024-04-23T10:04:23.546 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-23T10:04:23.546 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-23T10:04:23.547 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-23T10:04:23.549 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-23T10:04:23.549 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-23T10:04:23.552 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-23T10:04:23.552 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-23T10:04:23.578 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-23T10:04:23.578 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-23T10:04:23.592 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-23T10:04:23.592 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-23T10:04:23.603 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-23T10:04:23.615 INFO:teuthology.orchestra.run.smithi097.stderr:sudo: ntpd: command not found 2024-04-23T10:04:23.615 INFO:teuthology.orchestra.run.smithi183.stderr:sudo: ntpd: command not found 2024-04-23T10:04:23.617 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-23T10:04:23.625 INFO:teuthology.orchestra.run.smithi097.stdout:506 Cannot talk to daemon 2024-04-23T10:04:23.626 INFO:teuthology.orchestra.run.smithi183.stdout:506 Cannot talk to daemon 2024-04-23T10:04:23.638 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-23T10:04:23.639 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-23T10:04:23.640 INFO:teuthology.orchestra.run.smithi196.stderr:sudo: ntpd: command not found 2024-04-23T10:04:23.651 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-23T10:04:23.653 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-23T10:04:23.653 INFO:teuthology.orchestra.run.smithi196.stdout:506 Cannot talk to daemon 2024-04-23T10:04:23.665 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-23T10:04:23.678 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-23T10:04:23.708 INFO:teuthology.orchestra.run.smithi183.stderr:bash: line 1: ntpq: command not found 2024-04-23T10:04:23.711 INFO:teuthology.orchestra.run.smithi183.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-23T10:04:23.712 INFO:teuthology.orchestra.run.smithi183.stdout:=============================================================================== 2024-04-23T10:04:23.712 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T10:04:23.712 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T10:04:23.712 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T10:04:23.712 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T10:04:23.713 INFO:teuthology.orchestra.run.smithi097.stderr:bash: line 1: ntpq: command not found 2024-04-23T10:04:23.717 INFO:teuthology.orchestra.run.smithi097.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-23T10:04:23.717 INFO:teuthology.orchestra.run.smithi097.stdout:=============================================================================== 2024-04-23T10:04:23.717 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T10:04:23.717 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T10:04:23.717 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T10:04:23.717 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T10:04:23.736 INFO:teuthology.orchestra.run.smithi196.stderr:bash: line 1: ntpq: command not found 2024-04-23T10:04:23.740 INFO:teuthology.orchestra.run.smithi196.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-23T10:04:23.740 INFO:teuthology.orchestra.run.smithi196.stdout:=============================================================================== 2024-04-23T10:04:23.740 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T10:04:23.740 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T10:04:23.741 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T10:04:23.741 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T10:04:23.742 INFO:teuthology.run_tasks:Running task install... 2024-04-23T10:04:23.754 DEBUG:teuthology.task.install:project ceph 2024-04-23T10:04:23.754 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f'}} 2024-04-23T10:04:23.755 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f'} 2024-04-23T10:04:23.755 INFO:teuthology.task.install:Using flavor: default 2024-04-23T10:04:23.761 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-23T10:04:23.761 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-23T10:04:23.762 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-23T10:04:23.763 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-23T10:04:23.764 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-23T10:04:23.764 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-23T10:04:23.765 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-23T10:04:23.765 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-23T10:04:23.998 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-23T10:04:23.998 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-23T10:04:24.007 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-23T10:04:24.007 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-23T10:04:24.013 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-23T10:04:24.013 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-23T10:04:24.257 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-23T10:04:24.257 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-23T10:04:24.257 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-23T10:04:24.282 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-23T10:04:24.282 DEBUG:teuthology.orchestra.run.smithi183:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-23T10:04:24.341 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-23T10:04:24.341 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-23T10:04:24.341 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-23T10:04:24.344 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-23T10:04:24.344 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-23T10:04:24.344 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-23T10:04:24.349 DEBUG:teuthology.orchestra.run.smithi183:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-23T10:04:24.370 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-23T10:04:24.370 DEBUG:teuthology.orchestra.run.smithi097:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-23T10:04:24.371 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-23T10:04:24.371 DEBUG:teuthology.orchestra.run.smithi196:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-23T10:04:24.423 DEBUG:teuthology.orchestra.run.smithi183:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-23T10:04:24.435 DEBUG:teuthology.orchestra.run.smithi097:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-23T10:04:24.437 DEBUG:teuthology.orchestra.run.smithi196:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-23T10:04:24.485 INFO:teuthology.orchestra.run.smithi183.stdout:check_obsoletes = 1 2024-04-23T10:04:24.487 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum clean all 2024-04-23T10:04:24.510 DEBUG:teuthology.orchestra.run.smithi097:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-23T10:04:24.511 DEBUG:teuthology.orchestra.run.smithi196:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-23T10:04:24.573 INFO:teuthology.orchestra.run.smithi097.stdout:check_obsoletes = 1 2024-04-23T10:04:24.574 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum clean all 2024-04-23T10:04:24.575 INFO:teuthology.orchestra.run.smithi196.stdout:check_obsoletes = 1 2024-04-23T10:04:24.577 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum clean all 2024-04-23T10:04:24.713 INFO:teuthology.orchestra.run.smithi183.stdout:48 files removed 2024-04-23T10:04:24.741 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-23T10:04:24.799 INFO:teuthology.orchestra.run.smithi196.stdout:48 files removed 2024-04-23T10:04:24.800 INFO:teuthology.orchestra.run.smithi097.stdout:48 files removed 2024-04-23T10:04:24.826 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-23T10:04:24.827 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-23T10:04:25.676 INFO:teuthology.orchestra.run.smithi183.stdout:ceph packages for x86_64 123 kB/s | 83 kB 00:00 2024-04-23T10:04:25.750 INFO:teuthology.orchestra.run.smithi097.stdout:ceph packages for x86_64 127 kB/s | 83 kB 00:00 2024-04-23T10:04:25.790 INFO:teuthology.orchestra.run.smithi196.stdout:ceph packages for x86_64 118 kB/s | 83 kB 00:00 2024-04-23T10:04:26.377 INFO:teuthology.orchestra.run.smithi183.stdout:ceph noarch packages 18 kB/s | 12 kB 00:00 2024-04-23T10:04:26.490 INFO:teuthology.orchestra.run.smithi196.stdout:ceph noarch packages 18 kB/s | 12 kB 00:00 2024-04-23T10:04:26.517 INFO:teuthology.orchestra.run.smithi097.stdout:ceph noarch packages 16 kB/s | 12 kB 00:00 2024-04-23T10:04:27.103 INFO:teuthology.orchestra.run.smithi183.stdout:ceph source packages 2.7 kB/s | 1.9 kB 00:00 2024-04-23T10:04:27.151 INFO:teuthology.orchestra.run.smithi097.stdout:ceph source packages 3.2 kB/s | 1.9 kB 00:00 2024-04-23T10:04:27.166 INFO:teuthology.orchestra.run.smithi196.stdout:ceph source packages 3.0 kB/s | 1.9 kB 00:00 2024-04-23T10:04:27.803 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.0 MB 00:00 2024-04-23T10:04:27.976 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.0 MB 00:00 2024-04-23T10:04:28.058 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - BaseOS 9.4 MB/s | 8.0 MB 00:00 2024-04-23T10:04:29.944 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - AppStream 21 MB/s | 19 MB 00:00 2024-04-23T10:04:30.262 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - AppStream 13 MB/s | 19 MB 00:01 2024-04-23T10:04:31.758 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - AppStream 7.2 MB/s | 19 MB 00:02 2024-04-23T10:04:35.954 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - CRB 2.9 MB/s | 6.2 MB 00:02 2024-04-23T10:04:37.521 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - Extras packages 38 kB/s | 16 kB 00:00 2024-04-23T10:04:39.133 INFO:teuthology.orchestra.run.smithi183.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-04-23T10:04:40.750 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - CRB 1.1 MB/s | 6.2 MB 00:05 2024-04-23T10:04:42.119 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - CRB 736 kB/s | 6.2 MB 00:08 2024-04-23T10:04:42.325 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - Extras packages 37 kB/s | 16 kB 00:00 2024-04-23T10:04:43.760 INFO:teuthology.orchestra.run.smithi196.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-04-23T10:04:43.861 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - Extras packages 27 kB/s | 16 kB 00:00 2024-04-23T10:04:44.283 INFO:teuthology.orchestra.run.smithi183.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-04-23T10:04:45.849 INFO:teuthology.orchestra.run.smithi097.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-04-23T10:04:46.418 INFO:teuthology.orchestra.run.smithi183.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-23T10:04:46.418 INFO:teuthology.orchestra.run.smithi183.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-23T10:04:46.429 INFO:teuthology.orchestra.run.smithi183.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-23T10:04:46.430 INFO:teuthology.orchestra.run.smithi183.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-23T10:04:46.431 INFO:teuthology.orchestra.run.smithi183.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-23T10:04:46.431 INFO:teuthology.orchestra.run.smithi183.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-23T10:04:46.433 INFO:teuthology.orchestra.run.smithi183.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-23T10:04:46.435 INFO:teuthology.orchestra.run.smithi183.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-23T10:04:46.436 INFO:teuthology.orchestra.run.smithi183.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-23T10:04:46.496 INFO:teuthology.orchestra.run.smithi183.stdout:Dependencies resolved. 2024-04-23T10:04:46.516 INFO:teuthology.orchestra.run.smithi183.stdout:======================================================================================= 2024-04-23T10:04:46.516 INFO:teuthology.orchestra.run.smithi183.stdout: Package Arch Version Repository Size 2024-04-23T10:04:46.516 INFO:teuthology.orchestra.run.smithi183.stdout:======================================================================================= 2024-04-23T10:04:46.516 INFO:teuthology.orchestra.run.smithi183.stdout:Installing: 2024-04-23T10:04:46.516 INFO:teuthology.orchestra.run.smithi183.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-23T10:04:46.516 INFO:teuthology.orchestra.run.smithi183.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-23T10:04:46.516 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-23T10:04:46.516 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-23T10:04:46.516 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-23T10:04:46.517 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-23T10:04:46.517 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-23T10:04:46.517 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-23T10:04:46.517 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-23T10:04:46.517 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-23T10:04:46.517 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-23T10:04:46.517 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-23T10:04:46.517 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-23T10:04:46.517 INFO:teuthology.orchestra.run.smithi183.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-23T10:04:46.517 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-23T10:04:46.517 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-23T10:04:46.517 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-23T10:04:46.517 INFO:teuthology.orchestra.run.smithi183.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-23T10:04:46.517 INFO:teuthology.orchestra.run.smithi183.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-23T10:04:46.518 INFO:teuthology.orchestra.run.smithi183.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-23T10:04:46.518 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-23T10:04:46.518 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-23T10:04:46.518 INFO:teuthology.orchestra.run.smithi183.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-23T10:04:46.518 INFO:teuthology.orchestra.run.smithi183.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-23T10:04:46.518 INFO:teuthology.orchestra.run.smithi183.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-23T10:04:46.518 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-23T10:04:46.518 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-23T10:04:46.518 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-23T10:04:46.518 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-23T10:04:46.518 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-23T10:04:46.518 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-23T10:04:46.518 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-23T10:04:46.519 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-23T10:04:46.519 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-23T10:04:46.519 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-23T10:04:46.519 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-23T10:04:46.519 INFO:teuthology.orchestra.run.smithi183.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-23T10:04:46.519 INFO:teuthology.orchestra.run.smithi183.stdout:Upgrading: 2024-04-23T10:04:46.519 INFO:teuthology.orchestra.run.smithi183.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-23T10:04:46.519 INFO:teuthology.orchestra.run.smithi183.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-23T10:04:46.519 INFO:teuthology.orchestra.run.smithi183.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-23T10:04:46.519 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-23T10:04:46.519 INFO:teuthology.orchestra.run.smithi183.stdout:Installing dependencies: 2024-04-23T10:04:46.519 INFO:teuthology.orchestra.run.smithi183.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-23T10:04:46.519 INFO:teuthology.orchestra.run.smithi183.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-23T10:04:46.520 INFO:teuthology.orchestra.run.smithi183.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-23T10:04:46.520 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-23T10:04:46.520 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-23T10:04:46.520 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-23T10:04:46.520 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-23T10:04:46.520 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-23T10:04:46.520 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-23T10:04:46.520 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-23T10:04:46.520 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-23T10:04:46.520 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-23T10:04:46.520 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-23T10:04:46.520 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-23T10:04:46.520 INFO:teuthology.orchestra.run.smithi183.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-23T10:04:46.520 INFO:teuthology.orchestra.run.smithi183.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-23T10:04:46.521 INFO:teuthology.orchestra.run.smithi183.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-23T10:04:46.521 INFO:teuthology.orchestra.run.smithi183.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-23T10:04:46.521 INFO:teuthology.orchestra.run.smithi183.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-23T10:04:46.521 INFO:teuthology.orchestra.run.smithi183.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-23T10:04:46.521 INFO:teuthology.orchestra.run.smithi183.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-23T10:04:46.521 INFO:teuthology.orchestra.run.smithi183.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-23T10:04:46.521 INFO:teuthology.orchestra.run.smithi183.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-23T10:04:46.521 INFO:teuthology.orchestra.run.smithi183.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-23T10:04:46.521 INFO:teuthology.orchestra.run.smithi183.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-23T10:04:46.521 INFO:teuthology.orchestra.run.smithi183.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-23T10:04:46.521 INFO:teuthology.orchestra.run.smithi183.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-23T10:04:46.521 INFO:teuthology.orchestra.run.smithi183.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-23T10:04:46.521 INFO:teuthology.orchestra.run.smithi183.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-23T10:04:46.521 INFO:teuthology.orchestra.run.smithi183.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-23T10:04:46.522 INFO:teuthology.orchestra.run.smithi183.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-23T10:04:46.522 INFO:teuthology.orchestra.run.smithi183.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-23T10:04:46.522 INFO:teuthology.orchestra.run.smithi183.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-23T10:04:46.522 INFO:teuthology.orchestra.run.smithi183.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-23T10:04:46.522 INFO:teuthology.orchestra.run.smithi183.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-23T10:04:46.522 INFO:teuthology.orchestra.run.smithi183.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-23T10:04:46.522 INFO:teuthology.orchestra.run.smithi183.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-23T10:04:46.522 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-23T10:04:46.522 INFO:teuthology.orchestra.run.smithi183.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-23T10:04:46.522 INFO:teuthology.orchestra.run.smithi183.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-23T10:04:46.522 INFO:teuthology.orchestra.run.smithi183.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-23T10:04:46.522 INFO:teuthology.orchestra.run.smithi183.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-23T10:04:46.522 INFO:teuthology.orchestra.run.smithi183.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-23T10:04:46.523 INFO:teuthology.orchestra.run.smithi183.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-23T10:04:46.523 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-23T10:04:46.523 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-23T10:04:46.523 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-23T10:04:46.523 INFO:teuthology.orchestra.run.smithi183.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-23T10:04:46.523 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-23T10:04:46.523 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-23T10:04:46.523 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-23T10:04:46.523 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-23T10:04:46.523 INFO:teuthology.orchestra.run.smithi183.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-23T10:04:46.524 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-23T10:04:46.524 INFO:teuthology.orchestra.run.smithi183.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-23T10:04:46.524 INFO:teuthology.orchestra.run.smithi183.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-23T10:04:46.524 INFO:teuthology.orchestra.run.smithi183.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-23T10:04:46.524 INFO:teuthology.orchestra.run.smithi183.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-23T10:04:46.524 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-23T10:04:46.524 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-23T10:04:46.524 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-23T10:04:46.524 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-23T10:04:46.524 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-23T10:04:46.524 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-23T10:04:46.524 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-23T10:04:46.524 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-23T10:04:46.525 INFO:teuthology.orchestra.run.smithi183.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-23T10:04:46.525 INFO:teuthology.orchestra.run.smithi183.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-23T10:04:46.525 INFO:teuthology.orchestra.run.smithi183.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-23T10:04:46.525 INFO:teuthology.orchestra.run.smithi183.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-23T10:04:46.525 INFO:teuthology.orchestra.run.smithi183.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-23T10:04:46.525 INFO:teuthology.orchestra.run.smithi183.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-23T10:04:46.525 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-23T10:04:46.525 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-23T10:04:46.525 INFO:teuthology.orchestra.run.smithi183.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-23T10:04:46.525 INFO:teuthology.orchestra.run.smithi183.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-23T10:04:46.525 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-23T10:04:46.525 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-23T10:04:46.525 INFO:teuthology.orchestra.run.smithi183.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-23T10:04:46.525 INFO:teuthology.orchestra.run.smithi183.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-23T10:04:46.525 INFO:teuthology.orchestra.run.smithi183.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-23T10:04:46.526 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-23T10:04:46.526 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-23T10:04:46.526 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-23T10:04:46.526 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-23T10:04:46.526 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-23T10:04:46.526 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-23T10:04:46.526 INFO:teuthology.orchestra.run.smithi183.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-23T10:04:46.526 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-23T10:04:46.526 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-23T10:04:46.526 INFO:teuthology.orchestra.run.smithi183.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-23T10:04:46.526 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-23T10:04:46.526 INFO:teuthology.orchestra.run.smithi183.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-23T10:04:46.526 INFO:teuthology.orchestra.run.smithi183.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-23T10:04:46.527 INFO:teuthology.orchestra.run.smithi183.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-23T10:04:46.527 INFO:teuthology.orchestra.run.smithi183.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-23T10:04:46.527 INFO:teuthology.orchestra.run.smithi183.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-23T10:04:46.527 INFO:teuthology.orchestra.run.smithi183.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-23T10:04:46.527 INFO:teuthology.orchestra.run.smithi183.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-23T10:04:46.527 INFO:teuthology.orchestra.run.smithi183.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-23T10:04:46.527 INFO:teuthology.orchestra.run.smithi183.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-23T10:04:46.527 INFO:teuthology.orchestra.run.smithi183.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-23T10:04:46.527 INFO:teuthology.orchestra.run.smithi183.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-23T10:04:46.527 INFO:teuthology.orchestra.run.smithi183.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-23T10:04:46.527 INFO:teuthology.orchestra.run.smithi183.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-23T10:04:46.527 INFO:teuthology.orchestra.run.smithi183.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-23T10:04:46.527 INFO:teuthology.orchestra.run.smithi183.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-23T10:04:46.527 INFO:teuthology.orchestra.run.smithi183.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-23T10:04:46.528 INFO:teuthology.orchestra.run.smithi183.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-23T10:04:46.528 INFO:teuthology.orchestra.run.smithi183.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-23T10:04:46.528 INFO:teuthology.orchestra.run.smithi183.stdout:Installing weak dependencies: 2024-04-23T10:04:46.528 INFO:teuthology.orchestra.run.smithi183.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-23T10:04:46.528 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-23T10:04:46.528 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction Summary 2024-04-23T10:04:46.528 INFO:teuthology.orchestra.run.smithi183.stdout:======================================================================================= 2024-04-23T10:04:46.528 INFO:teuthology.orchestra.run.smithi183.stdout:Install 148 Packages 2024-04-23T10:04:46.528 INFO:teuthology.orchestra.run.smithi183.stdout:Upgrade 4 Packages 2024-04-23T10:04:46.528 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-23T10:04:46.529 INFO:teuthology.orchestra.run.smithi183.stdout:Total download size: 222 M 2024-04-23T10:04:46.529 INFO:teuthology.orchestra.run.smithi183.stdout:Downloading Packages: 2024-04-23T10:04:47.851 INFO:teuthology.orchestra.run.smithi183.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 12 kB/s | 6.6 kB 00:00 2024-04-23T10:04:48.893 INFO:teuthology.orchestra.run.smithi196.stdout:lab-extras 10 kB/s | 1.7 kB 00:00 2024-04-23T10:04:50.867 INFO:teuthology.orchestra.run.smithi183.stdout:(2/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 295 kB/s | 890 kB 00:03 2024-04-23T10:04:50.971 INFO:teuthology.orchestra.run.smithi097.stdout:lab-extras 8.3 kB/s | 1.7 kB 00:00 2024-04-23T10:04:51.019 INFO:teuthology.orchestra.run.smithi196.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-23T10:04:51.019 INFO:teuthology.orchestra.run.smithi196.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-23T10:04:51.030 INFO:teuthology.orchestra.run.smithi196.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-23T10:04:51.031 INFO:teuthology.orchestra.run.smithi196.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-23T10:04:51.031 INFO:teuthology.orchestra.run.smithi196.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-23T10:04:51.032 INFO:teuthology.orchestra.run.smithi196.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-23T10:04:51.034 INFO:teuthology.orchestra.run.smithi196.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-23T10:04:51.036 INFO:teuthology.orchestra.run.smithi196.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-23T10:04:51.037 INFO:teuthology.orchestra.run.smithi196.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-23T10:04:51.096 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-04-23T10:04:51.111 INFO:teuthology.orchestra.run.smithi196.stdout:======================================================================================= 2024-04-23T10:04:51.111 INFO:teuthology.orchestra.run.smithi196.stdout: Package Arch Version Repository Size 2024-04-23T10:04:51.111 INFO:teuthology.orchestra.run.smithi196.stdout:======================================================================================= 2024-04-23T10:04:51.111 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-04-23T10:04:51.111 INFO:teuthology.orchestra.run.smithi196.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-23T10:04:51.111 INFO:teuthology.orchestra.run.smithi196.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-23T10:04:51.111 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-23T10:04:51.111 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-23T10:04:51.111 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-23T10:04:51.111 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-23T10:04:51.112 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-23T10:04:51.112 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-23T10:04:51.112 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-23T10:04:51.112 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-23T10:04:51.112 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-23T10:04:51.112 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-23T10:04:51.112 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-23T10:04:51.112 INFO:teuthology.orchestra.run.smithi196.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-23T10:04:51.112 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-23T10:04:51.112 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-23T10:04:51.112 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-23T10:04:51.112 INFO:teuthology.orchestra.run.smithi196.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-23T10:04:51.112 INFO:teuthology.orchestra.run.smithi196.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-23T10:04:51.113 INFO:teuthology.orchestra.run.smithi196.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-23T10:04:51.113 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-23T10:04:51.113 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-23T10:04:51.113 INFO:teuthology.orchestra.run.smithi196.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-23T10:04:51.113 INFO:teuthology.orchestra.run.smithi196.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-23T10:04:51.113 INFO:teuthology.orchestra.run.smithi196.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-23T10:04:51.113 INFO:teuthology.orchestra.run.smithi196.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-23T10:04:51.113 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-23T10:04:51.113 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-23T10:04:51.113 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-23T10:04:51.113 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-23T10:04:51.113 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-23T10:04:51.113 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-23T10:04:51.114 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-23T10:04:51.114 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-23T10:04:51.114 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-23T10:04:51.114 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-23T10:04:51.114 INFO:teuthology.orchestra.run.smithi196.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-23T10:04:51.114 INFO:teuthology.orchestra.run.smithi196.stdout:Upgrading: 2024-04-23T10:04:51.114 INFO:teuthology.orchestra.run.smithi196.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-23T10:04:51.114 INFO:teuthology.orchestra.run.smithi196.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-23T10:04:51.114 INFO:teuthology.orchestra.run.smithi196.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-23T10:04:51.114 INFO:teuthology.orchestra.run.smithi196.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-23T10:04:51.114 INFO:teuthology.orchestra.run.smithi196.stdout:Installing dependencies: 2024-04-23T10:04:51.114 INFO:teuthology.orchestra.run.smithi196.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-23T10:04:51.114 INFO:teuthology.orchestra.run.smithi196.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-23T10:04:51.115 INFO:teuthology.orchestra.run.smithi196.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-23T10:04:51.115 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-23T10:04:51.115 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-23T10:04:51.115 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-23T10:04:51.115 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-23T10:04:51.115 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-23T10:04:51.115 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-23T10:04:51.115 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-23T10:04:51.115 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-23T10:04:51.115 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-23T10:04:51.115 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-23T10:04:51.115 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-23T10:04:51.115 INFO:teuthology.orchestra.run.smithi196.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-23T10:04:51.116 INFO:teuthology.orchestra.run.smithi196.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-23T10:04:51.116 INFO:teuthology.orchestra.run.smithi196.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-23T10:04:51.116 INFO:teuthology.orchestra.run.smithi196.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-23T10:04:51.116 INFO:teuthology.orchestra.run.smithi196.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-23T10:04:51.116 INFO:teuthology.orchestra.run.smithi196.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-23T10:04:51.116 INFO:teuthology.orchestra.run.smithi196.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-23T10:04:51.116 INFO:teuthology.orchestra.run.smithi196.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-23T10:04:51.116 INFO:teuthology.orchestra.run.smithi196.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-23T10:04:51.116 INFO:teuthology.orchestra.run.smithi196.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-23T10:04:51.116 INFO:teuthology.orchestra.run.smithi196.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-23T10:04:51.116 INFO:teuthology.orchestra.run.smithi196.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-23T10:04:51.116 INFO:teuthology.orchestra.run.smithi196.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-23T10:04:51.117 INFO:teuthology.orchestra.run.smithi196.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-23T10:04:51.117 INFO:teuthology.orchestra.run.smithi196.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-23T10:04:51.117 INFO:teuthology.orchestra.run.smithi196.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-23T10:04:51.117 INFO:teuthology.orchestra.run.smithi196.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-23T10:04:51.117 INFO:teuthology.orchestra.run.smithi196.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-23T10:04:51.117 INFO:teuthology.orchestra.run.smithi196.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-23T10:04:51.117 INFO:teuthology.orchestra.run.smithi196.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-23T10:04:51.117 INFO:teuthology.orchestra.run.smithi196.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-23T10:04:51.117 INFO:teuthology.orchestra.run.smithi196.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-23T10:04:51.117 INFO:teuthology.orchestra.run.smithi196.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-23T10:04:51.117 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-23T10:04:51.117 INFO:teuthology.orchestra.run.smithi196.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-23T10:04:51.117 INFO:teuthology.orchestra.run.smithi196.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-23T10:04:51.117 INFO:teuthology.orchestra.run.smithi196.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-23T10:04:51.118 INFO:teuthology.orchestra.run.smithi196.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-23T10:04:51.118 INFO:teuthology.orchestra.run.smithi196.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-23T10:04:51.118 INFO:teuthology.orchestra.run.smithi196.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-23T10:04:51.118 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-23T10:04:51.118 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-23T10:04:51.118 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-23T10:04:51.118 INFO:teuthology.orchestra.run.smithi196.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-23T10:04:51.118 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-23T10:04:51.118 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-23T10:04:51.118 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-23T10:04:51.118 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-23T10:04:51.118 INFO:teuthology.orchestra.run.smithi196.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-23T10:04:51.118 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-23T10:04:51.119 INFO:teuthology.orchestra.run.smithi196.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-23T10:04:51.119 INFO:teuthology.orchestra.run.smithi196.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-23T10:04:51.119 INFO:teuthology.orchestra.run.smithi196.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-23T10:04:51.119 INFO:teuthology.orchestra.run.smithi196.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-23T10:04:51.119 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-23T10:04:51.119 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-23T10:04:51.119 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-23T10:04:51.119 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-23T10:04:51.119 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-23T10:04:51.119 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-23T10:04:51.119 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-23T10:04:51.119 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-23T10:04:51.119 INFO:teuthology.orchestra.run.smithi196.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-23T10:04:51.119 INFO:teuthology.orchestra.run.smithi196.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-23T10:04:51.119 INFO:teuthology.orchestra.run.smithi196.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-23T10:04:51.119 INFO:teuthology.orchestra.run.smithi196.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-23T10:04:51.120 INFO:teuthology.orchestra.run.smithi196.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-23T10:04:51.120 INFO:teuthology.orchestra.run.smithi196.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-23T10:04:51.120 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-23T10:04:51.120 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-23T10:04:51.120 INFO:teuthology.orchestra.run.smithi196.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-23T10:04:51.120 INFO:teuthology.orchestra.run.smithi196.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-23T10:04:51.120 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-23T10:04:51.120 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-23T10:04:51.120 INFO:teuthology.orchestra.run.smithi196.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-23T10:04:51.120 INFO:teuthology.orchestra.run.smithi196.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-23T10:04:51.120 INFO:teuthology.orchestra.run.smithi196.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-23T10:04:51.120 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-23T10:04:51.120 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-23T10:04:51.120 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-23T10:04:51.120 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-23T10:04:51.121 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-23T10:04:51.121 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-23T10:04:51.121 INFO:teuthology.orchestra.run.smithi196.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-23T10:04:51.121 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-23T10:04:51.121 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-23T10:04:51.121 INFO:teuthology.orchestra.run.smithi196.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-23T10:04:51.121 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-23T10:04:51.121 INFO:teuthology.orchestra.run.smithi196.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-23T10:04:51.121 INFO:teuthology.orchestra.run.smithi196.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-23T10:04:51.121 INFO:teuthology.orchestra.run.smithi196.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-23T10:04:51.121 INFO:teuthology.orchestra.run.smithi196.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-23T10:04:51.121 INFO:teuthology.orchestra.run.smithi196.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-23T10:04:51.122 INFO:teuthology.orchestra.run.smithi196.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-23T10:04:51.122 INFO:teuthology.orchestra.run.smithi196.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-23T10:04:51.122 INFO:teuthology.orchestra.run.smithi196.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-23T10:04:51.122 INFO:teuthology.orchestra.run.smithi196.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-23T10:04:51.122 INFO:teuthology.orchestra.run.smithi196.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-23T10:04:51.122 INFO:teuthology.orchestra.run.smithi196.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-23T10:04:51.122 INFO:teuthology.orchestra.run.smithi196.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-23T10:04:51.122 INFO:teuthology.orchestra.run.smithi196.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-23T10:04:51.122 INFO:teuthology.orchestra.run.smithi196.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-23T10:04:51.122 INFO:teuthology.orchestra.run.smithi196.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-23T10:04:51.122 INFO:teuthology.orchestra.run.smithi196.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-23T10:04:51.122 INFO:teuthology.orchestra.run.smithi196.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-23T10:04:51.122 INFO:teuthology.orchestra.run.smithi196.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-23T10:04:51.123 INFO:teuthology.orchestra.run.smithi196.stdout:Installing weak dependencies: 2024-04-23T10:04:51.123 INFO:teuthology.orchestra.run.smithi196.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-23T10:04:51.123 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-23T10:04:51.123 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-04-23T10:04:51.123 INFO:teuthology.orchestra.run.smithi196.stdout:======================================================================================= 2024-04-23T10:04:51.123 INFO:teuthology.orchestra.run.smithi196.stdout:Install 148 Packages 2024-04-23T10:04:51.123 INFO:teuthology.orchestra.run.smithi196.stdout:Upgrade 4 Packages 2024-04-23T10:04:51.123 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-23T10:04:51.123 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 222 M 2024-04-23T10:04:51.123 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-04-23T10:04:51.317 INFO:teuthology.orchestra.run.smithi183.stdout:(3/152): ceph-immutable-object-cache-19.0.0-315 322 kB/s | 145 kB 00:00 2024-04-23T10:04:51.776 INFO:teuthology.orchestra.run.smithi183.stdout:(4/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 1.2 MB/s | 5.4 MB 00:04 2024-04-23T10:04:52.201 INFO:teuthology.orchestra.run.smithi183.stdout:(5/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 3.5 MB/s | 1.5 MB 00:00 2024-04-23T10:04:52.488 INFO:teuthology.orchestra.run.smithi196.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 11 kB/s | 6.6 kB 00:00 2024-04-23T10:04:53.083 INFO:teuthology.orchestra.run.smithi097.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-23T10:04:53.084 INFO:teuthology.orchestra.run.smithi097.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-23T10:04:53.095 INFO:teuthology.orchestra.run.smithi097.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-23T10:04:53.095 INFO:teuthology.orchestra.run.smithi097.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-23T10:04:53.096 INFO:teuthology.orchestra.run.smithi097.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-23T10:04:53.097 INFO:teuthology.orchestra.run.smithi097.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-23T10:04:53.099 INFO:teuthology.orchestra.run.smithi097.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-23T10:04:53.101 INFO:teuthology.orchestra.run.smithi097.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-23T10:04:53.102 INFO:teuthology.orchestra.run.smithi097.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-23T10:04:53.162 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-04-23T10:04:53.171 INFO:teuthology.orchestra.run.smithi097.stdout:======================================================================================= 2024-04-23T10:04:53.171 INFO:teuthology.orchestra.run.smithi097.stdout: Package Arch Version Repository Size 2024-04-23T10:04:53.171 INFO:teuthology.orchestra.run.smithi097.stdout:======================================================================================= 2024-04-23T10:04:53.171 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2024-04-23T10:04:53.172 INFO:teuthology.orchestra.run.smithi097.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-23T10:04:53.172 INFO:teuthology.orchestra.run.smithi097.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-23T10:04:53.172 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-23T10:04:53.172 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-23T10:04:53.172 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-23T10:04:53.172 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-23T10:04:53.172 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-23T10:04:53.172 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-23T10:04:53.172 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-23T10:04:53.172 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-23T10:04:53.172 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-23T10:04:53.172 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-23T10:04:53.172 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-23T10:04:53.173 INFO:teuthology.orchestra.run.smithi097.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-23T10:04:53.173 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-23T10:04:53.173 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-23T10:04:53.173 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-23T10:04:53.173 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-23T10:04:53.173 INFO:teuthology.orchestra.run.smithi097.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-23T10:04:53.173 INFO:teuthology.orchestra.run.smithi097.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-23T10:04:53.173 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-23T10:04:53.173 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-23T10:04:53.173 INFO:teuthology.orchestra.run.smithi097.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-23T10:04:53.173 INFO:teuthology.orchestra.run.smithi097.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-23T10:04:53.173 INFO:teuthology.orchestra.run.smithi097.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-23T10:04:53.173 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-23T10:04:53.173 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-23T10:04:53.173 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-23T10:04:53.174 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-23T10:04:53.174 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-23T10:04:53.174 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-23T10:04:53.174 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-23T10:04:53.174 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-23T10:04:53.174 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-23T10:04:53.174 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-23T10:04:53.174 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-23T10:04:53.174 INFO:teuthology.orchestra.run.smithi097.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-23T10:04:53.174 INFO:teuthology.orchestra.run.smithi097.stdout:Upgrading: 2024-04-23T10:04:53.174 INFO:teuthology.orchestra.run.smithi097.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-23T10:04:53.174 INFO:teuthology.orchestra.run.smithi097.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-23T10:04:53.174 INFO:teuthology.orchestra.run.smithi097.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-23T10:04:53.175 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-23T10:04:53.175 INFO:teuthology.orchestra.run.smithi097.stdout:Installing dependencies: 2024-04-23T10:04:53.175 INFO:teuthology.orchestra.run.smithi097.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-23T10:04:53.175 INFO:teuthology.orchestra.run.smithi097.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-23T10:04:53.175 INFO:teuthology.orchestra.run.smithi097.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-23T10:04:53.175 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-23T10:04:53.175 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-23T10:04:53.175 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-23T10:04:53.175 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-23T10:04:53.175 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-23T10:04:53.175 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-23T10:04:53.175 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-23T10:04:53.175 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-23T10:04:53.175 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-23T10:04:53.176 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-23T10:04:53.176 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-23T10:04:53.176 INFO:teuthology.orchestra.run.smithi097.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-23T10:04:53.176 INFO:teuthology.orchestra.run.smithi097.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-23T10:04:53.176 INFO:teuthology.orchestra.run.smithi097.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-23T10:04:53.176 INFO:teuthology.orchestra.run.smithi097.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-23T10:04:53.176 INFO:teuthology.orchestra.run.smithi097.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-23T10:04:53.176 INFO:teuthology.orchestra.run.smithi097.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-23T10:04:53.176 INFO:teuthology.orchestra.run.smithi097.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-23T10:04:53.176 INFO:teuthology.orchestra.run.smithi097.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-23T10:04:53.176 INFO:teuthology.orchestra.run.smithi097.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-23T10:04:53.176 INFO:teuthology.orchestra.run.smithi097.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-23T10:04:53.176 INFO:teuthology.orchestra.run.smithi097.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-23T10:04:53.176 INFO:teuthology.orchestra.run.smithi097.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-23T10:04:53.176 INFO:teuthology.orchestra.run.smithi097.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-23T10:04:53.177 INFO:teuthology.orchestra.run.smithi097.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-23T10:04:53.177 INFO:teuthology.orchestra.run.smithi097.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-23T10:04:53.177 INFO:teuthology.orchestra.run.smithi097.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-23T10:04:53.177 INFO:teuthology.orchestra.run.smithi097.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-23T10:04:53.177 INFO:teuthology.orchestra.run.smithi097.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-23T10:04:53.177 INFO:teuthology.orchestra.run.smithi097.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-23T10:04:53.177 INFO:teuthology.orchestra.run.smithi097.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-23T10:04:53.177 INFO:teuthology.orchestra.run.smithi097.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-23T10:04:53.177 INFO:teuthology.orchestra.run.smithi097.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-23T10:04:53.177 INFO:teuthology.orchestra.run.smithi097.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-23T10:04:53.177 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-23T10:04:53.177 INFO:teuthology.orchestra.run.smithi097.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-23T10:04:53.177 INFO:teuthology.orchestra.run.smithi097.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-23T10:04:53.177 INFO:teuthology.orchestra.run.smithi097.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-23T10:04:53.178 INFO:teuthology.orchestra.run.smithi097.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-23T10:04:53.178 INFO:teuthology.orchestra.run.smithi097.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-23T10:04:53.178 INFO:teuthology.orchestra.run.smithi097.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-23T10:04:53.178 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-23T10:04:53.178 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-23T10:04:53.178 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-23T10:04:53.178 INFO:teuthology.orchestra.run.smithi097.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-23T10:04:53.178 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-23T10:04:53.178 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-23T10:04:53.178 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-23T10:04:53.178 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-23T10:04:53.178 INFO:teuthology.orchestra.run.smithi097.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-23T10:04:53.178 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-23T10:04:53.179 INFO:teuthology.orchestra.run.smithi097.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-23T10:04:53.179 INFO:teuthology.orchestra.run.smithi097.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-23T10:04:53.179 INFO:teuthology.orchestra.run.smithi097.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-23T10:04:53.179 INFO:teuthology.orchestra.run.smithi097.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-23T10:04:53.179 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-23T10:04:53.179 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-23T10:04:53.179 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-23T10:04:53.179 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-23T10:04:53.179 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-23T10:04:53.179 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-23T10:04:53.179 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-23T10:04:53.179 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-23T10:04:53.179 INFO:teuthology.orchestra.run.smithi097.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-23T10:04:53.179 INFO:teuthology.orchestra.run.smithi097.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-23T10:04:53.180 INFO:teuthology.orchestra.run.smithi097.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-23T10:04:53.180 INFO:teuthology.orchestra.run.smithi097.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-23T10:04:53.180 INFO:teuthology.orchestra.run.smithi097.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-23T10:04:53.180 INFO:teuthology.orchestra.run.smithi097.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-23T10:04:53.180 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-23T10:04:53.180 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-23T10:04:53.180 INFO:teuthology.orchestra.run.smithi097.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-23T10:04:53.180 INFO:teuthology.orchestra.run.smithi097.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-23T10:04:53.180 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-23T10:04:53.180 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-23T10:04:53.180 INFO:teuthology.orchestra.run.smithi097.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-23T10:04:53.180 INFO:teuthology.orchestra.run.smithi097.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-23T10:04:53.180 INFO:teuthology.orchestra.run.smithi097.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-23T10:04:53.180 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-23T10:04:53.180 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-23T10:04:53.181 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-23T10:04:53.181 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-23T10:04:53.181 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-23T10:04:53.181 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-23T10:04:53.181 INFO:teuthology.orchestra.run.smithi097.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-23T10:04:53.181 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-23T10:04:53.181 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-23T10:04:53.181 INFO:teuthology.orchestra.run.smithi097.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-23T10:04:53.181 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-23T10:04:53.181 INFO:teuthology.orchestra.run.smithi097.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-23T10:04:53.181 INFO:teuthology.orchestra.run.smithi097.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-23T10:04:53.181 INFO:teuthology.orchestra.run.smithi097.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-23T10:04:53.181 INFO:teuthology.orchestra.run.smithi097.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-23T10:04:53.181 INFO:teuthology.orchestra.run.smithi097.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-23T10:04:53.182 INFO:teuthology.orchestra.run.smithi097.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-23T10:04:53.182 INFO:teuthology.orchestra.run.smithi097.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-23T10:04:53.182 INFO:teuthology.orchestra.run.smithi097.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-23T10:04:53.182 INFO:teuthology.orchestra.run.smithi097.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-23T10:04:53.182 INFO:teuthology.orchestra.run.smithi097.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-23T10:04:53.182 INFO:teuthology.orchestra.run.smithi097.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-23T10:04:53.182 INFO:teuthology.orchestra.run.smithi097.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-23T10:04:53.182 INFO:teuthology.orchestra.run.smithi097.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-23T10:04:53.182 INFO:teuthology.orchestra.run.smithi097.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-23T10:04:53.182 INFO:teuthology.orchestra.run.smithi097.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-23T10:04:53.182 INFO:teuthology.orchestra.run.smithi097.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-23T10:04:53.182 INFO:teuthology.orchestra.run.smithi097.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-23T10:04:53.182 INFO:teuthology.orchestra.run.smithi097.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-23T10:04:53.182 INFO:teuthology.orchestra.run.smithi097.stdout:Installing weak dependencies: 2024-04-23T10:04:53.183 INFO:teuthology.orchestra.run.smithi097.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-23T10:04:53.183 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:04:53.183 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-04-23T10:04:53.183 INFO:teuthology.orchestra.run.smithi097.stdout:======================================================================================= 2024-04-23T10:04:53.183 INFO:teuthology.orchestra.run.smithi097.stdout:Install 148 Packages 2024-04-23T10:04:53.183 INFO:teuthology.orchestra.run.smithi097.stdout:Upgrade 4 Packages 2024-04-23T10:04:53.183 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:04:53.183 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 222 M 2024-04-23T10:04:53.184 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-04-23T10:04:54.134 INFO:teuthology.orchestra.run.smithi183.stdout:(6/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 2.4 MB/s | 4.7 MB 00:01 2024-04-23T10:04:54.425 INFO:teuthology.orchestra.run.smithi097.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 12 kB/s | 6.6 kB 00:00 2024-04-23T10:04:55.996 INFO:teuthology.orchestra.run.smithi196.stdout:(2/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 254 kB/s | 890 kB 00:03 2024-04-23T10:04:56.554 INFO:teuthology.orchestra.run.smithi196.stdout:(3/152): ceph-immutable-object-cache-19.0.0-315 259 kB/s | 145 kB 00:00 2024-04-23T10:04:57.151 INFO:teuthology.orchestra.run.smithi183.stdout:(7/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 414 kB/s | 2.4 MB 00:05 2024-04-23T10:04:57.708 INFO:teuthology.orchestra.run.smithi097.stdout:(2/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 271 kB/s | 890 kB 00:03 2024-04-23T10:04:58.025 INFO:teuthology.orchestra.run.smithi097.stdout:(3/152): ceph-immutable-object-cache-19.0.0-315 457 kB/s | 145 kB 00:00 2024-04-23T10:05:02.175 INFO:teuthology.orchestra.run.smithi097.stdout:(4/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 583 kB/s | 2.4 MB 00:04 2024-04-23T10:05:04.254 INFO:teuthology.orchestra.run.smithi196.stdout:(4/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 314 kB/s | 2.4 MB 00:07 2024-04-23T10:05:05.308 INFO:teuthology.orchestra.run.smithi097.stdout:(5/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 489 kB/s | 1.5 MB 00:03 2024-04-23T10:05:09.089 INFO:teuthology.orchestra.run.smithi196.stdout:(5/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 317 kB/s | 1.5 MB 00:04 2024-04-23T10:05:10.658 INFO:teuthology.orchestra.run.smithi097.stdout:(6/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 328 kB/s | 5.4 MB 00:16 2024-04-23T10:05:11.470 INFO:teuthology.orchestra.run.smithi196.stdout:(6/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 281 kB/s | 5.4 MB 00:19 2024-04-23T10:05:15.649 INFO:teuthology.orchestra.run.smithi097.stdout:(7/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 465 kB/s | 4.7 MB 00:10 2024-04-23T10:05:16.736 INFO:teuthology.orchestra.run.smithi196.stdout:(7/152): ceph-common-19.0.0-3156.gd69f13d8.el7. 871 kB/s | 21 MB 00:24 2024-04-23T10:05:20.339 INFO:teuthology.orchestra.run.smithi183.stdout:(8/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x86 658 kB/s | 17 MB 00:26 2024-04-23T10:05:20.840 INFO:teuthology.orchestra.run.smithi183.stdout:(9/152): ceph-selinux-19.0.0-3156.gd69f13d8.el7 50 kB/s | 25 kB 00:00 2024-04-23T10:05:23.811 INFO:teuthology.orchestra.run.smithi196.stdout:(8/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 327 kB/s | 4.7 MB 00:14 2024-04-23T10:05:23.994 INFO:teuthology.orchestra.run.smithi196.stdout:(9/152): ceph-selinux-19.0.0-3156.gd69f13d8.el7 137 kB/s | 25 kB 00:00 2024-04-23T10:05:28.814 INFO:teuthology.orchestra.run.smithi183.stdout:(10/152): ceph-common-19.0.0-3156.gd69f13d8.el7 521 kB/s | 21 MB 00:41 2024-04-23T10:05:29.723 INFO:teuthology.orchestra.run.smithi183.stdout:(11/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 247 kB/s | 225 kB 00:00 2024-04-23T10:05:30.081 INFO:teuthology.orchestra.run.smithi183.stdout:(12/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 75 kB/s | 27 kB 00:00 2024-04-23T10:05:30.256 INFO:teuthology.orchestra.run.smithi183.stdout:(13/152): libcephfs-devel-19.0.0-3156.gd69f13d8 181 kB/s | 32 kB 00:00 2024-04-23T10:05:34.023 INFO:teuthology.orchestra.run.smithi183.stdout:(14/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 198 kB/s | 747 kB 00:03 2024-04-23T10:05:34.881 INFO:teuthology.orchestra.run.smithi183.stdout:(15/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 190 kB/s | 163 kB 00:00 2024-04-23T10:05:35.632 INFO:teuthology.orchestra.run.smithi183.stdout:(16/152): librados-devel-19.0.0-3156.gd69f13d8. 169 kB/s | 126 kB 00:00 2024-04-23T10:05:39.873 INFO:teuthology.orchestra.run.smithi183.stdout:(17/152): libradosstriper1-19.0.0-3156.gd69f13d 118 kB/s | 500 kB 00:04 2024-04-23T10:05:40.841 INFO:teuthology.orchestra.run.smithi183.stdout:(18/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el 248 kB/s | 11 MB 00:43 2024-04-23T10:05:41.049 INFO:teuthology.orchestra.run.smithi183.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 217 kB/s | 45 kB 00:00 2024-04-23T10:05:41.549 INFO:teuthology.orchestra.run.smithi183.stdout:(20/152): python3-ceph-common-19.0.0-3156.gd69f 269 kB/s | 135 kB 00:00 2024-04-23T10:05:42.541 INFO:teuthology.orchestra.run.smithi183.stdout:(21/152): python3-cephfs-19.0.0-3156.gd69f13d8. 163 kB/s | 161 kB 00:00 2024-04-23T10:05:44.124 INFO:teuthology.orchestra.run.smithi183.stdout:(22/152): python3-rados-19.0.0-3156.gd69f13d8.e 202 kB/s | 320 kB 00:01 2024-04-23T10:05:46.149 INFO:teuthology.orchestra.run.smithi183.stdout:(23/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 147 kB/s | 298 kB 00:02 2024-04-23T10:05:47.091 INFO:teuthology.orchestra.run.smithi183.stdout:(24/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 106 kB/s | 99 kB 00:00 2024-04-23T10:05:48.416 INFO:teuthology.orchestra.run.smithi183.stdout:(25/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 64 kB/s | 85 kB 00:01 2024-04-23T10:06:00.954 INFO:teuthology.orchestra.run.smithi097.stdout:(8/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el7 239 kB/s | 11 MB 00:45 2024-04-23T10:06:01.204 INFO:teuthology.orchestra.run.smithi097.stdout:(9/152): ceph-selinux-19.0.0-3156.gd69f13d8.el7 101 kB/s | 25 kB 00:00 2024-04-23T10:06:05.172 INFO:teuthology.orchestra.run.smithi183.stdout:(26/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 217 kB/s | 5.4 MB 00:25 2024-04-23T10:06:05.281 INFO:teuthology.orchestra.run.smithi196.stdout:(10/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el 223 kB/s | 11 MB 00:48 2024-04-23T10:06:06.130 INFO:teuthology.orchestra.run.smithi183.stdout:(27/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 178 kB/s | 3.1 MB 00:17 2024-04-23T10:06:06.214 INFO:teuthology.orchestra.run.smithi183.stdout:(28/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 164 kB/s | 170 kB 00:01 2024-04-23T10:06:06.339 INFO:teuthology.orchestra.run.smithi183.stdout:(29/152): ceph-grafana-dashboards-19.0.0-3156.g 164 kB/s | 34 kB 00:00 2024-04-23T10:06:06.730 INFO:teuthology.orchestra.run.smithi196.stdout:(11/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 155 kB/s | 225 kB 00:01 2024-04-23T10:06:06.823 INFO:teuthology.orchestra.run.smithi183.stdout:(30/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 234 kB/s | 142 kB 00:00 2024-04-23T10:06:06.954 INFO:teuthology.orchestra.run.smithi196.stdout:(12/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 120 kB/s | 27 kB 00:00 2024-04-23T10:06:07.322 INFO:teuthology.orchestra.run.smithi196.stdout:(13/152): libcephfs-devel-19.0.0-3156.gd69f13d8 86 kB/s | 32 kB 00:00 2024-04-23T10:06:10.779 INFO:teuthology.orchestra.run.smithi196.stdout:(14/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 216 kB/s | 747 kB 00:03 2024-04-23T10:06:11.338 INFO:teuthology.orchestra.run.smithi196.stdout:(15/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 292 kB/s | 163 kB 00:00 2024-04-23T10:06:11.813 INFO:teuthology.orchestra.run.smithi196.stdout:(16/152): librados-devel-19.0.0-3156.gd69f13d8. 266 kB/s | 126 kB 00:00 2024-04-23T10:06:15.213 INFO:teuthology.orchestra.run.smithi196.stdout:(17/152): libradosstriper1-19.0.0-3156.gd69f13d 147 kB/s | 500 kB 00:03 2024-04-23T10:06:19.827 INFO:teuthology.orchestra.run.smithi097.stdout:(10/152): ceph-common-19.0.0-3156.gd69f13d8.el7 252 kB/s | 21 MB 01:25 2024-04-23T10:06:21.336 INFO:teuthology.orchestra.run.smithi097.stdout:(11/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 149 kB/s | 225 kB 00:01 2024-04-23T10:06:21.636 INFO:teuthology.orchestra.run.smithi097.stdout:(12/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x8 243 kB/s | 17 MB 01:10 2024-04-23T10:06:21.661 INFO:teuthology.orchestra.run.smithi097.stdout:(13/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 83 kB/s | 27 kB 00:00 2024-04-23T10:06:21.903 INFO:teuthology.orchestra.run.smithi097.stdout:(14/152): libcephfs-devel-19.0.0-3156.gd69f13d8 119 kB/s | 32 kB 00:00 2024-04-23T10:06:22.612 INFO:teuthology.orchestra.run.smithi097.stdout:(15/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 230 kB/s | 163 kB 00:00 2024-04-23T10:06:23.087 INFO:teuthology.orchestra.run.smithi097.stdout:(16/152): librados-devel-19.0.0-3156.gd69f13d8. 266 kB/s | 126 kB 00:00 2024-04-23T10:06:25.254 INFO:teuthology.orchestra.run.smithi097.stdout:(17/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 208 kB/s | 747 kB 00:03 2024-04-23T10:06:25.287 INFO:teuthology.orchestra.run.smithi097.stdout:(18/152): libradosstriper1-19.0.0-3156.gd69f13d 227 kB/s | 500 kB 00:02 2024-04-23T10:06:25.504 INFO:teuthology.orchestra.run.smithi097.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 208 kB/s | 45 kB 00:00 2024-04-23T10:06:25.979 INFO:teuthology.orchestra.run.smithi097.stdout:(20/152): python3-ceph-common-19.0.0-3156.gd69f 284 kB/s | 135 kB 00:00 2024-04-23T10:06:26.613 INFO:teuthology.orchestra.run.smithi097.stdout:(21/152): python3-cephfs-19.0.0-3156.gd69f13d8. 255 kB/s | 161 kB 00:00 2024-04-23T10:06:28.221 INFO:teuthology.orchestra.run.smithi097.stdout:(22/152): python3-rados-19.0.0-3156.gd69f13d8.e 199 kB/s | 320 kB 00:01 2024-04-23T10:06:29.928 INFO:teuthology.orchestra.run.smithi183.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 154 kB/s | 3.5 MB 00:23 2024-04-23T10:06:30.263 INFO:teuthology.orchestra.run.smithi097.stdout:(23/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 146 kB/s | 298 kB 00:02 2024-04-23T10:06:30.905 INFO:teuthology.orchestra.run.smithi097.stdout:(24/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 155 kB/s | 99 kB 00:00 2024-04-23T10:06:31.447 INFO:teuthology.orchestra.run.smithi097.stdout:(25/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 157 kB/s | 85 kB 00:00 2024-04-23T10:06:32.986 INFO:teuthology.orchestra.run.smithi183.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3156.gd6 81 kB/s | 249 kB 00:03 2024-04-23T10:06:33.644 INFO:teuthology.orchestra.run.smithi183.stdout:(33/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 75 kB/s | 49 kB 00:00 2024-04-23T10:06:33.920 INFO:teuthology.orchestra.run.smithi183.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3156.gd 60 kB/s | 16 kB 00:00 2024-04-23T10:06:36.453 INFO:teuthology.orchestra.run.smithi183.stdout:(35/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 108 kB/s | 273 kB 00:02 2024-04-23T10:06:37.584 INFO:teuthology.orchestra.run.smithi196.stdout:(18/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 245 kB/s | 5.4 MB 00:22 2024-04-23T10:06:37.927 INFO:teuthology.orchestra.run.smithi196.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 132 kB/s | 45 kB 00:00 2024-04-23T10:06:39.227 INFO:teuthology.orchestra.run.smithi196.stdout:(20/152): python3-ceph-common-19.0.0-3156.gd69f 104 kB/s | 135 kB 00:01 2024-04-23T10:06:40.585 INFO:teuthology.orchestra.run.smithi196.stdout:(21/152): python3-cephfs-19.0.0-3156.gd69f13d8. 119 kB/s | 161 kB 00:01 2024-04-23T10:06:40.644 INFO:teuthology.orchestra.run.smithi183.stdout:(36/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 182 kB/s | 762 kB 00:04 2024-04-23T10:06:40.861 INFO:teuthology.orchestra.run.smithi183.stdout:(37/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 84 kB/s | 18 kB 00:00 2024-04-23T10:06:41.011 INFO:teuthology.orchestra.run.smithi183.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 538 kB/s | 80 kB 00:00 2024-04-23T10:06:41.286 INFO:teuthology.orchestra.run.smithi183.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 2.9 MB/s | 803 kB 00:00 2024-04-23T10:06:41.370 INFO:teuthology.orchestra.run.smithi183.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 195 kB 00:00 2024-04-23T10:06:41.495 INFO:teuthology.orchestra.run.smithi183.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.0 MB/s | 253 kB 00:00 2024-04-23T10:06:41.829 INFO:teuthology.orchestra.run.smithi183.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 3.7 MB/s | 1.2 MB 00:00 2024-04-23T10:06:41.888 INFO:teuthology.orchestra.run.smithi183.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-04-23T10:06:41.946 INFO:teuthology.orchestra.run.smithi183.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-04-23T10:06:42.005 INFO:teuthology.orchestra.run.smithi183.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-04-23T10:06:42.088 INFO:teuthology.orchestra.run.smithi183.stdout:(46/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-04-23T10:06:42.339 INFO:teuthology.orchestra.run.smithi183.stdout:(47/152): automake-1.16.2-8.el9.noarch.rpm 2.7 MB/s | 689 kB 00:00 2024-04-23T10:06:42.443 INFO:teuthology.orchestra.run.smithi196.stdout:(22/152): python3-rados-19.0.0-3156.gd69f13d8.e 172 kB/s | 320 kB 00:01 2024-04-23T10:06:42.464 INFO:teuthology.orchestra.run.smithi183.stdout:(48/152): bison-3.7.4-5.el9.x86_64.rpm 7.4 MB/s | 943 kB 00:00 2024-04-23T10:06:42.514 INFO:teuthology.orchestra.run.smithi183.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 107 kB 00:00 2024-04-23T10:06:42.556 INFO:teuthology.orchestra.run.smithi183.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 517 kB/s | 21 kB 00:00 2024-04-23T10:06:42.606 INFO:teuthology.orchestra.run.smithi183.stdout:(51/152): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-04-23T10:06:42.648 INFO:teuthology.orchestra.run.smithi183.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 787 kB/s | 33 kB 00:00 2024-04-23T10:06:42.890 INFO:teuthology.orchestra.run.smithi183.stdout:(53/152): ceph-mgr-diskprediction-local-19.0.0- 210 kB/s | 7.4 MB 00:36 2024-04-23T10:06:42.957 INFO:teuthology.orchestra.run.smithi183.stdout:(54/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 9.7 MB/s | 3.0 MB 00:00 2024-04-23T10:06:43.041 INFO:teuthology.orchestra.run.smithi183.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 134 kB/s | 11 kB 00:00 2024-04-23T10:06:43.091 INFO:teuthology.orchestra.run.smithi183.stdout:(56/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-04-23T10:06:43.141 INFO:teuthology.orchestra.run.smithi183.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 908 kB/s | 45 kB 00:00 2024-04-23T10:06:43.233 INFO:teuthology.orchestra.run.smithi183.stdout:(58/152): librdkafka-1.6.1-102.el9.x86_64.rpm 7.1 MB/s | 662 kB 00:00 2024-04-23T10:06:43.300 INFO:teuthology.orchestra.run.smithi183.stdout:(59/152): libtool-2.4.6-45.el9.x86_64.rpm 8.5 MB/s | 582 kB 00:00 2024-04-23T10:06:43.325 INFO:teuthology.orchestra.run.smithi183.stdout:(60/152): flexiblas-openblas-openmp-3.0.4-8.el9 42 kB/s | 18 kB 00:00 2024-04-23T10:06:43.351 INFO:teuthology.orchestra.run.smithi183.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 507 kB/s | 25 kB 00:00 2024-04-23T10:06:43.417 INFO:teuthology.orchestra.run.smithi183.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 516 kB/s | 47 kB 00:00 2024-04-23T10:06:43.468 INFO:teuthology.orchestra.run.smithi183.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-04-23T10:06:43.510 INFO:teuthology.orchestra.run.smithi183.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 2.0 MB/s | 188 kB 00:00 2024-04-23T10:06:43.535 INFO:teuthology.orchestra.run.smithi183.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 526 kB/s | 35 kB 00:00 2024-04-23T10:06:43.835 INFO:teuthology.orchestra.run.smithi196.stdout:(23/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 214 kB/s | 298 kB 00:01 2024-04-23T10:06:44.002 INFO:teuthology.orchestra.run.smithi183.stdout:(66/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 9.5 MB/s | 4.4 MB 00:00 2024-04-23T10:06:44.111 INFO:teuthology.orchestra.run.smithi183.stdout:(67/152): openblas-openmp-0.3.21-2.el9.x86_64.r 7.9 MB/s | 4.7 MB 00:00 2024-04-23T10:06:44.177 INFO:teuthology.orchestra.run.smithi196.stdout:(24/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 291 kB/s | 99 kB 00:00 2024-04-23T10:06:44.219 INFO:teuthology.orchestra.run.smithi183.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 7.3 MB/s | 1.6 MB 00:00 2024-04-23T10:06:44.269 INFO:teuthology.orchestra.run.smithi183.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 2.6 MB/s | 133 kB 00:00 2024-04-23T10:06:44.303 INFO:teuthology.orchestra.run.smithi183.stdout:(70/152): postgresql-contrib-13.14-1.el9.x86_64 4.6 MB/s | 911 kB 00:00 2024-04-23T10:06:44.437 INFO:teuthology.orchestra.run.smithi183.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 7.6 MB/s | 1.0 MB 00:00 2024-04-23T10:06:44.479 INFO:teuthology.orchestra.run.smithi183.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 2.1 MB/s | 87 kB 00:00 2024-04-23T10:06:44.519 INFO:teuthology.orchestra.run.smithi196.stdout:(25/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 249 kB/s | 85 kB 00:00 2024-04-23T10:06:44.788 INFO:teuthology.orchestra.run.smithi183.stdout:(73/152): python3-babel-2.9.1-2.el9.noarch.rpm 19 MB/s | 6.0 MB 00:00 2024-04-23T10:06:44.838 INFO:teuthology.orchestra.run.smithi183.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 4.8 MB/s | 244 kB 00:00 2024-04-23T10:06:44.862 INFO:teuthology.orchestra.run.smithi097.stdout:(26/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 236 kB/s | 3.1 MB 00:13 2024-04-23T10:06:44.921 INFO:teuthology.orchestra.run.smithi183.stdout:(75/152): postgresql-server-13.14-1.el9.x86_64. 8.9 MB/s | 5.8 MB 00:00 2024-04-23T10:06:44.946 INFO:teuthology.orchestra.run.smithi183.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.2 MB/s | 248 kB 00:00 2024-04-23T10:06:44.972 INFO:teuthology.orchestra.run.smithi183.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 512 kB/s | 26 kB 00:00 2024-04-23T10:06:44.988 INFO:teuthology.orchestra.run.smithi183.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 458 kB/s | 19 kB 00:00 2024-04-23T10:06:45.022 INFO:teuthology.orchestra.run.smithi183.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-04-23T10:06:45.072 INFO:teuthology.orchestra.run.smithi183.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 415 kB/s | 35 kB 00:00 2024-04-23T10:06:45.148 INFO:teuthology.orchestra.run.smithi183.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.3 MB/s | 250 kB 00:00 2024-04-23T10:06:45.198 INFO:teuthology.orchestra.run.smithi183.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-04-23T10:06:45.290 INFO:teuthology.orchestra.run.smithi183.stdout:(83/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 19 MB/s | 5.1 MB 00:00 2024-04-23T10:06:45.315 INFO:teuthology.orchestra.run.smithi183.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 661 kB/s | 77 kB 00:00 2024-04-23T10:06:45.340 INFO:teuthology.orchestra.run.smithi183.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 843 kB/s | 42 kB 00:00 2024-04-23T10:06:45.366 INFO:teuthology.orchestra.run.smithi183.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-04-23T10:06:45.416 INFO:teuthology.orchestra.run.smithi183.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-04-23T10:06:45.466 INFO:teuthology.orchestra.run.smithi183.stdout:(88/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-23T10:06:45.525 INFO:teuthology.orchestra.run.smithi183.stdout:(89/152): python3-requests-oauthlib-1.3.0-12.el 921 kB/s | 54 kB 00:00 2024-04-23T10:06:45.617 INFO:teuthology.orchestra.run.smithi183.stdout:(90/152): python3-pyasn1-modules-0.4.8-6.el9.no 1.1 MB/s | 279 kB 00:00 2024-04-23T10:06:45.675 INFO:teuthology.orchestra.run.smithi183.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 716 kB/s | 42 kB 00:00 2024-04-23T10:06:45.734 INFO:teuthology.orchestra.run.smithi183.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 762 kB/s | 44 kB 00:00 2024-04-23T10:06:45.934 INFO:teuthology.orchestra.run.smithi183.stdout:(93/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 40 MB/s | 16 MB 00:00 2024-04-23T10:06:45.959 INFO:teuthology.orchestra.run.smithi183.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 980 kB/s | 221 kB 00:00 2024-04-23T10:06:46.026 INFO:teuthology.orchestra.run.smithi183.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 392 kB/s | 36 kB 00:00 2024-04-23T10:06:46.060 INFO:teuthology.orchestra.run.smithi183.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 473 kB/s | 47 kB 00:00 2024-04-23T10:06:46.118 INFO:teuthology.orchestra.run.smithi183.stdout:(97/152): uuid-1.6.2-55.el9.x86_64.rpm 994 kB/s | 58 kB 00:00 2024-04-23T10:06:46.144 INFO:teuthology.orchestra.run.smithi183.stdout:(98/152): socat-1.7.4.1-5.el9.x86_64.rpm 3.6 MB/s | 305 kB 00:00 2024-04-23T10:06:46.169 INFO:teuthology.orchestra.run.smithi183.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-04-23T10:06:46.211 INFO:teuthology.orchestra.run.smithi183.stdout:(100/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.0 MB/s | 45 kB 00:00 2024-04-23T10:06:46.212 INFO:teuthology.orchestra.run.smithi097.stdout:(27/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 126 kB/s | 170 kB 00:01 2024-04-23T10:06:46.253 INFO:teuthology.orchestra.run.smithi183.stdout:(101/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 586 kB/s | 64 kB 00:00 2024-04-23T10:06:46.303 INFO:teuthology.orchestra.run.smithi183.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 242 kB/s | 22 kB 00:00 2024-04-23T10:06:46.362 INFO:teuthology.orchestra.run.smithi183.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 7.8 MB/s | 863 kB 00:00 2024-04-23T10:06:46.479 INFO:teuthology.orchestra.run.smithi097.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3156.g 128 kB/s | 34 kB 00:00 2024-04-23T10:06:46.779 INFO:teuthology.orchestra.run.smithi183.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.1 MB/s | 548 kB 00:00 2024-04-23T10:06:46.804 INFO:teuthology.orchestra.run.smithi183.stdout:(105/152): gperftools-libs-2.9.1-2.el9.x86_64.r 744 kB/s | 309 kB 00:00 2024-04-23T10:06:46.863 INFO:teuthology.orchestra.run.smithi183.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 233 kB/s | 19 kB 00:00 2024-04-23T10:06:46.938 INFO:teuthology.orchestra.run.smithi183.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 348 kB/s | 26 kB 00:00 2024-04-23T10:06:47.030 INFO:teuthology.orchestra.run.smithi183.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 530 kB/s | 49 kB 00:00 2024-04-23T10:06:47.147 INFO:teuthology.orchestra.run.smithi183.stdout:(109/152): libarrow-9.0.0-10.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-04-23T10:06:47.172 INFO:teuthology.orchestra.run.smithi183.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 474 kB/s | 67 kB 00:00 2024-04-23T10:06:47.222 INFO:teuthology.orchestra.run.smithi183.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-04-23T10:06:47.288 INFO:teuthology.orchestra.run.smithi097.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 176 kB/s | 142 kB 00:00 2024-04-23T10:06:47.306 INFO:teuthology.orchestra.run.smithi183.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 6.4 MB/s | 548 kB 00:00 2024-04-23T10:06:47.365 INFO:teuthology.orchestra.run.smithi183.stdout:(113/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.3 MB/s | 838 kB 00:00 2024-04-23T10:06:47.390 INFO:teuthology.orchestra.run.smithi183.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 519 kB/s | 43 kB 00:00 2024-04-23T10:06:47.432 INFO:teuthology.orchestra.run.smithi183.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 484 kB/s | 32 kB 00:00 2024-04-23T10:06:47.457 INFO:teuthology.orchestra.run.smithi183.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 224 kB/s | 15 kB 00:00 2024-04-23T10:06:47.507 INFO:teuthology.orchestra.run.smithi183.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-04-23T10:06:47.532 INFO:teuthology.orchestra.run.smithi183.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 4.7 MB/s | 358 kB 00:00 2024-04-23T10:06:47.583 INFO:teuthology.orchestra.run.smithi183.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-04-23T10:06:47.608 INFO:teuthology.orchestra.run.smithi183.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 469 kB/s | 35 kB 00:00 2024-04-23T10:06:47.658 INFO:teuthology.orchestra.run.smithi183.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 2.7 MB/s | 206 kB 00:00 2024-04-23T10:06:47.733 INFO:teuthology.orchestra.run.smithi183.stdout:(122/152): python3-grpcio-1.46.7-10.el9.x86_64. 16 MB/s | 2.0 MB 00:00 2024-04-23T10:06:47.758 INFO:teuthology.orchestra.run.smithi183.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.4 MB/s | 144 kB 00:00 2024-04-23T10:06:47.809 INFO:teuthology.orchestra.run.smithi183.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 142 kB/s | 11 kB 00:00 2024-04-23T10:06:47.834 INFO:teuthology.orchestra.run.smithi183.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 234 kB/s | 18 kB 00:00 2024-04-23T10:06:47.876 INFO:teuthology.orchestra.run.smithi183.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 346 kB/s | 23 kB 00:00 2024-04-23T10:06:47.893 INFO:teuthology.orchestra.run.smithi183.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 332 kB/s | 19 kB 00:00 2024-04-23T10:06:47.935 INFO:teuthology.orchestra.run.smithi183.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 335 kB/s | 20 kB 00:00 2024-04-23T10:06:48.002 INFO:teuthology.orchestra.run.smithi183.stdout:(129/152): python3-kubernetes-26.1.0-2.el9.noar 9.4 MB/s | 1.0 MB 00:00 2024-04-23T10:06:48.019 INFO:teuthology.orchestra.run.smithi183.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 554 kB/s | 46 kB 00:00 2024-04-23T10:06:48.086 INFO:teuthology.orchestra.run.smithi183.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 944 kB/s | 79 kB 00:00 2024-04-23T10:06:48.102 INFO:teuthology.orchestra.run.smithi183.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 691 kB/s | 58 kB 00:00 2024-04-23T10:06:48.161 INFO:teuthology.orchestra.run.smithi183.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-04-23T10:06:48.187 INFO:teuthology.orchestra.run.smithi183.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 196 kB/s | 16 kB 00:00 2024-04-23T10:06:48.237 INFO:teuthology.orchestra.run.smithi183.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-04-23T10:06:48.262 INFO:teuthology.orchestra.run.smithi183.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 375 kB/s | 28 kB 00:00 2024-04-23T10:06:48.304 INFO:teuthology.orchestra.run.smithi183.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 461 kB/s | 31 kB 00:00 2024-04-23T10:06:48.338 INFO:teuthology.orchestra.run.smithi183.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 2.5 MB/s | 188 kB 00:00 2024-04-23T10:06:48.380 INFO:teuthology.orchestra.run.smithi183.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 785 kB/s | 59 kB 00:00 2024-04-23T10:06:48.405 INFO:teuthology.orchestra.run.smithi183.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 535 kB/s | 36 kB 00:00 2024-04-23T10:06:48.455 INFO:teuthology.orchestra.run.smithi183.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 678 kB/s | 51 kB 00:00 2024-04-23T10:06:48.488 INFO:teuthology.orchestra.run.smithi183.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-04-23T10:06:48.531 INFO:teuthology.orchestra.run.smithi183.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-04-23T10:06:48.564 INFO:teuthology.orchestra.run.smithi183.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 5.5 MB/s | 427 kB 00:00 2024-04-23T10:06:48.598 INFO:teuthology.orchestra.run.smithi183.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 298 kB/s | 20 kB 00:00 2024-04-23T10:06:48.640 INFO:teuthology.orchestra.run.smithi183.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-04-23T10:06:48.724 INFO:teuthology.orchestra.run.smithi183.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-04-23T10:06:51.701 INFO:teuthology.orchestra.run.smithi196.stdout:(26/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x8 172 kB/s | 17 MB 01:40 2024-04-23T10:06:53.393 INFO:teuthology.orchestra.run.smithi196.stdout:(27/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 101 kB/s | 170 kB 00:01 2024-04-23T10:06:53.893 INFO:teuthology.orchestra.run.smithi196.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3156.g 68 kB/s | 34 kB 00:00 2024-04-23T10:06:54.112 INFO:teuthology.orchestra.run.smithi097.stdout:(30/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 190 kB/s | 5.4 MB 00:28 2024-04-23T10:06:55.251 INFO:teuthology.orchestra.run.smithi196.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 105 kB/s | 142 kB 00:01 2024-04-23T10:06:57.985 INFO:teuthology.orchestra.run.smithi196.stdout:(30/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 235 kB/s | 3.1 MB 00:13 2024-04-23T10:07:03.928 INFO:teuthology.orchestra.run.smithi097.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 218 kB/s | 3.5 MB 00:16 2024-04-23T10:07:05.013 INFO:teuthology.orchestra.run.smithi183.stdout:(148/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 198 kB/s | 3.1 MB 00:16 2024-04-23T10:07:05.255 INFO:teuthology.orchestra.run.smithi183.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 5.6 MB/s | 1.3 MB 00:00 2024-04-23T10:07:05.337 INFO:teuthology.orchestra.run.smithi097.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3156.gd6 177 kB/s | 249 kB 00:01 2024-04-23T10:07:05.447 INFO:teuthology.orchestra.run.smithi183.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 12 MB/s | 2.4 MB 00:00 2024-04-23T10:07:05.995 INFO:teuthology.orchestra.run.smithi097.stdout:(33/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 75 kB/s | 49 kB 00:00 2024-04-23T10:07:06.412 INFO:teuthology.orchestra.run.smithi097.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3156.gd 39 kB/s | 16 kB 00:00 2024-04-23T10:07:08.804 INFO:teuthology.orchestra.run.smithi097.stdout:(35/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 114 kB/s | 273 kB 00:02 2024-04-23T10:07:11.171 INFO:teuthology.orchestra.run.smithi183.stdout:(151/152): librados2-19.0.0-3156.gd69f13d8.el7. 154 kB/s | 3.4 MB 00:22 2024-04-23T10:07:13.612 INFO:teuthology.orchestra.run.smithi097.stdout:(36/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 159 kB/s | 762 kB 00:04 2024-04-23T10:07:13.803 INFO:teuthology.orchestra.run.smithi097.stdout:(37/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 95 kB/s | 18 kB 00:00 2024-04-23T10:07:13.946 INFO:teuthology.orchestra.run.smithi097.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 569 kB/s | 80 kB 00:00 2024-04-23T10:07:14.196 INFO:teuthology.orchestra.run.smithi097.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 3.1 MB/s | 803 kB 00:00 2024-04-23T10:07:14.279 INFO:teuthology.orchestra.run.smithi097.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 195 kB 00:00 2024-04-23T10:07:14.372 INFO:teuthology.orchestra.run.smithi097.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.7 MB/s | 253 kB 00:00 2024-04-23T10:07:14.689 INFO:teuthology.orchestra.run.smithi097.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 3.9 MB/s | 1.2 MB 00:00 2024-04-23T10:07:14.739 INFO:teuthology.orchestra.run.smithi097.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-04-23T10:07:14.789 INFO:teuthology.orchestra.run.smithi097.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-04-23T10:07:14.839 INFO:teuthology.orchestra.run.smithi097.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 2.5 MB/s | 125 kB 00:00 2024-04-23T10:07:14.923 INFO:teuthology.orchestra.run.smithi097.stdout:(46/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-04-23T10:07:15.307 INFO:teuthology.orchestra.run.smithi097.stdout:(47/152): automake-1.16.2-8.el9.noarch.rpm 1.8 MB/s | 689 kB 00:00 2024-04-23T10:07:15.415 INFO:teuthology.orchestra.run.smithi097.stdout:(48/152): bison-3.7.4-5.el9.x86_64.rpm 8.5 MB/s | 943 kB 00:00 2024-04-23T10:07:15.482 INFO:teuthology.orchestra.run.smithi097.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-04-23T10:07:15.524 INFO:teuthology.orchestra.run.smithi097.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 515 kB/s | 21 kB 00:00 2024-04-23T10:07:15.599 INFO:teuthology.orchestra.run.smithi097.stdout:(51/152): flex-2.6.4-9.el9.x86_64.rpm 4.1 MB/s | 313 kB 00:00 2024-04-23T10:07:15.641 INFO:teuthology.orchestra.run.smithi097.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 785 kB/s | 33 kB 00:00 2024-04-23T10:07:15.800 INFO:teuthology.orchestra.run.smithi097.stdout:(53/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 19 MB/s | 3.0 MB 00:00 2024-04-23T10:07:15.850 INFO:teuthology.orchestra.run.smithi097.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 364 kB/s | 18 kB 00:00 2024-04-23T10:07:15.892 INFO:teuthology.orchestra.run.smithi097.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 268 kB/s | 11 kB 00:00 2024-04-23T10:07:15.959 INFO:teuthology.orchestra.run.smithi097.stdout:(56/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.4 MB/s | 160 kB 00:00 2024-04-23T10:07:16.018 INFO:teuthology.orchestra.run.smithi097.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 774 kB/s | 45 kB 00:00 2024-04-23T10:07:16.110 INFO:teuthology.orchestra.run.smithi097.stdout:(58/152): librdkafka-1.6.1-102.el9.x86_64.rpm 7.1 MB/s | 662 kB 00:00 2024-04-23T10:07:16.218 INFO:teuthology.orchestra.run.smithi097.stdout:(59/152): libtool-2.4.6-45.el9.x86_64.rpm 5.3 MB/s | 582 kB 00:00 2024-04-23T10:07:16.269 INFO:teuthology.orchestra.run.smithi097.stdout:(60/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 511 kB/s | 25 kB 00:00 2024-04-23T10:07:16.327 INFO:teuthology.orchestra.run.smithi097.stdout:(61/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 813 kB/s | 47 kB 00:00 2024-04-23T10:07:16.411 INFO:teuthology.orchestra.run.smithi097.stdout:(62/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-04-23T10:07:16.478 INFO:teuthology.orchestra.run.smithi097.stdout:(63/152): lua-5.4.4-4.el9.x86_64.rpm 2.8 MB/s | 188 kB 00:00 2024-04-23T10:07:16.528 INFO:teuthology.orchestra.run.smithi097.stdout:(64/152): openblas-0.3.21-2.el9.x86_64.rpm 705 kB/s | 35 kB 00:00 2024-04-23T10:07:16.737 INFO:teuthology.orchestra.run.smithi097.stdout:(65/152): openblas-openmp-0.3.21-2.el9.x86_64.r 23 MB/s | 4.7 MB 00:00 2024-04-23T10:07:16.754 INFO:teuthology.orchestra.run.smithi097.stdout:[MIRROR] openssl-devel-3.2.1-1.el9.x86_64.rpm: Status code: 404 for http://mirror.rackspace.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/openssl-devel-3.2.1-1.el9.x86_64.rpm (IP: 72.4.120.219) 2024-04-23T10:07:16.808 INFO:teuthology.orchestra.run.smithi097.stdout:[MIRROR] openssl-devel-3.2.1-1.el9.x86_64.rpm: Status code: 404 for https://mirror.rackspace.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/openssl-devel-3.2.1-1.el9.x86_64.rpm (IP: 72.4.120.219) 2024-04-23T10:07:21.614 INFO:teuthology.orchestra.run.smithi196.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 137 kB/s | 3.5 MB 00:26 2024-04-23T10:07:22.837 INFO:teuthology.orchestra.run.smithi097.stdout:(66/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 743 kB/s | 4.4 MB 00:06 2024-04-23T10:07:22.995 INFO:teuthology.orchestra.run.smithi097.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2024-04-23T10:07:23.137 INFO:teuthology.orchestra.run.smithi097.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 6.3 MB/s | 911 kB 00:00 2024-04-23T10:07:23.239 INFO:teuthology.orchestra.run.smithi196.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3156.gd6 153 kB/s | 249 kB 00:01 2024-04-23T10:07:23.246 INFO:teuthology.orchestra.run.smithi097.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 1.2 MB/s | 133 kB 00:00 2024-04-23T10:07:23.521 INFO:teuthology.orchestra.run.smithi097.stdout:(70/152): postgresql-server-13.14-1.el9.x86_64. 21 MB/s | 5.8 MB 00:00 2024-04-23T10:07:23.639 INFO:teuthology.orchestra.run.smithi196.stdout:(33/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 123 kB/s | 49 kB 00:00 2024-04-23T10:07:23.671 INFO:teuthology.orchestra.run.smithi097.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 6.7 MB/s | 1.0 MB 00:00 2024-04-23T10:07:23.764 INFO:teuthology.orchestra.run.smithi097.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 954 kB/s | 87 kB 00:00 2024-04-23T10:07:23.848 INFO:teuthology.orchestra.run.smithi196.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3156.gd 79 kB/s | 16 kB 00:00 2024-04-23T10:07:24.039 INFO:teuthology.orchestra.run.smithi097.stdout:(73/152): python3-babel-2.9.1-2.el9.noarch.rpm 22 MB/s | 6.0 MB 00:00 2024-04-23T10:07:24.147 INFO:teuthology.orchestra.run.smithi097.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 2.2 MB/s | 244 kB 00:00 2024-04-23T10:07:24.256 INFO:teuthology.orchestra.run.smithi097.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.2 MB/s | 248 kB 00:00 2024-04-23T10:07:24.331 INFO:teuthology.orchestra.run.smithi097.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 342 kB/s | 26 kB 00:00 2024-04-23T10:07:24.407 INFO:teuthology.orchestra.run.smithi097.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 255 kB/s | 19 kB 00:00 2024-04-23T10:07:24.507 INFO:teuthology.orchestra.run.smithi097.stdout:(78/152): python3-mako-1.1.4-6.el9.noarch.rpm 1.7 MB/s | 172 kB 00:00 2024-04-23T10:07:24.607 INFO:teuthology.orchestra.run.smithi097.stdout:(79/152): python3-markupsafe-1.1.1-12.el9.x86_6 348 kB/s | 35 kB 00:00 2024-04-23T10:07:24.866 INFO:teuthology.orchestra.run.smithi097.stdout:(80/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 20 MB/s | 5.1 MB 00:00 2024-04-23T10:07:24.974 INFO:teuthology.orchestra.run.smithi097.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-04-23T10:07:25.083 INFO:teuthology.orchestra.run.smithi097.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.0 MB/s | 222 kB 00:00 2024-04-23T10:07:25.175 INFO:teuthology.orchestra.run.smithi097.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 843 kB/s | 77 kB 00:00 2024-04-23T10:07:25.259 INFO:teuthology.orchestra.run.smithi097.stdout:(84/152): python3-prettytable-0.7.2-27.el9.noar 508 kB/s | 42 kB 00:00 2024-04-23T10:07:25.384 INFO:teuthology.orchestra.run.smithi097.stdout:(85/152): python3-protobuf-3.14.0-13.el9.noarch 2.1 MB/s | 269 kB 00:00 2024-04-23T10:07:25.484 INFO:teuthology.orchestra.run.smithi097.stdout:(86/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.6 MB/s | 159 kB 00:00 2024-04-23T10:07:25.593 INFO:teuthology.orchestra.run.smithi097.stdout:(87/152): python3-pyasn1-modules-0.4.8-6.el9.no 2.5 MB/s | 279 kB 00:00 2024-04-23T10:07:25.685 INFO:teuthology.orchestra.run.smithi097.stdout:(88/152): python3-pytz-2021.1-5.el9.noarch.rpm 557 kB/s | 51 kB 00:00 2024-04-23T10:07:25.777 INFO:teuthology.orchestra.run.smithi097.stdout:(89/152): python3-requests-oauthlib-1.3.0-12.el 586 kB/s | 54 kB 00:00 2024-04-23T10:07:26.098 INFO:teuthology.orchestra.run.smithi196.stdout:(35/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 121 kB/s | 273 kB 00:02 2024-04-23T10:07:26.327 INFO:teuthology.orchestra.run.smithi097.stdout:(90/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 30 MB/s | 16 MB 00:00 2024-04-23T10:07:26.419 INFO:teuthology.orchestra.run.smithi097.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 458 kB/s | 42 kB 00:00 2024-04-23T10:07:26.511 INFO:teuthology.orchestra.run.smithi097.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 483 kB/s | 44 kB 00:00 2024-04-23T10:07:26.628 INFO:teuthology.orchestra.run.smithi097.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 1.8 MB/s | 221 kB 00:00 2024-04-23T10:07:26.720 INFO:teuthology.orchestra.run.smithi097.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 392 kB/s | 36 kB 00:00 2024-04-23T10:07:26.804 INFO:teuthology.orchestra.run.smithi097.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 569 kB/s | 47 kB 00:00 2024-04-23T10:07:26.921 INFO:teuthology.orchestra.run.smithi097.stdout:(96/152): socat-1.7.4.1-5.el9.x86_64.rpm 2.5 MB/s | 305 kB 00:00 2024-04-23T10:07:27.004 INFO:teuthology.orchestra.run.smithi097.stdout:(97/152): uuid-1.6.2-55.el9.x86_64.rpm 695 kB/s | 58 kB 00:00 2024-04-23T10:07:27.105 INFO:teuthology.orchestra.run.smithi097.stdout:(98/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 938 kB/s | 94 kB 00:00 2024-04-23T10:07:27.205 INFO:teuthology.orchestra.run.smithi097.stdout:(99/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 638 kB/s | 64 kB 00:00 2024-04-23T10:07:27.297 INFO:teuthology.orchestra.run.smithi097.stdout:(100/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 488 kB/s | 45 kB 00:00 2024-04-23T10:07:27.364 INFO:teuthology.orchestra.run.smithi097.stdout:(101/152): lua-devel-5.4.4-4.el9.x86_64.rpm 336 kB/s | 22 kB 00:00 2024-04-23T10:07:27.489 INFO:teuthology.orchestra.run.smithi097.stdout:(102/152): protobuf-compiler-3.14.0-13.el9.x86_ 6.7 MB/s | 863 kB 00:00 2024-04-23T10:07:27.881 INFO:teuthology.orchestra.run.smithi097.stdout:(103/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.4 MB/s | 548 kB 00:00 2024-04-23T10:07:28.058 INFO:teuthology.orchestra.run.smithi097.stdout:(104/152): gperftools-libs-2.9.1-2.el9.x86_64.r 1.7 MB/s | 309 kB 00:00 2024-04-23T10:07:28.198 INFO:teuthology.orchestra.run.smithi097.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 139 kB/s | 19 kB 00:00 2024-04-23T10:07:28.615 INFO:teuthology.orchestra.run.smithi097.stdout:(106/152): libarrow-9.0.0-10.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-04-23T10:07:28.766 INFO:teuthology.orchestra.run.smithi097.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 173 kB/s | 26 kB 00:00 2024-04-23T10:07:28.899 INFO:teuthology.orchestra.run.smithi097.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 364 kB/s | 49 kB 00:00 2024-04-23T10:07:29.025 INFO:teuthology.orchestra.run.smithi097.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 539 kB/s | 67 kB 00:00 2024-04-23T10:07:29.108 INFO:teuthology.orchestra.run.smithi097.stdout:(110/152): luarocks-3.9.2-1.el9.noarch.rpm 1.8 MB/s | 151 kB 00:00 2024-04-23T10:07:29.258 INFO:teuthology.orchestra.run.smithi097.stdout:(111/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 5.5 MB/s | 838 kB 00:00 2024-04-23T10:07:29.401 INFO:teuthology.orchestra.run.smithi097.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 3.8 MB/s | 548 kB 00:00 2024-04-23T10:07:29.526 INFO:teuthology.orchestra.run.smithi097.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 347 kB/s | 43 kB 00:00 2024-04-23T10:07:29.651 INFO:teuthology.orchestra.run.smithi097.stdout:(114/152): python3-cachetools-4.2.4-1.el9.noarc 257 kB/s | 32 kB 00:00 2024-04-23T10:07:29.751 INFO:teuthology.orchestra.run.smithi097.stdout:(115/152): python3-certifi-2021.10.8-2.el9.noar 150 kB/s | 15 kB 00:00 2024-04-23T10:07:29.893 INFO:teuthology.orchestra.run.smithi097.stdout:(116/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.2 MB/s | 172 kB 00:00 2024-04-23T10:07:29.993 INFO:teuthology.orchestra.run.smithi097.stdout:(117/152): python3-cherrypy-18.6.1-2.el9.noarch 3.5 MB/s | 358 kB 00:00 2024-04-23T10:07:30.081 INFO:teuthology.orchestra.run.smithi196.stdout:(36/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 191 kB/s | 762 kB 00:03 2024-04-23T10:07:30.102 INFO:teuthology.orchestra.run.smithi097.stdout:(118/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.7 MB/s | 303 kB 00:00 2024-04-23T10:07:30.194 INFO:teuthology.orchestra.run.smithi097.stdout:(119/152): python3-colorama-0.4.4-12.el9.noarch 385 kB/s | 35 kB 00:00 2024-04-23T10:07:30.286 INFO:teuthology.orchestra.run.smithi097.stdout:(120/152): python3-google-auth-2.29.0-1.el9.noa 2.2 MB/s | 206 kB 00:00 2024-04-23T10:07:30.289 INFO:teuthology.orchestra.run.smithi196.stdout:(37/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 88 kB/s | 18 kB 00:00 2024-04-23T10:07:30.348 INFO:teuthology.orchestra.run.smithi196.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 1.4 MB/s | 80 kB 00:00 2024-04-23T10:07:30.448 INFO:teuthology.orchestra.run.smithi196.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 7.8 MB/s | 803 kB 00:00 2024-04-23T10:07:30.512 INFO:teuthology.orchestra.run.smithi097.stdout:(121/152): python3-grpcio-1.46.7-10.el9.x86_64. 9.1 MB/s | 2.0 MB 00:00 2024-04-23T10:07:30.515 INFO:teuthology.orchestra.run.smithi196.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 2.9 MB/s | 195 kB 00:00 2024-04-23T10:07:30.590 INFO:teuthology.orchestra.run.smithi196.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.3 MB/s | 253 kB 00:00 2024-04-23T10:07:30.637 INFO:teuthology.orchestra.run.smithi097.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 1.1 MB/s | 144 kB 00:00 2024-04-23T10:07:30.707 INFO:teuthology.orchestra.run.smithi196.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.2 MB 00:00 2024-04-23T10:07:30.746 INFO:teuthology.orchestra.run.smithi097.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 99 kB/s | 11 kB 00:00 2024-04-23T10:07:30.807 INFO:teuthology.orchestra.run.smithi196.stdout:(43/152): ceph-mgr-diskprediction-local-19.0.0- 230 kB/s | 7.4 MB 00:32 2024-04-23T10:07:30.829 INFO:teuthology.orchestra.run.smithi097.stdout:(124/152): python3-jaraco-classes-3.2.1-5.el9.n 212 kB/s | 18 kB 00:00 2024-04-23T10:07:30.833 INFO:teuthology.orchestra.run.smithi196.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 851 kB/s | 106 kB 00:00 2024-04-23T10:07:30.892 INFO:teuthology.orchestra.run.smithi196.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 135 kB 00:00 2024-04-23T10:07:30.917 INFO:teuthology.orchestra.run.smithi196.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-04-23T10:07:30.979 INFO:teuthology.orchestra.run.smithi097.stdout:(125/152): python3-jaraco-collections-3.0.0-8.e 154 kB/s | 23 kB 00:00 2024-04-23T10:07:30.984 INFO:teuthology.orchestra.run.smithi196.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.3 MB/s | 215 kB 00:00 2024-04-23T10:07:31.072 INFO:teuthology.orchestra.run.smithi097.stdout:(126/152): python3-jaraco-functools-3.5.0-2.el9 212 kB/s | 19 kB 00:00 2024-04-23T10:07:31.084 INFO:teuthology.orchestra.run.smithi196.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 4.0 MB/s | 689 kB 00:00 2024-04-23T10:07:31.126 INFO:teuthology.orchestra.run.smithi196.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 2.5 MB/s | 107 kB 00:00 2024-04-23T10:07:31.168 INFO:teuthology.orchestra.run.smithi196.stdout:(50/152): bison-3.7.4-5.el9.x86_64.rpm 5.0 MB/s | 943 kB 00:00 2024-04-23T10:07:31.188 INFO:teuthology.orchestra.run.smithi097.stdout:(127/152): python3-jaraco-text-3.2.0-6.el9.noar 169 kB/s | 20 kB 00:00 2024-04-23T10:07:31.193 INFO:teuthology.orchestra.run.smithi196.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 321 kB/s | 21 kB 00:00 2024-04-23T10:07:31.219 INFO:teuthology.orchestra.run.smithi196.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-04-23T10:07:31.244 INFO:teuthology.orchestra.run.smithi196.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 653 kB/s | 33 kB 00:00 2024-04-23T10:07:31.286 INFO:teuthology.orchestra.run.smithi196.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 438 kB/s | 18 kB 00:00 2024-04-23T10:07:31.344 INFO:teuthology.orchestra.run.smithi196.stdout:(55/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2024-04-23T10:07:31.355 INFO:teuthology.orchestra.run.smithi097.stdout:(128/152): python3-kubernetes-26.1.0-2.el9.noar 6.1 MB/s | 1.0 MB 00:00 2024-04-23T10:07:31.370 INFO:teuthology.orchestra.run.smithi196.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 133 kB/s | 11 kB 00:00 2024-04-23T10:07:31.395 INFO:teuthology.orchestra.run.smithi196.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-04-23T10:07:31.412 INFO:teuthology.orchestra.run.smithi196.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-04-23T10:07:31.454 INFO:teuthology.orchestra.run.smithi196.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-04-23T10:07:31.479 INFO:teuthology.orchestra.run.smithi196.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 8.5 MB/s | 582 kB 00:00 2024-04-23T10:07:31.480 INFO:teuthology.orchestra.run.smithi097.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 370 kB/s | 46 kB 00:00 2024-04-23T10:07:31.496 INFO:teuthology.orchestra.run.smithi196.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 610 kB/s | 25 kB 00:00 2024-04-23T10:07:31.513 INFO:teuthology.orchestra.run.smithi196.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-04-23T10:07:31.538 INFO:teuthology.orchestra.run.smithi196.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.8 MB/s | 292 kB 00:00 2024-04-23T10:07:31.563 INFO:teuthology.orchestra.run.smithi196.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-04-23T10:07:31.580 INFO:teuthology.orchestra.run.smithi196.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 843 kB/s | 35 kB 00:00 2024-04-23T10:07:31.647 INFO:teuthology.orchestra.run.smithi097.stdout:(130/152): python3-more-itertools-8.12.0-2.el9. 473 kB/s | 79 kB 00:00 2024-04-23T10:07:31.681 INFO:teuthology.orchestra.run.smithi196.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 41 MB/s | 4.7 MB 00:00 2024-04-23T10:07:31.756 INFO:teuthology.orchestra.run.smithi196.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 25 MB/s | 4.4 MB 00:00 2024-04-23T10:07:31.764 INFO:teuthology.orchestra.run.smithi097.stdout:(131/152): python3-natsort-7.1.1-5.el9.noarch.r 494 kB/s | 58 kB 00:00 2024-04-23T10:07:31.798 INFO:teuthology.orchestra.run.smithi196.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-04-23T10:07:31.831 INFO:teuthology.orchestra.run.smithi196.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 12 MB/s | 911 kB 00:00 2024-04-23T10:07:31.857 INFO:teuthology.orchestra.run.smithi196.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-04-23T10:07:31.873 INFO:teuthology.orchestra.run.smithi097.stdout:(132/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.5 MB/s | 272 kB 00:00 2024-04-23T10:07:31.915 INFO:teuthology.orchestra.run.smithi196.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 17 MB/s | 1.0 MB 00:00 2024-04-23T10:07:31.999 INFO:teuthology.orchestra.run.smithi196.stdout:(72/152): postgresql-server-13.14-1.el9.x86_64. 35 MB/s | 5.8 MB 00:00 2024-04-23T10:07:32.024 INFO:teuthology.orchestra.run.smithi196.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 807 kB/s | 87 kB 00:00 2024-04-23T10:07:32.066 INFO:teuthology.orchestra.run.smithi196.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 5.7 MB/s | 244 kB 00:00 2024-04-23T10:07:32.082 INFO:teuthology.orchestra.run.smithi097.stdout:(133/152): python3-portend-3.1.0-2.el9.noarch.r 79 kB/s | 16 kB 00:00 2024-04-23T10:07:32.158 INFO:teuthology.orchestra.run.smithi196.stdout:(75/152): python3-babel-2.9.1-2.el9.noarch.rpm 38 MB/s | 6.0 MB 00:00 2024-04-23T10:07:32.184 INFO:teuthology.orchestra.run.smithi196.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.1 MB/s | 248 kB 00:00 2024-04-23T10:07:32.207 INFO:teuthology.orchestra.run.smithi097.stdout:(134/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 721 kB/s | 90 kB 00:00 2024-04-23T10:07:32.209 INFO:teuthology.orchestra.run.smithi196.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 512 kB/s | 26 kB 00:00 2024-04-23T10:07:32.234 INFO:teuthology.orchestra.run.smithi196.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-04-23T10:07:32.260 INFO:teuthology.orchestra.run.smithi196.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-04-23T10:07:32.285 INFO:teuthology.orchestra.run.smithi196.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 695 kB/s | 35 kB 00:00 2024-04-23T10:07:32.299 INFO:teuthology.orchestra.run.smithi097.stdout:(135/152): python3-pyperclip-1.8.2-2.el9.noarch 308 kB/s | 28 kB 00:00 2024-04-23T10:07:32.327 INFO:teuthology.orchestra.run.smithi196.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.9 MB/s | 250 kB 00:00 2024-04-23T10:07:32.369 INFO:teuthology.orchestra.run.smithi196.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 5.2 MB/s | 222 kB 00:00 2024-04-23T10:07:32.424 INFO:teuthology.orchestra.run.smithi097.stdout:(136/152): python3-repoze-lru-0.7-16.el9.noarch 246 kB/s | 31 kB 00:00 2024-04-23T10:07:32.436 INFO:teuthology.orchestra.run.smithi196.stdout:(83/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 29 MB/s | 5.1 MB 00:00 2024-04-23T10:07:32.461 INFO:teuthology.orchestra.run.smithi196.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 840 kB/s | 77 kB 00:00 2024-04-23T10:07:32.486 INFO:teuthology.orchestra.run.smithi196.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 845 kB/s | 42 kB 00:00 2024-04-23T10:07:32.511 INFO:teuthology.orchestra.run.smithi196.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-04-23T10:07:32.537 INFO:teuthology.orchestra.run.smithi196.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-04-23T10:07:32.549 INFO:teuthology.orchestra.run.smithi097.stdout:(137/152): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-04-23T10:07:32.562 INFO:teuthology.orchestra.run.smithi196.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-04-23T10:07:32.587 INFO:teuthology.orchestra.run.smithi196.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-23T10:07:32.612 INFO:teuthology.orchestra.run.smithi196.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-04-23T10:07:32.658 INFO:teuthology.orchestra.run.smithi097.stdout:(138/152): python3-rsa-4.9-2.el9.noarch.rpm 546 kB/s | 59 kB 00:00 2024-04-23T10:07:32.679 INFO:teuthology.orchestra.run.smithi196.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 627 kB/s | 42 kB 00:00 2024-04-23T10:07:32.721 INFO:teuthology.orchestra.run.smithi196.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-04-23T10:07:32.763 INFO:teuthology.orchestra.run.smithi196.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 5.2 MB/s | 221 kB 00:00 2024-04-23T10:07:32.775 INFO:teuthology.orchestra.run.smithi097.stdout:(139/152): python3-tempora-5.0.0-2.el9.noarch.r 306 kB/s | 36 kB 00:00 2024-04-23T10:07:32.900 INFO:teuthology.orchestra.run.smithi097.stdout:(140/152): python3-typing-extensions-4.4.0-2.el 408 kB/s | 51 kB 00:00 2024-04-23T10:07:32.939 INFO:teuthology.orchestra.run.smithi196.stdout:(94/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 46 MB/s | 16 MB 00:00 2024-04-23T10:07:32.964 INFO:teuthology.orchestra.run.smithi196.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 179 kB/s | 36 kB 00:00 2024-04-23T10:07:32.989 INFO:teuthology.orchestra.run.smithi196.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 951 kB/s | 47 kB 00:00 2024-04-23T10:07:33.001 INFO:teuthology.orchestra.run.smithi097.stdout:(141/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-04-23T10:07:33.014 INFO:teuthology.orchestra.run.smithi196.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-04-23T10:07:33.040 INFO:teuthology.orchestra.run.smithi196.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-04-23T10:07:33.065 INFO:teuthology.orchestra.run.smithi196.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-04-23T10:07:33.090 INFO:teuthology.orchestra.run.smithi196.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-23T10:07:33.115 INFO:teuthology.orchestra.run.smithi196.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 896 kB/s | 45 kB 00:00 2024-04-23T10:07:33.141 INFO:teuthology.orchestra.run.smithi196.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 444 kB/s | 22 kB 00:00 2024-04-23T10:07:33.143 INFO:teuthology.orchestra.run.smithi097.stdout:(142/152): python3-websocket-client-1.2.3-2.el9 633 kB/s | 90 kB 00:00 2024-04-23T10:07:33.174 INFO:teuthology.orchestra.run.smithi196.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 14 MB/s | 863 kB 00:00 2024-04-23T10:07:33.335 INFO:teuthology.orchestra.run.smithi097.stdout:(143/152): python3-werkzeug-2.0.3-3.el9.1.noarc 2.2 MB/s | 427 kB 00:00 2024-04-23T10:07:33.477 INFO:teuthology.orchestra.run.smithi097.stdout:(144/152): python3-zc-lockfile-2.0-10.el9.noarc 141 kB/s | 20 kB 00:00 2024-04-23T10:07:33.585 INFO:teuthology.orchestra.run.smithi097.stdout:(145/152): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-04-23T10:07:33.699 INFO:teuthology.orchestra.run.smithi196.stdout:(104/152): gperftools-libs-2.9.1-2.el9.x86_64.r 588 kB/s | 309 kB 00:00 2024-04-23T10:07:33.725 INFO:teuthology.orchestra.run.smithi196.stdout:(105/152): abseil-cpp-20211102.0-3.el9.x86_64.r 939 kB/s | 548 kB 00:00 2024-04-23T10:07:33.786 INFO:teuthology.orchestra.run.smithi097.stdout:(146/152): thrift-0.15.0-2.el9.x86_64.rpm 7.9 MB/s | 1.6 MB 00:00 2024-04-23T10:07:33.817 INFO:teuthology.orchestra.run.smithi196.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 166 kB/s | 19 kB 00:00 2024-04-23T10:07:33.917 INFO:teuthology.orchestra.run.smithi196.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 260 kB/s | 26 kB 00:00 2024-04-23T10:07:34.017 INFO:teuthology.orchestra.run.smithi196.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 487 kB/s | 49 kB 00:00 2024-04-23T10:07:34.126 INFO:teuthology.orchestra.run.smithi196.stdout:(109/152): libarrow-9.0.0-10.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-04-23T10:07:34.151 INFO:teuthology.orchestra.run.smithi196.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 504 kB/s | 67 kB 00:00 2024-04-23T10:07:34.229 INFO:teuthology.orchestra.run.smithi196.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.5 MB/s | 151 kB 00:00 2024-04-23T10:07:34.335 INFO:teuthology.orchestra.run.smithi196.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 5.0 MB/s | 548 kB 00:00 2024-04-23T10:07:34.394 INFO:teuthology.orchestra.run.smithi196.stdout:(113/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.4 MB/s | 838 kB 00:00 2024-04-23T10:07:34.427 INFO:teuthology.orchestra.run.smithi196.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 472 kB/s | 43 kB 00:00 2024-04-23T10:07:34.486 INFO:teuthology.orchestra.run.smithi196.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 350 kB/s | 32 kB 00:00 2024-04-23T10:07:34.511 INFO:teuthology.orchestra.run.smithi196.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 180 kB/s | 15 kB 00:00 2024-04-23T10:07:34.595 INFO:teuthology.orchestra.run.smithi196.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.5 MB/s | 172 kB 00:00 2024-04-23T10:07:34.620 INFO:teuthology.orchestra.run.smithi196.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 3.2 MB/s | 358 kB 00:00 2024-04-23T10:07:34.704 INFO:teuthology.orchestra.run.smithi196.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.7 MB/s | 303 kB 00:00 2024-04-23T10:07:34.720 INFO:teuthology.orchestra.run.smithi196.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 352 kB/s | 35 kB 00:00 2024-04-23T10:07:34.812 INFO:teuthology.orchestra.run.smithi196.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 1.9 MB/s | 206 kB 00:00 2024-04-23T10:07:34.871 INFO:teuthology.orchestra.run.smithi196.stdout:(122/152): python3-grpcio-1.46.7-10.el9.x86_64. 14 MB/s | 2.0 MB 00:00 2024-04-23T10:07:34.913 INFO:teuthology.orchestra.run.smithi196.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.4 MB/s | 144 kB 00:00 2024-04-23T10:07:34.972 INFO:teuthology.orchestra.run.smithi196.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 107 kB/s | 11 kB 00:00 2024-04-23T10:07:35.005 INFO:teuthology.orchestra.run.smithi196.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 192 kB/s | 18 kB 00:00 2024-04-23T10:07:35.089 INFO:teuthology.orchestra.run.smithi196.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 198 kB/s | 23 kB 00:00 2024-04-23T10:07:35.114 INFO:teuthology.orchestra.run.smithi196.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 179 kB/s | 19 kB 00:00 2024-04-23T10:07:35.189 INFO:teuthology.orchestra.run.smithi196.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 196 kB/s | 20 kB 00:00 2024-04-23T10:07:35.273 INFO:teuthology.orchestra.run.smithi196.stdout:(129/152): python3-kubernetes-26.1.0-2.el9.noar 6.4 MB/s | 1.0 MB 00:00 2024-04-23T10:07:35.298 INFO:teuthology.orchestra.run.smithi196.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 426 kB/s | 46 kB 00:00 2024-04-23T10:07:35.373 INFO:teuthology.orchestra.run.smithi196.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 788 kB/s | 79 kB 00:00 2024-04-23T10:07:35.399 INFO:teuthology.orchestra.run.smithi196.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 576 kB/s | 58 kB 00:00 2024-04-23T10:07:35.515 INFO:teuthology.orchestra.run.smithi196.stdout:(133/152): python3-portend-3.1.0-2.el9.noarch.r 141 kB/s | 16 kB 00:00 2024-04-23T10:07:35.541 INFO:teuthology.orchestra.run.smithi196.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 1.6 MB/s | 272 kB 00:00 2024-04-23T10:07:35.616 INFO:teuthology.orchestra.run.smithi196.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 898 kB/s | 90 kB 00:00 2024-04-23T10:07:35.641 INFO:teuthology.orchestra.run.smithi196.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 281 kB/s | 28 kB 00:00 2024-04-23T10:07:35.725 INFO:teuthology.orchestra.run.smithi196.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 283 kB/s | 31 kB 00:00 2024-04-23T10:07:35.750 INFO:teuthology.orchestra.run.smithi196.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-04-23T10:07:35.842 INFO:teuthology.orchestra.run.smithi196.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 506 kB/s | 59 kB 00:00 2024-04-23T10:07:35.867 INFO:teuthology.orchestra.run.smithi196.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 306 kB/s | 36 kB 00:00 2024-04-23T10:07:35.943 INFO:teuthology.orchestra.run.smithi196.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 509 kB/s | 51 kB 00:00 2024-04-23T10:07:35.968 INFO:teuthology.orchestra.run.smithi196.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-04-23T10:07:36.060 INFO:teuthology.orchestra.run.smithi196.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 767 kB/s | 90 kB 00:00 2024-04-23T10:07:36.102 INFO:teuthology.orchestra.run.smithi196.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-04-23T10:07:36.177 INFO:teuthology.orchestra.run.smithi196.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 171 kB/s | 20 kB 00:00 2024-04-23T10:07:36.219 INFO:teuthology.orchestra.run.smithi196.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 1.6 MB/s | 192 kB 00:00 2024-04-23T10:07:36.436 INFO:teuthology.orchestra.run.smithi196.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 6.1 MB/s | 1.6 MB 00:00 2024-04-23T10:07:39.360 INFO:teuthology.orchestra.run.smithi097.stdout:(147/152): ceph-mgr-diskprediction-local-19.0.0 167 kB/s | 7.4 MB 00:45 2024-04-23T10:07:45.368 INFO:teuthology.orchestra.run.smithi196.stdout:(148/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 360 kB/s | 3.1 MB 00:08 2024-04-23T10:07:45.386 INFO:teuthology.orchestra.run.smithi196.stdout:[MIRROR] openssl-3.2.1-1.el9.x86_64.rpm: Status code: 404 for http://mirror.rackspace.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/openssl-3.2.1-1.el9.x86_64.rpm (IP: 72.4.120.219) 2024-04-23T10:07:45.484 INFO:teuthology.orchestra.run.smithi196.stdout:[MIRROR] openssl-3.2.1-1.el9.x86_64.rpm: Status code: 404 for https://mirror.rackspace.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/openssl-3.2.1-1.el9.x86_64.rpm (IP: 72.4.120.219) 2024-04-23T10:07:46.285 INFO:teuthology.orchestra.run.smithi196.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 1.5 MB/s | 1.3 MB 00:00 2024-04-23T10:07:46.342 INFO:teuthology.orchestra.run.smithi196.stdout:[MIRROR] openssl-libs-3.2.1-1.el9.x86_64.rpm: Status code: 404 for https://mirror.rackspace.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/openssl-libs-3.2.1-1.el9.x86_64.rpm (IP: 72.4.120.219) 2024-04-23T10:07:46.359 INFO:teuthology.orchestra.run.smithi196.stdout:[MIRROR] openssl-libs-3.2.1-1.el9.x86_64.rpm: Status code: 404 for http://mirror.rackspace.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/openssl-libs-3.2.1-1.el9.x86_64.rpm (IP: 72.4.120.219) 2024-04-23T10:07:46.893 INFO:teuthology.orchestra.run.smithi196.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 3.9 MB/s | 2.4 MB 00:00 2024-04-23T10:07:47.294 INFO:teuthology.orchestra.run.smithi097.stdout:(148/152): librados2-19.0.0-3156.gd69f13d8.el7. 257 kB/s | 3.4 MB 00:13 2024-04-23T10:07:47.752 INFO:teuthology.orchestra.run.smithi097.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 2.9 MB/s | 1.3 MB 00:00 2024-04-23T10:07:48.352 INFO:teuthology.orchestra.run.smithi097.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 4.0 MB/s | 2.4 MB 00:00 2024-04-23T10:07:52.219 INFO:teuthology.orchestra.run.smithi097.stdout:(151/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 250 kB/s | 3.1 MB 00:12 2024-04-23T10:07:56.600 INFO:teuthology.orchestra.run.smithi196.stdout:(151/152): librados2-19.0.0-3156.gd69f13d8.el7. 171 kB/s | 3.4 MB 00:20 2024-04-23T10:08:39.574 INFO:teuthology.orchestra.run.smithi183.stdout:(152/152): ceph-test-19.0.0-3156.gd69f13d8.el7. 253 kB/s | 49 MB 03:18 2024-04-23T10:08:39.577 INFO:teuthology.orchestra.run.smithi183.stdout:-------------------------------------------------------------------------------- 2024-04-23T10:08:39.577 INFO:teuthology.orchestra.run.smithi183.stdout:Total 974 kB/s | 222 MB 03:53 2024-04-23T10:08:40.616 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction check 2024-04-23T10:08:40.755 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction check succeeded. 2024-04-23T10:08:40.755 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction test 2024-04-23T10:08:42.513 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction test succeeded. 2024-04-23T10:08:42.513 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction 2024-04-23T10:08:44.027 INFO:teuthology.orchestra.run.smithi196.stdout:(152/152): ceph-test-19.0.0-3156.gd69f13d8.el7. 251 kB/s | 49 MB 03:20 2024-04-23T10:08:44.030 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-04-23T10:08:44.030 INFO:teuthology.orchestra.run.smithi196.stdout:Total 974 kB/s | 222 MB 03:52 2024-04-23T10:08:44.755 INFO:teuthology.orchestra.run.smithi097.stdout:(152/152): ceph-test-19.0.0-3156.gd69f13d8.el7. 307 kB/s | 49 MB 02:43 2024-04-23T10:08:44.758 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-04-23T10:08:44.758 INFO:teuthology.orchestra.run.smithi097.stdout:Total 980 kB/s | 222 MB 03:51 2024-04-23T10:08:45.065 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-04-23T10:08:45.208 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-04-23T10:08:45.208 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-04-23T10:08:45.794 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-04-23T10:08:45.935 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-04-23T10:08:45.935 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-04-23T10:08:47.041 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-04-23T10:08:47.041 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-04-23T10:08:47.773 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-04-23T10:08:47.774 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-04-23T10:08:48.448 INFO:teuthology.orchestra.run.smithi183.stdout: Preparing : 1/1 2024-04-23T10:08:48.544 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-23T10:08:48.582 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-23T10:08:48.635 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-23T10:08:48.670 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-23T10:08:48.704 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-23T10:08:48.746 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-23T10:08:49.080 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-23T10:08:49.188 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-23T10:08:49.311 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-23T10:08:49.372 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-23T10:08:49.447 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-23T10:08:49.476 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-23T10:08:49.630 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-23T10:08:49.674 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-23T10:08:49.701 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-23T10:08:49.805 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-23T10:08:49.827 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-23T10:08:49.889 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-23T10:08:49.911 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-23T10:08:49.931 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-23T10:08:50.103 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-23T10:08:50.136 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-23T10:08:50.242 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-23T10:08:50.338 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-23T10:08:50.368 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-23T10:08:50.403 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-23T10:08:50.476 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-23T10:08:50.510 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-23T10:08:50.554 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-23T10:08:50.586 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-23T10:08:50.614 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-23T10:08:50.683 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-23T10:08:50.736 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-23T10:08:50.767 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-23T10:08:50.802 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-23T10:08:50.869 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-23T10:08:50.904 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-23T10:08:50.956 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-23T10:08:50.985 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-23T10:08:51.056 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-23T10:08:51.099 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-23T10:08:51.174 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-23T10:08:51.309 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-23T10:08:51.360 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-23T10:08:51.891 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-23T10:08:51.972 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-23T10:08:52.001 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-23T10:08:52.043 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-23T10:08:52.073 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-23T10:08:52.132 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-23T10:08:52.292 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-23T10:08:52.360 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-23T10:08:52.415 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-23T10:08:52.488 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-23T10:08:52.519 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-23T10:08:52.555 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-23T10:08:52.602 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-23T10:08:52.637 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-23T10:08:52.681 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-23T10:08:52.723 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-23T10:08:52.859 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-23T10:08:52.920 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-23T10:08:52.950 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-23T10:08:52.986 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-04-23T10:08:52.991 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-23T10:08:53.080 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-23T10:08:53.086 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-23T10:08:53.118 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-23T10:08:53.125 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-23T10:08:53.165 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-23T10:08:53.172 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-23T10:08:53.190 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-23T10:08:53.206 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-23T10:08:53.215 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-23T10:08:53.240 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-23T10:08:53.290 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-23T10:08:53.291 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-23T10:08:53.417 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-23T10:08:53.456 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-23T10:08:53.484 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-23T10:08:53.518 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-23T10:08:53.557 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-23T10:08:53.588 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-23T10:08:53.618 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-23T10:08:53.620 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-23T10:08:53.653 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-23T10:08:53.686 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-23T10:08:53.727 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-23T10:08:53.732 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-23T10:08:53.749 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-04-23T10:08:53.762 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-23T10:08:53.838 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-23T10:08:53.842 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-23T10:08:53.848 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-23T10:08:53.867 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-23T10:08:53.880 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-23T10:08:53.892 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-23T10:08:53.923 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-23T10:08:53.941 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-23T10:08:53.962 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-23T10:08:53.966 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-23T10:08:53.976 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-23T10:08:53.996 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-23T10:08:54.001 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-23T10:08:54.044 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-23T10:08:54.060 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-23T10:08:54.116 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-23T10:08:54.144 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-23T10:08:54.192 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-23T10:08:54.220 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-23T10:08:54.324 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-23T10:08:54.346 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-23T10:08:54.362 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-23T10:08:54.392 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-23T10:08:54.414 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-23T10:08:54.435 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-23T10:08:54.470 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-23T10:08:54.589 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-23T10:08:54.612 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-23T10:08:54.622 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-23T10:08:54.655 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-23T10:08:54.694 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-23T10:08:54.733 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-23T10:08:54.762 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-23T10:08:54.844 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-23T10:08:54.856 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-23T10:08:54.878 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-23T10:08:54.888 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-23T10:08:54.897 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-23T10:08:54.921 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-23T10:08:54.922 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-23T10:08:54.930 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-23T10:08:54.949 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-23T10:08:54.986 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-23T10:08:55.029 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-23T10:08:55.030 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-23T10:08:55.073 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-23T10:08:55.082 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-23T10:08:55.088 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-23T10:08:55.106 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-23T10:08:55.109 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-23T10:08:55.134 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-23T10:08:55.171 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-23T10:08:55.176 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-23T10:08:55.193 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-23T10:08:55.201 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-23T10:08:55.211 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-23T10:08:55.216 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-23T10:08:55.255 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-23T10:08:55.286 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-23T10:08:55.321 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-23T10:08:55.392 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-23T10:08:55.401 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-23T10:08:55.405 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-23T10:08:55.421 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-23T10:08:55.434 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-23T10:08:55.441 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-23T10:08:55.484 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-23T10:08:55.498 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-23T10:08:55.512 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-23T10:08:55.540 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-23T10:08:55.581 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-23T10:08:55.627 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-23T10:08:55.636 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-23T10:08:55.675 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-23T10:08:55.700 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-23T10:08:55.709 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-23T10:08:55.773 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-23T10:08:55.808 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-23T10:08:55.833 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-23T10:08:55.852 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-23T10:08:55.884 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-23T10:08:55.896 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-23T10:08:55.921 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-23T10:08:55.997 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-23T10:08:56.059 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-23T10:08:56.090 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-23T10:08:56.125 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-23T10:08:56.132 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-23T10:08:56.200 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-23T10:08:56.236 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-23T10:08:56.243 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-23T10:08:56.279 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-23T10:08:56.308 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-23T10:08:56.377 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-23T10:08:56.422 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-23T10:08:56.431 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-23T10:08:56.504 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-23T10:08:56.516 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-23T10:08:56.554 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-23T10:08:56.605 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-23T10:08:56.635 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-23T10:08:56.643 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-23T10:08:56.684 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-23T10:08:56.691 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-23T10:08:56.833 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-23T10:08:56.879 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-23T10:08:56.934 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-23T10:08:56.997 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-23T10:08:57.030 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-23T10:08:57.067 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-23T10:08:57.121 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-23T10:08:57.157 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-23T10:08:57.200 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-23T10:08:57.203 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-23T10:08:57.242 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-23T10:08:57.296 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-23T10:08:57.325 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-23T10:08:57.367 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-23T10:08:57.376 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-23T10:08:57.406 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-23T10:08:57.415 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-23T10:08:57.445 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-23T10:08:57.447 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-23T10:08:57.453 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-23T10:08:57.486 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-23T10:08:57.525 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-23T10:08:57.553 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-23T10:08:57.580 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-23T10:08:57.586 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-23T10:08:57.616 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-23T10:08:57.628 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-23T10:08:57.667 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-23T10:08:57.668 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-23T10:08:57.693 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-23T10:08:57.718 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-23T10:08:57.722 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-23T10:08:57.792 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-23T10:08:57.795 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-23T10:08:57.826 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-23T10:08:57.862 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-23T10:08:57.896 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-23T10:08:57.909 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-23T10:08:57.918 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-23T10:08:57.940 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-23T10:08:57.945 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-23T10:08:57.950 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-23T10:08:57.978 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-23T10:08:57.997 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-23T10:08:58.012 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-23T10:08:58.029 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-23T10:08:58.038 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-23T10:08:58.060 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-23T10:08:58.070 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-23T10:08:58.100 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-23T10:08:58.106 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-23T10:08:58.130 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-23T10:08:58.164 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-23T10:08:58.175 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-23T10:08:58.198 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-23T10:08:58.219 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-23T10:08:58.239 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-23T10:08:58.249 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-23T10:08:58.273 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-23T10:08:58.290 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-23T10:08:58.360 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-23T10:08:58.385 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-23T10:08:58.403 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-23T10:08:58.424 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-23T10:08:58.493 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-23T10:08:58.523 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-23T10:08:58.548 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-23T10:08:58.561 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-23T10:08:58.583 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-23T10:08:58.618 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-23T10:08:58.622 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-23T10:08:58.683 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-23T10:08:58.689 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-23T10:08:58.711 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-23T10:08:58.749 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-23T10:08:58.780 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-23T10:08:58.808 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-23T10:08:58.842 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-23T10:08:58.881 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-23T10:08:58.896 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-23T10:08:58.921 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-23T10:08:58.944 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-23T10:08:58.951 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-23T10:08:58.985 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-23T10:08:59.019 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-23T10:08:59.060 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-23T10:08:59.095 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-23T10:08:59.181 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-23T10:08:59.225 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-23T10:08:59.265 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-23T10:08:59.311 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-23T10:08:59.409 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-23T10:08:59.458 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-23T10:08:59.662 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-23T10:08:59.708 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-23T10:08:59.750 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-23T10:08:59.856 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-23T10:08:59.908 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-23T10:08:59.995 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-23T10:09:00.027 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-23T10:09:00.177 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-23T10:09:00.204 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-23T10:09:00.221 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-23T10:09:00.241 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-23T10:09:00.263 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-23T10:09:00.319 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-23T10:09:00.369 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-23T10:09:00.429 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-23T10:09:00.526 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-23T10:09:00.565 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-23T10:09:00.744 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-23T10:09:00.780 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-23T10:09:00.857 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-23T10:09:00.963 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-23T10:09:00.966 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-23T10:09:01.007 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-23T10:09:01.081 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-23T10:09:01.494 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-23T10:09:01.544 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-23T10:09:01.586 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-23T10:09:01.610 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-23T10:09:01.659 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-23T10:09:01.660 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-23T10:09:01.660 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-23T10:09:01.660 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-23T10:09:01.687 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-23T10:09:02.303 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-23T10:09:02.394 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-23T10:09:02.422 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-23T10:09:02.456 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-23T10:09:02.766 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-23T10:09:02.810 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-23T10:09:02.821 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-23T10:09:02.900 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-23T10:09:02.917 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-23T10:09:02.931 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-23T10:09:02.944 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-23T10:09:02.967 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-23T10:09:02.978 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-23T10:09:03.288 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-23T10:09:03.341 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-23T10:09:03.430 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-23T10:09:03.446 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-23T10:09:03.461 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-23T10:09:03.479 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-23T10:09:03.497 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-23T10:09:03.550 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-23T10:09:03.581 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-23T10:09:03.974 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-23T10:09:04.001 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-23T10:09:04.072 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-23T10:09:04.095 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-23T10:09:05.773 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-23T10:09:05.810 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-23T10:09:06.347 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-23T10:09:06.392 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-23T10:09:06.397 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-23T10:09:06.425 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-23T10:09:06.479 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-23T10:09:06.479 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-23T10:09:06.480 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-23T10:09:06.480 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-23T10:09:06.506 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-23T10:09:06.961 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-23T10:09:07.003 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-23T10:09:07.095 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-23T10:09:07.095 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-23T10:09:07.095 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-23T10:09:07.095 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:09:07.129 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-23T10:09:12.992 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-23T10:09:12.993 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /sys 2024-04-23T10:09:12.993 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /proc 2024-04-23T10:09:12.993 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /mnt 2024-04-23T10:09:12.993 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /var/tmp 2024-04-23T10:09:12.993 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /home 2024-04-23T10:09:12.993 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /root 2024-04-23T10:09:12.993 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /tmp 2024-04-23T10:09:12.993 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-23T10:09:13.165 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-23T10:09:13.180 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-23T10:09:13.180 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T10:09:13.180 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-23T10:09:13.180 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-23T10:09:14.643 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-23T10:09:14.668 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-23T10:09:14.668 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T10:09:14.668 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-23T10:09:14.668 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-23T10:09:14.668 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-23T10:09:14.668 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-23T10:09:14.893 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-23T10:09:14.915 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-23T10:09:14.915 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T10:09:14.915 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-23T10:09:14.915 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-23T10:09:14.916 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-23T10:09:14.916 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-23T10:09:15.352 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-23T10:09:15.374 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-23T10:09:15.374 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T10:09:15.374 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-23T10:09:15.374 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-23T10:09:15.375 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-23T10:09:15.375 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-23T10:09:15.474 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-23T10:09:15.522 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-23T10:09:15.586 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-23T10:09:15.773 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-23T10:09:15.827 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-23T10:09:15.897 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-23T10:09:15.986 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-23T10:09:16.126 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-23T10:09:16.169 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-23T10:09:16.201 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-23T10:09:16.229 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-23T10:09:16.689 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-23T10:09:16.719 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-23T10:09:16.751 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-23T10:09:16.812 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-23T10:09:16.850 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-23T10:09:17.877 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-23T10:09:17.877 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /sys 2024-04-23T10:09:17.877 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /proc 2024-04-23T10:09:17.877 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /mnt 2024-04-23T10:09:17.877 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /var/tmp 2024-04-23T10:09:17.878 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /home 2024-04-23T10:09:17.878 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /root 2024-04-23T10:09:17.878 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /tmp 2024-04-23T10:09:17.878 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-23T10:09:17.878 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-23T10:09:17.910 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-23T10:09:18.050 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-23T10:09:18.064 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-23T10:09:18.065 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T10:09:18.065 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-23T10:09:18.065 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-23T10:09:18.576 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-23T10:09:18.576 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /sys 2024-04-23T10:09:18.576 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /proc 2024-04-23T10:09:18.576 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /mnt 2024-04-23T10:09:18.576 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /var/tmp 2024-04-23T10:09:18.576 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /home 2024-04-23T10:09:18.576 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /root 2024-04-23T10:09:18.576 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /tmp 2024-04-23T10:09:18.576 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:09:18.749 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-23T10:09:18.764 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-23T10:09:18.764 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T10:09:18.764 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-23T10:09:18.764 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:09:18.866 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-23T10:09:18.902 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-23T10:09:19.003 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-23T10:09:19.166 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-23T10:09:19.188 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-23T10:09:19.210 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-23T10:09:19.210 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T10:09:19.210 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-23T10:09:19.210 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-23T10:09:19.210 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-23T10:09:19.210 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-23T10:09:19.255 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-23T10:09:19.266 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-23T10:09:19.364 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-23T10:09:19.411 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-23T10:09:19.545 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-23T10:09:19.570 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-23T10:09:19.570 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T10:09:19.570 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-23T10:09:19.570 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-23T10:09:19.570 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-23T10:09:19.570 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-23T10:09:19.787 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-23T10:09:19.809 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-23T10:09:19.809 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T10:09:19.809 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-23T10:09:19.809 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-23T10:09:19.809 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-23T10:09:19.809 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-23T10:09:20.235 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-23T10:09:20.260 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-23T10:09:20.260 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T10:09:20.261 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-23T10:09:20.261 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-23T10:09:20.261 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-23T10:09:20.261 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:09:20.263 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-23T10:09:20.284 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-23T10:09:20.284 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T10:09:20.284 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-23T10:09:20.285 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-23T10:09:20.285 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-23T10:09:20.285 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-23T10:09:20.334 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-23T10:09:20.366 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-23T10:09:20.388 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-23T10:09:20.389 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T10:09:20.389 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-23T10:09:20.389 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-23T10:09:20.389 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-23T10:09:20.389 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-23T10:09:20.392 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-23T10:09:20.433 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-23T10:09:20.441 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-23T10:09:20.455 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-23T10:09:20.455 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T10:09:20.455 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-23T10:09:20.455 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-23T10:09:20.496 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-23T10:09:20.500 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-23T10:09:20.503 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-23T10:09:20.522 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-23T10:09:20.522 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T10:09:20.522 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-23T10:09:20.523 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-23T10:09:20.523 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-23T10:09:20.523 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-23T10:09:20.525 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-23T10:09:20.525 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T10:09:20.525 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-23T10:09:20.525 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-23T10:09:20.525 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-23T10:09:20.525 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:09:20.689 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-23T10:09:20.738 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-23T10:09:20.815 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-23T10:09:20.835 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-23T10:09:20.857 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-23T10:09:20.857 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T10:09:20.857 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-23T10:09:20.857 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-23T10:09:20.857 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-23T10:09:20.857 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-23T10:09:20.903 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-23T10:09:20.970 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-23T10:09:20.992 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-23T10:09:20.992 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T10:09:20.992 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-23T10:09:20.992 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-23T10:09:20.992 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-23T10:09:20.992 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:09:21.043 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-23T10:09:21.096 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-23T10:09:21.100 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-23T10:09:21.128 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-23T10:09:21.149 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-23T10:09:21.156 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-23T10:09:21.203 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-23T10:09:21.390 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-23T10:09:21.445 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-23T10:09:21.515 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-23T10:09:21.554 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-23T10:09:21.588 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-23T10:09:21.604 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-23T10:09:21.620 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-23T10:09:21.681 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-23T10:09:21.711 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-23T10:09:21.733 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-23T10:09:21.779 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-23T10:09:21.811 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-23T10:09:21.838 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-23T10:09:22.266 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-23T10:09:22.303 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-23T10:09:22.335 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-23T10:09:22.397 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-23T10:09:22.435 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-23T10:09:22.749 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-23T10:09:22.796 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-23T10:09:23.466 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-23T10:09:23.523 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-23T10:09:23.756 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-23T10:09:23.788 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-23T10:09:23.888 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-23T10:09:24.062 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-23T10:09:24.098 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-23T10:09:24.121 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-23T10:09:24.121 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T10:09:24.121 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-23T10:09:24.121 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-23T10:09:24.121 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-23T10:09:24.121 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-23T10:09:24.165 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-23T10:09:24.176 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-23T10:09:24.275 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-23T10:09:24.322 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-23T10:09:24.490 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-23T10:09:24.549 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-23T10:09:24.649 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-23T10:09:24.909 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-23T10:09:24.939 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-23T10:09:24.961 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-23T10:09:24.961 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T10:09:24.961 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-23T10:09:24.961 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-23T10:09:24.962 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-23T10:09:24.962 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:09:25.006 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-23T10:09:25.017 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-23T10:09:25.115 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-23T10:09:25.163 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-23T10:09:25.244 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-23T10:09:25.277 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-23T10:09:25.299 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-23T10:09:25.299 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T10:09:25.299 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-23T10:09:25.299 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-23T10:09:25.299 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-23T10:09:25.299 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-23T10:09:25.344 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-23T10:09:25.365 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-23T10:09:25.365 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T10:09:25.365 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-23T10:09:25.365 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-23T10:09:25.420 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-23T10:09:25.441 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-23T10:09:25.442 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T10:09:25.442 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-23T10:09:25.442 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-23T10:09:25.442 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-23T10:09:25.442 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-23T10:09:25.496 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-23T10:09:25.538 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-23T10:09:25.579 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-23T10:09:25.610 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-23T10:09:25.638 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-23T10:09:25.753 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-23T10:09:25.775 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-23T10:09:25.775 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T10:09:25.775 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-23T10:09:25.775 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-23T10:09:25.775 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-23T10:09:25.775 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-23T10:09:25.825 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-23T10:09:25.875 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-23T10:09:25.905 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-23T10:09:26.086 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-23T10:09:26.109 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-23T10:09:26.131 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-23T10:09:26.131 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T10:09:26.131 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-23T10:09:26.131 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-23T10:09:26.131 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-23T10:09:26.132 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:09:26.197 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-23T10:09:26.219 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-23T10:09:26.219 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T10:09:26.219 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-23T10:09:26.219 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:09:26.269 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-23T10:09:26.290 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-23T10:09:26.291 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T10:09:26.291 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-23T10:09:26.291 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-23T10:09:26.291 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-23T10:09:26.291 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:09:26.478 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-23T10:09:26.511 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-23T10:09:26.530 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-23T10:09:26.603 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-23T10:09:26.624 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-23T10:09:26.625 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T10:09:26.625 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-23T10:09:26.625 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-23T10:09:26.625 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-23T10:09:26.625 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:09:27.359 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-23T10:09:27.449 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-23T10:09:27.490 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-23T10:09:27.573 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-23T10:09:27.611 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-23T10:09:27.611 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-23T10:09:27.678 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-23T10:09:27.711 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-23T10:09:27.712 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-23T10:09:27.786 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-23T10:09:27.852 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-23T10:09:30.382 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-23T10:09:30.423 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-23T10:09:30.464 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-23T10:09:30.496 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-23T10:09:30.523 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-23T10:09:30.706 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-23T10:09:30.752 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-23T10:09:30.782 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-23T10:09:31.206 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-23T10:09:31.247 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-23T10:09:31.288 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-23T10:09:31.320 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-23T10:09:31.347 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-23T10:09:31.387 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-23T10:09:31.430 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-23T10:09:31.448 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-23T10:09:31.531 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-23T10:09:31.569 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-23T10:09:31.598 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-23T10:09:32.217 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-23T10:09:32.255 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-23T10:09:32.274 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-23T10:09:32.277 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-23T10:09:32.361 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-23T10:09:32.401 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-23T10:09:32.483 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-23T10:09:32.515 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-23T10:09:32.515 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-23T10:09:32.599 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-23T10:09:32.632 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-23T10:09:32.632 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-23T10:09:32.721 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-23T10:09:32.790 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-23T10:09:33.102 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-23T10:09:33.186 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-23T10:09:33.227 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-23T10:09:33.309 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-23T10:09:33.341 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-23T10:09:33.341 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-23T10:09:33.425 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-23T10:09:33.449 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-23T10:09:33.449 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-23T10:09:33.543 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-23T10:09:33.552 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-23T10:09:33.553 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-23T10:09:33.553 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-23T10:09:33.553 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-23T10:09:33.553 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-23T10:09:33.553 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-23T10:09:33.553 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-23T10:09:33.553 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-23T10:09:33.553 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-23T10:09:33.553 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-23T10:09:33.553 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-23T10:09:33.553 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-23T10:09:33.553 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-23T10:09:33.553 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-23T10:09:33.553 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-23T10:09:33.554 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-23T10:09:33.554 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-23T10:09:33.554 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-23T10:09:33.554 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-23T10:09:33.554 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-23T10:09:33.554 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-23T10:09:33.554 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-23T10:09:33.554 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-23T10:09:33.554 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-23T10:09:33.554 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-23T10:09:33.554 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-23T10:09:33.554 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-23T10:09:33.554 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-23T10:09:33.555 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-23T10:09:33.555 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-23T10:09:33.555 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-23T10:09:33.555 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-23T10:09:33.555 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-23T10:09:33.555 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-23T10:09:33.555 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-23T10:09:33.555 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-23T10:09:33.555 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-23T10:09:33.555 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-23T10:09:33.555 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-23T10:09:33.555 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-23T10:09:33.555 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-23T10:09:33.555 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-23T10:09:33.555 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-23T10:09:33.555 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-23T10:09:33.556 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-23T10:09:33.556 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-23T10:09:33.556 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-23T10:09:33.556 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-23T10:09:33.556 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-23T10:09:33.556 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-23T10:09:33.556 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-23T10:09:33.556 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-23T10:09:33.556 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-23T10:09:33.556 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-23T10:09:33.556 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-23T10:09:33.556 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-23T10:09:33.556 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-23T10:09:33.556 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-23T10:09:33.557 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-23T10:09:33.557 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-23T10:09:33.557 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-23T10:09:33.557 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-23T10:09:33.557 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-23T10:09:33.557 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-23T10:09:33.557 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-23T10:09:33.557 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-23T10:09:33.557 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-23T10:09:33.557 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-23T10:09:33.557 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-23T10:09:33.557 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-23T10:09:33.557 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-23T10:09:33.557 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-23T10:09:33.557 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-23T10:09:33.557 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-23T10:09:33.558 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-23T10:09:33.558 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-23T10:09:33.558 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-23T10:09:33.558 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-23T10:09:33.558 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-23T10:09:33.558 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-23T10:09:33.558 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-23T10:09:33.558 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-23T10:09:33.558 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-23T10:09:33.558 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-23T10:09:33.558 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-23T10:09:33.558 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-23T10:09:33.558 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-23T10:09:33.558 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-23T10:09:33.558 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-23T10:09:33.559 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-23T10:09:33.559 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-23T10:09:33.559 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-23T10:09:33.559 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-23T10:09:33.559 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-23T10:09:33.559 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-23T10:09:33.559 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-23T10:09:33.559 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-23T10:09:33.559 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-23T10:09:33.559 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-23T10:09:33.559 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-23T10:09:33.559 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-23T10:09:33.559 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-23T10:09:33.559 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-23T10:09:33.560 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-23T10:09:33.560 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-23T10:09:33.560 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-23T10:09:33.560 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-23T10:09:33.560 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-23T10:09:33.560 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-23T10:09:33.560 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-23T10:09:33.560 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-23T10:09:33.560 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-23T10:09:33.560 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-23T10:09:33.560 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-23T10:09:33.560 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-23T10:09:33.560 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-23T10:09:33.560 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-23T10:09:33.561 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-23T10:09:33.561 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-23T10:09:33.561 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-23T10:09:33.561 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-23T10:09:33.561 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-23T10:09:33.561 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-23T10:09:33.561 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-23T10:09:33.561 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-23T10:09:33.561 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-23T10:09:33.561 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-23T10:09:33.561 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-23T10:09:33.561 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-23T10:09:33.561 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-23T10:09:33.561 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-23T10:09:33.561 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-23T10:09:33.561 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-23T10:09:33.562 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-23T10:09:33.562 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-23T10:09:33.562 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-23T10:09:33.562 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-23T10:09:33.562 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-23T10:09:33.562 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-23T10:09:33.562 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-23T10:09:33.562 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-23T10:09:33.562 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-23T10:09:33.562 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-23T10:09:33.562 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-23T10:09:33.562 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-23T10:09:33.562 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-23T10:09:33.562 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-23T10:09:33.563 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-23T10:09:33.563 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-23T10:09:33.563 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-23T10:09:33.563 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-23T10:09:33.563 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-23T10:09:33.563 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-23T10:09:33.563 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-23T10:09:33.563 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-23T10:09:33.563 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-23T10:09:33.606 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-23T10:09:35.740 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-23T10:09:35.740 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-23T10:09:35.740 INFO:teuthology.orchestra.run.smithi183.stdout:Upgraded: 2024-04-23T10:09:35.740 INFO:teuthology.orchestra.run.smithi183.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:35.740 INFO:teuthology.orchestra.run.smithi183.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:35.741 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-23T10:09:35.741 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-23T10:09:35.741 INFO:teuthology.orchestra.run.smithi183.stdout:Installed: 2024-04-23T10:09:35.741 INFO:teuthology.orchestra.run.smithi183.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-23T10:09:35.741 INFO:teuthology.orchestra.run.smithi183.stdout: automake-1.16.2-8.el9.noarch 2024-04-23T10:09:35.741 INFO:teuthology.orchestra.run.smithi183.stdout: bison-3.7.4-5.el9.x86_64 2024-04-23T10:09:35.741 INFO:teuthology.orchestra.run.smithi183.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-23T10:09:35.741 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:35.741 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:35.741 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:35.741 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:35.741 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T10:09:35.741 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:35.741 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:35.742 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:35.742 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T10:09:35.742 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T10:09:35.742 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T10:09:35.742 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T10:09:35.742 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T10:09:35.742 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:35.742 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:35.742 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T10:09:35.742 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:35.742 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:35.742 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:35.742 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T10:09:35.743 INFO:teuthology.orchestra.run.smithi183.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T10:09:35.743 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:35.743 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:35.743 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T10:09:35.743 INFO:teuthology.orchestra.run.smithi183.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-23T10:09:35.743 INFO:teuthology.orchestra.run.smithi183.stdout: flex-2.6.4-9.el9.x86_64 2024-04-23T10:09:35.743 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-23T10:09:35.743 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-23T10:09:35.743 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-23T10:09:35.743 INFO:teuthology.orchestra.run.smithi183.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-23T10:09:35.743 INFO:teuthology.orchestra.run.smithi183.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-23T10:09:35.743 INFO:teuthology.orchestra.run.smithi183.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-23T10:09:35.743 INFO:teuthology.orchestra.run.smithi183.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-23T10:09:35.743 INFO:teuthology.orchestra.run.smithi183.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-23T10:09:35.743 INFO:teuthology.orchestra.run.smithi183.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-23T10:09:35.744 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:35.744 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:35.744 INFO:teuthology.orchestra.run.smithi183.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:35.744 INFO:teuthology.orchestra.run.smithi183.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-23T10:09:35.744 INFO:teuthology.orchestra.run.smithi183.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-23T10:09:35.744 INFO:teuthology.orchestra.run.smithi183.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-23T10:09:35.744 INFO:teuthology.orchestra.run.smithi183.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-23T10:09:35.744 INFO:teuthology.orchestra.run.smithi183.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-23T10:09:35.744 INFO:teuthology.orchestra.run.smithi183.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:35.744 INFO:teuthology.orchestra.run.smithi183.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:35.744 INFO:teuthology.orchestra.run.smithi183.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-23T10:09:35.744 INFO:teuthology.orchestra.run.smithi183.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:35.744 INFO:teuthology.orchestra.run.smithi183.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-23T10:09:35.744 INFO:teuthology.orchestra.run.smithi183.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-23T10:09:35.745 INFO:teuthology.orchestra.run.smithi183.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-23T10:09:35.745 INFO:teuthology.orchestra.run.smithi183.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-23T10:09:35.745 INFO:teuthology.orchestra.run.smithi183.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-23T10:09:35.745 INFO:teuthology.orchestra.run.smithi183.stdout: lua-5.4.4-4.el9.x86_64 2024-04-23T10:09:35.745 INFO:teuthology.orchestra.run.smithi183.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-23T10:09:35.745 INFO:teuthology.orchestra.run.smithi183.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-23T10:09:35.745 INFO:teuthology.orchestra.run.smithi183.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-23T10:09:35.745 INFO:teuthology.orchestra.run.smithi183.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-23T10:09:35.745 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-23T10:09:35.745 INFO:teuthology.orchestra.run.smithi183.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-23T10:09:35.745 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-13.14-1.el9.x86_64 2024-04-23T10:09:35.745 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-23T10:09:35.745 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-23T10:09:35.745 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-23T10:09:35.745 INFO:teuthology.orchestra.run.smithi183.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-23T10:09:35.746 INFO:teuthology.orchestra.run.smithi183.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-23T10:09:35.746 INFO:teuthology.orchestra.run.smithi183.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-23T10:09:35.746 INFO:teuthology.orchestra.run.smithi183.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-23T10:09:35.746 INFO:teuthology.orchestra.run.smithi183.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-23T10:09:35.746 INFO:teuthology.orchestra.run.smithi183.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-23T10:09:35.746 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-23T10:09:35.746 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:35.746 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:35.746 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:35.746 INFO:teuthology.orchestra.run.smithi183.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-23T10:09:35.746 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-23T10:09:35.746 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-23T10:09:35.746 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-23T10:09:35.747 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-23T10:09:35.747 INFO:teuthology.orchestra.run.smithi183.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-23T10:09:35.747 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-23T10:09:35.747 INFO:teuthology.orchestra.run.smithi183.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-23T10:09:35.747 INFO:teuthology.orchestra.run.smithi183.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-23T10:09:35.747 INFO:teuthology.orchestra.run.smithi183.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-23T10:09:35.747 INFO:teuthology.orchestra.run.smithi183.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-23T10:09:35.747 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-23T10:09:35.747 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-23T10:09:35.747 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-23T10:09:35.747 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-23T10:09:35.747 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-23T10:09:35.747 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-23T10:09:35.747 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-23T10:09:35.748 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-23T10:09:35.748 INFO:teuthology.orchestra.run.smithi183.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-23T10:09:35.748 INFO:teuthology.orchestra.run.smithi183.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-23T10:09:35.748 INFO:teuthology.orchestra.run.smithi183.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-23T10:09:35.748 INFO:teuthology.orchestra.run.smithi183.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-23T10:09:35.748 INFO:teuthology.orchestra.run.smithi183.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-23T10:09:35.748 INFO:teuthology.orchestra.run.smithi183.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-23T10:09:35.748 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-23T10:09:35.748 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-23T10:09:35.748 INFO:teuthology.orchestra.run.smithi183.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-23T10:09:35.748 INFO:teuthology.orchestra.run.smithi183.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-23T10:09:35.748 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-23T10:09:35.748 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ply-3.11-14.el9.noarch 2024-04-23T10:09:35.748 INFO:teuthology.orchestra.run.smithi183.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-23T10:09:35.749 INFO:teuthology.orchestra.run.smithi183.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-23T10:09:35.749 INFO:teuthology.orchestra.run.smithi183.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-23T10:09:35.749 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-23T10:09:35.749 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-23T10:09:35.749 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-23T10:09:35.749 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-23T10:09:35.749 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-23T10:09:35.749 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-23T10:09:35.749 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:35.749 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:35.749 INFO:teuthology.orchestra.run.smithi183.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-23T10:09:35.749 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-23T10:09:35.749 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-23T10:09:35.749 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:35.749 INFO:teuthology.orchestra.run.smithi183.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-23T10:09:35.749 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-23T10:09:35.750 INFO:teuthology.orchestra.run.smithi183.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-23T10:09:35.750 INFO:teuthology.orchestra.run.smithi183.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-23T10:09:35.750 INFO:teuthology.orchestra.run.smithi183.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-23T10:09:35.750 INFO:teuthology.orchestra.run.smithi183.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-23T10:09:35.750 INFO:teuthology.orchestra.run.smithi183.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-23T10:09:35.750 INFO:teuthology.orchestra.run.smithi183.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-23T10:09:35.750 INFO:teuthology.orchestra.run.smithi183.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-23T10:09:35.750 INFO:teuthology.orchestra.run.smithi183.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-23T10:09:35.750 INFO:teuthology.orchestra.run.smithi183.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-23T10:09:35.750 INFO:teuthology.orchestra.run.smithi183.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-23T10:09:35.750 INFO:teuthology.orchestra.run.smithi183.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-23T10:09:35.750 INFO:teuthology.orchestra.run.smithi183.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-23T10:09:35.750 INFO:teuthology.orchestra.run.smithi183.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-23T10:09:35.751 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:35.751 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:35.751 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:35.751 INFO:teuthology.orchestra.run.smithi183.stdout: re2-1:20211101-3.el9.x86_64 2024-04-23T10:09:35.751 INFO:teuthology.orchestra.run.smithi183.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-23T10:09:35.751 INFO:teuthology.orchestra.run.smithi183.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-23T10:09:35.751 INFO:teuthology.orchestra.run.smithi183.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-23T10:09:35.751 INFO:teuthology.orchestra.run.smithi183.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-23T10:09:35.751 INFO:teuthology.orchestra.run.smithi183.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-23T10:09:35.751 INFO:teuthology.orchestra.run.smithi183.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-23T10:09:35.751 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-04-23T10:09:35.751 INFO:teuthology.orchestra.run.smithi183.stdout:Complete! 2024-04-23T10:09:36.157 DEBUG:teuthology.parallel:result is None 2024-04-23T10:09:38.936 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-23T10:09:38.936 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-23T10:09:38.936 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-23T10:09:38.936 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-23T10:09:38.936 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-23T10:09:38.936 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-23T10:09:38.936 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-23T10:09:38.936 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-23T10:09:38.936 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-23T10:09:38.937 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-23T10:09:38.937 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-23T10:09:38.937 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-23T10:09:38.937 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-23T10:09:38.937 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-23T10:09:38.937 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-23T10:09:38.937 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-23T10:09:38.937 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-23T10:09:38.937 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-23T10:09:38.937 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-23T10:09:38.937 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-23T10:09:38.937 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-23T10:09:38.937 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-23T10:09:38.937 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-23T10:09:38.938 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-23T10:09:38.938 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-23T10:09:38.938 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-23T10:09:38.938 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-23T10:09:38.938 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-23T10:09:38.938 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-23T10:09:38.938 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-23T10:09:38.938 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-23T10:09:38.938 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-23T10:09:38.938 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-23T10:09:38.938 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-23T10:09:38.938 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-23T10:09:38.938 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-23T10:09:38.939 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-23T10:09:38.939 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-23T10:09:38.939 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-23T10:09:38.939 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-23T10:09:38.939 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-23T10:09:38.939 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-23T10:09:38.939 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-23T10:09:38.939 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-23T10:09:38.939 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-23T10:09:38.939 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-23T10:09:38.939 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-23T10:09:38.939 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-23T10:09:38.939 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-23T10:09:38.939 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-23T10:09:38.939 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-23T10:09:38.939 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-23T10:09:38.940 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-23T10:09:38.940 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-23T10:09:38.940 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-23T10:09:38.940 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-23T10:09:38.940 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-23T10:09:38.940 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-23T10:09:38.940 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-23T10:09:38.940 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-23T10:09:38.940 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-23T10:09:38.940 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-23T10:09:38.940 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-23T10:09:38.940 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-23T10:09:38.940 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-23T10:09:38.940 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-23T10:09:38.941 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-23T10:09:38.941 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-23T10:09:38.941 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-23T10:09:38.941 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-23T10:09:38.941 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-23T10:09:38.941 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-23T10:09:38.941 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-23T10:09:38.941 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-23T10:09:38.941 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-23T10:09:38.941 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-23T10:09:38.941 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-23T10:09:38.941 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-23T10:09:38.941 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-23T10:09:38.941 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-23T10:09:38.941 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-23T10:09:38.942 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-23T10:09:38.942 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-23T10:09:38.942 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-23T10:09:38.942 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-23T10:09:38.942 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-23T10:09:38.942 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-23T10:09:38.942 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-23T10:09:38.942 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-23T10:09:38.942 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-23T10:09:38.942 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-23T10:09:38.942 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-23T10:09:38.942 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-23T10:09:38.942 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-23T10:09:38.942 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-23T10:09:38.943 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-23T10:09:38.943 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-23T10:09:38.943 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-23T10:09:38.943 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-23T10:09:38.943 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-23T10:09:38.943 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-23T10:09:38.943 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-23T10:09:38.943 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-23T10:09:38.943 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-23T10:09:38.943 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-23T10:09:38.943 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-23T10:09:38.943 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-23T10:09:38.943 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-23T10:09:38.943 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-23T10:09:38.944 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-23T10:09:38.944 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-23T10:09:38.944 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-23T10:09:38.944 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-23T10:09:38.944 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-23T10:09:38.944 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-23T10:09:38.944 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-23T10:09:38.944 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-23T10:09:38.944 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-23T10:09:38.944 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-23T10:09:38.944 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-23T10:09:38.944 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-23T10:09:38.944 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-23T10:09:38.944 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-23T10:09:38.944 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-23T10:09:38.944 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-23T10:09:38.945 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-23T10:09:38.945 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-23T10:09:38.945 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-23T10:09:38.945 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-23T10:09:38.945 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-23T10:09:38.945 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-23T10:09:38.945 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-23T10:09:38.945 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-23T10:09:38.945 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-23T10:09:38.945 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-23T10:09:38.945 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-23T10:09:38.945 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-23T10:09:38.945 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-23T10:09:38.945 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-23T10:09:38.945 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-23T10:09:38.946 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-23T10:09:38.946 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-23T10:09:38.946 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-23T10:09:38.946 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-23T10:09:38.946 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-23T10:09:38.946 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-23T10:09:38.946 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-23T10:09:38.946 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-23T10:09:38.946 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-23T10:09:38.946 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-23T10:09:38.946 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-23T10:09:38.946 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-23T10:09:38.946 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-23T10:09:38.946 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-23T10:09:38.946 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-23T10:09:38.947 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-23T10:09:39.585 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-23T10:09:39.586 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-23T10:09:39.586 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-23T10:09:39.586 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-23T10:09:39.586 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-23T10:09:39.586 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-23T10:09:39.586 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-23T10:09:39.586 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-23T10:09:39.586 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-23T10:09:39.586 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-23T10:09:39.586 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-23T10:09:39.586 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-23T10:09:39.586 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-23T10:09:39.587 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-23T10:09:39.587 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-23T10:09:39.587 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-23T10:09:39.587 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-23T10:09:39.587 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-23T10:09:39.587 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-23T10:09:39.587 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-23T10:09:39.587 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-23T10:09:39.587 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-23T10:09:39.587 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-23T10:09:39.587 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-23T10:09:39.587 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-23T10:09:39.587 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-23T10:09:39.587 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-23T10:09:39.588 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-23T10:09:39.588 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-23T10:09:39.588 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-23T10:09:39.600 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-23T10:09:39.600 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-23T10:09:39.600 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-23T10:09:39.600 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-23T10:09:39.600 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-23T10:09:39.600 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-23T10:09:39.600 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-23T10:09:39.600 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-23T10:09:39.600 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-23T10:09:39.600 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-23T10:09:39.600 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-23T10:09:39.600 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-23T10:09:39.600 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-23T10:09:39.601 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-23T10:09:39.601 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-23T10:09:39.601 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-23T10:09:39.601 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-23T10:09:39.601 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-23T10:09:39.601 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-23T10:09:39.601 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-23T10:09:39.601 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-23T10:09:39.601 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-23T10:09:39.601 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-23T10:09:39.601 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-23T10:09:39.601 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-23T10:09:39.601 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-23T10:09:39.602 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-23T10:09:39.602 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-23T10:09:39.602 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-23T10:09:39.602 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-23T10:09:39.602 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-23T10:09:39.602 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-23T10:09:39.602 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-23T10:09:39.602 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-23T10:09:39.602 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-23T10:09:39.602 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-23T10:09:39.602 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-23T10:09:39.602 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-23T10:09:39.602 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-23T10:09:39.603 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-23T10:09:39.603 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-23T10:09:39.603 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-23T10:09:39.603 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-23T10:09:39.603 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-23T10:09:39.603 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-23T10:09:39.603 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-23T10:09:39.603 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-23T10:09:39.603 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-23T10:09:39.603 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-23T10:09:39.603 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-23T10:09:39.603 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-23T10:09:39.603 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-23T10:09:39.603 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-23T10:09:39.604 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-23T10:09:39.604 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-23T10:09:39.604 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-23T10:09:39.604 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-23T10:09:39.604 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-23T10:09:39.604 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-23T10:09:39.604 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-23T10:09:39.604 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-23T10:09:39.604 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-23T10:09:39.604 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-23T10:09:39.604 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-23T10:09:39.604 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-23T10:09:39.604 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-23T10:09:39.604 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-23T10:09:39.605 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-23T10:09:39.605 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-23T10:09:39.605 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-23T10:09:39.605 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-23T10:09:39.605 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-23T10:09:39.605 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-23T10:09:39.605 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-23T10:09:39.605 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-23T10:09:39.605 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-23T10:09:39.605 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-23T10:09:39.605 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-23T10:09:39.605 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-23T10:09:39.605 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-23T10:09:39.605 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-23T10:09:39.605 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-23T10:09:39.606 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-23T10:09:39.606 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-23T10:09:39.606 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-23T10:09:39.606 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-23T10:09:39.606 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-23T10:09:39.606 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-23T10:09:39.606 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-23T10:09:39.606 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-23T10:09:39.606 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-23T10:09:39.606 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-23T10:09:39.606 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-23T10:09:39.606 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-23T10:09:39.606 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-23T10:09:39.607 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-23T10:09:39.607 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-23T10:09:39.607 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-23T10:09:39.607 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-23T10:09:39.607 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-23T10:09:39.607 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-23T10:09:39.607 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-23T10:09:39.607 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-23T10:09:39.607 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-23T10:09:39.607 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-23T10:09:39.607 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-23T10:09:39.607 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-23T10:09:39.607 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-23T10:09:39.608 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-23T10:09:39.608 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-23T10:09:39.608 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-23T10:09:39.608 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-23T10:09:39.608 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-23T10:09:39.608 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-23T10:09:39.608 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-23T10:09:39.608 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-23T10:09:39.608 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-23T10:09:39.608 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-23T10:09:39.608 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-23T10:09:39.608 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-23T10:09:39.608 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-23T10:09:39.608 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-23T10:09:39.609 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-23T10:09:39.609 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-23T10:09:39.609 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-23T10:09:39.609 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-23T10:09:41.114 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-23T10:09:41.114 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-23T10:09:41.114 INFO:teuthology.orchestra.run.smithi196.stdout:Upgraded: 2024-04-23T10:09:41.114 INFO:teuthology.orchestra.run.smithi196.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:41.114 INFO:teuthology.orchestra.run.smithi196.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:41.114 INFO:teuthology.orchestra.run.smithi196.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-23T10:09:41.114 INFO:teuthology.orchestra.run.smithi196.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-23T10:09:41.114 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-04-23T10:09:41.114 INFO:teuthology.orchestra.run.smithi196.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-23T10:09:41.115 INFO:teuthology.orchestra.run.smithi196.stdout: automake-1.16.2-8.el9.noarch 2024-04-23T10:09:41.115 INFO:teuthology.orchestra.run.smithi196.stdout: bison-3.7.4-5.el9.x86_64 2024-04-23T10:09:41.115 INFO:teuthology.orchestra.run.smithi196.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-23T10:09:41.115 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:41.115 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:41.115 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:41.115 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:41.115 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T10:09:41.115 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:41.115 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:41.115 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:41.115 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T10:09:41.115 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T10:09:41.115 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T10:09:41.116 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T10:09:41.116 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T10:09:41.116 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:41.116 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:41.116 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T10:09:41.116 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:41.116 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:41.116 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:41.116 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T10:09:41.116 INFO:teuthology.orchestra.run.smithi196.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T10:09:41.116 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:41.116 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:41.116 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T10:09:41.116 INFO:teuthology.orchestra.run.smithi196.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-23T10:09:41.117 INFO:teuthology.orchestra.run.smithi196.stdout: flex-2.6.4-9.el9.x86_64 2024-04-23T10:09:41.117 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-23T10:09:41.117 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-23T10:09:41.117 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-23T10:09:41.117 INFO:teuthology.orchestra.run.smithi196.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-23T10:09:41.117 INFO:teuthology.orchestra.run.smithi196.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-23T10:09:41.117 INFO:teuthology.orchestra.run.smithi196.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-23T10:09:41.117 INFO:teuthology.orchestra.run.smithi196.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-23T10:09:41.117 INFO:teuthology.orchestra.run.smithi196.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-23T10:09:41.117 INFO:teuthology.orchestra.run.smithi196.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-23T10:09:41.117 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:41.117 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:41.117 INFO:teuthology.orchestra.run.smithi196.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:41.117 INFO:teuthology.orchestra.run.smithi196.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-23T10:09:41.117 INFO:teuthology.orchestra.run.smithi196.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-23T10:09:41.117 INFO:teuthology.orchestra.run.smithi196.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-23T10:09:41.118 INFO:teuthology.orchestra.run.smithi196.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-23T10:09:41.118 INFO:teuthology.orchestra.run.smithi196.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-23T10:09:41.118 INFO:teuthology.orchestra.run.smithi196.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:41.118 INFO:teuthology.orchestra.run.smithi196.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:41.118 INFO:teuthology.orchestra.run.smithi196.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-23T10:09:41.118 INFO:teuthology.orchestra.run.smithi196.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:41.118 INFO:teuthology.orchestra.run.smithi196.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-23T10:09:41.118 INFO:teuthology.orchestra.run.smithi196.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-23T10:09:41.118 INFO:teuthology.orchestra.run.smithi196.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-23T10:09:41.118 INFO:teuthology.orchestra.run.smithi196.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-23T10:09:41.118 INFO:teuthology.orchestra.run.smithi196.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-23T10:09:41.118 INFO:teuthology.orchestra.run.smithi196.stdout: lua-5.4.4-4.el9.x86_64 2024-04-23T10:09:41.118 INFO:teuthology.orchestra.run.smithi196.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-23T10:09:41.118 INFO:teuthology.orchestra.run.smithi196.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-23T10:09:41.119 INFO:teuthology.orchestra.run.smithi196.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-23T10:09:41.119 INFO:teuthology.orchestra.run.smithi196.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-23T10:09:41.119 INFO:teuthology.orchestra.run.smithi196.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-23T10:09:41.119 INFO:teuthology.orchestra.run.smithi196.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-23T10:09:41.119 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-13.14-1.el9.x86_64 2024-04-23T10:09:41.119 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-23T10:09:41.119 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-23T10:09:41.119 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-23T10:09:41.119 INFO:teuthology.orchestra.run.smithi196.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-23T10:09:41.119 INFO:teuthology.orchestra.run.smithi196.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-23T10:09:41.119 INFO:teuthology.orchestra.run.smithi196.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-23T10:09:41.119 INFO:teuthology.orchestra.run.smithi196.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-23T10:09:41.119 INFO:teuthology.orchestra.run.smithi196.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-23T10:09:41.119 INFO:teuthology.orchestra.run.smithi196.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-23T10:09:41.120 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-23T10:09:41.120 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:41.120 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:41.120 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:41.120 INFO:teuthology.orchestra.run.smithi196.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-23T10:09:41.120 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-23T10:09:41.120 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-23T10:09:41.120 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-23T10:09:41.120 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-23T10:09:41.120 INFO:teuthology.orchestra.run.smithi196.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-23T10:09:41.120 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-23T10:09:41.120 INFO:teuthology.orchestra.run.smithi196.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-23T10:09:41.120 INFO:teuthology.orchestra.run.smithi196.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-23T10:09:41.120 INFO:teuthology.orchestra.run.smithi196.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-23T10:09:41.120 INFO:teuthology.orchestra.run.smithi196.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-23T10:09:41.121 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-23T10:09:41.121 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-23T10:09:41.121 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-23T10:09:41.121 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-23T10:09:41.121 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-23T10:09:41.121 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-23T10:09:41.121 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-23T10:09:41.121 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-23T10:09:41.121 INFO:teuthology.orchestra.run.smithi196.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-23T10:09:41.121 INFO:teuthology.orchestra.run.smithi196.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-23T10:09:41.121 INFO:teuthology.orchestra.run.smithi196.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-23T10:09:41.121 INFO:teuthology.orchestra.run.smithi196.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-23T10:09:41.121 INFO:teuthology.orchestra.run.smithi196.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-23T10:09:41.121 INFO:teuthology.orchestra.run.smithi196.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-23T10:09:41.122 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-23T10:09:41.122 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-23T10:09:41.122 INFO:teuthology.orchestra.run.smithi196.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-23T10:09:41.122 INFO:teuthology.orchestra.run.smithi196.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-23T10:09:41.122 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-23T10:09:41.122 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ply-3.11-14.el9.noarch 2024-04-23T10:09:41.122 INFO:teuthology.orchestra.run.smithi196.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-23T10:09:41.122 INFO:teuthology.orchestra.run.smithi196.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-23T10:09:41.122 INFO:teuthology.orchestra.run.smithi196.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-23T10:09:41.122 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-23T10:09:41.122 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-23T10:09:41.122 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-23T10:09:41.122 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-23T10:09:41.122 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-23T10:09:41.122 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-23T10:09:41.123 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:41.123 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:41.123 INFO:teuthology.orchestra.run.smithi196.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-23T10:09:41.123 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-23T10:09:41.123 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-23T10:09:41.123 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:41.123 INFO:teuthology.orchestra.run.smithi196.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-23T10:09:41.123 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-23T10:09:41.123 INFO:teuthology.orchestra.run.smithi196.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-23T10:09:41.123 INFO:teuthology.orchestra.run.smithi196.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-23T10:09:41.123 INFO:teuthology.orchestra.run.smithi196.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-23T10:09:41.123 INFO:teuthology.orchestra.run.smithi196.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-23T10:09:41.123 INFO:teuthology.orchestra.run.smithi196.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-23T10:09:41.123 INFO:teuthology.orchestra.run.smithi196.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-23T10:09:41.123 INFO:teuthology.orchestra.run.smithi196.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-23T10:09:41.123 INFO:teuthology.orchestra.run.smithi196.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-23T10:09:41.124 INFO:teuthology.orchestra.run.smithi196.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-23T10:09:41.124 INFO:teuthology.orchestra.run.smithi196.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-23T10:09:41.124 INFO:teuthology.orchestra.run.smithi196.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-23T10:09:41.124 INFO:teuthology.orchestra.run.smithi196.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-23T10:09:41.124 INFO:teuthology.orchestra.run.smithi196.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-23T10:09:41.124 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:41.124 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:41.124 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:41.124 INFO:teuthology.orchestra.run.smithi196.stdout: re2-1:20211101-3.el9.x86_64 2024-04-23T10:09:41.124 INFO:teuthology.orchestra.run.smithi196.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-23T10:09:41.124 INFO:teuthology.orchestra.run.smithi196.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-23T10:09:41.124 INFO:teuthology.orchestra.run.smithi196.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-23T10:09:41.124 INFO:teuthology.orchestra.run.smithi196.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-23T10:09:41.124 INFO:teuthology.orchestra.run.smithi196.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-23T10:09:41.125 INFO:teuthology.orchestra.run.smithi196.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-23T10:09:41.125 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-23T10:09:41.125 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-04-23T10:09:41.508 DEBUG:teuthology.parallel:result is None 2024-04-23T10:09:41.699 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-23T10:09:41.699 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:09:41.699 INFO:teuthology.orchestra.run.smithi097.stdout:Upgraded: 2024-04-23T10:09:41.699 INFO:teuthology.orchestra.run.smithi097.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:41.699 INFO:teuthology.orchestra.run.smithi097.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:41.699 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-23T10:09:41.699 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-23T10:09:41.699 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2024-04-23T10:09:41.699 INFO:teuthology.orchestra.run.smithi097.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-23T10:09:41.699 INFO:teuthology.orchestra.run.smithi097.stdout: automake-1.16.2-8.el9.noarch 2024-04-23T10:09:41.699 INFO:teuthology.orchestra.run.smithi097.stdout: bison-3.7.4-5.el9.x86_64 2024-04-23T10:09:41.699 INFO:teuthology.orchestra.run.smithi097.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-23T10:09:41.699 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:41.699 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:41.700 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:41.700 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:41.700 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T10:09:41.700 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:41.700 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:41.700 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:41.700 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T10:09:41.700 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T10:09:41.700 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T10:09:41.700 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T10:09:41.700 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T10:09:41.700 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:41.700 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:41.700 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T10:09:41.701 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:41.701 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:41.701 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:41.701 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T10:09:41.701 INFO:teuthology.orchestra.run.smithi097.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T10:09:41.701 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:41.701 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:41.701 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T10:09:41.701 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-23T10:09:41.701 INFO:teuthology.orchestra.run.smithi097.stdout: flex-2.6.4-9.el9.x86_64 2024-04-23T10:09:41.701 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-23T10:09:41.701 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-23T10:09:41.701 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-23T10:09:41.701 INFO:teuthology.orchestra.run.smithi097.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-23T10:09:41.701 INFO:teuthology.orchestra.run.smithi097.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-23T10:09:41.701 INFO:teuthology.orchestra.run.smithi097.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-23T10:09:41.702 INFO:teuthology.orchestra.run.smithi097.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-23T10:09:41.702 INFO:teuthology.orchestra.run.smithi097.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-23T10:09:41.702 INFO:teuthology.orchestra.run.smithi097.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-23T10:09:41.702 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:41.702 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:41.702 INFO:teuthology.orchestra.run.smithi097.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:41.702 INFO:teuthology.orchestra.run.smithi097.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-23T10:09:41.702 INFO:teuthology.orchestra.run.smithi097.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-23T10:09:41.702 INFO:teuthology.orchestra.run.smithi097.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-23T10:09:41.702 INFO:teuthology.orchestra.run.smithi097.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-23T10:09:41.702 INFO:teuthology.orchestra.run.smithi097.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-23T10:09:41.702 INFO:teuthology.orchestra.run.smithi097.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:41.702 INFO:teuthology.orchestra.run.smithi097.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:41.703 INFO:teuthology.orchestra.run.smithi097.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-23T10:09:41.703 INFO:teuthology.orchestra.run.smithi097.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:41.703 INFO:teuthology.orchestra.run.smithi097.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-23T10:09:41.703 INFO:teuthology.orchestra.run.smithi097.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-23T10:09:41.703 INFO:teuthology.orchestra.run.smithi097.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-23T10:09:41.703 INFO:teuthology.orchestra.run.smithi097.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-23T10:09:41.703 INFO:teuthology.orchestra.run.smithi097.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-23T10:09:41.703 INFO:teuthology.orchestra.run.smithi097.stdout: lua-5.4.4-4.el9.x86_64 2024-04-23T10:09:41.703 INFO:teuthology.orchestra.run.smithi097.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-23T10:09:41.703 INFO:teuthology.orchestra.run.smithi097.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-23T10:09:41.703 INFO:teuthology.orchestra.run.smithi097.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-23T10:09:41.703 INFO:teuthology.orchestra.run.smithi097.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-23T10:09:41.703 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-23T10:09:41.703 INFO:teuthology.orchestra.run.smithi097.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-23T10:09:41.703 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-13.14-1.el9.x86_64 2024-04-23T10:09:41.703 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-23T10:09:41.704 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-23T10:09:41.704 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-23T10:09:41.704 INFO:teuthology.orchestra.run.smithi097.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-23T10:09:41.704 INFO:teuthology.orchestra.run.smithi097.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-23T10:09:41.704 INFO:teuthology.orchestra.run.smithi097.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-23T10:09:41.704 INFO:teuthology.orchestra.run.smithi097.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-23T10:09:41.704 INFO:teuthology.orchestra.run.smithi097.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-23T10:09:41.704 INFO:teuthology.orchestra.run.smithi097.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-23T10:09:41.704 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-23T10:09:41.704 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:41.704 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:41.704 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:41.704 INFO:teuthology.orchestra.run.smithi097.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-23T10:09:41.704 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-23T10:09:41.704 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-23T10:09:41.704 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-23T10:09:41.705 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-23T10:09:41.705 INFO:teuthology.orchestra.run.smithi097.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-23T10:09:41.705 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-23T10:09:41.705 INFO:teuthology.orchestra.run.smithi097.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-23T10:09:41.705 INFO:teuthology.orchestra.run.smithi097.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-23T10:09:41.705 INFO:teuthology.orchestra.run.smithi097.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-23T10:09:41.705 INFO:teuthology.orchestra.run.smithi097.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-23T10:09:41.705 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-23T10:09:41.705 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-23T10:09:41.705 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-23T10:09:41.705 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-23T10:09:41.705 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-23T10:09:41.705 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-23T10:09:41.705 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-23T10:09:41.705 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-23T10:09:41.705 INFO:teuthology.orchestra.run.smithi097.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-23T10:09:41.706 INFO:teuthology.orchestra.run.smithi097.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-23T10:09:41.706 INFO:teuthology.orchestra.run.smithi097.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-23T10:09:41.706 INFO:teuthology.orchestra.run.smithi097.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-23T10:09:41.706 INFO:teuthology.orchestra.run.smithi097.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-23T10:09:41.706 INFO:teuthology.orchestra.run.smithi097.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-23T10:09:41.706 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-23T10:09:41.706 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-23T10:09:41.706 INFO:teuthology.orchestra.run.smithi097.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-23T10:09:41.706 INFO:teuthology.orchestra.run.smithi097.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-23T10:09:41.706 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-23T10:09:41.706 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ply-3.11-14.el9.noarch 2024-04-23T10:09:41.706 INFO:teuthology.orchestra.run.smithi097.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-23T10:09:41.706 INFO:teuthology.orchestra.run.smithi097.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-23T10:09:41.706 INFO:teuthology.orchestra.run.smithi097.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-23T10:09:41.706 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-23T10:09:41.706 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-23T10:09:41.707 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-23T10:09:41.707 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-23T10:09:41.707 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-23T10:09:41.707 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-23T10:09:41.707 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:41.707 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:41.707 INFO:teuthology.orchestra.run.smithi097.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-23T10:09:41.707 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-23T10:09:41.707 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-23T10:09:41.707 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:41.707 INFO:teuthology.orchestra.run.smithi097.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-23T10:09:41.707 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-23T10:09:41.707 INFO:teuthology.orchestra.run.smithi097.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-23T10:09:41.707 INFO:teuthology.orchestra.run.smithi097.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-23T10:09:41.707 INFO:teuthology.orchestra.run.smithi097.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-23T10:09:41.707 INFO:teuthology.orchestra.run.smithi097.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-23T10:09:41.707 INFO:teuthology.orchestra.run.smithi097.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-23T10:09:41.708 INFO:teuthology.orchestra.run.smithi097.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-23T10:09:41.708 INFO:teuthology.orchestra.run.smithi097.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-23T10:09:41.708 INFO:teuthology.orchestra.run.smithi097.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-23T10:09:41.708 INFO:teuthology.orchestra.run.smithi097.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-23T10:09:41.708 INFO:teuthology.orchestra.run.smithi097.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-23T10:09:41.708 INFO:teuthology.orchestra.run.smithi097.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-23T10:09:41.708 INFO:teuthology.orchestra.run.smithi097.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-23T10:09:41.708 INFO:teuthology.orchestra.run.smithi097.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-23T10:09:41.708 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:41.708 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:41.708 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T10:09:41.708 INFO:teuthology.orchestra.run.smithi097.stdout: re2-1:20211101-3.el9.x86_64 2024-04-23T10:09:41.708 INFO:teuthology.orchestra.run.smithi097.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-23T10:09:41.708 INFO:teuthology.orchestra.run.smithi097.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-23T10:09:41.708 INFO:teuthology.orchestra.run.smithi097.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-23T10:09:41.709 INFO:teuthology.orchestra.run.smithi097.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-23T10:09:41.709 INFO:teuthology.orchestra.run.smithi097.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-23T10:09:41.709 INFO:teuthology.orchestra.run.smithi097.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-23T10:09:41.709 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:09:41.709 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-04-23T10:09:42.114 DEBUG:teuthology.parallel:result is None 2024-04-23T10:09:42.115 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-23T10:09:42.332 DEBUG:teuthology.orchestra.run.smithi097:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-23T10:09:42.357 INFO:teuthology.orchestra.run.smithi097.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-23T10:09:42.358 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-23T10:09:42.358 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-23T10:09:42.359 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-23T10:09:42.575 DEBUG:teuthology.orchestra.run.smithi183:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-23T10:09:42.601 INFO:teuthology.orchestra.run.smithi183.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-23T10:09:42.602 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-23T10:09:42.602 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-23T10:09:42.604 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-23T10:09:42.820 DEBUG:teuthology.orchestra.run.smithi196:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-23T10:09:42.842 INFO:teuthology.orchestra.run.smithi196.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-23T10:09:42.842 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-23T10:09:42.842 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-23T10:09:42.844 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-23T10:09:42.844 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-23T10:09:42.844 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-23T10:09:42.870 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-23T10:09:42.870 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-23T10:09:42.901 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-23T10:09:42.901 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-23T10:09:42.926 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-23T10:09:42.926 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-23T10:09:42.926 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/daemon-helper 2024-04-23T10:09:42.950 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-23T10:09:43.017 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-23T10:09:43.017 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/daemon-helper 2024-04-23T10:09:43.045 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-23T10:09:43.113 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-23T10:09:43.113 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/usr/bin/daemon-helper 2024-04-23T10:09:43.138 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-23T10:09:43.201 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-23T10:09:43.201 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-23T10:09:43.201 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-23T10:09:43.224 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-23T10:09:43.288 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-23T10:09:43.288 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-23T10:09:43.315 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-23T10:09:43.381 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-23T10:09:43.381 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-23T10:09:43.404 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-23T10:09:43.468 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-23T10:09:43.468 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-23T10:09:43.468 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/stdin-killer 2024-04-23T10:09:43.492 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-23T10:09:43.554 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-23T10:09:43.555 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/stdin-killer 2024-04-23T10:09:43.581 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-23T10:09:43.647 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-23T10:09:43.647 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/usr/bin/stdin-killer 2024-04-23T10:09:43.670 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-23T10:09:43.734 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-23T10:09:43.822 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-04-23T10:09:43.822 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-23T10:09:43.823 INFO:tasks.cephadm:Cluster fsid is 9b22af88-0159-11ef-bc93-c7b262605968 2024-04-23T10:09:43.823 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-23T10:09:43.823 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.97', 'mon.b': '172.21.15.183', 'mon.c': '172.21.15.196'} 2024-04-23T10:09:43.824 INFO:tasks.cephadm:First mon is mon.a on smithi097 2024-04-23T10:09:43.824 INFO:tasks.cephadm:First mgr is x 2024-04-23T10:09:43.824 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-23T10:09:43.824 DEBUG:teuthology.orchestra.run.smithi097:> sudo hostname $(hostname -s) 2024-04-23T10:09:43.848 DEBUG:teuthology.orchestra.run.smithi183:> sudo hostname $(hostname -s) 2024-04-23T10:09:43.875 DEBUG:teuthology.orchestra.run.smithi196:> sudo hostname $(hostname -s) 2024-04-23T10:09:43.899 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-23T10:09:43.899 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-23T10:09:44.039 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78795/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3156-gd69f13d8', 'node_name': '172.21.5.39+adami09', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3156.gd69f13d8'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-22 12:00:06.590653', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-lusov-quiescer', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-23T10:09:44.247 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref wip-lusov-quiescer, sha1 d69f13d827952bb07bb9b7973bca14f97d764d6f from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-23T10:09:44.249 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm 2024-04-23T10:09:44.249 DEBUG:teuthology.orchestra.run.smithi097:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-23T10:09:44.731 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 23 10:09 /home/ubuntu/cephtest/cephadm 2024-04-23T10:09:44.731 DEBUG:teuthology.orchestra.run.smithi183:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-23T10:09:45.365 INFO:teuthology.orchestra.run.smithi183.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 23 10:09 /home/ubuntu/cephtest/cephadm 2024-04-23T10:09:45.365 DEBUG:teuthology.orchestra.run.smithi196:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-23T10:09:45.834 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 23 10:09 /home/ubuntu/cephtest/cephadm 2024-04-23T10:09:45.835 DEBUG:teuthology.orchestra.run.smithi097:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-23T10:09:45.854 DEBUG:teuthology.orchestra.run.smithi183:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-23T10:09:45.873 DEBUG:teuthology.orchestra.run.smithi196:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-23T10:09:45.906 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f on all hosts... 2024-04-23T10:09:45.906 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-23T10:09:45.909 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-23T10:09:45.915 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-23T10:09:47.569 INFO:teuthology.orchestra.run.smithi196.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-23T10:09:47.604 INFO:teuthology.orchestra.run.smithi183.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-23T10:09:47.621 INFO:teuthology.orchestra.run.smithi097.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-23T10:12:05.659 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-04-23T10:12:05.659 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-23T10:12:05.660 INFO:teuthology.orchestra.run.smithi183.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-23T10:12:05.660 INFO:teuthology.orchestra.run.smithi183.stdout: "repo_digests": [ 2024-04-23T10:12:05.660 INFO:teuthology.orchestra.run.smithi183.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-23T10:12:05.660 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-04-23T10:12:05.660 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-04-23T10:12:16.125 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-23T10:12:16.125 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-23T10:12:16.125 INFO:teuthology.orchestra.run.smithi097.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-23T10:12:16.125 INFO:teuthology.orchestra.run.smithi097.stdout: "repo_digests": [ 2024-04-23T10:12:16.125 INFO:teuthology.orchestra.run.smithi097.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-23T10:12:16.125 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-04-23T10:12:16.125 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-23T10:15:56.782 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-23T10:15:56.782 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-23T10:15:56.782 INFO:teuthology.orchestra.run.smithi196.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-23T10:15:56.782 INFO:teuthology.orchestra.run.smithi196.stdout: "repo_digests": [ 2024-04-23T10:15:56.783 INFO:teuthology.orchestra.run.smithi196.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-23T10:15:56.783 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-04-23T10:15:56.783 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-23T10:15:56.805 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /etc/ceph 2024-04-23T10:15:56.838 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /etc/ceph 2024-04-23T10:15:56.870 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /etc/ceph 2024-04-23T10:15:56.897 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 777 /etc/ceph 2024-04-23T10:15:56.925 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 777 /etc/ceph 2024-04-23T10:15:56.951 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 777 /etc/ceph 2024-04-23T10:15:56.974 INFO:tasks.cephadm:Writing seed config... 2024-04-23T10:15:56.975 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-23T10:15:56.975 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-23T10:15:56.975 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-23T10:15:56.975 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-23T10:15:56.975 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-23T10:15:56.975 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-23T10:15:56.976 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-23T10:15:56.976 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-23T10:15:56.976 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-23T10:15:56.976 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-23T10:15:56.976 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-23T10:15:56.976 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-23T10:15:56.976 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-23T10:15:56.976 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-23T10:15:56.976 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-23T10:15:56.976 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-23T10:15:56.976 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-23T10:15:56.976 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-04-23T10:15:56.976 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-04-23T10:15:56.976 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-23T10:15:56.976 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-23T10:15:56.976 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-23T10:15:56.977 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-23T10:15:56.977 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-04-23T10:15:56.977 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-23T10:15:56.977 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-23T10:15:56.977 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-23T10:15:56.977 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-23T10:15:56.977 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-23T10:15:56.977 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-23T10:15:56.977 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-23T10:15:56.977 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-23T10:15:56.977 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-23T10:15:56.977 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-04-23T10:15:56.977 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-23T10:15:56.977 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-04-23T10:15:56.977 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-04-23T10:15:56.977 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-23T10:15:56.978 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-23T10:15:56.978 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-04-23T10:15:56.978 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-23T10:15:56.978 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-23T10:15:56.978 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-23T10:15:56.978 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-23T10:15:56.978 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-23T10:15:56.978 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-23T10:15:56.978 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-23T10:15:56.978 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-04-23T10:15:56.979 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-23T10:15:56.979 DEBUG:teuthology.orchestra.run.smithi097:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-23T10:15:56.996 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 9b22af88-0159-11ef-bc93-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-23T10:15:56.997 DEBUG:teuthology.orchestra.run.smithi097:mon.a> sudo journalctl -f -n 0 -u ceph-9b22af88-0159-11ef-bc93-c7b262605968@mon.a.service 2024-04-23T10:15:57.039 DEBUG:teuthology.orchestra.run.smithi097:mgr.x> sudo journalctl -f -n 0 -u ceph-9b22af88-0159-11ef-bc93-c7b262605968@mgr.x.service 2024-04-23T10:15:57.082 INFO:tasks.cephadm:Bootstrapping... 2024-04-23T10:15:57.082 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f -v bootstrap --fsid 9b22af88-0159-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.97 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-23T10:15:57.275 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-04-23T10:15:57.276 INFO:teuthology.orchestra.run.smithi097.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f', '-v', 'bootstrap', '--fsid', '9b22af88-0159-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.97', '--skip-admin-label'] 2024-04-23T10:15:57.276 INFO:teuthology.orchestra.run.smithi097.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-23T10:15:57.276 INFO:teuthology.orchestra.run.smithi097.stdout:Verifying podman|docker is present... 2024-04-23T10:15:57.302 INFO:teuthology.orchestra.run.smithi097.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-23T10:15:57.302 INFO:teuthology.orchestra.run.smithi097.stdout:Verifying lvm2 is present... 2024-04-23T10:15:57.302 INFO:teuthology.orchestra.run.smithi097.stdout:Verifying time synchronization is in place... 2024-04-23T10:15:57.311 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-23T10:15:57.311 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-23T10:15:57.318 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-23T10:15:57.318 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout inactive 2024-04-23T10:15:57.325 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout enabled 2024-04-23T10:15:57.333 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout active 2024-04-23T10:15:57.333 INFO:teuthology.orchestra.run.smithi097.stdout:Unit chronyd.service is enabled and running 2024-04-23T10:15:57.333 INFO:teuthology.orchestra.run.smithi097.stdout:Repeating the final host check... 2024-04-23T10:15:57.373 INFO:teuthology.orchestra.run.smithi097.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-23T10:15:57.374 INFO:teuthology.orchestra.run.smithi097.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-23T10:15:57.374 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl is present 2024-04-23T10:15:57.374 INFO:teuthology.orchestra.run.smithi097.stdout:lvcreate is present 2024-04-23T10:15:57.380 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-23T10:15:57.380 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-23T10:15:57.387 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-23T10:15:57.387 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout inactive 2024-04-23T10:15:57.394 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout enabled 2024-04-23T10:15:57.400 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout active 2024-04-23T10:15:57.401 INFO:teuthology.orchestra.run.smithi097.stdout:Unit chronyd.service is enabled and running 2024-04-23T10:15:57.401 INFO:teuthology.orchestra.run.smithi097.stdout:Host looks OK 2024-04-23T10:15:57.401 INFO:teuthology.orchestra.run.smithi097.stdout:Cluster fsid: 9b22af88-0159-11ef-bc93-c7b262605968 2024-04-23T10:15:57.401 INFO:teuthology.orchestra.run.smithi097.stdout:Acquiring lock 140307223051424 on /run/cephadm/9b22af88-0159-11ef-bc93-c7b262605968.lock 2024-04-23T10:15:57.401 INFO:teuthology.orchestra.run.smithi097.stdout:Lock 140307223051424 acquired on /run/cephadm/9b22af88-0159-11ef-bc93-c7b262605968.lock 2024-04-23T10:15:57.401 INFO:teuthology.orchestra.run.smithi097.stdout:Verifying IP 172.21.15.97 port 3300 ... 2024-04-23T10:15:57.402 INFO:teuthology.orchestra.run.smithi097.stdout:Verifying IP 172.21.15.97 port 6789 ... 2024-04-23T10:15:57.402 INFO:teuthology.orchestra.run.smithi097.stdout:Base mon IP(s) is [172.21.15.97:3300, 172.21.15.97:6789], mon addrv is [v2:172.21.15.97:3300,v1:172.21.15.97:6789] 2024-04-23T10:15:57.406 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.97 metric 100 2024-04-23T10:15:57.406 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.97 metric 100 2024-04-23T10:15:57.409 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-23T10:15:57.409 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-04-23T10:15:57.409 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-04-23T10:15:57.413 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-23T10:15:57.413 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-23T10:15:57.413 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-23T10:15:57.413 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-04-23T10:15:57.413 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/ip: stdout inet6 fe80::d874:44e6:ef83:6bdd/64 scope link noprefixroute 2024-04-23T10:15:57.413 INFO:teuthology.orchestra.run.smithi097.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-23T10:15:57.413 INFO:teuthology.orchestra.run.smithi097.stdout:Mon IP `172.21.15.97` is in CIDR network `172.21.0.0/20` 2024-04-23T10:15:57.413 INFO:teuthology.orchestra.run.smithi097.stdout:Mon IP `172.21.15.97` is in CIDR network `172.21.0.0/20` 2024-04-23T10:15:57.413 INFO:teuthology.orchestra.run.smithi097.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-23T10:15:57.413 INFO:teuthology.orchestra.run.smithi097.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-23T10:15:57.415 INFO:teuthology.orchestra.run.smithi097.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-23T10:15:58.165 INFO:teuthology.orchestra.run.smithi097.stdout:/bin/podman: stdout da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 2024-04-23T10:15:58.165 INFO:teuthology.orchestra.run.smithi097.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-23T10:15:58.165 INFO:teuthology.orchestra.run.smithi097.stdout:/bin/podman: stderr Getting image source signatures 2024-04-23T10:15:58.165 INFO:teuthology.orchestra.run.smithi097.stdout:/bin/podman: stderr Copying blob sha256:7f03586bcf9c6a2b4d690c118c4aa5b771f9c6f252d75a20fb283dd07093a738 2024-04-23T10:15:58.165 INFO:teuthology.orchestra.run.smithi097.stdout:/bin/podman: stderr Copying blob sha256:426fede9fffa2c7dfe40a4f9322a34794b72405170134d54d2c30d9cc6c6a5be 2024-04-23T10:15:58.165 INFO:teuthology.orchestra.run.smithi097.stdout:/bin/podman: stderr Copying config sha256:da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 2024-04-23T10:15:58.166 INFO:teuthology.orchestra.run.smithi097.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-23T10:15:59.104 INFO:teuthology.orchestra.run.smithi097.stdout:ceph: stdout ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T10:15:59.105 INFO:teuthology.orchestra.run.smithi097.stdout:Ceph version: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T10:15:59.105 INFO:teuthology.orchestra.run.smithi097.stdout:Extracting ceph user uid/gid from container image... 2024-04-23T10:15:59.997 INFO:teuthology.orchestra.run.smithi097.stdout:stat: stdout 167 167 2024-04-23T10:15:59.997 INFO:teuthology.orchestra.run.smithi097.stdout:Creating initial keys... 2024-04-23T10:16:00.920 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph-authtool: stdout AQBgiidmXgZqEhAA5oaus7k7FuZ8oMv5srik+Q== 2024-04-23T10:16:01.841 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph-authtool: stdout AQBhiidmitRKDxAAco3VDYWYupy+4VL49pCSVQ== 2024-04-23T10:16:02.772 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph-authtool: stdout AQBiiidm+paSChAAbJCVj6KwNvZcw+kYKUSaqw== 2024-04-23T10:16:02.773 INFO:teuthology.orchestra.run.smithi097.stdout:Creating initial monmap... 2024-04-23T10:16:03.663 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-23T10:16:03.663 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-23T10:16:03.663 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 9b22af88-0159-11ef-bc93-c7b262605968 2024-04-23T10:16:03.663 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-23T10:16:03.663 INFO:teuthology.orchestra.run.smithi097.stdout:monmaptool for a [v2:172.21.15.97:3300,v1:172.21.15.97:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-23T10:16:03.663 INFO:teuthology.orchestra.run.smithi097.stdout:setting min_mon_release = quincy 2024-04-23T10:16:03.664 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/monmaptool: set fsid to 9b22af88-0159-11ef-bc93-c7b262605968 2024-04-23T10:16:03.664 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-23T10:16:03.664 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:16:03.664 INFO:teuthology.orchestra.run.smithi097.stdout:Creating mon... 2024-04-23T10:16:04.604 INFO:teuthology.orchestra.run.smithi097.stdout:create mon.a on 2024-04-23T10:16:04.809 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-23T10:16:04.948 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-23T10:16:05.122 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-9b22af88-0159-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-9b22af88-0159-11ef-bc93-c7b262605968.target. 2024-04-23T10:16:05.123 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-9b22af88-0159-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-9b22af88-0159-11ef-bc93-c7b262605968.target. 2024-04-23T10:16:05.342 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-9b22af88-0159-11ef-bc93-c7b262605968@mon.a 2024-04-23T10:16:05.342 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stderr Failed to reset failed state of unit ceph-9b22af88-0159-11ef-bc93-c7b262605968@mon.a.service: Unit ceph-9b22af88-0159-11ef-bc93-c7b262605968@mon.a.service not loaded. 2024-04-23T10:16:05.505 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-9b22af88-0159-11ef-bc93-c7b262605968.target.wants/ceph-9b22af88-0159-11ef-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-9b22af88-0159-11ef-bc93-c7b262605968@.service. 2024-04-23T10:16:06.016 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:05 smithi097 ceph-mon[28825]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-23T10:16:06.016 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:06 smithi097 bash[28806]: da307fca1800bf85723bfb492133a120425f3ddbfc0da887e2d64c5c3f6799be 2024-04-23T10:16:06.029 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-23T10:16:06.029 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout disabled 2024-04-23T10:16:06.035 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-23T10:16:06.035 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout inactive 2024-04-23T10:16:06.036 INFO:teuthology.orchestra.run.smithi097.stdout:firewalld.service is not enabled 2024-04-23T10:16:06.036 INFO:teuthology.orchestra.run.smithi097.stdout:Not possible to enable service . firewalld.service is not available 2024-04-23T10:16:06.036 INFO:teuthology.orchestra.run.smithi097.stdout:Waiting for mon to start... 2024-04-23T10:16:06.036 INFO:teuthology.orchestra.run.smithi097.stdout:Waiting for mon... 2024-04-23T10:16:06.266 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:06 smithi097 systemd[1]: Started Ceph mon.a for 9b22af88-0159-11ef-bc93-c7b262605968. 2024-04-23T10:16:07.008 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout cluster: 2024-04-23T10:16:07.008 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout id: 9b22af88-0159-11ef-bc93-c7b262605968 2024-04-23T10:16:07.009 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-23T10:16:07.009 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2024-04-23T10:16:07.009 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout services: 2024-04-23T10:16:07.009 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.562206s) 2024-04-23T10:16:07.009 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-23T10:16:07.009 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-23T10:16:07.009 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2024-04-23T10:16:07.009 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout data: 2024-04-23T10:16:07.009 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-23T10:16:07.009 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-23T10:16:07.009 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-23T10:16:07.009 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout pgs: 2024-04-23T10:16:07.009 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2024-04-23T10:16:07.010 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:06.467+0000 7f857da42700 1 Processor -- start 2024-04-23T10:16:07.010 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:06.467+0000 7f857da42700 1 -- start start 2024-04-23T10:16:07.010 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:06.467+0000 7f857da42700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8578109ab0 0x7f8578109ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:16:07.010 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:06.467+0000 7f857da42700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f857810a4a0 con 0x7f8578109ab0 2024-04-23T10:16:07.010 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:06.468+0000 7f8576ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8578109ab0 0x7f8578109ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:16:07.010 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:06.468+0000 7f8576ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8578109ab0 0x7f8578109ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43184/0 (socket says 172.21.15.97:43184) 2024-04-23T10:16:07.010 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:06.468+0000 7f8576ffd700 1 -- 172.21.15.97:0/1743238924 learned_addr learned my addr 172.21.15.97:0/1743238924 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:16:07.010 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:06.468+0000 7f8576ffd700 1 -- 172.21.15.97:0/1743238924 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f857810a620 con 0x7f8578109ab0 2024-04-23T10:16:07.010 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:06.468+0000 7f8576ffd700 1 --2- 172.21.15.97:0/1743238924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8578109ab0 0x7f8578109ed0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f8570009e80 tx=0x7f857000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57d8e0c95fb97774 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:16:07.010 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:06.468+0000 7f8575ffb700 1 -- 172.21.15.97:0/1743238924 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8570017070 con 0x7f8578109ab0 2024-04-23T10:16:07.010 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:06.468+0000 7f8575ffb700 1 -- 172.21.15.97:0/1743238924 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8570003a20 con 0x7f8578109ab0 2024-04-23T10:16:07.010 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:06.468+0000 7f8575ffb700 1 -- 172.21.15.97:0/1743238924 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8570003d20 con 0x7f8578109ab0 2024-04-23T10:16:07.010 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:06.469+0000 7f857da42700 1 -- 172.21.15.97:0/1743238924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8578109ab0 msgr2=0x7f8578109ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:16:07.011 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:06.469+0000 7f857da42700 1 --2- 172.21.15.97:0/1743238924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8578109ab0 0x7f8578109ed0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f8570009e80 tx=0x7f857000b330 comp rx=0 tx=0).stop 2024-04-23T10:16:07.011 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:06.469+0000 7f857da42700 1 -- 172.21.15.97:0/1743238924 shutdown_connections 2024-04-23T10:16:07.011 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:06.469+0000 7f857da42700 1 --2- 172.21.15.97:0/1743238924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8578109ab0 0x7f8578109ed0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:16:07.011 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:06.469+0000 7f857da42700 1 -- 172.21.15.97:0/1743238924 >> 172.21.15.97:0/1743238924 conn(0x7f8578102110 msgr2=0x7f8578104530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:16:07.011 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:06.469+0000 7f857da42700 1 -- 172.21.15.97:0/1743238924 shutdown_connections 2024-04-23T10:16:07.011 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:06.469+0000 7f857da42700 1 -- 172.21.15.97:0/1743238924 wait complete. 2024-04-23T10:16:07.011 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:06.470+0000 7f857da42700 1 Processor -- start 2024-04-23T10:16:07.011 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:06.470+0000 7f857da42700 1 -- start start 2024-04-23T10:16:07.011 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:06.470+0000 7f857da42700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8578109ab0 0x7f85781090d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:16:07.011 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:06.470+0000 7f857da42700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f85781067d0 con 0x7f8578109ab0 2024-04-23T10:16:07.011 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:06.470+0000 7f8576ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8578109ab0 0x7f85781090d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:16:07.011 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:06.470+0000 7f8576ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8578109ab0 0x7f85781090d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43198/0 (socket says 172.21.15.97:43198) 2024-04-23T10:16:07.012 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:06.470+0000 7f8576ffd700 1 -- 172.21.15.97:0/2788101876 learned_addr learned my addr 172.21.15.97:0/2788101876 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:16:07.012 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:06.470+0000 7f8576ffd700 1 -- 172.21.15.97:0/2788101876 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8578109610 con 0x7f8578109ab0 2024-04-23T10:16:07.012 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:06.470+0000 7f8576ffd700 1 --2- 172.21.15.97:0/2788101876 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8578109ab0 0x7f85781090d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8570000f80 tx=0x7f8570004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:16:07.012 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:06.471+0000 7f857ca40700 1 -- 172.21.15.97:0/2788101876 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8570022020 con 0x7f8578109ab0 2024-04-23T10:16:07.012 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:06.471+0000 7f857ca40700 1 -- 172.21.15.97:0/2788101876 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8570018040 con 0x7f8578109ab0 2024-04-23T10:16:07.012 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:06.471+0000 7f857ca40700 1 -- 172.21.15.97:0/2788101876 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8570017070 con 0x7f8578109ab0 2024-04-23T10:16:07.012 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:06.471+0000 7f857da42700 1 -- 172.21.15.97:0/2788101876 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8578107800 con 0x7f8578109ab0 2024-04-23T10:16:07.012 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:06.471+0000 7f857da42700 1 -- 172.21.15.97:0/2788101876 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8578107d40 con 0x7f8578109ab0 2024-04-23T10:16:07.012 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:06.471+0000 7f857ca40700 1 -- 172.21.15.97:0/2788101876 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8570025020 con 0x7f8578109ab0 2024-04-23T10:16:07.012 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:06.471+0000 7f857ca40700 1 -- 172.21.15.97:0/2788101876 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f857000f050 con 0x7f8578109ab0 2024-04-23T10:16:07.012 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:06.472+0000 7f857da42700 1 -- 172.21.15.97:0/2788101876 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85780640a0 con 0x7f8578109ab0 2024-04-23T10:16:07.012 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:06.474+0000 7f857ca40700 1 -- 172.21.15.97:0/2788101876 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f857002c030 con 0x7f8578109ab0 2024-04-23T10:16:07.012 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:06.543+0000 7f857da42700 1 -- 172.21.15.97:0/2788101876 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f8578195110 con 0x7f8578109ab0 2024-04-23T10:16:07.013 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:06.543+0000 7f857ca40700 1 -- 172.21.15.97:0/2788101876 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f8570012660 con 0x7f8578109ab0 2024-04-23T10:16:07.013 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:06.544+0000 7f857da42700 1 -- 172.21.15.97:0/2788101876 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8578109ab0 msgr2=0x7f85781090d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:16:07.013 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:06.544+0000 7f857da42700 1 --2- 172.21.15.97:0/2788101876 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8578109ab0 0x7f85781090d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8570000f80 tx=0x7f8570004ad0 comp rx=0 tx=0).stop 2024-04-23T10:16:07.013 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:06.544+0000 7f857da42700 1 -- 172.21.15.97:0/2788101876 shutdown_connections 2024-04-23T10:16:07.013 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:06.544+0000 7f857da42700 1 --2- 172.21.15.97:0/2788101876 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8578109ab0 0x7f85781090d0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:16:07.013 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:06.544+0000 7f857da42700 1 -- 172.21.15.97:0/2788101876 >> 172.21.15.97:0/2788101876 conn(0x7f8578102110 msgr2=0x7f8578192870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:16:07.013 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:06.544+0000 7f857da42700 1 -- 172.21.15.97:0/2788101876 shutdown_connections 2024-04-23T10:16:07.013 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:06.544+0000 7f857da42700 1 -- 172.21.15.97:0/2788101876 wait complete. 2024-04-23T10:16:07.013 INFO:teuthology.orchestra.run.smithi097.stdout:mon is available 2024-04-23T10:16:07.013 INFO:teuthology.orchestra.run.smithi097.stdout:Assimilating anything we can from ceph.conf... 2024-04-23T10:16:07.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:06 smithi097 ceph-mon[28825]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-23T10:16:07.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:06 smithi097 ceph-mon[28825]: monmap epoch 1 2024-04-23T10:16:07.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:06 smithi097 ceph-mon[28825]: fsid 9b22af88-0159-11ef-bc93-c7b262605968 2024-04-23T10:16:07.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:06 smithi097 ceph-mon[28825]: last_changed 2024-04-23T10:16:03.064570+0000 2024-04-23T10:16:07.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:06 smithi097 ceph-mon[28825]: created 2024-04-23T10:16:03.064570+0000 2024-04-23T10:16:07.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:06 smithi097 ceph-mon[28825]: min_mon_release 19 (squid) 2024-04-23T10:16:07.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:06 smithi097 ceph-mon[28825]: election_strategy: 1 2024-04-23T10:16:07.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:06 smithi097 ceph-mon[28825]: 0: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.a 2024-04-23T10:16:07.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:06 smithi097 ceph-mon[28825]: fsmap 2024-04-23T10:16:07.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:06 smithi097 ceph-mon[28825]: osdmap e1: 0 total, 0 up, 0 in 2024-04-23T10:16:07.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:06 smithi097 ceph-mon[28825]: mgrmap e1: no daemons active 2024-04-23T10:16:07.112 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:06 smithi097 ceph-mon[28825]: from='client.? 172.21.15.97:0/2788101876' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-23T10:16:08.157 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2024-04-23T10:16:08.157 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout [global] 2024-04-23T10:16:08.157 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout fsid = 9b22af88-0159-11ef-bc93-c7b262605968 2024-04-23T10:16:08.157 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-23T10:16:08.157 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.97:3300,v1:172.21.15.97:6789] 2024-04-23T10:16:08.157 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-23T10:16:08.157 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-23T10:16:08.157 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-23T10:16:08.157 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-23T10:16:08.157 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2024-04-23T10:16:08.158 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-23T10:16:08.158 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-23T10:16:08.158 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2024-04-23T10:16:08.158 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout [osd] 2024-04-23T10:16:08.158 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-23T10:16:08.158 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-23T10:16:08.158 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-23T10:16:08.158 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-23T10:16:08.158 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-23T10:16:08.158 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:07.442+0000 7f9942c55700 1 Processor -- start 2024-04-23T10:16:08.158 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:07.442+0000 7f9942c55700 1 -- start start 2024-04-23T10:16:08.158 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:07.442+0000 7f9942c55700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f993c108fc0 0x7f993c10b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:16:08.158 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:07.442+0000 7f9942c55700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f993c07be80 con 0x7f993c108fc0 2024-04-23T10:16:08.159 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:07.442+0000 7f99409f1700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f993c108fc0 0x7f993c10b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:16:08.159 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:07.442+0000 7f99409f1700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f993c108fc0 0x7f993c10b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43206/0 (socket says 172.21.15.97:43206) 2024-04-23T10:16:08.159 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:07.442+0000 7f99409f1700 1 -- 172.21.15.97:0/342706916 learned_addr learned my addr 172.21.15.97:0/342706916 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:16:08.159 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:07.442+0000 7f99409f1700 1 -- 172.21.15.97:0/342706916 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f993c10b8f0 con 0x7f993c108fc0 2024-04-23T10:16:08.159 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:07.442+0000 7f99409f1700 1 --2- 172.21.15.97:0/342706916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f993c108fc0 0x7f993c10b3b0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f99300098a0 tx=0x7f9930009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e7eaa7921dadb26 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:16:08.159 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:07.442+0000 7f993b7fe700 1 -- 172.21.15.97:0/342706916 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9930004160 con 0x7f993c108fc0 2024-04-23T10:16:08.159 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:07.443+0000 7f993b7fe700 1 -- 172.21.15.97:0/342706916 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9930004300 con 0x7f993c108fc0 2024-04-23T10:16:08.159 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:07.443+0000 7f993b7fe700 1 -- 172.21.15.97:0/342706916 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f99300036a0 con 0x7f993c108fc0 2024-04-23T10:16:08.159 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:07.443+0000 7f9942c55700 1 -- 172.21.15.97:0/342706916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f993c108fc0 msgr2=0x7f993c10b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:16:08.159 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:07.443+0000 7f9942c55700 1 --2- 172.21.15.97:0/342706916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f993c108fc0 0x7f993c10b3b0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f99300098a0 tx=0x7f9930009bb0 comp rx=0 tx=0).stop 2024-04-23T10:16:08.159 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:07.443+0000 7f9942c55700 1 -- 172.21.15.97:0/342706916 shutdown_connections 2024-04-23T10:16:08.159 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:07.443+0000 7f9942c55700 1 --2- 172.21.15.97:0/342706916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f993c108fc0 0x7f993c10b3b0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:16:08.159 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:07.443+0000 7f9942c55700 1 -- 172.21.15.97:0/342706916 >> 172.21.15.97:0/342706916 conn(0x7f993c102a70 msgr2=0x7f993c104e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:16:08.159 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:07.443+0000 7f9942c55700 1 -- 172.21.15.97:0/342706916 shutdown_connections 2024-04-23T10:16:08.160 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:07.443+0000 7f9942c55700 1 -- 172.21.15.97:0/342706916 wait complete. 2024-04-23T10:16:08.160 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:07.444+0000 7f9942c55700 1 Processor -- start 2024-04-23T10:16:08.160 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:07.444+0000 7f9942c55700 1 -- start start 2024-04-23T10:16:08.160 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:07.444+0000 7f9942c55700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f993c108fc0 0x7f993c1087f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:16:08.160 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:07.444+0000 7f9942c55700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f993c10bd60 con 0x7f993c108fc0 2024-04-23T10:16:08.160 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:07.444+0000 7f99409f1700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f993c108fc0 0x7f993c1087f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:16:08.160 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:07.444+0000 7f99409f1700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f993c108fc0 0x7f993c1087f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43218/0 (socket says 172.21.15.97:43218) 2024-04-23T10:16:08.160 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:07.444+0000 7f99409f1700 1 -- 172.21.15.97:0/4090139403 learned_addr learned my addr 172.21.15.97:0/4090139403 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:16:08.160 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:07.444+0000 7f99409f1700 1 -- 172.21.15.97:0/4090139403 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f993c108d30 con 0x7f993c108fc0 2024-04-23T10:16:08.160 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:07.444+0000 7f99409f1700 1 --2- 172.21.15.97:0/4090139403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f993c108fc0 0x7f993c1087f0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f9930000c00 tx=0x7f9930004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:16:08.160 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:07.445+0000 7f9939ffb700 1 -- 172.21.15.97:0/4090139403 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9930003db0 con 0x7f993c108fc0 2024-04-23T10:16:08.160 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:07.445+0000 7f9939ffb700 1 -- 172.21.15.97:0/4090139403 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9930011630 con 0x7f993c108fc0 2024-04-23T10:16:08.160 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:07.445+0000 7f9939ffb700 1 -- 172.21.15.97:0/4090139403 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9930011950 con 0x7f993c108fc0 2024-04-23T10:16:08.160 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:07.445+0000 7f9942c55700 1 -- 172.21.15.97:0/4090139403 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f993c106f70 con 0x7f993c108fc0 2024-04-23T10:16:08.160 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:07.445+0000 7f9942c55700 1 -- 172.21.15.97:0/4090139403 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f993c107450 con 0x7f993c108fc0 2024-04-23T10:16:08.160 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:07.445+0000 7f9939ffb700 1 -- 172.21.15.97:0/4090139403 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9930011af0 con 0x7f993c108fc0 2024-04-23T10:16:08.161 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:07.445+0000 7f9939ffb700 1 -- 172.21.15.97:0/4090139403 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f993001c450 con 0x7f993c108fc0 2024-04-23T10:16:08.161 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:07.446+0000 7f9942c55700 1 -- 172.21.15.97:0/4090139403 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f993c0640a0 con 0x7f993c108fc0 2024-04-23T10:16:08.161 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:07.448+0000 7f9939ffb700 1 -- 172.21.15.97:0/4090139403 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f9930004260 con 0x7f993c108fc0 2024-04-23T10:16:08.161 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:07.515+0000 7f9942c55700 1 -- 172.21.15.97:0/4090139403 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f993c195270 con 0x7f993c108fc0 2024-04-23T10:16:08.161 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:07.518+0000 7f9939ffb700 1 -- 172.21.15.97:0/4090139403 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+465 (secure 0 0 0) 0x7f993001b450 con 0x7f993c108fc0 2024-04-23T10:16:08.161 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:07.519+0000 7f9939ffb700 1 -- 172.21.15.97:0/4090139403 <== mon.0 v2:172.21.15.97:3300/0 8 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f993001b630 con 0x7f993c108fc0 2024-04-23T10:16:08.161 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:07.520+0000 7f9942c55700 1 -- 172.21.15.97:0/4090139403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f993c108fc0 msgr2=0x7f993c1087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:16:08.161 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:07.520+0000 7f9942c55700 1 --2- 172.21.15.97:0/4090139403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f993c108fc0 0x7f993c1087f0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f9930000c00 tx=0x7f9930004750 comp rx=0 tx=0).stop 2024-04-23T10:16:08.161 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:07.520+0000 7f9942c55700 1 -- 172.21.15.97:0/4090139403 shutdown_connections 2024-04-23T10:16:08.161 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:07.520+0000 7f9942c55700 1 --2- 172.21.15.97:0/4090139403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f993c108fc0 0x7f993c1087f0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:16:08.161 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:07.520+0000 7f9942c55700 1 -- 172.21.15.97:0/4090139403 >> 172.21.15.97:0/4090139403 conn(0x7f993c102a70 msgr2=0x7f993c104d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:16:08.161 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:07.520+0000 7f9942c55700 1 -- 172.21.15.97:0/4090139403 shutdown_connections 2024-04-23T10:16:08.161 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:07.520+0000 7f9942c55700 1 -- 172.21.15.97:0/4090139403 wait complete. 2024-04-23T10:16:08.162 INFO:teuthology.orchestra.run.smithi097.stdout:Generating new minimal ceph.conf... 2024-04-23T10:16:08.267 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:07 smithi097 ceph-mon[28825]: from='client.? 172.21.15.97:0/4090139403' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-23T10:16:08.267 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:07 smithi097 ceph-mon[28825]: from='client.? 172.21.15.97:0/4090139403' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-23T10:16:09.186 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:08.583+0000 7ff9a0494700 1 Processor -- start 2024-04-23T10:16:09.186 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:08.583+0000 7ff9a0494700 1 -- start start 2024-04-23T10:16:09.186 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:08.583+0000 7ff9a0494700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff998109b30 0x7ff998109f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:16:09.186 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:08.583+0000 7ff9a0494700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff99810a520 con 0x7ff998109b30 2024-04-23T10:16:09.186 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:08.584+0000 7ff99e230700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff998109b30 0x7ff998109f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:16:09.186 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:08.584+0000 7ff99e230700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff998109b30 0x7ff998109f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43224/0 (socket says 172.21.15.97:43224) 2024-04-23T10:16:09.187 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:08.584+0000 7ff99e230700 1 -- 172.21.15.97:0/618384532 learned_addr learned my addr 172.21.15.97:0/618384532 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:16:09.187 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:08.584+0000 7ff99e230700 1 -- 172.21.15.97:0/618384532 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff99810a6a0 con 0x7ff998109b30 2024-04-23T10:16:09.187 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:08.584+0000 7ff99e230700 1 --2- 172.21.15.97:0/618384532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff998109b30 0x7ff998109f50 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff990009e80 tx=0x7ff99000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0a1e190c53ad73a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:16:09.187 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:08.584+0000 7ff99d22e700 1 -- 172.21.15.97:0/618384532 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff990017070 con 0x7ff998109b30 2024-04-23T10:16:09.187 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:08.584+0000 7ff99d22e700 1 -- 172.21.15.97:0/618384532 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff99000b8c0 con 0x7ff998109b30 2024-04-23T10:16:09.187 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:08.584+0000 7ff99d22e700 1 -- 172.21.15.97:0/618384532 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff990004490 con 0x7ff998109b30 2024-04-23T10:16:09.187 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:08.585+0000 7ff9a0494700 1 -- 172.21.15.97:0/618384532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff998109b30 msgr2=0x7ff998109f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:16:09.187 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:08.585+0000 7ff9a0494700 1 --2- 172.21.15.97:0/618384532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff998109b30 0x7ff998109f50 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff990009e80 tx=0x7ff99000b330 comp rx=0 tx=0).stop 2024-04-23T10:16:09.187 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:08.585+0000 7ff9a0494700 1 -- 172.21.15.97:0/618384532 shutdown_connections 2024-04-23T10:16:09.187 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:08.585+0000 7ff9a0494700 1 --2- 172.21.15.97:0/618384532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff998109b30 0x7ff998109f50 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:16:09.187 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:08.585+0000 7ff9a0494700 1 -- 172.21.15.97:0/618384532 >> 172.21.15.97:0/618384532 conn(0x7ff9981021b0 msgr2=0x7ff9981045d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:16:09.187 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:08.585+0000 7ff9a0494700 1 -- 172.21.15.97:0/618384532 shutdown_connections 2024-04-23T10:16:09.187 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:08.585+0000 7ff9a0494700 1 -- 172.21.15.97:0/618384532 wait complete. 2024-04-23T10:16:09.187 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:08.585+0000 7ff9a0494700 1 Processor -- start 2024-04-23T10:16:09.188 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:08.585+0000 7ff9a0494700 1 -- start start 2024-04-23T10:16:09.188 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:08.585+0000 7ff9a0494700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff998109b30 0x7ff998109100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:16:09.188 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:08.585+0000 7ff9a0494700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff998107330 con 0x7ff998109b30 2024-04-23T10:16:09.188 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:08.586+0000 7ff99e230700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff998109b30 0x7ff998109100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:16:09.188 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:08.586+0000 7ff99e230700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff998109b30 0x7ff998109100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43228/0 (socket says 172.21.15.97:43228) 2024-04-23T10:16:09.188 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:08.586+0000 7ff99e230700 1 -- 172.21.15.97:0/3836865047 learned_addr learned my addr 172.21.15.97:0/3836865047 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:16:09.188 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:08.586+0000 7ff99e230700 1 -- 172.21.15.97:0/3836865047 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff998109640 con 0x7ff998109b30 2024-04-23T10:16:09.188 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:08.586+0000 7ff99e230700 1 --2- 172.21.15.97:0/3836865047 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff998109b30 0x7ff998109100 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff990004220 tx=0x7ff99000be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:16:09.188 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:08.586+0000 7ff9977fe700 1 -- 172.21.15.97:0/3836865047 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff990020020 con 0x7ff998109b30 2024-04-23T10:16:09.188 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:08.586+0000 7ff9a0494700 1 -- 172.21.15.97:0/3836865047 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff998107880 con 0x7ff998109b30 2024-04-23T10:16:09.188 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:08.586+0000 7ff9a0494700 1 -- 172.21.15.97:0/3836865047 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff998107dc0 con 0x7ff998109b30 2024-04-23T10:16:09.188 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:08.586+0000 7ff9977fe700 1 -- 172.21.15.97:0/3836865047 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff990012430 con 0x7ff998109b30 2024-04-23T10:16:09.188 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:08.586+0000 7ff9977fe700 1 -- 172.21.15.97:0/3836865047 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff990017040 con 0x7ff998109b30 2024-04-23T10:16:09.188 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:08.586+0000 7ff9977fe700 1 -- 172.21.15.97:0/3836865047 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff990023020 con 0x7ff998109b30 2024-04-23T10:16:09.188 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:08.586+0000 7ff9977fe700 1 -- 172.21.15.97:0/3836865047 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff99000f050 con 0x7ff998109b30 2024-04-23T10:16:09.188 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:08.587+0000 7ff9a0494700 1 -- 172.21.15.97:0/3836865047 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff984005360 con 0x7ff998109b30 2024-04-23T10:16:09.189 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:08.589+0000 7ff9977fe700 1 -- 172.21.15.97:0/3836865047 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7ff99002a020 con 0x7ff998109b30 2024-04-23T10:16:09.189 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:08.657+0000 7ff9a0494700 1 -- 172.21.15.97:0/3836865047 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7ff984005920 con 0x7ff998109b30 2024-04-23T10:16:09.189 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:08.657+0000 7ff9977fe700 1 -- 172.21.15.97:0/3836865047 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7ff99000b860 con 0x7ff998109b30 2024-04-23T10:16:09.189 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:08.658+0000 7ff9a0494700 1 -- 172.21.15.97:0/3836865047 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff998109b30 msgr2=0x7ff998109100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:16:09.189 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:08.658+0000 7ff9a0494700 1 --2- 172.21.15.97:0/3836865047 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff998109b30 0x7ff998109100 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff990004220 tx=0x7ff99000be90 comp rx=0 tx=0).stop 2024-04-23T10:16:09.189 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:08.658+0000 7ff9a0494700 1 -- 172.21.15.97:0/3836865047 shutdown_connections 2024-04-23T10:16:09.189 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:08.658+0000 7ff9a0494700 1 --2- 172.21.15.97:0/3836865047 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff998109b30 0x7ff998109100 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:16:09.189 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:08.658+0000 7ff9a0494700 1 -- 172.21.15.97:0/3836865047 >> 172.21.15.97:0/3836865047 conn(0x7ff9981021b0 msgr2=0x7ff998102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:16:09.189 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:08.658+0000 7ff9a0494700 1 -- 172.21.15.97:0/3836865047 shutdown_connections 2024-04-23T10:16:09.189 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:08.658+0000 7ff9a0494700 1 -- 172.21.15.97:0/3836865047 wait complete. 2024-04-23T10:16:09.189 INFO:teuthology.orchestra.run.smithi097.stdout:Restarting the monitor... 2024-04-23T10:16:09.201 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:08 smithi097 ceph-mon[28825]: from='client.? 172.21.15.97:0/3836865047' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:16:09.458 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:09 smithi097 systemd[1]: Stopping Ceph mon.a for 9b22af88-0159-11ef-bc93-c7b262605968... 2024-04-23T10:16:09.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:09 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mon-a[28821]: 2024-04-23T10:16:09.458+0000 7fb7ec75e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T10:16:09.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:09 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mon-a[28821]: 2024-04-23T10:16:09.458+0000 7fb7ec75e700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-23T10:16:09.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:09 smithi097 podman[29071]: 2024-04-23 10:16:09.674052781 +0000 UTC m=+0.331000785 container died da307fca1800bf85723bfb492133a120425f3ddbfc0da887e2d64c5c3f6799be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mon-a, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-23T10:16:10.325 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 podman[29071]: 2024-04-23 10:16:10.033886594 +0000 UTC m=+0.690834608 container cleanup da307fca1800bf85723bfb492133a120425f3ddbfc0da887e2d64c5c3f6799be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mon-a, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-23T10:16:10.325 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 bash[29071]: ceph-9b22af88-0159-11ef-bc93-c7b262605968-mon-a 2024-04-23T10:16:10.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 podman[29085]: 2024-04-23 10:16:10.326832718 +0000 UTC m=+0.646332005 container remove da307fca1800bf85723bfb492133a120425f3ddbfc0da887e2d64c5c3f6799be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-23T10:16:10.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 systemd[1]: ceph-9b22af88-0159-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-23T10:16:10.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 systemd[1]: Stopped Ceph mon.a for 9b22af88-0159-11ef-bc93-c7b262605968. 2024-04-23T10:16:10.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 systemd[1]: Starting Ceph mon.a for 9b22af88-0159-11ef-bc93-c7b262605968... 2024-04-23T10:16:10.903 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 podman[29177]: 2024-04-23 10:16:10.644293703 +0000 UTC m=+0.096871184 container create f2859bdee5d0a82e12711228b6fbee80a2e446ba32e6b3a320d9737e3d6e2b4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mon-a, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T10:16:10.903 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 podman[29177]: 2024-04-23 10:16:10.565637763 +0000 UTC m=+0.018215260 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-23T10:16:10.904 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 podman[29177]: 2024-04-23 10:16:10.821808556 +0000 UTC m=+0.274386042 container init f2859bdee5d0a82e12711228b6fbee80a2e446ba32e6b3a320d9737e3d6e2b4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mon-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS) 2024-04-23T10:16:10.904 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 podman[29177]: 2024-04-23 10:16:10.824869584 +0000 UTC m=+0.277447066 container start f2859bdee5d0a82e12711228b6fbee80a2e446ba32e6b3a320d9737e3d6e2b4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mon-a, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, ceph=True) 2024-04-23T10:16:10.904 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: set uid:gid to 167:167 (ceph:ceph) 2024-04-23T10:16:10.904 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev), process ceph-mon, pid 2 2024-04-23T10:16:10.904 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: pidfile_write: ignore empty --pid-file 2024-04-23T10:16:10.904 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: load: jerasure load: lrc 2024-04-23T10:16:10.904 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: RocksDB version: 7.9.2 2024-04-23T10:16:10.904 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Git sha 0 2024-04-23T10:16:10.904 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Compile date 2024-04-22 10:57:25 2024-04-23T10:16:10.904 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: DB SUMMARY 2024-04-23T10:16:10.904 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: DB Session ID: C1EBC9TS8K05P5XLH9K1 2024-04-23T10:16:10.905 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: CURRENT file: CURRENT 2024-04-23T10:16:10.905 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: IDENTITY file: IDENTITY 2024-04-23T10:16:10.905 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-23T10:16:10.905 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-23T10:16:10.905 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105682 ; 2024-04-23T10:16:10.905 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.error_if_exists: 0 2024-04-23T10:16:10.906 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.create_if_missing: 0 2024-04-23T10:16:10.906 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.paranoid_checks: 1 2024-04-23T10:16:10.906 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-23T10:16:10.907 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-23T10:16:10.907 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-23T10:16:10.907 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.env: 0x559cced26cc0 2024-04-23T10:16:10.907 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.fs: PosixFileSystem 2024-04-23T10:16:10.907 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.info_log: 0x559cd0005be0 2024-04-23T10:16:10.907 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.max_file_opening_threads: 16 2024-04-23T10:16:10.907 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.statistics: (nil) 2024-04-23T10:16:10.907 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.use_fsync: 0 2024-04-23T10:16:10.907 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.max_log_file_size: 0 2024-04-23T10:16:10.907 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-23T10:16:10.907 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-23T10:16:10.907 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.keep_log_file_num: 1000 2024-04-23T10:16:10.907 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.recycle_log_file_num: 0 2024-04-23T10:16:10.908 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.allow_fallocate: 1 2024-04-23T10:16:10.908 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.allow_mmap_reads: 0 2024-04-23T10:16:10.908 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.allow_mmap_writes: 0 2024-04-23T10:16:10.908 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.use_direct_reads: 0 2024-04-23T10:16:10.908 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-23T10:16:10.908 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.create_missing_column_families: 0 2024-04-23T10:16:10.908 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.db_log_dir: 2024-04-23T10:16:10.908 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.wal_dir: 2024-04-23T10:16:10.908 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-23T10:16:10.908 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-23T10:16:10.909 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-23T10:16:10.909 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-23T10:16:10.909 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-23T10:16:10.909 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-23T10:16:10.909 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.advise_random_on_open: 1 2024-04-23T10:16:10.909 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.db_write_buffer_size: 0 2024-04-23T10:16:10.909 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.write_buffer_manager: 0x559ccff990e0 2024-04-23T10:16:10.909 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-23T10:16:10.909 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-23T10:16:10.909 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-23T10:16:10.909 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.rate_limiter: (nil) 2024-04-23T10:16:10.909 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-23T10:16:10.909 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.wal_recovery_mode: 2 2024-04-23T10:16:10.909 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.enable_thread_tracking: 0 2024-04-23T10:16:10.909 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.enable_pipelined_write: 0 2024-04-23T10:16:10.910 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.unordered_write: 0 2024-04-23T10:16:10.910 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-23T10:16:10.910 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-23T10:16:10.910 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-23T10:16:10.910 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-23T10:16:10.910 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.row_cache: None 2024-04-23T10:16:10.910 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.wal_filter: None 2024-04-23T10:16:10.910 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-23T10:16:10.910 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.allow_ingest_behind: 0 2024-04-23T10:16:10.910 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.two_write_queues: 0 2024-04-23T10:16:10.910 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.manual_wal_flush: 0 2024-04-23T10:16:10.910 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.wal_compression: 0 2024-04-23T10:16:10.910 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.atomic_flush: 0 2024-04-23T10:16:10.910 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-23T10:16:10.911 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-23T10:16:10.911 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-23T10:16:10.911 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.log_readahead_size: 0 2024-04-23T10:16:10.911 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-23T10:16:10.911 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.best_efforts_recovery: 0 2024-04-23T10:16:10.911 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-23T10:16:10.911 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-23T10:16:10.911 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.allow_data_in_errors: 0 2024-04-23T10:16:10.911 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.db_host_id: __hostname__ 2024-04-23T10:16:10.911 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-23T10:16:10.911 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.max_background_jobs: 2 2024-04-23T10:16:10.911 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.max_background_compactions: -1 2024-04-23T10:16:10.911 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.max_subcompactions: 1 2024-04-23T10:16:10.911 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-23T10:16:10.911 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-23T10:16:10.912 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-23T10:16:10.912 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.max_total_wal_size: 0 2024-04-23T10:16:10.912 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-23T10:16:10.912 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-23T10:16:10.912 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-23T10:16:10.912 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-23T10:16:10.912 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.max_open_files: -1 2024-04-23T10:16:10.912 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.bytes_per_sync: 0 2024-04-23T10:16:10.912 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-23T10:16:10.912 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-23T10:16:10.912 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.compaction_readahead_size: 0 2024-04-23T10:16:10.912 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.max_background_flushes: -1 2024-04-23T10:16:10.912 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Compression algorithms supported: 2024-04-23T10:16:10.912 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-23T10:16:10.912 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: kZSTD supported: 0 2024-04-23T10:16:10.912 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: kXpressCompression supported: 0 2024-04-23T10:16:10.913 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: kLZ4HCCompression supported: 1 2024-04-23T10:16:10.913 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: kZlibCompression supported: 1 2024-04-23T10:16:10.913 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: kSnappyCompression supported: 1 2024-04-23T10:16:10.913 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: kLZ4Compression supported: 1 2024-04-23T10:16:10.913 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: kBZip2Compression supported: 0 2024-04-23T10:16:10.913 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-23T10:16:10.913 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-23T10:16:10.913 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-23T10:16:10.913 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-23T10:16:10.913 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-23T10:16:10.913 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.merge_operator: 2024-04-23T10:16:10.913 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.compaction_filter: None 2024-04-23T10:16:10.913 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.compaction_filter_factory: None 2024-04-23T10:16:10.913 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.sst_partitioner_factory: None 2024-04-23T10:16:10.913 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-23T10:16:10.914 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-23T10:16:10.914 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x559cd0005dc0) 2024-04-23T10:16:10.914 INFO:journalctl@ceph.mon.a.smithi097.stdout: cache_index_and_filter_blocks: 1 2024-04-23T10:16:10.914 INFO:journalctl@ceph.mon.a.smithi097.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-23T10:16:10.914 INFO:journalctl@ceph.mon.a.smithi097.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-23T10:16:10.914 INFO:journalctl@ceph.mon.a.smithi097.stdout: pin_top_level_index_and_filter: 1 2024-04-23T10:16:10.914 INFO:journalctl@ceph.mon.a.smithi097.stdout: index_type: 0 2024-04-23T10:16:10.914 INFO:journalctl@ceph.mon.a.smithi097.stdout: data_block_index_type: 0 2024-04-23T10:16:10.914 INFO:journalctl@ceph.mon.a.smithi097.stdout: index_shortening: 1 2024-04-23T10:16:10.914 INFO:journalctl@ceph.mon.a.smithi097.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-23T10:16:10.914 INFO:journalctl@ceph.mon.a.smithi097.stdout: checksum: 4 2024-04-23T10:16:10.914 INFO:journalctl@ceph.mon.a.smithi097.stdout: no_block_cache: 0 2024-04-23T10:16:10.914 INFO:journalctl@ceph.mon.a.smithi097.stdout: block_cache: 0x559ccffcb1f0 2024-04-23T10:16:10.914 INFO:journalctl@ceph.mon.a.smithi097.stdout: block_cache_name: BinnedLRUCache 2024-04-23T10:16:10.914 INFO:journalctl@ceph.mon.a.smithi097.stdout: block_cache_options: 2024-04-23T10:16:10.915 INFO:journalctl@ceph.mon.a.smithi097.stdout: capacity : 536870912 2024-04-23T10:16:10.915 INFO:journalctl@ceph.mon.a.smithi097.stdout: num_shard_bits : 4 2024-04-23T10:16:10.915 INFO:journalctl@ceph.mon.a.smithi097.stdout: strict_capacity_limit : 0 2024-04-23T10:16:10.915 INFO:journalctl@ceph.mon.a.smithi097.stdout: high_pri_pool_ratio: 0.000 2024-04-23T10:16:10.915 INFO:journalctl@ceph.mon.a.smithi097.stdout: block_cache_compressed: (nil) 2024-04-23T10:16:10.915 INFO:journalctl@ceph.mon.a.smithi097.stdout: persistent_cache: (nil) 2024-04-23T10:16:10.915 INFO:journalctl@ceph.mon.a.smithi097.stdout: block_size: 4096 2024-04-23T10:16:10.915 INFO:journalctl@ceph.mon.a.smithi097.stdout: block_size_deviation: 10 2024-04-23T10:16:10.915 INFO:journalctl@ceph.mon.a.smithi097.stdout: block_restart_interval: 16 2024-04-23T10:16:10.915 INFO:journalctl@ceph.mon.a.smithi097.stdout: index_block_restart_interval: 1 2024-04-23T10:16:10.915 INFO:journalctl@ceph.mon.a.smithi097.stdout: metadata_block_size: 4096 2024-04-23T10:16:10.915 INFO:journalctl@ceph.mon.a.smithi097.stdout: partition_filters: 0 2024-04-23T10:16:10.915 INFO:journalctl@ceph.mon.a.smithi097.stdout: use_delta_encoding: 1 2024-04-23T10:16:10.915 INFO:journalctl@ceph.mon.a.smithi097.stdout: filter_policy: bloomfilter 2024-04-23T10:16:10.915 INFO:journalctl@ceph.mon.a.smithi097.stdout: whole_key_filtering: 1 2024-04-23T10:16:10.915 INFO:journalctl@ceph.mon.a.smithi097.stdout: verify_compression: 0 2024-04-23T10:16:10.916 INFO:journalctl@ceph.mon.a.smithi097.stdout: read_amp_bytes_per_bit: 0 2024-04-23T10:16:10.916 INFO:journalctl@ceph.mon.a.smithi097.stdout: format_version: 5 2024-04-23T10:16:10.916 INFO:journalctl@ceph.mon.a.smithi097.stdout: enable_index_compression: 1 2024-04-23T10:16:10.916 INFO:journalctl@ceph.mon.a.smithi097.stdout: block_align: 0 2024-04-23T10:16:10.916 INFO:journalctl@ceph.mon.a.smithi097.stdout: max_auto_readahead_size: 262144 2024-04-23T10:16:10.916 INFO:journalctl@ceph.mon.a.smithi097.stdout: prepopulate_block_cache: 0 2024-04-23T10:16:10.916 INFO:journalctl@ceph.mon.a.smithi097.stdout: initial_auto_readahead_size: 8192 2024-04-23T10:16:10.916 INFO:journalctl@ceph.mon.a.smithi097.stdout: num_file_reads_for_auto_readahead: 2 2024-04-23T10:16:10.916 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.write_buffer_size: 33554432 2024-04-23T10:16:10.916 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.max_write_buffer_number: 2 2024-04-23T10:16:10.916 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.compression: NoCompression 2024-04-23T10:16:10.916 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.bottommost_compression: Disabled 2024-04-23T10:16:10.916 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.prefix_extractor: nullptr 2024-04-23T10:16:10.916 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-23T10:16:10.916 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.num_levels: 7 2024-04-23T10:16:10.916 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-23T10:16:10.917 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-23T10:16:10.917 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-23T10:16:10.917 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-23T10:16:10.917 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-23T10:16:10.917 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-23T10:16:10.917 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-23T10:16:10.917 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-23T10:16:10.917 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-23T10:16:10.917 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-23T10:16:10.917 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-23T10:16:10.917 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-23T10:16:10.917 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-23T10:16:10.917 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.compression_opts.level: 32767 2024-04-23T10:16:10.917 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.compression_opts.strategy: 0 2024-04-23T10:16:10.917 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-23T10:16:10.918 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-23T10:16:10.918 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-23T10:16:10.918 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-23T10:16:10.918 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.compression_opts.enabled: false 2024-04-23T10:16:10.918 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-23T10:16:10.918 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-23T10:16:10.918 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-23T10:16:10.918 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-23T10:16:10.918 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.target_file_size_base: 67108864 2024-04-23T10:16:10.918 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-23T10:16:10.918 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-23T10:16:10.918 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-23T10:16:10.918 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-23T10:16:10.918 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-23T10:16:10.919 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-23T10:16:10.919 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-23T10:16:10.919 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-23T10:16:10.919 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-23T10:16:10.919 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-23T10:16:10.919 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-23T10:16:10.919 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-23T10:16:10.919 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-23T10:16:10.919 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-23T10:16:10.919 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.arena_block_size: 1048576 2024-04-23T10:16:10.919 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-23T10:16:10.919 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-23T10:16:10.919 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.disable_auto_compactions: 0 2024-04-23T10:16:10.919 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-23T10:16:10.919 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-23T10:16:10.920 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-23T10:16:10.920 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-23T10:16:10.920 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-23T10:16:10.920 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-23T10:16:10.920 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-23T10:16:10.920 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-23T10:16:10.920 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-23T10:16:10.920 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-23T10:16:10.920 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-23T10:16:10.920 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.inplace_update_support: 0 2024-04-23T10:16:10.920 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-23T10:16:10.920 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-23T10:16:10.920 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-23T10:16:10.920 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-23T10:16:10.920 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.bloom_locality: 0 2024-04-23T10:16:10.920 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.max_successive_merges: 0 2024-04-23T10:16:10.920 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-23T10:16:10.921 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.paranoid_file_checks: 0 2024-04-23T10:16:10.921 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.force_consistency_checks: 1 2024-04-23T10:16:10.921 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.report_bg_io_stats: 0 2024-04-23T10:16:10.921 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.ttl: 2592000 2024-04-23T10:16:10.921 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-23T10:16:10.921 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-23T10:16:10.921 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-23T10:16:10.921 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.enable_blob_files: false 2024-04-23T10:16:10.921 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.min_blob_size: 0 2024-04-23T10:16:10.921 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.blob_file_size: 268435456 2024-04-23T10:16:10.921 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-23T10:16:10.921 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-23T10:16:10.921 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-23T10:16:10.921 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-23T10:16:10.921 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-23T10:16:10.921 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.blob_file_starting_level: 0 2024-04-23T10:16:10.922 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-23T10:16:10.922 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-23T10:16:10.922 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-23T10:16:10.922 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 0eadeb37-8a88-41df-b452-9c4dd549266c 2024-04-23T10:16:10.922 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713867370853067, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-23T10:16:10.922 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-23T10:16:10.922 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713867370854155, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100789, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 328, "table_properties": {"data_size": 98677, "index_size": 304, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14302, "raw_average_key_size": 49, "raw_value_size": 90895, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 287, "num_filter_entries": 287, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713867370, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "0eadeb37-8a88-41df-b452-9c4dd549266c", "db_session_id": "C1EBC9TS8K05P5XLH9K1", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-23T10:16:10.922 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713867370854235, "job": 1, "event": "recovery_finished"} 2024-04-23T10:16:10.922 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-23T10:16:10.922 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-23T10:16:10.922 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x559cd00e0000 2024-04-23T10:16:10.922 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: DB pointer 0x559cd00ca000 2024-04-23T10:16:10.922 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-23T10:16:10.922 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-23T10:16:10.922 INFO:journalctl@ceph.mon.a.smithi097.stdout: ** DB Stats ** 2024-04-23T10:16:10.923 INFO:journalctl@ceph.mon.a.smithi097.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-23T10:16:10.923 INFO:journalctl@ceph.mon.a.smithi097.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-23T10:16:10.923 INFO:journalctl@ceph.mon.a.smithi097.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-23T10:16:10.923 INFO:journalctl@ceph.mon.a.smithi097.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-23T10:16:10.923 INFO:journalctl@ceph.mon.a.smithi097.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-23T10:16:10.923 INFO:journalctl@ceph.mon.a.smithi097.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-23T10:16:10.923 INFO:journalctl@ceph.mon.a.smithi097.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-23T10:16:10.923 INFO:journalctl@ceph.mon.a.smithi097.stdout: 2024-04-23T10:16:10.923 INFO:journalctl@ceph.mon.a.smithi097.stdout: ** Compaction Stats [default] ** 2024-04-23T10:16:10.923 INFO:journalctl@ceph.mon.a.smithi097.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-23T10:16:10.923 INFO:journalctl@ceph.mon.a.smithi097.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-23T10:16:10.923 INFO:journalctl@ceph.mon.a.smithi097.stdout: L0 2/0 100.29 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 135.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-23T10:16:10.923 INFO:journalctl@ceph.mon.a.smithi097.stdout: Sum 2/0 100.29 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 135.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-23T10:16:10.923 INFO:journalctl@ceph.mon.a.smithi097.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 135.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-23T10:16:10.923 INFO:journalctl@ceph.mon.a.smithi097.stdout: 2024-04-23T10:16:10.924 INFO:journalctl@ceph.mon.a.smithi097.stdout: ** Compaction Stats [default] ** 2024-04-23T10:16:10.924 INFO:journalctl@ceph.mon.a.smithi097.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-23T10:16:10.924 INFO:journalctl@ceph.mon.a.smithi097.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-23T10:16:10.924 INFO:journalctl@ceph.mon.a.smithi097.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 135.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-23T10:16:10.924 INFO:journalctl@ceph.mon.a.smithi097.stdout: 2024-04-23T10:16:10.924 INFO:journalctl@ceph.mon.a.smithi097.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-23T10:16:10.924 INFO:journalctl@ceph.mon.a.smithi097.stdout: 2024-04-23T10:16:10.924 INFO:journalctl@ceph.mon.a.smithi097.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-23T10:16:10.924 INFO:journalctl@ceph.mon.a.smithi097.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-23T10:16:10.924 INFO:journalctl@ceph.mon.a.smithi097.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-23T10:16:10.924 INFO:journalctl@ceph.mon.a.smithi097.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-23T10:16:10.924 INFO:journalctl@ceph.mon.a.smithi097.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-23T10:16:10.924 INFO:journalctl@ceph.mon.a.smithi097.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-23T10:16:10.924 INFO:journalctl@ceph.mon.a.smithi097.stdout: Cumulative compaction: 0.00 GB write, 31.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-23T10:16:10.924 INFO:journalctl@ceph.mon.a.smithi097.stdout: Interval compaction: 0.00 GB write, 31.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-23T10:16:10.924 INFO:journalctl@ceph.mon.a.smithi097.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-23T10:16:10.925 INFO:journalctl@ceph.mon.a.smithi097.stdout: Block cache BinnedLRUCache@0x559ccffcb1f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-23T10:16:10.925 INFO:journalctl@ceph.mon.a.smithi097.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-23T10:16:10.925 INFO:journalctl@ceph.mon.a.smithi097.stdout: 2024-04-23T10:16:10.925 INFO:journalctl@ceph.mon.a.smithi097.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-23T10:16:10.925 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: starting mon.a rank 0 at public addrs [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] at bind addrs [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 9b22af88-0159-11ef-bc93-c7b262605968 2024-04-23T10:16:10.925 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: mon.a@-1(???) e1 preinit fsid 9b22af88-0159-11ef-bc93-c7b262605968 2024-04-23T10:16:10.925 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: mon.a@-1(???).mds e1 new map 2024-04-23T10:16:10.925 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: mon.a@-1(???).mds e1 print_map 2024-04-23T10:16:10.925 INFO:journalctl@ceph.mon.a.smithi097.stdout: e1 2024-04-23T10:16:10.925 INFO:journalctl@ceph.mon.a.smithi097.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T10:16:10.925 INFO:journalctl@ceph.mon.a.smithi097.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-23T10:16:10.925 INFO:journalctl@ceph.mon.a.smithi097.stdout: legacy client fscid: -1 2024-04-23T10:16:10.925 INFO:journalctl@ceph.mon.a.smithi097.stdout: 2024-04-23T10:16:10.925 INFO:journalctl@ceph.mon.a.smithi097.stdout: No filesystems configured 2024-04-23T10:16:10.925 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-23T10:16:10.925 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-23T10:16:10.925 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-23T10:16:10.926 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-23T10:16:10.926 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-23T10:16:10.926 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 bash[29177]: f2859bdee5d0a82e12711228b6fbee80a2e446ba32e6b3a320d9737e3d6e2b4d 2024-04-23T10:16:10.926 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 systemd[1]: Started Ceph mon.a for 9b22af88-0159-11ef-bc93-c7b262605968. 2024-04-23T10:16:10.926 INFO:teuthology.orchestra.run.smithi097.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-23T10:16:11.178 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-23T10:16:11.178 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: monmap epoch 1 2024-04-23T10:16:11.178 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: fsid 9b22af88-0159-11ef-bc93-c7b262605968 2024-04-23T10:16:11.178 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: last_changed 2024-04-23T10:16:03.064570+0000 2024-04-23T10:16:11.178 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: created 2024-04-23T10:16:03.064570+0000 2024-04-23T10:16:11.178 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: min_mon_release 19 (squid) 2024-04-23T10:16:11.178 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: election_strategy: 1 2024-04-23T10:16:11.178 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: 0: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.a 2024-04-23T10:16:11.178 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: fsmap 2024-04-23T10:16:11.178 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: osdmap e1: 0 total, 0 up, 0 in 2024-04-23T10:16:11.178 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:10 smithi097 ceph-mon[29197]: mgrmap e1: no daemons active 2024-04-23T10:16:12.017 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:11.384+0000 7f959c0c9700 1 Processor -- start 2024-04-23T10:16:12.017 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:11.384+0000 7f959c0c9700 1 -- start start 2024-04-23T10:16:12.017 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:11.385+0000 7f959c0c9700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9594108f80 0x7f959410b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:16:12.017 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:11.385+0000 7f959c0c9700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f959407be40 con 0x7f9594108f80 2024-04-23T10:16:12.017 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:11.385+0000 7f9599e65700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9594108f80 0x7f959410b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:16:12.017 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:11.385+0000 7f9599e65700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9594108f80 0x7f959410b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43238/0 (socket says 172.21.15.97:43238) 2024-04-23T10:16:12.017 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:11.385+0000 7f9599e65700 1 -- 172.21.15.97:0/1950449362 learned_addr learned my addr 172.21.15.97:0/1950449362 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:16:12.017 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:11.385+0000 7f9599e65700 1 -- 172.21.15.97:0/1950449362 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f959410b8b0 con 0x7f9594108f80 2024-04-23T10:16:12.018 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:11.385+0000 7f9599e65700 1 --2- 172.21.15.97:0/1950449362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9594108f80 0x7f959410b370 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f95900098a0 tx=0x7f9590009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc2b4141c6e91568 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:16:12.018 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:11.386+0000 7f9598e63700 1 -- 172.21.15.97:0/1950449362 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9590004160 con 0x7f9594108f80 2024-04-23T10:16:12.018 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:11.386+0000 7f9598e63700 1 -- 172.21.15.97:0/1950449362 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9590004300 con 0x7f9594108f80 2024-04-23T10:16:12.018 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:11.386+0000 7f9598e63700 1 -- 172.21.15.97:0/1950449362 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9590011af0 con 0x7f9594108f80 2024-04-23T10:16:12.018 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:11.386+0000 7f959c0c9700 1 -- 172.21.15.97:0/1950449362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9594108f80 msgr2=0x7f959410b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:16:12.018 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:11.386+0000 7f959c0c9700 1 --2- 172.21.15.97:0/1950449362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9594108f80 0x7f959410b370 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f95900098a0 tx=0x7f9590009bb0 comp rx=0 tx=0).stop 2024-04-23T10:16:12.018 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:11.386+0000 7f959c0c9700 1 -- 172.21.15.97:0/1950449362 shutdown_connections 2024-04-23T10:16:12.018 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:11.386+0000 7f959c0c9700 1 --2- 172.21.15.97:0/1950449362 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9594108f80 0x7f959410b370 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:16:12.018 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:11.386+0000 7f959c0c9700 1 -- 172.21.15.97:0/1950449362 >> 172.21.15.97:0/1950449362 conn(0x7f95941029b0 msgr2=0x7f9594104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:16:12.018 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:11.386+0000 7f959c0c9700 1 -- 172.21.15.97:0/1950449362 shutdown_connections 2024-04-23T10:16:12.018 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:11.386+0000 7f959c0c9700 1 -- 172.21.15.97:0/1950449362 wait complete. 2024-04-23T10:16:12.018 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:11.387+0000 7f959c0c9700 1 Processor -- start 2024-04-23T10:16:12.019 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:11.387+0000 7f959c0c9700 1 -- start start 2024-04-23T10:16:12.019 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:11.387+0000 7f959c0c9700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9594108f80 0x7f959419c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:16:12.019 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:11.387+0000 7f959c0c9700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f959410c810 con 0x7f9594108f80 2024-04-23T10:16:12.019 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:11.387+0000 7f9599e65700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9594108f80 0x7f959419c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:16:12.019 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:11.387+0000 7f9599e65700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9594108f80 0x7f959419c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43246/0 (socket says 172.21.15.97:43246) 2024-04-23T10:16:12.019 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:11.387+0000 7f9599e65700 1 -- 172.21.15.97:0/140606643 learned_addr learned my addr 172.21.15.97:0/140606643 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:16:12.019 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:11.387+0000 7f9599e65700 1 -- 172.21.15.97:0/140606643 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f959419c9c0 con 0x7f9594108f80 2024-04-23T10:16:12.019 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:11.387+0000 7f9599e65700 1 --2- 172.21.15.97:0/140606643 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9594108f80 0x7f959419c480 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9590009fb0 tx=0x7f9590011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:16:12.019 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:11.388+0000 7f958affd700 1 -- 172.21.15.97:0/140606643 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f95900036a0 con 0x7f9594108f80 2024-04-23T10:16:12.019 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:11.388+0000 7f958affd700 1 -- 172.21.15.97:0/140606643 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f959001a430 con 0x7f9594108f80 2024-04-23T10:16:12.019 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:11.388+0000 7f959c0c9700 1 -- 172.21.15.97:0/140606643 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f959419cc50 con 0x7f9594108f80 2024-04-23T10:16:12.019 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:11.388+0000 7f958affd700 1 -- 172.21.15.97:0/140606643 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9590019550 con 0x7f9594108f80 2024-04-23T10:16:12.019 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:11.388+0000 7f959c0c9700 1 -- 172.21.15.97:0/140606643 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f959419f940 con 0x7f9594108f80 2024-04-23T10:16:12.019 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:11.388+0000 7f958affd700 1 -- 172.21.15.97:0/140606643 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9590019b90 con 0x7f9594108f80 2024-04-23T10:16:12.019 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:11.388+0000 7f958affd700 1 -- 172.21.15.97:0/140606643 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9590024730 con 0x7f9594108f80 2024-04-23T10:16:12.020 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:11.389+0000 7f959c0c9700 1 -- 172.21.15.97:0/140606643 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95940640a0 con 0x7f9594108f80 2024-04-23T10:16:12.020 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:11.391+0000 7f958affd700 1 -- 172.21.15.97:0/140606643 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f9590043e50 con 0x7f9594108f80 2024-04-23T10:16:12.020 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:11.457+0000 7f959c0c9700 1 -- 172.21.15.97:0/140606643 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f959419fb10 con 0x7f9594108f80 2024-04-23T10:16:12.020 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:11.459+0000 7f958affd700 1 -- 172.21.15.97:0/140606643 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f9590032030 con 0x7f9594108f80 2024-04-23T10:16:12.020 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:11.460+0000 7f959c0c9700 1 -- 172.21.15.97:0/140606643 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9594108f80 msgr2=0x7f959419c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:16:12.020 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:11.460+0000 7f959c0c9700 1 --2- 172.21.15.97:0/140606643 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9594108f80 0x7f959419c480 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9590009fb0 tx=0x7f9590011940 comp rx=0 tx=0).stop 2024-04-23T10:16:12.020 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:11.460+0000 7f959c0c9700 1 -- 172.21.15.97:0/140606643 shutdown_connections 2024-04-23T10:16:12.020 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:11.460+0000 7f959c0c9700 1 --2- 172.21.15.97:0/140606643 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9594108f80 0x7f959419c480 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:16:12.020 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:11.460+0000 7f959c0c9700 1 -- 172.21.15.97:0/140606643 >> 172.21.15.97:0/140606643 conn(0x7f95941029b0 msgr2=0x7f9594103560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:16:12.020 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:11.460+0000 7f959c0c9700 1 -- 172.21.15.97:0/140606643 shutdown_connections 2024-04-23T10:16:12.020 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:11.460+0000 7f959c0c9700 1 -- 172.21.15.97:0/140606643 wait complete. 2024-04-23T10:16:12.020 INFO:teuthology.orchestra.run.smithi097.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-23T10:16:12.034 INFO:teuthology.orchestra.run.smithi097.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-23T10:16:12.034 INFO:teuthology.orchestra.run.smithi097.stdout:Creating mgr... 2024-04-23T10:16:12.035 INFO:teuthology.orchestra.run.smithi097.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-23T10:16:12.035 INFO:teuthology.orchestra.run.smithi097.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-23T10:16:12.280 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-9b22af88-0159-11ef-bc93-c7b262605968@mgr.x 2024-04-23T10:16:12.280 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stderr Failed to reset failed state of unit ceph-9b22af88-0159-11ef-bc93-c7b262605968@mgr.x.service: Unit ceph-9b22af88-0159-11ef-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-23T10:16:12.429 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-9b22af88-0159-11ef-bc93-c7b262605968.target.wants/ceph-9b22af88-0159-11ef-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-9b22af88-0159-11ef-bc93-c7b262605968@.service. 2024-04-23T10:16:12.447 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:12 smithi097 systemd[1]: Starting Ceph mgr.x for 9b22af88-0159-11ef-bc93-c7b262605968... 2024-04-23T10:16:12.759 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:12 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/140606643' entity='client.admin' 2024-04-23T10:16:12.759 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:12 smithi097 podman[29415]: 2024-04-23 10:16:12.663993441 +0000 UTC m=+0.113329851 container create 1f0c379183dda898c9277a28b5be4d44f9b6def4496432ea2a358dfe4af57094 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-23T10:16:12.759 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:12 smithi097 podman[29415]: 2024-04-23 10:16:12.569079931 +0000 UTC m=+0.018416311 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-23T10:16:12.911 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-23T10:16:12.911 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout disabled 2024-04-23T10:16:12.918 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-23T10:16:12.918 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout inactive 2024-04-23T10:16:12.918 INFO:teuthology.orchestra.run.smithi097.stdout:firewalld.service is not enabled 2024-04-23T10:16:12.918 INFO:teuthology.orchestra.run.smithi097.stdout:Not possible to enable service . firewalld.service is not available 2024-04-23T10:16:12.926 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-23T10:16:12.926 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout disabled 2024-04-23T10:16:12.933 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-23T10:16:12.933 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout inactive 2024-04-23T10:16:12.933 INFO:teuthology.orchestra.run.smithi097.stdout:firewalld.service is not enabled 2024-04-23T10:16:12.933 INFO:teuthology.orchestra.run.smithi097.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-23T10:16:12.933 INFO:teuthology.orchestra.run.smithi097.stdout:Waiting for mgr to start... 2024-04-23T10:16:12.933 INFO:teuthology.orchestra.run.smithi097.stdout:Waiting for mgr... 2024-04-23T10:16:13.058 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:12 smithi097 podman[29415]: 2024-04-23 10:16:12.841195228 +0000 UTC m=+0.290531596 container init 1f0c379183dda898c9277a28b5be4d44f9b6def4496432ea2a358dfe4af57094 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416) 2024-04-23T10:16:13.058 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:12 smithi097 podman[29415]: 2024-04-23 10:16:12.843675043 +0000 UTC m=+0.293011408 container start 1f0c379183dda898c9277a28b5be4d44f9b6def4496432ea2a358dfe4af57094 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-23T10:16:13.059 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:12 smithi097 bash[29415]: 1f0c379183dda898c9277a28b5be4d44f9b6def4496432ea2a358dfe4af57094 2024-04-23T10:16:13.059 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:12 smithi097 systemd[1]: Started Ceph mgr.x for 9b22af88-0159-11ef-bc93-c7b262605968. 2024-04-23T10:16:13.059 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:12 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:12.995+0000 7f3ce6813200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-23T10:16:13.361 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:13 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:13.182+0000 7f3ce6813200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-23T10:16:13.361 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:13 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:13.288+0000 7f3ce6813200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-23T10:16:13.696 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:13 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/2643087846' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-23T10:16:13.696 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:13 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:13.612+0000 7f3ce6813200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-23T10:16:13.696 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:13 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:13.696+0000 7f3ce6813200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-23T10:16:14.118 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2024-04-23T10:16:14.118 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout { 2024-04-23T10:16:14.118 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "fsid": "9b22af88-0159-11ef-bc93-c7b262605968", 2024-04-23T10:16:14.118 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "health": { 2024-04-23T10:16:14.118 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-23T10:16:14.118 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-23T10:16:14.118 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-23T10:16:14.118 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-04-23T10:16:14.118 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-23T10:16:14.118 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-23T10:16:14.119 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 0 2024-04-23T10:16:14.119 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2024-04-23T10:16:14.119 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-23T10:16:14.119 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "a" 2024-04-23T10:16:14.119 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2024-04-23T10:16:14.119 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-04-23T10:16:14.119 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-23T10:16:14.119 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T10:16:14.119 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-23T10:16:14.119 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-23T10:16:14.119 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-04-23T10:16:14.119 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-23T10:16:14.119 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T10:16:14.119 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-23T10:16:14.120 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-23T10:16:14.120 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-23T10:16:14.120 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-23T10:16:14.120 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-23T10:16:14.120 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-23T10:16:14.120 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-04-23T10:16:14.120 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-23T10:16:14.120 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-23T10:16:14.120 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-23T10:16:14.120 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-23T10:16:14.120 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-23T10:16:14.120 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-23T10:16:14.120 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-23T10:16:14.120 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-23T10:16:14.120 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-23T10:16:14.121 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-04-23T10:16:14.121 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-23T10:16:14.121 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T10:16:14.121 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-23T10:16:14.121 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-23T10:16:14.121 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-04-23T10:16:14.121 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-23T10:16:14.121 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-23T10:16:14.121 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-23T10:16:14.121 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-23T10:16:14.121 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-23T10:16:14.121 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-23T10:16:14.121 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "restful" 2024-04-23T10:16:14.121 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2024-04-23T10:16:14.121 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-23T10:16:14.122 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-04-23T10:16:14.122 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-23T10:16:14.122 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T10:16:14.122 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "modified": "2024-04-23T10:16:05.983765+0000", 2024-04-23T10:16:14.122 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-23T10:16:14.122 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-04-23T10:16:14.122 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-23T10:16:14.122 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout } 2024-04-23T10:16:14.122 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:13.404+0000 7f63af3dd700 1 Processor -- start 2024-04-23T10:16:14.122 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:13.404+0000 7f63af3dd700 1 -- start start 2024-04-23T10:16:14.122 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:13.404+0000 7f63af3dd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63a8108f70 0x7f63a810b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:16:14.122 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:13.404+0000 7f63af3dd700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f63a807bea0 con 0x7f63a8108f70 2024-04-23T10:16:14.122 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:13.404+0000 7f63ad179700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63a8108f70 0x7f63a810b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:16:14.123 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:13.404+0000 7f63ad179700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63a8108f70 0x7f63a810b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43266/0 (socket says 172.21.15.97:43266) 2024-04-23T10:16:14.123 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:13.404+0000 7f63ad179700 1 -- 172.21.15.97:0/1139274298 learned_addr learned my addr 172.21.15.97:0/1139274298 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:16:14.123 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:13.405+0000 7f63ad179700 1 -- 172.21.15.97:0/1139274298 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63a810b8a0 con 0x7f63a8108f70 2024-04-23T10:16:14.123 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:13.405+0000 7f63ad179700 1 --2- 172.21.15.97:0/1139274298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63a8108f70 0x7f63a810b360 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f63a40098a0 tx=0x7f63a4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fd5eb95eb65a6bb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:16:14.123 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:13.405+0000 7f639ffff700 1 -- 172.21.15.97:0/1139274298 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63a4004160 con 0x7f63a8108f70 2024-04-23T10:16:14.123 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:13.405+0000 7f639ffff700 1 -- 172.21.15.97:0/1139274298 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f63a4004300 con 0x7f63a8108f70 2024-04-23T10:16:14.123 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:13.405+0000 7f639ffff700 1 -- 172.21.15.97:0/1139274298 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63a4011af0 con 0x7f63a8108f70 2024-04-23T10:16:14.123 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:13.406+0000 7f63af3dd700 1 -- 172.21.15.97:0/1139274298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63a8108f70 msgr2=0x7f63a810b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:16:14.123 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:13.406+0000 7f63af3dd700 1 --2- 172.21.15.97:0/1139274298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63a8108f70 0x7f63a810b360 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f63a40098a0 tx=0x7f63a4009bb0 comp rx=0 tx=0).stop 2024-04-23T10:16:14.123 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:13.406+0000 7f63af3dd700 1 -- 172.21.15.97:0/1139274298 shutdown_connections 2024-04-23T10:16:14.123 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:13.406+0000 7f63af3dd700 1 --2- 172.21.15.97:0/1139274298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63a8108f70 0x7f63a810b360 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:16:14.123 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:13.406+0000 7f63af3dd700 1 -- 172.21.15.97:0/1139274298 >> 172.21.15.97:0/1139274298 conn(0x7f63a81029c0 msgr2=0x7f63a8104de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:16:14.123 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:13.406+0000 7f63af3dd700 1 -- 172.21.15.97:0/1139274298 shutdown_connections 2024-04-23T10:16:14.123 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:13.406+0000 7f63af3dd700 1 -- 172.21.15.97:0/1139274298 wait complete. 2024-04-23T10:16:14.124 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:13.406+0000 7f63af3dd700 1 Processor -- start 2024-04-23T10:16:14.124 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:13.406+0000 7f63af3dd700 1 -- start start 2024-04-23T10:16:14.124 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:13.407+0000 7f63af3dd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63a8108f70 0x7f63a819c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:16:14.124 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:13.407+0000 7f63af3dd700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f63a810c7f0 con 0x7f63a8108f70 2024-04-23T10:16:14.124 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:13.407+0000 7f63ad179700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63a8108f70 0x7f63a819c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:16:14.124 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:13.407+0000 7f63ad179700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63a8108f70 0x7f63a819c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43278/0 (socket says 172.21.15.97:43278) 2024-04-23T10:16:14.124 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:13.407+0000 7f63ad179700 1 -- 172.21.15.97:0/2643087846 learned_addr learned my addr 172.21.15.97:0/2643087846 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:16:14.124 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:13.407+0000 7f63ad179700 1 -- 172.21.15.97:0/2643087846 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63a819cc00 con 0x7f63a8108f70 2024-04-23T10:16:14.124 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:13.407+0000 7f63ad179700 1 --2- 172.21.15.97:0/2643087846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63a8108f70 0x7f63a819c6c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f63a4003f40 tx=0x7f63a4011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:16:14.124 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:13.407+0000 7f639e7fc700 1 -- 172.21.15.97:0/2643087846 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63a40038b0 con 0x7f63a8108f70 2024-04-23T10:16:14.124 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:13.407+0000 7f639e7fc700 1 -- 172.21.15.97:0/2643087846 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f63a4003a50 con 0x7f63a8108f70 2024-04-23T10:16:14.124 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:13.407+0000 7f63af3dd700 1 -- 172.21.15.97:0/2643087846 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63a819cef0 con 0x7f63a8108f70 2024-04-23T10:16:14.124 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:13.407+0000 7f63af3dd700 1 -- 172.21.15.97:0/2643087846 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63a819fb80 con 0x7f63a8108f70 2024-04-23T10:16:14.124 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:13.407+0000 7f639e7fc700 1 -- 172.21.15.97:0/2643087846 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63a401a5c0 con 0x7f63a8108f70 2024-04-23T10:16:14.124 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:13.408+0000 7f639e7fc700 1 -- 172.21.15.97:0/2643087846 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f63a401ac00 con 0x7f63a8108f70 2024-04-23T10:16:14.125 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:13.408+0000 7f639e7fc700 1 -- 172.21.15.97:0/2643087846 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f63a4019700 con 0x7f63a8108f70 2024-04-23T10:16:14.125 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:13.409+0000 7f63af3dd700 1 -- 172.21.15.97:0/2643087846 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63a80640a0 con 0x7f63a8108f70 2024-04-23T10:16:14.125 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:13.412+0000 7f639e7fc700 1 -- 172.21.15.97:0/2643087846 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f63a4032080 con 0x7f63a8108f70 2024-04-23T10:16:14.125 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:13.485+0000 7f63af3dd700 1 -- 172.21.15.97:0/2643087846 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f63a802df80 con 0x7f63a8108f70 2024-04-23T10:16:14.125 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:13.486+0000 7f639e7fc700 1 -- 172.21.15.97:0/2643087846 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f63a4019950 con 0x7f63a8108f70 2024-04-23T10:16:14.125 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:13.486+0000 7f63af3dd700 1 -- 172.21.15.97:0/2643087846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63a8108f70 msgr2=0x7f63a819c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:16:14.125 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:13.486+0000 7f63af3dd700 1 --2- 172.21.15.97:0/2643087846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63a8108f70 0x7f63a819c6c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f63a4003f40 tx=0x7f63a4011890 comp rx=0 tx=0).stop 2024-04-23T10:16:14.125 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:13.487+0000 7f63af3dd700 1 -- 172.21.15.97:0/2643087846 shutdown_connections 2024-04-23T10:16:14.125 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:13.487+0000 7f63af3dd700 1 --2- 172.21.15.97:0/2643087846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63a8108f70 0x7f63a819c6c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:16:14.125 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:13.487+0000 7f63af3dd700 1 -- 172.21.15.97:0/2643087846 >> 172.21.15.97:0/2643087846 conn(0x7f63a81029c0 msgr2=0x7f63a8103780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:16:14.125 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:13.487+0000 7f63af3dd700 1 -- 172.21.15.97:0/2643087846 shutdown_connections 2024-04-23T10:16:14.125 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:13.487+0000 7f63af3dd700 1 -- 172.21.15.97:0/2643087846 wait complete. 2024-04-23T10:16:14.125 INFO:teuthology.orchestra.run.smithi097.stdout:mgr not available, waiting (1/15)... 2024-04-23T10:16:15.233 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:14 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:14.898+0000 7f3ce6813200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-23T10:16:15.233 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:15 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:15.066+0000 7f3ce6813200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-23T10:16:15.233 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:15 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:15.233+0000 7f3ce6813200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-23T10:16:15.807 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:15 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:15.516+0000 7f3ce6813200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-23T10:16:15.807 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:15 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:15.607+0000 7f3ce6813200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-23T10:16:15.807 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:15 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:15.807+0000 7f3ce6813200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-23T10:16:16.111 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:16 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:15.999+0000 7f3ce6813200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-23T10:16:16.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:16 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/1303188531' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-23T10:16:16.861 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:16 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:16.616+0000 7f3ce6813200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-23T10:16:17.343 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:17 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:17.046+0000 7f3ce6813200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-23T10:16:17.343 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:17 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:17.143+0000 7f3ce6813200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-23T10:16:17.343 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:17 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:17.343+0000 7f3ce6813200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-23T10:16:17.462 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2024-04-23T10:16:17.462 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout { 2024-04-23T10:16:17.462 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "fsid": "9b22af88-0159-11ef-bc93-c7b262605968", 2024-04-23T10:16:17.462 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "health": { 2024-04-23T10:16:17.463 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-23T10:16:17.463 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-23T10:16:17.463 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-23T10:16:17.463 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-04-23T10:16:17.463 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-23T10:16:17.463 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-23T10:16:17.463 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 0 2024-04-23T10:16:17.463 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2024-04-23T10:16:17.463 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-23T10:16:17.463 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "a" 2024-04-23T10:16:17.464 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2024-04-23T10:16:17.464 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-04-23T10:16:17.464 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-23T10:16:17.464 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T10:16:17.464 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-23T10:16:17.464 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-23T10:16:17.464 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-04-23T10:16:17.464 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-23T10:16:17.464 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T10:16:17.464 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-23T10:16:17.465 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-23T10:16:17.465 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-23T10:16:17.465 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-23T10:16:17.465 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-23T10:16:17.465 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-23T10:16:17.465 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-04-23T10:16:17.465 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-23T10:16:17.465 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-23T10:16:17.465 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-23T10:16:17.465 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-23T10:16:17.466 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-23T10:16:17.466 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-23T10:16:17.466 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-23T10:16:17.466 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-23T10:16:17.466 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-23T10:16:17.466 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-04-23T10:16:17.466 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-23T10:16:17.466 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T10:16:17.466 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-23T10:16:17.466 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-23T10:16:17.467 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-04-23T10:16:17.467 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-23T10:16:17.467 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-23T10:16:17.467 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-23T10:16:17.467 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-23T10:16:17.467 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-23T10:16:17.467 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-23T10:16:17.467 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "restful" 2024-04-23T10:16:17.467 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2024-04-23T10:16:17.467 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-23T10:16:17.468 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-04-23T10:16:17.468 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-23T10:16:17.468 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T10:16:17.468 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "modified": "2024-04-23T10:16:05.983765+0000", 2024-04-23T10:16:17.468 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-23T10:16:17.468 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-04-23T10:16:17.468 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-23T10:16:17.468 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout } 2024-04-23T10:16:17.468 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:16.561+0000 7ffbc2858700 1 Processor -- start 2024-04-23T10:16:17.469 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:16.561+0000 7ffbc2858700 1 -- start start 2024-04-23T10:16:17.469 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:16.562+0000 7ffbc2858700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffbbc106b50 0x7ffbbc106f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:16:17.469 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:16.562+0000 7ffbc2858700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ffbbc107540 con 0x7ffbbc106b50 2024-04-23T10:16:17.469 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:16.562+0000 7ffbbbfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffbbc106b50 0x7ffbbc106f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:16:17.469 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:16.562+0000 7ffbbbfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffbbc106b50 0x7ffbbc106f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33140/0 (socket says 172.21.15.97:33140) 2024-04-23T10:16:17.469 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:16.562+0000 7ffbbbfff700 1 -- 172.21.15.97:0/2382316695 learned_addr learned my addr 172.21.15.97:0/2382316695 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:16:17.469 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:16.562+0000 7ffbbbfff700 1 -- 172.21.15.97:0/2382316695 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbbc107d50 con 0x7ffbbc106b50 2024-04-23T10:16:17.469 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:16.562+0000 7ffbbbfff700 1 --2- 172.21.15.97:0/2382316695 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffbbc106b50 0x7ffbbc106f70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ffbac00bb40 tx=0x7ffbac00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b4d2e2e42f33dba server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:16:17.469 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:16.562+0000 7ffbbaffd700 1 -- 172.21.15.97:0/2382316695 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffbac00dad0 con 0x7ffbbc106b50 2024-04-23T10:16:17.469 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:16.562+0000 7ffbbaffd700 1 -- 172.21.15.97:0/2382316695 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ffbac0044c0 con 0x7ffbbc106b50 2024-04-23T10:16:17.470 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:16.562+0000 7ffbbaffd700 1 -- 172.21.15.97:0/2382316695 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffbac003de0 con 0x7ffbbc106b50 2024-04-23T10:16:17.470 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:16.563+0000 7ffbc2858700 1 -- 172.21.15.97:0/2382316695 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffbbc106b50 msgr2=0x7ffbbc106f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:16:17.470 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:16.563+0000 7ffbc2858700 1 --2- 172.21.15.97:0/2382316695 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffbbc106b50 0x7ffbbc106f70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ffbac00bb40 tx=0x7ffbac00be50 comp rx=0 tx=0).stop 2024-04-23T10:16:17.470 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:16.563+0000 7ffbc2858700 1 -- 172.21.15.97:0/2382316695 shutdown_connections 2024-04-23T10:16:17.470 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:16.563+0000 7ffbc2858700 1 --2- 172.21.15.97:0/2382316695 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffbbc106b50 0x7ffbbc106f70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:16:17.470 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:16.563+0000 7ffbc2858700 1 -- 172.21.15.97:0/2382316695 >> 172.21.15.97:0/2382316695 conn(0x7ffbbc1021b0 msgr2=0x7ffbbc1045d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:16:17.470 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:16.563+0000 7ffbc2858700 1 -- 172.21.15.97:0/2382316695 shutdown_connections 2024-04-23T10:16:17.470 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:16.563+0000 7ffbc2858700 1 -- 172.21.15.97:0/2382316695 wait complete. 2024-04-23T10:16:17.470 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:16.564+0000 7ffbc2858700 1 Processor -- start 2024-04-23T10:16:17.471 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:16.564+0000 7ffbc2858700 1 -- start start 2024-04-23T10:16:17.471 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:16.564+0000 7ffbc2858700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffbbc106b50 0x7ffbbc19a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:16:17.471 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:16.564+0000 7ffbc2858700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ffbbc108470 con 0x7ffbbc106b50 2024-04-23T10:16:17.471 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:16.564+0000 7ffbbbfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffbbc106b50 0x7ffbbc19a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:16:17.471 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:16.564+0000 7ffbbbfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffbbc106b50 0x7ffbbc19a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33150/0 (socket says 172.21.15.97:33150) 2024-04-23T10:16:17.471 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:16.564+0000 7ffbbbfff700 1 -- 172.21.15.97:0/1303188531 learned_addr learned my addr 172.21.15.97:0/1303188531 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:16:17.471 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:16.565+0000 7ffbbbfff700 1 -- 172.21.15.97:0/1303188531 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbbc19aaa0 con 0x7ffbbc106b50 2024-04-23T10:16:17.471 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:16.565+0000 7ffbbbfff700 1 --2- 172.21.15.97:0/1303188531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffbbc106b50 0x7ffbbc19a560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ffbac019040 tx=0x7ffbac003ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:16:17.471 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:16.565+0000 7ffbb97fa700 1 -- 172.21.15.97:0/1303188531 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffbac025020 con 0x7ffbbc106b50 2024-04-23T10:16:17.471 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:16.565+0000 7ffbc2858700 1 -- 172.21.15.97:0/1303188531 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbbc19ad30 con 0x7ffbbc106b50 2024-04-23T10:16:17.472 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:16.565+0000 7ffbb97fa700 1 -- 172.21.15.97:0/1303188531 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ffbac0179d0 con 0x7ffbbc106b50 2024-04-23T10:16:17.472 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:16.565+0000 7ffbc2858700 1 -- 172.21.15.97:0/1303188531 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbbc19da20 con 0x7ffbbc106b50 2024-04-23T10:16:17.472 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:16.565+0000 7ffbb97fa700 1 -- 172.21.15.97:0/1303188531 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffbac021870 con 0x7ffbbc106b50 2024-04-23T10:16:17.472 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:16.565+0000 7ffbb97fa700 1 -- 172.21.15.97:0/1303188531 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ffbac028020 con 0x7ffbbc106b50 2024-04-23T10:16:17.472 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:16.565+0000 7ffbb97fa700 1 -- 172.21.15.97:0/1303188531 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ffbac02ab60 con 0x7ffbbc106b50 2024-04-23T10:16:17.472 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:16.566+0000 7ffbc2858700 1 -- 172.21.15.97:0/1303188531 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffba8005360 con 0x7ffbbc106b50 2024-04-23T10:16:17.472 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:16.568+0000 7ffbb97fa700 1 -- 172.21.15.97:0/1303188531 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7ffbac02f020 con 0x7ffbbc106b50 2024-04-23T10:16:17.472 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:16.640+0000 7ffbc2858700 1 -- 172.21.15.97:0/1303188531 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7ffba8005920 con 0x7ffbbc106b50 2024-04-23T10:16:17.472 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:16.640+0000 7ffbb97fa700 1 -- 172.21.15.97:0/1303188531 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7ffbac012070 con 0x7ffbbc106b50 2024-04-23T10:16:17.473 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:16.641+0000 7ffbc2858700 1 -- 172.21.15.97:0/1303188531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffbbc106b50 msgr2=0x7ffbbc19a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:16:17.473 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:16.641+0000 7ffbc2858700 1 --2- 172.21.15.97:0/1303188531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffbbc106b50 0x7ffbbc19a560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ffbac019040 tx=0x7ffbac003ee0 comp rx=0 tx=0).stop 2024-04-23T10:16:17.473 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:16.641+0000 7ffbc2858700 1 -- 172.21.15.97:0/1303188531 shutdown_connections 2024-04-23T10:16:17.473 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:16.641+0000 7ffbc2858700 1 --2- 172.21.15.97:0/1303188531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffbbc106b50 0x7ffbbc19a560 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:16:17.473 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:16.641+0000 7ffbc2858700 1 -- 172.21.15.97:0/1303188531 >> 172.21.15.97:0/1303188531 conn(0x7ffbbc1021b0 msgr2=0x7ffbbc190eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:16:17.473 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:16.641+0000 7ffbc2858700 1 -- 172.21.15.97:0/1303188531 shutdown_connections 2024-04-23T10:16:17.473 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:16.641+0000 7ffbc2858700 1 -- 172.21.15.97:0/1303188531 wait complete. 2024-04-23T10:16:17.473 INFO:teuthology.orchestra.run.smithi097.stdout:mgr not available, waiting (2/15)... 2024-04-23T10:16:18.195 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:17 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:17.866+0000 7f3ce6813200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-23T10:16:18.195 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:17 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:17.957+0000 7f3ce6813200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-23T10:16:18.534 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:18 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:18.196+0000 7f3ce6813200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-23T10:16:18.534 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:18 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:18.324+0000 7f3ce6813200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-23T10:16:18.535 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:18 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:18.418+0000 7f3ce6813200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-23T10:16:18.535 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:18 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:18.534+0000 7f3ce6813200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-23T10:16:19.111 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:18 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:18.833+0000 7f3ce6813200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-23T10:16:19.705 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:19 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:19.425+0000 7f3ce6813200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-23T10:16:19.705 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:19 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:19.650+0000 7f3ce6813200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-23T10:16:19.998 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:19 smithi097 ceph-mon[29197]: Activating manager daemon x 2024-04-23T10:16:19.998 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:19 smithi097 ceph-mon[29197]: mgrmap e2: x(active, starting, since 0.00315442s) 2024-04-23T10:16:19.999 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:19 smithi097 ceph-mon[29197]: from='mgr.14100 172.21.15.97:0/2664673218' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-23T10:16:19.999 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:19 smithi097 ceph-mon[29197]: from='mgr.14100 172.21.15.97:0/2664673218' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-23T10:16:19.999 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:19 smithi097 ceph-mon[29197]: from='mgr.14100 172.21.15.97:0/2664673218' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-23T10:16:19.999 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:19 smithi097 ceph-mon[29197]: from='mgr.14100 172.21.15.97:0/2664673218' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T10:16:19.999 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:19 smithi097 ceph-mon[29197]: from='mgr.14100 172.21.15.97:0/2664673218' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-23T10:16:19.999 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:19 smithi097 ceph-mon[29197]: Manager daemon x is now available 2024-04-23T10:16:19.999 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:19 smithi097 ceph-mon[29197]: from='mgr.14100 172.21.15.97:0/2664673218' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-23T10:16:19.999 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:19 smithi097 ceph-mon[29197]: from='mgr.14100 172.21.15.97:0/2664673218' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-23T10:16:19.999 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:19 smithi097 ceph-mon[29197]: from='mgr.14100 172.21.15.97:0/2664673218' entity='mgr.x' 2024-04-23T10:16:19.999 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:19 smithi097 ceph-mon[29197]: from='mgr.14100 172.21.15.97:0/2664673218' entity='mgr.x' 2024-04-23T10:16:19.999 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:19 smithi097 ceph-mon[29197]: from='mgr.14100 172.21.15.97:0/2664673218' entity='mgr.x' 2024-04-23T10:16:20.843 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:20 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/1878372434' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-23T10:16:20.844 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:20 smithi097 ceph-mon[29197]: mgrmap e3: x(active, since 1.00615s) 2024-04-23T10:16:20.849 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2024-04-23T10:16:20.849 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout { 2024-04-23T10:16:20.849 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "fsid": "9b22af88-0159-11ef-bc93-c7b262605968", 2024-04-23T10:16:20.849 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "health": { 2024-04-23T10:16:20.850 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-23T10:16:20.850 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-23T10:16:20.850 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-23T10:16:20.850 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-04-23T10:16:20.850 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-23T10:16:20.850 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-23T10:16:20.850 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 0 2024-04-23T10:16:20.850 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2024-04-23T10:16:20.850 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-23T10:16:20.850 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "a" 2024-04-23T10:16:20.851 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2024-04-23T10:16:20.851 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-04-23T10:16:20.851 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-23T10:16:20.851 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T10:16:20.851 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-23T10:16:20.851 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-23T10:16:20.851 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-04-23T10:16:20.851 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-23T10:16:20.851 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T10:16:20.851 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-23T10:16:20.852 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-23T10:16:20.852 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-23T10:16:20.852 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-23T10:16:20.862 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-23T10:16:20.862 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-23T10:16:20.862 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-04-23T10:16:20.862 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-23T10:16:20.862 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-23T10:16:20.862 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-23T10:16:20.862 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-23T10:16:20.863 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-23T10:16:20.863 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-23T10:16:20.863 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-23T10:16:20.863 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-23T10:16:20.863 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-23T10:16:20.863 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-04-23T10:16:20.863 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-23T10:16:20.863 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T10:16:20.863 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-23T10:16:20.863 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-23T10:16:20.863 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-04-23T10:16:20.864 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-23T10:16:20.864 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-23T10:16:20.864 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-23T10:16:20.864 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-23T10:16:20.864 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-23T10:16:20.864 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-23T10:16:20.864 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "restful" 2024-04-23T10:16:20.864 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2024-04-23T10:16:20.864 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-23T10:16:20.864 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-04-23T10:16:20.864 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-23T10:16:20.865 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T10:16:20.865 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "modified": "2024-04-23T10:16:05.983765+0000", 2024-04-23T10:16:20.865 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-23T10:16:20.865 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-04-23T10:16:20.865 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-23T10:16:20.865 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout } 2024-04-23T10:16:20.865 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:20.177+0000 7f0aa6701700 1 Processor -- start 2024-04-23T10:16:20.865 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:20.177+0000 7f0aa6701700 1 -- start start 2024-04-23T10:16:20.865 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:20.177+0000 7f0aa6701700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0aa0108510 0x7f0aa0108930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:16:20.865 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:20.177+0000 7f0aa6701700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa0108f00 con 0x7f0aa0108510 2024-04-23T10:16:20.865 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:20.177+0000 7f0a9ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0aa0108510 0x7f0aa0108930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:16:20.866 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:20.177+0000 7f0a9ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0aa0108510 0x7f0aa0108930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33218/0 (socket says 172.21.15.97:33218) 2024-04-23T10:16:20.866 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:20.177+0000 7f0a9ffff700 1 -- 172.21.15.97:0/78129710 learned_addr learned my addr 172.21.15.97:0/78129710 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:16:20.866 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:20.177+0000 7f0a9ffff700 1 -- 172.21.15.97:0/78129710 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0aa01096c0 con 0x7f0aa0108510 2024-04-23T10:16:20.866 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:20.178+0000 7f0a9ffff700 1 --2- 172.21.15.97:0/78129710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0aa0108510 0x7f0aa0108930 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0a900098a0 tx=0x7f0a90009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dc9aa3fce6c63e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:16:20.866 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:20.178+0000 7f0a9effd700 1 -- 172.21.15.97:0/78129710 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a90004160 con 0x7f0aa0108510 2024-04-23T10:16:20.866 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:20.178+0000 7f0a9effd700 1 -- 172.21.15.97:0/78129710 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0a90004300 con 0x7f0aa0108510 2024-04-23T10:16:20.866 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:20.178+0000 7f0a9effd700 1 -- 172.21.15.97:0/78129710 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a90011af0 con 0x7f0aa0108510 2024-04-23T10:16:20.866 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:20.178+0000 7f0aa6701700 1 -- 172.21.15.97:0/78129710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0aa0108510 msgr2=0x7f0aa0108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:16:20.866 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:20.178+0000 7f0aa6701700 1 --2- 172.21.15.97:0/78129710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0aa0108510 0x7f0aa0108930 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0a900098a0 tx=0x7f0a90009bb0 comp rx=0 tx=0).stop 2024-04-23T10:16:20.866 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:20.178+0000 7f0aa6701700 1 -- 172.21.15.97:0/78129710 shutdown_connections 2024-04-23T10:16:20.867 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:20.178+0000 7f0aa6701700 1 --2- 172.21.15.97:0/78129710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0aa0108510 0x7f0aa0108930 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:16:20.867 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:20.178+0000 7f0aa6701700 1 -- 172.21.15.97:0/78129710 >> 172.21.15.97:0/78129710 conn(0x7f0aa0103b70 msgr2=0x7f0aa0105f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:16:20.867 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:20.178+0000 7f0aa6701700 1 -- 172.21.15.97:0/78129710 shutdown_connections 2024-04-23T10:16:20.867 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:20.178+0000 7f0aa6701700 1 -- 172.21.15.97:0/78129710 wait complete. 2024-04-23T10:16:20.867 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:20.179+0000 7f0aa6701700 1 Processor -- start 2024-04-23T10:16:20.867 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:20.179+0000 7f0aa6701700 1 -- start start 2024-04-23T10:16:20.867 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:20.179+0000 7f0aa6701700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0aa0108510 0x7f0aa019c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:16:20.867 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:20.179+0000 7f0aa6701700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa010ca70 con 0x7f0aa0108510 2024-04-23T10:16:20.867 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:20.179+0000 7f0a9ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0aa0108510 0x7f0aa019c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:16:20.867 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:20.179+0000 7f0a9ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0aa0108510 0x7f0aa019c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33220/0 (socket says 172.21.15.97:33220) 2024-04-23T10:16:20.867 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:20.179+0000 7f0a9ffff700 1 -- 172.21.15.97:0/1878372434 learned_addr learned my addr 172.21.15.97:0/1878372434 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:16:20.868 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:20.180+0000 7f0a9ffff700 1 -- 172.21.15.97:0/1878372434 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0aa019ccc0 con 0x7f0aa0108510 2024-04-23T10:16:20.868 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:20.180+0000 7f0a9ffff700 1 --2- 172.21.15.97:0/1878372434 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0aa0108510 0x7f0aa019c780 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0a90003f40 tx=0x7f0a90011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:16:20.868 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:20.180+0000 7f0a9d7fa700 1 -- 172.21.15.97:0/1878372434 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a900038b0 con 0x7f0aa0108510 2024-04-23T10:16:20.868 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:20.180+0000 7f0a9d7fa700 1 -- 172.21.15.97:0/1878372434 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0a90003a50 con 0x7f0aa0108510 2024-04-23T10:16:20.868 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:20.180+0000 7f0aa6701700 1 -- 172.21.15.97:0/1878372434 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0aa019cf50 con 0x7f0aa0108510 2024-04-23T10:16:20.868 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:20.180+0000 7f0a9d7fa700 1 -- 172.21.15.97:0/1878372434 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a9001a5c0 con 0x7f0aa0108510 2024-04-23T10:16:20.868 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:20.180+0000 7f0aa6701700 1 -- 172.21.15.97:0/1878372434 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0aa019fbf0 con 0x7f0aa0108510 2024-04-23T10:16:20.868 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:20.181+0000 7f0a9d7fa700 1 -- 172.21.15.97:0/1878372434 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 2) ==== 46314+0+0 (secure 0 0 0) 0x7f0a9001a8c0 con 0x7f0aa0108510 2024-04-23T10:16:20.868 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:20.181+0000 7f0aa6701700 1 -- 172.21.15.97:0/1878372434 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0aa00640a0 con 0x7f0aa0108510 2024-04-23T10:16:20.868 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:20.181+0000 7f0a9d7fa700 1 -- 172.21.15.97:0/1878372434 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0a9004d6b0 con 0x7f0aa0108510 2024-04-23T10:16:20.868 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:20.183+0000 7f0a9d7fa700 1 -- 172.21.15.97:0/1878372434 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f0a90019c70 con 0x7f0aa0108510 2024-04-23T10:16:20.869 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:20.253+0000 7f0aa6701700 1 -- 172.21.15.97:0/1878372434 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f0aa002df80 con 0x7f0aa0108510 2024-04-23T10:16:20.869 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:20.253+0000 7f0a9d7fa700 1 -- 172.21.15.97:0/1878372434 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f0a9001abb0 con 0x7f0aa0108510 2024-04-23T10:16:20.869 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:20.254+0000 7f0aa6701700 1 -- 172.21.15.97:0/1878372434 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0aa0108510 msgr2=0x7f0aa019c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:16:20.869 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:20.254+0000 7f0aa6701700 1 --2- 172.21.15.97:0/1878372434 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0aa0108510 0x7f0aa019c780 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0a90003f40 tx=0x7f0a90011890 comp rx=0 tx=0).stop 2024-04-23T10:16:20.869 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:20.255+0000 7f0aa6701700 1 -- 172.21.15.97:0/1878372434 shutdown_connections 2024-04-23T10:16:20.869 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:20.255+0000 7f0aa6701700 1 --2- 172.21.15.97:0/1878372434 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0aa0108510 0x7f0aa019c780 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:16:20.869 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:20.255+0000 7f0aa6701700 1 -- 172.21.15.97:0/1878372434 >> 172.21.15.97:0/1878372434 conn(0x7f0aa0103b70 msgr2=0x7f0aa01048a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:16:20.869 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:20.255+0000 7f0aa6701700 1 -- 172.21.15.97:0/1878372434 shutdown_connections 2024-04-23T10:16:20.869 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:20.255+0000 7f0aa6701700 1 -- 172.21.15.97:0/1878372434 wait complete. 2024-04-23T10:16:20.869 INFO:teuthology.orchestra.run.smithi097.stdout:mgr not available, waiting (3/15)... 2024-04-23T10:16:22.979 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:22 smithi097 ceph-mon[29197]: mgrmap e4: x(active, since 2s) 2024-04-23T10:16:24.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:23 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/2974069258' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-23T10:16:24.414 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2024-04-23T10:16:24.414 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout { 2024-04-23T10:16:24.414 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "fsid": "9b22af88-0159-11ef-bc93-c7b262605968", 2024-04-23T10:16:24.414 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "health": { 2024-04-23T10:16:24.414 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-23T10:16:24.414 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-23T10:16:24.415 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-23T10:16:24.415 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-04-23T10:16:24.415 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-23T10:16:24.415 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-23T10:16:24.415 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 0 2024-04-23T10:16:24.415 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2024-04-23T10:16:24.415 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-23T10:16:24.415 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "a" 2024-04-23T10:16:24.415 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2024-04-23T10:16:24.415 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-04-23T10:16:24.416 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-23T10:16:24.416 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T10:16:24.416 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-23T10:16:24.416 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-23T10:16:24.416 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-04-23T10:16:24.416 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-23T10:16:24.416 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T10:16:24.416 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-23T10:16:24.416 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-23T10:16:24.416 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-23T10:16:24.427 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-23T10:16:24.427 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-23T10:16:24.427 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-23T10:16:24.427 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-04-23T10:16:24.428 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-23T10:16:24.428 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-23T10:16:24.428 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-23T10:16:24.428 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-23T10:16:24.428 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-23T10:16:24.428 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-23T10:16:24.428 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-23T10:16:24.428 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-23T10:16:24.428 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-23T10:16:24.428 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-04-23T10:16:24.429 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-23T10:16:24.429 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T10:16:24.429 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-23T10:16:24.429 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-23T10:16:24.429 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-04-23T10:16:24.429 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-23T10:16:24.429 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-23T10:16:24.429 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-23T10:16:24.429 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-23T10:16:24.429 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-23T10:16:24.429 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-23T10:16:24.430 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "restful" 2024-04-23T10:16:24.430 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ], 2024-04-23T10:16:24.430 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-23T10:16:24.430 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-04-23T10:16:24.430 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-23T10:16:24.430 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T10:16:24.430 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "modified": "2024-04-23T10:16:05.983765+0000", 2024-04-23T10:16:24.430 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-23T10:16:24.430 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout }, 2024-04-23T10:16:24.430 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-23T10:16:24.430 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout } 2024-04-23T10:16:24.431 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:23.528+0000 7f854013b700 1 Processor -- start 2024-04-23T10:16:24.431 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:23.528+0000 7f854013b700 1 -- start start 2024-04-23T10:16:24.431 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:23.528+0000 7f854013b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8538108510 0x7f8538108930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:16:24.431 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:23.528+0000 7f854013b700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8538108f00 con 0x7f8538108510 2024-04-23T10:16:24.431 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:23.528+0000 7f853ded7700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8538108510 0x7f8538108930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:16:24.431 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:23.528+0000 7f853ded7700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8538108510 0x7f8538108930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33222/0 (socket says 172.21.15.97:33222) 2024-04-23T10:16:24.431 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:23.528+0000 7f853ded7700 1 -- 172.21.15.97:0/1957123345 learned_addr learned my addr 172.21.15.97:0/1957123345 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:16:24.431 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:23.528+0000 7f853ded7700 1 -- 172.21.15.97:0/1957123345 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85381096c0 con 0x7f8538108510 2024-04-23T10:16:24.431 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:23.529+0000 7f853ded7700 1 --2- 172.21.15.97:0/1957123345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8538108510 0x7f8538108930 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f85340098a0 tx=0x7f8534009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=219378e72b7fcd83 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:16:24.431 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:23.529+0000 7f853ced5700 1 -- 172.21.15.97:0/1957123345 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8534004160 con 0x7f8538108510 2024-04-23T10:16:24.431 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:23.529+0000 7f853ced5700 1 -- 172.21.15.97:0/1957123345 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8534004300 con 0x7f8538108510 2024-04-23T10:16:24.432 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:23.529+0000 7f853ced5700 1 -- 172.21.15.97:0/1957123345 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8534011af0 con 0x7f8538108510 2024-04-23T10:16:24.432 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:23.529+0000 7f854013b700 1 -- 172.21.15.97:0/1957123345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8538108510 msgr2=0x7f8538108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:16:24.432 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:23.529+0000 7f854013b700 1 --2- 172.21.15.97:0/1957123345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8538108510 0x7f8538108930 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f85340098a0 tx=0x7f8534009bb0 comp rx=0 tx=0).stop 2024-04-23T10:16:24.432 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:23.529+0000 7f854013b700 1 -- 172.21.15.97:0/1957123345 shutdown_connections 2024-04-23T10:16:24.432 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:23.529+0000 7f854013b700 1 --2- 172.21.15.97:0/1957123345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8538108510 0x7f8538108930 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:16:24.432 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:23.529+0000 7f854013b700 1 -- 172.21.15.97:0/1957123345 >> 172.21.15.97:0/1957123345 conn(0x7f8538103b70 msgr2=0x7f8538105f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:16:24.432 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:23.530+0000 7f854013b700 1 -- 172.21.15.97:0/1957123345 shutdown_connections 2024-04-23T10:16:24.432 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:23.530+0000 7f854013b700 1 -- 172.21.15.97:0/1957123345 wait complete. 2024-04-23T10:16:24.432 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:23.530+0000 7f854013b700 1 Processor -- start 2024-04-23T10:16:24.432 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:23.530+0000 7f854013b700 1 -- start start 2024-04-23T10:16:24.432 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:23.530+0000 7f854013b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8538108510 0x7f853807db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:16:24.432 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:23.530+0000 7f854013b700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f853810ca70 con 0x7f8538108510 2024-04-23T10:16:24.433 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:23.530+0000 7f853ded7700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8538108510 0x7f853807db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:16:24.433 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:23.530+0000 7f853ded7700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8538108510 0x7f853807db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33232/0 (socket says 172.21.15.97:33232) 2024-04-23T10:16:24.433 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:23.530+0000 7f853ded7700 1 -- 172.21.15.97:0/2974069258 learned_addr learned my addr 172.21.15.97:0/2974069258 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:16:24.433 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:23.531+0000 7f853ded7700 1 -- 172.21.15.97:0/2974069258 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f853807e070 con 0x7f8538108510 2024-04-23T10:16:24.433 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:23.531+0000 7f853ded7700 1 --2- 172.21.15.97:0/2974069258 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8538108510 0x7f853807db30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8534003f40 tx=0x7f8534011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:16:24.433 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:23.531+0000 7f852effd700 1 -- 172.21.15.97:0/2974069258 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85340038b0 con 0x7f8538108510 2024-04-23T10:16:24.433 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:23.531+0000 7f852effd700 1 -- 172.21.15.97:0/2974069258 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8534003a50 con 0x7f8538108510 2024-04-23T10:16:24.433 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:23.531+0000 7f852effd700 1 -- 172.21.15.97:0/2974069258 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f853401a5c0 con 0x7f8538108510 2024-04-23T10:16:24.433 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:23.531+0000 7f854013b700 1 -- 172.21.15.97:0/2974069258 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f853807c2b0 con 0x7f8538108510 2024-04-23T10:16:24.433 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:23.531+0000 7f854013b700 1 -- 172.21.15.97:0/2974069258 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f853807c790 con 0x7f8538108510 2024-04-23T10:16:24.434 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:23.532+0000 7f852effd700 1 -- 172.21.15.97:0/2974069258 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f8534024430 con 0x7f8538108510 2024-04-23T10:16:24.434 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:23.532+0000 7f854013b700 1 -- 172.21.15.97:0/2974069258 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85380640a0 con 0x7f8538108510 2024-04-23T10:16:24.434 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:23.532+0000 7f852effd700 1 --2- 172.21.15.97:0/2974069258 >> [v2:172.21.15.97:6800/3706463017,v1:172.21.15.97:6801/3706463017] conn(0x7f8524039560 0x7f852403ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:16:24.434 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:23.532+0000 7f852effd700 1 -- 172.21.15.97:0/2974069258 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f853404da60 con 0x7f8538108510 2024-04-23T10:16:24.434 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:23.532+0000 7f853d6d6700 1 --2- 172.21.15.97:0/2974069258 >> [v2:172.21.15.97:6800/3706463017,v1:172.21.15.97:6801/3706463017] conn(0x7f8524039560 0x7f852403ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:16:24.434 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:23.533+0000 7f853d6d6700 1 --2- 172.21.15.97:0/2974069258 >> [v2:172.21.15.97:6800/3706463017,v1:172.21.15.97:6801/3706463017] conn(0x7f8524039560 0x7f852403ba20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8528006fd0 tx=0x7f8528006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:16:24.434 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:23.538+0000 7f852effd700 1 -- 172.21.15.97:0/2974069258 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f853404d480 con 0x7f8538108510 2024-04-23T10:16:24.434 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:23.816+0000 7f854013b700 1 -- 172.21.15.97:0/2974069258 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f8538195e80 con 0x7f8538108510 2024-04-23T10:16:24.434 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:23.817+0000 7f852effd700 1 -- 172.21.15.97:0/2974069258 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f853402d910 con 0x7f8538108510 2024-04-23T10:16:24.434 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:23.819+0000 7f854013b700 1 -- 172.21.15.97:0/2974069258 >> [v2:172.21.15.97:6800/3706463017,v1:172.21.15.97:6801/3706463017] conn(0x7f8524039560 msgr2=0x7f852403ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:16:24.435 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:23.819+0000 7f854013b700 1 --2- 172.21.15.97:0/2974069258 >> [v2:172.21.15.97:6800/3706463017,v1:172.21.15.97:6801/3706463017] conn(0x7f8524039560 0x7f852403ba20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8528006fd0 tx=0x7f8528006e40 comp rx=0 tx=0).stop 2024-04-23T10:16:24.435 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:23.819+0000 7f854013b700 1 -- 172.21.15.97:0/2974069258 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8538108510 msgr2=0x7f853807db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:16:24.435 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:23.819+0000 7f854013b700 1 --2- 172.21.15.97:0/2974069258 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8538108510 0x7f853807db30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8534003f40 tx=0x7f8534011890 comp rx=0 tx=0).stop 2024-04-23T10:16:24.435 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:23.819+0000 7f854013b700 1 -- 172.21.15.97:0/2974069258 shutdown_connections 2024-04-23T10:16:24.435 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:23.819+0000 7f854013b700 1 --2- 172.21.15.97:0/2974069258 >> [v2:172.21.15.97:6800/3706463017,v1:172.21.15.97:6801/3706463017] conn(0x7f8524039560 0x7f852403ba20 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:16:24.435 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:23.819+0000 7f854013b700 1 --2- 172.21.15.97:0/2974069258 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8538108510 0x7f853807db30 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:16:24.435 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:23.819+0000 7f854013b700 1 -- 172.21.15.97:0/2974069258 >> 172.21.15.97:0/2974069258 conn(0x7f8538103b70 msgr2=0x7f85381048a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:16:24.435 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:23.819+0000 7f854013b700 1 -- 172.21.15.97:0/2974069258 shutdown_connections 2024-04-23T10:16:24.435 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:23.819+0000 7f854013b700 1 -- 172.21.15.97:0/2974069258 wait complete. 2024-04-23T10:16:24.435 INFO:teuthology.orchestra.run.smithi097.stdout:mgr is available 2024-04-23T10:16:25.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:25 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/2582877608' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-23T10:16:25.575 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2024-04-23T10:16:25.575 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout [global] 2024-04-23T10:16:25.575 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout fsid = 9b22af88-0159-11ef-bc93-c7b262605968 2024-04-23T10:16:25.575 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-23T10:16:25.575 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-23T10:16:25.575 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-23T10:16:25.575 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-23T10:16:25.575 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-23T10:16:25.575 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2024-04-23T10:16:25.576 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-23T10:16:25.576 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-23T10:16:25.576 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 2024-04-23T10:16:25.576 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout [osd] 2024-04-23T10:16:25.576 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-23T10:16:25.576 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-23T10:16:25.576 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-23T10:16:25.576 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-23T10:16:25.576 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-23T10:16:25.576 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:24.799+0000 7f7c09a62700 1 Processor -- start 2024-04-23T10:16:25.576 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:24.799+0000 7f7c09a62700 1 -- start start 2024-04-23T10:16:25.576 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:24.800+0000 7f7c09a62700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c041073e0 0x7f7c04107800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:16:25.577 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:24.800+0000 7f7c09a62700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7c04107dd0 con 0x7f7c041073e0 2024-04-23T10:16:25.577 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:24.800+0000 7f7c037fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c041073e0 0x7f7c04107800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:16:25.577 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:24.800+0000 7f7c037fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c041073e0 0x7f7c04107800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33236/0 (socket says 172.21.15.97:33236) 2024-04-23T10:16:25.577 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:24.800+0000 7f7c037fe700 1 -- 172.21.15.97:0/738026848 learned_addr learned my addr 172.21.15.97:0/738026848 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:16:25.577 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:24.800+0000 7f7c037fe700 1 -- 172.21.15.97:0/738026848 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c041085e0 con 0x7f7c041073e0 2024-04-23T10:16:25.577 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:24.800+0000 7f7c037fe700 1 --2- 172.21.15.97:0/738026848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c041073e0 0x7f7c04107800 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7bf400bb40 tx=0x7f7bf400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8262cd98bfc6c8de server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:16:25.577 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:24.800+0000 7f7c027fc700 1 -- 172.21.15.97:0/738026848 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7bf400dbe0 con 0x7f7c041073e0 2024-04-23T10:16:25.577 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:24.800+0000 7f7c027fc700 1 -- 172.21.15.97:0/738026848 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7bf40044c0 con 0x7f7c041073e0 2024-04-23T10:16:25.577 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:24.800+0000 7f7c027fc700 1 -- 172.21.15.97:0/738026848 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7bf4010470 con 0x7f7c041073e0 2024-04-23T10:16:25.577 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:24.801+0000 7f7c09a62700 1 -- 172.21.15.97:0/738026848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c041073e0 msgr2=0x7f7c04107800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:16:25.577 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:24.801+0000 7f7c09a62700 1 --2- 172.21.15.97:0/738026848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c041073e0 0x7f7c04107800 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7bf400bb40 tx=0x7f7bf400be50 comp rx=0 tx=0).stop 2024-04-23T10:16:25.577 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:24.801+0000 7f7c09a62700 1 -- 172.21.15.97:0/738026848 shutdown_connections 2024-04-23T10:16:25.578 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:24.801+0000 7f7c09a62700 1 --2- 172.21.15.97:0/738026848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c041073e0 0x7f7c04107800 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:16:25.578 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:24.801+0000 7f7c09a62700 1 -- 172.21.15.97:0/738026848 >> 172.21.15.97:0/738026848 conn(0x7f7c041029c0 msgr2=0x7f7c04104de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:16:25.578 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:24.801+0000 7f7c09a62700 1 -- 172.21.15.97:0/738026848 shutdown_connections 2024-04-23T10:16:25.578 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:24.801+0000 7f7c09a62700 1 -- 172.21.15.97:0/738026848 wait complete. 2024-04-23T10:16:25.578 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:24.802+0000 7f7c09a62700 1 Processor -- start 2024-04-23T10:16:25.578 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:24.802+0000 7f7c09a62700 1 -- start start 2024-04-23T10:16:25.578 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:24.802+0000 7f7c09a62700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c041073e0 0x7f7c0419a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:16:25.578 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:24.802+0000 7f7c09a62700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0407afe0 con 0x7f7c041073e0 2024-04-23T10:16:25.578 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:24.802+0000 7f7c037fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c041073e0 0x7f7c0419a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:16:25.578 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:24.802+0000 7f7c037fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c041073e0 0x7f7c0419a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33246/0 (socket says 172.21.15.97:33246) 2024-04-23T10:16:25.578 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:24.802+0000 7f7c037fe700 1 -- 172.21.15.97:0/2582877608 learned_addr learned my addr 172.21.15.97:0/2582877608 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:16:25.579 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:24.802+0000 7f7c037fe700 1 -- 172.21.15.97:0/2582877608 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c0419aa10 con 0x7f7c041073e0 2024-04-23T10:16:25.579 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:24.803+0000 7f7c037fe700 1 --2- 172.21.15.97:0/2582877608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c041073e0 0x7f7c0419a4d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7bf400bfd0 tx=0x7f7bf4003ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:16:25.579 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:24.803+0000 7f7c00ff9700 1 -- 172.21.15.97:0/2582877608 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7bf4027070 con 0x7f7c041073e0 2024-04-23T10:16:25.579 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:24.803+0000 7f7c00ff9700 1 -- 172.21.15.97:0/2582877608 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7bf4010c00 con 0x7f7c041073e0 2024-04-23T10:16:25.579 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:24.803+0000 7f7c09a62700 1 -- 172.21.15.97:0/2582877608 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c0419aca0 con 0x7f7c041073e0 2024-04-23T10:16:25.579 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:24.803+0000 7f7c00ff9700 1 -- 172.21.15.97:0/2582877608 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7bf4022c10 con 0x7f7c041073e0 2024-04-23T10:16:25.579 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:24.803+0000 7f7c09a62700 1 -- 172.21.15.97:0/2582877608 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c0419d990 con 0x7f7c041073e0 2024-04-23T10:16:25.579 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:24.804+0000 7f7c00ff9700 1 -- 172.21.15.97:0/2582877608 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f7bf402a070 con 0x7f7c041073e0 2024-04-23T10:16:25.579 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:24.804+0000 7f7c09a62700 1 -- 172.21.15.97:0/2582877608 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c040640a0 con 0x7f7c041073e0 2024-04-23T10:16:25.579 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:24.804+0000 7f7c00ff9700 1 --2- 172.21.15.97:0/2582877608 >> [v2:172.21.15.97:6800/3706463017,v1:172.21.15.97:6801/3706463017] conn(0x7f7bec039510 0x7f7bec03b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:16:25.579 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:24.804+0000 7f7c00ff9700 1 -- 172.21.15.97:0/2582877608 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7bf401f070 con 0x7f7c041073e0 2024-04-23T10:16:25.580 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:24.804+0000 7f7c02ffd700 1 --2- 172.21.15.97:0/2582877608 >> [v2:172.21.15.97:6800/3706463017,v1:172.21.15.97:6801/3706463017] conn(0x7f7bec039510 0x7f7bec03b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:16:25.580 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:24.804+0000 7f7c02ffd700 1 --2- 172.21.15.97:0/2582877608 >> [v2:172.21.15.97:6800/3706463017,v1:172.21.15.97:6801/3706463017] conn(0x7f7bec039510 0x7f7bec03b9d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8006fd0 tx=0x7f7bf8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:16:25.580 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:24.809+0000 7f7c00ff9700 1 -- 172.21.15.97:0/2582877608 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7bf4015070 con 0x7f7c041073e0 2024-04-23T10:16:25.580 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:25.004+0000 7f7c09a62700 1 -- 172.21.15.97:0/2582877608 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f7c0419ee30 con 0x7f7c041073e0 2024-04-23T10:16:25.580 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:25.005+0000 7f7c00ff9700 1 -- 172.21.15.97:0/2582877608 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+409 (secure 0 0 0) 0x7f7bf4017300 con 0x7f7c041073e0 2024-04-23T10:16:25.580 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:25.008+0000 7f7c09a62700 1 -- 172.21.15.97:0/2582877608 >> [v2:172.21.15.97:6800/3706463017,v1:172.21.15.97:6801/3706463017] conn(0x7f7bec039510 msgr2=0x7f7bec03b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:16:25.580 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:25.008+0000 7f7c09a62700 1 --2- 172.21.15.97:0/2582877608 >> [v2:172.21.15.97:6800/3706463017,v1:172.21.15.97:6801/3706463017] conn(0x7f7bec039510 0x7f7bec03b9d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8006fd0 tx=0x7f7bf8006e40 comp rx=0 tx=0).stop 2024-04-23T10:16:25.580 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:25.008+0000 7f7c09a62700 1 -- 172.21.15.97:0/2582877608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c041073e0 msgr2=0x7f7c0419a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:16:25.580 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:25.008+0000 7f7c09a62700 1 --2- 172.21.15.97:0/2582877608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c041073e0 0x7f7c0419a4d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7bf400bfd0 tx=0x7f7bf4003ff0 comp rx=0 tx=0).stop 2024-04-23T10:16:25.580 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:25.008+0000 7f7c09a62700 1 -- 172.21.15.97:0/2582877608 shutdown_connections 2024-04-23T10:16:25.580 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:25.008+0000 7f7c09a62700 1 --2- 172.21.15.97:0/2582877608 >> [v2:172.21.15.97:6800/3706463017,v1:172.21.15.97:6801/3706463017] conn(0x7f7bec039510 0x7f7bec03b9d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:16:25.580 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:25.008+0000 7f7c09a62700 1 --2- 172.21.15.97:0/2582877608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c041073e0 0x7f7c0419a4d0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:16:25.581 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:25.008+0000 7f7c09a62700 1 -- 172.21.15.97:0/2582877608 >> 172.21.15.97:0/2582877608 conn(0x7f7c041029c0 msgr2=0x7f7c04190b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:16:25.581 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:25.008+0000 7f7c09a62700 1 -- 172.21.15.97:0/2582877608 shutdown_connections 2024-04-23T10:16:25.581 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:25.008+0000 7f7c09a62700 1 -- 172.21.15.97:0/2582877608 wait complete. 2024-04-23T10:16:25.581 INFO:teuthology.orchestra.run.smithi097.stdout:Enabling cephadm module... 2024-04-23T10:16:26.612 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:26 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/4055291827' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-23T10:16:26.612 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:26 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: ignoring --setuser ceph since I am not root 2024-04-23T10:16:26.612 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:26 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: ignoring --setgroup ceph since I am not root 2024-04-23T10:16:26.612 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:26 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:26.453+0000 7f1f4a1a5200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-23T10:16:26.905 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:26 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:26.640+0000 7f1f4a1a5200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-23T10:16:26.905 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:26 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:26.743+0000 7f1f4a1a5200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-23T10:16:26.915 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:25.974+0000 7fdf16f22700 1 Processor -- start 2024-04-23T10:16:26.915 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:25.974+0000 7fdf16f22700 1 -- start start 2024-04-23T10:16:26.915 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:25.975+0000 7fdf16f22700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf101078b0 0x7fdf10107cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:16:26.915 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:25.975+0000 7fdf16f22700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdf101082a0 con 0x7fdf101078b0 2024-04-23T10:16:26.915 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:25.975+0000 7fdf14cbe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf101078b0 0x7fdf10107cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:16:26.915 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:25.975+0000 7fdf14cbe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf101078b0 0x7fdf10107cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56536/0 (socket says 172.21.15.97:56536) 2024-04-23T10:16:26.915 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:25.975+0000 7fdf14cbe700 1 -- 172.21.15.97:0/4219407992 learned_addr learned my addr 172.21.15.97:0/4219407992 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:16:26.916 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:25.975+0000 7fdf14cbe700 1 -- 172.21.15.97:0/4219407992 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf101085d0 con 0x7fdf101078b0 2024-04-23T10:16:26.916 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:25.975+0000 7fdf14cbe700 1 --2- 172.21.15.97:0/4219407992 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf101078b0 0x7fdf10107cd0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fdf040098a0 tx=0x7fdf04009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5177bce05244d48f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:16:26.916 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:25.975+0000 7fdf0f7fe700 1 -- 172.21.15.97:0/4219407992 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf04004160 con 0x7fdf101078b0 2024-04-23T10:16:26.916 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:25.975+0000 7fdf0f7fe700 1 -- 172.21.15.97:0/4219407992 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fdf04004300 con 0x7fdf101078b0 2024-04-23T10:16:26.916 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:25.975+0000 7fdf0f7fe700 1 -- 172.21.15.97:0/4219407992 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf04011af0 con 0x7fdf101078b0 2024-04-23T10:16:26.916 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:25.976+0000 7fdf16f22700 1 -- 172.21.15.97:0/4219407992 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf101078b0 msgr2=0x7fdf10107cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:16:26.916 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:25.976+0000 7fdf16f22700 1 --2- 172.21.15.97:0/4219407992 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf101078b0 0x7fdf10107cd0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fdf040098a0 tx=0x7fdf04009bb0 comp rx=0 tx=0).stop 2024-04-23T10:16:26.916 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:25.976+0000 7fdf16f22700 1 -- 172.21.15.97:0/4219407992 shutdown_connections 2024-04-23T10:16:26.916 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:25.976+0000 7fdf16f22700 1 --2- 172.21.15.97:0/4219407992 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf101078b0 0x7fdf10107cd0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:16:26.916 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:25.976+0000 7fdf16f22700 1 -- 172.21.15.97:0/4219407992 >> 172.21.15.97:0/4219407992 conn(0x7fdf10102110 msgr2=0x7fdf10104530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:16:26.916 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:25.976+0000 7fdf16f22700 1 -- 172.21.15.97:0/4219407992 shutdown_connections 2024-04-23T10:16:26.916 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:25.976+0000 7fdf16f22700 1 -- 172.21.15.97:0/4219407992 wait complete. 2024-04-23T10:16:26.917 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:25.976+0000 7fdf16f22700 1 Processor -- start 2024-04-23T10:16:26.917 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:25.977+0000 7fdf16f22700 1 -- start start 2024-04-23T10:16:26.917 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:25.977+0000 7fdf16f22700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf101078b0 0x7fdf1019b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:16:26.917 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:25.977+0000 7fdf16f22700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1010b980 con 0x7fdf101078b0 2024-04-23T10:16:26.917 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:25.977+0000 7fdf14cbe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf101078b0 0x7fdf1019b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:16:26.917 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:25.977+0000 7fdf14cbe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf101078b0 0x7fdf1019b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56546/0 (socket says 172.21.15.97:56546) 2024-04-23T10:16:26.917 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:25.977+0000 7fdf14cbe700 1 -- 172.21.15.97:0/4055291827 learned_addr learned my addr 172.21.15.97:0/4055291827 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:16:26.917 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:25.977+0000 7fdf14cbe700 1 -- 172.21.15.97:0/4055291827 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf1019b8b0 con 0x7fdf101078b0 2024-04-23T10:16:26.917 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:25.977+0000 7fdf14cbe700 1 --2- 172.21.15.97:0/4055291827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf101078b0 0x7fdf1019b370 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fdf04011ac0 tx=0x7fdf04011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:16:26.917 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:25.977+0000 7fdf0dffb700 1 -- 172.21.15.97:0/4055291827 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf04003970 con 0x7fdf101078b0 2024-04-23T10:16:26.917 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:25.977+0000 7fdf0dffb700 1 -- 172.21.15.97:0/4055291827 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fdf04003b10 con 0x7fdf101078b0 2024-04-23T10:16:26.918 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:25.977+0000 7fdf16f22700 1 -- 172.21.15.97:0/4055291827 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf1019bb40 con 0x7fdf101078b0 2024-04-23T10:16:26.918 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:25.977+0000 7fdf0dffb700 1 -- 172.21.15.97:0/4055291827 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf04024610 con 0x7fdf101078b0 2024-04-23T10:16:26.918 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:25.978+0000 7fdf16f22700 1 -- 172.21.15.97:0/4055291827 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf1019e830 con 0x7fdf101078b0 2024-04-23T10:16:26.918 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:25.978+0000 7fdf0dffb700 1 -- 172.21.15.97:0/4055291827 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7fdf04024840 con 0x7fdf101078b0 2024-04-23T10:16:26.918 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:25.979+0000 7fdf16f22700 1 -- 172.21.15.97:0/4055291827 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf100640a0 con 0x7fdf101078b0 2024-04-23T10:16:26.918 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:25.979+0000 7fdf0dffb700 1 --2- 172.21.15.97:0/4055291827 >> [v2:172.21.15.97:6800/3706463017,v1:172.21.15.97:6801/3706463017] conn(0x7fdf000395b0 0x7fdf0003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:16:26.918 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:25.979+0000 7fdf0dffb700 1 -- 172.21.15.97:0/4055291827 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fdf0404d9f0 con 0x7fdf101078b0 2024-04-23T10:16:26.918 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:25.979+0000 7fdf0ffff700 1 --2- 172.21.15.97:0/4055291827 >> [v2:172.21.15.97:6800/3706463017,v1:172.21.15.97:6801/3706463017] conn(0x7fdf000395b0 0x7fdf0003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:16:26.918 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:25.979+0000 7fdf0ffff700 1 --2- 172.21.15.97:0/4055291827 >> [v2:172.21.15.97:6800/3706463017,v1:172.21.15.97:6801/3706463017] conn(0x7fdf000395b0 0x7fdf0003ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdefc006fd0 tx=0x7fdefc006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:16:26.918 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:25.984+0000 7fdf0dffb700 1 -- 172.21.15.97:0/4055291827 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdf0404d410 con 0x7fdf101078b0 2024-04-23T10:16:26.918 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:26.237+0000 7fdf16f22700 1 -- 172.21.15.97:0/4055291827 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7fdf1019ea00 con 0x7fdf101078b0 2024-04-23T10:16:26.919 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:26.288+0000 7fdf0dffb700 1 -- 172.21.15.97:0/4055291827 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7fdf1019ea00 con 0x7fdf101078b0 2024-04-23T10:16:26.919 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:26.290+0000 7fdf0dffb700 1 -- 172.21.15.97:0/4055291827 <== mon.0 v2:172.21.15.97:3300/0 8 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7fdf0404fd10 con 0x7fdf101078b0 2024-04-23T10:16:26.919 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:26.291+0000 7fdf16f22700 1 -- 172.21.15.97:0/4055291827 >> [v2:172.21.15.97:6800/3706463017,v1:172.21.15.97:6801/3706463017] conn(0x7fdf000395b0 msgr2=0x7fdf0003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:16:26.919 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:26.291+0000 7fdf16f22700 1 --2- 172.21.15.97:0/4055291827 >> [v2:172.21.15.97:6800/3706463017,v1:172.21.15.97:6801/3706463017] conn(0x7fdf000395b0 0x7fdf0003ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdefc006fd0 tx=0x7fdefc006e40 comp rx=0 tx=0).stop 2024-04-23T10:16:26.919 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:26.292+0000 7fdf16f22700 1 -- 172.21.15.97:0/4055291827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf101078b0 msgr2=0x7fdf1019b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:16:26.919 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:26.292+0000 7fdf16f22700 1 --2- 172.21.15.97:0/4055291827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf101078b0 0x7fdf1019b370 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fdf04011ac0 tx=0x7fdf04011ba0 comp rx=0 tx=0).stop 2024-04-23T10:16:26.919 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:26.292+0000 7fdf16f22700 1 -- 172.21.15.97:0/4055291827 shutdown_connections 2024-04-23T10:16:26.919 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:26.292+0000 7fdf16f22700 1 --2- 172.21.15.97:0/4055291827 >> [v2:172.21.15.97:6800/3706463017,v1:172.21.15.97:6801/3706463017] conn(0x7fdf000395b0 0x7fdf0003ba70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:16:26.919 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:26.292+0000 7fdf16f22700 1 --2- 172.21.15.97:0/4055291827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf101078b0 0x7fdf1019b370 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:16:26.919 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:26.292+0000 7fdf16f22700 1 -- 172.21.15.97:0/4055291827 >> 172.21.15.97:0/4055291827 conn(0x7fdf10102110 msgr2=0x7fdf10102c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:16:26.919 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:26.292+0000 7fdf16f22700 1 -- 172.21.15.97:0/4055291827 shutdown_connections 2024-04-23T10:16:26.919 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:26.292+0000 7fdf16f22700 1 -- 172.21.15.97:0/4055291827 wait complete. 2024-04-23T10:16:27.163 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:27 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:27.058+0000 7f1f4a1a5200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-23T10:16:27.163 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:27 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:27.142+0000 7f1f4a1a5200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-23T10:16:27.438 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:27 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/4055291827' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-23T10:16:27.438 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:27 smithi097 ceph-mon[29197]: mgrmap e5: x(active, since 6s) 2024-04-23T10:16:28.247 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout { 2024-04-23T10:16:28.247 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-23T10:16:28.248 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-23T10:16:28.248 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-23T10:16:28.248 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-23T10:16:28.248 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout } 2024-04-23T10:16:28.248 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:27.386+0000 7f367c0d1700 1 Processor -- start 2024-04-23T10:16:28.248 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:27.386+0000 7f367c0d1700 1 -- start start 2024-04-23T10:16:28.248 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:27.386+0000 7f367c0d1700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3674108540 0x7f3674108960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:16:28.248 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:27.386+0000 7f367c0d1700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3674108f30 con 0x7f3674108540 2024-04-23T10:16:28.248 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:27.386+0000 7f3679e6d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3674108540 0x7f3674108960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:16:28.248 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:27.386+0000 7f3679e6d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3674108540 0x7f3674108960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56568/0 (socket says 172.21.15.97:56568) 2024-04-23T10:16:28.248 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:27.386+0000 7f3679e6d700 1 -- 172.21.15.97:0/2913556158 learned_addr learned my addr 172.21.15.97:0/2913556158 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:16:28.248 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:27.387+0000 7f3679e6d700 1 -- 172.21.15.97:0/2913556158 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36741096f0 con 0x7f3674108540 2024-04-23T10:16:28.248 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:27.387+0000 7f3679e6d700 1 --2- 172.21.15.97:0/2913556158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3674108540 0x7f3674108960 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f36700098a0 tx=0x7f3670009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c206cee511da54b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:16:28.248 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:27.387+0000 7f3678e6b700 1 -- 172.21.15.97:0/2913556158 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3670004160 con 0x7f3674108540 2024-04-23T10:16:28.249 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:27.387+0000 7f3678e6b700 1 -- 172.21.15.97:0/2913556158 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3670004300 con 0x7f3674108540 2024-04-23T10:16:28.249 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:27.387+0000 7f3678e6b700 1 -- 172.21.15.97:0/2913556158 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3670011af0 con 0x7f3674108540 2024-04-23T10:16:28.249 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:27.388+0000 7f367c0d1700 1 -- 172.21.15.97:0/2913556158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3674108540 msgr2=0x7f3674108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:16:28.249 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:27.388+0000 7f367c0d1700 1 --2- 172.21.15.97:0/2913556158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3674108540 0x7f3674108960 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f36700098a0 tx=0x7f3670009bb0 comp rx=0 tx=0).stop 2024-04-23T10:16:28.249 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:27.388+0000 7f367c0d1700 1 -- 172.21.15.97:0/2913556158 shutdown_connections 2024-04-23T10:16:28.249 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:27.388+0000 7f367c0d1700 1 --2- 172.21.15.97:0/2913556158 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3674108540 0x7f3674108960 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:16:28.249 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:27.388+0000 7f367c0d1700 1 -- 172.21.15.97:0/2913556158 >> 172.21.15.97:0/2913556158 conn(0x7f3674103ba0 msgr2=0x7f3674105fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:16:28.249 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:27.388+0000 7f367c0d1700 1 -- 172.21.15.97:0/2913556158 shutdown_connections 2024-04-23T10:16:28.249 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:27.388+0000 7f367c0d1700 1 -- 172.21.15.97:0/2913556158 wait complete. 2024-04-23T10:16:28.249 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:27.389+0000 7f367c0d1700 1 Processor -- start 2024-04-23T10:16:28.249 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:27.389+0000 7f367c0d1700 1 -- start start 2024-04-23T10:16:28.249 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:27.389+0000 7f367c0d1700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3674108540 0x7f367419c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:16:28.249 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:27.389+0000 7f367c0d1700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f367410c920 con 0x7f3674108540 2024-04-23T10:16:28.249 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:27.389+0000 7f3679e6d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3674108540 0x7f367419c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:16:28.249 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:27.389+0000 7f3679e6d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3674108540 0x7f367419c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56580/0 (socket says 172.21.15.97:56580) 2024-04-23T10:16:28.250 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:27.389+0000 7f3679e6d700 1 -- 172.21.15.97:0/4268558603 learned_addr learned my addr 172.21.15.97:0/4268558603 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:16:28.250 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:27.389+0000 7f3679e6d700 1 -- 172.21.15.97:0/4268558603 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f367419c990 con 0x7f3674108540 2024-04-23T10:16:28.250 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:27.390+0000 7f3679e6d700 1 --2- 172.21.15.97:0/4268558603 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3674108540 0x7f367419c450 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f36700117b0 tx=0x7f3670011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:16:28.250 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:27.390+0000 7f366affd700 1 -- 172.21.15.97:0/4268558603 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3670003af0 con 0x7f3674108540 2024-04-23T10:16:28.250 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:27.390+0000 7f366affd700 1 -- 172.21.15.97:0/4268558603 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3670020070 con 0x7f3674108540 2024-04-23T10:16:28.250 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:27.390+0000 7f367c0d1700 1 -- 172.21.15.97:0/4268558603 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f367419cc20 con 0x7f3674108540 2024-04-23T10:16:28.250 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:27.391+0000 7f366affd700 1 -- 172.21.15.97:0/4268558603 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3670019630 con 0x7f3674108540 2024-04-23T10:16:28.250 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:27.391+0000 7f367c0d1700 1 -- 172.21.15.97:0/4268558603 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f367419f910 con 0x7f3674108540 2024-04-23T10:16:28.250 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:27.391+0000 7f366affd700 1 -- 172.21.15.97:0/4268558603 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f36700198a0 con 0x7f3674108540 2024-04-23T10:16:28.250 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:27.391+0000 7f366affd700 1 --2- 172.21.15.97:0/4268558603 >> [v2:172.21.15.97:6800/3706463017,v1:172.21.15.97:6801/3706463017] conn(0x7f3660039600 0x7f366003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:16:28.250 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:27.391+0000 7f366affd700 1 -- 172.21.15.97:0/4268558603 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3670052720 con 0x7f3674108540 2024-04-23T10:16:28.250 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:27.391+0000 7f367966c700 1 -- 172.21.15.97:0/4268558603 >> [v2:172.21.15.97:6800/3706463017,v1:172.21.15.97:6801/3706463017] conn(0x7f3660039600 msgr2=0x7f366003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/3706463017 2024-04-23T10:16:28.250 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:27.391+0000 7f367966c700 1 --2- 172.21.15.97:0/4268558603 >> [v2:172.21.15.97:6800/3706463017,v1:172.21.15.97:6801/3706463017] conn(0x7f3660039600 0x7f366003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T10:16:28.251 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:27.392+0000 7f367c0d1700 1 -- 172.21.15.97:0/4268558603 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f367410caa0 con 0x7f3674108540 2024-04-23T10:16:28.251 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:27.400+0000 7f366affd700 1 -- 172.21.15.97:0/4268558603 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f36700042c0 con 0x7f3674108540 2024-04-23T10:16:28.251 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:27.592+0000 7f367966c700 1 -- 172.21.15.97:0/4268558603 >> [v2:172.21.15.97:6800/3706463017,v1:172.21.15.97:6801/3706463017] conn(0x7f3660039600 msgr2=0x7f366003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/3706463017 2024-04-23T10:16:28.251 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:27.592+0000 7f367966c700 1 --2- 172.21.15.97:0/4268558603 >> [v2:172.21.15.97:6800/3706463017,v1:172.21.15.97:6801/3706463017] conn(0x7f3660039600 0x7f366003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-23T10:16:28.251 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:27.699+0000 7f367c0d1700 1 -- 172.21.15.97:0/4268558603 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f367419fe70 con 0x7f3674108540 2024-04-23T10:16:28.251 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:27.699+0000 7f366affd700 1 -- 172.21.15.97:0/4268558603 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f3670020220 con 0x7f3674108540 2024-04-23T10:16:28.251 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:27.701+0000 7f367c0d1700 1 -- 172.21.15.97:0/4268558603 >> [v2:172.21.15.97:6800/3706463017,v1:172.21.15.97:6801/3706463017] conn(0x7f3660039600 msgr2=0x7f366003bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T10:16:28.251 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:27.701+0000 7f367c0d1700 1 --2- 172.21.15.97:0/4268558603 >> [v2:172.21.15.97:6800/3706463017,v1:172.21.15.97:6801/3706463017] conn(0x7f3660039600 0x7f366003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:16:28.251 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:27.701+0000 7f367c0d1700 1 -- 172.21.15.97:0/4268558603 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3674108540 msgr2=0x7f367419c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:16:28.251 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:27.701+0000 7f367c0d1700 1 --2- 172.21.15.97:0/4268558603 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3674108540 0x7f367419c450 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f36700117b0 tx=0x7f3670011b70 comp rx=0 tx=0).stop 2024-04-23T10:16:28.251 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:27.701+0000 7f367c0d1700 1 -- 172.21.15.97:0/4268558603 shutdown_connections 2024-04-23T10:16:28.251 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:27.701+0000 7f367c0d1700 1 --2- 172.21.15.97:0/4268558603 >> [v2:172.21.15.97:6800/3706463017,v1:172.21.15.97:6801/3706463017] conn(0x7f3660039600 0x7f366003bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:16:28.251 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:27.701+0000 7f367c0d1700 1 --2- 172.21.15.97:0/4268558603 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3674108540 0x7f367419c450 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:16:28.251 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:27.701+0000 7f367c0d1700 1 -- 172.21.15.97:0/4268558603 >> 172.21.15.97:0/4268558603 conn(0x7f3674103ba0 msgr2=0x7f3674104710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:16:28.252 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:27.702+0000 7f367c0d1700 1 -- 172.21.15.97:0/4268558603 shutdown_connections 2024-04-23T10:16:28.252 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:27.702+0000 7f367c0d1700 1 -- 172.21.15.97:0/4268558603 wait complete. 2024-04-23T10:16:28.252 INFO:teuthology.orchestra.run.smithi097.stdout:Waiting for the mgr to restart... 2024-04-23T10:16:28.252 INFO:teuthology.orchestra.run.smithi097.stdout:Waiting for mgr epoch 5... 2024-04-23T10:16:28.512 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:28 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/4268558603' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-23T10:16:28.512 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:28 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:28.387+0000 7f1f4a1a5200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-23T10:16:28.861 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:28 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:28.564+0000 7f1f4a1a5200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-23T10:16:28.861 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:28 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:28.737+0000 7f1f4a1a5200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-23T10:16:29.318 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:29 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:29.028+0000 7f1f4a1a5200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-23T10:16:29.319 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:29 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:29.120+0000 7f1f4a1a5200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-23T10:16:29.319 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:29 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:29.319+0000 7f1f4a1a5200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-23T10:16:29.611 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:29 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:29.512+0000 7f1f4a1a5200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-23T10:16:30.543 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:30 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:30.123+0000 7f1f4a1a5200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-23T10:16:30.837 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:30 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:30.544+0000 7f1f4a1a5200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-23T10:16:30.837 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:30 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:30.639+0000 7f1f4a1a5200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-23T10:16:30.837 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:30 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:30.837+0000 7f1f4a1a5200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-23T10:16:31.611 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:31 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:31.352+0000 7f1f4a1a5200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-23T10:16:31.611 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:31 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:31.443+0000 7f1f4a1a5200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-23T10:16:32.020 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:31 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:31.681+0000 7f1f4a1a5200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-23T10:16:32.020 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:31 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:31.812+0000 7f1f4a1a5200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-23T10:16:32.020 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:31 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:31.905+0000 7f1f4a1a5200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-23T10:16:32.021 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:32 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:32.020+0000 7f1f4a1a5200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-23T10:16:32.611 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:32 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:32.315+0000 7f1f4a1a5200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-23T10:16:33.179 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:32 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:32.909+0000 7f1f4a1a5200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-23T10:16:33.179 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:33 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:33.126+0000 7f1f4a1a5200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-23T10:16:33.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:33 smithi097 ceph-mon[29197]: Active manager daemon x restarted 2024-04-23T10:16:33.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:33 smithi097 ceph-mon[29197]: Activating manager daemon x 2024-04-23T10:16:33.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:33 smithi097 ceph-mon[29197]: osdmap e2: 0 total, 0 up, 0 in 2024-04-23T10:16:33.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:33 smithi097 ceph-mon[29197]: mgrmap e6: x(active, starting, since 0.00332534s) 2024-04-23T10:16:33.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:33 smithi097 ceph-mon[29197]: from='mgr.14120 172.21.15.97:0/850926340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T10:16:33.612 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:33 smithi097 ceph-mon[29197]: from='mgr.14120 172.21.15.97:0/850926340' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-23T10:16:33.612 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:33 smithi097 ceph-mon[29197]: from='mgr.14120 172.21.15.97:0/850926340' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-23T10:16:33.612 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:33 smithi097 ceph-mon[29197]: from='mgr.14120 172.21.15.97:0/850926340' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-23T10:16:33.612 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:33 smithi097 ceph-mon[29197]: from='mgr.14120 172.21.15.97:0/850926340' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-23T10:16:33.612 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:33 smithi097 ceph-mon[29197]: Manager daemon x is now available 2024-04-23T10:16:33.612 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:33 smithi097 ceph-mon[29197]: from='mgr.14120 172.21.15.97:0/850926340' entity='mgr.x' 2024-04-23T10:16:33.612 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:33 smithi097 ceph-mon[29197]: from='mgr.14120 172.21.15.97:0/850926340' entity='mgr.x' 2024-04-23T10:16:33.612 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:33 smithi097 ceph-mon[29197]: from='mgr.14120 172.21.15.97:0/850926340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:16:33.612 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:33 smithi097 ceph-mon[29197]: from='mgr.14120 172.21.15.97:0/850926340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:16:33.612 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:33 smithi097 ceph-mon[29197]: from='mgr.14120 172.21.15.97:0/850926340' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-23T10:16:34.444 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:34 smithi097 ceph-mon[29197]: Found migration_current of "None". Setting to last migration. 2024-04-23T10:16:34.444 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:34 smithi097 ceph-mon[29197]: from='mgr.14120 172.21.15.97:0/850926340' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-23T10:16:34.444 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:34 smithi097 ceph-mon[29197]: mgrmap e7: x(active, since 1.00573s) 2024-04-23T10:16:34.798 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout { 2024-04-23T10:16:34.798 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-23T10:16:34.798 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-23T10:16:34.799 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout } 2024-04-23T10:16:34.799 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:28.751+0000 7f955713e700 1 Processor -- start 2024-04-23T10:16:34.799 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:28.751+0000 7f955713e700 1 -- start start 2024-04-23T10:16:34.799 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:28.751+0000 7f955713e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95501073e0 0x7f9550107800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:16:34.799 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:28.751+0000 7f955713e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9550107dd0 con 0x7f95501073e0 2024-04-23T10:16:34.799 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:28.752+0000 7f9554eda700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95501073e0 0x7f9550107800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:16:34.799 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:28.752+0000 7f9554eda700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95501073e0 0x7f9550107800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56596/0 (socket says 172.21.15.97:56596) 2024-04-23T10:16:34.799 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:28.752+0000 7f9554eda700 1 -- 172.21.15.97:0/957005585 learned_addr learned my addr 172.21.15.97:0/957005585 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:16:34.800 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:28.752+0000 7f9554eda700 1 -- 172.21.15.97:0/957005585 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95501085e0 con 0x7f95501073e0 2024-04-23T10:16:34.800 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:28.752+0000 7f9554eda700 1 --2- 172.21.15.97:0/957005585 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95501073e0 0x7f9550107800 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f95440098a0 tx=0x7f9544009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53c5d115f12e0ee3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:16:34.800 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:28.752+0000 7f954f7fe700 1 -- 172.21.15.97:0/957005585 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9544004160 con 0x7f95501073e0 2024-04-23T10:16:34.800 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:28.752+0000 7f954f7fe700 1 -- 172.21.15.97:0/957005585 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9544004300 con 0x7f95501073e0 2024-04-23T10:16:34.800 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:28.752+0000 7f954f7fe700 1 -- 172.21.15.97:0/957005585 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9544011af0 con 0x7f95501073e0 2024-04-23T10:16:34.800 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:28.753+0000 7f955713e700 1 -- 172.21.15.97:0/957005585 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95501073e0 msgr2=0x7f9550107800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:16:34.800 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:28.753+0000 7f955713e700 1 --2- 172.21.15.97:0/957005585 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95501073e0 0x7f9550107800 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f95440098a0 tx=0x7f9544009bb0 comp rx=0 tx=0).stop 2024-04-23T10:16:34.800 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:28.753+0000 7f955713e700 1 -- 172.21.15.97:0/957005585 shutdown_connections 2024-04-23T10:16:34.800 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:28.753+0000 7f955713e700 1 --2- 172.21.15.97:0/957005585 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95501073e0 0x7f9550107800 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:16:34.800 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:28.753+0000 7f955713e700 1 -- 172.21.15.97:0/957005585 >> 172.21.15.97:0/957005585 conn(0x7f95501029c0 msgr2=0x7f9550104de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:16:34.801 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:28.753+0000 7f955713e700 1 -- 172.21.15.97:0/957005585 shutdown_connections 2024-04-23T10:16:34.801 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:28.753+0000 7f955713e700 1 -- 172.21.15.97:0/957005585 wait complete. 2024-04-23T10:16:34.801 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:28.753+0000 7f955713e700 1 Processor -- start 2024-04-23T10:16:34.801 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:28.754+0000 7f955713e700 1 -- start start 2024-04-23T10:16:34.801 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:28.754+0000 7f955713e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95501073e0 0x7f955019c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:16:34.801 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:28.754+0000 7f955713e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f955010b990 con 0x7f95501073e0 2024-04-23T10:16:34.801 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:28.754+0000 7f9554eda700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95501073e0 0x7f955019c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:16:34.801 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:28.754+0000 7f9554eda700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95501073e0 0x7f955019c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56598/0 (socket says 172.21.15.97:56598) 2024-04-23T10:16:34.802 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:28.754+0000 7f9554eda700 1 -- 172.21.15.97:0/2377552176 learned_addr learned my addr 172.21.15.97:0/2377552176 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:16:34.802 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:28.754+0000 7f9554eda700 1 -- 172.21.15.97:0/2377552176 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f955019cd10 con 0x7f95501073e0 2024-04-23T10:16:34.802 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:28.754+0000 7f9554eda700 1 --2- 172.21.15.97:0/2377552176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95501073e0 0x7f955019c7d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9544000c00 tx=0x7f9544011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:16:34.802 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:28.754+0000 7f954dffb700 1 -- 172.21.15.97:0/2377552176 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9544003b20 con 0x7f95501073e0 2024-04-23T10:16:34.802 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:28.754+0000 7f954dffb700 1 -- 172.21.15.97:0/2377552176 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f954401a8c0 con 0x7f95501073e0 2024-04-23T10:16:34.802 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:28.754+0000 7f955713e700 1 -- 172.21.15.97:0/2377552176 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f955019cfa0 con 0x7f95501073e0 2024-04-23T10:16:34.802 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:28.754+0000 7f954dffb700 1 -- 172.21.15.97:0/2377552176 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9544024940 con 0x7f95501073e0 2024-04-23T10:16:34.802 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:28.755+0000 7f955713e700 1 -- 172.21.15.97:0/2377552176 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f955019fc90 con 0x7f95501073e0 2024-04-23T10:16:34.802 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:28.755+0000 7f954dffb700 1 -- 172.21.15.97:0/2377552176 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f9544022020 con 0x7f95501073e0 2024-04-23T10:16:34.802 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:28.756+0000 7f954dffb700 1 --2- 172.21.15.97:0/2377552176 >> [v2:172.21.15.97:6800/3706463017,v1:172.21.15.97:6801/3706463017] conn(0x7f95400395b0 0x7f954003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:16:34.802 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:28.756+0000 7f954dffb700 1 -- 172.21.15.97:0/2377552176 --> [v2:172.21.15.97:6800/3706463017,v1:172.21.15.97:6801/3706463017] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f954003c180 con 0x7f95400395b0 2024-04-23T10:16:34.803 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:28.756+0000 7f954dffb700 1 -- 172.21.15.97:0/2377552176 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f954404d510 con 0x7f95501073e0 2024-04-23T10:16:34.803 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:28.756+0000 7f954ffff700 1 -- 172.21.15.97:0/2377552176 >> [v2:172.21.15.97:6800/3706463017,v1:172.21.15.97:6801/3706463017] conn(0x7f95400395b0 msgr2=0x7f954003ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/3706463017 2024-04-23T10:16:34.803 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:28.756+0000 7f954ffff700 1 --2- 172.21.15.97:0/2377552176 >> [v2:172.21.15.97:6800/3706463017,v1:172.21.15.97:6801/3706463017] conn(0x7f95400395b0 0x7f954003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T10:16:34.803 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:28.957+0000 7f954ffff700 1 -- 172.21.15.97:0/2377552176 >> [v2:172.21.15.97:6800/3706463017,v1:172.21.15.97:6801/3706463017] conn(0x7f95400395b0 msgr2=0x7f954003ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/3706463017 2024-04-23T10:16:34.803 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:28.957+0000 7f954ffff700 1 --2- 172.21.15.97:0/2377552176 >> [v2:172.21.15.97:6800/3706463017,v1:172.21.15.97:6801/3706463017] conn(0x7f95400395b0 0x7f954003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-23T10:16:34.803 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:29.358+0000 7f954ffff700 1 -- 172.21.15.97:0/2377552176 >> [v2:172.21.15.97:6800/3706463017,v1:172.21.15.97:6801/3706463017] conn(0x7f95400395b0 msgr2=0x7f954003ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/3706463017 2024-04-23T10:16:34.803 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:29.358+0000 7f954ffff700 1 --2- 172.21.15.97:0/2377552176 >> [v2:172.21.15.97:6800/3706463017,v1:172.21.15.97:6801/3706463017] conn(0x7f95400395b0 0x7f954003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-23T10:16:34.803 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:30.159+0000 7f954ffff700 1 -- 172.21.15.97:0/2377552176 >> [v2:172.21.15.97:6800/3706463017,v1:172.21.15.97:6801/3706463017] conn(0x7f95400395b0 msgr2=0x7f954003ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/3706463017 2024-04-23T10:16:34.803 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:30.159+0000 7f954ffff700 1 --2- 172.21.15.97:0/2377552176 >> [v2:172.21.15.97:6800/3706463017,v1:172.21.15.97:6801/3706463017] conn(0x7f95400395b0 0x7f954003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-23T10:16:34.804 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:31.760+0000 7f954ffff700 1 -- 172.21.15.97:0/2377552176 >> [v2:172.21.15.97:6800/3706463017,v1:172.21.15.97:6801/3706463017] conn(0x7f95400395b0 msgr2=0x7f954003ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/3706463017 2024-04-23T10:16:34.804 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:31.760+0000 7f954ffff700 1 --2- 172.21.15.97:0/2377552176 >> [v2:172.21.15.97:6800/3706463017,v1:172.21.15.97:6801/3706463017] conn(0x7f95400395b0 0x7f954003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-23T10:16:34.804 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:33.130+0000 7f954dffb700 1 -- 172.21.15.97:0/2377552176 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mgrmap(e 6) ==== 46325+0+0 (secure 0 0 0) 0x7f954401aa70 con 0x7f95501073e0 2024-04-23T10:16:34.804 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:33.130+0000 7f954dffb700 1 -- 172.21.15.97:0/2377552176 >> [v2:172.21.15.97:6800/3706463017,v1:172.21.15.97:6801/3706463017] conn(0x7f95400395b0 msgr2=0x7f954003ba70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T10:16:34.804 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:33.130+0000 7f954dffb700 1 --2- 172.21.15.97:0/2377552176 >> [v2:172.21.15.97:6800/3706463017,v1:172.21.15.97:6801/3706463017] conn(0x7f95400395b0 0x7f954003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:16:34.804 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:34.131+0000 7f954dffb700 1 -- 172.21.15.97:0/2377552176 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f954404efe0 con 0x7f95501073e0 2024-04-23T10:16:34.804 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:34.131+0000 7f954dffb700 1 --2- 172.21.15.97:0/2377552176 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f954003cea0 0x7f954003f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:16:34.804 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:34.131+0000 7f954dffb700 1 -- 172.21.15.97:0/2377552176 --> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9544022e80 con 0x7f954003cea0 2024-04-23T10:16:34.804 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:34.131+0000 7f954ffff700 1 --2- 172.21.15.97:0/2377552176 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f954003cea0 0x7f954003f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:16:34.804 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:34.132+0000 7f954ffff700 1 --2- 172.21.15.97:0/2377552176 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f954003cea0 0x7f954003f290 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f953c003a10 tx=0x7f953c009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:16:34.804 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:34.133+0000 7f954dffb700 1 -- 172.21.15.97:0/2377552176 <== mgr.14120 v2:172.21.15.97:6800/3297166858 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f9544022e80 con 0x7f954003cea0 2024-04-23T10:16:34.805 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:34.139+0000 7f955713e700 1 -- 172.21.15.97:0/2377552176 --> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f9550069b90 con 0x7f954003cea0 2024-04-23T10:16:34.805 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:34.140+0000 7f954dffb700 1 -- 172.21.15.97:0/2377552176 <== mgr.14120 v2:172.21.15.97:6800/3297166858 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f9550069b90 con 0x7f954003cea0 2024-04-23T10:16:34.805 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:34.140+0000 7f955713e700 1 -- 172.21.15.97:0/2377552176 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f954003cea0 msgr2=0x7f954003f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:16:34.805 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:34.140+0000 7f955713e700 1 --2- 172.21.15.97:0/2377552176 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f954003cea0 0x7f954003f290 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f953c003a10 tx=0x7f953c009290 comp rx=0 tx=0).stop 2024-04-23T10:16:34.805 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:34.140+0000 7f955713e700 1 -- 172.21.15.97:0/2377552176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95501073e0 msgr2=0x7f955019c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:16:34.805 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:34.140+0000 7f955713e700 1 --2- 172.21.15.97:0/2377552176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95501073e0 0x7f955019c7d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9544000c00 tx=0x7f9544011b70 comp rx=0 tx=0).stop 2024-04-23T10:16:34.805 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:34.140+0000 7f955713e700 1 -- 172.21.15.97:0/2377552176 shutdown_connections 2024-04-23T10:16:34.805 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:34.140+0000 7f955713e700 1 --2- 172.21.15.97:0/2377552176 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f954003cea0 0x7f954003f290 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:16:34.805 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:34.140+0000 7f955713e700 1 --2- 172.21.15.97:0/2377552176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95501073e0 0x7f955019c7d0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:16:34.805 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:34.140+0000 7f955713e700 1 --2- 172.21.15.97:0/2377552176 >> [v2:172.21.15.97:6800/3706463017,v1:172.21.15.97:6801/3706463017] conn(0x7f95400395b0 0x7f954003ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:16:34.805 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:34.140+0000 7f955713e700 1 -- 172.21.15.97:0/2377552176 >> 172.21.15.97:0/2377552176 conn(0x7f95501029c0 msgr2=0x7f9550103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:16:34.806 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:34.140+0000 7f955713e700 1 -- 172.21.15.97:0/2377552176 shutdown_connections 2024-04-23T10:16:34.806 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:34.140+0000 7f955713e700 1 -- 172.21.15.97:0/2377552176 wait complete. 2024-04-23T10:16:34.806 INFO:teuthology.orchestra.run.smithi097.stdout:mgr epoch 5 is available 2024-04-23T10:16:34.806 INFO:teuthology.orchestra.run.smithi097.stdout:Setting orchestrator backend to cephadm... 2024-04-23T10:16:35.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:35 smithi097 ceph-mon[29197]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-23T10:16:35.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:35 smithi097 ceph-mon[29197]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-23T10:16:35.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:35 smithi097 ceph-mon[29197]: from='mgr.14120 172.21.15.97:0/850926340' entity='mgr.x' 2024-04-23T10:16:35.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:35 smithi097 ceph-mon[29197]: from='mgr.14120 172.21.15.97:0/850926340' entity='mgr.x' 2024-04-23T10:16:36.058 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:35.198+0000 7f147be32700 1 Processor -- start 2024-04-23T10:16:36.058 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:35.198+0000 7f147be32700 1 -- start start 2024-04-23T10:16:36.058 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:35.198+0000 7f147be32700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1474108f80 0x7f147410b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:16:36.059 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:35.198+0000 7f147be32700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f147407bec0 con 0x7f1474108f80 2024-04-23T10:16:36.059 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:35.198+0000 7f1479bce700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1474108f80 0x7f147410b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:16:36.059 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:35.199+0000 7f1479bce700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1474108f80 0x7f147410b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56654/0 (socket says 172.21.15.97:56654) 2024-04-23T10:16:36.059 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:35.199+0000 7f1479bce700 1 -- 172.21.15.97:0/849430119 learned_addr learned my addr 172.21.15.97:0/849430119 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:16:36.059 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:35.199+0000 7f1479bce700 1 -- 172.21.15.97:0/849430119 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f147410b8b0 con 0x7f1474108f80 2024-04-23T10:16:36.059 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:35.199+0000 7f1479bce700 1 --2- 172.21.15.97:0/849430119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1474108f80 0x7f147410b370 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f14700098a0 tx=0x7f1470009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27aa1dcc846cee99 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:16:36.059 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:35.199+0000 7f1478bcc700 1 -- 172.21.15.97:0/849430119 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1470004160 con 0x7f1474108f80 2024-04-23T10:16:36.059 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:35.199+0000 7f1478bcc700 1 -- 172.21.15.97:0/849430119 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1470004300 con 0x7f1474108f80 2024-04-23T10:16:36.059 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:35.199+0000 7f1478bcc700 1 -- 172.21.15.97:0/849430119 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1470011af0 con 0x7f1474108f80 2024-04-23T10:16:36.059 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:35.200+0000 7f147be32700 1 -- 172.21.15.97:0/849430119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1474108f80 msgr2=0x7f147410b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:16:36.059 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:35.200+0000 7f147be32700 1 --2- 172.21.15.97:0/849430119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1474108f80 0x7f147410b370 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f14700098a0 tx=0x7f1470009bb0 comp rx=0 tx=0).stop 2024-04-23T10:16:36.060 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:35.200+0000 7f147be32700 1 -- 172.21.15.97:0/849430119 shutdown_connections 2024-04-23T10:16:36.060 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:35.200+0000 7f147be32700 1 --2- 172.21.15.97:0/849430119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1474108f80 0x7f147410b370 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:16:36.060 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:35.200+0000 7f147be32700 1 -- 172.21.15.97:0/849430119 >> 172.21.15.97:0/849430119 conn(0x7f1474102a70 msgr2=0x7f1474104e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:16:36.060 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:35.200+0000 7f147be32700 1 -- 172.21.15.97:0/849430119 shutdown_connections 2024-04-23T10:16:36.060 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:35.200+0000 7f147be32700 1 -- 172.21.15.97:0/849430119 wait complete. 2024-04-23T10:16:36.060 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:35.200+0000 7f147be32700 1 Processor -- start 2024-04-23T10:16:36.060 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:35.200+0000 7f147be32700 1 -- start start 2024-04-23T10:16:36.060 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:35.201+0000 7f147be32700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1474108f80 0x7f147419c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:16:36.060 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:35.201+0000 7f147be32700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f147410c880 con 0x7f1474108f80 2024-04-23T10:16:36.060 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:35.201+0000 7f1479bce700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1474108f80 0x7f147419c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:16:36.060 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:35.201+0000 7f1479bce700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1474108f80 0x7f147419c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56660/0 (socket says 172.21.15.97:56660) 2024-04-23T10:16:36.060 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:35.201+0000 7f1479bce700 1 -- 172.21.15.97:0/3038862413 learned_addr learned my addr 172.21.15.97:0/3038862413 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:16:36.060 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:35.201+0000 7f1479bce700 1 -- 172.21.15.97:0/3038862413 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f147419cca0 con 0x7f1474108f80 2024-04-23T10:16:36.060 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:35.201+0000 7f1479bce700 1 --2- 172.21.15.97:0/3038862413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1474108f80 0x7f147419c760 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1470003f40 tx=0x7f1470011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:16:36.060 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:35.201+0000 7f146affd700 1 -- 172.21.15.97:0/3038862413 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14700038b0 con 0x7f1474108f80 2024-04-23T10:16:36.061 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:35.201+0000 7f146affd700 1 -- 172.21.15.97:0/3038862413 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1470003a50 con 0x7f1474108f80 2024-04-23T10:16:36.061 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:35.201+0000 7f146affd700 1 -- 172.21.15.97:0/3038862413 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f147001a5c0 con 0x7f1474108f80 2024-04-23T10:16:36.061 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:35.201+0000 7f147be32700 1 -- 172.21.15.97:0/3038862413 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f147419cf30 con 0x7f1474108f80 2024-04-23T10:16:36.061 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:35.201+0000 7f147be32700 1 -- 172.21.15.97:0/3038862413 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f147419fbd0 con 0x7f1474108f80 2024-04-23T10:16:36.061 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:35.202+0000 7f146affd700 1 -- 172.21.15.97:0/3038862413 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f1470024430 con 0x7f1474108f80 2024-04-23T10:16:36.061 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:35.202+0000 7f147be32700 1 -- 172.21.15.97:0/3038862413 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14740640a0 con 0x7f1474108f80 2024-04-23T10:16:36.061 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:35.203+0000 7f146affd700 1 --2- 172.21.15.97:0/3038862413 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f14600394a0 0x7f146003b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:16:36.061 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:35.203+0000 7f146affd700 1 -- 172.21.15.97:0/3038862413 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f147004d8f0 con 0x7f1474108f80 2024-04-23T10:16:36.061 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:35.203+0000 7f14793cd700 1 --2- 172.21.15.97:0/3038862413 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f14600394a0 0x7f146003b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:16:36.061 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:35.203+0000 7f14793cd700 1 --2- 172.21.15.97:0/3038862413 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f14600394a0 0x7f146003b960 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1464006fd0 tx=0x7f1464006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:16:36.061 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:35.208+0000 7f146affd700 1 -- 172.21.15.97:0/3038862413 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f147002d910 con 0x7f1474108f80 2024-04-23T10:16:36.061 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:35.432+0000 7f147be32700 1 -- 172.21.15.97:0/3038862413 --> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f147402dff0 con 0x7f14600394a0 2024-04-23T10:16:36.061 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:35.439+0000 7f146affd700 1 -- 172.21.15.97:0/3038862413 <== mgr.14120 v2:172.21.15.97:6800/3297166858 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f147402dff0 con 0x7f14600394a0 2024-04-23T10:16:36.061 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:35.441+0000 7f147be32700 1 -- 172.21.15.97:0/3038862413 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f14600394a0 msgr2=0x7f146003b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:16:36.061 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:35.441+0000 7f147be32700 1 --2- 172.21.15.97:0/3038862413 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f14600394a0 0x7f146003b960 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1464006fd0 tx=0x7f1464006e40 comp rx=0 tx=0).stop 2024-04-23T10:16:36.062 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:35.441+0000 7f147be32700 1 -- 172.21.15.97:0/3038862413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1474108f80 msgr2=0x7f147419c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:16:36.062 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:35.441+0000 7f147be32700 1 --2- 172.21.15.97:0/3038862413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1474108f80 0x7f147419c760 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1470003f40 tx=0x7f1470011890 comp rx=0 tx=0).stop 2024-04-23T10:16:36.062 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:35.441+0000 7f147be32700 1 -- 172.21.15.97:0/3038862413 shutdown_connections 2024-04-23T10:16:36.062 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:35.441+0000 7f147be32700 1 --2- 172.21.15.97:0/3038862413 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f14600394a0 0x7f146003b960 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:16:36.062 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:35.441+0000 7f147be32700 1 --2- 172.21.15.97:0/3038862413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1474108f80 0x7f147419c760 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:16:36.062 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:35.441+0000 7f147be32700 1 -- 172.21.15.97:0/3038862413 >> 172.21.15.97:0/3038862413 conn(0x7f1474102a70 msgr2=0x7f1474103740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:16:36.062 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:35.441+0000 7f147be32700 1 -- 172.21.15.97:0/3038862413 shutdown_connections 2024-04-23T10:16:36.062 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:35.441+0000 7f147be32700 1 -- 172.21.15.97:0/3038862413 wait complete. 2024-04-23T10:16:36.693 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:36 smithi097 ceph-mon[29197]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:16:36.693 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:36 smithi097 ceph-mon[29197]: from='mgr.14120 172.21.15.97:0/850926340' entity='mgr.x' 2024-04-23T10:16:36.693 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:36 smithi097 ceph-mon[29197]: [23/Apr/2024:10:16:35] ENGINE Bus STARTING 2024-04-23T10:16:36.693 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:36 smithi097 ceph-mon[29197]: from='mgr.14120 172.21.15.97:0/850926340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:16:36.693 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:36 smithi097 ceph-mon[29197]: mgrmap e8: x(active, since 2s) 2024-04-23T10:16:36.693 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:36 smithi097 ceph-mon[29197]: [23/Apr/2024:10:16:35] ENGINE Serving on https://172.21.15.97:7150 2024-04-23T10:16:36.693 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:36 smithi097 ceph-mon[29197]: [23/Apr/2024:10:16:35] ENGINE Serving on http://172.21.15.97:8765 2024-04-23T10:16:36.693 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:36 smithi097 ceph-mon[29197]: [23/Apr/2024:10:16:35] ENGINE Bus STARTED 2024-04-23T10:16:36.694 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:36 smithi097 ceph-mon[29197]: from='mgr.14120 172.21.15.97:0/850926340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:16:37.283 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-23T10:16:37.283 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:36.486+0000 7fa198f48700 1 Processor -- start 2024-04-23T10:16:37.283 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:36.486+0000 7fa198f48700 1 -- start start 2024-04-23T10:16:37.284 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:36.486+0000 7fa198f48700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa194108f40 0x7fa19410b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:16:37.284 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:36.487+0000 7fa198f48700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa19407be80 con 0x7fa194108f40 2024-04-23T10:16:37.284 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:36.487+0000 7fa19259c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa194108f40 0x7fa19410b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:16:37.284 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:36.487+0000 7fa19259c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa194108f40 0x7fa19410b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56576/0 (socket says 172.21.15.97:56576) 2024-04-23T10:16:37.284 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:36.487+0000 7fa19259c700 1 -- 172.21.15.97:0/239898996 learned_addr learned my addr 172.21.15.97:0/239898996 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:16:37.284 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:36.487+0000 7fa19259c700 1 -- 172.21.15.97:0/239898996 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa19410b870 con 0x7fa194108f40 2024-04-23T10:16:37.284 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:36.487+0000 7fa19259c700 1 --2- 172.21.15.97:0/239898996 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa194108f40 0x7fa19410b330 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa1840098a0 tx=0x7fa184009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f0271d736f796d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:16:37.284 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:36.487+0000 7fa19159a700 1 -- 172.21.15.97:0/239898996 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa184004160 con 0x7fa194108f40 2024-04-23T10:16:37.284 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:36.487+0000 7fa19159a700 1 -- 172.21.15.97:0/239898996 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa184004300 con 0x7fa194108f40 2024-04-23T10:16:37.284 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:36.487+0000 7fa19159a700 1 -- 172.21.15.97:0/239898996 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa184011af0 con 0x7fa194108f40 2024-04-23T10:16:37.284 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:36.488+0000 7fa198f48700 1 -- 172.21.15.97:0/239898996 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa194108f40 msgr2=0x7fa19410b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:16:37.285 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:36.488+0000 7fa198f48700 1 --2- 172.21.15.97:0/239898996 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa194108f40 0x7fa19410b330 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa1840098a0 tx=0x7fa184009bb0 comp rx=0 tx=0).stop 2024-04-23T10:16:37.285 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:36.488+0000 7fa198f48700 1 -- 172.21.15.97:0/239898996 shutdown_connections 2024-04-23T10:16:37.285 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:36.488+0000 7fa198f48700 1 --2- 172.21.15.97:0/239898996 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa194108f40 0x7fa19410b330 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:16:37.285 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:36.488+0000 7fa198f48700 1 -- 172.21.15.97:0/239898996 >> 172.21.15.97:0/239898996 conn(0x7fa1941029b0 msgr2=0x7fa194104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:16:37.285 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:36.488+0000 7fa198f48700 1 -- 172.21.15.97:0/239898996 shutdown_connections 2024-04-23T10:16:37.285 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:36.488+0000 7fa198f48700 1 -- 172.21.15.97:0/239898996 wait complete. 2024-04-23T10:16:37.285 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:36.489+0000 7fa198f48700 1 Processor -- start 2024-04-23T10:16:37.285 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:36.489+0000 7fa198f48700 1 -- start start 2024-04-23T10:16:37.285 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:36.489+0000 7fa198f48700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa194108f40 0x7fa19419c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:16:37.285 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:36.489+0000 7fa198f48700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa19410c830 con 0x7fa194108f40 2024-04-23T10:16:37.286 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:36.489+0000 7fa19259c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa194108f40 0x7fa19419c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:16:37.286 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:36.489+0000 7fa19259c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa194108f40 0x7fa19419c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56582/0 (socket says 172.21.15.97:56582) 2024-04-23T10:16:37.286 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:36.489+0000 7fa19259c700 1 -- 172.21.15.97:0/3459148990 learned_addr learned my addr 172.21.15.97:0/3459148990 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:16:37.286 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:36.489+0000 7fa19259c700 1 -- 172.21.15.97:0/3459148990 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa19419c990 con 0x7fa194108f40 2024-04-23T10:16:37.286 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:36.489+0000 7fa19259c700 1 --2- 172.21.15.97:0/3459148990 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa194108f40 0x7fa19419c450 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa184011ac0 tx=0x7fa184011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:16:37.286 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:36.489+0000 7fa1837fe700 1 -- 172.21.15.97:0/3459148990 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa184003970 con 0x7fa194108f40 2024-04-23T10:16:37.286 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:36.489+0000 7fa1837fe700 1 -- 172.21.15.97:0/3459148990 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa184003b10 con 0x7fa194108f40 2024-04-23T10:16:37.286 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:36.489+0000 7fa198f48700 1 -- 172.21.15.97:0/3459148990 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa19419cc20 con 0x7fa194108f40 2024-04-23T10:16:37.286 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:36.490+0000 7fa1837fe700 1 -- 172.21.15.97:0/3459148990 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa184024610 con 0x7fa194108f40 2024-04-23T10:16:37.286 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:36.490+0000 7fa198f48700 1 -- 172.21.15.97:0/3459148990 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa19419f910 con 0x7fa194108f40 2024-04-23T10:16:37.287 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:36.491+0000 7fa1837fe700 1 -- 172.21.15.97:0/3459148990 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fa184024890 con 0x7fa194108f40 2024-04-23T10:16:37.287 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:36.491+0000 7fa198f48700 1 -- 172.21.15.97:0/3459148990 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa194069b90 con 0x7fa194108f40 2024-04-23T10:16:37.287 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:36.491+0000 7fa1837fe700 1 --2- 172.21.15.97:0/3459148990 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7fa17c0395b0 0x7fa17c03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:16:37.287 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:36.491+0000 7fa1837fe700 1 -- 172.21.15.97:0/3459148990 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa18404d930 con 0x7fa194108f40 2024-04-23T10:16:37.287 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:36.491+0000 7fa191d9b700 1 --2- 172.21.15.97:0/3459148990 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7fa17c0395b0 0x7fa17c03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:16:37.287 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:36.491+0000 7fa191d9b700 1 --2- 172.21.15.97:0/3459148990 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7fa17c0395b0 0x7fa17c03ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa188006fd0 tx=0x7fa188006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:16:37.287 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:36.496+0000 7fa1837fe700 1 -- 172.21.15.97:0/3459148990 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa1840172b0 con 0x7fa194108f40 2024-04-23T10:16:37.287 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:36.693+0000 7fa198f48700 1 -- 172.21.15.97:0/3459148990 --> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fa194103c40 con 0x7fa17c0395b0 2024-04-23T10:16:37.287 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:36.694+0000 7fa1837fe700 1 -- 172.21.15.97:0/3459148990 <== mgr.14120 v2:172.21.15.97:6800/3297166858 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fa194103c40 con 0x7fa17c0395b0 2024-04-23T10:16:37.287 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:36.696+0000 7fa198f48700 1 -- 172.21.15.97:0/3459148990 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7fa17c0395b0 msgr2=0x7fa17c03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:16:37.287 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:36.696+0000 7fa198f48700 1 --2- 172.21.15.97:0/3459148990 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7fa17c0395b0 0x7fa17c03ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa188006fd0 tx=0x7fa188006e40 comp rx=0 tx=0).stop 2024-04-23T10:16:37.288 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:36.696+0000 7fa198f48700 1 -- 172.21.15.97:0/3459148990 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa194108f40 msgr2=0x7fa19419c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:16:37.288 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:36.696+0000 7fa198f48700 1 --2- 172.21.15.97:0/3459148990 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa194108f40 0x7fa19419c450 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa184011ac0 tx=0x7fa184011ba0 comp rx=0 tx=0).stop 2024-04-23T10:16:37.288 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:36.696+0000 7fa198f48700 1 -- 172.21.15.97:0/3459148990 shutdown_connections 2024-04-23T10:16:37.288 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:36.696+0000 7fa198f48700 1 --2- 172.21.15.97:0/3459148990 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7fa17c0395b0 0x7fa17c03ba70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:16:37.288 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:36.696+0000 7fa198f48700 1 --2- 172.21.15.97:0/3459148990 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa194108f40 0x7fa19419c450 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:16:37.288 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:36.696+0000 7fa198f48700 1 -- 172.21.15.97:0/3459148990 >> 172.21.15.97:0/3459148990 conn(0x7fa1941029b0 msgr2=0x7fa194103530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:16:37.288 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:36.697+0000 7fa198f48700 1 -- 172.21.15.97:0/3459148990 shutdown_connections 2024-04-23T10:16:37.288 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:36.697+0000 7fa198f48700 1 -- 172.21.15.97:0/3459148990 wait complete. 2024-04-23T10:16:37.288 INFO:teuthology.orchestra.run.smithi097.stdout:Generating ssh key... 2024-04-23T10:16:37.585 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:37 smithi097 ceph-mon[29197]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:16:38.361 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:38 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: Generating public/private rsa key pair. 2024-04-23T10:16:38.361 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:38 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: Your identification has been saved in /tmp/tmpov0jbgrr/key. 2024-04-23T10:16:38.361 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:38 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: Your public key has been saved in /tmp/tmpov0jbgrr/key.pub. 2024-04-23T10:16:38.361 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:38 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: The key fingerprint is: 2024-04-23T10:16:38.361 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:38 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: SHA256:7SPMsmcQeLnyLPLMfom6Scuw+kcW/K8jZAMwUciQbyI ceph-9b22af88-0159-11ef-bc93-c7b262605968 2024-04-23T10:16:38.361 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:38 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: The key's randomart image is: 2024-04-23T10:16:38.361 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:38 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: +---[RSA 3072]----+ 2024-04-23T10:16:38.362 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:38 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: |==. | 2024-04-23T10:16:38.362 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:38 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: |=. | 2024-04-23T10:16:38.362 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:38 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: | + . . . | 2024-04-23T10:16:38.362 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:38 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: |E + + + . | 2024-04-23T10:16:38.362 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:38 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: |.o . + oS . | 2024-04-23T10:16:38.362 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:38 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: | B +o . | 2024-04-23T10:16:38.362 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:38 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: |. .= *.++ o | 2024-04-23T10:16:38.362 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:38 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: | =.== *o+. . | 2024-04-23T10:16:38.362 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:38 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: |+.BB=+o= | 2024-04-23T10:16:38.362 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:38 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: +----[SHA256]-----+ 2024-04-23T10:16:38.748 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:37.682+0000 7f71c4e0c700 1 Processor -- start 2024-04-23T10:16:38.748 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:37.682+0000 7f71c4e0c700 1 -- start start 2024-04-23T10:16:38.749 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:37.683+0000 7f71c4e0c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71c0108fa0 0x7f71c010b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:16:38.749 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:37.683+0000 7f71c4e0c700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f71c007bea0 con 0x7f71c0108fa0 2024-04-23T10:16:38.749 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:37.683+0000 7f71be59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71c0108fa0 0x7f71c010b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:16:38.749 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:37.683+0000 7f71be59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71c0108fa0 0x7f71c010b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56590/0 (socket says 172.21.15.97:56590) 2024-04-23T10:16:38.749 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:37.683+0000 7f71be59c700 1 -- 172.21.15.97:0/738247632 learned_addr learned my addr 172.21.15.97:0/738247632 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:16:38.749 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:37.683+0000 7f71be59c700 1 -- 172.21.15.97:0/738247632 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71c010b8d0 con 0x7f71c0108fa0 2024-04-23T10:16:38.749 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:37.683+0000 7f71be59c700 1 --2- 172.21.15.97:0/738247632 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71c0108fa0 0x7f71c010b390 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f71b40098a0 tx=0x7f71b4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=500e752c0a24bb6f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:16:38.749 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:37.683+0000 7f71bd59a700 1 -- 172.21.15.97:0/738247632 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71b4004160 con 0x7f71c0108fa0 2024-04-23T10:16:38.749 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:37.683+0000 7f71bd59a700 1 -- 172.21.15.97:0/738247632 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f71b4004300 con 0x7f71c0108fa0 2024-04-23T10:16:38.749 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:37.683+0000 7f71bd59a700 1 -- 172.21.15.97:0/738247632 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71b4011af0 con 0x7f71c0108fa0 2024-04-23T10:16:38.750 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:37.684+0000 7f71c4e0c700 1 -- 172.21.15.97:0/738247632 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71c0108fa0 msgr2=0x7f71c010b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:16:38.750 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:37.684+0000 7f71c4e0c700 1 --2- 172.21.15.97:0/738247632 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71c0108fa0 0x7f71c010b390 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f71b40098a0 tx=0x7f71b4009bb0 comp rx=0 tx=0).stop 2024-04-23T10:16:38.750 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:37.684+0000 7f71c4e0c700 1 -- 172.21.15.97:0/738247632 shutdown_connections 2024-04-23T10:16:38.750 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:37.684+0000 7f71c4e0c700 1 --2- 172.21.15.97:0/738247632 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71c0108fa0 0x7f71c010b390 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:16:38.750 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:37.684+0000 7f71c4e0c700 1 -- 172.21.15.97:0/738247632 >> 172.21.15.97:0/738247632 conn(0x7f71c0102a70 msgr2=0x7f71c0104e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:16:38.750 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:37.684+0000 7f71c4e0c700 1 -- 172.21.15.97:0/738247632 shutdown_connections 2024-04-23T10:16:38.750 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:37.684+0000 7f71c4e0c700 1 -- 172.21.15.97:0/738247632 wait complete. 2024-04-23T10:16:38.750 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:37.685+0000 7f71c4e0c700 1 Processor -- start 2024-04-23T10:16:38.750 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:37.685+0000 7f71c4e0c700 1 -- start start 2024-04-23T10:16:38.750 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:37.685+0000 7f71c4e0c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71c0108fa0 0x7f71c019c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:16:38.750 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:37.685+0000 7f71c4e0c700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f71c010c820 con 0x7f71c0108fa0 2024-04-23T10:16:38.751 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:37.685+0000 7f71be59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71c0108fa0 0x7f71c019c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:16:38.751 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:37.685+0000 7f71be59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71c0108fa0 0x7f71c019c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56602/0 (socket says 172.21.15.97:56602) 2024-04-23T10:16:38.751 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:37.685+0000 7f71be59c700 1 -- 172.21.15.97:0/699799806 learned_addr learned my addr 172.21.15.97:0/699799806 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:16:38.751 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:37.685+0000 7f71be59c700 1 -- 172.21.15.97:0/699799806 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71c019cbf0 con 0x7f71c0108fa0 2024-04-23T10:16:38.751 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:37.686+0000 7f71be59c700 1 --2- 172.21.15.97:0/699799806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71c0108fa0 0x7f71c019c6b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f71b4011ac0 tx=0x7f71b4011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:16:38.751 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:37.686+0000 7f71af7fe700 1 -- 172.21.15.97:0/699799806 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71b4003b20 con 0x7f71c0108fa0 2024-04-23T10:16:38.751 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:37.686+0000 7f71af7fe700 1 -- 172.21.15.97:0/699799806 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f71b401a8c0 con 0x7f71c0108fa0 2024-04-23T10:16:38.751 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:37.686+0000 7f71af7fe700 1 -- 172.21.15.97:0/699799806 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71b40248d0 con 0x7f71c0108fa0 2024-04-23T10:16:38.751 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:37.686+0000 7f71c4e0c700 1 -- 172.21.15.97:0/699799806 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71c019cee0 con 0x7f71c0108fa0 2024-04-23T10:16:38.751 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:37.686+0000 7f71c4e0c700 1 -- 172.21.15.97:0/699799806 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71c019fb70 con 0x7f71c0108fa0 2024-04-23T10:16:38.751 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:37.687+0000 7f71af7fe700 1 -- 172.21.15.97:0/699799806 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f71b4022020 con 0x7f71c0108fa0 2024-04-23T10:16:38.751 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:37.687+0000 7f71c4e0c700 1 -- 172.21.15.97:0/699799806 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71c00640a0 con 0x7f71c0108fa0 2024-04-23T10:16:38.752 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:37.687+0000 7f71af7fe700 1 --2- 172.21.15.97:0/699799806 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f71a8039600 0x7f71a803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:16:38.752 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:37.687+0000 7f71af7fe700 1 -- 172.21.15.97:0/699799806 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f71b404d8a0 con 0x7f71c0108fa0 2024-04-23T10:16:38.752 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:37.687+0000 7f71bdd9b700 1 --2- 172.21.15.97:0/699799806 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f71a8039600 0x7f71a803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:16:38.752 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:37.688+0000 7f71bdd9b700 1 --2- 172.21.15.97:0/699799806 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f71a8039600 0x7f71a803bac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f71b0006fd0 tx=0x7f71b0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:16:38.752 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:37.692+0000 7f71af7fe700 1 -- 172.21.15.97:0/699799806 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f71b401ab20 con 0x7f71c0108fa0 2024-04-23T10:16:38.752 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:37.890+0000 7f71c4e0c700 1 -- 172.21.15.97:0/699799806 --> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f71c002e0f0 con 0x7f71a8039600 2024-04-23T10:16:38.752 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:38.119+0000 7f71af7fe700 1 -- 172.21.15.97:0/699799806 <== mgr.14120 v2:172.21.15.97:6800/3297166858 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f71c002e0f0 con 0x7f71a8039600 2024-04-23T10:16:38.752 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:38.121+0000 7f71c4e0c700 1 -- 172.21.15.97:0/699799806 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f71a8039600 msgr2=0x7f71a803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:16:38.752 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:38.121+0000 7f71c4e0c700 1 --2- 172.21.15.97:0/699799806 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f71a8039600 0x7f71a803bac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f71b0006fd0 tx=0x7f71b0006e40 comp rx=0 tx=0).stop 2024-04-23T10:16:38.752 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:38.121+0000 7f71c4e0c700 1 -- 172.21.15.97:0/699799806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71c0108fa0 msgr2=0x7f71c019c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:16:38.752 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:38.121+0000 7f71c4e0c700 1 --2- 172.21.15.97:0/699799806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71c0108fa0 0x7f71c019c6b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f71b4011ac0 tx=0x7f71b4011ba0 comp rx=0 tx=0).stop 2024-04-23T10:16:38.752 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:38.121+0000 7f71c4e0c700 1 -- 172.21.15.97:0/699799806 shutdown_connections 2024-04-23T10:16:38.753 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:38.121+0000 7f71c4e0c700 1 --2- 172.21.15.97:0/699799806 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f71a8039600 0x7f71a803bac0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:16:38.753 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:38.121+0000 7f71c4e0c700 1 --2- 172.21.15.97:0/699799806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71c0108fa0 0x7f71c019c6b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:16:38.753 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:38.121+0000 7f71c4e0c700 1 -- 172.21.15.97:0/699799806 >> 172.21.15.97:0/699799806 conn(0x7f71c0102a70 msgr2=0x7f71c01036e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:16:38.753 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:38.121+0000 7f71c4e0c700 1 -- 172.21.15.97:0/699799806 shutdown_connections 2024-04-23T10:16:38.753 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:38.121+0000 7f71c4e0c700 1 -- 172.21.15.97:0/699799806 wait complete. 2024-04-23T10:16:39.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:39 smithi097 ceph-mon[29197]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:16:39.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:39 smithi097 ceph-mon[29197]: Generating ssh key... 2024-04-23T10:16:39.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:39 smithi097 ceph-mon[29197]: from='mgr.14120 172.21.15.97:0/850926340' entity='mgr.x' 2024-04-23T10:16:39.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:39 smithi097 ceph-mon[29197]: from='mgr.14120 172.21.15.97:0/850926340' entity='mgr.x' 2024-04-23T10:16:39.964 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDwt497yLqXfRoxhJS9uddb2yvoHLs0CXwVBdPVspDDjWxj657SuhLHAsRUGViBz28qtETKZMFeMP0fBm4MkMCoFgZ+2vwKweKGyA2JMA41uVOTBnQrUG2Y6Ehkxrtdpcd619+SghjUIRyF2wfbNQDnQjo9TSAPEsAZfXR1BU8VFOqj3r+CBWkcKhjcA0BqlXFlCE7+UpIpdhf/ep7bp0ADzhP2q0k15Ni97VlVBkD9T4WQRsT5uzSAAqL15ttAUMtOHsDy2Fw6p1n2Veo4n03d7q2rN2CnDCRv/uQOrHjefoW+AHd2drU3oeKpuzFut0GX0q9S4TYBf4TiJAkn81Mky8Ntb1eeamGkDZcMbEc6vhYa6XpNpWmP92uartQKcy4DktFw9d15zhiP6t2T3+eegruggSWEMwaZJ+7anviJY4nuwmuXN3yEqLGM8XShaGzYkv8JYWD/Gm1mumwf+r5ObIx39wunaWl6lh7XvCo2aF3zsKuV24h8qVmsXHPm3hk= ceph-9b22af88-0159-11ef-bc93-c7b262605968 2024-04-23T10:16:39.964 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:39.166+0000 7faeb81fe700 1 Processor -- start 2024-04-23T10:16:39.964 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:39.166+0000 7faeb81fe700 1 -- start start 2024-04-23T10:16:39.964 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:39.166+0000 7faeb81fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faeb01089c0 0x7faeb0108de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:16:39.964 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:39.166+0000 7faeb81fe700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7faeb01093b0 con 0x7faeb01089c0 2024-04-23T10:16:39.964 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:39.166+0000 7faeb5f9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faeb01089c0 0x7faeb0108de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:16:39.964 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:39.166+0000 7faeb5f9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faeb01089c0 0x7faeb0108de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56606/0 (socket says 172.21.15.97:56606) 2024-04-23T10:16:39.964 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:39.166+0000 7faeb5f9a700 1 -- 172.21.15.97:0/3084172488 learned_addr learned my addr 172.21.15.97:0/3084172488 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:16:39.964 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:39.166+0000 7faeb5f9a700 1 -- 172.21.15.97:0/3084172488 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faeb0109530 con 0x7faeb01089c0 2024-04-23T10:16:39.964 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:39.167+0000 7faeb5f9a700 1 --2- 172.21.15.97:0/3084172488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faeb01089c0 0x7faeb0108de0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7faeac0098a0 tx=0x7faeac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd32ec05bf821aff server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:16:39.964 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:39.167+0000 7faeb4f98700 1 -- 172.21.15.97:0/3084172488 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faeac004160 con 0x7faeb01089c0 2024-04-23T10:16:39.965 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:39.167+0000 7faeb4f98700 1 -- 172.21.15.97:0/3084172488 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7faeac004300 con 0x7faeb01089c0 2024-04-23T10:16:39.965 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:39.167+0000 7faeb4f98700 1 -- 172.21.15.97:0/3084172488 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faeac011af0 con 0x7faeb01089c0 2024-04-23T10:16:39.965 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:39.167+0000 7faeb81fe700 1 -- 172.21.15.97:0/3084172488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faeb01089c0 msgr2=0x7faeb0108de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:16:39.965 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:39.167+0000 7faeb81fe700 1 --2- 172.21.15.97:0/3084172488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faeb01089c0 0x7faeb0108de0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7faeac0098a0 tx=0x7faeac009bb0 comp rx=0 tx=0).stop 2024-04-23T10:16:39.965 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:39.167+0000 7faeb81fe700 1 -- 172.21.15.97:0/3084172488 shutdown_connections 2024-04-23T10:16:39.965 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:39.167+0000 7faeb81fe700 1 --2- 172.21.15.97:0/3084172488 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faeb01089c0 0x7faeb0108de0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:16:39.965 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:39.167+0000 7faeb81fe700 1 -- 172.21.15.97:0/3084172488 >> 172.21.15.97:0/3084172488 conn(0x7faeb01021b0 msgr2=0x7faeb01045d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:16:39.965 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:39.167+0000 7faeb81fe700 1 -- 172.21.15.97:0/3084172488 shutdown_connections 2024-04-23T10:16:39.965 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:39.167+0000 7faeb81fe700 1 -- 172.21.15.97:0/3084172488 wait complete. 2024-04-23T10:16:39.965 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:39.168+0000 7faeb81fe700 1 Processor -- start 2024-04-23T10:16:39.965 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:39.168+0000 7faeb81fe700 1 -- start start 2024-04-23T10:16:39.965 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:39.168+0000 7faeb81fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faeb01089c0 0x7faeb019e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:16:39.965 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:39.168+0000 7faeb81fe700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7faeb0107320 con 0x7faeb01089c0 2024-04-23T10:16:39.966 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:39.168+0000 7faeb5f9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faeb01089c0 0x7faeb019e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:16:39.966 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:39.168+0000 7faeb5f9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faeb01089c0 0x7faeb019e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56618/0 (socket says 172.21.15.97:56618) 2024-04-23T10:16:39.966 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:39.168+0000 7faeb5f9a700 1 -- 172.21.15.97:0/3029266411 learned_addr learned my addr 172.21.15.97:0/3029266411 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:16:39.966 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:39.168+0000 7faeb5f9a700 1 -- 172.21.15.97:0/3029266411 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faeb019edf0 con 0x7faeb01089c0 2024-04-23T10:16:39.966 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:39.169+0000 7faeb5f9a700 1 --2- 172.21.15.97:0/3029266411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faeb01089c0 0x7faeb019e8b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7faeac003f40 tx=0x7faeac011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:16:39.966 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:39.169+0000 7faea6ffd700 1 -- 172.21.15.97:0/3029266411 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faeac0038b0 con 0x7faeb01089c0 2024-04-23T10:16:39.966 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:39.169+0000 7faea6ffd700 1 -- 172.21.15.97:0/3029266411 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7faeac003a50 con 0x7faeb01089c0 2024-04-23T10:16:39.966 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:39.169+0000 7faeb81fe700 1 -- 172.21.15.97:0/3029266411 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faeb019f0e0 con 0x7faeb01089c0 2024-04-23T10:16:39.966 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:39.169+0000 7faea6ffd700 1 -- 172.21.15.97:0/3029266411 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faeac01a5c0 con 0x7faeb01089c0 2024-04-23T10:16:39.966 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:39.169+0000 7faeb81fe700 1 -- 172.21.15.97:0/3029266411 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faeb01a1d70 con 0x7faeb01089c0 2024-04-23T10:16:39.966 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:39.170+0000 7faea6ffd700 1 -- 172.21.15.97:0/3029266411 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7faeac024430 con 0x7faeb01089c0 2024-04-23T10:16:39.966 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:39.170+0000 7faeb81fe700 1 -- 172.21.15.97:0/3029266411 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faeb00640a0 con 0x7faeb01089c0 2024-04-23T10:16:39.967 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:39.170+0000 7faea6ffd700 1 --2- 172.21.15.97:0/3029266411 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7fae9c0395b0 0x7fae9c03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:16:39.967 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:39.170+0000 7faea6ffd700 1 -- 172.21.15.97:0/3029266411 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7faeac04da40 con 0x7faeb01089c0 2024-04-23T10:16:39.967 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:39.170+0000 7faeb5799700 1 --2- 172.21.15.97:0/3029266411 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7fae9c0395b0 0x7fae9c03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:16:39.967 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:39.170+0000 7faeb5799700 1 --2- 172.21.15.97:0/3029266411 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7fae9c0395b0 0x7fae9c03ba70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7faea0006fd0 tx=0x7faea0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:16:39.967 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:39.175+0000 7faea6ffd700 1 -- 172.21.15.97:0/3029266411 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faeac02d3f0 con 0x7faeb01089c0 2024-04-23T10:16:39.967 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:39.371+0000 7faeb81fe700 1 -- 172.21.15.97:0/3029266411 --> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7faeb0062e80 con 0x7fae9c0395b0 2024-04-23T10:16:39.967 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:39.372+0000 7faea6ffd700 1 -- 172.21.15.97:0/3029266411 <== mgr.14120 v2:172.21.15.97:6800/3297166858 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7faeb0062e80 con 0x7fae9c0395b0 2024-04-23T10:16:39.968 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:39.374+0000 7faeb81fe700 1 -- 172.21.15.97:0/3029266411 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7fae9c0395b0 msgr2=0x7fae9c03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:16:39.968 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:39.374+0000 7faeb81fe700 1 --2- 172.21.15.97:0/3029266411 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7fae9c0395b0 0x7fae9c03ba70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7faea0006fd0 tx=0x7faea0006e40 comp rx=0 tx=0).stop 2024-04-23T10:16:39.968 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:39.374+0000 7faeb81fe700 1 -- 172.21.15.97:0/3029266411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faeb01089c0 msgr2=0x7faeb019e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:16:39.968 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:39.374+0000 7faeb81fe700 1 --2- 172.21.15.97:0/3029266411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faeb01089c0 0x7faeb019e8b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7faeac003f40 tx=0x7faeac011890 comp rx=0 tx=0).stop 2024-04-23T10:16:39.968 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:39.374+0000 7faeb81fe700 1 -- 172.21.15.97:0/3029266411 shutdown_connections 2024-04-23T10:16:39.968 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:39.374+0000 7faeb81fe700 1 --2- 172.21.15.97:0/3029266411 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7fae9c0395b0 0x7fae9c03ba70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:16:39.968 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:39.374+0000 7faeb81fe700 1 --2- 172.21.15.97:0/3029266411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faeb01089c0 0x7faeb019e8b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:16:39.968 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:39.374+0000 7faeb81fe700 1 -- 172.21.15.97:0/3029266411 >> 172.21.15.97:0/3029266411 conn(0x7faeb01021b0 msgr2=0x7faeb0194ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:16:39.968 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:39.374+0000 7faeb81fe700 1 -- 172.21.15.97:0/3029266411 shutdown_connections 2024-04-23T10:16:39.968 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:39.374+0000 7faeb81fe700 1 -- 172.21.15.97:0/3029266411 wait complete. 2024-04-23T10:16:39.968 INFO:teuthology.orchestra.run.smithi097.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-23T10:16:39.968 INFO:teuthology.orchestra.run.smithi097.stdout:Adding key to root@localhost authorized_keys... 2024-04-23T10:16:39.969 INFO:teuthology.orchestra.run.smithi097.stdout:Adding host smithi097... 2024-04-23T10:16:40.598 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:40 smithi097 ceph-mon[29197]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:16:41.429 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:41 smithi097 ceph-mon[29197]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi097", "addr": "172.21.15.97", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:16:41.430 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:41 smithi097 ceph-mon[29197]: Deploying cephadm binary to smithi097 2024-04-23T10:16:45.248 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout Added host 'smithi097' with addr '172.21.15.97' 2024-04-23T10:16:45.248 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:40.381+0000 7f216d93c700 1 Processor -- start 2024-04-23T10:16:45.248 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:40.381+0000 7f216d93c700 1 -- start start 2024-04-23T10:16:45.249 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:40.382+0000 7f216d93c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2168108510 0x7f2168108930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:16:45.249 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:40.382+0000 7f216d93c700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2168108f00 con 0x7f2168108510 2024-04-23T10:16:45.249 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:40.382+0000 7f2166ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2168108510 0x7f2168108930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:16:45.249 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:40.382+0000 7f2166ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2168108510 0x7f2168108930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56626/0 (socket says 172.21.15.97:56626) 2024-04-23T10:16:45.249 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:40.382+0000 7f2166ffd700 1 -- 172.21.15.97:0/3652163629 learned_addr learned my addr 172.21.15.97:0/3652163629 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:16:45.249 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:40.382+0000 7f2166ffd700 1 -- 172.21.15.97:0/3652163629 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21681096c0 con 0x7f2168108510 2024-04-23T10:16:45.249 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:40.382+0000 7f2166ffd700 1 --2- 172.21.15.97:0/3652163629 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2168108510 0x7f2168108930 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f21580098a0 tx=0x7f2158009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6f9b6706c8dbb05 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:16:45.249 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:40.382+0000 7f2165ffb700 1 -- 172.21.15.97:0/3652163629 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2158004160 con 0x7f2168108510 2024-04-23T10:16:45.249 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:40.382+0000 7f2165ffb700 1 -- 172.21.15.97:0/3652163629 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2158004300 con 0x7f2168108510 2024-04-23T10:16:45.249 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:40.382+0000 7f2165ffb700 1 -- 172.21.15.97:0/3652163629 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2158011af0 con 0x7f2168108510 2024-04-23T10:16:45.249 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:40.383+0000 7f216d93c700 1 -- 172.21.15.97:0/3652163629 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2168108510 msgr2=0x7f2168108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:16:45.250 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:40.383+0000 7f216d93c700 1 --2- 172.21.15.97:0/3652163629 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2168108510 0x7f2168108930 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f21580098a0 tx=0x7f2158009bb0 comp rx=0 tx=0).stop 2024-04-23T10:16:45.250 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:40.383+0000 7f216d93c700 1 -- 172.21.15.97:0/3652163629 shutdown_connections 2024-04-23T10:16:45.250 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:40.383+0000 7f216d93c700 1 --2- 172.21.15.97:0/3652163629 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2168108510 0x7f2168108930 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:16:45.250 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:40.383+0000 7f216d93c700 1 -- 172.21.15.97:0/3652163629 >> 172.21.15.97:0/3652163629 conn(0x7f2168103b70 msgr2=0x7f2168105f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:16:45.250 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:40.383+0000 7f216d93c700 1 -- 172.21.15.97:0/3652163629 shutdown_connections 2024-04-23T10:16:45.250 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:40.383+0000 7f216d93c700 1 -- 172.21.15.97:0/3652163629 wait complete. 2024-04-23T10:16:45.250 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:40.384+0000 7f216d93c700 1 Processor -- start 2024-04-23T10:16:45.250 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:40.384+0000 7f216d93c700 1 -- start start 2024-04-23T10:16:45.250 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:40.384+0000 7f216d93c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2168108510 0x7f216819c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:16:45.250 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:40.384+0000 7f216d93c700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f216810ca70 con 0x7f2168108510 2024-04-23T10:16:45.251 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:40.384+0000 7f2166ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2168108510 0x7f216819c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:16:45.251 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:40.384+0000 7f2166ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2168108510 0x7f216819c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56640/0 (socket says 172.21.15.97:56640) 2024-04-23T10:16:45.251 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:40.384+0000 7f2166ffd700 1 -- 172.21.15.97:0/3290509589 learned_addr learned my addr 172.21.15.97:0/3290509589 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:16:45.251 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:40.384+0000 7f2166ffd700 1 -- 172.21.15.97:0/3290509589 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f216819ccc0 con 0x7f2168108510 2024-04-23T10:16:45.251 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:40.385+0000 7f2166ffd700 1 --2- 172.21.15.97:0/3290509589 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2168108510 0x7f216819c780 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2158000c00 tx=0x7f2158011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:16:45.251 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:40.385+0000 7f216c93a700 1 -- 172.21.15.97:0/3290509589 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2158003b20 con 0x7f2168108510 2024-04-23T10:16:45.251 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:40.385+0000 7f216c93a700 1 -- 172.21.15.97:0/3290509589 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f215801a8c0 con 0x7f2168108510 2024-04-23T10:16:45.251 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:40.385+0000 7f216d93c700 1 -- 172.21.15.97:0/3290509589 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f216819cf50 con 0x7f2168108510 2024-04-23T10:16:45.251 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:40.385+0000 7f216c93a700 1 -- 172.21.15.97:0/3290509589 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2158024940 con 0x7f2168108510 2024-04-23T10:16:45.251 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:40.385+0000 7f216d93c700 1 -- 172.21.15.97:0/3290509589 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f216819fbf0 con 0x7f2168108510 2024-04-23T10:16:45.251 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:40.386+0000 7f216c93a700 1 -- 172.21.15.97:0/3290509589 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f2158022020 con 0x7f2168108510 2024-04-23T10:16:45.252 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:40.386+0000 7f216d93c700 1 -- 172.21.15.97:0/3290509589 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21680640a0 con 0x7f2168108510 2024-04-23T10:16:45.252 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:40.386+0000 7f216c93a700 1 --2- 172.21.15.97:0/3290509589 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f21500395b0 0x7f215003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:16:45.252 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:40.386+0000 7f216c93a700 1 -- 172.21.15.97:0/3290509589 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f215804d820 con 0x7f2168108510 2024-04-23T10:16:45.252 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:40.386+0000 7f21667fc700 1 --2- 172.21.15.97:0/3290509589 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f21500395b0 0x7f215003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:16:45.252 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:40.386+0000 7f21667fc700 1 --2- 172.21.15.97:0/3290509589 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f21500395b0 0x7f215003ba70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f215c006fd0 tx=0x7f215c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:16:45.252 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:40.391+0000 7f216c93a700 1 -- 172.21.15.97:0/3290509589 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f215800b3e0 con 0x7f2168108510 2024-04-23T10:16:45.252 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:40.598+0000 7f216d93c700 1 -- 172.21.15.97:0/3290509589 --> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi097", "addr": "172.21.15.97", "target": ["mon-mgr", ""]}) -- 0x7f216802dff0 con 0x7f21500395b0 2024-04-23T10:16:45.252 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:44.605+0000 7f216c93a700 1 -- 172.21.15.97:0/3290509589 <== mgr.14120 v2:172.21.15.97:6800/3297166858 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f216802dff0 con 0x7f21500395b0 2024-04-23T10:16:45.252 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:44.608+0000 7f216d93c700 1 -- 172.21.15.97:0/3290509589 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f21500395b0 msgr2=0x7f215003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:16:45.252 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:44.608+0000 7f216d93c700 1 --2- 172.21.15.97:0/3290509589 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f21500395b0 0x7f215003ba70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f215c006fd0 tx=0x7f215c006e40 comp rx=0 tx=0).stop 2024-04-23T10:16:45.253 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:44.608+0000 7f216d93c700 1 -- 172.21.15.97:0/3290509589 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2168108510 msgr2=0x7f216819c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:16:45.253 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:44.608+0000 7f216d93c700 1 --2- 172.21.15.97:0/3290509589 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2168108510 0x7f216819c780 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2158000c00 tx=0x7f2158011b70 comp rx=0 tx=0).stop 2024-04-23T10:16:45.253 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:44.608+0000 7f216d93c700 1 -- 172.21.15.97:0/3290509589 shutdown_connections 2024-04-23T10:16:45.253 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:44.608+0000 7f216d93c700 1 --2- 172.21.15.97:0/3290509589 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f21500395b0 0x7f215003ba70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:16:45.253 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:44.608+0000 7f216d93c700 1 --2- 172.21.15.97:0/3290509589 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2168108510 0x7f216819c780 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:16:45.253 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:44.608+0000 7f216d93c700 1 -- 172.21.15.97:0/3290509589 >> 172.21.15.97:0/3290509589 conn(0x7f2168103b70 msgr2=0x7f21681048a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:16:45.253 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:44.608+0000 7f216d93c700 1 -- 172.21.15.97:0/3290509589 shutdown_connections 2024-04-23T10:16:45.253 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:44.608+0000 7f216d93c700 1 -- 172.21.15.97:0/3290509589 wait complete. 2024-04-23T10:16:45.253 INFO:teuthology.orchestra.run.smithi097.stdout:Deploying unmanaged mon service... 2024-04-23T10:16:45.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:45 smithi097 ceph-mon[29197]: from='mgr.14120 172.21.15.97:0/850926340' entity='mgr.x' 2024-04-23T10:16:45.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:45 smithi097 ceph-mon[29197]: Added host smithi097 2024-04-23T10:16:45.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:45 smithi097 ceph-mon[29197]: from='mgr.14120 172.21.15.97:0/850926340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:16:46.657 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-23T10:16:46.657 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:45.660+0000 7f2f32de9700 1 Processor -- start 2024-04-23T10:16:46.657 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:45.660+0000 7f2f32de9700 1 -- start start 2024-04-23T10:16:46.657 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:45.660+0000 7f2f32de9700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f2c108f60 0x7f2f2c10b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:16:46.657 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:45.660+0000 7f2f32de9700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2f2c07be60 con 0x7f2f2c108f60 2024-04-23T10:16:46.657 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:45.660+0000 7f2f30b85700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f2c108f60 0x7f2f2c10b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:16:46.657 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:45.660+0000 7f2f30b85700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f2c108f60 0x7f2f2c10b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35572/0 (socket says 172.21.15.97:35572) 2024-04-23T10:16:46.657 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:45.660+0000 7f2f30b85700 1 -- 172.21.15.97:0/2692010566 learned_addr learned my addr 172.21.15.97:0/2692010566 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:16:46.658 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:45.660+0000 7f2f30b85700 1 -- 172.21.15.97:0/2692010566 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f2c10b890 con 0x7f2f2c108f60 2024-04-23T10:16:46.658 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:45.661+0000 7f2f30b85700 1 --2- 172.21.15.97:0/2692010566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f2c108f60 0x7f2f2c10b350 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2f200098a0 tx=0x7f2f20009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10a154af5cf89bf1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:16:46.658 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:45.661+0000 7f2f2b7fe700 1 -- 172.21.15.97:0/2692010566 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f20004160 con 0x7f2f2c108f60 2024-04-23T10:16:46.658 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:45.661+0000 7f2f2b7fe700 1 -- 172.21.15.97:0/2692010566 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2f20004300 con 0x7f2f2c108f60 2024-04-23T10:16:46.658 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:45.661+0000 7f2f2b7fe700 1 -- 172.21.15.97:0/2692010566 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f20011af0 con 0x7f2f2c108f60 2024-04-23T10:16:46.658 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:45.661+0000 7f2f32de9700 1 -- 172.21.15.97:0/2692010566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f2c108f60 msgr2=0x7f2f2c10b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:16:46.658 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:45.661+0000 7f2f32de9700 1 --2- 172.21.15.97:0/2692010566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f2c108f60 0x7f2f2c10b350 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2f200098a0 tx=0x7f2f20009bb0 comp rx=0 tx=0).stop 2024-04-23T10:16:46.658 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:45.661+0000 7f2f32de9700 1 -- 172.21.15.97:0/2692010566 shutdown_connections 2024-04-23T10:16:46.658 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:45.661+0000 7f2f32de9700 1 --2- 172.21.15.97:0/2692010566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f2c108f60 0x7f2f2c10b350 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:16:46.658 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:45.661+0000 7f2f32de9700 1 -- 172.21.15.97:0/2692010566 >> 172.21.15.97:0/2692010566 conn(0x7f2f2c1029b0 msgr2=0x7f2f2c104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:16:46.659 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:45.661+0000 7f2f32de9700 1 -- 172.21.15.97:0/2692010566 shutdown_connections 2024-04-23T10:16:46.659 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:45.661+0000 7f2f32de9700 1 -- 172.21.15.97:0/2692010566 wait complete. 2024-04-23T10:16:46.659 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:45.662+0000 7f2f32de9700 1 Processor -- start 2024-04-23T10:16:46.659 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:45.662+0000 7f2f32de9700 1 -- start start 2024-04-23T10:16:46.659 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:45.662+0000 7f2f32de9700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f2c108f60 0x7f2f2c19c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:16:46.659 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:45.662+0000 7f2f32de9700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2f2c10c800 con 0x7f2f2c108f60 2024-04-23T10:16:46.659 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:45.662+0000 7f2f30b85700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f2c108f60 0x7f2f2c19c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:16:46.659 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:45.662+0000 7f2f30b85700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f2c108f60 0x7f2f2c19c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35580/0 (socket says 172.21.15.97:35580) 2024-04-23T10:16:46.659 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:45.662+0000 7f2f30b85700 1 -- 172.21.15.97:0/4263138594 learned_addr learned my addr 172.21.15.97:0/4263138594 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:16:46.659 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:45.662+0000 7f2f30b85700 1 -- 172.21.15.97:0/4263138594 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f2c19c9a0 con 0x7f2f2c108f60 2024-04-23T10:16:46.659 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:45.663+0000 7f2f30b85700 1 --2- 172.21.15.97:0/4263138594 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f2c108f60 0x7f2f2c19c460 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2f20009fb0 tx=0x7f2f20011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:16:46.659 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:45.663+0000 7f2f29ffb700 1 -- 172.21.15.97:0/4263138594 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f200036a0 con 0x7f2f2c108f60 2024-04-23T10:16:46.660 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:45.663+0000 7f2f29ffb700 1 -- 172.21.15.97:0/4263138594 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2f2001a430 con 0x7f2f2c108f60 2024-04-23T10:16:46.660 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:45.663+0000 7f2f29ffb700 1 -- 172.21.15.97:0/4263138594 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f20019550 con 0x7f2f2c108f60 2024-04-23T10:16:46.660 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:45.663+0000 7f2f32de9700 1 -- 172.21.15.97:0/4263138594 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f2c19cc30 con 0x7f2f2c108f60 2024-04-23T10:16:46.660 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:45.663+0000 7f2f32de9700 1 -- 172.21.15.97:0/4263138594 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f2c19f920 con 0x7f2f2c108f60 2024-04-23T10:16:46.660 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:45.664+0000 7f2f29ffb700 1 -- 172.21.15.97:0/4263138594 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f2f2001a8e0 con 0x7f2f2c108f60 2024-04-23T10:16:46.660 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:45.664+0000 7f2f32de9700 1 -- 172.21.15.97:0/4263138594 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f2c0640a0 con 0x7f2f2c108f60 2024-04-23T10:16:46.660 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:45.664+0000 7f2f29ffb700 1 --2- 172.21.15.97:0/4263138594 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f2f1c039560 0x7f2f1c03ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:16:46.660 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:45.664+0000 7f2f29ffb700 1 -- 172.21.15.97:0/4263138594 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2f2004d940 con 0x7f2f2c108f60 2024-04-23T10:16:46.660 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:45.664+0000 7f2f2bfff700 1 --2- 172.21.15.97:0/4263138594 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f2f1c039560 0x7f2f1c03ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:16:46.660 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:45.665+0000 7f2f2bfff700 1 --2- 172.21.15.97:0/4263138594 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f2f1c039560 0x7f2f1c03ba20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2f18006fd0 tx=0x7f2f18006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:16:46.660 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:45.669+0000 7f2f29ffb700 1 -- 172.21.15.97:0/4263138594 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2f2000b3e0 con 0x7f2f2c108f60 2024-04-23T10:16:46.660 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:45.926+0000 7f2f32de9700 1 -- 172.21.15.97:0/4263138594 --> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f2f2c103ca0 con 0x7f2f1c039560 2024-04-23T10:16:46.661 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:45.934+0000 7f2f29ffb700 1 -- 172.21.15.97:0/4263138594 <== mgr.14120 v2:172.21.15.97:6800/3297166858 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f2f2c103ca0 con 0x7f2f1c039560 2024-04-23T10:16:46.661 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:45.936+0000 7f2f32de9700 1 -- 172.21.15.97:0/4263138594 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f2f1c039560 msgr2=0x7f2f1c03ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:16:46.661 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:45.936+0000 7f2f32de9700 1 --2- 172.21.15.97:0/4263138594 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f2f1c039560 0x7f2f1c03ba20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2f18006fd0 tx=0x7f2f18006e40 comp rx=0 tx=0).stop 2024-04-23T10:16:46.661 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:45.936+0000 7f2f32de9700 1 -- 172.21.15.97:0/4263138594 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f2c108f60 msgr2=0x7f2f2c19c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:16:46.661 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:45.936+0000 7f2f32de9700 1 --2- 172.21.15.97:0/4263138594 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f2c108f60 0x7f2f2c19c460 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2f20009fb0 tx=0x7f2f20011940 comp rx=0 tx=0).stop 2024-04-23T10:16:46.661 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:45.936+0000 7f2f32de9700 1 -- 172.21.15.97:0/4263138594 shutdown_connections 2024-04-23T10:16:46.661 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:45.936+0000 7f2f32de9700 1 --2- 172.21.15.97:0/4263138594 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f2f1c039560 0x7f2f1c03ba20 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:16:46.661 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:45.936+0000 7f2f32de9700 1 --2- 172.21.15.97:0/4263138594 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f2c108f60 0x7f2f2c19c460 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:16:46.661 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:45.936+0000 7f2f32de9700 1 -- 172.21.15.97:0/4263138594 >> 172.21.15.97:0/4263138594 conn(0x7f2f2c1029b0 msgr2=0x7f2f2c103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:16:46.661 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:45.936+0000 7f2f32de9700 1 -- 172.21.15.97:0/4263138594 shutdown_connections 2024-04-23T10:16:46.661 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:45.936+0000 7f2f32de9700 1 -- 172.21.15.97:0/4263138594 wait complete. 2024-04-23T10:16:46.661 INFO:teuthology.orchestra.run.smithi097.stdout:Deploying unmanaged mgr service... 2024-04-23T10:16:47.221 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:46 smithi097 ceph-mon[29197]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:16:47.221 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:46 smithi097 ceph-mon[29197]: Saving service mon spec with placement count:5 2024-04-23T10:16:47.221 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:46 smithi097 ceph-mon[29197]: from='mgr.14120 172.21.15.97:0/850926340' entity='mgr.x' 2024-04-23T10:16:48.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:48 smithi097 ceph-mon[29197]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:16:48.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:48 smithi097 ceph-mon[29197]: Saving service mgr spec with placement count:2 2024-04-23T10:16:48.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:48 smithi097 ceph-mon[29197]: from='mgr.14120 172.21.15.97:0/850926340' entity='mgr.x' 2024-04-23T10:16:48.612 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:48 smithi097 ceph-mon[29197]: from='mgr.14120 172.21.15.97:0/850926340' entity='mgr.x' 2024-04-23T10:16:48.695 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-23T10:16:48.695 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:47.078+0000 7f3c3ccbe700 1 Processor -- start 2024-04-23T10:16:48.695 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:47.079+0000 7f3c3ccbe700 1 -- start start 2024-04-23T10:16:48.695 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:47.079+0000 7f3c3ccbe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c38108510 0x7f3c38108930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:16:48.695 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:47.079+0000 7f3c3ccbe700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3c38108f00 con 0x7f3c38108510 2024-04-23T10:16:48.695 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:47.079+0000 7f3c3659c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c38108510 0x7f3c38108930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:16:48.695 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:47.079+0000 7f3c3659c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c38108510 0x7f3c38108930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35588/0 (socket says 172.21.15.97:35588) 2024-04-23T10:16:48.696 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:47.079+0000 7f3c3659c700 1 -- 172.21.15.97:0/2359317473 learned_addr learned my addr 172.21.15.97:0/2359317473 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:16:48.696 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:47.079+0000 7f3c3659c700 1 -- 172.21.15.97:0/2359317473 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c381096c0 con 0x7f3c38108510 2024-04-23T10:16:48.696 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:47.079+0000 7f3c3659c700 1 --2- 172.21.15.97:0/2359317473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c38108510 0x7f3c38108930 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3c280098a0 tx=0x7f3c28009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa50e95242cc7c3f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:16:48.696 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:47.080+0000 7f3c3559a700 1 -- 172.21.15.97:0/2359317473 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c28004160 con 0x7f3c38108510 2024-04-23T10:16:48.696 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:47.080+0000 7f3c3559a700 1 -- 172.21.15.97:0/2359317473 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3c28004300 con 0x7f3c38108510 2024-04-23T10:16:48.696 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:47.080+0000 7f3c3559a700 1 -- 172.21.15.97:0/2359317473 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c28011af0 con 0x7f3c38108510 2024-04-23T10:16:48.696 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:47.080+0000 7f3c3ccbe700 1 -- 172.21.15.97:0/2359317473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c38108510 msgr2=0x7f3c38108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:16:48.696 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:47.080+0000 7f3c3ccbe700 1 --2- 172.21.15.97:0/2359317473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c38108510 0x7f3c38108930 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3c280098a0 tx=0x7f3c28009bb0 comp rx=0 tx=0).stop 2024-04-23T10:16:48.696 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:47.080+0000 7f3c3ccbe700 1 -- 172.21.15.97:0/2359317473 shutdown_connections 2024-04-23T10:16:48.697 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:47.080+0000 7f3c3ccbe700 1 --2- 172.21.15.97:0/2359317473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c38108510 0x7f3c38108930 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:16:48.697 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:47.080+0000 7f3c3ccbe700 1 -- 172.21.15.97:0/2359317473 >> 172.21.15.97:0/2359317473 conn(0x7f3c38103b70 msgr2=0x7f3c38105f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:16:48.697 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:47.080+0000 7f3c3ccbe700 1 -- 172.21.15.97:0/2359317473 shutdown_connections 2024-04-23T10:16:48.697 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:47.080+0000 7f3c3ccbe700 1 -- 172.21.15.97:0/2359317473 wait complete. 2024-04-23T10:16:48.697 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:47.081+0000 7f3c3ccbe700 1 Processor -- start 2024-04-23T10:16:48.697 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:47.081+0000 7f3c3ccbe700 1 -- start start 2024-04-23T10:16:48.697 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:47.081+0000 7f3c3ccbe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c38108510 0x7f3c3819c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:16:48.697 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:47.081+0000 7f3c3ccbe700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3c3810ca70 con 0x7f3c38108510 2024-04-23T10:16:48.697 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:47.081+0000 7f3c3659c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c38108510 0x7f3c3819c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:16:48.697 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:47.082+0000 7f3c3659c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c38108510 0x7f3c3819c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35594/0 (socket says 172.21.15.97:35594) 2024-04-23T10:16:48.697 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:47.082+0000 7f3c3659c700 1 -- 172.21.15.97:0/3796884441 learned_addr learned my addr 172.21.15.97:0/3796884441 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:16:48.698 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:47.082+0000 7f3c3659c700 1 -- 172.21.15.97:0/3796884441 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c3819cca0 con 0x7f3c38108510 2024-04-23T10:16:48.698 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:47.082+0000 7f3c3659c700 1 --2- 172.21.15.97:0/3796884441 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c38108510 0x7f3c3819c760 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3c28000c00 tx=0x7f3c28011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:16:48.698 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:47.082+0000 7f3c277fe700 1 -- 172.21.15.97:0/3796884441 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c28003b20 con 0x7f3c38108510 2024-04-23T10:16:48.698 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:47.082+0000 7f3c277fe700 1 -- 172.21.15.97:0/3796884441 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3c2801a8c0 con 0x7f3c38108510 2024-04-23T10:16:48.698 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:47.082+0000 7f3c3ccbe700 1 -- 172.21.15.97:0/3796884441 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c3819cf30 con 0x7f3c38108510 2024-04-23T10:16:48.698 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:47.082+0000 7f3c277fe700 1 -- 172.21.15.97:0/3796884441 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c28024940 con 0x7f3c38108510 2024-04-23T10:16:48.698 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:47.082+0000 7f3c3ccbe700 1 -- 172.21.15.97:0/3796884441 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c3819fbd0 con 0x7f3c38108510 2024-04-23T10:16:48.698 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:47.083+0000 7f3c277fe700 1 -- 172.21.15.97:0/3796884441 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f3c28022020 con 0x7f3c38108510 2024-04-23T10:16:48.699 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:47.083+0000 7f3c3ccbe700 1 -- 172.21.15.97:0/3796884441 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c380640a0 con 0x7f3c38108510 2024-04-23T10:16:48.699 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:47.083+0000 7f3c277fe700 1 --2- 172.21.15.97:0/3796884441 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f3c200395b0 0x7f3c2003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:16:48.699 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:47.083+0000 7f3c277fe700 1 -- 172.21.15.97:0/3796884441 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3c2804d800 con 0x7f3c38108510 2024-04-23T10:16:48.699 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:47.083+0000 7f3c35d9b700 1 --2- 172.21.15.97:0/3796884441 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f3c200395b0 0x7f3c2003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:16:48.699 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:47.084+0000 7f3c35d9b700 1 --2- 172.21.15.97:0/3796884441 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f3c200395b0 0x7f3c2003ba70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c006fd0 tx=0x7f3c2c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:16:48.699 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:47.088+0000 7f3c277fe700 1 -- 172.21.15.97:0/3796884441 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3c28021d80 con 0x7f3c38108510 2024-04-23T10:16:48.699 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:47.303+0000 7f3c3ccbe700 1 -- 172.21.15.97:0/3796884441 --> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f3c3802dff0 con 0x7f3c200395b0 2024-04-23T10:16:48.699 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:47.310+0000 7f3c277fe700 1 -- 172.21.15.97:0/3796884441 <== mgr.14120 v2:172.21.15.97:6800/3297166858 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f3c3802dff0 con 0x7f3c200395b0 2024-04-23T10:16:48.699 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:47.312+0000 7f3c3ccbe700 1 -- 172.21.15.97:0/3796884441 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f3c200395b0 msgr2=0x7f3c2003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:16:48.699 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:47.312+0000 7f3c3ccbe700 1 --2- 172.21.15.97:0/3796884441 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f3c200395b0 0x7f3c2003ba70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c006fd0 tx=0x7f3c2c006e40 comp rx=0 tx=0).stop 2024-04-23T10:16:48.700 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:47.312+0000 7f3c3ccbe700 1 -- 172.21.15.97:0/3796884441 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c38108510 msgr2=0x7f3c3819c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:16:48.700 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:47.312+0000 7f3c3ccbe700 1 --2- 172.21.15.97:0/3796884441 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c38108510 0x7f3c3819c760 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3c28000c00 tx=0x7f3c28011b70 comp rx=0 tx=0).stop 2024-04-23T10:16:48.700 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:47.313+0000 7f3c3ccbe700 1 -- 172.21.15.97:0/3796884441 shutdown_connections 2024-04-23T10:16:48.700 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:47.313+0000 7f3c3ccbe700 1 --2- 172.21.15.97:0/3796884441 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f3c200395b0 0x7f3c2003ba70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:16:48.700 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:47.313+0000 7f3c3ccbe700 1 --2- 172.21.15.97:0/3796884441 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c38108510 0x7f3c3819c760 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:16:48.700 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:47.313+0000 7f3c3ccbe700 1 -- 172.21.15.97:0/3796884441 >> 172.21.15.97:0/3796884441 conn(0x7f3c38103b70 msgr2=0x7f3c38104880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:16:48.700 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:47.313+0000 7f3c3ccbe700 1 -- 172.21.15.97:0/3796884441 shutdown_connections 2024-04-23T10:16:48.700 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:47.313+0000 7f3c3ccbe700 1 -- 172.21.15.97:0/3796884441 wait complete. 2024-04-23T10:16:49.893 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:49.098+0000 7f5ae2b6d700 1 Processor -- start 2024-04-23T10:16:49.894 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:49.098+0000 7f5ae2b6d700 1 -- start start 2024-04-23T10:16:49.894 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:49.098+0000 7f5ae2b6d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5adc1067f0 0x7f5adc106c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:16:49.894 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:49.098+0000 7f5ae2b6d700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5adc1071e0 con 0x7f5adc1067f0 2024-04-23T10:16:49.894 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:49.098+0000 7f5ae0909700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5adc1067f0 0x7f5adc106c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:16:49.894 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:49.098+0000 7f5ae0909700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5adc1067f0 0x7f5adc106c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35602/0 (socket says 172.21.15.97:35602) 2024-04-23T10:16:49.894 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:49.098+0000 7f5ae0909700 1 -- 172.21.15.97:0/2454402509 learned_addr learned my addr 172.21.15.97:0/2454402509 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:16:49.894 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:49.098+0000 7f5ae0909700 1 -- 172.21.15.97:0/2454402509 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5adc1097d0 con 0x7f5adc1067f0 2024-04-23T10:16:49.894 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:49.098+0000 7f5ae0909700 1 --2- 172.21.15.97:0/2454402509 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5adc1067f0 0x7f5adc106c10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5ad00098a0 tx=0x7f5ad0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=280e92fd896ee22a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:16:49.894 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:49.099+0000 7f5adb7fe700 1 -- 172.21.15.97:0/2454402509 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ad0004160 con 0x7f5adc1067f0 2024-04-23T10:16:49.895 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:49.099+0000 7f5adb7fe700 1 -- 172.21.15.97:0/2454402509 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ad0004300 con 0x7f5adc1067f0 2024-04-23T10:16:49.895 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:49.099+0000 7f5adb7fe700 1 -- 172.21.15.97:0/2454402509 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ad0011a20 con 0x7f5adc1067f0 2024-04-23T10:16:49.895 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:49.099+0000 7f5ae2b6d700 1 -- 172.21.15.97:0/2454402509 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5adc1067f0 msgr2=0x7f5adc106c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:16:49.895 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:49.099+0000 7f5ae2b6d700 1 --2- 172.21.15.97:0/2454402509 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5adc1067f0 0x7f5adc106c10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5ad00098a0 tx=0x7f5ad0009bb0 comp rx=0 tx=0).stop 2024-04-23T10:16:49.895 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:49.099+0000 7f5ae2b6d700 1 -- 172.21.15.97:0/2454402509 shutdown_connections 2024-04-23T10:16:49.895 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:49.099+0000 7f5ae2b6d700 1 --2- 172.21.15.97:0/2454402509 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5adc1067f0 0x7f5adc106c10 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:16:49.895 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:49.099+0000 7f5ae2b6d700 1 -- 172.21.15.97:0/2454402509 >> 172.21.15.97:0/2454402509 conn(0x7f5adc1021b0 msgr2=0x7f5adc1045d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:16:49.895 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:49.099+0000 7f5ae2b6d700 1 -- 172.21.15.97:0/2454402509 shutdown_connections 2024-04-23T10:16:49.895 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:49.099+0000 7f5ae2b6d700 1 -- 172.21.15.97:0/2454402509 wait complete. 2024-04-23T10:16:49.895 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:49.100+0000 7f5ae2b6d700 1 Processor -- start 2024-04-23T10:16:49.895 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:49.100+0000 7f5ae2b6d700 1 -- start start 2024-04-23T10:16:49.896 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:49.100+0000 7f5ae2b6d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5adc1067f0 0x7f5adc19c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:16:49.896 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:49.100+0000 7f5ae2b6d700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5adc10cb80 con 0x7f5adc1067f0 2024-04-23T10:16:49.896 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:49.100+0000 7f5ae0909700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5adc1067f0 0x7f5adc19c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:16:49.896 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:49.100+0000 7f5ae0909700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5adc1067f0 0x7f5adc19c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35612/0 (socket says 172.21.15.97:35612) 2024-04-23T10:16:49.896 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:49.100+0000 7f5ae0909700 1 -- 172.21.15.97:0/2738391927 learned_addr learned my addr 172.21.15.97:0/2738391927 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:16:49.896 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:49.100+0000 7f5ae0909700 1 -- 172.21.15.97:0/2738391927 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5adc19ccd0 con 0x7f5adc1067f0 2024-04-23T10:16:49.896 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:49.100+0000 7f5ae0909700 1 --2- 172.21.15.97:0/2738391927 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5adc1067f0 0x7f5adc19c790 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5ad00116e0 tx=0x7f5ad0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:16:49.896 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:49.100+0000 7f5ad9ffb700 1 -- 172.21.15.97:0/2738391927 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ad00039e0 con 0x7f5adc1067f0 2024-04-23T10:16:49.896 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:49.100+0000 7f5ad9ffb700 1 -- 172.21.15.97:0/2738391927 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ad001b8f0 con 0x7f5adc1067f0 2024-04-23T10:16:49.897 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:49.100+0000 7f5ae2b6d700 1 -- 172.21.15.97:0/2738391927 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5adc19cf60 con 0x7f5adc1067f0 2024-04-23T10:16:49.897 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:49.100+0000 7f5ad9ffb700 1 -- 172.21.15.97:0/2738391927 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ad0023760 con 0x7f5adc1067f0 2024-04-23T10:16:49.897 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:49.101+0000 7f5ae2b6d700 1 -- 172.21.15.97:0/2738391927 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5adc19fc00 con 0x7f5adc1067f0 2024-04-23T10:16:49.897 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:49.101+0000 7f5ad9ffb700 1 -- 172.21.15.97:0/2738391927 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f5ad001b440 con 0x7f5adc1067f0 2024-04-23T10:16:49.897 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:49.102+0000 7f5ae2b6d700 1 -- 172.21.15.97:0/2738391927 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5adc0640a0 con 0x7f5adc1067f0 2024-04-23T10:16:49.897 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:49.102+0000 7f5ad9ffb700 1 --2- 172.21.15.97:0/2738391927 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f5acc039560 0x7f5acc03ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:16:49.897 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:49.102+0000 7f5ad9ffb700 1 -- 172.21.15.97:0/2738391927 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5ad004dec0 con 0x7f5adc1067f0 2024-04-23T10:16:49.897 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:49.102+0000 7f5adbfff700 1 --2- 172.21.15.97:0/2738391927 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f5acc039560 0x7f5acc03ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:16:49.897 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:49.102+0000 7f5adbfff700 1 --2- 172.21.15.97:0/2738391927 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f5acc039560 0x7f5acc03ba20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5ac8006fd0 tx=0x7f5ac8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:16:49.897 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:49.107+0000 7f5ad9ffb700 1 -- 172.21.15.97:0/2738391927 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5ad001f030 con 0x7f5adc1067f0 2024-04-23T10:16:49.897 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:49.312+0000 7f5ae2b6d700 1 -- 172.21.15.97:0/2738391927 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f5adc02df80 con 0x7f5adc1067f0 2024-04-23T10:16:49.898 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:49.317+0000 7f5ad9ffb700 1 -- 172.21.15.97:0/2738391927 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f5ad001a3d0 con 0x7f5adc1067f0 2024-04-23T10:16:49.898 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:49.320+0000 7f5ae2b6d700 1 -- 172.21.15.97:0/2738391927 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f5acc039560 msgr2=0x7f5acc03ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:16:49.898 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:49.320+0000 7f5ae2b6d700 1 --2- 172.21.15.97:0/2738391927 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f5acc039560 0x7f5acc03ba20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5ac8006fd0 tx=0x7f5ac8006e40 comp rx=0 tx=0).stop 2024-04-23T10:16:49.898 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:49.320+0000 7f5ae2b6d700 1 -- 172.21.15.97:0/2738391927 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5adc1067f0 msgr2=0x7f5adc19c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:16:49.898 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:49.320+0000 7f5ae2b6d700 1 --2- 172.21.15.97:0/2738391927 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5adc1067f0 0x7f5adc19c790 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5ad00116e0 tx=0x7f5ad0011aa0 comp rx=0 tx=0).stop 2024-04-23T10:16:49.898 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:49.320+0000 7f5ae2b6d700 1 -- 172.21.15.97:0/2738391927 shutdown_connections 2024-04-23T10:16:49.898 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:49.320+0000 7f5ae2b6d700 1 --2- 172.21.15.97:0/2738391927 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f5acc039560 0x7f5acc03ba20 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:16:49.898 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:49.320+0000 7f5ae2b6d700 1 --2- 172.21.15.97:0/2738391927 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5adc1067f0 0x7f5adc19c790 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:16:49.898 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:49.320+0000 7f5ae2b6d700 1 -- 172.21.15.97:0/2738391927 >> 172.21.15.97:0/2738391927 conn(0x7f5adc1021b0 msgr2=0x7f5adc102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:16:49.898 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:49.320+0000 7f5ae2b6d700 1 -- 172.21.15.97:0/2738391927 shutdown_connections 2024-04-23T10:16:49.898 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:49.320+0000 7f5ae2b6d700 1 -- 172.21.15.97:0/2738391927 wait complete. 2024-04-23T10:16:50.312 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:50 smithi097 ceph-mon[29197]: from='mgr.14120 172.21.15.97:0/850926340' entity='mgr.x' 2024-04-23T10:16:50.312 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:50 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/2738391927' entity='client.admin' 2024-04-23T10:16:51.778 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:51 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/3247811032' entity='client.admin' 2024-04-23T10:16:51.778 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:51 smithi097 ceph-mon[29197]: from='mgr.14120 172.21.15.97:0/850926340' entity='mgr.x' 2024-04-23T10:16:51.787 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:50.316+0000 7feb72109700 1 Processor -- start 2024-04-23T10:16:51.787 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:50.316+0000 7feb72109700 1 -- start start 2024-04-23T10:16:51.787 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:50.316+0000 7feb72109700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb6c107410 0x7feb6c107830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:16:51.787 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:50.316+0000 7feb72109700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7feb6c107e00 con 0x7feb6c107410 2024-04-23T10:16:51.787 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:50.316+0000 7feb6b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb6c107410 0x7feb6c107830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:16:51.787 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:50.316+0000 7feb6b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb6c107410 0x7feb6c107830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35628/0 (socket says 172.21.15.97:35628) 2024-04-23T10:16:51.787 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:50.316+0000 7feb6b7fe700 1 -- 172.21.15.97:0/1600868965 learned_addr learned my addr 172.21.15.97:0/1600868965 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:16:51.788 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:50.317+0000 7feb6b7fe700 1 -- 172.21.15.97:0/1600868965 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb6c108610 con 0x7feb6c107410 2024-04-23T10:16:51.788 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:50.317+0000 7feb6b7fe700 1 --2- 172.21.15.97:0/1600868965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb6c107410 0x7feb6c107830 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7feb600098a0 tx=0x7feb60009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f1e33ed734ff9e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:16:51.788 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:50.317+0000 7feb6a7fc700 1 -- 172.21.15.97:0/1600868965 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feb60004160 con 0x7feb6c107410 2024-04-23T10:16:51.788 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:50.317+0000 7feb6a7fc700 1 -- 172.21.15.97:0/1600868965 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb60004300 con 0x7feb6c107410 2024-04-23T10:16:51.788 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:50.317+0000 7feb6a7fc700 1 -- 172.21.15.97:0/1600868965 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feb60011a20 con 0x7feb6c107410 2024-04-23T10:16:51.788 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:50.318+0000 7feb72109700 1 -- 172.21.15.97:0/1600868965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb6c107410 msgr2=0x7feb6c107830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:16:51.788 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:50.318+0000 7feb72109700 1 --2- 172.21.15.97:0/1600868965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb6c107410 0x7feb6c107830 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7feb600098a0 tx=0x7feb60009bb0 comp rx=0 tx=0).stop 2024-04-23T10:16:51.788 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:50.318+0000 7feb72109700 1 -- 172.21.15.97:0/1600868965 shutdown_connections 2024-04-23T10:16:51.788 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:50.318+0000 7feb72109700 1 --2- 172.21.15.97:0/1600868965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb6c107410 0x7feb6c107830 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:16:51.788 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:50.318+0000 7feb72109700 1 -- 172.21.15.97:0/1600868965 >> 172.21.15.97:0/1600868965 conn(0x7feb6c102a70 msgr2=0x7feb6c104e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:16:51.788 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:50.318+0000 7feb72109700 1 -- 172.21.15.97:0/1600868965 shutdown_connections 2024-04-23T10:16:51.789 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:50.318+0000 7feb72109700 1 -- 172.21.15.97:0/1600868965 wait complete. 2024-04-23T10:16:51.789 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:50.318+0000 7feb72109700 1 Processor -- start 2024-04-23T10:16:51.789 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:50.319+0000 7feb72109700 1 -- start start 2024-04-23T10:16:51.789 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:50.319+0000 7feb72109700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb6c107410 0x7feb6c07bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:16:51.789 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:50.319+0000 7feb72109700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7feb6c10b9c0 con 0x7feb6c107410 2024-04-23T10:16:51.789 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:50.319+0000 7feb6b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb6c107410 0x7feb6c07bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:16:51.789 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:50.319+0000 7feb6b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb6c107410 0x7feb6c07bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35642/0 (socket says 172.21.15.97:35642) 2024-04-23T10:16:51.789 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:50.319+0000 7feb6b7fe700 1 -- 172.21.15.97:0/3247811032 learned_addr learned my addr 172.21.15.97:0/3247811032 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:16:51.789 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:50.319+0000 7feb6b7fe700 1 -- 172.21.15.97:0/3247811032 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb6c19d1a0 con 0x7feb6c107410 2024-04-23T10:16:51.789 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:50.319+0000 7feb6b7fe700 1 --2- 172.21.15.97:0/3247811032 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb6c107410 0x7feb6c07bac0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7feb600116e0 tx=0x7feb60011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:16:51.789 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:50.319+0000 7feb68ff9700 1 -- 172.21.15.97:0/3247811032 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feb600039e0 con 0x7feb6c107410 2024-04-23T10:16:51.790 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:50.320+0000 7feb72109700 1 -- 172.21.15.97:0/3247811032 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb6c1a17b0 con 0x7feb6c107410 2024-04-23T10:16:51.790 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:50.320+0000 7feb68ff9700 1 -- 172.21.15.97:0/3247811032 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb6001b8f0 con 0x7feb6c107410 2024-04-23T10:16:51.790 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:50.320+0000 7feb68ff9700 1 -- 172.21.15.97:0/3247811032 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feb60023760 con 0x7feb6c107410 2024-04-23T10:16:51.790 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:50.320+0000 7feb72109700 1 -- 172.21.15.97:0/3247811032 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb6c1a1bd0 con 0x7feb6c107410 2024-04-23T10:16:51.790 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:50.321+0000 7feb68ff9700 1 -- 172.21.15.97:0/3247811032 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7feb600239d0 con 0x7feb6c107410 2024-04-23T10:16:51.790 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:50.321+0000 7feb68ff9700 1 --2- 172.21.15.97:0/3247811032 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7feb54039560 0x7feb5403ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:16:51.790 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:50.321+0000 7feb72109700 1 -- 172.21.15.97:0/3247811032 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb6c0640a0 con 0x7feb6c107410 2024-04-23T10:16:51.790 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:50.321+0000 7feb68ff9700 1 -- 172.21.15.97:0/3247811032 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7feb6004df40 con 0x7feb6c107410 2024-04-23T10:16:51.790 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:50.321+0000 7feb6affd700 1 --2- 172.21.15.97:0/3247811032 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7feb54039560 0x7feb5403ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:16:51.790 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:50.321+0000 7feb6affd700 1 --2- 172.21.15.97:0/3247811032 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7feb54039560 0x7feb5403ba20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7feb5c006fd0 tx=0x7feb5c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:16:51.790 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:50.326+0000 7feb68ff9700 1 -- 172.21.15.97:0/3247811032 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feb6001f030 con 0x7feb6c107410 2024-04-23T10:16:51.791 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:50.553+0000 7feb72109700 1 -- 172.21.15.97:0/3247811032 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7feb6c02e090 con 0x7feb6c107410 2024-04-23T10:16:51.791 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:50.556+0000 7feb68ff9700 1 -- 172.21.15.97:0/3247811032 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7feb60018e30 con 0x7feb6c107410 2024-04-23T10:16:51.791 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:50.558+0000 7feb72109700 1 -- 172.21.15.97:0/3247811032 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7feb54039560 msgr2=0x7feb5403ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:16:51.791 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:50.558+0000 7feb72109700 1 --2- 172.21.15.97:0/3247811032 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7feb54039560 0x7feb5403ba20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7feb5c006fd0 tx=0x7feb5c006e40 comp rx=0 tx=0).stop 2024-04-23T10:16:51.791 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:50.558+0000 7feb72109700 1 -- 172.21.15.97:0/3247811032 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb6c107410 msgr2=0x7feb6c07bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:16:51.791 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:50.558+0000 7feb72109700 1 --2- 172.21.15.97:0/3247811032 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb6c107410 0x7feb6c07bac0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7feb600116e0 tx=0x7feb60011aa0 comp rx=0 tx=0).stop 2024-04-23T10:16:51.791 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:50.558+0000 7feb72109700 1 -- 172.21.15.97:0/3247811032 shutdown_connections 2024-04-23T10:16:51.791 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:50.558+0000 7feb72109700 1 --2- 172.21.15.97:0/3247811032 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7feb54039560 0x7feb5403ba20 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:16:51.791 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:50.558+0000 7feb72109700 1 --2- 172.21.15.97:0/3247811032 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb6c107410 0x7feb6c07bac0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:16:51.791 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:50.558+0000 7feb72109700 1 -- 172.21.15.97:0/3247811032 >> 172.21.15.97:0/3247811032 conn(0x7feb6c102a70 msgr2=0x7feb6c103660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:16:51.791 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:50.558+0000 7feb72109700 1 -- 172.21.15.97:0/3247811032 shutdown_connections 2024-04-23T10:16:51.791 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:50.558+0000 7feb72109700 1 -- 172.21.15.97:0/3247811032 wait complete. 2024-04-23T10:16:51.792 INFO:teuthology.orchestra.run.smithi097.stdout:Enabling the dashboard module... 2024-04-23T10:16:52.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:52 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/2543061848' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-23T10:16:52.861 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:52 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: ignoring --setuser ceph since I am not root 2024-04-23T10:16:52.861 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:52 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: ignoring --setgroup ceph since I am not root 2024-04-23T10:16:52.861 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:52 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:52.737+0000 7f1eb096a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-23T10:16:53.235 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:52 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:52.923+0000 7f1eb096a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-23T10:16:53.235 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:53 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:53.026+0000 7f1eb096a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-23T10:16:53.244 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:52.227+0000 7f80a5dd0700 1 Processor -- start 2024-04-23T10:16:53.244 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:52.227+0000 7f80a5dd0700 1 -- start start 2024-04-23T10:16:53.244 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:52.227+0000 7f80a5dd0700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80a0108f60 0x7f80a010b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:16:53.244 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:52.227+0000 7f80a5dd0700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f80a007be60 con 0x7f80a0108f60 2024-04-23T10:16:53.244 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:52.228+0000 7f809f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80a0108f60 0x7f80a010b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:16:53.244 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:52.228+0000 7f809f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80a0108f60 0x7f80a010b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35650/0 (socket says 172.21.15.97:35650) 2024-04-23T10:16:53.244 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:52.228+0000 7f809f7fe700 1 -- 172.21.15.97:0/1278347782 learned_addr learned my addr 172.21.15.97:0/1278347782 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:16:53.244 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:52.228+0000 7f809f7fe700 1 -- 172.21.15.97:0/1278347782 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80a010b890 con 0x7f80a0108f60 2024-04-23T10:16:53.244 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:52.228+0000 7f809f7fe700 1 --2- 172.21.15.97:0/1278347782 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80a0108f60 0x7f80a010b350 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f80900098a0 tx=0x7f8090009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14eb5fb5fe0423b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:16:53.244 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:52.228+0000 7f809e7fc700 1 -- 172.21.15.97:0/1278347782 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8090004160 con 0x7f80a0108f60 2024-04-23T10:16:53.244 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:52.228+0000 7f809e7fc700 1 -- 172.21.15.97:0/1278347782 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8090004300 con 0x7f80a0108f60 2024-04-23T10:16:53.244 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:52.228+0000 7f809e7fc700 1 -- 172.21.15.97:0/1278347782 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8090011a20 con 0x7f80a0108f60 2024-04-23T10:16:53.244 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:52.229+0000 7f80a5dd0700 1 -- 172.21.15.97:0/1278347782 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80a0108f60 msgr2=0x7f80a010b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:16:53.245 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:52.229+0000 7f80a5dd0700 1 --2- 172.21.15.97:0/1278347782 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80a0108f60 0x7f80a010b350 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f80900098a0 tx=0x7f8090009bb0 comp rx=0 tx=0).stop 2024-04-23T10:16:53.245 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:52.229+0000 7f80a5dd0700 1 -- 172.21.15.97:0/1278347782 shutdown_connections 2024-04-23T10:16:53.245 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:52.229+0000 7f80a5dd0700 1 --2- 172.21.15.97:0/1278347782 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80a0108f60 0x7f80a010b350 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:16:53.245 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:52.229+0000 7f80a5dd0700 1 -- 172.21.15.97:0/1278347782 >> 172.21.15.97:0/1278347782 conn(0x7f80a01029b0 msgr2=0x7f80a0104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:16:53.245 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:52.229+0000 7f80a5dd0700 1 -- 172.21.15.97:0/1278347782 shutdown_connections 2024-04-23T10:16:53.245 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:52.229+0000 7f80a5dd0700 1 -- 172.21.15.97:0/1278347782 wait complete. 2024-04-23T10:16:53.245 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:52.230+0000 7f80a5dd0700 1 Processor -- start 2024-04-23T10:16:53.245 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:52.230+0000 7f80a5dd0700 1 -- start start 2024-04-23T10:16:53.245 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:52.230+0000 7f80a5dd0700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80a0108f60 0x7f80a019c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:16:53.245 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:52.230+0000 7f80a5dd0700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f80a010c840 con 0x7f80a0108f60 2024-04-23T10:16:53.245 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:52.230+0000 7f809f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80a0108f60 0x7f80a019c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:16:53.245 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:52.230+0000 7f809f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80a0108f60 0x7f80a019c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35658/0 (socket says 172.21.15.97:35658) 2024-04-23T10:16:53.245 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:52.230+0000 7f809f7fe700 1 -- 172.21.15.97:0/2543061848 learned_addr learned my addr 172.21.15.97:0/2543061848 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:16:53.245 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:52.231+0000 7f809f7fe700 1 -- 172.21.15.97:0/2543061848 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80a019c940 con 0x7f80a0108f60 2024-04-23T10:16:53.246 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:52.231+0000 7f809f7fe700 1 --2- 172.21.15.97:0/2543061848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80a0108f60 0x7f80a019c400 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f80900116e0 tx=0x7f8090011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:16:53.246 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:52.231+0000 7f809cff9700 1 -- 172.21.15.97:0/2543061848 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8090003b20 con 0x7f80a0108f60 2024-04-23T10:16:53.246 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:52.231+0000 7f809cff9700 1 -- 172.21.15.97:0/2543061848 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f809001b8f0 con 0x7f80a0108f60 2024-04-23T10:16:53.246 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:52.231+0000 7f80a5dd0700 1 -- 172.21.15.97:0/2543061848 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80a019cbd0 con 0x7f80a0108f60 2024-04-23T10:16:53.246 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:52.231+0000 7f809cff9700 1 -- 172.21.15.97:0/2543061848 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80900238c0 con 0x7f80a0108f60 2024-04-23T10:16:53.246 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:52.231+0000 7f80a5dd0700 1 -- 172.21.15.97:0/2543061848 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80a019f8c0 con 0x7f80a0108f60 2024-04-23T10:16:53.246 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:52.232+0000 7f809cff9700 1 -- 172.21.15.97:0/2543061848 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f8090023a60 con 0x7f80a0108f60 2024-04-23T10:16:53.246 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:52.232+0000 7f809cff9700 1 --2- 172.21.15.97:0/2543061848 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f80880395b0 0x7f808803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:16:53.246 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:52.232+0000 7f80a5dd0700 1 -- 172.21.15.97:0/2543061848 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f808c005360 con 0x7f80a0108f60 2024-04-23T10:16:53.246 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:52.232+0000 7f809cff9700 1 -- 172.21.15.97:0/2543061848 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f809004dfd0 con 0x7f80a0108f60 2024-04-23T10:16:53.246 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:52.233+0000 7f809effd700 1 --2- 172.21.15.97:0/2543061848 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f80880395b0 0x7f808803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:16:53.246 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:52.233+0000 7f809effd700 1 --2- 172.21.15.97:0/2543061848 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f80880395b0 0x7f808803ba70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8094006fd0 tx=0x7f8094006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:16:53.246 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:52.238+0000 7f809cff9700 1 -- 172.21.15.97:0/2543061848 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f809001f030 con 0x7f80a0108f60 2024-04-23T10:16:53.246 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:52.500+0000 7f80a5dd0700 1 -- 172.21.15.97:0/2543061848 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f808c005650 con 0x7f80a0108f60 2024-04-23T10:16:53.247 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:52.558+0000 7f809cff9700 1 -- 172.21.15.97:0/2543061848 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f8090018e30 con 0x7f80a0108f60 2024-04-23T10:16:53.247 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:52.560+0000 7f809cff9700 1 -- 172.21.15.97:0/2543061848 <== mon.0 v2:172.21.15.97:3300/0 8 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f809001baa0 con 0x7f80a0108f60 2024-04-23T10:16:53.247 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:52.561+0000 7f80a5dd0700 1 -- 172.21.15.97:0/2543061848 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f80880395b0 msgr2=0x7f808803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:16:53.247 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:52.561+0000 7f80a5dd0700 1 --2- 172.21.15.97:0/2543061848 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f80880395b0 0x7f808803ba70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8094006fd0 tx=0x7f8094006e40 comp rx=0 tx=0).stop 2024-04-23T10:16:53.247 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:52.561+0000 7f80a5dd0700 1 -- 172.21.15.97:0/2543061848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80a0108f60 msgr2=0x7f80a019c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:16:53.247 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:52.561+0000 7f80a5dd0700 1 --2- 172.21.15.97:0/2543061848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80a0108f60 0x7f80a019c400 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f80900116e0 tx=0x7f8090011aa0 comp rx=0 tx=0).stop 2024-04-23T10:16:53.247 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:52.561+0000 7f80a5dd0700 1 -- 172.21.15.97:0/2543061848 shutdown_connections 2024-04-23T10:16:53.247 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:52.561+0000 7f80a5dd0700 1 --2- 172.21.15.97:0/2543061848 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f80880395b0 0x7f808803ba70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:16:53.247 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:52.561+0000 7f80a5dd0700 1 --2- 172.21.15.97:0/2543061848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f80a0108f60 0x7f80a019c400 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:16:53.247 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:52.561+0000 7f80a5dd0700 1 -- 172.21.15.97:0/2543061848 >> 172.21.15.97:0/2543061848 conn(0x7f80a01029b0 msgr2=0x7f80a0103540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:16:53.247 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:52.562+0000 7f80a5dd0700 1 -- 172.21.15.97:0/2543061848 shutdown_connections 2024-04-23T10:16:53.247 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:52.562+0000 7f80a5dd0700 1 -- 172.21.15.97:0/2543061848 wait complete. 2024-04-23T10:16:53.558 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:53 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:53.343+0000 7f1eb096a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-23T10:16:53.558 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:53 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:53.430+0000 7f1eb096a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-23T10:16:53.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:53 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/2543061848' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-23T10:16:53.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:53 smithi097 ceph-mon[29197]: mgrmap e9: x(active, since 19s) 2024-04-23T10:16:54.609 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:54 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/752100475' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-23T10:16:54.618 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout { 2024-04-23T10:16:54.618 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-23T10:16:54.618 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-23T10:16:54.618 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-23T10:16:54.618 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-23T10:16:54.618 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout } 2024-04-23T10:16:54.618 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:53.683+0000 7f07ec0c3700 1 Processor -- start 2024-04-23T10:16:54.619 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:53.683+0000 7f07ec0c3700 1 -- start start 2024-04-23T10:16:54.619 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:53.684+0000 7f07ec0c3700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07e4108fc0 0x7f07e410b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:16:54.619 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:53.684+0000 7f07ec0c3700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f07e407be40 con 0x7f07e4108fc0 2024-04-23T10:16:54.619 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:53.684+0000 7f07e9e5f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07e4108fc0 0x7f07e410b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:16:54.619 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:53.684+0000 7f07e9e5f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07e4108fc0 0x7f07e410b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35690/0 (socket says 172.21.15.97:35690) 2024-04-23T10:16:54.619 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:53.684+0000 7f07e9e5f700 1 -- 172.21.15.97:0/2561224344 learned_addr learned my addr 172.21.15.97:0/2561224344 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:16:54.619 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:53.684+0000 7f07e9e5f700 1 -- 172.21.15.97:0/2561224344 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07e410b8f0 con 0x7f07e4108fc0 2024-04-23T10:16:54.619 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:53.684+0000 7f07e9e5f700 1 --2- 172.21.15.97:0/2561224344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07e4108fc0 0x7f07e410b3b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f07d80098a0 tx=0x7f07d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b27d519f3900fae2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:16:54.619 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:53.685+0000 7f07e8e5d700 1 -- 172.21.15.97:0/2561224344 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f07d8004160 con 0x7f07e4108fc0 2024-04-23T10:16:54.619 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:53.685+0000 7f07e8e5d700 1 -- 172.21.15.97:0/2561224344 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07d8004300 con 0x7f07e4108fc0 2024-04-23T10:16:54.619 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:53.685+0000 7f07e8e5d700 1 -- 172.21.15.97:0/2561224344 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f07d8011a20 con 0x7f07e4108fc0 2024-04-23T10:16:54.619 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:53.685+0000 7f07ec0c3700 1 -- 172.21.15.97:0/2561224344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07e4108fc0 msgr2=0x7f07e410b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:16:54.619 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:53.685+0000 7f07ec0c3700 1 --2- 172.21.15.97:0/2561224344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07e4108fc0 0x7f07e410b3b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f07d80098a0 tx=0x7f07d8009bb0 comp rx=0 tx=0).stop 2024-04-23T10:16:54.620 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:53.685+0000 7f07ec0c3700 1 -- 172.21.15.97:0/2561224344 shutdown_connections 2024-04-23T10:16:54.620 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:53.685+0000 7f07ec0c3700 1 --2- 172.21.15.97:0/2561224344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07e4108fc0 0x7f07e410b3b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:16:54.620 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:53.685+0000 7f07ec0c3700 1 -- 172.21.15.97:0/2561224344 >> 172.21.15.97:0/2561224344 conn(0x7f07e4102a10 msgr2=0x7f07e4104e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:16:54.620 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:53.685+0000 7f07ec0c3700 1 -- 172.21.15.97:0/2561224344 shutdown_connections 2024-04-23T10:16:54.620 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:53.685+0000 7f07ec0c3700 1 -- 172.21.15.97:0/2561224344 wait complete. 2024-04-23T10:16:54.620 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:53.686+0000 7f07ec0c3700 1 Processor -- start 2024-04-23T10:16:54.620 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:53.686+0000 7f07ec0c3700 1 -- start start 2024-04-23T10:16:54.620 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:53.686+0000 7f07ec0c3700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07e4108fc0 0x7f07e419c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:16:54.620 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:53.686+0000 7f07ec0c3700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f07e410c890 con 0x7f07e4108fc0 2024-04-23T10:16:54.620 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:53.686+0000 7f07e9e5f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07e4108fc0 0x7f07e419c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:16:54.620 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:53.686+0000 7f07e9e5f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07e4108fc0 0x7f07e419c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35692/0 (socket says 172.21.15.97:35692) 2024-04-23T10:16:54.620 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:53.686+0000 7f07e9e5f700 1 -- 172.21.15.97:0/752100475 learned_addr learned my addr 172.21.15.97:0/752100475 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:16:54.620 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:53.687+0000 7f07e9e5f700 1 -- 172.21.15.97:0/752100475 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07e419c920 con 0x7f07e4108fc0 2024-04-23T10:16:54.620 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:53.687+0000 7f07e9e5f700 1 --2- 172.21.15.97:0/752100475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07e4108fc0 0x7f07e419c3e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f07d8004660 tx=0x7f07d8011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:16:54.621 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:53.687+0000 7f07d6ffd700 1 -- 172.21.15.97:0/752100475 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f07d8003aa0 con 0x7f07e4108fc0 2024-04-23T10:16:54.621 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:53.687+0000 7f07ec0c3700 1 -- 172.21.15.97:0/752100475 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07e419cbb0 con 0x7f07e4108fc0 2024-04-23T10:16:54.621 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:53.687+0000 7f07d6ffd700 1 -- 172.21.15.97:0/752100475 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07d801b8f0 con 0x7f07e4108fc0 2024-04-23T10:16:54.621 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:53.687+0000 7f07d6ffd700 1 -- 172.21.15.97:0/752100475 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f07d8023900 con 0x7f07e4108fc0 2024-04-23T10:16:54.621 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:53.687+0000 7f07ec0c3700 1 -- 172.21.15.97:0/752100475 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07e419f8a0 con 0x7f07e4108fc0 2024-04-23T10:16:54.621 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:53.688+0000 7f07ec0c3700 1 -- 172.21.15.97:0/752100475 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07e40640a0 con 0x7f07e4108fc0 2024-04-23T10:16:54.621 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:53.688+0000 7f07d6ffd700 1 -- 172.21.15.97:0/752100475 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f07d801baa0 con 0x7f07e4108fc0 2024-04-23T10:16:54.621 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:53.688+0000 7f07d6ffd700 1 --2- 172.21.15.97:0/752100475 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f07d0039600 0x7f07d003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:16:54.621 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:53.688+0000 7f07d6ffd700 1 -- 172.21.15.97:0/752100475 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f07d804dff0 con 0x7f07e4108fc0 2024-04-23T10:16:54.621 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:53.688+0000 7f07e965e700 1 -- 172.21.15.97:0/752100475 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f07d0039600 msgr2=0x7f07d003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/3297166858 2024-04-23T10:16:54.621 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:53.688+0000 7f07e965e700 1 --2- 172.21.15.97:0/752100475 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f07d0039600 0x7f07d003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T10:16:54.621 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:53.697+0000 7f07d6ffd700 1 -- 172.21.15.97:0/752100475 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f07d801f030 con 0x7f07e4108fc0 2024-04-23T10:16:54.621 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:53.889+0000 7f07e965e700 1 -- 172.21.15.97:0/752100475 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f07d0039600 msgr2=0x7f07d003bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/3297166858 2024-04-23T10:16:54.621 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:53.889+0000 7f07e965e700 1 --2- 172.21.15.97:0/752100475 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f07d0039600 0x7f07d003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-23T10:16:54.622 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:54.006+0000 7f07ec0c3700 1 -- 172.21.15.97:0/752100475 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f07e419fa70 con 0x7f07e4108fc0 2024-04-23T10:16:54.622 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:54.007+0000 7f07d6ffd700 1 -- 172.21.15.97:0/752100475 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f07d802c3f0 con 0x7f07e4108fc0 2024-04-23T10:16:54.622 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:54.009+0000 7f07ec0c3700 1 -- 172.21.15.97:0/752100475 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f07d0039600 msgr2=0x7f07d003bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T10:16:54.622 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:54.009+0000 7f07ec0c3700 1 --2- 172.21.15.97:0/752100475 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f07d0039600 0x7f07d003bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:16:54.622 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:54.009+0000 7f07ec0c3700 1 -- 172.21.15.97:0/752100475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07e4108fc0 msgr2=0x7f07e419c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:16:54.622 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:54.009+0000 7f07ec0c3700 1 --2- 172.21.15.97:0/752100475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07e4108fc0 0x7f07e419c3e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f07d8004660 tx=0x7f07d8011aa0 comp rx=0 tx=0).stop 2024-04-23T10:16:54.622 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:54.009+0000 7f07ec0c3700 1 -- 172.21.15.97:0/752100475 shutdown_connections 2024-04-23T10:16:54.622 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:54.009+0000 7f07ec0c3700 1 --2- 172.21.15.97:0/752100475 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f07d0039600 0x7f07d003bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:16:54.622 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:54.009+0000 7f07ec0c3700 1 --2- 172.21.15.97:0/752100475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07e4108fc0 0x7f07e419c3e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:16:54.622 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:54.009+0000 7f07ec0c3700 1 -- 172.21.15.97:0/752100475 >> 172.21.15.97:0/752100475 conn(0x7f07e4102a10 msgr2=0x7f07e41034b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:16:54.622 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:54.009+0000 7f07ec0c3700 1 -- 172.21.15.97:0/752100475 shutdown_connections 2024-04-23T10:16:54.622 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:54.009+0000 7f07ec0c3700 1 -- 172.21.15.97:0/752100475 wait complete. 2024-04-23T10:16:54.622 INFO:teuthology.orchestra.run.smithi097.stdout:Waiting for the mgr to restart... 2024-04-23T10:16:54.622 INFO:teuthology.orchestra.run.smithi097.stdout:Waiting for mgr epoch 9... 2024-04-23T10:16:54.860 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:54 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:54.650+0000 7f1eb096a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-23T10:16:54.861 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:54 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:54.825+0000 7f1eb096a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-23T10:16:55.260 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:55 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:55.001+0000 7f1eb096a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-23T10:16:55.591 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:55 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:55.293+0000 7f1eb096a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-23T10:16:55.591 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:55 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:55.386+0000 7f1eb096a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-23T10:16:55.591 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:55 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:55.591+0000 7f1eb096a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-23T10:16:55.861 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:55 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:55.784+0000 7f1eb096a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-23T10:16:56.762 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:56 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:56.375+0000 7f1eb096a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-23T10:16:57.033 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:56 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:56.797+0000 7f1eb096a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-23T10:16:57.033 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:56 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:56.897+0000 7f1eb096a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-23T10:16:57.361 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:57 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:57.111+0000 7f1eb096a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-23T10:16:57.991 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:57 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:57.654+0000 7f1eb096a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-23T10:16:57.991 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:57 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:57.750+0000 7f1eb096a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-23T10:16:58.348 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:58 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:58.002+0000 7f1eb096a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-23T10:16:58.348 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:58 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:58.136+0000 7f1eb096a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-23T10:16:58.348 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:58 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:58.231+0000 7f1eb096a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-23T10:16:58.349 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:58 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:58.349+0000 7f1eb096a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-23T10:16:59.111 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:58 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:58.648+0000 7f1eb096a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-23T10:16:59.518 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:59 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:59.248+0000 7f1eb096a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-23T10:16:59.518 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:16:59 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:16:59.465+0000 7f1eb096a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-23T10:16:59.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:59 smithi097 ceph-mon[29197]: Active manager daemon x restarted 2024-04-23T10:16:59.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:59 smithi097 ceph-mon[29197]: Activating manager daemon x 2024-04-23T10:16:59.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:59 smithi097 ceph-mon[29197]: osdmap e3: 0 total, 0 up, 0 in 2024-04-23T10:16:59.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:59 smithi097 ceph-mon[29197]: mgrmap e10: x(active, starting, since 0.00345877s) 2024-04-23T10:16:59.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:59 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T10:16:59.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:59 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-23T10:16:59.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:59 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-23T10:16:59.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:59 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-23T10:16:59.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:59 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-23T10:16:59.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:59 smithi097 ceph-mon[29197]: Manager daemon x is now available 2024-04-23T10:16:59.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:59 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:16:59.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:16:59 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-23T10:17:00.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:00 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-23T10:17:00.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:00 smithi097 ceph-mon[29197]: mgrmap e11: x(active, since 1.00544s) 2024-04-23T10:17:01.591 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout { 2024-04-23T10:17:01.591 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-23T10:17:01.591 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-23T10:17:01.592 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout } 2024-04-23T10:17:01.592 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:55.055+0000 7f242473b700 1 Processor -- start 2024-04-23T10:17:01.592 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:55.055+0000 7f242473b700 1 -- start start 2024-04-23T10:17:01.592 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:55.055+0000 7f242473b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f241c1084e0 0x7f241c108900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:17:01.592 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:55.055+0000 7f242473b700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f241c108ed0 con 0x7f241c1084e0 2024-04-23T10:17:01.592 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:55.056+0000 7f24224d7700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f241c1084e0 0x7f241c108900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:17:01.592 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:55.056+0000 7f24224d7700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f241c1084e0 0x7f241c108900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35704/0 (socket says 172.21.15.97:35704) 2024-04-23T10:17:01.593 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:55.056+0000 7f24224d7700 1 -- 172.21.15.97:0/2455501067 learned_addr learned my addr 172.21.15.97:0/2455501067 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:17:01.593 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:55.056+0000 7f24224d7700 1 -- 172.21.15.97:0/2455501067 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f241c109690 con 0x7f241c1084e0 2024-04-23T10:17:01.593 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:55.056+0000 7f24224d7700 1 --2- 172.21.15.97:0/2455501067 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f241c1084e0 0x7f241c108900 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f240c0098a0 tx=0x7f240c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a4e1cf9b369c1e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:17:01.593 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:55.056+0000 7f24214d5700 1 -- 172.21.15.97:0/2455501067 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f240c004160 con 0x7f241c1084e0 2024-04-23T10:17:01.593 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:55.056+0000 7f24214d5700 1 -- 172.21.15.97:0/2455501067 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f240c004300 con 0x7f241c1084e0 2024-04-23T10:17:01.593 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:55.056+0000 7f24214d5700 1 -- 172.21.15.97:0/2455501067 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f240c011a20 con 0x7f241c1084e0 2024-04-23T10:17:01.593 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:55.057+0000 7f242473b700 1 -- 172.21.15.97:0/2455501067 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f241c1084e0 msgr2=0x7f241c108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:17:01.593 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:55.057+0000 7f242473b700 1 --2- 172.21.15.97:0/2455501067 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f241c1084e0 0x7f241c108900 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f240c0098a0 tx=0x7f240c009bb0 comp rx=0 tx=0).stop 2024-04-23T10:17:01.593 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:55.057+0000 7f242473b700 1 -- 172.21.15.97:0/2455501067 shutdown_connections 2024-04-23T10:17:01.594 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:55.057+0000 7f242473b700 1 --2- 172.21.15.97:0/2455501067 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f241c1084e0 0x7f241c108900 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:17:01.594 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:55.057+0000 7f242473b700 1 -- 172.21.15.97:0/2455501067 >> 172.21.15.97:0/2455501067 conn(0x7f241c103ae0 msgr2=0x7f241c105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:17:01.594 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:55.057+0000 7f242473b700 1 -- 172.21.15.97:0/2455501067 shutdown_connections 2024-04-23T10:17:01.594 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:55.057+0000 7f242473b700 1 -- 172.21.15.97:0/2455501067 wait complete. 2024-04-23T10:17:01.594 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:55.057+0000 7f242473b700 1 Processor -- start 2024-04-23T10:17:01.594 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:55.058+0000 7f242473b700 1 -- start start 2024-04-23T10:17:01.594 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:55.058+0000 7f242473b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f241c1084e0 0x7f241c07db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:17:01.594 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:55.058+0000 7f242473b700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f241c10ca40 con 0x7f241c1084e0 2024-04-23T10:17:01.594 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:55.058+0000 7f24224d7700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f241c1084e0 0x7f241c07db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:17:01.594 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:55.058+0000 7f24224d7700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f241c1084e0 0x7f241c07db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35710/0 (socket says 172.21.15.97:35710) 2024-04-23T10:17:01.595 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:55.058+0000 7f24224d7700 1 -- 172.21.15.97:0/3369668848 learned_addr learned my addr 172.21.15.97:0/3369668848 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:17:01.595 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:55.058+0000 7f24224d7700 1 -- 172.21.15.97:0/3369668848 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f241c07e0c0 con 0x7f241c1084e0 2024-04-23T10:17:01.595 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:55.058+0000 7f24224d7700 1 --2- 172.21.15.97:0/3369668848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f241c1084e0 0x7f241c07db80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f240c0119f0 tx=0x7f240c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:17:01.595 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:55.058+0000 7f24137fe700 1 -- 172.21.15.97:0/3369668848 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f240c0039e0 con 0x7f241c1084e0 2024-04-23T10:17:01.595 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:55.058+0000 7f24137fe700 1 -- 172.21.15.97:0/3369668848 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f240c01b8f0 con 0x7f241c1084e0 2024-04-23T10:17:01.595 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:55.058+0000 7f24137fe700 1 -- 172.21.15.97:0/3369668848 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f240c0237a0 con 0x7f241c1084e0 2024-04-23T10:17:01.595 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:55.058+0000 7f242473b700 1 -- 172.21.15.97:0/3369668848 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f241c07c2b0 con 0x7f241c1084e0 2024-04-23T10:17:01.595 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:55.058+0000 7f242473b700 1 -- 172.21.15.97:0/3369668848 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f241c07c7f0 con 0x7f241c1084e0 2024-04-23T10:17:01.595 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:55.060+0000 7f24137fe700 1 -- 172.21.15.97:0/3369668848 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f240c01b440 con 0x7f241c1084e0 2024-04-23T10:17:01.595 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:55.060+0000 7f24137fe700 1 --2- 172.21.15.97:0/3369668848 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f2408039610 0x7f240803bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:17:01.595 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:55.060+0000 7f24137fe700 1 -- 172.21.15.97:0/3369668848 --> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f240803c1e0 con 0x7f2408039610 2024-04-23T10:17:01.596 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:55.060+0000 7f24137fe700 1 -- 172.21.15.97:0/3369668848 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f240c04df60 con 0x7f241c1084e0 2024-04-23T10:17:01.596 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:55.060+0000 7f2421cd6700 1 -- 172.21.15.97:0/3369668848 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f2408039610 msgr2=0x7f240803bad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/3297166858 2024-04-23T10:17:01.596 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:55.060+0000 7f2421cd6700 1 --2- 172.21.15.97:0/3369668848 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f2408039610 0x7f240803bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T10:17:01.596 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:55.261+0000 7f2421cd6700 1 -- 172.21.15.97:0/3369668848 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f2408039610 msgr2=0x7f240803bad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/3297166858 2024-04-23T10:17:01.596 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:55.261+0000 7f2421cd6700 1 --2- 172.21.15.97:0/3369668848 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f2408039610 0x7f240803bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-23T10:17:01.596 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:55.662+0000 7f2421cd6700 1 -- 172.21.15.97:0/3369668848 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f2408039610 msgr2=0x7f240803bad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/3297166858 2024-04-23T10:17:01.596 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:55.662+0000 7f2421cd6700 1 --2- 172.21.15.97:0/3369668848 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f2408039610 0x7f240803bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-23T10:17:01.596 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:56.463+0000 7f2421cd6700 1 -- 172.21.15.97:0/3369668848 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f2408039610 msgr2=0x7f240803bad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/3297166858 2024-04-23T10:17:01.596 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:56.463+0000 7f2421cd6700 1 --2- 172.21.15.97:0/3369668848 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f2408039610 0x7f240803bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-23T10:17:01.597 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:58.064+0000 7f2421cd6700 1 -- 172.21.15.97:0/3369668848 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f2408039610 msgr2=0x7f240803bad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/3297166858 2024-04-23T10:17:01.597 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:58.064+0000 7f2421cd6700 1 --2- 172.21.15.97:0/3369668848 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f2408039610 0x7f240803bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-23T10:17:01.597 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:59.469+0000 7f24137fe700 1 -- 172.21.15.97:0/3369668848 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mgrmap(e 10) ==== 46338+0+0 (secure 0 0 0) 0x7f240c02c770 con 0x7f241c1084e0 2024-04-23T10:17:01.597 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:59.469+0000 7f24137fe700 1 -- 172.21.15.97:0/3369668848 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f2408039610 msgr2=0x7f240803bad0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T10:17:01.597 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:16:59.469+0000 7f24137fe700 1 --2- 172.21.15.97:0/3369668848 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f2408039610 0x7f240803bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:17:01.597 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:00.470+0000 7f24137fe700 1 -- 172.21.15.97:0/3369668848 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7f240c04fba0 con 0x7f241c1084e0 2024-04-23T10:17:01.597 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:00.470+0000 7f24137fe700 1 --2- 172.21.15.97:0/3369668848 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f240803cff0 0x7f240803f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:17:01.597 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:00.470+0000 7f24137fe700 1 -- 172.21.15.97:0/3369668848 --> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f240c047000 con 0x7f240803cff0 2024-04-23T10:17:01.597 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:00.471+0000 7f2421cd6700 1 --2- 172.21.15.97:0/3369668848 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f240803cff0 0x7f240803f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:17:01.597 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:00.471+0000 7f2421cd6700 1 --2- 172.21.15.97:0/3369668848 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f240803cff0 0x7f240803f3e0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f2418003a10 tx=0x7f2418009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:17:01.598 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:00.472+0000 7f24137fe700 1 -- 172.21.15.97:0/3369668848 <== mgr.14152 v2:172.21.15.97:6800/3149043749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f240c047000 con 0x7f240803cff0 2024-04-23T10:17:01.598 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:00.479+0000 7f242473b700 1 -- 172.21.15.97:0/3369668848 --> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f241c069b90 con 0x7f240803cff0 2024-04-23T10:17:01.598 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:00.479+0000 7f24137fe700 1 -- 172.21.15.97:0/3369668848 <== mgr.14152 v2:172.21.15.97:6800/3149043749 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f241c069b90 con 0x7f240803cff0 2024-04-23T10:17:01.598 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:00.479+0000 7f242473b700 1 -- 172.21.15.97:0/3369668848 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f240803cff0 msgr2=0x7f240803f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:17:01.598 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:00.479+0000 7f242473b700 1 --2- 172.21.15.97:0/3369668848 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f240803cff0 0x7f240803f3e0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f2418003a10 tx=0x7f2418009270 comp rx=0 tx=0).stop 2024-04-23T10:17:01.598 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:00.479+0000 7f242473b700 1 -- 172.21.15.97:0/3369668848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f241c1084e0 msgr2=0x7f241c07db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:17:01.598 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:00.479+0000 7f242473b700 1 --2- 172.21.15.97:0/3369668848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f241c1084e0 0x7f241c07db80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f240c0119f0 tx=0x7f240c011ad0 comp rx=0 tx=0).stop 2024-04-23T10:17:01.598 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:00.479+0000 7f242473b700 1 -- 172.21.15.97:0/3369668848 shutdown_connections 2024-04-23T10:17:01.598 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:00.479+0000 7f242473b700 1 --2- 172.21.15.97:0/3369668848 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f240803cff0 0x7f240803f3e0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:17:01.598 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:00.479+0000 7f242473b700 1 --2- 172.21.15.97:0/3369668848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f241c1084e0 0x7f241c07db80 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:17:01.599 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:00.479+0000 7f242473b700 1 --2- 172.21.15.97:0/3369668848 >> [v2:172.21.15.97:6800/3297166858,v1:172.21.15.97:6801/3297166858] conn(0x7f2408039610 0x7f240803bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:17:01.599 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:00.479+0000 7f242473b700 1 -- 172.21.15.97:0/3369668848 >> 172.21.15.97:0/3369668848 conn(0x7f241c103ae0 msgr2=0x7f241c1048f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:17:01.599 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:00.479+0000 7f242473b700 1 -- 172.21.15.97:0/3369668848 shutdown_connections 2024-04-23T10:17:01.599 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:00.479+0000 7f242473b700 1 -- 172.21.15.97:0/3369668848 wait complete. 2024-04-23T10:17:01.599 INFO:teuthology.orchestra.run.smithi097.stdout:mgr epoch 9 is available 2024-04-23T10:17:01.599 INFO:teuthology.orchestra.run.smithi097.stdout:Generating a dashboard self-signed certificate... 2024-04-23T10:17:01.819 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:01 smithi097 ceph-mon[29197]: [23/Apr/2024:10:17:00] ENGINE Bus STARTING 2024-04-23T10:17:01.820 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:01 smithi097 ceph-mon[29197]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-23T10:17:01.820 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:01 smithi097 ceph-mon[29197]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-23T10:17:01.820 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:01 smithi097 ceph-mon[29197]: [23/Apr/2024:10:17:00] ENGINE Serving on https://172.21.15.97:7150 2024-04-23T10:17:01.820 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:01 smithi097 ceph-mon[29197]: [23/Apr/2024:10:17:00] ENGINE Serving on http://172.21.15.97:8765 2024-04-23T10:17:01.820 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:01 smithi097 ceph-mon[29197]: [23/Apr/2024:10:17:00] ENGINE Bus STARTED 2024-04-23T10:17:01.820 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:01 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:03.038 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-23T10:17:03.038 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:02.049+0000 7ff61a061700 1 Processor -- start 2024-04-23T10:17:03.038 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:02.049+0000 7ff61a061700 1 -- start start 2024-04-23T10:17:03.039 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:02.050+0000 7ff61a061700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6141073d0 0x7ff6141077f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:17:03.039 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:02.050+0000 7ff61a061700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff614107dc0 con 0x7ff6141073d0 2024-04-23T10:17:03.039 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:02.050+0000 7ff6137fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6141073d0 0x7ff6141077f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:17:03.039 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:02.050+0000 7ff6137fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6141073d0 0x7ff6141077f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58132/0 (socket says 172.21.15.97:58132) 2024-04-23T10:17:03.039 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:02.050+0000 7ff6137fe700 1 -- 172.21.15.97:0/343205432 learned_addr learned my addr 172.21.15.97:0/343205432 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:17:03.039 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:02.050+0000 7ff6137fe700 1 -- 172.21.15.97:0/343205432 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6141085d0 con 0x7ff6141073d0 2024-04-23T10:17:03.039 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:02.050+0000 7ff6137fe700 1 --2- 172.21.15.97:0/343205432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6141073d0 0x7ff6141077f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff6080098a0 tx=0x7ff608009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ac93162526f221b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:17:03.039 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:02.050+0000 7ff6127fc700 1 -- 172.21.15.97:0/343205432 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff608004160 con 0x7ff6141073d0 2024-04-23T10:17:03.039 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:02.050+0000 7ff6127fc700 1 -- 172.21.15.97:0/343205432 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff608004300 con 0x7ff6141073d0 2024-04-23T10:17:03.039 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:02.050+0000 7ff6127fc700 1 -- 172.21.15.97:0/343205432 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff608011a20 con 0x7ff6141073d0 2024-04-23T10:17:03.039 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:02.051+0000 7ff61a061700 1 -- 172.21.15.97:0/343205432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6141073d0 msgr2=0x7ff6141077f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:17:03.039 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:02.051+0000 7ff61a061700 1 --2- 172.21.15.97:0/343205432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6141073d0 0x7ff6141077f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff6080098a0 tx=0x7ff608009bb0 comp rx=0 tx=0).stop 2024-04-23T10:17:03.039 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:02.051+0000 7ff61a061700 1 -- 172.21.15.97:0/343205432 shutdown_connections 2024-04-23T10:17:03.039 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:02.051+0000 7ff61a061700 1 --2- 172.21.15.97:0/343205432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6141073d0 0x7ff6141077f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:17:03.039 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:02.051+0000 7ff61a061700 1 -- 172.21.15.97:0/343205432 >> 172.21.15.97:0/343205432 conn(0x7ff614102a10 msgr2=0x7ff614104e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:17:03.040 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:02.051+0000 7ff61a061700 1 -- 172.21.15.97:0/343205432 shutdown_connections 2024-04-23T10:17:03.040 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:02.051+0000 7ff61a061700 1 -- 172.21.15.97:0/343205432 wait complete. 2024-04-23T10:17:03.040 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:02.052+0000 7ff61a061700 1 Processor -- start 2024-04-23T10:17:03.040 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:02.052+0000 7ff61a061700 1 -- start start 2024-04-23T10:17:03.040 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:02.052+0000 7ff61a061700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6141073d0 0x7ff61407ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:17:03.040 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:02.052+0000 7ff61a061700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff61410b980 con 0x7ff6141073d0 2024-04-23T10:17:03.040 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:02.052+0000 7ff6137fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6141073d0 0x7ff61407ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:17:03.040 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:02.052+0000 7ff6137fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6141073d0 0x7ff61407ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58140/0 (socket says 172.21.15.97:58140) 2024-04-23T10:17:03.040 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:02.052+0000 7ff6137fe700 1 -- 172.21.15.97:0/3251794737 learned_addr learned my addr 172.21.15.97:0/3251794737 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:17:03.040 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:02.052+0000 7ff6137fe700 1 -- 172.21.15.97:0/3251794737 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff61419f170 con 0x7ff6141073d0 2024-04-23T10:17:03.040 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:02.053+0000 7ff6137fe700 1 --2- 172.21.15.97:0/3251794737 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6141073d0 0x7ff61407ba60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff608000c00 tx=0x7ff608011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:17:03.040 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:02.053+0000 7ff610ff9700 1 -- 172.21.15.97:0/3251794737 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff608003aa0 con 0x7ff6141073d0 2024-04-23T10:17:03.040 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:02.053+0000 7ff610ff9700 1 -- 172.21.15.97:0/3251794737 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff60801b8f0 con 0x7ff6141073d0 2024-04-23T10:17:03.040 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:02.053+0000 7ff610ff9700 1 -- 172.21.15.97:0/3251794737 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff608023890 con 0x7ff6141073d0 2024-04-23T10:17:03.040 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:02.053+0000 7ff61a061700 1 -- 172.21.15.97:0/3251794737 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff61419f340 con 0x7ff6141073d0 2024-04-23T10:17:03.040 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:02.053+0000 7ff61a061700 1 -- 172.21.15.97:0/3251794737 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff61419f6e0 con 0x7ff6141073d0 2024-04-23T10:17:03.041 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:02.054+0000 7ff610ff9700 1 -- 172.21.15.97:0/3251794737 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7ff60801b440 con 0x7ff6141073d0 2024-04-23T10:17:03.041 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:02.054+0000 7ff61a061700 1 -- 172.21.15.97:0/3251794737 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6140640a0 con 0x7ff6141073d0 2024-04-23T10:17:03.041 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:02.054+0000 7ff610ff9700 1 --2- 172.21.15.97:0/3251794737 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7ff5fc0390f0 0x7ff5fc03b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:17:03.041 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:02.055+0000 7ff610ff9700 1 -- 172.21.15.97:0/3251794737 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff60804dec0 con 0x7ff6141073d0 2024-04-23T10:17:03.041 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:02.055+0000 7ff612ffd700 1 --2- 172.21.15.97:0/3251794737 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7ff5fc0390f0 0x7ff5fc03b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:17:03.041 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:02.055+0000 7ff612ffd700 1 --2- 172.21.15.97:0/3251794737 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7ff5fc0390f0 0x7ff5fc03b5b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff604006fd0 tx=0x7ff604006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:17:03.041 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:02.060+0000 7ff610ff9700 1 -- 172.21.15.97:0/3251794737 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff608028060 con 0x7ff6141073d0 2024-04-23T10:17:03.041 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:02.126+0000 7ff610ff9700 1 -- 172.21.15.97:0/3251794737 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7ff608018350 con 0x7ff6141073d0 2024-04-23T10:17:03.041 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:02.285+0000 7ff61a061700 1 -- 172.21.15.97:0/3251794737 --> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7ff614103bf0 con 0x7ff5fc0390f0 2024-04-23T10:17:03.041 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:02.433+0000 7ff610ff9700 1 -- 172.21.15.97:0/3251794737 <== mgr.14152 v2:172.21.15.97:6800/3149043749 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7ff614103bf0 con 0x7ff5fc0390f0 2024-04-23T10:17:03.041 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:02.435+0000 7ff61a061700 1 -- 172.21.15.97:0/3251794737 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7ff5fc0390f0 msgr2=0x7ff5fc03b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:17:03.041 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:02.435+0000 7ff61a061700 1 --2- 172.21.15.97:0/3251794737 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7ff5fc0390f0 0x7ff5fc03b5b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff604006fd0 tx=0x7ff604006e40 comp rx=0 tx=0).stop 2024-04-23T10:17:03.041 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:02.435+0000 7ff61a061700 1 -- 172.21.15.97:0/3251794737 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6141073d0 msgr2=0x7ff61407ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:17:03.041 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:02.435+0000 7ff61a061700 1 --2- 172.21.15.97:0/3251794737 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6141073d0 0x7ff61407ba60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff608000c00 tx=0x7ff608011aa0 comp rx=0 tx=0).stop 2024-04-23T10:17:03.042 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:02.435+0000 7ff61a061700 1 -- 172.21.15.97:0/3251794737 shutdown_connections 2024-04-23T10:17:03.042 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:02.435+0000 7ff61a061700 1 --2- 172.21.15.97:0/3251794737 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7ff5fc0390f0 0x7ff5fc03b5b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:17:03.042 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:02.435+0000 7ff61a061700 1 --2- 172.21.15.97:0/3251794737 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6141073d0 0x7ff61407ba60 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:17:03.042 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:02.435+0000 7ff61a061700 1 -- 172.21.15.97:0/3251794737 >> 172.21.15.97:0/3251794737 conn(0x7ff614102a10 msgr2=0x7ff6141034e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:17:03.042 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:02.435+0000 7ff61a061700 1 -- 172.21.15.97:0/3251794737 shutdown_connections 2024-04-23T10:17:03.042 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:02.435+0000 7ff61a061700 1 -- 172.21.15.97:0/3251794737 wait complete. 2024-04-23T10:17:03.042 INFO:teuthology.orchestra.run.smithi097.stdout:Creating initial admin user... 2024-04-23T10:17:03.292 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:03 smithi097 ceph-mon[29197]: mgrmap e12: x(active, since 2s) 2024-04-23T10:17:03.292 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:03 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:03.292 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:03 smithi097 ceph-mon[29197]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:17:03.292 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:03 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:03.293 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:03 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:05.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:04 smithi097 ceph-mon[29197]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:17:05.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:04 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:05.232 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$13fmCGOw0u1WX8hYjv4/2OW2g/0yaHw7lSQHNickIbtRU3d8EPsCO", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1713867423, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-23T10:17:05.232 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:03.509+0000 7fdbd8114700 1 Processor -- start 2024-04-23T10:17:05.232 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:03.509+0000 7fdbd8114700 1 -- start start 2024-04-23T10:17:05.232 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:03.510+0000 7fdbd8114700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbd01084e0 0x7fdbd0108900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:17:05.233 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:03.510+0000 7fdbd8114700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd0108ed0 con 0x7fdbd01084e0 2024-04-23T10:17:05.233 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:03.510+0000 7fdbd5eb0700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbd01084e0 0x7fdbd0108900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:17:05.233 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:03.510+0000 7fdbd5eb0700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbd01084e0 0x7fdbd0108900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58156/0 (socket says 172.21.15.97:58156) 2024-04-23T10:17:05.233 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:03.510+0000 7fdbd5eb0700 1 -- 172.21.15.97:0/1788092098 learned_addr learned my addr 172.21.15.97:0/1788092098 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:17:05.233 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:03.510+0000 7fdbd5eb0700 1 -- 172.21.15.97:0/1788092098 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbd0109690 con 0x7fdbd01084e0 2024-04-23T10:17:05.233 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:03.510+0000 7fdbd5eb0700 1 --2- 172.21.15.97:0/1788092098 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbd01084e0 0x7fdbd0108900 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdbc00098a0 tx=0x7fdbc0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c06cdcdb5495cf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:17:05.233 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:03.510+0000 7fdbd4eae700 1 -- 172.21.15.97:0/1788092098 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdbc0004160 con 0x7fdbd01084e0 2024-04-23T10:17:05.233 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:03.511+0000 7fdbd4eae700 1 -- 172.21.15.97:0/1788092098 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbc0004300 con 0x7fdbd01084e0 2024-04-23T10:17:05.233 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:03.511+0000 7fdbd4eae700 1 -- 172.21.15.97:0/1788092098 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdbc0011a20 con 0x7fdbd01084e0 2024-04-23T10:17:05.233 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:03.511+0000 7fdbd8114700 1 -- 172.21.15.97:0/1788092098 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbd01084e0 msgr2=0x7fdbd0108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:17:05.233 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:03.511+0000 7fdbd8114700 1 --2- 172.21.15.97:0/1788092098 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbd01084e0 0x7fdbd0108900 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdbc00098a0 tx=0x7fdbc0009bb0 comp rx=0 tx=0).stop 2024-04-23T10:17:05.233 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:03.511+0000 7fdbd8114700 1 -- 172.21.15.97:0/1788092098 shutdown_connections 2024-04-23T10:17:05.233 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:03.511+0000 7fdbd8114700 1 --2- 172.21.15.97:0/1788092098 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbd01084e0 0x7fdbd0108900 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:17:05.233 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:03.511+0000 7fdbd8114700 1 -- 172.21.15.97:0/1788092098 >> 172.21.15.97:0/1788092098 conn(0x7fdbd0103b40 msgr2=0x7fdbd0105f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:17:05.234 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:03.511+0000 7fdbd8114700 1 -- 172.21.15.97:0/1788092098 shutdown_connections 2024-04-23T10:17:05.234 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:03.511+0000 7fdbd8114700 1 -- 172.21.15.97:0/1788092098 wait complete. 2024-04-23T10:17:05.234 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:03.512+0000 7fdbd8114700 1 Processor -- start 2024-04-23T10:17:05.234 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:03.512+0000 7fdbd8114700 1 -- start start 2024-04-23T10:17:05.234 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:03.512+0000 7fdbd8114700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbd01084e0 0x7fdbd019c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:17:05.234 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:03.512+0000 7fdbd8114700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd010ca40 con 0x7fdbd01084e0 2024-04-23T10:17:05.234 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:03.512+0000 7fdbd5eb0700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbd01084e0 0x7fdbd019c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:17:05.234 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:03.512+0000 7fdbd5eb0700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbd01084e0 0x7fdbd019c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58166/0 (socket says 172.21.15.97:58166) 2024-04-23T10:17:05.234 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:03.512+0000 7fdbd5eb0700 1 -- 172.21.15.97:0/671029130 learned_addr learned my addr 172.21.15.97:0/671029130 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:17:05.234 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:03.512+0000 7fdbd5eb0700 1 -- 172.21.15.97:0/671029130 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbd019c940 con 0x7fdbd01084e0 2024-04-23T10:17:05.234 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:03.512+0000 7fdbd5eb0700 1 --2- 172.21.15.97:0/671029130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbd01084e0 0x7fdbd019c400 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdbc00119f0 tx=0x7fdbc0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:17:05.234 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:03.513+0000 7fdbc6ffd700 1 -- 172.21.15.97:0/671029130 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdbc00039e0 con 0x7fdbd01084e0 2024-04-23T10:17:05.235 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:03.513+0000 7fdbc6ffd700 1 -- 172.21.15.97:0/671029130 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbc001b8f0 con 0x7fdbd01084e0 2024-04-23T10:17:05.235 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:03.513+0000 7fdbd8114700 1 -- 172.21.15.97:0/671029130 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbd019cbd0 con 0x7fdbd01084e0 2024-04-23T10:17:05.235 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:03.513+0000 7fdbc6ffd700 1 -- 172.21.15.97:0/671029130 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdbc00237a0 con 0x7fdbd01084e0 2024-04-23T10:17:05.235 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:03.513+0000 7fdbd8114700 1 -- 172.21.15.97:0/671029130 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbd019f8c0 con 0x7fdbd01084e0 2024-04-23T10:17:05.235 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:03.513+0000 7fdbc6ffd700 1 -- 172.21.15.97:0/671029130 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7fdbc002c3f0 con 0x7fdbd01084e0 2024-04-23T10:17:05.235 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:03.514+0000 7fdbc6ffd700 1 --2- 172.21.15.97:0/671029130 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fdbbc039610 0x7fdbbc03bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:17:05.235 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:03.514+0000 7fdbd8114700 1 -- 172.21.15.97:0/671029130 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbd00640a0 con 0x7fdbd01084e0 2024-04-23T10:17:05.235 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:03.514+0000 7fdbc6ffd700 1 -- 172.21.15.97:0/671029130 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fdbc004dfa0 con 0x7fdbd01084e0 2024-04-23T10:17:05.235 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:03.514+0000 7fdbd56af700 1 --2- 172.21.15.97:0/671029130 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fdbbc039610 0x7fdbbc03bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:17:05.235 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:03.514+0000 7fdbd56af700 1 --2- 172.21.15.97:0/671029130 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fdbbc039610 0x7fdbbc03bad0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc006fd0 tx=0x7fdbcc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:17:05.235 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:03.519+0000 7fdbc6ffd700 1 -- 172.21.15.97:0/671029130 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdbc001f030 con 0x7fdbd01084e0 2024-04-23T10:17:05.235 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:03.749+0000 7fdbd8114700 1 -- 172.21.15.97:0/671029130 --> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fdbd019d0b0 con 0x7fdbbc039610 2024-04-23T10:17:05.235 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:03.973+0000 7fdbc6ffd700 1 -- 172.21.15.97:0/671029130 <== mgr.14152 v2:172.21.15.97:6800/3149043749 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fdbd019d0b0 con 0x7fdbbc039610 2024-04-23T10:17:05.235 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:03.975+0000 7fdbd8114700 1 -- 172.21.15.97:0/671029130 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fdbbc039610 msgr2=0x7fdbbc03bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:17:05.235 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:03.975+0000 7fdbd8114700 1 --2- 172.21.15.97:0/671029130 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fdbbc039610 0x7fdbbc03bad0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc006fd0 tx=0x7fdbcc006e40 comp rx=0 tx=0).stop 2024-04-23T10:17:05.236 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:03.975+0000 7fdbd8114700 1 -- 172.21.15.97:0/671029130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbd01084e0 msgr2=0x7fdbd019c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:17:05.236 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:03.975+0000 7fdbd8114700 1 --2- 172.21.15.97:0/671029130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbd01084e0 0x7fdbd019c400 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdbc00119f0 tx=0x7fdbc0011ad0 comp rx=0 tx=0).stop 2024-04-23T10:17:05.236 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:03.975+0000 7fdbd8114700 1 -- 172.21.15.97:0/671029130 shutdown_connections 2024-04-23T10:17:05.236 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:03.975+0000 7fdbd8114700 1 --2- 172.21.15.97:0/671029130 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fdbbc039610 0x7fdbbc03bad0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:17:05.236 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:03.975+0000 7fdbd8114700 1 --2- 172.21.15.97:0/671029130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbd01084e0 0x7fdbd019c400 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:17:05.236 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:03.975+0000 7fdbd8114700 1 -- 172.21.15.97:0/671029130 >> 172.21.15.97:0/671029130 conn(0x7fdbd0103b40 msgr2=0x7fdbd01046b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:17:05.236 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:03.976+0000 7fdbd8114700 1 -- 172.21.15.97:0/671029130 shutdown_connections 2024-04-23T10:17:05.236 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:03.976+0000 7fdbd8114700 1 -- 172.21.15.97:0/671029130 wait complete. 2024-04-23T10:17:05.236 INFO:teuthology.orchestra.run.smithi097.stdout:Fetching dashboard port number... 2024-04-23T10:17:06.189 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:05 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/1246434857' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-23T10:17:06.498 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stdout 8443 2024-04-23T10:17:06.498 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:05.661+0000 7f8fcbb69700 1 Processor -- start 2024-04-23T10:17:06.499 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:05.661+0000 7f8fcbb69700 1 -- start start 2024-04-23T10:17:06.499 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:05.662+0000 7f8fcbb69700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fc4108f60 0x7f8fc410b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:17:06.499 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:05.662+0000 7f8fcbb69700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc407be60 con 0x7f8fc4108f60 2024-04-23T10:17:06.499 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:05.662+0000 7f8fc9905700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fc4108f60 0x7f8fc410b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:17:06.499 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:05.662+0000 7f8fc9905700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fc4108f60 0x7f8fc410b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40876/0 (socket says 172.21.15.97:40876) 2024-04-23T10:17:06.499 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:05.662+0000 7f8fc9905700 1 -- 172.21.15.97:0/3208626502 learned_addr learned my addr 172.21.15.97:0/3208626502 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:17:06.499 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:05.662+0000 7f8fc9905700 1 -- 172.21.15.97:0/3208626502 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fc410b890 con 0x7f8fc4108f60 2024-04-23T10:17:06.499 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:05.662+0000 7f8fc9905700 1 --2- 172.21.15.97:0/3208626502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fc4108f60 0x7f8fc410b350 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8fb40098a0 tx=0x7f8fb4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80160002c85f9726 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:17:06.499 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:05.662+0000 7f8fc8903700 1 -- 172.21.15.97:0/3208626502 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8fb4004160 con 0x7f8fc4108f60 2024-04-23T10:17:06.499 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:05.662+0000 7f8fc8903700 1 -- 172.21.15.97:0/3208626502 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fb4004300 con 0x7f8fc4108f60 2024-04-23T10:17:06.500 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:05.662+0000 7f8fc8903700 1 -- 172.21.15.97:0/3208626502 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8fb4011a20 con 0x7f8fc4108f60 2024-04-23T10:17:06.500 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:05.663+0000 7f8fcbb69700 1 -- 172.21.15.97:0/3208626502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fc4108f60 msgr2=0x7f8fc410b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:17:06.500 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:05.663+0000 7f8fcbb69700 1 --2- 172.21.15.97:0/3208626502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fc4108f60 0x7f8fc410b350 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8fb40098a0 tx=0x7f8fb4009bb0 comp rx=0 tx=0).stop 2024-04-23T10:17:06.500 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:05.663+0000 7f8fcbb69700 1 -- 172.21.15.97:0/3208626502 shutdown_connections 2024-04-23T10:17:06.500 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:05.663+0000 7f8fcbb69700 1 --2- 172.21.15.97:0/3208626502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fc4108f60 0x7f8fc410b350 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:17:06.500 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:05.663+0000 7f8fcbb69700 1 -- 172.21.15.97:0/3208626502 >> 172.21.15.97:0/3208626502 conn(0x7f8fc41029b0 msgr2=0x7f8fc4104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:17:06.500 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:05.663+0000 7f8fcbb69700 1 -- 172.21.15.97:0/3208626502 shutdown_connections 2024-04-23T10:17:06.500 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:05.663+0000 7f8fcbb69700 1 -- 172.21.15.97:0/3208626502 wait complete. 2024-04-23T10:17:06.500 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:05.663+0000 7f8fcbb69700 1 Processor -- start 2024-04-23T10:17:06.501 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:05.664+0000 7f8fcbb69700 1 -- start start 2024-04-23T10:17:06.501 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:05.664+0000 7f8fcbb69700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fc4108f60 0x7f8fc419c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:17:06.501 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:05.664+0000 7f8fcbb69700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc410c840 con 0x7f8fc4108f60 2024-04-23T10:17:06.501 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:05.664+0000 7f8fc9905700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fc4108f60 0x7f8fc419c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:17:06.501 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:05.664+0000 7f8fc9905700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fc4108f60 0x7f8fc419c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40878/0 (socket says 172.21.15.97:40878) 2024-04-23T10:17:06.501 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:05.664+0000 7f8fc9905700 1 -- 172.21.15.97:0/1246434857 learned_addr learned my addr 172.21.15.97:0/1246434857 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:17:06.501 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:05.664+0000 7f8fc9905700 1 -- 172.21.15.97:0/1246434857 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fc419c940 con 0x7f8fc4108f60 2024-04-23T10:17:06.501 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:05.664+0000 7f8fc9905700 1 --2- 172.21.15.97:0/1246434857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fc4108f60 0x7f8fc419c400 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4004660 tx=0x7f8fb4011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:17:06.501 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:05.664+0000 7f8fbaffd700 1 -- 172.21.15.97:0/1246434857 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8fb4003aa0 con 0x7f8fc4108f60 2024-04-23T10:17:06.501 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:05.664+0000 7f8fbaffd700 1 -- 172.21.15.97:0/1246434857 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fb401b8f0 con 0x7f8fc4108f60 2024-04-23T10:17:06.501 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:05.664+0000 7f8fcbb69700 1 -- 172.21.15.97:0/1246434857 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fc419cbd0 con 0x7f8fc4108f60 2024-04-23T10:17:06.502 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:05.664+0000 7f8fbaffd700 1 -- 172.21.15.97:0/1246434857 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8fb4023900 con 0x7f8fc4108f60 2024-04-23T10:17:06.502 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:05.665+0000 7f8fcbb69700 1 -- 172.21.15.97:0/1246434857 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fc419f8c0 con 0x7f8fc4108f60 2024-04-23T10:17:06.502 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:05.665+0000 7f8fbaffd700 1 -- 172.21.15.97:0/1246434857 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f8fb4023bc0 con 0x7f8fc4108f60 2024-04-23T10:17:06.502 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:05.665+0000 7f8fbaffd700 1 --2- 172.21.15.97:0/1246434857 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f8fb0039600 0x7f8fb003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:17:06.502 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:05.665+0000 7f8fbaffd700 1 -- 172.21.15.97:0/1246434857 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8fb404e040 con 0x7f8fc4108f60 2024-04-23T10:17:06.502 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:05.665+0000 7f8fcbb69700 1 -- 172.21.15.97:0/1246434857 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fc40640a0 con 0x7f8fc4108f60 2024-04-23T10:17:06.502 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:05.665+0000 7f8fc9104700 1 --2- 172.21.15.97:0/1246434857 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f8fb0039600 0x7f8fb003bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:17:06.502 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:05.666+0000 7f8fc9104700 1 --2- 172.21.15.97:0/1246434857 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f8fb0039600 0x7f8fb003bac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0006fd0 tx=0x7f8fc0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:17:06.503 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:05.670+0000 7f8fbaffd700 1 -- 172.21.15.97:0/1246434857 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8fb401f030 con 0x7f8fc4108f60 2024-04-23T10:17:06.503 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:05.867+0000 7f8fcbb69700 1 -- 172.21.15.97:0/1246434857 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f8fc419fa90 con 0x7f8fc4108f60 2024-04-23T10:17:06.503 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:05.867+0000 7f8fbaffd700 1 -- 172.21.15.97:0/1246434857 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f8fb4010380 con 0x7f8fc4108f60 2024-04-23T10:17:06.503 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:05.869+0000 7f8fcbb69700 1 -- 172.21.15.97:0/1246434857 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f8fb0039600 msgr2=0x7f8fb003bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:17:06.503 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:05.869+0000 7f8fcbb69700 1 --2- 172.21.15.97:0/1246434857 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f8fb0039600 0x7f8fb003bac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0006fd0 tx=0x7f8fc0006e40 comp rx=0 tx=0).stop 2024-04-23T10:17:06.503 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:05.869+0000 7f8fcbb69700 1 -- 172.21.15.97:0/1246434857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fc4108f60 msgr2=0x7f8fc419c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:17:06.503 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:05.869+0000 7f8fcbb69700 1 --2- 172.21.15.97:0/1246434857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fc4108f60 0x7f8fc419c400 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4004660 tx=0x7f8fb4011aa0 comp rx=0 tx=0).stop 2024-04-23T10:17:06.503 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:05.869+0000 7f8fcbb69700 1 -- 172.21.15.97:0/1246434857 shutdown_connections 2024-04-23T10:17:06.503 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:05.869+0000 7f8fcbb69700 1 --2- 172.21.15.97:0/1246434857 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f8fb0039600 0x7f8fb003bac0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:17:06.504 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:05.869+0000 7f8fcbb69700 1 --2- 172.21.15.97:0/1246434857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fc4108f60 0x7f8fc419c400 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:17:06.504 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:05.869+0000 7f8fcbb69700 1 -- 172.21.15.97:0/1246434857 >> 172.21.15.97:0/1246434857 conn(0x7f8fc41029b0 msgr2=0x7f8fc4103540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:17:06.504 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:05.869+0000 7f8fcbb69700 1 -- 172.21.15.97:0/1246434857 shutdown_connections 2024-04-23T10:17:06.504 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:05.869+0000 7f8fcbb69700 1 -- 172.21.15.97:0/1246434857 wait complete. 2024-04-23T10:17:06.504 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-23T10:17:06.504 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout disabled 2024-04-23T10:17:06.511 INFO:teuthology.orchestra.run.smithi097.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-23T10:17:06.511 INFO:teuthology.orchestra.run.smithi097.stdout:systemctl: stdout inactive 2024-04-23T10:17:06.511 INFO:teuthology.orchestra.run.smithi097.stdout:firewalld.service is not enabled 2024-04-23T10:17:06.511 INFO:teuthology.orchestra.run.smithi097.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-23T10:17:06.512 INFO:teuthology.orchestra.run.smithi097.stdout:Ceph Dashboard is now available at: 2024-04-23T10:17:06.512 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:17:06.512 INFO:teuthology.orchestra.run.smithi097.stdout: URL: https://smithi097:8443/ 2024-04-23T10:17:06.513 INFO:teuthology.orchestra.run.smithi097.stdout: User: admin 2024-04-23T10:17:06.513 INFO:teuthology.orchestra.run.smithi097.stdout: Password: 3o4avr8mzf 2024-04-23T10:17:06.513 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:17:06.513 INFO:teuthology.orchestra.run.smithi097.stdout:Saving cluster configuration to /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/config directory 2024-04-23T10:17:06.513 INFO:teuthology.orchestra.run.smithi097.stdout:Enabling autotune for osd_memory_target 2024-04-23T10:17:07.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:06 smithi097 ceph-mon[29197]: mgrmap e13: x(active, since 6s) 2024-04-23T10:17:07.747 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:06.977+0000 7f2f4f95a700 1 Processor -- start 2024-04-23T10:17:07.747 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:06.977+0000 7f2f4f95a700 1 -- start start 2024-04-23T10:17:07.747 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:06.977+0000 7f2f4f95a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f48106d50 0x7f2f48107170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:17:07.747 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:06.977+0000 7f2f4f95a700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2f4807be60 con 0x7f2f48106d50 2024-04-23T10:17:07.748 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:06.978+0000 7f2f4d6f6700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f48106d50 0x7f2f48107170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:17:07.748 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:06.978+0000 7f2f4d6f6700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f48106d50 0x7f2f48107170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40890/0 (socket says 172.21.15.97:40890) 2024-04-23T10:17:07.748 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:06.978+0000 7f2f4d6f6700 1 -- 172.21.15.97:0/1697267757 learned_addr learned my addr 172.21.15.97:0/1697267757 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:17:07.748 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:06.978+0000 7f2f4d6f6700 1 -- 172.21.15.97:0/1697267757 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f481076b0 con 0x7f2f48106d50 2024-04-23T10:17:07.748 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:06.978+0000 7f2f4d6f6700 1 --2- 172.21.15.97:0/1697267757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f48106d50 0x7f2f48107170 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2f440098a0 tx=0x7f2f44009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aaa1e3382a2db74 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:17:07.748 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:06.978+0000 7f2f3ffff700 1 -- 172.21.15.97:0/1697267757 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f44004160 con 0x7f2f48106d50 2024-04-23T10:17:07.748 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:06.979+0000 7f2f3ffff700 1 -- 172.21.15.97:0/1697267757 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f44004300 con 0x7f2f48106d50 2024-04-23T10:17:07.748 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:06.979+0000 7f2f3ffff700 1 -- 172.21.15.97:0/1697267757 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f44011a20 con 0x7f2f48106d50 2024-04-23T10:17:07.748 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:06.979+0000 7f2f4f95a700 1 -- 172.21.15.97:0/1697267757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f48106d50 msgr2=0x7f2f48107170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:17:07.748 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:06.979+0000 7f2f4f95a700 1 --2- 172.21.15.97:0/1697267757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f48106d50 0x7f2f48107170 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2f440098a0 tx=0x7f2f44009bb0 comp rx=0 tx=0).stop 2024-04-23T10:17:07.748 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:06.979+0000 7f2f4f95a700 1 -- 172.21.15.97:0/1697267757 shutdown_connections 2024-04-23T10:17:07.748 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:06.979+0000 7f2f4f95a700 1 --2- 172.21.15.97:0/1697267757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f48106d50 0x7f2f48107170 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:17:07.749 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:06.979+0000 7f2f4f95a700 1 -- 172.21.15.97:0/1697267757 >> 172.21.15.97:0/1697267757 conn(0x7f2f481029b0 msgr2=0x7f2f48104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:17:07.749 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:06.979+0000 7f2f4f95a700 1 -- 172.21.15.97:0/1697267757 shutdown_connections 2024-04-23T10:17:07.749 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:06.979+0000 7f2f4f95a700 1 -- 172.21.15.97:0/1697267757 wait complete. 2024-04-23T10:17:07.749 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:06.980+0000 7f2f4f95a700 1 Processor -- start 2024-04-23T10:17:07.749 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:06.980+0000 7f2f4f95a700 1 -- start start 2024-04-23T10:17:07.749 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:06.980+0000 7f2f4f95a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f48106d50 0x7f2f4819e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:17:07.749 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:06.980+0000 7f2f4f95a700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2f48108720 con 0x7f2f48106d50 2024-04-23T10:17:07.749 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:06.980+0000 7f2f4d6f6700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f48106d50 0x7f2f4819e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:17:07.749 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:06.980+0000 7f2f4d6f6700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f48106d50 0x7f2f4819e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40896/0 (socket says 172.21.15.97:40896) 2024-04-23T10:17:07.749 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:06.980+0000 7f2f4d6f6700 1 -- 172.21.15.97:0/3867085168 learned_addr learned my addr 172.21.15.97:0/3867085168 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:17:07.749 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:06.980+0000 7f2f4d6f6700 1 -- 172.21.15.97:0/3867085168 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f4819eb30 con 0x7f2f48106d50 2024-04-23T10:17:07.749 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:06.981+0000 7f2f4d6f6700 1 --2- 172.21.15.97:0/3867085168 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f48106d50 0x7f2f4819e5f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2f44004660 tx=0x7f2f44011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:17:07.749 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:06.981+0000 7f2f3e7fc700 1 -- 172.21.15.97:0/3867085168 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f44003aa0 con 0x7f2f48106d50 2024-04-23T10:17:07.749 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:06.981+0000 7f2f3e7fc700 1 -- 172.21.15.97:0/3867085168 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f4401b8f0 con 0x7f2f48106d50 2024-04-23T10:17:07.750 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:06.981+0000 7f2f4f95a700 1 -- 172.21.15.97:0/3867085168 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f4819edc0 con 0x7f2f48106d50 2024-04-23T10:17:07.750 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:06.981+0000 7f2f3e7fc700 1 -- 172.21.15.97:0/3867085168 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f44023900 con 0x7f2f48106d50 2024-04-23T10:17:07.750 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:06.981+0000 7f2f4f95a700 1 -- 172.21.15.97:0/3867085168 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f481a1ab0 con 0x7f2f48106d50 2024-04-23T10:17:07.750 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:06.982+0000 7f2f3e7fc700 1 -- 172.21.15.97:0/3867085168 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f2f44023b70 con 0x7f2f48106d50 2024-04-23T10:17:07.750 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:06.982+0000 7f2f4f95a700 1 -- 172.21.15.97:0/3867085168 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f480640a0 con 0x7f2f48106d50 2024-04-23T10:17:07.750 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:06.982+0000 7f2f3e7fc700 1 --2- 172.21.15.97:0/3867085168 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f2f34039700 0x7f2f3403bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:17:07.750 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:06.982+0000 7f2f3e7fc700 1 -- 172.21.15.97:0/3867085168 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f2f4404e150 con 0x7f2f48106d50 2024-04-23T10:17:07.750 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:06.982+0000 7f2f4cef5700 1 --2- 172.21.15.97:0/3867085168 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f2f34039700 0x7f2f3403bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:17:07.750 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:06.982+0000 7f2f4cef5700 1 --2- 172.21.15.97:0/3867085168 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f2f34039700 0x7f2f3403bbc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2f38006fd0 tx=0x7f2f38006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:17:07.750 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:06.987+0000 7f2f3e7fc700 1 -- 172.21.15.97:0/3867085168 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2f4401f030 con 0x7f2f48106d50 2024-04-23T10:17:07.750 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:07.184+0000 7f2f4f95a700 1 -- 172.21.15.97:0/3867085168 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f2f4802e2c0 con 0x7f2f48106d50 2024-04-23T10:17:07.750 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:07.185+0000 7f2f3e7fc700 1 -- 172.21.15.97:0/3867085168 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f2f4401bde0 con 0x7f2f48106d50 2024-04-23T10:17:07.751 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:07.187+0000 7f2f4f95a700 1 -- 172.21.15.97:0/3867085168 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f2f34039700 msgr2=0x7f2f3403bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:17:07.751 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:07.187+0000 7f2f4f95a700 1 --2- 172.21.15.97:0/3867085168 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f2f34039700 0x7f2f3403bbc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2f38006fd0 tx=0x7f2f38006e40 comp rx=0 tx=0).stop 2024-04-23T10:17:07.751 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:07.187+0000 7f2f4f95a700 1 -- 172.21.15.97:0/3867085168 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f48106d50 msgr2=0x7f2f4819e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:17:07.751 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:07.187+0000 7f2f4f95a700 1 --2- 172.21.15.97:0/3867085168 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f48106d50 0x7f2f4819e5f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2f44004660 tx=0x7f2f44011aa0 comp rx=0 tx=0).stop 2024-04-23T10:17:07.751 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:07.187+0000 7f2f4f95a700 1 -- 172.21.15.97:0/3867085168 shutdown_connections 2024-04-23T10:17:07.751 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:07.187+0000 7f2f4f95a700 1 --2- 172.21.15.97:0/3867085168 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f2f34039700 0x7f2f3403bbc0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:17:07.751 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:07.187+0000 7f2f4f95a700 1 --2- 172.21.15.97:0/3867085168 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2f48106d50 0x7f2f4819e5f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:17:07.751 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:07.187+0000 7f2f4f95a700 1 -- 172.21.15.97:0/3867085168 >> 172.21.15.97:0/3867085168 conn(0x7f2f481029b0 msgr2=0x7f2f48195260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:17:07.751 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:07.187+0000 7f2f4f95a700 1 -- 172.21.15.97:0/3867085168 shutdown_connections 2024-04-23T10:17:07.751 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:07.187+0000 7f2f4f95a700 1 -- 172.21.15.97:0/3867085168 wait complete. 2024-04-23T10:17:09.085 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:08.157+0000 7f48331d1700 1 Processor -- start 2024-04-23T10:17:09.086 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:08.157+0000 7f48331d1700 1 -- start start 2024-04-23T10:17:09.086 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:08.157+0000 7f48331d1700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f482c1067a0 0x7f482c106bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:17:09.086 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:08.157+0000 7f48331d1700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f482c107190 con 0x7f482c1067a0 2024-04-23T10:17:09.086 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:08.158+0000 7f4830f6d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f482c1067a0 0x7f482c106bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:17:09.086 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:08.158+0000 7f4830f6d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f482c1067a0 0x7f482c106bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40898/0 (socket says 172.21.15.97:40898) 2024-04-23T10:17:09.086 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:08.158+0000 7f4830f6d700 1 -- 172.21.15.97:0/1058392005 learned_addr learned my addr 172.21.15.97:0/1058392005 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:17:09.086 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:08.158+0000 7f4830f6d700 1 -- 172.21.15.97:0/1058392005 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f482c109730 con 0x7f482c1067a0 2024-04-23T10:17:09.086 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:08.158+0000 7f4830f6d700 1 --2- 172.21.15.97:0/1058392005 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f482c1067a0 0x7f482c106bc0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f48200098a0 tx=0x7f4820009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c7bbbbc2cd72fdd server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:17:09.086 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:08.158+0000 7f482b7fe700 1 -- 172.21.15.97:0/1058392005 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4820004160 con 0x7f482c1067a0 2024-04-23T10:17:09.086 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:08.158+0000 7f482b7fe700 1 -- 172.21.15.97:0/1058392005 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4820004300 con 0x7f482c1067a0 2024-04-23T10:17:09.086 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:08.158+0000 7f482b7fe700 1 -- 172.21.15.97:0/1058392005 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4820011a20 con 0x7f482c1067a0 2024-04-23T10:17:09.086 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:08.159+0000 7f48331d1700 1 -- 172.21.15.97:0/1058392005 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f482c1067a0 msgr2=0x7f482c106bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:17:09.086 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:08.159+0000 7f48331d1700 1 --2- 172.21.15.97:0/1058392005 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f482c1067a0 0x7f482c106bc0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f48200098a0 tx=0x7f4820009bb0 comp rx=0 tx=0).stop 2024-04-23T10:17:09.087 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:08.159+0000 7f48331d1700 1 -- 172.21.15.97:0/1058392005 shutdown_connections 2024-04-23T10:17:09.087 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:08.159+0000 7f48331d1700 1 --2- 172.21.15.97:0/1058392005 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f482c1067a0 0x7f482c106bc0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:17:09.087 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:08.159+0000 7f48331d1700 1 -- 172.21.15.97:0/1058392005 >> 172.21.15.97:0/1058392005 conn(0x7f482c102110 msgr2=0x7f482c104530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:17:09.087 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:08.159+0000 7f48331d1700 1 -- 172.21.15.97:0/1058392005 shutdown_connections 2024-04-23T10:17:09.087 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:08.159+0000 7f48331d1700 1 -- 172.21.15.97:0/1058392005 wait complete. 2024-04-23T10:17:09.087 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:08.159+0000 7f48331d1700 1 Processor -- start 2024-04-23T10:17:09.087 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:08.159+0000 7f48331d1700 1 -- start start 2024-04-23T10:17:09.087 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:08.160+0000 7f48331d1700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f482c1067a0 0x7f482c19c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:17:09.087 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:08.160+0000 7f48331d1700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f482c10cae0 con 0x7f482c1067a0 2024-04-23T10:17:09.087 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:08.160+0000 7f4830f6d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f482c1067a0 0x7f482c19c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:17:09.087 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:08.160+0000 7f4830f6d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f482c1067a0 0x7f482c19c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40914/0 (socket says 172.21.15.97:40914) 2024-04-23T10:17:09.087 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:08.160+0000 7f4830f6d700 1 -- 172.21.15.97:0/935146618 learned_addr learned my addr 172.21.15.97:0/935146618 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:17:09.087 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:08.160+0000 7f4830f6d700 1 -- 172.21.15.97:0/935146618 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f482c19c960 con 0x7f482c1067a0 2024-04-23T10:17:09.087 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:08.160+0000 7f4830f6d700 1 --2- 172.21.15.97:0/935146618 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f482c1067a0 0x7f482c19c420 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f48200119f0 tx=0x7f4820011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:17:09.087 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:08.160+0000 7f4829ffb700 1 -- 172.21.15.97:0/935146618 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48200039e0 con 0x7f482c1067a0 2024-04-23T10:17:09.088 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:08.160+0000 7f4829ffb700 1 -- 172.21.15.97:0/935146618 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f482001b8f0 con 0x7f482c1067a0 2024-04-23T10:17:09.088 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:08.160+0000 7f48331d1700 1 -- 172.21.15.97:0/935146618 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f482c19cbf0 con 0x7f482c1067a0 2024-04-23T10:17:09.088 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:08.160+0000 7f4829ffb700 1 -- 172.21.15.97:0/935146618 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48200237a0 con 0x7f482c1067a0 2024-04-23T10:17:09.088 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:08.160+0000 7f48331d1700 1 -- 172.21.15.97:0/935146618 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f482c19f8e0 con 0x7f482c1067a0 2024-04-23T10:17:09.088 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:08.161+0000 7f4829ffb700 1 -- 172.21.15.97:0/935146618 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f482001b440 con 0x7f482c1067a0 2024-04-23T10:17:09.088 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:08.161+0000 7f48331d1700 1 -- 172.21.15.97:0/935146618 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f482c0640a0 con 0x7f482c1067a0 2024-04-23T10:17:09.088 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:08.161+0000 7f4829ffb700 1 --2- 172.21.15.97:0/935146618 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f481c039650 0x7f481c03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:17:09.088 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:08.161+0000 7f4829ffb700 1 -- 172.21.15.97:0/935146618 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f482004e060 con 0x7f482c1067a0 2024-04-23T10:17:09.088 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:08.162+0000 7f482bfff700 1 --2- 172.21.15.97:0/935146618 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f481c039650 0x7f481c03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:17:09.088 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:08.162+0000 7f482bfff700 1 --2- 172.21.15.97:0/935146618 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f481c039650 0x7f481c03bb10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4818006fd0 tx=0x7f4818006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:17:09.088 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:08.167+0000 7f4829ffb700 1 -- 172.21.15.97:0/935146618 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f482001f030 con 0x7f482c1067a0 2024-04-23T10:17:09.088 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:08.473+0000 7f48331d1700 1 -- 172.21.15.97:0/935146618 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f482c19fe90 con 0x7f482c1067a0 2024-04-23T10:17:09.088 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:08.475+0000 7f4829ffb700 1 -- 172.21.15.97:0/935146618 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7f48200202b0 con 0x7f482c1067a0 2024-04-23T10:17:09.088 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:08.477+0000 7f48331d1700 1 -- 172.21.15.97:0/935146618 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f481c039650 msgr2=0x7f481c03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:17:09.089 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:08.477+0000 7f48331d1700 1 --2- 172.21.15.97:0/935146618 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f481c039650 0x7f481c03bb10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4818006fd0 tx=0x7f4818006e40 comp rx=0 tx=0).stop 2024-04-23T10:17:09.089 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:08.477+0000 7f48331d1700 1 -- 172.21.15.97:0/935146618 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f482c1067a0 msgr2=0x7f482c19c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:17:09.089 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:08.477+0000 7f48331d1700 1 --2- 172.21.15.97:0/935146618 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f482c1067a0 0x7f482c19c420 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f48200119f0 tx=0x7f4820011ad0 comp rx=0 tx=0).stop 2024-04-23T10:17:09.089 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:08.478+0000 7f48331d1700 1 -- 172.21.15.97:0/935146618 shutdown_connections 2024-04-23T10:17:09.089 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:08.478+0000 7f48331d1700 1 --2- 172.21.15.97:0/935146618 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f481c039650 0x7f481c03bb10 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:17:09.089 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:08.478+0000 7f48331d1700 1 --2- 172.21.15.97:0/935146618 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f482c1067a0 0x7f482c19c420 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:17:09.089 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:08.478+0000 7f48331d1700 1 -- 172.21.15.97:0/935146618 >> 172.21.15.97:0/935146618 conn(0x7f482c102110 msgr2=0x7f482c102b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:17:09.089 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:08.478+0000 7f48331d1700 1 -- 172.21.15.97:0/935146618 shutdown_connections 2024-04-23T10:17:09.089 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr 2024-04-23T10:17:08.478+0000 7f48331d1700 1 -- 172.21.15.97:0/935146618 wait complete. 2024-04-23T10:17:09.089 INFO:teuthology.orchestra.run.smithi097.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-23T10:17:09.089 INFO:teuthology.orchestra.run.smithi097.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-23T10:17:09.089 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:17:09.089 INFO:teuthology.orchestra.run.smithi097.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-23T10:17:09.089 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:17:09.089 INFO:teuthology.orchestra.run.smithi097.stdout:Or, if you are only running a single cluster on this host: 2024-04-23T10:17:09.090 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:17:09.090 INFO:teuthology.orchestra.run.smithi097.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-23T10:17:09.090 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:17:09.090 INFO:teuthology.orchestra.run.smithi097.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-23T10:17:09.090 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:17:09.090 INFO:teuthology.orchestra.run.smithi097.stdout: ceph telemetry on 2024-04-23T10:17:09.090 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:17:09.090 INFO:teuthology.orchestra.run.smithi097.stdout:For more information see: 2024-04-23T10:17:09.090 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:17:09.090 INFO:teuthology.orchestra.run.smithi097.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-23T10:17:09.090 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:17:09.090 INFO:teuthology.orchestra.run.smithi097.stdout:Bootstrap complete. 2024-04-23T10:17:09.119 INFO:tasks.cephadm:Fetching config... 2024-04-23T10:17:09.119 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-23T10:17:09.119 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-23T10:17:09.133 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-23T10:17:09.133 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-23T10:17:09.133 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-23T10:17:09.187 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-23T10:17:09.188 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-23T10:17:09.188 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-23T10:17:09.251 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-23T10:17:09.252 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-23T10:17:09.252 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-23T10:17:09.306 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-23T10:17:09.306 DEBUG:teuthology.orchestra.run.smithi097:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDwt497yLqXfRoxhJS9uddb2yvoHLs0CXwVBdPVspDDjWxj657SuhLHAsRUGViBz28qtETKZMFeMP0fBm4MkMCoFgZ+2vwKweKGyA2JMA41uVOTBnQrUG2Y6Ehkxrtdpcd619+SghjUIRyF2wfbNQDnQjo9TSAPEsAZfXR1BU8VFOqj3r+CBWkcKhjcA0BqlXFlCE7+UpIpdhf/ep7bp0ADzhP2q0k15Ni97VlVBkD9T4WQRsT5uzSAAqL15ttAUMtOHsDy2Fw6p1n2Veo4n03d7q2rN2CnDCRv/uQOrHjefoW+AHd2drU3oeKpuzFut0GX0q9S4TYBf4TiJAkn81Mky8Ntb1eeamGkDZcMbEc6vhYa6XpNpWmP92uartQKcy4DktFw9d15zhiP6t2T3+eegruggSWEMwaZJ+7anviJY4nuwmuXN3yEqLGM8XShaGzYkv8JYWD/Gm1mumwf+r5ObIx39wunaWl6lh7XvCo2aF3zsKuV24h8qVmsXHPm3hk= ceph-9b22af88-0159-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-23T10:17:09.378 INFO:teuthology.orchestra.run.smithi097.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDwt497yLqXfRoxhJS9uddb2yvoHLs0CXwVBdPVspDDjWxj657SuhLHAsRUGViBz28qtETKZMFeMP0fBm4MkMCoFgZ+2vwKweKGyA2JMA41uVOTBnQrUG2Y6Ehkxrtdpcd619+SghjUIRyF2wfbNQDnQjo9TSAPEsAZfXR1BU8VFOqj3r+CBWkcKhjcA0BqlXFlCE7+UpIpdhf/ep7bp0ADzhP2q0k15Ni97VlVBkD9T4WQRsT5uzSAAqL15ttAUMtOHsDy2Fw6p1n2Veo4n03d7q2rN2CnDCRv/uQOrHjefoW+AHd2drU3oeKpuzFut0GX0q9S4TYBf4TiJAkn81Mky8Ntb1eeamGkDZcMbEc6vhYa6XpNpWmP92uartQKcy4DktFw9d15zhiP6t2T3+eegruggSWEMwaZJ+7anviJY4nuwmuXN3yEqLGM8XShaGzYkv8JYWD/Gm1mumwf+r5ObIx39wunaWl6lh7XvCo2aF3zsKuV24h8qVmsXHPm3hk= ceph-9b22af88-0159-11ef-bc93-c7b262605968 2024-04-23T10:17:09.388 DEBUG:teuthology.orchestra.run.smithi183:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDwt497yLqXfRoxhJS9uddb2yvoHLs0CXwVBdPVspDDjWxj657SuhLHAsRUGViBz28qtETKZMFeMP0fBm4MkMCoFgZ+2vwKweKGyA2JMA41uVOTBnQrUG2Y6Ehkxrtdpcd619+SghjUIRyF2wfbNQDnQjo9TSAPEsAZfXR1BU8VFOqj3r+CBWkcKhjcA0BqlXFlCE7+UpIpdhf/ep7bp0ADzhP2q0k15Ni97VlVBkD9T4WQRsT5uzSAAqL15ttAUMtOHsDy2Fw6p1n2Veo4n03d7q2rN2CnDCRv/uQOrHjefoW+AHd2drU3oeKpuzFut0GX0q9S4TYBf4TiJAkn81Mky8Ntb1eeamGkDZcMbEc6vhYa6XpNpWmP92uartQKcy4DktFw9d15zhiP6t2T3+eegruggSWEMwaZJ+7anviJY4nuwmuXN3yEqLGM8XShaGzYkv8JYWD/Gm1mumwf+r5ObIx39wunaWl6lh7XvCo2aF3zsKuV24h8qVmsXHPm3hk= ceph-9b22af88-0159-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-23T10:17:09.426 INFO:teuthology.orchestra.run.smithi183.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDwt497yLqXfRoxhJS9uddb2yvoHLs0CXwVBdPVspDDjWxj657SuhLHAsRUGViBz28qtETKZMFeMP0fBm4MkMCoFgZ+2vwKweKGyA2JMA41uVOTBnQrUG2Y6Ehkxrtdpcd619+SghjUIRyF2wfbNQDnQjo9TSAPEsAZfXR1BU8VFOqj3r+CBWkcKhjcA0BqlXFlCE7+UpIpdhf/ep7bp0ADzhP2q0k15Ni97VlVBkD9T4WQRsT5uzSAAqL15ttAUMtOHsDy2Fw6p1n2Veo4n03d7q2rN2CnDCRv/uQOrHjefoW+AHd2drU3oeKpuzFut0GX0q9S4TYBf4TiJAkn81Mky8Ntb1eeamGkDZcMbEc6vhYa6XpNpWmP92uartQKcy4DktFw9d15zhiP6t2T3+eegruggSWEMwaZJ+7anviJY4nuwmuXN3yEqLGM8XShaGzYkv8JYWD/Gm1mumwf+r5ObIx39wunaWl6lh7XvCo2aF3zsKuV24h8qVmsXHPm3hk= ceph-9b22af88-0159-11ef-bc93-c7b262605968 2024-04-23T10:17:09.438 DEBUG:teuthology.orchestra.run.smithi196:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDwt497yLqXfRoxhJS9uddb2yvoHLs0CXwVBdPVspDDjWxj657SuhLHAsRUGViBz28qtETKZMFeMP0fBm4MkMCoFgZ+2vwKweKGyA2JMA41uVOTBnQrUG2Y6Ehkxrtdpcd619+SghjUIRyF2wfbNQDnQjo9TSAPEsAZfXR1BU8VFOqj3r+CBWkcKhjcA0BqlXFlCE7+UpIpdhf/ep7bp0ADzhP2q0k15Ni97VlVBkD9T4WQRsT5uzSAAqL15ttAUMtOHsDy2Fw6p1n2Veo4n03d7q2rN2CnDCRv/uQOrHjefoW+AHd2drU3oeKpuzFut0GX0q9S4TYBf4TiJAkn81Mky8Ntb1eeamGkDZcMbEc6vhYa6XpNpWmP92uartQKcy4DktFw9d15zhiP6t2T3+eegruggSWEMwaZJ+7anviJY4nuwmuXN3yEqLGM8XShaGzYkv8JYWD/Gm1mumwf+r5ObIx39wunaWl6lh7XvCo2aF3zsKuV24h8qVmsXHPm3hk= ceph-9b22af88-0159-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-23T10:17:09.476 INFO:teuthology.orchestra.run.smithi196.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDwt497yLqXfRoxhJS9uddb2yvoHLs0CXwVBdPVspDDjWxj657SuhLHAsRUGViBz28qtETKZMFeMP0fBm4MkMCoFgZ+2vwKweKGyA2JMA41uVOTBnQrUG2Y6Ehkxrtdpcd619+SghjUIRyF2wfbNQDnQjo9TSAPEsAZfXR1BU8VFOqj3r+CBWkcKhjcA0BqlXFlCE7+UpIpdhf/ep7bp0ADzhP2q0k15Ni97VlVBkD9T4WQRsT5uzSAAqL15ttAUMtOHsDy2Fw6p1n2Veo4n03d7q2rN2CnDCRv/uQOrHjefoW+AHd2drU3oeKpuzFut0GX0q9S4TYBf4TiJAkn81Mky8Ntb1eeamGkDZcMbEc6vhYa6XpNpWmP92uartQKcy4DktFw9d15zhiP6t2T3+eegruggSWEMwaZJ+7anviJY4nuwmuXN3yEqLGM8XShaGzYkv8JYWD/Gm1mumwf+r5ObIx39wunaWl6lh7XvCo2aF3zsKuV24h8qVmsXHPm3hk= ceph-9b22af88-0159-11ef-bc93-c7b262605968 2024-04-23T10:17:09.487 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-23T10:17:09.647 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:09 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/935146618' entity='client.admin' 2024-04-23T10:17:09.722 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T10:17:11.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:11.463+0000 7f30d7999700 1 -- 172.21.15.97:0/1152780252 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30d01024c0 msgr2=0x7f30d01028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:17:11.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:11.463+0000 7f30d7999700 1 --2- 172.21.15.97:0/1152780252 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30d01024c0 0x7f30d01028e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f30cc00b0a0 tx=0x7f30cc00b3b0 comp rx=0 tx=0).stop 2024-04-23T10:17:11.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:11.463+0000 7f30d7999700 1 -- 172.21.15.97:0/1152780252 shutdown_connections 2024-04-23T10:17:11.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:11.463+0000 7f30d7999700 1 --2- 172.21.15.97:0/1152780252 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30d01024c0 0x7f30d01028e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:17:11.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:11.463+0000 7f30d7999700 1 -- 172.21.15.97:0/1152780252 >> 172.21.15.97:0/1152780252 conn(0x7f30d00fda60 msgr2=0x7f30d00ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:17:11.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:11.463+0000 7f30d7999700 1 -- 172.21.15.97:0/1152780252 shutdown_connections 2024-04-23T10:17:11.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:11.463+0000 7f30d7999700 1 -- 172.21.15.97:0/1152780252 wait complete. 2024-04-23T10:17:11.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:11.464+0000 7f30d7999700 1 Processor -- start 2024-04-23T10:17:11.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:11.464+0000 7f30d7999700 1 -- start start 2024-04-23T10:17:11.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:11.464+0000 7f30d7999700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30d01024c0 0x7f30d0076700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:17:11.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:11.464+0000 7f30d7999700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f30d0108400 con 0x7f30d01024c0 2024-04-23T10:17:11.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:11.464+0000 7f30d5735700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30d01024c0 0x7f30d0076700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:17:11.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:11.464+0000 7f30d5735700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30d01024c0 0x7f30d0076700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40940/0 (socket says 172.21.15.97:40940) 2024-04-23T10:17:11.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:11.464+0000 7f30d5735700 1 -- 172.21.15.97:0/2049783858 learned_addr learned my addr 172.21.15.97:0/2049783858 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:17:11.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:11.464+0000 7f30d5735700 1 -- 172.21.15.97:0/2049783858 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30d0076c40 con 0x7f30d01024c0 2024-04-23T10:17:11.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:11.464+0000 7f30d5735700 1 --2- 172.21.15.97:0/2049783858 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30d01024c0 0x7f30d0076700 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f30cc00d010 tx=0x7f30cc003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:17:11.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:11.465+0000 7f30c67fc700 1 -- 172.21.15.97:0/2049783858 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f30cc021020 con 0x7f30d01024c0 2024-04-23T10:17:11.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:11.465+0000 7f30c67fc700 1 -- 172.21.15.97:0/2049783858 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30cc00bdd0 con 0x7f30d01024c0 2024-04-23T10:17:11.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:11.465+0000 7f30d7999700 1 -- 172.21.15.97:0/2049783858 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30d0074e30 con 0x7f30d01024c0 2024-04-23T10:17:11.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:11.465+0000 7f30c67fc700 1 -- 172.21.15.97:0/2049783858 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f30cc017040 con 0x7f30d01024c0 2024-04-23T10:17:11.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:11.465+0000 7f30d7999700 1 -- 172.21.15.97:0/2049783858 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30d00752f0 con 0x7f30d01024c0 2024-04-23T10:17:11.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:11.465+0000 7f30c67fc700 1 -- 172.21.15.97:0/2049783858 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f30cc024020 con 0x7f30d01024c0 2024-04-23T10:17:11.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:11.466+0000 7f30d7999700 1 -- 172.21.15.97:0/2049783858 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30d0108b80 con 0x7f30d01024c0 2024-04-23T10:17:11.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:11.466+0000 7f30c67fc700 1 --2- 172.21.15.97:0/2049783858 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f30bc0396b0 0x7f30bc03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:17:11.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:11.466+0000 7f30d4f34700 1 --2- 172.21.15.97:0/2049783858 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f30bc0396b0 0x7f30bc03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:17:11.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:11.466+0000 7f30c67fc700 1 -- 172.21.15.97:0/2049783858 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f30cc00f050 con 0x7f30d01024c0 2024-04-23T10:17:11.467 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:11.466+0000 7f30d4f34700 1 --2- 172.21.15.97:0/2049783858 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f30bc0396b0 0x7f30bc03bb70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f30c0006fd0 tx=0x7f30c0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:17:11.470 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:11.471+0000 7f30c67fc700 1 -- 172.21.15.97:0/2049783858 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f30cc02b020 con 0x7f30d01024c0 2024-04-23T10:17:11.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:11.671+0000 7f30d7999700 1 -- 172.21.15.97:0/2049783858 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f30d0190e20 con 0x7f30d01024c0 2024-04-23T10:17:11.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:11.674+0000 7f30c67fc700 1 -- 172.21.15.97:0/2049783858 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f30cc0253b0 con 0x7f30d01024c0 2024-04-23T10:17:11.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:11.676+0000 7f30d7999700 1 -- 172.21.15.97:0/2049783858 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f30bc0396b0 msgr2=0x7f30bc03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:17:11.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:11.676+0000 7f30d7999700 1 --2- 172.21.15.97:0/2049783858 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f30bc0396b0 0x7f30bc03bb70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f30c0006fd0 tx=0x7f30c0006e40 comp rx=0 tx=0).stop 2024-04-23T10:17:11.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:11.676+0000 7f30d7999700 1 -- 172.21.15.97:0/2049783858 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30d01024c0 msgr2=0x7f30d0076700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:17:11.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:11.676+0000 7f30d7999700 1 --2- 172.21.15.97:0/2049783858 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30d01024c0 0x7f30d0076700 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f30cc00d010 tx=0x7f30cc003ce0 comp rx=0 tx=0).stop 2024-04-23T10:17:11.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:11.676+0000 7f30d7999700 1 -- 172.21.15.97:0/2049783858 shutdown_connections 2024-04-23T10:17:11.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:11.676+0000 7f30d7999700 1 --2- 172.21.15.97:0/2049783858 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f30bc0396b0 0x7f30bc03bb70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:17:11.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:11.676+0000 7f30d7999700 1 --2- 172.21.15.97:0/2049783858 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30d01024c0 0x7f30d0076700 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:17:11.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:11.676+0000 7f30d7999700 1 -- 172.21.15.97:0/2049783858 >> 172.21.15.97:0/2049783858 conn(0x7f30d00fda60 msgr2=0x7f30d00fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:17:11.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:11.676+0000 7f30d7999700 1 -- 172.21.15.97:0/2049783858 shutdown_connections 2024-04-23T10:17:11.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:11.676+0000 7f30d7999700 1 -- 172.21.15.97:0/2049783858 wait complete. 2024-04-23T10:17:12.524 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-23T10:17:12.549 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-23T10:17:12.783 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T10:17:12.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:12 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/2049783858' entity='client.admin' 2024-04-23T10:17:12.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:12 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:12.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:12 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:12.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:12 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi097", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:17:12.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:12 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:12.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:12 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:17:12.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:12 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:14.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:14.330+0000 7f77f8684700 1 -- 172.21.15.97:0/2454136834 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77f01013f0 msgr2=0x7f77f0101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:17:14.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:14.330+0000 7f77f8684700 1 --2- 172.21.15.97:0/2454136834 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77f01013f0 0x7f77f0101810 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f77e0009910 tx=0x7f77e0009c20 comp rx=0 tx=0).stop 2024-04-23T10:17:14.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:14.330+0000 7f77f8684700 1 -- 172.21.15.97:0/2454136834 shutdown_connections 2024-04-23T10:17:14.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:14.330+0000 7f77f8684700 1 --2- 172.21.15.97:0/2454136834 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77f01013f0 0x7f77f0101810 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:17:14.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:14.330+0000 7f77f8684700 1 -- 172.21.15.97:0/2454136834 >> 172.21.15.97:0/2454136834 conn(0x7f77f00fc950 msgr2=0x7f77f00fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:17:14.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:14.330+0000 7f77f8684700 1 -- 172.21.15.97:0/2454136834 shutdown_connections 2024-04-23T10:17:14.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:14.330+0000 7f77f8684700 1 -- 172.21.15.97:0/2454136834 wait complete. 2024-04-23T10:17:14.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:14.331+0000 7f77f8684700 1 Processor -- start 2024-04-23T10:17:14.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:14.331+0000 7f77f8684700 1 -- start start 2024-04-23T10:17:14.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:14.331+0000 7f77f8684700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77f01013f0 0x7f77f006b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:17:14.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:14.331+0000 7f77f8684700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f77f0107330 con 0x7f77f01013f0 2024-04-23T10:17:14.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:14.331+0000 7f77f6420700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77f01013f0 0x7f77f006b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:17:14.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:14.331+0000 7f77f6420700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77f01013f0 0x7f77f006b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40970/0 (socket says 172.21.15.97:40970) 2024-04-23T10:17:14.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:14.331+0000 7f77f6420700 1 -- 172.21.15.97:0/1833114026 learned_addr learned my addr 172.21.15.97:0/1833114026 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:17:14.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:14.331+0000 7f77f6420700 1 -- 172.21.15.97:0/1833114026 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77f019a830 con 0x7f77f01013f0 2024-04-23T10:17:14.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:14.331+0000 7f77f6420700 1 --2- 172.21.15.97:0/1833114026 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77f01013f0 0x7f77f006b340 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f77e0005800 tx=0x7f77e0009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:17:14.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:14.331+0000 7f77e77fe700 1 -- 172.21.15.97:0/1833114026 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77e0027070 con 0x7f77f01013f0 2024-04-23T10:17:14.333 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:14.331+0000 7f77e77fe700 1 -- 172.21.15.97:0/1833114026 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77e000be10 con 0x7f77f01013f0 2024-04-23T10:17:14.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:14.331+0000 7f77f8684700 1 -- 172.21.15.97:0/1833114026 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77f019aa00 con 0x7f77f01013f0 2024-04-23T10:17:14.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:14.331+0000 7f77e77fe700 1 -- 172.21.15.97:0/1833114026 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f77e000f460 con 0x7f77f01013f0 2024-04-23T10:17:14.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:14.331+0000 7f77f8684700 1 -- 172.21.15.97:0/1833114026 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77f019ada0 con 0x7f77f01013f0 2024-04-23T10:17:14.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:14.332+0000 7f77e77fe700 1 -- 172.21.15.97:0/1833114026 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f77e002a070 con 0x7f77f01013f0 2024-04-23T10:17:14.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:14.332+0000 7f77f8684700 1 -- 172.21.15.97:0/1833114026 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77f0107ab0 con 0x7f77f01013f0 2024-04-23T10:17:14.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:14.332+0000 7f77e77fe700 1 --2- 172.21.15.97:0/1833114026 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f77dc039650 0x7f77dc03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:17:14.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:14.333+0000 7f77e77fe700 1 -- 172.21.15.97:0/1833114026 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f77e001e030 con 0x7f77f01013f0 2024-04-23T10:17:14.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:14.333+0000 7f77f5c1f700 1 --2- 172.21.15.97:0/1833114026 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f77dc039650 0x7f77dc03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:17:14.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:14.333+0000 7f77f5c1f700 1 --2- 172.21.15.97:0/1833114026 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f77dc039650 0x7f77dc03bb10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f77ec006fd0 tx=0x7f77ec006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:17:14.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:14.337+0000 7f77e77fe700 1 -- 172.21.15.97:0/1833114026 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f77e0013070 con 0x7f77f01013f0 2024-04-23T10:17:14.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:14.542+0000 7f77f8684700 1 -- 172.21.15.97:0/1833114026 --> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f77f002e240 con 0x7f77dc039650 2024-04-23T10:17:14.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:14.546+0000 7f77e77fe700 1 -- 172.21.15.97:0/1833114026 <== mgr.14152 v2:172.21.15.97:6800/3149043749 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f77f002e240 con 0x7f77dc039650 2024-04-23T10:17:14.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:14.548+0000 7f77f8684700 1 -- 172.21.15.97:0/1833114026 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f77dc039650 msgr2=0x7f77dc03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:17:14.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:14.548+0000 7f77f8684700 1 --2- 172.21.15.97:0/1833114026 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f77dc039650 0x7f77dc03bb10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f77ec006fd0 tx=0x7f77ec006e40 comp rx=0 tx=0).stop 2024-04-23T10:17:14.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:14.548+0000 7f77f8684700 1 -- 172.21.15.97:0/1833114026 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77f01013f0 msgr2=0x7f77f006b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:17:14.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:14.548+0000 7f77f8684700 1 --2- 172.21.15.97:0/1833114026 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77f01013f0 0x7f77f006b340 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f77e0005800 tx=0x7f77e0009f60 comp rx=0 tx=0).stop 2024-04-23T10:17:14.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:14.548+0000 7f77f8684700 1 -- 172.21.15.97:0/1833114026 shutdown_connections 2024-04-23T10:17:14.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:14.548+0000 7f77f8684700 1 --2- 172.21.15.97:0/1833114026 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f77dc039650 0x7f77dc03bb10 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:17:14.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:14.548+0000 7f77f8684700 1 --2- 172.21.15.97:0/1833114026 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f77f01013f0 0x7f77f006b340 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:17:14.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:14.548+0000 7f77f8684700 1 -- 172.21.15.97:0/1833114026 >> 172.21.15.97:0/1833114026 conn(0x7f77f00fc950 msgr2=0x7f77f00fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:17:14.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:14.548+0000 7f77f8684700 1 -- 172.21.15.97:0/1833114026 shutdown_connections 2024-04-23T10:17:14.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:14.548+0000 7f77f8684700 1 -- 172.21.15.97:0/1833114026 wait complete. 2024-04-23T10:17:15.492 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi183 2024-04-23T10:17:15.492 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-23T10:17:15.492 DEBUG:teuthology.orchestra.run.smithi183:> dd of=/etc/ceph/ceph.conf 2024-04-23T10:17:15.511 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-23T10:17:15.511 DEBUG:teuthology.orchestra.run.smithi183:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-23T10:17:15.569 INFO:tasks.cephadm:Adding host smithi183 to orchestrator... 2024-04-23T10:17:15.569 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph orch host add smithi183 2024-04-23T10:17:15.800 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T10:17:15.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:15 smithi097 ceph-mon[29197]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:17:15.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:15 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:15.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:15 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:17:15.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:15 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:17:15.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:15 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:17:15.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:15 smithi097 ceph-mon[29197]: Updating smithi097:/etc/ceph/ceph.conf 2024-04-23T10:17:15.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:15 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:15.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:15 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:15.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:15 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:16.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:16 smithi097 ceph-mon[29197]: Updating smithi097:/var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T10:17:16.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:16 smithi097 ceph-mon[29197]: Updating smithi097:/etc/ceph/ceph.client.admin.keyring 2024-04-23T10:17:16.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:16 smithi097 ceph-mon[29197]: Updating smithi097:/var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T10:17:17.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:17.410+0000 7f043bfa6700 1 -- 172.21.15.97:0/1051803472 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0434103650 msgr2=0x7f0434103a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:17:17.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:17.410+0000 7f043bfa6700 1 --2- 172.21.15.97:0/1051803472 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0434103650 0x7f0434103a30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0424009910 tx=0x7f0424009c20 comp rx=0 tx=0).stop 2024-04-23T10:17:17.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:17.410+0000 7f043bfa6700 1 -- 172.21.15.97:0/1051803472 shutdown_connections 2024-04-23T10:17:17.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:17.410+0000 7f043bfa6700 1 --2- 172.21.15.97:0/1051803472 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0434103650 0x7f0434103a30 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:17:17.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:17.410+0000 7f043bfa6700 1 -- 172.21.15.97:0/1051803472 >> 172.21.15.97:0/1051803472 conn(0x7f04340fce40 msgr2=0x7f04340ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:17:17.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:17.410+0000 7f043bfa6700 1 -- 172.21.15.97:0/1051803472 shutdown_connections 2024-04-23T10:17:17.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:17.410+0000 7f043bfa6700 1 -- 172.21.15.97:0/1051803472 wait complete. 2024-04-23T10:17:17.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:17.411+0000 7f043bfa6700 1 Processor -- start 2024-04-23T10:17:17.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:17.411+0000 7f043bfa6700 1 -- start start 2024-04-23T10:17:17.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:17.411+0000 7f043bfa6700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0434103650 0x7f0434197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:17:17.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:17.411+0000 7f043bfa6700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f043410c980 con 0x7f0434103650 2024-04-23T10:17:17.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:17.411+0000 7f0439d42700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0434103650 0x7f0434197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:17:17.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:17.411+0000 7f0439d42700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0434103650 0x7f0434197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48556/0 (socket says 172.21.15.97:48556) 2024-04-23T10:17:17.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:17.411+0000 7f0439d42700 1 -- 172.21.15.97:0/1344881803 learned_addr learned my addr 172.21.15.97:0/1344881803 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:17:17.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:17.411+0000 7f0439d42700 1 -- 172.21.15.97:0/1344881803 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0434198430 con 0x7f0434103650 2024-04-23T10:17:17.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:17.411+0000 7f0439d42700 1 --2- 172.21.15.97:0/1344881803 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0434103650 0x7f0434197ef0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f04240056d0 tx=0x7f04240057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:17:17.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:17.412+0000 7f042affd700 1 -- 172.21.15.97:0/1344881803 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0424026070 con 0x7f0434103650 2024-04-23T10:17:17.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:17.412+0000 7f042affd700 1 -- 172.21.15.97:0/1344881803 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0424004370 con 0x7f0434103650 2024-04-23T10:17:17.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:17.412+0000 7f043bfa6700 1 -- 172.21.15.97:0/1344881803 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04341986c0 con 0x7f0434103650 2024-04-23T10:17:17.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:17.412+0000 7f042affd700 1 -- 172.21.15.97:0/1344881803 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04240039b0 con 0x7f0434103650 2024-04-23T10:17:17.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:17.412+0000 7f043bfa6700 1 -- 172.21.15.97:0/1344881803 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0434198b20 con 0x7f0434103650 2024-04-23T10:17:17.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:17.413+0000 7f042affd700 1 -- 172.21.15.97:0/1344881803 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f0424029070 con 0x7f0434103650 2024-04-23T10:17:17.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:17.413+0000 7f043bfa6700 1 -- 172.21.15.97:0/1344881803 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f043406af70 con 0x7f0434103650 2024-04-23T10:17:17.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:17.413+0000 7f042affd700 1 --2- 172.21.15.97:0/1344881803 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f04200396b0 0x7f042003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:17:17.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:17.413+0000 7f042affd700 1 -- 172.21.15.97:0/1344881803 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f042401e070 con 0x7f0434103650 2024-04-23T10:17:17.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:17.413+0000 7f0439541700 1 --2- 172.21.15.97:0/1344881803 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f04200396b0 0x7f042003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:17:17.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:17.413+0000 7f0439541700 1 --2- 172.21.15.97:0/1344881803 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f04200396b0 0x7f042003bb70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0430006fd0 tx=0x7f0430006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:17:17.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:17.418+0000 7f042affd700 1 -- 172.21.15.97:0/1344881803 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0424013070 con 0x7f0434103650 2024-04-23T10:17:17.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:17.620+0000 7f043bfa6700 1 -- 172.21.15.97:0/1344881803 --> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi183", "target": ["mon-mgr", ""]}) -- 0x7f04341992c0 con 0x7f04200396b0 2024-04-23T10:17:19.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:18 smithi097 ceph-mon[29197]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi183", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:17:19.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:18 smithi097 ceph-mon[29197]: Deploying cephadm binary to smithi183 2024-04-23T10:17:20.825 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:20 smithi097 ceph-mon[29197]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:17:21.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:21.696+0000 7f042affd700 1 -- 172.21.15.97:0/1344881803 <== mgr.14152 v2:172.21.15.97:6800/3149043749 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f04341992c0 con 0x7f04200396b0 2024-04-23T10:17:21.696 INFO:teuthology.orchestra.run.smithi097.stdout:Added host 'smithi183' with addr '172.21.15.183' 2024-04-23T10:17:21.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:21.699+0000 7f043bfa6700 1 -- 172.21.15.97:0/1344881803 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f04200396b0 msgr2=0x7f042003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:17:21.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:21.699+0000 7f043bfa6700 1 --2- 172.21.15.97:0/1344881803 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f04200396b0 0x7f042003bb70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0430006fd0 tx=0x7f0430006e40 comp rx=0 tx=0).stop 2024-04-23T10:17:21.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:21.699+0000 7f043bfa6700 1 -- 172.21.15.97:0/1344881803 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0434103650 msgr2=0x7f0434197ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:17:21.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:21.699+0000 7f043bfa6700 1 --2- 172.21.15.97:0/1344881803 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0434103650 0x7f0434197ef0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f04240056d0 tx=0x7f04240057b0 comp rx=0 tx=0).stop 2024-04-23T10:17:21.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:21.699+0000 7f043bfa6700 1 -- 172.21.15.97:0/1344881803 shutdown_connections 2024-04-23T10:17:21.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:21.699+0000 7f043bfa6700 1 --2- 172.21.15.97:0/1344881803 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f04200396b0 0x7f042003bb70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:17:21.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:21.699+0000 7f043bfa6700 1 --2- 172.21.15.97:0/1344881803 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0434103650 0x7f0434197ef0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:17:21.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:21.699+0000 7f043bfa6700 1 -- 172.21.15.97:0/1344881803 >> 172.21.15.97:0/1344881803 conn(0x7f04340fce40 msgr2=0x7f0434104400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:17:21.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:21.699+0000 7f043bfa6700 1 -- 172.21.15.97:0/1344881803 shutdown_connections 2024-04-23T10:17:21.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:21.699+0000 7f043bfa6700 1 -- 172.21.15.97:0/1344881803 wait complete. 2024-04-23T10:17:22.338 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-23T10:17:22.570 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T10:17:22.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:22 smithi097 ceph-mon[29197]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:17:22.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:22 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:22.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:22 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:17:22.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:22 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:23.657 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:23 smithi097 ceph-mon[29197]: Added host smithi183 2024-04-23T10:17:23.658 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:23 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:24.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:24.127+0000 7f31c7f24700 1 -- 172.21.15.97:0/4091401769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31c01015b0 msgr2=0x7f31c0101990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:17:24.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:24.127+0000 7f31c7f24700 1 --2- 172.21.15.97:0/4091401769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31c01015b0 0x7f31c0101990 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f31bc009910 tx=0x7f31bc009c20 comp rx=0 tx=0).stop 2024-04-23T10:17:24.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:24.127+0000 7f31c7f24700 1 -- 172.21.15.97:0/4091401769 shutdown_connections 2024-04-23T10:17:24.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:24.127+0000 7f31c7f24700 1 --2- 172.21.15.97:0/4091401769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31c01015b0 0x7f31c0101990 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:17:24.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:24.127+0000 7f31c7f24700 1 -- 172.21.15.97:0/4091401769 >> 172.21.15.97:0/4091401769 conn(0x7f31c00fce60 msgr2=0x7f31c00ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:17:24.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:24.127+0000 7f31c7f24700 1 -- 172.21.15.97:0/4091401769 shutdown_connections 2024-04-23T10:17:24.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:24.127+0000 7f31c7f24700 1 -- 172.21.15.97:0/4091401769 wait complete. 2024-04-23T10:17:24.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:24.128+0000 7f31c7f24700 1 Processor -- start 2024-04-23T10:17:24.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:24.128+0000 7f31c7f24700 1 -- start start 2024-04-23T10:17:24.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:24.128+0000 7f31c7f24700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31c01015b0 0x7f31c0196d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:17:24.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:24.128+0000 7f31c7f24700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f31c010b830 con 0x7f31c01015b0 2024-04-23T10:17:24.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:24.128+0000 7f31c5cc0700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31c01015b0 0x7f31c0196d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:17:24.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:24.128+0000 7f31c5cc0700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31c01015b0 0x7f31c0196d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48572/0 (socket says 172.21.15.97:48572) 2024-04-23T10:17:24.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:24.128+0000 7f31c5cc0700 1 -- 172.21.15.97:0/3490772670 learned_addr learned my addr 172.21.15.97:0/3490772670 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:17:24.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:24.128+0000 7f31c5cc0700 1 -- 172.21.15.97:0/3490772670 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31c01972d0 con 0x7f31c01015b0 2024-04-23T10:17:24.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:24.129+0000 7f31c5cc0700 1 --2- 172.21.15.97:0/3490772670 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31c01015b0 0x7f31c0196d90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f31bc0056d0 tx=0x7f31bc0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:17:24.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:24.129+0000 7f31b6ffd700 1 -- 172.21.15.97:0/3490772670 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f31bc026070 con 0x7f31c01015b0 2024-04-23T10:17:24.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:24.129+0000 7f31b6ffd700 1 -- 172.21.15.97:0/3490772670 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31bc00bba0 con 0x7f31c01015b0 2024-04-23T10:17:24.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:24.129+0000 7f31c7f24700 1 -- 172.21.15.97:0/3490772670 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31c01975c0 con 0x7f31c01015b0 2024-04-23T10:17:24.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:24.129+0000 7f31b6ffd700 1 -- 172.21.15.97:0/3490772670 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f31bc003920 con 0x7f31c01015b0 2024-04-23T10:17:24.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:24.129+0000 7f31c7f24700 1 -- 172.21.15.97:0/3490772670 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31c019b660 con 0x7f31c01015b0 2024-04-23T10:17:24.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:24.130+0000 7f31b6ffd700 1 -- 172.21.15.97:0/3490772670 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f31bc029070 con 0x7f31c01015b0 2024-04-23T10:17:24.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:24.130+0000 7f31c7f24700 1 -- 172.21.15.97:0/3490772670 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31c0102a90 con 0x7f31c01015b0 2024-04-23T10:17:24.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:24.130+0000 7f31b6ffd700 1 --2- 172.21.15.97:0/3490772670 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f31ac039650 0x7f31ac03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:17:24.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:24.130+0000 7f31b6ffd700 1 -- 172.21.15.97:0/3490772670 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f31bc01e070 con 0x7f31c01015b0 2024-04-23T10:17:24.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:24.130+0000 7f31c54bf700 1 --2- 172.21.15.97:0/3490772670 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f31ac039650 0x7f31ac03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:17:24.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:24.130+0000 7f31c54bf700 1 --2- 172.21.15.97:0/3490772670 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f31ac039650 0x7f31ac03bb10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f31b0006fd0 tx=0x7f31b0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:17:24.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:24.135+0000 7f31b6ffd700 1 -- 172.21.15.97:0/3490772670 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f31bc013070 con 0x7f31c01015b0 2024-04-23T10:17:24.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:24.338+0000 7f31c7f24700 1 -- 172.21.15.97:0/3490772670 --> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f31c01980f0 con 0x7f31ac039650 2024-04-23T10:17:24.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:24.340+0000 7f31b6ffd700 1 -- 172.21.15.97:0/3490772670 <== mgr.14152 v2:172.21.15.97:6800/3149043749 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7f31c01980f0 con 0x7f31ac039650 2024-04-23T10:17:24.339 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:17:24.339 INFO:teuthology.orchestra.run.smithi097.stdout:[{"addr": "172.21.15.97", "hostname": "smithi097", "labels": [], "status": ""}, {"addr": "172.21.15.183", "hostname": "smithi183", "labels": [], "status": ""}] 2024-04-23T10:17:24.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:24.342+0000 7f31c7f24700 1 -- 172.21.15.97:0/3490772670 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f31ac039650 msgr2=0x7f31ac03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:17:24.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:24.342+0000 7f31c7f24700 1 --2- 172.21.15.97:0/3490772670 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f31ac039650 0x7f31ac03bb10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f31b0006fd0 tx=0x7f31b0006e40 comp rx=0 tx=0).stop 2024-04-23T10:17:24.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:24.342+0000 7f31c7f24700 1 -- 172.21.15.97:0/3490772670 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31c01015b0 msgr2=0x7f31c0196d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:17:24.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:24.342+0000 7f31c7f24700 1 --2- 172.21.15.97:0/3490772670 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31c01015b0 0x7f31c0196d90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f31bc0056d0 tx=0x7f31bc0057b0 comp rx=0 tx=0).stop 2024-04-23T10:17:24.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:24.342+0000 7f31c7f24700 1 -- 172.21.15.97:0/3490772670 shutdown_connections 2024-04-23T10:17:24.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:24.342+0000 7f31c7f24700 1 --2- 172.21.15.97:0/3490772670 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f31ac039650 0x7f31ac03bb10 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:17:24.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:24.342+0000 7f31c7f24700 1 --2- 172.21.15.97:0/3490772670 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31c01015b0 0x7f31c0196d90 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:17:24.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:24.342+0000 7f31c7f24700 1 -- 172.21.15.97:0/3490772670 >> 172.21.15.97:0/3490772670 conn(0x7f31c00fce60 msgr2=0x7f31c0102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:17:24.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:24.342+0000 7f31c7f24700 1 -- 172.21.15.97:0/3490772670 shutdown_connections 2024-04-23T10:17:24.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:24.342+0000 7f31c7f24700 1 -- 172.21.15.97:0/3490772670 wait complete. 2024-04-23T10:17:24.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:24 smithi097 ceph-mon[29197]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:17:24.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:24 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:25.002 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi196 2024-04-23T10:17:25.002 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-23T10:17:25.002 DEBUG:teuthology.orchestra.run.smithi196:> dd of=/etc/ceph/ceph.conf 2024-04-23T10:17:25.021 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-23T10:17:25.021 DEBUG:teuthology.orchestra.run.smithi196:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-23T10:17:25.078 INFO:tasks.cephadm:Adding host smithi196 to orchestrator... 2024-04-23T10:17:25.078 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph orch host add smithi196 2024-04-23T10:17:25.311 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T10:17:25.775 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:25 smithi097 ceph-mon[29197]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T10:17:26.710 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:26 smithi097 ceph-mon[29197]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:17:26.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:26.932+0000 7fd9cd027700 1 -- 172.21.15.97:0/2690846512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9c81084e0 msgr2=0x7fd9c81088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:17:26.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:26.932+0000 7fd9cd027700 1 --2- 172.21.15.97:0/2690846512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9c81084e0 0x7fd9c81088c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd9c000b0a0 tx=0x7fd9c000b3b0 comp rx=0 tx=0).stop 2024-04-23T10:17:26.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:26.932+0000 7fd9cd027700 1 -- 172.21.15.97:0/2690846512 shutdown_connections 2024-04-23T10:17:26.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:26.932+0000 7fd9cd027700 1 --2- 172.21.15.97:0/2690846512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9c81084e0 0x7fd9c81088c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:17:26.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:26.932+0000 7fd9cd027700 1 -- 172.21.15.97:0/2690846512 >> 172.21.15.97:0/2690846512 conn(0x7fd9c80fdf80 msgr2=0x7fd9c81003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:17:26.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:26.932+0000 7fd9cd027700 1 -- 172.21.15.97:0/2690846512 shutdown_connections 2024-04-23T10:17:26.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:26.932+0000 7fd9cd027700 1 -- 172.21.15.97:0/2690846512 wait complete. 2024-04-23T10:17:26.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:26.933+0000 7fd9cd027700 1 Processor -- start 2024-04-23T10:17:26.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:26.933+0000 7fd9cd027700 1 -- start start 2024-04-23T10:17:26.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:26.933+0000 7fd9cd027700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9c81084e0 0x7fd9c8074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:17:26.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:26.933+0000 7fd9cd027700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c810c940 con 0x7fd9c81084e0 2024-04-23T10:17:26.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:26.933+0000 7fd9c6d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9c81084e0 0x7fd9c8074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:17:26.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:26.933+0000 7fd9c6d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9c81084e0 0x7fd9c8074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44614/0 (socket says 172.21.15.97:44614) 2024-04-23T10:17:26.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:26.933+0000 7fd9c6d9d700 1 -- 172.21.15.97:0/2980501343 learned_addr learned my addr 172.21.15.97:0/2980501343 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:17:26.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:26.933+0000 7fd9c6d9d700 1 -- 172.21.15.97:0/2980501343 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9c8076c30 con 0x7fd9c81084e0 2024-04-23T10:17:26.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:26.934+0000 7fd9c6d9d700 1 --2- 172.21.15.97:0/2980501343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9c81084e0 0x7fd9c8074d40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd9c0006130 tx=0x7fd9c00097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:17:26.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:26.934+0000 7fd9affff700 1 -- 172.21.15.97:0/2980501343 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd9c0021020 con 0x7fd9c81084e0 2024-04-23T10:17:26.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:26.934+0000 7fd9cd027700 1 -- 172.21.15.97:0/2980501343 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9c80752e0 con 0x7fd9c81084e0 2024-04-23T10:17:26.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:26.934+0000 7fd9affff700 1 -- 172.21.15.97:0/2980501343 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9c00079b0 con 0x7fd9c81084e0 2024-04-23T10:17:26.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:26.934+0000 7fd9affff700 1 -- 172.21.15.97:0/2980501343 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd9c0017040 con 0x7fd9c81084e0 2024-04-23T10:17:26.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:26.934+0000 7fd9cd027700 1 -- 172.21.15.97:0/2980501343 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9c80757a0 con 0x7fd9c81084e0 2024-04-23T10:17:26.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:26.935+0000 7fd9affff700 1 -- 172.21.15.97:0/2980501343 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fd9c0024020 con 0x7fd9c81084e0 2024-04-23T10:17:26.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:26.935+0000 7fd9cd027700 1 -- 172.21.15.97:0/2980501343 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9c8103b90 con 0x7fd9c81084e0 2024-04-23T10:17:26.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:26.935+0000 7fd9affff700 1 --2- 172.21.15.97:0/2980501343 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fd9b00396b0 0x7fd9b003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:17:26.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:26.935+0000 7fd9affff700 1 -- 172.21.15.97:0/2980501343 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd9c000f050 con 0x7fd9c81084e0 2024-04-23T10:17:26.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:26.935+0000 7fd9c659c700 1 --2- 172.21.15.97:0/2980501343 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fd9b00396b0 0x7fd9b003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:17:26.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:26.935+0000 7fd9c659c700 1 --2- 172.21.15.97:0/2980501343 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fd9b00396b0 0x7fd9b003bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc006fd0 tx=0x7fd9bc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:17:26.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:26.940+0000 7fd9affff700 1 -- 172.21.15.97:0/2980501343 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd9c002b020 con 0x7fd9c81084e0 2024-04-23T10:17:27.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:27.147+0000 7fd9cd027700 1 -- 172.21.15.97:0/2980501343 --> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi196", "target": ["mon-mgr", ""]}) -- 0x7fd9c8076270 con 0x7fd9b00396b0 2024-04-23T10:17:28.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:28 smithi097 ceph-mon[29197]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi196", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:17:28.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:28 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:28.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:28 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:28.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:28 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:28.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:28 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:28.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:28 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi183", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:17:28.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:28 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:17:28.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:28 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:17:28.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:28 smithi097 ceph-mon[29197]: Updating smithi183:/etc/ceph/ceph.conf 2024-04-23T10:17:28.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:28 smithi097 ceph-mon[29197]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:17:28.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:28 smithi097 ceph-mon[29197]: Updating smithi183:/var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T10:17:29.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:29 smithi097 ceph-mon[29197]: Deploying cephadm binary to smithi196 2024-04-23T10:17:29.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:29 smithi097 ceph-mon[29197]: Updating smithi183:/etc/ceph/ceph.client.admin.keyring 2024-04-23T10:17:29.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:29 smithi097 ceph-mon[29197]: Updating smithi183:/var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T10:17:29.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:29 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:29.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:29 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:29.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:29 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:30.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:30 smithi097 ceph-mon[29197]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:17:31.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:31.294+0000 7fd9affff700 1 -- 172.21.15.97:0/2980501343 <== mgr.14152 v2:172.21.15.97:6800/3149043749 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fd9c8076270 con 0x7fd9b00396b0 2024-04-23T10:17:31.294 INFO:teuthology.orchestra.run.smithi097.stdout:Added host 'smithi196' with addr '172.21.15.196' 2024-04-23T10:17:31.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:31.297+0000 7fd9cd027700 1 -- 172.21.15.97:0/2980501343 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fd9b00396b0 msgr2=0x7fd9b003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:17:31.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:31.297+0000 7fd9cd027700 1 --2- 172.21.15.97:0/2980501343 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fd9b00396b0 0x7fd9b003bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc006fd0 tx=0x7fd9bc006e40 comp rx=0 tx=0).stop 2024-04-23T10:17:31.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:31.297+0000 7fd9cd027700 1 -- 172.21.15.97:0/2980501343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9c81084e0 msgr2=0x7fd9c8074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:17:31.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:31.297+0000 7fd9cd027700 1 --2- 172.21.15.97:0/2980501343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9c81084e0 0x7fd9c8074d40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd9c0006130 tx=0x7fd9c00097d0 comp rx=0 tx=0).stop 2024-04-23T10:17:31.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:31.297+0000 7fd9cd027700 1 -- 172.21.15.97:0/2980501343 shutdown_connections 2024-04-23T10:17:31.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:31.297+0000 7fd9cd027700 1 --2- 172.21.15.97:0/2980501343 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fd9b00396b0 0x7fd9b003bb70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:17:31.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:31.297+0000 7fd9cd027700 1 --2- 172.21.15.97:0/2980501343 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9c81084e0 0x7fd9c8074d40 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:17:31.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:31.297+0000 7fd9cd027700 1 -- 172.21.15.97:0/2980501343 >> 172.21.15.97:0/2980501343 conn(0x7fd9c80fdf80 msgr2=0x7fd9c8109fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:17:31.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:31.297+0000 7fd9cd027700 1 -- 172.21.15.97:0/2980501343 shutdown_connections 2024-04-23T10:17:31.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:31.297+0000 7fd9cd027700 1 -- 172.21.15.97:0/2980501343 wait complete. 2024-04-23T10:17:31.917 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-23T10:17:32.153 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T10:17:32.579 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:32 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:32.579 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:32 smithi097 ceph-mon[29197]: Added host smithi196 2024-04-23T10:17:32.579 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:32 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:17:32.579 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:32 smithi097 ceph-mon[29197]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:17:32.579 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:32 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:33.474 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:33 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:33.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:33.669+0000 7fcaa6c1f700 1 -- 172.21.15.97:0/3283328590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcaa01015b0 msgr2=0x7fcaa0101990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:17:33.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:33.669+0000 7fcaa6c1f700 1 --2- 172.21.15.97:0/3283328590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcaa01015b0 0x7fcaa0101990 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fca94009910 tx=0x7fca94009c20 comp rx=0 tx=0).stop 2024-04-23T10:17:33.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:33.669+0000 7fcaa6c1f700 1 -- 172.21.15.97:0/3283328590 shutdown_connections 2024-04-23T10:17:33.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:33.669+0000 7fcaa6c1f700 1 --2- 172.21.15.97:0/3283328590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcaa01015b0 0x7fcaa0101990 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:17:33.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:33.669+0000 7fcaa6c1f700 1 -- 172.21.15.97:0/3283328590 >> 172.21.15.97:0/3283328590 conn(0x7fcaa00fce60 msgr2=0x7fcaa00ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:17:33.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:33.669+0000 7fcaa6c1f700 1 -- 172.21.15.97:0/3283328590 shutdown_connections 2024-04-23T10:17:33.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:33.669+0000 7fcaa6c1f700 1 -- 172.21.15.97:0/3283328590 wait complete. 2024-04-23T10:17:33.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:33.670+0000 7fcaa6c1f700 1 Processor -- start 2024-04-23T10:17:33.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:33.670+0000 7fcaa6c1f700 1 -- start start 2024-04-23T10:17:33.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:33.670+0000 7fcaa6c1f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcaa01015b0 0x7fcaa0198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:17:33.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:33.670+0000 7fcaa6c1f700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa010b830 con 0x7fcaa01015b0 2024-04-23T10:17:33.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:33.670+0000 7fcaa49bb700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcaa01015b0 0x7fcaa0198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:17:33.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:33.670+0000 7fcaa49bb700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcaa01015b0 0x7fcaa0198cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44638/0 (socket says 172.21.15.97:44638) 2024-04-23T10:17:33.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:33.670+0000 7fcaa49bb700 1 -- 172.21.15.97:0/3610611843 learned_addr learned my addr 172.21.15.97:0/3610611843 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:17:33.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:33.670+0000 7fcaa49bb700 1 -- 172.21.15.97:0/3610611843 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaa0199230 con 0x7fcaa01015b0 2024-04-23T10:17:33.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:33.670+0000 7fcaa49bb700 1 --2- 172.21.15.97:0/3610611843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcaa01015b0 0x7fcaa0198cf0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fca940056d0 tx=0x7fca940057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:17:33.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:33.671+0000 7fca9dffb700 1 -- 172.21.15.97:0/3610611843 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fca94026070 con 0x7fcaa01015b0 2024-04-23T10:17:33.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:33.671+0000 7fca9dffb700 1 -- 172.21.15.97:0/3610611843 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca9400bba0 con 0x7fcaa01015b0 2024-04-23T10:17:33.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:33.671+0000 7fcaa6c1f700 1 -- 172.21.15.97:0/3610611843 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaa0199520 con 0x7fcaa01015b0 2024-04-23T10:17:33.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:33.671+0000 7fca9dffb700 1 -- 172.21.15.97:0/3610611843 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fca94003920 con 0x7fcaa01015b0 2024-04-23T10:17:33.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:33.671+0000 7fcaa6c1f700 1 -- 172.21.15.97:0/3610611843 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaa01999e0 con 0x7fcaa01015b0 2024-04-23T10:17:33.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:33.672+0000 7fca9dffb700 1 -- 172.21.15.97:0/3610611843 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fca94029070 con 0x7fcaa01015b0 2024-04-23T10:17:33.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:33.672+0000 7fcaa6c1f700 1 -- 172.21.15.97:0/3610611843 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcaa0102a90 con 0x7fcaa01015b0 2024-04-23T10:17:33.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:33.672+0000 7fca9dffb700 1 --2- 172.21.15.97:0/3610611843 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fca900396b0 0x7fca9003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:17:33.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:33.672+0000 7fca9dffb700 1 -- 172.21.15.97:0/3610611843 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fca9401e070 con 0x7fcaa01015b0 2024-04-23T10:17:33.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:33.672+0000 7fca9ffff700 1 --2- 172.21.15.97:0/3610611843 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fca900396b0 0x7fca9003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:17:33.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:33.672+0000 7fca9ffff700 1 --2- 172.21.15.97:0/3610611843 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fca900396b0 0x7fca9003bb70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fca8c006fd0 tx=0x7fca8c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:17:33.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:33.677+0000 7fca9dffb700 1 -- 172.21.15.97:0/3610611843 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fca94013070 con 0x7fcaa01015b0 2024-04-23T10:17:33.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:33.883+0000 7fcaa6c1f700 1 -- 172.21.15.97:0/3610611843 --> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcaa006aee0 con 0x7fca900396b0 2024-04-23T10:17:33.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:33.884+0000 7fca9dffb700 1 -- 172.21.15.97:0/3610611843 <== mgr.14152 v2:172.21.15.97:6800/3149043749 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7fcaa006aee0 con 0x7fca900396b0 2024-04-23T10:17:33.884 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:17:33.884 INFO:teuthology.orchestra.run.smithi097.stdout:[{"addr": "172.21.15.97", "hostname": "smithi097", "labels": [], "status": ""}, {"addr": "172.21.15.183", "hostname": "smithi183", "labels": [], "status": ""}, {"addr": "172.21.15.196", "hostname": "smithi196", "labels": [], "status": ""}] 2024-04-23T10:17:33.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:33.886+0000 7fcaa6c1f700 1 -- 172.21.15.97:0/3610611843 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fca900396b0 msgr2=0x7fca9003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:17:33.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:33.886+0000 7fcaa6c1f700 1 --2- 172.21.15.97:0/3610611843 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fca900396b0 0x7fca9003bb70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fca8c006fd0 tx=0x7fca8c006e40 comp rx=0 tx=0).stop 2024-04-23T10:17:33.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:33.886+0000 7fcaa6c1f700 1 -- 172.21.15.97:0/3610611843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcaa01015b0 msgr2=0x7fcaa0198cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:17:33.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:33.886+0000 7fcaa6c1f700 1 --2- 172.21.15.97:0/3610611843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcaa01015b0 0x7fcaa0198cf0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fca940056d0 tx=0x7fca940057b0 comp rx=0 tx=0).stop 2024-04-23T10:17:33.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:33.887+0000 7fcaa6c1f700 1 -- 172.21.15.97:0/3610611843 shutdown_connections 2024-04-23T10:17:33.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:33.887+0000 7fcaa6c1f700 1 --2- 172.21.15.97:0/3610611843 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fca900396b0 0x7fca9003bb70 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:17:33.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:33.887+0000 7fcaa6c1f700 1 --2- 172.21.15.97:0/3610611843 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcaa01015b0 0x7fcaa0198cf0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:17:33.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:33.887+0000 7fcaa6c1f700 1 -- 172.21.15.97:0/3610611843 >> 172.21.15.97:0/3610611843 conn(0x7fcaa00fce60 msgr2=0x7fcaa0102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:17:33.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:33.887+0000 7fcaa6c1f700 1 -- 172.21.15.97:0/3610611843 shutdown_connections 2024-04-23T10:17:33.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:33.887+0000 7fcaa6c1f700 1 -- 172.21.15.97:0/3610611843 wait complete. 2024-04-23T10:17:34.463 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:34 smithi097 ceph-mon[29197]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:17:34.463 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:34 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:34.490 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-23T10:17:34.490 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-23T10:17:34.720 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T10:17:35.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:35 smithi097 ceph-mon[29197]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T10:17:36.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:36.276+0000 7f38eaa44700 1 -- 172.21.15.97:0/1801086775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38e41015b0 msgr2=0x7f38e4101990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:17:36.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:36.276+0000 7f38eaa44700 1 --2- 172.21.15.97:0/1801086775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38e41015b0 0x7f38e4101990 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f38d4009910 tx=0x7f38d4009c20 comp rx=0 tx=0).stop 2024-04-23T10:17:36.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:36.276+0000 7f38eaa44700 1 -- 172.21.15.97:0/1801086775 shutdown_connections 2024-04-23T10:17:36.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:36.276+0000 7f38eaa44700 1 --2- 172.21.15.97:0/1801086775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38e41015b0 0x7f38e4101990 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:17:36.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:36.276+0000 7f38eaa44700 1 -- 172.21.15.97:0/1801086775 >> 172.21.15.97:0/1801086775 conn(0x7f38e40fce20 msgr2=0x7f38e40ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:17:36.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:36.276+0000 7f38eaa44700 1 -- 172.21.15.97:0/1801086775 shutdown_connections 2024-04-23T10:17:36.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:36.276+0000 7f38eaa44700 1 -- 172.21.15.97:0/1801086775 wait complete. 2024-04-23T10:17:36.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:36.276+0000 7f38eaa44700 1 Processor -- start 2024-04-23T10:17:36.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:36.276+0000 7f38eaa44700 1 -- start start 2024-04-23T10:17:36.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:36.277+0000 7f38eaa44700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38e41015b0 0x7f38e4197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:17:36.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:36.277+0000 7f38eaa44700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f38e406b300 con 0x7f38e41015b0 2024-04-23T10:17:36.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:36.277+0000 7f38e3fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38e41015b0 0x7f38e4197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:17:36.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:36.277+0000 7f38e3fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38e41015b0 0x7f38e4197ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50494/0 (socket says 172.21.15.97:50494) 2024-04-23T10:17:36.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:36.277+0000 7f38e3fff700 1 -- 172.21.15.97:0/3800077997 learned_addr learned my addr 172.21.15.97:0/3800077997 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:17:36.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:36.277+0000 7f38e3fff700 1 -- 172.21.15.97:0/3800077997 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38e4198420 con 0x7f38e41015b0 2024-04-23T10:17:36.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:36.277+0000 7f38e3fff700 1 --2- 172.21.15.97:0/3800077997 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38e41015b0 0x7f38e4197ee0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f38d40056d0 tx=0x7f38d40057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:17:36.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:36.277+0000 7f38e17fa700 1 -- 172.21.15.97:0/3800077997 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38d4026070 con 0x7f38e41015b0 2024-04-23T10:17:36.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:36.277+0000 7f38eaa44700 1 -- 172.21.15.97:0/3800077997 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38e4198710 con 0x7f38e41015b0 2024-04-23T10:17:36.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:36.277+0000 7f38e17fa700 1 -- 172.21.15.97:0/3800077997 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38d400bba0 con 0x7f38e41015b0 2024-04-23T10:17:36.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:36.277+0000 7f38e17fa700 1 -- 172.21.15.97:0/3800077997 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38d4003920 con 0x7f38e41015b0 2024-04-23T10:17:36.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:36.277+0000 7f38eaa44700 1 -- 172.21.15.97:0/3800077997 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38e419c7b0 con 0x7f38e41015b0 2024-04-23T10:17:36.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:36.278+0000 7f38e17fa700 1 -- 172.21.15.97:0/3800077997 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f38d4029070 con 0x7f38e41015b0 2024-04-23T10:17:36.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:36.278+0000 7f38eaa44700 1 -- 172.21.15.97:0/3800077997 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38e4102a90 con 0x7f38e41015b0 2024-04-23T10:17:36.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:36.278+0000 7f38e17fa700 1 --2- 172.21.15.97:0/3800077997 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f38cc0396b0 0x7f38cc03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:17:36.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:36.278+0000 7f38e17fa700 1 -- 172.21.15.97:0/3800077997 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f38d401e070 con 0x7f38e41015b0 2024-04-23T10:17:36.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:36.279+0000 7f38e37fe700 1 --2- 172.21.15.97:0/3800077997 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f38cc0396b0 0x7f38cc03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:17:36.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:36.279+0000 7f38e37fe700 1 --2- 172.21.15.97:0/3800077997 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f38cc0396b0 0x7f38cc03bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f38d8006fd0 tx=0x7f38d8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:17:36.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:36.283+0000 7f38e17fa700 1 -- 172.21.15.97:0/3800077997 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f38d4013070 con 0x7f38e41015b0 2024-04-23T10:17:36.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:36.489+0000 7f38eaa44700 1 -- 172.21.15.97:0/3800077997 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f38e40640d0 con 0x7f38e41015b0 2024-04-23T10:17:36.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:36 smithi097 ceph-mon[29197]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:17:37.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:37.350+0000 7f38e17fa700 1 -- 172.21.15.97:0/3800077997 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f38e40640d0 con 0x7f38e41015b0 2024-04-23T10:17:37.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:37.352+0000 7f38eaa44700 1 -- 172.21.15.97:0/3800077997 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f38cc0396b0 msgr2=0x7f38cc03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:17:37.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:37.352+0000 7f38eaa44700 1 --2- 172.21.15.97:0/3800077997 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f38cc0396b0 0x7f38cc03bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f38d8006fd0 tx=0x7f38d8006e40 comp rx=0 tx=0).stop 2024-04-23T10:17:37.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:37.352+0000 7f38eaa44700 1 -- 172.21.15.97:0/3800077997 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38e41015b0 msgr2=0x7f38e4197ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:17:37.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:37.352+0000 7f38eaa44700 1 --2- 172.21.15.97:0/3800077997 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38e41015b0 0x7f38e4197ee0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f38d40056d0 tx=0x7f38d40057b0 comp rx=0 tx=0).stop 2024-04-23T10:17:37.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:37.352+0000 7f38eaa44700 1 -- 172.21.15.97:0/3800077997 shutdown_connections 2024-04-23T10:17:37.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:37.352+0000 7f38eaa44700 1 --2- 172.21.15.97:0/3800077997 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f38cc0396b0 0x7f38cc03bb70 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:17:37.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:37.352+0000 7f38eaa44700 1 --2- 172.21.15.97:0/3800077997 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38e41015b0 0x7f38e4197ee0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:17:37.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:37.352+0000 7f38eaa44700 1 -- 172.21.15.97:0/3800077997 >> 172.21.15.97:0/3800077997 conn(0x7f38e40fce20 msgr2=0x7f38e4102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:17:37.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:37.353+0000 7f38eaa44700 1 -- 172.21.15.97:0/3800077997 shutdown_connections 2024-04-23T10:17:37.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:17:37.353+0000 7f38eaa44700 1 -- 172.21.15.97:0/3800077997 wait complete. 2024-04-23T10:17:37.353 INFO:teuthology.orchestra.run.smithi097.stderr:adjusted tunables profile to default 2024-04-23T10:17:37.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:37 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/3800077997' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-23T10:17:37.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:37 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:37.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:37 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:37.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:37 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:37.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:37 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:37.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:37 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi196", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:17:37.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:37 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:17:37.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:37 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:17:38.044 INFO:tasks.cephadm:Adding mon.a on smithi097 2024-04-23T10:17:38.044 INFO:tasks.cephadm:Adding mon.b on smithi183 2024-04-23T10:17:38.044 INFO:tasks.cephadm:Adding mon.c on smithi196 2024-04-23T10:17:38.045 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph orch apply mon '3;smithi097:172.21.15.97=a;smithi183:172.21.15.183=b;smithi196:172.21.15.196=c' 2024-04-23T10:17:38.267 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T10:17:38.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:38 smithi097 ceph-mon[29197]: Updating smithi196:/etc/ceph/ceph.conf 2024-04-23T10:17:38.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:38 smithi097 ceph-mon[29197]: Updating smithi196:/var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T10:17:38.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:38 smithi097 ceph-mon[29197]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-04-23T10:17:38.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:38 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/3800077997' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-23T10:17:38.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:38 smithi097 ceph-mon[29197]: osdmap e4: 0 total, 0 up, 0 in 2024-04-23T10:17:38.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:38 smithi097 ceph-mon[29197]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:17:38.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:38 smithi097 ceph-mon[29197]: Updating smithi196:/var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T10:17:38.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:38 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:38.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:38 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:38.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:38 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:39.571 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:39.572+0000 7fe5ee0e7700 1 -- 172.21.15.196:0/2835724194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5e8108430 msgr2=0x7fe5e8108810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:17:39.572 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:39.572+0000 7fe5ee0e7700 1 --2- 172.21.15.196:0/2835724194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5e8108430 0x7fe5e8108810 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8009910 tx=0x7fe5d8009c20 comp rx=0 tx=0).stop 2024-04-23T10:17:39.572 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:39.572+0000 7fe5ee0e7700 1 -- 172.21.15.196:0/2835724194 shutdown_connections 2024-04-23T10:17:39.572 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:39.572+0000 7fe5ee0e7700 1 --2- 172.21.15.196:0/2835724194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5e8108430 0x7fe5e8108810 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:17:39.572 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:39.572+0000 7fe5ee0e7700 1 -- 172.21.15.196:0/2835724194 >> 172.21.15.196:0/2835724194 conn(0x7fe5e80788d0 msgr2=0x7fe5e8078ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:17:39.572 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:39.572+0000 7fe5ee0e7700 1 -- 172.21.15.196:0/2835724194 shutdown_connections 2024-04-23T10:17:39.572 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:39.572+0000 7fe5ee0e7700 1 -- 172.21.15.196:0/2835724194 wait complete. 2024-04-23T10:17:39.574 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:39.573+0000 7fe5ee0e7700 1 Processor -- start 2024-04-23T10:17:39.574 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:39.573+0000 7fe5ee0e7700 1 -- start start 2024-04-23T10:17:39.574 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:39.573+0000 7fe5ee0e7700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5e8108430 0x7fe5e8197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:17:39.574 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:39.573+0000 7fe5ee0e7700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e810c920 con 0x7fe5e8108430 2024-04-23T10:17:39.574 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:39.574+0000 7fe5e77fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5e8108430 0x7fe5e8197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:17:39.574 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:39.574+0000 7fe5e77fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5e8108430 0x7fe5e8197e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.196:36362/0 (socket says 172.21.15.196:36362) 2024-04-23T10:17:39.575 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:39.574+0000 7fe5e77fe700 1 -- 172.21.15.196:0/1615788741 learned_addr learned my addr 172.21.15.196:0/1615788741 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-23T10:17:39.575 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:39.574+0000 7fe5e77fe700 1 -- 172.21.15.196:0/1615788741 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5e81983d0 con 0x7fe5e8108430 2024-04-23T10:17:39.575 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:39.574+0000 7fe5e77fe700 1 --2- 172.21.15.196:0/1615788741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5e8108430 0x7fe5e8197e90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8006010 tx=0x7fe5d8005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:17:39.575 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:39.574+0000 7fe5e4ff9700 1 -- 172.21.15.196:0/1615788741 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe5d8026070 con 0x7fe5e8108430 2024-04-23T10:17:39.575 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:39.575+0000 7fe5ee0e7700 1 -- 172.21.15.196:0/1615788741 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5e81986c0 con 0x7fe5e8108430 2024-04-23T10:17:39.575 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:39.575+0000 7fe5e4ff9700 1 -- 172.21.15.196:0/1615788741 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5d800bba0 con 0x7fe5e8108430 2024-04-23T10:17:39.575 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:39.575+0000 7fe5e4ff9700 1 -- 172.21.15.196:0/1615788741 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe5d8003910 con 0x7fe5e8108430 2024-04-23T10:17:39.575 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:39.575+0000 7fe5ee0e7700 1 -- 172.21.15.196:0/1615788741 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5e819c760 con 0x7fe5e8108430 2024-04-23T10:17:39.576 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:39.575+0000 7fe5ee0e7700 1 -- 172.21.15.196:0/1615788741 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5e806a620 con 0x7fe5e8108430 2024-04-23T10:17:39.576 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:39.575+0000 7fe5e4ff9700 1 -- 172.21.15.196:0/1615788741 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fe5d8029070 con 0x7fe5e8108430 2024-04-23T10:17:39.576 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:39.576+0000 7fe5e4ff9700 1 --2- 172.21.15.196:0/1615788741 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fe5d00396b0 0x7fe5d003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:17:39.576 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:39.576+0000 7fe5e4ff9700 1 -- 172.21.15.196:0/1615788741 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe5d801e070 con 0x7fe5e8108430 2024-04-23T10:17:39.576 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:39.576+0000 7fe5e6ffd700 1 --2- 172.21.15.196:0/1615788741 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fe5d00396b0 0x7fe5d003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:17:39.577 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:39.576+0000 7fe5e6ffd700 1 --2- 172.21.15.196:0/1615788741 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fe5d00396b0 0x7fe5d003bb70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc006fd0 tx=0x7fe5dc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:17:39.580 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:39.581+0000 7fe5e4ff9700 1 -- 172.21.15.196:0/1615788741 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe5d8013070 con 0x7fe5e8108430 2024-04-23T10:17:39.786 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:39.787+0000 7fe5ee0e7700 1 -- 172.21.15.196:0/1615788741 --> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi097:172.21.15.97=a;smithi183:172.21.15.183=b;smithi196:172.21.15.196=c", "target": ["mon-mgr", ""]}) -- 0x7fe5e81991f0 con 0x7fe5d00396b0 2024-04-23T10:17:39.795 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:39.796+0000 7fe5e4ff9700 1 -- 172.21.15.196:0/1615788741 <== mgr.14152 v2:172.21.15.97:6800/3149043749 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fe5e81991f0 con 0x7fe5d00396b0 2024-04-23T10:17:39.796 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled mon update... 2024-04-23T10:17:39.798 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:39.798+0000 7fe5ee0e7700 1 -- 172.21.15.196:0/1615788741 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fe5d00396b0 msgr2=0x7fe5d003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:17:39.798 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:39.798+0000 7fe5ee0e7700 1 --2- 172.21.15.196:0/1615788741 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fe5d00396b0 0x7fe5d003bb70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc006fd0 tx=0x7fe5dc006e40 comp rx=0 tx=0).stop 2024-04-23T10:17:39.798 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:39.798+0000 7fe5ee0e7700 1 -- 172.21.15.196:0/1615788741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5e8108430 msgr2=0x7fe5e8197e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:17:39.798 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:39.798+0000 7fe5ee0e7700 1 --2- 172.21.15.196:0/1615788741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5e8108430 0x7fe5e8197e90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8006010 tx=0x7fe5d8005780 comp rx=0 tx=0).stop 2024-04-23T10:17:39.798 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:39.798+0000 7fe5ee0e7700 1 -- 172.21.15.196:0/1615788741 shutdown_connections 2024-04-23T10:17:39.798 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:39.798+0000 7fe5ee0e7700 1 --2- 172.21.15.196:0/1615788741 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fe5d00396b0 0x7fe5d003bb70 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:17:39.798 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:39.798+0000 7fe5ee0e7700 1 --2- 172.21.15.196:0/1615788741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5e8108430 0x7fe5e8197e90 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:17:39.798 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:39.798+0000 7fe5ee0e7700 1 -- 172.21.15.196:0/1615788741 >> 172.21.15.196:0/1615788741 conn(0x7fe5e80788d0 msgr2=0x7fe5e80fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:17:39.798 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:39.798+0000 7fe5ee0e7700 1 -- 172.21.15.196:0/1615788741 shutdown_connections 2024-04-23T10:17:39.798 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:39.798+0000 7fe5ee0e7700 1 -- 172.21.15.196:0/1615788741 wait complete. 2024-04-23T10:17:40.343 DEBUG:teuthology.orchestra.run.smithi183:mon.b> sudo journalctl -f -n 0 -u ceph-9b22af88-0159-11ef-bc93-c7b262605968@mon.b.service 2024-04-23T10:17:40.346 DEBUG:teuthology.orchestra.run.smithi196:mon.c> sudo journalctl -f -n 0 -u ceph-9b22af88-0159-11ef-bc93-c7b262605968@mon.c.service 2024-04-23T10:17:40.349 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-23T10:17:40.349 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-23T10:17:40.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:40 smithi097 ceph-mon[29197]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:17:40.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:40 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:40.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:40 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:17:40.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:40 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:17:40.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:40 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:17:40.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:40 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:40.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:40 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T10:17:40.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:40 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:17:41.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:41 smithi097 ceph-mon[29197]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi097:172.21.15.97=a;smithi183:172.21.15.183=b;smithi196:172.21.15.196=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:17:41.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:41 smithi097 ceph-mon[29197]: Saving service mon spec with placement smithi097:172.21.15.97=a;smithi183:172.21.15.183=b;smithi196:172.21.15.196=c;count:3 2024-04-23T10:17:41.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:41 smithi097 ceph-mon[29197]: Deploying daemon mon.c on smithi196 2024-04-23T10:17:42.301 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T10:17:42.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:42 smithi097 ceph-mon[29197]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:17:43.580 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:43 smithi196 systemd[1]: Starting Ceph mon.c for 9b22af88-0159-11ef-bc93-c7b262605968... 2024-04-23T10:17:44.075 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:44.075+0000 7fb49ec6b700 1 -- 172.21.15.196:0/2757829439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb498108630 msgr2=0x7fb498108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:17:44.075 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:44.075+0000 7fb49ec6b700 1 --2- 172.21.15.196:0/2757829439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb498108630 0x7fb498108a10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb49400b0a0 tx=0x7fb49400b3b0 comp rx=0 tx=0).stop 2024-04-23T10:17:44.076 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:44.075+0000 7fb49ec6b700 1 -- 172.21.15.196:0/2757829439 shutdown_connections 2024-04-23T10:17:44.076 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:44.075+0000 7fb49ec6b700 1 --2- 172.21.15.196:0/2757829439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb498108630 0x7fb498108a10 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:17:44.076 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:44.075+0000 7fb49ec6b700 1 -- 172.21.15.196:0/2757829439 >> 172.21.15.196:0/2757829439 conn(0x7fb4980fe050 msgr2=0x7fb498100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:17:44.076 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:44.075+0000 7fb49ec6b700 1 -- 172.21.15.196:0/2757829439 shutdown_connections 2024-04-23T10:17:44.076 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:44.075+0000 7fb49ec6b700 1 -- 172.21.15.196:0/2757829439 wait complete. 2024-04-23T10:17:44.076 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:44.076+0000 7fb49ec6b700 1 Processor -- start 2024-04-23T10:17:44.076 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:44.076+0000 7fb49ec6b700 1 -- start start 2024-04-23T10:17:44.076 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:44.076+0000 7fb49ec6b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb498108630 0x7fb498197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:17:44.076 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:44.076+0000 7fb49ec6b700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb49810caa0 con 0x7fb498108630 2024-04-23T10:17:44.076 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:44.076+0000 7fb49ca07700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb498108630 0x7fb498197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:17:44.076 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:44.076+0000 7fb49ca07700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb498108630 0x7fb498197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.196:36382/0 (socket says 172.21.15.196:36382) 2024-04-23T10:17:44.076 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:44.076+0000 7fb49ca07700 1 -- 172.21.15.196:0/1031786227 learned_addr learned my addr 172.21.15.196:0/1031786227 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-23T10:17:44.077 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:44.076+0000 7fb49ca07700 1 -- 172.21.15.196:0/1031786227 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb498198410 con 0x7fb498108630 2024-04-23T10:17:44.077 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:44.077+0000 7fb49ca07700 1 --2- 172.21.15.196:0/1031786227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb498108630 0x7fb498197ed0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb494015040 tx=0x7fb494009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:17:44.078 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:44.077+0000 7fb48dffb700 1 -- 172.21.15.196:0/1031786227 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb494021020 con 0x7fb498108630 2024-04-23T10:17:44.078 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:44.077+0000 7fb48dffb700 1 -- 172.21.15.196:0/1031786227 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb494007970 con 0x7fb498108630 2024-04-23T10:17:44.078 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:44.077+0000 7fb49ec6b700 1 -- 172.21.15.196:0/1031786227 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4981986a0 con 0x7fb498108630 2024-04-23T10:17:44.078 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:44.077+0000 7fb48dffb700 1 -- 172.21.15.196:0/1031786227 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb494017040 con 0x7fb498108630 2024-04-23T10:17:44.078 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:44.077+0000 7fb49ec6b700 1 -- 172.21.15.196:0/1031786227 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb49819c7a0 con 0x7fb498108630 2024-04-23T10:17:44.078 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:44.078+0000 7fb49ec6b700 1 -- 172.21.15.196:0/1031786227 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb498103ca0 con 0x7fb498108630 2024-04-23T10:17:44.078 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:44.078+0000 7fb48dffb700 1 -- 172.21.15.196:0/1031786227 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fb494024020 con 0x7fb498108630 2024-04-23T10:17:44.078 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:44.078+0000 7fb48dffb700 1 --2- 172.21.15.196:0/1031786227 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fb480039650 0x7fb48003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:17:44.078 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:44.078+0000 7fb48dffb700 1 -- 172.21.15.196:0/1031786227 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb49400f050 con 0x7fb498108630 2024-04-23T10:17:44.078 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:44.078+0000 7fb48ffff700 1 --2- 172.21.15.196:0/1031786227 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fb480039650 0x7fb48003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:17:44.079 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:44.079+0000 7fb48ffff700 1 --2- 172.21.15.196:0/1031786227 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fb480039650 0x7fb48003bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb488006fd0 tx=0x7fb488006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:17:44.082 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:44.084+0000 7fb48dffb700 1 -- 172.21.15.196:0/1031786227 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb49402b020 con 0x7fb498108630 2024-04-23T10:17:44.380 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:44.381+0000 7fb49ec6b700 1 -- 172.21.15.196:0/1031786227 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fb498199230 con 0x7fb498108630 2024-04-23T10:17:44.381 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:44.382+0000 7fb48dffb700 1 -- 172.21.15.196:0/1031786227 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7fb494011e90 con 0x7fb498108630 2024-04-23T10:17:44.381 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-23T10:17:44.381 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":1,"fsid":"9b22af88-0159-11ef-bc93-c7b262605968","modified":"2024-04-23T10:16:03.064570Z","created":"2024-04-23T10:16:03.064570Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:3300","nonce":0},{"type":"v1","addr":"172.21.15.97:6789","nonce":0}]},"addr":"172.21.15.97:6789/0","public_addr":"172.21.15.97:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-23T10:17:44.383 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:44.384+0000 7fb49ec6b700 1 -- 172.21.15.196:0/1031786227 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fb480039650 msgr2=0x7fb48003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:17:44.383 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:44.384+0000 7fb49ec6b700 1 --2- 172.21.15.196:0/1031786227 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fb480039650 0x7fb48003bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb488006fd0 tx=0x7fb488006e40 comp rx=0 tx=0).stop 2024-04-23T10:17:44.383 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:44.384+0000 7fb49ec6b700 1 -- 172.21.15.196:0/1031786227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb498108630 msgr2=0x7fb498197ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:17:44.383 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:44.384+0000 7fb49ec6b700 1 --2- 172.21.15.196:0/1031786227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb498108630 0x7fb498197ed0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb494015040 tx=0x7fb494009790 comp rx=0 tx=0).stop 2024-04-23T10:17:44.383 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:44.384+0000 7fb49ec6b700 1 -- 172.21.15.196:0/1031786227 shutdown_connections 2024-04-23T10:17:44.383 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:44.384+0000 7fb49ec6b700 1 --2- 172.21.15.196:0/1031786227 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fb480039650 0x7fb48003bb10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:17:44.384 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:44.384+0000 7fb49ec6b700 1 --2- 172.21.15.196:0/1031786227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb498108630 0x7fb498197ed0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:17:44.384 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:44.384+0000 7fb49ec6b700 1 -- 172.21.15.196:0/1031786227 >> 172.21.15.196:0/1031786227 conn(0x7fb4980fe050 msgr2=0x7fb4980ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:17:44.384 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:44.384+0000 7fb49ec6b700 1 -- 172.21.15.196:0/1031786227 shutdown_connections 2024-04-23T10:17:44.384 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:44.384+0000 7fb49ec6b700 1 -- 172.21.15.196:0/1031786227 wait complete. 2024-04-23T10:17:44.384 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 1 2024-04-23T10:17:44.491 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 podman[32887]: 2024-04-23 10:17:44.140375654 +0000 UTC m=+0.022118567 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T10:17:44.491 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 podman[32887]: 2024-04-23 10:17:44.25648724 +0000 UTC m=+0.138230140 container create 2505a701babd24022931b21bf99653cc140802cf9bdd18f01a30bf8572b99de7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mon-c, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T10:17:44.491 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 podman[32887]: 2024-04-23 10:17:44.465785808 +0000 UTC m=+0.347528708 container init 2505a701babd24022931b21bf99653cc140802cf9bdd18f01a30bf8572b99de7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mon-c, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD) 2024-04-23T10:17:44.491 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 podman[32887]: 2024-04-23 10:17:44.46913538 +0000 UTC m=+0.350878280 container start 2505a701babd24022931b21bf99653cc140802cf9bdd18f01a30bf8572b99de7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mon-c, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416) 2024-04-23T10:17:44.491 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: set uid:gid to 167:167 (ceph:ceph) 2024-04-23T10:17:44.491 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev), process ceph-mon, pid 2 2024-04-23T10:17:44.492 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: pidfile_write: ignore empty --pid-file 2024-04-23T10:17:44.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: load: jerasure load: lrc 2024-04-23T10:17:44.843 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: RocksDB version: 7.9.2 2024-04-23T10:17:44.843 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Git sha 0 2024-04-23T10:17:44.843 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Compile date 2024-04-22 10:57:25 2024-04-23T10:17:44.843 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: DB SUMMARY 2024-04-23T10:17:44.843 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: DB Session ID: KBGHE8X138JA79M9GEOA 2024-04-23T10:17:44.843 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: CURRENT file: CURRENT 2024-04-23T10:17:44.843 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: IDENTITY file: IDENTITY 2024-04-23T10:17:44.843 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-23T10:17:44.843 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-04-23T10:17:44.843 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-04-23T10:17:44.844 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.error_if_exists: 0 2024-04-23T10:17:44.844 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.create_if_missing: 0 2024-04-23T10:17:44.844 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.paranoid_checks: 1 2024-04-23T10:17:44.844 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-23T10:17:44.844 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-23T10:17:44.844 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-23T10:17:44.844 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.env: 0x562e50e06cc0 2024-04-23T10:17:44.844 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.fs: PosixFileSystem 2024-04-23T10:17:44.844 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.info_log: 0x562e51fa3be0 2024-04-23T10:17:44.844 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.max_file_opening_threads: 16 2024-04-23T10:17:44.844 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.statistics: (nil) 2024-04-23T10:17:44.844 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.use_fsync: 0 2024-04-23T10:17:44.844 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.max_log_file_size: 0 2024-04-23T10:17:44.845 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-23T10:17:44.845 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-23T10:17:44.845 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.keep_log_file_num: 1000 2024-04-23T10:17:44.845 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.recycle_log_file_num: 0 2024-04-23T10:17:44.845 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.allow_fallocate: 1 2024-04-23T10:17:44.845 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.allow_mmap_reads: 0 2024-04-23T10:17:44.845 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.allow_mmap_writes: 0 2024-04-23T10:17:44.845 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.use_direct_reads: 0 2024-04-23T10:17:44.845 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-23T10:17:44.845 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.create_missing_column_families: 0 2024-04-23T10:17:44.845 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.db_log_dir: 2024-04-23T10:17:44.845 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.wal_dir: 2024-04-23T10:17:44.845 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-23T10:17:44.846 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-23T10:17:44.846 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-23T10:17:44.846 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-23T10:17:44.846 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-23T10:17:44.846 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-23T10:17:44.846 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.advise_random_on_open: 1 2024-04-23T10:17:44.846 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.db_write_buffer_size: 0 2024-04-23T10:17:44.846 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.write_buffer_manager: 0x562e51f370e0 2024-04-23T10:17:44.846 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-23T10:17:44.846 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-23T10:17:44.846 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-23T10:17:44.846 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.rate_limiter: (nil) 2024-04-23T10:17:44.846 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-23T10:17:44.846 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.wal_recovery_mode: 2 2024-04-23T10:17:44.846 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.enable_thread_tracking: 0 2024-04-23T10:17:44.847 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.enable_pipelined_write: 0 2024-04-23T10:17:44.847 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.unordered_write: 0 2024-04-23T10:17:44.847 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-23T10:17:44.847 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-23T10:17:44.847 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-23T10:17:44.847 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-23T10:17:44.847 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.row_cache: None 2024-04-23T10:17:44.847 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.wal_filter: None 2024-04-23T10:17:44.847 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-23T10:17:44.847 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.allow_ingest_behind: 0 2024-04-23T10:17:44.847 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.two_write_queues: 0 2024-04-23T10:17:44.847 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.manual_wal_flush: 0 2024-04-23T10:17:44.847 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.wal_compression: 0 2024-04-23T10:17:44.847 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.atomic_flush: 0 2024-04-23T10:17:44.847 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-23T10:17:44.847 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-23T10:17:44.848 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-23T10:17:44.848 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.log_readahead_size: 0 2024-04-23T10:17:44.848 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-23T10:17:44.848 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.best_efforts_recovery: 0 2024-04-23T10:17:44.848 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-23T10:17:44.848 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-23T10:17:44.848 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.allow_data_in_errors: 0 2024-04-23T10:17:44.848 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.db_host_id: __hostname__ 2024-04-23T10:17:44.848 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-23T10:17:44.848 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.max_background_jobs: 2 2024-04-23T10:17:44.848 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.max_background_compactions: -1 2024-04-23T10:17:44.848 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.max_subcompactions: 1 2024-04-23T10:17:44.850 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-23T10:17:44.850 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-23T10:17:44.850 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-23T10:17:44.850 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.max_total_wal_size: 0 2024-04-23T10:17:44.850 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-23T10:17:44.850 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-23T10:17:44.850 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-23T10:17:44.850 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-23T10:17:44.850 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.max_open_files: -1 2024-04-23T10:17:44.850 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.bytes_per_sync: 0 2024-04-23T10:17:44.850 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-23T10:17:44.850 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-23T10:17:44.851 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.compaction_readahead_size: 0 2024-04-23T10:17:44.851 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.max_background_flushes: -1 2024-04-23T10:17:44.851 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Compression algorithms supported: 2024-04-23T10:17:44.851 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-23T10:17:44.851 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: kZSTD supported: 0 2024-04-23T10:17:44.851 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: kXpressCompression supported: 0 2024-04-23T10:17:44.851 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: kLZ4HCCompression supported: 1 2024-04-23T10:17:44.851 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: kZlibCompression supported: 1 2024-04-23T10:17:44.851 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: kSnappyCompression supported: 1 2024-04-23T10:17:44.851 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: kLZ4Compression supported: 1 2024-04-23T10:17:44.851 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: kBZip2Compression supported: 0 2024-04-23T10:17:44.851 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-23T10:17:44.851 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-23T10:17:44.851 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-04-23T10:17:44.851 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-23T10:17:44.852 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-23T10:17:44.852 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.merge_operator: 2024-04-23T10:17:44.852 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.compaction_filter: None 2024-04-23T10:17:44.852 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.compaction_filter_factory: None 2024-04-23T10:17:44.852 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.sst_partitioner_factory: None 2024-04-23T10:17:44.852 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-23T10:17:44.852 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-23T10:17:44.852 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x562e51fa3dc0) 2024-04-23T10:17:44.852 INFO:journalctl@ceph.mon.c.smithi196.stdout: cache_index_and_filter_blocks: 1 2024-04-23T10:17:44.852 INFO:journalctl@ceph.mon.c.smithi196.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-23T10:17:44.852 INFO:journalctl@ceph.mon.c.smithi196.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-23T10:17:44.852 INFO:journalctl@ceph.mon.c.smithi196.stdout: pin_top_level_index_and_filter: 1 2024-04-23T10:17:44.852 INFO:journalctl@ceph.mon.c.smithi196.stdout: index_type: 0 2024-04-23T10:17:44.852 INFO:journalctl@ceph.mon.c.smithi196.stdout: data_block_index_type: 0 2024-04-23T10:17:44.852 INFO:journalctl@ceph.mon.c.smithi196.stdout: index_shortening: 1 2024-04-23T10:17:44.852 INFO:journalctl@ceph.mon.c.smithi196.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-23T10:17:44.853 INFO:journalctl@ceph.mon.c.smithi196.stdout: checksum: 4 2024-04-23T10:17:44.853 INFO:journalctl@ceph.mon.c.smithi196.stdout: no_block_cache: 0 2024-04-23T10:17:44.853 INFO:journalctl@ceph.mon.c.smithi196.stdout: block_cache: 0x562e51f691f0 2024-04-23T10:17:44.853 INFO:journalctl@ceph.mon.c.smithi196.stdout: block_cache_name: BinnedLRUCache 2024-04-23T10:17:44.853 INFO:journalctl@ceph.mon.c.smithi196.stdout: block_cache_options: 2024-04-23T10:17:44.853 INFO:journalctl@ceph.mon.c.smithi196.stdout: capacity : 536870912 2024-04-23T10:17:44.853 INFO:journalctl@ceph.mon.c.smithi196.stdout: num_shard_bits : 4 2024-04-23T10:17:44.853 INFO:journalctl@ceph.mon.c.smithi196.stdout: strict_capacity_limit : 0 2024-04-23T10:17:44.853 INFO:journalctl@ceph.mon.c.smithi196.stdout: high_pri_pool_ratio: 0.000 2024-04-23T10:17:44.853 INFO:journalctl@ceph.mon.c.smithi196.stdout: block_cache_compressed: (nil) 2024-04-23T10:17:44.853 INFO:journalctl@ceph.mon.c.smithi196.stdout: persistent_cache: (nil) 2024-04-23T10:17:44.853 INFO:journalctl@ceph.mon.c.smithi196.stdout: block_size: 4096 2024-04-23T10:17:44.853 INFO:journalctl@ceph.mon.c.smithi196.stdout: block_size_deviation: 10 2024-04-23T10:17:44.853 INFO:journalctl@ceph.mon.c.smithi196.stdout: block_restart_interval: 16 2024-04-23T10:17:44.853 INFO:journalctl@ceph.mon.c.smithi196.stdout: index_block_restart_interval: 1 2024-04-23T10:17:44.853 INFO:journalctl@ceph.mon.c.smithi196.stdout: metadata_block_size: 4096 2024-04-23T10:17:44.853 INFO:journalctl@ceph.mon.c.smithi196.stdout: partition_filters: 0 2024-04-23T10:17:44.854 INFO:journalctl@ceph.mon.c.smithi196.stdout: use_delta_encoding: 1 2024-04-23T10:17:44.854 INFO:journalctl@ceph.mon.c.smithi196.stdout: filter_policy: bloomfilter 2024-04-23T10:17:44.854 INFO:journalctl@ceph.mon.c.smithi196.stdout: whole_key_filtering: 1 2024-04-23T10:17:44.854 INFO:journalctl@ceph.mon.c.smithi196.stdout: verify_compression: 0 2024-04-23T10:17:44.854 INFO:journalctl@ceph.mon.c.smithi196.stdout: read_amp_bytes_per_bit: 0 2024-04-23T10:17:44.854 INFO:journalctl@ceph.mon.c.smithi196.stdout: format_version: 5 2024-04-23T10:17:44.854 INFO:journalctl@ceph.mon.c.smithi196.stdout: enable_index_compression: 1 2024-04-23T10:17:44.854 INFO:journalctl@ceph.mon.c.smithi196.stdout: block_align: 0 2024-04-23T10:17:44.854 INFO:journalctl@ceph.mon.c.smithi196.stdout: max_auto_readahead_size: 262144 2024-04-23T10:17:44.854 INFO:journalctl@ceph.mon.c.smithi196.stdout: prepopulate_block_cache: 0 2024-04-23T10:17:44.854 INFO:journalctl@ceph.mon.c.smithi196.stdout: initial_auto_readahead_size: 8192 2024-04-23T10:17:44.854 INFO:journalctl@ceph.mon.c.smithi196.stdout: num_file_reads_for_auto_readahead: 2 2024-04-23T10:17:44.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.write_buffer_size: 33554432 2024-04-23T10:17:44.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.max_write_buffer_number: 2 2024-04-23T10:17:44.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.compression: NoCompression 2024-04-23T10:17:44.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.bottommost_compression: Disabled 2024-04-23T10:17:44.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.prefix_extractor: nullptr 2024-04-23T10:17:44.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-23T10:17:44.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.num_levels: 7 2024-04-23T10:17:44.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-23T10:17:44.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-23T10:17:44.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-23T10:17:44.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-23T10:17:44.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-23T10:17:44.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-23T10:17:44.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-23T10:17:44.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-23T10:17:44.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-23T10:17:44.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-23T10:17:44.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-23T10:17:44.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-23T10:17:44.855 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-23T10:17:44.856 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.compression_opts.level: 32767 2024-04-23T10:17:44.856 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.compression_opts.strategy: 0 2024-04-23T10:17:44.856 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-23T10:17:44.856 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-23T10:17:44.856 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-23T10:17:44.856 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-23T10:17:44.856 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.compression_opts.enabled: false 2024-04-23T10:17:44.856 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-23T10:17:44.856 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-23T10:17:44.856 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-23T10:17:44.856 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-23T10:17:44.856 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.target_file_size_base: 67108864 2024-04-23T10:17:44.856 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-23T10:17:44.856 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-23T10:17:44.856 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-23T10:17:44.856 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-23T10:17:44.856 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-23T10:17:44.857 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-23T10:17:44.857 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-23T10:17:44.857 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-23T10:17:44.857 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-23T10:17:44.857 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-23T10:17:44.857 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-23T10:17:44.857 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-23T10:17:44.857 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-23T10:17:44.857 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-23T10:17:44.857 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.arena_block_size: 1048576 2024-04-23T10:17:44.857 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-23T10:17:44.857 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-23T10:17:44.857 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.disable_auto_compactions: 0 2024-04-23T10:17:44.857 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-23T10:17:44.857 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-23T10:17:44.857 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-23T10:17:44.858 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-23T10:17:44.858 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-23T10:17:44.858 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-23T10:17:44.858 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-23T10:17:44.858 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-23T10:17:44.858 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-23T10:17:44.858 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-23T10:17:44.858 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-23T10:17:44.858 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.inplace_update_support: 0 2024-04-23T10:17:44.858 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-23T10:17:44.858 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-23T10:17:44.858 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-23T10:17:44.858 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-23T10:17:44.858 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.bloom_locality: 0 2024-04-23T10:17:44.858 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.max_successive_merges: 0 2024-04-23T10:17:44.859 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-23T10:17:44.859 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.paranoid_file_checks: 0 2024-04-23T10:17:44.859 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.force_consistency_checks: 1 2024-04-23T10:17:44.859 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.report_bg_io_stats: 0 2024-04-23T10:17:44.859 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.ttl: 2592000 2024-04-23T10:17:44.859 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-23T10:17:44.859 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-23T10:17:44.859 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-23T10:17:44.859 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.enable_blob_files: false 2024-04-23T10:17:44.859 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.min_blob_size: 0 2024-04-23T10:17:44.859 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.blob_file_size: 268435456 2024-04-23T10:17:44.859 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-23T10:17:44.859 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-23T10:17:44.859 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-23T10:17:44.859 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-23T10:17:44.860 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-23T10:17:44.860 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.blob_file_starting_level: 0 2024-04-23T10:17:44.860 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-23T10:17:44.860 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-23T10:17:44.860 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-23T10:17:44.860 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 17714421-fa20-49b2-b906-418e501ecd63 2024-04-23T10:17:44.860 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713867464498687, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-23T10:17:44.860 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-23T10:17:44.860 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713867464499096, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713867464, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "17714421-fa20-49b2-b906-418e501ecd63", "db_session_id": "KBGHE8X138JA79M9GEOA", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-23T10:17:44.860 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713867464499173, "job": 1, "event": "recovery_finished"} 2024-04-23T10:17:44.860 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-23T10:17:44.860 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-23T10:17:44.860 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x562e5207e000 2024-04-23T10:17:44.860 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: DB pointer 0x562e52068000 2024-04-23T10:17:44.861 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-23T10:17:44.861 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-23T10:17:44.861 INFO:journalctl@ceph.mon.c.smithi196.stdout: ** DB Stats ** 2024-04-23T10:17:44.861 INFO:journalctl@ceph.mon.c.smithi196.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-23T10:17:44.861 INFO:journalctl@ceph.mon.c.smithi196.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-23T10:17:44.861 INFO:journalctl@ceph.mon.c.smithi196.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-23T10:17:44.861 INFO:journalctl@ceph.mon.c.smithi196.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-23T10:17:44.861 INFO:journalctl@ceph.mon.c.smithi196.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-23T10:17:44.861 INFO:journalctl@ceph.mon.c.smithi196.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-23T10:17:44.861 INFO:journalctl@ceph.mon.c.smithi196.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-23T10:17:44.861 INFO:journalctl@ceph.mon.c.smithi196.stdout: 2024-04-23T10:17:44.861 INFO:journalctl@ceph.mon.c.smithi196.stdout: ** Compaction Stats [default] ** 2024-04-23T10:17:44.861 INFO:journalctl@ceph.mon.c.smithi196.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-23T10:17:44.862 INFO:journalctl@ceph.mon.c.smithi196.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-23T10:17:44.862 INFO:journalctl@ceph.mon.c.smithi196.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-23T10:17:44.862 INFO:journalctl@ceph.mon.c.smithi196.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-23T10:17:44.862 INFO:journalctl@ceph.mon.c.smithi196.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-23T10:17:44.862 INFO:journalctl@ceph.mon.c.smithi196.stdout: 2024-04-23T10:17:44.862 INFO:journalctl@ceph.mon.c.smithi196.stdout: ** Compaction Stats [default] ** 2024-04-23T10:17:44.862 INFO:journalctl@ceph.mon.c.smithi196.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-23T10:17:44.862 INFO:journalctl@ceph.mon.c.smithi196.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-23T10:17:44.862 INFO:journalctl@ceph.mon.c.smithi196.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-23T10:17:44.862 INFO:journalctl@ceph.mon.c.smithi196.stdout: 2024-04-23T10:17:44.862 INFO:journalctl@ceph.mon.c.smithi196.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-23T10:17:44.862 INFO:journalctl@ceph.mon.c.smithi196.stdout: 2024-04-23T10:17:44.862 INFO:journalctl@ceph.mon.c.smithi196.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-23T10:17:44.862 INFO:journalctl@ceph.mon.c.smithi196.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-23T10:17:44.862 INFO:journalctl@ceph.mon.c.smithi196.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-23T10:17:44.863 INFO:journalctl@ceph.mon.c.smithi196.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-23T10:17:44.863 INFO:journalctl@ceph.mon.c.smithi196.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-23T10:17:44.863 INFO:journalctl@ceph.mon.c.smithi196.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-23T10:17:44.863 INFO:journalctl@ceph.mon.c.smithi196.stdout: Cumulative compaction: 0.00 GB write, 0.66 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-23T10:17:44.863 INFO:journalctl@ceph.mon.c.smithi196.stdout: Interval compaction: 0.00 GB write, 0.66 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-23T10:17:44.863 INFO:journalctl@ceph.mon.c.smithi196.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-23T10:17:44.863 INFO:journalctl@ceph.mon.c.smithi196.stdout: Block cache BinnedLRUCache@0x562e51f691f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-04-23T10:17:44.863 INFO:journalctl@ceph.mon.c.smithi196.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-23T10:17:44.863 INFO:journalctl@ceph.mon.c.smithi196.stdout: 2024-04-23T10:17:44.863 INFO:journalctl@ceph.mon.c.smithi196.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-23T10:17:44.863 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-04-23T10:17:44.863 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: using public_addr v2:172.21.15.196:0/0 -> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] 2024-04-23T10:17:44.863 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: starting mon.c rank -1 at public addrs [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] at bind addrs [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 9b22af88-0159-11ef-bc93-c7b262605968 2024-04-23T10:17:44.863 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: mon.c@-1(???) e0 preinit fsid 9b22af88-0159-11ef-bc93-c7b262605968 2024-04-23T10:17:44.863 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: mon.c@-1(synchronizing).mds e1 new map 2024-04-23T10:17:44.863 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: mon.c@-1(synchronizing).mds e1 print_map 2024-04-23T10:17:44.863 INFO:journalctl@ceph.mon.c.smithi196.stdout: e1 2024-04-23T10:17:44.864 INFO:journalctl@ceph.mon.c.smithi196.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T10:17:44.864 INFO:journalctl@ceph.mon.c.smithi196.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-23T10:17:44.864 INFO:journalctl@ceph.mon.c.smithi196.stdout: legacy client fscid: -1 2024-04-23T10:17:44.864 INFO:journalctl@ceph.mon.c.smithi196.stdout: 2024-04-23T10:17:44.864 INFO:journalctl@ceph.mon.c.smithi196.stdout: No filesystems configured 2024-04-23T10:17:44.864 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-23T10:17:44.864 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-23T10:17:44.864 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-23T10:17:44.864 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-23T10:17:44.864 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-23T10:17:44.864 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-23T10:17:44.864 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-23T10:17:44.864 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-23T10:17:44.864 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-23T10:17:44.864 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-23T10:17:44.865 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi196", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:17:44.865 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:44.865 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:44.865 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:44.865 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:44.865 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi183", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:17:44.865 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:17:44.865 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:17:44.865 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: Updating smithi183:/etc/ceph/ceph.conf 2024-04-23T10:17:44.865 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:17:44.865 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: Updating smithi183:/var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T10:17:44.865 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: Deploying cephadm binary to smithi196 2024-04-23T10:17:44.865 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: Updating smithi183:/etc/ceph/ceph.client.admin.keyring 2024-04-23T10:17:44.865 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: Updating smithi183:/var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T10:17:44.866 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:44.866 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:44.866 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:44.866 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:17:44.866 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:44.866 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: Added host smithi196 2024-04-23T10:17:44.866 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:17:44.866 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:17:44.866 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:44.866 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:44.866 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:17:44.866 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:44.866 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T10:17:44.866 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:17:44.866 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/3800077997' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-23T10:17:44.866 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:44.867 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:44.867 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:44.867 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:44.867 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi196", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:17:44.867 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:17:44.867 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:17:44.867 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: Updating smithi196:/etc/ceph/ceph.conf 2024-04-23T10:17:44.867 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: Updating smithi196:/var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T10:17:44.867 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-04-23T10:17:44.867 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/3800077997' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-23T10:17:44.867 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: osdmap e4: 0 total, 0 up, 0 in 2024-04-23T10:17:44.867 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:17:44.867 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: Updating smithi196:/var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T10:17:44.867 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:44.867 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:44.868 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:44.868 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:17:44.868 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:44.868 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:17:44.868 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:17:44.868 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:17:44.868 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:44.868 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T10:17:44.868 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:17:44.868 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi097:172.21.15.97=a;smithi183:172.21.15.183=b;smithi196:172.21.15.196=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:17:44.868 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: Saving service mon spec with placement smithi097:172.21.15.97=a;smithi183:172.21.15.183=b;smithi196:172.21.15.196=c;count:3 2024-04-23T10:17:44.868 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: Deploying daemon mon.c on smithi196 2024-04-23T10:17:44.868 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:17:44.868 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 ceph-mon[32919]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-23T10:17:44.868 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 bash[32887]: 2505a701babd24022931b21bf99653cc140802cf9bdd18f01a30bf8572b99de7 2024-04-23T10:17:44.868 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:44 smithi196 systemd[1]: Started Ceph mon.c for 9b22af88-0159-11ef-bc93-c7b262605968. 2024-04-23T10:17:46.111 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-23T10:17:46.111 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-23T10:17:46.346 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T10:17:49.843 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:49 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T10:17:49.843 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:49 smithi196 ceph-mon[32919]: mon.a calling monitor election 2024-04-23T10:17:49.843 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:49 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T10:17:49.843 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:49 smithi196 ceph-mon[32919]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:17:49.843 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:49 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T10:17:49.843 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:49 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T10:17:49.843 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:49 smithi196 ceph-mon[32919]: mon.c calling monitor election 2024-04-23T10:17:49.843 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:49 smithi196 ceph-mon[32919]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:17:49.843 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:49 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T10:17:49.843 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:49 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T10:17:49.843 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:49 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T10:17:49.843 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:49 smithi196 ceph-mon[32919]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-23T10:17:49.843 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:49 smithi196 ceph-mon[32919]: monmap epoch 2 2024-04-23T10:17:49.843 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:49 smithi196 ceph-mon[32919]: fsid 9b22af88-0159-11ef-bc93-c7b262605968 2024-04-23T10:17:49.844 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:49 smithi196 ceph-mon[32919]: last_changed 2024-04-23T10:17:44.522326+0000 2024-04-23T10:17:49.844 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:49 smithi196 ceph-mon[32919]: created 2024-04-23T10:16:03.064570+0000 2024-04-23T10:17:49.844 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:49 smithi196 ceph-mon[32919]: min_mon_release 19 (squid) 2024-04-23T10:17:49.844 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:49 smithi196 ceph-mon[32919]: election_strategy: 1 2024-04-23T10:17:49.844 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:49 smithi196 ceph-mon[32919]: 0: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.a 2024-04-23T10:17:49.844 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:49 smithi196 ceph-mon[32919]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-04-23T10:17:49.844 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:49 smithi196 ceph-mon[32919]: fsmap 2024-04-23T10:17:49.844 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:49 smithi196 ceph-mon[32919]: osdmap e4: 0 total, 0 up, 0 in 2024-04-23T10:17:49.844 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:49 smithi196 ceph-mon[32919]: mgrmap e13: x(active, since 50s) 2024-04-23T10:17:49.844 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:49 smithi196 ceph-mon[32919]: overall HEALTH_OK 2024-04-23T10:17:49.844 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:49 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:49.844 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:49 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:49.844 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:49 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:49.844 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:49 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T10:17:49.844 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:49 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:17:49.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:49 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T10:17:49.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:49 smithi097 ceph-mon[29197]: mon.a calling monitor election 2024-04-23T10:17:49.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:49 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T10:17:49.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:49 smithi097 ceph-mon[29197]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:17:49.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:49 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T10:17:49.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:49 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T10:17:49.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:49 smithi097 ceph-mon[29197]: mon.c calling monitor election 2024-04-23T10:17:49.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:49 smithi097 ceph-mon[29197]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:17:49.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:49 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T10:17:49.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:49 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T10:17:49.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:49 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T10:17:49.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:49 smithi097 ceph-mon[29197]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-23T10:17:49.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:49 smithi097 ceph-mon[29197]: monmap epoch 2 2024-04-23T10:17:49.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:49 smithi097 ceph-mon[29197]: fsid 9b22af88-0159-11ef-bc93-c7b262605968 2024-04-23T10:17:49.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:49 smithi097 ceph-mon[29197]: last_changed 2024-04-23T10:17:44.522326+0000 2024-04-23T10:17:49.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:49 smithi097 ceph-mon[29197]: created 2024-04-23T10:16:03.064570+0000 2024-04-23T10:17:49.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:49 smithi097 ceph-mon[29197]: min_mon_release 19 (squid) 2024-04-23T10:17:49.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:49 smithi097 ceph-mon[29197]: election_strategy: 1 2024-04-23T10:17:49.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:49 smithi097 ceph-mon[29197]: 0: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.a 2024-04-23T10:17:49.863 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:49 smithi097 ceph-mon[29197]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-04-23T10:17:49.863 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:49 smithi097 ceph-mon[29197]: fsmap 2024-04-23T10:17:49.863 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:49 smithi097 ceph-mon[29197]: osdmap e4: 0 total, 0 up, 0 in 2024-04-23T10:17:49.863 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:49 smithi097 ceph-mon[29197]: mgrmap e13: x(active, since 50s) 2024-04-23T10:17:49.863 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:49 smithi097 ceph-mon[29197]: overall HEALTH_OK 2024-04-23T10:17:49.863 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:49 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:49.863 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:49 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:49.863 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:49 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:49.863 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:49 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T10:17:49.863 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:49 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:17:50.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:50 smithi097 ceph-mon[29197]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:17:50.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:50 smithi097 ceph-mon[29197]: Deploying daemon mon.b on smithi183 2024-04-23T10:17:50.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:50 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T10:17:50.893 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:50 smithi196 ceph-mon[32919]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:17:50.894 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:50 smithi196 ceph-mon[32919]: Deploying daemon mon.b on smithi183 2024-04-23T10:17:50.894 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:50 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T10:17:50.895 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:50.894+0000 7f04cb071700 1 -- 172.21.15.196:0/80849649 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04ac0035c0 msgr2=0x7f04ac005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:17:50.895 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:50.894+0000 7f04cb071700 1 --2- 172.21.15.196:0/80849649 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04ac0035c0 0x7f04ac005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f04c406ad30 tx=0x7f04b400feb0 comp rx=0 tx=0).stop 2024-04-23T10:17:50.895 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:50.894+0000 7f04cb071700 1 -- 172.21.15.196:0/80849649 shutdown_connections 2024-04-23T10:17:50.895 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:50.894+0000 7f04cb071700 1 --2- 172.21.15.196:0/80849649 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04ac0035c0 0x7f04ac005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:17:50.896 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:50.894+0000 7f04cb071700 1 -- 172.21.15.196:0/80849649 >> 172.21.15.196:0/80849649 conn(0x7f04c40fcf80 msgr2=0x7f04c40ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:17:50.896 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:50.894+0000 7f04cb071700 1 -- 172.21.15.196:0/80849649 shutdown_connections 2024-04-23T10:17:50.896 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:50.894+0000 7f04cb071700 1 -- 172.21.15.196:0/80849649 wait complete. 2024-04-23T10:17:50.896 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:50.895+0000 7f04cb071700 1 Processor -- start 2024-04-23T10:17:50.896 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:50.895+0000 7f04cb071700 1 -- start start 2024-04-23T10:17:50.896 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:50.895+0000 7f04cb071700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04ac0035c0 0x7f04c419a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:17:50.896 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:50.895+0000 7f04cb071700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f04c419a780 0x7f04c419eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:17:50.896 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:50.895+0000 7f04cb071700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f04c410a4a0 con 0x7f04ac0035c0 2024-04-23T10:17:50.896 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:50.895+0000 7f04cb071700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f04c410a320 con 0x7f04c419a780 2024-04-23T10:17:50.896 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:50.896+0000 7f04c3fff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f04c419a780 0x7f04c419eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:17:50.896 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:50.896+0000 7f04c3fff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f04c419a780 0x7f04c419eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:42558/0 (socket says 172.21.15.196:42558) 2024-04-23T10:17:50.896 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:50.896+0000 7f04c3fff700 1 -- 172.21.15.196:0/3753630733 learned_addr learned my addr 172.21.15.196:0/3753630733 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-23T10:17:50.896 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:50.896+0000 7f04c8e0d700 1 --2- 172.21.15.196:0/3753630733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04ac0035c0 0x7f04c419a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:17:50.896 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:50.896+0000 7f04c3fff700 1 -- 172.21.15.196:0/3753630733 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f04c419a780 msgr2=0x7f04c419eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-23T10:17:50.896 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:50.896+0000 7f04c3fff700 1 -- 172.21.15.196:0/3753630733 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f04c419a780 msgr2=0x7f04c419eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-23T10:17:50.897 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:50.896+0000 7f04c3fff700 1 --2- 172.21.15.196:0/3753630733 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f04c419a780 0x7f04c419eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-23T10:17:50.897 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:50.896+0000 7f04c3fff700 1 --2- 172.21.15.196:0/3753630733 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f04c419a780 0x7f04c419eb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T10:17:50.897 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:50.896+0000 7f04c8e0d700 1 -- 172.21.15.196:0/3753630733 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f04c419a780 msgr2=0x7f04c419eb10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T10:17:50.897 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:50.896+0000 7f04c8e0d700 1 --2- 172.21.15.196:0/3753630733 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f04c419a780 0x7f04c419eb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:17:50.897 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:50.896+0000 7f04c8e0d700 1 -- 172.21.15.196:0/3753630733 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04c419adc0 con 0x7f04ac0035c0 2024-04-23T10:17:50.897 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:50.896+0000 7f04c8e0d700 1 --2- 172.21.15.196:0/3753630733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04ac0035c0 0x7f04c419a240 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f04b4000c00 tx=0x7f04b40046f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:17:50.898 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:50.896+0000 7f04c1ffb700 1 -- 172.21.15.196:0/3753630733 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f04b4013b00 con 0x7f04ac0035c0 2024-04-23T10:17:50.898 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:50.897+0000 7f04c1ffb700 1 -- 172.21.15.196:0/3753630733 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04b40044e0 con 0x7f04ac0035c0 2024-04-23T10:17:50.898 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:50.897+0000 7f04cb071700 1 -- 172.21.15.196:0/3753630733 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04c419f050 con 0x7f04ac0035c0 2024-04-23T10:17:50.898 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:50.897+0000 7f04c1ffb700 1 -- 172.21.15.196:0/3753630733 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f04b4011470 con 0x7f04ac0035c0 2024-04-23T10:17:50.898 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:50.897+0000 7f04cb071700 1 -- 172.21.15.196:0/3753630733 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04c419f5b0 con 0x7f04ac0035c0 2024-04-23T10:17:50.899 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:50.898+0000 7f04c1ffb700 1 -- 172.21.15.196:0/3753630733 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f04b4023070 con 0x7f04ac0035c0 2024-04-23T10:17:50.899 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:50.898+0000 7f04cb071700 1 -- 172.21.15.196:0/3753630733 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04c406b530 con 0x7f04ac0035c0 2024-04-23T10:17:50.899 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:50.898+0000 7f04c1ffb700 1 --2- 172.21.15.196:0/3753630733 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f04b0039440 0x7f04b003b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:17:50.899 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:50.898+0000 7f04c1ffb700 1 -- 172.21.15.196:0/3753630733 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f04b4020050 con 0x7f04ac0035c0 2024-04-23T10:17:50.899 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:50.898+0000 7f04c3fff700 1 --2- 172.21.15.196:0/3753630733 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f04b0039440 0x7f04b003b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:17:50.899 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:50.899+0000 7f04c3fff700 1 --2- 172.21.15.196:0/3753630733 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f04b0039440 0x7f04b003b900 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f04b8009180 tx=0x7f04b8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:17:50.903 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:50.904+0000 7f04c1ffb700 1 -- 172.21.15.196:0/3753630733 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f04b4028020 con 0x7f04ac0035c0 2024-04-23T10:17:51.186 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:51.187+0000 7f04cb071700 1 -- 172.21.15.196:0/3753630733 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f04c419f860 con 0x7f04ac0035c0 2024-04-23T10:17:51.186 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:51.187+0000 7f04c1ffb700 1 -- 172.21.15.196:0/3753630733 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f04b4024e20 con 0x7f04ac0035c0 2024-04-23T10:17:51.187 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-23T10:17:51.187 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":2,"fsid":"9b22af88-0159-11ef-bc93-c7b262605968","modified":"2024-04-23T10:17:44.522326Z","created":"2024-04-23T10:16:03.064570Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:3300","nonce":0},{"type":"v1","addr":"172.21.15.97:6789","nonce":0}]},"addr":"172.21.15.97:6789/0","public_addr":"172.21.15.97:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:3300","nonce":0},{"type":"v1","addr":"172.21.15.196:6789","nonce":0}]},"addr":"172.21.15.196:6789/0","public_addr":"172.21.15.196:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-23T10:17:51.189 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:51.190+0000 7f04cb071700 1 -- 172.21.15.196:0/3753630733 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f04b0039440 msgr2=0x7f04b003b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:17:51.189 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:51.190+0000 7f04cb071700 1 --2- 172.21.15.196:0/3753630733 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f04b0039440 0x7f04b003b900 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f04b8009180 tx=0x7f04b8008040 comp rx=0 tx=0).stop 2024-04-23T10:17:51.189 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:51.190+0000 7f04cb071700 1 -- 172.21.15.196:0/3753630733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04ac0035c0 msgr2=0x7f04c419a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:17:51.189 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:51.190+0000 7f04cb071700 1 --2- 172.21.15.196:0/3753630733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04ac0035c0 0x7f04c419a240 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f04b4000c00 tx=0x7f04b40046f0 comp rx=0 tx=0).stop 2024-04-23T10:17:51.190 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:51.190+0000 7f04cb071700 1 -- 172.21.15.196:0/3753630733 shutdown_connections 2024-04-23T10:17:51.190 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:51.190+0000 7f04cb071700 1 --2- 172.21.15.196:0/3753630733 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f04b0039440 0x7f04b003b900 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:17:51.190 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:51.190+0000 7f04cb071700 1 --2- 172.21.15.196:0/3753630733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f04ac0035c0 0x7f04c419a240 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:17:51.190 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:51.190+0000 7f04cb071700 1 --2- 172.21.15.196:0/3753630733 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f04c419a780 0x7f04c419eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:17:51.190 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:51.190+0000 7f04cb071700 1 -- 172.21.15.196:0/3753630733 >> 172.21.15.196:0/3753630733 conn(0x7f04c40fcf80 msgr2=0x7f04c40fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:17:51.190 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:51.190+0000 7f04cb071700 1 -- 172.21.15.196:0/3753630733 shutdown_connections 2024-04-23T10:17:51.190 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:51.190+0000 7f04cb071700 1 -- 172.21.15.196:0/3753630733 wait complete. 2024-04-23T10:17:51.190 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 2 2024-04-23T10:17:51.779 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:51 smithi196 ceph-mon[32919]: from='client.? 172.21.15.196:0/3753630733' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-23T10:17:51.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:51 smithi097 ceph-mon[29197]: from='client.? 172.21.15.196:0/3753630733' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-23T10:17:52.805 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-23T10:17:52.805 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-23T10:17:52.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:52 smithi097 ceph-mon[29197]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:17:53.058 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T10:17:53.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:52 smithi196 ceph-mon[32919]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:17:53.196 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: mon.b@-1(???) e0 preinit fsid 9b22af88-0159-11ef-bc93-c7b262605968 2024-04-23T10:17:53.197 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: mon.b@-1(synchronizing).mds e1 new map 2024-04-23T10:17:53.197 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: mon.b@-1(synchronizing).mds e1 print_map 2024-04-23T10:17:53.197 INFO:journalctl@ceph.mon.b.smithi183.stdout: e1 2024-04-23T10:17:53.197 INFO:journalctl@ceph.mon.b.smithi183.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T10:17:53.197 INFO:journalctl@ceph.mon.b.smithi183.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-23T10:17:53.197 INFO:journalctl@ceph.mon.b.smithi183.stdout: legacy client fscid: -1 2024-04-23T10:17:53.197 INFO:journalctl@ceph.mon.b.smithi183.stdout: 2024-04-23T10:17:53.197 INFO:journalctl@ceph.mon.b.smithi183.stdout: No filesystems configured 2024-04-23T10:17:53.197 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-23T10:17:53.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-23T10:17:53.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-23T10:17:53.652 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-23T10:17:53.652 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-23T10:17:53.652 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-23T10:17:53.652 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-23T10:17:53.652 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-23T10:17:53.652 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-23T10:17:53.652 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-23T10:17:53.652 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi196", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:17:53.652 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:53.652 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:53.652 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:53.652 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:53.652 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi183", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:17:53.653 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:17:53.653 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:17:53.653 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: Updating smithi183:/etc/ceph/ceph.conf 2024-04-23T10:17:53.653 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:17:53.653 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: Updating smithi183:/var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T10:17:53.653 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: Deploying cephadm binary to smithi196 2024-04-23T10:17:53.653 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: Updating smithi183:/etc/ceph/ceph.client.admin.keyring 2024-04-23T10:17:53.653 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: Updating smithi183:/var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T10:17:53.653 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:53.653 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:53.653 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:53.653 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:17:53.653 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:53.653 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: Added host smithi196 2024-04-23T10:17:53.653 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:17:53.654 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:17:53.654 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:53.654 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:53.654 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:17:53.654 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:53.654 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T10:17:53.654 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:17:53.654 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/3800077997' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-23T10:17:53.654 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:53.654 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:53.654 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:53.654 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:53.654 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi196", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:17:53.654 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:17:53.655 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:17:53.655 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: Updating smithi196:/etc/ceph/ceph.conf 2024-04-23T10:17:53.655 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: Updating smithi196:/var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T10:17:53.655 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-04-23T10:17:53.655 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/3800077997' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-23T10:17:53.655 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: osdmap e4: 0 total, 0 up, 0 in 2024-04-23T10:17:53.655 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:17:53.655 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: Updating smithi196:/var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T10:17:53.655 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:53.655 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:53.655 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:53.655 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:17:53.655 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:53.655 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:17:53.655 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:17:53.655 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:17:53.656 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:53.656 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T10:17:53.656 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:17:53.656 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi097:172.21.15.97=a;smithi183:172.21.15.183=b;smithi196:172.21.15.196=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:17:53.656 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: Saving service mon spec with placement smithi097:172.21.15.97=a;smithi183:172.21.15.183=b;smithi196:172.21.15.196=c;count:3 2024-04-23T10:17:53.656 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: Deploying daemon mon.c on smithi196 2024-04-23T10:17:53.656 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:17:53.656 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T10:17:53.656 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: mon.a calling monitor election 2024-04-23T10:17:53.656 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T10:17:53.656 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:17:53.656 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T10:17:53.656 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T10:17:53.656 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: mon.c calling monitor election 2024-04-23T10:17:53.657 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:17:53.657 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T10:17:53.657 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T10:17:53.657 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T10:17:53.657 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-23T10:17:53.657 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: monmap epoch 2 2024-04-23T10:17:53.657 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: fsid 9b22af88-0159-11ef-bc93-c7b262605968 2024-04-23T10:17:53.657 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: last_changed 2024-04-23T10:17:44.522326+0000 2024-04-23T10:17:53.657 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: created 2024-04-23T10:16:03.064570+0000 2024-04-23T10:17:53.657 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: min_mon_release 19 (squid) 2024-04-23T10:17:53.657 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: election_strategy: 1 2024-04-23T10:17:53.657 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: 0: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.a 2024-04-23T10:17:53.657 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-04-23T10:17:53.658 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: fsmap 2024-04-23T10:17:53.658 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: osdmap e4: 0 total, 0 up, 0 in 2024-04-23T10:17:53.658 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: mgrmap e13: x(active, since 50s) 2024-04-23T10:17:53.658 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: overall HEALTH_OK 2024-04-23T10:17:53.658 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:53.658 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:53.658 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:53.658 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T10:17:53.658 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:17:53.658 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:17:53.658 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: Deploying daemon mon.b on smithi183 2024-04-23T10:17:53.658 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T10:17:53.658 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: from='client.? 172.21.15.196:0/3753630733' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-23T10:17:53.658 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:17:53.658 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 ceph-mon[32317]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-23T10:17:53.659 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 bash[32298]: 7e8f30ab14b837cdf77364aa38b0cdd8f0e6540c6774d8d257d326a93fbef244 2024-04-23T10:17:53.659 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:17:53 smithi183 systemd[1]: Started Ceph mon.b for 9b22af88-0159-11ef-bc93-c7b262605968. 2024-04-23T10:17:58.568 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:58 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T10:17:58.568 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:58 smithi097 ceph-mon[29197]: mon.a calling monitor election 2024-04-23T10:17:58.568 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:58 smithi097 ceph-mon[29197]: mon.c calling monitor election 2024-04-23T10:17:58.568 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:58 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T10:17:58.568 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:58 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T10:17:58.569 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:58 smithi097 ceph-mon[29197]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:17:58.569 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:58 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T10:17:58.569 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:58 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T10:17:58.569 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:58 smithi097 ceph-mon[29197]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:17:58.569 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:58 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T10:17:58.569 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:58 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T10:17:58.569 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:58 smithi097 ceph-mon[29197]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:17:58.569 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:58 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T10:17:58.569 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:58 smithi097 ceph-mon[29197]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-23T10:17:58.569 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:58 smithi097 ceph-mon[29197]: monmap epoch 3 2024-04-23T10:17:58.569 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:58 smithi097 ceph-mon[29197]: fsid 9b22af88-0159-11ef-bc93-c7b262605968 2024-04-23T10:17:58.569 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:58 smithi097 ceph-mon[29197]: last_changed 2024-04-23T10:17:53.204963+0000 2024-04-23T10:17:58.569 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:58 smithi097 ceph-mon[29197]: created 2024-04-23T10:16:03.064570+0000 2024-04-23T10:17:58.569 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:58 smithi097 ceph-mon[29197]: min_mon_release 19 (squid) 2024-04-23T10:17:58.570 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:58 smithi097 ceph-mon[29197]: election_strategy: 1 2024-04-23T10:17:58.570 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:58 smithi097 ceph-mon[29197]: 0: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.a 2024-04-23T10:17:58.570 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:58 smithi097 ceph-mon[29197]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-04-23T10:17:58.570 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:58 smithi097 ceph-mon[29197]: 2: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.b 2024-04-23T10:17:58.570 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:58 smithi097 ceph-mon[29197]: fsmap 2024-04-23T10:17:58.570 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:58 smithi097 ceph-mon[29197]: osdmap e4: 0 total, 0 up, 0 in 2024-04-23T10:17:58.570 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:58 smithi097 ceph-mon[29197]: mgrmap e13: x(active, since 58s) 2024-04-23T10:17:58.570 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:58 smithi097 ceph-mon[29197]: overall HEALTH_OK 2024-04-23T10:17:58.570 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:58 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:58.570 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:58 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:58.570 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:58 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:58.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:58 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T10:17:58.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:58 smithi196 ceph-mon[32919]: mon.a calling monitor election 2024-04-23T10:17:58.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:58 smithi196 ceph-mon[32919]: mon.c calling monitor election 2024-04-23T10:17:58.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:58 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T10:17:58.593 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:58 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T10:17:58.593 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:58 smithi196 ceph-mon[32919]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:17:58.593 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:58 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T10:17:58.593 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:58 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T10:17:58.593 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:58 smithi196 ceph-mon[32919]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:17:58.593 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:58 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T10:17:58.593 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:58 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T10:17:58.593 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:58 smithi196 ceph-mon[32919]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:17:58.593 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:58 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T10:17:58.593 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:58 smithi196 ceph-mon[32919]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-23T10:17:58.593 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:58 smithi196 ceph-mon[32919]: monmap epoch 3 2024-04-23T10:17:58.593 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:58 smithi196 ceph-mon[32919]: fsid 9b22af88-0159-11ef-bc93-c7b262605968 2024-04-23T10:17:58.593 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:58 smithi196 ceph-mon[32919]: last_changed 2024-04-23T10:17:53.204963+0000 2024-04-23T10:17:58.593 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:58 smithi196 ceph-mon[32919]: created 2024-04-23T10:16:03.064570+0000 2024-04-23T10:17:58.593 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:58 smithi196 ceph-mon[32919]: min_mon_release 19 (squid) 2024-04-23T10:17:58.594 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:58 smithi196 ceph-mon[32919]: election_strategy: 1 2024-04-23T10:17:58.594 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:58 smithi196 ceph-mon[32919]: 0: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.a 2024-04-23T10:17:58.594 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:58 smithi196 ceph-mon[32919]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-04-23T10:17:58.594 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:58 smithi196 ceph-mon[32919]: 2: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.b 2024-04-23T10:17:58.594 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:58 smithi196 ceph-mon[32919]: fsmap 2024-04-23T10:17:58.594 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:58 smithi196 ceph-mon[32919]: osdmap e4: 0 total, 0 up, 0 in 2024-04-23T10:17:58.594 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:58 smithi196 ceph-mon[32919]: mgrmap e13: x(active, since 58s) 2024-04-23T10:17:58.594 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:58 smithi196 ceph-mon[32919]: overall HEALTH_OK 2024-04-23T10:17:58.594 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:58 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:58.594 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:58 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:58.594 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:58 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:58.907 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:58.908+0000 7f8fa4b27700 1 -- 172.21.15.196:0/1163786245 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f880035c0 msgr2=0x7f8f88005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:17:58.907 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:58.908+0000 7f8fa4b27700 1 --2- 172.21.15.196:0/1163786245 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f880035c0 0x7f8f88005a50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0079c80 tx=0x7f8f9000fe10 comp rx=0 tx=0).stop 2024-04-23T10:17:58.907 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:58.908+0000 7f8fa4b27700 1 -- 172.21.15.196:0/1163786245 shutdown_connections 2024-04-23T10:17:58.907 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:58.908+0000 7f8fa4b27700 1 --2- 172.21.15.196:0/1163786245 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f880035c0 0x7f8f88005a50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:17:58.907 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:58.908+0000 7f8fa4b27700 1 -- 172.21.15.196:0/1163786245 >> 172.21.15.196:0/1163786245 conn(0x7f8fa0077af0 msgr2=0x7f8fa0077f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:17:58.908 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:58.908+0000 7f8fa4b27700 1 -- 172.21.15.196:0/1163786245 shutdown_connections 2024-04-23T10:17:58.908 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:58.908+0000 7f8fa4b27700 1 -- 172.21.15.196:0/1163786245 wait complete. 2024-04-23T10:17:58.910 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:58.909+0000 7f8fa4b27700 1 Processor -- start 2024-04-23T10:17:58.910 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:58.909+0000 7f8fa4b27700 1 -- start start 2024-04-23T10:17:58.910 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:58.909+0000 7f8fa4b27700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f880035c0 0x7f8fa0196f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:17:58.910 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:58.909+0000 7f8fa4b27700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fa0197490 0x7f8fa019b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:17:58.910 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:58.909+0000 7f8fa4b27700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8fa019bd60 0x7f8fa019c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:17:58.910 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:58.909+0000 7f8fa4b27700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa0109380 con 0x7f8fa0197490 2024-04-23T10:17:58.910 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:58.909+0000 7f8fa4b27700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa0109200 con 0x7f8fa019bd60 2024-04-23T10:17:58.911 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:58.909+0000 7f8fa4b27700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa0109500 con 0x7f8f880035c0 2024-04-23T10:17:58.911 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:58.909+0000 7f8f9e59c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f880035c0 0x7f8fa0196f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:17:58.911 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:58.909+0000 7f8f9e59c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f880035c0 0x7f8fa0196f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:45380/0 (socket says 172.21.15.196:45380) 2024-04-23T10:17:58.911 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:58.909+0000 7f8f9e59c700 1 -- 172.21.15.196:0/2856444512 learned_addr learned my addr 172.21.15.196:0/2856444512 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-23T10:17:58.911 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:58.909+0000 7f8f9dd9b700 1 --2- 172.21.15.196:0/2856444512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fa0197490 0x7f8fa019b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:17:58.911 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:58.910+0000 7f8f9e59c700 1 -- 172.21.15.196:0/2856444512 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f880035c0 msgr2=0x7f8fa0196f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-23T10:17:58.911 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:58.910+0000 7f8f9e59c700 1 -- 172.21.15.196:0/2856444512 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f880035c0 msgr2=0x7f8fa0196f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-23T10:17:58.911 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:58.910+0000 7f8f9e59c700 1 --2- 172.21.15.196:0/2856444512 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f880035c0 0x7f8fa0196f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-23T10:17:58.911 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:58.910+0000 7f8f9e59c700 1 --2- 172.21.15.196:0/2856444512 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f880035c0 0x7f8fa0196f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T10:17:58.911 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:58.910+0000 7f8f9ed9d700 1 --2- 172.21.15.196:0/2856444512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8fa019bd60 0x7f8fa019c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:17:58.911 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:58.910+0000 7f8f9ed9d700 1 -- 172.21.15.196:0/2856444512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8fa019bd60 msgr2=0x7f8fa019c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-04-23T10:17:58.911 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:58.910+0000 7f8f9ed9d700 1 -- 172.21.15.196:0/2856444512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8fa019bd60 msgr2=0x7f8fa019c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-23T10:17:58.912 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:58.910+0000 7f8f9ed9d700 1 --2- 172.21.15.196:0/2856444512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8fa019bd60 0x7f8fa019c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-23T10:17:58.912 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:58.910+0000 7f8f9ed9d700 1 --2- 172.21.15.196:0/2856444512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8fa019bd60 0x7f8fa019c210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T10:17:58.912 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:58.910+0000 7f8f9dd9b700 1 -- 172.21.15.196:0/2856444512 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f880035c0 msgr2=0x7f8fa0196f50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T10:17:58.912 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:58.910+0000 7f8f9dd9b700 1 --2- 172.21.15.196:0/2856444512 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f880035c0 0x7f8fa0196f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:17:58.912 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:58.910+0000 7f8f9dd9b700 1 -- 172.21.15.196:0/2856444512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8fa019bd60 msgr2=0x7f8fa019c210 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T10:17:58.912 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:58.910+0000 7f8f9dd9b700 1 --2- 172.21.15.196:0/2856444512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8fa019bd60 0x7f8fa019c210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:17:58.912 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:58.910+0000 7f8f9dd9b700 1 -- 172.21.15.196:0/2856444512 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fa01a3ad0 con 0x7f8fa0197490 2024-04-23T10:17:58.912 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:58.910+0000 7f8f9dd9b700 1 --2- 172.21.15.196:0/2856444512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fa0197490 0x7f8fa019b820 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8f9400b550 tx=0x7f8f9400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:17:58.912 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:58.910+0000 7f8f8f7fe700 1 -- 172.21.15.196:0/2856444512 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f94004230 con 0x7f8fa0197490 2024-04-23T10:17:58.912 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:58.910+0000 7f8f8f7fe700 1 -- 172.21.15.196:0/2856444512 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f940043d0 con 0x7f8fa0197490 2024-04-23T10:17:58.912 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:58.910+0000 7f8fa4b27700 1 -- 172.21.15.196:0/2856444512 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fa01a3d00 con 0x7f8fa0197490 2024-04-23T10:17:58.912 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:58.910+0000 7f8f8f7fe700 1 -- 172.21.15.196:0/2856444512 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f94011b90 con 0x7f8fa0197490 2024-04-23T10:17:58.912 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:58.910+0000 7f8fa4b27700 1 -- 172.21.15.196:0/2856444512 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fa01a42b0 con 0x7f8fa0197490 2024-04-23T10:17:58.913 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:58.911+0000 7f8f8f7fe700 1 -- 172.21.15.196:0/2856444512 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f8f9401a400 con 0x7f8fa0197490 2024-04-23T10:17:58.913 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:58.911+0000 7f8f8f7fe700 1 --2- 172.21.15.196:0/2856444512 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f8f80039ae0 0x7f8f8003bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:17:58.913 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:58.912+0000 7f8f8f7fe700 1 -- 172.21.15.196:0/2856444512 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f8f9404c340 con 0x7f8fa0197490 2024-04-23T10:17:58.913 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:58.912+0000 7f8fa4b27700 1 -- 172.21.15.196:0/2856444512 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fa00640a0 con 0x7f8fa0197490 2024-04-23T10:17:58.913 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:58.912+0000 7f8f9e59c700 1 --2- 172.21.15.196:0/2856444512 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f8f80039ae0 0x7f8f8003bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:17:58.914 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:58.912+0000 7f8f9e59c700 1 --2- 172.21.15.196:0/2856444512 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f8f80039ae0 0x7f8f8003bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8f9000f9c0 tx=0x7f8f90005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:17:58.916 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:58.917+0000 7f8f8f7fe700 1 -- 172.21.15.196:0/2856444512 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8f940151c0 con 0x7f8fa0197490 2024-04-23T10:17:59.215 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:59.216+0000 7f8fa4b27700 1 -- 172.21.15.196:0/2856444512 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f8fa0100c30 con 0x7f8fa0197490 2024-04-23T10:17:59.215 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:59.216+0000 7f8f8f7fe700 1 -- 172.21.15.196:0/2856444512 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1283 (secure 0 0 0) 0x7f8f94018510 con 0x7f8fa0197490 2024-04-23T10:17:59.215 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-23T10:17:59.216 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":3,"fsid":"9b22af88-0159-11ef-bc93-c7b262605968","modified":"2024-04-23T10:17:53.204963Z","created":"2024-04-23T10:16:03.064570Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:3300","nonce":0},{"type":"v1","addr":"172.21.15.97:6789","nonce":0}]},"addr":"172.21.15.97:6789/0","public_addr":"172.21.15.97:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:3300","nonce":0},{"type":"v1","addr":"172.21.15.196:6789","nonce":0}]},"addr":"172.21.15.196:6789/0","public_addr":"172.21.15.196:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-23T10:17:59.217 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:59.218+0000 7f8fa4b27700 1 -- 172.21.15.196:0/2856444512 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f8f80039ae0 msgr2=0x7f8f8003bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:17:59.218 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:59.218+0000 7f8fa4b27700 1 --2- 172.21.15.196:0/2856444512 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f8f80039ae0 0x7f8f8003bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8f9000f9c0 tx=0x7f8f90005da0 comp rx=0 tx=0).stop 2024-04-23T10:17:59.218 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:59.218+0000 7f8fa4b27700 1 -- 172.21.15.196:0/2856444512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fa0197490 msgr2=0x7f8fa019b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:17:59.218 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:59.218+0000 7f8fa4b27700 1 --2- 172.21.15.196:0/2856444512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fa0197490 0x7f8fa019b820 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8f9400b550 tx=0x7f8f9400b910 comp rx=0 tx=0).stop 2024-04-23T10:17:59.218 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:59.218+0000 7f8fa4b27700 1 -- 172.21.15.196:0/2856444512 shutdown_connections 2024-04-23T10:17:59.218 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:59.218+0000 7f8fa4b27700 1 --2- 172.21.15.196:0/2856444512 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f8f80039ae0 0x7f8f8003bfa0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:17:59.218 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:59.218+0000 7f8fa4b27700 1 --2- 172.21.15.196:0/2856444512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8fa019bd60 0x7f8fa019c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:17:59.218 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:59.218+0000 7f8fa4b27700 1 --2- 172.21.15.196:0/2856444512 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f880035c0 0x7f8fa0196f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:17:59.218 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:59.218+0000 7f8fa4b27700 1 --2- 172.21.15.196:0/2856444512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fa0197490 0x7f8fa019b820 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:17:59.218 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:59.218+0000 7f8fa4b27700 1 -- 172.21.15.196:0/2856444512 >> 172.21.15.196:0/2856444512 conn(0x7f8fa0077af0 msgr2=0x7f8fa00fffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:17:59.218 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:59.218+0000 7f8fa4b27700 1 -- 172.21.15.196:0/2856444512 shutdown_connections 2024-04-23T10:17:59.218 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:17:59.218+0000 7f8fa4b27700 1 -- 172.21.15.196:0/2856444512 wait complete. 2024-04-23T10:17:59.219 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 3 2024-04-23T10:17:59.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:59 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:59.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:59 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:17:59.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:59 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T10:17:59.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:17:59 smithi196 ceph-mon[32919]: from='client.? 172.21.15.196:0/2856444512' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-23T10:17:59.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:59 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:17:59.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:59 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:17:59.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:59 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T10:17:59.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:17:59 smithi097 ceph-mon[29197]: from='client.? 172.21.15.196:0/2856444512' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-23T10:17:59.927 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-23T10:17:59.927 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-23T10:18:00.164 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T10:18:00.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:00 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T10:18:00.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:00 smithi183 ceph-mon[32317]: mon.a calling monitor election 2024-04-23T10:18:00.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:00 smithi183 ceph-mon[32317]: mon.c calling monitor election 2024-04-23T10:18:00.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:00 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T10:18:00.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:00 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T10:18:00.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:00 smithi183 ceph-mon[32317]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:18:00.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:00 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T10:18:00.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:00 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T10:18:00.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:00 smithi183 ceph-mon[32317]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:18:00.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:00 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T10:18:00.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:00 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T10:18:00.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:00 smithi183 ceph-mon[32317]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:18:00.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:00 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T10:18:00.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:00 smithi183 ceph-mon[32317]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-23T10:18:00.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:00 smithi183 ceph-mon[32317]: monmap epoch 3 2024-04-23T10:18:00.652 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:00 smithi183 ceph-mon[32317]: fsid 9b22af88-0159-11ef-bc93-c7b262605968 2024-04-23T10:18:00.652 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:00 smithi183 ceph-mon[32317]: last_changed 2024-04-23T10:17:53.204963+0000 2024-04-23T10:18:00.652 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:00 smithi183 ceph-mon[32317]: created 2024-04-23T10:16:03.064570+0000 2024-04-23T10:18:00.652 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:00 smithi183 ceph-mon[32317]: min_mon_release 19 (squid) 2024-04-23T10:18:00.652 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:00 smithi183 ceph-mon[32317]: election_strategy: 1 2024-04-23T10:18:00.652 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:00 smithi183 ceph-mon[32317]: 0: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.a 2024-04-23T10:18:00.652 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:00 smithi183 ceph-mon[32317]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-04-23T10:18:00.652 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:00 smithi183 ceph-mon[32317]: 2: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.b 2024-04-23T10:18:00.652 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:00 smithi183 ceph-mon[32317]: fsmap 2024-04-23T10:18:00.652 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:00 smithi183 ceph-mon[32317]: osdmap e4: 0 total, 0 up, 0 in 2024-04-23T10:18:00.652 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:00 smithi183 ceph-mon[32317]: mgrmap e13: x(active, since 58s) 2024-04-23T10:18:00.652 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:00 smithi183 ceph-mon[32317]: overall HEALTH_OK 2024-04-23T10:18:00.652 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:00 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:00.652 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:00 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:00.653 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:00 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:00.653 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:00 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:00.653 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:00 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:18:00.653 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:00 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T10:18:00.653 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:00 smithi183 ceph-mon[32317]: from='client.? 172.21.15.196:0/2856444512' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-23T10:18:00.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:00 smithi097 ceph-mon[29197]: mon.b calling monitor election 2024-04-23T10:18:00.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:00 smithi097 ceph-mon[29197]: mon.b calling monitor election 2024-04-23T10:18:00.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:00 smithi097 ceph-mon[29197]: mon.c calling monitor election 2024-04-23T10:18:00.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:00 smithi097 ceph-mon[29197]: mon.a calling monitor election 2024-04-23T10:18:00.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:00 smithi097 ceph-mon[29197]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-23T10:18:00.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:00 smithi097 ceph-mon[29197]: monmap epoch 3 2024-04-23T10:18:00.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:00 smithi097 ceph-mon[29197]: fsid 9b22af88-0159-11ef-bc93-c7b262605968 2024-04-23T10:18:00.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:00 smithi097 ceph-mon[29197]: last_changed 2024-04-23T10:17:53.204963+0000 2024-04-23T10:18:00.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:00 smithi097 ceph-mon[29197]: created 2024-04-23T10:16:03.064570+0000 2024-04-23T10:18:00.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:00 smithi097 ceph-mon[29197]: min_mon_release 19 (squid) 2024-04-23T10:18:00.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:00 smithi097 ceph-mon[29197]: election_strategy: 1 2024-04-23T10:18:00.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:00 smithi097 ceph-mon[29197]: 0: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.a 2024-04-23T10:18:00.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:00 smithi097 ceph-mon[29197]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-04-23T10:18:00.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:00 smithi097 ceph-mon[29197]: 2: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.b 2024-04-23T10:18:00.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:00 smithi097 ceph-mon[29197]: fsmap 2024-04-23T10:18:00.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:00 smithi097 ceph-mon[29197]: osdmap e4: 0 total, 0 up, 0 in 2024-04-23T10:18:00.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:00 smithi097 ceph-mon[29197]: mgrmap e13: x(active, since 60s) 2024-04-23T10:18:00.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:00 smithi097 ceph-mon[29197]: overall HEALTH_OK 2024-04-23T10:18:00.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:00 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:00.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:00 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:00.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:00 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:18:00.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:00 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:18:00.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:00 smithi097 ceph-mon[29197]: Updating smithi097:/etc/ceph/ceph.conf 2024-04-23T10:18:00.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:00 smithi097 ceph-mon[29197]: Updating smithi183:/etc/ceph/ceph.conf 2024-04-23T10:18:00.863 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:00 smithi097 ceph-mon[29197]: Updating smithi196:/etc/ceph/ceph.conf 2024-04-23T10:18:00.863 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:00 smithi097 ceph-mon[29197]: Updating smithi097:/var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T10:18:00.863 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:00 smithi097 ceph-mon[29197]: Updating smithi196:/var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T10:18:00.863 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:00 smithi097 ceph-mon[29197]: Updating smithi183:/var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T10:18:00.863 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:00 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:00.863 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:00 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:01.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:00 smithi196 ceph-mon[32919]: mon.b calling monitor election 2024-04-23T10:18:01.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:00 smithi196 ceph-mon[32919]: mon.b calling monitor election 2024-04-23T10:18:01.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:00 smithi196 ceph-mon[32919]: mon.c calling monitor election 2024-04-23T10:18:01.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:00 smithi196 ceph-mon[32919]: mon.a calling monitor election 2024-04-23T10:18:01.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:00 smithi196 ceph-mon[32919]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-23T10:18:01.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:00 smithi196 ceph-mon[32919]: monmap epoch 3 2024-04-23T10:18:01.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:00 smithi196 ceph-mon[32919]: fsid 9b22af88-0159-11ef-bc93-c7b262605968 2024-04-23T10:18:01.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:00 smithi196 ceph-mon[32919]: last_changed 2024-04-23T10:17:53.204963+0000 2024-04-23T10:18:01.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:00 smithi196 ceph-mon[32919]: created 2024-04-23T10:16:03.064570+0000 2024-04-23T10:18:01.093 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:00 smithi196 ceph-mon[32919]: min_mon_release 19 (squid) 2024-04-23T10:18:01.093 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:00 smithi196 ceph-mon[32919]: election_strategy: 1 2024-04-23T10:18:01.093 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:00 smithi196 ceph-mon[32919]: 0: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.a 2024-04-23T10:18:01.093 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:00 smithi196 ceph-mon[32919]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-04-23T10:18:01.093 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:00 smithi196 ceph-mon[32919]: 2: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.b 2024-04-23T10:18:01.093 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:00 smithi196 ceph-mon[32919]: fsmap 2024-04-23T10:18:01.093 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:00 smithi196 ceph-mon[32919]: osdmap e4: 0 total, 0 up, 0 in 2024-04-23T10:18:01.093 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:00 smithi196 ceph-mon[32919]: mgrmap e13: x(active, since 60s) 2024-04-23T10:18:01.093 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:00 smithi196 ceph-mon[32919]: overall HEALTH_OK 2024-04-23T10:18:01.093 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:00 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:01.093 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:00 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:01.093 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:00 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:18:01.093 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:00 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:18:01.093 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:00 smithi196 ceph-mon[32919]: Updating smithi097:/etc/ceph/ceph.conf 2024-04-23T10:18:01.093 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:00 smithi196 ceph-mon[32919]: Updating smithi183:/etc/ceph/ceph.conf 2024-04-23T10:18:01.094 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:00 smithi196 ceph-mon[32919]: Updating smithi196:/etc/ceph/ceph.conf 2024-04-23T10:18:01.094 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:00 smithi196 ceph-mon[32919]: Updating smithi097:/var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T10:18:01.094 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:00 smithi196 ceph-mon[32919]: Updating smithi196:/var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T10:18:01.094 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:00 smithi196 ceph-mon[32919]: Updating smithi183:/var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T10:18:01.094 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:00 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:01.094 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:00 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:01.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:00 smithi183 ceph-mon[32317]: mon.b calling monitor election 2024-04-23T10:18:01.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:00 smithi183 ceph-mon[32317]: mon.b calling monitor election 2024-04-23T10:18:01.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:00 smithi183 ceph-mon[32317]: mon.c calling monitor election 2024-04-23T10:18:01.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:00 smithi183 ceph-mon[32317]: mon.a calling monitor election 2024-04-23T10:18:01.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:00 smithi183 ceph-mon[32317]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-23T10:18:01.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:00 smithi183 ceph-mon[32317]: monmap epoch 3 2024-04-23T10:18:01.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:00 smithi183 ceph-mon[32317]: fsid 9b22af88-0159-11ef-bc93-c7b262605968 2024-04-23T10:18:01.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:00 smithi183 ceph-mon[32317]: last_changed 2024-04-23T10:17:53.204963+0000 2024-04-23T10:18:01.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:00 smithi183 ceph-mon[32317]: created 2024-04-23T10:16:03.064570+0000 2024-04-23T10:18:01.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:00 smithi183 ceph-mon[32317]: min_mon_release 19 (squid) 2024-04-23T10:18:01.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:00 smithi183 ceph-mon[32317]: election_strategy: 1 2024-04-23T10:18:01.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:00 smithi183 ceph-mon[32317]: 0: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.a 2024-04-23T10:18:01.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:00 smithi183 ceph-mon[32317]: 1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.c 2024-04-23T10:18:01.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:00 smithi183 ceph-mon[32317]: 2: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.b 2024-04-23T10:18:01.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:00 smithi183 ceph-mon[32317]: fsmap 2024-04-23T10:18:01.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:00 smithi183 ceph-mon[32317]: osdmap e4: 0 total, 0 up, 0 in 2024-04-23T10:18:01.152 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:00 smithi183 ceph-mon[32317]: mgrmap e13: x(active, since 60s) 2024-04-23T10:18:01.152 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:00 smithi183 ceph-mon[32317]: overall HEALTH_OK 2024-04-23T10:18:01.152 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:00 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:01.152 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:00 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:01.152 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:00 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:18:01.152 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:00 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:18:01.152 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:00 smithi183 ceph-mon[32317]: Updating smithi097:/etc/ceph/ceph.conf 2024-04-23T10:18:01.152 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:00 smithi183 ceph-mon[32317]: Updating smithi183:/etc/ceph/ceph.conf 2024-04-23T10:18:01.152 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:00 smithi183 ceph-mon[32317]: Updating smithi196:/etc/ceph/ceph.conf 2024-04-23T10:18:01.152 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:00 smithi183 ceph-mon[32317]: Updating smithi097:/var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T10:18:01.152 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:00 smithi183 ceph-mon[32317]: Updating smithi196:/var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T10:18:01.152 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:00 smithi183 ceph-mon[32317]: Updating smithi183:/var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T10:18:01.152 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:00 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:01.152 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:00 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:01.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:01.837+0000 7f3e4980f700 1 -- 172.21.15.97:0/2143094389 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e44103610 msgr2=0x7f3e441039f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:18:01.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:01.837+0000 7f3e4980f700 1 --2- 172.21.15.97:0/2143094389 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e44103610 0x7f3e441039f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3e34009910 tx=0x7f3e34009c20 comp rx=0 tx=0).stop 2024-04-23T10:18:01.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:01.837+0000 7f3e4980f700 1 -- 172.21.15.97:0/2143094389 shutdown_connections 2024-04-23T10:18:01.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:01.837+0000 7f3e4980f700 1 --2- 172.21.15.97:0/2143094389 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e44103610 0x7f3e441039f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:18:01.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:01.837+0000 7f3e4980f700 1 -- 172.21.15.97:0/2143094389 >> 172.21.15.97:0/2143094389 conn(0x7f3e440fce60 msgr2=0x7f3e440ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:18:01.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:01.837+0000 7f3e4980f700 1 -- 172.21.15.97:0/2143094389 shutdown_connections 2024-04-23T10:18:01.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:01.837+0000 7f3e4980f700 1 -- 172.21.15.97:0/2143094389 wait complete. 2024-04-23T10:18:01.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:01.838+0000 7f3e4980f700 1 Processor -- start 2024-04-23T10:18:01.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:01.838+0000 7f3e4980f700 1 -- start start 2024-04-23T10:18:01.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:01.838+0000 7f3e4980f700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e44103610 0x7f3e441981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:18:01.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:01.838+0000 7f3e4980f700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e441986e0 0x7f3e4419ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:18:01.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:01.838+0000 7f3e4980f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e4419cfb0 0x7f3e4419d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:18:01.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:01.838+0000 7f3e4980f700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4410c9b0 con 0x7f3e4419cfb0 2024-04-23T10:18:01.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:01.838+0000 7f3e4980f700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4410c830 con 0x7f3e441986e0 2024-04-23T10:18:01.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:01.838+0000 7f3e4980f700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4410cb30 con 0x7f3e44103610 2024-04-23T10:18:01.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:01.838+0000 7f3e437fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e4419cfb0 0x7f3e4419d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:18:01.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:01.838+0000 7f3e437fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e4419cfb0 0x7f3e4419d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57004/0 (socket says 172.21.15.97:57004) 2024-04-23T10:18:01.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:01.838+0000 7f3e437fe700 1 -- 172.21.15.97:0/3685593824 learned_addr learned my addr 172.21.15.97:0/3685593824 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:18:01.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:01.839+0000 7f3e437fe700 1 -- 172.21.15.97:0/3685593824 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e44103610 msgr2=0x7f3e441981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:18:01.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:01.839+0000 7f3e437fe700 1 --2- 172.21.15.97:0/3685593824 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e44103610 0x7f3e441981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:18:01.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:01.839+0000 7f3e427fc700 1 --2- 172.21.15.97:0/3685593824 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e441986e0 0x7f3e4419ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:18:01.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:01.839+0000 7f3e437fe700 1 -- 172.21.15.97:0/3685593824 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e441986e0 msgr2=0x7f3e4419ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:18:01.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:01.839+0000 7f3e437fe700 1 --2- 172.21.15.97:0/3685593824 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e441986e0 0x7f3e4419ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:18:01.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:01.839+0000 7f3e437fe700 1 -- 172.21.15.97:0/3685593824 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e441a4d20 con 0x7f3e4419cfb0 2024-04-23T10:18:01.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:01.839+0000 7f3e427fc700 1 --2- 172.21.15.97:0/3685593824 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e441986e0 0x7f3e4419ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:18:01.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:01.839+0000 7f3e437fe700 1 --2- 172.21.15.97:0/3685593824 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e4419cfb0 0x7f3e4419d460 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c00ba50 tx=0x7f3e3c00be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:18:01.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:01.839+0000 7f3e4880d700 1 -- 172.21.15.97:0/3685593824 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c019070 con 0x7f3e4419cfb0 2024-04-23T10:18:01.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:01.839+0000 7f3e4880d700 1 -- 172.21.15.97:0/3685593824 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e3c014440 con 0x7f3e4419cfb0 2024-04-23T10:18:01.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:01.839+0000 7f3e4980f700 1 -- 172.21.15.97:0/3685593824 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e441a4f50 con 0x7f3e4419cfb0 2024-04-23T10:18:01.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:01.839+0000 7f3e4880d700 1 -- 172.21.15.97:0/3685593824 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c007590 con 0x7f3e4419cfb0 2024-04-23T10:18:01.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:01.839+0000 7f3e4980f700 1 -- 172.21.15.97:0/3685593824 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e441a5450 con 0x7f3e4419cfb0 2024-04-23T10:18:01.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:01.840+0000 7f3e4880d700 1 -- 172.21.15.97:0/3685593824 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f3e3c01c070 con 0x7f3e4419cfb0 2024-04-23T10:18:01.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:01.840+0000 7f3e4880d700 1 --2- 172.21.15.97:0/3685593824 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f3e2c039ae0 0x7f3e2c03bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:18:01.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:01.840+0000 7f3e4980f700 1 -- 172.21.15.97:0/3685593824 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e440640a0 con 0x7f3e4419cfb0 2024-04-23T10:18:01.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:01.840+0000 7f3e4880d700 1 -- 172.21.15.97:0/3685593824 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f3e3c04be50 con 0x7f3e4419cfb0 2024-04-23T10:18:01.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:01.840+0000 7f3e42ffd700 1 --2- 172.21.15.97:0/3685593824 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f3e2c039ae0 0x7f3e2c03bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:18:01.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:01.841+0000 7f3e42ffd700 1 --2- 172.21.15.97:0/3685593824 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f3e2c039ae0 0x7f3e2c03bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3e3400b5c0 tx=0x7f3e34005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:18:01.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:01.845+0000 7f3e4880d700 1 -- 172.21.15.97:0/3685593824 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3e3c014910 con 0x7f3e4419cfb0 2024-04-23T10:18:01.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:01 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:01.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:01 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:01.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:01 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:01.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:01 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:01.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:01 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:01.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:01 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:01.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:01 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:01.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:01 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:01.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:01 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:01.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:01 smithi097 ceph-mon[29197]: Reconfiguring mon.a (unknown last config time)... 2024-04-23T10:18:01.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:01 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T10:18:01.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:01 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-23T10:18:01.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:01 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:18:01.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:01 smithi097 ceph-mon[29197]: Reconfiguring daemon mon.a on smithi097 2024-04-23T10:18:01.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:01 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T10:18:01.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:01 smithi097 ceph-mon[29197]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:18:02.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:02.047+0000 7f3e4980f700 1 -- 172.21.15.97:0/3685593824 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f3e441a57d0 con 0x7f3e4419cfb0 2024-04-23T10:18:02.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:02.047+0000 7f3e4880d700 1 -- 172.21.15.97:0/3685593824 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7f3e3c00e090 con 0x7f3e4419cfb0 2024-04-23T10:18:02.047 INFO:teuthology.orchestra.run.smithi097.stdout:# minimal ceph.conf for 9b22af88-0159-11ef-bc93-c7b262605968 2024-04-23T10:18:02.047 INFO:teuthology.orchestra.run.smithi097.stdout:[global] 2024-04-23T10:18:02.047 INFO:teuthology.orchestra.run.smithi097.stdout: fsid = 9b22af88-0159-11ef-bc93-c7b262605968 2024-04-23T10:18:02.047 INFO:teuthology.orchestra.run.smithi097.stdout: mon_host = [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] 2024-04-23T10:18:02.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:02.049+0000 7f3e4980f700 1 -- 172.21.15.97:0/3685593824 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f3e2c039ae0 msgr2=0x7f3e2c03bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:18:02.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:02.049+0000 7f3e4980f700 1 --2- 172.21.15.97:0/3685593824 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f3e2c039ae0 0x7f3e2c03bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3e3400b5c0 tx=0x7f3e34005da0 comp rx=0 tx=0).stop 2024-04-23T10:18:02.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:02.049+0000 7f3e4980f700 1 -- 172.21.15.97:0/3685593824 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e4419cfb0 msgr2=0x7f3e4419d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:18:02.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:02.049+0000 7f3e4980f700 1 --2- 172.21.15.97:0/3685593824 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e4419cfb0 0x7f3e4419d460 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c00ba50 tx=0x7f3e3c00be10 comp rx=0 tx=0).stop 2024-04-23T10:18:02.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:02.049+0000 7f3e4980f700 1 -- 172.21.15.97:0/3685593824 shutdown_connections 2024-04-23T10:18:02.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:02.049+0000 7f3e4980f700 1 --2- 172.21.15.97:0/3685593824 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f3e2c039ae0 0x7f3e2c03bfa0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:18:02.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:02.049+0000 7f3e4980f700 1 --2- 172.21.15.97:0/3685593824 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e4419cfb0 0x7f3e4419d460 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:18:02.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:02.049+0000 7f3e4980f700 1 --2- 172.21.15.97:0/3685593824 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e44103610 0x7f3e441981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:18:02.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:02.049+0000 7f3e4980f700 1 --2- 172.21.15.97:0/3685593824 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e441986e0 0x7f3e4419ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:18:02.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:02.049+0000 7f3e4980f700 1 -- 172.21.15.97:0/3685593824 >> 172.21.15.97:0/3685593824 conn(0x7f3e440fce60 msgr2=0x7f3e441063d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:18:02.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:02.049+0000 7f3e4980f700 1 -- 172.21.15.97:0/3685593824 shutdown_connections 2024-04-23T10:18:02.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:02.049+0000 7f3e4980f700 1 -- 172.21.15.97:0/3685593824 wait complete. 2024-04-23T10:18:02.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:01 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:02.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:01 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:02.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:01 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:02.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:01 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:02.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:01 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:02.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:01 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:02.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:01 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:02.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:01 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:02.093 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:01 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:02.093 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:01 smithi196 ceph-mon[32919]: Reconfiguring mon.a (unknown last config time)... 2024-04-23T10:18:02.093 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:01 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T10:18:02.093 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:01 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-23T10:18:02.093 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:01 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:18:02.093 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:01 smithi196 ceph-mon[32919]: Reconfiguring daemon mon.a on smithi097 2024-04-23T10:18:02.093 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:01 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T10:18:02.093 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:01 smithi196 ceph-mon[32919]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:18:02.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:01 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:02.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:01 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:02.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:01 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:02.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:01 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:02.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:01 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:02.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:01 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:02.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:01 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:02.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:01 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:02.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:01 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:02.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:01 smithi183 ceph-mon[32317]: Reconfiguring mon.a (unknown last config time)... 2024-04-23T10:18:02.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:01 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T10:18:02.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:01 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-23T10:18:02.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:01 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:18:02.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:01 smithi183 ceph-mon[32317]: Reconfiguring daemon mon.a on smithi097 2024-04-23T10:18:02.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:01 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T10:18:02.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:01 smithi183 ceph-mon[32317]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:18:02.729 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-23T10:18:02.729 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-23T10:18:02.729 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/ceph/ceph.conf 2024-04-23T10:18:02.756 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-23T10:18:02.756 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-23T10:18:02.820 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-23T10:18:02.820 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/ceph/ceph.conf 2024-04-23T10:18:02.850 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-23T10:18:02.850 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-23T10:18:02.917 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-23T10:18:02.917 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/ceph/ceph.conf 2024-04-23T10:18:02.948 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-23T10:18:02.948 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-23T10:18:02.993 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:02 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/3685593824' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:18:03.016 INFO:tasks.cephadm:Adding mgr.x on smithi097 2024-04-23T10:18:03.016 INFO:tasks.cephadm:Adding mgr.y on smithi183 2024-04-23T10:18:03.016 INFO:tasks.cephadm:Adding mgr.z on smithi196 2024-04-23T10:18:03.016 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi097=x;smithi183=y;smithi196=z' 2024-04-23T10:18:03.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:02 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/3685593824' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:18:03.290 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T10:18:03.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:02 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/3685593824' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:18:04.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:03 smithi196 ceph-mon[32919]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:18:04.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:03 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:04.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:03 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:04.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:03 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T10:18:04.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:03 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-23T10:18:04.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:03 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:18:04.292 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:03 smithi183 ceph-mon[32317]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:18:04.292 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:03 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:04.293 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:03 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:04.293 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:03 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T10:18:04.293 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:03 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-23T10:18:04.293 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:03 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:18:04.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:03 smithi097 ceph-mon[29197]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:18:04.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:03 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:04.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:03 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:04.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:03 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T10:18:04.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:03 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-23T10:18:04.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:03 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:18:04.774 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:18:04.774+0000 7f844e077700 1 -- 172.21.15.196:0/2397528947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84481025c0 msgr2=0x7f84481029e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:18:04.774 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:18:04.774+0000 7f844e077700 1 --2- 172.21.15.196:0/2397528947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84481025c0 0x7f84481029e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8438009910 tx=0x7f8438009c20 comp rx=0 tx=0).stop 2024-04-23T10:18:04.774 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:18:04.774+0000 7f844e077700 1 -- 172.21.15.196:0/2397528947 shutdown_connections 2024-04-23T10:18:04.774 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:18:04.774+0000 7f844e077700 1 --2- 172.21.15.196:0/2397528947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84481025c0 0x7f84481029e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:18:04.774 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:18:04.774+0000 7f844e077700 1 -- 172.21.15.196:0/2397528947 >> 172.21.15.196:0/2397528947 conn(0x7f84480fdb60 msgr2=0x7f84480fffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:18:04.774 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:18:04.774+0000 7f844e077700 1 -- 172.21.15.196:0/2397528947 shutdown_connections 2024-04-23T10:18:04.774 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:18:04.774+0000 7f844e077700 1 -- 172.21.15.196:0/2397528947 wait complete. 2024-04-23T10:18:04.777 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:18:04.775+0000 7f844e077700 1 Processor -- start 2024-04-23T10:18:04.777 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:18:04.775+0000 7f844e077700 1 -- start start 2024-04-23T10:18:04.777 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:18:04.776+0000 7f844e077700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84481025c0 0x7f8448197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:18:04.777 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:18:04.776+0000 7f844e077700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8448198280 0x7f844819d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:18:04.777 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:18:04.776+0000 7f844e077700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8448198700 0x7f844819f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:18:04.777 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:18:04.776+0000 7f844e077700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8448108500 con 0x7f8448198280 2024-04-23T10:18:04.777 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:18:04.776+0000 7f844e077700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8448108380 con 0x7f8448198700 2024-04-23T10:18:04.778 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:18:04.776+0000 7f844e077700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8448108680 con 0x7f84481025c0 2024-04-23T10:18:04.778 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:18:04.776+0000 7f84477fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84481025c0 0x7f8448197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:18:04.778 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:18:04.776+0000 7f84477fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84481025c0 0x7f8448197d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:45394/0 (socket says 172.21.15.196:45394) 2024-04-23T10:18:04.778 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:18:04.776+0000 7f84477fe700 1 -- 172.21.15.196:0/4158004678 learned_addr learned my addr 172.21.15.196:0/4158004678 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-23T10:18:04.778 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:18:04.776+0000 7f843effd700 1 --2- 172.21.15.196:0/4158004678 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8448198280 0x7f844819d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:18:04.778 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:18:04.776+0000 7f8447fff700 1 --2- 172.21.15.196:0/4158004678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8448198700 0x7f844819f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:18:04.778 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:18:04.776+0000 7f84477fe700 1 -- 172.21.15.196:0/4158004678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8448198700 msgr2=0x7f844819f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:18:04.778 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:18:04.776+0000 7f84477fe700 1 --2- 172.21.15.196:0/4158004678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8448198700 0x7f844819f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:18:04.778 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:18:04.776+0000 7f84477fe700 1 -- 172.21.15.196:0/4158004678 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8448198280 msgr2=0x7f844819d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:18:04.778 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:18:04.776+0000 7f84477fe700 1 --2- 172.21.15.196:0/4158004678 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8448198280 0x7f844819d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:18:04.778 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:18:04.776+0000 7f84477fe700 1 -- 172.21.15.196:0/4158004678 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84481a0160 con 0x7f84481025c0 2024-04-23T10:18:04.778 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:18:04.776+0000 7f8447fff700 1 --2- 172.21.15.196:0/4158004678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8448198700 0x7f844819f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:18:04.778 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:18:04.776+0000 7f84477fe700 1 --2- 172.21.15.196:0/4158004678 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84481025c0 0x7f8448197d40 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f8438005e00 tx=0x7f8438005020 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:18:04.779 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:18:04.776+0000 7f84457fa700 1 -- 172.21.15.196:0/4158004678 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8438027070 con 0x7f84481025c0 2024-04-23T10:18:04.779 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:18:04.776+0000 7f84457fa700 1 -- 172.21.15.196:0/4158004678 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f843800be10 con 0x7f84481025c0 2024-04-23T10:18:04.779 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:18:04.776+0000 7f844e077700 1 -- 172.21.15.196:0/4158004678 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84481a03f0 con 0x7f84481025c0 2024-04-23T10:18:04.779 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:18:04.776+0000 7f84457fa700 1 -- 172.21.15.196:0/4158004678 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8438022da0 con 0x7f84481025c0 2024-04-23T10:18:04.779 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:18:04.776+0000 7f844e077700 1 -- 172.21.15.196:0/4158004678 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84481a09a0 con 0x7f84481025c0 2024-04-23T10:18:04.779 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:18:04.777+0000 7f84457fa700 1 -- 172.21.15.196:0/4158004678 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f843802a070 con 0x7f84481025c0 2024-04-23T10:18:04.779 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:18:04.777+0000 7f84457fa700 1 --2- 172.21.15.196:0/4158004678 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f8434039ae0 0x7f843403bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:18:04.779 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:18:04.777+0000 7f84457fa700 1 -- 172.21.15.196:0/4158004678 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f843804ddd0 con 0x7f84481025c0 2024-04-23T10:18:04.779 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:18:04.777+0000 7f844e077700 1 -- 172.21.15.196:0/4158004678 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84480640a0 con 0x7f84481025c0 2024-04-23T10:18:04.779 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:18:04.778+0000 7f843effd700 1 --2- 172.21.15.196:0/4158004678 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f8434039ae0 0x7f843403bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:18:04.779 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:18:04.778+0000 7f843effd700 1 --2- 172.21.15.196:0/4158004678 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f8434039ae0 0x7f843403bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8430009ad0 tx=0x7f8430009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:18:04.782 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:18:04.783+0000 7f84457fa700 1 -- 172.21.15.196:0/4158004678 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8438031070 con 0x7f84481025c0 2024-04-23T10:18:04.990 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:18:04.991+0000 7f844e077700 1 -- 172.21.15.196:0/4158004678 --> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi097=x;smithi183=y;smithi196=z", "target": ["mon-mgr", ""]}) -- 0x7f8448062e80 con 0x7f8434039ae0 2024-04-23T10:18:04.999 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:18:05.000+0000 7f84457fa700 1 -- 172.21.15.196:0/4158004678 <== mgr.14152 v2:172.21.15.97:6800/3149043749 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f8448062e80 con 0x7f8434039ae0 2024-04-23T10:18:04.999 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled mgr update... 2024-04-23T10:18:05.001 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:18:05.002+0000 7f844e077700 1 -- 172.21.15.196:0/4158004678 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f8434039ae0 msgr2=0x7f843403bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:18:05.001 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:18:05.002+0000 7f844e077700 1 --2- 172.21.15.196:0/4158004678 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f8434039ae0 0x7f843403bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8430009ad0 tx=0x7f8430009210 comp rx=0 tx=0).stop 2024-04-23T10:18:05.001 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:18:05.002+0000 7f844e077700 1 -- 172.21.15.196:0/4158004678 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84481025c0 msgr2=0x7f8448197d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:18:05.001 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:18:05.002+0000 7f844e077700 1 --2- 172.21.15.196:0/4158004678 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84481025c0 0x7f8448197d40 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f8438005e00 tx=0x7f8438005020 comp rx=0 tx=0).stop 2024-04-23T10:18:05.001 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:18:05.002+0000 7f844e077700 1 -- 172.21.15.196:0/4158004678 shutdown_connections 2024-04-23T10:18:05.001 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:18:05.002+0000 7f844e077700 1 --2- 172.21.15.196:0/4158004678 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f8434039ae0 0x7f843403bfa0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:18:05.001 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:18:05.002+0000 7f844e077700 1 --2- 172.21.15.196:0/4158004678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8448198700 0x7f844819f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:18:05.001 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:18:05.002+0000 7f844e077700 1 --2- 172.21.15.196:0/4158004678 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84481025c0 0x7f8448197d40 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:18:05.001 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:18:05.002+0000 7f844e077700 1 --2- 172.21.15.196:0/4158004678 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8448198280 0x7f844819d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:18:05.001 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:18:05.002+0000 7f844e077700 1 -- 172.21.15.196:0/4158004678 >> 172.21.15.196:0/4158004678 conn(0x7f84480fdb60 msgr2=0x7f84480fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:18:05.002 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:18:05.002+0000 7f844e077700 1 -- 172.21.15.196:0/4158004678 shutdown_connections 2024-04-23T10:18:05.002 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:18:05.002+0000 7f844e077700 1 -- 172.21.15.196:0/4158004678 wait complete. 2024-04-23T10:18:05.318 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:04 smithi183 ceph-mon[32317]: Reconfiguring mon.b (monmap changed)... 2024-04-23T10:18:05.318 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:04 smithi183 ceph-mon[32317]: Reconfiguring daemon mon.b on smithi183 2024-04-23T10:18:05.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:04 smithi196 ceph-mon[32919]: Reconfiguring mon.b (monmap changed)... 2024-04-23T10:18:05.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:04 smithi196 ceph-mon[32919]: Reconfiguring daemon mon.b on smithi183 2024-04-23T10:18:05.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:04 smithi097 ceph-mon[29197]: Reconfiguring mon.b (monmap changed)... 2024-04-23T10:18:05.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:04 smithi097 ceph-mon[29197]: Reconfiguring daemon mon.b on smithi183 2024-04-23T10:18:05.661 DEBUG:teuthology.orchestra.run.smithi183:mgr.y> sudo journalctl -f -n 0 -u ceph-9b22af88-0159-11ef-bc93-c7b262605968@mgr.y.service 2024-04-23T10:18:05.663 DEBUG:teuthology.orchestra.run.smithi196:mgr.z> sudo journalctl -f -n 0 -u ceph-9b22af88-0159-11ef-bc93-c7b262605968@mgr.z.service 2024-04-23T10:18:05.665 INFO:tasks.cephadm:Deploying OSDs... 2024-04-23T10:18:05.665 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-23T10:18:05.665 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/scratch_devs of=/dev/stdout 2024-04-23T10:18:05.684 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-23T10:18:05.685 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_1 2024-04-23T10:18:05.742 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-23T10:18:05.742 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T10:18:05.742 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-23T10:18:05.742 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T10:18:05.742 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T10:18:05.743 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-04-23 10:17:10.155292569 +0000 2024-04-23T10:18:05.743 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-04-23 10:03:38.982375942 +0000 2024-04-23T10:18:05.743 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-04-23 10:03:38.982375942 +0000 2024-04-23T10:18:05.743 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-04-23 10:03:38.982375942 +0000 2024-04-23T10:18:05.743 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-23T10:18:05.810 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-23T10:18:05.810 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-23T10:18:05.810 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000247943 s, 2.1 MB/s 2024-04-23T10:18:05.811 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-23T10:18:05.869 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_2 2024-04-23T10:18:05.925 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-23T10:18:05.925 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T10:18:05.925 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-23T10:18:05.925 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T10:18:05.925 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T10:18:05.925 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-04-23 10:17:10.182292073 +0000 2024-04-23T10:18:05.925 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-04-23 10:03:39.338369480 +0000 2024-04-23T10:18:05.925 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-04-23 10:03:39.338369480 +0000 2024-04-23T10:18:05.925 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-04-23 10:03:39.338369480 +0000 2024-04-23T10:18:05.926 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-23T10:18:05.989 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-23T10:18:05.989 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-23T10:18:05.989 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000212987 s, 2.4 MB/s 2024-04-23T10:18:05.990 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-23T10:18:06.048 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_3 2024-04-23T10:18:06.103 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-23T10:18:06.103 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T10:18:06.103 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-23T10:18:06.103 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T10:18:06.103 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T10:18:06.103 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-04-23 10:17:10.207291614 +0000 2024-04-23T10:18:06.103 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-04-23 10:03:39.678363310 +0000 2024-04-23T10:18:06.103 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-04-23 10:03:39.678363310 +0000 2024-04-23T10:18:06.104 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-04-23 10:03:39.678363310 +0000 2024-04-23T10:18:06.104 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-23T10:18:06.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:06 smithi097 ceph-mon[29197]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi097=x;smithi183=y;smithi196=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:18:06.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:06 smithi097 ceph-mon[29197]: Saving service mgr spec with placement smithi097=x;smithi183=y;smithi196=z;count:3 2024-04-23T10:18:06.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:06 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:06.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:06 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:06.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:06 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:06.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:06 smithi097 ceph-mon[29197]: Reconfiguring mon.c (monmap changed)... 2024-04-23T10:18:06.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:06 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T10:18:06.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:06 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-23T10:18:06.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:06 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:18:06.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:06 smithi097 ceph-mon[29197]: Reconfiguring daemon mon.c on smithi196 2024-04-23T10:18:06.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:06 smithi097 ceph-mon[29197]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:18:06.133 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-23T10:18:06.133 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-23T10:18:06.133 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000230224 s, 2.2 MB/s 2024-04-23T10:18:06.134 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-23T10:18:06.190 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_4 2024-04-23T10:18:06.245 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-23T10:18:06.245 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T10:18:06.245 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-23T10:18:06.245 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T10:18:06.245 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T10:18:06.245 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-04-23 10:17:10.231291173 +0000 2024-04-23T10:18:06.246 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-04-23 10:03:40.060356377 +0000 2024-04-23T10:18:06.246 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-04-23 10:03:40.060356377 +0000 2024-04-23T10:18:06.246 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-04-23 10:03:40.060356377 +0000 2024-04-23T10:18:06.246 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-23T10:18:06.308 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-23T10:18:06.308 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-23T10:18:06.308 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000218021 s, 2.3 MB/s 2024-04-23T10:18:06.309 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-23T10:18:06.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:06 smithi196 ceph-mon[32919]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi097=x;smithi183=y;smithi196=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:18:06.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:06 smithi196 ceph-mon[32919]: Saving service mgr spec with placement smithi097=x;smithi183=y;smithi196=z;count:3 2024-04-23T10:18:06.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:06 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:06.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:06 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:06.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:06 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:06.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:06 smithi196 ceph-mon[32919]: Reconfiguring mon.c (monmap changed)... 2024-04-23T10:18:06.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:06 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T10:18:06.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:06 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-23T10:18:06.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:06 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:18:06.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:06 smithi196 ceph-mon[32919]: Reconfiguring daemon mon.c on smithi196 2024-04-23T10:18:06.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:06 smithi196 ceph-mon[32919]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:18:06.366 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-04-23T10:18:06.366 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/scratch_devs of=/dev/stdout 2024-04-23T10:18:06.385 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-23T10:18:06.385 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_1 2024-04-23T10:18:06.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:06 smithi183 ceph-mon[32317]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi097=x;smithi183=y;smithi196=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:18:06.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:06 smithi183 ceph-mon[32317]: Saving service mgr spec with placement smithi097=x;smithi183=y;smithi196=z;count:3 2024-04-23T10:18:06.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:06 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:06.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:06 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:06.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:06 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:06.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:06 smithi183 ceph-mon[32317]: Reconfiguring mon.c (monmap changed)... 2024-04-23T10:18:06.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:06 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T10:18:06.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:06 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-23T10:18:06.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:06 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:18:06.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:06 smithi183 ceph-mon[32317]: Reconfiguring daemon mon.c on smithi196 2024-04-23T10:18:06.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:06 smithi183 ceph-mon[32317]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:18:06.417 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-23T10:18:06.417 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T10:18:06.417 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-23T10:18:06.417 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T10:18:06.417 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T10:18:06.417 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-04-23 10:17:52.640855997 +0000 2024-04-23T10:18:06.417 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-04-23 10:03:58.578290805 +0000 2024-04-23T10:18:06.417 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-04-23 10:03:58.578290805 +0000 2024-04-23T10:18:06.417 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-04-23 10:03:58.578290805 +0000 2024-04-23T10:18:06.418 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-23T10:18:06.482 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-04-23T10:18:06.482 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-04-23T10:18:06.483 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000261781 s, 2.0 MB/s 2024-04-23T10:18:06.484 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-23T10:18:06.541 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_2 2024-04-23T10:18:06.598 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-23T10:18:06.598 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T10:18:06.598 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-23T10:18:06.598 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T10:18:06.598 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T10:18:06.598 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-04-23 10:17:52.641855970 +0000 2024-04-23T10:18:06.598 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-04-23 10:03:58.956280531 +0000 2024-04-23T10:18:06.599 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-04-23 10:03:58.956280531 +0000 2024-04-23T10:18:06.599 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-04-23 10:03:58.956280531 +0000 2024-04-23T10:18:06.599 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-23T10:18:06.662 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-04-23T10:18:06.662 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-04-23T10:18:06.662 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000227365 s, 2.3 MB/s 2024-04-23T10:18:06.663 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-23T10:18:06.719 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_3 2024-04-23T10:18:06.775 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-23T10:18:06.775 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T10:18:06.775 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-04-23T10:18:06.775 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T10:18:06.775 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T10:18:06.775 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-04-23 10:17:52.641855970 +0000 2024-04-23T10:18:06.775 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-04-23 10:03:59.285271588 +0000 2024-04-23T10:18:06.775 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-04-23 10:03:59.285271588 +0000 2024-04-23T10:18:06.776 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-04-23 10:03:59.285271588 +0000 2024-04-23T10:18:06.776 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-23T10:18:06.837 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-04-23T10:18:06.837 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-04-23T10:18:06.837 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000190697 s, 2.7 MB/s 2024-04-23T10:18:06.838 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-23T10:18:06.894 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_4 2024-04-23T10:18:06.948 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-23T10:18:06.949 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T10:18:06.949 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-23T10:18:06.949 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T10:18:06.949 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T10:18:06.949 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-04-23 10:17:52.641855970 +0000 2024-04-23T10:18:06.949 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-04-23 10:03:59.623262401 +0000 2024-04-23T10:18:06.949 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-04-23 10:03:59.623262401 +0000 2024-04-23T10:18:06.949 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-04-23 10:03:59.623262401 +0000 2024-04-23T10:18:06.949 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-23T10:18:07.011 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-04-23T10:18:07.011 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-04-23T10:18:07.011 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.00016959 s, 3.0 MB/s 2024-04-23T10:18:07.012 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-23T10:18:07.067 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-23T10:18:07.067 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/scratch_devs of=/dev/stdout 2024-04-23T10:18:07.084 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-23T10:18:07.084 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_1 2024-04-23T10:18:07.142 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-23T10:18:07.142 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T10:18:07.142 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-04-23T10:18:07.142 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T10:18:07.142 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T10:18:07.142 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-04-23 10:17:43.365188620 +0000 2024-04-23T10:18:07.142 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-04-23 10:04:03.210997504 +0000 2024-04-23T10:18:07.142 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-04-23 10:04:03.210997504 +0000 2024-04-23T10:18:07.143 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-04-23 10:04:03.210997504 +0000 2024-04-23T10:18:07.143 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-23T10:18:07.204 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-04-23T10:18:07.205 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-04-23T10:18:07.205 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000274896 s, 1.9 MB/s 2024-04-23T10:18:07.206 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-23T10:18:07.261 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_2 2024-04-23T10:18:07.318 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-23T10:18:07.318 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T10:18:07.318 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-23T10:18:07.318 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T10:18:07.318 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T10:18:07.318 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-04-23 10:17:43.365188620 +0000 2024-04-23T10:18:07.318 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-04-23 10:04:03.570989410 +0000 2024-04-23T10:18:07.318 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-04-23 10:04:03.570989410 +0000 2024-04-23T10:18:07.318 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-04-23 10:04:03.570989410 +0000 2024-04-23T10:18:07.319 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-23T10:18:07.380 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-04-23T10:18:07.380 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-04-23T10:18:07.380 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000185904 s, 2.8 MB/s 2024-04-23T10:18:07.381 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-23T10:18:07.438 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_3 2024-04-23T10:18:07.493 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-23T10:18:07.494 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T10:18:07.494 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-23T10:18:07.494 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T10:18:07.494 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T10:18:07.494 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-04-23 10:17:43.366188597 +0000 2024-04-23T10:18:07.494 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-04-23 10:04:03.899982015 +0000 2024-04-23T10:18:07.494 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-04-23 10:04:03.899982015 +0000 2024-04-23T10:18:07.494 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-04-23 10:04:03.899982015 +0000 2024-04-23T10:18:07.494 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-23T10:18:07.557 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-04-23T10:18:07.557 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-04-23T10:18:07.557 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000177437 s, 2.9 MB/s 2024-04-23T10:18:07.558 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-23T10:18:07.615 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_4 2024-04-23T10:18:07.670 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-23T10:18:07.670 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T10:18:07.670 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-23T10:18:07.670 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T10:18:07.670 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T10:18:07.670 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-04-23 10:17:43.366188597 +0000 2024-04-23T10:18:07.670 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-04-23 10:04:04.247974193 +0000 2024-04-23T10:18:07.670 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-04-23 10:04:04.247974193 +0000 2024-04-23T10:18:07.671 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-04-23 10:04:04.247974193 +0000 2024-04-23T10:18:07.671 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-23T10:18:07.732 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-04-23T10:18:07.732 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-04-23T10:18:07.732 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000208686 s, 2.5 MB/s 2024-04-23T10:18:07.734 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-23T10:18:07.790 INFO:tasks.cephadm:Deploying osd.0 on smithi097 with /dev/vg_nvme/lv_4... 2024-04-23T10:18:07.790 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-23T10:18:07.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:07 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:07.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:07 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:07.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:07 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:18:07.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:07 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:18:07.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:07 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:18:07.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:07 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:07.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:07 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-23T10:18:07.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:07 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-23T10:18:07.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:07 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-23T10:18:07.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:07 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:18:07.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:07 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:07.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:07 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:07.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:07 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:18:07.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:07 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:18:07.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:07 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:18:07.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:07 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:07.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:07 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-23T10:18:07.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:07 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-23T10:18:07.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:07 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-23T10:18:07.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:07 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:18:07.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:07 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:07.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:07 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:07.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:07 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:18:07.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:07 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:18:07.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:07 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:18:07.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:07 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:07.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:07 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-23T10:18:07.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:07 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-23T10:18:07.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:07 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-23T10:18:07.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:07 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:18:08.022 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T10:18:08.835 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:08 smithi196 ceph-mon[32919]: Deploying daemon mgr.z on smithi196 2024-04-23T10:18:08.835 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:08 smithi196 ceph-mon[32919]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:18:08.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:08 smithi097 ceph-mon[29197]: Deploying daemon mgr.z on smithi196 2024-04-23T10:18:08.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:08 smithi097 ceph-mon[29197]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:18:08.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:08 smithi183 ceph-mon[32317]: Deploying daemon mgr.z on smithi196 2024-04-23T10:18:08.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:08 smithi183 ceph-mon[32317]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:18:09.092 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 23 10:18:09 smithi196 systemd[1]: Starting Ceph mgr.z for 9b22af88-0159-11ef-bc93-c7b262605968... 2024-04-23T10:18:09.352 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 23 10:18:09 smithi196 podman[34755]: 2024-04-23 10:18:09.1239102 +0000 UTC m=+0.017440677 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T10:18:09.352 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 23 10:18:09 smithi196 podman[34755]: 2024-04-23 10:18:09.231113918 +0000 UTC m=+0.124644455 container create 488569418dfaa1f1067b1d681631ea167d2f6c1a66784d8a5825ced715c1f656 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-z, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-23T10:18:09.776 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 23 10:18:09 smithi196 podman[34755]: 2024-04-23 10:18:09.433538796 +0000 UTC m=+0.327069267 container init 488569418dfaa1f1067b1d681631ea167d2f6c1a66784d8a5825ced715c1f656 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T10:18:09.776 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 23 10:18:09 smithi196 podman[34755]: 2024-04-23 10:18:09.436214136 +0000 UTC m=+0.329744612 container start 488569418dfaa1f1067b1d681631ea167d2f6c1a66784d8a5825ced715c1f656 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True) 2024-04-23T10:18:09.776 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 23 10:18:09 smithi196 ceph-mgr[34774]: -- 172.21.15.196:0/2052360762 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ecad8065a0 con 0x55ecad7da800 2024-04-23T10:18:09.776 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 23 10:18:09 smithi196 bash[34755]: 488569418dfaa1f1067b1d681631ea167d2f6c1a66784d8a5825ced715c1f656 2024-04-23T10:18:09.777 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 23 10:18:09 smithi196 systemd[1]: Started Ceph mgr.z for 9b22af88-0159-11ef-bc93-c7b262605968. 2024-04-23T10:18:09.777 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 23 10:18:09 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-z[34770]: 2024-04-23T10:18:09.593+0000 7f236ac90200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-23T10:18:09.777 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 23 10:18:09 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-z[34770]: 2024-04-23T10:18:09.776+0000 7f236ac90200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-23T10:18:09.966 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-23T10:18:09.966 INFO:teuthology.orchestra.run.smithi097.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-23T10:18:09.966 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10+0 records in 2024-04-23T10:18:09.966 INFO:teuthology.orchestra.run.smithi097.stderr:10+0 records out 2024-04-23T10:18:09.966 INFO:teuthology.orchestra.run.smithi097.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112344 s, 933 MB/s 2024-04-23T10:18:09.966 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping successful for: 2024-04-23T10:18:10.092 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 23 10:18:09 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-z[34770]: 2024-04-23T10:18:09.877+0000 7f236ac90200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-23T10:18:10.578 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 23 10:18:10 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-z[34770]: 2024-04-23T10:18:10.180+0000 7f236ac90200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-23T10:18:10.578 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 23 10:18:10 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-z[34770]: 2024-04-23T10:18:10.264+0000 7f236ac90200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-23T10:18:10.603 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi097:vg_nvme/lv_4 2024-04-23T10:18:10.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:10 smithi196 ceph-mon[32919]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:18:10.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:10 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:10.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:10 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:10.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:10 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:10.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:10 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-23T10:18:10.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:10 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-23T10:18:10.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:10 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-23T10:18:10.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:10 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:18:10.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:10 smithi196 ceph-mon[32919]: Deploying daemon mgr.y on smithi183 2024-04-23T10:18:10.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:10 smithi097 ceph-mon[29197]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:18:10.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:10 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:10.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:10 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:10.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:10 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:10.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:10 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-23T10:18:10.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:10 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-23T10:18:10.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:10 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-23T10:18:10.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:10 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:18:10.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:10 smithi097 ceph-mon[29197]: Deploying daemon mgr.y on smithi183 2024-04-23T10:18:10.878 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T10:18:10.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:10 smithi183 ceph-mon[32317]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:18:10.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:10 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:10.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:10 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:10.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:10 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:10.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:10 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-23T10:18:10.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:10 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-23T10:18:10.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:10 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-23T10:18:10.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:10 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:18:10.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:10 smithi183 ceph-mon[32317]: Deploying daemon mgr.y on smithi183 2024-04-23T10:18:11.650 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 23 10:18:11 smithi183 systemd[1]: Starting Ceph mgr.y for 9b22af88-0159-11ef-bc93-c7b262605968... 2024-04-23T10:18:11.796 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 23 10:18:11 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-z[34770]: 2024-04-23T10:18:11.477+0000 7f236ac90200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-23T10:18:11.796 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 23 10:18:11 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-z[34770]: 2024-04-23T10:18:11.648+0000 7f236ac90200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-23T10:18:12.001 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 23 10:18:11 smithi183 podman[33684]: 2024-04-23 10:18:11.634751649 +0000 UTC m=+0.018017139 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T10:18:12.001 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 23 10:18:11 smithi183 podman[33684]: 2024-04-23 10:18:11.759037835 +0000 UTC m=+0.142303309 container create 6ba1874f12255e0c28bd2da29e055498c4ac5618c6a7e86ce53125b66325ac91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-23T10:18:12.001 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 23 10:18:11 smithi183 podman[33684]: 2024-04-23 10:18:11.936220518 +0000 UTC m=+0.319485991 container init 6ba1874f12255e0c28bd2da29e055498c4ac5618c6a7e86ce53125b66325ac91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240416, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-23T10:18:12.001 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 23 10:18:11 smithi183 podman[33684]: 2024-04-23 10:18:11.93865949 +0000 UTC m=+0.321924964 container start 6ba1874f12255e0c28bd2da29e055498c4ac5618c6a7e86ce53125b66325ac91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-23T10:18:12.001 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 23 10:18:11 smithi183 ceph-mgr[33703]: -- 172.21.15.183:0/4063584590 <== mon.2 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a2b383c5a0 con 0x55a2b3811000 2024-04-23T10:18:12.001 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 23 10:18:12 smithi183 bash[33684]: 6ba1874f12255e0c28bd2da29e055498c4ac5618c6a7e86ce53125b66325ac91 2024-04-23T10:18:12.091 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 23 10:18:11 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-z[34770]: 2024-04-23T10:18:11.820+0000 7f236ac90200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-23T10:18:12.295 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 23 10:18:12 smithi183 systemd[1]: Started Ceph mgr.y for 9b22af88-0159-11ef-bc93-c7b262605968. 2024-04-23T10:18:12.295 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 23 10:18:12 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-y[33699]: 2024-04-23T10:18:12.093+0000 7f6ccdd07200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-23T10:18:12.295 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 23 10:18:12 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-y[33699]: 2024-04-23T10:18:12.296+0000 7f6ccdd07200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-23T10:18:12.402 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 23 10:18:12 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-z[34770]: 2024-04-23T10:18:12.102+0000 7f236ac90200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-23T10:18:12.402 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 23 10:18:12 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-z[34770]: 2024-04-23T10:18:12.193+0000 7f236ac90200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-23T10:18:12.402 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 23 10:18:12 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-z[34770]: 2024-04-23T10:18:12.402+0000 7f236ac90200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-23T10:18:12.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:12.450+0000 7fd123ca4700 1 -- 172.21.15.97:0/2719082958 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd11c103d90 msgr2=0x7fd11c1041f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:18:12.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:12.450+0000 7fd123ca4700 1 --2- 172.21.15.97:0/2719082958 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd11c103d90 0x7fd11c1041f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd10c009990 tx=0x7fd10c009ca0 comp rx=0 tx=0).stop 2024-04-23T10:18:12.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:12.450+0000 7fd123ca4700 1 -- 172.21.15.97:0/2719082958 shutdown_connections 2024-04-23T10:18:12.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:12.450+0000 7fd123ca4700 1 --2- 172.21.15.97:0/2719082958 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd11c104730 0x7fd11c10afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:18:12.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:12.450+0000 7fd123ca4700 1 --2- 172.21.15.97:0/2719082958 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd11c100db0 0x7fd11c1011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:18:12.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:12.450+0000 7fd123ca4700 1 --2- 172.21.15.97:0/2719082958 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd11c103d90 0x7fd11c1041f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:18:12.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:12.450+0000 7fd123ca4700 1 -- 172.21.15.97:0/2719082958 >> 172.21.15.97:0/2719082958 conn(0x7fd11c0fc710 msgr2=0x7fd11c0feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:18:12.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:12.451+0000 7fd123ca4700 1 -- 172.21.15.97:0/2719082958 shutdown_connections 2024-04-23T10:18:12.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:12.451+0000 7fd123ca4700 1 -- 172.21.15.97:0/2719082958 wait complete. 2024-04-23T10:18:12.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:12.451+0000 7fd123ca4700 1 Processor -- start 2024-04-23T10:18:12.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:12.451+0000 7fd123ca4700 1 -- start start 2024-04-23T10:18:12.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:12.451+0000 7fd123ca4700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd11c100db0 0x7fd11c198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:18:12.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:12.451+0000 7fd123ca4700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd11c103d90 0x7fd11c199090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:18:12.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:12.451+0000 7fd123ca4700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd11c104730 0x7fd11c1a0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:18:12.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:12.451+0000 7fd123ca4700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd11c10e5a0 con 0x7fd11c100db0 2024-04-23T10:18:12.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:12.451+0000 7fd123ca4700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd11c10e420 con 0x7fd11c104730 2024-04-23T10:18:12.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:12.451+0000 7fd123ca4700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd11c10e720 con 0x7fd11c103d90 2024-04-23T10:18:12.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:12.452+0000 7fd121a40700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd11c100db0 0x7fd11c198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:18:12.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:12.452+0000 7fd121a40700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd11c100db0 0x7fd11c198b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39102/0 (socket says 172.21.15.97:39102) 2024-04-23T10:18:12.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:12.452+0000 7fd121a40700 1 -- 172.21.15.97:0/1497011719 learned_addr learned my addr 172.21.15.97:0/1497011719 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:18:12.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:12.452+0000 7fd12123f700 1 --2- 172.21.15.97:0/1497011719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd11c103d90 0x7fd11c199090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:18:12.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:12.452+0000 7fd122241700 1 --2- 172.21.15.97:0/1497011719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd11c104730 0x7fd11c1a0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:18:12.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:12.452+0000 7fd121a40700 1 -- 172.21.15.97:0/1497011719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd11c103d90 msgr2=0x7fd11c199090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:18:12.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:12.452+0000 7fd121a40700 1 --2- 172.21.15.97:0/1497011719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd11c103d90 0x7fd11c199090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:18:12.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:12.452+0000 7fd121a40700 1 -- 172.21.15.97:0/1497011719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd11c104730 msgr2=0x7fd11c1a0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:18:12.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:12.452+0000 7fd121a40700 1 --2- 172.21.15.97:0/1497011719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd11c104730 0x7fd11c1a0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:18:12.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:12.452+0000 7fd121a40700 1 -- 172.21.15.97:0/1497011719 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd11c1a07b0 con 0x7fd11c100db0 2024-04-23T10:18:12.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:12.452+0000 7fd121a40700 1 --2- 172.21.15.97:0/1497011719 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd11c100db0 0x7fd11c198b50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd11800c9c0 tx=0x7fd11800ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:18:12.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:12.452+0000 7fd112ffd700 1 -- 172.21.15.97:0/1497011719 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd118007ac0 con 0x7fd11c100db0 2024-04-23T10:18:12.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:12.452+0000 7fd112ffd700 1 -- 172.21.15.97:0/1497011719 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd118007c60 con 0x7fd11c100db0 2024-04-23T10:18:12.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:12.452+0000 7fd123ca4700 1 -- 172.21.15.97:0/1497011719 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd11c1a0aa0 con 0x7fd11c100db0 2024-04-23T10:18:12.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:12.452+0000 7fd112ffd700 1 -- 172.21.15.97:0/1497011719 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd118018660 con 0x7fd11c100db0 2024-04-23T10:18:12.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:12.452+0000 7fd123ca4700 1 -- 172.21.15.97:0/1497011719 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd11c1a1030 con 0x7fd11c100db0 2024-04-23T10:18:12.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:12.453+0000 7fd112ffd700 1 -- 172.21.15.97:0/1497011719 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fd118018800 con 0x7fd11c100db0 2024-04-23T10:18:12.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:12.453+0000 7fd123ca4700 1 -- 172.21.15.97:0/1497011719 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd11c069730 con 0x7fd11c100db0 2024-04-23T10:18:12.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:12.453+0000 7fd112ffd700 1 --2- 172.21.15.97:0/1497011719 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fd108039890 0x7fd10803bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:18:12.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:12.453+0000 7fd112ffd700 1 -- 172.21.15.97:0/1497011719 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd11804c950 con 0x7fd11c100db0 2024-04-23T10:18:12.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:12.453+0000 7fd12123f700 1 --2- 172.21.15.97:0/1497011719 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fd108039890 0x7fd10803bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:18:12.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:12.454+0000 7fd12123f700 1 --2- 172.21.15.97:0/1497011719 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fd108039890 0x7fd10803bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd10c00b5c0 tx=0x7fd10c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:18:12.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:12.458+0000 7fd112ffd700 1 -- 172.21.15.97:0/1497011719 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd118014030 con 0x7fd11c100db0 2024-04-23T10:18:12.581 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 23 10:18:12 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-y[33699]: 2024-04-23T10:18:12.405+0000 7f6ccdd07200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-23T10:18:12.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:12.672+0000 7fd123ca4700 1 -- 172.21.15.97:0/1497011719 --> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fd11c062e80 con 0x7fd108039890 2024-04-23T10:18:12.842 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 23 10:18:12 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-z[34770]: 2024-04-23T10:18:12.601+0000 7f236ac90200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-23T10:18:12.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:12 smithi196 ceph-mon[32919]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:18:12.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:12 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:12.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:12 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:12.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:12 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:12.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:12 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:12.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:12 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:18:12.848 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 23 10:18:12 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-y[33699]: 2024-04-23T10:18:12.747+0000 7f6ccdd07200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-23T10:18:12.848 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 23 10:18:12 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-y[33699]: 2024-04-23T10:18:12.849+0000 7f6ccdd07200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-23T10:18:12.848 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:12 smithi183 ceph-mon[32317]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:18:12.849 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:12 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:12.849 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:12 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:12.849 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:12 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:12.849 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:12 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:12.849 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:12 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:18:12.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:12 smithi097 ceph-mon[29197]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:18:12.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:12 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:12.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:12 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:12.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:12 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:12.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:12 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:12.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:12 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:18:13.342 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 23 10:18:13 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-z[34770]: 2024-04-23T10:18:13.233+0000 7f236ac90200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-23T10:18:13.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:13 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T10:18:13.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:13 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T10:18:13.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:13 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:18:13.842 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 23 10:18:13 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-z[34770]: 2024-04-23T10:18:13.666+0000 7f236ac90200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-23T10:18:13.842 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 23 10:18:13 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-z[34770]: 2024-04-23T10:18:13.764+0000 7f236ac90200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-23T10:18:13.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:13 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T10:18:13.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:13 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T10:18:13.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:13 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:18:13.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:13 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T10:18:13.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:13 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T10:18:13.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:13 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:18:14.342 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 23 10:18:13 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-z[34770]: 2024-04-23T10:18:13.966+0000 7f236ac90200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-23T10:18:14.400 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 23 10:18:14 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-y[33699]: 2024-04-23T10:18:14.131+0000 7f6ccdd07200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-23T10:18:14.400 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 23 10:18:14 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-y[33699]: 2024-04-23T10:18:14.300+0000 7f6ccdd07200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-23T10:18:14.758 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 23 10:18:14 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-y[33699]: 2024-04-23T10:18:14.471+0000 7f6ccdd07200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-23T10:18:14.759 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 23 10:18:14 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-y[33699]: 2024-04-23T10:18:14.759+0000 7f6ccdd07200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-23T10:18:14.759 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:14 smithi183 ceph-mon[32317]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:18:14.759 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:14 smithi183 ceph-mon[32317]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:18:14.759 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:14 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:14.759 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:14 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:14.759 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:14 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:14.759 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:14 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:14.759 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:14 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:18:14.759 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:14 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:18:14.759 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:14 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:14.760 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:14 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-23T10:18:14.760 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:14 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-23T10:18:14.760 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:14 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:18:14.760 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:14 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:14.842 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 23 10:18:14 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-z[34770]: 2024-04-23T10:18:14.507+0000 7f236ac90200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-23T10:18:14.842 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 23 10:18:14 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-z[34770]: 2024-04-23T10:18:14.600+0000 7f236ac90200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-23T10:18:14.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:14 smithi196 ceph-mon[32919]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:18:14.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:14 smithi196 ceph-mon[32919]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:18:14.843 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:14 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:14.843 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:14 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:14.843 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:14 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:14.843 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:14 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:14.843 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:14 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:18:14.843 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:14 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:18:14.843 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:14 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:14.843 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:14 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-23T10:18:14.843 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:14 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-23T10:18:14.843 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:14 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:18:14.843 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:14 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:14.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:14 smithi097 ceph-mon[29197]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:18:14.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:14 smithi097 ceph-mon[29197]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:18:14.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:14 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:14.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:14 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:14.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:14 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:14.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:14 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:14.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:14 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:18:14.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:14 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:18:14.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:14 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:14.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:14 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-23T10:18:14.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:14 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-23T10:18:14.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:14 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:18:14.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:14 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:15.054 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 23 10:18:14 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-y[33699]: 2024-04-23T10:18:14.851+0000 7f6ccdd07200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-23T10:18:15.054 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 23 10:18:15 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-y[33699]: 2024-04-23T10:18:15.054+0000 7f6ccdd07200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-23T10:18:15.183 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 23 10:18:14 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-z[34770]: 2024-04-23T10:18:14.843+0000 7f236ac90200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-23T10:18:15.183 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 23 10:18:14 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-z[34770]: 2024-04-23T10:18:14.975+0000 7f236ac90200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-23T10:18:15.183 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 23 10:18:15 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-z[34770]: 2024-04-23T10:18:15.069+0000 7f236ac90200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-23T10:18:15.183 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 23 10:18:15 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-z[34770]: 2024-04-23T10:18:15.184+0000 7f236ac90200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-23T10:18:15.400 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 23 10:18:15 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-y[33699]: 2024-04-23T10:18:15.245+0000 7f6ccdd07200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-23T10:18:15.842 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 23 10:18:15 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-z[34770]: 2024-04-23T10:18:15.487+0000 7f236ac90200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-23T10:18:15.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:15 smithi196 ceph-mon[32919]: Reconfiguring mgr.x (unknown last config time)... 2024-04-23T10:18:15.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:15 smithi196 ceph-mon[32919]: Reconfiguring daemon mgr.x on smithi097 2024-04-23T10:18:15.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:15 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/3562012086' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "58cb995d-de35-4522-ac81-4982401b260d"}]: dispatch 2024-04-23T10:18:15.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:15 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/3562012086' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "58cb995d-de35-4522-ac81-4982401b260d"}]': finished 2024-04-23T10:18:15.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:15 smithi196 ceph-mon[32919]: osdmap e5: 1 total, 0 up, 1 in 2024-04-23T10:18:15.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:15 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T10:18:15.843 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:15 smithi183 ceph-mon[32317]: Reconfiguring mgr.x (unknown last config time)... 2024-04-23T10:18:15.843 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:15 smithi183 ceph-mon[32317]: Reconfiguring daemon mgr.x on smithi097 2024-04-23T10:18:15.844 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:15 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/3562012086' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "58cb995d-de35-4522-ac81-4982401b260d"}]: dispatch 2024-04-23T10:18:15.844 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:15 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/3562012086' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "58cb995d-de35-4522-ac81-4982401b260d"}]': finished 2024-04-23T10:18:15.844 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:15 smithi183 ceph-mon[32317]: osdmap e5: 1 total, 0 up, 1 in 2024-04-23T10:18:15.844 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:15 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T10:18:15.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:15 smithi097 ceph-mon[29197]: Reconfiguring mgr.x (unknown last config time)... 2024-04-23T10:18:15.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:15 smithi097 ceph-mon[29197]: Reconfiguring daemon mgr.x on smithi097 2024-04-23T10:18:15.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:15 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/3562012086' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "58cb995d-de35-4522-ac81-4982401b260d"}]: dispatch 2024-04-23T10:18:15.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:15 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/3562012086' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "58cb995d-de35-4522-ac81-4982401b260d"}]': finished 2024-04-23T10:18:15.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:15 smithi097 ceph-mon[29197]: osdmap e5: 1 total, 0 up, 1 in 2024-04-23T10:18:15.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:15 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T10:18:16.150 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 23 10:18:15 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-y[33699]: 2024-04-23T10:18:15.844+0000 7f6ccdd07200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-23T10:18:16.342 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 23 10:18:16 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-z[34770]: 2024-04-23T10:18:16.085+0000 7f236ac90200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-23T10:18:16.342 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 23 10:18:16 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-z[34770]: 2024-04-23T10:18:16.302+0000 7f236ac90200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-23T10:18:16.557 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 23 10:18:16 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-y[33699]: 2024-04-23T10:18:16.264+0000 7f6ccdd07200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-23T10:18:16.557 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 23 10:18:16 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-y[33699]: 2024-04-23T10:18:16.359+0000 7f6ccdd07200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-23T10:18:16.557 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 23 10:18:16 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-y[33699]: 2024-04-23T10:18:16.557+0000 7f6ccdd07200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-23T10:18:16.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:16.599+0000 7fd112ffd700 1 -- 172.21.15.97:0/1497011719 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mgrmap(e 14) ==== 92602+0+0 (secure 0 0 0) 0x7fd118021910 con 0x7fd11c100db0 2024-04-23T10:18:16.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:16 smithi196 ceph-mon[32919]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:18:16.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:16 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/885734056' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T10:18:16.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:16 smithi196 ceph-mon[32919]: Standby manager daemon z started 2024-04-23T10:18:16.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:16 smithi196 ceph-mon[32919]: from='mgr.? 172.21.15.196:0/2690752770' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-23T10:18:16.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:16 smithi196 ceph-mon[32919]: from='mgr.? 172.21.15.196:0/2690752770' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T10:18:16.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:16 smithi196 ceph-mon[32919]: from='mgr.? 172.21.15.196:0/2690752770' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-23T10:18:16.843 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:16 smithi196 ceph-mon[32919]: from='mgr.? 172.21.15.196:0/2690752770' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T10:18:16.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:16 smithi097 ceph-mon[29197]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:18:16.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:16 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/885734056' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T10:18:16.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:16 smithi097 ceph-mon[29197]: Standby manager daemon z started 2024-04-23T10:18:16.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:16 smithi097 ceph-mon[29197]: from='mgr.? 172.21.15.196:0/2690752770' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-23T10:18:16.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:16 smithi097 ceph-mon[29197]: from='mgr.? 172.21.15.196:0/2690752770' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T10:18:16.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:16 smithi097 ceph-mon[29197]: from='mgr.? 172.21.15.196:0/2690752770' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-23T10:18:16.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:16 smithi097 ceph-mon[29197]: from='mgr.? 172.21.15.196:0/2690752770' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T10:18:16.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:16 smithi183 ceph-mon[32317]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:18:16.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:16 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/885734056' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T10:18:16.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:16 smithi183 ceph-mon[32317]: Standby manager daemon z started 2024-04-23T10:18:16.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:16 smithi183 ceph-mon[32317]: from='mgr.? 172.21.15.196:0/2690752770' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-23T10:18:16.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:16 smithi183 ceph-mon[32317]: from='mgr.? 172.21.15.196:0/2690752770' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T10:18:16.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:16 smithi183 ceph-mon[32317]: from='mgr.? 172.21.15.196:0/2690752770' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-23T10:18:16.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:16 smithi183 ceph-mon[32317]: from='mgr.? 172.21.15.196:0/2690752770' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T10:18:17.400 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 23 10:18:17 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-y[33699]: 2024-04-23T10:18:17.081+0000 7f6ccdd07200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-23T10:18:17.400 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 23 10:18:17 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-y[33699]: 2024-04-23T10:18:17.171+0000 7f6ccdd07200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-23T10:18:17.752 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:17 smithi183 ceph-mon[32317]: mgrmap e14: x(active, since 77s), standbys: z 2024-04-23T10:18:17.752 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:17 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-23T10:18:17.753 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 23 10:18:17 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-y[33699]: 2024-04-23T10:18:17.412+0000 7f6ccdd07200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-23T10:18:17.753 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 23 10:18:17 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-y[33699]: 2024-04-23T10:18:17.543+0000 7f6ccdd07200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-23T10:18:17.753 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 23 10:18:17 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-y[33699]: 2024-04-23T10:18:17.638+0000 7f6ccdd07200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-23T10:18:17.753 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 23 10:18:17 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-y[33699]: 2024-04-23T10:18:17.753+0000 7f6ccdd07200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-23T10:18:17.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:17 smithi097 ceph-mon[29197]: mgrmap e14: x(active, since 77s), standbys: z 2024-04-23T10:18:17.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:17 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-23T10:18:18.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:17 smithi196 ceph-mon[32919]: mgrmap e14: x(active, since 77s), standbys: z 2024-04-23T10:18:18.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:17 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-23T10:18:18.400 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 23 10:18:18 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-y[33699]: 2024-04-23T10:18:18.056+0000 7f6ccdd07200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-23T10:18:18.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:18 smithi097 ceph-mon[29197]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:18:18.871 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:18 smithi183 ceph-mon[32317]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:18:18.871 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 23 10:18:18 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-y[33699]: 2024-04-23T10:18:18.655+0000 7f6ccdd07200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-23T10:18:19.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:18 smithi196 ceph-mon[32919]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:18:19.150 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 23 10:18:18 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-y[33699]: 2024-04-23T10:18:18.870+0000 7f6ccdd07200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-23T10:18:19.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:19.616+0000 7fd112ffd700 1 -- 172.21.15.97:0/1497011719 <== mon.0 v2:172.21.15.97:3300/0 8 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7fd1180213f0 con 0x7fd11c100db0 2024-04-23T10:18:19.741 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:19 smithi097 ceph-mon[29197]: Standby manager daemon y started 2024-04-23T10:18:19.742 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:19 smithi097 ceph-mon[29197]: from='mgr.? 172.21.15.183:0/797362203' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-23T10:18:19.742 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:19 smithi097 ceph-mon[29197]: from='mgr.? 172.21.15.183:0/797362203' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T10:18:19.742 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:19 smithi097 ceph-mon[29197]: from='mgr.? 172.21.15.183:0/797362203' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-23T10:18:19.742 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:19 smithi097 ceph-mon[29197]: from='mgr.? 172.21.15.183:0/797362203' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T10:18:19.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:19 smithi183 ceph-mon[32317]: Standby manager daemon y started 2024-04-23T10:18:19.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:19 smithi183 ceph-mon[32317]: from='mgr.? 172.21.15.183:0/797362203' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-23T10:18:19.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:19 smithi183 ceph-mon[32317]: from='mgr.? 172.21.15.183:0/797362203' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T10:18:19.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:19 smithi183 ceph-mon[32317]: from='mgr.? 172.21.15.183:0/797362203' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-23T10:18:19.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:19 smithi183 ceph-mon[32317]: from='mgr.? 172.21.15.183:0/797362203' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T10:18:20.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:19 smithi196 ceph-mon[32919]: Standby manager daemon y started 2024-04-23T10:18:20.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:19 smithi196 ceph-mon[32919]: from='mgr.? 172.21.15.183:0/797362203' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-23T10:18:20.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:19 smithi196 ceph-mon[32919]: from='mgr.? 172.21.15.183:0/797362203' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T10:18:20.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:19 smithi196 ceph-mon[32919]: from='mgr.? 172.21.15.183:0/797362203' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-23T10:18:20.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:19 smithi196 ceph-mon[32919]: from='mgr.? 172.21.15.183:0/797362203' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T10:18:20.898 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:20 smithi097 ceph-mon[29197]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:18:20.899 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:20 smithi097 ceph-mon[29197]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-04-23T10:18:20.899 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:20 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-23T10:18:20.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:20 smithi183 ceph-mon[32317]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:18:20.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:20 smithi183 ceph-mon[32317]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-04-23T10:18:20.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:20 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-23T10:18:21.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:20 smithi196 ceph-mon[32919]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:18:21.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:20 smithi196 ceph-mon[32919]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-04-23T10:18:21.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:20 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-23T10:18:22.070 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:21 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:22.071 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:21 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:22.071 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:21 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:18:22.071 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:21 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:18:22.071 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:21 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:18:22.071 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:21 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:22.071 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:21 smithi097 ceph-mon[29197]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:18:22.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:21 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:22.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:21 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:22.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:21 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:18:22.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:21 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:18:22.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:21 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:18:22.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:21 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:22.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:21 smithi196 ceph-mon[32919]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:18:22.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:21 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:22.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:21 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:22.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:21 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:18:22.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:21 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:18:22.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:21 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:18:22.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:21 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:22.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:21 smithi183 ceph-mon[32317]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:18:23.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:23 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-23T10:18:23.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:23 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:18:23.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:23 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-23T10:18:23.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:23 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:18:23.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:23 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-23T10:18:23.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:23 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:18:24.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:24 smithi097 ceph-mon[29197]: Deploying daemon osd.0 on smithi097 2024-04-23T10:18:24.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:24 smithi097 ceph-mon[29197]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:18:24.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:24 smithi196 ceph-mon[32919]: Deploying daemon osd.0 on smithi097 2024-04-23T10:18:24.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:24 smithi196 ceph-mon[32919]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:18:24.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:24 smithi183 ceph-mon[32317]: Deploying daemon osd.0 on smithi097 2024-04-23T10:18:24.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:24 smithi183 ceph-mon[32317]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:18:26.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:26 smithi196 ceph-mon[32919]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:18:26.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:26 smithi097 ceph-mon[29197]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:18:26.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:26 smithi183 ceph-mon[32317]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:18:28.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:28 smithi196 ceph-mon[32919]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:18:28.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:28 smithi097 ceph-mon[29197]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:18:28.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:28 smithi183 ceph-mon[32317]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:18:29.837 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:29 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:29.837 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:29 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:29.837 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:29 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:18:29.837 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:29 smithi097 ceph-mon[29197]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:18:30.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:29 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:30.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:29 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:30.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:29 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:18:30.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:29 smithi196 ceph-mon[32919]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:18:30.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:29 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:30.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:29 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:30.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:29 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:18:30.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:29 smithi183 ceph-mon[32317]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:18:32.231 INFO:teuthology.orchestra.run.smithi097.stdout:Created osd(s) 0 on host 'smithi097' 2024-04-23T10:18:32.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:32.232+0000 7fd112ffd700 1 -- 172.21.15.97:0/1497011719 <== mgr.14152 v2:172.21.15.97:6800/3149043749 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd11c062e80 con 0x7fd108039890 2024-04-23T10:18:32.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:32.234+0000 7fd123ca4700 1 -- 172.21.15.97:0/1497011719 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fd108039890 msgr2=0x7fd10803bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:18:32.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:32.234+0000 7fd123ca4700 1 --2- 172.21.15.97:0/1497011719 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fd108039890 0x7fd10803bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd10c00b5c0 tx=0x7fd10c005fb0 comp rx=0 tx=0).stop 2024-04-23T10:18:32.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:32.234+0000 7fd123ca4700 1 -- 172.21.15.97:0/1497011719 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd11c100db0 msgr2=0x7fd11c198b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:18:32.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:32.234+0000 7fd123ca4700 1 --2- 172.21.15.97:0/1497011719 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd11c100db0 0x7fd11c198b50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd11800c9c0 tx=0x7fd11800ccd0 comp rx=0 tx=0).stop 2024-04-23T10:18:32.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:32.234+0000 7fd123ca4700 1 -- 172.21.15.97:0/1497011719 shutdown_connections 2024-04-23T10:18:32.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:32.234+0000 7fd123ca4700 1 --2- 172.21.15.97:0/1497011719 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fd108039890 0x7fd10803bd50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:18:32.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:32.234+0000 7fd123ca4700 1 --2- 172.21.15.97:0/1497011719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd11c104730 0x7fd11c1a0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:18:32.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:32.234+0000 7fd123ca4700 1 --2- 172.21.15.97:0/1497011719 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd11c100db0 0x7fd11c198b50 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:18:32.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:32.234+0000 7fd123ca4700 1 --2- 172.21.15.97:0/1497011719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd11c103d90 0x7fd11c199090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:18:32.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:32.234+0000 7fd123ca4700 1 -- 172.21.15.97:0/1497011719 >> 172.21.15.97:0/1497011719 conn(0x7fd11c0fc710 msgr2=0x7fd11c107090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:18:32.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:32.234+0000 7fd123ca4700 1 -- 172.21.15.97:0/1497011719 shutdown_connections 2024-04-23T10:18:32.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:18:32.234+0000 7fd123ca4700 1 -- 172.21.15.97:0/1497011719 wait complete. 2024-04-23T10:18:32.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:32 smithi097 ceph-mon[29197]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:18:32.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:32 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:32.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:32 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:32.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:32 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:32.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:32 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:32.841 DEBUG:teuthology.orchestra.run.smithi097:osd.0> sudo journalctl -f -n 0 -u ceph-9b22af88-0159-11ef-bc93-c7b262605968@osd.0.service 2024-04-23T10:18:32.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:32 smithi196 ceph-mon[32919]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:18:32.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:32 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:32.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:32 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:32.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:32 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:32.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:32 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:32.844 INFO:tasks.cephadm:Deploying osd.1 on smithi183 with /dev/vg_nvme/lv_4... 2024-04-23T10:18:32.844 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-23T10:18:32.871 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:32 smithi183 ceph-mon[32317]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:18:32.871 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:32 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:32.871 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:32 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:32.871 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:32 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:32.872 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:32 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:33.089 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.b/config 2024-04-23T10:18:33.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:33 smithi196 ceph-mon[32919]: from='osd.0 [v2:172.21.15.97:6802/3031221802,v1:172.21.15.97:6803/3031221802]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-23T10:18:33.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:33 smithi097 ceph-mon[29197]: from='osd.0 [v2:172.21.15.97:6802/3031221802,v1:172.21.15.97:6803/3031221802]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-23T10:18:33.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:33 smithi183 ceph-mon[32317]: from='osd.0 [v2:172.21.15.97:6802/3031221802,v1:172.21.15.97:6803/3031221802]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-23T10:18:34.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:34 smithi097 ceph-mon[29197]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:18:34.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:34 smithi097 ceph-mon[29197]: from='osd.0 [v2:172.21.15.97:6802/3031221802,v1:172.21.15.97:6803/3031221802]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-23T10:18:34.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:34 smithi097 ceph-mon[29197]: osdmap e6: 1 total, 0 up, 1 in 2024-04-23T10:18:34.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:34 smithi097 ceph-mon[29197]: from='osd.0 [v2:172.21.15.97:6802/3031221802,v1:172.21.15.97:6803/3031221802]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-23T10:18:34.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:34 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T10:18:34.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:34 smithi196 ceph-mon[32919]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:18:34.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:34 smithi196 ceph-mon[32919]: from='osd.0 [v2:172.21.15.97:6802/3031221802,v1:172.21.15.97:6803/3031221802]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-23T10:18:34.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:34 smithi196 ceph-mon[32919]: osdmap e6: 1 total, 0 up, 1 in 2024-04-23T10:18:34.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:34 smithi196 ceph-mon[32919]: from='osd.0 [v2:172.21.15.97:6802/3031221802,v1:172.21.15.97:6803/3031221802]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-23T10:18:34.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:34 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T10:18:34.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:34 smithi183 ceph-mon[32317]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:18:34.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:34 smithi183 ceph-mon[32317]: from='osd.0 [v2:172.21.15.97:6802/3031221802,v1:172.21.15.97:6803/3031221802]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-23T10:18:34.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:34 smithi183 ceph-mon[32317]: osdmap e6: 1 total, 0 up, 1 in 2024-04-23T10:18:34.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:34 smithi183 ceph-mon[32317]: from='osd.0 [v2:172.21.15.97:6802/3031221802,v1:172.21.15.97:6803/3031221802]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-23T10:18:34.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:34 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T10:18:34.986 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-23T10:18:34.986 INFO:teuthology.orchestra.run.smithi183.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-23T10:18:34.987 INFO:teuthology.orchestra.run.smithi183.stderr: stderr: 10+0 records in 2024-04-23T10:18:34.987 INFO:teuthology.orchestra.run.smithi183.stderr:10+0 records out 2024-04-23T10:18:34.987 INFO:teuthology.orchestra.run.smithi183.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.014083 s, 745 MB/s 2024-04-23T10:18:34.987 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping successful for: 2024-04-23T10:18:35.361 INFO:journalctl@ceph.osd.0.smithi097.stdout:Apr 23 10:18:34 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-0[41281]: 2024-04-23T10:18:34.994+0000 7f0e67c78700 -1 osd.0 0 waiting for initial osdmap 2024-04-23T10:18:35.361 INFO:journalctl@ceph.osd.0.smithi097.stdout:Apr 23 10:18:34 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-0[41281]: 2024-04-23T10:18:34.998+0000 7f0e5fa63700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T10:18:35.622 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi183:vg_nvme/lv_4 2024-04-23T10:18:35.821 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:35 smithi097 ceph-mon[29197]: from='osd.0 [v2:172.21.15.97:6802/3031221802,v1:172.21.15.97:6803/3031221802]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-23T10:18:35.821 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:35 smithi097 ceph-mon[29197]: osdmap e7: 1 total, 0 up, 1 in 2024-04-23T10:18:35.821 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:35 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T10:18:35.821 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:35 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T10:18:35.821 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:35 smithi097 ceph-mon[29197]: from='osd.0 [v2:172.21.15.97:6802/3031221802,v1:172.21.15.97:6803/3031221802]' entity='osd.0' 2024-04-23T10:18:35.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:35 smithi196 ceph-mon[32919]: from='osd.0 [v2:172.21.15.97:6802/3031221802,v1:172.21.15.97:6803/3031221802]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-23T10:18:35.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:35 smithi196 ceph-mon[32919]: osdmap e7: 1 total, 0 up, 1 in 2024-04-23T10:18:35.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:35 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T10:18:35.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:35 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T10:18:35.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:35 smithi196 ceph-mon[32919]: from='osd.0 [v2:172.21.15.97:6802/3031221802,v1:172.21.15.97:6803/3031221802]' entity='osd.0' 2024-04-23T10:18:35.900 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.b/config 2024-04-23T10:18:35.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:35 smithi183 ceph-mon[32317]: from='osd.0 [v2:172.21.15.97:6802/3031221802,v1:172.21.15.97:6803/3031221802]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-23T10:18:35.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:35 smithi183 ceph-mon[32317]: osdmap e7: 1 total, 0 up, 1 in 2024-04-23T10:18:35.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:35 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T10:18:35.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:35 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T10:18:35.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:35 smithi183 ceph-mon[32317]: from='osd.0 [v2:172.21.15.97:6802/3031221802,v1:172.21.15.97:6803/3031221802]' entity='osd.0' 2024-04-23T10:18:36.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:36 smithi196 ceph-mon[32919]: purged_snaps scrub starts 2024-04-23T10:18:36.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:36 smithi196 ceph-mon[32919]: purged_snaps scrub ok 2024-04-23T10:18:36.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:36 smithi196 ceph-mon[32919]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:18:36.843 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:36 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T10:18:36.843 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:36 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:36.843 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:36 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:36.843 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:36 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:18:36.843 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:36 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:36.843 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:36 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:18:36.843 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:36 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:18:36.843 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:36 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:36.843 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:36 smithi196 ceph-mon[32919]: osd.0 [v2:172.21.15.97:6802/3031221802,v1:172.21.15.97:6803/3031221802] boot 2024-04-23T10:18:36.843 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:36 smithi196 ceph-mon[32919]: osdmap e8: 1 total, 1 up, 1 in 2024-04-23T10:18:36.843 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:36 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T10:18:36.843 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:36 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:18:36.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:36 smithi097 ceph-mon[29197]: purged_snaps scrub starts 2024-04-23T10:18:36.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:36 smithi097 ceph-mon[29197]: purged_snaps scrub ok 2024-04-23T10:18:36.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:36 smithi097 ceph-mon[29197]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:18:36.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:36 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T10:18:36.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:36 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:36.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:36 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:36.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:36 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:18:36.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:36 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:36.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:36 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:18:36.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:36 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:18:36.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:36 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:36.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:36 smithi097 ceph-mon[29197]: osd.0 [v2:172.21.15.97:6802/3031221802,v1:172.21.15.97:6803/3031221802] boot 2024-04-23T10:18:36.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:36 smithi097 ceph-mon[29197]: osdmap e8: 1 total, 1 up, 1 in 2024-04-23T10:18:36.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:36 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T10:18:36.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:36 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:18:36.899 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:36 smithi183 ceph-mon[32317]: purged_snaps scrub starts 2024-04-23T10:18:36.899 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:36 smithi183 ceph-mon[32317]: purged_snaps scrub ok 2024-04-23T10:18:36.899 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:36 smithi183 ceph-mon[32317]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T10:18:36.899 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:36 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T10:18:36.899 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:36 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:36.899 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:36 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:36.899 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:36 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:18:36.899 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:36 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:36.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:36 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:18:36.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:36 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:18:36.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:36 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:36.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:36 smithi183 ceph-mon[32317]: osd.0 [v2:172.21.15.97:6802/3031221802,v1:172.21.15.97:6803/3031221802] boot 2024-04-23T10:18:36.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:36 smithi183 ceph-mon[32317]: osdmap e8: 1 total, 1 up, 1 in 2024-04-23T10:18:36.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:36 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T10:18:36.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:36 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:18:37.416 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:18:37.415+0000 7f213b7fe700 1 -- 172.21.15.183:0/2907760400 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2134102f60 msgr2=0x7f2134103380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:18:37.416 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:18:37.415+0000 7f213b7fe700 1 --2- 172.21.15.183:0/2907760400 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2134102f60 0x7f2134103380 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2130009990 tx=0x7f2130009ca0 comp rx=0 tx=0).stop 2024-04-23T10:18:37.416 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:18:37.415+0000 7f213b7fe700 1 -- 172.21.15.183:0/2907760400 shutdown_connections 2024-04-23T10:18:37.417 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:18:37.415+0000 7f213b7fe700 1 --2- 172.21.15.183:0/2907760400 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2134102640 0x7f2134109f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:18:37.417 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:18:37.415+0000 7f213b7fe700 1 --2- 172.21.15.183:0/2907760400 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2134102f60 0x7f2134103380 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:18:37.417 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:18:37.415+0000 7f213b7fe700 1 --2- 172.21.15.183:0/2907760400 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2134101ca0 0x7f2134102100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:18:37.417 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:18:37.415+0000 7f213b7fe700 1 -- 172.21.15.183:0/2907760400 >> 172.21.15.183:0/2907760400 conn(0x7f21340fc6f0 msgr2=0x7f21340feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:18:37.417 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:18:37.415+0000 7f213b7fe700 1 -- 172.21.15.183:0/2907760400 shutdown_connections 2024-04-23T10:18:37.417 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:18:37.415+0000 7f213b7fe700 1 -- 172.21.15.183:0/2907760400 wait complete. 2024-04-23T10:18:37.417 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:18:37.415+0000 7f213b7fe700 1 Processor -- start 2024-04-23T10:18:37.417 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:18:37.415+0000 7f213b7fe700 1 -- start start 2024-04-23T10:18:37.417 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:18:37.415+0000 7f213b7fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2134101ca0 0x7f2134198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:18:37.417 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:18:37.415+0000 7f213b7fe700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2134102640 0x7f2134199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:18:37.417 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:18:37.415+0000 7f213b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2134102f60 0x7f21341a0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:18:37.417 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:18:37.415+0000 7f213b7fe700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f213410d460 con 0x7f2134102f60 2024-04-23T10:18:37.417 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:18:37.415+0000 7f213b7fe700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f213410d2e0 con 0x7f2134102640 2024-04-23T10:18:37.417 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:18:37.415+0000 7f213b7fe700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f213410d5e0 con 0x7f2134101ca0 2024-04-23T10:18:37.418 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:18:37.416+0000 7f2138d99700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2134102640 0x7f2134199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:18:37.418 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:18:37.416+0000 7f213959a700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2134101ca0 0x7f2134198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:18:37.418 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:18:37.416+0000 7f2138d99700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2134102640 0x7f2134199080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:56646/0 (socket says 172.21.15.183:56646) 2024-04-23T10:18:37.418 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:18:37.416+0000 7f2138d99700 1 -- 172.21.15.183:0/1805378402 learned_addr learned my addr 172.21.15.183:0/1805378402 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-04-23T10:18:37.418 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:18:37.416+0000 7f2139d9b700 1 --2- 172.21.15.183:0/1805378402 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2134102f60 0x7f21341a0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:18:37.418 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:18:37.416+0000 7f2138d99700 1 -- 172.21.15.183:0/1805378402 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2134101ca0 msgr2=0x7f2134198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:18:37.418 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:18:37.416+0000 7f2138d99700 1 --2- 172.21.15.183:0/1805378402 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2134101ca0 0x7f2134198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:18:37.418 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:18:37.416+0000 7f2138d99700 1 -- 172.21.15.183:0/1805378402 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2134102f60 msgr2=0x7f21341a0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:18:37.418 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:18:37.416+0000 7f2138d99700 1 --2- 172.21.15.183:0/1805378402 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2134102f60 0x7f21341a0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:18:37.418 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:18:37.416+0000 7f2138d99700 1 -- 172.21.15.183:0/1805378402 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21341a07a0 con 0x7f2134102640 2024-04-23T10:18:37.418 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:18:37.416+0000 7f2138d99700 1 --2- 172.21.15.183:0/1805378402 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2134102640 0x7f2134199080 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f212400d980 tx=0x7f212400dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:18:37.418 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:18:37.416+0000 7f212a7fc700 1 -- 172.21.15.183:0/1805378402 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2124017070 con 0x7f2134102640 2024-04-23T10:18:37.419 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:18:37.416+0000 7f213b7fe700 1 -- 172.21.15.183:0/1805378402 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21341a0a90 con 0x7f2134102640 2024-04-23T10:18:37.419 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:18:37.416+0000 7f212a7fc700 1 -- 172.21.15.183:0/1805378402 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2124005020 con 0x7f2134102640 2024-04-23T10:18:37.419 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:18:37.416+0000 7f212a7fc700 1 -- 172.21.15.183:0/1805378402 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f212400fa90 con 0x7f2134102640 2024-04-23T10:18:37.419 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:18:37.416+0000 7f213b7fe700 1 -- 172.21.15.183:0/1805378402 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21341a0f80 con 0x7f2134102640 2024-04-23T10:18:37.419 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:18:37.417+0000 7f213b7fe700 1 -- 172.21.15.183:0/1805378402 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2134069730 con 0x7f2134102640 2024-04-23T10:18:37.420 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:18:37.418+0000 7f212a7fc700 1 -- 172.21.15.183:0/1805378402 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f212400fc30 con 0x7f2134102640 2024-04-23T10:18:37.420 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:18:37.419+0000 7f212a7fc700 1 --2- 172.21.15.183:0/1805378402 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f21200a3bb0 0x7f21200a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:18:37.420 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:18:37.419+0000 7f212a7fc700 1 -- 172.21.15.183:0/1805378402 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1767+0+0 (secure 0 0 0) 0x7f21240cebe0 con 0x7f2134102640 2024-04-23T10:18:37.420 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:18:37.419+0000 7f213959a700 1 --2- 172.21.15.183:0/1805378402 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f21200a3bb0 0x7f21200a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:18:37.420 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:18:37.420+0000 7f213959a700 1 --2- 172.21.15.183:0/1805378402 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f21200a3bb0 0x7f21200a6070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f21300038a0 tx=0x7f2130005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:18:37.422 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:18:37.423+0000 7f212a7fc700 1 -- 172.21.15.183:0/1805378402 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2124020020 con 0x7f2134102640 2024-04-23T10:18:37.628 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:18:37.628+0000 7f213b7fe700 1 -- 172.21.15.183:0/1805378402 --> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f21340fe940 con 0x7f21200a3bb0 2024-04-23T10:18:37.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:37 smithi196 ceph-mon[32919]: Detected new or changed devices on smithi097 2024-04-23T10:18:37.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:37 smithi196 ceph-mon[32919]: Adjusting osd_memory_target on smithi097 to 17088M 2024-04-23T10:18:37.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:37 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:18:37.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:37 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:18:37.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:37 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:37.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:37 smithi097 ceph-mon[29197]: Detected new or changed devices on smithi097 2024-04-23T10:18:37.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:37 smithi097 ceph-mon[29197]: Adjusting osd_memory_target on smithi097 to 17088M 2024-04-23T10:18:37.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:37 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:18:37.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:37 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:18:37.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:37 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:37.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:37 smithi183 ceph-mon[32317]: Detected new or changed devices on smithi097 2024-04-23T10:18:37.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:37 smithi183 ceph-mon[32317]: Adjusting osd_memory_target on smithi097 to 17088M 2024-04-23T10:18:37.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:37 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:18:37.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:37 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:18:37.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:37 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:38.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:38 smithi183 ceph-mon[32317]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T10:18:38.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:38 smithi183 ceph-mon[32317]: osdmap e9: 1 total, 1 up, 1 in 2024-04-23T10:18:38.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:38 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T10:18:38.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:38 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T10:18:38.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:38 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:18:39.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:38 smithi196 ceph-mon[32919]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T10:18:39.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:38 smithi196 ceph-mon[32919]: osdmap e9: 1 total, 1 up, 1 in 2024-04-23T10:18:39.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:38 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T10:18:39.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:38 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T10:18:39.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:38 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:18:39.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:38 smithi097 ceph-mon[29197]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T10:18:39.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:38 smithi097 ceph-mon[29197]: osdmap e9: 1 total, 1 up, 1 in 2024-04-23T10:18:39.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:38 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T10:18:39.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:38 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T10:18:39.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:38 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:18:39.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:39 smithi183 ceph-mon[32317]: from='client.24134 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:18:39.993 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:39 smithi196 ceph-mon[32919]: from='client.24134 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:18:40.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:39 smithi097 ceph-mon[29197]: from='client.24134 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:18:40.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:40 smithi183 ceph-mon[32317]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T10:18:40.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:40 smithi183 ceph-mon[32317]: from='client.? 172.21.15.183:0/623124701' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b47bd8ee-1f21-4834-acf5-bb40f97ab3ac"}]: dispatch 2024-04-23T10:18:40.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:40 smithi183 ceph-mon[32317]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b47bd8ee-1f21-4834-acf5-bb40f97ab3ac"}]: dispatch 2024-04-23T10:18:40.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:40 smithi183 ceph-mon[32317]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b47bd8ee-1f21-4834-acf5-bb40f97ab3ac"}]': finished 2024-04-23T10:18:40.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:40 smithi183 ceph-mon[32317]: osdmap e10: 2 total, 1 up, 2 in 2024-04-23T10:18:40.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:40 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T10:18:41.010 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:40 smithi196 ceph-mon[32919]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T10:18:41.010 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:40 smithi196 ceph-mon[32919]: from='client.? 172.21.15.183:0/623124701' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b47bd8ee-1f21-4834-acf5-bb40f97ab3ac"}]: dispatch 2024-04-23T10:18:41.010 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:40 smithi196 ceph-mon[32919]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b47bd8ee-1f21-4834-acf5-bb40f97ab3ac"}]: dispatch 2024-04-23T10:18:41.010 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:40 smithi196 ceph-mon[32919]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b47bd8ee-1f21-4834-acf5-bb40f97ab3ac"}]': finished 2024-04-23T10:18:41.010 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:40 smithi196 ceph-mon[32919]: osdmap e10: 2 total, 1 up, 2 in 2024-04-23T10:18:41.011 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:40 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T10:18:41.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:40 smithi097 ceph-mon[29197]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T10:18:41.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:40 smithi097 ceph-mon[29197]: from='client.? 172.21.15.183:0/623124701' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b47bd8ee-1f21-4834-acf5-bb40f97ab3ac"}]: dispatch 2024-04-23T10:18:41.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:40 smithi097 ceph-mon[29197]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b47bd8ee-1f21-4834-acf5-bb40f97ab3ac"}]: dispatch 2024-04-23T10:18:41.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:40 smithi097 ceph-mon[29197]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b47bd8ee-1f21-4834-acf5-bb40f97ab3ac"}]': finished 2024-04-23T10:18:41.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:40 smithi097 ceph-mon[29197]: osdmap e10: 2 total, 1 up, 2 in 2024-04-23T10:18:41.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:40 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T10:18:41.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:41 smithi183 ceph-mon[32317]: from='client.? 172.21.15.183:0/1633735213' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T10:18:41.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:41 smithi183 ceph-mon[32317]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T10:18:42.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:41 smithi196 ceph-mon[32919]: from='client.? 172.21.15.183:0/1633735213' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T10:18:42.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:41 smithi196 ceph-mon[32919]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T10:18:42.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:41 smithi097 ceph-mon[29197]: from='client.? 172.21.15.183:0/1633735213' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T10:18:42.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:41 smithi097 ceph-mon[29197]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T10:18:44.600 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:44 smithi183 ceph-mon[32317]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T10:18:44.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:44 smithi196 ceph-mon[32919]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T10:18:44.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:44 smithi097 ceph-mon[29197]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T10:18:46.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:46 smithi196 ceph-mon[32919]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T10:18:46.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:46 smithi097 ceph-mon[29197]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T10:18:46.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:46 smithi183 ceph-mon[32317]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T10:18:47.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:47 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-23T10:18:47.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:47 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:18:47.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:47 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-23T10:18:47.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:47 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:18:47.863 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:47 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-23T10:18:47.863 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:47 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:18:48.793 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:48 smithi183 ceph-mon[32317]: Deploying daemon osd.1 on smithi183 2024-04-23T10:18:48.793 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:48 smithi183 ceph-mon[32317]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T10:18:48.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:48 smithi196 ceph-mon[32919]: Deploying daemon osd.1 on smithi183 2024-04-23T10:18:48.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:48 smithi196 ceph-mon[32919]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T10:18:48.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:48 smithi097 ceph-mon[29197]: Deploying daemon osd.1 on smithi183 2024-04-23T10:18:48.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:48 smithi097 ceph-mon[29197]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T10:18:50.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:50 smithi183 ceph-mon[32317]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T10:18:50.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:50 smithi196 ceph-mon[32919]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T10:18:50.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:50 smithi097 ceph-mon[29197]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T10:18:52.594 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:52 smithi183 ceph-mon[32317]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T10:18:52.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:52 smithi196 ceph-mon[32919]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T10:18:52.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:52 smithi097 ceph-mon[29197]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T10:18:53.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:53 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:53.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:53 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:53.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:53 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:18:53.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:53 smithi183 ceph-mon[32317]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T10:18:54.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:53 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:54.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:53 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:54.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:53 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:18:54.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:53 smithi196 ceph-mon[32919]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T10:18:54.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:53 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:54.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:53 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:54.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:53 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:18:54.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:53 smithi097 ceph-mon[29197]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T10:18:56.292 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:18:56.293+0000 7f212a7fc700 1 -- 172.21.15.183:0/1805378402 <== mgr.14152 v2:172.21.15.97:6800/3149043749 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f21340fe940 con 0x7f21200a3bb0 2024-04-23T10:18:56.293 INFO:teuthology.orchestra.run.smithi183.stdout:Created osd(s) 1 on host 'smithi183' 2024-04-23T10:18:56.293 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:55 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:56.293 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:55 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:56.293 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:55 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:18:56.293 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:55 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:18:56.293 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:55 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:56.293 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:55 smithi183 ceph-mon[32317]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T10:18:56.295 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:18:56.295+0000 7f213b7fe700 1 -- 172.21.15.183:0/1805378402 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f21200a3bb0 msgr2=0x7f21200a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:18:56.295 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:18:56.295+0000 7f213b7fe700 1 --2- 172.21.15.183:0/1805378402 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f21200a3bb0 0x7f21200a6070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f21300038a0 tx=0x7f2130005fb0 comp rx=0 tx=0).stop 2024-04-23T10:18:56.295 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:18:56.295+0000 7f213b7fe700 1 -- 172.21.15.183:0/1805378402 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2134102640 msgr2=0x7f2134199080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:18:56.295 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:18:56.295+0000 7f213b7fe700 1 --2- 172.21.15.183:0/1805378402 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2134102640 0x7f2134199080 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f212400d980 tx=0x7f212400dc90 comp rx=0 tx=0).stop 2024-04-23T10:18:56.295 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:18:56.296+0000 7f213b7fe700 1 -- 172.21.15.183:0/1805378402 shutdown_connections 2024-04-23T10:18:56.295 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:18:56.296+0000 7f213b7fe700 1 --2- 172.21.15.183:0/1805378402 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f21200a3bb0 0x7f21200a6070 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:18:56.295 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:18:56.296+0000 7f213b7fe700 1 --2- 172.21.15.183:0/1805378402 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2134102f60 0x7f21341a0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:18:56.295 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:18:56.296+0000 7f213b7fe700 1 --2- 172.21.15.183:0/1805378402 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2134101ca0 0x7f2134198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:18:56.295 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:18:56.296+0000 7f213b7fe700 1 --2- 172.21.15.183:0/1805378402 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2134102640 0x7f2134199080 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:18:56.295 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:18:56.296+0000 7f213b7fe700 1 -- 172.21.15.183:0/1805378402 >> 172.21.15.183:0/1805378402 conn(0x7f21340fc6f0 msgr2=0x7f21340fe230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:18:56.296 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:18:56.296+0000 7f213b7fe700 1 -- 172.21.15.183:0/1805378402 shutdown_connections 2024-04-23T10:18:56.296 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:18:56.296+0000 7f213b7fe700 1 -- 172.21.15.183:0/1805378402 wait complete. 2024-04-23T10:18:56.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:55 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:56.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:55 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:56.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:55 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:18:56.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:55 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:18:56.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:55 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:56.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:55 smithi196 ceph-mon[32919]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T10:18:56.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:55 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:56.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:55 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:56.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:55 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:18:56.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:55 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:18:56.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:55 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:56.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:55 smithi097 ceph-mon[29197]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T10:18:56.903 DEBUG:teuthology.orchestra.run.smithi183:osd.1> sudo journalctl -f -n 0 -u ceph-9b22af88-0159-11ef-bc93-c7b262605968@osd.1.service 2024-04-23T10:18:56.906 INFO:tasks.cephadm:Deploying osd.2 on smithi196 with /dev/vg_nvme/lv_4... 2024-04-23T10:18:56.906 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-23T10:18:57.151 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T10:18:57.533 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:57 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:57.533 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:57 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:57.534 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:57 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:18:57.534 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:57 smithi196 ceph-mon[32919]: from='osd.1 [v2:172.21.15.183:6800/1108870440,v1:172.21.15.183:6801/1108870440]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-23T10:18:57.534 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:57 smithi196 ceph-mon[32919]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-23T10:18:57.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:57 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:57.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:57 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:57.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:57 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:18:57.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:57 smithi097 ceph-mon[29197]: from='osd.1 [v2:172.21.15.183:6800/1108870440,v1:172.21.15.183:6801/1108870440]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-23T10:18:57.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:57 smithi097 ceph-mon[29197]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-23T10:18:57.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:57 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:57.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:57 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:18:57.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:57 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:18:57.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:57 smithi183 ceph-mon[32317]: from='osd.1 [v2:172.21.15.183:6800/1108870440,v1:172.21.15.183:6801/1108870440]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-23T10:18:57.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:57 smithi183 ceph-mon[32317]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-23T10:18:58.433 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:58 smithi196 ceph-mon[32919]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-23T10:18:58.433 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:58 smithi196 ceph-mon[32919]: from='osd.1 [v2:172.21.15.183:6800/1108870440,v1:172.21.15.183:6801/1108870440]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-23T10:18:58.434 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:58 smithi196 ceph-mon[32919]: osdmap e11: 2 total, 1 up, 2 in 2024-04-23T10:18:58.434 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:58 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T10:18:58.434 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:58 smithi196 ceph-mon[32919]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-23T10:18:58.434 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:58 smithi196 ceph-mon[32919]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T10:18:58.512 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:58 smithi097 ceph-mon[29197]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-23T10:18:58.512 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:58 smithi097 ceph-mon[29197]: from='osd.1 [v2:172.21.15.183:6800/1108870440,v1:172.21.15.183:6801/1108870440]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-23T10:18:58.512 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:58 smithi097 ceph-mon[29197]: osdmap e11: 2 total, 1 up, 2 in 2024-04-23T10:18:58.512 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:58 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T10:18:58.512 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:58 smithi097 ceph-mon[29197]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-23T10:18:58.512 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:58 smithi097 ceph-mon[29197]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T10:18:58.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:58 smithi183 ceph-mon[32317]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-23T10:18:58.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:58 smithi183 ceph-mon[32317]: from='osd.1 [v2:172.21.15.183:6800/1108870440,v1:172.21.15.183:6801/1108870440]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-23T10:18:58.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:58 smithi183 ceph-mon[32317]: osdmap e11: 2 total, 1 up, 2 in 2024-04-23T10:18:58.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:58 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T10:18:58.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:58 smithi183 ceph-mon[32317]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-23T10:18:58.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:58 smithi183 ceph-mon[32317]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T10:18:58.651 INFO:journalctl@ceph.osd.1.smithi183.stdout:Apr 23 10:18:58 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-1[36908]: 2024-04-23T10:18:58.255+0000 7f6747204700 -1 osd.1 0 waiting for initial osdmap 2024-04-23T10:18:58.651 INFO:journalctl@ceph.osd.1.smithi183.stdout:Apr 23 10:18:58 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-1[36908]: 2024-04-23T10:18:58.258+0000 7f6742ff7700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T10:18:59.008 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-23T10:18:59.008 INFO:teuthology.orchestra.run.smithi196.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-23T10:18:59.008 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10+0 records in 2024-04-23T10:18:59.008 INFO:teuthology.orchestra.run.smithi196.stderr:10+0 records out 2024-04-23T10:18:59.008 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112069 s, 936 MB/s 2024-04-23T10:18:59.008 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping successful for: 2024-04-23T10:18:59.352 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:59 smithi183 ceph-mon[32317]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-04-23T10:18:59.352 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:59 smithi183 ceph-mon[32317]: osdmap e12: 2 total, 1 up, 2 in 2024-04-23T10:18:59.352 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:59 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T10:18:59.352 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:18:59 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T10:18:59.529 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:59 smithi097 ceph-mon[29197]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-04-23T10:18:59.529 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:59 smithi097 ceph-mon[29197]: osdmap e12: 2 total, 1 up, 2 in 2024-04-23T10:18:59.529 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:59 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T10:18:59.529 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:18:59 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T10:18:59.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:59 smithi196 ceph-mon[32919]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-04-23T10:18:59.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:59 smithi196 ceph-mon[32919]: osdmap e12: 2 total, 1 up, 2 in 2024-04-23T10:18:59.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:59 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T10:18:59.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:18:59 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T10:18:59.671 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi196:vg_nvme/lv_4 2024-04-23T10:18:59.937 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T10:19:00.330 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:00 smithi196 ceph-mon[32919]: purged_snaps scrub starts 2024-04-23T10:19:00.330 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:00 smithi196 ceph-mon[32919]: purged_snaps scrub ok 2024-04-23T10:19:00.330 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:00 smithi196 ceph-mon[32919]: osd.1 [v2:172.21.15.183:6800/1108870440,v1:172.21.15.183:6801/1108870440] boot 2024-04-23T10:19:00.330 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:00 smithi196 ceph-mon[32919]: osdmap e13: 2 total, 2 up, 2 in 2024-04-23T10:19:00.330 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:00 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T10:19:00.330 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:00 smithi196 ceph-mon[32919]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T10:19:00.330 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:00 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:00.330 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:00 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:00.331 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:00 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:19:00.331 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:00 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:00.331 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:00 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:19:00.331 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:00 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:19:00.331 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:00 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:00.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:00 smithi097 ceph-mon[29197]: purged_snaps scrub starts 2024-04-23T10:19:00.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:00 smithi097 ceph-mon[29197]: purged_snaps scrub ok 2024-04-23T10:19:00.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:00 smithi097 ceph-mon[29197]: osd.1 [v2:172.21.15.183:6800/1108870440,v1:172.21.15.183:6801/1108870440] boot 2024-04-23T10:19:00.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:00 smithi097 ceph-mon[29197]: osdmap e13: 2 total, 2 up, 2 in 2024-04-23T10:19:00.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:00 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T10:19:00.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:00 smithi097 ceph-mon[29197]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T10:19:00.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:00 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:00.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:00 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:00.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:00 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:19:00.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:00 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:00.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:00 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:19:00.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:00 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:19:00.612 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:00 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:00.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:00 smithi183 ceph-mon[32317]: purged_snaps scrub starts 2024-04-23T10:19:00.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:00 smithi183 ceph-mon[32317]: purged_snaps scrub ok 2024-04-23T10:19:00.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:00 smithi183 ceph-mon[32317]: osd.1 [v2:172.21.15.183:6800/1108870440,v1:172.21.15.183:6801/1108870440] boot 2024-04-23T10:19:00.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:00 smithi183 ceph-mon[32317]: osdmap e13: 2 total, 2 up, 2 in 2024-04-23T10:19:00.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:00 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T10:19:00.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:00 smithi183 ceph-mon[32317]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T10:19:00.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:00 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:00.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:00 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:00.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:00 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:19:00.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:00 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:00.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:00 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:19:00.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:00 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:19:00.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:00 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:01.415 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:19:01.415+0000 7fa67a53e700 1 -- 172.21.15.196:0/3918047500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa674103540 msgr2=0x7fa674109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:19:01.415 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:19:01.415+0000 7fa67a53e700 1 --2- 172.21.15.196:0/3918047500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa674103540 0x7fa674109df0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa66c00b0a0 tx=0x7fa66c00b3b0 comp rx=0 tx=0).stop 2024-04-23T10:19:01.415 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:19:01.415+0000 7fa67a53e700 1 -- 172.21.15.196:0/3918047500 shutdown_connections 2024-04-23T10:19:01.415 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:19:01.415+0000 7fa67a53e700 1 --2- 172.21.15.196:0/3918047500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa674103540 0x7fa674109df0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:19:01.415 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:19:01.415+0000 7fa67a53e700 1 --2- 172.21.15.196:0/3918047500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6741019a0 0x7fa674101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:19:01.415 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:19:01.415+0000 7fa67a53e700 1 --2- 172.21.15.196:0/3918047500 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa674102ba0 0x7fa674103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:19:01.415 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:19:01.415+0000 7fa67a53e700 1 -- 172.21.15.196:0/3918047500 >> 172.21.15.196:0/3918047500 conn(0x7fa6740fcf40 msgr2=0x7fa6740ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:19:01.415 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:19:01.415+0000 7fa67a53e700 1 -- 172.21.15.196:0/3918047500 shutdown_connections 2024-04-23T10:19:01.415 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:19:01.415+0000 7fa67a53e700 1 -- 172.21.15.196:0/3918047500 wait complete. 2024-04-23T10:19:01.417 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:19:01.416+0000 7fa67a53e700 1 Processor -- start 2024-04-23T10:19:01.417 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:19:01.416+0000 7fa67a53e700 1 -- start start 2024-04-23T10:19:01.417 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:19:01.417+0000 7fa67a53e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6741019a0 0x7fa67406bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:19:01.418 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:19:01.417+0000 7fa67a53e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa674102ba0 0x7fa67419d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:19:01.418 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:19:01.417+0000 7fa67a53e700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa674103540 0x7fa67419fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:19:01.418 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:19:01.417+0000 7fa67a53e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa67410d440 con 0x7fa6741019a0 2024-04-23T10:19:01.418 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:19:01.417+0000 7fa67a53e700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa67410d2c0 con 0x7fa674103540 2024-04-23T10:19:01.418 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:19:01.417+0000 7fa67a53e700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa67410d5c0 con 0x7fa674102ba0 2024-04-23T10:19:01.418 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:19:01.417+0000 7fa6737fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa674102ba0 0x7fa67419d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:19:01.418 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:19:01.417+0000 7fa6737fe700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa674102ba0 0x7fa67419d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:48736/0 (socket says 172.21.15.196:48736) 2024-04-23T10:19:01.418 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:19:01.417+0000 7fa6737fe700 1 -- 172.21.15.196:0/3415622481 learned_addr learned my addr 172.21.15.196:0/3415622481 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-23T10:19:01.418 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:19:01.417+0000 7fa678adb700 1 --2- 172.21.15.196:0/3415622481 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa674103540 0x7fa67419fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:19:01.418 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:19:01.417+0000 7fa673fff700 1 --2- 172.21.15.196:0/3415622481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6741019a0 0x7fa67406bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:19:01.418 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:19:01.417+0000 7fa6737fe700 1 -- 172.21.15.196:0/3415622481 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa674103540 msgr2=0x7fa67419fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:19:01.419 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:19:01.417+0000 7fa6737fe700 1 --2- 172.21.15.196:0/3415622481 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa674103540 0x7fa67419fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:19:01.419 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:19:01.417+0000 7fa6737fe700 1 -- 172.21.15.196:0/3415622481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6741019a0 msgr2=0x7fa67406bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:19:01.419 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:19:01.417+0000 7fa6737fe700 1 --2- 172.21.15.196:0/3415622481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6741019a0 0x7fa67406bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:19:01.419 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:19:01.417+0000 7fa6737fe700 1 -- 172.21.15.196:0/3415622481 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6741a07b0 con 0x7fa674102ba0 2024-04-23T10:19:01.419 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:19:01.417+0000 7fa6737fe700 1 --2- 172.21.15.196:0/3415622481 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa674102ba0 0x7fa67419d890 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa66800d980 tx=0x7fa66800dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:19:01.419 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:19:01.417+0000 7fa6717fa700 1 -- 172.21.15.196:0/3415622481 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa668017070 con 0x7fa674102ba0 2024-04-23T10:19:01.419 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:19:01.417+0000 7fa6717fa700 1 -- 172.21.15.196:0/3415622481 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa668005020 con 0x7fa674102ba0 2024-04-23T10:19:01.419 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:19:01.417+0000 7fa67a53e700 1 -- 172.21.15.196:0/3415622481 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6741a0aa0 con 0x7fa674102ba0 2024-04-23T10:19:01.419 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:19:01.417+0000 7fa6717fa700 1 -- 172.21.15.196:0/3415622481 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa66800fa10 con 0x7fa674102ba0 2024-04-23T10:19:01.419 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:19:01.418+0000 7fa67a53e700 1 -- 172.21.15.196:0/3415622481 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6741a0f90 con 0x7fa674102ba0 2024-04-23T10:19:01.419 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:19:01.418+0000 7fa67a53e700 1 -- 172.21.15.196:0/3415622481 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa674069730 con 0x7fa674102ba0 2024-04-23T10:19:01.420 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:19:01.420+0000 7fa6717fa700 1 -- 172.21.15.196:0/3415622481 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7fa668004a30 con 0x7fa674102ba0 2024-04-23T10:19:01.420 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:19:01.420+0000 7fa6717fa700 1 --2- 172.21.15.196:0/3415622481 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fa65c0a3bb0 0x7fa65c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:19:01.421 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:19:01.420+0000 7fa6717fa700 1 -- 172.21.15.196:0/3415622481 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7fa6680cee40 con 0x7fa674102ba0 2024-04-23T10:19:01.421 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:19:01.420+0000 7fa673fff700 1 --2- 172.21.15.196:0/3415622481 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fa65c0a3bb0 0x7fa65c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:19:01.421 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:19:01.421+0000 7fa673fff700 1 --2- 172.21.15.196:0/3415622481 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fa65c0a3bb0 0x7fa65c0a6070 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa674102a00 tx=0x7fa66400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:19:01.423 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:19:01.424+0000 7fa6717fa700 1 -- 172.21.15.196:0/3415622481 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa668020020 con 0x7fa674102ba0 2024-04-23T10:19:01.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:01 smithi196 ceph-mon[32919]: Detected new or changed devices on smithi183 2024-04-23T10:19:01.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:01 smithi196 ceph-mon[32919]: Adjusting osd_memory_target on smithi183 to 17088M 2024-04-23T10:19:01.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:01 smithi196 ceph-mon[32919]: osdmap e14: 2 total, 2 up, 2 in 2024-04-23T10:19:01.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:01 smithi097 ceph-mon[29197]: Detected new or changed devices on smithi183 2024-04-23T10:19:01.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:01 smithi097 ceph-mon[29197]: Adjusting osd_memory_target on smithi183 to 17088M 2024-04-23T10:19:01.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:01 smithi097 ceph-mon[29197]: osdmap e14: 2 total, 2 up, 2 in 2024-04-23T10:19:01.627 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:19:01.629+0000 7fa67a53e700 1 -- 172.21.15.196:0/3415622481 --> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fa674062e80 con 0x7fa65c0a3bb0 2024-04-23T10:19:01.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:01 smithi183 ceph-mon[32317]: Detected new or changed devices on smithi183 2024-04-23T10:19:01.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:01 smithi183 ceph-mon[32317]: Adjusting osd_memory_target on smithi183 to 17088M 2024-04-23T10:19:01.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:01 smithi183 ceph-mon[32317]: osdmap e14: 2 total, 2 up, 2 in 2024-04-23T10:19:02.505 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:02 smithi196 ceph-mon[32919]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-23T10:19:02.505 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:02 smithi196 ceph-mon[32919]: from='client.24154 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:19:02.505 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:02 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T10:19:02.505 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:02 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T10:19:02.505 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:02 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:19:02.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:02 smithi097 ceph-mon[29197]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-23T10:19:02.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:02 smithi097 ceph-mon[29197]: from='client.24154 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:19:02.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:02 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T10:19:02.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:02 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T10:19:02.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:02 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:19:02.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:02 smithi183 ceph-mon[32317]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-23T10:19:02.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:02 smithi183 ceph-mon[32317]: from='client.24154 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:19:02.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:02 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T10:19:02.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:02 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T10:19:02.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:02 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:19:04.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:04 smithi196 ceph-mon[32919]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-23T10:19:04.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:04 smithi196 ceph-mon[32919]: from='client.? 172.21.15.196:0/3741809948' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17c4eec3-6c47-4334-9b63-1d5ad9ec5988"}]: dispatch 2024-04-23T10:19:04.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:04 smithi196 ceph-mon[32919]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17c4eec3-6c47-4334-9b63-1d5ad9ec5988"}]: dispatch 2024-04-23T10:19:04.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:04 smithi196 ceph-mon[32919]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "17c4eec3-6c47-4334-9b63-1d5ad9ec5988"}]': finished 2024-04-23T10:19:04.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:04 smithi196 ceph-mon[32919]: osdmap e15: 3 total, 2 up, 3 in 2024-04-23T10:19:04.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:04 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T10:19:04.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:04 smithi097 ceph-mon[29197]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-23T10:19:04.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:04 smithi097 ceph-mon[29197]: from='client.? 172.21.15.196:0/3741809948' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17c4eec3-6c47-4334-9b63-1d5ad9ec5988"}]: dispatch 2024-04-23T10:19:04.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:04 smithi097 ceph-mon[29197]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17c4eec3-6c47-4334-9b63-1d5ad9ec5988"}]: dispatch 2024-04-23T10:19:04.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:04 smithi097 ceph-mon[29197]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "17c4eec3-6c47-4334-9b63-1d5ad9ec5988"}]': finished 2024-04-23T10:19:04.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:04 smithi097 ceph-mon[29197]: osdmap e15: 3 total, 2 up, 3 in 2024-04-23T10:19:04.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:04 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T10:19:04.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:04 smithi183 ceph-mon[32317]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-23T10:19:04.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:04 smithi183 ceph-mon[32317]: from='client.? 172.21.15.196:0/3741809948' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17c4eec3-6c47-4334-9b63-1d5ad9ec5988"}]: dispatch 2024-04-23T10:19:04.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:04 smithi183 ceph-mon[32317]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17c4eec3-6c47-4334-9b63-1d5ad9ec5988"}]: dispatch 2024-04-23T10:19:04.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:04 smithi183 ceph-mon[32317]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "17c4eec3-6c47-4334-9b63-1d5ad9ec5988"}]': finished 2024-04-23T10:19:04.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:04 smithi183 ceph-mon[32317]: osdmap e15: 3 total, 2 up, 3 in 2024-04-23T10:19:04.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:04 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T10:19:05.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:05 smithi196 ceph-mon[32919]: from='client.? 172.21.15.196:0/2166373772' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T10:19:05.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:05 smithi097 ceph-mon[29197]: from='client.? 172.21.15.196:0/2166373772' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T10:19:05.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:05 smithi183 ceph-mon[32317]: from='client.? 172.21.15.196:0/2166373772' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T10:19:06.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:06 smithi196 ceph-mon[32919]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-23T10:19:06.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:06 smithi097 ceph-mon[29197]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-23T10:19:06.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:06 smithi183 ceph-mon[32317]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-23T10:19:08.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:08 smithi196 ceph-mon[32919]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-23T10:19:08.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:08 smithi097 ceph-mon[29197]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-23T10:19:08.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:08 smithi183 ceph-mon[32317]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-23T10:19:10.775 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:10 smithi196 ceph-mon[32919]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-23T10:19:10.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:10 smithi097 ceph-mon[29197]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-23T10:19:10.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:10 smithi183 ceph-mon[32317]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-23T10:19:12.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:12 smithi196 ceph-mon[32919]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-23T10:19:12.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:12 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-23T10:19:12.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:12 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:19:12.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:12 smithi196 ceph-mon[32919]: Deploying daemon osd.2 on smithi196 2024-04-23T10:19:12.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:12 smithi097 ceph-mon[29197]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-23T10:19:12.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:12 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-23T10:19:12.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:12 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:19:12.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:12 smithi097 ceph-mon[29197]: Deploying daemon osd.2 on smithi196 2024-04-23T10:19:12.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:12 smithi183 ceph-mon[32317]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-23T10:19:12.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:12 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-23T10:19:12.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:12 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:19:12.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:12 smithi183 ceph-mon[32317]: Deploying daemon osd.2 on smithi196 2024-04-23T10:19:14.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:14 smithi196 ceph-mon[32919]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-23T10:19:14.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:14 smithi097 ceph-mon[29197]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-23T10:19:14.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:14 smithi183 ceph-mon[32317]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-23T10:19:16.616 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:16 smithi196 ceph-mon[32919]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-23T10:19:16.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:16 smithi097 ceph-mon[29197]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-23T10:19:16.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:16 smithi183 ceph-mon[32317]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-23T10:19:18.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:18 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:18.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:18 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:18.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:18 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:19:18.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:18 smithi196 ceph-mon[32919]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-23T10:19:18.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:18 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:18.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:18 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:18.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:18 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:19:18.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:18 smithi183 ceph-mon[32317]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-23T10:19:18.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:18 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:18.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:18 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:18.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:18 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:19:18.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:18 smithi097 ceph-mon[29197]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-23T10:19:20.550 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:19:20.549+0000 7fa6717fa700 1 -- 172.21.15.196:0/3415622481 <== mgr.14152 v2:172.21.15.97:6800/3149043749 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa674062e80 con 0x7fa65c0a3bb0 2024-04-23T10:19:20.550 INFO:teuthology.orchestra.run.smithi196.stdout:Created osd(s) 2 on host 'smithi196' 2024-04-23T10:19:20.552 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:19:20.552+0000 7fa67a53e700 1 -- 172.21.15.196:0/3415622481 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fa65c0a3bb0 msgr2=0x7fa65c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:19:20.553 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:19:20.552+0000 7fa67a53e700 1 --2- 172.21.15.196:0/3415622481 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fa65c0a3bb0 0x7fa65c0a6070 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa674102a00 tx=0x7fa66400a380 comp rx=0 tx=0).stop 2024-04-23T10:19:20.553 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:19:20.552+0000 7fa67a53e700 1 -- 172.21.15.196:0/3415622481 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa674102ba0 msgr2=0x7fa67419d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:19:20.553 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:19:20.552+0000 7fa67a53e700 1 --2- 172.21.15.196:0/3415622481 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa674102ba0 0x7fa67419d890 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa66800d980 tx=0x7fa66800dc90 comp rx=0 tx=0).stop 2024-04-23T10:19:20.553 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:19:20.552+0000 7fa67a53e700 1 -- 172.21.15.196:0/3415622481 shutdown_connections 2024-04-23T10:19:20.553 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:19:20.552+0000 7fa67a53e700 1 --2- 172.21.15.196:0/3415622481 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fa65c0a3bb0 0x7fa65c0a6070 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:19:20.553 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:19:20.552+0000 7fa67a53e700 1 --2- 172.21.15.196:0/3415622481 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa674103540 0x7fa67419fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:19:20.553 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:19:20.552+0000 7fa67a53e700 1 --2- 172.21.15.196:0/3415622481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6741019a0 0x7fa67406bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:19:20.553 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:19:20.552+0000 7fa67a53e700 1 --2- 172.21.15.196:0/3415622481 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa674102ba0 0x7fa67419d890 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:19:20.553 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:19:20.552+0000 7fa67a53e700 1 -- 172.21.15.196:0/3415622481 >> 172.21.15.196:0/3415622481 conn(0x7fa6740fcf40 msgr2=0x7fa674105ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:19:20.553 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:19:20.552+0000 7fa67a53e700 1 -- 172.21.15.196:0/3415622481 shutdown_connections 2024-04-23T10:19:20.553 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:19:20.552+0000 7fa67a53e700 1 -- 172.21.15.196:0/3415622481 wait complete. 2024-04-23T10:19:20.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:20 smithi196 ceph-mon[32919]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-23T10:19:20.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:20 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:20.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:20 smithi097 ceph-mon[29197]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-23T10:19:20.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:20 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:20.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:20 smithi183 ceph-mon[32317]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-23T10:19:20.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:20 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:21.230 DEBUG:teuthology.orchestra.run.smithi196:osd.2> sudo journalctl -f -n 0 -u ceph-9b22af88-0159-11ef-bc93-c7b262605968@osd.2.service 2024-04-23T10:19:21.233 INFO:tasks.cephadm:Deploying osd.3 on smithi097 with /dev/vg_nvme/lv_3... 2024-04-23T10:19:21.233 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-23T10:19:21.476 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T10:19:21.838 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:21 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:21.838 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:21 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:21.838 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:21 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:21.839 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:21 smithi097 ceph-mon[29197]: from='osd.2 [v2:172.21.15.196:6800/2877039731,v1:172.21.15.196:6801/2877039731]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-23T10:19:21.839 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:21 smithi097 ceph-mon[29197]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-23T10:19:21.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:21 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:21.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:21 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:21.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:21 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:21.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:21 smithi196 ceph-mon[32919]: from='osd.2 [v2:172.21.15.196:6800/2877039731,v1:172.21.15.196:6801/2877039731]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-23T10:19:21.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:21 smithi196 ceph-mon[32919]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-23T10:19:21.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:21 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:21.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:21 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:21.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:21 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:21.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:21 smithi183 ceph-mon[32317]: from='osd.2 [v2:172.21.15.196:6800/2877039731,v1:172.21.15.196:6801/2877039731]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-23T10:19:21.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:21 smithi183 ceph-mon[32317]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-23T10:19:22.682 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:22 smithi196 ceph-mon[32919]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-23T10:19:22.682 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:22 smithi196 ceph-mon[32919]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-23T10:19:22.683 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:22 smithi196 ceph-mon[32919]: from='osd.2 [v2:172.21.15.196:6800/2877039731,v1:172.21.15.196:6801/2877039731]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-23T10:19:22.683 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:22 smithi196 ceph-mon[32919]: osdmap e16: 3 total, 2 up, 3 in 2024-04-23T10:19:22.683 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:22 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T10:19:22.683 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:22 smithi196 ceph-mon[32919]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-23T10:19:22.683 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 23 10:19:22 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-2[38073]: 2024-04-23T10:19:22.582+0000 7f10812b9700 -1 osd.2 0 waiting for initial osdmap 2024-04-23T10:19:22.683 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 23 10:19:22 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-2[38073]: 2024-04-23T10:19:22.586+0000 7f107d0ac700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T10:19:22.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:22 smithi097 ceph-mon[29197]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-23T10:19:22.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:22 smithi097 ceph-mon[29197]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-23T10:19:22.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:22 smithi097 ceph-mon[29197]: from='osd.2 [v2:172.21.15.196:6800/2877039731,v1:172.21.15.196:6801/2877039731]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-23T10:19:22.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:22 smithi097 ceph-mon[29197]: osdmap e16: 3 total, 2 up, 3 in 2024-04-23T10:19:22.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:22 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T10:19:22.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:22 smithi097 ceph-mon[29197]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-23T10:19:22.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:22 smithi183 ceph-mon[32317]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-23T10:19:22.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:22 smithi183 ceph-mon[32317]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-23T10:19:22.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:22 smithi183 ceph-mon[32317]: from='osd.2 [v2:172.21.15.196:6800/2877039731,v1:172.21.15.196:6801/2877039731]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-23T10:19:22.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:22 smithi183 ceph-mon[32317]: osdmap e16: 3 total, 2 up, 3 in 2024-04-23T10:19:22.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:22 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T10:19:22.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:22 smithi183 ceph-mon[32317]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-23T10:19:23.439 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-23T10:19:23.439 INFO:teuthology.orchestra.run.smithi097.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-23T10:19:23.439 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10+0 records in 2024-04-23T10:19:23.439 INFO:teuthology.orchestra.run.smithi097.stderr:10+0 records out 2024-04-23T10:19:23.439 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0120426 s, 871 MB/s 2024-04-23T10:19:23.439 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping successful for: 2024-04-23T10:19:23.630 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:23 smithi097 ceph-mon[29197]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-04-23T10:19:23.630 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:23 smithi097 ceph-mon[29197]: osdmap e17: 3 total, 2 up, 3 in 2024-04-23T10:19:23.630 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:23 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T10:19:23.630 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:23 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T10:19:23.630 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:23 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T10:19:23.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:23 smithi196 ceph-mon[32919]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-04-23T10:19:23.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:23 smithi196 ceph-mon[32919]: osdmap e17: 3 total, 2 up, 3 in 2024-04-23T10:19:23.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:23 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T10:19:23.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:23 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T10:19:23.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:23 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T10:19:23.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:23 smithi183 ceph-mon[32317]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-04-23T10:19:23.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:23 smithi183 ceph-mon[32317]: osdmap e17: 3 total, 2 up, 3 in 2024-04-23T10:19:23.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:23 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T10:19:23.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:23 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T10:19:23.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:23 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T10:19:24.069 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi097:vg_nvme/lv_3 2024-04-23T10:19:24.318 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T10:19:24.778 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:24 smithi097 ceph-mon[29197]: purged_snaps scrub starts 2024-04-23T10:19:24.778 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:24 smithi097 ceph-mon[29197]: purged_snaps scrub ok 2024-04-23T10:19:24.778 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:24 smithi097 ceph-mon[29197]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-23T10:19:24.778 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:24 smithi097 ceph-mon[29197]: osd.2 [v2:172.21.15.196:6800/2877039731,v1:172.21.15.196:6801/2877039731] boot 2024-04-23T10:19:24.778 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:24 smithi097 ceph-mon[29197]: osdmap e18: 3 total, 3 up, 3 in 2024-04-23T10:19:24.778 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:24 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T10:19:24.778 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:24 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:24.778 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:24 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:24.779 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:24 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:19:24.779 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:24 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:24.779 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:24 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:19:24.779 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:24 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:19:24.779 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:24 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:24.779 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:24 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:19:24.779 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:24 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:19:24.779 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:24 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:19:24.779 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:24 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:24.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:24 smithi183 ceph-mon[32317]: purged_snaps scrub starts 2024-04-23T10:19:24.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:24 smithi183 ceph-mon[32317]: purged_snaps scrub ok 2024-04-23T10:19:24.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:24 smithi183 ceph-mon[32317]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-23T10:19:24.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:24 smithi183 ceph-mon[32317]: osd.2 [v2:172.21.15.196:6800/2877039731,v1:172.21.15.196:6801/2877039731] boot 2024-04-23T10:19:24.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:24 smithi183 ceph-mon[32317]: osdmap e18: 3 total, 3 up, 3 in 2024-04-23T10:19:24.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:24 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T10:19:24.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:24 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:24.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:24 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:24.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:24 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:19:24.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:24 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:24.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:24 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:19:24.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:24 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:19:24.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:24 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:24.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:24 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:19:24.902 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:24 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:19:24.902 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:24 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:19:24.902 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:24 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:25.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:24 smithi196 ceph-mon[32919]: purged_snaps scrub starts 2024-04-23T10:19:25.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:24 smithi196 ceph-mon[32919]: purged_snaps scrub ok 2024-04-23T10:19:25.093 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:24 smithi196 ceph-mon[32919]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-23T10:19:25.093 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:24 smithi196 ceph-mon[32919]: osd.2 [v2:172.21.15.196:6800/2877039731,v1:172.21.15.196:6801/2877039731] boot 2024-04-23T10:19:25.093 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:24 smithi196 ceph-mon[32919]: osdmap e18: 3 total, 3 up, 3 in 2024-04-23T10:19:25.093 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:24 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T10:19:25.093 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:24 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:25.093 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:24 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:25.093 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:24 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:19:25.093 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:24 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:25.094 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:24 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:19:25.094 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:24 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:19:25.094 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:24 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:25.094 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:24 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:19:25.094 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:24 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:19:25.094 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:24 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:19:25.094 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:24 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:25.826 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:25 smithi097 ceph-mon[29197]: Detected new or changed devices on smithi196 2024-04-23T10:19:25.827 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:25 smithi097 ceph-mon[29197]: Adjusting osd_memory_target on smithi196 to 17088M 2024-04-23T10:19:25.827 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:25 smithi097 ceph-mon[29197]: osdmap e19: 3 total, 3 up, 3 in 2024-04-23T10:19:25.827 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:25 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-23T10:19:25.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:25 smithi183 ceph-mon[32317]: Detected new or changed devices on smithi196 2024-04-23T10:19:25.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:25 smithi183 ceph-mon[32317]: Adjusting osd_memory_target on smithi196 to 17088M 2024-04-23T10:19:25.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:25 smithi183 ceph-mon[32317]: osdmap e19: 3 total, 3 up, 3 in 2024-04-23T10:19:25.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:25 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-23T10:19:25.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:19:25.934+0000 7fca6938b700 1 -- 172.21.15.97:0/3720021669 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca64103530 msgr2=0x7fca64105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:19:25.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:19:25.934+0000 7fca6938b700 1 --2- 172.21.15.97:0/3720021669 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca64103530 0x7fca64105920 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fca54009990 tx=0x7fca54009ca0 comp rx=0 tx=0).stop 2024-04-23T10:19:25.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:19:25.934+0000 7fca6938b700 1 -- 172.21.15.97:0/3720021669 shutdown_connections 2024-04-23T10:19:25.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:19:25.934+0000 7fca6938b700 1 --2- 172.21.15.97:0/3720021669 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca64108790 0x7fca6410ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:19:25.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:19:25.934+0000 7fca6938b700 1 --2- 172.21.15.97:0/3720021669 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca64103530 0x7fca64105920 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:19:25.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:19:25.934+0000 7fca6938b700 1 --2- 172.21.15.97:0/3720021669 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca64105e60 0x7fca64108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:19:25.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:19:25.934+0000 7fca6938b700 1 -- 172.21.15.97:0/3720021669 >> 172.21.15.97:0/3720021669 conn(0x7fca640fcf40 msgr2=0x7fca640ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:19:25.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:19:25.934+0000 7fca6938b700 1 -- 172.21.15.97:0/3720021669 shutdown_connections 2024-04-23T10:19:25.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:19:25.934+0000 7fca6938b700 1 -- 172.21.15.97:0/3720021669 wait complete. 2024-04-23T10:19:25.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:19:25.934+0000 7fca6938b700 1 Processor -- start 2024-04-23T10:19:25.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:19:25.935+0000 7fca6938b700 1 -- start start 2024-04-23T10:19:25.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:19:25.935+0000 7fca6938b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca64103530 0x7fca64198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:19:25.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:19:25.935+0000 7fca6938b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca64105e60 0x7fca641990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:19:25.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:19:25.935+0000 7fca6938b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca64108790 0x7fca641a0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:19:25.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:19:25.935+0000 7fca6938b700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fca6410e520 con 0x7fca64105e60 2024-04-23T10:19:25.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:19:25.935+0000 7fca6938b700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fca6410e3a0 con 0x7fca64103530 2024-04-23T10:19:25.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:19:25.935+0000 7fca6938b700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fca6410e6a0 con 0x7fca64108790 2024-04-23T10:19:25.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:19:25.935+0000 7fca5a7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca64105e60 0x7fca641990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:19:25.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:19:25.935+0000 7fca5a7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca64105e60 0x7fca641990f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48966/0 (socket says 172.21.15.97:48966) 2024-04-23T10:19:25.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:19:25.935+0000 7fca5a7fc700 1 -- 172.21.15.97:0/3768466500 learned_addr learned my addr 172.21.15.97:0/3768466500 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:19:25.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:19:25.935+0000 7fca637fe700 1 --2- 172.21.15.97:0/3768466500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca64108790 0x7fca641a0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:19:25.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:19:25.935+0000 7fca62ffd700 1 --2- 172.21.15.97:0/3768466500 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca64103530 0x7fca64198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:19:25.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:19:25.935+0000 7fca5a7fc700 1 -- 172.21.15.97:0/3768466500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca64108790 msgr2=0x7fca641a0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:19:25.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:19:25.935+0000 7fca5a7fc700 1 --2- 172.21.15.97:0/3768466500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca64108790 0x7fca641a0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:19:25.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:19:25.935+0000 7fca5a7fc700 1 -- 172.21.15.97:0/3768466500 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca64103530 msgr2=0x7fca64198bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:19:25.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:19:25.935+0000 7fca5a7fc700 1 --2- 172.21.15.97:0/3768466500 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca64103530 0x7fca64198bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:19:25.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:19:25.935+0000 7fca5a7fc700 1 -- 172.21.15.97:0/3768466500 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca641a0810 con 0x7fca64105e60 2024-04-23T10:19:25.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:19:25.935+0000 7fca62ffd700 1 --2- 172.21.15.97:0/3768466500 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca64103530 0x7fca64198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:19:25.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:19:25.936+0000 7fca5a7fc700 1 --2- 172.21.15.97:0/3768466500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca64105e60 0x7fca641990f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fca4c00d980 tx=0x7fca4c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:19:25.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:19:25.936+0000 7fca60ff9700 1 -- 172.21.15.97:0/3768466500 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca4c016070 con 0x7fca64105e60 2024-04-23T10:19:25.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:19:25.936+0000 7fca60ff9700 1 -- 172.21.15.97:0/3768466500 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca4c00de70 con 0x7fca64105e60 2024-04-23T10:19:25.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:19:25.936+0000 7fca6938b700 1 -- 172.21.15.97:0/3768466500 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca641a0b00 con 0x7fca64105e60 2024-04-23T10:19:25.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:19:25.936+0000 7fca60ff9700 1 -- 172.21.15.97:0/3768466500 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca4c00f3c0 con 0x7fca64105e60 2024-04-23T10:19:25.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:19:25.936+0000 7fca6938b700 1 -- 172.21.15.97:0/3768466500 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca641a1090 con 0x7fca64105e60 2024-04-23T10:19:25.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:19:25.937+0000 7fca6938b700 1 -- 172.21.15.97:0/3768466500 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca64069730 con 0x7fca64105e60 2024-04-23T10:19:25.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:19:25.938+0000 7fca60ff9700 1 -- 172.21.15.97:0/3768466500 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7fca4c019070 con 0x7fca64105e60 2024-04-23T10:19:25.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:19:25.939+0000 7fca60ff9700 1 --2- 172.21.15.97:0/3768466500 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fca500a3c30 0x7fca500a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:19:25.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:19:25.939+0000 7fca62ffd700 1 --2- 172.21.15.97:0/3768466500 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fca500a3c30 0x7fca500a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:19:25.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:19:25.939+0000 7fca60ff9700 1 -- 172.21.15.97:0/3768466500 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 3226+0+0 (secure 0 0 0) 0x7fca4c0ce9b0 con 0x7fca64105e60 2024-04-23T10:19:25.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:19:25.939+0000 7fca62ffd700 1 --2- 172.21.15.97:0/3768466500 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fca500a3c30 0x7fca500a60f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fca54005cd0 tx=0x7fca54005c20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:19:25.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:19:25.943+0000 7fca60ff9700 1 -- 172.21.15.97:0/3768466500 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fca4c09c850 con 0x7fca64105e60 2024-04-23T10:19:26.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:25 smithi196 ceph-mon[32919]: Detected new or changed devices on smithi196 2024-04-23T10:19:26.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:25 smithi196 ceph-mon[32919]: Adjusting osd_memory_target on smithi196 to 17088M 2024-04-23T10:19:26.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:25 smithi196 ceph-mon[32919]: osdmap e19: 3 total, 3 up, 3 in 2024-04-23T10:19:26.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:25 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-23T10:19:26.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:19:26.149+0000 7fca6938b700 1 -- 172.21.15.97:0/3768466500 --> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fca64062e80 con 0x7fca500a3c30 2024-04-23T10:19:26.861 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:26 smithi196 ceph-mon[32919]: pgmap v82: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-23T10:19:26.861 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:26 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-23T10:19:26.861 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:26 smithi196 ceph-mon[32919]: osdmap e20: 3 total, 3 up, 3 in 2024-04-23T10:19:26.861 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:26 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-23T10:19:26.861 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:26 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T10:19:26.862 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:26 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T10:19:26.862 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:26 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:19:26.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:26 smithi097 ceph-mon[29197]: pgmap v82: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-23T10:19:26.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:26 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-23T10:19:26.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:26 smithi097 ceph-mon[29197]: osdmap e20: 3 total, 3 up, 3 in 2024-04-23T10:19:26.863 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:26 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-23T10:19:26.863 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:26 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T10:19:26.863 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:26 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T10:19:26.863 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:26 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:19:26.863 INFO:journalctl@ceph.osd.0.smithi097.stdout:Apr 23 10:19:26 smithi097 sudo[45815]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-23T10:19:26.863 INFO:journalctl@ceph.osd.0.smithi097.stdout:Apr 23 10:19:26 smithi097 sudo[45815]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T10:19:26.863 INFO:journalctl@ceph.osd.0.smithi097.stdout:Apr 23 10:19:26 smithi097 sudo[45815]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T10:19:26.863 INFO:journalctl@ceph.osd.0.smithi097.stdout:Apr 23 10:19:26 smithi097 sudo[45815]: pam_unix(sudo:session): session closed for user root 2024-04-23T10:19:26.864 INFO:journalctl@ceph.osd.0.smithi097.stdout:Apr 23 10:19:26 smithi097 sudo[45818]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-23T10:19:26.864 INFO:journalctl@ceph.osd.0.smithi097.stdout:Apr 23 10:19:26 smithi097 sudo[45818]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T10:19:26.864 INFO:journalctl@ceph.osd.0.smithi097.stdout:Apr 23 10:19:26 smithi097 sudo[45818]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T10:19:26.864 INFO:journalctl@ceph.osd.0.smithi097.stdout:Apr 23 10:19:26 smithi097 sudo[45818]: pam_unix(sudo:session): session closed for user root 2024-04-23T10:19:26.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:26 smithi183 ceph-mon[32317]: pgmap v82: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-23T10:19:26.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:26 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-23T10:19:26.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:26 smithi183 ceph-mon[32317]: osdmap e20: 3 total, 3 up, 3 in 2024-04-23T10:19:26.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:26 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-23T10:19:26.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:26 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T10:19:26.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:26 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T10:19:26.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:26 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:19:26.901 INFO:journalctl@ceph.osd.1.smithi183.stdout:Apr 23 10:19:26 smithi183 sudo[41040]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-23T10:19:26.901 INFO:journalctl@ceph.osd.1.smithi183.stdout:Apr 23 10:19:26 smithi183 sudo[41040]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T10:19:26.901 INFO:journalctl@ceph.osd.1.smithi183.stdout:Apr 23 10:19:26 smithi183 sudo[41040]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T10:19:26.901 INFO:journalctl@ceph.osd.1.smithi183.stdout:Apr 23 10:19:26 smithi183 sudo[41040]: pam_unix(sudo:session): session closed for user root 2024-04-23T10:19:26.901 INFO:journalctl@ceph.osd.1.smithi183.stdout:Apr 23 10:19:26 smithi183 sudo[41043]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-23T10:19:26.901 INFO:journalctl@ceph.osd.1.smithi183.stdout:Apr 23 10:19:26 smithi183 sudo[41043]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T10:19:26.901 INFO:journalctl@ceph.osd.1.smithi183.stdout:Apr 23 10:19:26 smithi183 sudo[41043]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T10:19:26.902 INFO:journalctl@ceph.osd.1.smithi183.stdout:Apr 23 10:19:26 smithi183 sudo[41043]: pam_unix(sudo:session): session closed for user root 2024-04-23T10:19:27.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:26 smithi196 sudo[41826]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-23T10:19:27.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:26 smithi196 sudo[41826]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T10:19:27.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:26 smithi196 sudo[41826]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T10:19:27.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:26 smithi196 sudo[41826]: pam_unix(sudo:session): session closed for user root 2024-04-23T10:19:27.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:26 smithi196 sudo[41829]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-23T10:19:27.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:26 smithi196 sudo[41829]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T10:19:27.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:26 smithi196 sudo[41829]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T10:19:27.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:26 smithi196 sudo[41829]: pam_unix(sudo:session): session closed for user root 2024-04-23T10:19:27.343 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 23 10:19:26 smithi196 sudo[41820]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-23T10:19:27.343 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 23 10:19:26 smithi196 sudo[41820]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T10:19:27.343 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 23 10:19:26 smithi196 sudo[41820]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T10:19:27.343 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 23 10:19:26 smithi196 sudo[41820]: pam_unix(sudo:session): session closed for user root 2024-04-23T10:19:27.343 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 23 10:19:26 smithi196 sudo[41823]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-23T10:19:27.344 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 23 10:19:26 smithi196 sudo[41823]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T10:19:27.344 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 23 10:19:26 smithi196 sudo[41823]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T10:19:27.344 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 23 10:19:26 smithi196 sudo[41823]: pam_unix(sudo:session): session closed for user root 2024-04-23T10:19:27.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:26 smithi097 sudo[45826]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-23T10:19:27.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:26 smithi097 sudo[45826]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T10:19:27.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:26 smithi097 sudo[45826]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T10:19:27.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:26 smithi097 sudo[45826]: pam_unix(sudo:session): session closed for user root 2024-04-23T10:19:27.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:26 smithi097 sudo[45829]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-23T10:19:27.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:26 smithi097 sudo[45829]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T10:19:27.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:26 smithi097 sudo[45829]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T10:19:27.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:26 smithi097 sudo[45829]: pam_unix(sudo:session): session closed for user root 2024-04-23T10:19:27.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:26 smithi183 sudo[41046]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-23T10:19:27.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:26 smithi183 sudo[41046]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T10:19:27.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:26 smithi183 sudo[41046]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T10:19:27.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:27 smithi183 sudo[41046]: pam_unix(sudo:session): session closed for user root 2024-04-23T10:19:27.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:27 smithi183 sudo[41049]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-23T10:19:27.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:27 smithi183 sudo[41049]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T10:19:27.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:27 smithi183 sudo[41049]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T10:19:27.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:27 smithi183 sudo[41049]: pam_unix(sudo:session): session closed for user root 2024-04-23T10:19:27.759 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:27 smithi097 ceph-mon[29197]: from='client.14298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:19:27.759 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:27 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-23T10:19:27.760 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:27 smithi097 ceph-mon[29197]: osdmap e21: 3 total, 3 up, 3 in 2024-04-23T10:19:27.760 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:27 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:27.760 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:27 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:27.760 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:27 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:27.760 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:27 smithi097 ceph-mon[29197]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-23T10:19:27.760 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:27 smithi097 ceph-mon[29197]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-23T10:19:27.760 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:27 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T10:19:27.760 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:27 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T10:19:27.760 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:27 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T10:19:27.760 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:27 smithi097 ceph-mon[29197]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-23T10:19:27.761 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:27 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T10:19:27.761 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:27 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T10:19:27.761 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:27 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T10:19:27.761 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:27 smithi097 ceph-mon[29197]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-23T10:19:27.761 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:27 smithi097 ceph-mon[29197]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-23T10:19:27.761 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:27 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T10:19:27.761 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:27 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T10:19:27.761 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:27 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T10:19:27.761 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:27 smithi097 ceph-mon[29197]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-23T10:19:27.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:27 smithi183 ceph-mon[32317]: from='client.14298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:19:27.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:27 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-23T10:19:27.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:27 smithi183 ceph-mon[32317]: osdmap e21: 3 total, 3 up, 3 in 2024-04-23T10:19:27.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:27 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:27.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:27 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:27.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:27 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:27.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:27 smithi183 ceph-mon[32317]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-23T10:19:27.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:27 smithi183 ceph-mon[32317]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-23T10:19:27.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:27 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T10:19:27.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:27 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T10:19:27.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:27 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T10:19:27.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:27 smithi183 ceph-mon[32317]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-23T10:19:27.902 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:27 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T10:19:27.902 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:27 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T10:19:27.902 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:27 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T10:19:27.902 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:27 smithi183 ceph-mon[32317]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-23T10:19:27.902 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:27 smithi183 ceph-mon[32317]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-23T10:19:27.902 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:27 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T10:19:27.902 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:27 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T10:19:27.902 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:27 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T10:19:27.902 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:27 smithi183 ceph-mon[32317]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-23T10:19:27.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:19:27.921+0000 7fca60ff9700 1 -- 172.21.15.97:0/3768466500 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fca4c0987c0 con 0x7fca64105e60 2024-04-23T10:19:28.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:27 smithi196 ceph-mon[32919]: from='client.14298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:19:28.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:27 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-23T10:19:28.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:27 smithi196 ceph-mon[32919]: osdmap e21: 3 total, 3 up, 3 in 2024-04-23T10:19:28.093 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:27 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:28.093 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:27 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:28.093 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:27 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:28.093 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:27 smithi196 ceph-mon[32919]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-23T10:19:28.093 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:27 smithi196 ceph-mon[32919]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-23T10:19:28.093 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:27 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T10:19:28.093 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:27 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T10:19:28.093 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:27 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T10:19:28.093 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:27 smithi196 ceph-mon[32919]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-23T10:19:28.093 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:27 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T10:19:28.093 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:27 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T10:19:28.093 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:27 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T10:19:28.093 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:27 smithi196 ceph-mon[32919]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-23T10:19:28.094 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:27 smithi196 ceph-mon[32919]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-23T10:19:28.094 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:27 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T10:19:28.094 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:27 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T10:19:28.094 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:27 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T10:19:28.094 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:27 smithi196 ceph-mon[32919]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-23T10:19:28.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:28 smithi183 ceph-mon[32317]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-23T10:19:28.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:28 smithi183 ceph-mon[32317]: osdmap e22: 3 total, 3 up, 3 in 2024-04-23T10:19:28.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:28 smithi183 ceph-mon[32317]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-23T10:19:29.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:28 smithi196 ceph-mon[32919]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-23T10:19:29.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:28 smithi196 ceph-mon[32919]: osdmap e22: 3 total, 3 up, 3 in 2024-04-23T10:19:29.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:28 smithi196 ceph-mon[32919]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-23T10:19:29.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:28 smithi097 ceph-mon[29197]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-23T10:19:29.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:28 smithi097 ceph-mon[29197]: osdmap e22: 3 total, 3 up, 3 in 2024-04-23T10:19:29.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:28 smithi097 ceph-mon[29197]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-23T10:19:29.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:29 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/401729421' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3f731c38-2e45-4ca1-8dce-efc4a1b2166b"}]: dispatch 2024-04-23T10:19:29.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:29 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/401729421' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3f731c38-2e45-4ca1-8dce-efc4a1b2166b"}]': finished 2024-04-23T10:19:29.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:29 smithi183 ceph-mon[32317]: osdmap e23: 4 total, 3 up, 4 in 2024-04-23T10:19:29.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:29 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T10:19:29.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:29 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/1672128711' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T10:19:30.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:29 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/401729421' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3f731c38-2e45-4ca1-8dce-efc4a1b2166b"}]: dispatch 2024-04-23T10:19:30.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:29 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/401729421' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3f731c38-2e45-4ca1-8dce-efc4a1b2166b"}]': finished 2024-04-23T10:19:30.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:29 smithi196 ceph-mon[32919]: osdmap e23: 4 total, 3 up, 4 in 2024-04-23T10:19:30.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:29 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T10:19:30.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:29 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/1672128711' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T10:19:30.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:29 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/401729421' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3f731c38-2e45-4ca1-8dce-efc4a1b2166b"}]: dispatch 2024-04-23T10:19:30.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:29 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/401729421' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3f731c38-2e45-4ca1-8dce-efc4a1b2166b"}]': finished 2024-04-23T10:19:30.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:29 smithi097 ceph-mon[29197]: osdmap e23: 4 total, 3 up, 4 in 2024-04-23T10:19:30.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:29 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T10:19:30.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:29 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/1672128711' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T10:19:30.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:30 smithi183 ceph-mon[32317]: pgmap v88: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-23T10:19:31.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:30 smithi196 ceph-mon[32919]: pgmap v88: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-23T10:19:31.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:30 smithi097 ceph-mon[29197]: pgmap v88: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-23T10:19:32.822 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:32 smithi097 ceph-mon[29197]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-23T10:19:32.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:32 smithi183 ceph-mon[32317]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-23T10:19:33.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:32 smithi196 ceph-mon[32919]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-23T10:19:34.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:34 smithi183 ceph-mon[32317]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-23T10:19:34.924 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:34 smithi097 ceph-mon[29197]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-23T10:19:35.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:34 smithi196 ceph-mon[32919]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-23T10:19:36.898 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:36 smithi097 ceph-mon[29197]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-23T10:19:36.898 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:36 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-23T10:19:36.898 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:36 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:19:36.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:36 smithi183 ceph-mon[32317]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-23T10:19:36.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:36 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-23T10:19:36.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:36 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:19:37.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:36 smithi196 ceph-mon[32919]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-23T10:19:37.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:36 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-23T10:19:37.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:36 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:19:37.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:37 smithi183 ceph-mon[32317]: Deploying daemon osd.3 on smithi097 2024-04-23T10:19:37.977 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:37 smithi097 ceph-mon[29197]: Deploying daemon osd.3 on smithi097 2024-04-23T10:19:38.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:37 smithi196 ceph-mon[32919]: Deploying daemon osd.3 on smithi097 2024-04-23T10:19:38.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:38 smithi183 ceph-mon[32317]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-23T10:19:39.041 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:38 smithi097 ceph-mon[29197]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-23T10:19:39.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:38 smithi196 ceph-mon[32919]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-23T10:19:39.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:39 smithi183 ceph-mon[32317]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-23T10:19:39.993 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:39 smithi196 ceph-mon[32919]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-23T10:19:40.005 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:39 smithi097 ceph-mon[29197]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-23T10:19:42.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:42 smithi097 ceph-mon[29197]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-23T10:19:42.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:42 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:42.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:42 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:42.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:42 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:19:42.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:42 smithi183 ceph-mon[32317]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-23T10:19:42.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:42 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:42.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:42 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:42.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:42 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:19:43.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:42 smithi196 ceph-mon[32919]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-23T10:19:43.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:42 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:43.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:42 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:43.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:42 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:19:44.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:44 smithi097 ceph-mon[29197]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-23T10:19:44.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:44 smithi183 ceph-mon[32317]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-23T10:19:45.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:44 smithi196 ceph-mon[32919]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-23T10:19:45.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:19:45.509+0000 7fca60ff9700 1 -- 172.21.15.97:0/3768466500 <== mgr.14152 v2:172.21.15.97:6800/3149043749 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fca64062e80 con 0x7fca500a3c30 2024-04-23T10:19:45.509 INFO:teuthology.orchestra.run.smithi097.stdout:Created osd(s) 3 on host 'smithi097' 2024-04-23T10:19:45.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:19:45.512+0000 7fca6938b700 1 -- 172.21.15.97:0/3768466500 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fca500a3c30 msgr2=0x7fca500a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:19:45.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:19:45.512+0000 7fca6938b700 1 --2- 172.21.15.97:0/3768466500 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fca500a3c30 0x7fca500a60f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fca54005cd0 tx=0x7fca54005c20 comp rx=0 tx=0).stop 2024-04-23T10:19:45.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:19:45.512+0000 7fca6938b700 1 -- 172.21.15.97:0/3768466500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca64105e60 msgr2=0x7fca641990f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:19:45.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:19:45.512+0000 7fca6938b700 1 --2- 172.21.15.97:0/3768466500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca64105e60 0x7fca641990f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fca4c00d980 tx=0x7fca4c00dc90 comp rx=0 tx=0).stop 2024-04-23T10:19:45.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:19:45.512+0000 7fca6938b700 1 -- 172.21.15.97:0/3768466500 shutdown_connections 2024-04-23T10:19:45.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:19:45.512+0000 7fca6938b700 1 --2- 172.21.15.97:0/3768466500 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fca500a3c30 0x7fca500a60f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:19:45.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:19:45.512+0000 7fca6938b700 1 --2- 172.21.15.97:0/3768466500 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca64108790 0x7fca641a0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:19:45.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:19:45.512+0000 7fca6938b700 1 --2- 172.21.15.97:0/3768466500 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca64103530 0x7fca64198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:19:45.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:19:45.512+0000 7fca6938b700 1 --2- 172.21.15.97:0/3768466500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca64105e60 0x7fca641990f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:19:45.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:19:45.512+0000 7fca6938b700 1 -- 172.21.15.97:0/3768466500 >> 172.21.15.97:0/3768466500 conn(0x7fca640fcf40 msgr2=0x7fca64103df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:19:45.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:19:45.512+0000 7fca6938b700 1 -- 172.21.15.97:0/3768466500 shutdown_connections 2024-04-23T10:19:45.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:19:45.512+0000 7fca6938b700 1 -- 172.21.15.97:0/3768466500 wait complete. 2024-04-23T10:19:46.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:45 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:46.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:45 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:46.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:45 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:19:46.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:45 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:19:46.093 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:45 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:46.093 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:45 smithi196 ceph-mon[32919]: from='osd.3 [v2:172.21.15.97:6810/4276311194,v1:172.21.15.97:6811/4276311194]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-23T10:19:46.093 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:45 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:46.093 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:45 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:19:46.093 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:45 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:46.093 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:45 smithi196 ceph-mon[32919]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-23T10:19:46.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:45 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:46.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:45 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:46.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:45 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:19:46.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:45 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:19:46.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:45 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:46.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:45 smithi097 ceph-mon[29197]: from='osd.3 [v2:172.21.15.97:6810/4276311194,v1:172.21.15.97:6811/4276311194]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-23T10:19:46.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:45 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:46.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:45 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:19:46.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:45 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:46.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:45 smithi097 ceph-mon[29197]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-23T10:19:46.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:45 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:46.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:45 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:46.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:45 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:19:46.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:45 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:19:46.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:45 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:46.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:45 smithi183 ceph-mon[32317]: from='osd.3 [v2:172.21.15.97:6810/4276311194,v1:172.21.15.97:6811/4276311194]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-23T10:19:46.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:45 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:46.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:45 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:19:46.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:45 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:46.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:45 smithi183 ceph-mon[32317]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-23T10:19:46.217 DEBUG:teuthology.orchestra.run.smithi097:osd.3> sudo journalctl -f -n 0 -u ceph-9b22af88-0159-11ef-bc93-c7b262605968@osd.3.service 2024-04-23T10:19:46.220 INFO:tasks.cephadm:Deploying osd.4 on smithi183 with /dev/vg_nvme/lv_3... 2024-04-23T10:19:46.220 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-23T10:19:46.465 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.b/config 2024-04-23T10:19:46.885 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:46 smithi183 ceph-mon[32317]: from='osd.3 [v2:172.21.15.97:6810/4276311194,v1:172.21.15.97:6811/4276311194]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-23T10:19:46.885 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:46 smithi183 ceph-mon[32317]: osdmap e24: 4 total, 3 up, 4 in 2024-04-23T10:19:46.885 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:46 smithi183 ceph-mon[32317]: from='osd.3 [v2:172.21.15.97:6810/4276311194,v1:172.21.15.97:6811/4276311194]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-23T10:19:46.885 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:46 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T10:19:47.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:46 smithi196 ceph-mon[32919]: from='osd.3 [v2:172.21.15.97:6810/4276311194,v1:172.21.15.97:6811/4276311194]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-23T10:19:47.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:46 smithi196 ceph-mon[32919]: osdmap e24: 4 total, 3 up, 4 in 2024-04-23T10:19:47.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:46 smithi196 ceph-mon[32919]: from='osd.3 [v2:172.21.15.97:6810/4276311194,v1:172.21.15.97:6811/4276311194]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-23T10:19:47.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:46 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T10:19:47.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:46 smithi097 ceph-mon[29197]: from='osd.3 [v2:172.21.15.97:6810/4276311194,v1:172.21.15.97:6811/4276311194]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-23T10:19:47.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:46 smithi097 ceph-mon[29197]: osdmap e24: 4 total, 3 up, 4 in 2024-04-23T10:19:47.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:46 smithi097 ceph-mon[29197]: from='osd.3 [v2:172.21.15.97:6810/4276311194,v1:172.21.15.97:6811/4276311194]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-23T10:19:47.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:46 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T10:19:47.587 INFO:journalctl@ceph.osd.3.smithi097.stdout:Apr 23 10:19:47 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-3[47966]: 2024-04-23T10:19:47.212+0000 7f4c23c7b700 -1 osd.3 0 waiting for initial osdmap 2024-04-23T10:19:47.588 INFO:journalctl@ceph.osd.3.smithi097.stdout:Apr 23 10:19:47 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-3[47966]: 2024-04-23T10:19:47.217+0000 7f4c1ba66700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T10:19:47.853 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:47 smithi097 ceph-mon[29197]: from='osd.3 [v2:172.21.15.97:6810/4276311194,v1:172.21.15.97:6811/4276311194]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-23T10:19:47.853 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:47 smithi097 ceph-mon[29197]: osdmap e25: 4 total, 3 up, 4 in 2024-04-23T10:19:47.853 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:47 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T10:19:47.853 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:47 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T10:19:47.853 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:47 smithi097 ceph-mon[29197]: from='osd.3 [v2:172.21.15.97:6810/4276311194,v1:172.21.15.97:6811/4276311194]' entity='osd.3' 2024-04-23T10:19:47.853 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:47 smithi097 ceph-mon[29197]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-23T10:19:48.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:47 smithi196 ceph-mon[32919]: from='osd.3 [v2:172.21.15.97:6810/4276311194,v1:172.21.15.97:6811/4276311194]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-23T10:19:48.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:47 smithi196 ceph-mon[32919]: osdmap e25: 4 total, 3 up, 4 in 2024-04-23T10:19:48.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:47 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T10:19:48.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:47 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T10:19:48.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:47 smithi196 ceph-mon[32919]: from='osd.3 [v2:172.21.15.97:6810/4276311194,v1:172.21.15.97:6811/4276311194]' entity='osd.3' 2024-04-23T10:19:48.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:47 smithi196 ceph-mon[32919]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-23T10:19:48.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:47 smithi183 ceph-mon[32317]: from='osd.3 [v2:172.21.15.97:6810/4276311194,v1:172.21.15.97:6811/4276311194]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-23T10:19:48.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:47 smithi183 ceph-mon[32317]: osdmap e25: 4 total, 3 up, 4 in 2024-04-23T10:19:48.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:47 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T10:19:48.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:47 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T10:19:48.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:47 smithi183 ceph-mon[32317]: from='osd.3 [v2:172.21.15.97:6810/4276311194,v1:172.21.15.97:6811/4276311194]' entity='osd.3' 2024-04-23T10:19:48.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:47 smithi183 ceph-mon[32317]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-23T10:19:48.438 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-23T10:19:48.438 INFO:teuthology.orchestra.run.smithi183.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-23T10:19:48.438 INFO:teuthology.orchestra.run.smithi183.stderr: stderr: 10+0 records in 2024-04-23T10:19:48.438 INFO:teuthology.orchestra.run.smithi183.stderr:10+0 records out 2024-04-23T10:19:48.438 INFO:teuthology.orchestra.run.smithi183.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0130838 s, 801 MB/s 2024-04-23T10:19:48.438 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping successful for: 2024-04-23T10:19:49.037 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:48 smithi183 ceph-mon[32317]: purged_snaps scrub starts 2024-04-23T10:19:49.037 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:48 smithi183 ceph-mon[32317]: purged_snaps scrub ok 2024-04-23T10:19:49.037 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:48 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T10:19:49.037 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:48 smithi183 ceph-mon[32317]: osd.3 [v2:172.21.15.97:6810/4276311194,v1:172.21.15.97:6811/4276311194] boot 2024-04-23T10:19:49.037 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:48 smithi183 ceph-mon[32317]: osdmap e26: 4 total, 4 up, 4 in 2024-04-23T10:19:49.037 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:48 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T10:19:49.065 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi183:vg_nvme/lv_3 2024-04-23T10:19:49.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:48 smithi196 ceph-mon[32919]: purged_snaps scrub starts 2024-04-23T10:19:49.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:48 smithi196 ceph-mon[32919]: purged_snaps scrub ok 2024-04-23T10:19:49.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:48 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T10:19:49.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:48 smithi196 ceph-mon[32919]: osd.3 [v2:172.21.15.97:6810/4276311194,v1:172.21.15.97:6811/4276311194] boot 2024-04-23T10:19:49.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:48 smithi196 ceph-mon[32919]: osdmap e26: 4 total, 4 up, 4 in 2024-04-23T10:19:49.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:48 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T10:19:49.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:48 smithi097 ceph-mon[29197]: purged_snaps scrub starts 2024-04-23T10:19:49.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:48 smithi097 ceph-mon[29197]: purged_snaps scrub ok 2024-04-23T10:19:49.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:48 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T10:19:49.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:48 smithi097 ceph-mon[29197]: osd.3 [v2:172.21.15.97:6810/4276311194,v1:172.21.15.97:6811/4276311194] boot 2024-04-23T10:19:49.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:48 smithi097 ceph-mon[29197]: osdmap e26: 4 total, 4 up, 4 in 2024-04-23T10:19:49.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:48 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T10:19:49.311 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.b/config 2024-04-23T10:19:50.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:49 smithi196 ceph-mon[32919]: osdmap e27: 4 total, 4 up, 4 in 2024-04-23T10:19:50.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:49 smithi196 ceph-mon[32919]: Detected new or changed devices on smithi097 2024-04-23T10:19:50.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:49 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:50.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:49 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:50.093 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:49 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:19:50.093 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:49 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:19:50.093 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:49 smithi196 ceph-mon[32919]: Adjusting osd_memory_target on smithi097 to 8544M 2024-04-23T10:19:50.093 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:49 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:50.093 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:49 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:19:50.093 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:49 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:19:50.093 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:49 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:50.093 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:49 smithi196 ceph-mon[32919]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-23T10:19:50.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:49 smithi097 ceph-mon[29197]: osdmap e27: 4 total, 4 up, 4 in 2024-04-23T10:19:50.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:49 smithi097 ceph-mon[29197]: Detected new or changed devices on smithi097 2024-04-23T10:19:50.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:49 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:50.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:49 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:50.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:49 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:19:50.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:49 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:19:50.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:49 smithi097 ceph-mon[29197]: Adjusting osd_memory_target on smithi097 to 8544M 2024-04-23T10:19:50.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:49 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:50.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:49 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:19:50.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:49 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:19:50.112 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:49 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:50.112 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:49 smithi097 ceph-mon[29197]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-23T10:19:50.135 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:49 smithi183 ceph-mon[32317]: osdmap e27: 4 total, 4 up, 4 in 2024-04-23T10:19:50.135 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:49 smithi183 ceph-mon[32317]: Detected new or changed devices on smithi097 2024-04-23T10:19:50.135 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:49 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:50.135 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:49 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:50.135 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:49 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:19:50.135 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:49 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:19:50.136 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:49 smithi183 ceph-mon[32317]: Adjusting osd_memory_target on smithi097 to 8544M 2024-04-23T10:19:50.136 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:49 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:50.136 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:49 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:19:50.136 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:49 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:19:50.136 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:49 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:19:50.136 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:49 smithi183 ceph-mon[32317]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-23T10:19:50.989 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:19:50.985+0000 7f734c66e700 1 -- 172.21.15.183:0/1246006705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7344103550 msgr2=0x7f7344109e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:19:50.989 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:19:50.985+0000 7f734c66e700 1 --2- 172.21.15.183:0/1246006705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7344103550 0x7f7344109e00 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f733c00b0d0 tx=0x7f733c00b3e0 comp rx=0 tx=0).stop 2024-04-23T10:19:50.989 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:19:50.986+0000 7f734c66e700 1 -- 172.21.15.183:0/1246006705 shutdown_connections 2024-04-23T10:19:50.989 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:19:50.986+0000 7f734c66e700 1 --2- 172.21.15.183:0/1246006705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7344103550 0x7f7344109e00 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:19:50.989 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:19:50.986+0000 7f734c66e700 1 --2- 172.21.15.183:0/1246006705 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73441019b0 0x7f7344101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:19:50.990 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:19:50.986+0000 7f734c66e700 1 --2- 172.21.15.183:0/1246006705 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7344102bb0 0x7f7344103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:19:50.990 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:19:50.986+0000 7f734c66e700 1 -- 172.21.15.183:0/1246006705 >> 172.21.15.183:0/1246006705 conn(0x7f73440fcf70 msgr2=0x7f73440ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:19:50.990 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:19:50.986+0000 7f734c66e700 1 -- 172.21.15.183:0/1246006705 shutdown_connections 2024-04-23T10:19:50.990 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:19:50.986+0000 7f734c66e700 1 -- 172.21.15.183:0/1246006705 wait complete. 2024-04-23T10:19:50.990 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:19:50.986+0000 7f734c66e700 1 Processor -- start 2024-04-23T10:19:50.990 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:19:50.986+0000 7f734c66e700 1 -- start start 2024-04-23T10:19:50.990 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:19:50.986+0000 7f734c66e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73441019b0 0x7f7344197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:19:50.990 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:19:50.986+0000 7f734c66e700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7344102bb0 0x7f7344197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:19:50.990 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:19:50.986+0000 7f734c66e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7344103550 0x7f734419f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:19:50.990 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:19:50.986+0000 7f734c66e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f734410d480 con 0x7f73441019b0 2024-04-23T10:19:50.991 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:19:50.986+0000 7f734c66e700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f734410d300 con 0x7f7344102bb0 2024-04-23T10:19:50.991 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:19:50.986+0000 7f734c66e700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f734410d600 con 0x7f7344103550 2024-04-23T10:19:50.991 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:19:50.986+0000 7f7349c09700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7344102bb0 0x7f7344197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:19:50.991 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:19:50.986+0000 7f7349c09700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7344102bb0 0x7f7344197f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:35672/0 (socket says 172.21.15.183:35672) 2024-04-23T10:19:50.991 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:19:50.986+0000 7f7349c09700 1 -- 172.21.15.183:0/2743380465 learned_addr learned my addr 172.21.15.183:0/2743380465 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-04-23T10:19:50.991 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:19:50.987+0000 7f734a40a700 1 --2- 172.21.15.183:0/2743380465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73441019b0 0x7f7344197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:19:50.991 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:19:50.987+0000 7f734ac0b700 1 --2- 172.21.15.183:0/2743380465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7344103550 0x7f734419f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:19:50.991 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:19:50.987+0000 7f7349c09700 1 -- 172.21.15.183:0/2743380465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7344103550 msgr2=0x7f734419f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:19:50.992 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:19:50.987+0000 7f7349c09700 1 --2- 172.21.15.183:0/2743380465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7344103550 0x7f734419f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:19:50.992 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:19:50.987+0000 7f7349c09700 1 -- 172.21.15.183:0/2743380465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73441019b0 msgr2=0x7f7344197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:19:50.992 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:19:50.987+0000 7f7349c09700 1 --2- 172.21.15.183:0/2743380465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73441019b0 0x7f7344197a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:19:50.992 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:19:50.987+0000 7f7349c09700 1 -- 172.21.15.183:0/2743380465 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f734419f6a0 con 0x7f7344102bb0 2024-04-23T10:19:50.992 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:19:50.987+0000 7f734ac0b700 1 --2- 172.21.15.183:0/2743380465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7344103550 0x7f734419f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:19:50.992 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:19:50.987+0000 7f7349c09700 1 --2- 172.21.15.183:0/2743380465 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7344102bb0 0x7f7344197f80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f733400d980 tx=0x7f733400dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:19:50.992 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:19:50.987+0000 7f733b7fe700 1 -- 172.21.15.183:0/2743380465 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7334016070 con 0x7f7344102bb0 2024-04-23T10:19:50.992 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:19:50.987+0000 7f733b7fe700 1 -- 172.21.15.183:0/2743380465 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7334004830 con 0x7f7344102bb0 2024-04-23T10:19:50.992 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:19:50.987+0000 7f734c66e700 1 -- 172.21.15.183:0/2743380465 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f734419f990 con 0x7f7344102bb0 2024-04-23T10:19:50.993 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:19:50.987+0000 7f733b7fe700 1 -- 172.21.15.183:0/2743380465 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7334005520 con 0x7f7344102bb0 2024-04-23T10:19:50.993 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:19:50.987+0000 7f734c66e700 1 -- 172.21.15.183:0/2743380465 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f734419ff20 con 0x7f7344102bb0 2024-04-23T10:19:50.993 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:19:50.988+0000 7f734c66e700 1 -- 172.21.15.183:0/2743380465 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7344069730 con 0x7f7344102bb0 2024-04-23T10:19:50.993 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:19:50.989+0000 7f733b7fe700 1 -- 172.21.15.183:0/2743380465 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7334004220 con 0x7f7344102bb0 2024-04-23T10:19:50.993 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:19:50.990+0000 7f733b7fe700 1 --2- 172.21.15.183:0/2743380465 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f73300a3cc0 0x7f73300a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:19:50.993 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:19:50.990+0000 7f733b7fe700 1 -- 172.21.15.183:0/2743380465 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 3669+0+0 (secure 0 0 0) 0x7f73340cf4a0 con 0x7f7344102bb0 2024-04-23T10:19:50.993 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:19:50.990+0000 7f734a40a700 1 --2- 172.21.15.183:0/2743380465 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f73300a3cc0 0x7f73300a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:19:50.994 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:19:50.990+0000 7f734a40a700 1 --2- 172.21.15.183:0/2743380465 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f73300a3cc0 0x7f73300a6180 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7344102a10 tx=0x7f734000a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:19:50.994 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:19:50.994+0000 7f733b7fe700 1 -- 172.21.15.183:0/2743380465 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f733401f020 con 0x7f7344102bb0 2024-04-23T10:19:51.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:50 smithi196 ceph-mon[32919]: osdmap e28: 4 total, 4 up, 4 in 2024-04-23T10:19:51.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:50 smithi097 ceph-mon[29197]: osdmap e28: 4 total, 4 up, 4 in 2024-04-23T10:19:51.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:50 smithi183 ceph-mon[32317]: osdmap e28: 4 total, 4 up, 4 in 2024-04-23T10:19:51.198 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:19:51.199+0000 7f734c66e700 1 -- 172.21.15.183:0/2743380465 --> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f7344062e80 con 0x7f73300a3cc0 2024-04-23T10:19:52.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:51 smithi196 ceph-mon[32919]: osdmap e29: 4 total, 4 up, 4 in 2024-04-23T10:19:52.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:51 smithi196 ceph-mon[32919]: from='client.24194 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:19:52.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:51 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T10:19:52.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:51 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T10:19:52.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:51 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:19:52.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:51 smithi196 ceph-mon[32919]: pgmap v105: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-23T10:19:52.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:51 smithi097 ceph-mon[29197]: osdmap e29: 4 total, 4 up, 4 in 2024-04-23T10:19:52.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:51 smithi097 ceph-mon[29197]: from='client.24194 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:19:52.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:51 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T10:19:52.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:51 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T10:19:52.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:51 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:19:52.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:51 smithi097 ceph-mon[29197]: pgmap v105: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-23T10:19:52.117 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:51 smithi183 ceph-mon[32317]: osdmap e29: 4 total, 4 up, 4 in 2024-04-23T10:19:52.117 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:51 smithi183 ceph-mon[32317]: from='client.24194 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:19:52.117 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:51 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T10:19:52.117 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:51 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T10:19:52.117 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:51 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:19:52.117 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:51 smithi183 ceph-mon[32317]: pgmap v105: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-23T10:19:53.061 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:52 smithi183 ceph-mon[32317]: osdmap e30: 4 total, 4 up, 4 in 2024-04-23T10:19:53.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:52 smithi196 ceph-mon[32919]: osdmap e30: 4 total, 4 up, 4 in 2024-04-23T10:19:53.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:52 smithi097 ceph-mon[29197]: osdmap e30: 4 total, 4 up, 4 in 2024-04-23T10:19:54.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:53 smithi196 ceph-mon[32919]: pgmap v107: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-04-23T10:19:54.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:53 smithi097 ceph-mon[29197]: pgmap v107: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-04-23T10:19:54.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:53 smithi183 ceph-mon[32317]: pgmap v107: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-04-23T10:19:55.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:54 smithi196 ceph-mon[32919]: from='client.? 172.21.15.183:0/3650347213' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fdcafb14-9b82-4134-bd5b-79c0516f6cbb"}]: dispatch 2024-04-23T10:19:55.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:54 smithi196 ceph-mon[32919]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fdcafb14-9b82-4134-bd5b-79c0516f6cbb"}]: dispatch 2024-04-23T10:19:55.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:54 smithi196 ceph-mon[32919]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fdcafb14-9b82-4134-bd5b-79c0516f6cbb"}]': finished 2024-04-23T10:19:55.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:54 smithi196 ceph-mon[32919]: osdmap e31: 5 total, 4 up, 5 in 2024-04-23T10:19:55.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:54 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T10:19:55.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:54 smithi196 ceph-mon[32919]: from='client.? 172.21.15.183:0/1373867927' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T10:19:55.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:54 smithi097 ceph-mon[29197]: from='client.? 172.21.15.183:0/3650347213' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fdcafb14-9b82-4134-bd5b-79c0516f6cbb"}]: dispatch 2024-04-23T10:19:55.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:54 smithi097 ceph-mon[29197]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fdcafb14-9b82-4134-bd5b-79c0516f6cbb"}]: dispatch 2024-04-23T10:19:55.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:54 smithi097 ceph-mon[29197]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fdcafb14-9b82-4134-bd5b-79c0516f6cbb"}]': finished 2024-04-23T10:19:55.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:54 smithi097 ceph-mon[29197]: osdmap e31: 5 total, 4 up, 5 in 2024-04-23T10:19:55.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:54 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T10:19:55.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:54 smithi097 ceph-mon[29197]: from='client.? 172.21.15.183:0/1373867927' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T10:19:55.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:54 smithi183 ceph-mon[32317]: from='client.? 172.21.15.183:0/3650347213' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fdcafb14-9b82-4134-bd5b-79c0516f6cbb"}]: dispatch 2024-04-23T10:19:55.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:54 smithi183 ceph-mon[32317]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fdcafb14-9b82-4134-bd5b-79c0516f6cbb"}]: dispatch 2024-04-23T10:19:55.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:54 smithi183 ceph-mon[32317]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fdcafb14-9b82-4134-bd5b-79c0516f6cbb"}]': finished 2024-04-23T10:19:55.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:54 smithi183 ceph-mon[32317]: osdmap e31: 5 total, 4 up, 5 in 2024-04-23T10:19:55.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:54 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T10:19:55.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:54 smithi183 ceph-mon[32317]: from='client.? 172.21.15.183:0/1373867927' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T10:19:56.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:55 smithi196 ceph-mon[32919]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-04-23T10:19:56.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:55 smithi097 ceph-mon[29197]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-04-23T10:19:56.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:55 smithi183 ceph-mon[32317]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-04-23T10:19:58.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:19:58 smithi097 ceph-mon[29197]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-23T10:19:58.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:19:58 smithi196 ceph-mon[32919]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-23T10:19:58.860 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:19:58 smithi183 ceph-mon[32317]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-23T10:20:00.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:00 smithi183 ceph-mon[32317]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-23T10:20:00.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:00 smithi183 ceph-mon[32317]: overall HEALTH_OK 2024-04-23T10:20:00.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:00 smithi196 ceph-mon[32919]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-23T10:20:00.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:00 smithi196 ceph-mon[32919]: overall HEALTH_OK 2024-04-23T10:20:00.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:00 smithi097 ceph-mon[29197]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-23T10:20:00.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:00 smithi097 ceph-mon[29197]: overall HEALTH_OK 2024-04-23T10:20:01.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:01 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-23T10:20:01.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:01 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:20:01.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:01 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-23T10:20:01.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:01 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:20:01.871 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:01 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-23T10:20:01.871 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:01 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:20:02.616 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:02 smithi183 ceph-mon[32317]: Deploying daemon osd.4 on smithi183 2024-04-23T10:20:02.616 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:02 smithi183 ceph-mon[32317]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-23T10:20:02.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:02 smithi196 ceph-mon[32919]: Deploying daemon osd.4 on smithi183 2024-04-23T10:20:02.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:02 smithi196 ceph-mon[32919]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-23T10:20:02.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:02 smithi097 ceph-mon[29197]: Deploying daemon osd.4 on smithi183 2024-04-23T10:20:02.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:02 smithi097 ceph-mon[29197]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-23T10:20:04.839 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:04 smithi183 ceph-mon[32317]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-23T10:20:04.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:04 smithi196 ceph-mon[32919]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-23T10:20:04.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:04 smithi097 ceph-mon[29197]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-23T10:20:06.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:06 smithi183 ceph-mon[32317]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-23T10:20:06.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:06 smithi196 ceph-mon[32919]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-23T10:20:06.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:06 smithi097 ceph-mon[29197]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-23T10:20:07.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:07 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:07.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:07 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:07.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:07 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:20:08.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:07 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:08.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:07 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:08.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:07 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:20:08.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:07 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:08.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:07 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:08.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:07 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:20:08.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:08 smithi183 ceph-mon[32317]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-23T10:20:09.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:08 smithi196 ceph-mon[32919]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-23T10:20:09.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:08 smithi097 ceph-mon[29197]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-23T10:20:10.364 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:20:10.365+0000 7f733b7fe700 1 -- 172.21.15.183:0/2743380465 <== mgr.14152 v2:172.21.15.97:6800/3149043749 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7344062e80 con 0x7f73300a3cc0 2024-04-23T10:20:10.365 INFO:teuthology.orchestra.run.smithi183.stdout:Created osd(s) 4 on host 'smithi183' 2024-04-23T10:20:10.366 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:20:10.368+0000 7f734c66e700 1 -- 172.21.15.183:0/2743380465 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f73300a3cc0 msgr2=0x7f73300a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:20:10.366 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:20:10.368+0000 7f734c66e700 1 --2- 172.21.15.183:0/2743380465 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f73300a3cc0 0x7f73300a6180 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7344102a10 tx=0x7f734000a3b0 comp rx=0 tx=0).stop 2024-04-23T10:20:10.367 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:20:10.368+0000 7f734c66e700 1 -- 172.21.15.183:0/2743380465 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7344102bb0 msgr2=0x7f7344197f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:20:10.367 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:20:10.368+0000 7f734c66e700 1 --2- 172.21.15.183:0/2743380465 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7344102bb0 0x7f7344197f80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f733400d980 tx=0x7f733400dc90 comp rx=0 tx=0).stop 2024-04-23T10:20:10.367 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:20:10.368+0000 7f734c66e700 1 -- 172.21.15.183:0/2743380465 shutdown_connections 2024-04-23T10:20:10.367 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:20:10.368+0000 7f734c66e700 1 --2- 172.21.15.183:0/2743380465 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f73300a3cc0 0x7f73300a6180 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:20:10.367 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:20:10.368+0000 7f734c66e700 1 --2- 172.21.15.183:0/2743380465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7344103550 0x7f734419f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:20:10.367 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:20:10.368+0000 7f734c66e700 1 --2- 172.21.15.183:0/2743380465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73441019b0 0x7f7344197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:20:10.367 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:20:10.368+0000 7f734c66e700 1 --2- 172.21.15.183:0/2743380465 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7344102bb0 0x7f7344197f80 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:20:10.367 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:20:10.368+0000 7f734c66e700 1 -- 172.21.15.183:0/2743380465 >> 172.21.15.183:0/2743380465 conn(0x7f73440fcf70 msgr2=0x7f7344105eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:20:10.367 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:20:10.368+0000 7f734c66e700 1 -- 172.21.15.183:0/2743380465 shutdown_connections 2024-04-23T10:20:10.367 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:20:10.368+0000 7f734c66e700 1 -- 172.21.15.183:0/2743380465 wait complete. 2024-04-23T10:20:10.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:10 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:10.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:10 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:10.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:10 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:20:10.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:10 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:20:10.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:10 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:10.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:10 smithi097 ceph-mon[29197]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-23T10:20:10.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:10 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:10.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:10 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:20:10.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:10 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:10.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:10 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:10.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:10 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:10.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:10 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:20:10.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:10 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:20:10.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:10 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:10.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:10 smithi183 ceph-mon[32317]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-23T10:20:10.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:10 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:10.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:10 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:20:10.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:10 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:10.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:10 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:10.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:10 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:10.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:10 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:20:10.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:10 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:20:10.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:10 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:10.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:10 smithi196 ceph-mon[32919]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-23T10:20:10.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:10 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:10.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:10 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:20:10.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:10 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:11.013 DEBUG:teuthology.orchestra.run.smithi183:osd.4> sudo journalctl -f -n 0 -u ceph-9b22af88-0159-11ef-bc93-c7b262605968@osd.4.service 2024-04-23T10:20:11.015 INFO:tasks.cephadm:Deploying osd.5 on smithi196 with /dev/vg_nvme/lv_3... 2024-04-23T10:20:11.015 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-23T10:20:11.263 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T10:20:11.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:11 smithi097 ceph-mon[29197]: from='osd.4 [v2:172.21.15.183:6808/63913916,v1:172.21.15.183:6809/63913916]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-23T10:20:11.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:11 smithi097 ceph-mon[29197]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-23T10:20:11.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:11 smithi183 ceph-mon[32317]: from='osd.4 [v2:172.21.15.183:6808/63913916,v1:172.21.15.183:6809/63913916]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-23T10:20:11.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:11 smithi183 ceph-mon[32317]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-23T10:20:11.678 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:11 smithi196 ceph-mon[32919]: from='osd.4 [v2:172.21.15.183:6808/63913916,v1:172.21.15.183:6809/63913916]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-23T10:20:11.678 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:11 smithi196 ceph-mon[32919]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-23T10:20:12.529 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:12 smithi183 ceph-mon[32317]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-23T10:20:12.529 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:12 smithi183 ceph-mon[32317]: from='osd.4 [v2:172.21.15.183:6808/63913916,v1:172.21.15.183:6809/63913916]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-23T10:20:12.529 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:12 smithi183 ceph-mon[32317]: osdmap e32: 5 total, 4 up, 5 in 2024-04-23T10:20:12.529 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:12 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T10:20:12.529 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:12 smithi183 ceph-mon[32317]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-23T10:20:12.529 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:12 smithi183 ceph-mon[32317]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-23T10:20:12.585 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:12 smithi196 ceph-mon[32919]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-23T10:20:12.585 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:12 smithi196 ceph-mon[32919]: from='osd.4 [v2:172.21.15.183:6808/63913916,v1:172.21.15.183:6809/63913916]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-23T10:20:12.585 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:12 smithi196 ceph-mon[32919]: osdmap e32: 5 total, 4 up, 5 in 2024-04-23T10:20:12.585 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:12 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T10:20:12.585 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:12 smithi196 ceph-mon[32919]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-23T10:20:12.585 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:12 smithi196 ceph-mon[32919]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-23T10:20:12.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:12 smithi097 ceph-mon[29197]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-23T10:20:12.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:12 smithi097 ceph-mon[29197]: from='osd.4 [v2:172.21.15.183:6808/63913916,v1:172.21.15.183:6809/63913916]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-23T10:20:12.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:12 smithi097 ceph-mon[29197]: osdmap e32: 5 total, 4 up, 5 in 2024-04-23T10:20:12.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:12 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T10:20:12.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:12 smithi097 ceph-mon[29197]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-23T10:20:12.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:12 smithi097 ceph-mon[29197]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-23T10:20:13.117 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-23T10:20:13.117 INFO:teuthology.orchestra.run.smithi196.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-23T10:20:13.117 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10+0 records in 2024-04-23T10:20:13.117 INFO:teuthology.orchestra.run.smithi196.stderr:10+0 records out 2024-04-23T10:20:13.117 INFO:teuthology.orchestra.run.smithi196.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112327 s, 934 MB/s 2024-04-23T10:20:13.117 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 2024-04-23T10:20:13.117 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping successful for: 2024-04-23T10:20:13.150 INFO:journalctl@ceph.osd.4.smithi183.stdout:Apr 23 10:20:12 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-4[43863]: 2024-04-23T10:20:12.846+0000 7f6d6f3de700 -1 osd.4 0 waiting for initial osdmap 2024-04-23T10:20:13.151 INFO:journalctl@ceph.osd.4.smithi183.stdout:Apr 23 10:20:12 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-4[43863]: 2024-04-23T10:20:12.852+0000 7f6d671c9700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T10:20:13.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:13 smithi183 ceph-mon[32317]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-04-23T10:20:13.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:13 smithi183 ceph-mon[32317]: osdmap e33: 5 total, 4 up, 5 in 2024-04-23T10:20:13.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:13 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T10:20:13.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:13 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T10:20:13.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:13 smithi183 ceph-mon[32317]: from='osd.4 ' entity='osd.4' 2024-04-23T10:20:13.730 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:13 smithi196 ceph-mon[32919]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-04-23T10:20:13.730 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:13 smithi196 ceph-mon[32919]: osdmap e33: 5 total, 4 up, 5 in 2024-04-23T10:20:13.730 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:13 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T10:20:13.730 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:13 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T10:20:13.730 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:13 smithi196 ceph-mon[32919]: from='osd.4 ' entity='osd.4' 2024-04-23T10:20:13.758 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi196:vg_nvme/lv_3 2024-04-23T10:20:13.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:13 smithi097 ceph-mon[29197]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-04-23T10:20:13.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:13 smithi097 ceph-mon[29197]: osdmap e33: 5 total, 4 up, 5 in 2024-04-23T10:20:13.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:13 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T10:20:13.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:13 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T10:20:13.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:13 smithi097 ceph-mon[29197]: from='osd.4 ' entity='osd.4' 2024-04-23T10:20:13.998 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T10:20:14.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:14 smithi183 ceph-mon[32317]: purged_snaps scrub starts 2024-04-23T10:20:14.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:14 smithi183 ceph-mon[32317]: purged_snaps scrub ok 2024-04-23T10:20:14.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:14 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T10:20:14.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:14 smithi183 ceph-mon[32317]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-23T10:20:14.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:14 smithi183 ceph-mon[32317]: osd.4 [v2:172.21.15.183:6808/63913916,v1:172.21.15.183:6809/63913916] boot 2024-04-23T10:20:14.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:14 smithi183 ceph-mon[32317]: osdmap e34: 5 total, 5 up, 5 in 2024-04-23T10:20:14.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:14 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T10:20:14.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:14 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:14.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:14 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:14.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:14 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:20:14.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:14 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:20:14.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:14 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:14.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:14 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:20:14.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:14 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:20:14.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:14 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:14.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:14 smithi196 ceph-mon[32919]: purged_snaps scrub starts 2024-04-23T10:20:14.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:14 smithi196 ceph-mon[32919]: purged_snaps scrub ok 2024-04-23T10:20:14.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:14 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T10:20:14.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:14 smithi196 ceph-mon[32919]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-23T10:20:14.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:14 smithi196 ceph-mon[32919]: osd.4 [v2:172.21.15.183:6808/63913916,v1:172.21.15.183:6809/63913916] boot 2024-04-23T10:20:14.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:14 smithi196 ceph-mon[32919]: osdmap e34: 5 total, 5 up, 5 in 2024-04-23T10:20:14.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:14 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T10:20:14.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:14 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:14.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:14 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:14.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:14 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:20:14.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:14 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:20:14.843 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:14 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:14.843 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:14 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:20:14.843 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:14 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:20:14.843 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:14 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:14.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:14 smithi097 ceph-mon[29197]: purged_snaps scrub starts 2024-04-23T10:20:14.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:14 smithi097 ceph-mon[29197]: purged_snaps scrub ok 2024-04-23T10:20:14.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:14 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T10:20:14.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:14 smithi097 ceph-mon[29197]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-23T10:20:14.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:14 smithi097 ceph-mon[29197]: osd.4 [v2:172.21.15.183:6808/63913916,v1:172.21.15.183:6809/63913916] boot 2024-04-23T10:20:14.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:14 smithi097 ceph-mon[29197]: osdmap e34: 5 total, 5 up, 5 in 2024-04-23T10:20:14.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:14 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T10:20:14.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:14 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:14.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:14 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:14.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:14 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:20:14.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:14 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:20:14.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:14 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:14.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:14 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:20:14.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:14 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:20:14.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:14 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:15.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:15 smithi196 ceph-mon[32919]: Detected new or changed devices on smithi183 2024-04-23T10:20:15.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:15 smithi196 ceph-mon[32919]: Adjusting osd_memory_target on smithi183 to 8544M 2024-04-23T10:20:15.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:15 smithi183 ceph-mon[32317]: Detected new or changed devices on smithi183 2024-04-23T10:20:15.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:15 smithi183 ceph-mon[32317]: Adjusting osd_memory_target on smithi183 to 8544M 2024-04-23T10:20:15.792 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:20:15.792+0000 7f399593f700 1 -- 172.21.15.196:0/646844722 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39901087f0 msgr2=0x7f399010ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:20:15.792 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:20:15.792+0000 7f399593f700 1 --2- 172.21.15.196:0/646844722 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39901087f0 0x7f399010ac00 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f398800b0a0 tx=0x7f398800b3b0 comp rx=0 tx=0).stop 2024-04-23T10:20:15.792 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:20:15.792+0000 7f399593f700 1 -- 172.21.15.196:0/646844722 shutdown_connections 2024-04-23T10:20:15.792 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:20:15.792+0000 7f399593f700 1 --2- 172.21.15.196:0/646844722 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39901087f0 0x7f399010ac00 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:20:15.792 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:20:15.792+0000 7f399593f700 1 --2- 172.21.15.196:0/646844722 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3990103590 0x7f3990105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:20:15.792 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:20:15.793+0000 7f399593f700 1 --2- 172.21.15.196:0/646844722 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3990105ec0 0x7f39901082b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:20:15.792 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:20:15.793+0000 7f399593f700 1 -- 172.21.15.196:0/646844722 >> 172.21.15.196:0/646844722 conn(0x7f39900fcf40 msgr2=0x7f39900ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:20:15.792 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:20:15.793+0000 7f399593f700 1 -- 172.21.15.196:0/646844722 shutdown_connections 2024-04-23T10:20:15.792 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:20:15.793+0000 7f399593f700 1 -- 172.21.15.196:0/646844722 wait complete. 2024-04-23T10:20:15.794 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:20:15.793+0000 7f399593f700 1 Processor -- start 2024-04-23T10:20:15.795 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:20:15.793+0000 7f399593f700 1 -- start start 2024-04-23T10:20:15.795 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:20:15.793+0000 7f399593f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3990103590 0x7f3990198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:20:15.795 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:20:15.793+0000 7f399593f700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3990105ec0 0x7f3990199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:20:15.795 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:20:15.793+0000 7f399593f700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f39901087f0 0x7f39901a01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:20:15.795 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:20:15.793+0000 7f399593f700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f399010e5a0 con 0x7f3990103590 2024-04-23T10:20:15.795 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:20:15.793+0000 7f399593f700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f399010e420 con 0x7f39901087f0 2024-04-23T10:20:15.795 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:20:15.793+0000 7f399593f700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f399010e720 con 0x7f3990105ec0 2024-04-23T10:20:15.795 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:20:15.794+0000 7f398e7fc700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3990105ec0 0x7f3990199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:20:15.795 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:20:15.794+0000 7f398e7fc700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3990105ec0 0x7f3990199180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:55228/0 (socket says 172.21.15.196:55228) 2024-04-23T10:20:15.795 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:20:15.794+0000 7f398e7fc700 1 -- 172.21.15.196:0/93060699 learned_addr learned my addr 172.21.15.196:0/93060699 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-23T10:20:15.795 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:20:15.794+0000 7f398f7fe700 1 --2- 172.21.15.196:0/93060699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f39901087f0 0x7f39901a01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:20:15.795 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:20:15.794+0000 7f398effd700 1 --2- 172.21.15.196:0/93060699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3990103590 0x7f3990198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:20:15.795 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:20:15.794+0000 7f398e7fc700 1 -- 172.21.15.196:0/93060699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f39901087f0 msgr2=0x7f39901a01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:20:15.795 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:20:15.794+0000 7f398e7fc700 1 --2- 172.21.15.196:0/93060699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f39901087f0 0x7f39901a01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:20:15.796 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:20:15.794+0000 7f398e7fc700 1 -- 172.21.15.196:0/93060699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3990103590 msgr2=0x7f3990198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:20:15.796 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:20:15.794+0000 7f398e7fc700 1 --2- 172.21.15.196:0/93060699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3990103590 0x7f3990198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:20:15.796 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:20:15.794+0000 7f398e7fc700 1 -- 172.21.15.196:0/93060699 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39901a0890 con 0x7f3990105ec0 2024-04-23T10:20:15.796 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:20:15.794+0000 7f398e7fc700 1 --2- 172.21.15.196:0/93060699 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3990105ec0 0x7f3990199180 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f398400b530 tx=0x7f398400b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:20:15.796 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:20:15.794+0000 7f399493d700 1 -- 172.21.15.196:0/93060699 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3984015020 con 0x7f3990105ec0 2024-04-23T10:20:15.796 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:20:15.794+0000 7f399493d700 1 -- 172.21.15.196:0/93060699 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3984011ce0 con 0x7f3990105ec0 2024-04-23T10:20:15.796 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:20:15.794+0000 7f399593f700 1 -- 172.21.15.196:0/93060699 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39901a0b80 con 0x7f3990105ec0 2024-04-23T10:20:15.796 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:20:15.794+0000 7f399493d700 1 -- 172.21.15.196:0/93060699 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3984010e40 con 0x7f3990105ec0 2024-04-23T10:20:15.796 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:20:15.794+0000 7f399593f700 1 -- 172.21.15.196:0/93060699 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39901a10c0 con 0x7f3990105ec0 2024-04-23T10:20:15.796 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:20:15.795+0000 7f399593f700 1 -- 172.21.15.196:0/93060699 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3990069730 con 0x7f3990105ec0 2024-04-23T10:20:15.797 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:20:15.797+0000 7f399493d700 1 -- 172.21.15.196:0/93060699 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3984018020 con 0x7f3990105ec0 2024-04-23T10:20:15.797 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:20:15.798+0000 7f399493d700 1 --2- 172.21.15.196:0/93060699 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f39780a3c40 0x7f39780a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:20:15.797 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:20:15.798+0000 7f399493d700 1 -- 172.21.15.196:0/93060699 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4101+0+0 (secure 0 0 0) 0x7f39840cf0e0 con 0x7f3990105ec0 2024-04-23T10:20:15.797 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:20:15.798+0000 7f398effd700 1 --2- 172.21.15.196:0/93060699 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f39780a3c40 0x7f39780a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:20:15.798 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:20:15.798+0000 7f398effd700 1 --2- 172.21.15.196:0/93060699 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f39780a3c40 0x7f39780a6100 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3980005950 tx=0x7f3980005eb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:20:15.800 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:20:15.802+0000 7f399493d700 1 -- 172.21.15.196:0/93060699 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f398401f020 con 0x7f3990105ec0 2024-04-23T10:20:15.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:15 smithi097 ceph-mon[29197]: Detected new or changed devices on smithi183 2024-04-23T10:20:15.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:15 smithi097 ceph-mon[29197]: Adjusting osd_memory_target on smithi183 to 8544M 2024-04-23T10:20:16.005 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:20:16.006+0000 7f399593f700 1 -- 172.21.15.196:0/93060699 --> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f3990062e80 con 0x7f39780a3c40 2024-04-23T10:20:16.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:16 smithi183 ceph-mon[32317]: osdmap e35: 5 total, 5 up, 5 in 2024-04-23T10:20:16.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:16 smithi183 ceph-mon[32317]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-23T10:20:16.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:16 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T10:20:16.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:16 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T10:20:16.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:16 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:20:16.731 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:16 smithi196 ceph-mon[32919]: osdmap e35: 5 total, 5 up, 5 in 2024-04-23T10:20:16.731 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:16 smithi196 ceph-mon[32919]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-23T10:20:16.731 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:16 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T10:20:16.731 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:16 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T10:20:16.731 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:16 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:20:16.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:16 smithi097 ceph-mon[29197]: osdmap e35: 5 total, 5 up, 5 in 2024-04-23T10:20:16.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:16 smithi097 ceph-mon[29197]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-23T10:20:16.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:16 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T10:20:16.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:16 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T10:20:16.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:16 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:20:17.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:17 smithi183 ceph-mon[32317]: from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:20:17.772 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:17 smithi196 ceph-mon[32919]: from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:20:17.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:17 smithi097 ceph-mon[29197]: from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:20:18.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:18 smithi196 ceph-mon[32919]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-23T10:20:18.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:18 smithi097 ceph-mon[29197]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-23T10:20:18.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:18 smithi183 ceph-mon[32317]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-23T10:20:19.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:19 smithi196 ceph-mon[32919]: from='client.? 172.21.15.196:0/3605307595' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cdcc18d8-da3c-4012-bf12-876bb39e66eb"}]: dispatch 2024-04-23T10:20:19.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:19 smithi196 ceph-mon[32919]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cdcc18d8-da3c-4012-bf12-876bb39e66eb"}]: dispatch 2024-04-23T10:20:19.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:19 smithi196 ceph-mon[32919]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cdcc18d8-da3c-4012-bf12-876bb39e66eb"}]': finished 2024-04-23T10:20:19.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:19 smithi196 ceph-mon[32919]: osdmap e36: 6 total, 5 up, 6 in 2024-04-23T10:20:19.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:19 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T10:20:19.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:19 smithi196 ceph-mon[32919]: from='client.? 172.21.15.196:0/2052255093' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T10:20:19.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:19 smithi097 ceph-mon[29197]: from='client.? 172.21.15.196:0/3605307595' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cdcc18d8-da3c-4012-bf12-876bb39e66eb"}]: dispatch 2024-04-23T10:20:19.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:19 smithi097 ceph-mon[29197]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cdcc18d8-da3c-4012-bf12-876bb39e66eb"}]: dispatch 2024-04-23T10:20:19.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:19 smithi097 ceph-mon[29197]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cdcc18d8-da3c-4012-bf12-876bb39e66eb"}]': finished 2024-04-23T10:20:19.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:19 smithi097 ceph-mon[29197]: osdmap e36: 6 total, 5 up, 6 in 2024-04-23T10:20:19.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:19 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T10:20:19.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:19 smithi097 ceph-mon[29197]: from='client.? 172.21.15.196:0/2052255093' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T10:20:19.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:19 smithi183 ceph-mon[32317]: from='client.? 172.21.15.196:0/3605307595' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cdcc18d8-da3c-4012-bf12-876bb39e66eb"}]: dispatch 2024-04-23T10:20:19.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:19 smithi183 ceph-mon[32317]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cdcc18d8-da3c-4012-bf12-876bb39e66eb"}]: dispatch 2024-04-23T10:20:19.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:19 smithi183 ceph-mon[32317]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cdcc18d8-da3c-4012-bf12-876bb39e66eb"}]': finished 2024-04-23T10:20:19.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:19 smithi183 ceph-mon[32317]: osdmap e36: 6 total, 5 up, 6 in 2024-04-23T10:20:19.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:19 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T10:20:19.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:19 smithi183 ceph-mon[32317]: from='client.? 172.21.15.196:0/2052255093' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T10:20:20.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:20 smithi196 ceph-mon[32919]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-23T10:20:20.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:20 smithi097 ceph-mon[29197]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-23T10:20:20.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:20 smithi183 ceph-mon[32317]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-23T10:20:22.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:22 smithi196 ceph-mon[32919]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-23T10:20:22.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:22 smithi097 ceph-mon[29197]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-23T10:20:22.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:22 smithi183 ceph-mon[32317]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-23T10:20:24.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:24 smithi196 ceph-mon[32919]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-23T10:20:24.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:24 smithi097 ceph-mon[29197]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-23T10:20:24.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:24 smithi183 ceph-mon[32317]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-23T10:20:26.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:26 smithi196 ceph-mon[32919]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-23T10:20:26.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:26 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-23T10:20:26.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:26 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:20:26.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:26 smithi097 ceph-mon[29197]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-23T10:20:26.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:26 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-23T10:20:26.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:26 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:20:26.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:26 smithi183 ceph-mon[32317]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-23T10:20:26.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:26 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-23T10:20:26.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:26 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:20:27.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:27 smithi196 ceph-mon[32919]: Deploying daemon osd.5 on smithi196 2024-04-23T10:20:27.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:27 smithi097 ceph-mon[29197]: Deploying daemon osd.5 on smithi196 2024-04-23T10:20:27.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:27 smithi183 ceph-mon[32317]: Deploying daemon osd.5 on smithi196 2024-04-23T10:20:28.822 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:28 smithi196 ceph-mon[32919]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-23T10:20:28.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:28 smithi097 ceph-mon[29197]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-23T10:20:28.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:28 smithi183 ceph-mon[32317]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-23T10:20:30.838 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:30 smithi196 ceph-mon[32919]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-23T10:20:30.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:30 smithi097 ceph-mon[29197]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-23T10:20:30.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:30 smithi183 ceph-mon[32317]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-23T10:20:32.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:32 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:32.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:32 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:32.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:32 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:20:32.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:32 smithi196 ceph-mon[32919]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-23T10:20:32.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:32 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:32.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:32 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:32.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:32 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:20:32.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:32 smithi097 ceph-mon[29197]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-23T10:20:32.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:32 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:32.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:32 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:32.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:32 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:20:32.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:32 smithi183 ceph-mon[32317]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-23T10:20:34.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:34 smithi196 ceph-mon[32919]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-23T10:20:34.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:34 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:34.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:34 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:34.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:34 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:20:34.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:34 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:20:34.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:34 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:34.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:34 smithi097 ceph-mon[29197]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-23T10:20:34.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:34 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:34.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:34 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:34.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:34 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:20:34.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:34 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:20:34.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:34 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:34.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:34 smithi183 ceph-mon[32317]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-23T10:20:34.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:34 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:34.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:34 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:34.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:34 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:20:34.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:34 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:20:34.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:34 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:35.222 INFO:teuthology.orchestra.run.smithi196.stdout:Created osd(s) 5 on host 'smithi196' 2024-04-23T10:20:35.222 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:20:35.222+0000 7f399493d700 1 -- 172.21.15.196:0/93060699 <== mgr.14152 v2:172.21.15.97:6800/3149043749 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3990062e80 con 0x7f39780a3c40 2024-04-23T10:20:35.224 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:20:35.225+0000 7f399593f700 1 -- 172.21.15.196:0/93060699 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f39780a3c40 msgr2=0x7f39780a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:20:35.224 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:20:35.225+0000 7f399593f700 1 --2- 172.21.15.196:0/93060699 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f39780a3c40 0x7f39780a6100 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3980005950 tx=0x7f3980005eb0 comp rx=0 tx=0).stop 2024-04-23T10:20:35.224 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:20:35.225+0000 7f399593f700 1 -- 172.21.15.196:0/93060699 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3990105ec0 msgr2=0x7f3990199180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:20:35.224 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:20:35.225+0000 7f399593f700 1 --2- 172.21.15.196:0/93060699 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3990105ec0 0x7f3990199180 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f398400b530 tx=0x7f398400b840 comp rx=0 tx=0).stop 2024-04-23T10:20:35.224 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:20:35.225+0000 7f399593f700 1 -- 172.21.15.196:0/93060699 shutdown_connections 2024-04-23T10:20:35.224 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:20:35.225+0000 7f399593f700 1 --2- 172.21.15.196:0/93060699 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f39780a3c40 0x7f39780a6100 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:20:35.224 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:20:35.225+0000 7f399593f700 1 --2- 172.21.15.196:0/93060699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f39901087f0 0x7f39901a01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:20:35.224 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:20:35.225+0000 7f399593f700 1 --2- 172.21.15.196:0/93060699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3990103590 0x7f3990198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:20:35.225 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:20:35.225+0000 7f399593f700 1 --2- 172.21.15.196:0/93060699 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3990105ec0 0x7f3990199180 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:20:35.225 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:20:35.225+0000 7f399593f700 1 -- 172.21.15.196:0/93060699 >> 172.21.15.196:0/93060699 conn(0x7f39900fcf40 msgr2=0x7f3990103ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:20:35.225 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:20:35.225+0000 7f399593f700 1 -- 172.21.15.196:0/93060699 shutdown_connections 2024-04-23T10:20:35.225 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:20:35.225+0000 7f399593f700 1 -- 172.21.15.196:0/93060699 wait complete. 2024-04-23T10:20:35.980 DEBUG:teuthology.orchestra.run.smithi196:osd.5> sudo journalctl -f -n 0 -u ceph-9b22af88-0159-11ef-bc93-c7b262605968@osd.5.service 2024-04-23T10:20:35.983 INFO:tasks.cephadm:Deploying osd.6 on smithi097 with /dev/vg_nvme/lv_2... 2024-04-23T10:20:35.983 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-23T10:20:36.232 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T10:20:36.355 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:36 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:36.355 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:36 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:36.355 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:36 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:20:36.355 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:36 smithi097 ceph-mon[29197]: from='osd.5 [v2:172.21.15.196:6808/2804446896,v1:172.21.15.196:6809/2804446896]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-23T10:20:36.355 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:36 smithi097 ceph-mon[29197]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-23T10:20:36.355 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:36 smithi097 ceph-mon[29197]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-23T10:20:36.540 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:36 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:36.540 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:36 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:36.540 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:36 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:20:36.540 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:36 smithi196 ceph-mon[32919]: from='osd.5 [v2:172.21.15.196:6808/2804446896,v1:172.21.15.196:6809/2804446896]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-23T10:20:36.540 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:36 smithi196 ceph-mon[32919]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-23T10:20:36.540 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:36 smithi196 ceph-mon[32919]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-23T10:20:36.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:36 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:36.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:36 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:36.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:36 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:20:36.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:36 smithi183 ceph-mon[32317]: from='osd.5 [v2:172.21.15.196:6808/2804446896,v1:172.21.15.196:6809/2804446896]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-23T10:20:36.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:36 smithi183 ceph-mon[32317]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-23T10:20:36.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:36 smithi183 ceph-mon[32317]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-23T10:20:37.470 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:37 smithi196 ceph-mon[32919]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-23T10:20:37.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:37 smithi196 ceph-mon[32919]: osdmap e37: 6 total, 5 up, 6 in 2024-04-23T10:20:37.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:37 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T10:20:37.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:37 smithi196 ceph-mon[32919]: from='osd.5 [v2:172.21.15.196:6808/2804446896,v1:172.21.15.196:6809/2804446896]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-23T10:20:37.471 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:37 smithi196 ceph-mon[32919]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-23T10:20:37.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:37 smithi097 ceph-mon[29197]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-23T10:20:37.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:37 smithi097 ceph-mon[29197]: osdmap e37: 6 total, 5 up, 6 in 2024-04-23T10:20:37.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:37 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T10:20:37.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:37 smithi097 ceph-mon[29197]: from='osd.5 [v2:172.21.15.196:6808/2804446896,v1:172.21.15.196:6809/2804446896]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-23T10:20:37.564 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:37 smithi097 ceph-mon[29197]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-23T10:20:37.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:37 smithi183 ceph-mon[32317]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-23T10:20:37.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:37 smithi183 ceph-mon[32317]: osdmap e37: 6 total, 5 up, 6 in 2024-04-23T10:20:37.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:37 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T10:20:37.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:37 smithi183 ceph-mon[32317]: from='osd.5 [v2:172.21.15.196:6808/2804446896,v1:172.21.15.196:6809/2804446896]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-23T10:20:37.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:37 smithi183 ceph-mon[32317]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-23T10:20:37.842 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 23 10:20:37 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-5[44718]: 2024-04-23T10:20:37.666+0000 7f33f2ead700 -1 osd.5 0 waiting for initial osdmap 2024-04-23T10:20:37.842 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 23 10:20:37 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-5[44718]: 2024-04-23T10:20:37.676+0000 7f33eac98700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T10:20:38.287 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-23T10:20:38.287 INFO:teuthology.orchestra.run.smithi097.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-23T10:20:38.287 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10+0 records in 2024-04-23T10:20:38.287 INFO:teuthology.orchestra.run.smithi097.stderr:10+0 records out 2024-04-23T10:20:38.287 INFO:teuthology.orchestra.run.smithi097.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0131989 s, 794 MB/s 2024-04-23T10:20:38.287 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping successful for: 2024-04-23T10:20:38.590 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:38 smithi097 ceph-mon[29197]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-04-23T10:20:38.590 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:38 smithi097 ceph-mon[29197]: osdmap e38: 6 total, 5 up, 6 in 2024-04-23T10:20:38.590 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:38 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T10:20:38.590 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:38 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T10:20:38.591 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:38 smithi097 ceph-mon[29197]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-23T10:20:38.591 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:38 smithi097 ceph-mon[29197]: from='osd.5 ' entity='osd.5' 2024-04-23T10:20:38.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:38 smithi196 ceph-mon[32919]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-04-23T10:20:38.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:38 smithi196 ceph-mon[32919]: osdmap e38: 6 total, 5 up, 6 in 2024-04-23T10:20:38.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:38 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T10:20:38.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:38 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T10:20:38.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:38 smithi196 ceph-mon[32919]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-23T10:20:38.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:38 smithi196 ceph-mon[32919]: from='osd.5 ' entity='osd.5' 2024-04-23T10:20:38.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:38 smithi183 ceph-mon[32317]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-04-23T10:20:38.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:38 smithi183 ceph-mon[32317]: osdmap e38: 6 total, 5 up, 6 in 2024-04-23T10:20:38.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:38 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T10:20:38.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:38 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T10:20:38.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:38 smithi183 ceph-mon[32317]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-04-23T10:20:38.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:38 smithi183 ceph-mon[32317]: from='osd.5 ' entity='osd.5' 2024-04-23T10:20:38.924 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi097:vg_nvme/lv_2 2024-04-23T10:20:39.164 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T10:20:39.293 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:39 smithi196 ceph-mon[32919]: purged_snaps scrub starts 2024-04-23T10:20:39.293 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:39 smithi196 ceph-mon[32919]: purged_snaps scrub ok 2024-04-23T10:20:39.293 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:39 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T10:20:39.293 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:39 smithi196 ceph-mon[32919]: osd.5 [v2:172.21.15.196:6808/2804446896,v1:172.21.15.196:6809/2804446896] boot 2024-04-23T10:20:39.293 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:39 smithi196 ceph-mon[32919]: osdmap e39: 6 total, 6 up, 6 in 2024-04-23T10:20:39.293 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:39 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T10:20:39.506 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:39 smithi097 ceph-mon[29197]: purged_snaps scrub starts 2024-04-23T10:20:39.507 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:39 smithi097 ceph-mon[29197]: purged_snaps scrub ok 2024-04-23T10:20:39.507 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:39 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T10:20:39.507 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:39 smithi097 ceph-mon[29197]: osd.5 [v2:172.21.15.196:6808/2804446896,v1:172.21.15.196:6809/2804446896] boot 2024-04-23T10:20:39.507 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:39 smithi097 ceph-mon[29197]: osdmap e39: 6 total, 6 up, 6 in 2024-04-23T10:20:39.507 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:39 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T10:20:39.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:39 smithi183 ceph-mon[32317]: purged_snaps scrub starts 2024-04-23T10:20:39.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:39 smithi183 ceph-mon[32317]: purged_snaps scrub ok 2024-04-23T10:20:39.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:39 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T10:20:39.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:39 smithi183 ceph-mon[32317]: osd.5 [v2:172.21.15.196:6808/2804446896,v1:172.21.15.196:6809/2804446896] boot 2024-04-23T10:20:39.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:39 smithi183 ceph-mon[32317]: osdmap e39: 6 total, 6 up, 6 in 2024-04-23T10:20:39.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:39 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T10:20:40.458 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:40 smithi097 ceph-mon[29197]: osdmap e40: 6 total, 6 up, 6 in 2024-04-23T10:20:40.459 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:40 smithi097 ceph-mon[29197]: Detected new or changed devices on smithi196 2024-04-23T10:20:40.459 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:40 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:40.459 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:40 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:40.459 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:40 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:20:40.459 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:40 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:20:40.459 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:40 smithi097 ceph-mon[29197]: Adjusting osd_memory_target on smithi196 to 8544M 2024-04-23T10:20:40.459 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:40 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:40.459 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:40 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:20:40.459 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:40 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:20:40.459 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:40 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:40.459 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:40 smithi097 ceph-mon[29197]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-23T10:20:40.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:40 smithi196 ceph-mon[32919]: osdmap e40: 6 total, 6 up, 6 in 2024-04-23T10:20:40.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:40 smithi196 ceph-mon[32919]: Detected new or changed devices on smithi196 2024-04-23T10:20:40.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:40 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:40.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:40 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:40.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:40 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:20:40.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:40 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:20:40.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:40 smithi196 ceph-mon[32919]: Adjusting osd_memory_target on smithi196 to 8544M 2024-04-23T10:20:40.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:40 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:40.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:40 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:20:40.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:40 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:20:40.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:40 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:40.593 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:40 smithi196 ceph-mon[32919]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-23T10:20:40.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:40 smithi183 ceph-mon[32317]: osdmap e40: 6 total, 6 up, 6 in 2024-04-23T10:20:40.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:40 smithi183 ceph-mon[32317]: Detected new or changed devices on smithi196 2024-04-23T10:20:40.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:40 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:40.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:40 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:40.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:40 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:20:40.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:40 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:20:40.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:40 smithi183 ceph-mon[32317]: Adjusting osd_memory_target on smithi196 to 8544M 2024-04-23T10:20:40.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:40 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:40.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:40 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:20:40.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:40 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:20:40.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:40 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:40.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:40 smithi183 ceph-mon[32317]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-23T10:20:40.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:20:40.678+0000 7f34cdb7a700 1 -- 172.21.15.97:0/562535672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34c8105e60 msgr2=0x7f34c8108250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:20:40.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:20:40.678+0000 7f34cdb7a700 1 --2- 172.21.15.97:0/562535672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34c8105e60 0x7f34c8108250 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f34bc009990 tx=0x7f34bc009ca0 comp rx=0 tx=0).stop 2024-04-23T10:20:40.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:20:40.678+0000 7f34cdb7a700 1 -- 172.21.15.97:0/562535672 shutdown_connections 2024-04-23T10:20:40.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:20:40.678+0000 7f34cdb7a700 1 --2- 172.21.15.97:0/562535672 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34c8108790 0x7f34c810ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:20:40.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:20:40.678+0000 7f34cdb7a700 1 --2- 172.21.15.97:0/562535672 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f34c8103530 0x7f34c8105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:20:40.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:20:40.678+0000 7f34cdb7a700 1 --2- 172.21.15.97:0/562535672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34c8105e60 0x7f34c8108250 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:20:40.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:20:40.678+0000 7f34cdb7a700 1 -- 172.21.15.97:0/562535672 >> 172.21.15.97:0/562535672 conn(0x7f34c80fcf60 msgr2=0x7f34c80ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:20:40.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:20:40.678+0000 7f34cdb7a700 1 -- 172.21.15.97:0/562535672 shutdown_connections 2024-04-23T10:20:40.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:20:40.678+0000 7f34cdb7a700 1 -- 172.21.15.97:0/562535672 wait complete. 2024-04-23T10:20:40.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:20:40.679+0000 7f34cdb7a700 1 Processor -- start 2024-04-23T10:20:40.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:20:40.679+0000 7f34cdb7a700 1 -- start start 2024-04-23T10:20:40.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:20:40.679+0000 7f34cdb7a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34c8103530 0x7f34c8198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:20:40.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:20:40.679+0000 7f34cdb7a700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f34c8105e60 0x7f34c8199090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:20:40.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:20:40.679+0000 7f34cdb7a700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34c8108790 0x7f34c81a0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:20:40.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:20:40.679+0000 7f34cdb7a700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f34c810e550 con 0x7f34c8103530 2024-04-23T10:20:40.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:20:40.679+0000 7f34cdb7a700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f34c810e3d0 con 0x7f34c8105e60 2024-04-23T10:20:40.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:20:40.679+0000 7f34cdb7a700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f34c810e6d0 con 0x7f34c8108790 2024-04-23T10:20:40.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:20:40.679+0000 7f34c77fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34c8103530 0x7f34c8198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:20:40.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:20:40.679+0000 7f34c77fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34c8103530 0x7f34c8198b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42240/0 (socket says 172.21.15.97:42240) 2024-04-23T10:20:40.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:20:40.679+0000 7f34c77fe700 1 -- 172.21.15.97:0/1953636572 learned_addr learned my addr 172.21.15.97:0/1953636572 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:20:40.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:20:40.679+0000 7f34c6ffd700 1 --2- 172.21.15.97:0/1953636572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f34c8105e60 0x7f34c8199090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:20:40.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:20:40.679+0000 7f34c7fff700 1 --2- 172.21.15.97:0/1953636572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34c8108790 0x7f34c81a0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:20:40.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:20:40.679+0000 7f34c77fe700 1 -- 172.21.15.97:0/1953636572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34c8108790 msgr2=0x7f34c81a0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:20:40.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:20:40.679+0000 7f34c77fe700 1 --2- 172.21.15.97:0/1953636572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34c8108790 0x7f34c81a0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:20:40.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:20:40.679+0000 7f34c77fe700 1 -- 172.21.15.97:0/1953636572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f34c8105e60 msgr2=0x7f34c8199090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:20:40.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:20:40.679+0000 7f34c77fe700 1 --2- 172.21.15.97:0/1953636572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f34c8105e60 0x7f34c8199090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:20:40.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:20:40.679+0000 7f34c77fe700 1 -- 172.21.15.97:0/1953636572 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34c81a07b0 con 0x7f34c8103530 2024-04-23T10:20:40.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:20:40.679+0000 7f34c7fff700 1 --2- 172.21.15.97:0/1953636572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34c8108790 0x7f34c81a0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:20:40.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:20:40.680+0000 7f34c77fe700 1 --2- 172.21.15.97:0/1953636572 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34c8103530 0x7f34c8198b50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f34b800c9e0 tx=0x7f34b800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:20:40.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:20:40.680+0000 7f34c4ff9700 1 -- 172.21.15.97:0/1953636572 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b8007ac0 con 0x7f34c8103530 2024-04-23T10:20:40.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:20:40.680+0000 7f34c4ff9700 1 -- 172.21.15.97:0/1953636572 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34b8007c60 con 0x7f34c8103530 2024-04-23T10:20:40.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:20:40.680+0000 7f34cdb7a700 1 -- 172.21.15.97:0/1953636572 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34c81a0aa0 con 0x7f34c8103530 2024-04-23T10:20:40.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:20:40.680+0000 7f34c4ff9700 1 -- 172.21.15.97:0/1953636572 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b8018660 con 0x7f34c8103530 2024-04-23T10:20:40.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:20:40.680+0000 7f34cdb7a700 1 -- 172.21.15.97:0/1953636572 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34c81a0fe0 con 0x7f34c8103530 2024-04-23T10:20:40.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:20:40.681+0000 7f34cdb7a700 1 -- 172.21.15.97:0/1953636572 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34c8069730 con 0x7f34c8103530 2024-04-23T10:20:40.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:20:40.682+0000 7f34c4ff9700 1 -- 172.21.15.97:0/1953636572 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f34b800f450 con 0x7f34c8103530 2024-04-23T10:20:40.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:20:40.682+0000 7f34c4ff9700 1 --2- 172.21.15.97:0/1953636572 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f34b00a3c40 0x7f34b00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:20:40.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:20:40.682+0000 7f34c6ffd700 1 --2- 172.21.15.97:0/1953636572 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f34b00a3c40 0x7f34b00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:20:40.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:20:40.682+0000 7f34c4ff9700 1 -- 172.21.15.97:0/1953636572 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4562+0+0 (secure 0 0 0) 0x7f34b80cf7b0 con 0x7f34c8103530 2024-04-23T10:20:40.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:20:40.683+0000 7f34c6ffd700 1 --2- 172.21.15.97:0/1953636572 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f34b00a3c40 0x7f34b00a6100 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f34bc00b5c0 tx=0x7f34bc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:20:40.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:20:40.686+0000 7f34c4ff9700 1 -- 172.21.15.97:0/1953636572 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f34b8014030 con 0x7f34c8103530 2024-04-23T10:20:40.894 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:20:40.895+0000 7f34cdb7a700 1 -- 172.21.15.97:0/1953636572 --> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f34c8062e80 con 0x7f34b00a3c40 2024-04-23T10:20:41.503 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:41 smithi097 ceph-mon[29197]: osdmap e41: 6 total, 6 up, 6 in 2024-04-23T10:20:41.503 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:41 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T10:20:41.503 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:41 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T10:20:41.503 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:41 smithi097 ceph-mon[29197]: osdmap e42: 6 total, 6 up, 6 in 2024-04-23T10:20:41.503 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:41 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:20:41.512 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:41 smithi196 ceph-mon[32919]: osdmap e41: 6 total, 6 up, 6 in 2024-04-23T10:20:41.512 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:41 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T10:20:41.513 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:41 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T10:20:41.513 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:41 smithi196 ceph-mon[32919]: osdmap e42: 6 total, 6 up, 6 in 2024-04-23T10:20:41.513 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:41 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:20:41.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:41 smithi183 ceph-mon[32317]: osdmap e41: 6 total, 6 up, 6 in 2024-04-23T10:20:41.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:41 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T10:20:41.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:41 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T10:20:41.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:41 smithi183 ceph-mon[32317]: osdmap e42: 6 total, 6 up, 6 in 2024-04-23T10:20:41.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:41 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:20:42.528 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:42 smithi097 ceph-mon[29197]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:20:42.528 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:42 smithi097 ceph-mon[29197]: pgmap v143: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-04-23T10:20:42.528 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:42 smithi097 ceph-mon[29197]: osdmap e43: 6 total, 6 up, 6 in 2024-04-23T10:20:42.551 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:42 smithi183 ceph-mon[32317]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:20:42.551 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:42 smithi183 ceph-mon[32317]: pgmap v143: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-04-23T10:20:42.551 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:42 smithi183 ceph-mon[32317]: osdmap e43: 6 total, 6 up, 6 in 2024-04-23T10:20:42.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:42 smithi196 ceph-mon[32919]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:20:42.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:42 smithi196 ceph-mon[32919]: pgmap v143: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-04-23T10:20:42.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:42 smithi196 ceph-mon[32919]: osdmap e43: 6 total, 6 up, 6 in 2024-04-23T10:20:43.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:43 smithi196 ceph-mon[32919]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-04-23T10:20:43.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:43 smithi097 ceph-mon[29197]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-04-23T10:20:43.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:43 smithi183 ceph-mon[32317]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-04-23T10:20:44.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:44 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/3673653147' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "071abb5c-2edf-4832-b8aa-3b80d20096fc"}]: dispatch 2024-04-23T10:20:44.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:44 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/3673653147' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "071abb5c-2edf-4832-b8aa-3b80d20096fc"}]': finished 2024-04-23T10:20:44.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:44 smithi196 ceph-mon[32919]: osdmap e44: 7 total, 6 up, 7 in 2024-04-23T10:20:44.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:44 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T10:20:44.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:44 smithi196 ceph-mon[32919]: pgmap v146: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-04-23T10:20:44.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:44 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/932273917' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T10:20:44.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:44 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/3673653147' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "071abb5c-2edf-4832-b8aa-3b80d20096fc"}]: dispatch 2024-04-23T10:20:44.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:44 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/3673653147' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "071abb5c-2edf-4832-b8aa-3b80d20096fc"}]': finished 2024-04-23T10:20:44.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:44 smithi097 ceph-mon[29197]: osdmap e44: 7 total, 6 up, 7 in 2024-04-23T10:20:44.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:44 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T10:20:44.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:44 smithi097 ceph-mon[29197]: pgmap v146: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-04-23T10:20:44.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:44 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/932273917' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T10:20:44.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:44 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/3673653147' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "071abb5c-2edf-4832-b8aa-3b80d20096fc"}]: dispatch 2024-04-23T10:20:44.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:44 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/3673653147' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "071abb5c-2edf-4832-b8aa-3b80d20096fc"}]': finished 2024-04-23T10:20:44.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:44 smithi183 ceph-mon[32317]: osdmap e44: 7 total, 6 up, 7 in 2024-04-23T10:20:44.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:44 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T10:20:44.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:44 smithi183 ceph-mon[32317]: pgmap v146: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 2/6 objects degraded (33.333%) 2024-04-23T10:20:44.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:44 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/932273917' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T10:20:45.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:45 smithi183 ceph-mon[32317]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-04-23T10:20:45.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:45 smithi183 ceph-mon[32317]: Cluster is now healthy 2024-04-23T10:20:46.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:45 smithi196 ceph-mon[32919]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-04-23T10:20:46.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:45 smithi196 ceph-mon[32919]: Cluster is now healthy 2024-04-23T10:20:46.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:45 smithi097 ceph-mon[29197]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-04-23T10:20:46.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:45 smithi097 ceph-mon[29197]: Cluster is now healthy 2024-04-23T10:20:46.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:46 smithi183 ceph-mon[32317]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-04-23T10:20:47.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:46 smithi196 ceph-mon[32919]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-04-23T10:20:47.093 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:46 smithi097 ceph-mon[29197]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-04-23T10:20:48.771 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:48 smithi097 ceph-mon[29197]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-04-23T10:20:48.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:48 smithi183 ceph-mon[32317]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-04-23T10:20:49.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:48 smithi196 ceph-mon[32919]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-04-23T10:20:50.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:50 smithi097 ceph-mon[29197]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-23T10:20:50.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:50 smithi183 ceph-mon[32317]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-23T10:20:51.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:50 smithi196 ceph-mon[32919]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-23T10:20:51.813 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:51 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-23T10:20:51.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:51 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:20:51.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:51 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-23T10:20:51.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:51 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:20:52.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:51 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-23T10:20:52.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:51 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:20:52.791 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:52 smithi097 ceph-mon[29197]: Deploying daemon osd.6 on smithi097 2024-04-23T10:20:52.792 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:52 smithi097 ceph-mon[29197]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-23T10:20:52.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:52 smithi183 ceph-mon[32317]: Deploying daemon osd.6 on smithi097 2024-04-23T10:20:52.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:52 smithi183 ceph-mon[32317]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-23T10:20:53.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:52 smithi196 ceph-mon[32919]: Deploying daemon osd.6 on smithi097 2024-04-23T10:20:53.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:52 smithi196 ceph-mon[32919]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-23T10:20:54.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:54 smithi183 ceph-mon[32317]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-04-23T10:20:54.957 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:54 smithi097 ceph-mon[29197]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-04-23T10:20:55.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:54 smithi196 ceph-mon[32919]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-04-23T10:20:56.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:56 smithi097 ceph-mon[29197]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-23T10:20:56.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:56 smithi183 ceph-mon[32317]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-23T10:20:57.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:56 smithi196 ceph-mon[32919]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-23T10:20:58.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:57 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:58.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:57 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:20:58.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:57 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:58.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:20:57 smithi196 ceph-mon[32919]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-23T10:20:58.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:57 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:58.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:57 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:20:58.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:57 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:58.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:20:57 smithi097 ceph-mon[29197]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-23T10:20:58.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:57 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:58.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:57 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:20:58.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:57 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:20:58.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:20:57 smithi183 ceph-mon[32317]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-23T10:21:00.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:00 smithi196 ceph-mon[32919]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-23T10:21:00.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:00 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:00.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:00 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:00.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:00 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:21:00.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:00 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:21:00.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:00 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:00.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:00 smithi097 ceph-mon[29197]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-23T10:21:00.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:00 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:00.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:00 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:00.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:00 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:21:00.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:00 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:21:00.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:00 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:00.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:00 smithi183 ceph-mon[32317]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-23T10:21:00.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:00 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:00.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:00 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:00.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:00 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:21:00.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:00 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:21:00.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:00 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:01.073 INFO:teuthology.orchestra.run.smithi097.stdout:Created osd(s) 6 on host 'smithi097' 2024-04-23T10:21:01.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:21:01.074+0000 7f34c4ff9700 1 -- 172.21.15.97:0/1953636572 <== mgr.14152 v2:172.21.15.97:6800/3149043749 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f34c8062e80 con 0x7f34b00a3c40 2024-04-23T10:21:01.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:21:01.076+0000 7f34cdb7a700 1 -- 172.21.15.97:0/1953636572 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f34b00a3c40 msgr2=0x7f34b00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:21:01.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:21:01.076+0000 7f34cdb7a700 1 --2- 172.21.15.97:0/1953636572 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f34b00a3c40 0x7f34b00a6100 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f34bc00b5c0 tx=0x7f34bc005fb0 comp rx=0 tx=0).stop 2024-04-23T10:21:01.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:21:01.077+0000 7f34cdb7a700 1 -- 172.21.15.97:0/1953636572 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34c8103530 msgr2=0x7f34c8198b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:21:01.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:21:01.077+0000 7f34cdb7a700 1 --2- 172.21.15.97:0/1953636572 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34c8103530 0x7f34c8198b50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f34b800c9e0 tx=0x7f34b800ccf0 comp rx=0 tx=0).stop 2024-04-23T10:21:01.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:21:01.077+0000 7f34cdb7a700 1 -- 172.21.15.97:0/1953636572 shutdown_connections 2024-04-23T10:21:01.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:21:01.077+0000 7f34cdb7a700 1 --2- 172.21.15.97:0/1953636572 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f34b00a3c40 0x7f34b00a6100 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:21:01.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:21:01.077+0000 7f34cdb7a700 1 --2- 172.21.15.97:0/1953636572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34c8108790 0x7f34c81a0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:21:01.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:21:01.077+0000 7f34cdb7a700 1 --2- 172.21.15.97:0/1953636572 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34c8103530 0x7f34c8198b50 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:21:01.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:21:01.077+0000 7f34cdb7a700 1 --2- 172.21.15.97:0/1953636572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f34c8105e60 0x7f34c8199090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:21:01.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:21:01.077+0000 7f34cdb7a700 1 -- 172.21.15.97:0/1953636572 >> 172.21.15.97:0/1953636572 conn(0x7f34c80fcf60 msgr2=0x7f34c8103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:21:01.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:21:01.077+0000 7f34cdb7a700 1 -- 172.21.15.97:0/1953636572 shutdown_connections 2024-04-23T10:21:01.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:21:01.077+0000 7f34cdb7a700 1 -- 172.21.15.97:0/1953636572 wait complete. 2024-04-23T10:21:01.811 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:01 smithi097 ceph-mon[29197]: from='osd.6 [v2:172.21.15.97:6818/2080916705,v1:172.21.15.97:6819/2080916705]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-23T10:21:01.811 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:01 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:01.811 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:01 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:21:01.811 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:01 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:01.841 DEBUG:teuthology.orchestra.run.smithi097:osd.6> sudo journalctl -f -n 0 -u ceph-9b22af88-0159-11ef-bc93-c7b262605968@osd.6.service 2024-04-23T10:21:01.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:01 smithi196 ceph-mon[32919]: from='osd.6 [v2:172.21.15.97:6818/2080916705,v1:172.21.15.97:6819/2080916705]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-23T10:21:01.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:01 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:01.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:01 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:21:01.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:01 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:01.844 INFO:tasks.cephadm:Deploying osd.7 on smithi183 with /dev/vg_nvme/lv_2... 2024-04-23T10:21:01.844 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-23T10:21:01.871 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:01 smithi183 ceph-mon[32317]: from='osd.6 [v2:172.21.15.97:6818/2080916705,v1:172.21.15.97:6819/2080916705]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-23T10:21:01.871 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:01 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:01.871 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:01 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:21:01.871 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:01 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:02.090 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.b/config 2024-04-23T10:21:02.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:02 smithi196 ceph-mon[32919]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-23T10:21:02.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:02 smithi196 ceph-mon[32919]: from='osd.6 [v2:172.21.15.97:6818/2080916705,v1:172.21.15.97:6819/2080916705]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-23T10:21:02.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:02 smithi196 ceph-mon[32919]: osdmap e45: 7 total, 6 up, 7 in 2024-04-23T10:21:02.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:02 smithi196 ceph-mon[32919]: from='osd.6 [v2:172.21.15.97:6818/2080916705,v1:172.21.15.97:6819/2080916705]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-23T10:21:02.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:02 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T10:21:02.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:02 smithi097 ceph-mon[29197]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-23T10:21:02.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:02 smithi097 ceph-mon[29197]: from='osd.6 [v2:172.21.15.97:6818/2080916705,v1:172.21.15.97:6819/2080916705]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-23T10:21:02.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:02 smithi097 ceph-mon[29197]: osdmap e45: 7 total, 6 up, 7 in 2024-04-23T10:21:02.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:02 smithi097 ceph-mon[29197]: from='osd.6 [v2:172.21.15.97:6818/2080916705,v1:172.21.15.97:6819/2080916705]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-23T10:21:02.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:02 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T10:21:02.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:02 smithi183 ceph-mon[32317]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-23T10:21:02.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:02 smithi183 ceph-mon[32317]: from='osd.6 [v2:172.21.15.97:6818/2080916705,v1:172.21.15.97:6819/2080916705]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-23T10:21:02.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:02 smithi183 ceph-mon[32317]: osdmap e45: 7 total, 6 up, 7 in 2024-04-23T10:21:02.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:02 smithi183 ceph-mon[32317]: from='osd.6 [v2:172.21.15.97:6818/2080916705,v1:172.21.15.97:6819/2080916705]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-23T10:21:02.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:02 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T10:21:03.300 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:21:03 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6[54668]: 2024-04-23T10:21:03.020+0000 7f85abecd700 -1 osd.6 0 waiting for initial osdmap 2024-04-23T10:21:03.300 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:21:03 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6[54668]: 2024-04-23T10:21:03.030+0000 7f85a3cb8700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T10:21:03.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:03 smithi196 ceph-mon[32919]: from='osd.6 [v2:172.21.15.97:6818/2080916705,v1:172.21.15.97:6819/2080916705]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-23T10:21:03.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:03 smithi196 ceph-mon[32919]: osdmap e46: 7 total, 6 up, 7 in 2024-04-23T10:21:03.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:03 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T10:21:03.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:03 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T10:21:03.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:03 smithi196 ceph-mon[32919]: from='osd.6 [v2:172.21.15.97:6818/2080916705,v1:172.21.15.97:6819/2080916705]' entity='osd.6' 2024-04-23T10:21:03.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:03 smithi097 ceph-mon[29197]: from='osd.6 [v2:172.21.15.97:6818/2080916705,v1:172.21.15.97:6819/2080916705]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-23T10:21:03.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:03 smithi097 ceph-mon[29197]: osdmap e46: 7 total, 6 up, 7 in 2024-04-23T10:21:03.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:03 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T10:21:03.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:03 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T10:21:03.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:03 smithi097 ceph-mon[29197]: from='osd.6 [v2:172.21.15.97:6818/2080916705,v1:172.21.15.97:6819/2080916705]' entity='osd.6' 2024-04-23T10:21:03.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:03 smithi183 ceph-mon[32317]: from='osd.6 [v2:172.21.15.97:6818/2080916705,v1:172.21.15.97:6819/2080916705]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-23T10:21:03.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:03 smithi183 ceph-mon[32317]: osdmap e46: 7 total, 6 up, 7 in 2024-04-23T10:21:03.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:03 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T10:21:03.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:03 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T10:21:03.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:03 smithi183 ceph-mon[32317]: from='osd.6 [v2:172.21.15.97:6818/2080916705,v1:172.21.15.97:6819/2080916705]' entity='osd.6' 2024-04-23T10:21:04.089 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-23T10:21:04.089 INFO:teuthology.orchestra.run.smithi183.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-23T10:21:04.089 INFO:teuthology.orchestra.run.smithi183.stderr: stderr: 10+0 records in 2024-04-23T10:21:04.089 INFO:teuthology.orchestra.run.smithi183.stderr:10+0 records out 2024-04-23T10:21:04.089 INFO:teuthology.orchestra.run.smithi183.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113791 s, 921 MB/s 2024-04-23T10:21:04.089 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping successful for: 2024-04-23T10:21:04.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:04 smithi183 ceph-mon[32317]: purged_snaps scrub starts 2024-04-23T10:21:04.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:04 smithi183 ceph-mon[32317]: purged_snaps scrub ok 2024-04-23T10:21:04.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:04 smithi183 ceph-mon[32317]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-23T10:21:04.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:04 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T10:21:04.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:04 smithi183 ceph-mon[32317]: osd.6 [v2:172.21.15.97:6818/2080916705,v1:172.21.15.97:6819/2080916705] boot 2024-04-23T10:21:04.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:04 smithi183 ceph-mon[32317]: osdmap e47: 7 total, 7 up, 7 in 2024-04-23T10:21:04.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:04 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T10:21:04.751 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi183:vg_nvme/lv_2 2024-04-23T10:21:04.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:04 smithi196 ceph-mon[32919]: purged_snaps scrub starts 2024-04-23T10:21:04.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:04 smithi196 ceph-mon[32919]: purged_snaps scrub ok 2024-04-23T10:21:04.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:04 smithi196 ceph-mon[32919]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-23T10:21:04.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:04 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T10:21:04.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:04 smithi196 ceph-mon[32919]: osd.6 [v2:172.21.15.97:6818/2080916705,v1:172.21.15.97:6819/2080916705] boot 2024-04-23T10:21:04.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:04 smithi196 ceph-mon[32919]: osdmap e47: 7 total, 7 up, 7 in 2024-04-23T10:21:04.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:04 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T10:21:04.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:04 smithi097 ceph-mon[29197]: purged_snaps scrub starts 2024-04-23T10:21:04.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:04 smithi097 ceph-mon[29197]: purged_snaps scrub ok 2024-04-23T10:21:04.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:04 smithi097 ceph-mon[29197]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-23T10:21:04.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:04 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T10:21:04.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:04 smithi097 ceph-mon[29197]: osd.6 [v2:172.21.15.97:6818/2080916705,v1:172.21.15.97:6819/2080916705] boot 2024-04-23T10:21:04.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:04 smithi097 ceph-mon[29197]: osdmap e47: 7 total, 7 up, 7 in 2024-04-23T10:21:04.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:04 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T10:21:05.032 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.b/config 2024-04-23T10:21:06.632 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:21:06.625+0000 7f3c0e704700 1 -- 172.21.15.183:0/376430817 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c08100b90 msgr2=0x7f3c08100ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:21:06.632 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:21:06.625+0000 7f3c0e704700 1 --2- 172.21.15.183:0/376430817 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c08100b90 0x7f3c08100ff0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3bf8009990 tx=0x7f3bf8009ca0 comp rx=0 tx=0).stop 2024-04-23T10:21:06.632 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:21:06.625+0000 7f3c0e704700 1 -- 172.21.15.183:0/376430817 shutdown_connections 2024-04-23T10:21:06.632 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:21:06.625+0000 7f3c0e704700 1 --2- 172.21.15.183:0/376430817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c08101530 0x7f3c0810b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:21:06.632 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:21:06.625+0000 7f3c0e704700 1 --2- 172.21.15.183:0/376430817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c08104110 0x7f3c08104530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:21:06.632 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:21:06.625+0000 7f3c0e704700 1 --2- 172.21.15.183:0/376430817 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c08100b90 0x7f3c08100ff0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:21:06.632 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:21:06.625+0000 7f3c0e704700 1 -- 172.21.15.183:0/376430817 >> 172.21.15.183:0/376430817 conn(0x7f3c080fc710 msgr2=0x7f3c080feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:21:06.632 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:21:06.625+0000 7f3c0e704700 1 -- 172.21.15.183:0/376430817 shutdown_connections 2024-04-23T10:21:06.633 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:21:06.625+0000 7f3c0e704700 1 -- 172.21.15.183:0/376430817 wait complete. 2024-04-23T10:21:06.633 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:21:06.626+0000 7f3c0e704700 1 Processor -- start 2024-04-23T10:21:06.633 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:21:06.626+0000 7f3c0e704700 1 -- start start 2024-04-23T10:21:06.633 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:21:06.626+0000 7f3c0e704700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c08100b90 0x7f3c08198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:21:06.633 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:21:06.626+0000 7f3c0e704700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c08101530 0x7f3c081990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:21:06.633 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:21:06.626+0000 7f3c0e704700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c08104110 0x7f3c081a0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:21:06.633 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:21:06.626+0000 7f3c0e704700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0810e560 con 0x7f3c08101530 2024-04-23T10:21:06.633 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:21:06.626+0000 7f3c0e704700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0810e3e0 con 0x7f3c08104110 2024-04-23T10:21:06.633 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:21:06.626+0000 7f3c0e704700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0810e6e0 con 0x7f3c08100b90 2024-04-23T10:21:06.633 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:21:06.626+0000 7f3c0cca1700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c08104110 0x7f3c081a0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:21:06.633 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:21:06.626+0000 7f3c0cca1700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c08104110 0x7f3c081a0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:38546/0 (socket says 172.21.15.183:38546) 2024-04-23T10:21:06.633 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:21:06.626+0000 7f3c0cca1700 1 -- 172.21.15.183:0/397177282 learned_addr learned my addr 172.21.15.183:0/397177282 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-04-23T10:21:06.633 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:21:06.626+0000 7f3c07fff700 1 --2- 172.21.15.183:0/397177282 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c08100b90 0x7f3c08198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:21:06.633 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:21:06.627+0000 7f3c077fe700 1 --2- 172.21.15.183:0/397177282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c08101530 0x7f3c081990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:21:06.634 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:21:06.627+0000 7f3c0cca1700 1 -- 172.21.15.183:0/397177282 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c08100b90 msgr2=0x7f3c08198b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:21:06.634 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:21:06.627+0000 7f3c0cca1700 1 --2- 172.21.15.183:0/397177282 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c08100b90 0x7f3c08198b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:21:06.634 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:21:06.627+0000 7f3c0cca1700 1 -- 172.21.15.183:0/397177282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c08101530 msgr2=0x7f3c081990d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:21:06.634 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:21:06.627+0000 7f3c0cca1700 1 --2- 172.21.15.183:0/397177282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c08101530 0x7f3c081990d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:21:06.634 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:21:06.627+0000 7f3c0cca1700 1 -- 172.21.15.183:0/397177282 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c081a07f0 con 0x7f3c08104110 2024-04-23T10:21:06.634 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:21:06.627+0000 7f3c0cca1700 1 --2- 172.21.15.183:0/397177282 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c08104110 0x7f3c081a0150 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3c00007d70 tx=0x7f3c0000a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:21:06.634 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:21:06.627+0000 7f3c057fa700 1 -- 172.21.15.183:0/397177282 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c00016020 con 0x7f3c08104110 2024-04-23T10:21:06.634 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:21:06.627+0000 7f3c0e704700 1 -- 172.21.15.183:0/397177282 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c081a0ae0 con 0x7f3c08104110 2024-04-23T10:21:06.634 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:21:06.627+0000 7f3c0e704700 1 -- 172.21.15.183:0/397177282 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c081a1020 con 0x7f3c08104110 2024-04-23T10:21:06.634 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:21:06.627+0000 7f3c057fa700 1 -- 172.21.15.183:0/397177282 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c0000ad40 con 0x7f3c08104110 2024-04-23T10:21:06.634 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:21:06.627+0000 7f3c057fa700 1 -- 172.21.15.183:0/397177282 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0001b660 con 0x7f3c08104110 2024-04-23T10:21:06.634 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:21:06.628+0000 7f3c0e704700 1 -- 172.21.15.183:0/397177282 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3be8005360 con 0x7f3c08104110 2024-04-23T10:21:06.634 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:21:06.629+0000 7f3c057fa700 1 -- 172.21.15.183:0/397177282 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3c00019020 con 0x7f3c08104110 2024-04-23T10:21:06.634 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:21:06.630+0000 7f3c057fa700 1 --2- 172.21.15.183:0/397177282 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f3bf40a3c40 0x7f3bf40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:21:06.634 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:21:06.630+0000 7f3c057fa700 1 -- 172.21.15.183:0/397177282 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4965+0+0 (secure 0 0 0) 0x7f3c000cfba0 con 0x7f3c08104110 2024-04-23T10:21:06.635 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:21:06.630+0000 7f3c07fff700 1 --2- 172.21.15.183:0/397177282 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f3bf40a3c40 0x7f3bf40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:21:06.635 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:21:06.630+0000 7f3c07fff700 1 --2- 172.21.15.183:0/397177282 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f3bf40a3c40 0x7f3bf40a6100 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3c0806ae50 tx=0x7f3bf000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:21:06.635 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:21:06.633+0000 7f3c057fa700 1 -- 172.21.15.183:0/397177282 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3c00020030 con 0x7f3c08104110 2024-04-23T10:21:06.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:06 smithi183 ceph-mon[32317]: Detected new or changed devices on smithi097 2024-04-23T10:21:06.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:06 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:06.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:06 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:06.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:06 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:21:06.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:06 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:21:06.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:06 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:21:06.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:06 smithi183 ceph-mon[32317]: Adjusting osd_memory_target on smithi097 to 5696M 2024-04-23T10:21:06.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:06 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:06.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:06 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:21:06.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:06 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:21:06.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:06 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:06.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:06 smithi183 ceph-mon[32317]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T10:21:06.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:06 smithi183 ceph-mon[32317]: osdmap e48: 7 total, 7 up, 7 in 2024-04-23T10:21:06.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:06 smithi196 ceph-mon[32919]: Detected new or changed devices on smithi097 2024-04-23T10:21:06.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:06 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:06.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:06 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:06.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:06 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:21:06.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:06 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:21:06.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:06 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:21:06.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:06 smithi196 ceph-mon[32919]: Adjusting osd_memory_target on smithi097 to 5696M 2024-04-23T10:21:06.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:06 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:06.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:06 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:21:06.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:06 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:21:06.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:06 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:06.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:06 smithi196 ceph-mon[32919]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T10:21:06.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:06 smithi196 ceph-mon[32919]: osdmap e48: 7 total, 7 up, 7 in 2024-04-23T10:21:06.843 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:21:06.843+0000 7f3c0e704700 1 -- 172.21.15.183:0/397177282 --> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f3be8000bf0 con 0x7f3bf40a3c40 2024-04-23T10:21:06.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:06 smithi097 ceph-mon[29197]: Detected new or changed devices on smithi097 2024-04-23T10:21:06.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:06 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:06.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:06 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:06.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:06 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:21:06.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:06 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:21:06.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:06 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:21:06.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:06 smithi097 ceph-mon[29197]: Adjusting osd_memory_target on smithi097 to 5696M 2024-04-23T10:21:06.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:06 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:06.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:06 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:21:06.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:06 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:21:06.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:06 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:06.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:06 smithi097 ceph-mon[29197]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T10:21:06.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:06 smithi097 ceph-mon[29197]: osdmap e48: 7 total, 7 up, 7 in 2024-04-23T10:21:07.702 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:07 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T10:21:07.702 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:07 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T10:21:07.702 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:07 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:21:07.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:07 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T10:21:07.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:07 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T10:21:07.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:07 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:21:07.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:07 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T10:21:07.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:07 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T10:21:07.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:07 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:21:08.753 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:08 smithi183 ceph-mon[32317]: from='client.24254 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:21:08.753 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:08 smithi183 ceph-mon[32317]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T10:21:08.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:08 smithi196 ceph-mon[32919]: from='client.24254 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:21:08.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:08 smithi196 ceph-mon[32919]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T10:21:08.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:08 smithi097 ceph-mon[29197]: from='client.24254 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:21:08.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:08 smithi097 ceph-mon[29197]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T10:21:10.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:10 smithi196 ceph-mon[32919]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T10:21:10.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:10 smithi196 ceph-mon[32919]: from='client.? 172.21.15.183:0/3588213999' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9d54760b-66df-486b-8abb-017f3b64b4d7"}]: dispatch 2024-04-23T10:21:10.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:10 smithi196 ceph-mon[32919]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9d54760b-66df-486b-8abb-017f3b64b4d7"}]: dispatch 2024-04-23T10:21:10.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:10 smithi196 ceph-mon[32919]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9d54760b-66df-486b-8abb-017f3b64b4d7"}]': finished 2024-04-23T10:21:10.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:10 smithi196 ceph-mon[32919]: osdmap e49: 8 total, 7 up, 8 in 2024-04-23T10:21:10.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:10 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T10:21:10.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:10 smithi196 ceph-mon[32919]: from='client.? 172.21.15.183:0/2099125891' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T10:21:10.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:10 smithi097 ceph-mon[29197]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T10:21:10.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:10 smithi097 ceph-mon[29197]: from='client.? 172.21.15.183:0/3588213999' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9d54760b-66df-486b-8abb-017f3b64b4d7"}]: dispatch 2024-04-23T10:21:10.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:10 smithi097 ceph-mon[29197]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9d54760b-66df-486b-8abb-017f3b64b4d7"}]: dispatch 2024-04-23T10:21:10.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:10 smithi097 ceph-mon[29197]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9d54760b-66df-486b-8abb-017f3b64b4d7"}]': finished 2024-04-23T10:21:10.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:10 smithi097 ceph-mon[29197]: osdmap e49: 8 total, 7 up, 8 in 2024-04-23T10:21:10.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:10 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T10:21:10.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:10 smithi097 ceph-mon[29197]: from='client.? 172.21.15.183:0/2099125891' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T10:21:10.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:10 smithi183 ceph-mon[32317]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T10:21:10.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:10 smithi183 ceph-mon[32317]: from='client.? 172.21.15.183:0/3588213999' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9d54760b-66df-486b-8abb-017f3b64b4d7"}]: dispatch 2024-04-23T10:21:10.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:10 smithi183 ceph-mon[32317]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9d54760b-66df-486b-8abb-017f3b64b4d7"}]: dispatch 2024-04-23T10:21:10.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:10 smithi183 ceph-mon[32317]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9d54760b-66df-486b-8abb-017f3b64b4d7"}]': finished 2024-04-23T10:21:10.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:10 smithi183 ceph-mon[32317]: osdmap e49: 8 total, 7 up, 8 in 2024-04-23T10:21:10.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:10 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T10:21:10.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:10 smithi183 ceph-mon[32317]: from='client.? 172.21.15.183:0/2099125891' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T10:21:12.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:12 smithi196 ceph-mon[32919]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T10:21:12.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:12 smithi097 ceph-mon[29197]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T10:21:12.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:12 smithi183 ceph-mon[32317]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T10:21:14.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:14 smithi196 ceph-mon[32919]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T10:21:14.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:14 smithi183 ceph-mon[32317]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T10:21:14.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:14 smithi097 ceph-mon[29197]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T10:21:16.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:16 smithi196 ceph-mon[32919]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T10:21:16.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:16 smithi097 ceph-mon[29197]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T10:21:16.896 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:16 smithi183 ceph-mon[32317]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T10:21:17.789 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:17 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-23T10:21:17.789 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:17 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:21:17.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:17 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-23T10:21:17.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:17 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:21:17.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:17 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-23T10:21:17.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:17 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:21:18.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:18 smithi196 ceph-mon[32919]: Deploying daemon osd.7 on smithi183 2024-04-23T10:21:18.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:18 smithi196 ceph-mon[32919]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T10:21:18.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:18 smithi097 ceph-mon[29197]: Deploying daemon osd.7 on smithi183 2024-04-23T10:21:18.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:18 smithi097 ceph-mon[29197]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T10:21:18.875 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:18 smithi183 ceph-mon[32317]: Deploying daemon osd.7 on smithi183 2024-04-23T10:21:18.875 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:18 smithi183 ceph-mon[32317]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T10:21:20.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:20 smithi196 ceph-mon[32919]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T10:21:20.845 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:20 smithi183 ceph-mon[32317]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T10:21:20.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:20 smithi097 ceph-mon[29197]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T10:21:22.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:22 smithi196 ceph-mon[32919]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T10:21:22.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:22 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:22.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:22 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:21:22.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:22 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:22.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:22 smithi097 ceph-mon[29197]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T10:21:22.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:22 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:22.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:22 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:21:22.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:22 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:22.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:22 smithi183 ceph-mon[32317]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T10:21:22.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:22 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:22.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:22 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:21:22.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:22 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:24.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:24 smithi097 ceph-mon[29197]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T10:21:24.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:24 smithi183 ceph-mon[32317]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T10:21:25.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:24 smithi196 ceph-mon[32919]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T10:21:26.155 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:21:26.155+0000 7f3c057fa700 1 -- 172.21.15.183:0/397177282 <== mgr.14152 v2:172.21.15.97:6800/3149043749 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3be8000bf0 con 0x7f3bf40a3c40 2024-04-23T10:21:26.155 INFO:teuthology.orchestra.run.smithi183.stdout:Created osd(s) 7 on host 'smithi183' 2024-04-23T10:21:26.157 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:21:26.158+0000 7f3c0e704700 1 -- 172.21.15.183:0/397177282 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f3bf40a3c40 msgr2=0x7f3bf40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:21:26.157 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:21:26.158+0000 7f3c0e704700 1 --2- 172.21.15.183:0/397177282 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f3bf40a3c40 0x7f3bf40a6100 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3c0806ae50 tx=0x7f3bf000b410 comp rx=0 tx=0).stop 2024-04-23T10:21:26.157 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:21:26.158+0000 7f3c0e704700 1 -- 172.21.15.183:0/397177282 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c08104110 msgr2=0x7f3c081a0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:21:26.157 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:21:26.158+0000 7f3c0e704700 1 --2- 172.21.15.183:0/397177282 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c08104110 0x7f3c081a0150 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3c00007d70 tx=0x7f3c0000a340 comp rx=0 tx=0).stop 2024-04-23T10:21:26.157 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:21:26.158+0000 7f3c0e704700 1 -- 172.21.15.183:0/397177282 shutdown_connections 2024-04-23T10:21:26.157 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:21:26.158+0000 7f3c0e704700 1 --2- 172.21.15.183:0/397177282 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f3bf40a3c40 0x7f3bf40a6100 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:21:26.158 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:21:26.158+0000 7f3c0e704700 1 --2- 172.21.15.183:0/397177282 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c08104110 0x7f3c081a0150 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:21:26.158 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:21:26.158+0000 7f3c0e704700 1 --2- 172.21.15.183:0/397177282 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c08100b90 0x7f3c08198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:21:26.158 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:21:26.158+0000 7f3c0e704700 1 --2- 172.21.15.183:0/397177282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c08101530 0x7f3c081990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:21:26.158 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:21:26.158+0000 7f3c0e704700 1 -- 172.21.15.183:0/397177282 >> 172.21.15.183:0/397177282 conn(0x7f3c080fc710 msgr2=0x7f3c08107150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:21:26.158 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:21:26.158+0000 7f3c0e704700 1 -- 172.21.15.183:0/397177282 shutdown_connections 2024-04-23T10:21:26.158 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:21:26.158+0000 7f3c0e704700 1 -- 172.21.15.183:0/397177282 wait complete. 2024-04-23T10:21:26.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:26 smithi183 ceph-mon[32317]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T10:21:26.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:26 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:26.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:26 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:26.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:26 smithi183 ceph-mon[32317]: from='osd.7 [v2:172.21.15.183:6816/1865873033,v1:172.21.15.183:6817/1865873033]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-23T10:21:26.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:26 smithi183 ceph-mon[32317]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-23T10:21:26.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:26 smithi097 ceph-mon[29197]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T10:21:26.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:26 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:26.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:26 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:26.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:26 smithi097 ceph-mon[29197]: from='osd.7 [v2:172.21.15.183:6816/1865873033,v1:172.21.15.183:6817/1865873033]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-23T10:21:26.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:26 smithi097 ceph-mon[29197]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-23T10:21:26.869 DEBUG:teuthology.orchestra.run.smithi183:osd.7> sudo journalctl -f -n 0 -u ceph-9b22af88-0159-11ef-bc93-c7b262605968@osd.7.service 2024-04-23T10:21:26.872 INFO:tasks.cephadm:Deploying osd.8 on smithi196 with /dev/vg_nvme/lv_2... 2024-04-23T10:21:26.872 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-23T10:21:26.898 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:26 smithi196 ceph-mon[32919]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T10:21:26.898 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:26 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:26.898 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:26 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:26.898 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:26 smithi196 ceph-mon[32919]: from='osd.7 [v2:172.21.15.183:6816/1865873033,v1:172.21.15.183:6817/1865873033]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-23T10:21:26.898 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:26 smithi196 ceph-mon[32919]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-23T10:21:27.116 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T10:21:28.284 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:28 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:28.285 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:28 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:28.285 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:28 smithi196 ceph-mon[32919]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-23T10:21:28.285 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:28 smithi196 ceph-mon[32919]: osdmap e50: 8 total, 7 up, 8 in 2024-04-23T10:21:28.285 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:28 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T10:21:28.285 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:28 smithi196 ceph-mon[32919]: from='osd.7 [v2:172.21.15.183:6816/1865873033,v1:172.21.15.183:6817/1865873033]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-23T10:21:28.285 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:28 smithi196 ceph-mon[32919]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-23T10:21:28.285 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:28 smithi196 ceph-mon[32919]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T10:21:28.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:28 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:28.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:28 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:28.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:28 smithi097 ceph-mon[29197]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-23T10:21:28.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:28 smithi097 ceph-mon[29197]: osdmap e50: 8 total, 7 up, 8 in 2024-04-23T10:21:28.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:28 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T10:21:28.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:28 smithi097 ceph-mon[29197]: from='osd.7 [v2:172.21.15.183:6816/1865873033,v1:172.21.15.183:6817/1865873033]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-23T10:21:28.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:28 smithi097 ceph-mon[29197]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-23T10:21:28.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:28 smithi097 ceph-mon[29197]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T10:21:28.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:28 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:28.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:28 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:28.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:28 smithi183 ceph-mon[32317]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-23T10:21:28.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:28 smithi183 ceph-mon[32317]: osdmap e50: 8 total, 7 up, 8 in 2024-04-23T10:21:28.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:28 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T10:21:28.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:28 smithi183 ceph-mon[32317]: from='osd.7 [v2:172.21.15.183:6816/1865873033,v1:172.21.15.183:6817/1865873033]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-23T10:21:28.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:28 smithi183 ceph-mon[32317]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-23T10:21:28.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:28 smithi183 ceph-mon[32317]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T10:21:28.817 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:21:28 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7[50726]: 2024-04-23T10:21:28.540+0000 7f5dc3f35700 -1 osd.7 0 waiting for initial osdmap 2024-04-23T10:21:28.817 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:21:28 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7[50726]: 2024-04-23T10:21:28.550+0000 7f5dbbd20700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T10:21:29.287 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-23T10:21:29.287 INFO:teuthology.orchestra.run.smithi196.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-23T10:21:29.288 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10+0 records in 2024-04-23T10:21:29.288 INFO:teuthology.orchestra.run.smithi196.stderr:10+0 records out 2024-04-23T10:21:29.288 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111297 s, 942 MB/s 2024-04-23T10:21:29.288 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping successful for: 2024-04-23T10:21:29.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:29 smithi183 ceph-mon[32317]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-04-23T10:21:29.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:29 smithi183 ceph-mon[32317]: osdmap e51: 8 total, 7 up, 8 in 2024-04-23T10:21:29.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:29 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T10:21:29.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:29 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T10:21:29.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:29 smithi183 ceph-mon[32317]: from='osd.7 ' entity='osd.7' 2024-04-23T10:21:29.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:29 smithi196 ceph-mon[32919]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-04-23T10:21:29.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:29 smithi196 ceph-mon[32919]: osdmap e51: 8 total, 7 up, 8 in 2024-04-23T10:21:29.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:29 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T10:21:29.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:29 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T10:21:29.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:29 smithi196 ceph-mon[32919]: from='osd.7 ' entity='osd.7' 2024-04-23T10:21:29.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:29 smithi097 ceph-mon[29197]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-04-23T10:21:29.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:29 smithi097 ceph-mon[29197]: osdmap e51: 8 total, 7 up, 8 in 2024-04-23T10:21:29.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:29 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T10:21:29.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:29 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T10:21:29.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:29 smithi097 ceph-mon[29197]: from='osd.7 ' entity='osd.7' 2024-04-23T10:21:29.954 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi196:vg_nvme/lv_2 2024-04-23T10:21:30.236 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T10:21:30.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:30 smithi196 ceph-mon[32919]: purged_snaps scrub starts 2024-04-23T10:21:30.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:30 smithi196 ceph-mon[32919]: purged_snaps scrub ok 2024-04-23T10:21:30.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:30 smithi196 ceph-mon[32919]: osd.7 [v2:172.21.15.183:6816/1865873033,v1:172.21.15.183:6817/1865873033] boot 2024-04-23T10:21:30.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:30 smithi196 ceph-mon[32919]: osdmap e52: 8 total, 8 up, 8 in 2024-04-23T10:21:30.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:30 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T10:21:30.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:30 smithi196 ceph-mon[32919]: pgmap v177: 1 pgs: 1 remapped; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T10:21:30.357 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:30 smithi183 ceph-mon[32317]: purged_snaps scrub starts 2024-04-23T10:21:30.358 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:30 smithi183 ceph-mon[32317]: purged_snaps scrub ok 2024-04-23T10:21:30.358 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:30 smithi183 ceph-mon[32317]: osd.7 [v2:172.21.15.183:6816/1865873033,v1:172.21.15.183:6817/1865873033] boot 2024-04-23T10:21:30.358 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:30 smithi183 ceph-mon[32317]: osdmap e52: 8 total, 8 up, 8 in 2024-04-23T10:21:30.358 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:30 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T10:21:30.358 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:30 smithi183 ceph-mon[32317]: pgmap v177: 1 pgs: 1 remapped; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T10:21:30.577 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:30 smithi097 ceph-mon[29197]: purged_snaps scrub starts 2024-04-23T10:21:30.577 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:30 smithi097 ceph-mon[29197]: purged_snaps scrub ok 2024-04-23T10:21:30.577 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:30 smithi097 ceph-mon[29197]: osd.7 [v2:172.21.15.183:6816/1865873033,v1:172.21.15.183:6817/1865873033] boot 2024-04-23T10:21:30.577 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:30 smithi097 ceph-mon[29197]: osdmap e52: 8 total, 8 up, 8 in 2024-04-23T10:21:30.577 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:30 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T10:21:30.577 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:30 smithi097 ceph-mon[29197]: pgmap v177: 1 pgs: 1 remapped; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-04-23T10:21:31.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:31 smithi183 ceph-mon[32317]: osdmap e53: 8 total, 8 up, 8 in 2024-04-23T10:21:31.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:31 smithi183 ceph-mon[32317]: Detected new or changed devices on smithi183 2024-04-23T10:21:31.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:31 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:31.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:31 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:31.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:31 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:21:31.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:31 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:21:31.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:31 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:21:31.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:31 smithi183 ceph-mon[32317]: Adjusting osd_memory_target on smithi183 to 5696M 2024-04-23T10:21:31.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:31 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:31.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:31 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:21:31.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:31 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:21:31.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:31 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:31.402 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:31 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:21:31.402 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:31 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:21:31.402 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:31 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:21:31.402 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:31 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:31.535 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:31 smithi196 ceph-mon[32919]: osdmap e53: 8 total, 8 up, 8 in 2024-04-23T10:21:31.535 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:31 smithi196 ceph-mon[32919]: Detected new or changed devices on smithi183 2024-04-23T10:21:31.535 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:31 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:31.535 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:31 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:31.535 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:31 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:21:31.535 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:31 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:21:31.535 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:31 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:21:31.536 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:31 smithi196 ceph-mon[32919]: Adjusting osd_memory_target on smithi183 to 5696M 2024-04-23T10:21:31.536 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:31 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:31.536 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:31 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:21:31.536 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:31 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:21:31.536 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:31 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:31.536 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:31 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:21:31.536 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:31 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:21:31.536 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:31 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:21:31.536 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:31 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:31.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:31 smithi097 ceph-mon[29197]: osdmap e53: 8 total, 8 up, 8 in 2024-04-23T10:21:31.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:31 smithi097 ceph-mon[29197]: Detected new or changed devices on smithi183 2024-04-23T10:21:31.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:31 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:31.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:31 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:31.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:31 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:21:31.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:31 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:21:31.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:31 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:21:31.612 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:31 smithi097 ceph-mon[29197]: Adjusting osd_memory_target on smithi183 to 5696M 2024-04-23T10:21:31.612 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:31 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:31.612 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:31 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:21:31.612 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:31 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:21:31.612 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:31 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:31.612 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:31 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:21:31.612 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:31 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:21:31.612 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:31 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:21:31.612 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:31 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:31.881 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:21:31.858+0000 7fcc84b9d700 1 -- 172.21.15.196:0/378568127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc80103ca0 msgr2=0x7fcc80104100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:21:31.881 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:21:31.858+0000 7fcc84b9d700 1 --2- 172.21.15.196:0/378568127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc80103ca0 0x7fcc80104100 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcc74009990 tx=0x7fcc74009ca0 comp rx=0 tx=0).stop 2024-04-23T10:21:31.881 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:21:31.858+0000 7fcc84b9d700 1 -- 172.21.15.196:0/378568127 shutdown_connections 2024-04-23T10:21:31.881 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:21:31.858+0000 7fcc84b9d700 1 --2- 172.21.15.196:0/378568127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc80104640 0x7fcc8010aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:21:31.882 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:21:31.858+0000 7fcc84b9d700 1 --2- 172.21.15.196:0/378568127 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc80102aa0 0x7fcc80102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:21:31.882 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:21:31.858+0000 7fcc84b9d700 1 --2- 172.21.15.196:0/378568127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc80103ca0 0x7fcc80104100 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:21:31.882 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:21:31.859+0000 7fcc84b9d700 1 -- 172.21.15.196:0/378568127 >> 172.21.15.196:0/378568127 conn(0x7fcc800fe040 msgr2=0x7fcc80100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:21:31.882 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:21:31.859+0000 7fcc84b9d700 1 -- 172.21.15.196:0/378568127 shutdown_connections 2024-04-23T10:21:31.882 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:21:31.859+0000 7fcc84b9d700 1 -- 172.21.15.196:0/378568127 wait complete. 2024-04-23T10:21:31.882 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:21:31.859+0000 7fcc84b9d700 1 Processor -- start 2024-04-23T10:21:31.882 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:21:31.859+0000 7fcc84b9d700 1 -- start start 2024-04-23T10:21:31.882 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:21:31.859+0000 7fcc84b9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc80102aa0 0x7fcc80198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:21:31.882 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:21:31.859+0000 7fcc84b9d700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc80103ca0 0x7fcc80199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:21:31.883 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:21:31.860+0000 7fcc84b9d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc80104640 0x7fcc801a02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:21:31.883 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:21:31.860+0000 7fcc84b9d700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcc8010e680 con 0x7fcc80102aa0 2024-04-23T10:21:31.883 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:21:31.860+0000 7fcc84b9d700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcc8010e500 con 0x7fcc80103ca0 2024-04-23T10:21:31.883 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:21:31.860+0000 7fcc84b9d700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcc8010e800 con 0x7fcc80104640 2024-04-23T10:21:31.883 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:21:31.860+0000 7fcc7ed9d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc80104640 0x7fcc801a02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:21:31.883 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:21:31.860+0000 7fcc7ed9d700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc80104640 0x7fcc801a02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:39154/0 (socket says 172.21.15.196:39154) 2024-04-23T10:21:31.883 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:21:31.860+0000 7fcc7ed9d700 1 -- 172.21.15.196:0/3809973920 learned_addr learned my addr 172.21.15.196:0/3809973920 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-23T10:21:31.883 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:21:31.860+0000 7fcc7e59c700 1 --2- 172.21.15.196:0/3809973920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc80102aa0 0x7fcc80198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:21:31.883 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:21:31.860+0000 7fcc7dd9b700 1 --2- 172.21.15.196:0/3809973920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc80103ca0 0x7fcc80199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:21:31.883 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:21:31.860+0000 7fcc7ed9d700 1 -- 172.21.15.196:0/3809973920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc80103ca0 msgr2=0x7fcc80199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:21:31.884 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:21:31.860+0000 7fcc7ed9d700 1 --2- 172.21.15.196:0/3809973920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc80103ca0 0x7fcc80199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:21:31.884 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:21:31.860+0000 7fcc7ed9d700 1 -- 172.21.15.196:0/3809973920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc80102aa0 msgr2=0x7fcc80198d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:21:31.884 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:21:31.860+0000 7fcc7ed9d700 1 --2- 172.21.15.196:0/3809973920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc80102aa0 0x7fcc80198d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:21:31.884 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:21:31.860+0000 7fcc7ed9d700 1 -- 172.21.15.196:0/3809973920 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc801a0990 con 0x7fcc80104640 2024-04-23T10:21:31.884 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:21:31.860+0000 7fcc7e59c700 1 --2- 172.21.15.196:0/3809973920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc80102aa0 0x7fcc80198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:21:31.884 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:21:31.860+0000 7fcc7ed9d700 1 --2- 172.21.15.196:0/3809973920 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc80104640 0x7fcc801a02c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fcc7800b970 tx=0x7fcc7800bd30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:21:31.884 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:21:31.860+0000 7fcc6f7fe700 1 -- 172.21.15.196:0/3809973920 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc78019070 con 0x7fcc80104640 2024-04-23T10:21:31.884 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:21:31.860+0000 7fcc6f7fe700 1 -- 172.21.15.196:0/3809973920 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc78014440 con 0x7fcc80104640 2024-04-23T10:21:31.884 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:21:31.860+0000 7fcc84b9d700 1 -- 172.21.15.196:0/3809973920 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc801a0c80 con 0x7fcc80104640 2024-04-23T10:21:31.885 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:21:31.860+0000 7fcc6f7fe700 1 -- 172.21.15.196:0/3809973920 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc7800ce60 con 0x7fcc80104640 2024-04-23T10:21:31.885 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:21:31.860+0000 7fcc84b9d700 1 -- 172.21.15.196:0/3809973920 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc801a1170 con 0x7fcc80104640 2024-04-23T10:21:31.885 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:21:31.861+0000 7fcc84b9d700 1 -- 172.21.15.196:0/3809973920 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc80069730 con 0x7fcc80104640 2024-04-23T10:21:31.885 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:21:31.863+0000 7fcc6f7fe700 1 -- 172.21.15.196:0/3809973920 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fcc7801c070 con 0x7fcc80104640 2024-04-23T10:21:31.885 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:21:31.864+0000 7fcc6f7fe700 1 --2- 172.21.15.196:0/3809973920 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fcc680a3c40 0x7fcc680a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:21:31.885 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:21:31.864+0000 7fcc6f7fe700 1 -- 172.21.15.196:0/3809973920 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5426+0+0 (secure 0 0 0) 0x7fcc780cf1d0 con 0x7fcc80104640 2024-04-23T10:21:31.885 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:21:31.864+0000 7fcc7e59c700 1 --2- 172.21.15.196:0/3809973920 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fcc680a3c40 0x7fcc680a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:21:31.885 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:21:31.864+0000 7fcc7e59c700 1 --2- 172.21.15.196:0/3809973920 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fcc680a3c40 0x7fcc680a6100 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fcc8019a220 tx=0x7fcc70008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:21:31.885 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:21:31.867+0000 7fcc6f7fe700 1 -- 172.21.15.196:0/3809973920 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcc7809c6f0 con 0x7fcc80104640 2024-04-23T10:21:32.072 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:21:32.072+0000 7fcc84b9d700 1 -- 172.21.15.196:0/3809973920 --> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fcc80062e80 con 0x7fcc680a3c40 2024-04-23T10:21:32.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:32 smithi196 ceph-mon[32919]: osdmap e54: 8 total, 8 up, 8 in 2024-04-23T10:21:32.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:32 smithi196 ceph-mon[32919]: pgmap v180: 1 pgs: 1 remapped; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-04-23T10:21:32.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:32 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T10:21:32.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:32 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T10:21:32.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:32 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:21:32.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:32 smithi097 ceph-mon[29197]: osdmap e54: 8 total, 8 up, 8 in 2024-04-23T10:21:32.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:32 smithi097 ceph-mon[29197]: pgmap v180: 1 pgs: 1 remapped; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-04-23T10:21:32.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:32 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T10:21:32.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:32 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T10:21:32.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:32 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:21:32.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:32 smithi183 ceph-mon[32317]: osdmap e54: 8 total, 8 up, 8 in 2024-04-23T10:21:32.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:32 smithi183 ceph-mon[32317]: pgmap v180: 1 pgs: 1 remapped; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-04-23T10:21:32.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:32 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T10:21:32.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:32 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T10:21:32.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:32 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:21:33.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:33 smithi196 ceph-mon[32919]: from='client.24292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:21:33.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:33 smithi196 ceph-mon[32919]: osdmap e55: 8 total, 8 up, 8 in 2024-04-23T10:21:33.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:33 smithi097 ceph-mon[29197]: from='client.24292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:21:33.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:33 smithi097 ceph-mon[29197]: osdmap e55: 8 total, 8 up, 8 in 2024-04-23T10:21:33.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:33 smithi183 ceph-mon[32317]: from='client.24292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:21:33.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:33 smithi183 ceph-mon[32317]: osdmap e55: 8 total, 8 up, 8 in 2024-04-23T10:21:34.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:34 smithi196 ceph-mon[32919]: osdmap e56: 8 total, 8 up, 8 in 2024-04-23T10:21:34.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:34 smithi196 ceph-mon[32919]: pgmap v183: 1 pgs: 1 remapped; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-04-23T10:21:34.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:34 smithi097 ceph-mon[29197]: osdmap e56: 8 total, 8 up, 8 in 2024-04-23T10:21:34.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:34 smithi097 ceph-mon[29197]: pgmap v183: 1 pgs: 1 remapped; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-04-23T10:21:34.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:34 smithi183 ceph-mon[32317]: osdmap e56: 8 total, 8 up, 8 in 2024-04-23T10:21:34.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:34 smithi183 ceph-mon[32317]: pgmap v183: 1 pgs: 1 remapped; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-04-23T10:21:35.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:35 smithi196 ceph-mon[32919]: from='client.? 172.21.15.196:0/3741281644' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5cd3100a-bd6a-4be5-b5e9-0d0afc325193"}]: dispatch 2024-04-23T10:21:35.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:35 smithi196 ceph-mon[32919]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5cd3100a-bd6a-4be5-b5e9-0d0afc325193"}]: dispatch 2024-04-23T10:21:35.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:35 smithi196 ceph-mon[32919]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5cd3100a-bd6a-4be5-b5e9-0d0afc325193"}]': finished 2024-04-23T10:21:35.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:35 smithi196 ceph-mon[32919]: osdmap e57: 9 total, 8 up, 9 in 2024-04-23T10:21:35.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:35 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T10:21:35.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:35 smithi097 ceph-mon[29197]: from='client.? 172.21.15.196:0/3741281644' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5cd3100a-bd6a-4be5-b5e9-0d0afc325193"}]: dispatch 2024-04-23T10:21:35.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:35 smithi097 ceph-mon[29197]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5cd3100a-bd6a-4be5-b5e9-0d0afc325193"}]: dispatch 2024-04-23T10:21:35.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:35 smithi097 ceph-mon[29197]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5cd3100a-bd6a-4be5-b5e9-0d0afc325193"}]': finished 2024-04-23T10:21:35.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:35 smithi097 ceph-mon[29197]: osdmap e57: 9 total, 8 up, 9 in 2024-04-23T10:21:35.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:35 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T10:21:35.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:35 smithi183 ceph-mon[32317]: from='client.? 172.21.15.196:0/3741281644' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5cd3100a-bd6a-4be5-b5e9-0d0afc325193"}]: dispatch 2024-04-23T10:21:35.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:35 smithi183 ceph-mon[32317]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5cd3100a-bd6a-4be5-b5e9-0d0afc325193"}]: dispatch 2024-04-23T10:21:35.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:35 smithi183 ceph-mon[32317]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5cd3100a-bd6a-4be5-b5e9-0d0afc325193"}]': finished 2024-04-23T10:21:35.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:35 smithi183 ceph-mon[32317]: osdmap e57: 9 total, 8 up, 9 in 2024-04-23T10:21:35.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:35 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T10:21:36.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:36 smithi196 ceph-mon[32919]: from='client.? 172.21.15.196:0/3402371288' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T10:21:36.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:36 smithi196 ceph-mon[32919]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-23T10:21:36.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:36 smithi097 ceph-mon[29197]: from='client.? 172.21.15.196:0/3402371288' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T10:21:36.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:36 smithi097 ceph-mon[29197]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-23T10:21:36.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:36 smithi183 ceph-mon[32317]: from='client.? 172.21.15.196:0/3402371288' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T10:21:36.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:36 smithi183 ceph-mon[32317]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-23T10:21:38.722 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:38 smithi196 ceph-mon[32919]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-23T10:21:38.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:38 smithi097 ceph-mon[29197]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-23T10:21:38.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:38 smithi183 ceph-mon[32317]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-23T10:21:40.627 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:40 smithi196 ceph-mon[32919]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-23T10:21:40.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:40 smithi097 ceph-mon[29197]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-23T10:21:40.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:40 smithi183 ceph-mon[32317]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-23T10:21:42.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:42 smithi196 ceph-mon[32919]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-23T10:21:42.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:42 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-23T10:21:42.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:42 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:21:42.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:42 smithi097 ceph-mon[29197]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-23T10:21:42.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:42 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-23T10:21:42.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:42 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:21:42.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:42 smithi183 ceph-mon[32317]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-23T10:21:42.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:42 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-23T10:21:42.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:42 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:21:43.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:43 smithi196 ceph-mon[32919]: Deploying daemon osd.8 on smithi196 2024-04-23T10:21:43.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:43 smithi097 ceph-mon[29197]: Deploying daemon osd.8 on smithi196 2024-04-23T10:21:43.898 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:43 smithi183 ceph-mon[32317]: Deploying daemon osd.8 on smithi196 2024-04-23T10:21:44.816 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:44 smithi196 ceph-mon[32919]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-23T10:21:44.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:44 smithi097 ceph-mon[29197]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-23T10:21:44.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:44 smithi183 ceph-mon[32317]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-23T10:21:46.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:46 smithi196 ceph-mon[32919]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-23T10:21:46.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:46 smithi097 ceph-mon[29197]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-23T10:21:46.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:46 smithi183 ceph-mon[32317]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-23T10:21:48.622 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:48 smithi196 ceph-mon[32919]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-23T10:21:48.622 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:48 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:48.622 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:48 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:48.622 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:48 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:21:48.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:48 smithi097 ceph-mon[29197]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-23T10:21:48.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:48 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:48.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:48 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:48.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:48 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:21:48.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:48 smithi183 ceph-mon[32317]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-23T10:21:48.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:48 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:48.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:48 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:48.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:48 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:21:50.854 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:50 smithi196 ceph-mon[32919]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-23T10:21:50.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:50 smithi097 ceph-mon[29197]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-23T10:21:50.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:50 smithi183 ceph-mon[32317]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-23T10:21:52.040 INFO:teuthology.orchestra.run.smithi196.stdout:Created osd(s) 8 on host 'smithi196' 2024-04-23T10:21:52.040 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:21:52.040+0000 7fcc6f7fe700 1 -- 172.21.15.196:0/3809973920 <== mgr.14152 v2:172.21.15.97:6800/3149043749 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fcc80062e80 con 0x7fcc680a3c40 2024-04-23T10:21:52.042 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:21:52.043+0000 7fcc84b9d700 1 -- 172.21.15.196:0/3809973920 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fcc680a3c40 msgr2=0x7fcc680a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:21:52.042 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:21:52.043+0000 7fcc84b9d700 1 --2- 172.21.15.196:0/3809973920 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fcc680a3c40 0x7fcc680a6100 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fcc8019a220 tx=0x7fcc70008040 comp rx=0 tx=0).stop 2024-04-23T10:21:52.042 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:21:52.043+0000 7fcc84b9d700 1 -- 172.21.15.196:0/3809973920 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc80104640 msgr2=0x7fcc801a02c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:21:52.042 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:21:52.043+0000 7fcc84b9d700 1 --2- 172.21.15.196:0/3809973920 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc80104640 0x7fcc801a02c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fcc7800b970 tx=0x7fcc7800bd30 comp rx=0 tx=0).stop 2024-04-23T10:21:52.042 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:21:52.043+0000 7fcc84b9d700 1 -- 172.21.15.196:0/3809973920 shutdown_connections 2024-04-23T10:21:52.042 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:21:52.043+0000 7fcc84b9d700 1 --2- 172.21.15.196:0/3809973920 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fcc680a3c40 0x7fcc680a6100 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:21:52.042 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:21:52.043+0000 7fcc84b9d700 1 --2- 172.21.15.196:0/3809973920 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc80104640 0x7fcc801a02c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:21:52.042 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:21:52.043+0000 7fcc84b9d700 1 --2- 172.21.15.196:0/3809973920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc80102aa0 0x7fcc80198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:21:52.042 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:21:52.043+0000 7fcc84b9d700 1 --2- 172.21.15.196:0/3809973920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc80103ca0 0x7fcc80199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:21:52.042 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:21:52.043+0000 7fcc84b9d700 1 -- 172.21.15.196:0/3809973920 >> 172.21.15.196:0/3809973920 conn(0x7fcc800fe040 msgr2=0x7fcc80106ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:21:52.042 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:21:52.043+0000 7fcc84b9d700 1 -- 172.21.15.196:0/3809973920 shutdown_connections 2024-04-23T10:21:52.043 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:21:52.043+0000 7fcc84b9d700 1 -- 172.21.15.196:0/3809973920 wait complete. 2024-04-23T10:21:52.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:52 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:52.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:52 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:52.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:52 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:21:52.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:52 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:21:52.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:52 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:52.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:52 smithi196 ceph-mon[32919]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-23T10:21:52.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:52 smithi196 ceph-mon[32919]: from='osd.8 [v2:172.21.15.196:6816/4063932139,v1:172.21.15.196:6817/4063932139]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-23T10:21:52.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:52 smithi196 ceph-mon[32919]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-23T10:21:52.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:52 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:52.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:52 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:52.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:52 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:21:52.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:52 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:52.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:52 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:52.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:52 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:21:52.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:52 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:21:52.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:52 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:52.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:52 smithi097 ceph-mon[29197]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-23T10:21:52.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:52 smithi097 ceph-mon[29197]: from='osd.8 [v2:172.21.15.196:6816/4063932139,v1:172.21.15.196:6817/4063932139]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-23T10:21:52.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:52 smithi097 ceph-mon[29197]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-23T10:21:52.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:52 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:52.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:52 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:52.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:52 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:21:52.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:52 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:52.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:52 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:52.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:52 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:21:52.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:52 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:21:52.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:52 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:52.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:52 smithi183 ceph-mon[32317]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-23T10:21:52.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:52 smithi183 ceph-mon[32317]: from='osd.8 [v2:172.21.15.196:6816/4063932139,v1:172.21.15.196:6817/4063932139]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-23T10:21:52.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:52 smithi183 ceph-mon[32317]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-23T10:21:52.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:52 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:52.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:52 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:52.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:52 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:21:52.768 DEBUG:teuthology.orchestra.run.smithi196:osd.8> sudo journalctl -f -n 0 -u ceph-9b22af88-0159-11ef-bc93-c7b262605968@osd.8.service 2024-04-23T10:21:52.770 INFO:tasks.cephadm:Deploying osd.9 on smithi097 with /dev/vg_nvme/lv_1... 2024-04-23T10:21:52.771 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-23T10:21:53.017 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T10:21:53.314 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:53 smithi196 ceph-mon[32919]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-23T10:21:53.315 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:53 smithi196 ceph-mon[32919]: osdmap e58: 9 total, 8 up, 9 in 2024-04-23T10:21:53.315 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:53 smithi196 ceph-mon[32919]: from='osd.8 [v2:172.21.15.196:6816/4063932139,v1:172.21.15.196:6817/4063932139]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-23T10:21:53.315 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:53 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T10:21:53.315 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:53 smithi196 ceph-mon[32919]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-23T10:21:53.315 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:21:53 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:21:53.059+0000 7f31a074c700 -1 osd.8 0 waiting for initial osdmap 2024-04-23T10:21:53.315 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:21:53 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:21:53.068+0000 7f319c53f700 -1 osd.8 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T10:21:53.337 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:53 smithi097 ceph-mon[29197]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-23T10:21:53.337 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:53 smithi097 ceph-mon[29197]: osdmap e58: 9 total, 8 up, 9 in 2024-04-23T10:21:53.338 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:53 smithi097 ceph-mon[29197]: from='osd.8 [v2:172.21.15.196:6816/4063932139,v1:172.21.15.196:6817/4063932139]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-23T10:21:53.338 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:53 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T10:21:53.338 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:53 smithi097 ceph-mon[29197]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-23T10:21:53.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:53 smithi183 ceph-mon[32317]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-23T10:21:53.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:53 smithi183 ceph-mon[32317]: osdmap e58: 9 total, 8 up, 9 in 2024-04-23T10:21:53.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:53 smithi183 ceph-mon[32317]: from='osd.8 [v2:172.21.15.196:6816/4063932139,v1:172.21.15.196:6817/4063932139]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-23T10:21:53.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:53 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T10:21:53.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:53 smithi183 ceph-mon[32317]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-23T10:21:54.283 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:54 smithi196 ceph-mon[32919]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-04-23T10:21:54.284 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:54 smithi196 ceph-mon[32919]: osdmap e59: 9 total, 8 up, 9 in 2024-04-23T10:21:54.284 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:54 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T10:21:54.284 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:54 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T10:21:54.284 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:54 smithi196 ceph-mon[32919]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-23T10:21:54.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:54 smithi097 ceph-mon[29197]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-04-23T10:21:54.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:54 smithi097 ceph-mon[29197]: osdmap e59: 9 total, 8 up, 9 in 2024-04-23T10:21:54.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:54 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T10:21:54.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:54 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T10:21:54.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:54 smithi097 ceph-mon[29197]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-23T10:21:54.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:54 smithi183 ceph-mon[32317]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-04-23T10:21:54.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:54 smithi183 ceph-mon[32317]: osdmap e59: 9 total, 8 up, 9 in 2024-04-23T10:21:54.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:54 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T10:21:54.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:54 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T10:21:54.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:54 smithi183 ceph-mon[32317]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-04-23T10:21:55.271 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-23T10:21:55.271 INFO:teuthology.orchestra.run.smithi097.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-23T10:21:55.271 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10+0 records in 2024-04-23T10:21:55.271 INFO:teuthology.orchestra.run.smithi097.stderr:10+0 records out 2024-04-23T10:21:55.271 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114641 s, 915 MB/s 2024-04-23T10:21:55.271 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping successful for: 2024-04-23T10:21:55.323 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:55 smithi097 ceph-mon[29197]: purged_snaps scrub starts 2024-04-23T10:21:55.324 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:55 smithi097 ceph-mon[29197]: purged_snaps scrub ok 2024-04-23T10:21:55.324 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:55 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T10:21:55.324 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:55 smithi097 ceph-mon[29197]: osd.8 [v2:172.21.15.196:6816/4063932139,v1:172.21.15.196:6817/4063932139] boot 2024-04-23T10:21:55.324 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:55 smithi097 ceph-mon[29197]: osdmap e60: 9 total, 9 up, 9 in 2024-04-23T10:21:55.324 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:55 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T10:21:55.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:55 smithi196 ceph-mon[32919]: purged_snaps scrub starts 2024-04-23T10:21:55.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:55 smithi196 ceph-mon[32919]: purged_snaps scrub ok 2024-04-23T10:21:55.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:55 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T10:21:55.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:55 smithi196 ceph-mon[32919]: osd.8 [v2:172.21.15.196:6816/4063932139,v1:172.21.15.196:6817/4063932139] boot 2024-04-23T10:21:55.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:55 smithi196 ceph-mon[32919]: osdmap e60: 9 total, 9 up, 9 in 2024-04-23T10:21:55.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:55 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T10:21:55.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:55 smithi183 ceph-mon[32317]: purged_snaps scrub starts 2024-04-23T10:21:55.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:55 smithi183 ceph-mon[32317]: purged_snaps scrub ok 2024-04-23T10:21:55.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:55 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T10:21:55.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:55 smithi183 ceph-mon[32317]: osd.8 [v2:172.21.15.196:6816/4063932139,v1:172.21.15.196:6817/4063932139] boot 2024-04-23T10:21:55.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:55 smithi183 ceph-mon[32317]: osdmap e60: 9 total, 9 up, 9 in 2024-04-23T10:21:55.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:55 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T10:21:55.972 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi097:vg_nvme/lv_1 2024-04-23T10:21:56.225 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T10:21:56.337 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:56 smithi097 ceph-mon[29197]: osdmap e61: 9 total, 9 up, 9 in 2024-04-23T10:21:56.337 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:56 smithi097 ceph-mon[29197]: pgmap v199: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-04-23T10:21:56.337 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:56 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:56.338 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:56 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:56.338 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:56 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:21:56.338 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:56 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:21:56.338 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:56 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:21:56.338 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:56 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:56.338 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:56 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:21:56.338 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:56 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:21:56.338 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:56 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:56.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:56 smithi196 ceph-mon[32919]: osdmap e61: 9 total, 9 up, 9 in 2024-04-23T10:21:56.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:56 smithi196 ceph-mon[32919]: pgmap v199: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-04-23T10:21:56.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:56 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:56.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:56 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:56.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:56 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:21:56.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:56 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:21:56.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:56 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:21:56.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:56 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:56.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:56 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:21:56.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:56 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:21:56.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:56 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:56.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:56 smithi183 ceph-mon[32317]: osdmap e61: 9 total, 9 up, 9 in 2024-04-23T10:21:56.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:56 smithi183 ceph-mon[32317]: pgmap v199: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-04-23T10:21:56.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:56 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:56.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:56 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:56.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:56 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:21:56.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:56 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:21:56.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:56 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:21:56.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:56 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:56.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:56 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:21:56.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:56 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:21:56.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:56 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:21:57.335 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:57 smithi097 ceph-mon[29197]: Detected new or changed devices on smithi196 2024-04-23T10:21:57.335 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:57 smithi097 ceph-mon[29197]: Adjusting osd_memory_target on smithi196 to 5696M 2024-04-23T10:21:57.335 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:57 smithi097 ceph-mon[29197]: osdmap e62: 9 total, 9 up, 9 in 2024-04-23T10:21:57.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:57 smithi196 ceph-mon[32919]: Detected new or changed devices on smithi196 2024-04-23T10:21:57.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:57 smithi196 ceph-mon[32919]: Adjusting osd_memory_target on smithi196 to 5696M 2024-04-23T10:21:57.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:57 smithi196 ceph-mon[32919]: osdmap e62: 9 total, 9 up, 9 in 2024-04-23T10:21:57.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:57 smithi183 ceph-mon[32317]: Detected new or changed devices on smithi196 2024-04-23T10:21:57.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:57 smithi183 ceph-mon[32317]: Adjusting osd_memory_target on smithi196 to 5696M 2024-04-23T10:21:57.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:57 smithi183 ceph-mon[32317]: osdmap e62: 9 total, 9 up, 9 in 2024-04-23T10:21:57.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:21:57.936+0000 7fe91b85e700 1 -- 172.21.15.97:0/1646427495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe914102aa0 msgr2=0x7fe914102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:21:57.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:21:57.936+0000 7fe91b85e700 1 --2- 172.21.15.97:0/1646427495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe914102aa0 0x7fe914102ec0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe9100099c0 tx=0x7fe910009cd0 comp rx=0 tx=0).stop 2024-04-23T10:21:57.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:21:57.936+0000 7fe91b85e700 1 -- 172.21.15.97:0/1646427495 shutdown_connections 2024-04-23T10:21:57.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:21:57.936+0000 7fe91b85e700 1 --2- 172.21.15.97:0/1646427495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe914104640 0x7fe91410aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:21:57.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:21:57.936+0000 7fe91b85e700 1 --2- 172.21.15.97:0/1646427495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe914102aa0 0x7fe914102ec0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:21:57.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:21:57.936+0000 7fe91b85e700 1 --2- 172.21.15.97:0/1646427495 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe914103ca0 0x7fe914104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:21:57.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:21:57.936+0000 7fe91b85e700 1 -- 172.21.15.97:0/1646427495 >> 172.21.15.97:0/1646427495 conn(0x7fe9140fe040 msgr2=0x7fe914100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:21:57.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:21:57.936+0000 7fe91b85e700 1 -- 172.21.15.97:0/1646427495 shutdown_connections 2024-04-23T10:21:57.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:21:57.936+0000 7fe91b85e700 1 -- 172.21.15.97:0/1646427495 wait complete. 2024-04-23T10:21:57.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:21:57.937+0000 7fe91b85e700 1 Processor -- start 2024-04-23T10:21:57.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:21:57.937+0000 7fe91b85e700 1 -- start start 2024-04-23T10:21:57.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:21:57.937+0000 7fe91b85e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe914102aa0 0x7fe914198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:21:57.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:21:57.937+0000 7fe91b85e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe914103ca0 0x7fe914199030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:21:57.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:21:57.937+0000 7fe91b85e700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe914104640 0x7fe9141a00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:21:57.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:21:57.937+0000 7fe91b85e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe91410e510 con 0x7fe914103ca0 2024-04-23T10:21:57.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:21:57.937+0000 7fe91b85e700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe91410e390 con 0x7fe914104640 2024-04-23T10:21:57.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:21:57.937+0000 7fe91b85e700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe91410e690 con 0x7fe914102aa0 2024-04-23T10:21:57.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:21:57.937+0000 7fe918df9700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe914103ca0 0x7fe914199030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:21:57.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:21:57.937+0000 7fe918df9700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe914103ca0 0x7fe914199030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55556/0 (socket says 172.21.15.97:55556) 2024-04-23T10:21:57.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:21:57.937+0000 7fe918df9700 1 -- 172.21.15.97:0/3357419952 learned_addr learned my addr 172.21.15.97:0/3357419952 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:21:57.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:21:57.937+0000 7fe9195fa700 1 --2- 172.21.15.97:0/3357419952 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe914102aa0 0x7fe914198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:21:57.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:21:57.937+0000 7fe919dfb700 1 --2- 172.21.15.97:0/3357419952 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe914104640 0x7fe9141a00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:21:57.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:21:57.937+0000 7fe918df9700 1 -- 172.21.15.97:0/3357419952 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe914102aa0 msgr2=0x7fe914198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:21:57.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:21:57.937+0000 7fe918df9700 1 --2- 172.21.15.97:0/3357419952 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe914102aa0 0x7fe914198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:21:57.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:21:57.937+0000 7fe918df9700 1 -- 172.21.15.97:0/3357419952 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe914104640 msgr2=0x7fe9141a00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:21:57.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:21:57.937+0000 7fe918df9700 1 --2- 172.21.15.97:0/3357419952 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe914104640 0x7fe9141a00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:21:57.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:21:57.937+0000 7fe918df9700 1 -- 172.21.15.97:0/3357419952 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9141a0750 con 0x7fe914103ca0 2024-04-23T10:21:57.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:21:57.938+0000 7fe918df9700 1 --2- 172.21.15.97:0/3357419952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe914103ca0 0x7fe914199030 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe90400c9e0 tx=0x7fe90400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:21:57.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:21:57.938+0000 7fe90a7fc700 1 -- 172.21.15.97:0/3357419952 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe904007ac0 con 0x7fe914103ca0 2024-04-23T10:21:57.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:21:57.938+0000 7fe91b85e700 1 -- 172.21.15.97:0/3357419952 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9141a0a40 con 0x7fe914103ca0 2024-04-23T10:21:57.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:21:57.938+0000 7fe90a7fc700 1 -- 172.21.15.97:0/3357419952 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe904007c60 con 0x7fe914103ca0 2024-04-23T10:21:57.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:21:57.938+0000 7fe90a7fc700 1 -- 172.21.15.97:0/3357419952 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe904018660 con 0x7fe914103ca0 2024-04-23T10:21:57.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:21:57.938+0000 7fe91b85e700 1 -- 172.21.15.97:0/3357419952 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9141a0fd0 con 0x7fe914103ca0 2024-04-23T10:21:57.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:21:57.939+0000 7fe91b85e700 1 -- 172.21.15.97:0/3357419952 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe914069730 con 0x7fe914103ca0 2024-04-23T10:21:57.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:21:57.940+0000 7fe90a7fc700 1 -- 172.21.15.97:0/3357419952 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe90400f450 con 0x7fe914103ca0 2024-04-23T10:21:57.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:21:57.940+0000 7fe90a7fc700 1 --2- 172.21.15.97:0/3357419952 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fe9000a3c40 0x7fe9000a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:21:57.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:21:57.940+0000 7fe9195fa700 1 --2- 172.21.15.97:0/3357419952 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fe9000a3c40 0x7fe9000a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:21:57.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:21:57.940+0000 7fe90a7fc700 1 -- 172.21.15.97:0/3357419952 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5829+0+0 (secure 0 0 0) 0x7fe9040cfc00 con 0x7fe914103ca0 2024-04-23T10:21:57.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:21:57.941+0000 7fe9195fa700 1 --2- 172.21.15.97:0/3357419952 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fe9000a3c40 0x7fe9000a6100 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe9100038c0 tx=0x7fe91000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:21:57.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:21:57.944+0000 7fe90a7fc700 1 -- 172.21.15.97:0/3357419952 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe904014030 con 0x7fe914103ca0 2024-04-23T10:21:58.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:21:58.157+0000 7fe91b85e700 1 -- 172.21.15.97:0/3357419952 --> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fe914062e80 con 0x7fe9000a3c40 2024-04-23T10:21:58.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:58 smithi196 ceph-mon[32919]: osdmap e63: 9 total, 9 up, 9 in 2024-04-23T10:21:58.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:58 smithi196 ceph-mon[32919]: pgmap v202: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-04-23T10:21:58.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:58 smithi097 ceph-mon[29197]: osdmap e63: 9 total, 9 up, 9 in 2024-04-23T10:21:58.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:58 smithi097 ceph-mon[29197]: pgmap v202: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-04-23T10:21:58.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:58 smithi183 ceph-mon[32317]: osdmap e63: 9 total, 9 up, 9 in 2024-04-23T10:21:58.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:58 smithi183 ceph-mon[32317]: pgmap v202: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-04-23T10:21:59.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:59 smithi196 ceph-mon[32919]: osdmap e64: 9 total, 9 up, 9 in 2024-04-23T10:21:59.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:59 smithi196 ceph-mon[32919]: from='client.14421 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:21:59.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:59 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T10:21:59.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:59 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T10:21:59.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:21:59 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:21:59.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:59 smithi097 ceph-mon[29197]: osdmap e64: 9 total, 9 up, 9 in 2024-04-23T10:21:59.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:59 smithi097 ceph-mon[29197]: from='client.14421 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:21:59.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:59 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T10:21:59.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:59 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T10:21:59.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:21:59 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:21:59.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:59 smithi183 ceph-mon[32317]: osdmap e64: 9 total, 9 up, 9 in 2024-04-23T10:21:59.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:59 smithi183 ceph-mon[32317]: from='client.14421 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:21:59.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:59 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T10:21:59.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:59 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T10:21:59.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:21:59 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:22:00.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:00 smithi097 ceph-mon[29197]: pgmap v204: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-04-23T10:22:00.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:00 smithi183 ceph-mon[32317]: pgmap v204: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-04-23T10:22:00.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:00 smithi196 ceph-mon[32919]: pgmap v204: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-04-23T10:22:01.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:01 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/3733090119' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6a5f4a1-4ecc-4f73-a2df-07551d9298ae"}]: dispatch 2024-04-23T10:22:01.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:01 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/3733090119' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b6a5f4a1-4ecc-4f73-a2df-07551d9298ae"}]': finished 2024-04-23T10:22:01.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:01 smithi097 ceph-mon[29197]: osdmap e65: 10 total, 9 up, 10 in 2024-04-23T10:22:01.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:01 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T10:22:01.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:01 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/3733090119' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6a5f4a1-4ecc-4f73-a2df-07551d9298ae"}]: dispatch 2024-04-23T10:22:01.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:01 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/3733090119' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b6a5f4a1-4ecc-4f73-a2df-07551d9298ae"}]': finished 2024-04-23T10:22:01.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:01 smithi183 ceph-mon[32317]: osdmap e65: 10 total, 9 up, 10 in 2024-04-23T10:22:01.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:01 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T10:22:01.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:01 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/3733090119' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6a5f4a1-4ecc-4f73-a2df-07551d9298ae"}]: dispatch 2024-04-23T10:22:01.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:01 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/3733090119' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b6a5f4a1-4ecc-4f73-a2df-07551d9298ae"}]': finished 2024-04-23T10:22:01.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:01 smithi196 ceph-mon[32919]: osdmap e65: 10 total, 9 up, 10 in 2024-04-23T10:22:01.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:01 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T10:22:02.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:02 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/3600459045' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T10:22:02.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:02 smithi097 ceph-mon[29197]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-04-23T10:22:02.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:02 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/3600459045' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T10:22:02.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:02 smithi183 ceph-mon[32317]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-04-23T10:22:02.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:02 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/3600459045' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T10:22:02.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:02 smithi196 ceph-mon[32919]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-04-23T10:22:04.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:04 smithi196 ceph-mon[32919]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-04-23T10:22:04.849 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:04 smithi097 ceph-mon[29197]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-04-23T10:22:04.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:04 smithi183 ceph-mon[32317]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-04-23T10:22:06.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:06 smithi196 ceph-mon[32919]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-23T10:22:06.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:06 smithi097 ceph-mon[29197]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-23T10:22:06.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:06 smithi183 ceph-mon[32317]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-23T10:22:08.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:08 smithi196 ceph-mon[32919]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-04-23T10:22:08.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:08 smithi097 ceph-mon[29197]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-04-23T10:22:08.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:08 smithi183 ceph-mon[32317]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-04-23T10:22:09.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:09 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-23T10:22:09.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:09 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:22:09.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:09 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-23T10:22:09.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:09 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:22:09.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:09 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-23T10:22:09.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:09 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:22:10.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:10 smithi196 ceph-mon[32919]: Deploying daemon osd.9 on smithi097 2024-04-23T10:22:10.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:10 smithi196 ceph-mon[32919]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-23T10:22:10.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:10 smithi097 ceph-mon[29197]: Deploying daemon osd.9 on smithi097 2024-04-23T10:22:10.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:10 smithi097 ceph-mon[29197]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-23T10:22:10.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:10 smithi183 ceph-mon[32317]: Deploying daemon osd.9 on smithi097 2024-04-23T10:22:10.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:10 smithi183 ceph-mon[32317]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-23T10:22:12.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:12 smithi097 ceph-mon[29197]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-04-23T10:22:12.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:12 smithi183 ceph-mon[32317]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-04-23T10:22:12.936 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:12 smithi196 ceph-mon[32919]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-04-23T10:22:14.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:14 smithi097 ceph-mon[29197]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-04-23T10:22:14.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:14 smithi183 ceph-mon[32317]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-04-23T10:22:15.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:14 smithi196 ceph-mon[32919]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-04-23T10:22:16.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:15 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:22:16.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:15 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:22:16.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:15 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:22:16.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:15 smithi196 ceph-mon[32919]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-04-23T10:22:16.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:15 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:22:16.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:15 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:22:16.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:15 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:22:16.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:15 smithi097 ceph-mon[29197]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-04-23T10:22:16.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:15 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:22:16.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:15 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:22:16.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:15 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:22:16.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:15 smithi183 ceph-mon[32317]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-04-23T10:22:18.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:22:18.415+0000 7fe90a7fc700 1 -- 172.21.15.97:0/3357419952 <== mgr.14152 v2:172.21.15.97:6800/3149043749 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe914062e80 con 0x7fe9000a3c40 2024-04-23T10:22:18.415 INFO:teuthology.orchestra.run.smithi097.stdout:Created osd(s) 9 on host 'smithi097' 2024-04-23T10:22:18.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:22:18.417+0000 7fe91b85e700 1 -- 172.21.15.97:0/3357419952 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fe9000a3c40 msgr2=0x7fe9000a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:22:18.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:22:18.417+0000 7fe91b85e700 1 --2- 172.21.15.97:0/3357419952 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fe9000a3c40 0x7fe9000a6100 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe9100038c0 tx=0x7fe91000b540 comp rx=0 tx=0).stop 2024-04-23T10:22:18.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:22:18.417+0000 7fe91b85e700 1 -- 172.21.15.97:0/3357419952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe914103ca0 msgr2=0x7fe914199030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:22:18.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:22:18.417+0000 7fe91b85e700 1 --2- 172.21.15.97:0/3357419952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe914103ca0 0x7fe914199030 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe90400c9e0 tx=0x7fe90400ccf0 comp rx=0 tx=0).stop 2024-04-23T10:22:18.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:22:18.417+0000 7fe91b85e700 1 -- 172.21.15.97:0/3357419952 shutdown_connections 2024-04-23T10:22:18.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:22:18.417+0000 7fe91b85e700 1 --2- 172.21.15.97:0/3357419952 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fe9000a3c40 0x7fe9000a6100 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:22:18.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:22:18.417+0000 7fe91b85e700 1 --2- 172.21.15.97:0/3357419952 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe914104640 0x7fe9141a00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:22:18.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:22:18.417+0000 7fe91b85e700 1 --2- 172.21.15.97:0/3357419952 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe914102aa0 0x7fe914198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:22:18.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:22:18.417+0000 7fe91b85e700 1 --2- 172.21.15.97:0/3357419952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe914103ca0 0x7fe914199030 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:22:18.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:22:18.417+0000 7fe91b85e700 1 -- 172.21.15.97:0/3357419952 >> 172.21.15.97:0/3357419952 conn(0x7fe9140fe040 msgr2=0x7fe914106fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:22:18.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:22:18.417+0000 7fe91b85e700 1 -- 172.21.15.97:0/3357419952 shutdown_connections 2024-04-23T10:22:18.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:22:18.417+0000 7fe91b85e700 1 -- 172.21.15.97:0/3357419952 wait complete. 2024-04-23T10:22:18.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:18 smithi196 ceph-mon[32919]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-04-23T10:22:18.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:18 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:22:18.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:18 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:22:18.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:18 smithi097 ceph-mon[29197]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-04-23T10:22:18.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:18 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:22:18.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:18 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:22:18.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:18 smithi183 ceph-mon[32317]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-04-23T10:22:18.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:18 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:22:18.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:18 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:22:19.077 DEBUG:teuthology.orchestra.run.smithi097:osd.9> sudo journalctl -f -n 0 -u ceph-9b22af88-0159-11ef-bc93-c7b262605968@osd.9.service 2024-04-23T10:22:19.079 INFO:tasks.cephadm:Deploying osd.10 on smithi183 with /dev/vg_nvme/lv_1... 2024-04-23T10:22:19.080 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-23T10:22:19.329 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.b/config 2024-04-23T10:22:19.797 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:19 smithi183 ceph-mon[32317]: from='osd.9 [v2:172.21.15.97:6826/1887432474,v1:172.21.15.97:6827/1887432474]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-23T10:22:19.797 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:19 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:22:19.797 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:19 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:22:19.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:19 smithi196 ceph-mon[32919]: from='osd.9 [v2:172.21.15.97:6826/1887432474,v1:172.21.15.97:6827/1887432474]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-23T10:22:19.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:19 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:22:19.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:19 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:22:19.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:19 smithi097 ceph-mon[29197]: from='osd.9 [v2:172.21.15.97:6826/1887432474,v1:172.21.15.97:6827/1887432474]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-23T10:22:19.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:19 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:22:19.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:19 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:22:20.752 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:20 smithi183 ceph-mon[32317]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-04-23T10:22:20.752 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:20 smithi183 ceph-mon[32317]: from='osd.9 [v2:172.21.15.97:6826/1887432474,v1:172.21.15.97:6827/1887432474]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-23T10:22:20.752 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:20 smithi183 ceph-mon[32317]: osdmap e66: 10 total, 9 up, 10 in 2024-04-23T10:22:20.752 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:20 smithi183 ceph-mon[32317]: from='osd.9 [v2:172.21.15.97:6826/1887432474,v1:172.21.15.97:6827/1887432474]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-23T10:22:20.752 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:20 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T10:22:20.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:20 smithi097 ceph-mon[29197]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-04-23T10:22:20.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:20 smithi097 ceph-mon[29197]: from='osd.9 [v2:172.21.15.97:6826/1887432474,v1:172.21.15.97:6827/1887432474]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-23T10:22:20.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:20 smithi097 ceph-mon[29197]: osdmap e66: 10 total, 9 up, 10 in 2024-04-23T10:22:20.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:20 smithi097 ceph-mon[29197]: from='osd.9 [v2:172.21.15.97:6826/1887432474,v1:172.21.15.97:6827/1887432474]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-23T10:22:20.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:20 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T10:22:20.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:22:20 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:22:20.599+0000 7f96b3e10700 -1 osd.9 0 waiting for initial osdmap 2024-04-23T10:22:20.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:22:20 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:22:20.612+0000 7f96afc03700 -1 osd.9 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T10:22:21.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:20 smithi196 ceph-mon[32919]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-04-23T10:22:21.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:20 smithi196 ceph-mon[32919]: from='osd.9 [v2:172.21.15.97:6826/1887432474,v1:172.21.15.97:6827/1887432474]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-23T10:22:21.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:20 smithi196 ceph-mon[32919]: osdmap e66: 10 total, 9 up, 10 in 2024-04-23T10:22:21.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:20 smithi196 ceph-mon[32919]: from='osd.9 [v2:172.21.15.97:6826/1887432474,v1:172.21.15.97:6827/1887432474]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-23T10:22:21.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:20 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T10:22:21.461 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-23T10:22:21.462 INFO:teuthology.orchestra.run.smithi183.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-23T10:22:21.462 INFO:teuthology.orchestra.run.smithi183.stderr: stderr: 10+0 records in 2024-04-23T10:22:21.462 INFO:teuthology.orchestra.run.smithi183.stderr:10+0 records out 2024-04-23T10:22:21.462 INFO:teuthology.orchestra.run.smithi183.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0126623 s, 828 MB/s 2024-04-23T10:22:21.462 INFO:teuthology.orchestra.run.smithi183.stderr:--> Zapping successful for: 2024-04-23T10:22:21.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:21 smithi097 ceph-mon[29197]: from='osd.9 [v2:172.21.15.97:6826/1887432474,v1:172.21.15.97:6827/1887432474]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-23T10:22:21.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:21 smithi097 ceph-mon[29197]: osdmap e67: 10 total, 9 up, 10 in 2024-04-23T10:22:21.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:21 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T10:22:21.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:21 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T10:22:21.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:21 smithi183 ceph-mon[32317]: from='osd.9 [v2:172.21.15.97:6826/1887432474,v1:172.21.15.97:6827/1887432474]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-23T10:22:21.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:21 smithi183 ceph-mon[32317]: osdmap e67: 10 total, 9 up, 10 in 2024-04-23T10:22:21.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:21 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T10:22:21.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:21 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T10:22:22.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:21 smithi196 ceph-mon[32919]: from='osd.9 [v2:172.21.15.97:6826/1887432474,v1:172.21.15.97:6827/1887432474]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-23T10:22:22.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:21 smithi196 ceph-mon[32919]: osdmap e67: 10 total, 9 up, 10 in 2024-04-23T10:22:22.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:21 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T10:22:22.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:21 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T10:22:22.212 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi183:vg_nvme/lv_1 2024-04-23T10:22:22.451 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.b/config 2024-04-23T10:22:22.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:22 smithi196 ceph-mon[32919]: purged_snaps scrub starts 2024-04-23T10:22:22.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:22 smithi196 ceph-mon[32919]: purged_snaps scrub ok 2024-04-23T10:22:22.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:22 smithi196 ceph-mon[32919]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-04-23T10:22:22.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:22 smithi196 ceph-mon[32919]: osd.9 [v2:172.21.15.97:6826/1887432474,v1:172.21.15.97:6827/1887432474] boot 2024-04-23T10:22:22.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:22 smithi196 ceph-mon[32919]: osdmap e68: 10 total, 10 up, 10 in 2024-04-23T10:22:22.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:22 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T10:22:22.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:22 smithi183 ceph-mon[32317]: purged_snaps scrub starts 2024-04-23T10:22:22.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:22 smithi183 ceph-mon[32317]: purged_snaps scrub ok 2024-04-23T10:22:22.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:22 smithi183 ceph-mon[32317]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-04-23T10:22:22.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:22 smithi183 ceph-mon[32317]: osd.9 [v2:172.21.15.97:6826/1887432474,v1:172.21.15.97:6827/1887432474] boot 2024-04-23T10:22:22.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:22 smithi183 ceph-mon[32317]: osdmap e68: 10 total, 10 up, 10 in 2024-04-23T10:22:22.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:22 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T10:22:23.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:22 smithi097 ceph-mon[29197]: purged_snaps scrub starts 2024-04-23T10:22:23.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:22 smithi097 ceph-mon[29197]: purged_snaps scrub ok 2024-04-23T10:22:23.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:22 smithi097 ceph-mon[29197]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-04-23T10:22:23.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:22 smithi097 ceph-mon[29197]: osd.9 [v2:172.21.15.97:6826/1887432474,v1:172.21.15.97:6827/1887432474] boot 2024-04-23T10:22:23.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:22 smithi097 ceph-mon[29197]: osdmap e68: 10 total, 10 up, 10 in 2024-04-23T10:22:23.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:22 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T10:22:23.810 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:23 smithi183 ceph-mon[32317]: osdmap e69: 10 total, 10 up, 10 in 2024-04-23T10:22:23.811 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:23 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:22:23.811 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:23 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:22:23.811 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:23 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:22:23.811 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:23 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:22:23.811 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:23 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:22:23.811 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:23 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:22:23.811 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:23 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:22:23.811 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:23 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:22:23.811 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:23 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:22:23.811 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:23 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:22:23.811 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:23 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:22:23.811 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:23 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:22:23.811 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:23 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:22:23.811 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:23 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:22:24.031 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:22:24.031+0000 7fdeae6a2700 1 -- 172.21.15.183:0/3320062645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdea8102be0 msgr2=0x7fdea8103040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:22:24.031 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:22:24.031+0000 7fdeae6a2700 1 --2- 172.21.15.183:0/3320062645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdea8102be0 0x7fdea8103040 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fde9c0099c0 tx=0x7fde9c009cd0 comp rx=0 tx=0).stop 2024-04-23T10:22:24.031 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:22:24.032+0000 7fdeae6a2700 1 -- 172.21.15.183:0/3320062645 shutdown_connections 2024-04-23T10:22:24.031 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:22:24.032+0000 7fdeae6a2700 1 --2- 172.21.15.183:0/3320062645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdea8103580 0x7fdea8109e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:22:24.031 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:22:24.032+0000 7fdeae6a2700 1 --2- 172.21.15.183:0/3320062645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdea81019e0 0x7fdea8101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:22:24.032 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:22:24.032+0000 7fdeae6a2700 1 --2- 172.21.15.183:0/3320062645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdea8102be0 0x7fdea8103040 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:22:24.032 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:22:24.032+0000 7fdeae6a2700 1 -- 172.21.15.183:0/3320062645 >> 172.21.15.183:0/3320062645 conn(0x7fdea80fcfa0 msgr2=0x7fdea80ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:22:24.032 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:22:24.032+0000 7fdeae6a2700 1 -- 172.21.15.183:0/3320062645 shutdown_connections 2024-04-23T10:22:24.032 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:22:24.032+0000 7fdeae6a2700 1 -- 172.21.15.183:0/3320062645 wait complete. 2024-04-23T10:22:24.032 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:22:24.032+0000 7fdeae6a2700 1 Processor -- start 2024-04-23T10:22:24.032 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:22:24.032+0000 7fdeae6a2700 1 -- start start 2024-04-23T10:22:24.034 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:22:24.033+0000 7fdeae6a2700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdea81019e0 0x7fdea8198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:22:24.034 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:22:24.033+0000 7fdeae6a2700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdea8102be0 0x7fdea81991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:22:24.034 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:22:24.033+0000 7fdeae6a2700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdea8103580 0x7fdea81a0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:22:24.034 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:22:24.033+0000 7fdeae6a2700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdea810d570 con 0x7fdea81019e0 2024-04-23T10:22:24.034 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:22:24.033+0000 7fdeae6a2700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdea810d3f0 con 0x7fdea8102be0 2024-04-23T10:22:24.034 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:22:24.033+0000 7fdeae6a2700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdea810d6f0 con 0x7fdea8103580 2024-04-23T10:22:24.034 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:22:24.033+0000 7fdea77fe700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdea8102be0 0x7fdea81991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:22:24.035 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:22:24.033+0000 7fdea77fe700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdea8102be0 0x7fdea81991d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:53826/0 (socket says 172.21.15.183:53826) 2024-04-23T10:22:24.035 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:22:24.033+0000 7fdea77fe700 1 -- 172.21.15.183:0/2205177951 learned_addr learned my addr 172.21.15.183:0/2205177951 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-04-23T10:22:24.035 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:22:24.033+0000 7fdea7fff700 1 --2- 172.21.15.183:0/2205177951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdea81019e0 0x7fdea8198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:22:24.035 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:22:24.033+0000 7fdeacc3f700 1 --2- 172.21.15.183:0/2205177951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdea8103580 0x7fdea81a0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:22:24.035 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:22:24.033+0000 7fdea77fe700 1 -- 172.21.15.183:0/2205177951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdea8103580 msgr2=0x7fdea81a0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:22:24.035 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:22:24.033+0000 7fdea77fe700 1 --2- 172.21.15.183:0/2205177951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdea8103580 0x7fdea81a0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:22:24.035 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:22:24.033+0000 7fdea77fe700 1 -- 172.21.15.183:0/2205177951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdea81019e0 msgr2=0x7fdea8198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:22:24.035 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:22:24.033+0000 7fdea77fe700 1 --2- 172.21.15.183:0/2205177951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdea81019e0 0x7fdea8198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:22:24.035 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:22:24.033+0000 7fdea77fe700 1 -- 172.21.15.183:0/2205177951 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdea81a0910 con 0x7fdea8102be0 2024-04-23T10:22:24.035 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:22:24.033+0000 7fdeacc3f700 1 --2- 172.21.15.183:0/2205177951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdea8103580 0x7fdea81a0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:22:24.035 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:22:24.033+0000 7fdea77fe700 1 --2- 172.21.15.183:0/2205177951 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdea8102be0 0x7fdea81991d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fde9c00b580 tx=0x7fde9c005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:22:24.035 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:22:24.033+0000 7fdea57fa700 1 -- 172.21.15.183:0/2205177951 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde9c027070 con 0x7fdea8102be0 2024-04-23T10:22:24.035 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:22:24.033+0000 7fdeae6a2700 1 -- 172.21.15.183:0/2205177951 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdea81a0ba0 con 0x7fdea8102be0 2024-04-23T10:22:24.035 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:22:24.033+0000 7fdea57fa700 1 -- 172.21.15.183:0/2205177951 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde9c022470 con 0x7fdea8102be0 2024-04-23T10:22:24.036 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:22:24.033+0000 7fdea57fa700 1 -- 172.21.15.183:0/2205177951 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde9c00f6e0 con 0x7fdea8102be0 2024-04-23T10:22:24.036 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:22:24.033+0000 7fdeae6a2700 1 -- 172.21.15.183:0/2205177951 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdea81a1080 con 0x7fdea8102be0 2024-04-23T10:22:24.036 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:22:24.034+0000 7fdeae6a2700 1 -- 172.21.15.183:0/2205177951 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdea8069760 con 0x7fdea8102be0 2024-04-23T10:22:24.036 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:22:24.035+0000 7fdea57fa700 1 -- 172.21.15.183:0/2205177951 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fde9c02a070 con 0x7fdea8102be0 2024-04-23T10:22:24.037 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:22:24.036+0000 7fdea57fa700 1 --2- 172.21.15.183:0/2205177951 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fde900a3c40 0x7fde900a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:22:24.037 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:22:24.036+0000 7fdea57fa700 1 -- 172.21.15.183:0/2205177951 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6261+0+0 (secure 0 0 0) 0x7fde9c0d0db0 con 0x7fdea8102be0 2024-04-23T10:22:24.037 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:22:24.036+0000 7fdea7fff700 1 --2- 172.21.15.183:0/2205177951 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fde900a3c40 0x7fde900a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:22:24.037 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:22:24.037+0000 7fdea7fff700 1 --2- 172.21.15.183:0/2205177951 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fde900a3c40 0x7fde900a6100 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdea8199b70 tx=0x7fde98009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:22:24.039 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:22:24.040+0000 7fdea57fa700 1 -- 172.21.15.183:0/2205177951 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdea8069760 con 0x7fdea8102be0 2024-04-23T10:22:24.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:23 smithi196 ceph-mon[32919]: osdmap e69: 10 total, 10 up, 10 in 2024-04-23T10:22:24.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:23 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:22:24.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:23 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:22:24.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:23 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:22:24.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:23 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:22:24.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:23 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:22:24.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:23 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:22:24.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:23 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:22:24.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:23 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:22:24.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:23 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:22:24.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:23 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:22:24.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:23 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:22:24.093 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:23 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:22:24.093 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:23 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:22:24.093 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:23 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:22:24.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:23 smithi097 ceph-mon[29197]: osdmap e69: 10 total, 10 up, 10 in 2024-04-23T10:22:24.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:23 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:22:24.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:23 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:22:24.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:23 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:22:24.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:23 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:22:24.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:23 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:22:24.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:23 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:22:24.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:23 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:22:24.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:23 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:22:24.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:23 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:22:24.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:23 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:22:24.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:23 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:22:24.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:23 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:22:24.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:23 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:22:24.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:23 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:22:24.248 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:22:24.249+0000 7fdeae6a2700 1 -- 172.21.15.183:0/2205177951 --> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fdea80ff130 con 0x7fde900a3c40 2024-04-23T10:22:24.878 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:24 smithi183 ceph-mon[32317]: Detected new or changed devices on smithi097 2024-04-23T10:22:24.878 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:24 smithi183 ceph-mon[32317]: Adjusting osd_memory_target on smithi097 to 4272M 2024-04-23T10:22:24.879 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:24 smithi183 ceph-mon[32317]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-23T10:22:24.879 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:24 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T10:22:24.879 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:24 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T10:22:24.879 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:24 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:22:25.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:24 smithi196 ceph-mon[32919]: Detected new or changed devices on smithi097 2024-04-23T10:22:25.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:24 smithi196 ceph-mon[32919]: Adjusting osd_memory_target on smithi097 to 4272M 2024-04-23T10:22:25.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:24 smithi196 ceph-mon[32919]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-23T10:22:25.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:24 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T10:22:25.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:24 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T10:22:25.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:24 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:22:25.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:24 smithi097 ceph-mon[29197]: Detected new or changed devices on smithi097 2024-04-23T10:22:25.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:24 smithi097 ceph-mon[29197]: Adjusting osd_memory_target on smithi097 to 4272M 2024-04-23T10:22:25.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:24 smithi097 ceph-mon[29197]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-23T10:22:25.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:24 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T10:22:25.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:24 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T10:22:25.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:24 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:22:25.881 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:25 smithi183 ceph-mon[32317]: from='client.24314 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:22:26.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:25 smithi196 ceph-mon[32919]: from='client.24314 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:22:26.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:25 smithi097 ceph-mon[29197]: from='client.24314 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:22:26.621 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:26 smithi183 ceph-mon[32317]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-23T10:22:27.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:26 smithi196 ceph-mon[32919]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-23T10:22:27.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:26 smithi097 ceph-mon[29197]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-23T10:22:27.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:27 smithi183 ceph-mon[32317]: from='client.? 172.21.15.183:0/1774762709' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "86f59816-1ea6-4ae9-8796-cf5fc4831064"}]: dispatch 2024-04-23T10:22:27.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:27 smithi183 ceph-mon[32317]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "86f59816-1ea6-4ae9-8796-cf5fc4831064"}]: dispatch 2024-04-23T10:22:27.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:27 smithi183 ceph-mon[32317]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "86f59816-1ea6-4ae9-8796-cf5fc4831064"}]': finished 2024-04-23T10:22:27.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:27 smithi183 ceph-mon[32317]: osdmap e70: 11 total, 10 up, 11 in 2024-04-23T10:22:27.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:27 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T10:22:28.012 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:27 smithi097 ceph-mon[29197]: from='client.? 172.21.15.183:0/1774762709' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "86f59816-1ea6-4ae9-8796-cf5fc4831064"}]: dispatch 2024-04-23T10:22:28.012 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:27 smithi097 ceph-mon[29197]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "86f59816-1ea6-4ae9-8796-cf5fc4831064"}]: dispatch 2024-04-23T10:22:28.012 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:27 smithi097 ceph-mon[29197]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "86f59816-1ea6-4ae9-8796-cf5fc4831064"}]': finished 2024-04-23T10:22:28.012 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:27 smithi097 ceph-mon[29197]: osdmap e70: 11 total, 10 up, 11 in 2024-04-23T10:22:28.012 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:27 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T10:22:28.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:27 smithi196 ceph-mon[32919]: from='client.? 172.21.15.183:0/1774762709' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "86f59816-1ea6-4ae9-8796-cf5fc4831064"}]: dispatch 2024-04-23T10:22:28.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:27 smithi196 ceph-mon[32919]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "86f59816-1ea6-4ae9-8796-cf5fc4831064"}]: dispatch 2024-04-23T10:22:28.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:27 smithi196 ceph-mon[32919]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "86f59816-1ea6-4ae9-8796-cf5fc4831064"}]': finished 2024-04-23T10:22:28.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:27 smithi196 ceph-mon[32919]: osdmap e70: 11 total, 10 up, 11 in 2024-04-23T10:22:28.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:27 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T10:22:28.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:28 smithi183 ceph-mon[32317]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-23T10:22:28.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:28 smithi183 ceph-mon[32317]: from='client.? 172.21.15.183:0/301417547' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T10:22:29.032 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:28 smithi097 ceph-mon[29197]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-23T10:22:29.032 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:28 smithi097 ceph-mon[29197]: from='client.? 172.21.15.183:0/301417547' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T10:22:29.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:28 smithi196 ceph-mon[32919]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-23T10:22:29.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:28 smithi196 ceph-mon[32919]: from='client.? 172.21.15.183:0/301417547' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T10:22:30.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:30 smithi183 ceph-mon[32317]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-23T10:22:31.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:30 smithi196 ceph-mon[32919]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-23T10:22:31.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:30 smithi097 ceph-mon[29197]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-23T10:22:32.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:32 smithi183 ceph-mon[32317]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-23T10:22:33.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:32 smithi196 ceph-mon[32919]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-23T10:22:33.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:32 smithi097 ceph-mon[29197]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-23T10:22:34.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:34 smithi183 ceph-mon[32317]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-23T10:22:35.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:34 smithi196 ceph-mon[32919]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-23T10:22:35.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:34 smithi097 ceph-mon[29197]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-23T10:22:35.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:35 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-23T10:22:35.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:35 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:22:35.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:35 smithi183 ceph-mon[32317]: Deploying daemon osd.10 on smithi183 2024-04-23T10:22:36.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:35 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-23T10:22:36.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:35 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:22:36.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:35 smithi196 ceph-mon[32919]: Deploying daemon osd.10 on smithi183 2024-04-23T10:22:36.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:35 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-23T10:22:36.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:35 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:22:36.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:35 smithi097 ceph-mon[29197]: Deploying daemon osd.10 on smithi183 2024-04-23T10:22:36.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:36 smithi183 ceph-mon[32317]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-23T10:22:37.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:36 smithi196 ceph-mon[32919]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-23T10:22:37.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:36 smithi097 ceph-mon[29197]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-23T10:22:38.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:38 smithi183 ceph-mon[32317]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-23T10:22:39.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:38 smithi196 ceph-mon[32919]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-23T10:22:39.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:38 smithi097 ceph-mon[29197]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-23T10:22:40.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:40 smithi183 ceph-mon[32317]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-23T10:22:40.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:40 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:22:40.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:40 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:22:40.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:40 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:22:40.911 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:40 smithi196 ceph-mon[32919]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-23T10:22:40.911 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:40 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:22:40.911 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:40 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:22:40.911 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:40 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:22:41.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:40 smithi097 ceph-mon[29197]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-23T10:22:41.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:40 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:22:41.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:40 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:22:41.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:40 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:22:42.840 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:42 smithi183 ceph-mon[32317]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-23T10:22:43.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:42 smithi196 ceph-mon[32919]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-23T10:22:43.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:42 smithi097 ceph-mon[29197]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-23T10:22:44.265 INFO:teuthology.orchestra.run.smithi183.stdout:Created osd(s) 10 on host 'smithi183' 2024-04-23T10:22:44.265 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:22:44.265+0000 7fdea57fa700 1 -- 172.21.15.183:0/2205177951 <== mgr.14152 v2:172.21.15.97:6800/3149043749 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fdea80ff130 con 0x7fde900a3c40 2024-04-23T10:22:44.267 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:22:44.268+0000 7fdeae6a2700 1 -- 172.21.15.183:0/2205177951 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fde900a3c40 msgr2=0x7fde900a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:22:44.267 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:22:44.268+0000 7fdeae6a2700 1 --2- 172.21.15.183:0/2205177951 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fde900a3c40 0x7fde900a6100 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdea8199b70 tx=0x7fde98009190 comp rx=0 tx=0).stop 2024-04-23T10:22:44.268 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:22:44.268+0000 7fdeae6a2700 1 -- 172.21.15.183:0/2205177951 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdea8102be0 msgr2=0x7fdea81991d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:22:44.268 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:22:44.268+0000 7fdeae6a2700 1 --2- 172.21.15.183:0/2205177951 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdea8102be0 0x7fdea81991d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fde9c00b580 tx=0x7fde9c005e80 comp rx=0 tx=0).stop 2024-04-23T10:22:44.268 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:22:44.268+0000 7fdeae6a2700 1 -- 172.21.15.183:0/2205177951 shutdown_connections 2024-04-23T10:22:44.268 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:22:44.268+0000 7fdeae6a2700 1 --2- 172.21.15.183:0/2205177951 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fde900a3c40 0x7fde900a6100 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:22:44.268 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:22:44.268+0000 7fdeae6a2700 1 --2- 172.21.15.183:0/2205177951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdea8103580 0x7fdea81a0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:22:44.268 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:22:44.268+0000 7fdeae6a2700 1 --2- 172.21.15.183:0/2205177951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdea81019e0 0x7fdea8198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:22:44.268 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:22:44.268+0000 7fdeae6a2700 1 --2- 172.21.15.183:0/2205177951 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdea8102be0 0x7fdea81991d0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:22:44.268 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:22:44.268+0000 7fdeae6a2700 1 -- 172.21.15.183:0/2205177951 >> 172.21.15.183:0/2205177951 conn(0x7fdea80fcfa0 msgr2=0x7fdea80fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:22:44.268 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:22:44.268+0000 7fdeae6a2700 1 -- 172.21.15.183:0/2205177951 shutdown_connections 2024-04-23T10:22:44.268 INFO:teuthology.orchestra.run.smithi183.stderr:2024-04-23T10:22:44.268+0000 7fdeae6a2700 1 -- 172.21.15.183:0/2205177951 wait complete. 2024-04-23T10:22:44.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:44 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:22:44.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:44 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:22:44.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:44 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:22:44.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:44 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:22:44.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:44 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:22:44.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:44 smithi196 ceph-mon[32919]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-23T10:22:44.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:44 smithi196 ceph-mon[32919]: from='osd.10 [v2:172.21.15.183:6824/799434647,v1:172.21.15.183:6825/799434647]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-23T10:22:44.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:44 smithi196 ceph-mon[32919]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-23T10:22:44.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:44 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:22:44.843 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:44 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:22:44.843 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:44 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:22:44.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:44 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:22:44.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:44 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:22:44.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:44 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:22:44.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:44 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:22:44.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:44 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:22:44.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:44 smithi097 ceph-mon[29197]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-23T10:22:44.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:44 smithi097 ceph-mon[29197]: from='osd.10 [v2:172.21.15.183:6824/799434647,v1:172.21.15.183:6825/799434647]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-23T10:22:44.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:44 smithi097 ceph-mon[29197]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-23T10:22:44.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:44 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:22:44.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:44 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:22:44.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:44 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:22:44.889 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:44 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:22:44.890 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:44 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:22:44.890 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:44 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:22:44.890 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:44 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:22:44.890 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:44 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:22:44.890 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:44 smithi183 ceph-mon[32317]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-23T10:22:44.890 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:44 smithi183 ceph-mon[32317]: from='osd.10 [v2:172.21.15.183:6824/799434647,v1:172.21.15.183:6825/799434647]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-23T10:22:44.890 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:44 smithi183 ceph-mon[32317]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-23T10:22:44.890 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:44 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:22:44.890 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:44 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:22:44.890 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:44 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:22:44.915 DEBUG:teuthology.orchestra.run.smithi183:osd.10> sudo journalctl -f -n 0 -u ceph-9b22af88-0159-11ef-bc93-c7b262605968@osd.10.service 2024-04-23T10:22:44.917 INFO:tasks.cephadm:Deploying osd.11 on smithi196 with /dev/vg_nvme/lv_1... 2024-04-23T10:22:44.917 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-23T10:22:45.163 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T10:22:45.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:45 smithi196 ceph-mon[32919]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-23T10:22:45.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:45 smithi196 ceph-mon[32919]: osdmap e71: 11 total, 10 up, 11 in 2024-04-23T10:22:45.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:45 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T10:22:45.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:45 smithi196 ceph-mon[32919]: from='osd.10 [v2:172.21.15.183:6824/799434647,v1:172.21.15.183:6825/799434647]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-23T10:22:45.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:45 smithi196 ceph-mon[32919]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-23T10:22:45.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:45 smithi097 ceph-mon[29197]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-23T10:22:45.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:45 smithi097 ceph-mon[29197]: osdmap e71: 11 total, 10 up, 11 in 2024-04-23T10:22:45.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:45 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T10:22:45.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:45 smithi097 ceph-mon[29197]: from='osd.10 [v2:172.21.15.183:6824/799434647,v1:172.21.15.183:6825/799434647]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-23T10:22:45.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:45 smithi097 ceph-mon[29197]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-23T10:22:45.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:45 smithi183 ceph-mon[32317]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-23T10:22:45.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:45 smithi183 ceph-mon[32317]: osdmap e71: 11 total, 10 up, 11 in 2024-04-23T10:22:45.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:45 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T10:22:45.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:45 smithi183 ceph-mon[32317]: from='osd.10 [v2:172.21.15.183:6824/799434647,v1:172.21.15.183:6825/799434647]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-23T10:22:45.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:45 smithi183 ceph-mon[32317]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-04-23T10:22:45.901 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:22:45 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:22:45.504+0000 7fe4ca38f700 -1 osd.10 0 waiting for initial osdmap 2024-04-23T10:22:45.901 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:22:45 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:22:45.516+0000 7fe4c6182700 -1 osd.10 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T10:22:46.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:46 smithi196 ceph-mon[32919]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-04-23T10:22:46.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:46 smithi196 ceph-mon[32919]: osdmap e72: 11 total, 10 up, 11 in 2024-04-23T10:22:46.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:46 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T10:22:46.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:46 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T10:22:46.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:46 smithi196 ceph-mon[32919]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-23T10:22:46.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:46 smithi196 ceph-mon[32919]: osd.10 [v2:172.21.15.183:6824/799434647,v1:172.21.15.183:6825/799434647] boot 2024-04-23T10:22:46.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:46 smithi196 ceph-mon[32919]: osdmap e73: 11 total, 11 up, 11 in 2024-04-23T10:22:46.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:46 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T10:22:46.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:46 smithi097 ceph-mon[29197]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-04-23T10:22:46.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:46 smithi097 ceph-mon[29197]: osdmap e72: 11 total, 10 up, 11 in 2024-04-23T10:22:46.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:46 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T10:22:46.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:46 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T10:22:46.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:46 smithi097 ceph-mon[29197]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-23T10:22:46.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:46 smithi097 ceph-mon[29197]: osd.10 [v2:172.21.15.183:6824/799434647,v1:172.21.15.183:6825/799434647] boot 2024-04-23T10:22:46.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:46 smithi097 ceph-mon[29197]: osdmap e73: 11 total, 11 up, 11 in 2024-04-23T10:22:46.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:46 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T10:22:46.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:46 smithi183 ceph-mon[32317]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-04-23T10:22:46.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:46 smithi183 ceph-mon[32317]: osdmap e72: 11 total, 10 up, 11 in 2024-04-23T10:22:46.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:46 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T10:22:46.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:46 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T10:22:46.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:46 smithi183 ceph-mon[32317]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-23T10:22:46.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:46 smithi183 ceph-mon[32317]: osd.10 [v2:172.21.15.183:6824/799434647,v1:172.21.15.183:6825/799434647] boot 2024-04-23T10:22:46.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:46 smithi183 ceph-mon[32317]: osdmap e73: 11 total, 11 up, 11 in 2024-04-23T10:22:46.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:46 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T10:22:47.200 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-23T10:22:47.200 INFO:teuthology.orchestra.run.smithi196.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-23T10:22:47.200 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10+0 records in 2024-04-23T10:22:47.200 INFO:teuthology.orchestra.run.smithi196.stderr:10+0 records out 2024-04-23T10:22:47.200 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.012299 s, 853 MB/s 2024-04-23T10:22:47.200 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping successful for: 2024-04-23T10:22:47.858 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi196:vg_nvme/lv_1 2024-04-23T10:22:48.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:47 smithi196 ceph-mon[32919]: purged_snaps scrub starts 2024-04-23T10:22:48.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:47 smithi196 ceph-mon[32919]: purged_snaps scrub ok 2024-04-23T10:22:48.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:47 smithi196 ceph-mon[32919]: osdmap e74: 11 total, 11 up, 11 in 2024-04-23T10:22:48.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:47 smithi196 ceph-mon[32919]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-04-23T10:22:48.100 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T10:22:48.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:47 smithi183 ceph-mon[32317]: purged_snaps scrub starts 2024-04-23T10:22:48.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:47 smithi183 ceph-mon[32317]: purged_snaps scrub ok 2024-04-23T10:22:48.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:47 smithi183 ceph-mon[32317]: osdmap e74: 11 total, 11 up, 11 in 2024-04-23T10:22:48.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:47 smithi183 ceph-mon[32317]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-04-23T10:22:48.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:47 smithi097 ceph-mon[29197]: purged_snaps scrub starts 2024-04-23T10:22:48.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:47 smithi097 ceph-mon[29197]: purged_snaps scrub ok 2024-04-23T10:22:48.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:47 smithi097 ceph-mon[29197]: osdmap e74: 11 total, 11 up, 11 in 2024-04-23T10:22:48.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:47 smithi097 ceph-mon[29197]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-04-23T10:22:49.138 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:48 smithi196 ceph-mon[32919]: osdmap e75: 11 total, 11 up, 11 in 2024-04-23T10:22:49.139 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:48 smithi196 ceph-mon[32919]: Detected new or changed devices on smithi183 2024-04-23T10:22:49.139 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:48 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:22:49.139 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:48 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:22:49.139 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:48 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:22:49.139 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:48 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:22:49.139 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:48 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:22:49.139 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:48 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:22:49.139 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:48 smithi196 ceph-mon[32919]: Adjusting osd_memory_target on smithi183 to 4272M 2024-04-23T10:22:49.139 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:48 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:22:49.139 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:48 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:22:49.139 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:48 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:22:49.139 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:48 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:22:49.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:48 smithi097 ceph-mon[29197]: osdmap e75: 11 total, 11 up, 11 in 2024-04-23T10:22:49.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:48 smithi097 ceph-mon[29197]: Detected new or changed devices on smithi183 2024-04-23T10:22:49.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:48 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:22:49.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:48 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:22:49.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:48 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:22:49.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:48 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:22:49.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:48 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:22:49.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:48 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:22:49.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:48 smithi097 ceph-mon[29197]: Adjusting osd_memory_target on smithi183 to 4272M 2024-04-23T10:22:49.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:48 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:22:49.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:48 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:22:49.362 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:48 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:22:49.362 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:48 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:22:49.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:48 smithi183 ceph-mon[32317]: osdmap e75: 11 total, 11 up, 11 in 2024-04-23T10:22:49.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:48 smithi183 ceph-mon[32317]: Detected new or changed devices on smithi183 2024-04-23T10:22:49.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:48 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:22:49.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:48 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:22:49.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:48 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:22:49.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:48 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:22:49.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:48 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:22:49.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:48 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:22:49.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:48 smithi183 ceph-mon[32317]: Adjusting osd_memory_target on smithi183 to 4272M 2024-04-23T10:22:49.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:48 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:22:49.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:48 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:22:49.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:48 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:22:49.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:48 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:22:49.734 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:22:49.707+0000 7f54c2da8700 1 -- 172.21.15.196:0/3386771860 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54bc101990 msgr2=0x7f54bc101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:22:49.734 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:22:49.707+0000 7f54c2da8700 1 --2- 172.21.15.196:0/3386771860 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54bc101990 0x7f54bc101db0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f54b0009990 tx=0x7f54b0009ca0 comp rx=0 tx=0).stop 2024-04-23T10:22:49.735 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:22:49.707+0000 7f54c2da8700 1 -- 172.21.15.196:0/3386771860 shutdown_connections 2024-04-23T10:22:49.735 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:22:49.707+0000 7f54c2da8700 1 --2- 172.21.15.196:0/3386771860 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54bc103530 0x7f54bc109de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:22:49.735 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:22:49.707+0000 7f54c2da8700 1 --2- 172.21.15.196:0/3386771860 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54bc101990 0x7f54bc101db0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:22:49.735 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:22:49.707+0000 7f54c2da8700 1 --2- 172.21.15.196:0/3386771860 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54bc102b90 0x7f54bc102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:22:49.735 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:22:49.707+0000 7f54c2da8700 1 -- 172.21.15.196:0/3386771860 >> 172.21.15.196:0/3386771860 conn(0x7f54bc0fcf30 msgr2=0x7f54bc0ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:22:49.735 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:22:49.707+0000 7f54c2da8700 1 -- 172.21.15.196:0/3386771860 shutdown_connections 2024-04-23T10:22:49.735 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:22:49.707+0000 7f54c2da8700 1 -- 172.21.15.196:0/3386771860 wait complete. 2024-04-23T10:22:49.735 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:22:49.708+0000 7f54c2da8700 1 Processor -- start 2024-04-23T10:22:49.735 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:22:49.708+0000 7f54c2da8700 1 -- start start 2024-04-23T10:22:49.735 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:22:49.708+0000 7f54c2da8700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54bc101990 0x7f54bc198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:22:49.736 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:22:49.708+0000 7f54c2da8700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54bc102b90 0x7f54bc199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:22:49.736 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:22:49.708+0000 7f54c2da8700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54bc103530 0x7f54bc1a0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:22:49.736 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:22:49.708+0000 7f54c2da8700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f54bc10d400 con 0x7f54bc101990 2024-04-23T10:22:49.736 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:22:49.708+0000 7f54c2da8700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f54bc10d280 con 0x7f54bc103530 2024-04-23T10:22:49.736 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:22:49.708+0000 7f54c2da8700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f54bc10d580 con 0x7f54bc102b90 2024-04-23T10:22:49.736 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:22:49.708+0000 7f54bbfff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54bc102b90 0x7f54bc199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:22:49.736 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:22:49.708+0000 7f54bbfff700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54bc102b90 0x7f54bc199110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:58442/0 (socket says 172.21.15.196:58442) 2024-04-23T10:22:49.736 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:22:49.708+0000 7f54bbfff700 1 -- 172.21.15.196:0/4123853111 learned_addr learned my addr 172.21.15.196:0/4123853111 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-23T10:22:49.736 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:22:49.708+0000 7f54c0b44700 1 --2- 172.21.15.196:0/4123853111 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54bc101990 0x7f54bc198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:22:49.736 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:22:49.709+0000 7f54bbfff700 1 -- 172.21.15.196:0/4123853111 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54bc103530 msgr2=0x7f54bc1a0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:22:49.736 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:22:49.709+0000 7f54c1345700 1 --2- 172.21.15.196:0/4123853111 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54bc103530 0x7f54bc1a0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:22:49.736 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:22:49.709+0000 7f54bbfff700 1 --2- 172.21.15.196:0/4123853111 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54bc103530 0x7f54bc1a0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:22:49.736 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:22:49.709+0000 7f54bbfff700 1 -- 172.21.15.196:0/4123853111 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54bc101990 msgr2=0x7f54bc198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:22:49.737 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:22:49.709+0000 7f54bbfff700 1 --2- 172.21.15.196:0/4123853111 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54bc101990 0x7f54bc198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:22:49.737 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:22:49.709+0000 7f54bbfff700 1 -- 172.21.15.196:0/4123853111 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54bc1a07e0 con 0x7f54bc102b90 2024-04-23T10:22:49.737 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:22:49.709+0000 7f54c0b44700 1 --2- 172.21.15.196:0/4123853111 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54bc101990 0x7f54bc198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:22:49.737 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:22:49.709+0000 7f54c1345700 1 --2- 172.21.15.196:0/4123853111 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54bc103530 0x7f54bc1a0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:22:49.737 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:22:49.709+0000 7f54bbfff700 1 --2- 172.21.15.196:0/4123853111 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54bc102b90 0x7f54bc199110 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f54a800c9e0 tx=0x7f54a800cda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:22:49.737 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:22:49.709+0000 7f54b9ffb700 1 -- 172.21.15.196:0/4123853111 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a8007ac0 con 0x7f54bc102b90 2024-04-23T10:22:49.737 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:22:49.709+0000 7f54b9ffb700 1 -- 172.21.15.196:0/4123853111 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54a8007c60 con 0x7f54bc102b90 2024-04-23T10:22:49.737 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:22:49.709+0000 7f54c2da8700 1 -- 172.21.15.196:0/4123853111 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54bc1a0ad0 con 0x7f54bc102b90 2024-04-23T10:22:49.737 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:22:49.709+0000 7f54b9ffb700 1 -- 172.21.15.196:0/4123853111 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a8018660 con 0x7f54bc102b90 2024-04-23T10:22:49.737 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:22:49.709+0000 7f54c2da8700 1 -- 172.21.15.196:0/4123853111 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54bc1a1060 con 0x7f54bc102b90 2024-04-23T10:22:49.737 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:22:49.710+0000 7f54c2da8700 1 -- 172.21.15.196:0/4123853111 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54bc069730 con 0x7f54bc102b90 2024-04-23T10:22:49.737 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:22:49.712+0000 7f54b9ffb700 1 -- 172.21.15.196:0/4123853111 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f54a800f450 con 0x7f54bc102b90 2024-04-23T10:22:49.737 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:22:49.712+0000 7f54b9ffb700 1 --2- 172.21.15.196:0/4123853111 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f54ac0a3cc0 0x7f54ac0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:22:49.737 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:22:49.713+0000 7f54b9ffb700 1 -- 172.21.15.196:0/4123853111 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6694+0+0 (secure 0 0 0) 0x7f54a80d00d0 con 0x7f54bc102b90 2024-04-23T10:22:49.738 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:22:49.713+0000 7f54c0b44700 1 --2- 172.21.15.196:0/4123853111 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f54ac0a3cc0 0x7f54ac0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:22:49.738 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:22:49.713+0000 7f54c0b44700 1 --2- 172.21.15.196:0/4123853111 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f54ac0a3cc0 0x7f54ac0a6180 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f54b00038a0 tx=0x7f54b0003680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:22:49.738 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:22:49.716+0000 7f54b9ffb700 1 -- 172.21.15.196:0/4123853111 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f54a8014030 con 0x7f54bc102b90 2024-04-23T10:22:49.921 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:22:49.922+0000 7f54c2da8700 1 -- 172.21.15.196:0/4123853111 --> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f54bc0ff130 con 0x7f54ac0a3cc0 2024-04-23T10:22:50.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:49 smithi196 ceph-mon[32919]: osdmap e76: 11 total, 11 up, 11 in 2024-04-23T10:22:50.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:49 smithi196 ceph-mon[32919]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-23T10:22:50.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:49 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T10:22:50.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:49 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T10:22:50.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:49 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:22:50.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:49 smithi097 ceph-mon[29197]: osdmap e76: 11 total, 11 up, 11 in 2024-04-23T10:22:50.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:49 smithi097 ceph-mon[29197]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-23T10:22:50.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:49 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T10:22:50.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:49 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T10:22:50.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:49 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:22:50.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:49 smithi183 ceph-mon[32317]: osdmap e76: 11 total, 11 up, 11 in 2024-04-23T10:22:50.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:49 smithi183 ceph-mon[32317]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-23T10:22:50.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:49 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T10:22:50.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:49 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T10:22:50.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:49 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:22:51.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:50 smithi196 ceph-mon[32919]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:22:51.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:50 smithi196 ceph-mon[32919]: osdmap e77: 11 total, 11 up, 11 in 2024-04-23T10:22:51.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:50 smithi097 ceph-mon[29197]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:22:51.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:50 smithi097 ceph-mon[29197]: osdmap e77: 11 total, 11 up, 11 in 2024-04-23T10:22:51.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:50 smithi183 ceph-mon[32317]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:22:51.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:50 smithi183 ceph-mon[32317]: osdmap e77: 11 total, 11 up, 11 in 2024-04-23T10:22:52.170 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:51 smithi196 ceph-mon[32919]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 125 KiB/s, 0 objects/s recovering 2024-04-23T10:22:52.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:51 smithi097 ceph-mon[29197]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 125 KiB/s, 0 objects/s recovering 2024-04-23T10:22:52.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:51 smithi183 ceph-mon[32317]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 125 KiB/s, 0 objects/s recovering 2024-04-23T10:22:53.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:52 smithi196 ceph-mon[32919]: from='client.? 172.21.15.196:0/2348826350' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95d64a89-a9f8-4c7a-8c0d-da7fb26cfff8"}]: dispatch 2024-04-23T10:22:53.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:52 smithi196 ceph-mon[32919]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95d64a89-a9f8-4c7a-8c0d-da7fb26cfff8"}]: dispatch 2024-04-23T10:22:53.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:52 smithi196 ceph-mon[32919]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "95d64a89-a9f8-4c7a-8c0d-da7fb26cfff8"}]': finished 2024-04-23T10:22:53.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:52 smithi196 ceph-mon[32919]: osdmap e78: 12 total, 11 up, 12 in 2024-04-23T10:22:53.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:52 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T10:22:53.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:52 smithi097 ceph-mon[29197]: from='client.? 172.21.15.196:0/2348826350' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95d64a89-a9f8-4c7a-8c0d-da7fb26cfff8"}]: dispatch 2024-04-23T10:22:53.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:52 smithi097 ceph-mon[29197]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95d64a89-a9f8-4c7a-8c0d-da7fb26cfff8"}]: dispatch 2024-04-23T10:22:53.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:52 smithi097 ceph-mon[29197]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "95d64a89-a9f8-4c7a-8c0d-da7fb26cfff8"}]': finished 2024-04-23T10:22:53.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:52 smithi097 ceph-mon[29197]: osdmap e78: 12 total, 11 up, 12 in 2024-04-23T10:22:53.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:52 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T10:22:53.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:52 smithi183 ceph-mon[32317]: from='client.? 172.21.15.196:0/2348826350' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95d64a89-a9f8-4c7a-8c0d-da7fb26cfff8"}]: dispatch 2024-04-23T10:22:53.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:52 smithi183 ceph-mon[32317]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95d64a89-a9f8-4c7a-8c0d-da7fb26cfff8"}]: dispatch 2024-04-23T10:22:53.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:52 smithi183 ceph-mon[32317]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "95d64a89-a9f8-4c7a-8c0d-da7fb26cfff8"}]': finished 2024-04-23T10:22:53.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:52 smithi183 ceph-mon[32317]: osdmap e78: 12 total, 11 up, 12 in 2024-04-23T10:22:53.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:52 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T10:22:54.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:53 smithi183 ceph-mon[32317]: from='client.? 172.21.15.196:0/2637705023' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T10:22:54.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:53 smithi183 ceph-mon[32317]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-04-23T10:22:54.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:53 smithi196 ceph-mon[32919]: from='client.? 172.21.15.196:0/2637705023' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T10:22:54.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:53 smithi196 ceph-mon[32919]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-04-23T10:22:54.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:53 smithi097 ceph-mon[29197]: from='client.? 172.21.15.196:0/2637705023' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T10:22:54.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:53 smithi097 ceph-mon[29197]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-04-23T10:22:56.774 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:56 smithi196 ceph-mon[32919]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-04-23T10:22:56.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:56 smithi097 ceph-mon[29197]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-04-23T10:22:56.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:56 smithi183 ceph-mon[32317]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-04-23T10:22:58.844 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:22:58 smithi196 ceph-mon[32919]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-23T10:22:58.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:22:58 smithi097 ceph-mon[29197]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-23T10:22:58.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:22:58 smithi183 ceph-mon[32317]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-23T10:23:00.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:00 smithi196 ceph-mon[32919]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-23T10:23:00.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:00 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-23T10:23:00.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:00 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:23:00.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:00 smithi097 ceph-mon[29197]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-23T10:23:00.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:00 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-23T10:23:00.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:00 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:23:00.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:00 smithi183 ceph-mon[32317]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-23T10:23:00.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:00 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-23T10:23:00.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:00 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:23:01.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:01 smithi097 ceph-mon[29197]: Deploying daemon osd.11 on smithi196 2024-04-23T10:23:01.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:01 smithi183 ceph-mon[32317]: Deploying daemon osd.11 on smithi196 2024-04-23T10:23:01.990 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:01 smithi196 ceph-mon[32919]: Deploying daemon osd.11 on smithi196 2024-04-23T10:23:02.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:02 smithi097 ceph-mon[29197]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-23T10:23:02.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:02 smithi183 ceph-mon[32317]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-23T10:23:03.083 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:02 smithi196 ceph-mon[32919]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-23T10:23:04.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:04 smithi196 ceph-mon[32919]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-23T10:23:04.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:04 smithi097 ceph-mon[29197]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-23T10:23:04.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:04 smithi183 ceph-mon[32317]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-23T10:23:06.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:06 smithi097 ceph-mon[29197]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-23T10:23:06.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:06 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:06.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:06 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:23:06.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:06 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:06.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:06 smithi183 ceph-mon[32317]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-23T10:23:06.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:06 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:06.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:06 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:23:06.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:06 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:06.979 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:06 smithi196 ceph-mon[32919]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-23T10:23:06.979 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:06 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:06.979 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:06 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:23:06.979 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:06 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:08.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:08 smithi196 ceph-mon[32919]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-23T10:23:08.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:08 smithi183 ceph-mon[32317]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-23T10:23:09.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:08 smithi097 ceph-mon[29197]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-23T10:23:10.138 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:23:10.139+0000 7f54b9ffb700 1 -- 172.21.15.196:0/4123853111 <== mgr.14152 v2:172.21.15.97:6800/3149043749 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f54bc0ff130 con 0x7f54ac0a3cc0 2024-04-23T10:23:10.139 INFO:teuthology.orchestra.run.smithi196.stdout:Created osd(s) 11 on host 'smithi196' 2024-04-23T10:23:10.141 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:23:10.142+0000 7f54c2da8700 1 -- 172.21.15.196:0/4123853111 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f54ac0a3cc0 msgr2=0x7f54ac0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:10.141 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:23:10.142+0000 7f54c2da8700 1 --2- 172.21.15.196:0/4123853111 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f54ac0a3cc0 0x7f54ac0a6180 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f54b00038a0 tx=0x7f54b0003680 comp rx=0 tx=0).stop 2024-04-23T10:23:10.141 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:23:10.142+0000 7f54c2da8700 1 -- 172.21.15.196:0/4123853111 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54bc102b90 msgr2=0x7f54bc199110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:10.141 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:23:10.142+0000 7f54c2da8700 1 --2- 172.21.15.196:0/4123853111 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54bc102b90 0x7f54bc199110 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f54a800c9e0 tx=0x7f54a800cda0 comp rx=0 tx=0).stop 2024-04-23T10:23:10.141 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:23:10.142+0000 7f54c2da8700 1 -- 172.21.15.196:0/4123853111 shutdown_connections 2024-04-23T10:23:10.141 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:23:10.142+0000 7f54c2da8700 1 --2- 172.21.15.196:0/4123853111 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f54ac0a3cc0 0x7f54ac0a6180 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:10.141 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:23:10.142+0000 7f54c2da8700 1 --2- 172.21.15.196:0/4123853111 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54bc103530 0x7f54bc1a0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:10.141 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:23:10.142+0000 7f54c2da8700 1 --2- 172.21.15.196:0/4123853111 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54bc101990 0x7f54bc198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:10.141 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:23:10.142+0000 7f54c2da8700 1 --2- 172.21.15.196:0/4123853111 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54bc102b90 0x7f54bc199110 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:10.142 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:23:10.142+0000 7f54c2da8700 1 -- 172.21.15.196:0/4123853111 >> 172.21.15.196:0/4123853111 conn(0x7f54bc0fcf30 msgr2=0x7f54bc0fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:23:10.142 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:23:10.142+0000 7f54c2da8700 1 -- 172.21.15.196:0/4123853111 shutdown_connections 2024-04-23T10:23:10.142 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:23:10.142+0000 7f54c2da8700 1 -- 172.21.15.196:0/4123853111 wait complete. 2024-04-23T10:23:10.831 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:10 smithi196 ceph-mon[32919]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-23T10:23:10.831 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:10 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:10.831 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:10 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:10.831 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:10 smithi196 ceph-mon[32919]: from='osd.11 [v2:172.21.15.196:6824/1873969467,v1:172.21.15.196:6825/1873969467]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-23T10:23:10.831 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:10 smithi196 ceph-mon[32919]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-23T10:23:10.862 DEBUG:teuthology.orchestra.run.smithi196:osd.11> sudo journalctl -f -n 0 -u ceph-9b22af88-0159-11ef-bc93-c7b262605968@osd.11.service 2024-04-23T10:23:10.864 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-23T10:23:10.864 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-23T10:23:10.890 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:10 smithi097 ceph-mon[29197]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-23T10:23:10.891 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:10 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:10.891 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:10 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:10.891 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:10 smithi097 ceph-mon[29197]: from='osd.11 [v2:172.21.15.196:6824/1873969467,v1:172.21.15.196:6825/1873969467]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-23T10:23:10.891 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:10 smithi097 ceph-mon[29197]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-23T10:23:10.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:10 smithi183 ceph-mon[32317]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-23T10:23:10.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:10 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:10.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:10 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:10.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:10 smithi183 ceph-mon[32317]: from='osd.11 [v2:172.21.15.196:6824/1873969467,v1:172.21.15.196:6825/1873969467]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-23T10:23:10.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:10 smithi183 ceph-mon[32317]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-23T10:23:11.115 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T10:23:12.088 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:11 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:12.088 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:11 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:12.088 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:11 smithi196 ceph-mon[32919]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-23T10:23:12.088 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:11 smithi196 ceph-mon[32919]: osdmap e79: 12 total, 11 up, 12 in 2024-04-23T10:23:12.088 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:11 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T10:23:12.089 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:11 smithi196 ceph-mon[32919]: from='osd.11 [v2:172.21.15.196:6824/1873969467,v1:172.21.15.196:6825/1873969467]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-23T10:23:12.089 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:11 smithi196 ceph-mon[32919]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-23T10:23:12.089 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:11 smithi196 ceph-mon[32919]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-23T10:23:12.273 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:11 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:12.273 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:11 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:12.273 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:11 smithi097 ceph-mon[29197]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-23T10:23:12.273 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:11 smithi097 ceph-mon[29197]: osdmap e79: 12 total, 11 up, 12 in 2024-04-23T10:23:12.273 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:11 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T10:23:12.273 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:11 smithi097 ceph-mon[29197]: from='osd.11 [v2:172.21.15.196:6824/1873969467,v1:172.21.15.196:6825/1873969467]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-23T10:23:12.273 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:11 smithi097 ceph-mon[29197]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-23T10:23:12.274 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:11 smithi097 ceph-mon[29197]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-23T10:23:12.342 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:23:12 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:23:12.116+0000 7fa4fa475700 -1 osd.11 0 waiting for initial osdmap 2024-04-23T10:23:12.342 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:23:12 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:23:12.130+0000 7fa4f6268700 -1 osd.11 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T10:23:12.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:11 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:12.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:11 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:12.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:11 smithi183 ceph-mon[32317]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-23T10:23:12.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:11 smithi183 ceph-mon[32317]: osdmap e79: 12 total, 11 up, 12 in 2024-04-23T10:23:12.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:11 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T10:23:12.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:11 smithi183 ceph-mon[32317]: from='osd.11 [v2:172.21.15.196:6824/1873969467,v1:172.21.15.196:6825/1873969467]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-23T10:23:12.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:11 smithi183 ceph-mon[32317]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-04-23T10:23:12.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:11 smithi183 ceph-mon[32317]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-23T10:23:12.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:12.850+0000 7f0f28d12700 1 -- 172.21.15.97:0/2591984211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f241025c0 msgr2=0x7f0f24109e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:12.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:12.850+0000 7f0f28d12700 1 --2- 172.21.15.97:0/2591984211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f241025c0 0x7f0f24109e60 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c00b0a0 tx=0x7f0f1c00b3b0 comp rx=0 tx=0).stop 2024-04-23T10:23:12.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:12.850+0000 7f0f28d12700 1 -- 172.21.15.97:0/2591984211 shutdown_connections 2024-04-23T10:23:12.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:12.850+0000 7f0f28d12700 1 --2- 172.21.15.97:0/2591984211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f241025c0 0x7f0f24109e60 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:12.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:12.850+0000 7f0f28d12700 1 --2- 172.21.15.97:0/2591984211 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f24102fa0 0x7f0f241033c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:12.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:12.850+0000 7f0f28d12700 1 --2- 172.21.15.97:0/2591984211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f24101c00 0x7f0f24102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:12.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:12.850+0000 7f0f28d12700 1 -- 172.21.15.97:0/2591984211 >> 172.21.15.97:0/2591984211 conn(0x7f0f240fc6d0 msgr2=0x7f0f240feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:23:12.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:12.850+0000 7f0f28d12700 1 -- 172.21.15.97:0/2591984211 shutdown_connections 2024-04-23T10:23:12.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:12.850+0000 7f0f28d12700 1 -- 172.21.15.97:0/2591984211 wait complete. 2024-04-23T10:23:12.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:12.850+0000 7f0f28d12700 1 Processor -- start 2024-04-23T10:23:12.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:12.851+0000 7f0f28d12700 1 -- start start 2024-04-23T10:23:12.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:12.851+0000 7f0f28d12700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f24101c00 0x7f0f24101650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:12.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:12.851+0000 7f0f28d12700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f241025c0 0x7f0f2419d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:12.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:12.851+0000 7f0f28d12700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f24102fa0 0x7f0f2419fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:12.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:12.851+0000 7f0f28d12700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0f2410d3f0 con 0x7f0f24101c00 2024-04-23T10:23:12.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:12.851+0000 7f0f28d12700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0f2410d270 con 0x7f0f24102fa0 2024-04-23T10:23:12.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:12.851+0000 7f0f28d12700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0f2410d570 con 0x7f0f241025c0 2024-04-23T10:23:12.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:12.851+0000 7f0f2259c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f24101c00 0x7f0f24101650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:12.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:12.851+0000 7f0f2259c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f24101c00 0x7f0f24101650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48568/0 (socket says 172.21.15.97:48568) 2024-04-23T10:23:12.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:12.851+0000 7f0f2259c700 1 -- 172.21.15.97:0/3313597666 learned_addr learned my addr 172.21.15.97:0/3313597666 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:23:12.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:12.851+0000 7f0f22d9d700 1 --2- 172.21.15.97:0/3313597666 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f24102fa0 0x7f0f2419fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:12.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:12.851+0000 7f0f2259c700 1 -- 172.21.15.97:0/3313597666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f241025c0 msgr2=0x7f0f2419d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:12.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:12.851+0000 7f0f2259c700 1 --2- 172.21.15.97:0/3313597666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f241025c0 0x7f0f2419d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:12.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:12.851+0000 7f0f2259c700 1 -- 172.21.15.97:0/3313597666 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f24102fa0 msgr2=0x7f0f2419fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:12.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:12.851+0000 7f0f2259c700 1 --2- 172.21.15.97:0/3313597666 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f24102fa0 0x7f0f2419fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:12.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:12.851+0000 7f0f2259c700 1 -- 172.21.15.97:0/3313597666 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f241a0710 con 0x7f0f24101c00 2024-04-23T10:23:12.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:12.851+0000 7f0f2259c700 1 --2- 172.21.15.97:0/3313597666 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f24101c00 0x7f0f24101650 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0f1400c990 tx=0x7f0f1400cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:23:12.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:12.851+0000 7f0f1bfff700 1 -- 172.21.15.97:0/3313597666 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f14007b00 con 0x7f0f24101c00 2024-04-23T10:23:12.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:12.852+0000 7f0f1bfff700 1 -- 172.21.15.97:0/3313597666 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f14007ca0 con 0x7f0f24101c00 2024-04-23T10:23:12.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:12.852+0000 7f0f28d12700 1 -- 172.21.15.97:0/3313597666 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f241a0a00 con 0x7f0f24101c00 2024-04-23T10:23:12.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:12.852+0000 7f0f1bfff700 1 -- 172.21.15.97:0/3313597666 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f14018660 con 0x7f0f24101c00 2024-04-23T10:23:12.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:12.852+0000 7f0f28d12700 1 -- 172.21.15.97:0/3313597666 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f241a0f40 con 0x7f0f24101c00 2024-04-23T10:23:12.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:12.853+0000 7f0f28d12700 1 -- 172.21.15.97:0/3313597666 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f24069730 con 0x7f0f24101c00 2024-04-23T10:23:12.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:12.854+0000 7f0f1bfff700 1 -- 172.21.15.97:0/3313597666 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0f1400f450 con 0x7f0f24101c00 2024-04-23T10:23:12.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:12.854+0000 7f0f1bfff700 1 --2- 172.21.15.97:0/3313597666 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f0f100a3c40 0x7f0f100a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:12.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:12.854+0000 7f0f19d9b700 1 --2- 172.21.15.97:0/3313597666 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f0f100a3c40 0x7f0f100a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:12.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:12.854+0000 7f0f1bfff700 1 -- 172.21.15.97:0/3313597666 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6847+0+0 (secure 0 0 0) 0x7f0f140cfcf0 con 0x7f0f24101c00 2024-04-23T10:23:12.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:12.855+0000 7f0f19d9b700 1 --2- 172.21.15.97:0/3313597666 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f0f100a3c40 0x7f0f100a6100 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c005d90 tx=0x7f0f0c005d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:23:12.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:12.859+0000 7f0f1bfff700 1 -- 172.21.15.97:0/3313597666 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0f140d4070 con 0x7f0f24101c00 2024-04-23T10:23:13.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:13.055+0000 7f0f28d12700 1 -- 172.21.15.97:0/3313597666 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f0f24063650 con 0x7f0f24101c00 2024-04-23T10:23:13.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:13.055+0000 7f0f1bfff700 1 -- 172.21.15.97:0/3313597666 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v80) ==== 74+0+133 (secure 0 0 0) 0x7f0f14014030 con 0x7f0f24101c00 2024-04-23T10:23:13.055 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:23:13.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:13.057+0000 7f0f28d12700 1 -- 172.21.15.97:0/3313597666 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f0f100a3c40 msgr2=0x7f0f100a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:13.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:13.057+0000 7f0f28d12700 1 --2- 172.21.15.97:0/3313597666 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f0f100a3c40 0x7f0f100a6100 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c005d90 tx=0x7f0f0c005d00 comp rx=0 tx=0).stop 2024-04-23T10:23:13.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:13.057+0000 7f0f28d12700 1 -- 172.21.15.97:0/3313597666 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f24101c00 msgr2=0x7f0f24101650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:13.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:13.057+0000 7f0f28d12700 1 --2- 172.21.15.97:0/3313597666 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f24101c00 0x7f0f24101650 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0f1400c990 tx=0x7f0f1400cca0 comp rx=0 tx=0).stop 2024-04-23T10:23:13.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:13.058+0000 7f0f28d12700 1 -- 172.21.15.97:0/3313597666 shutdown_connections 2024-04-23T10:23:13.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:13.058+0000 7f0f28d12700 1 --2- 172.21.15.97:0/3313597666 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f0f100a3c40 0x7f0f100a6100 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:13.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:13.058+0000 7f0f28d12700 1 --2- 172.21.15.97:0/3313597666 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f24102fa0 0x7f0f2419fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:13.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:13.058+0000 7f0f28d12700 1 --2- 172.21.15.97:0/3313597666 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f24101c00 0x7f0f24101650 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:13.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:13.058+0000 7f0f28d12700 1 --2- 172.21.15.97:0/3313597666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f241025c0 0x7f0f2419d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:13.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:13.058+0000 7f0f28d12700 1 -- 172.21.15.97:0/3313597666 >> 172.21.15.97:0/3313597666 conn(0x7f0f240fc6d0 msgr2=0x7f0f24105f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:23:13.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:13.058+0000 7f0f28d12700 1 -- 172.21.15.97:0/3313597666 shutdown_connections 2024-04-23T10:23:13.058 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:13.058+0000 7f0f28d12700 1 -- 172.21.15.97:0/3313597666 wait complete. 2024-04-23T10:23:13.326 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:13 smithi097 ceph-mon[29197]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-04-23T10:23:13.327 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:13 smithi097 ceph-mon[29197]: osdmap e80: 12 total, 11 up, 12 in 2024-04-23T10:23:13.327 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:13 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T10:23:13.327 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:13 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T10:23:13.327 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:13 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/3313597666' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-23T10:23:13.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:13 smithi183 ceph-mon[32317]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-04-23T10:23:13.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:13 smithi183 ceph-mon[32317]: osdmap e80: 12 total, 11 up, 12 in 2024-04-23T10:23:13.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:13 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T10:23:13.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:13 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T10:23:13.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:13 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/3313597666' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-23T10:23:13.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:13 smithi196 ceph-mon[32919]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-04-23T10:23:13.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:13 smithi196 ceph-mon[32919]: osdmap e80: 12 total, 11 up, 12 in 2024-04-23T10:23:13.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:13 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T10:23:13.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:13 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T10:23:13.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:13 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/3313597666' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-23T10:23:13.849 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":80,"num_osds":12,"num_up_osds":11,"osd_up_since":1713867765,"num_in_osds":12,"osd_in_since":1713867772,"num_remapped_pgs":0} 2024-04-23T10:23:14.339 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:14 smithi196 ceph-mon[32919]: purged_snaps scrub starts 2024-04-23T10:23:14.339 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:14 smithi196 ceph-mon[32919]: purged_snaps scrub ok 2024-04-23T10:23:14.339 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:14 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T10:23:14.340 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:14 smithi196 ceph-mon[32919]: osd.11 [v2:172.21.15.196:6824/1873969467,v1:172.21.15.196:6825/1873969467] boot 2024-04-23T10:23:14.340 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:14 smithi196 ceph-mon[32919]: osdmap e81: 12 total, 12 up, 12 in 2024-04-23T10:23:14.340 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:14 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T10:23:14.340 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:14 smithi196 ceph-mon[32919]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-23T10:23:14.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:14 smithi097 ceph-mon[29197]: purged_snaps scrub starts 2024-04-23T10:23:14.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:14 smithi097 ceph-mon[29197]: purged_snaps scrub ok 2024-04-23T10:23:14.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:14 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T10:23:14.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:14 smithi097 ceph-mon[29197]: osd.11 [v2:172.21.15.196:6824/1873969467,v1:172.21.15.196:6825/1873969467] boot 2024-04-23T10:23:14.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:14 smithi097 ceph-mon[29197]: osdmap e81: 12 total, 12 up, 12 in 2024-04-23T10:23:14.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:14 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T10:23:14.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:14 smithi097 ceph-mon[29197]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-23T10:23:14.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:14 smithi183 ceph-mon[32317]: purged_snaps scrub starts 2024-04-23T10:23:14.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:14 smithi183 ceph-mon[32317]: purged_snaps scrub ok 2024-04-23T10:23:14.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:14 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T10:23:14.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:14 smithi183 ceph-mon[32317]: osd.11 [v2:172.21.15.196:6824/1873969467,v1:172.21.15.196:6825/1873969467] boot 2024-04-23T10:23:14.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:14 smithi183 ceph-mon[32317]: osdmap e81: 12 total, 12 up, 12 in 2024-04-23T10:23:14.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:14 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T10:23:14.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:14 smithi183 ceph-mon[32317]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-04-23T10:23:14.850 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-23T10:23:15.091 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T10:23:15.212 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:15 smithi097 ceph-mon[29197]: osdmap e82: 12 total, 12 up, 12 in 2024-04-23T10:23:15.213 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:15 smithi097 ceph-mon[29197]: Detected new or changed devices on smithi196 2024-04-23T10:23:15.213 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:15 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:15.213 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:15 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:15.213 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:15 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:23:15.213 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:15 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:23:15.213 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:15 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:23:15.213 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:15 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:23:15.213 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:15 smithi097 ceph-mon[29197]: Adjusting osd_memory_target on smithi196 to 4272M 2024-04-23T10:23:15.213 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:15 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:15.213 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:15 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:23:15.213 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:15 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:23:15.214 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:15 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:15.214 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:15 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:23:15.214 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:15 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:23:15.214 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:15 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:23:15.214 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:15 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:15.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:15 smithi183 ceph-mon[32317]: osdmap e82: 12 total, 12 up, 12 in 2024-04-23T10:23:15.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:15 smithi183 ceph-mon[32317]: Detected new or changed devices on smithi196 2024-04-23T10:23:15.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:15 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:15.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:15 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:15.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:15 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:23:15.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:15 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:23:15.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:15 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:23:15.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:15 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:23:15.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:15 smithi183 ceph-mon[32317]: Adjusting osd_memory_target on smithi196 to 4272M 2024-04-23T10:23:15.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:15 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:15.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:15 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:23:15.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:15 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:23:15.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:15 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:15.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:15 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:23:15.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:15 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:23:15.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:15 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:23:15.402 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:15 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:15.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:15 smithi196 ceph-mon[32919]: osdmap e82: 12 total, 12 up, 12 in 2024-04-23T10:23:15.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:15 smithi196 ceph-mon[32919]: Detected new or changed devices on smithi196 2024-04-23T10:23:15.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:15 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:15.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:15 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:15.593 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:15 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:23:15.593 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:15 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:23:15.593 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:15 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:23:15.593 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:15 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:23:15.593 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:15 smithi196 ceph-mon[32919]: Adjusting osd_memory_target on smithi196 to 4272M 2024-04-23T10:23:15.593 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:15 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:15.593 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:15 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:23:15.593 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:15 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:23:15.593 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:15 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:15.593 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:15 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:23:15.594 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:15 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:23:15.594 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:15 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:23:15.594 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:15 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:16.273 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:16 smithi097 ceph-mon[29197]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:23:16.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:16 smithi183 ceph-mon[32317]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:23:16.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:16 smithi196 ceph-mon[32919]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:23:16.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:16.859+0000 7fa2067a9700 1 -- 172.21.15.97:0/1613138660 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa200103540 msgr2=0x7fa200109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:16.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:16.859+0000 7fa2067a9700 1 --2- 172.21.15.97:0/1613138660 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa200103540 0x7fa200109df0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa1f800b0d0 tx=0x7fa1f800b3e0 comp rx=0 tx=0).stop 2024-04-23T10:23:16.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:16.859+0000 7fa2067a9700 1 -- 172.21.15.97:0/1613138660 shutdown_connections 2024-04-23T10:23:16.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:16.859+0000 7fa2067a9700 1 --2- 172.21.15.97:0/1613138660 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa200103540 0x7fa200109df0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:16.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:16.859+0000 7fa2067a9700 1 --2- 172.21.15.97:0/1613138660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2001019a0 0x7fa200101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:16.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:16.859+0000 7fa2067a9700 1 --2- 172.21.15.97:0/1613138660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa200102ba0 0x7fa200103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:16.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:16.859+0000 7fa2067a9700 1 -- 172.21.15.97:0/1613138660 >> 172.21.15.97:0/1613138660 conn(0x7fa2000fcf40 msgr2=0x7fa2000ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:23:16.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:16.859+0000 7fa2067a9700 1 -- 172.21.15.97:0/1613138660 shutdown_connections 2024-04-23T10:23:16.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:16.859+0000 7fa2067a9700 1 -- 172.21.15.97:0/1613138660 wait complete. 2024-04-23T10:23:16.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:16.860+0000 7fa2067a9700 1 Processor -- start 2024-04-23T10:23:16.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:16.860+0000 7fa2067a9700 1 -- start start 2024-04-23T10:23:16.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:16.860+0000 7fa2067a9700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2001019a0 0x7fa20006b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:16.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:16.860+0000 7fa2067a9700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa200102ba0 0x7fa20019d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:16.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:16.860+0000 7fa2067a9700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa200103540 0x7fa20019ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:16.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:16.860+0000 7fa2067a9700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa20010d440 con 0x7fa2001019a0 2024-04-23T10:23:16.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:16.860+0000 7fa2067a9700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa20010d2c0 con 0x7fa200103540 2024-04-23T10:23:16.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:16.860+0000 7fa2067a9700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa20010d5c0 con 0x7fa200102ba0 2024-04-23T10:23:16.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:16.861+0000 7fa1fffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2001019a0 0x7fa20006b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:16.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:16.861+0000 7fa1fffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2001019a0 0x7fa20006b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45308/0 (socket says 172.21.15.97:45308) 2024-04-23T10:23:16.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:16.861+0000 7fa1fffff700 1 -- 172.21.15.97:0/909144760 learned_addr learned my addr 172.21.15.97:0/909144760 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:23:16.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:16.861+0000 7fa1ff7fe700 1 --2- 172.21.15.97:0/909144760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa200102ba0 0x7fa20019d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:16.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:16.861+0000 7fa204d46700 1 --2- 172.21.15.97:0/909144760 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa200103540 0x7fa20019ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:16.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:16.861+0000 7fa1fffff700 1 -- 172.21.15.97:0/909144760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa200102ba0 msgr2=0x7fa20019d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:16.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:16.861+0000 7fa1fffff700 1 --2- 172.21.15.97:0/909144760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa200102ba0 0x7fa20019d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:16.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:16.861+0000 7fa1fffff700 1 -- 172.21.15.97:0/909144760 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa200103540 msgr2=0x7fa20019ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:16.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:16.861+0000 7fa1fffff700 1 --2- 172.21.15.97:0/909144760 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa200103540 0x7fa20019ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:16.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:16.861+0000 7fa1fffff700 1 -- 172.21.15.97:0/909144760 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2001a0840 con 0x7fa2001019a0 2024-04-23T10:23:16.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:16.861+0000 7fa204d46700 1 --2- 172.21.15.97:0/909144760 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa200103540 0x7fa20019ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:23:16.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:16.861+0000 7fa1fffff700 1 --2- 172.21.15.97:0/909144760 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2001019a0 0x7fa20006b2a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa1f000c8f0 tx=0x7fa1f000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:23:16.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:16.861+0000 7fa1fd7fa700 1 -- 172.21.15.97:0/909144760 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f0004220 con 0x7fa2001019a0 2024-04-23T10:23:16.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:16.861+0000 7fa1fd7fa700 1 -- 172.21.15.97:0/909144760 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1f0007db0 con 0x7fa2001019a0 2024-04-23T10:23:16.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:16.861+0000 7fa2067a9700 1 -- 172.21.15.97:0/909144760 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2001a0b30 con 0x7fa2001019a0 2024-04-23T10:23:16.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:16.861+0000 7fa1fd7fa700 1 -- 172.21.15.97:0/909144760 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f0003d10 con 0x7fa2001019a0 2024-04-23T10:23:16.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:16.861+0000 7fa2067a9700 1 -- 172.21.15.97:0/909144760 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2001a1070 con 0x7fa2001019a0 2024-04-23T10:23:16.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:16.863+0000 7fa2067a9700 1 -- 172.21.15.97:0/909144760 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa200069730 con 0x7fa2001019a0 2024-04-23T10:23:16.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:16.864+0000 7fa1fd7fa700 1 -- 172.21.15.97:0/909144760 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa1f0007900 con 0x7fa2001019a0 2024-04-23T10:23:16.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:16.864+0000 7fa1fd7fa700 1 --2- 172.21.15.97:0/909144760 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fa1e80a3cc0 0x7fa1e80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:16.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:16.864+0000 7fa1ff7fe700 1 --2- 172.21.15.97:0/909144760 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fa1e80a3cc0 0x7fa1e80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:16.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:16.864+0000 7fa1fd7fa700 1 -- 172.21.15.97:0/909144760 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fa1f00cfb90 con 0x7fa2001019a0 2024-04-23T10:23:16.867 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:16.865+0000 7fa1ff7fe700 1 --2- 172.21.15.97:0/909144760 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fa1e80a3cc0 0x7fa1e80a6180 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa20006b800 tx=0x7fa1f4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:23:16.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:16.869+0000 7fa1fd7fa700 1 -- 172.21.15.97:0/909144760 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa1f0014030 con 0x7fa2001019a0 2024-04-23T10:23:17.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:17.065+0000 7fa2067a9700 1 -- 172.21.15.97:0/909144760 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fa20002e040 con 0x7fa2001019a0 2024-04-23T10:23:17.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:17.065+0000 7fa1fd7fa700 1 -- 172.21.15.97:0/909144760 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v82) ==== 74+0+133 (secure 0 0 0) 0x7fa1f009ca10 con 0x7fa2001019a0 2024-04-23T10:23:17.064 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:23:17.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:17.067+0000 7fa2067a9700 1 -- 172.21.15.97:0/909144760 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fa1e80a3cc0 msgr2=0x7fa1e80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:17.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:17.067+0000 7fa2067a9700 1 --2- 172.21.15.97:0/909144760 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fa1e80a3cc0 0x7fa1e80a6180 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa20006b800 tx=0x7fa1f4009260 comp rx=0 tx=0).stop 2024-04-23T10:23:17.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:17.067+0000 7fa2067a9700 1 -- 172.21.15.97:0/909144760 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2001019a0 msgr2=0x7fa20006b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:17.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:17.067+0000 7fa2067a9700 1 --2- 172.21.15.97:0/909144760 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2001019a0 0x7fa20006b2a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa1f000c8f0 tx=0x7fa1f000ccb0 comp rx=0 tx=0).stop 2024-04-23T10:23:17.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:17.068+0000 7fa2067a9700 1 -- 172.21.15.97:0/909144760 shutdown_connections 2024-04-23T10:23:17.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:17.068+0000 7fa2067a9700 1 --2- 172.21.15.97:0/909144760 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fa1e80a3cc0 0x7fa1e80a6180 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:17.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:17.068+0000 7fa2067a9700 1 --2- 172.21.15.97:0/909144760 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa200103540 0x7fa20019ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:17.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:17.068+0000 7fa2067a9700 1 --2- 172.21.15.97:0/909144760 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2001019a0 0x7fa20006b2a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:17.067 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:17.068+0000 7fa2067a9700 1 --2- 172.21.15.97:0/909144760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa200102ba0 0x7fa20019d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:17.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:17.068+0000 7fa2067a9700 1 -- 172.21.15.97:0/909144760 >> 172.21.15.97:0/909144760 conn(0x7fa2000fcf40 msgr2=0x7fa200105fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:23:17.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:17.068+0000 7fa2067a9700 1 -- 172.21.15.97:0/909144760 shutdown_connections 2024-04-23T10:23:17.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:17.068+0000 7fa2067a9700 1 -- 172.21.15.97:0/909144760 wait complete. 2024-04-23T10:23:17.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:17 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/909144760' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-23T10:23:17.476 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:17 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/909144760' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-23T10:23:17.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:17 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/909144760' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-23T10:23:17.767 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":82,"num_osds":12,"num_up_osds":12,"osd_up_since":1713867793,"num_in_osds":12,"osd_in_since":1713867772,"num_remapped_pgs":0} 2024-04-23T10:23:17.767 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-23T10:23:18.007 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T10:23:18.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:18 smithi183 ceph-mon[32317]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:23:18.456 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:18 smithi097 ceph-mon[29197]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:23:18.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:18 smithi196 ceph-mon[32919]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:23:19.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:19.658+0000 7f99ea5a5700 1 -- 172.21.15.97:0/1568122064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99e410f440 msgr2=0x7f99e4111830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:19.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:19.658+0000 7f99ea5a5700 1 --2- 172.21.15.97:0/1568122064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99e410f440 0x7f99e4111830 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f99dc00b0a0 tx=0x7f99dc00b3b0 comp rx=0 tx=0).stop 2024-04-23T10:23:19.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:19.658+0000 7f99ea5a5700 1 -- 172.21.15.97:0/1568122064 shutdown_connections 2024-04-23T10:23:19.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:19.658+0000 7f99ea5a5700 1 --2- 172.21.15.97:0/1568122064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99e410f440 0x7f99e4111830 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:19.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:19.658+0000 7f99ea5a5700 1 --2- 172.21.15.97:0/1568122064 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99e4074c70 0x7f99e4075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:19.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:19.658+0000 7f99ea5a5700 1 --2- 172.21.15.97:0/1568122064 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99e4075590 0x7f99e410ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:19.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:19.658+0000 7f99ea5a5700 1 -- 172.21.15.97:0/1568122064 >> 172.21.15.97:0/1568122064 conn(0x7f99e4079ba0 msgr2=0x7f99e4079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:23:19.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:19.658+0000 7f99ea5a5700 1 -- 172.21.15.97:0/1568122064 shutdown_connections 2024-04-23T10:23:19.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:19.659+0000 7f99ea5a5700 1 -- 172.21.15.97:0/1568122064 wait complete. 2024-04-23T10:23:19.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:19.659+0000 7f99ea5a5700 1 Processor -- start 2024-04-23T10:23:19.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:19.659+0000 7f99ea5a5700 1 -- start start 2024-04-23T10:23:19.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:19.660+0000 7f99ea5a5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99e4074c70 0x7f99e419b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:19.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:19.660+0000 7f99ea5a5700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99e4075590 0x7f99e419b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:19.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:19.660+0000 7f99ea5a5700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99e410f440 0x7f99e419fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:19.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:19.660+0000 7f99ea5a5700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f99e4114dc0 con 0x7f99e4074c70 2024-04-23T10:23:19.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:19.660+0000 7f99ea5a5700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f99e4114c40 con 0x7f99e4075590 2024-04-23T10:23:19.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:19.660+0000 7f99ea5a5700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f99e4114f40 con 0x7f99e410f440 2024-04-23T10:23:19.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:19.660+0000 7f99e3fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99e4074c70 0x7f99e419b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:19.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:19.660+0000 7f99e3fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99e4074c70 0x7f99e419b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45328/0 (socket says 172.21.15.97:45328) 2024-04-23T10:23:19.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:19.660+0000 7f99e3fff700 1 -- 172.21.15.97:0/307185830 learned_addr learned my addr 172.21.15.97:0/307185830 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:23:19.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:19.660+0000 7f99e37fe700 1 --2- 172.21.15.97:0/307185830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99e4075590 0x7f99e419b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:19.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:19.660+0000 7f99e8b42700 1 --2- 172.21.15.97:0/307185830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99e410f440 0x7f99e419fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:19.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:19.660+0000 7f99e3fff700 1 -- 172.21.15.97:0/307185830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99e410f440 msgr2=0x7f99e419fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:19.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:19.660+0000 7f99e3fff700 1 --2- 172.21.15.97:0/307185830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99e410f440 0x7f99e419fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:19.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:19.660+0000 7f99e3fff700 1 -- 172.21.15.97:0/307185830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99e4075590 msgr2=0x7f99e419b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:19.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:19.660+0000 7f99e3fff700 1 --2- 172.21.15.97:0/307185830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99e4075590 0x7f99e419b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:19.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:19.660+0000 7f99e3fff700 1 -- 172.21.15.97:0/307185830 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99e41a02f0 con 0x7f99e4074c70 2024-04-23T10:23:19.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:19.660+0000 7f99e8b42700 1 --2- 172.21.15.97:0/307185830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99e410f440 0x7f99e419fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:23:19.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:19.660+0000 7f99e3fff700 1 --2- 172.21.15.97:0/307185830 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99e4074c70 0x7f99e419b330 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f99d400d950 tx=0x7f99d400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:23:19.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:19.660+0000 7f99e17fa700 1 -- 172.21.15.97:0/307185830 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d4016070 con 0x7f99e4074c70 2024-04-23T10:23:19.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:19.660+0000 7f99e17fa700 1 -- 172.21.15.97:0/307185830 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99d4009df0 con 0x7f99e4074c70 2024-04-23T10:23:19.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:19.660+0000 7f99ea5a5700 1 -- 172.21.15.97:0/307185830 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99e41a05e0 con 0x7f99e4074c70 2024-04-23T10:23:19.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:19.660+0000 7f99e17fa700 1 -- 172.21.15.97:0/307185830 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d400f3c0 con 0x7f99e4074c70 2024-04-23T10:23:19.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:19.660+0000 7f99ea5a5700 1 -- 172.21.15.97:0/307185830 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99e41a7e60 con 0x7f99e4074c70 2024-04-23T10:23:19.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:19.661+0000 7f99ea5a5700 1 -- 172.21.15.97:0/307185830 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99e4069730 con 0x7f99e4074c70 2024-04-23T10:23:19.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:19.662+0000 7f99e17fa700 1 -- 172.21.15.97:0/307185830 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f99d4019070 con 0x7f99e4074c70 2024-04-23T10:23:19.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:19.663+0000 7f99e17fa700 1 --2- 172.21.15.97:0/307185830 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f99cc0a3c40 0x7f99cc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:19.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:19.663+0000 7f99e37fe700 1 --2- 172.21.15.97:0/307185830 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f99cc0a3c40 0x7f99cc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:19.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:19.663+0000 7f99e17fa700 1 -- 172.21.15.97:0/307185830 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f99d40cfbb0 con 0x7f99e4074c70 2024-04-23T10:23:19.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:19.664+0000 7f99e37fe700 1 --2- 172.21.15.97:0/307185830 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f99cc0a3c40 0x7f99cc0a6100 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f99e41111d0 tx=0x7f99d800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:23:19.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:19.667+0000 7f99e17fa700 1 -- 172.21.15.97:0/307185830 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f99d409c990 con 0x7f99e4074c70 2024-04-23T10:23:19.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:19.865+0000 7f99ea5a5700 1 -- 172.21.15.97:0/307185830 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f99e402df80 con 0x7f99e4074c70 2024-04-23T10:23:19.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:19.866+0000 7f99e17fa700 1 -- 172.21.15.97:0/307185830 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18457 (secure 0 0 0) 0x7f99d409c290 con 0x7f99e4074c70 2024-04-23T10:23:19.866 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:23:19.866 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":82,"fsid":"9b22af88-0159-11ef-bc93-c7b262605968","created":"2024-04-23T10:16:05.983305+0000","modified":"2024-04-23T10:23:14.115959+0000","last_up_change":"2024-04-23T10:23:13.114761+0000","last_in_change":"2024-04-23T10:22:52.804880+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T10:19:25.559138+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"58cb995d-de35-4522-ac81-4982401b260d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6803","nonce":3031221802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6805","nonce":3031221802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6809","nonce":3031221802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6807","nonce":3031221802}]},"public_addr":"172.21.15.97:6803/3031221802","cluster_addr":"172.21.15.97:6805/3031221802","heartbeat_back_addr":"172.21.15.97:6809/3031221802","heartbeat_front_addr":"172.21.15.97:6807/3031221802","state":["exists","up"]},{"osd":1,"uuid":"b47bd8ee-1f21-4834-acf5-bb40f97ab3ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6801","nonce":1108870440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6803","nonce":1108870440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6807","nonce":1108870440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6805","nonce":1108870440}]},"public_addr":"172.21.15.183:6801/1108870440","cluster_addr":"172.21.15.183:6803/1108870440","heartbeat_back_addr":"172.21.15.183:6807/1108870440","heartbeat_front_addr":"172.21.15.183:6805/1108870440","state":["exists","up"]},{"osd":2,"uuid":"17c4eec3-6c47-4334-9b63-1d5ad9ec5988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6801","nonce":2877039731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6803","nonce":2877039731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6807","nonce":2877039731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6805","nonce":2877039731}]},"public_addr":"172.21.15.196:6801/2877039731","cluster_addr":"172.21.15.196:6803/2877039731","heartbeat_back_addr":"172.21.15.196:6807/2877039731","heartbeat_front_addr":"172.21.15.196:6805/2877039731","state":["exists","up"]},{"osd":3,"uuid":"3f731c38-2e45-4ca1-8dce-efc4a1b2166b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6811","nonce":4276311194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6813","nonce":4276311194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6817","nonce":4276311194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6815","nonce":4276311194}]},"public_addr":"172.21.15.97:6811/4276311194","cluster_addr":"172.21.15.97:6813/4276311194","heartbeat_back_addr":"172.21.15.97:6817/4276311194","heartbeat_front_addr":"172.21.15.97:6815/4276311194","state":["exists","up"]},{"osd":4,"uuid":"fdcafb14-9b82-4134-bd5b-79c0516f6cbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6809","nonce":63913916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6811","nonce":63913916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6815","nonce":63913916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6813","nonce":63913916}]},"public_addr":"172.21.15.183:6809/63913916","cluster_addr":"172.21.15.183:6811/63913916","heartbeat_back_addr":"172.21.15.183:6815/63913916","heartbeat_front_addr":"172.21.15.183:6813/63913916","state":["exists","up"]},{"osd":5,"uuid":"cdcc18d8-da3c-4012-bf12-876bb39e66eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6809","nonce":2804446896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6811","nonce":2804446896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6815","nonce":2804446896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6813","nonce":2804446896}]},"public_addr":"172.21.15.196:6809/2804446896","cluster_addr":"172.21.15.196:6811/2804446896","heartbeat_back_addr":"172.21.15.196:6815/2804446896","heartbeat_front_addr":"172.21.15.196:6813/2804446896","state":["exists","up"]},{"osd":6,"uuid":"071abb5c-2edf-4832-b8aa-3b80d20096fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6819","nonce":2080916705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6821","nonce":2080916705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6825","nonce":2080916705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6823","nonce":2080916705}]},"public_addr":"172.21.15.97:6819/2080916705","cluster_addr":"172.21.15.97:6821/2080916705","heartbeat_back_addr":"172.21.15.97:6825/2080916705","heartbeat_front_addr":"172.21.15.97:6823/2080916705","state":["exists","up"]},{"osd":7,"uuid":"9d54760b-66df-486b-8abb-017f3b64b4d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6817","nonce":1865873033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6819","nonce":1865873033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6823","nonce":1865873033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6821","nonce":1865873033}]},"public_addr":"172.21.15.183:6817/1865873033","cluster_addr":"172.21.15.183:6819/1865873033","heartbeat_back_addr":"172.21.15.183:6823/1865873033","heartbeat_front_addr":"172.21.15.183:6821/1865873033","state":["exists","up"]},{"osd":8,"uuid":"5cd3100a-bd6a-4be5-b5e9-0d0afc325193","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6817","nonce":4063932139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6819","nonce":4063932139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6823","nonce":4063932139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6821","nonce":4063932139}]},"public_addr":"172.21.15.196:6817/4063932139","cluster_addr":"172.21.15.196:6819/4063932139","heartbeat_back_addr":"172.21.15.196:6823/4063932139","heartbeat_front_addr":"172.21.15.196:6821/4063932139","state":["exists","up"]},{"osd":9,"uuid":"b6a5f4a1-4ecc-4f73-a2df-07551d9298ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6827","nonce":1887432474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6829","nonce":1887432474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6833","nonce":1887432474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6831","nonce":1887432474}]},"public_addr":"172.21.15.97:6827/1887432474","cluster_addr":"172.21.15.97:6829/1887432474","heartbeat_back_addr":"172.21.15.97:6833/1887432474","heartbeat_front_addr":"172.21.15.97:6831/1887432474","state":["exists","up"]},{"osd":10,"uuid":"86f59816-1ea6-4ae9-8796-cf5fc4831064","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6825","nonce":799434647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6827","nonce":799434647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6831","nonce":799434647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6829","nonce":799434647}]},"public_addr":"172.21.15.183:6825/799434647","cluster_addr":"172.21.15.183:6827/799434647","heartbeat_back_addr":"172.21.15.183:6831/799434647","heartbeat_front_addr":"172.21.15.183:6829/799434647","state":["exists","up"]},{"osd":11,"uuid":"95d64a89-a9f8-4c7a-8c0d-da7fb26cfff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6825","nonce":1873969467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6827","nonce":1873969467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6831","nonce":1873969467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6829","nonce":1873969467}]},"public_addr":"172.21.15.196:6825/1873969467","cluster_addr":"172.21.15.196:6827/1873969467","heartbeat_back_addr":"172.21.15.196:6831/1873969467","heartbeat_front_addr":"172.21.15.196:6829/1873969467","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:18:33.721156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:18:57.586709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:19:22.069458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:19:46.451597+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:20:11.561019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:20:36.298672+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:21:01.661804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:21:27.324128+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:21:52.898197+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:22:19.614326+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:22:45.154917+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:23:11.222612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/3866016040":"2024-04-24T10:16:59.468237+0000","172.21.15.97:6801/3297166858":"2024-04-24T10:16:59.468237+0000","172.21.15.97:0/3804743548":"2024-04-24T10:16:59.468237+0000","172.21.15.97:6800/3706463017":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/3715079438":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/786806834":"2024-04-24T10:16:59.468237+0000","172.21.15.97:6801/3706463017":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/3294175554":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/1449136760":"2024-04-24T10:16:33.128917+0000","172.21.15.97:6800/3297166858":"2024-04-24T10:16:59.468237+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T10:23:19.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:19.869+0000 7f99ea5a5700 1 -- 172.21.15.97:0/307185830 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f99cc0a3c40 msgr2=0x7f99cc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:19.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:19.869+0000 7f99ea5a5700 1 --2- 172.21.15.97:0/307185830 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f99cc0a3c40 0x7f99cc0a6100 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f99e41111d0 tx=0x7f99d800b410 comp rx=0 tx=0).stop 2024-04-23T10:23:19.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:19.869+0000 7f99ea5a5700 1 -- 172.21.15.97:0/307185830 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99e4074c70 msgr2=0x7f99e419b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:19.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:19.869+0000 7f99ea5a5700 1 --2- 172.21.15.97:0/307185830 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99e4074c70 0x7f99e419b330 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f99d400d950 tx=0x7f99d400dc60 comp rx=0 tx=0).stop 2024-04-23T10:23:19.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:19.869+0000 7f99ea5a5700 1 -- 172.21.15.97:0/307185830 shutdown_connections 2024-04-23T10:23:19.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:19.869+0000 7f99ea5a5700 1 --2- 172.21.15.97:0/307185830 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f99cc0a3c40 0x7f99cc0a6100 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:19.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:19.869+0000 7f99ea5a5700 1 --2- 172.21.15.97:0/307185830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99e410f440 0x7f99e419fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:19.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:19.869+0000 7f99ea5a5700 1 --2- 172.21.15.97:0/307185830 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f99e4074c70 0x7f99e419b330 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:19.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:19.869+0000 7f99ea5a5700 1 --2- 172.21.15.97:0/307185830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99e4075590 0x7f99e419b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:19.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:19.869+0000 7f99ea5a5700 1 -- 172.21.15.97:0/307185830 >> 172.21.15.97:0/307185830 conn(0x7f99e4079ba0 msgr2=0x7f99e40ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:23:19.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:19.869+0000 7f99ea5a5700 1 -- 172.21.15.97:0/307185830 shutdown_connections 2024-04-23T10:23:19.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:19.869+0000 7f99ea5a5700 1 -- 172.21.15.97:0/307185830 wait complete. 2024-04-23T10:23:20.652 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T10:19:25.559138+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T10:23:20.652 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-23T10:23:20.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:20 smithi097 ceph-mon[29197]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:23:20.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:20 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/307185830' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T10:23:20.890 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T10:23:20.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:20 smithi183 ceph-mon[32317]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:23:20.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:20 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/307185830' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T10:23:21.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:20 smithi196 ceph-mon[32919]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:23:21.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:20 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/307185830' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T10:23:22.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:22.658+0000 7ffbacfa8700 1 -- 172.21.15.97:0/3711379689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffba8101970 msgr2=0x7ffba8101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:22.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:22.658+0000 7ffbacfa8700 1 --2- 172.21.15.97:0/3711379689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffba8101970 0x7ffba8101df0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c009990 tx=0x7ffb9c009ca0 comp rx=0 tx=0).stop 2024-04-23T10:23:22.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:22.658+0000 7ffbacfa8700 1 -- 172.21.15.97:0/3711379689 shutdown_connections 2024-04-23T10:23:22.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:22.658+0000 7ffbacfa8700 1 --2- 172.21.15.97:0/3711379689 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffba8102330 0x7ffba810e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:22.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:22.658+0000 7ffbacfa8700 1 --2- 172.21.15.97:0/3711379689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffba8103760 0x7ffba8103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:22.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:22.658+0000 7ffbacfa8700 1 --2- 172.21.15.97:0/3711379689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffba8101970 0x7ffba8101df0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:22.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:22.658+0000 7ffbacfa8700 1 -- 172.21.15.97:0/3711379689 >> 172.21.15.97:0/3711379689 conn(0x7ffba80779b0 msgr2=0x7ffba8077dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:23:22.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:22.658+0000 7ffbacfa8700 1 -- 172.21.15.97:0/3711379689 shutdown_connections 2024-04-23T10:23:22.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:22.658+0000 7ffbacfa8700 1 -- 172.21.15.97:0/3711379689 wait complete. 2024-04-23T10:23:22.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:22.659+0000 7ffbacfa8700 1 Processor -- start 2024-04-23T10:23:22.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:22.659+0000 7ffbacfa8700 1 -- start start 2024-04-23T10:23:22.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:22.659+0000 7ffbacfa8700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffba8101970 0x7ffba819a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:22.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:22.659+0000 7ffbacfa8700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffba8102330 0x7ffba819a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:22.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:22.659+0000 7ffbacfa8700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffba8103760 0x7ffba81a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:22.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:22.659+0000 7ffbacfa8700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ffba8111be0 con 0x7ffba8102330 2024-04-23T10:23:22.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:22.659+0000 7ffbacfa8700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ffba8111a60 con 0x7ffba8103760 2024-04-23T10:23:22.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:22.659+0000 7ffbacfa8700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ffba8111d60 con 0x7ffba8101970 2024-04-23T10:23:22.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:22.659+0000 7ffba5d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffba8102330 0x7ffba819a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:22.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:22.659+0000 7ffba5d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffba8102330 0x7ffba819a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45344/0 (socket says 172.21.15.97:45344) 2024-04-23T10:23:22.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:22.659+0000 7ffba5d9b700 1 -- 172.21.15.97:0/1080096066 learned_addr learned my addr 172.21.15.97:0/1080096066 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:23:22.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:22.659+0000 7ffba659c700 1 --2- 172.21.15.97:0/1080096066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffba8101970 0x7ffba819a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:22.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:22.659+0000 7ffba6d9d700 1 --2- 172.21.15.97:0/1080096066 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffba8103760 0x7ffba81a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:22.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:22.659+0000 7ffba5d9b700 1 -- 172.21.15.97:0/1080096066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffba8101970 msgr2=0x7ffba819a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:22.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:22.659+0000 7ffba5d9b700 1 --2- 172.21.15.97:0/1080096066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffba8101970 0x7ffba819a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:22.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:22.660+0000 7ffba5d9b700 1 -- 172.21.15.97:0/1080096066 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffba8103760 msgr2=0x7ffba81a50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:22.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:22.660+0000 7ffba5d9b700 1 --2- 172.21.15.97:0/1080096066 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffba8103760 0x7ffba81a50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:22.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:22.660+0000 7ffba5d9b700 1 -- 172.21.15.97:0/1080096066 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffba81a5620 con 0x7ffba8102330 2024-04-23T10:23:22.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:22.660+0000 7ffba659c700 1 --2- 172.21.15.97:0/1080096066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffba8101970 0x7ffba819a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:23:22.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:22.660+0000 7ffba6d9d700 1 --2- 172.21.15.97:0/1080096066 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffba8103760 0x7ffba81a50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:23:22.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:22.660+0000 7ffba5d9b700 1 --2- 172.21.15.97:0/1080096066 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffba8102330 0x7ffba819a5a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c000c00 tx=0x7ffb9c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:23:22.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:22.660+0000 7ffb977fe700 1 -- 172.21.15.97:0/1080096066 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c027070 con 0x7ffba8102330 2024-04-23T10:23:22.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:22.660+0000 7ffbacfa8700 1 -- 172.21.15.97:0/1080096066 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffba81a57f0 con 0x7ffba8102330 2024-04-23T10:23:22.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:22.660+0000 7ffb977fe700 1 -- 172.21.15.97:0/1080096066 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb9c003fa0 con 0x7ffba8102330 2024-04-23T10:23:22.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:22.660+0000 7ffb977fe700 1 -- 172.21.15.97:0/1080096066 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c022db0 con 0x7ffba8102330 2024-04-23T10:23:22.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:22.660+0000 7ffbacfa8700 1 -- 172.21.15.97:0/1080096066 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffba81a5ca0 con 0x7ffba8102330 2024-04-23T10:23:22.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:22.661+0000 7ffbacfa8700 1 -- 172.21.15.97:0/1080096066 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffba8069730 con 0x7ffba8102330 2024-04-23T10:23:22.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:22.662+0000 7ffb977fe700 1 -- 172.21.15.97:0/1080096066 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ffb9c02a070 con 0x7ffba8102330 2024-04-23T10:23:22.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:22.662+0000 7ffb977fe700 1 --2- 172.21.15.97:0/1080096066 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7ffb900a3c40 0x7ffb900a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:22.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:22.662+0000 7ffba659c700 1 --2- 172.21.15.97:0/1080096066 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7ffb900a3c40 0x7ffb900a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:22.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:22.663+0000 7ffb977fe700 1 -- 172.21.15.97:0/1080096066 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7ffb9c0d0c50 con 0x7ffba8102330 2024-04-23T10:23:22.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:22.663+0000 7ffba659c700 1 --2- 172.21.15.97:0/1080096066 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7ffb900a3c40 0x7ffb900a6100 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ffb98005fd0 tx=0x7ffb98005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:23:22.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:22.667+0000 7ffb977fe700 1 -- 172.21.15.97:0/1080096066 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffb9c0d6020 con 0x7ffba8102330 2024-04-23T10:23:22.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:22 smithi196 ceph-mon[32919]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:23:22.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:22 smithi097 ceph-mon[29197]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:23:22.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:22.861+0000 7ffbacfa8700 1 -- 172.21.15.97:0/1080096066 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ffba802df80 con 0x7ffba8102330 2024-04-23T10:23:22.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:22.862+0000 7ffb977fe700 1 -- 172.21.15.97:0/1080096066 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7ffb9c013070 con 0x7ffba8102330 2024-04-23T10:23:22.861 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-23T10:23:22.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:22.864+0000 7ffbacfa8700 1 -- 172.21.15.97:0/1080096066 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7ffb900a3c40 msgr2=0x7ffb900a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:22.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:22.864+0000 7ffbacfa8700 1 --2- 172.21.15.97:0/1080096066 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7ffb900a3c40 0x7ffb900a6100 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ffb98005fd0 tx=0x7ffb98005db0 comp rx=0 tx=0).stop 2024-04-23T10:23:22.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:22.864+0000 7ffbacfa8700 1 -- 172.21.15.97:0/1080096066 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffba8102330 msgr2=0x7ffba819a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:22.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:22.864+0000 7ffbacfa8700 1 --2- 172.21.15.97:0/1080096066 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffba8102330 0x7ffba819a5a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c000c00 tx=0x7ffb9c0047b0 comp rx=0 tx=0).stop 2024-04-23T10:23:22.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:22.864+0000 7ffbacfa8700 1 -- 172.21.15.97:0/1080096066 shutdown_connections 2024-04-23T10:23:22.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:22.864+0000 7ffbacfa8700 1 --2- 172.21.15.97:0/1080096066 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7ffb900a3c40 0x7ffb900a6100 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:22.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:22.864+0000 7ffbacfa8700 1 --2- 172.21.15.97:0/1080096066 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffba8103760 0x7ffba81a50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:22.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:22.864+0000 7ffbacfa8700 1 --2- 172.21.15.97:0/1080096066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffba8101970 0x7ffba819a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:22.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:22.864+0000 7ffbacfa8700 1 --2- 172.21.15.97:0/1080096066 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffba8102330 0x7ffba819a5a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:22.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:22.864+0000 7ffbacfa8700 1 -- 172.21.15.97:0/1080096066 >> 172.21.15.97:0/1080096066 conn(0x7ffba80779b0 msgr2=0x7ffba8106770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:23:22.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:22.865+0000 7ffbacfa8700 1 -- 172.21.15.97:0/1080096066 shutdown_connections 2024-04-23T10:23:22.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:22.865+0000 7ffbacfa8700 1 -- 172.21.15.97:0/1080096066 wait complete. 2024-04-23T10:23:22.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:22 smithi183 ceph-mon[32317]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:23:23.625 INFO:tasks.cephadm:Adding mds.a on smithi097 2024-04-23T10:23:23.625 INFO:tasks.cephadm:Adding mds.d on smithi097 2024-04-23T10:23:23.625 INFO:tasks.cephadm:Adding mds.g on smithi097 2024-04-23T10:23:23.625 INFO:tasks.cephadm:Adding mds.j on smithi097 2024-04-23T10:23:23.625 INFO:tasks.cephadm:Adding mds.b on smithi183 2024-04-23T10:23:23.625 INFO:tasks.cephadm:Adding mds.e on smithi183 2024-04-23T10:23:23.625 INFO:tasks.cephadm:Adding mds.h on smithi183 2024-04-23T10:23:23.625 INFO:tasks.cephadm:Adding mds.k on smithi183 2024-04-23T10:23:23.625 INFO:tasks.cephadm:Adding mds.c on smithi196 2024-04-23T10:23:23.625 INFO:tasks.cephadm:Adding mds.f on smithi196 2024-04-23T10:23:23.626 INFO:tasks.cephadm:Adding mds.i on smithi196 2024-04-23T10:23:23.626 INFO:tasks.cephadm:Adding mds.l on smithi196 2024-04-23T10:23:23.626 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi097=a;smithi097=d;smithi097=g;smithi097=j;smithi183=b;smithi183=e;smithi183=h;smithi183=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l' 2024-04-23T10:23:23.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:23 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/1080096066' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T10:23:23.877 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T10:23:23.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:23 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/1080096066' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T10:23:23.999 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:23 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/1080096066' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T10:23:24.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:24 smithi097 ceph-mon[29197]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:23:24.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:24 smithi183 ceph-mon[32317]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:23:25.037 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:24 smithi196 ceph-mon[32919]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:23:25.588 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:23:25.589+0000 7f4b0ce4b700 1 -- 172.21.15.196:0/1777533678 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b08101560 msgr2=0x7f4b0810b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:25.589 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:23:25.589+0000 7f4b0ce4b700 1 --2- 172.21.15.196:0/1777533678 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b08101560 0x7f4b0810b130 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4b0000b0a0 tx=0x7f4b0000b3b0 comp rx=0 tx=0).stop 2024-04-23T10:23:25.589 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:23:25.589+0000 7f4b0ce4b700 1 -- 172.21.15.196:0/1777533678 shutdown_connections 2024-04-23T10:23:25.589 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:23:25.589+0000 7f4b0ce4b700 1 --2- 172.21.15.196:0/1777533678 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b08101560 0x7f4b0810b130 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:25.589 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:23:25.589+0000 7f4b0ce4b700 1 --2- 172.21.15.196:0/1777533678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b08104140 0x7f4b08104560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:25.589 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:23:25.589+0000 7f4b0ce4b700 1 --2- 172.21.15.196:0/1777533678 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b08100bc0 0x7f4b08101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:25.589 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:23:25.589+0000 7f4b0ce4b700 1 -- 172.21.15.196:0/1777533678 >> 172.21.15.196:0/1777533678 conn(0x7f4b080fc780 msgr2=0x7f4b080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:23:25.589 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:23:25.589+0000 7f4b0ce4b700 1 -- 172.21.15.196:0/1777533678 shutdown_connections 2024-04-23T10:23:25.589 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:23:25.589+0000 7f4b0ce4b700 1 -- 172.21.15.196:0/1777533678 wait complete. 2024-04-23T10:23:25.591 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:23:25.590+0000 7f4b0ce4b700 1 Processor -- start 2024-04-23T10:23:25.591 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:23:25.590+0000 7f4b0ce4b700 1 -- start start 2024-04-23T10:23:25.591 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:23:25.590+0000 7f4b0ce4b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b08100bc0 0x7f4b08198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:25.591 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:23:25.590+0000 7f4b0ce4b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b08101560 0x7f4b08199060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:25.591 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:23:25.590+0000 7f4b0ce4b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b08104140 0x7f4b081a00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:25.592 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:23:25.590+0000 7f4b0ce4b700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4b0810e540 con 0x7f4b08100bc0 2024-04-23T10:23:25.592 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:23:25.590+0000 7f4b0ce4b700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4b0810e3c0 con 0x7f4b08104140 2024-04-23T10:23:25.592 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:23:25.590+0000 7f4b0ce4b700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4b0810e6c0 con 0x7f4b08101560 2024-04-23T10:23:25.592 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:23:25.590+0000 7f4b05d9b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b08101560 0x7f4b08199060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:25.592 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:23:25.590+0000 7f4b05d9b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b08101560 0x7f4b08199060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:47742/0 (socket says 172.21.15.196:47742) 2024-04-23T10:23:25.592 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:23:25.590+0000 7f4b05d9b700 1 -- 172.21.15.196:0/308448701 learned_addr learned my addr 172.21.15.196:0/308448701 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-23T10:23:25.592 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:23:25.591+0000 7f4b06d9d700 1 --2- 172.21.15.196:0/308448701 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b08104140 0x7f4b081a00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:25.592 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:23:25.591+0000 7f4b0659c700 1 --2- 172.21.15.196:0/308448701 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b08100bc0 0x7f4b08198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:25.592 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:23:25.591+0000 7f4b05d9b700 1 -- 172.21.15.196:0/308448701 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b08104140 msgr2=0x7f4b081a00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:25.592 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:23:25.591+0000 7f4b05d9b700 1 --2- 172.21.15.196:0/308448701 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b08104140 0x7f4b081a00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:25.592 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:23:25.591+0000 7f4b05d9b700 1 -- 172.21.15.196:0/308448701 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b08100bc0 msgr2=0x7f4b08198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:25.592 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:23:25.591+0000 7f4b05d9b700 1 --2- 172.21.15.196:0/308448701 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b08100bc0 0x7f4b08198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:25.593 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:23:25.591+0000 7f4b05d9b700 1 -- 172.21.15.196:0/308448701 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b081a0780 con 0x7f4b08101560 2024-04-23T10:23:25.593 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:23:25.591+0000 7f4b05d9b700 1 --2- 172.21.15.196:0/308448701 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b08101560 0x7f4b08199060 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4af800e9d0 tx=0x7f4af800ece0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:23:25.593 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:23:25.591+0000 7f4af77fe700 1 -- 172.21.15.196:0/308448701 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af800cbd0 con 0x7f4b08101560 2024-04-23T10:23:25.593 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:23:25.591+0000 7f4b0ce4b700 1 -- 172.21.15.196:0/308448701 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b081a0a70 con 0x7f4b08101560 2024-04-23T10:23:25.593 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:23:25.591+0000 7f4af77fe700 1 -- 172.21.15.196:0/308448701 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4af800cd70 con 0x7f4b08101560 2024-04-23T10:23:25.593 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:23:25.591+0000 7f4af77fe700 1 -- 172.21.15.196:0/308448701 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af8018680 con 0x7f4b08101560 2024-04-23T10:23:25.593 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:23:25.591+0000 7f4b0ce4b700 1 -- 172.21.15.196:0/308448701 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b081a0fb0 con 0x7f4b08101560 2024-04-23T10:23:25.593 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:23:25.592+0000 7f4b0ce4b700 1 -- 172.21.15.196:0/308448701 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b08069730 con 0x7f4b08101560 2024-04-23T10:23:25.594 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:23:25.593+0000 7f4af77fe700 1 -- 172.21.15.196:0/308448701 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4af8010430 con 0x7f4b08101560 2024-04-23T10:23:25.594 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:23:25.594+0000 7f4af77fe700 1 --2- 172.21.15.196:0/308448701 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f4af00a3c40 0x7f4af00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:25.594 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:23:25.594+0000 7f4af77fe700 1 -- 172.21.15.196:0/308448701 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f4af80d01b0 con 0x7f4b08101560 2024-04-23T10:23:25.594 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:23:25.594+0000 7f4b0659c700 1 --2- 172.21.15.196:0/308448701 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f4af00a3c40 0x7f4af00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:25.594 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:23:25.594+0000 7f4b0659c700 1 --2- 172.21.15.196:0/308448701 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f4af00a3c40 0x7f4af00a6100 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4b0806aef0 tx=0x7f4afc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:23:25.597 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:23:25.598+0000 7f4af77fe700 1 -- 172.21.15.196:0/308448701 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4af809d030 con 0x7f4b08101560 2024-04-23T10:23:25.805 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:23:25.806+0000 7f4b0ce4b700 1 -- 172.21.15.196:0/308448701 --> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi097=a;smithi097=d;smithi097=g;smithi097=j;smithi183=b;smithi183=e;smithi183=h;smithi183=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l", "target": ["mon-mgr", ""]}) -- 0x7f4b08062f80 con 0x7f4af00a3c40 2024-04-23T10:23:25.813 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:23:25.815+0000 7f4af77fe700 1 -- 172.21.15.196:0/308448701 <== mgr.14152 v2:172.21.15.97:6800/3149043749 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f4b08062f80 con 0x7f4af00a3c40 2024-04-23T10:23:25.814 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled mds.all update... 2024-04-23T10:23:25.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:23:25.817+0000 7f4b0ce4b700 1 -- 172.21.15.196:0/308448701 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f4af00a3c40 msgr2=0x7f4af00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:25.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:23:25.817+0000 7f4b0ce4b700 1 --2- 172.21.15.196:0/308448701 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f4af00a3c40 0x7f4af00a6100 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4b0806aef0 tx=0x7f4afc009260 comp rx=0 tx=0).stop 2024-04-23T10:23:25.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:23:25.817+0000 7f4b0ce4b700 1 -- 172.21.15.196:0/308448701 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b08101560 msgr2=0x7f4b08199060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:25.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:23:25.817+0000 7f4b0ce4b700 1 --2- 172.21.15.196:0/308448701 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b08101560 0x7f4b08199060 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4af800e9d0 tx=0x7f4af800ece0 comp rx=0 tx=0).stop 2024-04-23T10:23:25.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:23:25.817+0000 7f4b0ce4b700 1 -- 172.21.15.196:0/308448701 shutdown_connections 2024-04-23T10:23:25.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:23:25.817+0000 7f4b0ce4b700 1 --2- 172.21.15.196:0/308448701 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f4af00a3c40 0x7f4af00a6100 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:25.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:23:25.817+0000 7f4b0ce4b700 1 --2- 172.21.15.196:0/308448701 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b08104140 0x7f4b081a00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:25.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:23:25.817+0000 7f4b0ce4b700 1 --2- 172.21.15.196:0/308448701 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b08100bc0 0x7f4b08198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:25.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:23:25.817+0000 7f4b0ce4b700 1 --2- 172.21.15.196:0/308448701 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b08101560 0x7f4b08199060 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:25.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:23:25.817+0000 7f4b0ce4b700 1 -- 172.21.15.196:0/308448701 >> 172.21.15.196:0/308448701 conn(0x7f4b080fc780 msgr2=0x7f4b08107180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:23:25.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:23:25.817+0000 7f4b0ce4b700 1 -- 172.21.15.196:0/308448701 shutdown_connections 2024-04-23T10:23:25.816 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-23T10:23:25.817+0000 7f4b0ce4b700 1 -- 172.21.15.196:0/308448701 wait complete. 2024-04-23T10:23:26.497 DEBUG:teuthology.orchestra.run.smithi097:mds.a> sudo journalctl -f -n 0 -u ceph-9b22af88-0159-11ef-bc93-c7b262605968@mds.a.service 2024-04-23T10:23:26.499 DEBUG:teuthology.orchestra.run.smithi097:mds.d> sudo journalctl -f -n 0 -u ceph-9b22af88-0159-11ef-bc93-c7b262605968@mds.d.service 2024-04-23T10:23:26.543 DEBUG:teuthology.orchestra.run.smithi097:mds.g> sudo journalctl -f -n 0 -u ceph-9b22af88-0159-11ef-bc93-c7b262605968@mds.g.service 2024-04-23T10:23:26.586 DEBUG:teuthology.orchestra.run.smithi097:mds.j> sudo journalctl -f -n 0 -u ceph-9b22af88-0159-11ef-bc93-c7b262605968@mds.j.service 2024-04-23T10:23:26.628 DEBUG:teuthology.orchestra.run.smithi183:mds.b> sudo journalctl -f -n 0 -u ceph-9b22af88-0159-11ef-bc93-c7b262605968@mds.b.service 2024-04-23T10:23:26.631 DEBUG:teuthology.orchestra.run.smithi183:mds.e> sudo journalctl -f -n 0 -u ceph-9b22af88-0159-11ef-bc93-c7b262605968@mds.e.service 2024-04-23T10:23:26.673 DEBUG:teuthology.orchestra.run.smithi183:mds.h> sudo journalctl -f -n 0 -u ceph-9b22af88-0159-11ef-bc93-c7b262605968@mds.h.service 2024-04-23T10:23:26.717 DEBUG:teuthology.orchestra.run.smithi183:mds.k> sudo journalctl -f -n 0 -u ceph-9b22af88-0159-11ef-bc93-c7b262605968@mds.k.service 2024-04-23T10:23:26.759 DEBUG:teuthology.orchestra.run.smithi196:mds.c> sudo journalctl -f -n 0 -u ceph-9b22af88-0159-11ef-bc93-c7b262605968@mds.c.service 2024-04-23T10:23:26.762 DEBUG:teuthology.orchestra.run.smithi196:mds.f> sudo journalctl -f -n 0 -u ceph-9b22af88-0159-11ef-bc93-c7b262605968@mds.f.service 2024-04-23T10:23:26.775 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:26 smithi196 ceph-mon[32919]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:23:26.775 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:26 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:26.775 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:26 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:23:26.776 DEBUG:teuthology.orchestra.run.smithi196:mds.i> sudo journalctl -f -n 0 -u ceph-9b22af88-0159-11ef-bc93-c7b262605968@mds.i.service 2024-04-23T10:23:26.819 DEBUG:teuthology.orchestra.run.smithi196:mds.l> sudo journalctl -f -n 0 -u ceph-9b22af88-0159-11ef-bc93-c7b262605968@mds.l.service 2024-04-23T10:23:26.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:26 smithi097 ceph-mon[29197]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:23:26.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:26 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:26.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:26 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:23:26.862 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-23T10:23:26.863 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T10:23:26.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:26 smithi183 ceph-mon[32317]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:23:26.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:26 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:26.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:26 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:23:26.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:26.971+0000 7fa585707640 1 -- 172.21.15.97:0/2449184537 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa580102040 msgr2=0x7fa5801024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:26.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:26.971+0000 7fa585707640 1 --2- 172.21.15.97:0/2449184537 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa580102040 0x7fa5801024c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa574009900 tx=0x7fa57402a410 comp rx=0 tx=0).stop 2024-04-23T10:23:26.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:26.971+0000 7fa585707640 1 -- 172.21.15.97:0/2449184537 shutdown_connections 2024-04-23T10:23:26.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:26.971+0000 7fa585707640 1 --2- 172.21.15.97:0/2449184537 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa580102a00 0x7fa580109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:26.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:26.971+0000 7fa585707640 1 --2- 172.21.15.97:0/2449184537 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa580102040 0x7fa5801024c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:26.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:26.971+0000 7fa585707640 1 --2- 172.21.15.97:0/2449184537 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa580100e60 0x7fa580101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:26.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:26.971+0000 7fa585707640 1 -- 172.21.15.97:0/2449184537 >> 172.21.15.97:0/2449184537 conn(0x7fa5800fc440 msgr2=0x7fa5800fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:23:26.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:26.971+0000 7fa585707640 1 -- 172.21.15.97:0/2449184537 shutdown_connections 2024-04-23T10:23:26.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:26.971+0000 7fa585707640 1 -- 172.21.15.97:0/2449184537 wait complete. 2024-04-23T10:23:26.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:26.972+0000 7fa585707640 1 Processor -- start 2024-04-23T10:23:26.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:26.972+0000 7fa585707640 1 -- start start 2024-04-23T10:23:26.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:26.972+0000 7fa585707640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa580100e60 0x7fa580197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:26.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:26.972+0000 7fa585707640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa580102040 0x7fa580197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:26.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:26.972+0000 7fa585707640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa580102a00 0x7fa58019e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:26.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:26.972+0000 7fa585707640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa58010ca10 con 0x7fa580100e60 2024-04-23T10:23:26.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:26.972+0000 7fa585707640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa58010c890 con 0x7fa580102a00 2024-04-23T10:23:26.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:26.972+0000 7fa585707640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa58010cb90 con 0x7fa580102040 2024-04-23T10:23:26.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:26.972+0000 7fa57effd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa580100e60 0x7fa580197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:26.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:26.972+0000 7fa57effd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa580100e60 0x7fa580197110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55708/0 (socket says 172.21.15.97:55708) 2024-04-23T10:23:26.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:26.972+0000 7fa57effd640 1 -- 172.21.15.97:0/238741627 learned_addr learned my addr 172.21.15.97:0/238741627 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:23:26.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:26.972+0000 7fa57e7fc640 1 --2- 172.21.15.97:0/238741627 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa580102040 0x7fa580197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:26.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:26.972+0000 7fa57f7fe640 1 --2- 172.21.15.97:0/238741627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa580102a00 0x7fa58019e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:26.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:26.972+0000 7fa57effd640 1 -- 172.21.15.97:0/238741627 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa580102040 msgr2=0x7fa580197650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:26.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:26.972+0000 7fa57effd640 1 --2- 172.21.15.97:0/238741627 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa580102040 0x7fa580197650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:26.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:26.972+0000 7fa57effd640 1 -- 172.21.15.97:0/238741627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa580102a00 msgr2=0x7fa58019e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:26.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:26.972+0000 7fa57effd640 1 --2- 172.21.15.97:0/238741627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa580102a00 0x7fa58019e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:26.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:26.972+0000 7fa57effd640 1 -- 172.21.15.97:0/238741627 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa58019ed30 con 0x7fa580100e60 2024-04-23T10:23:26.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:26.972+0000 7fa57f7fe640 1 --2- 172.21.15.97:0/238741627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa580102a00 0x7fa58019e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:23:26.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:26.972+0000 7fa57effd640 1 --2- 172.21.15.97:0/238741627 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa580100e60 0x7fa580197110 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa57000d950 tx=0x7fa57000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:23:26.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:26.973+0000 7fa567fff640 1 -- 172.21.15.97:0/238741627 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa570017070 con 0x7fa580100e60 2024-04-23T10:23:26.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:26.973+0000 7fa567fff640 1 -- 172.21.15.97:0/238741627 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa570009c30 con 0x7fa580100e60 2024-04-23T10:23:26.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:26.973+0000 7fa585707640 1 -- 172.21.15.97:0/238741627 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa58019efc0 con 0x7fa580100e60 2024-04-23T10:23:26.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:26.973+0000 7fa567fff640 1 -- 172.21.15.97:0/238741627 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa57000f620 con 0x7fa580100e60 2024-04-23T10:23:26.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:26.973+0000 7fa585707640 1 -- 172.21.15.97:0/238741627 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa58019f430 con 0x7fa580100e60 2024-04-23T10:23:26.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:26.974+0000 7fa585707640 1 -- 172.21.15.97:0/238741627 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5801082d0 con 0x7fa580100e60 2024-04-23T10:23:26.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:26.975+0000 7fa567fff640 1 -- 172.21.15.97:0/238741627 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa57001a070 con 0x7fa580100e60 2024-04-23T10:23:26.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:26.975+0000 7fa567fff640 1 --2- 172.21.15.97:0/238741627 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fa5680a3bb0 0x7fa5680a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:26.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:26.975+0000 7fa57e7fc640 1 --2- 172.21.15.97:0/238741627 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fa5680a3bb0 0x7fa5680a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:26.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:26.975+0000 7fa567fff640 1 -- 172.21.15.97:0/238741627 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fa5700cf980 con 0x7fa580100e60 2024-04-23T10:23:26.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:26.976+0000 7fa57e7fc640 1 --2- 172.21.15.97:0/238741627 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fa5680a3bb0 0x7fa5680a6070 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa574000c00 tx=0x7fa5740314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:23:26.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:26.979+0000 7fa567fff640 1 -- 172.21.15.97:0/238741627 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa570009de0 con 0x7fa580100e60 2024-04-23T10:23:27.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.160+0000 7fa585707640 1 -- 172.21.15.97:0/238741627 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa58002d7b0 con 0x7fa580100e60 2024-04-23T10:23:27.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.161+0000 7fa567fff640 1 -- 172.21.15.97:0/238741627 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18457 (secure 0 0 0) 0x7fa57009c800 con 0x7fa580100e60 2024-04-23T10:23:27.161 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:23:27.161 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":82,"fsid":"9b22af88-0159-11ef-bc93-c7b262605968","created":"2024-04-23T10:16:05.983305+0000","modified":"2024-04-23T10:23:14.115959+0000","last_up_change":"2024-04-23T10:23:13.114761+0000","last_in_change":"2024-04-23T10:22:52.804880+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T10:19:25.559138+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"58cb995d-de35-4522-ac81-4982401b260d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6803","nonce":3031221802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6805","nonce":3031221802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6809","nonce":3031221802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6807","nonce":3031221802}]},"public_addr":"172.21.15.97:6803/3031221802","cluster_addr":"172.21.15.97:6805/3031221802","heartbeat_back_addr":"172.21.15.97:6809/3031221802","heartbeat_front_addr":"172.21.15.97:6807/3031221802","state":["exists","up"]},{"osd":1,"uuid":"b47bd8ee-1f21-4834-acf5-bb40f97ab3ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6801","nonce":1108870440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6803","nonce":1108870440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6807","nonce":1108870440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6805","nonce":1108870440}]},"public_addr":"172.21.15.183:6801/1108870440","cluster_addr":"172.21.15.183:6803/1108870440","heartbeat_back_addr":"172.21.15.183:6807/1108870440","heartbeat_front_addr":"172.21.15.183:6805/1108870440","state":["exists","up"]},{"osd":2,"uuid":"17c4eec3-6c47-4334-9b63-1d5ad9ec5988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6801","nonce":2877039731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6803","nonce":2877039731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6807","nonce":2877039731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6805","nonce":2877039731}]},"public_addr":"172.21.15.196:6801/2877039731","cluster_addr":"172.21.15.196:6803/2877039731","heartbeat_back_addr":"172.21.15.196:6807/2877039731","heartbeat_front_addr":"172.21.15.196:6805/2877039731","state":["exists","up"]},{"osd":3,"uuid":"3f731c38-2e45-4ca1-8dce-efc4a1b2166b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6811","nonce":4276311194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6813","nonce":4276311194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6817","nonce":4276311194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6815","nonce":4276311194}]},"public_addr":"172.21.15.97:6811/4276311194","cluster_addr":"172.21.15.97:6813/4276311194","heartbeat_back_addr":"172.21.15.97:6817/4276311194","heartbeat_front_addr":"172.21.15.97:6815/4276311194","state":["exists","up"]},{"osd":4,"uuid":"fdcafb14-9b82-4134-bd5b-79c0516f6cbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6809","nonce":63913916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6811","nonce":63913916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6815","nonce":63913916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6813","nonce":63913916}]},"public_addr":"172.21.15.183:6809/63913916","cluster_addr":"172.21.15.183:6811/63913916","heartbeat_back_addr":"172.21.15.183:6815/63913916","heartbeat_front_addr":"172.21.15.183:6813/63913916","state":["exists","up"]},{"osd":5,"uuid":"cdcc18d8-da3c-4012-bf12-876bb39e66eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6809","nonce":2804446896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6811","nonce":2804446896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6815","nonce":2804446896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6813","nonce":2804446896}]},"public_addr":"172.21.15.196:6809/2804446896","cluster_addr":"172.21.15.196:6811/2804446896","heartbeat_back_addr":"172.21.15.196:6815/2804446896","heartbeat_front_addr":"172.21.15.196:6813/2804446896","state":["exists","up"]},{"osd":6,"uuid":"071abb5c-2edf-4832-b8aa-3b80d20096fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6819","nonce":2080916705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6821","nonce":2080916705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6825","nonce":2080916705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6823","nonce":2080916705}]},"public_addr":"172.21.15.97:6819/2080916705","cluster_addr":"172.21.15.97:6821/2080916705","heartbeat_back_addr":"172.21.15.97:6825/2080916705","heartbeat_front_addr":"172.21.15.97:6823/2080916705","state":["exists","up"]},{"osd":7,"uuid":"9d54760b-66df-486b-8abb-017f3b64b4d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6817","nonce":1865873033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6819","nonce":1865873033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6823","nonce":1865873033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6821","nonce":1865873033}]},"public_addr":"172.21.15.183:6817/1865873033","cluster_addr":"172.21.15.183:6819/1865873033","heartbeat_back_addr":"172.21.15.183:6823/1865873033","heartbeat_front_addr":"172.21.15.183:6821/1865873033","state":["exists","up"]},{"osd":8,"uuid":"5cd3100a-bd6a-4be5-b5e9-0d0afc325193","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6817","nonce":4063932139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6819","nonce":4063932139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6823","nonce":4063932139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6821","nonce":4063932139}]},"public_addr":"172.21.15.196:6817/4063932139","cluster_addr":"172.21.15.196:6819/4063932139","heartbeat_back_addr":"172.21.15.196:6823/4063932139","heartbeat_front_addr":"172.21.15.196:6821/4063932139","state":["exists","up"]},{"osd":9,"uuid":"b6a5f4a1-4ecc-4f73-a2df-07551d9298ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6827","nonce":1887432474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6829","nonce":1887432474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6833","nonce":1887432474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6831","nonce":1887432474}]},"public_addr":"172.21.15.97:6827/1887432474","cluster_addr":"172.21.15.97:6829/1887432474","heartbeat_back_addr":"172.21.15.97:6833/1887432474","heartbeat_front_addr":"172.21.15.97:6831/1887432474","state":["exists","up"]},{"osd":10,"uuid":"86f59816-1ea6-4ae9-8796-cf5fc4831064","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6825","nonce":799434647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6827","nonce":799434647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6831","nonce":799434647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6829","nonce":799434647}]},"public_addr":"172.21.15.183:6825/799434647","cluster_addr":"172.21.15.183:6827/799434647","heartbeat_back_addr":"172.21.15.183:6831/799434647","heartbeat_front_addr":"172.21.15.183:6829/799434647","state":["exists","up"]},{"osd":11,"uuid":"95d64a89-a9f8-4c7a-8c0d-da7fb26cfff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6825","nonce":1873969467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6827","nonce":1873969467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6831","nonce":1873969467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6829","nonce":1873969467}]},"public_addr":"172.21.15.196:6825/1873969467","cluster_addr":"172.21.15.196:6827/1873969467","heartbeat_back_addr":"172.21.15.196:6831/1873969467","heartbeat_front_addr":"172.21.15.196:6829/1873969467","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:18:33.721156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:18:57.586709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:19:22.069458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:19:46.451597+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:20:11.561019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:20:36.298672+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:21:01.661804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:21:27.324128+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:21:52.898197+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:22:19.614326+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:22:45.154917+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:23:11.222612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/3866016040":"2024-04-24T10:16:59.468237+0000","172.21.15.97:6801/3297166858":"2024-04-24T10:16:59.468237+0000","172.21.15.97:0/3804743548":"2024-04-24T10:16:59.468237+0000","172.21.15.97:6800/3706463017":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/3715079438":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/786806834":"2024-04-24T10:16:59.468237+0000","172.21.15.97:6801/3706463017":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/3294175554":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/1449136760":"2024-04-24T10:16:33.128917+0000","172.21.15.97:6800/3297166858":"2024-04-24T10:16:59.468237+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T10:23:27.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.163+0000 7fa585707640 1 -- 172.21.15.97:0/238741627 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fa5680a3bb0 msgr2=0x7fa5680a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:27.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.163+0000 7fa585707640 1 --2- 172.21.15.97:0/238741627 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fa5680a3bb0 0x7fa5680a6070 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa574000c00 tx=0x7fa5740314a0 comp rx=0 tx=0).stop 2024-04-23T10:23:27.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.163+0000 7fa585707640 1 -- 172.21.15.97:0/238741627 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa580100e60 msgr2=0x7fa580197110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:27.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.163+0000 7fa585707640 1 --2- 172.21.15.97:0/238741627 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa580100e60 0x7fa580197110 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa57000d950 tx=0x7fa57000de10 comp rx=0 tx=0).stop 2024-04-23T10:23:27.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.164+0000 7fa585707640 1 -- 172.21.15.97:0/238741627 shutdown_connections 2024-04-23T10:23:27.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.164+0000 7fa585707640 1 --2- 172.21.15.97:0/238741627 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fa5680a3bb0 0x7fa5680a6070 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:27.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.164+0000 7fa585707640 1 --2- 172.21.15.97:0/238741627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa580102a00 0x7fa58019e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:27.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.164+0000 7fa585707640 1 --2- 172.21.15.97:0/238741627 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa580102040 0x7fa580197650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:27.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.164+0000 7fa585707640 1 --2- 172.21.15.97:0/238741627 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa580100e60 0x7fa580197110 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:27.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.164+0000 7fa585707640 1 -- 172.21.15.97:0/238741627 >> 172.21.15.97:0/238741627 conn(0x7fa5800fc440 msgr2=0x7fa5801054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:23:27.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.164+0000 7fa585707640 1 -- 172.21.15.97:0/238741627 shutdown_connections 2024-04-23T10:23:27.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.164+0000 7fa585707640 1 -- 172.21.15.97:0/238741627 wait complete. 2024-04-23T10:23:27.174 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T10:19:25.559138+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T10:23:27.174 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T10:23:27.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.276+0000 7f1f2876f640 1 -- 172.21.15.97:0/1716489461 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f201029a0 msgr2=0x7f1f201092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:27.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.276+0000 7f1f2876f640 1 --2- 172.21.15.97:0/1716489461 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f201029a0 0x7f1f201092c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1f18009ca0 tx=0x7f1f1802a8c0 comp rx=0 tx=0).stop 2024-04-23T10:23:27.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.276+0000 7f1f2876f640 1 -- 172.21.15.97:0/1716489461 shutdown_connections 2024-04-23T10:23:27.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.276+0000 7f1f2876f640 1 --2- 172.21.15.97:0/1716489461 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f201029a0 0x7f1f201092c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:27.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.276+0000 7f1f2876f640 1 --2- 172.21.15.97:0/1716489461 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f20101fe0 0x7f1f20102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:27.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.276+0000 7f1f2876f640 1 --2- 172.21.15.97:0/1716489461 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f20100e90 0x7f1f201012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:27.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.276+0000 7f1f2876f640 1 -- 172.21.15.97:0/1716489461 >> 172.21.15.97:0/1716489461 conn(0x7f1f200fc400 msgr2=0x7f1f200fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:23:27.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.276+0000 7f1f2876f640 1 -- 172.21.15.97:0/1716489461 shutdown_connections 2024-04-23T10:23:27.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.276+0000 7f1f2876f640 1 -- 172.21.15.97:0/1716489461 wait complete. 2024-04-23T10:23:27.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.277+0000 7f1f2876f640 1 Processor -- start 2024-04-23T10:23:27.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.277+0000 7f1f2876f640 1 -- start start 2024-04-23T10:23:27.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.277+0000 7f1f2876f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f20100e90 0x7f1f20069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:27.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.277+0000 7f1f2876f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f20101fe0 0x7f1f2019cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:27.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.277+0000 7f1f2876f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f201029a0 0x7f1f2019f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:27.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.277+0000 7f1f2876f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1f2010c9e0 con 0x7f1f20100e90 2024-04-23T10:23:27.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.277+0000 7f1f2876f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1f2010c860 con 0x7f1f20101fe0 2024-04-23T10:23:27.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.277+0000 7f1f2876f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1f2010cb60 con 0x7f1f201029a0 2024-04-23T10:23:27.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.277+0000 7f1f2650b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f20100e90 0x7f1f20069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:27.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.277+0000 7f1f2650b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f20100e90 0x7f1f20069c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55714/0 (socket says 172.21.15.97:55714) 2024-04-23T10:23:27.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.277+0000 7f1f2650b640 1 -- 172.21.15.97:0/4105184252 learned_addr learned my addr 172.21.15.97:0/4105184252 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:23:27.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.277+0000 7f1f25d0a640 1 --2- 172.21.15.97:0/4105184252 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f20101fe0 0x7f1f2019cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:27.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.277+0000 7f1f26d0c640 1 --2- 172.21.15.97:0/4105184252 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f201029a0 0x7f1f2019f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:27.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.278+0000 7f1f2650b640 1 -- 172.21.15.97:0/4105184252 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f201029a0 msgr2=0x7f1f2019f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:27.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.278+0000 7f1f2650b640 1 --2- 172.21.15.97:0/4105184252 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f201029a0 0x7f1f2019f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:27.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.278+0000 7f1f2650b640 1 -- 172.21.15.97:0/4105184252 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f20101fe0 msgr2=0x7f1f2019cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:27.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.278+0000 7f1f2650b640 1 --2- 172.21.15.97:0/4105184252 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f20101fe0 0x7f1f2019cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:27.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.278+0000 7f1f2650b640 1 -- 172.21.15.97:0/4105184252 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f2019fe60 con 0x7f1f20100e90 2024-04-23T10:23:27.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.278+0000 7f1f2650b640 1 --2- 172.21.15.97:0/4105184252 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f20100e90 0x7f1f20069c50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c00d950 tx=0x7f1f1c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:23:27.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.278+0000 7f1f177fe640 1 -- 172.21.15.97:0/4105184252 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f1c017030 con 0x7f1f20100e90 2024-04-23T10:23:27.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.278+0000 7f1f177fe640 1 -- 172.21.15.97:0/4105184252 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f1c004020 con 0x7f1f20100e90 2024-04-23T10:23:27.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.278+0000 7f1f2876f640 1 -- 172.21.15.97:0/4105184252 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f201a00f0 con 0x7f1f20100e90 2024-04-23T10:23:27.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.278+0000 7f1f177fe640 1 -- 172.21.15.97:0/4105184252 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f1c0047a0 con 0x7f1f20100e90 2024-04-23T10:23:27.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.278+0000 7f1f2876f640 1 -- 172.21.15.97:0/4105184252 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f201a0640 con 0x7f1f20100e90 2024-04-23T10:23:27.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.279+0000 7f1f2876f640 1 -- 172.21.15.97:0/4105184252 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f20108210 con 0x7f1f20100e90 2024-04-23T10:23:27.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.280+0000 7f1f177fe640 1 -- 172.21.15.97:0/4105184252 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1f1c009a30 con 0x7f1f20100e90 2024-04-23T10:23:27.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.280+0000 7f1f177fe640 1 --2- 172.21.15.97:0/4105184252 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f1f0c0a3bb0 0x7f1f0c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:27.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.281+0000 7f1f25d0a640 1 --2- 172.21.15.97:0/4105184252 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f1f0c0a3bb0 0x7f1f0c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:27.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.281+0000 7f1f177fe640 1 -- 172.21.15.97:0/4105184252 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f1f1c0cf970 con 0x7f1f20100e90 2024-04-23T10:23:27.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.281+0000 7f1f25d0a640 1 --2- 172.21.15.97:0/4105184252 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f1f0c0a3bb0 0x7f1f0c0a6070 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1f200699c0 tx=0x7f1f10005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:23:27.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.285+0000 7f1f177fe640 1 -- 172.21.15.97:0/4105184252 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1f1c09c7f0 con 0x7f1f20100e90 2024-04-23T10:23:27.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.463+0000 7f1f2876f640 1 -- 172.21.15.97:0/4105184252 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1f2002d830 con 0x7f1f20100e90 2024-04-23T10:23:27.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.464+0000 7f1f177fe640 1 -- 172.21.15.97:0/4105184252 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f1f1c09c0f0 con 0x7f1f20100e90 2024-04-23T10:23:27.463 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-23T10:23:27.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.466+0000 7f1f2876f640 1 -- 172.21.15.97:0/4105184252 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f1f0c0a3bb0 msgr2=0x7f1f0c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:27.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.466+0000 7f1f2876f640 1 --2- 172.21.15.97:0/4105184252 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f1f0c0a3bb0 0x7f1f0c0a6070 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1f200699c0 tx=0x7f1f10005f90 comp rx=0 tx=0).stop 2024-04-23T10:23:27.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.466+0000 7f1f2876f640 1 -- 172.21.15.97:0/4105184252 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f20100e90 msgr2=0x7f1f20069c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:27.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.466+0000 7f1f2876f640 1 --2- 172.21.15.97:0/4105184252 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f20100e90 0x7f1f20069c50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c00d950 tx=0x7f1f1c00de10 comp rx=0 tx=0).stop 2024-04-23T10:23:27.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.466+0000 7f1f2876f640 1 -- 172.21.15.97:0/4105184252 shutdown_connections 2024-04-23T10:23:27.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.466+0000 7f1f2876f640 1 --2- 172.21.15.97:0/4105184252 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f1f0c0a3bb0 0x7f1f0c0a6070 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:27.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.466+0000 7f1f2876f640 1 --2- 172.21.15.97:0/4105184252 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f201029a0 0x7f1f2019f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:27.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.466+0000 7f1f2876f640 1 --2- 172.21.15.97:0/4105184252 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f20101fe0 0x7f1f2019cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:27.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.466+0000 7f1f2876f640 1 --2- 172.21.15.97:0/4105184252 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f20100e90 0x7f1f20069c50 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:27.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.466+0000 7f1f2876f640 1 -- 172.21.15.97:0/4105184252 >> 172.21.15.97:0/4105184252 conn(0x7f1f200fc400 msgr2=0x7f1f201053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:23:27.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.466+0000 7f1f2876f640 1 -- 172.21.15.97:0/4105184252 shutdown_connections 2024-04-23T10:23:27.466 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.466+0000 7f1f2876f640 1 -- 172.21.15.97:0/4105184252 wait complete. 2024-04-23T10:23:27.476 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T10:23:27.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.575+0000 7f6cae33d640 1 -- 172.21.15.97:0/3246721257 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ca81029b0 msgr2=0x7f6ca81092d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:27.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.575+0000 7f6cae33d640 1 --2- 172.21.15.97:0/3246721257 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ca81029b0 0x7f6ca81092d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6ca0009ca0 tx=0x7f6ca002a8c0 comp rx=0 tx=0).stop 2024-04-23T10:23:27.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.575+0000 7f6cae33d640 1 -- 172.21.15.97:0/3246721257 shutdown_connections 2024-04-23T10:23:27.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.575+0000 7f6cae33d640 1 --2- 172.21.15.97:0/3246721257 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ca81029b0 0x7f6ca81092d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:27.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.575+0000 7f6cae33d640 1 --2- 172.21.15.97:0/3246721257 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ca8101ff0 0x7f6ca8102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:27.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.575+0000 7f6cae33d640 1 --2- 172.21.15.97:0/3246721257 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ca8100ea0 0x7f6ca81012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:27.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.575+0000 7f6cae33d640 1 -- 172.21.15.97:0/3246721257 >> 172.21.15.97:0/3246721257 conn(0x7f6ca80fc450 msgr2=0x7f6ca80fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:23:27.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.575+0000 7f6cae33d640 1 -- 172.21.15.97:0/3246721257 shutdown_connections 2024-04-23T10:23:27.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.575+0000 7f6cae33d640 1 -- 172.21.15.97:0/3246721257 wait complete. 2024-04-23T10:23:27.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.576+0000 7f6cae33d640 1 Processor -- start 2024-04-23T10:23:27.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.576+0000 7f6cae33d640 1 -- start start 2024-04-23T10:23:27.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.576+0000 7f6cae33d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ca8100ea0 0x7f6ca806b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:27.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.576+0000 7f6cae33d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ca8101ff0 0x7f6ca819ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:27.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.576+0000 7f6cae33d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ca81029b0 0x7f6ca819f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:27.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.576+0000 7f6cae33d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca810c9e0 con 0x7f6ca8101ff0 2024-04-23T10:23:27.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.576+0000 7f6cae33d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca810c860 con 0x7f6ca8100ea0 2024-04-23T10:23:27.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.576+0000 7f6cae33d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca810cb60 con 0x7f6ca81029b0 2024-04-23T10:23:27.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.576+0000 7f6ca77fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ca8101ff0 0x7f6ca819ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:27.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.576+0000 7f6ca77fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ca8101ff0 0x7f6ca819ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55736/0 (socket says 172.21.15.97:55736) 2024-04-23T10:23:27.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.576+0000 7f6ca77fe640 1 -- 172.21.15.97:0/442665784 learned_addr learned my addr 172.21.15.97:0/442665784 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:23:27.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.576+0000 7f6cac8da640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ca81029b0 0x7f6ca819f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:27.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.576+0000 7f6ca7fff640 1 --2- 172.21.15.97:0/442665784 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ca8100ea0 0x7f6ca806b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:27.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.576+0000 7f6ca77fe640 1 -- 172.21.15.97:0/442665784 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ca81029b0 msgr2=0x7f6ca819f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:27.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.576+0000 7f6ca77fe640 1 --2- 172.21.15.97:0/442665784 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ca81029b0 0x7f6ca819f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:27.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.576+0000 7f6ca77fe640 1 -- 172.21.15.97:0/442665784 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ca8100ea0 msgr2=0x7f6ca806b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:27.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.576+0000 7f6ca77fe640 1 --2- 172.21.15.97:0/442665784 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ca8100ea0 0x7f6ca806b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:27.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.576+0000 7f6ca77fe640 1 -- 172.21.15.97:0/442665784 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ca819fd60 con 0x7f6ca8101ff0 2024-04-23T10:23:27.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.577+0000 7f6ca77fe640 1 --2- 172.21.15.97:0/442665784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ca8101ff0 0x7f6ca819ce60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6c9000e9c0 tx=0x7f6c9000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:23:27.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.577+0000 7f6ca57fa640 1 -- 172.21.15.97:0/442665784 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9000cd80 con 0x7f6ca8101ff0 2024-04-23T10:23:27.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.577+0000 7f6ca57fa640 1 -- 172.21.15.97:0/442665784 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c900035f0 con 0x7f6ca8101ff0 2024-04-23T10:23:27.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.577+0000 7f6cae33d640 1 -- 172.21.15.97:0/442665784 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ca819fff0 con 0x7f6ca8101ff0 2024-04-23T10:23:27.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.577+0000 7f6ca57fa640 1 -- 172.21.15.97:0/442665784 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c90010690 con 0x7f6ca8101ff0 2024-04-23T10:23:27.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.577+0000 7f6cae33d640 1 -- 172.21.15.97:0/442665784 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ca81a0510 con 0x7f6ca8101ff0 2024-04-23T10:23:27.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.578+0000 7f6cae33d640 1 -- 172.21.15.97:0/442665784 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ca81081b0 con 0x7f6ca8101ff0 2024-04-23T10:23:27.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.579+0000 7f6ca57fa640 1 -- 172.21.15.97:0/442665784 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6c900037a0 con 0x7f6ca8101ff0 2024-04-23T10:23:27.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.580+0000 7f6ca57fa640 1 --2- 172.21.15.97:0/442665784 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f6c940a3bb0 0x7f6c940a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:27.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.580+0000 7f6ca7fff640 1 --2- 172.21.15.97:0/442665784 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f6c940a3bb0 0x7f6c940a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:27.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.580+0000 7f6ca57fa640 1 -- 172.21.15.97:0/442665784 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f6c900d0330 con 0x7f6ca8101ff0 2024-04-23T10:23:27.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.580+0000 7f6ca7fff640 1 --2- 172.21.15.97:0/442665784 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f6c940a3bb0 0x7f6c940a6070 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6c98005d00 tx=0x7f6c9800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:23:27.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.584+0000 7f6ca57fa640 1 -- 172.21.15.97:0/442665784 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6c9009d1b0 con 0x7f6ca8101ff0 2024-04-23T10:23:27.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.803+0000 7f6cae33d640 1 -- 172.21.15.97:0/442665784 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6ca802d5c0 con 0x7f6ca8101ff0 2024-04-23T10:23:27.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.804+0000 7f6ca57fa640 1 -- 172.21.15.97:0/442665784 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f6c9009cab0 con 0x7f6ca8101ff0 2024-04-23T10:23:27.803 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:23:27.803 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-23T10:23:27.803 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 1 2024-04-23T10:23:27.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.806+0000 7f6cae33d640 1 -- 172.21.15.97:0/442665784 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f6c940a3bb0 msgr2=0x7f6c940a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:27.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.806+0000 7f6cae33d640 1 --2- 172.21.15.97:0/442665784 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f6c940a3bb0 0x7f6c940a6070 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6c98005d00 tx=0x7f6c9800b410 comp rx=0 tx=0).stop 2024-04-23T10:23:27.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.806+0000 7f6cae33d640 1 -- 172.21.15.97:0/442665784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ca8101ff0 msgr2=0x7f6ca819ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:27.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.806+0000 7f6cae33d640 1 --2- 172.21.15.97:0/442665784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ca8101ff0 0x7f6ca819ce60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6c9000e9c0 tx=0x7f6c9000ee80 comp rx=0 tx=0).stop 2024-04-23T10:23:27.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.806+0000 7f6cae33d640 1 -- 172.21.15.97:0/442665784 shutdown_connections 2024-04-23T10:23:27.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.806+0000 7f6cae33d640 1 --2- 172.21.15.97:0/442665784 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f6c940a3bb0 0x7f6c940a6070 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:27.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.806+0000 7f6cae33d640 1 --2- 172.21.15.97:0/442665784 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ca81029b0 0x7f6ca819f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:27.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.806+0000 7f6cae33d640 1 --2- 172.21.15.97:0/442665784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ca8101ff0 0x7f6ca819ce60 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:27.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.806+0000 7f6cae33d640 1 --2- 172.21.15.97:0/442665784 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ca8100ea0 0x7f6ca806b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:27.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.806+0000 7f6cae33d640 1 -- 172.21.15.97:0/442665784 >> 172.21.15.97:0/442665784 conn(0x7f6ca80fc450 msgr2=0x7f6ca8105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:23:27.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.806+0000 7f6cae33d640 1 -- 172.21.15.97:0/442665784 shutdown_connections 2024-04-23T10:23:27.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:27.806+0000 7f6cae33d640 1 -- 172.21.15.97:0/442665784 wait complete. 2024-04-23T10:23:27.815 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:27 smithi097 ceph-mon[29197]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi097=a;smithi097=d;smithi097=g;smithi097=j;smithi183=b;smithi183=e;smithi183=h;smithi183=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:23:27.815 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:27 smithi097 ceph-mon[29197]: Saving service mds.all spec with placement smithi097=a;smithi097=d;smithi097=g;smithi097=j;smithi183=b;smithi183=e;smithi183=h;smithi183=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l;count:12 2024-04-23T10:23:27.815 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:27 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:23:27.815 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:27 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:23:27.815 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:27 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:27.815 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:27 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T10:23:27.815 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:27 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T10:23:27.815 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:27 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:23:27.815 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:27 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/238741627' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T10:23:27.815 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:27 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/4105184252' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T10:23:27.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:27 smithi183 ceph-mon[32317]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi097=a;smithi097=d;smithi097=g;smithi097=j;smithi183=b;smithi183=e;smithi183=h;smithi183=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:23:27.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:27 smithi183 ceph-mon[32317]: Saving service mds.all spec with placement smithi097=a;smithi097=d;smithi097=g;smithi097=j;smithi183=b;smithi183=e;smithi183=h;smithi183=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l;count:12 2024-04-23T10:23:27.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:27 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:23:27.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:27 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:23:27.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:27 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:27.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:27 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T10:23:27.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:27 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T10:23:27.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:27 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:23:27.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:27 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/238741627' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T10:23:27.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:27 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/4105184252' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T10:23:28.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:27 smithi196 ceph-mon[32919]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi097=a;smithi097=d;smithi097=g;smithi097=j;smithi183=b;smithi183=e;smithi183=h;smithi183=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:23:28.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:27 smithi196 ceph-mon[32919]: Saving service mds.all spec with placement smithi097=a;smithi097=d;smithi097=g;smithi097=j;smithi183=b;smithi183=e;smithi183=h;smithi183=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l;count:12 2024-04-23T10:23:28.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:27 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:23:28.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:27 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:23:28.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:27 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:28.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:27 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T10:23:28.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:27 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T10:23:28.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:27 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:23:28.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:27 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/238741627' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T10:23:28.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:27 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/4105184252' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T10:23:28.732 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:28 smithi196 ceph-mon[32919]: Deploying daemon mds.l on smithi196 2024-04-23T10:23:28.733 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:28 smithi196 ceph-mon[32919]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:23:28.733 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:28 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/442665784' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:23:28.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:28 smithi097 ceph-mon[29197]: Deploying daemon mds.l on smithi196 2024-04-23T10:23:28.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:28 smithi097 ceph-mon[29197]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:23:28.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:28 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/442665784' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:23:28.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:28 smithi183 ceph-mon[32317]: Deploying daemon mds.l on smithi196 2024-04-23T10:23:28.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:28 smithi183 ceph-mon[32317]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:23:28.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:28 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/442665784' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:23:29.091 INFO:journalctl@ceph.mds.l.smithi196.stdout:Apr 23 10:23:28 smithi196 systemd[1]: Starting Ceph mds.l for 9b22af88-0159-11ef-bc93-c7b262605968... 2024-04-23T10:23:29.409 INFO:journalctl@ceph.mds.l.smithi196.stdout:Apr 23 10:23:29 smithi196 podman[63394]: 2024-04-23 10:23:29.030114445 +0000 UTC m=+0.017798476 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T10:23:29.409 INFO:journalctl@ceph.mds.l.smithi196.stdout:Apr 23 10:23:29 smithi196 podman[63394]: 2024-04-23 10:23:29.13282923 +0000 UTC m=+0.120513260 container create d00e288d04d9a2cced0ac98ab87f98eabca234e016a96df4a5fb410029905f08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-l, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-23T10:23:29.409 INFO:journalctl@ceph.mds.l.smithi196.stdout:Apr 23 10:23:29 smithi196 podman[63394]: 2024-04-23 10:23:29.410613493 +0000 UTC m=+0.398297523 container init d00e288d04d9a2cced0ac98ab87f98eabca234e016a96df4a5fb410029905f08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-l, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True) 2024-04-23T10:23:29.817 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T10:23:29.841 INFO:journalctl@ceph.mds.l.smithi196.stdout:Apr 23 10:23:29 smithi196 podman[63394]: 2024-04-23 10:23:29.413278275 +0000 UTC m=+0.400962318 container start d00e288d04d9a2cced0ac98ab87f98eabca234e016a96df4a5fb410029905f08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-l, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-23T10:23:29.842 INFO:journalctl@ceph.mds.l.smithi196.stdout:Apr 23 10:23:29 smithi196 ceph-mds[63413]: -- 172.21.15.196:0/1623928755 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d8d5655a40 con 0x55d8d5628800 2024-04-23T10:23:29.842 INFO:journalctl@ceph.mds.l.smithi196.stdout:Apr 23 10:23:29 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-l[63409]: starting mds.l at 2024-04-23T10:23:29.842 INFO:journalctl@ceph.mds.l.smithi196.stdout:Apr 23 10:23:29 smithi196 bash[63394]: d00e288d04d9a2cced0ac98ab87f98eabca234e016a96df4a5fb410029905f08 2024-04-23T10:23:29.842 INFO:journalctl@ceph.mds.l.smithi196.stdout:Apr 23 10:23:29 smithi196 systemd[1]: Started Ceph mds.l for 9b22af88-0159-11ef-bc93-c7b262605968. 2024-04-23T10:23:29.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:29.919+0000 7f2b21976640 1 -- 172.21.15.97:0/3407076738 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b1c1029a0 msgr2=0x7f2b1c1092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:29.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:29.919+0000 7f2b21976640 1 --2- 172.21.15.97:0/3407076738 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b1c1029a0 0x7f2b1c1092c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2b14009ca0 tx=0x7f2b1402a8c0 comp rx=0 tx=0).stop 2024-04-23T10:23:29.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:29.919+0000 7f2b21976640 1 -- 172.21.15.97:0/3407076738 shutdown_connections 2024-04-23T10:23:29.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:29.919+0000 7f2b21976640 1 --2- 172.21.15.97:0/3407076738 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b1c1029a0 0x7f2b1c1092c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:29.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:29.919+0000 7f2b21976640 1 --2- 172.21.15.97:0/3407076738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b1c101fe0 0x7f2b1c102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:29.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:29.919+0000 7f2b21976640 1 --2- 172.21.15.97:0/3407076738 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b1c100e90 0x7f2b1c1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:29.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:29.919+0000 7f2b21976640 1 -- 172.21.15.97:0/3407076738 >> 172.21.15.97:0/3407076738 conn(0x7f2b1c0fc400 msgr2=0x7f2b1c0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:23:29.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:29.919+0000 7f2b21976640 1 -- 172.21.15.97:0/3407076738 shutdown_connections 2024-04-23T10:23:29.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:29.919+0000 7f2b21976640 1 -- 172.21.15.97:0/3407076738 wait complete. 2024-04-23T10:23:29.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:29.920+0000 7f2b21976640 1 Processor -- start 2024-04-23T10:23:29.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:29.920+0000 7f2b21976640 1 -- start start 2024-04-23T10:23:29.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:29.920+0000 7f2b21976640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b1c100e90 0x7f2b1c197020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:29.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:29.920+0000 7f2b21976640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b1c101fe0 0x7f2b1c197560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:29.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:29.920+0000 7f2b21976640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b1c1029a0 0x7f2b1c19e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:29.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:29.920+0000 7f2b21976640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1c10c9d0 con 0x7f2b1c100e90 2024-04-23T10:23:29.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:29.920+0000 7f2b21976640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1c10c850 con 0x7f2b1c101fe0 2024-04-23T10:23:29.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:29.920+0000 7f2b21976640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1c10cb50 con 0x7f2b1c1029a0 2024-04-23T10:23:29.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:29.920+0000 7f2b1affd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b1c100e90 0x7f2b1c197020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:29.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:29.920+0000 7f2b1affd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b1c100e90 0x7f2b1c197020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55746/0 (socket says 172.21.15.97:55746) 2024-04-23T10:23:29.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:29.920+0000 7f2b1affd640 1 -- 172.21.15.97:0/3969585229 learned_addr learned my addr 172.21.15.97:0/3969585229 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:23:29.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:29.920+0000 7f2b13fff640 1 --2- 172.21.15.97:0/3969585229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b1c101fe0 0x7f2b1c197560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:29.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:29.920+0000 7f2b1b7fe640 1 --2- 172.21.15.97:0/3969585229 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b1c1029a0 0x7f2b1c19e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:29.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:29.920+0000 7f2b1affd640 1 -- 172.21.15.97:0/3969585229 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b1c1029a0 msgr2=0x7f2b1c19e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:29.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:29.920+0000 7f2b1affd640 1 --2- 172.21.15.97:0/3969585229 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b1c1029a0 0x7f2b1c19e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:29.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:29.920+0000 7f2b1affd640 1 -- 172.21.15.97:0/3969585229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b1c101fe0 msgr2=0x7f2b1c197560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:29.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:29.920+0000 7f2b1affd640 1 --2- 172.21.15.97:0/3969585229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b1c101fe0 0x7f2b1c197560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:29.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:29.920+0000 7f2b1affd640 1 -- 172.21.15.97:0/3969585229 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b1c19ec40 con 0x7f2b1c100e90 2024-04-23T10:23:29.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:29.920+0000 7f2b1b7fe640 1 --2- 172.21.15.97:0/3969585229 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b1c1029a0 0x7f2b1c19e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:23:29.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:29.920+0000 7f2b13fff640 1 --2- 172.21.15.97:0/3969585229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b1c101fe0 0x7f2b1c197560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T10:23:29.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:29.920+0000 7f2b1affd640 1 --2- 172.21.15.97:0/3969585229 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b1c100e90 0x7f2b1c197020 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c00e9f0 tx=0x7f2b0c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:23:29.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:29.921+0000 7f2b20974640 1 -- 172.21.15.97:0/3969585229 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c00cd80 con 0x7f2b1c100e90 2024-04-23T10:23:29.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:29.921+0000 7f2b20974640 1 -- 172.21.15.97:0/3969585229 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b0c0108e0 con 0x7f2b1c100e90 2024-04-23T10:23:29.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:29.921+0000 7f2b21976640 1 -- 172.21.15.97:0/3969585229 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b1c19ef30 con 0x7f2b1c100e90 2024-04-23T10:23:29.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:29.921+0000 7f2b20974640 1 -- 172.21.15.97:0/3969585229 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c018980 con 0x7f2b1c100e90 2024-04-23T10:23:29.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:29.921+0000 7f2b21976640 1 -- 172.21.15.97:0/3969585229 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b1c19f390 con 0x7f2b1c100e90 2024-04-23T10:23:29.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:29.922+0000 7f2b21976640 1 -- 172.21.15.97:0/3969585229 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b1c108140 con 0x7f2b1c100e90 2024-04-23T10:23:29.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:29.923+0000 7f2b20974640 1 -- 172.21.15.97:0/3969585229 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2b0c010430 con 0x7f2b1c100e90 2024-04-23T10:23:29.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:29.923+0000 7f2b20974640 1 --2- 172.21.15.97:0/3969585229 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f2b080a3bb0 0x7f2b080a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:29.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:29.923+0000 7f2b13fff640 1 --2- 172.21.15.97:0/3969585229 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f2b080a3bb0 0x7f2b080a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:29.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:29.923+0000 7f2b20974640 1 -- 172.21.15.97:0/3969585229 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f2b0c0d02e0 con 0x7f2b1c100e90 2024-04-23T10:23:29.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:29.924+0000 7f2b13fff640 1 --2- 172.21.15.97:0/3969585229 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f2b080a3bb0 0x7f2b080a6070 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c198530 tx=0x7f2b04005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:23:29.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:29.927+0000 7f2b20974640 1 -- 172.21.15.97:0/3969585229 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2b0c09d160 con 0x7f2b1c100e90 2024-04-23T10:23:30.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:30.150+0000 7f2b21976640 1 -- 172.21.15.97:0/3969585229 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2b1c02d850 con 0x7f2b1c100e90 2024-04-23T10:23:30.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:30.151+0000 7f2b20974640 1 -- 172.21.15.97:0/3969585229 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f2b0c09ca60 con 0x7f2b1c100e90 2024-04-23T10:23:30.150 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:23:30.150 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3885816565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3885816565},{"type":"v1","addr":"172.21.15.196:6833","nonce":3885816565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-23T10:23:30.150 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 2 2024-04-23T10:23:30.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:30.153+0000 7f2b21976640 1 -- 172.21.15.97:0/3969585229 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f2b080a3bb0 msgr2=0x7f2b080a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:30.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:30.153+0000 7f2b21976640 1 --2- 172.21.15.97:0/3969585229 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f2b080a3bb0 0x7f2b080a6070 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c198530 tx=0x7f2b04005f90 comp rx=0 tx=0).stop 2024-04-23T10:23:30.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:30.153+0000 7f2b21976640 1 -- 172.21.15.97:0/3969585229 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b1c100e90 msgr2=0x7f2b1c197020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:30.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:30.153+0000 7f2b21976640 1 --2- 172.21.15.97:0/3969585229 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b1c100e90 0x7f2b1c197020 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c00e9f0 tx=0x7f2b0c00eeb0 comp rx=0 tx=0).stop 2024-04-23T10:23:30.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:30.153+0000 7f2b21976640 1 -- 172.21.15.97:0/3969585229 shutdown_connections 2024-04-23T10:23:30.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:30.153+0000 7f2b21976640 1 --2- 172.21.15.97:0/3969585229 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f2b080a3bb0 0x7f2b080a6070 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:30.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:30.153+0000 7f2b21976640 1 --2- 172.21.15.97:0/3969585229 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b1c1029a0 0x7f2b1c19e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:30.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:30.153+0000 7f2b21976640 1 --2- 172.21.15.97:0/3969585229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b1c101fe0 0x7f2b1c197560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:30.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:30.153+0000 7f2b21976640 1 --2- 172.21.15.97:0/3969585229 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b1c100e90 0x7f2b1c197020 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:30.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:30.153+0000 7f2b21976640 1 -- 172.21.15.97:0/3969585229 >> 172.21.15.97:0/3969585229 conn(0x7f2b1c0fc400 msgr2=0x7f2b1c105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:23:30.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:30.153+0000 7f2b21976640 1 -- 172.21.15.97:0/3969585229 shutdown_connections 2024-04-23T10:23:30.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:30.153+0000 7f2b21976640 1 -- 172.21.15.97:0/3969585229 wait complete. 2024-04-23T10:23:30.572 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:30 smithi183 ceph-mon[32317]: mds.? [v2:172.21.15.196:6832/3885816565,v1:172.21.15.196:6833/3885816565] up:boot 2024-04-23T10:23:30.572 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:30 smithi183 ceph-mon[32317]: fsmap 1 up:standby 2024-04-23T10:23:30.572 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:30 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-23T10:23:30.573 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:30 smithi183 ceph-mon[32317]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:23:30.573 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:30 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:30.573 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:30 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:30.573 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:30 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:30.573 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:30 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T10:23:30.573 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:30 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T10:23:30.573 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:30 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:23:30.573 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:30 smithi183 ceph-mon[32317]: Deploying daemon mds.k on smithi183 2024-04-23T10:23:30.573 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:30 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/3969585229' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:23:30.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:30 smithi196 ceph-mon[32919]: mds.? [v2:172.21.15.196:6832/3885816565,v1:172.21.15.196:6833/3885816565] up:boot 2024-04-23T10:23:30.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:30 smithi196 ceph-mon[32919]: fsmap 1 up:standby 2024-04-23T10:23:30.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:30 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-23T10:23:30.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:30 smithi196 ceph-mon[32919]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:23:30.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:30 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:30.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:30 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:30.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:30 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:30.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:30 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T10:23:30.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:30 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T10:23:30.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:30 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:23:30.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:30 smithi196 ceph-mon[32919]: Deploying daemon mds.k on smithi183 2024-04-23T10:23:30.843 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:30 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/3969585229' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:23:30.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:30 smithi097 ceph-mon[29197]: mds.? [v2:172.21.15.196:6832/3885816565,v1:172.21.15.196:6833/3885816565] up:boot 2024-04-23T10:23:30.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:30 smithi097 ceph-mon[29197]: fsmap 1 up:standby 2024-04-23T10:23:30.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:30 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-23T10:23:30.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:30 smithi097 ceph-mon[29197]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:23:30.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:30 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:30.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:30 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:30.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:30 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:30.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:30 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T10:23:30.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:30 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T10:23:30.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:30 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:23:30.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:30 smithi097 ceph-mon[29197]: Deploying daemon mds.k on smithi183 2024-04-23T10:23:30.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:30 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/3969585229' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:23:32.151 INFO:journalctl@ceph.mds.k.smithi183.stdout:Apr 23 10:23:31 smithi183 systemd[1]: Starting Ceph mds.k for 9b22af88-0159-11ef-bc93-c7b262605968... 2024-04-23T10:23:32.164 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T10:23:32.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:32.275+0000 7f5e49c33640 1 -- 172.21.15.97:0/4057224268 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e44100ea0 msgr2=0x7f5e441012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:32.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:32.275+0000 7f5e49c33640 1 --2- 172.21.15.97:0/4057224268 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e44100ea0 0x7f5e441012c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5e38009900 tx=0x7f5e3802a410 comp rx=0 tx=0).stop 2024-04-23T10:23:32.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:32.275+0000 7f5e49c33640 1 -- 172.21.15.97:0/4057224268 shutdown_connections 2024-04-23T10:23:32.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:32.275+0000 7f5e49c33640 1 --2- 172.21.15.97:0/4057224268 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e441029b0 0x7f5e441092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:32.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:32.275+0000 7f5e49c33640 1 --2- 172.21.15.97:0/4057224268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e44101ff0 0x7f5e44102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:32.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:32.275+0000 7f5e49c33640 1 --2- 172.21.15.97:0/4057224268 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e44100ea0 0x7f5e441012c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:32.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:32.275+0000 7f5e49c33640 1 -- 172.21.15.97:0/4057224268 >> 172.21.15.97:0/4057224268 conn(0x7f5e440fc430 msgr2=0x7f5e440fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:23:32.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:32.275+0000 7f5e49c33640 1 -- 172.21.15.97:0/4057224268 shutdown_connections 2024-04-23T10:23:32.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:32.275+0000 7f5e49c33640 1 -- 172.21.15.97:0/4057224268 wait complete. 2024-04-23T10:23:32.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:32.276+0000 7f5e49c33640 1 Processor -- start 2024-04-23T10:23:32.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:32.276+0000 7f5e49c33640 1 -- start start 2024-04-23T10:23:32.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:32.276+0000 7f5e49c33640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e44100ea0 0x7f5e4410b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:32.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:32.276+0000 7f5e49c33640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e44101ff0 0x7f5e44109e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:32.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:32.276+0000 7f5e49c33640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e441029b0 0x7f5e4410a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:32.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:32.276+0000 7f5e49c33640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5e4410db80 con 0x7f5e44100ea0 2024-04-23T10:23:32.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:32.276+0000 7f5e49c33640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5e4410da00 con 0x7f5e441029b0 2024-04-23T10:23:32.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:32.276+0000 7f5e49c33640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5e4410dd00 con 0x7f5e44101ff0 2024-04-23T10:23:32.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:32.276+0000 7f5e437fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e44100ea0 0x7f5e4410b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:32.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:32.276+0000 7f5e437fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e44100ea0 0x7f5e4410b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55770/0 (socket says 172.21.15.97:55770) 2024-04-23T10:23:32.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:32.276+0000 7f5e437fe640 1 -- 172.21.15.97:0/1206526286 learned_addr learned my addr 172.21.15.97:0/1206526286 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:23:32.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:32.276+0000 7f5e43fff640 1 --2- 172.21.15.97:0/1206526286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e441029b0 0x7f5e4410a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:32.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:32.276+0000 7f5e42ffd640 1 --2- 172.21.15.97:0/1206526286 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e44101ff0 0x7f5e44109e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:32.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:32.277+0000 7f5e437fe640 1 -- 172.21.15.97:0/1206526286 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e44101ff0 msgr2=0x7f5e44109e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:32.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:32.277+0000 7f5e437fe640 1 --2- 172.21.15.97:0/1206526286 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e44101ff0 0x7f5e44109e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:32.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:32.277+0000 7f5e437fe640 1 -- 172.21.15.97:0/1206526286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e441029b0 msgr2=0x7f5e4410a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:32.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:32.277+0000 7f5e437fe640 1 --2- 172.21.15.97:0/1206526286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e441029b0 0x7f5e4410a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:32.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:32.277+0000 7f5e437fe640 1 -- 172.21.15.97:0/1206526286 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e4410ac50 con 0x7f5e44100ea0 2024-04-23T10:23:32.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:32.277+0000 7f5e437fe640 1 --2- 172.21.15.97:0/1206526286 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e44100ea0 0x7f5e4410b840 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5e38031500 tx=0x7f5e380043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:23:32.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:32.277+0000 7f5e40ff9640 1 -- 172.21.15.97:0/1206526286 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e3803f030 con 0x7f5e44100ea0 2024-04-23T10:23:32.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:32.277+0000 7f5e49c33640 1 -- 172.21.15.97:0/1206526286 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e441a2390 con 0x7f5e44100ea0 2024-04-23T10:23:32.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:32.277+0000 7f5e40ff9640 1 -- 172.21.15.97:0/1206526286 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e38003800 con 0x7f5e44100ea0 2024-04-23T10:23:32.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:32.277+0000 7f5e40ff9640 1 -- 172.21.15.97:0/1206526286 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e38035070 con 0x7f5e44100ea0 2024-04-23T10:23:32.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:32.277+0000 7f5e49c33640 1 -- 172.21.15.97:0/1206526286 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e441a29b0 con 0x7f5e44100ea0 2024-04-23T10:23:32.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:32.278+0000 7f5e49c33640 1 -- 172.21.15.97:0/1206526286 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e44108250 con 0x7f5e44100ea0 2024-04-23T10:23:32.282 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:32.279+0000 7f5e40ff9640 1 -- 172.21.15.97:0/1206526286 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5e380039b0 con 0x7f5e44100ea0 2024-04-23T10:23:32.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:32.280+0000 7f5e40ff9640 1 --2- 172.21.15.97:0/1206526286 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f5e2c0a3bb0 0x7f5e2c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:32.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:32.280+0000 7f5e42ffd640 1 --2- 172.21.15.97:0/1206526286 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f5e2c0a3bb0 0x7f5e2c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:32.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:32.280+0000 7f5e40ff9640 1 -- 172.21.15.97:0/1206526286 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f5e380f0f20 con 0x7f5e44100ea0 2024-04-23T10:23:32.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:32.280+0000 7f5e42ffd640 1 --2- 172.21.15.97:0/1206526286 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f5e2c0a3bb0 0x7f5e2c0a6070 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5e4410b5b0 tx=0x7f5e34009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:23:32.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:32.284+0000 7f5e40ff9640 1 -- 172.21.15.97:0/1206526286 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5e380bdda0 con 0x7f5e44100ea0 2024-04-23T10:23:32.422 INFO:journalctl@ceph.mds.k.smithi183.stdout:Apr 23 10:23:32 smithi183 podman[61610]: 2024-04-23 10:23:32.075906254 +0000 UTC m=+0.018211081 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T10:23:32.422 INFO:journalctl@ceph.mds.k.smithi183.stdout:Apr 23 10:23:32 smithi183 podman[61610]: 2024-04-23 10:23:32.178304249 +0000 UTC m=+0.120609112 container create 6d907df4143ee3df4254116a7906808d37864c786dc21f630e8cd8c00d860332 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-23T10:23:32.422 INFO:journalctl@ceph.mds.k.smithi183.stdout:Apr 23 10:23:32 smithi183 podman[61610]: 2024-04-23 10:23:32.397511355 +0000 UTC m=+0.339816173 container init 6d907df4143ee3df4254116a7906808d37864c786dc21f630e8cd8c00d860332 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-k, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T10:23:32.423 INFO:journalctl@ceph.mds.k.smithi183.stdout:Apr 23 10:23:32 smithi183 podman[61610]: 2024-04-23 10:23:32.40003353 +0000 UTC m=+0.342338358 container start 6d907df4143ee3df4254116a7906808d37864c786dc21f630e8cd8c00d860332 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-k, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416) 2024-04-23T10:23:32.423 INFO:journalctl@ceph.mds.k.smithi183.stdout:Apr 23 10:23:32 smithi183 ceph-mds[61629]: -- 172.21.15.183:0/818540290 <== mon.2 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5636e4751a40 con 0x5636e4724800 2024-04-23T10:23:32.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:32.500+0000 7f5e49c33640 1 -- 172.21.15.97:0/1206526286 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5e4402d5c0 con 0x7f5e44100ea0 2024-04-23T10:23:32.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:32.500+0000 7f5e40ff9640 1 -- 172.21.15.97:0/1206526286 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f5e380bd6a0 con 0x7f5e44100ea0 2024-04-23T10:23:32.500 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:23:32.500 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3885816565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3885816565},{"type":"v1","addr":"172.21.15.196:6833","nonce":3885816565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-23T10:23:32.500 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 2 2024-04-23T10:23:32.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:32.502+0000 7f5e49c33640 1 -- 172.21.15.97:0/1206526286 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f5e2c0a3bb0 msgr2=0x7f5e2c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:32.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:32.502+0000 7f5e49c33640 1 --2- 172.21.15.97:0/1206526286 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f5e2c0a3bb0 0x7f5e2c0a6070 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5e4410b5b0 tx=0x7f5e34009110 comp rx=0 tx=0).stop 2024-04-23T10:23:32.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:32.502+0000 7f5e49c33640 1 -- 172.21.15.97:0/1206526286 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e44100ea0 msgr2=0x7f5e4410b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:32.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:32.502+0000 7f5e49c33640 1 --2- 172.21.15.97:0/1206526286 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e44100ea0 0x7f5e4410b840 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5e38031500 tx=0x7f5e380043c0 comp rx=0 tx=0).stop 2024-04-23T10:23:32.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:32.502+0000 7f5e49c33640 1 -- 172.21.15.97:0/1206526286 shutdown_connections 2024-04-23T10:23:32.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:32.502+0000 7f5e49c33640 1 --2- 172.21.15.97:0/1206526286 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f5e2c0a3bb0 0x7f5e2c0a6070 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:32.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:32.502+0000 7f5e49c33640 1 --2- 172.21.15.97:0/1206526286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e441029b0 0x7f5e4410a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:32.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:32.502+0000 7f5e49c33640 1 --2- 172.21.15.97:0/1206526286 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5e44101ff0 0x7f5e44109e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:32.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:32.502+0000 7f5e49c33640 1 --2- 172.21.15.97:0/1206526286 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e44100ea0 0x7f5e4410b840 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:32.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:32.502+0000 7f5e49c33640 1 -- 172.21.15.97:0/1206526286 >> 172.21.15.97:0/1206526286 conn(0x7f5e440fc430 msgr2=0x7f5e44105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:23:32.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:32.502+0000 7f5e49c33640 1 -- 172.21.15.97:0/1206526286 shutdown_connections 2024-04-23T10:23:32.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:32.502+0000 7f5e49c33640 1 -- 172.21.15.97:0/1206526286 wait complete. 2024-04-23T10:23:32.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:32 smithi097 ceph-mon[29197]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:23:32.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:32 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/1206526286' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:23:32.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:32 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:32.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:32 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:32.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:32 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:32.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:32 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T10:23:32.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:32 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T10:23:32.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:32 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:23:32.900 INFO:journalctl@ceph.mds.k.smithi183.stdout:Apr 23 10:23:32 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-k[61625]: starting mds.k at 2024-04-23T10:23:32.900 INFO:journalctl@ceph.mds.k.smithi183.stdout:Apr 23 10:23:32 smithi183 bash[61610]: 6d907df4143ee3df4254116a7906808d37864c786dc21f630e8cd8c00d860332 2024-04-23T10:23:32.900 INFO:journalctl@ceph.mds.k.smithi183.stdout:Apr 23 10:23:32 smithi183 systemd[1]: Started Ceph mds.k for 9b22af88-0159-11ef-bc93-c7b262605968. 2024-04-23T10:23:32.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:32 smithi183 ceph-mon[32317]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:23:32.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:32 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/1206526286' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:23:32.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:32 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:32.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:32 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:32.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:32 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:32.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:32 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T10:23:32.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:32 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T10:23:32.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:32 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:23:33.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:32 smithi196 ceph-mon[32919]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:23:33.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:32 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/1206526286' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:23:33.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:32 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:33.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:32 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:33.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:32 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:33.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:32 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T10:23:33.093 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:32 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T10:23:33.093 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:32 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:23:33.824 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:33 smithi183 ceph-mon[32317]: Deploying daemon mds.j on smithi097 2024-04-23T10:23:33.824 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:33 smithi183 ceph-mon[32317]: mds.? [v2:172.21.15.183:6832/4292847121,v1:172.21.15.183:6833/4292847121] up:boot 2024-04-23T10:23:33.824 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:33 smithi183 ceph-mon[32317]: fsmap 2 up:standby 2024-04-23T10:23:33.824 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:33 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-23T10:23:33.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:33 smithi097 ceph-mon[29197]: Deploying daemon mds.j on smithi097 2024-04-23T10:23:33.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:33 smithi097 ceph-mon[29197]: mds.? [v2:172.21.15.183:6832/4292847121,v1:172.21.15.183:6833/4292847121] up:boot 2024-04-23T10:23:33.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:33 smithi097 ceph-mon[29197]: fsmap 2 up:standby 2024-04-23T10:23:33.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:33 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-23T10:23:34.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:33 smithi196 ceph-mon[32919]: Deploying daemon mds.j on smithi097 2024-04-23T10:23:34.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:33 smithi196 ceph-mon[32919]: mds.? [v2:172.21.15.183:6832/4292847121,v1:172.21.15.183:6833/4292847121] up:boot 2024-04-23T10:23:34.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:33 smithi196 ceph-mon[32919]: fsmap 2 up:standby 2024-04-23T10:23:34.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:33 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-23T10:23:34.513 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T10:23:34.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:34.754+0000 7f8fb77ad640 1 -- 172.21.15.97:0/1861422191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fb0102ab0 msgr2=0x7f8fb0104ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:34.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:34.754+0000 7f8fb77ad640 1 --2- 172.21.15.97:0/1861422191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fb0102ab0 0x7f8fb0104ea0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8fa8009ca0 tx=0x7f8fa802a8c0 comp rx=0 tx=0).stop 2024-04-23T10:23:34.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:34.754+0000 7f8fb77ad640 1 -- 172.21.15.97:0/1861422191 shutdown_connections 2024-04-23T10:23:34.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:34.754+0000 7f8fb77ad640 1 --2- 172.21.15.97:0/1861422191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8fb0107d10 0x7f8fb010a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:34.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:34.754+0000 7f8fb77ad640 1 --2- 172.21.15.97:0/1861422191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8fb01053e0 0x7f8fb01077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:34.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:34.754+0000 7f8fb77ad640 1 --2- 172.21.15.97:0/1861422191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fb0102ab0 0x7f8fb0104ea0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:34.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:34.754+0000 7f8fb77ad640 1 -- 172.21.15.97:0/1861422191 >> 172.21.15.97:0/1861422191 conn(0x7f8fb00fc440 msgr2=0x7f8fb00fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:23:34.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:34.754+0000 7f8fb77ad640 1 -- 172.21.15.97:0/1861422191 shutdown_connections 2024-04-23T10:23:34.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:34.754+0000 7f8fb77ad640 1 -- 172.21.15.97:0/1861422191 wait complete. 2024-04-23T10:23:34.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:34.754+0000 7f8fb77ad640 1 Processor -- start 2024-04-23T10:23:34.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:34.755+0000 7f8fb77ad640 1 -- start start 2024-04-23T10:23:34.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:34.755+0000 7f8fb77ad640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8fb0102ab0 0x7f8fb0198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:34.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:34.755+0000 7f8fb77ad640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fb01053e0 0x7f8fb01987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:34.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:34.755+0000 7f8fb77ad640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8fb0107d10 0x7f8fb019f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:34.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:34.755+0000 7f8fb77ad640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb010db10 con 0x7f8fb01053e0 2024-04-23T10:23:34.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:34.755+0000 7f8fb77ad640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb010d990 con 0x7f8fb0107d10 2024-04-23T10:23:34.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:34.755+0000 7f8fb77ad640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb010dc90 con 0x7f8fb0102ab0 2024-04-23T10:23:34.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:34.755+0000 7f8fb4d48640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fb01053e0 0x7f8fb01987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:34.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:34.755+0000 7f8fb4d48640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fb01053e0 0x7f8fb01987c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55784/0 (socket says 172.21.15.97:55784) 2024-04-23T10:23:34.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:34.755+0000 7f8fb4d48640 1 -- 172.21.15.97:0/176743110 learned_addr learned my addr 172.21.15.97:0/176743110 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:23:34.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:34.755+0000 7f8fb5d4a640 1 --2- 172.21.15.97:0/176743110 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8fb0107d10 0x7f8fb019f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:34.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:34.755+0000 7f8fb5549640 1 --2- 172.21.15.97:0/176743110 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8fb0102ab0 0x7f8fb0198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:34.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:34.755+0000 7f8fb5d4a640 1 -- 172.21.15.97:0/176743110 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8fb0102ab0 msgr2=0x7f8fb0198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:34.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:34.755+0000 7f8fb5d4a640 1 --2- 172.21.15.97:0/176743110 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8fb0102ab0 0x7f8fb0198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:34.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:34.755+0000 7f8fb5d4a640 1 -- 172.21.15.97:0/176743110 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fb01053e0 msgr2=0x7f8fb01987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:34.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:34.755+0000 7f8fb5d4a640 1 --2- 172.21.15.97:0/176743110 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fb01053e0 0x7f8fb01987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:34.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:34.755+0000 7f8fb5d4a640 1 -- 172.21.15.97:0/176743110 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fb019fea0 con 0x7f8fb0107d10 2024-04-23T10:23:34.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:34.755+0000 7f8fb4d48640 1 --2- 172.21.15.97:0/176743110 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fb01053e0 0x7f8fb01987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T10:23:34.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:34.755+0000 7f8fb5d4a640 1 --2- 172.21.15.97:0/176743110 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8fb0107d10 0x7f8fb019f830 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8fac00e970 tx=0x7f8fac00ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:23:34.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:34.756+0000 7f8fa67fc640 1 -- 172.21.15.97:0/176743110 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fac00ccc0 con 0x7f8fb0107d10 2024-04-23T10:23:34.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:34.756+0000 7f8fa67fc640 1 -- 172.21.15.97:0/176743110 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fac005020 con 0x7f8fb0107d10 2024-04-23T10:23:34.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:34.756+0000 7f8fb77ad640 1 -- 172.21.15.97:0/176743110 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fb01a0190 con 0x7f8fb0107d10 2024-04-23T10:23:34.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:34.756+0000 7f8fa67fc640 1 -- 172.21.15.97:0/176743110 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fac004a00 con 0x7f8fb0107d10 2024-04-23T10:23:34.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:34.756+0000 7f8fb77ad640 1 -- 172.21.15.97:0/176743110 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fb01a0690 con 0x7f8fb0107d10 2024-04-23T10:23:34.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:34.757+0000 7f8fb77ad640 1 -- 172.21.15.97:0/176743110 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fb00fdf30 con 0x7f8fb0107d10 2024-04-23T10:23:34.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:34.758+0000 7f8fa67fc640 1 -- 172.21.15.97:0/176743110 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8fac004ba0 con 0x7f8fb0107d10 2024-04-23T10:23:34.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:34.758+0000 7f8fa67fc640 1 --2- 172.21.15.97:0/176743110 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f8f9c0a3bb0 0x7f8f9c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:34.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:34.758+0000 7f8fb5549640 1 --2- 172.21.15.97:0/176743110 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f8f9c0a3bb0 0x7f8f9c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:34.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:34.758+0000 7f8fa67fc640 1 -- 172.21.15.97:0/176743110 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f8fac0d01f0 con 0x7f8fb0107d10 2024-04-23T10:23:34.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:34.759+0000 7f8fb5549640 1 --2- 172.21.15.97:0/176743110 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f8f9c0a3bb0 0x7f8f9c0a6070 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8fa8005fa0 tx=0x7f8fa8005ef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:23:34.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:34.762+0000 7f8fa67fc640 1 -- 172.21.15.97:0/176743110 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8fac09cfd0 con 0x7f8fb0107d10 2024-04-23T10:23:34.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:34 smithi097 ceph-mon[29197]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:23:34.861 INFO:journalctl@ceph.mds.j.smithi097.stdout:Apr 23 10:23:34 smithi097 systemd[1]: Starting Ceph mds.j for 9b22af88-0159-11ef-bc93-c7b262605968... 2024-04-23T10:23:34.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:34 smithi183 ceph-mon[32317]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:23:34.993 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:34.994+0000 7f8fb77ad640 1 -- 172.21.15.97:0/176743110 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8fb002d5c0 con 0x7f8fb0107d10 2024-04-23T10:23:34.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:34.995+0000 7f8fa67fc640 1 -- 172.21.15.97:0/176743110 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2224 (secure 0 0 0) 0x7f8fac09c8d0 con 0x7f8fb0107d10 2024-04-23T10:23:34.994 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:23:34.994 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/4292847121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4292847121},{"type":"v1","addr":"172.21.15.183:6833","nonce":4292847121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3885816565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3885816565},{"type":"v1","addr":"172.21.15.196:6833","nonce":3885816565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-23T10:23:34.995 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 3 2024-04-23T10:23:34.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:34.997+0000 7f8fb77ad640 1 -- 172.21.15.97:0/176743110 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f8f9c0a3bb0 msgr2=0x7f8f9c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:34.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:34.997+0000 7f8fb77ad640 1 --2- 172.21.15.97:0/176743110 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f8f9c0a3bb0 0x7f8f9c0a6070 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8fa8005fa0 tx=0x7f8fa8005ef0 comp rx=0 tx=0).stop 2024-04-23T10:23:34.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:34.997+0000 7f8fb77ad640 1 -- 172.21.15.97:0/176743110 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8fb0107d10 msgr2=0x7f8fb019f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:34.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:34.997+0000 7f8fb77ad640 1 --2- 172.21.15.97:0/176743110 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8fb0107d10 0x7f8fb019f830 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8fac00e970 tx=0x7f8fac00ee30 comp rx=0 tx=0).stop 2024-04-23T10:23:34.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:34.997+0000 7f8fb77ad640 1 -- 172.21.15.97:0/176743110 shutdown_connections 2024-04-23T10:23:34.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:34.997+0000 7f8fb77ad640 1 --2- 172.21.15.97:0/176743110 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f8f9c0a3bb0 0x7f8f9c0a6070 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:34.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:34.997+0000 7f8fb77ad640 1 --2- 172.21.15.97:0/176743110 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8fb0107d10 0x7f8fb019f830 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:34.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:34.997+0000 7f8fb77ad640 1 --2- 172.21.15.97:0/176743110 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fb01053e0 0x7f8fb01987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:34.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:34.997+0000 7f8fb77ad640 1 --2- 172.21.15.97:0/176743110 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8fb0102ab0 0x7f8fb0198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:34.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:34.997+0000 7f8fb77ad640 1 -- 172.21.15.97:0/176743110 >> 172.21.15.97:0/176743110 conn(0x7f8fb00fc440 msgr2=0x7f8fb0104950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:23:34.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:34.997+0000 7f8fb77ad640 1 -- 172.21.15.97:0/176743110 shutdown_connections 2024-04-23T10:23:34.997 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:34.997+0000 7f8fb77ad640 1 -- 172.21.15.97:0/176743110 wait complete. 2024-04-23T10:23:35.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:34 smithi196 ceph-mon[32919]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:23:35.171 INFO:journalctl@ceph.mds.j.smithi097.stdout:Apr 23 10:23:34 smithi097 podman[66960]: 2024-04-23 10:23:34.82408735 +0000 UTC m=+0.018402734 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T10:23:35.171 INFO:journalctl@ceph.mds.j.smithi097.stdout:Apr 23 10:23:34 smithi097 podman[66960]: 2024-04-23 10:23:34.936441148 +0000 UTC m=+0.130756505 container create 38fb77b0074693f124f67ae53d2e2400e839ae9f5f6660339270fc33697fed63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-j, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=) 2024-04-23T10:23:35.171 INFO:journalctl@ceph.mds.j.smithi097.stdout:Apr 23 10:23:35 smithi097 podman[66960]: 2024-04-23 10:23:35.172811404 +0000 UTC m=+0.367126766 container init 38fb77b0074693f124f67ae53d2e2400e839ae9f5f6660339270fc33697fed63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-j, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416) 2024-04-23T10:23:35.610 INFO:journalctl@ceph.mds.j.smithi097.stdout:Apr 23 10:23:35 smithi097 podman[66960]: 2024-04-23 10:23:35.175514555 +0000 UTC m=+0.369829913 container start 38fb77b0074693f124f67ae53d2e2400e839ae9f5f6660339270fc33697fed63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-j, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T10:23:35.610 INFO:journalctl@ceph.mds.j.smithi097.stdout:Apr 23 10:23:35 smithi097 ceph-mds[66981]: -- 172.21.15.97:0/4260417025 <== mon.0 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5597a61d1a40 con 0x5597a61a4800 2024-04-23T10:23:35.611 INFO:journalctl@ceph.mds.j.smithi097.stdout:Apr 23 10:23:35 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-j[66977]: starting mds.j at 2024-04-23T10:23:35.611 INFO:journalctl@ceph.mds.j.smithi097.stdout:Apr 23 10:23:35 smithi097 bash[66960]: 38fb77b0074693f124f67ae53d2e2400e839ae9f5f6660339270fc33697fed63 2024-04-23T10:23:35.611 INFO:journalctl@ceph.mds.j.smithi097.stdout:Apr 23 10:23:35 smithi097 systemd[1]: Started Ceph mds.j for 9b22af88-0159-11ef-bc93-c7b262605968. 2024-04-23T10:23:35.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:35 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/176743110' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:23:35.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:35 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:35.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:35 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:35.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:35 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:35.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:35 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T10:23:35.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:35 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T10:23:35.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:35 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:23:35.977 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:35 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/176743110' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:23:35.977 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:35 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:35.977 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:35 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:35.978 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:35 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:35.978 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:35 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T10:23:35.978 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:35 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T10:23:35.978 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:35 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:23:36.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:35 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/176743110' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:23:36.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:35 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:36.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:35 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:36.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:35 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:36.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:35 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T10:23:36.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:35 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T10:23:36.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:35 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:23:36.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:36 smithi183 ceph-mon[32317]: Deploying daemon mds.d on smithi097 2024-04-23T10:23:36.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:36 smithi183 ceph-mon[32317]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:23:36.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:36 smithi183 ceph-mon[32317]: mds.? [v2:172.21.15.97:6834/3946878762,v1:172.21.15.97:6835/3946878762] up:boot 2024-04-23T10:23:36.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:36 smithi183 ceph-mon[32317]: fsmap 3 up:standby 2024-04-23T10:23:36.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:36 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-23T10:23:37.007 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T10:23:37.009 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:36 smithi097 ceph-mon[29197]: Deploying daemon mds.d on smithi097 2024-04-23T10:23:37.009 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:36 smithi097 ceph-mon[29197]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:23:37.009 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:36 smithi097 ceph-mon[29197]: mds.? [v2:172.21.15.97:6834/3946878762,v1:172.21.15.97:6835/3946878762] up:boot 2024-04-23T10:23:37.009 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:36 smithi097 ceph-mon[29197]: fsmap 3 up:standby 2024-04-23T10:23:37.009 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:36 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-23T10:23:37.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:36 smithi196 ceph-mon[32919]: Deploying daemon mds.d on smithi097 2024-04-23T10:23:37.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:36 smithi196 ceph-mon[32919]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:23:37.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:36 smithi196 ceph-mon[32919]: mds.? [v2:172.21.15.97:6834/3946878762,v1:172.21.15.97:6835/3946878762] up:boot 2024-04-23T10:23:37.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:36 smithi196 ceph-mon[32919]: fsmap 3 up:standby 2024-04-23T10:23:37.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:36 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-23T10:23:37.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:37.113+0000 7f8e5a976640 1 -- 172.21.15.97:0/3996711123 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e54102a70 msgr2=0x7f8e54104e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:37.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:37.113+0000 7f8e5a976640 1 --2- 172.21.15.97:0/3996711123 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e54102a70 0x7f8e54104e60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8e44009900 tx=0x7f8e4402a410 comp rx=0 tx=0).stop 2024-04-23T10:23:37.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:37.113+0000 7f8e5a976640 1 -- 172.21.15.97:0/3996711123 shutdown_connections 2024-04-23T10:23:37.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:37.113+0000 7f8e5a976640 1 --2- 172.21.15.97:0/3996711123 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e54107cd0 0x7f8e5410a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:37.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:37.113+0000 7f8e5a976640 1 --2- 172.21.15.97:0/3996711123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e541053a0 0x7f8e54107790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:37.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:37.113+0000 7f8e5a976640 1 --2- 172.21.15.97:0/3996711123 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e54102a70 0x7f8e54104e60 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:37.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:37.114+0000 7f8e5a976640 1 -- 172.21.15.97:0/3996711123 >> 172.21.15.97:0/3996711123 conn(0x7f8e540fc400 msgr2=0x7f8e540fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:23:37.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:37.114+0000 7f8e5a976640 1 -- 172.21.15.97:0/3996711123 shutdown_connections 2024-04-23T10:23:37.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:37.114+0000 7f8e5a976640 1 -- 172.21.15.97:0/3996711123 wait complete. 2024-04-23T10:23:37.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:37.114+0000 7f8e5a976640 1 Processor -- start 2024-04-23T10:23:37.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:37.114+0000 7f8e5a976640 1 -- start start 2024-04-23T10:23:37.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:37.114+0000 7f8e5a976640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e54102a70 0x7f8e541981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:37.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:37.114+0000 7f8e5a976640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e541053a0 0x7f8e541986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:37.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:37.114+0000 7f8e5a976640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e54107cd0 0x7f8e5419f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:37.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:37.114+0000 7f8e5a976640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5410dac0 con 0x7f8e541053a0 2024-04-23T10:23:37.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:37.114+0000 7f8e5a976640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5410d940 con 0x7f8e54102a70 2024-04-23T10:23:37.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:37.114+0000 7f8e5a976640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5410dc40 con 0x7f8e54107cd0 2024-04-23T10:23:37.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:37.115+0000 7f8e537fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e541053a0 0x7f8e541986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:37.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:37.115+0000 7f8e537fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e541053a0 0x7f8e541986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43142/0 (socket says 172.21.15.97:43142) 2024-04-23T10:23:37.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:37.115+0000 7f8e537fe640 1 -- 172.21.15.97:0/1605510763 learned_addr learned my addr 172.21.15.97:0/1605510763 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:23:37.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:37.115+0000 7f8e58f13640 1 --2- 172.21.15.97:0/1605510763 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e54107cd0 0x7f8e5419f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:37.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:37.115+0000 7f8e53fff640 1 --2- 172.21.15.97:0/1605510763 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e54102a70 0x7f8e541981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:37.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:37.115+0000 7f8e537fe640 1 -- 172.21.15.97:0/1605510763 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e54107cd0 msgr2=0x7f8e5419f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:37.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:37.115+0000 7f8e537fe640 1 --2- 172.21.15.97:0/1605510763 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e54107cd0 0x7f8e5419f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:37.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:37.115+0000 7f8e537fe640 1 -- 172.21.15.97:0/1605510763 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e54102a70 msgr2=0x7f8e541981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:37.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:37.115+0000 7f8e537fe640 1 --2- 172.21.15.97:0/1605510763 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e54102a70 0x7f8e541981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:37.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:37.115+0000 7f8e537fe640 1 -- 172.21.15.97:0/1605510763 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e5419fdc0 con 0x7f8e541053a0 2024-04-23T10:23:37.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:37.115+0000 7f8e53fff640 1 --2- 172.21.15.97:0/1605510763 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e54102a70 0x7f8e541981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:23:37.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:37.115+0000 7f8e537fe640 1 --2- 172.21.15.97:0/1605510763 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e541053a0 0x7f8e541986e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8e4800d770 tx=0x7f8e4800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:23:37.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:37.115+0000 7f8e517fa640 1 -- 172.21.15.97:0/1605510763 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e48017070 con 0x7f8e541053a0 2024-04-23T10:23:37.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:37.115+0000 7f8e517fa640 1 -- 172.21.15.97:0/1605510763 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e48009c30 con 0x7f8e541053a0 2024-04-23T10:23:37.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:37.115+0000 7f8e5a976640 1 -- 172.21.15.97:0/1605510763 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e541a00b0 con 0x7f8e541053a0 2024-04-23T10:23:37.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:37.115+0000 7f8e517fa640 1 -- 172.21.15.97:0/1605510763 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e4800f620 con 0x7f8e541053a0 2024-04-23T10:23:37.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:37.115+0000 7f8e5a976640 1 -- 172.21.15.97:0/1605510763 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e541a0600 con 0x7f8e541053a0 2024-04-23T10:23:37.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:37.116+0000 7f8e5a976640 1 -- 172.21.15.97:0/1605510763 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e540fdef0 con 0x7f8e541053a0 2024-04-23T10:23:37.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:37.117+0000 7f8e517fa640 1 -- 172.21.15.97:0/1605510763 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8e4801a070 con 0x7f8e541053a0 2024-04-23T10:23:37.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:37.118+0000 7f8e517fa640 1 --2- 172.21.15.97:0/1605510763 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f8e3c0a3bb0 0x7f8e3c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:37.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:37.118+0000 7f8e53fff640 1 --2- 172.21.15.97:0/1605510763 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f8e3c0a3bb0 0x7f8e3c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:37.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:37.118+0000 7f8e517fa640 1 -- 172.21.15.97:0/1605510763 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f8e480cfac0 con 0x7f8e541053a0 2024-04-23T10:23:37.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:37.118+0000 7f8e53fff640 1 --2- 172.21.15.97:0/1605510763 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f8e3c0a3bb0 0x7f8e3c0a6070 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8e44006010 tx=0x7f8e44000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:23:37.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:37.123+0000 7f8e517fa640 1 -- 172.21.15.97:0/1605510763 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8e48009de0 con 0x7f8e541053a0 2024-04-23T10:23:37.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:37.360+0000 7f8e5a976640 1 -- 172.21.15.97:0/1605510763 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8e5402d810 con 0x7f8e541053a0 2024-04-23T10:23:37.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:37.361+0000 7f8e517fa640 1 -- 172.21.15.97:0/1605510763 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3049 (secure 0 0 0) 0x7f8e4809c940 con 0x7f8e541053a0 2024-04-23T10:23:37.360 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:23:37.360 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3946878762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3946878762},{"type":"v1","addr":"172.21.15.97:6835","nonce":3946878762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/4292847121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4292847121},{"type":"v1","addr":"172.21.15.183:6833","nonce":4292847121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3885816565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3885816565},{"type":"v1","addr":"172.21.15.196:6833","nonce":3885816565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-23T10:23:37.360 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 4 2024-04-23T10:23:37.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:37.363+0000 7f8e5a976640 1 -- 172.21.15.97:0/1605510763 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f8e3c0a3bb0 msgr2=0x7f8e3c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:37.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:37.363+0000 7f8e5a976640 1 --2- 172.21.15.97:0/1605510763 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f8e3c0a3bb0 0x7f8e3c0a6070 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8e44006010 tx=0x7f8e44000bc0 comp rx=0 tx=0).stop 2024-04-23T10:23:37.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:37.363+0000 7f8e5a976640 1 -- 172.21.15.97:0/1605510763 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e541053a0 msgr2=0x7f8e541986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:37.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:37.363+0000 7f8e5a976640 1 --2- 172.21.15.97:0/1605510763 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e541053a0 0x7f8e541986e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8e4800d770 tx=0x7f8e4800dc30 comp rx=0 tx=0).stop 2024-04-23T10:23:37.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:37.363+0000 7f8e5a976640 1 -- 172.21.15.97:0/1605510763 shutdown_connections 2024-04-23T10:23:37.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:37.363+0000 7f8e5a976640 1 --2- 172.21.15.97:0/1605510763 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f8e3c0a3bb0 0x7f8e3c0a6070 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:37.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:37.363+0000 7f8e5a976640 1 --2- 172.21.15.97:0/1605510763 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e54107cd0 0x7f8e5419f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:37.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:37.363+0000 7f8e5a976640 1 --2- 172.21.15.97:0/1605510763 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e541053a0 0x7f8e541986e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:37.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:37.363+0000 7f8e5a976640 1 --2- 172.21.15.97:0/1605510763 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e54102a70 0x7f8e541981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:37.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:37.363+0000 7f8e5a976640 1 -- 172.21.15.97:0/1605510763 >> 172.21.15.97:0/1605510763 conn(0x7f8e540fc400 msgr2=0x7f8e541033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:23:37.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:37.363+0000 7f8e5a976640 1 -- 172.21.15.97:0/1605510763 shutdown_connections 2024-04-23T10:23:37.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:37.363+0000 7f8e5a976640 1 -- 172.21.15.97:0/1605510763 wait complete. 2024-04-23T10:23:37.610 INFO:journalctl@ceph.mds.d.smithi097.stdout:Apr 23 10:23:37 smithi097 systemd[1]: Starting Ceph mds.d for 9b22af88-0159-11ef-bc93-c7b262605968... 2024-04-23T10:23:37.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:37 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/1605510763' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:23:37.917 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:37 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/1605510763' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:23:37.917 INFO:journalctl@ceph.mds.d.smithi097.stdout:Apr 23 10:23:37 smithi097 podman[67308]: 2024-04-23 10:23:37.665118889 +0000 UTC m=+0.017903411 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T10:23:37.917 INFO:journalctl@ceph.mds.d.smithi097.stdout:Apr 23 10:23:37 smithi097 podman[67308]: 2024-04-23 10:23:37.793430898 +0000 UTC m=+0.146215403 container create ed03f3b8a268626149c3c2a399acd09c3b7e7a13d040c80e1065199c19ed3cfc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-d, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-23T10:23:38.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:37 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/1605510763' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:23:38.611 INFO:journalctl@ceph.mds.d.smithi097.stdout:Apr 23 10:23:38 smithi097 podman[67308]: 2024-04-23 10:23:38.180095004 +0000 UTC m=+0.532879509 container init ed03f3b8a268626149c3c2a399acd09c3b7e7a13d040c80e1065199c19ed3cfc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-d, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-23T10:23:38.611 INFO:journalctl@ceph.mds.d.smithi097.stdout:Apr 23 10:23:38 smithi097 podman[67308]: 2024-04-23 10:23:38.182741057 +0000 UTC m=+0.535525563 container start ed03f3b8a268626149c3c2a399acd09c3b7e7a13d040c80e1065199c19ed3cfc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-d, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T10:23:38.611 INFO:journalctl@ceph.mds.d.smithi097.stdout:Apr 23 10:23:38 smithi097 ceph-mds[67327]: -- 172.21.15.97:0/3733446296 <== mon.0 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bc8ce83a40 con 0x55bc8ce57000 2024-04-23T10:23:38.611 INFO:journalctl@ceph.mds.d.smithi097.stdout:Apr 23 10:23:38 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-d[67323]: starting mds.d at 2024-04-23T10:23:38.611 INFO:journalctl@ceph.mds.d.smithi097.stdout:Apr 23 10:23:38 smithi097 bash[67308]: ed03f3b8a268626149c3c2a399acd09c3b7e7a13d040c80e1065199c19ed3cfc 2024-04-23T10:23:38.611 INFO:journalctl@ceph.mds.d.smithi097.stdout:Apr 23 10:23:38 smithi097 systemd[1]: Started Ceph mds.d for 9b22af88-0159-11ef-bc93-c7b262605968. 2024-04-23T10:23:38.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:38 smithi183 ceph-mon[32317]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:23:38.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:38 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:38.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:38 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:38.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:38 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:38.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:38 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T10:23:38.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:38 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T10:23:38.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:38 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:23:39.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:38 smithi196 ceph-mon[32919]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:23:39.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:38 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:39.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:38 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:39.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:38 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:39.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:38 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T10:23:39.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:38 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T10:23:39.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:38 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:23:39.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:38 smithi097 ceph-mon[29197]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:23:39.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:38 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:39.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:38 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:39.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:38 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:39.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:38 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T10:23:39.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:38 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T10:23:39.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:38 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:23:39.374 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T10:23:39.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:39.489+0000 7f29017a2640 1 -- 172.21.15.97:0/2265094813 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28fc1031b0 msgr2=0x7f28fc103630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:39.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:39.489+0000 7f29017a2640 1 --2- 172.21.15.97:0/2265094813 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28fc1031b0 0x7f28fc103630 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f28f0009900 tx=0x7f28f002a410 comp rx=0 tx=0).stop 2024-04-23T10:23:39.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:39.489+0000 7f29017a2640 1 -- 172.21.15.97:0/2265094813 shutdown_connections 2024-04-23T10:23:39.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:39.489+0000 7f29017a2640 1 --2- 172.21.15.97:0/2265094813 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28fc103b70 0x7f28fc10a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:39.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:39.489+0000 7f29017a2640 1 --2- 172.21.15.97:0/2265094813 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28fc1031b0 0x7f28fc103630 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:39.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:39.489+0000 7f29017a2640 1 --2- 172.21.15.97:0/2265094813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f28fc101fc0 0x7f28fc1023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:39.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:39.489+0000 7f29017a2640 1 -- 172.21.15.97:0/2265094813 >> 172.21.15.97:0/2265094813 conn(0x7f28fc0fd550 msgr2=0x7f28fc0ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:23:39.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:39.489+0000 7f29017a2640 1 -- 172.21.15.97:0/2265094813 shutdown_connections 2024-04-23T10:23:39.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:39.489+0000 7f29017a2640 1 -- 172.21.15.97:0/2265094813 wait complete. 2024-04-23T10:23:39.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:39.490+0000 7f29017a2640 1 Processor -- start 2024-04-23T10:23:39.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:39.490+0000 7f29017a2640 1 -- start start 2024-04-23T10:23:39.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:39.490+0000 7f29017a2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f28fc101fc0 0x7f28fc075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:39.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:39.490+0000 7f29017a2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28fc1031b0 0x7f28fc074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:39.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:39.490+0000 7f29017a2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28fc103b70 0x7f28fc074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:39.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:39.490+0000 7f29017a2640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f28fc10db00 con 0x7f28fc1031b0 2024-04-23T10:23:39.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:39.490+0000 7f29017a2640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f28fc10d980 con 0x7f28fc101fc0 2024-04-23T10:23:39.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:39.490+0000 7f29017a2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f28fc10dc80 con 0x7f28fc103b70 2024-04-23T10:23:39.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:39.490+0000 7f28fa7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28fc1031b0 0x7f28fc074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:39.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:39.490+0000 7f28fa7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28fc1031b0 0x7f28fc074340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43170/0 (socket says 172.21.15.97:43170) 2024-04-23T10:23:39.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:39.490+0000 7f28fa7fc640 1 -- 172.21.15.97:0/3056106133 learned_addr learned my addr 172.21.15.97:0/3056106133 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:23:39.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:39.491+0000 7f28faffd640 1 --2- 172.21.15.97:0/3056106133 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f28fc101fc0 0x7f28fc075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:39.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:39.491+0000 7f28fb7fe640 1 --2- 172.21.15.97:0/3056106133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28fc103b70 0x7f28fc074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:39.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:39.491+0000 7f28fa7fc640 1 -- 172.21.15.97:0/3056106133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28fc103b70 msgr2=0x7f28fc074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:39.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:39.491+0000 7f28fa7fc640 1 --2- 172.21.15.97:0/3056106133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28fc103b70 0x7f28fc074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:39.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:39.491+0000 7f28fa7fc640 1 -- 172.21.15.97:0/3056106133 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f28fc101fc0 msgr2=0x7f28fc075d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:39.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:39.491+0000 7f28fa7fc640 1 --2- 172.21.15.97:0/3056106133 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f28fc101fc0 0x7f28fc075d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:39.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:39.491+0000 7f28fa7fc640 1 -- 172.21.15.97:0/3056106133 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28fc075110 con 0x7f28fc1031b0 2024-04-23T10:23:39.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:39.491+0000 7f28faffd640 1 --2- 172.21.15.97:0/3056106133 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f28fc101fc0 0x7f28fc075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T10:23:39.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:39.491+0000 7f28fa7fc640 1 --2- 172.21.15.97:0/3056106133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28fc1031b0 0x7f28fc074340 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f28f0005940 tx=0x7f28f0031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:23:39.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:39.491+0000 7f28e3fff640 1 -- 172.21.15.97:0/3056106133 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f003f030 con 0x7f28fc1031b0 2024-04-23T10:23:39.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:39.491+0000 7f28e3fff640 1 -- 172.21.15.97:0/3056106133 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28f0003aa0 con 0x7f28fc1031b0 2024-04-23T10:23:39.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:39.491+0000 7f28e3fff640 1 -- 172.21.15.97:0/3056106133 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f0035070 con 0x7f28fc1031b0 2024-04-23T10:23:39.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:39.491+0000 7f29017a2640 1 -- 172.21.15.97:0/3056106133 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28fc1a2310 con 0x7f28fc1031b0 2024-04-23T10:23:39.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:39.491+0000 7f29017a2640 1 -- 172.21.15.97:0/3056106133 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28fc1a2810 con 0x7f28fc1031b0 2024-04-23T10:23:39.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:39.492+0000 7f29017a2640 1 -- 172.21.15.97:0/3056106133 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28fc109360 con 0x7f28fc1031b0 2024-04-23T10:23:39.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:39.493+0000 7f28e3fff640 1 -- 172.21.15.97:0/3056106133 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f28f0042030 con 0x7f28fc1031b0 2024-04-23T10:23:39.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:39.494+0000 7f28e3fff640 1 --2- 172.21.15.97:0/3056106133 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f28e40a3bb0 0x7f28e40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:39.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:39.494+0000 7f28faffd640 1 --2- 172.21.15.97:0/3056106133 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f28e40a3bb0 0x7f28e40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:39.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:39.494+0000 7f28e3fff640 1 -- 172.21.15.97:0/3056106133 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f28f00f05a0 con 0x7f28fc1031b0 2024-04-23T10:23:39.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:39.494+0000 7f28faffd640 1 --2- 172.21.15.97:0/3056106133 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f28e40a3bb0 0x7f28e40a6070 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f28fc103010 tx=0x7f28ec005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:23:39.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:39.498+0000 7f28e3fff640 1 -- 172.21.15.97:0/3056106133 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f28f00bd420 con 0x7f28fc1031b0 2024-04-23T10:23:39.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:39.721+0000 7f29017a2640 1 -- 172.21.15.97:0/3056106133 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f28fc02d810 con 0x7f28fc1031b0 2024-04-23T10:23:39.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:39.722+0000 7f28e3fff640 1 -- 172.21.15.97:0/3056106133 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3871 (secure 0 0 0) 0x7f28f00bcd20 con 0x7f28fc1031b0 2024-04-23T10:23:39.721 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:23:39.721 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3946878762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3946878762},{"type":"v1","addr":"172.21.15.97:6835","nonce":3946878762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/209800861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":209800861},{"type":"v1","addr":"172.21.15.97:6837","nonce":209800861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/4292847121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4292847121},{"type":"v1","addr":"172.21.15.183:6833","nonce":4292847121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3885816565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3885816565},{"type":"v1","addr":"172.21.15.196:6833","nonce":3885816565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-23T10:23:39.722 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 5 2024-04-23T10:23:39.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:39.724+0000 7f29017a2640 1 -- 172.21.15.97:0/3056106133 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f28e40a3bb0 msgr2=0x7f28e40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:39.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:39.724+0000 7f29017a2640 1 --2- 172.21.15.97:0/3056106133 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f28e40a3bb0 0x7f28e40a6070 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f28fc103010 tx=0x7f28ec005f90 comp rx=0 tx=0).stop 2024-04-23T10:23:39.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:39.724+0000 7f29017a2640 1 -- 172.21.15.97:0/3056106133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28fc1031b0 msgr2=0x7f28fc074340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:39.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:39.724+0000 7f29017a2640 1 --2- 172.21.15.97:0/3056106133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28fc1031b0 0x7f28fc074340 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f28f0005940 tx=0x7f28f0031a50 comp rx=0 tx=0).stop 2024-04-23T10:23:39.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:39.724+0000 7f29017a2640 1 -- 172.21.15.97:0/3056106133 shutdown_connections 2024-04-23T10:23:39.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:39.724+0000 7f29017a2640 1 --2- 172.21.15.97:0/3056106133 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f28e40a3bb0 0x7f28e40a6070 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:39.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:39.724+0000 7f29017a2640 1 --2- 172.21.15.97:0/3056106133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28fc103b70 0x7f28fc074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:39.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:39.724+0000 7f29017a2640 1 --2- 172.21.15.97:0/3056106133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28fc1031b0 0x7f28fc074340 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:39.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:39.724+0000 7f29017a2640 1 --2- 172.21.15.97:0/3056106133 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f28fc101fc0 0x7f28fc075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:39.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:39.724+0000 7f29017a2640 1 -- 172.21.15.97:0/3056106133 >> 172.21.15.97:0/3056106133 conn(0x7f28fc0fd550 msgr2=0x7f28fc106530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:23:39.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:39.724+0000 7f29017a2640 1 -- 172.21.15.97:0/3056106133 shutdown_connections 2024-04-23T10:23:39.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:39.724+0000 7f29017a2640 1 -- 172.21.15.97:0/3056106133 wait complete. 2024-04-23T10:23:39.733 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:39 smithi097 ceph-mon[29197]: Deploying daemon mds.e on smithi183 2024-04-23T10:23:39.733 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:39 smithi097 ceph-mon[29197]: mds.? [v2:172.21.15.97:6836/209800861,v1:172.21.15.97:6837/209800861] up:boot 2024-04-23T10:23:39.733 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:39 smithi097 ceph-mon[29197]: fsmap 4 up:standby 2024-04-23T10:23:39.733 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:39 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-23T10:23:39.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:39 smithi183 ceph-mon[32317]: Deploying daemon mds.e on smithi183 2024-04-23T10:23:39.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:39 smithi183 ceph-mon[32317]: mds.? [v2:172.21.15.97:6836/209800861,v1:172.21.15.97:6837/209800861] up:boot 2024-04-23T10:23:39.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:39 smithi183 ceph-mon[32317]: fsmap 4 up:standby 2024-04-23T10:23:39.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:39 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-23T10:23:40.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:39 smithi196 ceph-mon[32919]: Deploying daemon mds.e on smithi183 2024-04-23T10:23:40.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:39 smithi196 ceph-mon[32919]: mds.? [v2:172.21.15.97:6836/209800861,v1:172.21.15.97:6837/209800861] up:boot 2024-04-23T10:23:40.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:39 smithi196 ceph-mon[32919]: fsmap 4 up:standby 2024-04-23T10:23:40.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:39 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-23T10:23:40.640 INFO:journalctl@ceph.mds.e.smithi183.stdout:Apr 23 10:23:40 smithi183 systemd[1]: Starting Ceph mds.e for 9b22af88-0159-11ef-bc93-c7b262605968... 2024-04-23T10:23:40.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:40 smithi183 ceph-mon[32317]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:23:40.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:40 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/3056106133' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:23:40.900 INFO:journalctl@ceph.mds.e.smithi183.stdout:Apr 23 10:23:40 smithi183 podman[61935]: 2024-04-23 10:23:40.685943707 +0000 UTC m=+0.017768073 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T10:23:40.900 INFO:journalctl@ceph.mds.e.smithi183.stdout:Apr 23 10:23:40 smithi183 podman[61935]: 2024-04-23 10:23:40.808390574 +0000 UTC m=+0.140214912 container create bb92e4a07cf02d4aa12286edcb23a645d2053ab8f73de3af08b7c426cbb882a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-e, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2) 2024-04-23T10:23:40.993 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:40 smithi196 ceph-mon[32919]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:23:40.993 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:40 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/3056106133' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:23:41.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:40 smithi097 ceph-mon[29197]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:23:41.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:40 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/3056106133' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:23:41.400 INFO:journalctl@ceph.mds.e.smithi183.stdout:Apr 23 10:23:41 smithi183 podman[61935]: 2024-04-23 10:23:41.02795125 +0000 UTC m=+0.359775588 container init bb92e4a07cf02d4aa12286edcb23a645d2053ab8f73de3af08b7c426cbb882a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-e, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T10:23:41.400 INFO:journalctl@ceph.mds.e.smithi183.stdout:Apr 23 10:23:41 smithi183 podman[61935]: 2024-04-23 10:23:41.030508232 +0000 UTC m=+0.362332576 container start bb92e4a07cf02d4aa12286edcb23a645d2053ab8f73de3af08b7c426cbb882a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-e, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T10:23:41.400 INFO:journalctl@ceph.mds.e.smithi183.stdout:Apr 23 10:23:41 smithi183 ceph-mds[61954]: -- 172.21.15.183:0/3495872769 <== mon.2 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dc3b523a40 con 0x55dc3b4f7000 2024-04-23T10:23:41.400 INFO:journalctl@ceph.mds.e.smithi183.stdout:Apr 23 10:23:41 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-e[61950]: starting mds.e at 2024-04-23T10:23:41.400 INFO:journalctl@ceph.mds.e.smithi183.stdout:Apr 23 10:23:41 smithi183 bash[61935]: bb92e4a07cf02d4aa12286edcb23a645d2053ab8f73de3af08b7c426cbb882a1 2024-04-23T10:23:41.401 INFO:journalctl@ceph.mds.e.smithi183.stdout:Apr 23 10:23:41 smithi183 systemd[1]: Started Ceph mds.e for 9b22af88-0159-11ef-bc93-c7b262605968. 2024-04-23T10:23:41.735 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T10:23:41.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:41.846+0000 7f17f21f7640 1 -- 172.21.15.97:0/2362466857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17ec102a00 msgr2=0x7f17ec109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:41.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:41.846+0000 7f17f21f7640 1 --2- 172.21.15.97:0/2362466857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17ec102a00 0x7f17ec109320 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f17e4009ca0 tx=0x7f17e402a8c0 comp rx=0 tx=0).stop 2024-04-23T10:23:41.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:41.846+0000 7f17f21f7640 1 -- 172.21.15.97:0/2362466857 shutdown_connections 2024-04-23T10:23:41.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:41.846+0000 7f17f21f7640 1 --2- 172.21.15.97:0/2362466857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17ec102a00 0x7f17ec109320 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:41.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:41.846+0000 7f17f21f7640 1 --2- 172.21.15.97:0/2362466857 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17ec102040 0x7f17ec1024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:41.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:41.846+0000 7f17f21f7640 1 --2- 172.21.15.97:0/2362466857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17ec100e60 0x7f17ec101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:41.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:41.846+0000 7f17f21f7640 1 -- 172.21.15.97:0/2362466857 >> 172.21.15.97:0/2362466857 conn(0x7f17ec0fc440 msgr2=0x7f17ec0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:23:41.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:41.846+0000 7f17f21f7640 1 -- 172.21.15.97:0/2362466857 shutdown_connections 2024-04-23T10:23:41.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:41.846+0000 7f17f21f7640 1 -- 172.21.15.97:0/2362466857 wait complete. 2024-04-23T10:23:41.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:41.846+0000 7f17f21f7640 1 Processor -- start 2024-04-23T10:23:41.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:41.847+0000 7f17f21f7640 1 -- start start 2024-04-23T10:23:41.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:41.847+0000 7f17f21f7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17ec100e60 0x7f17ec104e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:41.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:41.847+0000 7f17f21f7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17ec102040 0x7f17ec1034b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:41.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:41.847+0000 7f17f21f7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17ec102a00 0x7f17ec1039f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:41.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:41.847+0000 7f17f21f7640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f17ec10ca10 con 0x7f17ec102040 2024-04-23T10:23:41.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:41.847+0000 7f17f21f7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f17ec10c890 con 0x7f17ec100e60 2024-04-23T10:23:41.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:41.847+0000 7f17f21f7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f17ec10cb90 con 0x7f17ec102a00 2024-04-23T10:23:41.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:41.847+0000 7f17eaffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17ec102040 0x7f17ec1034b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:41.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:41.847+0000 7f17eaffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17ec102040 0x7f17ec1034b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43182/0 (socket says 172.21.15.97:43182) 2024-04-23T10:23:41.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:41.847+0000 7f17eaffd640 1 -- 172.21.15.97:0/3900186268 learned_addr learned my addr 172.21.15.97:0/3900186268 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:23:41.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:41.847+0000 7f17eb7fe640 1 --2- 172.21.15.97:0/3900186268 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17ec100e60 0x7f17ec104e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:41.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:41.847+0000 7f17ebfff640 1 --2- 172.21.15.97:0/3900186268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17ec102a00 0x7f17ec1039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:41.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:41.847+0000 7f17eaffd640 1 -- 172.21.15.97:0/3900186268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17ec102a00 msgr2=0x7f17ec1039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:41.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:41.847+0000 7f17eaffd640 1 --2- 172.21.15.97:0/3900186268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17ec102a00 0x7f17ec1039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:41.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:41.847+0000 7f17eaffd640 1 -- 172.21.15.97:0/3900186268 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17ec100e60 msgr2=0x7f17ec104e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:41.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:41.847+0000 7f17eaffd640 1 --2- 172.21.15.97:0/3900186268 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17ec100e60 0x7f17ec104e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:41.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:41.847+0000 7f17eaffd640 1 -- 172.21.15.97:0/3900186268 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17ec104280 con 0x7f17ec102040 2024-04-23T10:23:41.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:41.847+0000 7f17eaffd640 1 --2- 172.21.15.97:0/3900186268 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17ec102040 0x7f17ec1034b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f17e000d950 tx=0x7f17e000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:23:41.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:41.847+0000 7f17e8ff9640 1 -- 172.21.15.97:0/3900186268 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e0017070 con 0x7f17ec102040 2024-04-23T10:23:41.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:41.847+0000 7f17f21f7640 1 -- 172.21.15.97:0/3900186268 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17ec1a2360 con 0x7f17ec102040 2024-04-23T10:23:41.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:41.848+0000 7f17e8ff9640 1 -- 172.21.15.97:0/3900186268 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17e0009c30 con 0x7f17ec102040 2024-04-23T10:23:41.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:41.848+0000 7f17e8ff9640 1 -- 172.21.15.97:0/3900186268 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e000f620 con 0x7f17ec102040 2024-04-23T10:23:41.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:41.848+0000 7f17f21f7640 1 -- 172.21.15.97:0/3900186268 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17ec1a2900 con 0x7f17ec102040 2024-04-23T10:23:41.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:41.849+0000 7f17f21f7640 1 -- 172.21.15.97:0/3900186268 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17ec1a2c20 con 0x7f17ec102040 2024-04-23T10:23:41.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:41.850+0000 7f17e8ff9640 1 -- 172.21.15.97:0/3900186268 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f17e0010460 con 0x7f17ec102040 2024-04-23T10:23:41.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:41.850+0000 7f17e8ff9640 1 --2- 172.21.15.97:0/3900186268 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f17d40a3bb0 0x7f17d40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:41.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:41.850+0000 7f17e8ff9640 1 -- 172.21.15.97:0/3900186268 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f17e00cfae0 con 0x7f17ec102040 2024-04-23T10:23:41.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:41.850+0000 7f17eb7fe640 1 --2- 172.21.15.97:0/3900186268 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f17d40a3bb0 0x7f17d40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:41.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:41.851+0000 7f17eb7fe640 1 --2- 172.21.15.97:0/3900186268 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f17d40a3bb0 0x7f17d40a6070 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f17ec101bb0 tx=0x7f17dc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:23:41.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:41.855+0000 7f17e8ff9640 1 -- 172.21.15.97:0/3900186268 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f17e0009de0 con 0x7f17ec102040 2024-04-23T10:23:42.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:42.069+0000 7f17f21f7640 1 -- 172.21.15.97:0/3900186268 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f17ec1a2df0 con 0x7f17ec102040 2024-04-23T10:23:42.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:42.070+0000 7f17e8ff9640 1 -- 172.21.15.97:0/3900186268 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4699 (secure 0 0 0) 0x7f17e009c310 con 0x7f17ec102040 2024-04-23T10:23:42.069 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:23:42.069 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3946878762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3946878762},{"type":"v1","addr":"172.21.15.97:6835","nonce":3946878762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/209800861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":209800861},{"type":"v1","addr":"172.21.15.97:6837","nonce":209800861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/4292847121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4292847121},{"type":"v1","addr":"172.21.15.183:6833","nonce":4292847121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3885816565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3885816565},{"type":"v1","addr":"172.21.15.196:6833","nonce":3885816565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1897093105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1897093105},{"type":"v1","addr":"172.21.15.183:6835","nonce":1897093105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-23T10:23:42.069 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 6 2024-04-23T10:23:42.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:42.072+0000 7f17f21f7640 1 -- 172.21.15.97:0/3900186268 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f17d40a3bb0 msgr2=0x7f17d40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:42.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:42.072+0000 7f17f21f7640 1 --2- 172.21.15.97:0/3900186268 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f17d40a3bb0 0x7f17d40a6070 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f17ec101bb0 tx=0x7f17dc005f90 comp rx=0 tx=0).stop 2024-04-23T10:23:42.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:42.072+0000 7f17f21f7640 1 -- 172.21.15.97:0/3900186268 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17ec102040 msgr2=0x7f17ec1034b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:42.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:42.072+0000 7f17f21f7640 1 --2- 172.21.15.97:0/3900186268 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17ec102040 0x7f17ec1034b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f17e000d950 tx=0x7f17e000de10 comp rx=0 tx=0).stop 2024-04-23T10:23:42.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:42.072+0000 7f17f21f7640 1 -- 172.21.15.97:0/3900186268 shutdown_connections 2024-04-23T10:23:42.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:42.072+0000 7f17f21f7640 1 --2- 172.21.15.97:0/3900186268 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f17d40a3bb0 0x7f17d40a6070 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:42.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:42.072+0000 7f17f21f7640 1 --2- 172.21.15.97:0/3900186268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17ec102a00 0x7f17ec1039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:42.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:42.072+0000 7f17f21f7640 1 --2- 172.21.15.97:0/3900186268 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17ec102040 0x7f17ec1034b0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:42.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:42.072+0000 7f17f21f7640 1 --2- 172.21.15.97:0/3900186268 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17ec100e60 0x7f17ec104e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:42.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:42.072+0000 7f17f21f7640 1 -- 172.21.15.97:0/3900186268 >> 172.21.15.97:0/3900186268 conn(0x7f17ec0fc440 msgr2=0x7f17ec0fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:23:42.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:42.072+0000 7f17f21f7640 1 -- 172.21.15.97:0/3900186268 shutdown_connections 2024-04-23T10:23:42.072 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:42.072+0000 7f17f21f7640 1 -- 172.21.15.97:0/3900186268 wait complete. 2024-04-23T10:23:42.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:42 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:42.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:42 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:42.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:42 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:42.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:42 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T10:23:42.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:42 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T10:23:42.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:42 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:23:42.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:42 smithi097 ceph-mon[29197]: Deploying daemon mds.f on smithi196 2024-04-23T10:23:42.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:42 smithi097 ceph-mon[29197]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:23:42.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:42 smithi097 ceph-mon[29197]: mds.? [v2:172.21.15.183:6834/1897093105,v1:172.21.15.183:6835/1897093105] up:boot 2024-04-23T10:23:42.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:42 smithi097 ceph-mon[29197]: fsmap 5 up:standby 2024-04-23T10:23:42.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:42 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-23T10:23:42.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:42 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/3900186268' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:23:42.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:42 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:42.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:42 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:42.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:42 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:42.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:42 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T10:23:42.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:42 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T10:23:42.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:42 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:23:42.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:42 smithi196 ceph-mon[32919]: Deploying daemon mds.f on smithi196 2024-04-23T10:23:42.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:42 smithi196 ceph-mon[32919]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:23:42.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:42 smithi196 ceph-mon[32919]: mds.? [v2:172.21.15.183:6834/1897093105,v1:172.21.15.183:6835/1897093105] up:boot 2024-04-23T10:23:42.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:42 smithi196 ceph-mon[32919]: fsmap 5 up:standby 2024-04-23T10:23:42.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:42 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-23T10:23:42.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:42 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/3900186268' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:23:42.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:42 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:42.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:42 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:42.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:42 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:42.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:42 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T10:23:42.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:42 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T10:23:42.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:42 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:23:42.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:42 smithi183 ceph-mon[32317]: Deploying daemon mds.f on smithi196 2024-04-23T10:23:42.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:42 smithi183 ceph-mon[32317]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:23:42.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:42 smithi183 ceph-mon[32317]: mds.? [v2:172.21.15.183:6834/1897093105,v1:172.21.15.183:6835/1897093105] up:boot 2024-04-23T10:23:42.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:42 smithi183 ceph-mon[32317]: fsmap 5 up:standby 2024-04-23T10:23:42.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:42 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-23T10:23:42.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:42 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/3900186268' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:23:43.586 INFO:journalctl@ceph.mds.f.smithi196.stdout:Apr 23 10:23:43 smithi196 systemd[1]: Starting Ceph mds.f for 9b22af88-0159-11ef-bc93-c7b262605968... 2024-04-23T10:23:43.841 INFO:journalctl@ceph.mds.f.smithi196.stdout:Apr 23 10:23:43 smithi196 podman[63716]: 2024-04-23 10:23:43.491065841 +0000 UTC m=+0.018298100 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T10:23:43.842 INFO:journalctl@ceph.mds.f.smithi196.stdout:Apr 23 10:23:43 smithi196 podman[63716]: 2024-04-23 10:23:43.603905467 +0000 UTC m=+0.131137775 container create 5fa315c4ec002895b34791b34d289501b9c4aa95171e7ba48c6931bc9b2a3520 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-f, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T10:23:44.083 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T10:23:44.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:44.193+0000 7fce9aa8a640 1 -- 172.21.15.97:0/2317646523 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce94100e90 msgr2=0x7fce941012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:44.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:44.193+0000 7fce9aa8a640 1 --2- 172.21.15.97:0/2317646523 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce94100e90 0x7fce941012b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fce88009900 tx=0x7fce8802a410 comp rx=0 tx=0).stop 2024-04-23T10:23:44.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:44.193+0000 7fce9aa8a640 1 -- 172.21.15.97:0/2317646523 shutdown_connections 2024-04-23T10:23:44.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:44.193+0000 7fce9aa8a640 1 --2- 172.21.15.97:0/2317646523 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce941029a0 0x7fce941092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:44.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:44.193+0000 7fce9aa8a640 1 --2- 172.21.15.97:0/2317646523 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce94101fe0 0x7fce94102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:44.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:44.193+0000 7fce9aa8a640 1 --2- 172.21.15.97:0/2317646523 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce94100e90 0x7fce941012b0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:44.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:44.193+0000 7fce9aa8a640 1 -- 172.21.15.97:0/2317646523 >> 172.21.15.97:0/2317646523 conn(0x7fce940fc400 msgr2=0x7fce940fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:23:44.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:44.193+0000 7fce9aa8a640 1 -- 172.21.15.97:0/2317646523 shutdown_connections 2024-04-23T10:23:44.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:44.193+0000 7fce9aa8a640 1 -- 172.21.15.97:0/2317646523 wait complete. 2024-04-23T10:23:44.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:44.193+0000 7fce9aa8a640 1 Processor -- start 2024-04-23T10:23:44.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:44.193+0000 7fce9aa8a640 1 -- start start 2024-04-23T10:23:44.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:44.193+0000 7fce9aa8a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce94100e90 0x7fce94198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:44.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:44.193+0000 7fce9aa8a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce94101fe0 0x7fce94198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:44.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:44.193+0000 7fce9aa8a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce941029a0 0x7fce9419f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:44.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:44.193+0000 7fce9aa8a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fce9410c9c0 con 0x7fce941029a0 2024-04-23T10:23:44.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:44.193+0000 7fce9aa8a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fce9410c840 con 0x7fce94101fe0 2024-04-23T10:23:44.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:44.193+0000 7fce9aa8a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fce9410cb40 con 0x7fce94100e90 2024-04-23T10:23:44.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:44.193+0000 7fce99027640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce941029a0 0x7fce9419f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:44.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:44.194+0000 7fce99027640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce941029a0 0x7fce9419f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43204/0 (socket says 172.21.15.97:43204) 2024-04-23T10:23:44.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:44.194+0000 7fce99027640 1 -- 172.21.15.97:0/1028209859 learned_addr learned my addr 172.21.15.97:0/1028209859 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:23:44.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:44.194+0000 7fce98826640 1 --2- 172.21.15.97:0/1028209859 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce94100e90 0x7fce94198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:44.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:44.194+0000 7fce93fff640 1 --2- 172.21.15.97:0/1028209859 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce94101fe0 0x7fce94198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:44.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:44.194+0000 7fce99027640 1 -- 172.21.15.97:0/1028209859 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce94100e90 msgr2=0x7fce94198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:44.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:44.194+0000 7fce99027640 1 --2- 172.21.15.97:0/1028209859 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce94100e90 0x7fce94198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:44.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:44.194+0000 7fce99027640 1 -- 172.21.15.97:0/1028209859 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce94101fe0 msgr2=0x7fce94198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:44.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:44.194+0000 7fce99027640 1 --2- 172.21.15.97:0/1028209859 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce94101fe0 0x7fce94198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:44.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:44.194+0000 7fce99027640 1 -- 172.21.15.97:0/1028209859 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce9419fe60 con 0x7fce941029a0 2024-04-23T10:23:44.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:44.194+0000 7fce93fff640 1 --2- 172.21.15.97:0/1028209859 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce94101fe0 0x7fce94198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:23:44.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:44.194+0000 7fce99027640 1 --2- 172.21.15.97:0/1028209859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce941029a0 0x7fce9419f7f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fce8c00eb10 tx=0x7fce8c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:23:44.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:44.194+0000 7fce91ffb640 1 -- 172.21.15.97:0/1028209859 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce8c004b20 con 0x7fce941029a0 2024-04-23T10:23:44.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:44.194+0000 7fce9aa8a640 1 -- 172.21.15.97:0/1028209859 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce941a0150 con 0x7fce941029a0 2024-04-23T10:23:44.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:44.194+0000 7fce91ffb640 1 -- 172.21.15.97:0/1028209859 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce8c004cc0 con 0x7fce941029a0 2024-04-23T10:23:44.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:44.194+0000 7fce91ffb640 1 -- 172.21.15.97:0/1028209859 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce8c010670 con 0x7fce941029a0 2024-04-23T10:23:44.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:44.194+0000 7fce9aa8a640 1 -- 172.21.15.97:0/1028209859 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce941a0650 con 0x7fce941029a0 2024-04-23T10:23:44.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:44.195+0000 7fce9aa8a640 1 -- 172.21.15.97:0/1028209859 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce941a0910 con 0x7fce941029a0 2024-04-23T10:23:44.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:44.196+0000 7fce91ffb640 1 -- 172.21.15.97:0/1028209859 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fce8c005020 con 0x7fce941029a0 2024-04-23T10:23:44.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:44.197+0000 7fce91ffb640 1 --2- 172.21.15.97:0/1028209859 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fce840a3bb0 0x7fce840a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:44.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:44.197+0000 7fce91ffb640 1 -- 172.21.15.97:0/1028209859 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fce8c0d0500 con 0x7fce941029a0 2024-04-23T10:23:44.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:44.197+0000 7fce98826640 1 --2- 172.21.15.97:0/1028209859 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fce840a3bb0 0x7fce840a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:44.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:44.197+0000 7fce98826640 1 --2- 172.21.15.97:0/1028209859 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fce840a3bb0 0x7fce840a6070 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fce88000c00 tx=0x7fce880314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:23:44.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:44.201+0000 7fce91ffb640 1 -- 172.21.15.97:0/1028209859 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fce8c09d380 con 0x7fce941029a0 2024-04-23T10:23:44.342 INFO:journalctl@ceph.mds.f.smithi196.stdout:Apr 23 10:23:43 smithi196 podman[63716]: 2024-04-23 10:23:43.860166031 +0000 UTC m=+0.387398277 container init 5fa315c4ec002895b34791b34d289501b9c4aa95171e7ba48c6931bc9b2a3520 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-f, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-23T10:23:44.342 INFO:journalctl@ceph.mds.f.smithi196.stdout:Apr 23 10:23:43 smithi196 podman[63716]: 2024-04-23 10:23:43.862827729 +0000 UTC m=+0.390059978 container start 5fa315c4ec002895b34791b34d289501b9c4aa95171e7ba48c6931bc9b2a3520 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-f, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux ) 2024-04-23T10:23:44.342 INFO:journalctl@ceph.mds.f.smithi196.stdout:Apr 23 10:23:43 smithi196 ceph-mds[63735]: -- 172.21.15.196:0/905513193 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561a2e141a40 con 0x561a2e115000 2024-04-23T10:23:44.342 INFO:journalctl@ceph.mds.f.smithi196.stdout:Apr 23 10:23:43 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-f[63731]: starting mds.f at 2024-04-23T10:23:44.342 INFO:journalctl@ceph.mds.f.smithi196.stdout:Apr 23 10:23:43 smithi196 bash[63716]: 5fa315c4ec002895b34791b34d289501b9c4aa95171e7ba48c6931bc9b2a3520 2024-04-23T10:23:44.342 INFO:journalctl@ceph.mds.f.smithi196.stdout:Apr 23 10:23:43 smithi196 systemd[1]: Started Ceph mds.f for 9b22af88-0159-11ef-bc93-c7b262605968. 2024-04-23T10:23:44.418 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:44.419+0000 7fce9aa8a640 1 -- 172.21.15.97:0/1028209859 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fce941a0b80 con 0x7fce941029a0 2024-04-23T10:23:44.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:44.420+0000 7fce91ffb640 1 -- 172.21.15.97:0/1028209859 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4699 (secure 0 0 0) 0x7fce8c09cc80 con 0x7fce941029a0 2024-04-23T10:23:44.419 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:23:44.419 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3946878762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3946878762},{"type":"v1","addr":"172.21.15.97:6835","nonce":3946878762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/209800861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":209800861},{"type":"v1","addr":"172.21.15.97:6837","nonce":209800861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/4292847121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4292847121},{"type":"v1","addr":"172.21.15.183:6833","nonce":4292847121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3885816565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3885816565},{"type":"v1","addr":"172.21.15.196:6833","nonce":3885816565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1897093105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1897093105},{"type":"v1","addr":"172.21.15.183:6835","nonce":1897093105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-23T10:23:44.420 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 6 2024-04-23T10:23:44.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:44.422+0000 7fce9aa8a640 1 -- 172.21.15.97:0/1028209859 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fce840a3bb0 msgr2=0x7fce840a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:44.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:44.422+0000 7fce9aa8a640 1 --2- 172.21.15.97:0/1028209859 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fce840a3bb0 0x7fce840a6070 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fce88000c00 tx=0x7fce880314a0 comp rx=0 tx=0).stop 2024-04-23T10:23:44.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:44.422+0000 7fce9aa8a640 1 -- 172.21.15.97:0/1028209859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce941029a0 msgr2=0x7fce9419f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:44.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:44.422+0000 7fce9aa8a640 1 --2- 172.21.15.97:0/1028209859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce941029a0 0x7fce9419f7f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fce8c00eb10 tx=0x7fce8c00c2d0 comp rx=0 tx=0).stop 2024-04-23T10:23:44.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:44.422+0000 7fce9aa8a640 1 -- 172.21.15.97:0/1028209859 shutdown_connections 2024-04-23T10:23:44.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:44.422+0000 7fce9aa8a640 1 --2- 172.21.15.97:0/1028209859 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fce840a3bb0 0x7fce840a6070 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:44.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:44.422+0000 7fce9aa8a640 1 --2- 172.21.15.97:0/1028209859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce941029a0 0x7fce9419f7f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:44.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:44.422+0000 7fce9aa8a640 1 --2- 172.21.15.97:0/1028209859 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce94101fe0 0x7fce94198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:44.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:44.422+0000 7fce9aa8a640 1 --2- 172.21.15.97:0/1028209859 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce94100e90 0x7fce94198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:44.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:44.422+0000 7fce9aa8a640 1 -- 172.21.15.97:0/1028209859 >> 172.21.15.97:0/1028209859 conn(0x7fce940fc400 msgr2=0x7fce94069150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:23:44.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:44.422+0000 7fce9aa8a640 1 -- 172.21.15.97:0/1028209859 shutdown_connections 2024-04-23T10:23:44.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:44.422+0000 7fce9aa8a640 1 -- 172.21.15.97:0/1028209859 wait complete. 2024-04-23T10:23:44.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:44 smithi097 ceph-mon[29197]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:23:44.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:44 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:44.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:44 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:44.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:44 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:44.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:44 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T10:23:44.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:44 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T10:23:44.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:44 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:23:44.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:44 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/1028209859' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:23:44.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:44 smithi183 ceph-mon[32317]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:23:44.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:44 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:44.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:44 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:44.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:44 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:44.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:44 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T10:23:44.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:44 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T10:23:44.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:44 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:23:44.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:44 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/1028209859' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:23:44.933 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:44 smithi196 ceph-mon[32919]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:23:44.933 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:44 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:44.933 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:44 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:44.933 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:44 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:44.933 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:44 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T10:23:44.933 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:44 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T10:23:44.934 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:44 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:23:44.934 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:44 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/1028209859' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:23:45.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:45 smithi183 ceph-mon[32317]: Deploying daemon mds.c on smithi196 2024-04-23T10:23:45.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:45 smithi183 ceph-mon[32317]: mds.? [v2:172.21.15.196:6834/1171477563,v1:172.21.15.196:6835/1171477563] up:boot 2024-04-23T10:23:45.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:45 smithi183 ceph-mon[32317]: fsmap 6 up:standby 2024-04-23T10:23:45.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:45 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-23T10:23:45.983 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:45 smithi196 ceph-mon[32919]: Deploying daemon mds.c on smithi196 2024-04-23T10:23:45.983 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:45 smithi196 ceph-mon[32919]: mds.? [v2:172.21.15.196:6834/1171477563,v1:172.21.15.196:6835/1171477563] up:boot 2024-04-23T10:23:45.983 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:45 smithi196 ceph-mon[32919]: fsmap 6 up:standby 2024-04-23T10:23:45.983 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:45 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-23T10:23:46.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:45 smithi097 ceph-mon[29197]: Deploying daemon mds.c on smithi196 2024-04-23T10:23:46.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:45 smithi097 ceph-mon[29197]: mds.? [v2:172.21.15.196:6834/1171477563,v1:172.21.15.196:6835/1171477563] up:boot 2024-04-23T10:23:46.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:45 smithi097 ceph-mon[29197]: fsmap 6 up:standby 2024-04-23T10:23:46.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:45 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-23T10:23:46.342 INFO:journalctl@ceph.mds.c.smithi196.stdout:Apr 23 10:23:46 smithi196 systemd[1]: Starting Ceph mds.c for 9b22af88-0159-11ef-bc93-c7b262605968... 2024-04-23T10:23:46.434 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T10:23:46.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:46.545+0000 7f5fc18dc640 1 -- 172.21.15.97:0/1815130593 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fbc1008a0 msgr2=0x7f5fbc100cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:46.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:46.545+0000 7f5fc18dc640 1 --2- 172.21.15.97:0/1815130593 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fbc1008a0 0x7f5fbc100cc0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5fac009900 tx=0x7f5fac02a480 comp rx=0 tx=0).stop 2024-04-23T10:23:46.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:46.545+0000 7f5fc18dc640 1 -- 172.21.15.97:0/1815130593 shutdown_connections 2024-04-23T10:23:46.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:46.545+0000 7f5fc18dc640 1 --2- 172.21.15.97:0/1815130593 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fbc101ba0 0x7f5fbc102020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:46.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:46.545+0000 7f5fc18dc640 1 --2- 172.21.15.97:0/1815130593 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5fbc101200 0x7f5fbc101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:46.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:46.545+0000 7f5fc18dc640 1 --2- 172.21.15.97:0/1815130593 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fbc1008a0 0x7f5fbc100cc0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:46.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:46.545+0000 7f5fc18dc640 1 -- 172.21.15.97:0/1815130593 >> 172.21.15.97:0/1815130593 conn(0x7f5fbc0fc440 msgr2=0x7f5fbc0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:23:46.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:46.545+0000 7f5fc18dc640 1 -- 172.21.15.97:0/1815130593 shutdown_connections 2024-04-23T10:23:46.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:46.545+0000 7f5fc18dc640 1 -- 172.21.15.97:0/1815130593 wait complete. 2024-04-23T10:23:46.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:46.546+0000 7f5fc18dc640 1 Processor -- start 2024-04-23T10:23:46.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:46.546+0000 7f5fc18dc640 1 -- start start 2024-04-23T10:23:46.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:46.546+0000 7f5fc18dc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fbc1008a0 0x7f5fbc19a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:46.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:46.546+0000 7f5fc18dc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fbc101200 0x7f5fbc19a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:46.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:46.546+0000 7f5fc18dc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5fbc101ba0 0x7f5fbc1a1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:46.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:46.546+0000 7f5fc18dc640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5fbc10fcf0 con 0x7f5fbc101200 2024-04-23T10:23:46.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:46.546+0000 7f5fc18dc640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5fbc10fb70 con 0x7f5fbc1008a0 2024-04-23T10:23:46.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:46.546+0000 7f5fc18dc640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5fbc10fe70 con 0x7f5fbc101ba0 2024-04-23T10:23:46.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:46.546+0000 7f5fba7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fbc101200 0x7f5fbc19a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:46.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:46.546+0000 7f5fba7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fbc101200 0x7f5fbc19a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40948/0 (socket says 172.21.15.97:40948) 2024-04-23T10:23:46.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:46.546+0000 7f5fba7fc640 1 -- 172.21.15.97:0/2134112350 learned_addr learned my addr 172.21.15.97:0/2134112350 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:23:46.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:46.546+0000 7f5fbb7fe640 1 --2- 172.21.15.97:0/2134112350 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5fbc101ba0 0x7f5fbc1a1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:46.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:46.546+0000 7f5fbaffd640 1 --2- 172.21.15.97:0/2134112350 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fbc1008a0 0x7f5fbc19a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:46.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:46.547+0000 7f5fba7fc640 1 -- 172.21.15.97:0/2134112350 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5fbc101ba0 msgr2=0x7f5fbc1a1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:46.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:46.547+0000 7f5fba7fc640 1 --2- 172.21.15.97:0/2134112350 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5fbc101ba0 0x7f5fbc1a1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:46.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:46.547+0000 7f5fba7fc640 1 -- 172.21.15.97:0/2134112350 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fbc1008a0 msgr2=0x7f5fbc19a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:46.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:46.547+0000 7f5fba7fc640 1 --2- 172.21.15.97:0/2134112350 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fbc1008a0 0x7f5fbc19a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:46.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:46.547+0000 7f5fba7fc640 1 -- 172.21.15.97:0/2134112350 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fbc1a2000 con 0x7f5fbc101200 2024-04-23T10:23:46.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:46.547+0000 7f5fba7fc640 1 --2- 172.21.15.97:0/2134112350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fbc101200 0x7f5fbc19a920 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5fb000e9f0 tx=0x7f5fb000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:23:46.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:46.547+0000 7f5fc08da640 1 -- 172.21.15.97:0/2134112350 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb000cd80 con 0x7f5fbc101200 2024-04-23T10:23:46.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:46.547+0000 7f5fc08da640 1 -- 172.21.15.97:0/2134112350 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fb00054d0 con 0x7f5fbc101200 2024-04-23T10:23:46.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:46.547+0000 7f5fc18dc640 1 -- 172.21.15.97:0/2134112350 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fbc1a22f0 con 0x7f5fbc101200 2024-04-23T10:23:46.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:46.547+0000 7f5fc08da640 1 -- 172.21.15.97:0/2134112350 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb0010690 con 0x7f5fbc101200 2024-04-23T10:23:46.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:46.547+0000 7f5fc18dc640 1 -- 172.21.15.97:0/2134112350 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fbc1a2840 con 0x7f5fbc101200 2024-04-23T10:23:46.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:46.548+0000 7f5fc18dc640 1 -- 172.21.15.97:0/2134112350 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fbc10b5f0 con 0x7f5fbc101200 2024-04-23T10:23:46.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:46.550+0000 7f5fc08da640 1 -- 172.21.15.97:0/2134112350 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5fb0004e30 con 0x7f5fbc101200 2024-04-23T10:23:46.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:46.550+0000 7f5fc08da640 1 --2- 172.21.15.97:0/2134112350 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f5fa40a3c30 0x7f5fa40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:46.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:46.551+0000 7f5fbaffd640 1 --2- 172.21.15.97:0/2134112350 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f5fa40a3c30 0x7f5fa40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:46.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:46.551+0000 7f5fc08da640 1 -- 172.21.15.97:0/2134112350 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f5fb00d05a0 con 0x7f5fbc101200 2024-04-23T10:23:46.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:46.551+0000 7f5fbaffd640 1 --2- 172.21.15.97:0/2134112350 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f5fa40a3c30 0x7f5fa40a60f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5fac000c00 tx=0x7f5fac0314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:23:46.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:46.554+0000 7f5fc08da640 1 -- 172.21.15.97:0/2134112350 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5fb0053ff0 con 0x7f5fbc101200 2024-04-23T10:23:46.667 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:46 smithi196 ceph-mon[32919]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:23:46.668 INFO:journalctl@ceph.mds.c.smithi196.stdout:Apr 23 10:23:46 smithi196 podman[64031]: 2024-04-23 10:23:46.291591233 +0000 UTC m=+0.017836181 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T10:23:46.668 INFO:journalctl@ceph.mds.c.smithi196.stdout:Apr 23 10:23:46 smithi196 podman[64031]: 2024-04-23 10:23:46.417079951 +0000 UTC m=+0.143324868 container create 5ceaaab165ca8b257190b3f4680b8d86c9109dc7426895dfe4e1ec7a278fbcd2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-c, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS) 2024-04-23T10:23:46.668 INFO:journalctl@ceph.mds.c.smithi196.stdout:Apr 23 10:23:46 smithi196 podman[64031]: 2024-04-23 10:23:46.66998691 +0000 UTC m=+0.396231828 container init 5ceaaab165ca8b257190b3f4680b8d86c9109dc7426895dfe4e1ec7a278fbcd2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS) 2024-04-23T10:23:46.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:46.771+0000 7f5fc18dc640 1 -- 172.21.15.97:0/2134112350 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5fbc02d810 con 0x7f5fbc101200 2024-04-23T10:23:46.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:46.772+0000 7f5fc08da640 1 -- 172.21.15.97:0/2134112350 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5527 (secure 0 0 0) 0x7f5fb009d420 con 0x7f5fbc101200 2024-04-23T10:23:46.802 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:23:46.802 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3946878762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3946878762},{"type":"v1","addr":"172.21.15.97:6835","nonce":3946878762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/209800861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":209800861},{"type":"v1","addr":"172.21.15.97:6837","nonce":209800861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/4292847121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4292847121},{"type":"v1","addr":"172.21.15.183:6833","nonce":4292847121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3885816565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3885816565},{"type":"v1","addr":"172.21.15.196:6833","nonce":3885816565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1897093105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1897093105},{"type":"v1","addr":"172.21.15.183:6835","nonce":1897093105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1171477563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1171477563},{"type":"v1","addr":"172.21.15.196:6835","nonce":1171477563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-04-23T10:23:46.803 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 7 2024-04-23T10:23:46.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:46.774+0000 7f5fc18dc640 1 -- 172.21.15.97:0/2134112350 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f5fa40a3c30 msgr2=0x7f5fa40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:46.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:46.774+0000 7f5fc18dc640 1 --2- 172.21.15.97:0/2134112350 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f5fa40a3c30 0x7f5fa40a60f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5fac000c00 tx=0x7f5fac0314a0 comp rx=0 tx=0).stop 2024-04-23T10:23:46.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:46.774+0000 7f5fc18dc640 1 -- 172.21.15.97:0/2134112350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fbc101200 msgr2=0x7f5fbc19a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:46.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:46.774+0000 7f5fc18dc640 1 --2- 172.21.15.97:0/2134112350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fbc101200 0x7f5fbc19a920 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5fb000e9f0 tx=0x7f5fb000edf0 comp rx=0 tx=0).stop 2024-04-23T10:23:46.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:46.774+0000 7f5fc18dc640 1 -- 172.21.15.97:0/2134112350 shutdown_connections 2024-04-23T10:23:46.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:46.774+0000 7f5fc18dc640 1 --2- 172.21.15.97:0/2134112350 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f5fa40a3c30 0x7f5fa40a60f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:46.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:46.774+0000 7f5fc18dc640 1 --2- 172.21.15.97:0/2134112350 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5fbc101ba0 0x7f5fbc1a1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:46.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:46.774+0000 7f5fc18dc640 1 --2- 172.21.15.97:0/2134112350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5fbc101200 0x7f5fbc19a920 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:46.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:46.774+0000 7f5fc18dc640 1 --2- 172.21.15.97:0/2134112350 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5fbc1008a0 0x7f5fbc19a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:46.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:46.774+0000 7f5fc18dc640 1 -- 172.21.15.97:0/2134112350 >> 172.21.15.97:0/2134112350 conn(0x7f5fbc0fc440 msgr2=0x7f5fbc0ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:23:46.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:46.774+0000 7f5fc18dc640 1 -- 172.21.15.97:0/2134112350 shutdown_connections 2024-04-23T10:23:46.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:46.774+0000 7f5fc18dc640 1 -- 172.21.15.97:0/2134112350 wait complete. 2024-04-23T10:23:46.804 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:46 smithi097 ceph-mon[29197]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:23:46.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:46 smithi183 ceph-mon[32317]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:23:47.092 INFO:journalctl@ceph.mds.c.smithi196.stdout:Apr 23 10:23:46 smithi196 podman[64031]: 2024-04-23 10:23:46.672776665 +0000 UTC m=+0.399021585 container start 5ceaaab165ca8b257190b3f4680b8d86c9109dc7426895dfe4e1ec7a278fbcd2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-c, GIT_CLEAN=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5) 2024-04-23T10:23:47.092 INFO:journalctl@ceph.mds.c.smithi196.stdout:Apr 23 10:23:46 smithi196 ceph-mds[64050]: -- 172.21.15.196:0/2907006661 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bd649bba40 con 0x55bd6498e800 2024-04-23T10:23:47.092 INFO:journalctl@ceph.mds.c.smithi196.stdout:Apr 23 10:23:46 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-c[64046]: starting mds.c at 2024-04-23T10:23:47.092 INFO:journalctl@ceph.mds.c.smithi196.stdout:Apr 23 10:23:46 smithi196 bash[64031]: 5ceaaab165ca8b257190b3f4680b8d86c9109dc7426895dfe4e1ec7a278fbcd2 2024-04-23T10:23:47.092 INFO:journalctl@ceph.mds.c.smithi196.stdout:Apr 23 10:23:46 smithi196 systemd[1]: Started Ceph mds.c for 9b22af88-0159-11ef-bc93-c7b262605968. 2024-04-23T10:23:47.758 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:47 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/2134112350' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:23:47.758 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:47 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:47.758 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:47 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:47.758 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:47 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:47.758 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:47 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T10:23:47.758 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:47 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T10:23:47.758 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:47 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:23:48.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:47 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/2134112350' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:23:48.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:47 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:48.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:47 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:48.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:47 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:48.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:47 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T10:23:48.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:47 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T10:23:48.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:47 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:23:48.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:47 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/2134112350' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:23:48.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:47 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:48.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:47 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:48.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:47 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:48.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:47 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T10:23:48.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:47 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T10:23:48.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:47 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:23:48.806 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T10:23:48.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:48 smithi183 ceph-mon[32317]: Deploying daemon mds.h on smithi183 2024-04-23T10:23:48.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:48 smithi183 ceph-mon[32317]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:23:48.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:48 smithi183 ceph-mon[32317]: mds.? [v2:172.21.15.196:6836/3094946502,v1:172.21.15.196:6837/3094946502] up:boot 2024-04-23T10:23:48.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:48 smithi183 ceph-mon[32317]: fsmap 7 up:standby 2024-04-23T10:23:48.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:48 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-23T10:23:48.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:48.918+0000 7f1ab699b640 1 -- 172.21.15.97:0/3582021249 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ab0105400 msgr2=0x7f1ab01077f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:48.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:48.918+0000 7f1ab699b640 1 --2- 172.21.15.97:0/3582021249 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ab0105400 0x7f1ab01077f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1a98009900 tx=0x7f1a9802a410 comp rx=0 tx=0).stop 2024-04-23T10:23:48.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:48.918+0000 7f1ab699b640 1 -- 172.21.15.97:0/3582021249 shutdown_connections 2024-04-23T10:23:48.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:48.918+0000 7f1ab699b640 1 --2- 172.21.15.97:0/3582021249 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ab0107d30 0x7f1ab010a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:48.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:48.918+0000 7f1ab699b640 1 --2- 172.21.15.97:0/3582021249 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ab0105400 0x7f1ab01077f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:48.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:48.918+0000 7f1ab699b640 1 --2- 172.21.15.97:0/3582021249 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ab0102ad0 0x7f1ab0104ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:48.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:48.918+0000 7f1ab699b640 1 -- 172.21.15.97:0/3582021249 >> 172.21.15.97:0/3582021249 conn(0x7f1ab00fc440 msgr2=0x7f1ab00fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:23:48.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:48.918+0000 7f1ab699b640 1 -- 172.21.15.97:0/3582021249 shutdown_connections 2024-04-23T10:23:48.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:48.918+0000 7f1ab699b640 1 -- 172.21.15.97:0/3582021249 wait complete. 2024-04-23T10:23:48.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:48.919+0000 7f1ab699b640 1 Processor -- start 2024-04-23T10:23:48.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:48.919+0000 7f1ab699b640 1 -- start start 2024-04-23T10:23:48.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:48.919+0000 7f1ab699b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ab0102ad0 0x7f1ab01981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:48.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:48.919+0000 7f1ab699b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ab0105400 0x7f1ab01986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:48.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:48.919+0000 7f1ab699b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ab0107d30 0x7f1ab019f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:48.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:48.919+0000 7f1ab699b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab010dae0 con 0x7f1ab0102ad0 2024-04-23T10:23:48.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:48.919+0000 7f1ab699b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab010d960 con 0x7f1ab0105400 2024-04-23T10:23:48.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:48.919+0000 7f1ab699b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab010dc60 con 0x7f1ab0107d30 2024-04-23T10:23:48.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:48.919+0000 7f1aaffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ab0102ad0 0x7f1ab01981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:48.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:48.919+0000 7f1aaffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ab0102ad0 0x7f1ab01981a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40962/0 (socket says 172.21.15.97:40962) 2024-04-23T10:23:48.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:48.919+0000 7f1aaffff640 1 -- 172.21.15.97:0/1869202387 learned_addr learned my addr 172.21.15.97:0/1869202387 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:23:48.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:48.919+0000 7f1ab4f38640 1 --2- 172.21.15.97:0/1869202387 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ab0107d30 0x7f1ab019f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:48.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:48.919+0000 7f1aaf7fe640 1 --2- 172.21.15.97:0/1869202387 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ab0105400 0x7f1ab01986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:48.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:48.919+0000 7f1aaffff640 1 -- 172.21.15.97:0/1869202387 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ab0107d30 msgr2=0x7f1ab019f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:48.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:48.919+0000 7f1aaffff640 1 --2- 172.21.15.97:0/1869202387 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ab0107d30 0x7f1ab019f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:48.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:48.919+0000 7f1aaffff640 1 -- 172.21.15.97:0/1869202387 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ab0105400 msgr2=0x7f1ab01986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:48.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:48.919+0000 7f1aaffff640 1 --2- 172.21.15.97:0/1869202387 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ab0105400 0x7f1ab01986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:48.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:48.919+0000 7f1aaffff640 1 -- 172.21.15.97:0/1869202387 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ab019fdc0 con 0x7f1ab0102ad0 2024-04-23T10:23:48.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:48.919+0000 7f1aaf7fe640 1 --2- 172.21.15.97:0/1869202387 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ab0105400 0x7f1ab01986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:23:48.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:48.919+0000 7f1ab4f38640 1 --2- 172.21.15.97:0/1869202387 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ab0107d30 0x7f1ab019f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:23:48.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:48.920+0000 7f1aaffff640 1 --2- 172.21.15.97:0/1869202387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ab0102ad0 0x7f1ab01981a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1aa000c920 tx=0x7f1aa000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:23:48.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:48.920+0000 7f1aad7fa640 1 -- 172.21.15.97:0/1869202387 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa0007c70 con 0x7f1ab0102ad0 2024-04-23T10:23:48.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:48.920+0000 7f1aad7fa640 1 -- 172.21.15.97:0/1869202387 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1aa0007e10 con 0x7f1ab0102ad0 2024-04-23T10:23:48.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:48.920+0000 7f1ab699b640 1 -- 172.21.15.97:0/1869202387 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ab01a00b0 con 0x7f1ab0102ad0 2024-04-23T10:23:48.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:48.920+0000 7f1aad7fa640 1 -- 172.21.15.97:0/1869202387 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa0018820 con 0x7f1ab0102ad0 2024-04-23T10:23:48.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:48.920+0000 7f1ab699b640 1 -- 172.21.15.97:0/1869202387 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ab01a05d0 con 0x7f1ab0102ad0 2024-04-23T10:23:48.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:48.921+0000 7f1ab699b640 1 -- 172.21.15.97:0/1869202387 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ab00fdf30 con 0x7f1ab0102ad0 2024-04-23T10:23:48.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:48.922+0000 7f1aad7fa640 1 -- 172.21.15.97:0/1869202387 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1aa000f450 con 0x7f1ab0102ad0 2024-04-23T10:23:48.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:48.922+0000 7f1aad7fa640 1 --2- 172.21.15.97:0/1869202387 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f1a9c0a3bb0 0x7f1a9c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:48.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:48.923+0000 7f1aaf7fe640 1 --2- 172.21.15.97:0/1869202387 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f1a9c0a3bb0 0x7f1a9c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:48.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:48.923+0000 7f1aad7fa640 1 -- 172.21.15.97:0/1869202387 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f1aa00d0120 con 0x7f1ab0102ad0 2024-04-23T10:23:48.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:48.923+0000 7f1aaf7fe640 1 --2- 172.21.15.97:0/1869202387 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f1a9c0a3bb0 0x7f1a9c0a6070 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1a98000c00 tx=0x7f1a980314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:23:48.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:48.927+0000 7f1aad7fa640 1 -- 172.21.15.97:0/1869202387 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1aa0014030 con 0x7f1ab0102ad0 2024-04-23T10:23:49.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:48 smithi196 ceph-mon[32919]: Deploying daemon mds.h on smithi183 2024-04-23T10:23:49.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:48 smithi196 ceph-mon[32919]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:23:49.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:48 smithi196 ceph-mon[32919]: mds.? [v2:172.21.15.196:6836/3094946502,v1:172.21.15.196:6837/3094946502] up:boot 2024-04-23T10:23:49.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:48 smithi196 ceph-mon[32919]: fsmap 7 up:standby 2024-04-23T10:23:49.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:48 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-23T10:23:49.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:48 smithi097 ceph-mon[29197]: Deploying daemon mds.h on smithi183 2024-04-23T10:23:49.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:48 smithi097 ceph-mon[29197]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:23:49.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:48 smithi097 ceph-mon[29197]: mds.? [v2:172.21.15.196:6836/3094946502,v1:172.21.15.196:6837/3094946502] up:boot 2024-04-23T10:23:49.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:48 smithi097 ceph-mon[29197]: fsmap 7 up:standby 2024-04-23T10:23:49.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:48 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-23T10:23:49.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:49.145+0000 7f1ab699b640 1 -- 172.21.15.97:0/1869202387 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1ab002d5c0 con 0x7f1ab0102ad0 2024-04-23T10:23:49.144 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:49.145+0000 7f1aad7fa640 1 -- 172.21.15.97:0/1869202387 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6355 (secure 0 0 0) 0x7f1aa009cfa0 con 0x7f1ab0102ad0 2024-04-23T10:23:49.145 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:23:49.145 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3946878762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3946878762},{"type":"v1","addr":"172.21.15.97:6835","nonce":3946878762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/209800861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":209800861},{"type":"v1","addr":"172.21.15.97:6837","nonce":209800861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/4292847121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4292847121},{"type":"v1","addr":"172.21.15.183:6833","nonce":4292847121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3885816565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3885816565},{"type":"v1","addr":"172.21.15.196:6833","nonce":3885816565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1897093105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1897093105},{"type":"v1","addr":"172.21.15.183:6835","nonce":1897093105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1171477563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1171477563},{"type":"v1","addr":"172.21.15.196:6835","nonce":1171477563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6837/3094946502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3094946502},{"type":"v1","addr":"172.21.15.196:6837","nonce":3094946502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-23T10:23:49.145 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 8 2024-04-23T10:23:49.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:49.147+0000 7f1ab699b640 1 -- 172.21.15.97:0/1869202387 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f1a9c0a3bb0 msgr2=0x7f1a9c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:49.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:49.147+0000 7f1ab699b640 1 --2- 172.21.15.97:0/1869202387 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f1a9c0a3bb0 0x7f1a9c0a6070 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1a98000c00 tx=0x7f1a980314a0 comp rx=0 tx=0).stop 2024-04-23T10:23:49.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:49.147+0000 7f1ab699b640 1 -- 172.21.15.97:0/1869202387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ab0102ad0 msgr2=0x7f1ab01981a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:49.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:49.147+0000 7f1ab699b640 1 --2- 172.21.15.97:0/1869202387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ab0102ad0 0x7f1ab01981a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1aa000c920 tx=0x7f1aa000cde0 comp rx=0 tx=0).stop 2024-04-23T10:23:49.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:49.148+0000 7f1ab699b640 1 -- 172.21.15.97:0/1869202387 shutdown_connections 2024-04-23T10:23:49.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:49.148+0000 7f1ab699b640 1 --2- 172.21.15.97:0/1869202387 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f1a9c0a3bb0 0x7f1a9c0a6070 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:49.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:49.148+0000 7f1ab699b640 1 --2- 172.21.15.97:0/1869202387 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ab0107d30 0x7f1ab019f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:49.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:49.148+0000 7f1ab699b640 1 --2- 172.21.15.97:0/1869202387 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ab0105400 0x7f1ab01986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:49.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:49.148+0000 7f1ab699b640 1 --2- 172.21.15.97:0/1869202387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ab0102ad0 0x7f1ab01981a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:49.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:49.148+0000 7f1ab699b640 1 -- 172.21.15.97:0/1869202387 >> 172.21.15.97:0/1869202387 conn(0x7f1ab00fc440 msgr2=0x7f1ab0103410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:23:49.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:49.148+0000 7f1ab699b640 1 -- 172.21.15.97:0/1869202387 shutdown_connections 2024-04-23T10:23:49.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:49.148+0000 7f1ab699b640 1 -- 172.21.15.97:0/1869202387 wait complete. 2024-04-23T10:23:49.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:49 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/1869202387' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:23:49.900 INFO:journalctl@ceph.mds.h.smithi183.stdout:Apr 23 10:23:49 smithi183 systemd[1]: Started Ceph mds.h for 9b22af88-0159-11ef-bc93-c7b262605968. 2024-04-23T10:23:50.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:49 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/1869202387' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:23:50.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:49 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/1869202387' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:23:50.732 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:50 smithi196 ceph-mon[32919]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:23:50.732 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:50 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:50.732 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:50 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:50.732 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:50 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:50.732 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:50 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T10:23:50.732 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:50 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T10:23:50.732 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:50 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:23:50.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:50 smithi183 ceph-mon[32317]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:23:50.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:50 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:50.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:50 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:50.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:50 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:50.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:50 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T10:23:50.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:50 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T10:23:50.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:50 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:23:51.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:50 smithi097 ceph-mon[29197]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:23:51.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:50 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:51.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:50 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:51.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:50 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:51.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:50 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T10:23:51.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:50 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T10:23:51.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:50 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:23:51.159 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T10:23:51.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:51.269+0000 7feca5ec2640 1 -- 172.21.15.97:0/4141127460 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feca0107c90 msgr2=0x7feca010a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:51.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:51.269+0000 7feca5ec2640 1 --2- 172.21.15.97:0/4141127460 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feca0107c90 0x7feca010a0e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fec98009ca0 tx=0x7fec9802a8c0 comp rx=0 tx=0).stop 2024-04-23T10:23:51.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:51.269+0000 7feca5ec2640 1 -- 172.21.15.97:0/4141127460 shutdown_connections 2024-04-23T10:23:51.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:51.269+0000 7feca5ec2640 1 --2- 172.21.15.97:0/4141127460 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feca0107c90 0x7feca010a0e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:51.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:51.269+0000 7feca5ec2640 1 --2- 172.21.15.97:0/4141127460 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feca0105360 0x7feca0107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:51.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:51.269+0000 7feca5ec2640 1 --2- 172.21.15.97:0/4141127460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feca0102a30 0x7feca0104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:51.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:51.269+0000 7feca5ec2640 1 -- 172.21.15.97:0/4141127460 >> 172.21.15.97:0/4141127460 conn(0x7feca00fc440 msgr2=0x7feca00fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:23:51.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:51.269+0000 7feca5ec2640 1 -- 172.21.15.97:0/4141127460 shutdown_connections 2024-04-23T10:23:51.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:51.269+0000 7feca5ec2640 1 -- 172.21.15.97:0/4141127460 wait complete. 2024-04-23T10:23:51.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:51.270+0000 7feca5ec2640 1 Processor -- start 2024-04-23T10:23:51.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:51.270+0000 7feca5ec2640 1 -- start start 2024-04-23T10:23:51.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:51.270+0000 7feca5ec2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feca0102a30 0x7feca0198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:51.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:51.270+0000 7feca5ec2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feca0105360 0x7feca01986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:51.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:51.270+0000 7feca5ec2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feca0107c90 0x7feca019f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:51.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:51.270+0000 7feca5ec2640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7feca010daf0 con 0x7feca0107c90 2024-04-23T10:23:51.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:51.270+0000 7feca5ec2640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7feca010d970 con 0x7feca0105360 2024-04-23T10:23:51.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:51.270+0000 7feca5ec2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7feca010dc70 con 0x7feca0102a30 2024-04-23T10:23:51.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:51.270+0000 7fec9ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feca0107c90 0x7feca019f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:51.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:51.270+0000 7fec9ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feca0107c90 0x7feca019f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40992/0 (socket says 172.21.15.97:40992) 2024-04-23T10:23:51.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:51.270+0000 7fec9ffff640 1 -- 172.21.15.97:0/1281480172 learned_addr learned my addr 172.21.15.97:0/1281480172 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:23:51.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:51.270+0000 7fec9f7fe640 1 --2- 172.21.15.97:0/1281480172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feca0102a30 0x7feca0198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:51.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:51.270+0000 7fec9effd640 1 --2- 172.21.15.97:0/1281480172 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feca0105360 0x7feca01986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:51.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:51.270+0000 7fec9ffff640 1 -- 172.21.15.97:0/1281480172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feca0102a30 msgr2=0x7feca0198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:51.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:51.270+0000 7fec9ffff640 1 --2- 172.21.15.97:0/1281480172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feca0102a30 0x7feca0198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:51.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:51.270+0000 7fec9ffff640 1 -- 172.21.15.97:0/1281480172 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feca0105360 msgr2=0x7feca01986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:51.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:51.270+0000 7fec9ffff640 1 --2- 172.21.15.97:0/1281480172 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feca0105360 0x7feca01986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:51.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:51.270+0000 7fec9ffff640 1 -- 172.21.15.97:0/1281480172 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feca019fdb0 con 0x7feca0107c90 2024-04-23T10:23:51.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:51.271+0000 7fec9ffff640 1 --2- 172.21.15.97:0/1281480172 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feca0107c90 0x7feca019f740 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fec980097b0 tx=0x7fec98003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:23:51.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:51.271+0000 7fec9cff9640 1 -- 172.21.15.97:0/1281480172 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec98004240 con 0x7feca0107c90 2024-04-23T10:23:51.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:51.271+0000 7fec9cff9640 1 -- 172.21.15.97:0/1281480172 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec980317c0 con 0x7feca0107c90 2024-04-23T10:23:51.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:51.271+0000 7feca5ec2640 1 -- 172.21.15.97:0/1281480172 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feca01a0040 con 0x7feca0107c90 2024-04-23T10:23:51.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:51.271+0000 7fec9cff9640 1 -- 172.21.15.97:0/1281480172 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec9802fd80 con 0x7feca0107c90 2024-04-23T10:23:51.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:51.271+0000 7feca5ec2640 1 -- 172.21.15.97:0/1281480172 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feca01a05e0 con 0x7feca0107c90 2024-04-23T10:23:51.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:51.272+0000 7feca5ec2640 1 -- 172.21.15.97:0/1281480172 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feca00fdf30 con 0x7feca0107c90 2024-04-23T10:23:51.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:51.273+0000 7fec9cff9640 1 -- 172.21.15.97:0/1281480172 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fec980043e0 con 0x7feca0107c90 2024-04-23T10:23:51.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:51.273+0000 7fec9cff9640 1 --2- 172.21.15.97:0/1281480172 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fec880a3c30 0x7fec880a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:51.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:51.274+0000 7fec9f7fe640 1 --2- 172.21.15.97:0/1281480172 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fec880a3c30 0x7fec880a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:51.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:51.274+0000 7fec9cff9640 1 -- 172.21.15.97:0/1281480172 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fec980f1770 con 0x7feca0107c90 2024-04-23T10:23:51.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:51.274+0000 7fec9f7fe640 1 --2- 172.21.15.97:0/1281480172 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fec880a3c30 0x7fec880a60f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fec90005fd0 tx=0x7fec9000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:23:51.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:51.278+0000 7fec9cff9640 1 -- 172.21.15.97:0/1281480172 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fec980be710 con 0x7feca0107c90 2024-04-23T10:23:51.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:51.495+0000 7feca5ec2640 1 -- 172.21.15.97:0/1281480172 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feca0062b50 con 0x7feca0107c90 2024-04-23T10:23:51.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:51.495+0000 7fec9cff9640 1 -- 172.21.15.97:0/1281480172 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7183 (secure 0 0 0) 0x7fec980098b0 con 0x7feca0107c90 2024-04-23T10:23:51.494 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:23:51.494 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3946878762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3946878762},{"type":"v1","addr":"172.21.15.97:6835","nonce":3946878762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/209800861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":209800861},{"type":"v1","addr":"172.21.15.97:6837","nonce":209800861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/4292847121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4292847121},{"type":"v1","addr":"172.21.15.183:6833","nonce":4292847121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3885816565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3885816565},{"type":"v1","addr":"172.21.15.196:6833","nonce":3885816565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1897093105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1897093105},{"type":"v1","addr":"172.21.15.183:6835","nonce":1897093105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1171477563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1171477563},{"type":"v1","addr":"172.21.15.196:6835","nonce":1171477563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24461,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/1898341911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1898341911},{"type":"v1","addr":"172.21.15.183:6837","nonce":1898341911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6837/3094946502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3094946502},{"type":"v1","addr":"172.21.15.196:6837","nonce":3094946502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-23T10:23:51.495 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 9 2024-04-23T10:23:51.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:51.497+0000 7feca5ec2640 1 -- 172.21.15.97:0/1281480172 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fec880a3c30 msgr2=0x7fec880a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:51.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:51.497+0000 7feca5ec2640 1 --2- 172.21.15.97:0/1281480172 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fec880a3c30 0x7fec880a60f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fec90005fd0 tx=0x7fec9000b390 comp rx=0 tx=0).stop 2024-04-23T10:23:51.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:51.497+0000 7feca5ec2640 1 -- 172.21.15.97:0/1281480172 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feca0107c90 msgr2=0x7feca019f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:51.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:51.497+0000 7feca5ec2640 1 --2- 172.21.15.97:0/1281480172 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feca0107c90 0x7feca019f740 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fec980097b0 tx=0x7fec98003c20 comp rx=0 tx=0).stop 2024-04-23T10:23:51.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:51.497+0000 7feca5ec2640 1 -- 172.21.15.97:0/1281480172 shutdown_connections 2024-04-23T10:23:51.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:51.497+0000 7feca5ec2640 1 --2- 172.21.15.97:0/1281480172 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fec880a3c30 0x7fec880a60f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:51.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:51.497+0000 7feca5ec2640 1 --2- 172.21.15.97:0/1281480172 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feca0107c90 0x7feca019f740 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:51.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:51.497+0000 7feca5ec2640 1 --2- 172.21.15.97:0/1281480172 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feca0105360 0x7feca01986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:51.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:51.497+0000 7feca5ec2640 1 --2- 172.21.15.97:0/1281480172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feca0102a30 0x7feca0198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:51.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:51.497+0000 7feca5ec2640 1 -- 172.21.15.97:0/1281480172 >> 172.21.15.97:0/1281480172 conn(0x7feca00fc440 msgr2=0x7feca0103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:23:51.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:51.497+0000 7feca5ec2640 1 -- 172.21.15.97:0/1281480172 shutdown_connections 2024-04-23T10:23:51.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:51.497+0000 7feca5ec2640 1 -- 172.21.15.97:0/1281480172 wait complete. 2024-04-23T10:23:51.793 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:51 smithi196 ceph-mon[32919]: Deploying daemon mds.i on smithi196 2024-04-23T10:23:51.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:51 smithi196 ceph-mon[32919]: mds.? [v2:172.21.15.183:6836/1898341911,v1:172.21.15.183:6837/1898341911] up:boot 2024-04-23T10:23:51.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:51 smithi196 ceph-mon[32919]: fsmap 8 up:standby 2024-04-23T10:23:51.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:51 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-23T10:23:51.794 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:51 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/1281480172' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:23:51.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:51 smithi097 ceph-mon[29197]: Deploying daemon mds.i on smithi196 2024-04-23T10:23:51.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:51 smithi097 ceph-mon[29197]: mds.? [v2:172.21.15.183:6836/1898341911,v1:172.21.15.183:6837/1898341911] up:boot 2024-04-23T10:23:51.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:51 smithi097 ceph-mon[29197]: fsmap 8 up:standby 2024-04-23T10:23:51.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:51 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-23T10:23:51.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:51 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/1281480172' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:23:51.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:51 smithi183 ceph-mon[32317]: Deploying daemon mds.i on smithi196 2024-04-23T10:23:51.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:51 smithi183 ceph-mon[32317]: mds.? [v2:172.21.15.183:6836/1898341911,v1:172.21.15.183:6837/1898341911] up:boot 2024-04-23T10:23:51.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:51 smithi183 ceph-mon[32317]: fsmap 8 up:standby 2024-04-23T10:23:51.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:51 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-23T10:23:51.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:51 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/1281480172' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:23:52.092 INFO:journalctl@ceph.mds.i.smithi196.stdout:Apr 23 10:23:52 smithi196 systemd[1]: Starting Ceph mds.i for 9b22af88-0159-11ef-bc93-c7b262605968... 2024-04-23T10:23:52.477 INFO:journalctl@ceph.mds.i.smithi196.stdout:Apr 23 10:23:52 smithi196 podman[64335]: 2024-04-23 10:23:52.130882227 +0000 UTC m=+0.018934355 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T10:23:52.477 INFO:journalctl@ceph.mds.i.smithi196.stdout:Apr 23 10:23:52 smithi196 podman[64335]: 2024-04-23 10:23:52.248487175 +0000 UTC m=+0.136539327 container create 13dc06b92bdf2afba3fbfd562d3120c9f9cf5973118fd94879aa914d0fcc4d80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T10:23:52.477 INFO:journalctl@ceph.mds.i.smithi196.stdout:Apr 23 10:23:52 smithi196 podman[64335]: 2024-04-23 10:23:52.476243821 +0000 UTC m=+0.364295931 container init 13dc06b92bdf2afba3fbfd562d3120c9f9cf5973118fd94879aa914d0fcc4d80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD) 2024-04-23T10:23:52.477 INFO:journalctl@ceph.mds.i.smithi196.stdout:Apr 23 10:23:52 smithi196 podman[64335]: 2024-04-23 10:23:52.478807325 +0000 UTC m=+0.366859437 container start 13dc06b92bdf2afba3fbfd562d3120c9f9cf5973118fd94879aa914d0fcc4d80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_CLEAN=True, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-23T10:23:52.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:52 smithi196 ceph-mon[32919]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:23:52.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:52 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:52.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:52 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:52.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:52 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:52.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:52 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T10:23:52.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:52 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T10:23:52.842 INFO:journalctl@ceph.mds.i.smithi196.stdout:Apr 23 10:23:52 smithi196 ceph-mds[64355]: -- 172.21.15.196:0/2032200526 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5557a211fa40 con 0x5557a20f2800 2024-04-23T10:23:52.842 INFO:journalctl@ceph.mds.i.smithi196.stdout:Apr 23 10:23:52 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-i[64351]: starting mds.i at 2024-04-23T10:23:52.842 INFO:journalctl@ceph.mds.i.smithi196.stdout:Apr 23 10:23:52 smithi196 bash[64335]: 13dc06b92bdf2afba3fbfd562d3120c9f9cf5973118fd94879aa914d0fcc4d80 2024-04-23T10:23:52.842 INFO:journalctl@ceph.mds.i.smithi196.stdout:Apr 23 10:23:52 smithi196 systemd[1]: Started Ceph mds.i for 9b22af88-0159-11ef-bc93-c7b262605968. 2024-04-23T10:23:52.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:52 smithi183 ceph-mon[32317]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:23:52.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:52 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:52.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:52 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:52.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:52 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:52.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:52 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T10:23:52.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:52 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T10:23:53.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:52 smithi097 ceph-mon[29197]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:23:53.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:52 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:53.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:52 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:53.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:52 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:53.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:52 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T10:23:53.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:52 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T10:23:53.508 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T10:23:53.614 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:53.613+0000 7faefa9d8640 1 -- 172.21.15.97:0/265283070 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faef4105340 msgr2=0x7faef4107730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:53.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:53.613+0000 7faefa9d8640 1 --2- 172.21.15.97:0/265283070 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faef4105340 0x7faef4107730 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7faee8009900 tx=0x7faee802a410 comp rx=0 tx=0).stop 2024-04-23T10:23:53.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:53.613+0000 7faefa9d8640 1 -- 172.21.15.97:0/265283070 shutdown_connections 2024-04-23T10:23:53.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:53.613+0000 7faefa9d8640 1 --2- 172.21.15.97:0/265283070 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faef4107c70 0x7faef410a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:53.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:53.613+0000 7faefa9d8640 1 --2- 172.21.15.97:0/265283070 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faef4105340 0x7faef4107730 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:53.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:53.613+0000 7faefa9d8640 1 --2- 172.21.15.97:0/265283070 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faef4102a10 0x7faef4104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:53.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:53.613+0000 7faefa9d8640 1 -- 172.21.15.97:0/265283070 >> 172.21.15.97:0/265283070 conn(0x7faef40fc400 msgr2=0x7faef40fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:23:53.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:53.613+0000 7faefa9d8640 1 -- 172.21.15.97:0/265283070 shutdown_connections 2024-04-23T10:23:53.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:53.613+0000 7faefa9d8640 1 -- 172.21.15.97:0/265283070 wait complete. 2024-04-23T10:23:53.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:53.613+0000 7faefa9d8640 1 Processor -- start 2024-04-23T10:23:53.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:53.613+0000 7faefa9d8640 1 -- start start 2024-04-23T10:23:53.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:53.614+0000 7faefa9d8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faef4102a10 0x7faef419a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:53.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:53.614+0000 7faefa9d8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faef4105340 0x7faef419aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:53.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:53.614+0000 7faefa9d8640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faef4107c70 0x7faef41a1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:53.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:53.614+0000 7faefa9d8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7faef410db90 con 0x7faef4105340 2024-04-23T10:23:53.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:53.614+0000 7faefa9d8640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7faef410da10 con 0x7faef4107c70 2024-04-23T10:23:53.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:53.614+0000 7faefa9d8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faef410dd10 con 0x7faef4102a10 2024-04-23T10:23:53.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:53.614+0000 7faef37fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faef4105340 0x7faef419aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:53.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:53.614+0000 7faef37fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faef4105340 0x7faef419aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41008/0 (socket says 172.21.15.97:41008) 2024-04-23T10:23:53.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:53.614+0000 7faef37fe640 1 -- 172.21.15.97:0/2295717807 learned_addr learned my addr 172.21.15.97:0/2295717807 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:23:53.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:53.614+0000 7faef3fff640 1 --2- 172.21.15.97:0/2295717807 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faef4102a10 0x7faef419a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:53.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:53.614+0000 7faef8f75640 1 --2- 172.21.15.97:0/2295717807 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faef4107c70 0x7faef41a1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:53.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:53.614+0000 7faef37fe640 1 -- 172.21.15.97:0/2295717807 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faef4102a10 msgr2=0x7faef419a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:53.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:53.614+0000 7faef37fe640 1 --2- 172.21.15.97:0/2295717807 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faef4102a10 0x7faef419a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:53.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:53.614+0000 7faef37fe640 1 -- 172.21.15.97:0/2295717807 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faef4107c70 msgr2=0x7faef41a1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:53.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:53.614+0000 7faef37fe640 1 --2- 172.21.15.97:0/2295717807 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faef4107c70 0x7faef41a1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:53.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:53.614+0000 7faef37fe640 1 -- 172.21.15.97:0/2295717807 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faef41a2190 con 0x7faef4105340 2024-04-23T10:23:53.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:53.614+0000 7faef3fff640 1 --2- 172.21.15.97:0/2295717807 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faef4102a10 0x7faef419a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:23:53.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:53.614+0000 7faef8f75640 1 --2- 172.21.15.97:0/2295717807 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faef4107c70 0x7faef41a1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:23:53.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:53.614+0000 7faef37fe640 1 --2- 172.21.15.97:0/2295717807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faef4105340 0x7faef419aab0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7faee8006010 tx=0x7faee8004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:23:53.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:53.614+0000 7faef17fa640 1 -- 172.21.15.97:0/2295717807 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee803f030 con 0x7faef4105340 2024-04-23T10:23:53.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:53.615+0000 7faefa9d8640 1 -- 172.21.15.97:0/2295717807 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faef41a2420 con 0x7faef4105340 2024-04-23T10:23:53.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:53.615+0000 7faef17fa640 1 -- 172.21.15.97:0/2295717807 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faee8003cb0 con 0x7faef4105340 2024-04-23T10:23:53.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:53.615+0000 7faef17fa640 1 -- 172.21.15.97:0/2295717807 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee8035070 con 0x7faef4105340 2024-04-23T10:23:53.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:53.615+0000 7faefa9d8640 1 -- 172.21.15.97:0/2295717807 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faef41a2990 con 0x7faef4105340 2024-04-23T10:23:53.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:53.616+0000 7faefa9d8640 1 -- 172.21.15.97:0/2295717807 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faef4109760 con 0x7faef4105340 2024-04-23T10:23:53.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:53.617+0000 7faef17fa640 1 -- 172.21.15.97:0/2295717807 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7faee8042030 con 0x7faef4105340 2024-04-23T10:23:53.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:53.617+0000 7faef17fa640 1 --2- 172.21.15.97:0/2295717807 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7faedc0a3bb0 0x7faedc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:53.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:53.617+0000 7faef3fff640 1 --2- 172.21.15.97:0/2295717807 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7faedc0a3bb0 0x7faedc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:53.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:53.618+0000 7faef17fa640 1 -- 172.21.15.97:0/2295717807 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7faee80f0720 con 0x7faef4105340 2024-04-23T10:23:53.618 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:53.618+0000 7faef3fff640 1 --2- 172.21.15.97:0/2295717807 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7faedc0a3bb0 0x7faedc0a6070 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7faee4009930 tx=0x7faee4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:23:53.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:53.622+0000 7faef17fa640 1 -- 172.21.15.97:0/2295717807 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faee8003e60 con 0x7faef4105340 2024-04-23T10:23:53.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:53.844+0000 7faefa9d8640 1 -- 172.21.15.97:0/2295717807 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faef402d810 con 0x7faef4105340 2024-04-23T10:23:53.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:53.845+0000 7faef17fa640 1 -- 172.21.15.97:0/2295717807 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8007 (secure 0 0 0) 0x7faee80bd5a0 con 0x7faef4105340 2024-04-23T10:23:53.844 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:23:53.844 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3946878762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3946878762},{"type":"v1","addr":"172.21.15.97:6835","nonce":3946878762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/209800861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":209800861},{"type":"v1","addr":"172.21.15.97:6837","nonce":209800861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/4292847121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4292847121},{"type":"v1","addr":"172.21.15.183:6833","nonce":4292847121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3885816565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3885816565},{"type":"v1","addr":"172.21.15.196:6833","nonce":3885816565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1897093105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1897093105},{"type":"v1","addr":"172.21.15.183:6835","nonce":1897093105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1171477563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1171477563},{"type":"v1","addr":"172.21.15.196:6835","nonce":1171477563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24461,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/1898341911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1898341911},{"type":"v1","addr":"172.21.15.183:6837","nonce":1898341911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6837/3094946502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3094946502},{"type":"v1","addr":"172.21.15.196:6837","nonce":3094946502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6839/83843556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":83843556},{"type":"v1","addr":"172.21.15.196:6839","nonce":83843556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-23T10:23:53.845 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 10 2024-04-23T10:23:53.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:53.847+0000 7faefa9d8640 1 -- 172.21.15.97:0/2295717807 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7faedc0a3bb0 msgr2=0x7faedc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:53.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:53.847+0000 7faefa9d8640 1 --2- 172.21.15.97:0/2295717807 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7faedc0a3bb0 0x7faedc0a6070 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7faee4009930 tx=0x7faee4008040 comp rx=0 tx=0).stop 2024-04-23T10:23:53.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:53.847+0000 7faefa9d8640 1 -- 172.21.15.97:0/2295717807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faef4105340 msgr2=0x7faef419aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:53.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:53.847+0000 7faefa9d8640 1 --2- 172.21.15.97:0/2295717807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faef4105340 0x7faef419aab0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7faee8006010 tx=0x7faee8004430 comp rx=0 tx=0).stop 2024-04-23T10:23:53.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:53.847+0000 7faefa9d8640 1 -- 172.21.15.97:0/2295717807 shutdown_connections 2024-04-23T10:23:53.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:53.847+0000 7faefa9d8640 1 --2- 172.21.15.97:0/2295717807 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7faedc0a3bb0 0x7faedc0a6070 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:53.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:53.847+0000 7faefa9d8640 1 --2- 172.21.15.97:0/2295717807 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faef4107c70 0x7faef41a1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:53.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:53.847+0000 7faefa9d8640 1 --2- 172.21.15.97:0/2295717807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faef4105340 0x7faef419aab0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:53.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:53.847+0000 7faefa9d8640 1 --2- 172.21.15.97:0/2295717807 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faef4102a10 0x7faef419a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:53.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:53.847+0000 7faefa9d8640 1 -- 172.21.15.97:0/2295717807 >> 172.21.15.97:0/2295717807 conn(0x7faef40fc400 msgr2=0x7faef4102210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:23:53.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:53.847+0000 7faefa9d8640 1 -- 172.21.15.97:0/2295717807 shutdown_connections 2024-04-23T10:23:53.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:53.847+0000 7faefa9d8640 1 -- 172.21.15.97:0/2295717807 wait complete. 2024-04-23T10:23:53.856 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:53 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:23:53.856 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:53 smithi097 ceph-mon[29197]: Deploying daemon mds.g on smithi097 2024-04-23T10:23:53.856 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:53 smithi097 ceph-mon[29197]: mds.? [v2:172.21.15.196:6838/83843556,v1:172.21.15.196:6839/83843556] up:boot 2024-04-23T10:23:53.856 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:53 smithi097 ceph-mon[29197]: fsmap 9 up:standby 2024-04-23T10:23:53.856 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:53 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-23T10:23:54.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:53 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:23:54.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:53 smithi196 ceph-mon[32919]: Deploying daemon mds.g on smithi097 2024-04-23T10:23:54.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:53 smithi196 ceph-mon[32919]: mds.? [v2:172.21.15.196:6838/83843556,v1:172.21.15.196:6839/83843556] up:boot 2024-04-23T10:23:54.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:53 smithi196 ceph-mon[32919]: fsmap 9 up:standby 2024-04-23T10:23:54.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:53 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-23T10:23:54.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:53 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:23:54.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:53 smithi183 ceph-mon[32317]: Deploying daemon mds.g on smithi097 2024-04-23T10:23:54.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:53 smithi183 ceph-mon[32317]: mds.? [v2:172.21.15.196:6838/83843556,v1:172.21.15.196:6839/83843556] up:boot 2024-04-23T10:23:54.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:53 smithi183 ceph-mon[32317]: fsmap 9 up:standby 2024-04-23T10:23:54.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:53 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-23T10:23:54.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:54 smithi097 ceph-mon[29197]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:23:54.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:54 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/2295717807' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:23:54.861 INFO:journalctl@ceph.mds.g.smithi097.stdout:Apr 23 10:23:54 smithi097 systemd[1]: Starting Ceph mds.g for 9b22af88-0159-11ef-bc93-c7b262605968... 2024-04-23T10:23:55.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:54 smithi196 ceph-mon[32919]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:23:55.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:54 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/2295717807' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:23:55.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:54 smithi183 ceph-mon[32317]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:23:55.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:54 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/2295717807' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:23:55.251 INFO:journalctl@ceph.mds.g.smithi097.stdout:Apr 23 10:23:54 smithi097 podman[67905]: 2024-04-23 10:23:54.854158075 +0000 UTC m=+0.017839688 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T10:23:55.251 INFO:journalctl@ceph.mds.g.smithi097.stdout:Apr 23 10:23:54 smithi097 podman[67905]: 2024-04-23 10:23:54.958344254 +0000 UTC m=+0.122025866 container create ac5e05633d0d7c18661e0c519582c7846d4220f6208d0d7c5a91212aa6425e29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T10:23:55.251 INFO:journalctl@ceph.mds.g.smithi097.stdout:Apr 23 10:23:55 smithi097 podman[67905]: 2024-04-23 10:23:55.253017066 +0000 UTC m=+0.416698679 container init ac5e05633d0d7c18661e0c519582c7846d4220f6208d0d7c5a91212aa6425e29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-g, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-23T10:23:55.610 INFO:journalctl@ceph.mds.g.smithi097.stdout:Apr 23 10:23:55 smithi097 podman[67905]: 2024-04-23 10:23:55.255770308 +0000 UTC m=+0.419451926 container start ac5e05633d0d7c18661e0c519582c7846d4220f6208d0d7c5a91212aa6425e29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T10:23:55.611 INFO:journalctl@ceph.mds.g.smithi097.stdout:Apr 23 10:23:55 smithi097 ceph-mds[67924]: -- 172.21.15.97:0/663783691 <== mon.0 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560c20351a40 con 0x560c20324800 2024-04-23T10:23:55.611 INFO:journalctl@ceph.mds.g.smithi097.stdout:Apr 23 10:23:55 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-g[67920]: starting mds.g at 2024-04-23T10:23:55.611 INFO:journalctl@ceph.mds.g.smithi097.stdout:Apr 23 10:23:55 smithi097 bash[67905]: ac5e05633d0d7c18661e0c519582c7846d4220f6208d0d7c5a91212aa6425e29 2024-04-23T10:23:55.611 INFO:journalctl@ceph.mds.g.smithi097.stdout:Apr 23 10:23:55 smithi097 systemd[1]: Started Ceph mds.g for 9b22af88-0159-11ef-bc93-c7b262605968. 2024-04-23T10:23:55.858 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T10:23:55.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:55.974+0000 7f90e5457640 1 -- 172.21.15.97:0/2509400819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90e0103190 msgr2=0x7f90e0103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:55.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:55.975+0000 7f90e5457640 1 --2- 172.21.15.97:0/2509400819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90e0103190 0x7f90e0103610 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f90c8009900 tx=0x7f90c802a410 comp rx=0 tx=0).stop 2024-04-23T10:23:55.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:55.975+0000 7f90e5457640 1 -- 172.21.15.97:0/2509400819 shutdown_connections 2024-04-23T10:23:55.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:55.975+0000 7f90e5457640 1 --2- 172.21.15.97:0/2509400819 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f90e0103b50 0x7f90e010a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:55.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:55.975+0000 7f90e5457640 1 --2- 172.21.15.97:0/2509400819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90e0103190 0x7f90e0103610 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:55.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:55.975+0000 7f90e5457640 1 --2- 172.21.15.97:0/2509400819 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f90e0101fa0 0x7f90e01023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:55.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:55.975+0000 7f90e5457640 1 -- 172.21.15.97:0/2509400819 >> 172.21.15.97:0/2509400819 conn(0x7f90e00fd550 msgr2=0x7f90e00ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:23:55.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:55.975+0000 7f90e5457640 1 -- 172.21.15.97:0/2509400819 shutdown_connections 2024-04-23T10:23:55.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:55.975+0000 7f90e5457640 1 -- 172.21.15.97:0/2509400819 wait complete. 2024-04-23T10:23:55.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:55.975+0000 7f90e5457640 1 Processor -- start 2024-04-23T10:23:55.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:55.975+0000 7f90e5457640 1 -- start start 2024-04-23T10:23:55.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:55.975+0000 7f90e5457640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f90e0101fa0 0x7f90e0198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:55.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:55.975+0000 7f90e5457640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90e0103190 0x7f90e0198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:55.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:55.975+0000 7f90e5457640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f90e0103b50 0x7f90e019f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:55.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:55.975+0000 7f90e5457640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f90e010dae0 con 0x7f90e0103190 2024-04-23T10:23:55.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:55.975+0000 7f90e5457640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f90e010d960 con 0x7f90e0103b50 2024-04-23T10:23:55.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:55.975+0000 7f90e5457640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f90e010dc60 con 0x7f90e0101fa0 2024-04-23T10:23:55.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:55.975+0000 7f90d7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90e0103190 0x7f90e0198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:55.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:55.975+0000 7f90d7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90e0103190 0x7f90e0198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44636/0 (socket says 172.21.15.97:44636) 2024-04-23T10:23:55.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:55.975+0000 7f90d7fff640 1 -- 172.21.15.97:0/2196651204 learned_addr learned my addr 172.21.15.97:0/2196651204 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:23:55.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:55.975+0000 7f90deffd640 1 --2- 172.21.15.97:0/2196651204 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f90e0101fa0 0x7f90e0198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:55.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:55.975+0000 7f90df7fe640 1 --2- 172.21.15.97:0/2196651204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f90e0103b50 0x7f90e019f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:55.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:55.976+0000 7f90d7fff640 1 -- 172.21.15.97:0/2196651204 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f90e0101fa0 msgr2=0x7f90e0198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:55.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:55.976+0000 7f90d7fff640 1 --2- 172.21.15.97:0/2196651204 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f90e0101fa0 0x7f90e0198200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:55.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:55.976+0000 7f90d7fff640 1 -- 172.21.15.97:0/2196651204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f90e0103b50 msgr2=0x7f90e019f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:55.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:55.976+0000 7f90d7fff640 1 --2- 172.21.15.97:0/2196651204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f90e0103b50 0x7f90e019f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:55.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:55.976+0000 7f90d7fff640 1 -- 172.21.15.97:0/2196651204 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90e019fe20 con 0x7f90e0103190 2024-04-23T10:23:55.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:55.976+0000 7f90df7fe640 1 --2- 172.21.15.97:0/2196651204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f90e0103b50 0x7f90e019f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:23:55.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:55.976+0000 7f90deffd640 1 --2- 172.21.15.97:0/2196651204 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f90e0101fa0 0x7f90e0198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:23:55.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:55.976+0000 7f90d7fff640 1 --2- 172.21.15.97:0/2196651204 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90e0103190 0x7f90e0198740 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f90c8005940 tx=0x7f90c8004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:23:55.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:55.976+0000 7f90dcff9640 1 -- 172.21.15.97:0/2196651204 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c803f030 con 0x7f90e0103190 2024-04-23T10:23:55.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:55.976+0000 7f90e5457640 1 -- 172.21.15.97:0/2196651204 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90e01a00b0 con 0x7f90e0103190 2024-04-23T10:23:55.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:55.976+0000 7f90dcff9640 1 -- 172.21.15.97:0/2196651204 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90c8003cb0 con 0x7f90e0103190 2024-04-23T10:23:55.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:55.976+0000 7f90dcff9640 1 -- 172.21.15.97:0/2196651204 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c8035070 con 0x7f90e0103190 2024-04-23T10:23:55.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:55.976+0000 7f90e5457640 1 -- 172.21.15.97:0/2196651204 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90e01a05f0 con 0x7f90e0103190 2024-04-23T10:23:55.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:55.977+0000 7f90e5457640 1 -- 172.21.15.97:0/2196651204 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90e01093e0 con 0x7f90e0103190 2024-04-23T10:23:55.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:55.978+0000 7f90dcff9640 1 -- 172.21.15.97:0/2196651204 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f90c8042030 con 0x7f90e0103190 2024-04-23T10:23:55.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:55.978+0000 7f90dcff9640 1 --2- 172.21.15.97:0/2196651204 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f90cc0a3bb0 0x7f90cc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:55.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:55.979+0000 7f90deffd640 1 --2- 172.21.15.97:0/2196651204 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f90cc0a3bb0 0x7f90cc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:55.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:55.979+0000 7f90dcff9640 1 -- 172.21.15.97:0/2196651204 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f90c80f0be0 con 0x7f90e0103190 2024-04-23T10:23:55.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:55.979+0000 7f90deffd640 1 --2- 172.21.15.97:0/2196651204 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f90cc0a3bb0 0x7f90cc0a6070 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f90d00058c0 tx=0x7f90d0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:23:55.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:55.982+0000 7f90dcff9640 1 -- 172.21.15.97:0/2196651204 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f90c80bda60 con 0x7f90e0103190 2024-04-23T10:23:56.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:56.204+0000 7f90e5457640 1 -- 172.21.15.97:0/2196651204 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f90e002d5c0 con 0x7f90e0103190 2024-04-23T10:23:56.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:56.205+0000 7f90dcff9640 1 -- 172.21.15.97:0/2196651204 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8830 (secure 0 0 0) 0x7f90c8003e60 con 0x7f90e0103190 2024-04-23T10:23:56.204 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:23:56.204 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3946878762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3946878762},{"type":"v1","addr":"172.21.15.97:6835","nonce":3946878762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/209800861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":209800861},{"type":"v1","addr":"172.21.15.97:6837","nonce":209800861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6839/418013575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":418013575},{"type":"v1","addr":"172.21.15.97:6839","nonce":418013575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/4292847121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4292847121},{"type":"v1","addr":"172.21.15.183:6833","nonce":4292847121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3885816565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3885816565},{"type":"v1","addr":"172.21.15.196:6833","nonce":3885816565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1897093105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1897093105},{"type":"v1","addr":"172.21.15.183:6835","nonce":1897093105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1171477563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1171477563},{"type":"v1","addr":"172.21.15.196:6835","nonce":1171477563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24461,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/1898341911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1898341911},{"type":"v1","addr":"172.21.15.183:6837","nonce":1898341911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6837/3094946502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3094946502},{"type":"v1","addr":"172.21.15.196:6837","nonce":3094946502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6839/83843556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":83843556},{"type":"v1","addr":"172.21.15.196:6839","nonce":83843556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-23T10:23:56.204 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 11 2024-04-23T10:23:56.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:56.207+0000 7f90e5457640 1 -- 172.21.15.97:0/2196651204 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f90cc0a3bb0 msgr2=0x7f90cc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:56.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:56.207+0000 7f90e5457640 1 --2- 172.21.15.97:0/2196651204 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f90cc0a3bb0 0x7f90cc0a6070 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f90d00058c0 tx=0x7f90d0005850 comp rx=0 tx=0).stop 2024-04-23T10:23:56.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:56.207+0000 7f90e5457640 1 -- 172.21.15.97:0/2196651204 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90e0103190 msgr2=0x7f90e0198740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:56.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:56.207+0000 7f90e5457640 1 --2- 172.21.15.97:0/2196651204 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90e0103190 0x7f90e0198740 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f90c8005940 tx=0x7f90c8004430 comp rx=0 tx=0).stop 2024-04-23T10:23:56.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:56.207+0000 7f90e5457640 1 -- 172.21.15.97:0/2196651204 shutdown_connections 2024-04-23T10:23:56.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:56.207+0000 7f90e5457640 1 --2- 172.21.15.97:0/2196651204 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f90cc0a3bb0 0x7f90cc0a6070 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:56.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:56.207+0000 7f90e5457640 1 --2- 172.21.15.97:0/2196651204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f90e0103b50 0x7f90e019f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:56.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:56.207+0000 7f90e5457640 1 --2- 172.21.15.97:0/2196651204 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90e0103190 0x7f90e0198740 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:56.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:56.207+0000 7f90e5457640 1 --2- 172.21.15.97:0/2196651204 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f90e0101fa0 0x7f90e0198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:56.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:56.207+0000 7f90e5457640 1 -- 172.21.15.97:0/2196651204 >> 172.21.15.97:0/2196651204 conn(0x7f90e00fd550 msgr2=0x7f90e01065b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:23:56.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:56.207+0000 7f90e5457640 1 -- 172.21.15.97:0/2196651204 shutdown_connections 2024-04-23T10:23:56.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:56.207+0000 7f90e5457640 1 -- 172.21.15.97:0/2196651204 wait complete. 2024-04-23T10:23:56.422 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:56 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:56.422 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:56 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:56.422 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:56 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:56.422 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:56 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T10:23:56.422 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:56 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T10:23:56.422 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:56 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:23:56.423 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:56 smithi097 ceph-mon[29197]: Deploying daemon mds.a on smithi097 2024-04-23T10:23:56.423 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:56 smithi097 ceph-mon[29197]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:23:56.423 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:56 smithi097 ceph-mon[29197]: mds.? [v2:172.21.15.97:6838/418013575,v1:172.21.15.97:6839/418013575] up:boot 2024-04-23T10:23:56.423 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:56 smithi097 ceph-mon[29197]: fsmap 10 up:standby 2024-04-23T10:23:56.423 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:56 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-23T10:23:56.423 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:56 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/2196651204' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:23:56.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:56 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:56.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:56 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:56.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:56 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:56.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:56 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T10:23:56.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:56 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T10:23:56.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:56 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:23:56.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:56 smithi183 ceph-mon[32317]: Deploying daemon mds.a on smithi097 2024-04-23T10:23:56.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:56 smithi183 ceph-mon[32317]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:23:56.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:56 smithi183 ceph-mon[32317]: mds.? [v2:172.21.15.97:6838/418013575,v1:172.21.15.97:6839/418013575] up:boot 2024-04-23T10:23:56.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:56 smithi183 ceph-mon[32317]: fsmap 10 up:standby 2024-04-23T10:23:56.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:56 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-23T10:23:56.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:56 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/2196651204' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:23:56.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:56 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:56.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:56 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:56.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:56 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:56.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:56 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T10:23:56.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:56 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T10:23:56.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:56 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:23:56.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:56 smithi196 ceph-mon[32919]: Deploying daemon mds.a on smithi097 2024-04-23T10:23:56.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:56 smithi196 ceph-mon[32919]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:23:56.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:56 smithi196 ceph-mon[32919]: mds.? [v2:172.21.15.97:6838/418013575,v1:172.21.15.97:6839/418013575] up:boot 2024-04-23T10:23:56.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:56 smithi196 ceph-mon[32919]: fsmap 10 up:standby 2024-04-23T10:23:56.843 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:56 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-23T10:23:56.843 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:56 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/2196651204' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:23:57.792 INFO:journalctl@ceph.mds.a.smithi097.stdout:Apr 23 10:23:57 smithi097 systemd[1]: Starting Ceph mds.a for 9b22af88-0159-11ef-bc93-c7b262605968... 2024-04-23T10:23:58.067 INFO:journalctl@ceph.mds.a.smithi097.stdout:Apr 23 10:23:57 smithi097 podman[68265]: 2024-04-23 10:23:57.69615051 +0000 UTC m=+0.017684522 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T10:23:58.067 INFO:journalctl@ceph.mds.a.smithi097.stdout:Apr 23 10:23:57 smithi097 podman[68265]: 2024-04-23 10:23:57.798516908 +0000 UTC m=+0.120050893 container create 1ac5fb4846ea1de32c2e0ce6bb29d77d76f7bcedc3b04f48db995270ba742ec4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-a, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T10:23:58.067 INFO:journalctl@ceph.mds.a.smithi097.stdout:Apr 23 10:23:58 smithi097 podman[68265]: 2024-04-23 10:23:58.068443993 +0000 UTC m=+0.389977987 container init 1ac5fb4846ea1de32c2e0ce6bb29d77d76f7bcedc3b04f48db995270ba742ec4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-a, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T10:23:58.218 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T10:23:58.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:58.323+0000 7fb60062a640 1 -- 172.21.15.97:0/1902451829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5f8103190 msgr2=0x7fb5f8103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:58.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:58.323+0000 7fb60062a640 1 --2- 172.21.15.97:0/1902451829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5f8103190 0x7fb5f8103610 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8009900 tx=0x7fb5e802a410 comp rx=0 tx=0).stop 2024-04-23T10:23:58.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:58.323+0000 7fb60062a640 1 -- 172.21.15.97:0/1902451829 shutdown_connections 2024-04-23T10:23:58.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:58.323+0000 7fb60062a640 1 --2- 172.21.15.97:0/1902451829 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5f8103b50 0x7fb5f810a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:58.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:58.323+0000 7fb60062a640 1 --2- 172.21.15.97:0/1902451829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5f8103190 0x7fb5f8103610 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:58.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:58.323+0000 7fb60062a640 1 --2- 172.21.15.97:0/1902451829 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb5f8101fa0 0x7fb5f81023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:58.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:58.323+0000 7fb60062a640 1 -- 172.21.15.97:0/1902451829 >> 172.21.15.97:0/1902451829 conn(0x7fb5f80fd530 msgr2=0x7fb5f80ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:23:58.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:58.323+0000 7fb60062a640 1 -- 172.21.15.97:0/1902451829 shutdown_connections 2024-04-23T10:23:58.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:58.323+0000 7fb60062a640 1 -- 172.21.15.97:0/1902451829 wait complete. 2024-04-23T10:23:58.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:58.324+0000 7fb60062a640 1 Processor -- start 2024-04-23T10:23:58.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:58.324+0000 7fb60062a640 1 -- start start 2024-04-23T10:23:58.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:58.324+0000 7fb60062a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5f8101fa0 0x7fb5f819a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:58.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:58.324+0000 7fb60062a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb5f8103190 0x7fb5f819a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:58.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:58.324+0000 7fb60062a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5f8103b50 0x7fb5f81a19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:58.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:58.324+0000 7fb60062a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f810dae0 con 0x7fb5f8101fa0 2024-04-23T10:23:58.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:58.324+0000 7fb60062a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f810d960 con 0x7fb5f8103190 2024-04-23T10:23:58.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:58.324+0000 7fb60062a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f810dc60 con 0x7fb5f8103b50 2024-04-23T10:23:58.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:58.324+0000 7fb5fe3c6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5f8101fa0 0x7fb5f819a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:58.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:58.324+0000 7fb5fe3c6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5f8101fa0 0x7fb5f819a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44684/0 (socket says 172.21.15.97:44684) 2024-04-23T10:23:58.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:58.324+0000 7fb5fe3c6640 1 -- 172.21.15.97:0/2029928880 learned_addr learned my addr 172.21.15.97:0/2029928880 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:23:58.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:58.324+0000 7fb5fdbc5640 1 --2- 172.21.15.97:0/2029928880 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb5f8103190 0x7fb5f819a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:58.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:58.324+0000 7fb5fe3c6640 1 -- 172.21.15.97:0/2029928880 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5f8103b50 msgr2=0x7fb5f81a19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:58.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:58.324+0000 7fb5fe3c6640 1 --2- 172.21.15.97:0/2029928880 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5f8103b50 0x7fb5f81a19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:58.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:58.324+0000 7fb5fe3c6640 1 -- 172.21.15.97:0/2029928880 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb5f8103190 msgr2=0x7fb5f819a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:58.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:58.324+0000 7fb5fe3c6640 1 --2- 172.21.15.97:0/2029928880 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb5f8103190 0x7fb5f819a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:58.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:58.324+0000 7fb5fe3c6640 1 -- 172.21.15.97:0/2029928880 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5f81a2030 con 0x7fb5f8101fa0 2024-04-23T10:23:58.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:58.325+0000 7fb5fe3c6640 1 --2- 172.21.15.97:0/2029928880 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5f8101fa0 0x7fb5f819a410 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb5f400e9c0 tx=0x7fb5f400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:23:58.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:58.325+0000 7fb5ef7fe640 1 -- 172.21.15.97:0/2029928880 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f400cd80 con 0x7fb5f8101fa0 2024-04-23T10:23:58.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:58.325+0000 7fb5ef7fe640 1 -- 172.21.15.97:0/2029928880 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5f40035f0 con 0x7fb5f8101fa0 2024-04-23T10:23:58.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:58.325+0000 7fb5ef7fe640 1 -- 172.21.15.97:0/2029928880 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f4010690 con 0x7fb5f8101fa0 2024-04-23T10:23:58.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:58.325+0000 7fb60062a640 1 -- 172.21.15.97:0/2029928880 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5f81a2320 con 0x7fb5f8101fa0 2024-04-23T10:23:58.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:58.325+0000 7fb60062a640 1 -- 172.21.15.97:0/2029928880 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5f81a2840 con 0x7fb5f8101fa0 2024-04-23T10:23:58.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:58.326+0000 7fb60062a640 1 -- 172.21.15.97:0/2029928880 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5f81a2b00 con 0x7fb5f8101fa0 2024-04-23T10:23:58.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:58.327+0000 7fb5ef7fe640 1 -- 172.21.15.97:0/2029928880 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb5f40037a0 con 0x7fb5f8101fa0 2024-04-23T10:23:58.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:58.327+0000 7fb5ef7fe640 1 --2- 172.21.15.97:0/2029928880 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fb5e40a3bb0 0x7fb5e40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:23:58.330 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:58.327+0000 7fb5fdbc5640 1 --2- 172.21.15.97:0/2029928880 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fb5e40a3bb0 0x7fb5e40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:23:58.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:58.328+0000 7fb5ef7fe640 1 -- 172.21.15.97:0/2029928880 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fb5f40d0490 con 0x7fb5f8101fa0 2024-04-23T10:23:58.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:58.328+0000 7fb5fdbc5640 1 --2- 172.21.15.97:0/2029928880 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fb5e40a3bb0 0x7fb5e40a6070 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8000c00 tx=0x7fb5e80314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:23:58.331 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:58.331+0000 7fb5ef7fe640 1 -- 172.21.15.97:0/2029928880 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb5f409d310 con 0x7fb5f8101fa0 2024-04-23T10:23:58.360 INFO:journalctl@ceph.mds.a.smithi097.stdout:Apr 23 10:23:58 smithi097 podman[68265]: 2024-04-23 10:23:58.071290655 +0000 UTC m=+0.392824641 container start 1ac5fb4846ea1de32c2e0ce6bb29d77d76f7bcedc3b04f48db995270ba742ec4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-a, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-23T10:23:58.361 INFO:journalctl@ceph.mds.a.smithi097.stdout:Apr 23 10:23:58 smithi097 ceph-mds[68284]: -- 172.21.15.97:0/3489059582 <== mon.0 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559532935a40 con 0x559532908800 2024-04-23T10:23:58.361 INFO:journalctl@ceph.mds.a.smithi097.stdout:Apr 23 10:23:58 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-a[68280]: starting mds.a at 2024-04-23T10:23:58.361 INFO:journalctl@ceph.mds.a.smithi097.stdout:Apr 23 10:23:58 smithi097 bash[68265]: 1ac5fb4846ea1de32c2e0ce6bb29d77d76f7bcedc3b04f48db995270ba742ec4 2024-04-23T10:23:58.361 INFO:journalctl@ceph.mds.a.smithi097.stdout:Apr 23 10:23:58 smithi097 systemd[1]: Started Ceph mds.a for 9b22af88-0159-11ef-bc93-c7b262605968. 2024-04-23T10:23:58.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:58.548+0000 7fb60062a640 1 -- 172.21.15.97:0/2029928880 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb5f81a2d70 con 0x7fb5f8101fa0 2024-04-23T10:23:58.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:58.548+0000 7fb5ef7fe640 1 -- 172.21.15.97:0/2029928880 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8830 (secure 0 0 0) 0x7fb5f409cc10 con 0x7fb5f8101fa0 2024-04-23T10:23:58.547 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:23:58.548 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3946878762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3946878762},{"type":"v1","addr":"172.21.15.97:6835","nonce":3946878762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/209800861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":209800861},{"type":"v1","addr":"172.21.15.97:6837","nonce":209800861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6839/418013575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":418013575},{"type":"v1","addr":"172.21.15.97:6839","nonce":418013575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/4292847121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4292847121},{"type":"v1","addr":"172.21.15.183:6833","nonce":4292847121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3885816565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3885816565},{"type":"v1","addr":"172.21.15.196:6833","nonce":3885816565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1897093105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1897093105},{"type":"v1","addr":"172.21.15.183:6835","nonce":1897093105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1171477563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1171477563},{"type":"v1","addr":"172.21.15.196:6835","nonce":1171477563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24461,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/1898341911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1898341911},{"type":"v1","addr":"172.21.15.183:6837","nonce":1898341911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6837/3094946502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3094946502},{"type":"v1","addr":"172.21.15.196:6837","nonce":3094946502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6839/83843556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":83843556},{"type":"v1","addr":"172.21.15.196:6839","nonce":83843556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-23T10:23:58.548 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 11 2024-04-23T10:23:58.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:58.550+0000 7fb60062a640 1 -- 172.21.15.97:0/2029928880 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fb5e40a3bb0 msgr2=0x7fb5e40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:58.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:58.550+0000 7fb60062a640 1 --2- 172.21.15.97:0/2029928880 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fb5e40a3bb0 0x7fb5e40a6070 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8000c00 tx=0x7fb5e80314a0 comp rx=0 tx=0).stop 2024-04-23T10:23:58.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:58.550+0000 7fb60062a640 1 -- 172.21.15.97:0/2029928880 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5f8101fa0 msgr2=0x7fb5f819a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:23:58.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:58.550+0000 7fb60062a640 1 --2- 172.21.15.97:0/2029928880 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5f8101fa0 0x7fb5f819a410 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb5f400e9c0 tx=0x7fb5f400ee80 comp rx=0 tx=0).stop 2024-04-23T10:23:58.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:58.550+0000 7fb60062a640 1 -- 172.21.15.97:0/2029928880 shutdown_connections 2024-04-23T10:23:58.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:58.550+0000 7fb60062a640 1 --2- 172.21.15.97:0/2029928880 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fb5e40a3bb0 0x7fb5e40a6070 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:58.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:58.550+0000 7fb60062a640 1 --2- 172.21.15.97:0/2029928880 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5f8103b50 0x7fb5f81a19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:58.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:58.550+0000 7fb60062a640 1 --2- 172.21.15.97:0/2029928880 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb5f8103190 0x7fb5f819a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:58.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:58.550+0000 7fb60062a640 1 --2- 172.21.15.97:0/2029928880 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5f8101fa0 0x7fb5f819a410 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:23:58.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:58.550+0000 7fb60062a640 1 -- 172.21.15.97:0/2029928880 >> 172.21.15.97:0/2029928880 conn(0x7fb5f80fd530 msgr2=0x7fb5f8074180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:23:58.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:58.551+0000 7fb60062a640 1 -- 172.21.15.97:0/2029928880 shutdown_connections 2024-04-23T10:23:58.550 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:23:58.551+0000 7fb60062a640 1 -- 172.21.15.97:0/2029928880 wait complete. 2024-04-23T10:23:58.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:58 smithi097 ceph-mon[29197]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:23:58.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:58 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:58.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:58 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:58.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:58 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:58.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:58 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T10:23:58.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:58 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T10:23:58.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:58 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:23:58.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:58 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/2029928880' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:23:58.873 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:58 smithi183 ceph-mon[32317]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:23:58.873 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:58 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:58.874 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:58 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:58.874 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:58 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:58.874 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:58 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T10:23:58.874 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:58 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T10:23:58.874 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:58 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:23:58.874 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:58 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/2029928880' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:23:59.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:58 smithi196 ceph-mon[32919]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:23:59.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:58 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:59.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:58 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:59.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:58 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:23:59.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:58 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T10:23:59.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:58 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T10:23:59.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:58 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:23:59.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:58 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/2029928880' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:23:59.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:59 smithi097 ceph-mon[29197]: Deploying daemon mds.b on smithi183 2024-04-23T10:23:59.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:59 smithi097 ceph-mon[29197]: mds.? [v2:172.21.15.97:6840/747242680,v1:172.21.15.97:6841/747242680] up:boot 2024-04-23T10:23:59.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:59 smithi097 ceph-mon[29197]: fsmap 11 up:standby 2024-04-23T10:23:59.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:23:59 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-23T10:23:59.877 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:59 smithi183 ceph-mon[32317]: Deploying daemon mds.b on smithi183 2024-04-23T10:23:59.877 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:59 smithi183 ceph-mon[32317]: mds.? [v2:172.21.15.97:6840/747242680,v1:172.21.15.97:6841/747242680] up:boot 2024-04-23T10:23:59.877 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:59 smithi183 ceph-mon[32317]: fsmap 11 up:standby 2024-04-23T10:23:59.877 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:23:59 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-23T10:24:00.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:59 smithi196 ceph-mon[32919]: Deploying daemon mds.b on smithi183 2024-04-23T10:24:00.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:59 smithi196 ceph-mon[32919]: mds.? [v2:172.21.15.97:6840/747242680,v1:172.21.15.97:6841/747242680] up:boot 2024-04-23T10:24:00.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:59 smithi196 ceph-mon[32919]: fsmap 11 up:standby 2024-04-23T10:24:00.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:23:59 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-23T10:24:00.562 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T10:24:00.619 INFO:journalctl@ceph.mds.b.smithi183.stdout:Apr 23 10:24:00 smithi183 systemd[1]: Starting Ceph mds.b for 9b22af88-0159-11ef-bc93-c7b262605968... 2024-04-23T10:24:00.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:00.677+0000 7f261e144640 1 -- 172.21.15.97:0/3760061826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2618100e90 msgr2=0x7f26181012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:00.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:00.677+0000 7f261e144640 1 --2- 172.21.15.97:0/3760061826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2618100e90 0x7f26181012b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f260c009900 tx=0x7f260c02a410 comp rx=0 tx=0).stop 2024-04-23T10:24:00.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:00.677+0000 7f261e144640 1 -- 172.21.15.97:0/3760061826 shutdown_connections 2024-04-23T10:24:00.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:00.677+0000 7f261e144640 1 --2- 172.21.15.97:0/3760061826 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26181029a0 0x7f26181092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:00.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:00.677+0000 7f261e144640 1 --2- 172.21.15.97:0/3760061826 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2618101fe0 0x7f2618102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:00.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:00.677+0000 7f261e144640 1 --2- 172.21.15.97:0/3760061826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2618100e90 0x7f26181012b0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:00.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:00.677+0000 7f261e144640 1 -- 172.21.15.97:0/3760061826 >> 172.21.15.97:0/3760061826 conn(0x7f26180fc400 msgr2=0x7f26180fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:00.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:00.677+0000 7f261e144640 1 -- 172.21.15.97:0/3760061826 shutdown_connections 2024-04-23T10:24:00.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:00.677+0000 7f261e144640 1 -- 172.21.15.97:0/3760061826 wait complete. 2024-04-23T10:24:00.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:00.677+0000 7f261e144640 1 Processor -- start 2024-04-23T10:24:00.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:00.677+0000 7f261e144640 1 -- start start 2024-04-23T10:24:00.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:00.678+0000 7f261e144640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2618100e90 0x7f2618069c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:00.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:00.678+0000 7f261e144640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2618101fe0 0x7f261819cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:00.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:00.678+0000 7f261e144640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26181029a0 0x7f261819f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:00.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:00.678+0000 7f261e144640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f261810ca30 con 0x7f2618100e90 2024-04-23T10:24:00.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:00.678+0000 7f261e144640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f261810c8b0 con 0x7f2618101fe0 2024-04-23T10:24:00.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:00.678+0000 7f261e144640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f261810cbb0 con 0x7f26181029a0 2024-04-23T10:24:00.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:00.678+0000 7f26177fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2618100e90 0x7f2618069c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:00.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:00.678+0000 7f26177fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2618100e90 0x7f2618069c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44698/0 (socket says 172.21.15.97:44698) 2024-04-23T10:24:00.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:00.678+0000 7f26177fe640 1 -- 172.21.15.97:0/2569069952 learned_addr learned my addr 172.21.15.97:0/2569069952 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:24:00.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:00.678+0000 7f2616ffd640 1 --2- 172.21.15.97:0/2569069952 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2618101fe0 0x7f261819cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:00.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:00.678+0000 7f2617fff640 1 --2- 172.21.15.97:0/2569069952 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26181029a0 0x7f261819f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:00.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:00.678+0000 7f26177fe640 1 -- 172.21.15.97:0/2569069952 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26181029a0 msgr2=0x7f261819f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:00.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:00.678+0000 7f26177fe640 1 --2- 172.21.15.97:0/2569069952 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26181029a0 0x7f261819f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:00.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:00.678+0000 7f26177fe640 1 -- 172.21.15.97:0/2569069952 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2618101fe0 msgr2=0x7f261819cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:00.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:00.678+0000 7f26177fe640 1 --2- 172.21.15.97:0/2569069952 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2618101fe0 0x7f261819cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:00.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:00.678+0000 7f26177fe640 1 -- 172.21.15.97:0/2569069952 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f261819fe00 con 0x7f2618100e90 2024-04-23T10:24:00.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:00.678+0000 7f2616ffd640 1 --2- 172.21.15.97:0/2569069952 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2618101fe0 0x7f261819cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T10:24:00.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:00.678+0000 7f26177fe640 1 --2- 172.21.15.97:0/2569069952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2618100e90 0x7f2618069c70 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f260c005ab0 tx=0x7f260c003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:00.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:00.678+0000 7f2614ff9640 1 -- 172.21.15.97:0/2569069952 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260c03f070 con 0x7f2618100e90 2024-04-23T10:24:00.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:00.679+0000 7f2614ff9640 1 -- 172.21.15.97:0/2569069952 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f260c03b460 con 0x7f2618100e90 2024-04-23T10:24:00.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:00.679+0000 7f261e144640 1 -- 172.21.15.97:0/2569069952 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26181a0090 con 0x7f2618100e90 2024-04-23T10:24:00.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:00.679+0000 7f2614ff9640 1 -- 172.21.15.97:0/2569069952 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260c035030 con 0x7f2618100e90 2024-04-23T10:24:00.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:00.679+0000 7f261e144640 1 -- 172.21.15.97:0/2569069952 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26181a0580 con 0x7f2618100e90 2024-04-23T10:24:00.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:00.679+0000 7f261e144640 1 -- 172.21.15.97:0/2569069952 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26181081a0 con 0x7f2618100e90 2024-04-23T10:24:00.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:00.681+0000 7f2614ff9640 1 -- 172.21.15.97:0/2569069952 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f260c042070 con 0x7f2618100e90 2024-04-23T10:24:00.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:00.681+0000 7f2614ff9640 1 --2- 172.21.15.97:0/2569069952 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f26000a3bb0 0x7f26000a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:00.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:00.681+0000 7f2616ffd640 1 --2- 172.21.15.97:0/2569069952 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f26000a3bb0 0x7f26000a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:00.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:00.681+0000 7f2614ff9640 1 -- 172.21.15.97:0/2569069952 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f260c0f0b90 con 0x7f2618100e90 2024-04-23T10:24:00.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:00.682+0000 7f2616ffd640 1 --2- 172.21.15.97:0/2569069952 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f26000a3bb0 0x7f26000a6070 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f26180699e0 tx=0x7f260800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:00.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:00.685+0000 7f2614ff9640 1 -- 172.21.15.97:0/2569069952 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f260c0bd3c0 con 0x7f2618100e90 2024-04-23T10:24:00.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:00 smithi097 ceph-mon[29197]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:24:00.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:00 smithi183 ceph-mon[32317]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:24:00.900 INFO:journalctl@ceph.mds.b.smithi183.stdout:Apr 23 10:24:00 smithi183 podman[62547]: 2024-04-23 10:24:00.718108769 +0000 UTC m=+0.112151956 container create fddaf1fb6bb473828f67bd8156924c51ab64bcddff9dca253bb81a695a386833 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-b, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-23T10:24:00.900 INFO:journalctl@ceph.mds.b.smithi183.stdout:Apr 23 10:24:00 smithi183 podman[62547]: 2024-04-23 10:24:00.624895517 +0000 UTC m=+0.018938673 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T10:24:00.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:00.903+0000 7f261e144640 1 -- 172.21.15.97:0/2569069952 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f261802d5c0 con 0x7f2618100e90 2024-04-23T10:24:00.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:00.904+0000 7f2614ff9640 1 -- 172.21.15.97:0/2569069952 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9653 (secure 0 0 0) 0x7f260c0c16c0 con 0x7f2618100e90 2024-04-23T10:24:00.903 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:24:00.903 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3946878762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3946878762},{"type":"v1","addr":"172.21.15.97:6835","nonce":3946878762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/209800861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":209800861},{"type":"v1","addr":"172.21.15.97:6837","nonce":209800861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6839/418013575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":418013575},{"type":"v1","addr":"172.21.15.97:6839","nonce":418013575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6841/747242680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":747242680},{"type":"v1","addr":"172.21.15.97:6841","nonce":747242680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/4292847121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4292847121},{"type":"v1","addr":"172.21.15.183:6833","nonce":4292847121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3885816565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3885816565},{"type":"v1","addr":"172.21.15.196:6833","nonce":3885816565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1897093105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1897093105},{"type":"v1","addr":"172.21.15.183:6835","nonce":1897093105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1171477563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1171477563},{"type":"v1","addr":"172.21.15.196:6835","nonce":1171477563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24461,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/1898341911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1898341911},{"type":"v1","addr":"172.21.15.183:6837","nonce":1898341911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6837/3094946502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3094946502},{"type":"v1","addr":"172.21.15.196:6837","nonce":3094946502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6839/83843556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":83843556},{"type":"v1","addr":"172.21.15.196:6839","nonce":83843556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-23T10:24:00.903 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 12 2024-04-23T10:24:00.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:00.906+0000 7f261e144640 1 -- 172.21.15.97:0/2569069952 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f26000a3bb0 msgr2=0x7f26000a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:00.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:00.906+0000 7f261e144640 1 --2- 172.21.15.97:0/2569069952 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f26000a3bb0 0x7f26000a6070 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f26180699e0 tx=0x7f260800b390 comp rx=0 tx=0).stop 2024-04-23T10:24:00.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:00.906+0000 7f261e144640 1 -- 172.21.15.97:0/2569069952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2618100e90 msgr2=0x7f2618069c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:00.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:00.906+0000 7f261e144640 1 --2- 172.21.15.97:0/2569069952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2618100e90 0x7f2618069c70 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f260c005ab0 tx=0x7f260c003cd0 comp rx=0 tx=0).stop 2024-04-23T10:24:00.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:00.906+0000 7f261e144640 1 -- 172.21.15.97:0/2569069952 shutdown_connections 2024-04-23T10:24:00.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:00.906+0000 7f261e144640 1 --2- 172.21.15.97:0/2569069952 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f26000a3bb0 0x7f26000a6070 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:00.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:00.906+0000 7f261e144640 1 --2- 172.21.15.97:0/2569069952 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26181029a0 0x7f261819f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:00.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:00.906+0000 7f261e144640 1 --2- 172.21.15.97:0/2569069952 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2618101fe0 0x7f261819cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:00.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:00.906+0000 7f261e144640 1 --2- 172.21.15.97:0/2569069952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2618100e90 0x7f2618069c70 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:00.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:00.906+0000 7f261e144640 1 -- 172.21.15.97:0/2569069952 >> 172.21.15.97:0/2569069952 conn(0x7f26180fc400 msgr2=0x7f2618105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:00.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:00.906+0000 7f261e144640 1 -- 172.21.15.97:0/2569069952 shutdown_connections 2024-04-23T10:24:00.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:00.906+0000 7f261e144640 1 -- 172.21.15.97:0/2569069952 wait complete. 2024-04-23T10:24:01.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:00 smithi196 ceph-mon[32919]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:24:01.400 INFO:journalctl@ceph.mds.b.smithi183.stdout:Apr 23 10:24:00 smithi183 podman[62547]: 2024-04-23 10:24:00.970885343 +0000 UTC m=+0.364928485 container init fddaf1fb6bb473828f67bd8156924c51ab64bcddff9dca253bb81a695a386833 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-b, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS) 2024-04-23T10:24:01.400 INFO:journalctl@ceph.mds.b.smithi183.stdout:Apr 23 10:24:00 smithi183 podman[62547]: 2024-04-23 10:24:00.973389643 +0000 UTC m=+0.367432785 container start fddaf1fb6bb473828f67bd8156924c51ab64bcddff9dca253bb81a695a386833 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T10:24:01.400 INFO:journalctl@ceph.mds.b.smithi183.stdout:Apr 23 10:24:00 smithi183 ceph-mds[62566]: -- 172.21.15.183:0/3156671961 <== mon.2 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56225d41ba40 con 0x56225d3eec00 2024-04-23T10:24:01.400 INFO:journalctl@ceph.mds.b.smithi183.stdout:Apr 23 10:24:00 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-b[62562]: starting mds.b at 2024-04-23T10:24:01.400 INFO:journalctl@ceph.mds.b.smithi183.stdout:Apr 23 10:24:01 smithi183 bash[62547]: fddaf1fb6bb473828f67bd8156924c51ab64bcddff9dca253bb81a695a386833 2024-04-23T10:24:01.400 INFO:journalctl@ceph.mds.b.smithi183.stdout:Apr 23 10:24:01 smithi183 systemd[1]: Started Ceph mds.b for 9b22af88-0159-11ef-bc93-c7b262605968. 2024-04-23T10:24:01.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:01 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/2569069952' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:24:01.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:01 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:24:01.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:01 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:24:01.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:01 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:24:01.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:01 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:24:01.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:01 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:24:01.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:01 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:24:02.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:01 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/2569069952' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:24:02.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:01 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:24:02.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:01 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:24:02.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:01 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:24:02.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:01 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:24:02.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:01 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:24:02.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:01 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:24:02.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:01 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/2569069952' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:24:02.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:01 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:24:02.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:01 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:24:02.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:01 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:24:02.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:01 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:24:02.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:01 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:24:02.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:01 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:24:02.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:02 smithi097 ceph-mon[29197]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:24:02.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:02 smithi097 ceph-mon[29197]: mds.? [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] up:boot 2024-04-23T10:24:02.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:02 smithi097 ceph-mon[29197]: fsmap 12 up:standby 2024-04-23T10:24:02.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:02 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-23T10:24:02.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:02 smithi183 ceph-mon[32317]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:24:02.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:02 smithi183 ceph-mon[32317]: mds.? [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] up:boot 2024-04-23T10:24:02.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:02 smithi183 ceph-mon[32317]: fsmap 12 up:standby 2024-04-23T10:24:02.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:02 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-23T10:24:02.918 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T10:24:03.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.046+0000 7f0965623640 1 -- 172.21.15.97:0/193369818 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0960107cf0 msgr2=0x7f096010a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:03.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.046+0000 7f0965623640 1 --2- 172.21.15.97:0/193369818 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0960107cf0 0x7f096010a0e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0958009ca0 tx=0x7f095802a8c0 comp rx=0 tx=0).stop 2024-04-23T10:24:03.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.046+0000 7f0965623640 1 -- 172.21.15.97:0/193369818 shutdown_connections 2024-04-23T10:24:03.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.046+0000 7f0965623640 1 --2- 172.21.15.97:0/193369818 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0960107cf0 0x7f096010a0e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:03.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.046+0000 7f0965623640 1 --2- 172.21.15.97:0/193369818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09601053c0 0x7f09601077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:03.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.046+0000 7f0965623640 1 --2- 172.21.15.97:0/193369818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0960102a90 0x7f0960104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:03.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.046+0000 7f0965623640 1 -- 172.21.15.97:0/193369818 >> 172.21.15.97:0/193369818 conn(0x7f09600fc440 msgr2=0x7f09600fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:03.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.046+0000 7f0965623640 1 -- 172.21.15.97:0/193369818 shutdown_connections 2024-04-23T10:24:03.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.046+0000 7f0965623640 1 -- 172.21.15.97:0/193369818 wait complete. 2024-04-23T10:24:03.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.046+0000 7f0965623640 1 Processor -- start 2024-04-23T10:24:03.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.047+0000 7f0965623640 1 -- start start 2024-04-23T10:24:03.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.047+0000 7f0965623640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0960102a90 0x7f09601981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:03.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.047+0000 7f0965623640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09601053c0 0x7f09601986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:03.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.047+0000 7f0965623640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0960107cf0 0x7f096019f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:03.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.047+0000 7f0965623640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f096010daf0 con 0x7f09601053c0 2024-04-23T10:24:03.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.047+0000 7f0965623640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f096010d970 con 0x7f0960102a90 2024-04-23T10:24:03.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.047+0000 7f0965623640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f096010dc70 con 0x7f0960107cf0 2024-04-23T10:24:03.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.047+0000 7f095e7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09601053c0 0x7f09601986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:03.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.047+0000 7f095e7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09601053c0 0x7f09601986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44720/0 (socket says 172.21.15.97:44720) 2024-04-23T10:24:03.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.047+0000 7f095e7fc640 1 -- 172.21.15.97:0/566280758 learned_addr learned my addr 172.21.15.97:0/566280758 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:24:03.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.047+0000 7f095f7fe640 1 --2- 172.21.15.97:0/566280758 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0960107cf0 0x7f096019f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:03.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.047+0000 7f095effd640 1 --2- 172.21.15.97:0/566280758 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0960102a90 0x7f09601981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:03.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.047+0000 7f095e7fc640 1 -- 172.21.15.97:0/566280758 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0960107cf0 msgr2=0x7f096019f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:03.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.047+0000 7f095e7fc640 1 --2- 172.21.15.97:0/566280758 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0960107cf0 0x7f096019f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:03.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.047+0000 7f095e7fc640 1 -- 172.21.15.97:0/566280758 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0960102a90 msgr2=0x7f09601981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:03.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.047+0000 7f095e7fc640 1 --2- 172.21.15.97:0/566280758 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0960102a90 0x7f09601981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:03.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.047+0000 7f095e7fc640 1 -- 172.21.15.97:0/566280758 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f096019fdc0 con 0x7f09601053c0 2024-04-23T10:24:03.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.047+0000 7f095e7fc640 1 --2- 172.21.15.97:0/566280758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09601053c0 0x7f09601986e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f095400e9f0 tx=0x7f095400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:03.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.048+0000 7f0947fff640 1 -- 172.21.15.97:0/566280758 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095400ccc0 con 0x7f09601053c0 2024-04-23T10:24:03.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.048+0000 7f0965623640 1 -- 172.21.15.97:0/566280758 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09601a00b0 con 0x7f09601053c0 2024-04-23T10:24:03.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.048+0000 7f0947fff640 1 -- 172.21.15.97:0/566280758 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09540054d0 con 0x7f09601053c0 2024-04-23T10:24:03.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.048+0000 7f0947fff640 1 -- 172.21.15.97:0/566280758 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0954010430 con 0x7f09601053c0 2024-04-23T10:24:03.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.048+0000 7f0965623640 1 -- 172.21.15.97:0/566280758 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09601a05b0 con 0x7f09601053c0 2024-04-23T10:24:03.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.049+0000 7f0965623640 1 -- 172.21.15.97:0/566280758 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09600fdf30 con 0x7f09601053c0 2024-04-23T10:24:03.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.050+0000 7f0947fff640 1 -- 172.21.15.97:0/566280758 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f09540046c0 con 0x7f09601053c0 2024-04-23T10:24:03.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.050+0000 7f0947fff640 1 --2- 172.21.15.97:0/566280758 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f09480a3bb0 0x7f09480a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:03.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.050+0000 7f095effd640 1 --2- 172.21.15.97:0/566280758 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f09480a3bb0 0x7f09480a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:03.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.050+0000 7f0947fff640 1 -- 172.21.15.97:0/566280758 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f09540d02a0 con 0x7f09601053c0 2024-04-23T10:24:03.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.051+0000 7f095effd640 1 --2- 172.21.15.97:0/566280758 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f09480a3bb0 0x7f09480a6070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f095000bbe0 tx=0x7f095000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:03.054 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.055+0000 7f0947fff640 1 -- 172.21.15.97:0/566280758 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f095409d120 con 0x7f09601053c0 2024-04-23T10:24:03.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:02 smithi196 ceph-mon[32919]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:24:03.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:02 smithi196 ceph-mon[32919]: mds.? [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] up:boot 2024-04-23T10:24:03.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:02 smithi196 ceph-mon[32919]: fsmap 12 up:standby 2024-04-23T10:24:03.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:02 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-23T10:24:03.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.305+0000 7f0965623640 1 -- 172.21.15.97:0/566280758 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f096002d5c0 con 0x7f09601053c0 2024-04-23T10:24:03.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.305+0000 7f0947fff640 1 -- 172.21.15.97:0/566280758 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10482 (secure 0 0 0) 0x7f095409ca20 con 0x7f09601053c0 2024-04-23T10:24:03.304 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:24:03.305 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3946878762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3946878762},{"type":"v1","addr":"172.21.15.97:6835","nonce":3946878762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/209800861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":209800861},{"type":"v1","addr":"172.21.15.97:6837","nonce":209800861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6839/418013575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":418013575},{"type":"v1","addr":"172.21.15.97:6839","nonce":418013575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6841/747242680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":747242680},{"type":"v1","addr":"172.21.15.97:6841","nonce":747242680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/4292847121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4292847121},{"type":"v1","addr":"172.21.15.183:6833","nonce":4292847121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3885816565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3885816565},{"type":"v1","addr":"172.21.15.196:6833","nonce":3885816565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/1897093105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1897093105},{"type":"v1","addr":"172.21.15.183:6835","nonce":1897093105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1171477563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1171477563},{"type":"v1","addr":"172.21.15.196:6835","nonce":1171477563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24461,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/1898341911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1898341911},{"type":"v1","addr":"172.21.15.183:6837","nonce":1898341911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6837/3094946502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3094946502},{"type":"v1","addr":"172.21.15.196:6837","nonce":3094946502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6839/83843556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":83843556},{"type":"v1","addr":"172.21.15.196:6839","nonce":83843556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24488,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/1763841130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1763841130},{"type":"v1","addr":"172.21.15.183:6839","nonce":1763841130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-23T10:24:03.305 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 13 2024-04-23T10:24:03.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.307+0000 7f0965623640 1 -- 172.21.15.97:0/566280758 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f09480a3bb0 msgr2=0x7f09480a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:03.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.307+0000 7f0965623640 1 --2- 172.21.15.97:0/566280758 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f09480a3bb0 0x7f09480a6070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f095000bbe0 tx=0x7f095000b390 comp rx=0 tx=0).stop 2024-04-23T10:24:03.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.307+0000 7f0965623640 1 -- 172.21.15.97:0/566280758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09601053c0 msgr2=0x7f09601986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:03.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.307+0000 7f0965623640 1 --2- 172.21.15.97:0/566280758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09601053c0 0x7f09601986e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f095400e9f0 tx=0x7f095400eeb0 comp rx=0 tx=0).stop 2024-04-23T10:24:03.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.307+0000 7f0965623640 1 -- 172.21.15.97:0/566280758 shutdown_connections 2024-04-23T10:24:03.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.307+0000 7f0965623640 1 --2- 172.21.15.97:0/566280758 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f09480a3bb0 0x7f09480a6070 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:03.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.307+0000 7f0965623640 1 --2- 172.21.15.97:0/566280758 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0960107cf0 0x7f096019f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:03.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.307+0000 7f0965623640 1 --2- 172.21.15.97:0/566280758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09601053c0 0x7f09601986e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:03.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.307+0000 7f0965623640 1 --2- 172.21.15.97:0/566280758 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0960102a90 0x7f09601981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:03.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.307+0000 7f0965623640 1 -- 172.21.15.97:0/566280758 >> 172.21.15.97:0/566280758 conn(0x7f09600fc440 msgr2=0x7f0960103330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:03.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.307+0000 7f0965623640 1 -- 172.21.15.97:0/566280758 shutdown_connections 2024-04-23T10:24:03.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.308+0000 7f0965623640 1 -- 172.21.15.97:0/566280758 wait complete. 2024-04-23T10:24:03.318 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T10:24:03.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.419+0000 7f82e2cfa640 1 -- 172.21.15.97:0/2193145921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82dc107d10 msgr2=0x7f82dc10a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:03.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.419+0000 7f82e2cfa640 1 --2- 172.21.15.97:0/2193145921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82dc107d10 0x7f82dc10a100 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f82d4009ca0 tx=0x7f82d402a8c0 comp rx=0 tx=0).stop 2024-04-23T10:24:03.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.419+0000 7f82e2cfa640 1 -- 172.21.15.97:0/2193145921 shutdown_connections 2024-04-23T10:24:03.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.419+0000 7f82e2cfa640 1 --2- 172.21.15.97:0/2193145921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82dc107d10 0x7f82dc10a100 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:03.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.419+0000 7f82e2cfa640 1 --2- 172.21.15.97:0/2193145921 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82dc1053e0 0x7f82dc1077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:03.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.419+0000 7f82e2cfa640 1 --2- 172.21.15.97:0/2193145921 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82dc102ab0 0x7f82dc104ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:03.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.419+0000 7f82e2cfa640 1 -- 172.21.15.97:0/2193145921 >> 172.21.15.97:0/2193145921 conn(0x7f82dc0fc400 msgr2=0x7f82dc0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:03.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.419+0000 7f82e2cfa640 1 -- 172.21.15.97:0/2193145921 shutdown_connections 2024-04-23T10:24:03.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.419+0000 7f82e2cfa640 1 -- 172.21.15.97:0/2193145921 wait complete. 2024-04-23T10:24:03.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.420+0000 7f82e2cfa640 1 Processor -- start 2024-04-23T10:24:03.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.420+0000 7f82e2cfa640 1 -- start start 2024-04-23T10:24:03.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.420+0000 7f82e2cfa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82dc102ab0 0x7f82dc1981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:03.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.420+0000 7f82e2cfa640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82dc1053e0 0x7f82dc1986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:03.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.420+0000 7f82e2cfa640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82dc107d10 0x7f82dc19f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:03.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.420+0000 7f82e2cfa640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f82dc10daf0 con 0x7f82dc102ab0 2024-04-23T10:24:03.421 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.420+0000 7f82e2cfa640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f82dc10d970 con 0x7f82dc1053e0 2024-04-23T10:24:03.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.420+0000 7f82e2cfa640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f82dc10dc70 con 0x7f82dc107d10 2024-04-23T10:24:03.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.420+0000 7f82e0a96640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82dc102ab0 0x7f82dc1981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:03.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.420+0000 7f82e0a96640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82dc102ab0 0x7f82dc1981a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44736/0 (socket says 172.21.15.97:44736) 2024-04-23T10:24:03.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.420+0000 7f82e0a96640 1 -- 172.21.15.97:0/3959029688 learned_addr learned my addr 172.21.15.97:0/3959029688 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:24:03.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.420+0000 7f82dbfff640 1 --2- 172.21.15.97:0/3959029688 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82dc1053e0 0x7f82dc1986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:03.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.420+0000 7f82e1297640 1 --2- 172.21.15.97:0/3959029688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82dc107d10 0x7f82dc19f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:03.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.420+0000 7f82e0a96640 1 -- 172.21.15.97:0/3959029688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82dc107d10 msgr2=0x7f82dc19f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:03.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.420+0000 7f82e0a96640 1 --2- 172.21.15.97:0/3959029688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82dc107d10 0x7f82dc19f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:03.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.420+0000 7f82e0a96640 1 -- 172.21.15.97:0/3959029688 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82dc1053e0 msgr2=0x7f82dc1986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:03.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.420+0000 7f82e0a96640 1 --2- 172.21.15.97:0/3959029688 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82dc1053e0 0x7f82dc1986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:03.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.420+0000 7f82e0a96640 1 -- 172.21.15.97:0/3959029688 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82dc19fdc0 con 0x7f82dc102ab0 2024-04-23T10:24:03.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.420+0000 7f82dbfff640 1 --2- 172.21.15.97:0/3959029688 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82dc1053e0 0x7f82dc1986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T10:24:03.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.421+0000 7f82e0a96640 1 --2- 172.21.15.97:0/3959029688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82dc102ab0 0x7f82dc1981a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f82d000e9f0 tx=0x7f82d000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:03.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.421+0000 7f82d9ffb640 1 -- 172.21.15.97:0/3959029688 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d000cd80 con 0x7f82dc102ab0 2024-04-23T10:24:03.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.421+0000 7f82e2cfa640 1 -- 172.21.15.97:0/3959029688 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82dc1a00b0 con 0x7f82dc102ab0 2024-04-23T10:24:03.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.421+0000 7f82d9ffb640 1 -- 172.21.15.97:0/3959029688 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82d00054d0 con 0x7f82dc102ab0 2024-04-23T10:24:03.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.421+0000 7f82d9ffb640 1 -- 172.21.15.97:0/3959029688 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d0010690 con 0x7f82dc102ab0 2024-04-23T10:24:03.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.421+0000 7f82e2cfa640 1 -- 172.21.15.97:0/3959029688 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82dc1a0600 con 0x7f82dc102ab0 2024-04-23T10:24:03.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.422+0000 7f82e2cfa640 1 -- 172.21.15.97:0/3959029688 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82dc0fdef0 con 0x7f82dc102ab0 2024-04-23T10:24:03.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.424+0000 7f82d9ffb640 1 -- 172.21.15.97:0/3959029688 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f82d0004e90 con 0x7f82dc102ab0 2024-04-23T10:24:03.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.424+0000 7f82d9ffb640 1 --2- 172.21.15.97:0/3959029688 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f82cc0a3bb0 0x7f82cc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:03.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.424+0000 7f82dbfff640 1 --2- 172.21.15.97:0/3959029688 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f82cc0a3bb0 0x7f82cc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:03.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.424+0000 7f82d9ffb640 1 -- 172.21.15.97:0/3959029688 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f82d00d0350 con 0x7f82dc102ab0 2024-04-23T10:24:03.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.425+0000 7f82dbfff640 1 --2- 172.21.15.97:0/3959029688 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f82cc0a3bb0 0x7f82cc0a6070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f82dc1996b0 tx=0x7f82c8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:03.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.429+0000 7f82d9ffb640 1 -- 172.21.15.97:0/3959029688 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f82d009d1d0 con 0x7f82dc102ab0 2024-04-23T10:24:03.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.612+0000 7f82e2cfa640 1 -- 172.21.15.97:0/3959029688 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f82dc062b50 con 0x7f82dc102ab0 2024-04-23T10:24:03.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.613+0000 7f82d9ffb640 1 -- 172.21.15.97:0/3959029688 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18457 (secure 0 0 0) 0x7f82d009cad0 con 0x7f82dc102ab0 2024-04-23T10:24:03.612 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:24:03.613 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":82,"fsid":"9b22af88-0159-11ef-bc93-c7b262605968","created":"2024-04-23T10:16:05.983305+0000","modified":"2024-04-23T10:23:14.115959+0000","last_up_change":"2024-04-23T10:23:13.114761+0000","last_in_change":"2024-04-23T10:22:52.804880+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T10:19:25.559138+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"58cb995d-de35-4522-ac81-4982401b260d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6803","nonce":3031221802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6805","nonce":3031221802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6809","nonce":3031221802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6807","nonce":3031221802}]},"public_addr":"172.21.15.97:6803/3031221802","cluster_addr":"172.21.15.97:6805/3031221802","heartbeat_back_addr":"172.21.15.97:6809/3031221802","heartbeat_front_addr":"172.21.15.97:6807/3031221802","state":["exists","up"]},{"osd":1,"uuid":"b47bd8ee-1f21-4834-acf5-bb40f97ab3ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6801","nonce":1108870440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6803","nonce":1108870440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6807","nonce":1108870440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6805","nonce":1108870440}]},"public_addr":"172.21.15.183:6801/1108870440","cluster_addr":"172.21.15.183:6803/1108870440","heartbeat_back_addr":"172.21.15.183:6807/1108870440","heartbeat_front_addr":"172.21.15.183:6805/1108870440","state":["exists","up"]},{"osd":2,"uuid":"17c4eec3-6c47-4334-9b63-1d5ad9ec5988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6801","nonce":2877039731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6803","nonce":2877039731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6807","nonce":2877039731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6805","nonce":2877039731}]},"public_addr":"172.21.15.196:6801/2877039731","cluster_addr":"172.21.15.196:6803/2877039731","heartbeat_back_addr":"172.21.15.196:6807/2877039731","heartbeat_front_addr":"172.21.15.196:6805/2877039731","state":["exists","up"]},{"osd":3,"uuid":"3f731c38-2e45-4ca1-8dce-efc4a1b2166b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6811","nonce":4276311194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6813","nonce":4276311194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6817","nonce":4276311194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6815","nonce":4276311194}]},"public_addr":"172.21.15.97:6811/4276311194","cluster_addr":"172.21.15.97:6813/4276311194","heartbeat_back_addr":"172.21.15.97:6817/4276311194","heartbeat_front_addr":"172.21.15.97:6815/4276311194","state":["exists","up"]},{"osd":4,"uuid":"fdcafb14-9b82-4134-bd5b-79c0516f6cbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6809","nonce":63913916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6811","nonce":63913916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6815","nonce":63913916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6813","nonce":63913916}]},"public_addr":"172.21.15.183:6809/63913916","cluster_addr":"172.21.15.183:6811/63913916","heartbeat_back_addr":"172.21.15.183:6815/63913916","heartbeat_front_addr":"172.21.15.183:6813/63913916","state":["exists","up"]},{"osd":5,"uuid":"cdcc18d8-da3c-4012-bf12-876bb39e66eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6809","nonce":2804446896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6811","nonce":2804446896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6815","nonce":2804446896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6813","nonce":2804446896}]},"public_addr":"172.21.15.196:6809/2804446896","cluster_addr":"172.21.15.196:6811/2804446896","heartbeat_back_addr":"172.21.15.196:6815/2804446896","heartbeat_front_addr":"172.21.15.196:6813/2804446896","state":["exists","up"]},{"osd":6,"uuid":"071abb5c-2edf-4832-b8aa-3b80d20096fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6819","nonce":2080916705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6821","nonce":2080916705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6825","nonce":2080916705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6823","nonce":2080916705}]},"public_addr":"172.21.15.97:6819/2080916705","cluster_addr":"172.21.15.97:6821/2080916705","heartbeat_back_addr":"172.21.15.97:6825/2080916705","heartbeat_front_addr":"172.21.15.97:6823/2080916705","state":["exists","up"]},{"osd":7,"uuid":"9d54760b-66df-486b-8abb-017f3b64b4d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6817","nonce":1865873033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6819","nonce":1865873033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6823","nonce":1865873033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6821","nonce":1865873033}]},"public_addr":"172.21.15.183:6817/1865873033","cluster_addr":"172.21.15.183:6819/1865873033","heartbeat_back_addr":"172.21.15.183:6823/1865873033","heartbeat_front_addr":"172.21.15.183:6821/1865873033","state":["exists","up"]},{"osd":8,"uuid":"5cd3100a-bd6a-4be5-b5e9-0d0afc325193","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6817","nonce":4063932139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6819","nonce":4063932139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6823","nonce":4063932139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6821","nonce":4063932139}]},"public_addr":"172.21.15.196:6817/4063932139","cluster_addr":"172.21.15.196:6819/4063932139","heartbeat_back_addr":"172.21.15.196:6823/4063932139","heartbeat_front_addr":"172.21.15.196:6821/4063932139","state":["exists","up"]},{"osd":9,"uuid":"b6a5f4a1-4ecc-4f73-a2df-07551d9298ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6827","nonce":1887432474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6829","nonce":1887432474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6833","nonce":1887432474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6831","nonce":1887432474}]},"public_addr":"172.21.15.97:6827/1887432474","cluster_addr":"172.21.15.97:6829/1887432474","heartbeat_back_addr":"172.21.15.97:6833/1887432474","heartbeat_front_addr":"172.21.15.97:6831/1887432474","state":["exists","up"]},{"osd":10,"uuid":"86f59816-1ea6-4ae9-8796-cf5fc4831064","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6825","nonce":799434647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6827","nonce":799434647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6831","nonce":799434647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6829","nonce":799434647}]},"public_addr":"172.21.15.183:6825/799434647","cluster_addr":"172.21.15.183:6827/799434647","heartbeat_back_addr":"172.21.15.183:6831/799434647","heartbeat_front_addr":"172.21.15.183:6829/799434647","state":["exists","up"]},{"osd":11,"uuid":"95d64a89-a9f8-4c7a-8c0d-da7fb26cfff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6825","nonce":1873969467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6827","nonce":1873969467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6831","nonce":1873969467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6829","nonce":1873969467}]},"public_addr":"172.21.15.196:6825/1873969467","cluster_addr":"172.21.15.196:6827/1873969467","heartbeat_back_addr":"172.21.15.196:6831/1873969467","heartbeat_front_addr":"172.21.15.196:6829/1873969467","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:18:33.721156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:18:57.586709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:19:22.069458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:19:46.451597+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:20:11.561019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:20:36.298672+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:21:01.661804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:21:27.324128+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:21:52.898197+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:22:19.614326+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:22:45.154917+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:23:11.222612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/3866016040":"2024-04-24T10:16:59.468237+0000","172.21.15.97:6801/3297166858":"2024-04-24T10:16:59.468237+0000","172.21.15.97:0/3804743548":"2024-04-24T10:16:59.468237+0000","172.21.15.97:6800/3706463017":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/3715079438":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/786806834":"2024-04-24T10:16:59.468237+0000","172.21.15.97:6801/3706463017":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/3294175554":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/1449136760":"2024-04-24T10:16:33.128917+0000","172.21.15.97:6800/3297166858":"2024-04-24T10:16:59.468237+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T10:24:03.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.615+0000 7f82e2cfa640 1 -- 172.21.15.97:0/3959029688 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f82cc0a3bb0 msgr2=0x7f82cc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:03.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.615+0000 7f82e2cfa640 1 --2- 172.21.15.97:0/3959029688 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f82cc0a3bb0 0x7f82cc0a6070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f82dc1996b0 tx=0x7f82c8005f90 comp rx=0 tx=0).stop 2024-04-23T10:24:03.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.615+0000 7f82e2cfa640 1 -- 172.21.15.97:0/3959029688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82dc102ab0 msgr2=0x7f82dc1981a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:03.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.615+0000 7f82e2cfa640 1 --2- 172.21.15.97:0/3959029688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82dc102ab0 0x7f82dc1981a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f82d000e9f0 tx=0x7f82d000edf0 comp rx=0 tx=0).stop 2024-04-23T10:24:03.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.615+0000 7f82e2cfa640 1 -- 172.21.15.97:0/3959029688 shutdown_connections 2024-04-23T10:24:03.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.615+0000 7f82e2cfa640 1 --2- 172.21.15.97:0/3959029688 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f82cc0a3bb0 0x7f82cc0a6070 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:03.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.615+0000 7f82e2cfa640 1 --2- 172.21.15.97:0/3959029688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f82dc107d10 0x7f82dc19f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:03.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.615+0000 7f82e2cfa640 1 --2- 172.21.15.97:0/3959029688 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82dc1053e0 0x7f82dc1986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:03.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.615+0000 7f82e2cfa640 1 --2- 172.21.15.97:0/3959029688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82dc102ab0 0x7f82dc1981a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:03.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.615+0000 7f82e2cfa640 1 -- 172.21.15.97:0/3959029688 >> 172.21.15.97:0/3959029688 conn(0x7f82dc0fc400 msgr2=0x7f82dc103410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:03.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.615+0000 7f82e2cfa640 1 -- 172.21.15.97:0/3959029688 shutdown_connections 2024-04-23T10:24:03.615 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.615+0000 7f82e2cfa640 1 -- 172.21.15.97:0/3959029688 wait complete. 2024-04-23T10:24:03.625 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T10:19:25.559138+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T10:24:03.626 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T10:24:03.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.729+0000 7f777a8e1640 1 -- 172.21.15.97:0/1403731083 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7774101fe0 msgr2=0x7f7774102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:03.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.729+0000 7f777a8e1640 1 --2- 172.21.15.97:0/1403731083 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7774101fe0 0x7f7774102460 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7768009900 tx=0x7f776802a410 comp rx=0 tx=0).stop 2024-04-23T10:24:03.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.730+0000 7f777a8e1640 1 -- 172.21.15.97:0/1403731083 shutdown_connections 2024-04-23T10:24:03.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.730+0000 7f777a8e1640 1 --2- 172.21.15.97:0/1403731083 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f77741029a0 0x7f77741092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:03.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.730+0000 7f777a8e1640 1 --2- 172.21.15.97:0/1403731083 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7774101fe0 0x7f7774102460 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:03.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.730+0000 7f777a8e1640 1 --2- 172.21.15.97:0/1403731083 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7774100e90 0x7f77741012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:03.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.730+0000 7f777a8e1640 1 -- 172.21.15.97:0/1403731083 >> 172.21.15.97:0/1403731083 conn(0x7f77740fc400 msgr2=0x7f77740fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:03.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.730+0000 7f777a8e1640 1 -- 172.21.15.97:0/1403731083 shutdown_connections 2024-04-23T10:24:03.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.730+0000 7f777a8e1640 1 -- 172.21.15.97:0/1403731083 wait complete. 2024-04-23T10:24:03.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.730+0000 7f777a8e1640 1 Processor -- start 2024-04-23T10:24:03.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.730+0000 7f777a8e1640 1 -- start start 2024-04-23T10:24:03.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.730+0000 7f777a8e1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7774100e90 0x7f7774069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:03.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.730+0000 7f777a8e1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7774101fe0 0x7f777419cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:03.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.730+0000 7f777a8e1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f77741029a0 0x7f777419f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:03.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.731+0000 7f777a8e1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f777410c9f0 con 0x7f7774100e90 2024-04-23T10:24:03.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.731+0000 7f777a8e1640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f777410c870 con 0x7f77741029a0 2024-04-23T10:24:03.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.731+0000 7f777a8e1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f777410cb70 con 0x7f7774101fe0 2024-04-23T10:24:03.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.731+0000 7f7773fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7774100e90 0x7f7774069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:03.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.731+0000 7f7773fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7774100e90 0x7f7774069c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44752/0 (socket says 172.21.15.97:44752) 2024-04-23T10:24:03.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.731+0000 7f7773fff640 1 -- 172.21.15.97:0/1051821624 learned_addr learned my addr 172.21.15.97:0/1051821624 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:24:03.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.731+0000 7f77737fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7774101fe0 0x7f777419cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:03.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.731+0000 7f7778e7e640 1 --2- 172.21.15.97:0/1051821624 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f77741029a0 0x7f777419f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:03.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.731+0000 7f7773fff640 1 -- 172.21.15.97:0/1051821624 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7774101fe0 msgr2=0x7f777419cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:03.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.731+0000 7f7773fff640 1 --2- 172.21.15.97:0/1051821624 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7774101fe0 0x7f777419cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:03.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.731+0000 7f7773fff640 1 -- 172.21.15.97:0/1051821624 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f77741029a0 msgr2=0x7f777419f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:03.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.731+0000 7f7773fff640 1 --2- 172.21.15.97:0/1051821624 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f77741029a0 0x7f777419f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:03.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.731+0000 7f7773fff640 1 -- 172.21.15.97:0/1051821624 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f777419ff00 con 0x7f7774100e90 2024-04-23T10:24:03.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.731+0000 7f7773fff640 1 --2- 172.21.15.97:0/1051821624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7774100e90 0x7f7774069c50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f776400e9f0 tx=0x7f776400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:03.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.731+0000 7f77717fa640 1 -- 172.21.15.97:0/1051821624 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776400cd80 con 0x7f7774100e90 2024-04-23T10:24:03.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.731+0000 7f777a8e1640 1 -- 172.21.15.97:0/1051821624 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77741a0190 con 0x7f7774100e90 2024-04-23T10:24:03.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.731+0000 7f77717fa640 1 -- 172.21.15.97:0/1051821624 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77640054d0 con 0x7f7774100e90 2024-04-23T10:24:03.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.731+0000 7f777a8e1640 1 -- 172.21.15.97:0/1051821624 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77741a06b0 con 0x7f7774100e90 2024-04-23T10:24:03.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.731+0000 7f77717fa640 1 -- 172.21.15.97:0/1051821624 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7764010690 con 0x7f7774100e90 2024-04-23T10:24:03.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.732+0000 7f777a8e1640 1 -- 172.21.15.97:0/1051821624 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7774068d70 con 0x7f7774100e90 2024-04-23T10:24:03.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.733+0000 7f77717fa640 1 -- 172.21.15.97:0/1051821624 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7764004e90 con 0x7f7774100e90 2024-04-23T10:24:03.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.734+0000 7f77717fa640 1 --2- 172.21.15.97:0/1051821624 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f775c0a3bb0 0x7f775c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:03.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.734+0000 7f77717fa640 1 -- 172.21.15.97:0/1051821624 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f77640d0380 con 0x7f7774100e90 2024-04-23T10:24:03.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.734+0000 7f77737fe640 1 --2- 172.21.15.97:0/1051821624 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f775c0a3bb0 0x7f775c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:03.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.734+0000 7f77737fe640 1 --2- 172.21.15.97:0/1051821624 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f775c0a3bb0 0x7f775c0a6070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7768009230 tx=0x7f77680314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:03.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.739+0000 7f77717fa640 1 -- 172.21.15.97:0/1051821624 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f776409d200 con 0x7f7774100e90 2024-04-23T10:24:03.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.949+0000 7f777a8e1640 1 -- 172.21.15.97:0/1051821624 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7774108310 con 0x7f7774100e90 2024-04-23T10:24:03.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.949+0000 7f77717fa640 1 -- 172.21.15.97:0/1051821624 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f7774108310 con 0x7f7774100e90 2024-04-23T10:24:03.948 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-23T10:24:03.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.951+0000 7f777a8e1640 1 -- 172.21.15.97:0/1051821624 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f775c0a3bb0 msgr2=0x7f775c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:03.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.951+0000 7f777a8e1640 1 --2- 172.21.15.97:0/1051821624 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f775c0a3bb0 0x7f775c0a6070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7768009230 tx=0x7f77680314a0 comp rx=0 tx=0).stop 2024-04-23T10:24:03.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.951+0000 7f777a8e1640 1 -- 172.21.15.97:0/1051821624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7774100e90 msgr2=0x7f7774069c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:03.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.951+0000 7f777a8e1640 1 --2- 172.21.15.97:0/1051821624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7774100e90 0x7f7774069c50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f776400e9f0 tx=0x7f776400edf0 comp rx=0 tx=0).stop 2024-04-23T10:24:03.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.951+0000 7f777a8e1640 1 -- 172.21.15.97:0/1051821624 shutdown_connections 2024-04-23T10:24:03.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.951+0000 7f777a8e1640 1 --2- 172.21.15.97:0/1051821624 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f775c0a3bb0 0x7f775c0a6070 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:03.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.951+0000 7f777a8e1640 1 --2- 172.21.15.97:0/1051821624 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f77741029a0 0x7f777419f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:03.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.951+0000 7f777a8e1640 1 --2- 172.21.15.97:0/1051821624 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7774101fe0 0x7f777419cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:03.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.951+0000 7f777a8e1640 1 --2- 172.21.15.97:0/1051821624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7774100e90 0x7f7774069c50 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:03.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.951+0000 7f777a8e1640 1 -- 172.21.15.97:0/1051821624 >> 172.21.15.97:0/1051821624 conn(0x7f77740fc400 msgr2=0x7f77741054e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:03.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.951+0000 7f777a8e1640 1 -- 172.21.15.97:0/1051821624 shutdown_connections 2024-04-23T10:24:03.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:03.951+0000 7f777a8e1640 1 -- 172.21.15.97:0/1051821624 wait complete. 2024-04-23T10:24:03.961 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-23T10:24:04.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.061+0000 7feed3843640 1 -- 172.21.15.97:0/595325123 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feecc100270 msgr2=0x7feecc100690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:04.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.061+0000 7feed3843640 1 --2- 172.21.15.97:0/595325123 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feecc100270 0x7feecc100690 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7feec8009930 tx=0x7feec802a420 comp rx=0 tx=0).stop 2024-04-23T10:24:04.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.061+0000 7feed3843640 1 -- 172.21.15.97:0/595325123 shutdown_connections 2024-04-23T10:24:04.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.061+0000 7feed3843640 1 --2- 172.21.15.97:0/595325123 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feecc103c00 0x7feecc10a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:04.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.061+0000 7feed3843640 1 --2- 172.21.15.97:0/595325123 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feecc103240 0x7feecc1036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:04.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.061+0000 7feed3843640 1 --2- 172.21.15.97:0/595325123 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feecc100270 0x7feecc100690 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:04.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.061+0000 7feed3843640 1 -- 172.21.15.97:0/595325123 >> 172.21.15.97:0/595325123 conn(0x7feecc0fbbc0 msgr2=0x7feecc0fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:04.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.061+0000 7feed3843640 1 -- 172.21.15.97:0/595325123 shutdown_connections 2024-04-23T10:24:04.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.061+0000 7feed3843640 1 -- 172.21.15.97:0/595325123 wait complete. 2024-04-23T10:24:04.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.061+0000 7feed3843640 1 Processor -- start 2024-04-23T10:24:04.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.061+0000 7feed3843640 1 -- start start 2024-04-23T10:24:04.062 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.062+0000 7feed3843640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feecc100270 0x7feecc198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:04.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.062+0000 7feed3843640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feecc103240 0x7feecc198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:04.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.062+0000 7feed3843640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feecc103c00 0x7feecc19f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:04.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.062+0000 7feed3843640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7feecc10db10 con 0x7feecc103c00 2024-04-23T10:24:04.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.062+0000 7feed3843640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7feecc10d990 con 0x7feecc103240 2024-04-23T10:24:04.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.062+0000 7feed3843640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7feecc10dc90 con 0x7feecc100270 2024-04-23T10:24:04.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.062+0000 7feed1de0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feecc103c00 0x7feecc19f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:04.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.062+0000 7feed1de0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feecc103c00 0x7feecc19f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44768/0 (socket says 172.21.15.97:44768) 2024-04-23T10:24:04.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.062+0000 7feed1de0640 1 -- 172.21.15.97:0/1855430485 learned_addr learned my addr 172.21.15.97:0/1855430485 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:24:04.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.062+0000 7feed15df640 1 --2- 172.21.15.97:0/1855430485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feecc100270 0x7feecc198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:04.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.062+0000 7feed0dde640 1 --2- 172.21.15.97:0/1855430485 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feecc103240 0x7feecc198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:04.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.062+0000 7feed1de0640 1 -- 172.21.15.97:0/1855430485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feecc100270 msgr2=0x7feecc198140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:04.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.062+0000 7feed1de0640 1 --2- 172.21.15.97:0/1855430485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feecc100270 0x7feecc198140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:04.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.062+0000 7feed1de0640 1 -- 172.21.15.97:0/1855430485 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feecc103240 msgr2=0x7feecc198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:04.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.062+0000 7feed1de0640 1 --2- 172.21.15.97:0/1855430485 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feecc103240 0x7feecc198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:04.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.062+0000 7feed1de0640 1 -- 172.21.15.97:0/1855430485 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feecc19fd60 con 0x7feecc103c00 2024-04-23T10:24:04.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.062+0000 7feed1de0640 1 --2- 172.21.15.97:0/1855430485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feecc103c00 0x7feecc19f6f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7feec400cce0 tx=0x7feec400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:04.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.062+0000 7feec27fc640 1 -- 172.21.15.97:0/1855430485 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec400f450 con 0x7feecc103c00 2024-04-23T10:24:04.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.062+0000 7feec27fc640 1 -- 172.21.15.97:0/1855430485 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feec400faa0 con 0x7feecc103c00 2024-04-23T10:24:04.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.062+0000 7feec27fc640 1 -- 172.21.15.97:0/1855430485 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec4018a80 con 0x7feecc103c00 2024-04-23T10:24:04.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.062+0000 7feed3843640 1 -- 172.21.15.97:0/1855430485 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feecc19fff0 con 0x7feecc103c00 2024-04-23T10:24:04.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.062+0000 7feed3843640 1 -- 172.21.15.97:0/1855430485 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feecc1a04c0 con 0x7feecc103c00 2024-04-23T10:24:04.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.063+0000 7feed3843640 1 -- 172.21.15.97:0/1855430485 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feecc109370 con 0x7feecc103c00 2024-04-23T10:24:04.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.064+0000 7feec27fc640 1 -- 172.21.15.97:0/1855430485 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7feec400fc50 con 0x7feecc103c00 2024-04-23T10:24:04.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.065+0000 7feec27fc640 1 --2- 172.21.15.97:0/1855430485 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7feeb80a3c30 0x7feeb80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:04.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.065+0000 7feed15df640 1 --2- 172.21.15.97:0/1855430485 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7feeb80a3c30 0x7feeb80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:04.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.065+0000 7feec27fc640 1 -- 172.21.15.97:0/1855430485 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7feec40d02e0 con 0x7feecc103c00 2024-04-23T10:24:04.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.065+0000 7feed15df640 1 --2- 172.21.15.97:0/1855430485 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7feeb80a3c30 0x7feeb80a60f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7feec8009900 tx=0x7feec80314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:04.068 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.069+0000 7feec27fc640 1 -- 172.21.15.97:0/1855430485 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feec4014030 con 0x7feecc103c00 2024-04-23T10:24:04.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.256+0000 7feed3843640 1 -- 172.21.15.97:0/1855430485 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7feecc062bf0 con 0x7feecc103c00 2024-04-23T10:24:04.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.257+0000 7feec27fc640 1 -- 172.21.15.97:0/1855430485 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) ==== 84+0+65 (secure 0 0 0) 0x7feec409d160 con 0x7feecc103c00 2024-04-23T10:24:04.256 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:24:04.256 INFO:teuthology.orchestra.run.smithi097.stdout:[ 2024-04-23T10:24:04.256 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:24:04.256 INFO:teuthology.orchestra.run.smithi097.stdout: "poolnum": 1, 2024-04-23T10:24:04.256 INFO:teuthology.orchestra.run.smithi097.stdout: "poolname": ".mgr" 2024-04-23T10:24:04.256 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:24:04.256 INFO:teuthology.orchestra.run.smithi097.stdout:] 2024-04-23T10:24:04.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.259+0000 7feed3843640 1 -- 172.21.15.97:0/1855430485 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7feeb80a3c30 msgr2=0x7feeb80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:04.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.259+0000 7feed3843640 1 --2- 172.21.15.97:0/1855430485 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7feeb80a3c30 0x7feeb80a60f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7feec8009900 tx=0x7feec80314a0 comp rx=0 tx=0).stop 2024-04-23T10:24:04.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.259+0000 7feed3843640 1 -- 172.21.15.97:0/1855430485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feecc103c00 msgr2=0x7feecc19f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:04.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.259+0000 7feed3843640 1 --2- 172.21.15.97:0/1855430485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feecc103c00 0x7feecc19f6f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7feec400cce0 tx=0x7feec400a620 comp rx=0 tx=0).stop 2024-04-23T10:24:04.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.259+0000 7feed3843640 1 -- 172.21.15.97:0/1855430485 shutdown_connections 2024-04-23T10:24:04.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.259+0000 7feed3843640 1 --2- 172.21.15.97:0/1855430485 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7feeb80a3c30 0x7feeb80a60f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:04.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.259+0000 7feed3843640 1 --2- 172.21.15.97:0/1855430485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feecc103c00 0x7feecc19f6f0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:04.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.259+0000 7feed3843640 1 --2- 172.21.15.97:0/1855430485 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feecc103240 0x7feecc198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:04.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.259+0000 7feed3843640 1 --2- 172.21.15.97:0/1855430485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feecc100270 0x7feecc198140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:04.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.259+0000 7feed3843640 1 -- 172.21.15.97:0/1855430485 >> 172.21.15.97:0/1855430485 conn(0x7feecc0fbbc0 msgr2=0x7feecc106540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:04.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.259+0000 7feed3843640 1 -- 172.21.15.97:0/1855430485 shutdown_connections 2024-04-23T10:24:04.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.259+0000 7feed3843640 1 -- 172.21.15.97:0/1855430485 wait complete. 2024-04-23T10:24:04.267 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:03 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:24:04.267 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:03 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:24:04.267 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:03 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:24:04.267 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:03 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:24:04.267 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:03 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:24:04.267 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:03 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:24:04.267 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:03 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:24:04.267 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:03 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:24:04.268 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:03 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:24:04.268 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:03 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:24:04.268 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:03 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/566280758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:24:04.268 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:03 smithi183 ceph-mon[32317]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:24:04.268 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:03 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/3959029688' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T10:24:04.270 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:03 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:24:04.270 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:03 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:24:04.270 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:03 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:24:04.270 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:03 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:24:04.270 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:03 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:24:04.270 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:03 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:24:04.270 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:03 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:24:04.270 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:03 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:24:04.270 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:03 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:24:04.271 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:03 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:24:04.271 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:03 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/566280758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:24:04.271 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:03 smithi097 ceph-mon[29197]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:24:04.271 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:03 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/3959029688' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T10:24:04.271 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-23T10:24:04.272 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-23T10:24:04.291 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:03 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:24:04.291 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:03 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:24:04.291 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:03 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:24:04.291 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:03 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:24:04.291 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:03 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:24:04.291 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:03 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:24:04.291 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:03 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:24:04.291 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:03 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:24:04.291 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:03 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:24:04.291 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:03 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:24:04.291 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:03 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/566280758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:24:04.291 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:03 smithi196 ceph-mon[32919]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:24:04.292 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:03 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/3959029688' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T10:24:04.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.375+0000 7f8877843640 1 -- 172.21.15.97:0/1137239386 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8870102040 msgr2=0x7f88701024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:04.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.375+0000 7f8877843640 1 --2- 172.21.15.97:0/1137239386 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8870102040 0x7f88701024c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8860009900 tx=0x7f886002a410 comp rx=0 tx=0).stop 2024-04-23T10:24:04.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.375+0000 7f8877843640 1 -- 172.21.15.97:0/1137239386 shutdown_connections 2024-04-23T10:24:04.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.375+0000 7f8877843640 1 --2- 172.21.15.97:0/1137239386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8870102a00 0x7f8870109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:04.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.375+0000 7f8877843640 1 --2- 172.21.15.97:0/1137239386 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8870102040 0x7f88701024c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:04.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.375+0000 7f8877843640 1 --2- 172.21.15.97:0/1137239386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8870100e60 0x7f8870101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:04.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.376+0000 7f8877843640 1 -- 172.21.15.97:0/1137239386 >> 172.21.15.97:0/1137239386 conn(0x7f88700fc440 msgr2=0x7f88700fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:04.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.376+0000 7f8877843640 1 -- 172.21.15.97:0/1137239386 shutdown_connections 2024-04-23T10:24:04.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.376+0000 7f8877843640 1 -- 172.21.15.97:0/1137239386 wait complete. 2024-04-23T10:24:04.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.376+0000 7f8877843640 1 Processor -- start 2024-04-23T10:24:04.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.376+0000 7f8877843640 1 -- start start 2024-04-23T10:24:04.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.376+0000 7f8877843640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8870100e60 0x7f887006b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:04.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.376+0000 7f8877843640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8870102040 0x7f887019cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:04.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.376+0000 7f8877843640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8870102a00 0x7f887019f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:04.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.376+0000 7f8877843640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f887010ca00 con 0x7f8870100e60 2024-04-23T10:24:04.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.376+0000 7f8877843640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f887010c880 con 0x7f8870102040 2024-04-23T10:24:04.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.376+0000 7f8877843640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f887010cb80 con 0x7f8870102a00 2024-04-23T10:24:04.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.376+0000 7f88755df640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8870100e60 0x7f887006b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:04.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.376+0000 7f88755df640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8870100e60 0x7f887006b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44776/0 (socket says 172.21.15.97:44776) 2024-04-23T10:24:04.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.376+0000 7f88755df640 1 -- 172.21.15.97:0/2491512929 learned_addr learned my addr 172.21.15.97:0/2491512929 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:24:04.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.376+0000 7f8875de0640 1 --2- 172.21.15.97:0/2491512929 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8870102a00 0x7f887019f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:04.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.377+0000 7f8874dde640 1 --2- 172.21.15.97:0/2491512929 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8870102040 0x7f887019cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:04.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.377+0000 7f88755df640 1 -- 172.21.15.97:0/2491512929 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8870102a00 msgr2=0x7f887019f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:04.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.377+0000 7f88755df640 1 --2- 172.21.15.97:0/2491512929 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8870102a00 0x7f887019f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:04.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.377+0000 7f88755df640 1 -- 172.21.15.97:0/2491512929 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8870102040 msgr2=0x7f887019cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:04.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.377+0000 7f88755df640 1 --2- 172.21.15.97:0/2491512929 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8870102040 0x7f887019cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:04.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.377+0000 7f88755df640 1 -- 172.21.15.97:0/2491512929 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f887019fe20 con 0x7f8870100e60 2024-04-23T10:24:04.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.377+0000 7f88755df640 1 --2- 172.21.15.97:0/2491512929 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8870100e60 0x7f887006b120 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f886c009fd0 tx=0x7f886c00c270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:04.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.377+0000 7f88667fc640 1 -- 172.21.15.97:0/2491512929 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f886c011470 con 0x7f8870100e60 2024-04-23T10:24:04.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.377+0000 7f88667fc640 1 -- 172.21.15.97:0/2491512929 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f886c011ac0 con 0x7f8870100e60 2024-04-23T10:24:04.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.377+0000 7f8877843640 1 -- 172.21.15.97:0/2491512929 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88701a0110 con 0x7f8870100e60 2024-04-23T10:24:04.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.377+0000 7f88667fc640 1 -- 172.21.15.97:0/2491512929 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f886c01aab0 con 0x7f8870100e60 2024-04-23T10:24:04.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.377+0000 7f8877843640 1 -- 172.21.15.97:0/2491512929 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88701a0630 con 0x7f8870100e60 2024-04-23T10:24:04.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.378+0000 7f8877843640 1 -- 172.21.15.97:0/2491512929 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8870108270 con 0x7f8870100e60 2024-04-23T10:24:04.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.379+0000 7f88667fc640 1 -- 172.21.15.97:0/2491512929 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f886c01ac50 con 0x7f8870100e60 2024-04-23T10:24:04.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.380+0000 7f88667fc640 1 --2- 172.21.15.97:0/2491512929 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f885c0a3bb0 0x7f885c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:04.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.380+0000 7f8874dde640 1 --2- 172.21.15.97:0/2491512929 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f885c0a3bb0 0x7f885c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:04.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.380+0000 7f88667fc640 1 -- 172.21.15.97:0/2491512929 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f886c0d0440 con 0x7f8870100e60 2024-04-23T10:24:04.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.380+0000 7f8874dde640 1 --2- 172.21.15.97:0/2491512929 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f885c0a3bb0 0x7f885c0a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8860000c00 tx=0x7f88600314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:04.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.384+0000 7f88667fc640 1 -- 172.21.15.97:0/2491512929 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f886c016030 con 0x7f8870100e60 2024-04-23T10:24:04.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:04.565+0000 7f8877843640 1 -- 172.21.15.97:0/2491512929 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f887002d5c0 con 0x7f8870100e60 2024-04-23T10:24:05.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:04 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/1051821624' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T10:24:05.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:04 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/1855430485' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-23T10:24:05.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:04 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:24:05.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:04 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/2491512929' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-23T10:24:05.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:04 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/1051821624' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T10:24:05.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:04 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/1855430485' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-23T10:24:05.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:04 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:24:05.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:04 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/2491512929' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-23T10:24:05.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:04 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/1051821624' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T10:24:05.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:04 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/1855430485' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-23T10:24:05.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:04 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:24:05.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:04 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/2491512929' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-23T10:24:05.545 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:05.546+0000 7f88667fc640 1 -- 172.21.15.97:0/2491512929 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) ==== 138+0+0 (secure 0 0 0) 0x7f886c09d2c0 con 0x7f8870100e60 2024-04-23T10:24:05.546 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' created 2024-04-23T10:24:05.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:05.549+0000 7f8877843640 1 -- 172.21.15.97:0/2491512929 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f885c0a3bb0 msgr2=0x7f885c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:05.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:05.549+0000 7f8877843640 1 --2- 172.21.15.97:0/2491512929 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f885c0a3bb0 0x7f885c0a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8860000c00 tx=0x7f88600314a0 comp rx=0 tx=0).stop 2024-04-23T10:24:05.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:05.549+0000 7f8877843640 1 -- 172.21.15.97:0/2491512929 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8870100e60 msgr2=0x7f887006b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:05.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:05.549+0000 7f8877843640 1 --2- 172.21.15.97:0/2491512929 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8870100e60 0x7f887006b120 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f886c009fd0 tx=0x7f886c00c270 comp rx=0 tx=0).stop 2024-04-23T10:24:05.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:05.549+0000 7f8877843640 1 -- 172.21.15.97:0/2491512929 shutdown_connections 2024-04-23T10:24:05.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:05.549+0000 7f8877843640 1 --2- 172.21.15.97:0/2491512929 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f885c0a3bb0 0x7f885c0a6070 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:05.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:05.549+0000 7f8877843640 1 --2- 172.21.15.97:0/2491512929 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8870102a00 0x7f887019f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:05.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:05.549+0000 7f8877843640 1 --2- 172.21.15.97:0/2491512929 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8870102040 0x7f887019cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:05.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:05.549+0000 7f8877843640 1 --2- 172.21.15.97:0/2491512929 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8870100e60 0x7f887006b120 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:05.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:05.549+0000 7f8877843640 1 -- 172.21.15.97:0/2491512929 >> 172.21.15.97:0/2491512929 conn(0x7f88700fc440 msgr2=0x7f8870105440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:05.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:05.550+0000 7f8877843640 1 -- 172.21.15.97:0/2491512929 shutdown_connections 2024-04-23T10:24:05.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:05.550+0000 7f8877843640 1 -- 172.21.15.97:0/2491512929 wait complete. 2024-04-23T10:24:05.563 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-23T10:24:05.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:05.670+0000 7ff4688c2640 1 -- 172.21.15.97:0/3403342043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff464105360 msgr2=0x7ff464107750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:05.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:05.670+0000 7ff4688c2640 1 --2- 172.21.15.97:0/3403342043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff464105360 0x7ff464107750 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff44c009900 tx=0x7ff44c02a410 comp rx=0 tx=0).stop 2024-04-23T10:24:05.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:05.670+0000 7ff4688c2640 1 -- 172.21.15.97:0/3403342043 shutdown_connections 2024-04-23T10:24:05.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:05.670+0000 7ff4688c2640 1 --2- 172.21.15.97:0/3403342043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff464107c90 0x7ff46410a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:05.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:05.670+0000 7ff4688c2640 1 --2- 172.21.15.97:0/3403342043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff464105360 0x7ff464107750 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:05.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:05.670+0000 7ff4688c2640 1 --2- 172.21.15.97:0/3403342043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff464102a30 0x7ff464104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:05.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:05.670+0000 7ff4688c2640 1 -- 172.21.15.97:0/3403342043 >> 172.21.15.97:0/3403342043 conn(0x7ff4640fc440 msgr2=0x7ff4640fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:05.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:05.670+0000 7ff4688c2640 1 -- 172.21.15.97:0/3403342043 shutdown_connections 2024-04-23T10:24:05.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:05.670+0000 7ff4688c2640 1 -- 172.21.15.97:0/3403342043 wait complete. 2024-04-23T10:24:05.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:05.671+0000 7ff4688c2640 1 Processor -- start 2024-04-23T10:24:05.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:05.671+0000 7ff4688c2640 1 -- start start 2024-04-23T10:24:05.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:05.671+0000 7ff4688c2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff464102a30 0x7ff464198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:05.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:05.671+0000 7ff4688c2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff464105360 0x7ff464198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:05.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:05.671+0000 7ff4688c2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff464107c90 0x7ff46419f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:05.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:05.671+0000 7ff4688c2640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff46410daf0 con 0x7ff464105360 2024-04-23T10:24:05.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:05.671+0000 7ff4688c2640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff46410d970 con 0x7ff464102a30 2024-04-23T10:24:05.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:05.671+0000 7ff4688c2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff46410dc70 con 0x7ff464107c90 2024-04-23T10:24:05.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:05.671+0000 7ff461d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff464105360 0x7ff464198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:05.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:05.671+0000 7ff461d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff464105360 0x7ff464198790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:32782/0 (socket says 172.21.15.97:32782) 2024-04-23T10:24:05.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:05.671+0000 7ff461d9b640 1 -- 172.21.15.97:0/2973973676 learned_addr learned my addr 172.21.15.97:0/2973973676 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:24:05.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:05.671+0000 7ff462d9d640 1 --2- 172.21.15.97:0/2973973676 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff464107c90 0x7ff46419f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:05.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:05.671+0000 7ff46259c640 1 --2- 172.21.15.97:0/2973973676 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff464102a30 0x7ff464198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:05.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:05.672+0000 7ff461d9b640 1 -- 172.21.15.97:0/2973973676 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff464107c90 msgr2=0x7ff46419f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:05.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:05.672+0000 7ff461d9b640 1 --2- 172.21.15.97:0/2973973676 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff464107c90 0x7ff46419f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:05.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:05.672+0000 7ff461d9b640 1 -- 172.21.15.97:0/2973973676 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff464102a30 msgr2=0x7ff464198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:05.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:05.672+0000 7ff461d9b640 1 --2- 172.21.15.97:0/2973973676 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff464102a30 0x7ff464198250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:05.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:05.672+0000 7ff461d9b640 1 -- 172.21.15.97:0/2973973676 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff46419fe70 con 0x7ff464105360 2024-04-23T10:24:05.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:05.672+0000 7ff461d9b640 1 --2- 172.21.15.97:0/2973973676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff464105360 0x7ff464198790 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff44c006010 tx=0x7ff44c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:05.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:05.672+0000 7ff45b7fe640 1 -- 172.21.15.97:0/2973973676 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff44c03f030 con 0x7ff464105360 2024-04-23T10:24:05.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:05.672+0000 7ff4688c2640 1 -- 172.21.15.97:0/2973973676 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4641a0100 con 0x7ff464105360 2024-04-23T10:24:05.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:05.672+0000 7ff45b7fe640 1 -- 172.21.15.97:0/2973973676 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff44c003cb0 con 0x7ff464105360 2024-04-23T10:24:05.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:05.672+0000 7ff45b7fe640 1 -- 172.21.15.97:0/2973973676 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff44c035070 con 0x7ff464105360 2024-04-23T10:24:05.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:05.672+0000 7ff4688c2640 1 -- 172.21.15.97:0/2973973676 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4641a05f0 con 0x7ff464105360 2024-04-23T10:24:05.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:05.673+0000 7ff4688c2640 1 -- 172.21.15.97:0/2973973676 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4640fdf30 con 0x7ff464105360 2024-04-23T10:24:05.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:05.674+0000 7ff45b7fe640 1 -- 172.21.15.97:0/2973973676 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff44c042030 con 0x7ff464105360 2024-04-23T10:24:05.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:05.675+0000 7ff45b7fe640 1 --2- 172.21.15.97:0/2973973676 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7ff4500a3bb0 0x7ff4500a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:05.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:05.675+0000 7ff46259c640 1 --2- 172.21.15.97:0/2973973676 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7ff4500a3bb0 0x7ff4500a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:05.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:05.675+0000 7ff45b7fe640 1 -- 172.21.15.97:0/2973973676 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7497+0+0 (secure 0 0 0) 0x7ff44c0f0a50 con 0x7ff464105360 2024-04-23T10:24:05.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:05.675+0000 7ff46259c640 1 --2- 172.21.15.97:0/2973973676 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7ff4500a3bb0 0x7ff4500a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff45c005fd0 tx=0x7ff45c009680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:05.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:05.679+0000 7ff45b7fe640 1 -- 172.21.15.97:0/2973973676 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff44c0bd110 con 0x7ff464105360 2024-04-23T10:24:05.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:05.856+0000 7ff4688c2640 1 -- 172.21.15.97:0/2973973676 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7ff46402d5c0 con 0x7ff464105360 2024-04-23T10:24:06.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:06 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:24:06.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:06 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:24:06.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:06 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:24:06.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:06 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:24:06.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:06 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:24:06.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:06 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:24:06.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:06 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:24:06.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:06 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:24:06.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:06 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:24:06.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:06 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/2491512929' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-23T10:24:06.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:06 smithi196 ceph-mon[32919]: osdmap e83: 12 total, 12 up, 12 in 2024-04-23T10:24:06.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:06 smithi196 ceph-mon[32919]: pgmap v286: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:24:06.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:06 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/2973973676' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-23T10:24:06.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:06 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:24:06.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:06 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:24:06.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:06 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:24:06.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:06 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:24:06.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:06 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:24:06.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:06 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:24:06.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:06 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:24:06.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:06 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:24:06.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:06 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:24:06.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:06 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/2491512929' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-23T10:24:06.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:06 smithi097 ceph-mon[29197]: osdmap e83: 12 total, 12 up, 12 in 2024-04-23T10:24:06.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:06 smithi097 ceph-mon[29197]: pgmap v286: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:24:06.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:06 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/2973973676' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-23T10:24:06.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:06 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:24:06.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:06 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:24:06.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:06 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:24:06.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:06 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:24:06.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:06 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:24:06.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:06 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:24:06.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:06 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:24:06.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:06 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:24:06.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:06 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:24:06.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:06 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/2491512929' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-23T10:24:06.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:06 smithi183 ceph-mon[32317]: osdmap e83: 12 total, 12 up, 12 in 2024-04-23T10:24:06.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:06 smithi183 ceph-mon[32317]: pgmap v286: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:24:06.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:06 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/2973973676' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-23T10:24:06.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:06.552+0000 7ff45b7fe640 1 -- 172.21.15.97:0/2973973676 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v84) ==== 170+0+0 (secure 0 0 0) 0x7ff44c0c1410 con 0x7ff464105360 2024-04-23T10:24:06.551 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' created 2024-04-23T10:24:06.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:06.556+0000 7ff4688c2640 1 -- 172.21.15.97:0/2973973676 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7ff4500a3bb0 msgr2=0x7ff4500a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:06.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:06.556+0000 7ff4688c2640 1 --2- 172.21.15.97:0/2973973676 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7ff4500a3bb0 0x7ff4500a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff45c005fd0 tx=0x7ff45c009680 comp rx=0 tx=0).stop 2024-04-23T10:24:06.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:06.556+0000 7ff4688c2640 1 -- 172.21.15.97:0/2973973676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff464105360 msgr2=0x7ff464198790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:06.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:06.556+0000 7ff4688c2640 1 --2- 172.21.15.97:0/2973973676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff464105360 0x7ff464198790 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff44c006010 tx=0x7ff44c0042f0 comp rx=0 tx=0).stop 2024-04-23T10:24:06.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:06.556+0000 7ff4688c2640 1 -- 172.21.15.97:0/2973973676 shutdown_connections 2024-04-23T10:24:06.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:06.556+0000 7ff4688c2640 1 --2- 172.21.15.97:0/2973973676 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7ff4500a3bb0 0x7ff4500a6070 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:06.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:06.556+0000 7ff4688c2640 1 --2- 172.21.15.97:0/2973973676 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff464107c90 0x7ff46419f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:06.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:06.556+0000 7ff4688c2640 1 --2- 172.21.15.97:0/2973973676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff464105360 0x7ff464198790 secure :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff44c006010 tx=0x7ff44c0042f0 comp rx=0 tx=0).stop 2024-04-23T10:24:06.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:06.556+0000 7ff4688c2640 1 --2- 172.21.15.97:0/2973973676 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff464102a30 0x7ff464198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:06.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:06.556+0000 7ff4688c2640 1 -- 172.21.15.97:0/2973973676 >> 172.21.15.97:0/2973973676 conn(0x7ff4640fc440 msgr2=0x7ff4641032f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:06.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:06.556+0000 7ff4688c2640 1 -- 172.21.15.97:0/2973973676 shutdown_connections 2024-04-23T10:24:06.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:06.556+0000 7ff4688c2640 1 -- 172.21.15.97:0/2973973676 wait complete. 2024-04-23T10:24:06.577 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-23T10:24:06.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:06.683+0000 7f8c4d55d640 1 -- 172.21.15.97:0/1208910589 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c48101fe0 msgr2=0x7f8c48102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:06.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:06.683+0000 7f8c4d55d640 1 --2- 172.21.15.97:0/1208910589 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c48101fe0 0x7f8c48102460 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c009900 tx=0x7f8c3c02a410 comp rx=0 tx=0).stop 2024-04-23T10:24:06.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:06.683+0000 7f8c4d55d640 1 -- 172.21.15.97:0/1208910589 shutdown_connections 2024-04-23T10:24:06.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:06.683+0000 7f8c4d55d640 1 --2- 172.21.15.97:0/1208910589 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8c481029a0 0x7f8c481092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:06.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:06.683+0000 7f8c4d55d640 1 --2- 172.21.15.97:0/1208910589 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c48101fe0 0x7f8c48102460 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:06.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:06.683+0000 7f8c4d55d640 1 --2- 172.21.15.97:0/1208910589 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c48100e90 0x7f8c481012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:06.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:06.683+0000 7f8c4d55d640 1 -- 172.21.15.97:0/1208910589 >> 172.21.15.97:0/1208910589 conn(0x7f8c480fc400 msgr2=0x7f8c480fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:06.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:06.683+0000 7f8c4d55d640 1 -- 172.21.15.97:0/1208910589 shutdown_connections 2024-04-23T10:24:06.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:06.683+0000 7f8c4d55d640 1 -- 172.21.15.97:0/1208910589 wait complete. 2024-04-23T10:24:06.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:06.684+0000 7f8c4d55d640 1 Processor -- start 2024-04-23T10:24:06.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:06.684+0000 7f8c4d55d640 1 -- start start 2024-04-23T10:24:06.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:06.684+0000 7f8c4d55d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c48100e90 0x7f8c481982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:06.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:06.684+0000 7f8c4d55d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c48101fe0 0x7f8c48198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:06.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:06.684+0000 7f8c4d55d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8c481029a0 0x7f8c4819f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:06.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:06.684+0000 7f8c4d55d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8c4810db70 con 0x7f8c48101fe0 2024-04-23T10:24:06.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:06.684+0000 7f8c4d55d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8c4810d9f0 con 0x7f8c48100e90 2024-04-23T10:24:06.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:06.684+0000 7f8c4d55d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8c4810dcf0 con 0x7f8c481029a0 2024-04-23T10:24:06.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:06.684+0000 7f8c467fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c48101fe0 0x7f8c48198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:06.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:06.684+0000 7f8c467fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c48101fe0 0x7f8c48198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:32804/0 (socket says 172.21.15.97:32804) 2024-04-23T10:24:06.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:06.684+0000 7f8c467fc640 1 -- 172.21.15.97:0/2837055699 learned_addr learned my addr 172.21.15.97:0/2837055699 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:24:06.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:06.685+0000 7f8c477fe640 1 --2- 172.21.15.97:0/2837055699 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8c481029a0 0x7f8c4819f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:06.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:06.685+0000 7f8c46ffd640 1 --2- 172.21.15.97:0/2837055699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c48100e90 0x7f8c481982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:06.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:06.685+0000 7f8c467fc640 1 -- 172.21.15.97:0/2837055699 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8c481029a0 msgr2=0x7f8c4819f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:06.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:06.685+0000 7f8c467fc640 1 --2- 172.21.15.97:0/2837055699 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8c481029a0 0x7f8c4819f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:06.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:06.685+0000 7f8c467fc640 1 -- 172.21.15.97:0/2837055699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c48100e90 msgr2=0x7f8c481982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:06.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:06.685+0000 7f8c467fc640 1 --2- 172.21.15.97:0/2837055699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c48100e90 0x7f8c481982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:06.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:06.685+0000 7f8c467fc640 1 -- 172.21.15.97:0/2837055699 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c4819fee0 con 0x7f8c48101fe0 2024-04-23T10:24:06.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:06.685+0000 7f8c46ffd640 1 --2- 172.21.15.97:0/2837055699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c48100e90 0x7f8c481982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:24:06.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:06.685+0000 7f8c467fc640 1 --2- 172.21.15.97:0/2837055699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c48101fe0 0x7f8c48198800 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c006010 tx=0x7f8c3c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:06.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:06.685+0000 7f8c2ffff640 1 -- 172.21.15.97:0/2837055699 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c03f030 con 0x7f8c48101fe0 2024-04-23T10:24:06.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:06.685+0000 7f8c4d55d640 1 -- 172.21.15.97:0/2837055699 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c481a0170 con 0x7f8c48101fe0 2024-04-23T10:24:06.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:06.685+0000 7f8c2ffff640 1 -- 172.21.15.97:0/2837055699 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c3c003cb0 con 0x7f8c48101fe0 2024-04-23T10:24:06.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:06.685+0000 7f8c2ffff640 1 -- 172.21.15.97:0/2837055699 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c035070 con 0x7f8c48101fe0 2024-04-23T10:24:06.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:06.685+0000 7f8c4d55d640 1 -- 172.21.15.97:0/2837055699 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c481a06e0 con 0x7f8c48101fe0 2024-04-23T10:24:06.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:06.686+0000 7f8c4d55d640 1 -- 172.21.15.97:0/2837055699 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c48108260 con 0x7f8c48101fe0 2024-04-23T10:24:06.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:06.688+0000 7f8c2ffff640 1 -- 172.21.15.97:0/2837055699 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8c3c042030 con 0x7f8c48101fe0 2024-04-23T10:24:06.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:06.688+0000 7f8c2ffff640 1 --2- 172.21.15.97:0/2837055699 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f8c300a3bb0 0x7f8c300a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:06.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:06.689+0000 7f8c46ffd640 1 --2- 172.21.15.97:0/2837055699 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f8c300a3bb0 0x7f8c300a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:06.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:06.689+0000 7f8c2ffff640 1 -- 172.21.15.97:0/2837055699 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7879+0+0 (secure 0 0 0) 0x7f8c3c0f0d20 con 0x7f8c48101fe0 2024-04-23T10:24:06.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:06.689+0000 7f8c46ffd640 1 --2- 172.21.15.97:0/2837055699 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f8c300a3bb0 0x7f8c300a6070 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8c38006fd0 tx=0x7f8c38008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:06.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:06.692+0000 7f8c2ffff640 1 -- 172.21.15.97:0/2837055699 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8c3c0bd810 con 0x7f8c48101fe0 2024-04-23T10:24:06.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:06.908+0000 7f8c4d55d640 1 -- 172.21.15.97:0/2837055699 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f8c48062b50 con 0x7f8c48101fe0 2024-04-23T10:24:07.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:07 smithi196 ceph-mon[32919]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-23T10:24:07.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:07 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/2973973676' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-23T10:24:07.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:07 smithi196 ceph-mon[32919]: osdmap e84: 12 total, 12 up, 12 in 2024-04-23T10:24:07.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:07 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/2837055699' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-23T10:24:07.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:07 smithi196 ceph-mon[32919]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-23T10:24:07.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:07 smithi097 ceph-mon[29197]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-23T10:24:07.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:07 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/2973973676' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-23T10:24:07.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:07 smithi097 ceph-mon[29197]: osdmap e84: 12 total, 12 up, 12 in 2024-04-23T10:24:07.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:07 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/2837055699' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-23T10:24:07.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:07 smithi097 ceph-mon[29197]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-23T10:24:07.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:07 smithi183 ceph-mon[32317]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-23T10:24:07.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:07 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/2973973676' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-23T10:24:07.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:07 smithi183 ceph-mon[32317]: osdmap e84: 12 total, 12 up, 12 in 2024-04-23T10:24:07.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:07 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/2837055699' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-23T10:24:07.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:07 smithi183 ceph-mon[32317]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-23T10:24:07.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.553+0000 7f8c2ffff640 1 -- 172.21.15.97:0/2837055699 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f8c3c0bd110 con 0x7f8c48101fe0 2024-04-23T10:24:07.552 INFO:teuthology.orchestra.run.smithi097.stderr:new fs with metadata pool 2 and data pool 3 2024-04-23T10:24:07.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.555+0000 7f8c4d55d640 1 -- 172.21.15.97:0/2837055699 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f8c300a3bb0 msgr2=0x7f8c300a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:07.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.555+0000 7f8c4d55d640 1 --2- 172.21.15.97:0/2837055699 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f8c300a3bb0 0x7f8c300a6070 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8c38006fd0 tx=0x7f8c38008040 comp rx=0 tx=0).stop 2024-04-23T10:24:07.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.555+0000 7f8c4d55d640 1 -- 172.21.15.97:0/2837055699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c48101fe0 msgr2=0x7f8c48198800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:07.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.555+0000 7f8c4d55d640 1 --2- 172.21.15.97:0/2837055699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c48101fe0 0x7f8c48198800 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c006010 tx=0x7f8c3c0042f0 comp rx=0 tx=0).stop 2024-04-23T10:24:07.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.556+0000 7f8c4d55d640 1 -- 172.21.15.97:0/2837055699 shutdown_connections 2024-04-23T10:24:07.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.556+0000 7f8c4d55d640 1 --2- 172.21.15.97:0/2837055699 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f8c300a3bb0 0x7f8c300a6070 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:07.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.556+0000 7f8c4d55d640 1 --2- 172.21.15.97:0/2837055699 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8c481029a0 0x7f8c4819f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:07.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.556+0000 7f8c4d55d640 1 --2- 172.21.15.97:0/2837055699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c48101fe0 0x7f8c48198800 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:07.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.556+0000 7f8c4d55d640 1 --2- 172.21.15.97:0/2837055699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c48100e90 0x7f8c481982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:07.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.556+0000 7f8c4d55d640 1 -- 172.21.15.97:0/2837055699 >> 172.21.15.97:0/2837055699 conn(0x7f8c480fc400 msgr2=0x7f8c48105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:07.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.556+0000 7f8c4d55d640 1 -- 172.21.15.97:0/2837055699 shutdown_connections 2024-04-23T10:24:07.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.556+0000 7f8c4d55d640 1 -- 172.21.15.97:0/2837055699 wait complete. 2024-04-23T10:24:07.569 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T10:24:07.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.668+0000 7f5135f44640 1 -- 172.21.15.97:0/2834842187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51301053a0 msgr2=0x7f5130107790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:07.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.668+0000 7f5135f44640 1 --2- 172.21.15.97:0/2834842187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51301053a0 0x7f5130107790 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5120009900 tx=0x7f512002a410 comp rx=0 tx=0).stop 2024-04-23T10:24:07.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.668+0000 7f5135f44640 1 -- 172.21.15.97:0/2834842187 shutdown_connections 2024-04-23T10:24:07.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.668+0000 7f5135f44640 1 --2- 172.21.15.97:0/2834842187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5130107cd0 0x7f513010a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:07.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.668+0000 7f5135f44640 1 --2- 172.21.15.97:0/2834842187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51301053a0 0x7f5130107790 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:07.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.668+0000 7f5135f44640 1 --2- 172.21.15.97:0/2834842187 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5130102a70 0x7f5130104e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:07.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.668+0000 7f5135f44640 1 -- 172.21.15.97:0/2834842187 >> 172.21.15.97:0/2834842187 conn(0x7f51300fc400 msgr2=0x7f51300fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:07.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.668+0000 7f5135f44640 1 -- 172.21.15.97:0/2834842187 shutdown_connections 2024-04-23T10:24:07.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.668+0000 7f5135f44640 1 -- 172.21.15.97:0/2834842187 wait complete. 2024-04-23T10:24:07.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.669+0000 7f5135f44640 1 Processor -- start 2024-04-23T10:24:07.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.669+0000 7f5135f44640 1 -- start start 2024-04-23T10:24:07.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.669+0000 7f5135f44640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5130102a70 0x7f51301981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:07.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.669+0000 7f5135f44640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51301053a0 0x7f5130198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:07.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.669+0000 7f5135f44640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5130107cd0 0x7f513019f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:07.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.669+0000 7f5135f44640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f513010db10 con 0x7f5130107cd0 2024-04-23T10:24:07.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.669+0000 7f5135f44640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f513010d990 con 0x7f5130102a70 2024-04-23T10:24:07.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.669+0000 7f5135f44640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f513010dc90 con 0x7f51301053a0 2024-04-23T10:24:07.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.669+0000 7f512ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5130107cd0 0x7f513019f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:07.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.669+0000 7f512ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5130107cd0 0x7f513019f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:32822/0 (socket says 172.21.15.97:32822) 2024-04-23T10:24:07.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.670+0000 7f512ffff640 1 -- 172.21.15.97:0/2271578882 learned_addr learned my addr 172.21.15.97:0/2271578882 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:24:07.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.670+0000 7f512f7fe640 1 --2- 172.21.15.97:0/2271578882 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5130102a70 0x7f51301981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:07.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.670+0000 7f512effd640 1 --2- 172.21.15.97:0/2271578882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51301053a0 0x7f5130198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:07.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.670+0000 7f512ffff640 1 -- 172.21.15.97:0/2271578882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51301053a0 msgr2=0x7f5130198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:07.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.670+0000 7f512ffff640 1 --2- 172.21.15.97:0/2271578882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51301053a0 0x7f5130198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:07.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.670+0000 7f512ffff640 1 -- 172.21.15.97:0/2271578882 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5130102a70 msgr2=0x7f51301981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:07.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.670+0000 7f512ffff640 1 --2- 172.21.15.97:0/2271578882 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5130102a70 0x7f51301981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:07.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.670+0000 7f512ffff640 1 -- 172.21.15.97:0/2271578882 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f513019fdf0 con 0x7f5130107cd0 2024-04-23T10:24:07.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.670+0000 7f512f7fe640 1 --2- 172.21.15.97:0/2271578882 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5130102a70 0x7f51301981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:24:07.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.670+0000 7f512effd640 1 --2- 172.21.15.97:0/2271578882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51301053a0 0x7f5130198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:24:07.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.670+0000 7f512ffff640 1 --2- 172.21.15.97:0/2271578882 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5130107cd0 0x7f513019f780 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f512800b970 tx=0x7f512800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:07.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.670+0000 7f512cff9640 1 -- 172.21.15.97:0/2271578882 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5128019070 con 0x7f5130107cd0 2024-04-23T10:24:07.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.670+0000 7f512cff9640 1 -- 172.21.15.97:0/2271578882 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5128014440 con 0x7f5130107cd0 2024-04-23T10:24:07.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.670+0000 7f5135f44640 1 -- 172.21.15.97:0/2271578882 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51301a00e0 con 0x7f5130107cd0 2024-04-23T10:24:07.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.670+0000 7f512cff9640 1 -- 172.21.15.97:0/2271578882 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51280075d0 con 0x7f5130107cd0 2024-04-23T10:24:07.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.670+0000 7f5135f44640 1 -- 172.21.15.97:0/2271578882 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51301a06d0 con 0x7f5130107cd0 2024-04-23T10:24:07.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.671+0000 7f5135f44640 1 -- 172.21.15.97:0/2271578882 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51300fdef0 con 0x7f5130107cd0 2024-04-23T10:24:07.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.672+0000 7f512cff9640 1 -- 172.21.15.97:0/2271578882 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f512801c070 con 0x7f5130107cd0 2024-04-23T10:24:07.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.673+0000 7f512cff9640 1 --2- 172.21.15.97:0/2271578882 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f51180a3c30 0x7f51180a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:07.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.673+0000 7f512f7fe640 1 --2- 172.21.15.97:0/2271578882 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f51180a3c30 0x7f51180a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:07.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.673+0000 7f512cff9640 1 -- 172.21.15.97:0/2271578882 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f51280cf810 con 0x7f5130107cd0 2024-04-23T10:24:07.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.673+0000 7f512f7fe640 1 --2- 172.21.15.97:0/2271578882 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f51180a3c30 0x7f51180a60f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f512400abe0 tx=0x7f512400a5c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:07.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.677+0000 7f512cff9640 1 -- 172.21.15.97:0/2271578882 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f512809c340 con 0x7f5130107cd0 2024-04-23T10:24:07.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.859+0000 7f5135f44640 1 -- 172.21.15.97:0/2271578882 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f513002d5c0 con 0x7f5130107cd0 2024-04-23T10:24:07.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.860+0000 7f512cff9640 1 -- 172.21.15.97:0/2271578882 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22233 (secure 0 0 0) 0x7f512809bc40 con 0x7f5130107cd0 2024-04-23T10:24:07.860 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:24:07.860 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":85,"fsid":"9b22af88-0159-11ef-bc93-c7b262605968","created":"2024-04-23T10:16:05.983305+0000","modified":"2024-04-23T10:24:06.910185+0000","last_up_change":"2024-04-23T10:23:13.114761+0000","last_in_change":"2024-04-23T10:22:52.804880+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T10:19:25.559138+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T10:24:04.570049+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T10:24:05.861631+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"58cb995d-de35-4522-ac81-4982401b260d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6803","nonce":3031221802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6805","nonce":3031221802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6809","nonce":3031221802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6807","nonce":3031221802}]},"public_addr":"172.21.15.97:6803/3031221802","cluster_addr":"172.21.15.97:6805/3031221802","heartbeat_back_addr":"172.21.15.97:6809/3031221802","heartbeat_front_addr":"172.21.15.97:6807/3031221802","state":["exists","up"]},{"osd":1,"uuid":"b47bd8ee-1f21-4834-acf5-bb40f97ab3ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6801","nonce":1108870440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6803","nonce":1108870440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6807","nonce":1108870440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6805","nonce":1108870440}]},"public_addr":"172.21.15.183:6801/1108870440","cluster_addr":"172.21.15.183:6803/1108870440","heartbeat_back_addr":"172.21.15.183:6807/1108870440","heartbeat_front_addr":"172.21.15.183:6805/1108870440","state":["exists","up"]},{"osd":2,"uuid":"17c4eec3-6c47-4334-9b63-1d5ad9ec5988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6801","nonce":2877039731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6803","nonce":2877039731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6807","nonce":2877039731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6805","nonce":2877039731}]},"public_addr":"172.21.15.196:6801/2877039731","cluster_addr":"172.21.15.196:6803/2877039731","heartbeat_back_addr":"172.21.15.196:6807/2877039731","heartbeat_front_addr":"172.21.15.196:6805/2877039731","state":["exists","up"]},{"osd":3,"uuid":"3f731c38-2e45-4ca1-8dce-efc4a1b2166b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6811","nonce":4276311194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6813","nonce":4276311194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6817","nonce":4276311194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6815","nonce":4276311194}]},"public_addr":"172.21.15.97:6811/4276311194","cluster_addr":"172.21.15.97:6813/4276311194","heartbeat_back_addr":"172.21.15.97:6817/4276311194","heartbeat_front_addr":"172.21.15.97:6815/4276311194","state":["exists","up"]},{"osd":4,"uuid":"fdcafb14-9b82-4134-bd5b-79c0516f6cbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6809","nonce":63913916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6811","nonce":63913916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6815","nonce":63913916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6813","nonce":63913916}]},"public_addr":"172.21.15.183:6809/63913916","cluster_addr":"172.21.15.183:6811/63913916","heartbeat_back_addr":"172.21.15.183:6815/63913916","heartbeat_front_addr":"172.21.15.183:6813/63913916","state":["exists","up"]},{"osd":5,"uuid":"cdcc18d8-da3c-4012-bf12-876bb39e66eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6809","nonce":2804446896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6811","nonce":2804446896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6815","nonce":2804446896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6813","nonce":2804446896}]},"public_addr":"172.21.15.196:6809/2804446896","cluster_addr":"172.21.15.196:6811/2804446896","heartbeat_back_addr":"172.21.15.196:6815/2804446896","heartbeat_front_addr":"172.21.15.196:6813/2804446896","state":["exists","up"]},{"osd":6,"uuid":"071abb5c-2edf-4832-b8aa-3b80d20096fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6819","nonce":2080916705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6821","nonce":2080916705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6825","nonce":2080916705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6823","nonce":2080916705}]},"public_addr":"172.21.15.97:6819/2080916705","cluster_addr":"172.21.15.97:6821/2080916705","heartbeat_back_addr":"172.21.15.97:6825/2080916705","heartbeat_front_addr":"172.21.15.97:6823/2080916705","state":["exists","up"]},{"osd":7,"uuid":"9d54760b-66df-486b-8abb-017f3b64b4d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6817","nonce":1865873033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6819","nonce":1865873033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6823","nonce":1865873033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6821","nonce":1865873033}]},"public_addr":"172.21.15.183:6817/1865873033","cluster_addr":"172.21.15.183:6819/1865873033","heartbeat_back_addr":"172.21.15.183:6823/1865873033","heartbeat_front_addr":"172.21.15.183:6821/1865873033","state":["exists","up"]},{"osd":8,"uuid":"5cd3100a-bd6a-4be5-b5e9-0d0afc325193","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6817","nonce":4063932139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6819","nonce":4063932139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6823","nonce":4063932139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6821","nonce":4063932139}]},"public_addr":"172.21.15.196:6817/4063932139","cluster_addr":"172.21.15.196:6819/4063932139","heartbeat_back_addr":"172.21.15.196:6823/4063932139","heartbeat_front_addr":"172.21.15.196:6821/4063932139","state":["exists","up"]},{"osd":9,"uuid":"b6a5f4a1-4ecc-4f73-a2df-07551d9298ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6827","nonce":1887432474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6829","nonce":1887432474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6833","nonce":1887432474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6831","nonce":1887432474}]},"public_addr":"172.21.15.97:6827/1887432474","cluster_addr":"172.21.15.97:6829/1887432474","heartbeat_back_addr":"172.21.15.97:6833/1887432474","heartbeat_front_addr":"172.21.15.97:6831/1887432474","state":["exists","up"]},{"osd":10,"uuid":"86f59816-1ea6-4ae9-8796-cf5fc4831064","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6825","nonce":799434647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6827","nonce":799434647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6831","nonce":799434647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6829","nonce":799434647}]},"public_addr":"172.21.15.183:6825/799434647","cluster_addr":"172.21.15.183:6827/799434647","heartbeat_back_addr":"172.21.15.183:6831/799434647","heartbeat_front_addr":"172.21.15.183:6829/799434647","state":["exists","up"]},{"osd":11,"uuid":"95d64a89-a9f8-4c7a-8c0d-da7fb26cfff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6825","nonce":1873969467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6827","nonce":1873969467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6831","nonce":1873969467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6829","nonce":1873969467}]},"public_addr":"172.21.15.196:6825/1873969467","cluster_addr":"172.21.15.196:6827/1873969467","heartbeat_back_addr":"172.21.15.196:6831/1873969467","heartbeat_front_addr":"172.21.15.196:6829/1873969467","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:18:33.721156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:18:57.586709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:19:22.069458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:19:46.451597+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:20:11.561019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:20:36.298672+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:21:01.661804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:21:27.324128+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:21:52.898197+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:22:19.614326+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:22:45.154917+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:23:11.222612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/3866016040":"2024-04-24T10:16:59.468237+0000","172.21.15.97:6801/3297166858":"2024-04-24T10:16:59.468237+0000","172.21.15.97:0/3804743548":"2024-04-24T10:16:59.468237+0000","172.21.15.97:6800/3706463017":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/3715079438":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/786806834":"2024-04-24T10:16:59.468237+0000","172.21.15.97:6801/3706463017":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/3294175554":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/1449136760":"2024-04-24T10:16:33.128917+0000","172.21.15.97:6800/3297166858":"2024-04-24T10:16:59.468237+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T10:24:07.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.862+0000 7f5135f44640 1 -- 172.21.15.97:0/2271578882 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f51180a3c30 msgr2=0x7f51180a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:07.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.862+0000 7f5135f44640 1 --2- 172.21.15.97:0/2271578882 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f51180a3c30 0x7f51180a60f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f512400abe0 tx=0x7f512400a5c0 comp rx=0 tx=0).stop 2024-04-23T10:24:07.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.862+0000 7f5135f44640 1 -- 172.21.15.97:0/2271578882 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5130107cd0 msgr2=0x7f513019f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:07.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.862+0000 7f5135f44640 1 --2- 172.21.15.97:0/2271578882 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5130107cd0 0x7f513019f780 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f512800b970 tx=0x7f512800be30 comp rx=0 tx=0).stop 2024-04-23T10:24:07.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.862+0000 7f5135f44640 1 -- 172.21.15.97:0/2271578882 shutdown_connections 2024-04-23T10:24:07.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.862+0000 7f5135f44640 1 --2- 172.21.15.97:0/2271578882 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f51180a3c30 0x7f51180a60f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:07.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.862+0000 7f5135f44640 1 --2- 172.21.15.97:0/2271578882 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5130107cd0 0x7f513019f780 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:07.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.862+0000 7f5135f44640 1 --2- 172.21.15.97:0/2271578882 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51301053a0 0x7f5130198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:07.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.862+0000 7f5135f44640 1 --2- 172.21.15.97:0/2271578882 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5130102a70 0x7f51301981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:07.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.862+0000 7f5135f44640 1 -- 172.21.15.97:0/2271578882 >> 172.21.15.97:0/2271578882 conn(0x7f51300fc400 msgr2=0x7f51301033d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:07.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.862+0000 7f5135f44640 1 -- 172.21.15.97:0/2271578882 shutdown_connections 2024-04-23T10:24:07.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.862+0000 7f5135f44640 1 -- 172.21.15.97:0/2271578882 wait complete. 2024-04-23T10:24:07.873 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T10:24:07.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.972+0000 7f1c25f20640 1 -- 172.21.15.97:0/761313130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c20103b50 msgr2=0x7f1c2010a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:07.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.972+0000 7f1c25f20640 1 --2- 172.21.15.97:0/761313130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c20103b50 0x7f1c2010a3e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1c18009ca0 tx=0x7f1c1802a8c0 comp rx=0 tx=0).stop 2024-04-23T10:24:07.973 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.972+0000 7f1c25f20640 1 -- 172.21.15.97:0/761313130 shutdown_connections 2024-04-23T10:24:07.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.972+0000 7f1c25f20640 1 --2- 172.21.15.97:0/761313130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c20103b50 0x7f1c2010a3e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:07.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.972+0000 7f1c25f20640 1 --2- 172.21.15.97:0/761313130 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c20103190 0x7f1c20103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:07.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.972+0000 7f1c25f20640 1 --2- 172.21.15.97:0/761313130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c20101fa0 0x7f1c201023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:07.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.972+0000 7f1c25f20640 1 -- 172.21.15.97:0/761313130 >> 172.21.15.97:0/761313130 conn(0x7f1c200fd530 msgr2=0x7f1c200ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:07.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.972+0000 7f1c25f20640 1 -- 172.21.15.97:0/761313130 shutdown_connections 2024-04-23T10:24:07.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.972+0000 7f1c25f20640 1 -- 172.21.15.97:0/761313130 wait complete. 2024-04-23T10:24:07.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.972+0000 7f1c25f20640 1 Processor -- start 2024-04-23T10:24:07.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.972+0000 7f1c25f20640 1 -- start start 2024-04-23T10:24:07.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.973+0000 7f1c25f20640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c20101fa0 0x7f1c20198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:07.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.973+0000 7f1c25f20640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c20103190 0x7f1c201986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:07.974 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.973+0000 7f1c25f20640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c20103b50 0x7f1c2019f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:07.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.973+0000 7f1c25f20640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1c2010dae0 con 0x7f1c20103190 2024-04-23T10:24:07.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.973+0000 7f1c25f20640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1c2010d960 con 0x7f1c20101fa0 2024-04-23T10:24:07.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.973+0000 7f1c25f20640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1c2010dc60 con 0x7f1c20103b50 2024-04-23T10:24:07.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.973+0000 7f1c17fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c20103190 0x7f1c201986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:07.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.973+0000 7f1c17fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c20103190 0x7f1c201986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:32838/0 (socket says 172.21.15.97:32838) 2024-04-23T10:24:07.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.973+0000 7f1c17fff640 1 -- 172.21.15.97:0/2901953254 learned_addr learned my addr 172.21.15.97:0/2901953254 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:24:07.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.973+0000 7f1c1f7fe640 1 --2- 172.21.15.97:0/2901953254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c20101fa0 0x7f1c20198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:07.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.973+0000 7f1c1ffff640 1 --2- 172.21.15.97:0/2901953254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c20103b50 0x7f1c2019f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:07.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.973+0000 7f1c17fff640 1 -- 172.21.15.97:0/2901953254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c20103b50 msgr2=0x7f1c2019f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:07.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.973+0000 7f1c17fff640 1 --2- 172.21.15.97:0/2901953254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c20103b50 0x7f1c2019f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:07.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.973+0000 7f1c17fff640 1 -- 172.21.15.97:0/2901953254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c20101fa0 msgr2=0x7f1c20198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:07.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.973+0000 7f1c17fff640 1 --2- 172.21.15.97:0/2901953254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c20101fa0 0x7f1c20198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:07.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.973+0000 7f1c17fff640 1 -- 172.21.15.97:0/2901953254 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c2019fdb0 con 0x7f1c20103190 2024-04-23T10:24:07.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.973+0000 7f1c1f7fe640 1 --2- 172.21.15.97:0/2901953254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c20101fa0 0x7f1c20198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:24:07.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.973+0000 7f1c17fff640 1 --2- 172.21.15.97:0/2901953254 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c20103190 0x7f1c201986d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1c0800d770 tx=0x7f1c0800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:07.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.973+0000 7f1c1d7fa640 1 -- 172.21.15.97:0/2901953254 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c08017070 con 0x7f1c20103190 2024-04-23T10:24:07.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.973+0000 7f1c25f20640 1 -- 172.21.15.97:0/2901953254 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c201a00a0 con 0x7f1c20103190 2024-04-23T10:24:07.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.973+0000 7f1c1d7fa640 1 -- 172.21.15.97:0/2901953254 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c080054d0 con 0x7f1c20103190 2024-04-23T10:24:07.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.973+0000 7f1c1d7fa640 1 -- 172.21.15.97:0/2901953254 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c08004c30 con 0x7f1c20103190 2024-04-23T10:24:07.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.973+0000 7f1c25f20640 1 -- 172.21.15.97:0/2901953254 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c201a05f0 con 0x7f1c20103190 2024-04-23T10:24:07.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.974+0000 7f1c25f20640 1 -- 172.21.15.97:0/2901953254 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c201092c0 con 0x7f1c20103190 2024-04-23T10:24:07.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.975+0000 7f1c1d7fa640 1 -- 172.21.15.97:0/2901953254 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1c08004dd0 con 0x7f1c20103190 2024-04-23T10:24:07.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.976+0000 7f1c1d7fa640 1 --2- 172.21.15.97:0/2901953254 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f1c0c0a3bb0 0x7f1c0c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:07.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.976+0000 7f1c1f7fe640 1 --2- 172.21.15.97:0/2901953254 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f1c0c0a3bb0 0x7f1c0c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:07.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.976+0000 7f1c1d7fa640 1 -- 172.21.15.97:0/2901953254 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f1c080d01c0 con 0x7f1c20103190 2024-04-23T10:24:07.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.976+0000 7f1c1f7fe640 1 --2- 172.21.15.97:0/2901953254 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f1c0c0a3bb0 0x7f1c0c0a6070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1c10005d00 tx=0x7f1c10005c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:07.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:07.980+0000 7f1c1d7fa640 1 -- 172.21.15.97:0/2901953254 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1c0809ccf0 con 0x7f1c20103190 2024-04-23T10:24:08.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.159+0000 7f1c25f20640 1 -- 172.21.15.97:0/2901953254 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1c2002d7c0 con 0x7f1c20103190 2024-04-23T10:24:08.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.161+0000 7f1c1d7fa640 1 -- 172.21.15.97:0/2901953254 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22220 (secure 0 0 0) 0x7f1c0809c5f0 con 0x7f1c20103190 2024-04-23T10:24:08.160 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:24:08.160 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":86,"fsid":"9b22af88-0159-11ef-bc93-c7b262605968","created":"2024-04-23T10:16:05.983305+0000","modified":"2024-04-23T10:24:08.021502+0000","last_up_change":"2024-04-23T10:23:13.114761+0000","last_in_change":"2024-04-23T10:22:52.804880+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T10:19:25.559138+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T10:24:04.570049+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T10:24:05.861631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"58cb995d-de35-4522-ac81-4982401b260d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6803","nonce":3031221802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6805","nonce":3031221802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6809","nonce":3031221802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6807","nonce":3031221802}]},"public_addr":"172.21.15.97:6803/3031221802","cluster_addr":"172.21.15.97:6805/3031221802","heartbeat_back_addr":"172.21.15.97:6809/3031221802","heartbeat_front_addr":"172.21.15.97:6807/3031221802","state":["exists","up"]},{"osd":1,"uuid":"b47bd8ee-1f21-4834-acf5-bb40f97ab3ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6801","nonce":1108870440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6803","nonce":1108870440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6807","nonce":1108870440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6805","nonce":1108870440}]},"public_addr":"172.21.15.183:6801/1108870440","cluster_addr":"172.21.15.183:6803/1108870440","heartbeat_back_addr":"172.21.15.183:6807/1108870440","heartbeat_front_addr":"172.21.15.183:6805/1108870440","state":["exists","up"]},{"osd":2,"uuid":"17c4eec3-6c47-4334-9b63-1d5ad9ec5988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6801","nonce":2877039731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6803","nonce":2877039731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6807","nonce":2877039731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6805","nonce":2877039731}]},"public_addr":"172.21.15.196:6801/2877039731","cluster_addr":"172.21.15.196:6803/2877039731","heartbeat_back_addr":"172.21.15.196:6807/2877039731","heartbeat_front_addr":"172.21.15.196:6805/2877039731","state":["exists","up"]},{"osd":3,"uuid":"3f731c38-2e45-4ca1-8dce-efc4a1b2166b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6811","nonce":4276311194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6813","nonce":4276311194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6817","nonce":4276311194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6815","nonce":4276311194}]},"public_addr":"172.21.15.97:6811/4276311194","cluster_addr":"172.21.15.97:6813/4276311194","heartbeat_back_addr":"172.21.15.97:6817/4276311194","heartbeat_front_addr":"172.21.15.97:6815/4276311194","state":["exists","up"]},{"osd":4,"uuid":"fdcafb14-9b82-4134-bd5b-79c0516f6cbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6809","nonce":63913916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6811","nonce":63913916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6815","nonce":63913916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6813","nonce":63913916}]},"public_addr":"172.21.15.183:6809/63913916","cluster_addr":"172.21.15.183:6811/63913916","heartbeat_back_addr":"172.21.15.183:6815/63913916","heartbeat_front_addr":"172.21.15.183:6813/63913916","state":["exists","up"]},{"osd":5,"uuid":"cdcc18d8-da3c-4012-bf12-876bb39e66eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6809","nonce":2804446896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6811","nonce":2804446896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6815","nonce":2804446896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6813","nonce":2804446896}]},"public_addr":"172.21.15.196:6809/2804446896","cluster_addr":"172.21.15.196:6811/2804446896","heartbeat_back_addr":"172.21.15.196:6815/2804446896","heartbeat_front_addr":"172.21.15.196:6813/2804446896","state":["exists","up"]},{"osd":6,"uuid":"071abb5c-2edf-4832-b8aa-3b80d20096fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6819","nonce":2080916705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6821","nonce":2080916705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6825","nonce":2080916705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6823","nonce":2080916705}]},"public_addr":"172.21.15.97:6819/2080916705","cluster_addr":"172.21.15.97:6821/2080916705","heartbeat_back_addr":"172.21.15.97:6825/2080916705","heartbeat_front_addr":"172.21.15.97:6823/2080916705","state":["exists","up"]},{"osd":7,"uuid":"9d54760b-66df-486b-8abb-017f3b64b4d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6817","nonce":1865873033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6819","nonce":1865873033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6823","nonce":1865873033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6821","nonce":1865873033}]},"public_addr":"172.21.15.183:6817/1865873033","cluster_addr":"172.21.15.183:6819/1865873033","heartbeat_back_addr":"172.21.15.183:6823/1865873033","heartbeat_front_addr":"172.21.15.183:6821/1865873033","state":["exists","up"]},{"osd":8,"uuid":"5cd3100a-bd6a-4be5-b5e9-0d0afc325193","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6817","nonce":4063932139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6819","nonce":4063932139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6823","nonce":4063932139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6821","nonce":4063932139}]},"public_addr":"172.21.15.196:6817/4063932139","cluster_addr":"172.21.15.196:6819/4063932139","heartbeat_back_addr":"172.21.15.196:6823/4063932139","heartbeat_front_addr":"172.21.15.196:6821/4063932139","state":["exists","up"]},{"osd":9,"uuid":"b6a5f4a1-4ecc-4f73-a2df-07551d9298ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6827","nonce":1887432474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6829","nonce":1887432474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6833","nonce":1887432474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6831","nonce":1887432474}]},"public_addr":"172.21.15.97:6827/1887432474","cluster_addr":"172.21.15.97:6829/1887432474","heartbeat_back_addr":"172.21.15.97:6833/1887432474","heartbeat_front_addr":"172.21.15.97:6831/1887432474","state":["exists","up"]},{"osd":10,"uuid":"86f59816-1ea6-4ae9-8796-cf5fc4831064","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6825","nonce":799434647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6827","nonce":799434647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6831","nonce":799434647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6829","nonce":799434647}]},"public_addr":"172.21.15.183:6825/799434647","cluster_addr":"172.21.15.183:6827/799434647","heartbeat_back_addr":"172.21.15.183:6831/799434647","heartbeat_front_addr":"172.21.15.183:6829/799434647","state":["exists","up"]},{"osd":11,"uuid":"95d64a89-a9f8-4c7a-8c0d-da7fb26cfff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6825","nonce":1873969467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6827","nonce":1873969467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6831","nonce":1873969467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6829","nonce":1873969467}]},"public_addr":"172.21.15.196:6825/1873969467","cluster_addr":"172.21.15.196:6827/1873969467","heartbeat_back_addr":"172.21.15.196:6831/1873969467","heartbeat_front_addr":"172.21.15.196:6829/1873969467","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:18:33.721156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:18:57.586709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:19:22.069458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:19:46.451597+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:20:11.561019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:20:36.298672+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:21:01.661804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:21:27.324128+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:21:52.898197+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:22:19.614326+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:22:45.154917+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:23:11.222612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/3866016040":"2024-04-24T10:16:59.468237+0000","172.21.15.97:6801/3297166858":"2024-04-24T10:16:59.468237+0000","172.21.15.97:0/3804743548":"2024-04-24T10:16:59.468237+0000","172.21.15.97:6800/3706463017":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/3715079438":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/786806834":"2024-04-24T10:16:59.468237+0000","172.21.15.97:6801/3706463017":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/3294175554":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/1449136760":"2024-04-24T10:16:33.128917+0000","172.21.15.97:6800/3297166858":"2024-04-24T10:16:59.468237+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T10:24:08.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.163+0000 7f1c25f20640 1 -- 172.21.15.97:0/2901953254 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f1c0c0a3bb0 msgr2=0x7f1c0c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:08.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.163+0000 7f1c25f20640 1 --2- 172.21.15.97:0/2901953254 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f1c0c0a3bb0 0x7f1c0c0a6070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1c10005d00 tx=0x7f1c10005c90 comp rx=0 tx=0).stop 2024-04-23T10:24:08.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.163+0000 7f1c25f20640 1 -- 172.21.15.97:0/2901953254 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c20103190 msgr2=0x7f1c201986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:08.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.163+0000 7f1c25f20640 1 --2- 172.21.15.97:0/2901953254 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c20103190 0x7f1c201986d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1c0800d770 tx=0x7f1c0800dc30 comp rx=0 tx=0).stop 2024-04-23T10:24:08.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.163+0000 7f1c25f20640 1 -- 172.21.15.97:0/2901953254 shutdown_connections 2024-04-23T10:24:08.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.163+0000 7f1c25f20640 1 --2- 172.21.15.97:0/2901953254 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f1c0c0a3bb0 0x7f1c0c0a6070 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:08.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.163+0000 7f1c25f20640 1 --2- 172.21.15.97:0/2901953254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c20103b50 0x7f1c2019f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:08.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.163+0000 7f1c25f20640 1 --2- 172.21.15.97:0/2901953254 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c20103190 0x7f1c201986d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:08.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.163+0000 7f1c25f20640 1 --2- 172.21.15.97:0/2901953254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c20101fa0 0x7f1c20198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:08.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.163+0000 7f1c25f20640 1 -- 172.21.15.97:0/2901953254 >> 172.21.15.97:0/2901953254 conn(0x7f1c200fd530 msgr2=0x7f1c20106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:08.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.163+0000 7f1c25f20640 1 -- 172.21.15.97:0/2901953254 shutdown_connections 2024-04-23T10:24:08.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.163+0000 7f1c25f20640 1 -- 172.21.15.97:0/2901953254 wait complete. 2024-04-23T10:24:08.172 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:08 smithi097 ceph-mon[29197]: osdmap e85: 12 total, 12 up, 12 in 2024-04-23T10:24:08.172 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:08 smithi097 ceph-mon[29197]: pgmap v289: 97 pgs: 11 creating+peering, 1 active+clean, 85 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:24:08.172 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:08 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/2837055699' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-23T10:24:08.172 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:08 smithi097 ceph-mon[29197]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-23T10:24:08.172 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:08 smithi097 ceph-mon[29197]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-23T10:24:08.172 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:08 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/2271578882' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T10:24:08.174 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-23T10:24:08.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.275+0000 7f886159b640 1 -- 172.21.15.97:0/2170479590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f885c102a00 msgr2=0x7f885c109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:08.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.275+0000 7f886159b640 1 --2- 172.21.15.97:0/2170479590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f885c102a00 0x7f885c109320 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8854009ca0 tx=0x7f885402a8c0 comp rx=0 tx=0).stop 2024-04-23T10:24:08.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.275+0000 7f886159b640 1 -- 172.21.15.97:0/2170479590 shutdown_connections 2024-04-23T10:24:08.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.275+0000 7f886159b640 1 --2- 172.21.15.97:0/2170479590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f885c102a00 0x7f885c109320 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:08.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.275+0000 7f886159b640 1 --2- 172.21.15.97:0/2170479590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f885c102040 0x7f885c1024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:08.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.275+0000 7f886159b640 1 --2- 172.21.15.97:0/2170479590 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f885c100e60 0x7f885c101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:08.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.275+0000 7f886159b640 1 -- 172.21.15.97:0/2170479590 >> 172.21.15.97:0/2170479590 conn(0x7f885c0fc420 msgr2=0x7f885c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:08.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.275+0000 7f886159b640 1 -- 172.21.15.97:0/2170479590 shutdown_connections 2024-04-23T10:24:08.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.275+0000 7f886159b640 1 -- 172.21.15.97:0/2170479590 wait complete. 2024-04-23T10:24:08.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.276+0000 7f886159b640 1 Processor -- start 2024-04-23T10:24:08.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.276+0000 7f886159b640 1 -- start start 2024-04-23T10:24:08.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.276+0000 7f886159b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f885c100e60 0x7f885c198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:08.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.276+0000 7f886159b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f885c102040 0x7f885c198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:08.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.276+0000 7f886159b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f885c102a00 0x7f885c19f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:08.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.276+0000 7f886159b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f885c10ca00 con 0x7f885c102a00 2024-04-23T10:24:08.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.276+0000 7f886159b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f885c10c880 con 0x7f885c100e60 2024-04-23T10:24:08.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.276+0000 7f886159b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f885c10cb80 con 0x7f885c102040 2024-04-23T10:24:08.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.276+0000 7f885b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f885c102a00 0x7f885c19f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:08.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.276+0000 7f885b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f885c102a00 0x7f885c19f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:32852/0 (socket says 172.21.15.97:32852) 2024-04-23T10:24:08.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.276+0000 7f885b7fe640 1 -- 172.21.15.97:0/2498292497 learned_addr learned my addr 172.21.15.97:0/2498292497 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:24:08.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.277+0000 7f885affd640 1 --2- 172.21.15.97:0/2498292497 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f885c100e60 0x7f885c198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:08.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.277+0000 7f885a7fc640 1 --2- 172.21.15.97:0/2498292497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f885c102040 0x7f885c198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:08.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.277+0000 7f885b7fe640 1 -- 172.21.15.97:0/2498292497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f885c102040 msgr2=0x7f885c198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:08.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.277+0000 7f885b7fe640 1 --2- 172.21.15.97:0/2498292497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f885c102040 0x7f885c198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:08.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.277+0000 7f885b7fe640 1 -- 172.21.15.97:0/2498292497 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f885c100e60 msgr2=0x7f885c198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:08.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.277+0000 7f885b7fe640 1 --2- 172.21.15.97:0/2498292497 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f885c100e60 0x7f885c198200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:08.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.277+0000 7f885b7fe640 1 -- 172.21.15.97:0/2498292497 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f885c19fdd0 con 0x7f885c102a00 2024-04-23T10:24:08.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.277+0000 7f885affd640 1 --2- 172.21.15.97:0/2498292497 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f885c100e60 0x7f885c198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:24:08.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.277+0000 7f885b7fe640 1 --2- 172.21.15.97:0/2498292497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f885c102a00 0x7f885c19f760 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f88540097b0 tx=0x7f8854031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:08.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.277+0000 7f8853fff640 1 -- 172.21.15.97:0/2498292497 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8854003bf0 con 0x7f885c102a00 2024-04-23T10:24:08.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.277+0000 7f8853fff640 1 -- 172.21.15.97:0/2498292497 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8854003d90 con 0x7f885c102a00 2024-04-23T10:24:08.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.277+0000 7f8853fff640 1 -- 172.21.15.97:0/2498292497 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f885402fc70 con 0x7f885c102a00 2024-04-23T10:24:08.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.277+0000 7f886159b640 1 -- 172.21.15.97:0/2498292497 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f885c1a0060 con 0x7f885c102a00 2024-04-23T10:24:08.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.277+0000 7f886159b640 1 -- 172.21.15.97:0/2498292497 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f885c1a0600 con 0x7f885c102a00 2024-04-23T10:24:08.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.278+0000 7f886159b640 1 -- 172.21.15.97:0/2498292497 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f885c1a0860 con 0x7f885c102a00 2024-04-23T10:24:08.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.279+0000 7f8853fff640 1 -- 172.21.15.97:0/2498292497 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8854004040 con 0x7f885c102a00 2024-04-23T10:24:08.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.280+0000 7f8853fff640 1 --2- 172.21.15.97:0/2498292497 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f88480a3c30 0x7f88480a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:08.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.280+0000 7f885affd640 1 --2- 172.21.15.97:0/2498292497 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f88480a3c30 0x7f88480a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:08.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.280+0000 7f8853fff640 1 -- 172.21.15.97:0/2498292497 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f88540f1940 con 0x7f885c102a00 2024-04-23T10:24:08.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.280+0000 7f885affd640 1 --2- 172.21.15.97:0/2498292497 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f88480a3c30 0x7f88480a60f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f885c101bb0 tx=0x7f8844005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:08.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.284+0000 7f8853fff640 1 -- 172.21.15.97:0/2498292497 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f88540be470 con 0x7f885c102a00 2024-04-23T10:24:08.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:08 smithi196 ceph-mon[32919]: osdmap e85: 12 total, 12 up, 12 in 2024-04-23T10:24:08.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:08 smithi196 ceph-mon[32919]: pgmap v289: 97 pgs: 11 creating+peering, 1 active+clean, 85 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:24:08.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:08 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/2837055699' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-23T10:24:08.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:08 smithi196 ceph-mon[32919]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-23T10:24:08.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:08 smithi196 ceph-mon[32919]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-23T10:24:08.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:08 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/2271578882' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T10:24:08.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:08 smithi183 ceph-mon[32317]: osdmap e85: 12 total, 12 up, 12 in 2024-04-23T10:24:08.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:08 smithi183 ceph-mon[32317]: pgmap v289: 97 pgs: 11 creating+peering, 1 active+clean, 85 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:24:08.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:08 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/2837055699' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-23T10:24:08.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:08 smithi183 ceph-mon[32317]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-23T10:24:08.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:08 smithi183 ceph-mon[32317]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-23T10:24:08.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:08 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/2271578882' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T10:24:08.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.503+0000 7f886159b640 1 -- 172.21.15.97:0/2498292497 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f885c1a0a30 con 0x7f885c102a00 2024-04-23T10:24:08.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.555+0000 7f8853fff640 1 -- 172.21.15.97:0/2498292497 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) ==== 118+0+0 (secure 0 0 0) 0x7f88540bdd70 con 0x7f885c102a00 2024-04-23T10:24:08.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.557+0000 7f886159b640 1 -- 172.21.15.97:0/2498292497 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f88480a3c30 msgr2=0x7f88480a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:08.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.557+0000 7f886159b640 1 --2- 172.21.15.97:0/2498292497 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f88480a3c30 0x7f88480a60f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f885c101bb0 tx=0x7f8844005f90 comp rx=0 tx=0).stop 2024-04-23T10:24:08.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.557+0000 7f886159b640 1 -- 172.21.15.97:0/2498292497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f885c102a00 msgr2=0x7f885c19f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:08.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.557+0000 7f886159b640 1 --2- 172.21.15.97:0/2498292497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f885c102a00 0x7f885c19f760 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f88540097b0 tx=0x7f8854031840 comp rx=0 tx=0).stop 2024-04-23T10:24:08.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.557+0000 7f886159b640 1 -- 172.21.15.97:0/2498292497 shutdown_connections 2024-04-23T10:24:08.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.557+0000 7f886159b640 1 --2- 172.21.15.97:0/2498292497 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f88480a3c30 0x7f88480a60f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:08.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.557+0000 7f886159b640 1 --2- 172.21.15.97:0/2498292497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f885c102a00 0x7f885c19f760 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:08.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.557+0000 7f886159b640 1 --2- 172.21.15.97:0/2498292497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f885c102040 0x7f885c198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:08.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.557+0000 7f886159b640 1 --2- 172.21.15.97:0/2498292497 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f885c100e60 0x7f885c198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:08.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.557+0000 7f886159b640 1 -- 172.21.15.97:0/2498292497 >> 172.21.15.97:0/2498292497 conn(0x7f885c0fc420 msgr2=0x7f885c0fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:08.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.557+0000 7f886159b640 1 -- 172.21.15.97:0/2498292497 shutdown_connections 2024-04-23T10:24:08.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.557+0000 7f886159b640 1 -- 172.21.15.97:0/2498292497 wait complete. 2024-04-23T10:24:08.567 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-04-23T10:24:08.568 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-04-23T10:24:08.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.670+0000 7f111015d640 1 -- 172.21.15.97:0/3964100225 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1108107c70 msgr2=0x7f110810a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:08.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.670+0000 7f111015d640 1 --2- 172.21.15.97:0/3964100225 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1108107c70 0x7f110810a0c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1100009ca0 tx=0x7f110002a8c0 comp rx=0 tx=0).stop 2024-04-23T10:24:08.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.670+0000 7f111015d640 1 -- 172.21.15.97:0/3964100225 shutdown_connections 2024-04-23T10:24:08.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.670+0000 7f111015d640 1 --2- 172.21.15.97:0/3964100225 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1108107c70 0x7f110810a0c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:08.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.670+0000 7f111015d640 1 --2- 172.21.15.97:0/3964100225 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1108105340 0x7f1108107730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:08.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.670+0000 7f111015d640 1 --2- 172.21.15.97:0/3964100225 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1108102a10 0x7f1108104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:08.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.670+0000 7f111015d640 1 -- 172.21.15.97:0/3964100225 >> 172.21.15.97:0/3964100225 conn(0x7f11080fc400 msgr2=0x7f11080fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:08.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.670+0000 7f111015d640 1 -- 172.21.15.97:0/3964100225 shutdown_connections 2024-04-23T10:24:08.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.670+0000 7f111015d640 1 -- 172.21.15.97:0/3964100225 wait complete. 2024-04-23T10:24:08.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.671+0000 7f111015d640 1 Processor -- start 2024-04-23T10:24:08.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.671+0000 7f111015d640 1 -- start start 2024-04-23T10:24:08.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.671+0000 7f111015d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1108102a10 0x7f11081982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:08.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.671+0000 7f111015d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1108105340 0x7f1108198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:08.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.671+0000 7f111015d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1108107c70 0x7f110819f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:08.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.671+0000 7f111015d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f110810dbb0 con 0x7f1108107c70 2024-04-23T10:24:08.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.671+0000 7f111015d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f110810da30 con 0x7f1108105340 2024-04-23T10:24:08.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.671+0000 7f111015d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f110810dd30 con 0x7f1108102a10 2024-04-23T10:24:08.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.671+0000 7f110e6fa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1108107c70 0x7f110819f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:08.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.671+0000 7f110e6fa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1108107c70 0x7f110819f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:32876/0 (socket says 172.21.15.97:32876) 2024-04-23T10:24:08.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.671+0000 7f110e6fa640 1 -- 172.21.15.97:0/2461132658 learned_addr learned my addr 172.21.15.97:0/2461132658 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:24:08.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.671+0000 7f110e6fa640 1 -- 172.21.15.97:0/2461132658 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1108102a10 msgr2=0x7f11081982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:08.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.671+0000 7f110e6fa640 1 --2- 172.21.15.97:0/2461132658 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1108102a10 0x7f11081982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:08.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.671+0000 7f110e6fa640 1 -- 172.21.15.97:0/2461132658 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1108105340 msgr2=0x7f1108198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:08.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.671+0000 7f110d6f8640 1 --2- 172.21.15.97:0/2461132658 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1108105340 0x7f1108198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:08.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.672+0000 7f110e6fa640 1 --2- 172.21.15.97:0/2461132658 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1108105340 0x7f1108198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:08.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.672+0000 7f110e6fa640 1 -- 172.21.15.97:0/2461132658 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f110819ff00 con 0x7f1108107c70 2024-04-23T10:24:08.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.672+0000 7f110d6f8640 1 --2- 172.21.15.97:0/2461132658 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1108105340 0x7f1108198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:24:08.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.672+0000 7f110e6fa640 1 --2- 172.21.15.97:0/2461132658 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1108107c70 0x7f110819f890 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1100031a20 tx=0x7f1100003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:08.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.672+0000 7f10feffd640 1 -- 172.21.15.97:0/2461132658 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1100004240 con 0x7f1108107c70 2024-04-23T10:24:08.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.672+0000 7f10feffd640 1 -- 172.21.15.97:0/2461132658 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1100003e10 con 0x7f1108107c70 2024-04-23T10:24:08.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.672+0000 7f111015d640 1 -- 172.21.15.97:0/2461132658 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11081a0190 con 0x7f1108107c70 2024-04-23T10:24:08.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.672+0000 7f10feffd640 1 -- 172.21.15.97:0/2461132658 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11000305c0 con 0x7f1108107c70 2024-04-23T10:24:08.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.672+0000 7f111015d640 1 -- 172.21.15.97:0/2461132658 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11081a07b0 con 0x7f1108107c70 2024-04-23T10:24:08.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.673+0000 7f111015d640 1 -- 172.21.15.97:0/2461132658 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11080fdef0 con 0x7f1108107c70 2024-04-23T10:24:08.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.674+0000 7f10feffd640 1 -- 172.21.15.97:0/2461132658 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f110002f660 con 0x7f1108107c70 2024-04-23T10:24:08.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.674+0000 7f10feffd640 1 --2- 172.21.15.97:0/2461132658 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f10f40a3c30 0x7f10f40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:08.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.675+0000 7f110def9640 1 --2- 172.21.15.97:0/2461132658 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f10f40a3c30 0x7f10f40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:08.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.675+0000 7f10feffd640 1 -- 172.21.15.97:0/2461132658 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f11000f1d30 con 0x7f1108107c70 2024-04-23T10:24:08.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.675+0000 7f110def9640 1 --2- 172.21.15.97:0/2461132658 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f10f40a3c30 0x7f10f40a60f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1104009710 tx=0x7f1104006c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:08.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.678+0000 7f10feffd640 1 -- 172.21.15.97:0/2461132658 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f11000be860 con 0x7f1108107c70 2024-04-23T10:24:08.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:08.896+0000 7f111015d640 1 -- 172.21.15.97:0/2461132658 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f110802d5c0 con 0x7f1108107c70 2024-04-23T10:24:09.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:09 smithi196 ceph-mon[32919]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-23T10:24:09.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:09 smithi196 ceph-mon[32919]: Cluster is now healthy 2024-04-23T10:24:09.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:09 smithi196 ceph-mon[32919]: osdmap e86: 12 total, 12 up, 12 in 2024-04-23T10:24:09.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:09 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/2901953254' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T10:24:09.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:09 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/2498292497' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-23T10:24:09.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:09 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/2498292497' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-23T10:24:09.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:09 smithi196 ceph-mon[32919]: mds.? [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] up:active 2024-04-23T10:24:09.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:09 smithi196 ceph-mon[32919]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-23T10:24:09.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:09 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/2461132658' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-23T10:24:09.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:09 smithi097 ceph-mon[29197]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-23T10:24:09.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:09 smithi097 ceph-mon[29197]: Cluster is now healthy 2024-04-23T10:24:09.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:09 smithi097 ceph-mon[29197]: osdmap e86: 12 total, 12 up, 12 in 2024-04-23T10:24:09.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:09 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/2901953254' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T10:24:09.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:09 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/2498292497' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-23T10:24:09.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:09 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/2498292497' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-23T10:24:09.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:09 smithi097 ceph-mon[29197]: mds.? [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] up:active 2024-04-23T10:24:09.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:09 smithi097 ceph-mon[29197]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-23T10:24:09.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:09 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/2461132658' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-23T10:24:09.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:09 smithi183 ceph-mon[32317]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-23T10:24:09.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:09 smithi183 ceph-mon[32317]: Cluster is now healthy 2024-04-23T10:24:09.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:09 smithi183 ceph-mon[32317]: osdmap e86: 12 total, 12 up, 12 in 2024-04-23T10:24:09.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:09 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/2901953254' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T10:24:09.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:09 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/2498292497' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-23T10:24:09.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:09 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/2498292497' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-23T10:24:09.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:09 smithi183 ceph-mon[32317]: mds.? [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] up:active 2024-04-23T10:24:09.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:09 smithi183 ceph-mon[32317]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-23T10:24:09.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:09 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/2461132658' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-23T10:24:09.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:09.556+0000 7f10feffd640 1 -- 172.21.15.97:0/2461132658 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v16) ==== 105+0+0 (secure 0 0 0) 0x7f11000be160 con 0x7f1108107c70 2024-04-23T10:24:09.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:09.559+0000 7f111015d640 1 -- 172.21.15.97:0/2461132658 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f10f40a3c30 msgr2=0x7f10f40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:09.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:09.559+0000 7f111015d640 1 --2- 172.21.15.97:0/2461132658 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f10f40a3c30 0x7f10f40a60f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1104009710 tx=0x7f1104006c90 comp rx=0 tx=0).stop 2024-04-23T10:24:09.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:09.559+0000 7f111015d640 1 -- 172.21.15.97:0/2461132658 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1108107c70 msgr2=0x7f110819f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:09.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:09.559+0000 7f111015d640 1 --2- 172.21.15.97:0/2461132658 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1108107c70 0x7f110819f890 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1100031a20 tx=0x7f1100003c20 comp rx=0 tx=0).stop 2024-04-23T10:24:09.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:09.559+0000 7f111015d640 1 -- 172.21.15.97:0/2461132658 shutdown_connections 2024-04-23T10:24:09.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:09.559+0000 7f111015d640 1 --2- 172.21.15.97:0/2461132658 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f10f40a3c30 0x7f10f40a60f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:09.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:09.559+0000 7f111015d640 1 --2- 172.21.15.97:0/2461132658 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1108107c70 0x7f110819f890 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:09.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:09.559+0000 7f111015d640 1 --2- 172.21.15.97:0/2461132658 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1108105340 0x7f1108198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:09.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:09.559+0000 7f111015d640 1 --2- 172.21.15.97:0/2461132658 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1108102a10 0x7f11081982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:09.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:09.559+0000 7f111015d640 1 -- 172.21.15.97:0/2461132658 >> 172.21.15.97:0/2461132658 conn(0x7f11080fc400 msgr2=0x7f1108103430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:09.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:09.559+0000 7f111015d640 1 -- 172.21.15.97:0/2461132658 shutdown_connections 2024-04-23T10:24:09.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:09.559+0000 7f111015d640 1 -- 172.21.15.97:0/2461132658 wait complete. 2024-04-23T10:24:09.571 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-23T10:24:09.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:09.681+0000 7fa07d605640 1 -- 172.21.15.97:0/2891792563 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa078101f90 msgr2=0x7fa0781023b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:09.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:09.681+0000 7fa07d605640 1 --2- 172.21.15.97:0/2891792563 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa078101f90 0x7fa0781023b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa070009ca0 tx=0x7fa07002a8c0 comp rx=0 tx=0).stop 2024-04-23T10:24:09.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:09.681+0000 7fa07d605640 1 -- 172.21.15.97:0/2891792563 shutdown_connections 2024-04-23T10:24:09.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:09.681+0000 7fa07d605640 1 --2- 172.21.15.97:0/2891792563 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa078103b40 0x7fa07810a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:09.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:09.681+0000 7fa07d605640 1 --2- 172.21.15.97:0/2891792563 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa078103180 0x7fa078103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:09.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:09.681+0000 7fa07d605640 1 --2- 172.21.15.97:0/2891792563 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa078101f90 0x7fa0781023b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:09.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:09.681+0000 7fa07d605640 1 -- 172.21.15.97:0/2891792563 >> 172.21.15.97:0/2891792563 conn(0x7fa0780fd500 msgr2=0x7fa0780ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:09.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:09.681+0000 7fa07d605640 1 -- 172.21.15.97:0/2891792563 shutdown_connections 2024-04-23T10:24:09.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:09.681+0000 7fa07d605640 1 -- 172.21.15.97:0/2891792563 wait complete. 2024-04-23T10:24:09.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:09.681+0000 7fa07d605640 1 Processor -- start 2024-04-23T10:24:09.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:09.681+0000 7fa07d605640 1 -- start start 2024-04-23T10:24:09.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:09.681+0000 7fa07d605640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa078101f90 0x7fa0781983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:09.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:09.681+0000 7fa07d605640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa078103180 0x7fa0781988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:09.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:09.682+0000 7fa07d605640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa078103b40 0x7fa07819f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:09.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:09.682+0000 7fa07d605640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa07810db90 con 0x7fa078103b40 2024-04-23T10:24:09.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:09.682+0000 7fa07d605640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa07810da10 con 0x7fa078101f90 2024-04-23T10:24:09.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:09.682+0000 7fa07d605640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa07810dd10 con 0x7fa078103180 2024-04-23T10:24:09.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:09.682+0000 7fa0777fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa078103b40 0x7fa07819f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:09.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:09.682+0000 7fa0777fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa078103b40 0x7fa07819f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:32894/0 (socket says 172.21.15.97:32894) 2024-04-23T10:24:09.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:09.682+0000 7fa0777fe640 1 -- 172.21.15.97:0/4193468833 learned_addr learned my addr 172.21.15.97:0/4193468833 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:24:09.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:09.682+0000 7fa0767fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa078103180 0x7fa0781988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:09.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:09.682+0000 7fa076ffd640 1 --2- 172.21.15.97:0/4193468833 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa078101f90 0x7fa0781983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:09.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:09.682+0000 7fa0777fe640 1 -- 172.21.15.97:0/4193468833 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa078103180 msgr2=0x7fa0781988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:09.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:09.682+0000 7fa0777fe640 1 --2- 172.21.15.97:0/4193468833 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa078103180 0x7fa0781988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:09.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:09.682+0000 7fa0777fe640 1 -- 172.21.15.97:0/4193468833 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa078101f90 msgr2=0x7fa0781983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:09.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:09.682+0000 7fa0777fe640 1 --2- 172.21.15.97:0/4193468833 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa078101f90 0x7fa0781983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:09.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:09.682+0000 7fa0777fe640 1 -- 172.21.15.97:0/4193468833 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa07819ffc0 con 0x7fa078103b40 2024-04-23T10:24:09.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:09.682+0000 7fa0777fe640 1 --2- 172.21.15.97:0/4193468833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa078103b40 0x7fa07819f950 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa068009fd0 tx=0x7fa06800b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:09.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:09.682+0000 7fa05ffff640 1 -- 172.21.15.97:0/4193468833 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa068015020 con 0x7fa078103b40 2024-04-23T10:24:09.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:09.682+0000 7fa07d605640 1 -- 172.21.15.97:0/4193468833 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0781a02b0 con 0x7fa078103b40 2024-04-23T10:24:09.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:09.682+0000 7fa05ffff640 1 -- 172.21.15.97:0/4193468833 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa068011ce0 con 0x7fa078103b40 2024-04-23T10:24:09.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:09.682+0000 7fa05ffff640 1 -- 172.21.15.97:0/4193468833 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa068010dc0 con 0x7fa078103b40 2024-04-23T10:24:09.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:09.682+0000 7fa07d605640 1 -- 172.21.15.97:0/4193468833 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0781a0830 con 0x7fa078103b40 2024-04-23T10:24:09.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:09.683+0000 7fa07d605640 1 -- 172.21.15.97:0/4193468833 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa078109470 con 0x7fa078103b40 2024-04-23T10:24:09.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:09.685+0000 7fa05ffff640 1 -- 172.21.15.97:0/4193468833 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa068018020 con 0x7fa078103b40 2024-04-23T10:24:09.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:09.685+0000 7fa05ffff640 1 --2- 172.21.15.97:0/4193468833 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fa0600a3bb0 0x7fa0600a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:09.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:09.685+0000 7fa076ffd640 1 --2- 172.21.15.97:0/4193468833 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fa0600a3bb0 0x7fa0600a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:09.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:09.685+0000 7fa05ffff640 1 -- 172.21.15.97:0/4193468833 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fa0680cfae0 con 0x7fa078103b40 2024-04-23T10:24:09.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:09.686+0000 7fa076ffd640 1 --2- 172.21.15.97:0/4193468833 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fa0600a3bb0 0x7fa0600a6070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa070000f80 tx=0x7fa070005c30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:09.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:09.690+0000 7fa05ffff640 1 -- 172.21.15.97:0/4193468833 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa06801f020 con 0x7fa078103b40 2024-04-23T10:24:09.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:09.905+0000 7fa07d605640 1 -- 172.21.15.97:0/4193468833 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fa07802d5c0 con 0x7fa078103b40 2024-04-23T10:24:10.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:10 smithi196 ceph-mon[32919]: pgmap v291: 97 pgs: 11 creating+peering, 38 active+clean, 48 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 2 op/s 2024-04-23T10:24:10.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:10 smithi196 ceph-mon[32919]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-23T10:24:10.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:10 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/2461132658' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-23T10:24:10.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:10 smithi196 ceph-mon[32919]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-23T10:24:10.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:10 smithi196 ceph-mon[32919]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-23T10:24:10.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:10 smithi196 ceph-mon[32919]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-23T10:24:10.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:10 smithi196 ceph-mon[32919]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-23T10:24:10.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:10 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/4193468833' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-23T10:24:10.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:10 smithi097 ceph-mon[29197]: pgmap v291: 97 pgs: 11 creating+peering, 38 active+clean, 48 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 2 op/s 2024-04-23T10:24:10.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:10 smithi097 ceph-mon[29197]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-23T10:24:10.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:10 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/2461132658' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-23T10:24:10.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:10 smithi097 ceph-mon[29197]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-23T10:24:10.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:10 smithi097 ceph-mon[29197]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-23T10:24:10.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:10 smithi097 ceph-mon[29197]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-23T10:24:10.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:10 smithi097 ceph-mon[29197]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-23T10:24:10.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:10 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/4193468833' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-23T10:24:10.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:10 smithi183 ceph-mon[32317]: pgmap v291: 97 pgs: 11 creating+peering, 38 active+clean, 48 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 2 op/s 2024-04-23T10:24:10.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:10 smithi183 ceph-mon[32317]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-23T10:24:10.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:10 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/2461132658' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-23T10:24:10.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:10 smithi183 ceph-mon[32317]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-23T10:24:10.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:10 smithi183 ceph-mon[32317]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-23T10:24:10.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:10 smithi183 ceph-mon[32317]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-23T10:24:10.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:10 smithi183 ceph-mon[32317]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-23T10:24:10.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:10 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/4193468833' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-23T10:24:10.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:10.565+0000 7fa05ffff640 1 -- 172.21.15.97:0/4193468833 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v18) ==== 121+0+0 (secure 0 0 0) 0x7fa06809c610 con 0x7fa078103b40 2024-04-23T10:24:10.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:10.568+0000 7fa07d605640 1 -- 172.21.15.97:0/4193468833 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fa0600a3bb0 msgr2=0x7fa0600a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:10.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:10.568+0000 7fa07d605640 1 --2- 172.21.15.97:0/4193468833 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fa0600a3bb0 0x7fa0600a6070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa070000f80 tx=0x7fa070005c30 comp rx=0 tx=0).stop 2024-04-23T10:24:10.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:10.568+0000 7fa07d605640 1 -- 172.21.15.97:0/4193468833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa078103b40 msgr2=0x7fa07819f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:10.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:10.568+0000 7fa07d605640 1 --2- 172.21.15.97:0/4193468833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa078103b40 0x7fa07819f950 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa068009fd0 tx=0x7fa06800b820 comp rx=0 tx=0).stop 2024-04-23T10:24:10.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:10.568+0000 7fa07d605640 1 -- 172.21.15.97:0/4193468833 shutdown_connections 2024-04-23T10:24:10.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:10.568+0000 7fa07d605640 1 --2- 172.21.15.97:0/4193468833 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fa0600a3bb0 0x7fa0600a6070 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:10.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:10.568+0000 7fa07d605640 1 --2- 172.21.15.97:0/4193468833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa078103b40 0x7fa07819f950 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:10.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:10.568+0000 7fa07d605640 1 --2- 172.21.15.97:0/4193468833 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa078103180 0x7fa0781988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:10.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:10.568+0000 7fa07d605640 1 --2- 172.21.15.97:0/4193468833 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa078101f90 0x7fa0781983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:10.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:10.568+0000 7fa07d605640 1 -- 172.21.15.97:0/4193468833 >> 172.21.15.97:0/4193468833 conn(0x7fa0780fd500 msgr2=0x7fa078106640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:10.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:10.568+0000 7fa07d605640 1 -- 172.21.15.97:0/4193468833 shutdown_connections 2024-04-23T10:24:10.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:10.568+0000 7fa07d605640 1 -- 172.21.15.97:0/4193468833 wait complete. 2024-04-23T10:24:10.582 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-23T10:24:10.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:10.687+0000 7f0008e31640 1 -- 172.21.15.97:0/3822453562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0004103190 msgr2=0x7f0004103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:10.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:10.687+0000 7f0008e31640 1 --2- 172.21.15.97:0/3822453562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0004103190 0x7f0004103610 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7effec009900 tx=0x7effec02a410 comp rx=0 tx=0).stop 2024-04-23T10:24:10.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:10.687+0000 7f0008e31640 1 -- 172.21.15.97:0/3822453562 shutdown_connections 2024-04-23T10:24:10.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:10.687+0000 7f0008e31640 1 --2- 172.21.15.97:0/3822453562 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0004103b50 0x7f000410a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:10.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:10.687+0000 7f0008e31640 1 --2- 172.21.15.97:0/3822453562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0004103190 0x7f0004103610 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:10.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:10.687+0000 7f0008e31640 1 --2- 172.21.15.97:0/3822453562 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0004101fa0 0x7f00041023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:10.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:10.687+0000 7f0008e31640 1 -- 172.21.15.97:0/3822453562 >> 172.21.15.97:0/3822453562 conn(0x7f00040fd4f0 msgr2=0x7f00040ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:10.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:10.687+0000 7f0008e31640 1 -- 172.21.15.97:0/3822453562 shutdown_connections 2024-04-23T10:24:10.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:10.687+0000 7f0008e31640 1 -- 172.21.15.97:0/3822453562 wait complete. 2024-04-23T10:24:10.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:10.688+0000 7f0008e31640 1 Processor -- start 2024-04-23T10:24:10.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:10.688+0000 7f0008e31640 1 -- start start 2024-04-23T10:24:10.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:10.688+0000 7f0008e31640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0004101fa0 0x7f0004198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:10.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:10.688+0000 7f0008e31640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0004103190 0x7f00041986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:10.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:10.688+0000 7f0008e31640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0004103b50 0x7f000419f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:10.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:10.688+0000 7f0008e31640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f000410dae0 con 0x7f0004103190 2024-04-23T10:24:10.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:10.688+0000 7f0008e31640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f000410d960 con 0x7f0004103b50 2024-04-23T10:24:10.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:10.688+0000 7f0008e31640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f000410dc60 con 0x7f0004101fa0 2024-04-23T10:24:10.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:10.688+0000 7efff9d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0004103190 0x7f00041986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:10.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:10.688+0000 7efff9d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0004103190 0x7f00041986a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:32916/0 (socket says 172.21.15.97:32916) 2024-04-23T10:24:10.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:10.688+0000 7efff9d9b640 1 -- 172.21.15.97:0/2150329423 learned_addr learned my addr 172.21.15.97:0/2150329423 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:24:10.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:10.688+0000 7f0002d9d640 1 --2- 172.21.15.97:0/2150329423 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0004103b50 0x7f000419f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:10.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:10.688+0000 7f000259c640 1 --2- 172.21.15.97:0/2150329423 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0004101fa0 0x7f0004198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:10.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:10.688+0000 7efff9d9b640 1 -- 172.21.15.97:0/2150329423 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0004101fa0 msgr2=0x7f0004198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:10.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:10.688+0000 7efff9d9b640 1 --2- 172.21.15.97:0/2150329423 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0004101fa0 0x7f0004198160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:10.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:10.688+0000 7efff9d9b640 1 -- 172.21.15.97:0/2150329423 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0004103b50 msgr2=0x7f000419f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:10.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:10.688+0000 7efff9d9b640 1 --2- 172.21.15.97:0/2150329423 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0004103b50 0x7f000419f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:10.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:10.688+0000 7efff9d9b640 1 -- 172.21.15.97:0/2150329423 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f000419fd80 con 0x7f0004103190 2024-04-23T10:24:10.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:10.689+0000 7efff9d9b640 1 --2- 172.21.15.97:0/2150329423 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0004103190 0x7f00041986a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7effec005940 tx=0x7effec031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:10.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:10.689+0000 7efffbfff640 1 -- 172.21.15.97:0/2150329423 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effec03f030 con 0x7f0004103190 2024-04-23T10:24:10.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:10.689+0000 7f0008e31640 1 -- 172.21.15.97:0/2150329423 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00041a0010 con 0x7f0004103190 2024-04-23T10:24:10.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:10.689+0000 7efffbfff640 1 -- 172.21.15.97:0/2150329423 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effec003aa0 con 0x7f0004103190 2024-04-23T10:24:10.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:10.689+0000 7efffbfff640 1 -- 172.21.15.97:0/2150329423 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effec035070 con 0x7f0004103190 2024-04-23T10:24:10.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:10.689+0000 7f0008e31640 1 -- 172.21.15.97:0/2150329423 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00041a0500 con 0x7f0004103190 2024-04-23T10:24:10.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:10.690+0000 7f0008e31640 1 -- 172.21.15.97:0/2150329423 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0004109360 con 0x7f0004103190 2024-04-23T10:24:10.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:10.691+0000 7efffbfff640 1 -- 172.21.15.97:0/2150329423 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7effec042030 con 0x7f0004103190 2024-04-23T10:24:10.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:10.691+0000 7efffbfff640 1 --2- 172.21.15.97:0/2150329423 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7efff00a3bb0 0x7efff00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:10.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:10.691+0000 7f000259c640 1 --2- 172.21.15.97:0/2150329423 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7efff00a3bb0 0x7efff00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:10.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:10.691+0000 7efffbfff640 1 -- 172.21.15.97:0/2150329423 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7effec0f08f0 con 0x7f0004103190 2024-04-23T10:24:10.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:10.692+0000 7f000259c640 1 --2- 172.21.15.97:0/2150329423 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7efff00a3bb0 0x7efff00a6070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0004102ff0 tx=0x7efff4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:10.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:10.696+0000 7efffbfff640 1 -- 172.21.15.97:0/2150329423 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7effec0bd420 con 0x7f0004103190 2024-04-23T10:24:10.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:10.913+0000 7f0008e31640 1 -- 172.21.15.97:0/2150329423 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f000402d810 con 0x7f0004103190 2024-04-23T10:24:10.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:10.945+0000 7efffbfff640 1 -- 172.21.15.97:0/2150329423 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v20) ==== 115+0+0 (secure 0 0 0) 0x7effec0bcd20 con 0x7f0004103190 2024-04-23T10:24:10.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:10.947+0000 7f0008e31640 1 -- 172.21.15.97:0/2150329423 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7efff00a3bb0 msgr2=0x7efff00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:10.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:10.947+0000 7f0008e31640 1 --2- 172.21.15.97:0/2150329423 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7efff00a3bb0 0x7efff00a6070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0004102ff0 tx=0x7efff4005f90 comp rx=0 tx=0).stop 2024-04-23T10:24:10.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:10.947+0000 7f0008e31640 1 -- 172.21.15.97:0/2150329423 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0004103190 msgr2=0x7f00041986a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:10.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:10.947+0000 7f0008e31640 1 --2- 172.21.15.97:0/2150329423 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0004103190 0x7f00041986a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7effec005940 tx=0x7effec031a50 comp rx=0 tx=0).stop 2024-04-23T10:24:10.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:10.947+0000 7f0008e31640 1 -- 172.21.15.97:0/2150329423 shutdown_connections 2024-04-23T10:24:10.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:10.947+0000 7f0008e31640 1 --2- 172.21.15.97:0/2150329423 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7efff00a3bb0 0x7efff00a6070 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:10.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:10.947+0000 7f0008e31640 1 --2- 172.21.15.97:0/2150329423 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0004103b50 0x7f000419f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:10.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:10.947+0000 7f0008e31640 1 --2- 172.21.15.97:0/2150329423 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0004103190 0x7f00041986a0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:10.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:10.947+0000 7f0008e31640 1 --2- 172.21.15.97:0/2150329423 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0004101fa0 0x7f0004198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:10.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:10.947+0000 7f0008e31640 1 -- 172.21.15.97:0/2150329423 >> 172.21.15.97:0/2150329423 conn(0x7f00040fd4f0 msgr2=0x7f0004106530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:10.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:10.948+0000 7f0008e31640 1 -- 172.21.15.97:0/2150329423 shutdown_connections 2024-04-23T10:24:10.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:10.948+0000 7f0008e31640 1 -- 172.21.15.97:0/2150329423 wait complete. 2024-04-23T10:24:10.960 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-04-23T10:24:10.960 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-23T10:24:11.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:11.061+0000 7f0296410640 1 -- 172.21.15.97:0/3557432911 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0290102a30 msgr2=0x7f0290104e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:11.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:11.061+0000 7f0296410640 1 --2- 172.21.15.97:0/3557432911 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0290102a30 0x7f0290104e20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0284009900 tx=0x7f028402a410 comp rx=0 tx=0).stop 2024-04-23T10:24:11.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:11.061+0000 7f0296410640 1 -- 172.21.15.97:0/3557432911 shutdown_connections 2024-04-23T10:24:11.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:11.061+0000 7f0296410640 1 --2- 172.21.15.97:0/3557432911 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0290107c90 0x7f029010a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:11.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:11.061+0000 7f0296410640 1 --2- 172.21.15.97:0/3557432911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0290105360 0x7f0290107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:11.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:11.061+0000 7f0296410640 1 --2- 172.21.15.97:0/3557432911 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0290102a30 0x7f0290104e20 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:11.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:11.061+0000 7f0296410640 1 -- 172.21.15.97:0/3557432911 >> 172.21.15.97:0/3557432911 conn(0x7f02900fc420 msgr2=0x7f02900fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:11.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:11.061+0000 7f0296410640 1 -- 172.21.15.97:0/3557432911 shutdown_connections 2024-04-23T10:24:11.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:11.061+0000 7f0296410640 1 -- 172.21.15.97:0/3557432911 wait complete. 2024-04-23T10:24:11.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:11.061+0000 7f0296410640 1 Processor -- start 2024-04-23T10:24:11.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:11.061+0000 7f0296410640 1 -- start start 2024-04-23T10:24:11.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:11.062+0000 7f0296410640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0290102a30 0x7f02901023a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:11.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:11.062+0000 7f0296410640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0290105360 0x7f02901009e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:11.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:11.062+0000 7f0296410640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0290107c90 0x7f0290100f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:11.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:11.062+0000 7f0296410640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f029010daf0 con 0x7f0290102a30 2024-04-23T10:24:11.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:11.062+0000 7f0296410640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f029010d970 con 0x7f0290105360 2024-04-23T10:24:11.063 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:11.062+0000 7f0296410640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f029010dc70 con 0x7f0290107c90 2024-04-23T10:24:11.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:11.062+0000 7f028ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0290102a30 0x7f02901023a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:11.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:11.062+0000 7f028ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0290102a30 0x7f02901023a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:32938/0 (socket says 172.21.15.97:32938) 2024-04-23T10:24:11.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:11.062+0000 7f028ffff640 1 -- 172.21.15.97:0/842155738 learned_addr learned my addr 172.21.15.97:0/842155738 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:24:11.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:11.062+0000 7f028f7fe640 1 --2- 172.21.15.97:0/842155738 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0290105360 0x7f02901009e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:11.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:11.062+0000 7f02949ad640 1 --2- 172.21.15.97:0/842155738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0290107c90 0x7f0290100f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:11.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:11.062+0000 7f028ffff640 1 -- 172.21.15.97:0/842155738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0290107c90 msgr2=0x7f0290100f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:11.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:11.062+0000 7f028ffff640 1 --2- 172.21.15.97:0/842155738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0290107c90 0x7f0290100f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:11.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:11.062+0000 7f028ffff640 1 -- 172.21.15.97:0/842155738 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0290105360 msgr2=0x7f02901009e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:11.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:11.062+0000 7f028ffff640 1 --2- 172.21.15.97:0/842155738 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0290105360 0x7f02901009e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:11.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:11.062+0000 7f028ffff640 1 -- 172.21.15.97:0/842155738 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02901017b0 con 0x7f0290102a30 2024-04-23T10:24:11.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:11.062+0000 7f028ffff640 1 --2- 172.21.15.97:0/842155738 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0290102a30 0x7f02901023a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0284005940 tx=0x7f0284031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:11.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:11.063+0000 7f028d7fa640 1 -- 172.21.15.97:0/842155738 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f028403f030 con 0x7f0290102a30 2024-04-23T10:24:11.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:11.063+0000 7f028d7fa640 1 -- 172.21.15.97:0/842155738 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0284003aa0 con 0x7f0290102a30 2024-04-23T10:24:11.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:11.063+0000 7f0296410640 1 -- 172.21.15.97:0/842155738 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02901a22a0 con 0x7f0290102a30 2024-04-23T10:24:11.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:11.063+0000 7f028d7fa640 1 -- 172.21.15.97:0/842155738 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0284035070 con 0x7f0290102a30 2024-04-23T10:24:11.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:11.063+0000 7f0296410640 1 -- 172.21.15.97:0/842155738 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02901a27f0 con 0x7f0290102a30 2024-04-23T10:24:11.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:11.064+0000 7f0296410640 1 -- 172.21.15.97:0/842155738 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02900fdf10 con 0x7f0290102a30 2024-04-23T10:24:11.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:11.065+0000 7f028d7fa640 1 -- 172.21.15.97:0/842155738 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0284042030 con 0x7f0290102a30 2024-04-23T10:24:11.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:11.065+0000 7f028d7fa640 1 --2- 172.21.15.97:0/842155738 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f02780a3bb0 0x7f02780a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:11.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:11.066+0000 7f028f7fe640 1 --2- 172.21.15.97:0/842155738 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f02780a3bb0 0x7f02780a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:11.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:11.066+0000 7f028d7fa640 1 -- 172.21.15.97:0/842155738 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f02840f0840 con 0x7f0290102a30 2024-04-23T10:24:11.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:11.066+0000 7f028f7fe640 1 --2- 172.21.15.97:0/842155738 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f02780a3bb0 0x7f02780a6070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0290102110 tx=0x7f028000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:11.069 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:11.070+0000 7f028d7fa640 1 -- 172.21.15.97:0/842155738 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f02840bcd20 con 0x7f0290102a30 2024-04-23T10:24:11.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:11.276+0000 7f0296410640 1 -- 172.21.15.97:0/842155738 --> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f0290062390 con 0x7f02780a3bb0 2024-04-23T10:24:11.566 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:24:11 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:24:11.280+0000 7f1e12d63700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T10:24:11.566 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:24:11 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:24:11.280+0000 7f1e12d63700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T10:24:11.566 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:24:11 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:24:11.280+0000 7f1e12d63700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T10:24:11.566 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:24:11 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:24:11.280+0000 7f1e12d63700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T10:24:11.566 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:24:11 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:24:11.280+0000 7f1e12d63700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T10:24:11.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:11 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/4193468833' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-23T10:24:11.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:11 smithi196 ceph-mon[32919]: mds.? [v2:172.21.15.196:6838/83843556,v1:172.21.15.196:6839/83843556] up:active 2024-04-23T10:24:11.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:11 smithi196 ceph-mon[32919]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-23T10:24:11.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:11 smithi196 ceph-mon[32919]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-23T10:24:11.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:11 smithi196 ceph-mon[32919]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-04-23T10:24:11.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:11 smithi196 ceph-mon[32919]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-04-23T10:24:11.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:11 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/2150329423' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-23T10:24:11.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:11 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/2150329423' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-23T10:24:11.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:11 smithi196 ceph-mon[32919]: mds.? [v2:172.21.15.196:6836/3094946502,v1:172.21.15.196:6837/3094946502] up:active 2024-04-23T10:24:11.843 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:11 smithi196 ceph-mon[32919]: daemon mds.l assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-23T10:24:11.843 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:11 smithi196 ceph-mon[32919]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T10:24:11.843 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:11 smithi196 ceph-mon[32919]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=l=up:creating} 3 up:standby-replay 5 up:standby 2024-04-23T10:24:11.843 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:11 smithi196 ceph-mon[32919]: daemon mds.l is now active in filesystem cephfs as rank 3 2024-04-23T10:24:11.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:11 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/4193468833' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-23T10:24:11.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:11 smithi097 ceph-mon[29197]: mds.? [v2:172.21.15.196:6838/83843556,v1:172.21.15.196:6839/83843556] up:active 2024-04-23T10:24:11.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:11 smithi097 ceph-mon[29197]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-23T10:24:11.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:11 smithi097 ceph-mon[29197]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-23T10:24:11.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:11 smithi097 ceph-mon[29197]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-04-23T10:24:11.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:11 smithi097 ceph-mon[29197]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-04-23T10:24:11.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:11 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/2150329423' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-23T10:24:11.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:11 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/2150329423' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-23T10:24:11.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:11 smithi097 ceph-mon[29197]: mds.? [v2:172.21.15.196:6836/3094946502,v1:172.21.15.196:6837/3094946502] up:active 2024-04-23T10:24:11.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:11 smithi097 ceph-mon[29197]: daemon mds.l assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-23T10:24:11.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:11 smithi097 ceph-mon[29197]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T10:24:11.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:11 smithi097 ceph-mon[29197]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=l=up:creating} 3 up:standby-replay 5 up:standby 2024-04-23T10:24:11.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:11 smithi097 ceph-mon[29197]: daemon mds.l is now active in filesystem cephfs as rank 3 2024-04-23T10:24:11.900 INFO:journalctl@ceph.mds.b.smithi183.stdout:Apr 23 10:24:11 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-b[62562]: 2024-04-23T10:24:11.628+0000 7f4185e42700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-23T10:24:11.900 INFO:journalctl@ceph.mds.b.smithi183.stdout:Apr 23 10:24:11 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-b[62562]: 2024-04-23T10:24:11.628+0000 7f4185e42700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-23T10:24:11.900 INFO:journalctl@ceph.mds.b.smithi183.stdout:Apr 23 10:24:11 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-b[62562]: 2024-04-23T10:24:11.628+0000 7f4185e42700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-04-23T10:24:11.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:11 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/4193468833' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-23T10:24:11.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:11 smithi183 ceph-mon[32317]: mds.? [v2:172.21.15.196:6838/83843556,v1:172.21.15.196:6839/83843556] up:active 2024-04-23T10:24:11.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:11 smithi183 ceph-mon[32317]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-23T10:24:11.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:11 smithi183 ceph-mon[32317]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-23T10:24:11.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:11 smithi183 ceph-mon[32317]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-04-23T10:24:11.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:11 smithi183 ceph-mon[32317]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-04-23T10:24:11.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:11 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/2150329423' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-23T10:24:11.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:11 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/2150329423' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-23T10:24:11.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:11 smithi183 ceph-mon[32317]: mds.? [v2:172.21.15.196:6836/3094946502,v1:172.21.15.196:6837/3094946502] up:active 2024-04-23T10:24:11.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:11 smithi183 ceph-mon[32317]: daemon mds.l assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-23T10:24:11.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:11 smithi183 ceph-mon[32317]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-23T10:24:11.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:11 smithi183 ceph-mon[32317]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=l=up:creating} 3 up:standby-replay 5 up:standby 2024-04-23T10:24:11.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:11 smithi183 ceph-mon[32317]: daemon mds.l is now active in filesystem cephfs as rank 3 2024-04-23T10:24:11.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:11.983+0000 7f028d7fa640 1 -- 172.21.15.97:0/842155738 <== mgr.14152 v2:172.21.15.97:6800/3149043749 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0290062390 con 0x7f02780a3bb0 2024-04-23T10:24:11.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:11.985+0000 7f0296410640 1 -- 172.21.15.97:0/842155738 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f02780a3bb0 msgr2=0x7f02780a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:11.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:11.985+0000 7f0296410640 1 --2- 172.21.15.97:0/842155738 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f02780a3bb0 0x7f02780a6070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0290102110 tx=0x7f028000b410 comp rx=0 tx=0).stop 2024-04-23T10:24:11.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:11.985+0000 7f0296410640 1 -- 172.21.15.97:0/842155738 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0290102a30 msgr2=0x7f02901023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:11.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:11.985+0000 7f0296410640 1 --2- 172.21.15.97:0/842155738 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0290102a30 0x7f02901023a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0284005940 tx=0x7f0284031a50 comp rx=0 tx=0).stop 2024-04-23T10:24:11.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:11.985+0000 7f0296410640 1 -- 172.21.15.97:0/842155738 shutdown_connections 2024-04-23T10:24:11.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:11.985+0000 7f0296410640 1 --2- 172.21.15.97:0/842155738 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f02780a3bb0 0x7f02780a6070 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:11.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:11.985+0000 7f0296410640 1 --2- 172.21.15.97:0/842155738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0290107c90 0x7f0290100f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:11.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:11.985+0000 7f0296410640 1 --2- 172.21.15.97:0/842155738 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0290105360 0x7f02901009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:11.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:11.985+0000 7f0296410640 1 --2- 172.21.15.97:0/842155738 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0290102a30 0x7f02901023a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:11.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:11.985+0000 7f0296410640 1 -- 172.21.15.97:0/842155738 >> 172.21.15.97:0/842155738 conn(0x7f02900fc420 msgr2=0x7f0290103530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:11.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:11.985+0000 7f0296410640 1 -- 172.21.15.97:0/842155738 shutdown_connections 2024-04-23T10:24:11.985 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:11.985+0000 7f0296410640 1 -- 172.21.15.97:0/842155738 wait complete. 2024-04-23T10:24:11.996 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-23T10:24:12.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.100+0000 7fac377bf640 1 -- 172.21.15.97:0/1829968939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac30101810 msgr2=0x7fac30101c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:12.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.100+0000 7fac377bf640 1 --2- 172.21.15.97:0/1829968939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac30101810 0x7fac30101c90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fac2c009900 tx=0x7fac2c02a410 comp rx=0 tx=0).stop 2024-04-23T10:24:12.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.101+0000 7fac377bf640 1 -- 172.21.15.97:0/1829968939 shutdown_connections 2024-04-23T10:24:12.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.101+0000 7fac377bf640 1 --2- 172.21.15.97:0/1829968939 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fac301021d0 0x7fac30108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:12.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.101+0000 7fac377bf640 1 --2- 172.21.15.97:0/1829968939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac30101810 0x7fac30101c90 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:12.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.101+0000 7fac377bf640 1 --2- 172.21.15.97:0/1829968939 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac30100620 0x7fac30100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:12.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.101+0000 7fac377bf640 1 -- 172.21.15.97:0/1829968939 >> 172.21.15.97:0/1829968939 conn(0x7fac300fbb90 msgr2=0x7fac300fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:12.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.101+0000 7fac377bf640 1 -- 172.21.15.97:0/1829968939 shutdown_connections 2024-04-23T10:24:12.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.101+0000 7fac377bf640 1 -- 172.21.15.97:0/1829968939 wait complete. 2024-04-23T10:24:12.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.101+0000 7fac377bf640 1 Processor -- start 2024-04-23T10:24:12.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.101+0000 7fac377bf640 1 -- start start 2024-04-23T10:24:12.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.101+0000 7fac377bf640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac30100620 0x7fac301981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:12.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.101+0000 7fac377bf640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fac30101810 0x7fac30198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:12.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.101+0000 7fac377bf640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac301021d0 0x7fac3019f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:12.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.101+0000 7fac377bf640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fac3010c9e0 con 0x7fac301021d0 2024-04-23T10:24:12.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.101+0000 7fac377bf640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fac3010c860 con 0x7fac30101810 2024-04-23T10:24:12.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.101+0000 7fac377bf640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fac3010cb60 con 0x7fac30100620 2024-04-23T10:24:12.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.101+0000 7fac35d5c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac301021d0 0x7fac3019f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:12.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.101+0000 7fac35d5c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac301021d0 0x7fac3019f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:32972/0 (socket says 172.21.15.97:32972) 2024-04-23T10:24:12.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.101+0000 7fac35d5c640 1 -- 172.21.15.97:0/437968305 learned_addr learned my addr 172.21.15.97:0/437968305 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:24:12.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.101+0000 7fac34d5a640 1 --2- 172.21.15.97:0/437968305 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fac30101810 0x7fac30198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:12.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.102+0000 7fac3555b640 1 --2- 172.21.15.97:0/437968305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac30100620 0x7fac301981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:12.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.102+0000 7fac35d5c640 1 -- 172.21.15.97:0/437968305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac30100620 msgr2=0x7fac301981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:12.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.102+0000 7fac35d5c640 1 --2- 172.21.15.97:0/437968305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac30100620 0x7fac301981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:12.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.102+0000 7fac35d5c640 1 -- 172.21.15.97:0/437968305 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fac30101810 msgr2=0x7fac30198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:12.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.102+0000 7fac35d5c640 1 --2- 172.21.15.97:0/437968305 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fac30101810 0x7fac30198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:12.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.102+0000 7fac35d5c640 1 -- 172.21.15.97:0/437968305 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac3019fe00 con 0x7fac301021d0 2024-04-23T10:24:12.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.102+0000 7fac35d5c640 1 --2- 172.21.15.97:0/437968305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac301021d0 0x7fac3019f790 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fac2800eb10 tx=0x7fac2800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:12.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.102+0000 7fac267fc640 1 -- 172.21.15.97:0/437968305 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac28004b20 con 0x7fac301021d0 2024-04-23T10:24:12.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.102+0000 7fac267fc640 1 -- 172.21.15.97:0/437968305 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac28004cc0 con 0x7fac301021d0 2024-04-23T10:24:12.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.102+0000 7fac377bf640 1 -- 172.21.15.97:0/437968305 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac301a00f0 con 0x7fac301021d0 2024-04-23T10:24:12.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.102+0000 7fac267fc640 1 -- 172.21.15.97:0/437968305 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac28010670 con 0x7fac301021d0 2024-04-23T10:24:12.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.102+0000 7fac377bf640 1 -- 172.21.15.97:0/437968305 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac301a05c0 con 0x7fac301021d0 2024-04-23T10:24:12.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.103+0000 7fac377bf640 1 -- 172.21.15.97:0/437968305 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac301a0880 con 0x7fac301021d0 2024-04-23T10:24:12.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.104+0000 7fac267fc640 1 -- 172.21.15.97:0/437968305 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fac28005020 con 0x7fac301021d0 2024-04-23T10:24:12.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.105+0000 7fac267fc640 1 --2- 172.21.15.97:0/437968305 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fac1c0a3bb0 0x7fac1c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:12.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.105+0000 7fac3555b640 1 --2- 172.21.15.97:0/437968305 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fac1c0a3bb0 0x7fac1c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:12.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.105+0000 7fac267fc640 1 -- 172.21.15.97:0/437968305 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fac280d0910 con 0x7fac301021d0 2024-04-23T10:24:12.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.105+0000 7fac3555b640 1 --2- 172.21.15.97:0/437968305 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fac1c0a3bb0 0x7fac1c0a6070 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fac20005fd0 tx=0x7fac20005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:12.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.109+0000 7fac267fc640 1 -- 172.21.15.97:0/437968305 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fac2809d440 con 0x7fac301021d0 2024-04-23T10:24:12.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.317+0000 7fac377bf640 1 -- 172.21.15.97:0/437968305 --> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fac300fddb0 con 0x7fac1c0a3bb0 2024-04-23T10:24:12.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.340+0000 7fac267fc640 1 -- 172.21.15.97:0/437968305 <== mgr.14152 v2:172.21.15.97:6800/3149043749 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fac300fddb0 con 0x7fac1c0a3bb0 2024-04-23T10:24:12.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.342+0000 7fac377bf640 1 -- 172.21.15.97:0/437968305 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fac1c0a3bb0 msgr2=0x7fac1c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:12.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.342+0000 7fac377bf640 1 --2- 172.21.15.97:0/437968305 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fac1c0a3bb0 0x7fac1c0a6070 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fac20005fd0 tx=0x7fac20005850 comp rx=0 tx=0).stop 2024-04-23T10:24:12.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.342+0000 7fac377bf640 1 -- 172.21.15.97:0/437968305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac301021d0 msgr2=0x7fac3019f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:12.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.342+0000 7fac377bf640 1 --2- 172.21.15.97:0/437968305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac301021d0 0x7fac3019f790 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fac2800eb10 tx=0x7fac2800c2d0 comp rx=0 tx=0).stop 2024-04-23T10:24:12.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.342+0000 7fac377bf640 1 -- 172.21.15.97:0/437968305 shutdown_connections 2024-04-23T10:24:12.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.342+0000 7fac377bf640 1 --2- 172.21.15.97:0/437968305 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fac1c0a3bb0 0x7fac1c0a6070 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:12.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.342+0000 7fac377bf640 1 --2- 172.21.15.97:0/437968305 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac301021d0 0x7fac3019f790 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:12.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.342+0000 7fac377bf640 1 --2- 172.21.15.97:0/437968305 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fac30101810 0x7fac30198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:12.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.342+0000 7fac377bf640 1 --2- 172.21.15.97:0/437968305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac30100620 0x7fac301981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:12.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.342+0000 7fac377bf640 1 -- 172.21.15.97:0/437968305 >> 172.21.15.97:0/437968305 conn(0x7fac300fbb90 msgr2=0x7fac300fd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:12.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.342+0000 7fac377bf640 1 -- 172.21.15.97:0/437968305 shutdown_connections 2024-04-23T10:24:12.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.342+0000 7fac377bf640 1 -- 172.21.15.97:0/437968305 wait complete. 2024-04-23T10:24:12.352 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-23T10:24:12.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.455+0000 7faf12cfc640 1 -- 172.21.15.97:0/4280711202 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf0c101fe0 msgr2=0x7faf0c102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:12.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.455+0000 7faf12cfc640 1 --2- 172.21.15.97:0/4280711202 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf0c101fe0 0x7faf0c102460 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7faef8009900 tx=0x7faef802a410 comp rx=0 tx=0).stop 2024-04-23T10:24:12.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.455+0000 7faf12cfc640 1 -- 172.21.15.97:0/4280711202 shutdown_connections 2024-04-23T10:24:12.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.455+0000 7faf12cfc640 1 --2- 172.21.15.97:0/4280711202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf0c1029a0 0x7faf0c1092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:12.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.455+0000 7faf12cfc640 1 --2- 172.21.15.97:0/4280711202 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf0c101fe0 0x7faf0c102460 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:12.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.455+0000 7faf12cfc640 1 --2- 172.21.15.97:0/4280711202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf0c100e90 0x7faf0c1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:12.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.455+0000 7faf12cfc640 1 -- 172.21.15.97:0/4280711202 >> 172.21.15.97:0/4280711202 conn(0x7faf0c0fc400 msgr2=0x7faf0c0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:12.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.456+0000 7faf12cfc640 1 -- 172.21.15.97:0/4280711202 shutdown_connections 2024-04-23T10:24:12.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.456+0000 7faf12cfc640 1 -- 172.21.15.97:0/4280711202 wait complete. 2024-04-23T10:24:12.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.456+0000 7faf12cfc640 1 Processor -- start 2024-04-23T10:24:12.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.456+0000 7faf12cfc640 1 -- start start 2024-04-23T10:24:12.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.456+0000 7faf12cfc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf0c100e90 0x7faf0c198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:12.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.456+0000 7faf12cfc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf0c101fe0 0x7faf0c1986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:12.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.456+0000 7faf12cfc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf0c1029a0 0x7faf0c19f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:12.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.456+0000 7faf12cfc640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7faf0c10c9d0 con 0x7faf0c101fe0 2024-04-23T10:24:12.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.456+0000 7faf12cfc640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7faf0c10c850 con 0x7faf0c100e90 2024-04-23T10:24:12.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.456+0000 7faf12cfc640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faf0c10cb50 con 0x7faf0c1029a0 2024-04-23T10:24:12.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.456+0000 7faf0bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf0c101fe0 0x7faf0c1986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:12.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.456+0000 7faf0bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf0c101fe0 0x7faf0c1986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:32998/0 (socket says 172.21.15.97:32998) 2024-04-23T10:24:12.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.456+0000 7faf0bfff640 1 -- 172.21.15.97:0/1837514624 learned_addr learned my addr 172.21.15.97:0/1837514624 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:24:12.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.456+0000 7faf10a98640 1 --2- 172.21.15.97:0/1837514624 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf0c100e90 0x7faf0c198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:12.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.456+0000 7faf11299640 1 --2- 172.21.15.97:0/1837514624 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf0c1029a0 0x7faf0c19f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:12.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.457+0000 7faf0bfff640 1 -- 172.21.15.97:0/1837514624 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf0c1029a0 msgr2=0x7faf0c19f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:12.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.457+0000 7faf0bfff640 1 --2- 172.21.15.97:0/1837514624 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf0c1029a0 0x7faf0c19f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:12.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.457+0000 7faf0bfff640 1 -- 172.21.15.97:0/1837514624 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf0c100e90 msgr2=0x7faf0c198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:12.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.457+0000 7faf0bfff640 1 --2- 172.21.15.97:0/1837514624 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf0c100e90 0x7faf0c198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:12.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.457+0000 7faf0bfff640 1 -- 172.21.15.97:0/1837514624 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf0c19fda0 con 0x7faf0c101fe0 2024-04-23T10:24:12.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.457+0000 7faf0bfff640 1 --2- 172.21.15.97:0/1837514624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf0c101fe0 0x7faf0c1986c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7faef8006010 tx=0x7faef80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:12.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.457+0000 7faf09ffb640 1 -- 172.21.15.97:0/1837514624 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef803f030 con 0x7faf0c101fe0 2024-04-23T10:24:12.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.457+0000 7faf12cfc640 1 -- 172.21.15.97:0/1837514624 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf0c1a0030 con 0x7faf0c101fe0 2024-04-23T10:24:12.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.457+0000 7faf09ffb640 1 -- 172.21.15.97:0/1837514624 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faef80318f0 con 0x7faf0c101fe0 2024-04-23T10:24:12.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.457+0000 7faf09ffb640 1 -- 172.21.15.97:0/1837514624 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef8035070 con 0x7faf0c101fe0 2024-04-23T10:24:12.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.457+0000 7faf12cfc640 1 -- 172.21.15.97:0/1837514624 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf0c1a0520 con 0x7faf0c101fe0 2024-04-23T10:24:12.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.458+0000 7faf12cfc640 1 -- 172.21.15.97:0/1837514624 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf0c1a0780 con 0x7faf0c101fe0 2024-04-23T10:24:12.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.459+0000 7faf09ffb640 1 -- 172.21.15.97:0/1837514624 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7faef8042030 con 0x7faf0c101fe0 2024-04-23T10:24:12.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.460+0000 7faf09ffb640 1 --2- 172.21.15.97:0/1837514624 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7faefc0a3bb0 0x7faefc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:12.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.460+0000 7faf10a98640 1 --2- 172.21.15.97:0/1837514624 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7faefc0a3bb0 0x7faefc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:12.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.460+0000 7faf09ffb640 1 -- 172.21.15.97:0/1837514624 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7faef80f0bd0 con 0x7faf0c101fe0 2024-04-23T10:24:12.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.460+0000 7faf10a98640 1 --2- 172.21.15.97:0/1837514624 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7faefc0a3bb0 0x7faefc0a6070 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7faf0000abe0 tx=0x7faf0000a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:12.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.463+0000 7faf09ffb640 1 -- 172.21.15.97:0/1837514624 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faef80bd700 con 0x7faf0c101fe0 2024-04-23T10:24:12.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:12 smithi097 ceph-mon[29197]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:24:12.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:12 smithi097 ceph-mon[29197]: pgmap v292: 97 pgs: 8 creating+peering, 89 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 2.8 KiB/s wr, 13 op/s 2024-04-23T10:24:12.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:12 smithi097 ceph-mon[29197]: mds.? [v2:172.21.15.196:6832/3885816565,v1:172.21.15.196:6833/3885816565] up:active 2024-04-23T10:24:12.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:12 smithi097 ceph-mon[29197]: daemon mds.k assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-23T10:24:12.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:12 smithi097 ceph-mon[29197]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-23T10:24:12.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:12 smithi097 ceph-mon[29197]: Cluster is now healthy 2024-04-23T10:24:12.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:12 smithi097 ceph-mon[29197]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=l=up:active} 3 up:standby-replay 5 up:standby 2024-04-23T10:24:12.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:12 smithi097 ceph-mon[29197]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2024-04-23T10:24:12.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:12 smithi097 ceph-mon[29197]: daemon mds.k is now active in filesystem cephfs as rank 4 2024-04-23T10:24:12.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.672+0000 7faf12cfc640 1 -- 172.21.15.97:0/1837514624 --> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7faf0c0fe600 con 0x7faefc0a3bb0 2024-04-23T10:24:12.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.677+0000 7faf09ffb640 1 -- 172.21.15.97:0/1837514624 <== mgr.14152 v2:172.21.15.97:6800/3149043749 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7faf0c0fe600 con 0x7faefc0a3bb0 2024-04-23T10:24:12.676 INFO:teuthology.orchestra.run.smithi097.stdout:/volumes/qa/sv_0/dcc1df0d-b5f1-461b-8b47-5b5dd7f9ec44 2024-04-23T10:24:12.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.679+0000 7faf12cfc640 1 -- 172.21.15.97:0/1837514624 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7faefc0a3bb0 msgr2=0x7faefc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:12.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.679+0000 7faf12cfc640 1 --2- 172.21.15.97:0/1837514624 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7faefc0a3bb0 0x7faefc0a6070 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7faf0000abe0 tx=0x7faf0000a2d0 comp rx=0 tx=0).stop 2024-04-23T10:24:12.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.679+0000 7faf12cfc640 1 -- 172.21.15.97:0/1837514624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf0c101fe0 msgr2=0x7faf0c1986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:12.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.679+0000 7faf12cfc640 1 --2- 172.21.15.97:0/1837514624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf0c101fe0 0x7faf0c1986c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7faef8006010 tx=0x7faef80042f0 comp rx=0 tx=0).stop 2024-04-23T10:24:12.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.679+0000 7faf12cfc640 1 -- 172.21.15.97:0/1837514624 shutdown_connections 2024-04-23T10:24:12.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.679+0000 7faf12cfc640 1 --2- 172.21.15.97:0/1837514624 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7faefc0a3bb0 0x7faefc0a6070 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:12.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.679+0000 7faf12cfc640 1 --2- 172.21.15.97:0/1837514624 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf0c1029a0 0x7faf0c19f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:12.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.679+0000 7faf12cfc640 1 --2- 172.21.15.97:0/1837514624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf0c101fe0 0x7faf0c1986c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:12.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.679+0000 7faf12cfc640 1 --2- 172.21.15.97:0/1837514624 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf0c100e90 0x7faf0c198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:12.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.679+0000 7faf12cfc640 1 -- 172.21.15.97:0/1837514624 >> 172.21.15.97:0/1837514624 conn(0x7faf0c0fc400 msgr2=0x7faf0c0fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:12.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.679+0000 7faf12cfc640 1 -- 172.21.15.97:0/1837514624 shutdown_connections 2024-04-23T10:24:12.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.679+0000 7faf12cfc640 1 -- 172.21.15.97:0/1837514624 wait complete. 2024-04-23T10:24:12.689 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-23T10:24:12.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.792+0000 7f9614423640 1 -- 172.21.15.97:0/3180969024 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f960c103b40 msgr2=0x7f960c10a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:12.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.792+0000 7f9614423640 1 --2- 172.21.15.97:0/3180969024 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f960c103b40 0x7f960c10a3d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9604009900 tx=0x7f960402a410 comp rx=0 tx=0).stop 2024-04-23T10:24:12.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.792+0000 7f9614423640 1 -- 172.21.15.97:0/3180969024 shutdown_connections 2024-04-23T10:24:12.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.792+0000 7f9614423640 1 --2- 172.21.15.97:0/3180969024 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f960c103b40 0x7f960c10a3d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:12.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.792+0000 7f9614423640 1 --2- 172.21.15.97:0/3180969024 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f960c103180 0x7f960c103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:12.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.792+0000 7f9614423640 1 --2- 172.21.15.97:0/3180969024 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f960c101f90 0x7f960c1023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:12.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.792+0000 7f9614423640 1 -- 172.21.15.97:0/3180969024 >> 172.21.15.97:0/3180969024 conn(0x7f960c0fd500 msgr2=0x7f960c0ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:12.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.792+0000 7f9614423640 1 -- 172.21.15.97:0/3180969024 shutdown_connections 2024-04-23T10:24:12.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.792+0000 7f9614423640 1 -- 172.21.15.97:0/3180969024 wait complete. 2024-04-23T10:24:12.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.793+0000 7f9614423640 1 Processor -- start 2024-04-23T10:24:12.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.793+0000 7f9614423640 1 -- start start 2024-04-23T10:24:12.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.793+0000 7f9614423640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f960c101f90 0x7f960c075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:12.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.793+0000 7f9614423640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f960c103180 0x7f960c074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:12.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.793+0000 7f9614423640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f960c103b40 0x7f960c074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:12.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.793+0000 7f9614423640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f960c10da90 con 0x7f960c103b40 2024-04-23T10:24:12.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.793+0000 7f9614423640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f960c10d910 con 0x7f960c101f90 2024-04-23T10:24:12.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.793+0000 7f9614423640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f960c10dc10 con 0x7f960c103180 2024-04-23T10:24:12.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.793+0000 7f96129c0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f960c103b40 0x7f960c074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:12.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.793+0000 7f96129c0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f960c103b40 0x7f960c074880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33020/0 (socket says 172.21.15.97:33020) 2024-04-23T10:24:12.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.793+0000 7f96129c0640 1 -- 172.21.15.97:0/1537355972 learned_addr learned my addr 172.21.15.97:0/1537355972 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:24:12.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.793+0000 7f96121bf640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f960c101f90 0x7f960c075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:12.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.793+0000 7f96119be640 1 --2- 172.21.15.97:0/1537355972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f960c103180 0x7f960c074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:12.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.793+0000 7f96129c0640 1 -- 172.21.15.97:0/1537355972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f960c103180 msgr2=0x7f960c074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:12.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.793+0000 7f96129c0640 1 --2- 172.21.15.97:0/1537355972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f960c103180 0x7f960c074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:12.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.793+0000 7f96129c0640 1 -- 172.21.15.97:0/1537355972 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f960c101f90 msgr2=0x7f960c075d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:12.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.793+0000 7f96129c0640 1 --2- 172.21.15.97:0/1537355972 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f960c101f90 0x7f960c075d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:12.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.793+0000 7f96129c0640 1 -- 172.21.15.97:0/1537355972 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f960c075110 con 0x7f960c103b40 2024-04-23T10:24:12.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.793+0000 7f96121bf640 1 --2- 172.21.15.97:0/1537355972 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f960c101f90 0x7f960c075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T10:24:12.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.794+0000 7f96129c0640 1 --2- 172.21.15.97:0/1537355972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f960c103b40 0x7f960c074880 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9604031500 tx=0x7f96040042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:12.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.794+0000 7f96037fe640 1 -- 172.21.15.97:0/1537355972 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f960403f030 con 0x7f960c103b40 2024-04-23T10:24:12.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.794+0000 7f96037fe640 1 -- 172.21.15.97:0/1537355972 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9604003cb0 con 0x7f960c103b40 2024-04-23T10:24:12.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.794+0000 7f9614423640 1 -- 172.21.15.97:0/1537355972 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f960c1a2230 con 0x7f960c103b40 2024-04-23T10:24:12.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.794+0000 7f96037fe640 1 -- 172.21.15.97:0/1537355972 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9604035070 con 0x7f960c103b40 2024-04-23T10:24:12.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.794+0000 7f9614423640 1 -- 172.21.15.97:0/1537355972 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f960c1a2780 con 0x7f960c103b40 2024-04-23T10:24:12.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.795+0000 7f9614423640 1 -- 172.21.15.97:0/1537355972 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f960c109420 con 0x7f960c103b40 2024-04-23T10:24:12.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.796+0000 7f96037fe640 1 -- 172.21.15.97:0/1537355972 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9604042030 con 0x7f960c103b40 2024-04-23T10:24:12.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.797+0000 7f96037fe640 1 --2- 172.21.15.97:0/1537355972 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f95f80a3bb0 0x7f95f80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:12.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.797+0000 7f96121bf640 1 --2- 172.21.15.97:0/1537355972 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f95f80a3bb0 0x7f95f80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:12.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.797+0000 7f96037fe640 1 -- 172.21.15.97:0/1537355972 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f96040f07c0 con 0x7f960c103b40 2024-04-23T10:24:12.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.797+0000 7f96121bf640 1 --2- 172.21.15.97:0/1537355972 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f95f80a3bb0 0x7f95f80a6070 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f96080099a0 tx=0x7f9608009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:12.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:12.801+0000 7f96037fe640 1 -- 172.21.15.97:0/1537355972 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f96040bcca0 con 0x7f960c103b40 2024-04-23T10:24:12.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:12 smithi196 ceph-mon[32919]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:24:12.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:12 smithi196 ceph-mon[32919]: pgmap v292: 97 pgs: 8 creating+peering, 89 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 2.8 KiB/s wr, 13 op/s 2024-04-23T10:24:12.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:12 smithi196 ceph-mon[32919]: mds.? [v2:172.21.15.196:6832/3885816565,v1:172.21.15.196:6833/3885816565] up:active 2024-04-23T10:24:12.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:12 smithi196 ceph-mon[32919]: daemon mds.k assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-23T10:24:12.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:12 smithi196 ceph-mon[32919]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-23T10:24:12.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:12 smithi196 ceph-mon[32919]: Cluster is now healthy 2024-04-23T10:24:12.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:12 smithi196 ceph-mon[32919]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=l=up:active} 3 up:standby-replay 5 up:standby 2024-04-23T10:24:12.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:12 smithi196 ceph-mon[32919]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2024-04-23T10:24:12.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:12 smithi196 ceph-mon[32919]: daemon mds.k is now active in filesystem cephfs as rank 4 2024-04-23T10:24:12.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:12 smithi183 ceph-mon[32317]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:24:12.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:12 smithi183 ceph-mon[32317]: pgmap v292: 97 pgs: 8 creating+peering, 89 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 2.8 KiB/s wr, 13 op/s 2024-04-23T10:24:12.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:12 smithi183 ceph-mon[32317]: mds.? [v2:172.21.15.196:6832/3885816565,v1:172.21.15.196:6833/3885816565] up:active 2024-04-23T10:24:12.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:12 smithi183 ceph-mon[32317]: daemon mds.k assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-23T10:24:12.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:12 smithi183 ceph-mon[32317]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-23T10:24:12.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:12 smithi183 ceph-mon[32317]: Cluster is now healthy 2024-04-23T10:24:12.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:12 smithi183 ceph-mon[32317]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=l=up:active} 3 up:standby-replay 5 up:standby 2024-04-23T10:24:12.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:12 smithi183 ceph-mon[32317]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2024-04-23T10:24:12.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:12 smithi183 ceph-mon[32317]: daemon mds.k is now active in filesystem cephfs as rank 4 2024-04-23T10:24:13.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.012+0000 7f9614423640 1 -- 172.21.15.97:0/1537355972 --> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f960c062420 con 0x7f95f80a3bb0 2024-04-23T10:24:13.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.036+0000 7f96037fe640 1 -- 172.21.15.97:0/1537355972 <== mgr.14152 v2:172.21.15.97:6800/3149043749 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f960c062420 con 0x7f95f80a3bb0 2024-04-23T10:24:13.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.038+0000 7f9614423640 1 -- 172.21.15.97:0/1537355972 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f95f80a3bb0 msgr2=0x7f95f80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:13.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.038+0000 7f9614423640 1 --2- 172.21.15.97:0/1537355972 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f95f80a3bb0 0x7f95f80a6070 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f96080099a0 tx=0x7f9608009500 comp rx=0 tx=0).stop 2024-04-23T10:24:13.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.038+0000 7f9614423640 1 -- 172.21.15.97:0/1537355972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f960c103b40 msgr2=0x7f960c074880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:13.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.038+0000 7f9614423640 1 --2- 172.21.15.97:0/1537355972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f960c103b40 0x7f960c074880 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9604031500 tx=0x7f96040042f0 comp rx=0 tx=0).stop 2024-04-23T10:24:13.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.038+0000 7f9614423640 1 -- 172.21.15.97:0/1537355972 shutdown_connections 2024-04-23T10:24:13.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.038+0000 7f9614423640 1 --2- 172.21.15.97:0/1537355972 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f95f80a3bb0 0x7f95f80a6070 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:13.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.038+0000 7f9614423640 1 --2- 172.21.15.97:0/1537355972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f960c103b40 0x7f960c074880 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:13.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.038+0000 7f9614423640 1 --2- 172.21.15.97:0/1537355972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f960c103180 0x7f960c074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:13.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.038+0000 7f9614423640 1 --2- 172.21.15.97:0/1537355972 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f960c101f90 0x7f960c075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:13.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.038+0000 7f9614423640 1 -- 172.21.15.97:0/1537355972 >> 172.21.15.97:0/1537355972 conn(0x7f960c0fd500 msgr2=0x7f960c1065f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:13.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.038+0000 7f9614423640 1 -- 172.21.15.97:0/1537355972 shutdown_connections 2024-04-23T10:24:13.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.038+0000 7f9614423640 1 -- 172.21.15.97:0/1537355972 wait complete. 2024-04-23T10:24:13.048 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-23T10:24:13.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.152+0000 7fbf1f260640 1 -- 172.21.15.97:0/1606294649 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf18101fa0 msgr2=0x7fbf181023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:13.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.152+0000 7fbf1f260640 1 --2- 172.21.15.97:0/1606294649 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf18101fa0 0x7fbf181023c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c009900 tx=0x7fbf0c02a410 comp rx=0 tx=0).stop 2024-04-23T10:24:13.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.152+0000 7fbf1f260640 1 -- 172.21.15.97:0/1606294649 shutdown_connections 2024-04-23T10:24:13.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.152+0000 7fbf1f260640 1 --2- 172.21.15.97:0/1606294649 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf18103b50 0x7fbf1810a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:13.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.152+0000 7fbf1f260640 1 --2- 172.21.15.97:0/1606294649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf18103190 0x7fbf18103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:13.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.152+0000 7fbf1f260640 1 --2- 172.21.15.97:0/1606294649 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf18101fa0 0x7fbf181023c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:13.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.152+0000 7fbf1f260640 1 -- 172.21.15.97:0/1606294649 >> 172.21.15.97:0/1606294649 conn(0x7fbf180fd530 msgr2=0x7fbf180ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:13.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.152+0000 7fbf1f260640 1 -- 172.21.15.97:0/1606294649 shutdown_connections 2024-04-23T10:24:13.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.152+0000 7fbf1f260640 1 -- 172.21.15.97:0/1606294649 wait complete. 2024-04-23T10:24:13.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.153+0000 7fbf1f260640 1 Processor -- start 2024-04-23T10:24:13.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.153+0000 7fbf1f260640 1 -- start start 2024-04-23T10:24:13.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.153+0000 7fbf1f260640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf18101fa0 0x7fbf18198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:13.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.153+0000 7fbf1f260640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf18103190 0x7fbf181986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:13.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.153+0000 7fbf1f260640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf18103b50 0x7fbf1819f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:13.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.153+0000 7fbf1f260640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbf1810db30 con 0x7fbf18103190 2024-04-23T10:24:13.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.153+0000 7fbf1f260640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbf1810d9b0 con 0x7fbf18103b50 2024-04-23T10:24:13.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.153+0000 7fbf1f260640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbf1810dcb0 con 0x7fbf18101fa0 2024-04-23T10:24:13.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.153+0000 7fbf17fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf18103190 0x7fbf181986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:13.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.153+0000 7fbf17fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf18103190 0x7fbf181986b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33038/0 (socket says 172.21.15.97:33038) 2024-04-23T10:24:13.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.153+0000 7fbf17fff640 1 -- 172.21.15.97:0/2762935645 learned_addr learned my addr 172.21.15.97:0/2762935645 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:24:13.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.153+0000 7fbf1d7fd640 1 --2- 172.21.15.97:0/2762935645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf18103b50 0x7fbf1819f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:13.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.153+0000 7fbf17fff640 1 -- 172.21.15.97:0/2762935645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf18101fa0 msgr2=0x7fbf18198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:13.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.153+0000 7fbf17fff640 1 --2- 172.21.15.97:0/2762935645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf18101fa0 0x7fbf18198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:13.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.153+0000 7fbf17fff640 1 -- 172.21.15.97:0/2762935645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf18103b50 msgr2=0x7fbf1819f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:13.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.153+0000 7fbf17fff640 1 --2- 172.21.15.97:0/2762935645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf18103b50 0x7fbf1819f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:13.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.153+0000 7fbf17fff640 1 -- 172.21.15.97:0/2762935645 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf1819fd90 con 0x7fbf18103190 2024-04-23T10:24:13.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.153+0000 7fbf17fff640 1 --2- 172.21.15.97:0/2762935645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf18103190 0x7fbf181986b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fbf0400c920 tx=0x7fbf0400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:13.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.153+0000 7fbf15ffb640 1 -- 172.21.15.97:0/2762935645 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf04007c70 con 0x7fbf18103190 2024-04-23T10:24:13.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.154+0000 7fbf15ffb640 1 -- 172.21.15.97:0/2762935645 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf04007e10 con 0x7fbf18103190 2024-04-23T10:24:13.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.154+0000 7fbf1f260640 1 -- 172.21.15.97:0/2762935645 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf181a0080 con 0x7fbf18103190 2024-04-23T10:24:13.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.154+0000 7fbf15ffb640 1 -- 172.21.15.97:0/2762935645 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf04018820 con 0x7fbf18103190 2024-04-23T10:24:13.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.154+0000 7fbf1f260640 1 -- 172.21.15.97:0/2762935645 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf181a05d0 con 0x7fbf18103190 2024-04-23T10:24:13.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.155+0000 7fbf1f260640 1 -- 172.21.15.97:0/2762935645 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf18109340 con 0x7fbf18103190 2024-04-23T10:24:13.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.156+0000 7fbf15ffb640 1 -- 172.21.15.97:0/2762935645 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fbf0400f450 con 0x7fbf18103190 2024-04-23T10:24:13.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.156+0000 7fbf15ffb640 1 --2- 172.21.15.97:0/2762935645 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fbf080a3bb0 0x7fbf080a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:13.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.157+0000 7fbf1cffc640 1 --2- 172.21.15.97:0/2762935645 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fbf080a3bb0 0x7fbf080a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:13.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.157+0000 7fbf15ffb640 1 -- 172.21.15.97:0/2762935645 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fbf040d03c0 con 0x7fbf18103190 2024-04-23T10:24:13.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.157+0000 7fbf1cffc640 1 --2- 172.21.15.97:0/2762935645 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fbf080a3bb0 0x7fbf080a6070 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c009230 tx=0x7fbf0c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:13.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.161+0000 7fbf15ffb640 1 -- 172.21.15.97:0/2762935645 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbf04014030 con 0x7fbf18103190 2024-04-23T10:24:13.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.367+0000 7fbf1f260640 1 -- 172.21.15.97:0/2762935645 --> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fbf18062390 con 0x7fbf080a3bb0 2024-04-23T10:24:13.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.373+0000 7fbf15ffb640 1 -- 172.21.15.97:0/2762935645 <== mgr.14152 v2:172.21.15.97:6800/3149043749 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fbf18062390 con 0x7fbf080a3bb0 2024-04-23T10:24:13.372 INFO:teuthology.orchestra.run.smithi097.stdout:/volumes/qa/sv_1/e80e6226-5911-43bb-a37a-f08d55e0fed9 2024-04-23T10:24:13.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.375+0000 7fbf1f260640 1 -- 172.21.15.97:0/2762935645 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fbf080a3bb0 msgr2=0x7fbf080a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:13.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.375+0000 7fbf1f260640 1 --2- 172.21.15.97:0/2762935645 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fbf080a3bb0 0x7fbf080a6070 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c009230 tx=0x7fbf0c000bc0 comp rx=0 tx=0).stop 2024-04-23T10:24:13.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.375+0000 7fbf1f260640 1 -- 172.21.15.97:0/2762935645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf18103190 msgr2=0x7fbf181986b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:13.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.375+0000 7fbf1f260640 1 --2- 172.21.15.97:0/2762935645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf18103190 0x7fbf181986b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fbf0400c920 tx=0x7fbf0400cde0 comp rx=0 tx=0).stop 2024-04-23T10:24:13.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.375+0000 7fbf1f260640 1 -- 172.21.15.97:0/2762935645 shutdown_connections 2024-04-23T10:24:13.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.375+0000 7fbf1f260640 1 --2- 172.21.15.97:0/2762935645 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fbf080a3bb0 0x7fbf080a6070 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:13.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.375+0000 7fbf1f260640 1 --2- 172.21.15.97:0/2762935645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf18103b50 0x7fbf1819f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:13.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.375+0000 7fbf1f260640 1 --2- 172.21.15.97:0/2762935645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf18103190 0x7fbf181986b0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:13.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.375+0000 7fbf1f260640 1 --2- 172.21.15.97:0/2762935645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf18101fa0 0x7fbf18198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:13.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.375+0000 7fbf1f260640 1 -- 172.21.15.97:0/2762935645 >> 172.21.15.97:0/2762935645 conn(0x7fbf180fd530 msgr2=0x7fbf18106510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:13.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.375+0000 7fbf1f260640 1 -- 172.21.15.97:0/2762935645 shutdown_connections 2024-04-23T10:24:13.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.375+0000 7fbf1f260640 1 -- 172.21.15.97:0/2762935645 wait complete. 2024-04-23T10:24:13.385 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T10:24:13.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.484+0000 7fc48aa91640 1 -- 172.21.15.97:0/3855651047 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc484101ba0 msgr2=0x7fc484102020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:13.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.484+0000 7fc48aa91640 1 --2- 172.21.15.97:0/3855651047 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc484101ba0 0x7fc484102020 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc47c009ca0 tx=0x7fc47c02a950 comp rx=0 tx=0).stop 2024-04-23T10:24:13.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.484+0000 7fc48aa91640 1 -- 172.21.15.97:0/3855651047 shutdown_connections 2024-04-23T10:24:13.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.484+0000 7fc48aa91640 1 --2- 172.21.15.97:0/3855651047 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc484101ba0 0x7fc484102020 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:13.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.484+0000 7fc48aa91640 1 --2- 172.21.15.97:0/3855651047 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc484101200 0x7fc484101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:13.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.484+0000 7fc48aa91640 1 --2- 172.21.15.97:0/3855651047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc4841008a0 0x7fc484100cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:13.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.484+0000 7fc48aa91640 1 -- 172.21.15.97:0/3855651047 >> 172.21.15.97:0/3855651047 conn(0x7fc4840fc440 msgr2=0x7fc4840fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:13.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.484+0000 7fc48aa91640 1 -- 172.21.15.97:0/3855651047 shutdown_connections 2024-04-23T10:24:13.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.484+0000 7fc48aa91640 1 -- 172.21.15.97:0/3855651047 wait complete. 2024-04-23T10:24:13.485 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.484+0000 7fc48aa91640 1 Processor -- start 2024-04-23T10:24:13.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.485+0000 7fc48aa91640 1 -- start start 2024-04-23T10:24:13.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.485+0000 7fc48aa91640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4841008a0 0x7fc48419a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:13.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.485+0000 7fc48aa91640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc484101200 0x7fc48419a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:13.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.485+0000 7fc48aa91640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc484101ba0 0x7fc4841a1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:13.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.485+0000 7fc48aa91640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc48410fcf0 con 0x7fc4841008a0 2024-04-23T10:24:13.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.485+0000 7fc48aa91640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc48410fb70 con 0x7fc484101ba0 2024-04-23T10:24:13.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.485+0000 7fc48aa91640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc48410fe70 con 0x7fc484101200 2024-04-23T10:24:13.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.485+0000 7fc48882d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4841008a0 0x7fc48419a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:13.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.485+0000 7fc48882d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4841008a0 0x7fc48419a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33052/0 (socket says 172.21.15.97:33052) 2024-04-23T10:24:13.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.485+0000 7fc48882d640 1 -- 172.21.15.97:0/705220008 learned_addr learned my addr 172.21.15.97:0/705220008 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:24:13.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.485+0000 7fc48902e640 1 --2- 172.21.15.97:0/705220008 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc484101ba0 0x7fc4841a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:13.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.485+0000 7fc483fff640 1 --2- 172.21.15.97:0/705220008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc484101200 0x7fc48419a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:13.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.485+0000 7fc48882d640 1 -- 172.21.15.97:0/705220008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc484101200 msgr2=0x7fc48419a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:13.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.485+0000 7fc48882d640 1 --2- 172.21.15.97:0/705220008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc484101200 0x7fc48419a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:13.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.485+0000 7fc48882d640 1 -- 172.21.15.97:0/705220008 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc484101ba0 msgr2=0x7fc4841a1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:13.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.485+0000 7fc48882d640 1 --2- 172.21.15.97:0/705220008 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc484101ba0 0x7fc4841a1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:13.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.485+0000 7fc48882d640 1 -- 172.21.15.97:0/705220008 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4841a1f90 con 0x7fc4841008a0 2024-04-23T10:24:13.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.485+0000 7fc483fff640 1 --2- 172.21.15.97:0/705220008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc484101200 0x7fc48419a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:24:13.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.485+0000 7fc48882d640 1 --2- 172.21.15.97:0/705220008 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4841008a0 0x7fc48419a370 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc47800d770 tx=0x7fc47800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:13.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.485+0000 7fc481ffb640 1 -- 172.21.15.97:0/705220008 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc478017070 con 0x7fc4841008a0 2024-04-23T10:24:13.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.485+0000 7fc481ffb640 1 -- 172.21.15.97:0/705220008 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4780054d0 con 0x7fc4841008a0 2024-04-23T10:24:13.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.485+0000 7fc48aa91640 1 -- 172.21.15.97:0/705220008 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4841a2280 con 0x7fc4841008a0 2024-04-23T10:24:13.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.485+0000 7fc481ffb640 1 -- 172.21.15.97:0/705220008 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc478004c30 con 0x7fc4841008a0 2024-04-23T10:24:13.487 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.486+0000 7fc48aa91640 1 -- 172.21.15.97:0/705220008 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4841a2780 con 0x7fc4841008a0 2024-04-23T10:24:13.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.486+0000 7fc48aa91640 1 -- 172.21.15.97:0/705220008 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc48410b5f0 con 0x7fc4841008a0 2024-04-23T10:24:13.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.487+0000 7fc481ffb640 1 -- 172.21.15.97:0/705220008 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc478004e10 con 0x7fc4841008a0 2024-04-23T10:24:13.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.488+0000 7fc481ffb640 1 --2- 172.21.15.97:0/705220008 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fc4740a3bb0 0x7fc4740a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:13.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.488+0000 7fc483fff640 1 --2- 172.21.15.97:0/705220008 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fc4740a3bb0 0x7fc4740a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:13.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.488+0000 7fc481ffb640 1 -- 172.21.15.97:0/705220008 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fc4780cfd70 con 0x7fc4841008a0 2024-04-23T10:24:13.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.489+0000 7fc483fff640 1 --2- 172.21.15.97:0/705220008 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fc4740a3bb0 0x7fc4740a6070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc48419b880 tx=0x7fc470005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:13.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.493+0000 7fc481ffb640 1 -- 172.21.15.97:0/705220008 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc47809c8a0 con 0x7fc4841008a0 2024-04-23T10:24:13.710 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.711+0000 7fc48aa91640 1 -- 172.21.15.97:0/705220008 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc484062b50 con 0x7fc4841008a0 2024-04-23T10:24:13.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.712+0000 7fc481ffb640 1 -- 172.21.15.97:0/705220008 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11931 (secure 0 0 0) 0x7fc47809c1a0 con 0x7fc4841008a0 2024-04-23T10:24:13.711 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:24:13.711 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3946878762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3946878762},{"type":"v1","addr":"172.21.15.97:6835","nonce":3946878762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/209800861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":209800861},{"type":"v1","addr":"172.21.15.97:6837","nonce":209800861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T10:24:06.910134+0000","modified":"2024-04-23T10:24:12.969478+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6839/418013575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":418013575},{"type":"v1","addr":"172.21.15.97:6839","nonce":418013575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6841/747242680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":747242680},{"type":"v1","addr":"172.21.15.97:6841","nonce":747242680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6833/4292847121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4292847121},{"type":"v1","addr":"172.21.15.183:6833","nonce":4292847121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6833/3885816565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3885816565},{"type":"v1","addr":"172.21.15.196:6833","nonce":3885816565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1897093105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1897093105},{"type":"v1","addr":"172.21.15.183:6835","nonce":1897093105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1171477563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1171477563},{"type":"v1","addr":"172.21.15.196:6835","nonce":1171477563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1898341911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1898341911},{"type":"v1","addr":"172.21.15.183:6837","nonce":1898341911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.196:6837/3094946502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3094946502},{"type":"v1","addr":"172.21.15.196:6837","nonce":3094946502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.196:6839/83843556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":83843556},{"type":"v1","addr":"172.21.15.196:6839","nonce":83843556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6839/1763841130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1763841130},{"type":"v1","addr":"172.21.15.183:6839","nonce":1763841130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24469,24488,24398,24403,24487]},"id":1}]} 2024-04-23T10:24:13.711 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 25 2024-04-23T10:24:13.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.714+0000 7fc48aa91640 1 -- 172.21.15.97:0/705220008 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fc4740a3bb0 msgr2=0x7fc4740a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:13.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.714+0000 7fc48aa91640 1 --2- 172.21.15.97:0/705220008 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fc4740a3bb0 0x7fc4740a6070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc48419b880 tx=0x7fc470005f90 comp rx=0 tx=0).stop 2024-04-23T10:24:13.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.714+0000 7fc48aa91640 1 -- 172.21.15.97:0/705220008 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4841008a0 msgr2=0x7fc48419a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:13.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.714+0000 7fc48aa91640 1 --2- 172.21.15.97:0/705220008 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4841008a0 0x7fc48419a370 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc47800d770 tx=0x7fc47800dc30 comp rx=0 tx=0).stop 2024-04-23T10:24:13.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.714+0000 7fc48aa91640 1 -- 172.21.15.97:0/705220008 shutdown_connections 2024-04-23T10:24:13.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.714+0000 7fc48aa91640 1 --2- 172.21.15.97:0/705220008 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fc4740a3bb0 0x7fc4740a6070 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:13.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.714+0000 7fc48aa91640 1 --2- 172.21.15.97:0/705220008 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc484101ba0 0x7fc4841a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:13.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.714+0000 7fc48aa91640 1 --2- 172.21.15.97:0/705220008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc484101200 0x7fc48419a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:13.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.714+0000 7fc48aa91640 1 --2- 172.21.15.97:0/705220008 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4841008a0 0x7fc48419a370 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:13.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.714+0000 7fc48aa91640 1 -- 172.21.15.97:0/705220008 >> 172.21.15.97:0/705220008 conn(0x7fc4840fc440 msgr2=0x7fc4840fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:13.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.714+0000 7fc48aa91640 1 -- 172.21.15.97:0/705220008 shutdown_connections 2024-04-23T10:24:13.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.714+0000 7fc48aa91640 1 -- 172.21.15.97:0/705220008 wait complete. 2024-04-23T10:24:13.723 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:13 smithi097 ceph-mon[29197]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:24:13.723 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:13 smithi097 ceph-mon[29197]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:24:13.723 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:13 smithi097 ceph-mon[29197]: mds.? [v2:172.21.15.183:6832/4292847121,v1:172.21.15.183:6833/4292847121] up:active 2024-04-23T10:24:13.723 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:13 smithi097 ceph-mon[29197]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-04-23T10:24:13.723 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:13 smithi097 ceph-mon[29197]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-23T10:24:13.724 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T10:24:13.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.822+0000 7fe060a91640 1 -- 172.21.15.97:0/3956142320 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe05c101800 msgr2=0x7fe05c101c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:13.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.822+0000 7fe060a91640 1 --2- 172.21.15.97:0/3956142320 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe05c101800 0x7fe05c101c80 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe050009900 tx=0x7fe05002a410 comp rx=0 tx=0).stop 2024-04-23T10:24:13.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.822+0000 7fe060a91640 1 -- 172.21.15.97:0/3956142320 shutdown_connections 2024-04-23T10:24:13.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.822+0000 7fe060a91640 1 --2- 172.21.15.97:0/3956142320 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe05c1021c0 0x7fe05c108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:13.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.822+0000 7fe060a91640 1 --2- 172.21.15.97:0/3956142320 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe05c101800 0x7fe05c101c80 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:13.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.822+0000 7fe060a91640 1 --2- 172.21.15.97:0/3956142320 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe05c100610 0x7fe05c100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:13.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.822+0000 7fe060a91640 1 -- 172.21.15.97:0/3956142320 >> 172.21.15.97:0/3956142320 conn(0x7fe05c0fbb80 msgr2=0x7fe05c0fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:13.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.822+0000 7fe060a91640 1 -- 172.21.15.97:0/3956142320 shutdown_connections 2024-04-23T10:24:13.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.822+0000 7fe060a91640 1 -- 172.21.15.97:0/3956142320 wait complete. 2024-04-23T10:24:13.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.823+0000 7fe060a91640 1 Processor -- start 2024-04-23T10:24:13.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.823+0000 7fe060a91640 1 -- start start 2024-04-23T10:24:13.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.823+0000 7fe060a91640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe05c100610 0x7fe05c198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:13.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.823+0000 7fe060a91640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe05c101800 0x7fe05c198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:13.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.823+0000 7fe060a91640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe05c1021c0 0x7fe05c19f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:13.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.823+0000 7fe060a91640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe05c10db40 con 0x7fe05c1021c0 2024-04-23T10:24:13.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.823+0000 7fe060a91640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe05c10d9c0 con 0x7fe05c100610 2024-04-23T10:24:13.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.823+0000 7fe060a91640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe05c10dcc0 con 0x7fe05c101800 2024-04-23T10:24:13.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.823+0000 7fe05ad9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe05c1021c0 0x7fe05c19f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:13.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.823+0000 7fe05ad9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe05c1021c0 0x7fe05c19f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33062/0 (socket says 172.21.15.97:33062) 2024-04-23T10:24:13.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.823+0000 7fe05ad9d640 1 -- 172.21.15.97:0/2712961394 learned_addr learned my addr 172.21.15.97:0/2712961394 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:24:13.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.823+0000 7fe05a59c640 1 --2- 172.21.15.97:0/2712961394 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe05c100610 0x7fe05c198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:13.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.824+0000 7fe059d9b640 1 --2- 172.21.15.97:0/2712961394 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe05c101800 0x7fe05c198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:13.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.824+0000 7fe05ad9d640 1 -- 172.21.15.97:0/2712961394 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe05c101800 msgr2=0x7fe05c198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:13.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.824+0000 7fe05ad9d640 1 --2- 172.21.15.97:0/2712961394 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe05c101800 0x7fe05c198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:13.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.824+0000 7fe05ad9d640 1 -- 172.21.15.97:0/2712961394 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe05c100610 msgr2=0x7fe05c198150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:13.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.824+0000 7fe05ad9d640 1 --2- 172.21.15.97:0/2712961394 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe05c100610 0x7fe05c198150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:13.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.824+0000 7fe05ad9d640 1 -- 172.21.15.97:0/2712961394 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe05c19fd70 con 0x7fe05c1021c0 2024-04-23T10:24:13.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.824+0000 7fe05ad9d640 1 --2- 172.21.15.97:0/2712961394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe05c1021c0 0x7fe05c19f700 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe05400eb10 tx=0x7fe05400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:13.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.824+0000 7fe04b7fe640 1 -- 172.21.15.97:0/2712961394 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe054004b20 con 0x7fe05c1021c0 2024-04-23T10:24:13.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.824+0000 7fe060a91640 1 -- 172.21.15.97:0/2712961394 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe05c1a0060 con 0x7fe05c1021c0 2024-04-23T10:24:13.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.824+0000 7fe04b7fe640 1 -- 172.21.15.97:0/2712961394 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe054004cc0 con 0x7fe05c1021c0 2024-04-23T10:24:13.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.824+0000 7fe04b7fe640 1 -- 172.21.15.97:0/2712961394 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe054010670 con 0x7fe05c1021c0 2024-04-23T10:24:13.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.824+0000 7fe060a91640 1 -- 172.21.15.97:0/2712961394 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe05c1a0530 con 0x7fe05c1021c0 2024-04-23T10:24:13.826 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:13 smithi183 ceph-mon[32317]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:24:13.826 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:13 smithi183 ceph-mon[32317]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:24:13.826 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:13 smithi183 ceph-mon[32317]: mds.? [v2:172.21.15.183:6832/4292847121,v1:172.21.15.183:6833/4292847121] up:active 2024-04-23T10:24:13.826 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:13 smithi183 ceph-mon[32317]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-04-23T10:24:13.826 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:13 smithi183 ceph-mon[32317]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-23T10:24:13.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.825+0000 7fe060a91640 1 -- 172.21.15.97:0/2712961394 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe05c107930 con 0x7fe05c1021c0 2024-04-23T10:24:13.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.826+0000 7fe04b7fe640 1 -- 172.21.15.97:0/2712961394 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe054005020 con 0x7fe05c1021c0 2024-04-23T10:24:13.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.827+0000 7fe04b7fe640 1 --2- 172.21.15.97:0/2712961394 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fe0440a3bb0 0x7fe0440a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:13.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.827+0000 7fe05a59c640 1 --2- 172.21.15.97:0/2712961394 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fe0440a3bb0 0x7fe0440a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:13.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.827+0000 7fe04b7fe640 1 -- 172.21.15.97:0/2712961394 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fe0540d0910 con 0x7fe05c1021c0 2024-04-23T10:24:13.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.827+0000 7fe05a59c640 1 --2- 172.21.15.97:0/2712961394 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fe0440a3bb0 0x7fe0440a6070 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe05c101660 tx=0x7fe04c00a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:13.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:13.831+0000 7fe04b7fe640 1 -- 172.21.15.97:0/2712961394 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe05409d440 con 0x7fe05c1021c0 2024-04-23T10:24:13.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:13 smithi196 ceph-mon[32919]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:24:13.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:13 smithi196 ceph-mon[32919]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:24:13.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:13 smithi196 ceph-mon[32919]: mds.? [v2:172.21.15.183:6832/4292847121,v1:172.21.15.183:6833/4292847121] up:active 2024-04-23T10:24:13.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:13 smithi196 ceph-mon[32919]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-04-23T10:24:13.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:13 smithi196 ceph-mon[32919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-23T10:24:14.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.013+0000 7fe060a91640 1 -- 172.21.15.97:0/2712961394 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe05c062b50 con 0x7fe05c1021c0 2024-04-23T10:24:14.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.014+0000 7fe04b7fe640 1 -- 172.21.15.97:0/2712961394 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22220 (secure 0 0 0) 0x7fe05409cd40 con 0x7fe05c1021c0 2024-04-23T10:24:14.013 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:24:14.014 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":86,"fsid":"9b22af88-0159-11ef-bc93-c7b262605968","created":"2024-04-23T10:16:05.983305+0000","modified":"2024-04-23T10:24:08.021502+0000","last_up_change":"2024-04-23T10:23:13.114761+0000","last_in_change":"2024-04-23T10:22:52.804880+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T10:19:25.559138+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T10:24:04.570049+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T10:24:05.861631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"58cb995d-de35-4522-ac81-4982401b260d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6803","nonce":3031221802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6805","nonce":3031221802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6809","nonce":3031221802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6807","nonce":3031221802}]},"public_addr":"172.21.15.97:6803/3031221802","cluster_addr":"172.21.15.97:6805/3031221802","heartbeat_back_addr":"172.21.15.97:6809/3031221802","heartbeat_front_addr":"172.21.15.97:6807/3031221802","state":["exists","up"]},{"osd":1,"uuid":"b47bd8ee-1f21-4834-acf5-bb40f97ab3ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6801","nonce":1108870440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6803","nonce":1108870440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6807","nonce":1108870440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6805","nonce":1108870440}]},"public_addr":"172.21.15.183:6801/1108870440","cluster_addr":"172.21.15.183:6803/1108870440","heartbeat_back_addr":"172.21.15.183:6807/1108870440","heartbeat_front_addr":"172.21.15.183:6805/1108870440","state":["exists","up"]},{"osd":2,"uuid":"17c4eec3-6c47-4334-9b63-1d5ad9ec5988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6801","nonce":2877039731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6803","nonce":2877039731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6807","nonce":2877039731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6805","nonce":2877039731}]},"public_addr":"172.21.15.196:6801/2877039731","cluster_addr":"172.21.15.196:6803/2877039731","heartbeat_back_addr":"172.21.15.196:6807/2877039731","heartbeat_front_addr":"172.21.15.196:6805/2877039731","state":["exists","up"]},{"osd":3,"uuid":"3f731c38-2e45-4ca1-8dce-efc4a1b2166b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6811","nonce":4276311194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6813","nonce":4276311194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6817","nonce":4276311194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6815","nonce":4276311194}]},"public_addr":"172.21.15.97:6811/4276311194","cluster_addr":"172.21.15.97:6813/4276311194","heartbeat_back_addr":"172.21.15.97:6817/4276311194","heartbeat_front_addr":"172.21.15.97:6815/4276311194","state":["exists","up"]},{"osd":4,"uuid":"fdcafb14-9b82-4134-bd5b-79c0516f6cbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6809","nonce":63913916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6811","nonce":63913916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6815","nonce":63913916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6813","nonce":63913916}]},"public_addr":"172.21.15.183:6809/63913916","cluster_addr":"172.21.15.183:6811/63913916","heartbeat_back_addr":"172.21.15.183:6815/63913916","heartbeat_front_addr":"172.21.15.183:6813/63913916","state":["exists","up"]},{"osd":5,"uuid":"cdcc18d8-da3c-4012-bf12-876bb39e66eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6809","nonce":2804446896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6811","nonce":2804446896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6815","nonce":2804446896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6813","nonce":2804446896}]},"public_addr":"172.21.15.196:6809/2804446896","cluster_addr":"172.21.15.196:6811/2804446896","heartbeat_back_addr":"172.21.15.196:6815/2804446896","heartbeat_front_addr":"172.21.15.196:6813/2804446896","state":["exists","up"]},{"osd":6,"uuid":"071abb5c-2edf-4832-b8aa-3b80d20096fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6819","nonce":2080916705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6821","nonce":2080916705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6825","nonce":2080916705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6823","nonce":2080916705}]},"public_addr":"172.21.15.97:6819/2080916705","cluster_addr":"172.21.15.97:6821/2080916705","heartbeat_back_addr":"172.21.15.97:6825/2080916705","heartbeat_front_addr":"172.21.15.97:6823/2080916705","state":["exists","up"]},{"osd":7,"uuid":"9d54760b-66df-486b-8abb-017f3b64b4d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6817","nonce":1865873033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6819","nonce":1865873033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6823","nonce":1865873033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6821","nonce":1865873033}]},"public_addr":"172.21.15.183:6817/1865873033","cluster_addr":"172.21.15.183:6819/1865873033","heartbeat_back_addr":"172.21.15.183:6823/1865873033","heartbeat_front_addr":"172.21.15.183:6821/1865873033","state":["exists","up"]},{"osd":8,"uuid":"5cd3100a-bd6a-4be5-b5e9-0d0afc325193","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6817","nonce":4063932139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6819","nonce":4063932139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6823","nonce":4063932139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6821","nonce":4063932139}]},"public_addr":"172.21.15.196:6817/4063932139","cluster_addr":"172.21.15.196:6819/4063932139","heartbeat_back_addr":"172.21.15.196:6823/4063932139","heartbeat_front_addr":"172.21.15.196:6821/4063932139","state":["exists","up"]},{"osd":9,"uuid":"b6a5f4a1-4ecc-4f73-a2df-07551d9298ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6827","nonce":1887432474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6829","nonce":1887432474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6833","nonce":1887432474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6831","nonce":1887432474}]},"public_addr":"172.21.15.97:6827/1887432474","cluster_addr":"172.21.15.97:6829/1887432474","heartbeat_back_addr":"172.21.15.97:6833/1887432474","heartbeat_front_addr":"172.21.15.97:6831/1887432474","state":["exists","up"]},{"osd":10,"uuid":"86f59816-1ea6-4ae9-8796-cf5fc4831064","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6825","nonce":799434647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6827","nonce":799434647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6831","nonce":799434647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6829","nonce":799434647}]},"public_addr":"172.21.15.183:6825/799434647","cluster_addr":"172.21.15.183:6827/799434647","heartbeat_back_addr":"172.21.15.183:6831/799434647","heartbeat_front_addr":"172.21.15.183:6829/799434647","state":["exists","up"]},{"osd":11,"uuid":"95d64a89-a9f8-4c7a-8c0d-da7fb26cfff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6825","nonce":1873969467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6827","nonce":1873969467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6831","nonce":1873969467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6829","nonce":1873969467}]},"public_addr":"172.21.15.196:6825/1873969467","cluster_addr":"172.21.15.196:6827/1873969467","heartbeat_back_addr":"172.21.15.196:6831/1873969467","heartbeat_front_addr":"172.21.15.196:6829/1873969467","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:18:33.721156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:18:57.586709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:19:22.069458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:19:46.451597+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:20:11.561019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:20:36.298672+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:21:01.661804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:21:27.324128+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:21:52.898197+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:22:19.614326+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:22:45.154917+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:23:11.222612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/3866016040":"2024-04-24T10:16:59.468237+0000","172.21.15.97:6801/3297166858":"2024-04-24T10:16:59.468237+0000","172.21.15.97:0/3804743548":"2024-04-24T10:16:59.468237+0000","172.21.15.97:6800/3706463017":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/3715079438":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/786806834":"2024-04-24T10:16:59.468237+0000","172.21.15.97:6801/3706463017":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/3294175554":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/1449136760":"2024-04-24T10:16:33.128917+0000","172.21.15.97:6800/3297166858":"2024-04-24T10:16:59.468237+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T10:24:14.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.016+0000 7fe060a91640 1 -- 172.21.15.97:0/2712961394 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fe0440a3bb0 msgr2=0x7fe0440a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:14.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.016+0000 7fe060a91640 1 --2- 172.21.15.97:0/2712961394 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fe0440a3bb0 0x7fe0440a6070 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe05c101660 tx=0x7fe04c00a2d0 comp rx=0 tx=0).stop 2024-04-23T10:24:14.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.016+0000 7fe060a91640 1 -- 172.21.15.97:0/2712961394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe05c1021c0 msgr2=0x7fe05c19f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:14.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.016+0000 7fe060a91640 1 --2- 172.21.15.97:0/2712961394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe05c1021c0 0x7fe05c19f700 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe05400eb10 tx=0x7fe05400c2d0 comp rx=0 tx=0).stop 2024-04-23T10:24:14.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.016+0000 7fe060a91640 1 -- 172.21.15.97:0/2712961394 shutdown_connections 2024-04-23T10:24:14.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.016+0000 7fe060a91640 1 --2- 172.21.15.97:0/2712961394 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fe0440a3bb0 0x7fe0440a6070 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:14.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.016+0000 7fe060a91640 1 --2- 172.21.15.97:0/2712961394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe05c1021c0 0x7fe05c19f700 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:14.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.016+0000 7fe060a91640 1 --2- 172.21.15.97:0/2712961394 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe05c101800 0x7fe05c198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:14.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.016+0000 7fe060a91640 1 --2- 172.21.15.97:0/2712961394 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe05c100610 0x7fe05c198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:14.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.016+0000 7fe060a91640 1 -- 172.21.15.97:0/2712961394 >> 172.21.15.97:0/2712961394 conn(0x7fe05c0fbb80 msgr2=0x7fe05c104b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:14.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.016+0000 7fe060a91640 1 -- 172.21.15.97:0/2712961394 shutdown_connections 2024-04-23T10:24:14.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.016+0000 7fe060a91640 1 -- 172.21.15.97:0/2712961394 wait complete. 2024-04-23T10:24:14.027 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-23T10:24:14.027 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T10:24:14.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.127+0000 7ff152491640 1 -- 172.21.15.97:0/796878330 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff14c1029b0 msgr2=0x7ff14c1092d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:14.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.127+0000 7ff152491640 1 --2- 172.21.15.97:0/796878330 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff14c1029b0 0x7ff14c1092d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff144009ca0 tx=0x7ff14402a8c0 comp rx=0 tx=0).stop 2024-04-23T10:24:14.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.127+0000 7ff152491640 1 -- 172.21.15.97:0/796878330 shutdown_connections 2024-04-23T10:24:14.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.127+0000 7ff152491640 1 --2- 172.21.15.97:0/796878330 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff14c1029b0 0x7ff14c1092d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:14.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.127+0000 7ff152491640 1 --2- 172.21.15.97:0/796878330 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff14c101ff0 0x7ff14c102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:14.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.127+0000 7ff152491640 1 --2- 172.21.15.97:0/796878330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff14c100ea0 0x7ff14c1012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:14.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.127+0000 7ff152491640 1 -- 172.21.15.97:0/796878330 >> 172.21.15.97:0/796878330 conn(0x7ff14c0fc430 msgr2=0x7ff14c0fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:14.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.127+0000 7ff152491640 1 -- 172.21.15.97:0/796878330 shutdown_connections 2024-04-23T10:24:14.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.127+0000 7ff152491640 1 -- 172.21.15.97:0/796878330 wait complete. 2024-04-23T10:24:14.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.127+0000 7ff152491640 1 Processor -- start 2024-04-23T10:24:14.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.127+0000 7ff152491640 1 -- start start 2024-04-23T10:24:14.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.128+0000 7ff152491640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff14c100ea0 0x7ff14c06b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:14.128 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.128+0000 7ff152491640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff14c101ff0 0x7ff14c19cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:14.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.128+0000 7ff152491640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff14c1029b0 0x7ff14c19f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:14.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.128+0000 7ff152491640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff14c10ca00 con 0x7ff14c1029b0 2024-04-23T10:24:14.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.128+0000 7ff152491640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff14c10c880 con 0x7ff14c100ea0 2024-04-23T10:24:14.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.128+0000 7ff152491640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff14c10cb80 con 0x7ff14c101ff0 2024-04-23T10:24:14.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.128+0000 7ff150a2e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff14c1029b0 0x7ff14c19f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:14.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.128+0000 7ff150a2e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff14c1029b0 0x7ff14c19f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33072/0 (socket says 172.21.15.97:33072) 2024-04-23T10:24:14.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.128+0000 7ff150a2e640 1 -- 172.21.15.97:0/1002919591 learned_addr learned my addr 172.21.15.97:0/1002919591 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:24:14.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.128+0000 7ff14b7fe640 1 --2- 172.21.15.97:0/1002919591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff14c101ff0 0x7ff14c19cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:14.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.128+0000 7ff14bfff640 1 --2- 172.21.15.97:0/1002919591 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff14c100ea0 0x7ff14c06b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:14.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.128+0000 7ff150a2e640 1 -- 172.21.15.97:0/1002919591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff14c101ff0 msgr2=0x7ff14c19cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:14.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.128+0000 7ff150a2e640 1 --2- 172.21.15.97:0/1002919591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff14c101ff0 0x7ff14c19cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:14.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.128+0000 7ff150a2e640 1 -- 172.21.15.97:0/1002919591 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff14c100ea0 msgr2=0x7ff14c06b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:14.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.128+0000 7ff150a2e640 1 --2- 172.21.15.97:0/1002919591 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff14c100ea0 0x7ff14c06b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:14.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.128+0000 7ff150a2e640 1 -- 172.21.15.97:0/1002919591 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff14c19fe30 con 0x7ff14c1029b0 2024-04-23T10:24:14.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.128+0000 7ff150a2e640 1 --2- 172.21.15.97:0/1002919591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff14c1029b0 0x7ff14c19f520 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff1440097b0 tx=0x7ff144003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:14.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.128+0000 7ff1497fa640 1 -- 172.21.15.97:0/1002919591 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff144004240 con 0x7ff14c1029b0 2024-04-23T10:24:14.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.128+0000 7ff1497fa640 1 -- 172.21.15.97:0/1002919591 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1440317c0 con 0x7ff14c1029b0 2024-04-23T10:24:14.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.128+0000 7ff152491640 1 -- 172.21.15.97:0/1002919591 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff14c1a0060 con 0x7ff14c1029b0 2024-04-23T10:24:14.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.128+0000 7ff1497fa640 1 -- 172.21.15.97:0/1002919591 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff14402fd80 con 0x7ff14c1029b0 2024-04-23T10:24:14.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.128+0000 7ff152491640 1 -- 172.21.15.97:0/1002919591 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff14c1a05b0 con 0x7ff14c1029b0 2024-04-23T10:24:14.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.129+0000 7ff152491640 1 -- 172.21.15.97:0/1002919591 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff14c108280 con 0x7ff14c1029b0 2024-04-23T10:24:14.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.130+0000 7ff1497fa640 1 -- 172.21.15.97:0/1002919591 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff1440043e0 con 0x7ff14c1029b0 2024-04-23T10:24:14.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.131+0000 7ff1497fa640 1 --2- 172.21.15.97:0/1002919591 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7ff1340a3bb0 0x7ff1340a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:14.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.131+0000 7ff14bfff640 1 --2- 172.21.15.97:0/1002919591 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7ff1340a3bb0 0x7ff1340a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:14.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.131+0000 7ff1497fa640 1 -- 172.21.15.97:0/1002919591 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7ff1440f1a90 con 0x7ff14c1029b0 2024-04-23T10:24:14.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.131+0000 7ff14bfff640 1 --2- 172.21.15.97:0/1002919591 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7ff1340a3bb0 0x7ff1340a6070 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff13c0058c0 tx=0x7ff13c00a250 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:14.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.136+0000 7ff1497fa640 1 -- 172.21.15.97:0/1002919591 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff1440be5c0 con 0x7ff14c1029b0 2024-04-23T10:24:14.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.310+0000 7ff152491640 1 -- 172.21.15.97:0/1002919591 --> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff14c062390 con 0x7ff1340a3bb0 2024-04-23T10:24:14.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.317+0000 7ff1497fa640 1 -- 172.21.15.97:0/1002919591 <== mgr.14152 v2:172.21.15.97:6800/3149043749 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258901 (secure 0 0 0) 0x7ff14c062390 con 0x7ff1340a3bb0 2024-04-23T10:24:14.320 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:24:14.321 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-04-23T10:24:14.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.321+0000 7ff152491640 1 -- 172.21.15.97:0/1002919591 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7ff1340a3bb0 msgr2=0x7ff1340a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:14.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.321+0000 7ff152491640 1 --2- 172.21.15.97:0/1002919591 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7ff1340a3bb0 0x7ff1340a6070 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff13c0058c0 tx=0x7ff13c00a250 comp rx=0 tx=0).stop 2024-04-23T10:24:14.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.321+0000 7ff152491640 1 -- 172.21.15.97:0/1002919591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff14c1029b0 msgr2=0x7ff14c19f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:14.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.321+0000 7ff152491640 1 --2- 172.21.15.97:0/1002919591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff14c1029b0 0x7ff14c19f520 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff1440097b0 tx=0x7ff144003c20 comp rx=0 tx=0).stop 2024-04-23T10:24:14.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.321+0000 7ff152491640 1 -- 172.21.15.97:0/1002919591 shutdown_connections 2024-04-23T10:24:14.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.321+0000 7ff152491640 1 --2- 172.21.15.97:0/1002919591 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7ff1340a3bb0 0x7ff1340a6070 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:14.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.321+0000 7ff152491640 1 --2- 172.21.15.97:0/1002919591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff14c1029b0 0x7ff14c19f520 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:14.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.321+0000 7ff152491640 1 --2- 172.21.15.97:0/1002919591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff14c101ff0 0x7ff14c19cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:14.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.321+0000 7ff152491640 1 --2- 172.21.15.97:0/1002919591 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff14c100ea0 0x7ff14c06b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:14.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.321+0000 7ff152491640 1 -- 172.21.15.97:0/1002919591 >> 172.21.15.97:0/1002919591 conn(0x7ff14c0fc430 msgr2=0x7ff14c105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:14.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.321+0000 7ff152491640 1 -- 172.21.15.97:0/1002919591 shutdown_connections 2024-04-23T10:24:14.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.321+0000 7ff152491640 1 -- 172.21.15.97:0/1002919591 wait complete. 2024-04-23T10:24:14.331 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2024-04-23T10:24:13.554081+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":593070,"num_objects":44,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":142,"num_read_kb":124,"num_write":280,"num_write_kb":2957,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":218,"ondisk_log_size":218,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":221,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360308,"kb_used_data":11100,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581516,"statfs":{"total":1151940427776,"available":1151571472384,"internally_reserved":0,"allocated":11366400,"data_stored":7063924,"data_compressed":106845,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2702,"num_objects":42,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":99,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.000127"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T10:24:07.041388+0000","last_change":"2024-04-23T10:24:07.040397+0000","last_active":"2024-04-23T10:24:07.041388+0000","last_peered":"2024-04-23T10:24:07.041388+0000","last_clean":"2024-04-23T10:24:07.041388+0000","last_became_active":"2024-04-23T10:24:07.040007+0000","last_became_peered":"2024-04-23T10:24:07.040007+0000","last_unstale":"2024-04-23T10:24:07.041388+0000","last_undegraded":"2024-04-23T10:24:07.041388+0000","last_fullsized":"2024-04-23T10:24:07.041388+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:34:16.967983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:09.418511+0000","last_change":"2024-04-23T10:24:07.044672+0000","last_active":"2024-04-23T10:24:09.418511+0000","last_peered":"2024-04-23T10:24:09.418511+0000","last_clean":"2024-04-23T10:24:09.418511+0000","last_became_active":"2024-04-23T10:24:07.044184+0000","last_became_peered":"2024-04-23T10:24:07.044184+0000","last_unstale":"2024-04-23T10:24:09.418511+0000","last_undegraded":"2024-04-23T10:24:09.418511+0000","last_fullsized":"2024-04-23T10:24:09.418511+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:07:16.729205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:09.418411+0000","last_change":"2024-04-23T10:24:07.041959+0000","last_active":"2024-04-23T10:24:09.418411+0000","last_peered":"2024-04-23T10:24:09.418411+0000","last_clean":"2024-04-23T10:24:09.418411+0000","last_became_active":"2024-04-23T10:24:07.041661+0000","last_became_peered":"2024-04-23T10:24:07.041661+0000","last_unstale":"2024-04-23T10:24:09.418411+0000","last_undegraded":"2024-04-23T10:24:09.418411+0000","last_fullsized":"2024-04-23T10:24:09.418411+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:23:14.694183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.030650+0000","last_change":"2024-04-23T10:24:07.048334+0000","last_active":"2024-04-23T10:24:08.030650+0000","last_peered":"2024-04-23T10:24:08.030650+0000","last_clean":"2024-04-23T10:24:08.030650+0000","last_became_active":"2024-04-23T10:24:07.047506+0000","last_became_peered":"2024-04-23T10:24:07.047506+0000","last_unstale":"2024-04-23T10:24:08.030650+0000","last_undegraded":"2024-04-23T10:24:08.030650+0000","last_fullsized":"2024-04-23T10:24:08.030650+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:27:27.674287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.561291+0000","last_change":"2024-04-23T10:24:07.035531+0000","last_active":"2024-04-23T10:24:08.561291+0000","last_peered":"2024-04-23T10:24:08.561291+0000","last_clean":"2024-04-23T10:24:08.561291+0000","last_became_active":"2024-04-23T10:24:07.035167+0000","last_became_peered":"2024-04-23T10:24:07.035167+0000","last_unstale":"2024-04-23T10:24:08.561291+0000","last_undegraded":"2024-04-23T10:24:08.561291+0000","last_fullsized":"2024-04-23T10:24:08.561291+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:39:17.198650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.787190+0000","last_change":"2024-04-23T10:24:07.040593+0000","last_active":"2024-04-23T10:24:08.787190+0000","last_peered":"2024-04-23T10:24:08.787190+0000","last_clean":"2024-04-23T10:24:08.787190+0000","last_became_active":"2024-04-23T10:24:07.040255+0000","last_became_peered":"2024-04-23T10:24:07.040255+0000","last_unstale":"2024-04-23T10:24:08.787190+0000","last_undegraded":"2024-04-23T10:24:08.787190+0000","last_fullsized":"2024-04-23T10:24:08.787190+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:05:52.663566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.101104+0000","last_change":"2024-04-23T10:24:07.051688+0000","last_active":"2024-04-23T10:24:08.101104+0000","last_peered":"2024-04-23T10:24:08.101104+0000","last_clean":"2024-04-23T10:24:08.101104+0000","last_became_active":"2024-04-23T10:24:07.050265+0000","last_became_peered":"2024-04-23T10:24:07.050265+0000","last_unstale":"2024-04-23T10:24:08.101104+0000","last_undegraded":"2024-04-23T10:24:08.101104+0000","last_fullsized":"2024-04-23T10:24:08.101104+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T13:13:36.126183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.034321+0000","last_change":"2024-04-23T10:24:07.045937+0000","last_active":"2024-04-23T10:24:08.034321+0000","last_peered":"2024-04-23T10:24:08.034321+0000","last_clean":"2024-04-23T10:24:08.034321+0000","last_became_active":"2024-04-23T10:24:07.045616+0000","last_became_peered":"2024-04-23T10:24:07.045616+0000","last_unstale":"2024-04-23T10:24:08.034321+0000","last_undegraded":"2024-04-23T10:24:08.034321+0000","last_fullsized":"2024-04-23T10:24:08.034321+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T10:45:27.250031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:09.108827+0000","last_change":"2024-04-23T10:24:07.039294+0000","last_active":"2024-04-23T10:24:09.108827+0000","last_peered":"2024-04-23T10:24:09.108827+0000","last_clean":"2024-04-23T10:24:09.108827+0000","last_became_active":"2024-04-23T10:24:07.038959+0000","last_became_peered":"2024-04-23T10:24:07.038959+0000","last_unstale":"2024-04-23T10:24:09.108827+0000","last_undegraded":"2024-04-23T10:24:09.108827+0000","last_fullsized":"2024-04-23T10:24:09.108827+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:53:41.345062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.787352+0000","last_change":"2024-04-23T10:24:07.044177+0000","last_active":"2024-04-23T10:24:08.787352+0000","last_peered":"2024-04-23T10:24:08.787352+0000","last_clean":"2024-04-23T10:24:08.787352+0000","last_became_active":"2024-04-23T10:24:07.043821+0000","last_became_peered":"2024-04-23T10:24:07.043821+0000","last_unstale":"2024-04-23T10:24:08.787352+0000","last_undegraded":"2024-04-23T10:24:08.787352+0000","last_fullsized":"2024-04-23T10:24:08.787352+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:09:38.782407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.101477+0000","last_change":"2024-04-23T10:24:07.052775+0000","last_active":"2024-04-23T10:24:08.101477+0000","last_peered":"2024-04-23T10:24:08.101477+0000","last_clean":"2024-04-23T10:24:08.101477+0000","last_became_active":"2024-04-23T10:24:07.051708+0000","last_became_peered":"2024-04-23T10:24:07.051708+0000","last_unstale":"2024-04-23T10:24:08.101477+0000","last_undegraded":"2024-04-23T10:24:08.101477+0000","last_fullsized":"2024-04-23T10:24:08.101477+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:47:55.360467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.030608+0000","last_change":"2024-04-23T10:24:07.048655+0000","last_active":"2024-04-23T10:24:08.030608+0000","last_peered":"2024-04-23T10:24:08.030608+0000","last_clean":"2024-04-23T10:24:08.030608+0000","last_became_active":"2024-04-23T10:24:07.047994+0000","last_became_peered":"2024-04-23T10:24:07.047994+0000","last_unstale":"2024-04-23T10:24:08.030608+0000","last_undegraded":"2024-04-23T10:24:08.030608+0000","last_fullsized":"2024-04-23T10:24:08.030608+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T10:25:51.425871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:09.418838+0000","last_change":"2024-04-23T10:24:07.036319+0000","last_active":"2024-04-23T10:24:09.418838+0000","last_peered":"2024-04-23T10:24:09.418838+0000","last_clean":"2024-04-23T10:24:09.418838+0000","last_became_active":"2024-04-23T10:24:07.035982+0000","last_became_peered":"2024-04-23T10:24:07.035982+0000","last_unstale":"2024-04-23T10:24:09.418838+0000","last_undegraded":"2024-04-23T10:24:09.418838+0000","last_fullsized":"2024-04-23T10:24:09.418838+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T10:25:46.191118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:09.417987+0000","last_change":"2024-04-23T10:24:07.033166+0000","last_active":"2024-04-23T10:24:09.417987+0000","last_peered":"2024-04-23T10:24:09.417987+0000","last_clean":"2024-04-23T10:24:09.417987+0000","last_became_active":"2024-04-23T10:24:07.032807+0000","last_became_peered":"2024-04-23T10:24:07.032807+0000","last_unstale":"2024-04-23T10:24:09.417987+0000","last_undegraded":"2024-04-23T10:24:09.417987+0000","last_fullsized":"2024-04-23T10:24:09.417987+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:52:35.654965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:09.418040+0000","last_change":"2024-04-23T10:24:07.041988+0000","last_active":"2024-04-23T10:24:09.418040+0000","last_peered":"2024-04-23T10:24:09.418040+0000","last_clean":"2024-04-23T10:24:09.418040+0000","last_became_active":"2024-04-23T10:24:07.041638+0000","last_became_peered":"2024-04-23T10:24:07.041638+0000","last_unstale":"2024-04-23T10:24:09.418040+0000","last_undegraded":"2024-04-23T10:24:09.418040+0000","last_fullsized":"2024-04-23T10:24:09.418040+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:17:40.565981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:09.108294+0000","last_change":"2024-04-23T10:24:07.045111+0000","last_active":"2024-04-23T10:24:09.108294+0000","last_peered":"2024-04-23T10:24:09.108294+0000","last_clean":"2024-04-23T10:24:09.108294+0000","last_became_active":"2024-04-23T10:24:07.044249+0000","last_became_peered":"2024-04-23T10:24:07.044249+0000","last_unstale":"2024-04-23T10:24:09.108294+0000","last_undegraded":"2024-04-23T10:24:09.108294+0000","last_fullsized":"2024-04-23T10:24:09.108294+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:08:36.029636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T10:24:07.041408+0000","last_change":"2024-04-23T10:24:07.040685+0000","last_active":"2024-04-23T10:24:07.041408+0000","last_peered":"2024-04-23T10:24:07.041408+0000","last_clean":"2024-04-23T10:24:07.041408+0000","last_became_active":"2024-04-23T10:24:07.038881+0000","last_became_peered":"2024-04-23T10:24:07.038881+0000","last_unstale":"2024-04-23T10:24:07.041408+0000","last_undegraded":"2024-04-23T10:24:07.041408+0000","last_fullsized":"2024-04-23T10:24:07.041408+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T11:27:08.211643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.392577+0000","last_change":"2024-04-23T10:24:07.041815+0000","last_active":"2024-04-23T10:24:08.392577+0000","last_peered":"2024-04-23T10:24:08.392577+0000","last_clean":"2024-04-23T10:24:08.392577+0000","last_became_active":"2024-04-23T10:24:07.041495+0000","last_became_peered":"2024-04-23T10:24:07.041495+0000","last_unstale":"2024-04-23T10:24:08.392577+0000","last_undegraded":"2024-04-23T10:24:08.392577+0000","last_fullsized":"2024-04-23T10:24:08.392577+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:29:02.006394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.647308+0000","last_change":"2024-04-23T10:24:07.041722+0000","last_active":"2024-04-23T10:24:08.647308+0000","last_peered":"2024-04-23T10:24:08.647308+0000","last_clean":"2024-04-23T10:24:08.647308+0000","last_became_active":"2024-04-23T10:24:07.039434+0000","last_became_peered":"2024-04-23T10:24:07.039434+0000","last_unstale":"2024-04-23T10:24:08.647308+0000","last_undegraded":"2024-04-23T10:24:08.647308+0000","last_fullsized":"2024-04-23T10:24:08.647308+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:03:44.033094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'13","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:11.975075+0000","last_change":"2024-04-23T10:24:06.576664+0000","last_active":"2024-04-23T10:24:11.975075+0000","last_peered":"2024-04-23T10:24:11.975075+0000","last_clean":"2024-04-23T10:24:11.975075+0000","last_became_active":"2024-04-23T10:24:06.576058+0000","last_became_peered":"2024-04-23T10:24:06.576058+0000","last_unstale":"2024-04-23T10:24:11.975075+0000","last_undegraded":"2024-04-23T10:24:11.975075+0000","last_fullsized":"2024-04-23T10:24:11.975075+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:39:57.657026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.034541+0000","last_change":"2024-04-23T10:24:07.047192+0000","last_active":"2024-04-23T10:24:08.034541+0000","last_peered":"2024-04-23T10:24:08.034541+0000","last_clean":"2024-04-23T10:24:08.034541+0000","last_became_active":"2024-04-23T10:24:07.046750+0000","last_became_peered":"2024-04-23T10:24:07.046750+0000","last_unstale":"2024-04-23T10:24:08.034541+0000","last_undegraded":"2024-04-23T10:24:08.034541+0000","last_fullsized":"2024-04-23T10:24:08.034541+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:13:09.399425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.034754+0000","last_change":"2024-04-23T10:24:06.577588+0000","last_active":"2024-04-23T10:24:08.034754+0000","last_peered":"2024-04-23T10:24:08.034754+0000","last_clean":"2024-04-23T10:24:08.034754+0000","last_became_active":"2024-04-23T10:24:06.576971+0000","last_became_peered":"2024-04-23T10:24:06.576971+0000","last_unstale":"2024-04-23T10:24:08.034754+0000","last_undegraded":"2024-04-23T10:24:08.034754+0000","last_fullsized":"2024-04-23T10:24:08.034754+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:07:21.577868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:09.418461+0000","last_change":"2024-04-23T10:24:07.030666+0000","last_active":"2024-04-23T10:24:09.418461+0000","last_peered":"2024-04-23T10:24:09.418461+0000","last_clean":"2024-04-23T10:24:09.418461+0000","last_became_active":"2024-04-23T10:24:07.030342+0000","last_became_peered":"2024-04-23T10:24:07.030342+0000","last_unstale":"2024-04-23T10:24:09.418461+0000","last_undegraded":"2024-04-23T10:24:09.418461+0000","last_fullsized":"2024-04-23T10:24:09.418461+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:09:14.818638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.561639+0000","last_change":"2024-04-23T10:24:06.576042+0000","last_active":"2024-04-23T10:24:08.561639+0000","last_peered":"2024-04-23T10:24:08.561639+0000","last_clean":"2024-04-23T10:24:08.561639+0000","last_became_active":"2024-04-23T10:24:06.574518+0000","last_became_peered":"2024-04-23T10:24:06.574518+0000","last_unstale":"2024-04-23T10:24:08.561639+0000","last_undegraded":"2024-04-23T10:24:08.561639+0000","last_fullsized":"2024-04-23T10:24:08.561639+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T11:56:40.100213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:09.108402+0000","last_change":"2024-04-23T10:24:07.046591+0000","last_active":"2024-04-23T10:24:09.108402+0000","last_peered":"2024-04-23T10:24:09.108402+0000","last_clean":"2024-04-23T10:24:09.108402+0000","last_became_active":"2024-04-23T10:24:07.046208+0000","last_became_peered":"2024-04-23T10:24:07.046208+0000","last_unstale":"2024-04-23T10:24:09.108402+0000","last_undegraded":"2024-04-23T10:24:09.108402+0000","last_fullsized":"2024-04-23T10:24:09.108402+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:27:37.711329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.561525+0000","last_change":"2024-04-23T10:24:06.568185+0000","last_active":"2024-04-23T10:24:08.561525+0000","last_peered":"2024-04-23T10:24:08.561525+0000","last_clean":"2024-04-23T10:24:08.561525+0000","last_became_active":"2024-04-23T10:24:06.567841+0000","last_became_peered":"2024-04-23T10:24:06.567841+0000","last_unstale":"2024-04-23T10:24:08.561525+0000","last_undegraded":"2024-04-23T10:24:08.561525+0000","last_fullsized":"2024-04-23T10:24:08.561525+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:29:30.214779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.036335+0000","last_change":"2024-04-23T10:24:07.047926+0000","last_active":"2024-04-23T10:24:08.036335+0000","last_peered":"2024-04-23T10:24:08.036335+0000","last_clean":"2024-04-23T10:24:08.036335+0000","last_became_active":"2024-04-23T10:24:07.047273+0000","last_became_peered":"2024-04-23T10:24:07.047273+0000","last_unstale":"2024-04-23T10:24:08.036335+0000","last_undegraded":"2024-04-23T10:24:08.036335+0000","last_fullsized":"2024-04-23T10:24:08.036335+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:37:25.068034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.036101+0000","last_change":"2024-04-23T10:24:06.574733+0000","last_active":"2024-04-23T10:24:08.036101+0000","last_peered":"2024-04-23T10:24:08.036101+0000","last_clean":"2024-04-23T10:24:08.036101+0000","last_became_active":"2024-04-23T10:24:06.574276+0000","last_became_peered":"2024-04-23T10:24:06.574276+0000","last_unstale":"2024-04-23T10:24:08.036101+0000","last_undegraded":"2024-04-23T10:24:08.036101+0000","last_fullsized":"2024-04-23T10:24:08.036101+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:27:57.080635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.561307+0000","last_change":"2024-04-23T10:24:07.040110+0000","last_active":"2024-04-23T10:24:08.561307+0000","last_peered":"2024-04-23T10:24:08.561307+0000","last_clean":"2024-04-23T10:24:08.561307+0000","last_became_active":"2024-04-23T10:24:07.039769+0000","last_became_peered":"2024-04-23T10:24:07.039769+0000","last_unstale":"2024-04-23T10:24:08.561307+0000","last_undegraded":"2024-04-23T10:24:08.561307+0000","last_fullsized":"2024-04-23T10:24:08.561307+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:10:07.884895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:10.961020+0000","last_change":"2024-04-23T10:24:06.567448+0000","last_active":"2024-04-23T10:24:10.961020+0000","last_peered":"2024-04-23T10:24:10.961020+0000","last_clean":"2024-04-23T10:24:10.961020+0000","last_became_active":"2024-04-23T10:24:06.566958+0000","last_became_peered":"2024-04-23T10:24:06.566958+0000","last_unstale":"2024-04-23T10:24:10.961020+0000","last_undegraded":"2024-04-23T10:24:10.961020+0000","last_fullsized":"2024-04-23T10:24:10.961020+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:45:52.863596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.036371+0000","last_change":"2024-04-23T10:24:07.040754+0000","last_active":"2024-04-23T10:24:08.036371+0000","last_peered":"2024-04-23T10:24:08.036371+0000","last_clean":"2024-04-23T10:24:08.036371+0000","last_became_active":"2024-04-23T10:24:07.040397+0000","last_became_peered":"2024-04-23T10:24:07.040397+0000","last_unstale":"2024-04-23T10:24:08.036371+0000","last_undegraded":"2024-04-23T10:24:08.036371+0000","last_fullsized":"2024-04-23T10:24:08.036371+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T11:01:03.626506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:09.418383+0000","last_change":"2024-04-23T10:24:06.568592+0000","last_active":"2024-04-23T10:24:09.418383+0000","last_peered":"2024-04-23T10:24:09.418383+0000","last_clean":"2024-04-23T10:24:09.418383+0000","last_became_active":"2024-04-23T10:24:06.568192+0000","last_became_peered":"2024-04-23T10:24:06.568192+0000","last_unstale":"2024-04-23T10:24:09.418383+0000","last_undegraded":"2024-04-23T10:24:09.418383+0000","last_fullsized":"2024-04-23T10:24:09.418383+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:14:42.589596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.034809+0000","last_change":"2024-04-23T10:24:07.037058+0000","last_active":"2024-04-23T10:24:08.034809+0000","last_peered":"2024-04-23T10:24:08.034809+0000","last_clean":"2024-04-23T10:24:08.034809+0000","last_became_active":"2024-04-23T10:24:07.036659+0000","last_became_peered":"2024-04-23T10:24:07.036659+0000","last_unstale":"2024-04-23T10:24:08.034809+0000","last_undegraded":"2024-04-23T10:24:08.034809+0000","last_fullsized":"2024-04-23T10:24:08.034809+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T12:53:49.298603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.035915+0000","last_change":"2024-04-23T10:24:06.576900+0000","last_active":"2024-04-23T10:24:08.035915+0000","last_peered":"2024-04-23T10:24:08.035915+0000","last_clean":"2024-04-23T10:24:08.035915+0000","last_became_active":"2024-04-23T10:24:06.575627+0000","last_became_peered":"2024-04-23T10:24:06.575627+0000","last_unstale":"2024-04-23T10:24:08.035915+0000","last_undegraded":"2024-04-23T10:24:08.035915+0000","last_fullsized":"2024-04-23T10:24:08.035915+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:16:07.973841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.392415+0000","last_change":"2024-04-23T10:24:07.035294+0000","last_active":"2024-04-23T10:24:08.392415+0000","last_peered":"2024-04-23T10:24:08.392415+0000","last_clean":"2024-04-23T10:24:08.392415+0000","last_became_active":"2024-04-23T10:24:07.035001+0000","last_became_peered":"2024-04-23T10:24:07.035001+0000","last_unstale":"2024-04-23T10:24:08.392415+0000","last_undegraded":"2024-04-23T10:24:08.392415+0000","last_fullsized":"2024-04-23T10:24:08.392415+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T13:12:47.013132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:10.952763+0000","last_change":"2024-04-23T10:24:06.573637+0000","last_active":"2024-04-23T10:24:10.952763+0000","last_peered":"2024-04-23T10:24:10.952763+0000","last_clean":"2024-04-23T10:24:10.952763+0000","last_became_active":"2024-04-23T10:24:06.573236+0000","last_became_peered":"2024-04-23T10:24:06.573236+0000","last_unstale":"2024-04-23T10:24:10.952763+0000","last_undegraded":"2024-04-23T10:24:10.952763+0000","last_fullsized":"2024-04-23T10:24:10.952763+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T12:48:06.740181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:09.108659+0000","last_change":"2024-04-23T10:24:07.037457+0000","last_active":"2024-04-23T10:24:09.108659+0000","last_peered":"2024-04-23T10:24:09.108659+0000","last_clean":"2024-04-23T10:24:09.108659+0000","last_became_active":"2024-04-23T10:24:07.036456+0000","last_became_peered":"2024-04-23T10:24:07.036456+0000","last_unstale":"2024-04-23T10:24:09.108659+0000","last_undegraded":"2024-04-23T10:24:09.108659+0000","last_fullsized":"2024-04-23T10:24:09.108659+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:42:57.750435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:09.576356+0000","last_change":"2024-04-23T10:24:06.575317+0000","last_active":"2024-04-23T10:24:09.576356+0000","last_peered":"2024-04-23T10:24:09.576356+0000","last_clean":"2024-04-23T10:24:09.576356+0000","last_became_active":"2024-04-23T10:24:06.574935+0000","last_became_peered":"2024-04-23T10:24:06.574935+0000","last_unstale":"2024-04-23T10:24:09.576356+0000","last_undegraded":"2024-04-23T10:24:09.576356+0000","last_fullsized":"2024-04-23T10:24:09.576356+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T11:03:47.864370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:09.418390+0000","last_change":"2024-04-23T10:24:07.040317+0000","last_active":"2024-04-23T10:24:09.418390+0000","last_peered":"2024-04-23T10:24:09.418390+0000","last_clean":"2024-04-23T10:24:09.418390+0000","last_became_active":"2024-04-23T10:24:07.039998+0000","last_became_peered":"2024-04-23T10:24:07.039998+0000","last_unstale":"2024-04-23T10:24:09.418390+0000","last_undegraded":"2024-04-23T10:24:09.418390+0000","last_fullsized":"2024-04-23T10:24:09.418390+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:50:30.412127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.036223+0000","last_change":"2024-04-23T10:24:06.570019+0000","last_active":"2024-04-23T10:24:08.036223+0000","last_peered":"2024-04-23T10:24:08.036223+0000","last_clean":"2024-04-23T10:24:08.036223+0000","last_became_active":"2024-04-23T10:24:06.569676+0000","last_became_peered":"2024-04-23T10:24:06.569676+0000","last_unstale":"2024-04-23T10:24:08.036223+0000","last_undegraded":"2024-04-23T10:24:08.036223+0000","last_fullsized":"2024-04-23T10:24:08.036223+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:31:46.994787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:09.108524+0000","last_change":"2024-04-23T10:24:07.040183+0000","last_active":"2024-04-23T10:24:09.108524+0000","last_peered":"2024-04-23T10:24:09.108524+0000","last_clean":"2024-04-23T10:24:09.108524+0000","last_became_active":"2024-04-23T10:24:07.039866+0000","last_became_peered":"2024-04-23T10:24:07.039866+0000","last_unstale":"2024-04-23T10:24:09.108524+0000","last_undegraded":"2024-04-23T10:24:09.108524+0000","last_fullsized":"2024-04-23T10:24:09.108524+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:33:04.411765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:10.953348+0000","last_change":"2024-04-23T10:24:06.568883+0000","last_active":"2024-04-23T10:24:10.953348+0000","last_peered":"2024-04-23T10:24:10.953348+0000","last_clean":"2024-04-23T10:24:10.953348+0000","last_became_active":"2024-04-23T10:24:06.568517+0000","last_became_peered":"2024-04-23T10:24:06.568517+0000","last_unstale":"2024-04-23T10:24:10.953348+0000","last_undegraded":"2024-04-23T10:24:10.953348+0000","last_fullsized":"2024-04-23T10:24:10.953348+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:51:42.440346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.561458+0000","last_change":"2024-04-23T10:24:07.046839+0000","last_active":"2024-04-23T10:24:08.561458+0000","last_peered":"2024-04-23T10:24:08.561458+0000","last_clean":"2024-04-23T10:24:08.561458+0000","last_became_active":"2024-04-23T10:24:07.046512+0000","last_became_peered":"2024-04-23T10:24:07.046512+0000","last_unstale":"2024-04-23T10:24:08.561458+0000","last_undegraded":"2024-04-23T10:24:08.561458+0000","last_fullsized":"2024-04-23T10:24:08.561458+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T12:42:11.822100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:09.577162+0000","last_change":"2024-04-23T10:24:06.578087+0000","last_active":"2024-04-23T10:24:09.577162+0000","last_peered":"2024-04-23T10:24:09.577162+0000","last_clean":"2024-04-23T10:24:09.577162+0000","last_became_active":"2024-04-23T10:24:06.577666+0000","last_became_peered":"2024-04-23T10:24:06.577666+0000","last_unstale":"2024-04-23T10:24:09.577162+0000","last_undegraded":"2024-04-23T10:24:09.577162+0000","last_fullsized":"2024-04-23T10:24:09.577162+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:55:15.053303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:09.418570+0000","last_change":"2024-04-23T10:24:07.043672+0000","last_active":"2024-04-23T10:24:09.418570+0000","last_peered":"2024-04-23T10:24:09.418570+0000","last_clean":"2024-04-23T10:24:09.418570+0000","last_became_active":"2024-04-23T10:24:07.043346+0000","last_became_peered":"2024-04-23T10:24:07.043346+0000","last_unstale":"2024-04-23T10:24:09.418570+0000","last_undegraded":"2024-04-23T10:24:09.418570+0000","last_fullsized":"2024-04-23T10:24:09.418570+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T13:21:56.751028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.035934+0000","last_change":"2024-04-23T10:24:06.576607+0000","last_active":"2024-04-23T10:24:08.035934+0000","last_peered":"2024-04-23T10:24:08.035934+0000","last_clean":"2024-04-23T10:24:08.035934+0000","last_became_active":"2024-04-23T10:24:06.576213+0000","last_became_peered":"2024-04-23T10:24:06.576213+0000","last_unstale":"2024-04-23T10:24:08.035934+0000","last_undegraded":"2024-04-23T10:24:08.035934+0000","last_fullsized":"2024-04-23T10:24:08.035934+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:09:14.918936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.101213+0000","last_change":"2024-04-23T10:24:07.052641+0000","last_active":"2024-04-23T10:24:08.101213+0000","last_peered":"2024-04-23T10:24:08.101213+0000","last_clean":"2024-04-23T10:24:08.101213+0000","last_became_active":"2024-04-23T10:24:07.052314+0000","last_became_peered":"2024-04-23T10:24:07.052314+0000","last_unstale":"2024-04-23T10:24:08.101213+0000","last_undegraded":"2024-04-23T10:24:08.101213+0000","last_fullsized":"2024-04-23T10:24:08.101213+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T10:29:18.098800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.030318+0000","last_change":"2024-04-23T10:24:06.574570+0000","last_active":"2024-04-23T10:24:08.030318+0000","last_peered":"2024-04-23T10:24:08.030318+0000","last_clean":"2024-04-23T10:24:08.030318+0000","last_became_active":"2024-04-23T10:24:06.574166+0000","last_became_peered":"2024-04-23T10:24:06.574166+0000","last_unstale":"2024-04-23T10:24:08.030318+0000","last_undegraded":"2024-04-23T10:24:08.030318+0000","last_fullsized":"2024-04-23T10:24:08.030318+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:28:31.619100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.100954+0000","last_change":"2024-04-23T10:24:07.052825+0000","last_active":"2024-04-23T10:24:08.100954+0000","last_peered":"2024-04-23T10:24:08.100954+0000","last_clean":"2024-04-23T10:24:08.100954+0000","last_became_active":"2024-04-23T10:24:07.052457+0000","last_became_peered":"2024-04-23T10:24:07.052457+0000","last_unstale":"2024-04-23T10:24:08.100954+0000","last_undegraded":"2024-04-23T10:24:08.100954+0000","last_fullsized":"2024-04-23T10:24:08.100954+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:47:09.545374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'10","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:09.576072+0000","last_change":"2024-04-23T10:24:06.572168+0000","last_active":"2024-04-23T10:24:09.576072+0000","last_peered":"2024-04-23T10:24:09.576072+0000","last_clean":"2024-04-23T10:24:09.576072+0000","last_became_active":"2024-04-23T10:24:06.571805+0000","last_became_peered":"2024-04-23T10:24:06.571805+0000","last_unstale":"2024-04-23T10:24:09.576072+0000","last_undegraded":"2024-04-23T10:24:09.576072+0000","last_fullsized":"2024-04-23T10:24:09.576072+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:29:56.817879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.036427+0000","last_change":"2024-04-23T10:24:07.048474+0000","last_active":"2024-04-23T10:24:08.036427+0000","last_peered":"2024-04-23T10:24:08.036427+0000","last_clean":"2024-04-23T10:24:08.036427+0000","last_became_active":"2024-04-23T10:24:07.047893+0000","last_became_peered":"2024-04-23T10:24:07.047893+0000","last_unstale":"2024-04-23T10:24:08.036427+0000","last_undegraded":"2024-04-23T10:24:08.036427+0000","last_fullsized":"2024-04-23T10:24:08.036427+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T12:59:46.545569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.561598+0000","last_change":"2024-04-23T10:24:06.570974+0000","last_active":"2024-04-23T10:24:08.561598+0000","last_peered":"2024-04-23T10:24:08.561598+0000","last_clean":"2024-04-23T10:24:08.561598+0000","last_became_active":"2024-04-23T10:24:06.570603+0000","last_became_peered":"2024-04-23T10:24:06.570603+0000","last_unstale":"2024-04-23T10:24:08.561598+0000","last_undegraded":"2024-04-23T10:24:08.561598+0000","last_fullsized":"2024-04-23T10:24:08.561598+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T12:46:33.518337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T10:24:07.051616+0000","last_change":"2024-04-23T10:24:07.050986+0000","last_active":"2024-04-23T10:24:07.051616+0000","last_peered":"2024-04-23T10:24:07.051616+0000","last_clean":"2024-04-23T10:24:07.051616+0000","last_became_active":"2024-04-23T10:24:07.050617+0000","last_became_peered":"2024-04-23T10:24:07.050617+0000","last_unstale":"2024-04-23T10:24:07.051616+0000","last_undegraded":"2024-04-23T10:24:07.051616+0000","last_fullsized":"2024-04-23T10:24:07.051616+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:47:19.951516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.392499+0000","last_change":"2024-04-23T10:24:06.579138+0000","last_active":"2024-04-23T10:24:08.392499+0000","last_peered":"2024-04-23T10:24:08.392499+0000","last_clean":"2024-04-23T10:24:08.392499+0000","last_became_active":"2024-04-23T10:24:06.578716+0000","last_became_peered":"2024-04-23T10:24:06.578716+0000","last_unstale":"2024-04-23T10:24:08.392499+0000","last_undegraded":"2024-04-23T10:24:08.392499+0000","last_fullsized":"2024-04-23T10:24:08.392499+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:06:59.797802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.787215+0000","last_change":"2024-04-23T10:24:07.039019+0000","last_active":"2024-04-23T10:24:08.787215+0000","last_peered":"2024-04-23T10:24:08.787215+0000","last_clean":"2024-04-23T10:24:08.787215+0000","last_became_active":"2024-04-23T10:24:07.038701+0000","last_became_peered":"2024-04-23T10:24:07.038701+0000","last_unstale":"2024-04-23T10:24:08.787215+0000","last_undegraded":"2024-04-23T10:24:08.787215+0000","last_fullsized":"2024-04-23T10:24:08.787215+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:26:32.716043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.101257+0000","last_change":"2024-04-23T10:24:06.574342+0000","last_active":"2024-04-23T10:24:08.101257+0000","last_peered":"2024-04-23T10:24:08.101257+0000","last_clean":"2024-04-23T10:24:08.101257+0000","last_became_active":"2024-04-23T10:24:06.573981+0000","last_became_peered":"2024-04-23T10:24:06.573981+0000","last_unstale":"2024-04-23T10:24:08.101257+0000","last_undegraded":"2024-04-23T10:24:08.101257+0000","last_fullsized":"2024-04-23T10:24:08.101257+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:48:40.325991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.787264+0000","last_change":"2024-04-23T10:24:07.044712+0000","last_active":"2024-04-23T10:24:08.787264+0000","last_peered":"2024-04-23T10:24:08.787264+0000","last_clean":"2024-04-23T10:24:08.787264+0000","last_became_active":"2024-04-23T10:24:07.044141+0000","last_became_peered":"2024-04-23T10:24:07.044141+0000","last_unstale":"2024-04-23T10:24:08.787264+0000","last_undegraded":"2024-04-23T10:24:08.787264+0000","last_fullsized":"2024-04-23T10:24:08.787264+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:14:38.818648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.036149+0000","last_change":"2024-04-23T10:24:07.049061+0000","last_active":"2024-04-23T10:24:08.036149+0000","last_peered":"2024-04-23T10:24:08.036149+0000","last_clean":"2024-04-23T10:24:08.036149+0000","last_became_active":"2024-04-23T10:24:07.046446+0000","last_became_peered":"2024-04-23T10:24:07.046446+0000","last_unstale":"2024-04-23T10:24:08.036149+0000","last_undegraded":"2024-04-23T10:24:08.036149+0000","last_fullsized":"2024-04-23T10:24:08.036149+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:08:15.062455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:09.573911+0000","last_change":"2024-04-23T10:24:06.578216+0000","last_active":"2024-04-23T10:24:09.573911+0000","last_peered":"2024-04-23T10:24:09.573911+0000","last_clean":"2024-04-23T10:24:09.573911+0000","last_became_active":"2024-04-23T10:24:06.577370+0000","last_became_peered":"2024-04-23T10:24:06.577370+0000","last_unstale":"2024-04-23T10:24:09.573911+0000","last_undegraded":"2024-04-23T10:24:09.573911+0000","last_fullsized":"2024-04-23T10:24:09.573911+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:02:06.554442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.034686+0000","last_change":"2024-04-23T10:24:07.039627+0000","last_active":"2024-04-23T10:24:08.034686+0000","last_peered":"2024-04-23T10:24:08.034686+0000","last_clean":"2024-04-23T10:24:08.034686+0000","last_became_active":"2024-04-23T10:24:07.039272+0000","last_became_peered":"2024-04-23T10:24:07.039272+0000","last_unstale":"2024-04-23T10:24:08.034686+0000","last_undegraded":"2024-04-23T10:24:08.034686+0000","last_fullsized":"2024-04-23T10:24:08.034686+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T13:45:44.277926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.030532+0000","last_change":"2024-04-23T10:24:06.569086+0000","last_active":"2024-04-23T10:24:08.030532+0000","last_peered":"2024-04-23T10:24:08.030532+0000","last_clean":"2024-04-23T10:24:08.030532+0000","last_became_active":"2024-04-23T10:24:06.568719+0000","last_became_peered":"2024-04-23T10:24:06.568719+0000","last_unstale":"2024-04-23T10:24:08.030532+0000","last_undegraded":"2024-04-23T10:24:08.030532+0000","last_fullsized":"2024-04-23T10:24:08.030532+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:22:10.583782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":350,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.100904+0000","last_change":"2024-04-23T10:22:49.957476+0000","last_active":"2024-04-23T10:24:08.100904+0000","last_peered":"2024-04-23T10:24:08.100904+0000","last_clean":"2024-04-23T10:24:08.100904+0000","last_became_active":"2024-04-23T10:22:49.957033+0000","last_became_peered":"2024-04-23T10:22:49.957033+0000","last_unstale":"2024-04-23T10:24:08.100904+0000","last_undegraded":"2024-04-23T10:24:08.100904+0000","last_fullsized":"2024-04-23T10:24:08.100904+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:19:25.604229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:19:25.604229+0000","last_clean_scrub_stamp":"2024-04-23T10:19:25.604229+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T12:56:17.601141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:09.418252+0000","last_change":"2024-04-23T10:24:07.037936+0000","last_active":"2024-04-23T10:24:09.418252+0000","last_peered":"2024-04-23T10:24:09.418252+0000","last_clean":"2024-04-23T10:24:09.418252+0000","last_became_active":"2024-04-23T10:24:07.037614+0000","last_became_peered":"2024-04-23T10:24:07.037614+0000","last_unstale":"2024-04-23T10:24:09.418252+0000","last_undegraded":"2024-04-23T10:24:09.418252+0000","last_fullsized":"2024-04-23T10:24:09.418252+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T11:01:39.807822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:09.575767+0000","last_change":"2024-04-23T10:24:06.582629+0000","last_active":"2024-04-23T10:24:09.575767+0000","last_peered":"2024-04-23T10:24:09.575767+0000","last_clean":"2024-04-23T10:24:09.575767+0000","last_became_active":"2024-04-23T10:24:06.582325+0000","last_became_peered":"2024-04-23T10:24:06.582325+0000","last_unstale":"2024-04-23T10:24:09.575767+0000","last_undegraded":"2024-04-23T10:24:09.575767+0000","last_fullsized":"2024-04-23T10:24:09.575767+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:57:09.879100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:09.574919+0000","last_change":"2024-04-23T10:24:06.582482+0000","last_active":"2024-04-23T10:24:09.574919+0000","last_peered":"2024-04-23T10:24:09.574919+0000","last_clean":"2024-04-23T10:24:09.574919+0000","last_became_active":"2024-04-23T10:24:06.582114+0000","last_became_peered":"2024-04-23T10:24:06.582114+0000","last_unstale":"2024-04-23T10:24:09.574919+0000","last_undegraded":"2024-04-23T10:24:09.574919+0000","last_fullsized":"2024-04-23T10:24:09.574919+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:45:42.939840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:09.418889+0000","last_change":"2024-04-23T10:24:07.031102+0000","last_active":"2024-04-23T10:24:09.418889+0000","last_peered":"2024-04-23T10:24:09.418889+0000","last_clean":"2024-04-23T10:24:09.418889+0000","last_became_active":"2024-04-23T10:24:07.030783+0000","last_became_peered":"2024-04-23T10:24:07.030783+0000","last_unstale":"2024-04-23T10:24:09.418889+0000","last_undegraded":"2024-04-23T10:24:09.418889+0000","last_fullsized":"2024-04-23T10:24:09.418889+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:49:46.736046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'8","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:10.963752+0000","last_change":"2024-04-23T10:24:06.577096+0000","last_active":"2024-04-23T10:24:10.963752+0000","last_peered":"2024-04-23T10:24:10.963752+0000","last_clean":"2024-04-23T10:24:10.963752+0000","last_became_active":"2024-04-23T10:24:06.576740+0000","last_became_peered":"2024-04-23T10:24:06.576740+0000","last_unstale":"2024-04-23T10:24:10.963752+0000","last_undegraded":"2024-04-23T10:24:10.963752+0000","last_fullsized":"2024-04-23T10:24:10.963752+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:44:06.532366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.034848+0000","last_change":"2024-04-23T10:24:07.043599+0000","last_active":"2024-04-23T10:24:08.034848+0000","last_peered":"2024-04-23T10:24:08.034848+0000","last_clean":"2024-04-23T10:24:08.034848+0000","last_became_active":"2024-04-23T10:24:07.042953+0000","last_became_peered":"2024-04-23T10:24:07.042953+0000","last_unstale":"2024-04-23T10:24:08.034848+0000","last_undegraded":"2024-04-23T10:24:08.034848+0000","last_fullsized":"2024-04-23T10:24:08.034848+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:07:50.487972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.030760+0000","last_change":"2024-04-23T10:24:06.570681+0000","last_active":"2024-04-23T10:24:08.030760+0000","last_peered":"2024-04-23T10:24:08.030760+0000","last_clean":"2024-04-23T10:24:08.030760+0000","last_became_active":"2024-04-23T10:24:06.570178+0000","last_became_peered":"2024-04-23T10:24:06.570178+0000","last_unstale":"2024-04-23T10:24:08.030760+0000","last_undegraded":"2024-04-23T10:24:08.030760+0000","last_fullsized":"2024-04-23T10:24:08.030760+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T12:16:56.629443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:09.418633+0000","last_change":"2024-04-23T10:24:07.040498+0000","last_active":"2024-04-23T10:24:09.418633+0000","last_peered":"2024-04-23T10:24:09.418633+0000","last_clean":"2024-04-23T10:24:09.418633+0000","last_became_active":"2024-04-23T10:24:07.040009+0000","last_became_peered":"2024-04-23T10:24:07.040009+0000","last_unstale":"2024-04-23T10:24:09.418633+0000","last_undegraded":"2024-04-23T10:24:09.418633+0000","last_fullsized":"2024-04-23T10:24:09.418633+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:43:17.319305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:09.575461+0000","last_change":"2024-04-23T10:24:06.576904+0000","last_active":"2024-04-23T10:24:09.575461+0000","last_peered":"2024-04-23T10:24:09.575461+0000","last_clean":"2024-04-23T10:24:09.575461+0000","last_became_active":"2024-04-23T10:24:06.576512+0000","last_became_peered":"2024-04-23T10:24:06.576512+0000","last_unstale":"2024-04-23T10:24:09.575461+0000","last_undegraded":"2024-04-23T10:24:09.575461+0000","last_fullsized":"2024-04-23T10:24:09.575461+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:43:49.008844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.561197+0000","last_change":"2024-04-23T10:24:07.043710+0000","last_active":"2024-04-23T10:24:08.561197+0000","last_peered":"2024-04-23T10:24:08.561197+0000","last_clean":"2024-04-23T10:24:08.561197+0000","last_became_active":"2024-04-23T10:24:07.043406+0000","last_became_peered":"2024-04-23T10:24:07.043406+0000","last_unstale":"2024-04-23T10:24:08.561197+0000","last_undegraded":"2024-04-23T10:24:08.561197+0000","last_fullsized":"2024-04-23T10:24:08.561197+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:09:14.471938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.034410+0000","last_change":"2024-04-23T10:24:06.577409+0000","last_active":"2024-04-23T10:24:08.034410+0000","last_peered":"2024-04-23T10:24:08.034410+0000","last_clean":"2024-04-23T10:24:08.034410+0000","last_became_active":"2024-04-23T10:24:06.576787+0000","last_became_peered":"2024-04-23T10:24:06.576787+0000","last_unstale":"2024-04-23T10:24:08.034410+0000","last_undegraded":"2024-04-23T10:24:08.034410+0000","last_fullsized":"2024-04-23T10:24:08.034410+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:05:28.423806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T10:24:07.045753+0000","last_change":"2024-04-23T10:24:07.045180+0000","last_active":"2024-04-23T10:24:07.045753+0000","last_peered":"2024-04-23T10:24:07.045753+0000","last_clean":"2024-04-23T10:24:07.045753+0000","last_became_active":"2024-04-23T10:24:07.044731+0000","last_became_peered":"2024-04-23T10:24:07.044731+0000","last_unstale":"2024-04-23T10:24:07.045753+0000","last_undegraded":"2024-04-23T10:24:07.045753+0000","last_fullsized":"2024-04-23T10:24:07.045753+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T13:33:22.092090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.647033+0000","last_change":"2024-04-23T10:24:06.578039+0000","last_active":"2024-04-23T10:24:08.647033+0000","last_peered":"2024-04-23T10:24:08.647033+0000","last_clean":"2024-04-23T10:24:08.647033+0000","last_became_active":"2024-04-23T10:24:06.577655+0000","last_became_peered":"2024-04-23T10:24:06.577655+0000","last_unstale":"2024-04-23T10:24:08.647033+0000","last_undegraded":"2024-04-23T10:24:08.647033+0000","last_fullsized":"2024-04-23T10:24:08.647033+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:57:35.172318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.034513+0000","last_change":"2024-04-23T10:24:07.037311+0000","last_active":"2024-04-23T10:24:08.034513+0000","last_peered":"2024-04-23T10:24:08.034513+0000","last_clean":"2024-04-23T10:24:08.034513+0000","last_became_active":"2024-04-23T10:24:07.036969+0000","last_became_peered":"2024-04-23T10:24:07.036969+0000","last_unstale":"2024-04-23T10:24:08.034513+0000","last_undegraded":"2024-04-23T10:24:08.034513+0000","last_fullsized":"2024-04-23T10:24:08.034513+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:13:18.998914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:09.576616+0000","last_change":"2024-04-23T10:24:06.575183+0000","last_active":"2024-04-23T10:24:09.576616+0000","last_peered":"2024-04-23T10:24:09.576616+0000","last_clean":"2024-04-23T10:24:09.576616+0000","last_became_active":"2024-04-23T10:24:06.574630+0000","last_became_peered":"2024-04-23T10:24:06.574630+0000","last_unstale":"2024-04-23T10:24:09.576616+0000","last_undegraded":"2024-04-23T10:24:09.576616+0000","last_fullsized":"2024-04-23T10:24:09.576616+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T11:18:18.189925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.101366+0000","last_change":"2024-04-23T10:24:07.048832+0000","last_active":"2024-04-23T10:24:08.101366+0000","last_peered":"2024-04-23T10:24:08.101366+0000","last_clean":"2024-04-23T10:24:08.101366+0000","last_became_active":"2024-04-23T10:24:07.048509+0000","last_became_peered":"2024-04-23T10:24:07.048509+0000","last_unstale":"2024-04-23T10:24:08.101366+0000","last_undegraded":"2024-04-23T10:24:08.101366+0000","last_fullsized":"2024-04-23T10:24:08.101366+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T13:45:26.793330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.647170+0000","last_change":"2024-04-23T10:24:06.575408+0000","last_active":"2024-04-23T10:24:08.647170+0000","last_peered":"2024-04-23T10:24:08.647170+0000","last_clean":"2024-04-23T10:24:08.647170+0000","last_became_active":"2024-04-23T10:24:06.575081+0000","last_became_peered":"2024-04-23T10:24:06.575081+0000","last_unstale":"2024-04-23T10:24:08.647170+0000","last_undegraded":"2024-04-23T10:24:08.647170+0000","last_fullsized":"2024-04-23T10:24:08.647170+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T11:28:12.765196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.101587+0000","last_change":"2024-04-23T10:24:07.043869+0000","last_active":"2024-04-23T10:24:08.101587+0000","last_peered":"2024-04-23T10:24:08.101587+0000","last_clean":"2024-04-23T10:24:08.101587+0000","last_became_active":"2024-04-23T10:24:07.043448+0000","last_became_peered":"2024-04-23T10:24:07.043448+0000","last_unstale":"2024-04-23T10:24:08.101587+0000","last_undegraded":"2024-04-23T10:24:08.101587+0000","last_fullsized":"2024-04-23T10:24:08.101587+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:00:35.821470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.036052+0000","last_change":"2024-04-23T10:24:06.575259+0000","last_active":"2024-04-23T10:24:08.036052+0000","last_peered":"2024-04-23T10:24:08.036052+0000","last_clean":"2024-04-23T10:24:08.036052+0000","last_became_active":"2024-04-23T10:24:06.574920+0000","last_became_peered":"2024-04-23T10:24:06.574920+0000","last_unstale":"2024-04-23T10:24:08.036052+0000","last_undegraded":"2024-04-23T10:24:08.036052+0000","last_fullsized":"2024-04-23T10:24:08.036052+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:52:14.558458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.101206+0000","last_change":"2024-04-23T10:24:07.043238+0000","last_active":"2024-04-23T10:24:08.101206+0000","last_peered":"2024-04-23T10:24:08.101206+0000","last_clean":"2024-04-23T10:24:08.101206+0000","last_became_active":"2024-04-23T10:24:07.042110+0000","last_became_peered":"2024-04-23T10:24:07.042110+0000","last_unstale":"2024-04-23T10:24:08.101206+0000","last_undegraded":"2024-04-23T10:24:08.101206+0000","last_fullsized":"2024-04-23T10:24:08.101206+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T12:36:13.474456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:09.418851+0000","last_change":"2024-04-23T10:24:06.579233+0000","last_active":"2024-04-23T10:24:09.418851+0000","last_peered":"2024-04-23T10:24:09.418851+0000","last_clean":"2024-04-23T10:24:09.418851+0000","last_became_active":"2024-04-23T10:24:06.578849+0000","last_became_peered":"2024-04-23T10:24:06.578849+0000","last_unstale":"2024-04-23T10:24:09.418851+0000","last_undegraded":"2024-04-23T10:24:09.418851+0000","last_fullsized":"2024-04-23T10:24:09.418851+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:34:23.193707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.101450+0000","last_change":"2024-04-23T10:24:07.049840+0000","last_active":"2024-04-23T10:24:08.101450+0000","last_peered":"2024-04-23T10:24:08.101450+0000","last_clean":"2024-04-23T10:24:08.101450+0000","last_became_active":"2024-04-23T10:24:07.049493+0000","last_became_peered":"2024-04-23T10:24:07.049493+0000","last_unstale":"2024-04-23T10:24:08.101450+0000","last_undegraded":"2024-04-23T10:24:08.101450+0000","last_fullsized":"2024-04-23T10:24:08.101450+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:03:22.606800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T10:24:07.053113+0000","last_change":"2024-04-23T10:24:07.052332+0000","last_active":"2024-04-23T10:24:07.053113+0000","last_peered":"2024-04-23T10:24:07.053113+0000","last_clean":"2024-04-23T10:24:07.053113+0000","last_became_active":"2024-04-23T10:24:07.051956+0000","last_became_peered":"2024-04-23T10:24:07.051956+0000","last_unstale":"2024-04-23T10:24:07.053113+0000","last_undegraded":"2024-04-23T10:24:07.053113+0000","last_fullsized":"2024-04-23T10:24:07.053113+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:55:24.834893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.647166+0000","last_change":"2024-04-23T10:24:07.041897+0000","last_active":"2024-04-23T10:24:08.647166+0000","last_peered":"2024-04-23T10:24:08.647166+0000","last_clean":"2024-04-23T10:24:08.647166+0000","last_became_active":"2024-04-23T10:24:07.041546+0000","last_became_peered":"2024-04-23T10:24:07.041546+0000","last_unstale":"2024-04-23T10:24:08.647166+0000","last_undegraded":"2024-04-23T10:24:08.647166+0000","last_fullsized":"2024-04-23T10:24:08.647166+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T11:31:23.211045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.647156+0000","last_change":"2024-04-23T10:24:07.040764+0000","last_active":"2024-04-23T10:24:08.647156+0000","last_peered":"2024-04-23T10:24:08.647156+0000","last_clean":"2024-04-23T10:24:08.647156+0000","last_became_active":"2024-04-23T10:24:07.039637+0000","last_became_peered":"2024-04-23T10:24:07.039637+0000","last_unstale":"2024-04-23T10:24:08.647156+0000","last_undegraded":"2024-04-23T10:24:08.647156+0000","last_fullsized":"2024-04-23T10:24:08.647156+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:50:33.298770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:09.418471+0000","last_change":"2024-04-23T10:24:07.039491+0000","last_active":"2024-04-23T10:24:09.418471+0000","last_peered":"2024-04-23T10:24:09.418471+0000","last_clean":"2024-04-23T10:24:09.418471+0000","last_became_active":"2024-04-23T10:24:07.039128+0000","last_became_peered":"2024-04-23T10:24:07.039128+0000","last_unstale":"2024-04-23T10:24:09.418471+0000","last_undegraded":"2024-04-23T10:24:09.418471+0000","last_fullsized":"2024-04-23T10:24:09.418471+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T12:30:40.326031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:09.418766+0000","last_change":"2024-04-23T10:24:07.038133+0000","last_active":"2024-04-23T10:24:09.418766+0000","last_peered":"2024-04-23T10:24:09.418766+0000","last_clean":"2024-04-23T10:24:09.418766+0000","last_became_active":"2024-04-23T10:24:07.037817+0000","last_became_peered":"2024-04-23T10:24:07.037817+0000","last_unstale":"2024-04-23T10:24:09.418766+0000","last_undegraded":"2024-04-23T10:24:09.418766+0000","last_fullsized":"2024-04-23T10:24:09.418766+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:09:25.059986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T10:24:07.053140+0000","last_change":"2024-04-23T10:24:07.052395+0000","last_active":"2024-04-23T10:24:07.053140+0000","last_peered":"2024-04-23T10:24:07.053140+0000","last_clean":"2024-04-23T10:24:07.053140+0000","last_became_active":"2024-04-23T10:24:07.052023+0000","last_became_peered":"2024-04-23T10:24:07.052023+0000","last_unstale":"2024-04-23T10:24:07.053140+0000","last_undegraded":"2024-04-23T10:24:07.053140+0000","last_fullsized":"2024-04-23T10:24:07.053140+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T11:16:06.471157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.647369+0000","last_change":"2024-04-23T10:24:07.042739+0000","last_active":"2024-04-23T10:24:08.647369+0000","last_peered":"2024-04-23T10:24:08.647369+0000","last_clean":"2024-04-23T10:24:08.647369+0000","last_became_active":"2024-04-23T10:24:07.042292+0000","last_became_peered":"2024-04-23T10:24:07.042292+0000","last_unstale":"2024-04-23T10:24:08.647369+0000","last_undegraded":"2024-04-23T10:24:08.647369+0000","last_fullsized":"2024-04-23T10:24:08.647369+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:14:23.229032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.034816+0000","last_change":"2024-04-23T10:24:07.042792+0000","last_active":"2024-04-23T10:24:08.034816+0000","last_peered":"2024-04-23T10:24:08.034816+0000","last_clean":"2024-04-23T10:24:08.034816+0000","last_became_active":"2024-04-23T10:24:07.040846+0000","last_became_peered":"2024-04-23T10:24:07.040846+0000","last_unstale":"2024-04-23T10:24:08.034816+0000","last_undegraded":"2024-04-23T10:24:08.034816+0000","last_fullsized":"2024-04-23T10:24:08.034816+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T13:50:24.973286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.036473+0000","last_change":"2024-04-23T10:24:07.040391+0000","last_active":"2024-04-23T10:24:08.036473+0000","last_peered":"2024-04-23T10:24:08.036473+0000","last_clean":"2024-04-23T10:24:08.036473+0000","last_became_active":"2024-04-23T10:24:07.040002+0000","last_became_peered":"2024-04-23T10:24:07.040002+0000","last_unstale":"2024-04-23T10:24:08.036473+0000","last_undegraded":"2024-04-23T10:24:08.036473+0000","last_fullsized":"2024-04-23T10:24:08.036473+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T10:30:13.922315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.034656+0000","last_change":"2024-04-23T10:24:07.040678+0000","last_active":"2024-04-23T10:24:08.034656+0000","last_peered":"2024-04-23T10:24:08.034656+0000","last_clean":"2024-04-23T10:24:08.034656+0000","last_became_active":"2024-04-23T10:24:07.039551+0000","last_became_peered":"2024-04-23T10:24:07.039551+0000","last_unstale":"2024-04-23T10:24:08.034656+0000","last_undegraded":"2024-04-23T10:24:08.034656+0000","last_fullsized":"2024-04-23T10:24:08.034656+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:03:50.467083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:09.108508+0000","last_change":"2024-04-23T10:24:07.044618+0000","last_active":"2024-04-23T10:24:09.108508+0000","last_peered":"2024-04-23T10:24:09.108508+0000","last_clean":"2024-04-23T10:24:09.108508+0000","last_became_active":"2024-04-23T10:24:07.042803+0000","last_became_peered":"2024-04-23T10:24:07.042803+0000","last_unstale":"2024-04-23T10:24:09.108508+0000","last_undegraded":"2024-04-23T10:24:09.108508+0000","last_fullsized":"2024-04-23T10:24:09.108508+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:37:46.872177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:09.418746+0000","last_change":"2024-04-23T10:24:07.043724+0000","last_active":"2024-04-23T10:24:09.418746+0000","last_peered":"2024-04-23T10:24:09.418746+0000","last_clean":"2024-04-23T10:24:09.418746+0000","last_became_active":"2024-04-23T10:24:07.043408+0000","last_became_peered":"2024-04-23T10:24:07.043408+0000","last_unstale":"2024-04-23T10:24:09.418746+0000","last_undegraded":"2024-04-23T10:24:09.418746+0000","last_fullsized":"2024-04-23T10:24:09.418746+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:18:46.615093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T10:24:07.045017+0000","last_change":"2024-04-23T10:24:07.043794+0000","last_active":"2024-04-23T10:24:07.045017+0000","last_peered":"2024-04-23T10:24:07.045017+0000","last_clean":"2024-04-23T10:24:07.045017+0000","last_became_active":"2024-04-23T10:24:07.043391+0000","last_became_peered":"2024-04-23T10:24:07.043391+0000","last_unstale":"2024-04-23T10:24:07.045017+0000","last_undegraded":"2024-04-23T10:24:07.045017+0000","last_fullsized":"2024-04-23T10:24:07.045017+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:43:56.783899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280003,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2702,"num_objects":42,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":99,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":208896,"data_stored":12911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350990,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":433002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612627,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1024000,"data_stored":1035254,"data_compressed":35615,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776152,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":442480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037790,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":933888,"data_stored":442198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299427,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":983040,"data_stored":1033144,"data_compressed":35615,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462952,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":909312,"data_stored":433002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":933888,"data_stored":443184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738438,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":950272,"data_stored":443050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411386,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":443218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149751,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":929792,"data_stored":441676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888114,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":925696,"data_stored":441594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724588,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":983040,"data_stored":1032122,"data_compressed":35615,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33976,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33976,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33976,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35615,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35615,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35615,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35615,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35615,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":3705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T10:24:14.335 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T10:24:14.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.471+0000 7fc026ee8640 1 -- 172.21.15.97:0/1337734164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc020101fe0 msgr2=0x7fc020102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:14.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.471+0000 7fc026ee8640 1 --2- 172.21.15.97:0/1337734164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc020101fe0 0x7fc020102460 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc008009900 tx=0x7fc00802a410 comp rx=0 tx=0).stop 2024-04-23T10:24:14.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.471+0000 7fc026ee8640 1 -- 172.21.15.97:0/1337734164 shutdown_connections 2024-04-23T10:24:14.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.471+0000 7fc026ee8640 1 --2- 172.21.15.97:0/1337734164 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0201029a0 0x7fc0201092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:14.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.471+0000 7fc026ee8640 1 --2- 172.21.15.97:0/1337734164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc020101fe0 0x7fc020102460 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:14.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.471+0000 7fc026ee8640 1 --2- 172.21.15.97:0/1337734164 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc020100e90 0x7fc0201012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:14.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.471+0000 7fc026ee8640 1 -- 172.21.15.97:0/1337734164 >> 172.21.15.97:0/1337734164 conn(0x7fc0200fc400 msgr2=0x7fc0200fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:14.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.471+0000 7fc026ee8640 1 -- 172.21.15.97:0/1337734164 shutdown_connections 2024-04-23T10:24:14.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.471+0000 7fc026ee8640 1 -- 172.21.15.97:0/1337734164 wait complete. 2024-04-23T10:24:14.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.471+0000 7fc026ee8640 1 Processor -- start 2024-04-23T10:24:14.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.471+0000 7fc026ee8640 1 -- start start 2024-04-23T10:24:14.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.471+0000 7fc026ee8640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc020100e90 0x7fc020104e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:14.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.471+0000 7fc026ee8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc020101fe0 0x7fc020103450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:14.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.471+0000 7fc026ee8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0201029a0 0x7fc020103990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:14.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.471+0000 7fc026ee8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc02010c9e0 con 0x7fc0201029a0 2024-04-23T10:24:14.472 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.471+0000 7fc026ee8640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc02010c860 con 0x7fc020100e90 2024-04-23T10:24:14.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.471+0000 7fc026ee8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc02010cb60 con 0x7fc020101fe0 2024-04-23T10:24:14.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.471+0000 7fc025485640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0201029a0 0x7fc020103990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:14.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.471+0000 7fc024c84640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc020100e90 0x7fc020104e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:14.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.471+0000 7fc025485640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0201029a0 0x7fc020103990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33096/0 (socket says 172.21.15.97:33096) 2024-04-23T10:24:14.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.471+0000 7fc025485640 1 -- 172.21.15.97:0/3626809705 learned_addr learned my addr 172.21.15.97:0/3626809705 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:24:14.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.472+0000 7fc01ffff640 1 --2- 172.21.15.97:0/3626809705 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc020101fe0 0x7fc020103450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:14.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.472+0000 7fc025485640 1 -- 172.21.15.97:0/3626809705 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc020101fe0 msgr2=0x7fc020103450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:14.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.472+0000 7fc025485640 1 --2- 172.21.15.97:0/3626809705 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc020101fe0 0x7fc020103450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:14.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.472+0000 7fc025485640 1 -- 172.21.15.97:0/3626809705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc020100e90 msgr2=0x7fc020104e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:14.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.472+0000 7fc025485640 1 --2- 172.21.15.97:0/3626809705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc020100e90 0x7fc020104e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:14.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.472+0000 7fc025485640 1 -- 172.21.15.97:0/3626809705 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc020104220 con 0x7fc0201029a0 2024-04-23T10:24:14.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.472+0000 7fc01ffff640 1 --2- 172.21.15.97:0/3626809705 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc020101fe0 0x7fc020103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:24:14.473 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.472+0000 7fc025485640 1 --2- 172.21.15.97:0/3626809705 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0201029a0 0x7fc020103990 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc01800eaa0 tx=0x7fc01800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:14.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.472+0000 7fc01dffb640 1 -- 172.21.15.97:0/3626809705 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc01800cdb0 con 0x7fc0201029a0 2024-04-23T10:24:14.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.472+0000 7fc026ee8640 1 -- 172.21.15.97:0/3626809705 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0201a2320 con 0x7fc0201029a0 2024-04-23T10:24:14.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.472+0000 7fc01dffb640 1 -- 172.21.15.97:0/3626809705 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc018005020 con 0x7fc0201029a0 2024-04-23T10:24:14.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.472+0000 7fc01dffb640 1 -- 172.21.15.97:0/3626809705 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0180116d0 con 0x7fc0201029a0 2024-04-23T10:24:14.474 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.472+0000 7fc026ee8640 1 -- 172.21.15.97:0/3626809705 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0201a2890 con 0x7fc0201029a0 2024-04-23T10:24:14.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.473+0000 7fc026ee8640 1 -- 172.21.15.97:0/3626809705 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0201a2bb0 con 0x7fc0201029a0 2024-04-23T10:24:14.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.474+0000 7fc01dffb640 1 -- 172.21.15.97:0/3626809705 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc0180054a0 con 0x7fc0201029a0 2024-04-23T10:24:14.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.475+0000 7fc01dffb640 1 --2- 172.21.15.97:0/3626809705 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fc00c0a3bb0 0x7fc00c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:14.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.475+0000 7fc024c84640 1 --2- 172.21.15.97:0/3626809705 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fc00c0a3bb0 0x7fc00c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:14.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.475+0000 7fc01dffb640 1 -- 172.21.15.97:0/3626809705 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fc0180d0890 con 0x7fc0201029a0 2024-04-23T10:24:14.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.475+0000 7fc024c84640 1 --2- 172.21.15.97:0/3626809705 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fc00c0a3bb0 0x7fc00c0a6070 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc010009bd0 tx=0x7fc010005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:14.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.478+0000 7fc01dffb640 1 -- 172.21.15.97:0/3626809705 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc018016030 con 0x7fc0201029a0 2024-04-23T10:24:14.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.588+0000 7fc01dffb640 1 -- 172.21.15.97:0/3626809705 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc018099330 con 0x7fc0201029a0 2024-04-23T10:24:14.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:14 smithi097 ceph-mon[29197]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:24:14.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:14 smithi097 ceph-mon[29197]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:24:14.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:14 smithi097 ceph-mon[29197]: pgmap v293: 97 pgs: 97 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 731 B/s rd, 2.7 KiB/s wr, 14 op/s 2024-04-23T10:24:14.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:14 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/705220008' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:24:14.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:14 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/2712961394' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T10:24:14.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.663+0000 7fc026ee8640 1 -- 172.21.15.97:0/3626809705 --> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc0201a2e40 con 0x7fc00c0a3bb0 2024-04-23T10:24:14.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.670+0000 7fc01dffb640 1 -- 172.21.15.97:0/3626809705 <== mgr.14152 v2:172.21.15.97:6800/3149043749 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258901 (secure 0 0 0) 0x7fc0201a2e40 con 0x7fc00c0a3bb0 2024-04-23T10:24:14.671 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:24:14.672 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-04-23T10:24:14.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.673+0000 7fc026ee8640 1 -- 172.21.15.97:0/3626809705 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fc00c0a3bb0 msgr2=0x7fc00c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:14.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.673+0000 7fc026ee8640 1 --2- 172.21.15.97:0/3626809705 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fc00c0a3bb0 0x7fc00c0a6070 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc010009bd0 tx=0x7fc010005f90 comp rx=0 tx=0).stop 2024-04-23T10:24:14.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.673+0000 7fc026ee8640 1 -- 172.21.15.97:0/3626809705 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0201029a0 msgr2=0x7fc020103990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:14.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.673+0000 7fc026ee8640 1 --2- 172.21.15.97:0/3626809705 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0201029a0 0x7fc020103990 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc01800eaa0 tx=0x7fc01800ef60 comp rx=0 tx=0).stop 2024-04-23T10:24:14.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.673+0000 7fc026ee8640 1 -- 172.21.15.97:0/3626809705 shutdown_connections 2024-04-23T10:24:14.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.673+0000 7fc026ee8640 1 --2- 172.21.15.97:0/3626809705 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fc00c0a3bb0 0x7fc00c0a6070 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:14.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.673+0000 7fc026ee8640 1 --2- 172.21.15.97:0/3626809705 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0201029a0 0x7fc020103990 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:14.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.673+0000 7fc026ee8640 1 --2- 172.21.15.97:0/3626809705 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc020101fe0 0x7fc020103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:14.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.673+0000 7fc026ee8640 1 --2- 172.21.15.97:0/3626809705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc020100e90 0x7fc020104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:14.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.673+0000 7fc026ee8640 1 -- 172.21.15.97:0/3626809705 >> 172.21.15.97:0/3626809705 conn(0x7fc0200fc400 msgr2=0x7fc0200fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:14.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.673+0000 7fc026ee8640 1 -- 172.21.15.97:0/3626809705 shutdown_connections 2024-04-23T10:24:14.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:14.673+0000 7fc026ee8640 1 -- 172.21.15.97:0/3626809705 wait complete. 2024-04-23T10:24:14.683 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2024-04-23T10:24:13.554081+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":593070,"num_objects":44,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":142,"num_read_kb":124,"num_write":280,"num_write_kb":2957,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":218,"ondisk_log_size":218,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":221,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360308,"kb_used_data":11100,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581516,"statfs":{"total":1151940427776,"available":1151571472384,"internally_reserved":0,"allocated":11366400,"data_stored":7063924,"data_compressed":106845,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2702,"num_objects":42,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":99,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.000127"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T10:24:07.041388+0000","last_change":"2024-04-23T10:24:07.040397+0000","last_active":"2024-04-23T10:24:07.041388+0000","last_peered":"2024-04-23T10:24:07.041388+0000","last_clean":"2024-04-23T10:24:07.041388+0000","last_became_active":"2024-04-23T10:24:07.040007+0000","last_became_peered":"2024-04-23T10:24:07.040007+0000","last_unstale":"2024-04-23T10:24:07.041388+0000","last_undegraded":"2024-04-23T10:24:07.041388+0000","last_fullsized":"2024-04-23T10:24:07.041388+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:34:16.967983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:09.418511+0000","last_change":"2024-04-23T10:24:07.044672+0000","last_active":"2024-04-23T10:24:09.418511+0000","last_peered":"2024-04-23T10:24:09.418511+0000","last_clean":"2024-04-23T10:24:09.418511+0000","last_became_active":"2024-04-23T10:24:07.044184+0000","last_became_peered":"2024-04-23T10:24:07.044184+0000","last_unstale":"2024-04-23T10:24:09.418511+0000","last_undegraded":"2024-04-23T10:24:09.418511+0000","last_fullsized":"2024-04-23T10:24:09.418511+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:07:16.729205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:09.418411+0000","last_change":"2024-04-23T10:24:07.041959+0000","last_active":"2024-04-23T10:24:09.418411+0000","last_peered":"2024-04-23T10:24:09.418411+0000","last_clean":"2024-04-23T10:24:09.418411+0000","last_became_active":"2024-04-23T10:24:07.041661+0000","last_became_peered":"2024-04-23T10:24:07.041661+0000","last_unstale":"2024-04-23T10:24:09.418411+0000","last_undegraded":"2024-04-23T10:24:09.418411+0000","last_fullsized":"2024-04-23T10:24:09.418411+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:23:14.694183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.030650+0000","last_change":"2024-04-23T10:24:07.048334+0000","last_active":"2024-04-23T10:24:08.030650+0000","last_peered":"2024-04-23T10:24:08.030650+0000","last_clean":"2024-04-23T10:24:08.030650+0000","last_became_active":"2024-04-23T10:24:07.047506+0000","last_became_peered":"2024-04-23T10:24:07.047506+0000","last_unstale":"2024-04-23T10:24:08.030650+0000","last_undegraded":"2024-04-23T10:24:08.030650+0000","last_fullsized":"2024-04-23T10:24:08.030650+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:27:27.674287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.561291+0000","last_change":"2024-04-23T10:24:07.035531+0000","last_active":"2024-04-23T10:24:08.561291+0000","last_peered":"2024-04-23T10:24:08.561291+0000","last_clean":"2024-04-23T10:24:08.561291+0000","last_became_active":"2024-04-23T10:24:07.035167+0000","last_became_peered":"2024-04-23T10:24:07.035167+0000","last_unstale":"2024-04-23T10:24:08.561291+0000","last_undegraded":"2024-04-23T10:24:08.561291+0000","last_fullsized":"2024-04-23T10:24:08.561291+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:39:17.198650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.787190+0000","last_change":"2024-04-23T10:24:07.040593+0000","last_active":"2024-04-23T10:24:08.787190+0000","last_peered":"2024-04-23T10:24:08.787190+0000","last_clean":"2024-04-23T10:24:08.787190+0000","last_became_active":"2024-04-23T10:24:07.040255+0000","last_became_peered":"2024-04-23T10:24:07.040255+0000","last_unstale":"2024-04-23T10:24:08.787190+0000","last_undegraded":"2024-04-23T10:24:08.787190+0000","last_fullsized":"2024-04-23T10:24:08.787190+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:05:52.663566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.101104+0000","last_change":"2024-04-23T10:24:07.051688+0000","last_active":"2024-04-23T10:24:08.101104+0000","last_peered":"2024-04-23T10:24:08.101104+0000","last_clean":"2024-04-23T10:24:08.101104+0000","last_became_active":"2024-04-23T10:24:07.050265+0000","last_became_peered":"2024-04-23T10:24:07.050265+0000","last_unstale":"2024-04-23T10:24:08.101104+0000","last_undegraded":"2024-04-23T10:24:08.101104+0000","last_fullsized":"2024-04-23T10:24:08.101104+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T13:13:36.126183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.034321+0000","last_change":"2024-04-23T10:24:07.045937+0000","last_active":"2024-04-23T10:24:08.034321+0000","last_peered":"2024-04-23T10:24:08.034321+0000","last_clean":"2024-04-23T10:24:08.034321+0000","last_became_active":"2024-04-23T10:24:07.045616+0000","last_became_peered":"2024-04-23T10:24:07.045616+0000","last_unstale":"2024-04-23T10:24:08.034321+0000","last_undegraded":"2024-04-23T10:24:08.034321+0000","last_fullsized":"2024-04-23T10:24:08.034321+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T10:45:27.250031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:09.108827+0000","last_change":"2024-04-23T10:24:07.039294+0000","last_active":"2024-04-23T10:24:09.108827+0000","last_peered":"2024-04-23T10:24:09.108827+0000","last_clean":"2024-04-23T10:24:09.108827+0000","last_became_active":"2024-04-23T10:24:07.038959+0000","last_became_peered":"2024-04-23T10:24:07.038959+0000","last_unstale":"2024-04-23T10:24:09.108827+0000","last_undegraded":"2024-04-23T10:24:09.108827+0000","last_fullsized":"2024-04-23T10:24:09.108827+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:53:41.345062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.787352+0000","last_change":"2024-04-23T10:24:07.044177+0000","last_active":"2024-04-23T10:24:08.787352+0000","last_peered":"2024-04-23T10:24:08.787352+0000","last_clean":"2024-04-23T10:24:08.787352+0000","last_became_active":"2024-04-23T10:24:07.043821+0000","last_became_peered":"2024-04-23T10:24:07.043821+0000","last_unstale":"2024-04-23T10:24:08.787352+0000","last_undegraded":"2024-04-23T10:24:08.787352+0000","last_fullsized":"2024-04-23T10:24:08.787352+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:09:38.782407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.101477+0000","last_change":"2024-04-23T10:24:07.052775+0000","last_active":"2024-04-23T10:24:08.101477+0000","last_peered":"2024-04-23T10:24:08.101477+0000","last_clean":"2024-04-23T10:24:08.101477+0000","last_became_active":"2024-04-23T10:24:07.051708+0000","last_became_peered":"2024-04-23T10:24:07.051708+0000","last_unstale":"2024-04-23T10:24:08.101477+0000","last_undegraded":"2024-04-23T10:24:08.101477+0000","last_fullsized":"2024-04-23T10:24:08.101477+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:47:55.360467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.030608+0000","last_change":"2024-04-23T10:24:07.048655+0000","last_active":"2024-04-23T10:24:08.030608+0000","last_peered":"2024-04-23T10:24:08.030608+0000","last_clean":"2024-04-23T10:24:08.030608+0000","last_became_active":"2024-04-23T10:24:07.047994+0000","last_became_peered":"2024-04-23T10:24:07.047994+0000","last_unstale":"2024-04-23T10:24:08.030608+0000","last_undegraded":"2024-04-23T10:24:08.030608+0000","last_fullsized":"2024-04-23T10:24:08.030608+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T10:25:51.425871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:09.418838+0000","last_change":"2024-04-23T10:24:07.036319+0000","last_active":"2024-04-23T10:24:09.418838+0000","last_peered":"2024-04-23T10:24:09.418838+0000","last_clean":"2024-04-23T10:24:09.418838+0000","last_became_active":"2024-04-23T10:24:07.035982+0000","last_became_peered":"2024-04-23T10:24:07.035982+0000","last_unstale":"2024-04-23T10:24:09.418838+0000","last_undegraded":"2024-04-23T10:24:09.418838+0000","last_fullsized":"2024-04-23T10:24:09.418838+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T10:25:46.191118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:09.417987+0000","last_change":"2024-04-23T10:24:07.033166+0000","last_active":"2024-04-23T10:24:09.417987+0000","last_peered":"2024-04-23T10:24:09.417987+0000","last_clean":"2024-04-23T10:24:09.417987+0000","last_became_active":"2024-04-23T10:24:07.032807+0000","last_became_peered":"2024-04-23T10:24:07.032807+0000","last_unstale":"2024-04-23T10:24:09.417987+0000","last_undegraded":"2024-04-23T10:24:09.417987+0000","last_fullsized":"2024-04-23T10:24:09.417987+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:52:35.654965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:09.418040+0000","last_change":"2024-04-23T10:24:07.041988+0000","last_active":"2024-04-23T10:24:09.418040+0000","last_peered":"2024-04-23T10:24:09.418040+0000","last_clean":"2024-04-23T10:24:09.418040+0000","last_became_active":"2024-04-23T10:24:07.041638+0000","last_became_peered":"2024-04-23T10:24:07.041638+0000","last_unstale":"2024-04-23T10:24:09.418040+0000","last_undegraded":"2024-04-23T10:24:09.418040+0000","last_fullsized":"2024-04-23T10:24:09.418040+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:17:40.565981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:09.108294+0000","last_change":"2024-04-23T10:24:07.045111+0000","last_active":"2024-04-23T10:24:09.108294+0000","last_peered":"2024-04-23T10:24:09.108294+0000","last_clean":"2024-04-23T10:24:09.108294+0000","last_became_active":"2024-04-23T10:24:07.044249+0000","last_became_peered":"2024-04-23T10:24:07.044249+0000","last_unstale":"2024-04-23T10:24:09.108294+0000","last_undegraded":"2024-04-23T10:24:09.108294+0000","last_fullsized":"2024-04-23T10:24:09.108294+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:08:36.029636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T10:24:07.041408+0000","last_change":"2024-04-23T10:24:07.040685+0000","last_active":"2024-04-23T10:24:07.041408+0000","last_peered":"2024-04-23T10:24:07.041408+0000","last_clean":"2024-04-23T10:24:07.041408+0000","last_became_active":"2024-04-23T10:24:07.038881+0000","last_became_peered":"2024-04-23T10:24:07.038881+0000","last_unstale":"2024-04-23T10:24:07.041408+0000","last_undegraded":"2024-04-23T10:24:07.041408+0000","last_fullsized":"2024-04-23T10:24:07.041408+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T11:27:08.211643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.392577+0000","last_change":"2024-04-23T10:24:07.041815+0000","last_active":"2024-04-23T10:24:08.392577+0000","last_peered":"2024-04-23T10:24:08.392577+0000","last_clean":"2024-04-23T10:24:08.392577+0000","last_became_active":"2024-04-23T10:24:07.041495+0000","last_became_peered":"2024-04-23T10:24:07.041495+0000","last_unstale":"2024-04-23T10:24:08.392577+0000","last_undegraded":"2024-04-23T10:24:08.392577+0000","last_fullsized":"2024-04-23T10:24:08.392577+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:29:02.006394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.647308+0000","last_change":"2024-04-23T10:24:07.041722+0000","last_active":"2024-04-23T10:24:08.647308+0000","last_peered":"2024-04-23T10:24:08.647308+0000","last_clean":"2024-04-23T10:24:08.647308+0000","last_became_active":"2024-04-23T10:24:07.039434+0000","last_became_peered":"2024-04-23T10:24:07.039434+0000","last_unstale":"2024-04-23T10:24:08.647308+0000","last_undegraded":"2024-04-23T10:24:08.647308+0000","last_fullsized":"2024-04-23T10:24:08.647308+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:03:44.033094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'13","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:11.975075+0000","last_change":"2024-04-23T10:24:06.576664+0000","last_active":"2024-04-23T10:24:11.975075+0000","last_peered":"2024-04-23T10:24:11.975075+0000","last_clean":"2024-04-23T10:24:11.975075+0000","last_became_active":"2024-04-23T10:24:06.576058+0000","last_became_peered":"2024-04-23T10:24:06.576058+0000","last_unstale":"2024-04-23T10:24:11.975075+0000","last_undegraded":"2024-04-23T10:24:11.975075+0000","last_fullsized":"2024-04-23T10:24:11.975075+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:39:57.657026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.034541+0000","last_change":"2024-04-23T10:24:07.047192+0000","last_active":"2024-04-23T10:24:08.034541+0000","last_peered":"2024-04-23T10:24:08.034541+0000","last_clean":"2024-04-23T10:24:08.034541+0000","last_became_active":"2024-04-23T10:24:07.046750+0000","last_became_peered":"2024-04-23T10:24:07.046750+0000","last_unstale":"2024-04-23T10:24:08.034541+0000","last_undegraded":"2024-04-23T10:24:08.034541+0000","last_fullsized":"2024-04-23T10:24:08.034541+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:13:09.399425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.034754+0000","last_change":"2024-04-23T10:24:06.577588+0000","last_active":"2024-04-23T10:24:08.034754+0000","last_peered":"2024-04-23T10:24:08.034754+0000","last_clean":"2024-04-23T10:24:08.034754+0000","last_became_active":"2024-04-23T10:24:06.576971+0000","last_became_peered":"2024-04-23T10:24:06.576971+0000","last_unstale":"2024-04-23T10:24:08.034754+0000","last_undegraded":"2024-04-23T10:24:08.034754+0000","last_fullsized":"2024-04-23T10:24:08.034754+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:07:21.577868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:09.418461+0000","last_change":"2024-04-23T10:24:07.030666+0000","last_active":"2024-04-23T10:24:09.418461+0000","last_peered":"2024-04-23T10:24:09.418461+0000","last_clean":"2024-04-23T10:24:09.418461+0000","last_became_active":"2024-04-23T10:24:07.030342+0000","last_became_peered":"2024-04-23T10:24:07.030342+0000","last_unstale":"2024-04-23T10:24:09.418461+0000","last_undegraded":"2024-04-23T10:24:09.418461+0000","last_fullsized":"2024-04-23T10:24:09.418461+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:09:14.818638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.561639+0000","last_change":"2024-04-23T10:24:06.576042+0000","last_active":"2024-04-23T10:24:08.561639+0000","last_peered":"2024-04-23T10:24:08.561639+0000","last_clean":"2024-04-23T10:24:08.561639+0000","last_became_active":"2024-04-23T10:24:06.574518+0000","last_became_peered":"2024-04-23T10:24:06.574518+0000","last_unstale":"2024-04-23T10:24:08.561639+0000","last_undegraded":"2024-04-23T10:24:08.561639+0000","last_fullsized":"2024-04-23T10:24:08.561639+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T11:56:40.100213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:09.108402+0000","last_change":"2024-04-23T10:24:07.046591+0000","last_active":"2024-04-23T10:24:09.108402+0000","last_peered":"2024-04-23T10:24:09.108402+0000","last_clean":"2024-04-23T10:24:09.108402+0000","last_became_active":"2024-04-23T10:24:07.046208+0000","last_became_peered":"2024-04-23T10:24:07.046208+0000","last_unstale":"2024-04-23T10:24:09.108402+0000","last_undegraded":"2024-04-23T10:24:09.108402+0000","last_fullsized":"2024-04-23T10:24:09.108402+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:27:37.711329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.561525+0000","last_change":"2024-04-23T10:24:06.568185+0000","last_active":"2024-04-23T10:24:08.561525+0000","last_peered":"2024-04-23T10:24:08.561525+0000","last_clean":"2024-04-23T10:24:08.561525+0000","last_became_active":"2024-04-23T10:24:06.567841+0000","last_became_peered":"2024-04-23T10:24:06.567841+0000","last_unstale":"2024-04-23T10:24:08.561525+0000","last_undegraded":"2024-04-23T10:24:08.561525+0000","last_fullsized":"2024-04-23T10:24:08.561525+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:29:30.214779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.036335+0000","last_change":"2024-04-23T10:24:07.047926+0000","last_active":"2024-04-23T10:24:08.036335+0000","last_peered":"2024-04-23T10:24:08.036335+0000","last_clean":"2024-04-23T10:24:08.036335+0000","last_became_active":"2024-04-23T10:24:07.047273+0000","last_became_peered":"2024-04-23T10:24:07.047273+0000","last_unstale":"2024-04-23T10:24:08.036335+0000","last_undegraded":"2024-04-23T10:24:08.036335+0000","last_fullsized":"2024-04-23T10:24:08.036335+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:37:25.068034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.036101+0000","last_change":"2024-04-23T10:24:06.574733+0000","last_active":"2024-04-23T10:24:08.036101+0000","last_peered":"2024-04-23T10:24:08.036101+0000","last_clean":"2024-04-23T10:24:08.036101+0000","last_became_active":"2024-04-23T10:24:06.574276+0000","last_became_peered":"2024-04-23T10:24:06.574276+0000","last_unstale":"2024-04-23T10:24:08.036101+0000","last_undegraded":"2024-04-23T10:24:08.036101+0000","last_fullsized":"2024-04-23T10:24:08.036101+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:27:57.080635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.561307+0000","last_change":"2024-04-23T10:24:07.040110+0000","last_active":"2024-04-23T10:24:08.561307+0000","last_peered":"2024-04-23T10:24:08.561307+0000","last_clean":"2024-04-23T10:24:08.561307+0000","last_became_active":"2024-04-23T10:24:07.039769+0000","last_became_peered":"2024-04-23T10:24:07.039769+0000","last_unstale":"2024-04-23T10:24:08.561307+0000","last_undegraded":"2024-04-23T10:24:08.561307+0000","last_fullsized":"2024-04-23T10:24:08.561307+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:10:07.884895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:10.961020+0000","last_change":"2024-04-23T10:24:06.567448+0000","last_active":"2024-04-23T10:24:10.961020+0000","last_peered":"2024-04-23T10:24:10.961020+0000","last_clean":"2024-04-23T10:24:10.961020+0000","last_became_active":"2024-04-23T10:24:06.566958+0000","last_became_peered":"2024-04-23T10:24:06.566958+0000","last_unstale":"2024-04-23T10:24:10.961020+0000","last_undegraded":"2024-04-23T10:24:10.961020+0000","last_fullsized":"2024-04-23T10:24:10.961020+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:45:52.863596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.036371+0000","last_change":"2024-04-23T10:24:07.040754+0000","last_active":"2024-04-23T10:24:08.036371+0000","last_peered":"2024-04-23T10:24:08.036371+0000","last_clean":"2024-04-23T10:24:08.036371+0000","last_became_active":"2024-04-23T10:24:07.040397+0000","last_became_peered":"2024-04-23T10:24:07.040397+0000","last_unstale":"2024-04-23T10:24:08.036371+0000","last_undegraded":"2024-04-23T10:24:08.036371+0000","last_fullsized":"2024-04-23T10:24:08.036371+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T11:01:03.626506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:09.418383+0000","last_change":"2024-04-23T10:24:06.568592+0000","last_active":"2024-04-23T10:24:09.418383+0000","last_peered":"2024-04-23T10:24:09.418383+0000","last_clean":"2024-04-23T10:24:09.418383+0000","last_became_active":"2024-04-23T10:24:06.568192+0000","last_became_peered":"2024-04-23T10:24:06.568192+0000","last_unstale":"2024-04-23T10:24:09.418383+0000","last_undegraded":"2024-04-23T10:24:09.418383+0000","last_fullsized":"2024-04-23T10:24:09.418383+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:14:42.589596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.034809+0000","last_change":"2024-04-23T10:24:07.037058+0000","last_active":"2024-04-23T10:24:08.034809+0000","last_peered":"2024-04-23T10:24:08.034809+0000","last_clean":"2024-04-23T10:24:08.034809+0000","last_became_active":"2024-04-23T10:24:07.036659+0000","last_became_peered":"2024-04-23T10:24:07.036659+0000","last_unstale":"2024-04-23T10:24:08.034809+0000","last_undegraded":"2024-04-23T10:24:08.034809+0000","last_fullsized":"2024-04-23T10:24:08.034809+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T12:53:49.298603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.035915+0000","last_change":"2024-04-23T10:24:06.576900+0000","last_active":"2024-04-23T10:24:08.035915+0000","last_peered":"2024-04-23T10:24:08.035915+0000","last_clean":"2024-04-23T10:24:08.035915+0000","last_became_active":"2024-04-23T10:24:06.575627+0000","last_became_peered":"2024-04-23T10:24:06.575627+0000","last_unstale":"2024-04-23T10:24:08.035915+0000","last_undegraded":"2024-04-23T10:24:08.035915+0000","last_fullsized":"2024-04-23T10:24:08.035915+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:16:07.973841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.392415+0000","last_change":"2024-04-23T10:24:07.035294+0000","last_active":"2024-04-23T10:24:08.392415+0000","last_peered":"2024-04-23T10:24:08.392415+0000","last_clean":"2024-04-23T10:24:08.392415+0000","last_became_active":"2024-04-23T10:24:07.035001+0000","last_became_peered":"2024-04-23T10:24:07.035001+0000","last_unstale":"2024-04-23T10:24:08.392415+0000","last_undegraded":"2024-04-23T10:24:08.392415+0000","last_fullsized":"2024-04-23T10:24:08.392415+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T13:12:47.013132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:10.952763+0000","last_change":"2024-04-23T10:24:06.573637+0000","last_active":"2024-04-23T10:24:10.952763+0000","last_peered":"2024-04-23T10:24:10.952763+0000","last_clean":"2024-04-23T10:24:10.952763+0000","last_became_active":"2024-04-23T10:24:06.573236+0000","last_became_peered":"2024-04-23T10:24:06.573236+0000","last_unstale":"2024-04-23T10:24:10.952763+0000","last_undegraded":"2024-04-23T10:24:10.952763+0000","last_fullsized":"2024-04-23T10:24:10.952763+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T12:48:06.740181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:09.108659+0000","last_change":"2024-04-23T10:24:07.037457+0000","last_active":"2024-04-23T10:24:09.108659+0000","last_peered":"2024-04-23T10:24:09.108659+0000","last_clean":"2024-04-23T10:24:09.108659+0000","last_became_active":"2024-04-23T10:24:07.036456+0000","last_became_peered":"2024-04-23T10:24:07.036456+0000","last_unstale":"2024-04-23T10:24:09.108659+0000","last_undegraded":"2024-04-23T10:24:09.108659+0000","last_fullsized":"2024-04-23T10:24:09.108659+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:42:57.750435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:09.576356+0000","last_change":"2024-04-23T10:24:06.575317+0000","last_active":"2024-04-23T10:24:09.576356+0000","last_peered":"2024-04-23T10:24:09.576356+0000","last_clean":"2024-04-23T10:24:09.576356+0000","last_became_active":"2024-04-23T10:24:06.574935+0000","last_became_peered":"2024-04-23T10:24:06.574935+0000","last_unstale":"2024-04-23T10:24:09.576356+0000","last_undegraded":"2024-04-23T10:24:09.576356+0000","last_fullsized":"2024-04-23T10:24:09.576356+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T11:03:47.864370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:09.418390+0000","last_change":"2024-04-23T10:24:07.040317+0000","last_active":"2024-04-23T10:24:09.418390+0000","last_peered":"2024-04-23T10:24:09.418390+0000","last_clean":"2024-04-23T10:24:09.418390+0000","last_became_active":"2024-04-23T10:24:07.039998+0000","last_became_peered":"2024-04-23T10:24:07.039998+0000","last_unstale":"2024-04-23T10:24:09.418390+0000","last_undegraded":"2024-04-23T10:24:09.418390+0000","last_fullsized":"2024-04-23T10:24:09.418390+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:50:30.412127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.036223+0000","last_change":"2024-04-23T10:24:06.570019+0000","last_active":"2024-04-23T10:24:08.036223+0000","last_peered":"2024-04-23T10:24:08.036223+0000","last_clean":"2024-04-23T10:24:08.036223+0000","last_became_active":"2024-04-23T10:24:06.569676+0000","last_became_peered":"2024-04-23T10:24:06.569676+0000","last_unstale":"2024-04-23T10:24:08.036223+0000","last_undegraded":"2024-04-23T10:24:08.036223+0000","last_fullsized":"2024-04-23T10:24:08.036223+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:31:46.994787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:09.108524+0000","last_change":"2024-04-23T10:24:07.040183+0000","last_active":"2024-04-23T10:24:09.108524+0000","last_peered":"2024-04-23T10:24:09.108524+0000","last_clean":"2024-04-23T10:24:09.108524+0000","last_became_active":"2024-04-23T10:24:07.039866+0000","last_became_peered":"2024-04-23T10:24:07.039866+0000","last_unstale":"2024-04-23T10:24:09.108524+0000","last_undegraded":"2024-04-23T10:24:09.108524+0000","last_fullsized":"2024-04-23T10:24:09.108524+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:33:04.411765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:10.953348+0000","last_change":"2024-04-23T10:24:06.568883+0000","last_active":"2024-04-23T10:24:10.953348+0000","last_peered":"2024-04-23T10:24:10.953348+0000","last_clean":"2024-04-23T10:24:10.953348+0000","last_became_active":"2024-04-23T10:24:06.568517+0000","last_became_peered":"2024-04-23T10:24:06.568517+0000","last_unstale":"2024-04-23T10:24:10.953348+0000","last_undegraded":"2024-04-23T10:24:10.953348+0000","last_fullsized":"2024-04-23T10:24:10.953348+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:51:42.440346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.561458+0000","last_change":"2024-04-23T10:24:07.046839+0000","last_active":"2024-04-23T10:24:08.561458+0000","last_peered":"2024-04-23T10:24:08.561458+0000","last_clean":"2024-04-23T10:24:08.561458+0000","last_became_active":"2024-04-23T10:24:07.046512+0000","last_became_peered":"2024-04-23T10:24:07.046512+0000","last_unstale":"2024-04-23T10:24:08.561458+0000","last_undegraded":"2024-04-23T10:24:08.561458+0000","last_fullsized":"2024-04-23T10:24:08.561458+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T12:42:11.822100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:09.577162+0000","last_change":"2024-04-23T10:24:06.578087+0000","last_active":"2024-04-23T10:24:09.577162+0000","last_peered":"2024-04-23T10:24:09.577162+0000","last_clean":"2024-04-23T10:24:09.577162+0000","last_became_active":"2024-04-23T10:24:06.577666+0000","last_became_peered":"2024-04-23T10:24:06.577666+0000","last_unstale":"2024-04-23T10:24:09.577162+0000","last_undegraded":"2024-04-23T10:24:09.577162+0000","last_fullsized":"2024-04-23T10:24:09.577162+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:55:15.053303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:09.418570+0000","last_change":"2024-04-23T10:24:07.043672+0000","last_active":"2024-04-23T10:24:09.418570+0000","last_peered":"2024-04-23T10:24:09.418570+0000","last_clean":"2024-04-23T10:24:09.418570+0000","last_became_active":"2024-04-23T10:24:07.043346+0000","last_became_peered":"2024-04-23T10:24:07.043346+0000","last_unstale":"2024-04-23T10:24:09.418570+0000","last_undegraded":"2024-04-23T10:24:09.418570+0000","last_fullsized":"2024-04-23T10:24:09.418570+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T13:21:56.751028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.035934+0000","last_change":"2024-04-23T10:24:06.576607+0000","last_active":"2024-04-23T10:24:08.035934+0000","last_peered":"2024-04-23T10:24:08.035934+0000","last_clean":"2024-04-23T10:24:08.035934+0000","last_became_active":"2024-04-23T10:24:06.576213+0000","last_became_peered":"2024-04-23T10:24:06.576213+0000","last_unstale":"2024-04-23T10:24:08.035934+0000","last_undegraded":"2024-04-23T10:24:08.035934+0000","last_fullsized":"2024-04-23T10:24:08.035934+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:09:14.918936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.101213+0000","last_change":"2024-04-23T10:24:07.052641+0000","last_active":"2024-04-23T10:24:08.101213+0000","last_peered":"2024-04-23T10:24:08.101213+0000","last_clean":"2024-04-23T10:24:08.101213+0000","last_became_active":"2024-04-23T10:24:07.052314+0000","last_became_peered":"2024-04-23T10:24:07.052314+0000","last_unstale":"2024-04-23T10:24:08.101213+0000","last_undegraded":"2024-04-23T10:24:08.101213+0000","last_fullsized":"2024-04-23T10:24:08.101213+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T10:29:18.098800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.030318+0000","last_change":"2024-04-23T10:24:06.574570+0000","last_active":"2024-04-23T10:24:08.030318+0000","last_peered":"2024-04-23T10:24:08.030318+0000","last_clean":"2024-04-23T10:24:08.030318+0000","last_became_active":"2024-04-23T10:24:06.574166+0000","last_became_peered":"2024-04-23T10:24:06.574166+0000","last_unstale":"2024-04-23T10:24:08.030318+0000","last_undegraded":"2024-04-23T10:24:08.030318+0000","last_fullsized":"2024-04-23T10:24:08.030318+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:28:31.619100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.100954+0000","last_change":"2024-04-23T10:24:07.052825+0000","last_active":"2024-04-23T10:24:08.100954+0000","last_peered":"2024-04-23T10:24:08.100954+0000","last_clean":"2024-04-23T10:24:08.100954+0000","last_became_active":"2024-04-23T10:24:07.052457+0000","last_became_peered":"2024-04-23T10:24:07.052457+0000","last_unstale":"2024-04-23T10:24:08.100954+0000","last_undegraded":"2024-04-23T10:24:08.100954+0000","last_fullsized":"2024-04-23T10:24:08.100954+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:47:09.545374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'10","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:09.576072+0000","last_change":"2024-04-23T10:24:06.572168+0000","last_active":"2024-04-23T10:24:09.576072+0000","last_peered":"2024-04-23T10:24:09.576072+0000","last_clean":"2024-04-23T10:24:09.576072+0000","last_became_active":"2024-04-23T10:24:06.571805+0000","last_became_peered":"2024-04-23T10:24:06.571805+0000","last_unstale":"2024-04-23T10:24:09.576072+0000","last_undegraded":"2024-04-23T10:24:09.576072+0000","last_fullsized":"2024-04-23T10:24:09.576072+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:29:56.817879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.036427+0000","last_change":"2024-04-23T10:24:07.048474+0000","last_active":"2024-04-23T10:24:08.036427+0000","last_peered":"2024-04-23T10:24:08.036427+0000","last_clean":"2024-04-23T10:24:08.036427+0000","last_became_active":"2024-04-23T10:24:07.047893+0000","last_became_peered":"2024-04-23T10:24:07.047893+0000","last_unstale":"2024-04-23T10:24:08.036427+0000","last_undegraded":"2024-04-23T10:24:08.036427+0000","last_fullsized":"2024-04-23T10:24:08.036427+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T12:59:46.545569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.561598+0000","last_change":"2024-04-23T10:24:06.570974+0000","last_active":"2024-04-23T10:24:08.561598+0000","last_peered":"2024-04-23T10:24:08.561598+0000","last_clean":"2024-04-23T10:24:08.561598+0000","last_became_active":"2024-04-23T10:24:06.570603+0000","last_became_peered":"2024-04-23T10:24:06.570603+0000","last_unstale":"2024-04-23T10:24:08.561598+0000","last_undegraded":"2024-04-23T10:24:08.561598+0000","last_fullsized":"2024-04-23T10:24:08.561598+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T12:46:33.518337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T10:24:07.051616+0000","last_change":"2024-04-23T10:24:07.050986+0000","last_active":"2024-04-23T10:24:07.051616+0000","last_peered":"2024-04-23T10:24:07.051616+0000","last_clean":"2024-04-23T10:24:07.051616+0000","last_became_active":"2024-04-23T10:24:07.050617+0000","last_became_peered":"2024-04-23T10:24:07.050617+0000","last_unstale":"2024-04-23T10:24:07.051616+0000","last_undegraded":"2024-04-23T10:24:07.051616+0000","last_fullsized":"2024-04-23T10:24:07.051616+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:47:19.951516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.392499+0000","last_change":"2024-04-23T10:24:06.579138+0000","last_active":"2024-04-23T10:24:08.392499+0000","last_peered":"2024-04-23T10:24:08.392499+0000","last_clean":"2024-04-23T10:24:08.392499+0000","last_became_active":"2024-04-23T10:24:06.578716+0000","last_became_peered":"2024-04-23T10:24:06.578716+0000","last_unstale":"2024-04-23T10:24:08.392499+0000","last_undegraded":"2024-04-23T10:24:08.392499+0000","last_fullsized":"2024-04-23T10:24:08.392499+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:06:59.797802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.787215+0000","last_change":"2024-04-23T10:24:07.039019+0000","last_active":"2024-04-23T10:24:08.787215+0000","last_peered":"2024-04-23T10:24:08.787215+0000","last_clean":"2024-04-23T10:24:08.787215+0000","last_became_active":"2024-04-23T10:24:07.038701+0000","last_became_peered":"2024-04-23T10:24:07.038701+0000","last_unstale":"2024-04-23T10:24:08.787215+0000","last_undegraded":"2024-04-23T10:24:08.787215+0000","last_fullsized":"2024-04-23T10:24:08.787215+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:26:32.716043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.101257+0000","last_change":"2024-04-23T10:24:06.574342+0000","last_active":"2024-04-23T10:24:08.101257+0000","last_peered":"2024-04-23T10:24:08.101257+0000","last_clean":"2024-04-23T10:24:08.101257+0000","last_became_active":"2024-04-23T10:24:06.573981+0000","last_became_peered":"2024-04-23T10:24:06.573981+0000","last_unstale":"2024-04-23T10:24:08.101257+0000","last_undegraded":"2024-04-23T10:24:08.101257+0000","last_fullsized":"2024-04-23T10:24:08.101257+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:48:40.325991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.787264+0000","last_change":"2024-04-23T10:24:07.044712+0000","last_active":"2024-04-23T10:24:08.787264+0000","last_peered":"2024-04-23T10:24:08.787264+0000","last_clean":"2024-04-23T10:24:08.787264+0000","last_became_active":"2024-04-23T10:24:07.044141+0000","last_became_peered":"2024-04-23T10:24:07.044141+0000","last_unstale":"2024-04-23T10:24:08.787264+0000","last_undegraded":"2024-04-23T10:24:08.787264+0000","last_fullsized":"2024-04-23T10:24:08.787264+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:14:38.818648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.036149+0000","last_change":"2024-04-23T10:24:07.049061+0000","last_active":"2024-04-23T10:24:08.036149+0000","last_peered":"2024-04-23T10:24:08.036149+0000","last_clean":"2024-04-23T10:24:08.036149+0000","last_became_active":"2024-04-23T10:24:07.046446+0000","last_became_peered":"2024-04-23T10:24:07.046446+0000","last_unstale":"2024-04-23T10:24:08.036149+0000","last_undegraded":"2024-04-23T10:24:08.036149+0000","last_fullsized":"2024-04-23T10:24:08.036149+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:08:15.062455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:09.573911+0000","last_change":"2024-04-23T10:24:06.578216+0000","last_active":"2024-04-23T10:24:09.573911+0000","last_peered":"2024-04-23T10:24:09.573911+0000","last_clean":"2024-04-23T10:24:09.573911+0000","last_became_active":"2024-04-23T10:24:06.577370+0000","last_became_peered":"2024-04-23T10:24:06.577370+0000","last_unstale":"2024-04-23T10:24:09.573911+0000","last_undegraded":"2024-04-23T10:24:09.573911+0000","last_fullsized":"2024-04-23T10:24:09.573911+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:02:06.554442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.034686+0000","last_change":"2024-04-23T10:24:07.039627+0000","last_active":"2024-04-23T10:24:08.034686+0000","last_peered":"2024-04-23T10:24:08.034686+0000","last_clean":"2024-04-23T10:24:08.034686+0000","last_became_active":"2024-04-23T10:24:07.039272+0000","last_became_peered":"2024-04-23T10:24:07.039272+0000","last_unstale":"2024-04-23T10:24:08.034686+0000","last_undegraded":"2024-04-23T10:24:08.034686+0000","last_fullsized":"2024-04-23T10:24:08.034686+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T13:45:44.277926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.030532+0000","last_change":"2024-04-23T10:24:06.569086+0000","last_active":"2024-04-23T10:24:08.030532+0000","last_peered":"2024-04-23T10:24:08.030532+0000","last_clean":"2024-04-23T10:24:08.030532+0000","last_became_active":"2024-04-23T10:24:06.568719+0000","last_became_peered":"2024-04-23T10:24:06.568719+0000","last_unstale":"2024-04-23T10:24:08.030532+0000","last_undegraded":"2024-04-23T10:24:08.030532+0000","last_fullsized":"2024-04-23T10:24:08.030532+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:22:10.583782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":350,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.100904+0000","last_change":"2024-04-23T10:22:49.957476+0000","last_active":"2024-04-23T10:24:08.100904+0000","last_peered":"2024-04-23T10:24:08.100904+0000","last_clean":"2024-04-23T10:24:08.100904+0000","last_became_active":"2024-04-23T10:22:49.957033+0000","last_became_peered":"2024-04-23T10:22:49.957033+0000","last_unstale":"2024-04-23T10:24:08.100904+0000","last_undegraded":"2024-04-23T10:24:08.100904+0000","last_fullsized":"2024-04-23T10:24:08.100904+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:19:25.604229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:19:25.604229+0000","last_clean_scrub_stamp":"2024-04-23T10:19:25.604229+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T12:56:17.601141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:09.418252+0000","last_change":"2024-04-23T10:24:07.037936+0000","last_active":"2024-04-23T10:24:09.418252+0000","last_peered":"2024-04-23T10:24:09.418252+0000","last_clean":"2024-04-23T10:24:09.418252+0000","last_became_active":"2024-04-23T10:24:07.037614+0000","last_became_peered":"2024-04-23T10:24:07.037614+0000","last_unstale":"2024-04-23T10:24:09.418252+0000","last_undegraded":"2024-04-23T10:24:09.418252+0000","last_fullsized":"2024-04-23T10:24:09.418252+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T11:01:39.807822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:09.575767+0000","last_change":"2024-04-23T10:24:06.582629+0000","last_active":"2024-04-23T10:24:09.575767+0000","last_peered":"2024-04-23T10:24:09.575767+0000","last_clean":"2024-04-23T10:24:09.575767+0000","last_became_active":"2024-04-23T10:24:06.582325+0000","last_became_peered":"2024-04-23T10:24:06.582325+0000","last_unstale":"2024-04-23T10:24:09.575767+0000","last_undegraded":"2024-04-23T10:24:09.575767+0000","last_fullsized":"2024-04-23T10:24:09.575767+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:57:09.879100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:09.574919+0000","last_change":"2024-04-23T10:24:06.582482+0000","last_active":"2024-04-23T10:24:09.574919+0000","last_peered":"2024-04-23T10:24:09.574919+0000","last_clean":"2024-04-23T10:24:09.574919+0000","last_became_active":"2024-04-23T10:24:06.582114+0000","last_became_peered":"2024-04-23T10:24:06.582114+0000","last_unstale":"2024-04-23T10:24:09.574919+0000","last_undegraded":"2024-04-23T10:24:09.574919+0000","last_fullsized":"2024-04-23T10:24:09.574919+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:45:42.939840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:09.418889+0000","last_change":"2024-04-23T10:24:07.031102+0000","last_active":"2024-04-23T10:24:09.418889+0000","last_peered":"2024-04-23T10:24:09.418889+0000","last_clean":"2024-04-23T10:24:09.418889+0000","last_became_active":"2024-04-23T10:24:07.030783+0000","last_became_peered":"2024-04-23T10:24:07.030783+0000","last_unstale":"2024-04-23T10:24:09.418889+0000","last_undegraded":"2024-04-23T10:24:09.418889+0000","last_fullsized":"2024-04-23T10:24:09.418889+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:49:46.736046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'8","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:10.963752+0000","last_change":"2024-04-23T10:24:06.577096+0000","last_active":"2024-04-23T10:24:10.963752+0000","last_peered":"2024-04-23T10:24:10.963752+0000","last_clean":"2024-04-23T10:24:10.963752+0000","last_became_active":"2024-04-23T10:24:06.576740+0000","last_became_peered":"2024-04-23T10:24:06.576740+0000","last_unstale":"2024-04-23T10:24:10.963752+0000","last_undegraded":"2024-04-23T10:24:10.963752+0000","last_fullsized":"2024-04-23T10:24:10.963752+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:44:06.532366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.034848+0000","last_change":"2024-04-23T10:24:07.043599+0000","last_active":"2024-04-23T10:24:08.034848+0000","last_peered":"2024-04-23T10:24:08.034848+0000","last_clean":"2024-04-23T10:24:08.034848+0000","last_became_active":"2024-04-23T10:24:07.042953+0000","last_became_peered":"2024-04-23T10:24:07.042953+0000","last_unstale":"2024-04-23T10:24:08.034848+0000","last_undegraded":"2024-04-23T10:24:08.034848+0000","last_fullsized":"2024-04-23T10:24:08.034848+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:07:50.487972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.030760+0000","last_change":"2024-04-23T10:24:06.570681+0000","last_active":"2024-04-23T10:24:08.030760+0000","last_peered":"2024-04-23T10:24:08.030760+0000","last_clean":"2024-04-23T10:24:08.030760+0000","last_became_active":"2024-04-23T10:24:06.570178+0000","last_became_peered":"2024-04-23T10:24:06.570178+0000","last_unstale":"2024-04-23T10:24:08.030760+0000","last_undegraded":"2024-04-23T10:24:08.030760+0000","last_fullsized":"2024-04-23T10:24:08.030760+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T12:16:56.629443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:09.418633+0000","last_change":"2024-04-23T10:24:07.040498+0000","last_active":"2024-04-23T10:24:09.418633+0000","last_peered":"2024-04-23T10:24:09.418633+0000","last_clean":"2024-04-23T10:24:09.418633+0000","last_became_active":"2024-04-23T10:24:07.040009+0000","last_became_peered":"2024-04-23T10:24:07.040009+0000","last_unstale":"2024-04-23T10:24:09.418633+0000","last_undegraded":"2024-04-23T10:24:09.418633+0000","last_fullsized":"2024-04-23T10:24:09.418633+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:43:17.319305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:09.575461+0000","last_change":"2024-04-23T10:24:06.576904+0000","last_active":"2024-04-23T10:24:09.575461+0000","last_peered":"2024-04-23T10:24:09.575461+0000","last_clean":"2024-04-23T10:24:09.575461+0000","last_became_active":"2024-04-23T10:24:06.576512+0000","last_became_peered":"2024-04-23T10:24:06.576512+0000","last_unstale":"2024-04-23T10:24:09.575461+0000","last_undegraded":"2024-04-23T10:24:09.575461+0000","last_fullsized":"2024-04-23T10:24:09.575461+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:43:49.008844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.561197+0000","last_change":"2024-04-23T10:24:07.043710+0000","last_active":"2024-04-23T10:24:08.561197+0000","last_peered":"2024-04-23T10:24:08.561197+0000","last_clean":"2024-04-23T10:24:08.561197+0000","last_became_active":"2024-04-23T10:24:07.043406+0000","last_became_peered":"2024-04-23T10:24:07.043406+0000","last_unstale":"2024-04-23T10:24:08.561197+0000","last_undegraded":"2024-04-23T10:24:08.561197+0000","last_fullsized":"2024-04-23T10:24:08.561197+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:09:14.471938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.034410+0000","last_change":"2024-04-23T10:24:06.577409+0000","last_active":"2024-04-23T10:24:08.034410+0000","last_peered":"2024-04-23T10:24:08.034410+0000","last_clean":"2024-04-23T10:24:08.034410+0000","last_became_active":"2024-04-23T10:24:06.576787+0000","last_became_peered":"2024-04-23T10:24:06.576787+0000","last_unstale":"2024-04-23T10:24:08.034410+0000","last_undegraded":"2024-04-23T10:24:08.034410+0000","last_fullsized":"2024-04-23T10:24:08.034410+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:05:28.423806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T10:24:07.045753+0000","last_change":"2024-04-23T10:24:07.045180+0000","last_active":"2024-04-23T10:24:07.045753+0000","last_peered":"2024-04-23T10:24:07.045753+0000","last_clean":"2024-04-23T10:24:07.045753+0000","last_became_active":"2024-04-23T10:24:07.044731+0000","last_became_peered":"2024-04-23T10:24:07.044731+0000","last_unstale":"2024-04-23T10:24:07.045753+0000","last_undegraded":"2024-04-23T10:24:07.045753+0000","last_fullsized":"2024-04-23T10:24:07.045753+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T13:33:22.092090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.647033+0000","last_change":"2024-04-23T10:24:06.578039+0000","last_active":"2024-04-23T10:24:08.647033+0000","last_peered":"2024-04-23T10:24:08.647033+0000","last_clean":"2024-04-23T10:24:08.647033+0000","last_became_active":"2024-04-23T10:24:06.577655+0000","last_became_peered":"2024-04-23T10:24:06.577655+0000","last_unstale":"2024-04-23T10:24:08.647033+0000","last_undegraded":"2024-04-23T10:24:08.647033+0000","last_fullsized":"2024-04-23T10:24:08.647033+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:57:35.172318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.034513+0000","last_change":"2024-04-23T10:24:07.037311+0000","last_active":"2024-04-23T10:24:08.034513+0000","last_peered":"2024-04-23T10:24:08.034513+0000","last_clean":"2024-04-23T10:24:08.034513+0000","last_became_active":"2024-04-23T10:24:07.036969+0000","last_became_peered":"2024-04-23T10:24:07.036969+0000","last_unstale":"2024-04-23T10:24:08.034513+0000","last_undegraded":"2024-04-23T10:24:08.034513+0000","last_fullsized":"2024-04-23T10:24:08.034513+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:13:18.998914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:09.576616+0000","last_change":"2024-04-23T10:24:06.575183+0000","last_active":"2024-04-23T10:24:09.576616+0000","last_peered":"2024-04-23T10:24:09.576616+0000","last_clean":"2024-04-23T10:24:09.576616+0000","last_became_active":"2024-04-23T10:24:06.574630+0000","last_became_peered":"2024-04-23T10:24:06.574630+0000","last_unstale":"2024-04-23T10:24:09.576616+0000","last_undegraded":"2024-04-23T10:24:09.576616+0000","last_fullsized":"2024-04-23T10:24:09.576616+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T11:18:18.189925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.101366+0000","last_change":"2024-04-23T10:24:07.048832+0000","last_active":"2024-04-23T10:24:08.101366+0000","last_peered":"2024-04-23T10:24:08.101366+0000","last_clean":"2024-04-23T10:24:08.101366+0000","last_became_active":"2024-04-23T10:24:07.048509+0000","last_became_peered":"2024-04-23T10:24:07.048509+0000","last_unstale":"2024-04-23T10:24:08.101366+0000","last_undegraded":"2024-04-23T10:24:08.101366+0000","last_fullsized":"2024-04-23T10:24:08.101366+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T13:45:26.793330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.647170+0000","last_change":"2024-04-23T10:24:06.575408+0000","last_active":"2024-04-23T10:24:08.647170+0000","last_peered":"2024-04-23T10:24:08.647170+0000","last_clean":"2024-04-23T10:24:08.647170+0000","last_became_active":"2024-04-23T10:24:06.575081+0000","last_became_peered":"2024-04-23T10:24:06.575081+0000","last_unstale":"2024-04-23T10:24:08.647170+0000","last_undegraded":"2024-04-23T10:24:08.647170+0000","last_fullsized":"2024-04-23T10:24:08.647170+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T11:28:12.765196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.101587+0000","last_change":"2024-04-23T10:24:07.043869+0000","last_active":"2024-04-23T10:24:08.101587+0000","last_peered":"2024-04-23T10:24:08.101587+0000","last_clean":"2024-04-23T10:24:08.101587+0000","last_became_active":"2024-04-23T10:24:07.043448+0000","last_became_peered":"2024-04-23T10:24:07.043448+0000","last_unstale":"2024-04-23T10:24:08.101587+0000","last_undegraded":"2024-04-23T10:24:08.101587+0000","last_fullsized":"2024-04-23T10:24:08.101587+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:00:35.821470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.036052+0000","last_change":"2024-04-23T10:24:06.575259+0000","last_active":"2024-04-23T10:24:08.036052+0000","last_peered":"2024-04-23T10:24:08.036052+0000","last_clean":"2024-04-23T10:24:08.036052+0000","last_became_active":"2024-04-23T10:24:06.574920+0000","last_became_peered":"2024-04-23T10:24:06.574920+0000","last_unstale":"2024-04-23T10:24:08.036052+0000","last_undegraded":"2024-04-23T10:24:08.036052+0000","last_fullsized":"2024-04-23T10:24:08.036052+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:52:14.558458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.101206+0000","last_change":"2024-04-23T10:24:07.043238+0000","last_active":"2024-04-23T10:24:08.101206+0000","last_peered":"2024-04-23T10:24:08.101206+0000","last_clean":"2024-04-23T10:24:08.101206+0000","last_became_active":"2024-04-23T10:24:07.042110+0000","last_became_peered":"2024-04-23T10:24:07.042110+0000","last_unstale":"2024-04-23T10:24:08.101206+0000","last_undegraded":"2024-04-23T10:24:08.101206+0000","last_fullsized":"2024-04-23T10:24:08.101206+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T12:36:13.474456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:09.418851+0000","last_change":"2024-04-23T10:24:06.579233+0000","last_active":"2024-04-23T10:24:09.418851+0000","last_peered":"2024-04-23T10:24:09.418851+0000","last_clean":"2024-04-23T10:24:09.418851+0000","last_became_active":"2024-04-23T10:24:06.578849+0000","last_became_peered":"2024-04-23T10:24:06.578849+0000","last_unstale":"2024-04-23T10:24:09.418851+0000","last_undegraded":"2024-04-23T10:24:09.418851+0000","last_fullsized":"2024-04-23T10:24:09.418851+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:34:23.193707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.101450+0000","last_change":"2024-04-23T10:24:07.049840+0000","last_active":"2024-04-23T10:24:08.101450+0000","last_peered":"2024-04-23T10:24:08.101450+0000","last_clean":"2024-04-23T10:24:08.101450+0000","last_became_active":"2024-04-23T10:24:07.049493+0000","last_became_peered":"2024-04-23T10:24:07.049493+0000","last_unstale":"2024-04-23T10:24:08.101450+0000","last_undegraded":"2024-04-23T10:24:08.101450+0000","last_fullsized":"2024-04-23T10:24:08.101450+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:03:22.606800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T10:24:07.053113+0000","last_change":"2024-04-23T10:24:07.052332+0000","last_active":"2024-04-23T10:24:07.053113+0000","last_peered":"2024-04-23T10:24:07.053113+0000","last_clean":"2024-04-23T10:24:07.053113+0000","last_became_active":"2024-04-23T10:24:07.051956+0000","last_became_peered":"2024-04-23T10:24:07.051956+0000","last_unstale":"2024-04-23T10:24:07.053113+0000","last_undegraded":"2024-04-23T10:24:07.053113+0000","last_fullsized":"2024-04-23T10:24:07.053113+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:55:24.834893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.647166+0000","last_change":"2024-04-23T10:24:07.041897+0000","last_active":"2024-04-23T10:24:08.647166+0000","last_peered":"2024-04-23T10:24:08.647166+0000","last_clean":"2024-04-23T10:24:08.647166+0000","last_became_active":"2024-04-23T10:24:07.041546+0000","last_became_peered":"2024-04-23T10:24:07.041546+0000","last_unstale":"2024-04-23T10:24:08.647166+0000","last_undegraded":"2024-04-23T10:24:08.647166+0000","last_fullsized":"2024-04-23T10:24:08.647166+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T11:31:23.211045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.647156+0000","last_change":"2024-04-23T10:24:07.040764+0000","last_active":"2024-04-23T10:24:08.647156+0000","last_peered":"2024-04-23T10:24:08.647156+0000","last_clean":"2024-04-23T10:24:08.647156+0000","last_became_active":"2024-04-23T10:24:07.039637+0000","last_became_peered":"2024-04-23T10:24:07.039637+0000","last_unstale":"2024-04-23T10:24:08.647156+0000","last_undegraded":"2024-04-23T10:24:08.647156+0000","last_fullsized":"2024-04-23T10:24:08.647156+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:50:33.298770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:09.418471+0000","last_change":"2024-04-23T10:24:07.039491+0000","last_active":"2024-04-23T10:24:09.418471+0000","last_peered":"2024-04-23T10:24:09.418471+0000","last_clean":"2024-04-23T10:24:09.418471+0000","last_became_active":"2024-04-23T10:24:07.039128+0000","last_became_peered":"2024-04-23T10:24:07.039128+0000","last_unstale":"2024-04-23T10:24:09.418471+0000","last_undegraded":"2024-04-23T10:24:09.418471+0000","last_fullsized":"2024-04-23T10:24:09.418471+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T12:30:40.326031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:09.418766+0000","last_change":"2024-04-23T10:24:07.038133+0000","last_active":"2024-04-23T10:24:09.418766+0000","last_peered":"2024-04-23T10:24:09.418766+0000","last_clean":"2024-04-23T10:24:09.418766+0000","last_became_active":"2024-04-23T10:24:07.037817+0000","last_became_peered":"2024-04-23T10:24:07.037817+0000","last_unstale":"2024-04-23T10:24:09.418766+0000","last_undegraded":"2024-04-23T10:24:09.418766+0000","last_fullsized":"2024-04-23T10:24:09.418766+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:09:25.059986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T10:24:07.053140+0000","last_change":"2024-04-23T10:24:07.052395+0000","last_active":"2024-04-23T10:24:07.053140+0000","last_peered":"2024-04-23T10:24:07.053140+0000","last_clean":"2024-04-23T10:24:07.053140+0000","last_became_active":"2024-04-23T10:24:07.052023+0000","last_became_peered":"2024-04-23T10:24:07.052023+0000","last_unstale":"2024-04-23T10:24:07.053140+0000","last_undegraded":"2024-04-23T10:24:07.053140+0000","last_fullsized":"2024-04-23T10:24:07.053140+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T11:16:06.471157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.647369+0000","last_change":"2024-04-23T10:24:07.042739+0000","last_active":"2024-04-23T10:24:08.647369+0000","last_peered":"2024-04-23T10:24:08.647369+0000","last_clean":"2024-04-23T10:24:08.647369+0000","last_became_active":"2024-04-23T10:24:07.042292+0000","last_became_peered":"2024-04-23T10:24:07.042292+0000","last_unstale":"2024-04-23T10:24:08.647369+0000","last_undegraded":"2024-04-23T10:24:08.647369+0000","last_fullsized":"2024-04-23T10:24:08.647369+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:14:23.229032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.034816+0000","last_change":"2024-04-23T10:24:07.042792+0000","last_active":"2024-04-23T10:24:08.034816+0000","last_peered":"2024-04-23T10:24:08.034816+0000","last_clean":"2024-04-23T10:24:08.034816+0000","last_became_active":"2024-04-23T10:24:07.040846+0000","last_became_peered":"2024-04-23T10:24:07.040846+0000","last_unstale":"2024-04-23T10:24:08.034816+0000","last_undegraded":"2024-04-23T10:24:08.034816+0000","last_fullsized":"2024-04-23T10:24:08.034816+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T13:50:24.973286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.036473+0000","last_change":"2024-04-23T10:24:07.040391+0000","last_active":"2024-04-23T10:24:08.036473+0000","last_peered":"2024-04-23T10:24:08.036473+0000","last_clean":"2024-04-23T10:24:08.036473+0000","last_became_active":"2024-04-23T10:24:07.040002+0000","last_became_peered":"2024-04-23T10:24:07.040002+0000","last_unstale":"2024-04-23T10:24:08.036473+0000","last_undegraded":"2024-04-23T10:24:08.036473+0000","last_fullsized":"2024-04-23T10:24:08.036473+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T10:30:13.922315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:08.034656+0000","last_change":"2024-04-23T10:24:07.040678+0000","last_active":"2024-04-23T10:24:08.034656+0000","last_peered":"2024-04-23T10:24:08.034656+0000","last_clean":"2024-04-23T10:24:08.034656+0000","last_became_active":"2024-04-23T10:24:07.039551+0000","last_became_peered":"2024-04-23T10:24:07.039551+0000","last_unstale":"2024-04-23T10:24:08.034656+0000","last_undegraded":"2024-04-23T10:24:08.034656+0000","last_fullsized":"2024-04-23T10:24:08.034656+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:03:50.467083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:09.108508+0000","last_change":"2024-04-23T10:24:07.044618+0000","last_active":"2024-04-23T10:24:09.108508+0000","last_peered":"2024-04-23T10:24:09.108508+0000","last_clean":"2024-04-23T10:24:09.108508+0000","last_became_active":"2024-04-23T10:24:07.042803+0000","last_became_peered":"2024-04-23T10:24:07.042803+0000","last_unstale":"2024-04-23T10:24:09.108508+0000","last_undegraded":"2024-04-23T10:24:09.108508+0000","last_fullsized":"2024-04-23T10:24:09.108508+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:37:46.872177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-23T10:24:09.418746+0000","last_change":"2024-04-23T10:24:07.043724+0000","last_active":"2024-04-23T10:24:09.418746+0000","last_peered":"2024-04-23T10:24:09.418746+0000","last_clean":"2024-04-23T10:24:09.418746+0000","last_became_active":"2024-04-23T10:24:07.043408+0000","last_became_peered":"2024-04-23T10:24:07.043408+0000","last_unstale":"2024-04-23T10:24:09.418746+0000","last_undegraded":"2024-04-23T10:24:09.418746+0000","last_fullsized":"2024-04-23T10:24:09.418746+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:18:46.615093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T10:24:07.045017+0000","last_change":"2024-04-23T10:24:07.043794+0000","last_active":"2024-04-23T10:24:07.045017+0000","last_peered":"2024-04-23T10:24:07.045017+0000","last_clean":"2024-04-23T10:24:07.045017+0000","last_became_active":"2024-04-23T10:24:07.043391+0000","last_became_peered":"2024-04-23T10:24:07.043391+0000","last_unstale":"2024-04-23T10:24:07.045017+0000","last_undegraded":"2024-04-23T10:24:07.045017+0000","last_fullsized":"2024-04-23T10:24:07.045017+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:43:56.783899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280003,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2702,"num_objects":42,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":99,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":208896,"data_stored":12911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350990,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":433002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612627,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1024000,"data_stored":1035254,"data_compressed":35615,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776152,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":442480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037790,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":933888,"data_stored":442198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299427,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":983040,"data_stored":1033144,"data_compressed":35615,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462952,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":909312,"data_stored":433002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":933888,"data_stored":443184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738438,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":950272,"data_stored":443050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411386,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":443218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149751,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":929792,"data_stored":441676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888114,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":925696,"data_stored":441594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724588,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":983040,"data_stored":1032122,"data_compressed":35615,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33976,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33976,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33976,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35615,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35615,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35615,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35615,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35615,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":3705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T10:24:14.686 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-23T10:24:14.687 INFO:tasks.cephadm:Setting up client nodes... 2024-04-23T10:24:14.687 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-23T10:24:14.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:14 smithi196 ceph-mon[32919]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:24:14.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:14 smithi196 ceph-mon[32919]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:24:14.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:14 smithi196 ceph-mon[32919]: pgmap v293: 97 pgs: 97 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 731 B/s rd, 2.7 KiB/s wr, 14 op/s 2024-04-23T10:24:14.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:14 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/705220008' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:24:14.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:14 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/2712961394' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T10:24:14.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:14 smithi183 ceph-mon[32317]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:24:14.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:14 smithi183 ceph-mon[32317]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:24:14.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:14 smithi183 ceph-mon[32317]: pgmap v293: 97 pgs: 97 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 731 B/s rd, 2.7 KiB/s wr, 14 op/s 2024-04-23T10:24:14.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:14 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/705220008' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:24:14.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:14 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/2712961394' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T10:24:14.960 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T10:24:15.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:15 smithi196 ceph-mon[32919]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T10:24:15.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:15 smithi196 ceph-mon[32919]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-23T10:24:15.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:15 smithi196 ceph-mon[32919]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T10:24:15.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:15 smithi097 ceph-mon[29197]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T10:24:15.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:15 smithi097 ceph-mon[29197]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-23T10:24:15.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:15 smithi097 ceph-mon[29197]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T10:24:15.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:15 smithi183 ceph-mon[32317]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T10:24:15.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:15 smithi183 ceph-mon[32317]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-23T10:24:15.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:15 smithi183 ceph-mon[32317]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T10:24:16.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:16.602+0000 7f2ba5093700 1 -- 172.21.15.97:0/4088309953 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ba0103560 msgr2=0x7f2ba0109e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:16.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:16.602+0000 7f2ba5093700 1 --2- 172.21.15.97:0/4088309953 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ba0103560 0x7f2ba0109e10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2b9800b0d0 tx=0x7f2b9800b3e0 comp rx=0 tx=0).stop 2024-04-23T10:24:16.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:16.602+0000 7f2ba5093700 1 -- 172.21.15.97:0/4088309953 shutdown_connections 2024-04-23T10:24:16.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:16.602+0000 7f2ba5093700 1 --2- 172.21.15.97:0/4088309953 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ba0103560 0x7f2ba0109e10 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:16.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:16.602+0000 7f2ba5093700 1 --2- 172.21.15.97:0/4088309953 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ba01019c0 0x7f2ba0101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:16.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:16.602+0000 7f2ba5093700 1 --2- 172.21.15.97:0/4088309953 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2ba0102bc0 0x7f2ba0103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:16.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:16.602+0000 7f2ba5093700 1 -- 172.21.15.97:0/4088309953 >> 172.21.15.97:0/4088309953 conn(0x7f2ba00fcf60 msgr2=0x7f2ba00ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:16.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:16.602+0000 7f2ba5093700 1 -- 172.21.15.97:0/4088309953 shutdown_connections 2024-04-23T10:24:16.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:16.602+0000 7f2ba5093700 1 -- 172.21.15.97:0/4088309953 wait complete. 2024-04-23T10:24:16.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:16.603+0000 7f2ba5093700 1 Processor -- start 2024-04-23T10:24:16.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:16.603+0000 7f2ba5093700 1 -- start start 2024-04-23T10:24:16.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:16.603+0000 7f2ba5093700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ba01019c0 0x7f2ba006b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:16.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:16.603+0000 7f2ba5093700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ba0102bc0 0x7f2ba019d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:16.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:16.603+0000 7f2ba5093700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2ba0103560 0x7f2ba019ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:16.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:16.603+0000 7f2ba5093700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba010d550 con 0x7f2ba01019c0 2024-04-23T10:24:16.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:16.603+0000 7f2ba5093700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba010d3d0 con 0x7f2ba0103560 2024-04-23T10:24:16.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:16.603+0000 7f2ba5093700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba010d6d0 con 0x7f2ba0102bc0 2024-04-23T10:24:16.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:16.603+0000 7f2b9ed9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ba01019c0 0x7f2ba006b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:16.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:16.603+0000 7f2b9ed9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ba01019c0 0x7f2ba006b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48554/0 (socket says 172.21.15.97:48554) 2024-04-23T10:24:16.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:16.603+0000 7f2b9ed9d700 1 -- 172.21.15.97:0/4204918166 learned_addr learned my addr 172.21.15.97:0/4204918166 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:24:16.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:16.603+0000 7f2b9f59e700 1 --2- 172.21.15.97:0/4204918166 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2ba0103560 0x7f2ba019ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:16.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:16.603+0000 7f2b9e59c700 1 --2- 172.21.15.97:0/4204918166 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ba0102bc0 0x7f2ba019d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:16.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:16.603+0000 7f2b9ed9d700 1 -- 172.21.15.97:0/4204918166 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ba0102bc0 msgr2=0x7f2ba019d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:16.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:16.603+0000 7f2b9ed9d700 1 --2- 172.21.15.97:0/4204918166 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ba0102bc0 0x7f2ba019d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:16.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:16.603+0000 7f2b9ed9d700 1 -- 172.21.15.97:0/4204918166 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2ba0103560 msgr2=0x7f2ba019ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:16.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:16.603+0000 7f2b9ed9d700 1 --2- 172.21.15.97:0/4204918166 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2ba0103560 0x7f2ba019ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:16.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:16.603+0000 7f2b9ed9d700 1 -- 172.21.15.97:0/4204918166 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ba01a0860 con 0x7f2ba01019c0 2024-04-23T10:24:16.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:16.603+0000 7f2b9e59c700 1 --2- 172.21.15.97:0/4204918166 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ba0102bc0 0x7f2ba019d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:24:16.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:16.604+0000 7f2b9ed9d700 1 --2- 172.21.15.97:0/4204918166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ba01019c0 0x7f2ba006b2e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2b9000b550 tx=0x7f2b9000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:16.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:16.604+0000 7f2b87fff700 1 -- 172.21.15.97:0/4204918166 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b90015020 con 0x7f2ba01019c0 2024-04-23T10:24:16.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:16.604+0000 7f2b87fff700 1 -- 172.21.15.97:0/4204918166 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b90011ce0 con 0x7f2ba01019c0 2024-04-23T10:24:16.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:16.604+0000 7f2ba5093700 1 -- 172.21.15.97:0/4204918166 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ba01a0b50 con 0x7f2ba01019c0 2024-04-23T10:24:16.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:16.604+0000 7f2b87fff700 1 -- 172.21.15.97:0/4204918166 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b90010d60 con 0x7f2ba01019c0 2024-04-23T10:24:16.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:16.604+0000 7f2ba5093700 1 -- 172.21.15.97:0/4204918166 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ba01a1090 con 0x7f2ba01019c0 2024-04-23T10:24:16.605 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:16.605+0000 7f2ba5093700 1 -- 172.21.15.97:0/4204918166 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ba0050f80 con 0x7f2ba01019c0 2024-04-23T10:24:16.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:16.606+0000 7f2b87fff700 1 -- 172.21.15.97:0/4204918166 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2b90018020 con 0x7f2ba01019c0 2024-04-23T10:24:16.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:16.606+0000 7f2b87fff700 1 --2- 172.21.15.97:0/4204918166 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f2b880a3d80 0x7f2b880a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:16.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:16.606+0000 7f2b9e59c700 1 --2- 172.21.15.97:0/4204918166 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f2b880a3d80 0x7f2b880a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:16.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:16.606+0000 7f2b87fff700 1 -- 172.21.15.97:0/4204918166 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f2b900cfae0 con 0x7f2ba01019c0 2024-04-23T10:24:16.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:16.607+0000 7f2b9e59c700 1 --2- 172.21.15.97:0/4204918166 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f2b880a3d80 0x7f2b880a6240 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2b94007900 tx=0x7f2b94008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:16.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:16.611+0000 7f2b87fff700 1 -- 172.21.15.97:0/4204918166 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2b9001f020 con 0x7f2ba01019c0 2024-04-23T10:24:16.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:16 smithi097 ceph-mon[29197]: pgmap v294: 97 pgs: 97 active+clean; 586 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 7.9 KiB/s wr, 30 op/s 2024-04-23T10:24:16.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:16 smithi196 ceph-mon[32919]: pgmap v294: 97 pgs: 97 active+clean; 586 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 7.9 KiB/s wr, 30 op/s 2024-04-23T10:24:16.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:16.889+0000 7f2ba5093700 1 -- 172.21.15.97:0/4204918166 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f2ba002e050 con 0x7f2ba01019c0 2024-04-23T10:24:16.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:16.891+0000 7f2b87fff700 1 -- 172.21.15.97:0/4204918166 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f2b9009c620 con 0x7f2ba01019c0 2024-04-23T10:24:16.890 INFO:teuthology.orchestra.run.smithi097.stdout:[client.0] 2024-04-23T10:24:16.890 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQBQjCdm8dgYNRAARmcfl2CPHyWTlaupzjd7qg== 2024-04-23T10:24:16.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:16.893+0000 7f2ba5093700 1 -- 172.21.15.97:0/4204918166 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f2b880a3d80 msgr2=0x7f2b880a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:16.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:16.893+0000 7f2ba5093700 1 --2- 172.21.15.97:0/4204918166 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f2b880a3d80 0x7f2b880a6240 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2b94007900 tx=0x7f2b94008040 comp rx=0 tx=0).stop 2024-04-23T10:24:16.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:16.893+0000 7f2ba5093700 1 -- 172.21.15.97:0/4204918166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ba01019c0 msgr2=0x7f2ba006b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:16.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:16.893+0000 7f2ba5093700 1 --2- 172.21.15.97:0/4204918166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ba01019c0 0x7f2ba006b2e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2b9000b550 tx=0x7f2b9000b910 comp rx=0 tx=0).stop 2024-04-23T10:24:16.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:16.893+0000 7f2ba5093700 1 -- 172.21.15.97:0/4204918166 shutdown_connections 2024-04-23T10:24:16.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:16.893+0000 7f2ba5093700 1 --2- 172.21.15.97:0/4204918166 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f2b880a3d80 0x7f2b880a6240 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:16.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:16.893+0000 7f2ba5093700 1 --2- 172.21.15.97:0/4204918166 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2ba0103560 0x7f2ba019ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:16.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:16.893+0000 7f2ba5093700 1 --2- 172.21.15.97:0/4204918166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ba01019c0 0x7f2ba006b2e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:16.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:16.893+0000 7f2ba5093700 1 --2- 172.21.15.97:0/4204918166 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ba0102bc0 0x7f2ba019d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:16.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:16.893+0000 7f2ba5093700 1 -- 172.21.15.97:0/4204918166 >> 172.21.15.97:0/4204918166 conn(0x7f2ba00fcf60 msgr2=0x7f2ba0105ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:16.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:16.894+0000 7f2ba5093700 1 -- 172.21.15.97:0/4204918166 shutdown_connections 2024-04-23T10:24:16.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:16.894+0000 7f2ba5093700 1 -- 172.21.15.97:0/4204918166 wait complete. 2024-04-23T10:24:16.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:16 smithi183 ceph-mon[32317]: pgmap v294: 97 pgs: 97 active+clean; 586 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 7.9 KiB/s wr, 30 op/s 2024-04-23T10:24:17.473 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-23T10:24:17.473 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-23T10:24:17.473 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-23T10:24:17.508 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-23T10:24:17.509 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-23T10:24:17.509 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-23T10:24:17.786 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T10:24:17.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:17 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/4204918166' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-23T10:24:17.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:17 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/4204918166' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-23T10:24:17.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:17 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/4204918166' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-23T10:24:17.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:17 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/4204918166' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-23T10:24:17.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:17 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/4204918166' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-23T10:24:17.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:17 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/4204918166' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-23T10:24:18.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:18 smithi196 ceph-mon[32919]: pgmap v295: 97 pgs: 97 active+clean; 627 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 11 KiB/s wr, 29 op/s 2024-04-23T10:24:18.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:18 smithi097 ceph-mon[29197]: pgmap v295: 97 pgs: 97 active+clean; 627 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 11 KiB/s wr, 29 op/s 2024-04-23T10:24:18.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:18 smithi183 ceph-mon[32317]: pgmap v295: 97 pgs: 97 active+clean; 627 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 11 KiB/s wr, 29 op/s 2024-04-23T10:24:19.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:19.517+0000 7f41b9292700 1 -- 172.21.15.97:0/2213279377 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41b4108a20 msgr2=0x7f41b4108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:19.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:19.517+0000 7f41b9292700 1 --2- 172.21.15.97:0/2213279377 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41b4108a20 0x7f41b4108e00 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f41ac00b0a0 tx=0x7f41ac00b3b0 comp rx=0 tx=0).stop 2024-04-23T10:24:19.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:19.517+0000 7f41b9292700 1 -- 172.21.15.97:0/2213279377 shutdown_connections 2024-04-23T10:24:19.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:19.517+0000 7f41b9292700 1 --2- 172.21.15.97:0/2213279377 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f41b406b980 0x7f41b410f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:19.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:19.517+0000 7f41b9292700 1 --2- 172.21.15.97:0/2213279377 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41b4108a20 0x7f41b4108e00 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:19.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:19.517+0000 7f41b9292700 1 --2- 172.21.15.97:0/2213279377 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41b406afe0 0x7f41b406b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:19.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:19.517+0000 7f41b9292700 1 -- 172.21.15.97:0/2213279377 >> 172.21.15.97:0/2213279377 conn(0x7f41b40788b0 msgr2=0x7f41b4078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:19.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:19.517+0000 7f41b9292700 1 -- 172.21.15.97:0/2213279377 shutdown_connections 2024-04-23T10:24:19.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:19.517+0000 7f41b9292700 1 -- 172.21.15.97:0/2213279377 wait complete. 2024-04-23T10:24:19.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:19.517+0000 7f41b9292700 1 Processor -- start 2024-04-23T10:24:19.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:19.518+0000 7f41b9292700 1 -- start start 2024-04-23T10:24:19.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:19.518+0000 7f41b9292700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41b406afe0 0x7f41b4101a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:19.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:19.518+0000 7f41b9292700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41b406b980 0x7f41b4101f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:19.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:19.518+0000 7f41b9292700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f41b4108a20 0x7f41b4102480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:19.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:19.518+0000 7f41b9292700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f41b4112c40 con 0x7f41b406afe0 2024-04-23T10:24:19.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:19.518+0000 7f41b9292700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f41b4112ac0 con 0x7f41b4108a20 2024-04-23T10:24:19.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:19.518+0000 7f41b9292700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f41b4112dc0 con 0x7f41b406b980 2024-04-23T10:24:19.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:19.518+0000 7f41b27fc700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41b406b980 0x7f41b4101f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:19.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:19.518+0000 7f41b27fc700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41b406b980 0x7f41b4101f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.97:34708/0 (socket says 172.21.15.97:34708) 2024-04-23T10:24:19.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:19.518+0000 7f41b27fc700 1 -- 172.21.15.97:0/3122546180 learned_addr learned my addr 172.21.15.97:0/3122546180 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:24:19.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:19.518+0000 7f41b2ffd700 1 --2- 172.21.15.97:0/3122546180 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41b406afe0 0x7f41b4101a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:19.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:19.518+0000 7f41b37fe700 1 --2- 172.21.15.97:0/3122546180 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f41b4108a20 0x7f41b4102480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:19.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:19.518+0000 7f41b2ffd700 1 -- 172.21.15.97:0/3122546180 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41b406b980 msgr2=0x7f41b4101f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:19.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:19.518+0000 7f41b2ffd700 1 --2- 172.21.15.97:0/3122546180 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41b406b980 0x7f41b4101f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:19.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:19.518+0000 7f41b2ffd700 1 -- 172.21.15.97:0/3122546180 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f41b4108a20 msgr2=0x7f41b4102480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:19.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:19.518+0000 7f41b2ffd700 1 --2- 172.21.15.97:0/3122546180 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f41b4108a20 0x7f41b4102480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:19.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:19.518+0000 7f41b2ffd700 1 -- 172.21.15.97:0/3122546180 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41b41a76e0 con 0x7f41b406afe0 2024-04-23T10:24:19.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:19.519+0000 7f41b2ffd700 1 --2- 172.21.15.97:0/3122546180 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41b406afe0 0x7f41b4101a00 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f41ac000f80 tx=0x7f41ac004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:19.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:19.519+0000 7f419bfff700 1 -- 172.21.15.97:0/3122546180 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ac021070 con 0x7f41b406afe0 2024-04-23T10:24:19.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:19.519+0000 7f41b9292700 1 -- 172.21.15.97:0/3122546180 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41b41a7970 con 0x7f41b406afe0 2024-04-23T10:24:19.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:19.519+0000 7f419bfff700 1 -- 172.21.15.97:0/3122546180 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41ac007b80 con 0x7f41b406afe0 2024-04-23T10:24:19.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:19.519+0000 7f419bfff700 1 -- 172.21.15.97:0/3122546180 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ac0052a0 con 0x7f41b406afe0 2024-04-23T10:24:19.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:19.519+0000 7f41b9292700 1 -- 172.21.15.97:0/3122546180 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41b41a7e70 con 0x7f41b406afe0 2024-04-23T10:24:19.521 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:19.520+0000 7f41b9292700 1 -- 172.21.15.97:0/3122546180 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41b4069730 con 0x7f41b406afe0 2024-04-23T10:24:19.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:19.521+0000 7f419bfff700 1 -- 172.21.15.97:0/3122546180 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f41ac024070 con 0x7f41b406afe0 2024-04-23T10:24:19.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:19.522+0000 7f419bfff700 1 --2- 172.21.15.97:0/3122546180 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f419c0a3cd0 0x7f419c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:19.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:19.522+0000 7f41b27fc700 1 --2- 172.21.15.97:0/3122546180 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f419c0a3cd0 0x7f419c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:19.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:19.522+0000 7f419bfff700 1 -- 172.21.15.97:0/3122546180 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f41ac00f020 con 0x7f41b406afe0 2024-04-23T10:24:19.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:19.522+0000 7f41b27fc700 1 --2- 172.21.15.97:0/3122546180 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f419c0a3cd0 0x7f419c0a6190 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f41b4078f90 tx=0x7f41a800b480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:19.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:19.526+0000 7f419bfff700 1 -- 172.21.15.97:0/3122546180 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f41ac02b020 con 0x7f41b406afe0 2024-04-23T10:24:19.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:19.784+0000 7f41b9292700 1 -- 172.21.15.97:0/3122546180 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f41b402df80 con 0x7f41b406afe0 2024-04-23T10:24:19.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:19.788+0000 7f419bfff700 1 -- 172.21.15.97:0/3122546180 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264995 (secure 0 0 0) 0x7f41ac09e2e0 con 0x7f41b406afe0 2024-04-23T10:24:19.791 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:24:19.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:19.791+0000 7f41b9292700 1 -- 172.21.15.97:0/3122546180 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f419c0a3cd0 msgr2=0x7f419c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:19.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:19.791+0000 7f41b9292700 1 --2- 172.21.15.97:0/3122546180 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f419c0a3cd0 0x7f419c0a6190 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f41b4078f90 tx=0x7f41a800b480 comp rx=0 tx=0).stop 2024-04-23T10:24:19.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:19.792+0000 7f41b9292700 1 -- 172.21.15.97:0/3122546180 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41b406afe0 msgr2=0x7f41b4101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:19.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:19.792+0000 7f41b9292700 1 --2- 172.21.15.97:0/3122546180 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41b406afe0 0x7f41b4101a00 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f41ac000f80 tx=0x7f41ac004ce0 comp rx=0 tx=0).stop 2024-04-23T10:24:19.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:19.792+0000 7f41b9292700 1 -- 172.21.15.97:0/3122546180 shutdown_connections 2024-04-23T10:24:19.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:19.792+0000 7f41b9292700 1 --2- 172.21.15.97:0/3122546180 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f419c0a3cd0 0x7f419c0a6190 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:19.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:19.792+0000 7f41b9292700 1 --2- 172.21.15.97:0/3122546180 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f41b4108a20 0x7f41b4102480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:19.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:19.792+0000 7f41b9292700 1 --2- 172.21.15.97:0/3122546180 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41b406afe0 0x7f41b4101a00 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:19.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:19.792+0000 7f41b9292700 1 --2- 172.21.15.97:0/3122546180 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41b406b980 0x7f41b4101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:19.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:19.792+0000 7f41b9292700 1 -- 172.21.15.97:0/3122546180 >> 172.21.15.97:0/3122546180 conn(0x7f41b40788b0 msgr2=0x7f41b40ffae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:19.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:19.792+0000 7f41b9292700 1 -- 172.21.15.97:0/3122546180 shutdown_connections 2024-04-23T10:24:19.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:19.792+0000 7f41b9292700 1 -- 172.21.15.97:0/3122546180 wait complete. 2024-04-23T10:24:20.519 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3149043749},{"type":"v1","addr":"172.21.15.97:6801","nonce":3149043749}]},"active_addr":"172.21.15.97:6801/3149043749","active_change":"2024-04-23T10:16:59.468346+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":14217,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24107,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.97:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.97:0","nonce":233335470}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.97:0","nonce":3198144747}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.97:0","nonce":597405418}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.97:0","nonce":338277142}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.97:0","nonce":2872798752}]}]} 2024-04-23T10:24:20.525 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-23T10:24:20.525 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-23T10:24:20.526 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-23T10:24:20.768 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T10:24:20.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:20 smithi196 ceph-mon[32919]: pgmap v296: 97 pgs: 97 active+clean; 627 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.8 KiB/s wr, 27 op/s 2024-04-23T10:24:20.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:20 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/3122546180' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-23T10:24:20.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:20 smithi097 ceph-mon[29197]: pgmap v296: 97 pgs: 97 active+clean; 627 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.8 KiB/s wr, 27 op/s 2024-04-23T10:24:20.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:20 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/3122546180' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-23T10:24:20.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:20 smithi183 ceph-mon[32317]: pgmap v296: 97 pgs: 97 active+clean; 627 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 9.8 KiB/s wr, 27 op/s 2024-04-23T10:24:20.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:20 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/3122546180' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-23T10:24:22.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:22.491+0000 7f7283a9e700 1 -- 172.21.15.97:0/1761252084 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f727c107b00 msgr2=0x7f727c107ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:22.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:22.491+0000 7f7283a9e700 1 --2- 172.21.15.97:0/1761252084 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f727c107b00 0x7f727c107ee0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7278009990 tx=0x7f7278009ca0 comp rx=0 tx=0).stop 2024-04-23T10:24:22.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:22.491+0000 7f7283a9e700 1 -- 172.21.15.97:0/1761252084 shutdown_connections 2024-04-23T10:24:22.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:22.491+0000 7f7283a9e700 1 --2- 172.21.15.97:0/1761252084 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f727c101da0 0x7f727c10f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:22.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:22.491+0000 7f7283a9e700 1 --2- 172.21.15.97:0/1761252084 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f727c107b00 0x7f727c107ee0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:22.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:22.491+0000 7f7283a9e700 1 --2- 172.21.15.97:0/1761252084 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f727c1013e0 0x7f727c101860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:22.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:22.491+0000 7f7283a9e700 1 -- 172.21.15.97:0/1761252084 >> 172.21.15.97:0/1761252084 conn(0x7f727c076c60 msgr2=0x7f727c077070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:22.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:22.491+0000 7f7283a9e700 1 -- 172.21.15.97:0/1761252084 shutdown_connections 2024-04-23T10:24:22.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:22.491+0000 7f7283a9e700 1 -- 172.21.15.97:0/1761252084 wait complete. 2024-04-23T10:24:22.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:22.491+0000 7f7283a9e700 1 Processor -- start 2024-04-23T10:24:22.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:22.492+0000 7f7283a9e700 1 -- start start 2024-04-23T10:24:22.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:22.492+0000 7f7283a9e700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f727c1013e0 0x7f727c1992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:22.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:22.492+0000 7f7283a9e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f727c101da0 0x7f727c1997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:22.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:22.492+0000 7f7283a9e700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f727c107b00 0x7f727c19db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:22.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:22.492+0000 7f7283a9e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f727c112cf0 con 0x7f727c101da0 2024-04-23T10:24:22.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:22.492+0000 7f7283a9e700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f727c112b70 con 0x7f727c107b00 2024-04-23T10:24:22.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:22.492+0000 7f7283a9e700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f727c112e70 con 0x7f727c1013e0 2024-04-23T10:24:22.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:22.492+0000 7f7281039700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f727c101da0 0x7f727c1997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:22.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:22.492+0000 7f7281039700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f727c101da0 0x7f727c1997e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48596/0 (socket says 172.21.15.97:48596) 2024-04-23T10:24:22.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:22.492+0000 7f7281039700 1 -- 172.21.15.97:0/2831814093 learned_addr learned my addr 172.21.15.97:0/2831814093 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:24:22.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:22.492+0000 7f728183a700 1 --2- 172.21.15.97:0/2831814093 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f727c1013e0 0x7f727c1992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:22.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:22.492+0000 7f728203b700 1 --2- 172.21.15.97:0/2831814093 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f727c107b00 0x7f727c19db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:22.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:22.492+0000 7f7281039700 1 -- 172.21.15.97:0/2831814093 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f727c1013e0 msgr2=0x7f727c1992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:22.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:22.492+0000 7f7281039700 1 --2- 172.21.15.97:0/2831814093 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f727c1013e0 0x7f727c1992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:22.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:22.492+0000 7f7281039700 1 -- 172.21.15.97:0/2831814093 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f727c107b00 msgr2=0x7f727c19db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:22.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:22.492+0000 7f7281039700 1 --2- 172.21.15.97:0/2831814093 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f727c107b00 0x7f727c19db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:22.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:22.492+0000 7f7281039700 1 -- 172.21.15.97:0/2831814093 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f727c19e2b0 con 0x7f727c101da0 2024-04-23T10:24:22.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:22.493+0000 7f7281039700 1 --2- 172.21.15.97:0/2831814093 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f727c101da0 0x7f727c1997e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f726c00b530 tx=0x7f726c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:22.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:22.493+0000 7f7272ffd700 1 -- 172.21.15.97:0/2831814093 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f726c015020 con 0x7f727c101da0 2024-04-23T10:24:22.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:22.493+0000 7f7283a9e700 1 -- 172.21.15.97:0/2831814093 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f727c19e5a0 con 0x7f727c101da0 2024-04-23T10:24:22.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:22.493+0000 7f7272ffd700 1 -- 172.21.15.97:0/2831814093 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f726c011ce0 con 0x7f727c101da0 2024-04-23T10:24:22.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:22.493+0000 7f7272ffd700 1 -- 172.21.15.97:0/2831814093 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f726c010dd0 con 0x7f727c101da0 2024-04-23T10:24:22.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:22.493+0000 7f7283a9e700 1 -- 172.21.15.97:0/2831814093 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f727c1a5e20 con 0x7f727c101da0 2024-04-23T10:24:22.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:22.494+0000 7f7283a9e700 1 -- 172.21.15.97:0/2831814093 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f727c069730 con 0x7f727c101da0 2024-04-23T10:24:22.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:22.495+0000 7f7272ffd700 1 -- 172.21.15.97:0/2831814093 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f726c018020 con 0x7f727c101da0 2024-04-23T10:24:22.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:22.496+0000 7f7272ffd700 1 --2- 172.21.15.97:0/2831814093 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f72680a3cd0 0x7f72680a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:22.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:22.496+0000 7f728183a700 1 --2- 172.21.15.97:0/2831814093 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f72680a3cd0 0x7f72680a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:22.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:22.496+0000 7f7272ffd700 1 -- 172.21.15.97:0/2831814093 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f726c0cff40 con 0x7f727c101da0 2024-04-23T10:24:22.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:22.496+0000 7f728183a700 1 --2- 172.21.15.97:0/2831814093 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f72680a3cd0 0x7f72680a6190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7278006010 tx=0x7f7278005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:22.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:22.500+0000 7f7272ffd700 1 -- 172.21.15.97:0/2831814093 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f726c01f020 con 0x7f727c101da0 2024-04-23T10:24:22.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:22.702+0000 7f7283a9e700 1 -- 172.21.15.97:0/2831814093 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f727c02df80 con 0x7f727c101da0 2024-04-23T10:24:22.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:22.704+0000 7f7272ffd700 1 -- 172.21.15.97:0/2831814093 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22220 (secure 0 0 0) 0x7f726c09ca80 con 0x7f727c101da0 2024-04-23T10:24:22.703 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:24:22.703 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":86,"fsid":"9b22af88-0159-11ef-bc93-c7b262605968","created":"2024-04-23T10:16:05.983305+0000","modified":"2024-04-23T10:24:08.021502+0000","last_up_change":"2024-04-23T10:23:13.114761+0000","last_in_change":"2024-04-23T10:22:52.804880+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T10:19:25.559138+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T10:24:04.570049+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T10:24:05.861631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"58cb995d-de35-4522-ac81-4982401b260d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6803","nonce":3031221802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6805","nonce":3031221802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6809","nonce":3031221802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6807","nonce":3031221802}]},"public_addr":"172.21.15.97:6803/3031221802","cluster_addr":"172.21.15.97:6805/3031221802","heartbeat_back_addr":"172.21.15.97:6809/3031221802","heartbeat_front_addr":"172.21.15.97:6807/3031221802","state":["exists","up"]},{"osd":1,"uuid":"b47bd8ee-1f21-4834-acf5-bb40f97ab3ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6801","nonce":1108870440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6803","nonce":1108870440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6807","nonce":1108870440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6805","nonce":1108870440}]},"public_addr":"172.21.15.183:6801/1108870440","cluster_addr":"172.21.15.183:6803/1108870440","heartbeat_back_addr":"172.21.15.183:6807/1108870440","heartbeat_front_addr":"172.21.15.183:6805/1108870440","state":["exists","up"]},{"osd":2,"uuid":"17c4eec3-6c47-4334-9b63-1d5ad9ec5988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6801","nonce":2877039731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6803","nonce":2877039731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6807","nonce":2877039731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6805","nonce":2877039731}]},"public_addr":"172.21.15.196:6801/2877039731","cluster_addr":"172.21.15.196:6803/2877039731","heartbeat_back_addr":"172.21.15.196:6807/2877039731","heartbeat_front_addr":"172.21.15.196:6805/2877039731","state":["exists","up"]},{"osd":3,"uuid":"3f731c38-2e45-4ca1-8dce-efc4a1b2166b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6811","nonce":4276311194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6813","nonce":4276311194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6817","nonce":4276311194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6815","nonce":4276311194}]},"public_addr":"172.21.15.97:6811/4276311194","cluster_addr":"172.21.15.97:6813/4276311194","heartbeat_back_addr":"172.21.15.97:6817/4276311194","heartbeat_front_addr":"172.21.15.97:6815/4276311194","state":["exists","up"]},{"osd":4,"uuid":"fdcafb14-9b82-4134-bd5b-79c0516f6cbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6809","nonce":63913916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6811","nonce":63913916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6815","nonce":63913916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6813","nonce":63913916}]},"public_addr":"172.21.15.183:6809/63913916","cluster_addr":"172.21.15.183:6811/63913916","heartbeat_back_addr":"172.21.15.183:6815/63913916","heartbeat_front_addr":"172.21.15.183:6813/63913916","state":["exists","up"]},{"osd":5,"uuid":"cdcc18d8-da3c-4012-bf12-876bb39e66eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6809","nonce":2804446896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6811","nonce":2804446896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6815","nonce":2804446896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6813","nonce":2804446896}]},"public_addr":"172.21.15.196:6809/2804446896","cluster_addr":"172.21.15.196:6811/2804446896","heartbeat_back_addr":"172.21.15.196:6815/2804446896","heartbeat_front_addr":"172.21.15.196:6813/2804446896","state":["exists","up"]},{"osd":6,"uuid":"071abb5c-2edf-4832-b8aa-3b80d20096fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6819","nonce":2080916705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6821","nonce":2080916705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6825","nonce":2080916705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6823","nonce":2080916705}]},"public_addr":"172.21.15.97:6819/2080916705","cluster_addr":"172.21.15.97:6821/2080916705","heartbeat_back_addr":"172.21.15.97:6825/2080916705","heartbeat_front_addr":"172.21.15.97:6823/2080916705","state":["exists","up"]},{"osd":7,"uuid":"9d54760b-66df-486b-8abb-017f3b64b4d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6817","nonce":1865873033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6819","nonce":1865873033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6823","nonce":1865873033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6821","nonce":1865873033}]},"public_addr":"172.21.15.183:6817/1865873033","cluster_addr":"172.21.15.183:6819/1865873033","heartbeat_back_addr":"172.21.15.183:6823/1865873033","heartbeat_front_addr":"172.21.15.183:6821/1865873033","state":["exists","up"]},{"osd":8,"uuid":"5cd3100a-bd6a-4be5-b5e9-0d0afc325193","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6817","nonce":4063932139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6819","nonce":4063932139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6823","nonce":4063932139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6821","nonce":4063932139}]},"public_addr":"172.21.15.196:6817/4063932139","cluster_addr":"172.21.15.196:6819/4063932139","heartbeat_back_addr":"172.21.15.196:6823/4063932139","heartbeat_front_addr":"172.21.15.196:6821/4063932139","state":["exists","up"]},{"osd":9,"uuid":"b6a5f4a1-4ecc-4f73-a2df-07551d9298ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6827","nonce":1887432474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6829","nonce":1887432474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6833","nonce":1887432474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6831","nonce":1887432474}]},"public_addr":"172.21.15.97:6827/1887432474","cluster_addr":"172.21.15.97:6829/1887432474","heartbeat_back_addr":"172.21.15.97:6833/1887432474","heartbeat_front_addr":"172.21.15.97:6831/1887432474","state":["exists","up"]},{"osd":10,"uuid":"86f59816-1ea6-4ae9-8796-cf5fc4831064","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6825","nonce":799434647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6827","nonce":799434647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6831","nonce":799434647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6829","nonce":799434647}]},"public_addr":"172.21.15.183:6825/799434647","cluster_addr":"172.21.15.183:6827/799434647","heartbeat_back_addr":"172.21.15.183:6831/799434647","heartbeat_front_addr":"172.21.15.183:6829/799434647","state":["exists","up"]},{"osd":11,"uuid":"95d64a89-a9f8-4c7a-8c0d-da7fb26cfff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6825","nonce":1873969467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6827","nonce":1873969467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6831","nonce":1873969467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6829","nonce":1873969467}]},"public_addr":"172.21.15.196:6825/1873969467","cluster_addr":"172.21.15.196:6827/1873969467","heartbeat_back_addr":"172.21.15.196:6831/1873969467","heartbeat_front_addr":"172.21.15.196:6829/1873969467","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:18:33.721156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:18:57.586709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:19:22.069458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:19:46.451597+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:20:11.561019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:20:36.298672+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:21:01.661804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:21:27.324128+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:21:52.898197+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:22:19.614326+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:22:45.154917+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:23:11.222612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/3866016040":"2024-04-24T10:16:59.468237+0000","172.21.15.97:6801/3297166858":"2024-04-24T10:16:59.468237+0000","172.21.15.97:0/3804743548":"2024-04-24T10:16:59.468237+0000","172.21.15.97:6800/3706463017":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/3715079438":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/786806834":"2024-04-24T10:16:59.468237+0000","172.21.15.97:6801/3706463017":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/3294175554":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/1449136760":"2024-04-24T10:16:33.128917+0000","172.21.15.97:6800/3297166858":"2024-04-24T10:16:59.468237+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T10:24:22.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:22.706+0000 7f7283a9e700 1 -- 172.21.15.97:0/2831814093 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f72680a3cd0 msgr2=0x7f72680a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:22.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:22.706+0000 7f7283a9e700 1 --2- 172.21.15.97:0/2831814093 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f72680a3cd0 0x7f72680a6190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7278006010 tx=0x7f7278005960 comp rx=0 tx=0).stop 2024-04-23T10:24:22.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:22.706+0000 7f7283a9e700 1 -- 172.21.15.97:0/2831814093 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f727c101da0 msgr2=0x7f727c1997e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:22.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:22.706+0000 7f7283a9e700 1 --2- 172.21.15.97:0/2831814093 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f727c101da0 0x7f727c1997e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f726c00b530 tx=0x7f726c00b840 comp rx=0 tx=0).stop 2024-04-23T10:24:22.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:22.706+0000 7f7283a9e700 1 -- 172.21.15.97:0/2831814093 shutdown_connections 2024-04-23T10:24:22.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:22.706+0000 7f7283a9e700 1 --2- 172.21.15.97:0/2831814093 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f72680a3cd0 0x7f72680a6190 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:22.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:22.706+0000 7f7283a9e700 1 --2- 172.21.15.97:0/2831814093 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f727c107b00 0x7f727c19db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:22.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:22.706+0000 7f7283a9e700 1 --2- 172.21.15.97:0/2831814093 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f727c1013e0 0x7f727c1992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:22.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:22.706+0000 7f7283a9e700 1 --2- 172.21.15.97:0/2831814093 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f727c101da0 0x7f727c1997e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:22.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:22.706+0000 7f7283a9e700 1 -- 172.21.15.97:0/2831814093 >> 172.21.15.97:0/2831814093 conn(0x7f727c076c60 msgr2=0x7f727c100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:22.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:22.706+0000 7f7283a9e700 1 -- 172.21.15.97:0/2831814093 shutdown_connections 2024-04-23T10:24:22.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:22.706+0000 7f7283a9e700 1 -- 172.21.15.97:0/2831814093 wait complete. 2024-04-23T10:24:22.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:22 smithi196 ceph-mon[32919]: pgmap v297: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 KiB/s wr, 28 op/s 2024-04-23T10:24:22.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:22 smithi097 ceph-mon[29197]: pgmap v297: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 KiB/s wr, 28 op/s 2024-04-23T10:24:22.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:22 smithi183 ceph-mon[32317]: pgmap v297: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 10 KiB/s wr, 28 op/s 2024-04-23T10:24:23.473 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-23T10:24:23.473 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-23T10:24:23.716 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T10:24:23.838 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:23 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/2831814093' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T10:24:23.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:23 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/2831814093' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T10:24:24.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:23 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/2831814093' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T10:24:24.855 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:24 smithi097 ceph-mon[29197]: pgmap v298: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.3 KiB/s wr, 23 op/s 2024-04-23T10:24:24.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:24 smithi183 ceph-mon[32317]: pgmap v298: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.3 KiB/s wr, 23 op/s 2024-04-23T10:24:25.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:24 smithi196 ceph-mon[32919]: pgmap v298: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.3 KiB/s wr, 23 op/s 2024-04-23T10:24:25.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:25.488+0000 7f604b2e3700 1 -- 172.21.15.97:0/3516279767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f604410f4e0 msgr2=0x7f60441118d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:25.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:25.488+0000 7f604b2e3700 1 --2- 172.21.15.97:0/3516279767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f604410f4e0 0x7f60441118d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f603c009990 tx=0x7f603c009ca0 comp rx=0 tx=0).stop 2024-04-23T10:24:25.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:25.488+0000 7f604b2e3700 1 -- 172.21.15.97:0/3516279767 shutdown_connections 2024-04-23T10:24:25.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:25.488+0000 7f604b2e3700 1 --2- 172.21.15.97:0/3516279767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f604410f4e0 0x7f60441118d0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:25.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:25.488+0000 7f604b2e3700 1 --2- 172.21.15.97:0/3516279767 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f604406afe0 0x7f604406b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:25.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:25.488+0000 7f604b2e3700 1 --2- 172.21.15.97:0/3516279767 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f604406b900 0x7f604410efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:25.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:25.488+0000 7f604b2e3700 1 -- 172.21.15.97:0/3516279767 >> 172.21.15.97:0/3516279767 conn(0x7f60440788b0 msgr2=0x7f6044078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:25.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:25.488+0000 7f604b2e3700 1 -- 172.21.15.97:0/3516279767 shutdown_connections 2024-04-23T10:24:25.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:25.488+0000 7f604b2e3700 1 -- 172.21.15.97:0/3516279767 wait complete. 2024-04-23T10:24:25.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:25.489+0000 7f604b2e3700 1 Processor -- start 2024-04-23T10:24:25.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:25.489+0000 7f604b2e3700 1 -- start start 2024-04-23T10:24:25.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:25.489+0000 7f604b2e3700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f604406afe0 0x7f604419b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:25.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:25.489+0000 7f604b2e3700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f604406b900 0x7f604419b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:25.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:25.489+0000 7f604b2e3700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f604410f4e0 0x7f604419fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:25.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:25.489+0000 7f604b2e3700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6044114e00 con 0x7f604406afe0 2024-04-23T10:24:25.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:25.489+0000 7f604b2e3700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6044114c80 con 0x7f604406b900 2024-04-23T10:24:25.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:25.489+0000 7f604b2e3700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6044114f80 con 0x7f604410f4e0 2024-04-23T10:24:25.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:25.489+0000 7f604907f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f604406afe0 0x7f604419b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:25.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:25.489+0000 7f604907f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f604406afe0 0x7f604419b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47274/0 (socket says 172.21.15.97:47274) 2024-04-23T10:24:25.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:25.489+0000 7f604907f700 1 -- 172.21.15.97:0/2683259508 learned_addr learned my addr 172.21.15.97:0/2683259508 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:24:25.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:25.489+0000 7f604887e700 1 --2- 172.21.15.97:0/2683259508 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f604406b900 0x7f604419b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:25.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:25.489+0000 7f6049880700 1 --2- 172.21.15.97:0/2683259508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f604410f4e0 0x7f604419fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:25.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:25.489+0000 7f604907f700 1 -- 172.21.15.97:0/2683259508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f604410f4e0 msgr2=0x7f604419fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:25.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:25.489+0000 7f604907f700 1 --2- 172.21.15.97:0/2683259508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f604410f4e0 0x7f604419fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:25.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:25.489+0000 7f604907f700 1 -- 172.21.15.97:0/2683259508 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f604406b900 msgr2=0x7f604419b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:25.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:25.489+0000 7f604907f700 1 --2- 172.21.15.97:0/2683259508 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f604406b900 0x7f604419b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:25.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:25.489+0000 7f604907f700 1 -- 172.21.15.97:0/2683259508 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60441a02c0 con 0x7f604406afe0 2024-04-23T10:24:25.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:25.489+0000 7f604907f700 1 --2- 172.21.15.97:0/2683259508 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f604406afe0 0x7f604419b2b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f60340060b0 tx=0x7f603400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:25.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:25.490+0000 7f60427fc700 1 -- 172.21.15.97:0/2683259508 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f603401a070 con 0x7f604406afe0 2024-04-23T10:24:25.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:25.490+0000 7f604b2e3700 1 -- 172.21.15.97:0/2683259508 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60441a05b0 con 0x7f604406afe0 2024-04-23T10:24:25.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:25.490+0000 7f60427fc700 1 -- 172.21.15.97:0/2683259508 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f603400ddd0 con 0x7f604406afe0 2024-04-23T10:24:25.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:25.490+0000 7f60427fc700 1 -- 172.21.15.97:0/2683259508 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6034014720 con 0x7f604406afe0 2024-04-23T10:24:25.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:25.490+0000 7f604b2e3700 1 -- 172.21.15.97:0/2683259508 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60441a7e30 con 0x7f604406afe0 2024-04-23T10:24:25.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:25.491+0000 7f604b2e3700 1 -- 172.21.15.97:0/2683259508 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6044069730 con 0x7f604406afe0 2024-04-23T10:24:25.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:25.492+0000 7f60427fc700 1 -- 172.21.15.97:0/2683259508 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f603401d070 con 0x7f604406afe0 2024-04-23T10:24:25.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:25.492+0000 7f60427fc700 1 --2- 172.21.15.97:0/2683259508 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f602c0a3cd0 0x7f602c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:25.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:25.492+0000 7f604887e700 1 --2- 172.21.15.97:0/2683259508 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f602c0a3cd0 0x7f602c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:25.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:25.492+0000 7f60427fc700 1 -- 172.21.15.97:0/2683259508 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f60340cfc50 con 0x7f604406afe0 2024-04-23T10:24:25.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:25.493+0000 7f604887e700 1 --2- 172.21.15.97:0/2683259508 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f602c0a3cd0 0x7f602c0a6190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6044102e00 tx=0x7f603800b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:25.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:25.497+0000 7f60427fc700 1 -- 172.21.15.97:0/2683259508 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6034010070 con 0x7f604406afe0 2024-04-23T10:24:25.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:25.699+0000 7f604b2e3700 1 -- 172.21.15.97:0/2683259508 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f604402df80 con 0x7f604406afe0 2024-04-23T10:24:25.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:25.700+0000 7f60427fc700 1 -- 172.21.15.97:0/2683259508 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22220 (secure 0 0 0) 0x7f603409c680 con 0x7f604406afe0 2024-04-23T10:24:25.700 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:24:25.700 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":86,"fsid":"9b22af88-0159-11ef-bc93-c7b262605968","created":"2024-04-23T10:16:05.983305+0000","modified":"2024-04-23T10:24:08.021502+0000","last_up_change":"2024-04-23T10:23:13.114761+0000","last_in_change":"2024-04-23T10:22:52.804880+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T10:19:25.559138+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T10:24:04.570049+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T10:24:05.861631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"58cb995d-de35-4522-ac81-4982401b260d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6803","nonce":3031221802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6805","nonce":3031221802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6809","nonce":3031221802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6807","nonce":3031221802}]},"public_addr":"172.21.15.97:6803/3031221802","cluster_addr":"172.21.15.97:6805/3031221802","heartbeat_back_addr":"172.21.15.97:6809/3031221802","heartbeat_front_addr":"172.21.15.97:6807/3031221802","state":["exists","up"]},{"osd":1,"uuid":"b47bd8ee-1f21-4834-acf5-bb40f97ab3ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6801","nonce":1108870440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6803","nonce":1108870440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6807","nonce":1108870440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6805","nonce":1108870440}]},"public_addr":"172.21.15.183:6801/1108870440","cluster_addr":"172.21.15.183:6803/1108870440","heartbeat_back_addr":"172.21.15.183:6807/1108870440","heartbeat_front_addr":"172.21.15.183:6805/1108870440","state":["exists","up"]},{"osd":2,"uuid":"17c4eec3-6c47-4334-9b63-1d5ad9ec5988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6801","nonce":2877039731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6803","nonce":2877039731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6807","nonce":2877039731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6805","nonce":2877039731}]},"public_addr":"172.21.15.196:6801/2877039731","cluster_addr":"172.21.15.196:6803/2877039731","heartbeat_back_addr":"172.21.15.196:6807/2877039731","heartbeat_front_addr":"172.21.15.196:6805/2877039731","state":["exists","up"]},{"osd":3,"uuid":"3f731c38-2e45-4ca1-8dce-efc4a1b2166b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6811","nonce":4276311194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6813","nonce":4276311194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6817","nonce":4276311194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6815","nonce":4276311194}]},"public_addr":"172.21.15.97:6811/4276311194","cluster_addr":"172.21.15.97:6813/4276311194","heartbeat_back_addr":"172.21.15.97:6817/4276311194","heartbeat_front_addr":"172.21.15.97:6815/4276311194","state":["exists","up"]},{"osd":4,"uuid":"fdcafb14-9b82-4134-bd5b-79c0516f6cbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6809","nonce":63913916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6811","nonce":63913916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6815","nonce":63913916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6813","nonce":63913916}]},"public_addr":"172.21.15.183:6809/63913916","cluster_addr":"172.21.15.183:6811/63913916","heartbeat_back_addr":"172.21.15.183:6815/63913916","heartbeat_front_addr":"172.21.15.183:6813/63913916","state":["exists","up"]},{"osd":5,"uuid":"cdcc18d8-da3c-4012-bf12-876bb39e66eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6809","nonce":2804446896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6811","nonce":2804446896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6815","nonce":2804446896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6813","nonce":2804446896}]},"public_addr":"172.21.15.196:6809/2804446896","cluster_addr":"172.21.15.196:6811/2804446896","heartbeat_back_addr":"172.21.15.196:6815/2804446896","heartbeat_front_addr":"172.21.15.196:6813/2804446896","state":["exists","up"]},{"osd":6,"uuid":"071abb5c-2edf-4832-b8aa-3b80d20096fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6819","nonce":2080916705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6821","nonce":2080916705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6825","nonce":2080916705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6823","nonce":2080916705}]},"public_addr":"172.21.15.97:6819/2080916705","cluster_addr":"172.21.15.97:6821/2080916705","heartbeat_back_addr":"172.21.15.97:6825/2080916705","heartbeat_front_addr":"172.21.15.97:6823/2080916705","state":["exists","up"]},{"osd":7,"uuid":"9d54760b-66df-486b-8abb-017f3b64b4d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6817","nonce":1865873033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6819","nonce":1865873033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6823","nonce":1865873033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6821","nonce":1865873033}]},"public_addr":"172.21.15.183:6817/1865873033","cluster_addr":"172.21.15.183:6819/1865873033","heartbeat_back_addr":"172.21.15.183:6823/1865873033","heartbeat_front_addr":"172.21.15.183:6821/1865873033","state":["exists","up"]},{"osd":8,"uuid":"5cd3100a-bd6a-4be5-b5e9-0d0afc325193","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6817","nonce":4063932139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6819","nonce":4063932139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6823","nonce":4063932139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6821","nonce":4063932139}]},"public_addr":"172.21.15.196:6817/4063932139","cluster_addr":"172.21.15.196:6819/4063932139","heartbeat_back_addr":"172.21.15.196:6823/4063932139","heartbeat_front_addr":"172.21.15.196:6821/4063932139","state":["exists","up"]},{"osd":9,"uuid":"b6a5f4a1-4ecc-4f73-a2df-07551d9298ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6827","nonce":1887432474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6829","nonce":1887432474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6833","nonce":1887432474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6831","nonce":1887432474}]},"public_addr":"172.21.15.97:6827/1887432474","cluster_addr":"172.21.15.97:6829/1887432474","heartbeat_back_addr":"172.21.15.97:6833/1887432474","heartbeat_front_addr":"172.21.15.97:6831/1887432474","state":["exists","up"]},{"osd":10,"uuid":"86f59816-1ea6-4ae9-8796-cf5fc4831064","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6825","nonce":799434647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6827","nonce":799434647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6831","nonce":799434647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6829","nonce":799434647}]},"public_addr":"172.21.15.183:6825/799434647","cluster_addr":"172.21.15.183:6827/799434647","heartbeat_back_addr":"172.21.15.183:6831/799434647","heartbeat_front_addr":"172.21.15.183:6829/799434647","state":["exists","up"]},{"osd":11,"uuid":"95d64a89-a9f8-4c7a-8c0d-da7fb26cfff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6825","nonce":1873969467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6827","nonce":1873969467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6831","nonce":1873969467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6829","nonce":1873969467}]},"public_addr":"172.21.15.196:6825/1873969467","cluster_addr":"172.21.15.196:6827/1873969467","heartbeat_back_addr":"172.21.15.196:6831/1873969467","heartbeat_front_addr":"172.21.15.196:6829/1873969467","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:18:33.721156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:18:57.586709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:19:22.069458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:19:46.451597+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:20:11.561019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:20:36.298672+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:21:01.661804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:21:27.324128+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:21:52.898197+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:22:19.614326+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:22:45.154917+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:23:11.222612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/3866016040":"2024-04-24T10:16:59.468237+0000","172.21.15.97:6801/3297166858":"2024-04-24T10:16:59.468237+0000","172.21.15.97:0/3804743548":"2024-04-24T10:16:59.468237+0000","172.21.15.97:6800/3706463017":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/3715079438":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/786806834":"2024-04-24T10:16:59.468237+0000","172.21.15.97:6801/3706463017":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/3294175554":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/1449136760":"2024-04-24T10:16:33.128917+0000","172.21.15.97:6800/3297166858":"2024-04-24T10:16:59.468237+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T10:24:25.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:25.703+0000 7f604b2e3700 1 -- 172.21.15.97:0/2683259508 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f602c0a3cd0 msgr2=0x7f602c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:25.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:25.703+0000 7f604b2e3700 1 --2- 172.21.15.97:0/2683259508 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f602c0a3cd0 0x7f602c0a6190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6044102e00 tx=0x7f603800b3c0 comp rx=0 tx=0).stop 2024-04-23T10:24:25.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:25.703+0000 7f604b2e3700 1 -- 172.21.15.97:0/2683259508 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f604406afe0 msgr2=0x7f604419b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:25.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:25.703+0000 7f604b2e3700 1 --2- 172.21.15.97:0/2683259508 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f604406afe0 0x7f604419b2b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f60340060b0 tx=0x7f603400d3b0 comp rx=0 tx=0).stop 2024-04-23T10:24:25.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:25.703+0000 7f604b2e3700 1 -- 172.21.15.97:0/2683259508 shutdown_connections 2024-04-23T10:24:25.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:25.703+0000 7f604b2e3700 1 --2- 172.21.15.97:0/2683259508 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f602c0a3cd0 0x7f602c0a6190 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:25.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:25.703+0000 7f604b2e3700 1 --2- 172.21.15.97:0/2683259508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f604410f4e0 0x7f604419fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:25.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:25.703+0000 7f604b2e3700 1 --2- 172.21.15.97:0/2683259508 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f604406afe0 0x7f604419b2b0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:25.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:25.703+0000 7f604b2e3700 1 --2- 172.21.15.97:0/2683259508 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f604406b900 0x7f604419b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:25.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:25.703+0000 7f604b2e3700 1 -- 172.21.15.97:0/2683259508 >> 172.21.15.97:0/2683259508 conn(0x7f60440788b0 msgr2=0x7f6044110930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:25.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:25.703+0000 7f604b2e3700 1 -- 172.21.15.97:0/2683259508 shutdown_connections 2024-04-23T10:24:25.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:25.703+0000 7f604b2e3700 1 -- 172.21.15.97:0/2683259508 wait complete. 2024-04-23T10:24:26.395 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-23T10:24:26.395 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-23T10:24:26.395 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-23T10:24:26.396 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-23T10:24:26.396 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-23T10:24:26.397 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-23T10:24:26.397 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-23T10:24:26.398 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-23T10:24:26.398 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-23T10:24:26.399 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-23T10:24:26.399 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-23T10:24:26.400 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-23T10:24:26.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:26 smithi097 ceph-mon[29197]: pgmap v299: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.2 KiB/s wr, 23 op/s 2024-04-23T10:24:26.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:26 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/2683259508' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T10:24:26.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:26 smithi183 ceph-mon[32317]: pgmap v299: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.2 KiB/s wr, 23 op/s 2024-04-23T10:24:26.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:26 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/2683259508' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T10:24:27.069 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T10:24:27.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:26 smithi196 ceph-mon[32919]: pgmap v299: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.2 KiB/s wr, 23 op/s 2024-04-23T10:24:27.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:26 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/2683259508' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T10:24:27.122 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T10:24:27.142 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T10:24:27.847 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T10:24:27.848 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T10:24:27.856 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T10:24:28.445 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T10:24:28.445 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T10:24:28.800 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:28 smithi097 ceph-mon[29197]: pgmap v300: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 5.2 KiB/s wr, 12 op/s 2024-04-23T10:24:28.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:28 smithi183 ceph-mon[32317]: pgmap v300: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 5.2 KiB/s wr, 12 op/s 2024-04-23T10:24:29.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:28 smithi196 ceph-mon[32919]: pgmap v300: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 5.2 KiB/s wr, 12 op/s 2024-04-23T10:24:30.810 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:30 smithi097 ceph-mon[29197]: pgmap v301: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-04-23T10:24:30.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:30 smithi183 ceph-mon[32317]: pgmap v301: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-04-23T10:24:31.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:30 smithi196 ceph-mon[32919]: pgmap v301: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-04-23T10:24:31.247 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T10:24:31.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:31.683+0000 7f5d2bcb0700 1 -- 172.21.15.97:0/2455163234 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d24102300 msgr2=0x7f5d2410e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:31.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:31.683+0000 7f5d2bcb0700 1 --2- 172.21.15.97:0/2455163234 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d24102300 0x7f5d2410e7f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c00b0a0 tx=0x7f5d1c00b3b0 comp rx=0 tx=0).stop 2024-04-23T10:24:31.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:31.683+0000 7f5d2bcb0700 1 -- 172.21.15.97:0/2455163234 shutdown_connections 2024-04-23T10:24:31.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:31.683+0000 7f5d2bcb0700 1 --2- 172.21.15.97:0/2455163234 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d24102300 0x7f5d2410e7f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:31.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:31.683+0000 7f5d2bcb0700 1 --2- 172.21.15.97:0/2455163234 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d24107990 0x7f5d24107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:31.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:31.683+0000 7f5d2bcb0700 1 --2- 172.21.15.97:0/2455163234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d24101940 0x7f5d24101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:31.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:31.683+0000 7f5d2bcb0700 1 -- 172.21.15.97:0/2455163234 >> 172.21.15.97:0/2455163234 conn(0x7f5d240788f0 msgr2=0x7f5d24078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:31.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:31.683+0000 7f5d2bcb0700 1 -- 172.21.15.97:0/2455163234 shutdown_connections 2024-04-23T10:24:31.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:31.683+0000 7f5d2bcb0700 1 -- 172.21.15.97:0/2455163234 wait complete. 2024-04-23T10:24:31.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:31.683+0000 7f5d2bcb0700 1 Processor -- start 2024-04-23T10:24:31.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:31.684+0000 7f5d2bcb0700 1 -- start start 2024-04-23T10:24:31.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:31.684+0000 7f5d2bcb0700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d24101940 0x7f5d24199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:31.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:31.684+0000 7f5d2bcb0700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d24102300 0x7f5d2419a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:31.687 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:31.684+0000 7f5d2bcb0700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d24107990 0x7f5d241a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:31.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:31.684+0000 7f5d2bcb0700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5d24111b70 con 0x7f5d24101940 2024-04-23T10:24:31.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:31.684+0000 7f5d2bcb0700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5d241119f0 con 0x7f5d24107990 2024-04-23T10:24:31.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:31.684+0000 7f5d2bcb0700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5d24111cf0 con 0x7f5d24102300 2024-04-23T10:24:31.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:31.684+0000 7f5d29a4c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d24101940 0x7f5d24199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:31.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:31.684+0000 7f5d29a4c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d24101940 0x7f5d24199ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47290/0 (socket says 172.21.15.97:47290) 2024-04-23T10:24:31.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:31.684+0000 7f5d29a4c700 1 -- 172.21.15.97:0/908658704 learned_addr learned my addr 172.21.15.97:0/908658704 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:24:31.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:31.684+0000 7f5d2a24d700 1 --2- 172.21.15.97:0/908658704 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d24107990 0x7f5d241a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:31.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:31.684+0000 7f5d2924b700 1 --2- 172.21.15.97:0/908658704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d24102300 0x7f5d2419a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:31.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:31.684+0000 7f5d29a4c700 1 -- 172.21.15.97:0/908658704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d24102300 msgr2=0x7f5d2419a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:31.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:31.684+0000 7f5d29a4c700 1 --2- 172.21.15.97:0/908658704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d24102300 0x7f5d2419a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:31.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:31.684+0000 7f5d29a4c700 1 -- 172.21.15.97:0/908658704 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d24107990 msgr2=0x7f5d241a5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:31.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:31.684+0000 7f5d29a4c700 1 --2- 172.21.15.97:0/908658704 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d24107990 0x7f5d241a5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:31.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:31.684+0000 7f5d29a4c700 1 -- 172.21.15.97:0/908658704 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d241a55b0 con 0x7f5d24101940 2024-04-23T10:24:31.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:31.684+0000 7f5d2924b700 1 --2- 172.21.15.97:0/908658704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d24102300 0x7f5d2419a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:24:31.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:31.684+0000 7f5d29a4c700 1 --2- 172.21.15.97:0/908658704 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d24101940 0x7f5d24199ff0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5d2000d980 tx=0x7f5d2000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:31.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:31.684+0000 7f5d1affd700 1 -- 172.21.15.97:0/908658704 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d20017070 con 0x7f5d24101940 2024-04-23T10:24:31.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:31.684+0000 7f5d1affd700 1 -- 172.21.15.97:0/908658704 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d20004830 con 0x7f5d24101940 2024-04-23T10:24:31.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:31.684+0000 7f5d2bcb0700 1 -- 172.21.15.97:0/908658704 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d241a5780 con 0x7f5d24101940 2024-04-23T10:24:31.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:31.684+0000 7f5d1affd700 1 -- 172.21.15.97:0/908658704 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d20005440 con 0x7f5d24101940 2024-04-23T10:24:31.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:31.684+0000 7f5d2bcb0700 1 -- 172.21.15.97:0/908658704 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d241a5c40 con 0x7f5d24101940 2024-04-23T10:24:31.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:31.685+0000 7f5d2bcb0700 1 -- 172.21.15.97:0/908658704 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5d24063150 con 0x7f5d24101940 2024-04-23T10:24:31.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:31.686+0000 7f5d1affd700 1 -- 172.21.15.97:0/908658704 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5d200055e0 con 0x7f5d24101940 2024-04-23T10:24:31.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:31.687+0000 7f5d1affd700 1 --2- 172.21.15.97:0/908658704 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f5d100a3cd0 0x7f5d100a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:31.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:31.687+0000 7f5d2924b700 1 --2- 172.21.15.97:0/908658704 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f5d100a3cd0 0x7f5d100a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:31.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:31.687+0000 7f5d1affd700 1 -- 172.21.15.97:0/908658704 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f5d200d08f0 con 0x7f5d24101940 2024-04-23T10:24:31.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:31.687+0000 7f5d1affd700 1 --2- 172.21.15.97:0/908658704 >> [v2:172.21.15.183:6800/1108870440,v1:172.21.15.183:6801/1108870440] conn(0x7f5d100ad4b0 0x7f5d100af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:31.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:31.687+0000 7f5d1affd700 1 -- 172.21.15.97:0/908658704 --> [v2:172.21.15.183:6800/1108870440,v1:172.21.15.183:6801/1108870440] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5d100aff80 con 0x7f5d100ad4b0 2024-04-23T10:24:31.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:31.687+0000 7f5d1affd700 1 -- 172.21.15.97:0/908658704 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f5d200d0dd0 con 0x7f5d24101940 2024-04-23T10:24:31.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:31.687+0000 7f5d2924b700 1 --2- 172.21.15.97:0/908658704 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f5d100a3cd0 0x7f5d100a6190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5d24078d10 tx=0x7f5d14008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:31.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:31.688+0000 7f5d2a24d700 1 --2- 172.21.15.97:0/908658704 >> [v2:172.21.15.183:6800/1108870440,v1:172.21.15.183:6801/1108870440] conn(0x7f5d100ad4b0 0x7f5d100af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:31.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:31.688+0000 7f5d2a24d700 1 --2- 172.21.15.97:0/908658704 >> [v2:172.21.15.183:6800/1108870440,v1:172.21.15.183:6801/1108870440] conn(0x7f5d100ad4b0 0x7f5d100af8d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:31.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:31.689+0000 7f5d1affd700 1 -- 172.21.15.97:0/908658704 <== osd.1 v2:172.21.15.183:6800/1108870440 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5d100aff80 con 0x7f5d100ad4b0 2024-04-23T10:24:31.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:31.705+0000 7f5d2bcb0700 1 -- 172.21.15.97:0/908658704 --> [v2:172.21.15.183:6800/1108870440,v1:172.21.15.183:6801/1108870440] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5d240635d0 con 0x7f5d100ad4b0 2024-04-23T10:24:31.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:31.706+0000 7f5d1affd700 1 -- 172.21.15.97:0/908658704 <== osd.1 v2:172.21.15.183:6800/1108870440 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5d240635d0 con 0x7f5d100ad4b0 2024-04-23T10:24:31.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:31.706+0000 7f5d2bcb0700 1 -- 172.21.15.97:0/908658704 >> [v2:172.21.15.183:6800/1108870440,v1:172.21.15.183:6801/1108870440] conn(0x7f5d100ad4b0 msgr2=0x7f5d100af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:31.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:31.706+0000 7f5d2bcb0700 1 --2- 172.21.15.97:0/908658704 >> [v2:172.21.15.183:6800/1108870440,v1:172.21.15.183:6801/1108870440] conn(0x7f5d100ad4b0 0x7f5d100af8d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:31.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:31.706+0000 7f5d2bcb0700 1 -- 172.21.15.97:0/908658704 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f5d100a3cd0 msgr2=0x7f5d100a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:31.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:31.706+0000 7f5d2bcb0700 1 --2- 172.21.15.97:0/908658704 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f5d100a3cd0 0x7f5d100a6190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5d24078d10 tx=0x7f5d14008040 comp rx=0 tx=0).stop 2024-04-23T10:24:31.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:31.706+0000 7f5d2bcb0700 1 -- 172.21.15.97:0/908658704 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d24101940 msgr2=0x7f5d24199ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:31.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:31.706+0000 7f5d2bcb0700 1 --2- 172.21.15.97:0/908658704 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d24101940 0x7f5d24199ff0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5d2000d980 tx=0x7f5d2000dd40 comp rx=0 tx=0).stop 2024-04-23T10:24:31.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:31.706+0000 7f5d2a24d700 1 -- 172.21.15.97:0/908658704 reap_dead start 2024-04-23T10:24:31.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:31.706+0000 7f5d2bcb0700 1 -- 172.21.15.97:0/908658704 shutdown_connections 2024-04-23T10:24:31.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:31.706+0000 7f5d2bcb0700 1 -- 172.21.15.97:0/908658704 >> 172.21.15.97:0/908658704 conn(0x7f5d240788f0 msgr2=0x7f5d240ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:31.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:31.707+0000 7f5d2bcb0700 1 -- 172.21.15.97:0/908658704 shutdown_connections 2024-04-23T10:24:31.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:31.707+0000 7f5d2bcb0700 1 -- 172.21.15.97:0/908658704 wait complete. 2024-04-23T10:24:32.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:32 smithi097 ceph-mon[29197]: pgmap v302: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-04-23T10:24:32.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:32 smithi183 ceph-mon[32317]: pgmap v302: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-04-23T10:24:33.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:32 smithi196 ceph-mon[32919]: pgmap v302: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-04-23T10:24:34.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:34 smithi097 ceph-mon[29197]: pgmap v303: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-23T10:24:34.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:34 smithi183 ceph-mon[32317]: pgmap v303: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-23T10:24:35.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:34 smithi196 ceph-mon[32919]: pgmap v303: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-23T10:24:35.347 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T10:24:35.901 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T10:24:35.938 INFO:teuthology.orchestra.run.smithi097.stdout:55834574918 2024-04-23T10:24:35.939 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-23T10:24:36.824 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:36 smithi097 ceph-mon[29197]: pgmap v304: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-23T10:24:36.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:36 smithi183 ceph-mon[32317]: pgmap v304: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-23T10:24:37.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:36 smithi196 ceph-mon[32919]: pgmap v304: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-23T10:24:38.722 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:38 smithi097 ceph-mon[29197]: pgmap v305: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-23T10:24:38.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:38 smithi183 ceph-mon[32317]: pgmap v305: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-23T10:24:39.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:38 smithi196 ceph-mon[32919]: pgmap v305: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-23T10:24:39.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.202+0000 7f28d37cc700 1 -- 172.21.15.97:0/1886465630 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28cc1019d0 msgr2=0x7f28cc10f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:39.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.202+0000 7f28d37cc700 1 --2- 172.21.15.97:0/1886465630 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28cc1019d0 0x7f28cc10f910 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f28c400b0a0 tx=0x7f28c400b3b0 comp rx=0 tx=0).stop 2024-04-23T10:24:39.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.203+0000 7f28d37cc700 1 -- 172.21.15.97:0/1886465630 shutdown_connections 2024-04-23T10:24:39.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.203+0000 7f28d37cc700 1 --2- 172.21.15.97:0/1886465630 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28cc1019d0 0x7f28cc10f910 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:39.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.203+0000 7f28d37cc700 1 --2- 172.21.15.97:0/1886465630 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f28cc108ad0 0x7f28cc108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:39.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.203+0000 7f28d37cc700 1 --2- 172.21.15.97:0/1886465630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28cc101030 0x7f28cc101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:39.202 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.203+0000 7f28d37cc700 1 -- 172.21.15.97:0/1886465630 >> 172.21.15.97:0/1886465630 conn(0x7f28cc077fb0 msgr2=0x7f28cc0ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:39.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.203+0000 7f28d37cc700 1 -- 172.21.15.97:0/1886465630 shutdown_connections 2024-04-23T10:24:39.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.203+0000 7f28d37cc700 1 -- 172.21.15.97:0/1886465630 wait complete. 2024-04-23T10:24:39.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.203+0000 7f28d37cc700 1 Processor -- start 2024-04-23T10:24:39.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.203+0000 7f28d37cc700 1 -- start start 2024-04-23T10:24:39.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.204+0000 7f28d37cc700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28cc101030 0x7f28cc199070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:39.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.204+0000 7f28d37cc700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f28cc1019d0 0x7f28cc1995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:39.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.204+0000 7f28d37cc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28cc108ad0 0x7f28cc19d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:39.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.204+0000 7f28d37cc700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f28cc112c80 con 0x7f28cc108ad0 2024-04-23T10:24:39.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.204+0000 7f28d37cc700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f28cc112b00 con 0x7f28cc1019d0 2024-04-23T10:24:39.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.204+0000 7f28d37cc700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f28cc112e00 con 0x7f28cc101030 2024-04-23T10:24:39.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.204+0000 7f28d1d69700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28cc108ad0 0x7f28cc19d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:39.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.204+0000 7f28d1d69700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28cc108ad0 0x7f28cc19d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37536/0 (socket says 172.21.15.97:37536) 2024-04-23T10:24:39.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.204+0000 7f28d1d69700 1 -- 172.21.15.97:0/1406163050 learned_addr learned my addr 172.21.15.97:0/1406163050 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:24:39.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.204+0000 7f28d1568700 1 --2- 172.21.15.97:0/1406163050 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28cc101030 0x7f28cc199070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:39.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.204+0000 7f28d0d67700 1 --2- 172.21.15.97:0/1406163050 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f28cc1019d0 0x7f28cc1995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:39.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.204+0000 7f28d1d69700 1 -- 172.21.15.97:0/1406163050 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28cc101030 msgr2=0x7f28cc199070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:39.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.204+0000 7f28d1d69700 1 --2- 172.21.15.97:0/1406163050 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28cc101030 0x7f28cc199070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:39.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.204+0000 7f28d1d69700 1 -- 172.21.15.97:0/1406163050 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f28cc1019d0 msgr2=0x7f28cc1995b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:39.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.204+0000 7f28d1d69700 1 --2- 172.21.15.97:0/1406163050 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f28cc1019d0 0x7f28cc1995b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:39.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.204+0000 7f28d1d69700 1 -- 172.21.15.97:0/1406163050 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28cc19e080 con 0x7f28cc108ad0 2024-04-23T10:24:39.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.204+0000 7f28d1568700 1 --2- 172.21.15.97:0/1406163050 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f28cc101030 0x7f28cc199070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:24:39.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.204+0000 7f28d0d67700 1 --2- 172.21.15.97:0/1406163050 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f28cc1019d0 0x7f28cc1995b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:24:39.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.204+0000 7f28d1d69700 1 --2- 172.21.15.97:0/1406163050 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28cc108ad0 0x7f28cc19d940 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f28c40062a0 tx=0x7f28c4004ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:39.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.204+0000 7f28c27fc700 1 -- 172.21.15.97:0/1406163050 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c4021070 con 0x7f28cc108ad0 2024-04-23T10:24:39.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.205+0000 7f28d37cc700 1 -- 172.21.15.97:0/1406163050 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28cc19e310 con 0x7f28cc108ad0 2024-04-23T10:24:39.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.205+0000 7f28c27fc700 1 -- 172.21.15.97:0/1406163050 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28c4007d00 con 0x7f28cc108ad0 2024-04-23T10:24:39.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.205+0000 7f28c27fc700 1 -- 172.21.15.97:0/1406163050 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c401d450 con 0x7f28cc108ad0 2024-04-23T10:24:39.206 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.205+0000 7f28d37cc700 1 -- 172.21.15.97:0/1406163050 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28cc102140 con 0x7f28cc108ad0 2024-04-23T10:24:39.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.206+0000 7f28d37cc700 1 -- 172.21.15.97:0/1406163050 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f28cc0631d0 con 0x7f28cc108ad0 2024-04-23T10:24:39.207 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.207+0000 7f28c27fc700 1 -- 172.21.15.97:0/1406163050 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f28c4004020 con 0x7f28cc108ad0 2024-04-23T10:24:39.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.207+0000 7f28c27fc700 1 --2- 172.21.15.97:0/1406163050 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f28b80a3d50 0x7f28b80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:39.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.207+0000 7f28d1568700 1 --2- 172.21.15.97:0/1406163050 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f28b80a3d50 0x7f28b80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:39.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.208+0000 7f28c27fc700 1 -- 172.21.15.97:0/1406163050 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f28c400f020 con 0x7f28cc108ad0 2024-04-23T10:24:39.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.208+0000 7f28c27fc700 1 --2- 172.21.15.97:0/1406163050 >> [v2:172.21.15.183:6824/799434647,v1:172.21.15.183:6825/799434647] conn(0x7f28b80ad530 0x7f28b80af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:39.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.208+0000 7f28c27fc700 1 -- 172.21.15.97:0/1406163050 --> [v2:172.21.15.183:6824/799434647,v1:172.21.15.183:6825/799434647] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f28c4013e70 con 0x7f28b80ad530 2024-04-23T10:24:39.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.208+0000 7f28c27fc700 1 -- 172.21.15.97:0/1406163050 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f28c40d62c0 con 0x7f28cc108ad0 2024-04-23T10:24:39.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.208+0000 7f28d0d67700 1 --2- 172.21.15.97:0/1406163050 >> [v2:172.21.15.183:6824/799434647,v1:172.21.15.183:6825/799434647] conn(0x7f28b80ad530 0x7f28b80af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:39.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.208+0000 7f28d1568700 1 --2- 172.21.15.97:0/1406163050 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f28b80a3d50 0x7f28b80a6210 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f28c8005fd0 tx=0x7f28c8005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:39.208 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.208+0000 7f28d0d67700 1 --2- 172.21.15.97:0/1406163050 >> [v2:172.21.15.183:6824/799434647,v1:172.21.15.183:6825/799434647] conn(0x7f28b80ad530 0x7f28b80af950 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:39.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.209+0000 7f28c27fc700 1 -- 172.21.15.97:0/1406163050 <== osd.10 v2:172.21.15.183:6824/799434647 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f28c4013e70 con 0x7f28b80ad530 2024-04-23T10:24:39.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.238+0000 7f28d37cc700 1 -- 172.21.15.97:0/1406163050 --> [v2:172.21.15.183:6824/799434647,v1:172.21.15.183:6825/799434647] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f28cc063600 con 0x7f28b80ad530 2024-04-23T10:24:39.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.238+0000 7f28c27fc700 1 -- 172.21.15.97:0/1406163050 <== osd.10 v2:172.21.15.183:6824/799434647 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f28cc063600 con 0x7f28b80ad530 2024-04-23T10:24:39.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.238+0000 7f28d37cc700 1 -- 172.21.15.97:0/1406163050 >> [v2:172.21.15.183:6824/799434647,v1:172.21.15.183:6825/799434647] conn(0x7f28b80ad530 msgr2=0x7f28b80af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:39.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.238+0000 7f28d37cc700 1 --2- 172.21.15.97:0/1406163050 >> [v2:172.21.15.183:6824/799434647,v1:172.21.15.183:6825/799434647] conn(0x7f28b80ad530 0x7f28b80af950 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:39.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.239+0000 7f28d37cc700 1 -- 172.21.15.97:0/1406163050 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f28b80a3d50 msgr2=0x7f28b80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:39.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.239+0000 7f28d37cc700 1 --2- 172.21.15.97:0/1406163050 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f28b80a3d50 0x7f28b80a6210 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f28c8005fd0 tx=0x7f28c8005e30 comp rx=0 tx=0).stop 2024-04-23T10:24:39.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.239+0000 7f28d37cc700 1 -- 172.21.15.97:0/1406163050 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28cc108ad0 msgr2=0x7f28cc19d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:39.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.239+0000 7f28d37cc700 1 --2- 172.21.15.97:0/1406163050 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28cc108ad0 0x7f28cc19d940 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f28c40062a0 tx=0x7f28c4004ed0 comp rx=0 tx=0).stop 2024-04-23T10:24:39.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.239+0000 7f28d1d69700 1 -- 172.21.15.97:0/1406163050 reap_dead start 2024-04-23T10:24:39.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.239+0000 7f28d37cc700 1 -- 172.21.15.97:0/1406163050 shutdown_connections 2024-04-23T10:24:39.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.239+0000 7f28d37cc700 1 -- 172.21.15.97:0/1406163050 >> 172.21.15.97:0/1406163050 conn(0x7f28cc077fb0 msgr2=0x7f28cc0ff680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:39.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.239+0000 7f28d37cc700 1 -- 172.21.15.97:0/1406163050 shutdown_connections 2024-04-23T10:24:39.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.239+0000 7f28d37cc700 1 -- 172.21.15.97:0/1406163050 wait complete. 2024-04-23T10:24:39.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.860+0000 7f4d7a2e5700 1 -- 172.21.15.97:0/1524626903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d74102330 msgr2=0x7f4d7410e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:39.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.860+0000 7f4d7a2e5700 1 --2- 172.21.15.97:0/1524626903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d74102330 0x7f4d7410e7f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c00b0a0 tx=0x7f4d6c00b3b0 comp rx=0 tx=0).stop 2024-04-23T10:24:39.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.860+0000 7f4d7a2e5700 1 -- 172.21.15.97:0/1524626903 shutdown_connections 2024-04-23T10:24:39.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.860+0000 7f4d7a2e5700 1 --2- 172.21.15.97:0/1524626903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d74102330 0x7f4d7410e7f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:39.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.860+0000 7f4d7a2e5700 1 --2- 172.21.15.97:0/1524626903 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d74107920 0x7f4d74107d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:39.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.860+0000 7f4d7a2e5700 1 --2- 172.21.15.97:0/1524626903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4d74101970 0x7f4d74101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:39.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.860+0000 7f4d7a2e5700 1 -- 172.21.15.97:0/1524626903 >> 172.21.15.97:0/1524626903 conn(0x7f4d74078910 msgr2=0x7f4d74078d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:39.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.860+0000 7f4d7a2e5700 1 -- 172.21.15.97:0/1524626903 shutdown_connections 2024-04-23T10:24:39.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.860+0000 7f4d7a2e5700 1 -- 172.21.15.97:0/1524626903 wait complete. 2024-04-23T10:24:39.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.861+0000 7f4d7a2e5700 1 Processor -- start 2024-04-23T10:24:39.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.861+0000 7f4d7a2e5700 1 -- start start 2024-04-23T10:24:39.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.861+0000 7f4d7a2e5700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d74101970 0x7f4d74198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:39.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.861+0000 7f4d7a2e5700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4d74102330 0x7f4d74198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:39.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.861+0000 7f4d7a2e5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d74107920 0x7f4d7419ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:39.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.862+0000 7f4d7a2e5700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4d74111bc0 con 0x7f4d74107920 2024-04-23T10:24:39.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.862+0000 7f4d7a2e5700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4d74111a40 con 0x7f4d74102330 2024-04-23T10:24:39.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.862+0000 7f4d7a2e5700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4d74111d40 con 0x7f4d74101970 2024-04-23T10:24:39.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.862+0000 7f4d78882700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d74107920 0x7f4d7419ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:39.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.862+0000 7f4d78882700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d74107920 0x7f4d7419ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37568/0 (socket says 172.21.15.97:37568) 2024-04-23T10:24:39.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.862+0000 7f4d78882700 1 -- 172.21.15.97:0/1530033673 learned_addr learned my addr 172.21.15.97:0/1530033673 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:24:39.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.862+0000 7f4d73fff700 1 --2- 172.21.15.97:0/1530033673 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d74101970 0x7f4d74198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:39.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.862+0000 7f4d737fe700 1 --2- 172.21.15.97:0/1530033673 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4d74102330 0x7f4d74198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:39.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.862+0000 7f4d78882700 1 -- 172.21.15.97:0/1530033673 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d74101970 msgr2=0x7f4d74198150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:39.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.862+0000 7f4d78882700 1 --2- 172.21.15.97:0/1530033673 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d74101970 0x7f4d74198150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:39.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.862+0000 7f4d78882700 1 -- 172.21.15.97:0/1530033673 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4d74102330 msgr2=0x7f4d74198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:39.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.862+0000 7f4d78882700 1 --2- 172.21.15.97:0/1530033673 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4d74102330 0x7f4d74198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:39.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.862+0000 7f4d78882700 1 -- 172.21.15.97:0/1530033673 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d7419d160 con 0x7f4d74107920 2024-04-23T10:24:39.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.862+0000 7f4d73fff700 1 --2- 172.21.15.97:0/1530033673 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d74101970 0x7f4d74198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:24:39.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.862+0000 7f4d78882700 1 --2- 172.21.15.97:0/1530033673 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d74107920 0x7f4d7419ca20 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c0062a0 tx=0x7f4d6c004ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:39.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.862+0000 7f4d717fa700 1 -- 172.21.15.97:0/1530033673 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c021070 con 0x7f4d74107920 2024-04-23T10:24:39.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.862+0000 7f4d717fa700 1 -- 172.21.15.97:0/1530033673 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d6c007d00 con 0x7f4d74107920 2024-04-23T10:24:39.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.862+0000 7f4d7a2e5700 1 -- 172.21.15.97:0/1530033673 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d7419d3f0 con 0x7f4d74107920 2024-04-23T10:24:39.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.862+0000 7f4d717fa700 1 -- 172.21.15.97:0/1530033673 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c01d450 con 0x7f4d74107920 2024-04-23T10:24:39.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.862+0000 7f4d7a2e5700 1 -- 172.21.15.97:0/1530033673 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d7406b190 con 0x7f4d74107920 2024-04-23T10:24:39.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.863+0000 7f4d7a2e5700 1 -- 172.21.15.97:0/1530033673 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4d7406b800 con 0x7f4d74107920 2024-04-23T10:24:39.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.864+0000 7f4d717fa700 1 -- 172.21.15.97:0/1530033673 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4d6c004000 con 0x7f4d74107920 2024-04-23T10:24:39.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.865+0000 7f4d717fa700 1 --2- 172.21.15.97:0/1530033673 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f4d5c0a3cd0 0x7f4d5c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:39.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.865+0000 7f4d73fff700 1 --2- 172.21.15.97:0/1530033673 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f4d5c0a3cd0 0x7f4d5c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:39.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.865+0000 7f4d717fa700 1 -- 172.21.15.97:0/1530033673 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f4d6c00f020 con 0x7f4d74107920 2024-04-23T10:24:39.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.865+0000 7f4d717fa700 1 --2- 172.21.15.97:0/1530033673 >> [v2:172.21.15.196:6824/1873969467,v1:172.21.15.196:6825/1873969467] conn(0x7f4d5c0ad4b0 0x7f4d5c0af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:39.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.865+0000 7f4d717fa700 1 -- 172.21.15.97:0/1530033673 --> [v2:172.21.15.196:6824/1873969467,v1:172.21.15.196:6825/1873969467] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4d6c0020b0 con 0x7f4d5c0ad4b0 2024-04-23T10:24:39.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.865+0000 7f4d717fa700 1 -- 172.21.15.97:0/1530033673 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f4d6c0d21b0 con 0x7f4d74107920 2024-04-23T10:24:39.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.865+0000 7f4d73fff700 1 --2- 172.21.15.97:0/1530033673 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f4d5c0a3cd0 0x7f4d5c0a6190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4d640099d0 tx=0x7f4d64008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:39.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.865+0000 7f4d737fe700 1 --2- 172.21.15.97:0/1530033673 >> [v2:172.21.15.196:6824/1873969467,v1:172.21.15.196:6825/1873969467] conn(0x7f4d5c0ad4b0 0x7f4d5c0af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:39.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.866+0000 7f4d737fe700 1 --2- 172.21.15.97:0/1530033673 >> [v2:172.21.15.196:6824/1873969467,v1:172.21.15.196:6825/1873969467] conn(0x7f4d5c0ad4b0 0x7f4d5c0af8d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:39.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.867+0000 7f4d717fa700 1 -- 172.21.15.97:0/1530033673 <== osd.11 v2:172.21.15.196:6824/1873969467 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4d6c0020b0 con 0x7f4d5c0ad4b0 2024-04-23T10:24:39.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.883+0000 7f4d7a2e5700 1 -- 172.21.15.97:0/1530033673 --> [v2:172.21.15.196:6824/1873969467,v1:172.21.15.196:6825/1873969467] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4d7406bc80 con 0x7f4d5c0ad4b0 2024-04-23T10:24:39.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.884+0000 7f4d717fa700 1 -- 172.21.15.97:0/1530033673 <== osd.11 v2:172.21.15.196:6824/1873969467 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4d7406bc80 con 0x7f4d5c0ad4b0 2024-04-23T10:24:39.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.884+0000 7f4d7a2e5700 1 -- 172.21.15.97:0/1530033673 >> [v2:172.21.15.196:6824/1873969467,v1:172.21.15.196:6825/1873969467] conn(0x7f4d5c0ad4b0 msgr2=0x7f4d5c0af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:39.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.884+0000 7f4d7a2e5700 1 --2- 172.21.15.97:0/1530033673 >> [v2:172.21.15.196:6824/1873969467,v1:172.21.15.196:6825/1873969467] conn(0x7f4d5c0ad4b0 0x7f4d5c0af8d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:39.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.885+0000 7f4d7a2e5700 1 -- 172.21.15.97:0/1530033673 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f4d5c0a3cd0 msgr2=0x7f4d5c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:39.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.885+0000 7f4d7a2e5700 1 --2- 172.21.15.97:0/1530033673 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f4d5c0a3cd0 0x7f4d5c0a6190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4d640099d0 tx=0x7f4d64008040 comp rx=0 tx=0).stop 2024-04-23T10:24:39.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.885+0000 7f4d7a2e5700 1 -- 172.21.15.97:0/1530033673 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d74107920 msgr2=0x7f4d7419ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:39.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.885+0000 7f4d7a2e5700 1 --2- 172.21.15.97:0/1530033673 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d74107920 0x7f4d7419ca20 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c0062a0 tx=0x7f4d6c004ed0 comp rx=0 tx=0).stop 2024-04-23T10:24:39.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.885+0000 7f4d78882700 1 -- 172.21.15.97:0/1530033673 reap_dead start 2024-04-23T10:24:39.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.885+0000 7f4d7a2e5700 1 -- 172.21.15.97:0/1530033673 shutdown_connections 2024-04-23T10:24:39.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.885+0000 7f4d7a2e5700 1 -- 172.21.15.97:0/1530033673 >> 172.21.15.97:0/1530033673 conn(0x7f4d74078910 msgr2=0x7f4d740ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:39.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.885+0000 7f4d7a2e5700 1 -- 172.21.15.97:0/1530033673 shutdown_connections 2024-04-23T10:24:39.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:39.885+0000 7f4d7a2e5700 1 -- 172.21.15.97:0/1530033673 wait complete. 2024-04-23T10:24:40.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:40.549+0000 7ff3856c2700 1 -- 172.21.15.97:0/1647177402 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff380107920 msgr2=0x7ff380107d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:40.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:40.549+0000 7ff3856c2700 1 --2- 172.21.15.97:0/1647177402 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff380107920 0x7ff380107d00 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff374009990 tx=0x7ff374009ca0 comp rx=0 tx=0).stop 2024-04-23T10:24:40.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:40.549+0000 7ff3856c2700 1 -- 172.21.15.97:0/1647177402 shutdown_connections 2024-04-23T10:24:40.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:40.549+0000 7ff3856c2700 1 --2- 172.21.15.97:0/1647177402 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff380102330 0x7ff38010e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:40.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:40.549+0000 7ff3856c2700 1 --2- 172.21.15.97:0/1647177402 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff380107920 0x7ff380107d00 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:40.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:40.549+0000 7ff3856c2700 1 --2- 172.21.15.97:0/1647177402 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff380101970 0x7ff380101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:40.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:40.549+0000 7ff3856c2700 1 -- 172.21.15.97:0/1647177402 >> 172.21.15.97:0/1647177402 conn(0x7ff3800788f0 msgr2=0x7ff380078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:40.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:40.549+0000 7ff3856c2700 1 -- 172.21.15.97:0/1647177402 shutdown_connections 2024-04-23T10:24:40.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:40.549+0000 7ff3856c2700 1 -- 172.21.15.97:0/1647177402 wait complete. 2024-04-23T10:24:40.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:40.550+0000 7ff3856c2700 1 Processor -- start 2024-04-23T10:24:40.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:40.550+0000 7ff3856c2700 1 -- start start 2024-04-23T10:24:40.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:40.550+0000 7ff3856c2700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff380101970 0x7ff380199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:40.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:40.550+0000 7ff3856c2700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff380102330 0x7ff38019a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:40.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:40.550+0000 7ff3856c2700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff380107920 0x7ff3801a4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:40.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:40.550+0000 7ff3856c2700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff380111b60 con 0x7ff380102330 2024-04-23T10:24:40.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:40.550+0000 7ff3856c2700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff3801119e0 con 0x7ff380101970 2024-04-23T10:24:40.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:40.550+0000 7ff3856c2700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff380111ce0 con 0x7ff380107920 2024-04-23T10:24:40.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:40.550+0000 7ff37e7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff380102330 0x7ff38019a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:40.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:40.550+0000 7ff37e7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff380102330 0x7ff38019a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37588/0 (socket says 172.21.15.97:37588) 2024-04-23T10:24:40.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:40.550+0000 7ff37e7fc700 1 -- 172.21.15.97:0/4283230329 learned_addr learned my addr 172.21.15.97:0/4283230329 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:24:40.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:40.550+0000 7ff37effd700 1 --2- 172.21.15.97:0/4283230329 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff380101970 0x7ff380199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:40.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:40.550+0000 7ff37e7fc700 1 -- 172.21.15.97:0/4283230329 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff380107920 msgr2=0x7ff3801a4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:40.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:40.550+0000 7ff37e7fc700 1 --2- 172.21.15.97:0/4283230329 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff380107920 0x7ff3801a4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:40.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:40.551+0000 7ff37e7fc700 1 -- 172.21.15.97:0/4283230329 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff380101970 msgr2=0x7ff380199f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:40.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:40.551+0000 7ff37e7fc700 1 --2- 172.21.15.97:0/4283230329 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff380101970 0x7ff380199f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:40.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:40.551+0000 7ff37e7fc700 1 -- 172.21.15.97:0/4283230329 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3801a54e0 con 0x7ff380102330 2024-04-23T10:24:40.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:40.551+0000 7ff37effd700 1 --2- 172.21.15.97:0/4283230329 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff380101970 0x7ff380199f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:24:40.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:40.551+0000 7ff37e7fc700 1 --2- 172.21.15.97:0/4283230329 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff380102330 0x7ff38019a460 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff37000b530 tx=0x7ff37000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:40.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:40.551+0000 7ff367fff700 1 -- 172.21.15.97:0/4283230329 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff370015020 con 0x7ff380102330 2024-04-23T10:24:40.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:40.551+0000 7ff367fff700 1 -- 172.21.15.97:0/4283230329 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff370011ce0 con 0x7ff380102330 2024-04-23T10:24:40.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:40.551+0000 7ff3856c2700 1 -- 172.21.15.97:0/4283230329 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3801a5710 con 0x7ff380102330 2024-04-23T10:24:40.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:40.551+0000 7ff367fff700 1 -- 172.21.15.97:0/4283230329 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff370010d20 con 0x7ff380102330 2024-04-23T10:24:40.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:40.551+0000 7ff3856c2700 1 -- 172.21.15.97:0/4283230329 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3801a5c00 con 0x7ff380102330 2024-04-23T10:24:40.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:40.552+0000 7ff3856c2700 1 -- 172.21.15.97:0/4283230329 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff380063150 con 0x7ff380102330 2024-04-23T10:24:40.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:40.553+0000 7ff367fff700 1 -- 172.21.15.97:0/4283230329 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff370018020 con 0x7ff380102330 2024-04-23T10:24:40.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:40.553+0000 7ff367fff700 1 --2- 172.21.15.97:0/4283230329 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7ff3680a3cd0 0x7ff3680a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:40.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:40.554+0000 7ff37effd700 1 --2- 172.21.15.97:0/4283230329 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7ff3680a3cd0 0x7ff3680a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:40.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:40.554+0000 7ff367fff700 1 -- 172.21.15.97:0/4283230329 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7ff3700d0280 con 0x7ff380102330 2024-04-23T10:24:40.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:40.554+0000 7ff367fff700 1 --2- 172.21.15.97:0/4283230329 >> [v2:172.21.15.196:6808/2804446896,v1:172.21.15.196:6809/2804446896] conn(0x7ff3680ad4b0 0x7ff3680af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:40.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:40.554+0000 7ff367fff700 1 -- 172.21.15.97:0/4283230329 --> [v2:172.21.15.196:6808/2804446896,v1:172.21.15.196:6809/2804446896] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff3680aff80 con 0x7ff3680ad4b0 2024-04-23T10:24:40.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:40.554+0000 7ff367fff700 1 -- 172.21.15.97:0/4283230329 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7ff3700d0760 con 0x7ff380102330 2024-04-23T10:24:40.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:40.554+0000 7ff37effd700 1 --2- 172.21.15.97:0/4283230329 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7ff3680a3cd0 0x7ff3680a6190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff374006010 tx=0x7ff37400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:40.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:40.554+0000 7ff37f7fe700 1 --2- 172.21.15.97:0/4283230329 >> [v2:172.21.15.196:6808/2804446896,v1:172.21.15.196:6809/2804446896] conn(0x7ff3680ad4b0 0x7ff3680af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:40.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:40.555+0000 7ff37f7fe700 1 --2- 172.21.15.97:0/4283230329 >> [v2:172.21.15.196:6808/2804446896,v1:172.21.15.196:6809/2804446896] conn(0x7ff3680ad4b0 0x7ff3680af8d0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:40.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:40.555+0000 7ff367fff700 1 -- 172.21.15.97:0/4283230329 <== osd.5 v2:172.21.15.196:6808/2804446896 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff3680aff80 con 0x7ff3680ad4b0 2024-04-23T10:24:40.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:40.571+0000 7ff3856c2700 1 -- 172.21.15.97:0/4283230329 --> [v2:172.21.15.196:6808/2804446896,v1:172.21.15.196:6809/2804446896] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff3800635d0 con 0x7ff3680ad4b0 2024-04-23T10:24:40.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:40.572+0000 7ff367fff700 1 -- 172.21.15.97:0/4283230329 <== osd.5 v2:172.21.15.196:6808/2804446896 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff3800635d0 con 0x7ff3680ad4b0 2024-04-23T10:24:40.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:40.572+0000 7ff3856c2700 1 -- 172.21.15.97:0/4283230329 >> [v2:172.21.15.196:6808/2804446896,v1:172.21.15.196:6809/2804446896] conn(0x7ff3680ad4b0 msgr2=0x7ff3680af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:40.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:40.572+0000 7ff3856c2700 1 --2- 172.21.15.97:0/4283230329 >> [v2:172.21.15.196:6808/2804446896,v1:172.21.15.196:6809/2804446896] conn(0x7ff3680ad4b0 0x7ff3680af8d0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:40.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:40.572+0000 7ff3856c2700 1 -- 172.21.15.97:0/4283230329 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7ff3680a3cd0 msgr2=0x7ff3680a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:40.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:40.572+0000 7ff3856c2700 1 --2- 172.21.15.97:0/4283230329 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7ff3680a3cd0 0x7ff3680a6190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff374006010 tx=0x7ff37400b540 comp rx=0 tx=0).stop 2024-04-23T10:24:40.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:40.572+0000 7ff3856c2700 1 -- 172.21.15.97:0/4283230329 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff380102330 msgr2=0x7ff38019a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:40.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:40.572+0000 7ff3856c2700 1 --2- 172.21.15.97:0/4283230329 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff380102330 0x7ff38019a460 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff37000b530 tx=0x7ff37000b8f0 comp rx=0 tx=0).stop 2024-04-23T10:24:40.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:40.572+0000 7ff37f7fe700 1 -- 172.21.15.97:0/4283230329 reap_dead start 2024-04-23T10:24:40.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:40.572+0000 7ff3856c2700 1 -- 172.21.15.97:0/4283230329 shutdown_connections 2024-04-23T10:24:40.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:40.572+0000 7ff3856c2700 1 -- 172.21.15.97:0/4283230329 >> 172.21.15.97:0/4283230329 conn(0x7ff3800788f0 msgr2=0x7ff3800ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:40.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:40.572+0000 7ff3856c2700 1 -- 172.21.15.97:0/4283230329 shutdown_connections 2024-04-23T10:24:40.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:40.573+0000 7ff3856c2700 1 -- 172.21.15.97:0/4283230329 wait complete. 2024-04-23T10:24:40.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:40 smithi097 ceph-mon[29197]: pgmap v306: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-23T10:24:40.894 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:40 smithi196 ceph-mon[32919]: pgmap v306: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-23T10:24:40.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:40 smithi183 ceph-mon[32317]: pgmap v306: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-23T10:24:41.400 INFO:teuthology.orchestra.run.smithi097.stdout:313532612633 2024-04-23T10:24:41.401 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-23T10:24:42.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:42.576+0000 7f8f47945700 1 -- 172.21.15.97:0/2127979112 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f40101d10 msgr2=0x7f8f40105d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:42.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:42.576+0000 7f8f47945700 1 --2- 172.21.15.97:0/2127979112 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f40101d10 0x7f8f40105d10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8f300099c0 tx=0x7f8f30009cd0 comp rx=0 tx=0).stop 2024-04-23T10:24:42.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:42.576+0000 7f8f47945700 1 -- 172.21.15.97:0/2127979112 shutdown_connections 2024-04-23T10:24:42.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:42.576+0000 7f8f47945700 1 --2- 172.21.15.97:0/2127979112 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f401063f0 0x7f8f4010dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:42.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:42.576+0000 7f8f47945700 1 --2- 172.21.15.97:0/2127979112 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f40101360 0x7f8f40101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:42.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:42.576+0000 7f8f47945700 1 --2- 172.21.15.97:0/2127979112 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f40101d10 0x7f8f40105d10 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:42.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:42.576+0000 7f8f47945700 1 -- 172.21.15.97:0/2127979112 >> 172.21.15.97:0/2127979112 conn(0x7f8f40077fb0 msgr2=0x7f8f400ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:42.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:42.576+0000 7f8f47945700 1 -- 172.21.15.97:0/2127979112 shutdown_connections 2024-04-23T10:24:42.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:42.576+0000 7f8f47945700 1 -- 172.21.15.97:0/2127979112 wait complete. 2024-04-23T10:24:42.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:42.577+0000 7f8f47945700 1 Processor -- start 2024-04-23T10:24:42.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:42.577+0000 7f8f47945700 1 -- start start 2024-04-23T10:24:42.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:42.577+0000 7f8f47945700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f40101360 0x7f8f40199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:42.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:42.577+0000 7f8f47945700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f40101d10 0x7f8f4019a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:42.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:42.577+0000 7f8f47945700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f401063f0 0x7f8f401a4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:42.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:42.577+0000 7f8f47945700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8f401112e0 con 0x7f8f40101d10 2024-04-23T10:24:42.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:42.577+0000 7f8f47945700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8f40111160 con 0x7f8f40101360 2024-04-23T10:24:42.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:42.577+0000 7f8f47945700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8f40111460 con 0x7f8f401063f0 2024-04-23T10:24:42.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:42.577+0000 7f8f44ee0700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f40101d10 0x7f8f4019a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:42.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:42.577+0000 7f8f44ee0700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f40101d10 0x7f8f4019a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37600/0 (socket says 172.21.15.97:37600) 2024-04-23T10:24:42.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:42.577+0000 7f8f44ee0700 1 -- 172.21.15.97:0/3750822427 learned_addr learned my addr 172.21.15.97:0/3750822427 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:24:42.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:42.577+0000 7f8f456e1700 1 --2- 172.21.15.97:0/3750822427 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f40101360 0x7f8f40199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:42.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:42.577+0000 7f8f45ee2700 1 --2- 172.21.15.97:0/3750822427 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f401063f0 0x7f8f401a4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:42.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:42.577+0000 7f8f44ee0700 1 -- 172.21.15.97:0/3750822427 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f401063f0 msgr2=0x7f8f401a4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:42.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:42.577+0000 7f8f44ee0700 1 --2- 172.21.15.97:0/3750822427 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f401063f0 0x7f8f401a4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:42.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:42.577+0000 7f8f44ee0700 1 -- 172.21.15.97:0/3750822427 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f40101360 msgr2=0x7f8f40199f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:42.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:42.577+0000 7f8f44ee0700 1 --2- 172.21.15.97:0/3750822427 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f40101360 0x7f8f40199f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:42.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:42.577+0000 7f8f44ee0700 1 -- 172.21.15.97:0/3750822427 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f401a54e0 con 0x7f8f40101d10 2024-04-23T10:24:42.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:42.577+0000 7f8f456e1700 1 --2- 172.21.15.97:0/3750822427 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f40101360 0x7f8f40199f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:24:42.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:42.577+0000 7f8f45ee2700 1 --2- 172.21.15.97:0/3750822427 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f401063f0 0x7f8f401a4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:24:42.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:42.577+0000 7f8f44ee0700 1 --2- 172.21.15.97:0/3750822427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f40101d10 0x7f8f4019a460 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8f30009f50 tx=0x7f8f300037c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:42.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:42.578+0000 7f8f367fc700 1 -- 172.21.15.97:0/3750822427 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f30027070 con 0x7f8f40101d10 2024-04-23T10:24:42.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:42.578+0000 7f8f47945700 1 -- 172.21.15.97:0/3750822427 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f401a56b0 con 0x7f8f40101d10 2024-04-23T10:24:42.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:42.578+0000 7f8f367fc700 1 -- 172.21.15.97:0/3750822427 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f30004140 con 0x7f8f40101d10 2024-04-23T10:24:42.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:42.578+0000 7f8f367fc700 1 -- 172.21.15.97:0/3750822427 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f3000f460 con 0x7f8f40101d10 2024-04-23T10:24:42.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:42.578+0000 7f8f47945700 1 -- 172.21.15.97:0/3750822427 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f401a5b60 con 0x7f8f40101d10 2024-04-23T10:24:42.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:42.578+0000 7f8f47945700 1 -- 172.21.15.97:0/3750822427 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8f400ffd40 con 0x7f8f40101d10 2024-04-23T10:24:42.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:42.580+0000 7f8f367fc700 1 -- 172.21.15.97:0/3750822427 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8f3002a070 con 0x7f8f40101d10 2024-04-23T10:24:42.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:42.580+0000 7f8f367fc700 1 --2- 172.21.15.97:0/3750822427 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f8f2c0a3cd0 0x7f8f2c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:42.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:42.580+0000 7f8f456e1700 1 --2- 172.21.15.97:0/3750822427 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f8f2c0a3cd0 0x7f8f2c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:42.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:42.580+0000 7f8f367fc700 1 -- 172.21.15.97:0/3750822427 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f8f300d1a60 con 0x7f8f40101d10 2024-04-23T10:24:42.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:42.581+0000 7f8f367fc700 1 --2- 172.21.15.97:0/3750822427 >> [v2:172.21.15.196:6800/2877039731,v1:172.21.15.196:6801/2877039731] conn(0x7f8f2c0ad4b0 0x7f8f2c0af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:42.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:42.581+0000 7f8f367fc700 1 -- 172.21.15.97:0/3750822427 --> [v2:172.21.15.196:6800/2877039731,v1:172.21.15.196:6801/2877039731] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8f2c0aff80 con 0x7f8f2c0ad4b0 2024-04-23T10:24:42.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:42.581+0000 7f8f367fc700 1 -- 172.21.15.97:0/3750822427 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f8f300d1f40 con 0x7f8f40101d10 2024-04-23T10:24:42.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:42.581+0000 7f8f456e1700 1 --2- 172.21.15.97:0/3750822427 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f8f2c0a3cd0 0x7f8f2c0a6190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c0099d0 tx=0x7f8f3c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:42.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:42.581+0000 7f8f45ee2700 1 --2- 172.21.15.97:0/3750822427 >> [v2:172.21.15.196:6800/2877039731,v1:172.21.15.196:6801/2877039731] conn(0x7f8f2c0ad4b0 0x7f8f2c0af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:42.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:42.581+0000 7f8f45ee2700 1 --2- 172.21.15.97:0/3750822427 >> [v2:172.21.15.196:6800/2877039731,v1:172.21.15.196:6801/2877039731] conn(0x7f8f2c0ad4b0 0x7f8f2c0af8d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:42.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:42.582+0000 7f8f367fc700 1 -- 172.21.15.97:0/3750822427 <== osd.2 v2:172.21.15.196:6800/2877039731 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8f2c0aff80 con 0x7f8f2c0ad4b0 2024-04-23T10:24:42.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:42.598+0000 7f8f47945700 1 -- 172.21.15.97:0/3750822427 --> [v2:172.21.15.196:6800/2877039731,v1:172.21.15.196:6801/2877039731] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8f401027f0 con 0x7f8f2c0ad4b0 2024-04-23T10:24:42.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:42.599+0000 7f8f367fc700 1 -- 172.21.15.97:0/3750822427 <== osd.2 v2:172.21.15.196:6800/2877039731 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8f401027f0 con 0x7f8f2c0ad4b0 2024-04-23T10:24:42.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:42.599+0000 7f8f47945700 1 -- 172.21.15.97:0/3750822427 >> [v2:172.21.15.196:6800/2877039731,v1:172.21.15.196:6801/2877039731] conn(0x7f8f2c0ad4b0 msgr2=0x7f8f2c0af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:42.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:42.599+0000 7f8f47945700 1 --2- 172.21.15.97:0/3750822427 >> [v2:172.21.15.196:6800/2877039731,v1:172.21.15.196:6801/2877039731] conn(0x7f8f2c0ad4b0 0x7f8f2c0af8d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:42.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:42.599+0000 7f8f47945700 1 -- 172.21.15.97:0/3750822427 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f8f2c0a3cd0 msgr2=0x7f8f2c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:42.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:42.599+0000 7f8f47945700 1 --2- 172.21.15.97:0/3750822427 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f8f2c0a3cd0 0x7f8f2c0a6190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c0099d0 tx=0x7f8f3c008040 comp rx=0 tx=0).stop 2024-04-23T10:24:42.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:42.599+0000 7f8f47945700 1 -- 172.21.15.97:0/3750822427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f40101d10 msgr2=0x7f8f4019a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:42.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:42.599+0000 7f8f47945700 1 --2- 172.21.15.97:0/3750822427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8f40101d10 0x7f8f4019a460 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8f30009f50 tx=0x7f8f300037c0 comp rx=0 tx=0).stop 2024-04-23T10:24:42.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:42.599+0000 7f8f45ee2700 1 -- 172.21.15.97:0/3750822427 reap_dead start 2024-04-23T10:24:42.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:42.599+0000 7f8f47945700 1 -- 172.21.15.97:0/3750822427 shutdown_connections 2024-04-23T10:24:42.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:42.599+0000 7f8f47945700 1 -- 172.21.15.97:0/3750822427 >> 172.21.15.97:0/3750822427 conn(0x7f8f40077fb0 msgr2=0x7f8f400ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:42.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:42.599+0000 7f8f47945700 1 -- 172.21.15.97:0/3750822427 shutdown_connections 2024-04-23T10:24:42.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:42.599+0000 7f8f47945700 1 -- 172.21.15.97:0/3750822427 wait complete. 2024-04-23T10:24:42.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:42 smithi097 ceph-mon[29197]: pgmap v307: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-23T10:24:42.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:42 smithi183 ceph-mon[32317]: pgmap v307: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-23T10:24:43.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:42 smithi196 ceph-mon[32919]: pgmap v307: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-23T10:24:44.884 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:44 smithi097 ceph-mon[29197]: pgmap v308: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-23T10:24:44.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:44 smithi183 ceph-mon[32317]: pgmap v308: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-23T10:24:44.910 INFO:teuthology.orchestra.run.smithi097.stdout:167503724595 2024-04-23T10:24:44.911 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-23T10:24:45.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:44 smithi196 ceph-mon[32919]: pgmap v308: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-23T10:24:45.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:37.076+0000 7fa548819700 1 -- 172.21.15.97:0/1648651456 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa544101940 msgr2=0x7fa544101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:45.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:37.076+0000 7fa548819700 1 --2- 172.21.15.97:0/1648651456 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa544101940 0x7fa544101dc0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa538009990 tx=0x7fa538009ca0 comp rx=0 tx=0).stop 2024-04-23T10:24:45.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:37.076+0000 7fa548819700 1 -- 172.21.15.97:0/1648651456 shutdown_connections 2024-04-23T10:24:45.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:37.077+0000 7fa548819700 1 --2- 172.21.15.97:0/1648651456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa544102300 0x7fa54410e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:45.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:37.077+0000 7fa548819700 1 --2- 172.21.15.97:0/1648651456 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa544107990 0x7fa544107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:45.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:37.077+0000 7fa548819700 1 --2- 172.21.15.97:0/1648651456 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa544101940 0x7fa544101dc0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:45.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:37.077+0000 7fa548819700 1 -- 172.21.15.97:0/1648651456 >> 172.21.15.97:0/1648651456 conn(0x7fa5440788f0 msgr2=0x7fa544078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:45.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:37.077+0000 7fa548819700 1 -- 172.21.15.97:0/1648651456 shutdown_connections 2024-04-23T10:24:45.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:37.077+0000 7fa548819700 1 -- 172.21.15.97:0/1648651456 wait complete. 2024-04-23T10:24:45.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:37.077+0000 7fa548819700 1 Processor -- start 2024-04-23T10:24:45.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:37.077+0000 7fa548819700 1 -- start start 2024-04-23T10:24:45.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:37.078+0000 7fa548819700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa544101940 0x7fa544199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:45.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:37.078+0000 7fa548819700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa544102300 0x7fa544199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:45.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:37.078+0000 7fa548819700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa544107990 0x7fa54419d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:45.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:37.078+0000 7fa548819700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa544111ae0 con 0x7fa544107990 2024-04-23T10:24:45.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:37.078+0000 7fa548819700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa544111960 con 0x7fa544101940 2024-04-23T10:24:45.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:37.078+0000 7fa548819700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa544111c60 con 0x7fa544102300 2024-04-23T10:24:45.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:37.078+0000 7fa542d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa544107990 0x7fa54419d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:45.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:37.078+0000 7fa542d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa544107990 0x7fa54419d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37520/0 (socket says 172.21.15.97:37520) 2024-04-23T10:24:45.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:37.078+0000 7fa542d9d700 1 -- 172.21.15.97:0/711621710 learned_addr learned my addr 172.21.15.97:0/711621710 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:24:45.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:37.078+0000 7fa54259c700 1 --2- 172.21.15.97:0/711621710 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa544101940 0x7fa544199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:45.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:37.078+0000 7fa541d9b700 1 --2- 172.21.15.97:0/711621710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa544102300 0x7fa544199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:45.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:37.078+0000 7fa542d9d700 1 -- 172.21.15.97:0/711621710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa544102300 msgr2=0x7fa544199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:45.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:37.078+0000 7fa542d9d700 1 --2- 172.21.15.97:0/711621710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa544102300 0x7fa544199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:45.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:37.078+0000 7fa542d9d700 1 -- 172.21.15.97:0/711621710 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa544101940 msgr2=0x7fa544199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:45.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:37.078+0000 7fa542d9d700 1 --2- 172.21.15.97:0/711621710 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa544101940 0x7fa544199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:45.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:37.078+0000 7fa542d9d700 1 -- 172.21.15.97:0/711621710 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa54419e0f0 con 0x7fa544107990 2024-04-23T10:24:45.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:37.078+0000 7fa54259c700 1 --2- 172.21.15.97:0/711621710 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa544101940 0x7fa544199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:24:45.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:37.078+0000 7fa542d9d700 1 --2- 172.21.15.97:0/711621710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa544107990 0x7fa54419d9b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa53c00ada0 tx=0x7fa53c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:45.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:37.078+0000 7fa5337fe700 1 -- 172.21.15.97:0/711621710 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa53c017020 con 0x7fa544107990 2024-04-23T10:24:45.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:37.078+0000 7fa548819700 1 -- 172.21.15.97:0/711621710 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa54419e3e0 con 0x7fa544107990 2024-04-23T10:24:45.276 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:37.078+0000 7fa5337fe700 1 -- 172.21.15.97:0/711621710 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa53c00ce30 con 0x7fa544107990 2024-04-23T10:24:45.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:37.078+0000 7fa5337fe700 1 -- 172.21.15.97:0/711621710 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa53c0054a0 con 0x7fa544107990 2024-04-23T10:24:45.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:37.078+0000 7fa548819700 1 -- 172.21.15.97:0/711621710 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5441a5c60 con 0x7fa544107990 2024-04-23T10:24:45.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:37.079+0000 7fa548819700 1 -- 172.21.15.97:0/711621710 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa544063110 con 0x7fa544107990 2024-04-23T10:24:45.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:37.081+0000 7fa5337fe700 1 -- 172.21.15.97:0/711621710 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa53c00c980 con 0x7fa544107990 2024-04-23T10:24:45.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:37.081+0000 7fa5337fe700 1 --2- 172.21.15.97:0/711621710 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fa52c0a3cd0 0x7fa52c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:45.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:37.082+0000 7fa54259c700 1 --2- 172.21.15.97:0/711621710 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fa52c0a3cd0 0x7fa52c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:45.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:37.082+0000 7fa5337fe700 1 -- 172.21.15.97:0/711621710 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fa53c0d0620 con 0x7fa544107990 2024-04-23T10:24:45.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:37.082+0000 7fa5337fe700 1 --2- 172.21.15.97:0/711621710 >> [v2:172.21.15.97:6802/3031221802,v1:172.21.15.97:6803/3031221802] conn(0x7fa52c0ad4b0 0x7fa52c0af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:45.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:37.082+0000 7fa5337fe700 1 -- 172.21.15.97:0/711621710 --> [v2:172.21.15.97:6802/3031221802,v1:172.21.15.97:6803/3031221802] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa53c019e50 con 0x7fa52c0ad4b0 2024-04-23T10:24:45.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:37.082+0000 7fa5337fe700 1 -- 172.21.15.97:0/711621710 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fa53c0d0b00 con 0x7fa544107990 2024-04-23T10:24:45.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:37.082+0000 7fa54259c700 1 --2- 172.21.15.97:0/711621710 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fa52c0a3cd0 0x7fa52c0a6190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa534005fd0 tx=0x7fa53400a5e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:45.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:37.082+0000 7fa541d9b700 1 --2- 172.21.15.97:0/711621710 >> [v2:172.21.15.97:6802/3031221802,v1:172.21.15.97:6803/3031221802] conn(0x7fa52c0ad4b0 0x7fa52c0af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:45.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:37.082+0000 7fa541d9b700 1 --2- 172.21.15.97:0/711621710 >> [v2:172.21.15.97:6802/3031221802,v1:172.21.15.97:6803/3031221802] conn(0x7fa52c0ad4b0 0x7fa52c0af8d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:45.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:37.083+0000 7fa5337fe700 1 -- 172.21.15.97:0/711621710 <== osd.0 v2:172.21.15.97:6802/3031221802 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa53c019e50 con 0x7fa52c0ad4b0 2024-04-23T10:24:45.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:37.099+0000 7fa548819700 1 -- 172.21.15.97:0/711621710 --> [v2:172.21.15.97:6802/3031221802,v1:172.21.15.97:6803/3031221802] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa544063590 con 0x7fa52c0ad4b0 2024-04-23T10:24:45.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:37.099+0000 7fa5337fe700 1 -- 172.21.15.97:0/711621710 <== osd.0 v2:172.21.15.97:6802/3031221802 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa544063590 con 0x7fa52c0ad4b0 2024-04-23T10:24:45.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:37.099+0000 7fa548819700 1 -- 172.21.15.97:0/711621710 >> [v2:172.21.15.97:6802/3031221802,v1:172.21.15.97:6803/3031221802] conn(0x7fa52c0ad4b0 msgr2=0x7fa52c0af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:45.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:37.099+0000 7fa548819700 1 --2- 172.21.15.97:0/711621710 >> [v2:172.21.15.97:6802/3031221802,v1:172.21.15.97:6803/3031221802] conn(0x7fa52c0ad4b0 0x7fa52c0af8d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:45.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:37.099+0000 7fa548819700 1 -- 172.21.15.97:0/711621710 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fa52c0a3cd0 msgr2=0x7fa52c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:45.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:37.099+0000 7fa548819700 1 --2- 172.21.15.97:0/711621710 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fa52c0a3cd0 0x7fa52c0a6190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa534005fd0 tx=0x7fa53400a5e0 comp rx=0 tx=0).stop 2024-04-23T10:24:45.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:37.099+0000 7fa548819700 1 -- 172.21.15.97:0/711621710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa544107990 msgr2=0x7fa54419d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:45.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:37.099+0000 7fa548819700 1 --2- 172.21.15.97:0/711621710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa544107990 0x7fa54419d9b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa53c00ada0 tx=0x7fa53c00c330 comp rx=0 tx=0).stop 2024-04-23T10:24:45.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:37.099+0000 7fa542d9d700 1 -- 172.21.15.97:0/711621710 reap_dead start 2024-04-23T10:24:45.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:37.100+0000 7fa548819700 1 -- 172.21.15.97:0/711621710 shutdown_connections 2024-04-23T10:24:45.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:37.100+0000 7fa548819700 1 -- 172.21.15.97:0/711621710 >> 172.21.15.97:0/711621710 conn(0x7fa5440788f0 msgr2=0x7fa54406b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:45.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:37.100+0000 7fa548819700 1 -- 172.21.15.97:0/711621710 shutdown_connections 2024-04-23T10:24:45.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:37.100+0000 7fa548819700 1 -- 172.21.15.97:0/711621710 wait complete. 2024-04-23T10:24:45.459 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T10:24:45.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:44.435+0000 7f37cf1dd700 1 -- 172.21.15.97:0/734883898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37c8102ed0 msgr2=0x7f37c810e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:45.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:44.435+0000 7f37cf1dd700 1 --2- 172.21.15.97:0/734883898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37c8102ed0 0x7f37c810e8e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f37c000b0d0 tx=0x7f37c000b3e0 comp rx=0 tx=0).stop 2024-04-23T10:24:45.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:44.435+0000 7f37cf1dd700 1 -- 172.21.15.97:0/734883898 shutdown_connections 2024-04-23T10:24:45.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:44.435+0000 7f37cf1dd700 1 --2- 172.21.15.97:0/734883898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37c8102ed0 0x7f37c810e8e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:45.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:44.435+0000 7f37cf1dd700 1 --2- 172.21.15.97:0/734883898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37c8107c30 0x7f37c8108010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:45.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:44.435+0000 7f37cf1dd700 1 --2- 172.21.15.97:0/734883898 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37c8102510 0x7f37c8102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:45.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:44.435+0000 7f37cf1dd700 1 -- 172.21.15.97:0/734883898 >> 172.21.15.97:0/734883898 conn(0x7f37c8076c60 msgr2=0x7f37c8077070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:45.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:44.436+0000 7f37cf1dd700 1 -- 172.21.15.97:0/734883898 shutdown_connections 2024-04-23T10:24:45.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:44.436+0000 7f37cf1dd700 1 -- 172.21.15.97:0/734883898 wait complete. 2024-04-23T10:24:45.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:44.436+0000 7f37cf1dd700 1 Processor -- start 2024-04-23T10:24:45.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:44.436+0000 7f37cf1dd700 1 -- start start 2024-04-23T10:24:45.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:44.437+0000 7f37cf1dd700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37c8102510 0x7f37c8198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:45.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:44.437+0000 7f37cf1dd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37c8102ed0 0x7f37c81986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:45.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:44.437+0000 7f37cf1dd700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37c8107c30 0x7f37c819ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:45.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:44.437+0000 7f37cf1dd700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f37c8111d30 con 0x7f37c8102ed0 2024-04-23T10:24:45.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:44.437+0000 7f37cf1dd700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f37c8111bb0 con 0x7f37c8102510 2024-04-23T10:24:45.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:44.437+0000 7f37cf1dd700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f37c8111eb0 con 0x7f37c8107c30 2024-04-23T10:24:45.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:44.437+0000 7f37c7fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37c8102ed0 0x7f37c81986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:45.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:44.437+0000 7f37c7fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37c8102ed0 0x7f37c81986a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37620/0 (socket says 172.21.15.97:37620) 2024-04-23T10:24:45.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:44.437+0000 7f37c7fff700 1 -- 172.21.15.97:0/3701798033 learned_addr learned my addr 172.21.15.97:0/3701798033 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:24:45.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:44.437+0000 7f37cd77a700 1 --2- 172.21.15.97:0/3701798033 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37c8107c30 0x7f37c819ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:45.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:44.437+0000 7f37ccf79700 1 --2- 172.21.15.97:0/3701798033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37c8102510 0x7f37c8198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:45.555 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:44.437+0000 7f37c7fff700 1 -- 172.21.15.97:0/3701798033 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37c8107c30 msgr2=0x7f37c819ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:45.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:44.437+0000 7f37c7fff700 1 --2- 172.21.15.97:0/3701798033 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37c8107c30 0x7f37c819ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:45.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:44.437+0000 7f37c7fff700 1 -- 172.21.15.97:0/3701798033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37c8102510 msgr2=0x7f37c8198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:45.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:44.437+0000 7f37c7fff700 1 --2- 172.21.15.97:0/3701798033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37c8102510 0x7f37c8198160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:45.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:44.437+0000 7f37c7fff700 1 -- 172.21.15.97:0/3701798033 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37c819d170 con 0x7f37c8102ed0 2024-04-23T10:24:45.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:44.437+0000 7f37ccf79700 1 --2- 172.21.15.97:0/3701798033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37c8102510 0x7f37c8198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:24:45.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:44.437+0000 7f37c7fff700 1 --2- 172.21.15.97:0/3701798033 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37c8102ed0 0x7f37c81986a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f37b400d9e0 tx=0x7f37b400dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:45.556 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:44.437+0000 7f37c5ffb700 1 -- 172.21.15.97:0/3701798033 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b401b070 con 0x7f37c8102ed0 2024-04-23T10:24:45.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:44.437+0000 7f37c5ffb700 1 -- 172.21.15.97:0/3701798033 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37b400ecd0 con 0x7f37c8102ed0 2024-04-23T10:24:45.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:44.437+0000 7f37cf1dd700 1 -- 172.21.15.97:0/3701798033 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37c819d460 con 0x7f37c8102ed0 2024-04-23T10:24:45.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:44.437+0000 7f37c5ffb700 1 -- 172.21.15.97:0/3701798033 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b40146c0 con 0x7f37c8102ed0 2024-04-23T10:24:45.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:44.438+0000 7f37cf1dd700 1 -- 172.21.15.97:0/3701798033 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37c8101400 con 0x7f37c8102ed0 2024-04-23T10:24:45.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:44.438+0000 7f37cf1dd700 1 -- 172.21.15.97:0/3701798033 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f37c8101a10 con 0x7f37c8102ed0 2024-04-23T10:24:45.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:44.440+0000 7f37c5ffb700 1 -- 172.21.15.97:0/3701798033 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f37b401e070 con 0x7f37c8102ed0 2024-04-23T10:24:45.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:44.440+0000 7f37c5ffb700 1 --2- 172.21.15.97:0/3701798033 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f37b80a3cd0 0x7f37b80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:45.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:44.440+0000 7f37ccf79700 1 --2- 172.21.15.97:0/3701798033 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f37b80a3cd0 0x7f37b80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:45.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:44.440+0000 7f37c5ffb700 1 -- 172.21.15.97:0/3701798033 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f37b40d20b0 con 0x7f37c8102ed0 2024-04-23T10:24:45.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:44.440+0000 7f37c5ffb700 1 --2- 172.21.15.97:0/3701798033 >> [v2:172.21.15.97:6826/1887432474,v1:172.21.15.97:6827/1887432474] conn(0x7f37b80ad4b0 0x7f37b80af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:45.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:44.440+0000 7f37c5ffb700 1 -- 172.21.15.97:0/3701798033 --> [v2:172.21.15.97:6826/1887432474,v1:172.21.15.97:6827/1887432474] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f37b80aff80 con 0x7f37b80ad4b0 2024-04-23T10:24:45.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:44.440+0000 7f37c5ffb700 1 -- 172.21.15.97:0/3701798033 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f37b40d2590 con 0x7f37c8102ed0 2024-04-23T10:24:45.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:44.441+0000 7f37ccf79700 1 --2- 172.21.15.97:0/3701798033 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f37b80a3cd0 0x7f37b80a6190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f37bc007900 tx=0x7f37bc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:45.557 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:44.441+0000 7f37cd77a700 1 --2- 172.21.15.97:0/3701798033 >> [v2:172.21.15.97:6826/1887432474,v1:172.21.15.97:6827/1887432474] conn(0x7f37b80ad4b0 0x7f37b80af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:45.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:44.441+0000 7f37cd77a700 1 --2- 172.21.15.97:0/3701798033 >> [v2:172.21.15.97:6826/1887432474,v1:172.21.15.97:6827/1887432474] conn(0x7f37b80ad4b0 0x7f37b80af8d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:45.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:44.442+0000 7f37c5ffb700 1 -- 172.21.15.97:0/3701798033 <== osd.9 v2:172.21.15.97:6826/1887432474 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f37b80aff80 con 0x7f37b80ad4b0 2024-04-23T10:24:45.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:44.459+0000 7f37cf1dd700 1 -- 172.21.15.97:0/3701798033 --> [v2:172.21.15.97:6826/1887432474,v1:172.21.15.97:6827/1887432474] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f37c8101e90 con 0x7f37b80ad4b0 2024-04-23T10:24:45.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:44.460+0000 7f37c5ffb700 1 -- 172.21.15.97:0/3701798033 <== osd.9 v2:172.21.15.97:6826/1887432474 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f37c8101e90 con 0x7f37b80ad4b0 2024-04-23T10:24:45.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:44.460+0000 7f37cf1dd700 1 -- 172.21.15.97:0/3701798033 >> [v2:172.21.15.97:6826/1887432474,v1:172.21.15.97:6827/1887432474] conn(0x7f37b80ad4b0 msgr2=0x7f37b80af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:45.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:44.460+0000 7f37cf1dd700 1 --2- 172.21.15.97:0/3701798033 >> [v2:172.21.15.97:6826/1887432474,v1:172.21.15.97:6827/1887432474] conn(0x7f37b80ad4b0 0x7f37b80af8d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:45.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:44.460+0000 7f37cf1dd700 1 -- 172.21.15.97:0/3701798033 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f37b80a3cd0 msgr2=0x7f37b80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:45.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:44.460+0000 7f37cf1dd700 1 --2- 172.21.15.97:0/3701798033 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f37b80a3cd0 0x7f37b80a6190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f37bc007900 tx=0x7f37bc008040 comp rx=0 tx=0).stop 2024-04-23T10:24:45.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:44.460+0000 7f37cf1dd700 1 -- 172.21.15.97:0/3701798033 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37c8102ed0 msgr2=0x7f37c81986a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:45.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:44.460+0000 7f37cf1dd700 1 --2- 172.21.15.97:0/3701798033 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37c8102ed0 0x7f37c81986a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f37b400d9e0 tx=0x7f37b400dda0 comp rx=0 tx=0).stop 2024-04-23T10:24:45.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:44.460+0000 7f37cd77a700 1 -- 172.21.15.97:0/3701798033 reap_dead start 2024-04-23T10:24:45.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:44.460+0000 7f37cf1dd700 1 -- 172.21.15.97:0/3701798033 shutdown_connections 2024-04-23T10:24:45.558 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:44.460+0000 7f37cf1dd700 1 -- 172.21.15.97:0/3701798033 >> 172.21.15.97:0/3701798033 conn(0x7f37c8076c60 msgr2=0x7f37c80ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:45.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:44.460+0000 7f37cf1dd700 1 -- 172.21.15.97:0/3701798033 shutdown_connections 2024-04-23T10:24:45.559 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:44.460+0000 7f37cf1dd700 1 -- 172.21.15.97:0/3701798033 wait complete. 2024-04-23T10:24:46.681 INFO:teuthology.orchestra.run.smithi097.stdout:77309411393 2024-04-23T10:24:46.681 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-23T10:24:46.791 INFO:teuthology.orchestra.run.smithi097.stdout:34359738444 2024-04-23T10:24:46.791 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-23T10:24:46.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:46 smithi183 ceph-mon[32317]: pgmap v309: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-23T10:24:46.907 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:46 smithi097 ceph-mon[29197]: pgmap v309: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-23T10:24:47.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:46 smithi196 ceph-mon[32919]: pgmap v309: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-23T10:24:47.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:47.920+0000 7fd0e7987700 1 -- 172.21.15.97:0/2294076729 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0e0108a20 msgr2=0x7fd0e0108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:47.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:47.920+0000 7fd0e7987700 1 --2- 172.21.15.97:0/2294076729 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0e0108a20 0x7fd0e0108e00 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc009990 tx=0x7fd0dc009ca0 comp rx=0 tx=0).stop 2024-04-23T10:24:47.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:47.921+0000 7fd0e7987700 1 -- 172.21.15.97:0/2294076729 shutdown_connections 2024-04-23T10:24:47.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:47.921+0000 7fd0e7987700 1 --2- 172.21.15.97:0/2294076729 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0e006b980 0x7fd0e010f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:47.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:47.921+0000 7fd0e7987700 1 --2- 172.21.15.97:0/2294076729 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0e0108a20 0x7fd0e0108e00 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:47.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:47.921+0000 7fd0e7987700 1 --2- 172.21.15.97:0/2294076729 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0e006afe0 0x7fd0e006b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:47.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:47.921+0000 7fd0e7987700 1 -- 172.21.15.97:0/2294076729 >> 172.21.15.97:0/2294076729 conn(0x7fd0e00788b0 msgr2=0x7fd0e0078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:47.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:47.921+0000 7fd0e7987700 1 -- 172.21.15.97:0/2294076729 shutdown_connections 2024-04-23T10:24:47.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:47.921+0000 7fd0e7987700 1 -- 172.21.15.97:0/2294076729 wait complete. 2024-04-23T10:24:47.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:47.921+0000 7fd0e7987700 1 Processor -- start 2024-04-23T10:24:47.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:47.921+0000 7fd0e7987700 1 -- start start 2024-04-23T10:24:47.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:47.921+0000 7fd0e7987700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0e006afe0 0x7fd0e0199090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:47.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:47.921+0000 7fd0e7987700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0e006b980 0x7fd0e01995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:47.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:47.921+0000 7fd0e7987700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0e0108a20 0x7fd0e019d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:47.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:47.921+0000 7fd0e7987700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e0112c20 con 0x7fd0e0108a20 2024-04-23T10:24:47.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:47.921+0000 7fd0e7987700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e0112aa0 con 0x7fd0e006b980 2024-04-23T10:24:47.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:47.921+0000 7fd0e7987700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e0112da0 con 0x7fd0e006afe0 2024-04-23T10:24:47.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:47.922+0000 7fd0e5f24700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0e0108a20 0x7fd0e019d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:47.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:47.922+0000 7fd0e5f24700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0e0108a20 0x7fd0e019d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58910/0 (socket says 172.21.15.97:58910) 2024-04-23T10:24:47.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:47.922+0000 7fd0e5f24700 1 -- 172.21.15.97:0/404370025 learned_addr learned my addr 172.21.15.97:0/404370025 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:24:47.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:47.922+0000 7fd0e5723700 1 --2- 172.21.15.97:0/404370025 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0e006afe0 0x7fd0e0199090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:47.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:47.922+0000 7fd0e4f22700 1 --2- 172.21.15.97:0/404370025 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0e006b980 0x7fd0e01995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:47.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:47.922+0000 7fd0e5f24700 1 -- 172.21.15.97:0/404370025 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0e006afe0 msgr2=0x7fd0e0199090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:47.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:47.922+0000 7fd0e5f24700 1 --2- 172.21.15.97:0/404370025 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0e006afe0 0x7fd0e0199090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:47.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:47.922+0000 7fd0e5f24700 1 -- 172.21.15.97:0/404370025 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0e006b980 msgr2=0x7fd0e01995d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:47.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:47.922+0000 7fd0e5f24700 1 --2- 172.21.15.97:0/404370025 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0e006b980 0x7fd0e01995d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:47.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:47.922+0000 7fd0e5f24700 1 -- 172.21.15.97:0/404370025 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0e019e0a0 con 0x7fd0e0108a20 2024-04-23T10:24:47.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:47.922+0000 7fd0e4f22700 1 --2- 172.21.15.97:0/404370025 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0e006b980 0x7fd0e01995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:24:47.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:47.922+0000 7fd0e5f24700 1 --2- 172.21.15.97:0/404370025 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0e0108a20 0x7fd0e019d960 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd0d800b970 tx=0x7fd0d800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:47.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:47.922+0000 7fd0d67fc700 1 -- 172.21.15.97:0/404370025 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d8019070 con 0x7fd0e0108a20 2024-04-23T10:24:47.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:47.922+0000 7fd0d67fc700 1 -- 172.21.15.97:0/404370025 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0d800cde0 con 0x7fd0e0108a20 2024-04-23T10:24:47.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:47.922+0000 7fd0e7987700 1 -- 172.21.15.97:0/404370025 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0e019e390 con 0x7fd0e0108a20 2024-04-23T10:24:47.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:47.922+0000 7fd0d67fc700 1 -- 172.21.15.97:0/404370025 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d8007370 con 0x7fd0e0108a20 2024-04-23T10:24:47.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:47.922+0000 7fd0e7987700 1 -- 172.21.15.97:0/404370025 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0e01a5c10 con 0x7fd0e0108a20 2024-04-23T10:24:47.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:47.923+0000 7fd0e7987700 1 -- 172.21.15.97:0/404370025 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd0e00631d0 con 0x7fd0e0108a20 2024-04-23T10:24:47.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:47.924+0000 7fd0d67fc700 1 -- 172.21.15.97:0/404370025 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd0d801c070 con 0x7fd0e0108a20 2024-04-23T10:24:47.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:47.925+0000 7fd0d67fc700 1 --2- 172.21.15.97:0/404370025 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fd0cc0a3cd0 0x7fd0cc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:47.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:47.925+0000 7fd0e5723700 1 --2- 172.21.15.97:0/404370025 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fd0cc0a3cd0 0x7fd0cc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:47.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:47.925+0000 7fd0d67fc700 1 -- 172.21.15.97:0/404370025 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fd0d80d01f0 con 0x7fd0e0108a20 2024-04-23T10:24:47.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:47.925+0000 7fd0d67fc700 1 --2- 172.21.15.97:0/404370025 >> [v2:172.21.15.183:6816/1865873033,v1:172.21.15.183:6817/1865873033] conn(0x7fd0cc0ad4b0 0x7fd0cc0af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:47.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:47.925+0000 7fd0d67fc700 1 -- 172.21.15.97:0/404370025 --> [v2:172.21.15.183:6816/1865873033,v1:172.21.15.183:6817/1865873033] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd0d8020e70 con 0x7fd0cc0ad4b0 2024-04-23T10:24:47.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:47.925+0000 7fd0d67fc700 1 -- 172.21.15.97:0/404370025 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fd0d80d06d0 con 0x7fd0e0108a20 2024-04-23T10:24:47.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:47.925+0000 7fd0e5723700 1 --2- 172.21.15.97:0/404370025 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fd0cc0a3cd0 0x7fd0cc0a6190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc0098e0 tx=0x7fd0dc005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:47.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:47.925+0000 7fd0e4f22700 1 --2- 172.21.15.97:0/404370025 >> [v2:172.21.15.183:6816/1865873033,v1:172.21.15.183:6817/1865873033] conn(0x7fd0cc0ad4b0 0x7fd0cc0af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:47.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:47.926+0000 7fd0e4f22700 1 --2- 172.21.15.97:0/404370025 >> [v2:172.21.15.183:6816/1865873033,v1:172.21.15.183:6817/1865873033] conn(0x7fd0cc0ad4b0 0x7fd0cc0af8d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:47.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:47.926+0000 7fd0d67fc700 1 -- 172.21.15.97:0/404370025 <== osd.7 v2:172.21.15.183:6816/1865873033 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd0d8020e70 con 0x7fd0cc0ad4b0 2024-04-23T10:24:47.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:47.943+0000 7fd0e7987700 1 -- 172.21.15.97:0/404370025 --> [v2:172.21.15.183:6816/1865873033,v1:172.21.15.183:6817/1865873033] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd0e0063650 con 0x7fd0cc0ad4b0 2024-04-23T10:24:47.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:47.943+0000 7fd0d67fc700 1 -- 172.21.15.97:0/404370025 <== osd.7 v2:172.21.15.183:6816/1865873033 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd0e0063650 con 0x7fd0cc0ad4b0 2024-04-23T10:24:47.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:47.944+0000 7fd0e7987700 1 -- 172.21.15.97:0/404370025 >> [v2:172.21.15.183:6816/1865873033,v1:172.21.15.183:6817/1865873033] conn(0x7fd0cc0ad4b0 msgr2=0x7fd0cc0af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:47.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:47.944+0000 7fd0e7987700 1 --2- 172.21.15.97:0/404370025 >> [v2:172.21.15.183:6816/1865873033,v1:172.21.15.183:6817/1865873033] conn(0x7fd0cc0ad4b0 0x7fd0cc0af8d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:47.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:47.944+0000 7fd0e7987700 1 -- 172.21.15.97:0/404370025 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fd0cc0a3cd0 msgr2=0x7fd0cc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:47.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:47.944+0000 7fd0e7987700 1 --2- 172.21.15.97:0/404370025 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fd0cc0a3cd0 0x7fd0cc0a6190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc0098e0 tx=0x7fd0dc005960 comp rx=0 tx=0).stop 2024-04-23T10:24:47.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:47.944+0000 7fd0e7987700 1 -- 172.21.15.97:0/404370025 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0e0108a20 msgr2=0x7fd0e019d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:47.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:47.944+0000 7fd0e7987700 1 --2- 172.21.15.97:0/404370025 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0e0108a20 0x7fd0e019d960 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd0d800b970 tx=0x7fd0d800bc80 comp rx=0 tx=0).stop 2024-04-23T10:24:47.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:47.944+0000 7fd0e5f24700 1 -- 172.21.15.97:0/404370025 reap_dead start 2024-04-23T10:24:47.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:47.944+0000 7fd0e7987700 1 -- 172.21.15.97:0/404370025 shutdown_connections 2024-04-23T10:24:47.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:47.944+0000 7fd0e7987700 1 -- 172.21.15.97:0/404370025 >> 172.21.15.97:0/404370025 conn(0x7fd0e00788b0 msgr2=0x7fd0e00ff9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:47.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:47.944+0000 7fd0e7987700 1 -- 172.21.15.97:0/404370025 shutdown_connections 2024-04-23T10:24:47.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:47.944+0000 7fd0e7987700 1 -- 172.21.15.97:0/404370025 wait complete. 2024-04-23T10:24:48.544 INFO:teuthology.orchestra.run.smithi097.stdout:347892350997 2024-04-23T10:24:48.545 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-23T10:24:48.862 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:48 smithi097 ceph-mon[29197]: pgmap v310: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-23T10:24:48.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:48 smithi183 ceph-mon[32317]: pgmap v310: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-23T10:24:49.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:48 smithi196 ceph-mon[32919]: pgmap v310: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-23T10:24:49.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:49.515+0000 7febb3a7b700 1 -- 172.21.15.97:0/859932814 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febac102a90 msgr2=0x7febac102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:49.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:49.515+0000 7febb3a7b700 1 --2- 172.21.15.97:0/859932814 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febac102a90 0x7febac102f10 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7feba8009990 tx=0x7feba8009ca0 comp rx=0 tx=0).stop 2024-04-23T10:24:49.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:49.515+0000 7febb3a7b700 1 -- 172.21.15.97:0/859932814 shutdown_connections 2024-04-23T10:24:49.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:49.515+0000 7febb3a7b700 1 --2- 172.21.15.97:0/859932814 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7febac103450 0x7febac10f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:49.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:49.515+0000 7febb3a7b700 1 --2- 172.21.15.97:0/859932814 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febac108a50 0x7febac108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:49.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:49.515+0000 7febb3a7b700 1 --2- 172.21.15.97:0/859932814 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febac102a90 0x7febac102f10 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:49.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:49.515+0000 7febb3a7b700 1 -- 172.21.15.97:0/859932814 >> 172.21.15.97:0/859932814 conn(0x7febac079ba0 msgr2=0x7febac079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:49.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:49.515+0000 7febb3a7b700 1 -- 172.21.15.97:0/859932814 shutdown_connections 2024-04-23T10:24:49.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:49.515+0000 7febb3a7b700 1 -- 172.21.15.97:0/859932814 wait complete. 2024-04-23T10:24:49.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:49.515+0000 7febb3a7b700 1 Processor -- start 2024-04-23T10:24:49.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:49.515+0000 7febb3a7b700 1 -- start start 2024-04-23T10:24:49.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:49.516+0000 7febb3a7b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febac102a90 0x7febac076930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:49.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:49.516+0000 7febb3a7b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febac103450 0x7febac074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:49.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:49.516+0000 7febb3a7b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7febac108a50 0x7febac0753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:49.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:49.516+0000 7febb3a7b700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7febac112be0 con 0x7febac103450 2024-04-23T10:24:49.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:49.516+0000 7febb3a7b700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7febac112a60 con 0x7febac102a90 2024-04-23T10:24:49.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:49.516+0000 7febb3a7b700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7febac112d60 con 0x7febac108a50 2024-04-23T10:24:49.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:49.516+0000 7febb1016700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febac103450 0x7febac074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:49.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:49.516+0000 7febb1016700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febac103450 0x7febac074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58920/0 (socket says 172.21.15.97:58920) 2024-04-23T10:24:49.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:49.516+0000 7febb1016700 1 -- 172.21.15.97:0/4037156777 learned_addr learned my addr 172.21.15.97:0/4037156777 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:24:49.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:49.516+0000 7febb1817700 1 --2- 172.21.15.97:0/4037156777 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febac102a90 0x7febac076930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:49.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:49.516+0000 7febb2018700 1 --2- 172.21.15.97:0/4037156777 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7febac108a50 0x7febac0753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:49.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:49.516+0000 7febb1016700 1 -- 172.21.15.97:0/4037156777 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7febac108a50 msgr2=0x7febac0753a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:49.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:49.516+0000 7febb1016700 1 --2- 172.21.15.97:0/4037156777 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7febac108a50 0x7febac0753a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:49.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:49.516+0000 7febb1016700 1 -- 172.21.15.97:0/4037156777 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febac102a90 msgr2=0x7febac076930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:49.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:49.516+0000 7febb1016700 1 --2- 172.21.15.97:0/4037156777 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febac102a90 0x7febac076930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:49.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:49.516+0000 7febb1016700 1 -- 172.21.15.97:0/4037156777 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febac1abb40 con 0x7febac103450 2024-04-23T10:24:49.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:49.516+0000 7febb1016700 1 --2- 172.21.15.97:0/4037156777 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febac103450 0x7febac074d40 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7feba8000c00 tx=0x7feba80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:49.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:49.516+0000 7feba2ffd700 1 -- 172.21.15.97:0/4037156777 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba8027070 con 0x7febac103450 2024-04-23T10:24:49.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:49.516+0000 7febb3a7b700 1 -- 172.21.15.97:0/4037156777 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febac1abd70 con 0x7febac103450 2024-04-23T10:24:49.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:49.516+0000 7feba2ffd700 1 -- 172.21.15.97:0/4037156777 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feba8003f20 con 0x7febac103450 2024-04-23T10:24:49.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:49.516+0000 7feba2ffd700 1 -- 172.21.15.97:0/4037156777 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba8022ca0 con 0x7febac103450 2024-04-23T10:24:49.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:49.517+0000 7febb3a7b700 1 -- 172.21.15.97:0/4037156777 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febac1ac1a0 con 0x7febac103450 2024-04-23T10:24:49.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:49.517+0000 7febb3a7b700 1 -- 172.21.15.97:0/4037156777 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7febac063110 con 0x7febac103450 2024-04-23T10:24:49.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:49.519+0000 7feba2ffd700 1 -- 172.21.15.97:0/4037156777 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7feba802a070 con 0x7febac103450 2024-04-23T10:24:49.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:49.519+0000 7feba2ffd700 1 --2- 172.21.15.97:0/4037156777 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7feb980a3cd0 0x7feb980a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:49.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:49.519+0000 7febb1817700 1 --2- 172.21.15.97:0/4037156777 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7feb980a3cd0 0x7feb980a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:49.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:49.519+0000 7feba2ffd700 1 -- 172.21.15.97:0/4037156777 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7feba80d1930 con 0x7febac103450 2024-04-23T10:24:49.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:49.520+0000 7feba2ffd700 1 --2- 172.21.15.97:0/4037156777 >> [v2:172.21.15.196:6816/4063932139,v1:172.21.15.196:6817/4063932139] conn(0x7feb980ad4b0 0x7feb980af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:49.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:49.520+0000 7feba2ffd700 1 -- 172.21.15.97:0/4037156777 --> [v2:172.21.15.196:6816/4063932139,v1:172.21.15.196:6817/4063932139] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7feba800ce90 con 0x7feb980ad4b0 2024-04-23T10:24:49.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:49.520+0000 7feba2ffd700 1 -- 172.21.15.97:0/4037156777 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7feba80d1e10 con 0x7febac103450 2024-04-23T10:24:49.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:49.520+0000 7febb1817700 1 --2- 172.21.15.97:0/4037156777 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7feb980a3cd0 0x7feb980a6190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7feb9c00abd0 tx=0x7feb9c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:49.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:49.520+0000 7febb2018700 1 --2- 172.21.15.97:0/4037156777 >> [v2:172.21.15.196:6816/4063932139,v1:172.21.15.196:6817/4063932139] conn(0x7feb980ad4b0 0x7feb980af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:49.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:49.520+0000 7febb2018700 1 --2- 172.21.15.97:0/4037156777 >> [v2:172.21.15.196:6816/4063932139,v1:172.21.15.196:6817/4063932139] conn(0x7feb980ad4b0 0x7feb980af8d0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:49.520 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:49.521+0000 7feba2ffd700 1 -- 172.21.15.97:0/4037156777 <== osd.8 v2:172.21.15.196:6816/4063932139 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7feba800ce90 con 0x7feb980ad4b0 2024-04-23T10:24:49.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:49.538+0000 7febb3a7b700 1 -- 172.21.15.97:0/4037156777 --> [v2:172.21.15.196:6816/4063932139,v1:172.21.15.196:6817/4063932139] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7febac063510 con 0x7feb980ad4b0 2024-04-23T10:24:49.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:49.539+0000 7feba2ffd700 1 -- 172.21.15.97:0/4037156777 <== osd.8 v2:172.21.15.196:6816/4063932139 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7febac063510 con 0x7feb980ad4b0 2024-04-23T10:24:49.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:49.539+0000 7febb3a7b700 1 -- 172.21.15.97:0/4037156777 >> [v2:172.21.15.196:6816/4063932139,v1:172.21.15.196:6817/4063932139] conn(0x7feb980ad4b0 msgr2=0x7feb980af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:49.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:49.539+0000 7febb3a7b700 1 --2- 172.21.15.97:0/4037156777 >> [v2:172.21.15.196:6816/4063932139,v1:172.21.15.196:6817/4063932139] conn(0x7feb980ad4b0 0x7feb980af8d0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:49.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:49.539+0000 7febb3a7b700 1 -- 172.21.15.97:0/4037156777 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7feb980a3cd0 msgr2=0x7feb980a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:49.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:49.539+0000 7febb3a7b700 1 --2- 172.21.15.97:0/4037156777 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7feb980a3cd0 0x7feb980a6190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7feb9c00abd0 tx=0x7feb9c00a380 comp rx=0 tx=0).stop 2024-04-23T10:24:49.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:49.539+0000 7febb3a7b700 1 -- 172.21.15.97:0/4037156777 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febac103450 msgr2=0x7febac074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:49.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:49.539+0000 7febb3a7b700 1 --2- 172.21.15.97:0/4037156777 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febac103450 0x7febac074d40 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7feba8000c00 tx=0x7feba80047b0 comp rx=0 tx=0).stop 2024-04-23T10:24:49.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:49.539+0000 7febb2018700 1 -- 172.21.15.97:0/4037156777 reap_dead start 2024-04-23T10:24:49.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:49.539+0000 7febb3a7b700 1 -- 172.21.15.97:0/4037156777 shutdown_connections 2024-04-23T10:24:49.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:49.539+0000 7febb3a7b700 1 -- 172.21.15.97:0/4037156777 >> 172.21.15.97:0/4037156777 conn(0x7febac079ba0 msgr2=0x7febac101720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:49.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:49.539+0000 7febb3a7b700 1 -- 172.21.15.97:0/4037156777 shutdown_connections 2024-04-23T10:24:49.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:49.539+0000 7febb3a7b700 1 -- 172.21.15.97:0/4037156777 wait complete. 2024-04-23T10:24:49.995 INFO:teuthology.orchestra.run.smithi097.stdout:292057776159 2024-04-23T10:24:49.996 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-23T10:24:50.663 INFO:teuthology.orchestra.run.smithi097.stdout:257698037798 2024-04-23T10:24:50.663 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-23T10:24:50.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:50 smithi183 ceph-mon[32317]: pgmap v311: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-23T10:24:51.044 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:50 smithi097 ceph-mon[29197]: pgmap v311: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-23T10:24:51.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:50 smithi196 ceph-mon[32919]: pgmap v311: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-23T10:24:52.132 INFO:teuthology.orchestra.run.smithi097.stdout:223338299435 2024-04-23T10:24:52.132 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-23T10:24:52.610 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T10:24:52.612 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T10:24:52.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.647+0000 7f685efc2700 1 -- 172.21.15.97:0/650742784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6858103480 msgr2=0x7f685810f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:52.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.647+0000 7f685efc2700 1 --2- 172.21.15.97:0/650742784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6858103480 0x7f685810f980 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f685000b0a0 tx=0x7f685000b3b0 comp rx=0 tx=0).stop 2024-04-23T10:24:52.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.648+0000 7f685efc2700 1 -- 172.21.15.97:0/650742784 shutdown_connections 2024-04-23T10:24:52.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.648+0000 7f685efc2700 1 --2- 172.21.15.97:0/650742784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6858103480 0x7f685810f980 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:52.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.648+0000 7f685efc2700 1 --2- 172.21.15.97:0/650742784 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6858108ab0 0x7f6858108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:52.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.648+0000 7f685efc2700 1 --2- 172.21.15.97:0/650742784 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6858102ac0 0x7f6858102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:52.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.648+0000 7f685efc2700 1 -- 172.21.15.97:0/650742784 >> 172.21.15.97:0/650742784 conn(0x7f6858079bf0 msgr2=0x7f685807a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:52.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.648+0000 7f685efc2700 1 -- 172.21.15.97:0/650742784 shutdown_connections 2024-04-23T10:24:52.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.648+0000 7f685efc2700 1 -- 172.21.15.97:0/650742784 wait complete. 2024-04-23T10:24:52.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.648+0000 7f685efc2700 1 Processor -- start 2024-04-23T10:24:52.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.648+0000 7f685efc2700 1 -- start start 2024-04-23T10:24:52.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.648+0000 7f685efc2700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6858102ac0 0x7f68581991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:52.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.649+0000 7f685efc2700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6858103480 0x7f68581996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:52.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.649+0000 7f685efc2700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6858108ab0 0x7f685819da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:52.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.649+0000 7f685efc2700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6858112cb0 con 0x7f6858103480 2024-04-23T10:24:52.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.649+0000 7f685efc2700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6858112b30 con 0x7f6858108ab0 2024-04-23T10:24:52.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.649+0000 7f685efc2700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6858112e30 con 0x7f6858102ac0 2024-04-23T10:24:52.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.649+0000 7f6857fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6858103480 0x7f68581996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:52.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.649+0000 7f6857fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6858103480 0x7f68581996e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58940/0 (socket says 172.21.15.97:58940) 2024-04-23T10:24:52.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.649+0000 7f6857fff700 1 -- 172.21.15.97:0/1366514887 learned_addr learned my addr 172.21.15.97:0/1366514887 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:24:52.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.649+0000 7f685cd5e700 1 --2- 172.21.15.97:0/1366514887 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6858102ac0 0x7f68581991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:52.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.649+0000 7f685d55f700 1 --2- 172.21.15.97:0/1366514887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6858108ab0 0x7f685819da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:52.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.649+0000 7f6857fff700 1 -- 172.21.15.97:0/1366514887 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6858102ac0 msgr2=0x7f68581991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:52.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.649+0000 7f6857fff700 1 --2- 172.21.15.97:0/1366514887 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6858102ac0 0x7f68581991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:52.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.649+0000 7f6857fff700 1 -- 172.21.15.97:0/1366514887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6858108ab0 msgr2=0x7f685819da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:52.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.649+0000 7f6857fff700 1 --2- 172.21.15.97:0/1366514887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6858108ab0 0x7f685819da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:52.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.649+0000 7f6857fff700 1 -- 172.21.15.97:0/1366514887 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f685819e1b0 con 0x7f6858103480 2024-04-23T10:24:52.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.649+0000 7f6857fff700 1 --2- 172.21.15.97:0/1366514887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6858103480 0x7f68581996e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f684400e9a0 tx=0x7f684400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:52.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.649+0000 7f6855ffb700 1 -- 172.21.15.97:0/1366514887 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f684400ca10 con 0x7f6858103480 2024-04-23T10:24:52.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.649+0000 7f685efc2700 1 -- 172.21.15.97:0/1366514887 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f685819e4a0 con 0x7f6858103480 2024-04-23T10:24:52.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.649+0000 7f6855ffb700 1 -- 172.21.15.97:0/1366514887 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f684400cbb0 con 0x7f6858103480 2024-04-23T10:24:52.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.649+0000 7f6855ffb700 1 -- 172.21.15.97:0/1366514887 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6844018680 con 0x7f6858103480 2024-04-23T10:24:52.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.650+0000 7f685efc2700 1 -- 172.21.15.97:0/1366514887 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6858074c70 con 0x7f6858103480 2024-04-23T10:24:52.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.650+0000 7f685efc2700 1 -- 172.21.15.97:0/1366514887 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6858063150 con 0x7f6858103480 2024-04-23T10:24:52.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.652+0000 7f6855ffb700 1 -- 172.21.15.97:0/1366514887 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6844010430 con 0x7f6858103480 2024-04-23T10:24:52.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.652+0000 7f6855ffb700 1 --2- 172.21.15.97:0/1366514887 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f68480a3cd0 0x7f68480a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:52.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.652+0000 7f685cd5e700 1 --2- 172.21.15.97:0/1366514887 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f68480a3cd0 0x7f68480a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:52.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.652+0000 7f6855ffb700 1 -- 172.21.15.97:0/1366514887 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f68440d0b40 con 0x7f6858103480 2024-04-23T10:24:52.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.652+0000 7f6855ffb700 1 --2- 172.21.15.97:0/1366514887 >> [v2:172.21.15.97:6818/2080916705,v1:172.21.15.97:6819/2080916705] conn(0x7f68480ad4b0 0x7f68480af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:52.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.652+0000 7f6855ffb700 1 -- 172.21.15.97:0/1366514887 --> [v2:172.21.15.97:6818/2080916705,v1:172.21.15.97:6819/2080916705] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f684400be50 con 0x7f68480ad4b0 2024-04-23T10:24:52.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.652+0000 7f6855ffb700 1 -- 172.21.15.97:0/1366514887 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f68440d1020 con 0x7f6858103480 2024-04-23T10:24:52.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.652+0000 7f685d55f700 1 --2- 172.21.15.97:0/1366514887 >> [v2:172.21.15.97:6818/2080916705,v1:172.21.15.97:6819/2080916705] conn(0x7f68480ad4b0 0x7f68480af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:52.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.653+0000 7f685cd5e700 1 --2- 172.21.15.97:0/1366514887 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f68480a3cd0 0x7f68480a6190 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f684c004490 tx=0x7f684c014040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:52.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.653+0000 7f685d55f700 1 --2- 172.21.15.97:0/1366514887 >> [v2:172.21.15.97:6818/2080916705,v1:172.21.15.97:6819/2080916705] conn(0x7f68480ad4b0 0x7f68480af8d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:52.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.654+0000 7f6855ffb700 1 -- 172.21.15.97:0/1366514887 <== osd.6 v2:172.21.15.97:6818/2080916705 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f684400be50 con 0x7f68480ad4b0 2024-04-23T10:24:52.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.670+0000 7f685efc2700 1 -- 172.21.15.97:0/1366514887 --> [v2:172.21.15.97:6818/2080916705,v1:172.21.15.97:6819/2080916705] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f68580633a0 con 0x7f68480ad4b0 2024-04-23T10:24:52.669 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:52 smithi097 ceph-mon[29197]: pgmap v312: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-23T10:24:52.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.670+0000 7f6855ffb700 1 -- 172.21.15.97:0/1366514887 <== osd.6 v2:172.21.15.97:6818/2080916705 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f68580633a0 con 0x7f68480ad4b0 2024-04-23T10:24:52.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.671+0000 7f685efc2700 1 -- 172.21.15.97:0/1366514887 >> [v2:172.21.15.97:6818/2080916705,v1:172.21.15.97:6819/2080916705] conn(0x7f68480ad4b0 msgr2=0x7f68480af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:52.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.671+0000 7f685efc2700 1 --2- 172.21.15.97:0/1366514887 >> [v2:172.21.15.97:6818/2080916705,v1:172.21.15.97:6819/2080916705] conn(0x7f68480ad4b0 0x7f68480af8d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:52.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.671+0000 7f685efc2700 1 -- 172.21.15.97:0/1366514887 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f68480a3cd0 msgr2=0x7f68480a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:52.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.671+0000 7f685efc2700 1 --2- 172.21.15.97:0/1366514887 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f68480a3cd0 0x7f68480a6190 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f684c004490 tx=0x7f684c014040 comp rx=0 tx=0).stop 2024-04-23T10:24:52.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.671+0000 7f685efc2700 1 -- 172.21.15.97:0/1366514887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6858103480 msgr2=0x7f68581996e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:52.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.671+0000 7f685efc2700 1 --2- 172.21.15.97:0/1366514887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6858103480 0x7f68581996e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f684400e9a0 tx=0x7f684400ecb0 comp rx=0 tx=0).stop 2024-04-23T10:24:52.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.671+0000 7f685d55f700 1 -- 172.21.15.97:0/1366514887 reap_dead start 2024-04-23T10:24:52.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.671+0000 7f685efc2700 1 -- 172.21.15.97:0/1366514887 shutdown_connections 2024-04-23T10:24:52.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.671+0000 7f685efc2700 1 -- 172.21.15.97:0/1366514887 >> 172.21.15.97:0/1366514887 conn(0x7f6858079bf0 msgr2=0x7f6858100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:52.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.671+0000 7f685efc2700 1 -- 172.21.15.97:0/1366514887 shutdown_connections 2024-04-23T10:24:52.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.671+0000 7f685efc2700 1 -- 172.21.15.97:0/1366514887 wait complete. 2024-04-23T10:24:52.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.833+0000 7f0a156b5700 1 -- 172.21.15.97:0/571668255 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a10103450 msgr2=0x7f0a1010f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:52.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.833+0000 7f0a156b5700 1 --2- 172.21.15.97:0/571668255 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a10103450 0x7f0a1010f8b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0a0800b0a0 tx=0x7f0a0800b3b0 comp rx=0 tx=0).stop 2024-04-23T10:24:52.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.833+0000 7f0a156b5700 1 -- 172.21.15.97:0/571668255 shutdown_connections 2024-04-23T10:24:52.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.833+0000 7f0a156b5700 1 --2- 172.21.15.97:0/571668255 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a10103450 0x7f0a1010f8b0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:52.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.833+0000 7f0a156b5700 1 --2- 172.21.15.97:0/571668255 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a10108a50 0x7f0a10108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:52.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.833+0000 7f0a156b5700 1 --2- 172.21.15.97:0/571668255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a10102a90 0x7f0a10102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:52.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.833+0000 7f0a156b5700 1 -- 172.21.15.97:0/571668255 >> 172.21.15.97:0/571668255 conn(0x7f0a10079ba0 msgr2=0x7f0a10079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:52.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.833+0000 7f0a156b5700 1 -- 172.21.15.97:0/571668255 shutdown_connections 2024-04-23T10:24:52.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.833+0000 7f0a156b5700 1 -- 172.21.15.97:0/571668255 wait complete. 2024-04-23T10:24:52.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.833+0000 7f0a156b5700 1 Processor -- start 2024-04-23T10:24:52.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.834+0000 7f0a156b5700 1 -- start start 2024-04-23T10:24:52.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.834+0000 7f0a156b5700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a10102a90 0x7f0a1019b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:52.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.834+0000 7f0a156b5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a10103450 0x7f0a1019b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:52.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.834+0000 7f0a156b5700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a10108a50 0x7f0a1019fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:52.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.834+0000 7f0a156b5700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0a10112be0 con 0x7f0a10103450 2024-04-23T10:24:52.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.834+0000 7f0a156b5700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0a10112a60 con 0x7f0a10102a90 2024-04-23T10:24:52.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.834+0000 7f0a156b5700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0a10112d60 con 0x7f0a10108a50 2024-04-23T10:24:52.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.834+0000 7f0a07fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a10103450 0x7f0a1019b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:52.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.834+0000 7f0a07fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a10103450 0x7f0a1019b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58958/0 (socket says 172.21.15.97:58958) 2024-04-23T10:24:52.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.834+0000 7f0a07fff700 1 -- 172.21.15.97:0/970435399 learned_addr learned my addr 172.21.15.97:0/970435399 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:24:52.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.834+0000 7f0a0effd700 1 --2- 172.21.15.97:0/970435399 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a10102a90 0x7f0a1019b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:52.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.834+0000 7f0a0f7fe700 1 --2- 172.21.15.97:0/970435399 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a10108a50 0x7f0a1019fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:52.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.834+0000 7f0a07fff700 1 -- 172.21.15.97:0/970435399 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a10108a50 msgr2=0x7f0a1019fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:52.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.834+0000 7f0a07fff700 1 --2- 172.21.15.97:0/970435399 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a10108a50 0x7f0a1019fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:52.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.834+0000 7f0a07fff700 1 -- 172.21.15.97:0/970435399 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a10102a90 msgr2=0x7f0a1019b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:52.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.834+0000 7f0a07fff700 1 --2- 172.21.15.97:0/970435399 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a10102a90 0x7f0a1019b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:52.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.834+0000 7f0a07fff700 1 -- 172.21.15.97:0/970435399 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a101a02a0 con 0x7f0a10103450 2024-04-23T10:24:52.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.834+0000 7f0a0effd700 1 --2- 172.21.15.97:0/970435399 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a10102a90 0x7f0a1019b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:24:52.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.834+0000 7f0a0f7fe700 1 --2- 172.21.15.97:0/970435399 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a10108a50 0x7f0a1019fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:24:52.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.834+0000 7f0a07fff700 1 --2- 172.21.15.97:0/970435399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a10103450 0x7f0a1019b7d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f09f800d980 tx=0x7f09f800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:52.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.834+0000 7f0a0cff9700 1 -- 172.21.15.97:0/970435399 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f8016070 con 0x7f0a10103450 2024-04-23T10:24:52.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.834+0000 7f0a0cff9700 1 -- 172.21.15.97:0/970435399 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09f80044d0 con 0x7f0a10103450 2024-04-23T10:24:52.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.834+0000 7f0a156b5700 1 -- 172.21.15.97:0/970435399 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a101a0590 con 0x7f0a10103450 2024-04-23T10:24:52.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.834+0000 7f0a156b5700 1 -- 172.21.15.97:0/970435399 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a101a7e10 con 0x7f0a10103450 2024-04-23T10:24:52.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.834+0000 7f0a0cff9700 1 -- 172.21.15.97:0/970435399 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f8004d00 con 0x7f0a10103450 2024-04-23T10:24:52.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.835+0000 7f0a156b5700 1 -- 172.21.15.97:0/970435399 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0a1010f930 con 0x7f0a10103450 2024-04-23T10:24:52.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.837+0000 7f0a0cff9700 1 -- 172.21.15.97:0/970435399 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f09f8004020 con 0x7f0a10103450 2024-04-23T10:24:52.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.837+0000 7f0a0cff9700 1 --2- 172.21.15.97:0/970435399 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f09fc0a3cd0 0x7f09fc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:52.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.837+0000 7f0a0effd700 1 --2- 172.21.15.97:0/970435399 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f09fc0a3cd0 0x7f09fc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:52.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.837+0000 7f0a0cff9700 1 -- 172.21.15.97:0/970435399 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f09f80d0990 con 0x7f0a10103450 2024-04-23T10:24:52.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.837+0000 7f0a0cff9700 1 --2- 172.21.15.97:0/970435399 >> [v2:172.21.15.97:6810/4276311194,v1:172.21.15.97:6811/4276311194] conn(0x7f09fc0ad4b0 0x7f09fc0af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:52.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.837+0000 7f0a0cff9700 1 -- 172.21.15.97:0/970435399 --> [v2:172.21.15.97:6810/4276311194,v1:172.21.15.97:6811/4276311194] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f09f8015ec0 con 0x7f09fc0ad4b0 2024-04-23T10:24:52.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.837+0000 7f0a0cff9700 1 -- 172.21.15.97:0/970435399 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f09f80d0e70 con 0x7f0a10103450 2024-04-23T10:24:52.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.837+0000 7f0a0effd700 1 --2- 172.21.15.97:0/970435399 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f09fc0a3cd0 0x7f09fc0a6190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0a00005fd0 tx=0x7f0a00005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:52.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.838+0000 7f0a0f7fe700 1 --2- 172.21.15.97:0/970435399 >> [v2:172.21.15.97:6810/4276311194,v1:172.21.15.97:6811/4276311194] conn(0x7f09fc0ad4b0 0x7f09fc0af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:52.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.838+0000 7f0a0f7fe700 1 --2- 172.21.15.97:0/970435399 >> [v2:172.21.15.97:6810/4276311194,v1:172.21.15.97:6811/4276311194] conn(0x7f09fc0ad4b0 0x7f09fc0af8d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:52.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.839+0000 7f0a0cff9700 1 -- 172.21.15.97:0/970435399 <== osd.3 v2:172.21.15.97:6810/4276311194 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f09f8015ec0 con 0x7f09fc0ad4b0 2024-04-23T10:24:52.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.854+0000 7f0a156b5700 1 -- 172.21.15.97:0/970435399 --> [v2:172.21.15.97:6810/4276311194,v1:172.21.15.97:6811/4276311194] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0a10104180 con 0x7f09fc0ad4b0 2024-04-23T10:24:52.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.855+0000 7f0a0cff9700 1 -- 172.21.15.97:0/970435399 <== osd.3 v2:172.21.15.97:6810/4276311194 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0a10104180 con 0x7f09fc0ad4b0 2024-04-23T10:24:52.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.855+0000 7f0a156b5700 1 -- 172.21.15.97:0/970435399 >> [v2:172.21.15.97:6810/4276311194,v1:172.21.15.97:6811/4276311194] conn(0x7f09fc0ad4b0 msgr2=0x7f09fc0af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:52.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.855+0000 7f0a156b5700 1 --2- 172.21.15.97:0/970435399 >> [v2:172.21.15.97:6810/4276311194,v1:172.21.15.97:6811/4276311194] conn(0x7f09fc0ad4b0 0x7f09fc0af8d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:52.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.855+0000 7f0a156b5700 1 -- 172.21.15.97:0/970435399 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f09fc0a3cd0 msgr2=0x7f09fc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:52.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.855+0000 7f0a156b5700 1 --2- 172.21.15.97:0/970435399 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f09fc0a3cd0 0x7f09fc0a6190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0a00005fd0 tx=0x7f0a00005db0 comp rx=0 tx=0).stop 2024-04-23T10:24:52.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.855+0000 7f0a156b5700 1 -- 172.21.15.97:0/970435399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a10103450 msgr2=0x7f0a1019b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:52.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.855+0000 7f0a156b5700 1 --2- 172.21.15.97:0/970435399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0a10103450 0x7f0a1019b7d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f09f800d980 tx=0x7f09f800dd40 comp rx=0 tx=0).stop 2024-04-23T10:24:52.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.855+0000 7f0a0f7fe700 1 -- 172.21.15.97:0/970435399 reap_dead start 2024-04-23T10:24:52.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.855+0000 7f0a156b5700 1 -- 172.21.15.97:0/970435399 shutdown_connections 2024-04-23T10:24:52.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.855+0000 7f0a156b5700 1 -- 172.21.15.97:0/970435399 >> 172.21.15.97:0/970435399 conn(0x7f0a10079ba0 msgr2=0x7f0a1010df00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:52.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.855+0000 7f0a156b5700 1 -- 172.21.15.97:0/970435399 shutdown_connections 2024-04-23T10:24:52.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:52.855+0000 7f0a156b5700 1 -- 172.21.15.97:0/970435399 wait complete. 2024-04-23T10:24:52.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:52 smithi183 ceph-mon[32317]: pgmap v312: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-23T10:24:53.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:52 smithi196 ceph-mon[32919]: pgmap v312: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-23T10:24:53.746 INFO:teuthology.orchestra.run.smithi097.stdout:111669149760 2024-04-23T10:24:53.747 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-23T10:24:54.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:54.234+0000 7f31af955700 1 -- 172.21.15.97:0/311320710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31a806a750 msgr2=0x7f31a806abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:54.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:54.234+0000 7f31af955700 1 --2- 172.21.15.97:0/311320710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31a806a750 0x7f31a806abb0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3198009990 tx=0x7f3198009ca0 comp rx=0 tx=0).stop 2024-04-23T10:24:54.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:54.235+0000 7f31af955700 1 -- 172.21.15.97:0/311320710 shutdown_connections 2024-04-23T10:24:54.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:54.235+0000 7f31af955700 1 --2- 172.21.15.97:0/311320710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f31a806b0f0 0x7f31a810f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:54.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:54.235+0000 7f31af955700 1 --2- 172.21.15.97:0/311320710 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f31a8108a30 0x7f31a8108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:54.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:54.235+0000 7f31af955700 1 --2- 172.21.15.97:0/311320710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31a806a750 0x7f31a806abb0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:54.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:54.235+0000 7f31af955700 1 -- 172.21.15.97:0/311320710 >> 172.21.15.97:0/311320710 conn(0x7f31a80788f0 msgr2=0x7f31a8078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:54.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:54.235+0000 7f31af955700 1 -- 172.21.15.97:0/311320710 shutdown_connections 2024-04-23T10:24:54.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:54.235+0000 7f31af955700 1 -- 172.21.15.97:0/311320710 wait complete. 2024-04-23T10:24:54.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:54.235+0000 7f31af955700 1 Processor -- start 2024-04-23T10:24:54.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:54.235+0000 7f31af955700 1 -- start start 2024-04-23T10:24:54.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:54.236+0000 7f31af955700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31a806a750 0x7f31a8101a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:54.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:54.236+0000 7f31af955700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f31a806b0f0 0x7f31a8101f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:54.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:54.236+0000 7f31af955700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f31a8108a30 0x7f31a8102490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:54.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:54.236+0000 7f31af955700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f31a8112bd0 con 0x7f31a806a750 2024-04-23T10:24:54.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:54.236+0000 7f31af955700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f31a8112a50 con 0x7f31a806b0f0 2024-04-23T10:24:54.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:54.236+0000 7f31af955700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f31a8112d50 con 0x7f31a8108a30 2024-04-23T10:24:54.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:54.236+0000 7f31ad6f1700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31a806a750 0x7f31a8101a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:54.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:54.236+0000 7f31ad6f1700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31a806a750 0x7f31a8101a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58982/0 (socket says 172.21.15.97:58982) 2024-04-23T10:24:54.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:54.236+0000 7f31ad6f1700 1 -- 172.21.15.97:0/3355802510 learned_addr learned my addr 172.21.15.97:0/3355802510 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:24:54.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:54.236+0000 7f31acef0700 1 --2- 172.21.15.97:0/3355802510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f31a806b0f0 0x7f31a8101f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:54.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:54.236+0000 7f31adef2700 1 --2- 172.21.15.97:0/3355802510 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f31a8108a30 0x7f31a8102490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:54.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:54.236+0000 7f31ad6f1700 1 -- 172.21.15.97:0/3355802510 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f31a8108a30 msgr2=0x7f31a8102490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:54.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:54.236+0000 7f31ad6f1700 1 --2- 172.21.15.97:0/3355802510 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f31a8108a30 0x7f31a8102490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:54.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:54.236+0000 7f31ad6f1700 1 -- 172.21.15.97:0/3355802510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f31a806b0f0 msgr2=0x7f31a8101f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:54.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:54.236+0000 7f31ad6f1700 1 --2- 172.21.15.97:0/3355802510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f31a806b0f0 0x7f31a8101f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:54.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:54.236+0000 7f31ad6f1700 1 -- 172.21.15.97:0/3355802510 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31a81a7650 con 0x7f31a806a750 2024-04-23T10:24:54.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:54.236+0000 7f31adef2700 1 --2- 172.21.15.97:0/3355802510 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f31a8108a30 0x7f31a8102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:24:54.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:54.236+0000 7f31ad6f1700 1 --2- 172.21.15.97:0/3355802510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31a806a750 0x7f31a8101a10 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f31a400c920 tx=0x7f31a400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:54.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:54.236+0000 7f319e7fc700 1 -- 172.21.15.97:0/3355802510 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a4007b00 con 0x7f31a806a750 2024-04-23T10:24:54.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:54.236+0000 7f31af955700 1 -- 172.21.15.97:0/3355802510 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31a81a7940 con 0x7f31a806a750 2024-04-23T10:24:54.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:54.236+0000 7f319e7fc700 1 -- 172.21.15.97:0/3355802510 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31a4004ce0 con 0x7f31a806a750 2024-04-23T10:24:54.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:54.236+0000 7f319e7fc700 1 -- 172.21.15.97:0/3355802510 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a4005500 con 0x7f31a806a750 2024-04-23T10:24:54.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:54.236+0000 7f31af955700 1 -- 172.21.15.97:0/3355802510 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31a81a7ea0 con 0x7f31a806a750 2024-04-23T10:24:54.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:54.237+0000 7f31af955700 1 -- 172.21.15.97:0/3355802510 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f31a80630d0 con 0x7f31a806a750 2024-04-23T10:24:54.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:54.238+0000 7f319e7fc700 1 -- 172.21.15.97:0/3355802510 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f31a4007ca0 con 0x7f31a806a750 2024-04-23T10:24:54.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:54.239+0000 7f319e7fc700 1 --2- 172.21.15.97:0/3355802510 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f31940a3cd0 0x7f31940a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:54.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:54.239+0000 7f31acef0700 1 --2- 172.21.15.97:0/3355802510 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f31940a3cd0 0x7f31940a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:54.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:54.239+0000 7f319e7fc700 1 -- 172.21.15.97:0/3355802510 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f31a40d02d0 con 0x7f31a806a750 2024-04-23T10:24:54.273 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:54.239+0000 7f319e7fc700 1 --2- 172.21.15.97:0/3355802510 >> [v2:172.21.15.183:6808/63913916,v1:172.21.15.183:6809/63913916] conn(0x7f31940ad4b0 0x7f31940af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:54.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:54.239+0000 7f319e7fc700 1 -- 172.21.15.97:0/3355802510 --> [v2:172.21.15.183:6808/63913916,v1:172.21.15.183:6809/63913916] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f31a4012ec0 con 0x7f31940ad4b0 2024-04-23T10:24:54.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:54.239+0000 7f319e7fc700 1 -- 172.21.15.97:0/3355802510 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f31a40d07b0 con 0x7f31a806a750 2024-04-23T10:24:54.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:54.239+0000 7f31acef0700 1 --2- 172.21.15.97:0/3355802510 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f31940a3cd0 0x7f31940a6190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f319800b5c0 tx=0x7f3198005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:54.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:54.239+0000 7f31adef2700 1 --2- 172.21.15.97:0/3355802510 >> [v2:172.21.15.183:6808/63913916,v1:172.21.15.183:6809/63913916] conn(0x7f31940ad4b0 0x7f31940af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:54.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:54.240+0000 7f31adef2700 1 --2- 172.21.15.97:0/3355802510 >> [v2:172.21.15.183:6808/63913916,v1:172.21.15.183:6809/63913916] conn(0x7f31940ad4b0 0x7f31940af8d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:54.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:54.241+0000 7f319e7fc700 1 -- 172.21.15.97:0/3355802510 <== osd.4 v2:172.21.15.183:6808/63913916 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f31a4012ec0 con 0x7f31940ad4b0 2024-04-23T10:24:54.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:54.257+0000 7f31af955700 1 -- 172.21.15.97:0/3355802510 --> [v2:172.21.15.183:6808/63913916,v1:172.21.15.183:6809/63913916] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f31a8063550 con 0x7f31940ad4b0 2024-04-23T10:24:54.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:54.257+0000 7f319e7fc700 1 -- 172.21.15.97:0/3355802510 <== osd.4 v2:172.21.15.183:6808/63913916 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f31a8063550 con 0x7f31940ad4b0 2024-04-23T10:24:54.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:54.257+0000 7f31af955700 1 -- 172.21.15.97:0/3355802510 >> [v2:172.21.15.183:6808/63913916,v1:172.21.15.183:6809/63913916] conn(0x7f31940ad4b0 msgr2=0x7f31940af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:54.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:54.257+0000 7f31af955700 1 --2- 172.21.15.97:0/3355802510 >> [v2:172.21.15.183:6808/63913916,v1:172.21.15.183:6809/63913916] conn(0x7f31940ad4b0 0x7f31940af8d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:54.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:54.258+0000 7f31af955700 1 -- 172.21.15.97:0/3355802510 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f31940a3cd0 msgr2=0x7f31940a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:54.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:54.258+0000 7f31af955700 1 --2- 172.21.15.97:0/3355802510 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f31940a3cd0 0x7f31940a6190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f319800b5c0 tx=0x7f3198005960 comp rx=0 tx=0).stop 2024-04-23T10:24:54.274 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:54.258+0000 7f31af955700 1 -- 172.21.15.97:0/3355802510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31a806a750 msgr2=0x7f31a8101a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:54.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:54.258+0000 7f31af955700 1 --2- 172.21.15.97:0/3355802510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31a806a750 0x7f31a8101a10 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f31a400c920 tx=0x7f31a400cc30 comp rx=0 tx=0).stop 2024-04-23T10:24:54.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:54.258+0000 7f31adef2700 1 -- 172.21.15.97:0/3355802510 reap_dead start 2024-04-23T10:24:54.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:54.258+0000 7f31af955700 1 -- 172.21.15.97:0/3355802510 shutdown_connections 2024-04-23T10:24:54.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:54.258+0000 7f31af955700 1 -- 172.21.15.97:0/3355802510 >> 172.21.15.97:0/3355802510 conn(0x7f31a80788f0 msgr2=0x7f31a80ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:54.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:54.258+0000 7f31af955700 1 -- 172.21.15.97:0/3355802510 shutdown_connections 2024-04-23T10:24:54.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:54.258+0000 7f31af955700 1 -- 172.21.15.97:0/3355802510 wait complete. 2024-04-23T10:24:54.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:54 smithi183 ceph-mon[32317]: pgmap v313: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 8 op/s 2024-04-23T10:24:54.937 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:54 smithi097 ceph-mon[29197]: pgmap v313: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 8 op/s 2024-04-23T10:24:54.966 INFO:teuthology.orchestra.run.smithi097.stdout:146028888123 2024-04-23T10:24:54.966 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-23T10:24:55.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:54 smithi196 ceph-mon[32919]: pgmap v313: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 8 op/s 2024-04-23T10:24:56.325 INFO:teuthology.orchestra.run.smithi097.stdout:201863462961 2024-04-23T10:24:56.325 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-23T10:24:56.748 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T10:24:56.749 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T10:24:56.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:56 smithi183 ceph-mon[32317]: pgmap v314: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-04-23T10:24:57.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:56 smithi196 ceph-mon[32919]: pgmap v314: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-04-23T10:24:57.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:56 smithi097 ceph-mon[29197]: pgmap v314: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-04-23T10:24:57.183 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T10:24:57.184 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T10:24:58.797 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:58 smithi097 ceph-mon[29197]: pgmap v315: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-04-23T10:24:59.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:58 smithi196 ceph-mon[32919]: pgmap v315: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-04-23T10:24:59.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:59.126+0000 7fa77cd86700 1 -- 172.21.15.97:0/1552056296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa77806b980 msgr2=0x7fa77810f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:59.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:59.126+0000 7fa77cd86700 1 --2- 172.21.15.97:0/1552056296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa77806b980 0x7fa77810f8f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa77000b0a0 tx=0x7fa77000b3b0 comp rx=0 tx=0).stop 2024-04-23T10:24:59.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:59.126+0000 7fa77cd86700 1 -- 172.21.15.97:0/1552056296 shutdown_connections 2024-04-23T10:24:59.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:59.126+0000 7fa77cd86700 1 --2- 172.21.15.97:0/1552056296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa77806b980 0x7fa77810f8f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:59.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:59.126+0000 7fa77cd86700 1 --2- 172.21.15.97:0/1552056296 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa778108a20 0x7fa778108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:59.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:59.126+0000 7fa77cd86700 1 --2- 172.21.15.97:0/1552056296 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa77806afe0 0x7fa77806b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:59.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:59.126+0000 7fa77cd86700 1 -- 172.21.15.97:0/1552056296 >> 172.21.15.97:0/1552056296 conn(0x7fa7780788b0 msgr2=0x7fa778078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:59.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:59.126+0000 7fa77cd86700 1 -- 172.21.15.97:0/1552056296 shutdown_connections 2024-04-23T10:24:59.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:59.126+0000 7fa77cd86700 1 -- 172.21.15.97:0/1552056296 wait complete. 2024-04-23T10:24:59.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:59.127+0000 7fa77cd86700 1 Processor -- start 2024-04-23T10:24:59.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:59.127+0000 7fa77cd86700 1 -- start start 2024-04-23T10:24:59.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:59.127+0000 7fa77cd86700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa77806afe0 0x7fa7781990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:59.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:59.127+0000 7fa77cd86700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa77806b980 0x7fa778199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:59.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:59.127+0000 7fa77cd86700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa778108a20 0x7fa77819d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:59.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:59.127+0000 7fa77cd86700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa778112bc0 con 0x7fa77806afe0 2024-04-23T10:24:59.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:59.127+0000 7fa77cd86700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa778112a40 con 0x7fa77806b980 2024-04-23T10:24:59.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:59.127+0000 7fa77cd86700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa778112d40 con 0x7fa778108a20 2024-04-23T10:24:59.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:59.127+0000 7fa77659c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa77806afe0 0x7fa7781990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:59.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:59.127+0000 7fa77659c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa77806afe0 0x7fa7781990f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46652/0 (socket says 172.21.15.97:46652) 2024-04-23T10:24:59.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:59.127+0000 7fa77659c700 1 -- 172.21.15.97:0/2202105659 learned_addr learned my addr 172.21.15.97:0/2202105659 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:24:59.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:59.127+0000 7fa776d9d700 1 --2- 172.21.15.97:0/2202105659 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa778108a20 0x7fa77819d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:59.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:59.127+0000 7fa77659c700 1 -- 172.21.15.97:0/2202105659 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa778108a20 msgr2=0x7fa77819d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:59.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:59.127+0000 7fa77659c700 1 --2- 172.21.15.97:0/2202105659 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa778108a20 0x7fa77819d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:59.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:59.127+0000 7fa77659c700 1 -- 172.21.15.97:0/2202105659 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa77806b980 msgr2=0x7fa778199630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:59.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:59.127+0000 7fa76dd9b700 1 --2- 172.21.15.97:0/2202105659 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa77806b980 0x7fa778199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:59.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:59.127+0000 7fa77659c700 1 --2- 172.21.15.97:0/2202105659 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa77806b980 0x7fa778199630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:59.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:59.127+0000 7fa77659c700 1 -- 172.21.15.97:0/2202105659 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa77819e0b0 con 0x7fa77806afe0 2024-04-23T10:24:59.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:59.128+0000 7fa77659c700 1 --2- 172.21.15.97:0/2202105659 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa77806afe0 0x7fa7781990f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa76800d980 tx=0x7fa76800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:59.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:59.128+0000 7fa76ffff700 1 -- 172.21.15.97:0/2202105659 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa768017070 con 0x7fa77806afe0 2024-04-23T10:24:59.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:59.128+0000 7fa77cd86700 1 -- 172.21.15.97:0/2202105659 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa77819e3a0 con 0x7fa77806afe0 2024-04-23T10:24:59.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:59.128+0000 7fa76ffff700 1 -- 172.21.15.97:0/2202105659 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa768005020 con 0x7fa77806afe0 2024-04-23T10:24:59.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:59.128+0000 7fa76ffff700 1 -- 172.21.15.97:0/2202105659 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa768009b50 con 0x7fa77806afe0 2024-04-23T10:24:59.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:59.128+0000 7fa77cd86700 1 -- 172.21.15.97:0/2202105659 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7781a5c20 con 0x7fa77806afe0 2024-04-23T10:24:59.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:59.129+0000 7fa77cd86700 1 -- 172.21.15.97:0/2202105659 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa778069730 con 0x7fa77806afe0 2024-04-23T10:24:59.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:59.130+0000 7fa76ffff700 1 -- 172.21.15.97:0/2202105659 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa7680051d0 con 0x7fa77806afe0 2024-04-23T10:24:59.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:59.130+0000 7fa76ffff700 1 --2- 172.21.15.97:0/2202105659 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fa7640a3cd0 0x7fa7640a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:24:59.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:59.131+0000 7fa76dd9b700 1 --2- 172.21.15.97:0/2202105659 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fa7640a3cd0 0x7fa7640a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:24:59.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:59.131+0000 7fa76ffff700 1 -- 172.21.15.97:0/2202105659 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fa7680d06b0 con 0x7fa77806afe0 2024-04-23T10:24:59.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:59.131+0000 7fa76dd9b700 1 --2- 172.21.15.97:0/2202105659 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fa7640a3cd0 0x7fa7640a6190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa760005950 tx=0x7fa7600058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:24:59.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:59.134+0000 7fa76ffff700 1 -- 172.21.15.97:0/2202105659 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa768020020 con 0x7fa77806afe0 2024-04-23T10:24:59.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:58 smithi183 ceph-mon[32317]: pgmap v315: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-04-23T10:24:59.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:59.336+0000 7fa77cd86700 1 -- 172.21.15.97:0/2202105659 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fa77802df80 con 0x7fa77806afe0 2024-04-23T10:24:59.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:59.336+0000 7fa76ffff700 1 -- 172.21.15.97:0/2202105659 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fa76809d060 con 0x7fa77806afe0 2024-04-23T10:24:59.336 INFO:teuthology.orchestra.run.smithi097.stdout:313532612637 2024-04-23T10:24:59.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:59.339+0000 7fa77cd86700 1 -- 172.21.15.97:0/2202105659 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fa7640a3cd0 msgr2=0x7fa7640a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:59.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:59.339+0000 7fa77cd86700 1 --2- 172.21.15.97:0/2202105659 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fa7640a3cd0 0x7fa7640a6190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa760005950 tx=0x7fa7600058e0 comp rx=0 tx=0).stop 2024-04-23T10:24:59.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:59.339+0000 7fa77cd86700 1 -- 172.21.15.97:0/2202105659 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa77806afe0 msgr2=0x7fa7781990f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:24:59.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:59.339+0000 7fa77cd86700 1 --2- 172.21.15.97:0/2202105659 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa77806afe0 0x7fa7781990f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa76800d980 tx=0x7fa76800dc90 comp rx=0 tx=0).stop 2024-04-23T10:24:59.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:59.339+0000 7fa77cd86700 1 -- 172.21.15.97:0/2202105659 shutdown_connections 2024-04-23T10:24:59.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:59.339+0000 7fa77cd86700 1 --2- 172.21.15.97:0/2202105659 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fa7640a3cd0 0x7fa7640a6190 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:59.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:59.339+0000 7fa77cd86700 1 --2- 172.21.15.97:0/2202105659 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa778108a20 0x7fa77819d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:59.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:59.339+0000 7fa77cd86700 1 --2- 172.21.15.97:0/2202105659 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa77806afe0 0x7fa7781990f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:59.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:59.339+0000 7fa77cd86700 1 --2- 172.21.15.97:0/2202105659 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa77806b980 0x7fa778199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:24:59.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:59.339+0000 7fa77cd86700 1 -- 172.21.15.97:0/2202105659 >> 172.21.15.97:0/2202105659 conn(0x7fa7780788b0 msgr2=0x7fa7780ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:24:59.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:59.339+0000 7fa77cd86700 1 -- 172.21.15.97:0/2202105659 shutdown_connections 2024-04-23T10:24:59.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:24:59.339+0000 7fa77cd86700 1 -- 172.21.15.97:0/2202105659 wait complete. 2024-04-23T10:25:00.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:59 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/2202105659' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-23T10:25:00.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:59 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-04-23T10:25:00.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:59 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-04-23T10:25:00.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:59 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-04-23T10:25:00.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:59 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4]}]: dispatch 2024-04-23T10:25:00.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:59 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-04-23T10:25:00.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:59 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-04-23T10:25:00.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:24:59 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-04-23T10:25:00.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:59 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/2202105659' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-23T10:25:00.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:59 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-04-23T10:25:00.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:59 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-04-23T10:25:00.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:59 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-04-23T10:25:00.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:59 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4]}]: dispatch 2024-04-23T10:25:00.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:59 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-04-23T10:25:00.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:59 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-04-23T10:25:00.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:24:59 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-04-23T10:25:00.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:59 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/2202105659' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-23T10:25:00.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:59 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-04-23T10:25:00.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:59 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-04-23T10:25:00.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:59 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-04-23T10:25:00.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:59 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4]}]: dispatch 2024-04-23T10:25:00.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:59 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-04-23T10:25:00.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:59 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-04-23T10:25:00.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:24:59 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-04-23T10:25:00.502 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612633 got 313532612637 for osd.10 2024-04-23T10:25:00.502 DEBUG:teuthology.parallel:result is None 2024-04-23T10:25:00.794 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:00 smithi097 ceph-mon[29197]: pgmap v316: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-04-23T10:25:00.794 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:00 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-04-23T10:25:00.794 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:00 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-04-23T10:25:00.794 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:00 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-04-23T10:25:00.795 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:00 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4]}]': finished 2024-04-23T10:25:00.795 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:00 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-04-23T10:25:00.795 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:00 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-04-23T10:25:00.795 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:00 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-04-23T10:25:00.795 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:00 smithi097 ceph-mon[29197]: osdmap e87: 12 total, 12 up, 12 in 2024-04-23T10:25:01.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:00 smithi196 ceph-mon[32919]: pgmap v316: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-04-23T10:25:01.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:00 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-04-23T10:25:01.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:00 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-04-23T10:25:01.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:00 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-04-23T10:25:01.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:00 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4]}]': finished 2024-04-23T10:25:01.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:00 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-04-23T10:25:01.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:00 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-04-23T10:25:01.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:00 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-04-23T10:25:01.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:00 smithi196 ceph-mon[32919]: osdmap e87: 12 total, 12 up, 12 in 2024-04-23T10:25:01.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:00 smithi183 ceph-mon[32317]: pgmap v316: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-04-23T10:25:01.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:00 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-04-23T10:25:01.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:00 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-04-23T10:25:01.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:00 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-04-23T10:25:01.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:00 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4]}]': finished 2024-04-23T10:25:01.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:00 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-04-23T10:25:01.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:00 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-04-23T10:25:01.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:00 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-04-23T10:25:01.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:00 smithi183 ceph-mon[32317]: osdmap e87: 12 total, 12 up, 12 in 2024-04-23T10:25:01.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:01 smithi097 ceph-mon[29197]: osdmap e88: 12 total, 12 up, 12 in 2024-04-23T10:25:02.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:01 smithi196 ceph-mon[32919]: osdmap e88: 12 total, 12 up, 12 in 2024-04-23T10:25:02.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:01 smithi183 ceph-mon[32317]: osdmap e88: 12 total, 12 up, 12 in 2024-04-23T10:25:03.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:02 smithi196 ceph-mon[32919]: pgmap v319: 97 pgs: 2 peering, 95 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 11 op/s 2024-04-23T10:25:03.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:02 smithi097 ceph-mon[29197]: pgmap v319: 97 pgs: 2 peering, 95 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 11 op/s 2024-04-23T10:25:03.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:02 smithi183 ceph-mon[32317]: pgmap v319: 97 pgs: 2 peering, 95 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 11 op/s 2024-04-23T10:25:03.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:03.803+0000 7f01a1424700 1 -- 172.21.15.97:0/151149871 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f019c102d50 msgr2=0x7f019c1065e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:03.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:03.803+0000 7f01a1424700 1 --2- 172.21.15.97:0/151149871 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f019c102d50 0x7f019c1065e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f01900099c0 tx=0x7f0190009cd0 comp rx=0 tx=0).stop 2024-04-23T10:25:03.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:03.803+0000 7f01a1424700 1 -- 172.21.15.97:0/151149871 shutdown_connections 2024-04-23T10:25:03.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:03.803+0000 7f01a1424700 1 --2- 172.21.15.97:0/151149871 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f019c106cc0 0x7f019c10e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:03.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:03.803+0000 7f01a1424700 1 --2- 172.21.15.97:0/151149871 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f019c1023a0 0x7f019c102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:03.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:03.803+0000 7f01a1424700 1 --2- 172.21.15.97:0/151149871 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f019c102d50 0x7f019c1065e0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:03.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:03.803+0000 7f01a1424700 1 -- 172.21.15.97:0/151149871 >> 172.21.15.97:0/151149871 conn(0x7f019c077fb0 msgr2=0x7f019c0ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:03.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:03.803+0000 7f01a1424700 1 -- 172.21.15.97:0/151149871 shutdown_connections 2024-04-23T10:25:03.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:03.803+0000 7f01a1424700 1 -- 172.21.15.97:0/151149871 wait complete. 2024-04-23T10:25:03.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:03.803+0000 7f01a1424700 1 Processor -- start 2024-04-23T10:25:03.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:03.804+0000 7f01a1424700 1 -- start start 2024-04-23T10:25:03.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:03.804+0000 7f01a1424700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f019c1023a0 0x7f019c199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:03.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:03.804+0000 7f01a1424700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f019c102d50 0x7f019c199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:03.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:03.804+0000 7f01a1424700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f019c106cc0 0x7f019c19dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:03.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:03.804+0000 7f01a1424700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f019c111c70 con 0x7f019c106cc0 2024-04-23T10:25:03.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:03.804+0000 7f01a1424700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f019c111af0 con 0x7f019c1023a0 2024-04-23T10:25:03.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:03.804+0000 7f01a1424700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f019c111df0 con 0x7f019c102d50 2024-04-23T10:25:03.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:03.804+0000 7f019b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f019c106cc0 0x7f019c19dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:03.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:03.804+0000 7f019b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f019c106cc0 0x7f019c19dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46670/0 (socket says 172.21.15.97:46670) 2024-04-23T10:25:03.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:03.804+0000 7f019b7fe700 1 -- 172.21.15.97:0/24207152 learned_addr learned my addr 172.21.15.97:0/24207152 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:25:03.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:03.804+0000 7f019affd700 1 --2- 172.21.15.97:0/24207152 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f019c1023a0 0x7f019c199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:03.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:03.804+0000 7f019a7fc700 1 --2- 172.21.15.97:0/24207152 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f019c102d50 0x7f019c199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:03.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:03.804+0000 7f019b7fe700 1 -- 172.21.15.97:0/24207152 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f019c102d50 msgr2=0x7f019c199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:03.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:03.804+0000 7f019b7fe700 1 --2- 172.21.15.97:0/24207152 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f019c102d50 0x7f019c199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:03.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:03.804+0000 7f019b7fe700 1 -- 172.21.15.97:0/24207152 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f019c1023a0 msgr2=0x7f019c199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:03.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:03.804+0000 7f019b7fe700 1 --2- 172.21.15.97:0/24207152 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f019c1023a0 0x7f019c199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:03.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:03.804+0000 7f019b7fe700 1 -- 172.21.15.97:0/24207152 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f019c19e320 con 0x7f019c106cc0 2024-04-23T10:25:03.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:03.804+0000 7f019a7fc700 1 --2- 172.21.15.97:0/24207152 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f019c102d50 0x7f019c199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:25:03.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:03.805+0000 7f019b7fe700 1 --2- 172.21.15.97:0/24207152 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f019c106cc0 0x7f019c19dbe0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f019400b970 tx=0x7f019400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:03.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:03.805+0000 7f0183fff700 1 -- 172.21.15.97:0/24207152 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0194019070 con 0x7f019c106cc0 2024-04-23T10:25:03.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:03.805+0000 7f01a1424700 1 -- 172.21.15.97:0/24207152 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f019c19e610 con 0x7f019c106cc0 2024-04-23T10:25:03.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:03.805+0000 7f0183fff700 1 -- 172.21.15.97:0/24207152 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f019400cde0 con 0x7f019c106cc0 2024-04-23T10:25:03.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:03.805+0000 7f0183fff700 1 -- 172.21.15.97:0/24207152 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0194007370 con 0x7f019c106cc0 2024-04-23T10:25:03.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:03.805+0000 7f01a1424700 1 -- 172.21.15.97:0/24207152 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f019c1a5e90 con 0x7f019c106cc0 2024-04-23T10:25:03.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:03.806+0000 7f01a1424700 1 -- 172.21.15.97:0/24207152 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f019c103150 con 0x7f019c106cc0 2024-04-23T10:25:03.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:03.807+0000 7f0183fff700 1 -- 172.21.15.97:0/24207152 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f019401c070 con 0x7f019c106cc0 2024-04-23T10:25:03.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:03.807+0000 7f0183fff700 1 --2- 172.21.15.97:0/24207152 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f01840a3cd0 0x7f01840a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:03.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:03.807+0000 7f0183fff700 1 -- 172.21.15.97:0/24207152 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f01940d0070 con 0x7f019c106cc0 2024-04-23T10:25:03.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:03.807+0000 7f019affd700 1 --2- 172.21.15.97:0/24207152 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f01840a3cd0 0x7f01840a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:03.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:03.808+0000 7f019affd700 1 --2- 172.21.15.97:0/24207152 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f01840a3cd0 0x7f01840a6190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f018c009a10 tx=0x7f018c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:03.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:03.811+0000 7f0183fff700 1 -- 172.21.15.97:0/24207152 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0194023070 con 0x7f019c106cc0 2024-04-23T10:25:04.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:04.015+0000 7f01a1424700 1 -- 172.21.15.97:0/24207152 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f019c1a6120 con 0x7f019c106cc0 2024-04-23T10:25:04.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:04.015+0000 7f0183fff700 1 -- 172.21.15.97:0/24207152 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f019409ca30 con 0x7f019c106cc0 2024-04-23T10:25:04.015 INFO:teuthology.orchestra.run.smithi097.stdout:347892351001 2024-04-23T10:25:04.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:04.017+0000 7f01a1424700 1 -- 172.21.15.97:0/24207152 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f01840a3cd0 msgr2=0x7f01840a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:04.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:04.017+0000 7f01a1424700 1 --2- 172.21.15.97:0/24207152 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f01840a3cd0 0x7f01840a6190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f018c009a10 tx=0x7f018c009190 comp rx=0 tx=0).stop 2024-04-23T10:25:04.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:04.017+0000 7f01a1424700 1 -- 172.21.15.97:0/24207152 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f019c106cc0 msgr2=0x7f019c19dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:04.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:04.017+0000 7f01a1424700 1 --2- 172.21.15.97:0/24207152 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f019c106cc0 0x7f019c19dbe0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f019400b970 tx=0x7f019400bc80 comp rx=0 tx=0).stop 2024-04-23T10:25:04.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:04.017+0000 7f01a1424700 1 -- 172.21.15.97:0/24207152 shutdown_connections 2024-04-23T10:25:04.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:04.017+0000 7f01a1424700 1 --2- 172.21.15.97:0/24207152 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f01840a3cd0 0x7f01840a6190 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:04.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:04.017+0000 7f01a1424700 1 --2- 172.21.15.97:0/24207152 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f019c106cc0 0x7f019c19dbe0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:04.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:04.017+0000 7f01a1424700 1 --2- 172.21.15.97:0/24207152 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f019c1023a0 0x7f019c199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:04.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:04.017+0000 7f01a1424700 1 --2- 172.21.15.97:0/24207152 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f019c102d50 0x7f019c199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:04.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:04.017+0000 7f01a1424700 1 -- 172.21.15.97:0/24207152 >> 172.21.15.97:0/24207152 conn(0x7f019c077fb0 msgr2=0x7f019c1012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:04.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:04.017+0000 7f01a1424700 1 -- 172.21.15.97:0/24207152 shutdown_connections 2024-04-23T10:25:04.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:04.018+0000 7f01a1424700 1 -- 172.21.15.97:0/24207152 wait complete. 2024-04-23T10:25:04.800 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:04 smithi097 ceph-mon[29197]: pgmap v320: 97 pgs: 2 peering, 95 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s; 83 B/s, 0 objects/s recovering 2024-04-23T10:25:04.800 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:04 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/24207152' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-23T10:25:04.833 INFO:tasks.cephadm.ceph_manager.ceph:need seq 347892350997 got 347892351001 for osd.11 2024-04-23T10:25:04.834 DEBUG:teuthology.parallel:result is None 2024-04-23T10:25:04.859 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T10:25:04.859 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T10:25:04.860 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T10:25:04.860 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T10:25:04.860 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T10:25:05.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:04 smithi196 ceph-mon[32919]: pgmap v320: 97 pgs: 2 peering, 95 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s; 83 B/s, 0 objects/s recovering 2024-04-23T10:25:05.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:04 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/24207152' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-23T10:25:05.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:04 smithi183 ceph-mon[32317]: pgmap v320: 97 pgs: 2 peering, 95 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s; 83 B/s, 0 objects/s recovering 2024-04-23T10:25:05.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:04 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/24207152' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-23T10:25:05.158 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T10:25:05.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:05.187+0000 7fdfcfce4700 1 -- 172.21.15.97:0/3125112844 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfc8101030 msgr2=0x7fdfc8101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:05.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:05.187+0000 7fdfcfce4700 1 --2- 172.21.15.97:0/3125112844 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfc8101030 0x7fdfc8101490 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8009990 tx=0x7fdfb8009ca0 comp rx=0 tx=0).stop 2024-04-23T10:25:05.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:05.187+0000 7fdfcfce4700 1 -- 172.21.15.97:0/3125112844 shutdown_connections 2024-04-23T10:25:05.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:05.187+0000 7fdfcfce4700 1 --2- 172.21.15.97:0/3125112844 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfc81019d0 0x7fdfc810f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:05.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:05.187+0000 7fdfcfce4700 1 --2- 172.21.15.97:0/3125112844 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdfc8108ad0 0x7fdfc8108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:05.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:05.187+0000 7fdfcfce4700 1 --2- 172.21.15.97:0/3125112844 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfc8101030 0x7fdfc8101490 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:05.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:05.187+0000 7fdfcfce4700 1 -- 172.21.15.97:0/3125112844 >> 172.21.15.97:0/3125112844 conn(0x7fdfc8077fb0 msgr2=0x7fdfc80ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:05.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:05.188+0000 7fdfcfce4700 1 -- 172.21.15.97:0/3125112844 shutdown_connections 2024-04-23T10:25:05.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:05.188+0000 7fdfcfce4700 1 -- 172.21.15.97:0/3125112844 wait complete. 2024-04-23T10:25:05.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:05.188+0000 7fdfcfce4700 1 Processor -- start 2024-04-23T10:25:05.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:05.188+0000 7fdfcfce4700 1 -- start start 2024-04-23T10:25:05.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:05.189+0000 7fdfcfce4700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfc8101030 0x7fdfc81990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:05.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:05.189+0000 7fdfcfce4700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdfc81019d0 0x7fdfc8199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:05.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:05.189+0000 7fdfcfce4700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfc8108ad0 0x7fdfc819d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:05.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:05.189+0000 7fdfcfce4700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc8112be0 con 0x7fdfc8101030 2024-04-23T10:25:05.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:05.189+0000 7fdfcfce4700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc8112a60 con 0x7fdfc8108ad0 2024-04-23T10:25:05.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:05.189+0000 7fdfcfce4700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc8112d60 con 0x7fdfc81019d0 2024-04-23T10:25:05.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:05.189+0000 7fdfcda80700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfc8101030 0x7fdfc81990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:05.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:05.189+0000 7fdfcda80700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfc8101030 0x7fdfc81990d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46688/0 (socket says 172.21.15.97:46688) 2024-04-23T10:25:05.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:05.189+0000 7fdfcda80700 1 -- 172.21.15.97:0/112478281 learned_addr learned my addr 172.21.15.97:0/112478281 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:25:05.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:05.189+0000 7fdfcd27f700 1 --2- 172.21.15.97:0/112478281 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdfc81019d0 0x7fdfc8199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:05.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:05.189+0000 7fdfce281700 1 --2- 172.21.15.97:0/112478281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfc8108ad0 0x7fdfc819d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:05.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:05.189+0000 7fdfcda80700 1 -- 172.21.15.97:0/112478281 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdfc81019d0 msgr2=0x7fdfc8199610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:05.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:05.189+0000 7fdfcda80700 1 --2- 172.21.15.97:0/112478281 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdfc81019d0 0x7fdfc8199610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:05.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:05.189+0000 7fdfcda80700 1 -- 172.21.15.97:0/112478281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfc8108ad0 msgr2=0x7fdfc819d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:05.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:05.189+0000 7fdfcda80700 1 --2- 172.21.15.97:0/112478281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfc8108ad0 0x7fdfc819d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:05.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:05.189+0000 7fdfcda80700 1 -- 172.21.15.97:0/112478281 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfc819e0e0 con 0x7fdfc8101030 2024-04-23T10:25:05.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:05.189+0000 7fdfcda80700 1 --2- 172.21.15.97:0/112478281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfc8101030 0x7fdfc81990d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fdfc400d950 tx=0x7fdfc400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:05.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:05.190+0000 7fdfbeffd700 1 -- 172.21.15.97:0/112478281 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc4016070 con 0x7fdfc8101030 2024-04-23T10:25:05.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:05.190+0000 7fdfcfce4700 1 -- 172.21.15.97:0/112478281 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfc819e3d0 con 0x7fdfc8101030 2024-04-23T10:25:05.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:05.190+0000 7fdfbeffd700 1 -- 172.21.15.97:0/112478281 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfc400fd40 con 0x7fdfc8101030 2024-04-23T10:25:05.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:05.190+0000 7fdfcfce4700 1 -- 172.21.15.97:0/112478281 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfc8102140 con 0x7fdfc8101030 2024-04-23T10:25:05.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:05.190+0000 7fdfbeffd700 1 -- 172.21.15.97:0/112478281 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc400e3b0 con 0x7fdfc8101030 2024-04-23T10:25:05.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:05.191+0000 7fdfcfce4700 1 -- 172.21.15.97:0/112478281 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfc8069730 con 0x7fdfc8101030 2024-04-23T10:25:05.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:05.192+0000 7fdfbeffd700 1 -- 172.21.15.97:0/112478281 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fdfc4019070 con 0x7fdfc8101030 2024-04-23T10:25:05.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:05.193+0000 7fdfbeffd700 1 --2- 172.21.15.97:0/112478281 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fdfb40a3cd0 0x7fdfb40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:05.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:05.193+0000 7fdfbeffd700 1 -- 172.21.15.97:0/112478281 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fdfc40d03f0 con 0x7fdfc8101030 2024-04-23T10:25:05.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:05.193+0000 7fdfcd27f700 1 --2- 172.21.15.97:0/112478281 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fdfb40a3cd0 0x7fdfb40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:05.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:05.193+0000 7fdfcd27f700 1 --2- 172.21.15.97:0/112478281 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fdfb40a3cd0 0x7fdfb40a6190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdfb800b5c0 tx=0x7fdfb8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:05.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:05.197+0000 7fdfbeffd700 1 -- 172.21.15.97:0/112478281 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdfc409cca0 con 0x7fdfc8101030 2024-04-23T10:25:05.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:05.410+0000 7fdfcfce4700 1 -- 172.21.15.97:0/112478281 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fdfc802df80 con 0x7fdfc8101030 2024-04-23T10:25:05.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:05.411+0000 7fdfbeffd700 1 -- 172.21.15.97:0/112478281 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fdfc409c5a0 con 0x7fdfc8101030 2024-04-23T10:25:05.410 INFO:teuthology.orchestra.run.smithi097.stdout:55834574924 2024-04-23T10:25:05.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:05.413+0000 7fdfcfce4700 1 -- 172.21.15.97:0/112478281 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fdfb40a3cd0 msgr2=0x7fdfb40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:05.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:05.413+0000 7fdfcfce4700 1 --2- 172.21.15.97:0/112478281 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fdfb40a3cd0 0x7fdfb40a6190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdfb800b5c0 tx=0x7fdfb8005960 comp rx=0 tx=0).stop 2024-04-23T10:25:05.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:05.413+0000 7fdfcfce4700 1 -- 172.21.15.97:0/112478281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfc8101030 msgr2=0x7fdfc81990d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:05.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:05.413+0000 7fdfcfce4700 1 --2- 172.21.15.97:0/112478281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfc8101030 0x7fdfc81990d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fdfc400d950 tx=0x7fdfc400dc60 comp rx=0 tx=0).stop 2024-04-23T10:25:05.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:05.414+0000 7fdfcfce4700 1 -- 172.21.15.97:0/112478281 shutdown_connections 2024-04-23T10:25:05.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:05.414+0000 7fdfcfce4700 1 --2- 172.21.15.97:0/112478281 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fdfb40a3cd0 0x7fdfb40a6190 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:05.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:05.414+0000 7fdfcfce4700 1 --2- 172.21.15.97:0/112478281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfc8108ad0 0x7fdfc819d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:05.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:05.414+0000 7fdfcfce4700 1 --2- 172.21.15.97:0/112478281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfc8101030 0x7fdfc81990d0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:05.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:05.414+0000 7fdfcfce4700 1 --2- 172.21.15.97:0/112478281 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdfc81019d0 0x7fdfc8199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:05.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:05.414+0000 7fdfcfce4700 1 -- 172.21.15.97:0/112478281 >> 172.21.15.97:0/112478281 conn(0x7fdfc8077fb0 msgr2=0x7fdfc80ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:05.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:05.414+0000 7fdfcfce4700 1 -- 172.21.15.97:0/112478281 shutdown_connections 2024-04-23T10:25:05.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:05.414+0000 7fdfcfce4700 1 -- 172.21.15.97:0/112478281 wait complete. 2024-04-23T10:25:05.983 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:05 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:25:05.984 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:05 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/112478281' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-23T10:25:05.984 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:05 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:25:05.984 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:05 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/112478281' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-23T10:25:06.042 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:05 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:25:06.042 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:05 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/112478281' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-23T10:25:06.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:06.574+0000 7f8d257f0700 1 -- 172.21.15.97:0/561979084 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d20103450 msgr2=0x7f8d2010f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:06.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:06.574+0000 7f8d257f0700 1 --2- 172.21.15.97:0/561979084 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d20103450 0x7f8d2010f8e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f8d1800b0a0 tx=0x7f8d1800b3b0 comp rx=0 tx=0).stop 2024-04-23T10:25:06.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:06.574+0000 7f8d257f0700 1 -- 172.21.15.97:0/561979084 shutdown_connections 2024-04-23T10:25:06.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:06.574+0000 7f8d257f0700 1 --2- 172.21.15.97:0/561979084 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d20103450 0x7f8d2010f8e0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:06.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:06.574+0000 7f8d257f0700 1 --2- 172.21.15.97:0/561979084 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d20108a80 0x7f8d20108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:06.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:06.574+0000 7f8d257f0700 1 --2- 172.21.15.97:0/561979084 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8d20102a90 0x7f8d20102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:06.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:06.574+0000 7f8d257f0700 1 -- 172.21.15.97:0/561979084 >> 172.21.15.97:0/561979084 conn(0x7f8d20079bf0 msgr2=0x7f8d2007a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:06.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:06.574+0000 7f8d257f0700 1 -- 172.21.15.97:0/561979084 shutdown_connections 2024-04-23T10:25:06.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:06.574+0000 7f8d257f0700 1 -- 172.21.15.97:0/561979084 wait complete. 2024-04-23T10:25:06.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:06.575+0000 7f8d257f0700 1 Processor -- start 2024-04-23T10:25:06.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:06.575+0000 7f8d257f0700 1 -- start start 2024-04-23T10:25:06.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:06.575+0000 7f8d257f0700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d20102a90 0x7f8d20199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:06.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:06.575+0000 7f8d257f0700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8d20103450 0x7f8d20199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:06.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:06.575+0000 7f8d257f0700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d20108a80 0x7f8d2019d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:06.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:06.575+0000 7f8d257f0700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8d20112c10 con 0x7f8d20102a90 2024-04-23T10:25:06.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:06.575+0000 7f8d257f0700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8d20112a90 con 0x7f8d20103450 2024-04-23T10:25:06.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:06.575+0000 7f8d257f0700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8d20112d90 con 0x7f8d20108a80 2024-04-23T10:25:06.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:06.576+0000 7f8d1effd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d20102a90 0x7f8d20199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:06.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:06.576+0000 7f8d1effd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d20102a90 0x7f8d20199120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36340/0 (socket says 172.21.15.97:36340) 2024-04-23T10:25:06.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:06.576+0000 7f8d1effd700 1 -- 172.21.15.97:0/594072191 learned_addr learned my addr 172.21.15.97:0/594072191 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:25:06.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:06.576+0000 7f8d1e7fc700 1 --2- 172.21.15.97:0/594072191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8d20103450 0x7f8d20199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:06.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:06.576+0000 7f8d1f7fe700 1 --2- 172.21.15.97:0/594072191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d20108a80 0x7f8d2019d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:06.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:06.576+0000 7f8d1effd700 1 -- 172.21.15.97:0/594072191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d20108a80 msgr2=0x7f8d2019d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:06.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:06.576+0000 7f8d1effd700 1 --2- 172.21.15.97:0/594072191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d20108a80 0x7f8d2019d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:06.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:06.576+0000 7f8d1effd700 1 -- 172.21.15.97:0/594072191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8d20103450 msgr2=0x7f8d20199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:06.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:06.576+0000 7f8d1effd700 1 --2- 172.21.15.97:0/594072191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8d20103450 0x7f8d20199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:06.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:06.576+0000 7f8d1effd700 1 -- 172.21.15.97:0/594072191 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d2019e0e0 con 0x7f8d20102a90 2024-04-23T10:25:06.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:06.576+0000 7f8d1effd700 1 --2- 172.21.15.97:0/594072191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d20102a90 0x7f8d20199120 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8d1400d980 tx=0x7f8d1400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:06.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:06.576+0000 7f8d07fff700 1 -- 172.21.15.97:0/594072191 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d14016070 con 0x7f8d20102a90 2024-04-23T10:25:06.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:06.576+0000 7f8d07fff700 1 -- 172.21.15.97:0/594072191 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d14004830 con 0x7f8d20102a90 2024-04-23T10:25:06.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:06.576+0000 7f8d257f0700 1 -- 172.21.15.97:0/594072191 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d2019e3d0 con 0x7f8d20102a90 2024-04-23T10:25:06.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:06.576+0000 7f8d07fff700 1 -- 172.21.15.97:0/594072191 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d14005540 con 0x7f8d20102a90 2024-04-23T10:25:06.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:06.576+0000 7f8d257f0700 1 -- 172.21.15.97:0/594072191 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d201a5c50 con 0x7f8d20102a90 2024-04-23T10:25:06.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:06.577+0000 7f8d257f0700 1 -- 172.21.15.97:0/594072191 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d20069730 con 0x7f8d20102a90 2024-04-23T10:25:06.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:06.579+0000 7f8d07fff700 1 -- 172.21.15.97:0/594072191 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8d14004220 con 0x7f8d20102a90 2024-04-23T10:25:06.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:06.579+0000 7f8d07fff700 1 --2- 172.21.15.97:0/594072191 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f8d080a3cd0 0x7f8d080a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:06.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:06.579+0000 7f8d1e7fc700 1 --2- 172.21.15.97:0/594072191 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f8d080a3cd0 0x7f8d080a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:06.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:06.580+0000 7f8d07fff700 1 -- 172.21.15.97:0/594072191 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f8d140d0770 con 0x7f8d20102a90 2024-04-23T10:25:06.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:06.580+0000 7f8d1e7fc700 1 --2- 172.21.15.97:0/594072191 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f8d080a3cd0 0x7f8d080a6190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8d2007a210 tx=0x7f8d1000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:06.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:06.583+0000 7f8d07fff700 1 -- 172.21.15.97:0/594072191 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8d1401f020 con 0x7f8d20102a90 2024-04-23T10:25:06.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:06.789+0000 7f8d257f0700 1 -- 172.21.15.97:0/594072191 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f8d2002df80 con 0x7f8d20102a90 2024-04-23T10:25:06.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:06.789+0000 7f8d07fff700 1 -- 172.21.15.97:0/594072191 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f8d1409d020 con 0x7f8d20102a90 2024-04-23T10:25:06.788 INFO:teuthology.orchestra.run.smithi097.stdout:167503724600 2024-04-23T10:25:06.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:06.791+0000 7f8d257f0700 1 -- 172.21.15.97:0/594072191 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f8d080a3cd0 msgr2=0x7f8d080a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:06.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:06.791+0000 7f8d257f0700 1 --2- 172.21.15.97:0/594072191 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f8d080a3cd0 0x7f8d080a6190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8d2007a210 tx=0x7f8d1000b410 comp rx=0 tx=0).stop 2024-04-23T10:25:06.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:06.791+0000 7f8d257f0700 1 -- 172.21.15.97:0/594072191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d20102a90 msgr2=0x7f8d20199120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:06.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:06.791+0000 7f8d257f0700 1 --2- 172.21.15.97:0/594072191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d20102a90 0x7f8d20199120 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8d1400d980 tx=0x7f8d1400dc90 comp rx=0 tx=0).stop 2024-04-23T10:25:06.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:06.792+0000 7f8d257f0700 1 -- 172.21.15.97:0/594072191 shutdown_connections 2024-04-23T10:25:06.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:06.792+0000 7f8d257f0700 1 --2- 172.21.15.97:0/594072191 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f8d080a3cd0 0x7f8d080a6190 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:06.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:06.792+0000 7f8d257f0700 1 --2- 172.21.15.97:0/594072191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d20108a80 0x7f8d2019d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:06.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:06.792+0000 7f8d257f0700 1 --2- 172.21.15.97:0/594072191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d20102a90 0x7f8d20199120 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:06.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:06.792+0000 7f8d257f0700 1 --2- 172.21.15.97:0/594072191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8d20103450 0x7f8d20199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:06.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:06.792+0000 7f8d257f0700 1 -- 172.21.15.97:0/594072191 >> 172.21.15.97:0/594072191 conn(0x7f8d20079bf0 msgr2=0x7f8d201016b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:06.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:06.792+0000 7f8d257f0700 1 -- 172.21.15.97:0/594072191 shutdown_connections 2024-04-23T10:25:06.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:06.792+0000 7f8d257f0700 1 -- 172.21.15.97:0/594072191 wait complete. 2024-04-23T10:25:07.017 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:06 smithi097 ceph-mon[29197]: pgmap v321: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 8 op/s; 156 B/s, 1 objects/s recovering 2024-04-23T10:25:07.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:06 smithi196 ceph-mon[32919]: pgmap v321: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 8 op/s; 156 B/s, 1 objects/s recovering 2024-04-23T10:25:07.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:06 smithi183 ceph-mon[32317]: pgmap v321: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 8 op/s; 156 B/s, 1 objects/s recovering 2024-04-23T10:25:07.718 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:07 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/594072191' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-23T10:25:07.718 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:07 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:25:07.718 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:07 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:25:07.718 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:07 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:25:07.718 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:07 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:25:08.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:07 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/594072191' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-23T10:25:08.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:07 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:25:08.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:07 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:25:08.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:07 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:25:08.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:07 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:25:08.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:07 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/594072191' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-23T10:25:08.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:07 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:25:08.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:07 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:25:08.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:07 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:25:08.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:07 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:25:08.783 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:08 smithi097 ceph-mon[29197]: pgmap v322: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s; 156 B/s, 1 objects/s recovering 2024-04-23T10:25:08.813 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574918 got 55834574924 for osd.1 2024-04-23T10:25:08.813 DEBUG:teuthology.parallel:result is None 2024-04-23T10:25:09.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:08 smithi196 ceph-mon[32919]: pgmap v322: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s; 156 B/s, 1 objects/s recovering 2024-04-23T10:25:09.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:08 smithi183 ceph-mon[32317]: pgmap v322: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s; 156 B/s, 1 objects/s recovering 2024-04-23T10:25:09.842 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:09 smithi097 ceph-mon[29197]: pgmap v323: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 8 op/s; 126 B/s, 1 objects/s recovering 2024-04-23T10:25:09.869 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724595 got 167503724600 for osd.5 2024-04-23T10:25:09.869 DEBUG:teuthology.parallel:result is None 2024-04-23T10:25:10.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:09 smithi196 ceph-mon[32919]: pgmap v323: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 8 op/s; 126 B/s, 1 objects/s recovering 2024-04-23T10:25:10.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:09 smithi183 ceph-mon[32317]: pgmap v323: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 8 op/s; 126 B/s, 1 objects/s recovering 2024-04-23T10:25:10.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:10.523+0000 7fd1f968c700 1 -- 172.21.15.97:0/3985834921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1f4107990 msgr2=0x7fd1f4107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:10.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:10.523+0000 7fd1f968c700 1 --2- 172.21.15.97:0/3985834921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1f4107990 0x7fd1f4107d70 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec00b0a0 tx=0x7fd1ec00b3b0 comp rx=0 tx=0).stop 2024-04-23T10:25:10.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:10.523+0000 7fd1f968c700 1 -- 172.21.15.97:0/3985834921 shutdown_connections 2024-04-23T10:25:10.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:10.523+0000 7fd1f968c700 1 --2- 172.21.15.97:0/3985834921 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1f4102300 0x7fd1f410e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:10.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:10.523+0000 7fd1f968c700 1 --2- 172.21.15.97:0/3985834921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1f4107990 0x7fd1f4107d70 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:10.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:10.523+0000 7fd1f968c700 1 --2- 172.21.15.97:0/3985834921 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1f4101940 0x7fd1f4101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:10.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:10.523+0000 7fd1f968c700 1 -- 172.21.15.97:0/3985834921 >> 172.21.15.97:0/3985834921 conn(0x7fd1f40788f0 msgr2=0x7fd1f4078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:10.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:10.523+0000 7fd1f968c700 1 -- 172.21.15.97:0/3985834921 shutdown_connections 2024-04-23T10:25:10.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:10.523+0000 7fd1f968c700 1 -- 172.21.15.97:0/3985834921 wait complete. 2024-04-23T10:25:10.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:10.523+0000 7fd1f968c700 1 Processor -- start 2024-04-23T10:25:10.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:10.524+0000 7fd1f968c700 1 -- start start 2024-04-23T10:25:10.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:10.524+0000 7fd1f968c700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1f4101940 0x7fd1f4199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:10.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:10.524+0000 7fd1f968c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1f4102300 0x7fd1f41997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:10.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:10.524+0000 7fd1f968c700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1f4107990 0x7fd1f419daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:10.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:10.524+0000 7fd1f968c700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f4111b30 con 0x7fd1f4102300 2024-04-23T10:25:10.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:10.524+0000 7fd1f968c700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f41119b0 con 0x7fd1f4107990 2024-04-23T10:25:10.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:10.524+0000 7fd1f968c700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f4111cb0 con 0x7fd1f4101940 2024-04-23T10:25:10.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:10.524+0000 7fd1f27fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1f4102300 0x7fd1f41997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:10.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:10.524+0000 7fd1f27fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1f4102300 0x7fd1f41997b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36356/0 (socket says 172.21.15.97:36356) 2024-04-23T10:25:10.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:10.524+0000 7fd1f27fc700 1 -- 172.21.15.97:0/2405064226 learned_addr learned my addr 172.21.15.97:0/2405064226 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:25:10.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:10.524+0000 7fd1f37fe700 1 --2- 172.21.15.97:0/2405064226 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1f4107990 0x7fd1f419daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:10.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:10.524+0000 7fd1f2ffd700 1 --2- 172.21.15.97:0/2405064226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1f4101940 0x7fd1f4199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:10.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:10.524+0000 7fd1f27fc700 1 -- 172.21.15.97:0/2405064226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1f4101940 msgr2=0x7fd1f4199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:10.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:10.524+0000 7fd1f27fc700 1 --2- 172.21.15.97:0/2405064226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1f4101940 0x7fd1f4199270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:10.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:10.524+0000 7fd1f27fc700 1 -- 172.21.15.97:0/2405064226 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1f4107990 msgr2=0x7fd1f419daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:10.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:10.524+0000 7fd1f27fc700 1 --2- 172.21.15.97:0/2405064226 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1f4107990 0x7fd1f419daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:10.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:10.524+0000 7fd1f27fc700 1 -- 172.21.15.97:0/2405064226 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1f419e230 con 0x7fd1f4102300 2024-04-23T10:25:10.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:10.524+0000 7fd1f27fc700 1 --2- 172.21.15.97:0/2405064226 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1f4102300 0x7fd1f41997b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc00c920 tx=0x7fd1dc00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:10.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:10.524+0000 7fd1ebfff700 1 -- 172.21.15.97:0/2405064226 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc007b00 con 0x7fd1f4102300 2024-04-23T10:25:10.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:10.525+0000 7fd1f968c700 1 -- 172.21.15.97:0/2405064226 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1f419e520 con 0x7fd1f4102300 2024-04-23T10:25:10.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:10.525+0000 7fd1ebfff700 1 -- 172.21.15.97:0/2405064226 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1dc004ce0 con 0x7fd1f4102300 2024-04-23T10:25:10.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:10.525+0000 7fd1ebfff700 1 -- 172.21.15.97:0/2405064226 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc005650 con 0x7fd1f4102300 2024-04-23T10:25:10.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:10.525+0000 7fd1f968c700 1 -- 172.21.15.97:0/2405064226 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1f41a5da0 con 0x7fd1f4102300 2024-04-23T10:25:10.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:10.525+0000 7fd1f968c700 1 -- 172.21.15.97:0/2405064226 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1f4069730 con 0x7fd1f4102300 2024-04-23T10:25:10.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:10.527+0000 7fd1ebfff700 1 -- 172.21.15.97:0/2405064226 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd1dc007ca0 con 0x7fd1f4102300 2024-04-23T10:25:10.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:10.527+0000 7fd1ebfff700 1 --2- 172.21.15.97:0/2405064226 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fd1e00a3cd0 0x7fd1e00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:10.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:10.527+0000 7fd1f2ffd700 1 --2- 172.21.15.97:0/2405064226 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fd1e00a3cd0 0x7fd1e00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:10.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:10.527+0000 7fd1ebfff700 1 -- 172.21.15.97:0/2405064226 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fd1dc0d06b0 con 0x7fd1f4102300 2024-04-23T10:25:10.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:10.528+0000 7fd1f2ffd700 1 --2- 172.21.15.97:0/2405064226 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fd1e00a3cd0 0x7fd1e00a6190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec00b070 tx=0x7fd1ec00b7b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:10.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:10.531+0000 7fd1ebfff700 1 -- 172.21.15.97:0/2405064226 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd1dc014030 con 0x7fd1f4102300 2024-04-23T10:25:10.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:10.742+0000 7fd1f968c700 1 -- 172.21.15.97:0/2405064226 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fd1f41a5fd0 con 0x7fd1f4102300 2024-04-23T10:25:10.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:10.742+0000 7fd1ebfff700 1 -- 172.21.15.97:0/2405064226 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd1dc09cf60 con 0x7fd1f4102300 2024-04-23T10:25:10.741 INFO:teuthology.orchestra.run.smithi097.stdout:34359738450 2024-04-23T10:25:10.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:10.744+0000 7fd1f968c700 1 -- 172.21.15.97:0/2405064226 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fd1e00a3cd0 msgr2=0x7fd1e00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:10.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:10.744+0000 7fd1f968c700 1 --2- 172.21.15.97:0/2405064226 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fd1e00a3cd0 0x7fd1e00a6190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec00b070 tx=0x7fd1ec00b7b0 comp rx=0 tx=0).stop 2024-04-23T10:25:10.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:10.744+0000 7fd1f968c700 1 -- 172.21.15.97:0/2405064226 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1f4102300 msgr2=0x7fd1f41997b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:10.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:10.744+0000 7fd1f968c700 1 --2- 172.21.15.97:0/2405064226 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1f4102300 0x7fd1f41997b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc00c920 tx=0x7fd1dc00cce0 comp rx=0 tx=0).stop 2024-04-23T10:25:10.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:10.744+0000 7fd1f968c700 1 -- 172.21.15.97:0/2405064226 shutdown_connections 2024-04-23T10:25:10.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:10.744+0000 7fd1f968c700 1 --2- 172.21.15.97:0/2405064226 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fd1e00a3cd0 0x7fd1e00a6190 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:10.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:10.744+0000 7fd1f968c700 1 --2- 172.21.15.97:0/2405064226 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1f4107990 0x7fd1f419daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:10.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:10.744+0000 7fd1f968c700 1 --2- 172.21.15.97:0/2405064226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd1f4101940 0x7fd1f4199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:10.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:10.744+0000 7fd1f968c700 1 --2- 172.21.15.97:0/2405064226 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1f4102300 0x7fd1f41997b0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:10.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:10.744+0000 7fd1f968c700 1 -- 172.21.15.97:0/2405064226 >> 172.21.15.97:0/2405064226 conn(0x7fd1f40788f0 msgr2=0x7fd1f406a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:10.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:10.745+0000 7fd1f968c700 1 -- 172.21.15.97:0/2405064226 shutdown_connections 2024-04-23T10:25:10.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:10.745+0000 7fd1f968c700 1 -- 172.21.15.97:0/2405064226 wait complete. 2024-04-23T10:25:11.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:10 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/2405064226' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-23T10:25:11.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:10 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/2405064226' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-23T10:25:11.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:10 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/2405064226' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-23T10:25:11.985 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:11 smithi097 ceph-mon[29197]: pgmap v324: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s; 114 B/s, 1 objects/s recovering 2024-04-23T10:25:12.051 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:11 smithi183 ceph-mon[32317]: pgmap v324: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s; 114 B/s, 1 objects/s recovering 2024-04-23T10:25:12.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:11 smithi196 ceph-mon[32919]: pgmap v324: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s; 114 B/s, 1 objects/s recovering 2024-04-23T10:25:12.345 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738444 got 34359738450 for osd.0 2024-04-23T10:25:12.346 DEBUG:teuthology.parallel:result is None 2024-04-23T10:25:13.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:13.045+0000 7fa14e297700 1 -- 172.21.15.97:0/1185467308 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa148101930 msgr2=0x7fa148101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:13.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:13.046+0000 7fa14e297700 1 --2- 172.21.15.97:0/1185467308 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa148101930 0x7fa148101db0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa13c009990 tx=0x7fa13c009ca0 comp rx=0 tx=0).stop 2024-04-23T10:25:13.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:13.046+0000 7fa14e297700 1 -- 172.21.15.97:0/1185467308 shutdown_connections 2024-04-23T10:25:13.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:13.046+0000 7fa14e297700 1 --2- 172.21.15.97:0/1185467308 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1481022f0 0x7fa14810e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:13.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:13.046+0000 7fa14e297700 1 --2- 172.21.15.97:0/1185467308 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa148107980 0x7fa148107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:13.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:13.046+0000 7fa14e297700 1 --2- 172.21.15.97:0/1185467308 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa148101930 0x7fa148101db0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:13.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:13.046+0000 7fa14e297700 1 -- 172.21.15.97:0/1185467308 >> 172.21.15.97:0/1185467308 conn(0x7fa1480788b0 msgr2=0x7fa148078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:13.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:13.046+0000 7fa14e297700 1 -- 172.21.15.97:0/1185467308 shutdown_connections 2024-04-23T10:25:13.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:13.046+0000 7fa14e297700 1 -- 172.21.15.97:0/1185467308 wait complete. 2024-04-23T10:25:13.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:13.046+0000 7fa14e297700 1 Processor -- start 2024-04-23T10:25:13.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:13.046+0000 7fa14e297700 1 -- start start 2024-04-23T10:25:13.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:13.047+0000 7fa14e297700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa148101930 0x7fa14819b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:13.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:13.047+0000 7fa14e297700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1481022f0 0x7fa14819b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:13.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:13.047+0000 7fa14e297700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa148107980 0x7fa14819fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:13.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:13.047+0000 7fa14e297700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa148111ad0 con 0x7fa148101930 2024-04-23T10:25:13.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:13.047+0000 7fa14e297700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa148111950 con 0x7fa1481022f0 2024-04-23T10:25:13.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:13.047+0000 7fa14e297700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa148111c50 con 0x7fa148107980 2024-04-23T10:25:13.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:13.047+0000 7fa147fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa148101930 0x7fa14819b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:13.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:13.047+0000 7fa147fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa148101930 0x7fa14819b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36402/0 (socket says 172.21.15.97:36402) 2024-04-23T10:25:13.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:13.047+0000 7fa147fff700 1 -- 172.21.15.97:0/4179438535 learned_addr learned my addr 172.21.15.97:0/4179438535 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:25:13.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:13.047+0000 7fa14c834700 1 --2- 172.21.15.97:0/4179438535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa148107980 0x7fa14819fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:13.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:13.047+0000 7fa1477fe700 1 --2- 172.21.15.97:0/4179438535 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1481022f0 0x7fa14819b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:13.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:13.047+0000 7fa147fff700 1 -- 172.21.15.97:0/4179438535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa148107980 msgr2=0x7fa14819fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:13.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:13.047+0000 7fa147fff700 1 --2- 172.21.15.97:0/4179438535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa148107980 0x7fa14819fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:13.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:13.047+0000 7fa147fff700 1 -- 172.21.15.97:0/4179438535 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1481022f0 msgr2=0x7fa14819b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:13.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:13.047+0000 7fa147fff700 1 --2- 172.21.15.97:0/4179438535 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1481022f0 0x7fa14819b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:13.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:13.047+0000 7fa147fff700 1 -- 172.21.15.97:0/4179438535 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1481a0300 con 0x7fa148101930 2024-04-23T10:25:13.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:13.047+0000 7fa147fff700 1 --2- 172.21.15.97:0/4179438535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa148101930 0x7fa14819b340 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa13800c9b0 tx=0x7fa13800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:13.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:13.047+0000 7fa1457fa700 1 -- 172.21.15.97:0/4179438535 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa138007b00 con 0x7fa148101930 2024-04-23T10:25:13.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:13.047+0000 7fa1457fa700 1 -- 172.21.15.97:0/4179438535 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa13800ce50 con 0x7fa148101930 2024-04-23T10:25:13.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:13.047+0000 7fa14e297700 1 -- 172.21.15.97:0/4179438535 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1481a05f0 con 0x7fa148101930 2024-04-23T10:25:13.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:13.048+0000 7fa1457fa700 1 -- 172.21.15.97:0/4179438535 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa138018660 con 0x7fa148101930 2024-04-23T10:25:13.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:13.048+0000 7fa14e297700 1 -- 172.21.15.97:0/4179438535 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1481a7e70 con 0x7fa148101930 2024-04-23T10:25:13.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:13.048+0000 7fa14e297700 1 -- 172.21.15.97:0/4179438535 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa148069730 con 0x7fa148101930 2024-04-23T10:25:13.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:13.051+0000 7fa1457fa700 1 -- 172.21.15.97:0/4179438535 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa13800f450 con 0x7fa148101930 2024-04-23T10:25:13.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:13.051+0000 7fa1457fa700 1 --2- 172.21.15.97:0/4179438535 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fa1300a3cd0 0x7fa1300a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:13.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:13.051+0000 7fa1477fe700 1 --2- 172.21.15.97:0/4179438535 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fa1300a3cd0 0x7fa1300a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:13.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:13.051+0000 7fa1457fa700 1 -- 172.21.15.97:0/4179438535 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fa1380d0670 con 0x7fa148101930 2024-04-23T10:25:13.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:13.052+0000 7fa1477fe700 1 --2- 172.21.15.97:0/4179438535 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fa1300a3cd0 0x7fa1300a6190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa13c0098e0 tx=0x7fa13c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:13.053 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:13.054+0000 7fa1457fa700 1 -- 172.21.15.97:0/4179438535 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa138014030 con 0x7fa148101930 2024-04-23T10:25:13.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:12.942+0000 7fc657048700 1 -- 172.21.15.97:0/2342340957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc650101ba0 msgr2=0x7fc6501076e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:13.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:12.942+0000 7fc657048700 1 --2- 172.21.15.97:0/2342340957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc650101ba0 0x7fc6501076e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc63c0099c0 tx=0x7fc63c009cd0 comp rx=0 tx=0).stop 2024-04-23T10:25:13.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:12.942+0000 7fc657048700 1 -- 172.21.15.97:0/2342340957 shutdown_connections 2024-04-23T10:25:13.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:12.942+0000 7fc657048700 1 --2- 172.21.15.97:0/2342340957 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc650107e10 0x7fc65010f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:13.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:12.942+0000 7fc657048700 1 --2- 172.21.15.97:0/2342340957 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6501011f0 0x7fc6501015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:13.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:12.942+0000 7fc657048700 1 --2- 172.21.15.97:0/2342340957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc650101ba0 0x7fc6501076e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:13.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:12.942+0000 7fc657048700 1 -- 172.21.15.97:0/2342340957 >> 172.21.15.97:0/2342340957 conn(0x7fc650077f40 msgr2=0x7fc6500ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:13.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:12.942+0000 7fc657048700 1 -- 172.21.15.97:0/2342340957 shutdown_connections 2024-04-23T10:25:13.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:12.942+0000 7fc657048700 1 -- 172.21.15.97:0/2342340957 wait complete. 2024-04-23T10:25:13.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:12.943+0000 7fc657048700 1 Processor -- start 2024-04-23T10:25:13.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:12.943+0000 7fc657048700 1 -- start start 2024-04-23T10:25:13.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:12.944+0000 7fc657048700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6501011f0 0x7fc650199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:13.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:12.944+0000 7fc657048700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc650101ba0 0x7fc650199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:13.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:12.944+0000 7fc657048700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc650107e10 0x7fc65019da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:13.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:12.944+0000 7fc657048700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc650112c30 con 0x7fc650107e10 2024-04-23T10:25:13.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:12.944+0000 7fc657048700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc650112ab0 con 0x7fc6501011f0 2024-04-23T10:25:13.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:12.944+0000 7fc657048700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc650112db0 con 0x7fc650101ba0 2024-04-23T10:25:13.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:12.944+0000 7fc6555e5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc650107e10 0x7fc65019da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:13.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:12.944+0000 7fc6555e5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc650107e10 0x7fc65019da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36374/0 (socket says 172.21.15.97:36374) 2024-04-23T10:25:13.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:12.944+0000 7fc6555e5700 1 -- 172.21.15.97:0/462504027 learned_addr learned my addr 172.21.15.97:0/462504027 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:25:13.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:12.944+0000 7fc654de4700 1 --2- 172.21.15.97:0/462504027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6501011f0 0x7fc650199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:13.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:12.944+0000 7fc64ffff700 1 --2- 172.21.15.97:0/462504027 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc650101ba0 0x7fc650199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:13.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:12.944+0000 7fc6555e5700 1 -- 172.21.15.97:0/462504027 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc650101ba0 msgr2=0x7fc650199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:13.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:12.944+0000 7fc6555e5700 1 --2- 172.21.15.97:0/462504027 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc650101ba0 0x7fc650199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:13.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:12.944+0000 7fc6555e5700 1 -- 172.21.15.97:0/462504027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6501011f0 msgr2=0x7fc650199140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:13.194 INFO:teuthology.orchestra.run.smithi097.stdout:77309411399 2024-04-23T10:25:13.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:12.944+0000 7fc6555e5700 1 --2- 172.21.15.97:0/462504027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6501011f0 0x7fc650199140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:13.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:12.944+0000 7fc6555e5700 1 -- 172.21.15.97:0/462504027 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc65019e150 con 0x7fc650107e10 2024-04-23T10:25:13.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:12.944+0000 7fc654de4700 1 --2- 172.21.15.97:0/462504027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6501011f0 0x7fc650199140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T10:25:13.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:12.944+0000 7fc6555e5700 1 --2- 172.21.15.97:0/462504027 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc650107e10 0x7fc65019da10 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc648007ee0 tx=0x7fc64800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:13.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:12.944+0000 7fc64dffb700 1 -- 172.21.15.97:0/462504027 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc64800dd20 con 0x7fc650107e10 2024-04-23T10:25:13.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:12.944+0000 7fc64dffb700 1 -- 172.21.15.97:0/462504027 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc648004020 con 0x7fc650107e10 2024-04-23T10:25:13.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:12.945+0000 7fc64dffb700 1 -- 172.21.15.97:0/462504027 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc648015660 con 0x7fc650107e10 2024-04-23T10:25:13.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:12.945+0000 7fc657048700 1 -- 172.21.15.97:0/462504027 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc65019e440 con 0x7fc650107e10 2024-04-23T10:25:13.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:12.945+0000 7fc657048700 1 -- 172.21.15.97:0/462504027 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6501a5cc0 con 0x7fc650107e10 2024-04-23T10:25:13.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:12.946+0000 7fc657048700 1 -- 172.21.15.97:0/462504027 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc65010b050 con 0x7fc650107e10 2024-04-23T10:25:13.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:12.947+0000 7fc64dffb700 1 -- 172.21.15.97:0/462504027 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc6480041d0 con 0x7fc650107e10 2024-04-23T10:25:13.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:12.947+0000 7fc64dffb700 1 --2- 172.21.15.97:0/462504027 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fc6400a3cd0 0x7fc6400a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:13.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:12.948+0000 7fc654de4700 1 --2- 172.21.15.97:0/462504027 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fc6400a3cd0 0x7fc6400a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:13.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:12.948+0000 7fc64dffb700 1 -- 172.21.15.97:0/462504027 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fc6480cfed0 con 0x7fc650107e10 2024-04-23T10:25:13.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:12.948+0000 7fc654de4700 1 --2- 172.21.15.97:0/462504027 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fc6400a3cd0 0x7fc6400a6190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc644011490 tx=0x7fc644009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:13.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:12.952+0000 7fc64dffb700 1 -- 172.21.15.97:0/462504027 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc648010070 con 0x7fc650107e10 2024-04-23T10:25:13.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:13.163+0000 7fc657048700 1 -- 172.21.15.97:0/462504027 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fc65019a4a0 con 0x7fc650107e10 2024-04-23T10:25:13.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:13.163+0000 7fc64dffb700 1 -- 172.21.15.97:0/462504027 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc64809c760 con 0x7fc650107e10 2024-04-23T10:25:13.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:13.165+0000 7fc657048700 1 -- 172.21.15.97:0/462504027 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fc6400a3cd0 msgr2=0x7fc6400a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:13.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:13.165+0000 7fc657048700 1 --2- 172.21.15.97:0/462504027 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fc6400a3cd0 0x7fc6400a6190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc644011490 tx=0x7fc644009260 comp rx=0 tx=0).stop 2024-04-23T10:25:13.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:13.165+0000 7fc657048700 1 -- 172.21.15.97:0/462504027 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc650107e10 msgr2=0x7fc65019da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:13.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:13.165+0000 7fc657048700 1 --2- 172.21.15.97:0/462504027 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc650107e10 0x7fc65019da10 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc648007ee0 tx=0x7fc64800d3b0 comp rx=0 tx=0).stop 2024-04-23T10:25:13.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:13.165+0000 7fc657048700 1 -- 172.21.15.97:0/462504027 shutdown_connections 2024-04-23T10:25:13.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:13.165+0000 7fc657048700 1 --2- 172.21.15.97:0/462504027 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fc6400a3cd0 0x7fc6400a6190 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:13.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:13.165+0000 7fc657048700 1 --2- 172.21.15.97:0/462504027 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc650107e10 0x7fc65019da10 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:13.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:13.165+0000 7fc657048700 1 --2- 172.21.15.97:0/462504027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc6501011f0 0x7fc650199140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:13.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:13.165+0000 7fc657048700 1 --2- 172.21.15.97:0/462504027 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc650101ba0 0x7fc650199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:13.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:13.165+0000 7fc657048700 1 -- 172.21.15.97:0/462504027 >> 172.21.15.97:0/462504027 conn(0x7fc650077f40 msgr2=0x7fc6500ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:13.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:13.165+0000 7fc657048700 1 -- 172.21.15.97:0/462504027 shutdown_connections 2024-04-23T10:25:13.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:13.165+0000 7fc657048700 1 -- 172.21.15.97:0/462504027 wait complete. 2024-04-23T10:25:13.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:13.267+0000 7fa14e297700 1 -- 172.21.15.97:0/4179438535 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fa1481a80a0 con 0x7fa148101930 2024-04-23T10:25:13.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:13.267+0000 7fa1457fa700 1 -- 172.21.15.97:0/4179438535 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fa13809cfb0 con 0x7fa148101930 2024-04-23T10:25:13.266 INFO:teuthology.orchestra.run.smithi097.stdout:257698037803 2024-04-23T10:25:13.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:13.269+0000 7fa14e297700 1 -- 172.21.15.97:0/4179438535 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fa1300a3cd0 msgr2=0x7fa1300a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:13.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:13.269+0000 7fa14e297700 1 --2- 172.21.15.97:0/4179438535 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fa1300a3cd0 0x7fa1300a6190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa13c0098e0 tx=0x7fa13c005960 comp rx=0 tx=0).stop 2024-04-23T10:25:13.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:13.269+0000 7fa14e297700 1 -- 172.21.15.97:0/4179438535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa148101930 msgr2=0x7fa14819b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:13.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:13.269+0000 7fa14e297700 1 --2- 172.21.15.97:0/4179438535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa148101930 0x7fa14819b340 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa13800c9b0 tx=0x7fa13800ccc0 comp rx=0 tx=0).stop 2024-04-23T10:25:13.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:13.270+0000 7fa14e297700 1 -- 172.21.15.97:0/4179438535 shutdown_connections 2024-04-23T10:25:13.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:13.270+0000 7fa14e297700 1 --2- 172.21.15.97:0/4179438535 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fa1300a3cd0 0x7fa1300a6190 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:13.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:13.270+0000 7fa14e297700 1 --2- 172.21.15.97:0/4179438535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa148107980 0x7fa14819fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:13.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:13.270+0000 7fa14e297700 1 --2- 172.21.15.97:0/4179438535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa148101930 0x7fa14819b340 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:13.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:13.270+0000 7fa14e297700 1 --2- 172.21.15.97:0/4179438535 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1481022f0 0x7fa14819b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:13.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:13.270+0000 7fa14e297700 1 -- 172.21.15.97:0/4179438535 >> 172.21.15.97:0/4179438535 conn(0x7fa1480788b0 msgr2=0x7fa14806b030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:13.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:13.270+0000 7fa14e297700 1 -- 172.21.15.97:0/4179438535 shutdown_connections 2024-04-23T10:25:13.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:13.270+0000 7fa14e297700 1 -- 172.21.15.97:0/4179438535 wait complete. 2024-04-23T10:25:13.326 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:13 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/462504027' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-23T10:25:13.503 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:13 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/462504027' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-23T10:25:13.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:13 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/462504027' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-23T10:25:14.144 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037798 got 257698037803 for osd.8 2024-04-23T10:25:14.145 DEBUG:teuthology.parallel:result is None 2024-04-23T10:25:14.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:14 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/4179438535' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-23T10:25:14.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:14 smithi196 ceph-mon[32919]: pgmap v325: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s; 104 B/s, 0 objects/s recovering 2024-04-23T10:25:14.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:14 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/4179438535' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-23T10:25:14.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:14 smithi097 ceph-mon[29197]: pgmap v325: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s; 104 B/s, 0 objects/s recovering 2024-04-23T10:25:14.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:14 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/4179438535' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-23T10:25:14.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:14 smithi183 ceph-mon[32317]: pgmap v325: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s; 104 B/s, 0 objects/s recovering 2024-04-23T10:25:16.754 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:16 smithi097 ceph-mon[29197]: pgmap v326: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 op/s; 48 B/s, 0 objects/s recovering 2024-04-23T10:25:16.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:16.863+0000 7fcc3b7f5700 1 -- 172.21.15.97:0/352688544 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc34101400 msgr2=0x7fcc34101880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:16.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:16.863+0000 7fcc3b7f5700 1 --2- 172.21.15.97:0/352688544 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc34101400 0x7fcc34101880 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fcc24009990 tx=0x7fcc24009ca0 comp rx=0 tx=0).stop 2024-04-23T10:25:16.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:16.863+0000 7fcc3b7f5700 1 -- 172.21.15.97:0/352688544 shutdown_connections 2024-04-23T10:25:16.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:16.863+0000 7fcc3b7f5700 1 --2- 172.21.15.97:0/352688544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc34101dc0 0x7fcc3410f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:16.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:16.863+0000 7fcc3b7f5700 1 --2- 172.21.15.97:0/352688544 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc34107b20 0x7fcc34107f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:16.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:16.863+0000 7fcc3b7f5700 1 --2- 172.21.15.97:0/352688544 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc34101400 0x7fcc34101880 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:16.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:16.863+0000 7fcc3b7f5700 1 -- 172.21.15.97:0/352688544 >> 172.21.15.97:0/352688544 conn(0x7fcc34076c60 msgr2=0x7fcc34077070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:16.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:16.864+0000 7fcc3b7f5700 1 -- 172.21.15.97:0/352688544 shutdown_connections 2024-04-23T10:25:16.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:16.864+0000 7fcc3b7f5700 1 -- 172.21.15.97:0/352688544 wait complete. 2024-04-23T10:25:16.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:16.864+0000 7fcc3b7f5700 1 Processor -- start 2024-04-23T10:25:16.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:16.864+0000 7fcc3b7f5700 1 -- start start 2024-04-23T10:25:16.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:16.864+0000 7fcc3b7f5700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc34101400 0x7fcc341025e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:16.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:16.864+0000 7fcc3b7f5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc34101dc0 0x7fcc34102b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:16.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:16.864+0000 7fcc3b7f5700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc34107b20 0x7fcc34103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:16.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:16.864+0000 7fcc3b7f5700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcc34112cb0 con 0x7fcc34101dc0 2024-04-23T10:25:16.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:16.864+0000 7fcc3b7f5700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcc34112b30 con 0x7fcc34107b20 2024-04-23T10:25:16.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:16.864+0000 7fcc3b7f5700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcc34112e30 con 0x7fcc34101400 2024-04-23T10:25:16.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:16.864+0000 7fcc38d90700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc34101dc0 0x7fcc34102b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:16.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:16.864+0000 7fcc38d90700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc34101dc0 0x7fcc34102b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33482/0 (socket says 172.21.15.97:33482) 2024-04-23T10:25:16.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:16.864+0000 7fcc38d90700 1 -- 172.21.15.97:0/2508286312 learned_addr learned my addr 172.21.15.97:0/2508286312 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:25:16.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:16.865+0000 7fcc39591700 1 --2- 172.21.15.97:0/2508286312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc34101400 0x7fcc341025e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:16.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:16.865+0000 7fcc38d90700 1 -- 172.21.15.97:0/2508286312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc34101400 msgr2=0x7fcc341025e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:16.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:16.865+0000 7fcc38d90700 1 --2- 172.21.15.97:0/2508286312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc34101400 0x7fcc341025e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:16.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:16.865+0000 7fcc39d92700 1 --2- 172.21.15.97:0/2508286312 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc34107b20 0x7fcc34103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:16.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:16.865+0000 7fcc38d90700 1 -- 172.21.15.97:0/2508286312 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc34107b20 msgr2=0x7fcc34103060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:16.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:16.865+0000 7fcc38d90700 1 --2- 172.21.15.97:0/2508286312 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc34107b20 0x7fcc34103060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:16.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:16.865+0000 7fcc38d90700 1 -- 172.21.15.97:0/2508286312 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc341a76d0 con 0x7fcc34101dc0 2024-04-23T10:25:16.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:16.865+0000 7fcc39591700 1 --2- 172.21.15.97:0/2508286312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc34101400 0x7fcc341025e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:25:16.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:16.865+0000 7fcc39d92700 1 --2- 172.21.15.97:0/2508286312 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc34107b20 0x7fcc34103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:25:16.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:16.865+0000 7fcc38d90700 1 --2- 172.21.15.97:0/2508286312 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc34101dc0 0x7fcc34102b20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fcc240057f0 tx=0x7fcc24003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:16.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:16.865+0000 7fcc2a7fc700 1 -- 172.21.15.97:0/2508286312 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc24027070 con 0x7fcc34101dc0 2024-04-23T10:25:16.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:16.865+0000 7fcc3b7f5700 1 -- 172.21.15.97:0/2508286312 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc341a7960 con 0x7fcc34101dc0 2024-04-23T10:25:16.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:16.865+0000 7fcc2a7fc700 1 -- 172.21.15.97:0/2508286312 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc240041c0 con 0x7fcc34101dc0 2024-04-23T10:25:16.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:16.865+0000 7fcc2a7fc700 1 -- 172.21.15.97:0/2508286312 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc24022e60 con 0x7fcc34101dc0 2024-04-23T10:25:16.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:16.865+0000 7fcc3b7f5700 1 -- 172.21.15.97:0/2508286312 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc341a7e40 con 0x7fcc34101dc0 2024-04-23T10:25:16.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:16.866+0000 7fcc3b7f5700 1 -- 172.21.15.97:0/2508286312 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc34069730 con 0x7fcc34101dc0 2024-04-23T10:25:16.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:16.868+0000 7fcc2a7fc700 1 -- 172.21.15.97:0/2508286312 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fcc2402a070 con 0x7fcc34101dc0 2024-04-23T10:25:16.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:16.868+0000 7fcc2a7fc700 1 --2- 172.21.15.97:0/2508286312 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fcc200a3cd0 0x7fcc200a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:16.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:16.868+0000 7fcc39591700 1 --2- 172.21.15.97:0/2508286312 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fcc200a3cd0 0x7fcc200a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:16.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:16.869+0000 7fcc2a7fc700 1 -- 172.21.15.97:0/2508286312 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fcc240d1830 con 0x7fcc34101dc0 2024-04-23T10:25:16.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:16.869+0000 7fcc39591700 1 --2- 172.21.15.97:0/2508286312 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fcc200a3cd0 0x7fcc200a6190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fcc340781e0 tx=0x7fcc30008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:16.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:16.872+0000 7fcc2a7fc700 1 -- 172.21.15.97:0/2508286312 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcc24013070 con 0x7fcc34101dc0 2024-04-23T10:25:16.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:16 smithi183 ceph-mon[32317]: pgmap v326: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 op/s; 48 B/s, 0 objects/s recovering 2024-04-23T10:25:17.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:17.072+0000 7fcc3b7f5700 1 -- 172.21.15.97:0/2508286312 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fcc3402e140 con 0x7fcc34101dc0 2024-04-23T10:25:17.071 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:17.072+0000 7fcc2a7fc700 1 -- 172.21.15.97:0/2508286312 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fcc2409e170 con 0x7fcc34101dc0 2024-04-23T10:25:17.071 INFO:teuthology.orchestra.run.smithi097.stdout:292057776165 2024-04-23T10:25:17.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:17.074+0000 7fcc3b7f5700 1 -- 172.21.15.97:0/2508286312 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fcc200a3cd0 msgr2=0x7fcc200a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:17.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:17.074+0000 7fcc3b7f5700 1 --2- 172.21.15.97:0/2508286312 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fcc200a3cd0 0x7fcc200a6190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fcc340781e0 tx=0x7fcc30008040 comp rx=0 tx=0).stop 2024-04-23T10:25:17.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:17.074+0000 7fcc3b7f5700 1 -- 172.21.15.97:0/2508286312 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc34101dc0 msgr2=0x7fcc34102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:17.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:17.074+0000 7fcc3b7f5700 1 --2- 172.21.15.97:0/2508286312 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc34101dc0 0x7fcc34102b20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fcc240057f0 tx=0x7fcc24003730 comp rx=0 tx=0).stop 2024-04-23T10:25:17.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:17.074+0000 7fcc3b7f5700 1 -- 172.21.15.97:0/2508286312 shutdown_connections 2024-04-23T10:25:17.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:17.074+0000 7fcc3b7f5700 1 --2- 172.21.15.97:0/2508286312 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fcc200a3cd0 0x7fcc200a6190 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:17.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:17.074+0000 7fcc3b7f5700 1 --2- 172.21.15.97:0/2508286312 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc34107b20 0x7fcc34103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:17.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:17.074+0000 7fcc3b7f5700 1 --2- 172.21.15.97:0/2508286312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc34101400 0x7fcc341025e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:17.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:17.074+0000 7fcc3b7f5700 1 --2- 172.21.15.97:0/2508286312 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcc34101dc0 0x7fcc34102b20 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:17.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:17.074+0000 7fcc3b7f5700 1 -- 172.21.15.97:0/2508286312 >> 172.21.15.97:0/2508286312 conn(0x7fcc34076c60 msgr2=0x7fcc340ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:17.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:17.074+0000 7fcc3b7f5700 1 -- 172.21.15.97:0/2508286312 shutdown_connections 2024-04-23T10:25:17.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:17.074+0000 7fcc3b7f5700 1 -- 172.21.15.97:0/2508286312 wait complete. 2024-04-23T10:25:17.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:16 smithi196 ceph-mon[32919]: pgmap v326: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 op/s; 48 B/s, 0 objects/s recovering 2024-04-23T10:25:17.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:17 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/2508286312' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-23T10:25:18.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:17 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/2508286312' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-23T10:25:18.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:17 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/2508286312' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-23T10:25:18.165 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411393 got 77309411399 for osd.2 2024-04-23T10:25:18.165 DEBUG:teuthology.parallel:result is None 2024-04-23T10:25:18.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.338+0000 7fa96481f700 1 -- 172.21.15.97:0/1923691538 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa960108a30 msgr2=0x7fa960108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:18.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.338+0000 7fa96481f700 1 --2- 172.21.15.97:0/1923691538 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa960108a30 0x7fa960108e10 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa954009990 tx=0x7fa954009ca0 comp rx=0 tx=0).stop 2024-04-23T10:25:18.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.338+0000 7fa96481f700 1 -- 172.21.15.97:0/1923691538 shutdown_connections 2024-04-23T10:25:18.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.338+0000 7fa96481f700 1 --2- 172.21.15.97:0/1923691538 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa96006b0f0 0x7fa96010f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:18.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.338+0000 7fa96481f700 1 --2- 172.21.15.97:0/1923691538 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa960108a30 0x7fa960108e10 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:18.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.338+0000 7fa96481f700 1 --2- 172.21.15.97:0/1923691538 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa96006a750 0x7fa96006abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:18.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.338+0000 7fa96481f700 1 -- 172.21.15.97:0/1923691538 >> 172.21.15.97:0/1923691538 conn(0x7fa9600788f0 msgr2=0x7fa960078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:18.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.338+0000 7fa96481f700 1 -- 172.21.15.97:0/1923691538 shutdown_connections 2024-04-23T10:25:18.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.338+0000 7fa96481f700 1 -- 172.21.15.97:0/1923691538 wait complete. 2024-04-23T10:25:18.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.339+0000 7fa96481f700 1 Processor -- start 2024-04-23T10:25:18.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.339+0000 7fa96481f700 1 -- start start 2024-04-23T10:25:18.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.339+0000 7fa96481f700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa96006a750 0x7fa960199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:18.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.339+0000 7fa96481f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa96006b0f0 0x7fa960199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:18.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.339+0000 7fa96481f700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa960108a30 0x7fa96019dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:18.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.339+0000 7fa96481f700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa960112d00 con 0x7fa96006b0f0 2024-04-23T10:25:18.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.339+0000 7fa96481f700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa960112b80 con 0x7fa960108a30 2024-04-23T10:25:18.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.339+0000 7fa96481f700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa960112e80 con 0x7fa96006a750 2024-04-23T10:25:18.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.339+0000 7fa95dd9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa96006b0f0 0x7fa960199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:18.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.339+0000 7fa95dd9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa96006b0f0 0x7fa960199840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33502/0 (socket says 172.21.15.97:33502) 2024-04-23T10:25:18.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.339+0000 7fa95dd9b700 1 -- 172.21.15.97:0/1853575520 learned_addr learned my addr 172.21.15.97:0/1853575520 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:25:18.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.340+0000 7fa95e59c700 1 --2- 172.21.15.97:0/1853575520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa96006a750 0x7fa960199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:18.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.340+0000 7fa95ed9d700 1 --2- 172.21.15.97:0/1853575520 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa960108a30 0x7fa96019dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:18.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.340+0000 7fa95dd9b700 1 -- 172.21.15.97:0/1853575520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa96006a750 msgr2=0x7fa960199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:18.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.340+0000 7fa95dd9b700 1 --2- 172.21.15.97:0/1853575520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa96006a750 0x7fa960199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:18.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.340+0000 7fa95dd9b700 1 -- 172.21.15.97:0/1853575520 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa960108a30 msgr2=0x7fa96019dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:18.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.340+0000 7fa95dd9b700 1 --2- 172.21.15.97:0/1853575520 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa960108a30 0x7fa96019dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:18.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.340+0000 7fa95dd9b700 1 -- 172.21.15.97:0/1853575520 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa96019e310 con 0x7fa96006b0f0 2024-04-23T10:25:18.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.340+0000 7fa95ed9d700 1 --2- 172.21.15.97:0/1853575520 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa960108a30 0x7fa96019dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:25:18.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.340+0000 7fa95e59c700 1 --2- 172.21.15.97:0/1853575520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa96006a750 0x7fa960199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:25:18.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.340+0000 7fa95dd9b700 1 --2- 172.21.15.97:0/1853575520 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa96006b0f0 0x7fa960199840 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa95000b530 tx=0x7fa95000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:18.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.340+0000 7fa94f7fe700 1 -- 172.21.15.97:0/1853575520 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa950015020 con 0x7fa96006b0f0 2024-04-23T10:25:18.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.340+0000 7fa94f7fe700 1 -- 172.21.15.97:0/1853575520 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa950011ce0 con 0x7fa96006b0f0 2024-04-23T10:25:18.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.340+0000 7fa96481f700 1 -- 172.21.15.97:0/1853575520 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa96019e600 con 0x7fa96006b0f0 2024-04-23T10:25:18.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.340+0000 7fa94f7fe700 1 -- 172.21.15.97:0/1853575520 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa950010d20 con 0x7fa96006b0f0 2024-04-23T10:25:18.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.340+0000 7fa96481f700 1 -- 172.21.15.97:0/1853575520 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9601a5e80 con 0x7fa96006b0f0 2024-04-23T10:25:18.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.341+0000 7fa96481f700 1 -- 172.21.15.97:0/1853575520 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa960069730 con 0x7fa96006b0f0 2024-04-23T10:25:18.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.342+0000 7fa94f7fe700 1 -- 172.21.15.97:0/1853575520 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa950018020 con 0x7fa96006b0f0 2024-04-23T10:25:18.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.343+0000 7fa94f7fe700 1 --2- 172.21.15.97:0/1853575520 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fa9480a3d50 0x7fa9480a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:18.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.343+0000 7fa95e59c700 1 --2- 172.21.15.97:0/1853575520 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fa9480a3d50 0x7fa9480a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:18.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.343+0000 7fa94f7fe700 1 -- 172.21.15.97:0/1853575520 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fa9500d0170 con 0x7fa96006b0f0 2024-04-23T10:25:18.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.343+0000 7fa95e59c700 1 --2- 172.21.15.97:0/1853575520 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fa9480a3d50 0x7fa9480a6210 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa95400b5c0 tx=0x7fa954005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:18.345 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.346+0000 7fa94f7fe700 1 -- 172.21.15.97:0/1853575520 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa95001f020 con 0x7fa96006b0f0 2024-04-23T10:25:18.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.549+0000 7fa96481f700 1 -- 172.21.15.97:0/1853575520 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fa96002df80 con 0x7fa96006b0f0 2024-04-23T10:25:18.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.549+0000 7fa94f7fe700 1 -- 172.21.15.97:0/1853575520 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fa95009cb30 con 0x7fa96006b0f0 2024-04-23T10:25:18.549 INFO:teuthology.orchestra.run.smithi097.stdout:146028888128 2024-04-23T10:25:18.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.553+0000 7fa96481f700 1 -- 172.21.15.97:0/1853575520 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fa9480a3d50 msgr2=0x7fa9480a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:18.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.553+0000 7fa96481f700 1 --2- 172.21.15.97:0/1853575520 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fa9480a3d50 0x7fa9480a6210 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa95400b5c0 tx=0x7fa954005fb0 comp rx=0 tx=0).stop 2024-04-23T10:25:18.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.553+0000 7fa96481f700 1 -- 172.21.15.97:0/1853575520 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa96006b0f0 msgr2=0x7fa960199840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:18.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.553+0000 7fa96481f700 1 --2- 172.21.15.97:0/1853575520 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa96006b0f0 0x7fa960199840 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa95000b530 tx=0x7fa95000b8f0 comp rx=0 tx=0).stop 2024-04-23T10:25:18.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.553+0000 7fa96481f700 1 -- 172.21.15.97:0/1853575520 shutdown_connections 2024-04-23T10:25:18.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.553+0000 7fa96481f700 1 --2- 172.21.15.97:0/1853575520 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fa9480a3d50 0x7fa9480a6210 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:18.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.553+0000 7fa96481f700 1 --2- 172.21.15.97:0/1853575520 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa960108a30 0x7fa96019dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:18.553 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.553+0000 7fa96481f700 1 --2- 172.21.15.97:0/1853575520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa96006a750 0x7fa960199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:18.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.553+0000 7fa96481f700 1 --2- 172.21.15.97:0/1853575520 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa96006b0f0 0x7fa960199840 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:18.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.553+0000 7fa96481f700 1 -- 172.21.15.97:0/1853575520 >> 172.21.15.97:0/1853575520 conn(0x7fa9600788f0 msgr2=0x7fa9600ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:18.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.553+0000 7fa96481f700 1 -- 172.21.15.97:0/1853575520 shutdown_connections 2024-04-23T10:25:18.554 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.553+0000 7fa96481f700 1 -- 172.21.15.97:0/1853575520 wait complete. 2024-04-23T10:25:18.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.574+0000 7febc11e2700 1 -- 172.21.15.97:0/1665998648 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febbc1022f0 msgr2=0x7febbc10e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:18.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.574+0000 7febc11e2700 1 --2- 172.21.15.97:0/1665998648 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febbc1022f0 0x7febbc10e7e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7febb400b0a0 tx=0x7febb400b3b0 comp rx=0 tx=0).stop 2024-04-23T10:25:18.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.574+0000 7febc11e2700 1 -- 172.21.15.97:0/1665998648 shutdown_connections 2024-04-23T10:25:18.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.574+0000 7febc11e2700 1 --2- 172.21.15.97:0/1665998648 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febbc1022f0 0x7febbc10e7e0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:18.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.574+0000 7febc11e2700 1 --2- 172.21.15.97:0/1665998648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7febbc107980 0x7febbc107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:18.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.574+0000 7febc11e2700 1 --2- 172.21.15.97:0/1665998648 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febbc101930 0x7febbc101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:18.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.574+0000 7febc11e2700 1 -- 172.21.15.97:0/1665998648 >> 172.21.15.97:0/1665998648 conn(0x7febbc0788b0 msgr2=0x7febbc078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:18.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.574+0000 7febc11e2700 1 -- 172.21.15.97:0/1665998648 shutdown_connections 2024-04-23T10:25:18.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.574+0000 7febc11e2700 1 -- 172.21.15.97:0/1665998648 wait complete. 2024-04-23T10:25:18.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.575+0000 7febc11e2700 1 Processor -- start 2024-04-23T10:25:18.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.575+0000 7febc11e2700 1 -- start start 2024-04-23T10:25:18.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.576+0000 7febc11e2700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febbc101930 0x7febbc1034e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:18.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.576+0000 7febc11e2700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febbc1022f0 0x7febbc103a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:18.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.576+0000 7febc11e2700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7febbc107980 0x7febbc103f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:18.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.576+0000 7febc11e2700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7febbc111af0 con 0x7febbc1022f0 2024-04-23T10:25:18.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.576+0000 7febc11e2700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7febbc111970 con 0x7febbc101930 2024-04-23T10:25:18.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.576+0000 7febc11e2700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7febbc111c70 con 0x7febbc107980 2024-04-23T10:25:18.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.576+0000 7febba59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febbc1022f0 0x7febbc103a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:18.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.576+0000 7febba59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febbc1022f0 0x7febbc103a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33512/0 (socket says 172.21.15.97:33512) 2024-04-23T10:25:18.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.576+0000 7febba59c700 1 -- 172.21.15.97:0/3216926814 learned_addr learned my addr 172.21.15.97:0/3216926814 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:25:18.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.576+0000 7febbad9d700 1 --2- 172.21.15.97:0/3216926814 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febbc101930 0x7febbc1034e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:18.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.576+0000 7febbb59e700 1 --2- 172.21.15.97:0/3216926814 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7febbc107980 0x7febbc103f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:18.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.576+0000 7febba59c700 1 -- 172.21.15.97:0/3216926814 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7febbc107980 msgr2=0x7febbc103f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:18.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.576+0000 7febba59c700 1 --2- 172.21.15.97:0/3216926814 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7febbc107980 0x7febbc103f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:18.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.576+0000 7febba59c700 1 -- 172.21.15.97:0/3216926814 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febbc101930 msgr2=0x7febbc1034e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:18.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.576+0000 7febba59c700 1 --2- 172.21.15.97:0/3216926814 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febbc101930 0x7febbc1034e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:18.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.576+0000 7febba59c700 1 -- 172.21.15.97:0/3216926814 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febbc1a7600 con 0x7febbc1022f0 2024-04-23T10:25:18.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.576+0000 7febba59c700 1 --2- 172.21.15.97:0/3216926814 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febbc1022f0 0x7febbc103a20 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7febb000d980 tx=0x7febb000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:18.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.577+0000 7feba3fff700 1 -- 172.21.15.97:0/3216926814 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb0016070 con 0x7febbc1022f0 2024-04-23T10:25:18.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.577+0000 7feba3fff700 1 -- 172.21.15.97:0/3216926814 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febb0004830 con 0x7febbc1022f0 2024-04-23T10:25:18.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.577+0000 7febc11e2700 1 -- 172.21.15.97:0/3216926814 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febbc1a78f0 con 0x7febbc1022f0 2024-04-23T10:25:18.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.577+0000 7febc11e2700 1 -- 172.21.15.97:0/3216926814 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febbc1a7e80 con 0x7febbc1022f0 2024-04-23T10:25:18.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.578+0000 7feba3fff700 1 -- 172.21.15.97:0/3216926814 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb0005540 con 0x7febbc1022f0 2024-04-23T10:25:18.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.578+0000 7febc11e2700 1 -- 172.21.15.97:0/3216926814 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feba8005330 con 0x7febbc1022f0 2024-04-23T10:25:18.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.580+0000 7feba3fff700 1 -- 172.21.15.97:0/3216926814 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7febb0004220 con 0x7febbc1022f0 2024-04-23T10:25:18.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.580+0000 7feba3fff700 1 --2- 172.21.15.97:0/3216926814 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7feba40a3d50 0x7feba40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:18.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.580+0000 7febbad9d700 1 --2- 172.21.15.97:0/3216926814 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7feba40a3d50 0x7feba40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:18.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.580+0000 7feba3fff700 1 -- 172.21.15.97:0/3216926814 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7febb00d0160 con 0x7febbc1022f0 2024-04-23T10:25:18.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.581+0000 7febbad9d700 1 --2- 172.21.15.97:0/3216926814 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7feba40a3d50 0x7feba40a6210 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7febac009be0 tx=0x7febac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:18.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.585+0000 7feba3fff700 1 -- 172.21.15.97:0/3216926814 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7febb001f020 con 0x7febbc1022f0 2024-04-23T10:25:18.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.792+0000 7febc11e2700 1 -- 172.21.15.97:0/3216926814 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7feba80058f0 con 0x7febbc1022f0 2024-04-23T10:25:18.791 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:18 smithi097 ceph-mon[29197]: pgmap v327: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-04-23T10:25:18.791 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:18 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/1853575520' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-23T10:25:18.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.792+0000 7feba3fff700 1 -- 172.21.15.97:0/3216926814 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7febb009ca10 con 0x7febbc1022f0 2024-04-23T10:25:18.792 INFO:teuthology.orchestra.run.smithi097.stdout:111669149765 2024-04-23T10:25:18.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.794+0000 7febc11e2700 1 -- 172.21.15.97:0/3216926814 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7feba40a3d50 msgr2=0x7feba40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:18.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.794+0000 7febc11e2700 1 --2- 172.21.15.97:0/3216926814 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7feba40a3d50 0x7feba40a6210 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7febac009be0 tx=0x7febac009260 comp rx=0 tx=0).stop 2024-04-23T10:25:18.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.794+0000 7febc11e2700 1 -- 172.21.15.97:0/3216926814 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febbc1022f0 msgr2=0x7febbc103a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:18.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.794+0000 7febc11e2700 1 --2- 172.21.15.97:0/3216926814 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febbc1022f0 0x7febbc103a20 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7febb000d980 tx=0x7febb000dc90 comp rx=0 tx=0).stop 2024-04-23T10:25:18.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.794+0000 7febc11e2700 1 -- 172.21.15.97:0/3216926814 shutdown_connections 2024-04-23T10:25:18.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.794+0000 7febc11e2700 1 --2- 172.21.15.97:0/3216926814 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7feba40a3d50 0x7feba40a6210 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:18.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.794+0000 7febc11e2700 1 --2- 172.21.15.97:0/3216926814 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7febbc107980 0x7febbc103f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:18.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.794+0000 7febc11e2700 1 --2- 172.21.15.97:0/3216926814 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febbc101930 0x7febbc1034e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:18.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.794+0000 7febc11e2700 1 --2- 172.21.15.97:0/3216926814 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febbc1022f0 0x7febbc103a20 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:18.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.794+0000 7febc11e2700 1 -- 172.21.15.97:0/3216926814 >> 172.21.15.97:0/3216926814 conn(0x7febbc0788b0 msgr2=0x7febbc06b330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:18.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.795+0000 7febc11e2700 1 -- 172.21.15.97:0/3216926814 shutdown_connections 2024-04-23T10:25:18.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:18.795+0000 7febc11e2700 1 -- 172.21.15.97:0/3216926814 wait complete. 2024-04-23T10:25:18.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:18 smithi183 ceph-mon[32317]: pgmap v327: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-04-23T10:25:18.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:18 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/1853575520' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-23T10:25:19.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:18 smithi196 ceph-mon[32919]: pgmap v327: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-04-23T10:25:19.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:18 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/1853575520' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-23T10:25:19.699 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776159 got 292057776165 for osd.9 2024-04-23T10:25:19.699 DEBUG:teuthology.parallel:result is None 2024-04-23T10:25:19.855 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:19 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/3216926814' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-23T10:25:19.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:19 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/3216926814' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-23T10:25:20.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:19 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/3216926814' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-23T10:25:20.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:20.201+0000 7fe062973700 1 -- 172.21.15.97:0/3551029831 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe05c103450 msgr2=0x7fe05c10f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:20.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:20.201+0000 7fe062973700 1 --2- 172.21.15.97:0/3551029831 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe05c103450 0x7fe05c10f8b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe05400b0a0 tx=0x7fe05400b3b0 comp rx=0 tx=0).stop 2024-04-23T10:25:20.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:20.201+0000 7fe062973700 1 -- 172.21.15.97:0/3551029831 shutdown_connections 2024-04-23T10:25:20.200 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:20.201+0000 7fe062973700 1 --2- 172.21.15.97:0/3551029831 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe05c103450 0x7fe05c10f8b0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:20.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:20.201+0000 7fe062973700 1 --2- 172.21.15.97:0/3551029831 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe05c108a50 0x7fe05c108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:20.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:20.201+0000 7fe062973700 1 --2- 172.21.15.97:0/3551029831 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe05c102a90 0x7fe05c102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:20.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:20.201+0000 7fe062973700 1 -- 172.21.15.97:0/3551029831 >> 172.21.15.97:0/3551029831 conn(0x7fe05c079ba0 msgr2=0x7fe05c079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:20.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:20.201+0000 7fe062973700 1 -- 172.21.15.97:0/3551029831 shutdown_connections 2024-04-23T10:25:20.201 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:20.201+0000 7fe062973700 1 -- 172.21.15.97:0/3551029831 wait complete. 2024-04-23T10:25:20.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:20.202+0000 7fe062973700 1 Processor -- start 2024-04-23T10:25:20.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:20.202+0000 7fe062973700 1 -- start start 2024-04-23T10:25:20.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:20.202+0000 7fe062973700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe05c102a90 0x7fe05c198f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:20.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:20.202+0000 7fe062973700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe05c103450 0x7fe05c1994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:20.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:20.202+0000 7fe062973700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe05c108a50 0x7fe05c19d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:20.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:20.202+0000 7fe062973700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe05c112be0 con 0x7fe05c103450 2024-04-23T10:25:20.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:20.202+0000 7fe062973700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe05c112a60 con 0x7fe05c102a90 2024-04-23T10:25:20.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:20.202+0000 7fe062973700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe05c112d60 con 0x7fe05c108a50 2024-04-23T10:25:20.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:20.202+0000 7fe05b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe05c103450 0x7fe05c1994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:20.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:20.202+0000 7fe05b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe05c103450 0x7fe05c1994d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33518/0 (socket says 172.21.15.97:33518) 2024-04-23T10:25:20.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:20.202+0000 7fe05b7fe700 1 -- 172.21.15.97:0/4291649175 learned_addr learned my addr 172.21.15.97:0/4291649175 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:25:20.203 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:20.202+0000 7fe05bfff700 1 --2- 172.21.15.97:0/4291649175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe05c102a90 0x7fe05c198f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:20.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:20.203+0000 7fe05b7fe700 1 -- 172.21.15.97:0/4291649175 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe05c108a50 msgr2=0x7fe05c19d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:20.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:20.203+0000 7fe05b7fe700 1 --2- 172.21.15.97:0/4291649175 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe05c108a50 0x7fe05c19d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:20.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:20.203+0000 7fe05b7fe700 1 -- 172.21.15.97:0/4291649175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe05c102a90 msgr2=0x7fe05c198f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:20.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:20.203+0000 7fe05b7fe700 1 --2- 172.21.15.97:0/4291649175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe05c102a90 0x7fe05c198f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:20.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:20.203+0000 7fe05b7fe700 1 -- 172.21.15.97:0/4291649175 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe05c19df50 con 0x7fe05c103450 2024-04-23T10:25:20.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:20.203+0000 7fe05bfff700 1 --2- 172.21.15.97:0/4291649175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe05c102a90 0x7fe05c198f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:25:20.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:20.203+0000 7fe05b7fe700 1 --2- 172.21.15.97:0/4291649175 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe05c103450 0x7fe05c1994d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe04c00c9b0 tx=0x7fe04c00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:20.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:20.203+0000 7fe0597fa700 1 -- 172.21.15.97:0/4291649175 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe04c007b00 con 0x7fe05c103450 2024-04-23T10:25:20.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:20.203+0000 7fe0597fa700 1 -- 172.21.15.97:0/4291649175 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe04c007ca0 con 0x7fe05c103450 2024-04-23T10:25:20.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:20.203+0000 7fe062973700 1 -- 172.21.15.97:0/4291649175 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe05c19e240 con 0x7fe05c103450 2024-04-23T10:25:20.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:20.203+0000 7fe0597fa700 1 -- 172.21.15.97:0/4291649175 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe04c018730 con 0x7fe05c103450 2024-04-23T10:25:20.204 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:20.203+0000 7fe062973700 1 -- 172.21.15.97:0/4291649175 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe05c1a5ac0 con 0x7fe05c103450 2024-04-23T10:25:20.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:20.204+0000 7fe062973700 1 -- 172.21.15.97:0/4291649175 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe05c069730 con 0x7fe05c103450 2024-04-23T10:25:20.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:20.205+0000 7fe0597fa700 1 -- 172.21.15.97:0/4291649175 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe04c00f450 con 0x7fe05c103450 2024-04-23T10:25:20.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:20.206+0000 7fe0597fa700 1 --2- 172.21.15.97:0/4291649175 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fe0440a3cd0 0x7fe0440a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:20.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:20.206+0000 7fe05bfff700 1 --2- 172.21.15.97:0/4291649175 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fe0440a3cd0 0x7fe0440a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:20.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:20.206+0000 7fe0597fa700 1 -- 172.21.15.97:0/4291649175 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fe04c0d05f0 con 0x7fe05c103450 2024-04-23T10:25:20.205 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:20.206+0000 7fe05bfff700 1 --2- 172.21.15.97:0/4291649175 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fe0440a3cd0 0x7fe0440a6190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe050005fd0 tx=0x7fe05000a5e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:20.209 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:20.210+0000 7fe0597fa700 1 -- 172.21.15.97:0/4291649175 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe04c014030 con 0x7fe05c103450 2024-04-23T10:25:20.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:20.429+0000 7fe062973700 1 -- 172.21.15.97:0/4291649175 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fe05c02df80 con 0x7fe05c103450 2024-04-23T10:25:20.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:20.429+0000 7fe0597fa700 1 -- 172.21.15.97:0/4291649175 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe04c09cea0 con 0x7fe05c103450 2024-04-23T10:25:20.428 INFO:teuthology.orchestra.run.smithi097.stdout:201863462966 2024-04-23T10:25:20.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:20.431+0000 7fe062973700 1 -- 172.21.15.97:0/4291649175 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fe0440a3cd0 msgr2=0x7fe0440a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:20.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:20.431+0000 7fe062973700 1 --2- 172.21.15.97:0/4291649175 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fe0440a3cd0 0x7fe0440a6190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe050005fd0 tx=0x7fe05000a5e0 comp rx=0 tx=0).stop 2024-04-23T10:25:20.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:20.431+0000 7fe062973700 1 -- 172.21.15.97:0/4291649175 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe05c103450 msgr2=0x7fe05c1994d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:20.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:20.431+0000 7fe062973700 1 --2- 172.21.15.97:0/4291649175 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe05c103450 0x7fe05c1994d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe04c00c9b0 tx=0x7fe04c00cd70 comp rx=0 tx=0).stop 2024-04-23T10:25:20.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:20.431+0000 7fe062973700 1 -- 172.21.15.97:0/4291649175 shutdown_connections 2024-04-23T10:25:20.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:20.431+0000 7fe062973700 1 --2- 172.21.15.97:0/4291649175 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fe0440a3cd0 0x7fe0440a6190 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:20.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:20.431+0000 7fe062973700 1 --2- 172.21.15.97:0/4291649175 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe05c108a50 0x7fe05c19d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:20.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:20.431+0000 7fe062973700 1 --2- 172.21.15.97:0/4291649175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe05c102a90 0x7fe05c198f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:20.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:20.431+0000 7fe062973700 1 --2- 172.21.15.97:0/4291649175 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe05c103450 0x7fe05c1994d0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:20.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:20.431+0000 7fe062973700 1 -- 172.21.15.97:0/4291649175 >> 172.21.15.97:0/4291649175 conn(0x7fe05c079ba0 msgr2=0x7fe05c101580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:20.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:20.431+0000 7fe062973700 1 -- 172.21.15.97:0/4291649175 shutdown_connections 2024-04-23T10:25:20.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:20.431+0000 7fe062973700 1 -- 172.21.15.97:0/4291649175 wait complete. 2024-04-23T10:25:20.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:20 smithi097 ceph-mon[29197]: pgmap v328: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-23T10:25:20.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:20 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/4291649175' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-23T10:25:21.052 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888123 got 146028888128 for osd.4 2024-04-23T10:25:21.052 DEBUG:teuthology.parallel:result is None 2024-04-23T10:25:21.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:20 smithi196 ceph-mon[32919]: pgmap v328: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-23T10:25:21.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:20 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/4291649175' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-23T10:25:21.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:20 smithi183 ceph-mon[32317]: pgmap v328: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-23T10:25:21.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:20 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/4291649175' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-23T10:25:21.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:21.267+0000 7f5d56d52700 1 -- 172.21.15.97:0/2332041315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d50102300 msgr2=0x7f5d5010e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:21.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:21.267+0000 7f5d56d52700 1 --2- 172.21.15.97:0/2332041315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d50102300 0x7f5d5010e7f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5d4800b0a0 tx=0x7f5d4800b3b0 comp rx=0 tx=0).stop 2024-04-23T10:25:21.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:21.267+0000 7f5d56d52700 1 -- 172.21.15.97:0/2332041315 shutdown_connections 2024-04-23T10:25:21.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:21.267+0000 7f5d56d52700 1 --2- 172.21.15.97:0/2332041315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d50102300 0x7f5d5010e7f0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:21.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:21.267+0000 7f5d56d52700 1 --2- 172.21.15.97:0/2332041315 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d50107990 0x7f5d50107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:21.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:21.267+0000 7f5d56d52700 1 --2- 172.21.15.97:0/2332041315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d50101940 0x7f5d50101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:21.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:21.267+0000 7f5d56d52700 1 -- 172.21.15.97:0/2332041315 >> 172.21.15.97:0/2332041315 conn(0x7f5d500788f0 msgr2=0x7f5d50078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:21.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:21.267+0000 7f5d56d52700 1 -- 172.21.15.97:0/2332041315 shutdown_connections 2024-04-23T10:25:21.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:21.267+0000 7f5d56d52700 1 -- 172.21.15.97:0/2332041315 wait complete. 2024-04-23T10:25:21.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:21.268+0000 7f5d56d52700 1 Processor -- start 2024-04-23T10:25:21.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:21.268+0000 7f5d56d52700 1 -- start start 2024-04-23T10:25:21.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:21.268+0000 7f5d56d52700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d50101940 0x7f5d501990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:21.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:21.268+0000 7f5d56d52700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d50102300 0x7f5d501995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:21.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:21.268+0000 7f5d56d52700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d50107990 0x7f5d5019d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:21.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:21.268+0000 7f5d56d52700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5d50111b50 con 0x7f5d50102300 2024-04-23T10:25:21.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:21.268+0000 7f5d56d52700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5d501119d0 con 0x7f5d50107990 2024-04-23T10:25:21.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:21.268+0000 7f5d56d52700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5d50111cd0 con 0x7f5d50101940 2024-04-23T10:25:21.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:21.268+0000 7f5d4ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d50102300 0x7f5d501995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:21.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:21.268+0000 7f5d4ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d50102300 0x7f5d501995e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33538/0 (socket says 172.21.15.97:33538) 2024-04-23T10:25:21.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:21.268+0000 7f5d4ffff700 1 -- 172.21.15.97:0/317529939 learned_addr learned my addr 172.21.15.97:0/317529939 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:25:21.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:21.268+0000 7f5d54aee700 1 --2- 172.21.15.97:0/317529939 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d50101940 0x7f5d501990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:21.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:21.268+0000 7f5d4ffff700 1 -- 172.21.15.97:0/317529939 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d50101940 msgr2=0x7f5d501990a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:21.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:21.268+0000 7f5d552ef700 1 --2- 172.21.15.97:0/317529939 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d50107990 0x7f5d5019d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:21.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:21.268+0000 7f5d4ffff700 1 --2- 172.21.15.97:0/317529939 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d50101940 0x7f5d501990a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:21.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:21.268+0000 7f5d4ffff700 1 -- 172.21.15.97:0/317529939 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d50107990 msgr2=0x7f5d5019d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:21.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:21.268+0000 7f5d4ffff700 1 --2- 172.21.15.97:0/317529939 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d50107990 0x7f5d5019d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:21.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:21.268+0000 7f5d4ffff700 1 -- 172.21.15.97:0/317529939 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d5019e0b0 con 0x7f5d50102300 2024-04-23T10:25:21.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:21.269+0000 7f5d4ffff700 1 --2- 172.21.15.97:0/317529939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d50102300 0x7f5d501995e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c00b530 tx=0x7f5d3c00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:21.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:21.269+0000 7f5d4dffb700 1 -- 172.21.15.97:0/317529939 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c015020 con 0x7f5d50102300 2024-04-23T10:25:21.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:21.269+0000 7f5d56d52700 1 -- 172.21.15.97:0/317529939 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d5019e3a0 con 0x7f5d50102300 2024-04-23T10:25:21.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:21.269+0000 7f5d4dffb700 1 -- 172.21.15.97:0/317529939 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d3c011ce0 con 0x7f5d50102300 2024-04-23T10:25:21.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:21.269+0000 7f5d4dffb700 1 -- 172.21.15.97:0/317529939 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c010d20 con 0x7f5d50102300 2024-04-23T10:25:21.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:21.269+0000 7f5d56d52700 1 -- 172.21.15.97:0/317529939 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d501a5c20 con 0x7f5d50102300 2024-04-23T10:25:21.270 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:21.270+0000 7f5d56d52700 1 -- 172.21.15.97:0/317529939 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d50069730 con 0x7f5d50102300 2024-04-23T10:25:21.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:21.271+0000 7f5d4dffb700 1 -- 172.21.15.97:0/317529939 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5d3c018020 con 0x7f5d50102300 2024-04-23T10:25:21.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:21.271+0000 7f5d4dffb700 1 --2- 172.21.15.97:0/317529939 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f5d400a3cd0 0x7f5d400a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:21.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:21.271+0000 7f5d54aee700 1 --2- 172.21.15.97:0/317529939 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f5d400a3cd0 0x7f5d400a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:21.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:21.272+0000 7f5d4dffb700 1 -- 172.21.15.97:0/317529939 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f5d3c0cfc90 con 0x7f5d50102300 2024-04-23T10:25:21.271 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:21.272+0000 7f5d54aee700 1 --2- 172.21.15.97:0/317529939 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f5d400a3cd0 0x7f5d400a6190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5d4400ba60 tx=0x7f5d44005d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:21.275 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:21.276+0000 7f5d4dffb700 1 -- 172.21.15.97:0/317529939 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5d3c0d5020 con 0x7f5d50102300 2024-04-23T10:25:21.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:21.480+0000 7f5d56d52700 1 -- 172.21.15.97:0/317529939 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f5d5019a400 con 0x7f5d50102300 2024-04-23T10:25:21.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:21.480+0000 7f5d4dffb700 1 -- 172.21.15.97:0/317529939 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f5d3c01f020 con 0x7f5d50102300 2024-04-23T10:25:21.480 INFO:teuthology.orchestra.run.smithi097.stdout:223338299441 2024-04-23T10:25:21.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:21.482+0000 7f5d56d52700 1 -- 172.21.15.97:0/317529939 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f5d400a3cd0 msgr2=0x7f5d400a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:21.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:21.483+0000 7f5d56d52700 1 --2- 172.21.15.97:0/317529939 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f5d400a3cd0 0x7f5d400a6190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5d4400ba60 tx=0x7f5d44005d50 comp rx=0 tx=0).stop 2024-04-23T10:25:21.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:21.483+0000 7f5d56d52700 1 -- 172.21.15.97:0/317529939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d50102300 msgr2=0x7f5d501995e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:21.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:21.483+0000 7f5d56d52700 1 --2- 172.21.15.97:0/317529939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d50102300 0x7f5d501995e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c00b530 tx=0x7f5d3c00b8f0 comp rx=0 tx=0).stop 2024-04-23T10:25:21.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:21.483+0000 7f5d56d52700 1 -- 172.21.15.97:0/317529939 shutdown_connections 2024-04-23T10:25:21.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:21.483+0000 7f5d56d52700 1 --2- 172.21.15.97:0/317529939 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f5d400a3cd0 0x7f5d400a6190 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:21.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:21.483+0000 7f5d56d52700 1 --2- 172.21.15.97:0/317529939 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d50107990 0x7f5d5019d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:21.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:21.483+0000 7f5d56d52700 1 --2- 172.21.15.97:0/317529939 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5d50101940 0x7f5d501990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:21.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:21.483+0000 7f5d56d52700 1 --2- 172.21.15.97:0/317529939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d50102300 0x7f5d501995e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:21.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:21.483+0000 7f5d56d52700 1 -- 172.21.15.97:0/317529939 >> 172.21.15.97:0/317529939 conn(0x7f5d500788f0 msgr2=0x7f5d5006aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:21.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:21.483+0000 7f5d56d52700 1 -- 172.21.15.97:0/317529939 shutdown_connections 2024-04-23T10:25:21.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:21.483+0000 7f5d56d52700 1 -- 172.21.15.97:0/317529939 wait complete. 2024-04-23T10:25:22.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:21 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/317529939' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-23T10:25:22.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:21 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/317529939' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-23T10:25:22.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:21 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/317529939' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-23T10:25:22.202 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299435 got 223338299441 for osd.7 2024-04-23T10:25:22.202 DEBUG:teuthology.parallel:result is None 2024-04-23T10:25:22.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:22 smithi097 ceph-mon[29197]: pgmap v329: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-23T10:25:23.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:22 smithi196 ceph-mon[32919]: pgmap v329: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-23T10:25:23.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:22 smithi183 ceph-mon[32317]: pgmap v329: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-23T10:25:23.393 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149760 got 111669149765 for osd.3 2024-04-23T10:25:23.393 DEBUG:teuthology.parallel:result is None 2024-04-23T10:25:23.909 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462961 got 201863462966 for osd.6 2024-04-23T10:25:23.909 DEBUG:teuthology.parallel:result is None 2024-04-23T10:25:23.909 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-23T10:25:23.910 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-23T10:25:24.195 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T10:25:25.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:24 smithi196 ceph-mon[32919]: pgmap v330: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-23T10:25:25.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:24 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:25:25.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:24 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:25:25.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:24 smithi097 ceph-mon[29197]: pgmap v330: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-23T10:25:25.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:24 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:25:25.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:24 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:25:25.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:24 smithi183 ceph-mon[32317]: pgmap v330: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-23T10:25:25.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:24 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:25:25.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:24 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:25:25.908 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:25 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:25:25.908 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:25 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:25:25.908 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:25 smithi097 ceph-mon[29197]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:25:26.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:26.010+0000 7ff3238ab700 1 -- 172.21.15.97:0/2532864242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff31c108a20 msgr2=0x7ff31c108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:26.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:26.010+0000 7ff3238ab700 1 --2- 172.21.15.97:0/2532864242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff31c108a20 0x7ff31c108e00 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff318009990 tx=0x7ff318009ca0 comp rx=0 tx=0).stop 2024-04-23T10:25:26.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:26.010+0000 7ff3238ab700 1 -- 172.21.15.97:0/2532864242 shutdown_connections 2024-04-23T10:25:26.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:26.010+0000 7ff3238ab700 1 --2- 172.21.15.97:0/2532864242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff31c06b980 0x7ff31c10f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:26.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:26.010+0000 7ff3238ab700 1 --2- 172.21.15.97:0/2532864242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff31c108a20 0x7ff31c108e00 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:26.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:26.010+0000 7ff3238ab700 1 --2- 172.21.15.97:0/2532864242 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff31c06afe0 0x7ff31c06b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:26.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:26.010+0000 7ff3238ab700 1 -- 172.21.15.97:0/2532864242 >> 172.21.15.97:0/2532864242 conn(0x7ff31c0788b0 msgr2=0x7ff31c078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:26.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:26.010+0000 7ff3238ab700 1 -- 172.21.15.97:0/2532864242 shutdown_connections 2024-04-23T10:25:26.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:26.010+0000 7ff3238ab700 1 -- 172.21.15.97:0/2532864242 wait complete. 2024-04-23T10:25:26.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:26.010+0000 7ff3238ab700 1 Processor -- start 2024-04-23T10:25:26.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:26.010+0000 7ff3238ab700 1 -- start start 2024-04-23T10:25:26.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:26.011+0000 7ff3238ab700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff31c06afe0 0x7ff31c1990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:26.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:26.011+0000 7ff3238ab700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff31c06b980 0x7ff31c199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:26.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:26.011+0000 7ff3238ab700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff31c108a20 0x7ff31c19d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:26.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:26.011+0000 7ff3238ab700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff31c112c20 con 0x7ff31c06afe0 2024-04-23T10:25:26.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:26.011+0000 7ff3238ab700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff31c112aa0 con 0x7ff31c108a20 2024-04-23T10:25:26.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:26.011+0000 7ff3238ab700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff31c112da0 con 0x7ff31c06b980 2024-04-23T10:25:26.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:26.011+0000 7ff321647700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff31c06afe0 0x7ff31c1990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:26.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:26.011+0000 7ff321647700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff31c06afe0 0x7ff31c1990f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33808/0 (socket says 172.21.15.97:33808) 2024-04-23T10:25:26.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:26.011+0000 7ff321647700 1 -- 172.21.15.97:0/2355006576 learned_addr learned my addr 172.21.15.97:0/2355006576 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:25:26.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:26.011+0000 7ff320e46700 1 --2- 172.21.15.97:0/2355006576 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff31c06b980 0x7ff31c199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:26.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:26.011+0000 7ff321e48700 1 --2- 172.21.15.97:0/2355006576 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff31c108a20 0x7ff31c19d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:26.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:26.011+0000 7ff321647700 1 -- 172.21.15.97:0/2355006576 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff31c06b980 msgr2=0x7ff31c199630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:26.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:26.011+0000 7ff321647700 1 --2- 172.21.15.97:0/2355006576 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff31c06b980 0x7ff31c199630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:26.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:26.011+0000 7ff321647700 1 -- 172.21.15.97:0/2355006576 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff31c108a20 msgr2=0x7ff31c19d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:26.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:26.011+0000 7ff321647700 1 --2- 172.21.15.97:0/2355006576 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff31c108a20 0x7ff31c19d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:26.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:26.011+0000 7ff321647700 1 -- 172.21.15.97:0/2355006576 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff31c19e0b0 con 0x7ff31c06afe0 2024-04-23T10:25:26.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:26.011+0000 7ff321e48700 1 --2- 172.21.15.97:0/2355006576 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff31c108a20 0x7ff31c19d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:25:26.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:26.011+0000 7ff321647700 1 --2- 172.21.15.97:0/2355006576 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff31c06afe0 0x7ff31c1990f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff3180057f0 tx=0x7ff318003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:26.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:26.011+0000 7ff3127fc700 1 -- 172.21.15.97:0/2355006576 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff318027070 con 0x7ff31c06afe0 2024-04-23T10:25:26.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:26.011+0000 7ff3238ab700 1 -- 172.21.15.97:0/2355006576 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff31c19e340 con 0x7ff31c06afe0 2024-04-23T10:25:26.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:26.011+0000 7ff3127fc700 1 -- 172.21.15.97:0/2355006576 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3180041c0 con 0x7ff31c06afe0 2024-04-23T10:25:26.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:26.011+0000 7ff3127fc700 1 -- 172.21.15.97:0/2355006576 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff318022e60 con 0x7ff31c06afe0 2024-04-23T10:25:26.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:26.011+0000 7ff3238ab700 1 -- 172.21.15.97:0/2355006576 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff31c1a5c20 con 0x7ff31c06afe0 2024-04-23T10:25:26.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:26.012+0000 7ff3238ab700 1 -- 172.21.15.97:0/2355006576 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff31c069730 con 0x7ff31c06afe0 2024-04-23T10:25:26.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:26.013+0000 7ff3127fc700 1 -- 172.21.15.97:0/2355006576 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff31802a070 con 0x7ff31c06afe0 2024-04-23T10:25:26.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:26.014+0000 7ff3127fc700 1 --2- 172.21.15.97:0/2355006576 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7ff3080a3cd0 0x7ff3080a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:26.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:26.014+0000 7ff320e46700 1 --2- 172.21.15.97:0/2355006576 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7ff3080a3cd0 0x7ff3080a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:26.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:26.015+0000 7ff3127fc700 1 -- 172.21.15.97:0/2355006576 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7ff3180d18a0 con 0x7ff31c06afe0 2024-04-23T10:25:26.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:26.015+0000 7ff320e46700 1 --2- 172.21.15.97:0/2355006576 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7ff3080a3cd0 0x7ff3080a6190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff30c005ea0 tx=0x7ff30c005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:26.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:26.018+0000 7ff3127fc700 1 -- 172.21.15.97:0/2355006576 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff318013070 con 0x7ff31c06afe0 2024-04-23T10:25:26.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:25 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:25:26.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:25 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:25:26.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:25 smithi196 ceph-mon[32919]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:25:26.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:25 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:25:26.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:25 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:25:26.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:25 smithi183 ceph-mon[32317]: from='mgr.14152 172.21.15.97:0/3134822600' entity='mgr.x' 2024-04-23T10:25:26.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:26.215+0000 7ff3238ab700 1 -- 172.21.15.97:0/2355006576 --> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff31c19a400 con 0x7ff3080a3cd0 2024-04-23T10:25:26.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:26.224+0000 7ff3127fc700 1 -- 172.21.15.97:0/2355006576 <== mgr.14152 v2:172.21.15.97:6800/3149043749 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258989 (secure 0 0 0) 0x7ff31c19a400 con 0x7ff3080a3cd0 2024-04-23T10:25:26.228 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:25:26.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:26.227+0000 7ff3238ab700 1 -- 172.21.15.97:0/2355006576 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7ff3080a3cd0 msgr2=0x7ff3080a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:26.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:26.227+0000 7ff3238ab700 1 --2- 172.21.15.97:0/2355006576 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7ff3080a3cd0 0x7ff3080a6190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff30c005ea0 tx=0x7ff30c005e30 comp rx=0 tx=0).stop 2024-04-23T10:25:26.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:26.227+0000 7ff3238ab700 1 -- 172.21.15.97:0/2355006576 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff31c06afe0 msgr2=0x7ff31c1990f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:26.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:26.227+0000 7ff3238ab700 1 --2- 172.21.15.97:0/2355006576 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff31c06afe0 0x7ff31c1990f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff3180057f0 tx=0x7ff318003730 comp rx=0 tx=0).stop 2024-04-23T10:25:26.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:26.228+0000 7ff3238ab700 1 -- 172.21.15.97:0/2355006576 shutdown_connections 2024-04-23T10:25:26.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:26.228+0000 7ff3238ab700 1 --2- 172.21.15.97:0/2355006576 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7ff3080a3cd0 0x7ff3080a6190 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:26.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:26.228+0000 7ff3238ab700 1 --2- 172.21.15.97:0/2355006576 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff31c108a20 0x7ff31c19d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:26.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:26.228+0000 7ff3238ab700 1 --2- 172.21.15.97:0/2355006576 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff31c06afe0 0x7ff31c1990f0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:26.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:26.228+0000 7ff3238ab700 1 --2- 172.21.15.97:0/2355006576 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff31c06b980 0x7ff31c199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:26.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:26.228+0000 7ff3238ab700 1 -- 172.21.15.97:0/2355006576 >> 172.21.15.97:0/2355006576 conn(0x7ff31c0788b0 msgr2=0x7ff31c0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:26.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:26.228+0000 7ff3238ab700 1 -- 172.21.15.97:0/2355006576 shutdown_connections 2024-04-23T10:25:26.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:26.228+0000 7ff3238ab700 1 -- 172.21.15.97:0/2355006576 wait complete. 2024-04-23T10:25:26.230 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-04-23T10:25:26.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:26 smithi097 ceph-mon[29197]: pgmap v331: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-23T10:25:26.925 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":331,"stamp":"2024-04-23T10:25:25.572117+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659243,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":887,"num_read_kb":562,"num_write":421,"num_write_kb":3074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1181990,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":346,"ondisk_log_size":346,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361016,"kb_used_data":11796,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580808,"statfs":{"total":1151940427776,"available":1151570747392,"internally_reserved":0,"allocated":12079104,"data_stored":7480677,"data_compressed":112413,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":125,"num_read_kb":55,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002782"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.664727+0000","last_change":"2024-04-23T10:24:07.040397+0000","last_active":"2024-04-23T10:25:00.664727+0000","last_peered":"2024-04-23T10:25:00.664727+0000","last_clean":"2024-04-23T10:25:00.664727+0000","last_became_active":"2024-04-23T10:24:07.040007+0000","last_became_peered":"2024-04-23T10:24:07.040007+0000","last_unstale":"2024-04-23T10:25:00.664727+0000","last_undegraded":"2024-04-23T10:25:00.664727+0000","last_fullsized":"2024-04-23T10:25:00.664727+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T13:50:24.533115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.669176+0000","last_change":"2024-04-23T10:24:07.044672+0000","last_active":"2024-04-23T10:25:00.669176+0000","last_peered":"2024-04-23T10:25:00.669176+0000","last_clean":"2024-04-23T10:25:00.669176+0000","last_became_active":"2024-04-23T10:24:07.044184+0000","last_became_peered":"2024-04-23T10:24:07.044184+0000","last_unstale":"2024-04-23T10:25:00.669176+0000","last_undegraded":"2024-04-23T10:25:00.669176+0000","last_fullsized":"2024-04-23T10:25:00.669176+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:07:16.729205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.669118+0000","last_change":"2024-04-23T10:24:07.041959+0000","last_active":"2024-04-23T10:25:00.669118+0000","last_peered":"2024-04-23T10:25:00.669118+0000","last_clean":"2024-04-23T10:25:00.669118+0000","last_became_active":"2024-04-23T10:24:07.041661+0000","last_became_peered":"2024-04-23T10:24:07.041661+0000","last_unstale":"2024-04-23T10:25:00.669118+0000","last_undegraded":"2024-04-23T10:25:00.669118+0000","last_fullsized":"2024-04-23T10:25:00.669118+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:23:14.694183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.728712+0000","last_change":"2024-04-23T10:24:07.048334+0000","last_active":"2024-04-23T10:25:00.728712+0000","last_peered":"2024-04-23T10:25:00.728712+0000","last_clean":"2024-04-23T10:25:00.728712+0000","last_became_active":"2024-04-23T10:24:07.047506+0000","last_became_peered":"2024-04-23T10:24:07.047506+0000","last_unstale":"2024-04-23T10:25:00.728712+0000","last_undegraded":"2024-04-23T10:25:00.728712+0000","last_fullsized":"2024-04-23T10:25:00.728712+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:27:27.674287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.666897+0000","last_change":"2024-04-23T10:24:07.035531+0000","last_active":"2024-04-23T10:25:00.666897+0000","last_peered":"2024-04-23T10:25:00.666897+0000","last_clean":"2024-04-23T10:25:00.666897+0000","last_became_active":"2024-04-23T10:24:07.035167+0000","last_became_peered":"2024-04-23T10:24:07.035167+0000","last_unstale":"2024-04-23T10:25:00.666897+0000","last_undegraded":"2024-04-23T10:25:00.666897+0000","last_fullsized":"2024-04-23T10:25:00.666897+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:39:17.198650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.664824+0000","last_change":"2024-04-23T10:24:07.040593+0000","last_active":"2024-04-23T10:25:00.664824+0000","last_peered":"2024-04-23T10:25:00.664824+0000","last_clean":"2024-04-23T10:25:00.664824+0000","last_became_active":"2024-04-23T10:24:07.040255+0000","last_became_peered":"2024-04-23T10:24:07.040255+0000","last_unstale":"2024-04-23T10:25:00.664824+0000","last_undegraded":"2024-04-23T10:25:00.664824+0000","last_fullsized":"2024-04-23T10:25:00.664824+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:05:52.663566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.667615+0000","last_change":"2024-04-23T10:24:07.051688+0000","last_active":"2024-04-23T10:25:00.667615+0000","last_peered":"2024-04-23T10:25:00.667615+0000","last_clean":"2024-04-23T10:25:00.667615+0000","last_became_active":"2024-04-23T10:24:07.050265+0000","last_became_peered":"2024-04-23T10:24:07.050265+0000","last_unstale":"2024-04-23T10:25:00.667615+0000","last_undegraded":"2024-04-23T10:25:00.667615+0000","last_fullsized":"2024-04-23T10:25:00.667615+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T13:13:36.126183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:01.274280+0000","last_change":"2024-04-23T10:24:07.045937+0000","last_active":"2024-04-23T10:25:01.274280+0000","last_peered":"2024-04-23T10:25:01.274280+0000","last_clean":"2024-04-23T10:25:01.274280+0000","last_became_active":"2024-04-23T10:24:07.045616+0000","last_became_peered":"2024-04-23T10:24:07.045616+0000","last_unstale":"2024-04-23T10:25:01.274280+0000","last_undegraded":"2024-04-23T10:25:01.274280+0000","last_fullsized":"2024-04-23T10:25:01.274280+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T10:45:27.250031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.667488+0000","last_change":"2024-04-23T10:24:07.039294+0000","last_active":"2024-04-23T10:25:00.667488+0000","last_peered":"2024-04-23T10:25:00.667488+0000","last_clean":"2024-04-23T10:25:00.667488+0000","last_became_active":"2024-04-23T10:24:07.038959+0000","last_became_peered":"2024-04-23T10:24:07.038959+0000","last_unstale":"2024-04-23T10:25:00.667488+0000","last_undegraded":"2024-04-23T10:25:00.667488+0000","last_fullsized":"2024-04-23T10:25:00.667488+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:53:41.345062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.665337+0000","last_change":"2024-04-23T10:24:07.044177+0000","last_active":"2024-04-23T10:25:00.665337+0000","last_peered":"2024-04-23T10:25:00.665337+0000","last_clean":"2024-04-23T10:25:00.665337+0000","last_became_active":"2024-04-23T10:24:07.043821+0000","last_became_peered":"2024-04-23T10:24:07.043821+0000","last_unstale":"2024-04-23T10:25:00.665337+0000","last_undegraded":"2024-04-23T10:25:00.665337+0000","last_fullsized":"2024-04-23T10:25:00.665337+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:09:38.782407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.667941+0000","last_change":"2024-04-23T10:24:07.052775+0000","last_active":"2024-04-23T10:25:00.667941+0000","last_peered":"2024-04-23T10:25:00.667941+0000","last_clean":"2024-04-23T10:25:00.667941+0000","last_became_active":"2024-04-23T10:24:07.051708+0000","last_became_peered":"2024-04-23T10:24:07.051708+0000","last_unstale":"2024-04-23T10:25:00.667941+0000","last_undegraded":"2024-04-23T10:25:00.667941+0000","last_fullsized":"2024-04-23T10:25:00.667941+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:47:55.360467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.728601+0000","last_change":"2024-04-23T10:24:07.048655+0000","last_active":"2024-04-23T10:25:00.728601+0000","last_peered":"2024-04-23T10:25:00.728601+0000","last_clean":"2024-04-23T10:25:00.728601+0000","last_became_active":"2024-04-23T10:24:07.047994+0000","last_became_peered":"2024-04-23T10:24:07.047994+0000","last_unstale":"2024-04-23T10:25:00.728601+0000","last_undegraded":"2024-04-23T10:25:00.728601+0000","last_fullsized":"2024-04-23T10:25:00.728601+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T10:25:51.425871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.667525+0000","last_change":"2024-04-23T10:24:07.036319+0000","last_active":"2024-04-23T10:25:00.667525+0000","last_peered":"2024-04-23T10:25:00.667525+0000","last_clean":"2024-04-23T10:25:00.667525+0000","last_became_active":"2024-04-23T10:24:07.035982+0000","last_became_peered":"2024-04-23T10:24:07.035982+0000","last_unstale":"2024-04-23T10:25:00.667525+0000","last_undegraded":"2024-04-23T10:25:00.667525+0000","last_fullsized":"2024-04-23T10:25:00.667525+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T10:25:46.191118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.669516+0000","last_change":"2024-04-23T10:24:07.033166+0000","last_active":"2024-04-23T10:25:00.669516+0000","last_peered":"2024-04-23T10:25:00.669516+0000","last_clean":"2024-04-23T10:25:00.669516+0000","last_became_active":"2024-04-23T10:24:07.032807+0000","last_became_peered":"2024-04-23T10:24:07.032807+0000","last_unstale":"2024-04-23T10:25:00.669516+0000","last_undegraded":"2024-04-23T10:25:00.669516+0000","last_fullsized":"2024-04-23T10:25:00.669516+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:52:35.654965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.668855+0000","last_change":"2024-04-23T10:24:07.041988+0000","last_active":"2024-04-23T10:25:00.668855+0000","last_peered":"2024-04-23T10:25:00.668855+0000","last_clean":"2024-04-23T10:25:00.668855+0000","last_became_active":"2024-04-23T10:24:07.041638+0000","last_became_peered":"2024-04-23T10:24:07.041638+0000","last_unstale":"2024-04-23T10:25:00.668855+0000","last_undegraded":"2024-04-23T10:25:00.668855+0000","last_fullsized":"2024-04-23T10:25:00.668855+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:17:40.565981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.666563+0000","last_change":"2024-04-23T10:24:07.045111+0000","last_active":"2024-04-23T10:25:00.666563+0000","last_peered":"2024-04-23T10:25:00.666563+0000","last_clean":"2024-04-23T10:25:00.666563+0000","last_became_active":"2024-04-23T10:24:07.044249+0000","last_became_peered":"2024-04-23T10:24:07.044249+0000","last_unstale":"2024-04-23T10:25:00.666563+0000","last_undegraded":"2024-04-23T10:25:00.666563+0000","last_fullsized":"2024-04-23T10:25:00.666563+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:08:36.029636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.664830+0000","last_change":"2024-04-23T10:24:07.040685+0000","last_active":"2024-04-23T10:25:00.664830+0000","last_peered":"2024-04-23T10:25:00.664830+0000","last_clean":"2024-04-23T10:25:00.664830+0000","last_became_active":"2024-04-23T10:24:07.038881+0000","last_became_peered":"2024-04-23T10:24:07.038881+0000","last_unstale":"2024-04-23T10:25:00.664830+0000","last_undegraded":"2024-04-23T10:25:00.664830+0000","last_fullsized":"2024-04-23T10:25:00.664830+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:29:12.489136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.664056+0000","last_change":"2024-04-23T10:24:07.041815+0000","last_active":"2024-04-23T10:25:00.664056+0000","last_peered":"2024-04-23T10:25:00.664056+0000","last_clean":"2024-04-23T10:25:00.664056+0000","last_became_active":"2024-04-23T10:24:07.041495+0000","last_became_peered":"2024-04-23T10:24:07.041495+0000","last_unstale":"2024-04-23T10:25:00.664056+0000","last_undegraded":"2024-04-23T10:25:00.664056+0000","last_fullsized":"2024-04-23T10:25:00.664056+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:29:02.006394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.670731+0000","last_change":"2024-04-23T10:24:07.041722+0000","last_active":"2024-04-23T10:25:00.670731+0000","last_peered":"2024-04-23T10:25:00.670731+0000","last_clean":"2024-04-23T10:25:00.670731+0000","last_became_active":"2024-04-23T10:24:07.039434+0000","last_became_peered":"2024-04-23T10:24:07.039434+0000","last_unstale":"2024-04-23T10:25:00.670731+0000","last_undegraded":"2024-04-23T10:25:00.670731+0000","last_fullsized":"2024-04-23T10:25:00.670731+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:03:44.033094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'13","reported_seq":122,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:20.756677+0000","last_change":"2024-04-23T10:25:00.739010+0000","last_active":"2024-04-23T10:25:20.756677+0000","last_peered":"2024-04-23T10:25:20.756677+0000","last_clean":"2024-04-23T10:25:20.756677+0000","last_became_active":"2024-04-23T10:25:00.729834+0000","last_became_peered":"2024-04-23T10:25:00.729834+0000","last_unstale":"2024-04-23T10:25:20.756677+0000","last_undegraded":"2024-04-23T10:25:20.756677+0000","last_fullsized":"2024-04-23T10:25:20.756677+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T12:57:45.871560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":65,"num_read_kb":65,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,4],"acting":[9,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:01.274678+0000","last_change":"2024-04-23T10:24:07.047192+0000","last_active":"2024-04-23T10:25:01.274678+0000","last_peered":"2024-04-23T10:25:01.274678+0000","last_clean":"2024-04-23T10:25:01.274678+0000","last_became_active":"2024-04-23T10:24:07.046750+0000","last_became_peered":"2024-04-23T10:24:07.046750+0000","last_unstale":"2024-04-23T10:25:01.274678+0000","last_undegraded":"2024-04-23T10:25:01.274678+0000","last_fullsized":"2024-04-23T10:25:01.274678+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:13:09.399425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:01.274791+0000","last_change":"2024-04-23T10:24:06.577588+0000","last_active":"2024-04-23T10:25:01.274791+0000","last_peered":"2024-04-23T10:25:01.274791+0000","last_clean":"2024-04-23T10:25:01.274791+0000","last_became_active":"2024-04-23T10:24:06.576971+0000","last_became_peered":"2024-04-23T10:24:06.576971+0000","last_unstale":"2024-04-23T10:25:01.274791+0000","last_undegraded":"2024-04-23T10:25:01.274791+0000","last_fullsized":"2024-04-23T10:25:01.274791+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:07:21.577868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.669348+0000","last_change":"2024-04-23T10:24:07.030666+0000","last_active":"2024-04-23T10:25:00.669348+0000","last_peered":"2024-04-23T10:25:00.669348+0000","last_clean":"2024-04-23T10:25:00.669348+0000","last_became_active":"2024-04-23T10:24:07.030342+0000","last_became_peered":"2024-04-23T10:24:07.030342+0000","last_unstale":"2024-04-23T10:25:00.669348+0000","last_undegraded":"2024-04-23T10:25:00.669348+0000","last_fullsized":"2024-04-23T10:25:00.669348+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:09:14.818638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.666771+0000","last_change":"2024-04-23T10:24:06.576042+0000","last_active":"2024-04-23T10:25:00.666771+0000","last_peered":"2024-04-23T10:25:00.666771+0000","last_clean":"2024-04-23T10:25:00.666771+0000","last_became_active":"2024-04-23T10:24:06.574518+0000","last_became_peered":"2024-04-23T10:24:06.574518+0000","last_unstale":"2024-04-23T10:25:00.666771+0000","last_undegraded":"2024-04-23T10:25:00.666771+0000","last_fullsized":"2024-04-23T10:25:00.666771+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T11:56:40.100213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.666998+0000","last_change":"2024-04-23T10:24:07.046591+0000","last_active":"2024-04-23T10:25:00.666998+0000","last_peered":"2024-04-23T10:25:00.666998+0000","last_clean":"2024-04-23T10:25:00.666998+0000","last_became_active":"2024-04-23T10:24:07.046208+0000","last_became_peered":"2024-04-23T10:24:07.046208+0000","last_unstale":"2024-04-23T10:25:00.666998+0000","last_undegraded":"2024-04-23T10:25:00.666998+0000","last_fullsized":"2024-04-23T10:25:00.666998+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:27:37.711329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.676709+0000","last_change":"2024-04-23T10:25:00.676255+0000","last_active":"2024-04-23T10:25:00.676709+0000","last_peered":"2024-04-23T10:25:00.676709+0000","last_clean":"2024-04-23T10:25:00.676709+0000","last_became_active":"2024-04-23T10:25:00.670146+0000","last_became_peered":"2024-04-23T10:25:00.670146+0000","last_unstale":"2024-04-23T10:25:00.676709+0000","last_undegraded":"2024-04-23T10:25:00.676709+0000","last_fullsized":"2024-04-23T10:25:00.676709+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:34:54.217317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.668016+0000","last_change":"2024-04-23T10:24:07.047926+0000","last_active":"2024-04-23T10:25:00.668016+0000","last_peered":"2024-04-23T10:25:00.668016+0000","last_clean":"2024-04-23T10:25:00.668016+0000","last_became_active":"2024-04-23T10:24:07.047273+0000","last_became_peered":"2024-04-23T10:24:07.047273+0000","last_unstale":"2024-04-23T10:25:00.668016+0000","last_undegraded":"2024-04-23T10:25:00.668016+0000","last_fullsized":"2024-04-23T10:25:00.668016+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:37:25.068034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'6","reported_seq":102,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:23.052202+0000","last_change":"2024-04-23T10:24:06.574733+0000","last_active":"2024-04-23T10:25:23.052202+0000","last_peered":"2024-04-23T10:25:23.052202+0000","last_clean":"2024-04-23T10:25:23.052202+0000","last_became_active":"2024-04-23T10:24:06.574276+0000","last_became_peered":"2024-04-23T10:24:06.574276+0000","last_unstale":"2024-04-23T10:25:23.052202+0000","last_undegraded":"2024-04-23T10:25:23.052202+0000","last_fullsized":"2024-04-23T10:25:23.052202+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:27:57.080635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":77,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.666430+0000","last_change":"2024-04-23T10:24:07.040110+0000","last_active":"2024-04-23T10:25:00.666430+0000","last_peered":"2024-04-23T10:25:00.666430+0000","last_clean":"2024-04-23T10:25:00.666430+0000","last_became_active":"2024-04-23T10:24:07.039769+0000","last_became_peered":"2024-04-23T10:24:07.039769+0000","last_unstale":"2024-04-23T10:25:00.666430+0000","last_undegraded":"2024-04-23T10:25:00.666430+0000","last_fullsized":"2024-04-23T10:25:00.666430+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:10:07.884895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.664807+0000","last_change":"2024-04-23T10:24:06.567448+0000","last_active":"2024-04-23T10:25:00.664807+0000","last_peered":"2024-04-23T10:25:00.664807+0000","last_clean":"2024-04-23T10:25:00.664807+0000","last_became_active":"2024-04-23T10:24:06.566958+0000","last_became_peered":"2024-04-23T10:24:06.566958+0000","last_unstale":"2024-04-23T10:25:00.664807+0000","last_undegraded":"2024-04-23T10:25:00.664807+0000","last_fullsized":"2024-04-23T10:25:00.664807+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:45:52.863596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.667724+0000","last_change":"2024-04-23T10:24:07.040754+0000","last_active":"2024-04-23T10:25:00.667724+0000","last_peered":"2024-04-23T10:25:00.667724+0000","last_clean":"2024-04-23T10:25:00.667724+0000","last_became_active":"2024-04-23T10:24:07.040397+0000","last_became_peered":"2024-04-23T10:24:07.040397+0000","last_unstale":"2024-04-23T10:25:00.667724+0000","last_undegraded":"2024-04-23T10:25:00.667724+0000","last_fullsized":"2024-04-23T10:25:00.667724+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T11:01:03.626506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.667441+0000","last_change":"2024-04-23T10:24:06.568592+0000","last_active":"2024-04-23T10:25:00.667441+0000","last_peered":"2024-04-23T10:25:00.667441+0000","last_clean":"2024-04-23T10:25:00.667441+0000","last_became_active":"2024-04-23T10:24:06.568192+0000","last_became_peered":"2024-04-23T10:24:06.568192+0000","last_unstale":"2024-04-23T10:25:00.667441+0000","last_undegraded":"2024-04-23T10:25:00.667441+0000","last_fullsized":"2024-04-23T10:25:00.667441+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:14:42.589596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:01.274328+0000","last_change":"2024-04-23T10:24:07.037058+0000","last_active":"2024-04-23T10:25:01.274328+0000","last_peered":"2024-04-23T10:25:01.274328+0000","last_clean":"2024-04-23T10:25:01.274328+0000","last_became_active":"2024-04-23T10:24:07.036659+0000","last_became_peered":"2024-04-23T10:24:07.036659+0000","last_unstale":"2024-04-23T10:25:01.274328+0000","last_undegraded":"2024-04-23T10:25:01.274328+0000","last_fullsized":"2024-04-23T10:25:01.274328+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T12:53:49.298603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.668070+0000","last_change":"2024-04-23T10:24:06.576900+0000","last_active":"2024-04-23T10:25:00.668070+0000","last_peered":"2024-04-23T10:25:00.668070+0000","last_clean":"2024-04-23T10:25:00.668070+0000","last_became_active":"2024-04-23T10:24:06.575627+0000","last_became_peered":"2024-04-23T10:24:06.575627+0000","last_unstale":"2024-04-23T10:25:00.668070+0000","last_undegraded":"2024-04-23T10:25:00.668070+0000","last_fullsized":"2024-04-23T10:25:00.668070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:16:07.973841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.663790+0000","last_change":"2024-04-23T10:24:07.035294+0000","last_active":"2024-04-23T10:25:00.663790+0000","last_peered":"2024-04-23T10:25:00.663790+0000","last_clean":"2024-04-23T10:25:00.663790+0000","last_became_active":"2024-04-23T10:24:07.035001+0000","last_became_peered":"2024-04-23T10:24:07.035001+0000","last_unstale":"2024-04-23T10:25:00.663790+0000","last_undegraded":"2024-04-23T10:25:00.663790+0000","last_fullsized":"2024-04-23T10:25:00.663790+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T13:12:47.013132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'4","reported_seq":163,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:21.048731+0000","last_change":"2024-04-23T10:24:06.573637+0000","last_active":"2024-04-23T10:25:21.048731+0000","last_peered":"2024-04-23T10:25:21.048731+0000","last_clean":"2024-04-23T10:25:21.048731+0000","last_became_active":"2024-04-23T10:24:06.573236+0000","last_became_peered":"2024-04-23T10:24:06.573236+0000","last_unstale":"2024-04-23T10:25:21.048731+0000","last_undegraded":"2024-04-23T10:25:21.048731+0000","last_fullsized":"2024-04-23T10:25:21.048731+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T12:48:06.740181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":140,"num_read_kb":140,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.667203+0000","last_change":"2024-04-23T10:24:07.037457+0000","last_active":"2024-04-23T10:25:00.667203+0000","last_peered":"2024-04-23T10:25:00.667203+0000","last_clean":"2024-04-23T10:25:00.667203+0000","last_became_active":"2024-04-23T10:24:07.036456+0000","last_became_peered":"2024-04-23T10:24:07.036456+0000","last_unstale":"2024-04-23T10:25:00.667203+0000","last_undegraded":"2024-04-23T10:25:00.667203+0000","last_fullsized":"2024-04-23T10:25:00.667203+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:42:57.750435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.741712+0000","last_change":"2024-04-23T10:25:00.741712+0000","last_active":"2024-04-23T10:25:00.741712+0000","last_peered":"2024-04-23T10:25:00.741712+0000","last_clean":"2024-04-23T10:25:00.741712+0000","last_became_active":"2024-04-23T10:25:00.730130+0000","last_became_peered":"2024-04-23T10:25:00.730130+0000","last_unstale":"2024-04-23T10:25:00.741712+0000","last_undegraded":"2024-04-23T10:25:00.741712+0000","last_fullsized":"2024-04-23T10:25:00.741712+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:08:14.409376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.669252+0000","last_change":"2024-04-23T10:24:07.040317+0000","last_active":"2024-04-23T10:25:00.669252+0000","last_peered":"2024-04-23T10:25:00.669252+0000","last_clean":"2024-04-23T10:25:00.669252+0000","last_became_active":"2024-04-23T10:24:07.039998+0000","last_became_peered":"2024-04-23T10:24:07.039998+0000","last_unstale":"2024-04-23T10:25:00.669252+0000","last_undegraded":"2024-04-23T10:25:00.669252+0000","last_fullsized":"2024-04-23T10:25:00.669252+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:50:30.412127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.668115+0000","last_change":"2024-04-23T10:24:06.570019+0000","last_active":"2024-04-23T10:25:00.668115+0000","last_peered":"2024-04-23T10:25:00.668115+0000","last_clean":"2024-04-23T10:25:00.668115+0000","last_became_active":"2024-04-23T10:24:06.569676+0000","last_became_peered":"2024-04-23T10:24:06.569676+0000","last_unstale":"2024-04-23T10:25:00.668115+0000","last_undegraded":"2024-04-23T10:25:00.668115+0000","last_fullsized":"2024-04-23T10:25:00.668115+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:31:46.994787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.667134+0000","last_change":"2024-04-23T10:24:07.040183+0000","last_active":"2024-04-23T10:25:00.667134+0000","last_peered":"2024-04-23T10:25:00.667134+0000","last_clean":"2024-04-23T10:25:00.667134+0000","last_became_active":"2024-04-23T10:24:07.039866+0000","last_became_peered":"2024-04-23T10:24:07.039866+0000","last_unstale":"2024-04-23T10:25:00.667134+0000","last_undegraded":"2024-04-23T10:25:00.667134+0000","last_fullsized":"2024-04-23T10:25:00.667134+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:33:04.411765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'30","reported_seq":199,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:21.048617+0000","last_change":"2024-04-23T10:24:06.568883+0000","last_active":"2024-04-23T10:25:21.048617+0000","last_peered":"2024-04-23T10:25:21.048617+0000","last_clean":"2024-04-23T10:25:21.048617+0000","last_became_active":"2024-04-23T10:24:06.568517+0000","last_became_peered":"2024-04-23T10:24:06.568517+0000","last_unstale":"2024-04-23T10:25:21.048617+0000","last_undegraded":"2024-04-23T10:25:21.048617+0000","last_fullsized":"2024-04-23T10:25:21.048617+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:51:42.440346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61046,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":150,"num_read_kb":134,"num_write":32,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.666600+0000","last_change":"2024-04-23T10:24:07.046839+0000","last_active":"2024-04-23T10:25:00.666600+0000","last_peered":"2024-04-23T10:25:00.666600+0000","last_clean":"2024-04-23T10:25:00.666600+0000","last_became_active":"2024-04-23T10:24:07.046512+0000","last_became_peered":"2024-04-23T10:24:07.046512+0000","last_unstale":"2024-04-23T10:25:00.666600+0000","last_undegraded":"2024-04-23T10:25:00.666600+0000","last_fullsized":"2024-04-23T10:25:00.666600+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T12:42:11.822100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.670189+0000","last_change":"2024-04-23T10:24:06.578087+0000","last_active":"2024-04-23T10:25:00.670189+0000","last_peered":"2024-04-23T10:25:00.670189+0000","last_clean":"2024-04-23T10:25:00.670189+0000","last_became_active":"2024-04-23T10:24:06.577666+0000","last_became_peered":"2024-04-23T10:24:06.577666+0000","last_unstale":"2024-04-23T10:25:00.670189+0000","last_undegraded":"2024-04-23T10:25:00.670189+0000","last_fullsized":"2024-04-23T10:25:00.670189+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:55:15.053303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.667210+0000","last_change":"2024-04-23T10:24:07.043672+0000","last_active":"2024-04-23T10:25:00.667210+0000","last_peered":"2024-04-23T10:25:00.667210+0000","last_clean":"2024-04-23T10:25:00.667210+0000","last_became_active":"2024-04-23T10:24:07.043346+0000","last_became_peered":"2024-04-23T10:24:07.043346+0000","last_unstale":"2024-04-23T10:25:00.667210+0000","last_undegraded":"2024-04-23T10:25:00.667210+0000","last_fullsized":"2024-04-23T10:25:00.667210+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T13:21:56.751028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.667535+0000","last_change":"2024-04-23T10:24:06.576607+0000","last_active":"2024-04-23T10:25:00.667535+0000","last_peered":"2024-04-23T10:25:00.667535+0000","last_clean":"2024-04-23T10:25:00.667535+0000","last_became_active":"2024-04-23T10:24:06.576213+0000","last_became_peered":"2024-04-23T10:24:06.576213+0000","last_unstale":"2024-04-23T10:25:00.667535+0000","last_undegraded":"2024-04-23T10:25:00.667535+0000","last_fullsized":"2024-04-23T10:25:00.667535+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:09:14.918936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.667860+0000","last_change":"2024-04-23T10:24:07.052641+0000","last_active":"2024-04-23T10:25:00.667860+0000","last_peered":"2024-04-23T10:25:00.667860+0000","last_clean":"2024-04-23T10:25:00.667860+0000","last_became_active":"2024-04-23T10:24:07.052314+0000","last_became_peered":"2024-04-23T10:24:07.052314+0000","last_unstale":"2024-04-23T10:25:00.667860+0000","last_undegraded":"2024-04-23T10:25:00.667860+0000","last_fullsized":"2024-04-23T10:25:00.667860+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T10:29:18.098800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'6","reported_seq":102,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:23.059719+0000","last_change":"2024-04-23T10:24:06.574570+0000","last_active":"2024-04-23T10:25:23.059719+0000","last_peered":"2024-04-23T10:25:23.059719+0000","last_clean":"2024-04-23T10:25:23.059719+0000","last_became_active":"2024-04-23T10:24:06.574166+0000","last_became_peered":"2024-04-23T10:24:06.574166+0000","last_unstale":"2024-04-23T10:25:23.059719+0000","last_undegraded":"2024-04-23T10:25:23.059719+0000","last_fullsized":"2024-04-23T10:25:23.059719+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:28:31.619100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":78,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.667617+0000","last_change":"2024-04-23T10:24:07.052825+0000","last_active":"2024-04-23T10:25:00.667617+0000","last_peered":"2024-04-23T10:25:00.667617+0000","last_clean":"2024-04-23T10:25:00.667617+0000","last_became_active":"2024-04-23T10:24:07.052457+0000","last_became_peered":"2024-04-23T10:24:07.052457+0000","last_unstale":"2024-04-23T10:25:00.667617+0000","last_undegraded":"2024-04-23T10:25:00.667617+0000","last_fullsized":"2024-04-23T10:25:00.667617+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:47:09.545374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.666369+0000","last_change":"2024-04-23T10:24:06.572168+0000","last_active":"2024-04-23T10:25:00.666369+0000","last_peered":"2024-04-23T10:25:00.666369+0000","last_clean":"2024-04-23T10:25:00.666369+0000","last_became_active":"2024-04-23T10:24:06.571805+0000","last_became_peered":"2024-04-23T10:24:06.571805+0000","last_unstale":"2024-04-23T10:25:00.666369+0000","last_undegraded":"2024-04-23T10:25:00.666369+0000","last_fullsized":"2024-04-23T10:25:00.666369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:29:56.817879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.668718+0000","last_change":"2024-04-23T10:24:07.048474+0000","last_active":"2024-04-23T10:25:00.668718+0000","last_peered":"2024-04-23T10:25:00.668718+0000","last_clean":"2024-04-23T10:25:00.668718+0000","last_became_active":"2024-04-23T10:24:07.047893+0000","last_became_peered":"2024-04-23T10:24:07.047893+0000","last_unstale":"2024-04-23T10:25:00.668718+0000","last_undegraded":"2024-04-23T10:25:00.668718+0000","last_fullsized":"2024-04-23T10:25:00.668718+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T12:59:46.545569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.667241+0000","last_change":"2024-04-23T10:24:06.570974+0000","last_active":"2024-04-23T10:25:00.667241+0000","last_peered":"2024-04-23T10:25:00.667241+0000","last_clean":"2024-04-23T10:25:00.667241+0000","last_became_active":"2024-04-23T10:24:06.570603+0000","last_became_peered":"2024-04-23T10:24:06.570603+0000","last_unstale":"2024-04-23T10:25:00.667241+0000","last_undegraded":"2024-04-23T10:25:00.667241+0000","last_fullsized":"2024-04-23T10:25:00.667241+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T12:46:33.518337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.664637+0000","last_change":"2024-04-23T10:24:07.050986+0000","last_active":"2024-04-23T10:25:00.664637+0000","last_peered":"2024-04-23T10:25:00.664637+0000","last_clean":"2024-04-23T10:25:00.664637+0000","last_became_active":"2024-04-23T10:24:07.050617+0000","last_became_peered":"2024-04-23T10:24:07.050617+0000","last_unstale":"2024-04-23T10:25:00.664637+0000","last_undegraded":"2024-04-23T10:25:00.664637+0000","last_fullsized":"2024-04-23T10:25:00.664637+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:13:25.355117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.663786+0000","last_change":"2024-04-23T10:24:06.579138+0000","last_active":"2024-04-23T10:25:00.663786+0000","last_peered":"2024-04-23T10:25:00.663786+0000","last_clean":"2024-04-23T10:25:00.663786+0000","last_became_active":"2024-04-23T10:24:06.578716+0000","last_became_peered":"2024-04-23T10:24:06.578716+0000","last_unstale":"2024-04-23T10:25:00.663786+0000","last_undegraded":"2024-04-23T10:25:00.663786+0000","last_fullsized":"2024-04-23T10:25:00.663786+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:06:59.797802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.665177+0000","last_change":"2024-04-23T10:24:07.039019+0000","last_active":"2024-04-23T10:25:00.665177+0000","last_peered":"2024-04-23T10:25:00.665177+0000","last_clean":"2024-04-23T10:25:00.665177+0000","last_became_active":"2024-04-23T10:24:07.038701+0000","last_became_peered":"2024-04-23T10:24:07.038701+0000","last_unstale":"2024-04-23T10:25:00.665177+0000","last_undegraded":"2024-04-23T10:25:00.665177+0000","last_fullsized":"2024-04-23T10:25:00.665177+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:26:32.716043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":97,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:23.761274+0000","last_change":"2024-04-23T10:24:06.574342+0000","last_active":"2024-04-23T10:25:23.761274+0000","last_peered":"2024-04-23T10:25:23.761274+0000","last_clean":"2024-04-23T10:25:23.761274+0000","last_became_active":"2024-04-23T10:24:06.573981+0000","last_became_peered":"2024-04-23T10:24:06.573981+0000","last_unstale":"2024-04-23T10:25:23.761274+0000","last_undegraded":"2024-04-23T10:25:23.761274+0000","last_fullsized":"2024-04-23T10:25:23.761274+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:48:40.325991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":73,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.665065+0000","last_change":"2024-04-23T10:24:07.044712+0000","last_active":"2024-04-23T10:25:00.665065+0000","last_peered":"2024-04-23T10:25:00.665065+0000","last_clean":"2024-04-23T10:25:00.665065+0000","last_became_active":"2024-04-23T10:24:07.044141+0000","last_became_peered":"2024-04-23T10:24:07.044141+0000","last_unstale":"2024-04-23T10:25:00.665065+0000","last_undegraded":"2024-04-23T10:25:00.665065+0000","last_fullsized":"2024-04-23T10:25:00.665065+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:14:38.818648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.729779+0000","last_change":"2024-04-23T10:25:00.729397+0000","last_active":"2024-04-23T10:25:00.729779+0000","last_peered":"2024-04-23T10:25:00.729779+0000","last_clean":"2024-04-23T10:25:00.729779+0000","last_became_active":"2024-04-23T10:25:00.729046+0000","last_became_peered":"2024-04-23T10:25:00.729046+0000","last_unstale":"2024-04-23T10:25:00.729779+0000","last_undegraded":"2024-04-23T10:25:00.729779+0000","last_fullsized":"2024-04-23T10:25:00.729779+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:25:55.189292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"86'5","reported_seq":99,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:23.058940+0000","last_change":"2024-04-23T10:24:06.578216+0000","last_active":"2024-04-23T10:25:23.058940+0000","last_peered":"2024-04-23T10:25:23.058940+0000","last_clean":"2024-04-23T10:25:23.058940+0000","last_became_active":"2024-04-23T10:24:06.577370+0000","last_became_peered":"2024-04-23T10:24:06.577370+0000","last_unstale":"2024-04-23T10:25:23.058940+0000","last_undegraded":"2024-04-23T10:25:23.058940+0000","last_fullsized":"2024-04-23T10:25:23.058940+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:02:06.554442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":75,"num_read_kb":75,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:01.274458+0000","last_change":"2024-04-23T10:24:07.039627+0000","last_active":"2024-04-23T10:25:01.274458+0000","last_peered":"2024-04-23T10:25:01.274458+0000","last_clean":"2024-04-23T10:25:01.274458+0000","last_became_active":"2024-04-23T10:24:07.039272+0000","last_became_peered":"2024-04-23T10:24:07.039272+0000","last_unstale":"2024-04-23T10:25:01.274458+0000","last_undegraded":"2024-04-23T10:25:01.274458+0000","last_fullsized":"2024-04-23T10:25:01.274458+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T13:45:44.277926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.728792+0000","last_change":"2024-04-23T10:24:06.569086+0000","last_active":"2024-04-23T10:25:00.728792+0000","last_peered":"2024-04-23T10:25:00.728792+0000","last_clean":"2024-04-23T10:25:00.728792+0000","last_became_active":"2024-04-23T10:24:06.568719+0000","last_became_peered":"2024-04-23T10:24:06.568719+0000","last_unstale":"2024-04-23T10:25:00.728792+0000","last_undegraded":"2024-04-23T10:25:00.728792+0000","last_fullsized":"2024-04-23T10:25:00.728792+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:22:10.583782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":354,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.667863+0000","last_change":"2024-04-23T10:22:49.957476+0000","last_active":"2024-04-23T10:25:00.667863+0000","last_peered":"2024-04-23T10:25:00.667863+0000","last_clean":"2024-04-23T10:25:00.667863+0000","last_became_active":"2024-04-23T10:22:49.957033+0000","last_became_peered":"2024-04-23T10:22:49.957033+0000","last_unstale":"2024-04-23T10:25:00.667863+0000","last_undegraded":"2024-04-23T10:25:00.667863+0000","last_fullsized":"2024-04-23T10:25:00.667863+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:19:25.604229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:19:25.604229+0000","last_clean_scrub_stamp":"2024-04-23T10:19:25.604229+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T12:56:17.601141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.671965+0000","last_change":"2024-04-23T10:25:00.671965+0000","last_active":"2024-04-23T10:25:00.671965+0000","last_peered":"2024-04-23T10:25:00.671965+0000","last_clean":"2024-04-23T10:25:00.671965+0000","last_became_active":"2024-04-23T10:25:00.671632+0000","last_became_peered":"2024-04-23T10:25:00.671632+0000","last_unstale":"2024-04-23T10:25:00.671965+0000","last_undegraded":"2024-04-23T10:25:00.671965+0000","last_fullsized":"2024-04-23T10:25:00.671965+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:08:40.262367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.668250+0000","last_change":"2024-04-23T10:24:06.582629+0000","last_active":"2024-04-23T10:25:00.668250+0000","last_peered":"2024-04-23T10:25:00.668250+0000","last_clean":"2024-04-23T10:25:00.668250+0000","last_became_active":"2024-04-23T10:24:06.582325+0000","last_became_peered":"2024-04-23T10:24:06.582325+0000","last_unstale":"2024-04-23T10:25:00.668250+0000","last_undegraded":"2024-04-23T10:25:00.668250+0000","last_fullsized":"2024-04-23T10:25:00.668250+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:57:09.879100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.666667+0000","last_change":"2024-04-23T10:24:06.582482+0000","last_active":"2024-04-23T10:25:00.666667+0000","last_peered":"2024-04-23T10:25:00.666667+0000","last_clean":"2024-04-23T10:25:00.666667+0000","last_became_active":"2024-04-23T10:24:06.582114+0000","last_became_peered":"2024-04-23T10:24:06.582114+0000","last_unstale":"2024-04-23T10:25:00.666667+0000","last_undegraded":"2024-04-23T10:25:00.666667+0000","last_fullsized":"2024-04-23T10:25:00.666667+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:45:42.939840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.667165+0000","last_change":"2024-04-23T10:24:07.031102+0000","last_active":"2024-04-23T10:25:00.667165+0000","last_peered":"2024-04-23T10:25:00.667165+0000","last_clean":"2024-04-23T10:25:00.667165+0000","last_became_active":"2024-04-23T10:24:07.030783+0000","last_became_peered":"2024-04-23T10:24:07.030783+0000","last_unstale":"2024-04-23T10:25:00.667165+0000","last_undegraded":"2024-04-23T10:25:00.667165+0000","last_fullsized":"2024-04-23T10:25:00.667165+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:49:46.736046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'12","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.664017+0000","last_change":"2024-04-23T10:24:06.577096+0000","last_active":"2024-04-23T10:25:00.664017+0000","last_peered":"2024-04-23T10:25:00.664017+0000","last_clean":"2024-04-23T10:25:00.664017+0000","last_became_active":"2024-04-23T10:24:06.576740+0000","last_became_peered":"2024-04-23T10:24:06.576740+0000","last_unstale":"2024-04-23T10:25:00.664017+0000","last_undegraded":"2024-04-23T10:25:00.664017+0000","last_fullsized":"2024-04-23T10:25:00.664017+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:44:06.532366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:01.274734+0000","last_change":"2024-04-23T10:24:07.043599+0000","last_active":"2024-04-23T10:25:01.274734+0000","last_peered":"2024-04-23T10:25:01.274734+0000","last_clean":"2024-04-23T10:25:01.274734+0000","last_became_active":"2024-04-23T10:24:07.042953+0000","last_became_peered":"2024-04-23T10:24:07.042953+0000","last_unstale":"2024-04-23T10:25:01.274734+0000","last_undegraded":"2024-04-23T10:25:01.274734+0000","last_fullsized":"2024-04-23T10:25:01.274734+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:07:50.487972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.728986+0000","last_change":"2024-04-23T10:24:06.570681+0000","last_active":"2024-04-23T10:25:00.728986+0000","last_peered":"2024-04-23T10:25:00.728986+0000","last_clean":"2024-04-23T10:25:00.728986+0000","last_became_active":"2024-04-23T10:24:06.570178+0000","last_became_peered":"2024-04-23T10:24:06.570178+0000","last_unstale":"2024-04-23T10:25:00.728986+0000","last_undegraded":"2024-04-23T10:25:00.728986+0000","last_fullsized":"2024-04-23T10:25:00.728986+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T12:16:56.629443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.669433+0000","last_change":"2024-04-23T10:24:07.040498+0000","last_active":"2024-04-23T10:25:00.669433+0000","last_peered":"2024-04-23T10:25:00.669433+0000","last_clean":"2024-04-23T10:25:00.669433+0000","last_became_active":"2024-04-23T10:24:07.040009+0000","last_became_peered":"2024-04-23T10:24:07.040009+0000","last_unstale":"2024-04-23T10:25:00.669433+0000","last_undegraded":"2024-04-23T10:25:00.669433+0000","last_fullsized":"2024-04-23T10:25:00.669433+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:43:17.319305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.673495+0000","last_change":"2024-04-23T10:25:00.673495+0000","last_active":"2024-04-23T10:25:00.673495+0000","last_peered":"2024-04-23T10:25:00.673495+0000","last_clean":"2024-04-23T10:25:00.673495+0000","last_became_active":"2024-04-23T10:25:00.670572+0000","last_became_peered":"2024-04-23T10:25:00.670572+0000","last_unstale":"2024-04-23T10:25:00.673495+0000","last_undegraded":"2024-04-23T10:25:00.673495+0000","last_fullsized":"2024-04-23T10:25:00.673495+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:08:04.279056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.666362+0000","last_change":"2024-04-23T10:24:07.043710+0000","last_active":"2024-04-23T10:25:00.666362+0000","last_peered":"2024-04-23T10:25:00.666362+0000","last_clean":"2024-04-23T10:25:00.666362+0000","last_became_active":"2024-04-23T10:24:07.043406+0000","last_became_peered":"2024-04-23T10:24:07.043406+0000","last_unstale":"2024-04-23T10:25:00.666362+0000","last_undegraded":"2024-04-23T10:25:00.666362+0000","last_fullsized":"2024-04-23T10:25:00.666362+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:09:14.471938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:01.274384+0000","last_change":"2024-04-23T10:24:06.577409+0000","last_active":"2024-04-23T10:25:01.274384+0000","last_peered":"2024-04-23T10:25:01.274384+0000","last_clean":"2024-04-23T10:25:01.274384+0000","last_became_active":"2024-04-23T10:24:06.576787+0000","last_became_peered":"2024-04-23T10:24:06.576787+0000","last_unstale":"2024-04-23T10:25:01.274384+0000","last_undegraded":"2024-04-23T10:25:01.274384+0000","last_fullsized":"2024-04-23T10:25:01.274384+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:05:28.423806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.664399+0000","last_change":"2024-04-23T10:24:07.045180+0000","last_active":"2024-04-23T10:25:00.664399+0000","last_peered":"2024-04-23T10:25:00.664399+0000","last_clean":"2024-04-23T10:25:00.664399+0000","last_became_active":"2024-04-23T10:24:07.044731+0000","last_became_peered":"2024-04-23T10:24:07.044731+0000","last_unstale":"2024-04-23T10:25:00.664399+0000","last_undegraded":"2024-04-23T10:25:00.664399+0000","last_fullsized":"2024-04-23T10:25:00.664399+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:42:34.474391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.669897+0000","last_change":"2024-04-23T10:24:06.578039+0000","last_active":"2024-04-23T10:25:00.669897+0000","last_peered":"2024-04-23T10:25:00.669897+0000","last_clean":"2024-04-23T10:25:00.669897+0000","last_became_active":"2024-04-23T10:24:06.577655+0000","last_became_peered":"2024-04-23T10:24:06.577655+0000","last_unstale":"2024-04-23T10:25:00.669897+0000","last_undegraded":"2024-04-23T10:25:00.669897+0000","last_fullsized":"2024-04-23T10:25:00.669897+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:57:35.172318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:01.274530+0000","last_change":"2024-04-23T10:24:07.037311+0000","last_active":"2024-04-23T10:25:01.274530+0000","last_peered":"2024-04-23T10:25:01.274530+0000","last_clean":"2024-04-23T10:25:01.274530+0000","last_became_active":"2024-04-23T10:24:07.036969+0000","last_became_peered":"2024-04-23T10:24:07.036969+0000","last_unstale":"2024-04-23T10:25:01.274530+0000","last_undegraded":"2024-04-23T10:25:01.274530+0000","last_fullsized":"2024-04-23T10:25:01.274530+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:13:18.998914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.666877+0000","last_change":"2024-04-23T10:24:06.575183+0000","last_active":"2024-04-23T10:25:00.666877+0000","last_peered":"2024-04-23T10:25:00.666877+0000","last_clean":"2024-04-23T10:25:00.666877+0000","last_became_active":"2024-04-23T10:24:06.574630+0000","last_became_peered":"2024-04-23T10:24:06.574630+0000","last_unstale":"2024-04-23T10:25:00.666877+0000","last_undegraded":"2024-04-23T10:25:00.666877+0000","last_fullsized":"2024-04-23T10:25:00.666877+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T11:18:18.189925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.667979+0000","last_change":"2024-04-23T10:24:07.048832+0000","last_active":"2024-04-23T10:25:00.667979+0000","last_peered":"2024-04-23T10:25:00.667979+0000","last_clean":"2024-04-23T10:25:00.667979+0000","last_became_active":"2024-04-23T10:24:07.048509+0000","last_became_peered":"2024-04-23T10:24:07.048509+0000","last_unstale":"2024-04-23T10:25:00.667979+0000","last_undegraded":"2024-04-23T10:25:00.667979+0000","last_fullsized":"2024-04-23T10:25:00.667979+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T13:45:26.793330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'6","reported_seq":99,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:23.081908+0000","last_change":"2024-04-23T10:24:06.575408+0000","last_active":"2024-04-23T10:25:23.081908+0000","last_peered":"2024-04-23T10:25:23.081908+0000","last_clean":"2024-04-23T10:25:23.081908+0000","last_became_active":"2024-04-23T10:24:06.575081+0000","last_became_peered":"2024-04-23T10:24:06.575081+0000","last_unstale":"2024-04-23T10:25:23.081908+0000","last_undegraded":"2024-04-23T10:25:23.081908+0000","last_fullsized":"2024-04-23T10:25:23.081908+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T11:28:12.765196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":74,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.668181+0000","last_change":"2024-04-23T10:24:07.043869+0000","last_active":"2024-04-23T10:25:00.668181+0000","last_peered":"2024-04-23T10:25:00.668181+0000","last_clean":"2024-04-23T10:25:00.668181+0000","last_became_active":"2024-04-23T10:24:07.043448+0000","last_became_peered":"2024-04-23T10:24:07.043448+0000","last_unstale":"2024-04-23T10:25:00.668181+0000","last_undegraded":"2024-04-23T10:25:00.668181+0000","last_fullsized":"2024-04-23T10:25:00.668181+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:00:35.821470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.667896+0000","last_change":"2024-04-23T10:24:06.575259+0000","last_active":"2024-04-23T10:25:00.667896+0000","last_peered":"2024-04-23T10:25:00.667896+0000","last_clean":"2024-04-23T10:25:00.667896+0000","last_became_active":"2024-04-23T10:24:06.574920+0000","last_became_peered":"2024-04-23T10:24:06.574920+0000","last_unstale":"2024-04-23T10:25:00.667896+0000","last_undegraded":"2024-04-23T10:25:00.667896+0000","last_fullsized":"2024-04-23T10:25:00.667896+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:52:14.558458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.667808+0000","last_change":"2024-04-23T10:24:07.043238+0000","last_active":"2024-04-23T10:25:00.667808+0000","last_peered":"2024-04-23T10:25:00.667808+0000","last_clean":"2024-04-23T10:25:00.667808+0000","last_became_active":"2024-04-23T10:24:07.042110+0000","last_became_peered":"2024-04-23T10:24:07.042110+0000","last_unstale":"2024-04-23T10:25:00.667808+0000","last_undegraded":"2024-04-23T10:25:00.667808+0000","last_fullsized":"2024-04-23T10:25:00.667808+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T12:36:13.474456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.667774+0000","last_change":"2024-04-23T10:24:06.579233+0000","last_active":"2024-04-23T10:25:00.667774+0000","last_peered":"2024-04-23T10:25:00.667774+0000","last_clean":"2024-04-23T10:25:00.667774+0000","last_became_active":"2024-04-23T10:24:06.578849+0000","last_became_peered":"2024-04-23T10:24:06.578849+0000","last_unstale":"2024-04-23T10:25:00.667774+0000","last_undegraded":"2024-04-23T10:25:00.667774+0000","last_fullsized":"2024-04-23T10:25:00.667774+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:34:23.193707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.668344+0000","last_change":"2024-04-23T10:24:07.049840+0000","last_active":"2024-04-23T10:25:00.668344+0000","last_peered":"2024-04-23T10:25:00.668344+0000","last_clean":"2024-04-23T10:25:00.668344+0000","last_became_active":"2024-04-23T10:24:07.049493+0000","last_became_peered":"2024-04-23T10:24:07.049493+0000","last_unstale":"2024-04-23T10:25:00.668344+0000","last_undegraded":"2024-04-23T10:25:00.668344+0000","last_fullsized":"2024-04-23T10:25:00.668344+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:03:22.606800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.664508+0000","last_change":"2024-04-23T10:24:07.052332+0000","last_active":"2024-04-23T10:25:00.664508+0000","last_peered":"2024-04-23T10:25:00.664508+0000","last_clean":"2024-04-23T10:25:00.664508+0000","last_became_active":"2024-04-23T10:24:07.051956+0000","last_became_peered":"2024-04-23T10:24:07.051956+0000","last_unstale":"2024-04-23T10:25:00.664508+0000","last_undegraded":"2024-04-23T10:25:00.664508+0000","last_fullsized":"2024-04-23T10:25:00.664508+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:56:43.606916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.669912+0000","last_change":"2024-04-23T10:24:07.041897+0000","last_active":"2024-04-23T10:25:00.669912+0000","last_peered":"2024-04-23T10:25:00.669912+0000","last_clean":"2024-04-23T10:25:00.669912+0000","last_became_active":"2024-04-23T10:24:07.041546+0000","last_became_peered":"2024-04-23T10:24:07.041546+0000","last_unstale":"2024-04-23T10:25:00.669912+0000","last_undegraded":"2024-04-23T10:25:00.669912+0000","last_fullsized":"2024-04-23T10:25:00.669912+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T11:31:23.211045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.669833+0000","last_change":"2024-04-23T10:24:07.040764+0000","last_active":"2024-04-23T10:25:00.669833+0000","last_peered":"2024-04-23T10:25:00.669833+0000","last_clean":"2024-04-23T10:25:00.669833+0000","last_became_active":"2024-04-23T10:24:07.039637+0000","last_became_peered":"2024-04-23T10:24:07.039637+0000","last_unstale":"2024-04-23T10:25:00.669833+0000","last_undegraded":"2024-04-23T10:25:00.669833+0000","last_fullsized":"2024-04-23T10:25:00.669833+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:50:33.298770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.669097+0000","last_change":"2024-04-23T10:24:07.039491+0000","last_active":"2024-04-23T10:25:00.669097+0000","last_peered":"2024-04-23T10:25:00.669097+0000","last_clean":"2024-04-23T10:25:00.669097+0000","last_became_active":"2024-04-23T10:24:07.039128+0000","last_became_peered":"2024-04-23T10:24:07.039128+0000","last_unstale":"2024-04-23T10:25:00.669097+0000","last_undegraded":"2024-04-23T10:25:00.669097+0000","last_fullsized":"2024-04-23T10:25:00.669097+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T12:30:40.326031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.671059+0000","last_change":"2024-04-23T10:25:00.671059+0000","last_active":"2024-04-23T10:25:00.671059+0000","last_peered":"2024-04-23T10:25:00.671059+0000","last_clean":"2024-04-23T10:25:00.671059+0000","last_became_active":"2024-04-23T10:25:00.670686+0000","last_became_peered":"2024-04-23T10:25:00.670686+0000","last_unstale":"2024-04-23T10:25:00.671059+0000","last_undegraded":"2024-04-23T10:25:00.671059+0000","last_fullsized":"2024-04-23T10:25:00.671059+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:04:39.991647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.664595+0000","last_change":"2024-04-23T10:24:07.052395+0000","last_active":"2024-04-23T10:25:00.664595+0000","last_peered":"2024-04-23T10:25:00.664595+0000","last_clean":"2024-04-23T10:25:00.664595+0000","last_became_active":"2024-04-23T10:24:07.052023+0000","last_became_peered":"2024-04-23T10:24:07.052023+0000","last_unstale":"2024-04-23T10:25:00.664595+0000","last_undegraded":"2024-04-23T10:25:00.664595+0000","last_fullsized":"2024-04-23T10:25:00.664595+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:25:52.517267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.671271+0000","last_change":"2024-04-23T10:24:07.042739+0000","last_active":"2024-04-23T10:25:00.671271+0000","last_peered":"2024-04-23T10:25:00.671271+0000","last_clean":"2024-04-23T10:25:00.671271+0000","last_became_active":"2024-04-23T10:24:07.042292+0000","last_became_peered":"2024-04-23T10:24:07.042292+0000","last_unstale":"2024-04-23T10:25:00.671271+0000","last_undegraded":"2024-04-23T10:25:00.671271+0000","last_fullsized":"2024-04-23T10:25:00.671271+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:14:23.229032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:01.274715+0000","last_change":"2024-04-23T10:24:07.042792+0000","last_active":"2024-04-23T10:25:01.274715+0000","last_peered":"2024-04-23T10:25:01.274715+0000","last_clean":"2024-04-23T10:25:01.274715+0000","last_became_active":"2024-04-23T10:24:07.040846+0000","last_became_peered":"2024-04-23T10:24:07.040846+0000","last_unstale":"2024-04-23T10:25:01.274715+0000","last_undegraded":"2024-04-23T10:25:01.274715+0000","last_fullsized":"2024-04-23T10:25:01.274715+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T13:50:24.973286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.668190+0000","last_change":"2024-04-23T10:24:07.040391+0000","last_active":"2024-04-23T10:25:00.668190+0000","last_peered":"2024-04-23T10:25:00.668190+0000","last_clean":"2024-04-23T10:25:00.668190+0000","last_became_active":"2024-04-23T10:24:07.040002+0000","last_became_peered":"2024-04-23T10:24:07.040002+0000","last_unstale":"2024-04-23T10:25:00.668190+0000","last_undegraded":"2024-04-23T10:25:00.668190+0000","last_fullsized":"2024-04-23T10:25:00.668190+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T10:30:13.922315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:01.274323+0000","last_change":"2024-04-23T10:24:07.040678+0000","last_active":"2024-04-23T10:25:01.274323+0000","last_peered":"2024-04-23T10:25:01.274323+0000","last_clean":"2024-04-23T10:25:01.274323+0000","last_became_active":"2024-04-23T10:24:07.039551+0000","last_became_peered":"2024-04-23T10:24:07.039551+0000","last_unstale":"2024-04-23T10:25:01.274323+0000","last_undegraded":"2024-04-23T10:25:01.274323+0000","last_fullsized":"2024-04-23T10:25:01.274323+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:03:50.467083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.666980+0000","last_change":"2024-04-23T10:24:07.044618+0000","last_active":"2024-04-23T10:25:00.666980+0000","last_peered":"2024-04-23T10:25:00.666980+0000","last_clean":"2024-04-23T10:25:00.666980+0000","last_became_active":"2024-04-23T10:24:07.042803+0000","last_became_peered":"2024-04-23T10:24:07.042803+0000","last_unstale":"2024-04-23T10:25:00.666980+0000","last_undegraded":"2024-04-23T10:25:00.666980+0000","last_fullsized":"2024-04-23T10:25:00.666980+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:37:46.872177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.667432+0000","last_change":"2024-04-23T10:24:07.043724+0000","last_active":"2024-04-23T10:25:00.667432+0000","last_peered":"2024-04-23T10:25:00.667432+0000","last_clean":"2024-04-23T10:25:00.667432+0000","last_became_active":"2024-04-23T10:24:07.043408+0000","last_became_peered":"2024-04-23T10:24:07.043408+0000","last_unstale":"2024-04-23T10:25:00.667432+0000","last_undegraded":"2024-04-23T10:25:00.667432+0000","last_fullsized":"2024-04-23T10:25:00.667432+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:18:46.615093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.664815+0000","last_change":"2024-04-23T10:24:07.043794+0000","last_active":"2024-04-23T10:25:00.664815+0000","last_peered":"2024-04-23T10:25:00.664815+0000","last_clean":"2024-04-23T10:25:00.664815+0000","last_became_active":"2024-04-23T10:24:07.043391+0000","last_became_peered":"2024-04-23T10:24:07.043391+0000","last_unstale":"2024-04-23T10:25:00.664815+0000","last_undegraded":"2024-04-23T10:25:00.664815+0000","last_fullsized":"2024-04-23T10:25:00.664815+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:06:04.584000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280003,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68641,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":751,"num_read_kb":443,"num_write":238,"num_write_kb":134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1254,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":205923,"data_compressed":5568,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":207,"ondisk_log_size":207,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351006,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":460744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612642,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1110016,"data_stored":1112237,"data_compressed":37471,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776167,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":459701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037806,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":458825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299443,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1032192,"data_stored":1052988,"data_compressed":35615,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462968,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":966656,"data_stored":459418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":521377,"data_compressed":1856,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738453,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":987136,"data_stored":461148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411402,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":522379,"data_compressed":1856,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149767,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":983040,"data_stored":459243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888130,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":974848,"data_stored":460342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724604,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1044480,"data_stored":1052275,"data_compressed":35615,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33976,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33976,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33976,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35615,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35615,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35615,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35615,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35615,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":62798,"data_compressed":1856,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":63800,"data_compressed":1856,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":63173,"data_compressed":1856,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T10:25:26.929 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-23T10:25:27.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:26 smithi196 ceph-mon[32919]: pgmap v331: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-23T10:25:27.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:26 smithi183 ceph-mon[32317]: pgmap v331: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-23T10:25:27.171 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T10:25:27.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:27 smithi097 ceph-mon[29197]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T10:25:28.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:27 smithi196 ceph-mon[32919]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T10:25:28.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:27 smithi183 ceph-mon[32317]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T10:25:28.743 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:28 smithi097 ceph-mon[29197]: pgmap v332: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-04-23T10:25:29.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:29.029+0000 7ff40f900700 1 -- 172.21.15.97:0/57815269 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff40806a750 msgr2=0x7ff40806abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:29.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:29.029+0000 7ff40f900700 1 --2- 172.21.15.97:0/57815269 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff40806a750 0x7ff40806abb0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff3f8009990 tx=0x7ff3f8009ca0 comp rx=0 tx=0).stop 2024-04-23T10:25:29.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:29.029+0000 7ff40f900700 1 -- 172.21.15.97:0/57815269 shutdown_connections 2024-04-23T10:25:29.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:29.029+0000 7ff40f900700 1 --2- 172.21.15.97:0/57815269 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff40806b0f0 0x7ff40810f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:29.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:29.029+0000 7ff40f900700 1 --2- 172.21.15.97:0/57815269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff408108a30 0x7ff408108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:29.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:29.029+0000 7ff40f900700 1 --2- 172.21.15.97:0/57815269 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff40806a750 0x7ff40806abb0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:29.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:29.029+0000 7ff40f900700 1 -- 172.21.15.97:0/57815269 >> 172.21.15.97:0/57815269 conn(0x7ff4080788f0 msgr2=0x7ff408078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:29.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:29.029+0000 7ff40f900700 1 -- 172.21.15.97:0/57815269 shutdown_connections 2024-04-23T10:25:29.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:29.029+0000 7ff40f900700 1 -- 172.21.15.97:0/57815269 wait complete. 2024-04-23T10:25:29.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:29.030+0000 7ff40f900700 1 Processor -- start 2024-04-23T10:25:29.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:29.030+0000 7ff40f900700 1 -- start start 2024-04-23T10:25:29.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:29.030+0000 7ff40f900700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff40806a750 0x7ff408199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:29.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:29.030+0000 7ff40f900700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff40806b0f0 0x7ff4081995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:29.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:29.030+0000 7ff40f900700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff408108a30 0x7ff40819d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:29.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:29.030+0000 7ff40f900700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff408112bd0 con 0x7ff40806a750 2024-04-23T10:25:29.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:29.030+0000 7ff40f900700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff408112a50 con 0x7ff408108a30 2024-04-23T10:25:29.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:29.030+0000 7ff40f900700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff408112d50 con 0x7ff40806b0f0 2024-04-23T10:25:29.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:29.030+0000 7ff40d69c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff40806a750 0x7ff408199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:29.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:29.030+0000 7ff40d69c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff40806a750 0x7ff408199080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33832/0 (socket says 172.21.15.97:33832) 2024-04-23T10:25:29.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:29.030+0000 7ff40d69c700 1 -- 172.21.15.97:0/868623410 learned_addr learned my addr 172.21.15.97:0/868623410 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:25:29.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:29.030+0000 7ff40ce9b700 1 --2- 172.21.15.97:0/868623410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff40806b0f0 0x7ff4081995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:29.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:29.030+0000 7ff40de9d700 1 --2- 172.21.15.97:0/868623410 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff408108a30 0x7ff40819d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:29.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:29.030+0000 7ff40d69c700 1 -- 172.21.15.97:0/868623410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff40806b0f0 msgr2=0x7ff4081995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:29.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:29.030+0000 7ff40d69c700 1 --2- 172.21.15.97:0/868623410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff40806b0f0 0x7ff4081995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:29.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:29.030+0000 7ff40d69c700 1 -- 172.21.15.97:0/868623410 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff408108a30 msgr2=0x7ff40819d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:29.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:29.030+0000 7ff40d69c700 1 --2- 172.21.15.97:0/868623410 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff408108a30 0x7ff40819d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:29.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:29.030+0000 7ff40d69c700 1 -- 172.21.15.97:0/868623410 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff40819e090 con 0x7ff40806a750 2024-04-23T10:25:29.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:29.030+0000 7ff40de9d700 1 --2- 172.21.15.97:0/868623410 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff408108a30 0x7ff40819d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:25:29.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:29.031+0000 7ff40d69c700 1 --2- 172.21.15.97:0/868623410 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff40806a750 0x7ff408199080 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff40400d980 tx=0x7ff40400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:29.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:29.031+0000 7ff3fe7fc700 1 -- 172.21.15.97:0/868623410 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff404016070 con 0x7ff40806a750 2024-04-23T10:25:29.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:29.031+0000 7ff3fe7fc700 1 -- 172.21.15.97:0/868623410 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff404004830 con 0x7ff40806a750 2024-04-23T10:25:29.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:29.031+0000 7ff40f900700 1 -- 172.21.15.97:0/868623410 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff40819e380 con 0x7ff40806a750 2024-04-23T10:25:29.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:29.031+0000 7ff3fe7fc700 1 -- 172.21.15.97:0/868623410 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4040054c0 con 0x7ff40806a750 2024-04-23T10:25:29.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:29.031+0000 7ff40f900700 1 -- 172.21.15.97:0/868623410 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4081a5c00 con 0x7ff40806a750 2024-04-23T10:25:29.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:29.032+0000 7ff40f900700 1 -- 172.21.15.97:0/868623410 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff408069730 con 0x7ff40806a750 2024-04-23T10:25:29.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:29.033+0000 7ff3fe7fc700 1 -- 172.21.15.97:0/868623410 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff404004220 con 0x7ff40806a750 2024-04-23T10:25:29.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:29.033+0000 7ff3fe7fc700 1 --2- 172.21.15.97:0/868623410 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7ff3f40a3cd0 0x7ff3f40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:29.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:29.034+0000 7ff40ce9b700 1 --2- 172.21.15.97:0/868623410 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7ff3f40a3cd0 0x7ff3f40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:29.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:29.034+0000 7ff3fe7fc700 1 -- 172.21.15.97:0/868623410 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7ff4040d02d0 con 0x7ff40806a750 2024-04-23T10:25:29.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:29.034+0000 7ff40ce9b700 1 --2- 172.21.15.97:0/868623410 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7ff3f40a3cd0 0x7ff3f40a6190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff3f80098e0 tx=0x7ff3f8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:29.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:29.038+0000 7ff3fe7fc700 1 -- 172.21.15.97:0/868623410 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff40401f020 con 0x7ff40806a750 2024-04-23T10:25:29.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:28 smithi196 ceph-mon[32919]: pgmap v332: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-04-23T10:25:29.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:28 smithi183 ceph-mon[32317]: pgmap v332: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-04-23T10:25:29.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:29.241+0000 7ff40f900700 1 -- 172.21.15.97:0/868623410 --> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff40819a3e0 con 0x7ff3f40a3cd0 2024-04-23T10:25:29.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:29.248+0000 7ff3fe7fc700 1 -- 172.21.15.97:0/868623410 <== mgr.14152 v2:172.21.15.97:6800/3149043749 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258989 (secure 0 0 0) 0x7ff40819a3e0 con 0x7ff3f40a3cd0 2024-04-23T10:25:29.250 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:25:29.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:29.252+0000 7ff40f900700 1 -- 172.21.15.97:0/868623410 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7ff3f40a3cd0 msgr2=0x7ff3f40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:29.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:29.252+0000 7ff40f900700 1 --2- 172.21.15.97:0/868623410 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7ff3f40a3cd0 0x7ff3f40a6190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff3f80098e0 tx=0x7ff3f8005960 comp rx=0 tx=0).stop 2024-04-23T10:25:29.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:29.252+0000 7ff40f900700 1 -- 172.21.15.97:0/868623410 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff40806a750 msgr2=0x7ff408199080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:29.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:29.252+0000 7ff40f900700 1 --2- 172.21.15.97:0/868623410 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff40806a750 0x7ff408199080 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff40400d980 tx=0x7ff40400dc90 comp rx=0 tx=0).stop 2024-04-23T10:25:29.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:29.252+0000 7ff40f900700 1 -- 172.21.15.97:0/868623410 shutdown_connections 2024-04-23T10:25:29.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:29.252+0000 7ff40f900700 1 --2- 172.21.15.97:0/868623410 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7ff3f40a3cd0 0x7ff3f40a6190 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:29.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:29.252+0000 7ff40f900700 1 --2- 172.21.15.97:0/868623410 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff408108a30 0x7ff40819d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:29.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:29.252+0000 7ff40f900700 1 --2- 172.21.15.97:0/868623410 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff40806a750 0x7ff408199080 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:29.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:29.252+0000 7ff40f900700 1 --2- 172.21.15.97:0/868623410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff40806b0f0 0x7ff4081995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:29.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:29.252+0000 7ff40f900700 1 -- 172.21.15.97:0/868623410 >> 172.21.15.97:0/868623410 conn(0x7ff4080788f0 msgr2=0x7ff4080ff910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:29.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:29.252+0000 7ff40f900700 1 -- 172.21.15.97:0/868623410 shutdown_connections 2024-04-23T10:25:29.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:29.252+0000 7ff40f900700 1 -- 172.21.15.97:0/868623410 wait complete. 2024-04-23T10:25:29.253 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-04-23T10:25:30.001 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":332,"stamp":"2024-04-23T10:25:27.572719+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659243,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":907,"num_read_kb":577,"num_write":421,"num_write_kb":3074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1181990,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":346,"ondisk_log_size":346,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361016,"kb_used_data":11796,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580808,"statfs":{"total":1151940427776,"available":1151570747392,"internally_reserved":0,"allocated":12079104,"data_stored":7480677,"data_compressed":112413,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":120,"num_read_kb":65,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002951"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.664727+0000","last_change":"2024-04-23T10:24:07.040397+0000","last_active":"2024-04-23T10:25:00.664727+0000","last_peered":"2024-04-23T10:25:00.664727+0000","last_clean":"2024-04-23T10:25:00.664727+0000","last_became_active":"2024-04-23T10:24:07.040007+0000","last_became_peered":"2024-04-23T10:24:07.040007+0000","last_unstale":"2024-04-23T10:25:00.664727+0000","last_undegraded":"2024-04-23T10:25:00.664727+0000","last_fullsized":"2024-04-23T10:25:00.664727+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T13:50:24.533115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.669176+0000","last_change":"2024-04-23T10:24:07.044672+0000","last_active":"2024-04-23T10:25:00.669176+0000","last_peered":"2024-04-23T10:25:00.669176+0000","last_clean":"2024-04-23T10:25:00.669176+0000","last_became_active":"2024-04-23T10:24:07.044184+0000","last_became_peered":"2024-04-23T10:24:07.044184+0000","last_unstale":"2024-04-23T10:25:00.669176+0000","last_undegraded":"2024-04-23T10:25:00.669176+0000","last_fullsized":"2024-04-23T10:25:00.669176+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:07:16.729205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.669118+0000","last_change":"2024-04-23T10:24:07.041959+0000","last_active":"2024-04-23T10:25:00.669118+0000","last_peered":"2024-04-23T10:25:00.669118+0000","last_clean":"2024-04-23T10:25:00.669118+0000","last_became_active":"2024-04-23T10:24:07.041661+0000","last_became_peered":"2024-04-23T10:24:07.041661+0000","last_unstale":"2024-04-23T10:25:00.669118+0000","last_undegraded":"2024-04-23T10:25:00.669118+0000","last_fullsized":"2024-04-23T10:25:00.669118+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:23:14.694183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.728712+0000","last_change":"2024-04-23T10:24:07.048334+0000","last_active":"2024-04-23T10:25:00.728712+0000","last_peered":"2024-04-23T10:25:00.728712+0000","last_clean":"2024-04-23T10:25:00.728712+0000","last_became_active":"2024-04-23T10:24:07.047506+0000","last_became_peered":"2024-04-23T10:24:07.047506+0000","last_unstale":"2024-04-23T10:25:00.728712+0000","last_undegraded":"2024-04-23T10:25:00.728712+0000","last_fullsized":"2024-04-23T10:25:00.728712+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:27:27.674287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.666897+0000","last_change":"2024-04-23T10:24:07.035531+0000","last_active":"2024-04-23T10:25:00.666897+0000","last_peered":"2024-04-23T10:25:00.666897+0000","last_clean":"2024-04-23T10:25:00.666897+0000","last_became_active":"2024-04-23T10:24:07.035167+0000","last_became_peered":"2024-04-23T10:24:07.035167+0000","last_unstale":"2024-04-23T10:25:00.666897+0000","last_undegraded":"2024-04-23T10:25:00.666897+0000","last_fullsized":"2024-04-23T10:25:00.666897+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:39:17.198650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.664824+0000","last_change":"2024-04-23T10:24:07.040593+0000","last_active":"2024-04-23T10:25:00.664824+0000","last_peered":"2024-04-23T10:25:00.664824+0000","last_clean":"2024-04-23T10:25:00.664824+0000","last_became_active":"2024-04-23T10:24:07.040255+0000","last_became_peered":"2024-04-23T10:24:07.040255+0000","last_unstale":"2024-04-23T10:25:00.664824+0000","last_undegraded":"2024-04-23T10:25:00.664824+0000","last_fullsized":"2024-04-23T10:25:00.664824+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:05:52.663566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.667615+0000","last_change":"2024-04-23T10:24:07.051688+0000","last_active":"2024-04-23T10:25:00.667615+0000","last_peered":"2024-04-23T10:25:00.667615+0000","last_clean":"2024-04-23T10:25:00.667615+0000","last_became_active":"2024-04-23T10:24:07.050265+0000","last_became_peered":"2024-04-23T10:24:07.050265+0000","last_unstale":"2024-04-23T10:25:00.667615+0000","last_undegraded":"2024-04-23T10:25:00.667615+0000","last_fullsized":"2024-04-23T10:25:00.667615+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T13:13:36.126183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:01.274280+0000","last_change":"2024-04-23T10:24:07.045937+0000","last_active":"2024-04-23T10:25:01.274280+0000","last_peered":"2024-04-23T10:25:01.274280+0000","last_clean":"2024-04-23T10:25:01.274280+0000","last_became_active":"2024-04-23T10:24:07.045616+0000","last_became_peered":"2024-04-23T10:24:07.045616+0000","last_unstale":"2024-04-23T10:25:01.274280+0000","last_undegraded":"2024-04-23T10:25:01.274280+0000","last_fullsized":"2024-04-23T10:25:01.274280+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T10:45:27.250031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.667488+0000","last_change":"2024-04-23T10:24:07.039294+0000","last_active":"2024-04-23T10:25:00.667488+0000","last_peered":"2024-04-23T10:25:00.667488+0000","last_clean":"2024-04-23T10:25:00.667488+0000","last_became_active":"2024-04-23T10:24:07.038959+0000","last_became_peered":"2024-04-23T10:24:07.038959+0000","last_unstale":"2024-04-23T10:25:00.667488+0000","last_undegraded":"2024-04-23T10:25:00.667488+0000","last_fullsized":"2024-04-23T10:25:00.667488+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:53:41.345062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.665337+0000","last_change":"2024-04-23T10:24:07.044177+0000","last_active":"2024-04-23T10:25:00.665337+0000","last_peered":"2024-04-23T10:25:00.665337+0000","last_clean":"2024-04-23T10:25:00.665337+0000","last_became_active":"2024-04-23T10:24:07.043821+0000","last_became_peered":"2024-04-23T10:24:07.043821+0000","last_unstale":"2024-04-23T10:25:00.665337+0000","last_undegraded":"2024-04-23T10:25:00.665337+0000","last_fullsized":"2024-04-23T10:25:00.665337+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:09:38.782407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.667941+0000","last_change":"2024-04-23T10:24:07.052775+0000","last_active":"2024-04-23T10:25:00.667941+0000","last_peered":"2024-04-23T10:25:00.667941+0000","last_clean":"2024-04-23T10:25:00.667941+0000","last_became_active":"2024-04-23T10:24:07.051708+0000","last_became_peered":"2024-04-23T10:24:07.051708+0000","last_unstale":"2024-04-23T10:25:00.667941+0000","last_undegraded":"2024-04-23T10:25:00.667941+0000","last_fullsized":"2024-04-23T10:25:00.667941+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:47:55.360467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.728601+0000","last_change":"2024-04-23T10:24:07.048655+0000","last_active":"2024-04-23T10:25:00.728601+0000","last_peered":"2024-04-23T10:25:00.728601+0000","last_clean":"2024-04-23T10:25:00.728601+0000","last_became_active":"2024-04-23T10:24:07.047994+0000","last_became_peered":"2024-04-23T10:24:07.047994+0000","last_unstale":"2024-04-23T10:25:00.728601+0000","last_undegraded":"2024-04-23T10:25:00.728601+0000","last_fullsized":"2024-04-23T10:25:00.728601+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T10:25:51.425871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.667525+0000","last_change":"2024-04-23T10:24:07.036319+0000","last_active":"2024-04-23T10:25:00.667525+0000","last_peered":"2024-04-23T10:25:00.667525+0000","last_clean":"2024-04-23T10:25:00.667525+0000","last_became_active":"2024-04-23T10:24:07.035982+0000","last_became_peered":"2024-04-23T10:24:07.035982+0000","last_unstale":"2024-04-23T10:25:00.667525+0000","last_undegraded":"2024-04-23T10:25:00.667525+0000","last_fullsized":"2024-04-23T10:25:00.667525+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T10:25:46.191118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.669516+0000","last_change":"2024-04-23T10:24:07.033166+0000","last_active":"2024-04-23T10:25:00.669516+0000","last_peered":"2024-04-23T10:25:00.669516+0000","last_clean":"2024-04-23T10:25:00.669516+0000","last_became_active":"2024-04-23T10:24:07.032807+0000","last_became_peered":"2024-04-23T10:24:07.032807+0000","last_unstale":"2024-04-23T10:25:00.669516+0000","last_undegraded":"2024-04-23T10:25:00.669516+0000","last_fullsized":"2024-04-23T10:25:00.669516+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:52:35.654965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.668855+0000","last_change":"2024-04-23T10:24:07.041988+0000","last_active":"2024-04-23T10:25:00.668855+0000","last_peered":"2024-04-23T10:25:00.668855+0000","last_clean":"2024-04-23T10:25:00.668855+0000","last_became_active":"2024-04-23T10:24:07.041638+0000","last_became_peered":"2024-04-23T10:24:07.041638+0000","last_unstale":"2024-04-23T10:25:00.668855+0000","last_undegraded":"2024-04-23T10:25:00.668855+0000","last_fullsized":"2024-04-23T10:25:00.668855+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:17:40.565981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.666563+0000","last_change":"2024-04-23T10:24:07.045111+0000","last_active":"2024-04-23T10:25:00.666563+0000","last_peered":"2024-04-23T10:25:00.666563+0000","last_clean":"2024-04-23T10:25:00.666563+0000","last_became_active":"2024-04-23T10:24:07.044249+0000","last_became_peered":"2024-04-23T10:24:07.044249+0000","last_unstale":"2024-04-23T10:25:00.666563+0000","last_undegraded":"2024-04-23T10:25:00.666563+0000","last_fullsized":"2024-04-23T10:25:00.666563+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:08:36.029636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.664830+0000","last_change":"2024-04-23T10:24:07.040685+0000","last_active":"2024-04-23T10:25:00.664830+0000","last_peered":"2024-04-23T10:25:00.664830+0000","last_clean":"2024-04-23T10:25:00.664830+0000","last_became_active":"2024-04-23T10:24:07.038881+0000","last_became_peered":"2024-04-23T10:24:07.038881+0000","last_unstale":"2024-04-23T10:25:00.664830+0000","last_undegraded":"2024-04-23T10:25:00.664830+0000","last_fullsized":"2024-04-23T10:25:00.664830+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:29:12.489136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.664056+0000","last_change":"2024-04-23T10:24:07.041815+0000","last_active":"2024-04-23T10:25:00.664056+0000","last_peered":"2024-04-23T10:25:00.664056+0000","last_clean":"2024-04-23T10:25:00.664056+0000","last_became_active":"2024-04-23T10:24:07.041495+0000","last_became_peered":"2024-04-23T10:24:07.041495+0000","last_unstale":"2024-04-23T10:25:00.664056+0000","last_undegraded":"2024-04-23T10:25:00.664056+0000","last_fullsized":"2024-04-23T10:25:00.664056+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:29:02.006394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.670731+0000","last_change":"2024-04-23T10:24:07.041722+0000","last_active":"2024-04-23T10:25:00.670731+0000","last_peered":"2024-04-23T10:25:00.670731+0000","last_clean":"2024-04-23T10:25:00.670731+0000","last_became_active":"2024-04-23T10:24:07.039434+0000","last_became_peered":"2024-04-23T10:24:07.039434+0000","last_unstale":"2024-04-23T10:25:00.670731+0000","last_undegraded":"2024-04-23T10:25:00.670731+0000","last_fullsized":"2024-04-23T10:25:00.670731+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:03:44.033094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'13","reported_seq":122,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:20.756677+0000","last_change":"2024-04-23T10:25:00.739010+0000","last_active":"2024-04-23T10:25:20.756677+0000","last_peered":"2024-04-23T10:25:20.756677+0000","last_clean":"2024-04-23T10:25:20.756677+0000","last_became_active":"2024-04-23T10:25:00.729834+0000","last_became_peered":"2024-04-23T10:25:00.729834+0000","last_unstale":"2024-04-23T10:25:20.756677+0000","last_undegraded":"2024-04-23T10:25:20.756677+0000","last_fullsized":"2024-04-23T10:25:20.756677+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T12:57:45.871560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":65,"num_read_kb":65,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,4],"acting":[9,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:01.274678+0000","last_change":"2024-04-23T10:24:07.047192+0000","last_active":"2024-04-23T10:25:01.274678+0000","last_peered":"2024-04-23T10:25:01.274678+0000","last_clean":"2024-04-23T10:25:01.274678+0000","last_became_active":"2024-04-23T10:24:07.046750+0000","last_became_peered":"2024-04-23T10:24:07.046750+0000","last_unstale":"2024-04-23T10:25:01.274678+0000","last_undegraded":"2024-04-23T10:25:01.274678+0000","last_fullsized":"2024-04-23T10:25:01.274678+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:13:09.399425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:01.274791+0000","last_change":"2024-04-23T10:24:06.577588+0000","last_active":"2024-04-23T10:25:01.274791+0000","last_peered":"2024-04-23T10:25:01.274791+0000","last_clean":"2024-04-23T10:25:01.274791+0000","last_became_active":"2024-04-23T10:24:06.576971+0000","last_became_peered":"2024-04-23T10:24:06.576971+0000","last_unstale":"2024-04-23T10:25:01.274791+0000","last_undegraded":"2024-04-23T10:25:01.274791+0000","last_fullsized":"2024-04-23T10:25:01.274791+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:07:21.577868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.669348+0000","last_change":"2024-04-23T10:24:07.030666+0000","last_active":"2024-04-23T10:25:00.669348+0000","last_peered":"2024-04-23T10:25:00.669348+0000","last_clean":"2024-04-23T10:25:00.669348+0000","last_became_active":"2024-04-23T10:24:07.030342+0000","last_became_peered":"2024-04-23T10:24:07.030342+0000","last_unstale":"2024-04-23T10:25:00.669348+0000","last_undegraded":"2024-04-23T10:25:00.669348+0000","last_fullsized":"2024-04-23T10:25:00.669348+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:09:14.818638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.666771+0000","last_change":"2024-04-23T10:24:06.576042+0000","last_active":"2024-04-23T10:25:00.666771+0000","last_peered":"2024-04-23T10:25:00.666771+0000","last_clean":"2024-04-23T10:25:00.666771+0000","last_became_active":"2024-04-23T10:24:06.574518+0000","last_became_peered":"2024-04-23T10:24:06.574518+0000","last_unstale":"2024-04-23T10:25:00.666771+0000","last_undegraded":"2024-04-23T10:25:00.666771+0000","last_fullsized":"2024-04-23T10:25:00.666771+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T11:56:40.100213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.666998+0000","last_change":"2024-04-23T10:24:07.046591+0000","last_active":"2024-04-23T10:25:00.666998+0000","last_peered":"2024-04-23T10:25:00.666998+0000","last_clean":"2024-04-23T10:25:00.666998+0000","last_became_active":"2024-04-23T10:24:07.046208+0000","last_became_peered":"2024-04-23T10:24:07.046208+0000","last_unstale":"2024-04-23T10:25:00.666998+0000","last_undegraded":"2024-04-23T10:25:00.666998+0000","last_fullsized":"2024-04-23T10:25:00.666998+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:27:37.711329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.676709+0000","last_change":"2024-04-23T10:25:00.676255+0000","last_active":"2024-04-23T10:25:00.676709+0000","last_peered":"2024-04-23T10:25:00.676709+0000","last_clean":"2024-04-23T10:25:00.676709+0000","last_became_active":"2024-04-23T10:25:00.670146+0000","last_became_peered":"2024-04-23T10:25:00.670146+0000","last_unstale":"2024-04-23T10:25:00.676709+0000","last_undegraded":"2024-04-23T10:25:00.676709+0000","last_fullsized":"2024-04-23T10:25:00.676709+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:34:54.217317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.668016+0000","last_change":"2024-04-23T10:24:07.047926+0000","last_active":"2024-04-23T10:25:00.668016+0000","last_peered":"2024-04-23T10:25:00.668016+0000","last_clean":"2024-04-23T10:25:00.668016+0000","last_became_active":"2024-04-23T10:24:07.047273+0000","last_became_peered":"2024-04-23T10:24:07.047273+0000","last_unstale":"2024-04-23T10:25:00.668016+0000","last_undegraded":"2024-04-23T10:25:00.668016+0000","last_fullsized":"2024-04-23T10:25:00.668016+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:37:25.068034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'6","reported_seq":102,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:23.052202+0000","last_change":"2024-04-23T10:24:06.574733+0000","last_active":"2024-04-23T10:25:23.052202+0000","last_peered":"2024-04-23T10:25:23.052202+0000","last_clean":"2024-04-23T10:25:23.052202+0000","last_became_active":"2024-04-23T10:24:06.574276+0000","last_became_peered":"2024-04-23T10:24:06.574276+0000","last_unstale":"2024-04-23T10:25:23.052202+0000","last_undegraded":"2024-04-23T10:25:23.052202+0000","last_fullsized":"2024-04-23T10:25:23.052202+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:27:57.080635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":77,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.666430+0000","last_change":"2024-04-23T10:24:07.040110+0000","last_active":"2024-04-23T10:25:00.666430+0000","last_peered":"2024-04-23T10:25:00.666430+0000","last_clean":"2024-04-23T10:25:00.666430+0000","last_became_active":"2024-04-23T10:24:07.039769+0000","last_became_peered":"2024-04-23T10:24:07.039769+0000","last_unstale":"2024-04-23T10:25:00.666430+0000","last_undegraded":"2024-04-23T10:25:00.666430+0000","last_fullsized":"2024-04-23T10:25:00.666430+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:10:07.884895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.664807+0000","last_change":"2024-04-23T10:24:06.567448+0000","last_active":"2024-04-23T10:25:00.664807+0000","last_peered":"2024-04-23T10:25:00.664807+0000","last_clean":"2024-04-23T10:25:00.664807+0000","last_became_active":"2024-04-23T10:24:06.566958+0000","last_became_peered":"2024-04-23T10:24:06.566958+0000","last_unstale":"2024-04-23T10:25:00.664807+0000","last_undegraded":"2024-04-23T10:25:00.664807+0000","last_fullsized":"2024-04-23T10:25:00.664807+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:45:52.863596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.667724+0000","last_change":"2024-04-23T10:24:07.040754+0000","last_active":"2024-04-23T10:25:00.667724+0000","last_peered":"2024-04-23T10:25:00.667724+0000","last_clean":"2024-04-23T10:25:00.667724+0000","last_became_active":"2024-04-23T10:24:07.040397+0000","last_became_peered":"2024-04-23T10:24:07.040397+0000","last_unstale":"2024-04-23T10:25:00.667724+0000","last_undegraded":"2024-04-23T10:25:00.667724+0000","last_fullsized":"2024-04-23T10:25:00.667724+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T11:01:03.626506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.667441+0000","last_change":"2024-04-23T10:24:06.568592+0000","last_active":"2024-04-23T10:25:00.667441+0000","last_peered":"2024-04-23T10:25:00.667441+0000","last_clean":"2024-04-23T10:25:00.667441+0000","last_became_active":"2024-04-23T10:24:06.568192+0000","last_became_peered":"2024-04-23T10:24:06.568192+0000","last_unstale":"2024-04-23T10:25:00.667441+0000","last_undegraded":"2024-04-23T10:25:00.667441+0000","last_fullsized":"2024-04-23T10:25:00.667441+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:14:42.589596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:01.274328+0000","last_change":"2024-04-23T10:24:07.037058+0000","last_active":"2024-04-23T10:25:01.274328+0000","last_peered":"2024-04-23T10:25:01.274328+0000","last_clean":"2024-04-23T10:25:01.274328+0000","last_became_active":"2024-04-23T10:24:07.036659+0000","last_became_peered":"2024-04-23T10:24:07.036659+0000","last_unstale":"2024-04-23T10:25:01.274328+0000","last_undegraded":"2024-04-23T10:25:01.274328+0000","last_fullsized":"2024-04-23T10:25:01.274328+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T12:53:49.298603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.668070+0000","last_change":"2024-04-23T10:24:06.576900+0000","last_active":"2024-04-23T10:25:00.668070+0000","last_peered":"2024-04-23T10:25:00.668070+0000","last_clean":"2024-04-23T10:25:00.668070+0000","last_became_active":"2024-04-23T10:24:06.575627+0000","last_became_peered":"2024-04-23T10:24:06.575627+0000","last_unstale":"2024-04-23T10:25:00.668070+0000","last_undegraded":"2024-04-23T10:25:00.668070+0000","last_fullsized":"2024-04-23T10:25:00.668070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:16:07.973841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.663790+0000","last_change":"2024-04-23T10:24:07.035294+0000","last_active":"2024-04-23T10:25:00.663790+0000","last_peered":"2024-04-23T10:25:00.663790+0000","last_clean":"2024-04-23T10:25:00.663790+0000","last_became_active":"2024-04-23T10:24:07.035001+0000","last_became_peered":"2024-04-23T10:24:07.035001+0000","last_unstale":"2024-04-23T10:25:00.663790+0000","last_undegraded":"2024-04-23T10:25:00.663790+0000","last_fullsized":"2024-04-23T10:25:00.663790+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T13:12:47.013132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'4","reported_seq":173,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:26.055930+0000","last_change":"2024-04-23T10:24:06.573637+0000","last_active":"2024-04-23T10:25:26.055930+0000","last_peered":"2024-04-23T10:25:26.055930+0000","last_clean":"2024-04-23T10:25:26.055930+0000","last_became_active":"2024-04-23T10:24:06.573236+0000","last_became_peered":"2024-04-23T10:24:06.573236+0000","last_unstale":"2024-04-23T10:25:26.055930+0000","last_undegraded":"2024-04-23T10:25:26.055930+0000","last_fullsized":"2024-04-23T10:25:26.055930+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T12:48:06.740181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":150,"num_read_kb":150,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.667203+0000","last_change":"2024-04-23T10:24:07.037457+0000","last_active":"2024-04-23T10:25:00.667203+0000","last_peered":"2024-04-23T10:25:00.667203+0000","last_clean":"2024-04-23T10:25:00.667203+0000","last_became_active":"2024-04-23T10:24:07.036456+0000","last_became_peered":"2024-04-23T10:24:07.036456+0000","last_unstale":"2024-04-23T10:25:00.667203+0000","last_undegraded":"2024-04-23T10:25:00.667203+0000","last_fullsized":"2024-04-23T10:25:00.667203+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:42:57.750435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.741712+0000","last_change":"2024-04-23T10:25:00.741712+0000","last_active":"2024-04-23T10:25:00.741712+0000","last_peered":"2024-04-23T10:25:00.741712+0000","last_clean":"2024-04-23T10:25:00.741712+0000","last_became_active":"2024-04-23T10:25:00.730130+0000","last_became_peered":"2024-04-23T10:25:00.730130+0000","last_unstale":"2024-04-23T10:25:00.741712+0000","last_undegraded":"2024-04-23T10:25:00.741712+0000","last_fullsized":"2024-04-23T10:25:00.741712+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:08:14.409376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.669252+0000","last_change":"2024-04-23T10:24:07.040317+0000","last_active":"2024-04-23T10:25:00.669252+0000","last_peered":"2024-04-23T10:25:00.669252+0000","last_clean":"2024-04-23T10:25:00.669252+0000","last_became_active":"2024-04-23T10:24:07.039998+0000","last_became_peered":"2024-04-23T10:24:07.039998+0000","last_unstale":"2024-04-23T10:25:00.669252+0000","last_undegraded":"2024-04-23T10:25:00.669252+0000","last_fullsized":"2024-04-23T10:25:00.669252+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:50:30.412127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.668115+0000","last_change":"2024-04-23T10:24:06.570019+0000","last_active":"2024-04-23T10:25:00.668115+0000","last_peered":"2024-04-23T10:25:00.668115+0000","last_clean":"2024-04-23T10:25:00.668115+0000","last_became_active":"2024-04-23T10:24:06.569676+0000","last_became_peered":"2024-04-23T10:24:06.569676+0000","last_unstale":"2024-04-23T10:25:00.668115+0000","last_undegraded":"2024-04-23T10:25:00.668115+0000","last_fullsized":"2024-04-23T10:25:00.668115+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:31:46.994787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.667134+0000","last_change":"2024-04-23T10:24:07.040183+0000","last_active":"2024-04-23T10:25:00.667134+0000","last_peered":"2024-04-23T10:25:00.667134+0000","last_clean":"2024-04-23T10:25:00.667134+0000","last_became_active":"2024-04-23T10:24:07.039866+0000","last_became_peered":"2024-04-23T10:24:07.039866+0000","last_unstale":"2024-04-23T10:25:00.667134+0000","last_undegraded":"2024-04-23T10:25:00.667134+0000","last_fullsized":"2024-04-23T10:25:00.667134+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:33:04.411765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'30","reported_seq":209,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:26.055930+0000","last_change":"2024-04-23T10:24:06.568883+0000","last_active":"2024-04-23T10:25:26.055930+0000","last_peered":"2024-04-23T10:25:26.055930+0000","last_clean":"2024-04-23T10:25:26.055930+0000","last_became_active":"2024-04-23T10:24:06.568517+0000","last_became_peered":"2024-04-23T10:24:06.568517+0000","last_unstale":"2024-04-23T10:25:26.055930+0000","last_undegraded":"2024-04-23T10:25:26.055930+0000","last_fullsized":"2024-04-23T10:25:26.055930+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:51:42.440346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61046,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":160,"num_read_kb":139,"num_write":32,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.666600+0000","last_change":"2024-04-23T10:24:07.046839+0000","last_active":"2024-04-23T10:25:00.666600+0000","last_peered":"2024-04-23T10:25:00.666600+0000","last_clean":"2024-04-23T10:25:00.666600+0000","last_became_active":"2024-04-23T10:24:07.046512+0000","last_became_peered":"2024-04-23T10:24:07.046512+0000","last_unstale":"2024-04-23T10:25:00.666600+0000","last_undegraded":"2024-04-23T10:25:00.666600+0000","last_fullsized":"2024-04-23T10:25:00.666600+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T12:42:11.822100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.670189+0000","last_change":"2024-04-23T10:24:06.578087+0000","last_active":"2024-04-23T10:25:00.670189+0000","last_peered":"2024-04-23T10:25:00.670189+0000","last_clean":"2024-04-23T10:25:00.670189+0000","last_became_active":"2024-04-23T10:24:06.577666+0000","last_became_peered":"2024-04-23T10:24:06.577666+0000","last_unstale":"2024-04-23T10:25:00.670189+0000","last_undegraded":"2024-04-23T10:25:00.670189+0000","last_fullsized":"2024-04-23T10:25:00.670189+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:55:15.053303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.667210+0000","last_change":"2024-04-23T10:24:07.043672+0000","last_active":"2024-04-23T10:25:00.667210+0000","last_peered":"2024-04-23T10:25:00.667210+0000","last_clean":"2024-04-23T10:25:00.667210+0000","last_became_active":"2024-04-23T10:24:07.043346+0000","last_became_peered":"2024-04-23T10:24:07.043346+0000","last_unstale":"2024-04-23T10:25:00.667210+0000","last_undegraded":"2024-04-23T10:25:00.667210+0000","last_fullsized":"2024-04-23T10:25:00.667210+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T13:21:56.751028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.667535+0000","last_change":"2024-04-23T10:24:06.576607+0000","last_active":"2024-04-23T10:25:00.667535+0000","last_peered":"2024-04-23T10:25:00.667535+0000","last_clean":"2024-04-23T10:25:00.667535+0000","last_became_active":"2024-04-23T10:24:06.576213+0000","last_became_peered":"2024-04-23T10:24:06.576213+0000","last_unstale":"2024-04-23T10:25:00.667535+0000","last_undegraded":"2024-04-23T10:25:00.667535+0000","last_fullsized":"2024-04-23T10:25:00.667535+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:09:14.918936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.667860+0000","last_change":"2024-04-23T10:24:07.052641+0000","last_active":"2024-04-23T10:25:00.667860+0000","last_peered":"2024-04-23T10:25:00.667860+0000","last_clean":"2024-04-23T10:25:00.667860+0000","last_became_active":"2024-04-23T10:24:07.052314+0000","last_became_peered":"2024-04-23T10:24:07.052314+0000","last_unstale":"2024-04-23T10:25:00.667860+0000","last_undegraded":"2024-04-23T10:25:00.667860+0000","last_fullsized":"2024-04-23T10:25:00.667860+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T10:29:18.098800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'6","reported_seq":102,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:23.059719+0000","last_change":"2024-04-23T10:24:06.574570+0000","last_active":"2024-04-23T10:25:23.059719+0000","last_peered":"2024-04-23T10:25:23.059719+0000","last_clean":"2024-04-23T10:25:23.059719+0000","last_became_active":"2024-04-23T10:24:06.574166+0000","last_became_peered":"2024-04-23T10:24:06.574166+0000","last_unstale":"2024-04-23T10:25:23.059719+0000","last_undegraded":"2024-04-23T10:25:23.059719+0000","last_fullsized":"2024-04-23T10:25:23.059719+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:28:31.619100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":78,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.667617+0000","last_change":"2024-04-23T10:24:07.052825+0000","last_active":"2024-04-23T10:25:00.667617+0000","last_peered":"2024-04-23T10:25:00.667617+0000","last_clean":"2024-04-23T10:25:00.667617+0000","last_became_active":"2024-04-23T10:24:07.052457+0000","last_became_peered":"2024-04-23T10:24:07.052457+0000","last_unstale":"2024-04-23T10:25:00.667617+0000","last_undegraded":"2024-04-23T10:25:00.667617+0000","last_fullsized":"2024-04-23T10:25:00.667617+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:47:09.545374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.666369+0000","last_change":"2024-04-23T10:24:06.572168+0000","last_active":"2024-04-23T10:25:00.666369+0000","last_peered":"2024-04-23T10:25:00.666369+0000","last_clean":"2024-04-23T10:25:00.666369+0000","last_became_active":"2024-04-23T10:24:06.571805+0000","last_became_peered":"2024-04-23T10:24:06.571805+0000","last_unstale":"2024-04-23T10:25:00.666369+0000","last_undegraded":"2024-04-23T10:25:00.666369+0000","last_fullsized":"2024-04-23T10:25:00.666369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:29:56.817879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.668718+0000","last_change":"2024-04-23T10:24:07.048474+0000","last_active":"2024-04-23T10:25:00.668718+0000","last_peered":"2024-04-23T10:25:00.668718+0000","last_clean":"2024-04-23T10:25:00.668718+0000","last_became_active":"2024-04-23T10:24:07.047893+0000","last_became_peered":"2024-04-23T10:24:07.047893+0000","last_unstale":"2024-04-23T10:25:00.668718+0000","last_undegraded":"2024-04-23T10:25:00.668718+0000","last_fullsized":"2024-04-23T10:25:00.668718+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T12:59:46.545569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.667241+0000","last_change":"2024-04-23T10:24:06.570974+0000","last_active":"2024-04-23T10:25:00.667241+0000","last_peered":"2024-04-23T10:25:00.667241+0000","last_clean":"2024-04-23T10:25:00.667241+0000","last_became_active":"2024-04-23T10:24:06.570603+0000","last_became_peered":"2024-04-23T10:24:06.570603+0000","last_unstale":"2024-04-23T10:25:00.667241+0000","last_undegraded":"2024-04-23T10:25:00.667241+0000","last_fullsized":"2024-04-23T10:25:00.667241+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T12:46:33.518337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.664637+0000","last_change":"2024-04-23T10:24:07.050986+0000","last_active":"2024-04-23T10:25:00.664637+0000","last_peered":"2024-04-23T10:25:00.664637+0000","last_clean":"2024-04-23T10:25:00.664637+0000","last_became_active":"2024-04-23T10:24:07.050617+0000","last_became_peered":"2024-04-23T10:24:07.050617+0000","last_unstale":"2024-04-23T10:25:00.664637+0000","last_undegraded":"2024-04-23T10:25:00.664637+0000","last_fullsized":"2024-04-23T10:25:00.664637+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:13:25.355117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.663786+0000","last_change":"2024-04-23T10:24:06.579138+0000","last_active":"2024-04-23T10:25:00.663786+0000","last_peered":"2024-04-23T10:25:00.663786+0000","last_clean":"2024-04-23T10:25:00.663786+0000","last_became_active":"2024-04-23T10:24:06.578716+0000","last_became_peered":"2024-04-23T10:24:06.578716+0000","last_unstale":"2024-04-23T10:25:00.663786+0000","last_undegraded":"2024-04-23T10:25:00.663786+0000","last_fullsized":"2024-04-23T10:25:00.663786+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:06:59.797802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.665177+0000","last_change":"2024-04-23T10:24:07.039019+0000","last_active":"2024-04-23T10:25:00.665177+0000","last_peered":"2024-04-23T10:25:00.665177+0000","last_clean":"2024-04-23T10:25:00.665177+0000","last_became_active":"2024-04-23T10:24:07.038701+0000","last_became_peered":"2024-04-23T10:24:07.038701+0000","last_unstale":"2024-04-23T10:25:00.665177+0000","last_undegraded":"2024-04-23T10:25:00.665177+0000","last_fullsized":"2024-04-23T10:25:00.665177+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:26:32.716043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":97,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:23.761274+0000","last_change":"2024-04-23T10:24:06.574342+0000","last_active":"2024-04-23T10:25:23.761274+0000","last_peered":"2024-04-23T10:25:23.761274+0000","last_clean":"2024-04-23T10:25:23.761274+0000","last_became_active":"2024-04-23T10:24:06.573981+0000","last_became_peered":"2024-04-23T10:24:06.573981+0000","last_unstale":"2024-04-23T10:25:23.761274+0000","last_undegraded":"2024-04-23T10:25:23.761274+0000","last_fullsized":"2024-04-23T10:25:23.761274+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:48:40.325991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":73,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.665065+0000","last_change":"2024-04-23T10:24:07.044712+0000","last_active":"2024-04-23T10:25:00.665065+0000","last_peered":"2024-04-23T10:25:00.665065+0000","last_clean":"2024-04-23T10:25:00.665065+0000","last_became_active":"2024-04-23T10:24:07.044141+0000","last_became_peered":"2024-04-23T10:24:07.044141+0000","last_unstale":"2024-04-23T10:25:00.665065+0000","last_undegraded":"2024-04-23T10:25:00.665065+0000","last_fullsized":"2024-04-23T10:25:00.665065+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:14:38.818648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.729779+0000","last_change":"2024-04-23T10:25:00.729397+0000","last_active":"2024-04-23T10:25:00.729779+0000","last_peered":"2024-04-23T10:25:00.729779+0000","last_clean":"2024-04-23T10:25:00.729779+0000","last_became_active":"2024-04-23T10:25:00.729046+0000","last_became_peered":"2024-04-23T10:25:00.729046+0000","last_unstale":"2024-04-23T10:25:00.729779+0000","last_undegraded":"2024-04-23T10:25:00.729779+0000","last_fullsized":"2024-04-23T10:25:00.729779+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:25:55.189292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"86'5","reported_seq":99,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:23.058940+0000","last_change":"2024-04-23T10:24:06.578216+0000","last_active":"2024-04-23T10:25:23.058940+0000","last_peered":"2024-04-23T10:25:23.058940+0000","last_clean":"2024-04-23T10:25:23.058940+0000","last_became_active":"2024-04-23T10:24:06.577370+0000","last_became_peered":"2024-04-23T10:24:06.577370+0000","last_unstale":"2024-04-23T10:25:23.058940+0000","last_undegraded":"2024-04-23T10:25:23.058940+0000","last_fullsized":"2024-04-23T10:25:23.058940+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:02:06.554442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":75,"num_read_kb":75,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:01.274458+0000","last_change":"2024-04-23T10:24:07.039627+0000","last_active":"2024-04-23T10:25:01.274458+0000","last_peered":"2024-04-23T10:25:01.274458+0000","last_clean":"2024-04-23T10:25:01.274458+0000","last_became_active":"2024-04-23T10:24:07.039272+0000","last_became_peered":"2024-04-23T10:24:07.039272+0000","last_unstale":"2024-04-23T10:25:01.274458+0000","last_undegraded":"2024-04-23T10:25:01.274458+0000","last_fullsized":"2024-04-23T10:25:01.274458+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T13:45:44.277926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.728792+0000","last_change":"2024-04-23T10:24:06.569086+0000","last_active":"2024-04-23T10:25:00.728792+0000","last_peered":"2024-04-23T10:25:00.728792+0000","last_clean":"2024-04-23T10:25:00.728792+0000","last_became_active":"2024-04-23T10:24:06.568719+0000","last_became_peered":"2024-04-23T10:24:06.568719+0000","last_unstale":"2024-04-23T10:25:00.728792+0000","last_undegraded":"2024-04-23T10:25:00.728792+0000","last_fullsized":"2024-04-23T10:25:00.728792+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:22:10.583782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":354,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.667863+0000","last_change":"2024-04-23T10:22:49.957476+0000","last_active":"2024-04-23T10:25:00.667863+0000","last_peered":"2024-04-23T10:25:00.667863+0000","last_clean":"2024-04-23T10:25:00.667863+0000","last_became_active":"2024-04-23T10:22:49.957033+0000","last_became_peered":"2024-04-23T10:22:49.957033+0000","last_unstale":"2024-04-23T10:25:00.667863+0000","last_undegraded":"2024-04-23T10:25:00.667863+0000","last_fullsized":"2024-04-23T10:25:00.667863+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:19:25.604229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:19:25.604229+0000","last_clean_scrub_stamp":"2024-04-23T10:19:25.604229+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T12:56:17.601141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.671965+0000","last_change":"2024-04-23T10:25:00.671965+0000","last_active":"2024-04-23T10:25:00.671965+0000","last_peered":"2024-04-23T10:25:00.671965+0000","last_clean":"2024-04-23T10:25:00.671965+0000","last_became_active":"2024-04-23T10:25:00.671632+0000","last_became_peered":"2024-04-23T10:25:00.671632+0000","last_unstale":"2024-04-23T10:25:00.671965+0000","last_undegraded":"2024-04-23T10:25:00.671965+0000","last_fullsized":"2024-04-23T10:25:00.671965+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:08:40.262367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.668250+0000","last_change":"2024-04-23T10:24:06.582629+0000","last_active":"2024-04-23T10:25:00.668250+0000","last_peered":"2024-04-23T10:25:00.668250+0000","last_clean":"2024-04-23T10:25:00.668250+0000","last_became_active":"2024-04-23T10:24:06.582325+0000","last_became_peered":"2024-04-23T10:24:06.582325+0000","last_unstale":"2024-04-23T10:25:00.668250+0000","last_undegraded":"2024-04-23T10:25:00.668250+0000","last_fullsized":"2024-04-23T10:25:00.668250+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:57:09.879100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.666667+0000","last_change":"2024-04-23T10:24:06.582482+0000","last_active":"2024-04-23T10:25:00.666667+0000","last_peered":"2024-04-23T10:25:00.666667+0000","last_clean":"2024-04-23T10:25:00.666667+0000","last_became_active":"2024-04-23T10:24:06.582114+0000","last_became_peered":"2024-04-23T10:24:06.582114+0000","last_unstale":"2024-04-23T10:25:00.666667+0000","last_undegraded":"2024-04-23T10:25:00.666667+0000","last_fullsized":"2024-04-23T10:25:00.666667+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:45:42.939840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.667165+0000","last_change":"2024-04-23T10:24:07.031102+0000","last_active":"2024-04-23T10:25:00.667165+0000","last_peered":"2024-04-23T10:25:00.667165+0000","last_clean":"2024-04-23T10:25:00.667165+0000","last_became_active":"2024-04-23T10:24:07.030783+0000","last_became_peered":"2024-04-23T10:24:07.030783+0000","last_unstale":"2024-04-23T10:25:00.667165+0000","last_undegraded":"2024-04-23T10:25:00.667165+0000","last_fullsized":"2024-04-23T10:25:00.667165+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:49:46.736046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'12","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.664017+0000","last_change":"2024-04-23T10:24:06.577096+0000","last_active":"2024-04-23T10:25:00.664017+0000","last_peered":"2024-04-23T10:25:00.664017+0000","last_clean":"2024-04-23T10:25:00.664017+0000","last_became_active":"2024-04-23T10:24:06.576740+0000","last_became_peered":"2024-04-23T10:24:06.576740+0000","last_unstale":"2024-04-23T10:25:00.664017+0000","last_undegraded":"2024-04-23T10:25:00.664017+0000","last_fullsized":"2024-04-23T10:25:00.664017+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:44:06.532366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:01.274734+0000","last_change":"2024-04-23T10:24:07.043599+0000","last_active":"2024-04-23T10:25:01.274734+0000","last_peered":"2024-04-23T10:25:01.274734+0000","last_clean":"2024-04-23T10:25:01.274734+0000","last_became_active":"2024-04-23T10:24:07.042953+0000","last_became_peered":"2024-04-23T10:24:07.042953+0000","last_unstale":"2024-04-23T10:25:01.274734+0000","last_undegraded":"2024-04-23T10:25:01.274734+0000","last_fullsized":"2024-04-23T10:25:01.274734+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:07:50.487972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.728986+0000","last_change":"2024-04-23T10:24:06.570681+0000","last_active":"2024-04-23T10:25:00.728986+0000","last_peered":"2024-04-23T10:25:00.728986+0000","last_clean":"2024-04-23T10:25:00.728986+0000","last_became_active":"2024-04-23T10:24:06.570178+0000","last_became_peered":"2024-04-23T10:24:06.570178+0000","last_unstale":"2024-04-23T10:25:00.728986+0000","last_undegraded":"2024-04-23T10:25:00.728986+0000","last_fullsized":"2024-04-23T10:25:00.728986+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T12:16:56.629443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.669433+0000","last_change":"2024-04-23T10:24:07.040498+0000","last_active":"2024-04-23T10:25:00.669433+0000","last_peered":"2024-04-23T10:25:00.669433+0000","last_clean":"2024-04-23T10:25:00.669433+0000","last_became_active":"2024-04-23T10:24:07.040009+0000","last_became_peered":"2024-04-23T10:24:07.040009+0000","last_unstale":"2024-04-23T10:25:00.669433+0000","last_undegraded":"2024-04-23T10:25:00.669433+0000","last_fullsized":"2024-04-23T10:25:00.669433+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:43:17.319305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.673495+0000","last_change":"2024-04-23T10:25:00.673495+0000","last_active":"2024-04-23T10:25:00.673495+0000","last_peered":"2024-04-23T10:25:00.673495+0000","last_clean":"2024-04-23T10:25:00.673495+0000","last_became_active":"2024-04-23T10:25:00.670572+0000","last_became_peered":"2024-04-23T10:25:00.670572+0000","last_unstale":"2024-04-23T10:25:00.673495+0000","last_undegraded":"2024-04-23T10:25:00.673495+0000","last_fullsized":"2024-04-23T10:25:00.673495+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:08:04.279056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.666362+0000","last_change":"2024-04-23T10:24:07.043710+0000","last_active":"2024-04-23T10:25:00.666362+0000","last_peered":"2024-04-23T10:25:00.666362+0000","last_clean":"2024-04-23T10:25:00.666362+0000","last_became_active":"2024-04-23T10:24:07.043406+0000","last_became_peered":"2024-04-23T10:24:07.043406+0000","last_unstale":"2024-04-23T10:25:00.666362+0000","last_undegraded":"2024-04-23T10:25:00.666362+0000","last_fullsized":"2024-04-23T10:25:00.666362+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:09:14.471938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:01.274384+0000","last_change":"2024-04-23T10:24:06.577409+0000","last_active":"2024-04-23T10:25:01.274384+0000","last_peered":"2024-04-23T10:25:01.274384+0000","last_clean":"2024-04-23T10:25:01.274384+0000","last_became_active":"2024-04-23T10:24:06.576787+0000","last_became_peered":"2024-04-23T10:24:06.576787+0000","last_unstale":"2024-04-23T10:25:01.274384+0000","last_undegraded":"2024-04-23T10:25:01.274384+0000","last_fullsized":"2024-04-23T10:25:01.274384+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:05:28.423806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.664399+0000","last_change":"2024-04-23T10:24:07.045180+0000","last_active":"2024-04-23T10:25:00.664399+0000","last_peered":"2024-04-23T10:25:00.664399+0000","last_clean":"2024-04-23T10:25:00.664399+0000","last_became_active":"2024-04-23T10:24:07.044731+0000","last_became_peered":"2024-04-23T10:24:07.044731+0000","last_unstale":"2024-04-23T10:25:00.664399+0000","last_undegraded":"2024-04-23T10:25:00.664399+0000","last_fullsized":"2024-04-23T10:25:00.664399+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:42:34.474391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.669897+0000","last_change":"2024-04-23T10:24:06.578039+0000","last_active":"2024-04-23T10:25:00.669897+0000","last_peered":"2024-04-23T10:25:00.669897+0000","last_clean":"2024-04-23T10:25:00.669897+0000","last_became_active":"2024-04-23T10:24:06.577655+0000","last_became_peered":"2024-04-23T10:24:06.577655+0000","last_unstale":"2024-04-23T10:25:00.669897+0000","last_undegraded":"2024-04-23T10:25:00.669897+0000","last_fullsized":"2024-04-23T10:25:00.669897+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:57:35.172318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:01.274530+0000","last_change":"2024-04-23T10:24:07.037311+0000","last_active":"2024-04-23T10:25:01.274530+0000","last_peered":"2024-04-23T10:25:01.274530+0000","last_clean":"2024-04-23T10:25:01.274530+0000","last_became_active":"2024-04-23T10:24:07.036969+0000","last_became_peered":"2024-04-23T10:24:07.036969+0000","last_unstale":"2024-04-23T10:25:01.274530+0000","last_undegraded":"2024-04-23T10:25:01.274530+0000","last_fullsized":"2024-04-23T10:25:01.274530+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:13:18.998914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.666877+0000","last_change":"2024-04-23T10:24:06.575183+0000","last_active":"2024-04-23T10:25:00.666877+0000","last_peered":"2024-04-23T10:25:00.666877+0000","last_clean":"2024-04-23T10:25:00.666877+0000","last_became_active":"2024-04-23T10:24:06.574630+0000","last_became_peered":"2024-04-23T10:24:06.574630+0000","last_unstale":"2024-04-23T10:25:00.666877+0000","last_undegraded":"2024-04-23T10:25:00.666877+0000","last_fullsized":"2024-04-23T10:25:00.666877+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T11:18:18.189925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.667979+0000","last_change":"2024-04-23T10:24:07.048832+0000","last_active":"2024-04-23T10:25:00.667979+0000","last_peered":"2024-04-23T10:25:00.667979+0000","last_clean":"2024-04-23T10:25:00.667979+0000","last_became_active":"2024-04-23T10:24:07.048509+0000","last_became_peered":"2024-04-23T10:24:07.048509+0000","last_unstale":"2024-04-23T10:25:00.667979+0000","last_undegraded":"2024-04-23T10:25:00.667979+0000","last_fullsized":"2024-04-23T10:25:00.667979+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T13:45:26.793330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'6","reported_seq":99,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:23.081908+0000","last_change":"2024-04-23T10:24:06.575408+0000","last_active":"2024-04-23T10:25:23.081908+0000","last_peered":"2024-04-23T10:25:23.081908+0000","last_clean":"2024-04-23T10:25:23.081908+0000","last_became_active":"2024-04-23T10:24:06.575081+0000","last_became_peered":"2024-04-23T10:24:06.575081+0000","last_unstale":"2024-04-23T10:25:23.081908+0000","last_undegraded":"2024-04-23T10:25:23.081908+0000","last_fullsized":"2024-04-23T10:25:23.081908+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T11:28:12.765196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":74,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.668181+0000","last_change":"2024-04-23T10:24:07.043869+0000","last_active":"2024-04-23T10:25:00.668181+0000","last_peered":"2024-04-23T10:25:00.668181+0000","last_clean":"2024-04-23T10:25:00.668181+0000","last_became_active":"2024-04-23T10:24:07.043448+0000","last_became_peered":"2024-04-23T10:24:07.043448+0000","last_unstale":"2024-04-23T10:25:00.668181+0000","last_undegraded":"2024-04-23T10:25:00.668181+0000","last_fullsized":"2024-04-23T10:25:00.668181+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:00:35.821470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.667896+0000","last_change":"2024-04-23T10:24:06.575259+0000","last_active":"2024-04-23T10:25:00.667896+0000","last_peered":"2024-04-23T10:25:00.667896+0000","last_clean":"2024-04-23T10:25:00.667896+0000","last_became_active":"2024-04-23T10:24:06.574920+0000","last_became_peered":"2024-04-23T10:24:06.574920+0000","last_unstale":"2024-04-23T10:25:00.667896+0000","last_undegraded":"2024-04-23T10:25:00.667896+0000","last_fullsized":"2024-04-23T10:25:00.667896+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:52:14.558458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.667808+0000","last_change":"2024-04-23T10:24:07.043238+0000","last_active":"2024-04-23T10:25:00.667808+0000","last_peered":"2024-04-23T10:25:00.667808+0000","last_clean":"2024-04-23T10:25:00.667808+0000","last_became_active":"2024-04-23T10:24:07.042110+0000","last_became_peered":"2024-04-23T10:24:07.042110+0000","last_unstale":"2024-04-23T10:25:00.667808+0000","last_undegraded":"2024-04-23T10:25:00.667808+0000","last_fullsized":"2024-04-23T10:25:00.667808+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T12:36:13.474456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.667774+0000","last_change":"2024-04-23T10:24:06.579233+0000","last_active":"2024-04-23T10:25:00.667774+0000","last_peered":"2024-04-23T10:25:00.667774+0000","last_clean":"2024-04-23T10:25:00.667774+0000","last_became_active":"2024-04-23T10:24:06.578849+0000","last_became_peered":"2024-04-23T10:24:06.578849+0000","last_unstale":"2024-04-23T10:25:00.667774+0000","last_undegraded":"2024-04-23T10:25:00.667774+0000","last_fullsized":"2024-04-23T10:25:00.667774+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:34:23.193707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.668344+0000","last_change":"2024-04-23T10:24:07.049840+0000","last_active":"2024-04-23T10:25:00.668344+0000","last_peered":"2024-04-23T10:25:00.668344+0000","last_clean":"2024-04-23T10:25:00.668344+0000","last_became_active":"2024-04-23T10:24:07.049493+0000","last_became_peered":"2024-04-23T10:24:07.049493+0000","last_unstale":"2024-04-23T10:25:00.668344+0000","last_undegraded":"2024-04-23T10:25:00.668344+0000","last_fullsized":"2024-04-23T10:25:00.668344+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:03:22.606800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.664508+0000","last_change":"2024-04-23T10:24:07.052332+0000","last_active":"2024-04-23T10:25:00.664508+0000","last_peered":"2024-04-23T10:25:00.664508+0000","last_clean":"2024-04-23T10:25:00.664508+0000","last_became_active":"2024-04-23T10:24:07.051956+0000","last_became_peered":"2024-04-23T10:24:07.051956+0000","last_unstale":"2024-04-23T10:25:00.664508+0000","last_undegraded":"2024-04-23T10:25:00.664508+0000","last_fullsized":"2024-04-23T10:25:00.664508+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:56:43.606916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.669912+0000","last_change":"2024-04-23T10:24:07.041897+0000","last_active":"2024-04-23T10:25:00.669912+0000","last_peered":"2024-04-23T10:25:00.669912+0000","last_clean":"2024-04-23T10:25:00.669912+0000","last_became_active":"2024-04-23T10:24:07.041546+0000","last_became_peered":"2024-04-23T10:24:07.041546+0000","last_unstale":"2024-04-23T10:25:00.669912+0000","last_undegraded":"2024-04-23T10:25:00.669912+0000","last_fullsized":"2024-04-23T10:25:00.669912+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T11:31:23.211045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.669833+0000","last_change":"2024-04-23T10:24:07.040764+0000","last_active":"2024-04-23T10:25:00.669833+0000","last_peered":"2024-04-23T10:25:00.669833+0000","last_clean":"2024-04-23T10:25:00.669833+0000","last_became_active":"2024-04-23T10:24:07.039637+0000","last_became_peered":"2024-04-23T10:24:07.039637+0000","last_unstale":"2024-04-23T10:25:00.669833+0000","last_undegraded":"2024-04-23T10:25:00.669833+0000","last_fullsized":"2024-04-23T10:25:00.669833+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:50:33.298770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.669097+0000","last_change":"2024-04-23T10:24:07.039491+0000","last_active":"2024-04-23T10:25:00.669097+0000","last_peered":"2024-04-23T10:25:00.669097+0000","last_clean":"2024-04-23T10:25:00.669097+0000","last_became_active":"2024-04-23T10:24:07.039128+0000","last_became_peered":"2024-04-23T10:24:07.039128+0000","last_unstale":"2024-04-23T10:25:00.669097+0000","last_undegraded":"2024-04-23T10:25:00.669097+0000","last_fullsized":"2024-04-23T10:25:00.669097+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T12:30:40.326031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.671059+0000","last_change":"2024-04-23T10:25:00.671059+0000","last_active":"2024-04-23T10:25:00.671059+0000","last_peered":"2024-04-23T10:25:00.671059+0000","last_clean":"2024-04-23T10:25:00.671059+0000","last_became_active":"2024-04-23T10:25:00.670686+0000","last_became_peered":"2024-04-23T10:25:00.670686+0000","last_unstale":"2024-04-23T10:25:00.671059+0000","last_undegraded":"2024-04-23T10:25:00.671059+0000","last_fullsized":"2024-04-23T10:25:00.671059+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:04:39.991647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.664595+0000","last_change":"2024-04-23T10:24:07.052395+0000","last_active":"2024-04-23T10:25:00.664595+0000","last_peered":"2024-04-23T10:25:00.664595+0000","last_clean":"2024-04-23T10:25:00.664595+0000","last_became_active":"2024-04-23T10:24:07.052023+0000","last_became_peered":"2024-04-23T10:24:07.052023+0000","last_unstale":"2024-04-23T10:25:00.664595+0000","last_undegraded":"2024-04-23T10:25:00.664595+0000","last_fullsized":"2024-04-23T10:25:00.664595+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:25:52.517267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.671271+0000","last_change":"2024-04-23T10:24:07.042739+0000","last_active":"2024-04-23T10:25:00.671271+0000","last_peered":"2024-04-23T10:25:00.671271+0000","last_clean":"2024-04-23T10:25:00.671271+0000","last_became_active":"2024-04-23T10:24:07.042292+0000","last_became_peered":"2024-04-23T10:24:07.042292+0000","last_unstale":"2024-04-23T10:25:00.671271+0000","last_undegraded":"2024-04-23T10:25:00.671271+0000","last_fullsized":"2024-04-23T10:25:00.671271+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:14:23.229032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:01.274715+0000","last_change":"2024-04-23T10:24:07.042792+0000","last_active":"2024-04-23T10:25:01.274715+0000","last_peered":"2024-04-23T10:25:01.274715+0000","last_clean":"2024-04-23T10:25:01.274715+0000","last_became_active":"2024-04-23T10:24:07.040846+0000","last_became_peered":"2024-04-23T10:24:07.040846+0000","last_unstale":"2024-04-23T10:25:01.274715+0000","last_undegraded":"2024-04-23T10:25:01.274715+0000","last_fullsized":"2024-04-23T10:25:01.274715+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T13:50:24.973286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.668190+0000","last_change":"2024-04-23T10:24:07.040391+0000","last_active":"2024-04-23T10:25:00.668190+0000","last_peered":"2024-04-23T10:25:00.668190+0000","last_clean":"2024-04-23T10:25:00.668190+0000","last_became_active":"2024-04-23T10:24:07.040002+0000","last_became_peered":"2024-04-23T10:24:07.040002+0000","last_unstale":"2024-04-23T10:25:00.668190+0000","last_undegraded":"2024-04-23T10:25:00.668190+0000","last_fullsized":"2024-04-23T10:25:00.668190+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T10:30:13.922315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:01.274323+0000","last_change":"2024-04-23T10:24:07.040678+0000","last_active":"2024-04-23T10:25:01.274323+0000","last_peered":"2024-04-23T10:25:01.274323+0000","last_clean":"2024-04-23T10:25:01.274323+0000","last_became_active":"2024-04-23T10:24:07.039551+0000","last_became_peered":"2024-04-23T10:24:07.039551+0000","last_unstale":"2024-04-23T10:25:01.274323+0000","last_undegraded":"2024-04-23T10:25:01.274323+0000","last_fullsized":"2024-04-23T10:25:01.274323+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:03:50.467083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.666980+0000","last_change":"2024-04-23T10:24:07.044618+0000","last_active":"2024-04-23T10:25:00.666980+0000","last_peered":"2024-04-23T10:25:00.666980+0000","last_clean":"2024-04-23T10:25:00.666980+0000","last_became_active":"2024-04-23T10:24:07.042803+0000","last_became_peered":"2024-04-23T10:24:07.042803+0000","last_unstale":"2024-04-23T10:25:00.666980+0000","last_undegraded":"2024-04-23T10:25:00.666980+0000","last_fullsized":"2024-04-23T10:25:00.666980+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:37:46.872177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.667432+0000","last_change":"2024-04-23T10:24:07.043724+0000","last_active":"2024-04-23T10:25:00.667432+0000","last_peered":"2024-04-23T10:25:00.667432+0000","last_clean":"2024-04-23T10:25:00.667432+0000","last_became_active":"2024-04-23T10:24:07.043408+0000","last_became_peered":"2024-04-23T10:24:07.043408+0000","last_unstale":"2024-04-23T10:25:00.667432+0000","last_undegraded":"2024-04-23T10:25:00.667432+0000","last_fullsized":"2024-04-23T10:25:00.667432+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:18:46.615093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T10:25:00.664815+0000","last_change":"2024-04-23T10:24:07.043794+0000","last_active":"2024-04-23T10:25:00.664815+0000","last_peered":"2024-04-23T10:25:00.664815+0000","last_clean":"2024-04-23T10:25:00.664815+0000","last_became_active":"2024-04-23T10:24:07.043391+0000","last_became_peered":"2024-04-23T10:24:07.043391+0000","last_unstale":"2024-04-23T10:25:00.664815+0000","last_undegraded":"2024-04-23T10:25:00.664815+0000","last_fullsized":"2024-04-23T10:25:00.664815+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:06:04.584000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280003,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68641,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":771,"num_read_kb":458,"num_write":238,"num_write_kb":134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1254,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":205923,"data_compressed":5568,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":207,"ondisk_log_size":207,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351006,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":460744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612643,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1110016,"data_stored":1112237,"data_compressed":37471,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776167,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":459701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037806,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":458825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299443,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1032192,"data_stored":1052988,"data_compressed":35615,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462968,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":966656,"data_stored":459418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":521377,"data_compressed":1856,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738454,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":987136,"data_stored":461148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411402,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":522379,"data_compressed":1856,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149767,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":983040,"data_stored":459243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888130,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":974848,"data_stored":460342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724604,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1044480,"data_stored":1052275,"data_compressed":35615,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33976,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33976,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33976,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35615,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35615,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35615,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35615,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35615,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":62798,"data_compressed":1856,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":63800,"data_compressed":1856,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":63173,"data_compressed":1856,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T10:25:30.005 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-23T10:25:30.005 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-23T10:25:30.005 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-23T10:25:30.005 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-23T10:25:30.256 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T10:25:31.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:30 smithi196 ceph-mon[32919]: from='client.14988 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T10:25:31.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:30 smithi196 ceph-mon[32919]: pgmap v333: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-23T10:25:31.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:30 smithi196 ceph-mon[32919]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-23T10:25:31.098 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:30 smithi097 ceph-mon[29197]: from='client.14988 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T10:25:31.098 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:30 smithi097 ceph-mon[29197]: pgmap v333: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-23T10:25:31.098 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:30 smithi097 ceph-mon[29197]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-23T10:25:31.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:30 smithi183 ceph-mon[32317]: from='client.14988 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T10:25:31.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:30 smithi183 ceph-mon[32317]: pgmap v333: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-23T10:25:31.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:30 smithi183 ceph-mon[32317]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-23T10:25:32.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:32.088+0000 7f03e384b700 1 -- 172.21.15.97:0/4166086555 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03dc108a50 msgr2=0x7f03dc108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:32.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:32.088+0000 7f03e384b700 1 --2- 172.21.15.97:0/4166086555 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03dc108a50 0x7f03dc108e30 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f03d8009990 tx=0x7f03d8009ca0 comp rx=0 tx=0).stop 2024-04-23T10:25:32.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:32.088+0000 7f03e384b700 1 -- 172.21.15.97:0/4166086555 shutdown_connections 2024-04-23T10:25:32.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:32.088+0000 7f03e384b700 1 --2- 172.21.15.97:0/4166086555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03dc103450 0x7f03dc10f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:32.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:32.088+0000 7f03e384b700 1 --2- 172.21.15.97:0/4166086555 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03dc108a50 0x7f03dc108e30 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:32.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:32.088+0000 7f03e384b700 1 --2- 172.21.15.97:0/4166086555 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03dc102a90 0x7f03dc102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:32.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:32.088+0000 7f03e384b700 1 -- 172.21.15.97:0/4166086555 >> 172.21.15.97:0/4166086555 conn(0x7f03dc079ba0 msgr2=0x7f03dc079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:32.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:32.088+0000 7f03e384b700 1 -- 172.21.15.97:0/4166086555 shutdown_connections 2024-04-23T10:25:32.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:32.088+0000 7f03e384b700 1 -- 172.21.15.97:0/4166086555 wait complete. 2024-04-23T10:25:32.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:32.089+0000 7f03e384b700 1 Processor -- start 2024-04-23T10:25:32.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:32.089+0000 7f03e384b700 1 -- start start 2024-04-23T10:25:32.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:32.089+0000 7f03e384b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03dc102a90 0x7f03dc1991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:32.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:32.089+0000 7f03e384b700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03dc103450 0x7f03dc1996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:32.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:32.089+0000 7f03e384b700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03dc108a50 0x7f03dc19da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:32.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:32.089+0000 7f03e384b700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f03dc112be0 con 0x7f03dc102a90 2024-04-23T10:25:32.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:32.089+0000 7f03e384b700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f03dc112a60 con 0x7f03dc108a50 2024-04-23T10:25:32.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:32.089+0000 7f03e384b700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f03dc112d60 con 0x7f03dc103450 2024-04-23T10:25:32.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:32.089+0000 7f03e15e7700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03dc102a90 0x7f03dc1991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:32.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:32.089+0000 7f03e15e7700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03dc102a90 0x7f03dc1991b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33854/0 (socket says 172.21.15.97:33854) 2024-04-23T10:25:32.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:32.089+0000 7f03e15e7700 1 -- 172.21.15.97:0/3518283536 learned_addr learned my addr 172.21.15.97:0/3518283536 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:25:32.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:32.089+0000 7f03e1de8700 1 --2- 172.21.15.97:0/3518283536 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03dc108a50 0x7f03dc19da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:32.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:32.089+0000 7f03e0de6700 1 --2- 172.21.15.97:0/3518283536 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03dc103450 0x7f03dc1996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:32.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:32.090+0000 7f03e15e7700 1 -- 172.21.15.97:0/3518283536 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03dc103450 msgr2=0x7f03dc1996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:32.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:32.090+0000 7f03e15e7700 1 --2- 172.21.15.97:0/3518283536 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03dc103450 0x7f03dc1996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:32.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:32.090+0000 7f03e15e7700 1 -- 172.21.15.97:0/3518283536 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03dc108a50 msgr2=0x7f03dc19da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:32.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:32.090+0000 7f03e15e7700 1 --2- 172.21.15.97:0/3518283536 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03dc108a50 0x7f03dc19da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:32.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:32.090+0000 7f03e15e7700 1 -- 172.21.15.97:0/3518283536 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03dc19e1c0 con 0x7f03dc102a90 2024-04-23T10:25:32.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:32.090+0000 7f03e15e7700 1 --2- 172.21.15.97:0/3518283536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03dc102a90 0x7f03dc1991b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f03d80057c0 tx=0x7f03d80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:32.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:32.090+0000 7f03d27fc700 1 -- 172.21.15.97:0/3518283536 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d8027070 con 0x7f03dc102a90 2024-04-23T10:25:32.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:32.090+0000 7f03d27fc700 1 -- 172.21.15.97:0/3518283536 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03d8003f20 con 0x7f03dc102a90 2024-04-23T10:25:32.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:32.090+0000 7f03e384b700 1 -- 172.21.15.97:0/3518283536 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03dc19e450 con 0x7f03dc102a90 2024-04-23T10:25:32.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:32.090+0000 7f03d27fc700 1 -- 172.21.15.97:0/3518283536 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d8022b80 con 0x7f03dc102a90 2024-04-23T10:25:32.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:32.090+0000 7f03e384b700 1 -- 172.21.15.97:0/3518283536 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03dc1a5d30 con 0x7f03dc102a90 2024-04-23T10:25:32.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:32.091+0000 7f03e384b700 1 -- 172.21.15.97:0/3518283536 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03dc069730 con 0x7f03dc102a90 2024-04-23T10:25:32.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:32.092+0000 7f03d27fc700 1 -- 172.21.15.97:0/3518283536 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f03d802a070 con 0x7f03dc102a90 2024-04-23T10:25:32.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:32.093+0000 7f03d27fc700 1 --2- 172.21.15.97:0/3518283536 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f03c80a3c40 0x7f03c80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:32.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:32.093+0000 7f03e0de6700 1 --2- 172.21.15.97:0/3518283536 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f03c80a3c40 0x7f03c80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:32.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:32.093+0000 7f03d27fc700 1 -- 172.21.15.97:0/3518283536 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f03d80d1c40 con 0x7f03dc102a90 2024-04-23T10:25:32.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:32.093+0000 7f03e0de6700 1 --2- 172.21.15.97:0/3518283536 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f03c80a3c40 0x7f03c80a6100 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f03dc07a2d0 tx=0x7f03cc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:32.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:32.096+0000 7f03d27fc700 1 -- 172.21.15.97:0/3518283536 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f03d8013070 con 0x7f03dc102a90 2024-04-23T10:25:32.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:32.340+0000 7f03e384b700 1 -- 172.21.15.97:0/3518283536 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f03dc02df80 con 0x7f03dc102a90 2024-04-23T10:25:32.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:32.341+0000 7f03d27fc700 1 -- 172.21.15.97:0/3518283536 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f03d809e6e0 con 0x7f03dc102a90 2024-04-23T10:25:32.340 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:25:32.340 INFO:teuthology.orchestra.run.smithi097.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-23T10:25:32.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:32.343+0000 7f03e384b700 1 -- 172.21.15.97:0/3518283536 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f03c80a3c40 msgr2=0x7f03c80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:32.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:32.343+0000 7f03e384b700 1 --2- 172.21.15.97:0/3518283536 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f03c80a3c40 0x7f03c80a6100 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f03dc07a2d0 tx=0x7f03cc00b410 comp rx=0 tx=0).stop 2024-04-23T10:25:32.342 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:32.343+0000 7f03e384b700 1 -- 172.21.15.97:0/3518283536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03dc102a90 msgr2=0x7f03dc1991b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:32.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:32.343+0000 7f03e384b700 1 --2- 172.21.15.97:0/3518283536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03dc102a90 0x7f03dc1991b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f03d80057c0 tx=0x7f03d80057f0 comp rx=0 tx=0).stop 2024-04-23T10:25:32.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:32.343+0000 7f03e384b700 1 -- 172.21.15.97:0/3518283536 shutdown_connections 2024-04-23T10:25:32.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:32.343+0000 7f03e384b700 1 --2- 172.21.15.97:0/3518283536 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f03c80a3c40 0x7f03c80a6100 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:32.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:32.343+0000 7f03e384b700 1 --2- 172.21.15.97:0/3518283536 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03dc108a50 0x7f03dc19da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:32.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:32.343+0000 7f03e384b700 1 --2- 172.21.15.97:0/3518283536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03dc102a90 0x7f03dc1991b0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:32.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:32.343+0000 7f03e384b700 1 --2- 172.21.15.97:0/3518283536 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03dc103450 0x7f03dc1996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:32.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:32.343+0000 7f03e384b700 1 -- 172.21.15.97:0/3518283536 >> 172.21.15.97:0/3518283536 conn(0x7f03dc079ba0 msgr2=0x7f03dc101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:32.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:32.343+0000 7f03e384b700 1 -- 172.21.15.97:0/3518283536 shutdown_connections 2024-04-23T10:25:32.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:32.343+0000 7f03e384b700 1 -- 172.21.15.97:0/3518283536 wait complete. 2024-04-23T10:25:33.042 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:32 smithi097 ceph-mon[29197]: pgmap v334: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-23T10:25:33.042 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:32 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/3518283536' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-23T10:25:33.074 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-23T10:25:33.075 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T10:25:33.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:32 smithi196 ceph-mon[32919]: pgmap v334: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-23T10:25:33.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:32 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/3518283536' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-23T10:25:33.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:32 smithi183 ceph-mon[32317]: pgmap v334: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-23T10:25:33.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:32 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/3518283536' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-23T10:25:33.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.183+0000 7fae37ad0640 1 -- 172.21.15.97:0/1586955637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae30100e90 msgr2=0x7fae301012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:33.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.183+0000 7fae37ad0640 1 --2- 172.21.15.97:0/1586955637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae30100e90 0x7fae301012b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fae2c009ca0 tx=0x7fae2c02a8c0 comp rx=0 tx=0).stop 2024-04-23T10:25:33.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.183+0000 7fae37ad0640 1 -- 172.21.15.97:0/1586955637 shutdown_connections 2024-04-23T10:25:33.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.183+0000 7fae37ad0640 1 --2- 172.21.15.97:0/1586955637 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae301029a0 0x7fae301092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:33.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.183+0000 7fae37ad0640 1 --2- 172.21.15.97:0/1586955637 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae30101fe0 0x7fae30102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:33.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.183+0000 7fae37ad0640 1 --2- 172.21.15.97:0/1586955637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae30100e90 0x7fae301012b0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:33.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.183+0000 7fae37ad0640 1 -- 172.21.15.97:0/1586955637 >> 172.21.15.97:0/1586955637 conn(0x7fae300fc400 msgr2=0x7fae300fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:33.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.183+0000 7fae37ad0640 1 -- 172.21.15.97:0/1586955637 shutdown_connections 2024-04-23T10:25:33.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.183+0000 7fae37ad0640 1 -- 172.21.15.97:0/1586955637 wait complete. 2024-04-23T10:25:33.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.183+0000 7fae37ad0640 1 Processor -- start 2024-04-23T10:25:33.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.183+0000 7fae37ad0640 1 -- start start 2024-04-23T10:25:33.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.183+0000 7fae37ad0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae30100e90 0x7fae301981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:33.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.183+0000 7fae37ad0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae30101fe0 0x7fae30198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:33.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.183+0000 7fae37ad0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae301029a0 0x7fae3019f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:33.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.183+0000 7fae37ad0640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fae3010daf0 con 0x7fae30101fe0 2024-04-23T10:25:33.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.183+0000 7fae37ad0640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fae3010d970 con 0x7fae301029a0 2024-04-23T10:25:33.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.183+0000 7fae37ad0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fae3010dc70 con 0x7fae30100e90 2024-04-23T10:25:33.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.184+0000 7fae3506b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae30101fe0 0x7fae30198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:33.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.184+0000 7fae3506b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae30101fe0 0x7fae30198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33874/0 (socket says 172.21.15.97:33874) 2024-04-23T10:25:33.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.184+0000 7fae3506b640 1 -- 172.21.15.97:0/3969743731 learned_addr learned my addr 172.21.15.97:0/3969743731 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:25:33.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.184+0000 7fae3606d640 1 --2- 172.21.15.97:0/3969743731 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae301029a0 0x7fae3019f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:33.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.184+0000 7fae3586c640 1 --2- 172.21.15.97:0/3969743731 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae30100e90 0x7fae301981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:33.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.184+0000 7fae3506b640 1 -- 172.21.15.97:0/3969743731 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae30100e90 msgr2=0x7fae301981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:33.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.184+0000 7fae3506b640 1 --2- 172.21.15.97:0/3969743731 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae30100e90 0x7fae301981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:33.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.184+0000 7fae3506b640 1 -- 172.21.15.97:0/3969743731 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae301029a0 msgr2=0x7fae3019f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:33.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.184+0000 7fae3506b640 1 --2- 172.21.15.97:0/3969743731 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae301029a0 0x7fae3019f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:33.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.184+0000 7fae3506b640 1 -- 172.21.15.97:0/3969743731 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae3019fe00 con 0x7fae30101fe0 2024-04-23T10:25:33.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.184+0000 7fae3586c640 1 --2- 172.21.15.97:0/3969743731 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae30100e90 0x7fae301981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:25:33.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.184+0000 7fae3506b640 1 --2- 172.21.15.97:0/3969743731 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae30101fe0 0x7fae30198720 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fae2000c970 tx=0x7fae2000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:33.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.184+0000 7fae26ffd640 1 -- 172.21.15.97:0/3969743731 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae20004220 con 0x7fae30101fe0 2024-04-23T10:25:33.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.184+0000 7fae26ffd640 1 -- 172.21.15.97:0/3969743731 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae200043c0 con 0x7fae30101fe0 2024-04-23T10:25:33.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.184+0000 7fae37ad0640 1 -- 172.21.15.97:0/3969743731 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae301a00f0 con 0x7fae30101fe0 2024-04-23T10:25:33.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.184+0000 7fae26ffd640 1 -- 172.21.15.97:0/3969743731 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2000f450 con 0x7fae30101fe0 2024-04-23T10:25:33.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.184+0000 7fae37ad0640 1 -- 172.21.15.97:0/3969743731 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae301a0640 con 0x7fae30101fe0 2024-04-23T10:25:33.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.185+0000 7fae37ad0640 1 -- 172.21.15.97:0/3969743731 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae30108290 con 0x7fae30101fe0 2024-04-23T10:25:33.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.186+0000 7fae26ffd640 1 -- 172.21.15.97:0/3969743731 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fae20003610 con 0x7fae30101fe0 2024-04-23T10:25:33.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.187+0000 7fae26ffd640 1 --2- 172.21.15.97:0/3969743731 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fae1c0a3bb0 0x7fae1c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:33.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.187+0000 7fae3586c640 1 --2- 172.21.15.97:0/3969743731 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fae1c0a3bb0 0x7fae1c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:33.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.187+0000 7fae26ffd640 1 -- 172.21.15.97:0/3969743731 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fae200d0300 con 0x7fae30101fe0 2024-04-23T10:25:33.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.187+0000 7fae3586c640 1 --2- 172.21.15.97:0/3969743731 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fae1c0a3bb0 0x7fae1c0a6070 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fae2c006010 tx=0x7fae2c0093f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:33.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.191+0000 7fae26ffd640 1 -- 172.21.15.97:0/3969743731 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fae20014030 con 0x7fae30101fe0 2024-04-23T10:25:33.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.371+0000 7fae37ad0640 1 -- 172.21.15.97:0/3969743731 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fae30062b50 con 0x7fae30101fe0 2024-04-23T10:25:33.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.372+0000 7fae26ffd640 1 -- 172.21.15.97:0/3969743731 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22606 (secure 0 0 0) 0x7fae2009cd10 con 0x7fae30101fe0 2024-04-23T10:25:33.371 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:25:33.372 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":88,"fsid":"9b22af88-0159-11ef-bc93-c7b262605968","created":"2024-04-23T10:16:05.983305+0000","modified":"2024-04-23T10:25:00.660133+0000","last_up_change":"2024-04-23T10:23:13.114761+0000","last_in_change":"2024-04-23T10:22:52.804880+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T10:19:25.559138+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T10:24:04.570049+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T10:24:05.861631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"58cb995d-de35-4522-ac81-4982401b260d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6803","nonce":3031221802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6805","nonce":3031221802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6809","nonce":3031221802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6807","nonce":3031221802}]},"public_addr":"172.21.15.97:6803/3031221802","cluster_addr":"172.21.15.97:6805/3031221802","heartbeat_back_addr":"172.21.15.97:6809/3031221802","heartbeat_front_addr":"172.21.15.97:6807/3031221802","state":["exists","up"]},{"osd":1,"uuid":"b47bd8ee-1f21-4834-acf5-bb40f97ab3ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6801","nonce":1108870440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6803","nonce":1108870440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6807","nonce":1108870440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6805","nonce":1108870440}]},"public_addr":"172.21.15.183:6801/1108870440","cluster_addr":"172.21.15.183:6803/1108870440","heartbeat_back_addr":"172.21.15.183:6807/1108870440","heartbeat_front_addr":"172.21.15.183:6805/1108870440","state":["exists","up"]},{"osd":2,"uuid":"17c4eec3-6c47-4334-9b63-1d5ad9ec5988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6801","nonce":2877039731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6803","nonce":2877039731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6807","nonce":2877039731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6805","nonce":2877039731}]},"public_addr":"172.21.15.196:6801/2877039731","cluster_addr":"172.21.15.196:6803/2877039731","heartbeat_back_addr":"172.21.15.196:6807/2877039731","heartbeat_front_addr":"172.21.15.196:6805/2877039731","state":["exists","up"]},{"osd":3,"uuid":"3f731c38-2e45-4ca1-8dce-efc4a1b2166b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6811","nonce":4276311194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6813","nonce":4276311194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6817","nonce":4276311194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6815","nonce":4276311194}]},"public_addr":"172.21.15.97:6811/4276311194","cluster_addr":"172.21.15.97:6813/4276311194","heartbeat_back_addr":"172.21.15.97:6817/4276311194","heartbeat_front_addr":"172.21.15.97:6815/4276311194","state":["exists","up"]},{"osd":4,"uuid":"fdcafb14-9b82-4134-bd5b-79c0516f6cbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6809","nonce":63913916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6811","nonce":63913916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6815","nonce":63913916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6813","nonce":63913916}]},"public_addr":"172.21.15.183:6809/63913916","cluster_addr":"172.21.15.183:6811/63913916","heartbeat_back_addr":"172.21.15.183:6815/63913916","heartbeat_front_addr":"172.21.15.183:6813/63913916","state":["exists","up"]},{"osd":5,"uuid":"cdcc18d8-da3c-4012-bf12-876bb39e66eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6809","nonce":2804446896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6811","nonce":2804446896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6815","nonce":2804446896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6813","nonce":2804446896}]},"public_addr":"172.21.15.196:6809/2804446896","cluster_addr":"172.21.15.196:6811/2804446896","heartbeat_back_addr":"172.21.15.196:6815/2804446896","heartbeat_front_addr":"172.21.15.196:6813/2804446896","state":["exists","up"]},{"osd":6,"uuid":"071abb5c-2edf-4832-b8aa-3b80d20096fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6819","nonce":2080916705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6821","nonce":2080916705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6825","nonce":2080916705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6823","nonce":2080916705}]},"public_addr":"172.21.15.97:6819/2080916705","cluster_addr":"172.21.15.97:6821/2080916705","heartbeat_back_addr":"172.21.15.97:6825/2080916705","heartbeat_front_addr":"172.21.15.97:6823/2080916705","state":["exists","up"]},{"osd":7,"uuid":"9d54760b-66df-486b-8abb-017f3b64b4d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6817","nonce":1865873033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6819","nonce":1865873033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6823","nonce":1865873033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6821","nonce":1865873033}]},"public_addr":"172.21.15.183:6817/1865873033","cluster_addr":"172.21.15.183:6819/1865873033","heartbeat_back_addr":"172.21.15.183:6823/1865873033","heartbeat_front_addr":"172.21.15.183:6821/1865873033","state":["exists","up"]},{"osd":8,"uuid":"5cd3100a-bd6a-4be5-b5e9-0d0afc325193","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6817","nonce":4063932139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6819","nonce":4063932139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6823","nonce":4063932139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6821","nonce":4063932139}]},"public_addr":"172.21.15.196:6817/4063932139","cluster_addr":"172.21.15.196:6819/4063932139","heartbeat_back_addr":"172.21.15.196:6823/4063932139","heartbeat_front_addr":"172.21.15.196:6821/4063932139","state":["exists","up"]},{"osd":9,"uuid":"b6a5f4a1-4ecc-4f73-a2df-07551d9298ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6827","nonce":1887432474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6829","nonce":1887432474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6833","nonce":1887432474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6831","nonce":1887432474}]},"public_addr":"172.21.15.97:6827/1887432474","cluster_addr":"172.21.15.97:6829/1887432474","heartbeat_back_addr":"172.21.15.97:6833/1887432474","heartbeat_front_addr":"172.21.15.97:6831/1887432474","state":["exists","up"]},{"osd":10,"uuid":"86f59816-1ea6-4ae9-8796-cf5fc4831064","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6825","nonce":799434647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6827","nonce":799434647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6831","nonce":799434647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6829","nonce":799434647}]},"public_addr":"172.21.15.183:6825/799434647","cluster_addr":"172.21.15.183:6827/799434647","heartbeat_back_addr":"172.21.15.183:6831/799434647","heartbeat_front_addr":"172.21.15.183:6829/799434647","state":["exists","up"]},{"osd":11,"uuid":"95d64a89-a9f8-4c7a-8c0d-da7fb26cfff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6825","nonce":1873969467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6827","nonce":1873969467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6831","nonce":1873969467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6829","nonce":1873969467}]},"public_addr":"172.21.15.196:6825/1873969467","cluster_addr":"172.21.15.196:6827/1873969467","heartbeat_back_addr":"172.21.15.196:6831/1873969467","heartbeat_front_addr":"172.21.15.196:6829/1873969467","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:18:33.721156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:18:57.586709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:19:22.069458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:19:46.451597+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:20:11.561019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:20:36.298672+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:21:01.661804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:21:27.324128+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:21:52.898197+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:22:19.614326+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:22:45.154917+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:23:11.222612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/3866016040":"2024-04-24T10:16:59.468237+0000","172.21.15.97:6801/3297166858":"2024-04-24T10:16:59.468237+0000","172.21.15.97:0/3804743548":"2024-04-24T10:16:59.468237+0000","172.21.15.97:6800/3706463017":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/3715079438":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/786806834":"2024-04-24T10:16:59.468237+0000","172.21.15.97:6801/3706463017":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/3294175554":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/1449136760":"2024-04-24T10:16:33.128917+0000","172.21.15.97:6800/3297166858":"2024-04-24T10:16:59.468237+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T10:25:33.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.374+0000 7fae37ad0640 1 -- 172.21.15.97:0/3969743731 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fae1c0a3bb0 msgr2=0x7fae1c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:33.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.374+0000 7fae37ad0640 1 --2- 172.21.15.97:0/3969743731 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fae1c0a3bb0 0x7fae1c0a6070 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fae2c006010 tx=0x7fae2c0093f0 comp rx=0 tx=0).stop 2024-04-23T10:25:33.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.374+0000 7fae37ad0640 1 -- 172.21.15.97:0/3969743731 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae30101fe0 msgr2=0x7fae30198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:33.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.374+0000 7fae37ad0640 1 --2- 172.21.15.97:0/3969743731 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae30101fe0 0x7fae30198720 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fae2000c970 tx=0x7fae2000ce30 comp rx=0 tx=0).stop 2024-04-23T10:25:33.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.374+0000 7fae37ad0640 1 -- 172.21.15.97:0/3969743731 shutdown_connections 2024-04-23T10:25:33.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.374+0000 7fae37ad0640 1 --2- 172.21.15.97:0/3969743731 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fae1c0a3bb0 0x7fae1c0a6070 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:33.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.374+0000 7fae37ad0640 1 --2- 172.21.15.97:0/3969743731 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae301029a0 0x7fae3019f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:33.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.374+0000 7fae37ad0640 1 --2- 172.21.15.97:0/3969743731 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae30101fe0 0x7fae30198720 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:33.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.374+0000 7fae37ad0640 1 --2- 172.21.15.97:0/3969743731 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae30100e90 0x7fae301981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:33.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.374+0000 7fae37ad0640 1 -- 172.21.15.97:0/3969743731 >> 172.21.15.97:0/3969743731 conn(0x7fae300fc400 msgr2=0x7fae30105460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:33.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.375+0000 7fae37ad0640 1 -- 172.21.15.97:0/3969743731 shutdown_connections 2024-04-23T10:25:33.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.375+0000 7fae37ad0640 1 -- 172.21.15.97:0/3969743731 wait complete. 2024-04-23T10:25:33.385 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T10:19:25.559138+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T10:24:04.570049+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T10:24:05.861631+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T10:25:33.385 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T10:25:33.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.487+0000 7f816ae79640 1 -- 172.21.15.97:0/1969419924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8164100ea0 msgr2=0x7f81641012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:33.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.487+0000 7f816ae79640 1 --2- 172.21.15.97:0/1969419924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8164100ea0 0x7f81641012c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8154009900 tx=0x7f815402a410 comp rx=0 tx=0).stop 2024-04-23T10:25:33.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.487+0000 7f816ae79640 1 -- 172.21.15.97:0/1969419924 shutdown_connections 2024-04-23T10:25:33.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.487+0000 7f816ae79640 1 --2- 172.21.15.97:0/1969419924 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f81641029b0 0x7f81641092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:33.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.487+0000 7f816ae79640 1 --2- 172.21.15.97:0/1969419924 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8164101ff0 0x7f8164102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:33.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.487+0000 7f816ae79640 1 --2- 172.21.15.97:0/1969419924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8164100ea0 0x7f81641012c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:33.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.487+0000 7f816ae79640 1 -- 172.21.15.97:0/1969419924 >> 172.21.15.97:0/1969419924 conn(0x7f81640fc410 msgr2=0x7f81640fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:33.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.488+0000 7f816ae79640 1 -- 172.21.15.97:0/1969419924 shutdown_connections 2024-04-23T10:25:33.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.488+0000 7f816ae79640 1 -- 172.21.15.97:0/1969419924 wait complete. 2024-04-23T10:25:33.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.488+0000 7f816ae79640 1 Processor -- start 2024-04-23T10:25:33.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.488+0000 7f816ae79640 1 -- start start 2024-04-23T10:25:33.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.488+0000 7f816ae79640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8164100ea0 0x7f816419a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:33.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.488+0000 7f816ae79640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8164101ff0 0x7f816419a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:33.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.488+0000 7f816ae79640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81641029b0 0x7f81641a1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:33.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.488+0000 7f816ae79640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f816410db00 con 0x7f8164100ea0 2024-04-23T10:25:33.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.488+0000 7f816ae79640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f816410d980 con 0x7f81641029b0 2024-04-23T10:25:33.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.488+0000 7f816ae79640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f816410dc80 con 0x7f8164101ff0 2024-04-23T10:25:33.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.488+0000 7f8168c15640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8164100ea0 0x7f816419a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:33.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.488+0000 7f8168c15640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8164100ea0 0x7f816419a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33890/0 (socket says 172.21.15.97:33890) 2024-04-23T10:25:33.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.488+0000 7f8168c15640 1 -- 172.21.15.97:0/1359063686 learned_addr learned my addr 172.21.15.97:0/1359063686 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:25:33.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.488+0000 7f8169416640 1 --2- 172.21.15.97:0/1359063686 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81641029b0 0x7f81641a1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:33.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.489+0000 7f8168c15640 1 -- 172.21.15.97:0/1359063686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8164101ff0 msgr2=0x7f816419a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:33.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.489+0000 7f8168c15640 1 --2- 172.21.15.97:0/1359063686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8164101ff0 0x7f816419a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:33.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.489+0000 7f8168c15640 1 -- 172.21.15.97:0/1359063686 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81641029b0 msgr2=0x7f81641a1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:33.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.489+0000 7f8168c15640 1 --2- 172.21.15.97:0/1359063686 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81641029b0 0x7f81641a1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:33.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.489+0000 7f8168c15640 1 -- 172.21.15.97:0/1359063686 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81641a2080 con 0x7f8164100ea0 2024-04-23T10:25:33.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.489+0000 7f8169416640 1 --2- 172.21.15.97:0/1359063686 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81641029b0 0x7f81641a1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:25:33.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.489+0000 7f8168c15640 1 --2- 172.21.15.97:0/1359063686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8164100ea0 0x7f816419a460 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8154031500 tx=0x7f81540043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:33.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.489+0000 7f8161ffb640 1 -- 172.21.15.97:0/1359063686 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f815403f030 con 0x7f8164100ea0 2024-04-23T10:25:33.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.489+0000 7f8161ffb640 1 -- 172.21.15.97:0/1359063686 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8154003800 con 0x7f8164100ea0 2024-04-23T10:25:33.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.489+0000 7f8161ffb640 1 -- 172.21.15.97:0/1359063686 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8154035070 con 0x7f8164100ea0 2024-04-23T10:25:33.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.489+0000 7f816ae79640 1 -- 172.21.15.97:0/1359063686 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81641a2310 con 0x7f8164100ea0 2024-04-23T10:25:33.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.489+0000 7f816ae79640 1 -- 172.21.15.97:0/1359063686 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81641a2800 con 0x7f8164100ea0 2024-04-23T10:25:33.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.490+0000 7f816ae79640 1 -- 172.21.15.97:0/1359063686 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81641a2b20 con 0x7f8164100ea0 2024-04-23T10:25:33.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.491+0000 7f8161ffb640 1 -- 172.21.15.97:0/1359063686 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f81540039b0 con 0x7f8164100ea0 2024-04-23T10:25:33.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.491+0000 7f8161ffb640 1 --2- 172.21.15.97:0/1359063686 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f814c0a3bb0 0x7f814c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:33.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.492+0000 7f8163fff640 1 --2- 172.21.15.97:0/1359063686 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f814c0a3bb0 0x7f814c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:33.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.492+0000 7f8161ffb640 1 -- 172.21.15.97:0/1359063686 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f81540f0ec0 con 0x7f8164100ea0 2024-04-23T10:25:33.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.492+0000 7f8163fff640 1 --2- 172.21.15.97:0/1359063686 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f814c0a3bb0 0x7f814c0a6070 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f816419b970 tx=0x7f8158005c10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:33.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.496+0000 7f8161ffb640 1 -- 172.21.15.97:0/1359063686 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f81540bd8d0 con 0x7f8164100ea0 2024-04-23T10:25:33.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.676+0000 7f816ae79640 1 -- 172.21.15.97:0/1359063686 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f81641a2cf0 con 0x7f8164100ea0 2024-04-23T10:25:33.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.677+0000 7f8161ffb640 1 -- 172.21.15.97:0/1359063686 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f81540bd1d0 con 0x7f8164100ea0 2024-04-23T10:25:33.676 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-23T10:25:33.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.679+0000 7f816ae79640 1 -- 172.21.15.97:0/1359063686 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f814c0a3bb0 msgr2=0x7f814c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:33.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.679+0000 7f816ae79640 1 --2- 172.21.15.97:0/1359063686 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f814c0a3bb0 0x7f814c0a6070 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f816419b970 tx=0x7f8158005c10 comp rx=0 tx=0).stop 2024-04-23T10:25:33.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.679+0000 7f816ae79640 1 -- 172.21.15.97:0/1359063686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8164100ea0 msgr2=0x7f816419a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:33.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.679+0000 7f816ae79640 1 --2- 172.21.15.97:0/1359063686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8164100ea0 0x7f816419a460 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8154031500 tx=0x7f81540043c0 comp rx=0 tx=0).stop 2024-04-23T10:25:33.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.679+0000 7f816ae79640 1 -- 172.21.15.97:0/1359063686 shutdown_connections 2024-04-23T10:25:33.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.679+0000 7f816ae79640 1 --2- 172.21.15.97:0/1359063686 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f814c0a3bb0 0x7f814c0a6070 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:33.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.679+0000 7f816ae79640 1 --2- 172.21.15.97:0/1359063686 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81641029b0 0x7f81641a1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:33.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.679+0000 7f816ae79640 1 --2- 172.21.15.97:0/1359063686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8164101ff0 0x7f816419a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:33.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.679+0000 7f816ae79640 1 --2- 172.21.15.97:0/1359063686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8164100ea0 0x7f816419a460 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:33.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.679+0000 7f816ae79640 1 -- 172.21.15.97:0/1359063686 >> 172.21.15.97:0/1359063686 conn(0x7f81640fc410 msgr2=0x7f8164109cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:33.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.679+0000 7f816ae79640 1 -- 172.21.15.97:0/1359063686 shutdown_connections 2024-04-23T10:25:33.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.679+0000 7f816ae79640 1 -- 172.21.15.97:0/1359063686 wait complete. 2024-04-23T10:25:33.689 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T10:25:33.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.791+0000 7f34311e2640 1 -- 172.21.15.97:0/1896338856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f342c101fc0 msgr2=0x7f342c102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:33.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.791+0000 7f34311e2640 1 --2- 172.21.15.97:0/1896338856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f342c101fc0 0x7f342c102440 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f3420009900 tx=0x7f342002a410 comp rx=0 tx=0).stop 2024-04-23T10:25:33.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.791+0000 7f34311e2640 1 -- 172.21.15.97:0/1896338856 shutdown_connections 2024-04-23T10:25:33.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.791+0000 7f34311e2640 1 --2- 172.21.15.97:0/1896338856 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f342c102980 0x7f342c1092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:33.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.791+0000 7f34311e2640 1 --2- 172.21.15.97:0/1896338856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f342c101fc0 0x7f342c102440 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:33.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.791+0000 7f34311e2640 1 --2- 172.21.15.97:0/1896338856 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f342c100e70 0x7f342c101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:33.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.791+0000 7f34311e2640 1 -- 172.21.15.97:0/1896338856 >> 172.21.15.97:0/1896338856 conn(0x7f342c0fc420 msgr2=0x7f342c0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:33.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.791+0000 7f34311e2640 1 -- 172.21.15.97:0/1896338856 shutdown_connections 2024-04-23T10:25:33.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.791+0000 7f34311e2640 1 -- 172.21.15.97:0/1896338856 wait complete. 2024-04-23T10:25:33.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.792+0000 7f34311e2640 1 Processor -- start 2024-04-23T10:25:33.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.792+0000 7f34311e2640 1 -- start start 2024-04-23T10:25:33.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.792+0000 7f34311e2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f342c100e70 0x7f342c06b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:33.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.792+0000 7f34311e2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f342c101fc0 0x7f342c19cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:33.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.792+0000 7f34311e2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f342c102980 0x7f342c19f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:33.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.792+0000 7f34311e2640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f342c10ca20 con 0x7f342c101fc0 2024-04-23T10:25:33.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.792+0000 7f34311e2640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f342c10c8a0 con 0x7f342c102980 2024-04-23T10:25:33.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.792+0000 7f34311e2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f342c10cba0 con 0x7f342c100e70 2024-04-23T10:25:33.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.792+0000 7f342a59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f342c101fc0 0x7f342c19cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:33.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.792+0000 7f342a59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f342c101fc0 0x7f342c19cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33902/0 (socket says 172.21.15.97:33902) 2024-04-23T10:25:33.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.792+0000 7f342a59c640 1 -- 172.21.15.97:0/2125157266 learned_addr learned my addr 172.21.15.97:0/2125157266 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:25:33.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.792+0000 7f342b59e640 1 --2- 172.21.15.97:0/2125157266 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f342c102980 0x7f342c19f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:33.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.792+0000 7f342ad9d640 1 --2- 172.21.15.97:0/2125157266 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f342c100e70 0x7f342c06b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:33.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.792+0000 7f342a59c640 1 -- 172.21.15.97:0/2125157266 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f342c100e70 msgr2=0x7f342c06b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:33.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.792+0000 7f342a59c640 1 --2- 172.21.15.97:0/2125157266 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f342c100e70 0x7f342c06b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:33.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.792+0000 7f342a59c640 1 -- 172.21.15.97:0/2125157266 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f342c102980 msgr2=0x7f342c19f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:33.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.792+0000 7f342a59c640 1 --2- 172.21.15.97:0/2125157266 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f342c102980 0x7f342c19f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:33.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.792+0000 7f342a59c640 1 -- 172.21.15.97:0/2125157266 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f342c19fdf0 con 0x7f342c101fc0 2024-04-23T10:25:33.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.792+0000 7f342b59e640 1 --2- 172.21.15.97:0/2125157266 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f342c102980 0x7f342c19f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:25:33.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.793+0000 7f342a59c640 1 --2- 172.21.15.97:0/2125157266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f342c101fc0 0x7f342c19cef0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3420031500 tx=0x7f3420003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:33.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.793+0000 7f3413fff640 1 -- 172.21.15.97:0/2125157266 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342003f070 con 0x7f342c101fc0 2024-04-23T10:25:33.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.793+0000 7f34311e2640 1 -- 172.21.15.97:0/2125157266 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f342c1a0080 con 0x7f342c101fc0 2024-04-23T10:25:33.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.793+0000 7f3413fff640 1 -- 172.21.15.97:0/2125157266 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3420003e70 con 0x7f342c101fc0 2024-04-23T10:25:33.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.793+0000 7f3413fff640 1 -- 172.21.15.97:0/2125157266 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3420035030 con 0x7f342c101fc0 2024-04-23T10:25:33.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.793+0000 7f34311e2640 1 -- 172.21.15.97:0/2125157266 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f342c1a0570 con 0x7f342c101fc0 2024-04-23T10:25:33.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.794+0000 7f34311e2640 1 -- 172.21.15.97:0/2125157266 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f342c108180 con 0x7f342c101fc0 2024-04-23T10:25:33.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.795+0000 7f3413fff640 1 -- 172.21.15.97:0/2125157266 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3420042070 con 0x7f342c101fc0 2024-04-23T10:25:33.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.795+0000 7f3413fff640 1 --2- 172.21.15.97:0/2125157266 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f34140a3bb0 0x7f34140a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:33.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.796+0000 7f342ad9d640 1 --2- 172.21.15.97:0/2125157266 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f34140a3bb0 0x7f34140a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:33.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.796+0000 7f3413fff640 1 -- 172.21.15.97:0/2125157266 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f34200f1100 con 0x7f342c101fc0 2024-04-23T10:25:33.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.796+0000 7f342ad9d640 1 --2- 172.21.15.97:0/2125157266 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f34140a3bb0 0x7f34140a6070 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f341c005fd0 tx=0x7f341c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:33.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.800+0000 7f3413fff640 1 -- 172.21.15.97:0/2125157266 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f34200bdb10 con 0x7f342c101fc0 2024-04-23T10:25:33.975 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.976+0000 7f34311e2640 1 -- 172.21.15.97:0/2125157266 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f342c062b50 con 0x7f342c101fc0 2024-04-23T10:25:33.976 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.977+0000 7f3413fff640 1 -- 172.21.15.97:0/2125157266 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f34200bd410 con 0x7f342c101fc0 2024-04-23T10:25:33.976 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-23T10:25:33.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.979+0000 7f34311e2640 1 -- 172.21.15.97:0/2125157266 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f34140a3bb0 msgr2=0x7f34140a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:33.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.979+0000 7f34311e2640 1 --2- 172.21.15.97:0/2125157266 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f34140a3bb0 0x7f34140a6070 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f341c005fd0 tx=0x7f341c005850 comp rx=0 tx=0).stop 2024-04-23T10:25:33.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.979+0000 7f34311e2640 1 -- 172.21.15.97:0/2125157266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f342c101fc0 msgr2=0x7f342c19cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:33.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.979+0000 7f34311e2640 1 --2- 172.21.15.97:0/2125157266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f342c101fc0 0x7f342c19cef0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3420031500 tx=0x7f3420003c80 comp rx=0 tx=0).stop 2024-04-23T10:25:33.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.979+0000 7f34311e2640 1 -- 172.21.15.97:0/2125157266 shutdown_connections 2024-04-23T10:25:33.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.979+0000 7f34311e2640 1 --2- 172.21.15.97:0/2125157266 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f34140a3bb0 0x7f34140a6070 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:33.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.979+0000 7f34311e2640 1 --2- 172.21.15.97:0/2125157266 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f342c102980 0x7f342c19f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:33.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.979+0000 7f34311e2640 1 --2- 172.21.15.97:0/2125157266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f342c101fc0 0x7f342c19cef0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:33.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.979+0000 7f34311e2640 1 --2- 172.21.15.97:0/2125157266 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f342c100e70 0x7f342c06b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:33.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.979+0000 7f34311e2640 1 -- 172.21.15.97:0/2125157266 >> 172.21.15.97:0/2125157266 conn(0x7f342c0fc420 msgr2=0x7f342c105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:33.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.979+0000 7f34311e2640 1 -- 172.21.15.97:0/2125157266 shutdown_connections 2024-04-23T10:25:33.979 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:33.979+0000 7f34311e2640 1 -- 172.21.15.97:0/2125157266 wait complete. 2024-04-23T10:25:33.988 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:33 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/3969743731' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T10:25:33.988 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:33 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/1359063686' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T10:25:33.989 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T10:25:34.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.091+0000 7f48303ad640 1 -- 172.21.15.97:0/4033953131 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4828105400 msgr2=0x7f48281077f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:34.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.091+0000 7f48303ad640 1 --2- 172.21.15.97:0/4033953131 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4828105400 0x7f48281077f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f481c009900 tx=0x7f481c02a410 comp rx=0 tx=0).stop 2024-04-23T10:25:34.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.091+0000 7f48303ad640 1 -- 172.21.15.97:0/4033953131 shutdown_connections 2024-04-23T10:25:34.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.091+0000 7f48303ad640 1 --2- 172.21.15.97:0/4033953131 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4828107d30 0x7f482810a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:34.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.091+0000 7f48303ad640 1 --2- 172.21.15.97:0/4033953131 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4828105400 0x7f48281077f0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:34.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.091+0000 7f48303ad640 1 --2- 172.21.15.97:0/4033953131 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4828102ad0 0x7f4828104ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:34.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.091+0000 7f48303ad640 1 -- 172.21.15.97:0/4033953131 >> 172.21.15.97:0/4033953131 conn(0x7f48280fc440 msgr2=0x7f48280fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:34.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.091+0000 7f48303ad640 1 -- 172.21.15.97:0/4033953131 shutdown_connections 2024-04-23T10:25:34.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.091+0000 7f48303ad640 1 -- 172.21.15.97:0/4033953131 wait complete. 2024-04-23T10:25:34.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.091+0000 7f48303ad640 1 Processor -- start 2024-04-23T10:25:34.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.092+0000 7f48303ad640 1 -- start start 2024-04-23T10:25:34.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.092+0000 7f48303ad640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4828102ad0 0x7f48281981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:34.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.092+0000 7f48303ad640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4828105400 0x7f4828198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:34.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.092+0000 7f48303ad640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4828107d30 0x7f482819f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:34.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.092+0000 7f48303ad640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f482810db10 con 0x7f4828102ad0 2024-04-23T10:25:34.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.092+0000 7f48303ad640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f482810d990 con 0x7f4828107d30 2024-04-23T10:25:34.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.092+0000 7f48303ad640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f482810dc90 con 0x7f4828105400 2024-04-23T10:25:34.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.092+0000 7f482e149640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4828102ad0 0x7f48281981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:34.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.092+0000 7f482e149640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4828102ad0 0x7f48281981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33920/0 (socket says 172.21.15.97:33920) 2024-04-23T10:25:34.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.092+0000 7f482e149640 1 -- 172.21.15.97:0/4251332212 learned_addr learned my addr 172.21.15.97:0/4251332212 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:25:34.093 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.092+0000 7f482d948640 1 --2- 172.21.15.97:0/4251332212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4828105400 0x7f4828198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:34.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.092+0000 7f482e94a640 1 --2- 172.21.15.97:0/4251332212 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4828107d30 0x7f482819f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:34.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.092+0000 7f482e149640 1 -- 172.21.15.97:0/4251332212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4828105400 msgr2=0x7f4828198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:34.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.092+0000 7f482e149640 1 --2- 172.21.15.97:0/4251332212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4828105400 0x7f4828198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:34.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.092+0000 7f482e149640 1 -- 172.21.15.97:0/4251332212 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4828107d30 msgr2=0x7f482819f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:34.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.092+0000 7f482e149640 1 --2- 172.21.15.97:0/4251332212 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4828107d30 0x7f482819f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:34.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.092+0000 7f482e149640 1 -- 172.21.15.97:0/4251332212 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f482819fe00 con 0x7f4828102ad0 2024-04-23T10:25:34.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.092+0000 7f482e94a640 1 --2- 172.21.15.97:0/4251332212 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4828107d30 0x7f482819f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:25:34.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.092+0000 7f482e149640 1 --2- 172.21.15.97:0/4251332212 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4828102ad0 0x7f48281981e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f482000cce0 tx=0x7f482000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:34.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.092+0000 7f48277fe640 1 -- 172.21.15.97:0/4251332212 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4820004b20 con 0x7f4828102ad0 2024-04-23T10:25:34.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.092+0000 7f48277fe640 1 -- 172.21.15.97:0/4251332212 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4820004cc0 con 0x7f4828102ad0 2024-04-23T10:25:34.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.092+0000 7f48303ad640 1 -- 172.21.15.97:0/4251332212 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48281a00f0 con 0x7f4828102ad0 2024-04-23T10:25:34.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.092+0000 7f48277fe640 1 -- 172.21.15.97:0/4251332212 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482000f6b0 con 0x7f4828102ad0 2024-04-23T10:25:34.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.092+0000 7f48303ad640 1 -- 172.21.15.97:0/4251332212 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48281a0610 con 0x7f4828102ad0 2024-04-23T10:25:34.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.093+0000 7f48303ad640 1 -- 172.21.15.97:0/4251332212 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48280fdf30 con 0x7f4828102ad0 2024-04-23T10:25:34.095 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:33 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/3969743731' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T10:25:34.095 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:33 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/1359063686' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T10:25:34.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.094+0000 7f48277fe640 1 -- 172.21.15.97:0/4251332212 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f482000f850 con 0x7f4828102ad0 2024-04-23T10:25:34.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.095+0000 7f48277fe640 1 --2- 172.21.15.97:0/4251332212 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f48100a3bb0 0x7f48100a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:34.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.095+0000 7f482d948640 1 --2- 172.21.15.97:0/4251332212 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f48100a3bb0 0x7f48100a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:34.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.095+0000 7f48277fe640 1 -- 172.21.15.97:0/4251332212 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f48200d08b0 con 0x7f4828102ad0 2024-04-23T10:25:34.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.095+0000 7f482d948640 1 --2- 172.21.15.97:0/4251332212 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f48100a3bb0 0x7f48100a6070 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f481c000c00 tx=0x7f481c0314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:34.098 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.099+0000 7f48277fe640 1 -- 172.21.15.97:0/4251332212 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4820014030 con 0x7f4828102ad0 2024-04-23T10:25:34.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:33 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/3969743731' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T10:25:34.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:33 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/1359063686' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T10:25:34.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.281+0000 7f48303ad640 1 -- 172.21.15.97:0/4251332212 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4828062b50 con 0x7f4828102ad0 2024-04-23T10:25:34.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.282+0000 7f48277fe640 1 -- 172.21.15.97:0/4251332212 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f482009d2c0 con 0x7f4828102ad0 2024-04-23T10:25:34.281 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-04-23T10:25:34.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.284+0000 7f48303ad640 1 -- 172.21.15.97:0/4251332212 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f48100a3bb0 msgr2=0x7f48100a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:34.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.284+0000 7f48303ad640 1 --2- 172.21.15.97:0/4251332212 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f48100a3bb0 0x7f48100a6070 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f481c000c00 tx=0x7f481c0314a0 comp rx=0 tx=0).stop 2024-04-23T10:25:34.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.284+0000 7f48303ad640 1 -- 172.21.15.97:0/4251332212 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4828102ad0 msgr2=0x7f48281981e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:34.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.284+0000 7f48303ad640 1 --2- 172.21.15.97:0/4251332212 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4828102ad0 0x7f48281981e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f482000cce0 tx=0x7f482000a620 comp rx=0 tx=0).stop 2024-04-23T10:25:34.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.284+0000 7f48303ad640 1 -- 172.21.15.97:0/4251332212 shutdown_connections 2024-04-23T10:25:34.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.284+0000 7f48303ad640 1 --2- 172.21.15.97:0/4251332212 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f48100a3bb0 0x7f48100a6070 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:34.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.284+0000 7f48303ad640 1 --2- 172.21.15.97:0/4251332212 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4828107d30 0x7f482819f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:34.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.284+0000 7f48303ad640 1 --2- 172.21.15.97:0/4251332212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4828105400 0x7f4828198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:34.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.284+0000 7f48303ad640 1 --2- 172.21.15.97:0/4251332212 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4828102ad0 0x7f48281981e0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:34.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.284+0000 7f48303ad640 1 -- 172.21.15.97:0/4251332212 >> 172.21.15.97:0/4251332212 conn(0x7f48280fc440 msgr2=0x7f48281033d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:34.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.284+0000 7f48303ad640 1 -- 172.21.15.97:0/4251332212 shutdown_connections 2024-04-23T10:25:34.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.284+0000 7f48303ad640 1 -- 172.21.15.97:0/4251332212 wait complete. 2024-04-23T10:25:34.294 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T10:25:34.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.393+0000 7f5dd769e640 1 -- 172.21.15.97:0/470621388 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5dd0102a00 msgr2=0x7f5dd0109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:34.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.393+0000 7f5dd769e640 1 --2- 172.21.15.97:0/470621388 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5dd0102a00 0x7f5dd0109320 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5dc8009ca0 tx=0x7f5dc802a8c0 comp rx=0 tx=0).stop 2024-04-23T10:25:34.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.393+0000 7f5dd769e640 1 -- 172.21.15.97:0/470621388 shutdown_connections 2024-04-23T10:25:34.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.393+0000 7f5dd769e640 1 --2- 172.21.15.97:0/470621388 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5dd0102a00 0x7f5dd0109320 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:34.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.393+0000 7f5dd769e640 1 --2- 172.21.15.97:0/470621388 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5dd0102040 0x7f5dd01024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:34.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.393+0000 7f5dd769e640 1 --2- 172.21.15.97:0/470621388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5dd0100e60 0x7f5dd0101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:34.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.393+0000 7f5dd769e640 1 -- 172.21.15.97:0/470621388 >> 172.21.15.97:0/470621388 conn(0x7f5dd00fc440 msgr2=0x7f5dd00fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:34.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.393+0000 7f5dd769e640 1 -- 172.21.15.97:0/470621388 shutdown_connections 2024-04-23T10:25:34.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.393+0000 7f5dd769e640 1 -- 172.21.15.97:0/470621388 wait complete. 2024-04-23T10:25:34.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.394+0000 7f5dd769e640 1 Processor -- start 2024-04-23T10:25:34.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.394+0000 7f5dd769e640 1 -- start start 2024-04-23T10:25:34.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.394+0000 7f5dd769e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5dd0100e60 0x7f5dd006b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:34.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.394+0000 7f5dd769e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5dd0102040 0x7f5dd019ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:34.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.394+0000 7f5dd769e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5dd0102a00 0x7f5dd019f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:34.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.394+0000 7f5dd769e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd010ca00 con 0x7f5dd0100e60 2024-04-23T10:25:34.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.394+0000 7f5dd769e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd010c880 con 0x7f5dd0102a00 2024-04-23T10:25:34.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.394+0000 7f5dd769e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd010cb80 con 0x7f5dd0102040 2024-04-23T10:25:34.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.394+0000 7f5dd543a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5dd0100e60 0x7f5dd006b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:34.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.394+0000 7f5dd543a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5dd0100e60 0x7f5dd006b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33934/0 (socket says 172.21.15.97:33934) 2024-04-23T10:25:34.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.394+0000 7f5dd543a640 1 -- 172.21.15.97:0/2148349299 learned_addr learned my addr 172.21.15.97:0/2148349299 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:25:34.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.394+0000 7f5dd5c3b640 1 --2- 172.21.15.97:0/2148349299 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5dd0102a00 0x7f5dd019f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:34.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.394+0000 7f5dd4c39640 1 --2- 172.21.15.97:0/2148349299 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5dd0102040 0x7f5dd019ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:34.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.394+0000 7f5dd543a640 1 -- 172.21.15.97:0/2148349299 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5dd0102040 msgr2=0x7f5dd019ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:34.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.394+0000 7f5dd543a640 1 --2- 172.21.15.97:0/2148349299 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5dd0102040 0x7f5dd019ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:34.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.394+0000 7f5dd543a640 1 -- 172.21.15.97:0/2148349299 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5dd0102a00 msgr2=0x7f5dd019f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:34.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.394+0000 7f5dd543a640 1 --2- 172.21.15.97:0/2148349299 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5dd0102a00 0x7f5dd019f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:34.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.394+0000 7f5dd543a640 1 -- 172.21.15.97:0/2148349299 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dd019fdd0 con 0x7f5dd0100e60 2024-04-23T10:25:34.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.394+0000 7f5dd5c3b640 1 --2- 172.21.15.97:0/2148349299 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5dd0102a00 0x7f5dd019f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:25:34.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.394+0000 7f5dd543a640 1 --2- 172.21.15.97:0/2148349299 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5dd0100e60 0x7f5dd006b140 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc00d770 tx=0x7f5dcc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:34.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.394+0000 7f5dc67fc640 1 -- 172.21.15.97:0/2148349299 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc017070 con 0x7f5dd0100e60 2024-04-23T10:25:34.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.394+0000 7f5dd769e640 1 -- 172.21.15.97:0/2148349299 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dd01a00c0 con 0x7f5dd0100e60 2024-04-23T10:25:34.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.394+0000 7f5dc67fc640 1 -- 172.21.15.97:0/2148349299 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dcc009c30 con 0x7f5dd0100e60 2024-04-23T10:25:34.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.394+0000 7f5dc67fc640 1 -- 172.21.15.97:0/2148349299 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc003cc0 con 0x7f5dd0100e60 2024-04-23T10:25:34.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.395+0000 7f5dd769e640 1 -- 172.21.15.97:0/2148349299 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dd01a0660 con 0x7f5dd0100e60 2024-04-23T10:25:34.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.395+0000 7f5dd769e640 1 -- 172.21.15.97:0/2148349299 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dd0108200 con 0x7f5dd0100e60 2024-04-23T10:25:34.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.397+0000 7f5dc67fc640 1 -- 172.21.15.97:0/2148349299 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5dcc003e60 con 0x7f5dd0100e60 2024-04-23T10:25:34.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.397+0000 7f5dc67fc640 1 --2- 172.21.15.97:0/2148349299 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f5dbc0a3bb0 0x7f5dbc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:34.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.397+0000 7f5dd4c39640 1 --2- 172.21.15.97:0/2148349299 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f5dbc0a3bb0 0x7f5dbc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:34.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.397+0000 7f5dc67fc640 1 -- 172.21.15.97:0/2148349299 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f5dcc0d0250 con 0x7f5dd0100e60 2024-04-23T10:25:34.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.397+0000 7f5dd4c39640 1 --2- 172.21.15.97:0/2148349299 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f5dbc0a3bb0 0x7f5dbc0a6070 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5dc0005fd0 tx=0x7f5dc0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:34.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.401+0000 7f5dc67fc640 1 -- 172.21.15.97:0/2148349299 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5dcc009de0 con 0x7f5dd0100e60 2024-04-23T10:25:34.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.620+0000 7f5dd769e640 1 -- 172.21.15.97:0/2148349299 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5dd002d800 con 0x7f5dd0100e60 2024-04-23T10:25:34.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.621+0000 7f5dc67fc640 1 -- 172.21.15.97:0/2148349299 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11931 (secure 0 0 0) 0x7f5dcc09cc60 con 0x7f5dd0100e60 2024-04-23T10:25:34.620 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:25:34.620 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3946878762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3946878762},{"type":"v1","addr":"172.21.15.97:6835","nonce":3946878762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/209800861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":209800861},{"type":"v1","addr":"172.21.15.97:6837","nonce":209800861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T10:24:06.910134+0000","modified":"2024-04-23T10:24:12.969478+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6839/418013575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":418013575},{"type":"v1","addr":"172.21.15.97:6839","nonce":418013575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6841/747242680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":747242680},{"type":"v1","addr":"172.21.15.97:6841","nonce":747242680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6833/4292847121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4292847121},{"type":"v1","addr":"172.21.15.183:6833","nonce":4292847121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6833/3885816565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3885816565},{"type":"v1","addr":"172.21.15.196:6833","nonce":3885816565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1897093105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1897093105},{"type":"v1","addr":"172.21.15.183:6835","nonce":1897093105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1171477563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1171477563},{"type":"v1","addr":"172.21.15.196:6835","nonce":1171477563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1898341911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1898341911},{"type":"v1","addr":"172.21.15.183:6837","nonce":1898341911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.196:6837/3094946502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3094946502},{"type":"v1","addr":"172.21.15.196:6837","nonce":3094946502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.196:6839/83843556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":83843556},{"type":"v1","addr":"172.21.15.196:6839","nonce":83843556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6839/1763841130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1763841130},{"type":"v1","addr":"172.21.15.183:6839","nonce":1763841130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24469,24488,24398,24403,24487]},"id":1}]} 2024-04-23T10:25:34.620 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 25 2024-04-23T10:25:34.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.623+0000 7f5dd769e640 1 -- 172.21.15.97:0/2148349299 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f5dbc0a3bb0 msgr2=0x7f5dbc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:34.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.623+0000 7f5dd769e640 1 --2- 172.21.15.97:0/2148349299 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f5dbc0a3bb0 0x7f5dbc0a6070 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5dc0005fd0 tx=0x7f5dc0005850 comp rx=0 tx=0).stop 2024-04-23T10:25:34.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.623+0000 7f5dd769e640 1 -- 172.21.15.97:0/2148349299 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5dd0100e60 msgr2=0x7f5dd006b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:34.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.623+0000 7f5dd769e640 1 --2- 172.21.15.97:0/2148349299 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5dd0100e60 0x7f5dd006b140 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc00d770 tx=0x7f5dcc00dc30 comp rx=0 tx=0).stop 2024-04-23T10:25:34.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.623+0000 7f5dd769e640 1 -- 172.21.15.97:0/2148349299 shutdown_connections 2024-04-23T10:25:34.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.623+0000 7f5dd769e640 1 --2- 172.21.15.97:0/2148349299 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f5dbc0a3bb0 0x7f5dbc0a6070 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:34.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.623+0000 7f5dd769e640 1 --2- 172.21.15.97:0/2148349299 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5dd0102a00 0x7f5dd019f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:34.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.623+0000 7f5dd769e640 1 --2- 172.21.15.97:0/2148349299 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5dd0102040 0x7f5dd019ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:34.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.623+0000 7f5dd769e640 1 --2- 172.21.15.97:0/2148349299 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5dd0100e60 0x7f5dd006b140 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:34.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.623+0000 7f5dd769e640 1 -- 172.21.15.97:0/2148349299 >> 172.21.15.97:0/2148349299 conn(0x7f5dd00fc440 msgr2=0x7f5dd01053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:34.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.623+0000 7f5dd769e640 1 -- 172.21.15.97:0/2148349299 shutdown_connections 2024-04-23T10:25:34.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.623+0000 7f5dd769e640 1 -- 172.21.15.97:0/2148349299 wait complete. 2024-04-23T10:25:34.633 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T10:25:34.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.733+0000 7f4c4828a640 1 -- 172.21.15.97:0/3609955294 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c40105340 msgr2=0x7f4c40107730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:34.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.733+0000 7f4c4828a640 1 --2- 172.21.15.97:0/3609955294 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c40105340 0x7f4c40107730 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f4c30009900 tx=0x7f4c3002a410 comp rx=0 tx=0).stop 2024-04-23T10:25:34.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.733+0000 7f4c4828a640 1 -- 172.21.15.97:0/3609955294 shutdown_connections 2024-04-23T10:25:34.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.733+0000 7f4c4828a640 1 --2- 172.21.15.97:0/3609955294 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c40107c70 0x7f4c4010a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:34.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.733+0000 7f4c4828a640 1 --2- 172.21.15.97:0/3609955294 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c40105340 0x7f4c40107730 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:34.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.733+0000 7f4c4828a640 1 --2- 172.21.15.97:0/3609955294 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c40102a10 0x7f4c40104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:34.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.734+0000 7f4c4828a640 1 -- 172.21.15.97:0/3609955294 >> 172.21.15.97:0/3609955294 conn(0x7f4c400fc400 msgr2=0x7f4c400fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:34.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.734+0000 7f4c4828a640 1 -- 172.21.15.97:0/3609955294 shutdown_connections 2024-04-23T10:25:34.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.734+0000 7f4c4828a640 1 -- 172.21.15.97:0/3609955294 wait complete. 2024-04-23T10:25:34.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.734+0000 7f4c4828a640 1 Processor -- start 2024-04-23T10:25:34.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.734+0000 7f4c4828a640 1 -- start start 2024-04-23T10:25:34.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.734+0000 7f4c4828a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c40102a10 0x7f4c40198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:34.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.734+0000 7f4c4828a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c40105340 0x7f4c40198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:34.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.734+0000 7f4c4828a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c40107c70 0x7f4c4019f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:34.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.734+0000 7f4c4828a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4c4010dab0 con 0x7f4c40107c70 2024-04-23T10:25:34.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.734+0000 7f4c4828a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4c4010d930 con 0x7f4c40102a10 2024-04-23T10:25:34.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.734+0000 7f4c4828a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4c4010dc30 con 0x7f4c40105340 2024-04-23T10:25:34.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.734+0000 7f4c46827640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c40107c70 0x7f4c4019f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:34.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.734+0000 7f4c46827640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c40107c70 0x7f4c4019f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33946/0 (socket says 172.21.15.97:33946) 2024-04-23T10:25:34.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.734+0000 7f4c46827640 1 -- 172.21.15.97:0/1644371043 learned_addr learned my addr 172.21.15.97:0/1644371043 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:25:34.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.734+0000 7f4c46026640 1 --2- 172.21.15.97:0/1644371043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c40102a10 0x7f4c40198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:34.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.735+0000 7f4c45825640 1 --2- 172.21.15.97:0/1644371043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c40105340 0x7f4c40198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:34.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.735+0000 7f4c46827640 1 -- 172.21.15.97:0/1644371043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c40105340 msgr2=0x7f4c40198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:34.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.735+0000 7f4c46827640 1 --2- 172.21.15.97:0/1644371043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c40105340 0x7f4c40198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:34.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.735+0000 7f4c46827640 1 -- 172.21.15.97:0/1644371043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c40102a10 msgr2=0x7f4c40198220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:34.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.735+0000 7f4c46827640 1 --2- 172.21.15.97:0/1644371043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c40102a10 0x7f4c40198220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:34.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.735+0000 7f4c46827640 1 -- 172.21.15.97:0/1644371043 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c4019fe40 con 0x7f4c40107c70 2024-04-23T10:25:34.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.735+0000 7f4c46827640 1 --2- 172.21.15.97:0/1644371043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c40107c70 0x7f4c4019f7d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f4c3800cad0 tx=0x7f4c3800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:34.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.735+0000 7f4c377fe640 1 -- 172.21.15.97:0/1644371043 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c3800ae20 con 0x7f4c40107c70 2024-04-23T10:25:34.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.735+0000 7f4c377fe640 1 -- 172.21.15.97:0/1644371043 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c38005020 con 0x7f4c40107c70 2024-04-23T10:25:34.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.735+0000 7f4c4828a640 1 -- 172.21.15.97:0/1644371043 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c401a0130 con 0x7f4c40107c70 2024-04-23T10:25:34.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.735+0000 7f4c377fe640 1 -- 172.21.15.97:0/1644371043 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c3800f6b0 con 0x7f4c40107c70 2024-04-23T10:25:34.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.735+0000 7f4c4828a640 1 -- 172.21.15.97:0/1644371043 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c401a0650 con 0x7f4c40107c70 2024-04-23T10:25:34.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.736+0000 7f4c4828a640 1 -- 172.21.15.97:0/1644371043 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c400fdef0 con 0x7f4c40107c70 2024-04-23T10:25:34.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.737+0000 7f4c377fe640 1 -- 172.21.15.97:0/1644371043 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4c380051d0 con 0x7f4c40107c70 2024-04-23T10:25:34.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.737+0000 7f4c377fe640 1 --2- 172.21.15.97:0/1644371043 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f4c2c0a3bb0 0x7f4c2c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:34.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.738+0000 7f4c46026640 1 --2- 172.21.15.97:0/1644371043 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f4c2c0a3bb0 0x7f4c2c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:34.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.738+0000 7f4c377fe640 1 -- 172.21.15.97:0/1644371043 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f4c380d03a0 con 0x7f4c40107c70 2024-04-23T10:25:34.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.738+0000 7f4c46026640 1 --2- 172.21.15.97:0/1644371043 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f4c2c0a3bb0 0x7f4c2c0a6070 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c005d00 tx=0x7f4c3c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:34.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.742+0000 7f4c377fe640 1 -- 172.21.15.97:0/1644371043 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4c38014030 con 0x7f4c40107c70 2024-04-23T10:25:34.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.923+0000 7f4c4828a640 1 -- 172.21.15.97:0/1644371043 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4c40062b50 con 0x7f4c40107c70 2024-04-23T10:25:34.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.924+0000 7f4c377fe640 1 -- 172.21.15.97:0/1644371043 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22606 (secure 0 0 0) 0x7f4c3809cdb0 con 0x7f4c40107c70 2024-04-23T10:25:34.924 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:25:34.924 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":88,"fsid":"9b22af88-0159-11ef-bc93-c7b262605968","created":"2024-04-23T10:16:05.983305+0000","modified":"2024-04-23T10:25:00.660133+0000","last_up_change":"2024-04-23T10:23:13.114761+0000","last_in_change":"2024-04-23T10:22:52.804880+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T10:19:25.559138+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T10:24:04.570049+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T10:24:05.861631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"58cb995d-de35-4522-ac81-4982401b260d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6803","nonce":3031221802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6805","nonce":3031221802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6809","nonce":3031221802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6807","nonce":3031221802}]},"public_addr":"172.21.15.97:6803/3031221802","cluster_addr":"172.21.15.97:6805/3031221802","heartbeat_back_addr":"172.21.15.97:6809/3031221802","heartbeat_front_addr":"172.21.15.97:6807/3031221802","state":["exists","up"]},{"osd":1,"uuid":"b47bd8ee-1f21-4834-acf5-bb40f97ab3ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6801","nonce":1108870440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6803","nonce":1108870440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6807","nonce":1108870440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6805","nonce":1108870440}]},"public_addr":"172.21.15.183:6801/1108870440","cluster_addr":"172.21.15.183:6803/1108870440","heartbeat_back_addr":"172.21.15.183:6807/1108870440","heartbeat_front_addr":"172.21.15.183:6805/1108870440","state":["exists","up"]},{"osd":2,"uuid":"17c4eec3-6c47-4334-9b63-1d5ad9ec5988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6801","nonce":2877039731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6803","nonce":2877039731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6807","nonce":2877039731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6805","nonce":2877039731}]},"public_addr":"172.21.15.196:6801/2877039731","cluster_addr":"172.21.15.196:6803/2877039731","heartbeat_back_addr":"172.21.15.196:6807/2877039731","heartbeat_front_addr":"172.21.15.196:6805/2877039731","state":["exists","up"]},{"osd":3,"uuid":"3f731c38-2e45-4ca1-8dce-efc4a1b2166b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6811","nonce":4276311194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6813","nonce":4276311194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6817","nonce":4276311194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6815","nonce":4276311194}]},"public_addr":"172.21.15.97:6811/4276311194","cluster_addr":"172.21.15.97:6813/4276311194","heartbeat_back_addr":"172.21.15.97:6817/4276311194","heartbeat_front_addr":"172.21.15.97:6815/4276311194","state":["exists","up"]},{"osd":4,"uuid":"fdcafb14-9b82-4134-bd5b-79c0516f6cbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6809","nonce":63913916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6811","nonce":63913916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6815","nonce":63913916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6813","nonce":63913916}]},"public_addr":"172.21.15.183:6809/63913916","cluster_addr":"172.21.15.183:6811/63913916","heartbeat_back_addr":"172.21.15.183:6815/63913916","heartbeat_front_addr":"172.21.15.183:6813/63913916","state":["exists","up"]},{"osd":5,"uuid":"cdcc18d8-da3c-4012-bf12-876bb39e66eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6809","nonce":2804446896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6811","nonce":2804446896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6815","nonce":2804446896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6813","nonce":2804446896}]},"public_addr":"172.21.15.196:6809/2804446896","cluster_addr":"172.21.15.196:6811/2804446896","heartbeat_back_addr":"172.21.15.196:6815/2804446896","heartbeat_front_addr":"172.21.15.196:6813/2804446896","state":["exists","up"]},{"osd":6,"uuid":"071abb5c-2edf-4832-b8aa-3b80d20096fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6819","nonce":2080916705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6821","nonce":2080916705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6825","nonce":2080916705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6823","nonce":2080916705}]},"public_addr":"172.21.15.97:6819/2080916705","cluster_addr":"172.21.15.97:6821/2080916705","heartbeat_back_addr":"172.21.15.97:6825/2080916705","heartbeat_front_addr":"172.21.15.97:6823/2080916705","state":["exists","up"]},{"osd":7,"uuid":"9d54760b-66df-486b-8abb-017f3b64b4d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6817","nonce":1865873033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6819","nonce":1865873033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6823","nonce":1865873033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6821","nonce":1865873033}]},"public_addr":"172.21.15.183:6817/1865873033","cluster_addr":"172.21.15.183:6819/1865873033","heartbeat_back_addr":"172.21.15.183:6823/1865873033","heartbeat_front_addr":"172.21.15.183:6821/1865873033","state":["exists","up"]},{"osd":8,"uuid":"5cd3100a-bd6a-4be5-b5e9-0d0afc325193","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6817","nonce":4063932139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6819","nonce":4063932139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6823","nonce":4063932139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6821","nonce":4063932139}]},"public_addr":"172.21.15.196:6817/4063932139","cluster_addr":"172.21.15.196:6819/4063932139","heartbeat_back_addr":"172.21.15.196:6823/4063932139","heartbeat_front_addr":"172.21.15.196:6821/4063932139","state":["exists","up"]},{"osd":9,"uuid":"b6a5f4a1-4ecc-4f73-a2df-07551d9298ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6827","nonce":1887432474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6829","nonce":1887432474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6833","nonce":1887432474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6831","nonce":1887432474}]},"public_addr":"172.21.15.97:6827/1887432474","cluster_addr":"172.21.15.97:6829/1887432474","heartbeat_back_addr":"172.21.15.97:6833/1887432474","heartbeat_front_addr":"172.21.15.97:6831/1887432474","state":["exists","up"]},{"osd":10,"uuid":"86f59816-1ea6-4ae9-8796-cf5fc4831064","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6825","nonce":799434647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6827","nonce":799434647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6831","nonce":799434647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6829","nonce":799434647}]},"public_addr":"172.21.15.183:6825/799434647","cluster_addr":"172.21.15.183:6827/799434647","heartbeat_back_addr":"172.21.15.183:6831/799434647","heartbeat_front_addr":"172.21.15.183:6829/799434647","state":["exists","up"]},{"osd":11,"uuid":"95d64a89-a9f8-4c7a-8c0d-da7fb26cfff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6825","nonce":1873969467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6827","nonce":1873969467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6831","nonce":1873969467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6829","nonce":1873969467}]},"public_addr":"172.21.15.196:6825/1873969467","cluster_addr":"172.21.15.196:6827/1873969467","heartbeat_back_addr":"172.21.15.196:6831/1873969467","heartbeat_front_addr":"172.21.15.196:6829/1873969467","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:18:33.721156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:18:57.586709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:19:22.069458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:19:46.451597+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:20:11.561019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:20:36.298672+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:21:01.661804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:21:27.324128+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:21:52.898197+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:22:19.614326+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:22:45.154917+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:23:11.222612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/3866016040":"2024-04-24T10:16:59.468237+0000","172.21.15.97:6801/3297166858":"2024-04-24T10:16:59.468237+0000","172.21.15.97:0/3804743548":"2024-04-24T10:16:59.468237+0000","172.21.15.97:6800/3706463017":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/3715079438":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/786806834":"2024-04-24T10:16:59.468237+0000","172.21.15.97:6801/3706463017":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/3294175554":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/1449136760":"2024-04-24T10:16:33.128917+0000","172.21.15.97:6800/3297166858":"2024-04-24T10:16:59.468237+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T10:25:34.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.927+0000 7f4c4828a640 1 -- 172.21.15.97:0/1644371043 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f4c2c0a3bb0 msgr2=0x7f4c2c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:34.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.927+0000 7f4c4828a640 1 --2- 172.21.15.97:0/1644371043 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f4c2c0a3bb0 0x7f4c2c0a6070 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c005d00 tx=0x7f4c3c00b410 comp rx=0 tx=0).stop 2024-04-23T10:25:34.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.927+0000 7f4c4828a640 1 -- 172.21.15.97:0/1644371043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c40107c70 msgr2=0x7f4c4019f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:34.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.927+0000 7f4c4828a640 1 --2- 172.21.15.97:0/1644371043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c40107c70 0x7f4c4019f7d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f4c3800cad0 tx=0x7f4c3800cf90 comp rx=0 tx=0).stop 2024-04-23T10:25:34.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.927+0000 7f4c4828a640 1 -- 172.21.15.97:0/1644371043 shutdown_connections 2024-04-23T10:25:34.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.927+0000 7f4c4828a640 1 --2- 172.21.15.97:0/1644371043 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f4c2c0a3bb0 0x7f4c2c0a6070 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:34.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.927+0000 7f4c4828a640 1 --2- 172.21.15.97:0/1644371043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c40107c70 0x7f4c4019f7d0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:34.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.927+0000 7f4c4828a640 1 --2- 172.21.15.97:0/1644371043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c40105340 0x7f4c40198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:34.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.927+0000 7f4c4828a640 1 --2- 172.21.15.97:0/1644371043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c40102a10 0x7f4c40198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:34.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.927+0000 7f4c4828a640 1 -- 172.21.15.97:0/1644371043 >> 172.21.15.97:0/1644371043 conn(0x7f4c400fc400 msgr2=0x7f4c40103350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:34.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.927+0000 7f4c4828a640 1 -- 172.21.15.97:0/1644371043 shutdown_connections 2024-04-23T10:25:34.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:34.927+0000 7f4c4828a640 1 -- 172.21.15.97:0/1644371043 wait complete. 2024-04-23T10:25:34.936 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:34 smithi097 ceph-mon[29197]: pgmap v335: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 9 op/s 2024-04-23T10:25:34.936 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:34 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/2125157266' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T10:25:34.936 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:34 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/4251332212' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T10:25:34.936 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:34 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/2148349299' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:25:34.937 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T10:19:25.559138+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T10:24:04.570049+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T10:24:05.861631+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T10:25:34.938 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T10:25:35.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.042+0000 7f4fd7e59640 1 -- 172.21.15.97:0/1100606897 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4fd0101f90 msgr2=0x7f4fd01023b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:35.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.042+0000 7f4fd7e59640 1 --2- 172.21.15.97:0/1100606897 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4fd0101f90 0x7f4fd01023b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc009900 tx=0x7f4fcc02a410 comp rx=0 tx=0).stop 2024-04-23T10:25:35.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.042+0000 7f4fd7e59640 1 -- 172.21.15.97:0/1100606897 shutdown_connections 2024-04-23T10:25:35.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.042+0000 7f4fd7e59640 1 --2- 172.21.15.97:0/1100606897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fd0103b40 0x7f4fd010a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:35.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.042+0000 7f4fd7e59640 1 --2- 172.21.15.97:0/1100606897 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fd0103180 0x7f4fd0103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:35.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.042+0000 7f4fd7e59640 1 --2- 172.21.15.97:0/1100606897 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4fd0101f90 0x7f4fd01023b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:35.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.042+0000 7f4fd7e59640 1 -- 172.21.15.97:0/1100606897 >> 172.21.15.97:0/1100606897 conn(0x7f4fd00fd500 msgr2=0x7f4fd00ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:35.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.042+0000 7f4fd7e59640 1 -- 172.21.15.97:0/1100606897 shutdown_connections 2024-04-23T10:25:35.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.042+0000 7f4fd7e59640 1 -- 172.21.15.97:0/1100606897 wait complete. 2024-04-23T10:25:35.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.043+0000 7f4fd7e59640 1 Processor -- start 2024-04-23T10:25:35.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.043+0000 7f4fd7e59640 1 -- start start 2024-04-23T10:25:35.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.043+0000 7f4fd7e59640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fd0101f90 0x7f4fd0075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:35.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.043+0000 7f4fd7e59640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4fd0103180 0x7f4fd0074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:35.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.043+0000 7f4fd7e59640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fd0103b40 0x7f4fd0074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:35.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.043+0000 7f4fd7e59640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd010dad0 con 0x7f4fd0103b40 2024-04-23T10:25:35.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.043+0000 7f4fd7e59640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd010d950 con 0x7f4fd0103180 2024-04-23T10:25:35.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.043+0000 7f4fd7e59640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd010dc50 con 0x7f4fd0101f90 2024-04-23T10:25:35.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.043+0000 7f4fd5bf5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fd0101f90 0x7f4fd0075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:35.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.043+0000 7f4fd63f6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fd0103b40 0x7f4fd0074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:35.044 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.043+0000 7f4fd5bf5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fd0101f90 0x7f4fd0075d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.97:58100/0 (socket says 172.21.15.97:58100) 2024-04-23T10:25:35.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.043+0000 7f4fd63f6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fd0103b40 0x7f4fd0074880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33956/0 (socket says 172.21.15.97:33956) 2024-04-23T10:25:35.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.043+0000 7f4fd5bf5640 1 -- 172.21.15.97:0/3187142372 learned_addr learned my addr 172.21.15.97:0/3187142372 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:25:35.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.044+0000 7f4fd53f4640 1 --2- 172.21.15.97:0/3187142372 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4fd0103180 0x7f4fd0074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:35.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.044+0000 7f4fd5bf5640 1 -- 172.21.15.97:0/3187142372 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4fd0103180 msgr2=0x7f4fd0074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:35.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.044+0000 7f4fd5bf5640 1 --2- 172.21.15.97:0/3187142372 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4fd0103180 0x7f4fd0074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:35.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.044+0000 7f4fd5bf5640 1 -- 172.21.15.97:0/3187142372 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fd0103b40 msgr2=0x7f4fd0074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:35.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.044+0000 7f4fd5bf5640 1 --2- 172.21.15.97:0/3187142372 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fd0103b40 0x7f4fd0074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:35.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.044+0000 7f4fd5bf5640 1 -- 172.21.15.97:0/3187142372 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fd0075110 con 0x7f4fd0101f90 2024-04-23T10:25:35.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.044+0000 7f4fd63f6640 1 --2- 172.21.15.97:0/3187142372 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fd0103b40 0x7f4fd0074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T10:25:35.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.044+0000 7f4fd5bf5640 1 --2- 172.21.15.97:0/3187142372 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fd0101f90 0x7f4fd0075d00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc006010 tx=0x7f4fcc0042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:35.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.044+0000 7f4fc6ffd640 1 -- 172.21.15.97:0/3187142372 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc03f030 con 0x7f4fd0101f90 2024-04-23T10:25:35.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.044+0000 7f4fd7e59640 1 -- 172.21.15.97:0/3187142372 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fd01a2300 con 0x7f4fd0101f90 2024-04-23T10:25:35.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.044+0000 7f4fc6ffd640 1 -- 172.21.15.97:0/3187142372 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fcc031910 con 0x7f4fd0101f90 2024-04-23T10:25:35.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.044+0000 7f4fc6ffd640 1 -- 172.21.15.97:0/3187142372 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc035070 con 0x7f4fd0101f90 2024-04-23T10:25:35.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.045+0000 7f4fd7e59640 1 -- 172.21.15.97:0/3187142372 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fd01a28a0 con 0x7f4fd0101f90 2024-04-23T10:25:35.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.045+0000 7f4fd7e59640 1 -- 172.21.15.97:0/3187142372 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fd01093c0 con 0x7f4fd0101f90 2024-04-23T10:25:35.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.047+0000 7f4fc6ffd640 1 -- 172.21.15.97:0/3187142372 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4fcc042030 con 0x7f4fd0101f90 2024-04-23T10:25:35.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.048+0000 7f4fc6ffd640 1 --2- 172.21.15.97:0/3187142372 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f4fbc0a3bb0 0x7f4fbc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:35.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.048+0000 7f4fd53f4640 1 --2- 172.21.15.97:0/3187142372 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f4fbc0a3bb0 0x7f4fbc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:35.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.048+0000 7f4fc6ffd640 1 -- 172.21.15.97:0/3187142372 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f4fcc0f0b70 con 0x7f4fd0101f90 2024-04-23T10:25:35.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.048+0000 7f4fd53f4640 1 --2- 172.21.15.97:0/3187142372 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f4fbc0a3bb0 0x7f4fbc0a6070 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0075a70 tx=0x7f4fc0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:35.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.052+0000 7f4fc6ffd640 1 -- 172.21.15.97:0/3187142372 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4fcc0bd580 con 0x7f4fd0101f90 2024-04-23T10:25:35.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:34 smithi196 ceph-mon[32919]: pgmap v335: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 9 op/s 2024-04-23T10:25:35.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:34 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/2125157266' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T10:25:35.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:34 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/4251332212' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T10:25:35.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:34 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/2148349299' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:25:35.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:34 smithi183 ceph-mon[32317]: pgmap v335: 97 pgs: 97 active+clean; 644 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 9 op/s 2024-04-23T10:25:35.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:34 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/2125157266' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T10:25:35.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:34 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/4251332212' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T10:25:35.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:34 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/2148349299' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:25:35.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.231+0000 7f4fd7e59640 1 -- 172.21.15.97:0/3187142372 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4fd002d810 con 0x7f4fd0101f90 2024-04-23T10:25:35.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.231+0000 7f4fc6ffd640 1 -- 172.21.15.97:0/3187142372 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f4fcc0bce80 con 0x7f4fd0101f90 2024-04-23T10:25:35.230 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-23T10:25:35.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.233+0000 7f4fd7e59640 1 -- 172.21.15.97:0/3187142372 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f4fbc0a3bb0 msgr2=0x7f4fbc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:35.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.233+0000 7f4fd7e59640 1 --2- 172.21.15.97:0/3187142372 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f4fbc0a3bb0 0x7f4fbc0a6070 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0075a70 tx=0x7f4fc0005f90 comp rx=0 tx=0).stop 2024-04-23T10:25:35.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.233+0000 7f4fd7e59640 1 -- 172.21.15.97:0/3187142372 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fd0101f90 msgr2=0x7f4fd0075d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:35.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.233+0000 7f4fd7e59640 1 --2- 172.21.15.97:0/3187142372 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fd0101f90 0x7f4fd0075d00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc006010 tx=0x7f4fcc0042f0 comp rx=0 tx=0).stop 2024-04-23T10:25:35.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.233+0000 7f4fd7e59640 1 -- 172.21.15.97:0/3187142372 shutdown_connections 2024-04-23T10:25:35.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.233+0000 7f4fd7e59640 1 --2- 172.21.15.97:0/3187142372 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f4fbc0a3bb0 0x7f4fbc0a6070 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:35.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.233+0000 7f4fd7e59640 1 --2- 172.21.15.97:0/3187142372 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4fd0103b40 0x7f4fd0074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:35.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.233+0000 7f4fd7e59640 1 --2- 172.21.15.97:0/3187142372 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4fd0103180 0x7f4fd0074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:35.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.233+0000 7f4fd7e59640 1 --2- 172.21.15.97:0/3187142372 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fd0101f90 0x7f4fd0075d00 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:35.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.233+0000 7f4fd7e59640 1 -- 172.21.15.97:0/3187142372 >> 172.21.15.97:0/3187142372 conn(0x7f4fd00fd500 msgr2=0x7f4fd0106590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:35.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.233+0000 7f4fd7e59640 1 -- 172.21.15.97:0/3187142372 shutdown_connections 2024-04-23T10:25:35.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.233+0000 7f4fd7e59640 1 -- 172.21.15.97:0/3187142372 wait complete. 2024-04-23T10:25:35.244 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T10:25:35.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.347+0000 7f7d0b249640 1 -- 172.21.15.97:0/1237156637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d041031a0 msgr2=0x7f7d04103620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:35.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.347+0000 7f7d0b249640 1 --2- 172.21.15.97:0/1237156637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d041031a0 0x7f7d04103620 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4009900 tx=0x7f7cf402a410 comp rx=0 tx=0).stop 2024-04-23T10:25:35.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.347+0000 7f7d0b249640 1 -- 172.21.15.97:0/1237156637 shutdown_connections 2024-04-23T10:25:35.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.347+0000 7f7d0b249640 1 --2- 172.21.15.97:0/1237156637 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d04103b60 0x7f7d0410a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:35.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.347+0000 7f7d0b249640 1 --2- 172.21.15.97:0/1237156637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d041031a0 0x7f7d04103620 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:35.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.347+0000 7f7d0b249640 1 --2- 172.21.15.97:0/1237156637 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d04101fb0 0x7f7d041023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:35.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.347+0000 7f7d0b249640 1 -- 172.21.15.97:0/1237156637 >> 172.21.15.97:0/1237156637 conn(0x7f7d040fd540 msgr2=0x7f7d040ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:35.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.347+0000 7f7d0b249640 1 -- 172.21.15.97:0/1237156637 shutdown_connections 2024-04-23T10:25:35.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.347+0000 7f7d0b249640 1 -- 172.21.15.97:0/1237156637 wait complete. 2024-04-23T10:25:35.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.348+0000 7f7d0b249640 1 Processor -- start 2024-04-23T10:25:35.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.348+0000 7f7d0b249640 1 -- start start 2024-04-23T10:25:35.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.348+0000 7f7d0b249640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d04101fb0 0x7f7d04198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:35.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.348+0000 7f7d0b249640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d041031a0 0x7f7d041986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:35.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.348+0000 7f7d0b249640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d04103b60 0x7f7d0419f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:35.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.348+0000 7f7d0b249640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7d0410daf0 con 0x7f7d041031a0 2024-04-23T10:25:35.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.348+0000 7f7d0b249640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7d0410d970 con 0x7f7d04101fb0 2024-04-23T10:25:35.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.348+0000 7f7d0b249640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7d0410dc70 con 0x7f7d04103b60 2024-04-23T10:25:35.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.348+0000 7f7d03fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d041031a0 0x7f7d041986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:35.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.348+0000 7f7d03fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d041031a0 0x7f7d041986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33966/0 (socket says 172.21.15.97:33966) 2024-04-23T10:25:35.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.348+0000 7f7d03fff640 1 -- 172.21.15.97:0/1029074349 learned_addr learned my addr 172.21.15.97:0/1029074349 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:25:35.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.348+0000 7f7d08fe5640 1 --2- 172.21.15.97:0/1029074349 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d04101fb0 0x7f7d04198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:35.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.348+0000 7f7d097e6640 1 --2- 172.21.15.97:0/1029074349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d04103b60 0x7f7d0419f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:35.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.348+0000 7f7d03fff640 1 -- 172.21.15.97:0/1029074349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d04103b60 msgr2=0x7f7d0419f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:35.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.348+0000 7f7d03fff640 1 --2- 172.21.15.97:0/1029074349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d04103b60 0x7f7d0419f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:35.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.348+0000 7f7d03fff640 1 -- 172.21.15.97:0/1029074349 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d04101fb0 msgr2=0x7f7d04198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:35.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.348+0000 7f7d03fff640 1 --2- 172.21.15.97:0/1029074349 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d04101fb0 0x7f7d04198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:35.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.348+0000 7f7d03fff640 1 -- 172.21.15.97:0/1029074349 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d0419fdb0 con 0x7f7d041031a0 2024-04-23T10:25:35.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.348+0000 7f7d03fff640 1 --2- 172.21.15.97:0/1029074349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d041031a0 0x7f7d041986d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4005940 tx=0x7f7cf4004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:35.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.348+0000 7f7d01ffb640 1 -- 172.21.15.97:0/1029074349 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf403f030 con 0x7f7d041031a0 2024-04-23T10:25:35.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.348+0000 7f7d01ffb640 1 -- 172.21.15.97:0/1029074349 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cf4003cb0 con 0x7f7d041031a0 2024-04-23T10:25:35.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.348+0000 7f7d0b249640 1 -- 172.21.15.97:0/1029074349 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d041a0040 con 0x7f7d041031a0 2024-04-23T10:25:35.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.348+0000 7f7d01ffb640 1 -- 172.21.15.97:0/1029074349 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf4035070 con 0x7f7d041031a0 2024-04-23T10:25:35.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.349+0000 7f7d0b249640 1 -- 172.21.15.97:0/1029074349 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d041a0530 con 0x7f7d041031a0 2024-04-23T10:25:35.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.350+0000 7f7d0b249640 1 -- 172.21.15.97:0/1029074349 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d041092d0 con 0x7f7d041031a0 2024-04-23T10:25:35.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.351+0000 7f7d01ffb640 1 -- 172.21.15.97:0/1029074349 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7cf4042030 con 0x7f7d041031a0 2024-04-23T10:25:35.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.351+0000 7f7d01ffb640 1 --2- 172.21.15.97:0/1029074349 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f7cf00a3bb0 0x7f7cf00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:35.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.351+0000 7f7d08fe5640 1 --2- 172.21.15.97:0/1029074349 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f7cf00a3bb0 0x7f7cf00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:35.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.351+0000 7f7d01ffb640 1 -- 172.21.15.97:0/1029074349 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f7cf40f0fa0 con 0x7f7d041031a0 2024-04-23T10:25:35.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.352+0000 7f7d08fe5640 1 --2- 172.21.15.97:0/1029074349 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f7cf00a3bb0 0x7f7cf00a6070 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7d04103000 tx=0x7f7cfc009720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:35.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.355+0000 7f7d01ffb640 1 -- 172.21.15.97:0/1029074349 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7cf40bd9b0 con 0x7f7d041031a0 2024-04-23T10:25:35.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.534+0000 7f7d0b249640 1 -- 172.21.15.97:0/1029074349 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7d0402d810 con 0x7f7d041031a0 2024-04-23T10:25:35.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.534+0000 7f7d01ffb640 1 -- 172.21.15.97:0/1029074349 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f7cf40b7e20 con 0x7f7d041031a0 2024-04-23T10:25:35.534 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-23T10:25:35.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.536+0000 7f7d0b249640 1 -- 172.21.15.97:0/1029074349 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f7cf00a3bb0 msgr2=0x7f7cf00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:35.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.536+0000 7f7d0b249640 1 --2- 172.21.15.97:0/1029074349 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f7cf00a3bb0 0x7f7cf00a6070 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7d04103000 tx=0x7f7cfc009720 comp rx=0 tx=0).stop 2024-04-23T10:25:35.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.536+0000 7f7d0b249640 1 -- 172.21.15.97:0/1029074349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d041031a0 msgr2=0x7f7d041986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:35.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.536+0000 7f7d0b249640 1 --2- 172.21.15.97:0/1029074349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d041031a0 0x7f7d041986d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4005940 tx=0x7f7cf4004430 comp rx=0 tx=0).stop 2024-04-23T10:25:35.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.536+0000 7f7d0b249640 1 -- 172.21.15.97:0/1029074349 shutdown_connections 2024-04-23T10:25:35.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.536+0000 7f7d0b249640 1 --2- 172.21.15.97:0/1029074349 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f7cf00a3bb0 0x7f7cf00a6070 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:35.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.536+0000 7f7d0b249640 1 --2- 172.21.15.97:0/1029074349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d04103b60 0x7f7d0419f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:35.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.536+0000 7f7d0b249640 1 --2- 172.21.15.97:0/1029074349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d041031a0 0x7f7d041986d0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:35.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.536+0000 7f7d0b249640 1 --2- 172.21.15.97:0/1029074349 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d04101fb0 0x7f7d04198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:35.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.536+0000 7f7d0b249640 1 -- 172.21.15.97:0/1029074349 >> 172.21.15.97:0/1029074349 conn(0x7f7d040fd540 msgr2=0x7f7d041064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:35.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.537+0000 7f7d0b249640 1 -- 172.21.15.97:0/1029074349 shutdown_connections 2024-04-23T10:25:35.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.537+0000 7f7d0b249640 1 -- 172.21.15.97:0/1029074349 wait complete. 2024-04-23T10:25:35.547 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T10:25:35.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.649+0000 7f4df867b640 1 -- 172.21.15.97:0/2523148057 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4df010e570 msgr2=0x7f4df0110960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:35.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.649+0000 7f4df867b640 1 --2- 172.21.15.97:0/2523148057 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4df010e570 0x7f4df0110960 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4de8009cd0 tx=0x7f4de802a8e0 comp rx=0 tx=0).stop 2024-04-23T10:25:35.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.649+0000 7f4df867b640 1 -- 172.21.15.97:0/2523148057 shutdown_connections 2024-04-23T10:25:35.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.649+0000 7f4df867b640 1 --2- 172.21.15.97:0/2523148057 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4df010e570 0x7f4df0110960 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:35.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.649+0000 7f4df867b640 1 --2- 172.21.15.97:0/2523148057 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4df00749b0 0x7f4df0074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:35.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.649+0000 7f4df867b640 1 --2- 172.21.15.97:0/2523148057 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4df0075f90 0x7f4df00743e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:35.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.649+0000 7f4df867b640 1 -- 172.21.15.97:0/2523148057 >> 172.21.15.97:0/2523148057 conn(0x7f4df00fd530 msgr2=0x7f4df00ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:35.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.649+0000 7f4df867b640 1 -- 172.21.15.97:0/2523148057 shutdown_connections 2024-04-23T10:25:35.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.649+0000 7f4df867b640 1 -- 172.21.15.97:0/2523148057 wait complete. 2024-04-23T10:25:35.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.650+0000 7f4df867b640 1 Processor -- start 2024-04-23T10:25:35.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.650+0000 7f4df867b640 1 -- start start 2024-04-23T10:25:35.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.650+0000 7f4df867b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4df00749b0 0x7f4df019e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:35.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.650+0000 7f4df867b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4df0075f90 0x7f4df019ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:35.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.650+0000 7f4df867b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4df010e570 0x7f4df01a5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:35.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.650+0000 7f4df867b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4df0114140 con 0x7f4df010e570 2024-04-23T10:25:35.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.650+0000 7f4df867b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4df0113fc0 con 0x7f4df00749b0 2024-04-23T10:25:35.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.650+0000 7f4df867b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4df01142c0 con 0x7f4df0075f90 2024-04-23T10:25:35.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.650+0000 7f4df6c18640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4df010e570 0x7f4df01a5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:35.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.650+0000 7f4df6c18640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4df010e570 0x7f4df01a5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59650/0 (socket says 172.21.15.97:59650) 2024-04-23T10:25:35.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.650+0000 7f4df6c18640 1 -- 172.21.15.97:0/2193923269 learned_addr learned my addr 172.21.15.97:0/2193923269 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:25:35.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.650+0000 7f4df6417640 1 --2- 172.21.15.97:0/2193923269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4df00749b0 0x7f4df019e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:35.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.650+0000 7f4df5c16640 1 --2- 172.21.15.97:0/2193923269 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4df0075f90 0x7f4df019ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:35.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.650+0000 7f4df6c18640 1 -- 172.21.15.97:0/2193923269 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4df0075f90 msgr2=0x7f4df019ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:35.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.650+0000 7f4df6c18640 1 --2- 172.21.15.97:0/2193923269 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4df0075f90 0x7f4df019ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:35.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.650+0000 7f4df6c18640 1 -- 172.21.15.97:0/2193923269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4df00749b0 msgr2=0x7f4df019e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:35.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.650+0000 7f4df6c18640 1 --2- 172.21.15.97:0/2193923269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4df00749b0 0x7f4df019e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:35.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.650+0000 7f4df6c18640 1 -- 172.21.15.97:0/2193923269 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4df01a6430 con 0x7f4df010e570 2024-04-23T10:25:35.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.650+0000 7f4df5c16640 1 --2- 172.21.15.97:0/2193923269 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4df0075f90 0x7f4df019ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:25:35.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.650+0000 7f4df6c18640 1 --2- 172.21.15.97:0/2193923269 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4df010e570 0x7f4df01a5dc0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f4de8004af0 tx=0x7f4de8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:35.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.650+0000 7f4de77fe640 1 -- 172.21.15.97:0/2193923269 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de8004250 con 0x7f4df010e570 2024-04-23T10:25:35.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.651+0000 7f4de77fe640 1 -- 172.21.15.97:0/2193923269 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4de80305a0 con 0x7f4df010e570 2024-04-23T10:25:35.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.651+0000 7f4df867b640 1 -- 172.21.15.97:0/2193923269 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4df01a66c0 con 0x7f4df010e570 2024-04-23T10:25:35.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.651+0000 7f4de77fe640 1 -- 172.21.15.97:0/2193923269 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de80466e0 con 0x7f4df010e570 2024-04-23T10:25:35.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.651+0000 7f4df867b640 1 -- 172.21.15.97:0/2193923269 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4df01a6c90 con 0x7f4df010e570 2024-04-23T10:25:35.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.652+0000 7f4df867b640 1 -- 172.21.15.97:0/2193923269 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4df0104630 con 0x7f4df010e570 2024-04-23T10:25:35.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.653+0000 7f4de77fe640 1 -- 172.21.15.97:0/2193923269 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4de8030750 con 0x7f4df010e570 2024-04-23T10:25:35.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.653+0000 7f4de77fe640 1 --2- 172.21.15.97:0/2193923269 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f4ddc0a3bb0 0x7f4ddc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:35.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.653+0000 7f4df6417640 1 --2- 172.21.15.97:0/2193923269 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f4ddc0a3bb0 0x7f4ddc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:35.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.653+0000 7f4de77fe640 1 -- 172.21.15.97:0/2193923269 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f4de802e030 con 0x7f4df010e570 2024-04-23T10:25:35.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.654+0000 7f4df6417640 1 --2- 172.21.15.97:0/2193923269 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f4ddc0a3bb0 0x7f4ddc0a6070 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4dec005fd0 tx=0x7f4dec00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:35.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.658+0000 7f4de77fe640 1 -- 172.21.15.97:0/2193923269 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4de804b060 con 0x7f4df010e570 2024-04-23T10:25:35.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.838+0000 7f4df867b640 1 -- 172.21.15.97:0/2193923269 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4df0062b50 con 0x7f4df010e570 2024-04-23T10:25:35.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.839+0000 7f4de77fe640 1 -- 172.21.15.97:0/2193923269 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f4de80be790 con 0x7f4df010e570 2024-04-23T10:25:35.838 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-04-23T10:25:35.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.841+0000 7f4df867b640 1 -- 172.21.15.97:0/2193923269 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f4ddc0a3bb0 msgr2=0x7f4ddc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:35.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.841+0000 7f4df867b640 1 --2- 172.21.15.97:0/2193923269 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f4ddc0a3bb0 0x7f4ddc0a6070 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4dec005fd0 tx=0x7f4dec00b390 comp rx=0 tx=0).stop 2024-04-23T10:25:35.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.841+0000 7f4df867b640 1 -- 172.21.15.97:0/2193923269 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4df010e570 msgr2=0x7f4df01a5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:35.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.841+0000 7f4df867b640 1 --2- 172.21.15.97:0/2193923269 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4df010e570 0x7f4df01a5dc0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f4de8004af0 tx=0x7f4de8003c10 comp rx=0 tx=0).stop 2024-04-23T10:25:35.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.841+0000 7f4df867b640 1 -- 172.21.15.97:0/2193923269 shutdown_connections 2024-04-23T10:25:35.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.841+0000 7f4df867b640 1 --2- 172.21.15.97:0/2193923269 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f4ddc0a3bb0 0x7f4ddc0a6070 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:35.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.841+0000 7f4df867b640 1 --2- 172.21.15.97:0/2193923269 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4df010e570 0x7f4df01a5dc0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:35.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.841+0000 7f4df867b640 1 --2- 172.21.15.97:0/2193923269 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4df0075f90 0x7f4df019ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:35.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.841+0000 7f4df867b640 1 --2- 172.21.15.97:0/2193923269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4df00749b0 0x7f4df019e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:35.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.841+0000 7f4df867b640 1 -- 172.21.15.97:0/2193923269 >> 172.21.15.97:0/2193923269 conn(0x7f4df00fd530 msgr2=0x7f4df0101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:35.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.841+0000 7f4df867b640 1 -- 172.21.15.97:0/2193923269 shutdown_connections 2024-04-23T10:25:35.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.841+0000 7f4df867b640 1 -- 172.21.15.97:0/2193923269 wait complete. 2024-04-23T10:25:35.850 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:35 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/1644371043' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T10:25:35.850 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:35 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/3187142372' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T10:25:35.850 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:35 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/1029074349' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T10:25:35.851 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T10:25:35.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.950+0000 7fb806eb4640 1 -- 172.21.15.97:0/1966469411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8001029b0 msgr2=0x7fb8001092d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:35.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.950+0000 7fb806eb4640 1 --2- 172.21.15.97:0/1966469411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8001029b0 0x7fb8001092d0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb7f8009ca0 tx=0x7fb7f802a8c0 comp rx=0 tx=0).stop 2024-04-23T10:25:35.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.950+0000 7fb806eb4640 1 -- 172.21.15.97:0/1966469411 shutdown_connections 2024-04-23T10:25:35.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.950+0000 7fb806eb4640 1 --2- 172.21.15.97:0/1966469411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8001029b0 0x7fb8001092d0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:35.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.950+0000 7fb806eb4640 1 --2- 172.21.15.97:0/1966469411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb800101ff0 0x7fb800102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:35.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.950+0000 7fb806eb4640 1 --2- 172.21.15.97:0/1966469411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb800100ea0 0x7fb8001012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:35.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.950+0000 7fb806eb4640 1 -- 172.21.15.97:0/1966469411 >> 172.21.15.97:0/1966469411 conn(0x7fb8000fc430 msgr2=0x7fb8000fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:35.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.950+0000 7fb806eb4640 1 -- 172.21.15.97:0/1966469411 shutdown_connections 2024-04-23T10:25:35.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.950+0000 7fb806eb4640 1 -- 172.21.15.97:0/1966469411 wait complete. 2024-04-23T10:25:35.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.950+0000 7fb806eb4640 1 Processor -- start 2024-04-23T10:25:35.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.951+0000 7fb806eb4640 1 -- start start 2024-04-23T10:25:35.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.951+0000 7fb806eb4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb800100ea0 0x7fb80006b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:35.951 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.951+0000 7fb806eb4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb800101ff0 0x7fb80019cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:35.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.951+0000 7fb806eb4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb8001029b0 0x7fb80019f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:35.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.951+0000 7fb806eb4640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb80010ca00 con 0x7fb800100ea0 2024-04-23T10:25:35.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.951+0000 7fb806eb4640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb80010c880 con 0x7fb8001029b0 2024-04-23T10:25:35.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.951+0000 7fb806eb4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb80010cb80 con 0x7fb800101ff0 2024-04-23T10:25:35.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.951+0000 7fb804c50640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb800100ea0 0x7fb80006b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:35.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.951+0000 7fb804c50640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb800100ea0 0x7fb80006b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59666/0 (socket says 172.21.15.97:59666) 2024-04-23T10:25:35.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.951+0000 7fb804c50640 1 -- 172.21.15.97:0/3355436790 learned_addr learned my addr 172.21.15.97:0/3355436790 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:25:35.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.951+0000 7fb805451640 1 --2- 172.21.15.97:0/3355436790 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb8001029b0 0x7fb80019f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:35.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.951+0000 7fb804c50640 1 -- 172.21.15.97:0/3355436790 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb800101ff0 msgr2=0x7fb80019cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:35.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.951+0000 7fb804c50640 1 --2- 172.21.15.97:0/3355436790 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb800101ff0 0x7fb80019cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:35.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.951+0000 7fb804c50640 1 -- 172.21.15.97:0/3355436790 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb8001029b0 msgr2=0x7fb80019f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:35.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.951+0000 7fb804c50640 1 --2- 172.21.15.97:0/3355436790 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb8001029b0 0x7fb80019f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:35.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.951+0000 7fb804c50640 1 -- 172.21.15.97:0/3355436790 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb80019fe30 con 0x7fb800100ea0 2024-04-23T10:25:35.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.951+0000 7fb805451640 1 --2- 172.21.15.97:0/3355436790 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb8001029b0 0x7fb80019f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:25:35.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.951+0000 7fb804c50640 1 --2- 172.21.15.97:0/3355436790 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb800100ea0 0x7fb80006b120 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb7f400c970 tx=0x7fb7f400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:35.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.951+0000 7fb7fdffb640 1 -- 172.21.15.97:0/3355436790 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f4007bb0 con 0x7fb800100ea0 2024-04-23T10:25:35.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.951+0000 7fb806eb4640 1 -- 172.21.15.97:0/3355436790 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8001a00c0 con 0x7fb800100ea0 2024-04-23T10:25:35.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.951+0000 7fb7fdffb640 1 -- 172.21.15.97:0/3355436790 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7f4007d50 con 0x7fb800100ea0 2024-04-23T10:25:35.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.951+0000 7fb7fdffb640 1 -- 172.21.15.97:0/3355436790 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f400f450 con 0x7fb800100ea0 2024-04-23T10:25:35.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.952+0000 7fb806eb4640 1 -- 172.21.15.97:0/3355436790 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8001a0610 con 0x7fb800100ea0 2024-04-23T10:25:35.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.953+0000 7fb806eb4640 1 -- 172.21.15.97:0/3355436790 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb800108280 con 0x7fb800100ea0 2024-04-23T10:25:35.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.953+0000 7fb7fdffb640 1 -- 172.21.15.97:0/3355436790 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb7f4004f20 con 0x7fb800100ea0 2024-04-23T10:25:35.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.954+0000 7fb7fdffb640 1 --2- 172.21.15.97:0/3355436790 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fb7f00a3bb0 0x7fb7f00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:35.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.954+0000 7fb7fffff640 1 --2- 172.21.15.97:0/3355436790 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fb7f00a3bb0 0x7fb7f00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:35.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.954+0000 7fb7fdffb640 1 -- 172.21.15.97:0/3355436790 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fb7f40d01f0 con 0x7fb800100ea0 2024-04-23T10:25:35.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.954+0000 7fb7fffff640 1 --2- 172.21.15.97:0/3355436790 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fb7f00a3bb0 0x7fb7f00a6070 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec005fd0 tx=0x7fb7ec00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:35.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:35.959+0000 7fb7fdffb640 1 -- 172.21.15.97:0/3355436790 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb7f4014030 con 0x7fb800100ea0 2024-04-23T10:25:36.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:35 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/1644371043' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T10:25:36.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:35 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/3187142372' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T10:25:36.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:35 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/1029074349' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T10:25:36.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:35 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/1644371043' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T10:25:36.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:35 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/3187142372' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T10:25:36.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:35 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/1029074349' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T10:25:36.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.175+0000 7fb806eb4640 1 -- 172.21.15.97:0/3355436790 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb800062b50 con 0x7fb800100ea0 2024-04-23T10:25:36.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.176+0000 7fb7fdffb640 1 -- 172.21.15.97:0/3355436790 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11931 (secure 0 0 0) 0x7fb7f409cc00 con 0x7fb800100ea0 2024-04-23T10:25:36.175 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:25:36.175 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3946878762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3946878762},{"type":"v1","addr":"172.21.15.97:6835","nonce":3946878762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/209800861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":209800861},{"type":"v1","addr":"172.21.15.97:6837","nonce":209800861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T10:24:06.910134+0000","modified":"2024-04-23T10:24:12.969478+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6839/418013575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":418013575},{"type":"v1","addr":"172.21.15.97:6839","nonce":418013575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6841/747242680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":747242680},{"type":"v1","addr":"172.21.15.97:6841","nonce":747242680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6833/4292847121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4292847121},{"type":"v1","addr":"172.21.15.183:6833","nonce":4292847121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6833/3885816565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3885816565},{"type":"v1","addr":"172.21.15.196:6833","nonce":3885816565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1897093105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1897093105},{"type":"v1","addr":"172.21.15.183:6835","nonce":1897093105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1171477563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1171477563},{"type":"v1","addr":"172.21.15.196:6835","nonce":1171477563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1898341911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1898341911},{"type":"v1","addr":"172.21.15.183:6837","nonce":1898341911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.196:6837/3094946502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3094946502},{"type":"v1","addr":"172.21.15.196:6837","nonce":3094946502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.196:6839/83843556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":83843556},{"type":"v1","addr":"172.21.15.196:6839","nonce":83843556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6839/1763841130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1763841130},{"type":"v1","addr":"172.21.15.183:6839","nonce":1763841130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24469,24488,24398,24403,24487]},"id":1}]} 2024-04-23T10:25:36.176 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 25 2024-04-23T10:25:36.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.178+0000 7fb806eb4640 1 -- 172.21.15.97:0/3355436790 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fb7f00a3bb0 msgr2=0x7fb7f00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:36.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.178+0000 7fb806eb4640 1 --2- 172.21.15.97:0/3355436790 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fb7f00a3bb0 0x7fb7f00a6070 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec005fd0 tx=0x7fb7ec00b410 comp rx=0 tx=0).stop 2024-04-23T10:25:36.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.178+0000 7fb806eb4640 1 -- 172.21.15.97:0/3355436790 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb800100ea0 msgr2=0x7fb80006b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:36.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.178+0000 7fb806eb4640 1 --2- 172.21.15.97:0/3355436790 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb800100ea0 0x7fb80006b120 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb7f400c970 tx=0x7fb7f400ce30 comp rx=0 tx=0).stop 2024-04-23T10:25:36.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.178+0000 7fb806eb4640 1 -- 172.21.15.97:0/3355436790 shutdown_connections 2024-04-23T10:25:36.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.178+0000 7fb806eb4640 1 --2- 172.21.15.97:0/3355436790 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fb7f00a3bb0 0x7fb7f00a6070 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:36.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.178+0000 7fb806eb4640 1 --2- 172.21.15.97:0/3355436790 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb8001029b0 0x7fb80019f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:36.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.178+0000 7fb806eb4640 1 --2- 172.21.15.97:0/3355436790 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb800101ff0 0x7fb80019cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:36.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.178+0000 7fb806eb4640 1 --2- 172.21.15.97:0/3355436790 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb800100ea0 0x7fb80006b120 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:36.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.178+0000 7fb806eb4640 1 -- 172.21.15.97:0/3355436790 >> 172.21.15.97:0/3355436790 conn(0x7fb8000fc430 msgr2=0x7fb800105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:36.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.178+0000 7fb806eb4640 1 -- 172.21.15.97:0/3355436790 shutdown_connections 2024-04-23T10:25:36.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.178+0000 7fb806eb4640 1 -- 172.21.15.97:0/3355436790 wait complete. 2024-04-23T10:25:36.189 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T10:25:36.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.288+0000 7f4406144640 1 -- 172.21.15.97:0/3603441272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4400101fe0 msgr2=0x7f4400102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:36.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.288+0000 7f4406144640 1 --2- 172.21.15.97:0/3603441272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4400101fe0 0x7f4400102460 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f43f0009900 tx=0x7f43f002a410 comp rx=0 tx=0).stop 2024-04-23T10:25:36.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.288+0000 7f4406144640 1 -- 172.21.15.97:0/3603441272 shutdown_connections 2024-04-23T10:25:36.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.288+0000 7f4406144640 1 --2- 172.21.15.97:0/3603441272 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f44001029a0 0x7f44001092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:36.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.288+0000 7f4406144640 1 --2- 172.21.15.97:0/3603441272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4400101fe0 0x7f4400102460 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:36.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.288+0000 7f4406144640 1 --2- 172.21.15.97:0/3603441272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4400100e90 0x7f44001012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:36.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.288+0000 7f4406144640 1 -- 172.21.15.97:0/3603441272 >> 172.21.15.97:0/3603441272 conn(0x7f44000fc400 msgr2=0x7f44000fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:36.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.289+0000 7f4406144640 1 -- 172.21.15.97:0/3603441272 shutdown_connections 2024-04-23T10:25:36.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.289+0000 7f4406144640 1 -- 172.21.15.97:0/3603441272 wait complete. 2024-04-23T10:25:36.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.289+0000 7f4406144640 1 Processor -- start 2024-04-23T10:25:36.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.289+0000 7f4406144640 1 -- start start 2024-04-23T10:25:36.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.289+0000 7f4406144640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4400100e90 0x7f4400069c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:36.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.289+0000 7f4406144640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4400101fe0 0x7f440019cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:36.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.289+0000 7f4406144640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44001029a0 0x7f440019f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:36.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.289+0000 7f4406144640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f440010c9f0 con 0x7f44001029a0 2024-04-23T10:25:36.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.289+0000 7f4406144640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f440010c870 con 0x7f4400100e90 2024-04-23T10:25:36.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.289+0000 7f4406144640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f440010cb70 con 0x7f4400101fe0 2024-04-23T10:25:36.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.289+0000 7f43fffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44001029a0 0x7f440019f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:36.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.289+0000 7f43fffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44001029a0 0x7f440019f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59678/0 (socket says 172.21.15.97:59678) 2024-04-23T10:25:36.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.289+0000 7f43fffff640 1 -- 172.21.15.97:0/109957840 learned_addr learned my addr 172.21.15.97:0/109957840 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:25:36.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.290+0000 7f43fffff640 1 -- 172.21.15.97:0/109957840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4400101fe0 msgr2=0x7f440019cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:36.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.290+0000 7f43feffd640 1 --2- 172.21.15.97:0/109957840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4400101fe0 0x7f440019cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:36.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.290+0000 7f43ff7fe640 1 --2- 172.21.15.97:0/109957840 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4400100e90 0x7f4400069c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:36.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.290+0000 7f43fffff640 1 --2- 172.21.15.97:0/109957840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4400101fe0 0x7f440019cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:36.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.290+0000 7f43fffff640 1 -- 172.21.15.97:0/109957840 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4400100e90 msgr2=0x7f4400069c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:36.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.290+0000 7f43fffff640 1 --2- 172.21.15.97:0/109957840 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4400100e90 0x7f4400069c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:36.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.290+0000 7f43fffff640 1 -- 172.21.15.97:0/109957840 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f440019fe20 con 0x7f44001029a0 2024-04-23T10:25:36.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.290+0000 7f43fffff640 1 --2- 172.21.15.97:0/109957840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44001029a0 0x7f440019f560 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f43f800eb10 tx=0x7f43f800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:36.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.290+0000 7f43fcff9640 1 -- 172.21.15.97:0/109957840 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f8004b20 con 0x7f44001029a0 2024-04-23T10:25:36.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.290+0000 7f4406144640 1 -- 172.21.15.97:0/109957840 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44001a0110 con 0x7f44001029a0 2024-04-23T10:25:36.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.290+0000 7f43fcff9640 1 -- 172.21.15.97:0/109957840 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43f8004cc0 con 0x7f44001029a0 2024-04-23T10:25:36.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.290+0000 7f43fcff9640 1 -- 172.21.15.97:0/109957840 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f8010670 con 0x7f44001029a0 2024-04-23T10:25:36.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.290+0000 7f4406144640 1 -- 172.21.15.97:0/109957840 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44001a0610 con 0x7f44001029a0 2024-04-23T10:25:36.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.291+0000 7f4406144640 1 -- 172.21.15.97:0/109957840 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4400108270 con 0x7f44001029a0 2024-04-23T10:25:36.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.292+0000 7f43fcff9640 1 -- 172.21.15.97:0/109957840 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f43f8005020 con 0x7f44001029a0 2024-04-23T10:25:36.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.293+0000 7f43fcff9640 1 --2- 172.21.15.97:0/109957840 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f43e80a3bb0 0x7f43e80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:36.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.293+0000 7f43ff7fe640 1 --2- 172.21.15.97:0/109957840 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f43e80a3bb0 0x7f43e80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:36.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.293+0000 7f43fcff9640 1 -- 172.21.15.97:0/109957840 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f43f80d0ab0 con 0x7f44001029a0 2024-04-23T10:25:36.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.293+0000 7f43ff7fe640 1 --2- 172.21.15.97:0/109957840 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f43e80a3bb0 0x7f43e80a6070 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f43f4005fd0 tx=0x7f43f4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:36.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.297+0000 7f43fcff9640 1 -- 172.21.15.97:0/109957840 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f43f809d4c0 con 0x7f44001029a0 2024-04-23T10:25:36.478 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.479+0000 7f4406144640 1 -- 172.21.15.97:0/109957840 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4400062b50 con 0x7f44001029a0 2024-04-23T10:25:36.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.480+0000 7f43fcff9640 1 -- 172.21.15.97:0/109957840 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22606 (secure 0 0 0) 0x7f43f809cdc0 con 0x7f44001029a0 2024-04-23T10:25:36.480 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:25:36.480 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":88,"fsid":"9b22af88-0159-11ef-bc93-c7b262605968","created":"2024-04-23T10:16:05.983305+0000","modified":"2024-04-23T10:25:00.660133+0000","last_up_change":"2024-04-23T10:23:13.114761+0000","last_in_change":"2024-04-23T10:22:52.804880+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T10:19:25.559138+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T10:24:04.570049+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T10:24:05.861631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"58cb995d-de35-4522-ac81-4982401b260d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6803","nonce":3031221802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6805","nonce":3031221802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6809","nonce":3031221802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6807","nonce":3031221802}]},"public_addr":"172.21.15.97:6803/3031221802","cluster_addr":"172.21.15.97:6805/3031221802","heartbeat_back_addr":"172.21.15.97:6809/3031221802","heartbeat_front_addr":"172.21.15.97:6807/3031221802","state":["exists","up"]},{"osd":1,"uuid":"b47bd8ee-1f21-4834-acf5-bb40f97ab3ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6801","nonce":1108870440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6803","nonce":1108870440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6807","nonce":1108870440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6805","nonce":1108870440}]},"public_addr":"172.21.15.183:6801/1108870440","cluster_addr":"172.21.15.183:6803/1108870440","heartbeat_back_addr":"172.21.15.183:6807/1108870440","heartbeat_front_addr":"172.21.15.183:6805/1108870440","state":["exists","up"]},{"osd":2,"uuid":"17c4eec3-6c47-4334-9b63-1d5ad9ec5988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6801","nonce":2877039731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6803","nonce":2877039731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6807","nonce":2877039731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6805","nonce":2877039731}]},"public_addr":"172.21.15.196:6801/2877039731","cluster_addr":"172.21.15.196:6803/2877039731","heartbeat_back_addr":"172.21.15.196:6807/2877039731","heartbeat_front_addr":"172.21.15.196:6805/2877039731","state":["exists","up"]},{"osd":3,"uuid":"3f731c38-2e45-4ca1-8dce-efc4a1b2166b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6811","nonce":4276311194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6813","nonce":4276311194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6817","nonce":4276311194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6815","nonce":4276311194}]},"public_addr":"172.21.15.97:6811/4276311194","cluster_addr":"172.21.15.97:6813/4276311194","heartbeat_back_addr":"172.21.15.97:6817/4276311194","heartbeat_front_addr":"172.21.15.97:6815/4276311194","state":["exists","up"]},{"osd":4,"uuid":"fdcafb14-9b82-4134-bd5b-79c0516f6cbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6809","nonce":63913916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6811","nonce":63913916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6815","nonce":63913916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6813","nonce":63913916}]},"public_addr":"172.21.15.183:6809/63913916","cluster_addr":"172.21.15.183:6811/63913916","heartbeat_back_addr":"172.21.15.183:6815/63913916","heartbeat_front_addr":"172.21.15.183:6813/63913916","state":["exists","up"]},{"osd":5,"uuid":"cdcc18d8-da3c-4012-bf12-876bb39e66eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6809","nonce":2804446896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6811","nonce":2804446896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6815","nonce":2804446896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6813","nonce":2804446896}]},"public_addr":"172.21.15.196:6809/2804446896","cluster_addr":"172.21.15.196:6811/2804446896","heartbeat_back_addr":"172.21.15.196:6815/2804446896","heartbeat_front_addr":"172.21.15.196:6813/2804446896","state":["exists","up"]},{"osd":6,"uuid":"071abb5c-2edf-4832-b8aa-3b80d20096fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6819","nonce":2080916705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6821","nonce":2080916705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6825","nonce":2080916705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6823","nonce":2080916705}]},"public_addr":"172.21.15.97:6819/2080916705","cluster_addr":"172.21.15.97:6821/2080916705","heartbeat_back_addr":"172.21.15.97:6825/2080916705","heartbeat_front_addr":"172.21.15.97:6823/2080916705","state":["exists","up"]},{"osd":7,"uuid":"9d54760b-66df-486b-8abb-017f3b64b4d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6817","nonce":1865873033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6819","nonce":1865873033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6823","nonce":1865873033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6821","nonce":1865873033}]},"public_addr":"172.21.15.183:6817/1865873033","cluster_addr":"172.21.15.183:6819/1865873033","heartbeat_back_addr":"172.21.15.183:6823/1865873033","heartbeat_front_addr":"172.21.15.183:6821/1865873033","state":["exists","up"]},{"osd":8,"uuid":"5cd3100a-bd6a-4be5-b5e9-0d0afc325193","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6817","nonce":4063932139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6819","nonce":4063932139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6823","nonce":4063932139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6821","nonce":4063932139}]},"public_addr":"172.21.15.196:6817/4063932139","cluster_addr":"172.21.15.196:6819/4063932139","heartbeat_back_addr":"172.21.15.196:6823/4063932139","heartbeat_front_addr":"172.21.15.196:6821/4063932139","state":["exists","up"]},{"osd":9,"uuid":"b6a5f4a1-4ecc-4f73-a2df-07551d9298ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6827","nonce":1887432474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6829","nonce":1887432474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6833","nonce":1887432474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6831","nonce":1887432474}]},"public_addr":"172.21.15.97:6827/1887432474","cluster_addr":"172.21.15.97:6829/1887432474","heartbeat_back_addr":"172.21.15.97:6833/1887432474","heartbeat_front_addr":"172.21.15.97:6831/1887432474","state":["exists","up"]},{"osd":10,"uuid":"86f59816-1ea6-4ae9-8796-cf5fc4831064","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6825","nonce":799434647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6827","nonce":799434647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6831","nonce":799434647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6829","nonce":799434647}]},"public_addr":"172.21.15.183:6825/799434647","cluster_addr":"172.21.15.183:6827/799434647","heartbeat_back_addr":"172.21.15.183:6831/799434647","heartbeat_front_addr":"172.21.15.183:6829/799434647","state":["exists","up"]},{"osd":11,"uuid":"95d64a89-a9f8-4c7a-8c0d-da7fb26cfff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6825","nonce":1873969467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6827","nonce":1873969467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6831","nonce":1873969467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6829","nonce":1873969467}]},"public_addr":"172.21.15.196:6825/1873969467","cluster_addr":"172.21.15.196:6827/1873969467","heartbeat_back_addr":"172.21.15.196:6831/1873969467","heartbeat_front_addr":"172.21.15.196:6829/1873969467","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:18:33.721156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:18:57.586709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:19:22.069458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:19:46.451597+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:20:11.561019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:20:36.298672+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:21:01.661804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:21:27.324128+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:21:52.898197+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:22:19.614326+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:22:45.154917+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:23:11.222612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/3866016040":"2024-04-24T10:16:59.468237+0000","172.21.15.97:6801/3297166858":"2024-04-24T10:16:59.468237+0000","172.21.15.97:0/3804743548":"2024-04-24T10:16:59.468237+0000","172.21.15.97:6800/3706463017":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/3715079438":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/786806834":"2024-04-24T10:16:59.468237+0000","172.21.15.97:6801/3706463017":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/3294175554":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/1449136760":"2024-04-24T10:16:33.128917+0000","172.21.15.97:6800/3297166858":"2024-04-24T10:16:59.468237+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T10:25:36.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.482+0000 7f4406144640 1 -- 172.21.15.97:0/109957840 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f43e80a3bb0 msgr2=0x7f43e80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:36.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.482+0000 7f4406144640 1 --2- 172.21.15.97:0/109957840 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f43e80a3bb0 0x7f43e80a6070 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f43f4005fd0 tx=0x7f43f4005850 comp rx=0 tx=0).stop 2024-04-23T10:25:36.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.482+0000 7f4406144640 1 -- 172.21.15.97:0/109957840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44001029a0 msgr2=0x7f440019f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:36.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.482+0000 7f4406144640 1 --2- 172.21.15.97:0/109957840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44001029a0 0x7f440019f560 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f43f800eb10 tx=0x7f43f800c2d0 comp rx=0 tx=0).stop 2024-04-23T10:25:36.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.482+0000 7f4406144640 1 -- 172.21.15.97:0/109957840 shutdown_connections 2024-04-23T10:25:36.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.483+0000 7f4406144640 1 --2- 172.21.15.97:0/109957840 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f43e80a3bb0 0x7f43e80a6070 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:36.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.483+0000 7f4406144640 1 --2- 172.21.15.97:0/109957840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44001029a0 0x7f440019f560 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:36.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.483+0000 7f4406144640 1 --2- 172.21.15.97:0/109957840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4400101fe0 0x7f440019cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:36.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.483+0000 7f4406144640 1 --2- 172.21.15.97:0/109957840 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4400100e90 0x7f4400069c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:36.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.483+0000 7f4406144640 1 -- 172.21.15.97:0/109957840 >> 172.21.15.97:0/109957840 conn(0x7f44000fc400 msgr2=0x7f4400105440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:36.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.483+0000 7f4406144640 1 -- 172.21.15.97:0/109957840 shutdown_connections 2024-04-23T10:25:36.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.483+0000 7f4406144640 1 -- 172.21.15.97:0/109957840 wait complete. 2024-04-23T10:25:36.493 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T10:25:36.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.592+0000 7f0303e23640 1 -- 172.21.15.97:0/2414695911 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02fc1029a0 msgr2=0x7f02fc1092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:36.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.592+0000 7f0303e23640 1 --2- 172.21.15.97:0/2414695911 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02fc1029a0 0x7f02fc1092c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f02ec009ca0 tx=0x7f02ec02a8c0 comp rx=0 tx=0).stop 2024-04-23T10:25:36.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.592+0000 7f0303e23640 1 -- 172.21.15.97:0/2414695911 shutdown_connections 2024-04-23T10:25:36.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.592+0000 7f0303e23640 1 --2- 172.21.15.97:0/2414695911 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02fc1029a0 0x7f02fc1092c0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:36.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.592+0000 7f0303e23640 1 --2- 172.21.15.97:0/2414695911 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02fc101fe0 0x7f02fc102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:36.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.592+0000 7f0303e23640 1 --2- 172.21.15.97:0/2414695911 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02fc100e90 0x7f02fc1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:36.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.592+0000 7f0303e23640 1 -- 172.21.15.97:0/2414695911 >> 172.21.15.97:0/2414695911 conn(0x7f02fc0fc400 msgr2=0x7f02fc0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:36.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.592+0000 7f0303e23640 1 -- 172.21.15.97:0/2414695911 shutdown_connections 2024-04-23T10:25:36.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.592+0000 7f0303e23640 1 -- 172.21.15.97:0/2414695911 wait complete. 2024-04-23T10:25:36.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.593+0000 7f0303e23640 1 Processor -- start 2024-04-23T10:25:36.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.593+0000 7f0303e23640 1 -- start start 2024-04-23T10:25:36.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.593+0000 7f0303e23640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02fc100e90 0x7f02fc197060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:36.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.593+0000 7f0303e23640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02fc101fe0 0x7f02fc1975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:36.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.593+0000 7f0303e23640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02fc1029a0 0x7f02fc19e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:36.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.593+0000 7f0303e23640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f02fc10c9e0 con 0x7f02fc100e90 2024-04-23T10:25:36.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.593+0000 7f0303e23640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f02fc10c860 con 0x7f02fc1029a0 2024-04-23T10:25:36.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.593+0000 7f0303e23640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f02fc10cb60 con 0x7f02fc101fe0 2024-04-23T10:25:36.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.593+0000 7f0301bbf640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02fc100e90 0x7f02fc197060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:36.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.593+0000 7f0301bbf640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02fc100e90 0x7f02fc197060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59688/0 (socket says 172.21.15.97:59688) 2024-04-23T10:25:36.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.593+0000 7f0301bbf640 1 -- 172.21.15.97:0/2696616206 learned_addr learned my addr 172.21.15.97:0/2696616206 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:25:36.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.593+0000 7f03023c0640 1 --2- 172.21.15.97:0/2696616206 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02fc1029a0 0x7f02fc19e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:36.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.593+0000 7f0301bbf640 1 -- 172.21.15.97:0/2696616206 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02fc101fe0 msgr2=0x7f02fc1975a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:36.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.593+0000 7f0301bbf640 1 --2- 172.21.15.97:0/2696616206 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02fc101fe0 0x7f02fc1975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:36.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.593+0000 7f0301bbf640 1 -- 172.21.15.97:0/2696616206 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02fc1029a0 msgr2=0x7f02fc19e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:36.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.593+0000 7f0301bbf640 1 --2- 172.21.15.97:0/2696616206 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02fc1029a0 0x7f02fc19e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:36.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.593+0000 7f0301bbf640 1 -- 172.21.15.97:0/2696616206 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02fc19ec80 con 0x7f02fc100e90 2024-04-23T10:25:36.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.593+0000 7f0301bbf640 1 --2- 172.21.15.97:0/2696616206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02fc100e90 0x7f02fc197060 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f02f400c890 tx=0x7f02f400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:36.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.593+0000 7f02faffd640 1 -- 172.21.15.97:0/2696616206 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f4004220 con 0x7f02fc100e90 2024-04-23T10:25:36.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.594+0000 7f02faffd640 1 -- 172.21.15.97:0/2696616206 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02f40043c0 con 0x7f02fc100e90 2024-04-23T10:25:36.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.594+0000 7f0303e23640 1 -- 172.21.15.97:0/2696616206 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02fc19ef70 con 0x7f02fc100e90 2024-04-23T10:25:36.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.594+0000 7f02faffd640 1 -- 172.21.15.97:0/2696616206 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f4004a00 con 0x7f02fc100e90 2024-04-23T10:25:36.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.594+0000 7f0303e23640 1 -- 172.21.15.97:0/2696616206 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02fc19f480 con 0x7f02fc100e90 2024-04-23T10:25:36.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.594+0000 7f0303e23640 1 -- 172.21.15.97:0/2696616206 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02fc1081a0 con 0x7f02fc100e90 2024-04-23T10:25:36.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.595+0000 7f02faffd640 1 -- 172.21.15.97:0/2696616206 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f02f40079b0 con 0x7f02fc100e90 2024-04-23T10:25:36.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.596+0000 7f02faffd640 1 --2- 172.21.15.97:0/2696616206 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f02e40a3c30 0x7f02e40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:36.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.596+0000 7f03013be640 1 --2- 172.21.15.97:0/2696616206 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f02e40a3c30 0x7f02e40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:36.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.596+0000 7f02faffd640 1 -- 172.21.15.97:0/2696616206 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f02f40d0500 con 0x7f02fc100e90 2024-04-23T10:25:36.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.596+0000 7f03013be640 1 --2- 172.21.15.97:0/2696616206 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f02e40a3c30 0x7f02e40a60f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f02fc198570 tx=0x7f02f0005c10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:36.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.600+0000 7f02faffd640 1 -- 172.21.15.97:0/2696616206 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f02f4014030 con 0x7f02fc100e90 2024-04-23T10:25:36.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.818+0000 7f0303e23640 1 -- 172.21.15.97:0/2696616206 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f02fc02d870 con 0x7f02fc100e90 2024-04-23T10:25:36.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.819+0000 7f02faffd640 1 -- 172.21.15.97:0/2696616206 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11931 (secure 0 0 0) 0x7f02f409cf90 con 0x7f02fc100e90 2024-04-23T10:25:36.821 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:25:36.821 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3946878762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3946878762},{"type":"v1","addr":"172.21.15.97:6835","nonce":3946878762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/209800861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":209800861},{"type":"v1","addr":"172.21.15.97:6837","nonce":209800861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T10:24:06.910134+0000","modified":"2024-04-23T10:24:12.969478+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6839/418013575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":418013575},{"type":"v1","addr":"172.21.15.97:6839","nonce":418013575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6841/747242680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":747242680},{"type":"v1","addr":"172.21.15.97:6841","nonce":747242680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6833/4292847121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4292847121},{"type":"v1","addr":"172.21.15.183:6833","nonce":4292847121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6833/3885816565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3885816565},{"type":"v1","addr":"172.21.15.196:6833","nonce":3885816565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1897093105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1897093105},{"type":"v1","addr":"172.21.15.183:6835","nonce":1897093105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1171477563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1171477563},{"type":"v1","addr":"172.21.15.196:6835","nonce":1171477563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1898341911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1898341911},{"type":"v1","addr":"172.21.15.183:6837","nonce":1898341911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.196:6837/3094946502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3094946502},{"type":"v1","addr":"172.21.15.196:6837","nonce":3094946502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.196:6839/83843556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":83843556},{"type":"v1","addr":"172.21.15.196:6839","nonce":83843556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6839/1763841130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1763841130},{"type":"v1","addr":"172.21.15.183:6839","nonce":1763841130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24469,24488,24398,24403,24487]},"id":1}]} 2024-04-23T10:25:36.822 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 25 2024-04-23T10:25:36.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.821+0000 7f0303e23640 1 -- 172.21.15.97:0/2696616206 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f02e40a3c30 msgr2=0x7f02e40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:36.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.821+0000 7f0303e23640 1 --2- 172.21.15.97:0/2696616206 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f02e40a3c30 0x7f02e40a60f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f02fc198570 tx=0x7f02f0005c10 comp rx=0 tx=0).stop 2024-04-23T10:25:36.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.821+0000 7f0303e23640 1 -- 172.21.15.97:0/2696616206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02fc100e90 msgr2=0x7f02fc197060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:36.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.821+0000 7f0303e23640 1 --2- 172.21.15.97:0/2696616206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02fc100e90 0x7f02fc197060 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f02f400c890 tx=0x7f02f400cd50 comp rx=0 tx=0).stop 2024-04-23T10:25:36.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.821+0000 7f0303e23640 1 -- 172.21.15.97:0/2696616206 shutdown_connections 2024-04-23T10:25:36.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.821+0000 7f0303e23640 1 --2- 172.21.15.97:0/2696616206 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f02e40a3c30 0x7f02e40a60f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:36.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.821+0000 7f0303e23640 1 --2- 172.21.15.97:0/2696616206 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02fc1029a0 0x7f02fc19e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:36.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.821+0000 7f0303e23640 1 --2- 172.21.15.97:0/2696616206 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02fc101fe0 0x7f02fc1975a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:36.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.821+0000 7f0303e23640 1 --2- 172.21.15.97:0/2696616206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f02fc100e90 0x7f02fc197060 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:36.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.821+0000 7f0303e23640 1 -- 172.21.15.97:0/2696616206 >> 172.21.15.97:0/2696616206 conn(0x7f02fc0fc400 msgr2=0x7f02fc105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:36.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.821+0000 7f0303e23640 1 -- 172.21.15.97:0/2696616206 shutdown_connections 2024-04-23T10:25:36.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.821+0000 7f0303e23640 1 -- 172.21.15.97:0/2696616206 wait complete. 2024-04-23T10:25:36.831 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:36 smithi097 ceph-mon[29197]: pgmap v336: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 11 op/s 2024-04-23T10:25:36.831 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:36 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/2193923269' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T10:25:36.831 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:36 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/3355436790' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:25:36.831 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:36 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/109957840' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T10:25:36.833 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-23T10:24:06.910134+0000', 'modified': '2024-04-23T10:24:12.969478+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24488, 'mds_1': 24487, 'mds_2': 24469, 'mds_3': 24403, 'mds_4': 24398}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14640': {'gid': 14640, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.97:6839/418013575', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6838', 'nonce': 418013575}, {'type': 'v1', 'addr': '172.21.15.97:6839', 'nonce': 418013575}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14652': {'gid': 14652, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.97:6841/747242680', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6840', 'nonce': 747242680}, {'type': 'v1', 'addr': '172.21.15.97:6841', 'nonce': 747242680}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24398': {'gid': 24398, 'name': 'k', 'rank': 4, 'incarnation': 23, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.183:6833/4292847121', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6832', 'nonce': 4292847121}, {'type': 'v1', 'addr': '172.21.15.183:6833', 'nonce': 4292847121}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24403': {'gid': 24403, 'name': 'l', 'rank': 3, 'incarnation': 21, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.196:6833/3885816565', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6832', 'nonce': 3885816565}, {'type': 'v1', 'addr': '172.21.15.196:6833', 'nonce': 3885816565}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24419': {'gid': 24419, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.183:6835/1897093105', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6834', 'nonce': 1897093105}, {'type': 'v1', 'addr': '172.21.15.183:6835', 'nonce': 1897093105}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24451': {'gid': 24451, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.196:6835/1171477563', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6834', 'nonce': 1171477563}, {'type': 'v1', 'addr': '172.21.15.196:6835', 'nonce': 1171477563}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24461': {'gid': 24461, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.183:6837/1898341911', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6836', 'nonce': 1898341911}, {'type': 'v1', 'addr': '172.21.15.183:6837', 'nonce': 1898341911}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24469': {'gid': 24469, 'name': 'c', 'rank': 2, 'incarnation': 19, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.196:6837/3094946502', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6836', 'nonce': 3094946502}, {'type': 'v1', 'addr': '172.21.15.196:6837', 'nonce': 3094946502}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24487': {'gid': 24487, 'name': 'i', 'rank': 1, 'incarnation': 17, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.196:6839/83843556', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6838', 'nonce': 83843556}, {'type': 'v1', 'addr': '172.21.15.196:6839', 'nonce': 83843556}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24488': {'gid': 24488, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.183:6839/1763841130', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6838', 'nonce': 1763841130}, {'type': 'v1', 'addr': '172.21.15.183:6839', 'nonce': 1763841130}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24488, 'qdb_cluster': [24469, 24488, 24398, 24403, 24487]} 2024-04-23T10:25:36.833 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-23T10:25:36.833 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2024-04-23T10:25:36.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.931+0000 7f159ee13640 1 -- 172.21.15.97:0/9015441 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1598103b40 msgr2=0x7f159810a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:36.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.931+0000 7f159ee13640 1 --2- 172.21.15.97:0/9015441 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1598103b40 0x7f159810a3d0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1590009ca0 tx=0x7f159002a8c0 comp rx=0 tx=0).stop 2024-04-23T10:25:36.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.931+0000 7f159ee13640 1 -- 172.21.15.97:0/9015441 shutdown_connections 2024-04-23T10:25:36.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.931+0000 7f159ee13640 1 --2- 172.21.15.97:0/9015441 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1598103b40 0x7f159810a3d0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:36.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.931+0000 7f159ee13640 1 --2- 172.21.15.97:0/9015441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1598103180 0x7f1598103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:36.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.931+0000 7f159ee13640 1 --2- 172.21.15.97:0/9015441 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1598101f90 0x7f15981023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:36.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.931+0000 7f159ee13640 1 -- 172.21.15.97:0/9015441 >> 172.21.15.97:0/9015441 conn(0x7f15980fd500 msgr2=0x7f15980ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:36.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.931+0000 7f159ee13640 1 -- 172.21.15.97:0/9015441 shutdown_connections 2024-04-23T10:25:36.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.931+0000 7f159ee13640 1 -- 172.21.15.97:0/9015441 wait complete. 2024-04-23T10:25:36.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.932+0000 7f159ee13640 1 Processor -- start 2024-04-23T10:25:36.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.932+0000 7f159ee13640 1 -- start start 2024-04-23T10:25:36.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.932+0000 7f159ee13640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1598101f90 0x7f1598198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:36.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.932+0000 7f159ee13640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1598103180 0x7f15981985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:36.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.932+0000 7f159ee13640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1598103b40 0x7f159819f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:36.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.932+0000 7f159ee13640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f159810da90 con 0x7f1598103b40 2024-04-23T10:25:36.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.932+0000 7f159ee13640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f159810d910 con 0x7f1598103180 2024-04-23T10:25:36.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.932+0000 7f159ee13640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f159810dc10 con 0x7f1598101f90 2024-04-23T10:25:36.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.932+0000 7f159d3b0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1598103b40 0x7f159819f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:36.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.932+0000 7f159d3b0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1598103b40 0x7f159819f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59710/0 (socket says 172.21.15.97:59710) 2024-04-23T10:25:36.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.932+0000 7f159d3b0640 1 -- 172.21.15.97:0/3629944595 learned_addr learned my addr 172.21.15.97:0/3629944595 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:25:36.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.932+0000 7f159cbaf640 1 --2- 172.21.15.97:0/3629944595 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1598101f90 0x7f1598198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:36.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.932+0000 7f159d3b0640 1 -- 172.21.15.97:0/3629944595 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1598101f90 msgr2=0x7f1598198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:36.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.932+0000 7f159d3b0640 1 --2- 172.21.15.97:0/3629944595 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1598101f90 0x7f1598198070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:36.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.933+0000 7f159d3b0640 1 -- 172.21.15.97:0/3629944595 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1598103180 msgr2=0x7f15981985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:36.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.933+0000 7f1597fff640 1 --2- 172.21.15.97:0/3629944595 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1598103180 0x7f15981985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:36.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.933+0000 7f159d3b0640 1 --2- 172.21.15.97:0/3629944595 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1598103180 0x7f15981985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:36.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.933+0000 7f159d3b0640 1 -- 172.21.15.97:0/3629944595 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f159819fc90 con 0x7f1598103b40 2024-04-23T10:25:36.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.933+0000 7f159d3b0640 1 --2- 172.21.15.97:0/3629944595 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1598103b40 0x7f159819f620 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1590031a20 tx=0x7f1590003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:36.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.933+0000 7f1595ffb640 1 -- 172.21.15.97:0/3629944595 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1590004240 con 0x7f1598103b40 2024-04-23T10:25:36.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.933+0000 7f159ee13640 1 -- 172.21.15.97:0/3629944595 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f159819ff20 con 0x7f1598103b40 2024-04-23T10:25:36.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.933+0000 7f1595ffb640 1 -- 172.21.15.97:0/3629944595 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1590003e10 con 0x7f1598103b40 2024-04-23T10:25:36.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.933+0000 7f1595ffb640 1 -- 172.21.15.97:0/3629944595 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15900305c0 con 0x7f1598103b40 2024-04-23T10:25:36.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.933+0000 7f159ee13640 1 -- 172.21.15.97:0/3629944595 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15981a0430 con 0x7f1598103b40 2024-04-23T10:25:36.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.935+0000 7f1595ffb640 1 -- 172.21.15.97:0/3629944595 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f159002f660 con 0x7f1598103b40 2024-04-23T10:25:36.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.935+0000 7f15a1c0a740 1 Processor -- start 2024-04-23T10:25:36.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.935+0000 7f15a1c0a740 1 -- start start 2024-04-23T10:25:36.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.936+0000 7f15a1c0a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d54ce51b80 0x55d54ce51fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:36.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.936+0000 7f15a1c0a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d54ce52d50 0x55d54ce531d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:36.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.936+0000 7f15a1c0a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d54ce53710 0x55d54ce59fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:36.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.936+0000 7f15a1c0a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d54cdb6c50 con 0x55d54ce53710 2024-04-23T10:25:36.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.936+0000 7f15a1c0a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d54cd53830 con 0x55d54ce51b80 2024-04-23T10:25:36.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.936+0000 7f15a1c0a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d54cdb2c10 con 0x55d54ce52d50 2024-04-23T10:25:36.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.936+0000 7f1597fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d54ce53710 0x55d54ce59fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:36.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.936+0000 7f1597fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d54ce53710 0x55d54ce59fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59720/0 (socket says 172.21.15.97:59720) 2024-04-23T10:25:36.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.936+0000 7f1597fff640 1 -- 172.21.15.97:0/2159370633 learned_addr learned my addr 172.21.15.97:0/2159370633 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:25:36.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.936+0000 7f1597fff640 1 --2- 172.21.15.97:0/2159370633 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d54ce51b80 0x55d54ce51fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:36.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.936+0000 7f1595ffb640 1 --2- 172.21.15.97:0/3629944595 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f15880a3bb0 0x7f15880a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:36.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.936+0000 7f159cbaf640 1 --2- 172.21.15.97:0/2159370633 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d54ce52d50 0x55d54ce531d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:36.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.936+0000 7f159d3b0640 1 --2- 172.21.15.97:0/3629944595 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f15880a3bb0 0x7f15880a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:36.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.936+0000 7f1597fff640 1 -- 172.21.15.97:0/2159370633 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d54ce52d50 msgr2=0x55d54ce531d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:25:36.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.936+0000 7f1597fff640 1 --2- 172.21.15.97:0/2159370633 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d54ce52d50 0x55d54ce531d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:36.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.936+0000 7f1597fff640 1 -- 172.21.15.97:0/2159370633 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d54ce51b80 msgr2=0x55d54ce51fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:25:36.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.936+0000 7f1597fff640 1 --2- 172.21.15.97:0/2159370633 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d54ce51b80 0x55d54ce51fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:36.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.936+0000 7f1597fff640 1 -- 172.21.15.97:0/2159370633 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d54cac92d0 con 0x55d54ce53710 2024-04-23T10:25:36.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.936+0000 7f1595ffb640 1 -- 172.21.15.97:0/3629944595 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f15900f1d30 con 0x7f1598103b40 2024-04-23T10:25:36.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.936+0000 7f1597fff640 1 --2- 172.21.15.97:0/2159370633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d54ce53710 0x55d54ce59fa0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f15840127d0 tx=0x7f158400d380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6631f9ef1d593d28 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:36.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.936+0000 7f1582ffd640 1 -- 172.21.15.97:0/2159370633 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1584005220 con 0x55d54ce53710 2024-04-23T10:25:36.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.936+0000 7f1582ffd640 1 -- 172.21.15.97:0/2159370633 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1584004c50 con 0x55d54ce53710 2024-04-23T10:25:36.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.936+0000 7f159d3b0640 1 --2- 172.21.15.97:0/3629944595 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f15880a3bb0 0x7f15880a6070 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f15900ba4a0 tx=0x7f15900e9150 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:36.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.936+0000 7f1582ffd640 1 -- 172.21.15.97:0/2159370633 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f158400a9b0 con 0x55d54ce53710 2024-04-23T10:25:36.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.937+0000 7f15a1c0a740 1 -- 172.21.15.97:0/2159370633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d54ce53710 msgr2=0x55d54ce59fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:36.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.937+0000 7f15a1c0a740 1 --2- 172.21.15.97:0/2159370633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d54ce53710 0x55d54ce59fa0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f15840127d0 tx=0x7f158400d380 comp rx=0 tx=0).stop 2024-04-23T10:25:36.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.937+0000 7f15a1c0a740 1 -- 172.21.15.97:0/2159370633 shutdown_connections 2024-04-23T10:25:36.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.937+0000 7f15a1c0a740 1 --2- 172.21.15.97:0/2159370633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d54ce53710 0x55d54ce59fa0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:36.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.937+0000 7f15a1c0a740 1 --2- 172.21.15.97:0/2159370633 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d54ce52d50 0x55d54ce531d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:36.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.937+0000 7f15a1c0a740 1 --2- 172.21.15.97:0/2159370633 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d54ce51b80 0x55d54ce51fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:36.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.937+0000 7f15a1c0a740 1 -- 172.21.15.97:0/2159370633 >> 172.21.15.97:0/2159370633 conn(0x55d54ce4d070 msgr2=0x55d54ce4d470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:36.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.937+0000 7f15a1c0a740 1 -- 172.21.15.97:0/2159370633 shutdown_connections 2024-04-23T10:25:36.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.937+0000 7f15a1c0a740 1 -- 172.21.15.97:0/2159370633 wait complete. 2024-04-23T10:25:36.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.937+0000 7f15a1c0a740 1 -- start start 2024-04-23T10:25:36.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.937+0000 7f15a1c0a740 1 Processor -- start 2024-04-23T10:25:36.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.938+0000 7f15a1c0a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d54ce51b80 0x55d54ce5bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:36.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.938+0000 7f15a1c0a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d54ce52d50 0x55d54ce5c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:36.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.938+0000 7f15a1c0a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d54ce53710 0x55d54ce63560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:36.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.938+0000 7f15a1c0a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d54ce5a720 con 0x55d54ce52d50 2024-04-23T10:25:36.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.938+0000 7f15a1c0a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d54cd51470 con 0x55d54ce53710 2024-04-23T10:25:36.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.938+0000 7f15a1c0a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d54ce5a8a0 con 0x55d54ce51b80 2024-04-23T10:25:36.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.938+0000 7f15a1c0a740 10 client.0 _ll_register_callbacks cb 0x55d54cd76f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:25:36.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.938+0000 7f159cbaf640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d54ce52d50 0x55d54ce5c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:36.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.938+0000 7f159cbaf640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d54ce52d50 0x55d54ce5c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59722/0 (socket says 172.21.15.97:59722) 2024-04-23T10:25:36.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.938+0000 7f159cbaf640 1 -- 172.21.15.97:0/1330670812 learned_addr learned my addr 172.21.15.97:0/1330670812 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:25:36.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.938+0000 7f1597fff640 1 --2- 172.21.15.97:0/1330670812 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d54ce53710 0x55d54ce63560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:36.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.938+0000 7f159cbaf640 1 -- 172.21.15.97:0/1330670812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d54ce51b80 msgr2=0x55d54ce5bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:25:36.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.938+0000 7f159cbaf640 1 --2- 172.21.15.97:0/1330670812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d54ce51b80 0x55d54ce5bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:36.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.938+0000 7f159cbaf640 1 -- 172.21.15.97:0/1330670812 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d54ce53710 msgr2=0x55d54ce63560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:25:36.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.938+0000 7f159cbaf640 1 --2- 172.21.15.97:0/1330670812 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d54ce53710 0x55d54ce63560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:36.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.938+0000 7f159cbaf640 1 -- 172.21.15.97:0/1330670812 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d54cb936b0 con 0x55d54ce52d50 2024-04-23T10:25:36.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.938+0000 7f1597fff640 1 --2- 172.21.15.97:0/1330670812 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d54ce53710 0x55d54ce63560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:25:36.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.938+0000 7f159cbaf640 1 --2- 172.21.15.97:0/1330670812 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d54ce52d50 0x55d54ce5c520 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f158c010780 tx=0x7f158c010c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3960dd932763454b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:36.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.938+0000 7f1577fff640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:25:36.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.938+0000 7f1577fff640 1 -- 172.21.15.97:0/1330670812 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f158c01d060 con 0x55d54ce52d50 2024-04-23T10:25:36.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.939+0000 7f1577fff640 1 -- 172.21.15.97:0/1330670812 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f158c00ed40 con 0x55d54ce52d50 2024-04-23T10:25:36.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.939+0000 7f15a1c0a740 1 -- 172.21.15.97:0/1330670812 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1584004c50 con 0x55d54ce52d50 2024-04-23T10:25:36.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.939+0000 7f1577fff640 1 -- 172.21.15.97:0/1330670812 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f158c018430 con 0x55d54ce52d50 2024-04-23T10:25:36.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.939+0000 7f15a1c0a740 10 client.15078 fetch_fsmap learned FSMap version 25 2024-04-23T10:25:36.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.939+0000 7f15a1c0a740 1 -- 172.21.15.97:0/1330670812 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d54ce669a0 con 0x55d54ce52d50 2024-04-23T10:25:36.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.939+0000 7f1577fff640 1 -- 172.21.15.97:0/1330670812 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1584004c50 con 0x55d54ce52d50 2024-04-23T10:25:36.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.939+0000 7f1577fff640 1 -- 172.21.15.97:0/1330670812 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f158c00d560 con 0x55d54ce52d50 2024-04-23T10:25:36.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.939+0000 7f15a1c0a740 10 client.15078 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T10:25:36.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.939+0000 7f15a1c0a740 10 client.15078 resolve_mds: resolved name 'k' to daemon mds.k 2024-04-23T10:25:36.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.939+0000 7f15a1c0a740 20 client.15078 populate_metadata read hostname 'smithi097' 2024-04-23T10:25:36.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.939+0000 7f15a1c0a740 1 --2- 172.21.15.97:0/1330670812 >> [v2:172.21.15.183:6832/4292847121,v1:172.21.15.183:6833/4292847121] conn(0x55d54ce690c0 0x55d54ce6a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:36.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.939+0000 7f15a1c0a740 4 client.15078 mds_command: new command op to 24398 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:25:36.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.939+0000 7f15a1c0a740 1 -- 172.21.15.97:0/1330670812 --> [v2:172.21.15.183:6832/4292847121,v1:172.21.15.183:6833/4292847121] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d54cdc4670 con 0x55d54ce690c0 2024-04-23T10:25:36.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.940+0000 7f1597fff640 1 --2- 172.21.15.97:0/1330670812 >> [v2:172.21.15.183:6832/4292847121,v1:172.21.15.183:6833/4292847121] conn(0x55d54ce690c0 0x55d54ce6a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:36.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.940+0000 7f1597fff640 1 --2- 172.21.15.97:0/1330670812 >> [v2:172.21.15.183:6832/4292847121,v1:172.21.15.183:6833/4292847121] conn(0x55d54ce690c0 0x55d54ce6a530 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bd25c0997e7b0814 server_cookie=659fa655924585b3 in_seq=0 out_seq=0 2024-04-23T10:25:36.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.940+0000 7f1577fff640 10 client.15078 ms_handle_connect on v2:172.21.15.183:6832/4292847121 2024-04-23T10:25:36.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.941+0000 7f1577fff640 1 -- 172.21.15.97:0/1330670812 <== mds.4 v2:172.21.15.183:6832/4292847121 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d54cdc4670 con 0x55d54ce690c0 2024-04-23T10:25:36.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.941+0000 7f1577fff640 10 client.15078 handle_command_reply: tid=0 2024-04-23T10:25:36.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.941+0000 7f15a1c0a740 1 client.15078 shutdown 2024-04-23T10:25:36.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.941+0000 7f15a1c0a740 1 -- 172.21.15.97:0/1330670812 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d54ce52d50 msgr2=0x55d54ce5c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:36.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.941+0000 7f15a1c0a740 1 --2- 172.21.15.97:0/1330670812 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d54ce52d50 0x55d54ce5c520 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f158c010780 tx=0x7f158c010c40 comp rx=0 tx=0).stop 2024-04-23T10:25:36.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.941+0000 7f15a1c0a740 1 -- 172.21.15.97:0/1330670812 shutdown_connections 2024-04-23T10:25:36.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.941+0000 7f15a1c0a740 1 --2- 172.21.15.97:0/1330670812 >> [v2:172.21.15.183:6832/4292847121,v1:172.21.15.183:6833/4292847121] conn(0x55d54ce690c0 0x55d54ce6a530 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:36.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.941+0000 7f15a1c0a740 1 --2- 172.21.15.97:0/1330670812 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d54ce53710 0x55d54ce63560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:36.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.941+0000 7f15a1c0a740 1 --2- 172.21.15.97:0/1330670812 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d54ce52d50 0x55d54ce5c520 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:36.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.941+0000 7f15a1c0a740 1 --2- 172.21.15.97:0/1330670812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d54ce51b80 0x55d54ce5bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:36.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.941+0000 7f1577fff640 0 client.15078 ms_handle_reset on v2:172.21.15.183:6832/4292847121 2024-04-23T10:25:36.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.941+0000 7f15a1c0a740 1 -- 172.21.15.97:0/1330670812 >> 172.21.15.97:0/1330670812 conn(0x55d54ce4d070 msgr2=0x55d54ce4c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:36.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.941+0000 7f15a1c0a740 1 -- 172.21.15.97:0/1330670812 shutdown_connections 2024-04-23T10:25:36.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.941+0000 7f15a1c0a740 1 -- 172.21.15.97:0/1330670812 wait complete. 2024-04-23T10:25:36.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.941+0000 7f15a1c0a740 20 client.15078 trim_cache size 0 max 16384 2024-04-23T10:25:36.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.959+0000 7f15a1c0a740 1 Processor -- start 2024-04-23T10:25:36.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.959+0000 7f15a1c0a740 1 -- start start 2024-04-23T10:25:36.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.959+0000 7f15a1c0a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d54ce51b80 0x55d54ce4fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:36.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.959+0000 7f15a1c0a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d54ce52d50 0x55d54ce63520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:36.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.959+0000 7f15a1c0a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d54ce53710 0x55d54ce50190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:36.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.959+0000 7f15a1c0a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d54ce68870 con 0x55d54ce52d50 2024-04-23T10:25:36.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.959+0000 7f15a1c0a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d54ce68f40 con 0x55d54ce51b80 2024-04-23T10:25:36.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.959+0000 7f15a1c0a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d54ce69610 con 0x55d54ce53710 2024-04-23T10:25:36.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.959+0000 7f159cbaf640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d54ce52d50 0x55d54ce63520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:36.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.959+0000 7f159cbaf640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d54ce52d50 0x55d54ce63520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59736/0 (socket says 172.21.15.97:59736) 2024-04-23T10:25:36.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.959+0000 7f159cbaf640 1 -- 172.21.15.97:0/2417155722 learned_addr learned my addr 172.21.15.97:0/2417155722 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:25:36.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.959+0000 7f1597fff640 1 --2- 172.21.15.97:0/2417155722 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d54ce53710 0x55d54ce50190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:36.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.959+0000 7f159cbaf640 1 -- 172.21.15.97:0/2417155722 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d54ce53710 msgr2=0x55d54ce50190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:25:36.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.959+0000 7f159cbaf640 1 --2- 172.21.15.97:0/2417155722 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d54ce53710 0x55d54ce50190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:36.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.959+0000 7f159cbaf640 1 -- 172.21.15.97:0/2417155722 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d54ce51b80 msgr2=0x55d54ce4fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:25:36.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.959+0000 7f159cbaf640 1 --2- 172.21.15.97:0/2417155722 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d54ce51b80 0x55d54ce4fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:36.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.959+0000 7f159cbaf640 1 -- 172.21.15.97:0/2417155722 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d54ce527d0 con 0x55d54ce52d50 2024-04-23T10:25:36.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.960+0000 7f159cbaf640 1 --2- 172.21.15.97:0/2417155722 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d54ce52d50 0x55d54ce63520 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f158c00e010 tx=0x7f158c00e800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c68d4153ad668a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:36.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.960+0000 7f15827fc640 1 -- 172.21.15.97:0/2417155722 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f158c028040 con 0x55d54ce52d50 2024-04-23T10:25:36.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.960+0000 7f15827fc640 1 -- 172.21.15.97:0/2417155722 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f158c010070 con 0x55d54ce52d50 2024-04-23T10:25:36.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.960+0000 7f15827fc640 1 -- 172.21.15.97:0/2417155722 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f158c004a40 con 0x55d54ce52d50 2024-04-23T10:25:36.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.960+0000 7f15a1c0a740 1 -- 172.21.15.97:0/2417155722 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d54ce52d50 msgr2=0x55d54ce63520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:36.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.960+0000 7f15a1c0a740 1 --2- 172.21.15.97:0/2417155722 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d54ce52d50 0x55d54ce63520 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f158c00e010 tx=0x7f158c00e800 comp rx=0 tx=0).stop 2024-04-23T10:25:36.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.960+0000 7f15a1c0a740 1 -- 172.21.15.97:0/2417155722 shutdown_connections 2024-04-23T10:25:36.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.960+0000 7f15a1c0a740 1 --2- 172.21.15.97:0/2417155722 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d54ce53710 0x55d54ce50190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:36.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.960+0000 7f15a1c0a740 1 --2- 172.21.15.97:0/2417155722 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d54ce52d50 0x55d54ce63520 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:36.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.960+0000 7f15a1c0a740 1 --2- 172.21.15.97:0/2417155722 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d54ce51b80 0x55d54ce4fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:36.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.960+0000 7f15a1c0a740 1 -- 172.21.15.97:0/2417155722 >> 172.21.15.97:0/2417155722 conn(0x55d54ce4d070 msgr2=0x55d54ce480b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:36.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.960+0000 7f15a1c0a740 1 -- 172.21.15.97:0/2417155722 shutdown_connections 2024-04-23T10:25:36.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.960+0000 7f15a1c0a740 1 -- 172.21.15.97:0/2417155722 wait complete. 2024-04-23T10:25:36.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.961+0000 7f15a1c0a740 1 -- start start 2024-04-23T10:25:36.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.961+0000 7f15a1c0a740 1 Processor -- start 2024-04-23T10:25:36.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.961+0000 7f15a1c0a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d54ce51b80 0x55d54ce5ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:36.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.961+0000 7f15a1c0a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d54ce52d50 0x55d54ce5f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:36.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.961+0000 7f15a1c0a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d54ce53710 0x55d54ce62470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:36.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.961+0000 7f15a1c0a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d54ce6a3b0 con 0x55d54ce51b80 2024-04-23T10:25:36.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.961+0000 7f15a1c0a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d54ce69ce0 con 0x55d54ce52d50 2024-04-23T10:25:36.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.961+0000 7f15a1c0a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d54ce6aa80 con 0x55d54ce53710 2024-04-23T10:25:36.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.961+0000 7f15a1c0a740 10 client.0 _ll_register_callbacks cb 0x7f158c00b6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:25:36.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.961+0000 7f1597fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d54ce51b80 0x55d54ce5ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:36.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.961+0000 7f1597fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d54ce51b80 0x55d54ce5ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59742/0 (socket says 172.21.15.97:59742) 2024-04-23T10:25:36.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.961+0000 7f1597fff640 1 -- 172.21.15.97:0/3925902714 learned_addr learned my addr 172.21.15.97:0/3925902714 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:25:36.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.961+0000 7f1597fff640 1 -- 172.21.15.97:0/3925902714 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d54ce53710 msgr2=0x55d54ce62470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:25:36.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.961+0000 7f1597fff640 1 --2- 172.21.15.97:0/3925902714 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d54ce53710 0x55d54ce62470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:36.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.961+0000 7f1597fff640 1 -- 172.21.15.97:0/3925902714 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d54ce52d50 msgr2=0x55d54ce5f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:25:36.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.961+0000 7f1597fff640 1 --2- 172.21.15.97:0/3925902714 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d54ce52d50 0x55d54ce5f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:36.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.961+0000 7f1597fff640 1 -- 172.21.15.97:0/3925902714 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d54ce62a60 con 0x55d54ce51b80 2024-04-23T10:25:36.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.962+0000 7f1597fff640 1 --2- 172.21.15.97:0/3925902714 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d54ce51b80 0x55d54ce5ef60 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f15840127a0 tx=0x7f1584005ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=929ffd3ac0d6c600 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:36.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.962+0000 7f1577fff640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:25:36.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.962+0000 7f1577fff640 1 -- 172.21.15.97:0/3925902714 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f158402d060 con 0x55d54ce51b80 2024-04-23T10:25:36.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.962+0000 7f1577fff640 1 -- 172.21.15.97:0/3925902714 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f1584012240 con 0x55d54ce51b80 2024-04-23T10:25:36.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.962+0000 7f15a1c0a740 1 -- 172.21.15.97:0/3925902714 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f158c010070 con 0x55d54ce51b80 2024-04-23T10:25:36.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.962+0000 7f1577fff640 1 -- 172.21.15.97:0/3925902714 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f158400daf0 con 0x55d54ce51b80 2024-04-23T10:25:36.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.962+0000 7f15a1c0a740 10 client.15084 fetch_fsmap learned FSMap version 25 2024-04-23T10:25:36.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.962+0000 7f15a1c0a740 1 -- 172.21.15.97:0/3925902714 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d54cecf3b0 con 0x55d54ce51b80 2024-04-23T10:25:36.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.962+0000 7f1577fff640 1 -- 172.21.15.97:0/3925902714 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f158c010070 con 0x55d54ce51b80 2024-04-23T10:25:36.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.962+0000 7f1577fff640 1 -- 172.21.15.97:0/3925902714 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f1584004f20 con 0x55d54ce51b80 2024-04-23T10:25:36.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.962+0000 7f15a1c0a740 10 client.15084 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T10:25:36.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.962+0000 7f15a1c0a740 10 client.15084 resolve_mds: resolved name 'k' to daemon mds.k 2024-04-23T10:25:36.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.962+0000 7f15a1c0a740 20 client.15084 populate_metadata read hostname 'smithi097' 2024-04-23T10:25:36.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.962+0000 7f15a1c0a740 1 --2- 172.21.15.97:0/3925902714 >> [v2:172.21.15.183:6832/4292847121,v1:172.21.15.183:6833/4292847121] conn(0x55d54ced3cc0 0x55d54ced29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:36.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.962+0000 7f15a1c0a740 4 client.15084 mds_command: new command op to 24398 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T10:25:36.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.962+0000 7f15a1c0a740 1 -- 172.21.15.97:0/3925902714 --> [v2:172.21.15.183:6832/4292847121,v1:172.21.15.183:6833/4292847121] -- command(tid 0: {"prefix": "status"}) -- 0x55d54cdbe4e0 con 0x55d54ced3cc0 2024-04-23T10:25:36.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.962+0000 7f159cbaf640 1 --2- 172.21.15.97:0/3925902714 >> [v2:172.21.15.183:6832/4292847121,v1:172.21.15.183:6833/4292847121] conn(0x55d54ced3cc0 0x55d54ced29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:36.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.963+0000 7f1577fff640 10 client.15084 ms_handle_connect on v2:172.21.15.183:6832/4292847121 2024-04-23T10:25:36.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.963+0000 7f159cbaf640 1 --2- 172.21.15.97:0/3925902714 >> [v2:172.21.15.183:6832/4292847121,v1:172.21.15.183:6833/4292847121] conn(0x55d54ced3cc0 0x55d54ced29d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6af7d94ebfd9b228 server_cookie=ddc8b0ce4e803b2d in_seq=0 out_seq=0 2024-04-23T10:25:36.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.963+0000 7f1577fff640 1 -- 172.21.15.97:0/3925902714 <== mds.4 v2:172.21.15.183:6832/4292847121 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55d54cdbe4e0 con 0x55d54ced3cc0 2024-04-23T10:25:36.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.963+0000 7f1577fff640 10 client.15084 handle_command_reply: tid=0 2024-04-23T10:25:36.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.963+0000 7f15a1c0a740 1 client.15084 shutdown 2024-04-23T10:25:36.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.963+0000 7f15a1c0a740 1 -- 172.21.15.97:0/3925902714 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d54ce51b80 msgr2=0x55d54ce5ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:36.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.963+0000 7f15a1c0a740 1 --2- 172.21.15.97:0/3925902714 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d54ce51b80 0x55d54ce5ef60 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f15840127a0 tx=0x7f1584005ca0 comp rx=0 tx=0).stop 2024-04-23T10:25:36.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.963+0000 7f15a1c0a740 1 -- 172.21.15.97:0/3925902714 shutdown_connections 2024-04-23T10:25:36.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.963+0000 7f15a1c0a740 1 --2- 172.21.15.97:0/3925902714 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d54ce53710 0x55d54ce62470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:36.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.963+0000 7f15a1c0a740 1 --2- 172.21.15.97:0/3925902714 >> [v2:172.21.15.183:6832/4292847121,v1:172.21.15.183:6833/4292847121] conn(0x55d54ced3cc0 0x55d54ced29d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:36.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.963+0000 7f15a1c0a740 1 --2- 172.21.15.97:0/3925902714 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d54ce52d50 0x55d54ce5f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:36.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.963+0000 7f15a1c0a740 1 --2- 172.21.15.97:0/3925902714 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d54ce51b80 0x55d54ce5ef60 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:36.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.963+0000 7f1577fff640 0 client.15084 ms_handle_reset on v2:172.21.15.183:6832/4292847121 2024-04-23T10:25:36.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.963+0000 7f15a1c0a740 1 -- 172.21.15.97:0/3925902714 >> 172.21.15.97:0/3925902714 conn(0x55d54ce4d070 msgr2=0x55d54ce4b940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:36.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.963+0000 7f15a1c0a740 1 -- 172.21.15.97:0/3925902714 shutdown_connections 2024-04-23T10:25:36.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.963+0000 7f15a1c0a740 1 -- 172.21.15.97:0/3925902714 wait complete. 2024-04-23T10:25:36.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.963+0000 7f15a1c0a740 20 client.15084 trim_cache size 0 max 16384 2024-04-23T10:25:36.970 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-23T10:25:36.970 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "9b22af88-0159-11ef-bc93-c7b262605968", 2024-04-23T10:25:36.970 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 4, 2024-04-23T10:25:36.970 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 24398, 2024-04-23T10:25:36.970 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-04-23T10:25:36.970 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-04-23T10:25:36.970 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-04-23T10:25:36.970 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 84.998253235000007, 2024-04-23T10:25:36.970 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 25, 2024-04-23T10:25:36.970 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 88, 2024-04-23T10:25:36.970 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 86, 2024-04-23T10:25:36.970 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 124.540098449 2024-04-23T10:25:36.970 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-23T10:25:36.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.964+0000 7f159ee13640 1 -- 172.21.15.97:0/3629944595 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f15880a3bb0 msgr2=0x7f15880a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:36.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.964+0000 7f159ee13640 1 --2- 172.21.15.97:0/3629944595 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f15880a3bb0 0x7f15880a6070 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f15900ba4a0 tx=0x7f15900e9150 comp rx=0 tx=0).stop 2024-04-23T10:25:36.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.964+0000 7f159ee13640 1 -- 172.21.15.97:0/3629944595 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1598103b40 msgr2=0x7f159819f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:36.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.964+0000 7f159ee13640 1 --2- 172.21.15.97:0/3629944595 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1598103b40 0x7f159819f620 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1590031a20 tx=0x7f1590003c20 comp rx=0 tx=0).stop 2024-04-23T10:25:36.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.964+0000 7f159ee13640 1 -- 172.21.15.97:0/3629944595 shutdown_connections 2024-04-23T10:25:36.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.964+0000 7f159ee13640 1 --2- 172.21.15.97:0/3629944595 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f15880a3bb0 0x7f15880a6070 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:36.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.964+0000 7f159ee13640 1 --2- 172.21.15.97:0/3629944595 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1598103b40 0x7f159819f620 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:36.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.964+0000 7f159ee13640 1 --2- 172.21.15.97:0/3629944595 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1598103180 0x7f15981985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:36.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.964+0000 7f159ee13640 1 --2- 172.21.15.97:0/3629944595 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1598101f90 0x7f1598198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:36.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.964+0000 7f159ee13640 1 -- 172.21.15.97:0/3629944595 >> 172.21.15.97:0/3629944595 conn(0x7f15980fd500 msgr2=0x7f15981063b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:36.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.964+0000 7f159ee13640 1 -- 172.21.15.97:0/3629944595 shutdown_connections 2024-04-23T10:25:36.972 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:36.964+0000 7f159ee13640 1 -- 172.21.15.97:0/3629944595 wait complete. 2024-04-23T10:25:36.975 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.l status 2024-04-23T10:25:37.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.074+0000 7f3382f07640 1 -- 172.21.15.97:0/196814684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f337c102040 msgr2=0x7f337c1024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:37.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.074+0000 7f3382f07640 1 --2- 172.21.15.97:0/196814684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f337c102040 0x7f337c1024c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3368009900 tx=0x7f336802a410 comp rx=0 tx=0).stop 2024-04-23T10:25:37.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.074+0000 7f3382f07640 1 -- 172.21.15.97:0/196814684 shutdown_connections 2024-04-23T10:25:37.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.074+0000 7f3382f07640 1 --2- 172.21.15.97:0/196814684 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f337c102a00 0x7f337c109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.074+0000 7f3382f07640 1 --2- 172.21.15.97:0/196814684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f337c102040 0x7f337c1024c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.074+0000 7f3382f07640 1 --2- 172.21.15.97:0/196814684 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f337c100e60 0x7f337c101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.074+0000 7f3382f07640 1 -- 172.21.15.97:0/196814684 >> 172.21.15.97:0/196814684 conn(0x7f337c0fc440 msgr2=0x7f337c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:37.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.074+0000 7f3382f07640 1 -- 172.21.15.97:0/196814684 shutdown_connections 2024-04-23T10:25:37.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.074+0000 7f3382f07640 1 -- 172.21.15.97:0/196814684 wait complete. 2024-04-23T10:25:37.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.074+0000 7f3382f07640 1 Processor -- start 2024-04-23T10:25:37.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.074+0000 7f3382f07640 1 -- start start 2024-04-23T10:25:37.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.075+0000 7f3382f07640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f337c100e60 0x7f337c1981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.075+0000 7f3382f07640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f337c102040 0x7f337c198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.075+0000 7f3382f07640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f337c102a00 0x7f337c19f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.075+0000 7f3382f07640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f337c10ca10 con 0x7f337c102a00 2024-04-23T10:25:37.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.075+0000 7f3382f07640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f337c10c890 con 0x7f337c100e60 2024-04-23T10:25:37.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.075+0000 7f3382f07640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f337c10cb90 con 0x7f337c102040 2024-04-23T10:25:37.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.075+0000 7f33814a4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f337c102a00 0x7f337c19f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:37.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.075+0000 7f33814a4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f337c102a00 0x7f337c19f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59756/0 (socket says 172.21.15.97:59756) 2024-04-23T10:25:37.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.075+0000 7f33814a4640 1 -- 172.21.15.97:0/392778462 learned_addr learned my addr 172.21.15.97:0/392778462 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:25:37.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.075+0000 7f3380ca3640 1 --2- 172.21.15.97:0/392778462 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f337c100e60 0x7f337c1981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:37.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.075+0000 7f337bfff640 1 --2- 172.21.15.97:0/392778462 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f337c102040 0x7f337c198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:37.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.075+0000 7f33814a4640 1 -- 172.21.15.97:0/392778462 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f337c102040 msgr2=0x7f337c198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:37.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.075+0000 7f33814a4640 1 --2- 172.21.15.97:0/392778462 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f337c102040 0x7f337c198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.075+0000 7f33814a4640 1 -- 172.21.15.97:0/392778462 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f337c100e60 msgr2=0x7f337c1981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:37.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.075+0000 7f33814a4640 1 --2- 172.21.15.97:0/392778462 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f337c100e60 0x7f337c1981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.075+0000 7f33814a4640 1 -- 172.21.15.97:0/392778462 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f337c19fdf0 con 0x7f337c102a00 2024-04-23T10:25:37.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.075+0000 7f33814a4640 1 --2- 172.21.15.97:0/392778462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f337c102a00 0x7f337c19f780 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f337400eaa0 tx=0x7f337400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:37.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.075+0000 7f3379ffb640 1 -- 172.21.15.97:0/392778462 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f337400cdb0 con 0x7f337c102a00 2024-04-23T10:25:37.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.075+0000 7f3382f07640 1 -- 172.21.15.97:0/392778462 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f337c1a00e0 con 0x7f337c102a00 2024-04-23T10:25:37.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.075+0000 7f3379ffb640 1 -- 172.21.15.97:0/392778462 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3374005020 con 0x7f337c102a00 2024-04-23T10:25:37.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.075+0000 7f3379ffb640 1 -- 172.21.15.97:0/392778462 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33740116d0 con 0x7f337c102a00 2024-04-23T10:25:37.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.076+0000 7f3382f07640 1 -- 172.21.15.97:0/392778462 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f337c1a05e0 con 0x7f337c102a00 2024-04-23T10:25:37.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.077+0000 7f3379ffb640 1 -- 172.21.15.97:0/392778462 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f33740051d0 con 0x7f337c102a00 2024-04-23T10:25:37.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.078+0000 7f3385c1a740 1 Processor -- start 2024-04-23T10:25:37.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.078+0000 7f3379ffb640 1 --2- 172.21.15.97:0/392778462 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f336c0a3bb0 0x7f336c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.078+0000 7f3385c1a740 1 -- start start 2024-04-23T10:25:37.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.078+0000 7f337bfff640 1 --2- 172.21.15.97:0/392778462 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f336c0a3bb0 0x7f336c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:37.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.078+0000 7f3385c1a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598ffcb6f90 0x5598ffcb73b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.078+0000 7f3379ffb640 1 -- 172.21.15.97:0/392778462 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f33740d05d0 con 0x7f337c102a00 2024-04-23T10:25:37.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.078+0000 7f3385c1a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598ffcb8160 0x5598ffcb85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.078+0000 7f3385c1a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598ffcb8b20 0x5598ffcbf3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.078+0000 7f3385c1a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5598ffc17c50 con 0x5598ffcb8b20 2024-04-23T10:25:37.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.078+0000 7f3385c1a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5598ffbb4830 con 0x5598ffcb8160 2024-04-23T10:25:37.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.078+0000 7f3385c1a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5598ffc13c10 con 0x5598ffcb6f90 2024-04-23T10:25:37.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.078+0000 7f33814a4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598ffcb8b20 0x5598ffcbf3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:37.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.078+0000 7f33814a4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598ffcb8b20 0x5598ffcbf3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59764/0 (socket says 172.21.15.97:59764) 2024-04-23T10:25:37.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.078+0000 7f33814a4640 1 -- 172.21.15.97:0/2932700324 learned_addr learned my addr 172.21.15.97:0/2932700324 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:25:37.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.078+0000 7f337bfff640 1 --2- 172.21.15.97:0/2932700324 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598ffcb8160 0x5598ffcb85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:37.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.079+0000 7f3380ca3640 1 --2- 172.21.15.97:0/2932700324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598ffcb6f90 0x5598ffcb73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:37.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.079+0000 7f33814a4640 1 -- 172.21.15.97:0/2932700324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598ffcb6f90 msgr2=0x5598ffcb73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:25:37.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.079+0000 7f33814a4640 1 --2- 172.21.15.97:0/2932700324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598ffcb6f90 0x5598ffcb73b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.079+0000 7f33814a4640 1 -- 172.21.15.97:0/2932700324 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598ffcb8160 msgr2=0x5598ffcb85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:25:37.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.079+0000 7f33814a4640 1 --2- 172.21.15.97:0/2932700324 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598ffcb8160 0x5598ffcb85e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.079+0000 7f33814a4640 1 -- 172.21.15.97:0/2932700324 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598ff92a2d0 con 0x5598ffcb8b20 2024-04-23T10:25:37.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.079+0000 7f3380ca3640 1 --2- 172.21.15.97:0/2932700324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598ffcb6f90 0x5598ffcb73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:25:37.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.079+0000 7f337bfff640 1 --2- 172.21.15.97:0/392778462 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f336c0a3bb0 0x7f336c0a6070 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f336802a930 tx=0x7f3368032e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:37.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.079+0000 7f33814a4640 1 --2- 172.21.15.97:0/2932700324 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598ffcb8b20 0x5598ffcbf3b0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f337400efd0 tx=0x7f337409d120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b04c6f5e13180d33 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:37.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.079+0000 7f3366ffd640 1 -- 172.21.15.97:0/2932700324 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f337401f040 con 0x5598ffcb8b20 2024-04-23T10:25:37.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.079+0000 7f3366ffd640 1 -- 172.21.15.97:0/2932700324 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33740a5550 con 0x5598ffcb8b20 2024-04-23T10:25:37.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.079+0000 7f3366ffd640 1 -- 172.21.15.97:0/2932700324 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3374087320 con 0x5598ffcb8b20 2024-04-23T10:25:37.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.080+0000 7f3385c1a740 1 -- 172.21.15.97:0/2932700324 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598ffcb8b20 msgr2=0x5598ffcbf3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:37.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.080+0000 7f3385c1a740 1 --2- 172.21.15.97:0/2932700324 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598ffcb8b20 0x5598ffcbf3b0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f337400efd0 tx=0x7f337409d120 comp rx=0 tx=0).stop 2024-04-23T10:25:37.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.080+0000 7f3385c1a740 1 -- 172.21.15.97:0/2932700324 shutdown_connections 2024-04-23T10:25:37.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.080+0000 7f3385c1a740 1 --2- 172.21.15.97:0/2932700324 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598ffcb8b20 0x5598ffcbf3b0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.080+0000 7f3385c1a740 1 --2- 172.21.15.97:0/2932700324 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598ffcb8160 0x5598ffcb85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.080+0000 7f3385c1a740 1 --2- 172.21.15.97:0/2932700324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598ffcb6f90 0x5598ffcb73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.080+0000 7f3385c1a740 1 -- 172.21.15.97:0/2932700324 >> 172.21.15.97:0/2932700324 conn(0x5598ffcb2480 msgr2=0x5598ffcb2880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:37.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.080+0000 7f3385c1a740 1 -- 172.21.15.97:0/2932700324 shutdown_connections 2024-04-23T10:25:37.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.080+0000 7f3385c1a740 1 -- 172.21.15.97:0/2932700324 wait complete. 2024-04-23T10:25:37.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.080+0000 7f3385c1a740 1 -- start start 2024-04-23T10:25:37.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.080+0000 7f3385c1a740 1 Processor -- start 2024-04-23T10:25:37.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.080+0000 7f3385c1a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598ffcb6f90 0x5598ffcc1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.080+0000 7f3385c1a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598ffcb8160 0x5598ffcc19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.080+0000 7f3385c1a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598ffcb8b20 0x5598ffcc89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.080+0000 7f3385c1a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5598ffcbfad0 con 0x5598ffcb8b20 2024-04-23T10:25:37.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.080+0000 7f3385c1a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5598ffbb2470 con 0x5598ffcb6f90 2024-04-23T10:25:37.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.080+0000 7f3385c1a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5598ffcbfc50 con 0x5598ffcb8160 2024-04-23T10:25:37.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.080+0000 7f3385c1a740 10 client.0 _ll_register_callbacks cb 0x5598ffbd7f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:25:37.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.080+0000 7f33814a4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598ffcb8b20 0x5598ffcc89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:37.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.080+0000 7f3380ca3640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598ffcb6f90 0x5598ffcc1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:37.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.080+0000 7f33814a4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598ffcb8b20 0x5598ffcc89e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59772/0 (socket says 172.21.15.97:59772) 2024-04-23T10:25:37.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.081+0000 7f33814a4640 1 -- 172.21.15.97:0/2566408427 learned_addr learned my addr 172.21.15.97:0/2566408427 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:25:37.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.081+0000 7f337bfff640 1 --2- 172.21.15.97:0/2566408427 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598ffcb8160 0x5598ffcc19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:37.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.081+0000 7f33814a4640 1 -- 172.21.15.97:0/2566408427 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598ffcb8160 msgr2=0x5598ffcc19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:25:37.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.081+0000 7f33814a4640 1 --2- 172.21.15.97:0/2566408427 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598ffcb8160 0x5598ffcc19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.081+0000 7f33814a4640 1 -- 172.21.15.97:0/2566408427 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598ffcb6f90 msgr2=0x5598ffcc1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:25:37.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.081+0000 7f33814a4640 1 --2- 172.21.15.97:0/2566408427 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598ffcb6f90 0x5598ffcc1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.081+0000 7f33814a4640 1 -- 172.21.15.97:0/2566408427 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598ff9f46b0 con 0x5598ffcb8b20 2024-04-23T10:25:37.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.081+0000 7f3380ca3640 1 --2- 172.21.15.97:0/2566408427 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598ffcb6f90 0x5598ffcc1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T10:25:37.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.081+0000 7f33814a4640 1 --2- 172.21.15.97:0/2566408427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598ffcb8b20 0x5598ffcc89e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f337409c5e0 tx=0x7f33740984d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d76a1d4191c9fee server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:37.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.081+0000 7f335bfff640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:25:37.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.081+0000 7f335bfff640 1 -- 172.21.15.97:0/2566408427 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f337401f070 con 0x5598ffcb8b20 2024-04-23T10:25:37.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.081+0000 7f335bfff640 1 -- 172.21.15.97:0/2566408427 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f33740d0110 con 0x5598ffcb8b20 2024-04-23T10:25:37.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.081+0000 7f3385c1a740 1 -- 172.21.15.97:0/2566408427 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33740a5550 con 0x5598ffcb8b20 2024-04-23T10:25:37.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.081+0000 7f335bfff640 1 -- 172.21.15.97:0/2566408427 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f337409aca0 con 0x5598ffcb8b20 2024-04-23T10:25:37.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.081+0000 7f3385c1a740 10 client.15096 fetch_fsmap learned FSMap version 25 2024-04-23T10:25:37.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.081+0000 7f3385c1a740 1 -- 172.21.15.97:0/2566408427 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5598ffccbcc0 con 0x5598ffcb8b20 2024-04-23T10:25:37.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.081+0000 7f335bfff640 1 -- 172.21.15.97:0/2566408427 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33740a5550 con 0x5598ffcb8b20 2024-04-23T10:25:37.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.082+0000 7f335bfff640 1 -- 172.21.15.97:0/2566408427 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f337400f040 con 0x5598ffcb8b20 2024-04-23T10:25:37.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.082+0000 7f3385c1a740 10 client.15096 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T10:25:37.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.082+0000 7f3385c1a740 10 client.15096 resolve_mds: resolved name 'l' to daemon mds.l 2024-04-23T10:25:37.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.082+0000 7f3385c1a740 20 client.15096 populate_metadata read hostname 'smithi097' 2024-04-23T10:25:37.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.082+0000 7f3385c1a740 1 --2- 172.21.15.97:0/2566408427 >> [v2:172.21.15.196:6832/3885816565,v1:172.21.15.196:6833/3885816565] conn(0x5598ffccde00 0x5598ffccf270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.082+0000 7f3385c1a740 4 client.15096 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:25:37.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.082+0000 7f3385c1a740 1 -- 172.21.15.97:0/2566408427 --> [v2:172.21.15.196:6832/3885816565,v1:172.21.15.196:6833/3885816565] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5598ffc25670 con 0x5598ffccde00 2024-04-23T10:25:37.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.082+0000 7f3380ca3640 1 --2- 172.21.15.97:0/2566408427 >> [v2:172.21.15.196:6832/3885816565,v1:172.21.15.196:6833/3885816565] conn(0x5598ffccde00 0x5598ffccf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:37.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.082+0000 7f335bfff640 10 client.15096 ms_handle_connect on v2:172.21.15.196:6832/3885816565 2024-04-23T10:25:37.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.082+0000 7f3380ca3640 1 --2- 172.21.15.97:0/2566408427 >> [v2:172.21.15.196:6832/3885816565,v1:172.21.15.196:6833/3885816565] conn(0x5598ffccde00 0x5598ffccf270 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d8be6ed719c77df6 server_cookie=c963c428443a8aa0 in_seq=0 out_seq=0 2024-04-23T10:25:37.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.083+0000 7f335bfff640 1 -- 172.21.15.97:0/2566408427 <== mds.3 v2:172.21.15.196:6832/3885816565 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5598ffc25670 con 0x5598ffccde00 2024-04-23T10:25:37.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.083+0000 7f335bfff640 10 client.15096 handle_command_reply: tid=0 2024-04-23T10:25:37.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.083+0000 7f3385c1a740 1 client.15096 shutdown 2024-04-23T10:25:37.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.083+0000 7f3385c1a740 1 -- 172.21.15.97:0/2566408427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598ffcb8b20 msgr2=0x5598ffcc89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:37.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.083+0000 7f3385c1a740 1 --2- 172.21.15.97:0/2566408427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598ffcb8b20 0x5598ffcc89e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f337409c5e0 tx=0x7f33740984d0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.083+0000 7f3385c1a740 1 -- 172.21.15.97:0/2566408427 shutdown_connections 2024-04-23T10:25:37.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.083+0000 7f3385c1a740 1 --2- 172.21.15.97:0/2566408427 >> [v2:172.21.15.196:6832/3885816565,v1:172.21.15.196:6833/3885816565] conn(0x5598ffccde00 0x5598ffccf270 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.083+0000 7f3385c1a740 1 --2- 172.21.15.97:0/2566408427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598ffcb8b20 0x5598ffcc89e0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.083+0000 7f3385c1a740 1 --2- 172.21.15.97:0/2566408427 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598ffcb8160 0x5598ffcc19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.083+0000 7f335bfff640 0 client.15096 ms_handle_reset on v2:172.21.15.196:6832/3885816565 2024-04-23T10:25:37.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.083+0000 7f3385c1a740 1 --2- 172.21.15.97:0/2566408427 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598ffcb6f90 0x5598ffcc1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.083+0000 7f3385c1a740 1 -- 172.21.15.97:0/2566408427 >> 172.21.15.97:0/2566408427 conn(0x5598ffcb2480 msgr2=0x5598ffcad1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:37.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.084+0000 7f3385c1a740 1 -- 172.21.15.97:0/2566408427 shutdown_connections 2024-04-23T10:25:37.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.084+0000 7f3385c1a740 1 -- 172.21.15.97:0/2566408427 wait complete. 2024-04-23T10:25:37.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.084+0000 7f3385c1a740 20 client.15096 trim_cache size 0 max 16384 2024-04-23T10:25:37.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:36 smithi196 ceph-mon[32919]: pgmap v336: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 11 op/s 2024-04-23T10:25:37.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:36 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/2193923269' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T10:25:37.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:36 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/3355436790' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:25:37.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:36 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/109957840' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T10:25:37.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.102+0000 7f3385c1a740 1 Processor -- start 2024-04-23T10:25:37.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.102+0000 7f3385c1a740 1 -- start start 2024-04-23T10:25:37.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.102+0000 7f3385c1a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598ffcb6f90 0x5598ffcc0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.102+0000 7f3385c1a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598ffcb8160 0x5598ffcc8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.102+0000 7f3385c1a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598ffcb8b20 0x5598ffcbbd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.102+0000 7f3385c1a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5598ffccd5b0 con 0x5598ffcb8160 2024-04-23T10:25:37.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.102+0000 7f3385c1a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5598ffccdc80 con 0x5598ffcb8b20 2024-04-23T10:25:37.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.102+0000 7f3385c1a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5598ffcce350 con 0x5598ffcb6f90 2024-04-23T10:25:37.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.102+0000 7f337bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598ffcb8160 0x5598ffcc8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:37.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.102+0000 7f337bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598ffcb8160 0x5598ffcc8900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59788/0 (socket says 172.21.15.97:59788) 2024-04-23T10:25:37.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.102+0000 7f337bfff640 1 -- 172.21.15.97:0/3889801977 learned_addr learned my addr 172.21.15.97:0/3889801977 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:25:37.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.102+0000 7f3380ca3640 1 --2- 172.21.15.97:0/3889801977 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598ffcb6f90 0x5598ffcc0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:37.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.102+0000 7f33814a4640 1 --2- 172.21.15.97:0/3889801977 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598ffcb8b20 0x5598ffcbbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:37.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.102+0000 7f337bfff640 1 -- 172.21.15.97:0/3889801977 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598ffcb6f90 msgr2=0x5598ffcc0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:25:37.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.102+0000 7f337bfff640 1 --2- 172.21.15.97:0/3889801977 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598ffcb6f90 0x5598ffcc0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.102+0000 7f337bfff640 1 -- 172.21.15.97:0/3889801977 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598ffcb8b20 msgr2=0x5598ffcbbd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:25:37.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.102+0000 7f337bfff640 1 --2- 172.21.15.97:0/3889801977 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598ffcb8b20 0x5598ffcbbd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.102+0000 7f337bfff640 1 -- 172.21.15.97:0/3889801977 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598ffccf5a0 con 0x5598ffcb8160 2024-04-23T10:25:37.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.102+0000 7f337bfff640 1 --2- 172.21.15.97:0/3889801977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598ffcb8160 0x5598ffcc8900 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3368009230 tx=0x7f3368042000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18d055fa48386504 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:37.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.102+0000 7f33667fc640 1 -- 172.21.15.97:0/3889801977 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3368035050 con 0x5598ffcb8160 2024-04-23T10:25:37.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.102+0000 7f33667fc640 1 -- 172.21.15.97:0/3889801977 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33680054e0 con 0x5598ffcb8160 2024-04-23T10:25:37.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.102+0000 7f33667fc640 1 -- 172.21.15.97:0/3889801977 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3368004c20 con 0x5598ffcb8160 2024-04-23T10:25:37.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.103+0000 7f3385c1a740 1 -- 172.21.15.97:0/3889801977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598ffcb8160 msgr2=0x5598ffcc8900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:37.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.103+0000 7f3385c1a740 1 --2- 172.21.15.97:0/3889801977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598ffcb8160 0x5598ffcc8900 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3368009230 tx=0x7f3368042000 comp rx=0 tx=0).stop 2024-04-23T10:25:37.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.103+0000 7f3385c1a740 1 -- 172.21.15.97:0/3889801977 shutdown_connections 2024-04-23T10:25:37.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.103+0000 7f3385c1a740 1 --2- 172.21.15.97:0/3889801977 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598ffcb8b20 0x5598ffcbbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.103+0000 7f3385c1a740 1 --2- 172.21.15.97:0/3889801977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598ffcb8160 0x5598ffcc8900 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.103+0000 7f3385c1a740 1 --2- 172.21.15.97:0/3889801977 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598ffcb6f90 0x5598ffcc0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.103+0000 7f3385c1a740 1 -- 172.21.15.97:0/3889801977 >> 172.21.15.97:0/3889801977 conn(0x5598ffcb2480 msgr2=0x5598ffca90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:37.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.103+0000 7f3385c1a740 1 -- 172.21.15.97:0/3889801977 shutdown_connections 2024-04-23T10:25:37.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.103+0000 7f3385c1a740 1 -- 172.21.15.97:0/3889801977 wait complete. 2024-04-23T10:25:37.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.103+0000 7f3385c1a740 1 -- start start 2024-04-23T10:25:37.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.103+0000 7f3385c1a740 1 Processor -- start 2024-04-23T10:25:37.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.104+0000 7f3385c1a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598ffcb6f90 0x5598ffcc48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.104+0000 7f3385c1a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598ffcb8160 0x5598ffcc6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.104+0000 7f3385c1a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598ffcb8b20 0x5598ffcc7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.104+0000 7f3385c1a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5598ffccf0f0 con 0x5598ffcb6f90 2024-04-23T10:25:37.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.104+0000 7f3385c1a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5598ffccea20 con 0x5598ffcb8160 2024-04-23T10:25:37.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.104+0000 7f3385c1a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5598ffccf7c0 con 0x5598ffcb8b20 2024-04-23T10:25:37.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.104+0000 7f3385c1a740 10 client.0 _ll_register_callbacks cb 0x7f3368009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:25:37.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.104+0000 7f3380ca3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598ffcb6f90 0x5598ffcc48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:37.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.104+0000 7f3380ca3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598ffcb6f90 0x5598ffcc48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59798/0 (socket says 172.21.15.97:59798) 2024-04-23T10:25:37.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.104+0000 7f3380ca3640 1 -- 172.21.15.97:0/3805088472 learned_addr learned my addr 172.21.15.97:0/3805088472 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:25:37.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.104+0000 7f33814a4640 1 --2- 172.21.15.97:0/3805088472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598ffcb8b20 0x5598ffcc7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:37.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.104+0000 7f337bfff640 1 --2- 172.21.15.97:0/3805088472 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598ffcb8160 0x5598ffcc6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:37.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.104+0000 7f3380ca3640 1 -- 172.21.15.97:0/3805088472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598ffcb8b20 msgr2=0x5598ffcc7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:25:37.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.104+0000 7f3380ca3640 1 --2- 172.21.15.97:0/3805088472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598ffcb8b20 0x5598ffcc7370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.104+0000 7f3380ca3640 1 -- 172.21.15.97:0/3805088472 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598ffcb8160 msgr2=0x5598ffcc6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:25:37.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.104+0000 7f3380ca3640 1 --2- 172.21.15.97:0/3805088472 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598ffcb8160 0x5598ffcc6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.104+0000 7f3380ca3640 1 -- 172.21.15.97:0/3805088472 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598ffcc78e0 con 0x5598ffcb6f90 2024-04-23T10:25:37.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.104+0000 7f3380ca3640 1 --2- 172.21.15.97:0/3805088472 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598ffcb6f90 0x5598ffcc48e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3370004700 tx=0x7f337001e880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74a2a1346f82368f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:37.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.104+0000 7f335bfff640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:25:37.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.104+0000 7f335bfff640 1 -- 172.21.15.97:0/3805088472 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3370012dd0 con 0x5598ffcb6f90 2024-04-23T10:25:37.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.104+0000 7f335bfff640 1 -- 172.21.15.97:0/3805088472 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f3370005020 con 0x5598ffcb6f90 2024-04-23T10:25:37.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.104+0000 7f3385c1a740 1 -- 172.21.15.97:0/3805088472 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33680054e0 con 0x5598ffcb6f90 2024-04-23T10:25:37.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.105+0000 7f335bfff640 1 -- 172.21.15.97:0/3805088472 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3370005220 con 0x5598ffcb6f90 2024-04-23T10:25:37.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.105+0000 7f3385c1a740 10 client.15102 fetch_fsmap learned FSMap version 25 2024-04-23T10:25:37.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.105+0000 7f3385c1a740 1 -- 172.21.15.97:0/3805088472 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5598ffcc8cb0 con 0x5598ffcb6f90 2024-04-23T10:25:37.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.105+0000 7f335bfff640 1 -- 172.21.15.97:0/3805088472 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33680054e0 con 0x5598ffcb6f90 2024-04-23T10:25:37.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.105+0000 7f335bfff640 1 -- 172.21.15.97:0/3805088472 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f3370025030 con 0x5598ffcb6f90 2024-04-23T10:25:37.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.105+0000 7f3385c1a740 10 client.15102 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T10:25:37.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.105+0000 7f3385c1a740 10 client.15102 resolve_mds: resolved name 'l' to daemon mds.l 2024-04-23T10:25:37.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.105+0000 7f3385c1a740 20 client.15102 populate_metadata read hostname 'smithi097' 2024-04-23T10:25:37.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.105+0000 7f3385c1a740 1 --2- 172.21.15.97:0/3805088472 >> [v2:172.21.15.196:6832/3885816565,v1:172.21.15.196:6833/3885816565] conn(0x5598ffd38690 0x5598ffd373a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.105+0000 7f3385c1a740 4 client.15102 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T10:25:37.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.105+0000 7f3385c1a740 1 -- 172.21.15.97:0/3805088472 --> [v2:172.21.15.196:6832/3885816565,v1:172.21.15.196:6833/3885816565] -- command(tid 0: {"prefix": "status"}) -- 0x5598ffc1f4e0 con 0x5598ffd38690 2024-04-23T10:25:37.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.105+0000 7f337bfff640 1 --2- 172.21.15.97:0/3805088472 >> [v2:172.21.15.196:6832/3885816565,v1:172.21.15.196:6833/3885816565] conn(0x5598ffd38690 0x5598ffd373a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:37.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.106+0000 7f337bfff640 1 --2- 172.21.15.97:0/3805088472 >> [v2:172.21.15.196:6832/3885816565,v1:172.21.15.196:6833/3885816565] conn(0x5598ffd38690 0x5598ffd373a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ab8a6196e5f337cc server_cookie=71b655daaca55d8b in_seq=0 out_seq=0 2024-04-23T10:25:37.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.106+0000 7f335bfff640 10 client.15102 ms_handle_connect on v2:172.21.15.196:6832/3885816565 2024-04-23T10:25:37.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.106+0000 7f335bfff640 1 -- 172.21.15.97:0/3805088472 <== mds.3 v2:172.21.15.196:6832/3885816565 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5598ffc1f4e0 con 0x5598ffd38690 2024-04-23T10:25:37.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.106+0000 7f335bfff640 10 client.15102 handle_command_reply: tid=0 2024-04-23T10:25:37.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.106+0000 7f3385c1a740 1 client.15102 shutdown 2024-04-23T10:25:37.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.106+0000 7f3385c1a740 1 -- 172.21.15.97:0/3805088472 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598ffcb6f90 msgr2=0x5598ffcc48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:37.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.106+0000 7f3385c1a740 1 --2- 172.21.15.97:0/3805088472 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598ffcb6f90 0x5598ffcc48e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3370004700 tx=0x7f337001e880 comp rx=0 tx=0).stop 2024-04-23T10:25:37.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.106+0000 7f3385c1a740 1 -- 172.21.15.97:0/3805088472 shutdown_connections 2024-04-23T10:25:37.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.106+0000 7f3385c1a740 1 --2- 172.21.15.97:0/3805088472 >> [v2:172.21.15.196:6832/3885816565,v1:172.21.15.196:6833/3885816565] conn(0x5598ffd38690 0x5598ffd373a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.106+0000 7f3385c1a740 1 --2- 172.21.15.97:0/3805088472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5598ffcb8b20 0x5598ffcc7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.106+0000 7f335bfff640 0 client.15102 ms_handle_reset on v2:172.21.15.196:6832/3885816565 2024-04-23T10:25:37.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.106+0000 7f3385c1a740 1 --2- 172.21.15.97:0/3805088472 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598ffcb8160 0x5598ffcc6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.106+0000 7f3385c1a740 1 --2- 172.21.15.97:0/3805088472 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598ffcb6f90 0x5598ffcc48e0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.106+0000 7f3385c1a740 1 -- 172.21.15.97:0/3805088472 >> 172.21.15.97:0/3805088472 conn(0x5598ffcb2480 msgr2=0x5598ffcbbd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:37.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.106+0000 7f3385c1a740 1 -- 172.21.15.97:0/3805088472 shutdown_connections 2024-04-23T10:25:37.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.106+0000 7f3385c1a740 1 -- 172.21.15.97:0/3805088472 wait complete. 2024-04-23T10:25:37.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.106+0000 7f3385c1a740 20 client.15102 trim_cache size 0 max 16384 2024-04-23T10:25:37.113 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-23T10:25:37.113 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "9b22af88-0159-11ef-bc93-c7b262605968", 2024-04-23T10:25:37.113 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 3, 2024-04-23T10:25:37.113 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 24403, 2024-04-23T10:25:37.113 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-04-23T10:25:37.113 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-04-23T10:25:37.113 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-04-23T10:25:37.113 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 86.154752131999999, 2024-04-23T10:25:37.113 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 25, 2024-04-23T10:25:37.113 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 88, 2024-04-23T10:25:37.113 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 86, 2024-04-23T10:25:37.114 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 127.669278573 2024-04-23T10:25:37.114 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-23T10:25:37.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.107+0000 7f3382f07640 1 -- 172.21.15.97:0/392778462 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f336c0a3bb0 msgr2=0x7f336c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:37.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.107+0000 7f3382f07640 1 --2- 172.21.15.97:0/392778462 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f336c0a3bb0 0x7f336c0a6070 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f336802a930 tx=0x7f3368032e40 comp rx=0 tx=0).stop 2024-04-23T10:25:37.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.107+0000 7f3382f07640 1 -- 172.21.15.97:0/392778462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f337c102a00 msgr2=0x7f337c19f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:37.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.107+0000 7f3382f07640 1 --2- 172.21.15.97:0/392778462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f337c102a00 0x7f337c19f780 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f337400eaa0 tx=0x7f337400ef60 comp rx=0 tx=0).stop 2024-04-23T10:25:37.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.107+0000 7f3382f07640 1 -- 172.21.15.97:0/392778462 shutdown_connections 2024-04-23T10:25:37.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.107+0000 7f3382f07640 1 --2- 172.21.15.97:0/392778462 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f336c0a3bb0 0x7f336c0a6070 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.107+0000 7f3382f07640 1 --2- 172.21.15.97:0/392778462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f337c102a00 0x7f337c19f780 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.107+0000 7f3382f07640 1 --2- 172.21.15.97:0/392778462 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f337c102040 0x7f337c198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.107+0000 7f3382f07640 1 --2- 172.21.15.97:0/392778462 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f337c100e60 0x7f337c1981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.107+0000 7f3382f07640 1 -- 172.21.15.97:0/392778462 >> 172.21.15.97:0/392778462 conn(0x7f337c0fc440 msgr2=0x7f337c06a570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:37.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.107+0000 7f3382f07640 1 -- 172.21.15.97:0/392778462 shutdown_connections 2024-04-23T10:25:37.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.107+0000 7f3382f07640 1 -- 172.21.15.97:0/392778462 wait complete. 2024-04-23T10:25:37.118 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-23T10:25:37.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:36 smithi183 ceph-mon[32317]: pgmap v336: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 11 op/s 2024-04-23T10:25:37.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:36 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/2193923269' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T10:25:37.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:36 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/3355436790' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:25:37.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:36 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/109957840' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T10:25:37.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.217+0000 7faa06dfe640 1 -- 172.21.15.97:0/979342968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa00101b30 msgr2=0x7faa001093d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:37.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.217+0000 7faa06dfe640 1 --2- 172.21.15.97:0/979342968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa00101b30 0x7faa001093d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0009900 tx=0x7fa9f002a410 comp rx=0 tx=0).stop 2024-04-23T10:25:37.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.217+0000 7faa06dfe640 1 -- 172.21.15.97:0/979342968 shutdown_connections 2024-04-23T10:25:37.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.217+0000 7faa06dfe640 1 --2- 172.21.15.97:0/979342968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa00101b30 0x7faa001093d0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.217+0000 7faa06dfe640 1 --2- 172.21.15.97:0/979342968 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa00101170 0x7faa001015f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.217+0000 7faa06dfe640 1 --2- 172.21.15.97:0/979342968 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa00102440 0x7faa00102860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.217+0000 7faa06dfe640 1 -- 172.21.15.97:0/979342968 >> 172.21.15.97:0/979342968 conn(0x7faa000fbb80 msgr2=0x7faa000fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:37.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.217+0000 7faa06dfe640 1 -- 172.21.15.97:0/979342968 shutdown_connections 2024-04-23T10:25:37.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.217+0000 7faa06dfe640 1 -- 172.21.15.97:0/979342968 wait complete. 2024-04-23T10:25:37.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.217+0000 7faa06dfe640 1 Processor -- start 2024-04-23T10:25:37.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.217+0000 7faa06dfe640 1 -- start start 2024-04-23T10:25:37.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.217+0000 7faa06dfe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa00101170 0x7faa00198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.217+0000 7faa06dfe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa00101b30 0x7faa00198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.217+0000 7faa06dfe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa00102440 0x7faa0019f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.217+0000 7faa06dfe640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7faa0010db10 con 0x7faa00102440 2024-04-23T10:25:37.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.217+0000 7faa06dfe640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7faa0010d990 con 0x7faa00101170 2024-04-23T10:25:37.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.217+0000 7faa06dfe640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7faa0010dc90 con 0x7faa00101b30 2024-04-23T10:25:37.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.218+0000 7faa0539b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa00102440 0x7faa0019f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:37.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.218+0000 7faa0539b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa00102440 0x7faa0019f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59826/0 (socket says 172.21.15.97:59826) 2024-04-23T10:25:37.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.218+0000 7faa0539b640 1 -- 172.21.15.97:0/1306237315 learned_addr learned my addr 172.21.15.97:0/1306237315 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:25:37.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.218+0000 7faa04b9a640 1 --2- 172.21.15.97:0/1306237315 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa00101170 0x7faa00198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:37.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.218+0000 7fa9fffff640 1 --2- 172.21.15.97:0/1306237315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa00101b30 0x7faa00198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:37.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.218+0000 7faa0539b640 1 -- 172.21.15.97:0/1306237315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa00101b30 msgr2=0x7faa00198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:37.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.218+0000 7faa0539b640 1 --2- 172.21.15.97:0/1306237315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa00101b30 0x7faa00198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.218+0000 7faa0539b640 1 -- 172.21.15.97:0/1306237315 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa00101170 msgr2=0x7faa00198220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:37.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.218+0000 7faa0539b640 1 --2- 172.21.15.97:0/1306237315 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa00101170 0x7faa00198220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.218+0000 7faa0539b640 1 -- 172.21.15.97:0/1306237315 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa0019fda0 con 0x7faa00102440 2024-04-23T10:25:37.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.218+0000 7fa9fffff640 1 --2- 172.21.15.97:0/1306237315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa00101b30 0x7faa00198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:25:37.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.218+0000 7faa0539b640 1 --2- 172.21.15.97:0/1306237315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa00102440 0x7faa0019f730 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0031500 tx=0x7fa9f0032f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:37.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.218+0000 7fa9fdffb640 1 -- 172.21.15.97:0/1306237315 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f003f030 con 0x7faa00102440 2024-04-23T10:25:37.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.218+0000 7fa9fdffb640 1 -- 172.21.15.97:0/1306237315 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9f0003aa0 con 0x7faa00102440 2024-04-23T10:25:37.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.218+0000 7fa9fdffb640 1 -- 172.21.15.97:0/1306237315 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f0035070 con 0x7faa00102440 2024-04-23T10:25:37.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.218+0000 7faa06dfe640 1 -- 172.21.15.97:0/1306237315 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa001a0030 con 0x7faa00102440 2024-04-23T10:25:37.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.218+0000 7faa06dfe640 1 -- 172.21.15.97:0/1306237315 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa001a0520 con 0x7faa00102440 2024-04-23T10:25:37.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.220+0000 7fa9fdffb640 1 -- 172.21.15.97:0/1306237315 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa9f0042030 con 0x7faa00102440 2024-04-23T10:25:37.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.221+0000 7faa09bed740 1 Processor -- start 2024-04-23T10:25:37.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.221+0000 7faa09bed740 1 -- start start 2024-04-23T10:25:37.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.221+0000 7faa09bed740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644fa9c4b80 0x5644fa9c4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.221+0000 7faa09bed740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644fa9c5d50 0x5644fa9c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.221+0000 7faa09bed740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644fa9c6710 0x5644fa9ccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.221+0000 7faa09bed740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5644fa929c50 con 0x5644fa9c4b80 2024-04-23T10:25:37.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.221+0000 7faa09bed740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5644fa8c6830 con 0x5644fa9c5d50 2024-04-23T10:25:37.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.221+0000 7faa09bed740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5644fa925c10 con 0x5644fa9c6710 2024-04-23T10:25:37.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.221+0000 7fa9fdffb640 1 --2- 172.21.15.97:0/1306237315 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fa9ec0a3bb0 0x7fa9ec0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.221+0000 7fa9fffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644fa9c4b80 0x5644fa9c4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:37.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.221+0000 7fa9fffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644fa9c4b80 0x5644fa9c4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59832/0 (socket says 172.21.15.97:59832) 2024-04-23T10:25:37.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.221+0000 7fa9fffff640 1 -- 172.21.15.97:0/340224896 learned_addr learned my addr 172.21.15.97:0/340224896 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:25:37.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.221+0000 7faa0539b640 1 --2- 172.21.15.97:0/1306237315 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fa9ec0a3bb0 0x7fa9ec0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:37.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.221+0000 7faa04b9a640 1 --2- 172.21.15.97:0/340224896 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644fa9c5d50 0x5644fa9c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:37.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.221+0000 7fa9fdffb640 1 -- 172.21.15.97:0/1306237315 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fa9f00f0990 con 0x7faa00102440 2024-04-23T10:25:37.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.221+0000 7fa9fffff640 1 -- 172.21.15.97:0/340224896 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644fa9c6710 msgr2=0x5644fa9ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:25:37.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.221+0000 7fa9fffff640 1 --2- 172.21.15.97:0/340224896 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644fa9c6710 0x5644fa9ccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.221+0000 7fa9fffff640 1 -- 172.21.15.97:0/340224896 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644fa9c5d50 msgr2=0x5644fa9c61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:25:37.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.221+0000 7fa9fffff640 1 --2- 172.21.15.97:0/340224896 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644fa9c5d50 0x5644fa9c61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.221+0000 7fa9fffff640 1 -- 172.21.15.97:0/340224896 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644fa63c2d0 con 0x5644fa9c4b80 2024-04-23T10:25:37.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.222+0000 7fa9fffff640 1 --2- 172.21.15.97:0/340224896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644fa9c4b80 0x5644fa9c4fa0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa9e800bdb0 tx=0x7fa9e8011750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3040064cb5bbac67 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:37.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.222+0000 7faa0539b640 1 --2- 172.21.15.97:0/1306237315 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fa9ec0a3bb0 0x7fa9ec0a6070 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa9f00bc9a0 tx=0x7fa9f00dd040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:37.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.222+0000 7fa9f6ffd640 1 -- 172.21.15.97:0/340224896 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e8011e60 con 0x5644fa9c4b80 2024-04-23T10:25:37.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.222+0000 7fa9f6ffd640 1 -- 172.21.15.97:0/340224896 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9e801aab0 con 0x5644fa9c4b80 2024-04-23T10:25:37.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.222+0000 7fa9f6ffd640 1 -- 172.21.15.97:0/340224896 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e8012ab0 con 0x5644fa9c4b80 2024-04-23T10:25:37.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.222+0000 7faa09bed740 1 -- 172.21.15.97:0/340224896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644fa9c4b80 msgr2=0x5644fa9c4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:37.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.222+0000 7faa09bed740 1 --2- 172.21.15.97:0/340224896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644fa9c4b80 0x5644fa9c4fa0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa9e800bdb0 tx=0x7fa9e8011750 comp rx=0 tx=0).stop 2024-04-23T10:25:37.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.222+0000 7faa09bed740 1 -- 172.21.15.97:0/340224896 shutdown_connections 2024-04-23T10:25:37.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.222+0000 7faa09bed740 1 --2- 172.21.15.97:0/340224896 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644fa9c6710 0x5644fa9ccfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.222+0000 7faa09bed740 1 --2- 172.21.15.97:0/340224896 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644fa9c5d50 0x5644fa9c61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.222+0000 7faa09bed740 1 --2- 172.21.15.97:0/340224896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644fa9c4b80 0x5644fa9c4fa0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.222+0000 7faa09bed740 1 -- 172.21.15.97:0/340224896 >> 172.21.15.97:0/340224896 conn(0x5644fa9c0070 msgr2=0x5644fa9c0470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:37.230 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.222+0000 7faa09bed740 1 -- 172.21.15.97:0/340224896 shutdown_connections 2024-04-23T10:25:37.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.222+0000 7faa09bed740 1 -- 172.21.15.97:0/340224896 wait complete. 2024-04-23T10:25:37.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.223+0000 7faa09bed740 1 -- start start 2024-04-23T10:25:37.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.223+0000 7faa09bed740 1 Processor -- start 2024-04-23T10:25:37.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.223+0000 7faa09bed740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644fa9c4b80 0x5644fa9cf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.223+0000 7faa09bed740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644fa9c5d50 0x5644fa9cf5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.223+0000 7faa09bed740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644fa9c6710 0x5644fa9d6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.223+0000 7faa09bed740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5644fa9cd720 con 0x5644fa9c6710 2024-04-23T10:25:37.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.223+0000 7faa09bed740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5644fa8c4470 con 0x5644fa9c5d50 2024-04-23T10:25:37.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.223+0000 7faa09bed740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5644fa9cd8a0 con 0x5644fa9c4b80 2024-04-23T10:25:37.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.223+0000 7faa09bed740 10 client.0 _ll_register_callbacks cb 0x5644fa8e9f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:25:37.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.223+0000 7fa9fffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644fa9c6710 0x5644fa9d6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:37.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.223+0000 7fa9fffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644fa9c6710 0x5644fa9d6610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59844/0 (socket says 172.21.15.97:59844) 2024-04-23T10:25:37.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.223+0000 7fa9fffff640 1 -- 172.21.15.97:0/773961215 learned_addr learned my addr 172.21.15.97:0/773961215 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:25:37.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.223+0000 7fa9fffff640 1 --2- 172.21.15.97:0/773961215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644fa9c4b80 0x5644fa9cf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:37.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.223+0000 7faa04b9a640 1 --2- 172.21.15.97:0/773961215 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644fa9c5d50 0x5644fa9cf5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:37.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.223+0000 7fa9fffff640 1 -- 172.21.15.97:0/773961215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644fa9c4b80 msgr2=0x5644fa9cf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:25:37.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.223+0000 7fa9fffff640 1 --2- 172.21.15.97:0/773961215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644fa9c4b80 0x5644fa9cf090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.224+0000 7fa9fffff640 1 -- 172.21.15.97:0/773961215 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644fa9c5d50 msgr2=0x5644fa9cf5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:25:37.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.224+0000 7fa9fffff640 1 --2- 172.21.15.97:0/773961215 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644fa9c5d50 0x5644fa9cf5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.224+0000 7fa9fffff640 1 -- 172.21.15.97:0/773961215 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644fa7066b0 con 0x5644fa9c6710 2024-04-23T10:25:37.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.224+0000 7faa04b9a640 1 --2- 172.21.15.97:0/773961215 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644fa9c5d50 0x5644fa9cf5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:25:37.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.224+0000 7fa9fffff640 1 --2- 172.21.15.97:0/773961215 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644fa9c6710 0x5644fa9d6610 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8005780 tx=0x7fa9e80056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c741105ae430a82e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:37.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.224+0000 7fa9dffff640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:25:37.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.224+0000 7fa9dffff640 1 -- 172.21.15.97:0/773961215 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e801f060 con 0x5644fa9c6710 2024-04-23T10:25:37.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.224+0000 7fa9dffff640 1 -- 172.21.15.97:0/773961215 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fa9e801a840 con 0x5644fa9c6710 2024-04-23T10:25:37.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.224+0000 7faa09bed740 1 -- 172.21.15.97:0/773961215 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9e801aab0 con 0x5644fa9c6710 2024-04-23T10:25:37.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.224+0000 7fa9dffff640 1 -- 172.21.15.97:0/773961215 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa9e8004030 con 0x5644fa9c6710 2024-04-23T10:25:37.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.224+0000 7faa09bed740 10 client.15114 fetch_fsmap learned FSMap version 25 2024-04-23T10:25:37.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.224+0000 7faa09bed740 1 -- 172.21.15.97:0/773961215 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5644fa9d9980 con 0x5644fa9c6710 2024-04-23T10:25:37.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.224+0000 7fa9dffff640 1 -- 172.21.15.97:0/773961215 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9e801aab0 con 0x5644fa9c6710 2024-04-23T10:25:37.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.224+0000 7fa9dffff640 1 -- 172.21.15.97:0/773961215 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa9e80035f0 con 0x5644fa9c6710 2024-04-23T10:25:37.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.224+0000 7faa09bed740 10 client.15114 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T10:25:37.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.224+0000 7faa09bed740 10 client.15114 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-23T10:25:37.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.224+0000 7faa09bed740 20 client.15114 populate_metadata read hostname 'smithi097' 2024-04-23T10:25:37.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.224+0000 7faa09bed740 1 --2- 172.21.15.97:0/773961215 >> [v2:172.21.15.196:6836/3094946502,v1:172.21.15.196:6837/3094946502] conn(0x5644fa9dbc40 0x5644fa9dd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.224+0000 7faa09bed740 4 client.15114 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:25:37.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.224+0000 7faa09bed740 1 -- 172.21.15.97:0/773961215 --> [v2:172.21.15.196:6836/3094946502,v1:172.21.15.196:6837/3094946502] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644fa937670 con 0x5644fa9dbc40 2024-04-23T10:25:37.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.225+0000 7faa04b9a640 1 --2- 172.21.15.97:0/773961215 >> [v2:172.21.15.196:6836/3094946502,v1:172.21.15.196:6837/3094946502] conn(0x5644fa9dbc40 0x5644fa9dd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:37.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.225+0000 7fa9dffff640 10 client.15114 ms_handle_connect on v2:172.21.15.196:6836/3094946502 2024-04-23T10:25:37.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.225+0000 7faa04b9a640 1 --2- 172.21.15.97:0/773961215 >> [v2:172.21.15.196:6836/3094946502,v1:172.21.15.196:6837/3094946502] conn(0x5644fa9dbc40 0x5644fa9dd0b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ccb2fe7cc3faa7f2 server_cookie=b812bfad4d83faa8 in_seq=0 out_seq=0 2024-04-23T10:25:37.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.226+0000 7fa9dffff640 1 -- 172.21.15.97:0/773961215 <== mds.2 v2:172.21.15.196:6836/3094946502 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5644fa937670 con 0x5644fa9dbc40 2024-04-23T10:25:37.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.226+0000 7fa9dffff640 10 client.15114 handle_command_reply: tid=0 2024-04-23T10:25:37.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.226+0000 7faa09bed740 1 client.15114 shutdown 2024-04-23T10:25:37.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.226+0000 7faa09bed740 1 -- 172.21.15.97:0/773961215 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644fa9c6710 msgr2=0x5644fa9d6610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:37.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.226+0000 7faa09bed740 1 --2- 172.21.15.97:0/773961215 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644fa9c6710 0x5644fa9d6610 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8005780 tx=0x7fa9e80056d0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.226+0000 7faa09bed740 1 -- 172.21.15.97:0/773961215 shutdown_connections 2024-04-23T10:25:37.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.226+0000 7faa09bed740 1 --2- 172.21.15.97:0/773961215 >> [v2:172.21.15.196:6836/3094946502,v1:172.21.15.196:6837/3094946502] conn(0x5644fa9dbc40 0x5644fa9dd0b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.226+0000 7faa09bed740 1 --2- 172.21.15.97:0/773961215 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644fa9c6710 0x5644fa9d6610 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.226+0000 7fa9dffff640 0 client.15114 ms_handle_reset on v2:172.21.15.196:6836/3094946502 2024-04-23T10:25:37.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.226+0000 7faa09bed740 1 --2- 172.21.15.97:0/773961215 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644fa9c5d50 0x5644fa9cf5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.226+0000 7faa09bed740 1 --2- 172.21.15.97:0/773961215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644fa9c4b80 0x5644fa9cf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.226+0000 7faa09bed740 1 -- 172.21.15.97:0/773961215 >> 172.21.15.97:0/773961215 conn(0x5644fa9c0070 msgr2=0x5644fa9bf1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:37.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.226+0000 7faa09bed740 1 -- 172.21.15.97:0/773961215 shutdown_connections 2024-04-23T10:25:37.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.226+0000 7faa09bed740 1 -- 172.21.15.97:0/773961215 wait complete. 2024-04-23T10:25:37.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.226+0000 7faa09bed740 20 client.15114 trim_cache size 0 max 16384 2024-04-23T10:25:37.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.244+0000 7faa09bed740 1 Processor -- start 2024-04-23T10:25:37.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.244+0000 7faa09bed740 1 -- start start 2024-04-23T10:25:37.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.244+0000 7faa09bed740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644fa9c4b80 0x5644fa9c6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.244+0000 7faa09bed740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644fa9c5d50 0x5644fa9cf530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.244+0000 7faa09bed740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644fa9c6710 0x5644fa9d6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.244+0000 7faa09bed740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5644fa9db3f0 con 0x5644fa9c5d50 2024-04-23T10:25:37.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.244+0000 7faa09bed740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5644fa9dbac0 con 0x5644fa9c6710 2024-04-23T10:25:37.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.244+0000 7faa09bed740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5644fa9dc190 con 0x5644fa9c4b80 2024-04-23T10:25:37.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.245+0000 7faa04b9a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644fa9c5d50 0x5644fa9cf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:37.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.245+0000 7faa04b9a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644fa9c5d50 0x5644fa9cf530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59850/0 (socket says 172.21.15.97:59850) 2024-04-23T10:25:37.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.245+0000 7faa04b9a640 1 -- 172.21.15.97:0/3062497406 learned_addr learned my addr 172.21.15.97:0/3062497406 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:25:37.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.245+0000 7faa04b9a640 1 -- 172.21.15.97:0/3062497406 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644fa9c4b80 msgr2=0x5644fa9c6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:25:37.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.245+0000 7faa04b9a640 1 --2- 172.21.15.97:0/3062497406 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644fa9c4b80 0x5644fa9c6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.245+0000 7faa04b9a640 1 -- 172.21.15.97:0/3062497406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644fa9c6710 msgr2=0x5644fa9d6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:25:37.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.245+0000 7faa04b9a640 1 --2- 172.21.15.97:0/3062497406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644fa9c6710 0x5644fa9d6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.245+0000 7faa04b9a640 1 -- 172.21.15.97:0/3062497406 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644fa9c5570 con 0x5644fa9c5d50 2024-04-23T10:25:37.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.245+0000 7faa04b9a640 1 --2- 172.21.15.97:0/3062497406 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644fa9c5d50 0x5644fa9cf530 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8005c40 tx=0x7fa9f8010a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a591d3237b3a4c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:37.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.245+0000 7fa9f67fc640 1 -- 172.21.15.97:0/3062497406 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f8004d20 con 0x5644fa9c5d50 2024-04-23T10:25:37.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.245+0000 7fa9f67fc640 1 -- 172.21.15.97:0/3062497406 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9f8023030 con 0x5644fa9c5d50 2024-04-23T10:25:37.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.245+0000 7fa9f67fc640 1 -- 172.21.15.97:0/3062497406 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f800fb20 con 0x5644fa9c5d50 2024-04-23T10:25:37.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.246+0000 7faa09bed740 1 -- 172.21.15.97:0/3062497406 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644fa9c5d50 msgr2=0x5644fa9cf530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:37.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.246+0000 7faa09bed740 1 --2- 172.21.15.97:0/3062497406 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644fa9c5d50 0x5644fa9cf530 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8005c40 tx=0x7fa9f8010a80 comp rx=0 tx=0).stop 2024-04-23T10:25:37.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.246+0000 7faa09bed740 1 -- 172.21.15.97:0/3062497406 shutdown_connections 2024-04-23T10:25:37.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.246+0000 7faa09bed740 1 --2- 172.21.15.97:0/3062497406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644fa9c6710 0x5644fa9d6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.246+0000 7faa09bed740 1 --2- 172.21.15.97:0/3062497406 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644fa9c5d50 0x5644fa9cf530 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.246+0000 7faa09bed740 1 --2- 172.21.15.97:0/3062497406 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644fa9c4b80 0x5644fa9c6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.246+0000 7faa09bed740 1 -- 172.21.15.97:0/3062497406 >> 172.21.15.97:0/3062497406 conn(0x5644fa9c0070 msgr2=0x5644fa9c0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:37.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.246+0000 7faa09bed740 1 -- 172.21.15.97:0/3062497406 shutdown_connections 2024-04-23T10:25:37.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.246+0000 7faa09bed740 1 -- 172.21.15.97:0/3062497406 wait complete. 2024-04-23T10:25:37.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.246+0000 7faa09bed740 1 -- start start 2024-04-23T10:25:37.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.246+0000 7faa09bed740 1 Processor -- start 2024-04-23T10:25:37.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.246+0000 7faa09bed740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644fa9c4b80 0x5644fa9cba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.246+0000 7faa09bed740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644fa9c5d50 0x5644fa9cbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.246+0000 7faa09bed740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644fa9c6710 0x5644fa9d4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.246+0000 7faa09bed740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5644fa9dcf30 con 0x5644fa9c6710 2024-04-23T10:25:37.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.246+0000 7faa09bed740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5644fa9dc860 con 0x5644fa9c4b80 2024-04-23T10:25:37.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.246+0000 7faa09bed740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5644fa9dd600 con 0x5644fa9c5d50 2024-04-23T10:25:37.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.246+0000 7faa09bed740 10 client.0 _ll_register_callbacks cb 0x7fa9f800a250 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:25:37.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.246+0000 7fa9fffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644fa9c6710 0x5644fa9d4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:37.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.247+0000 7fa9fffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644fa9c6710 0x5644fa9d4b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59852/0 (socket says 172.21.15.97:59852) 2024-04-23T10:25:37.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.247+0000 7fa9fffff640 1 -- 172.21.15.97:0/4113005680 learned_addr learned my addr 172.21.15.97:0/4113005680 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:25:37.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.247+0000 7faa04b9a640 1 --2- 172.21.15.97:0/4113005680 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644fa9c5d50 0x5644fa9cbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:37.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.247+0000 7fa9fffff640 1 --2- 172.21.15.97:0/4113005680 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644fa9c4b80 0x5644fa9cba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:37.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.247+0000 7fa9fffff640 1 -- 172.21.15.97:0/4113005680 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644fa9c5d50 msgr2=0x5644fa9cbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:25:37.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.247+0000 7fa9fffff640 1 --2- 172.21.15.97:0/4113005680 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644fa9c5d50 0x5644fa9cbfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.247+0000 7fa9fffff640 1 -- 172.21.15.97:0/4113005680 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644fa9c4b80 msgr2=0x5644fa9cba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:25:37.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.247+0000 7fa9fffff640 1 --2- 172.21.15.97:0/4113005680 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644fa9c4b80 0x5644fa9cba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.247+0000 7fa9fffff640 1 -- 172.21.15.97:0/4113005680 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644fa9d5180 con 0x5644fa9c6710 2024-04-23T10:25:37.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.247+0000 7fa9fffff640 1 --2- 172.21.15.97:0/4113005680 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644fa9c6710 0x5644fa9d4b90 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8003970 tx=0x7fa9e8005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6945e033afeedd9d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:37.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.247+0000 7fa9dffff640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:25:37.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.247+0000 7fa9dffff640 1 -- 172.21.15.97:0/4113005680 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e801f070 con 0x5644fa9c6710 2024-04-23T10:25:37.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.247+0000 7faa09bed740 1 -- 172.21.15.97:0/4113005680 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9f8023030 con 0x5644fa9c6710 2024-04-23T10:25:37.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.247+0000 7fa9dffff640 1 -- 172.21.15.97:0/4113005680 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fa9e80041d0 con 0x5644fa9c6710 2024-04-23T10:25:37.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.247+0000 7fa9dffff640 1 -- 172.21.15.97:0/4113005680 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa9e8040070 con 0x5644fa9c6710 2024-04-23T10:25:37.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.247+0000 7faa09bed740 10 client.15120 fetch_fsmap learned FSMap version 25 2024-04-23T10:25:37.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.247+0000 7faa09bed740 1 -- 172.21.15.97:0/4113005680 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5644fa9d6480 con 0x5644fa9c6710 2024-04-23T10:25:37.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.247+0000 7fa9dffff640 1 -- 172.21.15.97:0/4113005680 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9f8023030 con 0x5644fa9c6710 2024-04-23T10:25:37.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.247+0000 7fa9dffff640 1 -- 172.21.15.97:0/4113005680 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa9e8036020 con 0x5644fa9c6710 2024-04-23T10:25:37.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.247+0000 7faa09bed740 10 client.15120 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T10:25:37.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.248+0000 7faa09bed740 10 client.15120 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-23T10:25:37.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.248+0000 7faa09bed740 20 client.15120 populate_metadata read hostname 'smithi097' 2024-04-23T10:25:37.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.248+0000 7faa09bed740 1 --2- 172.21.15.97:0/4113005680 >> [v2:172.21.15.196:6836/3094946502,v1:172.21.15.196:6837/3094946502] conn(0x5644faa46090 0x5644faa424d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.248+0000 7faa09bed740 4 client.15120 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T10:25:37.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.248+0000 7faa09bed740 1 -- 172.21.15.97:0/4113005680 --> [v2:172.21.15.196:6836/3094946502,v1:172.21.15.196:6837/3094946502] -- command(tid 0: {"prefix": "status"}) -- 0x5644fa9314e0 con 0x5644faa46090 2024-04-23T10:25:37.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.248+0000 7faa04b9a640 1 --2- 172.21.15.97:0/4113005680 >> [v2:172.21.15.196:6836/3094946502,v1:172.21.15.196:6837/3094946502] conn(0x5644faa46090 0x5644faa424d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:37.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.248+0000 7faa04b9a640 1 --2- 172.21.15.97:0/4113005680 >> [v2:172.21.15.196:6836/3094946502,v1:172.21.15.196:6837/3094946502] conn(0x5644faa46090 0x5644faa424d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=17325715771e8cd server_cookie=d9982f8646530011 in_seq=0 out_seq=0 2024-04-23T10:25:37.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.248+0000 7fa9dffff640 10 client.15120 ms_handle_connect on v2:172.21.15.196:6836/3094946502 2024-04-23T10:25:37.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.248+0000 7fa9dffff640 1 -- 172.21.15.97:0/4113005680 <== mds.2 v2:172.21.15.196:6836/3094946502 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5644fa9314e0 con 0x5644faa46090 2024-04-23T10:25:37.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.248+0000 7fa9dffff640 10 client.15120 handle_command_reply: tid=0 2024-04-23T10:25:37.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.248+0000 7faa09bed740 1 client.15120 shutdown 2024-04-23T10:25:37.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.248+0000 7faa09bed740 1 -- 172.21.15.97:0/4113005680 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644fa9c6710 msgr2=0x5644fa9d4b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:37.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.248+0000 7faa09bed740 1 --2- 172.21.15.97:0/4113005680 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644fa9c6710 0x5644fa9d4b90 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8003970 tx=0x7fa9e8005fb0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.248+0000 7faa09bed740 1 -- 172.21.15.97:0/4113005680 shutdown_connections 2024-04-23T10:25:37.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.248+0000 7faa09bed740 1 --2- 172.21.15.97:0/4113005680 >> [v2:172.21.15.196:6836/3094946502,v1:172.21.15.196:6837/3094946502] conn(0x5644faa46090 0x5644faa424d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.248+0000 7faa09bed740 1 --2- 172.21.15.97:0/4113005680 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644fa9c6710 0x5644fa9d4b90 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.248+0000 7faa09bed740 1 --2- 172.21.15.97:0/4113005680 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644fa9c5d50 0x5644fa9cbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.248+0000 7fa9dffff640 0 client.15120 ms_handle_reset on v2:172.21.15.196:6836/3094946502 2024-04-23T10:25:37.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.248+0000 7faa09bed740 1 --2- 172.21.15.97:0/4113005680 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644fa9c4b80 0x5644fa9cba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.248+0000 7faa09bed740 1 -- 172.21.15.97:0/4113005680 >> 172.21.15.97:0/4113005680 conn(0x5644fa9c0070 msgr2=0x5644fa9bb0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:37.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.249+0000 7faa09bed740 1 -- 172.21.15.97:0/4113005680 shutdown_connections 2024-04-23T10:25:37.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.249+0000 7faa09bed740 1 -- 172.21.15.97:0/4113005680 wait complete. 2024-04-23T10:25:37.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.249+0000 7faa09bed740 20 client.15120 trim_cache size 0 max 16384 2024-04-23T10:25:37.255 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-23T10:25:37.255 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "9b22af88-0159-11ef-bc93-c7b262605968", 2024-04-23T10:25:37.255 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 2, 2024-04-23T10:25:37.255 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 24469, 2024-04-23T10:25:37.255 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-04-23T10:25:37.255 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-04-23T10:25:37.255 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-04-23T10:25:37.255 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 86.676317338000004, 2024-04-23T10:25:37.255 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 25, 2024-04-23T10:25:37.256 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 88, 2024-04-23T10:25:37.256 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 86, 2024-04-23T10:25:37.256 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 110.552061557 2024-04-23T10:25:37.256 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-23T10:25:37.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.249+0000 7faa06dfe640 1 -- 172.21.15.97:0/1306237315 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fa9ec0a3bb0 msgr2=0x7fa9ec0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:37.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.249+0000 7faa06dfe640 1 --2- 172.21.15.97:0/1306237315 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fa9ec0a3bb0 0x7fa9ec0a6070 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa9f00bc9a0 tx=0x7fa9f00dd040 comp rx=0 tx=0).stop 2024-04-23T10:25:37.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.249+0000 7faa06dfe640 1 -- 172.21.15.97:0/1306237315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa00102440 msgr2=0x7faa0019f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:37.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.249+0000 7faa06dfe640 1 --2- 172.21.15.97:0/1306237315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa00102440 0x7faa0019f730 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0031500 tx=0x7fa9f0032f90 comp rx=0 tx=0).stop 2024-04-23T10:25:37.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.249+0000 7faa06dfe640 1 -- 172.21.15.97:0/1306237315 shutdown_connections 2024-04-23T10:25:37.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.249+0000 7faa06dfe640 1 --2- 172.21.15.97:0/1306237315 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fa9ec0a3bb0 0x7fa9ec0a6070 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.249+0000 7faa06dfe640 1 --2- 172.21.15.97:0/1306237315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faa00102440 0x7faa0019f730 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.249+0000 7faa06dfe640 1 --2- 172.21.15.97:0/1306237315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa00101b30 0x7faa00198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.249+0000 7faa06dfe640 1 --2- 172.21.15.97:0/1306237315 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa00101170 0x7faa00198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.249+0000 7faa06dfe640 1 -- 172.21.15.97:0/1306237315 >> 172.21.15.97:0/1306237315 conn(0x7faa000fbb80 msgr2=0x7faa00105480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:37.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.249+0000 7faa06dfe640 1 -- 172.21.15.97:0/1306237315 shutdown_connections 2024-04-23T10:25:37.257 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.249+0000 7faa06dfe640 1 -- 172.21.15.97:0/1306237315 wait complete. 2024-04-23T10:25:37.260 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-23T10:25:37.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.359+0000 7fd83aaa9640 1 -- 172.21.15.97:0/3055091562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd834101fe0 msgr2=0x7fd834102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:37.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.359+0000 7fd83aaa9640 1 --2- 172.21.15.97:0/3055091562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd834101fe0 0x7fd834102460 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd820009900 tx=0x7fd82002a410 comp rx=0 tx=0).stop 2024-04-23T10:25:37.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.359+0000 7fd83aaa9640 1 -- 172.21.15.97:0/3055091562 shutdown_connections 2024-04-23T10:25:37.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.359+0000 7fd83aaa9640 1 --2- 172.21.15.97:0/3055091562 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8341029a0 0x7fd8341092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.359+0000 7fd83aaa9640 1 --2- 172.21.15.97:0/3055091562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd834101fe0 0x7fd834102460 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.359+0000 7fd83aaa9640 1 --2- 172.21.15.97:0/3055091562 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd834100e90 0x7fd8341012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.359+0000 7fd83aaa9640 1 -- 172.21.15.97:0/3055091562 >> 172.21.15.97:0/3055091562 conn(0x7fd8340fc440 msgr2=0x7fd8340fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:37.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.359+0000 7fd83aaa9640 1 -- 172.21.15.97:0/3055091562 shutdown_connections 2024-04-23T10:25:37.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.359+0000 7fd83aaa9640 1 -- 172.21.15.97:0/3055091562 wait complete. 2024-04-23T10:25:37.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.359+0000 7fd83aaa9640 1 Processor -- start 2024-04-23T10:25:37.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.360+0000 7fd83aaa9640 1 -- start start 2024-04-23T10:25:37.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.360+0000 7fd83aaa9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd834100e90 0x7fd83406b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.360+0000 7fd83aaa9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd834101fe0 0x7fd83419cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.360+0000 7fd83aaa9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8341029a0 0x7fd83419f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.360+0000 7fd83aaa9640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd83410ca40 con 0x7fd834100e90 2024-04-23T10:25:37.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.360+0000 7fd83aaa9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd83410c8c0 con 0x7fd8341029a0 2024-04-23T10:25:37.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.360+0000 7fd83aaa9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd83410cbc0 con 0x7fd834101fe0 2024-04-23T10:25:37.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.360+0000 7fd838845640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd834100e90 0x7fd83406b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:37.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.360+0000 7fd838845640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd834100e90 0x7fd83406b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59870/0 (socket says 172.21.15.97:59870) 2024-04-23T10:25:37.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.360+0000 7fd838845640 1 -- 172.21.15.97:0/3808608143 learned_addr learned my addr 172.21.15.97:0/3808608143 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:25:37.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.360+0000 7fd833fff640 1 --2- 172.21.15.97:0/3808608143 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd834101fe0 0x7fd83419cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:37.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.360+0000 7fd838845640 1 -- 172.21.15.97:0/3808608143 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd834101fe0 msgr2=0x7fd83419cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:37.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.360+0000 7fd838845640 1 --2- 172.21.15.97:0/3808608143 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd834101fe0 0x7fd83419cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.360+0000 7fd838845640 1 -- 172.21.15.97:0/3808608143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8341029a0 msgr2=0x7fd83419f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:37.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.360+0000 7fd838845640 1 --2- 172.21.15.97:0/3808608143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8341029a0 0x7fd83419f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.360+0000 7fd838845640 1 -- 172.21.15.97:0/3808608143 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd83419fec0 con 0x7fd834100e90 2024-04-23T10:25:37.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.360+0000 7fd838845640 1 --2- 172.21.15.97:0/3808608143 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd834100e90 0x7fd83406b140 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd82800b4a0 tx=0x7fd82800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:37.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.360+0000 7fd831ffb640 1 -- 172.21.15.97:0/3808608143 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd828014070 con 0x7fd834100e90 2024-04-23T10:25:37.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.360+0000 7fd831ffb640 1 -- 172.21.15.97:0/3808608143 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd828010ac0 con 0x7fd834100e90 2024-04-23T10:25:37.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.360+0000 7fd83aaa9640 1 -- 172.21.15.97:0/3808608143 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8341a0150 con 0x7fd834100e90 2024-04-23T10:25:37.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.360+0000 7fd831ffb640 1 -- 172.21.15.97:0/3808608143 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd828017c30 con 0x7fd834100e90 2024-04-23T10:25:37.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.361+0000 7fd83aaa9640 1 -- 172.21.15.97:0/3808608143 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8341a06a0 con 0x7fd834100e90 2024-04-23T10:25:37.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.362+0000 7fd831ffb640 1 -- 172.21.15.97:0/3808608143 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd828017070 con 0x7fd834100e90 2024-04-23T10:25:37.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.363+0000 7fd831ffb640 1 --2- 172.21.15.97:0/3808608143 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fd8240a3bb0 0x7fd8240a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.363+0000 7fd83d7bd740 1 Processor -- start 2024-04-23T10:25:37.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.363+0000 7fd83d7bd740 1 -- start start 2024-04-23T10:25:37.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.363+0000 7fd833fff640 1 --2- 172.21.15.97:0/3808608143 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fd8240a3bb0 0x7fd8240a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:37.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.363+0000 7fd831ffb640 1 -- 172.21.15.97:0/3808608143 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fd8280cf820 con 0x7fd834100e90 2024-04-23T10:25:37.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.363+0000 7fd83d7bd740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648df525b80 0x5648df525fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.363+0000 7fd83d7bd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648df526d50 0x5648df5271d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.363+0000 7fd83d7bd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648df527710 0x5648df52dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.363+0000 7fd83d7bd740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5648df48ac50 con 0x5648df525b80 2024-04-23T10:25:37.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.363+0000 7fd83d7bd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5648df427830 con 0x5648df527710 2024-04-23T10:25:37.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.363+0000 7fd83d7bd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5648df486c10 con 0x5648df526d50 2024-04-23T10:25:37.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.363+0000 7fd838845640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648df525b80 0x5648df525fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:37.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.363+0000 7fd838845640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648df525b80 0x5648df525fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59884/0 (socket says 172.21.15.97:59884) 2024-04-23T10:25:37.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.363+0000 7fd838845640 1 -- 172.21.15.97:0/2942251070 learned_addr learned my addr 172.21.15.97:0/2942251070 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:25:37.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.363+0000 7fd833fff640 1 --2- 172.21.15.97:0/2942251070 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648df526d50 0x5648df5271d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:37.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.363+0000 7fd839046640 1 --2- 172.21.15.97:0/2942251070 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648df527710 0x5648df52dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:37.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.364+0000 7fd838845640 1 -- 172.21.15.97:0/2942251070 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648df526d50 msgr2=0x5648df5271d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:25:37.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.364+0000 7fd838845640 1 --2- 172.21.15.97:0/2942251070 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648df526d50 0x5648df5271d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.364+0000 7fd833fff640 1 --2- 172.21.15.97:0/3808608143 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fd8240a3bb0 0x7fd8240a6070 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd820000c00 tx=0x7fd820031c30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:37.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.364+0000 7fd838845640 1 -- 172.21.15.97:0/2942251070 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648df527710 msgr2=0x5648df52dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:25:37.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.364+0000 7fd838845640 1 --2- 172.21.15.97:0/2942251070 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648df527710 0x5648df52dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.364+0000 7fd838845640 1 -- 172.21.15.97:0/2942251070 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648df19d2d0 con 0x5648df525b80 2024-04-23T10:25:37.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.364+0000 7fd838845640 1 --2- 172.21.15.97:0/2942251070 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648df525b80 0x5648df525fa0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd82800b470 tx=0x7fd828017650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e73823561052f82b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:37.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.364+0000 7fd81effd640 1 -- 172.21.15.97:0/2942251070 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd82809ce60 con 0x5648df525b80 2024-04-23T10:25:37.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.364+0000 7fd81effd640 1 -- 172.21.15.97:0/2942251070 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd82809fee0 con 0x5648df525b80 2024-04-23T10:25:37.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.364+0000 7fd81effd640 1 -- 172.21.15.97:0/2942251070 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd82801e050 con 0x5648df525b80 2024-04-23T10:25:37.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.364+0000 7fd83d7bd740 1 -- 172.21.15.97:0/2942251070 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648df525b80 msgr2=0x5648df525fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:37.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.364+0000 7fd83d7bd740 1 --2- 172.21.15.97:0/2942251070 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648df525b80 0x5648df525fa0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd82800b470 tx=0x7fd828017650 comp rx=0 tx=0).stop 2024-04-23T10:25:37.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.365+0000 7fd83d7bd740 1 -- 172.21.15.97:0/2942251070 shutdown_connections 2024-04-23T10:25:37.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.365+0000 7fd83d7bd740 1 --2- 172.21.15.97:0/2942251070 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648df527710 0x5648df52dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.365+0000 7fd83d7bd740 1 --2- 172.21.15.97:0/2942251070 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648df526d50 0x5648df5271d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.365+0000 7fd83d7bd740 1 --2- 172.21.15.97:0/2942251070 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648df525b80 0x5648df525fa0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.365+0000 7fd83d7bd740 1 -- 172.21.15.97:0/2942251070 >> 172.21.15.97:0/2942251070 conn(0x5648df521070 msgr2=0x5648df521470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:37.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.365+0000 7fd83d7bd740 1 -- 172.21.15.97:0/2942251070 shutdown_connections 2024-04-23T10:25:37.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.365+0000 7fd83d7bd740 1 -- 172.21.15.97:0/2942251070 wait complete. 2024-04-23T10:25:37.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.365+0000 7fd83d7bd740 1 -- start start 2024-04-23T10:25:37.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.365+0000 7fd83d7bd740 1 Processor -- start 2024-04-23T10:25:37.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.365+0000 7fd83d7bd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648df525b80 0x5648df530070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.365+0000 7fd83d7bd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648df526d50 0x5648df5305b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.365+0000 7fd83d7bd740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648df527710 0x5648df5375f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.365+0000 7fd83d7bd740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5648df52e720 con 0x5648df527710 2024-04-23T10:25:37.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.365+0000 7fd83d7bd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5648df425470 con 0x5648df526d50 2024-04-23T10:25:37.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.365+0000 7fd83d7bd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5648df52e8a0 con 0x5648df525b80 2024-04-23T10:25:37.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.365+0000 7fd83d7bd740 10 client.0 _ll_register_callbacks cb 0x5648df44af30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:25:37.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.365+0000 7fd839046640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648df527710 0x5648df5375f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:37.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.365+0000 7fd839046640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648df527710 0x5648df5375f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59892/0 (socket says 172.21.15.97:59892) 2024-04-23T10:25:37.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.365+0000 7fd839046640 1 -- 172.21.15.97:0/365025065 learned_addr learned my addr 172.21.15.97:0/365025065 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:25:37.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.366+0000 7fd838845640 1 --2- 172.21.15.97:0/365025065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648df525b80 0x5648df530070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:37.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.366+0000 7fd839046640 1 -- 172.21.15.97:0/365025065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648df525b80 msgr2=0x5648df530070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:25:37.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.366+0000 7fd839046640 1 --2- 172.21.15.97:0/365025065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648df525b80 0x5648df530070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.366+0000 7fd839046640 1 -- 172.21.15.97:0/365025065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648df526d50 msgr2=0x5648df5305b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:25:37.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.366+0000 7fd839046640 1 --2- 172.21.15.97:0/365025065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648df526d50 0x5648df5305b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.366+0000 7fd839046640 1 -- 172.21.15.97:0/365025065 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648df2676b0 con 0x5648df527710 2024-04-23T10:25:37.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.366+0000 7fd839046640 1 --2- 172.21.15.97:0/365025065 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648df527710 0x5648df5375f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd82c00e7c0 tx=0x7fd82c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a306e757a87c3b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:37.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.366+0000 7fd813fff640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:25:37.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.366+0000 7fd813fff640 1 -- 172.21.15.97:0/365025065 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd82c00cc60 con 0x5648df527710 2024-04-23T10:25:37.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.366+0000 7fd813fff640 1 -- 172.21.15.97:0/365025065 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fd82c00ce00 con 0x5648df527710 2024-04-23T10:25:37.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.366+0000 7fd83d7bd740 1 -- 172.21.15.97:0/365025065 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd82809fee0 con 0x5648df527710 2024-04-23T10:25:37.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.366+0000 7fd813fff640 1 -- 172.21.15.97:0/365025065 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd82c00ee70 con 0x5648df527710 2024-04-23T10:25:37.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.366+0000 7fd83d7bd740 10 client.15132 fetch_fsmap learned FSMap version 25 2024-04-23T10:25:37.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.366+0000 7fd83d7bd740 1 -- 172.21.15.97:0/365025065 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5648df53aa30 con 0x5648df527710 2024-04-23T10:25:37.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.366+0000 7fd813fff640 1 -- 172.21.15.97:0/365025065 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd82809fee0 con 0x5648df527710 2024-04-23T10:25:37.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.366+0000 7fd813fff640 1 -- 172.21.15.97:0/365025065 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fd82c010430 con 0x5648df527710 2024-04-23T10:25:37.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.366+0000 7fd83d7bd740 10 client.15132 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T10:25:37.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.366+0000 7fd83d7bd740 10 client.15132 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-23T10:25:37.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.366+0000 7fd83d7bd740 20 client.15132 populate_metadata read hostname 'smithi097' 2024-04-23T10:25:37.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.367+0000 7fd83d7bd740 1 --2- 172.21.15.97:0/365025065 >> [v2:172.21.15.196:6838/83843556,v1:172.21.15.196:6839/83843556] conn(0x5648df53d1a0 0x5648df53e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.367+0000 7fd83d7bd740 4 client.15132 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:25:37.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.367+0000 7fd83d7bd740 1 -- 172.21.15.97:0/365025065 --> [v2:172.21.15.196:6838/83843556,v1:172.21.15.196:6839/83843556] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5648df498670 con 0x5648df53d1a0 2024-04-23T10:25:37.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.367+0000 7fd838845640 1 --2- 172.21.15.97:0/365025065 >> [v2:172.21.15.196:6838/83843556,v1:172.21.15.196:6839/83843556] conn(0x5648df53d1a0 0x5648df53e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:37.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.367+0000 7fd838845640 1 --2- 172.21.15.97:0/365025065 >> [v2:172.21.15.196:6838/83843556,v1:172.21.15.196:6839/83843556] conn(0x5648df53d1a0 0x5648df53e610 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3b75d791fd60c67e server_cookie=cfeba4e92f844c1d in_seq=0 out_seq=0 2024-04-23T10:25:37.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.367+0000 7fd813fff640 10 client.15132 ms_handle_connect on v2:172.21.15.196:6838/83843556 2024-04-23T10:25:37.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.368+0000 7fd813fff640 1 -- 172.21.15.97:0/365025065 <== mds.1 v2:172.21.15.196:6838/83843556 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5648df498670 con 0x5648df53d1a0 2024-04-23T10:25:37.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.368+0000 7fd813fff640 10 client.15132 handle_command_reply: tid=0 2024-04-23T10:25:37.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.368+0000 7fd83d7bd740 1 client.15132 shutdown 2024-04-23T10:25:37.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.368+0000 7fd83d7bd740 1 -- 172.21.15.97:0/365025065 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648df527710 msgr2=0x5648df5375f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:37.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.368+0000 7fd83d7bd740 1 --2- 172.21.15.97:0/365025065 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648df527710 0x5648df5375f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd82c00e7c0 tx=0x7fd82c00ec80 comp rx=0 tx=0).stop 2024-04-23T10:25:37.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.368+0000 7fd83d7bd740 1 -- 172.21.15.97:0/365025065 shutdown_connections 2024-04-23T10:25:37.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.368+0000 7fd83d7bd740 1 --2- 172.21.15.97:0/365025065 >> [v2:172.21.15.196:6838/83843556,v1:172.21.15.196:6839/83843556] conn(0x5648df53d1a0 0x5648df53e610 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.368+0000 7fd83d7bd740 1 --2- 172.21.15.97:0/365025065 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648df527710 0x5648df5375f0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.368+0000 7fd83d7bd740 1 --2- 172.21.15.97:0/365025065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648df526d50 0x5648df5305b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.368+0000 7fd83d7bd740 1 --2- 172.21.15.97:0/365025065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648df525b80 0x5648df530070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.368+0000 7fd813fff640 0 client.15132 ms_handle_reset on v2:172.21.15.196:6838/83843556 2024-04-23T10:25:37.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.368+0000 7fd83d7bd740 1 -- 172.21.15.97:0/365025065 >> 172.21.15.97:0/365025065 conn(0x5648df521070 msgr2=0x5648df5201d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:37.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.368+0000 7fd83d7bd740 1 -- 172.21.15.97:0/365025065 shutdown_connections 2024-04-23T10:25:37.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.368+0000 7fd83d7bd740 1 -- 172.21.15.97:0/365025065 wait complete. 2024-04-23T10:25:37.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.368+0000 7fd83d7bd740 20 client.15132 trim_cache size 0 max 16384 2024-04-23T10:25:37.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.386+0000 7fd83d7bd740 1 Processor -- start 2024-04-23T10:25:37.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.386+0000 7fd83d7bd740 1 -- start start 2024-04-23T10:25:37.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.386+0000 7fd83d7bd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648df525b80 0x5648df521450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.386+0000 7fd83d7bd740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648df526d50 0x5648df5375b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.386+0000 7fd83d7bd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648df527710 0x5648df530050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.386+0000 7fd83d7bd740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5648df53c950 con 0x5648df526d50 2024-04-23T10:25:37.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.386+0000 7fd83d7bd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5648df53d020 con 0x5648df525b80 2024-04-23T10:25:37.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.386+0000 7fd83d7bd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5648df53d6f0 con 0x5648df527710 2024-04-23T10:25:37.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.386+0000 7fd833fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648df526d50 0x5648df5375b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:37.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.386+0000 7fd833fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648df526d50 0x5648df5375b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59898/0 (socket says 172.21.15.97:59898) 2024-04-23T10:25:37.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.386+0000 7fd833fff640 1 -- 172.21.15.97:0/2299012972 learned_addr learned my addr 172.21.15.97:0/2299012972 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:25:37.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.386+0000 7fd833fff640 1 -- 172.21.15.97:0/2299012972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648df527710 msgr2=0x5648df530050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:25:37.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.386+0000 7fd833fff640 1 --2- 172.21.15.97:0/2299012972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648df527710 0x5648df530050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.386+0000 7fd833fff640 1 -- 172.21.15.97:0/2299012972 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648df525b80 msgr2=0x5648df521450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:25:37.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.386+0000 7fd833fff640 1 --2- 172.21.15.97:0/2299012972 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648df525b80 0x5648df521450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.386+0000 7fd833fff640 1 -- 172.21.15.97:0/2299012972 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648df5267d0 con 0x5648df526d50 2024-04-23T10:25:37.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.387+0000 7fd833fff640 1 --2- 172.21.15.97:0/2299012972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648df526d50 0x5648df5375b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fd834100bb0 tx=0x7fd82003f070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2619661a02977be5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:37.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.387+0000 7fd81e7fc640 1 -- 172.21.15.97:0/2299012972 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd820004240 con 0x5648df526d50 2024-04-23T10:25:37.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.387+0000 7fd81e7fc640 1 -- 172.21.15.97:0/2299012972 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd82004c070 con 0x5648df526d50 2024-04-23T10:25:37.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.387+0000 7fd81e7fc640 1 -- 172.21.15.97:0/2299012972 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd820004af0 con 0x5648df526d50 2024-04-23T10:25:37.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.387+0000 7fd83d7bd740 1 -- 172.21.15.97:0/2299012972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648df526d50 msgr2=0x5648df5375b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:37.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.387+0000 7fd83d7bd740 1 --2- 172.21.15.97:0/2299012972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648df526d50 0x5648df5375b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fd834100bb0 tx=0x7fd82003f070 comp rx=0 tx=0).stop 2024-04-23T10:25:37.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.387+0000 7fd83d7bd740 1 -- 172.21.15.97:0/2299012972 shutdown_connections 2024-04-23T10:25:37.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.387+0000 7fd83d7bd740 1 --2- 172.21.15.97:0/2299012972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648df527710 0x5648df530050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.387+0000 7fd83d7bd740 1 --2- 172.21.15.97:0/2299012972 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648df526d50 0x5648df5375b0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.387+0000 7fd83d7bd740 1 --2- 172.21.15.97:0/2299012972 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648df525b80 0x5648df521450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.387+0000 7fd83d7bd740 1 -- 172.21.15.97:0/2299012972 >> 172.21.15.97:0/2299012972 conn(0x5648df521070 msgr2=0x5648df529d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:37.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.387+0000 7fd83d7bd740 1 -- 172.21.15.97:0/2299012972 shutdown_connections 2024-04-23T10:25:37.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.387+0000 7fd83d7bd740 1 -- 172.21.15.97:0/2299012972 wait complete. 2024-04-23T10:25:37.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.388+0000 7fd83d7bd740 1 -- start start 2024-04-23T10:25:37.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.388+0000 7fd83d7bd740 1 Processor -- start 2024-04-23T10:25:37.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.388+0000 7fd83d7bd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648df525b80 0x5648df5334f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.388+0000 7fd83d7bd740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648df526d50 0x5648df535a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.388+0000 7fd83d7bd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648df527710 0x5648df535f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.388+0000 7fd83d7bd740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5648df53e490 con 0x5648df526d50 2024-04-23T10:25:37.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.388+0000 7fd83d7bd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5648df53ddc0 con 0x5648df525b80 2024-04-23T10:25:37.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.388+0000 7fd83d7bd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5648df53eb60 con 0x5648df527710 2024-04-23T10:25:37.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.388+0000 7fd83d7bd740 10 client.0 _ll_register_callbacks cb 0x7fd82c0074e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:25:37.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.388+0000 7fd833fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648df526d50 0x5648df535a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:37.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.388+0000 7fd833fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648df526d50 0x5648df535a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59906/0 (socket says 172.21.15.97:59906) 2024-04-23T10:25:37.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.388+0000 7fd833fff640 1 -- 172.21.15.97:0/2420133251 learned_addr learned my addr 172.21.15.97:0/2420133251 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:25:37.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.388+0000 7fd833fff640 1 -- 172.21.15.97:0/2420133251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648df527710 msgr2=0x5648df535f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:25:37.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.388+0000 7fd833fff640 1 --2- 172.21.15.97:0/2420133251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648df527710 0x5648df535f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.388+0000 7fd833fff640 1 -- 172.21.15.97:0/2420133251 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648df525b80 msgr2=0x5648df5334f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:25:37.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.388+0000 7fd833fff640 1 --2- 172.21.15.97:0/2420133251 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648df525b80 0x5648df5334f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.388+0000 7fd833fff640 1 -- 172.21.15.97:0/2420133251 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648df5365c0 con 0x5648df526d50 2024-04-23T10:25:37.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.389+0000 7fd833fff640 1 --2- 172.21.15.97:0/2420133251 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648df526d50 0x5648df535a40 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd820032fd0 tx=0x7fd820004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c45bbd25b48333 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:37.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.389+0000 7fd813fff640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:25:37.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.389+0000 7fd813fff640 1 -- 172.21.15.97:0/2420133251 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd820003800 con 0x5648df526d50 2024-04-23T10:25:37.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.389+0000 7fd813fff640 1 -- 172.21.15.97:0/2420133251 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fd8200039a0 con 0x5648df526d50 2024-04-23T10:25:37.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.389+0000 7fd83d7bd740 1 -- 172.21.15.97:0/2420133251 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd82004c070 con 0x5648df526d50 2024-04-23T10:25:37.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.389+0000 7fd813fff640 1 -- 172.21.15.97:0/2420133251 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd820003ba0 con 0x5648df526d50 2024-04-23T10:25:37.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.389+0000 7fd83d7bd740 10 client.15138 fetch_fsmap learned FSMap version 25 2024-04-23T10:25:37.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.389+0000 7fd83d7bd740 1 -- 172.21.15.97:0/2420133251 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5648df5378c0 con 0x5648df526d50 2024-04-23T10:25:37.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.389+0000 7fd813fff640 1 -- 172.21.15.97:0/2420133251 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd82004c070 con 0x5648df526d50 2024-04-23T10:25:37.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.389+0000 7fd813fff640 1 -- 172.21.15.97:0/2420133251 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fd820063430 con 0x5648df526d50 2024-04-23T10:25:37.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.389+0000 7fd83d7bd740 10 client.15138 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T10:25:37.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.389+0000 7fd83d7bd740 10 client.15138 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-23T10:25:37.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.389+0000 7fd83d7bd740 20 client.15138 populate_metadata read hostname 'smithi097' 2024-04-23T10:25:37.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.389+0000 7fd83d7bd740 1 --2- 172.21.15.97:0/2420133251 >> [v2:172.21.15.196:6838/83843556,v1:172.21.15.196:6839/83843556] conn(0x5648df5a7740 0x5648df5a6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.389+0000 7fd83d7bd740 4 client.15138 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T10:25:37.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.389+0000 7fd83d7bd740 1 -- 172.21.15.97:0/2420133251 --> [v2:172.21.15.196:6838/83843556,v1:172.21.15.196:6839/83843556] -- command(tid 0: {"prefix": "status"}) -- 0x5648df4924e0 con 0x5648df5a7740 2024-04-23T10:25:37.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.389+0000 7fd838845640 1 --2- 172.21.15.97:0/2420133251 >> [v2:172.21.15.196:6838/83843556,v1:172.21.15.196:6839/83843556] conn(0x5648df5a7740 0x5648df5a6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:37.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.390+0000 7fd813fff640 10 client.15138 ms_handle_connect on v2:172.21.15.196:6838/83843556 2024-04-23T10:25:37.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.390+0000 7fd838845640 1 --2- 172.21.15.97:0/2420133251 >> [v2:172.21.15.196:6838/83843556,v1:172.21.15.196:6839/83843556] conn(0x5648df5a7740 0x5648df5a6450 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7abd340b4c54b3f8 server_cookie=40e3d4c457bd15d0 in_seq=0 out_seq=0 2024-04-23T10:25:37.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.390+0000 7fd813fff640 1 -- 172.21.15.97:0/2420133251 <== mds.1 v2:172.21.15.196:6838/83843556 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5648df4924e0 con 0x5648df5a7740 2024-04-23T10:25:37.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.390+0000 7fd813fff640 10 client.15138 handle_command_reply: tid=0 2024-04-23T10:25:37.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.390+0000 7fd83d7bd740 1 client.15138 shutdown 2024-04-23T10:25:37.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.390+0000 7fd83d7bd740 1 -- 172.21.15.97:0/2420133251 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648df526d50 msgr2=0x5648df535a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:37.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.390+0000 7fd83d7bd740 1 --2- 172.21.15.97:0/2420133251 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648df526d50 0x5648df535a40 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd820032fd0 tx=0x7fd820004270 comp rx=0 tx=0).stop 2024-04-23T10:25:37.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.390+0000 7fd83d7bd740 1 -- 172.21.15.97:0/2420133251 shutdown_connections 2024-04-23T10:25:37.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.390+0000 7fd83d7bd740 1 --2- 172.21.15.97:0/2420133251 >> [v2:172.21.15.196:6838/83843556,v1:172.21.15.196:6839/83843556] conn(0x5648df5a7740 0x5648df5a6450 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.390+0000 7fd83d7bd740 1 --2- 172.21.15.97:0/2420133251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648df527710 0x5648df535f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.390+0000 7fd813fff640 0 client.15138 ms_handle_reset on v2:172.21.15.196:6838/83843556 2024-04-23T10:25:37.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.390+0000 7fd83d7bd740 1 --2- 172.21.15.97:0/2420133251 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648df526d50 0x5648df535a40 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.390+0000 7fd83d7bd740 1 --2- 172.21.15.97:0/2420133251 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648df525b80 0x5648df5334f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.390+0000 7fd83d7bd740 1 -- 172.21.15.97:0/2420133251 >> 172.21.15.97:0/2420133251 conn(0x5648df521070 msgr2=0x5648df51f940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:37.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.390+0000 7fd83d7bd740 1 -- 172.21.15.97:0/2420133251 shutdown_connections 2024-04-23T10:25:37.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.390+0000 7fd83d7bd740 1 -- 172.21.15.97:0/2420133251 wait complete. 2024-04-23T10:25:37.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.391+0000 7fd83d7bd740 20 client.15138 trim_cache size 0 max 16384 2024-04-23T10:25:37.396 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-23T10:25:37.396 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "9b22af88-0159-11ef-bc93-c7b262605968", 2024-04-23T10:25:37.396 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 1, 2024-04-23T10:25:37.396 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 24487, 2024-04-23T10:25:37.397 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-04-23T10:25:37.397 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-04-23T10:25:37.397 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-04-23T10:25:37.397 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 87.825909617999997, 2024-04-23T10:25:37.397 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 25, 2024-04-23T10:25:37.397 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 88, 2024-04-23T10:25:37.397 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 86, 2024-04-23T10:25:37.397 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 104.887916467 2024-04-23T10:25:37.397 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-23T10:25:37.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.391+0000 7fd83aaa9640 1 -- 172.21.15.97:0/3808608143 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fd8240a3bb0 msgr2=0x7fd8240a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:37.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.391+0000 7fd83aaa9640 1 --2- 172.21.15.97:0/3808608143 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fd8240a3bb0 0x7fd8240a6070 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd820000c00 tx=0x7fd820031c30 comp rx=0 tx=0).stop 2024-04-23T10:25:37.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.391+0000 7fd83aaa9640 1 -- 172.21.15.97:0/3808608143 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd834100e90 msgr2=0x7fd83406b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:37.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.391+0000 7fd83aaa9640 1 --2- 172.21.15.97:0/3808608143 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd834100e90 0x7fd83406b140 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd82800b4a0 tx=0x7fd82800b960 comp rx=0 tx=0).stop 2024-04-23T10:25:37.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.391+0000 7fd83aaa9640 1 -- 172.21.15.97:0/3808608143 shutdown_connections 2024-04-23T10:25:37.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.391+0000 7fd83aaa9640 1 --2- 172.21.15.97:0/3808608143 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fd8240a3bb0 0x7fd8240a6070 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.391+0000 7fd83aaa9640 1 --2- 172.21.15.97:0/3808608143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8341029a0 0x7fd83419f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.391+0000 7fd83aaa9640 1 --2- 172.21.15.97:0/3808608143 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd834101fe0 0x7fd83419cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.391+0000 7fd83aaa9640 1 --2- 172.21.15.97:0/3808608143 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd834100e90 0x7fd83406b140 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.391+0000 7fd83aaa9640 1 -- 172.21.15.97:0/3808608143 >> 172.21.15.97:0/3808608143 conn(0x7fd8340fc440 msgr2=0x7fd8341053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:37.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.391+0000 7fd83aaa9640 1 -- 172.21.15.97:0/3808608143 shutdown_connections 2024-04-23T10:25:37.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.391+0000 7fd83aaa9640 1 -- 172.21.15.97:0/3808608143 wait complete. 2024-04-23T10:25:37.402 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-23T10:25:37.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.501+0000 7ff21fe12640 1 -- 172.21.15.97:0/415373394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff218103210 msgr2=0x7ff218105680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:37.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.501+0000 7ff21fe12640 1 --2- 172.21.15.97:0/415373394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff218103210 0x7ff218105680 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff208009900 tx=0x7ff20802a410 comp rx=0 tx=0).stop 2024-04-23T10:25:37.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.501+0000 7ff21fe12640 1 -- 172.21.15.97:0/415373394 shutdown_connections 2024-04-23T10:25:37.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.501+0000 7ff21fe12640 1 --2- 172.21.15.97:0/415373394 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff218105bc0 0x7ff218106040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.501+0000 7ff21fe12640 1 --2- 172.21.15.97:0/415373394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff218103210 0x7ff218105680 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.501+0000 7ff21fe12640 1 --2- 172.21.15.97:0/415373394 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2181008a0 0x7ff218102cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.501+0000 7ff21fe12640 1 -- 172.21.15.97:0/415373394 >> 172.21.15.97:0/415373394 conn(0x7ff2180fc4a0 msgr2=0x7ff2180fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:37.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.501+0000 7ff21fe12640 1 -- 172.21.15.97:0/415373394 shutdown_connections 2024-04-23T10:25:37.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.501+0000 7ff21fe12640 1 -- 172.21.15.97:0/415373394 wait complete. 2024-04-23T10:25:37.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.502+0000 7ff21fe12640 1 Processor -- start 2024-04-23T10:25:37.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.502+0000 7ff21fe12640 1 -- start start 2024-04-23T10:25:37.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.502+0000 7ff21fe12640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2181008a0 0x7ff21819e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.502+0000 7ff21fe12640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff218103210 0x7ff21819ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.502+0000 7ff21fe12640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff218105bc0 0x7ff2181a5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.502+0000 7ff21fe12640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff218114160 con 0x7ff218105bc0 2024-04-23T10:25:37.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.502+0000 7ff21fe12640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff218113fe0 con 0x7ff2181008a0 2024-04-23T10:25:37.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.502+0000 7ff21fe12640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff2181142e0 con 0x7ff218103210 2024-04-23T10:25:37.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.502+0000 7ff21e3af640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff218105bc0 0x7ff2181a5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:37.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.502+0000 7ff21e3af640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff218105bc0 0x7ff2181a5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59926/0 (socket says 172.21.15.97:59926) 2024-04-23T10:25:37.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.502+0000 7ff21e3af640 1 -- 172.21.15.97:0/916090486 learned_addr learned my addr 172.21.15.97:0/916090486 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:25:37.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.502+0000 7ff21dbae640 1 --2- 172.21.15.97:0/916090486 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2181008a0 0x7ff21819e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:37.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.502+0000 7ff21d3ad640 1 --2- 172.21.15.97:0/916090486 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff218103210 0x7ff21819ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:37.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.502+0000 7ff21e3af640 1 -- 172.21.15.97:0/916090486 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff218103210 msgr2=0x7ff21819ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:37.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.502+0000 7ff21e3af640 1 --2- 172.21.15.97:0/916090486 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff218103210 0x7ff21819ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.502+0000 7ff21e3af640 1 -- 172.21.15.97:0/916090486 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2181008a0 msgr2=0x7ff21819e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:37.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.502+0000 7ff21e3af640 1 --2- 172.21.15.97:0/916090486 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2181008a0 0x7ff21819e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.502+0000 7ff21e3af640 1 -- 172.21.15.97:0/916090486 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2181a6400 con 0x7ff218105bc0 2024-04-23T10:25:37.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.503+0000 7ff21e3af640 1 --2- 172.21.15.97:0/916090486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff218105bc0 0x7ff2181a5d90 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff21000e970 tx=0x7ff21000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:37.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.503+0000 7ff20effd640 1 -- 172.21.15.97:0/916090486 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff21000ccc0 con 0x7ff218105bc0 2024-04-23T10:25:37.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.503+0000 7ff20effd640 1 -- 172.21.15.97:0/916090486 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff21000ce60 con 0x7ff218105bc0 2024-04-23T10:25:37.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.503+0000 7ff20effd640 1 -- 172.21.15.97:0/916090486 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff210010430 con 0x7ff218105bc0 2024-04-23T10:25:37.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.503+0000 7ff21fe12640 1 -- 172.21.15.97:0/916090486 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2181a66f0 con 0x7ff218105bc0 2024-04-23T10:25:37.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.503+0000 7ff21fe12640 1 -- 172.21.15.97:0/916090486 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2181a6c40 con 0x7ff218105bc0 2024-04-23T10:25:37.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.505+0000 7ff20effd640 1 -- 172.21.15.97:0/916090486 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff210004f90 con 0x7ff218105bc0 2024-04-23T10:25:37.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.505+0000 7ff222bfd740 1 Processor -- start 2024-04-23T10:25:37.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.505+0000 7ff222bfd740 1 -- start start 2024-04-23T10:25:37.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.506+0000 7ff222bfd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8585b1b80 0x55e8585b1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.506+0000 7ff222bfd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8585b2d50 0x55e8585b31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.506+0000 7ff222bfd740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8585b3710 0x55e8585b9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.506+0000 7ff222bfd740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e858516c50 con 0x55e8585b3710 2024-04-23T10:25:37.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.506+0000 7ff222bfd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e8584b3830 con 0x55e8585b1b80 2024-04-23T10:25:37.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.506+0000 7ff222bfd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e858512c10 con 0x55e8585b2d50 2024-04-23T10:25:37.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.506+0000 7ff21d3ad640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8585b3710 0x55e8585b9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:37.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.506+0000 7ff21dbae640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8585b2d50 0x55e8585b31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:37.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.506+0000 7ff21d3ad640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8585b3710 0x55e8585b9fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59936/0 (socket says 172.21.15.97:59936) 2024-04-23T10:25:37.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.506+0000 7ff21d3ad640 1 -- 172.21.15.97:0/2731972849 learned_addr learned my addr 172.21.15.97:0/2731972849 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:25:37.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.506+0000 7ff20effd640 1 --2- 172.21.15.97:0/916090486 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7ff2040a3bb0 0x7ff2040a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.506+0000 7ff21d3ad640 1 --2- 172.21.15.97:0/2731972849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8585b1b80 0x55e8585b1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:37.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.506+0000 7ff21e3af640 1 --2- 172.21.15.97:0/916090486 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7ff2040a3bb0 0x7ff2040a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:37.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.506+0000 7ff21d3ad640 1 -- 172.21.15.97:0/2731972849 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8585b2d50 msgr2=0x55e8585b31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:25:37.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.506+0000 7ff21d3ad640 1 --2- 172.21.15.97:0/2731972849 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8585b2d50 0x55e8585b31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.506+0000 7ff21d3ad640 1 -- 172.21.15.97:0/2731972849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8585b1b80 msgr2=0x55e8585b1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:25:37.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.506+0000 7ff21d3ad640 1 --2- 172.21.15.97:0/2731972849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8585b1b80 0x55e8585b1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.506+0000 7ff20effd640 1 -- 172.21.15.97:0/916090486 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7ff2100d0850 con 0x7ff218105bc0 2024-04-23T10:25:37.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.506+0000 7ff21d3ad640 1 -- 172.21.15.97:0/2731972849 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8582292d0 con 0x55e8585b3710 2024-04-23T10:25:37.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.506+0000 7ff21d3ad640 1 --2- 172.21.15.97:0/2731972849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8585b3710 0x55e8585b9fa0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff208005940 tx=0x7ff2080058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90754edec947492f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:37.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.506+0000 7ff21e3af640 1 --2- 172.21.15.97:0/916090486 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7ff2040a3bb0 0x7ff2040a6070 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff210099530 tx=0x7ff2100bd040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:37.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.506+0000 7ff203fff640 1 -- 172.21.15.97:0/2731972849 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff208035040 con 0x55e8585b3710 2024-04-23T10:25:37.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.506+0000 7ff203fff640 1 -- 172.21.15.97:0/2731972849 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2080035f0 con 0x55e8585b3710 2024-04-23T10:25:37.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.506+0000 7ff203fff640 1 -- 172.21.15.97:0/2731972849 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2080044e0 con 0x55e8585b3710 2024-04-23T10:25:37.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.507+0000 7ff222bfd740 1 -- 172.21.15.97:0/2731972849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8585b3710 msgr2=0x55e8585b9fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:37.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.507+0000 7ff222bfd740 1 --2- 172.21.15.97:0/2731972849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8585b3710 0x55e8585b9fa0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff208005940 tx=0x7ff2080058b0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.507+0000 7ff222bfd740 1 -- 172.21.15.97:0/2731972849 shutdown_connections 2024-04-23T10:25:37.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.507+0000 7ff222bfd740 1 --2- 172.21.15.97:0/2731972849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8585b3710 0x55e8585b9fa0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.507+0000 7ff222bfd740 1 --2- 172.21.15.97:0/2731972849 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8585b2d50 0x55e8585b31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.507+0000 7ff222bfd740 1 --2- 172.21.15.97:0/2731972849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8585b1b80 0x55e8585b1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.507+0000 7ff222bfd740 1 -- 172.21.15.97:0/2731972849 >> 172.21.15.97:0/2731972849 conn(0x55e8585ad070 msgr2=0x55e8585ad470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:37.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.507+0000 7ff222bfd740 1 -- 172.21.15.97:0/2731972849 shutdown_connections 2024-04-23T10:25:37.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.507+0000 7ff222bfd740 1 -- 172.21.15.97:0/2731972849 wait complete. 2024-04-23T10:25:37.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.508+0000 7ff222bfd740 1 -- start start 2024-04-23T10:25:37.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.508+0000 7ff222bfd740 1 Processor -- start 2024-04-23T10:25:37.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.508+0000 7ff222bfd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8585b1b80 0x55e8585bbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.508+0000 7ff222bfd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8585b2d50 0x55e8585bc520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.508+0000 7ff222bfd740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8585b3710 0x55e8585c3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.508+0000 7ff222bfd740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e8585ba6c0 con 0x55e8585b3710 2024-04-23T10:25:37.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.508+0000 7ff222bfd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e8584b1470 con 0x55e8585b1b80 2024-04-23T10:25:37.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.508+0000 7ff222bfd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e8585ba840 con 0x55e8585b2d50 2024-04-23T10:25:37.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.508+0000 7ff222bfd740 10 client.0 _ll_register_callbacks cb 0x55e8584d6f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:25:37.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.508+0000 7ff21d3ad640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8585b3710 0x55e8585c3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:37.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.508+0000 7ff21d3ad640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8585b3710 0x55e8585c3560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59938/0 (socket says 172.21.15.97:59938) 2024-04-23T10:25:37.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.508+0000 7ff21d3ad640 1 -- 172.21.15.97:0/993529077 learned_addr learned my addr 172.21.15.97:0/993529077 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:25:37.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.508+0000 7ff21d3ad640 1 --2- 172.21.15.97:0/993529077 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8585b1b80 0x55e8585bbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:37.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.508+0000 7ff21d3ad640 1 -- 172.21.15.97:0/993529077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8585b2d50 msgr2=0x55e8585bc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:25:37.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.508+0000 7ff21d3ad640 1 --2- 172.21.15.97:0/993529077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8585b2d50 0x55e8585bc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.508+0000 7ff21d3ad640 1 -- 172.21.15.97:0/993529077 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8585b1b80 msgr2=0x55e8585bbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:25:37.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.508+0000 7ff21d3ad640 1 --2- 172.21.15.97:0/993529077 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8585b1b80 0x55e8585bbfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.508+0000 7ff21d3ad640 1 -- 172.21.15.97:0/993529077 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8582f36b0 con 0x55e8585b3710 2024-04-23T10:25:37.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.509+0000 7ff21d3ad640 1 --2- 172.21.15.97:0/993529077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8585b3710 0x55e8585c3560 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff2080096c0 tx=0x7ff20804b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dd8095e6d3fda25 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:37.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.509+0000 7ff2017fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:25:37.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.509+0000 7ff2017fa640 1 -- 172.21.15.97:0/993529077 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff208035040 con 0x55e8585b3710 2024-04-23T10:25:37.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.509+0000 7ff2017fa640 1 -- 172.21.15.97:0/993529077 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7ff208004000 con 0x55e8585b3710 2024-04-23T10:25:37.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.509+0000 7ff222bfd740 1 -- 172.21.15.97:0/993529077 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2080035f0 con 0x55e8585b3710 2024-04-23T10:25:37.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.509+0000 7ff2017fa640 1 -- 172.21.15.97:0/993529077 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff208003bb0 con 0x55e8585b3710 2024-04-23T10:25:37.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.509+0000 7ff222bfd740 10 client.15150 fetch_fsmap learned FSMap version 25 2024-04-23T10:25:37.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.509+0000 7ff222bfd740 1 -- 172.21.15.97:0/993529077 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e8585ba6c0 con 0x55e8585b3710 2024-04-23T10:25:37.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.509+0000 7ff2017fa640 1 -- 172.21.15.97:0/993529077 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2080035f0 con 0x55e8585b3710 2024-04-23T10:25:37.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.509+0000 7ff2017fa640 1 -- 172.21.15.97:0/993529077 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ff20803a040 con 0x55e8585b3710 2024-04-23T10:25:37.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.509+0000 7ff222bfd740 10 client.15150 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T10:25:37.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.509+0000 7ff222bfd740 10 client.15150 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-23T10:25:37.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.509+0000 7ff222bfd740 20 client.15150 populate_metadata read hostname 'smithi097' 2024-04-23T10:25:37.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.509+0000 7ff222bfd740 1 --2- 172.21.15.97:0/993529077 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55e8585c8a80 0x55e8585c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.509+0000 7ff222bfd740 4 client.15150 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:25:37.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.509+0000 7ff222bfd740 1 -- 172.21.15.97:0/993529077 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e858524670 con 0x55e8585c8a80 2024-04-23T10:25:37.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.510+0000 7ff21dbae640 1 --2- 172.21.15.97:0/993529077 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55e8585c8a80 0x55e8585c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:37.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.510+0000 7ff21dbae640 1 --2- 172.21.15.97:0/993529077 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55e8585c8a80 0x55e8585c9ef0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2907248a83c41ceb server_cookie=fd0f4829f85c2fda in_seq=0 out_seq=0 2024-04-23T10:25:37.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.510+0000 7ff2017fa640 10 client.15150 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:25:37.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.511+0000 7ff2017fa640 1 -- 172.21.15.97:0/993529077 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e858524670 con 0x55e8585c8a80 2024-04-23T10:25:37.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.511+0000 7ff2017fa640 10 client.15150 handle_command_reply: tid=0 2024-04-23T10:25:37.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.511+0000 7ff222bfd740 1 client.15150 shutdown 2024-04-23T10:25:37.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.511+0000 7ff222bfd740 1 -- 172.21.15.97:0/993529077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8585b3710 msgr2=0x55e8585c3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:37.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.511+0000 7ff222bfd740 1 --2- 172.21.15.97:0/993529077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8585b3710 0x55e8585c3560 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff2080096c0 tx=0x7ff20804b040 comp rx=0 tx=0).stop 2024-04-23T10:25:37.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.511+0000 7ff222bfd740 1 -- 172.21.15.97:0/993529077 shutdown_connections 2024-04-23T10:25:37.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.511+0000 7ff222bfd740 1 --2- 172.21.15.97:0/993529077 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55e8585c8a80 0x55e8585c9ef0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.511+0000 7ff222bfd740 1 --2- 172.21.15.97:0/993529077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8585b3710 0x55e8585c3560 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.511+0000 7ff2017fa640 0 client.15150 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:25:37.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.511+0000 7ff222bfd740 1 --2- 172.21.15.97:0/993529077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8585b2d50 0x55e8585bc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.511+0000 7ff222bfd740 1 --2- 172.21.15.97:0/993529077 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8585b1b80 0x55e8585bbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.511+0000 7ff222bfd740 1 -- 172.21.15.97:0/993529077 >> 172.21.15.97:0/993529077 conn(0x55e8585ad070 msgr2=0x55e8585ac1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:37.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.511+0000 7ff222bfd740 1 -- 172.21.15.97:0/993529077 shutdown_connections 2024-04-23T10:25:37.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.511+0000 7ff222bfd740 1 -- 172.21.15.97:0/993529077 wait complete. 2024-04-23T10:25:37.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.511+0000 7ff222bfd740 20 client.15150 trim_cache size 0 max 16384 2024-04-23T10:25:37.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.529+0000 7ff222bfd740 1 Processor -- start 2024-04-23T10:25:37.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.529+0000 7ff222bfd740 1 -- start start 2024-04-23T10:25:37.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.529+0000 7ff222bfd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8585b1b80 0x55e8585bc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.529+0000 7ff222bfd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8585b2d50 0x55e8585c3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.529+0000 7ff222bfd740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8585b3710 0x55e8585b5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.529+0000 7ff222bfd740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e8585c8230 con 0x55e8585b3710 2024-04-23T10:25:37.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.529+0000 7ff222bfd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e8585c8900 con 0x55e8585b1b80 2024-04-23T10:25:37.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.529+0000 7ff222bfd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e8585c8fd0 con 0x55e8585b2d50 2024-04-23T10:25:37.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.529+0000 7ff21d3ad640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8585b3710 0x55e8585b5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:37.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.529+0000 7ff21d3ad640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8585b3710 0x55e8585b5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59948/0 (socket says 172.21.15.97:59948) 2024-04-23T10:25:37.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.529+0000 7ff21d3ad640 1 -- 172.21.15.97:0/3872099942 learned_addr learned my addr 172.21.15.97:0/3872099942 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:25:37.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.529+0000 7ff21d3ad640 1 -- 172.21.15.97:0/3872099942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8585b2d50 msgr2=0x55e8585c3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:25:37.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.529+0000 7ff21d3ad640 1 --2- 172.21.15.97:0/3872099942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8585b2d50 0x55e8585c3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.529+0000 7ff21d3ad640 1 -- 172.21.15.97:0/3872099942 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8585b1b80 msgr2=0x55e8585bc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:25:37.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.529+0000 7ff21d3ad640 1 --2- 172.21.15.97:0/3872099942 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8585b1b80 0x55e8585bc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.529+0000 7ff21d3ad640 1 -- 172.21.15.97:0/3872099942 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8585bcfd0 con 0x55e8585b3710 2024-04-23T10:25:37.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.530+0000 7ff21d3ad640 1 --2- 172.21.15.97:0/3872099942 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8585b3710 0x55e8585b5d50 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff20803dab0 tx=0x7ff20803dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f897ed071a38ea27 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:37.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.530+0000 7ff2037fe640 1 -- 172.21.15.97:0/3872099942 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff208035040 con 0x55e8585b3710 2024-04-23T10:25:37.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.530+0000 7ff2037fe640 1 -- 172.21.15.97:0/3872099942 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2080035f0 con 0x55e8585b3710 2024-04-23T10:25:37.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.530+0000 7ff2037fe640 1 -- 172.21.15.97:0/3872099942 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff20803d480 con 0x55e8585b3710 2024-04-23T10:25:37.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.530+0000 7ff222bfd740 1 -- 172.21.15.97:0/3872099942 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8585b3710 msgr2=0x55e8585b5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:37.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.530+0000 7ff222bfd740 1 --2- 172.21.15.97:0/3872099942 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8585b3710 0x55e8585b5d50 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff20803dab0 tx=0x7ff20803dd50 comp rx=0 tx=0).stop 2024-04-23T10:25:37.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.530+0000 7ff222bfd740 1 -- 172.21.15.97:0/3872099942 shutdown_connections 2024-04-23T10:25:37.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.530+0000 7ff222bfd740 1 --2- 172.21.15.97:0/3872099942 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8585b3710 0x55e8585b5d50 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.530+0000 7ff222bfd740 1 --2- 172.21.15.97:0/3872099942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8585b2d50 0x55e8585c3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.530+0000 7ff222bfd740 1 --2- 172.21.15.97:0/3872099942 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8585b1b80 0x55e8585bc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.530+0000 7ff222bfd740 1 -- 172.21.15.97:0/3872099942 >> 172.21.15.97:0/3872099942 conn(0x55e8585ad070 msgr2=0x55e8585c7a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:37.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.530+0000 7ff222bfd740 1 -- 172.21.15.97:0/3872099942 shutdown_connections 2024-04-23T10:25:37.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.530+0000 7ff222bfd740 1 -- 172.21.15.97:0/3872099942 wait complete. 2024-04-23T10:25:37.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.531+0000 7ff222bfd740 1 -- start start 2024-04-23T10:25:37.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.531+0000 7ff222bfd740 1 Processor -- start 2024-04-23T10:25:37.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.531+0000 7ff222bfd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8585b1b80 0x55e8585bf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.531+0000 7ff222bfd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8585b2d50 0x55e8585c1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.531+0000 7ff222bfd740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8585b3710 0x55e8585c21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.531+0000 7ff222bfd740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e8585c9d70 con 0x55e8585b3710 2024-04-23T10:25:37.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.531+0000 7ff222bfd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e8585c96a0 con 0x55e8585b1b80 2024-04-23T10:25:37.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.531+0000 7ff222bfd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e8585ca440 con 0x55e8585b2d50 2024-04-23T10:25:37.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.531+0000 7ff222bfd740 10 client.0 _ll_register_callbacks cb 0x7ff210005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:25:37.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.531+0000 7ff21d3ad640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8585b3710 0x55e8585c21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:37.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.531+0000 7ff21d3ad640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8585b3710 0x55e8585c21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59950/0 (socket says 172.21.15.97:59950) 2024-04-23T10:25:37.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.531+0000 7ff21d3ad640 1 -- 172.21.15.97:0/4250865514 learned_addr learned my addr 172.21.15.97:0/4250865514 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:25:37.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.531+0000 7ff21d3ad640 1 --2- 172.21.15.97:0/4250865514 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8585b1b80 0x55e8585bf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:37.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.531+0000 7ff21dbae640 1 --2- 172.21.15.97:0/4250865514 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8585b2d50 0x55e8585c1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:37.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.531+0000 7ff21d3ad640 1 -- 172.21.15.97:0/4250865514 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8585b2d50 msgr2=0x55e8585c1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:25:37.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.531+0000 7ff21d3ad640 1 --2- 172.21.15.97:0/4250865514 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8585b2d50 0x55e8585c1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.531+0000 7ff21d3ad640 1 -- 172.21.15.97:0/4250865514 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8585b1b80 msgr2=0x55e8585bf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:25:37.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.531+0000 7ff21d3ad640 1 --2- 172.21.15.97:0/4250865514 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8585b1b80 0x55e8585bf750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.531+0000 7ff21d3ad640 1 -- 172.21.15.97:0/4250865514 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8585c2920 con 0x55e8585b3710 2024-04-23T10:25:37.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.532+0000 7ff21d3ad640 1 --2- 172.21.15.97:0/4250865514 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8585b3710 0x55e8585c21e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff208055510 tx=0x7ff208005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9304353ca677bb55 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:37.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.532+0000 7ff2017fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:25:37.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.532+0000 7ff2017fa640 1 -- 172.21.15.97:0/4250865514 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff208035040 con 0x55e8585b3710 2024-04-23T10:25:37.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.532+0000 7ff2017fa640 1 -- 172.21.15.97:0/4250865514 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7ff20803d690 con 0x55e8585b3710 2024-04-23T10:25:37.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.532+0000 7ff222bfd740 1 -- 172.21.15.97:0/4250865514 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2080035f0 con 0x55e8585b3710 2024-04-23T10:25:37.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.532+0000 7ff2017fa640 1 -- 172.21.15.97:0/4250865514 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff208031df0 con 0x55e8585b3710 2024-04-23T10:25:37.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.532+0000 7ff222bfd740 10 client.15156 fetch_fsmap learned FSMap version 25 2024-04-23T10:25:37.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.532+0000 7ff222bfd740 1 -- 172.21.15.97:0/4250865514 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e85862eb20 con 0x55e8585b3710 2024-04-23T10:25:37.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.532+0000 7ff2017fa640 1 -- 172.21.15.97:0/4250865514 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2080035f0 con 0x55e8585b3710 2024-04-23T10:25:37.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.532+0000 7ff2017fa640 1 -- 172.21.15.97:0/4250865514 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ff208060430 con 0x55e8585b3710 2024-04-23T10:25:37.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.532+0000 7ff222bfd740 10 client.15156 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T10:25:37.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.532+0000 7ff222bfd740 10 client.15156 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-23T10:25:37.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.532+0000 7ff222bfd740 20 client.15156 populate_metadata read hostname 'smithi097' 2024-04-23T10:25:37.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.532+0000 7ff222bfd740 1 --2- 172.21.15.97:0/4250865514 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55e8586332b0 0x55e858631fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:37.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.532+0000 7ff222bfd740 4 client.15156 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T10:25:37.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.532+0000 7ff222bfd740 1 -- 172.21.15.97:0/4250865514 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "status"}) -- 0x55e85851e4e0 con 0x55e8586332b0 2024-04-23T10:25:37.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.533+0000 7ff21dbae640 1 --2- 172.21.15.97:0/4250865514 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55e8586332b0 0x55e858631fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:37.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.533+0000 7ff2017fa640 10 client.15156 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:25:37.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.533+0000 7ff21dbae640 1 --2- 172.21.15.97:0/4250865514 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55e8586332b0 0x55e858631fc0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48f41a6e00173f3a server_cookie=af9f31f63a981471 in_seq=0 out_seq=0 2024-04-23T10:25:37.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.533+0000 7ff2017fa640 1 -- 172.21.15.97:0/4250865514 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+320 (crc 0 0 0) 0x55e85851e4e0 con 0x55e8586332b0 2024-04-23T10:25:37.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.533+0000 7ff2017fa640 10 client.15156 handle_command_reply: tid=0 2024-04-23T10:25:37.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.533+0000 7ff222bfd740 1 client.15156 shutdown 2024-04-23T10:25:37.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.533+0000 7ff222bfd740 1 -- 172.21.15.97:0/4250865514 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8585b3710 msgr2=0x55e8585c21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:37.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.533+0000 7ff222bfd740 1 --2- 172.21.15.97:0/4250865514 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8585b3710 0x55e8585c21e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff208055510 tx=0x7ff208005e50 comp rx=0 tx=0).stop 2024-04-23T10:25:37.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.533+0000 7ff222bfd740 1 -- 172.21.15.97:0/4250865514 shutdown_connections 2024-04-23T10:25:37.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.533+0000 7ff222bfd740 1 --2- 172.21.15.97:0/4250865514 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55e8586332b0 0x55e858631fc0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.533+0000 7ff222bfd740 1 --2- 172.21.15.97:0/4250865514 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8585b3710 0x55e8585c21e0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.533+0000 7ff222bfd740 1 --2- 172.21.15.97:0/4250865514 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8585b2d50 0x55e8585c1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.533+0000 7ff222bfd740 1 --2- 172.21.15.97:0/4250865514 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8585b1b80 0x55e8585bf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.533+0000 7ff2017fa640 0 client.15156 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:25:37.539 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.533+0000 7ff222bfd740 1 -- 172.21.15.97:0/4250865514 >> 172.21.15.97:0/4250865514 conn(0x55e8585ad070 msgr2=0x55e8585a80b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:37.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.533+0000 7ff222bfd740 1 -- 172.21.15.97:0/4250865514 shutdown_connections 2024-04-23T10:25:37.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.533+0000 7ff222bfd740 1 -- 172.21.15.97:0/4250865514 wait complete. 2024-04-23T10:25:37.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.534+0000 7ff222bfd740 20 client.15156 trim_cache size 0 max 16384 2024-04-23T10:25:37.540 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-23T10:25:37.540 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "9b22af88-0159-11ef-bc93-c7b262605968", 2024-04-23T10:25:37.540 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2024-04-23T10:25:37.540 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 24488, 2024-04-23T10:25:37.540 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-04-23T10:25:37.540 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-04-23T10:25:37.540 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-04-23T10:25:37.540 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 89.979354203, 2024-04-23T10:25:37.540 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 25, 2024-04-23T10:25:37.541 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 88, 2024-04-23T10:25:37.541 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 86, 2024-04-23T10:25:37.541 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 96.536435509 2024-04-23T10:25:37.541 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-23T10:25:37.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.534+0000 7ff21fe12640 1 -- 172.21.15.97:0/916090486 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7ff2040a3bb0 msgr2=0x7ff2040a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:37.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.534+0000 7ff21fe12640 1 --2- 172.21.15.97:0/916090486 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7ff2040a3bb0 0x7ff2040a6070 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff210099530 tx=0x7ff2100bd040 comp rx=0 tx=0).stop 2024-04-23T10:25:37.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.534+0000 7ff21fe12640 1 -- 172.21.15.97:0/916090486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff218105bc0 msgr2=0x7ff2181a5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:37.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.534+0000 7ff21fe12640 1 --2- 172.21.15.97:0/916090486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff218105bc0 0x7ff2181a5d90 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff21000e970 tx=0x7ff21000ed70 comp rx=0 tx=0).stop 2024-04-23T10:25:37.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.534+0000 7ff21fe12640 1 -- 172.21.15.97:0/916090486 shutdown_connections 2024-04-23T10:25:37.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.534+0000 7ff21fe12640 1 --2- 172.21.15.97:0/916090486 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7ff2040a3bb0 0x7ff2040a6070 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.534+0000 7ff21fe12640 1 --2- 172.21.15.97:0/916090486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff218105bc0 0x7ff2181a5d90 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.534+0000 7ff21fe12640 1 --2- 172.21.15.97:0/916090486 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff218103210 0x7ff21819ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.534+0000 7ff21fe12640 1 --2- 172.21.15.97:0/916090486 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2181008a0 0x7ff21819e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:37.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.534+0000 7ff21fe12640 1 -- 172.21.15.97:0/916090486 >> 172.21.15.97:0/916090486 conn(0x7ff2180fc4a0 msgr2=0x7ff218101300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:37.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.534+0000 7ff21fe12640 1 -- 172.21.15.97:0/916090486 shutdown_connections 2024-04-23T10:25:37.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:37.534+0000 7ff21fe12640 1 -- 172.21.15.97:0/916090486 wait complete. 2024-04-23T10:25:37.545 INFO:tasks.cephadm:Setup complete, yielding 2024-04-23T10:25:37.545 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-23T10:25:37.554 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi097.front.sepia.ceph.com 2024-04-23T10:25:37.554 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-23T10:25:37.795 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T10:25:38.043 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:37 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/2696616206' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:25:38.043 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:37 smithi097 ceph-mon[29197]: pgmap v337: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-04-23T10:25:38.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:37 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/2696616206' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:25:38.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:37 smithi196 ceph-mon[32919]: pgmap v337: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-04-23T10:25:38.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:37 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/2696616206' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:25:38.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:37 smithi183 ceph-mon[32317]: pgmap v337: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-04-23T10:25:39.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:39.638+0000 7ff911f3f700 1 -- 172.21.15.97:0/3582870169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff90c101bb0 msgr2=0x7ff90c102030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:39.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:39.639+0000 7ff911f3f700 1 --2- 172.21.15.97:0/3582870169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff90c101bb0 0x7ff90c102030 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff900009990 tx=0x7ff900009ca0 comp rx=0 tx=0).stop 2024-04-23T10:25:39.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:39.639+0000 7ff911f3f700 1 -- 172.21.15.97:0/3582870169 shutdown_connections 2024-04-23T10:25:39.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:39.639+0000 7ff911f3f700 1 --2- 172.21.15.97:0/3582870169 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff90c102570 0x7ff90c106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:39.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:39.639+0000 7ff911f3f700 1 --2- 172.21.15.97:0/3582870169 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff90c100e40 0x7ff90c101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:39.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:39.639+0000 7ff911f3f700 1 --2- 172.21.15.97:0/3582870169 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff90c101bb0 0x7ff90c102030 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:39.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:39.639+0000 7ff911f3f700 1 -- 172.21.15.97:0/3582870169 >> 172.21.15.97:0/3582870169 conn(0x7ff90c0fc7a0 msgr2=0x7ff90c0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:39.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:39.639+0000 7ff911f3f700 1 -- 172.21.15.97:0/3582870169 shutdown_connections 2024-04-23T10:25:39.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:39.639+0000 7ff911f3f700 1 -- 172.21.15.97:0/3582870169 wait complete. 2024-04-23T10:25:39.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:39.639+0000 7ff911f3f700 1 Processor -- start 2024-04-23T10:25:39.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:39.639+0000 7ff911f3f700 1 -- start start 2024-04-23T10:25:39.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:39.639+0000 7ff911f3f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff90c100e40 0x7ff90c19f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:39.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:39.640+0000 7ff911f3f700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff90c101bb0 0x7ff90c19f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:39.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:39.640+0000 7ff911f3f700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff90c102570 0x7ff90c1a6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:39.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:39.640+0000 7ff911f3f700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff90c114c90 con 0x7ff90c100e40 2024-04-23T10:25:39.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:39.640+0000 7ff911f3f700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff90c114b10 con 0x7ff90c101bb0 2024-04-23T10:25:39.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:39.640+0000 7ff911f3f700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff90c114e10 con 0x7ff90c102570 2024-04-23T10:25:39.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:39.640+0000 7ff90b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff90c100e40 0x7ff90c19f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:39.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:39.640+0000 7ff90b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff90c100e40 0x7ff90c19f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59962/0 (socket says 172.21.15.97:59962) 2024-04-23T10:25:39.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:39.640+0000 7ff90b7fe700 1 -- 172.21.15.97:0/2498387170 learned_addr learned my addr 172.21.15.97:0/2498387170 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:25:39.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:39.640+0000 7ff90affd700 1 --2- 172.21.15.97:0/2498387170 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff90c101bb0 0x7ff90c19f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:39.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:39.640+0000 7ff90bfff700 1 --2- 172.21.15.97:0/2498387170 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff90c102570 0x7ff90c1a6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:39.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:39.640+0000 7ff90b7fe700 1 -- 172.21.15.97:0/2498387170 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff90c102570 msgr2=0x7ff90c1a6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:39.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:39.640+0000 7ff90b7fe700 1 --2- 172.21.15.97:0/2498387170 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff90c102570 0x7ff90c1a6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:39.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:39.640+0000 7ff90b7fe700 1 -- 172.21.15.97:0/2498387170 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff90c101bb0 msgr2=0x7ff90c19f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:39.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:39.640+0000 7ff90b7fe700 1 --2- 172.21.15.97:0/2498387170 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff90c101bb0 0x7ff90c19f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:39.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:39.640+0000 7ff90b7fe700 1 -- 172.21.15.97:0/2498387170 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff90c1a6ee0 con 0x7ff90c100e40 2024-04-23T10:25:39.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:39.640+0000 7ff90b7fe700 1 --2- 172.21.15.97:0/2498387170 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff90c100e40 0x7ff90c19f280 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc00b560 tx=0x7ff8fc00b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:39.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:39.640+0000 7ff908ff9700 1 -- 172.21.15.97:0/2498387170 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc015020 con 0x7ff90c100e40 2024-04-23T10:25:39.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:39.640+0000 7ff908ff9700 1 -- 172.21.15.97:0/2498387170 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8fc011ce0 con 0x7ff90c100e40 2024-04-23T10:25:39.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:39.640+0000 7ff911f3f700 1 -- 172.21.15.97:0/2498387170 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff90c1a71d0 con 0x7ff90c100e40 2024-04-23T10:25:39.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:39.640+0000 7ff908ff9700 1 -- 172.21.15.97:0/2498387170 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc010da0 con 0x7ff90c100e40 2024-04-23T10:25:39.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:39.640+0000 7ff911f3f700 1 -- 172.21.15.97:0/2498387170 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff90c1a76c0 con 0x7ff90c100e40 2024-04-23T10:25:39.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:39.641+0000 7ff911f3f700 1 -- 172.21.15.97:0/2498387170 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff90c069730 con 0x7ff90c100e40 2024-04-23T10:25:39.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:39.643+0000 7ff908ff9700 1 -- 172.21.15.97:0/2498387170 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff8fc018020 con 0x7ff90c100e40 2024-04-23T10:25:39.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:39.643+0000 7ff908ff9700 1 --2- 172.21.15.97:0/2498387170 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7ff8f40a3c40 0x7ff8f40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:39.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:39.643+0000 7ff90affd700 1 --2- 172.21.15.97:0/2498387170 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7ff8f40a3c40 0x7ff8f40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:39.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:39.643+0000 7ff908ff9700 1 -- 172.21.15.97:0/2498387170 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7ff8fc0d0160 con 0x7ff90c100e40 2024-04-23T10:25:39.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:39.644+0000 7ff90affd700 1 --2- 172.21.15.97:0/2498387170 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7ff8f40a3c40 0x7ff8f40a6100 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff90000b5c0 tx=0x7ff900005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:39.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:39.647+0000 7ff908ff9700 1 -- 172.21.15.97:0/2498387170 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff8fc01f020 con 0x7ff90c100e40 2024-04-23T10:25:39.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:39.866+0000 7ff911f3f700 1 -- 172.21.15.97:0/2498387170 --> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7ff90c062e80 con 0x7ff8f40a3c40 2024-04-23T10:25:39.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:39.867+0000 7ff908ff9700 1 -- 172.21.15.97:0/2498387170 <== mgr.14152 v2:172.21.15.97:6800/3149043749 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7ff90c062e80 con 0x7ff8f40a3c40 2024-04-23T10:25:39.866 INFO:teuthology.orchestra.run.smithi097.stdout:Backend: cephadm 2024-04-23T10:25:39.866 INFO:teuthology.orchestra.run.smithi097.stdout:Available: Yes 2024-04-23T10:25:39.866 INFO:teuthology.orchestra.run.smithi097.stdout:Paused: No 2024-04-23T10:25:39.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:39.869+0000 7ff911f3f700 1 -- 172.21.15.97:0/2498387170 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7ff8f40a3c40 msgr2=0x7ff8f40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:39.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:39.869+0000 7ff911f3f700 1 --2- 172.21.15.97:0/2498387170 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7ff8f40a3c40 0x7ff8f40a6100 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff90000b5c0 tx=0x7ff900005fd0 comp rx=0 tx=0).stop 2024-04-23T10:25:39.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:39.869+0000 7ff911f3f700 1 -- 172.21.15.97:0/2498387170 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff90c100e40 msgr2=0x7ff90c19f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:39.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:39.869+0000 7ff911f3f700 1 --2- 172.21.15.97:0/2498387170 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff90c100e40 0x7ff90c19f280 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc00b560 tx=0x7ff8fc00b920 comp rx=0 tx=0).stop 2024-04-23T10:25:39.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:39.869+0000 7ff911f3f700 1 -- 172.21.15.97:0/2498387170 shutdown_connections 2024-04-23T10:25:39.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:39.869+0000 7ff911f3f700 1 --2- 172.21.15.97:0/2498387170 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7ff8f40a3c40 0x7ff8f40a6100 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:39.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:39.869+0000 7ff911f3f700 1 --2- 172.21.15.97:0/2498387170 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff90c102570 0x7ff90c1a6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:39.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:39.869+0000 7ff911f3f700 1 --2- 172.21.15.97:0/2498387170 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff90c100e40 0x7ff90c19f280 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:39.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:39.869+0000 7ff911f3f700 1 --2- 172.21.15.97:0/2498387170 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff90c101bb0 0x7ff90c19f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:39.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:39.869+0000 7ff911f3f700 1 -- 172.21.15.97:0/2498387170 >> 172.21.15.97:0/2498387170 conn(0x7ff90c0fc7a0 msgr2=0x7ff90c0fd410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:39.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:39.869+0000 7ff911f3f700 1 -- 172.21.15.97:0/2498387170 shutdown_connections 2024-04-23T10:25:39.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:39.869+0000 7ff911f3f700 1 -- 172.21.15.97:0/2498387170 wait complete. 2024-04-23T10:25:40.690 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-23T10:25:40.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:40 smithi183 ceph-mon[32317]: pgmap v338: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 10 op/s 2024-04-23T10:25:40.929 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T10:25:41.052 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:40 smithi097 ceph-mon[29197]: pgmap v338: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 10 op/s 2024-04-23T10:25:41.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:40 smithi196 ceph-mon[32919]: pgmap v338: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 10 op/s 2024-04-23T10:25:41.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:41 smithi183 ceph-mon[32317]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:25:42.087 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:41 smithi097 ceph-mon[29197]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:25:42.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:41 smithi196 ceph-mon[32919]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:25:42.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:42.637+0000 7fe931919700 1 -- 172.21.15.97:0/3263284139 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe92c1019a0 msgr2=0x7fe92c101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:42.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:42.637+0000 7fe931919700 1 --2- 172.21.15.97:0/3263284139 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe92c1019a0 0x7fe92c101dc0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fe91c0099c0 tx=0x7fe91c009cd0 comp rx=0 tx=0).stop 2024-04-23T10:25:42.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:42.638+0000 7fe931919700 1 -- 172.21.15.97:0/3263284139 shutdown_connections 2024-04-23T10:25:42.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:42.638+0000 7fe931919700 1 --2- 172.21.15.97:0/3263284139 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe92c103540 0x7fe92c109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:42.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:42.638+0000 7fe931919700 1 --2- 172.21.15.97:0/3263284139 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe92c1019a0 0x7fe92c101dc0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:42.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:42.638+0000 7fe931919700 1 --2- 172.21.15.97:0/3263284139 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe92c102ba0 0x7fe92c103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:42.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:42.638+0000 7fe931919700 1 -- 172.21.15.97:0/3263284139 >> 172.21.15.97:0/3263284139 conn(0x7fe92c0fcf40 msgr2=0x7fe92c0ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:42.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:42.638+0000 7fe931919700 1 -- 172.21.15.97:0/3263284139 shutdown_connections 2024-04-23T10:25:42.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:42.638+0000 7fe931919700 1 -- 172.21.15.97:0/3263284139 wait complete. 2024-04-23T10:25:42.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:42.638+0000 7fe931919700 1 Processor -- start 2024-04-23T10:25:42.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:42.638+0000 7fe931919700 1 -- start start 2024-04-23T10:25:42.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:42.639+0000 7fe931919700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe92c1019a0 0x7fe92c06bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:42.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:42.639+0000 7fe931919700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe92c102ba0 0x7fe92c19d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:42.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:42.639+0000 7fe931919700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe92c103540 0x7fe92c19ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:42.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:42.639+0000 7fe931919700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe92c10d410 con 0x7fe92c102ba0 2024-04-23T10:25:42.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:42.639+0000 7fe931919700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe92c10d290 con 0x7fe92c1019a0 2024-04-23T10:25:42.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:42.639+0000 7fe931919700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe92c10d590 con 0x7fe92c103540 2024-04-23T10:25:42.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:42.639+0000 7fe92a7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe92c102ba0 0x7fe92c19d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:42.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:42.639+0000 7fe92a7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe92c102ba0 0x7fe92c19d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59984/0 (socket says 172.21.15.97:59984) 2024-04-23T10:25:42.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:42.639+0000 7fe92a7fc700 1 -- 172.21.15.97:0/1470085895 learned_addr learned my addr 172.21.15.97:0/1470085895 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:25:42.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:42.639+0000 7fe92affd700 1 --2- 172.21.15.97:0/1470085895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe92c1019a0 0x7fe92c06bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:42.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:42.639+0000 7fe92b7fe700 1 --2- 172.21.15.97:0/1470085895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe92c103540 0x7fe92c19ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:42.640 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:42.639+0000 7fe92a7fc700 1 -- 172.21.15.97:0/1470085895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe92c103540 msgr2=0x7fe92c19ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:42.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:42.639+0000 7fe92a7fc700 1 --2- 172.21.15.97:0/1470085895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe92c103540 0x7fe92c19ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:42.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:42.639+0000 7fe92a7fc700 1 -- 172.21.15.97:0/1470085895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe92c1019a0 msgr2=0x7fe92c06bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:42.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:42.639+0000 7fe92a7fc700 1 --2- 172.21.15.97:0/1470085895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe92c1019a0 0x7fe92c06bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:42.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:42.639+0000 7fe92a7fc700 1 -- 172.21.15.97:0/1470085895 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe92c1a0840 con 0x7fe92c102ba0 2024-04-23T10:25:42.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:42.639+0000 7fe92a7fc700 1 --2- 172.21.15.97:0/1470085895 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe92c102ba0 0x7fe92c19d930 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe92000d980 tx=0x7fe92000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:42.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:42.639+0000 7fe930917700 1 -- 172.21.15.97:0/1470085895 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe920016070 con 0x7fe92c102ba0 2024-04-23T10:25:42.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:42.640+0000 7fe930917700 1 -- 172.21.15.97:0/1470085895 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe920004830 con 0x7fe92c102ba0 2024-04-23T10:25:42.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:42.640+0000 7fe931919700 1 -- 172.21.15.97:0/1470085895 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe92c1a0b30 con 0x7fe92c102ba0 2024-04-23T10:25:42.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:42.640+0000 7fe930917700 1 -- 172.21.15.97:0/1470085895 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe920005540 con 0x7fe92c102ba0 2024-04-23T10:25:42.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:42.640+0000 7fe931919700 1 -- 172.21.15.97:0/1470085895 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe92c1a1070 con 0x7fe92c102ba0 2024-04-23T10:25:42.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:42.641+0000 7fe931919700 1 -- 172.21.15.97:0/1470085895 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe92c069730 con 0x7fe92c102ba0 2024-04-23T10:25:42.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:42.642+0000 7fe930917700 1 -- 172.21.15.97:0/1470085895 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe920004220 con 0x7fe92c102ba0 2024-04-23T10:25:42.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:42.642+0000 7fe930917700 1 --2- 172.21.15.97:0/1470085895 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fe9140a3c40 0x7fe9140a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:42.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:42.642+0000 7fe92affd700 1 --2- 172.21.15.97:0/1470085895 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fe9140a3c40 0x7fe9140a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:42.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:42.642+0000 7fe930917700 1 -- 172.21.15.97:0/1470085895 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fe9200d01c0 con 0x7fe92c102ba0 2024-04-23T10:25:42.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:42.643+0000 7fe92affd700 1 --2- 172.21.15.97:0/1470085895 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fe9140a3c40 0x7fe9140a6100 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe91c006010 tx=0x7fe91c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:42.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:42.647+0000 7fe930917700 1 -- 172.21.15.97:0/1470085895 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe92001f020 con 0x7fe92c102ba0 2024-04-23T10:25:42.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:42.855+0000 7fe931919700 1 -- 172.21.15.97:0/1470085895 --> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fe92c062e80 con 0x7fe9140a3c40 2024-04-23T10:25:42.855 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:42 smithi097 ceph-mon[29197]: pgmap v339: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 12 op/s 2024-04-23T10:25:42.868 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:42.869+0000 7fe930917700 1 -- 172.21.15.97:0/1470085895 <== mgr.14152 v2:172.21.15.97:6800/3149043749 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7fe92c062e80 con 0x7fe9140a3c40 2024-04-23T10:25:42.869 INFO:teuthology.orchestra.run.smithi097.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T10:25:42.869 INFO:teuthology.orchestra.run.smithi097.stdout:mds.a smithi097 running (105s) 18s ago 104s 17.1M - 19.0.0-3156-gd69f13d8 da4b004c09e8 1ac5fb4846ea 2024-04-23T10:25:42.869 INFO:teuthology.orchestra.run.smithi097.stdout:mds.b smithi183 running (102s) 36s ago 101s 28.7M - 19.0.0-3156-gd69f13d8 da4b004c09e8 fddaf1fb6bb4 2024-04-23T10:25:42.869 INFO:teuthology.orchestra.run.smithi097.stdout:mds.c smithi196 running (116s) 35s ago 116s 20.4M - 19.0.0-3156-gd69f13d8 da4b004c09e8 5ceaaab165ca 2024-04-23T10:25:42.869 INFO:teuthology.orchestra.run.smithi097.stdout:mds.d smithi097 running (2m) 18s ago 2m 15.5M - 19.0.0-3156-gd69f13d8 da4b004c09e8 ed03f3b8a268 2024-04-23T10:25:42.869 INFO:teuthology.orchestra.run.smithi097.stdout:mds.e smithi183 running (2m) 36s ago 2m 15.9M - 19.0.0-3156-gd69f13d8 da4b004c09e8 bb92e4a07cf0 2024-04-23T10:25:42.869 INFO:teuthology.orchestra.run.smithi097.stdout:mds.f smithi196 running (119s) 35s ago 118s 17.0M - 19.0.0-3156-gd69f13d8 da4b004c09e8 5fa315c4ec00 2024-04-23T10:25:42.869 INFO:teuthology.orchestra.run.smithi097.stdout:mds.g smithi097 running (108s) 18s ago 107s 17.2M - 19.0.0-3156-gd69f13d8 da4b004c09e8 ac5e05633d0d 2024-04-23T10:25:42.869 INFO:teuthology.orchestra.run.smithi097.stdout:mds.h smithi183 running (113s) 36s ago 113s 19.2M - 19.0.0-3156-gd69f13d8 da4b004c09e8 0340b54e4316 2024-04-23T10:25:42.869 INFO:teuthology.orchestra.run.smithi097.stdout:mds.i smithi196 running (110s) 35s ago 110s 17.0M - 19.0.0-3156-gd69f13d8 da4b004c09e8 13dc06b92bdf 2024-04-23T10:25:42.869 INFO:teuthology.orchestra.run.smithi097.stdout:mds.j smithi097 running (2m) 18s ago 2m 16.3M - 19.0.0-3156-gd69f13d8 da4b004c09e8 38fb77b00746 2024-04-23T10:25:42.869 INFO:teuthology.orchestra.run.smithi097.stdout:mds.k smithi183 running (2m) 36s ago 2m 17.8M - 19.0.0-3156-gd69f13d8 da4b004c09e8 6d907df4143e 2024-04-23T10:25:42.869 INFO:teuthology.orchestra.run.smithi097.stdout:mds.l smithi196 running (2m) 35s ago 2m 17.9M - 19.0.0-3156-gd69f13d8 da4b004c09e8 d00e288d04d9 2024-04-23T10:25:42.870 INFO:teuthology.orchestra.run.smithi097.stdout:mgr.x smithi097 *:9283,8765 running (9m) 18s ago 9m 515M - 19.0.0-3156-gd69f13d8 da4b004c09e8 1f0c379183dd 2024-04-23T10:25:42.870 INFO:teuthology.orchestra.run.smithi097.stdout:mgr.y smithi183 *:8443,8765 running (7m) 36s ago 7m 452M - 19.0.0-3156-gd69f13d8 da4b004c09e8 6ba1874f1225 2024-04-23T10:25:42.870 INFO:teuthology.orchestra.run.smithi097.stdout:mgr.z smithi196 *:8443,8765 running (7m) 35s ago 7m 454M - 19.0.0-3156-gd69f13d8 da4b004c09e8 488569418dfa 2024-04-23T10:25:42.870 INFO:teuthology.orchestra.run.smithi097.stdout:mon.a smithi097 running (9m) 18s ago 9m 51.5M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 f2859bdee5d0 2024-04-23T10:25:42.870 INFO:teuthology.orchestra.run.smithi097.stdout:mon.b smithi183 running (7m) 36s ago 7m 41.5M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 7e8f30ab14b8 2024-04-23T10:25:42.870 INFO:teuthology.orchestra.run.smithi097.stdout:mon.c smithi196 running (7m) 35s ago 7m 36.7M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 2505a701babd 2024-04-23T10:25:42.870 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0 smithi097 running (7m) 18s ago 7m 56.0M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 5a27b7127abe 2024-04-23T10:25:42.870 INFO:teuthology.orchestra.run.smithi097.stdout:osd.1 smithi183 running (6m) 36s ago 6m 45.9M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 0f8de5477269 2024-04-23T10:25:42.870 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2 smithi196 running (6m) 35s ago 6m 46.9M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 f1b76491119e 2024-04-23T10:25:42.870 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3 smithi097 running (6m) 18s ago 6m 53.8M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 bcf103c5245d 2024-04-23T10:25:42.870 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4 smithi183 running (5m) 36s ago 5m 53.5M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 cfbf02a3871c 2024-04-23T10:25:42.870 INFO:teuthology.orchestra.run.smithi097.stdout:osd.5 smithi196 running (5m) 35s ago 5m 58.0M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 aa998b567460 2024-04-23T10:25:42.870 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6 smithi097 running (4m) 18s ago 4m 51.4M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 8c5f7ac7861f 2024-04-23T10:25:42.870 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7 smithi183 running (4m) 36s ago 4m 54.2M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 b8fd49bcf7cb 2024-04-23T10:25:42.870 INFO:teuthology.orchestra.run.smithi097.stdout:osd.8 smithi196 running (3m) 35s ago 3m 45.0M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 854ca2fb9aa1 2024-04-23T10:25:42.871 INFO:teuthology.orchestra.run.smithi097.stdout:osd.9 smithi097 running (3m) 18s ago 3m 42.1M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 a3ffebf4ff7e 2024-04-23T10:25:42.871 INFO:teuthology.orchestra.run.smithi097.stdout:osd.10 smithi183 running (3m) 36s ago 3m 42.7M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 5ff072c98c74 2024-04-23T10:25:42.871 INFO:teuthology.orchestra.run.smithi097.stdout:osd.11 smithi196 running (2m) 35s ago 2m 41.5M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 dd014384992f 2024-04-23T10:25:42.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:42.871+0000 7fe931919700 1 -- 172.21.15.97:0/1470085895 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fe9140a3c40 msgr2=0x7fe9140a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:42.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:42.871+0000 7fe931919700 1 --2- 172.21.15.97:0/1470085895 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fe9140a3c40 0x7fe9140a6100 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe91c006010 tx=0x7fe91c000bc0 comp rx=0 tx=0).stop 2024-04-23T10:25:42.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:42.871+0000 7fe931919700 1 -- 172.21.15.97:0/1470085895 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe92c102ba0 msgr2=0x7fe92c19d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:42.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:42.871+0000 7fe931919700 1 --2- 172.21.15.97:0/1470085895 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe92c102ba0 0x7fe92c19d930 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe92000d980 tx=0x7fe92000dc90 comp rx=0 tx=0).stop 2024-04-23T10:25:42.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:42.871+0000 7fe931919700 1 -- 172.21.15.97:0/1470085895 shutdown_connections 2024-04-23T10:25:42.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:42.871+0000 7fe931919700 1 --2- 172.21.15.97:0/1470085895 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fe9140a3c40 0x7fe9140a6100 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:42.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:42.871+0000 7fe931919700 1 --2- 172.21.15.97:0/1470085895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe92c103540 0x7fe92c19ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:42.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:42.871+0000 7fe931919700 1 --2- 172.21.15.97:0/1470085895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe92c1019a0 0x7fe92c06bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:42.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:42.871+0000 7fe931919700 1 --2- 172.21.15.97:0/1470085895 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe92c102ba0 0x7fe92c19d930 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:42.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:42.871+0000 7fe931919700 1 -- 172.21.15.97:0/1470085895 >> 172.21.15.97:0/1470085895 conn(0x7fe92c0fcf40 msgr2=0x7fe92c105fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:42.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:42.872+0000 7fe931919700 1 -- 172.21.15.97:0/1470085895 shutdown_connections 2024-04-23T10:25:42.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:42.872+0000 7fe931919700 1 -- 172.21.15.97:0/1470085895 wait complete. 2024-04-23T10:25:42.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:42 smithi183 ceph-mon[32317]: pgmap v339: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 12 op/s 2024-04-23T10:25:43.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:42 smithi196 ceph-mon[32919]: pgmap v339: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 12 op/s 2024-04-23T10:25:43.601 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-23T10:25:43.844 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T10:25:44.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:44 smithi183 ceph-mon[32317]: from='client.15168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:25:44.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:44 smithi183 ceph-mon[32317]: pgmap v340: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-23T10:25:44.963 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:44 smithi097 ceph-mon[29197]: from='client.15168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:25:44.963 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:44 smithi097 ceph-mon[29197]: pgmap v340: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-23T10:25:45.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:44 smithi196 ceph-mon[32919]: from='client.15168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:25:45.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:44 smithi196 ceph-mon[32919]: pgmap v340: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 170 B/s wr, 9 op/s 2024-04-23T10:25:45.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:45.539+0000 7ff81bc74700 1 -- 172.21.15.97:0/3921236824 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff814106680 msgr2=0x7ff814106b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:45.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:45.539+0000 7ff81bc74700 1 --2- 172.21.15.97:0/3921236824 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff814106680 0x7ff814106b00 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff80c00b0a0 tx=0x7ff80c00b3b0 comp rx=0 tx=0).stop 2024-04-23T10:25:45.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:45.539+0000 7ff81bc74700 1 -- 172.21.15.97:0/3921236824 shutdown_connections 2024-04-23T10:25:45.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:45.539+0000 7ff81bc74700 1 --2- 172.21.15.97:0/3921236824 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff814106680 0x7ff814106b00 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:45.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:45.539+0000 7ff81bc74700 1 --2- 172.21.15.97:0/3921236824 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff814101340 0x7ff814103770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:45.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:45.539+0000 7ff81bc74700 1 --2- 172.21.15.97:0/3921236824 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff814103cb0 0x7ff814106140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:45.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:45.539+0000 7ff81bc74700 1 -- 172.21.15.97:0/3921236824 >> 172.21.15.97:0/3921236824 conn(0x7ff8140fcfa0 msgr2=0x7ff8140ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:45.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:45.539+0000 7ff81bc74700 1 -- 172.21.15.97:0/3921236824 shutdown_connections 2024-04-23T10:25:45.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:45.539+0000 7ff81bc74700 1 -- 172.21.15.97:0/3921236824 wait complete. 2024-04-23T10:25:45.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:45.539+0000 7ff81bc74700 1 Processor -- start 2024-04-23T10:25:45.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:45.539+0000 7ff81bc74700 1 -- start start 2024-04-23T10:25:45.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:45.540+0000 7ff81bc74700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff814101340 0x7ff81419f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:45.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:45.540+0000 7ff81bc74700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff814103cb0 0x7ff81419f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:45.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:45.540+0000 7ff81bc74700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff814106680 0x7ff8141a6740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:45.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:45.540+0000 7ff81bc74700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff814114b30 con 0x7ff814103cb0 2024-04-23T10:25:45.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:45.540+0000 7ff81bc74700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff8141149b0 con 0x7ff814101340 2024-04-23T10:25:45.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:45.540+0000 7ff81bc74700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff814114cb0 con 0x7ff814106680 2024-04-23T10:25:45.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:45.540+0000 7ff81920f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff814103cb0 0x7ff81419f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:45.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:45.540+0000 7ff81920f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff814103cb0 0x7ff81419f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58448/0 (socket says 172.21.15.97:58448) 2024-04-23T10:25:45.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:45.540+0000 7ff81920f700 1 -- 172.21.15.97:0/3295813670 learned_addr learned my addr 172.21.15.97:0/3295813670 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:25:45.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:45.540+0000 7ff819a10700 1 --2- 172.21.15.97:0/3295813670 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff814101340 0x7ff81419f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:45.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:45.540+0000 7ff81a211700 1 --2- 172.21.15.97:0/3295813670 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff814106680 0x7ff8141a6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:45.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:45.540+0000 7ff81920f700 1 -- 172.21.15.97:0/3295813670 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff814106680 msgr2=0x7ff8141a6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:45.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:45.540+0000 7ff81920f700 1 --2- 172.21.15.97:0/3295813670 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff814106680 0x7ff8141a6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:45.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:45.540+0000 7ff81920f700 1 -- 172.21.15.97:0/3295813670 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff814101340 msgr2=0x7ff81419f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:45.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:45.540+0000 7ff81920f700 1 --2- 172.21.15.97:0/3295813670 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff814101340 0x7ff81419f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:45.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:45.540+0000 7ff81920f700 1 -- 172.21.15.97:0/3295813670 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8141a6de0 con 0x7ff814103cb0 2024-04-23T10:25:45.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:45.540+0000 7ff81a211700 1 --2- 172.21.15.97:0/3295813670 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff814106680 0x7ff8141a6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:25:45.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:45.540+0000 7ff81920f700 1 --2- 172.21.15.97:0/3295813670 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff814103cb0 0x7ff81419f710 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff80400d980 tx=0x7ff80400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:45.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:45.540+0000 7ff80affd700 1 -- 172.21.15.97:0/3295813670 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff804016070 con 0x7ff814103cb0 2024-04-23T10:25:45.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:45.540+0000 7ff81bc74700 1 -- 172.21.15.97:0/3295813670 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8141a70d0 con 0x7ff814103cb0 2024-04-23T10:25:45.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:45.540+0000 7ff80affd700 1 -- 172.21.15.97:0/3295813670 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff80400de70 con 0x7ff814103cb0 2024-04-23T10:25:45.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:45.540+0000 7ff80affd700 1 -- 172.21.15.97:0/3295813670 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff80400f3c0 con 0x7ff814103cb0 2024-04-23T10:25:45.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:45.540+0000 7ff81bc74700 1 -- 172.21.15.97:0/3295813670 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8141a76e0 con 0x7ff814103cb0 2024-04-23T10:25:45.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:45.541+0000 7ff81bc74700 1 -- 172.21.15.97:0/3295813670 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff814069730 con 0x7ff814103cb0 2024-04-23T10:25:45.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:45.543+0000 7ff80affd700 1 -- 172.21.15.97:0/3295813670 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff804019070 con 0x7ff814103cb0 2024-04-23T10:25:45.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:45.544+0000 7ff80affd700 1 --2- 172.21.15.97:0/3295813670 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7ff8000a3c40 0x7ff8000a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:45.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:45.544+0000 7ff819a10700 1 --2- 172.21.15.97:0/3295813670 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7ff8000a3c40 0x7ff8000a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:45.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:45.544+0000 7ff80affd700 1 -- 172.21.15.97:0/3295813670 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7ff8040cfec0 con 0x7ff814103cb0 2024-04-23T10:25:45.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:45.544+0000 7ff819a10700 1 --2- 172.21.15.97:0/3295813670 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7ff8000a3c40 0x7ff8000a6100 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff810009a10 tx=0x7ff810009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:45.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:45.547+0000 7ff80affd700 1 -- 172.21.15.97:0/3295813670 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff80409c8d0 con 0x7ff814103cb0 2024-04-23T10:25:45.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:45.758+0000 7ff81bc74700 1 -- 172.21.15.97:0/3295813670 --> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7ff814062e80 con 0x7ff8000a3c40 2024-04-23T10:25:45.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:45.764+0000 7ff80affd700 1 -- 172.21.15.97:0/3295813670 <== mgr.14152 v2:172.21.15.97:6800/3149043749 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7ff814062e80 con 0x7ff8000a3c40 2024-04-23T10:25:45.763 INFO:teuthology.orchestra.run.smithi097.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-23T10:25:45.763 INFO:teuthology.orchestra.run.smithi097.stdout:mds.all 12/12 38s ago 2m smithi097=a;smithi097=d;smithi097=g;smithi097=j;smithi183=b;smithi183=e;smithi183=h;smithi183=k;smithi196=c;smithi196=f;smithi196=i;smithi196=l;count:12 2024-04-23T10:25:45.763 INFO:teuthology.orchestra.run.smithi097.stdout:mgr 3/3 38s ago 7m smithi097=x;smithi183=y;smithi196=z;count:3 2024-04-23T10:25:45.763 INFO:teuthology.orchestra.run.smithi097.stdout:mon 3/3 38s ago 8m smithi097:172.21.15.97=a;smithi183:172.21.15.183=b;smithi196:172.21.15.196=c;count:3 2024-04-23T10:25:45.763 INFO:teuthology.orchestra.run.smithi097.stdout:osd 12 38s ago - 2024-04-23T10:25:45.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:45.766+0000 7ff81bc74700 1 -- 172.21.15.97:0/3295813670 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7ff8000a3c40 msgr2=0x7ff8000a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:45.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:45.766+0000 7ff81bc74700 1 --2- 172.21.15.97:0/3295813670 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7ff8000a3c40 0x7ff8000a6100 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff810009a10 tx=0x7ff810009190 comp rx=0 tx=0).stop 2024-04-23T10:25:45.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:45.766+0000 7ff81bc74700 1 -- 172.21.15.97:0/3295813670 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff814103cb0 msgr2=0x7ff81419f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:45.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:45.766+0000 7ff81bc74700 1 --2- 172.21.15.97:0/3295813670 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff814103cb0 0x7ff81419f710 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff80400d980 tx=0x7ff80400dc90 comp rx=0 tx=0).stop 2024-04-23T10:25:45.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:45.766+0000 7ff81bc74700 1 -- 172.21.15.97:0/3295813670 shutdown_connections 2024-04-23T10:25:45.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:45.766+0000 7ff81bc74700 1 --2- 172.21.15.97:0/3295813670 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7ff8000a3c40 0x7ff8000a6100 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:45.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:45.766+0000 7ff81bc74700 1 --2- 172.21.15.97:0/3295813670 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff814106680 0x7ff8141a6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:45.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:45.766+0000 7ff81bc74700 1 --2- 172.21.15.97:0/3295813670 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff814101340 0x7ff81419f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:45.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:45.766+0000 7ff81bc74700 1 --2- 172.21.15.97:0/3295813670 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff814103cb0 0x7ff81419f710 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:45.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:45.766+0000 7ff81bc74700 1 -- 172.21.15.97:0/3295813670 >> 172.21.15.97:0/3295813670 conn(0x7ff8140fcfa0 msgr2=0x7ff814101ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:45.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:45.766+0000 7ff81bc74700 1 -- 172.21.15.97:0/3295813670 shutdown_connections 2024-04-23T10:25:45.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:45.766+0000 7ff81bc74700 1 -- 172.21.15.97:0/3295813670 wait complete. 2024-04-23T10:25:46.465 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-23T10:25:46.705 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T10:25:46.827 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:46 smithi097 ceph-mon[29197]: pgmap v341: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-23T10:25:46.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:46 smithi183 ceph-mon[32317]: pgmap v341: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-23T10:25:47.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:46 smithi196 ceph-mon[32919]: pgmap v341: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-23T10:25:47.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:47 smithi183 ceph-mon[32317]: from='client.15174 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:25:47.987 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:47 smithi097 ceph-mon[29197]: from='client.15174 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:25:48.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:47 smithi196 ceph-mon[32919]: from='client.15174 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:25:48.574 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:48.573+0000 7f7f1c650700 1 -- 172.21.15.97:0/2270262820 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f14104640 msgr2=0x7f7f1410aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:48.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:48.573+0000 7f7f1c650700 1 --2- 172.21.15.97:0/2270262820 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f14104640 0x7f7f1410aef0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c00b0d0 tx=0x7f7f0c00b3e0 comp rx=0 tx=0).stop 2024-04-23T10:25:48.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:48.573+0000 7f7f1c650700 1 -- 172.21.15.97:0/2270262820 shutdown_connections 2024-04-23T10:25:48.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:48.573+0000 7f7f1c650700 1 --2- 172.21.15.97:0/2270262820 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f14104640 0x7f7f1410aef0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:48.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:48.573+0000 7f7f1c650700 1 --2- 172.21.15.97:0/2270262820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f14102aa0 0x7f7f14102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:48.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:48.573+0000 7f7f1c650700 1 --2- 172.21.15.97:0/2270262820 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7f14103ca0 0x7f7f14104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:48.575 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:48.573+0000 7f7f1c650700 1 -- 172.21.15.97:0/2270262820 >> 172.21.15.97:0/2270262820 conn(0x7f7f140fe040 msgr2=0x7f7f14100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:48.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:48.574+0000 7f7f1c650700 1 -- 172.21.15.97:0/2270262820 shutdown_connections 2024-04-23T10:25:48.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:48.574+0000 7f7f1c650700 1 -- 172.21.15.97:0/2270262820 wait complete. 2024-04-23T10:25:48.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:48.574+0000 7f7f1c650700 1 Processor -- start 2024-04-23T10:25:48.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:48.574+0000 7f7f1c650700 1 -- start start 2024-04-23T10:25:48.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:48.574+0000 7f7f1c650700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f14102aa0 0x7f7f14198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:48.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:48.574+0000 7f7f1c650700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7f14103ca0 0x7f7f14199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:48.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:48.574+0000 7f7f1c650700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f14104640 0x7f7f141a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:48.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:48.574+0000 7f7f1c650700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7f1410e5b0 con 0x7f7f14104640 2024-04-23T10:25:48.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:48.574+0000 7f7f1c650700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7f1410e430 con 0x7f7f14103ca0 2024-04-23T10:25:48.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:48.574+0000 7f7f1c650700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7f1410e730 con 0x7f7f14102aa0 2024-04-23T10:25:48.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:48.574+0000 7f7f1abed700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f14104640 0x7f7f141a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:48.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:48.574+0000 7f7f1abed700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f14104640 0x7f7f141a0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58456/0 (socket says 172.21.15.97:58456) 2024-04-23T10:25:48.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:48.574+0000 7f7f1abed700 1 -- 172.21.15.97:0/37476818 learned_addr learned my addr 172.21.15.97:0/37476818 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:25:48.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:48.574+0000 7f7f19beb700 1 --2- 172.21.15.97:0/37476818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7f14103ca0 0x7f7f14199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:48.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:48.574+0000 7f7f1a3ec700 1 --2- 172.21.15.97:0/37476818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f14102aa0 0x7f7f14198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:48.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:48.575+0000 7f7f1abed700 1 -- 172.21.15.97:0/37476818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f14102aa0 msgr2=0x7f7f14198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:48.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:48.575+0000 7f7f1abed700 1 --2- 172.21.15.97:0/37476818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f14102aa0 0x7f7f14198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:48.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:48.575+0000 7f7f1abed700 1 -- 172.21.15.97:0/37476818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7f14103ca0 msgr2=0x7f7f14199150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:48.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:48.575+0000 7f7f1abed700 1 --2- 172.21.15.97:0/37476818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7f14103ca0 0x7f7f14199150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:48.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:48.575+0000 7f7f1abed700 1 -- 172.21.15.97:0/37476818 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f141a0860 con 0x7f7f14104640 2024-04-23T10:25:48.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:48.575+0000 7f7f1abed700 1 --2- 172.21.15.97:0/37476818 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f14104640 0x7f7f141a0190 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c009040 tx=0x7f7f0c004d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:48.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:48.575+0000 7f7f0b7fe700 1 -- 172.21.15.97:0/37476818 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c01f030 con 0x7f7f14104640 2024-04-23T10:25:48.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:48.575+0000 7f7f1c650700 1 -- 172.21.15.97:0/37476818 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f141a0af0 con 0x7f7f14104640 2024-04-23T10:25:48.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:48.575+0000 7f7f0b7fe700 1 -- 172.21.15.97:0/37476818 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f0c003a00 con 0x7f7f14104640 2024-04-23T10:25:48.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:48.575+0000 7f7f0b7fe700 1 -- 172.21.15.97:0/37476818 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c0046e0 con 0x7f7f14104640 2024-04-23T10:25:48.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:48.575+0000 7f7f1c650700 1 -- 172.21.15.97:0/37476818 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f141a0fc0 con 0x7f7f14104640 2024-04-23T10:25:48.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:48.576+0000 7f7f1c650700 1 -- 172.21.15.97:0/37476818 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f1402df80 con 0x7f7f14104640 2024-04-23T10:25:48.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:48.577+0000 7f7f0b7fe700 1 -- 172.21.15.97:0/37476818 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7f0c003bb0 con 0x7f7f14104640 2024-04-23T10:25:48.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:48.578+0000 7f7f0b7fe700 1 --2- 172.21.15.97:0/37476818 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f7f000a3c40 0x7f7f000a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:48.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:48.578+0000 7f7f1a3ec700 1 --2- 172.21.15.97:0/37476818 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f7f000a3c40 0x7f7f000a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:48.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:48.578+0000 7f7f0b7fe700 1 -- 172.21.15.97:0/37476818 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f7f0c00f020 con 0x7f7f14104640 2024-04-23T10:25:48.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:48.578+0000 7f7f1a3ec700 1 --2- 172.21.15.97:0/37476818 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f7f000a3c40 0x7f7f000a6100 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7f1419a0f0 tx=0x7f7f04009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:48.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:48.582+0000 7f7f0b7fe700 1 -- 172.21.15.97:0/37476818 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7f0c029020 con 0x7f7f14104640 2024-04-23T10:25:48.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:48.788+0000 7f7f1c650700 1 -- 172.21.15.97:0/37476818 --> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f7f14062e80 con 0x7f7f000a3c40 2024-04-23T10:25:48.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:48.790+0000 7f7f0b7fe700 1 -- 172.21.15.97:0/37476818 <== mgr.14152 v2:172.21.15.97:6800/3149043749 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f7f14062e80 con 0x7f7f000a3c40 2024-04-23T10:25:48.789 INFO:teuthology.orchestra.run.smithi097.stdout:HOST ADDR LABELS STATUS 2024-04-23T10:25:48.789 INFO:teuthology.orchestra.run.smithi097.stdout:smithi097 172.21.15.97 2024-04-23T10:25:48.789 INFO:teuthology.orchestra.run.smithi097.stdout:smithi183 172.21.15.183 2024-04-23T10:25:48.789 INFO:teuthology.orchestra.run.smithi097.stdout:smithi196 172.21.15.196 2024-04-23T10:25:48.789 INFO:teuthology.orchestra.run.smithi097.stdout:3 hosts in cluster 2024-04-23T10:25:48.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:48.792+0000 7f7f1c650700 1 -- 172.21.15.97:0/37476818 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f7f000a3c40 msgr2=0x7f7f000a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:48.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:48.792+0000 7f7f1c650700 1 --2- 172.21.15.97:0/37476818 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f7f000a3c40 0x7f7f000a6100 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7f1419a0f0 tx=0x7f7f04009190 comp rx=0 tx=0).stop 2024-04-23T10:25:48.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:48.792+0000 7f7f1c650700 1 -- 172.21.15.97:0/37476818 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f14104640 msgr2=0x7f7f141a0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:48.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:48.792+0000 7f7f1c650700 1 --2- 172.21.15.97:0/37476818 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f14104640 0x7f7f141a0190 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c009040 tx=0x7f7f0c004d50 comp rx=0 tx=0).stop 2024-04-23T10:25:48.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:48.792+0000 7f7f1c650700 1 -- 172.21.15.97:0/37476818 shutdown_connections 2024-04-23T10:25:48.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:48.792+0000 7f7f1c650700 1 --2- 172.21.15.97:0/37476818 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f7f000a3c40 0x7f7f000a6100 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:48.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:48.792+0000 7f7f1c650700 1 --2- 172.21.15.97:0/37476818 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f14104640 0x7f7f141a0190 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:48.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:48.792+0000 7f7f1c650700 1 --2- 172.21.15.97:0/37476818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f14102aa0 0x7f7f14198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:48.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:48.792+0000 7f7f1c650700 1 --2- 172.21.15.97:0/37476818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7f14103ca0 0x7f7f14199150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:48.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:48.792+0000 7f7f1c650700 1 -- 172.21.15.97:0/37476818 >> 172.21.15.97:0/37476818 conn(0x7f7f140fe040 msgr2=0x7f7f14106fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:48.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:48.792+0000 7f7f1c650700 1 -- 172.21.15.97:0/37476818 shutdown_connections 2024-04-23T10:25:48.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:48.792+0000 7f7f1c650700 1 -- 172.21.15.97:0/37476818 wait complete. 2024-04-23T10:25:48.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:48 smithi097 ceph-mon[29197]: pgmap v342: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-04-23T10:25:48.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:48 smithi183 ceph-mon[32317]: pgmap v342: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-04-23T10:25:49.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:48 smithi196 ceph-mon[32919]: pgmap v342: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-04-23T10:25:49.618 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-23T10:25:49.859 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T10:25:50.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:50 smithi183 ceph-mon[32317]: from='client.15180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:25:50.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:50 smithi183 ceph-mon[32317]: pgmap v343: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-23T10:25:51.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:50 smithi196 ceph-mon[32919]: from='client.15180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:25:51.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:50 smithi196 ceph-mon[32919]: pgmap v343: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-23T10:25:51.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:50 smithi097 ceph-mon[29197]: from='client.15180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:25:51.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:50 smithi097 ceph-mon[29197]: pgmap v343: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-23T10:25:51.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:51.708+0000 7f7ed2aa5700 1 -- 172.21.15.97:0/2989450441 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ecc104670 msgr2=0x7f7ecc10af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:51.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:51.708+0000 7f7ed2aa5700 1 --2- 172.21.15.97:0/2989450441 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ecc104670 0x7f7ecc10af20 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7ec400b0d0 tx=0x7f7ec400b3e0 comp rx=0 tx=0).stop 2024-04-23T10:25:51.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:51.708+0000 7f7ed2aa5700 1 -- 172.21.15.97:0/2989450441 shutdown_connections 2024-04-23T10:25:51.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:51.708+0000 7f7ed2aa5700 1 --2- 172.21.15.97:0/2989450441 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ecc104670 0x7f7ecc10af20 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:51.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:51.708+0000 7f7ed2aa5700 1 --2- 172.21.15.97:0/2989450441 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ecc102ad0 0x7f7ecc102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:51.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:51.708+0000 7f7ed2aa5700 1 --2- 172.21.15.97:0/2989450441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ecc103cd0 0x7f7ecc104130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:51.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:51.708+0000 7f7ed2aa5700 1 -- 172.21.15.97:0/2989450441 >> 172.21.15.97:0/2989450441 conn(0x7f7ecc0fe090 msgr2=0x7f7ecc1004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:51.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:51.708+0000 7f7ed2aa5700 1 -- 172.21.15.97:0/2989450441 shutdown_connections 2024-04-23T10:25:51.738 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:51.708+0000 7f7ed2aa5700 1 -- 172.21.15.97:0/2989450441 wait complete. 2024-04-23T10:25:51.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:51.709+0000 7f7ed2aa5700 1 Processor -- start 2024-04-23T10:25:51.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:51.709+0000 7f7ed2aa5700 1 -- start start 2024-04-23T10:25:51.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:51.709+0000 7f7ed2aa5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ecc102ad0 0x7f7ecc0767e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:51.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:51.709+0000 7f7ed2aa5700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ecc103cd0 0x7f7ecc074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:51.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:51.709+0000 7f7ed2aa5700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ecc104670 0x7f7ecc075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:51.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:51.709+0000 7f7ed2aa5700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc10e5c0 con 0x7f7ecc102ad0 2024-04-23T10:25:51.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:51.709+0000 7f7ed2aa5700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc10e440 con 0x7f7ecc103cd0 2024-04-23T10:25:51.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:51.709+0000 7f7ed2aa5700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc10e740 con 0x7f7ecc104670 2024-04-23T10:25:51.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:51.709+0000 7f7ed0841700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ecc102ad0 0x7f7ecc0767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:51.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:51.709+0000 7f7ed0841700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ecc102ad0 0x7f7ecc0767e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58478/0 (socket says 172.21.15.97:58478) 2024-04-23T10:25:51.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:51.709+0000 7f7ed0841700 1 -- 172.21.15.97:0/2891008255 learned_addr learned my addr 172.21.15.97:0/2891008255 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:25:51.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:51.709+0000 7f7ed0841700 1 -- 172.21.15.97:0/2891008255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ecc104670 msgr2=0x7f7ecc075370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:51.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:51.709+0000 7f7ecbfff700 1 --2- 172.21.15.97:0/2891008255 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ecc103cd0 0x7f7ecc074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:51.739 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:51.709+0000 7f7ed0841700 1 --2- 172.21.15.97:0/2891008255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ecc104670 0x7f7ecc075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:51.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:51.709+0000 7f7ed0841700 1 -- 172.21.15.97:0/2891008255 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ecc103cd0 msgr2=0x7f7ecc074e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:51.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:51.709+0000 7f7ed0841700 1 --2- 172.21.15.97:0/2891008255 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ecc103cd0 0x7f7ecc074e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:51.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:51.709+0000 7f7ed0841700 1 -- 172.21.15.97:0/2891008255 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ecc075c20 con 0x7f7ecc102ad0 2024-04-23T10:25:51.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:51.709+0000 7f7ecbfff700 1 --2- 172.21.15.97:0/2891008255 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ecc103cd0 0x7f7ecc074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:25:51.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:51.710+0000 7f7ed0841700 1 --2- 172.21.15.97:0/2891008255 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ecc102ad0 0x7f7ecc0767e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7ec000c990 tx=0x7f7ec000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:51.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:51.710+0000 7f7ec9ffb700 1 -- 172.21.15.97:0/2891008255 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec0007b00 con 0x7f7ecc102ad0 2024-04-23T10:25:51.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:51.710+0000 7f7ec9ffb700 1 -- 172.21.15.97:0/2891008255 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ec0007ca0 con 0x7f7ecc102ad0 2024-04-23T10:25:51.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:51.710+0000 7f7ec9ffb700 1 -- 172.21.15.97:0/2891008255 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec0018660 con 0x7f7ecc102ad0 2024-04-23T10:25:51.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:51.710+0000 7f7ed2aa5700 1 -- 172.21.15.97:0/2891008255 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ecc1a2ca0 con 0x7f7ecc102ad0 2024-04-23T10:25:51.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:51.710+0000 7f7ed2aa5700 1 -- 172.21.15.97:0/2891008255 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ecc1a31b0 con 0x7f7ecc102ad0 2024-04-23T10:25:51.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:51.711+0000 7f7ed2aa5700 1 -- 172.21.15.97:0/2891008255 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ecc069730 con 0x7f7ecc102ad0 2024-04-23T10:25:51.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:51.712+0000 7f7ec9ffb700 1 -- 172.21.15.97:0/2891008255 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7ec000f450 con 0x7f7ecc102ad0 2024-04-23T10:25:51.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:51.712+0000 7f7ec9ffb700 1 --2- 172.21.15.97:0/2891008255 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f7ebc0a3c40 0x7f7ebc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:51.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:51.712+0000 7f7ecbfff700 1 --2- 172.21.15.97:0/2891008255 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f7ebc0a3c40 0x7f7ebc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:51.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:51.712+0000 7f7ec9ffb700 1 -- 172.21.15.97:0/2891008255 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f7ec00d03b0 con 0x7f7ecc102ad0 2024-04-23T10:25:51.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:51.713+0000 7f7ecbfff700 1 --2- 172.21.15.97:0/2891008255 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f7ebc0a3c40 0x7f7ebc0a6100 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8007900 tx=0x7f7eb8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:51.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:51.717+0000 7f7ec9ffb700 1 -- 172.21.15.97:0/2891008255 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7ec0014030 con 0x7f7ecc102ad0 2024-04-23T10:25:51.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:51.926+0000 7f7ed2aa5700 1 -- 172.21.15.97:0/2891008255 --> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f7ecc062e80 con 0x7f7ebc0a3c40 2024-04-23T10:25:51.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:51.927+0000 7f7ec9ffb700 1 -- 172.21.15.97:0/2891008255 <== mgr.14152 v2:172.21.15.97:6800/3149043749 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f7ecc062e80 con 0x7f7ebc0a3c40 2024-04-23T10:25:51.926 INFO:teuthology.orchestra.run.smithi097.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-23T10:25:51.927 INFO:teuthology.orchestra.run.smithi097.stdout:smithi097 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT6204013T400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-23T10:25:51.927 INFO:teuthology.orchestra.run.smithi097.stdout:smithi183 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500QJ400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-23T10:25:51.927 INFO:teuthology.orchestra.run.smithi097.stdout:smithi196 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6246000Z400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-23T10:25:51.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:51.930+0000 7f7ed2aa5700 1 -- 172.21.15.97:0/2891008255 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f7ebc0a3c40 msgr2=0x7f7ebc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:51.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:51.930+0000 7f7ed2aa5700 1 --2- 172.21.15.97:0/2891008255 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f7ebc0a3c40 0x7f7ebc0a6100 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8007900 tx=0x7f7eb8008040 comp rx=0 tx=0).stop 2024-04-23T10:25:51.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:51.930+0000 7f7ed2aa5700 1 -- 172.21.15.97:0/2891008255 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ecc102ad0 msgr2=0x7f7ecc0767e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:51.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:51.930+0000 7f7ed2aa5700 1 --2- 172.21.15.97:0/2891008255 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ecc102ad0 0x7f7ecc0767e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7ec000c990 tx=0x7f7ec000cd50 comp rx=0 tx=0).stop 2024-04-23T10:25:51.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:51.930+0000 7f7ed2aa5700 1 -- 172.21.15.97:0/2891008255 shutdown_connections 2024-04-23T10:25:51.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:51.930+0000 7f7ed2aa5700 1 --2- 172.21.15.97:0/2891008255 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f7ebc0a3c40 0x7f7ebc0a6100 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:51.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:51.930+0000 7f7ed2aa5700 1 --2- 172.21.15.97:0/2891008255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ecc104670 0x7f7ecc075370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:51.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:51.930+0000 7f7ed2aa5700 1 --2- 172.21.15.97:0/2891008255 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ecc102ad0 0x7f7ecc0767e0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:51.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:51.930+0000 7f7ed2aa5700 1 --2- 172.21.15.97:0/2891008255 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ecc103cd0 0x7f7ecc074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:51.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:51.930+0000 7f7ed2aa5700 1 -- 172.21.15.97:0/2891008255 >> 172.21.15.97:0/2891008255 conn(0x7f7ecc0fe090 msgr2=0x7f7ecc106f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:51.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:51.930+0000 7f7ed2aa5700 1 -- 172.21.15.97:0/2891008255 shutdown_connections 2024-04-23T10:25:51.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:51.930+0000 7f7ed2aa5700 1 -- 172.21.15.97:0/2891008255 wait complete. 2024-04-23T10:25:52.797 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-23T10:25:52.806 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi097.front.sepia.ceph.com 2024-04-23T10:25:52.806 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-23T10:25:53.045 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T10:25:53.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:52 smithi196 ceph-mon[32919]: pgmap v344: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-23T10:25:53.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:52 smithi097 ceph-mon[29197]: pgmap v344: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-23T10:25:53.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:52 smithi183 ceph-mon[32317]: pgmap v344: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-23T10:25:53.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:53 smithi097 ceph-mon[29197]: from='client.15186 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:25:54.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:53 smithi196 ceph-mon[32919]: from='client.15186 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:25:54.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:53 smithi183 ceph-mon[32317]: from='client.15186 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:25:54.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:54.833+0000 7fa2c7ec2700 1 -- 172.21.15.97:0/896102497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2c0103580 msgr2=0x7fa2c0109e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:54.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:54.833+0000 7fa2c7ec2700 1 --2- 172.21.15.97:0/896102497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2c0103580 0x7fa2c0109e30 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fa2b00099c0 tx=0x7fa2b0009cd0 comp rx=0 tx=0).stop 2024-04-23T10:25:54.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:54.833+0000 7fa2c7ec2700 1 -- 172.21.15.97:0/896102497 shutdown_connections 2024-04-23T10:25:54.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:54.833+0000 7fa2c7ec2700 1 --2- 172.21.15.97:0/896102497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2c0103580 0x7fa2c0109e30 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:54.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:54.833+0000 7fa2c7ec2700 1 --2- 172.21.15.97:0/896102497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2c01019e0 0x7fa2c0101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:54.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:54.833+0000 7fa2c7ec2700 1 --2- 172.21.15.97:0/896102497 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa2c0102be0 0x7fa2c0103040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:54.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:54.833+0000 7fa2c7ec2700 1 -- 172.21.15.97:0/896102497 >> 172.21.15.97:0/896102497 conn(0x7fa2c00fcfa0 msgr2=0x7fa2c00ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:54.833 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:54 smithi097 ceph-mon[29197]: pgmap v345: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-23T10:25:54.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:54.833+0000 7fa2c7ec2700 1 -- 172.21.15.97:0/896102497 shutdown_connections 2024-04-23T10:25:54.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:54.833+0000 7fa2c7ec2700 1 -- 172.21.15.97:0/896102497 wait complete. 2024-04-23T10:25:54.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:54.834+0000 7fa2c7ec2700 1 Processor -- start 2024-04-23T10:25:54.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:54.834+0000 7fa2c7ec2700 1 -- start start 2024-04-23T10:25:54.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:54.834+0000 7fa2c7ec2700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa2c01019e0 0x7fa2c010c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:54.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:54.834+0000 7fa2c7ec2700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2c0102be0 0x7fa2c010aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:54.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:54.834+0000 7fa2c7ec2700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2c0103580 0x7fa2c010af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:54.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:54.834+0000 7fa2c7ec2700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c010e5f0 con 0x7fa2c0103580 2024-04-23T10:25:54.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:54.834+0000 7fa2c7ec2700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c010e470 con 0x7fa2c01019e0 2024-04-23T10:25:54.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:54.834+0000 7fa2c7ec2700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c010e770 con 0x7fa2c0102be0 2024-04-23T10:25:54.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:54.834+0000 7fa2c645f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2c0103580 0x7fa2c010af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:54.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:54.834+0000 7fa2c645f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2c0103580 0x7fa2c010af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58490/0 (socket says 172.21.15.97:58490) 2024-04-23T10:25:54.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:54.834+0000 7fa2c645f700 1 -- 172.21.15.97:0/402507555 learned_addr learned my addr 172.21.15.97:0/402507555 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:25:54.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:54.834+0000 7fa2c545d700 1 --2- 172.21.15.97:0/402507555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2c0102be0 0x7fa2c010aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:54.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:54.834+0000 7fa2c5c5e700 1 --2- 172.21.15.97:0/402507555 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa2c01019e0 0x7fa2c010c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:54.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:54.834+0000 7fa2c645f700 1 -- 172.21.15.97:0/402507555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2c0102be0 msgr2=0x7fa2c010aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:54.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:54.834+0000 7fa2c645f700 1 --2- 172.21.15.97:0/402507555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2c0102be0 0x7fa2c010aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:54.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:54.834+0000 7fa2c645f700 1 -- 172.21.15.97:0/402507555 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa2c01019e0 msgr2=0x7fa2c010c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:54.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:54.834+0000 7fa2c645f700 1 --2- 172.21.15.97:0/402507555 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa2c01019e0 0x7fa2c010c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:54.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:54.835+0000 7fa2c645f700 1 -- 172.21.15.97:0/402507555 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2c010b7f0 con 0x7fa2c0103580 2024-04-23T10:25:54.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:54.835+0000 7fa2c645f700 1 --2- 172.21.15.97:0/402507555 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2c0103580 0x7fa2c010af40 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa2b00059d0 tx=0x7fa2b00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:54.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:54.835+0000 7fa2beffd700 1 -- 172.21.15.97:0/402507555 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b0027070 con 0x7fa2c0103580 2024-04-23T10:25:54.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:54.835+0000 7fa2c7ec2700 1 -- 172.21.15.97:0/402507555 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2c01a2c10 con 0x7fa2c0103580 2024-04-23T10:25:54.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:54.835+0000 7fa2beffd700 1 -- 172.21.15.97:0/402507555 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2b00041e0 con 0x7fa2c0103580 2024-04-23T10:25:54.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:54.835+0000 7fa2c7ec2700 1 -- 172.21.15.97:0/402507555 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2c01a3060 con 0x7fa2c0103580 2024-04-23T10:25:54.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:54.836+0000 7fa2beffd700 1 -- 172.21.15.97:0/402507555 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b000f460 con 0x7fa2c0103580 2024-04-23T10:25:54.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:54.836+0000 7fa2c7ec2700 1 -- 172.21.15.97:0/402507555 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2ac005360 con 0x7fa2c0103580 2024-04-23T10:25:54.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:54.837+0000 7fa2beffd700 1 -- 172.21.15.97:0/402507555 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa2b002a070 con 0x7fa2c0103580 2024-04-23T10:25:54.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:54.838+0000 7fa2beffd700 1 --2- 172.21.15.97:0/402507555 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fa2a80a3cc0 0x7fa2a80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:54.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:54.838+0000 7fa2c5c5e700 1 --2- 172.21.15.97:0/402507555 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fa2a80a3cc0 0x7fa2a80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:54.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:54.838+0000 7fa2beffd700 1 -- 172.21.15.97:0/402507555 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fa2b00d1210 con 0x7fa2c0103580 2024-04-23T10:25:54.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:54.838+0000 7fa2c5c5e700 1 --2- 172.21.15.97:0/402507555 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fa2a80a3cc0 0x7fa2a80a6180 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa2b8005950 tx=0x7fa2b800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:54.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:54.842+0000 7fa2beffd700 1 -- 172.21.15.97:0/402507555 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa2b005eff0 con 0x7fa2c0103580 2024-04-23T10:25:55.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:55.089+0000 7fa2c7ec2700 1 -- 172.21.15.97:0/402507555 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fa2ac005e60 con 0x7fa2c0103580 2024-04-23T10:25:55.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:55.090+0000 7fa2beffd700 1 -- 172.21.15.97:0/402507555 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) ==== 76+0+3090 (secure 0 0 0) 0x7fa2b0013070 con 0x7fa2c0103580 2024-04-23T10:25:55.089 INFO:teuthology.orchestra.run.smithi097.stdout:e25 2024-04-23T10:25:55.089 INFO:teuthology.orchestra.run.smithi097.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T10:25:55.089 INFO:teuthology.orchestra.run.smithi097.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-23T10:25:55.089 INFO:teuthology.orchestra.run.smithi097.stdout:legacy client fscid: 1 2024-04-23T10:25:55.089 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:25:55.090 INFO:teuthology.orchestra.run.smithi097.stdout:Filesystem 'cephfs' (1) 2024-04-23T10:25:55.090 INFO:teuthology.orchestra.run.smithi097.stdout:fs_name cephfs 2024-04-23T10:25:55.090 INFO:teuthology.orchestra.run.smithi097.stdout:epoch 25 2024-04-23T10:25:55.090 INFO:teuthology.orchestra.run.smithi097.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-23T10:25:55.090 INFO:teuthology.orchestra.run.smithi097.stdout:created 2024-04-23T10:24:06.910134+0000 2024-04-23T10:25:55.090 INFO:teuthology.orchestra.run.smithi097.stdout:modified 2024-04-23T10:24:12.969478+0000 2024-04-23T10:25:55.090 INFO:teuthology.orchestra.run.smithi097.stdout:tableserver 0 2024-04-23T10:25:55.090 INFO:teuthology.orchestra.run.smithi097.stdout:root 0 2024-04-23T10:25:55.090 INFO:teuthology.orchestra.run.smithi097.stdout:session_timeout 300 2024-04-23T10:25:55.090 INFO:teuthology.orchestra.run.smithi097.stdout:session_autoclose 300 2024-04-23T10:25:55.090 INFO:teuthology.orchestra.run.smithi097.stdout:max_file_size 1099511627776 2024-04-23T10:25:55.090 INFO:teuthology.orchestra.run.smithi097.stdout:max_xattr_size 65536 2024-04-23T10:25:55.090 INFO:teuthology.orchestra.run.smithi097.stdout:required_client_features {} 2024-04-23T10:25:55.091 INFO:teuthology.orchestra.run.smithi097.stdout:last_failure 0 2024-04-23T10:25:55.091 INFO:teuthology.orchestra.run.smithi097.stdout:last_failure_osd_epoch 0 2024-04-23T10:25:55.091 INFO:teuthology.orchestra.run.smithi097.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-23T10:25:55.091 INFO:teuthology.orchestra.run.smithi097.stdout:max_mds 5 2024-04-23T10:25:55.091 INFO:teuthology.orchestra.run.smithi097.stdout:in 0,1,2,3,4 2024-04-23T10:25:55.091 INFO:teuthology.orchestra.run.smithi097.stdout:up {0=24488,1=24487,2=24469,3=24403,4=24398} 2024-04-23T10:25:55.091 INFO:teuthology.orchestra.run.smithi097.stdout:failed 2024-04-23T10:25:55.091 INFO:teuthology.orchestra.run.smithi097.stdout:damaged 2024-04-23T10:25:55.091 INFO:teuthology.orchestra.run.smithi097.stdout:stopped 2024-04-23T10:25:55.091 INFO:teuthology.orchestra.run.smithi097.stdout:data_pools [3] 2024-04-23T10:25:55.091 INFO:teuthology.orchestra.run.smithi097.stdout:metadata_pool 2 2024-04-23T10:25:55.091 INFO:teuthology.orchestra.run.smithi097.stdout:inline_data disabled 2024-04-23T10:25:55.091 INFO:teuthology.orchestra.run.smithi097.stdout:balancer 2024-04-23T10:25:55.091 INFO:teuthology.orchestra.run.smithi097.stdout:bal_rank_mask -1 2024-04-23T10:25:55.091 INFO:teuthology.orchestra.run.smithi097.stdout:standby_count_wanted 0 2024-04-23T10:25:55.092 INFO:teuthology.orchestra.run.smithi097.stdout:qdb_cluster leader: 24488 members: 24469,24488,24398,24403,24487 2024-04-23T10:25:55.092 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.b{0:24488} state up:active seq 3 addr [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T10:25:55.092 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.h{0:24461} state up:standby-replay seq 1 addr [v2:172.21.15.183:6836/1898341911,v1:172.21.15.183:6837/1898341911] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T10:25:55.092 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.i{1:24487} state up:active seq 6 addr [v2:172.21.15.196:6838/83843556,v1:172.21.15.196:6839/83843556] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T10:25:55.092 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.f{1:24451} state up:standby-replay seq 1 addr [v2:172.21.15.196:6834/1171477563,v1:172.21.15.196:6835/1171477563] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T10:25:55.092 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.c{2:24469} state up:active seq 7 addr [v2:172.21.15.196:6836/3094946502,v1:172.21.15.196:6837/3094946502] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T10:25:55.092 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.e{2:24419} state up:standby-replay seq 1 addr [v2:172.21.15.183:6834/1897093105,v1:172.21.15.183:6835/1897093105] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T10:25:55.092 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.l{3:24403} state up:active seq 12 addr [v2:172.21.15.196:6832/3885816565,v1:172.21.15.196:6833/3885816565] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T10:25:55.092 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.a{3:14652} state up:standby-replay seq 1 addr [v2:172.21.15.97:6840/747242680,v1:172.21.15.97:6841/747242680] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T10:25:55.092 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.k{4:24398} state up:active seq 11 addr [v2:172.21.15.183:6832/4292847121,v1:172.21.15.183:6833/4292847121] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T10:25:55.092 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.g{4:14640} state up:standby-replay seq 1 addr [v2:172.21.15.97:6838/418013575,v1:172.21.15.97:6839/418013575] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T10:25:55.092 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:25:55.092 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:25:55.092 INFO:teuthology.orchestra.run.smithi097.stdout:Standby daemons: 2024-04-23T10:25:55.092 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:25:55.093 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.j{-1:14559} state up:standby seq 1 addr [v2:172.21.15.97:6834/3946878762,v1:172.21.15.97:6835/3946878762] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T10:25:55.093 INFO:teuthology.orchestra.run.smithi097.stdout:[mds.d{-1:14571} state up:standby seq 1 addr [v2:172.21.15.97:6836/209800861,v1:172.21.15.97:6837/209800861] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T10:25:55.093 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:54 smithi196 ceph-mon[32919]: pgmap v345: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-23T10:25:55.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:55.092+0000 7fa2c7ec2700 1 -- 172.21.15.97:0/402507555 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fa2a80a3cc0 msgr2=0x7fa2a80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:55.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:55.092+0000 7fa2c7ec2700 1 --2- 172.21.15.97:0/402507555 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fa2a80a3cc0 0x7fa2a80a6180 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa2b8005950 tx=0x7fa2b800a300 comp rx=0 tx=0).stop 2024-04-23T10:25:55.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:55.092+0000 7fa2c7ec2700 1 -- 172.21.15.97:0/402507555 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2c0103580 msgr2=0x7fa2c010af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:55.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:55.092+0000 7fa2c7ec2700 1 --2- 172.21.15.97:0/402507555 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2c0103580 0x7fa2c010af40 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa2b00059d0 tx=0x7fa2b00047b0 comp rx=0 tx=0).stop 2024-04-23T10:25:55.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:55.092+0000 7fa2c7ec2700 1 -- 172.21.15.97:0/402507555 shutdown_connections 2024-04-23T10:25:55.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:55.092+0000 7fa2c7ec2700 1 --2- 172.21.15.97:0/402507555 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fa2a80a3cc0 0x7fa2a80a6180 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:55.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:55.092+0000 7fa2c7ec2700 1 --2- 172.21.15.97:0/402507555 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2c0103580 0x7fa2c010af40 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:55.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:55.092+0000 7fa2c7ec2700 1 --2- 172.21.15.97:0/402507555 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa2c01019e0 0x7fa2c010c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:55.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:55.092+0000 7fa2c7ec2700 1 --2- 172.21.15.97:0/402507555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2c0102be0 0x7fa2c010aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:55.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:55.092+0000 7fa2c7ec2700 1 -- 172.21.15.97:0/402507555 >> 172.21.15.97:0/402507555 conn(0x7fa2c00fcfa0 msgr2=0x7fa2c0105e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:55.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:55.092+0000 7fa2c7ec2700 1 -- 172.21.15.97:0/402507555 shutdown_connections 2024-04-23T10:25:55.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:55.092+0000 7fa2c7ec2700 1 -- 172.21.15.97:0/402507555 wait complete. 2024-04-23T10:25:55.095 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 25 2024-04-23T10:25:55.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:54 smithi183 ceph-mon[32317]: pgmap v345: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-23T10:25:55.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:55 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/402507555' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T10:25:55.898 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-23T10:25:56.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:55 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/402507555' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T10:25:56.138 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T10:25:56.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:55 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/402507555' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T10:25:56.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:56 smithi097 ceph-mon[29197]: pgmap v346: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-23T10:25:57.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:56 smithi196 ceph-mon[32919]: pgmap v346: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-23T10:25:57.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:56 smithi183 ceph-mon[32317]: pgmap v346: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-23T10:25:57.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:57.852+0000 7f5482f47700 1 -- 172.21.15.97:0/4230658484 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f547c1025c0 msgr2=0x7f547c109e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:57.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:57.852+0000 7f5482f47700 1 --2- 172.21.15.97:0/4230658484 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f547c1025c0 0x7f547c109e60 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f5474009f90 tx=0x7f547400b330 comp rx=0 tx=0).stop 2024-04-23T10:25:57.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:57.852+0000 7f5482f47700 1 -- 172.21.15.97:0/4230658484 shutdown_connections 2024-04-23T10:25:57.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:57.852+0000 7f5482f47700 1 --2- 172.21.15.97:0/4230658484 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f547c1025c0 0x7f547c109e60 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:57.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:57.852+0000 7f5482f47700 1 --2- 172.21.15.97:0/4230658484 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f547c103030 0x7f547c103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:57.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:57.852+0000 7f5482f47700 1 --2- 172.21.15.97:0/4230658484 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f547c101c00 0x7f547c102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:57.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:57.852+0000 7f5482f47700 1 -- 172.21.15.97:0/4230658484 >> 172.21.15.97:0/4230658484 conn(0x7f547c0fc6b0 msgr2=0x7f547c0feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:57.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:57.852+0000 7f5482f47700 1 -- 172.21.15.97:0/4230658484 shutdown_connections 2024-04-23T10:25:57.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:57.852+0000 7f5482f47700 1 -- 172.21.15.97:0/4230658484 wait complete. 2024-04-23T10:25:57.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:57.852+0000 7f5482f47700 1 Processor -- start 2024-04-23T10:25:57.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:57.852+0000 7f5482f47700 1 -- start start 2024-04-23T10:25:57.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:57.852+0000 7f5482f47700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f547c101c00 0x7f547c198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:57.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:57.852+0000 7f5482f47700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f547c1025c0 0x7f547c199050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:57.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:57.852+0000 7f5482f47700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f547c103030 0x7f547c1a00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:57.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:57.852+0000 7f5482f47700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f547c10e520 con 0x7f547c1025c0 2024-04-23T10:25:57.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:57.852+0000 7f5482f47700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f547c10e3a0 con 0x7f547c101c00 2024-04-23T10:25:57.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:57.852+0000 7f5482f47700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f547c10e6a0 con 0x7f547c103030 2024-04-23T10:25:57.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:57.853+0000 7f547bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f547c1025c0 0x7f547c199050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:57.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:57.853+0000 7f547bfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f547c1025c0 0x7f547c199050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46490/0 (socket says 172.21.15.97:46490) 2024-04-23T10:25:57.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:57.853+0000 7f547bfff700 1 -- 172.21.15.97:0/3507173354 learned_addr learned my addr 172.21.15.97:0/3507173354 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:25:57.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:57.853+0000 7f54814e4700 1 --2- 172.21.15.97:0/3507173354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f547c103030 0x7f547c1a00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:57.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:57.853+0000 7f5480ce3700 1 --2- 172.21.15.97:0/3507173354 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f547c101c00 0x7f547c198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:57.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:57.853+0000 7f547bfff700 1 -- 172.21.15.97:0/3507173354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f547c103030 msgr2=0x7f547c1a00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:57.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:57.853+0000 7f547bfff700 1 --2- 172.21.15.97:0/3507173354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f547c103030 0x7f547c1a00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:57.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:57.853+0000 7f547bfff700 1 -- 172.21.15.97:0/3507173354 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f547c101c00 msgr2=0x7f547c198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:57.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:57.853+0000 7f547bfff700 1 --2- 172.21.15.97:0/3507173354 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f547c101c00 0x7f547c198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:57.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:57.853+0000 7f547bfff700 1 -- 172.21.15.97:0/3507173354 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f547c1a0770 con 0x7f547c1025c0 2024-04-23T10:25:57.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:57.853+0000 7f5480ce3700 1 --2- 172.21.15.97:0/3507173354 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f547c101c00 0x7f547c198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:25:57.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:57.853+0000 7f547bfff700 1 --2- 172.21.15.97:0/3507173354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f547c1025c0 0x7f547c199050 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f546800e9f0 tx=0x7f546800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:57.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:57.853+0000 7f5479ffb700 1 -- 172.21.15.97:0/3507173354 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f546800cbd0 con 0x7f547c1025c0 2024-04-23T10:25:57.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:57.853+0000 7f5479ffb700 1 -- 172.21.15.97:0/3507173354 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f546800cd70 con 0x7f547c1025c0 2024-04-23T10:25:57.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:57.853+0000 7f5482f47700 1 -- 172.21.15.97:0/3507173354 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f547c1a0a60 con 0x7f547c1025c0 2024-04-23T10:25:57.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:57.853+0000 7f5479ffb700 1 -- 172.21.15.97:0/3507173354 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5468005610 con 0x7f547c1025c0 2024-04-23T10:25:57.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:57.853+0000 7f5482f47700 1 -- 172.21.15.97:0/3507173354 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f547c1a0fa0 con 0x7f547c1025c0 2024-04-23T10:25:57.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:57.854+0000 7f5482f47700 1 -- 172.21.15.97:0/3507173354 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f547c069730 con 0x7f547c1025c0 2024-04-23T10:25:57.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:57.855+0000 7f5479ffb700 1 -- 172.21.15.97:0/3507173354 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5468004750 con 0x7f547c1025c0 2024-04-23T10:25:57.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:57.856+0000 7f5479ffb700 1 --2- 172.21.15.97:0/3507173354 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f546c0a3c40 0x7f546c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:57.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:57.856+0000 7f5480ce3700 1 --2- 172.21.15.97:0/3507173354 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f546c0a3c40 0x7f546c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:57.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:57.856+0000 7f5479ffb700 1 -- 172.21.15.97:0/3507173354 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f54680d05e0 con 0x7f547c1025c0 2024-04-23T10:25:57.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:57.856+0000 7f5480ce3700 1 --2- 172.21.15.97:0/3507173354 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f546c0a3c40 0x7f546c0a6100 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f547000bc10 tx=0x7f547000b5f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:57.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:57.860+0000 7f5479ffb700 1 -- 172.21.15.97:0/3507173354 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f546809cff0 con 0x7f547c1025c0 2024-04-23T10:25:58.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:58.061+0000 7f5482f47700 1 -- 172.21.15.97:0/3507173354 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f547c063650 con 0x7f547c1025c0 2024-04-23T10:25:58.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:58.062+0000 7f5479ffb700 1 -- 172.21.15.97:0/3507173354 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v88) ==== 56+0+5258 (secure 0 0 0) 0x7f546809c8f0 con 0x7f547c1025c0 2024-04-23T10:25:58.061 INFO:teuthology.orchestra.run.smithi097.stdout:epoch 88 2024-04-23T10:25:58.062 INFO:teuthology.orchestra.run.smithi097.stdout:fsid 9b22af88-0159-11ef-bc93-c7b262605968 2024-04-23T10:25:58.062 INFO:teuthology.orchestra.run.smithi097.stdout:created 2024-04-23T10:16:05.983305+0000 2024-04-23T10:25:58.062 INFO:teuthology.orchestra.run.smithi097.stdout:modified 2024-04-23T10:25:00.660133+0000 2024-04-23T10:25:58.062 INFO:teuthology.orchestra.run.smithi097.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-23T10:25:58.062 INFO:teuthology.orchestra.run.smithi097.stdout:crush_version 26 2024-04-23T10:25:58.062 INFO:teuthology.orchestra.run.smithi097.stdout:full_ratio 0.95 2024-04-23T10:25:58.062 INFO:teuthology.orchestra.run.smithi097.stdout:backfillfull_ratio 0.9 2024-04-23T10:25:58.062 INFO:teuthology.orchestra.run.smithi097.stdout:nearfull_ratio 0.85 2024-04-23T10:25:58.062 INFO:teuthology.orchestra.run.smithi097.stdout:require_min_compat_client luminous 2024-04-23T10:25:58.062 INFO:teuthology.orchestra.run.smithi097.stdout:min_compat_client luminous 2024-04-23T10:25:58.062 INFO:teuthology.orchestra.run.smithi097.stdout:require_osd_release squid 2024-04-23T10:25:58.062 INFO:teuthology.orchestra.run.smithi097.stdout:stretch_mode_enabled false 2024-04-23T10:25:58.062 INFO:teuthology.orchestra.run.smithi097.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-23T10:25:58.063 INFO:teuthology.orchestra.run.smithi097.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-04-23T10:25:58.063 INFO:teuthology.orchestra.run.smithi097.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-04-23T10:25:58.063 INFO:teuthology.orchestra.run.smithi097.stdout:max_osd 12 2024-04-23T10:25:58.063 INFO:teuthology.orchestra.run.smithi097.stdout:osd.0 up in weight 1 up_from 8 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.97:6802/3031221802,v1:172.21.15.97:6803/3031221802] [v2:172.21.15.97:6804/3031221802,v1:172.21.15.97:6805/3031221802] exists,up 58cb995d-de35-4522-ac81-4982401b260d 2024-04-23T10:25:58.063 INFO:teuthology.orchestra.run.smithi097.stdout:osd.1 up in weight 1 up_from 13 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6800/1108870440,v1:172.21.15.183:6801/1108870440] [v2:172.21.15.183:6802/1108870440,v1:172.21.15.183:6803/1108870440] exists,up b47bd8ee-1f21-4834-acf5-bb40f97ab3ac 2024-04-23T10:25:58.063 INFO:teuthology.orchestra.run.smithi097.stdout:osd.2 up in weight 1 up_from 18 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.196:6800/2877039731,v1:172.21.15.196:6801/2877039731] [v2:172.21.15.196:6802/2877039731,v1:172.21.15.196:6803/2877039731] exists,up 17c4eec3-6c47-4334-9b63-1d5ad9ec5988 2024-04-23T10:25:58.063 INFO:teuthology.orchestra.run.smithi097.stdout:osd.3 up in weight 1 up_from 26 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.97:6810/4276311194,v1:172.21.15.97:6811/4276311194] [v2:172.21.15.97:6812/4276311194,v1:172.21.15.97:6813/4276311194] exists,up 3f731c38-2e45-4ca1-8dce-efc4a1b2166b 2024-04-23T10:25:58.063 INFO:teuthology.orchestra.run.smithi097.stdout:osd.4 up in weight 1 up_from 34 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6808/63913916,v1:172.21.15.183:6809/63913916] [v2:172.21.15.183:6810/63913916,v1:172.21.15.183:6811/63913916] exists,up fdcafb14-9b82-4134-bd5b-79c0516f6cbb 2024-04-23T10:25:58.063 INFO:teuthology.orchestra.run.smithi097.stdout:osd.5 up in weight 1 up_from 39 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.196:6808/2804446896,v1:172.21.15.196:6809/2804446896] [v2:172.21.15.196:6810/2804446896,v1:172.21.15.196:6811/2804446896] exists,up cdcc18d8-da3c-4012-bf12-876bb39e66eb 2024-04-23T10:25:58.063 INFO:teuthology.orchestra.run.smithi097.stdout:osd.6 up in weight 1 up_from 47 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.97:6818/2080916705,v1:172.21.15.97:6819/2080916705] [v2:172.21.15.97:6820/2080916705,v1:172.21.15.97:6821/2080916705] exists,up 071abb5c-2edf-4832-b8aa-3b80d20096fc 2024-04-23T10:25:58.063 INFO:teuthology.orchestra.run.smithi097.stdout:osd.7 up in weight 1 up_from 52 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6816/1865873033,v1:172.21.15.183:6817/1865873033] [v2:172.21.15.183:6818/1865873033,v1:172.21.15.183:6819/1865873033] exists,up 9d54760b-66df-486b-8abb-017f3b64b4d7 2024-04-23T10:25:58.063 INFO:teuthology.orchestra.run.smithi097.stdout:osd.8 up in weight 1 up_from 60 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.196:6816/4063932139,v1:172.21.15.196:6817/4063932139] [v2:172.21.15.196:6818/4063932139,v1:172.21.15.196:6819/4063932139] exists,up 5cd3100a-bd6a-4be5-b5e9-0d0afc325193 2024-04-23T10:25:58.063 INFO:teuthology.orchestra.run.smithi097.stdout:osd.9 up in weight 1 up_from 68 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.97:6826/1887432474,v1:172.21.15.97:6827/1887432474] [v2:172.21.15.97:6828/1887432474,v1:172.21.15.97:6829/1887432474] exists,up b6a5f4a1-4ecc-4f73-a2df-07551d9298ae 2024-04-23T10:25:58.063 INFO:teuthology.orchestra.run.smithi097.stdout:osd.10 up in weight 1 up_from 73 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6824/799434647,v1:172.21.15.183:6825/799434647] [v2:172.21.15.183:6826/799434647,v1:172.21.15.183:6827/799434647] exists,up 86f59816-1ea6-4ae9-8796-cf5fc4831064 2024-04-23T10:25:58.063 INFO:teuthology.orchestra.run.smithi097.stdout:osd.11 up in weight 1 up_from 81 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.196:6824/1873969467,v1:172.21.15.196:6825/1873969467] [v2:172.21.15.196:6826/1873969467,v1:172.21.15.196:6827/1873969467] exists,up 95d64a89-a9f8-4c7a-8c0d-da7fb26cfff8 2024-04-23T10:25:58.063 INFO:teuthology.orchestra.run.smithi097.stdout:pg_upmap_items 2.8 [2,11] 2024-04-23T10:25:58.064 INFO:teuthology.orchestra.run.smithi097.stdout:pg_upmap_items 2.16 [2,4] 2024-04-23T10:25:58.064 INFO:teuthology.orchestra.run.smithi097.stdout:pg_upmap_items 2.1c [8,11] 2024-04-23T10:25:58.064 INFO:teuthology.orchestra.run.smithi097.stdout:pg_upmap_items 2.1f [8,4] 2024-04-23T10:25:58.064 INFO:teuthology.orchestra.run.smithi097.stdout:pg_upmap_items 3.0 [5,9] 2024-04-23T10:25:58.064 INFO:teuthology.orchestra.run.smithi097.stdout:pg_upmap_items 3.1 [4,9] 2024-04-23T10:25:58.064 INFO:teuthology.orchestra.run.smithi097.stdout:pg_upmap_items 3.24 [4,9] 2024-04-23T10:25:58.064 INFO:teuthology.orchestra.run.smithi097.stdout:blocklist 172.21.15.97:0/3866016040 expires 2024-04-24T10:16:59.468237+0000 2024-04-23T10:25:58.064 INFO:teuthology.orchestra.run.smithi097.stdout:blocklist 172.21.15.97:6801/3297166858 expires 2024-04-24T10:16:59.468237+0000 2024-04-23T10:25:58.064 INFO:teuthology.orchestra.run.smithi097.stdout:blocklist 172.21.15.97:0/3804743548 expires 2024-04-24T10:16:59.468237+0000 2024-04-23T10:25:58.064 INFO:teuthology.orchestra.run.smithi097.stdout:blocklist 172.21.15.97:6800/3706463017 expires 2024-04-24T10:16:33.128917+0000 2024-04-23T10:25:58.064 INFO:teuthology.orchestra.run.smithi097.stdout:blocklist 172.21.15.97:0/3715079438 expires 2024-04-24T10:16:33.128917+0000 2024-04-23T10:25:58.064 INFO:teuthology.orchestra.run.smithi097.stdout:blocklist 172.21.15.97:0/786806834 expires 2024-04-24T10:16:59.468237+0000 2024-04-23T10:25:58.065 INFO:teuthology.orchestra.run.smithi097.stdout:blocklist 172.21.15.97:6801/3706463017 expires 2024-04-24T10:16:33.128917+0000 2024-04-23T10:25:58.065 INFO:teuthology.orchestra.run.smithi097.stdout:blocklist 172.21.15.97:0/3294175554 expires 2024-04-24T10:16:33.128917+0000 2024-04-23T10:25:58.065 INFO:teuthology.orchestra.run.smithi097.stdout:blocklist 172.21.15.97:0/1449136760 expires 2024-04-24T10:16:33.128917+0000 2024-04-23T10:25:58.065 INFO:teuthology.orchestra.run.smithi097.stdout:blocklist 172.21.15.97:6800/3297166858 expires 2024-04-24T10:16:59.468237+0000 2024-04-23T10:25:58.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:58.065+0000 7f5482f47700 1 -- 172.21.15.97:0/3507173354 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f546c0a3c40 msgr2=0x7f546c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:58.065 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:58.065+0000 7f5482f47700 1 --2- 172.21.15.97:0/3507173354 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f546c0a3c40 0x7f546c0a6100 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f547000bc10 tx=0x7f547000b5f0 comp rx=0 tx=0).stop 2024-04-23T10:25:58.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:58.065+0000 7f5482f47700 1 -- 172.21.15.97:0/3507173354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f547c1025c0 msgr2=0x7f547c199050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:58.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:58.065+0000 7f5482f47700 1 --2- 172.21.15.97:0/3507173354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f547c1025c0 0x7f547c199050 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f546800e9f0 tx=0x7f546800ed00 comp rx=0 tx=0).stop 2024-04-23T10:25:58.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:58.066+0000 7f5482f47700 1 -- 172.21.15.97:0/3507173354 shutdown_connections 2024-04-23T10:25:58.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:58.066+0000 7f5482f47700 1 --2- 172.21.15.97:0/3507173354 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f546c0a3c40 0x7f546c0a6100 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:58.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:58.066+0000 7f5482f47700 1 --2- 172.21.15.97:0/3507173354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f547c103030 0x7f547c1a00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:58.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:58.066+0000 7f5482f47700 1 --2- 172.21.15.97:0/3507173354 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f547c101c00 0x7f547c198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:58.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:58.066+0000 7f5482f47700 1 --2- 172.21.15.97:0/3507173354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f547c1025c0 0x7f547c199050 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:58.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:58.066+0000 7f5482f47700 1 -- 172.21.15.97:0/3507173354 >> 172.21.15.97:0/3507173354 conn(0x7f547c0fc6b0 msgr2=0x7f547c105f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:58.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:58.066+0000 7f5482f47700 1 -- 172.21.15.97:0/3507173354 shutdown_connections 2024-04-23T10:25:58.066 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:58.066+0000 7f5482f47700 1 -- 172.21.15.97:0/3507173354 wait complete. 2024-04-23T10:25:58.799 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-23T10:25:58.809 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T10:25:58.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:58.914+0000 7fc1a3329640 1 -- 172.21.15.97:0/2929038193 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc19c1029a0 msgr2=0x7fc19c1092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:58.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:58.914+0000 7fc1a3329640 1 --2- 172.21.15.97:0/2929038193 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc19c1029a0 0x7fc19c1092c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc194009ca0 tx=0x7fc19402a8c0 comp rx=0 tx=0).stop 2024-04-23T10:25:58.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:58.914+0000 7fc1a3329640 1 -- 172.21.15.97:0/2929038193 shutdown_connections 2024-04-23T10:25:58.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:58.914+0000 7fc1a3329640 1 --2- 172.21.15.97:0/2929038193 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc19c1029a0 0x7fc19c1092c0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:58.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:58.914+0000 7fc1a3329640 1 --2- 172.21.15.97:0/2929038193 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc19c101fe0 0x7fc19c102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:58.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:58.914+0000 7fc1a3329640 1 --2- 172.21.15.97:0/2929038193 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc19c100e90 0x7fc19c1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:58.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:58.914+0000 7fc1a3329640 1 -- 172.21.15.97:0/2929038193 >> 172.21.15.97:0/2929038193 conn(0x7fc19c0fc400 msgr2=0x7fc19c0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:58.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:58.914+0000 7fc1a3329640 1 -- 172.21.15.97:0/2929038193 shutdown_connections 2024-04-23T10:25:58.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:58.914+0000 7fc1a3329640 1 -- 172.21.15.97:0/2929038193 wait complete. 2024-04-23T10:25:58.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:58.915+0000 7fc1a3329640 1 Processor -- start 2024-04-23T10:25:58.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:58.915+0000 7fc1a3329640 1 -- start start 2024-04-23T10:25:58.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:58.915+0000 7fc1a3329640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc19c100e90 0x7fc19c197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:58.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:58.915+0000 7fc1a3329640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc19c101fe0 0x7fc19c197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:58.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:58.915+0000 7fc1a3329640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc19c1029a0 0x7fc19c19e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:58.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:58.915+0000 7fc1a3329640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc19c10cae0 con 0x7fc19c1029a0 2024-04-23T10:25:58.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:58.915+0000 7fc1a3329640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc19c10c960 con 0x7fc19c100e90 2024-04-23T10:25:58.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:58.915+0000 7fc1a3329640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc19c10cc60 con 0x7fc19c101fe0 2024-04-23T10:25:58.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:58.915+0000 7fc1a18c6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc19c1029a0 0x7fc19c19e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:58.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:58.915+0000 7fc1a18c6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc19c1029a0 0x7fc19c19e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46512/0 (socket says 172.21.15.97:46512) 2024-04-23T10:25:58.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:58.915+0000 7fc1a18c6640 1 -- 172.21.15.97:0/2741277809 learned_addr learned my addr 172.21.15.97:0/2741277809 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:25:58.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:58.915+0000 7fc1a10c5640 1 --2- 172.21.15.97:0/2741277809 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc19c100e90 0x7fc19c197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:58.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:58.915+0000 7fc1a08c4640 1 --2- 172.21.15.97:0/2741277809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc19c101fe0 0x7fc19c197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:58.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:58.915+0000 7fc1a18c6640 1 -- 172.21.15.97:0/2741277809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc19c101fe0 msgr2=0x7fc19c197740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:58.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:58.915+0000 7fc1a18c6640 1 --2- 172.21.15.97:0/2741277809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc19c101fe0 0x7fc19c197740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:58.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:58.915+0000 7fc1a18c6640 1 -- 172.21.15.97:0/2741277809 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc19c100e90 msgr2=0x7fc19c197200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:58.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:58.915+0000 7fc1a18c6640 1 --2- 172.21.15.97:0/2741277809 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc19c100e90 0x7fc19c197200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:58.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:58.915+0000 7fc1a18c6640 1 -- 172.21.15.97:0/2741277809 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc19c19ee20 con 0x7fc19c1029a0 2024-04-23T10:25:58.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:58.915+0000 7fc1a08c4640 1 --2- 172.21.15.97:0/2741277809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc19c101fe0 0x7fc19c197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:25:58.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:58.915+0000 7fc1a18c6640 1 --2- 172.21.15.97:0/2741277809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc19c1029a0 0x7fc19c19e7b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc194031a20 tx=0x7fc194003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:58.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:58.916+0000 7fc1927fc640 1 -- 172.21.15.97:0/2741277809 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc194004240 con 0x7fc19c1029a0 2024-04-23T10:25:58.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:58.916+0000 7fc1927fc640 1 -- 172.21.15.97:0/2741277809 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc194003e10 con 0x7fc19c1029a0 2024-04-23T10:25:58.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:58.916+0000 7fc1a3329640 1 -- 172.21.15.97:0/2741277809 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc19c19f0b0 con 0x7fc19c1029a0 2024-04-23T10:25:58.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:58.916+0000 7fc1927fc640 1 -- 172.21.15.97:0/2741277809 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19402fd90 con 0x7fc19c1029a0 2024-04-23T10:25:58.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:58.916+0000 7fc1a3329640 1 -- 172.21.15.97:0/2741277809 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc19c19f680 con 0x7fc19c1029a0 2024-04-23T10:25:58.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:58.917+0000 7fc1a3329640 1 -- 172.21.15.97:0/2741277809 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc19c108270 con 0x7fc19c1029a0 2024-04-23T10:25:58.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:58.918+0000 7fc1927fc640 1 -- 172.21.15.97:0/2741277809 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc1940043e0 con 0x7fc19c1029a0 2024-04-23T10:25:58.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:58.918+0000 7fc1927fc640 1 --2- 172.21.15.97:0/2741277809 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fc1880a3bb0 0x7fc1880a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:58.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:58.918+0000 7fc1a10c5640 1 --2- 172.21.15.97:0/2741277809 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fc1880a3bb0 0x7fc1880a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:58.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:58.918+0000 7fc1927fc640 1 -- 172.21.15.97:0/2741277809 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fc1940f1cc0 con 0x7fc19c1029a0 2024-04-23T10:25:58.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:58.919+0000 7fc1a10c5640 1 --2- 172.21.15.97:0/2741277809 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fc1880a3bb0 0x7fc1880a6070 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc198009990 tx=0x7fc198009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:58.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:58.927+0000 7fc1927fc640 1 -- 172.21.15.97:0/2741277809 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc1940be6d0 con 0x7fc19c1029a0 2024-04-23T10:25:59.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:58 smithi196 ceph-mon[32919]: pgmap v347: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-04-23T10:25:59.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:58 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/3507173354' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-23T10:25:59.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:58 smithi097 ceph-mon[29197]: pgmap v347: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-04-23T10:25:59.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:58 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/3507173354' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-23T10:25:59.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.117+0000 7fc1a3329640 1 -- 172.21.15.97:0/2741277809 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc19c062b50 con 0x7fc19c1029a0 2024-04-23T10:25:59.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.120+0000 7fc1927fc640 1 -- 172.21.15.97:0/2741277809 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22606 (secure 0 0 0) 0x7fc1940bdfd0 con 0x7fc19c1029a0 2024-04-23T10:25:59.119 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:25:59.119 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":88,"fsid":"9b22af88-0159-11ef-bc93-c7b262605968","created":"2024-04-23T10:16:05.983305+0000","modified":"2024-04-23T10:25:00.660133+0000","last_up_change":"2024-04-23T10:23:13.114761+0000","last_in_change":"2024-04-23T10:22:52.804880+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T10:19:25.559138+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T10:24:04.570049+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T10:24:05.861631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"58cb995d-de35-4522-ac81-4982401b260d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6803","nonce":3031221802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6805","nonce":3031221802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6809","nonce":3031221802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6807","nonce":3031221802}]},"public_addr":"172.21.15.97:6803/3031221802","cluster_addr":"172.21.15.97:6805/3031221802","heartbeat_back_addr":"172.21.15.97:6809/3031221802","heartbeat_front_addr":"172.21.15.97:6807/3031221802","state":["exists","up"]},{"osd":1,"uuid":"b47bd8ee-1f21-4834-acf5-bb40f97ab3ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6801","nonce":1108870440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6803","nonce":1108870440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6807","nonce":1108870440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6805","nonce":1108870440}]},"public_addr":"172.21.15.183:6801/1108870440","cluster_addr":"172.21.15.183:6803/1108870440","heartbeat_back_addr":"172.21.15.183:6807/1108870440","heartbeat_front_addr":"172.21.15.183:6805/1108870440","state":["exists","up"]},{"osd":2,"uuid":"17c4eec3-6c47-4334-9b63-1d5ad9ec5988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6801","nonce":2877039731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6803","nonce":2877039731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6807","nonce":2877039731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6805","nonce":2877039731}]},"public_addr":"172.21.15.196:6801/2877039731","cluster_addr":"172.21.15.196:6803/2877039731","heartbeat_back_addr":"172.21.15.196:6807/2877039731","heartbeat_front_addr":"172.21.15.196:6805/2877039731","state":["exists","up"]},{"osd":3,"uuid":"3f731c38-2e45-4ca1-8dce-efc4a1b2166b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6811","nonce":4276311194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6813","nonce":4276311194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6817","nonce":4276311194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6815","nonce":4276311194}]},"public_addr":"172.21.15.97:6811/4276311194","cluster_addr":"172.21.15.97:6813/4276311194","heartbeat_back_addr":"172.21.15.97:6817/4276311194","heartbeat_front_addr":"172.21.15.97:6815/4276311194","state":["exists","up"]},{"osd":4,"uuid":"fdcafb14-9b82-4134-bd5b-79c0516f6cbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6809","nonce":63913916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6811","nonce":63913916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6815","nonce":63913916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6813","nonce":63913916}]},"public_addr":"172.21.15.183:6809/63913916","cluster_addr":"172.21.15.183:6811/63913916","heartbeat_back_addr":"172.21.15.183:6815/63913916","heartbeat_front_addr":"172.21.15.183:6813/63913916","state":["exists","up"]},{"osd":5,"uuid":"cdcc18d8-da3c-4012-bf12-876bb39e66eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6809","nonce":2804446896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6811","nonce":2804446896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6815","nonce":2804446896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6813","nonce":2804446896}]},"public_addr":"172.21.15.196:6809/2804446896","cluster_addr":"172.21.15.196:6811/2804446896","heartbeat_back_addr":"172.21.15.196:6815/2804446896","heartbeat_front_addr":"172.21.15.196:6813/2804446896","state":["exists","up"]},{"osd":6,"uuid":"071abb5c-2edf-4832-b8aa-3b80d20096fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6819","nonce":2080916705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6821","nonce":2080916705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6825","nonce":2080916705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6823","nonce":2080916705}]},"public_addr":"172.21.15.97:6819/2080916705","cluster_addr":"172.21.15.97:6821/2080916705","heartbeat_back_addr":"172.21.15.97:6825/2080916705","heartbeat_front_addr":"172.21.15.97:6823/2080916705","state":["exists","up"]},{"osd":7,"uuid":"9d54760b-66df-486b-8abb-017f3b64b4d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6817","nonce":1865873033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6819","nonce":1865873033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6823","nonce":1865873033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6821","nonce":1865873033}]},"public_addr":"172.21.15.183:6817/1865873033","cluster_addr":"172.21.15.183:6819/1865873033","heartbeat_back_addr":"172.21.15.183:6823/1865873033","heartbeat_front_addr":"172.21.15.183:6821/1865873033","state":["exists","up"]},{"osd":8,"uuid":"5cd3100a-bd6a-4be5-b5e9-0d0afc325193","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6817","nonce":4063932139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6819","nonce":4063932139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6823","nonce":4063932139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6821","nonce":4063932139}]},"public_addr":"172.21.15.196:6817/4063932139","cluster_addr":"172.21.15.196:6819/4063932139","heartbeat_back_addr":"172.21.15.196:6823/4063932139","heartbeat_front_addr":"172.21.15.196:6821/4063932139","state":["exists","up"]},{"osd":9,"uuid":"b6a5f4a1-4ecc-4f73-a2df-07551d9298ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6827","nonce":1887432474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6829","nonce":1887432474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6833","nonce":1887432474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6831","nonce":1887432474}]},"public_addr":"172.21.15.97:6827/1887432474","cluster_addr":"172.21.15.97:6829/1887432474","heartbeat_back_addr":"172.21.15.97:6833/1887432474","heartbeat_front_addr":"172.21.15.97:6831/1887432474","state":["exists","up"]},{"osd":10,"uuid":"86f59816-1ea6-4ae9-8796-cf5fc4831064","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6825","nonce":799434647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6827","nonce":799434647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6831","nonce":799434647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6829","nonce":799434647}]},"public_addr":"172.21.15.183:6825/799434647","cluster_addr":"172.21.15.183:6827/799434647","heartbeat_back_addr":"172.21.15.183:6831/799434647","heartbeat_front_addr":"172.21.15.183:6829/799434647","state":["exists","up"]},{"osd":11,"uuid":"95d64a89-a9f8-4c7a-8c0d-da7fb26cfff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6825","nonce":1873969467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6827","nonce":1873969467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6831","nonce":1873969467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6829","nonce":1873969467}]},"public_addr":"172.21.15.196:6825/1873969467","cluster_addr":"172.21.15.196:6827/1873969467","heartbeat_back_addr":"172.21.15.196:6831/1873969467","heartbeat_front_addr":"172.21.15.196:6829/1873969467","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:18:33.721156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:18:57.586709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:19:22.069458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:19:46.451597+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:20:11.561019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:20:36.298672+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:21:01.661804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:21:27.324128+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:21:52.898197+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:22:19.614326+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:22:45.154917+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:23:11.222612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/3866016040":"2024-04-24T10:16:59.468237+0000","172.21.15.97:6801/3297166858":"2024-04-24T10:16:59.468237+0000","172.21.15.97:0/3804743548":"2024-04-24T10:16:59.468237+0000","172.21.15.97:6800/3706463017":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/3715079438":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/786806834":"2024-04-24T10:16:59.468237+0000","172.21.15.97:6801/3706463017":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/3294175554":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/1449136760":"2024-04-24T10:16:33.128917+0000","172.21.15.97:6800/3297166858":"2024-04-24T10:16:59.468237+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T10:25:59.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.122+0000 7fc1a3329640 1 -- 172.21.15.97:0/2741277809 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fc1880a3bb0 msgr2=0x7fc1880a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:59.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.122+0000 7fc1a3329640 1 --2- 172.21.15.97:0/2741277809 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fc1880a3bb0 0x7fc1880a6070 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc198009990 tx=0x7fc198009110 comp rx=0 tx=0).stop 2024-04-23T10:25:59.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.122+0000 7fc1a3329640 1 -- 172.21.15.97:0/2741277809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc19c1029a0 msgr2=0x7fc19c19e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:59.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.122+0000 7fc1a3329640 1 --2- 172.21.15.97:0/2741277809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc19c1029a0 0x7fc19c19e7b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc194031a20 tx=0x7fc194003c20 comp rx=0 tx=0).stop 2024-04-23T10:25:59.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.122+0000 7fc1a3329640 1 -- 172.21.15.97:0/2741277809 shutdown_connections 2024-04-23T10:25:59.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.122+0000 7fc1a3329640 1 --2- 172.21.15.97:0/2741277809 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fc1880a3bb0 0x7fc1880a6070 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:59.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.122+0000 7fc1a3329640 1 --2- 172.21.15.97:0/2741277809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc19c1029a0 0x7fc19c19e7b0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:59.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.122+0000 7fc1a3329640 1 --2- 172.21.15.97:0/2741277809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc19c101fe0 0x7fc19c197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:59.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.122+0000 7fc1a3329640 1 --2- 172.21.15.97:0/2741277809 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc19c100e90 0x7fc19c197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:59.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.122+0000 7fc1a3329640 1 -- 172.21.15.97:0/2741277809 >> 172.21.15.97:0/2741277809 conn(0x7fc19c0fc400 msgr2=0x7fc19c105440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:59.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.122+0000 7fc1a3329640 1 -- 172.21.15.97:0/2741277809 shutdown_connections 2024-04-23T10:25:59.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.122+0000 7fc1a3329640 1 -- 172.21.15.97:0/2741277809 wait complete. 2024-04-23T10:25:59.133 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T10:19:25.559138+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T10:24:04.570049+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T10:24:05.861631+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T10:25:59.133 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T10:25:59.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:58 smithi183 ceph-mon[32317]: pgmap v347: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-04-23T10:25:59.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:58 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/3507173354' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-23T10:25:59.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.236+0000 7ff974968640 1 -- 172.21.15.97:0/3653421659 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff970100e90 msgr2=0x7ff9701012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:59.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.236+0000 7ff974968640 1 --2- 172.21.15.97:0/3653421659 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff970100e90 0x7ff9701012b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff960009900 tx=0x7ff96002a410 comp rx=0 tx=0).stop 2024-04-23T10:25:59.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.236+0000 7ff974968640 1 -- 172.21.15.97:0/3653421659 shutdown_connections 2024-04-23T10:25:59.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.236+0000 7ff974968640 1 --2- 172.21.15.97:0/3653421659 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9701029a0 0x7ff9701092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:59.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.236+0000 7ff974968640 1 --2- 172.21.15.97:0/3653421659 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff970101fe0 0x7ff970102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:59.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.236+0000 7ff974968640 1 --2- 172.21.15.97:0/3653421659 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff970100e90 0x7ff9701012b0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:59.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.236+0000 7ff974968640 1 -- 172.21.15.97:0/3653421659 >> 172.21.15.97:0/3653421659 conn(0x7ff9700fc400 msgr2=0x7ff9700fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:59.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.236+0000 7ff974968640 1 -- 172.21.15.97:0/3653421659 shutdown_connections 2024-04-23T10:25:59.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.236+0000 7ff974968640 1 -- 172.21.15.97:0/3653421659 wait complete. 2024-04-23T10:25:59.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.237+0000 7ff974968640 1 Processor -- start 2024-04-23T10:25:59.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.237+0000 7ff974968640 1 -- start start 2024-04-23T10:25:59.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.237+0000 7ff974968640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff970100e90 0x7ff970069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:59.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.238+0000 7ff974968640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff970101fe0 0x7ff97019ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:59.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.238+0000 7ff974968640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9701029a0 0x7ff97019f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:59.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.238+0000 7ff974968640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff97010c9e0 con 0x7ff970101fe0 2024-04-23T10:25:59.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.238+0000 7ff974968640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff97010c860 con 0x7ff970100e90 2024-04-23T10:25:59.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.238+0000 7ff974968640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff97010cb60 con 0x7ff9701029a0 2024-04-23T10:25:59.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.238+0000 7ff96dd9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff970101fe0 0x7ff97019ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:59.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.238+0000 7ff96dd9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff970101fe0 0x7ff97019ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46540/0 (socket says 172.21.15.97:46540) 2024-04-23T10:25:59.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.238+0000 7ff96dd9b640 1 -- 172.21.15.97:0/2688213966 learned_addr learned my addr 172.21.15.97:0/2688213966 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:25:59.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.238+0000 7ff96e59c640 1 --2- 172.21.15.97:0/2688213966 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff970100e90 0x7ff970069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:59.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.238+0000 7ff96ed9d640 1 --2- 172.21.15.97:0/2688213966 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9701029a0 0x7ff97019f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:59.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.238+0000 7ff96dd9b640 1 -- 172.21.15.97:0/2688213966 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9701029a0 msgr2=0x7ff97019f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:59.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.238+0000 7ff96dd9b640 1 --2- 172.21.15.97:0/2688213966 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9701029a0 0x7ff97019f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:59.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.238+0000 7ff96dd9b640 1 -- 172.21.15.97:0/2688213966 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff970100e90 msgr2=0x7ff970069c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:59.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.238+0000 7ff96dd9b640 1 --2- 172.21.15.97:0/2688213966 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff970100e90 0x7ff970069c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:59.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.238+0000 7ff96dd9b640 1 -- 172.21.15.97:0/2688213966 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff97019fe00 con 0x7ff970101fe0 2024-04-23T10:25:59.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.238+0000 7ff96dd9b640 1 --2- 172.21.15.97:0/2688213966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff970101fe0 0x7ff97019ced0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff96400c970 tx=0x7ff96400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:59.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.238+0000 7ff95f7fe640 1 -- 172.21.15.97:0/2688213966 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff964007c10 con 0x7ff970101fe0 2024-04-23T10:25:59.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.238+0000 7ff974968640 1 -- 172.21.15.97:0/2688213966 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9701a0090 con 0x7ff970101fe0 2024-04-23T10:25:59.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.238+0000 7ff95f7fe640 1 -- 172.21.15.97:0/2688213966 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff964007db0 con 0x7ff970101fe0 2024-04-23T10:25:59.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.238+0000 7ff95f7fe640 1 -- 172.21.15.97:0/2688213966 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96400f7d0 con 0x7ff970101fe0 2024-04-23T10:25:59.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.238+0000 7ff974968640 1 -- 172.21.15.97:0/2688213966 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9701a05e0 con 0x7ff970101fe0 2024-04-23T10:25:59.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.239+0000 7ff974968640 1 -- 172.21.15.97:0/2688213966 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9701081a0 con 0x7ff970101fe0 2024-04-23T10:25:59.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.240+0000 7ff95f7fe640 1 -- 172.21.15.97:0/2688213966 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff964018480 con 0x7ff970101fe0 2024-04-23T10:25:59.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.241+0000 7ff95f7fe640 1 --2- 172.21.15.97:0/2688213966 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7ff9580a3bb0 0x7ff9580a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:59.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.241+0000 7ff96e59c640 1 --2- 172.21.15.97:0/2688213966 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7ff9580a3bb0 0x7ff9580a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:59.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.241+0000 7ff95f7fe640 1 -- 172.21.15.97:0/2688213966 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7ff9640d0680 con 0x7ff970101fe0 2024-04-23T10:25:59.241 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.241+0000 7ff96e59c640 1 --2- 172.21.15.97:0/2688213966 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7ff9580a3bb0 0x7ff9580a6070 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff960000c00 tx=0x7ff9600314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:59.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.245+0000 7ff95f7fe640 1 -- 172.21.15.97:0/2688213966 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff96409d120 con 0x7ff970101fe0 2024-04-23T10:25:59.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.429+0000 7ff974968640 1 -- 172.21.15.97:0/2688213966 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff970062b50 con 0x7ff970101fe0 2024-04-23T10:25:59.428 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.429+0000 7ff95f7fe640 1 -- 172.21.15.97:0/2688213966 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7ff96409ca20 con 0x7ff970101fe0 2024-04-23T10:25:59.428 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-23T10:25:59.430 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.431+0000 7ff974968640 1 -- 172.21.15.97:0/2688213966 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7ff9580a3bb0 msgr2=0x7ff9580a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:59.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.431+0000 7ff974968640 1 --2- 172.21.15.97:0/2688213966 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7ff9580a3bb0 0x7ff9580a6070 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff960000c00 tx=0x7ff9600314a0 comp rx=0 tx=0).stop 2024-04-23T10:25:59.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.431+0000 7ff974968640 1 -- 172.21.15.97:0/2688213966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff970101fe0 msgr2=0x7ff97019ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:59.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.431+0000 7ff974968640 1 --2- 172.21.15.97:0/2688213966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff970101fe0 0x7ff97019ced0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff96400c970 tx=0x7ff96400ce30 comp rx=0 tx=0).stop 2024-04-23T10:25:59.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.431+0000 7ff974968640 1 -- 172.21.15.97:0/2688213966 shutdown_connections 2024-04-23T10:25:59.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.431+0000 7ff974968640 1 --2- 172.21.15.97:0/2688213966 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7ff9580a3bb0 0x7ff9580a6070 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:59.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.431+0000 7ff974968640 1 --2- 172.21.15.97:0/2688213966 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9701029a0 0x7ff97019f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:59.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.431+0000 7ff974968640 1 --2- 172.21.15.97:0/2688213966 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff970101fe0 0x7ff97019ced0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:59.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.431+0000 7ff974968640 1 --2- 172.21.15.97:0/2688213966 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff970100e90 0x7ff970069c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:59.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.431+0000 7ff974968640 1 -- 172.21.15.97:0/2688213966 >> 172.21.15.97:0/2688213966 conn(0x7ff9700fc400 msgr2=0x7ff970105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:59.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.431+0000 7ff974968640 1 -- 172.21.15.97:0/2688213966 shutdown_connections 2024-04-23T10:25:59.431 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.431+0000 7ff974968640 1 -- 172.21.15.97:0/2688213966 wait complete. 2024-04-23T10:25:59.441 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T10:25:59.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.544+0000 7fe260ded640 1 -- 172.21.15.97:0/354083875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe25c101fa0 msgr2=0x7fe25c1023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:59.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.544+0000 7fe260ded640 1 --2- 172.21.15.97:0/354083875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe25c101fa0 0x7fe25c1023c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fe24c009900 tx=0x7fe24c02a410 comp rx=0 tx=0).stop 2024-04-23T10:25:59.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.544+0000 7fe260ded640 1 -- 172.21.15.97:0/354083875 shutdown_connections 2024-04-23T10:25:59.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.544+0000 7fe260ded640 1 --2- 172.21.15.97:0/354083875 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe25c103b50 0x7fe25c10a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:59.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.544+0000 7fe260ded640 1 --2- 172.21.15.97:0/354083875 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe25c103190 0x7fe25c103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:59.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.544+0000 7fe260ded640 1 --2- 172.21.15.97:0/354083875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe25c101fa0 0x7fe25c1023c0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:59.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.544+0000 7fe260ded640 1 -- 172.21.15.97:0/354083875 >> 172.21.15.97:0/354083875 conn(0x7fe25c0fd530 msgr2=0x7fe25c0ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:59.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.544+0000 7fe260ded640 1 -- 172.21.15.97:0/354083875 shutdown_connections 2024-04-23T10:25:59.544 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.544+0000 7fe260ded640 1 -- 172.21.15.97:0/354083875 wait complete. 2024-04-23T10:25:59.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.545+0000 7fe260ded640 1 Processor -- start 2024-04-23T10:25:59.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.545+0000 7fe260ded640 1 -- start start 2024-04-23T10:25:59.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.545+0000 7fe260ded640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe25c101fa0 0x7fe25c1981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:59.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.545+0000 7fe260ded640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe25c103190 0x7fe25c1986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:59.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.545+0000 7fe260ded640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe25c103b50 0x7fe25c19f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:59.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.545+0000 7fe260ded640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe25c10dae0 con 0x7fe25c101fa0 2024-04-23T10:25:59.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.545+0000 7fe260ded640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe25c10d960 con 0x7fe25c103190 2024-04-23T10:25:59.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.545+0000 7fe260ded640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe25c10dc60 con 0x7fe25c103b50 2024-04-23T10:25:59.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.545+0000 7fe25a59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe25c101fa0 0x7fe25c1981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:59.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.545+0000 7fe25a59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe25c101fa0 0x7fe25c1981a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46558/0 (socket says 172.21.15.97:46558) 2024-04-23T10:25:59.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.545+0000 7fe25a59c640 1 -- 172.21.15.97:0/3367841646 learned_addr learned my addr 172.21.15.97:0/3367841646 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:25:59.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.545+0000 7fe25ad9d640 1 --2- 172.21.15.97:0/3367841646 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe25c103b50 0x7fe25c19f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:59.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.545+0000 7fe259d9b640 1 --2- 172.21.15.97:0/3367841646 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe25c103190 0x7fe25c1986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:59.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.545+0000 7fe25a59c640 1 -- 172.21.15.97:0/3367841646 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe25c103b50 msgr2=0x7fe25c19f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:59.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.545+0000 7fe25a59c640 1 --2- 172.21.15.97:0/3367841646 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe25c103b50 0x7fe25c19f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:59.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.545+0000 7fe25a59c640 1 -- 172.21.15.97:0/3367841646 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe25c103190 msgr2=0x7fe25c1986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:59.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.545+0000 7fe25a59c640 1 --2- 172.21.15.97:0/3367841646 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe25c103190 0x7fe25c1986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:59.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.545+0000 7fe25a59c640 1 -- 172.21.15.97:0/3367841646 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe25c19fdc0 con 0x7fe25c101fa0 2024-04-23T10:25:59.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.546+0000 7fe25a59c640 1 --2- 172.21.15.97:0/3367841646 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe25c101fa0 0x7fe25c1981a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe24c006010 tx=0x7fe24c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:59.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.546+0000 7fe24b7fe640 1 -- 172.21.15.97:0/3367841646 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe24c03f030 con 0x7fe25c101fa0 2024-04-23T10:25:59.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.546+0000 7fe260ded640 1 -- 172.21.15.97:0/3367841646 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe25c1a0050 con 0x7fe25c101fa0 2024-04-23T10:25:59.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.546+0000 7fe24b7fe640 1 -- 172.21.15.97:0/3367841646 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe24c003cb0 con 0x7fe25c101fa0 2024-04-23T10:25:59.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.546+0000 7fe24b7fe640 1 -- 172.21.15.97:0/3367841646 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe24c035070 con 0x7fe25c101fa0 2024-04-23T10:25:59.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.546+0000 7fe260ded640 1 -- 172.21.15.97:0/3367841646 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe25c1a0540 con 0x7fe25c101fa0 2024-04-23T10:25:59.547 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.547+0000 7fe260ded640 1 -- 172.21.15.97:0/3367841646 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe25c1092e0 con 0x7fe25c101fa0 2024-04-23T10:25:59.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.549+0000 7fe24b7fe640 1 -- 172.21.15.97:0/3367841646 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe24c042030 con 0x7fe25c101fa0 2024-04-23T10:25:59.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.549+0000 7fe24b7fe640 1 --2- 172.21.15.97:0/3367841646 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fe2440a3bb0 0x7fe2440a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:59.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.549+0000 7fe259d9b640 1 --2- 172.21.15.97:0/3367841646 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fe2440a3bb0 0x7fe2440a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:59.548 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.550+0000 7fe24b7fe640 1 -- 172.21.15.97:0/3367841646 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fe24c0f0db0 con 0x7fe25c101fa0 2024-04-23T10:25:59.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.550+0000 7fe259d9b640 1 --2- 172.21.15.97:0/3367841646 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fe2440a3bb0 0x7fe2440a6070 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe25c1996b0 tx=0x7fe25000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:59.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.552+0000 7fe24b7fe640 1 -- 172.21.15.97:0/3367841646 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe24c0bd110 con 0x7fe25c101fa0 2024-04-23T10:25:59.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.738+0000 7fe260ded640 1 -- 172.21.15.97:0/3367841646 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe25c062b50 con 0x7fe25c101fa0 2024-04-23T10:25:59.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.738+0000 7fe24b7fe640 1 -- 172.21.15.97:0/3367841646 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7fe24c0c1410 con 0x7fe25c101fa0 2024-04-23T10:25:59.737 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-23T10:25:59.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.740+0000 7fe260ded640 1 -- 172.21.15.97:0/3367841646 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fe2440a3bb0 msgr2=0x7fe2440a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:59.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.740+0000 7fe260ded640 1 --2- 172.21.15.97:0/3367841646 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fe2440a3bb0 0x7fe2440a6070 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe25c1996b0 tx=0x7fe25000b390 comp rx=0 tx=0).stop 2024-04-23T10:25:59.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.740+0000 7fe260ded640 1 -- 172.21.15.97:0/3367841646 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe25c101fa0 msgr2=0x7fe25c1981a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:59.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.740+0000 7fe260ded640 1 --2- 172.21.15.97:0/3367841646 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe25c101fa0 0x7fe25c1981a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe24c006010 tx=0x7fe24c0042f0 comp rx=0 tx=0).stop 2024-04-23T10:25:59.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.741+0000 7fe260ded640 1 -- 172.21.15.97:0/3367841646 shutdown_connections 2024-04-23T10:25:59.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.741+0000 7fe260ded640 1 --2- 172.21.15.97:0/3367841646 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fe2440a3bb0 0x7fe2440a6070 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:59.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.741+0000 7fe260ded640 1 --2- 172.21.15.97:0/3367841646 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe25c103b50 0x7fe25c19f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:59.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.741+0000 7fe260ded640 1 --2- 172.21.15.97:0/3367841646 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe25c103190 0x7fe25c1986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:59.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.741+0000 7fe260ded640 1 --2- 172.21.15.97:0/3367841646 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe25c101fa0 0x7fe25c1981a0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:59.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.741+0000 7fe260ded640 1 -- 172.21.15.97:0/3367841646 >> 172.21.15.97:0/3367841646 conn(0x7fe25c0fd530 msgr2=0x7fe25c1064b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:59.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.741+0000 7fe260ded640 1 -- 172.21.15.97:0/3367841646 shutdown_connections 2024-04-23T10:25:59.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.741+0000 7fe260ded640 1 -- 172.21.15.97:0/3367841646 wait complete. 2024-04-23T10:25:59.749 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:59 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/2741277809' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T10:25:59.750 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:25:59 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/2688213966' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T10:25:59.750 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T10:25:59.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.853+0000 7f7a5b120640 1 -- 172.21.15.97:0/1782448806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a54103b50 msgr2=0x7f7a5410a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:59.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.853+0000 7f7a5b120640 1 --2- 172.21.15.97:0/1782448806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a54103b50 0x7f7a5410a3e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c009ca0 tx=0x7f7a4c02a8c0 comp rx=0 tx=0).stop 2024-04-23T10:25:59.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.853+0000 7f7a5b120640 1 -- 172.21.15.97:0/1782448806 shutdown_connections 2024-04-23T10:25:59.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.853+0000 7f7a5b120640 1 --2- 172.21.15.97:0/1782448806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a54103b50 0x7f7a5410a3e0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:59.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.853+0000 7f7a5b120640 1 --2- 172.21.15.97:0/1782448806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a54103190 0x7f7a54103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:59.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.853+0000 7f7a5b120640 1 --2- 172.21.15.97:0/1782448806 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a54101fa0 0x7f7a541023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:59.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.853+0000 7f7a5b120640 1 -- 172.21.15.97:0/1782448806 >> 172.21.15.97:0/1782448806 conn(0x7f7a540fd530 msgr2=0x7f7a540ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:25:59.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.853+0000 7f7a5b120640 1 -- 172.21.15.97:0/1782448806 shutdown_connections 2024-04-23T10:25:59.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.853+0000 7f7a5b120640 1 -- 172.21.15.97:0/1782448806 wait complete. 2024-04-23T10:25:59.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.854+0000 7f7a5b120640 1 Processor -- start 2024-04-23T10:25:59.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.854+0000 7f7a5b120640 1 -- start start 2024-04-23T10:25:59.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.854+0000 7f7a5b120640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a54101fa0 0x7f7a54075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:59.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.854+0000 7f7a5b120640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a54103190 0x7f7a54074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:59.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.854+0000 7f7a5b120640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a54103b50 0x7f7a54074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:59.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.854+0000 7f7a5b120640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5410dae0 con 0x7f7a54103b50 2024-04-23T10:25:59.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.854+0000 7f7a5b120640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5410d960 con 0x7f7a54101fa0 2024-04-23T10:25:59.855 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.854+0000 7f7a5b120640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5410dc60 con 0x7f7a54103190 2024-04-23T10:25:59.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.854+0000 7f7a596bd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a54103b50 0x7f7a54074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:59.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.854+0000 7f7a596bd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a54103b50 0x7f7a54074880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46580/0 (socket says 172.21.15.97:46580) 2024-04-23T10:25:59.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.854+0000 7f7a596bd640 1 -- 172.21.15.97:0/1236425876 learned_addr learned my addr 172.21.15.97:0/1236425876 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:25:59.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.854+0000 7f7a58ebc640 1 --2- 172.21.15.97:0/1236425876 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a54101fa0 0x7f7a54075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:59.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.854+0000 7f7a596bd640 1 -- 172.21.15.97:0/1236425876 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a54103190 msgr2=0x7f7a54074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:59.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.854+0000 7f7a53fff640 1 --2- 172.21.15.97:0/1236425876 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a54103190 0x7f7a54074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:59.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.854+0000 7f7a596bd640 1 --2- 172.21.15.97:0/1236425876 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a54103190 0x7f7a54074340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:59.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.854+0000 7f7a596bd640 1 -- 172.21.15.97:0/1236425876 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a54101fa0 msgr2=0x7f7a54075d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:25:59.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.854+0000 7f7a596bd640 1 --2- 172.21.15.97:0/1236425876 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a54101fa0 0x7f7a54075d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:25:59.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.854+0000 7f7a596bd640 1 -- 172.21.15.97:0/1236425876 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a54075110 con 0x7f7a54103b50 2024-04-23T10:25:59.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.854+0000 7f7a58ebc640 1 --2- 172.21.15.97:0/1236425876 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a54101fa0 0x7f7a54075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:25:59.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.854+0000 7f7a596bd640 1 --2- 172.21.15.97:0/1236425876 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a54103b50 0x7f7a54074880 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c0097b0 tx=0x7f7a4c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:59.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.854+0000 7f7a51ffb640 1 -- 172.21.15.97:0/1236425876 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c004240 con 0x7f7a54103b50 2024-04-23T10:25:59.856 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.855+0000 7f7a51ffb640 1 -- 172.21.15.97:0/1236425876 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a4c0317c0 con 0x7f7a54103b50 2024-04-23T10:25:59.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.855+0000 7f7a5b120640 1 -- 172.21.15.97:0/1236425876 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a541a22a0 con 0x7f7a54103b50 2024-04-23T10:25:59.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.855+0000 7f7a51ffb640 1 -- 172.21.15.97:0/1236425876 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c02fd80 con 0x7f7a54103b50 2024-04-23T10:25:59.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.855+0000 7f7a5b120640 1 -- 172.21.15.97:0/1236425876 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a541a27f0 con 0x7f7a54103b50 2024-04-23T10:25:59.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.856+0000 7f7a5b120640 1 -- 172.21.15.97:0/1236425876 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a541092c0 con 0x7f7a54103b50 2024-04-23T10:25:59.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.857+0000 7f7a51ffb640 1 -- 172.21.15.97:0/1236425876 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7a4c0043e0 con 0x7f7a54103b50 2024-04-23T10:25:59.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.857+0000 7f7a51ffb640 1 --2- 172.21.15.97:0/1236425876 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f7a440a3bb0 0x7f7a440a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:25:59.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.857+0000 7f7a58ebc640 1 --2- 172.21.15.97:0/1236425876 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f7a440a3bb0 0x7f7a440a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:25:59.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.857+0000 7f7a51ffb640 1 -- 172.21.15.97:0/1236425876 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f7a4c0f1ce0 con 0x7f7a54103b50 2024-04-23T10:25:59.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.858+0000 7f7a58ebc640 1 --2- 172.21.15.97:0/1236425876 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f7a440a3bb0 0x7f7a440a6070 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7a54102ff0 tx=0x7f7a4800a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:25:59.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:25:59.862+0000 7f7a51ffb640 1 -- 172.21.15.97:0/1236425876 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7a4c0be6f0 con 0x7f7a54103b50 2024-04-23T10:26:00.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.041+0000 7f7a5b120640 1 -- 172.21.15.97:0/1236425876 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7a54062b50 con 0x7f7a54103b50 2024-04-23T10:26:00.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.041+0000 7f7a51ffb640 1 -- 172.21.15.97:0/1236425876 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f7a4c0de070 con 0x7f7a54103b50 2024-04-23T10:26:00.040 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-04-23T10:26:00.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.043+0000 7f7a5b120640 1 -- 172.21.15.97:0/1236425876 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f7a440a3bb0 msgr2=0x7f7a440a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:00.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.043+0000 7f7a5b120640 1 --2- 172.21.15.97:0/1236425876 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f7a440a3bb0 0x7f7a440a6070 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7a54102ff0 tx=0x7f7a4800a560 comp rx=0 tx=0).stop 2024-04-23T10:26:00.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.043+0000 7f7a5b120640 1 -- 172.21.15.97:0/1236425876 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a54103b50 msgr2=0x7f7a54074880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:00.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.043+0000 7f7a5b120640 1 --2- 172.21.15.97:0/1236425876 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a54103b50 0x7f7a54074880 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c0097b0 tx=0x7f7a4c003c20 comp rx=0 tx=0).stop 2024-04-23T10:26:00.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.043+0000 7f7a5b120640 1 -- 172.21.15.97:0/1236425876 shutdown_connections 2024-04-23T10:26:00.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.043+0000 7f7a5b120640 1 --2- 172.21.15.97:0/1236425876 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f7a440a3bb0 0x7f7a440a6070 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:00.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.043+0000 7f7a5b120640 1 --2- 172.21.15.97:0/1236425876 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a54103b50 0x7f7a54074880 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:00.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.043+0000 7f7a5b120640 1 --2- 172.21.15.97:0/1236425876 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a54103190 0x7f7a54074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:00.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.043+0000 7f7a5b120640 1 --2- 172.21.15.97:0/1236425876 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a54101fa0 0x7f7a54075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:00.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.043+0000 7f7a5b120640 1 -- 172.21.15.97:0/1236425876 >> 172.21.15.97:0/1236425876 conn(0x7f7a540fd530 msgr2=0x7f7a54106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:00.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.043+0000 7f7a5b120640 1 -- 172.21.15.97:0/1236425876 shutdown_connections 2024-04-23T10:26:00.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.043+0000 7f7a5b120640 1 -- 172.21.15.97:0/1236425876 wait complete. 2024-04-23T10:26:00.053 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T10:26:00.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:59 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/2741277809' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T10:26:00.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:25:59 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/2688213966' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T10:26:00.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:59 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/2741277809' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T10:26:00.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:25:59 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/2688213966' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T10:26:00.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.152+0000 7f197678e640 1 -- 172.21.15.97:0/2323866285 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1970102a30 msgr2=0x7f1970104e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:00.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.152+0000 7f197678e640 1 --2- 172.21.15.97:0/2323866285 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1970102a30 0x7f1970104e20 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1964009900 tx=0x7f196402a410 comp rx=0 tx=0).stop 2024-04-23T10:26:00.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.153+0000 7f197678e640 1 -- 172.21.15.97:0/2323866285 shutdown_connections 2024-04-23T10:26:00.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.153+0000 7f197678e640 1 --2- 172.21.15.97:0/2323866285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1970107c90 0x7f197010a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:00.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.153+0000 7f197678e640 1 --2- 172.21.15.97:0/2323866285 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1970105360 0x7f1970107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:00.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.153+0000 7f197678e640 1 --2- 172.21.15.97:0/2323866285 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1970102a30 0x7f1970104e20 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:00.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.153+0000 7f197678e640 1 -- 172.21.15.97:0/2323866285 >> 172.21.15.97:0/2323866285 conn(0x7f19700fc440 msgr2=0x7f19700fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:00.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.153+0000 7f197678e640 1 -- 172.21.15.97:0/2323866285 shutdown_connections 2024-04-23T10:26:00.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.153+0000 7f197678e640 1 -- 172.21.15.97:0/2323866285 wait complete. 2024-04-23T10:26:00.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.153+0000 7f197678e640 1 Processor -- start 2024-04-23T10:26:00.153 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.153+0000 7f197678e640 1 -- start start 2024-04-23T10:26:00.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.153+0000 7f197678e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1970102a30 0x7f19701982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:00.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.153+0000 7f197678e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1970105360 0x7f19701987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:00.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.153+0000 7f197678e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1970107c90 0x7f197019f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:00.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.153+0000 7f197678e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f197010db70 con 0x7f1970107c90 2024-04-23T10:26:00.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.153+0000 7f197678e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f197010d9f0 con 0x7f1970102a30 2024-04-23T10:26:00.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.153+0000 7f197678e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f197010dcf0 con 0x7f1970105360 2024-04-23T10:26:00.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.154+0000 7f1974d2b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1970107c90 0x7f197019f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:00.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.154+0000 7f1974d2b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1970107c90 0x7f197019f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46588/0 (socket says 172.21.15.97:46588) 2024-04-23T10:26:00.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.154+0000 7f1974d2b640 1 -- 172.21.15.97:0/1463553237 learned_addr learned my addr 172.21.15.97:0/1463553237 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:00.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.154+0000 7f196ffff640 1 --2- 172.21.15.97:0/1463553237 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1970102a30 0x7f19701982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:00.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.154+0000 7f196f7fe640 1 --2- 172.21.15.97:0/1463553237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1970105360 0x7f19701987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:00.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.154+0000 7f1974d2b640 1 -- 172.21.15.97:0/1463553237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1970105360 msgr2=0x7f19701987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:00.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.154+0000 7f1974d2b640 1 --2- 172.21.15.97:0/1463553237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1970105360 0x7f19701987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:00.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.154+0000 7f1974d2b640 1 -- 172.21.15.97:0/1463553237 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1970102a30 msgr2=0x7f19701982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:00.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.154+0000 7f1974d2b640 1 --2- 172.21.15.97:0/1463553237 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1970102a30 0x7f19701982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:00.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.154+0000 7f1974d2b640 1 -- 172.21.15.97:0/1463553237 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f197019fec0 con 0x7f1970107c90 2024-04-23T10:26:00.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.154+0000 7f196f7fe640 1 --2- 172.21.15.97:0/1463553237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1970105360 0x7f19701987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:26:00.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.154+0000 7f1974d2b640 1 --2- 172.21.15.97:0/1463553237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1970107c90 0x7f197019f850 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f196800b970 tx=0x7f196800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:00.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.154+0000 7f196d7fa640 1 -- 172.21.15.97:0/1463553237 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1968019070 con 0x7f1970107c90 2024-04-23T10:26:00.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.154+0000 7f197678e640 1 -- 172.21.15.97:0/1463553237 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19701a01b0 con 0x7f1970107c90 2024-04-23T10:26:00.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.154+0000 7f196d7fa640 1 -- 172.21.15.97:0/1463553237 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1968014440 con 0x7f1970107c90 2024-04-23T10:26:00.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.154+0000 7f196d7fa640 1 -- 172.21.15.97:0/1463553237 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19680075d0 con 0x7f1970107c90 2024-04-23T10:26:00.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.154+0000 7f197678e640 1 -- 172.21.15.97:0/1463553237 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19701a0730 con 0x7f1970107c90 2024-04-23T10:26:00.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.155+0000 7f197678e640 1 -- 172.21.15.97:0/1463553237 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19700fdf30 con 0x7f1970107c90 2024-04-23T10:26:00.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.156+0000 7f196d7fa640 1 -- 172.21.15.97:0/1463553237 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f196801c070 con 0x7f1970107c90 2024-04-23T10:26:00.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.157+0000 7f196d7fa640 1 --2- 172.21.15.97:0/1463553237 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f19580a3bb0 0x7f19580a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:00.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.157+0000 7f196d7fa640 1 -- 172.21.15.97:0/1463553237 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f19680cf950 con 0x7f1970107c90 2024-04-23T10:26:00.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.157+0000 7f196ffff640 1 --2- 172.21.15.97:0/1463553237 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f19580a3bb0 0x7f19580a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:00.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.157+0000 7f196ffff640 1 --2- 172.21.15.97:0/1463553237 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f19580a3bb0 0x7f19580a6070 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1964000c00 tx=0x7f19640314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:00.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.161+0000 7f196d7fa640 1 -- 172.21.15.97:0/1463553237 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f196809bd10 con 0x7f1970107c90 2024-04-23T10:26:00.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.376+0000 7f197678e640 1 -- 172.21.15.97:0/1463553237 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f197002d5c0 con 0x7f1970107c90 2024-04-23T10:26:00.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.377+0000 7f196d7fa640 1 -- 172.21.15.97:0/1463553237 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11931 (secure 0 0 0) 0x7f19680bae20 con 0x7f1970107c90 2024-04-23T10:26:00.376 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:26:00.376 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3946878762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3946878762},{"type":"v1","addr":"172.21.15.97:6835","nonce":3946878762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/209800861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":209800861},{"type":"v1","addr":"172.21.15.97:6837","nonce":209800861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T10:24:06.910134+0000","modified":"2024-04-23T10:24:12.969478+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6839/418013575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":418013575},{"type":"v1","addr":"172.21.15.97:6839","nonce":418013575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6841/747242680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":747242680},{"type":"v1","addr":"172.21.15.97:6841","nonce":747242680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6833/4292847121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4292847121},{"type":"v1","addr":"172.21.15.183:6833","nonce":4292847121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6833/3885816565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3885816565},{"type":"v1","addr":"172.21.15.196:6833","nonce":3885816565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1897093105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1897093105},{"type":"v1","addr":"172.21.15.183:6835","nonce":1897093105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1171477563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1171477563},{"type":"v1","addr":"172.21.15.196:6835","nonce":1171477563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1898341911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1898341911},{"type":"v1","addr":"172.21.15.183:6837","nonce":1898341911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.196:6837/3094946502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3094946502},{"type":"v1","addr":"172.21.15.196:6837","nonce":3094946502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.196:6839/83843556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":83843556},{"type":"v1","addr":"172.21.15.196:6839","nonce":83843556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6839/1763841130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1763841130},{"type":"v1","addr":"172.21.15.183:6839","nonce":1763841130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24469,24488,24398,24403,24487]},"id":1}]} 2024-04-23T10:26:00.377 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 25 2024-04-23T10:26:00.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.379+0000 7f197678e640 1 -- 172.21.15.97:0/1463553237 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f19580a3bb0 msgr2=0x7f19580a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:00.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.379+0000 7f197678e640 1 --2- 172.21.15.97:0/1463553237 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f19580a3bb0 0x7f19580a6070 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1964000c00 tx=0x7f19640314a0 comp rx=0 tx=0).stop 2024-04-23T10:26:00.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.379+0000 7f197678e640 1 -- 172.21.15.97:0/1463553237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1970107c90 msgr2=0x7f197019f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:00.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.379+0000 7f197678e640 1 --2- 172.21.15.97:0/1463553237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1970107c90 0x7f197019f850 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f196800b970 tx=0x7f196800be30 comp rx=0 tx=0).stop 2024-04-23T10:26:00.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.379+0000 7f197678e640 1 -- 172.21.15.97:0/1463553237 shutdown_connections 2024-04-23T10:26:00.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.379+0000 7f197678e640 1 --2- 172.21.15.97:0/1463553237 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f19580a3bb0 0x7f19580a6070 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:00.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.379+0000 7f197678e640 1 --2- 172.21.15.97:0/1463553237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1970107c90 0x7f197019f850 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:00.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.379+0000 7f197678e640 1 --2- 172.21.15.97:0/1463553237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1970105360 0x7f19701987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:00.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.379+0000 7f197678e640 1 --2- 172.21.15.97:0/1463553237 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1970102a30 0x7f19701982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:00.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.379+0000 7f197678e640 1 -- 172.21.15.97:0/1463553237 >> 172.21.15.97:0/1463553237 conn(0x7f19700fc440 msgr2=0x7f1970103490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:00.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.379+0000 7f197678e640 1 -- 172.21.15.97:0/1463553237 shutdown_connections 2024-04-23T10:26:00.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.379+0000 7f197678e640 1 -- 172.21.15.97:0/1463553237 wait complete. 2024-04-23T10:26:00.390 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T10:26:00.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.489+0000 7fa25b14b640 1 -- 172.21.15.97:0/3128882748 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2541029b0 msgr2=0x7fa2541092d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:00.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.489+0000 7fa25b14b640 1 --2- 172.21.15.97:0/3128882748 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2541029b0 0x7fa2541092d0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa244009900 tx=0x7fa24402a410 comp rx=0 tx=0).stop 2024-04-23T10:26:00.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.489+0000 7fa25b14b640 1 -- 172.21.15.97:0/3128882748 shutdown_connections 2024-04-23T10:26:00.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.489+0000 7fa25b14b640 1 --2- 172.21.15.97:0/3128882748 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2541029b0 0x7fa2541092d0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:00.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.489+0000 7fa25b14b640 1 --2- 172.21.15.97:0/3128882748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa254101ff0 0x7fa254102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:00.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.489+0000 7fa25b14b640 1 --2- 172.21.15.97:0/3128882748 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa254100ea0 0x7fa2541012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:00.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.489+0000 7fa25b14b640 1 -- 172.21.15.97:0/3128882748 >> 172.21.15.97:0/3128882748 conn(0x7fa2540fc430 msgr2=0x7fa2540fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:00.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.489+0000 7fa25b14b640 1 -- 172.21.15.97:0/3128882748 shutdown_connections 2024-04-23T10:26:00.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.489+0000 7fa25b14b640 1 -- 172.21.15.97:0/3128882748 wait complete. 2024-04-23T10:26:00.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.489+0000 7fa25b14b640 1 Processor -- start 2024-04-23T10:26:00.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.489+0000 7fa25b14b640 1 -- start start 2024-04-23T10:26:00.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.490+0000 7fa25b14b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa254100ea0 0x7fa2541981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:00.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.490+0000 7fa25b14b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa254101ff0 0x7fa2541986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:00.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.490+0000 7fa25b14b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa2541029b0 0x7fa25419f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:00.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.490+0000 7fa25b14b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa25410db40 con 0x7fa254101ff0 2024-04-23T10:26:00.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.490+0000 7fa25b14b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa25410d9c0 con 0x7fa2541029b0 2024-04-23T10:26:00.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.490+0000 7fa25b14b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa25410dcc0 con 0x7fa254100ea0 2024-04-23T10:26:00.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.490+0000 7fa253fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa254101ff0 0x7fa2541986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:00.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.490+0000 7fa253fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa254101ff0 0x7fa2541986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46604/0 (socket says 172.21.15.97:46604) 2024-04-23T10:26:00.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.490+0000 7fa253fff640 1 -- 172.21.15.97:0/3390750251 learned_addr learned my addr 172.21.15.97:0/3390750251 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:00.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.490+0000 7fa2596e8640 1 --2- 172.21.15.97:0/3390750251 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa2541029b0 0x7fa25419f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:00.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.490+0000 7fa258ee7640 1 --2- 172.21.15.97:0/3390750251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa254100ea0 0x7fa2541981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:00.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.490+0000 7fa253fff640 1 -- 172.21.15.97:0/3390750251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa254100ea0 msgr2=0x7fa2541981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:00.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.490+0000 7fa253fff640 1 --2- 172.21.15.97:0/3390750251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa254100ea0 0x7fa2541981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:00.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.490+0000 7fa253fff640 1 -- 172.21.15.97:0/3390750251 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa2541029b0 msgr2=0x7fa25419f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:00.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.490+0000 7fa253fff640 1 --2- 172.21.15.97:0/3390750251 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa2541029b0 0x7fa25419f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:00.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.490+0000 7fa253fff640 1 -- 172.21.15.97:0/3390750251 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa25419fdc0 con 0x7fa254101ff0 2024-04-23T10:26:00.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.490+0000 7fa2596e8640 1 --2- 172.21.15.97:0/3390750251 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa2541029b0 0x7fa25419f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:26:00.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.490+0000 7fa258ee7640 1 --2- 172.21.15.97:0/3390750251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa254100ea0 0x7fa2541981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:26:00.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.490+0000 7fa253fff640 1 --2- 172.21.15.97:0/3390750251 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa254101ff0 0x7fa2541986e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa23c00d950 tx=0x7fa23c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:00.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.490+0000 7fa251ffb640 1 -- 172.21.15.97:0/3390750251 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23c017070 con 0x7fa254101ff0 2024-04-23T10:26:00.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.490+0000 7fa251ffb640 1 -- 172.21.15.97:0/3390750251 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa23c009c30 con 0x7fa254101ff0 2024-04-23T10:26:00.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.490+0000 7fa25b14b640 1 -- 172.21.15.97:0/3390750251 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2541a00b0 con 0x7fa254101ff0 2024-04-23T10:26:00.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.490+0000 7fa251ffb640 1 -- 172.21.15.97:0/3390750251 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23c00f620 con 0x7fa254101ff0 2024-04-23T10:26:00.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.491+0000 7fa25b14b640 1 -- 172.21.15.97:0/3390750251 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2541a05b0 con 0x7fa254101ff0 2024-04-23T10:26:00.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.491+0000 7fa25b14b640 1 -- 172.21.15.97:0/3390750251 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2541081b0 con 0x7fa254101ff0 2024-04-23T10:26:00.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.493+0000 7fa251ffb640 1 -- 172.21.15.97:0/3390750251 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa23c01a070 con 0x7fa254101ff0 2024-04-23T10:26:00.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.493+0000 7fa251ffb640 1 --2- 172.21.15.97:0/3390750251 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fa2400a3bb0 0x7fa2400a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:00.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.493+0000 7fa258ee7640 1 --2- 172.21.15.97:0/3390750251 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fa2400a3bb0 0x7fa2400a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:00.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.493+0000 7fa251ffb640 1 -- 172.21.15.97:0/3390750251 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fa23c0cfea0 con 0x7fa254101ff0 2024-04-23T10:26:00.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.494+0000 7fa258ee7640 1 --2- 172.21.15.97:0/3390750251 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fa2400a3bb0 0x7fa2400a6070 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa24c005fd0 tx=0x7fa24c009510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:00.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.497+0000 7fa251ffb640 1 -- 172.21.15.97:0/3390750251 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa23c009de0 con 0x7fa254101ff0 2024-04-23T10:26:00.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.677+0000 7fa25b14b640 1 -- 172.21.15.97:0/3390750251 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa25402d830 con 0x7fa254101ff0 2024-04-23T10:26:00.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.679+0000 7fa251ffb640 1 -- 172.21.15.97:0/3390750251 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22606 (secure 0 0 0) 0x7fa23c09c8b0 con 0x7fa254101ff0 2024-04-23T10:26:00.678 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:26:00.678 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":88,"fsid":"9b22af88-0159-11ef-bc93-c7b262605968","created":"2024-04-23T10:16:05.983305+0000","modified":"2024-04-23T10:25:00.660133+0000","last_up_change":"2024-04-23T10:23:13.114761+0000","last_in_change":"2024-04-23T10:22:52.804880+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T10:19:25.559138+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T10:24:04.570049+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T10:24:05.861631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"58cb995d-de35-4522-ac81-4982401b260d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6803","nonce":3031221802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6805","nonce":3031221802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6809","nonce":3031221802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6807","nonce":3031221802}]},"public_addr":"172.21.15.97:6803/3031221802","cluster_addr":"172.21.15.97:6805/3031221802","heartbeat_back_addr":"172.21.15.97:6809/3031221802","heartbeat_front_addr":"172.21.15.97:6807/3031221802","state":["exists","up"]},{"osd":1,"uuid":"b47bd8ee-1f21-4834-acf5-bb40f97ab3ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6801","nonce":1108870440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6803","nonce":1108870440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6807","nonce":1108870440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6805","nonce":1108870440}]},"public_addr":"172.21.15.183:6801/1108870440","cluster_addr":"172.21.15.183:6803/1108870440","heartbeat_back_addr":"172.21.15.183:6807/1108870440","heartbeat_front_addr":"172.21.15.183:6805/1108870440","state":["exists","up"]},{"osd":2,"uuid":"17c4eec3-6c47-4334-9b63-1d5ad9ec5988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6801","nonce":2877039731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6803","nonce":2877039731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6807","nonce":2877039731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6805","nonce":2877039731}]},"public_addr":"172.21.15.196:6801/2877039731","cluster_addr":"172.21.15.196:6803/2877039731","heartbeat_back_addr":"172.21.15.196:6807/2877039731","heartbeat_front_addr":"172.21.15.196:6805/2877039731","state":["exists","up"]},{"osd":3,"uuid":"3f731c38-2e45-4ca1-8dce-efc4a1b2166b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6811","nonce":4276311194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6813","nonce":4276311194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6817","nonce":4276311194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6815","nonce":4276311194}]},"public_addr":"172.21.15.97:6811/4276311194","cluster_addr":"172.21.15.97:6813/4276311194","heartbeat_back_addr":"172.21.15.97:6817/4276311194","heartbeat_front_addr":"172.21.15.97:6815/4276311194","state":["exists","up"]},{"osd":4,"uuid":"fdcafb14-9b82-4134-bd5b-79c0516f6cbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6809","nonce":63913916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6811","nonce":63913916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6815","nonce":63913916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6813","nonce":63913916}]},"public_addr":"172.21.15.183:6809/63913916","cluster_addr":"172.21.15.183:6811/63913916","heartbeat_back_addr":"172.21.15.183:6815/63913916","heartbeat_front_addr":"172.21.15.183:6813/63913916","state":["exists","up"]},{"osd":5,"uuid":"cdcc18d8-da3c-4012-bf12-876bb39e66eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6809","nonce":2804446896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6811","nonce":2804446896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6815","nonce":2804446896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6813","nonce":2804446896}]},"public_addr":"172.21.15.196:6809/2804446896","cluster_addr":"172.21.15.196:6811/2804446896","heartbeat_back_addr":"172.21.15.196:6815/2804446896","heartbeat_front_addr":"172.21.15.196:6813/2804446896","state":["exists","up"]},{"osd":6,"uuid":"071abb5c-2edf-4832-b8aa-3b80d20096fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6819","nonce":2080916705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6821","nonce":2080916705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6825","nonce":2080916705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6823","nonce":2080916705}]},"public_addr":"172.21.15.97:6819/2080916705","cluster_addr":"172.21.15.97:6821/2080916705","heartbeat_back_addr":"172.21.15.97:6825/2080916705","heartbeat_front_addr":"172.21.15.97:6823/2080916705","state":["exists","up"]},{"osd":7,"uuid":"9d54760b-66df-486b-8abb-017f3b64b4d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6817","nonce":1865873033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6819","nonce":1865873033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6823","nonce":1865873033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6821","nonce":1865873033}]},"public_addr":"172.21.15.183:6817/1865873033","cluster_addr":"172.21.15.183:6819/1865873033","heartbeat_back_addr":"172.21.15.183:6823/1865873033","heartbeat_front_addr":"172.21.15.183:6821/1865873033","state":["exists","up"]},{"osd":8,"uuid":"5cd3100a-bd6a-4be5-b5e9-0d0afc325193","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6817","nonce":4063932139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6819","nonce":4063932139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6823","nonce":4063932139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6821","nonce":4063932139}]},"public_addr":"172.21.15.196:6817/4063932139","cluster_addr":"172.21.15.196:6819/4063932139","heartbeat_back_addr":"172.21.15.196:6823/4063932139","heartbeat_front_addr":"172.21.15.196:6821/4063932139","state":["exists","up"]},{"osd":9,"uuid":"b6a5f4a1-4ecc-4f73-a2df-07551d9298ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6827","nonce":1887432474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6829","nonce":1887432474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6833","nonce":1887432474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6831","nonce":1887432474}]},"public_addr":"172.21.15.97:6827/1887432474","cluster_addr":"172.21.15.97:6829/1887432474","heartbeat_back_addr":"172.21.15.97:6833/1887432474","heartbeat_front_addr":"172.21.15.97:6831/1887432474","state":["exists","up"]},{"osd":10,"uuid":"86f59816-1ea6-4ae9-8796-cf5fc4831064","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6825","nonce":799434647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6827","nonce":799434647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6831","nonce":799434647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6829","nonce":799434647}]},"public_addr":"172.21.15.183:6825/799434647","cluster_addr":"172.21.15.183:6827/799434647","heartbeat_back_addr":"172.21.15.183:6831/799434647","heartbeat_front_addr":"172.21.15.183:6829/799434647","state":["exists","up"]},{"osd":11,"uuid":"95d64a89-a9f8-4c7a-8c0d-da7fb26cfff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6825","nonce":1873969467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6827","nonce":1873969467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6831","nonce":1873969467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6829","nonce":1873969467}]},"public_addr":"172.21.15.196:6825/1873969467","cluster_addr":"172.21.15.196:6827/1873969467","heartbeat_back_addr":"172.21.15.196:6831/1873969467","heartbeat_front_addr":"172.21.15.196:6829/1873969467","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:18:33.721156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:18:57.586709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:19:22.069458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:19:46.451597+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:20:11.561019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:20:36.298672+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:21:01.661804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:21:27.324128+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:21:52.898197+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:22:19.614326+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:22:45.154917+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:23:11.222612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/3866016040":"2024-04-24T10:16:59.468237+0000","172.21.15.97:6801/3297166858":"2024-04-24T10:16:59.468237+0000","172.21.15.97:0/3804743548":"2024-04-24T10:16:59.468237+0000","172.21.15.97:6800/3706463017":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/3715079438":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/786806834":"2024-04-24T10:16:59.468237+0000","172.21.15.97:6801/3706463017":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/3294175554":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/1449136760":"2024-04-24T10:16:33.128917+0000","172.21.15.97:6800/3297166858":"2024-04-24T10:16:59.468237+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T10:26:00.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.681+0000 7fa25b14b640 1 -- 172.21.15.97:0/3390750251 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fa2400a3bb0 msgr2=0x7fa2400a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:00.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.681+0000 7fa25b14b640 1 --2- 172.21.15.97:0/3390750251 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fa2400a3bb0 0x7fa2400a6070 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa24c005fd0 tx=0x7fa24c009510 comp rx=0 tx=0).stop 2024-04-23T10:26:00.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.681+0000 7fa25b14b640 1 -- 172.21.15.97:0/3390750251 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa254101ff0 msgr2=0x7fa2541986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:00.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.681+0000 7fa25b14b640 1 --2- 172.21.15.97:0/3390750251 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa254101ff0 0x7fa2541986e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa23c00d950 tx=0x7fa23c00de10 comp rx=0 tx=0).stop 2024-04-23T10:26:00.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.681+0000 7fa25b14b640 1 -- 172.21.15.97:0/3390750251 shutdown_connections 2024-04-23T10:26:00.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.681+0000 7fa25b14b640 1 --2- 172.21.15.97:0/3390750251 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fa2400a3bb0 0x7fa2400a6070 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:00.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.681+0000 7fa25b14b640 1 --2- 172.21.15.97:0/3390750251 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa2541029b0 0x7fa25419f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:00.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.681+0000 7fa25b14b640 1 --2- 172.21.15.97:0/3390750251 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa254101ff0 0x7fa2541986e0 secure :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa23c00d950 tx=0x7fa23c00de10 comp rx=0 tx=0).stop 2024-04-23T10:26:00.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.681+0000 7fa25b14b640 1 --2- 172.21.15.97:0/3390750251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa254100ea0 0x7fa2541981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:00.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.681+0000 7fa25b14b640 1 -- 172.21.15.97:0/3390750251 >> 172.21.15.97:0/3390750251 conn(0x7fa2540fc430 msgr2=0x7fa254105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:00.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.681+0000 7fa25b14b640 1 -- 172.21.15.97:0/3390750251 shutdown_connections 2024-04-23T10:26:00.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.681+0000 7fa25b14b640 1 -- 172.21.15.97:0/3390750251 wait complete. 2024-04-23T10:26:00.692 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T10:19:25.559138+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T10:24:04.570049+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T10:24:05.861631+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T10:26:00.693 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T10:26:00.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.795+0000 7f3221def640 1 -- 172.21.15.97:0/1662110656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f321c1009f0 msgr2=0x7f321c10a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:00.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.795+0000 7f3221def640 1 --2- 172.21.15.97:0/1662110656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f321c1009f0 0x7f321c10a5a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3214009ca0 tx=0x7f321402a8c0 comp rx=0 tx=0).stop 2024-04-23T10:26:00.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.795+0000 7f3221def640 1 -- 172.21.15.97:0/1662110656 shutdown_connections 2024-04-23T10:26:00.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.795+0000 7f3221def640 1 --2- 172.21.15.97:0/1662110656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f321c1009f0 0x7f321c10a5a0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:00.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.795+0000 7f3221def640 1 --2- 172.21.15.97:0/1662110656 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f321c100030 0x7f321c1004b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:00.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.795+0000 7f3221def640 1 --2- 172.21.15.97:0/1662110656 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f321c1035c0 0x7f321c1039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:00.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.795+0000 7f3221def640 1 -- 172.21.15.97:0/1662110656 >> 172.21.15.97:0/1662110656 conn(0x7f321c0fbb90 msgr2=0x7f321c0fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:00.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.795+0000 7f3221def640 1 -- 172.21.15.97:0/1662110656 shutdown_connections 2024-04-23T10:26:00.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.795+0000 7f3221def640 1 -- 172.21.15.97:0/1662110656 wait complete. 2024-04-23T10:26:00.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.796+0000 7f3221def640 1 Processor -- start 2024-04-23T10:26:00.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.796+0000 7f3221def640 1 -- start start 2024-04-23T10:26:00.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.796+0000 7f3221def640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f321c100030 0x7f321c198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:00.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.796+0000 7f3221def640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f321c1009f0 0x7f321c198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:00.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.796+0000 7f3221def640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f321c1035c0 0x7f321c19f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:00.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.796+0000 7f3221def640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f321c10db00 con 0x7f321c100030 2024-04-23T10:26:00.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.796+0000 7f3221def640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f321c10d980 con 0x7f321c1009f0 2024-04-23T10:26:00.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.796+0000 7f3221def640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f321c10dc80 con 0x7f321c1035c0 2024-04-23T10:26:00.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.796+0000 7f321b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f321c100030 0x7f321c198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:00.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.796+0000 7f321b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f321c100030 0x7f321c198020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46610/0 (socket says 172.21.15.97:46610) 2024-04-23T10:26:00.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.796+0000 7f321b7fe640 1 -- 172.21.15.97:0/3090270836 learned_addr learned my addr 172.21.15.97:0/3090270836 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:00.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.796+0000 7f321affd640 1 --2- 172.21.15.97:0/3090270836 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f321c1009f0 0x7f321c198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:00.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.796+0000 7f321b7fe640 1 -- 172.21.15.97:0/3090270836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f321c1035c0 msgr2=0x7f321c19f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:00.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.796+0000 7f321b7fe640 1 --2- 172.21.15.97:0/3090270836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f321c1035c0 0x7f321c19f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:00.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.796+0000 7f321b7fe640 1 -- 172.21.15.97:0/3090270836 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f321c1009f0 msgr2=0x7f321c198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:00.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.796+0000 7f321b7fe640 1 --2- 172.21.15.97:0/3090270836 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f321c1009f0 0x7f321c198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:00.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.796+0000 7f321b7fe640 1 -- 172.21.15.97:0/3090270836 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f321c19fc40 con 0x7f321c100030 2024-04-23T10:26:00.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.796+0000 7f321b7fe640 1 --2- 172.21.15.97:0/3090270836 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f321c100030 0x7f321c198020 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f320c00c970 tx=0x7f320c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:00.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.797+0000 7f3218ff9640 1 -- 172.21.15.97:0/3090270836 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320c007c10 con 0x7f321c100030 2024-04-23T10:26:00.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.797+0000 7f3218ff9640 1 -- 172.21.15.97:0/3090270836 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f320c007db0 con 0x7f321c100030 2024-04-23T10:26:00.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.797+0000 7f3221def640 1 -- 172.21.15.97:0/3090270836 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f321c19ff30 con 0x7f321c100030 2024-04-23T10:26:00.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.797+0000 7f3218ff9640 1 -- 172.21.15.97:0/3090270836 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320c00f870 con 0x7f321c100030 2024-04-23T10:26:00.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.797+0000 7f3221def640 1 -- 172.21.15.97:0/3090270836 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f321c1a03a0 con 0x7f321c100030 2024-04-23T10:26:00.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.798+0000 7f3221def640 1 -- 172.21.15.97:0/3090270836 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f321c109420 con 0x7f321c100030 2024-04-23T10:26:00.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.799+0000 7f3218ff9640 1 -- 172.21.15.97:0/3090270836 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f320c00fa10 con 0x7f321c100030 2024-04-23T10:26:00.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.799+0000 7f3218ff9640 1 --2- 172.21.15.97:0/3090270836 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f32040a3bb0 0x7f32040a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:00.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.799+0000 7f321affd640 1 --2- 172.21.15.97:0/3090270836 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f32040a3bb0 0x7f32040a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:00.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.800+0000 7f3218ff9640 1 -- 172.21.15.97:0/3090270836 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f320c0d0810 con 0x7f321c100030 2024-04-23T10:26:00.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.800+0000 7f321affd640 1 --2- 172.21.15.97:0/3090270836 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f32040a3bb0 0x7f32040a6070 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3210005e10 tx=0x7f3210005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:00.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.804+0000 7f3218ff9640 1 -- 172.21.15.97:0/3090270836 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f320c09d2b0 con 0x7f321c100030 2024-04-23T10:26:00.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.985+0000 7f3221def640 1 -- 172.21.15.97:0/3090270836 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f321c062c50 con 0x7f321c100030 2024-04-23T10:26:00.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.985+0000 7f3218ff9640 1 -- 172.21.15.97:0/3090270836 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f320c09cbb0 con 0x7f321c100030 2024-04-23T10:26:00.984 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-23T10:26:00.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.987+0000 7f3221def640 1 -- 172.21.15.97:0/3090270836 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f32040a3bb0 msgr2=0x7f32040a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:00.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.988+0000 7f3221def640 1 --2- 172.21.15.97:0/3090270836 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f32040a3bb0 0x7f32040a6070 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3210005e10 tx=0x7f3210005da0 comp rx=0 tx=0).stop 2024-04-23T10:26:00.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.988+0000 7f3221def640 1 -- 172.21.15.97:0/3090270836 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f321c100030 msgr2=0x7f321c198020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:00.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.988+0000 7f3221def640 1 --2- 172.21.15.97:0/3090270836 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f321c100030 0x7f321c198020 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f320c00c970 tx=0x7f320c00ce30 comp rx=0 tx=0).stop 2024-04-23T10:26:00.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.988+0000 7f3221def640 1 -- 172.21.15.97:0/3090270836 shutdown_connections 2024-04-23T10:26:00.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.988+0000 7f3221def640 1 --2- 172.21.15.97:0/3090270836 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f32040a3bb0 0x7f32040a6070 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:00.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.988+0000 7f3221def640 1 --2- 172.21.15.97:0/3090270836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f321c1035c0 0x7f321c19f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:00.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.988+0000 7f3221def640 1 --2- 172.21.15.97:0/3090270836 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f321c1009f0 0x7f321c198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:00.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.988+0000 7f3221def640 1 --2- 172.21.15.97:0/3090270836 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f321c100030 0x7f321c198020 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:00.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.988+0000 7f3221def640 1 -- 172.21.15.97:0/3090270836 >> 172.21.15.97:0/3090270836 conn(0x7f321c0fbb90 msgr2=0x7f321c1065f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:00.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.988+0000 7f3221def640 1 -- 172.21.15.97:0/3090270836 shutdown_connections 2024-04-23T10:26:00.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:00.988+0000 7f3221def640 1 -- 172.21.15.97:0/3090270836 wait complete. 2024-04-23T10:26:00.997 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:00 smithi097 ceph-mon[29197]: pgmap v348: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-23T10:26:00.997 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:00 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/3367841646' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T10:26:00.997 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:00 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/1236425876' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T10:26:00.997 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:00 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/1463553237' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:26:00.997 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:00 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/3390750251' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T10:26:00.998 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T10:26:01.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:00 smithi196 ceph-mon[32919]: pgmap v348: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-23T10:26:01.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:00 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/3367841646' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T10:26:01.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:00 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/1236425876' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T10:26:01.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:00 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/1463553237' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:26:01.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:00 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/3390750251' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T10:26:01.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.100+0000 7f08944bb640 1 -- 172.21.15.97:0/1749194454 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f088c102040 msgr2=0x7f088c1024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:01.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.100+0000 7f08944bb640 1 --2- 172.21.15.97:0/1749194454 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f088c102040 0x7f088c1024c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f0880009900 tx=0x7f088002a410 comp rx=0 tx=0).stop 2024-04-23T10:26:01.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.100+0000 7f08944bb640 1 -- 172.21.15.97:0/1749194454 shutdown_connections 2024-04-23T10:26:01.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.100+0000 7f08944bb640 1 --2- 172.21.15.97:0/1749194454 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f088c102a00 0x7f088c109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:01.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.100+0000 7f08944bb640 1 --2- 172.21.15.97:0/1749194454 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f088c102040 0x7f088c1024c0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:01.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.100+0000 7f08944bb640 1 --2- 172.21.15.97:0/1749194454 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f088c100e60 0x7f088c101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:01.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.100+0000 7f08944bb640 1 -- 172.21.15.97:0/1749194454 >> 172.21.15.97:0/1749194454 conn(0x7f088c0fc440 msgr2=0x7f088c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:01.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.100+0000 7f08944bb640 1 -- 172.21.15.97:0/1749194454 shutdown_connections 2024-04-23T10:26:01.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.100+0000 7f08944bb640 1 -- 172.21.15.97:0/1749194454 wait complete. 2024-04-23T10:26:01.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.100+0000 7f08944bb640 1 Processor -- start 2024-04-23T10:26:01.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.100+0000 7f08944bb640 1 -- start start 2024-04-23T10:26:01.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.100+0000 7f08944bb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f088c100e60 0x7f088c06b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:01.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.100+0000 7f08944bb640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f088c102040 0x7f088c19ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:01.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.100+0000 7f08944bb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f088c102a00 0x7f088c19f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:01.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.100+0000 7f08944bb640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f088c10ca10 con 0x7f088c102a00 2024-04-23T10:26:01.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.100+0000 7f08944bb640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f088c10c890 con 0x7f088c102040 2024-04-23T10:26:01.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.100+0000 7f08944bb640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f088c10cb90 con 0x7f088c100e60 2024-04-23T10:26:01.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.101+0000 7f0892a58640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f088c102a00 0x7f088c19f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:01.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.101+0000 7f0892a58640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f088c102a00 0x7f088c19f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46628/0 (socket says 172.21.15.97:46628) 2024-04-23T10:26:01.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.101+0000 7f0892a58640 1 -- 172.21.15.97:0/3141346164 learned_addr learned my addr 172.21.15.97:0/3141346164 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:01.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.101+0000 7f0891a56640 1 --2- 172.21.15.97:0/3141346164 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f088c102040 0x7f088c19ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:01.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.101+0000 7f0892257640 1 --2- 172.21.15.97:0/3141346164 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f088c100e60 0x7f088c06b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:01.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.101+0000 7f0892a58640 1 -- 172.21.15.97:0/3141346164 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f088c100e60 msgr2=0x7f088c06b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:01.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.101+0000 7f0892a58640 1 --2- 172.21.15.97:0/3141346164 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f088c100e60 0x7f088c06b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:01.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.101+0000 7f0892a58640 1 -- 172.21.15.97:0/3141346164 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f088c102040 msgr2=0x7f088c19ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:01.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.101+0000 7f0892a58640 1 --2- 172.21.15.97:0/3141346164 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f088c102040 0x7f088c19ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:01.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.101+0000 7f0892a58640 1 -- 172.21.15.97:0/3141346164 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f088c19fe00 con 0x7f088c102a00 2024-04-23T10:26:01.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.101+0000 7f0892a58640 1 --2- 172.21.15.97:0/3141346164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f088c102a00 0x7f088c19f4f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f087c00e970 tx=0x7f087c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:01.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.101+0000 7f088b7fe640 1 -- 172.21.15.97:0/3141346164 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f087c00ccc0 con 0x7f088c102a00 2024-04-23T10:26:01.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.101+0000 7f08944bb640 1 -- 172.21.15.97:0/3141346164 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f088c1a0090 con 0x7f088c102a00 2024-04-23T10:26:01.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.101+0000 7f088b7fe640 1 -- 172.21.15.97:0/3141346164 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f087c00ce60 con 0x7f088c102a00 2024-04-23T10:26:01.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.101+0000 7f088b7fe640 1 -- 172.21.15.97:0/3141346164 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f087c010430 con 0x7f088c102a00 2024-04-23T10:26:01.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.101+0000 7f08944bb640 1 -- 172.21.15.97:0/3141346164 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f088c1a0560 con 0x7f088c102a00 2024-04-23T10:26:01.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.102+0000 7f08944bb640 1 -- 172.21.15.97:0/3141346164 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f088c108200 con 0x7f088c102a00 2024-04-23T10:26:01.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.103+0000 7f088b7fe640 1 -- 172.21.15.97:0/3141346164 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f087c004fb0 con 0x7f088c102a00 2024-04-23T10:26:01.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.104+0000 7f088b7fe640 1 --2- 172.21.15.97:0/3141346164 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f08740a3bb0 0x7f08740a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:01.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.104+0000 7f0892257640 1 --2- 172.21.15.97:0/3141346164 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f08740a3bb0 0x7f08740a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:01.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.104+0000 7f088b7fe640 1 -- 172.21.15.97:0/3141346164 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f087c0d0ad0 con 0x7f088c102a00 2024-04-23T10:26:01.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.104+0000 7f0892257640 1 --2- 172.21.15.97:0/3141346164 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f08740a3bb0 0x7f08740a6070 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f088c101bb0 tx=0x7f0884009870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:01.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.107+0000 7f088b7fe640 1 -- 172.21.15.97:0/3141346164 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f087c053ff0 con 0x7f088c102a00 2024-04-23T10:26:01.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:00 smithi183 ceph-mon[32317]: pgmap v348: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-23T10:26:01.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:00 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/3367841646' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T10:26:01.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:00 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/1236425876' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T10:26:01.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:00 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/1463553237' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:26:01.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:00 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/3390750251' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T10:26:01.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.286+0000 7f08944bb640 1 -- 172.21.15.97:0/3141346164 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f088c02d750 con 0x7f088c102a00 2024-04-23T10:26:01.286 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.287+0000 7f088b7fe640 1 -- 172.21.15.97:0/3141346164 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f087c09d480 con 0x7f088c102a00 2024-04-23T10:26:01.286 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-23T10:26:01.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.289+0000 7f08944bb640 1 -- 172.21.15.97:0/3141346164 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f08740a3bb0 msgr2=0x7f08740a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:01.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.289+0000 7f08944bb640 1 --2- 172.21.15.97:0/3141346164 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f08740a3bb0 0x7f08740a6070 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f088c101bb0 tx=0x7f0884009870 comp rx=0 tx=0).stop 2024-04-23T10:26:01.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.289+0000 7f08944bb640 1 -- 172.21.15.97:0/3141346164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f088c102a00 msgr2=0x7f088c19f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:01.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.289+0000 7f08944bb640 1 --2- 172.21.15.97:0/3141346164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f088c102a00 0x7f088c19f4f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f087c00e970 tx=0x7f087c00ed70 comp rx=0 tx=0).stop 2024-04-23T10:26:01.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.289+0000 7f08944bb640 1 -- 172.21.15.97:0/3141346164 shutdown_connections 2024-04-23T10:26:01.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.289+0000 7f08944bb640 1 --2- 172.21.15.97:0/3141346164 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f08740a3bb0 0x7f08740a6070 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:01.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.289+0000 7f08944bb640 1 --2- 172.21.15.97:0/3141346164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f088c102a00 0x7f088c19f4f0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:01.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.289+0000 7f08944bb640 1 --2- 172.21.15.97:0/3141346164 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f088c102040 0x7f088c19ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:01.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.289+0000 7f08944bb640 1 --2- 172.21.15.97:0/3141346164 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f088c100e60 0x7f088c06b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:01.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.289+0000 7f08944bb640 1 -- 172.21.15.97:0/3141346164 >> 172.21.15.97:0/3141346164 conn(0x7f088c0fc440 msgr2=0x7f088c1053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:01.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.289+0000 7f08944bb640 1 -- 172.21.15.97:0/3141346164 shutdown_connections 2024-04-23T10:26:01.289 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.289+0000 7f08944bb640 1 -- 172.21.15.97:0/3141346164 wait complete. 2024-04-23T10:26:01.299 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T10:26:01.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.401+0000 7fbf9974b640 1 -- 172.21.15.97:0/1927198391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf94102a00 msgr2=0x7fbf94109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:01.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.401+0000 7fbf9974b640 1 --2- 172.21.15.97:0/1927198391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf94102a00 0x7fbf94109320 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c009ca0 tx=0x7fbf8c02a8c0 comp rx=0 tx=0).stop 2024-04-23T10:26:01.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.401+0000 7fbf9974b640 1 -- 172.21.15.97:0/1927198391 shutdown_connections 2024-04-23T10:26:01.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.401+0000 7fbf9974b640 1 --2- 172.21.15.97:0/1927198391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf94102a00 0x7fbf94109320 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:01.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.401+0000 7fbf9974b640 1 --2- 172.21.15.97:0/1927198391 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf94102040 0x7fbf941024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:01.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.401+0000 7fbf9974b640 1 --2- 172.21.15.97:0/1927198391 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf94100e60 0x7fbf94101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:01.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.401+0000 7fbf9974b640 1 -- 172.21.15.97:0/1927198391 >> 172.21.15.97:0/1927198391 conn(0x7fbf940fc420 msgr2=0x7fbf940fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:01.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.401+0000 7fbf9974b640 1 -- 172.21.15.97:0/1927198391 shutdown_connections 2024-04-23T10:26:01.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.401+0000 7fbf9974b640 1 -- 172.21.15.97:0/1927198391 wait complete. 2024-04-23T10:26:01.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.402+0000 7fbf9974b640 1 Processor -- start 2024-04-23T10:26:01.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.402+0000 7fbf9974b640 1 -- start start 2024-04-23T10:26:01.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.402+0000 7fbf9974b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf94100e60 0x7fbf941981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:01.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.402+0000 7fbf9974b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf94102040 0x7fbf941986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:01.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.402+0000 7fbf9974b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf94102a00 0x7fbf9419f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:01.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.402+0000 7fbf9974b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9410ca00 con 0x7fbf94102040 2024-04-23T10:26:01.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.402+0000 7fbf9974b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9410c880 con 0x7fbf94100e60 2024-04-23T10:26:01.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.402+0000 7fbf9974b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9410cb80 con 0x7fbf94102a00 2024-04-23T10:26:01.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.402+0000 7fbf8bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf94102040 0x7fbf941986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:01.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.402+0000 7fbf8bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf94102040 0x7fbf941986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46640/0 (socket says 172.21.15.97:46640) 2024-04-23T10:26:01.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.402+0000 7fbf8bfff640 1 -- 172.21.15.97:0/2986237404 learned_addr learned my addr 172.21.15.97:0/2986237404 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:01.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.402+0000 7fbf92ffd640 1 --2- 172.21.15.97:0/2986237404 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf94100e60 0x7fbf941981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:01.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.402+0000 7fbf937fe640 1 --2- 172.21.15.97:0/2986237404 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf94102a00 0x7fbf9419f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:01.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.402+0000 7fbf8bfff640 1 -- 172.21.15.97:0/2986237404 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf94102a00 msgr2=0x7fbf9419f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:01.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.402+0000 7fbf8bfff640 1 --2- 172.21.15.97:0/2986237404 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf94102a00 0x7fbf9419f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:01.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.402+0000 7fbf8bfff640 1 -- 172.21.15.97:0/2986237404 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf94100e60 msgr2=0x7fbf941981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:01.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.402+0000 7fbf8bfff640 1 --2- 172.21.15.97:0/2986237404 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf94100e60 0x7fbf941981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:01.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.402+0000 7fbf8bfff640 1 -- 172.21.15.97:0/2986237404 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf9419fdc0 con 0x7fbf94102040 2024-04-23T10:26:01.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.403+0000 7fbf8bfff640 1 --2- 172.21.15.97:0/2986237404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf94102040 0x7fbf941986e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c00d770 tx=0x7fbf7c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:01.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.403+0000 7fbf90ff9640 1 -- 172.21.15.97:0/2986237404 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c017070 con 0x7fbf94102040 2024-04-23T10:26:01.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.403+0000 7fbf90ff9640 1 -- 172.21.15.97:0/2986237404 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf7c0054d0 con 0x7fbf94102040 2024-04-23T10:26:01.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.403+0000 7fbf9974b640 1 -- 172.21.15.97:0/2986237404 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf941a00b0 con 0x7fbf94102040 2024-04-23T10:26:01.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.403+0000 7fbf90ff9640 1 -- 172.21.15.97:0/2986237404 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c004c30 con 0x7fbf94102040 2024-04-23T10:26:01.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.403+0000 7fbf9974b640 1 -- 172.21.15.97:0/2986237404 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf941a05b0 con 0x7fbf94102040 2024-04-23T10:26:01.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.404+0000 7fbf9974b640 1 -- 172.21.15.97:0/2986237404 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf941a0810 con 0x7fbf94102040 2024-04-23T10:26:01.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.405+0000 7fbf90ff9640 1 -- 172.21.15.97:0/2986237404 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fbf7c004e10 con 0x7fbf94102040 2024-04-23T10:26:01.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.405+0000 7fbf90ff9640 1 --2- 172.21.15.97:0/2986237404 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fbf800a3bb0 0x7fbf800a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:01.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.405+0000 7fbf92ffd640 1 --2- 172.21.15.97:0/2986237404 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fbf800a3bb0 0x7fbf800a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:01.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.405+0000 7fbf90ff9640 1 -- 172.21.15.97:0/2986237404 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fbf7c0cfde0 con 0x7fbf94102040 2024-04-23T10:26:01.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.406+0000 7fbf92ffd640 1 --2- 172.21.15.97:0/2986237404 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fbf800a3bb0 0x7fbf800a6070 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fbf94101bb0 tx=0x7fbf8400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:01.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.410+0000 7fbf90ff9640 1 -- 172.21.15.97:0/2986237404 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbf7c09c1a0 con 0x7fbf94102040 2024-04-23T10:26:01.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.589+0000 7fbf9974b640 1 -- 172.21.15.97:0/2986237404 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbf941a09e0 con 0x7fbf94102040 2024-04-23T10:26:01.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.590+0000 7fbf90ff9640 1 -- 172.21.15.97:0/2986237404 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7fbf7c0a04a0 con 0x7fbf94102040 2024-04-23T10:26:01.589 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-04-23T10:26:01.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.592+0000 7fbf9974b640 1 -- 172.21.15.97:0/2986237404 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fbf800a3bb0 msgr2=0x7fbf800a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:01.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.592+0000 7fbf9974b640 1 --2- 172.21.15.97:0/2986237404 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fbf800a3bb0 0x7fbf800a6070 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fbf94101bb0 tx=0x7fbf8400b390 comp rx=0 tx=0).stop 2024-04-23T10:26:01.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.592+0000 7fbf9974b640 1 -- 172.21.15.97:0/2986237404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf94102040 msgr2=0x7fbf941986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:01.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.592+0000 7fbf9974b640 1 --2- 172.21.15.97:0/2986237404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf94102040 0x7fbf941986e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c00d770 tx=0x7fbf7c00dc30 comp rx=0 tx=0).stop 2024-04-23T10:26:01.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.592+0000 7fbf9974b640 1 -- 172.21.15.97:0/2986237404 shutdown_connections 2024-04-23T10:26:01.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.592+0000 7fbf9974b640 1 --2- 172.21.15.97:0/2986237404 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fbf800a3bb0 0x7fbf800a6070 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:01.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.592+0000 7fbf9974b640 1 --2- 172.21.15.97:0/2986237404 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf94102a00 0x7fbf9419f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:01.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.592+0000 7fbf9974b640 1 --2- 172.21.15.97:0/2986237404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf94102040 0x7fbf941986e0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:01.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.592+0000 7fbf9974b640 1 --2- 172.21.15.97:0/2986237404 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf94100e60 0x7fbf941981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:01.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.592+0000 7fbf9974b640 1 -- 172.21.15.97:0/2986237404 >> 172.21.15.97:0/2986237404 conn(0x7fbf940fc420 msgr2=0x7fbf940fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:01.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.592+0000 7fbf9974b640 1 -- 172.21.15.97:0/2986237404 shutdown_connections 2024-04-23T10:26:01.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.592+0000 7fbf9974b640 1 -- 172.21.15.97:0/2986237404 wait complete. 2024-04-23T10:26:01.602 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T10:26:01.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.701+0000 7f17c8bed640 1 -- 172.21.15.97:0/1036592211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17c4101fa0 msgr2=0x7f17c41023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:01.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.701+0000 7f17c8bed640 1 --2- 172.21.15.97:0/1036592211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17c4101fa0 0x7f17c41023c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f17bc009ca0 tx=0x7f17bc02a8c0 comp rx=0 tx=0).stop 2024-04-23T10:26:01.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.701+0000 7f17c8bed640 1 -- 172.21.15.97:0/1036592211 shutdown_connections 2024-04-23T10:26:01.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.701+0000 7f17c8bed640 1 --2- 172.21.15.97:0/1036592211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17c4103b50 0x7f17c410a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:01.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.701+0000 7f17c8bed640 1 --2- 172.21.15.97:0/1036592211 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17c4103190 0x7f17c4103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:01.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.701+0000 7f17c8bed640 1 --2- 172.21.15.97:0/1036592211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17c4101fa0 0x7f17c41023c0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:01.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.701+0000 7f17c8bed640 1 -- 172.21.15.97:0/1036592211 >> 172.21.15.97:0/1036592211 conn(0x7f17c40fd550 msgr2=0x7f17c40ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:01.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.701+0000 7f17c8bed640 1 -- 172.21.15.97:0/1036592211 shutdown_connections 2024-04-23T10:26:01.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.701+0000 7f17c8bed640 1 -- 172.21.15.97:0/1036592211 wait complete. 2024-04-23T10:26:01.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.702+0000 7f17c8bed640 1 Processor -- start 2024-04-23T10:26:01.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.702+0000 7f17c8bed640 1 -- start start 2024-04-23T10:26:01.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.702+0000 7f17c8bed640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17c4101fa0 0x7f17c4075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:01.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.702+0000 7f17c8bed640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17c4103190 0x7f17c4074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:01.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.702+0000 7f17c8bed640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17c4103b50 0x7f17c4074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:01.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.702+0000 7f17c8bed640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f17c410dae0 con 0x7f17c4103190 2024-04-23T10:26:01.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.702+0000 7f17c8bed640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f17c410d960 con 0x7f17c4101fa0 2024-04-23T10:26:01.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.702+0000 7f17c8bed640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f17c410dc60 con 0x7f17c4103b50 2024-04-23T10:26:01.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.702+0000 7f17c1d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17c4103190 0x7f17c4074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:01.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.702+0000 7f17c1d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17c4103190 0x7f17c4074340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46658/0 (socket says 172.21.15.97:46658) 2024-04-23T10:26:01.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.702+0000 7f17c1d9b640 1 -- 172.21.15.97:0/2241725834 learned_addr learned my addr 172.21.15.97:0/2241725834 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:01.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.702+0000 7f17c259c640 1 --2- 172.21.15.97:0/2241725834 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17c4101fa0 0x7f17c4075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:01.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.702+0000 7f17c2d9d640 1 --2- 172.21.15.97:0/2241725834 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17c4103b50 0x7f17c4074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:01.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.702+0000 7f17c1d9b640 1 -- 172.21.15.97:0/2241725834 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17c4103b50 msgr2=0x7f17c4074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:01.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.702+0000 7f17c1d9b640 1 --2- 172.21.15.97:0/2241725834 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17c4103b50 0x7f17c4074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:01.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.702+0000 7f17c1d9b640 1 -- 172.21.15.97:0/2241725834 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17c4101fa0 msgr2=0x7f17c4075d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:01.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.702+0000 7f17c1d9b640 1 --2- 172.21.15.97:0/2241725834 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17c4101fa0 0x7f17c4075d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:01.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.702+0000 7f17c1d9b640 1 -- 172.21.15.97:0/2241725834 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17c4075110 con 0x7f17c4103190 2024-04-23T10:26:01.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.702+0000 7f17c1d9b640 1 --2- 172.21.15.97:0/2241725834 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17c4103190 0x7f17c4074340 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f17ac00c920 tx=0x7f17ac00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:01.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.702+0000 7f17bb7fe640 1 -- 172.21.15.97:0/2241725834 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ac007c70 con 0x7f17c4103190 2024-04-23T10:26:01.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.702+0000 7f17c8bed640 1 -- 172.21.15.97:0/2241725834 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17c41a22b0 con 0x7f17c4103190 2024-04-23T10:26:01.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.702+0000 7f17bb7fe640 1 -- 172.21.15.97:0/2241725834 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17ac007e10 con 0x7f17c4103190 2024-04-23T10:26:01.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.703+0000 7f17bb7fe640 1 -- 172.21.15.97:0/2241725834 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ac018820 con 0x7f17c4103190 2024-04-23T10:26:01.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.703+0000 7f17c8bed640 1 -- 172.21.15.97:0/2241725834 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17c41a2800 con 0x7f17c4103190 2024-04-23T10:26:01.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.704+0000 7f17c8bed640 1 -- 172.21.15.97:0/2241725834 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17c4109340 con 0x7f17c4103190 2024-04-23T10:26:01.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.705+0000 7f17bb7fe640 1 -- 172.21.15.97:0/2241725834 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f17ac00f450 con 0x7f17c4103190 2024-04-23T10:26:01.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.706+0000 7f17bb7fe640 1 --2- 172.21.15.97:0/2241725834 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f17b00a3bb0 0x7f17b00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:01.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.706+0000 7f17c259c640 1 --2- 172.21.15.97:0/2241725834 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f17b00a3bb0 0x7f17b00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:01.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.706+0000 7f17bb7fe640 1 -- 172.21.15.97:0/2241725834 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f17ac0d0710 con 0x7f17c4103190 2024-04-23T10:26:01.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.706+0000 7f17c259c640 1 --2- 172.21.15.97:0/2241725834 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f17b00a3bb0 0x7f17b00a6070 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f17bc000f80 tx=0x7f17bc005c30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:01.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.709+0000 7f17bb7fe640 1 -- 172.21.15.97:0/2241725834 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f17ac014030 con 0x7f17c4103190 2024-04-23T10:26:01.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.927+0000 7f17c8bed640 1 -- 172.21.15.97:0/2241725834 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f17c402d810 con 0x7f17c4103190 2024-04-23T10:26:01.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.927+0000 7f17bb7fe640 1 -- 172.21.15.97:0/2241725834 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11931 (secure 0 0 0) 0x7f17ac09d1b0 con 0x7f17c4103190 2024-04-23T10:26:01.927 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:26:01.927 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3946878762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3946878762},{"type":"v1","addr":"172.21.15.97:6835","nonce":3946878762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/209800861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":209800861},{"type":"v1","addr":"172.21.15.97:6837","nonce":209800861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T10:24:06.910134+0000","modified":"2024-04-23T10:24:12.969478+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6839/418013575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":418013575},{"type":"v1","addr":"172.21.15.97:6839","nonce":418013575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6841/747242680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":747242680},{"type":"v1","addr":"172.21.15.97:6841","nonce":747242680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6833/4292847121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4292847121},{"type":"v1","addr":"172.21.15.183:6833","nonce":4292847121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6833/3885816565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3885816565},{"type":"v1","addr":"172.21.15.196:6833","nonce":3885816565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1897093105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1897093105},{"type":"v1","addr":"172.21.15.183:6835","nonce":1897093105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1171477563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1171477563},{"type":"v1","addr":"172.21.15.196:6835","nonce":1171477563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1898341911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1898341911},{"type":"v1","addr":"172.21.15.183:6837","nonce":1898341911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.196:6837/3094946502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3094946502},{"type":"v1","addr":"172.21.15.196:6837","nonce":3094946502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.196:6839/83843556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":83843556},{"type":"v1","addr":"172.21.15.196:6839","nonce":83843556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6839/1763841130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1763841130},{"type":"v1","addr":"172.21.15.183:6839","nonce":1763841130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24469,24488,24398,24403,24487]},"id":1}]} 2024-04-23T10:26:01.927 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 25 2024-04-23T10:26:01.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.929+0000 7f17c8bed640 1 -- 172.21.15.97:0/2241725834 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f17b00a3bb0 msgr2=0x7f17b00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:01.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.929+0000 7f17c8bed640 1 --2- 172.21.15.97:0/2241725834 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f17b00a3bb0 0x7f17b00a6070 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f17bc000f80 tx=0x7f17bc005c30 comp rx=0 tx=0).stop 2024-04-23T10:26:01.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.929+0000 7f17c8bed640 1 -- 172.21.15.97:0/2241725834 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17c4103190 msgr2=0x7f17c4074340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:01.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.929+0000 7f17c8bed640 1 --2- 172.21.15.97:0/2241725834 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17c4103190 0x7f17c4074340 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f17ac00c920 tx=0x7f17ac00cde0 comp rx=0 tx=0).stop 2024-04-23T10:26:01.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.929+0000 7f17c8bed640 1 -- 172.21.15.97:0/2241725834 shutdown_connections 2024-04-23T10:26:01.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.929+0000 7f17c8bed640 1 --2- 172.21.15.97:0/2241725834 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f17b00a3bb0 0x7f17b00a6070 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:01.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.929+0000 7f17c8bed640 1 --2- 172.21.15.97:0/2241725834 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17c4103b50 0x7f17c4074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:01.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.929+0000 7f17c8bed640 1 --2- 172.21.15.97:0/2241725834 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17c4103190 0x7f17c4074340 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:01.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.929+0000 7f17c8bed640 1 --2- 172.21.15.97:0/2241725834 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17c4101fa0 0x7f17c4075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:01.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.929+0000 7f17c8bed640 1 -- 172.21.15.97:0/2241725834 >> 172.21.15.97:0/2241725834 conn(0x7f17c40fd550 msgr2=0x7f17c4106510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:01.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.929+0000 7f17c8bed640 1 -- 172.21.15.97:0/2241725834 shutdown_connections 2024-04-23T10:26:01.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:01.929+0000 7f17c8bed640 1 -- 172.21.15.97:0/2241725834 wait complete. 2024-04-23T10:26:01.940 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T10:26:02.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.043+0000 7fe0c3720640 1 -- 172.21.15.97:0/1720738475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0bc101fa0 msgr2=0x7fe0bc1023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:02.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.043+0000 7fe0c3720640 1 --2- 172.21.15.97:0/1720738475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0bc101fa0 0x7fe0bc1023c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8009900 tx=0x7fe0b802a410 comp rx=0 tx=0).stop 2024-04-23T10:26:02.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.043+0000 7fe0c3720640 1 -- 172.21.15.97:0/1720738475 shutdown_connections 2024-04-23T10:26:02.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.043+0000 7fe0c3720640 1 --2- 172.21.15.97:0/1720738475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0bc103b50 0x7fe0bc10a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.043+0000 7fe0c3720640 1 --2- 172.21.15.97:0/1720738475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0bc103190 0x7fe0bc103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.043+0000 7fe0c3720640 1 --2- 172.21.15.97:0/1720738475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0bc101fa0 0x7fe0bc1023c0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.043+0000 7fe0c3720640 1 -- 172.21.15.97:0/1720738475 >> 172.21.15.97:0/1720738475 conn(0x7fe0bc0fd550 msgr2=0x7fe0bc0ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:02.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.043+0000 7fe0c3720640 1 -- 172.21.15.97:0/1720738475 shutdown_connections 2024-04-23T10:26:02.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.043+0000 7fe0c3720640 1 -- 172.21.15.97:0/1720738475 wait complete. 2024-04-23T10:26:02.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.044+0000 7fe0c3720640 1 Processor -- start 2024-04-23T10:26:02.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.044+0000 7fe0c3720640 1 -- start start 2024-04-23T10:26:02.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.044+0000 7fe0c3720640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0bc101fa0 0x7fe0bc198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.044+0000 7fe0c3720640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0bc103190 0x7fe0bc198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.044+0000 7fe0c3720640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0bc103b50 0x7fe0bc19f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.044+0000 7fe0c3720640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe0bc10dae0 con 0x7fe0bc103b50 2024-04-23T10:26:02.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.044+0000 7fe0c3720640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe0bc10d960 con 0x7fe0bc103190 2024-04-23T10:26:02.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.044+0000 7fe0c3720640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe0bc10dc60 con 0x7fe0bc101fa0 2024-04-23T10:26:02.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.044+0000 7fe0c1cbd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0bc103b50 0x7fe0bc19f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.044+0000 7fe0c1cbd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0bc103b50 0x7fe0bc19f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46684/0 (socket says 172.21.15.97:46684) 2024-04-23T10:26:02.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.044+0000 7fe0c1cbd640 1 -- 172.21.15.97:0/3278193651 learned_addr learned my addr 172.21.15.97:0/3278193651 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:02.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.044+0000 7fe0c0cbb640 1 --2- 172.21.15.97:0/3278193651 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0bc103190 0x7fe0bc198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.044+0000 7fe0c14bc640 1 --2- 172.21.15.97:0/3278193651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0bc101fa0 0x7fe0bc198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.044+0000 7fe0c1cbd640 1 -- 172.21.15.97:0/3278193651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0bc101fa0 msgr2=0x7fe0bc198220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:02.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.044+0000 7fe0c1cbd640 1 --2- 172.21.15.97:0/3278193651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0bc101fa0 0x7fe0bc198220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.044+0000 7fe0c1cbd640 1 -- 172.21.15.97:0/3278193651 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0bc103190 msgr2=0x7fe0bc198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:02.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.044+0000 7fe0c1cbd640 1 --2- 172.21.15.97:0/3278193651 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0bc103190 0x7fe0bc198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.044+0000 7fe0c1cbd640 1 -- 172.21.15.97:0/3278193651 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0bc19fe40 con 0x7fe0bc103b50 2024-04-23T10:26:02.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.044+0000 7fe0c14bc640 1 --2- 172.21.15.97:0/3278193651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0bc101fa0 0x7fe0bc198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:26:02.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.044+0000 7fe0c1cbd640 1 --2- 172.21.15.97:0/3278193651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0bc103b50 0x7fe0bc19f7d0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe0b400cad0 tx=0x7fe0b400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:02.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.044+0000 7fe0b27fc640 1 -- 172.21.15.97:0/3278193651 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b400ae20 con 0x7fe0bc103b50 2024-04-23T10:26:02.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.045+0000 7fe0b27fc640 1 -- 172.21.15.97:0/3278193651 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0b4005020 con 0x7fe0bc103b50 2024-04-23T10:26:02.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.045+0000 7fe0b27fc640 1 -- 172.21.15.97:0/3278193651 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b400f6b0 con 0x7fe0bc103b50 2024-04-23T10:26:02.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.045+0000 7fe0c3720640 1 -- 172.21.15.97:0/3278193651 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0bc1a0130 con 0x7fe0bc103b50 2024-04-23T10:26:02.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.045+0000 7fe0c3720640 1 -- 172.21.15.97:0/3278193651 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0bc1a06d0 con 0x7fe0bc103b50 2024-04-23T10:26:02.047 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.045+0000 7fe0c3720640 1 -- 172.21.15.97:0/3278193651 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0bc109430 con 0x7fe0bc103b50 2024-04-23T10:26:02.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.046+0000 7fe0b27fc640 1 -- 172.21.15.97:0/3278193651 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe0b40051d0 con 0x7fe0bc103b50 2024-04-23T10:26:02.048 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.047+0000 7fe0b27fc640 1 --2- 172.21.15.97:0/3278193651 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fe0a80a3c30 0x7fe0a80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.047+0000 7fe0c14bc640 1 --2- 172.21.15.97:0/3278193651 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fe0a80a3c30 0x7fe0a80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.047+0000 7fe0b27fc640 1 -- 172.21.15.97:0/3278193651 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fe0b40d0430 con 0x7fe0bc103b50 2024-04-23T10:26:02.049 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.047+0000 7fe0c14bc640 1 --2- 172.21.15.97:0/3278193651 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fe0a80a3c30 0x7fe0a80a60f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8006010 tx=0x7fe0b8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:02.050 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.051+0000 7fe0b27fc640 1 -- 172.21.15.97:0/3278193651 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe0b4014030 con 0x7fe0bc103b50 2024-04-23T10:26:02.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.232+0000 7fe0c3720640 1 -- 172.21.15.97:0/3278193651 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe0bc062b50 con 0x7fe0bc103b50 2024-04-23T10:26:02.233 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.233+0000 7fe0b27fc640 1 -- 172.21.15.97:0/3278193651 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22606 (secure 0 0 0) 0x7fe0b409ce40 con 0x7fe0bc103b50 2024-04-23T10:26:02.233 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:26:02.233 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":88,"fsid":"9b22af88-0159-11ef-bc93-c7b262605968","created":"2024-04-23T10:16:05.983305+0000","modified":"2024-04-23T10:25:00.660133+0000","last_up_change":"2024-04-23T10:23:13.114761+0000","last_in_change":"2024-04-23T10:22:52.804880+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T10:19:25.559138+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T10:24:04.570049+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T10:24:05.861631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"58cb995d-de35-4522-ac81-4982401b260d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6803","nonce":3031221802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6805","nonce":3031221802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6809","nonce":3031221802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6807","nonce":3031221802}]},"public_addr":"172.21.15.97:6803/3031221802","cluster_addr":"172.21.15.97:6805/3031221802","heartbeat_back_addr":"172.21.15.97:6809/3031221802","heartbeat_front_addr":"172.21.15.97:6807/3031221802","state":["exists","up"]},{"osd":1,"uuid":"b47bd8ee-1f21-4834-acf5-bb40f97ab3ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6801","nonce":1108870440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6803","nonce":1108870440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6807","nonce":1108870440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6805","nonce":1108870440}]},"public_addr":"172.21.15.183:6801/1108870440","cluster_addr":"172.21.15.183:6803/1108870440","heartbeat_back_addr":"172.21.15.183:6807/1108870440","heartbeat_front_addr":"172.21.15.183:6805/1108870440","state":["exists","up"]},{"osd":2,"uuid":"17c4eec3-6c47-4334-9b63-1d5ad9ec5988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6801","nonce":2877039731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6803","nonce":2877039731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6807","nonce":2877039731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6805","nonce":2877039731}]},"public_addr":"172.21.15.196:6801/2877039731","cluster_addr":"172.21.15.196:6803/2877039731","heartbeat_back_addr":"172.21.15.196:6807/2877039731","heartbeat_front_addr":"172.21.15.196:6805/2877039731","state":["exists","up"]},{"osd":3,"uuid":"3f731c38-2e45-4ca1-8dce-efc4a1b2166b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6811","nonce":4276311194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6813","nonce":4276311194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6817","nonce":4276311194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6815","nonce":4276311194}]},"public_addr":"172.21.15.97:6811/4276311194","cluster_addr":"172.21.15.97:6813/4276311194","heartbeat_back_addr":"172.21.15.97:6817/4276311194","heartbeat_front_addr":"172.21.15.97:6815/4276311194","state":["exists","up"]},{"osd":4,"uuid":"fdcafb14-9b82-4134-bd5b-79c0516f6cbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6809","nonce":63913916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6811","nonce":63913916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6815","nonce":63913916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6813","nonce":63913916}]},"public_addr":"172.21.15.183:6809/63913916","cluster_addr":"172.21.15.183:6811/63913916","heartbeat_back_addr":"172.21.15.183:6815/63913916","heartbeat_front_addr":"172.21.15.183:6813/63913916","state":["exists","up"]},{"osd":5,"uuid":"cdcc18d8-da3c-4012-bf12-876bb39e66eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6809","nonce":2804446896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6811","nonce":2804446896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6815","nonce":2804446896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6813","nonce":2804446896}]},"public_addr":"172.21.15.196:6809/2804446896","cluster_addr":"172.21.15.196:6811/2804446896","heartbeat_back_addr":"172.21.15.196:6815/2804446896","heartbeat_front_addr":"172.21.15.196:6813/2804446896","state":["exists","up"]},{"osd":6,"uuid":"071abb5c-2edf-4832-b8aa-3b80d20096fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6819","nonce":2080916705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6821","nonce":2080916705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6825","nonce":2080916705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6823","nonce":2080916705}]},"public_addr":"172.21.15.97:6819/2080916705","cluster_addr":"172.21.15.97:6821/2080916705","heartbeat_back_addr":"172.21.15.97:6825/2080916705","heartbeat_front_addr":"172.21.15.97:6823/2080916705","state":["exists","up"]},{"osd":7,"uuid":"9d54760b-66df-486b-8abb-017f3b64b4d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6817","nonce":1865873033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6819","nonce":1865873033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6823","nonce":1865873033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6821","nonce":1865873033}]},"public_addr":"172.21.15.183:6817/1865873033","cluster_addr":"172.21.15.183:6819/1865873033","heartbeat_back_addr":"172.21.15.183:6823/1865873033","heartbeat_front_addr":"172.21.15.183:6821/1865873033","state":["exists","up"]},{"osd":8,"uuid":"5cd3100a-bd6a-4be5-b5e9-0d0afc325193","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6817","nonce":4063932139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6819","nonce":4063932139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6823","nonce":4063932139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6821","nonce":4063932139}]},"public_addr":"172.21.15.196:6817/4063932139","cluster_addr":"172.21.15.196:6819/4063932139","heartbeat_back_addr":"172.21.15.196:6823/4063932139","heartbeat_front_addr":"172.21.15.196:6821/4063932139","state":["exists","up"]},{"osd":9,"uuid":"b6a5f4a1-4ecc-4f73-a2df-07551d9298ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6827","nonce":1887432474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6829","nonce":1887432474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6833","nonce":1887432474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6831","nonce":1887432474}]},"public_addr":"172.21.15.97:6827/1887432474","cluster_addr":"172.21.15.97:6829/1887432474","heartbeat_back_addr":"172.21.15.97:6833/1887432474","heartbeat_front_addr":"172.21.15.97:6831/1887432474","state":["exists","up"]},{"osd":10,"uuid":"86f59816-1ea6-4ae9-8796-cf5fc4831064","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6825","nonce":799434647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6827","nonce":799434647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6831","nonce":799434647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6829","nonce":799434647}]},"public_addr":"172.21.15.183:6825/799434647","cluster_addr":"172.21.15.183:6827/799434647","heartbeat_back_addr":"172.21.15.183:6831/799434647","heartbeat_front_addr":"172.21.15.183:6829/799434647","state":["exists","up"]},{"osd":11,"uuid":"95d64a89-a9f8-4c7a-8c0d-da7fb26cfff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6825","nonce":1873969467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6827","nonce":1873969467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6831","nonce":1873969467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6829","nonce":1873969467}]},"public_addr":"172.21.15.196:6825/1873969467","cluster_addr":"172.21.15.196:6827/1873969467","heartbeat_back_addr":"172.21.15.196:6831/1873969467","heartbeat_front_addr":"172.21.15.196:6829/1873969467","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:18:33.721156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:18:57.586709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:19:22.069458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:19:46.451597+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:20:11.561019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:20:36.298672+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:21:01.661804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:21:27.324128+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:21:52.898197+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:22:19.614326+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:22:45.154917+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:23:11.222612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/3866016040":"2024-04-24T10:16:59.468237+0000","172.21.15.97:6801/3297166858":"2024-04-24T10:16:59.468237+0000","172.21.15.97:0/3804743548":"2024-04-24T10:16:59.468237+0000","172.21.15.97:6800/3706463017":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/3715079438":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/786806834":"2024-04-24T10:16:59.468237+0000","172.21.15.97:6801/3706463017":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/3294175554":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/1449136760":"2024-04-24T10:16:33.128917+0000","172.21.15.97:6800/3297166858":"2024-04-24T10:16:59.468237+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T10:26:02.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.236+0000 7fe0c3720640 1 -- 172.21.15.97:0/3278193651 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fe0a80a3c30 msgr2=0x7fe0a80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:02.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.236+0000 7fe0c3720640 1 --2- 172.21.15.97:0/3278193651 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fe0a80a3c30 0x7fe0a80a60f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8006010 tx=0x7fe0b8000bc0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.236+0000 7fe0c3720640 1 -- 172.21.15.97:0/3278193651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0bc103b50 msgr2=0x7fe0bc19f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:02.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.236+0000 7fe0c3720640 1 --2- 172.21.15.97:0/3278193651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0bc103b50 0x7fe0bc19f7d0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe0b400cad0 tx=0x7fe0b400cf90 comp rx=0 tx=0).stop 2024-04-23T10:26:02.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.236+0000 7fe0c3720640 1 -- 172.21.15.97:0/3278193651 shutdown_connections 2024-04-23T10:26:02.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.236+0000 7fe0c3720640 1 --2- 172.21.15.97:0/3278193651 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fe0a80a3c30 0x7fe0a80a60f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.236+0000 7fe0c3720640 1 --2- 172.21.15.97:0/3278193651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe0bc103b50 0x7fe0bc19f7d0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.236+0000 7fe0c3720640 1 --2- 172.21.15.97:0/3278193651 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0bc103190 0x7fe0bc198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.236+0000 7fe0c3720640 1 --2- 172.21.15.97:0/3278193651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0bc101fa0 0x7fe0bc198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.236+0000 7fe0c3720640 1 -- 172.21.15.97:0/3278193651 >> 172.21.15.97:0/3278193651 conn(0x7fe0bc0fd550 msgr2=0x7fe0bc106600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:02.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.236+0000 7fe0c3720640 1 -- 172.21.15.97:0/3278193651 shutdown_connections 2024-04-23T10:26:02.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.236+0000 7fe0c3720640 1 -- 172.21.15.97:0/3278193651 wait complete. 2024-04-23T10:26:02.245 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:01 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/3090270836' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T10:26:02.245 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:01 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/3141346164' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T10:26:02.245 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:01 smithi097 ceph-mon[29197]: pgmap v349: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-23T10:26:02.245 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:01 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/2986237404' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T10:26:02.245 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:01 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/2241725834' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:26:02.247 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-23T10:24:06.910134+0000', 'modified': '2024-04-23T10:24:12.969478+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24488, 'mds_1': 24487, 'mds_2': 24469, 'mds_3': 24403, 'mds_4': 24398}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14640': {'gid': 14640, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.97:6839/418013575', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6838', 'nonce': 418013575}, {'type': 'v1', 'addr': '172.21.15.97:6839', 'nonce': 418013575}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14652': {'gid': 14652, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.97:6841/747242680', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6840', 'nonce': 747242680}, {'type': 'v1', 'addr': '172.21.15.97:6841', 'nonce': 747242680}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24398': {'gid': 24398, 'name': 'k', 'rank': 4, 'incarnation': 23, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.183:6833/4292847121', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6832', 'nonce': 4292847121}, {'type': 'v1', 'addr': '172.21.15.183:6833', 'nonce': 4292847121}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24403': {'gid': 24403, 'name': 'l', 'rank': 3, 'incarnation': 21, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.196:6833/3885816565', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6832', 'nonce': 3885816565}, {'type': 'v1', 'addr': '172.21.15.196:6833', 'nonce': 3885816565}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24419': {'gid': 24419, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.183:6835/1897093105', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6834', 'nonce': 1897093105}, {'type': 'v1', 'addr': '172.21.15.183:6835', 'nonce': 1897093105}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24451': {'gid': 24451, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.196:6835/1171477563', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6834', 'nonce': 1171477563}, {'type': 'v1', 'addr': '172.21.15.196:6835', 'nonce': 1171477563}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24461': {'gid': 24461, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.183:6837/1898341911', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6836', 'nonce': 1898341911}, {'type': 'v1', 'addr': '172.21.15.183:6837', 'nonce': 1898341911}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24469': {'gid': 24469, 'name': 'c', 'rank': 2, 'incarnation': 19, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.196:6837/3094946502', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6836', 'nonce': 3094946502}, {'type': 'v1', 'addr': '172.21.15.196:6837', 'nonce': 3094946502}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24487': {'gid': 24487, 'name': 'i', 'rank': 1, 'incarnation': 17, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.196:6839/83843556', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6838', 'nonce': 83843556}, {'type': 'v1', 'addr': '172.21.15.196:6839', 'nonce': 83843556}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24488': {'gid': 24488, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.183:6839/1763841130', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6838', 'nonce': 1763841130}, {'type': 'v1', 'addr': '172.21.15.183:6839', 'nonce': 1763841130}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24488, 'qdb_cluster': [24469, 24488, 24398, 24403, 24487]} 2024-04-23T10:26:02.247 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-23T10:26:02.247 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2024-04-23T10:26:02.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:01 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/3090270836' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T10:26:02.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:01 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/3141346164' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T10:26:02.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:01 smithi196 ceph-mon[32919]: pgmap v349: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-23T10:26:02.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:01 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/2986237404' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T10:26:02.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:01 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/2241725834' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:26:02.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.347+0000 7f35aec13640 1 -- 172.21.15.97:0/1416170891 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35a8102a30 msgr2=0x7f35a8104e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:02.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.347+0000 7f35aec13640 1 --2- 172.21.15.97:0/1416170891 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35a8102a30 0x7f35a8104e20 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f359c009900 tx=0x7f359c02a410 comp rx=0 tx=0).stop 2024-04-23T10:26:02.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.347+0000 7f35aec13640 1 -- 172.21.15.97:0/1416170891 shutdown_connections 2024-04-23T10:26:02.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.347+0000 7f35aec13640 1 --2- 172.21.15.97:0/1416170891 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f35a8107c90 0x7f35a810a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.347+0000 7f35aec13640 1 --2- 172.21.15.97:0/1416170891 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35a8105360 0x7f35a8107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.347+0000 7f35aec13640 1 --2- 172.21.15.97:0/1416170891 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35a8102a30 0x7f35a8104e20 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.347+0000 7f35aec13640 1 -- 172.21.15.97:0/1416170891 >> 172.21.15.97:0/1416170891 conn(0x7f35a80fc440 msgr2=0x7f35a80fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:02.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.347+0000 7f35aec13640 1 -- 172.21.15.97:0/1416170891 shutdown_connections 2024-04-23T10:26:02.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.347+0000 7f35aec13640 1 -- 172.21.15.97:0/1416170891 wait complete. 2024-04-23T10:26:02.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.347+0000 7f35aec13640 1 Processor -- start 2024-04-23T10:26:02.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.347+0000 7f35aec13640 1 -- start start 2024-04-23T10:26:02.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.348+0000 7f35aec13640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f35a8102a30 0x7f35a8198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.348+0000 7f35aec13640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35a8105360 0x7f35a8198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.348+0000 7f35aec13640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35a8107c90 0x7f35a819f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.348+0000 7f35aec13640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f35a810daf0 con 0x7f35a8107c90 2024-04-23T10:26:02.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.348+0000 7f35aec13640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f35a810d970 con 0x7f35a8102a30 2024-04-23T10:26:02.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.348+0000 7f35aec13640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f35a810dc70 con 0x7f35a8105360 2024-04-23T10:26:02.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.348+0000 7f35ad1b0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35a8107c90 0x7f35a819f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.348+0000 7f35ad1b0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35a8107c90 0x7f35a819f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46708/0 (socket says 172.21.15.97:46708) 2024-04-23T10:26:02.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.348+0000 7f35ad1b0640 1 -- 172.21.15.97:0/2288409552 learned_addr learned my addr 172.21.15.97:0/2288409552 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:02.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.348+0000 7f35ac9af640 1 --2- 172.21.15.97:0/2288409552 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f35a8102a30 0x7f35a8198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.350 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.348+0000 7f35a7fff640 1 --2- 172.21.15.97:0/2288409552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35a8105360 0x7f35a8198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.348+0000 7f35ad1b0640 1 -- 172.21.15.97:0/2288409552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35a8105360 msgr2=0x7f35a8198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:02.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.348+0000 7f35ad1b0640 1 --2- 172.21.15.97:0/2288409552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35a8105360 0x7f35a8198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.348+0000 7f35ad1b0640 1 -- 172.21.15.97:0/2288409552 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f35a8102a30 msgr2=0x7f35a8198220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:02.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.348+0000 7f35ad1b0640 1 --2- 172.21.15.97:0/2288409552 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f35a8102a30 0x7f35a8198220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.348+0000 7f35ad1b0640 1 -- 172.21.15.97:0/2288409552 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35a819fe40 con 0x7f35a8107c90 2024-04-23T10:26:02.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.348+0000 7f35a7fff640 1 --2- 172.21.15.97:0/2288409552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35a8105360 0x7f35a8198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:26:02.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.348+0000 7f35ad1b0640 1 --2- 172.21.15.97:0/2288409552 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35a8107c90 0x7f35a819f7d0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f35a000eb10 tx=0x7f35a000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:02.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.348+0000 7f35a5ffb640 1 -- 172.21.15.97:0/2288409552 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a0004b20 con 0x7f35a8107c90 2024-04-23T10:26:02.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.348+0000 7f35a5ffb640 1 -- 172.21.15.97:0/2288409552 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35a0004cc0 con 0x7f35a8107c90 2024-04-23T10:26:02.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.348+0000 7f35aec13640 1 -- 172.21.15.97:0/2288409552 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35a81a0130 con 0x7f35a8107c90 2024-04-23T10:26:02.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.348+0000 7f35a5ffb640 1 -- 172.21.15.97:0/2288409552 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a0010670 con 0x7f35a8107c90 2024-04-23T10:26:02.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.348+0000 7f35aec13640 1 -- 172.21.15.97:0/2288409552 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35a81a0630 con 0x7f35a8107c90 2024-04-23T10:26:02.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.350+0000 7f35a5ffb640 1 -- 172.21.15.97:0/2288409552 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f35a0005020 con 0x7f35a8107c90 2024-04-23T10:26:02.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.351+0000 7f35b1a08740 1 Processor -- start 2024-04-23T10:26:02.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.351+0000 7f35a5ffb640 1 --2- 172.21.15.97:0/2288409552 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f35980a3bb0 0x7f35980a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.351+0000 7f35b1a08740 1 -- start start 2024-04-23T10:26:02.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.351+0000 7f35b1a08740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55691661eb80 0x55691661efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.351+0000 7f35b1a08740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55691661fd50 0x5569166201d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.351+0000 7f35b1a08740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556916620710 0x556916626fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.351+0000 7f35b1a08740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x556916583c50 con 0x55691661eb80 2024-04-23T10:26:02.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.351+0000 7f35b1a08740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556916520830 con 0x556916620710 2024-04-23T10:26:02.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.351+0000 7f35b1a08740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55691657fc10 con 0x55691661fd50 2024-04-23T10:26:02.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.351+0000 7f35a7fff640 1 --2- 172.21.15.97:0/2288409552 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f35980a3bb0 0x7f35980a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.351+0000 7f35a5ffb640 1 -- 172.21.15.97:0/2288409552 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f35a00d0ab0 con 0x7f35a8107c90 2024-04-23T10:26:02.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.351+0000 7f35ac9af640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55691661eb80 0x55691661efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.351+0000 7f35ac9af640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55691661eb80 0x55691661efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46716/0 (socket says 172.21.15.97:46716) 2024-04-23T10:26:02.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.351+0000 7f35ac9af640 1 -- 172.21.15.97:0/3733750686 learned_addr learned my addr 172.21.15.97:0/3733750686 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:02.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.351+0000 7f35ac9af640 1 -- 172.21.15.97:0/3733750686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55691661fd50 msgr2=0x5569166201d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:02.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.351+0000 7f35ad1b0640 1 --2- 172.21.15.97:0/3733750686 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556916620710 0x556916626fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.351+0000 7f35a7fff640 1 --2- 172.21.15.97:0/3733750686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55691661fd50 0x5569166201d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.352+0000 7f35ac9af640 1 --2- 172.21.15.97:0/3733750686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55691661fd50 0x5569166201d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.352+0000 7f35ac9af640 1 -- 172.21.15.97:0/3733750686 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556916620710 msgr2=0x556916626fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:02.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.352+0000 7f35ac9af640 1 --2- 172.21.15.97:0/3733750686 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556916620710 0x556916626fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.352+0000 7f35a7fff640 1 --2- 172.21.15.97:0/2288409552 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f35980a3bb0 0x7f35980a6070 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f35940058c0 tx=0x7f3594009700 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:02.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.352+0000 7f35ac9af640 1 -- 172.21.15.97:0/3733750686 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569162962d0 con 0x55691661eb80 2024-04-23T10:26:02.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.352+0000 7f35ac9af640 1 --2- 172.21.15.97:0/3733750686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55691661eb80 0x55691661efa0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f359c009900 tx=0x7f359c032b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4271791145ec8e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:02.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.352+0000 7f3592ffd640 1 -- 172.21.15.97:0/3733750686 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359c03f070 con 0x55691661eb80 2024-04-23T10:26:02.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.352+0000 7f3592ffd640 1 -- 172.21.15.97:0/3733750686 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f359c004040 con 0x55691661eb80 2024-04-23T10:26:02.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.352+0000 7f3592ffd640 1 -- 172.21.15.97:0/3733750686 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359c035070 con 0x55691661eb80 2024-04-23T10:26:02.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.352+0000 7f35b1a08740 1 -- 172.21.15.97:0/3733750686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55691661eb80 msgr2=0x55691661efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:02.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.352+0000 7f35b1a08740 1 --2- 172.21.15.97:0/3733750686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55691661eb80 0x55691661efa0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f359c009900 tx=0x7f359c032b20 comp rx=0 tx=0).stop 2024-04-23T10:26:02.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.353+0000 7f35b1a08740 1 -- 172.21.15.97:0/3733750686 shutdown_connections 2024-04-23T10:26:02.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.353+0000 7f35b1a08740 1 --2- 172.21.15.97:0/3733750686 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556916620710 0x556916626fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.353+0000 7f35b1a08740 1 --2- 172.21.15.97:0/3733750686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55691661fd50 0x5569166201d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.353+0000 7f35b1a08740 1 --2- 172.21.15.97:0/3733750686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55691661eb80 0x55691661efa0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.353+0000 7f35b1a08740 1 -- 172.21.15.97:0/3733750686 >> 172.21.15.97:0/3733750686 conn(0x55691661a070 msgr2=0x55691661a470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:02.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.353+0000 7f35b1a08740 1 -- 172.21.15.97:0/3733750686 shutdown_connections 2024-04-23T10:26:02.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.353+0000 7f35b1a08740 1 -- 172.21.15.97:0/3733750686 wait complete. 2024-04-23T10:26:02.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.353+0000 7f35b1a08740 1 -- start start 2024-04-23T10:26:02.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.353+0000 7f35b1a08740 1 Processor -- start 2024-04-23T10:26:02.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.353+0000 7f35b1a08740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55691661eb80 0x556916628fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.353+0000 7f35b1a08740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55691661fd50 0x5569166294f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.353+0000 7f35b1a08740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556916620710 0x556916630530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.353+0000 7f35b1a08740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5569166276c0 con 0x556916620710 2024-04-23T10:26:02.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.353+0000 7f35b1a08740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55691651e470 con 0x55691661fd50 2024-04-23T10:26:02.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.353+0000 7f35b1a08740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556916627840 con 0x55691661eb80 2024-04-23T10:26:02.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.353+0000 7f35b1a08740 10 client.0 _ll_register_callbacks cb 0x556916543f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:26:02.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.353+0000 7f35ad1b0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556916620710 0x556916630530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.353+0000 7f35ad1b0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556916620710 0x556916630530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46722/0 (socket says 172.21.15.97:46722) 2024-04-23T10:26:02.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.353+0000 7f35ad1b0640 1 -- 172.21.15.97:0/50879209 learned_addr learned my addr 172.21.15.97:0/50879209 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:02.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.354+0000 7f35ac9af640 1 --2- 172.21.15.97:0/50879209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55691661eb80 0x556916628fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.354+0000 7f35a7fff640 1 --2- 172.21.15.97:0/50879209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55691661fd50 0x5569166294f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.354+0000 7f35ad1b0640 1 -- 172.21.15.97:0/50879209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55691661eb80 msgr2=0x556916628fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:02.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.354+0000 7f35ad1b0640 1 --2- 172.21.15.97:0/50879209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55691661eb80 0x556916628fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.354+0000 7f35ad1b0640 1 -- 172.21.15.97:0/50879209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55691661fd50 msgr2=0x5569166294f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:02.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.354+0000 7f35ad1b0640 1 --2- 172.21.15.97:0/50879209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55691661fd50 0x5569166294f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.354+0000 7f35ad1b0640 1 -- 172.21.15.97:0/50879209 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569163606b0 con 0x556916620710 2024-04-23T10:26:02.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.354+0000 7f35ad1b0640 1 --2- 172.21.15.97:0/50879209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556916620710 0x556916630530 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f35a00a5500 tx=0x7f35a009c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75e4d1522b437a86 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:02.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.354+0000 7f3587fff640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:26:02.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.354+0000 7f3587fff640 1 -- 172.21.15.97:0/50879209 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a009cdc0 con 0x556916620710 2024-04-23T10:26:02.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.354+0000 7f3587fff640 1 -- 172.21.15.97:0/50879209 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f35a0005310 con 0x556916620710 2024-04-23T10:26:02.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.354+0000 7f35b1a08740 1 -- 172.21.15.97:0/50879209 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f359c004040 con 0x556916620710 2024-04-23T10:26:02.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.354+0000 7f3587fff640 1 -- 172.21.15.97:0/50879209 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f35a00a1170 con 0x556916620710 2024-04-23T10:26:02.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.354+0000 7f35b1a08740 10 client.15276 fetch_fsmap learned FSMap version 25 2024-04-23T10:26:02.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.354+0000 7f35b1a08740 1 -- 172.21.15.97:0/50879209 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556916633910 con 0x556916620710 2024-04-23T10:26:02.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.354+0000 7f3587fff640 1 -- 172.21.15.97:0/50879209 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f359c004040 con 0x556916620710 2024-04-23T10:26:02.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.354+0000 7f3587fff640 1 -- 172.21.15.97:0/50879209 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f35a00bd150 con 0x556916620710 2024-04-23T10:26:02.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.354+0000 7f35b1a08740 10 client.15276 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T10:26:02.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.354+0000 7f35b1a08740 10 client.15276 resolve_mds: resolved name 'k' to daemon mds.k 2024-04-23T10:26:02.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.355+0000 7f35b1a08740 20 client.15276 populate_metadata read hostname 'smithi097' 2024-04-23T10:26:02.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.355+0000 7f35b1a08740 1 --2- 172.21.15.97:0/50879209 >> [v2:172.21.15.183:6832/4292847121,v1:172.21.15.183:6833/4292847121] conn(0x556916635f90 0x556916637400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.355+0000 7f35b1a08740 4 client.15276 mds_command: new command op to 24398 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:26:02.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.355+0000 7f35b1a08740 1 -- 172.21.15.97:0/50879209 --> [v2:172.21.15.183:6832/4292847121,v1:172.21.15.183:6833/4292847121] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556916591670 con 0x556916635f90 2024-04-23T10:26:02.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.355+0000 7f35ac9af640 1 --2- 172.21.15.97:0/50879209 >> [v2:172.21.15.183:6832/4292847121,v1:172.21.15.183:6833/4292847121] conn(0x556916635f90 0x556916637400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.355+0000 7f35ac9af640 1 --2- 172.21.15.97:0/50879209 >> [v2:172.21.15.183:6832/4292847121,v1:172.21.15.183:6833/4292847121] conn(0x556916635f90 0x556916637400 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7c9dee23d6850a7a server_cookie=f4b1424a702d162a in_seq=0 out_seq=0 2024-04-23T10:26:02.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.355+0000 7f3587fff640 10 client.15276 ms_handle_connect on v2:172.21.15.183:6832/4292847121 2024-04-23T10:26:02.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.356+0000 7f3587fff640 1 -- 172.21.15.97:0/50879209 <== mds.4 v2:172.21.15.183:6832/4292847121 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556916591670 con 0x556916635f90 2024-04-23T10:26:02.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.356+0000 7f3587fff640 10 client.15276 handle_command_reply: tid=0 2024-04-23T10:26:02.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.356+0000 7f35b1a08740 1 client.15276 shutdown 2024-04-23T10:26:02.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.356+0000 7f35b1a08740 1 -- 172.21.15.97:0/50879209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556916620710 msgr2=0x556916630530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:02.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.356+0000 7f35b1a08740 1 --2- 172.21.15.97:0/50879209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556916620710 0x556916630530 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f35a00a5500 tx=0x7f35a009c9c0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.356+0000 7f35b1a08740 1 -- 172.21.15.97:0/50879209 shutdown_connections 2024-04-23T10:26:02.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.356+0000 7f35b1a08740 1 --2- 172.21.15.97:0/50879209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556916620710 0x556916630530 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.356+0000 7f35b1a08740 1 --2- 172.21.15.97:0/50879209 >> [v2:172.21.15.183:6832/4292847121,v1:172.21.15.183:6833/4292847121] conn(0x556916635f90 0x556916637400 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.356+0000 7f35b1a08740 1 --2- 172.21.15.97:0/50879209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55691661fd50 0x5569166294f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.356+0000 7f35b1a08740 1 --2- 172.21.15.97:0/50879209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55691661eb80 0x556916628fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.356+0000 7f3587fff640 0 client.15276 ms_handle_reset on v2:172.21.15.183:6832/4292847121 2024-04-23T10:26:02.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.356+0000 7f35b1a08740 1 -- 172.21.15.97:0/50879209 >> 172.21.15.97:0/50879209 conn(0x55691661a070 msgr2=0x5569166191d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:02.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.356+0000 7f35b1a08740 1 -- 172.21.15.97:0/50879209 shutdown_connections 2024-04-23T10:26:02.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.356+0000 7f35b1a08740 1 -- 172.21.15.97:0/50879209 wait complete. 2024-04-23T10:26:02.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.357+0000 7f35b1a08740 20 client.15276 trim_cache size 0 max 16384 2024-04-23T10:26:02.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.374+0000 7f35b1a08740 1 Processor -- start 2024-04-23T10:26:02.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.374+0000 7f35b1a08740 1 -- start start 2024-04-23T10:26:02.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.375+0000 7f35b1a08740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55691661eb80 0x5569166150b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.375+0000 7f35b1a08740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55691661fd50 0x5569166304f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.375+0000 7f35b1a08740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556916620710 0x556916623870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.375+0000 7f35b1a08740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x556916635740 con 0x55691661fd50 2024-04-23T10:26:02.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.375+0000 7f35b1a08740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556916635e10 con 0x55691661eb80 2024-04-23T10:26:02.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.375+0000 7f35b1a08740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5569166364e0 con 0x556916620710 2024-04-23T10:26:02.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.375+0000 7f35a7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55691661fd50 0x5569166304f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.375+0000 7f35a7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55691661fd50 0x5569166304f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46728/0 (socket says 172.21.15.97:46728) 2024-04-23T10:26:02.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.375+0000 7f35a7fff640 1 -- 172.21.15.97:0/2581440706 learned_addr learned my addr 172.21.15.97:0/2581440706 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:02.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.375+0000 7f35ac9af640 1 --2- 172.21.15.97:0/2581440706 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55691661eb80 0x5569166150b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.375+0000 7f35ad1b0640 1 --2- 172.21.15.97:0/2581440706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556916620710 0x556916623870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.375+0000 7f35a7fff640 1 -- 172.21.15.97:0/2581440706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556916620710 msgr2=0x556916623870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:02.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.375+0000 7f35a7fff640 1 --2- 172.21.15.97:0/2581440706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556916620710 0x556916623870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.375+0000 7f35a7fff640 1 -- 172.21.15.97:0/2581440706 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55691661eb80 msgr2=0x5569166150b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:02.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.375+0000 7f35a7fff640 1 --2- 172.21.15.97:0/2581440706 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55691661eb80 0x5569166150b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.375+0000 7f35a7fff640 1 -- 172.21.15.97:0/2581440706 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55691661f7d0 con 0x55691661fd50 2024-04-23T10:26:02.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.375+0000 7f35a7fff640 1 --2- 172.21.15.97:0/2581440706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55691661fd50 0x5569166304f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f3594009d80 tx=0x7f3594015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf52ecbbbd1f73c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:02.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.375+0000 7f35927fc640 1 -- 172.21.15.97:0/2581440706 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3594029470 con 0x55691661fd50 2024-04-23T10:26:02.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.375+0000 7f35927fc640 1 -- 172.21.15.97:0/2581440706 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3594024070 con 0x55691661fd50 2024-04-23T10:26:02.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.375+0000 7f35927fc640 1 -- 172.21.15.97:0/2581440706 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3594020850 con 0x55691661fd50 2024-04-23T10:26:02.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.376+0000 7f35b1a08740 1 -- 172.21.15.97:0/2581440706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55691661fd50 msgr2=0x5569166304f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:02.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.376+0000 7f35b1a08740 1 --2- 172.21.15.97:0/2581440706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55691661fd50 0x5569166304f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f3594009d80 tx=0x7f3594015000 comp rx=0 tx=0).stop 2024-04-23T10:26:02.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.376+0000 7f35b1a08740 1 -- 172.21.15.97:0/2581440706 shutdown_connections 2024-04-23T10:26:02.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.376+0000 7f35b1a08740 1 --2- 172.21.15.97:0/2581440706 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556916620710 0x556916623870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.376+0000 7f35b1a08740 1 --2- 172.21.15.97:0/2581440706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55691661fd50 0x5569166304f0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.376+0000 7f35b1a08740 1 --2- 172.21.15.97:0/2581440706 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55691661eb80 0x5569166150b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.376+0000 7f35b1a08740 1 -- 172.21.15.97:0/2581440706 >> 172.21.15.97:0/2581440706 conn(0x55691661a070 msgr2=0x55691661a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:02.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.376+0000 7f35b1a08740 1 -- 172.21.15.97:0/2581440706 shutdown_connections 2024-04-23T10:26:02.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.376+0000 7f35b1a08740 1 -- 172.21.15.97:0/2581440706 wait complete. 2024-04-23T10:26:02.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.376+0000 7f35b1a08740 1 -- start start 2024-04-23T10:26:02.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.376+0000 7f35b1a08740 1 Processor -- start 2024-04-23T10:26:02.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.377+0000 7f35b1a08740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55691661eb80 0x55691662bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.377+0000 7f35b1a08740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55691661fd50 0x55691662c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.377+0000 7f35b1a08740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556916620710 0x55691662f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.377+0000 7f35b1a08740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x556916637280 con 0x55691661eb80 2024-04-23T10:26:02.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.377+0000 7f35b1a08740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556916636bb0 con 0x556916620710 2024-04-23T10:26:02.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.377+0000 7f35b1a08740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556916637950 con 0x55691661fd50 2024-04-23T10:26:02.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.377+0000 7f35b1a08740 10 client.0 _ll_register_callbacks cb 0x7f35a0099150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:26:02.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.377+0000 7f35ac9af640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55691661eb80 0x55691662bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.377+0000 7f35ac9af640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55691661eb80 0x55691662bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46740/0 (socket says 172.21.15.97:46740) 2024-04-23T10:26:02.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.377+0000 7f35ac9af640 1 -- 172.21.15.97:0/2400216878 learned_addr learned my addr 172.21.15.97:0/2400216878 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:02.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.377+0000 7f35ad1b0640 1 --2- 172.21.15.97:0/2400216878 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556916620710 0x55691662f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.377+0000 7f35ac9af640 1 -- 172.21.15.97:0/2400216878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55691661fd50 msgr2=0x55691662c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:02.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.377+0000 7f35ac9af640 1 --2- 172.21.15.97:0/2400216878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55691661fd50 0x55691662c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.377+0000 7f35ac9af640 1 -- 172.21.15.97:0/2400216878 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556916620710 msgr2=0x55691662f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:02.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.377+0000 7f35ac9af640 1 --2- 172.21.15.97:0/2400216878 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556916620710 0x55691662f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.377+0000 7f35ac9af640 1 -- 172.21.15.97:0/2400216878 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55691662f930 con 0x55691661eb80 2024-04-23T10:26:02.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.377+0000 7f35ac9af640 1 --2- 172.21.15.97:0/2400216878 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55691661eb80 0x55691662bde0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f359c009900 tx=0x7f359c0055c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e1b3cf8641492ee server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:02.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.377+0000 7f3587fff640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:26:02.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.377+0000 7f3587fff640 1 -- 172.21.15.97:0/2400216878 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359c03f040 con 0x55691661eb80 2024-04-23T10:26:02.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.377+0000 7f3587fff640 1 -- 172.21.15.97:0/2400216878 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f359c0317a0 con 0x55691661eb80 2024-04-23T10:26:02.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.377+0000 7f35b1a08740 1 -- 172.21.15.97:0/2400216878 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3594024070 con 0x55691661eb80 2024-04-23T10:26:02.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.377+0000 7f3587fff640 1 -- 172.21.15.97:0/2400216878 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f359c05e050 con 0x55691661eb80 2024-04-23T10:26:02.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.377+0000 7f35b1a08740 10 client.15282 fetch_fsmap learned FSMap version 25 2024-04-23T10:26:02.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.377+0000 7f35b1a08740 1 -- 172.21.15.97:0/2400216878 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55691669c150 con 0x55691661eb80 2024-04-23T10:26:02.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.378+0000 7f3587fff640 1 -- 172.21.15.97:0/2400216878 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3594024070 con 0x55691661eb80 2024-04-23T10:26:02.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.378+0000 7f3587fff640 1 -- 172.21.15.97:0/2400216878 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f359c0447c0 con 0x55691661eb80 2024-04-23T10:26:02.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.378+0000 7f35b1a08740 10 client.15282 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T10:26:02.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.378+0000 7f35b1a08740 10 client.15282 resolve_mds: resolved name 'k' to daemon mds.k 2024-04-23T10:26:02.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.378+0000 7f35b1a08740 20 client.15282 populate_metadata read hostname 'smithi097' 2024-04-23T10:26:02.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.378+0000 7f35b1a08740 1 --2- 172.21.15.97:0/2400216878 >> [v2:172.21.15.183:6832/4292847121,v1:172.21.15.183:6833/4292847121] conn(0x5569166a09e0 0x55691669f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.378+0000 7f35b1a08740 4 client.15282 mds_command: new command op to 24398 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T10:26:02.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.378+0000 7f35b1a08740 1 -- 172.21.15.97:0/2400216878 --> [v2:172.21.15.183:6832/4292847121,v1:172.21.15.183:6833/4292847121] -- command(tid 0: {"prefix": "status"}) -- 0x55691658b4e0 con 0x5569166a09e0 2024-04-23T10:26:02.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.378+0000 7f35a7fff640 1 --2- 172.21.15.97:0/2400216878 >> [v2:172.21.15.183:6832/4292847121,v1:172.21.15.183:6833/4292847121] conn(0x5569166a09e0 0x55691669f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.378+0000 7f35a7fff640 1 --2- 172.21.15.97:0/2400216878 >> [v2:172.21.15.183:6832/4292847121,v1:172.21.15.183:6833/4292847121] conn(0x5569166a09e0 0x55691669f6f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f22c6b33bbb66394 server_cookie=62e307644269fc95 in_seq=0 out_seq=0 2024-04-23T10:26:02.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.378+0000 7f3587fff640 10 client.15282 ms_handle_connect on v2:172.21.15.183:6832/4292847121 2024-04-23T10:26:02.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.379+0000 7f3587fff640 1 -- 172.21.15.97:0/2400216878 <== mds.4 v2:172.21.15.183:6832/4292847121 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55691658b4e0 con 0x5569166a09e0 2024-04-23T10:26:02.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.379+0000 7f3587fff640 10 client.15282 handle_command_reply: tid=0 2024-04-23T10:26:02.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.379+0000 7f35b1a08740 1 client.15282 shutdown 2024-04-23T10:26:02.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.379+0000 7f35b1a08740 1 -- 172.21.15.97:0/2400216878 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55691661eb80 msgr2=0x55691662bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:02.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.379+0000 7f35b1a08740 1 --2- 172.21.15.97:0/2400216878 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55691661eb80 0x55691662bde0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f359c009900 tx=0x7f359c0055c0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.379+0000 7f35b1a08740 1 -- 172.21.15.97:0/2400216878 shutdown_connections 2024-04-23T10:26:02.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.379+0000 7f35b1a08740 1 --2- 172.21.15.97:0/2400216878 >> [v2:172.21.15.183:6832/4292847121,v1:172.21.15.183:6833/4292847121] conn(0x5569166a09e0 0x55691669f6f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.379+0000 7f35b1a08740 1 --2- 172.21.15.97:0/2400216878 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556916620710 0x55691662f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.379+0000 7f35b1a08740 1 --2- 172.21.15.97:0/2400216878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55691661fd50 0x55691662c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.379+0000 7f35b1a08740 1 --2- 172.21.15.97:0/2400216878 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55691661eb80 0x55691662bde0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.379+0000 7f3587fff640 0 client.15282 ms_handle_reset on v2:172.21.15.183:6832/4292847121 2024-04-23T10:26:02.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.379+0000 7f35b1a08740 1 -- 172.21.15.97:0/2400216878 >> 172.21.15.97:0/2400216878 conn(0x55691661a070 msgr2=0x5569166150b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:02.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.379+0000 7f35b1a08740 1 -- 172.21.15.97:0/2400216878 shutdown_connections 2024-04-23T10:26:02.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.379+0000 7f35b1a08740 1 -- 172.21.15.97:0/2400216878 wait complete. 2024-04-23T10:26:02.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.379+0000 7f35b1a08740 20 client.15282 trim_cache size 0 max 16384 2024-04-23T10:26:02.387 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-23T10:26:02.387 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "9b22af88-0159-11ef-bc93-c7b262605968", 2024-04-23T10:26:02.387 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 4, 2024-04-23T10:26:02.387 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 24398, 2024-04-23T10:26:02.387 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-04-23T10:26:02.387 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-04-23T10:26:02.387 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-04-23T10:26:02.387 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 110.412748784, 2024-04-23T10:26:02.387 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 25, 2024-04-23T10:26:02.387 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 88, 2024-04-23T10:26:02.387 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 86, 2024-04-23T10:26:02.388 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 149.954593964 2024-04-23T10:26:02.388 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-23T10:26:02.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.379+0000 7f35aec13640 1 -- 172.21.15.97:0/2288409552 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f35980a3bb0 msgr2=0x7f35980a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:02.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.379+0000 7f35aec13640 1 --2- 172.21.15.97:0/2288409552 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f35980a3bb0 0x7f35980a6070 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f35940058c0 tx=0x7f3594009700 comp rx=0 tx=0).stop 2024-04-23T10:26:02.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.379+0000 7f35aec13640 1 -- 172.21.15.97:0/2288409552 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35a8107c90 msgr2=0x7f35a819f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:02.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.379+0000 7f35aec13640 1 --2- 172.21.15.97:0/2288409552 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35a8107c90 0x7f35a819f7d0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f35a000eb10 tx=0x7f35a000c2d0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.379+0000 7f35aec13640 1 -- 172.21.15.97:0/2288409552 shutdown_connections 2024-04-23T10:26:02.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.379+0000 7f35aec13640 1 --2- 172.21.15.97:0/2288409552 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f35980a3bb0 0x7f35980a6070 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.379+0000 7f35aec13640 1 --2- 172.21.15.97:0/2288409552 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f35a8107c90 0x7f35a819f7d0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.379+0000 7f35aec13640 1 --2- 172.21.15.97:0/2288409552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35a8105360 0x7f35a8198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.379+0000 7f35aec13640 1 --2- 172.21.15.97:0/2288409552 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f35a8102a30 0x7f35a8198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.379+0000 7f35aec13640 1 -- 172.21.15.97:0/2288409552 >> 172.21.15.97:0/2288409552 conn(0x7f35a80fc440 msgr2=0x7f35a81035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:02.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.380+0000 7f35aec13640 1 -- 172.21.15.97:0/2288409552 shutdown_connections 2024-04-23T10:26:02.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.380+0000 7f35aec13640 1 -- 172.21.15.97:0/2288409552 wait complete. 2024-04-23T10:26:02.390 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.l status 2024-04-23T10:26:02.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:01 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/3090270836' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T10:26:02.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:01 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/3141346164' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T10:26:02.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:01 smithi183 ceph-mon[32317]: pgmap v349: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-23T10:26:02.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:01 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/2986237404' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T10:26:02.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:01 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/2241725834' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:26:02.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.490+0000 7f74de1cd640 1 -- 172.21.15.97:0/1628214016 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74d8107c90 msgr2=0x7f74d810a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:02.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.490+0000 7f74de1cd640 1 --2- 172.21.15.97:0/1628214016 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74d8107c90 0x7f74d810a0e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f74d0009ca0 tx=0x7f74d002a8c0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.490+0000 7f74de1cd640 1 -- 172.21.15.97:0/1628214016 shutdown_connections 2024-04-23T10:26:02.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.490+0000 7f74de1cd640 1 --2- 172.21.15.97:0/1628214016 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74d8107c90 0x7f74d810a0e0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.490+0000 7f74de1cd640 1 --2- 172.21.15.97:0/1628214016 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74d8105360 0x7f74d8107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.490+0000 7f74de1cd640 1 --2- 172.21.15.97:0/1628214016 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f74d8102a30 0x7f74d8104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.490+0000 7f74de1cd640 1 -- 172.21.15.97:0/1628214016 >> 172.21.15.97:0/1628214016 conn(0x7f74d80fc440 msgr2=0x7f74d80fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:02.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.490+0000 7f74de1cd640 1 -- 172.21.15.97:0/1628214016 shutdown_connections 2024-04-23T10:26:02.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.490+0000 7f74de1cd640 1 -- 172.21.15.97:0/1628214016 wait complete. 2024-04-23T10:26:02.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.491+0000 7f74de1cd640 1 Processor -- start 2024-04-23T10:26:02.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.491+0000 7f74de1cd640 1 -- start start 2024-04-23T10:26:02.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.491+0000 7f74de1cd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74d8102a30 0x7f74d8198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.491+0000 7f74de1cd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74d8105360 0x7f74d81986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.491+0000 7f74de1cd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f74d8107c90 0x7f74d819f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.491+0000 7f74de1cd640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f74d810daf0 con 0x7f74d8102a30 2024-04-23T10:26:02.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.491+0000 7f74de1cd640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f74d810d970 con 0x7f74d8105360 2024-04-23T10:26:02.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.491+0000 7f74de1cd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f74d810dc70 con 0x7f74d8107c90 2024-04-23T10:26:02.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.491+0000 7f74d77fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74d8102a30 0x7f74d8198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.491+0000 7f74d77fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74d8102a30 0x7f74d8198180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46758/0 (socket says 172.21.15.97:46758) 2024-04-23T10:26:02.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.491+0000 7f74d77fe640 1 -- 172.21.15.97:0/4070196668 learned_addr learned my addr 172.21.15.97:0/4070196668 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:02.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.491+0000 7f74d7fff640 1 --2- 172.21.15.97:0/4070196668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f74d8107c90 0x7f74d819f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.491+0000 7f74d6ffd640 1 --2- 172.21.15.97:0/4070196668 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74d8105360 0x7f74d81986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.491+0000 7f74d77fe640 1 -- 172.21.15.97:0/4070196668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f74d8107c90 msgr2=0x7f74d819f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:02.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.492+0000 7f74d77fe640 1 --2- 172.21.15.97:0/4070196668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f74d8107c90 0x7f74d819f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.492+0000 7f74d77fe640 1 -- 172.21.15.97:0/4070196668 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74d8105360 msgr2=0x7f74d81986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:02.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.492+0000 7f74d77fe640 1 --2- 172.21.15.97:0/4070196668 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74d8105360 0x7f74d81986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.492+0000 7f74d77fe640 1 -- 172.21.15.97:0/4070196668 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74d819fda0 con 0x7f74d8102a30 2024-04-23T10:26:02.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.492+0000 7f74d77fe640 1 --2- 172.21.15.97:0/4070196668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74d8102a30 0x7f74d8198180 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f74c800c890 tx=0x7f74c800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:02.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.492+0000 7f74d4ff9640 1 -- 172.21.15.97:0/4070196668 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c8004220 con 0x7f74d8102a30 2024-04-23T10:26:02.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.492+0000 7f74de1cd640 1 -- 172.21.15.97:0/4070196668 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74d81a0090 con 0x7f74d8102a30 2024-04-23T10:26:02.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.492+0000 7f74d4ff9640 1 -- 172.21.15.97:0/4070196668 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74c80043c0 con 0x7f74d8102a30 2024-04-23T10:26:02.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.492+0000 7f74d4ff9640 1 -- 172.21.15.97:0/4070196668 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c8004a00 con 0x7f74d8102a30 2024-04-23T10:26:02.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.492+0000 7f74de1cd640 1 -- 172.21.15.97:0/4070196668 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74d81a0630 con 0x7f74d8102a30 2024-04-23T10:26:02.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.494+0000 7f74d4ff9640 1 -- 172.21.15.97:0/4070196668 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f74c80079b0 con 0x7f74d8102a30 2024-04-23T10:26:02.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.494+0000 7f74e0fc4740 1 Processor -- start 2024-04-23T10:26:02.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.494+0000 7f74e0fc4740 1 -- start start 2024-04-23T10:26:02.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.495+0000 7f74e0fc4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55afa7cd2f90 0x55afa7cd33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.495+0000 7f74e0fc4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55afa7cd4160 0x55afa7cd45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.495+0000 7f74e0fc4740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afa7cd4b20 0x55afa7cdb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.495+0000 7f74e0fc4740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55afa7c33c50 con 0x55afa7cd4b20 2024-04-23T10:26:02.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.495+0000 7f74e0fc4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55afa7bd0830 con 0x55afa7cd4160 2024-04-23T10:26:02.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.495+0000 7f74e0fc4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55afa7c2fc10 con 0x55afa7cd2f90 2024-04-23T10:26:02.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.495+0000 7f74d6ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afa7cd4b20 0x55afa7cdb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.495+0000 7f74d6ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afa7cd4b20 0x55afa7cdb3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46762/0 (socket says 172.21.15.97:46762) 2024-04-23T10:26:02.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.495+0000 7f74d6ffd640 1 -- 172.21.15.97:0/3806764077 learned_addr learned my addr 172.21.15.97:0/3806764077 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:02.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.495+0000 7f74d7fff640 1 --2- 172.21.15.97:0/3806764077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55afa7cd2f90 0x55afa7cd33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.495+0000 7f74d77fe640 1 --2- 172.21.15.97:0/3806764077 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55afa7cd4160 0x55afa7cd45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.495+0000 7f74d4ff9640 1 --2- 172.21.15.97:0/4070196668 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f74c00a3bb0 0x7f74c00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.495+0000 7f74d6ffd640 1 -- 172.21.15.97:0/3806764077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55afa7cd2f90 msgr2=0x55afa7cd33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:02.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.495+0000 7f74d6ffd640 1 --2- 172.21.15.97:0/3806764077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55afa7cd2f90 0x55afa7cd33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.495+0000 7f74d6ffd640 1 -- 172.21.15.97:0/3806764077 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55afa7cd4160 msgr2=0x55afa7cd45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:02.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.495+0000 7f74d6ffd640 1 --2- 172.21.15.97:0/3806764077 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55afa7cd4160 0x55afa7cd45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.495+0000 7f74d6ffd640 1 -- 172.21.15.97:0/3806764077 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afa79462d0 con 0x55afa7cd4b20 2024-04-23T10:26:02.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.495+0000 7f74d7fff640 1 --2- 172.21.15.97:0/4070196668 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f74c00a3bb0 0x7f74c00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.495+0000 7f74d77fe640 1 --2- 172.21.15.97:0/3806764077 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55afa7cd4160 0x55afa7cd45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:26:02.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.495+0000 7f74d4ff9640 1 -- 172.21.15.97:0/4070196668 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f74c80cffc0 con 0x7f74d8102a30 2024-04-23T10:26:02.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.495+0000 7f74d6ffd640 1 --2- 172.21.15.97:0/3806764077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afa7cd4b20 0x55afa7cdb3b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f74cc00bc70 tx=0x7f74cc0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3555d33a55e6b86b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:02.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.495+0000 7f74bdffb640 1 -- 172.21.15.97:0/3806764077 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74cc005220 con 0x55afa7cd4b20 2024-04-23T10:26:02.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.495+0000 7f74d7fff640 1 --2- 172.21.15.97:0/4070196668 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f74c00a3bb0 0x7f74c00a6070 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f74d0031b50 tx=0x7f74d0040040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:02.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.495+0000 7f74bdffb640 1 -- 172.21.15.97:0/3806764077 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74cc00be20 con 0x55afa7cd4b20 2024-04-23T10:26:02.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.495+0000 7f74bdffb640 1 -- 172.21.15.97:0/3806764077 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74cc0116a0 con 0x55afa7cd4b20 2024-04-23T10:26:02.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.496+0000 7f74e0fc4740 1 -- 172.21.15.97:0/3806764077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afa7cd4b20 msgr2=0x55afa7cdb3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:02.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.496+0000 7f74e0fc4740 1 --2- 172.21.15.97:0/3806764077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afa7cd4b20 0x55afa7cdb3b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f74cc00bc70 tx=0x7f74cc0107a0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.496+0000 7f74e0fc4740 1 -- 172.21.15.97:0/3806764077 shutdown_connections 2024-04-23T10:26:02.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.496+0000 7f74e0fc4740 1 --2- 172.21.15.97:0/3806764077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afa7cd4b20 0x55afa7cdb3b0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.496+0000 7f74e0fc4740 1 --2- 172.21.15.97:0/3806764077 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55afa7cd4160 0x55afa7cd45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.496+0000 7f74e0fc4740 1 --2- 172.21.15.97:0/3806764077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55afa7cd2f90 0x55afa7cd33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.496+0000 7f74e0fc4740 1 -- 172.21.15.97:0/3806764077 >> 172.21.15.97:0/3806764077 conn(0x55afa7cca070 msgr2=0x55afa7cca470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:02.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.496+0000 7f74e0fc4740 1 -- 172.21.15.97:0/3806764077 shutdown_connections 2024-04-23T10:26:02.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.496+0000 7f74e0fc4740 1 -- 172.21.15.97:0/3806764077 wait complete. 2024-04-23T10:26:02.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.496+0000 7f74e0fc4740 1 -- start start 2024-04-23T10:26:02.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.497+0000 7f74e0fc4740 1 Processor -- start 2024-04-23T10:26:02.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.497+0000 7f74e0fc4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55afa7cd2f90 0x55afa7cdd400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.497+0000 7f74e0fc4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55afa7cd4160 0x55afa7cdd940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.497+0000 7f74e0fc4740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afa7cd4b20 0x55afa7ce4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.497+0000 7f74e0fc4740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55afa7cdbb30 con 0x55afa7cd4b20 2024-04-23T10:26:02.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.497+0000 7f74e0fc4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55afa7bce470 con 0x55afa7cd4160 2024-04-23T10:26:02.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.497+0000 7f74e0fc4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55afa7cdbcb0 con 0x55afa7cd2f90 2024-04-23T10:26:02.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.497+0000 7f74e0fc4740 10 client.0 _ll_register_callbacks cb 0x55afa7bf3f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:26:02.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.497+0000 7f74d7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afa7cd4b20 0x55afa7ce4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.497+0000 7f74d7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afa7cd4b20 0x55afa7ce4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46772/0 (socket says 172.21.15.97:46772) 2024-04-23T10:26:02.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.497+0000 7f74d7fff640 1 -- 172.21.15.97:0/3554616542 learned_addr learned my addr 172.21.15.97:0/3554616542 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:02.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.497+0000 7f74d77fe640 1 --2- 172.21.15.97:0/3554616542 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55afa7cd2f90 0x55afa7cdd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.497+0000 7f74d6ffd640 1 --2- 172.21.15.97:0/3554616542 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55afa7cd4160 0x55afa7cdd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.497+0000 7f74d7fff640 1 -- 172.21.15.97:0/3554616542 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55afa7cd2f90 msgr2=0x55afa7cdd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:02.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.497+0000 7f74d7fff640 1 --2- 172.21.15.97:0/3554616542 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55afa7cd2f90 0x55afa7cdd400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.497+0000 7f74d7fff640 1 -- 172.21.15.97:0/3554616542 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55afa7cd4160 msgr2=0x55afa7cdd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:02.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.497+0000 7f74d7fff640 1 --2- 172.21.15.97:0/3554616542 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55afa7cd4160 0x55afa7cdd940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.497+0000 7f74d7fff640 1 -- 172.21.15.97:0/3554616542 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afa7a106b0 con 0x55afa7cd4b20 2024-04-23T10:26:02.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.497+0000 7f74d7fff640 1 --2- 172.21.15.97:0/3554616542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afa7cd4b20 0x55afa7ce4980 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f74d0035000 tx=0x7f74d00095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=513d8a8779aebb99 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:02.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.497+0000 7f74b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:26:02.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.498+0000 7f74b6ffd640 1 -- 172.21.15.97:0/3554616542 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d002f660 con 0x55afa7cd4b20 2024-04-23T10:26:02.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.498+0000 7f74b6ffd640 1 -- 172.21.15.97:0/3554616542 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f74d002f800 con 0x55afa7cd4b20 2024-04-23T10:26:02.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.498+0000 7f74e0fc4740 1 -- 172.21.15.97:0/3554616542 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74cc00be20 con 0x55afa7cd4b20 2024-04-23T10:26:02.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.498+0000 7f74b6ffd640 1 -- 172.21.15.97:0/3554616542 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f74d0059070 con 0x55afa7cd4b20 2024-04-23T10:26:02.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.498+0000 7f74e0fc4740 10 client.15294 fetch_fsmap learned FSMap version 25 2024-04-23T10:26:02.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.498+0000 7f74e0fc4740 1 -- 172.21.15.97:0/3554616542 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55afa7ce7cf0 con 0x55afa7cd4b20 2024-04-23T10:26:02.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.498+0000 7f74b6ffd640 1 -- 172.21.15.97:0/3554616542 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74cc00be20 con 0x55afa7cd4b20 2024-04-23T10:26:02.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.498+0000 7f74b6ffd640 1 -- 172.21.15.97:0/3554616542 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f74d0050020 con 0x55afa7cd4b20 2024-04-23T10:26:02.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.498+0000 7f74e0fc4740 10 client.15294 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T10:26:02.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.498+0000 7f74e0fc4740 10 client.15294 resolve_mds: resolved name 'l' to daemon mds.l 2024-04-23T10:26:02.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.498+0000 7f74e0fc4740 20 client.15294 populate_metadata read hostname 'smithi097' 2024-04-23T10:26:02.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.498+0000 7f74e0fc4740 1 --2- 172.21.15.97:0/3554616542 >> [v2:172.21.15.196:6832/3885816565,v1:172.21.15.196:6833/3885816565] conn(0x55afa7cea410 0x55afa7ceb880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.498+0000 7f74e0fc4740 4 client.15294 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:26:02.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.498+0000 7f74e0fc4740 1 -- 172.21.15.97:0/3554616542 --> [v2:172.21.15.196:6832/3885816565,v1:172.21.15.196:6833/3885816565] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55afa7c41670 con 0x55afa7cea410 2024-04-23T10:26:02.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.498+0000 7f74d77fe640 1 --2- 172.21.15.97:0/3554616542 >> [v2:172.21.15.196:6832/3885816565,v1:172.21.15.196:6833/3885816565] conn(0x55afa7cea410 0x55afa7ceb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.499+0000 7f74b6ffd640 10 client.15294 ms_handle_connect on v2:172.21.15.196:6832/3885816565 2024-04-23T10:26:02.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.499+0000 7f74d77fe640 1 --2- 172.21.15.97:0/3554616542 >> [v2:172.21.15.196:6832/3885816565,v1:172.21.15.196:6833/3885816565] conn(0x55afa7cea410 0x55afa7ceb880 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b120f0e80ecf1df1 server_cookie=3c2630f66e190bc0 in_seq=0 out_seq=0 2024-04-23T10:26:02.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.500+0000 7f74b6ffd640 1 -- 172.21.15.97:0/3554616542 <== mds.3 v2:172.21.15.196:6832/3885816565 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55afa7c41670 con 0x55afa7cea410 2024-04-23T10:26:02.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.500+0000 7f74b6ffd640 10 client.15294 handle_command_reply: tid=0 2024-04-23T10:26:02.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.500+0000 7f74e0fc4740 1 client.15294 shutdown 2024-04-23T10:26:02.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.500+0000 7f74e0fc4740 1 -- 172.21.15.97:0/3554616542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afa7cd4b20 msgr2=0x55afa7ce4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:02.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.500+0000 7f74e0fc4740 1 --2- 172.21.15.97:0/3554616542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afa7cd4b20 0x55afa7ce4980 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f74d0035000 tx=0x7f74d00095c0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.500+0000 7f74e0fc4740 1 -- 172.21.15.97:0/3554616542 shutdown_connections 2024-04-23T10:26:02.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.500+0000 7f74e0fc4740 1 --2- 172.21.15.97:0/3554616542 >> [v2:172.21.15.196:6832/3885816565,v1:172.21.15.196:6833/3885816565] conn(0x55afa7cea410 0x55afa7ceb880 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.500+0000 7f74e0fc4740 1 --2- 172.21.15.97:0/3554616542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afa7cd4b20 0x55afa7ce4980 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.500+0000 7f74e0fc4740 1 --2- 172.21.15.97:0/3554616542 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55afa7cd4160 0x55afa7cdd940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.500+0000 7f74b6ffd640 0 client.15294 ms_handle_reset on v2:172.21.15.196:6832/3885816565 2024-04-23T10:26:02.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.500+0000 7f74e0fc4740 1 --2- 172.21.15.97:0/3554616542 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55afa7cd2f90 0x55afa7cdd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.500+0000 7f74e0fc4740 1 -- 172.21.15.97:0/3554616542 >> 172.21.15.97:0/3554616542 conn(0x55afa7cca070 msgr2=0x55afa7cc91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:02.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.500+0000 7f74e0fc4740 1 -- 172.21.15.97:0/3554616542 shutdown_connections 2024-04-23T10:26:02.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.500+0000 7f74e0fc4740 1 -- 172.21.15.97:0/3554616542 wait complete. 2024-04-23T10:26:02.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.500+0000 7f74e0fc4740 20 client.15294 trim_cache size 0 max 16384 2024-04-23T10:26:02.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.518+0000 7f74e0fc4740 1 Processor -- start 2024-04-23T10:26:02.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.518+0000 7f74e0fc4740 1 -- start start 2024-04-23T10:26:02.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.518+0000 7f74e0fc4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55afa7cd2f90 0x55afa7cc50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.518+0000 7f74e0fc4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55afa7cd4160 0x55afa7ce48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.518+0000 7f74e0fc4740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afa7cd4b20 0x55afa7cd7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.518+0000 7f74e0fc4740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55afa7ce9bc0 con 0x55afa7cd4b20 2024-04-23T10:26:02.524 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.518+0000 7f74e0fc4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55afa7cea290 con 0x55afa7cd2f90 2024-04-23T10:26:02.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.518+0000 7f74e0fc4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55afa7cea960 con 0x55afa7cd4160 2024-04-23T10:26:02.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.518+0000 7f74d7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afa7cd4b20 0x55afa7cd7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.518+0000 7f74d7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afa7cd4b20 0x55afa7cd7ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46786/0 (socket says 172.21.15.97:46786) 2024-04-23T10:26:02.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.518+0000 7f74d7fff640 1 -- 172.21.15.97:0/602365672 learned_addr learned my addr 172.21.15.97:0/602365672 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:02.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.518+0000 7f74d77fe640 1 --2- 172.21.15.97:0/602365672 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55afa7cd2f90 0x55afa7cc50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.518+0000 7f74d7fff640 1 -- 172.21.15.97:0/602365672 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55afa7cd4160 msgr2=0x55afa7ce48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:02.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.518+0000 7f74d7fff640 1 --2- 172.21.15.97:0/602365672 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55afa7cd4160 0x55afa7ce48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.518+0000 7f74d7fff640 1 -- 172.21.15.97:0/602365672 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55afa7cd2f90 msgr2=0x55afa7cc50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:02.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.518+0000 7f74d7fff640 1 --2- 172.21.15.97:0/602365672 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55afa7cd2f90 0x55afa7cc50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.518+0000 7f74d7fff640 1 -- 172.21.15.97:0/602365672 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afa7cd3be0 con 0x55afa7cd4b20 2024-04-23T10:26:02.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.518+0000 7f74d77fe640 1 --2- 172.21.15.97:0/602365672 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55afa7cd2f90 0x55afa7cc50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:26:02.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.518+0000 7f74d7fff640 1 --2- 172.21.15.97:0/602365672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afa7cd4b20 0x55afa7cd7ca0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f74d0005e30 tx=0x7f74d0030fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a14816e882b5f08 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:02.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.518+0000 7f74bd7fa640 1 -- 172.21.15.97:0/602365672 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d00305c0 con 0x55afa7cd4b20 2024-04-23T10:26:02.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.518+0000 7f74bd7fa640 1 -- 172.21.15.97:0/602365672 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74d0036050 con 0x55afa7cd4b20 2024-04-23T10:26:02.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.518+0000 7f74bd7fa640 1 -- 172.21.15.97:0/602365672 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d00476d0 con 0x55afa7cd4b20 2024-04-23T10:26:02.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.519+0000 7f74e0fc4740 1 -- 172.21.15.97:0/602365672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afa7cd4b20 msgr2=0x55afa7cd7ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:02.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.519+0000 7f74e0fc4740 1 --2- 172.21.15.97:0/602365672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afa7cd4b20 0x55afa7cd7ca0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f74d0005e30 tx=0x7f74d0030fe0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.519+0000 7f74e0fc4740 1 -- 172.21.15.97:0/602365672 shutdown_connections 2024-04-23T10:26:02.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.519+0000 7f74e0fc4740 1 --2- 172.21.15.97:0/602365672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afa7cd4b20 0x55afa7cd7ca0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.519+0000 7f74e0fc4740 1 --2- 172.21.15.97:0/602365672 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55afa7cd4160 0x55afa7ce48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.519+0000 7f74e0fc4740 1 --2- 172.21.15.97:0/602365672 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55afa7cd2f90 0x55afa7cc50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.519+0000 7f74e0fc4740 1 -- 172.21.15.97:0/602365672 >> 172.21.15.97:0/602365672 conn(0x55afa7cca070 msgr2=0x55afa7cca450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:02.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.519+0000 7f74e0fc4740 1 -- 172.21.15.97:0/602365672 shutdown_connections 2024-04-23T10:26:02.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.519+0000 7f74e0fc4740 1 -- 172.21.15.97:0/602365672 wait complete. 2024-04-23T10:26:02.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.519+0000 7f74e0fc4740 1 -- start start 2024-04-23T10:26:02.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.520+0000 7f74e0fc4740 1 Processor -- start 2024-04-23T10:26:02.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.520+0000 7f74e0fc4740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afa7cd2f90 0x55afa7cda660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.520+0000 7f74e0fc4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55afa7cd4160 0x55afa7ce48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.520+0000 7f74e0fc4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55afa7cd4b20 0x55afa7d4e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.520+0000 7f74e0fc4740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55afa7ceb700 con 0x55afa7cd2f90 2024-04-23T10:26:02.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.520+0000 7f74e0fc4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55afa7ceb030 con 0x55afa7cd4b20 2024-04-23T10:26:02.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.520+0000 7f74e0fc4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55afa7cebdd0 con 0x55afa7cd4160 2024-04-23T10:26:02.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.520+0000 7f74e0fc4740 10 client.0 _ll_register_callbacks cb 0x7f74d0009920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:26:02.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.520+0000 7f74d77fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afa7cd2f90 0x55afa7cda660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.520+0000 7f74d77fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afa7cd2f90 0x55afa7cda660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46798/0 (socket says 172.21.15.97:46798) 2024-04-23T10:26:02.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.520+0000 7f74d77fe640 1 -- 172.21.15.97:0/4064673501 learned_addr learned my addr 172.21.15.97:0/4064673501 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:02.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.520+0000 7f74d6ffd640 1 --2- 172.21.15.97:0/4064673501 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55afa7cd4160 0x55afa7ce48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.520+0000 7f74d77fe640 1 -- 172.21.15.97:0/4064673501 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55afa7cd4160 msgr2=0x55afa7ce48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:02.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.520+0000 7f74d77fe640 1 --2- 172.21.15.97:0/4064673501 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55afa7cd4160 0x55afa7ce48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.520+0000 7f74d77fe640 1 -- 172.21.15.97:0/4064673501 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55afa7cd4b20 msgr2=0x55afa7d4e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:02.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.520+0000 7f74d7fff640 1 --2- 172.21.15.97:0/4064673501 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55afa7cd4b20 0x55afa7d4e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.520+0000 7f74d77fe640 1 --2- 172.21.15.97:0/4064673501 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55afa7cd4b20 0x55afa7d4e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.520+0000 7f74d77fe640 1 -- 172.21.15.97:0/4064673501 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afa7d4ec40 con 0x55afa7cd2f90 2024-04-23T10:26:02.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.520+0000 7f74d6ffd640 1 --2- 172.21.15.97:0/4064673501 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55afa7cd4160 0x55afa7ce48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:26:02.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.520+0000 7f74d7fff640 1 --2- 172.21.15.97:0/4064673501 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55afa7cd4b20 0x55afa7d4e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:26:02.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.520+0000 7f74d77fe640 1 --2- 172.21.15.97:0/4064673501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afa7cd2f90 0x55afa7cda660 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f74c809c3b0 tx=0x7f74c8098040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb35e159cfe7514d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:02.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.520+0000 7f74b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:26:02.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.520+0000 7f74b6ffd640 1 -- 172.21.15.97:0/4064673501 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c801d040 con 0x55afa7cd2f90 2024-04-23T10:26:02.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.520+0000 7f74b6ffd640 1 -- 172.21.15.97:0/4064673501 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f74c8007ca0 con 0x55afa7cd2f90 2024-04-23T10:26:02.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.520+0000 7f74e0fc4740 1 -- 172.21.15.97:0/4064673501 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74d0036050 con 0x55afa7cd2f90 2024-04-23T10:26:02.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.521+0000 7f74b6ffd640 1 -- 172.21.15.97:0/4064673501 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f74c8005290 con 0x55afa7cd2f90 2024-04-23T10:26:02.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.521+0000 7f74e0fc4740 10 client.15300 fetch_fsmap learned FSMap version 25 2024-04-23T10:26:02.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.521+0000 7f74e0fc4740 1 -- 172.21.15.97:0/4064673501 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55afa7d4ff20 con 0x55afa7cd2f90 2024-04-23T10:26:02.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.521+0000 7f74b6ffd640 1 -- 172.21.15.97:0/4064673501 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74d0036050 con 0x55afa7cd2f90 2024-04-23T10:26:02.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.521+0000 7f74b6ffd640 1 -- 172.21.15.97:0/4064673501 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f74c80d20d0 con 0x55afa7cd2f90 2024-04-23T10:26:02.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.521+0000 7f74e0fc4740 10 client.15300 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T10:26:02.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.521+0000 7f74e0fc4740 10 client.15300 resolve_mds: resolved name 'l' to daemon mds.l 2024-04-23T10:26:02.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.521+0000 7f74e0fc4740 20 client.15300 populate_metadata read hostname 'smithi097' 2024-04-23T10:26:02.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.521+0000 7f74e0fc4740 1 --2- 172.21.15.97:0/4064673501 >> [v2:172.21.15.196:6832/3885816565,v1:172.21.15.196:6833/3885816565] conn(0x55afa7d54b30 0x55afa7d53840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.521+0000 7f74e0fc4740 4 client.15300 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T10:26:02.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.521+0000 7f74e0fc4740 1 -- 172.21.15.97:0/4064673501 --> [v2:172.21.15.196:6832/3885816565,v1:172.21.15.196:6833/3885816565] -- command(tid 0: {"prefix": "status"}) -- 0x55afa7c3b4e0 con 0x55afa7d54b30 2024-04-23T10:26:02.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.521+0000 7f74d6ffd640 1 --2- 172.21.15.97:0/4064673501 >> [v2:172.21.15.196:6832/3885816565,v1:172.21.15.196:6833/3885816565] conn(0x55afa7d54b30 0x55afa7d53840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.522+0000 7f74d6ffd640 1 --2- 172.21.15.97:0/4064673501 >> [v2:172.21.15.196:6832/3885816565,v1:172.21.15.196:6833/3885816565] conn(0x55afa7d54b30 0x55afa7d53840 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c8629c39e27634cd server_cookie=340c80bbf17e2308 in_seq=0 out_seq=0 2024-04-23T10:26:02.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.522+0000 7f74b6ffd640 10 client.15300 ms_handle_connect on v2:172.21.15.196:6832/3885816565 2024-04-23T10:26:02.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.522+0000 7f74b6ffd640 1 -- 172.21.15.97:0/4064673501 <== mds.3 v2:172.21.15.196:6832/3885816565 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55afa7c3b4e0 con 0x55afa7d54b30 2024-04-23T10:26:02.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.522+0000 7f74b6ffd640 10 client.15300 handle_command_reply: tid=0 2024-04-23T10:26:02.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.522+0000 7f74e0fc4740 1 client.15300 shutdown 2024-04-23T10:26:02.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.522+0000 7f74e0fc4740 1 -- 172.21.15.97:0/4064673501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afa7cd2f90 msgr2=0x55afa7cda660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:02.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.522+0000 7f74e0fc4740 1 --2- 172.21.15.97:0/4064673501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afa7cd2f90 0x55afa7cda660 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f74c809c3b0 tx=0x7f74c8098040 comp rx=0 tx=0).stop 2024-04-23T10:26:02.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.522+0000 7f74e0fc4740 1 -- 172.21.15.97:0/4064673501 shutdown_connections 2024-04-23T10:26:02.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.522+0000 7f74e0fc4740 1 --2- 172.21.15.97:0/4064673501 >> [v2:172.21.15.196:6832/3885816565,v1:172.21.15.196:6833/3885816565] conn(0x55afa7d54b30 0x55afa7d53840 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.522+0000 7f74e0fc4740 1 --2- 172.21.15.97:0/4064673501 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55afa7cd4b20 0x55afa7d4e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.522+0000 7f74e0fc4740 1 --2- 172.21.15.97:0/4064673501 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55afa7cd4160 0x55afa7ce48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.522+0000 7f74b6ffd640 0 client.15300 ms_handle_reset on v2:172.21.15.196:6832/3885816565 2024-04-23T10:26:02.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.522+0000 7f74e0fc4740 1 --2- 172.21.15.97:0/4064673501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55afa7cd2f90 0x55afa7cda660 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.522+0000 7f74e0fc4740 1 -- 172.21.15.97:0/4064673501 >> 172.21.15.97:0/4064673501 conn(0x55afa7cca070 msgr2=0x55afa7cdae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:02.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.522+0000 7f74e0fc4740 1 -- 172.21.15.97:0/4064673501 shutdown_connections 2024-04-23T10:26:02.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.522+0000 7f74e0fc4740 1 -- 172.21.15.97:0/4064673501 wait complete. 2024-04-23T10:26:02.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.522+0000 7f74e0fc4740 20 client.15300 trim_cache size 0 max 16384 2024-04-23T10:26:02.532 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-23T10:26:02.532 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "9b22af88-0159-11ef-bc93-c7b262605968", 2024-04-23T10:26:02.532 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 3, 2024-04-23T10:26:02.532 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 24403, 2024-04-23T10:26:02.532 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-04-23T10:26:02.532 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-04-23T10:26:02.532 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-04-23T10:26:02.532 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 111.569757354, 2024-04-23T10:26:02.532 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 25, 2024-04-23T10:26:02.532 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 88, 2024-04-23T10:26:02.533 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 86, 2024-04-23T10:26:02.533 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 153.08428520000001 2024-04-23T10:26:02.533 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-23T10:26:02.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.523+0000 7f74de1cd640 1 -- 172.21.15.97:0/4070196668 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f74c00a3bb0 msgr2=0x7f74c00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:02.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.523+0000 7f74de1cd640 1 --2- 172.21.15.97:0/4070196668 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f74c00a3bb0 0x7f74c00a6070 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f74d0031b50 tx=0x7f74d0040040 comp rx=0 tx=0).stop 2024-04-23T10:26:02.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.523+0000 7f74de1cd640 1 -- 172.21.15.97:0/4070196668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74d8102a30 msgr2=0x7f74d8198180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:02.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.523+0000 7f74de1cd640 1 --2- 172.21.15.97:0/4070196668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74d8102a30 0x7f74d8198180 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f74c800c890 tx=0x7f74c800cd50 comp rx=0 tx=0).stop 2024-04-23T10:26:02.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.523+0000 7f74de1cd640 1 -- 172.21.15.97:0/4070196668 shutdown_connections 2024-04-23T10:26:02.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.523+0000 7f74de1cd640 1 --2- 172.21.15.97:0/4070196668 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f74c00a3bb0 0x7f74c00a6070 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.523+0000 7f74de1cd640 1 --2- 172.21.15.97:0/4070196668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f74d8107c90 0x7f74d819f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.523+0000 7f74de1cd640 1 --2- 172.21.15.97:0/4070196668 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74d8105360 0x7f74d81986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.523+0000 7f74de1cd640 1 --2- 172.21.15.97:0/4070196668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74d8102a30 0x7f74d8198180 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.523+0000 7f74de1cd640 1 -- 172.21.15.97:0/4070196668 >> 172.21.15.97:0/4070196668 conn(0x7f74d80fc440 msgr2=0x7f74d8103390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:02.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.523+0000 7f74de1cd640 1 -- 172.21.15.97:0/4070196668 shutdown_connections 2024-04-23T10:26:02.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.523+0000 7f74de1cd640 1 -- 172.21.15.97:0/4070196668 wait complete. 2024-04-23T10:26:02.535 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-23T10:26:02.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.635+0000 7fdbb74e1640 1 -- 172.21.15.97:0/1563829421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbb0102a30 msgr2=0x7fdbb0104e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:02.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.635+0000 7fdbb74e1640 1 --2- 172.21.15.97:0/1563829421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbb0102a30 0x7fdbb0104e20 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fdbac009900 tx=0x7fdbac02a410 comp rx=0 tx=0).stop 2024-04-23T10:26:02.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.635+0000 7fdbb74e1640 1 -- 172.21.15.97:0/1563829421 shutdown_connections 2024-04-23T10:26:02.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.635+0000 7fdbb74e1640 1 --2- 172.21.15.97:0/1563829421 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdbb0107c90 0x7fdbb010a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.635+0000 7fdbb74e1640 1 --2- 172.21.15.97:0/1563829421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbb0105360 0x7fdbb0107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.635+0000 7fdbb74e1640 1 --2- 172.21.15.97:0/1563829421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbb0102a30 0x7fdbb0104e20 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.635+0000 7fdbb74e1640 1 -- 172.21.15.97:0/1563829421 >> 172.21.15.97:0/1563829421 conn(0x7fdbb00fc440 msgr2=0x7fdbb00fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:02.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.635+0000 7fdbb74e1640 1 -- 172.21.15.97:0/1563829421 shutdown_connections 2024-04-23T10:26:02.637 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.635+0000 7fdbb74e1640 1 -- 172.21.15.97:0/1563829421 wait complete. 2024-04-23T10:26:02.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.636+0000 7fdbb74e1640 1 Processor -- start 2024-04-23T10:26:02.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.636+0000 7fdbb74e1640 1 -- start start 2024-04-23T10:26:02.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.636+0000 7fdbb74e1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbb0102a30 0x7fdbb0198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.636+0000 7fdbb74e1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbb0105360 0x7fdbb0198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.636+0000 7fdbb74e1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdbb0107c90 0x7fdbb019f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.636+0000 7fdbb74e1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb010dbd0 con 0x7fdbb0105360 2024-04-23T10:26:02.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.636+0000 7fdbb74e1640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb010da50 con 0x7fdbb0107c90 2024-04-23T10:26:02.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.636+0000 7fdbb74e1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb010dd50 con 0x7fdbb0102a30 2024-04-23T10:26:02.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.636+0000 7fdbb4a7c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbb0105360 0x7fdbb0198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.636+0000 7fdbb4a7c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbb0105360 0x7fdbb0198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46810/0 (socket says 172.21.15.97:46810) 2024-04-23T10:26:02.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.636+0000 7fdbb4a7c640 1 -- 172.21.15.97:0/2675681518 learned_addr learned my addr 172.21.15.97:0/2675681518 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:02.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.636+0000 7fdbb527d640 1 --2- 172.21.15.97:0/2675681518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbb0102a30 0x7fdbb0198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.636+0000 7fdbb5a7e640 1 --2- 172.21.15.97:0/2675681518 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdbb0107c90 0x7fdbb019f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.636+0000 7fdbb4a7c640 1 -- 172.21.15.97:0/2675681518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbb0102a30 msgr2=0x7fdbb0198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:02.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.636+0000 7fdbb4a7c640 1 --2- 172.21.15.97:0/2675681518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbb0102a30 0x7fdbb0198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.636+0000 7fdbb4a7c640 1 -- 172.21.15.97:0/2675681518 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdbb0107c90 msgr2=0x7fdbb019f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:02.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.636+0000 7fdbb4a7c640 1 --2- 172.21.15.97:0/2675681518 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdbb0107c90 0x7fdbb019f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.636+0000 7fdbb4a7c640 1 -- 172.21.15.97:0/2675681518 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbb019ff50 con 0x7fdbb0105360 2024-04-23T10:26:02.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.636+0000 7fdbb5a7e640 1 --2- 172.21.15.97:0/2675681518 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdbb0107c90 0x7fdbb019f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:26:02.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.637+0000 7fdbb4a7c640 1 --2- 172.21.15.97:0/2675681518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbb0105360 0x7fdbb0198870 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fdba000b450 tx=0x7fdba000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:02.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.637+0000 7fdba67fc640 1 -- 172.21.15.97:0/2675681518 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba0015020 con 0x7fdbb0105360 2024-04-23T10:26:02.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.637+0000 7fdba67fc640 1 -- 172.21.15.97:0/2675681518 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdba0011d40 con 0x7fdbb0105360 2024-04-23T10:26:02.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.637+0000 7fdbb74e1640 1 -- 172.21.15.97:0/2675681518 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbb01a0240 con 0x7fdbb0105360 2024-04-23T10:26:02.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.637+0000 7fdba67fc640 1 -- 172.21.15.97:0/2675681518 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba000f340 con 0x7fdbb0105360 2024-04-23T10:26:02.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.637+0000 7fdbb74e1640 1 -- 172.21.15.97:0/2675681518 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbb01a07c0 con 0x7fdbb0105360 2024-04-23T10:26:02.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.639+0000 7fdba67fc640 1 -- 172.21.15.97:0/2675681518 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fdba0018020 con 0x7fdbb0105360 2024-04-23T10:26:02.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.639+0000 7fdbba1f7740 1 Processor -- start 2024-04-23T10:26:02.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.639+0000 7fdbba1f7740 1 -- start start 2024-04-23T10:26:02.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.639+0000 7fdbba1f7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f7405eb80 0x555f7405efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.639+0000 7fdbba1f7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f7405fd50 0x555f740601d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.639+0000 7fdbba1f7740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555f74060710 0x555f74066fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.639+0000 7fdbba1f7740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x555f73fc3c50 con 0x555f74060710 2024-04-23T10:26:02.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.639+0000 7fdbba1f7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555f73f60830 con 0x555f7405fd50 2024-04-23T10:26:02.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.639+0000 7fdbba1f7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555f73fbfc10 con 0x555f7405eb80 2024-04-23T10:26:02.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.639+0000 7fdbb4a7c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555f74060710 0x555f74066fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.640+0000 7fdbb4a7c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555f74060710 0x555f74066fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46822/0 (socket says 172.21.15.97:46822) 2024-04-23T10:26:02.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.640+0000 7fdbb4a7c640 1 -- 172.21.15.97:0/2361476107 learned_addr learned my addr 172.21.15.97:0/2361476107 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:02.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.640+0000 7fdbb5a7e640 1 --2- 172.21.15.97:0/2361476107 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f7405eb80 0x555f7405efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.640+0000 7fdba67fc640 1 --2- 172.21.15.97:0/2675681518 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fdb9c0a3bb0 0x7fdb9c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.640+0000 7fdbb527d640 1 --2- 172.21.15.97:0/2361476107 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f7405fd50 0x555f740601d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.640+0000 7fdbb4a7c640 1 -- 172.21.15.97:0/2361476107 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f7405eb80 msgr2=0x555f7405efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:02.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.640+0000 7fdbb5a7e640 1 --2- 172.21.15.97:0/2675681518 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fdb9c0a3bb0 0x7fdb9c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.640+0000 7fdbb4a7c640 1 --2- 172.21.15.97:0/2361476107 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f7405eb80 0x555f7405efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.640+0000 7fdbb4a7c640 1 -- 172.21.15.97:0/2361476107 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f7405fd50 msgr2=0x555f740601d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:02.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.640+0000 7fdbb4a7c640 1 --2- 172.21.15.97:0/2361476107 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f7405fd50 0x555f740601d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.640+0000 7fdbb4a7c640 1 -- 172.21.15.97:0/2361476107 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f73cd62d0 con 0x555f74060710 2024-04-23T10:26:02.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.640+0000 7fdba67fc640 1 -- 172.21.15.97:0/2675681518 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fdba00cff30 con 0x7fdbb0105360 2024-04-23T10:26:02.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.640+0000 7fdbb4a7c640 1 --2- 172.21.15.97:0/2361476107 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555f74060710 0x555f74066fa0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fdba005f080 tx=0x7fdba0086dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d166ea7831c6477 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:02.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.640+0000 7fdbb5a7e640 1 --2- 172.21.15.97:0/2675681518 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fdb9c0a3bb0 0x7fdb9c0a6070 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x555f7405fc50 tx=0x7fdba8004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:02.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.640+0000 7fdb9b7fe640 1 -- 172.21.15.97:0/2361476107 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba009c010 con 0x555f74060710 2024-04-23T10:26:02.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.640+0000 7fdb9b7fe640 1 -- 172.21.15.97:0/2361476107 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdba005f650 con 0x555f74060710 2024-04-23T10:26:02.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.640+0000 7fdb9b7fe640 1 -- 172.21.15.97:0/2361476107 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba0009280 con 0x555f74060710 2024-04-23T10:26:02.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.641+0000 7fdbba1f7740 1 -- 172.21.15.97:0/2361476107 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555f74060710 msgr2=0x555f74066fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:02.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.641+0000 7fdbba1f7740 1 --2- 172.21.15.97:0/2361476107 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555f74060710 0x555f74066fa0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fdba005f080 tx=0x7fdba0086dc0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.641+0000 7fdbba1f7740 1 -- 172.21.15.97:0/2361476107 shutdown_connections 2024-04-23T10:26:02.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.641+0000 7fdbba1f7740 1 --2- 172.21.15.97:0/2361476107 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555f74060710 0x555f74066fa0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.641+0000 7fdbba1f7740 1 --2- 172.21.15.97:0/2361476107 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f7405fd50 0x555f740601d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.641+0000 7fdbba1f7740 1 --2- 172.21.15.97:0/2361476107 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f7405eb80 0x555f7405efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.641+0000 7fdbba1f7740 1 -- 172.21.15.97:0/2361476107 >> 172.21.15.97:0/2361476107 conn(0x555f7405a070 msgr2=0x555f7405a470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:02.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.641+0000 7fdbba1f7740 1 -- 172.21.15.97:0/2361476107 shutdown_connections 2024-04-23T10:26:02.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.641+0000 7fdbba1f7740 1 -- 172.21.15.97:0/2361476107 wait complete. 2024-04-23T10:26:02.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.641+0000 7fdbba1f7740 1 -- start start 2024-04-23T10:26:02.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.641+0000 7fdbba1f7740 1 Processor -- start 2024-04-23T10:26:02.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.642+0000 7fdbba1f7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f7405eb80 0x555f74068f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.642+0000 7fdbba1f7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f7405fd50 0x555f74069490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.642+0000 7fdbba1f7740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555f74060710 0x555f740704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.642+0000 7fdbba1f7740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x555f740676c0 con 0x555f74060710 2024-04-23T10:26:02.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.642+0000 7fdbba1f7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555f73f5e470 con 0x555f7405eb80 2024-04-23T10:26:02.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.642+0000 7fdbba1f7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555f74067840 con 0x555f7405fd50 2024-04-23T10:26:02.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.642+0000 7fdbba1f7740 10 client.0 _ll_register_callbacks cb 0x555f73f83f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:26:02.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.642+0000 7fdbb5a7e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555f74060710 0x555f740704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.642+0000 7fdbb5a7e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555f74060710 0x555f740704d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46828/0 (socket says 172.21.15.97:46828) 2024-04-23T10:26:02.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.642+0000 7fdbb5a7e640 1 -- 172.21.15.97:0/2754132335 learned_addr learned my addr 172.21.15.97:0/2754132335 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:02.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.642+0000 7fdbb527d640 1 --2- 172.21.15.97:0/2754132335 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f7405eb80 0x555f74068f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.642+0000 7fdbb4a7c640 1 --2- 172.21.15.97:0/2754132335 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f7405fd50 0x555f74069490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.642+0000 7fdbb5a7e640 1 -- 172.21.15.97:0/2754132335 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f7405fd50 msgr2=0x555f74069490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:02.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.642+0000 7fdbb5a7e640 1 --2- 172.21.15.97:0/2754132335 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f7405fd50 0x555f74069490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.642+0000 7fdbb5a7e640 1 -- 172.21.15.97:0/2754132335 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f7405eb80 msgr2=0x555f74068f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:02.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.642+0000 7fdbb5a7e640 1 --2- 172.21.15.97:0/2754132335 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f7405eb80 0x555f74068f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.642+0000 7fdbb5a7e640 1 -- 172.21.15.97:0/2754132335 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f73da06b0 con 0x555f74060710 2024-04-23T10:26:02.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.642+0000 7fdbb4a7c640 1 --2- 172.21.15.97:0/2754132335 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f7405fd50 0x555f74069490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:26:02.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.642+0000 7fdbb5a7e640 1 --2- 172.21.15.97:0/2754132335 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555f74060710 0x555f740704d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fdba8007870 tx=0x7fdba80075f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28448646b485907b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:02.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.642+0000 7fdb98ff9640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:26:02.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.642+0000 7fdb98ff9640 1 -- 172.21.15.97:0/2754132335 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba801f450 con 0x555f74060710 2024-04-23T10:26:02.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.642+0000 7fdb98ff9640 1 -- 172.21.15.97:0/2754132335 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fdba801f5f0 con 0x555f74060710 2024-04-23T10:26:02.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.642+0000 7fdbba1f7740 1 -- 172.21.15.97:0/2754132335 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555f73fd1670 con 0x555f74060710 2024-04-23T10:26:02.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.643+0000 7fdb98ff9640 1 -- 172.21.15.97:0/2754132335 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdba801f7f0 con 0x555f74060710 2024-04-23T10:26:02.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.643+0000 7fdbba1f7740 10 client.15312 fetch_fsmap learned FSMap version 25 2024-04-23T10:26:02.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.643+0000 7fdbba1f7740 1 -- 172.21.15.97:0/2754132335 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555f740738b0 con 0x555f74060710 2024-04-23T10:26:02.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.643+0000 7fdb98ff9640 1 -- 172.21.15.97:0/2754132335 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x555f73fd1670 con 0x555f74060710 2024-04-23T10:26:02.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.643+0000 7fdb98ff9640 1 -- 172.21.15.97:0/2754132335 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fdba8028b40 con 0x555f74060710 2024-04-23T10:26:02.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.643+0000 7fdbba1f7740 10 client.15312 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T10:26:02.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.643+0000 7fdbba1f7740 10 client.15312 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-23T10:26:02.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.643+0000 7fdbba1f7740 20 client.15312 populate_metadata read hostname 'smithi097' 2024-04-23T10:26:02.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.643+0000 7fdbba1f7740 1 --2- 172.21.15.97:0/2754132335 >> [v2:172.21.15.196:6836/3094946502,v1:172.21.15.196:6837/3094946502] conn(0x555f74075f90 0x555f74077400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.643+0000 7fdbba1f7740 4 client.15312 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:26:02.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.643+0000 7fdbba1f7740 1 -- 172.21.15.97:0/2754132335 --> [v2:172.21.15.196:6836/3094946502,v1:172.21.15.196:6837/3094946502] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f73fcb4e0 con 0x555f74075f90 2024-04-23T10:26:02.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.643+0000 7fdbb527d640 1 --2- 172.21.15.97:0/2754132335 >> [v2:172.21.15.196:6836/3094946502,v1:172.21.15.196:6837/3094946502] conn(0x555f74075f90 0x555f74077400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.644+0000 7fdb98ff9640 10 client.15312 ms_handle_connect on v2:172.21.15.196:6836/3094946502 2024-04-23T10:26:02.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.644+0000 7fdbb527d640 1 --2- 172.21.15.97:0/2754132335 >> [v2:172.21.15.196:6836/3094946502,v1:172.21.15.196:6837/3094946502] conn(0x555f74075f90 0x555f74077400 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e087569e28759c1e server_cookie=a314e84dd6ed835f in_seq=0 out_seq=0 2024-04-23T10:26:02.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.644+0000 7fdb98ff9640 1 -- 172.21.15.97:0/2754132335 <== mds.2 v2:172.21.15.196:6836/3094946502 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555f73fcb4e0 con 0x555f74075f90 2024-04-23T10:26:02.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.644+0000 7fdb98ff9640 10 client.15312 handle_command_reply: tid=0 2024-04-23T10:26:02.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.644+0000 7fdbba1f7740 1 client.15312 shutdown 2024-04-23T10:26:02.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.645+0000 7fdbba1f7740 1 -- 172.21.15.97:0/2754132335 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555f74060710 msgr2=0x555f740704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:02.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.645+0000 7fdbba1f7740 1 --2- 172.21.15.97:0/2754132335 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555f74060710 0x555f740704d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fdba8007870 tx=0x7fdba80075f0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.645+0000 7fdbba1f7740 1 -- 172.21.15.97:0/2754132335 shutdown_connections 2024-04-23T10:26:02.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.645+0000 7fdbba1f7740 1 --2- 172.21.15.97:0/2754132335 >> [v2:172.21.15.196:6836/3094946502,v1:172.21.15.196:6837/3094946502] conn(0x555f74075f90 0x555f74077400 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.645+0000 7fdbba1f7740 1 --2- 172.21.15.97:0/2754132335 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555f74060710 0x555f740704d0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.645+0000 7fdbba1f7740 1 --2- 172.21.15.97:0/2754132335 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f7405fd50 0x555f74069490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.645+0000 7fdb98ff9640 0 client.15312 ms_handle_reset on v2:172.21.15.196:6836/3094946502 2024-04-23T10:26:02.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.645+0000 7fdbba1f7740 1 --2- 172.21.15.97:0/2754132335 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f7405eb80 0x555f74068f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.645+0000 7fdbba1f7740 1 -- 172.21.15.97:0/2754132335 >> 172.21.15.97:0/2754132335 conn(0x555f7405a070 msgr2=0x555f740591d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:02.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.645+0000 7fdbba1f7740 1 -- 172.21.15.97:0/2754132335 shutdown_connections 2024-04-23T10:26:02.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.645+0000 7fdbba1f7740 1 -- 172.21.15.97:0/2754132335 wait complete. 2024-04-23T10:26:02.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.645+0000 7fdbba1f7740 20 client.15312 trim_cache size 0 max 16384 2024-04-23T10:26:02.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.663+0000 7fdbba1f7740 1 Processor -- start 2024-04-23T10:26:02.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.663+0000 7fdbba1f7740 1 -- start start 2024-04-23T10:26:02.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.663+0000 7fdbba1f7740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555f7405eb80 0x555f74068f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.663+0000 7fdbba1f7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f7405fd50 0x555f74069490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.663+0000 7fdbba1f7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f74060710 0x555f740703f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.663+0000 7fdbba1f7740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x555f74075740 con 0x555f7405eb80 2024-04-23T10:26:02.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.663+0000 7fdbba1f7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555f74075e10 con 0x555f74060710 2024-04-23T10:26:02.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.663+0000 7fdbba1f7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555f740764e0 con 0x555f7405fd50 2024-04-23T10:26:02.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.663+0000 7fdbb527d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555f7405eb80 0x555f74068f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.663+0000 7fdbb527d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555f7405eb80 0x555f74068f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46834/0 (socket says 172.21.15.97:46834) 2024-04-23T10:26:02.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.663+0000 7fdbb527d640 1 -- 172.21.15.97:0/570691703 learned_addr learned my addr 172.21.15.97:0/570691703 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:02.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.663+0000 7fdbb4a7c640 1 --2- 172.21.15.97:0/570691703 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f7405fd50 0x555f74069490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.663+0000 7fdbb527d640 1 -- 172.21.15.97:0/570691703 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f7405fd50 msgr2=0x555f74069490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:02.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.663+0000 7fdbb527d640 1 --2- 172.21.15.97:0/570691703 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f7405fd50 0x555f74069490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.663+0000 7fdbb527d640 1 -- 172.21.15.97:0/570691703 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f74060710 msgr2=0x555f740703f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:02.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.663+0000 7fdbb527d640 1 --2- 172.21.15.97:0/570691703 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f74060710 0x555f740703f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.663+0000 7fdbb527d640 1 -- 172.21.15.97:0/570691703 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f7405f7d0 con 0x555f7405eb80 2024-04-23T10:26:02.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.663+0000 7fdbb4a7c640 1 --2- 172.21.15.97:0/570691703 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f7405fd50 0x555f74069490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:26:02.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.663+0000 7fdbb527d640 1 --2- 172.21.15.97:0/570691703 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555f7405eb80 0x555f74068f50 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fdbac009900 tx=0x7fdbac005df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e01e795860fc854 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:02.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.663+0000 7fdb9affd640 1 -- 172.21.15.97:0/570691703 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbac048070 con 0x555f7405eb80 2024-04-23T10:26:02.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.663+0000 7fdb9affd640 1 -- 172.21.15.97:0/570691703 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbac031890 con 0x555f7405eb80 2024-04-23T10:26:02.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.663+0000 7fdb9affd640 1 -- 172.21.15.97:0/570691703 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbac035040 con 0x555f7405eb80 2024-04-23T10:26:02.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.664+0000 7fdbba1f7740 1 -- 172.21.15.97:0/570691703 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555f7405eb80 msgr2=0x555f74068f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:02.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.664+0000 7fdbba1f7740 1 --2- 172.21.15.97:0/570691703 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555f7405eb80 0x555f74068f50 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fdbac009900 tx=0x7fdbac005df0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.664+0000 7fdbba1f7740 1 -- 172.21.15.97:0/570691703 shutdown_connections 2024-04-23T10:26:02.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.664+0000 7fdbba1f7740 1 --2- 172.21.15.97:0/570691703 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f74060710 0x555f740703f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.664+0000 7fdbba1f7740 1 --2- 172.21.15.97:0/570691703 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f7405fd50 0x555f74069490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.664+0000 7fdbba1f7740 1 --2- 172.21.15.97:0/570691703 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555f7405eb80 0x555f74068f50 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.664+0000 7fdbba1f7740 1 -- 172.21.15.97:0/570691703 >> 172.21.15.97:0/570691703 conn(0x555f7405a070 msgr2=0x555f74054e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:02.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.664+0000 7fdbba1f7740 1 -- 172.21.15.97:0/570691703 shutdown_connections 2024-04-23T10:26:02.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.664+0000 7fdbba1f7740 1 -- 172.21.15.97:0/570691703 wait complete. 2024-04-23T10:26:02.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.664+0000 7fdbba1f7740 1 -- start start 2024-04-23T10:26:02.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.664+0000 7fdbba1f7740 1 Processor -- start 2024-04-23T10:26:02.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.665+0000 7fdbba1f7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f7405eb80 0x555f7406b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.665+0000 7fdbba1f7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f7405fd50 0x555f7406bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.665+0000 7fdbba1f7740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555f74060710 0x555f7406ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.665+0000 7fdbba1f7740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x555f74077280 con 0x555f74060710 2024-04-23T10:26:02.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.665+0000 7fdbba1f7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555f74076bb0 con 0x555f7405fd50 2024-04-23T10:26:02.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.665+0000 7fdbba1f7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555f74077950 con 0x555f7405eb80 2024-04-23T10:26:02.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.665+0000 7fdbba1f7740 10 client.0 _ll_register_callbacks cb 0x7fdba8007480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:26:02.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.665+0000 7fdbb5a7e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555f74060710 0x555f7406ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.665+0000 7fdbb5a7e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555f74060710 0x555f7406ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46842/0 (socket says 172.21.15.97:46842) 2024-04-23T10:26:02.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.665+0000 7fdbb5a7e640 1 -- 172.21.15.97:0/1305963345 learned_addr learned my addr 172.21.15.97:0/1305963345 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:02.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.665+0000 7fdbb527d640 1 --2- 172.21.15.97:0/1305963345 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f7405eb80 0x555f7406b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.665+0000 7fdbb4a7c640 1 --2- 172.21.15.97:0/1305963345 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f7405fd50 0x555f7406bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.665+0000 7fdbb5a7e640 1 -- 172.21.15.97:0/1305963345 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f7405eb80 msgr2=0x555f7406b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:02.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.665+0000 7fdbb5a7e640 1 --2- 172.21.15.97:0/1305963345 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f7405eb80 0x555f7406b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.665+0000 7fdbb5a7e640 1 -- 172.21.15.97:0/1305963345 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f7405fd50 msgr2=0x555f7406bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:02.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.665+0000 7fdbb5a7e640 1 --2- 172.21.15.97:0/1305963345 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f7405fd50 0x555f7406bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.665+0000 7fdbb5a7e640 1 -- 172.21.15.97:0/1305963345 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f7406f400 con 0x555f74060710 2024-04-23T10:26:02.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.665+0000 7fdbb4a7c640 1 --2- 172.21.15.97:0/1305963345 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f7405fd50 0x555f7406bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:26:02.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.665+0000 7fdbb5a7e640 1 --2- 172.21.15.97:0/1305963345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555f74060710 0x555f7406ee10 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fdba80115a0 tx=0x7fdba801e7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b0156657d994731 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:02.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.665+0000 7fdb98ff9640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:26:02.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.665+0000 7fdb98ff9640 1 -- 172.21.15.97:0/1305963345 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba8012c90 con 0x555f74060710 2024-04-23T10:26:02.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.665+0000 7fdb98ff9640 1 -- 172.21.15.97:0/1305963345 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fdba801fba0 con 0x555f74060710 2024-04-23T10:26:02.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.665+0000 7fdbba1f7740 1 -- 172.21.15.97:0/1305963345 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbac031890 con 0x555f74060710 2024-04-23T10:26:02.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.666+0000 7fdb98ff9640 1 -- 172.21.15.97:0/1305963345 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdba801fda0 con 0x555f74060710 2024-04-23T10:26:02.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.666+0000 7fdbba1f7740 10 client.15318 fetch_fsmap learned FSMap version 25 2024-04-23T10:26:02.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.666+0000 7fdbba1f7740 1 -- 172.21.15.97:0/1305963345 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555f74070700 con 0x555f74060710 2024-04-23T10:26:02.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.666+0000 7fdb98ff9640 1 -- 172.21.15.97:0/1305963345 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbac031890 con 0x555f74060710 2024-04-23T10:26:02.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.666+0000 7fdb98ff9640 1 -- 172.21.15.97:0/1305963345 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fdba8028600 con 0x555f74060710 2024-04-23T10:26:02.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.666+0000 7fdbba1f7740 10 client.15318 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T10:26:02.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.666+0000 7fdbba1f7740 10 client.15318 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-23T10:26:02.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.666+0000 7fdbba1f7740 20 client.15318 populate_metadata read hostname 'smithi097' 2024-04-23T10:26:02.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.666+0000 7fdbba1f7740 1 --2- 172.21.15.97:0/1305963345 >> [v2:172.21.15.196:6836/3094946502,v1:172.21.15.196:6837/3094946502] conn(0x555f740e0830 0x555f740df540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.666+0000 7fdbba1f7740 4 client.15318 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T10:26:02.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.666+0000 7fdbba1f7740 1 -- 172.21.15.97:0/1305963345 --> [v2:172.21.15.196:6836/3094946502,v1:172.21.15.196:6837/3094946502] -- command(tid 0: {"prefix": "status"}) -- 0x555f73f479a0 con 0x555f740e0830 2024-04-23T10:26:02.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.666+0000 7fdbb527d640 1 --2- 172.21.15.97:0/1305963345 >> [v2:172.21.15.196:6836/3094946502,v1:172.21.15.196:6837/3094946502] conn(0x555f740e0830 0x555f740df540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.667+0000 7fdb98ff9640 10 client.15318 ms_handle_connect on v2:172.21.15.196:6836/3094946502 2024-04-23T10:26:02.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.667+0000 7fdbb527d640 1 --2- 172.21.15.97:0/1305963345 >> [v2:172.21.15.196:6836/3094946502,v1:172.21.15.196:6837/3094946502] conn(0x555f740e0830 0x555f740df540 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d0a8c0ea2b82e829 server_cookie=a64be16b664ba1b2 in_seq=0 out_seq=0 2024-04-23T10:26:02.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.667+0000 7fdb98ff9640 1 -- 172.21.15.97:0/1305963345 <== mds.2 v2:172.21.15.196:6836/3094946502 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x555f73f479a0 con 0x555f740e0830 2024-04-23T10:26:02.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.667+0000 7fdb98ff9640 10 client.15318 handle_command_reply: tid=0 2024-04-23T10:26:02.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.667+0000 7fdbba1f7740 1 client.15318 shutdown 2024-04-23T10:26:02.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.667+0000 7fdbba1f7740 1 -- 172.21.15.97:0/1305963345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555f74060710 msgr2=0x555f7406ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:02.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.667+0000 7fdbba1f7740 1 --2- 172.21.15.97:0/1305963345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555f74060710 0x555f7406ee10 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fdba80115a0 tx=0x7fdba801e7f0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.667+0000 7fdbba1f7740 1 -- 172.21.15.97:0/1305963345 shutdown_connections 2024-04-23T10:26:02.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.667+0000 7fdbba1f7740 1 --2- 172.21.15.97:0/1305963345 >> [v2:172.21.15.196:6836/3094946502,v1:172.21.15.196:6837/3094946502] conn(0x555f740e0830 0x555f740df540 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.667+0000 7fdbba1f7740 1 --2- 172.21.15.97:0/1305963345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555f74060710 0x555f7406ee10 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.667+0000 7fdbba1f7740 1 --2- 172.21.15.97:0/1305963345 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f7405fd50 0x555f7406bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.667+0000 7fdb98ff9640 0 client.15318 ms_handle_reset on v2:172.21.15.196:6836/3094946502 2024-04-23T10:26:02.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.667+0000 7fdbba1f7740 1 --2- 172.21.15.97:0/1305963345 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555f7405eb80 0x555f7406b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.667+0000 7fdbba1f7740 1 -- 172.21.15.97:0/1305963345 >> 172.21.15.97:0/1305963345 conn(0x555f7405a070 msgr2=0x555f74054e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:02.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.667+0000 7fdbba1f7740 1 -- 172.21.15.97:0/1305963345 shutdown_connections 2024-04-23T10:26:02.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.667+0000 7fdbba1f7740 1 -- 172.21.15.97:0/1305963345 wait complete. 2024-04-23T10:26:02.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.667+0000 7fdbba1f7740 20 client.15318 trim_cache size 0 max 16384 2024-04-23T10:26:02.675 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-23T10:26:02.675 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "9b22af88-0159-11ef-bc93-c7b262605968", 2024-04-23T10:26:02.675 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 2, 2024-04-23T10:26:02.675 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 24469, 2024-04-23T10:26:02.675 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-04-23T10:26:02.675 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-04-23T10:26:02.675 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-04-23T10:26:02.675 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 112.093747486, 2024-04-23T10:26:02.675 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 25, 2024-04-23T10:26:02.675 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 88, 2024-04-23T10:26:02.675 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 86, 2024-04-23T10:26:02.675 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 135.969488119 2024-04-23T10:26:02.676 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-23T10:26:02.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.668+0000 7fdbb74e1640 1 -- 172.21.15.97:0/2675681518 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fdb9c0a3bb0 msgr2=0x7fdb9c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:02.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.668+0000 7fdbb74e1640 1 --2- 172.21.15.97:0/2675681518 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fdb9c0a3bb0 0x7fdb9c0a6070 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x555f7405fc50 tx=0x7fdba8004a40 comp rx=0 tx=0).stop 2024-04-23T10:26:02.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.668+0000 7fdbb74e1640 1 -- 172.21.15.97:0/2675681518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbb0105360 msgr2=0x7fdbb0198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:02.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.668+0000 7fdbb74e1640 1 --2- 172.21.15.97:0/2675681518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbb0105360 0x7fdbb0198870 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fdba000b450 tx=0x7fdba000b910 comp rx=0 tx=0).stop 2024-04-23T10:26:02.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.668+0000 7fdbb74e1640 1 -- 172.21.15.97:0/2675681518 shutdown_connections 2024-04-23T10:26:02.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.668+0000 7fdbb74e1640 1 --2- 172.21.15.97:0/2675681518 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fdb9c0a3bb0 0x7fdb9c0a6070 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.668+0000 7fdbb74e1640 1 --2- 172.21.15.97:0/2675681518 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdbb0107c90 0x7fdbb019f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.668+0000 7fdbb74e1640 1 --2- 172.21.15.97:0/2675681518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdbb0105360 0x7fdbb0198870 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.668+0000 7fdbb74e1640 1 --2- 172.21.15.97:0/2675681518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbb0102a30 0x7fdbb0198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.668+0000 7fdbb74e1640 1 -- 172.21.15.97:0/2675681518 >> 172.21.15.97:0/2675681518 conn(0x7fdbb00fc440 msgr2=0x7fdbb0103780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:02.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.668+0000 7fdbb74e1640 1 -- 172.21.15.97:0/2675681518 shutdown_connections 2024-04-23T10:26:02.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.668+0000 7fdbb74e1640 1 -- 172.21.15.97:0/2675681518 wait complete. 2024-04-23T10:26:02.678 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-23T10:26:02.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.778+0000 7f545f3cc640 1 -- 172.21.15.97:0/3511870643 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5458105360 msgr2=0x7f5458107750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:02.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.778+0000 7f545f3cc640 1 --2- 172.21.15.97:0/3511870643 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5458105360 0x7f5458107750 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f5448009900 tx=0x7f544802a410 comp rx=0 tx=0).stop 2024-04-23T10:26:02.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.778+0000 7f545f3cc640 1 -- 172.21.15.97:0/3511870643 shutdown_connections 2024-04-23T10:26:02.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.778+0000 7f545f3cc640 1 --2- 172.21.15.97:0/3511870643 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5458107c90 0x7f545810a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.778+0000 7f545f3cc640 1 --2- 172.21.15.97:0/3511870643 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5458105360 0x7f5458107750 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.778+0000 7f545f3cc640 1 --2- 172.21.15.97:0/3511870643 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5458102a30 0x7f5458104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.778+0000 7f545f3cc640 1 -- 172.21.15.97:0/3511870643 >> 172.21.15.97:0/3511870643 conn(0x7f54580fc440 msgr2=0x7f54580fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:02.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.778+0000 7f545f3cc640 1 -- 172.21.15.97:0/3511870643 shutdown_connections 2024-04-23T10:26:02.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.778+0000 7f545f3cc640 1 -- 172.21.15.97:0/3511870643 wait complete. 2024-04-23T10:26:02.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.779+0000 7f545f3cc640 1 Processor -- start 2024-04-23T10:26:02.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.779+0000 7f545f3cc640 1 -- start start 2024-04-23T10:26:02.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.779+0000 7f545f3cc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5458102a30 0x7f5458198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.779+0000 7f545f3cc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5458105360 0x7f5458198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.779+0000 7f545f3cc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5458107c90 0x7f545819f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.779+0000 7f545f3cc640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f545810db50 con 0x7f5458107c90 2024-04-23T10:26:02.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.779+0000 7f545f3cc640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f545810d9d0 con 0x7f5458105360 2024-04-23T10:26:02.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.779+0000 7f545f3cc640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f545810dcd0 con 0x7f5458102a30 2024-04-23T10:26:02.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.779+0000 7f545d969640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5458107c90 0x7f545819f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.779+0000 7f545d969640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5458107c90 0x7f545819f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46854/0 (socket says 172.21.15.97:46854) 2024-04-23T10:26:02.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.779+0000 7f545d969640 1 -- 172.21.15.97:0/3578676135 learned_addr learned my addr 172.21.15.97:0/3578676135 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:02.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.779+0000 7f545c967640 1 --2- 172.21.15.97:0/3578676135 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5458105360 0x7f5458198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.779+0000 7f545d168640 1 --2- 172.21.15.97:0/3578676135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5458102a30 0x7f5458198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.779+0000 7f545d969640 1 -- 172.21.15.97:0/3578676135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5458102a30 msgr2=0x7f5458198220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:02.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.779+0000 7f545d969640 1 --2- 172.21.15.97:0/3578676135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5458102a30 0x7f5458198220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.779+0000 7f545d969640 1 -- 172.21.15.97:0/3578676135 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5458105360 msgr2=0x7f5458198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:02.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.779+0000 7f545d969640 1 --2- 172.21.15.97:0/3578676135 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5458105360 0x7f5458198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.779+0000 7f545d969640 1 -- 172.21.15.97:0/3578676135 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f545819fe40 con 0x7f5458107c90 2024-04-23T10:26:02.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.779+0000 7f545c967640 1 --2- 172.21.15.97:0/3578676135 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5458105360 0x7f5458198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:26:02.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.779+0000 7f545d969640 1 --2- 172.21.15.97:0/3578676135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5458107c90 0x7f545819f7d0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f545000eaa0 tx=0x7f545000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:02.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.780+0000 7f544e7fc640 1 -- 172.21.15.97:0/3578676135 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f545000cdb0 con 0x7f5458107c90 2024-04-23T10:26:02.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.780+0000 7f544e7fc640 1 -- 172.21.15.97:0/3578676135 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5450005020 con 0x7f5458107c90 2024-04-23T10:26:02.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.780+0000 7f545f3cc640 1 -- 172.21.15.97:0/3578676135 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54581a0130 con 0x7f5458107c90 2024-04-23T10:26:02.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.780+0000 7f544e7fc640 1 -- 172.21.15.97:0/3578676135 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54500116d0 con 0x7f5458107c90 2024-04-23T10:26:02.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.780+0000 7f545f3cc640 1 -- 172.21.15.97:0/3578676135 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54581a06d0 con 0x7f5458107c90 2024-04-23T10:26:02.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.782+0000 7f544e7fc640 1 -- 172.21.15.97:0/3578676135 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f54500051d0 con 0x7f5458107c90 2024-04-23T10:26:02.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.782+0000 7f54621b6740 1 Processor -- start 2024-04-23T10:26:02.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.782+0000 7f54621b6740 1 -- start start 2024-04-23T10:26:02.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.782+0000 7f54621b6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ad1e72f90 0x559ad1e733b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.782+0000 7f54621b6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ad1e74160 0x559ad1e745e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.782+0000 7f54621b6740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ad1e74b20 0x559ad1e7b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.782+0000 7f54621b6740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559ad1dd3c50 con 0x559ad1e74b20 2024-04-23T10:26:02.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.782+0000 7f54621b6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559ad1d70830 con 0x559ad1e74160 2024-04-23T10:26:02.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.782+0000 7f54621b6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559ad1dcfc10 con 0x559ad1e72f90 2024-04-23T10:26:02.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.782+0000 7f544e7fc640 1 --2- 172.21.15.97:0/3578676135 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f54440a3bb0 0x7f54440a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.783+0000 7f545c967640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ad1e74b20 0x559ad1e7b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.783+0000 7f545c967640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ad1e74b20 0x559ad1e7b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46864/0 (socket says 172.21.15.97:46864) 2024-04-23T10:26:02.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.783+0000 7f545c967640 1 -- 172.21.15.97:0/3418744040 learned_addr learned my addr 172.21.15.97:0/3418744040 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:02.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.783+0000 7f545d969640 1 --2- 172.21.15.97:0/3578676135 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f54440a3bb0 0x7f54440a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.783+0000 7f545c967640 1 --2- 172.21.15.97:0/3418744040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ad1e72f90 0x559ad1e733b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.783+0000 7f545d168640 1 --2- 172.21.15.97:0/3418744040 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ad1e74160 0x559ad1e745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.783+0000 7f544e7fc640 1 -- 172.21.15.97:0/3578676135 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f54500d0640 con 0x7f5458107c90 2024-04-23T10:26:02.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.783+0000 7f545c967640 1 -- 172.21.15.97:0/3418744040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ad1e72f90 msgr2=0x559ad1e733b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:02.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.783+0000 7f545c967640 1 --2- 172.21.15.97:0/3418744040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ad1e72f90 0x559ad1e733b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.783+0000 7f545c967640 1 -- 172.21.15.97:0/3418744040 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ad1e74160 msgr2=0x559ad1e745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:02.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.783+0000 7f545c967640 1 --2- 172.21.15.97:0/3418744040 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ad1e74160 0x559ad1e745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.783+0000 7f545c967640 1 -- 172.21.15.97:0/3418744040 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ad1ae62d0 con 0x559ad1e74b20 2024-04-23T10:26:02.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.783+0000 7f545d969640 1 --2- 172.21.15.97:0/3578676135 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f54440a3bb0 0x7f54440a6070 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5450009270 tx=0x7f54500bd040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:02.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.783+0000 7f545c967640 1 --2- 172.21.15.97:0/3418744040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ad1e74b20 0x559ad1e7b3b0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f5448032fd0 tx=0x7f54480058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e08fedb49066b99 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:02.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.783+0000 7f54437fe640 1 -- 172.21.15.97:0/3418744040 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5448035040 con 0x559ad1e74b20 2024-04-23T10:26:02.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.783+0000 7f54437fe640 1 -- 172.21.15.97:0/3418744040 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5448004230 con 0x559ad1e74b20 2024-04-23T10:26:02.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.783+0000 7f54437fe640 1 -- 172.21.15.97:0/3418744040 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544803d480 con 0x559ad1e74b20 2024-04-23T10:26:02.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.784+0000 7f54621b6740 1 -- 172.21.15.97:0/3418744040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ad1e74b20 msgr2=0x559ad1e7b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:02.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.784+0000 7f54621b6740 1 --2- 172.21.15.97:0/3418744040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ad1e74b20 0x559ad1e7b3b0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f5448032fd0 tx=0x7f54480058b0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.784+0000 7f54621b6740 1 -- 172.21.15.97:0/3418744040 shutdown_connections 2024-04-23T10:26:02.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.784+0000 7f54621b6740 1 --2- 172.21.15.97:0/3418744040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ad1e74b20 0x559ad1e7b3b0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.784+0000 7f54621b6740 1 --2- 172.21.15.97:0/3418744040 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ad1e74160 0x559ad1e745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.784+0000 7f54621b6740 1 --2- 172.21.15.97:0/3418744040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ad1e72f90 0x559ad1e733b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.784+0000 7f54621b6740 1 -- 172.21.15.97:0/3418744040 >> 172.21.15.97:0/3418744040 conn(0x559ad1e6a070 msgr2=0x559ad1e6a470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:02.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.784+0000 7f54621b6740 1 -- 172.21.15.97:0/3418744040 shutdown_connections 2024-04-23T10:26:02.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.784+0000 7f54621b6740 1 -- 172.21.15.97:0/3418744040 wait complete. 2024-04-23T10:26:02.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.784+0000 7f54621b6740 1 -- start start 2024-04-23T10:26:02.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.784+0000 7f54621b6740 1 Processor -- start 2024-04-23T10:26:02.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.785+0000 7f54621b6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ad1e72f90 0x559ad1e7d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.785+0000 7f54621b6740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ad1e74160 0x559ad1e7d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.785+0000 7f54621b6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ad1e74b20 0x559ad1e84940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.785+0000 7f54621b6740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559ad1e7bad0 con 0x559ad1e74160 2024-04-23T10:26:02.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.785+0000 7f54621b6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559ad1d6e470 con 0x559ad1e74b20 2024-04-23T10:26:02.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.785+0000 7f54621b6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559ad1e7bc50 con 0x559ad1e72f90 2024-04-23T10:26:02.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.785+0000 7f54621b6740 10 client.0 _ll_register_callbacks cb 0x559ad1d93f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:26:02.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.785+0000 7f545d168640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ad1e74160 0x559ad1e7d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.785+0000 7f545d168640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ad1e74160 0x559ad1e7d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46874/0 (socket says 172.21.15.97:46874) 2024-04-23T10:26:02.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.785+0000 7f545d168640 1 -- 172.21.15.97:0/1148669717 learned_addr learned my addr 172.21.15.97:0/1148669717 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:02.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.785+0000 7f545c967640 1 --2- 172.21.15.97:0/1148669717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ad1e72f90 0x559ad1e7d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.785+0000 7f545c967640 1 --2- 172.21.15.97:0/1148669717 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ad1e74b20 0x559ad1e84940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.785+0000 7f545d168640 1 -- 172.21.15.97:0/1148669717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ad1e72f90 msgr2=0x559ad1e7d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:02.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.785+0000 7f545d168640 1 --2- 172.21.15.97:0/1148669717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ad1e72f90 0x559ad1e7d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.785+0000 7f545d168640 1 -- 172.21.15.97:0/1148669717 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ad1e74b20 msgr2=0x559ad1e84940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:02.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.785+0000 7f545d168640 1 --2- 172.21.15.97:0/1148669717 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ad1e74b20 0x559ad1e84940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.785+0000 7f545d168640 1 -- 172.21.15.97:0/1148669717 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ad1bb06b0 con 0x559ad1e74160 2024-04-23T10:26:02.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.785+0000 7f545c967640 1 --2- 172.21.15.97:0/1148669717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ad1e72f90 0x559ad1e7d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:26:02.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.785+0000 7f545d168640 1 --2- 172.21.15.97:0/1148669717 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ad1e74160 0x559ad1e7d900 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f54540104d0 tx=0x7f5454010990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48d87a5bd532ca9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:02.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.785+0000 7f5440ff9640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:26:02.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.785+0000 7f5440ff9640 1 -- 172.21.15.97:0/1148669717 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f545401a020 con 0x559ad1e74160 2024-04-23T10:26:02.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.785+0000 7f5440ff9640 1 -- 172.21.15.97:0/1148669717 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f5454016690 con 0x559ad1e74160 2024-04-23T10:26:02.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.785+0000 7f54621b6740 1 -- 172.21.15.97:0/1148669717 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5448004230 con 0x559ad1e74160 2024-04-23T10:26:02.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.786+0000 7f5440ff9640 1 -- 172.21.15.97:0/1148669717 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5454016890 con 0x559ad1e74160 2024-04-23T10:26:02.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.786+0000 7f54621b6740 10 client.15330 fetch_fsmap learned FSMap version 25 2024-04-23T10:26:02.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.786+0000 7f54621b6740 1 -- 172.21.15.97:0/1148669717 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559ad1e87d80 con 0x559ad1e74160 2024-04-23T10:26:02.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.786+0000 7f5440ff9640 1 -- 172.21.15.97:0/1148669717 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5448004230 con 0x559ad1e74160 2024-04-23T10:26:02.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.786+0000 7f5440ff9640 1 -- 172.21.15.97:0/1148669717 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f5454014b70 con 0x559ad1e74160 2024-04-23T10:26:02.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.786+0000 7f54621b6740 10 client.15330 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T10:26:02.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.786+0000 7f54621b6740 10 client.15330 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-23T10:26:02.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.786+0000 7f54621b6740 20 client.15330 populate_metadata read hostname 'smithi097' 2024-04-23T10:26:02.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.786+0000 7f54621b6740 1 --2- 172.21.15.97:0/1148669717 >> [v2:172.21.15.196:6838/83843556,v1:172.21.15.196:6839/83843556] conn(0x559ad1e8a490 0x559ad1e8b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.786+0000 7f54621b6740 4 client.15330 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:26:02.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.786+0000 7f54621b6740 1 -- 172.21.15.97:0/1148669717 --> [v2:172.21.15.196:6838/83843556,v1:172.21.15.196:6839/83843556] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ad1de1670 con 0x559ad1e8a490 2024-04-23T10:26:02.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.786+0000 7f545c967640 1 --2- 172.21.15.97:0/1148669717 >> [v2:172.21.15.196:6838/83843556,v1:172.21.15.196:6839/83843556] conn(0x559ad1e8a490 0x559ad1e8b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.787+0000 7f5440ff9640 10 client.15330 ms_handle_connect on v2:172.21.15.196:6838/83843556 2024-04-23T10:26:02.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.787+0000 7f545c967640 1 --2- 172.21.15.97:0/1148669717 >> [v2:172.21.15.196:6838/83843556,v1:172.21.15.196:6839/83843556] conn(0x559ad1e8a490 0x559ad1e8b900 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=89b0646f10642ddd server_cookie=ef2cda5296b31ad1 in_seq=0 out_seq=0 2024-04-23T10:26:02.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.787+0000 7f5440ff9640 1 -- 172.21.15.97:0/1148669717 <== mds.1 v2:172.21.15.196:6838/83843556 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559ad1de1670 con 0x559ad1e8a490 2024-04-23T10:26:02.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.787+0000 7f5440ff9640 10 client.15330 handle_command_reply: tid=0 2024-04-23T10:26:02.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.787+0000 7f54621b6740 1 client.15330 shutdown 2024-04-23T10:26:02.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.787+0000 7f54621b6740 1 -- 172.21.15.97:0/1148669717 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ad1e74160 msgr2=0x559ad1e7d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:02.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.787+0000 7f54621b6740 1 --2- 172.21.15.97:0/1148669717 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ad1e74160 0x559ad1e7d900 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f54540104d0 tx=0x7f5454010990 comp rx=0 tx=0).stop 2024-04-23T10:26:02.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.787+0000 7f54621b6740 1 -- 172.21.15.97:0/1148669717 shutdown_connections 2024-04-23T10:26:02.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.787+0000 7f54621b6740 1 --2- 172.21.15.97:0/1148669717 >> [v2:172.21.15.196:6838/83843556,v1:172.21.15.196:6839/83843556] conn(0x559ad1e8a490 0x559ad1e8b900 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.787+0000 7f54621b6740 1 --2- 172.21.15.97:0/1148669717 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ad1e74b20 0x559ad1e84940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.787+0000 7f5440ff9640 0 client.15330 ms_handle_reset on v2:172.21.15.196:6838/83843556 2024-04-23T10:26:02.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.787+0000 7f54621b6740 1 --2- 172.21.15.97:0/1148669717 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ad1e74160 0x559ad1e7d900 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.787+0000 7f54621b6740 1 --2- 172.21.15.97:0/1148669717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ad1e72f90 0x559ad1e7d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.787+0000 7f54621b6740 1 -- 172.21.15.97:0/1148669717 >> 172.21.15.97:0/1148669717 conn(0x559ad1e6a070 msgr2=0x559ad1e691d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:02.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.788+0000 7f54621b6740 1 -- 172.21.15.97:0/1148669717 shutdown_connections 2024-04-23T10:26:02.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.788+0000 7f54621b6740 1 -- 172.21.15.97:0/1148669717 wait complete. 2024-04-23T10:26:02.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.788+0000 7f54621b6740 20 client.15330 trim_cache size 0 max 16384 2024-04-23T10:26:02.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.805+0000 7f54621b6740 1 Processor -- start 2024-04-23T10:26:02.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.806+0000 7f54621b6740 1 -- start start 2024-04-23T10:26:02.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.806+0000 7f54621b6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ad1e72f90 0x559ad1e650b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.806+0000 7f54621b6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ad1e74160 0x559ad1e84900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.806+0000 7f54621b6740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ad1e74b20 0x559ad1e68940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.806+0000 7f54621b6740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559ad1e89c40 con 0x559ad1e74b20 2024-04-23T10:26:02.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.806+0000 7f54621b6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559ad1e8a310 con 0x559ad1e74160 2024-04-23T10:26:02.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.806+0000 7f54621b6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559ad1e8a9e0 con 0x559ad1e72f90 2024-04-23T10:26:02.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.806+0000 7f545c967640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ad1e74b20 0x559ad1e68940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.806+0000 7f545c967640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ad1e74b20 0x559ad1e68940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46882/0 (socket says 172.21.15.97:46882) 2024-04-23T10:26:02.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.806+0000 7f545c967640 1 -- 172.21.15.97:0/3981128280 learned_addr learned my addr 172.21.15.97:0/3981128280 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:02.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.806+0000 7f545c967640 1 -- 172.21.15.97:0/3981128280 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ad1e72f90 msgr2=0x559ad1e650b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:02.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.806+0000 7f545c967640 1 --2- 172.21.15.97:0/3981128280 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ad1e72f90 0x559ad1e650b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.806+0000 7f545c967640 1 -- 172.21.15.97:0/3981128280 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ad1e74160 msgr2=0x559ad1e84900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:02.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.806+0000 7f545d168640 1 --2- 172.21.15.97:0/3981128280 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ad1e74160 0x559ad1e84900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.806+0000 7f545c967640 1 --2- 172.21.15.97:0/3981128280 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ad1e74160 0x559ad1e84900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.806+0000 7f545c967640 1 -- 172.21.15.97:0/3981128280 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ad1e7af20 con 0x559ad1e74b20 2024-04-23T10:26:02.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.806+0000 7f545d168640 1 --2- 172.21.15.97:0/3981128280 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ad1e74160 0x559ad1e84900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:26:02.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.806+0000 7f545c967640 1 --2- 172.21.15.97:0/3981128280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ad1e74b20 0x559ad1e68940 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f5448005940 tx=0x7f5448032c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c62cf00347d245a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:02.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.806+0000 7f5442ffd640 1 -- 172.21.15.97:0/3981128280 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5448058070 con 0x559ad1e74b20 2024-04-23T10:26:02.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.806+0000 7f5442ffd640 1 -- 172.21.15.97:0/3981128280 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5448004420 con 0x559ad1e74b20 2024-04-23T10:26:02.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.806+0000 7f5442ffd640 1 -- 172.21.15.97:0/3981128280 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5448035040 con 0x559ad1e74b20 2024-04-23T10:26:02.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.807+0000 7f54621b6740 1 -- 172.21.15.97:0/3981128280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ad1e74b20 msgr2=0x559ad1e68940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:02.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.807+0000 7f54621b6740 1 --2- 172.21.15.97:0/3981128280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ad1e74b20 0x559ad1e68940 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f5448005940 tx=0x7f5448032c20 comp rx=0 tx=0).stop 2024-04-23T10:26:02.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.807+0000 7f54621b6740 1 -- 172.21.15.97:0/3981128280 shutdown_connections 2024-04-23T10:26:02.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.807+0000 7f54621b6740 1 --2- 172.21.15.97:0/3981128280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ad1e74b20 0x559ad1e68940 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.807+0000 7f54621b6740 1 --2- 172.21.15.97:0/3981128280 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ad1e74160 0x559ad1e84900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.807+0000 7f54621b6740 1 --2- 172.21.15.97:0/3981128280 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ad1e72f90 0x559ad1e650b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.807+0000 7f54621b6740 1 -- 172.21.15.97:0/3981128280 >> 172.21.15.97:0/3981128280 conn(0x559ad1e6a070 msgr2=0x559ad1e6a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:02.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.807+0000 7f54621b6740 1 -- 172.21.15.97:0/3981128280 shutdown_connections 2024-04-23T10:26:02.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.807+0000 7f54621b6740 1 -- 172.21.15.97:0/3981128280 wait complete. 2024-04-23T10:26:02.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.807+0000 7f54621b6740 1 -- start start 2024-04-23T10:26:02.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.807+0000 7f54621b6740 1 Processor -- start 2024-04-23T10:26:02.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.808+0000 7f54621b6740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ad1e72f90 0x559ad1e7fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.808+0000 7f54621b6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ad1e74160 0x559ad1e80170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.808+0000 7f54621b6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ad1e74b20 0x559ad1e83140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.808+0000 7f54621b6740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559ad1e8b780 con 0x559ad1e72f90 2024-04-23T10:26:02.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.808+0000 7f54621b6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559ad1e8b0b0 con 0x559ad1e74160 2024-04-23T10:26:02.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.808+0000 7f54621b6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559ad1e8be50 con 0x559ad1e74b20 2024-04-23T10:26:02.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.808+0000 7f54621b6740 10 client.0 _ll_register_callbacks cb 0x7f545400b520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:26:02.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.808+0000 7f545c967640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ad1e72f90 0x559ad1e7fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.808+0000 7f545c967640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ad1e72f90 0x559ad1e7fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46886/0 (socket says 172.21.15.97:46886) 2024-04-23T10:26:02.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.808+0000 7f545c967640 1 -- 172.21.15.97:0/805475224 learned_addr learned my addr 172.21.15.97:0/805475224 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:02.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.808+0000 7f545d168640 1 --2- 172.21.15.97:0/805475224 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ad1e74160 0x559ad1e80170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.808+0000 7f545c967640 1 -- 172.21.15.97:0/805475224 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ad1e74b20 msgr2=0x559ad1e83140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:02.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.808+0000 7f545c967640 1 --2- 172.21.15.97:0/805475224 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ad1e74b20 0x559ad1e83140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.808+0000 7f545c967640 1 -- 172.21.15.97:0/805475224 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ad1e74160 msgr2=0x559ad1e80170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:02.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.808+0000 7f545c967640 1 --2- 172.21.15.97:0/805475224 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ad1e74160 0x559ad1e80170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.808+0000 7f545c967640 1 -- 172.21.15.97:0/805475224 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ad1e83730 con 0x559ad1e72f90 2024-04-23T10:26:02.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.808+0000 7f545d168640 1 --2- 172.21.15.97:0/805475224 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ad1e74160 0x559ad1e80170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:26:02.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.808+0000 7f545c967640 1 --2- 172.21.15.97:0/805475224 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ad1e72f90 0x559ad1e7fc30 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f5448031850 tx=0x7f5448032d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5a009c68546c215 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:02.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.808+0000 7f5440ff9640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:26:02.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.808+0000 7f5440ff9640 1 -- 172.21.15.97:0/805475224 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5448035070 con 0x559ad1e72f90 2024-04-23T10:26:02.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.808+0000 7f5440ff9640 1 -- 172.21.15.97:0/805475224 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f5448053880 con 0x559ad1e72f90 2024-04-23T10:26:02.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.808+0000 7f54621b6740 1 -- 172.21.15.97:0/805475224 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5448004420 con 0x559ad1e72f90 2024-04-23T10:26:02.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.808+0000 7f5440ff9640 1 -- 172.21.15.97:0/805475224 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5448065070 con 0x559ad1e72f90 2024-04-23T10:26:02.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.808+0000 7f54621b6740 10 client.15336 fetch_fsmap learned FSMap version 25 2024-04-23T10:26:02.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.808+0000 7f54621b6740 1 -- 172.21.15.97:0/805475224 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559ad1e849a0 con 0x559ad1e72f90 2024-04-23T10:26:02.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.808+0000 7f5440ff9640 1 -- 172.21.15.97:0/805475224 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5448004420 con 0x559ad1e72f90 2024-04-23T10:26:02.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.809+0000 7f5440ff9640 1 -- 172.21.15.97:0/805475224 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f544803a040 con 0x559ad1e72f90 2024-04-23T10:26:02.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.809+0000 7f54621b6740 10 client.15336 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T10:26:02.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.809+0000 7f54621b6740 10 client.15336 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-23T10:26:02.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.809+0000 7f54621b6740 20 client.15336 populate_metadata read hostname 'smithi097' 2024-04-23T10:26:02.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.809+0000 7f54621b6740 1 --2- 172.21.15.97:0/805475224 >> [v2:172.21.15.196:6838/83843556,v1:172.21.15.196:6839/83843556] conn(0x559ad1ef4390 0x559ad1ef30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.809+0000 7f54621b6740 4 client.15336 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T10:26:02.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.809+0000 7f54621b6740 1 -- 172.21.15.97:0/805475224 --> [v2:172.21.15.196:6838/83843556,v1:172.21.15.196:6839/83843556] -- command(tid 0: {"prefix": "status"}) -- 0x559ad1ddb4e0 con 0x559ad1ef4390 2024-04-23T10:26:02.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.809+0000 7f545d168640 1 --2- 172.21.15.97:0/805475224 >> [v2:172.21.15.196:6838/83843556,v1:172.21.15.196:6839/83843556] conn(0x559ad1ef4390 0x559ad1ef30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.809+0000 7f5440ff9640 10 client.15336 ms_handle_connect on v2:172.21.15.196:6838/83843556 2024-04-23T10:26:02.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.809+0000 7f545d168640 1 --2- 172.21.15.97:0/805475224 >> [v2:172.21.15.196:6838/83843556,v1:172.21.15.196:6839/83843556] conn(0x559ad1ef4390 0x559ad1ef30a0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ae3631eb17386ed1 server_cookie=6f2082bd424e3bdd in_seq=0 out_seq=0 2024-04-23T10:26:02.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.810+0000 7f5440ff9640 1 -- 172.21.15.97:0/805475224 <== mds.1 v2:172.21.15.196:6838/83843556 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x559ad1ddb4e0 con 0x559ad1ef4390 2024-04-23T10:26:02.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.810+0000 7f5440ff9640 10 client.15336 handle_command_reply: tid=0 2024-04-23T10:26:02.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.810+0000 7f54621b6740 1 client.15336 shutdown 2024-04-23T10:26:02.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.810+0000 7f54621b6740 1 -- 172.21.15.97:0/805475224 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ad1e72f90 msgr2=0x559ad1e7fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:02.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.810+0000 7f54621b6740 1 --2- 172.21.15.97:0/805475224 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ad1e72f90 0x559ad1e7fc30 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f5448031850 tx=0x7f5448032d90 comp rx=0 tx=0).stop 2024-04-23T10:26:02.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.810+0000 7f54621b6740 1 -- 172.21.15.97:0/805475224 shutdown_connections 2024-04-23T10:26:02.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.810+0000 7f54621b6740 1 --2- 172.21.15.97:0/805475224 >> [v2:172.21.15.196:6838/83843556,v1:172.21.15.196:6839/83843556] conn(0x559ad1ef4390 0x559ad1ef30a0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.810+0000 7f54621b6740 1 --2- 172.21.15.97:0/805475224 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559ad1e74b20 0x559ad1e83140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.810+0000 7f54621b6740 1 --2- 172.21.15.97:0/805475224 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559ad1e74160 0x559ad1e80170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.810+0000 7f5440ff9640 0 client.15336 ms_handle_reset on v2:172.21.15.196:6838/83843556 2024-04-23T10:26:02.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.810+0000 7f54621b6740 1 --2- 172.21.15.97:0/805475224 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559ad1e72f90 0x559ad1e7fc30 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.810+0000 7f54621b6740 1 -- 172.21.15.97:0/805475224 >> 172.21.15.97:0/805475224 conn(0x559ad1e6a070 msgr2=0x559ad1e650b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:02.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.810+0000 7f54621b6740 1 -- 172.21.15.97:0/805475224 shutdown_connections 2024-04-23T10:26:02.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.810+0000 7f54621b6740 1 -- 172.21.15.97:0/805475224 wait complete. 2024-04-23T10:26:02.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.810+0000 7f54621b6740 20 client.15336 trim_cache size 0 max 16384 2024-04-23T10:26:02.816 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-23T10:26:02.816 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "9b22af88-0159-11ef-bc93-c7b262605968", 2024-04-23T10:26:02.816 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 1, 2024-04-23T10:26:02.816 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 24487, 2024-04-23T10:26:02.816 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-04-23T10:26:02.816 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-04-23T10:26:02.816 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-04-23T10:26:02.817 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 113.244411755, 2024-04-23T10:26:02.817 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 25, 2024-04-23T10:26:02.817 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 88, 2024-04-23T10:26:02.817 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 86, 2024-04-23T10:26:02.817 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 130.30641781599999 2024-04-23T10:26:02.817 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-23T10:26:02.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.810+0000 7f545f3cc640 1 -- 172.21.15.97:0/3578676135 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f54440a3bb0 msgr2=0x7f54440a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:02.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.810+0000 7f545f3cc640 1 --2- 172.21.15.97:0/3578676135 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f54440a3bb0 0x7f54440a6070 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5450009270 tx=0x7f54500bd040 comp rx=0 tx=0).stop 2024-04-23T10:26:02.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.810+0000 7f545f3cc640 1 -- 172.21.15.97:0/3578676135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5458107c90 msgr2=0x7f545819f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:02.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.810+0000 7f545f3cc640 1 --2- 172.21.15.97:0/3578676135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5458107c90 0x7f545819f7d0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f545000eaa0 tx=0x7f545000ef60 comp rx=0 tx=0).stop 2024-04-23T10:26:02.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.810+0000 7f545f3cc640 1 -- 172.21.15.97:0/3578676135 shutdown_connections 2024-04-23T10:26:02.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.810+0000 7f545f3cc640 1 --2- 172.21.15.97:0/3578676135 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f54440a3bb0 0x7f54440a6070 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.810+0000 7f545f3cc640 1 --2- 172.21.15.97:0/3578676135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5458107c90 0x7f545819f7d0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.810+0000 7f545f3cc640 1 --2- 172.21.15.97:0/3578676135 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5458105360 0x7f5458198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.810+0000 7f545f3cc640 1 --2- 172.21.15.97:0/3578676135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5458102a30 0x7f5458198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.810+0000 7f545f3cc640 1 -- 172.21.15.97:0/3578676135 >> 172.21.15.97:0/3578676135 conn(0x7f54580fc440 msgr2=0x7f5458103570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:02.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.811+0000 7f545f3cc640 1 -- 172.21.15.97:0/3578676135 shutdown_connections 2024-04-23T10:26:02.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.811+0000 7f545f3cc640 1 -- 172.21.15.97:0/3578676135 wait complete. 2024-04-23T10:26:02.820 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-23T10:26:02.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.920+0000 7f6ac3e31640 1 -- 172.21.15.97:0/3479237595 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6abc102a30 msgr2=0x7f6abc104e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:02.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.920+0000 7f6ac3e31640 1 --2- 172.21.15.97:0/3479237595 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6abc102a30 0x7f6abc104e20 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f6aac009900 tx=0x7f6aac02a410 comp rx=0 tx=0).stop 2024-04-23T10:26:02.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.920+0000 7f6ac3e31640 1 -- 172.21.15.97:0/3479237595 shutdown_connections 2024-04-23T10:26:02.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.920+0000 7f6ac3e31640 1 --2- 172.21.15.97:0/3479237595 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6abc107c90 0x7f6abc10a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.920+0000 7f6ac3e31640 1 --2- 172.21.15.97:0/3479237595 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6abc105360 0x7f6abc107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.920+0000 7f6ac3e31640 1 --2- 172.21.15.97:0/3479237595 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6abc102a30 0x7f6abc104e20 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.920+0000 7f6ac3e31640 1 -- 172.21.15.97:0/3479237595 >> 172.21.15.97:0/3479237595 conn(0x7f6abc0fc440 msgr2=0x7f6abc0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:02.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.920+0000 7f6ac3e31640 1 -- 172.21.15.97:0/3479237595 shutdown_connections 2024-04-23T10:26:02.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.920+0000 7f6ac3e31640 1 -- 172.21.15.97:0/3479237595 wait complete. 2024-04-23T10:26:02.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.920+0000 7f6ac3e31640 1 Processor -- start 2024-04-23T10:26:02.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.920+0000 7f6ac3e31640 1 -- start start 2024-04-23T10:26:02.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.921+0000 7f6ac3e31640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6abc102a30 0x7f6abc198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.921+0000 7f6ac3e31640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6abc105360 0x7f6abc1987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.921+0000 7f6ac3e31640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6abc107c90 0x7f6abc19f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.921+0000 7f6ac3e31640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6abc10daf0 con 0x7f6abc105360 2024-04-23T10:26:02.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.921+0000 7f6ac3e31640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6abc10d970 con 0x7f6abc107c90 2024-04-23T10:26:02.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.921+0000 7f6ac3e31640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6abc10dc70 con 0x7f6abc102a30 2024-04-23T10:26:02.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.921+0000 7f6ac13cc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6abc105360 0x7f6abc1987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.921+0000 7f6ac13cc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6abc105360 0x7f6abc1987b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46896/0 (socket says 172.21.15.97:46896) 2024-04-23T10:26:02.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.921+0000 7f6ac13cc640 1 -- 172.21.15.97:0/3498181498 learned_addr learned my addr 172.21.15.97:0/3498181498 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:02.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.921+0000 7f6ac1bcd640 1 --2- 172.21.15.97:0/3498181498 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6abc102a30 0x7f6abc198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.921+0000 7f6ac23ce640 1 --2- 172.21.15.97:0/3498181498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6abc107c90 0x7f6abc19f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.921+0000 7f6ac13cc640 1 -- 172.21.15.97:0/3498181498 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6abc102a30 msgr2=0x7f6abc198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:02.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.921+0000 7f6ac13cc640 1 --2- 172.21.15.97:0/3498181498 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6abc102a30 0x7f6abc198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.921+0000 7f6ac13cc640 1 -- 172.21.15.97:0/3498181498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6abc107c90 msgr2=0x7f6abc19f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:02.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.921+0000 7f6ac13cc640 1 --2- 172.21.15.97:0/3498181498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6abc107c90 0x7f6abc19f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.921+0000 7f6ac13cc640 1 -- 172.21.15.97:0/3498181498 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6abc19fe40 con 0x7f6abc105360 2024-04-23T10:26:02.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.921+0000 7f6ac1bcd640 1 --2- 172.21.15.97:0/3498181498 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6abc102a30 0x7f6abc198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:26:02.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.921+0000 7f6ac23ce640 1 --2- 172.21.15.97:0/3498181498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6abc107c90 0x7f6abc19f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:26:02.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.921+0000 7f6ac13cc640 1 --2- 172.21.15.97:0/3498181498 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6abc105360 0x7f6abc1987b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f6ab800d770 tx=0x7f6ab800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:02.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.921+0000 7f6ab2ffd640 1 -- 172.21.15.97:0/3498181498 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab8017070 con 0x7f6abc105360 2024-04-23T10:26:02.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.921+0000 7f6ab2ffd640 1 -- 172.21.15.97:0/3498181498 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ab8009c30 con 0x7f6abc105360 2024-04-23T10:26:02.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.921+0000 7f6ab2ffd640 1 -- 172.21.15.97:0/3498181498 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab800f620 con 0x7f6abc105360 2024-04-23T10:26:02.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.921+0000 7f6ac3e31640 1 -- 172.21.15.97:0/3498181498 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6abc1a0130 con 0x7f6abc105360 2024-04-23T10:26:02.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.921+0000 7f6ac3e31640 1 -- 172.21.15.97:0/3498181498 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6abc1a0680 con 0x7f6abc105360 2024-04-23T10:26:02.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.923+0000 7f6ab2ffd640 1 -- 172.21.15.97:0/3498181498 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f6ab801a070 con 0x7f6abc105360 2024-04-23T10:26:02.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.924+0000 7f6ac6c23740 1 Processor -- start 2024-04-23T10:26:02.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.924+0000 7f6ac6c23740 1 -- start start 2024-04-23T10:26:02.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.924+0000 7f6ac6c23740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558bb2c49f90 0x558bb2c4a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.924+0000 7f6ac6c23740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bb2c4b160 0x558bb2c4b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.924+0000 7f6ac6c23740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bb2c4bb20 0x558bb2c523b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.924+0000 7f6ac6c23740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x558bb2baac50 con 0x558bb2c4b160 2024-04-23T10:26:02.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.924+0000 7f6ac6c23740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558bb2b47830 con 0x558bb2c4bb20 2024-04-23T10:26:02.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.924+0000 7f6ac6c23740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558bb2ba6c10 con 0x558bb2c49f90 2024-04-23T10:26:02.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.924+0000 7f6ab2ffd640 1 --2- 172.21.15.97:0/3498181498 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f6aa80a3bb0 0x7f6aa80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.925+0000 7f6ac1bcd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bb2c4b160 0x558bb2c4b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.925+0000 7f6ac1bcd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bb2c4b160 0x558bb2c4b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46898/0 (socket says 172.21.15.97:46898) 2024-04-23T10:26:02.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.925+0000 7f6ac1bcd640 1 -- 172.21.15.97:0/3465968864 learned_addr learned my addr 172.21.15.97:0/3465968864 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:02.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.925+0000 7f6ac23ce640 1 --2- 172.21.15.97:0/3498181498 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f6aa80a3bb0 0x7f6aa80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.925+0000 7f6ac23ce640 1 --2- 172.21.15.97:0/3465968864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558bb2c49f90 0x558bb2c4a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.925+0000 7f6ab2ffd640 1 -- 172.21.15.97:0/3498181498 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f6ab80cffe0 con 0x7f6abc105360 2024-04-23T10:26:02.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.925+0000 7f6ac13cc640 1 --2- 172.21.15.97:0/3465968864 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bb2c4bb20 0x558bb2c523b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.925+0000 7f6ac1bcd640 1 -- 172.21.15.97:0/3465968864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558bb2c49f90 msgr2=0x558bb2c4a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:02.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.925+0000 7f6ac1bcd640 1 --2- 172.21.15.97:0/3465968864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558bb2c49f90 0x558bb2c4a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.925+0000 7f6ac1bcd640 1 -- 172.21.15.97:0/3465968864 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bb2c4bb20 msgr2=0x558bb2c523b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:02.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.925+0000 7f6ac1bcd640 1 --2- 172.21.15.97:0/3465968864 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bb2c4bb20 0x558bb2c523b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.925+0000 7f6ac1bcd640 1 -- 172.21.15.97:0/3465968864 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bb28bd2d0 con 0x558bb2c4b160 2024-04-23T10:26:02.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.925+0000 7f6ac23ce640 1 --2- 172.21.15.97:0/3498181498 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f6aa80a3bb0 0x7f6aa80a6070 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6ab4005c40 tx=0x7f6ab4009dd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:02.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.925+0000 7f6ac1bcd640 1 --2- 172.21.15.97:0/3465968864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bb2c4b160 0x558bb2c4b5e0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f6aac009900 tx=0x7f6aac032c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c93c137569561f5f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:02.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.925+0000 7f6aa7fff640 1 -- 172.21.15.97:0/3465968864 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aac03f070 con 0x558bb2c4b160 2024-04-23T10:26:02.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.925+0000 7f6aa7fff640 1 -- 172.21.15.97:0/3465968864 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6aac032e50 con 0x558bb2c4b160 2024-04-23T10:26:02.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.925+0000 7f6aa7fff640 1 -- 172.21.15.97:0/3465968864 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aac035070 con 0x558bb2c4b160 2024-04-23T10:26:02.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.926+0000 7f6ac6c23740 1 -- 172.21.15.97:0/3465968864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bb2c4b160 msgr2=0x558bb2c4b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:02.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.926+0000 7f6ac6c23740 1 --2- 172.21.15.97:0/3465968864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bb2c4b160 0x558bb2c4b5e0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f6aac009900 tx=0x7f6aac032c60 comp rx=0 tx=0).stop 2024-04-23T10:26:02.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.926+0000 7f6ac6c23740 1 -- 172.21.15.97:0/3465968864 shutdown_connections 2024-04-23T10:26:02.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.926+0000 7f6ac6c23740 1 --2- 172.21.15.97:0/3465968864 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bb2c4bb20 0x558bb2c523b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.926+0000 7f6ac6c23740 1 --2- 172.21.15.97:0/3465968864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bb2c4b160 0x558bb2c4b5e0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.926+0000 7f6ac6c23740 1 --2- 172.21.15.97:0/3465968864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558bb2c49f90 0x558bb2c4a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.926+0000 7f6ac6c23740 1 -- 172.21.15.97:0/3465968864 >> 172.21.15.97:0/3465968864 conn(0x558bb2c41070 msgr2=0x558bb2c41470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:02.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.926+0000 7f6ac6c23740 1 -- 172.21.15.97:0/3465968864 shutdown_connections 2024-04-23T10:26:02.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.926+0000 7f6ac6c23740 1 -- 172.21.15.97:0/3465968864 wait complete. 2024-04-23T10:26:02.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.926+0000 7f6ac6c23740 1 -- start start 2024-04-23T10:26:02.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.926+0000 7f6ac6c23740 1 Processor -- start 2024-04-23T10:26:02.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.927+0000 7f6ac6c23740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bb2c49f90 0x558bb2c543e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.927+0000 7f6ac6c23740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558bb2c4b160 0x558bb2c54920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.927+0000 7f6ac6c23740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bb2c4bb20 0x558bb2c5b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.927+0000 7f6ac6c23740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x558bb2c52ad0 con 0x558bb2c4bb20 2024-04-23T10:26:02.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.927+0000 7f6ac6c23740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558bb2b45470 con 0x558bb2c49f90 2024-04-23T10:26:02.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.927+0000 7f6ac6c23740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558bb2c52c50 con 0x558bb2c4b160 2024-04-23T10:26:02.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.927+0000 7f6ac6c23740 10 client.0 _ll_register_callbacks cb 0x558bb2b6af30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:26:02.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.927+0000 7f6ac23ce640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bb2c4bb20 0x558bb2c5b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.927+0000 7f6ac23ce640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bb2c4bb20 0x558bb2c5b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46904/0 (socket says 172.21.15.97:46904) 2024-04-23T10:26:02.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.927+0000 7f6ac23ce640 1 -- 172.21.15.97:0/1860995308 learned_addr learned my addr 172.21.15.97:0/1860995308 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:02.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.927+0000 7f6ac1bcd640 1 --2- 172.21.15.97:0/1860995308 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bb2c49f90 0x558bb2c543e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.927+0000 7f6ac23ce640 1 -- 172.21.15.97:0/1860995308 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558bb2c4b160 msgr2=0x558bb2c54920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:02.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.927+0000 7f6ac13cc640 1 --2- 172.21.15.97:0/1860995308 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558bb2c4b160 0x558bb2c54920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.927+0000 7f6ac23ce640 1 --2- 172.21.15.97:0/1860995308 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558bb2c4b160 0x558bb2c54920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.927+0000 7f6ac23ce640 1 -- 172.21.15.97:0/1860995308 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bb2c49f90 msgr2=0x558bb2c543e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:02.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.927+0000 7f6ac23ce640 1 --2- 172.21.15.97:0/1860995308 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bb2c49f90 0x558bb2c543e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.927+0000 7f6ac23ce640 1 -- 172.21.15.97:0/1860995308 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558bb29876b0 con 0x558bb2c4bb20 2024-04-23T10:26:02.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.927+0000 7f6ac1bcd640 1 --2- 172.21.15.97:0/1860995308 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bb2c49f90 0x558bb2c543e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:26:02.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.927+0000 7f6ac23ce640 1 --2- 172.21.15.97:0/1860995308 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bb2c4bb20 0x558bb2c5b960 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f6ab4009d20 tx=0x7f6ab4005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a074e8a8350a14 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:02.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.927+0000 7f6aa57fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:26:02.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.927+0000 7f6aa57fa640 1 -- 172.21.15.97:0/1860995308 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab4024070 con 0x558bb2c4bb20 2024-04-23T10:26:02.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.927+0000 7f6aa57fa640 1 -- 172.21.15.97:0/1860995308 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f6ab4010d50 con 0x558bb2c4bb20 2024-04-23T10:26:02.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.927+0000 7f6ac6c23740 1 -- 172.21.15.97:0/1860995308 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6aac032e50 con 0x558bb2c4bb20 2024-04-23T10:26:02.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.928+0000 7f6aa57fa640 1 -- 172.21.15.97:0/1860995308 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6ab402f070 con 0x558bb2c4bb20 2024-04-23T10:26:02.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.928+0000 7f6ac6c23740 10 client.15348 fetch_fsmap learned FSMap version 25 2024-04-23T10:26:02.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.928+0000 7f6ac6c23740 1 -- 172.21.15.97:0/1860995308 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558bb2c5ed00 con 0x558bb2c4bb20 2024-04-23T10:26:02.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.928+0000 7f6aa57fa640 1 -- 172.21.15.97:0/1860995308 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6aac032e50 con 0x558bb2c4bb20 2024-04-23T10:26:02.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.928+0000 7f6aa57fa640 1 -- 172.21.15.97:0/1860995308 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f6ab401f070 con 0x558bb2c4bb20 2024-04-23T10:26:02.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.928+0000 7f6ac6c23740 10 client.15348 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T10:26:02.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.928+0000 7f6ac6c23740 10 client.15348 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-23T10:26:02.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.928+0000 7f6ac6c23740 20 client.15348 populate_metadata read hostname 'smithi097' 2024-04-23T10:26:02.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.928+0000 7f6ac6c23740 1 --2- 172.21.15.97:0/1860995308 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x558bb2c61400 0x558bb2c62870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.928+0000 7f6ac6c23740 4 client.15348 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:26:02.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.928+0000 7f6ac6c23740 1 -- 172.21.15.97:0/1860995308 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558bb2bb8670 con 0x558bb2c61400 2024-04-23T10:26:02.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.928+0000 7f6ac1bcd640 1 --2- 172.21.15.97:0/1860995308 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x558bb2c61400 0x558bb2c62870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.929+0000 7f6aa57fa640 10 client.15348 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:26:02.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.929+0000 7f6ac1bcd640 1 --2- 172.21.15.97:0/1860995308 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x558bb2c61400 0x558bb2c62870 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65a2231258d209b2 server_cookie=91eebd22d9cdf65a in_seq=0 out_seq=0 2024-04-23T10:26:02.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.929+0000 7f6aa57fa640 1 -- 172.21.15.97:0/1860995308 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558bb2bb8670 con 0x558bb2c61400 2024-04-23T10:26:02.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.929+0000 7f6aa57fa640 10 client.15348 handle_command_reply: tid=0 2024-04-23T10:26:02.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.929+0000 7f6ac6c23740 1 client.15348 shutdown 2024-04-23T10:26:02.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.929+0000 7f6ac6c23740 1 -- 172.21.15.97:0/1860995308 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bb2c4bb20 msgr2=0x558bb2c5b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:02.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.929+0000 7f6ac6c23740 1 --2- 172.21.15.97:0/1860995308 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bb2c4bb20 0x558bb2c5b960 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f6ab4009d20 tx=0x7f6ab4005bd0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.930+0000 7f6ac6c23740 1 -- 172.21.15.97:0/1860995308 shutdown_connections 2024-04-23T10:26:02.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.930+0000 7f6ac6c23740 1 --2- 172.21.15.97:0/1860995308 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x558bb2c61400 0x558bb2c62870 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.930+0000 7f6ac6c23740 1 --2- 172.21.15.97:0/1860995308 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bb2c4bb20 0x558bb2c5b960 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.930+0000 7f6ac6c23740 1 --2- 172.21.15.97:0/1860995308 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558bb2c4b160 0x558bb2c54920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.930+0000 7f6aa57fa640 0 client.15348 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:26:02.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.930+0000 7f6ac6c23740 1 --2- 172.21.15.97:0/1860995308 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bb2c49f90 0x558bb2c543e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.930+0000 7f6ac6c23740 1 -- 172.21.15.97:0/1860995308 >> 172.21.15.97:0/1860995308 conn(0x558bb2c41070 msgr2=0x558bb2c401d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:02.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.930+0000 7f6ac6c23740 1 -- 172.21.15.97:0/1860995308 shutdown_connections 2024-04-23T10:26:02.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.930+0000 7f6ac6c23740 1 -- 172.21.15.97:0/1860995308 wait complete. 2024-04-23T10:26:02.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.930+0000 7f6ac6c23740 20 client.15348 trim_cache size 0 max 16384 2024-04-23T10:26:02.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.948+0000 7f6ac6c23740 1 Processor -- start 2024-04-23T10:26:02.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.948+0000 7f6ac6c23740 1 -- start start 2024-04-23T10:26:02.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.948+0000 7f6ac6c23740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bb2c49f90 0x558bb2c4ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.948+0000 7f6ac6c23740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558bb2c4b160 0x558bb2c4f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.948+0000 7f6ac6c23740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bb2c4bb20 0x558bb2c5b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.948+0000 7f6ac6c23740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x558bb2c60bb0 con 0x558bb2c4bb20 2024-04-23T10:26:02.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.948+0000 7f6ac6c23740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558bb2c61280 con 0x558bb2c49f90 2024-04-23T10:26:02.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.948+0000 7f6ac6c23740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558bb2c61950 con 0x558bb2c4b160 2024-04-23T10:26:02.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.948+0000 7f6ac23ce640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bb2c4bb20 0x558bb2c5b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.948+0000 7f6ac23ce640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bb2c4bb20 0x558bb2c5b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46920/0 (socket says 172.21.15.97:46920) 2024-04-23T10:26:02.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.948+0000 7f6ac23ce640 1 -- 172.21.15.97:0/2900817317 learned_addr learned my addr 172.21.15.97:0/2900817317 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:02.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.948+0000 7f6ac1bcd640 1 --2- 172.21.15.97:0/2900817317 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bb2c49f90 0x558bb2c4ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.948+0000 7f6ac23ce640 1 -- 172.21.15.97:0/2900817317 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558bb2c4b160 msgr2=0x558bb2c4f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:02.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.948+0000 7f6ac23ce640 1 --2- 172.21.15.97:0/2900817317 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558bb2c4b160 0x558bb2c4f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.948+0000 7f6ac23ce640 1 -- 172.21.15.97:0/2900817317 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bb2c49f90 msgr2=0x558bb2c4ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:02.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.948+0000 7f6ac23ce640 1 --2- 172.21.15.97:0/2900817317 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bb2c49f90 0x558bb2c4ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.948+0000 7f6ac23ce640 1 -- 172.21.15.97:0/2900817317 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bb2c51f00 con 0x558bb2c4bb20 2024-04-23T10:26:02.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.948+0000 7f6ac23ce640 1 --2- 172.21.15.97:0/2900817317 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bb2c4bb20 0x558bb2c5b880 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f6ab4010820 tx=0x7f6ab40095b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=732b0b4fd13cf381 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:02.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.948+0000 7f6aa77fe640 1 -- 172.21.15.97:0/2900817317 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab4038020 con 0x558bb2c4bb20 2024-04-23T10:26:02.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.948+0000 7f6aa77fe640 1 -- 172.21.15.97:0/2900817317 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ab4010d30 con 0x558bb2c4bb20 2024-04-23T10:26:02.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.948+0000 7f6aa77fe640 1 -- 172.21.15.97:0/2900817317 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab4024070 con 0x558bb2c4bb20 2024-04-23T10:26:02.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.949+0000 7f6ac6c23740 1 -- 172.21.15.97:0/2900817317 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bb2c4bb20 msgr2=0x558bb2c5b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:02.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.949+0000 7f6ac6c23740 1 --2- 172.21.15.97:0/2900817317 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bb2c4bb20 0x558bb2c5b880 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f6ab4010820 tx=0x7f6ab40095b0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.949+0000 7f6ac6c23740 1 -- 172.21.15.97:0/2900817317 shutdown_connections 2024-04-23T10:26:02.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.949+0000 7f6ac6c23740 1 --2- 172.21.15.97:0/2900817317 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bb2c4bb20 0x558bb2c5b880 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.949+0000 7f6ac6c23740 1 --2- 172.21.15.97:0/2900817317 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558bb2c4b160 0x558bb2c4f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.949+0000 7f6ac6c23740 1 --2- 172.21.15.97:0/2900817317 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bb2c49f90 0x558bb2c4ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.949+0000 7f6ac6c23740 1 -- 172.21.15.97:0/2900817317 >> 172.21.15.97:0/2900817317 conn(0x558bb2c41070 msgr2=0x558bb2c3c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:02.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.949+0000 7f6ac6c23740 1 -- 172.21.15.97:0/2900817317 shutdown_connections 2024-04-23T10:26:02.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.949+0000 7f6ac6c23740 1 -- 172.21.15.97:0/2900817317 wait complete. 2024-04-23T10:26:02.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.949+0000 7f6ac6c23740 1 -- start start 2024-04-23T10:26:02.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.949+0000 7f6ac6c23740 1 Processor -- start 2024-04-23T10:26:02.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.950+0000 7f6ac6c23740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bb2c49f90 0x558bb2c56e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.950+0000 7f6ac6c23740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bb2c4b160 0x558bb2c57350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.950+0000 7f6ac6c23740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558bb2c4bb20 0x558bb2c5a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.950+0000 7f6ac6c23740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x558bb2c626f0 con 0x558bb2c4b160 2024-04-23T10:26:02.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.950+0000 7f6ac6c23740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558bb2c62020 con 0x558bb2c49f90 2024-04-23T10:26:02.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.950+0000 7f6ac6c23740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558bb2c62dc0 con 0x558bb2c4bb20 2024-04-23T10:26:02.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.950+0000 7f6ac6c23740 10 client.0 _ll_register_callbacks cb 0x7f6ab40097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:26:02.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.950+0000 7f6ac13cc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bb2c4b160 0x558bb2c57350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.950+0000 7f6ac13cc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bb2c4b160 0x558bb2c57350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46930/0 (socket says 172.21.15.97:46930) 2024-04-23T10:26:02.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.950+0000 7f6ac13cc640 1 -- 172.21.15.97:0/229001235 learned_addr learned my addr 172.21.15.97:0/229001235 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:02.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.950+0000 7f6ac23ce640 1 --2- 172.21.15.97:0/229001235 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558bb2c4bb20 0x558bb2c5a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.950+0000 7f6ac13cc640 1 -- 172.21.15.97:0/229001235 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558bb2c4bb20 msgr2=0x558bb2c5a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:02.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.950+0000 7f6ac1bcd640 1 --2- 172.21.15.97:0/229001235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bb2c49f90 0x558bb2c56e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.950+0000 7f6ac13cc640 1 --2- 172.21.15.97:0/229001235 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558bb2c4bb20 0x558bb2c5a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.950+0000 7f6ac13cc640 1 -- 172.21.15.97:0/229001235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bb2c49f90 msgr2=0x558bb2c56e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:02.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.950+0000 7f6ac13cc640 1 --2- 172.21.15.97:0/229001235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bb2c49f90 0x558bb2c56e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.950+0000 7f6ac13cc640 1 -- 172.21.15.97:0/229001235 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558bb2c5a960 con 0x558bb2c4b160 2024-04-23T10:26:02.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.950+0000 7f6ac13cc640 1 --2- 172.21.15.97:0/229001235 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bb2c4b160 0x558bb2c57350 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f6ab80cde80 tx=0x7f6ab809beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6608fbc81771d0b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:02.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.950+0000 7f6aa57fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:26:02.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.950+0000 7f6aa57fa640 1 -- 172.21.15.97:0/229001235 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab80b5060 con 0x558bb2c4b160 2024-04-23T10:26:02.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.950+0000 7f6aa57fa640 1 -- 172.21.15.97:0/229001235 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f6ab80a0170 con 0x558bb2c4b160 2024-04-23T10:26:02.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.950+0000 7f6ac6c23740 1 -- 172.21.15.97:0/229001235 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ab4010d30 con 0x558bb2c4b160 2024-04-23T10:26:02.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.950+0000 7f6aa57fa640 1 -- 172.21.15.97:0/229001235 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6ab8021040 con 0x558bb2c4b160 2024-04-23T10:26:02.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.950+0000 7f6ac6c23740 10 client.15354 fetch_fsmap learned FSMap version 25 2024-04-23T10:26:02.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.950+0000 7f6ac6c23740 1 -- 172.21.15.97:0/229001235 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558bb2c5bcd0 con 0x558bb2c4b160 2024-04-23T10:26:02.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.950+0000 7f6aa57fa640 1 -- 172.21.15.97:0/229001235 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ab4010d30 con 0x558bb2c4b160 2024-04-23T10:26:02.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.951+0000 7f6aa57fa640 1 -- 172.21.15.97:0/229001235 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f6ab8098040 con 0x558bb2c4b160 2024-04-23T10:26:02.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.951+0000 7f6ac6c23740 10 client.15354 fetch_fsmap finished waiting for FSMap version 25 2024-04-23T10:26:02.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.951+0000 7f6ac6c23740 10 client.15354 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-23T10:26:02.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.951+0000 7f6ac6c23740 20 client.15354 populate_metadata read hostname 'smithi097' 2024-04-23T10:26:02.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.951+0000 7f6ac6c23740 1 --2- 172.21.15.97:0/229001235 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x558bb2ccbd50 0x558bb2cc8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:02.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.951+0000 7f6ac6c23740 4 client.15354 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T10:26:02.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.951+0000 7f6ac6c23740 1 -- 172.21.15.97:0/229001235 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "status"}) -- 0x558bb2bb24e0 con 0x558bb2ccbd50 2024-04-23T10:26:02.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.951+0000 7f6ac1bcd640 1 --2- 172.21.15.97:0/229001235 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x558bb2ccbd50 0x558bb2cc8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:02.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.951+0000 7f6aa57fa640 10 client.15354 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:26:02.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.951+0000 7f6ac1bcd640 1 --2- 172.21.15.97:0/229001235 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x558bb2ccbd50 0x558bb2cc8860 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=667473b9b4e31175 server_cookie=6f3e29df77673375 in_seq=0 out_seq=0 2024-04-23T10:26:02.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.951+0000 7f6aa57fa640 1 -- 172.21.15.97:0/229001235 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x558bb2bb24e0 con 0x558bb2ccbd50 2024-04-23T10:26:02.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.951+0000 7f6aa57fa640 10 client.15354 handle_command_reply: tid=0 2024-04-23T10:26:02.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.951+0000 7f6ac6c23740 1 client.15354 shutdown 2024-04-23T10:26:02.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.952+0000 7f6ac6c23740 1 -- 172.21.15.97:0/229001235 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bb2c4b160 msgr2=0x558bb2c57350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:02.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.952+0000 7f6ac6c23740 1 --2- 172.21.15.97:0/229001235 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bb2c4b160 0x558bb2c57350 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f6ab80cde80 tx=0x7f6ab809beb0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.952+0000 7f6ac6c23740 1 -- 172.21.15.97:0/229001235 shutdown_connections 2024-04-23T10:26:02.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.952+0000 7f6ac6c23740 1 --2- 172.21.15.97:0/229001235 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x558bb2ccbd50 0x558bb2cc8860 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.952+0000 7f6ac6c23740 1 --2- 172.21.15.97:0/229001235 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558bb2c4bb20 0x558bb2c5a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.952+0000 7f6aa57fa640 0 client.15354 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:26:02.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.952+0000 7f6ac6c23740 1 --2- 172.21.15.97:0/229001235 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558bb2c4b160 0x558bb2c57350 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.952+0000 7f6ac6c23740 1 --2- 172.21.15.97:0/229001235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bb2c49f90 0x558bb2c56e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.952+0000 7f6ac6c23740 1 -- 172.21.15.97:0/229001235 >> 172.21.15.97:0/229001235 conn(0x558bb2c41070 msgr2=0x558bb2c4ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:02.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.952+0000 7f6ac6c23740 1 -- 172.21.15.97:0/229001235 shutdown_connections 2024-04-23T10:26:02.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.952+0000 7f6ac6c23740 1 -- 172.21.15.97:0/229001235 wait complete. 2024-04-23T10:26:02.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.952+0000 7f6ac6c23740 20 client.15354 trim_cache size 0 max 16384 2024-04-23T10:26:02.959 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-23T10:26:02.959 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "9b22af88-0159-11ef-bc93-c7b262605968", 2024-04-23T10:26:02.959 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2024-04-23T10:26:02.959 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 24488, 2024-04-23T10:26:02.959 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-04-23T10:26:02.959 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-04-23T10:26:02.959 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-04-23T10:26:02.959 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 115.39660848299999, 2024-04-23T10:26:02.959 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 25, 2024-04-23T10:26:02.960 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 88, 2024-04-23T10:26:02.960 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 86, 2024-04-23T10:26:02.960 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 121.95370162499999 2024-04-23T10:26:02.960 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-23T10:26:02.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.952+0000 7f6ac3e31640 1 -- 172.21.15.97:0/3498181498 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f6aa80a3bb0 msgr2=0x7f6aa80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:02.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.952+0000 7f6ac3e31640 1 --2- 172.21.15.97:0/3498181498 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f6aa80a3bb0 0x7f6aa80a6070 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6ab4005c40 tx=0x7f6ab4009dd0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.952+0000 7f6ac3e31640 1 -- 172.21.15.97:0/3498181498 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6abc105360 msgr2=0x7f6abc1987b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:02.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.952+0000 7f6ac3e31640 1 --2- 172.21.15.97:0/3498181498 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6abc105360 0x7f6abc1987b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f6ab800d770 tx=0x7f6ab800dc30 comp rx=0 tx=0).stop 2024-04-23T10:26:02.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.952+0000 7f6ac3e31640 1 -- 172.21.15.97:0/3498181498 shutdown_connections 2024-04-23T10:26:02.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.952+0000 7f6ac3e31640 1 --2- 172.21.15.97:0/3498181498 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f6aa80a3bb0 0x7f6aa80a6070 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.952+0000 7f6ac3e31640 1 --2- 172.21.15.97:0/3498181498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6abc107c90 0x7f6abc19f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.952+0000 7f6ac3e31640 1 --2- 172.21.15.97:0/3498181498 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6abc105360 0x7f6abc1987b0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.952+0000 7f6ac3e31640 1 --2- 172.21.15.97:0/3498181498 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6abc102a30 0x7f6abc198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:02.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.952+0000 7f6ac3e31640 1 -- 172.21.15.97:0/3498181498 >> 172.21.15.97:0/3498181498 conn(0x7f6abc0fc440 msgr2=0x7f6abc103550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:02.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.952+0000 7f6ac3e31640 1 -- 172.21.15.97:0/3498181498 shutdown_connections 2024-04-23T10:26:02.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:02.952+0000 7f6ac3e31640 1 -- 172.21.15.97:0/3498181498 wait complete. 2024-04-23T10:26:02.962 INFO:teuthology.run_tasks:Running task sequential... 2024-04-23T10:26:02.972 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-23T10:26:02.972 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-23T10:26:02.972 INFO:teuthology.task.print:Enabling mgr modules 2024-04-23T10:26:02.973 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-04-23T10:26:02.973 INFO:teuthology.task.exec:Executing custom commands... 2024-04-23T10:26:02.973 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi097.front.sepia.ceph.com 2024-04-23T10:26:02.973 DEBUG:teuthology.orchestra.run.smithi097:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-04-23T10:26:03.073 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:03.072+0000 7f1d4734a640 1 -- 172.21.15.97:0/2650715993 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d40102a00 msgr2=0x7f1d40109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:03.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:03.072+0000 7f1d4734a640 1 --2- 172.21.15.97:0/2650715993 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d40102a00 0x7f1d40109320 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f1d38009ca0 tx=0x7f1d3802a8c0 comp rx=0 tx=0).stop 2024-04-23T10:26:03.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:03.072+0000 7f1d4734a640 1 -- 172.21.15.97:0/2650715993 shutdown_connections 2024-04-23T10:26:03.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:03.072+0000 7f1d4734a640 1 --2- 172.21.15.97:0/2650715993 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d40102a00 0x7f1d40109320 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:03.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:03.072+0000 7f1d4734a640 1 --2- 172.21.15.97:0/2650715993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d40102040 0x7f1d401024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:03.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:03.072+0000 7f1d4734a640 1 --2- 172.21.15.97:0/2650715993 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d40100e60 0x7f1d40101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:03.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:03.072+0000 7f1d4734a640 1 -- 172.21.15.97:0/2650715993 >> 172.21.15.97:0/2650715993 conn(0x7f1d400fc420 msgr2=0x7f1d400fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:03.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:03.072+0000 7f1d4734a640 1 -- 172.21.15.97:0/2650715993 shutdown_connections 2024-04-23T10:26:03.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:03.072+0000 7f1d4734a640 1 -- 172.21.15.97:0/2650715993 wait complete. 2024-04-23T10:26:03.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:03.073+0000 7f1d4734a640 1 Processor -- start 2024-04-23T10:26:03.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:03.073+0000 7f1d4734a640 1 -- start start 2024-04-23T10:26:03.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:03.073+0000 7f1d4734a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d40100e60 0x7f1d4006b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:03.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:03.073+0000 7f1d4734a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d40102040 0x7f1d4019cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:03.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:03.073+0000 7f1d4734a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d40102a00 0x7f1d4019f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:03.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:03.073+0000 7f1d4734a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1d4010c9f0 con 0x7f1d40102a00 2024-04-23T10:26:03.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:03.073+0000 7f1d4734a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1d4010c870 con 0x7f1d40102040 2024-04-23T10:26:03.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:03.073+0000 7f1d4734a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1d4010cb70 con 0x7f1d40100e60 2024-04-23T10:26:03.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:03.073+0000 7f1d458e7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d40102a00 0x7f1d4019f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:03.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:03.073+0000 7f1d458e7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d40102a00 0x7f1d4019f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46954/0 (socket says 172.21.15.97:46954) 2024-04-23T10:26:03.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:03.073+0000 7f1d458e7640 1 -- 172.21.15.97:0/1966818331 learned_addr learned my addr 172.21.15.97:0/1966818331 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:03.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:03.073+0000 7f1d448e5640 1 --2- 172.21.15.97:0/1966818331 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d40102040 0x7f1d4019cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:03.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:03.073+0000 7f1d450e6640 1 --2- 172.21.15.97:0/1966818331 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d40100e60 0x7f1d4006b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:03.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:03.073+0000 7f1d458e7640 1 -- 172.21.15.97:0/1966818331 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d40100e60 msgr2=0x7f1d4006b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:03.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:03.073+0000 7f1d458e7640 1 --2- 172.21.15.97:0/1966818331 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d40100e60 0x7f1d4006b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:03.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:03.073+0000 7f1d458e7640 1 -- 172.21.15.97:0/1966818331 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d40102040 msgr2=0x7f1d4019cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:03.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:03.073+0000 7f1d458e7640 1 --2- 172.21.15.97:0/1966818331 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d40102040 0x7f1d4019cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:03.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:03.073+0000 7f1d458e7640 1 -- 172.21.15.97:0/1966818331 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d4019fc30 con 0x7f1d40102a00 2024-04-23T10:26:03.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:03.073+0000 7f1d458e7640 1 --2- 172.21.15.97:0/1966818331 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d40102a00 0x7f1d4019f370 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f1d38031a20 tx=0x7f1d38003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:03.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:03.074+0000 7f1d367fc640 1 -- 172.21.15.97:0/1966818331 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d38004240 con 0x7f1d40102a00 2024-04-23T10:26:03.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:03.074+0000 7f1d367fc640 1 -- 172.21.15.97:0/1966818331 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d38003e10 con 0x7f1d40102a00 2024-04-23T10:26:03.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:03.074+0000 7f1d4734a640 1 -- 172.21.15.97:0/1966818331 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d4019fec0 con 0x7f1d40102a00 2024-04-23T10:26:03.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:03.074+0000 7f1d367fc640 1 -- 172.21.15.97:0/1966818331 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d380305c0 con 0x7f1d40102a00 2024-04-23T10:26:03.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:03.074+0000 7f1d4734a640 1 -- 172.21.15.97:0/1966818331 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d401a03d0 con 0x7f1d40102a00 2024-04-23T10:26:03.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:03.074+0000 7f1d4734a640 1 -- 172.21.15.97:0/1966818331 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d40108130 con 0x7f1d40102a00 2024-04-23T10:26:03.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:03.076+0000 7f1d367fc640 1 -- 172.21.15.97:0/1966818331 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1d3802f660 con 0x7f1d40102a00 2024-04-23T10:26:03.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:03.076+0000 7f1d367fc640 1 --2- 172.21.15.97:0/1966818331 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f1d2c0a3cd0 0x7f1d2c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:03.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:03.076+0000 7f1d450e6640 1 --2- 172.21.15.97:0/1966818331 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f1d2c0a3cd0 0x7f1d2c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:03.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:03.076+0000 7f1d367fc640 1 -- 172.21.15.97:0/1966818331 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f1d380f1e30 con 0x7f1d40102a00 2024-04-23T10:26:03.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:03.077+0000 7f1d450e6640 1 --2- 172.21.15.97:0/1966818331 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f1d2c0a3cd0 0x7f1d2c0a6190 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c005fd0 tx=0x7f1d3c005ce0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:03.078 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:03.080+0000 7f1d367fc640 1 -- 172.21.15.97:0/1966818331 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1d380be840 con 0x7f1d40102a00 2024-04-23T10:26:03.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:02 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/3278193651' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T10:26:03.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:03.306+0000 7f1d4734a640 1 -- 172.21.15.97:0/1966818331 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f1d4002d5c0 con 0x7f1d40102a00 2024-04-23T10:26:03.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:02 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/3278193651' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T10:26:03.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:02 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/3278193651' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T10:26:03.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:03.989+0000 7f1d367fc640 1 -- 172.21.15.97:0/1966818331 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f1d380efe20 con 0x7f1d40102a00 2024-04-23T10:26:03.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:03.992+0000 7f1d4734a640 1 -- 172.21.15.97:0/1966818331 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f1d2c0a3cd0 msgr2=0x7f1d2c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:03.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:03.992+0000 7f1d4734a640 1 --2- 172.21.15.97:0/1966818331 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f1d2c0a3cd0 0x7f1d2c0a6190 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c005fd0 tx=0x7f1d3c005ce0 comp rx=0 tx=0).stop 2024-04-23T10:26:03.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:03.992+0000 7f1d4734a640 1 -- 172.21.15.97:0/1966818331 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d40102a00 msgr2=0x7f1d4019f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:03.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:03.992+0000 7f1d4734a640 1 --2- 172.21.15.97:0/1966818331 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d40102a00 0x7f1d4019f370 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f1d38031a20 tx=0x7f1d38003c20 comp rx=0 tx=0).stop 2024-04-23T10:26:03.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:03.992+0000 7f1d4734a640 1 -- 172.21.15.97:0/1966818331 shutdown_connections 2024-04-23T10:26:03.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:03.992+0000 7f1d4734a640 1 --2- 172.21.15.97:0/1966818331 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f1d2c0a3cd0 0x7f1d2c0a6190 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:03.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:03.992+0000 7f1d4734a640 1 --2- 172.21.15.97:0/1966818331 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d40102a00 0x7f1d4019f370 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:03.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:03.992+0000 7f1d4734a640 1 --2- 172.21.15.97:0/1966818331 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d40102040 0x7f1d4019cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:03.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:03.992+0000 7f1d4734a640 1 --2- 172.21.15.97:0/1966818331 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d40100e60 0x7f1d4006b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:03.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:03.993+0000 7f1d4734a640 1 -- 172.21.15.97:0/1966818331 >> 172.21.15.97:0/1966818331 conn(0x7f1d400fc420 msgr2=0x7f1d40105300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:03.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:03.993+0000 7f1d4734a640 1 -- 172.21.15.97:0/1966818331 shutdown_connections 2024-04-23T10:26:03.992 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:03.993+0000 7f1d4734a640 1 -- 172.21.15.97:0/1966818331 wait complete. 2024-04-23T10:26:04.011 DEBUG:teuthology.orchestra.run.smithi097:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-04-23T10:26:04.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.117+0000 7fc0dd744640 1 -- 172.21.15.97:0/3447691830 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0d8102000 msgr2=0x7fc0d8102420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:04.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.117+0000 7fc0dd744640 1 --2- 172.21.15.97:0/3447691830 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0d8102000 0x7fc0d8102420 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc0c8009900 tx=0x7fc0c802a410 comp rx=0 tx=0).stop 2024-04-23T10:26:04.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.117+0000 7fc0dd744640 1 -- 172.21.15.97:0/3447691830 shutdown_connections 2024-04-23T10:26:04.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.117+0000 7fc0dd744640 1 --2- 172.21.15.97:0/3447691830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc0d8103bb0 0x7fc0d810a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:04.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.117+0000 7fc0dd744640 1 --2- 172.21.15.97:0/3447691830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0d81031f0 0x7fc0d8103670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:04.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.117+0000 7fc0dd744640 1 --2- 172.21.15.97:0/3447691830 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0d8102000 0x7fc0d8102420 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:04.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.117+0000 7fc0dd744640 1 -- 172.21.15.97:0/3447691830 >> 172.21.15.97:0/3447691830 conn(0x7fc0d80fd570 msgr2=0x7fc0d80ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:04.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.117+0000 7fc0dd744640 1 -- 172.21.15.97:0/3447691830 shutdown_connections 2024-04-23T10:26:04.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.117+0000 7fc0dd744640 1 -- 172.21.15.97:0/3447691830 wait complete. 2024-04-23T10:26:04.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.118+0000 7fc0dd744640 1 Processor -- start 2024-04-23T10:26:04.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.118+0000 7fc0dd744640 1 -- start start 2024-04-23T10:26:04.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.118+0000 7fc0dd744640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0d8102000 0x7fc0d81981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:04.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.118+0000 7fc0dd744640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0d81031f0 0x7fc0d81986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:04.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.118+0000 7fc0dd744640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc0d8103bb0 0x7fc0d819f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:04.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.118+0000 7fc0dd744640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d810db40 con 0x7fc0d81031f0 2024-04-23T10:26:04.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.118+0000 7fc0dd744640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d810d9c0 con 0x7fc0d8103bb0 2024-04-23T10:26:04.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.118+0000 7fc0dd744640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d810dcc0 con 0x7fc0d8102000 2024-04-23T10:26:04.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.118+0000 7fc0cffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0d81031f0 0x7fc0d81986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:04.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.118+0000 7fc0cffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0d81031f0 0x7fc0d81986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46994/0 (socket says 172.21.15.97:46994) 2024-04-23T10:26:04.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.118+0000 7fc0d6ffd640 1 --2- 172.21.15.97:0/4228545334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0d8102000 0x7fc0d81981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:04.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.118+0000 7fc0cffff640 1 -- 172.21.15.97:0/4228545334 learned_addr learned my addr 172.21.15.97:0/4228545334 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:04.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.118+0000 7fc0d77fe640 1 --2- 172.21.15.97:0/4228545334 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc0d8103bb0 0x7fc0d819f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:04.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.119+0000 7fc0d6ffd640 1 -- 172.21.15.97:0/4228545334 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc0d8103bb0 msgr2=0x7fc0d819f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:04.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.119+0000 7fc0d6ffd640 1 --2- 172.21.15.97:0/4228545334 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc0d8103bb0 0x7fc0d819f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:04.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.119+0000 7fc0d6ffd640 1 -- 172.21.15.97:0/4228545334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0d81031f0 msgr2=0x7fc0d81986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:04.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.119+0000 7fc0d6ffd640 1 --2- 172.21.15.97:0/4228545334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0d81031f0 0x7fc0d81986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:04.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.119+0000 7fc0d6ffd640 1 -- 172.21.15.97:0/4228545334 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0d819fd70 con 0x7fc0d8102000 2024-04-23T10:26:04.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.119+0000 7fc0cffff640 1 --2- 172.21.15.97:0/4228545334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0d81031f0 0x7fc0d81986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T10:26:04.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.119+0000 7fc0d6ffd640 1 --2- 172.21.15.97:0/4228545334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0d8102000 0x7fc0d81981a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc0c8006010 tx=0x7fc0c80042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:04.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.119+0000 7fc0d4ff9640 1 -- 172.21.15.97:0/4228545334 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c803f030 con 0x7fc0d8102000 2024-04-23T10:26:04.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.119+0000 7fc0dd744640 1 -- 172.21.15.97:0/4228545334 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0d81a0000 con 0x7fc0d8102000 2024-04-23T10:26:04.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.119+0000 7fc0d4ff9640 1 -- 172.21.15.97:0/4228545334 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0c8031910 con 0x7fc0d8102000 2024-04-23T10:26:04.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.119+0000 7fc0d4ff9640 1 -- 172.21.15.97:0/4228545334 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c8035070 con 0x7fc0d8102000 2024-04-23T10:26:04.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.119+0000 7fc0dd744640 1 -- 172.21.15.97:0/4228545334 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0d81a0540 con 0x7fc0d8102000 2024-04-23T10:26:04.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.120+0000 7fc0dd744640 1 -- 172.21.15.97:0/4228545334 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0d8109320 con 0x7fc0d8102000 2024-04-23T10:26:04.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.121+0000 7fc0d4ff9640 1 -- 172.21.15.97:0/4228545334 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7fc0c8042030 con 0x7fc0d8102000 2024-04-23T10:26:04.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.122+0000 7fc0d4ff9640 1 --2- 172.21.15.97:0/4228545334 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fc0c40a3ca0 0x7fc0c40a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:04.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.122+0000 7fc0cffff640 1 -- 172.21.15.97:0/4228545334 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fc0c40a3ca0 msgr2=0x7fc0c40a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/3149043749 2024-04-23T10:26:04.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.122+0000 7fc0d4ff9640 1 -- 172.21.15.97:0/4228545334 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fc0c80f0920 con 0x7fc0d8102000 2024-04-23T10:26:04.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.122+0000 7fc0cffff640 1 --2- 172.21.15.97:0/4228545334 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fc0c40a3ca0 0x7fc0c40a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T10:26:04.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.126+0000 7fc0d4ff9640 1 -- 172.21.15.97:0/4228545334 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc0c80bcce0 con 0x7fc0d8102000 2024-04-23T10:26:04.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.312+0000 7fc0dd744640 1 -- 172.21.15.97:0/4228545334 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7fc0d802d5c0 con 0x7fc0d8102000 2024-04-23T10:26:04.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.318+0000 7fc0d4ff9640 1 -- 172.21.15.97:0/4228545334 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v29)=0 v29) ==== 138+0+0 (secure 0 0 0) 0x7fc0c80c0fe0 con 0x7fc0d8102000 2024-04-23T10:26:04.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.320+0000 7fc0dd744640 1 -- 172.21.15.97:0/4228545334 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fc0c40a3ca0 msgr2=0x7fc0c40a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T10:26:04.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.320+0000 7fc0dd744640 1 --2- 172.21.15.97:0/4228545334 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fc0c40a3ca0 0x7fc0c40a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:04.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.320+0000 7fc0dd744640 1 -- 172.21.15.97:0/4228545334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0d8102000 msgr2=0x7fc0d81981a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:04.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.320+0000 7fc0dd744640 1 --2- 172.21.15.97:0/4228545334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0d8102000 0x7fc0d81981a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc0c8006010 tx=0x7fc0c80042f0 comp rx=0 tx=0).stop 2024-04-23T10:26:04.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.320+0000 7fc0dd744640 1 -- 172.21.15.97:0/4228545334 shutdown_connections 2024-04-23T10:26:04.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.320+0000 7fc0dd744640 1 --2- 172.21.15.97:0/4228545334 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fc0c40a3ca0 0x7fc0c40a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:04.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.320+0000 7fc0dd744640 1 --2- 172.21.15.97:0/4228545334 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc0d8103bb0 0x7fc0d819f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:04.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.320+0000 7fc0dd744640 1 --2- 172.21.15.97:0/4228545334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc0d81031f0 0x7fc0d81986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:04.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.320+0000 7fc0dd744640 1 --2- 172.21.15.97:0/4228545334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0d8102000 0x7fc0d81981a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:04.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.320+0000 7fc0dd744640 1 -- 172.21.15.97:0/4228545334 >> 172.21.15.97:0/4228545334 conn(0x7fc0d80fd570 msgr2=0x7fc0d81064f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:04.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.320+0000 7fc0dd744640 1 -- 172.21.15.97:0/4228545334 shutdown_connections 2024-04-23T10:26:04.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.320+0000 7fc0dd744640 1 -- 172.21.15.97:0/4228545334 wait complete. 2024-04-23T10:26:04.329 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:03 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/1966818331' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-23T10:26:04.329 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:03 smithi097 ceph-mon[29197]: pgmap v350: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-23T10:26:04.330 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:26:04 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: ignoring --setuser ceph since I am not root 2024-04-23T10:26:04.330 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:26:04 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: ignoring --setgroup ceph since I am not root 2024-04-23T10:26:04.330 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:26:04 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:26:04.178+0000 7ff779856200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-23T10:26:04.330 DEBUG:teuthology.orchestra.run.smithi097:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-04-23T10:26:04.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:03 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/1966818331' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-23T10:26:04.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:03 smithi196 ceph-mon[32919]: pgmap v350: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-23T10:26:04.342 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 23 10:26:04 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-z[34770]: ignoring --setuser ceph since I am not root 2024-04-23T10:26:04.342 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 23 10:26:04 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-z[34770]: ignoring --setgroup ceph since I am not root 2024-04-23T10:26:04.342 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 23 10:26:04 smithi196 ceph-mgr[34774]: -- 172.21.15.196:0/894098597 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56113e4b05a0 con 0x56113e484c00 2024-04-23T10:26:04.342 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 23 10:26:04 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-z[34770]: 2024-04-23T10:26:04.171+0000 7f10c1c6e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-23T10:26:04.357 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:03 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/1966818331' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-23T10:26:04.357 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:03 smithi183 ceph-mon[32317]: pgmap v350: 97 pgs: 97 active+clean; 644 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-23T10:26:04.357 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 23 10:26:04 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-y[33699]: ignoring --setuser ceph since I am not root 2024-04-23T10:26:04.357 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 23 10:26:04 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-y[33699]: ignoring --setgroup ceph since I am not root 2024-04-23T10:26:04.357 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 23 10:26:04 smithi183 ceph-mgr[33703]: -- 172.21.15.183:0/1892326531 <== mon.2 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556058aba5a0 con 0x556058a8f000 2024-04-23T10:26:04.357 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 23 10:26:04 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-y[33699]: 2024-04-23T10:26:04.173+0000 7fe32ad0e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-23T10:26:04.357 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 23 10:26:04 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-y[33699]: 2024-04-23T10:26:04.357+0000 7fe32ad0e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-23T10:26:04.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.432+0000 7f27fe4bb640 1 -- 172.21.15.97:0/1908058814 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27f8102a00 msgr2=0x7f27f8109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:04.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.432+0000 7f27fe4bb640 1 --2- 172.21.15.97:0/1908058814 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27f8102a00 0x7f27f8109320 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f27f0009ca0 tx=0x7f27f002a8c0 comp rx=0 tx=0).stop 2024-04-23T10:26:04.433 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.432+0000 7f27fe4bb640 1 -- 172.21.15.97:0/1908058814 shutdown_connections 2024-04-23T10:26:04.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.432+0000 7f27fe4bb640 1 --2- 172.21.15.97:0/1908058814 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27f8102a00 0x7f27f8109320 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:04.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.432+0000 7f27fe4bb640 1 --2- 172.21.15.97:0/1908058814 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27f8102040 0x7f27f81024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:04.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.432+0000 7f27fe4bb640 1 --2- 172.21.15.97:0/1908058814 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27f8100e60 0x7f27f8101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:04.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.432+0000 7f27fe4bb640 1 -- 172.21.15.97:0/1908058814 >> 172.21.15.97:0/1908058814 conn(0x7f27f80fc420 msgr2=0x7f27f80fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:04.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.432+0000 7f27fe4bb640 1 -- 172.21.15.97:0/1908058814 shutdown_connections 2024-04-23T10:26:04.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.432+0000 7f27fe4bb640 1 -- 172.21.15.97:0/1908058814 wait complete. 2024-04-23T10:26:04.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.433+0000 7f27fe4bb640 1 Processor -- start 2024-04-23T10:26:04.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.433+0000 7f27fe4bb640 1 -- start start 2024-04-23T10:26:04.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.433+0000 7f27fe4bb640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27f8100e60 0x7f27f8197010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:04.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.433+0000 7f27fe4bb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27f8102040 0x7f27f8197550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:04.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.433+0000 7f27fe4bb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27f8102a00 0x7f27f819e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:04.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.433+0000 7f27fe4bb640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f27f810ca00 con 0x7f27f8102a00 2024-04-23T10:26:04.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.433+0000 7f27fe4bb640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f27f810c880 con 0x7f27f8100e60 2024-04-23T10:26:04.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.433+0000 7f27fe4bb640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f27f810cb80 con 0x7f27f8102040 2024-04-23T10:26:04.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.433+0000 7f27fca58640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27f8102a00 0x7f27f819e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:04.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.433+0000 7f27fca58640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27f8102a00 0x7f27f819e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47016/0 (socket says 172.21.15.97:47016) 2024-04-23T10:26:04.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.433+0000 7f27fca58640 1 -- 172.21.15.97:0/2879922610 learned_addr learned my addr 172.21.15.97:0/2879922610 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:04.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.433+0000 7f27f7fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27f8100e60 0x7f27f8197010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:04.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.433+0000 7f27f77fe640 1 --2- 172.21.15.97:0/2879922610 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27f8102040 0x7f27f8197550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:04.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.434+0000 7f27fca58640 1 -- 172.21.15.97:0/2879922610 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27f8102040 msgr2=0x7f27f8197550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:04.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.434+0000 7f27fca58640 1 --2- 172.21.15.97:0/2879922610 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27f8102040 0x7f27f8197550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:04.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.434+0000 7f27fca58640 1 -- 172.21.15.97:0/2879922610 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27f8100e60 msgr2=0x7f27f8197010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:04.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.434+0000 7f27fca58640 1 --2- 172.21.15.97:0/2879922610 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27f8100e60 0x7f27f8197010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:04.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.434+0000 7f27fca58640 1 -- 172.21.15.97:0/2879922610 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27f819ec30 con 0x7f27f8102a00 2024-04-23T10:26:04.435 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.434+0000 7f27fca58640 1 --2- 172.21.15.97:0/2879922610 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27f8102a00 0x7f27f819e5c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f27f00097b0 tx=0x7f27f0031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:04.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.434+0000 7f27f57fa640 1 -- 172.21.15.97:0/2879922610 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f0003bf0 con 0x7f27f8102a00 2024-04-23T10:26:04.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.434+0000 7f27f57fa640 1 -- 172.21.15.97:0/2879922610 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27f0003d90 con 0x7f27f8102a00 2024-04-23T10:26:04.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.434+0000 7f27fe4bb640 1 -- 172.21.15.97:0/2879922610 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27f819eec0 con 0x7f27f8102a00 2024-04-23T10:26:04.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.434+0000 7f27f57fa640 1 -- 172.21.15.97:0/2879922610 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f002fc70 con 0x7f27f8102a00 2024-04-23T10:26:04.436 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.434+0000 7f27fe4bb640 1 -- 172.21.15.97:0/2879922610 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27f819f380 con 0x7f27f8102a00 2024-04-23T10:26:04.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.435+0000 7f27fe4bb640 1 -- 172.21.15.97:0/2879922610 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27f8108200 con 0x7f27f8102a00 2024-04-23T10:26:04.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.436+0000 7f27f57fa640 1 -- 172.21.15.97:0/2879922610 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f27f0004b40 con 0x7f27f8102a00 2024-04-23T10:26:04.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.437+0000 7f27f57fa640 1 --2- 172.21.15.97:0/2879922610 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f27e00a3ca0 0x7f27e00a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:04.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.437+0000 7f27f7fff640 1 -- 172.21.15.97:0/2879922610 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f27e00a3ca0 msgr2=0x7f27e00a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/3149043749 2024-04-23T10:26:04.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.437+0000 7f27f57fa640 1 -- 172.21.15.97:0/2879922610 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f27f00f1b70 con 0x7f27f8102a00 2024-04-23T10:26:04.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.437+0000 7f27f7fff640 1 --2- 172.21.15.97:0/2879922610 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f27e00a3ca0 0x7f27e00a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T10:26:04.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.444+0000 7f27f57fa640 1 -- 172.21.15.97:0/2879922610 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f27f00be580 con 0x7f27f8102a00 2024-04-23T10:26:04.610 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:26:04 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:26:04.368+0000 7ff779856200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-23T10:26:04.610 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:26:04 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:26:04.480+0000 7ff779856200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-23T10:26:04.630 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.631+0000 7f27fe4bb640 1 -- 172.21.15.97:0/2879922610 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f27f8062c60 con 0x7f27f8102a00 2024-04-23T10:26:04.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.637+0000 7f27f57fa640 1 -- 172.21.15.97:0/2879922610 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v30)=0 v30) ==== 133+0+0 (secure 0 0 0) 0x7f27f00bde80 con 0x7f27f8102a00 2024-04-23T10:26:04.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.638+0000 7f27f7fff640 1 -- 172.21.15.97:0/2879922610 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f27e00a3ca0 msgr2=0x7f27e00a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/3149043749 2024-04-23T10:26:04.636 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.638+0000 7f27f7fff640 1 --2- 172.21.15.97:0/2879922610 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f27e00a3ca0 0x7f27e00a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-23T10:26:04.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.639+0000 7f27fe4bb640 1 -- 172.21.15.97:0/2879922610 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f27e00a3ca0 msgr2=0x7f27e00a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T10:26:04.638 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.639+0000 7f27fe4bb640 1 --2- 172.21.15.97:0/2879922610 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f27e00a3ca0 0x7f27e00a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:04.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.639+0000 7f27fe4bb640 1 -- 172.21.15.97:0/2879922610 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27f8102a00 msgr2=0x7f27f819e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:04.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.639+0000 7f27fe4bb640 1 --2- 172.21.15.97:0/2879922610 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27f8102a00 0x7f27f819e5c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f27f00097b0 tx=0x7f27f0031840 comp rx=0 tx=0).stop 2024-04-23T10:26:04.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.639+0000 7f27fe4bb640 1 -- 172.21.15.97:0/2879922610 shutdown_connections 2024-04-23T10:26:04.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.639+0000 7f27fe4bb640 1 --2- 172.21.15.97:0/2879922610 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f27e00a3ca0 0x7f27e00a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:04.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.639+0000 7f27fe4bb640 1 --2- 172.21.15.97:0/2879922610 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27f8102a00 0x7f27f819e5c0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:04.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.639+0000 7f27fe4bb640 1 --2- 172.21.15.97:0/2879922610 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27f8102040 0x7f27f8197550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:04.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.639+0000 7f27fe4bb640 1 --2- 172.21.15.97:0/2879922610 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27f8100e60 0x7f27f8197010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:04.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.639+0000 7f27fe4bb640 1 -- 172.21.15.97:0/2879922610 >> 172.21.15.97:0/2879922610 conn(0x7f27f80fc420 msgr2=0x7f27f81053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:04.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.639+0000 7f27fe4bb640 1 -- 172.21.15.97:0/2879922610 shutdown_connections 2024-04-23T10:26:04.639 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:04.639+0000 7f27fe4bb640 1 -- 172.21.15.97:0/2879922610 wait complete. 2024-04-23T10:26:04.649 INFO:teuthology.run_tasks:Running task kclient... 2024-04-23T10:26:04.672 INFO:tasks.kclient:Mounting kernel clients... 2024-04-23T10:26:04.672 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-23T10:26:04.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T10:26:04.672 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-04-23T10:26:04.673 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 23 10:26:04 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-y[33699]: 2024-04-23T10:26:04.460+0000 7fe32ad0e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-23T10:26:04.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T10:26:04.687 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link delete ceph-brx 2024-04-23T10:26:04.753 INFO:teuthology.orchestra.run.smithi097.stderr:Cannot find device "ceph-brx" 2024-04-23T10:26:04.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T10:26:04.754 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'nowsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-04-23T10:26:04.755 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-23T10:26:04.755 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-23T10:26:04.755 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-23T10:26:04.767 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 23 10:26:04 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-z[34770]: 2024-04-23T10:26:04.355+0000 7f10c1c6e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-23T10:26:04.767 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 23 10:26:04 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-z[34770]: 2024-04-23T10:26:04.459+0000 7f10c1c6e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-23T10:26:04.993 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:26:04 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:26:04.801+0000 7ff779856200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-23T10:26:05.004 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:26:04 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:26:04.891+0000 7ff779856200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-23T10:26:05.048 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T10:26:05.091 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 23 10:26:04 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-z[34770]: 2024-04-23T10:26:04.768+0000 7f10c1c6e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-23T10:26:05.091 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 23 10:26:04 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-z[34770]: 2024-04-23T10:26:04.853+0000 7f10c1c6e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-23T10:26:05.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:04 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/1966818331' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-23T10:26:05.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:04 smithi196 ceph-mon[32919]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-23T10:26:05.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:04 smithi196 ceph-mon[32919]: from='client.? ' entity='client.admin' 2024-04-23T10:26:05.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:04 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/2879922610' entity='client.admin' 2024-04-23T10:26:05.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:04 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/1966818331' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-23T10:26:05.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:04 smithi183 ceph-mon[32317]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-23T10:26:05.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:04 smithi183 ceph-mon[32317]: from='client.? ' entity='client.admin' 2024-04-23T10:26:05.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:04 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/2879922610' entity='client.admin' 2024-04-23T10:26:05.150 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 23 10:26:04 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-y[33699]: 2024-04-23T10:26:04.770+0000 7fe32ad0e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-23T10:26:05.150 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 23 10:26:04 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-y[33699]: 2024-04-23T10:26:04.856+0000 7fe32ad0e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-23T10:26:05.290 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:04 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/1966818331' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-23T10:26:05.290 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:04 smithi097 ceph-mon[29197]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-23T10:26:05.290 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:04 smithi097 ceph-mon[29197]: from='client.? ' entity='client.admin' 2024-04-23T10:26:05.290 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:04 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/2879922610' entity='client.admin' 2024-04-23T10:26:06.341 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 23 10:26:06 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-z[34770]: 2024-04-23T10:26:06.031+0000 7f10c1c6e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-23T10:26:06.342 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 23 10:26:06 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-z[34770]: 2024-04-23T10:26:06.201+0000 7f10c1c6e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-23T10:26:06.385 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 23 10:26:06 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-y[33699]: 2024-04-23T10:26:06.044+0000 7fe32ad0e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-23T10:26:06.385 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 23 10:26:06 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-y[33699]: 2024-04-23T10:26:06.214+0000 7fe32ad0e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-23T10:26:06.385 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 23 10:26:06 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-y[33699]: 2024-04-23T10:26:06.386+0000 7fe32ad0e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-23T10:26:06.464 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:26:06 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:26:06.113+0000 7ff779856200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-23T10:26:06.464 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:26:06 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:26:06.286+0000 7ff779856200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-23T10:26:06.464 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:26:06 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:26:06.464+0000 7ff779856200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-23T10:26:06.662 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 23 10:26:06 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-z[34770]: 2024-04-23T10:26:06.373+0000 7f10c1c6e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-23T10:26:06.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:06.900+0000 7f0f78698700 1 -- 172.21.15.97:0/660282826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f70107990 msgr2=0x7f0f70107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:06.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:06.900+0000 7f0f78698700 1 --2- 172.21.15.97:0/660282826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f70107990 0x7f0f70107d70 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c009990 tx=0x7f0f6c009ca0 comp rx=0 tx=0).stop 2024-04-23T10:26:06.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:06.901+0000 7f0f78698700 1 -- 172.21.15.97:0/660282826 shutdown_connections 2024-04-23T10:26:06.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:06.901+0000 7f0f78698700 1 --2- 172.21.15.97:0/660282826 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f70102300 0x7f0f7010e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:06.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:06.901+0000 7f0f78698700 1 --2- 172.21.15.97:0/660282826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f70107990 0x7f0f70107d70 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:06.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:06.901+0000 7f0f78698700 1 --2- 172.21.15.97:0/660282826 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f70101940 0x7f0f70101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:06.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:06.901+0000 7f0f78698700 1 -- 172.21.15.97:0/660282826 >> 172.21.15.97:0/660282826 conn(0x7f0f700788f0 msgr2=0x7f0f70078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:06.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:06.901+0000 7f0f78698700 1 -- 172.21.15.97:0/660282826 shutdown_connections 2024-04-23T10:26:06.900 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:06.901+0000 7f0f78698700 1 -- 172.21.15.97:0/660282826 wait complete. 2024-04-23T10:26:06.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:06.901+0000 7f0f78698700 1 Processor -- start 2024-04-23T10:26:06.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:06.901+0000 7f0f78698700 1 -- start start 2024-04-23T10:26:06.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:06.902+0000 7f0f78698700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f70101940 0x7f0f70103940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:06.902 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:06.902+0000 7f0f78698700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f70102300 0x7f0f70103e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:06.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:06.902+0000 7f0f78698700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f70107990 0x7f0f701043c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:06.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:06.902+0000 7f0f78698700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0f70111b00 con 0x7f0f70107990 2024-04-23T10:26:06.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:06.902+0000 7f0f78698700 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0f70111980 con 0x7f0f70101940 2024-04-23T10:26:06.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:06.902+0000 7f0f78698700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0f70111c80 con 0x7f0f70102300 2024-04-23T10:26:06.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:06.902+0000 7f0f76434700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f70101940 0x7f0f70103940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:06.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:06.902+0000 7f0f75c33700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f70102300 0x7f0f70103e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:06.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:06.902+0000 7f0f76434700 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f70101940 0x7f0f70103940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.97:37480/0 (socket says 172.21.15.97:37480) 2024-04-23T10:26:06.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:06.902+0000 7f0f75c33700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f70102300 0x7f0f70103e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.97:58680/0 (socket says 172.21.15.97:58680) 2024-04-23T10:26:06.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:06.902+0000 7f0f76434700 1 -- 172.21.15.97:0/1935103005 learned_addr learned my addr 172.21.15.97:0/1935103005 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:06.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:06.902+0000 7f0f76c35700 1 --2- 172.21.15.97:0/1935103005 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f70107990 0x7f0f701043c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:06.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:06.902+0000 7f0f76434700 1 -- 172.21.15.97:0/1935103005 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f70102300 msgr2=0x7f0f70103e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:06.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:06.902+0000 7f0f76434700 1 --2- 172.21.15.97:0/1935103005 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f70102300 0x7f0f70103e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:06.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:06.902+0000 7f0f76434700 1 -- 172.21.15.97:0/1935103005 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f70107990 msgr2=0x7f0f701043c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:06.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:06.902+0000 7f0f76434700 1 --2- 172.21.15.97:0/1935103005 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f70107990 0x7f0f701043c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:06.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:06.902+0000 7f0f76434700 1 -- 172.21.15.97:0/1935103005 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f701a7660 con 0x7f0f70101940 2024-04-23T10:26:06.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:06.902+0000 7f0f75c33700 1 --2- 172.21.15.97:0/1935103005 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f70102300 0x7f0f70103e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T10:26:06.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:06.902+0000 7f0f76434700 1 --2- 172.21.15.97:0/1935103005 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f70101940 0x7f0f70103940 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c00b5c0 tx=0x7f0f6c0058e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:06.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:06.902+0000 7f0f677fe700 1 -- 172.21.15.97:0/1935103005 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c027070 con 0x7f0f70101940 2024-04-23T10:26:06.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:06.903+0000 7f0f78698700 1 -- 172.21.15.97:0/1935103005 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f701a78f0 con 0x7f0f70101940 2024-04-23T10:26:06.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:06.903+0000 7f0f677fe700 1 -- 172.21.15.97:0/1935103005 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f6c003fb0 con 0x7f0f70101940 2024-04-23T10:26:06.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:06.903+0000 7f0f677fe700 1 -- 172.21.15.97:0/1935103005 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c022c80 con 0x7f0f70101940 2024-04-23T10:26:06.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:06.903+0000 7f0f78698700 1 -- 172.21.15.97:0/1935103005 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f701a7da0 con 0x7f0f70101940 2024-04-23T10:26:06.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:06.903+0000 7f0f78698700 1 -- 172.21.15.97:0/1935103005 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f70069730 con 0x7f0f70101940 2024-04-23T10:26:06.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:06.905+0000 7f0f677fe700 1 -- 172.21.15.97:0/1935103005 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f0f6c02a070 con 0x7f0f70101940 2024-04-23T10:26:06.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:06.906+0000 7f0f677fe700 1 --2- 172.21.15.97:0/1935103005 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f0f5c0a3c90 0x7f0f5c0a6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:06.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:06.906+0000 7f0f75c33700 1 -- 172.21.15.97:0/1935103005 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f0f5c0a3c90 msgr2=0x7f0f5c0a6150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/3149043749 2024-04-23T10:26:06.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:06.906+0000 7f0f677fe700 1 -- 172.21.15.97:0/1935103005 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f0f6c0d13e0 con 0x7f0f70101940 2024-04-23T10:26:06.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:06.906+0000 7f0f75c33700 1 --2- 172.21.15.97:0/1935103005 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f0f5c0a3c90 0x7f0f5c0a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T10:26:06.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:06.909+0000 7f0f677fe700 1 -- 172.21.15.97:0/1935103005 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0f6c022e20 con 0x7f0f70101940 2024-04-23T10:26:06.960 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 23 10:26:06 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-z[34770]: 2024-04-23T10:26:06.663+0000 7f10c1c6e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-23T10:26:06.960 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 23 10:26:06 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-z[34770]: 2024-04-23T10:26:06.756+0000 7f10c1c6e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-23T10:26:06.973 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 23 10:26:06 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-y[33699]: 2024-04-23T10:26:06.676+0000 7fe32ad0e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-23T10:26:06.973 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 23 10:26:06 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-y[33699]: 2024-04-23T10:26:06.769+0000 7fe32ad0e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-23T10:26:06.973 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 23 10:26:06 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-y[33699]: 2024-04-23T10:26:06.974+0000 7fe32ad0e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-23T10:26:07.085 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:26:06 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:26:06.763+0000 7ff779856200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-23T10:26:07.085 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:26:06 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:26:06.861+0000 7ff779856200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-23T10:26:07.086 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:26:07 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:26:07.086+0000 7ff779856200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-23T10:26:07.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:07.107+0000 7f0f75c33700 1 -- 172.21.15.97:0/1935103005 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f0f5c0a3c90 msgr2=0x7f0f5c0a6150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/3149043749 2024-04-23T10:26:07.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:07.107+0000 7f0f75c33700 1 --2- 172.21.15.97:0/1935103005 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f0f5c0a3c90 0x7f0f5c0a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-23T10:26:07.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:07.182+0000 7f0f78698700 1 -- 172.21.15.97:0/1935103005 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f0f701a7f70 con 0x7f0f70101940 2024-04-23T10:26:07.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:07.183+0000 7f0f677fe700 1 -- 172.21.15.97:0/1935103005 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) ==== 53+0+73 (secure 0 0 0) 0x7f0f6c013070 con 0x7f0f70101940 2024-04-23T10:26:07.182 INFO:teuthology.orchestra.run.smithi097.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-23T10:26:07.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:07.185+0000 7f0f78698700 1 -- 172.21.15.97:0/1935103005 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f0f5c0a3c90 msgr2=0x7f0f5c0a6150 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T10:26:07.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:07.185+0000 7f0f78698700 1 --2- 172.21.15.97:0/1935103005 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f0f5c0a3c90 0x7f0f5c0a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:07.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:07.185+0000 7f0f78698700 1 -- 172.21.15.97:0/1935103005 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f70101940 msgr2=0x7f0f70103940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:07.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:07.185+0000 7f0f78698700 1 --2- 172.21.15.97:0/1935103005 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f70101940 0x7f0f70103940 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c00b5c0 tx=0x7f0f6c0058e0 comp rx=0 tx=0).stop 2024-04-23T10:26:07.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:07.185+0000 7f0f78698700 1 -- 172.21.15.97:0/1935103005 shutdown_connections 2024-04-23T10:26:07.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:07.185+0000 7f0f78698700 1 --2- 172.21.15.97:0/1935103005 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7f0f5c0a3c90 0x7f0f5c0a6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:07.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:07.185+0000 7f0f78698700 1 --2- 172.21.15.97:0/1935103005 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f70107990 0x7f0f701043c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:07.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:07.185+0000 7f0f78698700 1 --2- 172.21.15.97:0/1935103005 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f70101940 0x7f0f70103940 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:07.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:07.185+0000 7f0f78698700 1 --2- 172.21.15.97:0/1935103005 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f70102300 0x7f0f70103e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:07.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:07.185+0000 7f0f78698700 1 -- 172.21.15.97:0/1935103005 >> 172.21.15.97:0/1935103005 conn(0x7f0f700788f0 msgr2=0x7f0f70105c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:07.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:07.185+0000 7f0f78698700 1 -- 172.21.15.97:0/1935103005 shutdown_connections 2024-04-23T10:26:07.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:07.185+0000 7f0f78698700 1 -- 172.21.15.97:0/1935103005 wait complete. 2024-04-23T10:26:07.233 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 23 10:26:06 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-z[34770]: 2024-04-23T10:26:06.961+0000 7f10c1c6e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-23T10:26:07.233 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 23 10:26:07 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-z[34770]: 2024-04-23T10:26:07.158+0000 7f10c1c6e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-23T10:26:07.233 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 23 10:26:07 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-y[33699]: 2024-04-23T10:26:07.172+0000 7fe32ad0e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-23T10:26:07.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:07 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/1935103005' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-23T10:26:07.360 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:26:07 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:26:07.293+0000 7ff779856200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-23T10:26:07.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:07 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/1935103005' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-23T10:26:07.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:07 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/1935103005' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-23T10:26:07.857 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-23T10:26:07.857 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-23T10:26:07.857 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2024-04-23T10:26:07.857 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-23T10:26:07.857 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-23T10:26:07.857 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-23T10:26:07.857 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-23T10:26:07.857 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-23T10:26:07.857 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-23T10:26:07.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T10:26:07.858 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2024-04-23T10:26:07.874 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-23T10:26:07.874 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-23T10:26:07.874 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2024-04-23T10:26:07.874 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-23T10:26:07.874 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2024-04-23T10:26:07.874 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-23T10:26:07.874 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T10:26:07.874 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2024-04-23T10:26:07.874 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2024-04-23T10:26:07.874 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T10:26:07.874 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2024-04-23T10:26:07.874 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2024-04-23T10:26:07.874 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T10:26:07.875 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2024-04-23T10:26:07.875 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-23T10:26:07.875 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2024-04-23T10:26:07.875 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-23T10:26:07.875 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 41562sec preferred_lft 41562sec 2024-04-23T10:26:07.875 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::d874:44e6:ef83:6bdd/64 scope link noprefixroute 2024-04-23T10:26:07.875 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-04-23T10:26:07.875 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-23T10:26:07.875 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T10:26:07.876 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-23T10:26:07.876 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link add name ceph-brx type bridge 2024-04-23T10:26:07.876 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip addr flush dev ceph-brx 2024-04-23T10:26:07.876 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set ceph-brx up 2024-04-23T10:26:07.876 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-23T10:26:07.876 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-23T10:26:07.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T10:26:08.051 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T10:26:08.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T10:26:08.058 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-23T10:26:08.076 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:26:07 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:26:07.904+0000 7ff779856200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-23T10:26:08.083 INFO:teuthology.orchestra.run.smithi097.stdout:1 2024-04-23T10:26:08.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T10:26:08.085 DEBUG:teuthology.orchestra.run.smithi097:> route 2024-04-23T10:26:08.091 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 23 10:26:07 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-z[34770]: 2024-04-23T10:26:07.759+0000 7f10c1c6e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-23T10:26:08.150 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 23 10:26:07 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-y[33699]: 2024-04-23T10:26:07.771+0000 7fe32ad0e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-23T10:26:08.154 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2024-04-23T10:26:08.154 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-23T10:26:08.154 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-23T10:26:08.154 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-23T10:26:08.155 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-23T10:26:08.155 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T10:26:08.155 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-23T10:26:08.155 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-23T10:26:08.155 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-23T10:26:08.155 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-23T10:26:08.155 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-23T10:26:08.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T10:26:08.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T10:26:08.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T10:26:08.430 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-04-23T10:26:08.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T10:26:08.446 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list-id 2024-04-23T10:26:08.478 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 23 10:26:08 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-z[34770]: 2024-04-23T10:26:08.181+0000 7f10c1c6e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-23T10:26:08.478 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 23 10:26:08 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-z[34770]: 2024-04-23T10:26:08.276+0000 7f10c1c6e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-23T10:26:08.491 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 23 10:26:08 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-y[33699]: 2024-04-23T10:26:08.195+0000 7fe32ad0e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-23T10:26:08.491 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 23 10:26:08 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-y[33699]: 2024-04-23T10:26:08.291+0000 7fe32ad0e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-23T10:26:08.491 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 23 10:26:08 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-y[33699]: 2024-04-23T10:26:08.491+0000 7fe32ad0e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-23T10:26:08.500 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T10:26:08.501 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-23T10:26:08.501 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T10:26:08.501 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-23T10:26:08.501 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-23T10:26:08.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T10:26:08.596 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:26:08 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:26:08.362+0000 7ff779856200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-23T10:26:08.596 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:26:08 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:26:08.467+0000 7ff779856200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-23T10:26:08.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T10:26:08.619 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-23T10:26:08.619 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T10:26:08.619 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-23T10:26:08.619 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-23T10:26:08.619 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-23T10:26:08.619 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-23T10:26:08.619 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-23T10:26:08.619 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-23T10:26:08.619 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-23T10:26:08.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T10:26:08.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T10:26:08.743 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T10:26:08.743 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-04-23T10:26:08.743 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2024-04-23T10:26:08.743 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set dev brx.0 master ceph-brx 2024-04-23T10:26:08.743 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-04-23T10:26:08.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T10:26:08.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T10:26:08.820 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T10:26:08.820 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-23T10:26:08.834 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-23T10:26:08.835 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T10:26:08.835 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-23T10:26:08.841 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 23 10:26:08 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-z[34770]: 2024-04-23T10:26:08.478+0000 7f10c1c6e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-23T10:26:08.860 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:26:08 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:26:08.682+0000 7ff779856200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-23T10:26:08.875 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-23T10:26:08.875 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-23T10:26:08.875 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/e80e6226-5911-43bb-a37a-f08d55e0fed9 2024-04-23T10:26:08.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T10:26:08.875 DEBUG:teuthology.orchestra.run.smithi097:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/e80e6226-5911-43bb-a37a-f08d55e0fed9 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,nowsync,nofallback 2024-04-23T10:26:08.966 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,nowsync,nofallback 2024-04-23T10:26:08.966 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,nowsync". 2024-04-23T10:26:09.102 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,nowsync,nofallback 2024-04-23T10:26:09.103 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,nowsync". 2024-04-23T10:26:09.103 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.97:3300,172.21.15.183:3300,172.21.15.196:3300" 2024-04-23T10:26:09.103 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: trying mount with new device syntax: 0@9b22af88-0159-11ef-bc93-c7b262605968.cephfs=/volumes/qa/sv_1/e80e6226-5911-43bb-a37a-f08d55e0fed9 2024-04-23T10:26:09.103 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,nowsync,name=0,key=0,mon_addr=172.21.15.97:3300/172.21.15.183:3300/172.21.15.196:3300" will pass to kernel 2024-04-23T10:26:09.104 INFO:teuthology.orchestra.run.smithi097.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-23T10:26:09.104 INFO:teuthology.orchestra.run.smithi097.stdout: You just mounted a file system that supports labels which does not 2024-04-23T10:26:09.104 INFO:teuthology.orchestra.run.smithi097.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-23T10:26:09.104 INFO:teuthology.orchestra.run.smithi097.stdout: applications will generate AVC messages and not be allowed access to 2024-04-23T10:26:09.104 INFO:teuthology.orchestra.run.smithi097.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-23T10:26:09.106 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-23T10:26:09.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T10:26:09.106 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-23T10:26:09.131 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-23T10:26:09.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:09 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-23T10:26:09.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T10:26:09.230 INFO:teuthology.orchestra.run.smithi097.stdout:client24943 2024-04-23T10:26:09.230 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-23T10:26:09.230 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T10:26:09.246 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-23T10:26:09.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:09 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-23T10:26:09.336 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 23 10:26:09 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-z[34770]: 2024-04-23T10:26:09.001+0000 7f10c1c6e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-23T10:26:09.336 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 23 10:26:09 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-z[34770]: 2024-04-23T10:26:09.094+0000 7f10c1c6e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-23T10:26:09.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T10:26:09.343 INFO:teuthology.orchestra.run.smithi097.stdout:client24943 2024-04-23T10:26:09.344 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-23T10:26:09.353 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 23 10:26:09 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-y[33699]: 2024-04-23T10:26:09.016+0000 7fe32ad0e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-23T10:26:09.353 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 23 10:26:09 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-y[33699]: 2024-04-23T10:26:09.109+0000 7fe32ad0e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-23T10:26:09.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:09 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-23T10:26:09.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T10:26:09.400 INFO:teuthology.orchestra.run.smithi097.stdout:client24943 2024-04-23T10:26:09.401 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/9b22af88-0159-11ef-bc93-c7b262605968.client24943/status') 2024-04-23T10:26:09.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T10:26:09.459 INFO:teuthology.orchestra.run.smithi097.stdout:instance: client.24943 (3)172.21.15.97:0/507110464 2024-04-23T10:26:09.459 INFO:teuthology.orchestra.run.smithi097.stdout:blocklisted: false 2024-04-23T10:26:09.459 INFO:teuthology.orchestra.run.smithi097.stderr:0+1 records in 2024-04-23T10:26:09.460 INFO:teuthology.orchestra.run.smithi097.stderr:0+1 records out 2024-04-23T10:26:09.460 INFO:teuthology.orchestra.run.smithi097.stderr:70 bytes copied, 2.5974e-05 s, 2.7 MB/s 2024-04-23T10:26:09.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T10:26:09.467 INFO:teuthology.run_tasks:Running task exec... 2024-04-23T10:26:09.476 INFO:teuthology.task.exec:Executing custom commands... 2024-04-23T10:26:09.476 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi097.front.sepia.ceph.com 2024-04-23T10:26:09.476 DEBUG:teuthology.orchestra.run.smithi097:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-04-23T10:26:09.496 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:26:09 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:26:09.235+0000 7ff779856200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-23T10:26:09.497 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:26:09 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:26:09.333+0000 7ff779856200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-23T10:26:09.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:09.579+0000 7fe589ecf640 1 -- 172.21.15.97:0/1257481637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5841021d0 msgr2=0x7fe584108a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:09.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:09.579+0000 7fe589ecf640 1 --2- 172.21.15.97:0/1257481637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5841021d0 0x7fe584108a60 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe57c009ca0 tx=0x7fe57c02a8c0 comp rx=0 tx=0).stop 2024-04-23T10:26:09.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:09.579+0000 7fe589ecf640 1 -- 172.21.15.97:0/1257481637 shutdown_connections 2024-04-23T10:26:09.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:09.579+0000 7fe589ecf640 1 --2- 172.21.15.97:0/1257481637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5841021d0 0x7fe584108a60 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:09.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:09.579+0000 7fe589ecf640 1 --2- 172.21.15.97:0/1257481637 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe584101810 0x7fe584101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:09.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:09.579+0000 7fe589ecf640 1 --2- 172.21.15.97:0/1257481637 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe584100620 0x7fe584100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:09.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:09.579+0000 7fe589ecf640 1 -- 172.21.15.97:0/1257481637 >> 172.21.15.97:0/1257481637 conn(0x7fe5840fbbb0 msgr2=0x7fe5840fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:09.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:09.579+0000 7fe589ecf640 1 -- 172.21.15.97:0/1257481637 shutdown_connections 2024-04-23T10:26:09.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:09.579+0000 7fe589ecf640 1 -- 172.21.15.97:0/1257481637 wait complete. 2024-04-23T10:26:09.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:09.579+0000 7fe589ecf640 1 Processor -- start 2024-04-23T10:26:09.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:09.580+0000 7fe589ecf640 1 -- start start 2024-04-23T10:26:09.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:09.580+0000 7fe589ecf640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe584100620 0x7fe58419a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:09.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:09.580+0000 7fe589ecf640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe584101810 0x7fe58419a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:09.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:09.580+0000 7fe589ecf640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5841021d0 0x7fe5841a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:09.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:09.580+0000 7fe589ecf640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe58410b8e0 con 0x7fe5841021d0 2024-04-23T10:26:09.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:09.580+0000 7fe589ecf640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe58410b760 con 0x7fe584100620 2024-04-23T10:26:09.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:09.580+0000 7fe589ecf640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe58410ba60 con 0x7fe584101810 2024-04-23T10:26:09.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:09.580+0000 7fe583fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5841021d0 0x7fe5841a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:09.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:09.580+0000 7fe583fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5841021d0 0x7fe5841a18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58120/0 (socket says 172.21.15.97:58120) 2024-04-23T10:26:09.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:09.580+0000 7fe583fff640 1 -- 172.21.15.97:0/2079864056 learned_addr learned my addr 172.21.15.97:0/2079864056 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:09.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:09.580+0000 7fe582ffd640 1 --2- 172.21.15.97:0/2079864056 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe584101810 0x7fe58419a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:09.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:09.580+0000 7fe5837fe640 1 --2- 172.21.15.97:0/2079864056 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe584100620 0x7fe58419a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:09.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:09.580+0000 7fe583fff640 1 -- 172.21.15.97:0/2079864056 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe584101810 msgr2=0x7fe58419a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:09.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:09.580+0000 7fe583fff640 1 --2- 172.21.15.97:0/2079864056 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe584101810 0x7fe58419a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:09.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:09.580+0000 7fe583fff640 1 -- 172.21.15.97:0/2079864056 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe584100620 msgr2=0x7fe58419a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:09.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:09.580+0000 7fe583fff640 1 --2- 172.21.15.97:0/2079864056 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe584100620 0x7fe58419a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:09.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:09.580+0000 7fe583fff640 1 -- 172.21.15.97:0/2079864056 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5841a1f10 con 0x7fe5841021d0 2024-04-23T10:26:09.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:09.580+0000 7fe583fff640 1 --2- 172.21.15.97:0/2079864056 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5841021d0 0x7fe5841a18a0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe57c0097b0 tx=0x7fe57c031770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:09.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:09.581+0000 7fe580ff9640 1 -- 172.21.15.97:0/2079864056 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe57c003b70 con 0x7fe5841021d0 2024-04-23T10:26:09.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:09.581+0000 7fe589ecf640 1 -- 172.21.15.97:0/2079864056 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5841a21a0 con 0x7fe5841021d0 2024-04-23T10:26:09.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:09.581+0000 7fe580ff9640 1 -- 172.21.15.97:0/2079864056 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe57c003d10 con 0x7fe5841021d0 2024-04-23T10:26:09.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:09.581+0000 7fe580ff9640 1 -- 172.21.15.97:0/2079864056 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe57c02fc10 con 0x7fe5841021d0 2024-04-23T10:26:09.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:09.581+0000 7fe589ecf640 1 -- 172.21.15.97:0/2079864056 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5841a2740 con 0x7fe5841021d0 2024-04-23T10:26:09.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:09.582+0000 7fe589ecf640 1 -- 172.21.15.97:0/2079864056 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe584107870 con 0x7fe5841021d0 2024-04-23T10:26:09.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:09.583+0000 7fe580ff9640 1 -- 172.21.15.97:0/2079864056 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7fe57c004040 con 0x7fe5841021d0 2024-04-23T10:26:09.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:09.583+0000 7fe580ff9640 1 --2- 172.21.15.97:0/2079864056 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fe56c0a3ca0 0x7fe56c0a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:09.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:09.583+0000 7fe5837fe640 1 -- 172.21.15.97:0/2079864056 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fe56c0a3ca0 msgr2=0x7fe56c0a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/3149043749 2024-04-23T10:26:09.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:09.583+0000 7fe580ff9640 1 -- 172.21.15.97:0/2079864056 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fe57c0f1910 con 0x7fe5841021d0 2024-04-23T10:26:09.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:09.583+0000 7fe5837fe640 1 --2- 172.21.15.97:0/2079864056 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fe56c0a3ca0 0x7fe56c0a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T10:26:09.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:09.592+0000 7fe580ff9640 1 -- 172.21.15.97:0/2079864056 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe57c0be320 con 0x7fe5841021d0 2024-04-23T10:26:09.591 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 23 10:26:09 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-z[34770]: 2024-04-23T10:26:09.337+0000 7f10c1c6e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-23T10:26:09.591 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 23 10:26:09 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-z[34770]: 2024-04-23T10:26:09.469+0000 7f10c1c6e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-23T10:26:09.591 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 23 10:26:09 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-z[34770]: 2024-04-23T10:26:09.564+0000 7f10c1c6e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-23T10:26:09.650 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 23 10:26:09 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-y[33699]: 2024-04-23T10:26:09.354+0000 7fe32ad0e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-23T10:26:09.650 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 23 10:26:09 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-y[33699]: 2024-04-23T10:26:09.488+0000 7fe32ad0e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-23T10:26:09.650 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 23 10:26:09 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-y[33699]: 2024-04-23T10:26:09.584+0000 7fe32ad0e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-23T10:26:09.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:09.784+0000 7fe5837fe640 1 -- 172.21.15.97:0/2079864056 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fe56c0a3ca0 msgr2=0x7fe56c0a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/3149043749 2024-04-23T10:26:09.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:09.784+0000 7fe5837fe640 1 --2- 172.21.15.97:0/2079864056 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fe56c0a3ca0 0x7fe56c0a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-23T10:26:09.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:09.819+0000 7fe589ecf640 1 -- 172.21.15.97:0/2079864056 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7fe584062b50 con 0x7fe5841021d0 2024-04-23T10:26:09.838 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:26:09 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:26:09.599+0000 7ff779856200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-23T10:26:09.838 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:26:09 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:26:09.738+0000 7ff779856200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-23T10:26:09.839 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:26:09 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:26:09.839+0000 7ff779856200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-23T10:26:09.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:09.871+0000 7fe580ff9640 1 -- 172.21.15.97:0/2079864056 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v26) ==== 118+0+0 (secure 0 0 0) 0x7fe57c0bdc20 con 0x7fe5841021d0 2024-04-23T10:26:09.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:09.873+0000 7fe589ecf640 1 -- 172.21.15.97:0/2079864056 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fe56c0a3ca0 msgr2=0x7fe56c0a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T10:26:09.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:09.873+0000 7fe589ecf640 1 --2- 172.21.15.97:0/2079864056 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fe56c0a3ca0 0x7fe56c0a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:09.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:09.873+0000 7fe589ecf640 1 -- 172.21.15.97:0/2079864056 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5841021d0 msgr2=0x7fe5841a18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:09.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:09.873+0000 7fe589ecf640 1 --2- 172.21.15.97:0/2079864056 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5841021d0 0x7fe5841a18a0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe57c0097b0 tx=0x7fe57c031770 comp rx=0 tx=0).stop 2024-04-23T10:26:09.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:09.873+0000 7fe589ecf640 1 -- 172.21.15.97:0/2079864056 shutdown_connections 2024-04-23T10:26:09.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:09.873+0000 7fe589ecf640 1 --2- 172.21.15.97:0/2079864056 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fe56c0a3ca0 0x7fe56c0a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:09.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:09.873+0000 7fe589ecf640 1 --2- 172.21.15.97:0/2079864056 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5841021d0 0x7fe5841a18a0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:09.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:09.873+0000 7fe589ecf640 1 --2- 172.21.15.97:0/2079864056 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe584101810 0x7fe58419a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:09.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:09.873+0000 7fe589ecf640 1 --2- 172.21.15.97:0/2079864056 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe584100620 0x7fe58419a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:09.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:09.873+0000 7fe589ecf640 1 -- 172.21.15.97:0/2079864056 >> 172.21.15.97:0/2079864056 conn(0x7fe5840fbbb0 msgr2=0x7fe584104a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:09.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:09.874+0000 7fe589ecf640 1 -- 172.21.15.97:0/2079864056 shutdown_connections 2024-04-23T10:26:09.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:09.874+0000 7fe589ecf640 1 -- 172.21.15.97:0/2079864056 wait complete. 2024-04-23T10:26:09.885 DEBUG:teuthology.orchestra.run.smithi097:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-04-23T10:26:09.990 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 23 10:26:09 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-z[34770]: 2024-04-23T10:26:09.682+0000 7f10c1c6e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-23T10:26:09.991 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:09 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/2079864056' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-23T10:26:10.005 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:09 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/2079864056' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-23T10:26:10.005 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 23 10:26:09 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-y[33699]: 2024-04-23T10:26:09.700+0000 7fe32ad0e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-23T10:26:10.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:10.031+0000 7fcea0e4e640 1 -- 172.21.15.97:0/4006902785 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce9c102a40 msgr2=0x7fce9c109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:10.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:10.031+0000 7fcea0e4e640 1 --2- 172.21.15.97:0/4006902785 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce9c102a40 0x7fce9c109360 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fce94009ca0 tx=0x7fce9402a8c0 comp rx=0 tx=0).stop 2024-04-23T10:26:10.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:10.031+0000 7fcea0e4e640 1 -- 172.21.15.97:0/4006902785 shutdown_connections 2024-04-23T10:26:10.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:10.031+0000 7fcea0e4e640 1 --2- 172.21.15.97:0/4006902785 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce9c102a40 0x7fce9c109360 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:10.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:10.031+0000 7fcea0e4e640 1 --2- 172.21.15.97:0/4006902785 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce9c102080 0x7fce9c102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:10.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:10.031+0000 7fcea0e4e640 1 --2- 172.21.15.97:0/4006902785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce9c100ea0 0x7fce9c1012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:10.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:10.031+0000 7fcea0e4e640 1 -- 172.21.15.97:0/4006902785 >> 172.21.15.97:0/4006902785 conn(0x7fce9c0fc460 msgr2=0x7fce9c0fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:10.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:10.031+0000 7fcea0e4e640 1 -- 172.21.15.97:0/4006902785 shutdown_connections 2024-04-23T10:26:10.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:10.031+0000 7fcea0e4e640 1 -- 172.21.15.97:0/4006902785 wait complete. 2024-04-23T10:26:10.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:10.032+0000 7fcea0e4e640 1 Processor -- start 2024-04-23T10:26:10.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:10.032+0000 7fcea0e4e640 1 -- start start 2024-04-23T10:26:10.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:10.032+0000 7fcea0e4e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce9c100ea0 0x7fce9c06acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:10.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:10.032+0000 7fcea0e4e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce9c102080 0x7fce9c19cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:10.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:10.032+0000 7fcea0e4e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce9c102a40 0x7fce9c19f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:10.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:10.032+0000 7fcea0e4e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fce9c10ca40 con 0x7fce9c100ea0 2024-04-23T10:26:10.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:10.032+0000 7fcea0e4e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fce9c10c8c0 con 0x7fce9c102080 2024-04-23T10:26:10.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:10.032+0000 7fcea0e4e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fce9c10cbc0 con 0x7fce9c102a40 2024-04-23T10:26:10.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:10.032+0000 7fce9a59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce9c100ea0 0x7fce9c06acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:10.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:10.032+0000 7fce9a59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce9c100ea0 0x7fce9c06acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58140/0 (socket says 172.21.15.97:58140) 2024-04-23T10:26:10.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:10.032+0000 7fce9a59c640 1 -- 172.21.15.97:0/1759343973 learned_addr learned my addr 172.21.15.97:0/1759343973 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:10.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:10.032+0000 7fce99d9b640 1 --2- 172.21.15.97:0/1759343973 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce9c102080 0x7fce9c19cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:10.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:10.032+0000 7fce9ad9d640 1 --2- 172.21.15.97:0/1759343973 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce9c102a40 0x7fce9c19f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:10.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:10.032+0000 7fce9a59c640 1 -- 172.21.15.97:0/1759343973 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce9c102a40 msgr2=0x7fce9c19f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:10.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:10.032+0000 7fce9a59c640 1 --2- 172.21.15.97:0/1759343973 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce9c102a40 0x7fce9c19f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:10.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:10.032+0000 7fce9a59c640 1 -- 172.21.15.97:0/1759343973 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce9c102080 msgr2=0x7fce9c19cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:10.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:10.032+0000 7fce9a59c640 1 --2- 172.21.15.97:0/1759343973 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce9c102080 0x7fce9c19cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:10.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:10.032+0000 7fce9a59c640 1 -- 172.21.15.97:0/1759343973 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce9c19fd30 con 0x7fce9c100ea0 2024-04-23T10:26:10.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:10.032+0000 7fce9a59c640 1 --2- 172.21.15.97:0/1759343973 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce9c100ea0 0x7fce9c06acb0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fce9000e9c0 tx=0x7fce9000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:10.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:10.032+0000 7fce8b7fe640 1 -- 172.21.15.97:0/1759343973 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce9000cd80 con 0x7fce9c100ea0 2024-04-23T10:26:10.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:10.033+0000 7fce8b7fe640 1 -- 172.21.15.97:0/1759343973 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce900035f0 con 0x7fce9c100ea0 2024-04-23T10:26:10.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:10.033+0000 7fcea0e4e640 1 -- 172.21.15.97:0/1759343973 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce9c19ffc0 con 0x7fce9c100ea0 2024-04-23T10:26:10.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:10.033+0000 7fce8b7fe640 1 -- 172.21.15.97:0/1759343973 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce90010690 con 0x7fce9c100ea0 2024-04-23T10:26:10.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:10.033+0000 7fcea0e4e640 1 -- 172.21.15.97:0/1759343973 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce9c1a0460 con 0x7fce9c100ea0 2024-04-23T10:26:10.035 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:10.034+0000 7fcea0e4e640 1 -- 172.21.15.97:0/1759343973 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce9c108240 con 0x7fce9c100ea0 2024-04-23T10:26:10.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:10.035+0000 7fce8b7fe640 1 -- 172.21.15.97:0/1759343973 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7fce90004000 con 0x7fce9c100ea0 2024-04-23T10:26:10.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:10.035+0000 7fce8b7fe640 1 --2- 172.21.15.97:0/1759343973 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fce840a3ca0 0x7fce840a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:10.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:10.035+0000 7fce99d9b640 1 -- 172.21.15.97:0/1759343973 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fce840a3ca0 msgr2=0x7fce840a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/3149043749 2024-04-23T10:26:10.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:10.035+0000 7fce8b7fe640 1 -- 172.21.15.97:0/1759343973 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fce900d0920 con 0x7fce9c100ea0 2024-04-23T10:26:10.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:10.035+0000 7fce99d9b640 1 --2- 172.21.15.97:0/1759343973 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fce840a3ca0 0x7fce840a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T10:26:10.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:10.040+0000 7fce8b7fe640 1 -- 172.21.15.97:0/1759343973 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fce9009d330 con 0x7fce9c100ea0 2024-04-23T10:26:10.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:09 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/2079864056' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-23T10:26:10.111 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:26:09 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:26:09.964+0000 7ff779856200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-23T10:26:10.234 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:10.236+0000 7fce99d9b640 1 -- 172.21.15.97:0/1759343973 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fce840a3ca0 msgr2=0x7fce840a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/3149043749 2024-04-23T10:26:10.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:10.236+0000 7fce99d9b640 1 --2- 172.21.15.97:0/1759343973 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fce840a3ca0 0x7fce840a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-23T10:26:10.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:10.256+0000 7fcea0e4e640 1 -- 172.21.15.97:0/1759343973 --> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7fce9c062430 con 0x7fce840a3ca0 2024-04-23T10:26:10.341 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 23 10:26:09 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-z[34770]: 2024-04-23T10:26:09.991+0000 7f10c1c6e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-23T10:26:10.400 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 23 10:26:10 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-y[33699]: 2024-04-23T10:26:10.005+0000 7fe32ad0e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-23T10:26:10.610 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:26:10 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:26:10.275+0000 7ff779856200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-23T10:26:10.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:10.637+0000 7fce99d9b640 1 -- 172.21.15.97:0/1759343973 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fce840a3ca0 msgr2=0x7fce840a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.97:6800/3149043749 2024-04-23T10:26:10.635 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:10.637+0000 7fce99d9b640 1 --2- 172.21.15.97:0/1759343973 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fce840a3ca0 0x7fce840a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-23T10:26:10.878 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 23 10:26:10 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-z[34770]: 2024-04-23T10:26:10.594+0000 7f10c1c6e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-23T10:26:10.878 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 23 10:26:10 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-z[34770]: 2024-04-23T10:26:10.810+0000 7f10c1c6e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-23T10:26:10.878 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 23 10:26:10 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-y[33699]: 2024-04-23T10:26:10.604+0000 7fe32ad0e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-23T10:26:10.879 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 23 10:26:10 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-y[33699]: 2024-04-23T10:26:10.820+0000 7fe32ad0e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-23T10:26:10.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:10.894+0000 7fce8b7fe640 1 -- 172.21.15.97:0/1759343973 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mgrmap(e 20) ==== 138663+0+0 (secure 0 0 0) 0x7fce900992a0 con 0x7fce9c100ea0 2024-04-23T10:26:11.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:11.121+0000 7fce8b7fe640 1 -- 172.21.15.97:0/1759343973 <== mon.0 v2:172.21.15.97:3300/0 8 ==== mgrmap(e 21) ==== 138331+0+0 (secure 0 0 0) 0x7fce900042f0 con 0x7fce9c100ea0 2024-04-23T10:26:11.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:11.122+0000 7fce8b7fe640 1 -- 172.21.15.97:0/1759343973 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fce840a3ca0 msgr2=0x7fce840a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T10:26:11.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:11.122+0000 7fce8b7fe640 1 --2- 172.21.15.97:0/1759343973 >> [v2:172.21.15.97:6800/3149043749,v1:172.21.15.97:6801/3149043749] conn(0x7fce840a3ca0 0x7fce840a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:11.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:10 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/2079864056' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-23T10:26:11.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:10 smithi183 ceph-mon[32317]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-23T10:26:11.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:10 smithi183 ceph-mon[32317]: Standby manager daemon z restarted 2024-04-23T10:26:11.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:10 smithi183 ceph-mon[32317]: Standby manager daemon z started 2024-04-23T10:26:11.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:10 smithi183 ceph-mon[32317]: from='mgr.? 172.21.15.196:0/1941881304' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-23T10:26:11.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:10 smithi183 ceph-mon[32317]: from='mgr.? 172.21.15.196:0/1941881304' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T10:26:11.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:10 smithi183 ceph-mon[32317]: from='mgr.? 172.21.15.196:0/1941881304' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-23T10:26:11.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:10 smithi183 ceph-mon[32317]: from='mgr.? 172.21.15.196:0/1941881304' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T10:26:11.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:10 smithi183 ceph-mon[32317]: Standby manager daemon y restarted 2024-04-23T10:26:11.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:10 smithi183 ceph-mon[32317]: Standby manager daemon y started 2024-04-23T10:26:11.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:10 smithi183 ceph-mon[32317]: from='mgr.? 172.21.15.183:0/1809912091' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-23T10:26:11.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:10 smithi183 ceph-mon[32317]: from='mgr.? 172.21.15.183:0/1809912091' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T10:26:11.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:10 smithi183 ceph-mon[32317]: from='mgr.? 172.21.15.183:0/1809912091' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-23T10:26:11.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:10 smithi183 ceph-mon[32317]: from='mgr.? 172.21.15.183:0/1809912091' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T10:26:11.252 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:10 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/2079864056' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-23T10:26:11.252 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:10 smithi196 ceph-mon[32919]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-23T10:26:11.252 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:10 smithi196 ceph-mon[32919]: Standby manager daemon z restarted 2024-04-23T10:26:11.252 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:10 smithi196 ceph-mon[32919]: Standby manager daemon z started 2024-04-23T10:26:11.252 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:10 smithi196 ceph-mon[32919]: from='mgr.? 172.21.15.196:0/1941881304' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-23T10:26:11.252 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:10 smithi196 ceph-mon[32919]: from='mgr.? 172.21.15.196:0/1941881304' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T10:26:11.252 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:10 smithi196 ceph-mon[32919]: from='mgr.? 172.21.15.196:0/1941881304' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-23T10:26:11.252 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:10 smithi196 ceph-mon[32919]: from='mgr.? 172.21.15.196:0/1941881304' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T10:26:11.252 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:10 smithi196 ceph-mon[32919]: Standby manager daemon y restarted 2024-04-23T10:26:11.253 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:10 smithi196 ceph-mon[32919]: Standby manager daemon y started 2024-04-23T10:26:11.253 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:10 smithi196 ceph-mon[32919]: from='mgr.? 172.21.15.183:0/1809912091' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-23T10:26:11.253 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:10 smithi196 ceph-mon[32919]: from='mgr.? 172.21.15.183:0/1809912091' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T10:26:11.253 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:10 smithi196 ceph-mon[32919]: from='mgr.? 172.21.15.183:0/1809912091' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-23T10:26:11.253 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:10 smithi196 ceph-mon[32919]: from='mgr.? 172.21.15.183:0/1809912091' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T10:26:11.253 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:26:10 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:26:10.883+0000 7ff779856200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-23T10:26:11.253 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:26:11 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:26:11.107+0000 7ff779856200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-23T10:26:11.254 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:10 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/2079864056' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-23T10:26:11.254 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:10 smithi097 ceph-mon[29197]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-23T10:26:11.254 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:10 smithi097 ceph-mon[29197]: Standby manager daemon z restarted 2024-04-23T10:26:11.254 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:10 smithi097 ceph-mon[29197]: Standby manager daemon z started 2024-04-23T10:26:11.254 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:10 smithi097 ceph-mon[29197]: from='mgr.? 172.21.15.196:0/1941881304' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-23T10:26:11.254 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:10 smithi097 ceph-mon[29197]: from='mgr.? 172.21.15.196:0/1941881304' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T10:26:11.254 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:10 smithi097 ceph-mon[29197]: from='mgr.? 172.21.15.196:0/1941881304' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-23T10:26:11.254 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:10 smithi097 ceph-mon[29197]: from='mgr.? 172.21.15.196:0/1941881304' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T10:26:11.254 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:10 smithi097 ceph-mon[29197]: Standby manager daemon y restarted 2024-04-23T10:26:11.254 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:10 smithi097 ceph-mon[29197]: Standby manager daemon y started 2024-04-23T10:26:11.254 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:10 smithi097 ceph-mon[29197]: from='mgr.? 172.21.15.183:0/1809912091' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-23T10:26:11.254 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:10 smithi097 ceph-mon[29197]: from='mgr.? 172.21.15.183:0/1809912091' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T10:26:11.255 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:10 smithi097 ceph-mon[29197]: from='mgr.? 172.21.15.183:0/1809912091' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-23T10:26:11.255 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:10 smithi097 ceph-mon[29197]: from='mgr.? 172.21.15.183:0/1809912091' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T10:26:11.610 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:26:11 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:26:11.342+0000 7ff6dc1d0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T10:26:11.611 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:26:11 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:26:11.342+0000 7ff6dc1d0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T10:26:11.611 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:26:11 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:26:11.342+0000 7ff6dc1d0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T10:26:11.611 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:26:11 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:26:11.342+0000 7ff6dc1d0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T10:26:11.611 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:26:11 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:26:11.342+0000 7ff6dc1d0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T10:26:11.611 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:26:11 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:26:11.342+0000 7ff6d91ca700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T10:26:11.611 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:26:11 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:26:11.342+0000 7ff6d91ca700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T10:26:11.611 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:26:11 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:26:11.342+0000 7ff6d91ca700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T10:26:11.611 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:26:11 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:26:11.342+0000 7ff6d91ca700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T10:26:11.611 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:26:11 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:26:11.342+0000 7ff6d91ca700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T10:26:12.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.126+0000 7fce8b7fe640 1 -- 172.21.15.97:0/1759343973 <== mon.0 v2:172.21.15.97:3300/0 9 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7fce90078cd0 con 0x7fce9c100ea0 2024-04-23T10:26:12.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.127+0000 7fce8b7fe640 1 --2- 172.21.15.97:0/1759343973 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fce84035890 0x7fce84035c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:12.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.127+0000 7fce8b7fe640 1 -- 172.21.15.97:0/1759343973 --> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7fce90021b20 con 0x7fce84035890 2024-04-23T10:26:12.129 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.131+0000 7fce99d9b640 1 --2- 172.21.15.97:0/1759343973 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fce84035890 0x7fce84035c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:12.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.133+0000 7fce99d9b640 1 --2- 172.21.15.97:0/1759343973 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fce84035890 0x7fce84035c70 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fce9c06aa20 tx=0x7fce8c005f90 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:12.142 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:11 smithi097 ceph-mon[29197]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-04-23T10:26:12.142 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:11 smithi097 ceph-mon[29197]: Active manager daemon x restarted 2024-04-23T10:26:12.142 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:11 smithi097 ceph-mon[29197]: Activating manager daemon x 2024-04-23T10:26:12.142 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:11 smithi097 ceph-mon[29197]: osdmap e89: 12 total, 12 up, 12 in 2024-04-23T10:26:12.142 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:11 smithi097 ceph-mon[29197]: mgrmap e21: x(active, starting, since 0.0149011s), standbys: z, y 2024-04-23T10:26:12.142 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:11 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T10:26:12.142 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:11 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T10:26:12.142 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:11 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T10:26:12.142 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:11 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-23T10:26:12.142 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:11 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-23T10:26:12.142 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:11 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-23T10:26:12.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:11 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-23T10:26:12.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:11 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-23T10:26:12.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:11 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-23T10:26:12.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:11 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-23T10:26:12.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:11 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-23T10:26:12.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:11 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-23T10:26:12.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:11 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-23T10:26:12.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:11 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-23T10:26:12.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:11 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-23T10:26:12.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:11 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-23T10:26:12.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:11 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-23T10:26:12.143 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:11 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-23T10:26:12.144 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:11 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T10:26:12.144 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:11 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T10:26:12.144 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:11 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T10:26:12.144 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:11 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T10:26:12.144 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:11 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T10:26:12.144 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:11 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T10:26:12.144 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:11 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T10:26:12.144 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:11 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T10:26:12.144 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:11 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T10:26:12.144 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:11 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T10:26:12.144 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:11 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T10:26:12.144 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:11 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T10:26:12.145 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:11 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-23T10:26:12.145 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:11 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-23T10:26:12.145 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:11 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-23T10:26:12.145 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:11 smithi097 ceph-mon[29197]: Manager daemon x is now available 2024-04-23T10:26:12.145 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:11 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:26:12.145 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:11 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-23T10:26:12.145 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:11 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-23T10:26:12.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:11 smithi183 ceph-mon[32317]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-04-23T10:26:12.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:11 smithi183 ceph-mon[32317]: Active manager daemon x restarted 2024-04-23T10:26:12.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:11 smithi183 ceph-mon[32317]: Activating manager daemon x 2024-04-23T10:26:12.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:11 smithi183 ceph-mon[32317]: osdmap e89: 12 total, 12 up, 12 in 2024-04-23T10:26:12.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:11 smithi183 ceph-mon[32317]: mgrmap e21: x(active, starting, since 0.0149011s), standbys: z, y 2024-04-23T10:26:12.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:11 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T10:26:12.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:11 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T10:26:12.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:11 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T10:26:12.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:11 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-23T10:26:12.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:11 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-23T10:26:12.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:11 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-23T10:26:12.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:11 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-23T10:26:12.152 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:11 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-23T10:26:12.152 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:11 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-23T10:26:12.152 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:11 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-23T10:26:12.152 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:11 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-23T10:26:12.152 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:11 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-23T10:26:12.152 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:11 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-23T10:26:12.152 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:11 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-23T10:26:12.152 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:11 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-23T10:26:12.152 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:11 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-23T10:26:12.152 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:11 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-23T10:26:12.152 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:11 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-23T10:26:12.152 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:11 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T10:26:12.152 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:11 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T10:26:12.153 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:11 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T10:26:12.153 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:11 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T10:26:12.153 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:11 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T10:26:12.153 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:11 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T10:26:12.153 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:11 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T10:26:12.153 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:11 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T10:26:12.153 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:11 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T10:26:12.153 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:11 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T10:26:12.153 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:11 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T10:26:12.153 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:11 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T10:26:12.153 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:11 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-23T10:26:12.153 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:11 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-23T10:26:12.153 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:11 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-23T10:26:12.153 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:11 smithi183 ceph-mon[32317]: Manager daemon x is now available 2024-04-23T10:26:12.153 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:11 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:26:12.154 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:11 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-23T10:26:12.154 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:11 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-23T10:26:12.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.158+0000 7fce8b7fe640 1 -- 172.21.15.97:0/1759343973 <== mgr.15378 v2:172.21.15.97:6800/3567268502 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7fce90021b20 con 0x7fce84035890 2024-04-23T10:26:12.156 INFO:teuthology.orchestra.run.smithi097.stdout:{} 2024-04-23T10:26:12.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.160+0000 7fcea0e4e640 1 -- 172.21.15.97:0/1759343973 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fce84035890 msgr2=0x7fce84035c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:12.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.160+0000 7fcea0e4e640 1 --2- 172.21.15.97:0/1759343973 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fce84035890 0x7fce84035c70 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fce9c06aa20 tx=0x7fce8c005f90 comp rx=0 tx=0).stop 2024-04-23T10:26:12.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.160+0000 7fcea0e4e640 1 -- 172.21.15.97:0/1759343973 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce9c100ea0 msgr2=0x7fce9c06acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:12.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.160+0000 7fcea0e4e640 1 --2- 172.21.15.97:0/1759343973 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce9c100ea0 0x7fce9c06acb0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fce9000e9c0 tx=0x7fce9000ee80 comp rx=0 tx=0).stop 2024-04-23T10:26:12.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.160+0000 7fce9ad9d640 1 -- 172.21.15.97:0/1759343973 reap_dead start 2024-04-23T10:26:12.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.160+0000 7fcea0e4e640 1 -- 172.21.15.97:0/1759343973 shutdown_connections 2024-04-23T10:26:12.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.160+0000 7fcea0e4e640 1 -- 172.21.15.97:0/1759343973 >> 172.21.15.97:0/1759343973 conn(0x7fce9c0fc460 msgr2=0x7fce9c105410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:12.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.160+0000 7fcea0e4e640 1 -- 172.21.15.97:0/1759343973 shutdown_connections 2024-04-23T10:26:12.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.160+0000 7fcea0e4e640 1 -- 172.21.15.97:0/1759343973 wait complete. 2024-04-23T10:26:12.170 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-23T10:26:12.180 INFO:teuthology.run_tasks:Running task exec... 2024-04-23T10:26:12.221 INFO:teuthology.task.exec:Executing custom commands... 2024-04-23T10:26:12.221 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi097.front.sepia.ceph.com 2024-04-23T10:26:12.221 DEBUG:teuthology.orchestra.run.smithi097:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-04-23T10:26:12.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.324+0000 7f4660b9b640 1 -- 172.21.15.97:0/2598379380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f465c101fb0 msgr2=0x7f465c1023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:12.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.324+0000 7f4660b9b640 1 --2- 172.21.15.97:0/2598379380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f465c101fb0 0x7f465c1023d0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f464c009900 tx=0x7f464c02a410 comp rx=0 tx=0).stop 2024-04-23T10:26:12.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.324+0000 7f4660b9b640 1 -- 172.21.15.97:0/2598379380 shutdown_connections 2024-04-23T10:26:12.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.324+0000 7f4660b9b640 1 --2- 172.21.15.97:0/2598379380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f465c103b60 0x7f465c10a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:12.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.324+0000 7f4660b9b640 1 --2- 172.21.15.97:0/2598379380 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f465c1031a0 0x7f465c103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:12.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.324+0000 7f4660b9b640 1 --2- 172.21.15.97:0/2598379380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f465c101fb0 0x7f465c1023d0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:12.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.324+0000 7f4660b9b640 1 -- 172.21.15.97:0/2598379380 >> 172.21.15.97:0/2598379380 conn(0x7f465c0fd540 msgr2=0x7f465c0ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:12.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.324+0000 7f4660b9b640 1 -- 172.21.15.97:0/2598379380 shutdown_connections 2024-04-23T10:26:12.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.324+0000 7f4660b9b640 1 -- 172.21.15.97:0/2598379380 wait complete. 2024-04-23T10:26:12.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.325+0000 7f4660b9b640 1 Processor -- start 2024-04-23T10:26:12.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.325+0000 7f4660b9b640 1 -- start start 2024-04-23T10:26:12.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.325+0000 7f4660b9b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f465c101fb0 0x7f465c198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:12.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.325+0000 7f4660b9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f465c1031a0 0x7f465c198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:12.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.325+0000 7f4660b9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f465c103b60 0x7f465c19f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:12.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.325+0000 7f4660b9b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f465c10daf0 con 0x7f465c1031a0 2024-04-23T10:26:12.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.325+0000 7f4660b9b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f465c10d970 con 0x7f465c101fb0 2024-04-23T10:26:12.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.325+0000 7f4660b9b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f465c10dc70 con 0x7f465c103b60 2024-04-23T10:26:12.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.325+0000 7f4659d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f465c1031a0 0x7f465c198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:12.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.325+0000 7f4659d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f465c1031a0 0x7f465c198650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58310/0 (socket says 172.21.15.97:58310) 2024-04-23T10:26:12.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.325+0000 7f4659d9b640 1 -- 172.21.15.97:0/3360786352 learned_addr learned my addr 172.21.15.97:0/3360786352 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:12.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.325+0000 7f465a59c640 1 --2- 172.21.15.97:0/3360786352 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f465c101fb0 0x7f465c198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:12.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.325+0000 7f465ad9d640 1 --2- 172.21.15.97:0/3360786352 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f465c103b60 0x7f465c19f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:12.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.325+0000 7f4659d9b640 1 -- 172.21.15.97:0/3360786352 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f465c103b60 msgr2=0x7f465c19f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:12.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.325+0000 7f4659d9b640 1 --2- 172.21.15.97:0/3360786352 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f465c103b60 0x7f465c19f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:12.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.325+0000 7f4659d9b640 1 -- 172.21.15.97:0/3360786352 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f465c101fb0 msgr2=0x7f465c198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:12.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.325+0000 7f4659d9b640 1 --2- 172.21.15.97:0/3360786352 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f465c101fb0 0x7f465c198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:12.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.325+0000 7f4659d9b640 1 -- 172.21.15.97:0/3360786352 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f465c19fd30 con 0x7f465c1031a0 2024-04-23T10:26:12.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.326+0000 7f4659d9b640 1 --2- 172.21.15.97:0/3360786352 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f465c1031a0 0x7f465c198650 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f465000e9f0 tx=0x7f465000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:12.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.326+0000 7f464b7fe640 1 -- 172.21.15.97:0/3360786352 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465000cd80 con 0x7f465c1031a0 2024-04-23T10:26:12.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.326+0000 7f464b7fe640 1 -- 172.21.15.97:0/3360786352 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46500108e0 con 0x7f465c1031a0 2024-04-23T10:26:12.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.326+0000 7f4660b9b640 1 -- 172.21.15.97:0/3360786352 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f465c19ffc0 con 0x7f465c1031a0 2024-04-23T10:26:12.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.326+0000 7f464b7fe640 1 -- 172.21.15.97:0/3360786352 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4650018980 con 0x7f465c1031a0 2024-04-23T10:26:12.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.326+0000 7f4660b9b640 1 -- 172.21.15.97:0/3360786352 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f465c1a04e0 con 0x7f465c1031a0 2024-04-23T10:26:12.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.327+0000 7f4660b9b640 1 -- 172.21.15.97:0/3360786352 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f465c1092d0 con 0x7f465c1031a0 2024-04-23T10:26:12.328 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.328+0000 7f464b7fe640 1 -- 172.21.15.97:0/3360786352 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f4650010430 con 0x7f465c1031a0 2024-04-23T10:26:12.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.328+0000 7f464b7fe640 1 --2- 172.21.15.97:0/3360786352 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f46440a3a60 0x7f46440a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:12.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.328+0000 7f464b7fe640 1 -- 172.21.15.97:0/3360786352 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f46500d0380 con 0x7f465c1031a0 2024-04-23T10:26:12.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.328+0000 7f465a59c640 1 --2- 172.21.15.97:0/3360786352 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f46440a3a60 0x7f46440a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:12.329 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.329+0000 7f465a59c640 1 --2- 172.21.15.97:0/3360786352 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f46440a3a60 0x7f46440a5f20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f464c009230 tx=0x7f464c0314a0 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:12.332 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.333+0000 7f464b7fe640 1 -- 172.21.15.97:0/3360786352 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f465009c930 con 0x7f465c1031a0 2024-04-23T10:26:12.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:11 smithi196 ceph-mon[32919]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-04-23T10:26:12.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:11 smithi196 ceph-mon[32919]: Active manager daemon x restarted 2024-04-23T10:26:12.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:11 smithi196 ceph-mon[32919]: Activating manager daemon x 2024-04-23T10:26:12.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:11 smithi196 ceph-mon[32919]: osdmap e89: 12 total, 12 up, 12 in 2024-04-23T10:26:12.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:11 smithi196 ceph-mon[32919]: mgrmap e21: x(active, starting, since 0.0149011s), standbys: z, y 2024-04-23T10:26:12.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:11 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T10:26:12.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:11 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T10:26:12.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:11 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T10:26:12.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:11 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-23T10:26:12.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:11 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-23T10:26:12.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:11 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-23T10:26:12.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:11 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-23T10:26:12.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:11 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-23T10:26:12.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:11 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-23T10:26:12.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:11 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-23T10:26:12.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:11 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-23T10:26:12.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:11 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-23T10:26:12.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:11 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-23T10:26:12.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:11 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-23T10:26:12.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:11 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-23T10:26:12.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:11 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-23T10:26:12.344 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:11 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-23T10:26:12.344 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:11 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-23T10:26:12.344 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:11 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T10:26:12.344 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:11 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T10:26:12.344 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:11 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T10:26:12.344 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:11 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T10:26:12.344 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:11 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T10:26:12.344 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:11 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T10:26:12.344 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:11 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T10:26:12.344 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:11 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T10:26:12.344 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:11 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T10:26:12.344 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:11 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T10:26:12.344 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:11 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T10:26:12.344 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:11 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T10:26:12.344 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:11 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-23T10:26:12.344 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:11 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-23T10:26:12.345 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:11 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-23T10:26:12.345 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:11 smithi196 ceph-mon[32919]: Manager daemon x is now available 2024-04-23T10:26:12.345 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:11 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:26:12.345 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:11 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-23T10:26:12.345 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:11 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-23T10:26:12.502 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:26:12 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:26:12.141+0000 7ff6d61c4700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T10:26:12.502 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:26:12 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:26:12.141+0000 7ff6d61c4700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T10:26:12.502 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:26:12 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:26:12.141+0000 7ff6d61c4700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T10:26:12.502 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:26:12 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:26:12.141+0000 7ff6d61c4700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T10:26:12.502 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:26:12 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:26:12.141+0000 7ff6d61c4700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T10:26:12.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.534+0000 7f4660b9b640 1 -- 172.21.15.97:0/3360786352 --> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f465c062390 con 0x7f46440a3a60 2024-04-23T10:26:12.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.596+0000 7f464b7fe640 1 -- 172.21.15.97:0/3360786352 <== mgr.15378 v2:172.21.15.97:6800/3567268502 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f465c062390 con 0x7f46440a3a60 2024-04-23T10:26:12.595 INFO:teuthology.orchestra.run.smithi097.stdout:Schedule set for path / 2024-04-23T10:26:12.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.598+0000 7f4660b9b640 1 -- 172.21.15.97:0/3360786352 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f46440a3a60 msgr2=0x7f46440a5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:12.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.598+0000 7f4660b9b640 1 --2- 172.21.15.97:0/3360786352 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f46440a3a60 0x7f46440a5f20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f464c009230 tx=0x7f464c0314a0 comp rx=0 tx=0).stop 2024-04-23T10:26:12.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.598+0000 7f4660b9b640 1 -- 172.21.15.97:0/3360786352 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f465c1031a0 msgr2=0x7f465c198650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:12.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.598+0000 7f4660b9b640 1 --2- 172.21.15.97:0/3360786352 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f465c1031a0 0x7f465c198650 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f465000e9f0 tx=0x7f465000eeb0 comp rx=0 tx=0).stop 2024-04-23T10:26:12.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.598+0000 7f4660b9b640 1 -- 172.21.15.97:0/3360786352 shutdown_connections 2024-04-23T10:26:12.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.598+0000 7f4660b9b640 1 --2- 172.21.15.97:0/3360786352 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f46440a3a60 0x7f46440a5f20 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:12.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.598+0000 7f4660b9b640 1 --2- 172.21.15.97:0/3360786352 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f465c103b60 0x7f465c19f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:12.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.598+0000 7f4660b9b640 1 --2- 172.21.15.97:0/3360786352 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f465c1031a0 0x7f465c198650 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:12.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.598+0000 7f4660b9b640 1 --2- 172.21.15.97:0/3360786352 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f465c101fb0 0x7f465c198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:12.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.598+0000 7f4660b9b640 1 -- 172.21.15.97:0/3360786352 >> 172.21.15.97:0/3360786352 conn(0x7f465c0fd540 msgr2=0x7f465c1064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:12.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.598+0000 7f4660b9b640 1 -- 172.21.15.97:0/3360786352 shutdown_connections 2024-04-23T10:26:12.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.598+0000 7f4660b9b640 1 -- 172.21.15.97:0/3360786352 wait complete. 2024-04-23T10:26:12.609 DEBUG:teuthology.orchestra.run.smithi097:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-04-23T10:26:12.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.713+0000 7f7d69074640 1 -- 172.21.15.97:0/3053425352 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d64100e60 msgr2=0x7f7d64101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:12.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.713+0000 7f7d69074640 1 --2- 172.21.15.97:0/3053425352 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d64100e60 0x7f7d64101280 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c009ca0 tx=0x7f7d5c02a8c0 comp rx=0 tx=0).stop 2024-04-23T10:26:12.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.713+0000 7f7d69074640 1 -- 172.21.15.97:0/3053425352 shutdown_connections 2024-04-23T10:26:12.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.713+0000 7f7d69074640 1 --2- 172.21.15.97:0/3053425352 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d64102a00 0x7f7d64109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:12.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.713+0000 7f7d69074640 1 --2- 172.21.15.97:0/3053425352 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d64102040 0x7f7d641024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:12.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.713+0000 7f7d69074640 1 --2- 172.21.15.97:0/3053425352 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d64100e60 0x7f7d64101280 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:12.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.714+0000 7f7d69074640 1 -- 172.21.15.97:0/3053425352 >> 172.21.15.97:0/3053425352 conn(0x7f7d640fc420 msgr2=0x7f7d640fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:12.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.714+0000 7f7d69074640 1 -- 172.21.15.97:0/3053425352 shutdown_connections 2024-04-23T10:26:12.713 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.714+0000 7f7d69074640 1 -- 172.21.15.97:0/3053425352 wait complete. 2024-04-23T10:26:12.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.714+0000 7f7d69074640 1 Processor -- start 2024-04-23T10:26:12.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.714+0000 7f7d69074640 1 -- start start 2024-04-23T10:26:12.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.715+0000 7f7d69074640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d64100e60 0x7f7d6406a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:12.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.715+0000 7f7d69074640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d64102040 0x7f7d6419f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:12.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.715+0000 7f7d69074640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d64102a00 0x7f7d641a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:12.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.715+0000 7f7d69074640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6410c9f0 con 0x7f7d64102a00 2024-04-23T10:26:12.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.715+0000 7f7d69074640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6410c870 con 0x7f7d64100e60 2024-04-23T10:26:12.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.715+0000 7f7d69074640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6410cb70 con 0x7f7d64102040 2024-04-23T10:26:12.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.715+0000 7f7d6359e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d64102a00 0x7f7d641a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:12.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.715+0000 7f7d6359e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d64102a00 0x7f7d641a1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58330/0 (socket says 172.21.15.97:58330) 2024-04-23T10:26:12.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.715+0000 7f7d6359e640 1 -- 172.21.15.97:0/1780648088 learned_addr learned my addr 172.21.15.97:0/1780648088 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:12.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.715+0000 7f7d62d9d640 1 --2- 172.21.15.97:0/1780648088 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d64100e60 0x7f7d6406a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:12.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.715+0000 7f7d6259c640 1 --2- 172.21.15.97:0/1780648088 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d64102040 0x7f7d6419f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:12.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.715+0000 7f7d62d9d640 1 -- 172.21.15.97:0/1780648088 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d64102040 msgr2=0x7f7d6419f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:12.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.715+0000 7f7d62d9d640 1 --2- 172.21.15.97:0/1780648088 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d64102040 0x7f7d6419f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:12.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.715+0000 7f7d62d9d640 1 -- 172.21.15.97:0/1780648088 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d64102a00 msgr2=0x7f7d641a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:12.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.715+0000 7f7d62d9d640 1 --2- 172.21.15.97:0/1780648088 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d64102a00 0x7f7d641a1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:12.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.715+0000 7f7d62d9d640 1 -- 172.21.15.97:0/1780648088 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d641a1fd0 con 0x7f7d64100e60 2024-04-23T10:26:12.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.715+0000 7f7d6359e640 1 --2- 172.21.15.97:0/1780648088 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d64102a00 0x7f7d641a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T10:26:12.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.715+0000 7f7d62d9d640 1 --2- 172.21.15.97:0/1780648088 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d64100e60 0x7f7d6406a120 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c009ca0 tx=0x7f7d5c0316e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:12.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.715+0000 7f7d5bfff640 1 -- 172.21.15.97:0/1780648088 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c003b70 con 0x7f7d64100e60 2024-04-23T10:26:12.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.715+0000 7f7d69074640 1 -- 172.21.15.97:0/1780648088 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d641a2260 con 0x7f7d64100e60 2024-04-23T10:26:12.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.715+0000 7f7d5bfff640 1 -- 172.21.15.97:0/1780648088 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d5c003d10 con 0x7f7d64100e60 2024-04-23T10:26:12.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.715+0000 7f7d5bfff640 1 -- 172.21.15.97:0/1780648088 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c02fad0 con 0x7f7d64100e60 2024-04-23T10:26:12.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.715+0000 7f7d69074640 1 -- 172.21.15.97:0/1780648088 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d641a27b0 con 0x7f7d64100e60 2024-04-23T10:26:12.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.716+0000 7f7d69074640 1 -- 172.21.15.97:0/1780648088 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d641082d0 con 0x7f7d64100e60 2024-04-23T10:26:12.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.717+0000 7f7d5bfff640 1 -- 172.21.15.97:0/1780648088 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f7d5c004610 con 0x7f7d64100e60 2024-04-23T10:26:12.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.718+0000 7f7d5bfff640 1 --2- 172.21.15.97:0/1780648088 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f7d500a3a90 0x7f7d500a5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:12.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.718+0000 7f7d6259c640 1 --2- 172.21.15.97:0/1780648088 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f7d500a3a90 0x7f7d500a5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:12.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.718+0000 7f7d5bfff640 1 -- 172.21.15.97:0/1780648088 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f7d5c0f18f0 con 0x7f7d64100e60 2024-04-23T10:26:12.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.718+0000 7f7d6259c640 1 --2- 172.21.15.97:0/1780648088 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f7d500a3a90 0x7f7d500a5f50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7d64069e90 tx=0x7f7d4c005f90 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:12.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.722+0000 7f7d5bfff640 1 -- 172.21.15.97:0/1780648088 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7d5c0be4f0 con 0x7f7d64100e60 2024-04-23T10:26:12.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.933+0000 7f7d69074640 1 -- 172.21.15.97:0/1780648088 --> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f7d64062430 con 0x7f7d500a3a90 2024-04-23T10:26:12.991 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.992+0000 7f7d5bfff640 1 -- 172.21.15.97:0/1780648088 <== mgr.15378 v2:172.21.15.97:6800/3567268502 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f7d64062430 con 0x7f7d500a3a90 2024-04-23T10:26:12.991 INFO:teuthology.orchestra.run.smithi097.stdout:Retention added to path / 2024-04-23T10:26:12.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.994+0000 7f7d69074640 1 -- 172.21.15.97:0/1780648088 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f7d500a3a90 msgr2=0x7f7d500a5f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:12.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.994+0000 7f7d69074640 1 --2- 172.21.15.97:0/1780648088 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f7d500a3a90 0x7f7d500a5f50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7d64069e90 tx=0x7f7d4c005f90 comp rx=0 tx=0).stop 2024-04-23T10:26:12.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.994+0000 7f7d69074640 1 -- 172.21.15.97:0/1780648088 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d64100e60 msgr2=0x7f7d6406a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:12.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.994+0000 7f7d69074640 1 --2- 172.21.15.97:0/1780648088 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d64100e60 0x7f7d6406a120 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c009ca0 tx=0x7f7d5c0316e0 comp rx=0 tx=0).stop 2024-04-23T10:26:12.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.995+0000 7f7d69074640 1 -- 172.21.15.97:0/1780648088 shutdown_connections 2024-04-23T10:26:12.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.995+0000 7f7d69074640 1 --2- 172.21.15.97:0/1780648088 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f7d500a3a90 0x7f7d500a5f50 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:12.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.995+0000 7f7d69074640 1 --2- 172.21.15.97:0/1780648088 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d64102a00 0x7f7d641a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:12.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.995+0000 7f7d69074640 1 --2- 172.21.15.97:0/1780648088 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d64102040 0x7f7d6419f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:12.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.995+0000 7f7d69074640 1 --2- 172.21.15.97:0/1780648088 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d64100e60 0x7f7d6406a120 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:12.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.995+0000 7f7d69074640 1 -- 172.21.15.97:0/1780648088 >> 172.21.15.97:0/1780648088 conn(0x7f7d640fc420 msgr2=0x7f7d641054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:12.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.995+0000 7f7d69074640 1 -- 172.21.15.97:0/1780648088 shutdown_connections 2024-04-23T10:26:12.994 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:12.995+0000 7f7d69074640 1 -- 172.21.15.97:0/1780648088 wait complete. 2024-04-23T10:26:13.005 DEBUG:teuthology.orchestra.run.smithi097:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-23T10:26:13.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:12 smithi196 ceph-mon[32919]: [23/Apr/2024:10:26:11] ENGINE Bus STARTING 2024-04-23T10:26:13.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:12 smithi196 ceph-mon[32919]: [23/Apr/2024:10:26:11] ENGINE Serving on https://172.21.15.97:7150 2024-04-23T10:26:13.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:12 smithi196 ceph-mon[32919]: [23/Apr/2024:10:26:12] ENGINE Serving on http://172.21.15.97:8765 2024-04-23T10:26:13.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:12 smithi196 ceph-mon[32919]: [23/Apr/2024:10:26:12] ENGINE Bus STARTED 2024-04-23T10:26:13.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:12 smithi196 ceph-mon[32919]: mgrmap e22: x(active, since 1.02148s), standbys: z, y 2024-04-23T10:26:13.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.114+0000 7fdf7be59640 1 -- 172.21.15.97:0/2432721909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf74100e60 msgr2=0x7fdf74101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:13.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.114+0000 7fdf7be59640 1 --2- 172.21.15.97:0/2432721909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf74100e60 0x7fdf74101280 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fdf70009900 tx=0x7fdf7002a410 comp rx=0 tx=0).stop 2024-04-23T10:26:13.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.114+0000 7fdf7be59640 1 -- 172.21.15.97:0/2432721909 shutdown_connections 2024-04-23T10:26:13.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.114+0000 7fdf7be59640 1 --2- 172.21.15.97:0/2432721909 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf74102a00 0x7fdf74109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:13.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.114+0000 7fdf7be59640 1 --2- 172.21.15.97:0/2432721909 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf74102040 0x7fdf741024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:13.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.114+0000 7fdf7be59640 1 --2- 172.21.15.97:0/2432721909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf74100e60 0x7fdf74101280 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:13.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.114+0000 7fdf7be59640 1 -- 172.21.15.97:0/2432721909 >> 172.21.15.97:0/2432721909 conn(0x7fdf740fc420 msgr2=0x7fdf740fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:13.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.114+0000 7fdf7be59640 1 -- 172.21.15.97:0/2432721909 shutdown_connections 2024-04-23T10:26:13.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.114+0000 7fdf7be59640 1 -- 172.21.15.97:0/2432721909 wait complete. 2024-04-23T10:26:13.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.115+0000 7fdf7be59640 1 Processor -- start 2024-04-23T10:26:13.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.115+0000 7fdf7be59640 1 -- start start 2024-04-23T10:26:13.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.115+0000 7fdf7be59640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf74100e60 0x7fdf741981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:13.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.115+0000 7fdf7be59640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf74102040 0x7fdf741986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:13.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.115+0000 7fdf7be59640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf74102a00 0x7fdf7419f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:13.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.115+0000 7fdf7be59640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdf7410c9d0 con 0x7fdf74102040 2024-04-23T10:26:13.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.115+0000 7fdf7be59640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdf7410c850 con 0x7fdf74102a00 2024-04-23T10:26:13.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.115+0000 7fdf7be59640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdf7410cb50 con 0x7fdf74100e60 2024-04-23T10:26:13.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.115+0000 7fdf793f4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf74102040 0x7fdf741986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:13.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.115+0000 7fdf793f4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf74102040 0x7fdf741986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58348/0 (socket says 172.21.15.97:58348) 2024-04-23T10:26:13.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.115+0000 7fdf793f4640 1 -- 172.21.15.97:0/3326887596 learned_addr learned my addr 172.21.15.97:0/3326887596 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:13.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.115+0000 7fdf7a3f6640 1 --2- 172.21.15.97:0/3326887596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf74102a00 0x7fdf7419f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:13.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.115+0000 7fdf79bf5640 1 --2- 172.21.15.97:0/3326887596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf74100e60 0x7fdf741981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:13.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.116+0000 7fdf793f4640 1 -- 172.21.15.97:0/3326887596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf74100e60 msgr2=0x7fdf741981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:13.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.116+0000 7fdf793f4640 1 --2- 172.21.15.97:0/3326887596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf74100e60 0x7fdf741981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:13.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.116+0000 7fdf793f4640 1 -- 172.21.15.97:0/3326887596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf74102a00 msgr2=0x7fdf7419f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:13.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.116+0000 7fdf793f4640 1 --2- 172.21.15.97:0/3326887596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf74102a00 0x7fdf7419f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:13.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.116+0000 7fdf793f4640 1 -- 172.21.15.97:0/3326887596 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf7419fd80 con 0x7fdf74102040 2024-04-23T10:26:13.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.116+0000 7fdf793f4640 1 --2- 172.21.15.97:0/3326887596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf74102040 0x7fdf741986f0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fdf6400d950 tx=0x7fdf6400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:13.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.116+0000 7fdf6affd640 1 -- 172.21.15.97:0/3326887596 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf64017070 con 0x7fdf74102040 2024-04-23T10:26:13.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.116+0000 7fdf7be59640 1 -- 172.21.15.97:0/3326887596 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf741a0070 con 0x7fdf74102040 2024-04-23T10:26:13.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.116+0000 7fdf7be59640 1 -- 172.21.15.97:0/3326887596 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf741a0610 con 0x7fdf74102040 2024-04-23T10:26:13.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.116+0000 7fdf6affd640 1 -- 172.21.15.97:0/3326887596 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf64009c30 con 0x7fdf74102040 2024-04-23T10:26:13.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.116+0000 7fdf6affd640 1 -- 172.21.15.97:0/3326887596 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf64020060 con 0x7fdf74102040 2024-04-23T10:26:13.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.117+0000 7fdf7be59640 1 -- 172.21.15.97:0/3326887596 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf580052c0 con 0x7fdf74102040 2024-04-23T10:26:13.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.118+0000 7fdf6affd640 1 -- 172.21.15.97:0/3326887596 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7fdf64010460 con 0x7fdf74102040 2024-04-23T10:26:13.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.119+0000 7fdf6affd640 1 --2- 172.21.15.97:0/3326887596 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fdf600a3a60 0x7fdf600a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:13.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.119+0000 7fdf79bf5640 1 --2- 172.21.15.97:0/3326887596 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fdf600a3a60 0x7fdf600a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:13.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.119+0000 7fdf6affd640 1 -- 172.21.15.97:0/3326887596 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fdf640cfb30 con 0x7fdf74102040 2024-04-23T10:26:13.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.119+0000 7fdf79bf5640 1 --2- 172.21.15.97:0/3326887596 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fdf600a3a60 0x7fdf600a5f20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fdf70009230 tx=0x7fdf700314a0 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:13.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.125+0000 7fdf6affd640 1 -- 172.21.15.97:0/3326887596 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdf64009de0 con 0x7fdf74102040 2024-04-23T10:26:13.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:12 smithi183 ceph-mon[32317]: [23/Apr/2024:10:26:11] ENGINE Bus STARTING 2024-04-23T10:26:13.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:12 smithi183 ceph-mon[32317]: [23/Apr/2024:10:26:11] ENGINE Serving on https://172.21.15.97:7150 2024-04-23T10:26:13.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:12 smithi183 ceph-mon[32317]: [23/Apr/2024:10:26:12] ENGINE Serving on http://172.21.15.97:8765 2024-04-23T10:26:13.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:12 smithi183 ceph-mon[32317]: [23/Apr/2024:10:26:12] ENGINE Bus STARTED 2024-04-23T10:26:13.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:12 smithi183 ceph-mon[32317]: mgrmap e22: x(active, since 1.02148s), standbys: z, y 2024-04-23T10:26:13.346 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.347+0000 7fdf7be59640 1 -- 172.21.15.97:0/3326887596 --> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fdf58000bf0 con 0x7fdf600a3a60 2024-04-23T10:26:13.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.355+0000 7fdf6affd640 1 -- 172.21.15.97:0/3326887596 <== mgr.15378 v2:172.21.15.97:6800/3567268502 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7fdf58000bf0 con 0x7fdf600a3a60 2024-04-23T10:26:13.354 INFO:teuthology.orchestra.run.smithi097.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-23T00:00:00", "created": "2024-04-23T10:26:12", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-04-23T10:26:13.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.357+0000 7fdf7be59640 1 -- 172.21.15.97:0/3326887596 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fdf600a3a60 msgr2=0x7fdf600a5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:13.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.357+0000 7fdf7be59640 1 --2- 172.21.15.97:0/3326887596 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fdf600a3a60 0x7fdf600a5f20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fdf70009230 tx=0x7fdf700314a0 comp rx=0 tx=0).stop 2024-04-23T10:26:13.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.357+0000 7fdf7be59640 1 -- 172.21.15.97:0/3326887596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf74102040 msgr2=0x7fdf741986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:13.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.357+0000 7fdf7be59640 1 --2- 172.21.15.97:0/3326887596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf74102040 0x7fdf741986f0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fdf6400d950 tx=0x7fdf6400de10 comp rx=0 tx=0).stop 2024-04-23T10:26:13.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.357+0000 7fdf7be59640 1 -- 172.21.15.97:0/3326887596 shutdown_connections 2024-04-23T10:26:13.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.357+0000 7fdf7be59640 1 --2- 172.21.15.97:0/3326887596 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fdf600a3a60 0x7fdf600a5f20 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:13.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.357+0000 7fdf7be59640 1 --2- 172.21.15.97:0/3326887596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf74102a00 0x7fdf7419f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:13.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.357+0000 7fdf7be59640 1 --2- 172.21.15.97:0/3326887596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf74102040 0x7fdf741986f0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:13.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.357+0000 7fdf7be59640 1 --2- 172.21.15.97:0/3326887596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf74100e60 0x7fdf741981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:13.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.357+0000 7fdf7be59640 1 -- 172.21.15.97:0/3326887596 >> 172.21.15.97:0/3326887596 conn(0x7fdf740fc420 msgr2=0x7fdf74069600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:13.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.357+0000 7fdf7be59640 1 -- 172.21.15.97:0/3326887596 shutdown_connections 2024-04-23T10:26:13.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.357+0000 7fdf7be59640 1 -- 172.21.15.97:0/3326887596 wait complete. 2024-04-23T10:26:13.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:12 smithi097 ceph-mon[29197]: [23/Apr/2024:10:26:11] ENGINE Bus STARTING 2024-04-23T10:26:13.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:12 smithi097 ceph-mon[29197]: [23/Apr/2024:10:26:11] ENGINE Serving on https://172.21.15.97:7150 2024-04-23T10:26:13.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:12 smithi097 ceph-mon[29197]: [23/Apr/2024:10:26:12] ENGINE Serving on http://172.21.15.97:8765 2024-04-23T10:26:13.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:12 smithi097 ceph-mon[29197]: [23/Apr/2024:10:26:12] ENGINE Bus STARTED 2024-04-23T10:26:13.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:12 smithi097 ceph-mon[29197]: mgrmap e22: x(active, since 1.02148s), standbys: z, y 2024-04-23T10:26:13.372 DEBUG:teuthology.orchestra.run.smithi097:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-23T10:26:13.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.478+0000 7f5926a79640 1 -- 172.21.15.97:0/3595758604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5920100e60 msgr2=0x7f5920101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:13.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.478+0000 7f5926a79640 1 --2- 172.21.15.97:0/3595758604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5920100e60 0x7f5920101280 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f5918009ca0 tx=0x7f591802a8c0 comp rx=0 tx=0).stop 2024-04-23T10:26:13.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.478+0000 7f5926a79640 1 -- 172.21.15.97:0/3595758604 shutdown_connections 2024-04-23T10:26:13.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.478+0000 7f5926a79640 1 --2- 172.21.15.97:0/3595758604 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5920102a00 0x7f5920109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:13.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.478+0000 7f5926a79640 1 --2- 172.21.15.97:0/3595758604 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5920102040 0x7f59201024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:13.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.478+0000 7f5926a79640 1 --2- 172.21.15.97:0/3595758604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5920100e60 0x7f5920101280 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:13.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.478+0000 7f5926a79640 1 -- 172.21.15.97:0/3595758604 >> 172.21.15.97:0/3595758604 conn(0x7f59200fc420 msgr2=0x7f59200fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:13.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.479+0000 7f5926a79640 1 -- 172.21.15.97:0/3595758604 shutdown_connections 2024-04-23T10:26:13.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.479+0000 7f5926a79640 1 -- 172.21.15.97:0/3595758604 wait complete. 2024-04-23T10:26:13.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.479+0000 7f5926a79640 1 Processor -- start 2024-04-23T10:26:13.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.479+0000 7f5926a79640 1 -- start start 2024-04-23T10:26:13.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.479+0000 7f5926a79640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5920100e60 0x7f592010b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:13.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.479+0000 7f5926a79640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5920102040 0x7f5920109ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:13.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.479+0000 7f5926a79640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5920102a00 0x7f592010a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:13.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.479+0000 7f5926a79640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f592010db30 con 0x7f5920102a00 2024-04-23T10:26:13.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.479+0000 7f5926a79640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f592010d9b0 con 0x7f5920100e60 2024-04-23T10:26:13.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.479+0000 7f5926a79640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f592010dcb0 con 0x7f5920102040 2024-04-23T10:26:13.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.479+0000 7f5925016640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5920102a00 0x7f592010a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:13.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.479+0000 7f5925016640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5920102a00 0x7f592010a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58376/0 (socket says 172.21.15.97:58376) 2024-04-23T10:26:13.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.479+0000 7f5925016640 1 -- 172.21.15.97:0/524344487 learned_addr learned my addr 172.21.15.97:0/524344487 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:13.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.479+0000 7f5924815640 1 --2- 172.21.15.97:0/524344487 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5920100e60 0x7f592010b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:13.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.479+0000 7f591ffff640 1 --2- 172.21.15.97:0/524344487 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5920102040 0x7f5920109ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:13.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.480+0000 7f5925016640 1 -- 172.21.15.97:0/524344487 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5920102040 msgr2=0x7f5920109ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:13.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.480+0000 7f5925016640 1 --2- 172.21.15.97:0/524344487 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5920102040 0x7f5920109ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:13.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.480+0000 7f5925016640 1 -- 172.21.15.97:0/524344487 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5920100e60 msgr2=0x7f592010b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:13.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.480+0000 7f5925016640 1 --2- 172.21.15.97:0/524344487 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5920100e60 0x7f592010b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:13.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.480+0000 7f5925016640 1 -- 172.21.15.97:0/524344487 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f592010ac70 con 0x7f5920102a00 2024-04-23T10:26:13.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.480+0000 7f5925016640 1 --2- 172.21.15.97:0/524344487 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5920102a00 0x7f592010a3e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f591000d950 tx=0x7f591000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:13.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.480+0000 7f591dffb640 1 -- 172.21.15.97:0/524344487 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5910017030 con 0x7f5920102a00 2024-04-23T10:26:13.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.480+0000 7f591dffb640 1 -- 172.21.15.97:0/524344487 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5910004020 con 0x7f5920102a00 2024-04-23T10:26:13.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.480+0000 7f5926a79640 1 -- 172.21.15.97:0/524344487 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59201a22d0 con 0x7f5920102a00 2024-04-23T10:26:13.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.480+0000 7f591dffb640 1 -- 172.21.15.97:0/524344487 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5910003b50 con 0x7f5920102a00 2024-04-23T10:26:13.481 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.480+0000 7f5926a79640 1 -- 172.21.15.97:0/524344487 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59201a27d0 con 0x7f5920102a00 2024-04-23T10:26:13.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.481+0000 7f5926a79640 1 -- 172.21.15.97:0/524344487 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5920108280 con 0x7f5920102a00 2024-04-23T10:26:13.482 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.482+0000 7f591dffb640 1 -- 172.21.15.97:0/524344487 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f591001a070 con 0x7f5920102a00 2024-04-23T10:26:13.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.483+0000 7f591dffb640 1 --2- 172.21.15.97:0/524344487 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f590c0a3a60 0x7f590c0a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:13.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.483+0000 7f5924815640 1 --2- 172.21.15.97:0/524344487 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f590c0a3a60 0x7f590c0a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:13.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.483+0000 7f591dffb640 1 -- 172.21.15.97:0/524344487 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f59100cf830 con 0x7f5920102a00 2024-04-23T10:26:13.483 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.484+0000 7f5924815640 1 --2- 172.21.15.97:0/524344487 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f590c0a3a60 0x7f590c0a5f20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5918000f80 tx=0x7f5918005dc0 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:13.486 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.488+0000 7f591dffb640 1 -- 172.21.15.97:0/524344487 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5910021070 con 0x7f5920102a00 2024-04-23T10:26:13.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.692+0000 7f5926a79640 1 -- 172.21.15.97:0/524344487 --> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f5920062390 con 0x7f590c0a3a60 2024-04-23T10:26:13.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.698+0000 7f591dffb640 1 -- 172.21.15.97:0/524344487 <== mgr.15378 v2:172.21.15.97:6800/3567268502 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f5920062390 con 0x7f590c0a3a60 2024-04-23T10:26:13.697 INFO:teuthology.orchestra.run.smithi097.stdout:/ 1m 6m3h 2024-04-23T10:26:13.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.701+0000 7f5926a79640 1 -- 172.21.15.97:0/524344487 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f590c0a3a60 msgr2=0x7f590c0a5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:13.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.701+0000 7f5926a79640 1 --2- 172.21.15.97:0/524344487 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f590c0a3a60 0x7f590c0a5f20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5918000f80 tx=0x7f5918005dc0 comp rx=0 tx=0).stop 2024-04-23T10:26:13.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.701+0000 7f5926a79640 1 -- 172.21.15.97:0/524344487 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5920102a00 msgr2=0x7f592010a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:13.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.701+0000 7f5926a79640 1 --2- 172.21.15.97:0/524344487 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5920102a00 0x7f592010a3e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f591000d950 tx=0x7f591000de10 comp rx=0 tx=0).stop 2024-04-23T10:26:13.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.701+0000 7f5926a79640 1 -- 172.21.15.97:0/524344487 shutdown_connections 2024-04-23T10:26:13.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.701+0000 7f5926a79640 1 --2- 172.21.15.97:0/524344487 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f590c0a3a60 0x7f590c0a5f20 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:13.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.701+0000 7f5926a79640 1 --2- 172.21.15.97:0/524344487 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5920102a00 0x7f592010a3e0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:13.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.701+0000 7f5926a79640 1 --2- 172.21.15.97:0/524344487 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5920102040 0x7f5920109ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:13.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.701+0000 7f5926a79640 1 --2- 172.21.15.97:0/524344487 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5920100e60 0x7f592010b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:13.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.701+0000 7f5926a79640 1 -- 172.21.15.97:0/524344487 >> 172.21.15.97:0/524344487 conn(0x7f59200fc420 msgr2=0x7f5920105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:13.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.701+0000 7f5926a79640 1 -- 172.21.15.97:0/524344487 shutdown_connections 2024-04-23T10:26:13.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.701+0000 7f5926a79640 1 -- 172.21.15.97:0/524344487 wait complete. 2024-04-23T10:26:13.710 DEBUG:teuthology.orchestra.run.smithi097:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-04-23T10:26:13.735 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-04-23T10:26:13.745 INFO:teuthology.run_tasks:Running task background_exec... 2024-04-23T10:26:13.754 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi097.front.sepia.ceph.com 2024-04-23T10:26:13.754 DEBUG:teuthology.orchestra.run.smithi097:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-04-23T10:26:13.778 INFO:teuthology.run_tasks:Running task quiescer... 2024-04-23T10:26:13.789 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T10:26:13.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.896+0000 7fedf7d20640 1 -- 172.21.15.97:0/2070181467 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fedf0103b50 msgr2=0x7fedf010a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:13.895 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.896+0000 7fedf7d20640 1 --2- 172.21.15.97:0/2070181467 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fedf0103b50 0x7fedf010a3e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fede0009900 tx=0x7fede002a410 comp rx=0 tx=0).stop 2024-04-23T10:26:13.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.896+0000 7fedf7d20640 1 -- 172.21.15.97:0/2070181467 shutdown_connections 2024-04-23T10:26:13.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.896+0000 7fedf7d20640 1 --2- 172.21.15.97:0/2070181467 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fedf0103b50 0x7fedf010a3e0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:13.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.897+0000 7fedf7d20640 1 --2- 172.21.15.97:0/2070181467 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fedf0103190 0x7fedf0103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:13.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.897+0000 7fedf7d20640 1 --2- 172.21.15.97:0/2070181467 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fedf0101fa0 0x7fedf01023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:13.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.897+0000 7fedf7d20640 1 -- 172.21.15.97:0/2070181467 >> 172.21.15.97:0/2070181467 conn(0x7fedf00fd550 msgr2=0x7fedf00ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:13.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.897+0000 7fedf7d20640 1 -- 172.21.15.97:0/2070181467 shutdown_connections 2024-04-23T10:26:13.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.897+0000 7fedf7d20640 1 -- 172.21.15.97:0/2070181467 wait complete. 2024-04-23T10:26:13.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.897+0000 7fedf7d20640 1 Processor -- start 2024-04-23T10:26:13.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.897+0000 7fedf7d20640 1 -- start start 2024-04-23T10:26:13.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.897+0000 7fedf7d20640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fedf0101fa0 0x7fedf0198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:13.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.897+0000 7fedf7d20640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fedf0103190 0x7fedf01986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:13.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.897+0000 7fedf7d20640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fedf0103b50 0x7fedf019f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:13.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.898+0000 7fedf7d20640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fedf010daf0 con 0x7fedf0103b50 2024-04-23T10:26:13.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.898+0000 7fedf7d20640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fedf010d970 con 0x7fedf0103190 2024-04-23T10:26:13.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.898+0000 7fedf7d20640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fedf010dc70 con 0x7fedf0101fa0 2024-04-23T10:26:13.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.898+0000 7fedf62bd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fedf0103b50 0x7fedf019f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:13.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.898+0000 7fedf62bd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fedf0103b50 0x7fedf019f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58398/0 (socket says 172.21.15.97:58398) 2024-04-23T10:26:13.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.898+0000 7fedf52bb640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fedf0103190 0x7fedf01986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:13.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.898+0000 7fedf62bd640 1 -- 172.21.15.97:0/2234761069 learned_addr learned my addr 172.21.15.97:0/2234761069 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:13.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.898+0000 7fedf5abc640 1 --2- 172.21.15.97:0/2234761069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fedf0101fa0 0x7fedf0198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:13.898 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.898+0000 7fedf62bd640 1 -- 172.21.15.97:0/2234761069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fedf0101fa0 msgr2=0x7fedf0198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:13.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.898+0000 7fedf62bd640 1 --2- 172.21.15.97:0/2234761069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fedf0101fa0 0x7fedf0198160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:13.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.898+0000 7fedf62bd640 1 -- 172.21.15.97:0/2234761069 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fedf0103190 msgr2=0x7fedf01986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:13.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.898+0000 7fedf62bd640 1 --2- 172.21.15.97:0/2234761069 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fedf0103190 0x7fedf01986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:13.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.898+0000 7fedf62bd640 1 -- 172.21.15.97:0/2234761069 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedf019fd80 con 0x7fedf0103b50 2024-04-23T10:26:13.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.898+0000 7fedf62bd640 1 --2- 172.21.15.97:0/2234761069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fedf0103b50 0x7fedf019f710 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fede0005ab0 tx=0x7fede0003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:13.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.908+0000 7fede6ffd640 1 -- 172.21.15.97:0/2234761069 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede003f070 con 0x7fedf0103b50 2024-04-23T10:26:13.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.908+0000 7fede6ffd640 1 -- 172.21.15.97:0/2234761069 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fede0031c50 con 0x7fedf0103b50 2024-04-23T10:26:13.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.908+0000 7fede6ffd640 1 -- 172.21.15.97:0/2234761069 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede0035030 con 0x7fedf0103b50 2024-04-23T10:26:13.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.908+0000 7fedf7d20640 1 -- 172.21.15.97:0/2234761069 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedf01a0010 con 0x7fedf0103b50 2024-04-23T10:26:13.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.908+0000 7fedf7d20640 1 -- 172.21.15.97:0/2234761069 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedf01a0550 con 0x7fedf0103b50 2024-04-23T10:26:13.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.909+0000 7fedf7d20640 1 -- 172.21.15.97:0/2234761069 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedf01092c0 con 0x7fedf0103b50 2024-04-23T10:26:13.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.910+0000 7fede6ffd640 1 -- 172.21.15.97:0/2234761069 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7fede0042070 con 0x7fedf0103b50 2024-04-23T10:26:13.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.911+0000 7fede6ffd640 1 --2- 172.21.15.97:0/2234761069 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7feddc0a3e30 0x7feddc0a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:13.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.911+0000 7fedf5abc640 1 --2- 172.21.15.97:0/2234761069 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7feddc0a3e30 0x7feddc0a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:13.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.911+0000 7fede6ffd640 1 -- 172.21.15.97:0/2234761069 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fede00f1080 con 0x7fedf0103b50 2024-04-23T10:26:13.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.912+0000 7fedf5abc640 1 --2- 172.21.15.97:0/2234761069 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7feddc0a3e30 0x7feddc0a62f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fedf0102ff0 tx=0x7fede8009720 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:13.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:13.916+0000 7fede6ffd640 1 -- 172.21.15.97:0/2234761069 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fede00bcfb0 con 0x7fedf0103b50 2024-04-23T10:26:14.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:13 smithi097 ceph-mon[29197]: from='client.15408 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:26:14.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:13 smithi097 ceph-mon[29197]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:26:14.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:13 smithi097 ceph-mon[29197]: from='client.15456 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:26:14.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:13 smithi097 ceph-mon[29197]: from='client.25016 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:26:14.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:13 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:26:14.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:13 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:26:14.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:13 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:26:14.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:13 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:26:14.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:13 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:26:14.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:13 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:26:14.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.114+0000 7fedf7d20640 1 -- 172.21.15.97:0/2234761069 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fedf0062b50 con 0x7fedf0103b50 2024-04-23T10:26:14.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.116+0000 7fede6ffd640 1 -- 172.21.15.97:0/2234761069 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22981 (secure 0 0 0) 0x7fede00c12b0 con 0x7fedf0103b50 2024-04-23T10:26:14.115 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:26:14.116 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":89,"fsid":"9b22af88-0159-11ef-bc93-c7b262605968","created":"2024-04-23T10:16:05.983305+0000","modified":"2024-04-23T10:26:11.109574+0000","last_up_change":"2024-04-23T10:23:13.114761+0000","last_in_change":"2024-04-23T10:22:52.804880+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T10:19:25.559138+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T10:24:04.570049+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T10:24:05.861631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"58cb995d-de35-4522-ac81-4982401b260d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6803","nonce":3031221802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6805","nonce":3031221802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6809","nonce":3031221802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6807","nonce":3031221802}]},"public_addr":"172.21.15.97:6803/3031221802","cluster_addr":"172.21.15.97:6805/3031221802","heartbeat_back_addr":"172.21.15.97:6809/3031221802","heartbeat_front_addr":"172.21.15.97:6807/3031221802","state":["exists","up"]},{"osd":1,"uuid":"b47bd8ee-1f21-4834-acf5-bb40f97ab3ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6801","nonce":1108870440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6803","nonce":1108870440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6807","nonce":1108870440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6805","nonce":1108870440}]},"public_addr":"172.21.15.183:6801/1108870440","cluster_addr":"172.21.15.183:6803/1108870440","heartbeat_back_addr":"172.21.15.183:6807/1108870440","heartbeat_front_addr":"172.21.15.183:6805/1108870440","state":["exists","up"]},{"osd":2,"uuid":"17c4eec3-6c47-4334-9b63-1d5ad9ec5988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6801","nonce":2877039731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6803","nonce":2877039731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6807","nonce":2877039731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6805","nonce":2877039731}]},"public_addr":"172.21.15.196:6801/2877039731","cluster_addr":"172.21.15.196:6803/2877039731","heartbeat_back_addr":"172.21.15.196:6807/2877039731","heartbeat_front_addr":"172.21.15.196:6805/2877039731","state":["exists","up"]},{"osd":3,"uuid":"3f731c38-2e45-4ca1-8dce-efc4a1b2166b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6811","nonce":4276311194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6813","nonce":4276311194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6817","nonce":4276311194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6815","nonce":4276311194}]},"public_addr":"172.21.15.97:6811/4276311194","cluster_addr":"172.21.15.97:6813/4276311194","heartbeat_back_addr":"172.21.15.97:6817/4276311194","heartbeat_front_addr":"172.21.15.97:6815/4276311194","state":["exists","up"]},{"osd":4,"uuid":"fdcafb14-9b82-4134-bd5b-79c0516f6cbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6809","nonce":63913916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6811","nonce":63913916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6815","nonce":63913916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6813","nonce":63913916}]},"public_addr":"172.21.15.183:6809/63913916","cluster_addr":"172.21.15.183:6811/63913916","heartbeat_back_addr":"172.21.15.183:6815/63913916","heartbeat_front_addr":"172.21.15.183:6813/63913916","state":["exists","up"]},{"osd":5,"uuid":"cdcc18d8-da3c-4012-bf12-876bb39e66eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6809","nonce":2804446896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6811","nonce":2804446896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6815","nonce":2804446896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6813","nonce":2804446896}]},"public_addr":"172.21.15.196:6809/2804446896","cluster_addr":"172.21.15.196:6811/2804446896","heartbeat_back_addr":"172.21.15.196:6815/2804446896","heartbeat_front_addr":"172.21.15.196:6813/2804446896","state":["exists","up"]},{"osd":6,"uuid":"071abb5c-2edf-4832-b8aa-3b80d20096fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6819","nonce":2080916705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6821","nonce":2080916705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6825","nonce":2080916705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6823","nonce":2080916705}]},"public_addr":"172.21.15.97:6819/2080916705","cluster_addr":"172.21.15.97:6821/2080916705","heartbeat_back_addr":"172.21.15.97:6825/2080916705","heartbeat_front_addr":"172.21.15.97:6823/2080916705","state":["exists","up"]},{"osd":7,"uuid":"9d54760b-66df-486b-8abb-017f3b64b4d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6817","nonce":1865873033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6819","nonce":1865873033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6823","nonce":1865873033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6821","nonce":1865873033}]},"public_addr":"172.21.15.183:6817/1865873033","cluster_addr":"172.21.15.183:6819/1865873033","heartbeat_back_addr":"172.21.15.183:6823/1865873033","heartbeat_front_addr":"172.21.15.183:6821/1865873033","state":["exists","up"]},{"osd":8,"uuid":"5cd3100a-bd6a-4be5-b5e9-0d0afc325193","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6817","nonce":4063932139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6819","nonce":4063932139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6823","nonce":4063932139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6821","nonce":4063932139}]},"public_addr":"172.21.15.196:6817/4063932139","cluster_addr":"172.21.15.196:6819/4063932139","heartbeat_back_addr":"172.21.15.196:6823/4063932139","heartbeat_front_addr":"172.21.15.196:6821/4063932139","state":["exists","up"]},{"osd":9,"uuid":"b6a5f4a1-4ecc-4f73-a2df-07551d9298ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6827","nonce":1887432474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6829","nonce":1887432474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6833","nonce":1887432474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6831","nonce":1887432474}]},"public_addr":"172.21.15.97:6827/1887432474","cluster_addr":"172.21.15.97:6829/1887432474","heartbeat_back_addr":"172.21.15.97:6833/1887432474","heartbeat_front_addr":"172.21.15.97:6831/1887432474","state":["exists","up"]},{"osd":10,"uuid":"86f59816-1ea6-4ae9-8796-cf5fc4831064","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6825","nonce":799434647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6827","nonce":799434647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6831","nonce":799434647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6829","nonce":799434647}]},"public_addr":"172.21.15.183:6825/799434647","cluster_addr":"172.21.15.183:6827/799434647","heartbeat_back_addr":"172.21.15.183:6831/799434647","heartbeat_front_addr":"172.21.15.183:6829/799434647","state":["exists","up"]},{"osd":11,"uuid":"95d64a89-a9f8-4c7a-8c0d-da7fb26cfff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6825","nonce":1873969467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6827","nonce":1873969467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6831","nonce":1873969467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6829","nonce":1873969467}]},"public_addr":"172.21.15.196:6825/1873969467","cluster_addr":"172.21.15.196:6827/1873969467","heartbeat_back_addr":"172.21.15.196:6831/1873969467","heartbeat_front_addr":"172.21.15.196:6829/1873969467","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:18:33.721156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:18:57.586709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:19:22.069458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:19:46.451597+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:20:11.561019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:20:36.298672+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:21:01.661804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:21:27.324128+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:21:52.898197+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:22:19.614326+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:22:45.154917+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:23:11.222612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/233335470":"2024-04-24T10:26:11.109535+0000","172.21.15.97:6800/3149043749":"2024-04-24T10:26:11.109535+0000","172.21.15.97:0/338277142":"2024-04-24T10:26:11.109535+0000","172.21.15.97:0/3866016040":"2024-04-24T10:16:59.468237+0000","172.21.15.97:6801/3297166858":"2024-04-24T10:16:59.468237+0000","172.21.15.97:0/3804743548":"2024-04-24T10:16:59.468237+0000","172.21.15.97:6800/3706463017":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/3715079438":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/597405418":"2024-04-24T10:26:11.109535+0000","172.21.15.97:0/786806834":"2024-04-24T10:16:59.468237+0000","172.21.15.97:6801/3706463017":"2024-04-24T10:16:33.128917+0000","172.21.15.97:6801/3149043749":"2024-04-24T10:26:11.109535+0000","172.21.15.97:0/3294175554":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/1449136760":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/3198144747":"2024-04-24T10:26:11.109535+0000","172.21.15.97:6800/3297166858":"2024-04-24T10:16:59.468237+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T10:26:14.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.118+0000 7fedf7d20640 1 -- 172.21.15.97:0/2234761069 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7feddc0a3e30 msgr2=0x7feddc0a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:14.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.118+0000 7fedf7d20640 1 --2- 172.21.15.97:0/2234761069 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7feddc0a3e30 0x7feddc0a62f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fedf0102ff0 tx=0x7fede8009720 comp rx=0 tx=0).stop 2024-04-23T10:26:14.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.118+0000 7fedf7d20640 1 -- 172.21.15.97:0/2234761069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fedf0103b50 msgr2=0x7fedf019f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:14.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.118+0000 7fedf7d20640 1 --2- 172.21.15.97:0/2234761069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fedf0103b50 0x7fedf019f710 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fede0005ab0 tx=0x7fede0003b90 comp rx=0 tx=0).stop 2024-04-23T10:26:14.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.119+0000 7fedf7d20640 1 -- 172.21.15.97:0/2234761069 shutdown_connections 2024-04-23T10:26:14.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.119+0000 7fedf7d20640 1 --2- 172.21.15.97:0/2234761069 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7feddc0a3e30 0x7feddc0a62f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:14.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.119+0000 7fedf7d20640 1 --2- 172.21.15.97:0/2234761069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fedf0103b50 0x7fedf019f710 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:14.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.119+0000 7fedf7d20640 1 --2- 172.21.15.97:0/2234761069 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fedf0103190 0x7fedf01986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:14.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.119+0000 7fedf7d20640 1 --2- 172.21.15.97:0/2234761069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fedf0101fa0 0x7fedf0198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:14.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.119+0000 7fedf7d20640 1 -- 172.21.15.97:0/2234761069 >> 172.21.15.97:0/2234761069 conn(0x7fedf00fd550 msgr2=0x7fedf0106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:14.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.119+0000 7fedf7d20640 1 -- 172.21.15.97:0/2234761069 shutdown_connections 2024-04-23T10:26:14.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.119+0000 7fedf7d20640 1 -- 172.21.15.97:0/2234761069 wait complete. 2024-04-23T10:26:14.129 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T10:19:25.559138+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T10:24:04.570049+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T10:24:05.861631+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T10:26:14.130 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T10:26:14.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:13 smithi183 ceph-mon[32317]: from='client.15408 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:26:14.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:13 smithi183 ceph-mon[32317]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:26:14.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:13 smithi183 ceph-mon[32317]: from='client.15456 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:26:14.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:13 smithi183 ceph-mon[32317]: from='client.25016 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:26:14.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:13 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:26:14.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:13 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:26:14.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:13 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:26:14.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:13 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:26:14.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:13 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:26:14.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:13 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:26:14.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.235+0000 7fc72dc91640 1 -- 172.21.15.97:0/996473288 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc728100e90 msgr2=0x7fc7281012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:14.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.235+0000 7fc72dc91640 1 --2- 172.21.15.97:0/996473288 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc728100e90 0x7fc7281012b0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fc718009900 tx=0x7fc71802a410 comp rx=0 tx=0).stop 2024-04-23T10:26:14.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.235+0000 7fc72dc91640 1 -- 172.21.15.97:0/996473288 shutdown_connections 2024-04-23T10:26:14.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.235+0000 7fc72dc91640 1 --2- 172.21.15.97:0/996473288 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7281029a0 0x7fc7281092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:14.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.235+0000 7fc72dc91640 1 --2- 172.21.15.97:0/996473288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc728101fe0 0x7fc728102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:14.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.235+0000 7fc72dc91640 1 --2- 172.21.15.97:0/996473288 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc728100e90 0x7fc7281012b0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:14.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.235+0000 7fc72dc91640 1 -- 172.21.15.97:0/996473288 >> 172.21.15.97:0/996473288 conn(0x7fc7280fc400 msgr2=0x7fc7280fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:14.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.235+0000 7fc72dc91640 1 -- 172.21.15.97:0/996473288 shutdown_connections 2024-04-23T10:26:14.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.235+0000 7fc72dc91640 1 -- 172.21.15.97:0/996473288 wait complete. 2024-04-23T10:26:14.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.235+0000 7fc72dc91640 1 Processor -- start 2024-04-23T10:26:14.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.236+0000 7fc72dc91640 1 -- start start 2024-04-23T10:26:14.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.236+0000 7fc72dc91640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc728100e90 0x7fc728069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:14.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.236+0000 7fc72dc91640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc728101fe0 0x7fc72819f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:14.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.236+0000 7fc72dc91640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7281029a0 0x7fc7281a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:14.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.236+0000 7fc72dc91640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc72810c9e0 con 0x7fc7281029a0 2024-04-23T10:26:14.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.236+0000 7fc72dc91640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc72810c860 con 0x7fc728101fe0 2024-04-23T10:26:14.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.236+0000 7fc72dc91640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc72810cb60 con 0x7fc728100e90 2024-04-23T10:26:14.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.236+0000 7fc727fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7281029a0 0x7fc7281a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:14.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.236+0000 7fc727fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7281029a0 0x7fc7281a18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58410/0 (socket says 172.21.15.97:58410) 2024-04-23T10:26:14.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.236+0000 7fc726ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc728101fe0 0x7fc72819f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:14.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.236+0000 7fc727fff640 1 -- 172.21.15.97:0/1882928253 learned_addr learned my addr 172.21.15.97:0/1882928253 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:14.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.236+0000 7fc7277fe640 1 --2- 172.21.15.97:0/1882928253 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc728100e90 0x7fc728069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:14.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.236+0000 7fc727fff640 1 -- 172.21.15.97:0/1882928253 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc728100e90 msgr2=0x7fc728069c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:14.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.236+0000 7fc727fff640 1 --2- 172.21.15.97:0/1882928253 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc728100e90 0x7fc728069c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:14.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.236+0000 7fc727fff640 1 -- 172.21.15.97:0/1882928253 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc728101fe0 msgr2=0x7fc72819f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:14.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.236+0000 7fc727fff640 1 --2- 172.21.15.97:0/1882928253 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc728101fe0 0x7fc72819f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:14.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.236+0000 7fc727fff640 1 -- 172.21.15.97:0/1882928253 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7281a20a0 con 0x7fc7281029a0 2024-04-23T10:26:14.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.236+0000 7fc726ffd640 1 --2- 172.21.15.97:0/1882928253 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc728101fe0 0x7fc72819f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T10:26:14.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.236+0000 7fc727fff640 1 --2- 172.21.15.97:0/1882928253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7281029a0 0x7fc7281a18b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fc72000eb10 tx=0x7fc72000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:14.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.237+0000 7fc724ff9640 1 -- 172.21.15.97:0/1882928253 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc720004b20 con 0x7fc7281029a0 2024-04-23T10:26:14.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.237+0000 7fc72dc91640 1 -- 172.21.15.97:0/1882928253 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7281a2330 con 0x7fc7281029a0 2024-04-23T10:26:14.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.237+0000 7fc724ff9640 1 -- 172.21.15.97:0/1882928253 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc720004cc0 con 0x7fc7281029a0 2024-04-23T10:26:14.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.237+0000 7fc724ff9640 1 -- 172.21.15.97:0/1882928253 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc720010690 con 0x7fc7281029a0 2024-04-23T10:26:14.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.237+0000 7fc72dc91640 1 -- 172.21.15.97:0/1882928253 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7281a2880 con 0x7fc7281029a0 2024-04-23T10:26:14.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.238+0000 7fc72dc91640 1 -- 172.21.15.97:0/1882928253 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc728108210 con 0x7fc7281029a0 2024-04-23T10:26:14.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.239+0000 7fc724ff9640 1 -- 172.21.15.97:0/1882928253 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7fc720005020 con 0x7fc7281029a0 2024-04-23T10:26:14.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.239+0000 7fc724ff9640 1 --2- 172.21.15.97:0/1882928253 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fc7100a3de0 0x7fc7100a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:14.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.240+0000 7fc7277fe640 1 --2- 172.21.15.97:0/1882928253 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fc7100a3de0 0x7fc7100a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:14.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.240+0000 7fc724ff9640 1 -- 172.21.15.97:0/1882928253 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fc7200d0e70 con 0x7fc7281029a0 2024-04-23T10:26:14.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.240+0000 7fc7277fe640 1 --2- 172.21.15.97:0/1882928253 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fc7100a3de0 0x7fc7100a62a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc718000c00 tx=0x7fc7180314a0 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:14.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.248+0000 7fc724ff9640 1 -- 172.21.15.97:0/1882928253 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc72009d3f0 con 0x7fc7281029a0 2024-04-23T10:26:14.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:13 smithi196 ceph-mon[32919]: from='client.15408 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:26:14.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:13 smithi196 ceph-mon[32919]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:26:14.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:13 smithi196 ceph-mon[32919]: from='client.15456 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:26:14.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:13 smithi196 ceph-mon[32919]: from='client.25016 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:26:14.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:13 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:26:14.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:13 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:26:14.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:13 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:26:14.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:13 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:26:14.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:13 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:26:14.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:13 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:26:14.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.435+0000 7fc72dc91640 1 -- 172.21.15.97:0/1882928253 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc72802d7c0 con 0x7fc7281029a0 2024-04-23T10:26:14.434 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.436+0000 7fc724ff9640 1 -- 172.21.15.97:0/1882928253 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7fc72009ccf0 con 0x7fc7281029a0 2024-04-23T10:26:14.435 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-23T10:26:14.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.438+0000 7fc72dc91640 1 -- 172.21.15.97:0/1882928253 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fc7100a3de0 msgr2=0x7fc7100a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:14.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.438+0000 7fc72dc91640 1 --2- 172.21.15.97:0/1882928253 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fc7100a3de0 0x7fc7100a62a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc718000c00 tx=0x7fc7180314a0 comp rx=0 tx=0).stop 2024-04-23T10:26:14.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.438+0000 7fc72dc91640 1 -- 172.21.15.97:0/1882928253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7281029a0 msgr2=0x7fc7281a18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:14.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.438+0000 7fc72dc91640 1 --2- 172.21.15.97:0/1882928253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7281029a0 0x7fc7281a18b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fc72000eb10 tx=0x7fc72000c2d0 comp rx=0 tx=0).stop 2024-04-23T10:26:14.437 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.438+0000 7fc72dc91640 1 -- 172.21.15.97:0/1882928253 shutdown_connections 2024-04-23T10:26:14.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.438+0000 7fc72dc91640 1 --2- 172.21.15.97:0/1882928253 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fc7100a3de0 0x7fc7100a62a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:14.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.438+0000 7fc72dc91640 1 --2- 172.21.15.97:0/1882928253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7281029a0 0x7fc7281a18b0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:14.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.438+0000 7fc72dc91640 1 --2- 172.21.15.97:0/1882928253 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc728101fe0 0x7fc72819f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:14.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.438+0000 7fc72dc91640 1 --2- 172.21.15.97:0/1882928253 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc728100e90 0x7fc728069c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:14.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.438+0000 7fc72dc91640 1 -- 172.21.15.97:0/1882928253 >> 172.21.15.97:0/1882928253 conn(0x7fc7280fc400 msgr2=0x7fc7281053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:14.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.438+0000 7fc72dc91640 1 -- 172.21.15.97:0/1882928253 shutdown_connections 2024-04-23T10:26:14.438 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.438+0000 7fc72dc91640 1 -- 172.21.15.97:0/1882928253 wait complete. 2024-04-23T10:26:14.448 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T10:26:14.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.561+0000 7f9196a0b640 1 -- 172.21.15.97:0/3681784828 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9190100860 msgr2=0x7f9190100c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:14.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.561+0000 7f9196a0b640 1 --2- 172.21.15.97:0/3681784828 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9190100860 0x7f9190100c80 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f9180009900 tx=0x7f918002a480 comp rx=0 tx=0).stop 2024-04-23T10:26:14.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.561+0000 7f9196a0b640 1 -- 172.21.15.97:0/3681784828 shutdown_connections 2024-04-23T10:26:14.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.561+0000 7f9196a0b640 1 --2- 172.21.15.97:0/3681784828 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9190101b60 0x7f9190101fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:14.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.561+0000 7f9196a0b640 1 --2- 172.21.15.97:0/3681784828 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91901011c0 0x7f9190101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:14.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.561+0000 7f9196a0b640 1 --2- 172.21.15.97:0/3681784828 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9190100860 0x7f9190100c80 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:14.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.561+0000 7f9196a0b640 1 -- 172.21.15.97:0/3681784828 >> 172.21.15.97:0/3681784828 conn(0x7f91900fc3e0 msgr2=0x7f91900fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:14.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.561+0000 7f9196a0b640 1 -- 172.21.15.97:0/3681784828 shutdown_connections 2024-04-23T10:26:14.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.561+0000 7f9196a0b640 1 -- 172.21.15.97:0/3681784828 wait complete. 2024-04-23T10:26:14.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.562+0000 7f9196a0b640 1 Processor -- start 2024-04-23T10:26:14.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.562+0000 7f9196a0b640 1 -- start start 2024-04-23T10:26:14.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.562+0000 7f9196a0b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9190100860 0x7f919019a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:14.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.562+0000 7f9196a0b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f91901011c0 0x7f919019a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:14.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.562+0000 7f9196a0b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9190101b60 0x7f91901a1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:14.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.562+0000 7f9196a0b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f919010fcb0 con 0x7f9190101b60 2024-04-23T10:26:14.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.562+0000 7f9196a0b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f919010fb30 con 0x7f91901011c0 2024-04-23T10:26:14.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.562+0000 7f9196a0b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f919010fe30 con 0x7f9190100860 2024-04-23T10:26:14.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.562+0000 7f9194fa8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9190101b60 0x7f91901a1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:14.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.562+0000 7f918f7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f91901011c0 0x7f919019a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:14.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.562+0000 7f9194fa8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9190101b60 0x7f91901a1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58416/0 (socket says 172.21.15.97:58416) 2024-04-23T10:26:14.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.562+0000 7f9194fa8640 1 -- 172.21.15.97:0/1364190033 learned_addr learned my addr 172.21.15.97:0/1364190033 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:14.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.562+0000 7f918ffff640 1 --2- 172.21.15.97:0/1364190033 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9190100860 0x7f919019a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:14.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.563+0000 7f9194fa8640 1 -- 172.21.15.97:0/1364190033 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9190100860 msgr2=0x7f919019a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:14.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.563+0000 7f9194fa8640 1 --2- 172.21.15.97:0/1364190033 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9190100860 0x7f919019a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:14.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.563+0000 7f9194fa8640 1 -- 172.21.15.97:0/1364190033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f91901011c0 msgr2=0x7f919019a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:14.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.563+0000 7f9194fa8640 1 --2- 172.21.15.97:0/1364190033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f91901011c0 0x7f919019a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:14.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.563+0000 7f9194fa8640 1 -- 172.21.15.97:0/1364190033 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91901a2000 con 0x7f9190101b60 2024-04-23T10:26:14.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.563+0000 7f9194fa8640 1 --2- 172.21.15.97:0/1364190033 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9190101b60 0x7f91901a1990 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f9188009fd0 tx=0x7f918800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:14.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.563+0000 7f918d7fa640 1 -- 172.21.15.97:0/1364190033 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9188017070 con 0x7f9190101b60 2024-04-23T10:26:14.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.563+0000 7f918d7fa640 1 -- 172.21.15.97:0/1364190033 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9188003e20 con 0x7f9190101b60 2024-04-23T10:26:14.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.563+0000 7f918d7fa640 1 -- 172.21.15.97:0/1364190033 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9188010460 con 0x7f9190101b60 2024-04-23T10:26:14.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.563+0000 7f9196a0b640 1 -- 172.21.15.97:0/1364190033 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91901a22f0 con 0x7f9190101b60 2024-04-23T10:26:14.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.563+0000 7f9196a0b640 1 -- 172.21.15.97:0/1364190033 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91901a2840 con 0x7f9190101b60 2024-04-23T10:26:14.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.564+0000 7f9196a0b640 1 -- 172.21.15.97:0/1364190033 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f919010b7d0 con 0x7f9190101b60 2024-04-23T10:26:14.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.565+0000 7f918d7fa640 1 -- 172.21.15.97:0/1364190033 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7f9188003970 con 0x7f9190101b60 2024-04-23T10:26:14.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.566+0000 7f918d7fa640 1 --2- 172.21.15.97:0/1364190033 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f91780a3db0 0x7f91780a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:14.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.566+0000 7f918ffff640 1 --2- 172.21.15.97:0/1364190033 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f91780a3db0 0x7f91780a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:14.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.566+0000 7f918d7fa640 1 -- 172.21.15.97:0/1364190033 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f91880d05e0 con 0x7f9190101b60 2024-04-23T10:26:14.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.567+0000 7f918ffff640 1 --2- 172.21.15.97:0/1364190033 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f91780a3db0 0x7f91780a6270 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9180000c00 tx=0x7f91800314a0 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:14.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.571+0000 7f918d7fa640 1 -- 172.21.15.97:0/1364190033 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f918809c510 con 0x7f9190101b60 2024-04-23T10:26:14.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.795+0000 7f9196a0b640 1 -- 172.21.15.97:0/1364190033 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9190062b50 con 0x7f9190101b60 2024-04-23T10:26:14.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.795+0000 7f918d7fa640 1 -- 172.21.15.97:0/1364190033 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7f91880a0810 con 0x7f9190101b60 2024-04-23T10:26:14.794 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-23T10:26:14.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.797+0000 7f9196a0b640 1 -- 172.21.15.97:0/1364190033 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f91780a3db0 msgr2=0x7f91780a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:14.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.797+0000 7f9196a0b640 1 --2- 172.21.15.97:0/1364190033 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f91780a3db0 0x7f91780a6270 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9180000c00 tx=0x7f91800314a0 comp rx=0 tx=0).stop 2024-04-23T10:26:14.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.797+0000 7f9196a0b640 1 -- 172.21.15.97:0/1364190033 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9190101b60 msgr2=0x7f91901a1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:14.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.797+0000 7f9196a0b640 1 --2- 172.21.15.97:0/1364190033 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9190101b60 0x7f91901a1990 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f9188009fd0 tx=0x7f918800b260 comp rx=0 tx=0).stop 2024-04-23T10:26:14.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.798+0000 7f9196a0b640 1 -- 172.21.15.97:0/1364190033 shutdown_connections 2024-04-23T10:26:14.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.798+0000 7f9196a0b640 1 --2- 172.21.15.97:0/1364190033 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f91780a3db0 0x7f91780a6270 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:14.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.798+0000 7f9196a0b640 1 --2- 172.21.15.97:0/1364190033 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9190101b60 0x7f91901a1990 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:14.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.798+0000 7f9196a0b640 1 --2- 172.21.15.97:0/1364190033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f91901011c0 0x7f919019a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:14.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.798+0000 7f9196a0b640 1 --2- 172.21.15.97:0/1364190033 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9190100860 0x7f919019a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:14.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.798+0000 7f9196a0b640 1 -- 172.21.15.97:0/1364190033 >> 172.21.15.97:0/1364190033 conn(0x7f91900fc3e0 msgr2=0x7f91900fdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:14.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.798+0000 7f9196a0b640 1 -- 172.21.15.97:0/1364190033 shutdown_connections 2024-04-23T10:26:14.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.798+0000 7f9196a0b640 1 -- 172.21.15.97:0/1364190033 wait complete. 2024-04-23T10:26:14.808 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T10:26:14.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.915+0000 7f9b7c330640 1 -- 172.21.15.97:0/2521344035 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9b74101fe0 msgr2=0x7f9b74102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:14.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.915+0000 7f9b7c330640 1 --2- 172.21.15.97:0/2521344035 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9b74101fe0 0x7f9b74102460 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9b70009900 tx=0x7f9b7002a410 comp rx=0 tx=0).stop 2024-04-23T10:26:14.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.915+0000 7f9b7c330640 1 -- 172.21.15.97:0/2521344035 shutdown_connections 2024-04-23T10:26:14.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.915+0000 7f9b7c330640 1 --2- 172.21.15.97:0/2521344035 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b741029a0 0x7f9b741092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:14.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.915+0000 7f9b7c330640 1 --2- 172.21.15.97:0/2521344035 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9b74101fe0 0x7f9b74102460 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:14.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.915+0000 7f9b7c330640 1 --2- 172.21.15.97:0/2521344035 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b74100e90 0x7f9b741012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:14.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.915+0000 7f9b7c330640 1 -- 172.21.15.97:0/2521344035 >> 172.21.15.97:0/2521344035 conn(0x7f9b740fc400 msgr2=0x7f9b740fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:14.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.915+0000 7f9b7c330640 1 -- 172.21.15.97:0/2521344035 shutdown_connections 2024-04-23T10:26:14.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.915+0000 7f9b7c330640 1 -- 172.21.15.97:0/2521344035 wait complete. 2024-04-23T10:26:14.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.915+0000 7f9b7c330640 1 Processor -- start 2024-04-23T10:26:14.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.915+0000 7f9b7c330640 1 -- start start 2024-04-23T10:26:14.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.915+0000 7f9b7c330640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b74100e90 0x7f9b74069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:14.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.915+0000 7f9b7c330640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b74101fe0 0x7f9b7419f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:14.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.915+0000 7f9b7c330640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9b741029a0 0x7f9b741a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:14.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.915+0000 7f9b7c330640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9b7410c9f0 con 0x7f9b74100e90 2024-04-23T10:26:14.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.915+0000 7f9b7c330640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9b7410c870 con 0x7f9b74101fe0 2024-04-23T10:26:14.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.915+0000 7f9b7c330640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9b7410cb70 con 0x7f9b741029a0 2024-04-23T10:26:14.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.916+0000 7f9b7a0cc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b74100e90 0x7f9b74069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:14.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.916+0000 7f9b7a0cc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b74100e90 0x7f9b74069c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58438/0 (socket says 172.21.15.97:58438) 2024-04-23T10:26:14.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.916+0000 7f9b7a0cc640 1 -- 172.21.15.97:0/1361204800 learned_addr learned my addr 172.21.15.97:0/1361204800 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:14.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.916+0000 7f9b7a8cd640 1 --2- 172.21.15.97:0/1361204800 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9b741029a0 0x7f9b741a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:14.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.916+0000 7f9b798cb640 1 --2- 172.21.15.97:0/1361204800 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b74101fe0 0x7f9b7419f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:14.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.916+0000 7f9b7a8cd640 1 -- 172.21.15.97:0/1361204800 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b74101fe0 msgr2=0x7f9b7419f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:14.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.916+0000 7f9b7a8cd640 1 --2- 172.21.15.97:0/1361204800 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b74101fe0 0x7f9b7419f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:14.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.916+0000 7f9b7a8cd640 1 -- 172.21.15.97:0/1361204800 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b74100e90 msgr2=0x7f9b74069c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:14.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.916+0000 7f9b7a8cd640 1 --2- 172.21.15.97:0/1361204800 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b74100e90 0x7f9b74069c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:14.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.916+0000 7f9b7a8cd640 1 -- 172.21.15.97:0/1361204800 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b741a20f0 con 0x7f9b741029a0 2024-04-23T10:26:14.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.916+0000 7f9b7a0cc640 1 --2- 172.21.15.97:0/1361204800 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b74100e90 0x7f9b74069c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T10:26:14.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.916+0000 7f9b7a8cd640 1 --2- 172.21.15.97:0/1361204800 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9b741029a0 0x7f9b741a1900 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c00ec50 tx=0x7f9b6c00c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:14.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.916+0000 7f9b6b7fe640 1 -- 172.21.15.97:0/1361204800 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b6c003b70 con 0x7f9b741029a0 2024-04-23T10:26:14.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.916+0000 7f9b7c330640 1 -- 172.21.15.97:0/1361204800 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b741a2380 con 0x7f9b741029a0 2024-04-23T10:26:14.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.916+0000 7f9b6b7fe640 1 -- 172.21.15.97:0/1361204800 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b6c003d10 con 0x7f9b741029a0 2024-04-23T10:26:14.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.916+0000 7f9b6b7fe640 1 -- 172.21.15.97:0/1361204800 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b6c010830 con 0x7f9b741029a0 2024-04-23T10:26:14.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.916+0000 7f9b7c330640 1 -- 172.21.15.97:0/1361204800 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b741a2850 con 0x7f9b741029a0 2024-04-23T10:26:14.918 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.917+0000 7f9b7c330640 1 -- 172.21.15.97:0/1361204800 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b74108310 con 0x7f9b741029a0 2024-04-23T10:26:14.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.919+0000 7f9b6b7fe640 1 -- 172.21.15.97:0/1361204800 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7f9b6c004b40 con 0x7f9b741029a0 2024-04-23T10:26:14.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.920+0000 7f9b6b7fe640 1 --2- 172.21.15.97:0/1361204800 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f9b600a3de0 0x7f9b600a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:14.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.920+0000 7f9b7a0cc640 1 --2- 172.21.15.97:0/1361204800 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f9b600a3de0 0x7f9b600a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:14.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.920+0000 7f9b6b7fe640 1 -- 172.21.15.97:0/1361204800 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f9b6c0d1000 con 0x7f9b741029a0 2024-04-23T10:26:14.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.920+0000 7f9b7a0cc640 1 --2- 172.21.15.97:0/1361204800 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f9b600a3de0 0x7f9b600a62a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9b64011500 tx=0x7f9b64005f90 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:14.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:14.923+0000 7f9b6b7fe640 1 -- 172.21.15.97:0/1361204800 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9b6c09d580 con 0x7f9b741029a0 2024-04-23T10:26:15.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:14 smithi097 ceph-mon[29197]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:26:15.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:14 smithi097 ceph-mon[29197]: from='client.15468 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:26:15.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:14 smithi097 ceph-mon[29197]: from='client.15480 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:26:15.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:14 smithi097 ceph-mon[29197]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-23T10:26:15.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:14 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/2234761069' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T10:26:15.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:14 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/1882928253' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T10:26:15.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:14 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:26:15.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:14 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:26:15.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:14 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:26:15.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:14 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:26:15.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:14 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:26:15.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:14 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:26:15.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:14 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:26:15.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:14 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:26:15.112 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:14 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:26:15.112 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:14 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:26:15.112 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:14 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:26:15.112 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:14 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:26:15.112 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:14 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:26:15.112 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:14 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:26:15.112 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:14 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:26:15.112 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:14 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:26:15.112 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:14 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:26:15.112 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:14 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:26:15.112 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:14 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:26:15.112 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:14 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:26:15.112 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:14 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/1364190033' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T10:26:15.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.144+0000 7f9b7c330640 1 -- 172.21.15.97:0/1361204800 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9b7402d810 con 0x7f9b741029a0 2024-04-23T10:26:15.143 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.144+0000 7f9b6b7fe640 1 -- 172.21.15.97:0/1361204800 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7f9b6c09ce80 con 0x7f9b741029a0 2024-04-23T10:26:15.143 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-04-23T10:26:15.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.147+0000 7f9b7c330640 1 -- 172.21.15.97:0/1361204800 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f9b600a3de0 msgr2=0x7f9b600a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:15.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.147+0000 7f9b7c330640 1 --2- 172.21.15.97:0/1361204800 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f9b600a3de0 0x7f9b600a62a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9b64011500 tx=0x7f9b64005f90 comp rx=0 tx=0).stop 2024-04-23T10:26:15.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.147+0000 7f9b7c330640 1 -- 172.21.15.97:0/1361204800 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9b741029a0 msgr2=0x7f9b741a1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:15.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.147+0000 7f9b7c330640 1 --2- 172.21.15.97:0/1361204800 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9b741029a0 0x7f9b741a1900 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c00ec50 tx=0x7f9b6c00c6a0 comp rx=0 tx=0).stop 2024-04-23T10:26:15.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.148+0000 7f9b7c330640 1 -- 172.21.15.97:0/1361204800 shutdown_connections 2024-04-23T10:26:15.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.148+0000 7f9b7c330640 1 --2- 172.21.15.97:0/1361204800 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f9b600a3de0 0x7f9b600a62a0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:15.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.148+0000 7f9b7c330640 1 --2- 172.21.15.97:0/1361204800 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9b741029a0 0x7f9b741a1900 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:15.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.148+0000 7f9b7c330640 1 --2- 172.21.15.97:0/1361204800 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b74101fe0 0x7f9b7419f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:15.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.148+0000 7f9b7c330640 1 --2- 172.21.15.97:0/1361204800 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9b74100e90 0x7f9b74069c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:15.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.148+0000 7f9b7c330640 1 -- 172.21.15.97:0/1361204800 >> 172.21.15.97:0/1361204800 conn(0x7f9b740fc400 msgr2=0x7f9b741054e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:15.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.148+0000 7f9b7c330640 1 -- 172.21.15.97:0/1361204800 shutdown_connections 2024-04-23T10:26:15.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.148+0000 7f9b7c330640 1 -- 172.21.15.97:0/1361204800 wait complete. 2024-04-23T10:26:15.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:14 smithi183 ceph-mon[32317]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:26:15.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:14 smithi183 ceph-mon[32317]: from='client.15468 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:26:15.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:14 smithi183 ceph-mon[32317]: from='client.15480 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:26:15.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:14 smithi183 ceph-mon[32317]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-23T10:26:15.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:14 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/2234761069' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T10:26:15.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:14 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/1882928253' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T10:26:15.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:14 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:26:15.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:14 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:26:15.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:14 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:26:15.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:14 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:26:15.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:14 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:26:15.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:14 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:26:15.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:14 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:26:15.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:14 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:26:15.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:14 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:26:15.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:14 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:26:15.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:14 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:26:15.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:14 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:26:15.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:14 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:26:15.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:14 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:26:15.151 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:14 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:26:15.152 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:14 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:26:15.152 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:14 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:26:15.152 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:14 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:26:15.152 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:14 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:26:15.152 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:14 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:26:15.152 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:14 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/1364190033' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T10:26:15.159 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T10:26:15.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.262+0000 7fd760c64640 1 -- 172.21.15.97:0/1317856104 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd75c102ad0 msgr2=0x7fd75c104ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:15.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.262+0000 7fd760c64640 1 --2- 172.21.15.97:0/1317856104 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd75c102ad0 0x7fd75c104ec0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fd74c009900 tx=0x7fd74c02a410 comp rx=0 tx=0).stop 2024-04-23T10:26:15.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.262+0000 7fd760c64640 1 -- 172.21.15.97:0/1317856104 shutdown_connections 2024-04-23T10:26:15.261 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.262+0000 7fd760c64640 1 --2- 172.21.15.97:0/1317856104 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd75c107d30 0x7fd75c10a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:15.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.262+0000 7fd760c64640 1 --2- 172.21.15.97:0/1317856104 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd75c105400 0x7fd75c1077f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:15.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.262+0000 7fd760c64640 1 --2- 172.21.15.97:0/1317856104 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd75c102ad0 0x7fd75c104ec0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:15.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.262+0000 7fd760c64640 1 -- 172.21.15.97:0/1317856104 >> 172.21.15.97:0/1317856104 conn(0x7fd75c0fc400 msgr2=0x7fd75c0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:15.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.262+0000 7fd760c64640 1 -- 172.21.15.97:0/1317856104 shutdown_connections 2024-04-23T10:26:15.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.262+0000 7fd760c64640 1 -- 172.21.15.97:0/1317856104 wait complete. 2024-04-23T10:26:15.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.263+0000 7fd760c64640 1 Processor -- start 2024-04-23T10:26:15.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.263+0000 7fd760c64640 1 -- start start 2024-04-23T10:26:15.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.263+0000 7fd760c64640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd75c102ad0 0x7fd75c198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:15.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.263+0000 7fd760c64640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd75c105400 0x7fd75c1987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:15.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.263+0000 7fd760c64640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd75c107d30 0x7fd75c19f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:15.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.263+0000 7fd760c64640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd75c10dae0 con 0x7fd75c105400 2024-04-23T10:26:15.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.263+0000 7fd760c64640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd75c10d960 con 0x7fd75c107d30 2024-04-23T10:26:15.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.263+0000 7fd760c64640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd75c10dc60 con 0x7fd75c102ad0 2024-04-23T10:26:15.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.263+0000 7fd759d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd75c105400 0x7fd75c1987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:15.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.263+0000 7fd759d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd75c105400 0x7fd75c1987c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58466/0 (socket says 172.21.15.97:58466) 2024-04-23T10:26:15.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.263+0000 7fd75a59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd75c102ad0 0x7fd75c198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:15.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.263+0000 7fd759d9b640 1 -- 172.21.15.97:0/367237289 learned_addr learned my addr 172.21.15.97:0/367237289 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:15.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.263+0000 7fd75ad9d640 1 --2- 172.21.15.97:0/367237289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd75c107d30 0x7fd75c19f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:15.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.264+0000 7fd759d9b640 1 -- 172.21.15.97:0/367237289 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd75c102ad0 msgr2=0x7fd75c198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:15.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.264+0000 7fd759d9b640 1 --2- 172.21.15.97:0/367237289 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd75c102ad0 0x7fd75c198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:15.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.264+0000 7fd759d9b640 1 -- 172.21.15.97:0/367237289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd75c107d30 msgr2=0x7fd75c19f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:15.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.264+0000 7fd759d9b640 1 --2- 172.21.15.97:0/367237289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd75c107d30 0x7fd75c19f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:15.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.264+0000 7fd759d9b640 1 -- 172.21.15.97:0/367237289 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd75c19fe50 con 0x7fd75c105400 2024-04-23T10:26:15.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.264+0000 7fd75ad9d640 1 --2- 172.21.15.97:0/367237289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd75c107d30 0x7fd75c19f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T10:26:15.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.264+0000 7fd759d9b640 1 --2- 172.21.15.97:0/367237289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd75c105400 0x7fd75c1987c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd75000e9f0 tx=0x7fd75000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:15.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.264+0000 7fd74b7fe640 1 -- 172.21.15.97:0/367237289 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd75000ccc0 con 0x7fd75c105400 2024-04-23T10:26:15.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.264+0000 7fd760c64640 1 -- 172.21.15.97:0/367237289 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd75c1a0140 con 0x7fd75c105400 2024-04-23T10:26:15.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.264+0000 7fd74b7fe640 1 -- 172.21.15.97:0/367237289 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7500054d0 con 0x7fd75c105400 2024-04-23T10:26:15.265 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.264+0000 7fd74b7fe640 1 -- 172.21.15.97:0/367237289 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd750010430 con 0x7fd75c105400 2024-04-23T10:26:15.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.264+0000 7fd760c64640 1 -- 172.21.15.97:0/367237289 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd75c1a0640 con 0x7fd75c105400 2024-04-23T10:26:15.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.265+0000 7fd760c64640 1 -- 172.21.15.97:0/367237289 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd75c0fdef0 con 0x7fd75c105400 2024-04-23T10:26:15.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.266+0000 7fd74b7fe640 1 -- 172.21.15.97:0/367237289 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7fd7500046c0 con 0x7fd75c105400 2024-04-23T10:26:15.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.267+0000 7fd74b7fe640 1 --2- 172.21.15.97:0/367237289 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fd7440a3db0 0x7fd7440a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:15.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.267+0000 7fd74b7fe640 1 -- 172.21.15.97:0/367237289 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fd7500d0ec0 con 0x7fd75c105400 2024-04-23T10:26:15.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.267+0000 7fd75a59c640 1 --2- 172.21.15.97:0/367237289 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fd7440a3db0 0x7fd7440a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:15.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.267+0000 7fd75a59c640 1 --2- 172.21.15.97:0/367237289 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fd7440a3db0 0x7fd7440a6270 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd74c000c00 tx=0x7fd74c0314a0 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:15.269 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.270+0000 7fd74b7fe640 1 -- 172.21.15.97:0/367237289 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd750053ff0 con 0x7fd75c105400 2024-04-23T10:26:15.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:14 smithi196 ceph-mon[32919]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:26:15.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:14 smithi196 ceph-mon[32919]: from='client.15468 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:26:15.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:14 smithi196 ceph-mon[32919]: from='client.15480 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:26:15.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:14 smithi196 ceph-mon[32919]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-23T10:26:15.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:14 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/2234761069' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T10:26:15.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:14 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/1882928253' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T10:26:15.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:14 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:26:15.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:14 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:26:15.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:14 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:26:15.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:14 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:26:15.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:14 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:26:15.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:14 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:26:15.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:14 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:26:15.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:14 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:26:15.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:14 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:26:15.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:14 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:26:15.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:14 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:26:15.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:14 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:26:15.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:14 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:26:15.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:14 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:26:15.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:14 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:26:15.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:14 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:26:15.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:14 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:26:15.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:14 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:26:15.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:14 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:26:15.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:14 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:26:15.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:14 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/1364190033' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T10:26:15.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.461+0000 7fd760c64640 1 -- 172.21.15.97:0/367237289 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd75c02d810 con 0x7fd75c105400 2024-04-23T10:26:15.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.462+0000 7fd74b7fe640 1 -- 172.21.15.97:0/367237289 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22981 (secure 0 0 0) 0x7fd75009d440 con 0x7fd75c105400 2024-04-23T10:26:15.462 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:26:15.462 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":89,"fsid":"9b22af88-0159-11ef-bc93-c7b262605968","created":"2024-04-23T10:16:05.983305+0000","modified":"2024-04-23T10:26:11.109574+0000","last_up_change":"2024-04-23T10:23:13.114761+0000","last_in_change":"2024-04-23T10:22:52.804880+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T10:19:25.559138+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T10:24:04.570049+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T10:24:05.861631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"58cb995d-de35-4522-ac81-4982401b260d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6803","nonce":3031221802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6805","nonce":3031221802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6809","nonce":3031221802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6807","nonce":3031221802}]},"public_addr":"172.21.15.97:6803/3031221802","cluster_addr":"172.21.15.97:6805/3031221802","heartbeat_back_addr":"172.21.15.97:6809/3031221802","heartbeat_front_addr":"172.21.15.97:6807/3031221802","state":["exists","up"]},{"osd":1,"uuid":"b47bd8ee-1f21-4834-acf5-bb40f97ab3ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6801","nonce":1108870440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6803","nonce":1108870440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6807","nonce":1108870440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6805","nonce":1108870440}]},"public_addr":"172.21.15.183:6801/1108870440","cluster_addr":"172.21.15.183:6803/1108870440","heartbeat_back_addr":"172.21.15.183:6807/1108870440","heartbeat_front_addr":"172.21.15.183:6805/1108870440","state":["exists","up"]},{"osd":2,"uuid":"17c4eec3-6c47-4334-9b63-1d5ad9ec5988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6801","nonce":2877039731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6803","nonce":2877039731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6807","nonce":2877039731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6805","nonce":2877039731}]},"public_addr":"172.21.15.196:6801/2877039731","cluster_addr":"172.21.15.196:6803/2877039731","heartbeat_back_addr":"172.21.15.196:6807/2877039731","heartbeat_front_addr":"172.21.15.196:6805/2877039731","state":["exists","up"]},{"osd":3,"uuid":"3f731c38-2e45-4ca1-8dce-efc4a1b2166b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6811","nonce":4276311194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6813","nonce":4276311194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6817","nonce":4276311194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6815","nonce":4276311194}]},"public_addr":"172.21.15.97:6811/4276311194","cluster_addr":"172.21.15.97:6813/4276311194","heartbeat_back_addr":"172.21.15.97:6817/4276311194","heartbeat_front_addr":"172.21.15.97:6815/4276311194","state":["exists","up"]},{"osd":4,"uuid":"fdcafb14-9b82-4134-bd5b-79c0516f6cbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6809","nonce":63913916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6811","nonce":63913916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6815","nonce":63913916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6813","nonce":63913916}]},"public_addr":"172.21.15.183:6809/63913916","cluster_addr":"172.21.15.183:6811/63913916","heartbeat_back_addr":"172.21.15.183:6815/63913916","heartbeat_front_addr":"172.21.15.183:6813/63913916","state":["exists","up"]},{"osd":5,"uuid":"cdcc18d8-da3c-4012-bf12-876bb39e66eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6809","nonce":2804446896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6811","nonce":2804446896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6815","nonce":2804446896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6813","nonce":2804446896}]},"public_addr":"172.21.15.196:6809/2804446896","cluster_addr":"172.21.15.196:6811/2804446896","heartbeat_back_addr":"172.21.15.196:6815/2804446896","heartbeat_front_addr":"172.21.15.196:6813/2804446896","state":["exists","up"]},{"osd":6,"uuid":"071abb5c-2edf-4832-b8aa-3b80d20096fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6819","nonce":2080916705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6821","nonce":2080916705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6825","nonce":2080916705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6823","nonce":2080916705}]},"public_addr":"172.21.15.97:6819/2080916705","cluster_addr":"172.21.15.97:6821/2080916705","heartbeat_back_addr":"172.21.15.97:6825/2080916705","heartbeat_front_addr":"172.21.15.97:6823/2080916705","state":["exists","up"]},{"osd":7,"uuid":"9d54760b-66df-486b-8abb-017f3b64b4d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6817","nonce":1865873033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6819","nonce":1865873033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6823","nonce":1865873033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6821","nonce":1865873033}]},"public_addr":"172.21.15.183:6817/1865873033","cluster_addr":"172.21.15.183:6819/1865873033","heartbeat_back_addr":"172.21.15.183:6823/1865873033","heartbeat_front_addr":"172.21.15.183:6821/1865873033","state":["exists","up"]},{"osd":8,"uuid":"5cd3100a-bd6a-4be5-b5e9-0d0afc325193","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6817","nonce":4063932139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6819","nonce":4063932139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6823","nonce":4063932139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6821","nonce":4063932139}]},"public_addr":"172.21.15.196:6817/4063932139","cluster_addr":"172.21.15.196:6819/4063932139","heartbeat_back_addr":"172.21.15.196:6823/4063932139","heartbeat_front_addr":"172.21.15.196:6821/4063932139","state":["exists","up"]},{"osd":9,"uuid":"b6a5f4a1-4ecc-4f73-a2df-07551d9298ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6827","nonce":1887432474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6829","nonce":1887432474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6833","nonce":1887432474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6831","nonce":1887432474}]},"public_addr":"172.21.15.97:6827/1887432474","cluster_addr":"172.21.15.97:6829/1887432474","heartbeat_back_addr":"172.21.15.97:6833/1887432474","heartbeat_front_addr":"172.21.15.97:6831/1887432474","state":["exists","up"]},{"osd":10,"uuid":"86f59816-1ea6-4ae9-8796-cf5fc4831064","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6825","nonce":799434647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6827","nonce":799434647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6831","nonce":799434647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6829","nonce":799434647}]},"public_addr":"172.21.15.183:6825/799434647","cluster_addr":"172.21.15.183:6827/799434647","heartbeat_back_addr":"172.21.15.183:6831/799434647","heartbeat_front_addr":"172.21.15.183:6829/799434647","state":["exists","up"]},{"osd":11,"uuid":"95d64a89-a9f8-4c7a-8c0d-da7fb26cfff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6825","nonce":1873969467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6827","nonce":1873969467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6831","nonce":1873969467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6829","nonce":1873969467}]},"public_addr":"172.21.15.196:6825/1873969467","cluster_addr":"172.21.15.196:6827/1873969467","heartbeat_back_addr":"172.21.15.196:6831/1873969467","heartbeat_front_addr":"172.21.15.196:6829/1873969467","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:18:33.721156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:18:57.586709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:19:22.069458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:19:46.451597+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:20:11.561019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:20:36.298672+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:21:01.661804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:21:27.324128+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:21:52.898197+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:22:19.614326+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:22:45.154917+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:23:11.222612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/233335470":"2024-04-24T10:26:11.109535+0000","172.21.15.97:6800/3149043749":"2024-04-24T10:26:11.109535+0000","172.21.15.97:0/338277142":"2024-04-24T10:26:11.109535+0000","172.21.15.97:0/3866016040":"2024-04-24T10:16:59.468237+0000","172.21.15.97:6801/3297166858":"2024-04-24T10:16:59.468237+0000","172.21.15.97:0/3804743548":"2024-04-24T10:16:59.468237+0000","172.21.15.97:6800/3706463017":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/3715079438":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/597405418":"2024-04-24T10:26:11.109535+0000","172.21.15.97:0/786806834":"2024-04-24T10:16:59.468237+0000","172.21.15.97:6801/3706463017":"2024-04-24T10:16:33.128917+0000","172.21.15.97:6801/3149043749":"2024-04-24T10:26:11.109535+0000","172.21.15.97:0/3294175554":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/1449136760":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/3198144747":"2024-04-24T10:26:11.109535+0000","172.21.15.97:6800/3297166858":"2024-04-24T10:16:59.468237+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T10:26:15.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.464+0000 7fd760c64640 1 -- 172.21.15.97:0/367237289 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fd7440a3db0 msgr2=0x7fd7440a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:15.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.464+0000 7fd760c64640 1 --2- 172.21.15.97:0/367237289 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fd7440a3db0 0x7fd7440a6270 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd74c000c00 tx=0x7fd74c0314a0 comp rx=0 tx=0).stop 2024-04-23T10:26:15.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.464+0000 7fd760c64640 1 -- 172.21.15.97:0/367237289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd75c105400 msgr2=0x7fd75c1987c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:15.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.464+0000 7fd760c64640 1 --2- 172.21.15.97:0/367237289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd75c105400 0x7fd75c1987c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd75000e9f0 tx=0x7fd75000eeb0 comp rx=0 tx=0).stop 2024-04-23T10:26:15.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.464+0000 7fd760c64640 1 -- 172.21.15.97:0/367237289 shutdown_connections 2024-04-23T10:26:15.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.464+0000 7fd760c64640 1 --2- 172.21.15.97:0/367237289 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fd7440a3db0 0x7fd7440a6270 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:15.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.464+0000 7fd760c64640 1 --2- 172.21.15.97:0/367237289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd75c107d30 0x7fd75c19f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:15.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.464+0000 7fd760c64640 1 --2- 172.21.15.97:0/367237289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd75c105400 0x7fd75c1987c0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:15.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.464+0000 7fd760c64640 1 --2- 172.21.15.97:0/367237289 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd75c102ad0 0x7fd75c198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:15.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.464+0000 7fd760c64640 1 -- 172.21.15.97:0/367237289 >> 172.21.15.97:0/367237289 conn(0x7fd75c0fc400 msgr2=0x7fd75c103510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:15.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.465+0000 7fd760c64640 1 -- 172.21.15.97:0/367237289 shutdown_connections 2024-04-23T10:26:15.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.465+0000 7fd760c64640 1 -- 172.21.15.97:0/367237289 wait complete. 2024-04-23T10:26:15.475 INFO:tasks.quiescer.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T10:19:25.559138+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T10:24:04.570049+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T10:24:05.861631+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T10:26:15.475 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T10:26:15.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.581+0000 7f27f5ee1640 1 -- 172.21.15.97:0/691024734 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27f006ac80 msgr2=0x7f27f010e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:15.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.581+0000 7f27f5ee1640 1 --2- 172.21.15.97:0/691024734 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27f006ac80 0x7f27f010e420 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f27e4009900 tx=0x7f27e402a410 comp rx=0 tx=0).stop 2024-04-23T10:26:15.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.581+0000 7f27f5ee1640 1 -- 172.21.15.97:0/691024734 shutdown_connections 2024-04-23T10:26:15.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.581+0000 7f27f5ee1640 1 --2- 172.21.15.97:0/691024734 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27f010e960 0x7f27f0110d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:15.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.581+0000 7f27f5ee1640 1 --2- 172.21.15.97:0/691024734 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27f006ac80 0x7f27f010e420 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:15.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.581+0000 7f27f5ee1640 1 --2- 172.21.15.97:0/691024734 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27f006a360 0x7f27f006a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:15.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.581+0000 7f27f5ee1640 1 -- 172.21.15.97:0/691024734 >> 172.21.15.97:0/691024734 conn(0x7f27f0077f80 msgr2=0x7f27f0078390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:15.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.581+0000 7f27f5ee1640 1 -- 172.21.15.97:0/691024734 shutdown_connections 2024-04-23T10:26:15.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.581+0000 7f27f5ee1640 1 -- 172.21.15.97:0/691024734 wait complete. 2024-04-23T10:26:15.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.581+0000 7f27f5ee1640 1 Processor -- start 2024-04-23T10:26:15.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.581+0000 7f27f5ee1640 1 -- start start 2024-04-23T10:26:15.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.582+0000 7f27f5ee1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27f006a360 0x7f27f01a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:15.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.582+0000 7f27f5ee1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27f006ac80 0x7f27f01a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:15.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.582+0000 7f27f5ee1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27f010e960 0x7f27f019a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:15.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.582+0000 7f27f5ee1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f27f0114420 con 0x7f27f006ac80 2024-04-23T10:26:15.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.582+0000 7f27f5ee1640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f27f01142a0 con 0x7f27f010e960 2024-04-23T10:26:15.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.582+0000 7f27f5ee1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f27f01145a0 con 0x7f27f006a360 2024-04-23T10:26:15.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.582+0000 7f27eeffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27f006ac80 0x7f27f01a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:15.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.582+0000 7f27eeffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27f006ac80 0x7f27f01a0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46888/0 (socket says 172.21.15.97:46888) 2024-04-23T10:26:15.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.582+0000 7f27eeffd640 1 -- 172.21.15.97:0/3697438452 learned_addr learned my addr 172.21.15.97:0/3697438452 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:15.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.582+0000 7f27ef7fe640 1 --2- 172.21.15.97:0/3697438452 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27f006a360 0x7f27f01a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:15.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.582+0000 7f27effff640 1 --2- 172.21.15.97:0/3697438452 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27f010e960 0x7f27f019a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:15.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.582+0000 7f27eeffd640 1 -- 172.21.15.97:0/3697438452 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27f006a360 msgr2=0x7f27f01a06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:15.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.582+0000 7f27eeffd640 1 --2- 172.21.15.97:0/3697438452 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27f006a360 0x7f27f01a06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:15.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.582+0000 7f27eeffd640 1 -- 172.21.15.97:0/3697438452 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27f010e960 msgr2=0x7f27f019a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:15.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.582+0000 7f27eeffd640 1 --2- 172.21.15.97:0/3697438452 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27f010e960 0x7f27f019a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:15.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.582+0000 7f27eeffd640 1 -- 172.21.15.97:0/3697438452 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27f019b010 con 0x7f27f006ac80 2024-04-23T10:26:15.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.582+0000 7f27ef7fe640 1 --2- 172.21.15.97:0/3697438452 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27f006a360 0x7f27f01a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T10:26:15.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.582+0000 7f27eeffd640 1 --2- 172.21.15.97:0/3697438452 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27f006ac80 0x7f27f01a0c10 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f27e4005ab0 tx=0x7f27e4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:15.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.582+0000 7f27ecff9640 1 -- 172.21.15.97:0/3697438452 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e403f070 con 0x7f27f006ac80 2024-04-23T10:26:15.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.582+0000 7f27ecff9640 1 -- 172.21.15.97:0/3697438452 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27e4031b60 con 0x7f27f006ac80 2024-04-23T10:26:15.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.582+0000 7f27f5ee1640 1 -- 172.21.15.97:0/3697438452 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27f019b2a0 con 0x7f27f006ac80 2024-04-23T10:26:15.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.582+0000 7f27ecff9640 1 -- 172.21.15.97:0/3697438452 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e4035030 con 0x7f27f006ac80 2024-04-23T10:26:15.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.582+0000 7f27f5ee1640 1 -- 172.21.15.97:0/3697438452 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27f01a7520 con 0x7f27f006ac80 2024-04-23T10:26:15.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.584+0000 7f27f5ee1640 1 -- 172.21.15.97:0/3697438452 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27f0101460 con 0x7f27f006ac80 2024-04-23T10:26:15.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.584+0000 7f27ecff9640 1 -- 172.21.15.97:0/3697438452 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7f27e4042070 con 0x7f27f006ac80 2024-04-23T10:26:15.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.585+0000 7f27ecff9640 1 --2- 172.21.15.97:0/3697438452 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f27d80a3db0 0x7f27d80a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:15.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.585+0000 7f27ef7fe640 1 --2- 172.21.15.97:0/3697438452 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f27d80a3db0 0x7f27d80a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:15.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.585+0000 7f27ecff9640 1 -- 172.21.15.97:0/3697438452 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f27e40f1060 con 0x7f27f006ac80 2024-04-23T10:26:15.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.586+0000 7f27ef7fe640 1 --2- 172.21.15.97:0/3697438452 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f27d80a3db0 0x7f27d80a6270 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f27e0009fc0 tx=0x7f27e0005f90 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:15.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.590+0000 7f27ecff9640 1 -- 172.21.15.97:0/3697438452 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f27e40bcf90 con 0x7f27f006ac80 2024-04-23T10:26:15.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.836+0000 7f27f5ee1640 1 -- 172.21.15.97:0/3697438452 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f27f0109af0 con 0x7f27f006ac80 2024-04-23T10:26:15.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.836+0000 7f27ecff9640 1 -- 172.21.15.97:0/3697438452 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7f27e40c1290 con 0x7f27f006ac80 2024-04-23T10:26:15.835 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-23T10:26:15.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.838+0000 7f27f5ee1640 1 -- 172.21.15.97:0/3697438452 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f27d80a3db0 msgr2=0x7f27d80a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:15.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.838+0000 7f27f5ee1640 1 --2- 172.21.15.97:0/3697438452 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f27d80a3db0 0x7f27d80a6270 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f27e0009fc0 tx=0x7f27e0005f90 comp rx=0 tx=0).stop 2024-04-23T10:26:15.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.838+0000 7f27f5ee1640 1 -- 172.21.15.97:0/3697438452 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27f006ac80 msgr2=0x7f27f01a0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:15.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.838+0000 7f27f5ee1640 1 --2- 172.21.15.97:0/3697438452 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27f006ac80 0x7f27f01a0c10 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f27e4005ab0 tx=0x7f27e4003c10 comp rx=0 tx=0).stop 2024-04-23T10:26:15.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.838+0000 7f27f5ee1640 1 -- 172.21.15.97:0/3697438452 shutdown_connections 2024-04-23T10:26:15.837 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.838+0000 7f27f5ee1640 1 --2- 172.21.15.97:0/3697438452 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f27d80a3db0 0x7f27d80a6270 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:15.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.838+0000 7f27f5ee1640 1 --2- 172.21.15.97:0/3697438452 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27f010e960 0x7f27f019a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:15.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.838+0000 7f27f5ee1640 1 --2- 172.21.15.97:0/3697438452 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27f006ac80 0x7f27f01a0c10 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:15.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.838+0000 7f27f5ee1640 1 --2- 172.21.15.97:0/3697438452 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27f006a360 0x7f27f01a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:15.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.838+0000 7f27f5ee1640 1 -- 172.21.15.97:0/3697438452 >> 172.21.15.97:0/3697438452 conn(0x7f27f0077f80 msgr2=0x7f27f010ff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:15.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.838+0000 7f27f5ee1640 1 -- 172.21.15.97:0/3697438452 shutdown_connections 2024-04-23T10:26:15.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.838+0000 7f27f5ee1640 1 -- 172.21.15.97:0/3697438452 wait complete. 2024-04-23T10:26:15.848 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T10:26:15.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.959+0000 7fdb2802a640 1 -- 172.21.15.97:0/3237523409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb2010e7c0 msgr2=0x7fdb20110bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:15.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.959+0000 7fdb2802a640 1 --2- 172.21.15.97:0/3237523409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb2010e7c0 0x7fdb20110bb0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fdb18009ca0 tx=0x7fdb1802a950 comp rx=0 tx=0).stop 2024-04-23T10:26:15.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.959+0000 7fdb2802a640 1 -- 172.21.15.97:0/3237523409 shutdown_connections 2024-04-23T10:26:15.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.959+0000 7fdb2802a640 1 --2- 172.21.15.97:0/3237523409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb2010e7c0 0x7fdb20110bb0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:15.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.959+0000 7fdb2802a640 1 --2- 172.21.15.97:0/3237523409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb20074c20 0x7fdb2010e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:15.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.959+0000 7fdb2802a640 1 --2- 172.21.15.97:0/3237523409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb20074300 0x7fdb200746e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:15.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.959+0000 7fdb2802a640 1 -- 172.21.15.97:0/3237523409 >> 172.21.15.97:0/3237523409 conn(0x7fdb200fda10 msgr2=0x7fdb200ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:15.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.959+0000 7fdb2802a640 1 -- 172.21.15.97:0/3237523409 shutdown_connections 2024-04-23T10:26:15.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.959+0000 7fdb2802a640 1 -- 172.21.15.97:0/3237523409 wait complete. 2024-04-23T10:26:15.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.960+0000 7fdb2802a640 1 Processor -- start 2024-04-23T10:26:15.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.960+0000 7fdb2802a640 1 -- start start 2024-04-23T10:26:15.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.960+0000 7fdb2802a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb20074300 0x7fdb201a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:15.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.960+0000 7fdb2802a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb20074c20 0x7fdb201a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:15.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.960+0000 7fdb2802a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb2010e7c0 0x7fdb2019a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:15.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.960+0000 7fdb2802a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdb20114370 con 0x7fdb20074300 2024-04-23T10:26:15.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.960+0000 7fdb2802a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdb201141f0 con 0x7fdb2010e7c0 2024-04-23T10:26:15.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.960+0000 7fdb2802a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdb201144f0 con 0x7fdb20074c20 2024-04-23T10:26:15.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.960+0000 7fdb25dc6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb20074300 0x7fdb201a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:15.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.960+0000 7fdb25dc6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb20074300 0x7fdb201a0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46906/0 (socket says 172.21.15.97:46906) 2024-04-23T10:26:15.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.960+0000 7fdb25dc6640 1 -- 172.21.15.97:0/2705289938 learned_addr learned my addr 172.21.15.97:0/2705289938 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:15.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.960+0000 7fdb265c7640 1 --2- 172.21.15.97:0/2705289938 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb2010e7c0 0x7fdb2019a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:15.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.960+0000 7fdb255c5640 1 --2- 172.21.15.97:0/2705289938 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb20074c20 0x7fdb201a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:15.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.961+0000 7fdb25dc6640 1 -- 172.21.15.97:0/2705289938 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb20074c20 msgr2=0x7fdb201a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:15.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.961+0000 7fdb25dc6640 1 --2- 172.21.15.97:0/2705289938 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb20074c20 0x7fdb201a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:15.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.961+0000 7fdb25dc6640 1 -- 172.21.15.97:0/2705289938 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb2010e7c0 msgr2=0x7fdb2019a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:15.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.961+0000 7fdb25dc6640 1 --2- 172.21.15.97:0/2705289938 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb2010e7c0 0x7fdb2019a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:15.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.961+0000 7fdb25dc6640 1 -- 172.21.15.97:0/2705289938 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb2019afb0 con 0x7fdb20074300 2024-04-23T10:26:15.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.961+0000 7fdb25dc6640 1 --2- 172.21.15.97:0/2705289938 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb20074300 0x7fdb201a0630 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fdb1000c970 tx=0x7fdb1000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:15.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.961+0000 7fdb16ffd640 1 -- 172.21.15.97:0/2705289938 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb10007bb0 con 0x7fdb20074300 2024-04-23T10:26:15.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.961+0000 7fdb16ffd640 1 -- 172.21.15.97:0/2705289938 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb10007d50 con 0x7fdb20074300 2024-04-23T10:26:15.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.961+0000 7fdb2802a640 1 -- 172.21.15.97:0/2705289938 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb2019b2a0 con 0x7fdb20074300 2024-04-23T10:26:15.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.961+0000 7fdb16ffd640 1 -- 172.21.15.97:0/2705289938 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1000f450 con 0x7fdb20074300 2024-04-23T10:26:15.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.961+0000 7fdb2802a640 1 -- 172.21.15.97:0/2705289938 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb201a75b0 con 0x7fdb20074300 2024-04-23T10:26:15.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.962+0000 7fdb2802a640 1 -- 172.21.15.97:0/2705289938 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb200759b0 con 0x7fdb20074300 2024-04-23T10:26:15.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.963+0000 7fdb16ffd640 1 -- 172.21.15.97:0/2705289938 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fdb10004f20 con 0x7fdb20074300 2024-04-23T10:26:15.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.964+0000 7fdb16ffd640 1 --2- 172.21.15.97:0/2705289938 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fdb0c0a3e40 0x7fdb0c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:15.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.964+0000 7fdb255c5640 1 --2- 172.21.15.97:0/2705289938 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fdb0c0a3e40 0x7fdb0c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:15.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.964+0000 7fdb16ffd640 1 -- 172.21.15.97:0/2705289938 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fdb100d0710 con 0x7fdb20074300 2024-04-23T10:26:15.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.964+0000 7fdb255c5640 1 --2- 172.21.15.97:0/2705289938 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fdb0c0a3e40 0x7fdb0c0a6300 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdb2019bf40 tx=0x7fdb1c00b410 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:15.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:15.968+0000 7fdb16ffd640 1 -- 172.21.15.97:0/2705289938 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdb10014030 con 0x7fdb20074300 2024-04-23T10:26:16.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:15 smithi097 ceph-mon[29197]: Adjusting osd_memory_target on smithi183 to 176.2M 2024-04-23T10:26:16.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:15 smithi097 ceph-mon[29197]: Unable to set osd_memory_target on smithi183 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-23T10:26:16.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:15 smithi097 ceph-mon[29197]: Adjusting osd_memory_target on smithi196 to 176.2M 2024-04-23T10:26:16.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:15 smithi097 ceph-mon[29197]: Unable to set osd_memory_target on smithi196 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-04-23T10:26:16.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:15 smithi097 ceph-mon[29197]: Adjusting osd_memory_target on smithi097 to 176.2M 2024-04-23T10:26:16.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:15 smithi097 ceph-mon[29197]: Unable to set osd_memory_target on smithi097 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-04-23T10:26:16.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:15 smithi097 ceph-mon[29197]: Updating smithi097:/etc/ceph/ceph.conf 2024-04-23T10:26:16.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:15 smithi097 ceph-mon[29197]: Updating smithi183:/etc/ceph/ceph.conf 2024-04-23T10:26:16.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:15 smithi097 ceph-mon[29197]: Updating smithi196:/etc/ceph/ceph.conf 2024-04-23T10:26:16.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:15 smithi097 ceph-mon[29197]: Updating smithi183:/var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T10:26:16.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:15 smithi097 ceph-mon[29197]: Updating smithi097:/var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T10:26:16.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:15 smithi097 ceph-mon[29197]: Updating smithi196:/var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T10:26:16.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:15 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/1361204800' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T10:26:16.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:15 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/367237289' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T10:26:16.112 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:15 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:26:16.112 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:15 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:26:16.112 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:15 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:26:16.112 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:15 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:26:16.112 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:15 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:26:16.112 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:15 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:26:16.112 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:15 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:26:16.112 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:15 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/3697438452' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T10:26:16.112 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:15 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:26:16.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.149+0000 7fdb2802a640 1 -- 172.21.15.97:0/2705289938 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdb2019b4d0 con 0x7fdb20074300 2024-04-23T10:26:16.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.149+0000 7fdb16ffd640 1 -- 172.21.15.97:0/2705289938 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7fdb1009c560 con 0x7fdb20074300 2024-04-23T10:26:16.148 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-23T10:26:16.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.151+0000 7fdb2802a640 1 -- 172.21.15.97:0/2705289938 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fdb0c0a3e40 msgr2=0x7fdb0c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:16.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.151+0000 7fdb2802a640 1 --2- 172.21.15.97:0/2705289938 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fdb0c0a3e40 0x7fdb0c0a6300 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdb2019bf40 tx=0x7fdb1c00b410 comp rx=0 tx=0).stop 2024-04-23T10:26:16.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.151+0000 7fdb2802a640 1 -- 172.21.15.97:0/2705289938 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb20074300 msgr2=0x7fdb201a0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:16.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.151+0000 7fdb2802a640 1 --2- 172.21.15.97:0/2705289938 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb20074300 0x7fdb201a0630 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fdb1000c970 tx=0x7fdb1000ce30 comp rx=0 tx=0).stop 2024-04-23T10:26:16.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.151+0000 7fdb2802a640 1 -- 172.21.15.97:0/2705289938 shutdown_connections 2024-04-23T10:26:16.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.151+0000 7fdb2802a640 1 --2- 172.21.15.97:0/2705289938 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fdb0c0a3e40 0x7fdb0c0a6300 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:16.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.151+0000 7fdb2802a640 1 --2- 172.21.15.97:0/2705289938 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb2010e7c0 0x7fdb2019a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:16.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.151+0000 7fdb2802a640 1 --2- 172.21.15.97:0/2705289938 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb20074c20 0x7fdb201a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:16.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.151+0000 7fdb2802a640 1 --2- 172.21.15.97:0/2705289938 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb20074300 0x7fdb201a0630 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:16.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.151+0000 7fdb2802a640 1 -- 172.21.15.97:0/2705289938 >> 172.21.15.97:0/2705289938 conn(0x7fdb200fda10 msgr2=0x7fdb200ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:16.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.152+0000 7fdb2802a640 1 -- 172.21.15.97:0/2705289938 shutdown_connections 2024-04-23T10:26:16.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.152+0000 7fdb2802a640 1 -- 172.21.15.97:0/2705289938 wait complete. 2024-04-23T10:26:16.161 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T10:26:16.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.264+0000 7fe7ffb30640 1 -- 172.21.15.97:0/2778968342 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7f8074220 msgr2=0x7fe7f8074600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:16.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.264+0000 7fe7ffb30640 1 --2- 172.21.15.97:0/2778968342 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7f8074220 0x7fe7f8074600 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4009900 tx=0x7fe7f402a480 comp rx=0 tx=0).stop 2024-04-23T10:26:16.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.264+0000 7fe7ffb30640 1 -- 172.21.15.97:0/2778968342 shutdown_connections 2024-04-23T10:26:16.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.264+0000 7fe7ffb30640 1 --2- 172.21.15.97:0/2778968342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7f810e830 0x7fe7f8110c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:16.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.264+0000 7fe7ffb30640 1 --2- 172.21.15.97:0/2778968342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7f8074b40 0x7fe7f810e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:16.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.264+0000 7fe7ffb30640 1 --2- 172.21.15.97:0/2778968342 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7f8074220 0x7fe7f8074600 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:16.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.264+0000 7fe7ffb30640 1 -- 172.21.15.97:0/2778968342 >> 172.21.15.97:0/2778968342 conn(0x7fe7f80793a0 msgr2=0x7fe7f80797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:16.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.264+0000 7fe7ffb30640 1 -- 172.21.15.97:0/2778968342 shutdown_connections 2024-04-23T10:26:16.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.264+0000 7fe7ffb30640 1 -- 172.21.15.97:0/2778968342 wait complete. 2024-04-23T10:26:16.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.265+0000 7fe7ffb30640 1 Processor -- start 2024-04-23T10:26:16.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.265+0000 7fe7ffb30640 1 -- start start 2024-04-23T10:26:16.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.265+0000 7fe7ffb30640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7f8074220 0x7fe7f81a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:16.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.265+0000 7fe7ffb30640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7f8074b40 0x7fe7f81a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:16.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.265+0000 7fe7ffb30640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7f810e830 0x7fe7f819a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:16.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.265+0000 7fe7ffb30640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f81143f0 con 0x7fe7f810e830 2024-04-23T10:26:16.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.265+0000 7fe7ffb30640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f8114270 con 0x7fe7f8074220 2024-04-23T10:26:16.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.265+0000 7fe7ffb30640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f8114570 con 0x7fe7f8074b40 2024-04-23T10:26:16.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.265+0000 7fe7fe0cd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7f810e830 0x7fe7f819a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:16.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.265+0000 7fe7fe0cd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7f810e830 0x7fe7f819a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46924/0 (socket says 172.21.15.97:46924) 2024-04-23T10:26:16.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.265+0000 7fe7fe0cd640 1 -- 172.21.15.97:0/1917864722 learned_addr learned my addr 172.21.15.97:0/1917864722 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:16.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.265+0000 7fe7fd0cb640 1 --2- 172.21.15.97:0/1917864722 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7f8074b40 0x7fe7f81a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:16.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.265+0000 7fe7fd8cc640 1 --2- 172.21.15.97:0/1917864722 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7f8074220 0x7fe7f81a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:16.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.265+0000 7fe7fe0cd640 1 -- 172.21.15.97:0/1917864722 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7f8074b40 msgr2=0x7fe7f81a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:16.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.265+0000 7fe7fe0cd640 1 --2- 172.21.15.97:0/1917864722 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7f8074b40 0x7fe7f81a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:16.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.265+0000 7fe7fe0cd640 1 -- 172.21.15.97:0/1917864722 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7f8074220 msgr2=0x7fe7f81a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:16.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.265+0000 7fe7fe0cd640 1 --2- 172.21.15.97:0/1917864722 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7f8074220 0x7fe7f81a0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:16.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.265+0000 7fe7fe0cd640 1 -- 172.21.15.97:0/1917864722 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7f819af80 con 0x7fe7f810e830 2024-04-23T10:26:16.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.266+0000 7fe7fe0cd640 1 --2- 172.21.15.97:0/1917864722 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7f810e830 0x7fe7f819a7b0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fe7f000e9d0 tx=0x7fe7f000ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:16.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.266+0000 7fe7eeffd640 1 -- 172.21.15.97:0/1917864722 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f000cd80 con 0x7fe7f810e830 2024-04-23T10:26:16.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.266+0000 7fe7eeffd640 1 -- 172.21.15.97:0/1917864722 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7f0005020 con 0x7fe7f810e830 2024-04-23T10:26:16.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.266+0000 7fe7ffb30640 1 -- 172.21.15.97:0/1917864722 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7f819b270 con 0x7fe7f810e830 2024-04-23T10:26:16.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.266+0000 7fe7eeffd640 1 -- 172.21.15.97:0/1917864722 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f00116d0 con 0x7fe7f810e830 2024-04-23T10:26:16.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.266+0000 7fe7ffb30640 1 -- 172.21.15.97:0/1917864722 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7f81a7490 con 0x7fe7f810e830 2024-04-23T10:26:16.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.267+0000 7fe7ffb30640 1 -- 172.21.15.97:0/1917864722 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7f8075960 con 0x7fe7f810e830 2024-04-23T10:26:16.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.268+0000 7fe7eeffd640 1 -- 172.21.15.97:0/1917864722 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fe7f00051d0 con 0x7fe7f810e830 2024-04-23T10:26:16.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.268+0000 7fe7eeffd640 1 --2- 172.21.15.97:0/1917864722 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fe7e40a3e40 0x7fe7e40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:16.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.268+0000 7fe7fd8cc640 1 --2- 172.21.15.97:0/1917864722 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fe7e40a3e40 0x7fe7e40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:16.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.268+0000 7fe7eeffd640 1 -- 172.21.15.97:0/1917864722 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fe7f00d0c20 con 0x7fe7f810e830 2024-04-23T10:26:16.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.269+0000 7fe7fd8cc640 1 --2- 172.21.15.97:0/1917864722 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fe7e40a3e40 0x7fe7e40a6300 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4006010 tx=0x7fe7f4000bc0 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:16.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.273+0000 7fe7eeffd640 1 -- 172.21.15.97:0/1917864722 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe7f0016030 con 0x7fe7f810e830 2024-04-23T10:26:16.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:15 smithi196 ceph-mon[32919]: Adjusting osd_memory_target on smithi183 to 176.2M 2024-04-23T10:26:16.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:15 smithi196 ceph-mon[32919]: Unable to set osd_memory_target on smithi183 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-23T10:26:16.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:15 smithi196 ceph-mon[32919]: Adjusting osd_memory_target on smithi196 to 176.2M 2024-04-23T10:26:16.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:15 smithi196 ceph-mon[32919]: Unable to set osd_memory_target on smithi196 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-04-23T10:26:16.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:15 smithi196 ceph-mon[32919]: Adjusting osd_memory_target on smithi097 to 176.2M 2024-04-23T10:26:16.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:15 smithi196 ceph-mon[32919]: Unable to set osd_memory_target on smithi097 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-04-23T10:26:16.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:15 smithi196 ceph-mon[32919]: Updating smithi097:/etc/ceph/ceph.conf 2024-04-23T10:26:16.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:15 smithi196 ceph-mon[32919]: Updating smithi183:/etc/ceph/ceph.conf 2024-04-23T10:26:16.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:15 smithi196 ceph-mon[32919]: Updating smithi196:/etc/ceph/ceph.conf 2024-04-23T10:26:16.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:15 smithi196 ceph-mon[32919]: Updating smithi183:/var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T10:26:16.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:15 smithi196 ceph-mon[32919]: Updating smithi097:/var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T10:26:16.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:15 smithi196 ceph-mon[32919]: Updating smithi196:/var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T10:26:16.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:15 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/1361204800' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T10:26:16.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:15 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/367237289' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T10:26:16.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:15 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:26:16.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:15 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:26:16.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:15 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:26:16.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:15 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:26:16.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:15 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:26:16.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:15 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:26:16.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:15 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:26:16.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:15 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/3697438452' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T10:26:16.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:15 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:26:16.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:15 smithi183 ceph-mon[32317]: Adjusting osd_memory_target on smithi183 to 176.2M 2024-04-23T10:26:16.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:15 smithi183 ceph-mon[32317]: Unable to set osd_memory_target on smithi183 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-23T10:26:16.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:15 smithi183 ceph-mon[32317]: Adjusting osd_memory_target on smithi196 to 176.2M 2024-04-23T10:26:16.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:15 smithi183 ceph-mon[32317]: Unable to set osd_memory_target on smithi196 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-04-23T10:26:16.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:15 smithi183 ceph-mon[32317]: Adjusting osd_memory_target on smithi097 to 176.2M 2024-04-23T10:26:16.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:15 smithi183 ceph-mon[32317]: Unable to set osd_memory_target on smithi097 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-04-23T10:26:16.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:15 smithi183 ceph-mon[32317]: Updating smithi097:/etc/ceph/ceph.conf 2024-04-23T10:26:16.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:15 smithi183 ceph-mon[32317]: Updating smithi183:/etc/ceph/ceph.conf 2024-04-23T10:26:16.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:15 smithi183 ceph-mon[32317]: Updating smithi196:/etc/ceph/ceph.conf 2024-04-23T10:26:16.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:15 smithi183 ceph-mon[32317]: Updating smithi183:/var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T10:26:16.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:15 smithi183 ceph-mon[32317]: Updating smithi097:/var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T10:26:16.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:15 smithi183 ceph-mon[32317]: Updating smithi196:/var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T10:26:16.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:15 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/1361204800' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T10:26:16.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:15 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/367237289' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T10:26:16.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:15 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:26:16.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:15 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:26:16.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:15 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:26:16.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:15 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:26:16.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:15 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:26:16.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:15 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:26:16.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:15 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:26:16.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:15 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/3697438452' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T10:26:16.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:15 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:26:16.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.451+0000 7fe7ffb30640 1 -- 172.21.15.97:0/1917864722 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe7f8109420 con 0x7fe7f810e830 2024-04-23T10:26:16.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.451+0000 7fe7eeffd640 1 -- 172.21.15.97:0/1917864722 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7fe7f009d0c0 con 0x7fe7f810e830 2024-04-23T10:26:16.451 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-04-23T10:26:16.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.454+0000 7fe7ffb30640 1 -- 172.21.15.97:0/1917864722 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fe7e40a3e40 msgr2=0x7fe7e40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:16.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.454+0000 7fe7ffb30640 1 --2- 172.21.15.97:0/1917864722 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fe7e40a3e40 0x7fe7e40a6300 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4006010 tx=0x7fe7f4000bc0 comp rx=0 tx=0).stop 2024-04-23T10:26:16.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.454+0000 7fe7ffb30640 1 -- 172.21.15.97:0/1917864722 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7f810e830 msgr2=0x7fe7f819a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:16.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.454+0000 7fe7ffb30640 1 --2- 172.21.15.97:0/1917864722 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7f810e830 0x7fe7f819a7b0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fe7f000e9d0 tx=0x7fe7f000ee90 comp rx=0 tx=0).stop 2024-04-23T10:26:16.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.454+0000 7fe7ffb30640 1 -- 172.21.15.97:0/1917864722 shutdown_connections 2024-04-23T10:26:16.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.454+0000 7fe7ffb30640 1 --2- 172.21.15.97:0/1917864722 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fe7e40a3e40 0x7fe7e40a6300 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:16.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.454+0000 7fe7ffb30640 1 --2- 172.21.15.97:0/1917864722 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7f810e830 0x7fe7f819a7b0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:16.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.454+0000 7fe7ffb30640 1 --2- 172.21.15.97:0/1917864722 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7f8074b40 0x7fe7f81a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:16.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.454+0000 7fe7ffb30640 1 --2- 172.21.15.97:0/1917864722 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7f8074220 0x7fe7f81a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:16.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.454+0000 7fe7ffb30640 1 -- 172.21.15.97:0/1917864722 >> 172.21.15.97:0/1917864722 conn(0x7fe7f80793a0 msgr2=0x7fe7f80fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:16.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.454+0000 7fe7ffb30640 1 -- 172.21.15.97:0/1917864722 shutdown_connections 2024-04-23T10:26:16.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.454+0000 7fe7ffb30640 1 -- 172.21.15.97:0/1917864722 wait complete. 2024-04-23T10:26:16.464 INFO:tasks.quiescer.ceph_manager:waiting for clean 2024-04-23T10:26:16.464 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T10:26:16.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.563+0000 7ff4136a7640 1 -- 172.21.15.97:0/3040236506 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff40c10c580 msgr2=0x7ff40c10e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:16.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.563+0000 7ff4136a7640 1 --2- 172.21.15.97:0/3040236506 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff40c10c580 0x7ff40c10e9b0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7ff404009ca0 tx=0x7ff40402a950 comp rx=0 tx=0).stop 2024-04-23T10:26:16.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.563+0000 7ff4136a7640 1 -- 172.21.15.97:0/3040236506 shutdown_connections 2024-04-23T10:26:16.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.563+0000 7ff4136a7640 1 --2- 172.21.15.97:0/3040236506 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff40c10c580 0x7ff40c10e9b0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:16.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.563+0000 7ff4136a7640 1 --2- 172.21.15.97:0/3040236506 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff40c103ad0 0x7ff40c10bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:16.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.563+0000 7ff4136a7640 1 --2- 172.21.15.97:0/3040236506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff40c1031b0 0x7ff40c103590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:16.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.563+0000 7ff4136a7640 1 -- 172.21.15.97:0/3040236506 >> 172.21.15.97:0/3040236506 conn(0x7ff40c0771d0 msgr2=0x7ff40c0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:16.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.563+0000 7ff4136a7640 1 -- 172.21.15.97:0/3040236506 shutdown_connections 2024-04-23T10:26:16.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.563+0000 7ff4136a7640 1 -- 172.21.15.97:0/3040236506 wait complete. 2024-04-23T10:26:16.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.564+0000 7ff4136a7640 1 Processor -- start 2024-04-23T10:26:16.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.564+0000 7ff4136a7640 1 -- start start 2024-04-23T10:26:16.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.564+0000 7ff4136a7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff40c1031b0 0x7ff40c198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:16.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.564+0000 7ff4136a7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff40c103ad0 0x7ff40c198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:16.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.564+0000 7ff4136a7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff40c10c580 0x7ff40c19cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:16.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.564+0000 7ff4136a7640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff40c112180 con 0x7ff40c1031b0 2024-04-23T10:26:16.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.564+0000 7ff4136a7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff40c112000 con 0x7ff40c10c580 2024-04-23T10:26:16.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.564+0000 7ff4136a7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff40c112300 con 0x7ff40c103ad0 2024-04-23T10:26:16.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.564+0000 7ff411443640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff40c1031b0 0x7ff40c198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:16.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.564+0000 7ff411443640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff40c1031b0 0x7ff40c198760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46932/0 (socket says 172.21.15.97:46932) 2024-04-23T10:26:16.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.564+0000 7ff411443640 1 -- 172.21.15.97:0/711144014 learned_addr learned my addr 172.21.15.97:0/711144014 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:16.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.564+0000 7ff410c42640 1 --2- 172.21.15.97:0/711144014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff40c103ad0 0x7ff40c198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:16.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.564+0000 7ff411c44640 1 --2- 172.21.15.97:0/711144014 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff40c10c580 0x7ff40c19cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:16.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.565+0000 7ff411443640 1 -- 172.21.15.97:0/711144014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff40c103ad0 msgr2=0x7ff40c198ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:16.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.565+0000 7ff411443640 1 --2- 172.21.15.97:0/711144014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff40c103ad0 0x7ff40c198ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:16.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.565+0000 7ff411443640 1 -- 172.21.15.97:0/711144014 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff40c10c580 msgr2=0x7ff40c19cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:16.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.565+0000 7ff411443640 1 --2- 172.21.15.97:0/711144014 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff40c10c580 0x7ff40c19cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:16.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.565+0000 7ff411443640 1 -- 172.21.15.97:0/711144014 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff40c19d700 con 0x7ff40c1031b0 2024-04-23T10:26:16.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.565+0000 7ff410c42640 1 --2- 172.21.15.97:0/711144014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff40c103ad0 0x7ff40c198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:26:16.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.565+0000 7ff411443640 1 --2- 172.21.15.97:0/711144014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff40c1031b0 0x7ff40c198760 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff40800d950 tx=0x7ff40800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:16.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.565+0000 7ff4027fc640 1 -- 172.21.15.97:0/711144014 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff408017070 con 0x7ff40c1031b0 2024-04-23T10:26:16.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.565+0000 7ff4027fc640 1 -- 172.21.15.97:0/711144014 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff408009c30 con 0x7ff40c1031b0 2024-04-23T10:26:16.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.565+0000 7ff4136a7640 1 -- 172.21.15.97:0/711144014 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff40c19d9f0 con 0x7ff40c1031b0 2024-04-23T10:26:16.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.565+0000 7ff4027fc640 1 -- 172.21.15.97:0/711144014 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40800f620 con 0x7ff40c1031b0 2024-04-23T10:26:16.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.565+0000 7ff4136a7640 1 -- 172.21.15.97:0/711144014 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff40c1a5280 con 0x7ff40c1031b0 2024-04-23T10:26:16.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.566+0000 7ff4136a7640 1 -- 172.21.15.97:0/711144014 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff40c0691c0 con 0x7ff40c1031b0 2024-04-23T10:26:16.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.567+0000 7ff4027fc640 1 -- 172.21.15.97:0/711144014 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7ff408010460 con 0x7ff40c1031b0 2024-04-23T10:26:16.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.567+0000 7ff4027fc640 1 --2- 172.21.15.97:0/711144014 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7ff3f80a3e40 0x7ff3f80a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:16.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.567+0000 7ff410c42640 1 --2- 172.21.15.97:0/711144014 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7ff3f80a3e40 0x7ff3f80a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:16.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.568+0000 7ff4027fc640 1 -- 172.21.15.97:0/711144014 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7ff4080d06c0 con 0x7ff40c1031b0 2024-04-23T10:26:16.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.568+0000 7ff410c42640 1 --2- 172.21.15.97:0/711144014 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7ff3f80a3e40 0x7ff3f80a6300 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff40c10e3c0 tx=0x7ff3fc005f90 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:16.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.572+0000 7ff4027fc640 1 -- 172.21.15.97:0/711144014 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff40809cb60 con 0x7ff40c1031b0 2024-04-23T10:26:16.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.748+0000 7ff4136a7640 1 -- 172.21.15.97:0/711144014 --> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff40c1076e0 con 0x7ff3f80a3e40 2024-04-23T10:26:16.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.755+0000 7ff4027fc640 1 -- 172.21.15.97:0/711144014 <== mgr.15378 v2:172.21.15.97:6800/3567268502 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258076 (secure 0 0 0) 0x7ff3fc016070 con 0x7ff3f80a3e40 2024-04-23T10:26:16.757 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:26:16.758 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-04-23T10:26:16.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.758+0000 7ff4136a7640 1 -- 172.21.15.97:0/711144014 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7ff3f80a3e40 msgr2=0x7ff3f80a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:16.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.758+0000 7ff4136a7640 1 --2- 172.21.15.97:0/711144014 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7ff3f80a3e40 0x7ff3f80a6300 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff40c10e3c0 tx=0x7ff3fc005f90 comp rx=0 tx=0).stop 2024-04-23T10:26:16.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.758+0000 7ff4136a7640 1 -- 172.21.15.97:0/711144014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff40c1031b0 msgr2=0x7ff40c198760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:16.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.758+0000 7ff4136a7640 1 --2- 172.21.15.97:0/711144014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff40c1031b0 0x7ff40c198760 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff40800d950 tx=0x7ff40800de10 comp rx=0 tx=0).stop 2024-04-23T10:26:16.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.758+0000 7ff4136a7640 1 -- 172.21.15.97:0/711144014 shutdown_connections 2024-04-23T10:26:16.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.758+0000 7ff4136a7640 1 --2- 172.21.15.97:0/711144014 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7ff3f80a3e40 0x7ff3f80a6300 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:16.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.758+0000 7ff4136a7640 1 --2- 172.21.15.97:0/711144014 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff40c10c580 0x7ff40c19cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:16.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.758+0000 7ff4136a7640 1 --2- 172.21.15.97:0/711144014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff40c103ad0 0x7ff40c198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:16.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.758+0000 7ff4136a7640 1 --2- 172.21.15.97:0/711144014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff40c1031b0 0x7ff40c198760 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:16.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.758+0000 7ff4136a7640 1 -- 172.21.15.97:0/711144014 >> 172.21.15.97:0/711144014 conn(0x7ff40c0771d0 msgr2=0x7ff40c0fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:16.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.758+0000 7ff4136a7640 1 -- 172.21.15.97:0/711144014 shutdown_connections 2024-04-23T10:26:16.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.758+0000 7ff4136a7640 1 -- 172.21.15.97:0/711144014 wait complete. 2024-04-23T10:26:16.768 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-04-23T10:26:15.131026+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1192758,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1478,"num_read_kb":904,"num_write":558,"num_write_kb":4409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1181990,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":943,"ondisk_log_size":943,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361296,"kb_used_data":12096,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580528,"statfs":{"total":1151940427776,"available":1151570460672,"internally_reserved":0,"allocated":12386304,"data_stored":9187998,"data_compressed":223143,"data_compressed_allocated":294912,"data_compressed_original":3440640,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.991977"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.356030+0000","last_change":"2024-04-23T10:24:07.040397+0000","last_active":"2024-04-23T10:26:11.356030+0000","last_peered":"2024-04-23T10:26:11.356030+0000","last_clean":"2024-04-23T10:26:11.356030+0000","last_became_active":"2024-04-23T10:24:07.040007+0000","last_became_peered":"2024-04-23T10:24:07.040007+0000","last_unstale":"2024-04-23T10:26:11.356030+0000","last_undegraded":"2024-04-23T10:26:11.356030+0000","last_fullsized":"2024-04-23T10:26:11.356030+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T13:50:24.533115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.350839+0000","last_change":"2024-04-23T10:24:07.044672+0000","last_active":"2024-04-23T10:26:11.350839+0000","last_peered":"2024-04-23T10:26:11.350839+0000","last_clean":"2024-04-23T10:26:11.350839+0000","last_became_active":"2024-04-23T10:24:07.044184+0000","last_became_peered":"2024-04-23T10:24:07.044184+0000","last_unstale":"2024-04-23T10:26:11.350839+0000","last_undegraded":"2024-04-23T10:26:11.350839+0000","last_fullsized":"2024-04-23T10:26:11.350839+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:07:16.729205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.350394+0000","last_change":"2024-04-23T10:24:07.041959+0000","last_active":"2024-04-23T10:26:11.350394+0000","last_peered":"2024-04-23T10:26:11.350394+0000","last_clean":"2024-04-23T10:26:11.350394+0000","last_became_active":"2024-04-23T10:24:07.041661+0000","last_became_peered":"2024-04-23T10:24:07.041661+0000","last_unstale":"2024-04-23T10:26:11.350394+0000","last_undegraded":"2024-04-23T10:26:11.350394+0000","last_fullsized":"2024-04-23T10:26:11.350394+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:23:14.694183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.131765+0000","last_change":"2024-04-23T10:24:07.048334+0000","last_active":"2024-04-23T10:26:11.131765+0000","last_peered":"2024-04-23T10:26:11.131765+0000","last_clean":"2024-04-23T10:26:11.131765+0000","last_became_active":"2024-04-23T10:24:07.047506+0000","last_became_peered":"2024-04-23T10:24:07.047506+0000","last_unstale":"2024-04-23T10:26:11.131765+0000","last_undegraded":"2024-04-23T10:26:11.131765+0000","last_fullsized":"2024-04-23T10:26:11.131765+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:27:27.674287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.157602+0000","last_change":"2024-04-23T10:24:07.035531+0000","last_active":"2024-04-23T10:26:11.157602+0000","last_peered":"2024-04-23T10:26:11.157602+0000","last_clean":"2024-04-23T10:26:11.157602+0000","last_became_active":"2024-04-23T10:24:07.035167+0000","last_became_peered":"2024-04-23T10:24:07.035167+0000","last_unstale":"2024-04-23T10:26:11.157602+0000","last_undegraded":"2024-04-23T10:26:11.157602+0000","last_fullsized":"2024-04-23T10:26:11.157602+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:39:17.198650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.124553+0000","last_change":"2024-04-23T10:24:07.040593+0000","last_active":"2024-04-23T10:26:11.124553+0000","last_peered":"2024-04-23T10:26:11.124553+0000","last_clean":"2024-04-23T10:26:11.124553+0000","last_became_active":"2024-04-23T10:24:07.040255+0000","last_became_peered":"2024-04-23T10:24:07.040255+0000","last_unstale":"2024-04-23T10:26:11.124553+0000","last_undegraded":"2024-04-23T10:26:11.124553+0000","last_fullsized":"2024-04-23T10:26:11.124553+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:05:52.663566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.116448+0000","last_change":"2024-04-23T10:24:07.051688+0000","last_active":"2024-04-23T10:26:11.116448+0000","last_peered":"2024-04-23T10:26:11.116448+0000","last_clean":"2024-04-23T10:26:11.116448+0000","last_became_active":"2024-04-23T10:24:07.050265+0000","last_became_peered":"2024-04-23T10:24:07.050265+0000","last_unstale":"2024-04-23T10:26:11.116448+0000","last_undegraded":"2024-04-23T10:26:11.116448+0000","last_fullsized":"2024-04-23T10:26:11.116448+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T13:13:36.126183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.131842+0000","last_change":"2024-04-23T10:24:07.045937+0000","last_active":"2024-04-23T10:26:11.131842+0000","last_peered":"2024-04-23T10:26:11.131842+0000","last_clean":"2024-04-23T10:26:11.131842+0000","last_became_active":"2024-04-23T10:24:07.045616+0000","last_became_peered":"2024-04-23T10:24:07.045616+0000","last_unstale":"2024-04-23T10:26:11.131842+0000","last_undegraded":"2024-04-23T10:26:11.131842+0000","last_fullsized":"2024-04-23T10:26:11.131842+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T10:45:27.250031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.349840+0000","last_change":"2024-04-23T10:24:07.039294+0000","last_active":"2024-04-23T10:26:11.349840+0000","last_peered":"2024-04-23T10:26:11.349840+0000","last_clean":"2024-04-23T10:26:11.349840+0000","last_became_active":"2024-04-23T10:24:07.038959+0000","last_became_peered":"2024-04-23T10:24:07.038959+0000","last_unstale":"2024-04-23T10:26:11.349840+0000","last_undegraded":"2024-04-23T10:26:11.349840+0000","last_fullsized":"2024-04-23T10:26:11.349840+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:53:41.345062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.124874+0000","last_change":"2024-04-23T10:24:07.044177+0000","last_active":"2024-04-23T10:26:11.124874+0000","last_peered":"2024-04-23T10:26:11.124874+0000","last_clean":"2024-04-23T10:26:11.124874+0000","last_became_active":"2024-04-23T10:24:07.043821+0000","last_became_peered":"2024-04-23T10:24:07.043821+0000","last_unstale":"2024-04-23T10:26:11.124874+0000","last_undegraded":"2024-04-23T10:26:11.124874+0000","last_fullsized":"2024-04-23T10:26:11.124874+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:09:38.782407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.116882+0000","last_change":"2024-04-23T10:24:07.052775+0000","last_active":"2024-04-23T10:26:11.116882+0000","last_peered":"2024-04-23T10:26:11.116882+0000","last_clean":"2024-04-23T10:26:11.116882+0000","last_became_active":"2024-04-23T10:24:07.051708+0000","last_became_peered":"2024-04-23T10:24:07.051708+0000","last_unstale":"2024-04-23T10:26:11.116882+0000","last_undegraded":"2024-04-23T10:26:11.116882+0000","last_fullsized":"2024-04-23T10:26:11.116882+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:47:55.360467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.131663+0000","last_change":"2024-04-23T10:24:07.048655+0000","last_active":"2024-04-23T10:26:11.131663+0000","last_peered":"2024-04-23T10:26:11.131663+0000","last_clean":"2024-04-23T10:26:11.131663+0000","last_became_active":"2024-04-23T10:24:07.047994+0000","last_became_peered":"2024-04-23T10:24:07.047994+0000","last_unstale":"2024-04-23T10:26:11.131663+0000","last_undegraded":"2024-04-23T10:26:11.131663+0000","last_fullsized":"2024-04-23T10:26:11.131663+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T10:25:51.425871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.129951+0000","last_change":"2024-04-23T10:24:07.036319+0000","last_active":"2024-04-23T10:26:11.129951+0000","last_peered":"2024-04-23T10:26:11.129951+0000","last_clean":"2024-04-23T10:26:11.129951+0000","last_became_active":"2024-04-23T10:24:07.035982+0000","last_became_peered":"2024-04-23T10:24:07.035982+0000","last_unstale":"2024-04-23T10:26:11.129951+0000","last_undegraded":"2024-04-23T10:26:11.129951+0000","last_fullsized":"2024-04-23T10:26:11.129951+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T10:25:46.191118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.349736+0000","last_change":"2024-04-23T10:24:07.033166+0000","last_active":"2024-04-23T10:26:11.349736+0000","last_peered":"2024-04-23T10:26:11.349736+0000","last_clean":"2024-04-23T10:26:11.349736+0000","last_became_active":"2024-04-23T10:24:07.032807+0000","last_became_peered":"2024-04-23T10:24:07.032807+0000","last_unstale":"2024-04-23T10:26:11.349736+0000","last_undegraded":"2024-04-23T10:26:11.349736+0000","last_fullsized":"2024-04-23T10:26:11.349736+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:52:35.654965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.349923+0000","last_change":"2024-04-23T10:24:07.041988+0000","last_active":"2024-04-23T10:26:11.349923+0000","last_peered":"2024-04-23T10:26:11.349923+0000","last_clean":"2024-04-23T10:26:11.349923+0000","last_became_active":"2024-04-23T10:24:07.041638+0000","last_became_peered":"2024-04-23T10:24:07.041638+0000","last_unstale":"2024-04-23T10:26:11.349923+0000","last_undegraded":"2024-04-23T10:26:11.349923+0000","last_fullsized":"2024-04-23T10:26:11.349923+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:17:40.565981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.349144+0000","last_change":"2024-04-23T10:24:07.045111+0000","last_active":"2024-04-23T10:26:11.349144+0000","last_peered":"2024-04-23T10:26:11.349144+0000","last_clean":"2024-04-23T10:26:11.349144+0000","last_became_active":"2024-04-23T10:24:07.044249+0000","last_became_peered":"2024-04-23T10:24:07.044249+0000","last_unstale":"2024-04-23T10:26:11.349144+0000","last_undegraded":"2024-04-23T10:26:11.349144+0000","last_fullsized":"2024-04-23T10:26:11.349144+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:08:36.029636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.356095+0000","last_change":"2024-04-23T10:24:07.040685+0000","last_active":"2024-04-23T10:26:11.356095+0000","last_peered":"2024-04-23T10:26:11.356095+0000","last_clean":"2024-04-23T10:26:11.356095+0000","last_became_active":"2024-04-23T10:24:07.038881+0000","last_became_peered":"2024-04-23T10:24:07.038881+0000","last_unstale":"2024-04-23T10:26:11.356095+0000","last_undegraded":"2024-04-23T10:26:11.356095+0000","last_fullsized":"2024-04-23T10:26:11.356095+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:29:12.489136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.117665+0000","last_change":"2024-04-23T10:24:07.041815+0000","last_active":"2024-04-23T10:26:11.117665+0000","last_peered":"2024-04-23T10:26:11.117665+0000","last_clean":"2024-04-23T10:26:11.117665+0000","last_became_active":"2024-04-23T10:24:07.041495+0000","last_became_peered":"2024-04-23T10:24:07.041495+0000","last_unstale":"2024-04-23T10:26:11.117665+0000","last_undegraded":"2024-04-23T10:26:11.117665+0000","last_fullsized":"2024-04-23T10:26:11.117665+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:29:02.006394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.116347+0000","last_change":"2024-04-23T10:24:07.041722+0000","last_active":"2024-04-23T10:26:11.116347+0000","last_peered":"2024-04-23T10:26:11.116347+0000","last_clean":"2024-04-23T10:26:11.116347+0000","last_became_active":"2024-04-23T10:24:07.039434+0000","last_became_peered":"2024-04-23T10:24:07.039434+0000","last_unstale":"2024-04-23T10:26:11.116347+0000","last_undegraded":"2024-04-23T10:26:11.116347+0000","last_fullsized":"2024-04-23T10:26:11.116347+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:03:44.033094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'14","reported_seq":176,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.825209+0000","last_change":"2024-04-23T10:25:00.739010+0000","last_active":"2024-04-23T10:26:11.825209+0000","last_peered":"2024-04-23T10:26:11.825209+0000","last_clean":"2024-04-23T10:26:11.825209+0000","last_became_active":"2024-04-23T10:25:00.729834+0000","last_became_peered":"2024-04-23T10:25:00.729834+0000","last_unstale":"2024-04-23T10:26:11.825209+0000","last_undegraded":"2024-04-23T10:26:11.825209+0000","last_fullsized":"2024-04-23T10:26:11.825209+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T12:57:45.871560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,4],"acting":[9,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.132261+0000","last_change":"2024-04-23T10:24:07.047192+0000","last_active":"2024-04-23T10:26:11.132261+0000","last_peered":"2024-04-23T10:26:11.132261+0000","last_clean":"2024-04-23T10:26:11.132261+0000","last_became_active":"2024-04-23T10:24:07.046750+0000","last_became_peered":"2024-04-23T10:24:07.046750+0000","last_unstale":"2024-04-23T10:26:11.132261+0000","last_undegraded":"2024-04-23T10:26:11.132261+0000","last_fullsized":"2024-04-23T10:26:11.132261+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:13:09.399425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.132396+0000","last_change":"2024-04-23T10:24:06.577588+0000","last_active":"2024-04-23T10:26:11.132396+0000","last_peered":"2024-04-23T10:26:11.132396+0000","last_clean":"2024-04-23T10:26:11.132396+0000","last_became_active":"2024-04-23T10:24:06.576971+0000","last_became_peered":"2024-04-23T10:24:06.576971+0000","last_unstale":"2024-04-23T10:26:11.132396+0000","last_undegraded":"2024-04-23T10:26:11.132396+0000","last_fullsized":"2024-04-23T10:26:11.132396+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:07:21.577868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.350711+0000","last_change":"2024-04-23T10:24:07.030666+0000","last_active":"2024-04-23T10:26:11.350711+0000","last_peered":"2024-04-23T10:26:11.350711+0000","last_clean":"2024-04-23T10:26:11.350711+0000","last_became_active":"2024-04-23T10:24:07.030342+0000","last_became_peered":"2024-04-23T10:24:07.030342+0000","last_unstale":"2024-04-23T10:26:11.350711+0000","last_undegraded":"2024-04-23T10:26:11.350711+0000","last_fullsized":"2024-04-23T10:26:11.350711+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:09:14.818638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"89'13","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.157995+0000","last_change":"2024-04-23T10:24:06.576042+0000","last_active":"2024-04-23T10:26:11.157995+0000","last_peered":"2024-04-23T10:26:11.157995+0000","last_clean":"2024-04-23T10:26:11.157995+0000","last_became_active":"2024-04-23T10:24:06.574518+0000","last_became_peered":"2024-04-23T10:24:06.574518+0000","last_unstale":"2024-04-23T10:26:11.157995+0000","last_undegraded":"2024-04-23T10:26:11.157995+0000","last_fullsized":"2024-04-23T10:26:11.157995+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T11:56:40.100213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.349417+0000","last_change":"2024-04-23T10:24:07.046591+0000","last_active":"2024-04-23T10:26:11.349417+0000","last_peered":"2024-04-23T10:26:11.349417+0000","last_clean":"2024-04-23T10:26:11.349417+0000","last_became_active":"2024-04-23T10:24:07.046208+0000","last_became_peered":"2024-04-23T10:24:07.046208+0000","last_unstale":"2024-04-23T10:26:11.349417+0000","last_undegraded":"2024-04-23T10:26:11.349417+0000","last_fullsized":"2024-04-23T10:26:11.349417+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:27:37.711329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.355666+0000","last_change":"2024-04-23T10:25:00.676255+0000","last_active":"2024-04-23T10:26:11.355666+0000","last_peered":"2024-04-23T10:26:11.355666+0000","last_clean":"2024-04-23T10:26:11.355666+0000","last_became_active":"2024-04-23T10:25:00.670146+0000","last_became_peered":"2024-04-23T10:25:00.670146+0000","last_unstale":"2024-04-23T10:26:11.355666+0000","last_undegraded":"2024-04-23T10:26:11.355666+0000","last_fullsized":"2024-04-23T10:26:11.355666+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:34:54.217317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.127034+0000","last_change":"2024-04-23T10:24:07.047926+0000","last_active":"2024-04-23T10:26:11.127034+0000","last_peered":"2024-04-23T10:26:11.127034+0000","last_clean":"2024-04-23T10:26:11.127034+0000","last_became_active":"2024-04-23T10:24:07.047273+0000","last_became_peered":"2024-04-23T10:24:07.047273+0000","last_unstale":"2024-04-23T10:26:11.127034+0000","last_undegraded":"2024-04-23T10:26:11.127034+0000","last_fullsized":"2024-04-23T10:26:11.127034+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:37:25.068034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'7","reported_seq":154,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.127172+0000","last_change":"2024-04-23T10:24:06.574733+0000","last_active":"2024-04-23T10:26:11.127172+0000","last_peered":"2024-04-23T10:26:11.127172+0000","last_clean":"2024-04-23T10:26:11.127172+0000","last_became_active":"2024-04-23T10:24:06.574276+0000","last_became_peered":"2024-04-23T10:24:06.574276+0000","last_unstale":"2024-04-23T10:26:11.127172+0000","last_undegraded":"2024-04-23T10:26:11.127172+0000","last_fullsized":"2024-04-23T10:26:11.127172+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:27:57.080635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.157562+0000","last_change":"2024-04-23T10:24:07.040110+0000","last_active":"2024-04-23T10:26:11.157562+0000","last_peered":"2024-04-23T10:26:11.157562+0000","last_clean":"2024-04-23T10:26:11.157562+0000","last_became_active":"2024-04-23T10:24:07.039769+0000","last_became_peered":"2024-04-23T10:24:07.039769+0000","last_unstale":"2024-04-23T10:26:11.157562+0000","last_undegraded":"2024-04-23T10:26:11.157562+0000","last_fullsized":"2024-04-23T10:26:11.157562+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:10:07.884895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.124478+0000","last_change":"2024-04-23T10:24:06.567448+0000","last_active":"2024-04-23T10:26:11.124478+0000","last_peered":"2024-04-23T10:26:11.124478+0000","last_clean":"2024-04-23T10:26:11.124478+0000","last_became_active":"2024-04-23T10:24:06.566958+0000","last_became_peered":"2024-04-23T10:24:06.566958+0000","last_unstale":"2024-04-23T10:26:11.124478+0000","last_undegraded":"2024-04-23T10:26:11.124478+0000","last_fullsized":"2024-04-23T10:26:11.124478+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:45:52.863596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.126797+0000","last_change":"2024-04-23T10:24:07.040754+0000","last_active":"2024-04-23T10:26:11.126797+0000","last_peered":"2024-04-23T10:26:11.126797+0000","last_clean":"2024-04-23T10:26:11.126797+0000","last_became_active":"2024-04-23T10:24:07.040397+0000","last_became_peered":"2024-04-23T10:24:07.040397+0000","last_unstale":"2024-04-23T10:26:11.126797+0000","last_undegraded":"2024-04-23T10:26:11.126797+0000","last_fullsized":"2024-04-23T10:26:11.126797+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T11:01:03.626506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.129719+0000","last_change":"2024-04-23T10:24:06.568592+0000","last_active":"2024-04-23T10:26:11.129719+0000","last_peered":"2024-04-23T10:26:11.129719+0000","last_clean":"2024-04-23T10:26:11.129719+0000","last_became_active":"2024-04-23T10:24:06.568192+0000","last_became_peered":"2024-04-23T10:24:06.568192+0000","last_unstale":"2024-04-23T10:26:11.129719+0000","last_undegraded":"2024-04-23T10:26:11.129719+0000","last_fullsized":"2024-04-23T10:26:11.129719+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:14:42.589596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.131909+0000","last_change":"2024-04-23T10:24:07.037058+0000","last_active":"2024-04-23T10:26:11.131909+0000","last_peered":"2024-04-23T10:26:11.131909+0000","last_clean":"2024-04-23T10:26:11.131909+0000","last_became_active":"2024-04-23T10:24:07.036659+0000","last_became_peered":"2024-04-23T10:24:07.036659+0000","last_unstale":"2024-04-23T10:26:11.131909+0000","last_undegraded":"2024-04-23T10:26:11.131909+0000","last_fullsized":"2024-04-23T10:26:11.131909+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T12:53:49.298603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.127061+0000","last_change":"2024-04-23T10:24:06.576900+0000","last_active":"2024-04-23T10:26:11.127061+0000","last_peered":"2024-04-23T10:26:11.127061+0000","last_clean":"2024-04-23T10:26:11.127061+0000","last_became_active":"2024-04-23T10:24:06.575627+0000","last_became_peered":"2024-04-23T10:24:06.575627+0000","last_unstale":"2024-04-23T10:26:11.127061+0000","last_undegraded":"2024-04-23T10:26:11.127061+0000","last_fullsized":"2024-04-23T10:26:11.127061+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:16:07.973841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.117469+0000","last_change":"2024-04-23T10:24:07.035294+0000","last_active":"2024-04-23T10:26:11.117469+0000","last_peered":"2024-04-23T10:26:11.117469+0000","last_clean":"2024-04-23T10:26:11.117469+0000","last_became_active":"2024-04-23T10:24:07.035001+0000","last_became_peered":"2024-04-23T10:24:07.035001+0000","last_unstale":"2024-04-23T10:26:11.117469+0000","last_undegraded":"2024-04-23T10:26:11.117469+0000","last_fullsized":"2024-04-23T10:26:11.117469+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T13:12:47.013132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"89'7","reported_seq":269,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.355330+0000","last_change":"2024-04-23T10:24:06.573637+0000","last_active":"2024-04-23T10:26:11.355330+0000","last_peered":"2024-04-23T10:26:11.355330+0000","last_clean":"2024-04-23T10:26:11.355330+0000","last_became_active":"2024-04-23T10:24:06.573236+0000","last_became_peered":"2024-04-23T10:24:06.573236+0000","last_unstale":"2024-04-23T10:26:11.355330+0000","last_undegraded":"2024-04-23T10:26:11.355330+0000","last_fullsized":"2024-04-23T10:26:11.355330+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T12:48:06.740181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":241,"num_read_kb":241,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.349816+0000","last_change":"2024-04-23T10:24:07.037457+0000","last_active":"2024-04-23T10:26:11.349816+0000","last_peered":"2024-04-23T10:26:11.349816+0000","last_clean":"2024-04-23T10:26:11.349816+0000","last_became_active":"2024-04-23T10:24:07.036456+0000","last_became_peered":"2024-04-23T10:24:07.036456+0000","last_unstale":"2024-04-23T10:26:11.349816+0000","last_undegraded":"2024-04-23T10:26:11.349816+0000","last_fullsized":"2024-04-23T10:26:11.349816+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:42:57.750435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.116889+0000","last_change":"2024-04-23T10:25:00.741712+0000","last_active":"2024-04-23T10:26:11.116889+0000","last_peered":"2024-04-23T10:26:11.116889+0000","last_clean":"2024-04-23T10:26:11.116889+0000","last_became_active":"2024-04-23T10:25:00.730130+0000","last_became_peered":"2024-04-23T10:25:00.730130+0000","last_unstale":"2024-04-23T10:26:11.116889+0000","last_undegraded":"2024-04-23T10:26:11.116889+0000","last_fullsized":"2024-04-23T10:26:11.116889+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:08:14.409376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.350621+0000","last_change":"2024-04-23T10:24:07.040317+0000","last_active":"2024-04-23T10:26:11.350621+0000","last_peered":"2024-04-23T10:26:11.350621+0000","last_clean":"2024-04-23T10:26:11.350621+0000","last_became_active":"2024-04-23T10:24:07.039998+0000","last_became_peered":"2024-04-23T10:24:07.039998+0000","last_unstale":"2024-04-23T10:26:11.350621+0000","last_undegraded":"2024-04-23T10:26:11.350621+0000","last_fullsized":"2024-04-23T10:26:11.350621+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:50:30.412127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.127037+0000","last_change":"2024-04-23T10:24:06.570019+0000","last_active":"2024-04-23T10:26:11.127037+0000","last_peered":"2024-04-23T10:26:11.127037+0000","last_clean":"2024-04-23T10:26:11.127037+0000","last_became_active":"2024-04-23T10:24:06.569676+0000","last_became_peered":"2024-04-23T10:24:06.569676+0000","last_unstale":"2024-04-23T10:26:11.127037+0000","last_undegraded":"2024-04-23T10:26:11.127037+0000","last_fullsized":"2024-04-23T10:26:11.127037+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:31:46.994787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.349539+0000","last_change":"2024-04-23T10:24:07.040183+0000","last_active":"2024-04-23T10:26:11.349539+0000","last_peered":"2024-04-23T10:26:11.349539+0000","last_clean":"2024-04-23T10:26:11.349539+0000","last_became_active":"2024-04-23T10:24:07.039866+0000","last_became_peered":"2024-04-23T10:24:07.039866+0000","last_unstale":"2024-04-23T10:26:11.349539+0000","last_undegraded":"2024-04-23T10:26:11.349539+0000","last_fullsized":"2024-04-23T10:26:11.349539+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:33:04.411765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"89'38","reported_seq":315,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:12.128873+0000","last_change":"2024-04-23T10:24:06.568883+0000","last_active":"2024-04-23T10:26:12.128873+0000","last_peered":"2024-04-23T10:26:12.128873+0000","last_clean":"2024-04-23T10:26:12.128873+0000","last_became_active":"2024-04-23T10:24:06.568517+0000","last_became_peered":"2024-04-23T10:24:06.568517+0000","last_unstale":"2024-04-23T10:26:12.128873+0000","last_undegraded":"2024-04-23T10:26:12.128873+0000","last_fullsized":"2024-04-23T10:26:12.128873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:51:42.440346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67758,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":256,"num_read_kb":194,"num_write":40,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.157789+0000","last_change":"2024-04-23T10:24:07.046839+0000","last_active":"2024-04-23T10:26:11.157789+0000","last_peered":"2024-04-23T10:26:11.157789+0000","last_clean":"2024-04-23T10:26:11.157789+0000","last_became_active":"2024-04-23T10:24:07.046512+0000","last_became_peered":"2024-04-23T10:24:07.046512+0000","last_unstale":"2024-04-23T10:26:11.157789+0000","last_undegraded":"2024-04-23T10:26:11.157789+0000","last_fullsized":"2024-04-23T10:26:11.157789+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T12:42:11.822100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.116303+0000","last_change":"2024-04-23T10:24:06.578087+0000","last_active":"2024-04-23T10:26:11.116303+0000","last_peered":"2024-04-23T10:26:11.116303+0000","last_clean":"2024-04-23T10:26:11.116303+0000","last_became_active":"2024-04-23T10:24:06.577666+0000","last_became_peered":"2024-04-23T10:24:06.577666+0000","last_unstale":"2024-04-23T10:26:11.116303+0000","last_undegraded":"2024-04-23T10:26:11.116303+0000","last_fullsized":"2024-04-23T10:26:11.116303+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:55:15.053303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.129816+0000","last_change":"2024-04-23T10:24:07.043672+0000","last_active":"2024-04-23T10:26:11.129816+0000","last_peered":"2024-04-23T10:26:11.129816+0000","last_clean":"2024-04-23T10:26:11.129816+0000","last_became_active":"2024-04-23T10:24:07.043346+0000","last_became_peered":"2024-04-23T10:24:07.043346+0000","last_unstale":"2024-04-23T10:26:11.129816+0000","last_undegraded":"2024-04-23T10:26:11.129816+0000","last_fullsized":"2024-04-23T10:26:11.129816+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T13:21:56.751028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.126748+0000","last_change":"2024-04-23T10:24:06.576607+0000","last_active":"2024-04-23T10:26:11.126748+0000","last_peered":"2024-04-23T10:26:11.126748+0000","last_clean":"2024-04-23T10:26:11.126748+0000","last_became_active":"2024-04-23T10:24:06.576213+0000","last_became_peered":"2024-04-23T10:24:06.576213+0000","last_unstale":"2024-04-23T10:26:11.126748+0000","last_undegraded":"2024-04-23T10:26:11.126748+0000","last_fullsized":"2024-04-23T10:26:11.126748+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:09:14.918936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.116719+0000","last_change":"2024-04-23T10:24:07.052641+0000","last_active":"2024-04-23T10:26:11.116719+0000","last_peered":"2024-04-23T10:26:11.116719+0000","last_clean":"2024-04-23T10:26:11.116719+0000","last_became_active":"2024-04-23T10:24:07.052314+0000","last_became_peered":"2024-04-23T10:24:07.052314+0000","last_unstale":"2024-04-23T10:26:11.116719+0000","last_undegraded":"2024-04-23T10:26:11.116719+0000","last_fullsized":"2024-04-23T10:26:11.116719+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T10:29:18.098800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'8","reported_seq":155,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.349410+0000","last_change":"2024-04-23T10:24:06.574570+0000","last_active":"2024-04-23T10:26:11.349410+0000","last_peered":"2024-04-23T10:26:11.349410+0000","last_clean":"2024-04-23T10:26:11.349410+0000","last_became_active":"2024-04-23T10:24:06.574166+0000","last_became_peered":"2024-04-23T10:24:06.574166+0000","last_unstale":"2024-04-23T10:26:11.349410+0000","last_undegraded":"2024-04-23T10:26:11.349410+0000","last_fullsized":"2024-04-23T10:26:11.349410+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:28:31.619100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":127,"num_read_kb":5,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.116415+0000","last_change":"2024-04-23T10:24:07.052825+0000","last_active":"2024-04-23T10:26:11.116415+0000","last_peered":"2024-04-23T10:26:11.116415+0000","last_clean":"2024-04-23T10:26:11.116415+0000","last_became_active":"2024-04-23T10:24:07.052457+0000","last_became_peered":"2024-04-23T10:24:07.052457+0000","last_unstale":"2024-04-23T10:26:11.116415+0000","last_undegraded":"2024-04-23T10:26:11.116415+0000","last_fullsized":"2024-04-23T10:26:11.116415+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:47:09.545374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.157540+0000","last_change":"2024-04-23T10:24:06.572168+0000","last_active":"2024-04-23T10:26:11.157540+0000","last_peered":"2024-04-23T10:26:11.157540+0000","last_clean":"2024-04-23T10:26:11.157540+0000","last_became_active":"2024-04-23T10:24:06.571805+0000","last_became_peered":"2024-04-23T10:24:06.571805+0000","last_unstale":"2024-04-23T10:26:11.157540+0000","last_undegraded":"2024-04-23T10:26:11.157540+0000","last_fullsized":"2024-04-23T10:26:11.157540+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:29:56.817879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.126680+0000","last_change":"2024-04-23T10:24:07.048474+0000","last_active":"2024-04-23T10:26:11.126680+0000","last_peered":"2024-04-23T10:26:11.126680+0000","last_clean":"2024-04-23T10:26:11.126680+0000","last_became_active":"2024-04-23T10:24:07.047893+0000","last_became_peered":"2024-04-23T10:24:07.047893+0000","last_unstale":"2024-04-23T10:26:11.126680+0000","last_undegraded":"2024-04-23T10:26:11.126680+0000","last_fullsized":"2024-04-23T10:26:11.126680+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T12:59:46.545569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.157883+0000","last_change":"2024-04-23T10:24:06.570974+0000","last_active":"2024-04-23T10:26:11.157883+0000","last_peered":"2024-04-23T10:26:11.157883+0000","last_clean":"2024-04-23T10:26:11.157883+0000","last_became_active":"2024-04-23T10:24:06.570603+0000","last_became_peered":"2024-04-23T10:24:06.570603+0000","last_unstale":"2024-04-23T10:26:11.157883+0000","last_undegraded":"2024-04-23T10:26:11.157883+0000","last_fullsized":"2024-04-23T10:26:11.157883+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T12:46:33.518337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.355892+0000","last_change":"2024-04-23T10:24:07.050986+0000","last_active":"2024-04-23T10:26:11.355892+0000","last_peered":"2024-04-23T10:26:11.355892+0000","last_clean":"2024-04-23T10:26:11.355892+0000","last_became_active":"2024-04-23T10:24:07.050617+0000","last_became_peered":"2024-04-23T10:24:07.050617+0000","last_unstale":"2024-04-23T10:26:11.355892+0000","last_undegraded":"2024-04-23T10:26:11.355892+0000","last_fullsized":"2024-04-23T10:26:11.355892+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:13:25.355117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.117580+0000","last_change":"2024-04-23T10:24:06.579138+0000","last_active":"2024-04-23T10:26:11.117580+0000","last_peered":"2024-04-23T10:26:11.117580+0000","last_clean":"2024-04-23T10:26:11.117580+0000","last_became_active":"2024-04-23T10:24:06.578716+0000","last_became_peered":"2024-04-23T10:24:06.578716+0000","last_unstale":"2024-04-23T10:26:11.117580+0000","last_undegraded":"2024-04-23T10:26:11.117580+0000","last_fullsized":"2024-04-23T10:26:11.117580+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:06:59.797802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.124849+0000","last_change":"2024-04-23T10:24:07.039019+0000","last_active":"2024-04-23T10:26:11.124849+0000","last_peered":"2024-04-23T10:26:11.124849+0000","last_clean":"2024-04-23T10:26:11.124849+0000","last_became_active":"2024-04-23T10:24:07.038701+0000","last_became_peered":"2024-04-23T10:24:07.038701+0000","last_unstale":"2024-04-23T10:26:11.124849+0000","last_undegraded":"2024-04-23T10:26:11.124849+0000","last_fullsized":"2024-04-23T10:26:11.124849+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:26:32.716043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'7","reported_seq":149,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.825787+0000","last_change":"2024-04-23T10:24:06.574342+0000","last_active":"2024-04-23T10:26:11.825787+0000","last_peered":"2024-04-23T10:26:11.825787+0000","last_clean":"2024-04-23T10:26:11.825787+0000","last_became_active":"2024-04-23T10:24:06.573981+0000","last_became_peered":"2024-04-23T10:24:06.573981+0000","last_unstale":"2024-04-23T10:26:11.825787+0000","last_undegraded":"2024-04-23T10:26:11.825787+0000","last_fullsized":"2024-04-23T10:26:11.825787+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:48:40.325991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1370,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.124690+0000","last_change":"2024-04-23T10:24:07.044712+0000","last_active":"2024-04-23T10:26:11.124690+0000","last_peered":"2024-04-23T10:26:11.124690+0000","last_clean":"2024-04-23T10:26:11.124690+0000","last_became_active":"2024-04-23T10:24:07.044141+0000","last_became_peered":"2024-04-23T10:24:07.044141+0000","last_unstale":"2024-04-23T10:26:11.124690+0000","last_undegraded":"2024-04-23T10:26:11.124690+0000","last_fullsized":"2024-04-23T10:26:11.124690+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:14:38.818648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.117318+0000","last_change":"2024-04-23T10:25:00.729397+0000","last_active":"2024-04-23T10:26:11.117318+0000","last_peered":"2024-04-23T10:26:11.117318+0000","last_clean":"2024-04-23T10:26:11.117318+0000","last_became_active":"2024-04-23T10:25:00.729046+0000","last_became_peered":"2024-04-23T10:25:00.729046+0000","last_unstale":"2024-04-23T10:26:11.117318+0000","last_undegraded":"2024-04-23T10:26:11.117318+0000","last_fullsized":"2024-04-23T10:26:11.117318+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:25:55.189292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"89'7","reported_seq":152,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:12.133438+0000","last_change":"2024-04-23T10:24:06.578216+0000","last_active":"2024-04-23T10:26:12.133438+0000","last_peered":"2024-04-23T10:26:12.133438+0000","last_clean":"2024-04-23T10:26:12.133438+0000","last_became_active":"2024-04-23T10:24:06.577370+0000","last_became_peered":"2024-04-23T10:24:06.577370+0000","last_unstale":"2024-04-23T10:26:12.133438+0000","last_undegraded":"2024-04-23T10:26:12.133438+0000","last_fullsized":"2024-04-23T10:26:12.133438+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:02:06.554442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":123,"num_read_kb":123,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.132207+0000","last_change":"2024-04-23T10:24:07.039627+0000","last_active":"2024-04-23T10:26:11.132207+0000","last_peered":"2024-04-23T10:26:11.132207+0000","last_clean":"2024-04-23T10:26:11.132207+0000","last_became_active":"2024-04-23T10:24:07.039272+0000","last_became_peered":"2024-04-23T10:24:07.039272+0000","last_unstale":"2024-04-23T10:26:11.132207+0000","last_undegraded":"2024-04-23T10:26:11.132207+0000","last_fullsized":"2024-04-23T10:26:11.132207+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T13:45:44.277926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.131705+0000","last_change":"2024-04-23T10:24:06.569086+0000","last_active":"2024-04-23T10:26:11.131705+0000","last_peered":"2024-04-23T10:26:11.131705+0000","last_clean":"2024-04-23T10:26:11.131705+0000","last_became_active":"2024-04-23T10:24:06.568719+0000","last_became_peered":"2024-04-23T10:24:06.568719+0000","last_unstale":"2024-04-23T10:26:11.131705+0000","last_undegraded":"2024-04-23T10:26:11.131705+0000","last_fullsized":"2024-04-23T10:26:11.131705+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:22:10.583782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":357,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.116495+0000","last_change":"2024-04-23T10:22:49.957476+0000","last_active":"2024-04-23T10:26:11.116495+0000","last_peered":"2024-04-23T10:26:11.116495+0000","last_clean":"2024-04-23T10:26:11.116495+0000","last_became_active":"2024-04-23T10:22:49.957033+0000","last_became_peered":"2024-04-23T10:22:49.957033+0000","last_unstale":"2024-04-23T10:26:11.116495+0000","last_undegraded":"2024-04-23T10:26:11.116495+0000","last_fullsized":"2024-04-23T10:26:11.116495+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:19:25.604229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:19:25.604229+0000","last_clean_scrub_stamp":"2024-04-23T10:19:25.604229+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T12:56:17.601141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.129630+0000","last_change":"2024-04-23T10:25:00.671965+0000","last_active":"2024-04-23T10:26:11.129630+0000","last_peered":"2024-04-23T10:26:11.129630+0000","last_clean":"2024-04-23T10:26:11.129630+0000","last_became_active":"2024-04-23T10:25:00.671632+0000","last_became_peered":"2024-04-23T10:25:00.671632+0000","last_unstale":"2024-04-23T10:26:11.129630+0000","last_undegraded":"2024-04-23T10:26:11.129630+0000","last_fullsized":"2024-04-23T10:26:11.129630+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:08:40.262367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.116642+0000","last_change":"2024-04-23T10:24:06.582629+0000","last_active":"2024-04-23T10:26:11.116642+0000","last_peered":"2024-04-23T10:26:11.116642+0000","last_clean":"2024-04-23T10:26:11.116642+0000","last_became_active":"2024-04-23T10:24:06.582325+0000","last_became_peered":"2024-04-23T10:24:06.582325+0000","last_unstale":"2024-04-23T10:26:11.116642+0000","last_undegraded":"2024-04-23T10:26:11.116642+0000","last_fullsized":"2024-04-23T10:26:11.116642+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:57:09.879100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.157861+0000","last_change":"2024-04-23T10:24:06.582482+0000","last_active":"2024-04-23T10:26:11.157861+0000","last_peered":"2024-04-23T10:26:11.157861+0000","last_clean":"2024-04-23T10:26:11.157861+0000","last_became_active":"2024-04-23T10:24:06.582114+0000","last_became_peered":"2024-04-23T10:24:06.582114+0000","last_unstale":"2024-04-23T10:26:11.157861+0000","last_undegraded":"2024-04-23T10:26:11.157861+0000","last_fullsized":"2024-04-23T10:26:11.157861+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:45:42.939840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.129988+0000","last_change":"2024-04-23T10:24:07.031102+0000","last_active":"2024-04-23T10:26:11.129988+0000","last_peered":"2024-04-23T10:26:11.129988+0000","last_clean":"2024-04-23T10:26:11.129988+0000","last_became_active":"2024-04-23T10:24:07.030783+0000","last_became_peered":"2024-04-23T10:24:07.030783+0000","last_unstale":"2024-04-23T10:26:11.129988+0000","last_undegraded":"2024-04-23T10:26:11.129988+0000","last_fullsized":"2024-04-23T10:26:11.129988+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:49:46.736046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"89'49","reported_seq":93,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.318009+0000","last_change":"2024-04-23T10:24:06.577096+0000","last_active":"2024-04-23T10:26:11.318009+0000","last_peered":"2024-04-23T10:26:11.318009+0000","last_clean":"2024-04-23T10:26:11.318009+0000","last_became_active":"2024-04-23T10:24:06.576740+0000","last_became_peered":"2024-04-23T10:24:06.576740+0000","last_unstale":"2024-04-23T10:26:11.318009+0000","last_undegraded":"2024-04-23T10:26:11.318009+0000","last_fullsized":"2024-04-23T10:26:11.318009+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:44:06.532366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":86,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.132294+0000","last_change":"2024-04-23T10:24:07.043599+0000","last_active":"2024-04-23T10:26:11.132294+0000","last_peered":"2024-04-23T10:26:11.132294+0000","last_clean":"2024-04-23T10:26:11.132294+0000","last_became_active":"2024-04-23T10:24:07.042953+0000","last_became_peered":"2024-04-23T10:24:07.042953+0000","last_unstale":"2024-04-23T10:26:11.132294+0000","last_undegraded":"2024-04-23T10:26:11.132294+0000","last_fullsized":"2024-04-23T10:26:11.132294+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:07:50.487972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.131879+0000","last_change":"2024-04-23T10:24:06.570681+0000","last_active":"2024-04-23T10:26:11.131879+0000","last_peered":"2024-04-23T10:26:11.131879+0000","last_clean":"2024-04-23T10:26:11.131879+0000","last_became_active":"2024-04-23T10:24:06.570178+0000","last_became_peered":"2024-04-23T10:24:06.570178+0000","last_unstale":"2024-04-23T10:26:11.131879+0000","last_undegraded":"2024-04-23T10:26:11.131879+0000","last_fullsized":"2024-04-23T10:26:11.131879+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T12:16:56.629443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.350527+0000","last_change":"2024-04-23T10:24:07.040498+0000","last_active":"2024-04-23T10:26:11.350527+0000","last_peered":"2024-04-23T10:26:11.350527+0000","last_clean":"2024-04-23T10:26:11.350527+0000","last_became_active":"2024-04-23T10:24:07.040009+0000","last_became_peered":"2024-04-23T10:24:07.040009+0000","last_unstale":"2024-04-23T10:26:11.350527+0000","last_undegraded":"2024-04-23T10:26:11.350527+0000","last_fullsized":"2024-04-23T10:26:11.350527+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:43:17.319305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.349057+0000","last_change":"2024-04-23T10:25:00.673495+0000","last_active":"2024-04-23T10:26:11.349057+0000","last_peered":"2024-04-23T10:26:11.349057+0000","last_clean":"2024-04-23T10:26:11.349057+0000","last_became_active":"2024-04-23T10:25:00.670572+0000","last_became_peered":"2024-04-23T10:25:00.670572+0000","last_unstale":"2024-04-23T10:26:11.349057+0000","last_undegraded":"2024-04-23T10:26:11.349057+0000","last_fullsized":"2024-04-23T10:26:11.349057+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:08:04.279056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.157542+0000","last_change":"2024-04-23T10:24:07.043710+0000","last_active":"2024-04-23T10:26:11.157542+0000","last_peered":"2024-04-23T10:26:11.157542+0000","last_clean":"2024-04-23T10:26:11.157542+0000","last_became_active":"2024-04-23T10:24:07.043406+0000","last_became_peered":"2024-04-23T10:24:07.043406+0000","last_unstale":"2024-04-23T10:26:11.157542+0000","last_undegraded":"2024-04-23T10:26:11.157542+0000","last_fullsized":"2024-04-23T10:26:11.157542+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:09:14.471938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.131969+0000","last_change":"2024-04-23T10:24:06.577409+0000","last_active":"2024-04-23T10:26:11.131969+0000","last_peered":"2024-04-23T10:26:11.131969+0000","last_clean":"2024-04-23T10:26:11.131969+0000","last_became_active":"2024-04-23T10:24:06.576787+0000","last_became_peered":"2024-04-23T10:24:06.576787+0000","last_unstale":"2024-04-23T10:26:11.131969+0000","last_undegraded":"2024-04-23T10:26:11.131969+0000","last_fullsized":"2024-04-23T10:26:11.131969+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:05:28.423806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.355736+0000","last_change":"2024-04-23T10:24:07.045180+0000","last_active":"2024-04-23T10:26:11.355736+0000","last_peered":"2024-04-23T10:26:11.355736+0000","last_clean":"2024-04-23T10:26:11.355736+0000","last_became_active":"2024-04-23T10:24:07.044731+0000","last_became_peered":"2024-04-23T10:24:07.044731+0000","last_unstale":"2024-04-23T10:26:11.355736+0000","last_undegraded":"2024-04-23T10:26:11.355736+0000","last_fullsized":"2024-04-23T10:26:11.355736+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:42:34.474391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.116406+0000","last_change":"2024-04-23T10:24:06.578039+0000","last_active":"2024-04-23T10:26:11.116406+0000","last_peered":"2024-04-23T10:26:11.116406+0000","last_clean":"2024-04-23T10:26:11.116406+0000","last_became_active":"2024-04-23T10:24:06.577655+0000","last_became_peered":"2024-04-23T10:24:06.577655+0000","last_unstale":"2024-04-23T10:26:11.116406+0000","last_undegraded":"2024-04-23T10:26:11.116406+0000","last_fullsized":"2024-04-23T10:26:11.116406+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:57:35.172318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.132097+0000","last_change":"2024-04-23T10:24:07.037311+0000","last_active":"2024-04-23T10:26:11.132097+0000","last_peered":"2024-04-23T10:26:11.132097+0000","last_clean":"2024-04-23T10:26:11.132097+0000","last_became_active":"2024-04-23T10:24:07.036969+0000","last_became_peered":"2024-04-23T10:24:07.036969+0000","last_unstale":"2024-04-23T10:26:11.132097+0000","last_undegraded":"2024-04-23T10:26:11.132097+0000","last_fullsized":"2024-04-23T10:26:11.132097+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:13:18.998914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.157662+0000","last_change":"2024-04-23T10:24:06.575183+0000","last_active":"2024-04-23T10:26:11.157662+0000","last_peered":"2024-04-23T10:26:11.157662+0000","last_clean":"2024-04-23T10:26:11.157662+0000","last_became_active":"2024-04-23T10:24:06.574630+0000","last_became_peered":"2024-04-23T10:24:06.574630+0000","last_unstale":"2024-04-23T10:26:11.157662+0000","last_undegraded":"2024-04-23T10:26:11.157662+0000","last_fullsized":"2024-04-23T10:26:11.157662+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T11:18:18.189925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.116657+0000","last_change":"2024-04-23T10:24:07.048832+0000","last_active":"2024-04-23T10:26:11.116657+0000","last_peered":"2024-04-23T10:26:11.116657+0000","last_clean":"2024-04-23T10:26:11.116657+0000","last_became_active":"2024-04-23T10:24:07.048509+0000","last_became_peered":"2024-04-23T10:24:07.048509+0000","last_unstale":"2024-04-23T10:26:11.116657+0000","last_undegraded":"2024-04-23T10:26:11.116657+0000","last_fullsized":"2024-04-23T10:26:11.116657+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T13:45:26.793330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'8","reported_seq":151,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.156578+0000","last_change":"2024-04-23T10:24:06.575408+0000","last_active":"2024-04-23T10:26:11.156578+0000","last_peered":"2024-04-23T10:26:11.156578+0000","last_clean":"2024-04-23T10:26:11.156578+0000","last_became_active":"2024-04-23T10:24:06.575081+0000","last_became_peered":"2024-04-23T10:24:06.575081+0000","last_unstale":"2024-04-23T10:26:11.156578+0000","last_undegraded":"2024-04-23T10:26:11.156578+0000","last_fullsized":"2024-04-23T10:26:11.156578+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T11:28:12.765196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2093,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":122,"num_read_kb":3,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.116971+0000","last_change":"2024-04-23T10:24:07.043869+0000","last_active":"2024-04-23T10:26:11.116971+0000","last_peered":"2024-04-23T10:26:11.116971+0000","last_clean":"2024-04-23T10:26:11.116971+0000","last_became_active":"2024-04-23T10:24:07.043448+0000","last_became_peered":"2024-04-23T10:24:07.043448+0000","last_unstale":"2024-04-23T10:26:11.116971+0000","last_undegraded":"2024-04-23T10:26:11.116971+0000","last_fullsized":"2024-04-23T10:26:11.116971+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:00:35.821470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'41","reported_seq":70,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.310840+0000","last_change":"2024-04-23T10:24:06.575259+0000","last_active":"2024-04-23T10:26:11.310840+0000","last_peered":"2024-04-23T10:26:11.310840+0000","last_clean":"2024-04-23T10:26:11.310840+0000","last_became_active":"2024-04-23T10:24:06.574920+0000","last_became_peered":"2024-04-23T10:24:06.574920+0000","last_unstale":"2024-04-23T10:26:11.310840+0000","last_undegraded":"2024-04-23T10:26:11.310840+0000","last_fullsized":"2024-04-23T10:26:11.310840+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:52:14.558458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.116672+0000","last_change":"2024-04-23T10:24:07.043238+0000","last_active":"2024-04-23T10:26:11.116672+0000","last_peered":"2024-04-23T10:26:11.116672+0000","last_clean":"2024-04-23T10:26:11.116672+0000","last_became_active":"2024-04-23T10:24:07.042110+0000","last_became_peered":"2024-04-23T10:24:07.042110+0000","last_unstale":"2024-04-23T10:26:11.116672+0000","last_undegraded":"2024-04-23T10:26:11.116672+0000","last_fullsized":"2024-04-23T10:26:11.116672+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T12:36:13.474456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.130204+0000","last_change":"2024-04-23T10:24:06.579233+0000","last_active":"2024-04-23T10:26:11.130204+0000","last_peered":"2024-04-23T10:26:11.130204+0000","last_clean":"2024-04-23T10:26:11.130204+0000","last_became_active":"2024-04-23T10:24:06.578849+0000","last_became_peered":"2024-04-23T10:24:06.578849+0000","last_unstale":"2024-04-23T10:26:11.130204+0000","last_undegraded":"2024-04-23T10:26:11.130204+0000","last_fullsized":"2024-04-23T10:26:11.130204+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:34:23.193707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.116824+0000","last_change":"2024-04-23T10:24:07.049840+0000","last_active":"2024-04-23T10:26:11.116824+0000","last_peered":"2024-04-23T10:26:11.116824+0000","last_clean":"2024-04-23T10:26:11.116824+0000","last_became_active":"2024-04-23T10:24:07.049493+0000","last_became_peered":"2024-04-23T10:24:07.049493+0000","last_unstale":"2024-04-23T10:26:11.116824+0000","last_undegraded":"2024-04-23T10:26:11.116824+0000","last_fullsized":"2024-04-23T10:26:11.116824+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:03:22.606800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.355883+0000","last_change":"2024-04-23T10:24:07.052332+0000","last_active":"2024-04-23T10:26:11.355883+0000","last_peered":"2024-04-23T10:26:11.355883+0000","last_clean":"2024-04-23T10:26:11.355883+0000","last_became_active":"2024-04-23T10:24:07.051956+0000","last_became_peered":"2024-04-23T10:24:07.051956+0000","last_unstale":"2024-04-23T10:26:11.355883+0000","last_undegraded":"2024-04-23T10:26:11.355883+0000","last_fullsized":"2024-04-23T10:26:11.355883+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:56:43.606916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.116207+0000","last_change":"2024-04-23T10:24:07.041897+0000","last_active":"2024-04-23T10:26:11.116207+0000","last_peered":"2024-04-23T10:26:11.116207+0000","last_clean":"2024-04-23T10:26:11.116207+0000","last_became_active":"2024-04-23T10:24:07.041546+0000","last_became_peered":"2024-04-23T10:24:07.041546+0000","last_unstale":"2024-04-23T10:26:11.116207+0000","last_undegraded":"2024-04-23T10:26:11.116207+0000","last_fullsized":"2024-04-23T10:26:11.116207+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T11:31:23.211045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.116192+0000","last_change":"2024-04-23T10:24:07.040764+0000","last_active":"2024-04-23T10:26:11.116192+0000","last_peered":"2024-04-23T10:26:11.116192+0000","last_clean":"2024-04-23T10:26:11.116192+0000","last_became_active":"2024-04-23T10:24:07.039637+0000","last_became_peered":"2024-04-23T10:24:07.039637+0000","last_unstale":"2024-04-23T10:26:11.116192+0000","last_undegraded":"2024-04-23T10:26:11.116192+0000","last_fullsized":"2024-04-23T10:26:11.116192+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:50:33.298770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.350168+0000","last_change":"2024-04-23T10:24:07.039491+0000","last_active":"2024-04-23T10:26:11.350168+0000","last_peered":"2024-04-23T10:26:11.350168+0000","last_clean":"2024-04-23T10:26:11.350168+0000","last_became_active":"2024-04-23T10:24:07.039128+0000","last_became_peered":"2024-04-23T10:24:07.039128+0000","last_unstale":"2024-04-23T10:26:11.350168+0000","last_undegraded":"2024-04-23T10:26:11.350168+0000","last_fullsized":"2024-04-23T10:26:11.350168+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T12:30:40.326031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.130013+0000","last_change":"2024-04-23T10:25:00.671059+0000","last_active":"2024-04-23T10:26:11.130013+0000","last_peered":"2024-04-23T10:26:11.130013+0000","last_clean":"2024-04-23T10:26:11.130013+0000","last_became_active":"2024-04-23T10:25:00.670686+0000","last_became_peered":"2024-04-23T10:25:00.670686+0000","last_unstale":"2024-04-23T10:26:11.130013+0000","last_undegraded":"2024-04-23T10:26:11.130013+0000","last_fullsized":"2024-04-23T10:26:11.130013+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:04:39.991647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.355865+0000","last_change":"2024-04-23T10:24:07.052395+0000","last_active":"2024-04-23T10:26:11.355865+0000","last_peered":"2024-04-23T10:26:11.355865+0000","last_clean":"2024-04-23T10:26:11.355865+0000","last_became_active":"2024-04-23T10:24:07.052023+0000","last_became_peered":"2024-04-23T10:24:07.052023+0000","last_unstale":"2024-04-23T10:26:11.355865+0000","last_undegraded":"2024-04-23T10:26:11.355865+0000","last_fullsized":"2024-04-23T10:26:11.355865+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:25:52.517267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.116425+0000","last_change":"2024-04-23T10:24:07.042739+0000","last_active":"2024-04-23T10:26:11.116425+0000","last_peered":"2024-04-23T10:26:11.116425+0000","last_clean":"2024-04-23T10:26:11.116425+0000","last_became_active":"2024-04-23T10:24:07.042292+0000","last_became_peered":"2024-04-23T10:24:07.042292+0000","last_unstale":"2024-04-23T10:26:11.116425+0000","last_undegraded":"2024-04-23T10:26:11.116425+0000","last_fullsized":"2024-04-23T10:26:11.116425+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:14:23.229032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.132286+0000","last_change":"2024-04-23T10:24:07.042792+0000","last_active":"2024-04-23T10:26:11.132286+0000","last_peered":"2024-04-23T10:26:11.132286+0000","last_clean":"2024-04-23T10:26:11.132286+0000","last_became_active":"2024-04-23T10:24:07.040846+0000","last_became_peered":"2024-04-23T10:24:07.040846+0000","last_unstale":"2024-04-23T10:26:11.132286+0000","last_undegraded":"2024-04-23T10:26:11.132286+0000","last_fullsized":"2024-04-23T10:26:11.132286+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T13:50:24.973286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.127085+0000","last_change":"2024-04-23T10:24:07.040391+0000","last_active":"2024-04-23T10:26:11.127085+0000","last_peered":"2024-04-23T10:26:11.127085+0000","last_clean":"2024-04-23T10:26:11.127085+0000","last_became_active":"2024-04-23T10:24:07.040002+0000","last_became_peered":"2024-04-23T10:24:07.040002+0000","last_unstale":"2024-04-23T10:26:11.127085+0000","last_undegraded":"2024-04-23T10:26:11.127085+0000","last_fullsized":"2024-04-23T10:26:11.127085+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T10:30:13.922315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.131939+0000","last_change":"2024-04-23T10:24:07.040678+0000","last_active":"2024-04-23T10:26:11.131939+0000","last_peered":"2024-04-23T10:26:11.131939+0000","last_clean":"2024-04-23T10:26:11.131939+0000","last_became_active":"2024-04-23T10:24:07.039551+0000","last_became_peered":"2024-04-23T10:24:07.039551+0000","last_unstale":"2024-04-23T10:26:11.131939+0000","last_undegraded":"2024-04-23T10:26:11.131939+0000","last_fullsized":"2024-04-23T10:26:11.131939+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:03:50.467083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.349510+0000","last_change":"2024-04-23T10:24:07.044618+0000","last_active":"2024-04-23T10:26:11.349510+0000","last_peered":"2024-04-23T10:26:11.349510+0000","last_clean":"2024-04-23T10:26:11.349510+0000","last_became_active":"2024-04-23T10:24:07.042803+0000","last_became_peered":"2024-04-23T10:24:07.042803+0000","last_unstale":"2024-04-23T10:26:11.349510+0000","last_undegraded":"2024-04-23T10:26:11.349510+0000","last_fullsized":"2024-04-23T10:26:11.349510+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:37:46.872177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.129943+0000","last_change":"2024-04-23T10:24:07.043724+0000","last_active":"2024-04-23T10:26:11.129943+0000","last_peered":"2024-04-23T10:26:11.129943+0000","last_clean":"2024-04-23T10:26:11.129943+0000","last_became_active":"2024-04-23T10:24:07.043408+0000","last_became_peered":"2024-04-23T10:24:07.043408+0000","last_unstale":"2024-04-23T10:26:11.129943+0000","last_undegraded":"2024-04-23T10:26:11.129943+0000","last_fullsized":"2024-04-23T10:26:11.129943+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:18:46.615093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.355748+0000","last_change":"2024-04-23T10:24:07.043794+0000","last_active":"2024-04-23T10:26:11.355748+0000","last_peered":"2024-04-23T10:26:11.355748+0000","last_clean":"2024-04-23T10:26:11.355748+0000","last_became_active":"2024-04-23T10:24:07.043391+0000","last_became_peered":"2024-04-23T10:24:07.043391+0000","last_unstale":"2024-04-23T10:26:11.355748+0000","last_undegraded":"2024-04-23T10:26:11.355748+0000","last_fullsized":"2024-04-23T10:26:11.355748+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:06:04.584000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":106845,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":602156,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1342,"num_read_kb":785,"num_write":375,"num_write_kb":1469,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1254,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":712704,"data_stored":1806468,"data_compressed":81474,"data_compressed_allocated":110592,"data_compressed_original":1572864,"omap_allocated":0,"internal_metadata":0},"log_size":304,"ondisk_log_size":304,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351014,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":974848,"data_stored":470174,"data_compressed":2902,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612651,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28144,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717008,"statfs":{"total":95995035648,"available":95966216192,"internally_reserved":0,"allocated":1150976,"data_stored":1259979,"data_compressed":46529,"data_compressed_allocated":61440,"data_compressed_original":729088,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776176,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1019904,"data_stored":861815,"data_compressed":22048,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037814,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1003520,"data_stored":860939,"data_compressed":22048,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299451,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1044480,"data_stored":1063341,"data_compressed":38517,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462976,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":978944,"data_stored":469098,"data_compressed":2902,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1044480,"data_stored":537769,"data_compressed":4758,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":999424,"data_stored":470578,"data_compressed":2902,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1064960,"data_stored":538662,"data_compressed":4758,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149776,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1040384,"data_stored":992957,"data_compressed":28204,"data_compressed_allocated":36864,"data_compressed_original":524288,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888138,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":987136,"data_stored":469772,"data_compressed":2902,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724612,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32240,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712912,"statfs":{"total":95995035648,"available":95962021888,"internally_reserved":0,"allocated":1077248,"data_stored":1192914,"data_compressed":44673,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35615,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35615,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35615,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":70292,"data_compressed":1856,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":71185,"data_compressed":1856,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":134952,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":393462,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":202017,"data_compressed":8012,"data_compressed_allocated":12288,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T10:26:16.772 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T10:26:16.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.911+0000 7fd0c85cd640 1 -- 172.21.15.97:0/2840496711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0c01010a0 msgr2=0x7fd0c0101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:16.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.911+0000 7fd0c85cd640 1 --2- 172.21.15.97:0/2840496711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0c01010a0 0x7fd0c0101480 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc009900 tx=0x7fd0bc02a410 comp rx=0 tx=0).stop 2024-04-23T10:26:16.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.911+0000 7fd0c85cd640 1 -- 172.21.15.97:0/2840496711 shutdown_connections 2024-04-23T10:26:16.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.911+0000 7fd0c85cd640 1 --2- 172.21.15.97:0/2840496711 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0c0106170 0x7fd0c010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:16.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.911+0000 7fd0c85cd640 1 --2- 172.21.15.97:0/2840496711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0c0101a50 0x7fd0c0105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:16.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.911+0000 7fd0c85cd640 1 --2- 172.21.15.97:0/2840496711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0c01010a0 0x7fd0c0101480 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:16.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.911+0000 7fd0c85cd640 1 -- 172.21.15.97:0/2840496711 >> 172.21.15.97:0/2840496711 conn(0x7fd0c0077f50 msgr2=0x7fd0c0078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:16.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.911+0000 7fd0c85cd640 1 -- 172.21.15.97:0/2840496711 shutdown_connections 2024-04-23T10:26:16.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.911+0000 7fd0c85cd640 1 -- 172.21.15.97:0/2840496711 wait complete. 2024-04-23T10:26:16.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.912+0000 7fd0c85cd640 1 Processor -- start 2024-04-23T10:26:16.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.912+0000 7fd0c85cd640 1 -- start start 2024-04-23T10:26:16.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.912+0000 7fd0c85cd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0c01010a0 0x7fd0c01995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:16.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.912+0000 7fd0c85cd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0c0106170 0x7fd0c0199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:16.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.912+0000 7fd0c85cd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0c019a0b0 0x7fd0c01a4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:16.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.912+0000 7fd0c85cd640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c0111120 con 0x7fd0c019a0b0 2024-04-23T10:26:16.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.912+0000 7fd0c85cd640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c0110fa0 con 0x7fd0c01010a0 2024-04-23T10:26:16.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.912+0000 7fd0c85cd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c01112a0 con 0x7fd0c0106170 2024-04-23T10:26:16.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.912+0000 7fd0c6b6a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0c019a0b0 0x7fd0c01a4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:16.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.912+0000 7fd0c6b6a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0c019a0b0 0x7fd0c01a4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46948/0 (socket says 172.21.15.97:46948) 2024-04-23T10:26:16.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.912+0000 7fd0c6b6a640 1 -- 172.21.15.97:0/2094774203 learned_addr learned my addr 172.21.15.97:0/2094774203 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:16.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.912+0000 7fd0c6369640 1 --2- 172.21.15.97:0/2094774203 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0c01010a0 0x7fd0c01995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:16.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.912+0000 7fd0c5b68640 1 --2- 172.21.15.97:0/2094774203 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0c0106170 0x7fd0c0199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:16.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.912+0000 7fd0c6b6a640 1 -- 172.21.15.97:0/2094774203 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0c0106170 msgr2=0x7fd0c0199b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:16.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.912+0000 7fd0c6b6a640 1 --2- 172.21.15.97:0/2094774203 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0c0106170 0x7fd0c0199b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:16.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.912+0000 7fd0c6b6a640 1 -- 172.21.15.97:0/2094774203 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0c01010a0 msgr2=0x7fd0c01995e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:16.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.912+0000 7fd0c6b6a640 1 --2- 172.21.15.97:0/2094774203 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0c01010a0 0x7fd0c01995e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:16.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.912+0000 7fd0c6b6a640 1 -- 172.21.15.97:0/2094774203 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0c01a4e30 con 0x7fd0c019a0b0 2024-04-23T10:26:16.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.912+0000 7fd0c6369640 1 --2- 172.21.15.97:0/2094774203 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0c01010a0 0x7fd0c01995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T10:26:16.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.913+0000 7fd0c6b6a640 1 --2- 172.21.15.97:0/2094774203 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0c019a0b0 0x7fd0c01a4660 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fd0b800eb10 tx=0x7fd0b800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:16.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.913+0000 7fd0b77fe640 1 -- 172.21.15.97:0/2094774203 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b8004b20 con 0x7fd0c019a0b0 2024-04-23T10:26:16.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.913+0000 7fd0b77fe640 1 -- 172.21.15.97:0/2094774203 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0b8004cc0 con 0x7fd0c019a0b0 2024-04-23T10:26:16.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.913+0000 7fd0c85cd640 1 -- 172.21.15.97:0/2094774203 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0c01a5120 con 0x7fd0c019a0b0 2024-04-23T10:26:16.915 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.913+0000 7fd0b77fe640 1 -- 172.21.15.97:0/2094774203 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b8010670 con 0x7fd0c019a0b0 2024-04-23T10:26:16.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.913+0000 7fd0c85cd640 1 -- 172.21.15.97:0/2094774203 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0c01a5620 con 0x7fd0c019a0b0 2024-04-23T10:26:16.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.914+0000 7fd0c85cd640 1 -- 172.21.15.97:0/2094774203 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0c0068d70 con 0x7fd0c019a0b0 2024-04-23T10:26:16.916 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.915+0000 7fd0b77fe640 1 -- 172.21.15.97:0/2094774203 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fd0b8005020 con 0x7fd0c019a0b0 2024-04-23T10:26:16.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.915+0000 7fd0b77fe640 1 --2- 172.21.15.97:0/2094774203 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fd0ac0a3e40 0x7fd0ac0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:16.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.916+0000 7fd0c6369640 1 --2- 172.21.15.97:0/2094774203 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fd0ac0a3e40 0x7fd0ac0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:16.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.916+0000 7fd0b77fe640 1 -- 172.21.15.97:0/2094774203 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fd0b80d0f60 con 0x7fd0c019a0b0 2024-04-23T10:26:16.917 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.916+0000 7fd0c6369640 1 --2- 172.21.15.97:0/2094774203 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fd0ac0a3e40 0x7fd0ac0a6300 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc005ab0 tx=0x7fd0bc005a20 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:16.919 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:16.920+0000 7fd0b77fe640 1 -- 172.21.15.97:0/2094774203 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd0b809d400 con 0x7fd0c019a0b0 2024-04-23T10:26:17.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.095+0000 7fd0c85cd640 1 -- 172.21.15.97:0/2094774203 --> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd0c0101a50 con 0x7fd0ac0a3e40 2024-04-23T10:26:17.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.102+0000 7fd0b77fe640 1 -- 172.21.15.97:0/2094774203 <== mgr.15378 v2:172.21.15.97:6800/3567268502 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258076 (secure 0 0 0) 0x7fd0c0101a50 con 0x7fd0ac0a3e40 2024-04-23T10:26:17.104 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:26:17.105 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-04-23T10:26:17.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.105+0000 7fd0c85cd640 1 -- 172.21.15.97:0/2094774203 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fd0ac0a3e40 msgr2=0x7fd0ac0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:17.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.105+0000 7fd0c85cd640 1 --2- 172.21.15.97:0/2094774203 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fd0ac0a3e40 0x7fd0ac0a6300 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc005ab0 tx=0x7fd0bc005a20 comp rx=0 tx=0).stop 2024-04-23T10:26:17.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.105+0000 7fd0c85cd640 1 -- 172.21.15.97:0/2094774203 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0c019a0b0 msgr2=0x7fd0c01a4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:17.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.105+0000 7fd0c85cd640 1 --2- 172.21.15.97:0/2094774203 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0c019a0b0 0x7fd0c01a4660 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fd0b800eb10 tx=0x7fd0b800c2d0 comp rx=0 tx=0).stop 2024-04-23T10:26:17.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.105+0000 7fd0c85cd640 1 -- 172.21.15.97:0/2094774203 shutdown_connections 2024-04-23T10:26:17.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.105+0000 7fd0c85cd640 1 --2- 172.21.15.97:0/2094774203 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fd0ac0a3e40 0x7fd0ac0a6300 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:17.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.105+0000 7fd0c85cd640 1 --2- 172.21.15.97:0/2094774203 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0c019a0b0 0x7fd0c01a4660 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:17.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.105+0000 7fd0c85cd640 1 --2- 172.21.15.97:0/2094774203 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0c0106170 0x7fd0c0199b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:17.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.105+0000 7fd0c85cd640 1 --2- 172.21.15.97:0/2094774203 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0c01010a0 0x7fd0c01995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:17.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.105+0000 7fd0c85cd640 1 -- 172.21.15.97:0/2094774203 >> 172.21.15.97:0/2094774203 conn(0x7fd0c0077f50 msgr2=0x7fd0c00ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:17.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.106+0000 7fd0c85cd640 1 -- 172.21.15.97:0/2094774203 shutdown_connections 2024-04-23T10:26:17.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.106+0000 7fd0c85cd640 1 -- 172.21.15.97:0/2094774203 wait complete. 2024-04-23T10:26:17.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:16 smithi097 ceph-mon[29197]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:26:17.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:16 smithi097 ceph-mon[29197]: Updating smithi183:/etc/ceph/ceph.client.admin.keyring 2024-04-23T10:26:17.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:16 smithi097 ceph-mon[29197]: Updating smithi097:/etc/ceph/ceph.client.admin.keyring 2024-04-23T10:26:17.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:16 smithi097 ceph-mon[29197]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-04-23T10:26:17.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:16 smithi097 ceph-mon[29197]: Updating smithi097:/var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T10:26:17.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:16 smithi097 ceph-mon[29197]: Updating smithi183:/var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T10:26:17.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:16 smithi097 ceph-mon[29197]: Updating smithi196:/var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T10:26:17.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:16 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:26:17.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:16 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:26:17.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:16 smithi097 ceph-mon[29197]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-23T10:26:17.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:16 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:26:17.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:16 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/2705289938' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T10:26:17.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:16 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/1917864722' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T10:26:17.115 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-04-23T10:26:15.131026+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1192758,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1478,"num_read_kb":904,"num_write":558,"num_write_kb":4409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1181990,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":943,"ondisk_log_size":943,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361296,"kb_used_data":12096,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580528,"statfs":{"total":1151940427776,"available":1151570460672,"internally_reserved":0,"allocated":12386304,"data_stored":9187998,"data_compressed":223143,"data_compressed_allocated":294912,"data_compressed_original":3440640,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.991977"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.356030+0000","last_change":"2024-04-23T10:24:07.040397+0000","last_active":"2024-04-23T10:26:11.356030+0000","last_peered":"2024-04-23T10:26:11.356030+0000","last_clean":"2024-04-23T10:26:11.356030+0000","last_became_active":"2024-04-23T10:24:07.040007+0000","last_became_peered":"2024-04-23T10:24:07.040007+0000","last_unstale":"2024-04-23T10:26:11.356030+0000","last_undegraded":"2024-04-23T10:26:11.356030+0000","last_fullsized":"2024-04-23T10:26:11.356030+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T13:50:24.533115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.350839+0000","last_change":"2024-04-23T10:24:07.044672+0000","last_active":"2024-04-23T10:26:11.350839+0000","last_peered":"2024-04-23T10:26:11.350839+0000","last_clean":"2024-04-23T10:26:11.350839+0000","last_became_active":"2024-04-23T10:24:07.044184+0000","last_became_peered":"2024-04-23T10:24:07.044184+0000","last_unstale":"2024-04-23T10:26:11.350839+0000","last_undegraded":"2024-04-23T10:26:11.350839+0000","last_fullsized":"2024-04-23T10:26:11.350839+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:07:16.729205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.350394+0000","last_change":"2024-04-23T10:24:07.041959+0000","last_active":"2024-04-23T10:26:11.350394+0000","last_peered":"2024-04-23T10:26:11.350394+0000","last_clean":"2024-04-23T10:26:11.350394+0000","last_became_active":"2024-04-23T10:24:07.041661+0000","last_became_peered":"2024-04-23T10:24:07.041661+0000","last_unstale":"2024-04-23T10:26:11.350394+0000","last_undegraded":"2024-04-23T10:26:11.350394+0000","last_fullsized":"2024-04-23T10:26:11.350394+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:23:14.694183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.131765+0000","last_change":"2024-04-23T10:24:07.048334+0000","last_active":"2024-04-23T10:26:11.131765+0000","last_peered":"2024-04-23T10:26:11.131765+0000","last_clean":"2024-04-23T10:26:11.131765+0000","last_became_active":"2024-04-23T10:24:07.047506+0000","last_became_peered":"2024-04-23T10:24:07.047506+0000","last_unstale":"2024-04-23T10:26:11.131765+0000","last_undegraded":"2024-04-23T10:26:11.131765+0000","last_fullsized":"2024-04-23T10:26:11.131765+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:27:27.674287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.157602+0000","last_change":"2024-04-23T10:24:07.035531+0000","last_active":"2024-04-23T10:26:11.157602+0000","last_peered":"2024-04-23T10:26:11.157602+0000","last_clean":"2024-04-23T10:26:11.157602+0000","last_became_active":"2024-04-23T10:24:07.035167+0000","last_became_peered":"2024-04-23T10:24:07.035167+0000","last_unstale":"2024-04-23T10:26:11.157602+0000","last_undegraded":"2024-04-23T10:26:11.157602+0000","last_fullsized":"2024-04-23T10:26:11.157602+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:39:17.198650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.124553+0000","last_change":"2024-04-23T10:24:07.040593+0000","last_active":"2024-04-23T10:26:11.124553+0000","last_peered":"2024-04-23T10:26:11.124553+0000","last_clean":"2024-04-23T10:26:11.124553+0000","last_became_active":"2024-04-23T10:24:07.040255+0000","last_became_peered":"2024-04-23T10:24:07.040255+0000","last_unstale":"2024-04-23T10:26:11.124553+0000","last_undegraded":"2024-04-23T10:26:11.124553+0000","last_fullsized":"2024-04-23T10:26:11.124553+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:05:52.663566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.116448+0000","last_change":"2024-04-23T10:24:07.051688+0000","last_active":"2024-04-23T10:26:11.116448+0000","last_peered":"2024-04-23T10:26:11.116448+0000","last_clean":"2024-04-23T10:26:11.116448+0000","last_became_active":"2024-04-23T10:24:07.050265+0000","last_became_peered":"2024-04-23T10:24:07.050265+0000","last_unstale":"2024-04-23T10:26:11.116448+0000","last_undegraded":"2024-04-23T10:26:11.116448+0000","last_fullsized":"2024-04-23T10:26:11.116448+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T13:13:36.126183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.131842+0000","last_change":"2024-04-23T10:24:07.045937+0000","last_active":"2024-04-23T10:26:11.131842+0000","last_peered":"2024-04-23T10:26:11.131842+0000","last_clean":"2024-04-23T10:26:11.131842+0000","last_became_active":"2024-04-23T10:24:07.045616+0000","last_became_peered":"2024-04-23T10:24:07.045616+0000","last_unstale":"2024-04-23T10:26:11.131842+0000","last_undegraded":"2024-04-23T10:26:11.131842+0000","last_fullsized":"2024-04-23T10:26:11.131842+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T10:45:27.250031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.349840+0000","last_change":"2024-04-23T10:24:07.039294+0000","last_active":"2024-04-23T10:26:11.349840+0000","last_peered":"2024-04-23T10:26:11.349840+0000","last_clean":"2024-04-23T10:26:11.349840+0000","last_became_active":"2024-04-23T10:24:07.038959+0000","last_became_peered":"2024-04-23T10:24:07.038959+0000","last_unstale":"2024-04-23T10:26:11.349840+0000","last_undegraded":"2024-04-23T10:26:11.349840+0000","last_fullsized":"2024-04-23T10:26:11.349840+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:53:41.345062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.124874+0000","last_change":"2024-04-23T10:24:07.044177+0000","last_active":"2024-04-23T10:26:11.124874+0000","last_peered":"2024-04-23T10:26:11.124874+0000","last_clean":"2024-04-23T10:26:11.124874+0000","last_became_active":"2024-04-23T10:24:07.043821+0000","last_became_peered":"2024-04-23T10:24:07.043821+0000","last_unstale":"2024-04-23T10:26:11.124874+0000","last_undegraded":"2024-04-23T10:26:11.124874+0000","last_fullsized":"2024-04-23T10:26:11.124874+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:09:38.782407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.116882+0000","last_change":"2024-04-23T10:24:07.052775+0000","last_active":"2024-04-23T10:26:11.116882+0000","last_peered":"2024-04-23T10:26:11.116882+0000","last_clean":"2024-04-23T10:26:11.116882+0000","last_became_active":"2024-04-23T10:24:07.051708+0000","last_became_peered":"2024-04-23T10:24:07.051708+0000","last_unstale":"2024-04-23T10:26:11.116882+0000","last_undegraded":"2024-04-23T10:26:11.116882+0000","last_fullsized":"2024-04-23T10:26:11.116882+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:47:55.360467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.131663+0000","last_change":"2024-04-23T10:24:07.048655+0000","last_active":"2024-04-23T10:26:11.131663+0000","last_peered":"2024-04-23T10:26:11.131663+0000","last_clean":"2024-04-23T10:26:11.131663+0000","last_became_active":"2024-04-23T10:24:07.047994+0000","last_became_peered":"2024-04-23T10:24:07.047994+0000","last_unstale":"2024-04-23T10:26:11.131663+0000","last_undegraded":"2024-04-23T10:26:11.131663+0000","last_fullsized":"2024-04-23T10:26:11.131663+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T10:25:51.425871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.129951+0000","last_change":"2024-04-23T10:24:07.036319+0000","last_active":"2024-04-23T10:26:11.129951+0000","last_peered":"2024-04-23T10:26:11.129951+0000","last_clean":"2024-04-23T10:26:11.129951+0000","last_became_active":"2024-04-23T10:24:07.035982+0000","last_became_peered":"2024-04-23T10:24:07.035982+0000","last_unstale":"2024-04-23T10:26:11.129951+0000","last_undegraded":"2024-04-23T10:26:11.129951+0000","last_fullsized":"2024-04-23T10:26:11.129951+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T10:25:46.191118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.349736+0000","last_change":"2024-04-23T10:24:07.033166+0000","last_active":"2024-04-23T10:26:11.349736+0000","last_peered":"2024-04-23T10:26:11.349736+0000","last_clean":"2024-04-23T10:26:11.349736+0000","last_became_active":"2024-04-23T10:24:07.032807+0000","last_became_peered":"2024-04-23T10:24:07.032807+0000","last_unstale":"2024-04-23T10:26:11.349736+0000","last_undegraded":"2024-04-23T10:26:11.349736+0000","last_fullsized":"2024-04-23T10:26:11.349736+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:52:35.654965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.349923+0000","last_change":"2024-04-23T10:24:07.041988+0000","last_active":"2024-04-23T10:26:11.349923+0000","last_peered":"2024-04-23T10:26:11.349923+0000","last_clean":"2024-04-23T10:26:11.349923+0000","last_became_active":"2024-04-23T10:24:07.041638+0000","last_became_peered":"2024-04-23T10:24:07.041638+0000","last_unstale":"2024-04-23T10:26:11.349923+0000","last_undegraded":"2024-04-23T10:26:11.349923+0000","last_fullsized":"2024-04-23T10:26:11.349923+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:17:40.565981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.349144+0000","last_change":"2024-04-23T10:24:07.045111+0000","last_active":"2024-04-23T10:26:11.349144+0000","last_peered":"2024-04-23T10:26:11.349144+0000","last_clean":"2024-04-23T10:26:11.349144+0000","last_became_active":"2024-04-23T10:24:07.044249+0000","last_became_peered":"2024-04-23T10:24:07.044249+0000","last_unstale":"2024-04-23T10:26:11.349144+0000","last_undegraded":"2024-04-23T10:26:11.349144+0000","last_fullsized":"2024-04-23T10:26:11.349144+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:08:36.029636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.356095+0000","last_change":"2024-04-23T10:24:07.040685+0000","last_active":"2024-04-23T10:26:11.356095+0000","last_peered":"2024-04-23T10:26:11.356095+0000","last_clean":"2024-04-23T10:26:11.356095+0000","last_became_active":"2024-04-23T10:24:07.038881+0000","last_became_peered":"2024-04-23T10:24:07.038881+0000","last_unstale":"2024-04-23T10:26:11.356095+0000","last_undegraded":"2024-04-23T10:26:11.356095+0000","last_fullsized":"2024-04-23T10:26:11.356095+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:29:12.489136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.117665+0000","last_change":"2024-04-23T10:24:07.041815+0000","last_active":"2024-04-23T10:26:11.117665+0000","last_peered":"2024-04-23T10:26:11.117665+0000","last_clean":"2024-04-23T10:26:11.117665+0000","last_became_active":"2024-04-23T10:24:07.041495+0000","last_became_peered":"2024-04-23T10:24:07.041495+0000","last_unstale":"2024-04-23T10:26:11.117665+0000","last_undegraded":"2024-04-23T10:26:11.117665+0000","last_fullsized":"2024-04-23T10:26:11.117665+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:29:02.006394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.116347+0000","last_change":"2024-04-23T10:24:07.041722+0000","last_active":"2024-04-23T10:26:11.116347+0000","last_peered":"2024-04-23T10:26:11.116347+0000","last_clean":"2024-04-23T10:26:11.116347+0000","last_became_active":"2024-04-23T10:24:07.039434+0000","last_became_peered":"2024-04-23T10:24:07.039434+0000","last_unstale":"2024-04-23T10:26:11.116347+0000","last_undegraded":"2024-04-23T10:26:11.116347+0000","last_fullsized":"2024-04-23T10:26:11.116347+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:03:44.033094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'14","reported_seq":176,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.825209+0000","last_change":"2024-04-23T10:25:00.739010+0000","last_active":"2024-04-23T10:26:11.825209+0000","last_peered":"2024-04-23T10:26:11.825209+0000","last_clean":"2024-04-23T10:26:11.825209+0000","last_became_active":"2024-04-23T10:25:00.729834+0000","last_became_peered":"2024-04-23T10:25:00.729834+0000","last_unstale":"2024-04-23T10:26:11.825209+0000","last_undegraded":"2024-04-23T10:26:11.825209+0000","last_fullsized":"2024-04-23T10:26:11.825209+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T12:57:45.871560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,4],"acting":[9,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.132261+0000","last_change":"2024-04-23T10:24:07.047192+0000","last_active":"2024-04-23T10:26:11.132261+0000","last_peered":"2024-04-23T10:26:11.132261+0000","last_clean":"2024-04-23T10:26:11.132261+0000","last_became_active":"2024-04-23T10:24:07.046750+0000","last_became_peered":"2024-04-23T10:24:07.046750+0000","last_unstale":"2024-04-23T10:26:11.132261+0000","last_undegraded":"2024-04-23T10:26:11.132261+0000","last_fullsized":"2024-04-23T10:26:11.132261+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:13:09.399425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.132396+0000","last_change":"2024-04-23T10:24:06.577588+0000","last_active":"2024-04-23T10:26:11.132396+0000","last_peered":"2024-04-23T10:26:11.132396+0000","last_clean":"2024-04-23T10:26:11.132396+0000","last_became_active":"2024-04-23T10:24:06.576971+0000","last_became_peered":"2024-04-23T10:24:06.576971+0000","last_unstale":"2024-04-23T10:26:11.132396+0000","last_undegraded":"2024-04-23T10:26:11.132396+0000","last_fullsized":"2024-04-23T10:26:11.132396+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:07:21.577868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.350711+0000","last_change":"2024-04-23T10:24:07.030666+0000","last_active":"2024-04-23T10:26:11.350711+0000","last_peered":"2024-04-23T10:26:11.350711+0000","last_clean":"2024-04-23T10:26:11.350711+0000","last_became_active":"2024-04-23T10:24:07.030342+0000","last_became_peered":"2024-04-23T10:24:07.030342+0000","last_unstale":"2024-04-23T10:26:11.350711+0000","last_undegraded":"2024-04-23T10:26:11.350711+0000","last_fullsized":"2024-04-23T10:26:11.350711+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:09:14.818638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"89'13","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.157995+0000","last_change":"2024-04-23T10:24:06.576042+0000","last_active":"2024-04-23T10:26:11.157995+0000","last_peered":"2024-04-23T10:26:11.157995+0000","last_clean":"2024-04-23T10:26:11.157995+0000","last_became_active":"2024-04-23T10:24:06.574518+0000","last_became_peered":"2024-04-23T10:24:06.574518+0000","last_unstale":"2024-04-23T10:26:11.157995+0000","last_undegraded":"2024-04-23T10:26:11.157995+0000","last_fullsized":"2024-04-23T10:26:11.157995+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T11:56:40.100213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.349417+0000","last_change":"2024-04-23T10:24:07.046591+0000","last_active":"2024-04-23T10:26:11.349417+0000","last_peered":"2024-04-23T10:26:11.349417+0000","last_clean":"2024-04-23T10:26:11.349417+0000","last_became_active":"2024-04-23T10:24:07.046208+0000","last_became_peered":"2024-04-23T10:24:07.046208+0000","last_unstale":"2024-04-23T10:26:11.349417+0000","last_undegraded":"2024-04-23T10:26:11.349417+0000","last_fullsized":"2024-04-23T10:26:11.349417+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:27:37.711329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.355666+0000","last_change":"2024-04-23T10:25:00.676255+0000","last_active":"2024-04-23T10:26:11.355666+0000","last_peered":"2024-04-23T10:26:11.355666+0000","last_clean":"2024-04-23T10:26:11.355666+0000","last_became_active":"2024-04-23T10:25:00.670146+0000","last_became_peered":"2024-04-23T10:25:00.670146+0000","last_unstale":"2024-04-23T10:26:11.355666+0000","last_undegraded":"2024-04-23T10:26:11.355666+0000","last_fullsized":"2024-04-23T10:26:11.355666+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:34:54.217317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.127034+0000","last_change":"2024-04-23T10:24:07.047926+0000","last_active":"2024-04-23T10:26:11.127034+0000","last_peered":"2024-04-23T10:26:11.127034+0000","last_clean":"2024-04-23T10:26:11.127034+0000","last_became_active":"2024-04-23T10:24:07.047273+0000","last_became_peered":"2024-04-23T10:24:07.047273+0000","last_unstale":"2024-04-23T10:26:11.127034+0000","last_undegraded":"2024-04-23T10:26:11.127034+0000","last_fullsized":"2024-04-23T10:26:11.127034+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:37:25.068034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'7","reported_seq":154,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.127172+0000","last_change":"2024-04-23T10:24:06.574733+0000","last_active":"2024-04-23T10:26:11.127172+0000","last_peered":"2024-04-23T10:26:11.127172+0000","last_clean":"2024-04-23T10:26:11.127172+0000","last_became_active":"2024-04-23T10:24:06.574276+0000","last_became_peered":"2024-04-23T10:24:06.574276+0000","last_unstale":"2024-04-23T10:26:11.127172+0000","last_undegraded":"2024-04-23T10:26:11.127172+0000","last_fullsized":"2024-04-23T10:26:11.127172+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:27:57.080635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.157562+0000","last_change":"2024-04-23T10:24:07.040110+0000","last_active":"2024-04-23T10:26:11.157562+0000","last_peered":"2024-04-23T10:26:11.157562+0000","last_clean":"2024-04-23T10:26:11.157562+0000","last_became_active":"2024-04-23T10:24:07.039769+0000","last_became_peered":"2024-04-23T10:24:07.039769+0000","last_unstale":"2024-04-23T10:26:11.157562+0000","last_undegraded":"2024-04-23T10:26:11.157562+0000","last_fullsized":"2024-04-23T10:26:11.157562+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:10:07.884895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.124478+0000","last_change":"2024-04-23T10:24:06.567448+0000","last_active":"2024-04-23T10:26:11.124478+0000","last_peered":"2024-04-23T10:26:11.124478+0000","last_clean":"2024-04-23T10:26:11.124478+0000","last_became_active":"2024-04-23T10:24:06.566958+0000","last_became_peered":"2024-04-23T10:24:06.566958+0000","last_unstale":"2024-04-23T10:26:11.124478+0000","last_undegraded":"2024-04-23T10:26:11.124478+0000","last_fullsized":"2024-04-23T10:26:11.124478+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:45:52.863596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.126797+0000","last_change":"2024-04-23T10:24:07.040754+0000","last_active":"2024-04-23T10:26:11.126797+0000","last_peered":"2024-04-23T10:26:11.126797+0000","last_clean":"2024-04-23T10:26:11.126797+0000","last_became_active":"2024-04-23T10:24:07.040397+0000","last_became_peered":"2024-04-23T10:24:07.040397+0000","last_unstale":"2024-04-23T10:26:11.126797+0000","last_undegraded":"2024-04-23T10:26:11.126797+0000","last_fullsized":"2024-04-23T10:26:11.126797+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T11:01:03.626506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.129719+0000","last_change":"2024-04-23T10:24:06.568592+0000","last_active":"2024-04-23T10:26:11.129719+0000","last_peered":"2024-04-23T10:26:11.129719+0000","last_clean":"2024-04-23T10:26:11.129719+0000","last_became_active":"2024-04-23T10:24:06.568192+0000","last_became_peered":"2024-04-23T10:24:06.568192+0000","last_unstale":"2024-04-23T10:26:11.129719+0000","last_undegraded":"2024-04-23T10:26:11.129719+0000","last_fullsized":"2024-04-23T10:26:11.129719+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:14:42.589596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.131909+0000","last_change":"2024-04-23T10:24:07.037058+0000","last_active":"2024-04-23T10:26:11.131909+0000","last_peered":"2024-04-23T10:26:11.131909+0000","last_clean":"2024-04-23T10:26:11.131909+0000","last_became_active":"2024-04-23T10:24:07.036659+0000","last_became_peered":"2024-04-23T10:24:07.036659+0000","last_unstale":"2024-04-23T10:26:11.131909+0000","last_undegraded":"2024-04-23T10:26:11.131909+0000","last_fullsized":"2024-04-23T10:26:11.131909+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T12:53:49.298603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.127061+0000","last_change":"2024-04-23T10:24:06.576900+0000","last_active":"2024-04-23T10:26:11.127061+0000","last_peered":"2024-04-23T10:26:11.127061+0000","last_clean":"2024-04-23T10:26:11.127061+0000","last_became_active":"2024-04-23T10:24:06.575627+0000","last_became_peered":"2024-04-23T10:24:06.575627+0000","last_unstale":"2024-04-23T10:26:11.127061+0000","last_undegraded":"2024-04-23T10:26:11.127061+0000","last_fullsized":"2024-04-23T10:26:11.127061+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:16:07.973841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.117469+0000","last_change":"2024-04-23T10:24:07.035294+0000","last_active":"2024-04-23T10:26:11.117469+0000","last_peered":"2024-04-23T10:26:11.117469+0000","last_clean":"2024-04-23T10:26:11.117469+0000","last_became_active":"2024-04-23T10:24:07.035001+0000","last_became_peered":"2024-04-23T10:24:07.035001+0000","last_unstale":"2024-04-23T10:26:11.117469+0000","last_undegraded":"2024-04-23T10:26:11.117469+0000","last_fullsized":"2024-04-23T10:26:11.117469+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T13:12:47.013132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"89'7","reported_seq":269,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.355330+0000","last_change":"2024-04-23T10:24:06.573637+0000","last_active":"2024-04-23T10:26:11.355330+0000","last_peered":"2024-04-23T10:26:11.355330+0000","last_clean":"2024-04-23T10:26:11.355330+0000","last_became_active":"2024-04-23T10:24:06.573236+0000","last_became_peered":"2024-04-23T10:24:06.573236+0000","last_unstale":"2024-04-23T10:26:11.355330+0000","last_undegraded":"2024-04-23T10:26:11.355330+0000","last_fullsized":"2024-04-23T10:26:11.355330+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T12:48:06.740181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":241,"num_read_kb":241,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.349816+0000","last_change":"2024-04-23T10:24:07.037457+0000","last_active":"2024-04-23T10:26:11.349816+0000","last_peered":"2024-04-23T10:26:11.349816+0000","last_clean":"2024-04-23T10:26:11.349816+0000","last_became_active":"2024-04-23T10:24:07.036456+0000","last_became_peered":"2024-04-23T10:24:07.036456+0000","last_unstale":"2024-04-23T10:26:11.349816+0000","last_undegraded":"2024-04-23T10:26:11.349816+0000","last_fullsized":"2024-04-23T10:26:11.349816+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:42:57.750435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.116889+0000","last_change":"2024-04-23T10:25:00.741712+0000","last_active":"2024-04-23T10:26:11.116889+0000","last_peered":"2024-04-23T10:26:11.116889+0000","last_clean":"2024-04-23T10:26:11.116889+0000","last_became_active":"2024-04-23T10:25:00.730130+0000","last_became_peered":"2024-04-23T10:25:00.730130+0000","last_unstale":"2024-04-23T10:26:11.116889+0000","last_undegraded":"2024-04-23T10:26:11.116889+0000","last_fullsized":"2024-04-23T10:26:11.116889+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:08:14.409376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.350621+0000","last_change":"2024-04-23T10:24:07.040317+0000","last_active":"2024-04-23T10:26:11.350621+0000","last_peered":"2024-04-23T10:26:11.350621+0000","last_clean":"2024-04-23T10:26:11.350621+0000","last_became_active":"2024-04-23T10:24:07.039998+0000","last_became_peered":"2024-04-23T10:24:07.039998+0000","last_unstale":"2024-04-23T10:26:11.350621+0000","last_undegraded":"2024-04-23T10:26:11.350621+0000","last_fullsized":"2024-04-23T10:26:11.350621+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:50:30.412127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.127037+0000","last_change":"2024-04-23T10:24:06.570019+0000","last_active":"2024-04-23T10:26:11.127037+0000","last_peered":"2024-04-23T10:26:11.127037+0000","last_clean":"2024-04-23T10:26:11.127037+0000","last_became_active":"2024-04-23T10:24:06.569676+0000","last_became_peered":"2024-04-23T10:24:06.569676+0000","last_unstale":"2024-04-23T10:26:11.127037+0000","last_undegraded":"2024-04-23T10:26:11.127037+0000","last_fullsized":"2024-04-23T10:26:11.127037+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:31:46.994787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.349539+0000","last_change":"2024-04-23T10:24:07.040183+0000","last_active":"2024-04-23T10:26:11.349539+0000","last_peered":"2024-04-23T10:26:11.349539+0000","last_clean":"2024-04-23T10:26:11.349539+0000","last_became_active":"2024-04-23T10:24:07.039866+0000","last_became_peered":"2024-04-23T10:24:07.039866+0000","last_unstale":"2024-04-23T10:26:11.349539+0000","last_undegraded":"2024-04-23T10:26:11.349539+0000","last_fullsized":"2024-04-23T10:26:11.349539+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:33:04.411765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"89'38","reported_seq":315,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:12.128873+0000","last_change":"2024-04-23T10:24:06.568883+0000","last_active":"2024-04-23T10:26:12.128873+0000","last_peered":"2024-04-23T10:26:12.128873+0000","last_clean":"2024-04-23T10:26:12.128873+0000","last_became_active":"2024-04-23T10:24:06.568517+0000","last_became_peered":"2024-04-23T10:24:06.568517+0000","last_unstale":"2024-04-23T10:26:12.128873+0000","last_undegraded":"2024-04-23T10:26:12.128873+0000","last_fullsized":"2024-04-23T10:26:12.128873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:51:42.440346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67758,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":256,"num_read_kb":194,"num_write":40,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.157789+0000","last_change":"2024-04-23T10:24:07.046839+0000","last_active":"2024-04-23T10:26:11.157789+0000","last_peered":"2024-04-23T10:26:11.157789+0000","last_clean":"2024-04-23T10:26:11.157789+0000","last_became_active":"2024-04-23T10:24:07.046512+0000","last_became_peered":"2024-04-23T10:24:07.046512+0000","last_unstale":"2024-04-23T10:26:11.157789+0000","last_undegraded":"2024-04-23T10:26:11.157789+0000","last_fullsized":"2024-04-23T10:26:11.157789+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T12:42:11.822100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.116303+0000","last_change":"2024-04-23T10:24:06.578087+0000","last_active":"2024-04-23T10:26:11.116303+0000","last_peered":"2024-04-23T10:26:11.116303+0000","last_clean":"2024-04-23T10:26:11.116303+0000","last_became_active":"2024-04-23T10:24:06.577666+0000","last_became_peered":"2024-04-23T10:24:06.577666+0000","last_unstale":"2024-04-23T10:26:11.116303+0000","last_undegraded":"2024-04-23T10:26:11.116303+0000","last_fullsized":"2024-04-23T10:26:11.116303+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:55:15.053303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.129816+0000","last_change":"2024-04-23T10:24:07.043672+0000","last_active":"2024-04-23T10:26:11.129816+0000","last_peered":"2024-04-23T10:26:11.129816+0000","last_clean":"2024-04-23T10:26:11.129816+0000","last_became_active":"2024-04-23T10:24:07.043346+0000","last_became_peered":"2024-04-23T10:24:07.043346+0000","last_unstale":"2024-04-23T10:26:11.129816+0000","last_undegraded":"2024-04-23T10:26:11.129816+0000","last_fullsized":"2024-04-23T10:26:11.129816+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T13:21:56.751028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.126748+0000","last_change":"2024-04-23T10:24:06.576607+0000","last_active":"2024-04-23T10:26:11.126748+0000","last_peered":"2024-04-23T10:26:11.126748+0000","last_clean":"2024-04-23T10:26:11.126748+0000","last_became_active":"2024-04-23T10:24:06.576213+0000","last_became_peered":"2024-04-23T10:24:06.576213+0000","last_unstale":"2024-04-23T10:26:11.126748+0000","last_undegraded":"2024-04-23T10:26:11.126748+0000","last_fullsized":"2024-04-23T10:26:11.126748+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:09:14.918936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.116719+0000","last_change":"2024-04-23T10:24:07.052641+0000","last_active":"2024-04-23T10:26:11.116719+0000","last_peered":"2024-04-23T10:26:11.116719+0000","last_clean":"2024-04-23T10:26:11.116719+0000","last_became_active":"2024-04-23T10:24:07.052314+0000","last_became_peered":"2024-04-23T10:24:07.052314+0000","last_unstale":"2024-04-23T10:26:11.116719+0000","last_undegraded":"2024-04-23T10:26:11.116719+0000","last_fullsized":"2024-04-23T10:26:11.116719+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T10:29:18.098800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'8","reported_seq":155,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.349410+0000","last_change":"2024-04-23T10:24:06.574570+0000","last_active":"2024-04-23T10:26:11.349410+0000","last_peered":"2024-04-23T10:26:11.349410+0000","last_clean":"2024-04-23T10:26:11.349410+0000","last_became_active":"2024-04-23T10:24:06.574166+0000","last_became_peered":"2024-04-23T10:24:06.574166+0000","last_unstale":"2024-04-23T10:26:11.349410+0000","last_undegraded":"2024-04-23T10:26:11.349410+0000","last_fullsized":"2024-04-23T10:26:11.349410+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:28:31.619100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":127,"num_read_kb":5,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.116415+0000","last_change":"2024-04-23T10:24:07.052825+0000","last_active":"2024-04-23T10:26:11.116415+0000","last_peered":"2024-04-23T10:26:11.116415+0000","last_clean":"2024-04-23T10:26:11.116415+0000","last_became_active":"2024-04-23T10:24:07.052457+0000","last_became_peered":"2024-04-23T10:24:07.052457+0000","last_unstale":"2024-04-23T10:26:11.116415+0000","last_undegraded":"2024-04-23T10:26:11.116415+0000","last_fullsized":"2024-04-23T10:26:11.116415+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:47:09.545374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.157540+0000","last_change":"2024-04-23T10:24:06.572168+0000","last_active":"2024-04-23T10:26:11.157540+0000","last_peered":"2024-04-23T10:26:11.157540+0000","last_clean":"2024-04-23T10:26:11.157540+0000","last_became_active":"2024-04-23T10:24:06.571805+0000","last_became_peered":"2024-04-23T10:24:06.571805+0000","last_unstale":"2024-04-23T10:26:11.157540+0000","last_undegraded":"2024-04-23T10:26:11.157540+0000","last_fullsized":"2024-04-23T10:26:11.157540+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:29:56.817879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.126680+0000","last_change":"2024-04-23T10:24:07.048474+0000","last_active":"2024-04-23T10:26:11.126680+0000","last_peered":"2024-04-23T10:26:11.126680+0000","last_clean":"2024-04-23T10:26:11.126680+0000","last_became_active":"2024-04-23T10:24:07.047893+0000","last_became_peered":"2024-04-23T10:24:07.047893+0000","last_unstale":"2024-04-23T10:26:11.126680+0000","last_undegraded":"2024-04-23T10:26:11.126680+0000","last_fullsized":"2024-04-23T10:26:11.126680+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T12:59:46.545569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.157883+0000","last_change":"2024-04-23T10:24:06.570974+0000","last_active":"2024-04-23T10:26:11.157883+0000","last_peered":"2024-04-23T10:26:11.157883+0000","last_clean":"2024-04-23T10:26:11.157883+0000","last_became_active":"2024-04-23T10:24:06.570603+0000","last_became_peered":"2024-04-23T10:24:06.570603+0000","last_unstale":"2024-04-23T10:26:11.157883+0000","last_undegraded":"2024-04-23T10:26:11.157883+0000","last_fullsized":"2024-04-23T10:26:11.157883+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T12:46:33.518337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.355892+0000","last_change":"2024-04-23T10:24:07.050986+0000","last_active":"2024-04-23T10:26:11.355892+0000","last_peered":"2024-04-23T10:26:11.355892+0000","last_clean":"2024-04-23T10:26:11.355892+0000","last_became_active":"2024-04-23T10:24:07.050617+0000","last_became_peered":"2024-04-23T10:24:07.050617+0000","last_unstale":"2024-04-23T10:26:11.355892+0000","last_undegraded":"2024-04-23T10:26:11.355892+0000","last_fullsized":"2024-04-23T10:26:11.355892+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:13:25.355117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.117580+0000","last_change":"2024-04-23T10:24:06.579138+0000","last_active":"2024-04-23T10:26:11.117580+0000","last_peered":"2024-04-23T10:26:11.117580+0000","last_clean":"2024-04-23T10:26:11.117580+0000","last_became_active":"2024-04-23T10:24:06.578716+0000","last_became_peered":"2024-04-23T10:24:06.578716+0000","last_unstale":"2024-04-23T10:26:11.117580+0000","last_undegraded":"2024-04-23T10:26:11.117580+0000","last_fullsized":"2024-04-23T10:26:11.117580+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:06:59.797802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.124849+0000","last_change":"2024-04-23T10:24:07.039019+0000","last_active":"2024-04-23T10:26:11.124849+0000","last_peered":"2024-04-23T10:26:11.124849+0000","last_clean":"2024-04-23T10:26:11.124849+0000","last_became_active":"2024-04-23T10:24:07.038701+0000","last_became_peered":"2024-04-23T10:24:07.038701+0000","last_unstale":"2024-04-23T10:26:11.124849+0000","last_undegraded":"2024-04-23T10:26:11.124849+0000","last_fullsized":"2024-04-23T10:26:11.124849+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:26:32.716043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'7","reported_seq":149,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.825787+0000","last_change":"2024-04-23T10:24:06.574342+0000","last_active":"2024-04-23T10:26:11.825787+0000","last_peered":"2024-04-23T10:26:11.825787+0000","last_clean":"2024-04-23T10:26:11.825787+0000","last_became_active":"2024-04-23T10:24:06.573981+0000","last_became_peered":"2024-04-23T10:24:06.573981+0000","last_unstale":"2024-04-23T10:26:11.825787+0000","last_undegraded":"2024-04-23T10:26:11.825787+0000","last_fullsized":"2024-04-23T10:26:11.825787+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:48:40.325991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1370,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.124690+0000","last_change":"2024-04-23T10:24:07.044712+0000","last_active":"2024-04-23T10:26:11.124690+0000","last_peered":"2024-04-23T10:26:11.124690+0000","last_clean":"2024-04-23T10:26:11.124690+0000","last_became_active":"2024-04-23T10:24:07.044141+0000","last_became_peered":"2024-04-23T10:24:07.044141+0000","last_unstale":"2024-04-23T10:26:11.124690+0000","last_undegraded":"2024-04-23T10:26:11.124690+0000","last_fullsized":"2024-04-23T10:26:11.124690+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:14:38.818648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.117318+0000","last_change":"2024-04-23T10:25:00.729397+0000","last_active":"2024-04-23T10:26:11.117318+0000","last_peered":"2024-04-23T10:26:11.117318+0000","last_clean":"2024-04-23T10:26:11.117318+0000","last_became_active":"2024-04-23T10:25:00.729046+0000","last_became_peered":"2024-04-23T10:25:00.729046+0000","last_unstale":"2024-04-23T10:26:11.117318+0000","last_undegraded":"2024-04-23T10:26:11.117318+0000","last_fullsized":"2024-04-23T10:26:11.117318+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:25:55.189292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"89'7","reported_seq":152,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:12.133438+0000","last_change":"2024-04-23T10:24:06.578216+0000","last_active":"2024-04-23T10:26:12.133438+0000","last_peered":"2024-04-23T10:26:12.133438+0000","last_clean":"2024-04-23T10:26:12.133438+0000","last_became_active":"2024-04-23T10:24:06.577370+0000","last_became_peered":"2024-04-23T10:24:06.577370+0000","last_unstale":"2024-04-23T10:26:12.133438+0000","last_undegraded":"2024-04-23T10:26:12.133438+0000","last_fullsized":"2024-04-23T10:26:12.133438+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:02:06.554442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":123,"num_read_kb":123,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.132207+0000","last_change":"2024-04-23T10:24:07.039627+0000","last_active":"2024-04-23T10:26:11.132207+0000","last_peered":"2024-04-23T10:26:11.132207+0000","last_clean":"2024-04-23T10:26:11.132207+0000","last_became_active":"2024-04-23T10:24:07.039272+0000","last_became_peered":"2024-04-23T10:24:07.039272+0000","last_unstale":"2024-04-23T10:26:11.132207+0000","last_undegraded":"2024-04-23T10:26:11.132207+0000","last_fullsized":"2024-04-23T10:26:11.132207+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T13:45:44.277926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.131705+0000","last_change":"2024-04-23T10:24:06.569086+0000","last_active":"2024-04-23T10:26:11.131705+0000","last_peered":"2024-04-23T10:26:11.131705+0000","last_clean":"2024-04-23T10:26:11.131705+0000","last_became_active":"2024-04-23T10:24:06.568719+0000","last_became_peered":"2024-04-23T10:24:06.568719+0000","last_unstale":"2024-04-23T10:26:11.131705+0000","last_undegraded":"2024-04-23T10:26:11.131705+0000","last_fullsized":"2024-04-23T10:26:11.131705+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:22:10.583782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":357,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.116495+0000","last_change":"2024-04-23T10:22:49.957476+0000","last_active":"2024-04-23T10:26:11.116495+0000","last_peered":"2024-04-23T10:26:11.116495+0000","last_clean":"2024-04-23T10:26:11.116495+0000","last_became_active":"2024-04-23T10:22:49.957033+0000","last_became_peered":"2024-04-23T10:22:49.957033+0000","last_unstale":"2024-04-23T10:26:11.116495+0000","last_undegraded":"2024-04-23T10:26:11.116495+0000","last_fullsized":"2024-04-23T10:26:11.116495+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:19:25.604229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:19:25.604229+0000","last_clean_scrub_stamp":"2024-04-23T10:19:25.604229+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T12:56:17.601141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.129630+0000","last_change":"2024-04-23T10:25:00.671965+0000","last_active":"2024-04-23T10:26:11.129630+0000","last_peered":"2024-04-23T10:26:11.129630+0000","last_clean":"2024-04-23T10:26:11.129630+0000","last_became_active":"2024-04-23T10:25:00.671632+0000","last_became_peered":"2024-04-23T10:25:00.671632+0000","last_unstale":"2024-04-23T10:26:11.129630+0000","last_undegraded":"2024-04-23T10:26:11.129630+0000","last_fullsized":"2024-04-23T10:26:11.129630+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:08:40.262367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.116642+0000","last_change":"2024-04-23T10:24:06.582629+0000","last_active":"2024-04-23T10:26:11.116642+0000","last_peered":"2024-04-23T10:26:11.116642+0000","last_clean":"2024-04-23T10:26:11.116642+0000","last_became_active":"2024-04-23T10:24:06.582325+0000","last_became_peered":"2024-04-23T10:24:06.582325+0000","last_unstale":"2024-04-23T10:26:11.116642+0000","last_undegraded":"2024-04-23T10:26:11.116642+0000","last_fullsized":"2024-04-23T10:26:11.116642+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:57:09.879100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.157861+0000","last_change":"2024-04-23T10:24:06.582482+0000","last_active":"2024-04-23T10:26:11.157861+0000","last_peered":"2024-04-23T10:26:11.157861+0000","last_clean":"2024-04-23T10:26:11.157861+0000","last_became_active":"2024-04-23T10:24:06.582114+0000","last_became_peered":"2024-04-23T10:24:06.582114+0000","last_unstale":"2024-04-23T10:26:11.157861+0000","last_undegraded":"2024-04-23T10:26:11.157861+0000","last_fullsized":"2024-04-23T10:26:11.157861+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:45:42.939840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.129988+0000","last_change":"2024-04-23T10:24:07.031102+0000","last_active":"2024-04-23T10:26:11.129988+0000","last_peered":"2024-04-23T10:26:11.129988+0000","last_clean":"2024-04-23T10:26:11.129988+0000","last_became_active":"2024-04-23T10:24:07.030783+0000","last_became_peered":"2024-04-23T10:24:07.030783+0000","last_unstale":"2024-04-23T10:26:11.129988+0000","last_undegraded":"2024-04-23T10:26:11.129988+0000","last_fullsized":"2024-04-23T10:26:11.129988+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:49:46.736046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"89'49","reported_seq":93,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.318009+0000","last_change":"2024-04-23T10:24:06.577096+0000","last_active":"2024-04-23T10:26:11.318009+0000","last_peered":"2024-04-23T10:26:11.318009+0000","last_clean":"2024-04-23T10:26:11.318009+0000","last_became_active":"2024-04-23T10:24:06.576740+0000","last_became_peered":"2024-04-23T10:24:06.576740+0000","last_unstale":"2024-04-23T10:26:11.318009+0000","last_undegraded":"2024-04-23T10:26:11.318009+0000","last_fullsized":"2024-04-23T10:26:11.318009+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:44:06.532366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":86,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.132294+0000","last_change":"2024-04-23T10:24:07.043599+0000","last_active":"2024-04-23T10:26:11.132294+0000","last_peered":"2024-04-23T10:26:11.132294+0000","last_clean":"2024-04-23T10:26:11.132294+0000","last_became_active":"2024-04-23T10:24:07.042953+0000","last_became_peered":"2024-04-23T10:24:07.042953+0000","last_unstale":"2024-04-23T10:26:11.132294+0000","last_undegraded":"2024-04-23T10:26:11.132294+0000","last_fullsized":"2024-04-23T10:26:11.132294+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:07:50.487972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.131879+0000","last_change":"2024-04-23T10:24:06.570681+0000","last_active":"2024-04-23T10:26:11.131879+0000","last_peered":"2024-04-23T10:26:11.131879+0000","last_clean":"2024-04-23T10:26:11.131879+0000","last_became_active":"2024-04-23T10:24:06.570178+0000","last_became_peered":"2024-04-23T10:24:06.570178+0000","last_unstale":"2024-04-23T10:26:11.131879+0000","last_undegraded":"2024-04-23T10:26:11.131879+0000","last_fullsized":"2024-04-23T10:26:11.131879+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T12:16:56.629443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.350527+0000","last_change":"2024-04-23T10:24:07.040498+0000","last_active":"2024-04-23T10:26:11.350527+0000","last_peered":"2024-04-23T10:26:11.350527+0000","last_clean":"2024-04-23T10:26:11.350527+0000","last_became_active":"2024-04-23T10:24:07.040009+0000","last_became_peered":"2024-04-23T10:24:07.040009+0000","last_unstale":"2024-04-23T10:26:11.350527+0000","last_undegraded":"2024-04-23T10:26:11.350527+0000","last_fullsized":"2024-04-23T10:26:11.350527+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:43:17.319305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.349057+0000","last_change":"2024-04-23T10:25:00.673495+0000","last_active":"2024-04-23T10:26:11.349057+0000","last_peered":"2024-04-23T10:26:11.349057+0000","last_clean":"2024-04-23T10:26:11.349057+0000","last_became_active":"2024-04-23T10:25:00.670572+0000","last_became_peered":"2024-04-23T10:25:00.670572+0000","last_unstale":"2024-04-23T10:26:11.349057+0000","last_undegraded":"2024-04-23T10:26:11.349057+0000","last_fullsized":"2024-04-23T10:26:11.349057+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:08:04.279056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.157542+0000","last_change":"2024-04-23T10:24:07.043710+0000","last_active":"2024-04-23T10:26:11.157542+0000","last_peered":"2024-04-23T10:26:11.157542+0000","last_clean":"2024-04-23T10:26:11.157542+0000","last_became_active":"2024-04-23T10:24:07.043406+0000","last_became_peered":"2024-04-23T10:24:07.043406+0000","last_unstale":"2024-04-23T10:26:11.157542+0000","last_undegraded":"2024-04-23T10:26:11.157542+0000","last_fullsized":"2024-04-23T10:26:11.157542+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:09:14.471938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.131969+0000","last_change":"2024-04-23T10:24:06.577409+0000","last_active":"2024-04-23T10:26:11.131969+0000","last_peered":"2024-04-23T10:26:11.131969+0000","last_clean":"2024-04-23T10:26:11.131969+0000","last_became_active":"2024-04-23T10:24:06.576787+0000","last_became_peered":"2024-04-23T10:24:06.576787+0000","last_unstale":"2024-04-23T10:26:11.131969+0000","last_undegraded":"2024-04-23T10:26:11.131969+0000","last_fullsized":"2024-04-23T10:26:11.131969+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:05:28.423806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.355736+0000","last_change":"2024-04-23T10:24:07.045180+0000","last_active":"2024-04-23T10:26:11.355736+0000","last_peered":"2024-04-23T10:26:11.355736+0000","last_clean":"2024-04-23T10:26:11.355736+0000","last_became_active":"2024-04-23T10:24:07.044731+0000","last_became_peered":"2024-04-23T10:24:07.044731+0000","last_unstale":"2024-04-23T10:26:11.355736+0000","last_undegraded":"2024-04-23T10:26:11.355736+0000","last_fullsized":"2024-04-23T10:26:11.355736+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:42:34.474391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.116406+0000","last_change":"2024-04-23T10:24:06.578039+0000","last_active":"2024-04-23T10:26:11.116406+0000","last_peered":"2024-04-23T10:26:11.116406+0000","last_clean":"2024-04-23T10:26:11.116406+0000","last_became_active":"2024-04-23T10:24:06.577655+0000","last_became_peered":"2024-04-23T10:24:06.577655+0000","last_unstale":"2024-04-23T10:26:11.116406+0000","last_undegraded":"2024-04-23T10:26:11.116406+0000","last_fullsized":"2024-04-23T10:26:11.116406+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:57:35.172318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.132097+0000","last_change":"2024-04-23T10:24:07.037311+0000","last_active":"2024-04-23T10:26:11.132097+0000","last_peered":"2024-04-23T10:26:11.132097+0000","last_clean":"2024-04-23T10:26:11.132097+0000","last_became_active":"2024-04-23T10:24:07.036969+0000","last_became_peered":"2024-04-23T10:24:07.036969+0000","last_unstale":"2024-04-23T10:26:11.132097+0000","last_undegraded":"2024-04-23T10:26:11.132097+0000","last_fullsized":"2024-04-23T10:26:11.132097+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:13:18.998914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.157662+0000","last_change":"2024-04-23T10:24:06.575183+0000","last_active":"2024-04-23T10:26:11.157662+0000","last_peered":"2024-04-23T10:26:11.157662+0000","last_clean":"2024-04-23T10:26:11.157662+0000","last_became_active":"2024-04-23T10:24:06.574630+0000","last_became_peered":"2024-04-23T10:24:06.574630+0000","last_unstale":"2024-04-23T10:26:11.157662+0000","last_undegraded":"2024-04-23T10:26:11.157662+0000","last_fullsized":"2024-04-23T10:26:11.157662+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T11:18:18.189925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.116657+0000","last_change":"2024-04-23T10:24:07.048832+0000","last_active":"2024-04-23T10:26:11.116657+0000","last_peered":"2024-04-23T10:26:11.116657+0000","last_clean":"2024-04-23T10:26:11.116657+0000","last_became_active":"2024-04-23T10:24:07.048509+0000","last_became_peered":"2024-04-23T10:24:07.048509+0000","last_unstale":"2024-04-23T10:26:11.116657+0000","last_undegraded":"2024-04-23T10:26:11.116657+0000","last_fullsized":"2024-04-23T10:26:11.116657+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T13:45:26.793330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'8","reported_seq":151,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.156578+0000","last_change":"2024-04-23T10:24:06.575408+0000","last_active":"2024-04-23T10:26:11.156578+0000","last_peered":"2024-04-23T10:26:11.156578+0000","last_clean":"2024-04-23T10:26:11.156578+0000","last_became_active":"2024-04-23T10:24:06.575081+0000","last_became_peered":"2024-04-23T10:24:06.575081+0000","last_unstale":"2024-04-23T10:26:11.156578+0000","last_undegraded":"2024-04-23T10:26:11.156578+0000","last_fullsized":"2024-04-23T10:26:11.156578+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T11:28:12.765196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2093,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":122,"num_read_kb":3,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.116971+0000","last_change":"2024-04-23T10:24:07.043869+0000","last_active":"2024-04-23T10:26:11.116971+0000","last_peered":"2024-04-23T10:26:11.116971+0000","last_clean":"2024-04-23T10:26:11.116971+0000","last_became_active":"2024-04-23T10:24:07.043448+0000","last_became_peered":"2024-04-23T10:24:07.043448+0000","last_unstale":"2024-04-23T10:26:11.116971+0000","last_undegraded":"2024-04-23T10:26:11.116971+0000","last_fullsized":"2024-04-23T10:26:11.116971+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:00:35.821470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'41","reported_seq":70,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.310840+0000","last_change":"2024-04-23T10:24:06.575259+0000","last_active":"2024-04-23T10:26:11.310840+0000","last_peered":"2024-04-23T10:26:11.310840+0000","last_clean":"2024-04-23T10:26:11.310840+0000","last_became_active":"2024-04-23T10:24:06.574920+0000","last_became_peered":"2024-04-23T10:24:06.574920+0000","last_unstale":"2024-04-23T10:26:11.310840+0000","last_undegraded":"2024-04-23T10:26:11.310840+0000","last_fullsized":"2024-04-23T10:26:11.310840+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:52:14.558458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.116672+0000","last_change":"2024-04-23T10:24:07.043238+0000","last_active":"2024-04-23T10:26:11.116672+0000","last_peered":"2024-04-23T10:26:11.116672+0000","last_clean":"2024-04-23T10:26:11.116672+0000","last_became_active":"2024-04-23T10:24:07.042110+0000","last_became_peered":"2024-04-23T10:24:07.042110+0000","last_unstale":"2024-04-23T10:26:11.116672+0000","last_undegraded":"2024-04-23T10:26:11.116672+0000","last_fullsized":"2024-04-23T10:26:11.116672+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T12:36:13.474456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.130204+0000","last_change":"2024-04-23T10:24:06.579233+0000","last_active":"2024-04-23T10:26:11.130204+0000","last_peered":"2024-04-23T10:26:11.130204+0000","last_clean":"2024-04-23T10:26:11.130204+0000","last_became_active":"2024-04-23T10:24:06.578849+0000","last_became_peered":"2024-04-23T10:24:06.578849+0000","last_unstale":"2024-04-23T10:26:11.130204+0000","last_undegraded":"2024-04-23T10:26:11.130204+0000","last_fullsized":"2024-04-23T10:26:11.130204+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:34:23.193707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.116824+0000","last_change":"2024-04-23T10:24:07.049840+0000","last_active":"2024-04-23T10:26:11.116824+0000","last_peered":"2024-04-23T10:26:11.116824+0000","last_clean":"2024-04-23T10:26:11.116824+0000","last_became_active":"2024-04-23T10:24:07.049493+0000","last_became_peered":"2024-04-23T10:24:07.049493+0000","last_unstale":"2024-04-23T10:26:11.116824+0000","last_undegraded":"2024-04-23T10:26:11.116824+0000","last_fullsized":"2024-04-23T10:26:11.116824+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:03:22.606800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.355883+0000","last_change":"2024-04-23T10:24:07.052332+0000","last_active":"2024-04-23T10:26:11.355883+0000","last_peered":"2024-04-23T10:26:11.355883+0000","last_clean":"2024-04-23T10:26:11.355883+0000","last_became_active":"2024-04-23T10:24:07.051956+0000","last_became_peered":"2024-04-23T10:24:07.051956+0000","last_unstale":"2024-04-23T10:26:11.355883+0000","last_undegraded":"2024-04-23T10:26:11.355883+0000","last_fullsized":"2024-04-23T10:26:11.355883+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:56:43.606916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.116207+0000","last_change":"2024-04-23T10:24:07.041897+0000","last_active":"2024-04-23T10:26:11.116207+0000","last_peered":"2024-04-23T10:26:11.116207+0000","last_clean":"2024-04-23T10:26:11.116207+0000","last_became_active":"2024-04-23T10:24:07.041546+0000","last_became_peered":"2024-04-23T10:24:07.041546+0000","last_unstale":"2024-04-23T10:26:11.116207+0000","last_undegraded":"2024-04-23T10:26:11.116207+0000","last_fullsized":"2024-04-23T10:26:11.116207+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T11:31:23.211045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.116192+0000","last_change":"2024-04-23T10:24:07.040764+0000","last_active":"2024-04-23T10:26:11.116192+0000","last_peered":"2024-04-23T10:26:11.116192+0000","last_clean":"2024-04-23T10:26:11.116192+0000","last_became_active":"2024-04-23T10:24:07.039637+0000","last_became_peered":"2024-04-23T10:24:07.039637+0000","last_unstale":"2024-04-23T10:26:11.116192+0000","last_undegraded":"2024-04-23T10:26:11.116192+0000","last_fullsized":"2024-04-23T10:26:11.116192+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:50:33.298770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.350168+0000","last_change":"2024-04-23T10:24:07.039491+0000","last_active":"2024-04-23T10:26:11.350168+0000","last_peered":"2024-04-23T10:26:11.350168+0000","last_clean":"2024-04-23T10:26:11.350168+0000","last_became_active":"2024-04-23T10:24:07.039128+0000","last_became_peered":"2024-04-23T10:24:07.039128+0000","last_unstale":"2024-04-23T10:26:11.350168+0000","last_undegraded":"2024-04-23T10:26:11.350168+0000","last_fullsized":"2024-04-23T10:26:11.350168+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T12:30:40.326031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.130013+0000","last_change":"2024-04-23T10:25:00.671059+0000","last_active":"2024-04-23T10:26:11.130013+0000","last_peered":"2024-04-23T10:26:11.130013+0000","last_clean":"2024-04-23T10:26:11.130013+0000","last_became_active":"2024-04-23T10:25:00.670686+0000","last_became_peered":"2024-04-23T10:25:00.670686+0000","last_unstale":"2024-04-23T10:26:11.130013+0000","last_undegraded":"2024-04-23T10:26:11.130013+0000","last_fullsized":"2024-04-23T10:26:11.130013+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:04:39.991647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.355865+0000","last_change":"2024-04-23T10:24:07.052395+0000","last_active":"2024-04-23T10:26:11.355865+0000","last_peered":"2024-04-23T10:26:11.355865+0000","last_clean":"2024-04-23T10:26:11.355865+0000","last_became_active":"2024-04-23T10:24:07.052023+0000","last_became_peered":"2024-04-23T10:24:07.052023+0000","last_unstale":"2024-04-23T10:26:11.355865+0000","last_undegraded":"2024-04-23T10:26:11.355865+0000","last_fullsized":"2024-04-23T10:26:11.355865+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:25:52.517267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.116425+0000","last_change":"2024-04-23T10:24:07.042739+0000","last_active":"2024-04-23T10:26:11.116425+0000","last_peered":"2024-04-23T10:26:11.116425+0000","last_clean":"2024-04-23T10:26:11.116425+0000","last_became_active":"2024-04-23T10:24:07.042292+0000","last_became_peered":"2024-04-23T10:24:07.042292+0000","last_unstale":"2024-04-23T10:26:11.116425+0000","last_undegraded":"2024-04-23T10:26:11.116425+0000","last_fullsized":"2024-04-23T10:26:11.116425+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:14:23.229032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.132286+0000","last_change":"2024-04-23T10:24:07.042792+0000","last_active":"2024-04-23T10:26:11.132286+0000","last_peered":"2024-04-23T10:26:11.132286+0000","last_clean":"2024-04-23T10:26:11.132286+0000","last_became_active":"2024-04-23T10:24:07.040846+0000","last_became_peered":"2024-04-23T10:24:07.040846+0000","last_unstale":"2024-04-23T10:26:11.132286+0000","last_undegraded":"2024-04-23T10:26:11.132286+0000","last_fullsized":"2024-04-23T10:26:11.132286+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T13:50:24.973286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.127085+0000","last_change":"2024-04-23T10:24:07.040391+0000","last_active":"2024-04-23T10:26:11.127085+0000","last_peered":"2024-04-23T10:26:11.127085+0000","last_clean":"2024-04-23T10:26:11.127085+0000","last_became_active":"2024-04-23T10:24:07.040002+0000","last_became_peered":"2024-04-23T10:24:07.040002+0000","last_unstale":"2024-04-23T10:26:11.127085+0000","last_undegraded":"2024-04-23T10:26:11.127085+0000","last_fullsized":"2024-04-23T10:26:11.127085+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T10:30:13.922315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.131939+0000","last_change":"2024-04-23T10:24:07.040678+0000","last_active":"2024-04-23T10:26:11.131939+0000","last_peered":"2024-04-23T10:26:11.131939+0000","last_clean":"2024-04-23T10:26:11.131939+0000","last_became_active":"2024-04-23T10:24:07.039551+0000","last_became_peered":"2024-04-23T10:24:07.039551+0000","last_unstale":"2024-04-23T10:26:11.131939+0000","last_undegraded":"2024-04-23T10:26:11.131939+0000","last_fullsized":"2024-04-23T10:26:11.131939+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:03:50.467083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.349510+0000","last_change":"2024-04-23T10:24:07.044618+0000","last_active":"2024-04-23T10:26:11.349510+0000","last_peered":"2024-04-23T10:26:11.349510+0000","last_clean":"2024-04-23T10:26:11.349510+0000","last_became_active":"2024-04-23T10:24:07.042803+0000","last_became_peered":"2024-04-23T10:24:07.042803+0000","last_unstale":"2024-04-23T10:26:11.349510+0000","last_undegraded":"2024-04-23T10:26:11.349510+0000","last_fullsized":"2024-04-23T10:26:11.349510+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:37:46.872177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.129943+0000","last_change":"2024-04-23T10:24:07.043724+0000","last_active":"2024-04-23T10:26:11.129943+0000","last_peered":"2024-04-23T10:26:11.129943+0000","last_clean":"2024-04-23T10:26:11.129943+0000","last_became_active":"2024-04-23T10:24:07.043408+0000","last_became_peered":"2024-04-23T10:24:07.043408+0000","last_unstale":"2024-04-23T10:26:11.129943+0000","last_undegraded":"2024-04-23T10:26:11.129943+0000","last_fullsized":"2024-04-23T10:26:11.129943+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:18:46.615093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.355748+0000","last_change":"2024-04-23T10:24:07.043794+0000","last_active":"2024-04-23T10:26:11.355748+0000","last_peered":"2024-04-23T10:26:11.355748+0000","last_clean":"2024-04-23T10:26:11.355748+0000","last_became_active":"2024-04-23T10:24:07.043391+0000","last_became_peered":"2024-04-23T10:24:07.043391+0000","last_unstale":"2024-04-23T10:26:11.355748+0000","last_undegraded":"2024-04-23T10:26:11.355748+0000","last_fullsized":"2024-04-23T10:26:11.355748+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:06:04.584000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":106845,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":602156,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1342,"num_read_kb":785,"num_write":375,"num_write_kb":1469,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1254,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":712704,"data_stored":1806468,"data_compressed":81474,"data_compressed_allocated":110592,"data_compressed_original":1572864,"omap_allocated":0,"internal_metadata":0},"log_size":304,"ondisk_log_size":304,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351014,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":974848,"data_stored":470174,"data_compressed":2902,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612651,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28144,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717008,"statfs":{"total":95995035648,"available":95966216192,"internally_reserved":0,"allocated":1150976,"data_stored":1259979,"data_compressed":46529,"data_compressed_allocated":61440,"data_compressed_original":729088,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776176,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1019904,"data_stored":861815,"data_compressed":22048,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037814,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1003520,"data_stored":860939,"data_compressed":22048,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299451,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1044480,"data_stored":1063341,"data_compressed":38517,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462976,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":978944,"data_stored":469098,"data_compressed":2902,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1044480,"data_stored":537769,"data_compressed":4758,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":999424,"data_stored":470578,"data_compressed":2902,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1064960,"data_stored":538662,"data_compressed":4758,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149776,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1040384,"data_stored":992957,"data_compressed":28204,"data_compressed_allocated":36864,"data_compressed_original":524288,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888138,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":987136,"data_stored":469772,"data_compressed":2902,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724612,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32240,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712912,"statfs":{"total":95995035648,"available":95962021888,"internally_reserved":0,"allocated":1077248,"data_stored":1192914,"data_compressed":44673,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35615,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35615,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35615,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":70292,"data_compressed":1856,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":71185,"data_compressed":1856,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":134952,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":393462,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":202017,"data_compressed":8012,"data_compressed_allocated":12288,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T10:26:17.119 INFO:tasks.quiescer.ceph_manager:clean! 2024-04-23T10:26:17.119 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T10:26:17.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.275+0000 7f0413a6b640 1 -- 172.21.15.97:0/3965238768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f040c1018f0 msgr2=0x7f040c105db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:17.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.275+0000 7f0413a6b640 1 --2- 172.21.15.97:0/3965238768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f040c1018f0 0x7f040c105db0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f03fc009900 tx=0x7f03fc02a510 comp rx=0 tx=0).stop 2024-04-23T10:26:17.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.275+0000 7f0413a6b640 1 -- 172.21.15.97:0/3965238768 shutdown_connections 2024-04-23T10:26:17.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.275+0000 7f0413a6b640 1 --2- 172.21.15.97:0/3965238768 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f040c1062f0 0x7f040c106780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:17.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.275+0000 7f0413a6b640 1 --2- 172.21.15.97:0/3965238768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f040c1018f0 0x7f040c105db0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:17.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.275+0000 7f0413a6b640 1 --2- 172.21.15.97:0/3965238768 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f040c100fd0 0x7f040c1013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:17.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.275+0000 7f0413a6b640 1 -- 172.21.15.97:0/3965238768 >> 172.21.15.97:0/3965238768 conn(0x7f040c077f50 msgr2=0x7f040c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:17.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.275+0000 7f0413a6b640 1 -- 172.21.15.97:0/3965238768 shutdown_connections 2024-04-23T10:26:17.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.275+0000 7f0413a6b640 1 -- 172.21.15.97:0/3965238768 wait complete. 2024-04-23T10:26:17.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.276+0000 7f0413a6b640 1 Processor -- start 2024-04-23T10:26:17.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.276+0000 7f0413a6b640 1 -- start start 2024-04-23T10:26:17.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.276+0000 7f0413a6b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f040c100fd0 0x7f040c1a48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:17.277 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.276+0000 7f0413a6b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f040c1018f0 0x7f040c1a4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:17.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.276+0000 7f0413a6b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f040c1062f0 0x7f040c19ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:17.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.276+0000 7f0413a6b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f040c1187e0 con 0x7f040c1018f0 2024-04-23T10:26:17.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.276+0000 7f0413a6b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f040c118660 con 0x7f040c1062f0 2024-04-23T10:26:17.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.276+0000 7f0413a6b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f040c118960 con 0x7f040c100fd0 2024-04-23T10:26:17.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.276+0000 7f0411006640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f040c1018f0 0x7f040c1a4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:17.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.276+0000 7f0411006640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f040c1018f0 0x7f040c1a4df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46960/0 (socket says 172.21.15.97:46960) 2024-04-23T10:26:17.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.276+0000 7f0411006640 1 -- 172.21.15.97:0/3881818812 learned_addr learned my addr 172.21.15.97:0/3881818812 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:17.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.277+0000 7f0411807640 1 --2- 172.21.15.97:0/3881818812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f040c100fd0 0x7f040c1a48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:17.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.277+0000 7f0412008640 1 --2- 172.21.15.97:0/3881818812 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f040c1062f0 0x7f040c19ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:17.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.277+0000 7f0411006640 1 -- 172.21.15.97:0/3881818812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f040c100fd0 msgr2=0x7f040c1a48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:17.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.277+0000 7f0411006640 1 --2- 172.21.15.97:0/3881818812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f040c100fd0 0x7f040c1a48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:17.278 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.277+0000 7f0411006640 1 -- 172.21.15.97:0/3881818812 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f040c1062f0 msgr2=0x7f040c19ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:17.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.277+0000 7f0411006640 1 --2- 172.21.15.97:0/3881818812 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f040c1062f0 0x7f040c19ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:17.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.277+0000 7f0411006640 1 -- 172.21.15.97:0/3881818812 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f040c19f1e0 con 0x7f040c1018f0 2024-04-23T10:26:17.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.277+0000 7f0412008640 1 --2- 172.21.15.97:0/3881818812 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f040c1062f0 0x7f040c19ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:26:17.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.277+0000 7f0411006640 1 --2- 172.21.15.97:0/3881818812 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f040c1018f0 0x7f040c1a4df0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f03fc005ab0 tx=0x7f03fc003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:17.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.277+0000 7f0402ffd640 1 -- 172.21.15.97:0/3881818812 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03fc03f070 con 0x7f040c1018f0 2024-04-23T10:26:17.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.277+0000 7f0402ffd640 1 -- 172.21.15.97:0/3881818812 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03fc031d60 con 0x7f040c1018f0 2024-04-23T10:26:17.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.277+0000 7f0413a6b640 1 -- 172.21.15.97:0/3881818812 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f040c19f470 con 0x7f040c1018f0 2024-04-23T10:26:17.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.277+0000 7f0402ffd640 1 -- 172.21.15.97:0/3881818812 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03fc035030 con 0x7f040c1018f0 2024-04-23T10:26:17.279 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.277+0000 7f0413a6b640 1 -- 172.21.15.97:0/3881818812 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f040c1ab770 con 0x7f040c1018f0 2024-04-23T10:26:17.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.278+0000 7f0413a6b640 1 -- 172.21.15.97:0/3881818812 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f040c068d70 con 0x7f040c1018f0 2024-04-23T10:26:17.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.279+0000 7f0402ffd640 1 -- 172.21.15.97:0/3881818812 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f03fc042070 con 0x7f040c1018f0 2024-04-23T10:26:17.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.280+0000 7f0402ffd640 1 --2- 172.21.15.97:0/3881818812 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f03f80a3e40 0x7f03f80a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:17.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.280+0000 7f0402ffd640 1 -- 172.21.15.97:0/3881818812 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f03fc0f1420 con 0x7f040c1018f0 2024-04-23T10:26:17.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.280+0000 7f0411807640 1 --2- 172.21.15.97:0/3881818812 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f03f80a3e40 0x7f03f80a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:17.280 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.280+0000 7f0411807640 1 --2- 172.21.15.97:0/3881818812 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f03f80a3e40 0x7f03f80a6300 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0408005e10 tx=0x7f0408005da0 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:17.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.285+0000 7f0402ffd640 1 -- 172.21.15.97:0/3881818812 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f03fc0bd990 con 0x7f040c1018f0 2024-04-23T10:26:17.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:16 smithi196 ceph-mon[32919]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:26:17.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:16 smithi196 ceph-mon[32919]: Updating smithi183:/etc/ceph/ceph.client.admin.keyring 2024-04-23T10:26:17.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:16 smithi196 ceph-mon[32919]: Updating smithi097:/etc/ceph/ceph.client.admin.keyring 2024-04-23T10:26:17.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:16 smithi196 ceph-mon[32919]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-04-23T10:26:17.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:16 smithi196 ceph-mon[32919]: Updating smithi097:/var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T10:26:17.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:16 smithi196 ceph-mon[32919]: Updating smithi183:/var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T10:26:17.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:16 smithi196 ceph-mon[32919]: Updating smithi196:/var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T10:26:17.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:16 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:26:17.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:16 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:26:17.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:16 smithi196 ceph-mon[32919]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-23T10:26:17.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:16 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:26:17.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:16 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/2705289938' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T10:26:17.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:16 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/1917864722' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T10:26:17.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:16 smithi183 ceph-mon[32317]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T10:26:17.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:16 smithi183 ceph-mon[32317]: Updating smithi183:/etc/ceph/ceph.client.admin.keyring 2024-04-23T10:26:17.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:16 smithi183 ceph-mon[32317]: Updating smithi097:/etc/ceph/ceph.client.admin.keyring 2024-04-23T10:26:17.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:16 smithi183 ceph-mon[32317]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-04-23T10:26:17.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:16 smithi183 ceph-mon[32317]: Updating smithi097:/var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T10:26:17.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:16 smithi183 ceph-mon[32317]: Updating smithi183:/var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T10:26:17.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:16 smithi183 ceph-mon[32317]: Updating smithi196:/var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T10:26:17.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:16 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:26:17.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:16 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:26:17.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:16 smithi183 ceph-mon[32317]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-23T10:26:17.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:16 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:26:17.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:16 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/2705289938' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T10:26:17.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:16 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/1917864722' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T10:26:17.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.459+0000 7f0413a6b640 1 -- 172.21.15.97:0/3881818812 --> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f040c1054a0 con 0x7f03f80a3e40 2024-04-23T10:26:17.465 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.467+0000 7f0402ffd640 1 -- 172.21.15.97:0/3881818812 <== mgr.15378 v2:172.21.15.97:6800/3567268502 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258096 (secure 0 0 0) 0x7f040c1054a0 con 0x7f03f80a3e40 2024-04-23T10:26:17.468 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:26:17.469 INFO:teuthology.orchestra.run.smithi097.stderr:dumped all 2024-04-23T10:26:17.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.469+0000 7f0413a6b640 1 -- 172.21.15.97:0/3881818812 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f03f80a3e40 msgr2=0x7f03f80a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:17.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.469+0000 7f0413a6b640 1 --2- 172.21.15.97:0/3881818812 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f03f80a3e40 0x7f03f80a6300 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0408005e10 tx=0x7f0408005da0 comp rx=0 tx=0).stop 2024-04-23T10:26:17.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.469+0000 7f0413a6b640 1 -- 172.21.15.97:0/3881818812 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f040c1018f0 msgr2=0x7f040c1a4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:17.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.469+0000 7f0413a6b640 1 --2- 172.21.15.97:0/3881818812 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f040c1018f0 0x7f040c1a4df0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f03fc005ab0 tx=0x7f03fc003d50 comp rx=0 tx=0).stop 2024-04-23T10:26:17.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.469+0000 7f0413a6b640 1 -- 172.21.15.97:0/3881818812 shutdown_connections 2024-04-23T10:26:17.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.469+0000 7f0413a6b640 1 --2- 172.21.15.97:0/3881818812 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f03f80a3e40 0x7f03f80a6300 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:17.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.469+0000 7f0413a6b640 1 --2- 172.21.15.97:0/3881818812 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f040c1062f0 0x7f040c19ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:17.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.469+0000 7f0413a6b640 1 --2- 172.21.15.97:0/3881818812 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f040c1018f0 0x7f040c1a4df0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:17.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.469+0000 7f0413a6b640 1 --2- 172.21.15.97:0/3881818812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f040c100fd0 0x7f040c1a48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:17.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.470+0000 7f0413a6b640 1 -- 172.21.15.97:0/3881818812 >> 172.21.15.97:0/3881818812 conn(0x7f040c077f50 msgr2=0x7f040c103d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:17.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.470+0000 7f0413a6b640 1 -- 172.21.15.97:0/3881818812 shutdown_connections 2024-04-23T10:26:17.471 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.470+0000 7f0413a6b640 1 -- 172.21.15.97:0/3881818812 wait complete. 2024-04-23T10:26:17.480 INFO:teuthology.orchestra.run.smithi097.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-04-23T10:26:17.131307+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1193149,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1719,"num_read_kb":1117,"num_write":662,"num_write_kb":4860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":1181990,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":999,"ondisk_log_size":999,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":365480,"kb_used_data":12124,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576344,"statfs":{"total":1151940427776,"available":1151566176256,"internally_reserved":0,"allocated":12414976,"data_stored":9389105,"data_compressed":232618,"data_compressed_allocated":307200,"data_compressed_original":3624960,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":391,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":241,"num_read_kb":213,"num_write":104,"num_write_kb":451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.992257"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.356030+0000","last_change":"2024-04-23T10:24:07.040397+0000","last_active":"2024-04-23T10:26:11.356030+0000","last_peered":"2024-04-23T10:26:11.356030+0000","last_clean":"2024-04-23T10:26:11.356030+0000","last_became_active":"2024-04-23T10:24:07.040007+0000","last_became_peered":"2024-04-23T10:24:07.040007+0000","last_unstale":"2024-04-23T10:26:11.356030+0000","last_undegraded":"2024-04-23T10:26:11.356030+0000","last_fullsized":"2024-04-23T10:26:11.356030+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T13:50:24.533115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.350839+0000","last_change":"2024-04-23T10:24:07.044672+0000","last_active":"2024-04-23T10:26:11.350839+0000","last_peered":"2024-04-23T10:26:11.350839+0000","last_clean":"2024-04-23T10:26:11.350839+0000","last_became_active":"2024-04-23T10:24:07.044184+0000","last_became_peered":"2024-04-23T10:24:07.044184+0000","last_unstale":"2024-04-23T10:26:11.350839+0000","last_undegraded":"2024-04-23T10:26:11.350839+0000","last_fullsized":"2024-04-23T10:26:11.350839+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:07:16.729205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.350394+0000","last_change":"2024-04-23T10:24:07.041959+0000","last_active":"2024-04-23T10:26:11.350394+0000","last_peered":"2024-04-23T10:26:11.350394+0000","last_clean":"2024-04-23T10:26:11.350394+0000","last_became_active":"2024-04-23T10:24:07.041661+0000","last_became_peered":"2024-04-23T10:24:07.041661+0000","last_unstale":"2024-04-23T10:26:11.350394+0000","last_undegraded":"2024-04-23T10:26:11.350394+0000","last_fullsized":"2024-04-23T10:26:11.350394+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:23:14.694183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.131765+0000","last_change":"2024-04-23T10:24:07.048334+0000","last_active":"2024-04-23T10:26:11.131765+0000","last_peered":"2024-04-23T10:26:11.131765+0000","last_clean":"2024-04-23T10:26:11.131765+0000","last_became_active":"2024-04-23T10:24:07.047506+0000","last_became_peered":"2024-04-23T10:24:07.047506+0000","last_unstale":"2024-04-23T10:26:11.131765+0000","last_undegraded":"2024-04-23T10:26:11.131765+0000","last_fullsized":"2024-04-23T10:26:11.131765+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:27:27.674287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.157602+0000","last_change":"2024-04-23T10:24:07.035531+0000","last_active":"2024-04-23T10:26:11.157602+0000","last_peered":"2024-04-23T10:26:11.157602+0000","last_clean":"2024-04-23T10:26:11.157602+0000","last_became_active":"2024-04-23T10:24:07.035167+0000","last_became_peered":"2024-04-23T10:24:07.035167+0000","last_unstale":"2024-04-23T10:26:11.157602+0000","last_undegraded":"2024-04-23T10:26:11.157602+0000","last_fullsized":"2024-04-23T10:26:11.157602+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:39:17.198650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.124553+0000","last_change":"2024-04-23T10:24:07.040593+0000","last_active":"2024-04-23T10:26:11.124553+0000","last_peered":"2024-04-23T10:26:11.124553+0000","last_clean":"2024-04-23T10:26:11.124553+0000","last_became_active":"2024-04-23T10:24:07.040255+0000","last_became_peered":"2024-04-23T10:24:07.040255+0000","last_unstale":"2024-04-23T10:26:11.124553+0000","last_undegraded":"2024-04-23T10:26:11.124553+0000","last_fullsized":"2024-04-23T10:26:11.124553+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:05:52.663566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.116448+0000","last_change":"2024-04-23T10:24:07.051688+0000","last_active":"2024-04-23T10:26:11.116448+0000","last_peered":"2024-04-23T10:26:11.116448+0000","last_clean":"2024-04-23T10:26:11.116448+0000","last_became_active":"2024-04-23T10:24:07.050265+0000","last_became_peered":"2024-04-23T10:24:07.050265+0000","last_unstale":"2024-04-23T10:26:11.116448+0000","last_undegraded":"2024-04-23T10:26:11.116448+0000","last_fullsized":"2024-04-23T10:26:11.116448+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T13:13:36.126183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.131842+0000","last_change":"2024-04-23T10:24:07.045937+0000","last_active":"2024-04-23T10:26:11.131842+0000","last_peered":"2024-04-23T10:26:11.131842+0000","last_clean":"2024-04-23T10:26:11.131842+0000","last_became_active":"2024-04-23T10:24:07.045616+0000","last_became_peered":"2024-04-23T10:24:07.045616+0000","last_unstale":"2024-04-23T10:26:11.131842+0000","last_undegraded":"2024-04-23T10:26:11.131842+0000","last_fullsized":"2024-04-23T10:26:11.131842+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T10:45:27.250031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.349840+0000","last_change":"2024-04-23T10:24:07.039294+0000","last_active":"2024-04-23T10:26:11.349840+0000","last_peered":"2024-04-23T10:26:11.349840+0000","last_clean":"2024-04-23T10:26:11.349840+0000","last_became_active":"2024-04-23T10:24:07.038959+0000","last_became_peered":"2024-04-23T10:24:07.038959+0000","last_unstale":"2024-04-23T10:26:11.349840+0000","last_undegraded":"2024-04-23T10:26:11.349840+0000","last_fullsized":"2024-04-23T10:26:11.349840+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:53:41.345062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.124874+0000","last_change":"2024-04-23T10:24:07.044177+0000","last_active":"2024-04-23T10:26:11.124874+0000","last_peered":"2024-04-23T10:26:11.124874+0000","last_clean":"2024-04-23T10:26:11.124874+0000","last_became_active":"2024-04-23T10:24:07.043821+0000","last_became_peered":"2024-04-23T10:24:07.043821+0000","last_unstale":"2024-04-23T10:26:11.124874+0000","last_undegraded":"2024-04-23T10:26:11.124874+0000","last_fullsized":"2024-04-23T10:26:11.124874+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:09:38.782407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.116882+0000","last_change":"2024-04-23T10:24:07.052775+0000","last_active":"2024-04-23T10:26:11.116882+0000","last_peered":"2024-04-23T10:26:11.116882+0000","last_clean":"2024-04-23T10:26:11.116882+0000","last_became_active":"2024-04-23T10:24:07.051708+0000","last_became_peered":"2024-04-23T10:24:07.051708+0000","last_unstale":"2024-04-23T10:26:11.116882+0000","last_undegraded":"2024-04-23T10:26:11.116882+0000","last_fullsized":"2024-04-23T10:26:11.116882+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:47:55.360467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.131663+0000","last_change":"2024-04-23T10:24:07.048655+0000","last_active":"2024-04-23T10:26:11.131663+0000","last_peered":"2024-04-23T10:26:11.131663+0000","last_clean":"2024-04-23T10:26:11.131663+0000","last_became_active":"2024-04-23T10:24:07.047994+0000","last_became_peered":"2024-04-23T10:24:07.047994+0000","last_unstale":"2024-04-23T10:26:11.131663+0000","last_undegraded":"2024-04-23T10:26:11.131663+0000","last_fullsized":"2024-04-23T10:26:11.131663+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T10:25:51.425871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.129951+0000","last_change":"2024-04-23T10:24:07.036319+0000","last_active":"2024-04-23T10:26:11.129951+0000","last_peered":"2024-04-23T10:26:11.129951+0000","last_clean":"2024-04-23T10:26:11.129951+0000","last_became_active":"2024-04-23T10:24:07.035982+0000","last_became_peered":"2024-04-23T10:24:07.035982+0000","last_unstale":"2024-04-23T10:26:11.129951+0000","last_undegraded":"2024-04-23T10:26:11.129951+0000","last_fullsized":"2024-04-23T10:26:11.129951+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T10:25:46.191118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.349736+0000","last_change":"2024-04-23T10:24:07.033166+0000","last_active":"2024-04-23T10:26:11.349736+0000","last_peered":"2024-04-23T10:26:11.349736+0000","last_clean":"2024-04-23T10:26:11.349736+0000","last_became_active":"2024-04-23T10:24:07.032807+0000","last_became_peered":"2024-04-23T10:24:07.032807+0000","last_unstale":"2024-04-23T10:26:11.349736+0000","last_undegraded":"2024-04-23T10:26:11.349736+0000","last_fullsized":"2024-04-23T10:26:11.349736+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:52:35.654965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.349923+0000","last_change":"2024-04-23T10:24:07.041988+0000","last_active":"2024-04-23T10:26:11.349923+0000","last_peered":"2024-04-23T10:26:11.349923+0000","last_clean":"2024-04-23T10:26:11.349923+0000","last_became_active":"2024-04-23T10:24:07.041638+0000","last_became_peered":"2024-04-23T10:24:07.041638+0000","last_unstale":"2024-04-23T10:26:11.349923+0000","last_undegraded":"2024-04-23T10:26:11.349923+0000","last_fullsized":"2024-04-23T10:26:11.349923+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:17:40.565981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.349144+0000","last_change":"2024-04-23T10:24:07.045111+0000","last_active":"2024-04-23T10:26:11.349144+0000","last_peered":"2024-04-23T10:26:11.349144+0000","last_clean":"2024-04-23T10:26:11.349144+0000","last_became_active":"2024-04-23T10:24:07.044249+0000","last_became_peered":"2024-04-23T10:24:07.044249+0000","last_unstale":"2024-04-23T10:26:11.349144+0000","last_undegraded":"2024-04-23T10:26:11.349144+0000","last_fullsized":"2024-04-23T10:26:11.349144+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:08:36.029636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.356095+0000","last_change":"2024-04-23T10:24:07.040685+0000","last_active":"2024-04-23T10:26:11.356095+0000","last_peered":"2024-04-23T10:26:11.356095+0000","last_clean":"2024-04-23T10:26:11.356095+0000","last_became_active":"2024-04-23T10:24:07.038881+0000","last_became_peered":"2024-04-23T10:24:07.038881+0000","last_unstale":"2024-04-23T10:26:11.356095+0000","last_undegraded":"2024-04-23T10:26:11.356095+0000","last_fullsized":"2024-04-23T10:26:11.356095+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:29:12.489136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.117665+0000","last_change":"2024-04-23T10:24:07.041815+0000","last_active":"2024-04-23T10:26:11.117665+0000","last_peered":"2024-04-23T10:26:11.117665+0000","last_clean":"2024-04-23T10:26:11.117665+0000","last_became_active":"2024-04-23T10:24:07.041495+0000","last_became_peered":"2024-04-23T10:24:07.041495+0000","last_unstale":"2024-04-23T10:26:11.117665+0000","last_undegraded":"2024-04-23T10:26:11.117665+0000","last_fullsized":"2024-04-23T10:26:11.117665+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:29:02.006394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.116347+0000","last_change":"2024-04-23T10:24:07.041722+0000","last_active":"2024-04-23T10:26:11.116347+0000","last_peered":"2024-04-23T10:26:11.116347+0000","last_clean":"2024-04-23T10:26:11.116347+0000","last_became_active":"2024-04-23T10:24:07.039434+0000","last_became_peered":"2024-04-23T10:24:07.039434+0000","last_unstale":"2024-04-23T10:26:11.116347+0000","last_undegraded":"2024-04-23T10:26:11.116347+0000","last_fullsized":"2024-04-23T10:26:11.116347+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:03:44.033094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'14","reported_seq":181,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:15.830687+0000","last_change":"2024-04-23T10:25:00.739010+0000","last_active":"2024-04-23T10:26:15.830687+0000","last_peered":"2024-04-23T10:26:15.830687+0000","last_clean":"2024-04-23T10:26:15.830687+0000","last_became_active":"2024-04-23T10:25:00.729834+0000","last_became_peered":"2024-04-23T10:25:00.729834+0000","last_unstale":"2024-04-23T10:26:15.830687+0000","last_undegraded":"2024-04-23T10:26:15.830687+0000","last_fullsized":"2024-04-23T10:26:15.830687+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T12:57:45.871560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":120,"num_read_kb":120,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,4],"acting":[9,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.132261+0000","last_change":"2024-04-23T10:24:07.047192+0000","last_active":"2024-04-23T10:26:11.132261+0000","last_peered":"2024-04-23T10:26:11.132261+0000","last_clean":"2024-04-23T10:26:11.132261+0000","last_became_active":"2024-04-23T10:24:07.046750+0000","last_became_peered":"2024-04-23T10:24:07.046750+0000","last_unstale":"2024-04-23T10:26:11.132261+0000","last_undegraded":"2024-04-23T10:26:11.132261+0000","last_fullsized":"2024-04-23T10:26:11.132261+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:13:09.399425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.132396+0000","last_change":"2024-04-23T10:24:06.577588+0000","last_active":"2024-04-23T10:26:11.132396+0000","last_peered":"2024-04-23T10:26:11.132396+0000","last_clean":"2024-04-23T10:26:11.132396+0000","last_became_active":"2024-04-23T10:24:06.576971+0000","last_became_peered":"2024-04-23T10:24:06.576971+0000","last_unstale":"2024-04-23T10:26:11.132396+0000","last_undegraded":"2024-04-23T10:26:11.132396+0000","last_fullsized":"2024-04-23T10:26:11.132396+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:07:21.577868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.350711+0000","last_change":"2024-04-23T10:24:07.030666+0000","last_active":"2024-04-23T10:26:11.350711+0000","last_peered":"2024-04-23T10:26:11.350711+0000","last_clean":"2024-04-23T10:26:11.350711+0000","last_became_active":"2024-04-23T10:24:07.030342+0000","last_became_peered":"2024-04-23T10:24:07.030342+0000","last_unstale":"2024-04-23T10:26:11.350711+0000","last_undegraded":"2024-04-23T10:26:11.350711+0000","last_fullsized":"2024-04-23T10:26:11.350711+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:09:14.818638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"89'13","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.157995+0000","last_change":"2024-04-23T10:24:06.576042+0000","last_active":"2024-04-23T10:26:11.157995+0000","last_peered":"2024-04-23T10:26:11.157995+0000","last_clean":"2024-04-23T10:26:11.157995+0000","last_became_active":"2024-04-23T10:24:06.574518+0000","last_became_peered":"2024-04-23T10:24:06.574518+0000","last_unstale":"2024-04-23T10:26:11.157995+0000","last_undegraded":"2024-04-23T10:26:11.157995+0000","last_fullsized":"2024-04-23T10:26:11.157995+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T11:56:40.100213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.349417+0000","last_change":"2024-04-23T10:24:07.046591+0000","last_active":"2024-04-23T10:26:11.349417+0000","last_peered":"2024-04-23T10:26:11.349417+0000","last_clean":"2024-04-23T10:26:11.349417+0000","last_became_active":"2024-04-23T10:24:07.046208+0000","last_became_peered":"2024-04-23T10:24:07.046208+0000","last_unstale":"2024-04-23T10:26:11.349417+0000","last_undegraded":"2024-04-23T10:26:11.349417+0000","last_fullsized":"2024-04-23T10:26:11.349417+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:27:37.711329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.355666+0000","last_change":"2024-04-23T10:25:00.676255+0000","last_active":"2024-04-23T10:26:11.355666+0000","last_peered":"2024-04-23T10:26:11.355666+0000","last_clean":"2024-04-23T10:26:11.355666+0000","last_became_active":"2024-04-23T10:25:00.670146+0000","last_became_peered":"2024-04-23T10:25:00.670146+0000","last_unstale":"2024-04-23T10:26:11.355666+0000","last_undegraded":"2024-04-23T10:26:11.355666+0000","last_fullsized":"2024-04-23T10:26:11.355666+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:34:54.217317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.127034+0000","last_change":"2024-04-23T10:24:07.047926+0000","last_active":"2024-04-23T10:26:11.127034+0000","last_peered":"2024-04-23T10:26:11.127034+0000","last_clean":"2024-04-23T10:26:11.127034+0000","last_became_active":"2024-04-23T10:24:07.047273+0000","last_became_peered":"2024-04-23T10:24:07.047273+0000","last_unstale":"2024-04-23T10:26:11.127034+0000","last_undegraded":"2024-04-23T10:26:11.127034+0000","last_fullsized":"2024-04-23T10:26:11.127034+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:37:25.068034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'7","reported_seq":154,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.127172+0000","last_change":"2024-04-23T10:24:06.574733+0000","last_active":"2024-04-23T10:26:11.127172+0000","last_peered":"2024-04-23T10:26:11.127172+0000","last_clean":"2024-04-23T10:26:11.127172+0000","last_became_active":"2024-04-23T10:24:06.574276+0000","last_became_peered":"2024-04-23T10:24:06.574276+0000","last_unstale":"2024-04-23T10:26:11.127172+0000","last_undegraded":"2024-04-23T10:26:11.127172+0000","last_fullsized":"2024-04-23T10:26:11.127172+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:27:57.080635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.157562+0000","last_change":"2024-04-23T10:24:07.040110+0000","last_active":"2024-04-23T10:26:11.157562+0000","last_peered":"2024-04-23T10:26:11.157562+0000","last_clean":"2024-04-23T10:26:11.157562+0000","last_became_active":"2024-04-23T10:24:07.039769+0000","last_became_peered":"2024-04-23T10:24:07.039769+0000","last_unstale":"2024-04-23T10:26:11.157562+0000","last_undegraded":"2024-04-23T10:26:11.157562+0000","last_fullsized":"2024-04-23T10:26:11.157562+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:10:07.884895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.124478+0000","last_change":"2024-04-23T10:24:06.567448+0000","last_active":"2024-04-23T10:26:11.124478+0000","last_peered":"2024-04-23T10:26:11.124478+0000","last_clean":"2024-04-23T10:26:11.124478+0000","last_became_active":"2024-04-23T10:24:06.566958+0000","last_became_peered":"2024-04-23T10:24:06.566958+0000","last_unstale":"2024-04-23T10:26:11.124478+0000","last_undegraded":"2024-04-23T10:26:11.124478+0000","last_fullsized":"2024-04-23T10:26:11.124478+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:45:52.863596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.126797+0000","last_change":"2024-04-23T10:24:07.040754+0000","last_active":"2024-04-23T10:26:11.126797+0000","last_peered":"2024-04-23T10:26:11.126797+0000","last_clean":"2024-04-23T10:26:11.126797+0000","last_became_active":"2024-04-23T10:24:07.040397+0000","last_became_peered":"2024-04-23T10:24:07.040397+0000","last_unstale":"2024-04-23T10:26:11.126797+0000","last_undegraded":"2024-04-23T10:26:11.126797+0000","last_fullsized":"2024-04-23T10:26:11.126797+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T11:01:03.626506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.129719+0000","last_change":"2024-04-23T10:24:06.568592+0000","last_active":"2024-04-23T10:26:11.129719+0000","last_peered":"2024-04-23T10:26:11.129719+0000","last_clean":"2024-04-23T10:26:11.129719+0000","last_became_active":"2024-04-23T10:24:06.568192+0000","last_became_peered":"2024-04-23T10:24:06.568192+0000","last_unstale":"2024-04-23T10:26:11.129719+0000","last_undegraded":"2024-04-23T10:26:11.129719+0000","last_fullsized":"2024-04-23T10:26:11.129719+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:14:42.589596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.131909+0000","last_change":"2024-04-23T10:24:07.037058+0000","last_active":"2024-04-23T10:26:11.131909+0000","last_peered":"2024-04-23T10:26:11.131909+0000","last_clean":"2024-04-23T10:26:11.131909+0000","last_became_active":"2024-04-23T10:24:07.036659+0000","last_became_peered":"2024-04-23T10:24:07.036659+0000","last_unstale":"2024-04-23T10:26:11.131909+0000","last_undegraded":"2024-04-23T10:26:11.131909+0000","last_fullsized":"2024-04-23T10:26:11.131909+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T12:53:49.298603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.127061+0000","last_change":"2024-04-23T10:24:06.576900+0000","last_active":"2024-04-23T10:26:11.127061+0000","last_peered":"2024-04-23T10:26:11.127061+0000","last_clean":"2024-04-23T10:26:11.127061+0000","last_became_active":"2024-04-23T10:24:06.575627+0000","last_became_peered":"2024-04-23T10:24:06.575627+0000","last_unstale":"2024-04-23T10:26:11.127061+0000","last_undegraded":"2024-04-23T10:26:11.127061+0000","last_fullsized":"2024-04-23T10:26:11.127061+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:16:07.973841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.117469+0000","last_change":"2024-04-23T10:24:07.035294+0000","last_active":"2024-04-23T10:26:11.117469+0000","last_peered":"2024-04-23T10:26:11.117469+0000","last_clean":"2024-04-23T10:26:11.117469+0000","last_became_active":"2024-04-23T10:24:07.035001+0000","last_became_peered":"2024-04-23T10:24:07.035001+0000","last_unstale":"2024-04-23T10:26:11.117469+0000","last_undegraded":"2024-04-23T10:26:11.117469+0000","last_fullsized":"2024-04-23T10:26:11.117469+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T13:12:47.013132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"89'7","reported_seq":269,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.355330+0000","last_change":"2024-04-23T10:24:06.573637+0000","last_active":"2024-04-23T10:26:11.355330+0000","last_peered":"2024-04-23T10:26:11.355330+0000","last_clean":"2024-04-23T10:26:11.355330+0000","last_became_active":"2024-04-23T10:24:06.573236+0000","last_became_peered":"2024-04-23T10:24:06.573236+0000","last_unstale":"2024-04-23T10:26:11.355330+0000","last_undegraded":"2024-04-23T10:26:11.355330+0000","last_fullsized":"2024-04-23T10:26:11.355330+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T12:48:06.740181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":241,"num_read_kb":241,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.349816+0000","last_change":"2024-04-23T10:24:07.037457+0000","last_active":"2024-04-23T10:26:11.349816+0000","last_peered":"2024-04-23T10:26:11.349816+0000","last_clean":"2024-04-23T10:26:11.349816+0000","last_became_active":"2024-04-23T10:24:07.036456+0000","last_became_peered":"2024-04-23T10:24:07.036456+0000","last_unstale":"2024-04-23T10:26:11.349816+0000","last_undegraded":"2024-04-23T10:26:11.349816+0000","last_fullsized":"2024-04-23T10:26:11.349816+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:42:57.750435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.116889+0000","last_change":"2024-04-23T10:25:00.741712+0000","last_active":"2024-04-23T10:26:11.116889+0000","last_peered":"2024-04-23T10:26:11.116889+0000","last_clean":"2024-04-23T10:26:11.116889+0000","last_became_active":"2024-04-23T10:25:00.730130+0000","last_became_peered":"2024-04-23T10:25:00.730130+0000","last_unstale":"2024-04-23T10:26:11.116889+0000","last_undegraded":"2024-04-23T10:26:11.116889+0000","last_fullsized":"2024-04-23T10:26:11.116889+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:08:14.409376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.350621+0000","last_change":"2024-04-23T10:24:07.040317+0000","last_active":"2024-04-23T10:26:11.350621+0000","last_peered":"2024-04-23T10:26:11.350621+0000","last_clean":"2024-04-23T10:26:11.350621+0000","last_became_active":"2024-04-23T10:24:07.039998+0000","last_became_peered":"2024-04-23T10:24:07.039998+0000","last_unstale":"2024-04-23T10:26:11.350621+0000","last_undegraded":"2024-04-23T10:26:11.350621+0000","last_fullsized":"2024-04-23T10:26:11.350621+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:50:30.412127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.127037+0000","last_change":"2024-04-23T10:24:06.570019+0000","last_active":"2024-04-23T10:26:11.127037+0000","last_peered":"2024-04-23T10:26:11.127037+0000","last_clean":"2024-04-23T10:26:11.127037+0000","last_became_active":"2024-04-23T10:24:06.569676+0000","last_became_peered":"2024-04-23T10:24:06.569676+0000","last_unstale":"2024-04-23T10:26:11.127037+0000","last_undegraded":"2024-04-23T10:26:11.127037+0000","last_fullsized":"2024-04-23T10:26:11.127037+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:31:46.994787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.349539+0000","last_change":"2024-04-23T10:24:07.040183+0000","last_active":"2024-04-23T10:26:11.349539+0000","last_peered":"2024-04-23T10:26:11.349539+0000","last_clean":"2024-04-23T10:26:11.349539+0000","last_became_active":"2024-04-23T10:24:07.039866+0000","last_became_peered":"2024-04-23T10:24:07.039866+0000","last_unstale":"2024-04-23T10:26:11.349539+0000","last_undegraded":"2024-04-23T10:26:11.349539+0000","last_fullsized":"2024-04-23T10:26:11.349539+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:33:04.411765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"89'38","reported_seq":315,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:12.128873+0000","last_change":"2024-04-23T10:24:06.568883+0000","last_active":"2024-04-23T10:26:12.128873+0000","last_peered":"2024-04-23T10:26:12.128873+0000","last_clean":"2024-04-23T10:26:12.128873+0000","last_became_active":"2024-04-23T10:24:06.568517+0000","last_became_peered":"2024-04-23T10:24:06.568517+0000","last_unstale":"2024-04-23T10:26:12.128873+0000","last_undegraded":"2024-04-23T10:26:12.128873+0000","last_fullsized":"2024-04-23T10:26:12.128873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:51:42.440346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67758,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":256,"num_read_kb":194,"num_write":40,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.157789+0000","last_change":"2024-04-23T10:24:07.046839+0000","last_active":"2024-04-23T10:26:11.157789+0000","last_peered":"2024-04-23T10:26:11.157789+0000","last_clean":"2024-04-23T10:26:11.157789+0000","last_became_active":"2024-04-23T10:24:07.046512+0000","last_became_peered":"2024-04-23T10:24:07.046512+0000","last_unstale":"2024-04-23T10:26:11.157789+0000","last_undegraded":"2024-04-23T10:26:11.157789+0000","last_fullsized":"2024-04-23T10:26:11.157789+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T12:42:11.822100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.116303+0000","last_change":"2024-04-23T10:24:06.578087+0000","last_active":"2024-04-23T10:26:11.116303+0000","last_peered":"2024-04-23T10:26:11.116303+0000","last_clean":"2024-04-23T10:26:11.116303+0000","last_became_active":"2024-04-23T10:24:06.577666+0000","last_became_peered":"2024-04-23T10:24:06.577666+0000","last_unstale":"2024-04-23T10:26:11.116303+0000","last_undegraded":"2024-04-23T10:26:11.116303+0000","last_fullsized":"2024-04-23T10:26:11.116303+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:55:15.053303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.129816+0000","last_change":"2024-04-23T10:24:07.043672+0000","last_active":"2024-04-23T10:26:11.129816+0000","last_peered":"2024-04-23T10:26:11.129816+0000","last_clean":"2024-04-23T10:26:11.129816+0000","last_became_active":"2024-04-23T10:24:07.043346+0000","last_became_peered":"2024-04-23T10:24:07.043346+0000","last_unstale":"2024-04-23T10:26:11.129816+0000","last_undegraded":"2024-04-23T10:26:11.129816+0000","last_fullsized":"2024-04-23T10:26:11.129816+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T13:21:56.751028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.126748+0000","last_change":"2024-04-23T10:24:06.576607+0000","last_active":"2024-04-23T10:26:11.126748+0000","last_peered":"2024-04-23T10:26:11.126748+0000","last_clean":"2024-04-23T10:26:11.126748+0000","last_became_active":"2024-04-23T10:24:06.576213+0000","last_became_peered":"2024-04-23T10:24:06.576213+0000","last_unstale":"2024-04-23T10:26:11.126748+0000","last_undegraded":"2024-04-23T10:26:11.126748+0000","last_fullsized":"2024-04-23T10:26:11.126748+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:09:14.918936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.116719+0000","last_change":"2024-04-23T10:24:07.052641+0000","last_active":"2024-04-23T10:26:11.116719+0000","last_peered":"2024-04-23T10:26:11.116719+0000","last_clean":"2024-04-23T10:26:11.116719+0000","last_became_active":"2024-04-23T10:24:07.052314+0000","last_became_peered":"2024-04-23T10:24:07.052314+0000","last_unstale":"2024-04-23T10:26:11.116719+0000","last_undegraded":"2024-04-23T10:26:11.116719+0000","last_fullsized":"2024-04-23T10:26:11.116719+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T10:29:18.098800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'8","reported_seq":155,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.349410+0000","last_change":"2024-04-23T10:24:06.574570+0000","last_active":"2024-04-23T10:26:11.349410+0000","last_peered":"2024-04-23T10:26:11.349410+0000","last_clean":"2024-04-23T10:26:11.349410+0000","last_became_active":"2024-04-23T10:24:06.574166+0000","last_became_peered":"2024-04-23T10:24:06.574166+0000","last_unstale":"2024-04-23T10:26:11.349410+0000","last_undegraded":"2024-04-23T10:26:11.349410+0000","last_fullsized":"2024-04-23T10:26:11.349410+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:28:31.619100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":127,"num_read_kb":5,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.116415+0000","last_change":"2024-04-23T10:24:07.052825+0000","last_active":"2024-04-23T10:26:11.116415+0000","last_peered":"2024-04-23T10:26:11.116415+0000","last_clean":"2024-04-23T10:26:11.116415+0000","last_became_active":"2024-04-23T10:24:07.052457+0000","last_became_peered":"2024-04-23T10:24:07.052457+0000","last_unstale":"2024-04-23T10:26:11.116415+0000","last_undegraded":"2024-04-23T10:26:11.116415+0000","last_fullsized":"2024-04-23T10:26:11.116415+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:47:09.545374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.157540+0000","last_change":"2024-04-23T10:24:06.572168+0000","last_active":"2024-04-23T10:26:11.157540+0000","last_peered":"2024-04-23T10:26:11.157540+0000","last_clean":"2024-04-23T10:26:11.157540+0000","last_became_active":"2024-04-23T10:24:06.571805+0000","last_became_peered":"2024-04-23T10:24:06.571805+0000","last_unstale":"2024-04-23T10:26:11.157540+0000","last_undegraded":"2024-04-23T10:26:11.157540+0000","last_fullsized":"2024-04-23T10:26:11.157540+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:29:56.817879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.126680+0000","last_change":"2024-04-23T10:24:07.048474+0000","last_active":"2024-04-23T10:26:11.126680+0000","last_peered":"2024-04-23T10:26:11.126680+0000","last_clean":"2024-04-23T10:26:11.126680+0000","last_became_active":"2024-04-23T10:24:07.047893+0000","last_became_peered":"2024-04-23T10:24:07.047893+0000","last_unstale":"2024-04-23T10:26:11.126680+0000","last_undegraded":"2024-04-23T10:26:11.126680+0000","last_fullsized":"2024-04-23T10:26:11.126680+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T12:59:46.545569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.157883+0000","last_change":"2024-04-23T10:24:06.570974+0000","last_active":"2024-04-23T10:26:11.157883+0000","last_peered":"2024-04-23T10:26:11.157883+0000","last_clean":"2024-04-23T10:26:11.157883+0000","last_became_active":"2024-04-23T10:24:06.570603+0000","last_became_peered":"2024-04-23T10:24:06.570603+0000","last_unstale":"2024-04-23T10:26:11.157883+0000","last_undegraded":"2024-04-23T10:26:11.157883+0000","last_fullsized":"2024-04-23T10:26:11.157883+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T12:46:33.518337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.355892+0000","last_change":"2024-04-23T10:24:07.050986+0000","last_active":"2024-04-23T10:26:11.355892+0000","last_peered":"2024-04-23T10:26:11.355892+0000","last_clean":"2024-04-23T10:26:11.355892+0000","last_became_active":"2024-04-23T10:24:07.050617+0000","last_became_peered":"2024-04-23T10:24:07.050617+0000","last_unstale":"2024-04-23T10:26:11.355892+0000","last_undegraded":"2024-04-23T10:26:11.355892+0000","last_fullsized":"2024-04-23T10:26:11.355892+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:13:25.355117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.117580+0000","last_change":"2024-04-23T10:24:06.579138+0000","last_active":"2024-04-23T10:26:11.117580+0000","last_peered":"2024-04-23T10:26:11.117580+0000","last_clean":"2024-04-23T10:26:11.117580+0000","last_became_active":"2024-04-23T10:24:06.578716+0000","last_became_peered":"2024-04-23T10:24:06.578716+0000","last_unstale":"2024-04-23T10:26:11.117580+0000","last_undegraded":"2024-04-23T10:26:11.117580+0000","last_fullsized":"2024-04-23T10:26:11.117580+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:06:59.797802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.124849+0000","last_change":"2024-04-23T10:24:07.039019+0000","last_active":"2024-04-23T10:26:11.124849+0000","last_peered":"2024-04-23T10:26:11.124849+0000","last_clean":"2024-04-23T10:26:11.124849+0000","last_became_active":"2024-04-23T10:24:07.038701+0000","last_became_peered":"2024-04-23T10:24:07.038701+0000","last_unstale":"2024-04-23T10:26:11.124849+0000","last_undegraded":"2024-04-23T10:26:11.124849+0000","last_fullsized":"2024-04-23T10:26:11.124849+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:26:32.716043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'7","reported_seq":149,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.825787+0000","last_change":"2024-04-23T10:24:06.574342+0000","last_active":"2024-04-23T10:26:11.825787+0000","last_peered":"2024-04-23T10:26:11.825787+0000","last_clean":"2024-04-23T10:26:11.825787+0000","last_became_active":"2024-04-23T10:24:06.573981+0000","last_became_peered":"2024-04-23T10:24:06.573981+0000","last_unstale":"2024-04-23T10:26:11.825787+0000","last_undegraded":"2024-04-23T10:26:11.825787+0000","last_fullsized":"2024-04-23T10:26:11.825787+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:48:40.325991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1370,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.124690+0000","last_change":"2024-04-23T10:24:07.044712+0000","last_active":"2024-04-23T10:26:11.124690+0000","last_peered":"2024-04-23T10:26:11.124690+0000","last_clean":"2024-04-23T10:26:11.124690+0000","last_became_active":"2024-04-23T10:24:07.044141+0000","last_became_peered":"2024-04-23T10:24:07.044141+0000","last_unstale":"2024-04-23T10:26:11.124690+0000","last_undegraded":"2024-04-23T10:26:11.124690+0000","last_fullsized":"2024-04-23T10:26:11.124690+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:14:38.818648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.117318+0000","last_change":"2024-04-23T10:25:00.729397+0000","last_active":"2024-04-23T10:26:11.117318+0000","last_peered":"2024-04-23T10:26:11.117318+0000","last_clean":"2024-04-23T10:26:11.117318+0000","last_became_active":"2024-04-23T10:25:00.729046+0000","last_became_peered":"2024-04-23T10:25:00.729046+0000","last_unstale":"2024-04-23T10:26:11.117318+0000","last_undegraded":"2024-04-23T10:26:11.117318+0000","last_fullsized":"2024-04-23T10:26:11.117318+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:25:55.189292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"89'7","reported_seq":152,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:12.133438+0000","last_change":"2024-04-23T10:24:06.578216+0000","last_active":"2024-04-23T10:26:12.133438+0000","last_peered":"2024-04-23T10:26:12.133438+0000","last_clean":"2024-04-23T10:26:12.133438+0000","last_became_active":"2024-04-23T10:24:06.577370+0000","last_became_peered":"2024-04-23T10:24:06.577370+0000","last_unstale":"2024-04-23T10:26:12.133438+0000","last_undegraded":"2024-04-23T10:26:12.133438+0000","last_fullsized":"2024-04-23T10:26:12.133438+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:02:06.554442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":123,"num_read_kb":123,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.132207+0000","last_change":"2024-04-23T10:24:07.039627+0000","last_active":"2024-04-23T10:26:11.132207+0000","last_peered":"2024-04-23T10:26:11.132207+0000","last_clean":"2024-04-23T10:26:11.132207+0000","last_became_active":"2024-04-23T10:24:07.039272+0000","last_became_peered":"2024-04-23T10:24:07.039272+0000","last_unstale":"2024-04-23T10:26:11.132207+0000","last_undegraded":"2024-04-23T10:26:11.132207+0000","last_fullsized":"2024-04-23T10:26:11.132207+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T13:45:44.277926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.131705+0000","last_change":"2024-04-23T10:24:06.569086+0000","last_active":"2024-04-23T10:26:11.131705+0000","last_peered":"2024-04-23T10:26:11.131705+0000","last_clean":"2024-04-23T10:26:11.131705+0000","last_became_active":"2024-04-23T10:24:06.568719+0000","last_became_peered":"2024-04-23T10:24:06.568719+0000","last_unstale":"2024-04-23T10:26:11.131705+0000","last_undegraded":"2024-04-23T10:26:11.131705+0000","last_fullsized":"2024-04-23T10:26:11.131705+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:22:10.583782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":357,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.116495+0000","last_change":"2024-04-23T10:22:49.957476+0000","last_active":"2024-04-23T10:26:11.116495+0000","last_peered":"2024-04-23T10:26:11.116495+0000","last_clean":"2024-04-23T10:26:11.116495+0000","last_became_active":"2024-04-23T10:22:49.957033+0000","last_became_peered":"2024-04-23T10:22:49.957033+0000","last_unstale":"2024-04-23T10:26:11.116495+0000","last_undegraded":"2024-04-23T10:26:11.116495+0000","last_fullsized":"2024-04-23T10:26:11.116495+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:19:25.604229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:19:25.604229+0000","last_clean_scrub_stamp":"2024-04-23T10:19:25.604229+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T12:56:17.601141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.129630+0000","last_change":"2024-04-23T10:25:00.671965+0000","last_active":"2024-04-23T10:26:11.129630+0000","last_peered":"2024-04-23T10:26:11.129630+0000","last_clean":"2024-04-23T10:26:11.129630+0000","last_became_active":"2024-04-23T10:25:00.671632+0000","last_became_peered":"2024-04-23T10:25:00.671632+0000","last_unstale":"2024-04-23T10:26:11.129630+0000","last_undegraded":"2024-04-23T10:26:11.129630+0000","last_fullsized":"2024-04-23T10:26:11.129630+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:08:40.262367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.116642+0000","last_change":"2024-04-23T10:24:06.582629+0000","last_active":"2024-04-23T10:26:11.116642+0000","last_peered":"2024-04-23T10:26:11.116642+0000","last_clean":"2024-04-23T10:26:11.116642+0000","last_became_active":"2024-04-23T10:24:06.582325+0000","last_became_peered":"2024-04-23T10:24:06.582325+0000","last_unstale":"2024-04-23T10:26:11.116642+0000","last_undegraded":"2024-04-23T10:26:11.116642+0000","last_fullsized":"2024-04-23T10:26:11.116642+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:57:09.879100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.157913+0000","last_change":"2024-04-23T10:24:06.582482+0000","last_active":"2024-04-23T10:26:11.157913+0000","last_peered":"2024-04-23T10:26:11.157913+0000","last_clean":"2024-04-23T10:26:11.157913+0000","last_became_active":"2024-04-23T10:24:06.582114+0000","last_became_peered":"2024-04-23T10:24:06.582114+0000","last_unstale":"2024-04-23T10:26:11.157913+0000","last_undegraded":"2024-04-23T10:26:11.157913+0000","last_fullsized":"2024-04-23T10:26:11.157913+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:45:42.939840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.129988+0000","last_change":"2024-04-23T10:24:07.031102+0000","last_active":"2024-04-23T10:26:11.129988+0000","last_peered":"2024-04-23T10:26:11.129988+0000","last_clean":"2024-04-23T10:26:11.129988+0000","last_became_active":"2024-04-23T10:24:07.030783+0000","last_became_peered":"2024-04-23T10:24:07.030783+0000","last_unstale":"2024-04-23T10:26:11.129988+0000","last_undegraded":"2024-04-23T10:26:11.129988+0000","last_fullsized":"2024-04-23T10:26:11.129988+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:49:46.736046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"89'103","reported_seq":217,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:13.699413+0000","last_change":"2024-04-23T10:24:06.577096+0000","last_active":"2024-04-23T10:26:13.699413+0000","last_peered":"2024-04-23T10:26:13.699413+0000","last_clean":"2024-04-23T10:26:13.699413+0000","last_became_active":"2024-04-23T10:24:06.576740+0000","last_became_peered":"2024-04-23T10:24:06.576740+0000","last_unstale":"2024-04-23T10:26:13.699413+0000","last_undegraded":"2024-04-23T10:26:13.699413+0000","last_fullsized":"2024-04-23T10:26:13.699413+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:44:06.532366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":187,"num_write_kb":1286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.132346+0000","last_change":"2024-04-23T10:24:07.043599+0000","last_active":"2024-04-23T10:26:11.132346+0000","last_peered":"2024-04-23T10:26:11.132346+0000","last_clean":"2024-04-23T10:26:11.132346+0000","last_became_active":"2024-04-23T10:24:07.042953+0000","last_became_peered":"2024-04-23T10:24:07.042953+0000","last_unstale":"2024-04-23T10:26:11.132346+0000","last_undegraded":"2024-04-23T10:26:11.132346+0000","last_fullsized":"2024-04-23T10:26:11.132346+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:07:50.487972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.131879+0000","last_change":"2024-04-23T10:24:06.570681+0000","last_active":"2024-04-23T10:26:11.131879+0000","last_peered":"2024-04-23T10:26:11.131879+0000","last_clean":"2024-04-23T10:26:11.131879+0000","last_became_active":"2024-04-23T10:24:06.570178+0000","last_became_peered":"2024-04-23T10:24:06.570178+0000","last_unstale":"2024-04-23T10:26:11.131879+0000","last_undegraded":"2024-04-23T10:26:11.131879+0000","last_fullsized":"2024-04-23T10:26:11.131879+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T12:16:56.629443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.350578+0000","last_change":"2024-04-23T10:24:07.040498+0000","last_active":"2024-04-23T10:26:11.350578+0000","last_peered":"2024-04-23T10:26:11.350578+0000","last_clean":"2024-04-23T10:26:11.350578+0000","last_became_active":"2024-04-23T10:24:07.040009+0000","last_became_peered":"2024-04-23T10:24:07.040009+0000","last_unstale":"2024-04-23T10:26:11.350578+0000","last_undegraded":"2024-04-23T10:26:11.350578+0000","last_fullsized":"2024-04-23T10:26:11.350578+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:43:17.319305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.349057+0000","last_change":"2024-04-23T10:25:00.673495+0000","last_active":"2024-04-23T10:26:11.349057+0000","last_peered":"2024-04-23T10:26:11.349057+0000","last_clean":"2024-04-23T10:26:11.349057+0000","last_became_active":"2024-04-23T10:25:00.670572+0000","last_became_peered":"2024-04-23T10:25:00.670572+0000","last_unstale":"2024-04-23T10:26:11.349057+0000","last_undegraded":"2024-04-23T10:26:11.349057+0000","last_fullsized":"2024-04-23T10:26:11.349057+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:08:04.279056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.157542+0000","last_change":"2024-04-23T10:24:07.043710+0000","last_active":"2024-04-23T10:26:11.157542+0000","last_peered":"2024-04-23T10:26:11.157542+0000","last_clean":"2024-04-23T10:26:11.157542+0000","last_became_active":"2024-04-23T10:24:07.043406+0000","last_became_peered":"2024-04-23T10:24:07.043406+0000","last_unstale":"2024-04-23T10:26:11.157542+0000","last_undegraded":"2024-04-23T10:26:11.157542+0000","last_fullsized":"2024-04-23T10:26:11.157542+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:09:14.471938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.131969+0000","last_change":"2024-04-23T10:24:06.577409+0000","last_active":"2024-04-23T10:26:11.131969+0000","last_peered":"2024-04-23T10:26:11.131969+0000","last_clean":"2024-04-23T10:26:11.131969+0000","last_became_active":"2024-04-23T10:24:06.576787+0000","last_became_peered":"2024-04-23T10:24:06.576787+0000","last_unstale":"2024-04-23T10:26:11.131969+0000","last_undegraded":"2024-04-23T10:26:11.131969+0000","last_fullsized":"2024-04-23T10:26:11.131969+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:05:28.423806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.355802+0000","last_change":"2024-04-23T10:24:07.045180+0000","last_active":"2024-04-23T10:26:11.355802+0000","last_peered":"2024-04-23T10:26:11.355802+0000","last_clean":"2024-04-23T10:26:11.355802+0000","last_became_active":"2024-04-23T10:24:07.044731+0000","last_became_peered":"2024-04-23T10:24:07.044731+0000","last_unstale":"2024-04-23T10:26:11.355802+0000","last_undegraded":"2024-04-23T10:26:11.355802+0000","last_fullsized":"2024-04-23T10:26:11.355802+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:42:34.474391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"89'4","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.116462+0000","last_change":"2024-04-23T10:24:06.578039+0000","last_active":"2024-04-23T10:26:11.116462+0000","last_peered":"2024-04-23T10:26:11.116462+0000","last_clean":"2024-04-23T10:26:11.116462+0000","last_became_active":"2024-04-23T10:24:06.577655+0000","last_became_peered":"2024-04-23T10:24:06.577655+0000","last_unstale":"2024-04-23T10:26:11.116462+0000","last_undegraded":"2024-04-23T10:26:11.116462+0000","last_fullsized":"2024-04-23T10:26:11.116462+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:57:35.172318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.132097+0000","last_change":"2024-04-23T10:24:07.037311+0000","last_active":"2024-04-23T10:26:11.132097+0000","last_peered":"2024-04-23T10:26:11.132097+0000","last_clean":"2024-04-23T10:26:11.132097+0000","last_became_active":"2024-04-23T10:24:07.036969+0000","last_became_peered":"2024-04-23T10:24:07.036969+0000","last_unstale":"2024-04-23T10:26:11.132097+0000","last_undegraded":"2024-04-23T10:26:11.132097+0000","last_fullsized":"2024-04-23T10:26:11.132097+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:13:18.998914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.157662+0000","last_change":"2024-04-23T10:24:06.575183+0000","last_active":"2024-04-23T10:26:11.157662+0000","last_peered":"2024-04-23T10:26:11.157662+0000","last_clean":"2024-04-23T10:26:11.157662+0000","last_became_active":"2024-04-23T10:24:06.574630+0000","last_became_peered":"2024-04-23T10:24:06.574630+0000","last_unstale":"2024-04-23T10:26:11.157662+0000","last_undegraded":"2024-04-23T10:26:11.157662+0000","last_fullsized":"2024-04-23T10:26:11.157662+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T11:18:18.189925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.116657+0000","last_change":"2024-04-23T10:24:07.048832+0000","last_active":"2024-04-23T10:26:11.116657+0000","last_peered":"2024-04-23T10:26:11.116657+0000","last_clean":"2024-04-23T10:26:11.116657+0000","last_became_active":"2024-04-23T10:24:07.048509+0000","last_became_peered":"2024-04-23T10:24:07.048509+0000","last_unstale":"2024-04-23T10:26:11.116657+0000","last_undegraded":"2024-04-23T10:26:11.116657+0000","last_fullsized":"2024-04-23T10:26:11.116657+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T13:45:26.793330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'9","reported_seq":158,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:15.163214+0000","last_change":"2024-04-23T10:24:06.575408+0000","last_active":"2024-04-23T10:26:15.163214+0000","last_peered":"2024-04-23T10:26:15.163214+0000","last_clean":"2024-04-23T10:26:15.163214+0000","last_became_active":"2024-04-23T10:24:06.575081+0000","last_became_peered":"2024-04-23T10:24:06.575081+0000","last_unstale":"2024-04-23T10:26:15.163214+0000","last_undegraded":"2024-04-23T10:26:15.163214+0000","last_fullsized":"2024-04-23T10:26:15.163214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T11:28:12.765196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2484,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":128,"num_read_kb":4,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.116971+0000","last_change":"2024-04-23T10:24:07.043869+0000","last_active":"2024-04-23T10:26:11.116971+0000","last_peered":"2024-04-23T10:26:11.116971+0000","last_clean":"2024-04-23T10:26:11.116971+0000","last_became_active":"2024-04-23T10:24:07.043448+0000","last_became_peered":"2024-04-23T10:24:07.043448+0000","last_unstale":"2024-04-23T10:26:11.116971+0000","last_undegraded":"2024-04-23T10:26:11.116971+0000","last_fullsized":"2024-04-23T10:26:11.116971+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:00:35.821470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'41","reported_seq":70,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.310840+0000","last_change":"2024-04-23T10:24:06.575259+0000","last_active":"2024-04-23T10:26:11.310840+0000","last_peered":"2024-04-23T10:26:11.310840+0000","last_clean":"2024-04-23T10:26:11.310840+0000","last_became_active":"2024-04-23T10:24:06.574920+0000","last_became_peered":"2024-04-23T10:24:06.574920+0000","last_unstale":"2024-04-23T10:26:11.310840+0000","last_undegraded":"2024-04-23T10:26:11.310840+0000","last_fullsized":"2024-04-23T10:26:11.310840+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:52:14.558458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.116672+0000","last_change":"2024-04-23T10:24:07.043238+0000","last_active":"2024-04-23T10:26:11.116672+0000","last_peered":"2024-04-23T10:26:11.116672+0000","last_clean":"2024-04-23T10:26:11.116672+0000","last_became_active":"2024-04-23T10:24:07.042110+0000","last_became_peered":"2024-04-23T10:24:07.042110+0000","last_unstale":"2024-04-23T10:26:11.116672+0000","last_undegraded":"2024-04-23T10:26:11.116672+0000","last_fullsized":"2024-04-23T10:26:11.116672+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T12:36:13.474456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.130204+0000","last_change":"2024-04-23T10:24:06.579233+0000","last_active":"2024-04-23T10:26:11.130204+0000","last_peered":"2024-04-23T10:26:11.130204+0000","last_clean":"2024-04-23T10:26:11.130204+0000","last_became_active":"2024-04-23T10:24:06.578849+0000","last_became_peered":"2024-04-23T10:24:06.578849+0000","last_unstale":"2024-04-23T10:26:11.130204+0000","last_undegraded":"2024-04-23T10:26:11.130204+0000","last_fullsized":"2024-04-23T10:26:11.130204+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:05.545929+0000","last_clean_scrub_stamp":"2024-04-23T10:24:05.545929+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:34:23.193707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.116824+0000","last_change":"2024-04-23T10:24:07.049840+0000","last_active":"2024-04-23T10:26:11.116824+0000","last_peered":"2024-04-23T10:26:11.116824+0000","last_clean":"2024-04-23T10:26:11.116824+0000","last_became_active":"2024-04-23T10:24:07.049493+0000","last_became_peered":"2024-04-23T10:24:07.049493+0000","last_unstale":"2024-04-23T10:26:11.116824+0000","last_undegraded":"2024-04-23T10:26:11.116824+0000","last_fullsized":"2024-04-23T10:26:11.116824+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:03:22.606800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.355883+0000","last_change":"2024-04-23T10:24:07.052332+0000","last_active":"2024-04-23T10:26:11.355883+0000","last_peered":"2024-04-23T10:26:11.355883+0000","last_clean":"2024-04-23T10:26:11.355883+0000","last_became_active":"2024-04-23T10:24:07.051956+0000","last_became_peered":"2024-04-23T10:24:07.051956+0000","last_unstale":"2024-04-23T10:26:11.355883+0000","last_undegraded":"2024-04-23T10:26:11.355883+0000","last_fullsized":"2024-04-23T10:26:11.355883+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T15:56:43.606916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.116258+0000","last_change":"2024-04-23T10:24:07.041897+0000","last_active":"2024-04-23T10:26:11.116258+0000","last_peered":"2024-04-23T10:26:11.116258+0000","last_clean":"2024-04-23T10:26:11.116258+0000","last_became_active":"2024-04-23T10:24:07.041546+0000","last_became_peered":"2024-04-23T10:24:07.041546+0000","last_unstale":"2024-04-23T10:26:11.116258+0000","last_undegraded":"2024-04-23T10:26:11.116258+0000","last_fullsized":"2024-04-23T10:26:11.116258+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T11:31:23.211045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.116257+0000","last_change":"2024-04-23T10:24:07.040764+0000","last_active":"2024-04-23T10:26:11.116257+0000","last_peered":"2024-04-23T10:26:11.116257+0000","last_clean":"2024-04-23T10:26:11.116257+0000","last_became_active":"2024-04-23T10:24:07.039637+0000","last_became_peered":"2024-04-23T10:24:07.039637+0000","last_unstale":"2024-04-23T10:26:11.116257+0000","last_undegraded":"2024-04-23T10:26:11.116257+0000","last_fullsized":"2024-04-23T10:26:11.116257+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:50:33.298770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.350168+0000","last_change":"2024-04-23T10:24:07.039491+0000","last_active":"2024-04-23T10:26:11.350168+0000","last_peered":"2024-04-23T10:26:11.350168+0000","last_clean":"2024-04-23T10:26:11.350168+0000","last_became_active":"2024-04-23T10:24:07.039128+0000","last_became_peered":"2024-04-23T10:24:07.039128+0000","last_unstale":"2024-04-23T10:26:11.350168+0000","last_undegraded":"2024-04-23T10:26:11.350168+0000","last_fullsized":"2024-04-23T10:26:11.350168+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T12:30:40.326031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.130013+0000","last_change":"2024-04-23T10:25:00.671059+0000","last_active":"2024-04-23T10:26:11.130013+0000","last_peered":"2024-04-23T10:26:11.130013+0000","last_clean":"2024-04-23T10:26:11.130013+0000","last_became_active":"2024-04-23T10:25:00.670686+0000","last_became_peered":"2024-04-23T10:25:00.670686+0000","last_unstale":"2024-04-23T10:26:11.130013+0000","last_undegraded":"2024-04-23T10:26:11.130013+0000","last_fullsized":"2024-04-23T10:26:11.130013+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T17:04:39.991647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.355901+0000","last_change":"2024-04-23T10:24:07.052395+0000","last_active":"2024-04-23T10:26:11.355901+0000","last_peered":"2024-04-23T10:26:11.355901+0000","last_clean":"2024-04-23T10:26:11.355901+0000","last_became_active":"2024-04-23T10:24:07.052023+0000","last_became_peered":"2024-04-23T10:24:07.052023+0000","last_unstale":"2024-04-23T10:26:11.355901+0000","last_undegraded":"2024-04-23T10:26:11.355901+0000","last_fullsized":"2024-04-23T10:26:11.355901+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:25:52.517267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.116425+0000","last_change":"2024-04-23T10:24:07.042739+0000","last_active":"2024-04-23T10:26:11.116425+0000","last_peered":"2024-04-23T10:26:11.116425+0000","last_clean":"2024-04-23T10:26:11.116425+0000","last_became_active":"2024-04-23T10:24:07.042292+0000","last_became_peered":"2024-04-23T10:24:07.042292+0000","last_unstale":"2024-04-23T10:26:11.116425+0000","last_undegraded":"2024-04-23T10:26:11.116425+0000","last_fullsized":"2024-04-23T10:26:11.116425+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:14:23.229032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.132286+0000","last_change":"2024-04-23T10:24:07.042792+0000","last_active":"2024-04-23T10:26:11.132286+0000","last_peered":"2024-04-23T10:26:11.132286+0000","last_clean":"2024-04-23T10:26:11.132286+0000","last_became_active":"2024-04-23T10:24:07.040846+0000","last_became_peered":"2024-04-23T10:24:07.040846+0000","last_unstale":"2024-04-23T10:26:11.132286+0000","last_undegraded":"2024-04-23T10:26:11.132286+0000","last_fullsized":"2024-04-23T10:26:11.132286+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T13:50:24.973286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.127085+0000","last_change":"2024-04-23T10:24:07.040391+0000","last_active":"2024-04-23T10:26:11.127085+0000","last_peered":"2024-04-23T10:26:11.127085+0000","last_clean":"2024-04-23T10:26:11.127085+0000","last_became_active":"2024-04-23T10:24:07.040002+0000","last_became_peered":"2024-04-23T10:24:07.040002+0000","last_unstale":"2024-04-23T10:26:11.127085+0000","last_undegraded":"2024-04-23T10:26:11.127085+0000","last_fullsized":"2024-04-23T10:26:11.127085+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T10:30:13.922315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.131939+0000","last_change":"2024-04-23T10:24:07.040678+0000","last_active":"2024-04-23T10:26:11.131939+0000","last_peered":"2024-04-23T10:26:11.131939+0000","last_clean":"2024-04-23T10:26:11.131939+0000","last_became_active":"2024-04-23T10:24:07.039551+0000","last_became_peered":"2024-04-23T10:24:07.039551+0000","last_unstale":"2024-04-23T10:26:11.131939+0000","last_undegraded":"2024-04-23T10:26:11.131939+0000","last_fullsized":"2024-04-23T10:26:11.131939+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T16:03:50.467083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.349510+0000","last_change":"2024-04-23T10:24:07.044618+0000","last_active":"2024-04-23T10:26:11.349510+0000","last_peered":"2024-04-23T10:26:11.349510+0000","last_clean":"2024-04-23T10:26:11.349510+0000","last_became_active":"2024-04-23T10:24:07.042803+0000","last_became_peered":"2024-04-23T10:24:07.042803+0000","last_unstale":"2024-04-23T10:26:11.349510+0000","last_undegraded":"2024-04-23T10:26:11.349510+0000","last_fullsized":"2024-04-23T10:26:11.349510+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:37:46.872177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.129943+0000","last_change":"2024-04-23T10:24:07.043724+0000","last_active":"2024-04-23T10:26:11.129943+0000","last_peered":"2024-04-23T10:26:11.129943+0000","last_clean":"2024-04-23T10:26:11.129943+0000","last_became_active":"2024-04-23T10:24:07.043408+0000","last_became_peered":"2024-04-23T10:24:07.043408+0000","last_unstale":"2024-04-23T10:26:11.129943+0000","last_undegraded":"2024-04-23T10:26:11.129943+0000","last_fullsized":"2024-04-23T10:26:11.129943+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T14:18:46.615093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-23T10:26:11.355748+0000","last_change":"2024-04-23T10:24:07.043794+0000","last_active":"2024-04-23T10:26:11.355748+0000","last_peered":"2024-04-23T10:26:11.355748+0000","last_clean":"2024-04-23T10:26:11.355748+0000","last_became_active":"2024-04-23T10:24:07.043391+0000","last_became_peered":"2024-04-23T10:24:07.043391+0000","last_unstale":"2024-04-23T10:26:11.355748+0000","last_undegraded":"2024-04-23T10:26:11.355748+0000","last_fullsized":"2024-04-23T10:26:11.355748+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T10:24:06.548166+0000","last_clean_scrub_stamp":"2024-04-23T10:24:06.548166+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:06:04.584000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":106845,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":602547,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1583,"num_read_kb":998,"num_write":479,"num_write_kb":1920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1254,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":741376,"data_stored":2007575,"data_compressed":90949,"data_compressed_allocated":122880,"data_compressed_original":1757184,"omap_allocated":0,"internal_metadata":0},"log_size":360,"ondisk_log_size":360,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351015,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":974848,"data_stored":470565,"data_compressed":2902,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612651,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28144,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717008,"statfs":{"total":95995035648,"available":95966216192,"internally_reserved":0,"allocated":1150976,"data_stored":1259979,"data_compressed":46529,"data_compressed_allocated":61440,"data_compressed_original":729088,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776177,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":861815,"data_compressed":22191,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037815,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":860939,"data_compressed":22191,"data_compressed_allocated":28672,"data_compressed_original":401408,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299451,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1044480,"data_stored":1063341,"data_compressed":38517,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462976,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":978944,"data_stored":469098,"data_compressed":2902,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1044480,"data_stored":537769,"data_compressed":4758,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738463,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":999424,"data_stored":470969,"data_compressed":2902,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1069056,"data_stored":542355,"data_compressed":4758,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149777,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1064960,"data_stored":1189589,"data_compressed":37393,"data_compressed_allocated":49152,"data_compressed_original":708608,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888138,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":987136,"data_stored":469772,"data_compressed":2902,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724612,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32240,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712912,"statfs":{"total":95995035648,"available":95962021888,"internally_reserved":0,"allocated":1077248,"data_stored":1192914,"data_compressed":44673,"data_compressed_allocated":57344,"data_compressed_original":720896,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35615,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35615,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35615,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":70292,"data_compressed":1856,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":74878,"data_compressed":1856,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":721878,"data_compressed":34491,"data_compressed_allocated":45056,"data_compressed_original":700416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":134952,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":393462,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":202017,"data_compressed":8012,"data_compressed_allocated":12288,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T10:26:17.483 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T10:26:17.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.619+0000 7f72d687c640 1 -- 172.21.15.97:0/1607500680 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72d0101a50 msgr2=0x7f72d0105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:17.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.619+0000 7f72d687c640 1 --2- 172.21.15.97:0/1607500680 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72d0101a50 0x7f72d0105a90 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f72c0009900 tx=0x7f72c002a410 comp rx=0 tx=0).stop 2024-04-23T10:26:17.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.619+0000 7f72d687c640 1 -- 172.21.15.97:0/1607500680 shutdown_connections 2024-04-23T10:26:17.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.619+0000 7f72d687c640 1 --2- 172.21.15.97:0/1607500680 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f72d0106170 0x7f72d010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:17.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.619+0000 7f72d687c640 1 --2- 172.21.15.97:0/1607500680 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72d0101a50 0x7f72d0105a90 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:17.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.619+0000 7f72d687c640 1 --2- 172.21.15.97:0/1607500680 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72d01010a0 0x7f72d0101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:17.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.619+0000 7f72d687c640 1 -- 172.21.15.97:0/1607500680 >> 172.21.15.97:0/1607500680 conn(0x7f72d0077fe0 msgr2=0x7f72d00783f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:17.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.619+0000 7f72d687c640 1 -- 172.21.15.97:0/1607500680 shutdown_connections 2024-04-23T10:26:17.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.619+0000 7f72d687c640 1 -- 172.21.15.97:0/1607500680 wait complete. 2024-04-23T10:26:17.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.619+0000 7f72d687c640 1 Processor -- start 2024-04-23T10:26:17.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.619+0000 7f72d687c640 1 -- start start 2024-04-23T10:26:17.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.620+0000 7f72d687c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f72d01010a0 0x7f72d0198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:17.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.620+0000 7f72d687c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72d0106170 0x7f72d0198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:17.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.620+0000 7f72d687c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72d019d020 0x7f72d019d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:17.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.620+0000 7f72d687c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f72d0111090 con 0x7f72d0106170 2024-04-23T10:26:17.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.620+0000 7f72d687c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f72d0110f10 con 0x7f72d019d020 2024-04-23T10:26:17.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.620+0000 7f72d687c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f72d0111210 con 0x7f72d01010a0 2024-04-23T10:26:17.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.620+0000 7f72cf7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72d0106170 0x7f72d0198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:17.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.620+0000 7f72cf7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72d0106170 0x7f72d0198ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46984/0 (socket says 172.21.15.97:46984) 2024-04-23T10:26:17.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.620+0000 7f72cf7fe640 1 -- 172.21.15.97:0/1768413598 learned_addr learned my addr 172.21.15.97:0/1768413598 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:17.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.620+0000 7f72cffff640 1 --2- 172.21.15.97:0/1768413598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f72d01010a0 0x7f72d0198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:17.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.620+0000 7f72d4e19640 1 --2- 172.21.15.97:0/1768413598 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72d019d020 0x7f72d019d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:17.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.620+0000 7f72cf7fe640 1 -- 172.21.15.97:0/1768413598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f72d01010a0 msgr2=0x7f72d0198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:17.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.620+0000 7f72cf7fe640 1 --2- 172.21.15.97:0/1768413598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f72d01010a0 0x7f72d0198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:17.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.620+0000 7f72cf7fe640 1 -- 172.21.15.97:0/1768413598 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72d019d020 msgr2=0x7f72d019d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:17.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.620+0000 7f72cf7fe640 1 --2- 172.21.15.97:0/1768413598 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72d019d020 0x7f72d019d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:17.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.620+0000 7f72cf7fe640 1 -- 172.21.15.97:0/1768413598 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72d019db10 con 0x7f72d0106170 2024-04-23T10:26:17.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.620+0000 7f72cffff640 1 --2- 172.21.15.97:0/1768413598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f72d01010a0 0x7f72d0198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:26:17.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.620+0000 7f72cf7fe640 1 --2- 172.21.15.97:0/1768413598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72d0106170 0x7f72d0198ca0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f72c0006010 tx=0x7f72c0004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:17.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.620+0000 7f72cd7fa640 1 -- 172.21.15.97:0/1768413598 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c003f030 con 0x7f72d0106170 2024-04-23T10:26:17.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.620+0000 7f72cd7fa640 1 -- 172.21.15.97:0/1768413598 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72c0003aa0 con 0x7f72d0106170 2024-04-23T10:26:17.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.620+0000 7f72d687c640 1 -- 172.21.15.97:0/1768413598 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72d019dda0 con 0x7f72d0106170 2024-04-23T10:26:17.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.620+0000 7f72cd7fa640 1 -- 172.21.15.97:0/1768413598 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c0035070 con 0x7f72d0106170 2024-04-23T10:26:17.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.621+0000 7f72d687c640 1 -- 172.21.15.97:0/1768413598 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72d01a5560 con 0x7f72d0106170 2024-04-23T10:26:17.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.622+0000 7f72d687c640 1 -- 172.21.15.97:0/1768413598 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72d00691c0 con 0x7f72d0106170 2024-04-23T10:26:17.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.623+0000 7f72cd7fa640 1 -- 172.21.15.97:0/1768413598 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f72c0042030 con 0x7f72d0106170 2024-04-23T10:26:17.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.623+0000 7f72cd7fa640 1 --2- 172.21.15.97:0/1768413598 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f72b80a3e40 0x7f72b80a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:17.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.624+0000 7f72cffff640 1 --2- 172.21.15.97:0/1768413598 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f72b80a3e40 0x7f72b80a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:17.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.624+0000 7f72cd7fa640 1 -- 172.21.15.97:0/1768413598 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f72c00f12a0 con 0x7f72d0106170 2024-04-23T10:26:17.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.624+0000 7f72cffff640 1 --2- 172.21.15.97:0/1768413598 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f72b80a3e40 0x7f72b80a6300 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f72c4009b50 tx=0x7f72c4005f90 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:17.626 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.627+0000 7f72cd7fa640 1 -- 172.21.15.97:0/1768413598 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f72c00bd810 con 0x7f72d0106170 2024-04-23T10:26:17.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.843+0000 7f72d687c640 1 -- 172.21.15.97:0/1768413598 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f72d0199ab0 con 0x7f72d0106170 2024-04-23T10:26:17.843 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.844+0000 7f72cd7fa640 1 -- 172.21.15.97:0/1768413598 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11931 (secure 0 0 0) 0x7f72d0199ab0 con 0x7f72d0106170 2024-04-23T10:26:17.843 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:26:17.843 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3946878762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3946878762},{"type":"v1","addr":"172.21.15.97:6835","nonce":3946878762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/209800861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":209800861},{"type":"v1","addr":"172.21.15.97:6837","nonce":209800861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T10:24:06.910134+0000","modified":"2024-04-23T10:26:09.820985+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6839/418013575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":418013575},{"type":"v1","addr":"172.21.15.97:6839","nonce":418013575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6841/747242680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":747242680},{"type":"v1","addr":"172.21.15.97:6841","nonce":747242680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6833/4292847121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4292847121},{"type":"v1","addr":"172.21.15.183:6833","nonce":4292847121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6833/3885816565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3885816565},{"type":"v1","addr":"172.21.15.196:6833","nonce":3885816565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1897093105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1897093105},{"type":"v1","addr":"172.21.15.183:6835","nonce":1897093105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1171477563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1171477563},{"type":"v1","addr":"172.21.15.196:6835","nonce":1171477563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1898341911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1898341911},{"type":"v1","addr":"172.21.15.183:6837","nonce":1898341911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.196:6837/3094946502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3094946502},{"type":"v1","addr":"172.21.15.196:6837","nonce":3094946502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.196:6839/83843556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":83843556},{"type":"v1","addr":"172.21.15.196:6839","nonce":83843556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6839/1763841130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1763841130},{"type":"v1","addr":"172.21.15.183:6839","nonce":1763841130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24487,24403,24398,24469,24488]},"id":1}]} 2024-04-23T10:26:17.843 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 26 2024-04-23T10:26:17.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.846+0000 7f72d687c640 1 -- 172.21.15.97:0/1768413598 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f72b80a3e40 msgr2=0x7f72b80a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:17.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.846+0000 7f72d687c640 1 --2- 172.21.15.97:0/1768413598 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f72b80a3e40 0x7f72b80a6300 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f72c4009b50 tx=0x7f72c4005f90 comp rx=0 tx=0).stop 2024-04-23T10:26:17.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.846+0000 7f72d687c640 1 -- 172.21.15.97:0/1768413598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72d0106170 msgr2=0x7f72d0198ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:17.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.846+0000 7f72d687c640 1 --2- 172.21.15.97:0/1768413598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72d0106170 0x7f72d0198ca0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f72c0006010 tx=0x7f72c0004060 comp rx=0 tx=0).stop 2024-04-23T10:26:17.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.846+0000 7f72d687c640 1 -- 172.21.15.97:0/1768413598 shutdown_connections 2024-04-23T10:26:17.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.846+0000 7f72d687c640 1 --2- 172.21.15.97:0/1768413598 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f72b80a3e40 0x7f72b80a6300 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:17.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.846+0000 7f72d687c640 1 --2- 172.21.15.97:0/1768413598 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72d019d020 0x7f72d019d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:17.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.846+0000 7f72d687c640 1 --2- 172.21.15.97:0/1768413598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72d0106170 0x7f72d0198ca0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:17.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.846+0000 7f72d687c640 1 --2- 172.21.15.97:0/1768413598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f72d01010a0 0x7f72d0198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:17.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.846+0000 7f72d687c640 1 -- 172.21.15.97:0/1768413598 >> 172.21.15.97:0/1768413598 conn(0x7f72d0077fe0 msgr2=0x7f72d006aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:17.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.846+0000 7f72d687c640 1 -- 172.21.15.97:0/1768413598 shutdown_connections 2024-04-23T10:26:17.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.846+0000 7f72d687c640 1 -- 172.21.15.97:0/1768413598 wait complete. 2024-04-23T10:26:17.856 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T10:26:17.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.956+0000 7fc375443640 1 -- 172.21.15.97:0/3655015220 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc370103b00 msgr2=0x7fc37010bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:17.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.956+0000 7fc375443640 1 --2- 172.21.15.97:0/3655015220 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc370103b00 0x7fc37010bfe0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fc364009900 tx=0x7fc36402a510 comp rx=0 tx=0).stop 2024-04-23T10:26:17.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.956+0000 7fc375443640 1 -- 172.21.15.97:0/3655015220 shutdown_connections 2024-04-23T10:26:17.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.956+0000 7fc375443640 1 --2- 172.21.15.97:0/3655015220 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc37010c520 0x7fc37010e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:17.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.956+0000 7fc375443640 1 --2- 172.21.15.97:0/3655015220 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc370103b00 0x7fc37010bfe0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:17.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.956+0000 7fc375443640 1 --2- 172.21.15.97:0/3655015220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3701031e0 0x7fc3701035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:17.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.956+0000 7fc375443640 1 -- 172.21.15.97:0/3655015220 >> 172.21.15.97:0/3655015220 conn(0x7fc370077f50 msgr2=0x7fc370078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:17.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.956+0000 7fc375443640 1 -- 172.21.15.97:0/3655015220 shutdown_connections 2024-04-23T10:26:17.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.956+0000 7fc375443640 1 -- 172.21.15.97:0/3655015220 wait complete. 2024-04-23T10:26:17.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.957+0000 7fc375443640 1 Processor -- start 2024-04-23T10:26:17.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.957+0000 7fc375443640 1 -- start start 2024-04-23T10:26:17.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.957+0000 7fc375443640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3701031e0 0x7fc37019e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:17.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.957+0000 7fc375443640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc370103b00 0x7fc37019e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:17.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.957+0000 7fc375443640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc37010c520 0x7fc370198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:17.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.957+0000 7fc375443640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc3701121b0 con 0x7fc370103b00 2024-04-23T10:26:17.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.957+0000 7fc375443640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc370112030 con 0x7fc3701031e0 2024-04-23T10:26:17.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.957+0000 7fc375443640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc370112330 con 0x7fc37010c520 2024-04-23T10:26:17.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.957+0000 7fc36e7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc370103b00 0x7fc37019e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:17.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.958+0000 7fc36e7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc370103b00 0x7fc37019e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47004/0 (socket says 172.21.15.97:47004) 2024-04-23T10:26:17.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.958+0000 7fc36e7fc640 1 -- 172.21.15.97:0/1388590846 learned_addr learned my addr 172.21.15.97:0/1388590846 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:17.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.958+0000 7fc36effd640 1 --2- 172.21.15.97:0/1388590846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3701031e0 0x7fc37019e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:17.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.958+0000 7fc36f7fe640 1 --2- 172.21.15.97:0/1388590846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc37010c520 0x7fc370198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:17.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.958+0000 7fc36e7fc640 1 -- 172.21.15.97:0/1388590846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc37010c520 msgr2=0x7fc370198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:17.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.958+0000 7fc36e7fc640 1 --2- 172.21.15.97:0/1388590846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc37010c520 0x7fc370198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:17.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.958+0000 7fc36e7fc640 1 -- 172.21.15.97:0/1388590846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3701031e0 msgr2=0x7fc37019e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:17.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.958+0000 7fc36e7fc640 1 --2- 172.21.15.97:0/1388590846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3701031e0 0x7fc37019e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:17.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.958+0000 7fc36e7fc640 1 -- 172.21.15.97:0/1388590846 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc370198d10 con 0x7fc370103b00 2024-04-23T10:26:17.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.958+0000 7fc36f7fe640 1 --2- 172.21.15.97:0/1388590846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc37010c520 0x7fc370198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:26:17.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.958+0000 7fc36e7fc640 1 --2- 172.21.15.97:0/1388590846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc370103b00 0x7fc37019e920 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fc364005940 tx=0x7fc3640042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:17.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.958+0000 7fc357fff640 1 -- 172.21.15.97:0/1388590846 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc36403f030 con 0x7fc370103b00 2024-04-23T10:26:17.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.958+0000 7fc375443640 1 -- 172.21.15.97:0/1388590846 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc370198fa0 con 0x7fc370103b00 2024-04-23T10:26:17.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.958+0000 7fc357fff640 1 -- 172.21.15.97:0/1388590846 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc364003cb0 con 0x7fc370103b00 2024-04-23T10:26:17.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.958+0000 7fc375443640 1 -- 172.21.15.97:0/1388590846 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3701a52a0 con 0x7fc370103b00 2024-04-23T10:26:17.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.958+0000 7fc357fff640 1 -- 172.21.15.97:0/1388590846 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc364049060 con 0x7fc370103b00 2024-04-23T10:26:17.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.959+0000 7fc375443640 1 -- 172.21.15.97:0/1388590846 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc370068d70 con 0x7fc370103b00 2024-04-23T10:26:17.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.961+0000 7fc357fff640 1 -- 172.21.15.97:0/1388590846 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fc364042030 con 0x7fc370103b00 2024-04-23T10:26:17.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.961+0000 7fc357fff640 1 --2- 172.21.15.97:0/1388590846 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fc3580a3e40 0x7fc3580a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:17.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.962+0000 7fc36effd640 1 --2- 172.21.15.97:0/1388590846 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fc3580a3e40 0x7fc3580a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:17.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.962+0000 7fc357fff640 1 -- 172.21.15.97:0/1388590846 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fc3640f14c0 con 0x7fc370103b00 2024-04-23T10:26:17.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.962+0000 7fc36effd640 1 --2- 172.21.15.97:0/1388590846 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fc3580a3e40 0x7fc3580a6300 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc3600058c0 tx=0x7fc36000a250 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:17.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:17.965+0000 7fc357fff640 1 -- 172.21.15.97:0/1388590846 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc3640bd9d0 con 0x7fc370103b00 2024-04-23T10:26:18.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:17 smithi097 ceph-mon[29197]: from='client.15534 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T10:26:18.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:17 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/1768413598' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:26:18.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.147+0000 7fc375443640 1 -- 172.21.15.97:0/1388590846 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc370199ac0 con 0x7fc370103b00 2024-04-23T10:26:18.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.148+0000 7fc357fff640 1 -- 172.21.15.97:0/1388590846 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22981 (secure 0 0 0) 0x7fc370199ac0 con 0x7fc370103b00 2024-04-23T10:26:18.148 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:26:18.148 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":89,"fsid":"9b22af88-0159-11ef-bc93-c7b262605968","created":"2024-04-23T10:16:05.983305+0000","modified":"2024-04-23T10:26:11.109574+0000","last_up_change":"2024-04-23T10:23:13.114761+0000","last_in_change":"2024-04-23T10:22:52.804880+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T10:19:25.559138+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T10:24:04.570049+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T10:24:05.861631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"58cb995d-de35-4522-ac81-4982401b260d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6803","nonce":3031221802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6805","nonce":3031221802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6809","nonce":3031221802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6807","nonce":3031221802}]},"public_addr":"172.21.15.97:6803/3031221802","cluster_addr":"172.21.15.97:6805/3031221802","heartbeat_back_addr":"172.21.15.97:6809/3031221802","heartbeat_front_addr":"172.21.15.97:6807/3031221802","state":["exists","up"]},{"osd":1,"uuid":"b47bd8ee-1f21-4834-acf5-bb40f97ab3ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6801","nonce":1108870440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6803","nonce":1108870440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6807","nonce":1108870440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6805","nonce":1108870440}]},"public_addr":"172.21.15.183:6801/1108870440","cluster_addr":"172.21.15.183:6803/1108870440","heartbeat_back_addr":"172.21.15.183:6807/1108870440","heartbeat_front_addr":"172.21.15.183:6805/1108870440","state":["exists","up"]},{"osd":2,"uuid":"17c4eec3-6c47-4334-9b63-1d5ad9ec5988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6801","nonce":2877039731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6803","nonce":2877039731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6807","nonce":2877039731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6805","nonce":2877039731}]},"public_addr":"172.21.15.196:6801/2877039731","cluster_addr":"172.21.15.196:6803/2877039731","heartbeat_back_addr":"172.21.15.196:6807/2877039731","heartbeat_front_addr":"172.21.15.196:6805/2877039731","state":["exists","up"]},{"osd":3,"uuid":"3f731c38-2e45-4ca1-8dce-efc4a1b2166b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6811","nonce":4276311194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6813","nonce":4276311194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6817","nonce":4276311194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6815","nonce":4276311194}]},"public_addr":"172.21.15.97:6811/4276311194","cluster_addr":"172.21.15.97:6813/4276311194","heartbeat_back_addr":"172.21.15.97:6817/4276311194","heartbeat_front_addr":"172.21.15.97:6815/4276311194","state":["exists","up"]},{"osd":4,"uuid":"fdcafb14-9b82-4134-bd5b-79c0516f6cbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6809","nonce":63913916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6811","nonce":63913916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6815","nonce":63913916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6813","nonce":63913916}]},"public_addr":"172.21.15.183:6809/63913916","cluster_addr":"172.21.15.183:6811/63913916","heartbeat_back_addr":"172.21.15.183:6815/63913916","heartbeat_front_addr":"172.21.15.183:6813/63913916","state":["exists","up"]},{"osd":5,"uuid":"cdcc18d8-da3c-4012-bf12-876bb39e66eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6809","nonce":2804446896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6811","nonce":2804446896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6815","nonce":2804446896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6813","nonce":2804446896}]},"public_addr":"172.21.15.196:6809/2804446896","cluster_addr":"172.21.15.196:6811/2804446896","heartbeat_back_addr":"172.21.15.196:6815/2804446896","heartbeat_front_addr":"172.21.15.196:6813/2804446896","state":["exists","up"]},{"osd":6,"uuid":"071abb5c-2edf-4832-b8aa-3b80d20096fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6819","nonce":2080916705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6821","nonce":2080916705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6825","nonce":2080916705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6823","nonce":2080916705}]},"public_addr":"172.21.15.97:6819/2080916705","cluster_addr":"172.21.15.97:6821/2080916705","heartbeat_back_addr":"172.21.15.97:6825/2080916705","heartbeat_front_addr":"172.21.15.97:6823/2080916705","state":["exists","up"]},{"osd":7,"uuid":"9d54760b-66df-486b-8abb-017f3b64b4d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6817","nonce":1865873033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6819","nonce":1865873033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6823","nonce":1865873033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6821","nonce":1865873033}]},"public_addr":"172.21.15.183:6817/1865873033","cluster_addr":"172.21.15.183:6819/1865873033","heartbeat_back_addr":"172.21.15.183:6823/1865873033","heartbeat_front_addr":"172.21.15.183:6821/1865873033","state":["exists","up"]},{"osd":8,"uuid":"5cd3100a-bd6a-4be5-b5e9-0d0afc325193","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6817","nonce":4063932139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6819","nonce":4063932139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6823","nonce":4063932139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6821","nonce":4063932139}]},"public_addr":"172.21.15.196:6817/4063932139","cluster_addr":"172.21.15.196:6819/4063932139","heartbeat_back_addr":"172.21.15.196:6823/4063932139","heartbeat_front_addr":"172.21.15.196:6821/4063932139","state":["exists","up"]},{"osd":9,"uuid":"b6a5f4a1-4ecc-4f73-a2df-07551d9298ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6827","nonce":1887432474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6829","nonce":1887432474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6833","nonce":1887432474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6831","nonce":1887432474}]},"public_addr":"172.21.15.97:6827/1887432474","cluster_addr":"172.21.15.97:6829/1887432474","heartbeat_back_addr":"172.21.15.97:6833/1887432474","heartbeat_front_addr":"172.21.15.97:6831/1887432474","state":["exists","up"]},{"osd":10,"uuid":"86f59816-1ea6-4ae9-8796-cf5fc4831064","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6825","nonce":799434647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6827","nonce":799434647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6831","nonce":799434647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6829","nonce":799434647}]},"public_addr":"172.21.15.183:6825/799434647","cluster_addr":"172.21.15.183:6827/799434647","heartbeat_back_addr":"172.21.15.183:6831/799434647","heartbeat_front_addr":"172.21.15.183:6829/799434647","state":["exists","up"]},{"osd":11,"uuid":"95d64a89-a9f8-4c7a-8c0d-da7fb26cfff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6825","nonce":1873969467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6827","nonce":1873969467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6831","nonce":1873969467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6829","nonce":1873969467}]},"public_addr":"172.21.15.196:6825/1873969467","cluster_addr":"172.21.15.196:6827/1873969467","heartbeat_back_addr":"172.21.15.196:6831/1873969467","heartbeat_front_addr":"172.21.15.196:6829/1873969467","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:18:33.721156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:18:57.586709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:19:22.069458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:19:46.451597+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:20:11.561019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:20:36.298672+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:21:01.661804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:21:27.324128+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:21:52.898197+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:22:19.614326+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:22:45.154917+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:23:11.222612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/233335470":"2024-04-24T10:26:11.109535+0000","172.21.15.97:6800/3149043749":"2024-04-24T10:26:11.109535+0000","172.21.15.97:0/338277142":"2024-04-24T10:26:11.109535+0000","172.21.15.97:0/3866016040":"2024-04-24T10:16:59.468237+0000","172.21.15.97:6801/3297166858":"2024-04-24T10:16:59.468237+0000","172.21.15.97:0/3804743548":"2024-04-24T10:16:59.468237+0000","172.21.15.97:6800/3706463017":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/3715079438":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/597405418":"2024-04-24T10:26:11.109535+0000","172.21.15.97:0/786806834":"2024-04-24T10:16:59.468237+0000","172.21.15.97:6801/3706463017":"2024-04-24T10:16:33.128917+0000","172.21.15.97:6801/3149043749":"2024-04-24T10:26:11.109535+0000","172.21.15.97:0/3294175554":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/1449136760":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/3198144747":"2024-04-24T10:26:11.109535+0000","172.21.15.97:6800/3297166858":"2024-04-24T10:16:59.468237+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T10:26:18.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.150+0000 7fc375443640 1 -- 172.21.15.97:0/1388590846 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fc3580a3e40 msgr2=0x7fc3580a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:18.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.150+0000 7fc375443640 1 --2- 172.21.15.97:0/1388590846 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fc3580a3e40 0x7fc3580a6300 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc3600058c0 tx=0x7fc36000a250 comp rx=0 tx=0).stop 2024-04-23T10:26:18.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.150+0000 7fc375443640 1 -- 172.21.15.97:0/1388590846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc370103b00 msgr2=0x7fc37019e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:18.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.150+0000 7fc375443640 1 --2- 172.21.15.97:0/1388590846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc370103b00 0x7fc37019e920 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fc364005940 tx=0x7fc3640042f0 comp rx=0 tx=0).stop 2024-04-23T10:26:18.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.150+0000 7fc375443640 1 -- 172.21.15.97:0/1388590846 shutdown_connections 2024-04-23T10:26:18.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.150+0000 7fc375443640 1 --2- 172.21.15.97:0/1388590846 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fc3580a3e40 0x7fc3580a6300 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:18.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.150+0000 7fc375443640 1 --2- 172.21.15.97:0/1388590846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc37010c520 0x7fc370198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:18.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.150+0000 7fc375443640 1 --2- 172.21.15.97:0/1388590846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc370103b00 0x7fc37019e920 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:18.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.150+0000 7fc375443640 1 --2- 172.21.15.97:0/1388590846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3701031e0 0x7fc37019e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:18.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.150+0000 7fc375443640 1 -- 172.21.15.97:0/1388590846 >> 172.21.15.97:0/1388590846 conn(0x7fc370077f50 msgr2=0x7fc3700fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:18.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.151+0000 7fc375443640 1 -- 172.21.15.97:0/1388590846 shutdown_connections 2024-04-23T10:26:18.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.151+0000 7fc375443640 1 -- 172.21.15.97:0/1388590846 wait complete. 2024-04-23T10:26:18.161 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T10:19:25.559138+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T10:24:04.570049+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T10:24:05.861631+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T10:26:18.161 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T10:26:18.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.264+0000 7f850e520640 1 -- 172.21.15.97:0/2379728665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8508106170 msgr2=0x7f850810dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:18.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.264+0000 7f850e520640 1 --2- 172.21.15.97:0/2379728665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8508106170 0x7f850810dd40 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f8500009ca0 tx=0x7f850002a8c0 comp rx=0 tx=0).stop 2024-04-23T10:26:18.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.264+0000 7f850e520640 1 -- 172.21.15.97:0/2379728665 shutdown_connections 2024-04-23T10:26:18.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.264+0000 7f850e520640 1 --2- 172.21.15.97:0/2379728665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8508106170 0x7f850810dd40 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:18.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.264+0000 7f850e520640 1 --2- 172.21.15.97:0/2379728665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8508101a50 0x7f8508105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:18.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.264+0000 7f850e520640 1 --2- 172.21.15.97:0/2379728665 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f85081010a0 0x7f8508101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:18.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.264+0000 7f850e520640 1 -- 172.21.15.97:0/2379728665 >> 172.21.15.97:0/2379728665 conn(0x7f8508077eb0 msgr2=0x7f85080782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:18.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.264+0000 7f850e520640 1 -- 172.21.15.97:0/2379728665 shutdown_connections 2024-04-23T10:26:18.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.264+0000 7f850e520640 1 -- 172.21.15.97:0/2379728665 wait complete. 2024-04-23T10:26:18.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.265+0000 7f850e520640 1 Processor -- start 2024-04-23T10:26:18.264 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.265+0000 7f850e520640 1 -- start start 2024-04-23T10:26:18.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.265+0000 7f850e520640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85081010a0 0x7f85081985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:18.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.265+0000 7f850e520640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8508106170 0x7f8508198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:18.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.265+0000 7f850e520640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f850819ceb0 0x7f850819d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:18.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.265+0000 7f850e520640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f85081110c0 con 0x7f85081010a0 2024-04-23T10:26:18.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.265+0000 7f850e520640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8508110f40 con 0x7f850819ceb0 2024-04-23T10:26:18.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.265+0000 7f850e520640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8508111240 con 0x7f8508106170 2024-04-23T10:26:18.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.265+0000 7f8507fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85081010a0 0x7f85081985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:18.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.265+0000 7f8507fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85081010a0 0x7f85081985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47020/0 (socket says 172.21.15.97:47020) 2024-04-23T10:26:18.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.265+0000 7f8507fff640 1 -- 172.21.15.97:0/2834681549 learned_addr learned my addr 172.21.15.97:0/2834681549 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:18.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.265+0000 7f85077fe640 1 --2- 172.21.15.97:0/2834681549 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8508106170 0x7f8508198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:18.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.265+0000 7f850cabd640 1 --2- 172.21.15.97:0/2834681549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f850819ceb0 0x7f850819d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:18.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.265+0000 7f8507fff640 1 -- 172.21.15.97:0/2834681549 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8508106170 msgr2=0x7f8508198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:18.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.265+0000 7f8507fff640 1 --2- 172.21.15.97:0/2834681549 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8508106170 0x7f8508198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:18.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.265+0000 7f8507fff640 1 -- 172.21.15.97:0/2834681549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f850819ceb0 msgr2=0x7f850819d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:18.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.265+0000 7f8507fff640 1 --2- 172.21.15.97:0/2834681549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f850819ceb0 0x7f850819d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:18.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.266+0000 7f8507fff640 1 -- 172.21.15.97:0/2834681549 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f850819d9a0 con 0x7f85081010a0 2024-04-23T10:26:18.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.266+0000 7f8507fff640 1 --2- 172.21.15.97:0/2834681549 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85081010a0 0x7f85081985f0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f84f800d950 tx=0x7f84f800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:18.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.266+0000 7f85057fa640 1 -- 172.21.15.97:0/2834681549 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f8017030 con 0x7f85081010a0 2024-04-23T10:26:18.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.266+0000 7f850e520640 1 -- 172.21.15.97:0/2834681549 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f850819dc90 con 0x7f85081010a0 2024-04-23T10:26:18.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.266+0000 7f85057fa640 1 -- 172.21.15.97:0/2834681549 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84f8004020 con 0x7f85081010a0 2024-04-23T10:26:18.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.266+0000 7f85057fa640 1 -- 172.21.15.97:0/2834681549 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f80047a0 con 0x7f85081010a0 2024-04-23T10:26:18.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.266+0000 7f850e520640 1 -- 172.21.15.97:0/2834681549 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85081a54d0 con 0x7f85081010a0 2024-04-23T10:26:18.267 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.267+0000 7f850e520640 1 -- 172.21.15.97:0/2834681549 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85080691c0 con 0x7f85081010a0 2024-04-23T10:26:18.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.268+0000 7f85057fa640 1 -- 172.21.15.97:0/2834681549 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f84f8009a30 con 0x7f85081010a0 2024-04-23T10:26:18.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.268+0000 7f85057fa640 1 --2- 172.21.15.97:0/2834681549 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f84f00a3e40 0x7f84f00a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:18.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.268+0000 7f85077fe640 1 --2- 172.21.15.97:0/2834681549 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f84f00a3e40 0x7f84f00a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:18.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.269+0000 7f85057fa640 1 -- 172.21.15.97:0/2834681549 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f84f80d0570 con 0x7f85081010a0 2024-04-23T10:26:18.268 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.269+0000 7f85077fe640 1 --2- 172.21.15.97:0/2834681549 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f84f00a3e40 0x7f84f00a6300 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f850806b630 tx=0x7f84fc00c4a0 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:18.272 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.273+0000 7f85057fa640 1 -- 172.21.15.97:0/2834681549 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f84f809ca10 con 0x7f85081010a0 2024-04-23T10:26:18.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:17 smithi196 ceph-mon[32919]: from='client.15534 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T10:26:18.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:17 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/1768413598' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:26:18.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:17 smithi183 ceph-mon[32317]: from='client.15534 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T10:26:18.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:17 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/1768413598' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:26:18.449 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.451+0000 7f850e520640 1 -- 172.21.15.97:0/2834681549 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8508101a80 con 0x7f85081010a0 2024-04-23T10:26:18.450 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.451+0000 7f85057fa640 1 -- 172.21.15.97:0/2834681549 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7f84f809c310 con 0x7f85081010a0 2024-04-23T10:26:18.450 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-23T10:26:18.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.453+0000 7f850e520640 1 -- 172.21.15.97:0/2834681549 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f84f00a3e40 msgr2=0x7f84f00a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:18.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.453+0000 7f850e520640 1 --2- 172.21.15.97:0/2834681549 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f84f00a3e40 0x7f84f00a6300 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f850806b630 tx=0x7f84fc00c4a0 comp rx=0 tx=0).stop 2024-04-23T10:26:18.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.453+0000 7f850e520640 1 -- 172.21.15.97:0/2834681549 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85081010a0 msgr2=0x7f85081985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:18.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.453+0000 7f850e520640 1 --2- 172.21.15.97:0/2834681549 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85081010a0 0x7f85081985f0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f84f800d950 tx=0x7f84f800de10 comp rx=0 tx=0).stop 2024-04-23T10:26:18.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.453+0000 7f850e520640 1 -- 172.21.15.97:0/2834681549 shutdown_connections 2024-04-23T10:26:18.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.453+0000 7f850e520640 1 --2- 172.21.15.97:0/2834681549 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f84f00a3e40 0x7f84f00a6300 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:18.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.453+0000 7f850e520640 1 --2- 172.21.15.97:0/2834681549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f850819ceb0 0x7f850819d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:18.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.453+0000 7f850e520640 1 --2- 172.21.15.97:0/2834681549 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8508106170 0x7f8508198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:18.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.453+0000 7f850e520640 1 --2- 172.21.15.97:0/2834681549 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85081010a0 0x7f85081985f0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:18.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.453+0000 7f850e520640 1 -- 172.21.15.97:0/2834681549 >> 172.21.15.97:0/2834681549 conn(0x7f8508077eb0 msgr2=0x7f8508104c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:18.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.453+0000 7f850e520640 1 -- 172.21.15.97:0/2834681549 shutdown_connections 2024-04-23T10:26:18.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.453+0000 7f850e520640 1 -- 172.21.15.97:0/2834681549 wait complete. 2024-04-23T10:26:18.463 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T10:26:18.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.566+0000 7f27254fa640 1 -- 172.21.15.97:0/118928877 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2720101070 msgr2=0x7f2720106ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:18.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.566+0000 7f27254fa640 1 --2- 172.21.15.97:0/118928877 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2720101070 0x7f2720106ba0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f2710009900 tx=0x7f271002a410 comp rx=0 tx=0).stop 2024-04-23T10:26:18.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.566+0000 7f27254fa640 1 -- 172.21.15.97:0/118928877 shutdown_connections 2024-04-23T10:26:18.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.566+0000 7f27254fa640 1 --2- 172.21.15.97:0/118928877 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2720107280 0x7f272010ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:18.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.566+0000 7f27254fa640 1 --2- 172.21.15.97:0/118928877 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2720101070 0x7f2720106ba0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:18.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.566+0000 7f27254fa640 1 --2- 172.21.15.97:0/118928877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27201006c0 0x7f2720100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:18.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.566+0000 7f27254fa640 1 -- 172.21.15.97:0/118928877 >> 172.21.15.97:0/118928877 conn(0x7f27200773a0 msgr2=0x7f27200fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:18.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.566+0000 7f27254fa640 1 -- 172.21.15.97:0/118928877 shutdown_connections 2024-04-23T10:26:18.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.566+0000 7f27254fa640 1 -- 172.21.15.97:0/118928877 wait complete. 2024-04-23T10:26:18.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.566+0000 7f27254fa640 1 Processor -- start 2024-04-23T10:26:18.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.566+0000 7f27254fa640 1 -- start start 2024-04-23T10:26:18.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.567+0000 7f27254fa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27201006c0 0x7f272019e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:18.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.567+0000 7f27254fa640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2720107280 0x7f272019e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:18.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.567+0000 7f27254fa640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27201985b0 0x7f2720198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:18.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.567+0000 7f27254fa640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2720112210 con 0x7f27201006c0 2024-04-23T10:26:18.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.567+0000 7f27254fa640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2720112090 con 0x7f2720107280 2024-04-23T10:26:18.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.567+0000 7f27254fa640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2720112390 con 0x7f27201985b0 2024-04-23T10:26:18.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.567+0000 7f271effd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27201006c0 0x7f272019e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:18.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.567+0000 7f271effd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27201006c0 0x7f272019e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47036/0 (socket says 172.21.15.97:47036) 2024-04-23T10:26:18.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.567+0000 7f271effd640 1 -- 172.21.15.97:0/1290657624 learned_addr learned my addr 172.21.15.97:0/1290657624 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:18.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.567+0000 7f271e7fc640 1 --2- 172.21.15.97:0/1290657624 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2720107280 0x7f272019e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:18.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.567+0000 7f271f7fe640 1 --2- 172.21.15.97:0/1290657624 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27201985b0 0x7f2720198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:18.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.567+0000 7f271effd640 1 -- 172.21.15.97:0/1290657624 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27201985b0 msgr2=0x7f2720198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:18.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.567+0000 7f271effd640 1 --2- 172.21.15.97:0/1290657624 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27201985b0 0x7f2720198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:18.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.567+0000 7f271effd640 1 -- 172.21.15.97:0/1290657624 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2720107280 msgr2=0x7f272019e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:18.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.567+0000 7f271effd640 1 --2- 172.21.15.97:0/1290657624 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2720107280 0x7f272019e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:18.568 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.567+0000 7f271effd640 1 -- 172.21.15.97:0/1290657624 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27201990d0 con 0x7f27201006c0 2024-04-23T10:26:18.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.567+0000 7f271f7fe640 1 --2- 172.21.15.97:0/1290657624 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27201985b0 0x7f2720198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:26:18.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.567+0000 7f271effd640 1 --2- 172.21.15.97:0/1290657624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27201006c0 0x7f272019e400 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f271400e9f0 tx=0x7f271400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:18.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.567+0000 7f2707fff640 1 -- 172.21.15.97:0/1290657624 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f271400cd80 con 0x7f27201006c0 2024-04-23T10:26:18.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.567+0000 7f27254fa640 1 -- 172.21.15.97:0/1290657624 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27201993c0 con 0x7f27201006c0 2024-04-23T10:26:18.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.567+0000 7f2707fff640 1 -- 172.21.15.97:0/1290657624 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27140054d0 con 0x7f27201006c0 2024-04-23T10:26:18.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.567+0000 7f2707fff640 1 -- 172.21.15.97:0/1290657624 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2714010690 con 0x7f27201006c0 2024-04-23T10:26:18.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.567+0000 7f27254fa640 1 -- 172.21.15.97:0/1290657624 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27201a55c0 con 0x7f27201006c0 2024-04-23T10:26:18.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.568+0000 7f27254fa640 1 -- 172.21.15.97:0/1290657624 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27200691c0 con 0x7f27201006c0 2024-04-23T10:26:18.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.570+0000 7f2707fff640 1 -- 172.21.15.97:0/1290657624 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f2714004e30 con 0x7f27201006c0 2024-04-23T10:26:18.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.570+0000 7f2707fff640 1 --2- 172.21.15.97:0/1290657624 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f27080a3e40 0x7f27080a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:18.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.570+0000 7f2707fff640 1 -- 172.21.15.97:0/1290657624 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f27140d1090 con 0x7f27201006c0 2024-04-23T10:26:18.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.570+0000 7f271e7fc640 1 --2- 172.21.15.97:0/1290657624 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f27080a3e40 0x7f27080a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:18.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.571+0000 7f271e7fc640 1 --2- 172.21.15.97:0/1290657624 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f27080a3e40 0x7f27080a6300 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2710031440 tx=0x7f2710005a20 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:18.573 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.574+0000 7f2707fff640 1 -- 172.21.15.97:0/1290657624 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2714053ff0 con 0x7f27201006c0 2024-04-23T10:26:18.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.753+0000 7f27254fa640 1 -- 172.21.15.97:0/1290657624 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2720101070 con 0x7f27201006c0 2024-04-23T10:26:18.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.753+0000 7f2707fff640 1 -- 172.21.15.97:0/1290657624 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7f271409d5d0 con 0x7f27201006c0 2024-04-23T10:26:18.752 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-23T10:26:18.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.755+0000 7f27254fa640 1 -- 172.21.15.97:0/1290657624 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f27080a3e40 msgr2=0x7f27080a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:18.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.755+0000 7f27254fa640 1 --2- 172.21.15.97:0/1290657624 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f27080a3e40 0x7f27080a6300 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2710031440 tx=0x7f2710005a20 comp rx=0 tx=0).stop 2024-04-23T10:26:18.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.755+0000 7f27254fa640 1 -- 172.21.15.97:0/1290657624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27201006c0 msgr2=0x7f272019e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:18.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.755+0000 7f27254fa640 1 --2- 172.21.15.97:0/1290657624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27201006c0 0x7f272019e400 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f271400e9f0 tx=0x7f271400edf0 comp rx=0 tx=0).stop 2024-04-23T10:26:18.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.755+0000 7f27254fa640 1 -- 172.21.15.97:0/1290657624 shutdown_connections 2024-04-23T10:26:18.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.755+0000 7f27254fa640 1 --2- 172.21.15.97:0/1290657624 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f27080a3e40 0x7f27080a6300 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:18.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.755+0000 7f27254fa640 1 --2- 172.21.15.97:0/1290657624 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27201985b0 0x7f2720198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:18.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.755+0000 7f27254fa640 1 --2- 172.21.15.97:0/1290657624 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2720107280 0x7f272019e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:18.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.755+0000 7f27254fa640 1 --2- 172.21.15.97:0/1290657624 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27201006c0 0x7f272019e400 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:18.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.756+0000 7f27254fa640 1 -- 172.21.15.97:0/1290657624 >> 172.21.15.97:0/1290657624 conn(0x7f27200773a0 msgr2=0x7f27201054e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:18.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.756+0000 7f27254fa640 1 -- 172.21.15.97:0/1290657624 shutdown_connections 2024-04-23T10:26:18.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.756+0000 7f27254fa640 1 -- 172.21.15.97:0/1290657624 wait complete. 2024-04-23T10:26:18.765 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T10:26:18.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.868+0000 7f2dd6d9b640 1 -- 172.21.15.97:0/1573486687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2dd0106170 msgr2=0x7f2dd010dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:18.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.868+0000 7f2dd6d9b640 1 --2- 172.21.15.97:0/1573486687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2dd0106170 0x7f2dd010dd40 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f2dc8009ca0 tx=0x7f2dc802a8c0 comp rx=0 tx=0).stop 2024-04-23T10:26:18.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.868+0000 7f2dd6d9b640 1 -- 172.21.15.97:0/1573486687 shutdown_connections 2024-04-23T10:26:18.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.868+0000 7f2dd6d9b640 1 --2- 172.21.15.97:0/1573486687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2dd0106170 0x7f2dd010dd40 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:18.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.868+0000 7f2dd6d9b640 1 --2- 172.21.15.97:0/1573486687 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2dd0101a50 0x7f2dd0105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:18.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.868+0000 7f2dd6d9b640 1 --2- 172.21.15.97:0/1573486687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2dd01010a0 0x7f2dd0101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:18.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.868+0000 7f2dd6d9b640 1 -- 172.21.15.97:0/1573486687 >> 172.21.15.97:0/1573486687 conn(0x7f2dd0077eb0 msgr2=0x7f2dd00782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:18.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.868+0000 7f2dd6d9b640 1 -- 172.21.15.97:0/1573486687 shutdown_connections 2024-04-23T10:26:18.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.868+0000 7f2dd6d9b640 1 -- 172.21.15.97:0/1573486687 wait complete. 2024-04-23T10:26:18.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.869+0000 7f2dd6d9b640 1 Processor -- start 2024-04-23T10:26:18.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.869+0000 7f2dd6d9b640 1 -- start start 2024-04-23T10:26:18.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.869+0000 7f2dd6d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2dd01010a0 0x7f2dd019e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:18.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.869+0000 7f2dd6d9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2dd0106170 0x7f2dd019e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:18.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.869+0000 7f2dd6d9b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2dd0198590 0x7f2dd0198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:18.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.869+0000 7f2dd6d9b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd01110e0 con 0x7f2dd01010a0 2024-04-23T10:26:18.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.869+0000 7f2dd6d9b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd0110f60 con 0x7f2dd0198590 2024-04-23T10:26:18.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.869+0000 7f2dd6d9b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd0111260 con 0x7f2dd0106170 2024-04-23T10:26:18.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.869+0000 7f2dd4b37640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2dd01010a0 0x7f2dd019e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:18.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.869+0000 7f2dd4b37640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2dd01010a0 0x7f2dd019e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47054/0 (socket says 172.21.15.97:47054) 2024-04-23T10:26:18.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.869+0000 7f2dd4b37640 1 -- 172.21.15.97:0/3840928823 learned_addr learned my addr 172.21.15.97:0/3840928823 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:18.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.869+0000 7f2dd4b37640 1 -- 172.21.15.97:0/3840928823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2dd0106170 msgr2=0x7f2dd019e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:18.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.869+0000 7f2dd4b37640 1 --2- 172.21.15.97:0/3840928823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2dd0106170 0x7f2dd019e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:18.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.869+0000 7f2dd4b37640 1 -- 172.21.15.97:0/3840928823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2dd0198590 msgr2=0x7f2dd0198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:18.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.869+0000 7f2dd4b37640 1 --2- 172.21.15.97:0/3840928823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2dd0198590 0x7f2dd0198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:18.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.869+0000 7f2dd4b37640 1 -- 172.21.15.97:0/3840928823 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dd01990b0 con 0x7f2dd01010a0 2024-04-23T10:26:18.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.869+0000 7f2dd4b37640 1 --2- 172.21.15.97:0/3840928823 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2dd01010a0 0x7f2dd019e3e0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f2dc400c970 tx=0x7f2dc400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:18.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.869+0000 7f2dcdffb640 1 -- 172.21.15.97:0/3840928823 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc4007bb0 con 0x7f2dd01010a0 2024-04-23T10:26:18.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.870+0000 7f2dcdffb640 1 -- 172.21.15.97:0/3840928823 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2dc4007d50 con 0x7f2dd01010a0 2024-04-23T10:26:18.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.870+0000 7f2dd6d9b640 1 -- 172.21.15.97:0/3840928823 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dd01993a0 con 0x7f2dd01010a0 2024-04-23T10:26:18.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.870+0000 7f2dcdffb640 1 -- 172.21.15.97:0/3840928823 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc400f450 con 0x7f2dd01010a0 2024-04-23T10:26:18.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.870+0000 7f2dd6d9b640 1 -- 172.21.15.97:0/3840928823 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dd01a55a0 con 0x7f2dd01010a0 2024-04-23T10:26:18.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.871+0000 7f2dd6d9b640 1 -- 172.21.15.97:0/3840928823 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2dd00691c0 con 0x7f2dd01010a0 2024-04-23T10:26:18.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.872+0000 7f2dcdffb640 1 -- 172.21.15.97:0/3840928823 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f2dc4004f20 con 0x7f2dd01010a0 2024-04-23T10:26:18.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.872+0000 7f2dcdffb640 1 --2- 172.21.15.97:0/3840928823 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f2dc00a3e40 0x7f2dc00a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:18.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.872+0000 7f2dcffff640 1 --2- 172.21.15.97:0/3840928823 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f2dc00a3e40 0x7f2dc00a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:18.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.872+0000 7f2dcdffb640 1 -- 172.21.15.97:0/3840928823 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f2dc40d0890 con 0x7f2dd01010a0 2024-04-23T10:26:18.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.873+0000 7f2dcffff640 1 --2- 172.21.15.97:0/3840928823 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f2dc00a3e40 0x7f2dc00a6300 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc005d80 tx=0x7f2dbc005c90 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:18.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:18.877+0000 7f2dcdffb640 1 -- 172.21.15.97:0/3840928823 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2dc4014030 con 0x7f2dd01010a0 2024-04-23T10:26:19.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.058+0000 7f2dd6d9b640 1 -- 172.21.15.97:0/3840928823 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2dd0101a50 con 0x7f2dd01010a0 2024-04-23T10:26:19.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.058+0000 7f2dcdffb640 1 -- 172.21.15.97:0/3840928823 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7f2dc409cd30 con 0x7f2dd01010a0 2024-04-23T10:26:19.058 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-04-23T10:26:19.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.060+0000 7f2dd6d9b640 1 -- 172.21.15.97:0/3840928823 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f2dc00a3e40 msgr2=0x7f2dc00a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:19.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.060+0000 7f2dd6d9b640 1 --2- 172.21.15.97:0/3840928823 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f2dc00a3e40 0x7f2dc00a6300 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc005d80 tx=0x7f2dbc005c90 comp rx=0 tx=0).stop 2024-04-23T10:26:19.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.061+0000 7f2dd6d9b640 1 -- 172.21.15.97:0/3840928823 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2dd01010a0 msgr2=0x7f2dd019e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:19.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.061+0000 7f2dd6d9b640 1 --2- 172.21.15.97:0/3840928823 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2dd01010a0 0x7f2dd019e3e0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f2dc400c970 tx=0x7f2dc400ce30 comp rx=0 tx=0).stop 2024-04-23T10:26:19.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.061+0000 7f2dd6d9b640 1 -- 172.21.15.97:0/3840928823 shutdown_connections 2024-04-23T10:26:19.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.061+0000 7f2dd6d9b640 1 --2- 172.21.15.97:0/3840928823 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f2dc00a3e40 0x7f2dc00a6300 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:19.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.061+0000 7f2dd6d9b640 1 --2- 172.21.15.97:0/3840928823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2dd0198590 0x7f2dd0198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:19.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.061+0000 7f2dd6d9b640 1 --2- 172.21.15.97:0/3840928823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2dd0106170 0x7f2dd019e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:19.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.061+0000 7f2dd6d9b640 1 --2- 172.21.15.97:0/3840928823 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2dd01010a0 0x7f2dd019e3e0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:19.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.061+0000 7f2dd6d9b640 1 -- 172.21.15.97:0/3840928823 >> 172.21.15.97:0/3840928823 conn(0x7f2dd0077eb0 msgr2=0x7f2dd006a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:19.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.061+0000 7f2dd6d9b640 1 -- 172.21.15.97:0/3840928823 shutdown_connections 2024-04-23T10:26:19.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.061+0000 7f2dd6d9b640 1 -- 172.21.15.97:0/3840928823 wait complete. 2024-04-23T10:26:19.069 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:18 smithi097 ceph-mon[29197]: from='client.15540 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T10:26:19.070 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:18 smithi097 ceph-mon[29197]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 90 KiB/s wr, 69 op/s 2024-04-23T10:26:19.070 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:18 smithi097 ceph-mon[29197]: from='client.15546 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T10:26:19.070 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:18 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/1388590846' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T10:26:19.070 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:18 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/2834681549' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T10:26:19.070 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:18 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/1290657624' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T10:26:19.071 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T10:26:19.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.171+0000 7f4d849fe640 1 -- 172.21.15.97:0/2881063567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d80102b40 msgr2=0x7f4d80106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:19.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.171+0000 7f4d849fe640 1 --2- 172.21.15.97:0/2881063567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d80102b40 0x7f4d80106b80 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4d70009900 tx=0x7f4d7002a410 comp rx=0 tx=0).stop 2024-04-23T10:26:19.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.171+0000 7f4d849fe640 1 -- 172.21.15.97:0/2881063567 shutdown_connections 2024-04-23T10:26:19.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.171+0000 7f4d849fe640 1 --2- 172.21.15.97:0/2881063567 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d80107260 0x7f4d8010ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:19.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.171+0000 7f4d849fe640 1 --2- 172.21.15.97:0/2881063567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d80102b40 0x7f4d80106b80 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:19.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.171+0000 7f4d849fe640 1 --2- 172.21.15.97:0/2881063567 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4d80102190 0x7f4d80102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:19.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.171+0000 7f4d849fe640 1 -- 172.21.15.97:0/2881063567 >> 172.21.15.97:0/2881063567 conn(0x7f4d80079310 msgr2=0x7f4d80079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:19.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.171+0000 7f4d849fe640 1 -- 172.21.15.97:0/2881063567 shutdown_connections 2024-04-23T10:26:19.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.171+0000 7f4d849fe640 1 -- 172.21.15.97:0/2881063567 wait complete. 2024-04-23T10:26:19.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.172+0000 7f4d849fe640 1 Processor -- start 2024-04-23T10:26:19.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.172+0000 7f4d849fe640 1 -- start start 2024-04-23T10:26:19.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.172+0000 7f4d849fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d80102190 0x7f4d8019e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:19.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.172+0000 7f4d849fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d80107260 0x7f4d8019e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:19.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.172+0000 7f4d849fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4d801985a0 0x7f4d80198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:19.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.172+0000 7f4d849fe640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4d801121a0 con 0x7f4d80107260 2024-04-23T10:26:19.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.172+0000 7f4d849fe640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4d80112020 con 0x7f4d801985a0 2024-04-23T10:26:19.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.172+0000 7f4d849fe640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4d80112320 con 0x7f4d80102190 2024-04-23T10:26:19.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.172+0000 7f4d7dd9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d80107260 0x7f4d8019e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:19.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.172+0000 7f4d7e59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d80102190 0x7f4d8019e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:19.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.172+0000 7f4d7dd9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d80107260 0x7f4d8019e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47070/0 (socket says 172.21.15.97:47070) 2024-04-23T10:26:19.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.172+0000 7f4d7dd9b640 1 -- 172.21.15.97:0/3785077638 learned_addr learned my addr 172.21.15.97:0/3785077638 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:19.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.172+0000 7f4d7ed9d640 1 --2- 172.21.15.97:0/3785077638 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4d801985a0 0x7f4d80198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:19.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.173+0000 7f4d7dd9b640 1 -- 172.21.15.97:0/3785077638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d80102190 msgr2=0x7f4d8019e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:19.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.173+0000 7f4d7dd9b640 1 --2- 172.21.15.97:0/3785077638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d80102190 0x7f4d8019e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:19.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.173+0000 7f4d7dd9b640 1 -- 172.21.15.97:0/3785077638 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4d801985a0 msgr2=0x7f4d80198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:19.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.173+0000 7f4d7dd9b640 1 --2- 172.21.15.97:0/3785077638 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4d801985a0 0x7f4d80198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:19.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.173+0000 7f4d7dd9b640 1 -- 172.21.15.97:0/3785077638 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d801990c0 con 0x7f4d80107260 2024-04-23T10:26:19.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.173+0000 7f4d7dd9b640 1 --2- 172.21.15.97:0/3785077638 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d80107260 0x7f4d8019e930 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f4d70032fd0 tx=0x7f4d70031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:19.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.173+0000 7f4d6f7fe640 1 -- 172.21.15.97:0/3785077638 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7003f030 con 0x7f4d80107260 2024-04-23T10:26:19.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.173+0000 7f4d6f7fe640 1 -- 172.21.15.97:0/3785077638 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d70004490 con 0x7f4d80107260 2024-04-23T10:26:19.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.173+0000 7f4d849fe640 1 -- 172.21.15.97:0/3785077638 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d80199350 con 0x7f4d80107260 2024-04-23T10:26:19.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.173+0000 7f4d6f7fe640 1 -- 172.21.15.97:0/3785077638 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d70035070 con 0x7f4d80107260 2024-04-23T10:26:19.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.173+0000 7f4d849fe640 1 -- 172.21.15.97:0/3785077638 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d80074410 con 0x7f4d80107260 2024-04-23T10:26:19.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.174+0000 7f4d849fe640 1 -- 172.21.15.97:0/3785077638 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d800691c0 con 0x7f4d80107260 2024-04-23T10:26:19.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.175+0000 7f4d6f7fe640 1 -- 172.21.15.97:0/3785077638 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f4d70042030 con 0x7f4d80107260 2024-04-23T10:26:19.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.175+0000 7f4d6f7fe640 1 --2- 172.21.15.97:0/3785077638 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f4d680a3e40 0x7f4d680a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:19.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.176+0000 7f4d6f7fe640 1 -- 172.21.15.97:0/3785077638 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f4d700f0cb0 con 0x7f4d80107260 2024-04-23T10:26:19.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.176+0000 7f4d7e59c640 1 --2- 172.21.15.97:0/3785077638 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f4d680a3e40 0x7f4d680a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:19.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.176+0000 7f4d7e59c640 1 --2- 172.21.15.97:0/3785077638 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f4d680a3e40 0x7f4d680a6300 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4d74010d80 tx=0x7f4d74012040 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:19.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.180+0000 7f4d6f7fe640 1 -- 172.21.15.97:0/3785077638 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4d700bcbd0 con 0x7f4d80107260 2024-04-23T10:26:19.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:18 smithi196 ceph-mon[32919]: from='client.15540 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T10:26:19.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:18 smithi196 ceph-mon[32919]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 90 KiB/s wr, 69 op/s 2024-04-23T10:26:19.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:18 smithi196 ceph-mon[32919]: from='client.15546 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T10:26:19.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:18 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/1388590846' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T10:26:19.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:18 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/2834681549' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T10:26:19.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:18 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/1290657624' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T10:26:19.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.395+0000 7f4d849fe640 1 -- 172.21.15.97:0/3785077638 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4d80102b40 con 0x7f4d80107260 2024-04-23T10:26:19.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.396+0000 7f4d6f7fe640 1 -- 172.21.15.97:0/3785077638 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11931 (secure 0 0 0) 0x7f4d700c0ed0 con 0x7f4d80107260 2024-04-23T10:26:19.395 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:26:19.395 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3946878762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3946878762},{"type":"v1","addr":"172.21.15.97:6835","nonce":3946878762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/209800861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":209800861},{"type":"v1","addr":"172.21.15.97:6837","nonce":209800861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T10:24:06.910134+0000","modified":"2024-04-23T10:26:09.820985+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6839/418013575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":418013575},{"type":"v1","addr":"172.21.15.97:6839","nonce":418013575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6841/747242680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":747242680},{"type":"v1","addr":"172.21.15.97:6841","nonce":747242680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6833/4292847121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4292847121},{"type":"v1","addr":"172.21.15.183:6833","nonce":4292847121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6833/3885816565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3885816565},{"type":"v1","addr":"172.21.15.196:6833","nonce":3885816565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1897093105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1897093105},{"type":"v1","addr":"172.21.15.183:6835","nonce":1897093105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1171477563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1171477563},{"type":"v1","addr":"172.21.15.196:6835","nonce":1171477563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1898341911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1898341911},{"type":"v1","addr":"172.21.15.183:6837","nonce":1898341911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.196:6837/3094946502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3094946502},{"type":"v1","addr":"172.21.15.196:6837","nonce":3094946502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.196:6839/83843556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":83843556},{"type":"v1","addr":"172.21.15.196:6839","nonce":83843556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6839/1763841130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1763841130},{"type":"v1","addr":"172.21.15.183:6839","nonce":1763841130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24487,24403,24398,24469,24488]},"id":1}]} 2024-04-23T10:26:19.395 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 26 2024-04-23T10:26:19.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.398+0000 7f4d849fe640 1 -- 172.21.15.97:0/3785077638 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f4d680a3e40 msgr2=0x7f4d680a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:19.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.398+0000 7f4d849fe640 1 --2- 172.21.15.97:0/3785077638 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f4d680a3e40 0x7f4d680a6300 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4d74010d80 tx=0x7f4d74012040 comp rx=0 tx=0).stop 2024-04-23T10:26:19.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.398+0000 7f4d849fe640 1 -- 172.21.15.97:0/3785077638 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d80107260 msgr2=0x7f4d8019e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:19.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.398+0000 7f4d849fe640 1 --2- 172.21.15.97:0/3785077638 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d80107260 0x7f4d8019e930 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f4d70032fd0 tx=0x7f4d70031a20 comp rx=0 tx=0).stop 2024-04-23T10:26:19.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.398+0000 7f4d849fe640 1 -- 172.21.15.97:0/3785077638 shutdown_connections 2024-04-23T10:26:19.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.398+0000 7f4d849fe640 1 --2- 172.21.15.97:0/3785077638 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f4d680a3e40 0x7f4d680a6300 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:19.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.398+0000 7f4d849fe640 1 --2- 172.21.15.97:0/3785077638 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4d801985a0 0x7f4d80198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:19.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.398+0000 7f4d849fe640 1 --2- 172.21.15.97:0/3785077638 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4d80107260 0x7f4d8019e930 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:19.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.398+0000 7f4d849fe640 1 --2- 172.21.15.97:0/3785077638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d80102190 0x7f4d8019e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:19.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.398+0000 7f4d849fe640 1 -- 172.21.15.97:0/3785077638 >> 172.21.15.97:0/3785077638 conn(0x7f4d80079310 msgr2=0x7f4d80100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:19.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.398+0000 7f4d849fe640 1 -- 172.21.15.97:0/3785077638 shutdown_connections 2024-04-23T10:26:19.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.398+0000 7f4d849fe640 1 -- 172.21.15.97:0/3785077638 wait complete. 2024-04-23T10:26:19.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:18 smithi183 ceph-mon[32317]: from='client.15540 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T10:26:19.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:18 smithi183 ceph-mon[32317]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 90 KiB/s wr, 69 op/s 2024-04-23T10:26:19.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:18 smithi183 ceph-mon[32317]: from='client.15546 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T10:26:19.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:18 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/1388590846' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T10:26:19.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:18 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/2834681549' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T10:26:19.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:18 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/1290657624' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T10:26:19.408 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T10:26:19.508 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.507+0000 7f7d1a107640 1 -- 172.21.15.97:0/1269650269 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d14107280 msgr2=0x7f7d1410ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:19.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.507+0000 7f7d1a107640 1 --2- 172.21.15.97:0/1269650269 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d14107280 0x7f7d1410ee50 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c009ca0 tx=0x7f7d0c02a8c0 comp rx=0 tx=0).stop 2024-04-23T10:26:19.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.508+0000 7f7d1a107640 1 -- 172.21.15.97:0/1269650269 shutdown_connections 2024-04-23T10:26:19.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.508+0000 7f7d1a107640 1 --2- 172.21.15.97:0/1269650269 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d14107280 0x7f7d1410ee50 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:19.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.508+0000 7f7d1a107640 1 --2- 172.21.15.97:0/1269650269 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d14102bf0 0x7f7d14106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:19.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.508+0000 7f7d1a107640 1 --2- 172.21.15.97:0/1269650269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d14102240 0x7f7d14102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:19.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.508+0000 7f7d1a107640 1 -- 172.21.15.97:0/1269650269 >> 172.21.15.97:0/1269650269 conn(0x7f7d140790e0 msgr2=0x7f7d140794f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:19.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.508+0000 7f7d1a107640 1 -- 172.21.15.97:0/1269650269 shutdown_connections 2024-04-23T10:26:19.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.508+0000 7f7d1a107640 1 -- 172.21.15.97:0/1269650269 wait complete. 2024-04-23T10:26:19.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.508+0000 7f7d1a107640 1 Processor -- start 2024-04-23T10:26:19.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.508+0000 7f7d1a107640 1 -- start start 2024-04-23T10:26:19.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.508+0000 7f7d1a107640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d14102240 0x7f7d14198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:19.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.508+0000 7f7d1a107640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d14107280 0x7f7d14198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:19.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.509+0000 7f7d1a107640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d1419cfc0 0x7f7d1419d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:19.509 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.509+0000 7f7d1a107640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7d14112200 con 0x7f7d1419cfc0 2024-04-23T10:26:19.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.509+0000 7f7d1a107640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7d14112080 con 0x7f7d14102240 2024-04-23T10:26:19.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.509+0000 7f7d1a107640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7d14112380 con 0x7f7d14107280 2024-04-23T10:26:19.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.509+0000 7f7d13fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d1419cfc0 0x7f7d1419d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:19.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.509+0000 7f7d13fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d1419cfc0 0x7f7d1419d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47094/0 (socket says 172.21.15.97:47094) 2024-04-23T10:26:19.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.509+0000 7f7d13fff640 1 -- 172.21.15.97:0/582176515 learned_addr learned my addr 172.21.15.97:0/582176515 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:19.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.509+0000 7f7d12ffd640 1 --2- 172.21.15.97:0/582176515 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d14107280 0x7f7d14198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:19.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.509+0000 7f7d137fe640 1 --2- 172.21.15.97:0/582176515 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d14102240 0x7f7d14198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:19.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.509+0000 7f7d13fff640 1 -- 172.21.15.97:0/582176515 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d14107280 msgr2=0x7f7d14198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:19.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.509+0000 7f7d13fff640 1 --2- 172.21.15.97:0/582176515 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d14107280 0x7f7d14198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:19.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.509+0000 7f7d13fff640 1 -- 172.21.15.97:0/582176515 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d14102240 msgr2=0x7f7d14198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:19.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.509+0000 7f7d13fff640 1 --2- 172.21.15.97:0/582176515 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d14102240 0x7f7d14198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:19.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.509+0000 7f7d13fff640 1 -- 172.21.15.97:0/582176515 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d1419dab0 con 0x7f7d1419cfc0 2024-04-23T10:26:19.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.509+0000 7f7d13fff640 1 --2- 172.21.15.97:0/582176515 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d1419cfc0 0x7f7d1419d470 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c005cc0 tx=0x7f7d0c02f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:19.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.509+0000 7f7d10ff9640 1 -- 172.21.15.97:0/582176515 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c003bf0 con 0x7f7d1419cfc0 2024-04-23T10:26:19.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.509+0000 7f7d1a107640 1 -- 172.21.15.97:0/582176515 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d1419dd40 con 0x7f7d1419cfc0 2024-04-23T10:26:19.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.509+0000 7f7d10ff9640 1 -- 172.21.15.97:0/582176515 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d0c003d90 con 0x7f7d1419cfc0 2024-04-23T10:26:19.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.509+0000 7f7d10ff9640 1 -- 172.21.15.97:0/582176515 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c02fbc0 con 0x7f7d1419cfc0 2024-04-23T10:26:19.511 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.509+0000 7f7d1a107640 1 -- 172.21.15.97:0/582176515 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d141a5540 con 0x7f7d1419cfc0 2024-04-23T10:26:19.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.510+0000 7f7d1a107640 1 -- 172.21.15.97:0/582176515 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d140691c0 con 0x7f7d1419cfc0 2024-04-23T10:26:19.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.511+0000 7f7d10ff9640 1 -- 172.21.15.97:0/582176515 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f7d0c02fd60 con 0x7f7d1419cfc0 2024-04-23T10:26:19.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.512+0000 7f7d10ff9640 1 --2- 172.21.15.97:0/582176515 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f7cfc0a3e40 0x7f7cfc0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:19.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.512+0000 7f7d10ff9640 1 -- 172.21.15.97:0/582176515 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f7d0c0f20a0 con 0x7f7d1419cfc0 2024-04-23T10:26:19.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.512+0000 7f7d137fe640 1 --2- 172.21.15.97:0/582176515 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f7cfc0a3e40 0x7f7cfc0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:19.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.513+0000 7f7d137fe640 1 --2- 172.21.15.97:0/582176515 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f7cfc0a3e40 0x7f7cfc0a6300 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7d0800bbe0 tx=0x7f7d0800b390 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:19.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.516+0000 7f7d10ff9640 1 -- 172.21.15.97:0/582176515 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7d0c0be6f0 con 0x7f7d1419cfc0 2024-04-23T10:26:19.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.694+0000 7f7d1a107640 1 -- 172.21.15.97:0/582176515 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7d14199a50 con 0x7f7d1419cfc0 2024-04-23T10:26:19.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.695+0000 7f7d10ff9640 1 -- 172.21.15.97:0/582176515 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22981 (secure 0 0 0) 0x7f7d14199a50 con 0x7f7d1419cfc0 2024-04-23T10:26:19.695 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:26:19.695 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":89,"fsid":"9b22af88-0159-11ef-bc93-c7b262605968","created":"2024-04-23T10:16:05.983305+0000","modified":"2024-04-23T10:26:11.109574+0000","last_up_change":"2024-04-23T10:23:13.114761+0000","last_in_change":"2024-04-23T10:22:52.804880+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T10:19:25.559138+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T10:24:04.570049+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T10:24:05.861631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"58cb995d-de35-4522-ac81-4982401b260d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6803","nonce":3031221802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6805","nonce":3031221802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6809","nonce":3031221802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6807","nonce":3031221802}]},"public_addr":"172.21.15.97:6803/3031221802","cluster_addr":"172.21.15.97:6805/3031221802","heartbeat_back_addr":"172.21.15.97:6809/3031221802","heartbeat_front_addr":"172.21.15.97:6807/3031221802","state":["exists","up"]},{"osd":1,"uuid":"b47bd8ee-1f21-4834-acf5-bb40f97ab3ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6801","nonce":1108870440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6803","nonce":1108870440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6807","nonce":1108870440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6805","nonce":1108870440}]},"public_addr":"172.21.15.183:6801/1108870440","cluster_addr":"172.21.15.183:6803/1108870440","heartbeat_back_addr":"172.21.15.183:6807/1108870440","heartbeat_front_addr":"172.21.15.183:6805/1108870440","state":["exists","up"]},{"osd":2,"uuid":"17c4eec3-6c47-4334-9b63-1d5ad9ec5988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6801","nonce":2877039731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6803","nonce":2877039731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6807","nonce":2877039731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6805","nonce":2877039731}]},"public_addr":"172.21.15.196:6801/2877039731","cluster_addr":"172.21.15.196:6803/2877039731","heartbeat_back_addr":"172.21.15.196:6807/2877039731","heartbeat_front_addr":"172.21.15.196:6805/2877039731","state":["exists","up"]},{"osd":3,"uuid":"3f731c38-2e45-4ca1-8dce-efc4a1b2166b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6811","nonce":4276311194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6813","nonce":4276311194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6817","nonce":4276311194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6815","nonce":4276311194}]},"public_addr":"172.21.15.97:6811/4276311194","cluster_addr":"172.21.15.97:6813/4276311194","heartbeat_back_addr":"172.21.15.97:6817/4276311194","heartbeat_front_addr":"172.21.15.97:6815/4276311194","state":["exists","up"]},{"osd":4,"uuid":"fdcafb14-9b82-4134-bd5b-79c0516f6cbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6809","nonce":63913916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6811","nonce":63913916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6815","nonce":63913916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6813","nonce":63913916}]},"public_addr":"172.21.15.183:6809/63913916","cluster_addr":"172.21.15.183:6811/63913916","heartbeat_back_addr":"172.21.15.183:6815/63913916","heartbeat_front_addr":"172.21.15.183:6813/63913916","state":["exists","up"]},{"osd":5,"uuid":"cdcc18d8-da3c-4012-bf12-876bb39e66eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6809","nonce":2804446896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6811","nonce":2804446896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6815","nonce":2804446896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6813","nonce":2804446896}]},"public_addr":"172.21.15.196:6809/2804446896","cluster_addr":"172.21.15.196:6811/2804446896","heartbeat_back_addr":"172.21.15.196:6815/2804446896","heartbeat_front_addr":"172.21.15.196:6813/2804446896","state":["exists","up"]},{"osd":6,"uuid":"071abb5c-2edf-4832-b8aa-3b80d20096fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6819","nonce":2080916705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6821","nonce":2080916705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6825","nonce":2080916705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6823","nonce":2080916705}]},"public_addr":"172.21.15.97:6819/2080916705","cluster_addr":"172.21.15.97:6821/2080916705","heartbeat_back_addr":"172.21.15.97:6825/2080916705","heartbeat_front_addr":"172.21.15.97:6823/2080916705","state":["exists","up"]},{"osd":7,"uuid":"9d54760b-66df-486b-8abb-017f3b64b4d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6817","nonce":1865873033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6819","nonce":1865873033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6823","nonce":1865873033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6821","nonce":1865873033}]},"public_addr":"172.21.15.183:6817/1865873033","cluster_addr":"172.21.15.183:6819/1865873033","heartbeat_back_addr":"172.21.15.183:6823/1865873033","heartbeat_front_addr":"172.21.15.183:6821/1865873033","state":["exists","up"]},{"osd":8,"uuid":"5cd3100a-bd6a-4be5-b5e9-0d0afc325193","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6817","nonce":4063932139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6819","nonce":4063932139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6823","nonce":4063932139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6821","nonce":4063932139}]},"public_addr":"172.21.15.196:6817/4063932139","cluster_addr":"172.21.15.196:6819/4063932139","heartbeat_back_addr":"172.21.15.196:6823/4063932139","heartbeat_front_addr":"172.21.15.196:6821/4063932139","state":["exists","up"]},{"osd":9,"uuid":"b6a5f4a1-4ecc-4f73-a2df-07551d9298ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6827","nonce":1887432474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6829","nonce":1887432474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6833","nonce":1887432474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6831","nonce":1887432474}]},"public_addr":"172.21.15.97:6827/1887432474","cluster_addr":"172.21.15.97:6829/1887432474","heartbeat_back_addr":"172.21.15.97:6833/1887432474","heartbeat_front_addr":"172.21.15.97:6831/1887432474","state":["exists","up"]},{"osd":10,"uuid":"86f59816-1ea6-4ae9-8796-cf5fc4831064","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6825","nonce":799434647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6827","nonce":799434647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6831","nonce":799434647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6829","nonce":799434647}]},"public_addr":"172.21.15.183:6825/799434647","cluster_addr":"172.21.15.183:6827/799434647","heartbeat_back_addr":"172.21.15.183:6831/799434647","heartbeat_front_addr":"172.21.15.183:6829/799434647","state":["exists","up"]},{"osd":11,"uuid":"95d64a89-a9f8-4c7a-8c0d-da7fb26cfff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6825","nonce":1873969467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6827","nonce":1873969467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6831","nonce":1873969467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6829","nonce":1873969467}]},"public_addr":"172.21.15.196:6825/1873969467","cluster_addr":"172.21.15.196:6827/1873969467","heartbeat_back_addr":"172.21.15.196:6831/1873969467","heartbeat_front_addr":"172.21.15.196:6829/1873969467","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:18:33.721156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:18:57.586709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:19:22.069458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:19:46.451597+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:20:11.561019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:20:36.298672+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:21:01.661804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:21:27.324128+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:21:52.898197+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:22:19.614326+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:22:45.154917+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:23:11.222612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/233335470":"2024-04-24T10:26:11.109535+0000","172.21.15.97:6800/3149043749":"2024-04-24T10:26:11.109535+0000","172.21.15.97:0/338277142":"2024-04-24T10:26:11.109535+0000","172.21.15.97:0/3866016040":"2024-04-24T10:16:59.468237+0000","172.21.15.97:6801/3297166858":"2024-04-24T10:16:59.468237+0000","172.21.15.97:0/3804743548":"2024-04-24T10:16:59.468237+0000","172.21.15.97:6800/3706463017":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/3715079438":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/597405418":"2024-04-24T10:26:11.109535+0000","172.21.15.97:0/786806834":"2024-04-24T10:16:59.468237+0000","172.21.15.97:6801/3706463017":"2024-04-24T10:16:33.128917+0000","172.21.15.97:6801/3149043749":"2024-04-24T10:26:11.109535+0000","172.21.15.97:0/3294175554":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/1449136760":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/3198144747":"2024-04-24T10:26:11.109535+0000","172.21.15.97:6800/3297166858":"2024-04-24T10:16:59.468237+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T10:26:19.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.697+0000 7f7d1a107640 1 -- 172.21.15.97:0/582176515 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f7cfc0a3e40 msgr2=0x7f7cfc0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:19.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.697+0000 7f7d1a107640 1 --2- 172.21.15.97:0/582176515 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f7cfc0a3e40 0x7f7cfc0a6300 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7d0800bbe0 tx=0x7f7d0800b390 comp rx=0 tx=0).stop 2024-04-23T10:26:19.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.697+0000 7f7d1a107640 1 -- 172.21.15.97:0/582176515 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d1419cfc0 msgr2=0x7f7d1419d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:19.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.697+0000 7f7d1a107640 1 --2- 172.21.15.97:0/582176515 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d1419cfc0 0x7f7d1419d470 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c005cc0 tx=0x7f7d0c02f380 comp rx=0 tx=0).stop 2024-04-23T10:26:19.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.698+0000 7f7d1a107640 1 -- 172.21.15.97:0/582176515 shutdown_connections 2024-04-23T10:26:19.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.698+0000 7f7d1a107640 1 --2- 172.21.15.97:0/582176515 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f7cfc0a3e40 0x7f7cfc0a6300 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:19.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.698+0000 7f7d1a107640 1 --2- 172.21.15.97:0/582176515 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d1419cfc0 0x7f7d1419d470 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:19.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.698+0000 7f7d1a107640 1 --2- 172.21.15.97:0/582176515 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d14107280 0x7f7d14198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:19.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.698+0000 7f7d1a107640 1 --2- 172.21.15.97:0/582176515 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d14102240 0x7f7d14198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:19.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.698+0000 7f7d1a107640 1 -- 172.21.15.97:0/582176515 >> 172.21.15.97:0/582176515 conn(0x7f7d140790e0 msgr2=0x7f7d141001b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:19.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.698+0000 7f7d1a107640 1 -- 172.21.15.97:0/582176515 shutdown_connections 2024-04-23T10:26:19.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.698+0000 7f7d1a107640 1 -- 172.21.15.97:0/582176515 wait complete. 2024-04-23T10:26:19.708 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with random seed 260202 2024-04-23T10:26:19.708 DEBUG:tasks.quiescer:Yielding 2024-04-23T10:26:19.709 INFO:teuthology.run_tasks:Running task workunit... 2024-04-23T10:26:19.722 INFO:tasks.workunit:Pulling workunits from ref 0a6c3ed699031b80a2b419e7e795368719871394 2024-04-23T10:26:19.722 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-23T10:26:19.722 INFO:tasks.workunit:timeout=3h 2024-04-23T10:26:19.722 INFO:tasks.workunit:cleanup=True 2024-04-23T10:26:19.723 DEBUG:teuthology.orchestra.run.smithi097:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-23T10:26:19.723 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T10:26:19.738 INFO:teuthology.orchestra.run.smithi097.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-23T10:26:19.738 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-23T10:26:19.738 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 73h/115d Inode: 1099511628283 Links: 2 2024-04-23T10:26:19.738 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T10:26:19.738 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-23T10:26:19.738 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-04-23 10:24:13.017958474 +0000 2024-04-23T10:26:19.738 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-04-23 10:24:13.017958474 +0000 2024-04-23T10:26:19.738 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-04-23 10:26:09.129408099 +0000 2024-04-23T10:26:19.738 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-04-23 10:24:13.017958474 +0000 2024-04-23T10:26:19.739 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-23T10:26:19.739 DEBUG:teuthology.orchestra.run.smithi097:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-23T10:26:19.810 DEBUG:teuthology.orchestra.run.smithi097:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 0a6c3ed699031b80a2b419e7e795368719871394 2024-04-23T10:26:19.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.825+0000 7fa250d94640 1 -- 172.21.15.97:0/1657915709 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa24c100b10 msgr2=0x7fa24c100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:19.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.825+0000 7fa250d94640 1 --2- 172.21.15.97:0/1657915709 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa24c100b10 0x7fa24c100ef0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa23c009900 tx=0x7fa23c02a410 comp rx=0 tx=0).stop 2024-04-23T10:26:19.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.825+0000 7fa250d94640 1 -- 172.21.15.97:0/1657915709 shutdown_connections 2024-04-23T10:26:19.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.825+0000 7fa250d94640 1 --2- 172.21.15.97:0/1657915709 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa24c1072d0 0x7fa24c10eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:19.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.825+0000 7fa250d94640 1 --2- 172.21.15.97:0/1657915709 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa24c1014c0 0x7fa24c106bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:19.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.825+0000 7fa250d94640 1 --2- 172.21.15.97:0/1657915709 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa24c100b10 0x7fa24c100ef0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:19.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.825+0000 7fa250d94640 1 -- 172.21.15.97:0/1657915709 >> 172.21.15.97:0/1657915709 conn(0x7fa24c076130 msgr2=0x7fa24c076540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:19.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.825+0000 7fa250d94640 1 -- 172.21.15.97:0/1657915709 shutdown_connections 2024-04-23T10:26:19.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.825+0000 7fa250d94640 1 -- 172.21.15.97:0/1657915709 wait complete. 2024-04-23T10:26:19.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.826+0000 7fa250d94640 1 Processor -- start 2024-04-23T10:26:19.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.826+0000 7fa250d94640 1 -- start start 2024-04-23T10:26:19.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.826+0000 7fa250d94640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa24c100b10 0x7fa24c19e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:19.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.826+0000 7fa250d94640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa24c1072d0 0x7fa24c19eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:19.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.826+0000 7fa250d94640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa24c198660 0x7fa24c198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:19.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.826+0000 7fa250d94640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa24c112300 con 0x7fa24c100b10 2024-04-23T10:26:19.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.826+0000 7fa250d94640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa24c112180 con 0x7fa24c1072d0 2024-04-23T10:26:19.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.826+0000 7fa250d94640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa24c112480 con 0x7fa24c198660 2024-04-23T10:26:19.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.826+0000 7fa24a59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa24c100b10 0x7fa24c19e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:19.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.826+0000 7fa24a59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa24c100b10 0x7fa24c19e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47122/0 (socket says 172.21.15.97:47122) 2024-04-23T10:26:19.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.826+0000 7fa24a59c640 1 -- 172.21.15.97:0/3635958980 learned_addr learned my addr 172.21.15.97:0/3635958980 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:19.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.827+0000 7fa243fff640 1 --2- 172.21.15.97:0/3635958980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa24c1072d0 0x7fa24c19eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:19.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.827+0000 7fa24ad9d640 1 --2- 172.21.15.97:0/3635958980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa24c198660 0x7fa24c198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:19.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.827+0000 7fa24a59c640 1 -- 172.21.15.97:0/3635958980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa24c198660 msgr2=0x7fa24c198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:19.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.827+0000 7fa24a59c640 1 --2- 172.21.15.97:0/3635958980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa24c198660 0x7fa24c198ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:19.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.827+0000 7fa24a59c640 1 -- 172.21.15.97:0/3635958980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa24c1072d0 msgr2=0x7fa24c19eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:19.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.827+0000 7fa24a59c640 1 --2- 172.21.15.97:0/3635958980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa24c1072d0 0x7fa24c19eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:19.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.827+0000 7fa24a59c640 1 -- 172.21.15.97:0/3635958980 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa24c199260 con 0x7fa24c100b10 2024-04-23T10:26:19.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.827+0000 7fa243fff640 1 --2- 172.21.15.97:0/3635958980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa24c1072d0 0x7fa24c19eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:26:19.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.827+0000 7fa24a59c640 1 --2- 172.21.15.97:0/3635958980 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa24c100b10 0x7fa24c19e5a0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fa23c005ab0 tx=0x7fa23c003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:19.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.827+0000 7fa2437fe640 1 -- 172.21.15.97:0/3635958980 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23c03f070 con 0x7fa24c100b10 2024-04-23T10:26:19.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.827+0000 7fa250d94640 1 -- 172.21.15.97:0/3635958980 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa24c1994f0 con 0x7fa24c100b10 2024-04-23T10:26:19.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.827+0000 7fa2437fe640 1 -- 172.21.15.97:0/3635958980 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa23c031d60 con 0x7fa24c100b10 2024-04-23T10:26:19.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.827+0000 7fa2437fe640 1 -- 172.21.15.97:0/3635958980 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23c035030 con 0x7fa24c100b10 2024-04-23T10:26:19.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.827+0000 7fa250d94640 1 -- 172.21.15.97:0/3635958980 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa24c1a5680 con 0x7fa24c100b10 2024-04-23T10:26:19.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.828+0000 7fa250d94640 1 -- 172.21.15.97:0/3635958980 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa24c06a0c0 con 0x7fa24c100b10 2024-04-23T10:26:19.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.829+0000 7fa2437fe640 1 -- 172.21.15.97:0/3635958980 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fa23c042070 con 0x7fa24c100b10 2024-04-23T10:26:19.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.830+0000 7fa2437fe640 1 --2- 172.21.15.97:0/3635958980 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fa2380a3e40 0x7fa2380a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:19.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.830+0000 7fa243fff640 1 --2- 172.21.15.97:0/3635958980 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fa2380a3e40 0x7fa2380a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:19.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.830+0000 7fa2437fe640 1 -- 172.21.15.97:0/3635958980 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fa23c0f1520 con 0x7fa24c100b10 2024-04-23T10:26:19.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.831+0000 7fa243fff640 1 --2- 172.21.15.97:0/3635958980 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fa2380a3e40 0x7fa2380a6300 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa24c199eb0 tx=0x7fa234005f90 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:19.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:19.833+0000 7fa2437fe640 1 -- 172.21.15.97:0/3635958980 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa23c0bda90 con 0x7fa24c100b10 2024-04-23T10:26:19.839 INFO:tasks.workunit.client.0.smithi097.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-23T10:26:20.056 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.058+0000 7fa250d94640 1 -- 172.21.15.97:0/3635958980 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa24c0691c0 con 0x7fa24c100b10 2024-04-23T10:26:20.057 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.058+0000 7fa2437fe640 1 -- 172.21.15.97:0/3635958980 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11931 (secure 0 0 0) 0x7fa23c0bd390 con 0x7fa24c100b10 2024-04-23T10:26:20.057 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:26:20.058 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3946878762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3946878762},{"type":"v1","addr":"172.21.15.97:6835","nonce":3946878762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/209800861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":209800861},{"type":"v1","addr":"172.21.15.97:6837","nonce":209800861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T10:24:06.910134+0000","modified":"2024-04-23T10:26:09.820985+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6839/418013575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":418013575},{"type":"v1","addr":"172.21.15.97:6839","nonce":418013575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6841/747242680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":747242680},{"type":"v1","addr":"172.21.15.97:6841","nonce":747242680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6833/4292847121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4292847121},{"type":"v1","addr":"172.21.15.183:6833","nonce":4292847121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6833/3885816565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3885816565},{"type":"v1","addr":"172.21.15.196:6833","nonce":3885816565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1897093105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1897093105},{"type":"v1","addr":"172.21.15.183:6835","nonce":1897093105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1171477563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1171477563},{"type":"v1","addr":"172.21.15.196:6835","nonce":1171477563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1898341911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1898341911},{"type":"v1","addr":"172.21.15.183:6837","nonce":1898341911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.196:6837/3094946502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3094946502},{"type":"v1","addr":"172.21.15.196:6837","nonce":3094946502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.196:6839/83843556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":83843556},{"type":"v1","addr":"172.21.15.196:6839","nonce":83843556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6839/1763841130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1763841130},{"type":"v1","addr":"172.21.15.183:6839","nonce":1763841130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24487,24403,24398,24469,24488]},"id":1}]} 2024-04-23T10:26:20.058 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 26 2024-04-23T10:26:20.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.061+0000 7fa250d94640 1 -- 172.21.15.97:0/3635958980 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fa2380a3e40 msgr2=0x7fa2380a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:20.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.061+0000 7fa250d94640 1 --2- 172.21.15.97:0/3635958980 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fa2380a3e40 0x7fa2380a6300 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa24c199eb0 tx=0x7fa234005f90 comp rx=0 tx=0).stop 2024-04-23T10:26:20.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.061+0000 7fa250d94640 1 -- 172.21.15.97:0/3635958980 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa24c100b10 msgr2=0x7fa24c19e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:20.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.061+0000 7fa250d94640 1 --2- 172.21.15.97:0/3635958980 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa24c100b10 0x7fa24c19e5a0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fa23c005ab0 tx=0x7fa23c003c80 comp rx=0 tx=0).stop 2024-04-23T10:26:20.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.061+0000 7fa250d94640 1 -- 172.21.15.97:0/3635958980 shutdown_connections 2024-04-23T10:26:20.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.061+0000 7fa250d94640 1 --2- 172.21.15.97:0/3635958980 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fa2380a3e40 0x7fa2380a6300 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.061+0000 7fa250d94640 1 --2- 172.21.15.97:0/3635958980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa24c198660 0x7fa24c198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.061+0000 7fa250d94640 1 --2- 172.21.15.97:0/3635958980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa24c1072d0 0x7fa24c19eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.061+0000 7fa250d94640 1 --2- 172.21.15.97:0/3635958980 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa24c100b10 0x7fa24c19e5a0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.060 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.061+0000 7fa250d94640 1 -- 172.21.15.97:0/3635958980 >> 172.21.15.97:0/3635958980 conn(0x7fa24c076130 msgr2=0x7fa24c0fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:20.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.061+0000 7fa250d94640 1 -- 172.21.15.97:0/3635958980 shutdown_connections 2024-04-23T10:26:20.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.061+0000 7fa250d94640 1 -- 172.21.15.97:0/3635958980 wait complete. 2024-04-23T10:26:20.070 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:19 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/3840928823' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T10:26:20.070 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:19 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/3785077638' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:26:20.070 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:19 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/582176515' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T10:26:20.071 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-23T10:24:06.910134+0000', 'modified': '2024-04-23T10:26:09.820985+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24488, 'mds_1': 24487, 'mds_2': 24469, 'mds_3': 24403, 'mds_4': 24398}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14640': {'gid': 14640, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.97:6839/418013575', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6838', 'nonce': 418013575}, {'type': 'v1', 'addr': '172.21.15.97:6839', 'nonce': 418013575}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14652': {'gid': 14652, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.97:6841/747242680', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6840', 'nonce': 747242680}, {'type': 'v1', 'addr': '172.21.15.97:6841', 'nonce': 747242680}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24398': {'gid': 24398, 'name': 'k', 'rank': 4, 'incarnation': 23, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.183:6833/4292847121', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6832', 'nonce': 4292847121}, {'type': 'v1', 'addr': '172.21.15.183:6833', 'nonce': 4292847121}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24403': {'gid': 24403, 'name': 'l', 'rank': 3, 'incarnation': 21, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.196:6833/3885816565', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6832', 'nonce': 3885816565}, {'type': 'v1', 'addr': '172.21.15.196:6833', 'nonce': 3885816565}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24419': {'gid': 24419, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.183:6835/1897093105', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6834', 'nonce': 1897093105}, {'type': 'v1', 'addr': '172.21.15.183:6835', 'nonce': 1897093105}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24451': {'gid': 24451, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.196:6835/1171477563', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6834', 'nonce': 1171477563}, {'type': 'v1', 'addr': '172.21.15.196:6835', 'nonce': 1171477563}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24461': {'gid': 24461, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.183:6837/1898341911', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6836', 'nonce': 1898341911}, {'type': 'v1', 'addr': '172.21.15.183:6837', 'nonce': 1898341911}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24469': {'gid': 24469, 'name': 'c', 'rank': 2, 'incarnation': 19, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.196:6837/3094946502', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6836', 'nonce': 3094946502}, {'type': 'v1', 'addr': '172.21.15.196:6837', 'nonce': 3094946502}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24487': {'gid': 24487, 'name': 'i', 'rank': 1, 'incarnation': 17, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.196:6839/83843556', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6838', 'nonce': 83843556}, {'type': 'v1', 'addr': '172.21.15.196:6839', 'nonce': 83843556}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24488': {'gid': 24488, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.183:6839/1763841130', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6838', 'nonce': 1763841130}, {'type': 'v1', 'addr': '172.21.15.183:6839', 'nonce': 1763841130}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24488, 'qdb_cluster': [24487, 24403, 24398, 24469, 24488]} 2024-04-23T10:26:20.071 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-23T10:26:20.071 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2024-04-23T10:26:20.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.211+0000 7ff2d5e2a640 1 -- 172.21.15.97:0/777137041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2d0103b50 msgr2=0x7ff2d010c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:20.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.211+0000 7ff2d5e2a640 1 --2- 172.21.15.97:0/777137041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2d0103b50 0x7ff2d010c030 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4009900 tx=0x7ff2c402a480 comp rx=0 tx=0).stop 2024-04-23T10:26:20.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.211+0000 7ff2d5e2a640 1 -- 172.21.15.97:0/777137041 shutdown_connections 2024-04-23T10:26:20.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.211+0000 7ff2d5e2a640 1 --2- 172.21.15.97:0/777137041 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2d010c600 0x7ff2d010ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.211+0000 7ff2d5e2a640 1 --2- 172.21.15.97:0/777137041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2d0103b50 0x7ff2d010c030 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.211+0000 7ff2d5e2a640 1 --2- 172.21.15.97:0/777137041 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2d0103230 0x7ff2d0103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.211+0000 7ff2d5e2a640 1 -- 172.21.15.97:0/777137041 >> 172.21.15.97:0/777137041 conn(0x7ff2d0077f80 msgr2=0x7ff2d0078390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:20.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.211+0000 7ff2d5e2a640 1 -- 172.21.15.97:0/777137041 shutdown_connections 2024-04-23T10:26:20.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.211+0000 7ff2d5e2a640 1 -- 172.21.15.97:0/777137041 wait complete. 2024-04-23T10:26:20.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.212+0000 7ff2d5e2a640 1 Processor -- start 2024-04-23T10:26:20.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.212+0000 7ff2d5e2a640 1 -- start start 2024-04-23T10:26:20.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.212+0000 7ff2d5e2a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2d0103230 0x7ff2d019e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.212+0000 7ff2d5e2a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2d0103b50 0x7ff2d019eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.212+0000 7ff2d5e2a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2d010c600 0x7ff2d0198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.212+0000 7ff2d5e2a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d01122a0 con 0x7ff2d010c600 2024-04-23T10:26:20.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.212+0000 7ff2d5e2a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d0112120 con 0x7ff2d0103b50 2024-04-23T10:26:20.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.212+0000 7ff2d5e2a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d0112420 con 0x7ff2d0103230 2024-04-23T10:26:20.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.212+0000 7ff2cf7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2d0103230 0x7ff2d019e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.212+0000 7ff2cffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2d010c600 0x7ff2d0198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.212+0000 7ff2cffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2d010c600 0x7ff2d0198680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47130/0 (socket says 172.21.15.97:47130) 2024-04-23T10:26:20.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.212+0000 7ff2cf7fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2d0103230 0x7ff2d019e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.97:35850/0 (socket says 172.21.15.97:35850) 2024-04-23T10:26:20.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.212+0000 7ff2cffff640 1 -- 172.21.15.97:0/3122552572 learned_addr learned my addr 172.21.15.97:0/3122552572 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:20.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.212+0000 7ff2ceffd640 1 --2- 172.21.15.97:0/3122552572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2d0103b50 0x7ff2d019eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.213+0000 7ff2cffff640 1 -- 172.21.15.97:0/3122552572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2d0103230 msgr2=0x7ff2d019e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:20.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.213+0000 7ff2cffff640 1 --2- 172.21.15.97:0/3122552572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2d0103230 0x7ff2d019e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.213+0000 7ff2cffff640 1 -- 172.21.15.97:0/3122552572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2d0103b50 msgr2=0x7ff2d019eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:20.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.213+0000 7ff2cffff640 1 --2- 172.21.15.97:0/3122552572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2d0103b50 0x7ff2d019eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.213+0000 7ff2cffff640 1 -- 172.21.15.97:0/3122552572 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2d0198f40 con 0x7ff2d010c600 2024-04-23T10:26:20.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.213+0000 7ff2cffff640 1 --2- 172.21.15.97:0/3122552572 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2d010c600 0x7ff2d0198680 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7ff2c800cad0 tx=0x7ff2c800ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:20.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.213+0000 7ff2ccff9640 1 -- 172.21.15.97:0/3122552572 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c800ad60 con 0x7ff2d010c600 2024-04-23T10:26:20.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.213+0000 7ff2ccff9640 1 -- 172.21.15.97:0/3122552572 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2c8005020 con 0x7ff2d010c600 2024-04-23T10:26:20.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.213+0000 7ff2d5e2a640 1 -- 172.21.15.97:0/3122552572 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2d0199230 con 0x7ff2d010c600 2024-04-23T10:26:20.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.213+0000 7ff2ccff9640 1 -- 172.21.15.97:0/3122552572 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c800f6b0 con 0x7ff2d010c600 2024-04-23T10:26:20.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.213+0000 7ff2d5e2a640 1 -- 172.21.15.97:0/3122552572 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2d01a5540 con 0x7ff2d010c600 2024-04-23T10:26:20.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.215+0000 7ff2ccff9640 1 -- 172.21.15.97:0/3122552572 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7ff2c80051d0 con 0x7ff2d010c600 2024-04-23T10:26:20.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.215+0000 7ff2d8c1b740 1 Processor -- start 2024-04-23T10:26:20.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.215+0000 7ff2d8c1b740 1 -- start start 2024-04-23T10:26:20.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.216+0000 7ff2d8c1b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56010ba96c90 0x56010ba97070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.216+0000 7ff2d8c1b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56010ba97640 0x56010baa5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.216+0000 7ff2ccff9640 1 --2- 172.21.15.97:0/3122552572 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7ff2b80a3e40 0x7ff2b80a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.216+0000 7ff2d8c1b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56010baa61e0 0x56010baa85f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.216+0000 7ff2d8c1b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56010b9fcc50 con 0x56010ba97640 2024-04-23T10:26:20.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.216+0000 7ff2d8c1b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56010b999830 con 0x56010ba96c90 2024-04-23T10:26:20.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.216+0000 7ff2d8c1b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56010b9f8c10 con 0x56010baa61e0 2024-04-23T10:26:20.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.216+0000 7ff2cf7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56010ba97640 0x56010baa5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.216+0000 7ff2ceffd640 1 --2- 172.21.15.97:0/3122552572 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7ff2b80a3e40 0x7ff2b80a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.216+0000 7ff2cf7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56010ba97640 0x56010baa5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47138/0 (socket says 172.21.15.97:47138) 2024-04-23T10:26:20.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.216+0000 7ff2cf7fe640 1 -- 172.21.15.97:0/3709405405 learned_addr learned my addr 172.21.15.97:0/3709405405 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:20.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.216+0000 7ff2ceffd640 1 --2- 172.21.15.97:0/3709405405 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56010ba96c90 0x56010ba97070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.216+0000 7ff2ccff9640 1 -- 172.21.15.97:0/3122552572 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7ff2c80d0a40 con 0x7ff2d010c600 2024-04-23T10:26:20.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.216+0000 7ff2cffff640 1 --2- 172.21.15.97:0/3709405405 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56010baa61e0 0x56010baa85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.216+0000 7ff2cf7fe640 1 -- 172.21.15.97:0/3709405405 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56010baa61e0 msgr2=0x56010baa85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:20.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.216+0000 7ff2cf7fe640 1 --2- 172.21.15.97:0/3709405405 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56010baa61e0 0x56010baa85f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.216+0000 7ff2cf7fe640 1 -- 172.21.15.97:0/3709405405 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56010ba96c90 msgr2=0x56010ba97070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:20.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.216+0000 7ff2cf7fe640 1 --2- 172.21.15.97:0/3709405405 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56010ba96c90 0x56010ba97070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.216+0000 7ff2cf7fe640 1 -- 172.21.15.97:0/3709405405 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56010b70f2d0 con 0x56010ba97640 2024-04-23T10:26:20.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.216+0000 7ff2cffff640 1 --2- 172.21.15.97:0/3709405405 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56010baa61e0 0x56010baa85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:26:20.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.216+0000 7ff2ceffd640 1 --2- 172.21.15.97:0/3122552572 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7ff2b80a3e40 0x7ff2b80a6300 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4031440 tx=0x7ff2c4032c70 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:20.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.216+0000 7ff2cf7fe640 1 --2- 172.21.15.97:0/3709405405 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56010ba97640 0x56010baa5c10 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7ff2c0009a70 tx=0x7ff2c0009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30d39920e486dbec server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:20.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.216+0000 7ff2b5ffb640 1 -- 172.21.15.97:0/3709405405 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c0017070 con 0x56010ba97640 2024-04-23T10:26:20.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.216+0000 7ff2b5ffb640 1 -- 172.21.15.97:0/3709405405 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2c0013470 con 0x56010ba97640 2024-04-23T10:26:20.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.216+0000 7ff2b5ffb640 1 -- 172.21.15.97:0/3709405405 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c0012610 con 0x56010ba97640 2024-04-23T10:26:20.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.217+0000 7ff2d8c1b740 1 -- 172.21.15.97:0/3709405405 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56010ba97640 msgr2=0x56010baa5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:20.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.217+0000 7ff2d8c1b740 1 --2- 172.21.15.97:0/3709405405 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56010ba97640 0x56010baa5c10 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7ff2c0009a70 tx=0x7ff2c0009f30 comp rx=0 tx=0).stop 2024-04-23T10:26:20.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.217+0000 7ff2d8c1b740 1 -- 172.21.15.97:0/3709405405 shutdown_connections 2024-04-23T10:26:20.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.217+0000 7ff2d8c1b740 1 --2- 172.21.15.97:0/3709405405 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56010baa61e0 0x56010baa85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.217+0000 7ff2d8c1b740 1 --2- 172.21.15.97:0/3709405405 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56010ba97640 0x56010baa5c10 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.217+0000 7ff2d8c1b740 1 --2- 172.21.15.97:0/3709405405 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56010ba96c90 0x56010ba97070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.217+0000 7ff2d8c1b740 1 -- 172.21.15.97:0/3709405405 >> 172.21.15.97:0/3709405405 conn(0x56010ba93070 msgr2=0x56010ba93450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:20.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.217+0000 7ff2d8c1b740 1 -- 172.21.15.97:0/3709405405 shutdown_connections 2024-04-23T10:26:20.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.217+0000 7ff2d8c1b740 1 -- 172.21.15.97:0/3709405405 wait complete. 2024-04-23T10:26:20.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.217+0000 7ff2d8c1b740 1 -- start start 2024-04-23T10:26:20.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.218+0000 7ff2d8c1b740 1 Processor -- start 2024-04-23T10:26:20.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.218+0000 7ff2d8c1b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56010ba96c90 0x56010baa3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.218+0000 7ff2d8c1b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56010ba97640 0x56010baa3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.218+0000 7ff2d8c1b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56010baa61e0 0x56010baa3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.218+0000 7ff2d8c1b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56010baa9110 con 0x56010ba97640 2024-04-23T10:26:20.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.218+0000 7ff2d8c1b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56010b997470 con 0x56010ba96c90 2024-04-23T10:26:20.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.218+0000 7ff2d8c1b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56010baa9290 con 0x56010baa61e0 2024-04-23T10:26:20.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.218+0000 7ff2d8c1b740 10 client.0 _ll_register_callbacks cb 0x56010b9bcf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:26:20.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.218+0000 7ff2ceffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56010ba97640 0x56010baa3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.218+0000 7ff2ceffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56010ba97640 0x56010baa3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47142/0 (socket says 172.21.15.97:47142) 2024-04-23T10:26:20.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.218+0000 7ff2ceffd640 1 -- 172.21.15.97:0/4145087447 learned_addr learned my addr 172.21.15.97:0/4145087447 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:20.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.218+0000 7ff2cf7fe640 1 --2- 172.21.15.97:0/4145087447 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56010ba96c90 0x56010baa3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.218+0000 7ff2cffff640 1 --2- 172.21.15.97:0/4145087447 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56010baa61e0 0x56010baa3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.218+0000 7ff2ceffd640 1 -- 172.21.15.97:0/4145087447 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56010baa61e0 msgr2=0x56010baa3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:20.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.218+0000 7ff2ceffd640 1 --2- 172.21.15.97:0/4145087447 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56010baa61e0 0x56010baa3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.218+0000 7ff2ceffd640 1 -- 172.21.15.97:0/4145087447 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56010ba96c90 msgr2=0x56010baa3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:20.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.218+0000 7ff2ceffd640 1 --2- 172.21.15.97:0/4145087447 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56010ba96c90 0x56010baa3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.218+0000 7ff2ceffd640 1 -- 172.21.15.97:0/4145087447 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56010b7d96b0 con 0x56010ba97640 2024-04-23T10:26:20.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.218+0000 7ff2cf7fe640 1 --2- 172.21.15.97:0/4145087447 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56010ba96c90 0x56010baa3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:26:20.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.218+0000 7ff2cffff640 1 --2- 172.21.15.97:0/4145087447 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56010baa61e0 0x56010baa3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:26:20.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.219+0000 7ff2ceffd640 1 --2- 172.21.15.97:0/4145087447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56010ba97640 0x56010baa3590 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4033000 tx=0x7ff2c4005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc13d27882725509 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:20.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.219+0000 7ff2aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:26:20.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.219+0000 7ff2aeffd640 1 -- 172.21.15.97:0/4145087447 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c40047a0 con 0x56010ba97640 2024-04-23T10:26:20.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.219+0000 7ff2aeffd640 1 -- 172.21.15.97:0/4145087447 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7ff2c4004940 con 0x56010ba97640 2024-04-23T10:26:20.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.219+0000 7ff2d8c1b740 1 -- 172.21.15.97:0/4145087447 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2c0013470 con 0x56010ba97640 2024-04-23T10:26:20.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.219+0000 7ff2aeffd640 1 -- 172.21.15.97:0/4145087447 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff2c4059050 con 0x56010ba97640 2024-04-23T10:26:20.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.219+0000 7ff2d8c1b740 10 client.15606 fetch_fsmap learned FSMap version 26 2024-04-23T10:26:20.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.219+0000 7ff2d8c1b740 1 -- 172.21.15.97:0/4145087447 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56010bab52c0 con 0x56010ba97640 2024-04-23T10:26:20.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.219+0000 7ff2aeffd640 1 -- 172.21.15.97:0/4145087447 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2c0013470 con 0x56010ba97640 2024-04-23T10:26:20.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.219+0000 7ff2aeffd640 1 -- 172.21.15.97:0/4145087447 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff2c403d450 con 0x56010ba97640 2024-04-23T10:26:20.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.219+0000 7ff2d8c1b740 10 client.15606 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:26:20.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.219+0000 7ff2d8c1b740 10 client.15606 resolve_mds: resolved name 'k' to daemon mds.k 2024-04-23T10:26:20.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.219+0000 7ff2d8c1b740 20 client.15606 populate_metadata read hostname 'smithi097' 2024-04-23T10:26:20.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.219+0000 7ff2d8c1b740 1 --2- 172.21.15.97:0/4145087447 >> [v2:172.21.15.183:6832/4292847121,v1:172.21.15.183:6833/4292847121] conn(0x56010bab7990 0x56010bab8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.219+0000 7ff2d8c1b740 4 client.15606 mds_command: new command op to 24398 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:26:20.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.219+0000 7ff2d8c1b740 1 -- 172.21.15.97:0/4145087447 --> [v2:172.21.15.183:6832/4292847121,v1:172.21.15.183:6833/4292847121] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56010ba0a670 con 0x56010bab7990 2024-04-23T10:26:20.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.220+0000 7ff2cf7fe640 1 --2- 172.21.15.97:0/4145087447 >> [v2:172.21.15.183:6832/4292847121,v1:172.21.15.183:6833/4292847121] conn(0x56010bab7990 0x56010bab8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.220+0000 7ff2aeffd640 10 client.15606 ms_handle_connect on v2:172.21.15.183:6832/4292847121 2024-04-23T10:26:20.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.220+0000 7ff2cf7fe640 1 --2- 172.21.15.97:0/4145087447 >> [v2:172.21.15.183:6832/4292847121,v1:172.21.15.183:6833/4292847121] conn(0x56010bab7990 0x56010bab8e00 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6a72691cd0ddad2b server_cookie=7ed2b328fa978f2f in_seq=0 out_seq=0 2024-04-23T10:26:20.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.221+0000 7ff2aeffd640 1 -- 172.21.15.97:0/4145087447 <== mds.4 v2:172.21.15.183:6832/4292847121 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56010ba0a670 con 0x56010bab7990 2024-04-23T10:26:20.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.221+0000 7ff2aeffd640 10 client.15606 handle_command_reply: tid=0 2024-04-23T10:26:20.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.221+0000 7ff2d8c1b740 1 client.15606 shutdown 2024-04-23T10:26:20.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.221+0000 7ff2d8c1b740 1 -- 172.21.15.97:0/4145087447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56010ba97640 msgr2=0x56010baa3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:20.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.221+0000 7ff2d8c1b740 1 --2- 172.21.15.97:0/4145087447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56010ba97640 0x56010baa3590 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4033000 tx=0x7ff2c4005e80 comp rx=0 tx=0).stop 2024-04-23T10:26:20.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.221+0000 7ff2d8c1b740 1 -- 172.21.15.97:0/4145087447 shutdown_connections 2024-04-23T10:26:20.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.221+0000 7ff2d8c1b740 1 --2- 172.21.15.97:0/4145087447 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56010baa61e0 0x56010baa3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.221+0000 7ff2d8c1b740 1 --2- 172.21.15.97:0/4145087447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56010ba97640 0x56010baa3590 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.221+0000 7ff2d8c1b740 1 --2- 172.21.15.97:0/4145087447 >> [v2:172.21.15.183:6832/4292847121,v1:172.21.15.183:6833/4292847121] conn(0x56010bab7990 0x56010bab8e00 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.221+0000 7ff2d8c1b740 1 --2- 172.21.15.97:0/4145087447 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56010ba96c90 0x56010baa3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.221+0000 7ff2aeffd640 0 client.15606 ms_handle_reset on v2:172.21.15.183:6832/4292847121 2024-04-23T10:26:20.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.221+0000 7ff2d8c1b740 1 -- 172.21.15.97:0/4145087447 >> 172.21.15.97:0/4145087447 conn(0x56010ba93070 msgr2=0x56010ba98ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:20.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.221+0000 7ff2d8c1b740 1 -- 172.21.15.97:0/4145087447 shutdown_connections 2024-04-23T10:26:20.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.221+0000 7ff2d8c1b740 1 -- 172.21.15.97:0/4145087447 wait complete. 2024-04-23T10:26:20.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.221+0000 7ff2d8c1b740 20 client.15606 trim_cache size 0 max 16384 2024-04-23T10:26:20.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.239+0000 7ff2d8c1b740 1 Processor -- start 2024-04-23T10:26:20.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.239+0000 7ff2d8c1b740 1 -- start start 2024-04-23T10:26:20.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.239+0000 7ff2d8c1b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56010ba96c90 0x56010ba9e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.239+0000 7ff2d8c1b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56010ba97640 0x56010ba9e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.239+0000 7ff2d8c1b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56010baa61e0 0x56010baa1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.243 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.239+0000 7ff2d8c1b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56010bab7140 con 0x56010baa61e0 2024-04-23T10:26:20.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.239+0000 7ff2d8c1b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56010bab7810 con 0x56010ba96c90 2024-04-23T10:26:20.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.239+0000 7ff2d8c1b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56010bab7ee0 con 0x56010ba97640 2024-04-23T10:26:20.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.239+0000 7ff2cffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56010baa61e0 0x56010baa1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.239+0000 7ff2cffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56010baa61e0 0x56010baa1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47146/0 (socket says 172.21.15.97:47146) 2024-04-23T10:26:20.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.239+0000 7ff2cffff640 1 -- 172.21.15.97:0/2101404754 learned_addr learned my addr 172.21.15.97:0/2101404754 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:20.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.239+0000 7ff2ceffd640 1 --2- 172.21.15.97:0/2101404754 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56010ba97640 0x56010ba9e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.239+0000 7ff2cf7fe640 1 --2- 172.21.15.97:0/2101404754 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56010ba96c90 0x56010ba9e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.239+0000 7ff2cffff640 1 -- 172.21.15.97:0/2101404754 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56010ba97640 msgr2=0x56010ba9e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:20.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.239+0000 7ff2cffff640 1 --2- 172.21.15.97:0/2101404754 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56010ba97640 0x56010ba9e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.239+0000 7ff2cffff640 1 -- 172.21.15.97:0/2101404754 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56010ba96c90 msgr2=0x56010ba9e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:20.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.239+0000 7ff2cffff640 1 --2- 172.21.15.97:0/2101404754 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56010ba96c90 0x56010ba9e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.240+0000 7ff2cffff640 1 -- 172.21.15.97:0/2101404754 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56010ba973e0 con 0x56010baa61e0 2024-04-23T10:26:20.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.240+0000 7ff2ceffd640 1 --2- 172.21.15.97:0/2101404754 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56010ba97640 0x56010ba9e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:26:20.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.240+0000 7ff2cf7fe640 1 --2- 172.21.15.97:0/2101404754 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56010ba96c90 0x56010ba9e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:26:20.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.240+0000 7ff2cffff640 1 --2- 172.21.15.97:0/2101404754 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56010baa61e0 0x56010baa1960 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7ff2c80ce7e0 tx=0x7ff2c809c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd5efee699e95756 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:20.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.240+0000 7ff2b57fa640 1 -- 172.21.15.97:0/2101404754 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c801d040 con 0x56010baa61e0 2024-04-23T10:26:20.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.240+0000 7ff2b57fa640 1 -- 172.21.15.97:0/2101404754 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2c809d170 con 0x56010baa61e0 2024-04-23T10:26:20.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.240+0000 7ff2b57fa640 1 -- 172.21.15.97:0/2101404754 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c80054f0 con 0x56010baa61e0 2024-04-23T10:26:20.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.240+0000 7ff2d8c1b740 1 -- 172.21.15.97:0/2101404754 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56010baa61e0 msgr2=0x56010baa1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:20.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.240+0000 7ff2d8c1b740 1 --2- 172.21.15.97:0/2101404754 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56010baa61e0 0x56010baa1960 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7ff2c80ce7e0 tx=0x7ff2c809c6c0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.240+0000 7ff2d8c1b740 1 -- 172.21.15.97:0/2101404754 shutdown_connections 2024-04-23T10:26:20.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.240+0000 7ff2d8c1b740 1 --2- 172.21.15.97:0/2101404754 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56010baa61e0 0x56010baa1960 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.240+0000 7ff2d8c1b740 1 --2- 172.21.15.97:0/2101404754 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56010ba97640 0x56010ba9e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.240+0000 7ff2d8c1b740 1 --2- 172.21.15.97:0/2101404754 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56010ba96c90 0x56010ba9e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.240+0000 7ff2d8c1b740 1 -- 172.21.15.97:0/2101404754 >> 172.21.15.97:0/2101404754 conn(0x56010ba93070 msgr2=0x56010ba8de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:20.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.240+0000 7ff2d8c1b740 1 -- 172.21.15.97:0/2101404754 shutdown_connections 2024-04-23T10:26:20.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.240+0000 7ff2d8c1b740 1 -- 172.21.15.97:0/2101404754 wait complete. 2024-04-23T10:26:20.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.241+0000 7ff2d8c1b740 1 -- start start 2024-04-23T10:26:20.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.241+0000 7ff2d8c1b740 1 Processor -- start 2024-04-23T10:26:20.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.241+0000 7ff2d8c1b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56010ba96c90 0x56010ba9f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.241+0000 7ff2d8c1b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56010ba97640 0x56010ba9f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.245 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.241+0000 7ff2d8c1b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56010baa61e0 0x56010ba9fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.241+0000 7ff2d8c1b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56010bab8c80 con 0x56010ba97640 2024-04-23T10:26:20.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.241+0000 7ff2d8c1b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56010bab85b0 con 0x56010ba96c90 2024-04-23T10:26:20.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.241+0000 7ff2d8c1b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56010bab9350 con 0x56010baa61e0 2024-04-23T10:26:20.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.241+0000 7ff2d8c1b740 10 client.0 _ll_register_callbacks cb 0x7ff2c4032e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:26:20.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.241+0000 7ff2ceffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56010ba97640 0x56010ba9f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.241+0000 7ff2ceffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56010ba97640 0x56010ba9f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47150/0 (socket says 172.21.15.97:47150) 2024-04-23T10:26:20.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.241+0000 7ff2ceffd640 1 -- 172.21.15.97:0/4121251514 learned_addr learned my addr 172.21.15.97:0/4121251514 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:20.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.241+0000 7ff2cffff640 1 --2- 172.21.15.97:0/4121251514 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56010baa61e0 0x56010ba9fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.241+0000 7ff2cf7fe640 1 --2- 172.21.15.97:0/4121251514 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56010ba96c90 0x56010ba9f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.241+0000 7ff2ceffd640 1 -- 172.21.15.97:0/4121251514 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56010baa61e0 msgr2=0x56010ba9fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:20.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.241+0000 7ff2ceffd640 1 --2- 172.21.15.97:0/4121251514 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56010baa61e0 0x56010ba9fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.241+0000 7ff2ceffd640 1 -- 172.21.15.97:0/4121251514 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56010ba96c90 msgr2=0x56010ba9f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:20.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.241+0000 7ff2ceffd640 1 --2- 172.21.15.97:0/4121251514 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56010ba96c90 0x56010ba9f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.241+0000 7ff2ceffd640 1 -- 172.21.15.97:0/4121251514 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56010bac0180 con 0x56010ba97640 2024-04-23T10:26:20.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.241+0000 7ff2cf7fe640 1 --2- 172.21.15.97:0/4121251514 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56010ba96c90 0x56010ba9f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:26:20.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.242+0000 7ff2ceffd640 1 --2- 172.21.15.97:0/4121251514 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56010ba97640 0x56010ba9f880 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4006010 tx=0x7ff2c4005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17e7a79573b84be3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:20.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.242+0000 7ff2aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:26:20.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.242+0000 7ff2aeffd640 1 -- 172.21.15.97:0/4121251514 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c4063030 con 0x56010ba97640 2024-04-23T10:26:20.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.242+0000 7ff2aeffd640 1 -- 172.21.15.97:0/4121251514 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7ff2c403d450 con 0x56010ba97640 2024-04-23T10:26:20.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.242+0000 7ff2d8c1b740 1 -- 172.21.15.97:0/4121251514 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2c809d170 con 0x56010ba97640 2024-04-23T10:26:20.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.242+0000 7ff2aeffd640 1 -- 172.21.15.97:0/4121251514 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff2c403d650 con 0x56010ba97640 2024-04-23T10:26:20.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.242+0000 7ff2d8c1b740 10 client.15612 fetch_fsmap learned FSMap version 26 2024-04-23T10:26:20.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.242+0000 7ff2d8c1b740 1 -- 172.21.15.97:0/4121251514 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56010bac1410 con 0x56010ba97640 2024-04-23T10:26:20.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.242+0000 7ff2aeffd640 1 -- 172.21.15.97:0/4121251514 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2c809d170 con 0x56010ba97640 2024-04-23T10:26:20.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.242+0000 7ff2aeffd640 1 -- 172.21.15.97:0/4121251514 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff2c4003960 con 0x56010ba97640 2024-04-23T10:26:20.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.242+0000 7ff2d8c1b740 10 client.15612 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:26:20.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.242+0000 7ff2d8c1b740 10 client.15612 resolve_mds: resolved name 'k' to daemon mds.k 2024-04-23T10:26:20.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.242+0000 7ff2d8c1b740 20 client.15612 populate_metadata read hostname 'smithi097' 2024-04-23T10:26:20.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.242+0000 7ff2d8c1b740 1 --2- 172.21.15.97:0/4121251514 >> [v2:172.21.15.183:6832/4292847121,v1:172.21.15.183:6833/4292847121] conn(0x56010bb21dc0 0x56010bb20ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.242+0000 7ff2d8c1b740 4 client.15612 mds_command: new command op to 24398 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T10:26:20.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.242+0000 7ff2d8c1b740 1 -- 172.21.15.97:0/4121251514 --> [v2:172.21.15.183:6832/4292847121,v1:172.21.15.183:6833/4292847121] -- command(tid 0: {"prefix": "status"}) -- 0x56010ba044e0 con 0x56010bb21dc0 2024-04-23T10:26:20.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.243+0000 7ff2cf7fe640 1 --2- 172.21.15.97:0/4121251514 >> [v2:172.21.15.183:6832/4292847121,v1:172.21.15.183:6833/4292847121] conn(0x56010bb21dc0 0x56010bb20ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.243+0000 7ff2aeffd640 10 client.15612 ms_handle_connect on v2:172.21.15.183:6832/4292847121 2024-04-23T10:26:20.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.243+0000 7ff2cf7fe640 1 --2- 172.21.15.97:0/4121251514 >> [v2:172.21.15.183:6832/4292847121,v1:172.21.15.183:6833/4292847121] conn(0x56010bb21dc0 0x56010bb20ad0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=aae3aaf9664fd51a server_cookie=d4e79cb8902921b5 in_seq=0 out_seq=0 2024-04-23T10:26:20.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.243+0000 7ff2aeffd640 1 -- 172.21.15.97:0/4121251514 <== mds.4 v2:172.21.15.183:6832/4292847121 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56010ba044e0 con 0x56010bb21dc0 2024-04-23T10:26:20.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.243+0000 7ff2aeffd640 10 client.15612 handle_command_reply: tid=0 2024-04-23T10:26:20.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.243+0000 7ff2d8c1b740 1 client.15612 shutdown 2024-04-23T10:26:20.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.243+0000 7ff2d8c1b740 1 -- 172.21.15.97:0/4121251514 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56010ba97640 msgr2=0x56010ba9f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:20.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.243+0000 7ff2d8c1b740 1 --2- 172.21.15.97:0/4121251514 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56010ba97640 0x56010ba9f880 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4006010 tx=0x7ff2c4005710 comp rx=0 tx=0).stop 2024-04-23T10:26:20.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.243+0000 7ff2d8c1b740 1 -- 172.21.15.97:0/4121251514 shutdown_connections 2024-04-23T10:26:20.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.243+0000 7ff2d8c1b740 1 --2- 172.21.15.97:0/4121251514 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56010baa61e0 0x56010ba9fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.243+0000 7ff2d8c1b740 1 --2- 172.21.15.97:0/4121251514 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56010ba97640 0x56010ba9f880 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.243+0000 7ff2d8c1b740 1 --2- 172.21.15.97:0/4121251514 >> [v2:172.21.15.183:6832/4292847121,v1:172.21.15.183:6833/4292847121] conn(0x56010bb21dc0 0x56010bb20ad0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.243+0000 7ff2d8c1b740 1 --2- 172.21.15.97:0/4121251514 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56010ba96c90 0x56010ba9f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.243+0000 7ff2aeffd640 0 client.15612 ms_handle_reset on v2:172.21.15.183:6832/4292847121 2024-04-23T10:26:20.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.243+0000 7ff2d8c1b740 1 -- 172.21.15.97:0/4121251514 >> 172.21.15.97:0/4121251514 conn(0x56010ba93070 msgr2=0x56010ba8de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:20.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.243+0000 7ff2d8c1b740 1 -- 172.21.15.97:0/4121251514 shutdown_connections 2024-04-23T10:26:20.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.243+0000 7ff2d8c1b740 1 -- 172.21.15.97:0/4121251514 wait complete. 2024-04-23T10:26:20.249 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.243+0000 7ff2d8c1b740 20 client.15612 trim_cache size 0 max 16384 2024-04-23T10:26:20.249 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-23T10:26:20.249 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "9b22af88-0159-11ef-bc93-c7b262605968", 2024-04-23T10:26:20.249 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 4, 2024-04-23T10:26:20.249 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 24398, 2024-04-23T10:26:20.249 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-04-23T10:26:20.249 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-04-23T10:26:20.249 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-04-23T10:26:20.249 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 128.277249195, 2024-04-23T10:26:20.249 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 26, 2024-04-23T10:26:20.249 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 89, 2024-04-23T10:26:20.250 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 88, 2024-04-23T10:26:20.250 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 167.81909438100001 2024-04-23T10:26:20.250 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-23T10:26:20.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.244+0000 7ff2d5e2a640 1 -- 172.21.15.97:0/3122552572 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7ff2b80a3e40 msgr2=0x7ff2b80a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:20.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.244+0000 7ff2d5e2a640 1 --2- 172.21.15.97:0/3122552572 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7ff2b80a3e40 0x7ff2b80a6300 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4031440 tx=0x7ff2c4032c70 comp rx=0 tx=0).stop 2024-04-23T10:26:20.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.244+0000 7ff2d5e2a640 1 -- 172.21.15.97:0/3122552572 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2d010c600 msgr2=0x7ff2d0198680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:20.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.244+0000 7ff2d5e2a640 1 --2- 172.21.15.97:0/3122552572 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2d010c600 0x7ff2d0198680 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7ff2c800cad0 tx=0x7ff2c800ced0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.244+0000 7ff2d5e2a640 1 -- 172.21.15.97:0/3122552572 shutdown_connections 2024-04-23T10:26:20.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.244+0000 7ff2d5e2a640 1 --2- 172.21.15.97:0/3122552572 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7ff2b80a3e40 0x7ff2b80a6300 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.244+0000 7ff2d5e2a640 1 --2- 172.21.15.97:0/3122552572 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2d010c600 0x7ff2d0198680 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.244+0000 7ff2d5e2a640 1 --2- 172.21.15.97:0/3122552572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2d0103b50 0x7ff2d019eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.244+0000 7ff2d5e2a640 1 --2- 172.21.15.97:0/3122552572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2d0103230 0x7ff2d019e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.244+0000 7ff2d5e2a640 1 -- 172.21.15.97:0/3122552572 >> 172.21.15.97:0/3122552572 conn(0x7ff2d0077f80 msgr2=0x7ff2d00fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:20.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.244+0000 7ff2d5e2a640 1 -- 172.21.15.97:0/3122552572 shutdown_connections 2024-04-23T10:26:20.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.244+0000 7ff2d5e2a640 1 -- 172.21.15.97:0/3122552572 wait complete. 2024-04-23T10:26:20.253 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.l status 2024-04-23T10:26:20.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:19 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/3840928823' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T10:26:20.370 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:19 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/3785077638' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:26:20.370 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:19 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/582176515' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T10:26:20.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.354+0000 7f929096f640 1 -- 172.21.15.97:0/582443987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f928c10e960 msgr2=0x7f928c110d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:20.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.354+0000 7f929096f640 1 --2- 172.21.15.97:0/582443987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f928c10e960 0x7f928c110d50 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f9284009ca0 tx=0x7f928402a8c0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.354+0000 7f929096f640 1 -- 172.21.15.97:0/582443987 shutdown_connections 2024-04-23T10:26:20.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.354+0000 7f929096f640 1 --2- 172.21.15.97:0/582443987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f928c10e960 0x7f928c110d50 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.354+0000 7f929096f640 1 --2- 172.21.15.97:0/582443987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f928c1016d0 0x7f928c10e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.354+0000 7f929096f640 1 --2- 172.21.15.97:0/582443987 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f928c100db0 0x7f928c101190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.354+0000 7f929096f640 1 -- 172.21.15.97:0/582443987 >> 172.21.15.97:0/582443987 conn(0x7f928c077f80 msgr2=0x7f928c078390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:20.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.354+0000 7f929096f640 1 -- 172.21.15.97:0/582443987 shutdown_connections 2024-04-23T10:26:20.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.354+0000 7f929096f640 1 -- 172.21.15.97:0/582443987 wait complete. 2024-04-23T10:26:20.378 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.355+0000 7f929096f640 1 Processor -- start 2024-04-23T10:26:20.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.355+0000 7f929096f640 1 -- start start 2024-04-23T10:26:20.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.355+0000 7f929096f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f928c100db0 0x7f928c1a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.355+0000 7f929096f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f928c1016d0 0x7f928c1a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.355+0000 7f929096f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f928c10e960 0x7f928c19a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.355+0000 7f929096f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f928c114450 con 0x7f928c1016d0 2024-04-23T10:26:20.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.355+0000 7f929096f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f928c1142d0 con 0x7f928c100db0 2024-04-23T10:26:20.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.355+0000 7f929096f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f928c1145d0 con 0x7f928c10e960 2024-04-23T10:26:20.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.355+0000 7f9283fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f928c1016d0 0x7f928c1a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.355+0000 7f9283fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f928c1016d0 0x7f928c1a0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47182/0 (socket says 172.21.15.97:47182) 2024-04-23T10:26:20.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.355+0000 7f9283fff640 1 -- 172.21.15.97:0/509991330 learned_addr learned my addr 172.21.15.97:0/509991330 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:20.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.355+0000 7f928a59c640 1 --2- 172.21.15.97:0/509991330 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f928c100db0 0x7f928c1a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.355+0000 7f9283fff640 1 -- 172.21.15.97:0/509991330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f928c10e960 msgr2=0x7f928c19a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:20.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.355+0000 7f9283fff640 1 --2- 172.21.15.97:0/509991330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f928c10e960 0x7f928c19a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.379 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.355+0000 7f9283fff640 1 -- 172.21.15.97:0/509991330 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f928c100db0 msgr2=0x7f928c1a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:20.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.355+0000 7f9283fff640 1 --2- 172.21.15.97:0/509991330 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f928c100db0 0x7f928c1a06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.355+0000 7f9283fff640 1 -- 172.21.15.97:0/509991330 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f928c19b030 con 0x7f928c1016d0 2024-04-23T10:26:20.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.356+0000 7f9283fff640 1 --2- 172.21.15.97:0/509991330 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f928c1016d0 0x7f928c1a0bf0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f927400d770 tx=0x7f927400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:20.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.356+0000 7f92837fe640 1 -- 172.21.15.97:0/509991330 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9274017070 con 0x7f928c1016d0 2024-04-23T10:26:20.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.356+0000 7f92837fe640 1 -- 172.21.15.97:0/509991330 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92740054d0 con 0x7f928c1016d0 2024-04-23T10:26:20.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.356+0000 7f929096f640 1 -- 172.21.15.97:0/509991330 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f928c19b320 con 0x7f928c1016d0 2024-04-23T10:26:20.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.356+0000 7f92837fe640 1 -- 172.21.15.97:0/509991330 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9274004c30 con 0x7f928c1016d0 2024-04-23T10:26:20.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.356+0000 7f929096f640 1 -- 172.21.15.97:0/509991330 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f928c1a7630 con 0x7f928c1016d0 2024-04-23T10:26:20.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.358+0000 7f92837fe640 1 -- 172.21.15.97:0/509991330 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f9274004df0 con 0x7f928c1016d0 2024-04-23T10:26:20.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.358+0000 7f9293766740 1 Processor -- start 2024-04-23T10:26:20.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.359+0000 7f92837fe640 1 --2- 172.21.15.97:0/509991330 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f92780a3ec0 0x7f92780a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.359+0000 7f9293766740 1 -- start start 2024-04-23T10:26:20.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.359+0000 7f9293766740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e47ea0edb0 0x55e47ea0f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.380 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.359+0000 7f928ad9d640 1 --2- 172.21.15.97:0/509991330 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f92780a3ec0 0x7f92780a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.359+0000 7f9293766740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e47ea0f760 0x55e47ea19800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.359+0000 7f9293766740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e47ea19dd0 0x55e47ea1c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.359+0000 7f9293766740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e47e974c50 con 0x55e47ea0f760 2024-04-23T10:26:20.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.359+0000 7f9293766740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e47e911830 con 0x55e47ea19dd0 2024-04-23T10:26:20.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.359+0000 7f9293766740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e47e970c10 con 0x55e47ea0edb0 2024-04-23T10:26:20.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.359+0000 7f92837fe640 1 -- 172.21.15.97:0/509991330 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f92740d0500 con 0x7f928c1016d0 2024-04-23T10:26:20.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.359+0000 7f9283fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e47ea0f760 0x55e47ea19800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.359+0000 7f9283fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e47ea0f760 0x55e47ea19800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47198/0 (socket says 172.21.15.97:47198) 2024-04-23T10:26:20.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.359+0000 7f9283fff640 1 -- 172.21.15.97:0/3742351833 learned_addr learned my addr 172.21.15.97:0/3742351833 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:20.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.359+0000 7f928ad9d640 1 --2- 172.21.15.97:0/3742351833 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e47ea19dd0 0x55e47ea1c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.359+0000 7f928a59c640 1 --2- 172.21.15.97:0/3742351833 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e47ea0edb0 0x55e47ea0f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.359+0000 7f9283fff640 1 -- 172.21.15.97:0/3742351833 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e47ea0edb0 msgr2=0x55e47ea0f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:20.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.359+0000 7f9283fff640 1 --2- 172.21.15.97:0/3742351833 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e47ea0edb0 0x55e47ea0f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.359+0000 7f9283fff640 1 -- 172.21.15.97:0/3742351833 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e47ea19dd0 msgr2=0x55e47ea1c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:20.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.359+0000 7f9283fff640 1 --2- 172.21.15.97:0/3742351833 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e47ea19dd0 0x55e47ea1c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.359+0000 7f9283fff640 1 -- 172.21.15.97:0/3742351833 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e47e6872d0 con 0x55e47ea0f760 2024-04-23T10:26:20.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.359+0000 7f928ad9d640 1 --2- 172.21.15.97:0/509991330 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f92780a3ec0 0x7f92780a6380 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f928402ade0 tx=0x7f9284005fc0 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:20.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.359+0000 7f9283fff640 1 --2- 172.21.15.97:0/3742351833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e47ea0f760 0x55e47ea19800 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f92740ce2a0 tx=0x7f927409cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2176c92675453c9b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:20.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.359+0000 7f9280ff9640 1 -- 172.21.15.97:0/3742351833 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9274020040 con 0x55e47ea0f760 2024-04-23T10:26:20.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.360+0000 7f9280ff9640 1 -- 172.21.15.97:0/3742351833 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f927409a5a0 con 0x55e47ea0f760 2024-04-23T10:26:20.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.360+0000 7f9280ff9640 1 -- 172.21.15.97:0/3742351833 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92740ce000 con 0x55e47ea0f760 2024-04-23T10:26:20.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.360+0000 7f9293766740 1 -- 172.21.15.97:0/3742351833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e47ea0f760 msgr2=0x55e47ea19800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:20.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.360+0000 7f9293766740 1 --2- 172.21.15.97:0/3742351833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e47ea0f760 0x55e47ea19800 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f92740ce2a0 tx=0x7f927409cb10 comp rx=0 tx=0).stop 2024-04-23T10:26:20.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.360+0000 7f9293766740 1 -- 172.21.15.97:0/3742351833 shutdown_connections 2024-04-23T10:26:20.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.360+0000 7f9293766740 1 --2- 172.21.15.97:0/3742351833 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e47ea19dd0 0x55e47ea1c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.360+0000 7f9293766740 1 --2- 172.21.15.97:0/3742351833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e47ea0f760 0x55e47ea19800 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.360+0000 7f9293766740 1 --2- 172.21.15.97:0/3742351833 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e47ea0edb0 0x55e47ea0f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.360+0000 7f9293766740 1 -- 172.21.15.97:0/3742351833 >> 172.21.15.97:0/3742351833 conn(0x55e47ea0b070 msgr2=0x55e47ea0b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:20.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.360+0000 7f9293766740 1 -- 172.21.15.97:0/3742351833 shutdown_connections 2024-04-23T10:26:20.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.360+0000 7f9293766740 1 -- 172.21.15.97:0/3742351833 wait complete. 2024-04-23T10:26:20.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.361+0000 7f9293766740 1 -- start start 2024-04-23T10:26:20.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.361+0000 7f9293766740 1 Processor -- start 2024-04-23T10:26:20.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.361+0000 7f9293766740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e47ea0edb0 0x55e47ea16d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.361+0000 7f9293766740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e47ea0f760 0x55e47ea17280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.361+0000 7f9293766740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e47ea19dd0 0x55e47ea177c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.361+0000 7f9293766740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e47ea1cd40 con 0x55e47ea19dd0 2024-04-23T10:26:20.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.361+0000 7f9293766740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e47e90f470 con 0x55e47ea0edb0 2024-04-23T10:26:20.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.361+0000 7f9293766740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e47ea1cec0 con 0x55e47ea0f760 2024-04-23T10:26:20.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.361+0000 7f9293766740 10 client.0 _ll_register_callbacks cb 0x55e47e934f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:26:20.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.361+0000 7f928ad9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e47ea19dd0 0x55e47ea177c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.361+0000 7f928ad9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e47ea19dd0 0x55e47ea177c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47212/0 (socket says 172.21.15.97:47212) 2024-04-23T10:26:20.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.361+0000 7f928ad9d640 1 -- 172.21.15.97:0/64268910 learned_addr learned my addr 172.21.15.97:0/64268910 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:20.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.361+0000 7f928a59c640 1 --2- 172.21.15.97:0/64268910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e47ea0edb0 0x55e47ea16d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.361+0000 7f9283fff640 1 --2- 172.21.15.97:0/64268910 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e47ea0f760 0x55e47ea17280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.361+0000 7f928ad9d640 1 -- 172.21.15.97:0/64268910 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e47ea0f760 msgr2=0x55e47ea17280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:20.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.361+0000 7f928ad9d640 1 --2- 172.21.15.97:0/64268910 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e47ea0f760 0x55e47ea17280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.361+0000 7f928ad9d640 1 -- 172.21.15.97:0/64268910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e47ea0edb0 msgr2=0x55e47ea16d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:20.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.361+0000 7f928ad9d640 1 --2- 172.21.15.97:0/64268910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e47ea0edb0 0x55e47ea16d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.361+0000 7f928ad9d640 1 -- 172.21.15.97:0/64268910 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e47e7516b0 con 0x55e47ea19dd0 2024-04-23T10:26:20.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.361+0000 7f9283fff640 1 --2- 172.21.15.97:0/64268910 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e47ea0f760 0x55e47ea17280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:26:20.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.362+0000 7f928ad9d640 1 --2- 172.21.15.97:0/64268910 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e47ea19dd0 0x55e47ea177c0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f9284009a60 tx=0x7f928403f070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cf63f476194de4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:20.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.362+0000 7f9271ffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:26:20.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.362+0000 7f9271ffb640 1 -- 172.21.15.97:0/64268910 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92840045d0 con 0x55e47ea19dd0 2024-04-23T10:26:20.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.362+0000 7f9271ffb640 1 -- 172.21.15.97:0/64268910 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f9284004770 con 0x55e47ea19dd0 2024-04-23T10:26:20.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.362+0000 7f9293766740 1 -- 172.21.15.97:0/64268910 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f927409a5a0 con 0x55e47ea19dd0 2024-04-23T10:26:20.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.362+0000 7f9271ffb640 1 -- 172.21.15.97:0/64268910 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f928402f660 con 0x55e47ea19dd0 2024-04-23T10:26:20.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.362+0000 7f9293766740 10 client.15624 fetch_fsmap learned FSMap version 26 2024-04-23T10:26:20.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.362+0000 7f9293766740 1 -- 172.21.15.97:0/64268910 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e47ea29040 con 0x55e47ea19dd0 2024-04-23T10:26:20.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.362+0000 7f9271ffb640 1 -- 172.21.15.97:0/64268910 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f927409a5a0 con 0x55e47ea19dd0 2024-04-23T10:26:20.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.362+0000 7f9271ffb640 1 -- 172.21.15.97:0/64268910 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f9284030960 con 0x55e47ea19dd0 2024-04-23T10:26:20.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.362+0000 7f9293766740 10 client.15624 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:26:20.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.362+0000 7f9293766740 10 client.15624 resolve_mds: resolved name 'l' to daemon mds.l 2024-04-23T10:26:20.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.362+0000 7f9293766740 20 client.15624 populate_metadata read hostname 'smithi097' 2024-04-23T10:26:20.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.362+0000 7f9293766740 1 --2- 172.21.15.97:0/64268910 >> [v2:172.21.15.196:6832/3885816565,v1:172.21.15.196:6833/3885816565] conn(0x55e47ea2b670 0x55e47ea2cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.362+0000 7f9293766740 4 client.15624 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:26:20.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.362+0000 7f9293766740 1 -- 172.21.15.97:0/64268910 --> [v2:172.21.15.196:6832/3885816565,v1:172.21.15.196:6833/3885816565] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e47e982670 con 0x55e47ea2b670 2024-04-23T10:26:20.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.363+0000 7f928a59c640 1 --2- 172.21.15.97:0/64268910 >> [v2:172.21.15.196:6832/3885816565,v1:172.21.15.196:6833/3885816565] conn(0x55e47ea2b670 0x55e47ea2cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.363+0000 7f928a59c640 1 --2- 172.21.15.97:0/64268910 >> [v2:172.21.15.196:6832/3885816565,v1:172.21.15.196:6833/3885816565] conn(0x55e47ea2b670 0x55e47ea2cae0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=528d20ec94a5e523 server_cookie=5e6f64daccd09fc5 in_seq=0 out_seq=0 2024-04-23T10:26:20.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.363+0000 7f9271ffb640 10 client.15624 ms_handle_connect on v2:172.21.15.196:6832/3885816565 2024-04-23T10:26:20.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.364+0000 7f9271ffb640 1 -- 172.21.15.97:0/64268910 <== mds.3 v2:172.21.15.196:6832/3885816565 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e47e982670 con 0x55e47ea2b670 2024-04-23T10:26:20.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.364+0000 7f9271ffb640 10 client.15624 handle_command_reply: tid=0 2024-04-23T10:26:20.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.364+0000 7f9293766740 1 client.15624 shutdown 2024-04-23T10:26:20.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.364+0000 7f9293766740 1 -- 172.21.15.97:0/64268910 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e47ea19dd0 msgr2=0x55e47ea177c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:20.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.364+0000 7f9293766740 1 --2- 172.21.15.97:0/64268910 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e47ea19dd0 0x55e47ea177c0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f9284009a60 tx=0x7f928403f070 comp rx=0 tx=0).stop 2024-04-23T10:26:20.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.364+0000 7f9293766740 1 -- 172.21.15.97:0/64268910 shutdown_connections 2024-04-23T10:26:20.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.364+0000 7f9293766740 1 --2- 172.21.15.97:0/64268910 >> [v2:172.21.15.196:6832/3885816565,v1:172.21.15.196:6833/3885816565] conn(0x55e47ea2b670 0x55e47ea2cae0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.364+0000 7f9293766740 1 --2- 172.21.15.97:0/64268910 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e47ea19dd0 0x55e47ea177c0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.364+0000 7f9271ffb640 0 client.15624 ms_handle_reset on v2:172.21.15.196:6832/3885816565 2024-04-23T10:26:20.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.364+0000 7f9293766740 1 --2- 172.21.15.97:0/64268910 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e47ea0f760 0x55e47ea17280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.364+0000 7f9293766740 1 --2- 172.21.15.97:0/64268910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e47ea0edb0 0x55e47ea16d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.364+0000 7f9293766740 1 -- 172.21.15.97:0/64268910 >> 172.21.15.97:0/64268910 conn(0x55e47ea0b070 msgr2=0x55e47ea10ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:20.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.364+0000 7f9293766740 1 -- 172.21.15.97:0/64268910 shutdown_connections 2024-04-23T10:26:20.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.364+0000 7f9293766740 1 -- 172.21.15.97:0/64268910 wait complete. 2024-04-23T10:26:20.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.364+0000 7f9293766740 20 client.15624 trim_cache size 0 max 16384 2024-04-23T10:26:20.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.382+0000 7f9293766740 1 Processor -- start 2024-04-23T10:26:20.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.382+0000 7f9293766740 1 -- start start 2024-04-23T10:26:20.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.382+0000 7f9293766740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e47ea0edb0 0x55e47ea115f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.382+0000 7f9293766740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e47ea0f760 0x55e47ea12020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.382+0000 7f9293766740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e47ea19dd0 0x55e47ea12560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.382+0000 7f9293766740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e47ea2ae20 con 0x55e47ea0edb0 2024-04-23T10:26:20.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.382+0000 7f9293766740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e47ea2b4f0 con 0x55e47ea19dd0 2024-04-23T10:26:20.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.382+0000 7f9293766740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e47ea2bbc0 con 0x55e47ea0f760 2024-04-23T10:26:20.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.382+0000 7f928a59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e47ea0edb0 0x55e47ea115f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.382+0000 7f928a59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e47ea0edb0 0x55e47ea115f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47214/0 (socket says 172.21.15.97:47214) 2024-04-23T10:26:20.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.382+0000 7f928a59c640 1 -- 172.21.15.97:0/1866590830 learned_addr learned my addr 172.21.15.97:0/1866590830 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:20.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.382+0000 7f928ad9d640 1 --2- 172.21.15.97:0/1866590830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e47ea19dd0 0x55e47ea12560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.382+0000 7f928a59c640 1 -- 172.21.15.97:0/1866590830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e47ea0f760 msgr2=0x55e47ea12020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:20.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.382+0000 7f928a59c640 1 --2- 172.21.15.97:0/1866590830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e47ea0f760 0x55e47ea12020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.382+0000 7f928a59c640 1 -- 172.21.15.97:0/1866590830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e47ea19dd0 msgr2=0x55e47ea12560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:20.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.382+0000 7f928a59c640 1 --2- 172.21.15.97:0/1866590830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e47ea19dd0 0x55e47ea12560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.382+0000 7f928a59c640 1 -- 172.21.15.97:0/1866590830 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e47ea15290 con 0x55e47ea0edb0 2024-04-23T10:26:20.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.382+0000 7f928ad9d640 1 --2- 172.21.15.97:0/1866590830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e47ea19dd0 0x55e47ea12560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:26:20.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.383+0000 7f928a59c640 1 --2- 172.21.15.97:0/1866590830 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e47ea0edb0 0x55e47ea115f0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f927c00ca50 tx=0x7f927c00cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99f71e3a75e915e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:20.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.383+0000 7f9273fff640 1 -- 172.21.15.97:0/1866590830 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f927c01c630 con 0x55e47ea0edb0 2024-04-23T10:26:20.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.383+0000 7f9273fff640 1 -- 172.21.15.97:0/1866590830 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f927c00edf0 con 0x55e47ea0edb0 2024-04-23T10:26:20.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.383+0000 7f9273fff640 1 -- 172.21.15.97:0/1866590830 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f927c0047a0 con 0x55e47ea0edb0 2024-04-23T10:26:20.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.383+0000 7f9293766740 1 -- 172.21.15.97:0/1866590830 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e47ea0edb0 msgr2=0x55e47ea115f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:20.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.383+0000 7f9293766740 1 --2- 172.21.15.97:0/1866590830 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e47ea0edb0 0x55e47ea115f0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f927c00ca50 tx=0x7f927c00cca0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.383+0000 7f9293766740 1 -- 172.21.15.97:0/1866590830 shutdown_connections 2024-04-23T10:26:20.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.383+0000 7f9293766740 1 --2- 172.21.15.97:0/1866590830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e47ea19dd0 0x55e47ea12560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.383+0000 7f9293766740 1 --2- 172.21.15.97:0/1866590830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e47ea0f760 0x55e47ea12020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.383+0000 7f9293766740 1 --2- 172.21.15.97:0/1866590830 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e47ea0edb0 0x55e47ea115f0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.383+0000 7f9293766740 1 -- 172.21.15.97:0/1866590830 >> 172.21.15.97:0/1866590830 conn(0x55e47ea0b070 msgr2=0x55e47ea0b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:20.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.383+0000 7f9293766740 1 -- 172.21.15.97:0/1866590830 shutdown_connections 2024-04-23T10:26:20.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.383+0000 7f9293766740 1 -- 172.21.15.97:0/1866590830 wait complete. 2024-04-23T10:26:20.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.384+0000 7f9293766740 1 -- start start 2024-04-23T10:26:20.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.384+0000 7f9293766740 1 Processor -- start 2024-04-23T10:26:20.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.384+0000 7f9293766740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e47ea0edb0 0x55e47ea35150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.384+0000 7f9293766740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e47ea0f760 0x55e47ea2e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.384+0000 7f9293766740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e47ea19dd0 0x55e47ea2ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.384+0000 7f9293766740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e47ea2c960 con 0x55e47ea0edb0 2024-04-23T10:26:20.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.384+0000 7f9293766740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e47ea2c290 con 0x55e47ea19dd0 2024-04-23T10:26:20.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.384+0000 7f9293766740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e47ea2d030 con 0x55e47ea0f760 2024-04-23T10:26:20.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.384+0000 7f9293766740 10 client.0 _ll_register_callbacks cb 0x7f9284009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:26:20.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.384+0000 7f928a59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e47ea0edb0 0x55e47ea35150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.384+0000 7f928a59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e47ea0edb0 0x55e47ea35150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47224/0 (socket says 172.21.15.97:47224) 2024-04-23T10:26:20.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.384+0000 7f928a59c640 1 -- 172.21.15.97:0/374445263 learned_addr learned my addr 172.21.15.97:0/374445263 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:20.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.384+0000 7f928ad9d640 1 --2- 172.21.15.97:0/374445263 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e47ea19dd0 0x55e47ea2ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.384+0000 7f928a59c640 1 -- 172.21.15.97:0/374445263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e47ea0f760 msgr2=0x55e47ea2e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:20.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.384+0000 7f9283fff640 1 --2- 172.21.15.97:0/374445263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e47ea0f760 0x55e47ea2e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.384+0000 7f928a59c640 1 --2- 172.21.15.97:0/374445263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e47ea0f760 0x55e47ea2e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.384+0000 7f928a59c640 1 -- 172.21.15.97:0/374445263 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e47ea19dd0 msgr2=0x55e47ea2ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:20.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.384+0000 7f928a59c640 1 --2- 172.21.15.97:0/374445263 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e47ea19dd0 0x55e47ea2ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.384+0000 7f928a59c640 1 -- 172.21.15.97:0/374445263 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e47ea2f260 con 0x55e47ea0edb0 2024-04-23T10:26:20.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.384+0000 7f928a59c640 1 --2- 172.21.15.97:0/374445263 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e47ea0edb0 0x55e47ea35150 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f927c005db0 tx=0x7f927c0054d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7fab9c5bb8d7344 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:20.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.384+0000 7f9271ffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:26:20.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.385+0000 7f9271ffb640 1 -- 172.21.15.97:0/374445263 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f927c0056c0 con 0x55e47ea0edb0 2024-04-23T10:26:20.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.385+0000 7f9271ffb640 1 -- 172.21.15.97:0/374445263 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f927c01c430 con 0x55e47ea0edb0 2024-04-23T10:26:20.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.385+0000 7f9293766740 1 -- 172.21.15.97:0/374445263 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f927c00edf0 con 0x55e47ea0edb0 2024-04-23T10:26:20.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.385+0000 7f9271ffb640 1 -- 172.21.15.97:0/374445263 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f927c01c630 con 0x55e47ea0edb0 2024-04-23T10:26:20.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.385+0000 7f9293766740 10 client.15630 fetch_fsmap learned FSMap version 26 2024-04-23T10:26:20.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.385+0000 7f9293766740 1 -- 172.21.15.97:0/374445263 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e47ea91620 con 0x55e47ea0edb0 2024-04-23T10:26:20.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.385+0000 7f9271ffb640 1 -- 172.21.15.97:0/374445263 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f927c00edf0 con 0x55e47ea0edb0 2024-04-23T10:26:20.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.385+0000 7f9271ffb640 1 -- 172.21.15.97:0/374445263 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f927c02d020 con 0x55e47ea0edb0 2024-04-23T10:26:20.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.385+0000 7f9293766740 10 client.15630 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:26:20.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.385+0000 7f9293766740 10 client.15630 resolve_mds: resolved name 'l' to daemon mds.l 2024-04-23T10:26:20.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.385+0000 7f9293766740 20 client.15630 populate_metadata read hostname 'smithi097' 2024-04-23T10:26:20.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.385+0000 7f9293766740 1 --2- 172.21.15.97:0/374445263 >> [v2:172.21.15.196:6832/3885816565,v1:172.21.15.196:6833/3885816565] conn(0x55e47ea96070 0x55e47ea94d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.385+0000 7f9293766740 4 client.15630 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T10:26:20.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.385+0000 7f9293766740 1 -- 172.21.15.97:0/374445263 --> [v2:172.21.15.196:6832/3885816565,v1:172.21.15.196:6833/3885816565] -- command(tid 0: {"prefix": "status"}) -- 0x55e47e97c4e0 con 0x55e47ea96070 2024-04-23T10:26:20.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.385+0000 7f9283fff640 1 --2- 172.21.15.97:0/374445263 >> [v2:172.21.15.196:6832/3885816565,v1:172.21.15.196:6833/3885816565] conn(0x55e47ea96070 0x55e47ea94d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.386+0000 7f9283fff640 1 --2- 172.21.15.97:0/374445263 >> [v2:172.21.15.196:6832/3885816565,v1:172.21.15.196:6833/3885816565] conn(0x55e47ea96070 0x55e47ea94d80 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3100d1ef15aab1a6 server_cookie=658709bff307c1e3 in_seq=0 out_seq=0 2024-04-23T10:26:20.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.386+0000 7f9271ffb640 10 client.15630 ms_handle_connect on v2:172.21.15.196:6832/3885816565 2024-04-23T10:26:20.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.386+0000 7f9271ffb640 1 -- 172.21.15.97:0/374445263 <== mds.3 v2:172.21.15.196:6832/3885816565 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55e47e97c4e0 con 0x55e47ea96070 2024-04-23T10:26:20.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.386+0000 7f9271ffb640 10 client.15630 handle_command_reply: tid=0 2024-04-23T10:26:20.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.386+0000 7f9293766740 1 client.15630 shutdown 2024-04-23T10:26:20.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.386+0000 7f9293766740 1 -- 172.21.15.97:0/374445263 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e47ea0edb0 msgr2=0x55e47ea35150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:20.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.386+0000 7f9293766740 1 --2- 172.21.15.97:0/374445263 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e47ea0edb0 0x55e47ea35150 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f927c005db0 tx=0x7f927c0054d0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.386+0000 7f9293766740 1 -- 172.21.15.97:0/374445263 shutdown_connections 2024-04-23T10:26:20.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.386+0000 7f9293766740 1 --2- 172.21.15.97:0/374445263 >> [v2:172.21.15.196:6832/3885816565,v1:172.21.15.196:6833/3885816565] conn(0x55e47ea96070 0x55e47ea94d80 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.386+0000 7f9293766740 1 --2- 172.21.15.97:0/374445263 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e47ea19dd0 0x55e47ea2ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.386+0000 7f9293766740 1 --2- 172.21.15.97:0/374445263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e47ea0f760 0x55e47ea2e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.386+0000 7f9271ffb640 0 client.15630 ms_handle_reset on v2:172.21.15.196:6832/3885816565 2024-04-23T10:26:20.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.386+0000 7f9293766740 1 --2- 172.21.15.97:0/374445263 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e47ea0edb0 0x55e47ea35150 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.386+0000 7f9293766740 1 -- 172.21.15.97:0/374445263 >> 172.21.15.97:0/374445263 conn(0x55e47ea0b070 msgr2=0x55e47ea12020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:20.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.386+0000 7f9293766740 1 -- 172.21.15.97:0/374445263 shutdown_connections 2024-04-23T10:26:20.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.386+0000 7f9293766740 1 -- 172.21.15.97:0/374445263 wait complete. 2024-04-23T10:26:20.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.386+0000 7f9293766740 20 client.15630 trim_cache size 0 max 16384 2024-04-23T10:26:20.398 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-23T10:26:20.398 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "9b22af88-0159-11ef-bc93-c7b262605968", 2024-04-23T10:26:20.398 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 3, 2024-04-23T10:26:20.398 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 24403, 2024-04-23T10:26:20.398 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-04-23T10:26:20.398 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-04-23T10:26:20.398 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-04-23T10:26:20.398 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 129.43398945000001, 2024-04-23T10:26:20.398 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 26, 2024-04-23T10:26:20.398 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 89, 2024-04-23T10:26:20.398 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 88, 2024-04-23T10:26:20.398 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 170.94851609200001 2024-04-23T10:26:20.399 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-23T10:26:20.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.387+0000 7f929096f640 1 -- 172.21.15.97:0/509991330 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f92780a3ec0 msgr2=0x7f92780a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:20.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.387+0000 7f929096f640 1 --2- 172.21.15.97:0/509991330 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f92780a3ec0 0x7f92780a6380 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f928402ade0 tx=0x7f9284005fc0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.387+0000 7f929096f640 1 -- 172.21.15.97:0/509991330 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f928c1016d0 msgr2=0x7f928c1a0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:20.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.387+0000 7f929096f640 1 --2- 172.21.15.97:0/509991330 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f928c1016d0 0x7f928c1a0bf0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f927400d770 tx=0x7f927400dc30 comp rx=0 tx=0).stop 2024-04-23T10:26:20.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.387+0000 7f929096f640 1 -- 172.21.15.97:0/509991330 shutdown_connections 2024-04-23T10:26:20.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.387+0000 7f929096f640 1 --2- 172.21.15.97:0/509991330 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f92780a3ec0 0x7f92780a6380 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.387+0000 7f929096f640 1 --2- 172.21.15.97:0/509991330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f928c10e960 0x7f928c19a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.387+0000 7f929096f640 1 --2- 172.21.15.97:0/509991330 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f928c1016d0 0x7f928c1a0bf0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.387+0000 7f929096f640 1 --2- 172.21.15.97:0/509991330 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f928c100db0 0x7f928c1a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.387+0000 7f929096f640 1 -- 172.21.15.97:0/509991330 >> 172.21.15.97:0/509991330 conn(0x7f928c077f80 msgr2=0x7f928c10fed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:20.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.387+0000 7f929096f640 1 -- 172.21.15.97:0/509991330 shutdown_connections 2024-04-23T10:26:20.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.387+0000 7f929096f640 1 -- 172.21.15.97:0/509991330 wait complete. 2024-04-23T10:26:20.401 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-23T10:26:20.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:19 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/3840928823' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T10:26:20.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:19 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/3785077638' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:26:20.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:19 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/582176515' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T10:26:20.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.500+0000 7f8eb80ef640 1 -- 172.21.15.97:0/382116105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8eb0106200 msgr2=0x7f8eb010ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:20.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.500+0000 7f8eb80ef640 1 --2- 172.21.15.97:0/382116105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8eb0106200 0x7f8eb010ddd0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8009ca0 tx=0x7f8ea802a8c0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.501+0000 7f8eb80ef640 1 -- 172.21.15.97:0/382116105 shutdown_connections 2024-04-23T10:26:20.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.501+0000 7f8eb80ef640 1 --2- 172.21.15.97:0/382116105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8eb0106200 0x7f8eb010ddd0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.501+0000 7f8eb80ef640 1 --2- 172.21.15.97:0/382116105 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8eb0101ae0 0x7f8eb0105b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.501+0000 7f8eb80ef640 1 --2- 172.21.15.97:0/382116105 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8eb0101130 0x7f8eb0101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.501+0000 7f8eb80ef640 1 -- 172.21.15.97:0/382116105 >> 172.21.15.97:0/382116105 conn(0x7f8eb0077f80 msgr2=0x7f8eb0078390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:20.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.501+0000 7f8eb80ef640 1 -- 172.21.15.97:0/382116105 shutdown_connections 2024-04-23T10:26:20.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.501+0000 7f8eb80ef640 1 -- 172.21.15.97:0/382116105 wait complete. 2024-04-23T10:26:20.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.501+0000 7f8eb80ef640 1 Processor -- start 2024-04-23T10:26:20.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.501+0000 7f8eb80ef640 1 -- start start 2024-04-23T10:26:20.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.501+0000 7f8eb80ef640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8eb0101130 0x7f8eb0199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.501+0000 7f8eb80ef640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8eb0106200 0x7f8eb0199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.501+0000 7f8eb80ef640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8eb019a040 0x7f8eb01a45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.501+0000 7f8eb80ef640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb0111170 con 0x7f8eb0106200 2024-04-23T10:26:20.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.501+0000 7f8eb80ef640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb0110ff0 con 0x7f8eb0101130 2024-04-23T10:26:20.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.501+0000 7f8eb80ef640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb01112f0 con 0x7f8eb019a040 2024-04-23T10:26:20.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.502+0000 7f8eb568a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8eb0106200 0x7f8eb0199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.502+0000 7f8eb568a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8eb0106200 0x7f8eb0199ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47240/0 (socket says 172.21.15.97:47240) 2024-04-23T10:26:20.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.502+0000 7f8eb568a640 1 -- 172.21.15.97:0/1796677694 learned_addr learned my addr 172.21.15.97:0/1796677694 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:20.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.502+0000 7f8eb668c640 1 --2- 172.21.15.97:0/1796677694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8eb019a040 0x7f8eb01a45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.502+0000 7f8eb568a640 1 -- 172.21.15.97:0/1796677694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8eb019a040 msgr2=0x7f8eb01a45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:20.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.502+0000 7f8eb568a640 1 --2- 172.21.15.97:0/1796677694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8eb019a040 0x7f8eb01a45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.502+0000 7f8eb5e8b640 1 --2- 172.21.15.97:0/1796677694 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8eb0101130 0x7f8eb0199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.502+0000 7f8eb568a640 1 -- 172.21.15.97:0/1796677694 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8eb0101130 msgr2=0x7f8eb0199570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:20.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.502+0000 7f8eb568a640 1 --2- 172.21.15.97:0/1796677694 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8eb0101130 0x7f8eb0199570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.502+0000 7f8eb568a640 1 -- 172.21.15.97:0/1796677694 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eb01a4dc0 con 0x7f8eb0106200 2024-04-23T10:26:20.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.502+0000 7f8eb5e8b640 1 --2- 172.21.15.97:0/1796677694 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8eb0101130 0x7f8eb0199570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:26:20.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.502+0000 7f8eb568a640 1 --2- 172.21.15.97:0/1796677694 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8eb0106200 0x7f8eb0199ab0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f8eac00c970 tx=0x7f8eac00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:20.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.502+0000 7f8ea2ffd640 1 -- 172.21.15.97:0/1796677694 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eac014070 con 0x7f8eb0106200 2024-04-23T10:26:20.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.502+0000 7f8eb80ef640 1 -- 172.21.15.97:0/1796677694 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8eb01a50b0 con 0x7f8eb0106200 2024-04-23T10:26:20.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.502+0000 7f8ea2ffd640 1 -- 172.21.15.97:0/1796677694 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8eac004440 con 0x7f8eb0106200 2024-04-23T10:26:20.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.502+0000 7f8ea2ffd640 1 -- 172.21.15.97:0/1796677694 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eac003e50 con 0x7f8eb0106200 2024-04-23T10:26:20.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.502+0000 7f8eb80ef640 1 -- 172.21.15.97:0/1796677694 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8eb01a5650 con 0x7f8eb0106200 2024-04-23T10:26:20.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.504+0000 7f8ea2ffd640 1 -- 172.21.15.97:0/1796677694 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f8eac017070 con 0x7f8eb0106200 2024-04-23T10:26:20.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.505+0000 7f8ebae08740 1 Processor -- start 2024-04-23T10:26:20.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.505+0000 7f8ebae08740 1 -- start start 2024-04-23T10:26:20.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.505+0000 7f8ebae08740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648f6c09db0 0x5648f6c0a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.505+0000 7f8ebae08740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648f6c0a760 0x5648f6c14800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.505+0000 7f8ebae08740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648f6c14dd0 0x5648f6c17220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.505+0000 7f8ebae08740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5648f6b6fc50 con 0x5648f6c14dd0 2024-04-23T10:26:20.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.505+0000 7f8ebae08740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5648f6b0c830 con 0x5648f6c09db0 2024-04-23T10:26:20.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.505+0000 7f8ebae08740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5648f6b6bc10 con 0x5648f6c0a760 2024-04-23T10:26:20.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.505+0000 7f8ea2ffd640 1 --2- 172.21.15.97:0/1796677694 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f8e9c0a3fe0 0x7f8e9c0a64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.505+0000 7f8eb568a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648f6c14dd0 0x5648f6c17220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.505+0000 7f8eb568a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648f6c14dd0 0x5648f6c17220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47248/0 (socket says 172.21.15.97:47248) 2024-04-23T10:26:20.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.505+0000 7f8eb568a640 1 -- 172.21.15.97:0/853071119 learned_addr learned my addr 172.21.15.97:0/853071119 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:20.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.505+0000 7f8eb668c640 1 --2- 172.21.15.97:0/1796677694 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f8e9c0a3fe0 0x7f8e9c0a64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.505+0000 7f8eb668c640 1 --2- 172.21.15.97:0/853071119 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648f6c09db0 0x5648f6c0a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.513 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.505+0000 7f8ea2ffd640 1 -- 172.21.15.97:0/1796677694 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f8eac0d0910 con 0x7f8eb0106200 2024-04-23T10:26:20.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.506+0000 7f8eb5e8b640 1 --2- 172.21.15.97:0/853071119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648f6c0a760 0x5648f6c14800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.506+0000 7f8eb568a640 1 -- 172.21.15.97:0/853071119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648f6c0a760 msgr2=0x5648f6c14800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:20.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.506+0000 7f8eb568a640 1 --2- 172.21.15.97:0/853071119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648f6c0a760 0x5648f6c14800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.506+0000 7f8eb568a640 1 -- 172.21.15.97:0/853071119 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648f6c09db0 msgr2=0x5648f6c0a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:20.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.506+0000 7f8eb568a640 1 --2- 172.21.15.97:0/853071119 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648f6c09db0 0x5648f6c0a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.506+0000 7f8eb568a640 1 -- 172.21.15.97:0/853071119 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648f68822d0 con 0x5648f6c14dd0 2024-04-23T10:26:20.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.506+0000 7f8eb668c640 1 --2- 172.21.15.97:0/1796677694 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f8e9c0a3fe0 0x7f8e9c0a64a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8005cc0 tx=0x7f8ea8032ca0 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:20.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.506+0000 7f8eb568a640 1 --2- 172.21.15.97:0/853071119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648f6c14dd0 0x5648f6c17220 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f8eac0a4df0 tx=0x7f8eac007ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b13ab128c3a1f3d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:20.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.506+0000 7f8e9bfff640 1 -- 172.21.15.97:0/853071119 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eac09ac70 con 0x5648f6c14dd0 2024-04-23T10:26:20.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.506+0000 7f8e9bfff640 1 -- 172.21.15.97:0/853071119 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8eac09cdb0 con 0x5648f6c14dd0 2024-04-23T10:26:20.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.506+0000 7f8e9bfff640 1 -- 172.21.15.97:0/853071119 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eac0870b0 con 0x5648f6c14dd0 2024-04-23T10:26:20.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.507+0000 7f8ebae08740 1 -- 172.21.15.97:0/853071119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648f6c14dd0 msgr2=0x5648f6c17220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:20.514 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.507+0000 7f8ebae08740 1 --2- 172.21.15.97:0/853071119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648f6c14dd0 0x5648f6c17220 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f8eac0a4df0 tx=0x7f8eac007ec0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.507+0000 7f8ebae08740 1 -- 172.21.15.97:0/853071119 shutdown_connections 2024-04-23T10:26:20.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.507+0000 7f8ebae08740 1 --2- 172.21.15.97:0/853071119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648f6c14dd0 0x5648f6c17220 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.507+0000 7f8ebae08740 1 --2- 172.21.15.97:0/853071119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648f6c0a760 0x5648f6c14800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.507+0000 7f8ebae08740 1 --2- 172.21.15.97:0/853071119 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648f6c09db0 0x5648f6c0a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.507+0000 7f8ebae08740 1 -- 172.21.15.97:0/853071119 >> 172.21.15.97:0/853071119 conn(0x5648f6c06070 msgr2=0x5648f6c06450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:20.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.507+0000 7f8ebae08740 1 -- 172.21.15.97:0/853071119 shutdown_connections 2024-04-23T10:26:20.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.507+0000 7f8ebae08740 1 -- 172.21.15.97:0/853071119 wait complete. 2024-04-23T10:26:20.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.507+0000 7f8ebae08740 1 -- start start 2024-04-23T10:26:20.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.507+0000 7f8ebae08740 1 Processor -- start 2024-04-23T10:26:20.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.508+0000 7f8ebae08740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648f6c09db0 0x5648f6c11d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.508+0000 7f8ebae08740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648f6c0a760 0x5648f6c12240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.508+0000 7f8ebae08740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648f6c14dd0 0x5648f6c12780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.508+0000 7f8ebae08740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5648f6c17ce0 con 0x5648f6c09db0 2024-04-23T10:26:20.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.508+0000 7f8ebae08740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5648f6b0a470 con 0x5648f6c14dd0 2024-04-23T10:26:20.515 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.508+0000 7f8ebae08740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5648f6c17e60 con 0x5648f6c0a760 2024-04-23T10:26:20.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.508+0000 7f8ebae08740 10 client.0 _ll_register_callbacks cb 0x5648f6b2ff30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:26:20.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.508+0000 7f8eb5e8b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648f6c09db0 0x5648f6c11d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.508+0000 7f8eb5e8b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648f6c09db0 0x5648f6c11d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47250/0 (socket says 172.21.15.97:47250) 2024-04-23T10:26:20.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.508+0000 7f8eb5e8b640 1 -- 172.21.15.97:0/367690677 learned_addr learned my addr 172.21.15.97:0/367690677 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:20.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.508+0000 7f8eb568a640 1 --2- 172.21.15.97:0/367690677 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648f6c0a760 0x5648f6c12240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.508+0000 7f8eb668c640 1 --2- 172.21.15.97:0/367690677 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648f6c14dd0 0x5648f6c12780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.508+0000 7f8eb5e8b640 1 -- 172.21.15.97:0/367690677 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648f6c0a760 msgr2=0x5648f6c12240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:20.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.508+0000 7f8eb5e8b640 1 --2- 172.21.15.97:0/367690677 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648f6c0a760 0x5648f6c12240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.508+0000 7f8eb5e8b640 1 -- 172.21.15.97:0/367690677 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648f6c14dd0 msgr2=0x5648f6c12780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:20.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.508+0000 7f8eb5e8b640 1 --2- 172.21.15.97:0/367690677 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648f6c14dd0 0x5648f6c12780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.508+0000 7f8eb5e8b640 1 -- 172.21.15.97:0/367690677 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648f694c6b0 con 0x5648f6c09db0 2024-04-23T10:26:20.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.508+0000 7f8eb668c640 1 --2- 172.21.15.97:0/367690677 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648f6c14dd0 0x5648f6c12780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:26:20.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.508+0000 7f8eb5e8b640 1 --2- 172.21.15.97:0/367690677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648f6c09db0 0x5648f6c11d00 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f8ea400f6f0 tx=0x7f8ea400faf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb3a726287704536 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:20.516 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.508+0000 7f8e997fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:26:20.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.508+0000 7f8e997fa640 1 -- 172.21.15.97:0/367690677 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea4016070 con 0x5648f6c09db0 2024-04-23T10:26:20.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.508+0000 7f8e997fa640 1 -- 172.21.15.97:0/367690677 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f8ea400ba60 con 0x5648f6c09db0 2024-04-23T10:26:20.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.508+0000 7f8ebae08740 1 -- 172.21.15.97:0/367690677 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8eac09cdb0 con 0x5648f6c09db0 2024-04-23T10:26:20.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.508+0000 7f8e997fa640 1 -- 172.21.15.97:0/367690677 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8ea400bc60 con 0x5648f6c09db0 2024-04-23T10:26:20.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.508+0000 7f8ebae08740 10 client.15642 fetch_fsmap learned FSMap version 26 2024-04-23T10:26:20.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.508+0000 7f8ebae08740 1 -- 172.21.15.97:0/367690677 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5648f6c24010 con 0x5648f6c09db0 2024-04-23T10:26:20.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.509+0000 7f8e997fa640 1 -- 172.21.15.97:0/367690677 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8eac09cdb0 con 0x5648f6c09db0 2024-04-23T10:26:20.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.509+0000 7f8e997fa640 1 -- 172.21.15.97:0/367690677 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8ea401a400 con 0x5648f6c09db0 2024-04-23T10:26:20.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.509+0000 7f8ebae08740 10 client.15642 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:26:20.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.509+0000 7f8ebae08740 10 client.15642 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-23T10:26:20.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.509+0000 7f8ebae08740 20 client.15642 populate_metadata read hostname 'smithi097' 2024-04-23T10:26:20.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.509+0000 7f8ebae08740 1 --2- 172.21.15.97:0/367690677 >> [v2:172.21.15.196:6836/3094946502,v1:172.21.15.196:6837/3094946502] conn(0x5648f6c26690 0x5648f6c27b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.509+0000 7f8ebae08740 4 client.15642 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:26:20.517 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.509+0000 7f8ebae08740 1 -- 172.21.15.97:0/367690677 --> [v2:172.21.15.196:6836/3094946502,v1:172.21.15.196:6837/3094946502] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5648f6b7d670 con 0x5648f6c26690 2024-04-23T10:26:20.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.509+0000 7f8eb568a640 1 --2- 172.21.15.97:0/367690677 >> [v2:172.21.15.196:6836/3094946502,v1:172.21.15.196:6837/3094946502] conn(0x5648f6c26690 0x5648f6c27b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.509+0000 7f8e997fa640 10 client.15642 ms_handle_connect on v2:172.21.15.196:6836/3094946502 2024-04-23T10:26:20.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.509+0000 7f8eb568a640 1 --2- 172.21.15.97:0/367690677 >> [v2:172.21.15.196:6836/3094946502,v1:172.21.15.196:6837/3094946502] conn(0x5648f6c26690 0x5648f6c27b00 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ea9711901dab8664 server_cookie=eb86ba90066d615a in_seq=0 out_seq=0 2024-04-23T10:26:20.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.510+0000 7f8e997fa640 1 -- 172.21.15.97:0/367690677 <== mds.2 v2:172.21.15.196:6836/3094946502 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5648f6b7d670 con 0x5648f6c26690 2024-04-23T10:26:20.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.510+0000 7f8e997fa640 10 client.15642 handle_command_reply: tid=0 2024-04-23T10:26:20.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.510+0000 7f8ebae08740 1 client.15642 shutdown 2024-04-23T10:26:20.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.510+0000 7f8ebae08740 1 -- 172.21.15.97:0/367690677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648f6c09db0 msgr2=0x5648f6c11d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:20.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.510+0000 7f8ebae08740 1 --2- 172.21.15.97:0/367690677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648f6c09db0 0x5648f6c11d00 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f8ea400f6f0 tx=0x7f8ea400faf0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.510+0000 7f8ebae08740 1 -- 172.21.15.97:0/367690677 shutdown_connections 2024-04-23T10:26:20.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.510+0000 7f8ebae08740 1 --2- 172.21.15.97:0/367690677 >> [v2:172.21.15.196:6836/3094946502,v1:172.21.15.196:6837/3094946502] conn(0x5648f6c26690 0x5648f6c27b00 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.510+0000 7f8ebae08740 1 --2- 172.21.15.97:0/367690677 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648f6c14dd0 0x5648f6c12780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.510+0000 7f8e997fa640 0 client.15642 ms_handle_reset on v2:172.21.15.196:6836/3094946502 2024-04-23T10:26:20.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.510+0000 7f8ebae08740 1 --2- 172.21.15.97:0/367690677 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648f6c0a760 0x5648f6c12240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.510+0000 7f8ebae08740 1 --2- 172.21.15.97:0/367690677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648f6c09db0 0x5648f6c11d00 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.518 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.510+0000 7f8ebae08740 1 -- 172.21.15.97:0/367690677 >> 172.21.15.97:0/367690677 conn(0x5648f6c06070 msgr2=0x5648f6c0bee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:20.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.510+0000 7f8ebae08740 1 -- 172.21.15.97:0/367690677 shutdown_connections 2024-04-23T10:26:20.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.510+0000 7f8ebae08740 1 -- 172.21.15.97:0/367690677 wait complete. 2024-04-23T10:26:20.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.511+0000 7f8ebae08740 20 client.15642 trim_cache size 0 max 16384 2024-04-23T10:26:20.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.528+0000 7f8ebae08740 1 Processor -- start 2024-04-23T10:26:20.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.529+0000 7f8ebae08740 1 -- start start 2024-04-23T10:26:20.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.529+0000 7f8ebae08740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648f6c09db0 0x5648f6c14430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.529+0000 7f8ebae08740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648f6c0a760 0x5648f6c0c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.529+0000 7f8ebae08740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648f6c14dd0 0x5648f6c125a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.529+0000 7f8ebae08740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5648f6c25e40 con 0x5648f6c09db0 2024-04-23T10:26:20.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.529+0000 7f8ebae08740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5648f6c26510 con 0x5648f6c0a760 2024-04-23T10:26:20.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.529+0000 7f8ebae08740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5648f6c26be0 con 0x5648f6c14dd0 2024-04-23T10:26:20.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.529+0000 7f8eb5e8b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648f6c09db0 0x5648f6c14430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.529+0000 7f8eb5e8b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648f6c09db0 0x5648f6c14430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47258/0 (socket says 172.21.15.97:47258) 2024-04-23T10:26:20.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.529+0000 7f8eb5e8b640 1 -- 172.21.15.97:0/4094481821 learned_addr learned my addr 172.21.15.97:0/4094481821 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:20.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.529+0000 7f8eb5e8b640 1 -- 172.21.15.97:0/4094481821 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648f6c14dd0 msgr2=0x5648f6c125a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:20.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.529+0000 7f8eb5e8b640 1 --2- 172.21.15.97:0/4094481821 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648f6c14dd0 0x5648f6c125a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.529+0000 7f8eb5e8b640 1 -- 172.21.15.97:0/4094481821 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648f6c0a760 msgr2=0x5648f6c0c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:20.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.529+0000 7f8eb5e8b640 1 --2- 172.21.15.97:0/4094481821 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648f6c0a760 0x5648f6c0c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.529+0000 7f8eb5e8b640 1 -- 172.21.15.97:0/4094481821 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648f6c17600 con 0x5648f6c09db0 2024-04-23T10:26:20.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.529+0000 7f8eb5e8b640 1 --2- 172.21.15.97:0/4094481821 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648f6c09db0 0x5648f6c14430 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f8ea401a820 tx=0x7f8ea400d500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=987a78d73f3f9cbe server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:20.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.529+0000 7f8e9b7fe640 1 -- 172.21.15.97:0/4094481821 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea4030070 con 0x5648f6c09db0 2024-04-23T10:26:20.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.529+0000 7f8e9b7fe640 1 -- 172.21.15.97:0/4094481821 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ea401ac90 con 0x5648f6c09db0 2024-04-23T10:26:20.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.529+0000 7f8e9b7fe640 1 -- 172.21.15.97:0/4094481821 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea4016040 con 0x5648f6c09db0 2024-04-23T10:26:20.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.530+0000 7f8ebae08740 1 -- 172.21.15.97:0/4094481821 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648f6c09db0 msgr2=0x5648f6c14430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:20.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.530+0000 7f8ebae08740 1 --2- 172.21.15.97:0/4094481821 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648f6c09db0 0x5648f6c14430 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f8ea401a820 tx=0x7f8ea400d500 comp rx=0 tx=0).stop 2024-04-23T10:26:20.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.530+0000 7f8ebae08740 1 -- 172.21.15.97:0/4094481821 shutdown_connections 2024-04-23T10:26:20.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.530+0000 7f8ebae08740 1 --2- 172.21.15.97:0/4094481821 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648f6c14dd0 0x5648f6c125a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.530+0000 7f8ebae08740 1 --2- 172.21.15.97:0/4094481821 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648f6c0a760 0x5648f6c0c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.530+0000 7f8ebae08740 1 --2- 172.21.15.97:0/4094481821 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648f6c09db0 0x5648f6c14430 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.534 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.530+0000 7f8ebae08740 1 -- 172.21.15.97:0/4094481821 >> 172.21.15.97:0/4094481821 conn(0x5648f6c06070 msgr2=0x5648f6c0cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:20.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.530+0000 7f8ebae08740 1 -- 172.21.15.97:0/4094481821 shutdown_connections 2024-04-23T10:26:20.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.530+0000 7f8ebae08740 1 -- 172.21.15.97:0/4094481821 wait complete. 2024-04-23T10:26:20.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.530+0000 7f8ebae08740 1 -- start start 2024-04-23T10:26:20.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.530+0000 7f8ebae08740 1 Processor -- start 2024-04-23T10:26:20.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.531+0000 7f8ebae08740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648f6c09db0 0x5648f6c2feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.531+0000 7f8ebae08740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648f6c0a760 0x5648f6c29300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.531+0000 7f8ebae08740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648f6c14dd0 0x5648f6c29840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.531+0000 7f8ebae08740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5648f6c27980 con 0x5648f6c14dd0 2024-04-23T10:26:20.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.531+0000 7f8ebae08740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5648f6c272b0 con 0x5648f6c09db0 2024-04-23T10:26:20.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.531+0000 7f8ebae08740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5648f6c28050 con 0x5648f6c0a760 2024-04-23T10:26:20.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.531+0000 7f8ebae08740 10 client.0 _ll_register_callbacks cb 0x7f8ea4005f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:26:20.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.531+0000 7f8eb668c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648f6c14dd0 0x5648f6c29840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.531+0000 7f8eb668c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648f6c14dd0 0x5648f6c29840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47270/0 (socket says 172.21.15.97:47270) 2024-04-23T10:26:20.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.531+0000 7f8eb668c640 1 -- 172.21.15.97:0/572847332 learned_addr learned my addr 172.21.15.97:0/572847332 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:20.535 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.531+0000 7f8eb5e8b640 1 --2- 172.21.15.97:0/572847332 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648f6c09db0 0x5648f6c2feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.531+0000 7f8eb568a640 1 --2- 172.21.15.97:0/572847332 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648f6c0a760 0x5648f6c29300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.531+0000 7f8eb668c640 1 -- 172.21.15.97:0/572847332 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648f6c0a760 msgr2=0x5648f6c29300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:20.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.531+0000 7f8eb668c640 1 --2- 172.21.15.97:0/572847332 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648f6c0a760 0x5648f6c29300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.531+0000 7f8eb668c640 1 -- 172.21.15.97:0/572847332 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648f6c09db0 msgr2=0x5648f6c2feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:20.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.531+0000 7f8eb668c640 1 --2- 172.21.15.97:0/572847332 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648f6c09db0 0x5648f6c2feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.531+0000 7f8eb668c640 1 -- 172.21.15.97:0/572847332 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648f6c29db0 con 0x5648f6c14dd0 2024-04-23T10:26:20.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.531+0000 7f8eb668c640 1 --2- 172.21.15.97:0/572847332 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648f6c14dd0 0x5648f6c29840 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8009ca0 tx=0x7f8ea80095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9380eb7d9bc7ad9f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:20.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.531+0000 7f8e997fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:26:20.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.531+0000 7f8e997fa640 1 -- 172.21.15.97:0/572847332 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea8003b70 con 0x5648f6c14dd0 2024-04-23T10:26:20.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.531+0000 7f8e997fa640 1 -- 172.21.15.97:0/572847332 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f8ea8003d10 con 0x5648f6c14dd0 2024-04-23T10:26:20.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.531+0000 7f8ebae08740 1 -- 172.21.15.97:0/572847332 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ea401ac90 con 0x5648f6c14dd0 2024-04-23T10:26:20.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.532+0000 7f8e997fa640 1 -- 172.21.15.97:0/572847332 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8ea8004040 con 0x5648f6c14dd0 2024-04-23T10:26:20.536 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.532+0000 7f8ebae08740 10 client.15648 fetch_fsmap learned FSMap version 26 2024-04-23T10:26:20.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.532+0000 7f8ebae08740 1 -- 172.21.15.97:0/572847332 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5648f6c2e7b0 con 0x5648f6c14dd0 2024-04-23T10:26:20.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.532+0000 7f8e997fa640 1 -- 172.21.15.97:0/572847332 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ea401ac90 con 0x5648f6c14dd0 2024-04-23T10:26:20.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.532+0000 7f8e997fa640 1 -- 172.21.15.97:0/572847332 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8ea802fa00 con 0x5648f6c14dd0 2024-04-23T10:26:20.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.532+0000 7f8ebae08740 10 client.15648 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:26:20.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.532+0000 7f8ebae08740 10 client.15648 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-23T10:26:20.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.532+0000 7f8ebae08740 20 client.15648 populate_metadata read hostname 'smithi097' 2024-04-23T10:26:20.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.532+0000 7f8ebae08740 1 --2- 172.21.15.97:0/572847332 >> [v2:172.21.15.196:6836/3094946502,v1:172.21.15.196:6837/3094946502] conn(0x5648f6c26690 0x5648f6c8fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.532+0000 7f8ebae08740 4 client.15648 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T10:26:20.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.532+0000 7f8ebae08740 1 -- 172.21.15.97:0/572847332 --> [v2:172.21.15.196:6836/3094946502,v1:172.21.15.196:6837/3094946502] -- command(tid 0: {"prefix": "status"}) -- 0x5648f6b774e0 con 0x5648f6c26690 2024-04-23T10:26:20.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.532+0000 7f8eb5e8b640 1 --2- 172.21.15.97:0/572847332 >> [v2:172.21.15.196:6836/3094946502,v1:172.21.15.196:6837/3094946502] conn(0x5648f6c26690 0x5648f6c8fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.533+0000 7f8e997fa640 10 client.15648 ms_handle_connect on v2:172.21.15.196:6836/3094946502 2024-04-23T10:26:20.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.533+0000 7f8eb5e8b640 1 --2- 172.21.15.97:0/572847332 >> [v2:172.21.15.196:6836/3094946502,v1:172.21.15.196:6837/3094946502] conn(0x5648f6c26690 0x5648f6c8fcc0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=951ddddfe32e2740 server_cookie=cd661f4fa18e1ea8 in_seq=0 out_seq=0 2024-04-23T10:26:20.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.533+0000 7f8e997fa640 1 -- 172.21.15.97:0/572847332 <== mds.2 v2:172.21.15.196:6836/3094946502 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5648f6b774e0 con 0x5648f6c26690 2024-04-23T10:26:20.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.533+0000 7f8e997fa640 10 client.15648 handle_command_reply: tid=0 2024-04-23T10:26:20.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.533+0000 7f8ebae08740 1 client.15648 shutdown 2024-04-23T10:26:20.537 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.533+0000 7f8ebae08740 1 -- 172.21.15.97:0/572847332 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648f6c14dd0 msgr2=0x5648f6c29840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:20.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.533+0000 7f8ebae08740 1 --2- 172.21.15.97:0/572847332 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648f6c14dd0 0x5648f6c29840 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8009ca0 tx=0x7f8ea80095c0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.533+0000 7f8ebae08740 1 -- 172.21.15.97:0/572847332 shutdown_connections 2024-04-23T10:26:20.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.533+0000 7f8ebae08740 1 --2- 172.21.15.97:0/572847332 >> [v2:172.21.15.196:6836/3094946502,v1:172.21.15.196:6837/3094946502] conn(0x5648f6c26690 0x5648f6c8fcc0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.533+0000 7f8ebae08740 1 --2- 172.21.15.97:0/572847332 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5648f6c14dd0 0x5648f6c29840 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.533+0000 7f8e997fa640 0 client.15648 ms_handle_reset on v2:172.21.15.196:6836/3094946502 2024-04-23T10:26:20.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.533+0000 7f8ebae08740 1 --2- 172.21.15.97:0/572847332 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5648f6c0a760 0x5648f6c29300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.533+0000 7f8ebae08740 1 --2- 172.21.15.97:0/572847332 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648f6c09db0 0x5648f6c2feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.533+0000 7f8ebae08740 1 -- 172.21.15.97:0/572847332 >> 172.21.15.97:0/572847332 conn(0x5648f6c06070 msgr2=0x5648f6c10590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:20.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.533+0000 7f8ebae08740 1 -- 172.21.15.97:0/572847332 shutdown_connections 2024-04-23T10:26:20.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.533+0000 7f8ebae08740 1 -- 172.21.15.97:0/572847332 wait complete. 2024-04-23T10:26:20.538 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.533+0000 7f8ebae08740 20 client.15648 trim_cache size 0 max 16384 2024-04-23T10:26:20.538 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-23T10:26:20.539 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "9b22af88-0159-11ef-bc93-c7b262605968", 2024-04-23T10:26:20.539 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 2, 2024-04-23T10:26:20.539 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 24469, 2024-04-23T10:26:20.539 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-04-23T10:26:20.539 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-04-23T10:26:20.539 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-04-23T10:26:20.539 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 129.95977519600001, 2024-04-23T10:26:20.539 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 26, 2024-04-23T10:26:20.539 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 89, 2024-04-23T10:26:20.539 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 88, 2024-04-23T10:26:20.539 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 153.83551911500001 2024-04-23T10:26:20.539 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-23T10:26:20.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.533+0000 7f8eb80ef640 1 -- 172.21.15.97:0/1796677694 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f8e9c0a3fe0 msgr2=0x7f8e9c0a64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:20.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.534+0000 7f8eb80ef640 1 --2- 172.21.15.97:0/1796677694 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f8e9c0a3fe0 0x7f8e9c0a64a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8005cc0 tx=0x7f8ea8032ca0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.534+0000 7f8eb80ef640 1 -- 172.21.15.97:0/1796677694 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8eb0106200 msgr2=0x7f8eb0199ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:20.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.534+0000 7f8eb80ef640 1 --2- 172.21.15.97:0/1796677694 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8eb0106200 0x7f8eb0199ab0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f8eac00c970 tx=0x7f8eac00ce30 comp rx=0 tx=0).stop 2024-04-23T10:26:20.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.534+0000 7f8eb80ef640 1 -- 172.21.15.97:0/1796677694 shutdown_connections 2024-04-23T10:26:20.540 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.534+0000 7f8eb80ef640 1 --2- 172.21.15.97:0/1796677694 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f8e9c0a3fe0 0x7f8e9c0a64a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.534+0000 7f8eb80ef640 1 --2- 172.21.15.97:0/1796677694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8eb019a040 0x7f8eb01a45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.534+0000 7f8eb80ef640 1 --2- 172.21.15.97:0/1796677694 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8eb0106200 0x7f8eb0199ab0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.534+0000 7f8eb80ef640 1 --2- 172.21.15.97:0/1796677694 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8eb0101130 0x7f8eb0199570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.534+0000 7f8eb80ef640 1 -- 172.21.15.97:0/1796677694 >> 172.21.15.97:0/1796677694 conn(0x7f8eb0077f80 msgr2=0x7f8eb00fee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:20.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.534+0000 7f8eb80ef640 1 -- 172.21.15.97:0/1796677694 shutdown_connections 2024-04-23T10:26:20.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.534+0000 7f8eb80ef640 1 -- 172.21.15.97:0/1796677694 wait complete. 2024-04-23T10:26:20.543 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-23T10:26:20.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.644+0000 7f5cb7ccf640 1 -- 172.21.15.97:0/3494353862 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cb0101a30 msgr2=0x7f5cb0105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:20.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.644+0000 7f5cb7ccf640 1 --2- 172.21.15.97:0/3494353862 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cb0101a30 0x7f5cb0105a70 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f5cac009900 tx=0x7f5cac02a410 comp rx=0 tx=0).stop 2024-04-23T10:26:20.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.644+0000 7f5cb7ccf640 1 -- 172.21.15.97:0/3494353862 shutdown_connections 2024-04-23T10:26:20.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.644+0000 7f5cb7ccf640 1 --2- 172.21.15.97:0/3494353862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5cb0106150 0x7f5cb010dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.644+0000 7f5cb7ccf640 1 --2- 172.21.15.97:0/3494353862 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cb0101a30 0x7f5cb0105a70 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.644+0000 7f5cb7ccf640 1 --2- 172.21.15.97:0/3494353862 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5cb0101080 0x7f5cb0101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.644+0000 7f5cb7ccf640 1 -- 172.21.15.97:0/3494353862 >> 172.21.15.97:0/3494353862 conn(0x7f5cb00771d0 msgr2=0x7f5cb00775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:20.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.644+0000 7f5cb7ccf640 1 -- 172.21.15.97:0/3494353862 shutdown_connections 2024-04-23T10:26:20.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.644+0000 7f5cb7ccf640 1 -- 172.21.15.97:0/3494353862 wait complete. 2024-04-23T10:26:20.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.645+0000 7f5cb7ccf640 1 Processor -- start 2024-04-23T10:26:20.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.645+0000 7f5cb7ccf640 1 -- start start 2024-04-23T10:26:20.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.645+0000 7f5cb7ccf640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cb0101080 0x7f5cb0197630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.645+0000 7f5cb7ccf640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5cb0106150 0x7f5cb0197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.645+0000 7f5cb7ccf640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5cb019bef0 0x7f5cb019c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.645+0000 7f5cb7ccf640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb0111070 con 0x7f5cb0101080 2024-04-23T10:26:20.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.645+0000 7f5cb7ccf640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb0110ef0 con 0x7f5cb019bef0 2024-04-23T10:26:20.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.645+0000 7f5cb7ccf640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb01111f0 con 0x7f5cb0106150 2024-04-23T10:26:20.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.645+0000 7f5cb5a6b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cb0101080 0x7f5cb0197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.645+0000 7f5cb5a6b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cb0101080 0x7f5cb0197630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47284/0 (socket says 172.21.15.97:47284) 2024-04-23T10:26:20.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.645+0000 7f5cb5a6b640 1 -- 172.21.15.97:0/437905115 learned_addr learned my addr 172.21.15.97:0/437905115 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:20.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.645+0000 7f5cb626c640 1 --2- 172.21.15.97:0/437905115 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5cb019bef0 0x7f5cb019c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.645+0000 7f5cb5a6b640 1 -- 172.21.15.97:0/437905115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5cb0106150 msgr2=0x7f5cb0197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:20.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.645+0000 7f5cb5a6b640 1 --2- 172.21.15.97:0/437905115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5cb0106150 0x7f5cb0197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.645+0000 7f5cb5a6b640 1 -- 172.21.15.97:0/437905115 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5cb019bef0 msgr2=0x7f5cb019c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:20.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.645+0000 7f5cb5a6b640 1 --2- 172.21.15.97:0/437905115 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5cb019bef0 0x7f5cb019c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.645+0000 7f5cb5a6b640 1 -- 172.21.15.97:0/437905115 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cb019c9e0 con 0x7f5cb0101080 2024-04-23T10:26:20.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.646+0000 7f5cb5a6b640 1 --2- 172.21.15.97:0/437905115 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cb0101080 0x7f5cb0197630 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f5ca800ebc0 tx=0x7f5ca800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:20.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.646+0000 7f5ca2ffd640 1 -- 172.21.15.97:0/437905115 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca8010630 con 0x7f5cb0101080 2024-04-23T10:26:20.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.646+0000 7f5cb7ccf640 1 -- 172.21.15.97:0/437905115 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cb019ccd0 con 0x7f5cb0101080 2024-04-23T10:26:20.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.646+0000 7f5ca2ffd640 1 -- 172.21.15.97:0/437905115 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ca8010c80 con 0x7f5cb0101080 2024-04-23T10:26:20.647 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.646+0000 7f5ca2ffd640 1 -- 172.21.15.97:0/437905115 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca8018b60 con 0x7f5cb0101080 2024-04-23T10:26:20.648 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.646+0000 7f5cb7ccf640 1 -- 172.21.15.97:0/437905115 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cb01a4510 con 0x7f5cb0101080 2024-04-23T10:26:20.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.648+0000 7f5ca2ffd640 1 -- 172.21.15.97:0/437905115 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f5ca8018d00 con 0x7f5cb0101080 2024-04-23T10:26:20.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.648+0000 7f5cba9ed740 1 Processor -- start 2024-04-23T10:26:20.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.649+0000 7f5cba9ed740 1 -- start start 2024-04-23T10:26:20.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.649+0000 7f5ca2ffd640 1 --2- 172.21.15.97:0/437905115 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f5c9c0a3e40 0x7f5c9c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.649+0000 7f5cba9ed740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1e64fcc70 0x55c1e64fd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.649+0000 7f5cba9ed740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1e64fd620 0x55c1e6509e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.649+0000 7f5cba9ed740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c1e650a340 0x55c1e650c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.649+0000 7f5cba9ed740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c1e6460c50 con 0x55c1e64fcc70 2024-04-23T10:26:20.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.649+0000 7f5cba9ed740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c1e63fd830 con 0x55c1e650a340 2024-04-23T10:26:20.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.649+0000 7f5cba9ed740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c1e645cc10 con 0x55c1e64fd620 2024-04-23T10:26:20.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.649+0000 7f5cb626c640 1 --2- 172.21.15.97:0/437905115 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f5c9c0a3e40 0x7f5c9c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.649+0000 7f5ca2ffd640 1 -- 172.21.15.97:0/437905115 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f5ca80d11a0 con 0x7f5cb0101080 2024-04-23T10:26:20.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.649+0000 7f5cb5a6b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1e64fcc70 0x55c1e64fd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.649+0000 7f5cb5a6b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1e64fcc70 0x55c1e64fd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47286/0 (socket says 172.21.15.97:47286) 2024-04-23T10:26:20.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.649+0000 7f5cb5a6b640 1 -- 172.21.15.97:0/1786206501 learned_addr learned my addr 172.21.15.97:0/1786206501 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:20.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.649+0000 7f5cb526a640 1 --2- 172.21.15.97:0/1786206501 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1e64fd620 0x55c1e6509e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.649+0000 7f5cb626c640 1 --2- 172.21.15.97:0/1786206501 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c1e650a340 0x55c1e650c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.649+0000 7f5cb5a6b640 1 -- 172.21.15.97:0/1786206501 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1e64fd620 msgr2=0x55c1e6509e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:20.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.649+0000 7f5cb5a6b640 1 --2- 172.21.15.97:0/1786206501 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1e64fd620 0x55c1e6509e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.649+0000 7f5cb5a6b640 1 -- 172.21.15.97:0/1786206501 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c1e650a340 msgr2=0x55c1e650c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:20.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.649+0000 7f5cb5a6b640 1 --2- 172.21.15.97:0/1786206501 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c1e650a340 0x55c1e650c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.649+0000 7f5cb5a6b640 1 -- 172.21.15.97:0/1786206501 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1e61732d0 con 0x55c1e64fcc70 2024-04-23T10:26:20.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.649+0000 7f5cb626c640 1 --2- 172.21.15.97:0/437905115 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f5c9c0a3e40 0x7f5c9c0a6300 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5ca4005940 tx=0x7f5ca400cf90 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:20.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.650+0000 7f5cb5a6b640 1 --2- 172.21.15.97:0/1786206501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1e64fcc70 0x55c1e64fd050 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f5ca80993a0 tx=0x7f5ca80099b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60959ae9f985ef4f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:20.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.650+0000 7f5c9bfff640 1 -- 172.21.15.97:0/1786206501 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca80b66c0 con 0x55c1e64fcc70 2024-04-23T10:26:20.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.650+0000 7f5c9bfff640 1 -- 172.21.15.97:0/1786206501 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ca8018420 con 0x55c1e64fcc70 2024-04-23T10:26:20.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.650+0000 7f5c9bfff640 1 -- 172.21.15.97:0/1786206501 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca80995b0 con 0x55c1e64fcc70 2024-04-23T10:26:20.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.650+0000 7f5cba9ed740 1 -- 172.21.15.97:0/1786206501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1e64fcc70 msgr2=0x55c1e64fd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:20.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.650+0000 7f5cba9ed740 1 --2- 172.21.15.97:0/1786206501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1e64fcc70 0x55c1e64fd050 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f5ca80993a0 tx=0x7f5ca80099b0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.650+0000 7f5cba9ed740 1 -- 172.21.15.97:0/1786206501 shutdown_connections 2024-04-23T10:26:20.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.650+0000 7f5cba9ed740 1 --2- 172.21.15.97:0/1786206501 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c1e650a340 0x55c1e650c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.650+0000 7f5cba9ed740 1 --2- 172.21.15.97:0/1786206501 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1e64fd620 0x55c1e6509e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.650+0000 7f5cba9ed740 1 --2- 172.21.15.97:0/1786206501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1e64fcc70 0x55c1e64fd050 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.650+0000 7f5cba9ed740 1 -- 172.21.15.97:0/1786206501 >> 172.21.15.97:0/1786206501 conn(0x55c1e64f9280 msgr2=0x55c1e64f9660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:20.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.651+0000 7f5cba9ed740 1 -- 172.21.15.97:0/1786206501 shutdown_connections 2024-04-23T10:26:20.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.651+0000 7f5cba9ed740 1 -- 172.21.15.97:0/1786206501 wait complete. 2024-04-23T10:26:20.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.651+0000 7f5cba9ed740 1 -- start start 2024-04-23T10:26:20.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.651+0000 7f5cba9ed740 1 Processor -- start 2024-04-23T10:26:20.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.651+0000 7f5cba9ed740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1e64fcc70 0x55c1e65073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.651+0000 7f5cba9ed740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c1e64fd620 0x55c1e6507910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.651+0000 7f5cba9ed740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1e650a340 0x55c1e6507e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.651+0000 7f5cba9ed740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c1e650d130 con 0x55c1e650a340 2024-04-23T10:26:20.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.651+0000 7f5cba9ed740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c1e63fb470 con 0x55c1e64fd620 2024-04-23T10:26:20.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.651+0000 7f5cba9ed740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c1e650d2b0 con 0x55c1e64fcc70 2024-04-23T10:26:20.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.651+0000 7f5cba9ed740 10 client.0 _ll_register_callbacks cb 0x55c1e6420f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:26:20.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.651+0000 7f5cb626c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1e650a340 0x55c1e6507e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.651+0000 7f5cb626c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1e650a340 0x55c1e6507e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47294/0 (socket says 172.21.15.97:47294) 2024-04-23T10:26:20.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.651+0000 7f5cb626c640 1 -- 172.21.15.97:0/346466452 learned_addr learned my addr 172.21.15.97:0/346466452 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:20.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.652+0000 7f5cb5a6b640 1 --2- 172.21.15.97:0/346466452 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1e64fcc70 0x55c1e65073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.652+0000 7f5cb526a640 1 --2- 172.21.15.97:0/346466452 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c1e64fd620 0x55c1e6507910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.652+0000 7f5cb626c640 1 -- 172.21.15.97:0/346466452 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1e64fcc70 msgr2=0x55c1e65073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:20.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.652+0000 7f5cb626c640 1 --2- 172.21.15.97:0/346466452 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1e64fcc70 0x55c1e65073d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.652+0000 7f5cb626c640 1 -- 172.21.15.97:0/346466452 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c1e64fd620 msgr2=0x55c1e6507910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:20.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.652+0000 7f5cb626c640 1 --2- 172.21.15.97:0/346466452 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c1e64fd620 0x55c1e6507910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.652+0000 7f5cb626c640 1 -- 172.21.15.97:0/346466452 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1e623d6b0 con 0x55c1e650a340 2024-04-23T10:26:20.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.652+0000 7f5cb526a640 1 --2- 172.21.15.97:0/346466452 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c1e64fd620 0x55c1e6507910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:26:20.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.652+0000 7f5cb626c640 1 --2- 172.21.15.97:0/346466452 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1e650a340 0x55c1e6507e50 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f5ca401bd10 tx=0x7f5ca4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5af5d6d818375d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:20.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.652+0000 7f5c997fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:26:20.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.652+0000 7f5c997fa640 1 -- 172.21.15.97:0/346466452 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca4004020 con 0x55c1e650a340 2024-04-23T10:26:20.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.652+0000 7f5c997fa640 1 -- 172.21.15.97:0/346466452 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f5ca40041c0 con 0x55c1e650a340 2024-04-23T10:26:20.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.652+0000 7f5cba9ed740 1 -- 172.21.15.97:0/346466452 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c1e646e670 con 0x55c1e650a340 2024-04-23T10:26:20.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.652+0000 7f5c997fa640 1 -- 172.21.15.97:0/346466452 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5ca40043c0 con 0x55c1e650a340 2024-04-23T10:26:20.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.652+0000 7f5cba9ed740 10 client.15660 fetch_fsmap learned FSMap version 26 2024-04-23T10:26:20.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.652+0000 7f5cba9ed740 1 -- 172.21.15.97:0/346466452 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c1e6519570 con 0x55c1e650a340 2024-04-23T10:26:20.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.652+0000 7f5c997fa640 1 -- 172.21.15.97:0/346466452 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55c1e646e670 con 0x55c1e650a340 2024-04-23T10:26:20.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.652+0000 7f5c997fa640 1 -- 172.21.15.97:0/346466452 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5ca4004c50 con 0x55c1e650a340 2024-04-23T10:26:20.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.653+0000 7f5cba9ed740 10 client.15660 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:26:20.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.653+0000 7f5cba9ed740 10 client.15660 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-23T10:26:20.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.653+0000 7f5cba9ed740 20 client.15660 populate_metadata read hostname 'smithi097' 2024-04-23T10:26:20.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.653+0000 7f5cba9ed740 1 --2- 172.21.15.97:0/346466452 >> [v2:172.21.15.196:6838/83843556,v1:172.21.15.196:6839/83843556] conn(0x55c1e651bbe0 0x55c1e651d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.653+0000 7f5cba9ed740 4 client.15660 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:26:20.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.653+0000 7f5cba9ed740 1 -- 172.21.15.97:0/346466452 --> [v2:172.21.15.196:6838/83843556,v1:172.21.15.196:6839/83843556] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c1e64684e0 con 0x55c1e651bbe0 2024-04-23T10:26:20.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.653+0000 7f5cb5a6b640 1 --2- 172.21.15.97:0/346466452 >> [v2:172.21.15.196:6838/83843556,v1:172.21.15.196:6839/83843556] conn(0x55c1e651bbe0 0x55c1e651d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.653+0000 7f5cb5a6b640 1 --2- 172.21.15.97:0/346466452 >> [v2:172.21.15.196:6838/83843556,v1:172.21.15.196:6839/83843556] conn(0x55c1e651bbe0 0x55c1e651d050 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=217be65b94b20e98 server_cookie=6543d6eb7c9852b1 in_seq=0 out_seq=0 2024-04-23T10:26:20.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.653+0000 7f5c997fa640 10 client.15660 ms_handle_connect on v2:172.21.15.196:6838/83843556 2024-04-23T10:26:20.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.654+0000 7f5c997fa640 1 -- 172.21.15.97:0/346466452 <== mds.1 v2:172.21.15.196:6838/83843556 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c1e64684e0 con 0x55c1e651bbe0 2024-04-23T10:26:20.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.654+0000 7f5c997fa640 10 client.15660 handle_command_reply: tid=0 2024-04-23T10:26:20.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.654+0000 7f5cba9ed740 1 client.15660 shutdown 2024-04-23T10:26:20.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.654+0000 7f5cba9ed740 1 -- 172.21.15.97:0/346466452 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1e650a340 msgr2=0x55c1e6507e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:20.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.654+0000 7f5cba9ed740 1 --2- 172.21.15.97:0/346466452 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1e650a340 0x55c1e6507e50 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f5ca401bd10 tx=0x7f5ca4004730 comp rx=0 tx=0).stop 2024-04-23T10:26:20.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.654+0000 7f5cba9ed740 1 -- 172.21.15.97:0/346466452 shutdown_connections 2024-04-23T10:26:20.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.654+0000 7f5cba9ed740 1 --2- 172.21.15.97:0/346466452 >> [v2:172.21.15.196:6838/83843556,v1:172.21.15.196:6839/83843556] conn(0x55c1e651bbe0 0x55c1e651d050 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.654+0000 7f5cba9ed740 1 --2- 172.21.15.97:0/346466452 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1e650a340 0x55c1e6507e50 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.654+0000 7f5cba9ed740 1 --2- 172.21.15.97:0/346466452 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c1e64fd620 0x55c1e6507910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.654+0000 7f5cba9ed740 1 --2- 172.21.15.97:0/346466452 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1e64fcc70 0x55c1e65073d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.654+0000 7f5c997fa640 0 client.15660 ms_handle_reset on v2:172.21.15.196:6838/83843556 2024-04-23T10:26:20.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.654+0000 7f5cba9ed740 1 -- 172.21.15.97:0/346466452 >> 172.21.15.97:0/346466452 conn(0x55c1e64f9280 msgr2=0x55c1e64ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:20.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.654+0000 7f5cba9ed740 1 -- 172.21.15.97:0/346466452 shutdown_connections 2024-04-23T10:26:20.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.654+0000 7f5cba9ed740 1 -- 172.21.15.97:0/346466452 wait complete. 2024-04-23T10:26:20.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.655+0000 7f5cba9ed740 20 client.15660 trim_cache size 0 max 16384 2024-04-23T10:26:20.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.673+0000 7f5cba9ed740 1 Processor -- start 2024-04-23T10:26:20.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.673+0000 7f5cba9ed740 1 -- start start 2024-04-23T10:26:20.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.673+0000 7f5cba9ed740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1e64fcc70 0x55c1e6504dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.673+0000 7f5cba9ed740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c1e64fd620 0x55c1e64f9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.673+0000 7f5cba9ed740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1e650a340 0x55c1e64f6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.673+0000 7f5cba9ed740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c1e651b390 con 0x55c1e64fcc70 2024-04-23T10:26:20.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.673+0000 7f5cba9ed740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c1e651ba60 con 0x55c1e64fd620 2024-04-23T10:26:20.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.673+0000 7f5cba9ed740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c1e651c130 con 0x55c1e650a340 2024-04-23T10:26:20.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.673+0000 7f5cb5a6b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1e64fcc70 0x55c1e6504dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.673+0000 7f5cb5a6b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1e64fcc70 0x55c1e6504dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47300/0 (socket says 172.21.15.97:47300) 2024-04-23T10:26:20.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.673+0000 7f5cb5a6b640 1 -- 172.21.15.97:0/3777746262 learned_addr learned my addr 172.21.15.97:0/3777746262 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:20.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.673+0000 7f5cb526a640 1 --2- 172.21.15.97:0/3777746262 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c1e64fd620 0x55c1e64f9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.673+0000 7f5cb626c640 1 --2- 172.21.15.97:0/3777746262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1e650a340 0x55c1e64f6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.673+0000 7f5cb5a6b640 1 -- 172.21.15.97:0/3777746262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1e650a340 msgr2=0x55c1e64f6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:20.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.673+0000 7f5cb5a6b640 1 --2- 172.21.15.97:0/3777746262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1e650a340 0x55c1e64f6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.673+0000 7f5cb5a6b640 1 -- 172.21.15.97:0/3777746262 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c1e64fd620 msgr2=0x55c1e64f9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:20.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.673+0000 7f5cb5a6b640 1 --2- 172.21.15.97:0/3777746262 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c1e64fd620 0x55c1e64f9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.673+0000 7f5cb5a6b640 1 -- 172.21.15.97:0/3777746262 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1e64fdb30 con 0x55c1e64fcc70 2024-04-23T10:26:20.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.673+0000 7f5cb5a6b640 1 --2- 172.21.15.97:0/3777746262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1e64fcc70 0x55c1e6504dd0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f5ca80ce8d0 tx=0x7f5ca809c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce81794df2835edc server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:20.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.673+0000 7f5c9b7fe640 1 -- 172.21.15.97:0/3777746262 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca80b6070 con 0x55c1e64fcc70 2024-04-23T10:26:20.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.674+0000 7f5c9b7fe640 1 -- 172.21.15.97:0/3777746262 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ca809d640 con 0x55c1e64fcc70 2024-04-23T10:26:20.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.674+0000 7f5c9b7fe640 1 -- 172.21.15.97:0/3777746262 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca80ceaa0 con 0x55c1e64fcc70 2024-04-23T10:26:20.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.674+0000 7f5cba9ed740 1 -- 172.21.15.97:0/3777746262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1e64fcc70 msgr2=0x55c1e6504dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:20.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.674+0000 7f5cba9ed740 1 --2- 172.21.15.97:0/3777746262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1e64fcc70 0x55c1e6504dd0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f5ca80ce8d0 tx=0x7f5ca809c9a0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.674+0000 7f5cba9ed740 1 -- 172.21.15.97:0/3777746262 shutdown_connections 2024-04-23T10:26:20.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.674+0000 7f5cba9ed740 1 --2- 172.21.15.97:0/3777746262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1e650a340 0x55c1e64f6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.674+0000 7f5cba9ed740 1 --2- 172.21.15.97:0/3777746262 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c1e64fd620 0x55c1e64f9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.674+0000 7f5cba9ed740 1 --2- 172.21.15.97:0/3777746262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1e64fcc70 0x55c1e6504dd0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.674+0000 7f5cba9ed740 1 -- 172.21.15.97:0/3777746262 >> 172.21.15.97:0/3777746262 conn(0x55c1e64f9280 msgr2=0x55c1e64f8230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:20.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.674+0000 7f5cba9ed740 1 -- 172.21.15.97:0/3777746262 shutdown_connections 2024-04-23T10:26:20.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.674+0000 7f5cba9ed740 1 -- 172.21.15.97:0/3777746262 wait complete. 2024-04-23T10:26:20.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.674+0000 7f5cba9ed740 1 -- start start 2024-04-23T10:26:20.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.674+0000 7f5cba9ed740 1 Processor -- start 2024-04-23T10:26:20.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.675+0000 7f5cba9ed740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c1e64fcc70 0x55c1e6503dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.675+0000 7f5cba9ed740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1e64fd620 0x55c1e6504300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.675+0000 7f5cba9ed740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1e650a340 0x55c1e6504840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.675+0000 7f5cba9ed740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c1e651ced0 con 0x55c1e64fd620 2024-04-23T10:26:20.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.675+0000 7f5cba9ed740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c1e651c800 con 0x55c1e64fcc70 2024-04-23T10:26:20.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.675+0000 7f5cba9ed740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c1e651d5a0 con 0x55c1e650a340 2024-04-23T10:26:20.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.675+0000 7f5cba9ed740 10 client.0 _ll_register_callbacks cb 0x7f5ca4005ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:26:20.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.675+0000 7f5cb526a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1e64fd620 0x55c1e6504300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.675+0000 7f5cb526a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1e64fd620 0x55c1e6504300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47308/0 (socket says 172.21.15.97:47308) 2024-04-23T10:26:20.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.675+0000 7f5cb526a640 1 -- 172.21.15.97:0/291011046 learned_addr learned my addr 172.21.15.97:0/291011046 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:20.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.675+0000 7f5cb5a6b640 1 --2- 172.21.15.97:0/291011046 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c1e64fcc70 0x55c1e6503dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.675+0000 7f5cb626c640 1 --2- 172.21.15.97:0/291011046 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1e650a340 0x55c1e6504840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.675+0000 7f5cb526a640 1 -- 172.21.15.97:0/291011046 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1e650a340 msgr2=0x55c1e6504840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:20.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.675+0000 7f5cb526a640 1 --2- 172.21.15.97:0/291011046 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1e650a340 0x55c1e6504840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.675+0000 7f5cb526a640 1 -- 172.21.15.97:0/291011046 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c1e64fcc70 msgr2=0x55c1e6503dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:20.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.675+0000 7f5cb526a640 1 --2- 172.21.15.97:0/291011046 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c1e64fcc70 0x55c1e6503dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.675+0000 7f5cb526a640 1 -- 172.21.15.97:0/291011046 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1e6524270 con 0x55c1e64fd620 2024-04-23T10:26:20.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.675+0000 7f5cb5a6b640 1 --2- 172.21.15.97:0/291011046 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c1e64fcc70 0x55c1e6503dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:26:20.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.675+0000 7f5cb526a640 1 --2- 172.21.15.97:0/291011046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1e64fd620 0x55c1e6504300 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f5cac033000 tx=0x7f5cac0314a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f267d33d333874a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:20.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.675+0000 7f5c997fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:26:20.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.676+0000 7f5c997fa640 1 -- 172.21.15.97:0/291011046 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cac035040 con 0x55c1e64fd620 2024-04-23T10:26:20.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.676+0000 7f5c997fa640 1 -- 172.21.15.97:0/291011046 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f5cac004030 con 0x55c1e64fd620 2024-04-23T10:26:20.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.676+0000 7f5cba9ed740 1 -- 172.21.15.97:0/291011046 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ca809d640 con 0x55c1e64fd620 2024-04-23T10:26:20.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.676+0000 7f5c997fa640 1 -- 172.21.15.97:0/291011046 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5cac04a020 con 0x55c1e64fd620 2024-04-23T10:26:20.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.676+0000 7f5cba9ed740 10 client.15666 fetch_fsmap learned FSMap version 26 2024-04-23T10:26:20.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.676+0000 7f5cba9ed740 1 -- 172.21.15.97:0/291011046 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c1e6525550 con 0x55c1e64fd620 2024-04-23T10:26:20.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.676+0000 7f5c997fa640 1 -- 172.21.15.97:0/291011046 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ca809d640 con 0x55c1e64fd620 2024-04-23T10:26:20.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.676+0000 7f5c997fa640 1 -- 172.21.15.97:0/291011046 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5cac004880 con 0x55c1e64fd620 2024-04-23T10:26:20.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.676+0000 7f5cba9ed740 10 client.15666 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:26:20.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.676+0000 7f5cba9ed740 10 client.15666 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-23T10:26:20.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.676+0000 7f5cba9ed740 20 client.15666 populate_metadata read hostname 'smithi097' 2024-04-23T10:26:20.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.676+0000 7f5cba9ed740 1 --2- 172.21.15.97:0/291011046 >> [v2:172.21.15.196:6838/83843556,v1:172.21.15.196:6839/83843556] conn(0x55c1e6585f30 0x55c1e6584c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.676+0000 7f5cba9ed740 4 client.15666 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T10:26:20.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.676+0000 7f5cba9ed740 1 -- 172.21.15.97:0/291011046 --> [v2:172.21.15.196:6838/83843556,v1:172.21.15.196:6839/83843556] -- command(tid 0: {"prefix": "status"}) -- 0x55c1e63e49a0 con 0x55c1e6585f30 2024-04-23T10:26:20.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.676+0000 7f5cb5a6b640 1 --2- 172.21.15.97:0/291011046 >> [v2:172.21.15.196:6838/83843556,v1:172.21.15.196:6839/83843556] conn(0x55c1e6585f30 0x55c1e6584c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.677+0000 7f5c997fa640 10 client.15666 ms_handle_connect on v2:172.21.15.196:6838/83843556 2024-04-23T10:26:20.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.677+0000 7f5cb5a6b640 1 --2- 172.21.15.97:0/291011046 >> [v2:172.21.15.196:6838/83843556,v1:172.21.15.196:6839/83843556] conn(0x55c1e6585f30 0x55c1e6584c40 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=de61ef3365b9f3fc server_cookie=1b8f48ac499ba549 in_seq=0 out_seq=0 2024-04-23T10:26:20.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.677+0000 7f5c997fa640 1 -- 172.21.15.97:0/291011046 <== mds.1 v2:172.21.15.196:6838/83843556 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55c1e63e49a0 con 0x55c1e6585f30 2024-04-23T10:26:20.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.677+0000 7f5c997fa640 10 client.15666 handle_command_reply: tid=0 2024-04-23T10:26:20.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.677+0000 7f5cba9ed740 1 client.15666 shutdown 2024-04-23T10:26:20.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.677+0000 7f5cba9ed740 1 -- 172.21.15.97:0/291011046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1e64fd620 msgr2=0x55c1e6504300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:20.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.677+0000 7f5cba9ed740 1 --2- 172.21.15.97:0/291011046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1e64fd620 0x55c1e6504300 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f5cac033000 tx=0x7f5cac0314a0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.677+0000 7f5cba9ed740 1 -- 172.21.15.97:0/291011046 shutdown_connections 2024-04-23T10:26:20.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.677+0000 7f5cba9ed740 1 --2- 172.21.15.97:0/291011046 >> [v2:172.21.15.196:6838/83843556,v1:172.21.15.196:6839/83843556] conn(0x55c1e6585f30 0x55c1e6584c40 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.677+0000 7f5cba9ed740 1 --2- 172.21.15.97:0/291011046 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1e650a340 0x55c1e6504840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.677+0000 7f5c997fa640 0 client.15666 ms_handle_reset on v2:172.21.15.196:6838/83843556 2024-04-23T10:26:20.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.677+0000 7f5cba9ed740 1 --2- 172.21.15.97:0/291011046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1e64fd620 0x55c1e6504300 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.677+0000 7f5cba9ed740 1 --2- 172.21.15.97:0/291011046 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c1e64fcc70 0x55c1e6503dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.677+0000 7f5cba9ed740 1 -- 172.21.15.97:0/291011046 >> 172.21.15.97:0/291011046 conn(0x55c1e64f9280 msgr2=0x55c1e64f9660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:20.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.677+0000 7f5cba9ed740 1 -- 172.21.15.97:0/291011046 shutdown_connections 2024-04-23T10:26:20.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.677+0000 7f5cba9ed740 1 -- 172.21.15.97:0/291011046 wait complete. 2024-04-23T10:26:20.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.677+0000 7f5cba9ed740 20 client.15666 trim_cache size 0 max 16384 2024-04-23T10:26:20.683 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-23T10:26:20.683 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "9b22af88-0159-11ef-bc93-c7b262605968", 2024-04-23T10:26:20.683 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 1, 2024-04-23T10:26:20.683 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 24487, 2024-04-23T10:26:20.683 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-04-23T10:26:20.684 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-04-23T10:26:20.684 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-04-23T10:26:20.684 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 131.11185534000001, 2024-04-23T10:26:20.684 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 26, 2024-04-23T10:26:20.684 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 89, 2024-04-23T10:26:20.684 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 88, 2024-04-23T10:26:20.684 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 148.17386189499999 2024-04-23T10:26:20.684 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-23T10:26:20.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.678+0000 7f5cb7ccf640 1 -- 172.21.15.97:0/437905115 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f5c9c0a3e40 msgr2=0x7f5c9c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:20.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.678+0000 7f5cb7ccf640 1 --2- 172.21.15.97:0/437905115 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f5c9c0a3e40 0x7f5c9c0a6300 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5ca4005940 tx=0x7f5ca400cf90 comp rx=0 tx=0).stop 2024-04-23T10:26:20.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.678+0000 7f5cb7ccf640 1 -- 172.21.15.97:0/437905115 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cb0101080 msgr2=0x7f5cb0197630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:20.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.678+0000 7f5cb7ccf640 1 --2- 172.21.15.97:0/437905115 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cb0101080 0x7f5cb0197630 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f5ca800ebc0 tx=0x7f5ca800c6a0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.678+0000 7f5cb7ccf640 1 -- 172.21.15.97:0/437905115 shutdown_connections 2024-04-23T10:26:20.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.678+0000 7f5cb7ccf640 1 --2- 172.21.15.97:0/437905115 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f5c9c0a3e40 0x7f5c9c0a6300 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.678+0000 7f5cb7ccf640 1 --2- 172.21.15.97:0/437905115 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5cb019bef0 0x7f5cb019c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.678+0000 7f5cb7ccf640 1 --2- 172.21.15.97:0/437905115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5cb0106150 0x7f5cb0197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.678+0000 7f5cb7ccf640 1 --2- 172.21.15.97:0/437905115 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cb0101080 0x7f5cb0197630 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.678+0000 7f5cb7ccf640 1 -- 172.21.15.97:0/437905115 >> 172.21.15.97:0/437905115 conn(0x7f5cb00771d0 msgr2=0x7f5cb00fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:20.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.678+0000 7f5cb7ccf640 1 -- 172.21.15.97:0/437905115 shutdown_connections 2024-04-23T10:26:20.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.678+0000 7f5cb7ccf640 1 -- 172.21.15.97:0/437905115 wait complete. 2024-04-23T10:26:20.688 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-23T10:26:20.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.787+0000 7f53a61ad640 1 -- 172.21.15.97:0/1863935396 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53a0101af0 msgr2=0x7f53a010dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:20.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.787+0000 7f53a61ad640 1 --2- 172.21.15.97:0/1863935396 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53a0101af0 0x7f53a010dc40 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f5398009cd0 tx=0x7f539802a8e0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.787+0000 7f53a61ad640 1 -- 172.21.15.97:0/1863935396 shutdown_connections 2024-04-23T10:26:20.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.787+0000 7f53a61ad640 1 --2- 172.21.15.97:0/1863935396 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53a0101af0 0x7f53a010dc40 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.787+0000 7f53a61ad640 1 --2- 172.21.15.97:0/1863935396 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53a0101130 0x7f53a01015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.787+0000 7f53a61ad640 1 --2- 172.21.15.97:0/1863935396 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53a0107010 0x7f53a01073f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.787+0000 7f53a61ad640 1 -- 172.21.15.97:0/1863935396 >> 172.21.15.97:0/1863935396 conn(0x7f53a0077be0 msgr2=0x7f53a0077ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:20.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.788+0000 7f53a61ad640 1 -- 172.21.15.97:0/1863935396 shutdown_connections 2024-04-23T10:26:20.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.788+0000 7f53a61ad640 1 -- 172.21.15.97:0/1863935396 wait complete. 2024-04-23T10:26:20.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.788+0000 7f53a61ad640 1 Processor -- start 2024-04-23T10:26:20.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.788+0000 7f53a61ad640 1 -- start start 2024-04-23T10:26:20.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.788+0000 7f53a61ad640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53a0101130 0x7f53a0197660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.789 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.788+0000 7f53a61ad640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53a0101af0 0x7f53a0197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.788+0000 7f53a61ad640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53a0107010 0x7f53a019bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.788+0000 7f53a61ad640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f53a01110e0 con 0x7f53a0101af0 2024-04-23T10:26:20.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.788+0000 7f53a61ad640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f53a0110f60 con 0x7f53a0101130 2024-04-23T10:26:20.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.788+0000 7f53a61ad640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f53a0111260 con 0x7f53a0107010 2024-04-23T10:26:20.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.789+0000 7f539effd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53a0101af0 0x7f53a0197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.789+0000 7f539effd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53a0101af0 0x7f53a0197ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47322/0 (socket says 172.21.15.97:47322) 2024-04-23T10:26:20.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.789+0000 7f539effd640 1 -- 172.21.15.97:0/93149130 learned_addr learned my addr 172.21.15.97:0/93149130 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:20.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.789+0000 7f539ffff640 1 --2- 172.21.15.97:0/93149130 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53a0107010 0x7f53a019bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.789+0000 7f539f7fe640 1 --2- 172.21.15.97:0/93149130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53a0101130 0x7f53a0197660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.789+0000 7f539effd640 1 -- 172.21.15.97:0/93149130 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53a0107010 msgr2=0x7f53a019bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:20.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.789+0000 7f539effd640 1 --2- 172.21.15.97:0/93149130 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53a0107010 0x7f53a019bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.789+0000 7f539effd640 1 -- 172.21.15.97:0/93149130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53a0101130 msgr2=0x7f53a0197660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:20.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.789+0000 7f539effd640 1 --2- 172.21.15.97:0/93149130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53a0101130 0x7f53a0197660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.789+0000 7f539effd640 1 -- 172.21.15.97:0/93149130 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53a019c650 con 0x7f53a0101af0 2024-04-23T10:26:20.790 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.789+0000 7f539effd640 1 --2- 172.21.15.97:0/93149130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53a0101af0 0x7f53a0197ba0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f539400d770 tx=0x7f539400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:20.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.789+0000 7f539cff9640 1 -- 172.21.15.97:0/93149130 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5394017070 con 0x7f53a0101af0 2024-04-23T10:26:20.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.789+0000 7f539cff9640 1 -- 172.21.15.97:0/93149130 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53940054d0 con 0x7f53a0101af0 2024-04-23T10:26:20.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.789+0000 7f53a61ad640 1 -- 172.21.15.97:0/93149130 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53a019c940 con 0x7f53a0101af0 2024-04-23T10:26:20.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.789+0000 7f539cff9640 1 -- 172.21.15.97:0/93149130 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5394004c30 con 0x7f53a0101af0 2024-04-23T10:26:20.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.789+0000 7f53a61ad640 1 -- 172.21.15.97:0/93149130 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53a01a41d0 con 0x7f53a0101af0 2024-04-23T10:26:20.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.791+0000 7f539cff9640 1 -- 172.21.15.97:0/93149130 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f5394004e10 con 0x7f53a0101af0 2024-04-23T10:26:20.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.792+0000 7f53a8fa3740 1 Processor -- start 2024-04-23T10:26:20.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.792+0000 7f53a8fa3740 1 -- start start 2024-04-23T10:26:20.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.792+0000 7f53a8fa3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ff4619db0 0x555ff461a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.792+0000 7f53a8fa3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ff461a760 0x555ff4624800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.792+0000 7f53a8fa3740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ff4624dd0 0x555ff4627220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.792+0000 7f53a8fa3740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x555ff457fc50 con 0x555ff4624dd0 2024-04-23T10:26:20.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.792+0000 7f53a8fa3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555ff451c830 con 0x555ff461a760 2024-04-23T10:26:20.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.792+0000 7f53a8fa3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555ff457bc10 con 0x555ff4619db0 2024-04-23T10:26:20.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.792+0000 7f539cff9640 1 --2- 172.21.15.97:0/93149130 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f53880a3e40 0x7f53880a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.792+0000 7f539effd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ff4624dd0 0x555ff4627220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.792+0000 7f539ffff640 1 --2- 172.21.15.97:0/93149130 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f53880a3e40 0x7f53880a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.792+0000 7f539f7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ff461a760 0x555ff4624800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.792+0000 7f539effd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ff4624dd0 0x555ff4627220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47326/0 (socket says 172.21.15.97:47326) 2024-04-23T10:26:20.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.792+0000 7f539effd640 1 -- 172.21.15.97:0/2487565574 learned_addr learned my addr 172.21.15.97:0/2487565574 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:20.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.793+0000 7f539ffff640 1 --2- 172.21.15.97:0/2487565574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ff4619db0 0x555ff461a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.793+0000 7f539cff9640 1 -- 172.21.15.97:0/93149130 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f53940d04e0 con 0x7f53a0101af0 2024-04-23T10:26:20.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.793+0000 7f539effd640 1 -- 172.21.15.97:0/2487565574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ff4619db0 msgr2=0x555ff461a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:20.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.793+0000 7f539effd640 1 --2- 172.21.15.97:0/2487565574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ff4619db0 0x555ff461a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.793+0000 7f539effd640 1 -- 172.21.15.97:0/2487565574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ff461a760 msgr2=0x555ff4624800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:20.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.793+0000 7f539effd640 1 --2- 172.21.15.97:0/2487565574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ff461a760 0x555ff4624800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.793+0000 7f539effd640 1 -- 172.21.15.97:0/2487565574 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ff42922d0 con 0x555ff4624dd0 2024-04-23T10:26:20.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.793+0000 7f539effd640 1 --2- 172.21.15.97:0/2487565574 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ff4624dd0 0x555ff4627220 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f539409c260 tx=0x7f539406edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb51c2377c0757e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:20.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.793+0000 7f539ffff640 1 --2- 172.21.15.97:0/93149130 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f53880a3e40 0x7f53880a6300 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5398004af0 tx=0x7f5398009910 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:20.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.793+0000 7f5385ffb640 1 -- 172.21.15.97:0/2487565574 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5394020070 con 0x555ff4624dd0 2024-04-23T10:26:20.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.793+0000 7f5385ffb640 1 -- 172.21.15.97:0/2487565574 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53940a05d0 con 0x555ff4624dd0 2024-04-23T10:26:20.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.793+0000 7f5385ffb640 1 -- 172.21.15.97:0/2487565574 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53940669f0 con 0x555ff4624dd0 2024-04-23T10:26:20.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.794+0000 7f53a8fa3740 1 -- 172.21.15.97:0/2487565574 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ff4624dd0 msgr2=0x555ff4627220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:20.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.794+0000 7f53a8fa3740 1 --2- 172.21.15.97:0/2487565574 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ff4624dd0 0x555ff4627220 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f539409c260 tx=0x7f539406edc0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.794+0000 7f53a8fa3740 1 -- 172.21.15.97:0/2487565574 shutdown_connections 2024-04-23T10:26:20.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.794+0000 7f53a8fa3740 1 --2- 172.21.15.97:0/2487565574 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ff4624dd0 0x555ff4627220 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.794+0000 7f53a8fa3740 1 --2- 172.21.15.97:0/2487565574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ff461a760 0x555ff4624800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.794+0000 7f53a8fa3740 1 --2- 172.21.15.97:0/2487565574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ff4619db0 0x555ff461a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.794+0000 7f53a8fa3740 1 -- 172.21.15.97:0/2487565574 >> 172.21.15.97:0/2487565574 conn(0x555ff4616070 msgr2=0x555ff4616450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:20.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.794+0000 7f53a8fa3740 1 -- 172.21.15.97:0/2487565574 shutdown_connections 2024-04-23T10:26:20.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.794+0000 7f53a8fa3740 1 -- 172.21.15.97:0/2487565574 wait complete. 2024-04-23T10:26:20.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.794+0000 7f53a8fa3740 1 -- start start 2024-04-23T10:26:20.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.794+0000 7f53a8fa3740 1 Processor -- start 2024-04-23T10:26:20.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.794+0000 7f53a8fa3740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ff4619db0 0x555ff4621d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.794+0000 7f53a8fa3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ff461a760 0x555ff4622250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.794+0000 7f53a8fa3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ff4624dd0 0x555ff4622790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.794+0000 7f53a8fa3740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x555ff4627ce0 con 0x555ff4619db0 2024-04-23T10:26:20.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.794+0000 7f53a8fa3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555ff451a470 con 0x555ff461a760 2024-04-23T10:26:20.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.794+0000 7f53a8fa3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555ff4627e60 con 0x555ff4624dd0 2024-04-23T10:26:20.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.795+0000 7f53a8fa3740 10 client.0 _ll_register_callbacks cb 0x555ff453ff30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:26:20.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.795+0000 7f539f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ff4619db0 0x555ff4621d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.795+0000 7f539f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ff4619db0 0x555ff4621d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47328/0 (socket says 172.21.15.97:47328) 2024-04-23T10:26:20.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.795+0000 7f539f7fe640 1 -- 172.21.15.97:0/26255878 learned_addr learned my addr 172.21.15.97:0/26255878 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:20.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.795+0000 7f539effd640 1 --2- 172.21.15.97:0/26255878 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ff461a760 0x555ff4622250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.795+0000 7f539ffff640 1 --2- 172.21.15.97:0/26255878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ff4624dd0 0x555ff4622790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.795+0000 7f539f7fe640 1 -- 172.21.15.97:0/26255878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ff4624dd0 msgr2=0x555ff4622790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:20.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.795+0000 7f539f7fe640 1 --2- 172.21.15.97:0/26255878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ff4624dd0 0x555ff4622790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.795+0000 7f539f7fe640 1 -- 172.21.15.97:0/26255878 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ff461a760 msgr2=0x555ff4622250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:20.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.795+0000 7f539f7fe640 1 --2- 172.21.15.97:0/26255878 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ff461a760 0x555ff4622250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.795+0000 7f539f7fe640 1 -- 172.21.15.97:0/26255878 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ff435c6b0 con 0x555ff4619db0 2024-04-23T10:26:20.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.795+0000 7f539ffff640 1 --2- 172.21.15.97:0/26255878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ff4624dd0 0x555ff4622790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:26:20.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.795+0000 7f539f7fe640 1 --2- 172.21.15.97:0/26255878 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ff4619db0 0x555ff4621d10 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f539000bad0 tx=0x7f539000bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ff60bf92540850f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:20.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.795+0000 7f537effd640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:26:20.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.795+0000 7f537effd640 1 -- 172.21.15.97:0/26255878 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5390003e00 con 0x555ff4619db0 2024-04-23T10:26:20.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.795+0000 7f537effd640 1 -- 172.21.15.97:0/26255878 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f5390011440 con 0x555ff4619db0 2024-04-23T10:26:20.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.795+0000 7f53a8fa3740 1 -- 172.21.15.97:0/26255878 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53940a05d0 con 0x555ff4619db0 2024-04-23T10:26:20.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.795+0000 7f537effd640 1 -- 172.21.15.97:0/26255878 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5390011640 con 0x555ff4619db0 2024-04-23T10:26:20.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.795+0000 7f53a8fa3740 10 client.15678 fetch_fsmap learned FSMap version 26 2024-04-23T10:26:20.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.795+0000 7f53a8fa3740 1 -- 172.21.15.97:0/26255878 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555ff4634030 con 0x555ff4619db0 2024-04-23T10:26:20.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.795+0000 7f537effd640 1 -- 172.21.15.97:0/26255878 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53940a05d0 con 0x555ff4619db0 2024-04-23T10:26:20.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.796+0000 7f537effd640 1 -- 172.21.15.97:0/26255878 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f539001b9a0 con 0x555ff4619db0 2024-04-23T10:26:20.801 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.796+0000 7f53a8fa3740 10 client.15678 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:26:20.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.796+0000 7f53a8fa3740 10 client.15678 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-23T10:26:20.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.796+0000 7f53a8fa3740 20 client.15678 populate_metadata read hostname 'smithi097' 2024-04-23T10:26:20.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.796+0000 7f53a8fa3740 1 --2- 172.21.15.97:0/26255878 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x555ff46366b0 0x555ff4637b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.796+0000 7f53a8fa3740 4 client.15678 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:26:20.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.796+0000 7f53a8fa3740 1 -- 172.21.15.97:0/26255878 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ff458d670 con 0x555ff46366b0 2024-04-23T10:26:20.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.796+0000 7f539effd640 1 --2- 172.21.15.97:0/26255878 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x555ff46366b0 0x555ff4637b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.796+0000 7f537effd640 10 client.15678 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:26:20.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.796+0000 7f539effd640 1 --2- 172.21.15.97:0/26255878 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x555ff46366b0 0x555ff4637b20 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efaf53f300a7653c server_cookie=20240f333e342202 in_seq=0 out_seq=0 2024-04-23T10:26:20.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.797+0000 7f537effd640 1 -- 172.21.15.97:0/26255878 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555ff458d670 con 0x555ff46366b0 2024-04-23T10:26:20.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.797+0000 7f537effd640 10 client.15678 handle_command_reply: tid=0 2024-04-23T10:26:20.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.797+0000 7f53a8fa3740 1 client.15678 shutdown 2024-04-23T10:26:20.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.797+0000 7f53a8fa3740 1 -- 172.21.15.97:0/26255878 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ff4619db0 msgr2=0x555ff4621d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:20.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.797+0000 7f53a8fa3740 1 --2- 172.21.15.97:0/26255878 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ff4619db0 0x555ff4621d10 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f539000bad0 tx=0x7f539000bf90 comp rx=0 tx=0).stop 2024-04-23T10:26:20.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.797+0000 7f53a8fa3740 1 -- 172.21.15.97:0/26255878 shutdown_connections 2024-04-23T10:26:20.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.797+0000 7f53a8fa3740 1 --2- 172.21.15.97:0/26255878 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x555ff46366b0 0x555ff4637b20 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.797+0000 7f53a8fa3740 1 --2- 172.21.15.97:0/26255878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ff4624dd0 0x555ff4622790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.797+0000 7f537effd640 0 client.15678 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:26:20.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.797+0000 7f53a8fa3740 1 --2- 172.21.15.97:0/26255878 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ff461a760 0x555ff4622250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.797+0000 7f53a8fa3740 1 --2- 172.21.15.97:0/26255878 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ff4619db0 0x555ff4621d10 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.797+0000 7f53a8fa3740 1 -- 172.21.15.97:0/26255878 >> 172.21.15.97:0/26255878 conn(0x555ff4616070 msgr2=0x555ff461bee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:20.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.798+0000 7f53a8fa3740 1 -- 172.21.15.97:0/26255878 shutdown_connections 2024-04-23T10:26:20.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.798+0000 7f53a8fa3740 1 -- 172.21.15.97:0/26255878 wait complete. 2024-04-23T10:26:20.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.798+0000 7f53a8fa3740 20 client.15678 trim_cache size 0 max 16384 2024-04-23T10:26:20.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.815+0000 7f53a8fa3740 1 Processor -- start 2024-04-23T10:26:20.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.816+0000 7f53a8fa3740 1 -- start start 2024-04-23T10:26:20.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.816+0000 7f53a8fa3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ff4619db0 0x555ff4635fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.816+0000 7f53a8fa3740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ff461a760 0x555ff4624440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.816+0000 7f53a8fa3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ff4624dd0 0x555ff461c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.816+0000 7f53a8fa3740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x555ff4635e60 con 0x555ff461a760 2024-04-23T10:26:20.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.816+0000 7f53a8fa3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555ff4636530 con 0x555ff4624dd0 2024-04-23T10:26:20.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.816+0000 7f53a8fa3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555ff4636c00 con 0x555ff4619db0 2024-04-23T10:26:20.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.816+0000 7f539effd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ff461a760 0x555ff4624440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.816+0000 7f539effd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ff461a760 0x555ff4624440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47338/0 (socket says 172.21.15.97:47338) 2024-04-23T10:26:20.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.816+0000 7f539effd640 1 -- 172.21.15.97:0/714756222 learned_addr learned my addr 172.21.15.97:0/714756222 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:20.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.816+0000 7f539f7fe640 1 --2- 172.21.15.97:0/714756222 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ff4619db0 0x555ff4635fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.816+0000 7f539effd640 1 -- 172.21.15.97:0/714756222 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ff4619db0 msgr2=0x555ff4635fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:20.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.816+0000 7f539effd640 1 --2- 172.21.15.97:0/714756222 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ff4619db0 0x555ff4635fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.816+0000 7f539effd640 1 -- 172.21.15.97:0/714756222 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ff4624dd0 msgr2=0x555ff461c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:20.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.816+0000 7f539effd640 1 --2- 172.21.15.97:0/714756222 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ff4624dd0 0x555ff461c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.816+0000 7f539effd640 1 -- 172.21.15.97:0/714756222 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ff4620260 con 0x555ff461a760 2024-04-23T10:26:20.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.816+0000 7f539effd640 1 --2- 172.21.15.97:0/714756222 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ff461a760 0x555ff4624440 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f539409bfd0 tx=0x7f539401d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66c2299f0f6e169c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:20.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.816+0000 7f53857fa640 1 -- 172.21.15.97:0/714756222 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5394020040 con 0x555ff461a760 2024-04-23T10:26:20.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.816+0000 7f53857fa640 1 -- 172.21.15.97:0/714756222 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53940b5070 con 0x555ff461a760 2024-04-23T10:26:20.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.816+0000 7f53857fa640 1 -- 172.21.15.97:0/714756222 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5394098940 con 0x555ff461a760 2024-04-23T10:26:20.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.817+0000 7f53a8fa3740 1 -- 172.21.15.97:0/714756222 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ff461a760 msgr2=0x555ff4624440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:20.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.817+0000 7f53a8fa3740 1 --2- 172.21.15.97:0/714756222 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ff461a760 0x555ff4624440 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f539409bfd0 tx=0x7f539401d040 comp rx=0 tx=0).stop 2024-04-23T10:26:20.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.817+0000 7f53a8fa3740 1 -- 172.21.15.97:0/714756222 shutdown_connections 2024-04-23T10:26:20.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.817+0000 7f53a8fa3740 1 --2- 172.21.15.97:0/714756222 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ff4624dd0 0x555ff461c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.817+0000 7f53a8fa3740 1 --2- 172.21.15.97:0/714756222 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ff461a760 0x555ff4624440 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.817+0000 7f53a8fa3740 1 --2- 172.21.15.97:0/714756222 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ff4619db0 0x555ff4635fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.817+0000 7f53a8fa3740 1 -- 172.21.15.97:0/714756222 >> 172.21.15.97:0/714756222 conn(0x555ff4616070 msgr2=0x555ff46205e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:20.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.817+0000 7f53a8fa3740 1 -- 172.21.15.97:0/714756222 shutdown_connections 2024-04-23T10:26:20.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.817+0000 7f53a8fa3740 1 -- 172.21.15.97:0/714756222 wait complete. 2024-04-23T10:26:20.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.817+0000 7f53a8fa3740 1 -- start start 2024-04-23T10:26:20.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.818+0000 7f53a8fa3740 1 Processor -- start 2024-04-23T10:26:20.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.818+0000 7f53a8fa3740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ff4619db0 0x555ff46400c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.818+0000 7f53a8fa3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ff461a760 0x555ff4639460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.818+0000 7f53a8fa3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ff4624dd0 0x555ff46399a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.818+0000 7f53a8fa3740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x555ff46379a0 con 0x555ff4619db0 2024-04-23T10:26:20.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.818+0000 7f53a8fa3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555ff46372d0 con 0x555ff4624dd0 2024-04-23T10:26:20.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.818+0000 7f53a8fa3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555ff4638070 con 0x555ff461a760 2024-04-23T10:26:20.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.818+0000 7f53a8fa3740 10 client.0 _ll_register_callbacks cb 0x7f5390004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:26:20.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.818+0000 7f539f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ff4619db0 0x555ff46400c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.818+0000 7f539f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ff4619db0 0x555ff46400c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47354/0 (socket says 172.21.15.97:47354) 2024-04-23T10:26:20.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.818+0000 7f539f7fe640 1 -- 172.21.15.97:0/3944369517 learned_addr learned my addr 172.21.15.97:0/3944369517 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:20.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.818+0000 7f539effd640 1 --2- 172.21.15.97:0/3944369517 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ff461a760 0x555ff4639460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.818+0000 7f539f7fe640 1 -- 172.21.15.97:0/3944369517 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ff461a760 msgr2=0x555ff4639460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:20.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.818+0000 7f539f7fe640 1 --2- 172.21.15.97:0/3944369517 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ff461a760 0x555ff4639460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.818+0000 7f539f7fe640 1 -- 172.21.15.97:0/3944369517 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ff4624dd0 msgr2=0x555ff46399a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:20.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.818+0000 7f539ffff640 1 --2- 172.21.15.97:0/3944369517 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ff4624dd0 0x555ff46399a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.818+0000 7f539f7fe640 1 --2- 172.21.15.97:0/3944369517 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ff4624dd0 0x555ff46399a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.818+0000 7f539f7fe640 1 -- 172.21.15.97:0/3944369517 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ff463a110 con 0x555ff4619db0 2024-04-23T10:26:20.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.818+0000 7f539f7fe640 1 --2- 172.21.15.97:0/3944369517 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ff4619db0 0x555ff46400c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f539000baa0 tx=0x7f5390003e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24e02cc75fb76fa5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:20.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.818+0000 7f537effd640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:26:20.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.818+0000 7f537effd640 1 -- 172.21.15.97:0/3944369517 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5390030020 con 0x555ff4619db0 2024-04-23T10:26:20.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.818+0000 7f537effd640 1 -- 172.21.15.97:0/3944369517 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f539000b400 con 0x555ff4619db0 2024-04-23T10:26:20.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.818+0000 7f53a8fa3740 1 -- 172.21.15.97:0/3944369517 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53940b5070 con 0x555ff4619db0 2024-04-23T10:26:20.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.819+0000 7f537effd640 1 -- 172.21.15.97:0/3944369517 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5390028040 con 0x555ff4619db0 2024-04-23T10:26:20.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.819+0000 7f53a8fa3740 10 client.15684 fetch_fsmap learned FSMap version 26 2024-04-23T10:26:20.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.819+0000 7f53a8fa3740 1 -- 172.21.15.97:0/3944369517 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555ff469c660 con 0x555ff4619db0 2024-04-23T10:26:20.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.819+0000 7f537effd640 1 -- 172.21.15.97:0/3944369517 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53940b5070 con 0x555ff4619db0 2024-04-23T10:26:20.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.819+0000 7f537effd640 1 -- 172.21.15.97:0/3944369517 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f539001b700 con 0x555ff4619db0 2024-04-23T10:26:20.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.819+0000 7f53a8fa3740 10 client.15684 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:26:20.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.819+0000 7f53a8fa3740 10 client.15684 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-23T10:26:20.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.819+0000 7f53a8fa3740 20 client.15684 populate_metadata read hostname 'smithi097' 2024-04-23T10:26:20.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.819+0000 7f53a8fa3740 1 --2- 172.21.15.97:0/3944369517 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x555ff46a1230 0x555ff469ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:20.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.819+0000 7f53a8fa3740 4 client.15684 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T10:26:20.824 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.819+0000 7f53a8fa3740 1 -- 172.21.15.97:0/3944369517 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "status"}) -- 0x555ff45874e0 con 0x555ff46a1230 2024-04-23T10:26:20.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.819+0000 7f539effd640 1 --2- 172.21.15.97:0/3944369517 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x555ff46a1230 0x555ff469ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:20.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.820+0000 7f537effd640 10 client.15684 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:26:20.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.820+0000 7f539effd640 1 --2- 172.21.15.97:0/3944369517 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x555ff46a1230 0x555ff469ff40 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eea55f729ba8f586 server_cookie=ef3cfd685d51813e in_seq=0 out_seq=0 2024-04-23T10:26:20.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.820+0000 7f537effd640 1 -- 172.21.15.97:0/3944369517 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x555ff45874e0 con 0x555ff46a1230 2024-04-23T10:26:20.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.820+0000 7f537effd640 10 client.15684 handle_command_reply: tid=0 2024-04-23T10:26:20.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.820+0000 7f53a8fa3740 1 client.15684 shutdown 2024-04-23T10:26:20.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.820+0000 7f53a8fa3740 1 -- 172.21.15.97:0/3944369517 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ff4619db0 msgr2=0x555ff46400c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:20.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.820+0000 7f53a8fa3740 1 --2- 172.21.15.97:0/3944369517 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ff4619db0 0x555ff46400c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f539000baa0 tx=0x7f5390003e80 comp rx=0 tx=0).stop 2024-04-23T10:26:20.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.820+0000 7f53a8fa3740 1 -- 172.21.15.97:0/3944369517 shutdown_connections 2024-04-23T10:26:20.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.820+0000 7f53a8fa3740 1 --2- 172.21.15.97:0/3944369517 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x555ff46a1230 0x555ff469ff40 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.820+0000 7f53a8fa3740 1 --2- 172.21.15.97:0/3944369517 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555ff4624dd0 0x555ff46399a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.820+0000 7f53a8fa3740 1 --2- 172.21.15.97:0/3944369517 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555ff461a760 0x555ff4639460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.820+0000 7f537effd640 0 client.15684 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:26:20.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.820+0000 7f53a8fa3740 1 --2- 172.21.15.97:0/3944369517 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555ff4619db0 0x555ff46400c0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.820+0000 7f53a8fa3740 1 -- 172.21.15.97:0/3944369517 >> 172.21.15.97:0/3944369517 conn(0x555ff4616070 msgr2=0x555ff4620520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:20.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.820+0000 7f53a8fa3740 1 -- 172.21.15.97:0/3944369517 shutdown_connections 2024-04-23T10:26:20.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.820+0000 7f53a8fa3740 1 -- 172.21.15.97:0/3944369517 wait complete. 2024-04-23T10:26:20.826 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.820+0000 7f53a8fa3740 20 client.15684 trim_cache size 0 max 16384 2024-04-23T10:26:20.826 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-23T10:26:20.826 INFO:teuthology.orchestra.run.smithi097.stdout: "cluster_fsid": "9b22af88-0159-11ef-bc93-c7b262605968", 2024-04-23T10:26:20.826 INFO:teuthology.orchestra.run.smithi097.stdout: "whoami": 0, 2024-04-23T10:26:20.826 INFO:teuthology.orchestra.run.smithi097.stdout: "id": 24488, 2024-04-23T10:26:20.826 INFO:teuthology.orchestra.run.smithi097.stdout: "want_state": "up:active", 2024-04-23T10:26:20.826 INFO:teuthology.orchestra.run.smithi097.stdout: "state": "up:active", 2024-04-23T10:26:20.826 INFO:teuthology.orchestra.run.smithi097.stdout: "fs_name": "cephfs", 2024-04-23T10:26:20.826 INFO:teuthology.orchestra.run.smithi097.stdout: "rank_uptime": 133.26505374600001, 2024-04-23T10:26:20.826 INFO:teuthology.orchestra.run.smithi097.stdout: "mdsmap_epoch": 26, 2024-04-23T10:26:20.827 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch": 89, 2024-04-23T10:26:20.827 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_epoch_barrier": 88, 2024-04-23T10:26:20.827 INFO:teuthology.orchestra.run.smithi097.stdout: "uptime": 139.82213988500001 2024-04-23T10:26:20.827 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-23T10:26:20.827 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.821+0000 7f53a61ad640 1 -- 172.21.15.97:0/93149130 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f53880a3e40 msgr2=0x7f53880a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:20.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.821+0000 7f53a61ad640 1 --2- 172.21.15.97:0/93149130 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f53880a3e40 0x7f53880a6300 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5398004af0 tx=0x7f5398009910 comp rx=0 tx=0).stop 2024-04-23T10:26:20.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.821+0000 7f53a61ad640 1 -- 172.21.15.97:0/93149130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53a0101af0 msgr2=0x7f53a0197ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:20.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.821+0000 7f53a61ad640 1 --2- 172.21.15.97:0/93149130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53a0101af0 0x7f53a0197ba0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f539400d770 tx=0x7f539400dc30 comp rx=0 tx=0).stop 2024-04-23T10:26:20.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.821+0000 7f53a61ad640 1 -- 172.21.15.97:0/93149130 shutdown_connections 2024-04-23T10:26:20.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.821+0000 7f53a61ad640 1 --2- 172.21.15.97:0/93149130 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f53880a3e40 0x7f53880a6300 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.821+0000 7f53a61ad640 1 --2- 172.21.15.97:0/93149130 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53a0107010 0x7f53a019bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.821+0000 7f53a61ad640 1 --2- 172.21.15.97:0/93149130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53a0101af0 0x7f53a0197ba0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.821+0000 7f53a61ad640 1 --2- 172.21.15.97:0/93149130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53a0101130 0x7f53a0197660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:20.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.821+0000 7f53a61ad640 1 -- 172.21.15.97:0/93149130 >> 172.21.15.97:0/93149130 conn(0x7f53a0077be0 msgr2=0x7f53a00ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:20.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.821+0000 7f53a61ad640 1 -- 172.21.15.97:0/93149130 shutdown_connections 2024-04-23T10:26:20.828 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:20.821+0000 7f53a61ad640 1 -- 172.21.15.97:0/93149130 wait complete. 2024-04-23T10:26:20.831 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-04-23T10:26:21.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:20 smithi097 ceph-mon[29197]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 132 KiB/s wr, 80 op/s 2024-04-23T10:26:21.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:20 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/3635958980' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:26:21.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:20 smithi196 ceph-mon[32919]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 132 KiB/s wr, 80 op/s 2024-04-23T10:26:21.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:20 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/3635958980' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:26:21.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:20 smithi183 ceph-mon[32317]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 132 KiB/s wr, 80 op/s 2024-04-23T10:26:21.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:20 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/3635958980' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:26:23.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:22 smithi183 ceph-mon[32317]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 103 KiB/s wr, 62 op/s 2024-04-23T10:26:23.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:22 smithi196 ceph-mon[32919]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 103 KiB/s wr, 62 op/s 2024-04-23T10:26:23.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:22 smithi097 ceph-mon[29197]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 103 KiB/s wr, 62 op/s 2024-04-23T10:26:25.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:24 smithi196 ceph-mon[32919]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 85 KiB/s wr, 56 op/s 2024-04-23T10:26:25.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:24 smithi097 ceph-mon[29197]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 85 KiB/s wr, 56 op/s 2024-04-23T10:26:25.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:24 smithi183 ceph-mon[32317]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 85 KiB/s wr, 56 op/s 2024-04-23T10:26:26.909 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.910+0000 7f7a9cf7d640 1 -- 172.21.15.97:0/2314877227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a98106170 msgr2=0x7f7a9810dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:26.909 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.910+0000 7f7a9cf7d640 1 --2- 172.21.15.97:0/2314877227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a98106170 0x7f7a9810dd40 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f7a90009ca0 tx=0x7f7a9002a8c0 comp rx=0 tx=0).stop 2024-04-23T10:26:26.909 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.910+0000 7f7a9cf7d640 1 -- 172.21.15.97:0/2314877227 shutdown_connections 2024-04-23T10:26:26.909 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.910+0000 7f7a9cf7d640 1 --2- 172.21.15.97:0/2314877227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a98106170 0x7f7a9810dd40 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:26.909 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.910+0000 7f7a9cf7d640 1 --2- 172.21.15.97:0/2314877227 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a98101aa0 0x7f7a98105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:26.910 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.910+0000 7f7a9cf7d640 1 --2- 172.21.15.97:0/2314877227 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a981010f0 0x7f7a981014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:26.910 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.910+0000 7f7a9cf7d640 1 -- 172.21.15.97:0/2314877227 >> 172.21.15.97:0/2314877227 conn(0x7f7a98077fd0 msgr2=0x7f7a980783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:26.910 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.910+0000 7f7a9cf7d640 1 -- 172.21.15.97:0/2314877227 shutdown_connections 2024-04-23T10:26:26.910 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.910+0000 7f7a9cf7d640 1 -- 172.21.15.97:0/2314877227 wait complete. 2024-04-23T10:26:26.912 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.911+0000 7f7a9cf7d640 1 Processor -- start 2024-04-23T10:26:26.912 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.911+0000 7f7a9cf7d640 1 -- start start 2024-04-23T10:26:26.912 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.911+0000 7f7a9cf7d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a981010f0 0x7f7a98198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:26.912 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.911+0000 7f7a9cf7d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a98106170 0x7f7a98198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:26.912 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.911+0000 7f7a9cf7d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a9819cf00 0x7f7a9819d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:26.912 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.911+0000 7f7a9cf7d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7a981110c0 con 0x7f7a9819cf00 2024-04-23T10:26:26.912 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.911+0000 7f7a9cf7d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7a98110f40 con 0x7f7a981010f0 2024-04-23T10:26:26.912 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.911+0000 7f7a9cf7d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7a98111240 con 0x7f7a98106170 2024-04-23T10:26:26.912 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.911+0000 7f7a96d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a9819cf00 0x7f7a9819d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:26.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.911+0000 7f7a96d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a9819cf00 0x7f7a9819d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56302/0 (socket says 172.21.15.97:56302) 2024-04-23T10:26:26.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.911+0000 7f7a96d9d640 1 -- 172.21.15.97:0/3516765461 learned_addr learned my addr 172.21.15.97:0/3516765461 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:26.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.911+0000 7f7a9659c640 1 --2- 172.21.15.97:0/3516765461 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a981010f0 0x7f7a98198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:26.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.911+0000 7f7a95d9b640 1 --2- 172.21.15.97:0/3516765461 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a98106170 0x7f7a98198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:26.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.911+0000 7f7a96d9d640 1 -- 172.21.15.97:0/3516765461 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a98106170 msgr2=0x7f7a98198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:26.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.911+0000 7f7a96d9d640 1 --2- 172.21.15.97:0/3516765461 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a98106170 0x7f7a98198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:26.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.911+0000 7f7a96d9d640 1 -- 172.21.15.97:0/3516765461 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a981010f0 msgr2=0x7f7a98198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:26.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.911+0000 7f7a96d9d640 1 --2- 172.21.15.97:0/3516765461 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a981010f0 0x7f7a98198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:26.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.911+0000 7f7a96d9d640 1 -- 172.21.15.97:0/3516765461 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a9819d9f0 con 0x7f7a9819cf00 2024-04-23T10:26:26.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.911+0000 7f7a9659c640 1 --2- 172.21.15.97:0/3516765461 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a981010f0 0x7f7a98198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:26:26.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.911+0000 7f7a95d9b640 1 --2- 172.21.15.97:0/3516765461 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a98106170 0x7f7a98198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:26:26.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.912+0000 7f7a96d9d640 1 --2- 172.21.15.97:0/3516765461 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a9819cf00 0x7f7a9819d3b0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7a90005cc0 tx=0x7f7a9002f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:26.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.912+0000 7f7a877fe640 1 -- 172.21.15.97:0/3516765461 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a90003bf0 con 0x7f7a9819cf00 2024-04-23T10:26:26.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.912+0000 7f7a877fe640 1 -- 172.21.15.97:0/3516765461 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a90003d90 con 0x7f7a9819cf00 2024-04-23T10:26:26.914 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.912+0000 7f7a9cf7d640 1 -- 172.21.15.97:0/3516765461 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a9819dc80 con 0x7f7a9819cf00 2024-04-23T10:26:26.914 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.912+0000 7f7a877fe640 1 -- 172.21.15.97:0/3516765461 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9002fbc0 con 0x7f7a9819cf00 2024-04-23T10:26:26.914 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.912+0000 7f7a9cf7d640 1 -- 172.21.15.97:0/3516765461 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a981a54a0 con 0x7f7a9819cf00 2024-04-23T10:26:26.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.914+0000 7f7a9fd75740 1 Processor -- start 2024-04-23T10:26:26.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.914+0000 7f7a877fe640 1 -- 172.21.15.97:0/3516765461 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f7a9002fd60 con 0x7f7a9819cf00 2024-04-23T10:26:26.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.915+0000 7f7a9fd75740 1 -- start start 2024-04-23T10:26:26.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.915+0000 7f7a9fd75740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a761fc520 0x559a761fc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:26.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.915+0000 7f7a9fd75740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a761fce40 0x559a7620b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:26.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.915+0000 7f7a9fd75740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a7620bc10 0x559a7620e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:26.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.915+0000 7f7a9fd75740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559a76161970 con 0x559a761fce40 2024-04-23T10:26:26.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.915+0000 7f7a9fd75740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559a760fe4a0 con 0x559a761fc520 2024-04-23T10:26:26.920 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.915+0000 7f7a9fd75740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559a7615d930 con 0x559a7620bc10 2024-04-23T10:26:26.920 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.915+0000 7f7a9659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a761fce40 0x559a7620b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:26.920 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.915+0000 7f7a95d9b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a761fc520 0x559a761fc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:26.920 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.915+0000 7f7a9659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a761fce40 0x559a7620b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56304/0 (socket says 172.21.15.97:56304) 2024-04-23T10:26:26.920 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.915+0000 7f7a95d9b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a761fc520 0x559a761fc900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.97:58696/0 (socket says 172.21.15.97:58696) 2024-04-23T10:26:26.920 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.915+0000 7f7a9659c640 1 -- 172.21.15.97:0/2687760553 learned_addr learned my addr 172.21.15.97:0/2687760553 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:26.920 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.915+0000 7f7a95d9b640 1 --2- 172.21.15.97:0/2687760553 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a7620bc10 0x559a7620e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:26.920 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.915+0000 7f7a877fe640 1 --2- 172.21.15.97:0/3516765461 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f7a800a3ee0 0x7f7a800a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:26.920 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.916+0000 7f7a9659c640 1 -- 172.21.15.97:0/2687760553 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a7620bc10 msgr2=0x559a7620e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:26.920 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.916+0000 7f7a9659c640 1 --2- 172.21.15.97:0/2687760553 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a7620bc10 0x559a7620e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:26.920 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.916+0000 7f7a9659c640 1 -- 172.21.15.97:0/2687760553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a761fc520 msgr2=0x559a761fc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:26.920 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.916+0000 7f7a9659c640 1 --2- 172.21.15.97:0/2687760553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a761fc520 0x559a761fc900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:26.920 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.916+0000 7f7a9659c640 1 -- 172.21.15.97:0/2687760553 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a75e74210 con 0x559a761fce40 2024-04-23T10:26:26.920 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.916+0000 7f7a95d9b640 1 --2- 172.21.15.97:0/2687760553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a761fc520 0x559a761fc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T10:26:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.916+0000 7f7a96d9d640 1 --2- 172.21.15.97:0/3516765461 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f7a800a3ee0 0x7f7a800a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.916+0000 7f7a9659c640 1 --2- 172.21.15.97:0/2687760553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a761fce40 0x559a7620b6d0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f7a8800cca0 tx=0x7f7a8800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dce280e1c175feb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.916+0000 7f7a877fe640 1 -- 172.21.15.97:0/3516765461 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f7a900f20f0 con 0x7f7a9819cf00 2024-04-23T10:26:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.916+0000 7f7a84ff9640 1 -- 172.21.15.97:0/2687760553 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a88005220 con 0x559a761fce40 2024-04-23T10:26:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.916+0000 7f7a84ff9640 1 -- 172.21.15.97:0/2687760553 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a88004c50 con 0x559a761fce40 2024-04-23T10:26:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.916+0000 7f7a84ff9640 1 -- 172.21.15.97:0/2687760553 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a880107b0 con 0x559a761fce40 2024-04-23T10:26:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.917+0000 7f7a9fd75740 1 -- 172.21.15.97:0/2687760553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a761fce40 msgr2=0x559a7620b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.917+0000 7f7a96d9d640 1 --2- 172.21.15.97:0/3516765461 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f7a800a3ee0 0x7f7a800a63a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7a900ef880 tx=0x7f7a9008d880 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.917+0000 7f7a9fd75740 1 --2- 172.21.15.97:0/2687760553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a761fce40 0x559a7620b6d0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f7a8800cca0 tx=0x7f7a8800e720 comp rx=0 tx=0).stop 2024-04-23T10:26:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.917+0000 7f7a9fd75740 1 -- 172.21.15.97:0/2687760553 shutdown_connections 2024-04-23T10:26:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.917+0000 7f7a9fd75740 1 --2- 172.21.15.97:0/2687760553 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a7620bc10 0x559a7620e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.917+0000 7f7a9fd75740 1 --2- 172.21.15.97:0/2687760553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a761fce40 0x559a7620b6d0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.917+0000 7f7a9fd75740 1 --2- 172.21.15.97:0/2687760553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a761fc520 0x559a761fc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.917+0000 7f7a9fd75740 1 -- 172.21.15.97:0/2687760553 >> 172.21.15.97:0/2687760553 conn(0x559a761f8b30 msgr2=0x559a761f8f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:26.921 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.917+0000 7f7a9fd75740 1 -- 172.21.15.97:0/2687760553 shutdown_connections 2024-04-23T10:26:26.922 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.917+0000 7f7a9fd75740 1 -- 172.21.15.97:0/2687760553 wait complete. 2024-04-23T10:26:26.922 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.917+0000 7f7a9fd75740 1 -- start start 2024-04-23T10:26:26.922 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.917+0000 7f7a9fd75740 1 Processor -- start 2024-04-23T10:26:26.922 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.918+0000 7f7a9fd75740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a761fc520 0x559a7620b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:26.922 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.918+0000 7f7a9fd75740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a761fce40 0x559a762087a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:26.922 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.918+0000 7f7a9fd75740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a7620bc10 0x559a76208ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:26.922 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.918+0000 7f7a9fd75740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559a7620e840 con 0x559a761fc520 2024-04-23T10:26:26.922 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.918+0000 7f7a9fd75740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559a760fc0b0 con 0x559a7620bc10 2024-04-23T10:26:26.922 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.918+0000 7f7a9fd75740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559a7620e9c0 con 0x559a761fce40 2024-04-23T10:26:26.922 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.918+0000 7f7a9fd75740 10 client.0 _ll_register_callbacks cb 0x559a760e9d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:26:26.922 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.918+0000 7f7a95d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a761fc520 0x559a7620b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:26.922 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.918+0000 7f7a95d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a761fc520 0x559a7620b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56318/0 (socket says 172.21.15.97:56318) 2024-04-23T10:26:26.922 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.918+0000 7f7a95d9b640 1 -- 172.21.15.97:0/4118537710 learned_addr learned my addr 172.21.15.97:0/4118537710 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:26.922 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.918+0000 7f7a9659c640 1 --2- 172.21.15.97:0/4118537710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a761fce40 0x559a762087a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.918+0000 7f7a95d9b640 1 --2- 172.21.15.97:0/4118537710 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a7620bc10 0x559a76208ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.918+0000 7f7a95d9b640 1 -- 172.21.15.97:0/4118537710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a761fce40 msgr2=0x559a762087a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.918+0000 7f7a95d9b640 1 --2- 172.21.15.97:0/4118537710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a761fce40 0x559a762087a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.918+0000 7f7a95d9b640 1 -- 172.21.15.97:0/4118537710 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a7620bc10 msgr2=0x559a76208ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.918+0000 7f7a95d9b640 1 --2- 172.21.15.97:0/4118537710 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a7620bc10 0x559a76208ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.918+0000 7f7a95d9b640 1 -- 172.21.15.97:0/4118537710 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a75f40620 con 0x559a761fc520 2024-04-23T10:26:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.918+0000 7f7a95d9b640 1 --2- 172.21.15.97:0/4118537710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a761fc520 0x559a7620b4b0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f7a98100c30 tx=0x7f7a8c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6702ba142c047cb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.918+0000 7f7a7dffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:26:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.918+0000 7f7a7dffb640 1 -- 172.21.15.97:0/4118537710 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a8c004220 con 0x559a761fc520 2024-04-23T10:26:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.918+0000 7f7a7dffb640 1 -- 172.21.15.97:0/4118537710 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f7a8c0043c0 con 0x559a761fc520 2024-04-23T10:26:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.918+0000 7f7a9fd75740 1 -- 172.21.15.97:0/4118537710 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a88004c50 con 0x559a761fc520 2024-04-23T10:26:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.919+0000 7f7a7dffb640 1 -- 172.21.15.97:0/4118537710 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7a8c0035f0 con 0x559a761fc520 2024-04-23T10:26:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.919+0000 7f7a9fd75740 10 client.15696 fetch_fsmap learned FSMap version 26 2024-04-23T10:26:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.919+0000 7f7a9fd75740 1 -- 172.21.15.97:0/4118537710 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559a7621a770 con 0x559a761fc520 2024-04-23T10:26:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.919+0000 7f7a7dffb640 1 -- 172.21.15.97:0/4118537710 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a88004c50 con 0x559a761fc520 2024-04-23T10:26:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.919+0000 7f7a7dffb640 1 -- 172.21.15.97:0/4118537710 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f7a8c01c030 con 0x559a761fc520 2024-04-23T10:26:26.924 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.919+0000 7f7a9fd75740 10 client.15696 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:26:26.924 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.919+0000 7f7a9fd75740 10 client.15696 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:26:26.924 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.919+0000 7f7a9fd75740 20 client.15696 populate_metadata read hostname 'smithi097' 2024-04-23T10:26:26.924 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.919+0000 7f7a9fd75740 1 --2- 172.21.15.97:0/4118537710 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x559a7621bbe0 0x559a7621dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:26.924 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.919+0000 7f7a9fd75740 4 client.15696 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:26:26.924 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.919+0000 7f7a9fd75740 1 -- 172.21.15.97:0/4118537710 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a7616f390 con 0x559a7621bbe0 2024-04-23T10:26:26.924 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.919+0000 7f7a9659c640 1 --2- 172.21.15.97:0/4118537710 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x559a7621bbe0 0x559a7621dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:26.924 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.920+0000 7f7a9659c640 1 --2- 172.21.15.97:0/4118537710 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x559a7621bbe0 0x559a7621dfd0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7813298903cce003 server_cookie=a35fc48ebb467176 in_seq=0 out_seq=0 2024-04-23T10:26:26.924 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.920+0000 7f7a7dffb640 10 client.15696 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:26:26.925 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.920+0000 7f7a7dffb640 1 -- 172.21.15.97:0/4118537710 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559a7616f390 con 0x559a7621bbe0 2024-04-23T10:26:26.925 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.920+0000 7f7a7dffb640 10 client.15696 handle_command_reply: tid=0 2024-04-23T10:26:26.925 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.921+0000 7f7a9fd75740 1 client.15696 shutdown 2024-04-23T10:26:26.925 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.921+0000 7f7a9fd75740 1 -- 172.21.15.97:0/4118537710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a761fc520 msgr2=0x559a7620b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:26.925 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.921+0000 7f7a9fd75740 1 --2- 172.21.15.97:0/4118537710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a761fc520 0x559a7620b4b0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f7a98100c30 tx=0x7f7a8c004730 comp rx=0 tx=0).stop 2024-04-23T10:26:26.926 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.921+0000 7f7a9fd75740 1 -- 172.21.15.97:0/4118537710 shutdown_connections 2024-04-23T10:26:26.926 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.921+0000 7f7a9fd75740 1 --2- 172.21.15.97:0/4118537710 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x559a7621bbe0 0x559a7621dfd0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:26.926 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.921+0000 7f7a9fd75740 1 --2- 172.21.15.97:0/4118537710 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a7620bc10 0x559a76208ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:26.926 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.921+0000 7f7a9fd75740 1 --2- 172.21.15.97:0/4118537710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a761fce40 0x559a762087a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:26.926 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.921+0000 7f7a9fd75740 1 --2- 172.21.15.97:0/4118537710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a761fc520 0x559a7620b4b0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:26.926 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.921+0000 7f7a7dffb640 0 client.15696 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:26:26.926 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.921+0000 7f7a9fd75740 1 -- 172.21.15.97:0/4118537710 >> 172.21.15.97:0/4118537710 conn(0x559a761f8b30 msgr2=0x559a761fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:26.926 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.921+0000 7f7a9fd75740 1 -- 172.21.15.97:0/4118537710 shutdown_connections 2024-04-23T10:26:26.926 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.921+0000 7f7a9fd75740 1 -- 172.21.15.97:0/4118537710 wait complete. 2024-04-23T10:26:26.926 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.921+0000 7f7a9fd75740 20 client.15696 trim_cache size 0 max 16384 2024-04-23T10:26:26.937 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.937+0000 7f7a9fd75740 1 Processor -- start 2024-04-23T10:26:26.937 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.937+0000 7f7a9fd75740 1 -- start start 2024-04-23T10:26:26.937 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.937+0000 7f7a9fd75740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a761fc520 0x559a76208080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:26.937 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.937+0000 7f7a9fd75740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a761fce40 0x559a7620afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:26.937 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.937+0000 7f7a9fd75740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a7620bc10 0x559a76223890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:26.937 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.937+0000 7f7a9fd75740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7a981ab810 con 0x559a7620bc10 2024-04-23T10:26:26.937 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.937+0000 7f7a9fd75740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7a981ad350 con 0x559a761fc520 2024-04-23T10:26:26.938 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.937+0000 7f7a9fd75740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7a981abee0 con 0x559a761fce40 2024-04-23T10:26:26.938 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.937+0000 7f7a95d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a7620bc10 0x559a76223890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:26.938 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.937+0000 7f7a95d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a7620bc10 0x559a76223890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56328/0 (socket says 172.21.15.97:56328) 2024-04-23T10:26:26.938 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.937+0000 7f7a95d9b640 1 -- 172.21.15.97:0/2298083151 learned_addr learned my addr 172.21.15.97:0/2298083151 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:26.938 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.937+0000 7f7a95d9b640 1 --2- 172.21.15.97:0/2298083151 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a761fc520 0x559a76208080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:26.938 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.938+0000 7f7a9659c640 1 --2- 172.21.15.97:0/2298083151 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a761fce40 0x559a7620afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:26.938 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.938+0000 7f7a95d9b640 1 -- 172.21.15.97:0/2298083151 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a761fce40 msgr2=0x559a7620afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:26.938 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.938+0000 7f7a95d9b640 1 --2- 172.21.15.97:0/2298083151 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a761fce40 0x559a7620afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:26.938 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.938+0000 7f7a95d9b640 1 -- 172.21.15.97:0/2298083151 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a761fc520 msgr2=0x559a76208080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:26.938 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.938+0000 7f7a95d9b640 1 --2- 172.21.15.97:0/2298083151 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a761fc520 0x559a76208080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:26.938 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.938+0000 7f7a95d9b640 1 -- 172.21.15.97:0/2298083151 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a761fcc70 con 0x559a7620bc10 2024-04-23T10:26:26.938 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.938+0000 7f7a9659c640 1 --2- 172.21.15.97:0/2298083151 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a761fce40 0x559a7620afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:26:26.938 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.938+0000 7f7a95d9b640 1 --2- 172.21.15.97:0/2298083151 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a7620bc10 0x559a76223890 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c00efd0 tx=0x7f7a8c00e3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=528add116b4536d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:26.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.938+0000 7f7a7ffff640 1 -- 172.21.15.97:0/2298083151 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a8c02c030 con 0x559a7620bc10 2024-04-23T10:26:26.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.938+0000 7f7a7ffff640 1 -- 172.21.15.97:0/2298083151 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a8c026ad0 con 0x559a7620bc10 2024-04-23T10:26:26.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.938+0000 7f7a7ffff640 1 -- 172.21.15.97:0/2298083151 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a8c035b40 con 0x559a7620bc10 2024-04-23T10:26:26.942 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.939+0000 7f7a9fd75740 1 -- 172.21.15.97:0/2298083151 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a7620bc10 msgr2=0x559a76223890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:26.942 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.939+0000 7f7a9fd75740 1 --2- 172.21.15.97:0/2298083151 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a7620bc10 0x559a76223890 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c00efd0 tx=0x7f7a8c00e3b0 comp rx=0 tx=0).stop 2024-04-23T10:26:26.942 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.939+0000 7f7a9fd75740 1 -- 172.21.15.97:0/2298083151 shutdown_connections 2024-04-23T10:26:26.942 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.939+0000 7f7a9fd75740 1 --2- 172.21.15.97:0/2298083151 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a7620bc10 0x559a76223890 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:26.942 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.939+0000 7f7a9fd75740 1 --2- 172.21.15.97:0/2298083151 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a761fce40 0x559a7620afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:26.942 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.939+0000 7f7a9fd75740 1 --2- 172.21.15.97:0/2298083151 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a761fc520 0x559a76208080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:26.943 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.939+0000 7f7a9fd75740 1 -- 172.21.15.97:0/2298083151 >> 172.21.15.97:0/2298083151 conn(0x559a761f8b30 msgr2=0x559a761f7cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:26.943 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.939+0000 7f7a9fd75740 1 -- 172.21.15.97:0/2298083151 shutdown_connections 2024-04-23T10:26:26.943 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.939+0000 7f7a9fd75740 1 -- 172.21.15.97:0/2298083151 wait complete. 2024-04-23T10:26:26.943 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.940+0000 7f7a9fd75740 1 -- start start 2024-04-23T10:26:26.943 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.940+0000 7f7a9fd75740 1 Processor -- start 2024-04-23T10:26:26.943 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.940+0000 7f7a9fd75740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a761fc520 0x559a76213ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:26.943 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.940+0000 7f7a9fd75740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a761fce40 0x559a76205010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:26.943 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.940+0000 7f7a9fd75740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a7620bc10 0x559a76205550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:26.943 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.940+0000 7f7a9fd75740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7a981acc80 con 0x559a7620bc10 2024-04-23T10:26:26.943 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.940+0000 7f7a9fd75740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7a981ac5b0 con 0x559a761fce40 2024-04-23T10:26:26.943 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.940+0000 7f7a9fd75740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7a981a9810 con 0x559a761fc520 2024-04-23T10:26:26.943 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.940+0000 7f7a9fd75740 10 client.0 _ll_register_callbacks cb 0x7f7a8c0137f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:26:26.943 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.940+0000 7f7a95d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a7620bc10 0x559a76205550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:26.943 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.940+0000 7f7a95d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a7620bc10 0x559a76205550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56332/0 (socket says 172.21.15.97:56332) 2024-04-23T10:26:26.943 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.940+0000 7f7a95d9b640 1 -- 172.21.15.97:0/400031055 learned_addr learned my addr 172.21.15.97:0/400031055 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:26.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.940+0000 7f7a9659c640 1 --2- 172.21.15.97:0/400031055 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a761fce40 0x559a76205010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:26.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.940+0000 7f7a95d9b640 1 --2- 172.21.15.97:0/400031055 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a761fc520 0x559a76213ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:26.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.940+0000 7f7a95d9b640 1 -- 172.21.15.97:0/400031055 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a761fc520 msgr2=0x559a76213ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:26.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.940+0000 7f7a95d9b640 1 --2- 172.21.15.97:0/400031055 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a761fc520 0x559a76213ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:26.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.940+0000 7f7a95d9b640 1 -- 172.21.15.97:0/400031055 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a761fce40 msgr2=0x559a76205010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:26.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.940+0000 7f7a95d9b640 1 --2- 172.21.15.97:0/400031055 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a761fce40 0x559a76205010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:26.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.940+0000 7f7a95d9b640 1 -- 172.21.15.97:0/400031055 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a76205bc0 con 0x559a7620bc10 2024-04-23T10:26:26.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.941+0000 7f7a95d9b640 1 --2- 172.21.15.97:0/400031055 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a7620bc10 0x559a76205550 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c000c00 tx=0x7f7a8c0265d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b3db0e2315acfa2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:26.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.941+0000 7f7a7dffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:26:26.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.941+0000 7f7a7dffb640 1 -- 172.21.15.97:0/400031055 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a8c02c070 con 0x559a7620bc10 2024-04-23T10:26:26.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.941+0000 7f7a7dffb640 1 -- 172.21.15.97:0/400031055 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f7a8c004d70 con 0x559a7620bc10 2024-04-23T10:26:26.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.941+0000 7f7a9fd75740 1 -- 172.21.15.97:0/400031055 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a8c026ad0 con 0x559a7620bc10 2024-04-23T10:26:26.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.941+0000 7f7a7dffb640 1 -- 172.21.15.97:0/400031055 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7a8c00ead0 con 0x559a7620bc10 2024-04-23T10:26:26.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.941+0000 7f7a9fd75740 10 client.15702 fetch_fsmap learned FSMap version 26 2024-04-23T10:26:26.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.941+0000 7f7a9fd75740 1 -- 172.21.15.97:0/400031055 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559a76281ef0 con 0x559a7620bc10 2024-04-23T10:26:26.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.941+0000 7f7a7dffb640 1 -- 172.21.15.97:0/400031055 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a8c026ad0 con 0x559a7620bc10 2024-04-23T10:26:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.941+0000 7f7a7dffb640 1 -- 172.21.15.97:0/400031055 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f7a8c01c050 con 0x559a7620bc10 2024-04-23T10:26:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.941+0000 7f7a9fd75740 10 client.15702 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:26:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.941+0000 7f7a9fd75740 10 client.15702 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:26:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.941+0000 7f7a9fd75740 20 client.15702 populate_metadata read hostname 'smithi097' 2024-04-23T10:26:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.941+0000 7f7a9fd75740 1 --2- 172.21.15.97:0/400031055 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x559a76282540 0x559a76284930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.941+0000 7f7a9fd75740 4 client.15702 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:26:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.941+0000 7f7a9fd75740 1 -- 172.21.15.97:0/400031055 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559a76209070 con 0x559a76282540 2024-04-23T10:26:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.942+0000 7f7a9659c640 1 --2- 172.21.15.97:0/400031055 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x559a76282540 0x559a76284930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.942+0000 7f7a9659c640 1 --2- 172.21.15.97:0/400031055 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x559a76282540 0x559a76284930 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=859b1deeff2a48ad server_cookie=446bd1237330eb8b in_seq=0 out_seq=0 2024-04-23T10:26:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.942+0000 7f7a7dffb640 10 client.15702 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:26:26.953 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.954+0000 7f7a7dffb640 1 -- 172.21.15.97:0/400031055 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559a76209070 con 0x559a76282540 2024-04-23T10:26:26.953 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.954+0000 7f7a7dffb640 10 client.15702 handle_command_reply: tid=0 2024-04-23T10:26:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.954+0000 7f7a9fd75740 1 client.15702 shutdown 2024-04-23T10:26:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.954+0000 7f7a9fd75740 1 -- 172.21.15.97:0/400031055 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a7620bc10 msgr2=0x559a76205550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.954+0000 7f7a9fd75740 1 --2- 172.21.15.97:0/400031055 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a7620bc10 0x559a76205550 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c000c00 tx=0x7f7a8c0265d0 comp rx=0 tx=0).stop 2024-04-23T10:26:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.954+0000 7f7a9fd75740 1 -- 172.21.15.97:0/400031055 shutdown_connections 2024-04-23T10:26:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.954+0000 7f7a9fd75740 1 --2- 172.21.15.97:0/400031055 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x559a76282540 0x559a76284930 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.954+0000 7f7a9fd75740 1 --2- 172.21.15.97:0/400031055 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559a7620bc10 0x559a76205550 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.954+0000 7f7a9fd75740 1 --2- 172.21.15.97:0/400031055 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a761fce40 0x559a76205010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.954+0000 7f7a7dffb640 0 client.15702 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:26:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.954+0000 7f7a9fd75740 1 --2- 172.21.15.97:0/400031055 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559a761fc520 0x559a76213ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.954+0000 7f7a9fd75740 1 -- 172.21.15.97:0/400031055 >> 172.21.15.97:0/400031055 conn(0x559a761f8b30 msgr2=0x559a761fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.954+0000 7f7a9fd75740 1 -- 172.21.15.97:0/400031055 shutdown_connections 2024-04-23T10:26:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.954+0000 7f7a9fd75740 1 -- 172.21.15.97:0/400031055 wait complete. 2024-04-23T10:26:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.954+0000 7f7a9fd75740 20 client.15702 trim_cache size 0 max 16384 2024-04-23T10:26:26.955 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:26:26.955 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:26:26.955 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:26:26.955 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:26:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.955+0000 7f7a9cf7d640 1 -- 172.21.15.97:0/3516765461 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f7a800a3ee0 msgr2=0x7f7a800a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.955+0000 7f7a9cf7d640 1 --2- 172.21.15.97:0/3516765461 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f7a800a3ee0 0x7f7a800a63a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7a900ef880 tx=0x7f7a9008d880 comp rx=0 tx=0).stop 2024-04-23T10:26:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.955+0000 7f7a9cf7d640 1 -- 172.21.15.97:0/3516765461 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a9819cf00 msgr2=0x7f7a9819d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.955+0000 7f7a9cf7d640 1 --2- 172.21.15.97:0/3516765461 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a9819cf00 0x7f7a9819d3b0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7a90005cc0 tx=0x7f7a9002f380 comp rx=0 tx=0).stop 2024-04-23T10:26:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.955+0000 7f7a9cf7d640 1 -- 172.21.15.97:0/3516765461 shutdown_connections 2024-04-23T10:26:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.955+0000 7f7a9cf7d640 1 --2- 172.21.15.97:0/3516765461 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f7a800a3ee0 0x7f7a800a63a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.955+0000 7f7a9cf7d640 1 --2- 172.21.15.97:0/3516765461 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a9819cf00 0x7f7a9819d3b0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.955+0000 7f7a9cf7d640 1 --2- 172.21.15.97:0/3516765461 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a98106170 0x7f7a98198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.955+0000 7f7a9cf7d640 1 --2- 172.21.15.97:0/3516765461 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a981010f0 0x7f7a98198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.955+0000 7f7a9cf7d640 1 -- 172.21.15.97:0/3516765461 >> 172.21.15.97:0/3516765461 conn(0x7f7a98077fd0 msgr2=0x7f7a98069880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.955+0000 7f7a9cf7d640 1 -- 172.21.15.97:0/3516765461 shutdown_connections 2024-04-23T10:26:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:26.955+0000 7f7a9cf7d640 1 -- 172.21.15.97:0/3516765461 wait complete. 2024-04-23T10:26:27.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:26 smithi196 ceph-mon[32919]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 78 KiB/s wr, 51 op/s 2024-04-23T10:26:27.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:26 smithi097 ceph-mon[29197]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 78 KiB/s wr, 51 op/s 2024-04-23T10:26:27.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:26 smithi183 ceph-mon[32317]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 78 KiB/s wr, 51 op/s 2024-04-23T10:26:29.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:28 smithi196 ceph-mon[32919]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 78 KiB/s wr, 54 op/s 2024-04-23T10:26:29.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:28 smithi097 ceph-mon[29197]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 78 KiB/s wr, 54 op/s 2024-04-23T10:26:29.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:28 smithi183 ceph-mon[32317]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 78 KiB/s wr, 54 op/s 2024-04-23T10:26:31.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:30 smithi196 ceph-mon[32919]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 42 KiB/s wr, 31 op/s 2024-04-23T10:26:31.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:30 smithi097 ceph-mon[29197]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 42 KiB/s wr, 31 op/s 2024-04-23T10:26:31.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:30 smithi183 ceph-mon[32317]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 42 KiB/s wr, 31 op/s 2024-04-23T10:26:33.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:32 smithi196 ceph-mon[32919]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.3 KiB/s wr, 13 op/s 2024-04-23T10:26:33.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:32 smithi097 ceph-mon[29197]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.3 KiB/s wr, 13 op/s 2024-04-23T10:26:33.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:32 smithi183 ceph-mon[32317]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.3 KiB/s wr, 13 op/s 2024-04-23T10:26:35.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:34 smithi196 ceph-mon[32919]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.3 KiB/s wr, 17 op/s 2024-04-23T10:26:35.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:34 smithi097 ceph-mon[29197]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.3 KiB/s wr, 17 op/s 2024-04-23T10:26:35.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:34 smithi183 ceph-mon[32317]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.3 KiB/s wr, 17 op/s 2024-04-23T10:26:35.831 INFO:tasks.quiescer.fs.[cephfs]:Total duration (45.8) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-04-23T10:26:35.831 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.9 2024-04-23T10:26:35.831 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T10:26:35.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:35.937+0000 7fd12e95d640 1 -- 172.21.15.97:0/3667517520 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd128103b00 msgr2=0x7fd12810bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:35.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:35.937+0000 7fd12e95d640 1 --2- 172.21.15.97:0/3667517520 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd128103b00 0x7fd12810bfe0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fd11c009900 tx=0x7fd11c02a510 comp rx=0 tx=0).stop 2024-04-23T10:26:35.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:35.937+0000 7fd12e95d640 1 -- 172.21.15.97:0/3667517520 shutdown_connections 2024-04-23T10:26:35.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:35.937+0000 7fd12e95d640 1 --2- 172.21.15.97:0/3667517520 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd12810c520 0x7fd12810e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:35.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:35.937+0000 7fd12e95d640 1 --2- 172.21.15.97:0/3667517520 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd128103b00 0x7fd12810bfe0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:35.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:35.937+0000 7fd12e95d640 1 --2- 172.21.15.97:0/3667517520 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1281031e0 0x7fd1281035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:35.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:35.937+0000 7fd12e95d640 1 -- 172.21.15.97:0/3667517520 >> 172.21.15.97:0/3667517520 conn(0x7fd128077f50 msgr2=0x7fd128078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:35.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:35.937+0000 7fd12e95d640 1 -- 172.21.15.97:0/3667517520 shutdown_connections 2024-04-23T10:26:35.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:35.937+0000 7fd12e95d640 1 -- 172.21.15.97:0/3667517520 wait complete. 2024-04-23T10:26:35.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:35.938+0000 7fd12e95d640 1 Processor -- start 2024-04-23T10:26:35.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:35.938+0000 7fd12e95d640 1 -- start start 2024-04-23T10:26:35.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:35.938+0000 7fd12e95d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1281031e0 0x7fd12819e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:35.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:35.938+0000 7fd12e95d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd128103b00 0x7fd12819e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:35.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:35.938+0000 7fd12e95d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd12810c520 0x7fd1281985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:35.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:35.938+0000 7fd12e95d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd128112210 con 0x7fd12810c520 2024-04-23T10:26:35.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:35.938+0000 7fd12e95d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd128112090 con 0x7fd1281031e0 2024-04-23T10:26:35.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:35.938+0000 7fd12e95d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd128112390 con 0x7fd128103b00 2024-04-23T10:26:35.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:35.938+0000 7fd12cefa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd12810c520 0x7fd1281985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:35.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:35.938+0000 7fd12cefa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd12810c520 0x7fd1281985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48618/0 (socket says 172.21.15.97:48618) 2024-04-23T10:26:35.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:35.938+0000 7fd12cefa640 1 -- 172.21.15.97:0/3072384383 learned_addr learned my addr 172.21.15.97:0/3072384383 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:35.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:35.938+0000 7fd127fff640 1 --2- 172.21.15.97:0/3072384383 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1281031e0 0x7fd12819e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:35.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:35.938+0000 7fd12cefa640 1 -- 172.21.15.97:0/3072384383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd128103b00 msgr2=0x7fd12819e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:35.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:35.938+0000 7fd1277fe640 1 --2- 172.21.15.97:0/3072384383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd128103b00 0x7fd12819e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:35.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:35.938+0000 7fd12cefa640 1 --2- 172.21.15.97:0/3072384383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd128103b00 0x7fd12819e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:35.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:35.938+0000 7fd12cefa640 1 -- 172.21.15.97:0/3072384383 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1281031e0 msgr2=0x7fd12819e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:35.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:35.938+0000 7fd12cefa640 1 --2- 172.21.15.97:0/3072384383 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1281031e0 0x7fd12819e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:35.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:35.939+0000 7fd12cefa640 1 -- 172.21.15.97:0/3072384383 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd128198d70 con 0x7fd12810c520 2024-04-23T10:26:35.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:35.939+0000 7fd127fff640 1 --2- 172.21.15.97:0/3072384383 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1281031e0 0x7fd12819e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:26:35.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:35.939+0000 7fd1277fe640 1 --2- 172.21.15.97:0/3072384383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd128103b00 0x7fd12819e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:26:35.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:35.939+0000 7fd12cefa640 1 --2- 172.21.15.97:0/3072384383 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd12810c520 0x7fd1281985f0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fd12000b970 tx=0x7fd12000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:35.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:35.939+0000 7fd1257fa640 1 -- 172.21.15.97:0/3072384383 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd120019070 con 0x7fd12810c520 2024-04-23T10:26:35.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:35.939+0000 7fd12e95d640 1 -- 172.21.15.97:0/3072384383 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd128199060 con 0x7fd12810c520 2024-04-23T10:26:35.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:35.939+0000 7fd1257fa640 1 -- 172.21.15.97:0/3072384383 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd120014440 con 0x7fd12810c520 2024-04-23T10:26:35.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:35.939+0000 7fd1257fa640 1 -- 172.21.15.97:0/3072384383 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1200075d0 con 0x7fd12810c520 2024-04-23T10:26:35.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:35.939+0000 7fd12e95d640 1 -- 172.21.15.97:0/3072384383 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1281a5300 con 0x7fd12810c520 2024-04-23T10:26:35.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:35.940+0000 7fd12e95d640 1 -- 172.21.15.97:0/3072384383 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd128068d70 con 0x7fd12810c520 2024-04-23T10:26:35.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:35.942+0000 7fd1257fa640 1 -- 172.21.15.97:0/3072384383 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fd12001c070 con 0x7fd12810c520 2024-04-23T10:26:35.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:35.942+0000 7fd1257fa640 1 --2- 172.21.15.97:0/3072384383 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fd1100a3e40 0x7fd1100a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:35.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:35.943+0000 7fd127fff640 1 --2- 172.21.15.97:0/3072384383 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fd1100a3e40 0x7fd1100a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:35.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:35.943+0000 7fd1257fa640 1 -- 172.21.15.97:0/3072384383 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fd1200d0470 con 0x7fd12810c520 2024-04-23T10:26:35.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:35.943+0000 7fd127fff640 1 --2- 172.21.15.97:0/3072384383 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fd1100a3e40 0x7fd1100a6300 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd1180058c0 tx=0x7fd118005850 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:35.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:35.948+0000 7fd1257fa640 1 -- 172.21.15.97:0/3072384383 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd12009ca20 con 0x7fd12810c520 2024-04-23T10:26:36.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.180+0000 7fd12e95d640 1 -- 172.21.15.97:0/3072384383 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd128199b20 con 0x7fd12810c520 2024-04-23T10:26:36.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.181+0000 7fd1257fa640 1 -- 172.21.15.97:0/3072384383 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11931 (secure 0 0 0) 0x7fd128199b20 con 0x7fd12810c520 2024-04-23T10:26:36.180 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:26:36.180 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3946878762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3946878762},{"type":"v1","addr":"172.21.15.97:6835","nonce":3946878762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/209800861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":209800861},{"type":"v1","addr":"172.21.15.97:6837","nonce":209800861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T10:24:06.910134+0000","modified":"2024-04-23T10:26:09.820985+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6839/418013575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":418013575},{"type":"v1","addr":"172.21.15.97:6839","nonce":418013575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6841/747242680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":747242680},{"type":"v1","addr":"172.21.15.97:6841","nonce":747242680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6833/4292847121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4292847121},{"type":"v1","addr":"172.21.15.183:6833","nonce":4292847121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6833/3885816565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3885816565},{"type":"v1","addr":"172.21.15.196:6833","nonce":3885816565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1897093105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1897093105},{"type":"v1","addr":"172.21.15.183:6835","nonce":1897093105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1171477563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1171477563},{"type":"v1","addr":"172.21.15.196:6835","nonce":1171477563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1898341911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1898341911},{"type":"v1","addr":"172.21.15.183:6837","nonce":1898341911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.196:6837/3094946502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3094946502},{"type":"v1","addr":"172.21.15.196:6837","nonce":3094946502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.196:6839/83843556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":83843556},{"type":"v1","addr":"172.21.15.196:6839","nonce":83843556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6839/1763841130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1763841130},{"type":"v1","addr":"172.21.15.183:6839","nonce":1763841130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24487,24403,24398,24469,24488]},"id":1}]} 2024-04-23T10:26:36.181 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 26 2024-04-23T10:26:36.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.183+0000 7fd12e95d640 1 -- 172.21.15.97:0/3072384383 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fd1100a3e40 msgr2=0x7fd1100a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:36.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.183+0000 7fd12e95d640 1 --2- 172.21.15.97:0/3072384383 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fd1100a3e40 0x7fd1100a6300 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd1180058c0 tx=0x7fd118005850 comp rx=0 tx=0).stop 2024-04-23T10:26:36.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.183+0000 7fd12e95d640 1 -- 172.21.15.97:0/3072384383 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd12810c520 msgr2=0x7fd1281985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:36.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.183+0000 7fd12e95d640 1 --2- 172.21.15.97:0/3072384383 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd12810c520 0x7fd1281985f0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fd12000b970 tx=0x7fd12000be30 comp rx=0 tx=0).stop 2024-04-23T10:26:36.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.183+0000 7fd12e95d640 1 -- 172.21.15.97:0/3072384383 shutdown_connections 2024-04-23T10:26:36.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.183+0000 7fd12e95d640 1 --2- 172.21.15.97:0/3072384383 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fd1100a3e40 0x7fd1100a6300 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:36.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.183+0000 7fd12e95d640 1 --2- 172.21.15.97:0/3072384383 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd12810c520 0x7fd1281985f0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:36.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.183+0000 7fd12e95d640 1 --2- 172.21.15.97:0/3072384383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd128103b00 0x7fd12819e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:36.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.183+0000 7fd12e95d640 1 --2- 172.21.15.97:0/3072384383 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1281031e0 0x7fd12819e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:36.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.183+0000 7fd12e95d640 1 -- 172.21.15.97:0/3072384383 >> 172.21.15.97:0/3072384383 conn(0x7fd128077f50 msgr2=0x7fd1280fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:36.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.183+0000 7fd12e95d640 1 -- 172.21.15.97:0/3072384383 shutdown_connections 2024-04-23T10:26:36.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.183+0000 7fd12e95d640 1 -- 172.21.15.97:0/3072384383 wait complete. 2024-04-23T10:26:36.194 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db / --timeout 60 --expiration 82.9 --await-for 42.0' 2024-04-23T10:26:36.194 DEBUG:teuthology.orchestra.run.smithi097:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24488 quiesce db / --timeout 60 --expiration 82.9 --await-for 42.0 2024-04-23T10:26:36.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.310+0000 7f419f937640 1 -- 172.21.15.97:0/460428645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4198101ac0 msgr2=0x7f4198105ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:36.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.310+0000 7f419f937640 1 --2- 172.21.15.97:0/460428645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4198101ac0 0x7f4198105ab0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f4188009900 tx=0x7f418802a410 comp rx=0 tx=0).stop 2024-04-23T10:26:36.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.310+0000 7f419f937640 1 -- 172.21.15.97:0/460428645 shutdown_connections 2024-04-23T10:26:36.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.310+0000 7f419f937640 1 --2- 172.21.15.97:0/460428645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4198106190 0x7f419810dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:36.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.310+0000 7f419f937640 1 --2- 172.21.15.97:0/460428645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4198101ac0 0x7f4198105ab0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:36.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.310+0000 7f419f937640 1 --2- 172.21.15.97:0/460428645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4198101110 0x7f41981014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:36.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.310+0000 7f419f937640 1 -- 172.21.15.97:0/460428645 >> 172.21.15.97:0/460428645 conn(0x7f4198077fb0 msgr2=0x7f41980783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:36.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.311+0000 7f419f937640 1 -- 172.21.15.97:0/460428645 shutdown_connections 2024-04-23T10:26:36.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.311+0000 7f419f937640 1 -- 172.21.15.97:0/460428645 wait complete. 2024-04-23T10:26:36.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.311+0000 7f419f937640 1 Processor -- start 2024-04-23T10:26:36.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.311+0000 7f419f937640 1 -- start start 2024-04-23T10:26:36.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.311+0000 7f419f937640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4198101110 0x7f419819e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:36.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.311+0000 7f419f937640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4198106190 0x7f419819e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:36.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.311+0000 7f419f937640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f41981985a0 0x7f4198198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:36.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.311+0000 7f419f937640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f41981110b0 con 0x7f4198106190 2024-04-23T10:26:36.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.311+0000 7f419f937640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4198110f30 con 0x7f41981985a0 2024-04-23T10:26:36.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.311+0000 7f419f937640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4198111230 con 0x7f4198101110 2024-04-23T10:26:36.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.312+0000 7f419ced2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4198106190 0x7f419819e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:36.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.312+0000 7f419ced2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4198106190 0x7f419819e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48628/0 (socket says 172.21.15.97:48628) 2024-04-23T10:26:36.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.312+0000 7f419ced2640 1 -- 172.21.15.97:0/781276726 learned_addr learned my addr 172.21.15.97:0/781276726 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:36.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.312+0000 7f419d6d3640 1 --2- 172.21.15.97:0/781276726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4198101110 0x7f419819e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:36.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.312+0000 7f419ded4640 1 --2- 172.21.15.97:0/781276726 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f41981985a0 0x7f4198198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:36.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.312+0000 7f419ced2640 1 -- 172.21.15.97:0/781276726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4198101110 msgr2=0x7f419819e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:36.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.312+0000 7f419ced2640 1 --2- 172.21.15.97:0/781276726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4198101110 0x7f419819e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:36.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.312+0000 7f419ced2640 1 -- 172.21.15.97:0/781276726 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f41981985a0 msgr2=0x7f4198198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:36.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.312+0000 7f419ced2640 1 --2- 172.21.15.97:0/781276726 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f41981985a0 0x7f4198198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:36.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.312+0000 7f419ced2640 1 -- 172.21.15.97:0/781276726 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41981990c0 con 0x7f4198106190 2024-04-23T10:26:36.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.312+0000 7f419ced2640 1 --2- 172.21.15.97:0/781276726 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4198106190 0x7f419819e930 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f4188032fd0 tx=0x7f4188004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:36.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.312+0000 7f418e7fc640 1 -- 172.21.15.97:0/781276726 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f418803f030 con 0x7f4198106190 2024-04-23T10:26:36.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.312+0000 7f418e7fc640 1 -- 172.21.15.97:0/781276726 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4188004490 con 0x7f4198106190 2024-04-23T10:26:36.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.312+0000 7f419f937640 1 -- 172.21.15.97:0/781276726 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4198199350 con 0x7f4198106190 2024-04-23T10:26:36.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.312+0000 7f418e7fc640 1 -- 172.21.15.97:0/781276726 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4188035070 con 0x7f4198106190 2024-04-23T10:26:36.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.312+0000 7f419f937640 1 -- 172.21.15.97:0/781276726 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41981a55a0 con 0x7f4198106190 2024-04-23T10:26:36.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.315+0000 7f418e7fc640 1 -- 172.21.15.97:0/781276726 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f4188042030 con 0x7f4198106190 2024-04-23T10:26:36.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.315+0000 7f41a2650740 1 Processor -- start 2024-04-23T10:26:36.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.315+0000 7f41a2650740 1 -- start start 2024-04-23T10:26:36.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.315+0000 7f418e7fc640 1 --2- 172.21.15.97:0/781276726 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f41840a3ec0 0x7f41840a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:36.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.315+0000 7f41a2650740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622a13c60a0 0x5622a13c6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:36.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.315+0000 7f41a2650740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622a13c69c0 0x5622a13d0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:36.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.315+0000 7f41a2650740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622a13d1090 0x5622a13d3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:36.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.315+0000 7f41a2650740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5622a132b370 con 0x5622a13d1090 2024-04-23T10:26:36.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.315+0000 7f41a2650740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5622a12c82e0 con 0x5622a13c69c0 2024-04-23T10:26:36.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.315+0000 7f41a2650740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5622a1327330 con 0x5622a13c60a0 2024-04-23T10:26:36.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.315+0000 7f419ded4640 1 --2- 172.21.15.97:0/781276726 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f41840a3ec0 0x7f41840a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:36.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.315+0000 7f418e7fc640 1 -- 172.21.15.97:0/781276726 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f41880f0de0 con 0x7f4198106190 2024-04-23T10:26:36.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.316+0000 7f419ded4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622a13d1090 0x5622a13d3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:36.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.316+0000 7f419ded4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622a13d1090 0x5622a13d3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48636/0 (socket says 172.21.15.97:48636) 2024-04-23T10:26:36.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.316+0000 7f419ded4640 1 -- 172.21.15.97:0/2276065285 learned_addr learned my addr 172.21.15.97:0/2276065285 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:36.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.316+0000 7f419d6d3640 1 --2- 172.21.15.97:0/2276065285 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622a13c60a0 0x5622a13c6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:36.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.316+0000 7f419ced2640 1 --2- 172.21.15.97:0/2276065285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622a13c69c0 0x5622a13d0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:36.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.316+0000 7f419ded4640 1 -- 172.21.15.97:0/2276065285 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622a13c60a0 msgr2=0x5622a13c6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:36.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.316+0000 7f419ded4640 1 --2- 172.21.15.97:0/2276065285 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622a13c60a0 0x5622a13c6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:36.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.316+0000 7f419ded4640 1 -- 172.21.15.97:0/2276065285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622a13c69c0 msgr2=0x5622a13d0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:36.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.316+0000 7f419ded4640 1 --2- 172.21.15.97:0/2276065285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622a13c69c0 0x5622a13d0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:36.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.316+0000 7f419ded4640 1 -- 172.21.15.97:0/2276065285 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622a1109d80 con 0x5622a13d1090 2024-04-23T10:26:36.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.316+0000 7f419ded4640 1 --2- 172.21.15.97:0/2276065285 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622a13d1090 0x5622a13d3480 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4190011c80 tx=0x7f4190014f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaddc1845e237a42 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:36.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.316+0000 7f419ded4640 1 --2- 172.21.15.97:0/781276726 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f41840a3ec0 0x7f41840a6380 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4190014ad0 tx=0x7f4190009db0 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:36.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.316+0000 7f41837fe640 1 -- 172.21.15.97:0/2276065285 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4190014510 con 0x5622a13d1090 2024-04-23T10:26:36.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.316+0000 7f41837fe640 1 -- 172.21.15.97:0/2276065285 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4190014030 con 0x5622a13d1090 2024-04-23T10:26:36.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.316+0000 7f41837fe640 1 -- 172.21.15.97:0/2276065285 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4190003b90 con 0x5622a13d1090 2024-04-23T10:26:36.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.317+0000 7f41a2650740 1 -- 172.21.15.97:0/2276065285 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622a13d1090 msgr2=0x5622a13d3480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:36.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.317+0000 7f41a2650740 1 --2- 172.21.15.97:0/2276065285 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622a13d1090 0x5622a13d3480 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4190011c80 tx=0x7f4190014f90 comp rx=0 tx=0).stop 2024-04-23T10:26:36.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.317+0000 7f41a2650740 1 -- 172.21.15.97:0/2276065285 shutdown_connections 2024-04-23T10:26:36.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.317+0000 7f41a2650740 1 --2- 172.21.15.97:0/2276065285 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622a13d1090 0x5622a13d3480 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:36.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.317+0000 7f41a2650740 1 --2- 172.21.15.97:0/2276065285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622a13c69c0 0x5622a13d0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:36.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.317+0000 7f41a2650740 1 --2- 172.21.15.97:0/2276065285 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622a13c60a0 0x5622a13c6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:36.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.317+0000 7f41a2650740 1 -- 172.21.15.97:0/2276065285 >> 172.21.15.97:0/2276065285 conn(0x5622a13c23c0 msgr2=0x5622a13c27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:36.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.317+0000 7f41a2650740 1 -- 172.21.15.97:0/2276065285 shutdown_connections 2024-04-23T10:26:36.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.317+0000 7f41a2650740 1 -- 172.21.15.97:0/2276065285 wait complete. 2024-04-23T10:26:36.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.317+0000 7f41a2650740 1 -- start start 2024-04-23T10:26:36.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.317+0000 7f41a2650740 1 Processor -- start 2024-04-23T10:26:36.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.318+0000 7f41a2650740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622a13c60a0 0x5622a13ce050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:36.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.318+0000 7f41a2650740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622a13c69c0 0x5622a13ce590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:36.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.318+0000 7f41a2650740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622a13d1090 0x5622a13cead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:36.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.318+0000 7f41a2650740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5622a13d3dc0 con 0x5622a13d1090 2024-04-23T10:26:36.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.318+0000 7f41a2650740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5622a12c5fb0 con 0x5622a13c69c0 2024-04-23T10:26:36.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.318+0000 7f41a2650740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5622a13d3f40 con 0x5622a13c60a0 2024-04-23T10:26:36.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.318+0000 7f41a2650740 10 client.0 _ll_register_callbacks cb 0x5622a12c59a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:26:36.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.318+0000 7f419ded4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622a13d1090 0x5622a13cead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:36.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.318+0000 7f419ded4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622a13d1090 0x5622a13cead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48652/0 (socket says 172.21.15.97:48652) 2024-04-23T10:26:36.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.318+0000 7f419ded4640 1 -- 172.21.15.97:0/3120942084 learned_addr learned my addr 172.21.15.97:0/3120942084 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:36.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.318+0000 7f419ced2640 1 --2- 172.21.15.97:0/3120942084 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622a13c69c0 0x5622a13ce590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:36.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.318+0000 7f419ded4640 1 -- 172.21.15.97:0/3120942084 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622a13c60a0 msgr2=0x5622a13ce050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:36.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.318+0000 7f419d6d3640 1 --2- 172.21.15.97:0/3120942084 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622a13c60a0 0x5622a13ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:36.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.318+0000 7f419ded4640 1 --2- 172.21.15.97:0/3120942084 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622a13c60a0 0x5622a13ce050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:36.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.318+0000 7f419ded4640 1 -- 172.21.15.97:0/3120942084 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622a13c69c0 msgr2=0x5622a13ce590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:36.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.318+0000 7f419ded4640 1 --2- 172.21.15.97:0/3120942084 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622a13c69c0 0x5622a13ce590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:36.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.318+0000 7f419ded4640 1 -- 172.21.15.97:0/3120942084 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622a116a510 con 0x5622a13d1090 2024-04-23T10:26:36.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.318+0000 7f419ced2640 1 --2- 172.21.15.97:0/3120942084 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622a13c69c0 0x5622a13ce590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:26:36.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.318+0000 7f419ded4640 1 --2- 172.21.15.97:0/3120942084 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622a13d1090 0x5622a13cead0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f41900092f0 tx=0x7f4190011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea6073d31b2a368a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:36.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.318+0000 7f4180ff9640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:26:36.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.318+0000 7f4180ff9640 1 -- 172.21.15.97:0/3120942084 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f419002c070 con 0x5622a13d1090 2024-04-23T10:26:36.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.318+0000 7f4180ff9640 1 -- 172.21.15.97:0/3120942084 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f4190011d70 con 0x5622a13d1090 2024-04-23T10:26:36.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.318+0000 7f41a2650740 1 -- 172.21.15.97:0/3120942084 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4190014030 con 0x5622a13d1090 2024-04-23T10:26:36.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.319+0000 7f4180ff9640 1 -- 172.21.15.97:0/3120942084 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4190038050 con 0x5622a13d1090 2024-04-23T10:26:36.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.319+0000 7f41a2650740 10 client.15720 fetch_fsmap learned FSMap version 26 2024-04-23T10:26:36.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.319+0000 7f41a2650740 1 -- 172.21.15.97:0/3120942084 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5622a0ffd670 con 0x5622a13d1090 2024-04-23T10:26:36.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.319+0000 7f4180ff9640 1 -- 172.21.15.97:0/3120942084 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4190014030 con 0x5622a13d1090 2024-04-23T10:26:36.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.319+0000 7f4180ff9640 1 -- 172.21.15.97:0/3120942084 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f419000f050 con 0x5622a13d1090 2024-04-23T10:26:36.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.319+0000 7f41a2650740 10 client.15720 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:26:36.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.319+0000 7f41a2650740 10 client.15720 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:26:36.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.319+0000 7f41a2650740 20 client.15720 populate_metadata read hostname 'smithi097' 2024-04-23T10:26:36.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.319+0000 7f41a2650740 1 --2- 172.21.15.97:0/3120942084 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5622a13e12c0 0x5622a13e36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:36.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.319+0000 7f41a2650740 4 client.15720 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:26:36.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.319+0000 7f41a2650740 1 -- 172.21.15.97:0/3120942084 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5622a1338d90 con 0x5622a13e12c0 2024-04-23T10:26:36.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.319+0000 7f419d6d3640 1 --2- 172.21.15.97:0/3120942084 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5622a13e12c0 0x5622a13e36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:36.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.320+0000 7f4180ff9640 10 client.15720 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:26:36.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.320+0000 7f419d6d3640 1 --2- 172.21.15.97:0/3120942084 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5622a13e12c0 0x5622a13e36b0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e367356e0c307731 server_cookie=756b062c351a4f0d in_seq=0 out_seq=0 2024-04-23T10:26:36.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.320+0000 7f4180ff9640 1 -- 172.21.15.97:0/3120942084 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5622a1338d90 con 0x5622a13e12c0 2024-04-23T10:26:36.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.320+0000 7f4180ff9640 10 client.15720 handle_command_reply: tid=0 2024-04-23T10:26:36.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.320+0000 7f41a2650740 1 client.15720 shutdown 2024-04-23T10:26:36.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.320+0000 7f41a2650740 1 -- 172.21.15.97:0/3120942084 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622a13d1090 msgr2=0x5622a13cead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:36.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.320+0000 7f41a2650740 1 --2- 172.21.15.97:0/3120942084 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622a13d1090 0x5622a13cead0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f41900092f0 tx=0x7f4190011ae0 comp rx=0 tx=0).stop 2024-04-23T10:26:36.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.321+0000 7f41a2650740 1 -- 172.21.15.97:0/3120942084 shutdown_connections 2024-04-23T10:26:36.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.321+0000 7f41a2650740 1 --2- 172.21.15.97:0/3120942084 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5622a13e12c0 0x5622a13e36b0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:36.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.321+0000 7f41a2650740 1 --2- 172.21.15.97:0/3120942084 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622a13d1090 0x5622a13cead0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:36.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.321+0000 7f41a2650740 1 --2- 172.21.15.97:0/3120942084 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622a13c69c0 0x5622a13ce590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:36.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.321+0000 7f4180ff9640 0 client.15720 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:26:36.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.321+0000 7f41a2650740 1 --2- 172.21.15.97:0/3120942084 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622a13c60a0 0x5622a13ce050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:36.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.321+0000 7f41a2650740 1 -- 172.21.15.97:0/3120942084 >> 172.21.15.97:0/3120942084 conn(0x5622a13c23c0 msgr2=0x5622a13c81c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:36.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.321+0000 7f41a2650740 1 -- 172.21.15.97:0/3120942084 shutdown_connections 2024-04-23T10:26:36.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.321+0000 7f41a2650740 1 -- 172.21.15.97:0/3120942084 wait complete. 2024-04-23T10:26:36.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.321+0000 7f41a2650740 20 client.15720 trim_cache size 0 max 16384 2024-04-23T10:26:36.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.346+0000 7f41a2650740 1 Processor -- start 2024-04-23T10:26:36.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.346+0000 7f41a2650740 1 -- start start 2024-04-23T10:26:36.347 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.347+0000 7f41a2650740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622a13c60a0 0x5622a13c13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:36.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.347+0000 7f41a2650740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622a13c69c0 0x5622a13dccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:36.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.347+0000 7f41a2650740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622a13d1090 0x5622a13e8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:36.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.347+0000 7f41a2650740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f41981adaf0 con 0x5622a13c69c0 2024-04-23T10:26:36.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.347+0000 7f41a2650740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f41981af630 con 0x5622a13c60a0 2024-04-23T10:26:36.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.347+0000 7f41a2650740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f41981ae1c0 con 0x5622a13d1090 2024-04-23T10:26:36.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.347+0000 7f419ced2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622a13c69c0 0x5622a13dccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:36.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.347+0000 7f419ced2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622a13c69c0 0x5622a13dccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48656/0 (socket says 172.21.15.97:48656) 2024-04-23T10:26:36.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.347+0000 7f419ced2640 1 -- 172.21.15.97:0/3192637131 learned_addr learned my addr 172.21.15.97:0/3192637131 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:36.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.347+0000 7f419d6d3640 1 --2- 172.21.15.97:0/3192637131 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622a13c60a0 0x5622a13c13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:36.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.347+0000 7f419ded4640 1 --2- 172.21.15.97:0/3192637131 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622a13d1090 0x5622a13e8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:36.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.347+0000 7f419ced2640 1 -- 172.21.15.97:0/3192637131 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622a13d1090 msgr2=0x5622a13e8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:36.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.347+0000 7f419ced2640 1 --2- 172.21.15.97:0/3192637131 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622a13d1090 0x5622a13e8c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:36.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.347+0000 7f419ced2640 1 -- 172.21.15.97:0/3192637131 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622a13c60a0 msgr2=0x5622a13c13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:36.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.347+0000 7f419ced2640 1 --2- 172.21.15.97:0/3192637131 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622a13c60a0 0x5622a13c13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:36.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.347+0000 7f419ced2640 1 -- 172.21.15.97:0/3192637131 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622a13cefb0 con 0x5622a13c69c0 2024-04-23T10:26:36.348 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.347+0000 7f419ced2640 1 --2- 172.21.15.97:0/3192637131 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622a13c69c0 0x5622a13dccd0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f41880bc950 tx=0x7f41880bc850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8e65363cf44185a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:36.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.347+0000 7f4182ffd640 1 -- 172.21.15.97:0/3192637131 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4188049040 con 0x5622a13c69c0 2024-04-23T10:26:36.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.347+0000 7f4182ffd640 1 -- 172.21.15.97:0/3192637131 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41880bd350 con 0x5622a13c69c0 2024-04-23T10:26:36.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.347+0000 7f4182ffd640 1 -- 172.21.15.97:0/3192637131 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41880b92c0 con 0x5622a13c69c0 2024-04-23T10:26:36.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.348+0000 7f41a2650740 1 -- 172.21.15.97:0/3192637131 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622a13c69c0 msgr2=0x5622a13dccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:36.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.348+0000 7f41a2650740 1 --2- 172.21.15.97:0/3192637131 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622a13c69c0 0x5622a13dccd0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f41880bc950 tx=0x7f41880bc850 comp rx=0 tx=0).stop 2024-04-23T10:26:36.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.348+0000 7f41a2650740 1 -- 172.21.15.97:0/3192637131 shutdown_connections 2024-04-23T10:26:36.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.348+0000 7f41a2650740 1 --2- 172.21.15.97:0/3192637131 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622a13d1090 0x5622a13e8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:36.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.348+0000 7f41a2650740 1 --2- 172.21.15.97:0/3192637131 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622a13c69c0 0x5622a13dccd0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:36.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.348+0000 7f41a2650740 1 --2- 172.21.15.97:0/3192637131 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622a13c60a0 0x5622a13c13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:36.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.348+0000 7f41a2650740 1 -- 172.21.15.97:0/3192637131 >> 172.21.15.97:0/3192637131 conn(0x5622a13c23c0 msgr2=0x5622a13ce8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:36.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.348+0000 7f41a2650740 1 -- 172.21.15.97:0/3192637131 shutdown_connections 2024-04-23T10:26:36.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.348+0000 7f41a2650740 1 -- 172.21.15.97:0/3192637131 wait complete. 2024-04-23T10:26:36.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.349+0000 7f41a2650740 1 -- start start 2024-04-23T10:26:36.349 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.349+0000 7f41a2650740 1 Processor -- start 2024-04-23T10:26:36.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.349+0000 7f41a2650740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622a13c60a0 0x5622a1444e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:36.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.349+0000 7f41a2650740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622a13c69c0 0x5622a1445340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:36.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.349+0000 7f41a2650740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622a13d1090 0x5622a143f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:36.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.349+0000 7f41a2650740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f41981aef60 con 0x5622a13d1090 2024-04-23T10:26:36.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.349+0000 7f41a2650740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f41981ae890 con 0x5622a13c60a0 2024-04-23T10:26:36.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.349+0000 7f41a2650740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f41981abad0 con 0x5622a13c69c0 2024-04-23T10:26:36.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.349+0000 7f41a2650740 10 client.0 _ll_register_callbacks cb 0x7f4190003cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:26:36.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.349+0000 7f419ded4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622a13d1090 0x5622a143f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:36.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.349+0000 7f419ded4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622a13d1090 0x5622a143f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48670/0 (socket says 172.21.15.97:48670) 2024-04-23T10:26:36.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.349+0000 7f419ded4640 1 -- 172.21.15.97:0/3146124288 learned_addr learned my addr 172.21.15.97:0/3146124288 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:36.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.349+0000 7f419d6d3640 1 --2- 172.21.15.97:0/3146124288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622a13c60a0 0x5622a1444e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:36.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.349+0000 7f419ced2640 1 --2- 172.21.15.97:0/3146124288 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622a13c69c0 0x5622a1445340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:36.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.349+0000 7f419ded4640 1 -- 172.21.15.97:0/3146124288 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622a13c69c0 msgr2=0x5622a1445340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:36.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.349+0000 7f419ded4640 1 --2- 172.21.15.97:0/3146124288 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622a13c69c0 0x5622a1445340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:36.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.349+0000 7f419ded4640 1 -- 172.21.15.97:0/3146124288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622a13c60a0 msgr2=0x5622a1444e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:36.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.349+0000 7f419ded4640 1 --2- 172.21.15.97:0/3146124288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622a13c60a0 0x5622a1444e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:36.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.350+0000 7f419ded4640 1 -- 172.21.15.97:0/3146124288 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622a143fce0 con 0x5622a13d1090 2024-04-23T10:26:36.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.350+0000 7f419ded4640 1 --2- 172.21.15.97:0/3146124288 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622a13d1090 0x5622a143f4b0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f41900145e0 tx=0x7f41900117d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa8c3dfb90445954 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:36.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.350+0000 7f4180ff9640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:26:36.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.350+0000 7f4180ff9640 1 -- 172.21.15.97:0/3146124288 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4190041060 con 0x5622a13d1090 2024-04-23T10:26:36.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.350+0000 7f41a2650740 1 -- 172.21.15.97:0/3146124288 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41880bd350 con 0x5622a13d1090 2024-04-23T10:26:36.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.350+0000 7f4180ff9640 1 -- 172.21.15.97:0/3146124288 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f4190014040 con 0x5622a13d1090 2024-04-23T10:26:36.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.350+0000 7f4180ff9640 1 -- 172.21.15.97:0/3146124288 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f419002c040 con 0x5622a13d1090 2024-04-23T10:26:36.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.350+0000 7f41a2650740 10 client.15726 fetch_fsmap learned FSMap version 26 2024-04-23T10:26:36.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.350+0000 7f41a2650740 1 -- 172.21.15.97:0/3146124288 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5622a144c5b0 con 0x5622a13d1090 2024-04-23T10:26:36.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.350+0000 7f4180ff9640 1 -- 172.21.15.97:0/3146124288 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41880bd350 con 0x5622a13d1090 2024-04-23T10:26:36.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.350+0000 7f4180ff9640 1 -- 172.21.15.97:0/3146124288 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f41900050d0 con 0x5622a13d1090 2024-04-23T10:26:36.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.350+0000 7f41a2650740 10 client.15726 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:26:36.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.350+0000 7f41a2650740 10 client.15726 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:26:36.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.350+0000 7f41a2650740 20 client.15726 populate_metadata read hostname 'smithi097' 2024-04-23T10:26:36.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.350+0000 7f41a2650740 1 --2- 172.21.15.97:0/3146124288 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5622a144cc50 0x5622a144f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:36.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.350+0000 7f41a2650740 4 client.15726 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 82.9, "await_for": 42.0}] 2024-04-23T10:26:36.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.350+0000 7f41a2650740 1 -- 172.21.15.97:0/3146124288 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 82.9, "await_for": 42.0}) -- 0x5622a1332c00 con 0x5622a144cc50 2024-04-23T10:26:36.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.351+0000 7f419d6d3640 1 --2- 172.21.15.97:0/3146124288 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5622a144cc50 0x5622a144f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:36.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.351+0000 7f419d6d3640 1 --2- 172.21.15.97:0/3146124288 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5622a144cc50 0x5622a144f070 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8f9d86251f6c57f server_cookie=b52337556184dbb in_seq=0 out_seq=0 2024-04-23T10:26:36.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.351+0000 7f4180ff9640 10 client.15726 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:26:36.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.364+0000 7f4180ff9640 1 -- 172.21.15.97:0/3146124288 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x5622a1332c00 con 0x5622a144cc50 2024-04-23T10:26:36.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.364+0000 7f4180ff9640 10 client.15726 handle_command_reply: tid=0 2024-04-23T10:26:36.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.365+0000 7f41a2650740 1 client.15726 shutdown 2024-04-23T10:26:36.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.365+0000 7f41a2650740 1 -- 172.21.15.97:0/3146124288 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622a13d1090 msgr2=0x5622a143f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:36.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.365+0000 7f41a2650740 1 --2- 172.21.15.97:0/3146124288 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622a13d1090 0x5622a143f4b0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f41900145e0 tx=0x7f41900117d0 comp rx=0 tx=0).stop 2024-04-23T10:26:36.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.365+0000 7f41a2650740 1 -- 172.21.15.97:0/3146124288 shutdown_connections 2024-04-23T10:26:36.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.365+0000 7f41a2650740 1 --2- 172.21.15.97:0/3146124288 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5622a144cc50 0x5622a144f070 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:36.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.365+0000 7f41a2650740 1 --2- 172.21.15.97:0/3146124288 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622a13d1090 0x5622a143f4b0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:36.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.365+0000 7f41a2650740 1 --2- 172.21.15.97:0/3146124288 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622a13c69c0 0x5622a1445340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:36.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.365+0000 7f41a2650740 1 --2- 172.21.15.97:0/3146124288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622a13c60a0 0x5622a1444e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:36.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.365+0000 7f4180ff9640 0 client.15726 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:26:36.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.365+0000 7f41a2650740 1 -- 172.21.15.97:0/3146124288 >> 172.21.15.97:0/3146124288 conn(0x5622a13c23c0 msgr2=0x5622a13cf880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:36.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.365+0000 7f41a2650740 1 -- 172.21.15.97:0/3146124288 shutdown_connections 2024-04-23T10:26:36.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.365+0000 7f41a2650740 1 -- 172.21.15.97:0/3146124288 wait complete. 2024-04-23T10:26:36.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.365+0000 7f41a2650740 20 client.15726 trim_cache size 0 max 16384 2024-04-23T10:26:36.365 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-23T10:26:36.365 INFO:teuthology.orchestra.run.smithi097.stdout: "epoch": 26, 2024-04-23T10:26:36.365 INFO:teuthology.orchestra.run.smithi097.stdout: "leader": 24488, 2024-04-23T10:26:36.365 INFO:teuthology.orchestra.run.smithi097.stdout: "set_version": 2, 2024-04-23T10:26:36.365 INFO:teuthology.orchestra.run.smithi097.stdout: "sets": { 2024-04-23T10:26:36.365 INFO:teuthology.orchestra.run.smithi097.stdout: "55d3c626": { 2024-04-23T10:26:36.365 INFO:teuthology.orchestra.run.smithi097.stdout: "version": 2, 2024-04-23T10:26:36.366 INFO:teuthology.orchestra.run.smithi097.stdout: "age_ref": 0.0, 2024-04-23T10:26:36.366 INFO:teuthology.orchestra.run.smithi097.stdout: "state": { 2024-04-23T10:26:36.366 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "QUIESCED", 2024-04-23T10:26:36.366 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0 2024-04-23T10:26:36.366 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:26:36.366 INFO:teuthology.orchestra.run.smithi097.stdout: "timeout": 60.0, 2024-04-23T10:26:36.366 INFO:teuthology.orchestra.run.smithi097.stdout: "expiration": 82.9, 2024-04-23T10:26:36.366 INFO:teuthology.orchestra.run.smithi097.stdout: "members": { 2024-04-23T10:26:36.366 INFO:teuthology.orchestra.run.smithi097.stdout: "file:/": { 2024-04-23T10:26:36.366 INFO:teuthology.orchestra.run.smithi097.stdout: "excluded": false, 2024-04-23T10:26:36.366 INFO:teuthology.orchestra.run.smithi097.stdout: "state": { 2024-04-23T10:26:36.366 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "QUIESCED", 2024-04-23T10:26:36.366 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0 2024-04-23T10:26:36.366 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:26:36.366 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:26:36.366 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:26:36.367 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:26:36.367 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:26:36.367 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-23T10:26:36.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.365+0000 7f419f937640 1 -- 172.21.15.97:0/781276726 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f41840a3ec0 msgr2=0x7f41840a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:36.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.365+0000 7f419f937640 1 --2- 172.21.15.97:0/781276726 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f41840a3ec0 0x7f41840a6380 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4190014ad0 tx=0x7f4190009db0 comp rx=0 tx=0).stop 2024-04-23T10:26:36.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.365+0000 7f419f937640 1 -- 172.21.15.97:0/781276726 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4198106190 msgr2=0x7f419819e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:36.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.365+0000 7f419f937640 1 --2- 172.21.15.97:0/781276726 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4198106190 0x7f419819e930 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f4188032fd0 tx=0x7f4188004270 comp rx=0 tx=0).stop 2024-04-23T10:26:36.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.366+0000 7f419f937640 1 -- 172.21.15.97:0/781276726 shutdown_connections 2024-04-23T10:26:36.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.366+0000 7f419f937640 1 --2- 172.21.15.97:0/781276726 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f41840a3ec0 0x7f41840a6380 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:36.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.366+0000 7f419f937640 1 --2- 172.21.15.97:0/781276726 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f41981985a0 0x7f4198198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:36.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.366+0000 7f419f937640 1 --2- 172.21.15.97:0/781276726 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4198106190 0x7f419819e930 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:36.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.366+0000 7f419f937640 1 --2- 172.21.15.97:0/781276726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4198101110 0x7f419819e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:36.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.366+0000 7f419f937640 1 -- 172.21.15.97:0/781276726 >> 172.21.15.97:0/781276726 conn(0x7f4198077fb0 msgr2=0x7f4198104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:36.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.366+0000 7f419f937640 1 -- 172.21.15.97:0/781276726 shutdown_connections 2024-04-23T10:26:36.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:36.366+0000 7f419f937640 1 -- 172.21.15.97:0/781276726 wait complete. 2024-04-23T10:26:36.378 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '55d3c626', quiesce took 0.5 seconds. Will release after: 22.9 2024-04-23T10:26:37.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:36 smithi196 ceph-mon[32919]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-04-23T10:26:37.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:36 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/3072384383' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:26:37.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:36 smithi097 ceph-mon[29197]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-04-23T10:26:37.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:36 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/3072384383' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:26:37.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:36 smithi183 ceph-mon[32317]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-04-23T10:26:37.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:36 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/3072384383' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:26:39.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:38 smithi196 ceph-mon[32919]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-04-23T10:26:39.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:38 smithi097 ceph-mon[29197]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-04-23T10:26:39.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:38 smithi183 ceph-mon[32317]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-04-23T10:26:40.084 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.084+0000 7f830fa8a640 1 -- 172.21.15.97:0/1372411752 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8308103200 msgr2=0x7f83081035e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:40.084 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.084+0000 7f830fa8a640 1 --2- 172.21.15.97:0/1372411752 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8308103200 0x7f83081035e0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f8304009900 tx=0x7f830402a480 comp rx=0 tx=0).stop 2024-04-23T10:26:40.084 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.085+0000 7f830fa8a640 1 -- 172.21.15.97:0/1372411752 shutdown_connections 2024-04-23T10:26:40.084 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.085+0000 7f830fa8a640 1 --2- 172.21.15.97:0/1372411752 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f830810c540 0x7f830810e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:40.085 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.085+0000 7f830fa8a640 1 --2- 172.21.15.97:0/1372411752 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8308103b20 0x7f830810c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:40.085 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.085+0000 7f830fa8a640 1 --2- 172.21.15.97:0/1372411752 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8308103200 0x7f83081035e0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:40.085 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.085+0000 7f830fa8a640 1 -- 172.21.15.97:0/1372411752 >> 172.21.15.97:0/1372411752 conn(0x7f8308077fd0 msgr2=0x7f83080783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:40.085 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.085+0000 7f830fa8a640 1 -- 172.21.15.97:0/1372411752 shutdown_connections 2024-04-23T10:26:40.085 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.085+0000 7f830fa8a640 1 -- 172.21.15.97:0/1372411752 wait complete. 2024-04-23T10:26:40.085 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.085+0000 7f830fa8a640 1 Processor -- start 2024-04-23T10:26:40.085 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.085+0000 7f830fa8a640 1 -- start start 2024-04-23T10:26:40.087 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.086+0000 7f830fa8a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8308103200 0x7f8308198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:40.087 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.086+0000 7f830fa8a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8308103b20 0x7f8308198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:40.087 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.086+0000 7f830fa8a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f830810c540 0x7f830819cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:40.087 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.086+0000 7f830fa8a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f83081121d0 con 0x7f830810c540 2024-04-23T10:26:40.087 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.086+0000 7f830fa8a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8308112050 con 0x7f8308103b20 2024-04-23T10:26:40.087 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.086+0000 7f830fa8a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8308112350 con 0x7f8308103200 2024-04-23T10:26:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.086+0000 7f830e027640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f830810c540 0x7f830819cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.086+0000 7f830e027640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f830810c540 0x7f830819cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48684/0 (socket says 172.21.15.97:48684) 2024-04-23T10:26:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.086+0000 7f830e027640 1 -- 172.21.15.97:0/3169117921 learned_addr learned my addr 172.21.15.97:0/3169117921 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.086+0000 7f830d826640 1 --2- 172.21.15.97:0/3169117921 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8308103200 0x7f8308198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.086+0000 7f830d025640 1 --2- 172.21.15.97:0/3169117921 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8308103b20 0x7f8308198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.086+0000 7f830e027640 1 -- 172.21.15.97:0/3169117921 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8308103200 msgr2=0x7f8308198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.086+0000 7f830e027640 1 --2- 172.21.15.97:0/3169117921 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8308103200 0x7f8308198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.086+0000 7f830e027640 1 -- 172.21.15.97:0/3169117921 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8308103b20 msgr2=0x7f8308198b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.086+0000 7f830e027640 1 --2- 172.21.15.97:0/3169117921 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8308103b20 0x7f8308198b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.086+0000 7f830e027640 1 -- 172.21.15.97:0/3169117921 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f830819d640 con 0x7f830810c540 2024-04-23T10:26:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.086+0000 7f830d826640 1 --2- 172.21.15.97:0/3169117921 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8308103200 0x7f8308198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T10:26:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.086+0000 7f830d025640 1 --2- 172.21.15.97:0/3169117921 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8308103b20 0x7f8308198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T10:26:40.089 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.087+0000 7f830e027640 1 --2- 172.21.15.97:0/3169117921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f830810c540 0x7f830819cf10 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f830000cce0 tx=0x7f830000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:40.089 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.087+0000 7f82feffd640 1 -- 172.21.15.97:0/3169117921 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830000f450 con 0x7f830810c540 2024-04-23T10:26:40.089 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.087+0000 7f82feffd640 1 -- 172.21.15.97:0/3169117921 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f830000faa0 con 0x7f830810c540 2024-04-23T10:26:40.089 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.087+0000 7f830fa8a640 1 -- 172.21.15.97:0/3169117921 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f830819d930 con 0x7f830810c540 2024-04-23T10:26:40.089 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.087+0000 7f82feffd640 1 -- 172.21.15.97:0/3169117921 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8300018b70 con 0x7f830810c540 2024-04-23T10:26:40.089 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.087+0000 7f830fa8a640 1 -- 172.21.15.97:0/3169117921 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83081a51c0 con 0x7f830810c540 2024-04-23T10:26:40.089 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.090+0000 7f82feffd640 1 -- 172.21.15.97:0/3169117921 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f8300018d10 con 0x7f830810c540 2024-04-23T10:26:40.090 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.090+0000 7f83127a4740 1 Processor -- start 2024-04-23T10:26:40.092 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.091+0000 7f82feffd640 1 --2- 172.21.15.97:0/3169117921 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f82f40a3fe0 0x7f82f40a64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:40.092 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.091+0000 7f82feffd640 1 -- 172.21.15.97:0/3169117921 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f83000d0da0 con 0x7f830810c540 2024-04-23T10:26:40.092 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.091+0000 7f83127a4740 1 -- start start 2024-04-23T10:26:40.092 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.091+0000 7f830d025640 1 --2- 172.21.15.97:0/3169117921 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f82f40a3fe0 0x7f82f40a64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:40.092 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.092+0000 7f83127a4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d78a70b60 0x560d78a6f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:40.092 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.092+0000 7f83127a4740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d78a6fa20 0x560d78a6fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:40.092 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.092+0000 7f83127a4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d78a82ea0 0x560d78a85290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:40.092 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.092+0000 7f83127a4740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560d789d4970 con 0x560d78a6fa20 2024-04-23T10:26:40.093 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.092+0000 7f83127a4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560d789714a0 con 0x560d78a70b60 2024-04-23T10:26:40.093 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.092+0000 7f83127a4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560d789d0930 con 0x560d78a82ea0 2024-04-23T10:26:40.093 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.092+0000 7f830e027640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d78a82ea0 0x560d78a85290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:40.093 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.092+0000 7f830e027640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d78a82ea0 0x560d78a85290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.97:47614/0 (socket says 172.21.15.97:47614) 2024-04-23T10:26:40.093 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.092+0000 7f830e027640 1 -- 172.21.15.97:0/441044550 learned_addr learned my addr 172.21.15.97:0/441044550 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:40.093 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.092+0000 7f830d826640 1 --2- 172.21.15.97:0/441044550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d78a70b60 0x560d78a6f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:40.093 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.092+0000 7f830d025640 1 --2- 172.21.15.97:0/3169117921 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f82f40a3fe0 0x7f82f40a64a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8308100830 tx=0x7f82f800b410 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:40.093 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.092+0000 7f830d025640 1 --2- 172.21.15.97:0/441044550 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d78a6fa20 0x560d78a6fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:40.093 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.092+0000 7f830e027640 1 -- 172.21.15.97:0/441044550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d78a70b60 msgr2=0x560d78a6f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:40.093 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.092+0000 7f830e027640 1 --2- 172.21.15.97:0/441044550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d78a70b60 0x560d78a6f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:40.093 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.092+0000 7f830e027640 1 -- 172.21.15.97:0/441044550 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d78a6fa20 msgr2=0x560d78a6fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:40.094 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.092+0000 7f830e027640 1 --2- 172.21.15.97:0/441044550 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d78a6fa20 0x560d78a6fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:40.094 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.092+0000 7f830e027640 1 -- 172.21.15.97:0/441044550 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d786e7210 con 0x560d78a82ea0 2024-04-23T10:26:40.094 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.092+0000 7f830d025640 1 --2- 172.21.15.97:0/441044550 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d78a6fa20 0x560d78a6fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:26:40.094 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.092+0000 7f830e027640 1 --2- 172.21.15.97:0/441044550 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d78a82ea0 0x560d78a85290 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f830001f000 tx=0x7f830009c440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=676efdbbb1edfd0a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:40.094 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.093+0000 7f82f3fff640 1 -- 172.21.15.97:0/441044550 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83000a0e90 con 0x560d78a82ea0 2024-04-23T10:26:40.094 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.093+0000 7f82f3fff640 1 -- 172.21.15.97:0/441044550 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83000a5750 con 0x560d78a82ea0 2024-04-23T10:26:40.094 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.093+0000 7f82f3fff640 1 -- 172.21.15.97:0/441044550 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83000a5540 con 0x560d78a82ea0 2024-04-23T10:26:40.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.094+0000 7f83127a4740 1 -- 172.21.15.97:0/441044550 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d78a82ea0 msgr2=0x560d78a85290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:40.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.094+0000 7f83127a4740 1 --2- 172.21.15.97:0/441044550 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d78a82ea0 0x560d78a85290 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f830001f000 tx=0x7f830009c440 comp rx=0 tx=0).stop 2024-04-23T10:26:40.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.094+0000 7f83127a4740 1 -- 172.21.15.97:0/441044550 shutdown_connections 2024-04-23T10:26:40.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.094+0000 7f83127a4740 1 --2- 172.21.15.97:0/441044550 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d78a82ea0 0x560d78a85290 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:40.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.094+0000 7f83127a4740 1 --2- 172.21.15.97:0/441044550 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d78a6fa20 0x560d78a6fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:40.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.094+0000 7f83127a4740 1 --2- 172.21.15.97:0/441044550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d78a70b60 0x560d78a6f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:40.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.094+0000 7f83127a4740 1 -- 172.21.15.97:0/441044550 >> 172.21.15.97:0/441044550 conn(0x560d78a6bb30 msgr2=0x560d78a6bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:40.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.094+0000 7f83127a4740 1 -- 172.21.15.97:0/441044550 shutdown_connections 2024-04-23T10:26:40.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.094+0000 7f83127a4740 1 -- 172.21.15.97:0/441044550 wait complete. 2024-04-23T10:26:40.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.094+0000 7f83127a4740 1 -- start start 2024-04-23T10:26:40.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.094+0000 7f83127a4740 1 Processor -- start 2024-04-23T10:26:40.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.095+0000 7f83127a4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d78a6fa20 0x560d78a75c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:40.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.095+0000 7f83127a4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d78a70b60 0x560d78a72f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:40.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.095+0000 7f83127a4740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d78a82ea0 0x560d78a734d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:40.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.095+0000 7f83127a4740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560d78a85bf0 con 0x560d78a82ea0 2024-04-23T10:26:40.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.095+0000 7f83127a4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560d7896f0b0 con 0x560d78a6fa20 2024-04-23T10:26:40.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.095+0000 7f83127a4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560d78a85d70 con 0x560d78a70b60 2024-04-23T10:26:40.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.095+0000 7f83127a4740 10 client.0 _ll_register_callbacks cb 0x560d7895cd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:26:40.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.095+0000 7f830e027640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d78a82ea0 0x560d78a734d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:40.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.095+0000 7f830e027640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d78a82ea0 0x560d78a734d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48690/0 (socket says 172.21.15.97:48690) 2024-04-23T10:26:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.095+0000 7f830e027640 1 -- 172.21.15.97:0/1083761014 learned_addr learned my addr 172.21.15.97:0/1083761014 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.095+0000 7f830d826640 1 --2- 172.21.15.97:0/1083761014 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d78a6fa20 0x560d78a75c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.095+0000 7f830d025640 1 --2- 172.21.15.97:0/1083761014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d78a70b60 0x560d78a72f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.096+0000 7f830e027640 1 -- 172.21.15.97:0/1083761014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d78a70b60 msgr2=0x560d78a72f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.096+0000 7f830e027640 1 --2- 172.21.15.97:0/1083761014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d78a70b60 0x560d78a72f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.096+0000 7f830e027640 1 -- 172.21.15.97:0/1083761014 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d78a6fa20 msgr2=0x560d78a75c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.096+0000 7f830e027640 1 --2- 172.21.15.97:0/1083761014 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d78a6fa20 0x560d78a75c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.096+0000 7f830e027640 1 -- 172.21.15.97:0/1083761014 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d787b3620 con 0x560d78a82ea0 2024-04-23T10:26:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.096+0000 7f830d826640 1 --2- 172.21.15.97:0/1083761014 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d78a6fa20 0x560d78a75c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T10:26:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.096+0000 7f830e027640 1 --2- 172.21.15.97:0/1083761014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d78a82ea0 0x560d78a734d0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f830009ca70 tx=0x7f830000f9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3e3d9eaf862f43b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.096+0000 7f82f17fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:26:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.096+0000 7f82f17fa640 1 -- 172.21.15.97:0/1083761014 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83000b6040 con 0x560d78a82ea0 2024-04-23T10:26:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.096+0000 7f83127a4740 1 -- 172.21.15.97:0/1083761014 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83000a5750 con 0x560d78a82ea0 2024-04-23T10:26:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.096+0000 7f82f17fa640 1 -- 172.21.15.97:0/1083761014 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f8300099150 con 0x560d78a82ea0 2024-04-23T10:26:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.096+0000 7f82f17fa640 1 -- 172.21.15.97:0/1083761014 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f83000a0e90 con 0x560d78a82ea0 2024-04-23T10:26:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.096+0000 7f83127a4740 10 client.15735 fetch_fsmap learned FSMap version 26 2024-04-23T10:26:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.096+0000 7f83127a4740 1 -- 172.21.15.97:0/1083761014 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560d78a91b10 con 0x560d78a82ea0 2024-04-23T10:26:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.097+0000 7f82f17fa640 1 -- 172.21.15.97:0/1083761014 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83000a5750 con 0x560d78a82ea0 2024-04-23T10:26:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.097+0000 7f82f17fa640 1 -- 172.21.15.97:0/1083761014 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f83000988c0 con 0x560d78a82ea0 2024-04-23T10:26:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.097+0000 7f83127a4740 10 client.15735 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:26:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.097+0000 7f83127a4740 10 client.15735 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:26:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.097+0000 7f83127a4740 20 client.15735 populate_metadata read hostname 'smithi097' 2024-04-23T10:26:40.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.097+0000 7f83127a4740 1 --2- 172.21.15.97:0/1083761014 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x560d78a931a0 0x560d78a955d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:40.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.097+0000 7f83127a4740 4 client.15735 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:26:40.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.097+0000 7f83127a4740 1 -- 172.21.15.97:0/1083761014 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d789e2390 con 0x560d78a931a0 2024-04-23T10:26:40.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.097+0000 7f830d826640 1 --2- 172.21.15.97:0/1083761014 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x560d78a931a0 0x560d78a955d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:40.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.098+0000 7f830d826640 1 --2- 172.21.15.97:0/1083761014 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x560d78a931a0 0x560d78a955d0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84117ba59f55c15a server_cookie=e528cc669e674653 in_seq=0 out_seq=0 2024-04-23T10:26:40.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.098+0000 7f82f17fa640 10 client.15735 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:26:40.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.098+0000 7f82f17fa640 1 -- 172.21.15.97:0/1083761014 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560d789e2390 con 0x560d78a931a0 2024-04-23T10:26:40.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.098+0000 7f82f17fa640 10 client.15735 handle_command_reply: tid=0 2024-04-23T10:26:40.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.098+0000 7f83127a4740 1 client.15735 shutdown 2024-04-23T10:26:40.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.099+0000 7f83127a4740 1 -- 172.21.15.97:0/1083761014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d78a82ea0 msgr2=0x560d78a734d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:40.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.099+0000 7f83127a4740 1 --2- 172.21.15.97:0/1083761014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d78a82ea0 0x560d78a734d0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f830009ca70 tx=0x7f830000f9e0 comp rx=0 tx=0).stop 2024-04-23T10:26:40.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.099+0000 7f83127a4740 1 -- 172.21.15.97:0/1083761014 shutdown_connections 2024-04-23T10:26:40.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.099+0000 7f83127a4740 1 --2- 172.21.15.97:0/1083761014 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x560d78a931a0 0x560d78a955d0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:40.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.099+0000 7f83127a4740 1 --2- 172.21.15.97:0/1083761014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d78a82ea0 0x560d78a734d0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:40.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.099+0000 7f83127a4740 1 --2- 172.21.15.97:0/1083761014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d78a70b60 0x560d78a72f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:40.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.099+0000 7f83127a4740 1 --2- 172.21.15.97:0/1083761014 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d78a6fa20 0x560d78a75c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:40.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.099+0000 7f83127a4740 1 -- 172.21.15.97:0/1083761014 >> 172.21.15.97:0/1083761014 conn(0x560d78a6bb30 msgr2=0x560d78a71980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:40.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.099+0000 7f82f17fa640 0 client.15735 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:26:40.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.099+0000 7f83127a4740 1 -- 172.21.15.97:0/1083761014 shutdown_connections 2024-04-23T10:26:40.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.099+0000 7f83127a4740 1 -- 172.21.15.97:0/1083761014 wait complete. 2024-04-23T10:26:40.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.099+0000 7f83127a4740 20 client.15735 trim_cache size 0 max 16384 2024-04-23T10:26:40.127 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.127+0000 7f83127a4740 1 Processor -- start 2024-04-23T10:26:40.127 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.127+0000 7f83127a4740 1 -- start start 2024-04-23T10:26:40.127 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.127+0000 7f83127a4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d78a6fa20 0x560d78a8eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:40.128 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.127+0000 7f83127a4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d78a70b60 0x560d78a757b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:40.128 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.127+0000 7f83127a4740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d78a82ea0 0x560d78a9aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:40.128 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.127+0000 7f83127a4740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f83081ab740 con 0x560d78a82ea0 2024-04-23T10:26:40.128 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.127+0000 7f83127a4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f83081ad280 con 0x560d78a6fa20 2024-04-23T10:26:40.128 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.127+0000 7f83127a4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f83081abe10 con 0x560d78a70b60 2024-04-23T10:26:40.128 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.127+0000 7f830e027640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d78a82ea0 0x560d78a9aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:40.128 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.127+0000 7f830e027640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d78a82ea0 0x560d78a9aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48704/0 (socket says 172.21.15.97:48704) 2024-04-23T10:26:40.128 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.128+0000 7f830e027640 1 -- 172.21.15.97:0/2719107474 learned_addr learned my addr 172.21.15.97:0/2719107474 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:40.128 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.128+0000 7f830d826640 1 --2- 172.21.15.97:0/2719107474 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d78a6fa20 0x560d78a8eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:40.128 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.128+0000 7f830d025640 1 --2- 172.21.15.97:0/2719107474 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d78a70b60 0x560d78a757b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:40.128 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.128+0000 7f830e027640 1 -- 172.21.15.97:0/2719107474 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d78a70b60 msgr2=0x560d78a757b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:40.128 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.128+0000 7f830e027640 1 --2- 172.21.15.97:0/2719107474 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d78a70b60 0x560d78a757b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:40.129 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.128+0000 7f830e027640 1 -- 172.21.15.97:0/2719107474 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d78a6fa20 msgr2=0x560d78a8eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:40.129 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.128+0000 7f830e027640 1 --2- 172.21.15.97:0/2719107474 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d78a6fa20 0x560d78a8eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:40.129 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.128+0000 7f830e027640 1 -- 172.21.15.97:0/2719107474 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d78a672e0 con 0x560d78a82ea0 2024-04-23T10:26:40.129 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.128+0000 7f830d826640 1 --2- 172.21.15.97:0/2719107474 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d78a6fa20 0x560d78a8eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T10:26:40.129 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.128+0000 7f830e027640 1 --2- 172.21.15.97:0/2719107474 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d78a82ea0 0x560d78a9aee0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f830000ccb0 tx=0x7f830009aff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b514672d48b4e296 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:40.129 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.128+0000 7f82f37fe640 1 -- 172.21.15.97:0/2719107474 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83000a5750 con 0x560d78a82ea0 2024-04-23T10:26:40.129 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.128+0000 7f82f37fe640 1 -- 172.21.15.97:0/2719107474 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83000ce2a0 con 0x560d78a82ea0 2024-04-23T10:26:40.129 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.128+0000 7f82f37fe640 1 -- 172.21.15.97:0/2719107474 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8300098a90 con 0x560d78a82ea0 2024-04-23T10:26:40.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.129+0000 7f83127a4740 1 -- 172.21.15.97:0/2719107474 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d78a82ea0 msgr2=0x560d78a9aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:40.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.129+0000 7f83127a4740 1 --2- 172.21.15.97:0/2719107474 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d78a82ea0 0x560d78a9aee0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f830000ccb0 tx=0x7f830009aff0 comp rx=0 tx=0).stop 2024-04-23T10:26:40.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.129+0000 7f83127a4740 1 -- 172.21.15.97:0/2719107474 shutdown_connections 2024-04-23T10:26:40.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.129+0000 7f83127a4740 1 --2- 172.21.15.97:0/2719107474 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d78a82ea0 0x560d78a9aee0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:40.134 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.129+0000 7f83127a4740 1 --2- 172.21.15.97:0/2719107474 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d78a70b60 0x560d78a757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:40.134 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.129+0000 7f83127a4740 1 --2- 172.21.15.97:0/2719107474 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d78a6fa20 0x560d78a8eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:40.134 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.129+0000 7f83127a4740 1 -- 172.21.15.97:0/2719107474 >> 172.21.15.97:0/2719107474 conn(0x560d78a6bb30 msgr2=0x560d78a7ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:40.134 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.129+0000 7f83127a4740 1 -- 172.21.15.97:0/2719107474 shutdown_connections 2024-04-23T10:26:40.134 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.129+0000 7f83127a4740 1 -- 172.21.15.97:0/2719107474 wait complete. 2024-04-23T10:26:40.134 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.129+0000 7f83127a4740 1 -- start start 2024-04-23T10:26:40.134 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.130+0000 7f83127a4740 1 Processor -- start 2024-04-23T10:26:40.134 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.130+0000 7f83127a4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d78a6fa20 0x560d78a8b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:40.134 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.130+0000 7f83127a4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d78a70b60 0x560d78a81240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:40.134 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.130+0000 7f83127a4740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d78a82ea0 0x560d78a81850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:40.134 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.130+0000 7f83127a4740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f83081acbb0 con 0x560d78a82ea0 2024-04-23T10:26:40.134 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.130+0000 7f83127a4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f83081ac4e0 con 0x560d78a6fa20 2024-04-23T10:26:40.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.130+0000 7f83127a4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f83081a9720 con 0x560d78a70b60 2024-04-23T10:26:40.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.130+0000 7f83127a4740 10 client.0 _ll_register_callbacks cb 0x7f830009c940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:26:40.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.130+0000 7f830e027640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d78a82ea0 0x560d78a81850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:40.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.130+0000 7f830e027640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d78a82ea0 0x560d78a81850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48710/0 (socket says 172.21.15.97:48710) 2024-04-23T10:26:40.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.130+0000 7f830e027640 1 -- 172.21.15.97:0/2768798214 learned_addr learned my addr 172.21.15.97:0/2768798214 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:40.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.130+0000 7f830d025640 1 --2- 172.21.15.97:0/2768798214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d78a70b60 0x560d78a81240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:40.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.130+0000 7f830d826640 1 --2- 172.21.15.97:0/2768798214 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d78a6fa20 0x560d78a8b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:40.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.131+0000 7f830e027640 1 -- 172.21.15.97:0/2768798214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d78a70b60 msgr2=0x560d78a81240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:40.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.131+0000 7f830e027640 1 --2- 172.21.15.97:0/2768798214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d78a70b60 0x560d78a81240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:40.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.131+0000 7f830e027640 1 -- 172.21.15.97:0/2768798214 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d78a6fa20 msgr2=0x560d78a8b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:40.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.131+0000 7f830e027640 1 --2- 172.21.15.97:0/2768798214 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d78a6fa20 0x560d78a8b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:40.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.131+0000 7f830e027640 1 -- 172.21.15.97:0/2768798214 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d78a800e0 con 0x560d78a82ea0 2024-04-23T10:26:40.136 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.131+0000 7f830d826640 1 --2- 172.21.15.97:0/2768798214 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d78a6fa20 0x560d78a8b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:26:40.136 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.131+0000 7f830e027640 1 --2- 172.21.15.97:0/2768798214 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d78a82ea0 0x560d78a81850 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f830009c7e0 tx=0x7f830009bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=549feeb2cd61f625 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:40.136 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.131+0000 7f82f17fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:26:40.136 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.131+0000 7f82f17fa640 1 -- 172.21.15.97:0/2768798214 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83000b6030 con 0x560d78a82ea0 2024-04-23T10:26:40.136 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.131+0000 7f82f17fa640 1 -- 172.21.15.97:0/2768798214 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f830009bf30 con 0x560d78a82ea0 2024-04-23T10:26:40.136 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.131+0000 7f83127a4740 1 -- 172.21.15.97:0/2768798214 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83000ce2a0 con 0x560d78a82ea0 2024-04-23T10:26:40.136 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.131+0000 7f82f17fa640 1 -- 172.21.15.97:0/2768798214 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f830009b9a0 con 0x560d78a82ea0 2024-04-23T10:26:40.136 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.131+0000 7f83127a4740 10 client.15741 fetch_fsmap learned FSMap version 26 2024-04-23T10:26:40.136 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.131+0000 7f83127a4740 1 -- 172.21.15.97:0/2768798214 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560d78a800e0 con 0x560d78a82ea0 2024-04-23T10:26:40.136 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.131+0000 7f82f17fa640 1 -- 172.21.15.97:0/2768798214 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83000a5750 con 0x560d78a82ea0 2024-04-23T10:26:40.136 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.132+0000 7f82f17fa640 1 -- 172.21.15.97:0/2768798214 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8300004a40 con 0x560d78a82ea0 2024-04-23T10:26:40.136 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.132+0000 7f83127a4740 10 client.15741 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:26:40.136 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.132+0000 7f83127a4740 10 client.15741 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:26:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.132+0000 7f83127a4740 20 client.15741 populate_metadata read hostname 'smithi097' 2024-04-23T10:26:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.132+0000 7f83127a4740 1 --2- 172.21.15.97:0/2768798214 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x560d78af8f40 0x560d78afb330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.132+0000 7f83127a4740 4 client.15741 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:26:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.132+0000 7f83127a4740 1 -- 172.21.15.97:0/2768798214 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f83000ce2a0 con 0x560d78af8f40 2024-04-23T10:26:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.132+0000 7f830d826640 1 --2- 172.21.15.97:0/2768798214 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x560d78af8f40 0x560d78afb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.133+0000 7f82f17fa640 10 client.15741 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:26:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.133+0000 7f830d826640 1 --2- 172.21.15.97:0/2768798214 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x560d78af8f40 0x560d78afb330 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fd76b9e50599668 server_cookie=d39bb192d6e2f70a in_seq=0 out_seq=0 2024-04-23T10:26:40.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.140+0000 7f82f17fa640 1 -- 172.21.15.97:0/2768798214 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f83000ce2a0 con 0x560d78af8f40 2024-04-23T10:26:40.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.140+0000 7f82f17fa640 10 client.15741 handle_command_reply: tid=0 2024-04-23T10:26:40.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.140+0000 7f83127a4740 1 client.15741 shutdown 2024-04-23T10:26:40.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.140+0000 7f83127a4740 1 -- 172.21.15.97:0/2768798214 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d78a82ea0 msgr2=0x560d78a81850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:40.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.140+0000 7f83127a4740 1 --2- 172.21.15.97:0/2768798214 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d78a82ea0 0x560d78a81850 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f830009c7e0 tx=0x7f830009bce0 comp rx=0 tx=0).stop 2024-04-23T10:26:40.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.140+0000 7f83127a4740 1 -- 172.21.15.97:0/2768798214 shutdown_connections 2024-04-23T10:26:40.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.140+0000 7f83127a4740 1 --2- 172.21.15.97:0/2768798214 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x560d78af8f40 0x560d78afb330 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:40.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.140+0000 7f83127a4740 1 --2- 172.21.15.97:0/2768798214 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d78a82ea0 0x560d78a81850 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:40.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.140+0000 7f83127a4740 1 --2- 172.21.15.97:0/2768798214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d78a70b60 0x560d78a81240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:40.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.140+0000 7f82f17fa640 0 client.15741 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:26:40.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.140+0000 7f83127a4740 1 --2- 172.21.15.97:0/2768798214 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d78a6fa20 0x560d78a8b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:40.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.140+0000 7f83127a4740 1 -- 172.21.15.97:0/2768798214 >> 172.21.15.97:0/2768798214 conn(0x560d78a6bb30 msgr2=0x560d78a710c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:40.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.140+0000 7f83127a4740 1 -- 172.21.15.97:0/2768798214 shutdown_connections 2024-04-23T10:26:40.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.141+0000 7f83127a4740 1 -- 172.21.15.97:0/2768798214 wait complete. 2024-04-23T10:26:40.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.141+0000 7f83127a4740 20 client.15741 trim_cache size 0 max 16384 2024-04-23T10:26:40.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:26:40.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:26:40.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:26:40.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:26:40.142 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.141+0000 7f830fa8a640 1 -- 172.21.15.97:0/3169117921 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f82f40a3fe0 msgr2=0x7f82f40a64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:40.142 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.141+0000 7f830fa8a640 1 --2- 172.21.15.97:0/3169117921 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f82f40a3fe0 0x7f82f40a64a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8308100830 tx=0x7f82f800b410 comp rx=0 tx=0).stop 2024-04-23T10:26:40.142 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.141+0000 7f830fa8a640 1 -- 172.21.15.97:0/3169117921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f830810c540 msgr2=0x7f830819cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:40.143 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.141+0000 7f830fa8a640 1 --2- 172.21.15.97:0/3169117921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f830810c540 0x7f830819cf10 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f830000cce0 tx=0x7f830000a620 comp rx=0 tx=0).stop 2024-04-23T10:26:40.143 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.141+0000 7f830fa8a640 1 -- 172.21.15.97:0/3169117921 shutdown_connections 2024-04-23T10:26:40.143 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.141+0000 7f830fa8a640 1 --2- 172.21.15.97:0/3169117921 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f82f40a3fe0 0x7f82f40a64a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:40.143 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.141+0000 7f830fa8a640 1 --2- 172.21.15.97:0/3169117921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f830810c540 0x7f830819cf10 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:40.143 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.141+0000 7f830fa8a640 1 --2- 172.21.15.97:0/3169117921 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8308103b20 0x7f8308198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:40.143 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.141+0000 7f830fa8a640 1 --2- 172.21.15.97:0/3169117921 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8308103200 0x7f8308198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:40.143 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.141+0000 7f830fa8a640 1 -- 172.21.15.97:0/3169117921 >> 172.21.15.97:0/3169117921 conn(0x7f8308077fd0 msgr2=0x7f83080fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:40.143 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.141+0000 7f830fa8a640 1 -- 172.21.15.97:0/3169117921 shutdown_connections 2024-04-23T10:26:40.143 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:40.141+0000 7f830fa8a640 1 -- 172.21.15.97:0/3169117921 wait complete. 2024-04-23T10:26:41.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:40 smithi196 ceph-mon[32919]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.3 KiB/s wr, 13 op/s 2024-04-23T10:26:41.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:40 smithi097 ceph-mon[29197]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.3 KiB/s wr, 13 op/s 2024-04-23T10:26:41.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:40 smithi183 ceph-mon[32317]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.3 KiB/s wr, 13 op/s 2024-04-23T10:26:43.202 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:42 smithi183 ceph-mon[32317]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-23T10:26:43.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:42 smithi196 ceph-mon[32919]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-23T10:26:43.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:42 smithi097 ceph-mon[29197]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-23T10:26:45.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:44 smithi196 ceph-mon[32919]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-04-23T10:26:45.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:44 smithi097 ceph-mon[29197]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-04-23T10:26:45.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:44 smithi183 ceph-mon[32317]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-04-23T10:26:47.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:46 smithi196 ceph-mon[32919]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-23T10:26:47.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:46 smithi097 ceph-mon[29197]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-23T10:26:47.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:46 smithi183 ceph-mon[32317]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-23T10:26:49.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:48 smithi196 ceph-mon[32919]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.3 KiB/s wr, 14 op/s 2024-04-23T10:26:49.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:48 smithi097 ceph-mon[29197]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.3 KiB/s wr, 14 op/s 2024-04-23T10:26:49.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:48 smithi183 ceph-mon[32317]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.3 KiB/s wr, 14 op/s 2024-04-23T10:26:51.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:50 smithi196 ceph-mon[32919]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.3 KiB/s wr, 13 op/s 2024-04-23T10:26:51.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:50 smithi097 ceph-mon[29197]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.3 KiB/s wr, 13 op/s 2024-04-23T10:26:51.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:50 smithi183 ceph-mon[32317]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.3 KiB/s wr, 13 op/s 2024-04-23T10:26:53.263 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.263+0000 7fcb0019b640 1 -- 172.21.15.97:0/3229458560 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcaf81072a0 msgr2=0x7fcaf810ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:53.263 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.263+0000 7fcb0019b640 1 --2- 172.21.15.97:0/3229458560 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcaf81072a0 0x7fcaf810ee70 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0009ca0 tx=0x7fcaf002a8c0 comp rx=0 tx=0).stop 2024-04-23T10:26:53.263 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.263+0000 7fcb0019b640 1 -- 172.21.15.97:0/3229458560 shutdown_connections 2024-04-23T10:26:53.263 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.263+0000 7fcb0019b640 1 --2- 172.21.15.97:0/3229458560 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcaf81072a0 0x7fcaf810ee70 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:53.263 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.263+0000 7fcb0019b640 1 --2- 172.21.15.97:0/3229458560 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcaf8102b80 0x7fcaf8106bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:53.263 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.263+0000 7fcb0019b640 1 --2- 172.21.15.97:0/3229458560 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcaf81021d0 0x7fcaf81025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:53.263 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.263+0000 7fcb0019b640 1 -- 172.21.15.97:0/3229458560 >> 172.21.15.97:0/3229458560 conn(0x7fcaf80fda50 msgr2=0x7fcaf80ffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:53.263 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.264+0000 7fcb0019b640 1 -- 172.21.15.97:0/3229458560 shutdown_connections 2024-04-23T10:26:53.263 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.264+0000 7fcb0019b640 1 -- 172.21.15.97:0/3229458560 wait complete. 2024-04-23T10:26:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.264+0000 7fcb0019b640 1 Processor -- start 2024-04-23T10:26:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.264+0000 7fcb0019b640 1 -- start start 2024-04-23T10:26:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.265+0000 7fcb0019b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcaf81021d0 0x7fcaf81986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.265+0000 7fcb0019b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcaf81072a0 0x7fcaf8198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.265+0000 7fcb0019b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcaf819cf60 0x7fcaf819d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.265+0000 7fcb0019b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf8112210 con 0x7fcaf81021d0 2024-04-23T10:26:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.265+0000 7fcb0019b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf8112090 con 0x7fcaf81072a0 2024-04-23T10:26:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.265+0000 7fcb0019b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf8112390 con 0x7fcaf819cf60 2024-04-23T10:26:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.265+0000 7fcafdf37640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcaf81021d0 0x7fcaf81986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.265+0000 7fcafdf37640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcaf81021d0 0x7fcaf81986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42734/0 (socket says 172.21.15.97:42734) 2024-04-23T10:26:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.265+0000 7fcafdf37640 1 -- 172.21.15.97:0/929107738 learned_addr learned my addr 172.21.15.97:0/929107738 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.265+0000 7fcafd736640 1 --2- 172.21.15.97:0/929107738 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcaf81072a0 0x7fcaf8198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.265+0000 7fcafe738640 1 --2- 172.21.15.97:0/929107738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcaf819cf60 0x7fcaf819d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:53.266 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.265+0000 7fcafdf37640 1 -- 172.21.15.97:0/929107738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcaf819cf60 msgr2=0x7fcaf819d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:53.267 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.265+0000 7fcafdf37640 1 --2- 172.21.15.97:0/929107738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcaf819cf60 0x7fcaf819d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:53.267 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.265+0000 7fcafdf37640 1 -- 172.21.15.97:0/929107738 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcaf81072a0 msgr2=0x7fcaf8198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:53.267 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.265+0000 7fcafdf37640 1 --2- 172.21.15.97:0/929107738 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcaf81072a0 0x7fcaf8198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:53.267 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.265+0000 7fcafdf37640 1 -- 172.21.15.97:0/929107738 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaf819da50 con 0x7fcaf81021d0 2024-04-23T10:26:53.267 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.266+0000 7fcafdf37640 1 --2- 172.21.15.97:0/929107738 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcaf81021d0 0x7fcaf81986f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fcaf400d950 tx=0x7fcaf400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:53.267 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.266+0000 7fcaeeffd640 1 -- 172.21.15.97:0/929107738 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf4017030 con 0x7fcaf81021d0 2024-04-23T10:26:53.267 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.266+0000 7fcaeeffd640 1 -- 172.21.15.97:0/929107738 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcaf4004020 con 0x7fcaf81021d0 2024-04-23T10:26:53.267 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.266+0000 7fcaeeffd640 1 -- 172.21.15.97:0/929107738 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf40047a0 con 0x7fcaf81021d0 2024-04-23T10:26:53.267 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.266+0000 7fcb0019b640 1 -- 172.21.15.97:0/929107738 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaf819dd40 con 0x7fcaf81021d0 2024-04-23T10:26:53.267 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.266+0000 7fcb0019b640 1 -- 172.21.15.97:0/929107738 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaf81a5580 con 0x7fcaf81021d0 2024-04-23T10:26:53.268 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.269+0000 7fcb02f92740 1 Processor -- start 2024-04-23T10:26:53.269 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.269+0000 7fcb02f92740 1 -- start start 2024-04-23T10:26:53.269 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.270+0000 7fcb02f92740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fe9486620 0x560fe9486a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:53.269 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.270+0000 7fcb02f92740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fe9486f40 0x560fe94912c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:53.269 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.270+0000 7fcb02f92740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560fe9491800 0x560fe9493bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:53.269 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.270+0000 7fcb02f92740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560fe93eb970 con 0x560fe9486620 2024-04-23T10:26:53.269 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.270+0000 7fcb02f92740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560fe93884a0 con 0x560fe9491800 2024-04-23T10:26:53.269 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.270+0000 7fcb02f92740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560fe93e7930 con 0x560fe9486f40 2024-04-23T10:26:53.270 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.270+0000 7fcafe738640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fe9486620 0x560fe9486a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:53.270 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.270+0000 7fcafe738640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fe9486620 0x560fe9486a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42744/0 (socket says 172.21.15.97:42744) 2024-04-23T10:26:53.270 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.270+0000 7fcafe738640 1 -- 172.21.15.97:0/1337741316 learned_addr learned my addr 172.21.15.97:0/1337741316 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:53.270 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.270+0000 7fcaeeffd640 1 -- 172.21.15.97:0/929107738 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fcaf4009a30 con 0x7fcaf81021d0 2024-04-23T10:26:53.270 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.270+0000 7fcafd736640 1 --2- 172.21.15.97:0/1337741316 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560fe9491800 0x560fe9493bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:53.270 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.270+0000 7fcafe738640 1 -- 172.21.15.97:0/1337741316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fe9486f40 msgr2=0x560fe94912c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-23T10:26:53.270 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.270+0000 7fcafe738640 1 --2- 172.21.15.97:0/1337741316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fe9486f40 0x560fe94912c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:53.270 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.270+0000 7fcafe738640 1 -- 172.21.15.97:0/1337741316 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560fe9491800 msgr2=0x560fe9493bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:53.270 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.270+0000 7fcafe738640 1 --2- 172.21.15.97:0/1337741316 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560fe9491800 0x560fe9493bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:53.270 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.270+0000 7fcafe738640 1 -- 172.21.15.97:0/1337741316 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fe90fe210 con 0x560fe9486620 2024-04-23T10:26:53.270 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.270+0000 7fcafe738640 1 --2- 172.21.15.97:0/1337741316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fe9486620 0x560fe9486a00 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0005fa0 tx=0x7fcaf0009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94cb837c271eda03 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:53.272 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.271+0000 7fcae3fff640 1 -- 172.21.15.97:0/1337741316 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf0031cf0 con 0x560fe9486620 2024-04-23T10:26:53.272 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.271+0000 7fcae3fff640 1 -- 172.21.15.97:0/1337741316 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcaf0036030 con 0x560fe9486620 2024-04-23T10:26:53.272 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.271+0000 7fcaeeffd640 1 --2- 172.21.15.97:0/929107738 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fcae40a3f60 0x7fcae40a6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:53.272 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.271+0000 7fcae3fff640 1 -- 172.21.15.97:0/1337741316 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf00042a0 con 0x560fe9486620 2024-04-23T10:26:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.271+0000 7fcaeeffd640 1 -- 172.21.15.97:0/929107738 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fcaf40d03b0 con 0x7fcaf81021d0 2024-04-23T10:26:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.271+0000 7fcafdf37640 1 --2- 172.21.15.97:0/929107738 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fcae40a3f60 0x7fcae40a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.272+0000 7fcb02f92740 1 -- 172.21.15.97:0/1337741316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fe9486620 msgr2=0x560fe9486a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.272+0000 7fcafdf37640 1 --2- 172.21.15.97:0/929107738 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fcae40a3f60 0x7fcae40a6420 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcaf40cda30 tx=0x7fcaf40046c0 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.272+0000 7fcb02f92740 1 --2- 172.21.15.97:0/1337741316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fe9486620 0x560fe9486a00 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0005fa0 tx=0x7fcaf0009800 comp rx=0 tx=0).stop 2024-04-23T10:26:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.272+0000 7fcb02f92740 1 -- 172.21.15.97:0/1337741316 shutdown_connections 2024-04-23T10:26:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.272+0000 7fcb02f92740 1 --2- 172.21.15.97:0/1337741316 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560fe9491800 0x560fe9493bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.272+0000 7fcb02f92740 1 --2- 172.21.15.97:0/1337741316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fe9486f40 0x560fe94912c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.272+0000 7fcb02f92740 1 --2- 172.21.15.97:0/1337741316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fe9486620 0x560fe9486a00 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.272+0000 7fcb02f92740 1 -- 172.21.15.97:0/1337741316 >> 172.21.15.97:0/1337741316 conn(0x560fe9482b30 msgr2=0x560fe9482f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.272+0000 7fcb02f92740 1 -- 172.21.15.97:0/1337741316 shutdown_connections 2024-04-23T10:26:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.273+0000 7fcb02f92740 1 -- 172.21.15.97:0/1337741316 wait complete. 2024-04-23T10:26:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.273+0000 7fcb02f92740 1 -- start start 2024-04-23T10:26:53.273 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.273+0000 7fcb02f92740 1 Processor -- start 2024-04-23T10:26:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.274+0000 7fcb02f92740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560fe9486620 0x560fe9491150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.274+0000 7fcb02f92740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fe9486f40 0x560fe948e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.274+0000 7fcb02f92740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fe9491800 0x560fe948e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.274+0000 7fcb02f92740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560fe9498660 con 0x560fe9491800 2024-04-23T10:26:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.274+0000 7fcafe738640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fe9491800 0x560fe948e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.274+0000 7fcafe738640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fe9491800 0x560fe948e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42750/0 (socket says 172.21.15.97:42750) 2024-04-23T10:26:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.274+0000 7fcafe738640 1 -- 172.21.15.97:0/1194335550 learned_addr learned my addr 172.21.15.97:0/1194335550 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.274+0000 7fcb02f92740 1 -- 172.21.15.97:0/1194335550 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560fe93860b0 con 0x560fe9486620 2024-04-23T10:26:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.274+0000 7fcb02f92740 1 -- 172.21.15.97:0/1194335550 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560fe94987e0 con 0x560fe9486f40 2024-04-23T10:26:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.274+0000 7fcafd736640 1 --2- 172.21.15.97:0/1194335550 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fe9486f40 0x560fe948e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.275+0000 7fcb02f92740 10 client.0 _ll_register_callbacks cb 0x560fe9373d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:26:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.275+0000 7fcafe738640 1 -- 172.21.15.97:0/1194335550 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fe9486f40 msgr2=0x560fe948e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.275+0000 7fcafe738640 1 --2- 172.21.15.97:0/1194335550 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fe9486f40 0x560fe948e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.275+0000 7fcafe738640 1 -- 172.21.15.97:0/1194335550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560fe9486620 msgr2=0x560fe9491150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.275+0000 7fcafe738640 1 --2- 172.21.15.97:0/1194335550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560fe9486620 0x560fe9491150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.275+0000 7fcafe738640 1 -- 172.21.15.97:0/1194335550 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fe91ca620 con 0x560fe9491800 2024-04-23T10:26:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.275+0000 7fcb02f92740 1 -- 172.21.15.97:0/1194335550 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcaf0036030 con 0x560fe9491800 2024-04-23T10:26:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.275+0000 7fcafe738640 1 --2- 172.21.15.97:0/1194335550 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fe9491800 0x560fe948e9a0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fcaf002ade0 tx=0x7fcaf0003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b48acb37b61f2c92 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.275+0000 7fcae17fa640 10 client.15753 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:26:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.275+0000 7fcae17fa640 1 -- 172.21.15.97:0/1194335550 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf0056060 con 0x560fe9491800 2024-04-23T10:26:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.275+0000 7fcae17fa640 1 -- 172.21.15.97:0/1194335550 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fcaf0004480 con 0x560fe9491800 2024-04-23T10:26:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.275+0000 7fcae17fa640 1 -- 172.21.15.97:0/1194335550 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcaf00046e0 con 0x560fe9491800 2024-04-23T10:26:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.275+0000 7fcb02f92740 10 client.15753 fetch_fsmap learned FSMap version 26 2024-04-23T10:26:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.275+0000 7fcb02f92740 1 -- 172.21.15.97:0/1194335550 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({fsmap=26}) -- 0x560fe94a43f0 con 0x560fe9491800 2024-04-23T10:26:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.275+0000 7fcae17fa640 1 -- 172.21.15.97:0/1194335550 <== mon.0 v2:172.21.15.97:3300/0 4 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fcaf0043070 con 0x560fe9491800 2024-04-23T10:26:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.275+0000 7fcb02f92740 10 client.15753 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:26:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.275+0000 7fcb02f92740 10 client.15753 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:26:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.275+0000 7fcb02f92740 20 client.15753 populate_metadata read hostname 'smithi097' 2024-04-23T10:26:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.276+0000 7fcb02f92740 1 --2- 172.21.15.97:0/1194335550 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x560fe94a5740 0x560fe94a7b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.276+0000 7fcb02f92740 4 client.15753 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:26:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.276+0000 7fcb02f92740 1 -- 172.21.15.97:0/1194335550 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560fe93f9390 con 0x560fe94a5740 2024-04-23T10:26:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.276+0000 7fcafd736640 1 --2- 172.21.15.97:0/1194335550 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x560fe94a5740 0x560fe94a7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.276+0000 7fcae17fa640 10 client.15753 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:26:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.276+0000 7fcafd736640 1 --2- 172.21.15.97:0/1194335550 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x560fe94a5740 0x560fe94a7b30 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7533e3776ea21f6 server_cookie=dd815629367e5d3 in_seq=0 out_seq=0 2024-04-23T10:26:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.277+0000 7fcae17fa640 1 -- 172.21.15.97:0/1194335550 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560fe93f9390 con 0x560fe94a5740 2024-04-23T10:26:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.277+0000 7fcae17fa640 10 client.15753 handle_command_reply: tid=0 2024-04-23T10:26:53.279 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.277+0000 7fcb02f92740 1 client.15753 shutdown 2024-04-23T10:26:53.279 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.277+0000 7fcb02f92740 1 -- 172.21.15.97:0/1194335550 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fe9491800 msgr2=0x560fe948e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:53.279 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.277+0000 7fcb02f92740 1 --2- 172.21.15.97:0/1194335550 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fe9491800 0x560fe948e9a0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fcaf002ade0 tx=0x7fcaf0003f10 comp rx=0 tx=0).stop 2024-04-23T10:26:53.279 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.277+0000 7fcb02f92740 1 -- 172.21.15.97:0/1194335550 shutdown_connections 2024-04-23T10:26:53.279 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.277+0000 7fcb02f92740 1 --2- 172.21.15.97:0/1194335550 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x560fe94a5740 0x560fe94a7b30 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:53.279 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.277+0000 7fcb02f92740 1 --2- 172.21.15.97:0/1194335550 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fe9491800 0x560fe948e9a0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:53.279 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.277+0000 7fcae17fa640 0 client.15753 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:26:53.279 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.277+0000 7fcb02f92740 1 --2- 172.21.15.97:0/1194335550 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fe9486f40 0x560fe948e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:53.279 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.277+0000 7fcb02f92740 1 --2- 172.21.15.97:0/1194335550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560fe9486620 0x560fe9491150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:53.279 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.277+0000 7fcb02f92740 1 -- 172.21.15.97:0/1194335550 >> 172.21.15.97:0/1194335550 conn(0x560fe9482b30 msgr2=0x560fe9488860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:53.279 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.277+0000 7fcb02f92740 1 -- 172.21.15.97:0/1194335550 shutdown_connections 2024-04-23T10:26:53.279 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.277+0000 7fcb02f92740 1 -- 172.21.15.97:0/1194335550 wait complete. 2024-04-23T10:26:53.279 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.277+0000 7fcb02f92740 20 client.15753 trim_cache size 0 max 16384 2024-04-23T10:26:53.292 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.292+0000 7fcb02f92740 1 Processor -- start 2024-04-23T10:26:53.292 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.292+0000 7fcb02f92740 1 -- start start 2024-04-23T10:26:53.292 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.292+0000 7fcb02f92740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fe9486620 0x560fe94a1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:53.292 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.292+0000 7fcb02f92740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560fe9486f40 0x560fe94ad2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:53.292 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.292+0000 7fcb02f92740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fe9491800 0x560fe94ad800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:53.292 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.292+0000 7fcb02f92740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf81ab800 con 0x560fe9486620 2024-04-23T10:26:53.292 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.292+0000 7fcb02f92740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf81ad340 con 0x560fe9486f40 2024-04-23T10:26:53.293 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.292+0000 7fcb02f92740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf81abed0 con 0x560fe9491800 2024-04-23T10:26:53.293 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.292+0000 7fcafe738640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fe9486620 0x560fe94a1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:53.293 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.293+0000 7fcafe738640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fe9486620 0x560fe94a1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42752/0 (socket says 172.21.15.97:42752) 2024-04-23T10:26:53.293 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.293+0000 7fcafe738640 1 -- 172.21.15.97:0/1409187570 learned_addr learned my addr 172.21.15.97:0/1409187570 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:53.293 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.293+0000 7fcafd736640 1 --2- 172.21.15.97:0/1409187570 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560fe9486f40 0x560fe94ad2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:53.293 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.293+0000 7fcafe738640 1 -- 172.21.15.97:0/1409187570 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fe9491800 msgr2=0x560fe94ad800 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-23T10:26:53.293 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.293+0000 7fcafe738640 1 --2- 172.21.15.97:0/1409187570 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fe9491800 0x560fe94ad800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:53.293 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.293+0000 7fcafe738640 1 -- 172.21.15.97:0/1409187570 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560fe9486f40 msgr2=0x560fe94ad2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:53.293 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.293+0000 7fcafe738640 1 --2- 172.21.15.97:0/1409187570 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560fe9486f40 0x560fe94ad2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:53.293 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.293+0000 7fcafe738640 1 -- 172.21.15.97:0/1409187570 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fe948ed10 con 0x560fe9486620 2024-04-23T10:26:53.293 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.293+0000 7fcafe738640 1 --2- 172.21.15.97:0/1409187570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fe9486620 0x560fe94a1520 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0002230 tx=0x7fcaf004c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b86fe66813429c70 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:53.293 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.293+0000 7fcae37fe640 1 -- 172.21.15.97:0/1409187570 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf0004440 con 0x560fe9486620 2024-04-23T10:26:53.293 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.293+0000 7fcae37fe640 1 -- 172.21.15.97:0/1409187570 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcaf0036030 con 0x560fe9486620 2024-04-23T10:26:53.293 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.293+0000 7fcae37fe640 1 -- 172.21.15.97:0/1409187570 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf002fb10 con 0x560fe9486620 2024-04-23T10:26:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.294+0000 7fcb02f92740 1 -- 172.21.15.97:0/1409187570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fe9486620 msgr2=0x560fe94a1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.294+0000 7fcb02f92740 1 --2- 172.21.15.97:0/1409187570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fe9486620 0x560fe94a1520 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0002230 tx=0x7fcaf004c000 comp rx=0 tx=0).stop 2024-04-23T10:26:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.294+0000 7fcb02f92740 1 -- 172.21.15.97:0/1409187570 shutdown_connections 2024-04-23T10:26:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.294+0000 7fcb02f92740 1 --2- 172.21.15.97:0/1409187570 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fe9491800 0x560fe94ad800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.294+0000 7fcb02f92740 1 --2- 172.21.15.97:0/1409187570 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560fe9486f40 0x560fe94ad2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.294+0000 7fcb02f92740 1 --2- 172.21.15.97:0/1409187570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fe9486620 0x560fe94a1520 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.294+0000 7fcb02f92740 1 -- 172.21.15.97:0/1409187570 >> 172.21.15.97:0/1409187570 conn(0x560fe9482b30 msgr2=0x560fe9481cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.294+0000 7fcb02f92740 1 -- 172.21.15.97:0/1409187570 shutdown_connections 2024-04-23T10:26:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.294+0000 7fcb02f92740 1 -- 172.21.15.97:0/1409187570 wait complete. 2024-04-23T10:26:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.294+0000 7fcb02f92740 1 -- start start 2024-04-23T10:26:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.295+0000 7fcb02f92740 1 Processor -- start 2024-04-23T10:26:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.295+0000 7fcb02f92740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fe9486620 0x560fe949dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.295+0000 7fcb02f92740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fe9486f40 0x560fe949b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:53.298 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.295+0000 7fcb02f92740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560fe9491800 0x560fe949b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:53.298 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.295+0000 7fcb02f92740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf81acc70 con 0x560fe9486f40 2024-04-23T10:26:53.298 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.295+0000 7fcb02f92740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf81ac5a0 con 0x560fe9491800 2024-04-23T10:26:53.298 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.295+0000 7fcb02f92740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf81a9800 con 0x560fe9486620 2024-04-23T10:26:53.298 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.295+0000 7fcb02f92740 10 client.0 _ll_register_callbacks cb 0x7fcaf0004390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:26:53.298 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.295+0000 7fcafd736640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fe9486f40 0x560fe949b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:53.298 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.295+0000 7fcafd736640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fe9486f40 0x560fe949b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42760/0 (socket says 172.21.15.97:42760) 2024-04-23T10:26:53.298 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.295+0000 7fcafd736640 1 -- 172.21.15.97:0/3592652679 learned_addr learned my addr 172.21.15.97:0/3592652679 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:53.298 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.295+0000 7fcafe738640 1 --2- 172.21.15.97:0/3592652679 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560fe9491800 0x560fe949b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:53.298 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.295+0000 7fcafe738640 1 --2- 172.21.15.97:0/3592652679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fe9486620 0x560fe949dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:53.298 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.295+0000 7fcafd736640 1 -- 172.21.15.97:0/3592652679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fe9486620 msgr2=0x560fe949dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:53.298 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.295+0000 7fcafd736640 1 --2- 172.21.15.97:0/3592652679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fe9486620 0x560fe949dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:53.298 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.295+0000 7fcafd736640 1 -- 172.21.15.97:0/3592652679 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560fe9491800 msgr2=0x560fe949b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:26:53.299 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.295+0000 7fcafd736640 1 --2- 172.21.15.97:0/3592652679 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560fe9491800 0x560fe949b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:53.299 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.295+0000 7fcafd736640 1 -- 172.21.15.97:0/3592652679 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fe949bc20 con 0x560fe9486f40 2024-04-23T10:26:53.299 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.295+0000 7fcafe738640 1 --2- 172.21.15.97:0/3592652679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fe9486620 0x560fe949dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:26:53.299 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.296+0000 7fcafd736640 1 --2- 172.21.15.97:0/3592652679 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fe9486f40 0x560fe949b040 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fcaf8108880 tx=0x7fcae80206a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa4876e06dcb1142 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:53.299 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.296+0000 7fcae17fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:26:53.299 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.296+0000 7fcae17fa640 1 -- 172.21.15.97:0/3592652679 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae8010c90 con 0x560fe9486f40 2024-04-23T10:26:53.299 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.296+0000 7fcae17fa640 1 -- 172.21.15.97:0/3592652679 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fcae8005020 con 0x560fe9486f40 2024-04-23T10:26:53.299 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.296+0000 7fcb02f92740 1 -- 172.21.15.97:0/3592652679 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcaf0036030 con 0x560fe9486f40 2024-04-23T10:26:53.299 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.296+0000 7fcae17fa640 1 -- 172.21.15.97:0/3592652679 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcae8010e30 con 0x560fe9486f40 2024-04-23T10:26:53.299 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.296+0000 7fcb02f92740 10 client.15759 fetch_fsmap learned FSMap version 26 2024-04-23T10:26:53.299 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.296+0000 7fcb02f92740 1 -- 172.21.15.97:0/3592652679 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560fe950ba60 con 0x560fe9486f40 2024-04-23T10:26:53.299 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.296+0000 7fcae17fa640 1 -- 172.21.15.97:0/3592652679 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcaf0036030 con 0x560fe9486f40 2024-04-23T10:26:53.299 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.296+0000 7fcae17fa640 1 -- 172.21.15.97:0/3592652679 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fcae8032480 con 0x560fe9486f40 2024-04-23T10:26:53.299 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.296+0000 7fcb02f92740 10 client.15759 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:26:53.300 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.296+0000 7fcb02f92740 10 client.15759 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:26:53.300 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.296+0000 7fcb02f92740 20 client.15759 populate_metadata read hostname 'smithi097' 2024-04-23T10:26:53.300 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.297+0000 7fcb02f92740 1 --2- 172.21.15.97:0/3592652679 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x560fe94a5740 0x560fe950e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:53.300 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.297+0000 7fcb02f92740 4 client.15759 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:26:53.300 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.297+0000 7fcb02f92740 1 -- 172.21.15.97:0/3592652679 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560fe93f3200 con 0x560fe94a5740 2024-04-23T10:26:53.300 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.297+0000 7fcafe738640 1 --2- 172.21.15.97:0/3592652679 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x560fe94a5740 0x560fe950e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:53.300 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.297+0000 7fcae17fa640 10 client.15759 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:26:53.300 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.297+0000 7fcafe738640 1 --2- 172.21.15.97:0/3592652679 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x560fe94a5740 0x560fe950e170 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c83b77624f4b0634 server_cookie=5ec39292f88aff4b in_seq=0 out_seq=0 2024-04-23T10:26:53.301 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.301+0000 7fcae17fa640 1 -- 172.21.15.97:0/3592652679 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560fe93f3200 con 0x560fe94a5740 2024-04-23T10:26:53.301 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.301+0000 7fcae17fa640 10 client.15759 handle_command_reply: tid=0 2024-04-23T10:26:53.301 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.301+0000 7fcb02f92740 1 client.15759 shutdown 2024-04-23T10:26:53.302 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.301+0000 7fcb02f92740 1 -- 172.21.15.97:0/3592652679 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fe9486f40 msgr2=0x560fe949b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:53.302 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.301+0000 7fcb02f92740 1 --2- 172.21.15.97:0/3592652679 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fe9486f40 0x560fe949b040 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fcaf8108880 tx=0x7fcae80206a0 comp rx=0 tx=0).stop 2024-04-23T10:26:53.302 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.302+0000 7fcb02f92740 1 -- 172.21.15.97:0/3592652679 shutdown_connections 2024-04-23T10:26:53.302 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.302+0000 7fcb02f92740 1 --2- 172.21.15.97:0/3592652679 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x560fe94a5740 0x560fe950e170 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:53.302 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.302+0000 7fcb02f92740 1 --2- 172.21.15.97:0/3592652679 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560fe9491800 0x560fe949b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:53.302 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.302+0000 7fcb02f92740 1 --2- 172.21.15.97:0/3592652679 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560fe9486f40 0x560fe949b040 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:53.302 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.302+0000 7fcb02f92740 1 --2- 172.21.15.97:0/3592652679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560fe9486620 0x560fe949dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:53.302 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.302+0000 7fcb02f92740 1 -- 172.21.15.97:0/3592652679 >> 172.21.15.97:0/3592652679 conn(0x560fe9482b30 msgr2=0x560fe94a0510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:53.302 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.302+0000 7fcae17fa640 0 client.15759 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:26:53.302 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.302+0000 7fcb02f92740 1 -- 172.21.15.97:0/3592652679 shutdown_connections 2024-04-23T10:26:53.302 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.302+0000 7fcb02f92740 1 -- 172.21.15.97:0/3592652679 wait complete. 2024-04-23T10:26:53.302 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.302+0000 7fcb02f92740 20 client.15759 trim_cache size 0 max 16384 2024-04-23T10:26:53.303 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:26:53.303 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:26:53.303 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:26:53.303 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:26:53.304 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.302+0000 7fcb0019b640 1 -- 172.21.15.97:0/929107738 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fcae40a3f60 msgr2=0x7fcae40a6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:53.304 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.302+0000 7fcb0019b640 1 --2- 172.21.15.97:0/929107738 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fcae40a3f60 0x7fcae40a6420 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcaf40cda30 tx=0x7fcaf40046c0 comp rx=0 tx=0).stop 2024-04-23T10:26:53.304 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.302+0000 7fcb0019b640 1 -- 172.21.15.97:0/929107738 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcaf81021d0 msgr2=0x7fcaf81986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:53.304 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.302+0000 7fcb0019b640 1 --2- 172.21.15.97:0/929107738 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcaf81021d0 0x7fcaf81986f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fcaf400d950 tx=0x7fcaf400de10 comp rx=0 tx=0).stop 2024-04-23T10:26:53.304 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.303+0000 7fcb0019b640 1 -- 172.21.15.97:0/929107738 shutdown_connections 2024-04-23T10:26:53.304 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.303+0000 7fcb0019b640 1 --2- 172.21.15.97:0/929107738 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fcae40a3f60 0x7fcae40a6420 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:53.304 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.303+0000 7fcb0019b640 1 --2- 172.21.15.97:0/929107738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcaf819cf60 0x7fcaf819d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:53.304 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.303+0000 7fcb0019b640 1 --2- 172.21.15.97:0/929107738 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcaf81072a0 0x7fcaf8198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:53.304 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.303+0000 7fcb0019b640 1 --2- 172.21.15.97:0/929107738 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcaf81021d0 0x7fcaf81986f0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:53.304 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.303+0000 7fcb0019b640 1 -- 172.21.15.97:0/929107738 >> 172.21.15.97:0/929107738 conn(0x7fcaf80fda50 msgr2=0x7fcaf80ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:53.304 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.303+0000 7fcb0019b640 1 -- 172.21.15.97:0/929107738 shutdown_connections 2024-04-23T10:26:53.305 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:26:53.303+0000 7fcb0019b640 1 -- 172.21.15.97:0/929107738 wait complete. 2024-04-23T10:26:53.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:52 smithi196 ceph-mon[32919]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 10 op/s 2024-04-23T10:26:53.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:52 smithi097 ceph-mon[29197]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 10 op/s 2024-04-23T10:26:53.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:52 smithi183 ceph-mon[32317]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 10 op/s 2024-04-23T10:26:55.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:54 smithi196 ceph-mon[32919]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 853 B/s wr, 14 op/s 2024-04-23T10:26:55.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:54 smithi097 ceph-mon[29197]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 853 B/s wr, 14 op/s 2024-04-23T10:26:55.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:54 smithi183 ceph-mon[32317]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 853 B/s wr, 14 op/s 2024-04-23T10:26:57.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:56 smithi196 ceph-mon[32919]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-23T10:26:57.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:56 smithi097 ceph-mon[29197]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-23T10:26:57.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:56 smithi183 ceph-mon[32317]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-23T10:26:59.279 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T10:26:59.297 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:58 smithi097 ceph-mon[29197]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-23T10:26:59.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:58 smithi196 ceph-mon[32919]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-23T10:26:59.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:58 smithi183 ceph-mon[32317]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-23T10:26:59.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:59.410+0000 7f06276a0640 1 -- 172.21.15.97:0/687848127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0620103b00 msgr2=0x7f062010bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:59.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:59.410+0000 7f06276a0640 1 --2- 172.21.15.97:0/687848127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0620103b00 0x7f062010bfe0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f0610009900 tx=0x7f061002a510 comp rx=0 tx=0).stop 2024-04-23T10:26:59.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:59.411+0000 7f06276a0640 1 -- 172.21.15.97:0/687848127 shutdown_connections 2024-04-23T10:26:59.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:59.411+0000 7f06276a0640 1 --2- 172.21.15.97:0/687848127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f062010c520 0x7f062010e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:59.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:59.411+0000 7f06276a0640 1 --2- 172.21.15.97:0/687848127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0620103b00 0x7f062010bfe0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:59.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:59.411+0000 7f06276a0640 1 --2- 172.21.15.97:0/687848127 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06201031e0 0x7f06201035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:59.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:59.411+0000 7f06276a0640 1 -- 172.21.15.97:0/687848127 >> 172.21.15.97:0/687848127 conn(0x7f0620077f50 msgr2=0x7f0620078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:59.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:59.411+0000 7f06276a0640 1 -- 172.21.15.97:0/687848127 shutdown_connections 2024-04-23T10:26:59.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:59.411+0000 7f06276a0640 1 -- 172.21.15.97:0/687848127 wait complete. 2024-04-23T10:26:59.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:59.411+0000 7f06276a0640 1 Processor -- start 2024-04-23T10:26:59.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:59.411+0000 7f06276a0640 1 -- start start 2024-04-23T10:26:59.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:59.412+0000 7f06276a0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06201031e0 0x7f0620198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:59.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:59.412+0000 7f06276a0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0620103b00 0x7f0620198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:59.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:59.412+0000 7f06276a0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f062010c520 0x7f062019cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:59.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:59.412+0000 7f06276a0640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f06201121b0 con 0x7f0620103b00 2024-04-23T10:26:59.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:59.412+0000 7f06276a0640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0620112030 con 0x7f06201031e0 2024-04-23T10:26:59.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:59.412+0000 7f06276a0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0620112330 con 0x7f062010c520 2024-04-23T10:26:59.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:59.412+0000 7f0624c3b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0620103b00 0x7f0620198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:59.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:59.412+0000 7f062543c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06201031e0 0x7f0620198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:59.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:59.412+0000 7f062543c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06201031e0 0x7f0620198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.97:55250/0 (socket says 172.21.15.97:55250) 2024-04-23T10:26:59.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:59.412+0000 7f062543c640 1 -- 172.21.15.97:0/1399202133 learned_addr learned my addr 172.21.15.97:0/1399202133 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:26:59.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:59.412+0000 7f0625c3d640 1 --2- 172.21.15.97:0/1399202133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f062010c520 0x7f062019cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:59.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:59.412+0000 7f0624c3b640 1 -- 172.21.15.97:0/1399202133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f062010c520 msgr2=0x7f062019cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:59.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:59.412+0000 7f0624c3b640 1 --2- 172.21.15.97:0/1399202133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f062010c520 0x7f062019cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:59.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:59.412+0000 7f0624c3b640 1 -- 172.21.15.97:0/1399202133 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06201031e0 msgr2=0x7f0620198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:59.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:59.412+0000 7f0624c3b640 1 --2- 172.21.15.97:0/1399202133 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06201031e0 0x7f0620198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:59.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:59.412+0000 7f0624c3b640 1 -- 172.21.15.97:0/1399202133 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f062019d720 con 0x7f0620103b00 2024-04-23T10:26:59.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:59.412+0000 7f0624c3b640 1 --2- 172.21.15.97:0/1399202133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0620103b00 0x7f0620198c70 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f061002aa30 tx=0x7f0610004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:59.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:59.413+0000 7f06167fc640 1 -- 172.21.15.97:0/1399202133 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061003f030 con 0x7f0620103b00 2024-04-23T10:26:59.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:59.413+0000 7f06276a0640 1 -- 172.21.15.97:0/1399202133 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f062019d9b0 con 0x7f0620103b00 2024-04-23T10:26:59.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:59.413+0000 7f06167fc640 1 -- 172.21.15.97:0/1399202133 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0610003aa0 con 0x7f0620103b00 2024-04-23T10:26:59.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:59.413+0000 7f06167fc640 1 -- 172.21.15.97:0/1399202133 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0610035070 con 0x7f0620103b00 2024-04-23T10:26:59.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:59.413+0000 7f06276a0640 1 -- 172.21.15.97:0/1399202133 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0620100d60 con 0x7f0620103b00 2024-04-23T10:26:59.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:59.414+0000 7f06276a0640 1 -- 172.21.15.97:0/1399202133 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0604005290 con 0x7f0620103b00 2024-04-23T10:26:59.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:59.415+0000 7f06167fc640 1 -- 172.21.15.97:0/1399202133 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f0610042030 con 0x7f0620103b00 2024-04-23T10:26:59.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:59.416+0000 7f06167fc640 1 --2- 172.21.15.97:0/1399202133 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f060c0a3e40 0x7f060c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:26:59.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:59.416+0000 7f06167fc640 1 -- 172.21.15.97:0/1399202133 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f06100f0b70 con 0x7f0620103b00 2024-04-23T10:26:59.416 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:59.418+0000 7f062543c640 1 --2- 172.21.15.97:0/1399202133 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f060c0a3e40 0x7f060c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:26:59.417 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:59.418+0000 7f062543c640 1 --2- 172.21.15.97:0/1399202133 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f060c0a3e40 0x7f060c0a6300 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f061c005fd0 tx=0x7f061c00f8b0 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:26:59.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:59.424+0000 7f06167fc640 1 -- 172.21.15.97:0/1399202133 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f06100bca90 con 0x7f0620103b00 2024-04-23T10:26:59.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:59.816+0000 7f06276a0640 1 -- 172.21.15.97:0/1399202133 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0604005850 con 0x7f0620103b00 2024-04-23T10:26:59.815 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:59.817+0000 7f06167fc640 1 -- 172.21.15.97:0/1399202133 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11931 (secure 0 0 0) 0x7f06100c0d90 con 0x7f0620103b00 2024-04-23T10:26:59.816 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:26:59.816 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3946878762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3946878762},{"type":"v1","addr":"172.21.15.97:6835","nonce":3946878762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/209800861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":209800861},{"type":"v1","addr":"172.21.15.97:6837","nonce":209800861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T10:24:06.910134+0000","modified":"2024-04-23T10:26:09.820985+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6839/418013575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":418013575},{"type":"v1","addr":"172.21.15.97:6839","nonce":418013575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6841/747242680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":747242680},{"type":"v1","addr":"172.21.15.97:6841","nonce":747242680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6833/4292847121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4292847121},{"type":"v1","addr":"172.21.15.183:6833","nonce":4292847121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6833/3885816565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3885816565},{"type":"v1","addr":"172.21.15.196:6833","nonce":3885816565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1897093105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1897093105},{"type":"v1","addr":"172.21.15.183:6835","nonce":1897093105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1171477563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1171477563},{"type":"v1","addr":"172.21.15.196:6835","nonce":1171477563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1898341911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1898341911},{"type":"v1","addr":"172.21.15.183:6837","nonce":1898341911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.196:6837/3094946502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3094946502},{"type":"v1","addr":"172.21.15.196:6837","nonce":3094946502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.196:6839/83843556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":83843556},{"type":"v1","addr":"172.21.15.196:6839","nonce":83843556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6839/1763841130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1763841130},{"type":"v1","addr":"172.21.15.183:6839","nonce":1763841130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24487,24403,24398,24469,24488]},"id":1}]} 2024-04-23T10:26:59.816 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 26 2024-04-23T10:26:59.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:59.820+0000 7f06276a0640 1 -- 172.21.15.97:0/1399202133 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f060c0a3e40 msgr2=0x7f060c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:59.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:59.820+0000 7f06276a0640 1 --2- 172.21.15.97:0/1399202133 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f060c0a3e40 0x7f060c0a6300 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f061c005fd0 tx=0x7f061c00f8b0 comp rx=0 tx=0).stop 2024-04-23T10:26:59.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:59.820+0000 7f06276a0640 1 -- 172.21.15.97:0/1399202133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0620103b00 msgr2=0x7f0620198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:26:59.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:59.820+0000 7f06276a0640 1 --2- 172.21.15.97:0/1399202133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0620103b00 0x7f0620198c70 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f061002aa30 tx=0x7f0610004270 comp rx=0 tx=0).stop 2024-04-23T10:26:59.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:59.820+0000 7f06276a0640 1 -- 172.21.15.97:0/1399202133 shutdown_connections 2024-04-23T10:26:59.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:59.820+0000 7f06276a0640 1 --2- 172.21.15.97:0/1399202133 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f060c0a3e40 0x7f060c0a6300 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:59.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:59.820+0000 7f06276a0640 1 --2- 172.21.15.97:0/1399202133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f062010c520 0x7f062019cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:59.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:59.820+0000 7f06276a0640 1 --2- 172.21.15.97:0/1399202133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0620103b00 0x7f0620198c70 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:59.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:59.820+0000 7f06276a0640 1 --2- 172.21.15.97:0/1399202133 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06201031e0 0x7f0620198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:26:59.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:59.820+0000 7f06276a0640 1 -- 172.21.15.97:0/1399202133 >> 172.21.15.97:0/1399202133 conn(0x7f0620077f50 msgr2=0x7f06200fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:26:59.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:59.820+0000 7f06276a0640 1 -- 172.21.15.97:0/1399202133 shutdown_connections 2024-04-23T10:26:59.819 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:26:59.820+0000 7f06276a0640 1 -- 172.21.15.97:0/1399202133 wait complete. 2024-04-23T10:26:59.832 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --set-id 55d3c626 --release --await' 2024-04-23T10:26:59.832 DEBUG:teuthology.orchestra.run.smithi097:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24488 quiesce db --set-id 55d3c626 --release --await 2024-04-23T10:27:00.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.001+0000 7f8a126a9640 1 -- 172.21.15.97:0/690939984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a0c1005b0 msgr2=0x7f8a0c100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:00.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.001+0000 7f8a126a9640 1 --2- 172.21.15.97:0/690939984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a0c1005b0 0x7f8a0c100a30 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f8a00009900 tx=0x7f8a0002b520 comp rx=0 tx=0).stop 2024-04-23T10:27:00.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.001+0000 7f8a126a9640 1 -- 172.21.15.97:0/690939984 shutdown_connections 2024-04-23T10:27:00.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.001+0000 7f8a126a9640 1 --2- 172.21.15.97:0/690939984 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a0c100f70 0x7f8a0c10d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.001+0000 7f8a126a9640 1 --2- 172.21.15.97:0/690939984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a0c1005b0 0x7f8a0c100a30 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.001+0000 7f8a126a9640 1 --2- 172.21.15.97:0/690939984 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a0c106670 0x7f8a0c106a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.001+0000 7f8a126a9640 1 -- 172.21.15.97:0/690939984 >> 172.21.15.97:0/690939984 conn(0x7f8a0c0773f0 msgr2=0x7f8a0c0fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:27:00.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.002+0000 7f8a126a9640 1 -- 172.21.15.97:0/690939984 shutdown_connections 2024-04-23T10:27:00.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.002+0000 7f8a126a9640 1 -- 172.21.15.97:0/690939984 wait complete. 2024-04-23T10:27:00.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.002+0000 7f8a126a9640 1 Processor -- start 2024-04-23T10:27:00.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.002+0000 7f8a126a9640 1 -- start start 2024-04-23T10:27:00.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.003+0000 7f8a126a9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a0c1005b0 0x7f8a0c196470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:00.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.003+0000 7f8a126a9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a0c100f70 0x7f8a0c1969b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:00.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.003+0000 7f8a126a9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a0c106670 0x7f8a0c19ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:00.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.003+0000 7f8a126a9640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0c10fff0 con 0x7f8a0c100f70 2024-04-23T10:27:00.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.003+0000 7f8a126a9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0c10fe70 con 0x7f8a0c1005b0 2024-04-23T10:27:00.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.003+0000 7f8a126a9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0c110170 con 0x7f8a0c106670 2024-04-23T10:27:00.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.003+0000 7f8a0b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a0c100f70 0x7f8a0c1969b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:00.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.003+0000 7f8a0b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a0c100f70 0x7f8a0c1969b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53130/0 (socket says 172.21.15.97:53130) 2024-04-23T10:27:00.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.003+0000 7f8a0b7fe640 1 -- 172.21.15.97:0/1077564010 learned_addr learned my addr 172.21.15.97:0/1077564010 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:27:00.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.003+0000 7f8a0bfff640 1 --2- 172.21.15.97:0/1077564010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a0c1005b0 0x7f8a0c196470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:00.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.003+0000 7f8a10c46640 1 --2- 172.21.15.97:0/1077564010 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a0c106670 0x7f8a0c19ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:00.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.003+0000 7f8a0bfff640 1 -- 172.21.15.97:0/1077564010 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a0c106670 msgr2=0x7f8a0c19ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:00.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.003+0000 7f8a0bfff640 1 --2- 172.21.15.97:0/1077564010 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a0c106670 0x7f8a0c19ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.003+0000 7f8a0bfff640 1 -- 172.21.15.97:0/1077564010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a0c100f70 msgr2=0x7f8a0c1969b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:00.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.003+0000 7f8a0bfff640 1 --2- 172.21.15.97:0/1077564010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a0c100f70 0x7f8a0c1969b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.003+0000 7f8a0bfff640 1 -- 172.21.15.97:0/1077564010 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a0c19b460 con 0x7f8a0c1005b0 2024-04-23T10:27:00.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.003+0000 7f8a0b7fe640 1 --2- 172.21.15.97:0/1077564010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a0c100f70 0x7f8a0c1969b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T10:27:00.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.004+0000 7f8a0bfff640 1 --2- 172.21.15.97:0/1077564010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a0c1005b0 0x7f8a0c196470 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f89fc005fd0 tx=0x7f89fc00dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:27:00.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.004+0000 7f8a097fa640 1 -- 172.21.15.97:0/1077564010 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89fc009c30 con 0x7f8a0c1005b0 2024-04-23T10:27:00.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.004+0000 7f8a097fa640 1 -- 172.21.15.97:0/1077564010 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89fc009dd0 con 0x7f8a0c1005b0 2024-04-23T10:27:00.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.004+0000 7f8a126a9640 1 -- 172.21.15.97:0/1077564010 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a0c19b750 con 0x7f8a0c1005b0 2024-04-23T10:27:00.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.004+0000 7f8a126a9640 1 -- 172.21.15.97:0/1077564010 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a0c1a51f0 con 0x7f8a0c1005b0 2024-04-23T10:27:00.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.004+0000 7f8a097fa640 1 -- 172.21.15.97:0/1077564010 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89fc010460 con 0x7f8a0c1005b0 2024-04-23T10:27:00.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.006+0000 7f8a097fa640 1 -- 172.21.15.97:0/1077564010 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f89fc003e80 con 0x7f8a0c1005b0 2024-04-23T10:27:00.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.007+0000 7f8a097fa640 1 --2- 172.21.15.97:0/1077564010 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f89f40a3ec0 0x7f89f40a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:00.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.007+0000 7f8a097fa640 1 -- 172.21.15.97:0/1077564010 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f89fc0d0a30 con 0x7f8a0c1005b0 2024-04-23T10:27:00.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.007+0000 7f8a0b7fe640 1 --2- 172.21.15.97:0/1077564010 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f89f40a3ec0 0x7f89f40a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:00.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.007+0000 7f8a153bd740 1 Processor -- start 2024-04-23T10:27:00.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.007+0000 7f8a153bd740 1 -- start start 2024-04-23T10:27:00.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.008+0000 7f8a153bd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af62547390 0x55af62547770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:00.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.008+0000 7f8a153bd740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af62547cb0 0x55af6255ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:00.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.008+0000 7f8a0b7fe640 1 --2- 172.21.15.97:0/1077564010 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f89f40a3ec0 0x7f89f40a6380 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8a000096c0 tx=0x7f8a000046c0 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:27:00.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.008+0000 7f8a153bd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af6254df00 0x55af6255d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:00.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.008+0000 7f8a153bd740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55af624ac6d0 con 0x55af62547cb0 2024-04-23T10:27:00.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.008+0000 7f8a153bd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55af624491a0 con 0x55af6254df00 2024-04-23T10:27:00.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.008+0000 7f8a153bd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55af624a8690 con 0x55af62547390 2024-04-23T10:27:00.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.008+0000 7f8a0b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af62547cb0 0x55af6255ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:00.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.008+0000 7f8a0b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af62547cb0 0x55af6255ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53134/0 (socket says 172.21.15.97:53134) 2024-04-23T10:27:00.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.008+0000 7f8a0b7fe640 1 -- 172.21.15.97:0/3788997133 learned_addr learned my addr 172.21.15.97:0/3788997133 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:27:00.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.008+0000 7f8a0bfff640 1 --2- 172.21.15.97:0/3788997133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af62547390 0x55af62547770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:00.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.008+0000 7f8a10c46640 1 --2- 172.21.15.97:0/3788997133 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af6254df00 0x55af6255d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:00.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.008+0000 7f8a0bfff640 1 -- 172.21.15.97:0/3788997133 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af6254df00 msgr2=0x55af6255d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:27:00.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.008+0000 7f8a0bfff640 1 --2- 172.21.15.97:0/3788997133 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af6254df00 0x55af6255d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.008+0000 7f8a0bfff640 1 -- 172.21.15.97:0/3788997133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af62547cb0 msgr2=0x55af6255ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:27:00.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.008+0000 7f8a0bfff640 1 --2- 172.21.15.97:0/3788997133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af62547cb0 0x55af6255ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.008+0000 7f8a0bfff640 1 -- 172.21.15.97:0/3788997133 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af624535b0 con 0x55af62547390 2024-04-23T10:27:00.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.008+0000 7f8a0b7fe640 1 --2- 172.21.15.97:0/3788997133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af62547cb0 0x55af6255ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T10:27:00.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.008+0000 7f8a0bfff640 1 --2- 172.21.15.97:0/3788997133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af62547390 0x55af62547770 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f89fc0cdfc0 tx=0x7f89fc0036b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6078f2adc75dfdf server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:27:00.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.009+0000 7f89f27fc640 1 -- 172.21.15.97:0/3788997133 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89fc020070 con 0x55af62547390 2024-04-23T10:27:00.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.009+0000 7f89f27fc640 1 -- 172.21.15.97:0/3788997133 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89fc0a0b20 con 0x55af62547390 2024-04-23T10:27:00.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.009+0000 7f89f27fc640 1 -- 172.21.15.97:0/3788997133 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89fc09bbc0 con 0x55af62547390 2024-04-23T10:27:00.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.009+0000 7f8a153bd740 1 -- 172.21.15.97:0/3788997133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af62547390 msgr2=0x55af62547770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:00.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.009+0000 7f8a153bd740 1 --2- 172.21.15.97:0/3788997133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af62547390 0x55af62547770 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f89fc0cdfc0 tx=0x7f89fc0036b0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.010+0000 7f8a153bd740 1 -- 172.21.15.97:0/3788997133 shutdown_connections 2024-04-23T10:27:00.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.010+0000 7f8a153bd740 1 --2- 172.21.15.97:0/3788997133 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af6254df00 0x55af6255d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.010+0000 7f8a153bd740 1 --2- 172.21.15.97:0/3788997133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af62547cb0 0x55af6255ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.010+0000 7f8a153bd740 1 --2- 172.21.15.97:0/3788997133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af62547390 0x55af62547770 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.010+0000 7f8a153bd740 1 -- 172.21.15.97:0/3788997133 >> 172.21.15.97:0/3788997133 conn(0x55af625437b0 msgr2=0x55af62543b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:27:00.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.010+0000 7f8a153bd740 1 -- 172.21.15.97:0/3788997133 shutdown_connections 2024-04-23T10:27:00.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.010+0000 7f8a153bd740 1 -- 172.21.15.97:0/3788997133 wait complete. 2024-04-23T10:27:00.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.010+0000 7f8a153bd740 1 -- start start 2024-04-23T10:27:00.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.010+0000 7f8a153bd740 1 Processor -- start 2024-04-23T10:27:00.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.011+0000 7f8a153bd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af62547390 0x55af62551d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:00.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.011+0000 7f8a153bd740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af62547cb0 0x55af6254f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:00.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.011+0000 7f8a153bd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af6254f700 0x55af62566f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:00.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.011+0000 7f8a153bd740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55af6255daa0 con 0x55af62547cb0 2024-04-23T10:27:00.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.011+0000 7f8a153bd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55af62446b20 con 0x55af6254f700 2024-04-23T10:27:00.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.011+0000 7f8a153bd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55af6255dc20 con 0x55af62547390 2024-04-23T10:27:00.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.011+0000 7f8a153bd740 10 client.0 _ll_register_callbacks cb 0x55af62446530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:27:00.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.011+0000 7f8a0b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af62547cb0 0x55af6254f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:00.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.011+0000 7f8a10c46640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af6254f700 0x55af62566f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:00.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.011+0000 7f8a0b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af62547cb0 0x55af6254f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53144/0 (socket says 172.21.15.97:53144) 2024-04-23T10:27:00.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.011+0000 7f8a0b7fe640 1 -- 172.21.15.97:0/450340644 learned_addr learned my addr 172.21.15.97:0/450340644 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:27:00.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.011+0000 7f8a10c46640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af6254f700 0x55af62566f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.97:55282/0 (socket says 172.21.15.97:55282) 2024-04-23T10:27:00.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.011+0000 7f8a0bfff640 1 --2- 172.21.15.97:0/450340644 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af62547390 0x55af62551d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:00.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.011+0000 7f8a10c46640 1 -- 172.21.15.97:0/450340644 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af62547390 msgr2=0x55af62551d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:27:00.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.011+0000 7f8a10c46640 1 --2- 172.21.15.97:0/450340644 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af62547390 0x55af62551d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.011+0000 7f8a10c46640 1 -- 172.21.15.97:0/450340644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af62547cb0 msgr2=0x55af6254f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:27:00.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.011+0000 7f8a10c46640 1 --2- 172.21.15.97:0/450340644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af62547cb0 0x55af6254f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.011+0000 7f8a10c46640 1 -- 172.21.15.97:0/450340644 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af621be920 con 0x55af6254f700 2024-04-23T10:27:00.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.011+0000 7f8a0b7fe640 1 --2- 172.21.15.97:0/450340644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af62547cb0 0x55af6254f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T10:27:00.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.012+0000 7f8a10c46640 1 --2- 172.21.15.97:0/450340644 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af6254f700 0x55af62566f50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8a04005fd0 tx=0x7f8a04009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88a67dae3272b602 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:27:00.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.012+0000 7f89eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-23T10:27:00.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.012+0000 7f89eb7fe640 1 -- 172.21.15.97:0/450340644 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a04011770 con 0x55af6254f700 2024-04-23T10:27:00.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.012+0000 7f8a153bd740 1 -- 172.21.15.97:0/450340644 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89fc0a0b20 con 0x55af6254f700 2024-04-23T10:27:00.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.012+0000 7f89eb7fe640 1 -- 172.21.15.97:0/450340644 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f8a04011910 con 0x55af6254f700 2024-04-23T10:27:00.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.012+0000 7f89eb7fe640 1 -- 172.21.15.97:0/450340644 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8a04011b10 con 0x55af6254f700 2024-04-23T10:27:00.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.012+0000 7f8a153bd740 10 client.25268 fetch_fsmap learned FSMap version 26 2024-04-23T10:27:00.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.012+0000 7f8a153bd740 1 -- 172.21.15.97:0/450340644 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55af6228ad30 con 0x55af6254f700 2024-04-23T10:27:00.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.012+0000 7f89eb7fe640 1 -- 172.21.15.97:0/450340644 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89fc0a0b20 con 0x55af6254f700 2024-04-23T10:27:00.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.013+0000 7f89eb7fe640 1 -- 172.21.15.97:0/450340644 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8a0401b420 con 0x55af6254f700 2024-04-23T10:27:00.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.013+0000 7f8a153bd740 10 client.25268 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:27:00.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.013+0000 7f8a153bd740 10 client.25268 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:27:00.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.013+0000 7f8a153bd740 20 client.25268 populate_metadata read hostname 'smithi097' 2024-04-23T10:27:00.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.013+0000 7f8a153bd740 1 --2- 172.21.15.97:0/450340644 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55af6256b3f0 0x55af6256d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:00.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.013+0000 7f8a153bd740 4 client.25268 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:27:00.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.013+0000 7f8a153bd740 1 -- 172.21.15.97:0/450340644 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af6246a500 con 0x55af6256b3f0 2024-04-23T10:27:00.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.014+0000 7f8a0bfff640 1 --2- 172.21.15.97:0/450340644 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55af6256b3f0 0x55af6256d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:00.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.014+0000 7f89eb7fe640 10 client.25268 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:27:00.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.014+0000 7f8a0bfff640 1 --2- 172.21.15.97:0/450340644 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55af6256b3f0 0x55af6256d830 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b354b654afccb710 server_cookie=ea6105c8e60549a7 in_seq=0 out_seq=0 2024-04-23T10:27:00.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.015+0000 7f89eb7fe640 1 -- 172.21.15.97:0/450340644 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55af6246a500 con 0x55af6256b3f0 2024-04-23T10:27:00.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.015+0000 7f89eb7fe640 10 client.25268 handle_command_reply: tid=0 2024-04-23T10:27:00.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.015+0000 7f8a153bd740 1 client.25268 shutdown 2024-04-23T10:27:00.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.015+0000 7f8a153bd740 1 -- 172.21.15.97:0/450340644 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af6254f700 msgr2=0x55af62566f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:00.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.015+0000 7f8a153bd740 1 --2- 172.21.15.97:0/450340644 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af6254f700 0x55af62566f50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8a04005fd0 tx=0x7f8a04009f90 comp rx=0 tx=0).stop 2024-04-23T10:27:00.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.015+0000 7f8a153bd740 1 -- 172.21.15.97:0/450340644 shutdown_connections 2024-04-23T10:27:00.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.015+0000 7f8a153bd740 1 --2- 172.21.15.97:0/450340644 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55af6256b3f0 0x55af6256d830 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.015+0000 7f8a153bd740 1 --2- 172.21.15.97:0/450340644 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af6254f700 0x55af62566f50 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.015+0000 7f8a153bd740 1 --2- 172.21.15.97:0/450340644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af62547cb0 0x55af6254f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.015+0000 7f8a153bd740 1 --2- 172.21.15.97:0/450340644 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af62547390 0x55af62551d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.015+0000 7f8a153bd740 1 -- 172.21.15.97:0/450340644 >> 172.21.15.97:0/450340644 conn(0x55af625437b0 msgr2=0x55af625495e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:27:00.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.015+0000 7f89eb7fe640 0 client.25268 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:27:00.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.015+0000 7f8a153bd740 1 -- 172.21.15.97:0/450340644 shutdown_connections 2024-04-23T10:27:00.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.015+0000 7f8a153bd740 1 -- 172.21.15.97:0/450340644 wait complete. 2024-04-23T10:27:00.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.015+0000 7f8a153bd740 20 client.25268 trim_cache size 0 max 16384 2024-04-23T10:27:00.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.032+0000 7f8a153bd740 1 Processor -- start 2024-04-23T10:27:00.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.033+0000 7f8a153bd740 1 -- start start 2024-04-23T10:27:00.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.033+0000 7f8a153bd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af62547390 0x55af62542850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:00.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.033+0000 7f8a153bd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af62547cb0 0x55af62573140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:00.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.033+0000 7f8a153bd740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af6254f700 0x55af62573680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:00.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.033+0000 7f8a153bd740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0c1ab870 con 0x55af6254f700 2024-04-23T10:27:00.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.033+0000 7f8a153bd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0c1ad3b0 con 0x55af62547cb0 2024-04-23T10:27:00.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.033+0000 7f8a153bd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0c1abf40 con 0x55af62547390 2024-04-23T10:27:00.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.033+0000 7f8a10c46640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af6254f700 0x55af62573680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:00.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.033+0000 7f8a0bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af62547390 0x55af62542850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:00.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.033+0000 7f8a0bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af62547390 0x55af62542850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.97:59910/0 (socket says 172.21.15.97:59910) 2024-04-23T10:27:00.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.033+0000 7f8a10c46640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af6254f700 0x55af62573680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53158/0 (socket says 172.21.15.97:53158) 2024-04-23T10:27:00.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.033+0000 7f8a0bfff640 1 -- 172.21.15.97:0/3402193007 learned_addr learned my addr 172.21.15.97:0/3402193007 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:27:00.033 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.033+0000 7f8a0b7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af62547cb0 0x55af62573140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:00.036 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.033+0000 7f8a10c46640 1 -- 172.21.15.97:0/3402193007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af62547390 msgr2=0x55af62542850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:27:00.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.033+0000 7f8a10c46640 1 --2- 172.21.15.97:0/3402193007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af62547390 0x55af62542850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.033+0000 7f8a10c46640 1 -- 172.21.15.97:0/3402193007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af62547cb0 msgr2=0x55af62573140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:27:00.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.033+0000 7f8a10c46640 1 --2- 172.21.15.97:0/3402193007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af62547cb0 0x55af62573140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.033+0000 7f8a10c46640 1 -- 172.21.15.97:0/3402193007 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af6253f170 con 0x55af6254f700 2024-04-23T10:27:00.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.034+0000 7f8a10c46640 1 --2- 172.21.15.97:0/3402193007 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af6254f700 0x55af62573680 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f8a0401b820 tx=0x7f8a04009af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83301685876f2533 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:27:00.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.034+0000 7f89f1ffb640 1 -- 172.21.15.97:0/3402193007 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0402b070 con 0x55af6254f700 2024-04-23T10:27:00.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.034+0000 7f89f1ffb640 1 -- 172.21.15.97:0/3402193007 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a04011770 con 0x55af6254f700 2024-04-23T10:27:00.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.034+0000 7f89f1ffb640 1 -- 172.21.15.97:0/3402193007 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a04025b90 con 0x55af6254f700 2024-04-23T10:27:00.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.035+0000 7f8a153bd740 1 -- 172.21.15.97:0/3402193007 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af6254f700 msgr2=0x55af62573680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:00.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.035+0000 7f8a153bd740 1 --2- 172.21.15.97:0/3402193007 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af6254f700 0x55af62573680 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f8a0401b820 tx=0x7f8a04009af0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.035+0000 7f8a153bd740 1 -- 172.21.15.97:0/3402193007 shutdown_connections 2024-04-23T10:27:00.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.035+0000 7f8a153bd740 1 --2- 172.21.15.97:0/3402193007 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af6254f700 0x55af62573680 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.037 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.035+0000 7f8a153bd740 1 --2- 172.21.15.97:0/3402193007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af62547cb0 0x55af62573140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.035+0000 7f8a153bd740 1 --2- 172.21.15.97:0/3402193007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af62547390 0x55af62542850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.035+0000 7f8a153bd740 1 -- 172.21.15.97:0/3402193007 >> 172.21.15.97:0/3402193007 conn(0x55af625437b0 msgr2=0x55af62542140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:27:00.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.035+0000 7f8a153bd740 1 -- 172.21.15.97:0/3402193007 shutdown_connections 2024-04-23T10:27:00.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.035+0000 7f8a153bd740 1 -- 172.21.15.97:0/3402193007 wait complete. 2024-04-23T10:27:00.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.035+0000 7f8a153bd740 1 -- start start 2024-04-23T10:27:00.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.035+0000 7f8a153bd740 1 Processor -- start 2024-04-23T10:27:00.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.036+0000 7f8a153bd740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af62547390 0x55af6254b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:00.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.036+0000 7f8a153bd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af62547cb0 0x55af6254b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:00.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.036+0000 7f8a0bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af62547390 0x55af6254b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:00.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.036+0000 7f8a0bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af62547390 0x55af6254b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53164/0 (socket says 172.21.15.97:53164) 2024-04-23T10:27:00.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.036+0000 7f8a0bfff640 1 -- 172.21.15.97:0/2057536512 learned_addr learned my addr 172.21.15.97:0/2057536512 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:27:00.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.036+0000 7f8a153bd740 1 --2- 172.21.15.97:0/2057536512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af6254f700 0x55af62561eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:00.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.036+0000 7f8a153bd740 1 -- 172.21.15.97:0/2057536512 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0c1acce0 con 0x55af62547390 2024-04-23T10:27:00.038 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.036+0000 7f8a153bd740 1 -- 172.21.15.97:0/2057536512 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0c1ac610 con 0x55af6254f700 2024-04-23T10:27:00.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.036+0000 7f8a153bd740 1 -- 172.21.15.97:0/2057536512 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0c1a9850 con 0x55af62547cb0 2024-04-23T10:27:00.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.036+0000 7f8a153bd740 10 client.0 _ll_register_callbacks cb 0x7f8a04003e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:27:00.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.036+0000 7f8a0b7fe640 1 --2- 172.21.15.97:0/2057536512 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af62547cb0 0x55af6254b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:00.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.036+0000 7f8a10c46640 1 --2- 172.21.15.97:0/2057536512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af6254f700 0x55af62561eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:00.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.036+0000 7f8a0bfff640 1 -- 172.21.15.97:0/2057536512 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af62547cb0 msgr2=0x55af6254b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:27:00.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.036+0000 7f8a0bfff640 1 --2- 172.21.15.97:0/2057536512 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af62547cb0 0x55af6254b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.036+0000 7f8a0bfff640 1 -- 172.21.15.97:0/2057536512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af6254f700 msgr2=0x55af62561eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:27:00.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.036+0000 7f8a0bfff640 1 --2- 172.21.15.97:0/2057536512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af6254f700 0x55af62561eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.036+0000 7f8a0bfff640 1 -- 172.21.15.97:0/2057536512 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af625623f0 con 0x55af62547390 2024-04-23T10:27:00.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.036+0000 7f8a10c46640 1 --2- 172.21.15.97:0/2057536512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af6254f700 0x55af62561eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:27:00.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.036+0000 7f8a0bfff640 1 --2- 172.21.15.97:0/2057536512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af62547390 0x55af6254b2b0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f89fc09aa20 tx=0x7f89fc005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a771e6ae2ffa713e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:27:00.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.036+0000 7f89eb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:27:00.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.037+0000 7f89eb7fe640 1 -- 172.21.15.97:0/2057536512 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89fc020040 con 0x55af62547390 2024-04-23T10:27:00.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.037+0000 7f89eb7fe640 1 -- 172.21.15.97:0/2057536512 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f89fc098de0 con 0x55af62547390 2024-04-23T10:27:00.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.037+0000 7f8a153bd740 1 -- 172.21.15.97:0/2057536512 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a04011770 con 0x55af62547390 2024-04-23T10:27:00.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.037+0000 7f89eb7fe640 1 -- 172.21.15.97:0/2057536512 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f89fc0a0b20 con 0x55af62547390 2024-04-23T10:27:00.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.037+0000 7f8a153bd740 10 client.15783 fetch_fsmap learned FSMap version 26 2024-04-23T10:27:00.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.037+0000 7f8a153bd740 1 -- 172.21.15.97:0/2057536512 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55af625636a0 con 0x55af62547390 2024-04-23T10:27:00.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.037+0000 7f89eb7fe640 1 -- 172.21.15.97:0/2057536512 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89fc080070 con 0x55af62547390 2024-04-23T10:27:00.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.037+0000 7f89eb7fe640 1 -- 172.21.15.97:0/2057536512 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f89fc004170 con 0x55af62547390 2024-04-23T10:27:00.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.037+0000 7f8a153bd740 10 client.15783 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:27:00.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.037+0000 7f8a153bd740 10 client.15783 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:27:00.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.037+0000 7f8a153bd740 20 client.15783 populate_metadata read hostname 'smithi097' 2024-04-23T10:27:00.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.037+0000 7f8a153bd740 1 --2- 172.21.15.97:0/2057536512 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55af625d19b0 0x55af625d3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:00.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.037+0000 7f8a153bd740 4 client.15783 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "55d3c626", "release": true, "await": true}] 2024-04-23T10:27:00.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.037+0000 7f8a153bd740 1 -- 172.21.15.97:0/2057536512 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "quiesce db", "set_id": "55d3c626", "release": true, "await": true}) -- 0x7f8a04011770 con 0x55af625d19b0 2024-04-23T10:27:00.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.038+0000 7f8a0b7fe640 1 --2- 172.21.15.97:0/2057536512 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55af625d19b0 0x55af625d3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:00.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.038+0000 7f8a0b7fe640 1 --2- 172.21.15.97:0/2057536512 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55af625d19b0 0x55af625d3da0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0f739b70a030ccb server_cookie=23db419b8ff43187 in_seq=0 out_seq=0 2024-04-23T10:27:00.041 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.038+0000 7f89eb7fe640 10 client.15783 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:27:00.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.041+0000 7f89eb7fe640 1 -- 172.21.15.97:0/2057536512 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x7f8a04011770 con 0x55af625d19b0 2024-04-23T10:27:00.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.041+0000 7f89eb7fe640 10 client.15783 handle_command_reply: tid=0 2024-04-23T10:27:00.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.041+0000 7f8a153bd740 1 client.15783 shutdown 2024-04-23T10:27:00.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.041+0000 7f8a153bd740 1 -- 172.21.15.97:0/2057536512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af62547390 msgr2=0x55af6254b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:00.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.041+0000 7f8a153bd740 1 --2- 172.21.15.97:0/2057536512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af62547390 0x55af6254b2b0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f89fc09aa20 tx=0x7f89fc005be0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.041+0000 7f8a153bd740 1 -- 172.21.15.97:0/2057536512 shutdown_connections 2024-04-23T10:27:00.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.041+0000 7f8a153bd740 1 --2- 172.21.15.97:0/2057536512 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55af625d19b0 0x55af625d3da0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.041+0000 7f8a153bd740 1 --2- 172.21.15.97:0/2057536512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55af6254f700 0x55af62561eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.041+0000 7f8a153bd740 1 --2- 172.21.15.97:0/2057536512 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55af62547cb0 0x55af6254b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.041+0000 7f8a153bd740 1 --2- 172.21.15.97:0/2057536512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55af62547390 0x55af6254b2b0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.041+0000 7f8a153bd740 1 -- 172.21.15.97:0/2057536512 >> 172.21.15.97:0/2057536512 conn(0x55af625437b0 msgr2=0x55af6255cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:27:00.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.041+0000 7f89eb7fe640 0 client.15783 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:27:00.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.041+0000 7f8a153bd740 1 -- 172.21.15.97:0/2057536512 shutdown_connections 2024-04-23T10:27:00.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.041+0000 7f8a153bd740 1 -- 172.21.15.97:0/2057536512 wait complete. 2024-04-23T10:27:00.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.041+0000 7f8a153bd740 20 client.15783 trim_cache size 0 max 16384 2024-04-23T10:27:00.043 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-23T10:27:00.043 INFO:teuthology.orchestra.run.smithi097.stdout: "epoch": 26, 2024-04-23T10:27:00.043 INFO:teuthology.orchestra.run.smithi097.stdout: "leader": 24488, 2024-04-23T10:27:00.043 INFO:teuthology.orchestra.run.smithi097.stdout: "set_version": 4, 2024-04-23T10:27:00.043 INFO:teuthology.orchestra.run.smithi097.stdout: "sets": { 2024-04-23T10:27:00.043 INFO:teuthology.orchestra.run.smithi097.stdout: "55d3c626": { 2024-04-23T10:27:00.043 INFO:teuthology.orchestra.run.smithi097.stdout: "version": 4, 2024-04-23T10:27:00.043 INFO:teuthology.orchestra.run.smithi097.stdout: "age_ref": 0.0, 2024-04-23T10:27:00.043 INFO:teuthology.orchestra.run.smithi097.stdout: "state": { 2024-04-23T10:27:00.043 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RELEASED", 2024-04-23T10:27:00.043 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0 2024-04-23T10:27:00.044 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:27:00.044 INFO:teuthology.orchestra.run.smithi097.stdout: "timeout": 60.0, 2024-04-23T10:27:00.044 INFO:teuthology.orchestra.run.smithi097.stdout: "expiration": 82.9, 2024-04-23T10:27:00.044 INFO:teuthology.orchestra.run.smithi097.stdout: "members": { 2024-04-23T10:27:00.044 INFO:teuthology.orchestra.run.smithi097.stdout: "file:/": { 2024-04-23T10:27:00.044 INFO:teuthology.orchestra.run.smithi097.stdout: "excluded": false, 2024-04-23T10:27:00.044 INFO:teuthology.orchestra.run.smithi097.stdout: "state": { 2024-04-23T10:27:00.044 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RELEASED", 2024-04-23T10:27:00.044 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0 2024-04-23T10:27:00.044 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:27:00.044 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:27:00.044 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:27:00.044 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:27:00.044 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:27:00.044 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-23T10:27:00.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.042+0000 7f8a126a9640 1 -- 172.21.15.97:0/1077564010 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f89f40a3ec0 msgr2=0x7f89f40a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:00.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.042+0000 7f8a126a9640 1 --2- 172.21.15.97:0/1077564010 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f89f40a3ec0 0x7f89f40a6380 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8a000096c0 tx=0x7f8a000046c0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.045 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.042+0000 7f8a126a9640 1 -- 172.21.15.97:0/1077564010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a0c1005b0 msgr2=0x7f8a0c196470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:00.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.042+0000 7f8a126a9640 1 --2- 172.21.15.97:0/1077564010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a0c1005b0 0x7f8a0c196470 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f89fc005fd0 tx=0x7f89fc00dde0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.042+0000 7f8a126a9640 1 -- 172.21.15.97:0/1077564010 shutdown_connections 2024-04-23T10:27:00.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.042+0000 7f8a126a9640 1 --2- 172.21.15.97:0/1077564010 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f89f40a3ec0 0x7f89f40a6380 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.042+0000 7f8a126a9640 1 --2- 172.21.15.97:0/1077564010 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8a0c106670 0x7f8a0c19ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.042+0000 7f8a126a9640 1 --2- 172.21.15.97:0/1077564010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a0c100f70 0x7f8a0c1969b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.042+0000 7f8a126a9640 1 --2- 172.21.15.97:0/1077564010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a0c1005b0 0x7f8a0c196470 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.042+0000 7f8a126a9640 1 -- 172.21.15.97:0/1077564010 >> 172.21.15.97:0/1077564010 conn(0x7f8a0c0773f0 msgr2=0x7f8a0c0feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:27:00.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.042+0000 7f8a126a9640 1 -- 172.21.15.97:0/1077564010 shutdown_connections 2024-04-23T10:27:00.046 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.042+0000 7f8a126a9640 1 -- 172.21.15.97:0/1077564010 wait complete. 2024-04-23T10:27:00.052 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '55d3c626', seconds elapsed: 24.2 2024-04-23T10:27:00.052 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.9 2024-04-23T10:27:00.052 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T10:27:00.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.188+0000 7fc3d2659640 1 -- 172.21.15.97:0/3035612880 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3cc101a50 msgr2=0x7fc3cc105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:00.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.188+0000 7fc3d2659640 1 --2- 172.21.15.97:0/3035612880 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3cc101a50 0x7fc3cc105a90 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fc3c0009900 tx=0x7fc3c002a410 comp rx=0 tx=0).stop 2024-04-23T10:27:00.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.188+0000 7fc3d2659640 1 -- 172.21.15.97:0/3035612880 shutdown_connections 2024-04-23T10:27:00.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.188+0000 7fc3d2659640 1 --2- 172.21.15.97:0/3035612880 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3cc106170 0x7fc3cc10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.188+0000 7fc3d2659640 1 --2- 172.21.15.97:0/3035612880 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3cc101a50 0x7fc3cc105a90 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.188+0000 7fc3d2659640 1 --2- 172.21.15.97:0/3035612880 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3cc1010a0 0x7fc3cc101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.188+0000 7fc3d2659640 1 -- 172.21.15.97:0/3035612880 >> 172.21.15.97:0/3035612880 conn(0x7fc3cc077f50 msgr2=0x7fc3cc078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:27:00.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.188+0000 7fc3d2659640 1 -- 172.21.15.97:0/3035612880 shutdown_connections 2024-04-23T10:27:00.187 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.188+0000 7fc3d2659640 1 -- 172.21.15.97:0/3035612880 wait complete. 2024-04-23T10:27:00.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.189+0000 7fc3d2659640 1 Processor -- start 2024-04-23T10:27:00.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.189+0000 7fc3d2659640 1 -- start start 2024-04-23T10:27:00.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.189+0000 7fc3d2659640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3cc1010a0 0x7fc3cc199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:00.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.189+0000 7fc3d2659640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3cc106170 0x7fc3cc199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:00.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.189+0000 7fc3d2659640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3cc19a0d0 0x7fc3cc1a4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:00.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.189+0000 7fc3d2659640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc3cc1110b0 con 0x7fc3cc19a0d0 2024-04-23T10:27:00.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.189+0000 7fc3d2659640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc3cc110f30 con 0x7fc3cc106170 2024-04-23T10:27:00.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.189+0000 7fc3d2659640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc3cc111230 con 0x7fc3cc1010a0 2024-04-23T10:27:00.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.189+0000 7fc3d0bf6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3cc19a0d0 0x7fc3cc1a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:00.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.189+0000 7fc3d0bf6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3cc19a0d0 0x7fc3cc1a4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53184/0 (socket says 172.21.15.97:53184) 2024-04-23T10:27:00.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.189+0000 7fc3d0bf6640 1 -- 172.21.15.97:0/1445939885 learned_addr learned my addr 172.21.15.97:0/1445939885 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:27:00.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.189+0000 7fc3cb7fe640 1 --2- 172.21.15.97:0/1445939885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3cc106170 0x7fc3cc199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:00.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.189+0000 7fc3cbfff640 1 --2- 172.21.15.97:0/1445939885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3cc1010a0 0x7fc3cc199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:00.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.190+0000 7fc3d0bf6640 1 -- 172.21.15.97:0/1445939885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3cc1010a0 msgr2=0x7fc3cc199600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:00.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.190+0000 7fc3d0bf6640 1 --2- 172.21.15.97:0/1445939885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3cc1010a0 0x7fc3cc199600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.190+0000 7fc3d0bf6640 1 -- 172.21.15.97:0/1445939885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3cc106170 msgr2=0x7fc3cc199b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:00.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.190+0000 7fc3d0bf6640 1 --2- 172.21.15.97:0/1445939885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3cc106170 0x7fc3cc199b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.190+0000 7fc3d0bf6640 1 -- 172.21.15.97:0/1445939885 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3cc1974f0 con 0x7fc3cc19a0d0 2024-04-23T10:27:00.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.190+0000 7fc3d0bf6640 1 --2- 172.21.15.97:0/1445939885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3cc19a0d0 0x7fc3cc1a4680 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fc3c400eaa0 tx=0x7fc3c400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:27:00.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.190+0000 7fc3c97fa640 1 -- 172.21.15.97:0/1445939885 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c400cdb0 con 0x7fc3cc19a0d0 2024-04-23T10:27:00.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.190+0000 7fc3c97fa640 1 -- 172.21.15.97:0/1445939885 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3c4005020 con 0x7fc3cc19a0d0 2024-04-23T10:27:00.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.190+0000 7fc3c97fa640 1 -- 172.21.15.97:0/1445939885 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c40116d0 con 0x7fc3cc19a0d0 2024-04-23T10:27:00.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.190+0000 7fc3d2659640 1 -- 172.21.15.97:0/1445939885 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3cc1977e0 con 0x7fc3cc19a0d0 2024-04-23T10:27:00.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.190+0000 7fc3d2659640 1 -- 172.21.15.97:0/1445939885 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3cc197cb0 con 0x7fc3cc19a0d0 2024-04-23T10:27:00.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.191+0000 7fc3d2659640 1 -- 172.21.15.97:0/1445939885 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3cc102570 con 0x7fc3cc19a0d0 2024-04-23T10:27:00.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.193+0000 7fc3c97fa640 1 -- 172.21.15.97:0/1445939885 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fc3c40051d0 con 0x7fc3cc19a0d0 2024-04-23T10:27:00.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.194+0000 7fc3c97fa640 1 --2- 172.21.15.97:0/1445939885 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fc3b40a3e40 0x7fc3b40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:00.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.194+0000 7fc3c97fa640 1 -- 172.21.15.97:0/1445939885 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fc3c40d0b30 con 0x7fc3cc19a0d0 2024-04-23T10:27:00.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.195+0000 7fc3cbfff640 1 --2- 172.21.15.97:0/1445939885 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fc3b40a3e40 0x7fc3b40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:00.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.195+0000 7fc3cbfff640 1 --2- 172.21.15.97:0/1445939885 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fc3b40a3e40 0x7fc3b40a6300 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc009bd0 tx=0x7fc3bc005f90 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:27:00.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.199+0000 7fc3c97fa640 1 -- 172.21.15.97:0/1445939885 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc3c4016030 con 0x7fc3cc19a0d0 2024-04-23T10:27:00.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:26:59 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/1399202133' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:27:00.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:26:59 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/1399202133' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:27:00.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:26:59 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/1399202133' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:27:00.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.577+0000 7fc3d2659640 1 -- 172.21.15.97:0/1445939885 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc3cc101a50 con 0x7fc3cc19a0d0 2024-04-23T10:27:00.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.578+0000 7fc3c97fa640 1 -- 172.21.15.97:0/1445939885 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11931 (secure 0 0 0) 0x7fc3c409c920 con 0x7fc3cc19a0d0 2024-04-23T10:27:00.577 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:27:00.577 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3946878762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3946878762},{"type":"v1","addr":"172.21.15.97:6835","nonce":3946878762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/209800861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":209800861},{"type":"v1","addr":"172.21.15.97:6837","nonce":209800861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T10:24:06.910134+0000","modified":"2024-04-23T10:26:09.820985+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6839/418013575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":418013575},{"type":"v1","addr":"172.21.15.97:6839","nonce":418013575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6841/747242680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":747242680},{"type":"v1","addr":"172.21.15.97:6841","nonce":747242680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6833/4292847121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4292847121},{"type":"v1","addr":"172.21.15.183:6833","nonce":4292847121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6833/3885816565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3885816565},{"type":"v1","addr":"172.21.15.196:6833","nonce":3885816565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1897093105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1897093105},{"type":"v1","addr":"172.21.15.183:6835","nonce":1897093105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1171477563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1171477563},{"type":"v1","addr":"172.21.15.196:6835","nonce":1171477563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1898341911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1898341911},{"type":"v1","addr":"172.21.15.183:6837","nonce":1898341911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.196:6837/3094946502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3094946502},{"type":"v1","addr":"172.21.15.196:6837","nonce":3094946502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.196:6839/83843556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":83843556},{"type":"v1","addr":"172.21.15.196:6839","nonce":83843556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6839/1763841130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1763841130},{"type":"v1","addr":"172.21.15.183:6839","nonce":1763841130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24487,24403,24398,24469,24488]},"id":1}]} 2024-04-23T10:27:00.578 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 26 2024-04-23T10:27:00.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.581+0000 7fc3d2659640 1 -- 172.21.15.97:0/1445939885 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fc3b40a3e40 msgr2=0x7fc3b40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:00.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.581+0000 7fc3d2659640 1 --2- 172.21.15.97:0/1445939885 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fc3b40a3e40 0x7fc3b40a6300 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc009bd0 tx=0x7fc3bc005f90 comp rx=0 tx=0).stop 2024-04-23T10:27:00.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.581+0000 7fc3d2659640 1 -- 172.21.15.97:0/1445939885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3cc19a0d0 msgr2=0x7fc3cc1a4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:00.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.581+0000 7fc3d2659640 1 --2- 172.21.15.97:0/1445939885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3cc19a0d0 0x7fc3cc1a4680 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fc3c400eaa0 tx=0x7fc3c400ef60 comp rx=0 tx=0).stop 2024-04-23T10:27:00.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.581+0000 7fc3d2659640 1 -- 172.21.15.97:0/1445939885 shutdown_connections 2024-04-23T10:27:00.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.581+0000 7fc3d2659640 1 --2- 172.21.15.97:0/1445939885 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fc3b40a3e40 0x7fc3b40a6300 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.581+0000 7fc3d2659640 1 --2- 172.21.15.97:0/1445939885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3cc19a0d0 0x7fc3cc1a4680 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.581+0000 7fc3d2659640 1 --2- 172.21.15.97:0/1445939885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3cc106170 0x7fc3cc199b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.581+0000 7fc3d2659640 1 --2- 172.21.15.97:0/1445939885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3cc1010a0 0x7fc3cc199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.581+0000 7fc3d2659640 1 -- 172.21.15.97:0/1445939885 >> 172.21.15.97:0/1445939885 conn(0x7fc3cc077f50 msgr2=0x7fc3cc0ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:27:00.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.581+0000 7fc3d2659640 1 -- 172.21.15.97:0/1445939885 shutdown_connections 2024-04-23T10:27:00.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.581+0000 7fc3d2659640 1 -- 172.21.15.97:0/1445939885 wait complete. 2024-04-23T10:27:00.595 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db / --timeout 60 --expiration 82.9 --await-for 42.0' 2024-04-23T10:27:00.595 DEBUG:teuthology.orchestra.run.smithi097:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24488 quiesce db / --timeout 60 --expiration 82.9 --await-for 42.0 2024-04-23T10:27:00.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.749+0000 7fa2d899b640 1 -- 172.21.15.97:0/1872602399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2d4102f10 msgr2=0x7fa2d4100460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:00.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.749+0000 7fa2d899b640 1 --2- 172.21.15.97:0/1872602399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2d4102f10 0x7fa2d4100460 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fa2c4009900 tx=0x7fa2c402a410 comp rx=0 tx=0).stop 2024-04-23T10:27:00.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.749+0000 7fa2d899b640 1 -- 172.21.15.97:0/1872602399 shutdown_connections 2024-04-23T10:27:00.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.749+0000 7fa2d899b640 1 --2- 172.21.15.97:0/1872602399 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa2d410ebe0 0x7fa2d4110fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.749+0000 7fa2d899b640 1 --2- 172.21.15.97:0/1872602399 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2d4103380 0x7fa2d41009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.749+0000 7fa2d899b640 1 --2- 172.21.15.97:0/1872602399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2d4102f10 0x7fa2d4100460 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.749+0000 7fa2d899b640 1 -- 172.21.15.97:0/1872602399 >> 172.21.15.97:0/1872602399 conn(0x7fa2d40773f0 msgr2=0x7fa2d40fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:27:00.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.749+0000 7fa2d899b640 1 -- 172.21.15.97:0/1872602399 shutdown_connections 2024-04-23T10:27:00.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.750+0000 7fa2d899b640 1 -- 172.21.15.97:0/1872602399 wait complete. 2024-04-23T10:27:00.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.750+0000 7fa2d899b640 1 Processor -- start 2024-04-23T10:27:00.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.750+0000 7fa2d899b640 1 -- start start 2024-04-23T10:27:00.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.751+0000 7fa2d899b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2d4102f10 0x7fa2d41a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:00.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.751+0000 7fa2d899b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2d4103380 0x7fa2d41a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:00.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.751+0000 7fa2d899b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa2d410ebe0 0x7fa2d419a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:00.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.751+0000 7fa2d899b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d4114460 con 0x7fa2d4103380 2024-04-23T10:27:00.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.751+0000 7fa2d899b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d41142e0 con 0x7fa2d410ebe0 2024-04-23T10:27:00.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.751+0000 7fa2d899b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d41145e0 con 0x7fa2d4102f10 2024-04-23T10:27:00.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.751+0000 7fa2d1d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2d4103380 0x7fa2d41a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:00.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.751+0000 7fa2d1d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2d4103380 0x7fa2d41a0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53208/0 (socket says 172.21.15.97:53208) 2024-04-23T10:27:00.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.751+0000 7fa2d1d9b640 1 -- 172.21.15.97:0/3137715550 learned_addr learned my addr 172.21.15.97:0/3137715550 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:27:00.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.751+0000 7fa2d259c640 1 --2- 172.21.15.97:0/3137715550 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2d4102f10 0x7fa2d41a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:00.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.751+0000 7fa2d2d9d640 1 --2- 172.21.15.97:0/3137715550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa2d410ebe0 0x7fa2d419a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:00.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.751+0000 7fa2d1d9b640 1 -- 172.21.15.97:0/3137715550 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2d4102f10 msgr2=0x7fa2d41a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:00.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.751+0000 7fa2d1d9b640 1 --2- 172.21.15.97:0/3137715550 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2d4102f10 0x7fa2d41a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.751+0000 7fa2d1d9b640 1 -- 172.21.15.97:0/3137715550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa2d410ebe0 msgr2=0x7fa2d419a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:00.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.751+0000 7fa2d1d9b640 1 --2- 172.21.15.97:0/3137715550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa2d410ebe0 0x7fa2d419a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.751+0000 7fa2d1d9b640 1 -- 172.21.15.97:0/3137715550 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2d419afd0 con 0x7fa2d4103380 2024-04-23T10:27:00.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.752+0000 7fa2d1d9b640 1 --2- 172.21.15.97:0/3137715550 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2d4103380 0x7fa2d41a0bd0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fa2c800e9f0 tx=0x7fa2c800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:27:00.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.752+0000 7fa2c37fe640 1 -- 172.21.15.97:0/3137715550 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c800ccc0 con 0x7fa2d4103380 2024-04-23T10:27:00.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.752+0000 7fa2c37fe640 1 -- 172.21.15.97:0/3137715550 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2c80054d0 con 0x7fa2d4103380 2024-04-23T10:27:00.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.752+0000 7fa2c37fe640 1 -- 172.21.15.97:0/3137715550 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c8010430 con 0x7fa2d4103380 2024-04-23T10:27:00.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.752+0000 7fa2d899b640 1 -- 172.21.15.97:0/3137715550 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2d419b2c0 con 0x7fa2d4103380 2024-04-23T10:27:00.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.753+0000 7fa2d899b640 1 -- 172.21.15.97:0/3137715550 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2d41a74e0 con 0x7fa2d4103380 2024-04-23T10:27:00.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.755+0000 7fa2c37fe640 1 -- 172.21.15.97:0/3137715550 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fa2c80046c0 con 0x7fa2d4103380 2024-04-23T10:27:00.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.756+0000 7fa2db78f740 1 Processor -- start 2024-04-23T10:27:00.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.756+0000 7fa2db78f740 1 -- start start 2024-04-23T10:27:00.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.756+0000 7fa2db78f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a0605690a0 0x55a060569480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:00.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.756+0000 7fa2db78f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a0605699c0 0x55a060573b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:00.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.756+0000 7fa2db78f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a060574090 0x55a060576480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:00.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.756+0000 7fa2db78f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55a0604ce370 con 0x55a0605699c0 2024-04-23T10:27:00.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.756+0000 7fa2db78f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a06046b2e0 con 0x55a060574090 2024-04-23T10:27:00.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.756+0000 7fa2db78f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a0604ca330 con 0x55a0605690a0 2024-04-23T10:27:00.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.756+0000 7fa2c37fe640 1 --2- 172.21.15.97:0/3137715550 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fa2bc0a3ec0 0x7fa2bc0a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:00.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.756+0000 7fa2d259c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a0605699c0 0x55a060573b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:00.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.757+0000 7fa2d2d9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a0605690a0 0x55a060569480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:00.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.757+0000 7fa2d2d9d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a0605690a0 0x55a060569480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.97:59954/0 (socket says 172.21.15.97:59954) 2024-04-23T10:27:00.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.757+0000 7fa2d2d9d640 1 -- 172.21.15.97:0/1567657414 learned_addr learned my addr 172.21.15.97:0/1567657414 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:27:00.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.757+0000 7fa2d1d9b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a060574090 0x55a060576480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:00.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.757+0000 7fa2d2d9d640 1 --2- 172.21.15.97:0/3137715550 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fa2bc0a3ec0 0x7fa2bc0a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:00.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.757+0000 7fa2c37fe640 1 -- 172.21.15.97:0/3137715550 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fa2c80d0d80 con 0x7fa2d4103380 2024-04-23T10:27:00.756 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.757+0000 7fa2d259c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a0605699c0 0x55a060573b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53214/0 (socket says 172.21.15.97:53214) 2024-04-23T10:27:00.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.757+0000 7fa2d2d9d640 1 -- 172.21.15.97:0/1567657414 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a060574090 msgr2=0x55a060576480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:27:00.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.757+0000 7fa2d2d9d640 1 --2- 172.21.15.97:0/1567657414 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a060574090 0x55a060576480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.757+0000 7fa2d2d9d640 1 -- 172.21.15.97:0/1567657414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a0605699c0 msgr2=0x55a060573b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:27:00.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.757+0000 7fa2d2d9d640 1 --2- 172.21.15.97:0/1567657414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a0605699c0 0x55a060573b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.757+0000 7fa2d2d9d640 1 -- 172.21.15.97:0/1567657414 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0602acd80 con 0x55a0605690a0 2024-04-23T10:27:00.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.757+0000 7fa2d2d9d640 1 --2- 172.21.15.97:0/3137715550 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fa2bc0a3ec0 0x7fa2bc0a6380 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc009fd0 tx=0x7fa2cc017000 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:27:00.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.757+0000 7fa2d2d9d640 1 --2- 172.21.15.97:0/1567657414 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a0605690a0 0x55a060569480 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc005c40 tx=0x7fa2cc010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0d24671b6d83a77 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:27:00.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.758+0000 7fa2c0ff9640 1 -- 172.21.15.97:0/1567657414 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc023030 con 0x55a0605690a0 2024-04-23T10:27:00.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.758+0000 7fa2c0ff9640 1 -- 172.21.15.97:0/1567657414 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2cc009d30 con 0x55a0605690a0 2024-04-23T10:27:00.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.758+0000 7fa2c0ff9640 1 -- 172.21.15.97:0/1567657414 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc003970 con 0x55a0605690a0 2024-04-23T10:27:00.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.758+0000 7fa2db78f740 1 -- 172.21.15.97:0/1567657414 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a0605690a0 msgr2=0x55a060569480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:00.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.758+0000 7fa2db78f740 1 --2- 172.21.15.97:0/1567657414 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a0605690a0 0x55a060569480 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc005c40 tx=0x7fa2cc010740 comp rx=0 tx=0).stop 2024-04-23T10:27:00.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.759+0000 7fa2db78f740 1 -- 172.21.15.97:0/1567657414 shutdown_connections 2024-04-23T10:27:00.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.759+0000 7fa2db78f740 1 --2- 172.21.15.97:0/1567657414 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a060574090 0x55a060576480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.759+0000 7fa2db78f740 1 --2- 172.21.15.97:0/1567657414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a0605699c0 0x55a060573b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.759+0000 7fa2db78f740 1 --2- 172.21.15.97:0/1567657414 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a0605690a0 0x55a060569480 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.759+0000 7fa2db78f740 1 -- 172.21.15.97:0/1567657414 >> 172.21.15.97:0/1567657414 conn(0x55a0605653c0 msgr2=0x55a0605657a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:27:00.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.759+0000 7fa2db78f740 1 -- 172.21.15.97:0/1567657414 shutdown_connections 2024-04-23T10:27:00.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.759+0000 7fa2db78f740 1 -- 172.21.15.97:0/1567657414 wait complete. 2024-04-23T10:27:00.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.759+0000 7fa2db78f740 1 -- start start 2024-04-23T10:27:00.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.759+0000 7fa2db78f740 1 Processor -- start 2024-04-23T10:27:00.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.759+0000 7fa2db78f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a0605690a0 0x55a060571000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:00.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.759+0000 7fa2db78f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0605699c0 0x55a060571540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:00.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.759+0000 7fa2db78f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a060574090 0x55a060571a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:00.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.759+0000 7fa2db78f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55a060576e20 con 0x55a060574090 2024-04-23T10:27:00.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.760+0000 7fa2db78f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a060468fb0 con 0x55a0605699c0 2024-04-23T10:27:00.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.760+0000 7fa2db78f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a060576fa0 con 0x55a0605690a0 2024-04-23T10:27:00.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.760+0000 7fa2db78f740 10 client.0 _ll_register_callbacks cb 0x55a0604689a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:27:00.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.760+0000 7fa2d1d9b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0605699c0 0x55a060571540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:00.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.760+0000 7fa2d1d9b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0605699c0 0x55a060571540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.97:55380/0 (socket says 172.21.15.97:55380) 2024-04-23T10:27:00.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.760+0000 7fa2d259c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a0605690a0 0x55a060571000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:00.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.760+0000 7fa2d1d9b640 1 -- 172.21.15.97:0/1006916072 learned_addr learned my addr 172.21.15.97:0/1006916072 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:27:00.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.760+0000 7fa2d2d9d640 1 --2- 172.21.15.97:0/1006916072 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a060574090 0x55a060571a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:00.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.760+0000 7fa2d1d9b640 1 -- 172.21.15.97:0/1006916072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a0605690a0 msgr2=0x55a060571000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:27:00.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.760+0000 7fa2d1d9b640 1 --2- 172.21.15.97:0/1006916072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a0605690a0 0x55a060571000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.760+0000 7fa2d1d9b640 1 -- 172.21.15.97:0/1006916072 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a060574090 msgr2=0x55a060571a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:27:00.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.760+0000 7fa2d1d9b640 1 --2- 172.21.15.97:0/1006916072 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a060574090 0x55a060571a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.760+0000 7fa2d1d9b640 1 -- 172.21.15.97:0/1006916072 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a06030d510 con 0x55a0605699c0 2024-04-23T10:27:00.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.760+0000 7fa2d1d9b640 1 --2- 172.21.15.97:0/1006916072 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0605699c0 0x55a060571540 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa2c80994f0 tx=0x7fa2c80073b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d76dcad0f48900e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:27:00.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.760+0000 7fa2b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-23T10:27:00.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.760+0000 7fa2b9ffb640 1 -- 172.21.15.97:0/1006916072 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c80b6060 con 0x55a0605699c0 2024-04-23T10:27:00.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.761+0000 7fa2db78f740 1 -- 172.21.15.97:0/1006916072 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2cc009d30 con 0x55a0605699c0 2024-04-23T10:27:00.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.761+0000 7fa2b9ffb640 1 -- 172.21.15.97:0/1006916072 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fa2c806b1d0 con 0x55a0605699c0 2024-04-23T10:27:00.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.761+0000 7fa2b9ffb640 1 -- 172.21.15.97:0/1006916072 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa2c806b030 con 0x55a0605699c0 2024-04-23T10:27:00.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.761+0000 7fa2db78f740 10 client.25292 fetch_fsmap learned FSMap version 26 2024-04-23T10:27:00.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.761+0000 7fa2db78f740 1 -- 172.21.15.97:0/1006916072 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a0601a0670 con 0x55a0605699c0 2024-04-23T10:27:00.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.761+0000 7fa2b9ffb640 1 -- 172.21.15.97:0/1006916072 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2cc009d30 con 0x55a0605699c0 2024-04-23T10:27:00.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.761+0000 7fa2b9ffb640 1 -- 172.21.15.97:0/1006916072 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa2c80b8050 con 0x55a0605699c0 2024-04-23T10:27:00.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.761+0000 7fa2db78f740 10 client.25292 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:27:00.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.761+0000 7fa2db78f740 10 client.25292 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:27:00.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.761+0000 7fa2db78f740 20 client.25292 populate_metadata read hostname 'smithi097' 2024-04-23T10:27:00.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.761+0000 7fa2db78f740 1 --2- 172.21.15.97:0/1006916072 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55a060584320 0x55a060586710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:00.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.761+0000 7fa2db78f740 4 client.25292 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:27:00.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.761+0000 7fa2db78f740 1 -- 172.21.15.97:0/1006916072 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0604dbd90 con 0x55a060584320 2024-04-23T10:27:00.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.762+0000 7fa2d259c640 1 --2- 172.21.15.97:0/1006916072 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55a060584320 0x55a060586710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:00.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.762+0000 7fa2b9ffb640 10 client.25292 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:27:00.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.762+0000 7fa2d259c640 1 --2- 172.21.15.97:0/1006916072 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55a060584320 0x55a060586710 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fc7c8178b24919a server_cookie=66e728be08acb2b8 in_seq=0 out_seq=0 2024-04-23T10:27:00.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.763+0000 7fa2b9ffb640 1 -- 172.21.15.97:0/1006916072 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a0604dbd90 con 0x55a060584320 2024-04-23T10:27:00.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.763+0000 7fa2b9ffb640 10 client.25292 handle_command_reply: tid=0 2024-04-23T10:27:00.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.763+0000 7fa2db78f740 1 client.25292 shutdown 2024-04-23T10:27:00.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.763+0000 7fa2db78f740 1 -- 172.21.15.97:0/1006916072 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0605699c0 msgr2=0x55a060571540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:00.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.763+0000 7fa2db78f740 1 --2- 172.21.15.97:0/1006916072 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0605699c0 0x55a060571540 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa2c80994f0 tx=0x7fa2c80073b0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.763+0000 7fa2db78f740 1 -- 172.21.15.97:0/1006916072 shutdown_connections 2024-04-23T10:27:00.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.763+0000 7fa2db78f740 1 --2- 172.21.15.97:0/1006916072 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55a060584320 0x55a060586710 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.763+0000 7fa2db78f740 1 --2- 172.21.15.97:0/1006916072 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a060574090 0x55a060571a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.763+0000 7fa2b9ffb640 0 client.25292 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:27:00.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.763+0000 7fa2db78f740 1 --2- 172.21.15.97:0/1006916072 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0605699c0 0x55a060571540 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.763+0000 7fa2db78f740 1 --2- 172.21.15.97:0/1006916072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a0605690a0 0x55a060571000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.763+0000 7fa2db78f740 1 -- 172.21.15.97:0/1006916072 >> 172.21.15.97:0/1006916072 conn(0x55a0605653c0 msgr2=0x55a06056b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:27:00.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.763+0000 7fa2db78f740 1 -- 172.21.15.97:0/1006916072 shutdown_connections 2024-04-23T10:27:00.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.763+0000 7fa2db78f740 1 -- 172.21.15.97:0/1006916072 wait complete. 2024-04-23T10:27:00.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.763+0000 7fa2db78f740 20 client.25292 trim_cache size 0 max 16384 2024-04-23T10:27:00.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.793+0000 7fa2db78f740 1 Processor -- start 2024-04-23T10:27:00.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.793+0000 7fa2db78f740 1 -- start start 2024-04-23T10:27:00.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.793+0000 7fa2db78f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a0605690a0 0x55a06057fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:00.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.793+0000 7fa2db78f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a0605699c0 0x55a060571630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:00.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.793+0000 7fa2db78f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a060574090 0x55a0605708e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:00.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.793+0000 7fa2db78f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d41afc30 con 0x55a0605699c0 2024-04-23T10:27:00.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.793+0000 7fa2db78f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d41b1770 con 0x55a060574090 2024-04-23T10:27:00.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.793+0000 7fa2db78f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d41b0300 con 0x55a0605690a0 2024-04-23T10:27:00.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.793+0000 7fa2d1d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a0605699c0 0x55a060571630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:00.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.793+0000 7fa2d1d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a0605699c0 0x55a060571630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53226/0 (socket says 172.21.15.97:53226) 2024-04-23T10:27:00.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.793+0000 7fa2d1d9b640 1 -- 172.21.15.97:0/1130889699 learned_addr learned my addr 172.21.15.97:0/1130889699 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:27:00.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.793+0000 7fa2d259c640 1 --2- 172.21.15.97:0/1130889699 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a0605690a0 0x55a06057fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:00.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.793+0000 7fa2d2d9d640 1 --2- 172.21.15.97:0/1130889699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a060574090 0x55a0605708e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:00.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.793+0000 7fa2d1d9b640 1 -- 172.21.15.97:0/1130889699 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a0605690a0 msgr2=0x55a06057fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:27:00.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.793+0000 7fa2d1d9b640 1 --2- 172.21.15.97:0/1130889699 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a0605690a0 0x55a06057fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.793+0000 7fa2d1d9b640 1 -- 172.21.15.97:0/1130889699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a060574090 msgr2=0x55a0605708e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:27:00.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.793+0000 7fa2d1d9b640 1 --2- 172.21.15.97:0/1130889699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a060574090 0x55a0605708e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.794+0000 7fa2d1d9b640 1 -- 172.21.15.97:0/1130889699 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a060565f90 con 0x55a0605699c0 2024-04-23T10:27:00.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.794+0000 7fa2d1d9b640 1 --2- 172.21.15.97:0/1130889699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a0605699c0 0x55a060571630 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fa2c80994a0 tx=0x7fa2c8004a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2be05d487ecc52c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:27:00.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.794+0000 7fa2bbfff640 1 -- 172.21.15.97:0/1130889699 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c80b6040 con 0x55a0605699c0 2024-04-23T10:27:00.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.794+0000 7fa2bbfff640 1 -- 172.21.15.97:0/1130889699 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2c8096030 con 0x55a0605699c0 2024-04-23T10:27:00.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.794+0000 7fa2bbfff640 1 -- 172.21.15.97:0/1130889699 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c80a5560 con 0x55a0605699c0 2024-04-23T10:27:00.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.795+0000 7fa2db78f740 1 -- 172.21.15.97:0/1130889699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a0605699c0 msgr2=0x55a060571630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:00.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.795+0000 7fa2db78f740 1 --2- 172.21.15.97:0/1130889699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a0605699c0 0x55a060571630 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fa2c80994a0 tx=0x7fa2c8004a60 comp rx=0 tx=0).stop 2024-04-23T10:27:00.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.795+0000 7fa2db78f740 1 -- 172.21.15.97:0/1130889699 shutdown_connections 2024-04-23T10:27:00.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.795+0000 7fa2db78f740 1 --2- 172.21.15.97:0/1130889699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a060574090 0x55a0605708e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.795+0000 7fa2db78f740 1 --2- 172.21.15.97:0/1130889699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a0605699c0 0x55a060571630 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.795+0000 7fa2db78f740 1 --2- 172.21.15.97:0/1130889699 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a0605690a0 0x55a06057fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.795+0000 7fa2db78f740 1 -- 172.21.15.97:0/1130889699 >> 172.21.15.97:0/1130889699 conn(0x55a0605653c0 msgr2=0x55a0605730c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:27:00.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.795+0000 7fa2db78f740 1 -- 172.21.15.97:0/1130889699 shutdown_connections 2024-04-23T10:27:00.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.795+0000 7fa2db78f740 1 -- 172.21.15.97:0/1130889699 wait complete. 2024-04-23T10:27:00.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.795+0000 7fa2db78f740 1 -- start start 2024-04-23T10:27:00.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.795+0000 7fa2db78f740 1 Processor -- start 2024-04-23T10:27:00.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.796+0000 7fa2db78f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a0605690a0 0x55a06056e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:00.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.796+0000 7fa2db78f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a0605699c0 0x55a06056e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:00.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.796+0000 7fa2db78f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a060574090 0x55a0605e9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:00.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.796+0000 7fa2db78f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d41b10a0 con 0x55a0605690a0 2024-04-23T10:27:00.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.796+0000 7fa2db78f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d41b09d0 con 0x55a060574090 2024-04-23T10:27:00.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.796+0000 7fa2db78f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d41adc10 con 0x55a0605699c0 2024-04-23T10:27:00.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.796+0000 7fa2db78f740 10 client.0 _ll_register_callbacks cb 0x7fa2c809c660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:27:00.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.796+0000 7fa2d259c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a0605690a0 0x55a06056e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:00.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.796+0000 7fa2d2d9d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a060574090 0x55a0605e9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:00.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.796+0000 7fa2d1d9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a0605699c0 0x55a06056e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:00.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.796+0000 7fa2d2d9d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a060574090 0x55a0605e9800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.97:55408/0 (socket says 172.21.15.97:55408) 2024-04-23T10:27:00.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.796+0000 7fa2d259c640 1 --2- 172.21.15.97:0/3488230339 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a0605690a0 0x55a06056e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53234/0 (socket says 172.21.15.97:53234) 2024-04-23T10:27:00.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.796+0000 7fa2d2d9d640 1 -- 172.21.15.97:0/3488230339 learned_addr learned my addr 172.21.15.97:0/3488230339 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:27:00.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.796+0000 7fa2d2d9d640 1 -- 172.21.15.97:0/3488230339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a0605699c0 msgr2=0x55a06056e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:27:00.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.796+0000 7fa2d2d9d640 1 --2- 172.21.15.97:0/3488230339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a0605699c0 0x55a06056e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.796+0000 7fa2d2d9d640 1 -- 172.21.15.97:0/3488230339 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a0605690a0 msgr2=0x55a06056e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:27:00.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.796+0000 7fa2d2d9d640 1 --2- 172.21.15.97:0/3488230339 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a0605690a0 0x55a06056e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.796+0000 7fa2d2d9d640 1 -- 172.21.15.97:0/3488230339 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0605e9d40 con 0x55a060574090 2024-04-23T10:27:00.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.796+0000 7fa2d2d9d640 1 --2- 172.21.15.97:0/3488230339 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a060574090 0x55a0605e9800 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc005c40 tx=0x7fa2cc012c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a7163592a9fee56 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:27:00.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.796+0000 7fa2b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-04-23T10:27:00.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.797+0000 7fa2b9ffb640 1 -- 172.21.15.97:0/3488230339 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc003970 con 0x55a060574090 2024-04-23T10:27:00.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.797+0000 7fa2b9ffb640 1 -- 172.21.15.97:0/3488230339 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fa2cc012870 con 0x55a060574090 2024-04-23T10:27:00.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.797+0000 7fa2db78f740 1 -- 172.21.15.97:0/3488230339 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2c8096030 con 0x55a060574090 2024-04-23T10:27:00.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.797+0000 7fa2b9ffb640 1 -- 172.21.15.97:0/3488230339 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa2cc0050c0 con 0x55a060574090 2024-04-23T10:27:00.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.797+0000 7fa2db78f740 10 client.25298 fetch_fsmap learned FSMap version 26 2024-04-23T10:27:00.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.797+0000 7fa2db78f740 1 -- 172.21.15.97:0/3488230339 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a0605eafc0 con 0x55a060574090 2024-04-23T10:27:00.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.797+0000 7fa2b9ffb640 1 -- 172.21.15.97:0/3488230339 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2cc005710 con 0x55a060574090 2024-04-23T10:27:00.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.797+0000 7fa2b9ffb640 1 -- 172.21.15.97:0/3488230339 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa2cc01f8f0 con 0x55a060574090 2024-04-23T10:27:00.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.797+0000 7fa2db78f740 10 client.25298 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:27:00.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.797+0000 7fa2db78f740 10 client.25298 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:27:00.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.797+0000 7fa2db78f740 20 client.25298 populate_metadata read hostname 'smithi097' 2024-04-23T10:27:00.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.797+0000 7fa2db78f740 1 --2- 172.21.15.97:0/3488230339 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55a0605eb4a0 0x55a0605ed8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:00.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.797+0000 7fa2db78f740 4 client.25298 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 82.9, "await_for": 42.0}] 2024-04-23T10:27:00.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.797+0000 7fa2db78f740 1 -- 172.21.15.97:0/3488230339 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 82.9, "await_for": 42.0}) -- 0x7fa2c8096030 con 0x55a0605eb4a0 2024-04-23T10:27:00.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.798+0000 7fa2d259c640 1 --2- 172.21.15.97:0/3488230339 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55a0605eb4a0 0x55a0605ed8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:00.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.798+0000 7fa2d259c640 1 --2- 172.21.15.97:0/3488230339 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55a0605eb4a0 0x55a0605ed8c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2aedd62d1d599f57 server_cookie=a8af692985875cb7 in_seq=0 out_seq=0 2024-04-23T10:27:00.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.798+0000 7fa2b9ffb640 10 client.25298 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:27:00.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.808+0000 7fa2b9ffb640 1 -- 172.21.15.97:0/3488230339 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x7fa2c8096030 con 0x55a0605eb4a0 2024-04-23T10:27:00.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.808+0000 7fa2b9ffb640 10 client.25298 handle_command_reply: tid=0 2024-04-23T10:27:00.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.808+0000 7fa2db78f740 1 client.25298 shutdown 2024-04-23T10:27:00.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.808+0000 7fa2db78f740 1 -- 172.21.15.97:0/3488230339 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a060574090 msgr2=0x55a0605e9800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:00.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.808+0000 7fa2db78f740 1 --2- 172.21.15.97:0/3488230339 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a060574090 0x55a0605e9800 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc005c40 tx=0x7fa2cc012c10 comp rx=0 tx=0).stop 2024-04-23T10:27:00.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.808+0000 7fa2db78f740 1 -- 172.21.15.97:0/3488230339 shutdown_connections 2024-04-23T10:27:00.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.808+0000 7fa2db78f740 1 --2- 172.21.15.97:0/3488230339 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55a0605eb4a0 0x55a0605ed8c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.808+0000 7fa2db78f740 1 --2- 172.21.15.97:0/3488230339 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a060574090 0x55a0605e9800 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.808+0000 7fa2b9ffb640 0 client.25298 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:27:00.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.808+0000 7fa2db78f740 1 --2- 172.21.15.97:0/3488230339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a0605699c0 0x55a06056e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.808+0000 7fa2db78f740 1 --2- 172.21.15.97:0/3488230339 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a0605690a0 0x55a06056e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.808+0000 7fa2db78f740 1 -- 172.21.15.97:0/3488230339 >> 172.21.15.97:0/3488230339 conn(0x55a0605653c0 msgr2=0x55a06056f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:27:00.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.808+0000 7fa2db78f740 1 -- 172.21.15.97:0/3488230339 shutdown_connections 2024-04-23T10:27:00.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.808+0000 7fa2db78f740 1 -- 172.21.15.97:0/3488230339 wait complete. 2024-04-23T10:27:00.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.808+0000 7fa2db78f740 20 client.25298 trim_cache size 0 max 16384 2024-04-23T10:27:00.809 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-23T10:27:00.809 INFO:teuthology.orchestra.run.smithi097.stdout: "epoch": 26, 2024-04-23T10:27:00.809 INFO:teuthology.orchestra.run.smithi097.stdout: "leader": 24488, 2024-04-23T10:27:00.809 INFO:teuthology.orchestra.run.smithi097.stdout: "set_version": 6, 2024-04-23T10:27:00.809 INFO:teuthology.orchestra.run.smithi097.stdout: "sets": { 2024-04-23T10:27:00.809 INFO:teuthology.orchestra.run.smithi097.stdout: "c703558e": { 2024-04-23T10:27:00.809 INFO:teuthology.orchestra.run.smithi097.stdout: "version": 6, 2024-04-23T10:27:00.809 INFO:teuthology.orchestra.run.smithi097.stdout: "age_ref": 0.0, 2024-04-23T10:27:00.809 INFO:teuthology.orchestra.run.smithi097.stdout: "state": { 2024-04-23T10:27:00.809 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "QUIESCED", 2024-04-23T10:27:00.810 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0 2024-04-23T10:27:00.810 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:27:00.810 INFO:teuthology.orchestra.run.smithi097.stdout: "timeout": 60.0, 2024-04-23T10:27:00.810 INFO:teuthology.orchestra.run.smithi097.stdout: "expiration": 82.9, 2024-04-23T10:27:00.810 INFO:teuthology.orchestra.run.smithi097.stdout: "members": { 2024-04-23T10:27:00.810 INFO:teuthology.orchestra.run.smithi097.stdout: "file:/": { 2024-04-23T10:27:00.810 INFO:teuthology.orchestra.run.smithi097.stdout: "excluded": false, 2024-04-23T10:27:00.810 INFO:teuthology.orchestra.run.smithi097.stdout: "state": { 2024-04-23T10:27:00.810 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "QUIESCED", 2024-04-23T10:27:00.810 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0 2024-04-23T10:27:00.810 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:27:00.810 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:27:00.810 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:27:00.810 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:27:00.811 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:27:00.811 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-23T10:27:00.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.809+0000 7fa2d899b640 1 -- 172.21.15.97:0/3137715550 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fa2bc0a3ec0 msgr2=0x7fa2bc0a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:00.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.809+0000 7fa2d899b640 1 --2- 172.21.15.97:0/3137715550 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fa2bc0a3ec0 0x7fa2bc0a6380 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc009fd0 tx=0x7fa2cc017000 comp rx=0 tx=0).stop 2024-04-23T10:27:00.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.809+0000 7fa2d899b640 1 -- 172.21.15.97:0/3137715550 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2d4103380 msgr2=0x7fa2d41a0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:00.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.809+0000 7fa2d899b640 1 --2- 172.21.15.97:0/3137715550 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2d4103380 0x7fa2d41a0bd0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fa2c800e9f0 tx=0x7fa2c800eeb0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.809+0000 7fa2d899b640 1 -- 172.21.15.97:0/3137715550 shutdown_connections 2024-04-23T10:27:00.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.809+0000 7fa2d899b640 1 --2- 172.21.15.97:0/3137715550 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fa2bc0a3ec0 0x7fa2bc0a6380 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.809+0000 7fa2d899b640 1 --2- 172.21.15.97:0/3137715550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa2d410ebe0 0x7fa2d419a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.809+0000 7fa2d899b640 1 --2- 172.21.15.97:0/3137715550 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2d4103380 0x7fa2d41a0bd0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.809+0000 7fa2d899b640 1 --2- 172.21.15.97:0/3137715550 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2d4102f10 0x7fa2d41a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:00.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.809+0000 7fa2d899b640 1 -- 172.21.15.97:0/3137715550 >> 172.21.15.97:0/3137715550 conn(0x7fa2d40773f0 msgr2=0x7fa2d4110180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:27:00.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.809+0000 7fa2d899b640 1 -- 172.21.15.97:0/3137715550 shutdown_connections 2024-04-23T10:27:00.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:00.809+0000 7fa2d899b640 1 -- 172.21.15.97:0/3137715550 wait complete. 2024-04-23T10:27:00.823 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'c703558e', quiesce took 0.8 seconds. Will release after: 22.9 2024-04-23T10:27:01.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:27:00 smithi196 ceph-mon[32919]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-23T10:27:01.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:27:00 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/1445939885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:27:01.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:27:00 smithi097 ceph-mon[29197]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-23T10:27:01.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:27:00 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/1445939885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:27:01.361 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:27:01 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:27:01.002+0000 7ff6b5503700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T10:27:01.361 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:27:01 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:27:01.002+0000 7ff6b5503700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T10:27:01.361 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:27:01 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:27:01.002+0000 7ff6b5503700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T10:27:01.361 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:27:01 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:27:01.002+0000 7ff6b5503700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T10:27:01.361 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:27:01 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:27:01.002+0000 7ff6b5503700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T10:27:01.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:27:00 smithi183 ceph-mon[32317]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-23T10:27:01.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:27:00 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/1445939885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:27:03.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:27:02 smithi196 ceph-mon[32919]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T10:27:03.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:27:02 smithi196 ceph-mon[32919]: mgrmap e25: x(active, since 50s), standbys: z, y 2024-04-23T10:27:03.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:27:02 smithi097 ceph-mon[29197]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T10:27:03.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:27:02 smithi097 ceph-mon[29197]: mgrmap e25: x(active, since 50s), standbys: z, y 2024-04-23T10:27:03.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:27:02 smithi183 ceph-mon[32317]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T10:27:03.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:27:02 smithi183 ceph-mon[32317]: mgrmap e25: x(active, since 50s), standbys: z, y 2024-04-23T10:27:05.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:27:04 smithi196 ceph-mon[32919]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-23T10:27:05.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:27:04 smithi097 ceph-mon[29197]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-23T10:27:05.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:27:04 smithi183 ceph-mon[32317]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 28 op/s 2024-04-23T10:27:06.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.391+0000 7f8e2bfcc640 1 -- 172.21.15.97:0/4114118622 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e24107290 msgr2=0x7f8e2410ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:06.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.391+0000 7f8e2bfcc640 1 --2- 172.21.15.97:0/4114118622 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e24107290 0x7f8e2410ee60 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c009ca0 tx=0x7f8e1c02a8c0 comp rx=0 tx=0).stop 2024-04-23T10:27:06.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.391+0000 7f8e2bfcc640 1 -- 172.21.15.97:0/4114118622 shutdown_connections 2024-04-23T10:27:06.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.391+0000 7f8e2bfcc640 1 --2- 172.21.15.97:0/4114118622 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e24107290 0x7f8e2410ee60 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:06.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.391+0000 7f8e2bfcc640 1 --2- 172.21.15.97:0/4114118622 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e24102b70 0x7f8e24106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:06.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.391+0000 7f8e2bfcc640 1 --2- 172.21.15.97:0/4114118622 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e241021c0 0x7f8e241025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:06.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.391+0000 7f8e2bfcc640 1 -- 172.21.15.97:0/4114118622 >> 172.21.15.97:0/4114118622 conn(0x7f8e240fda40 msgr2=0x7f8e240ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:27:06.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.391+0000 7f8e2bfcc640 1 -- 172.21.15.97:0/4114118622 shutdown_connections 2024-04-23T10:27:06.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.391+0000 7f8e2bfcc640 1 -- 172.21.15.97:0/4114118622 wait complete. 2024-04-23T10:27:06.393 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.391+0000 7f8e2bfcc640 1 Processor -- start 2024-04-23T10:27:06.393 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.391+0000 7f8e2bfcc640 1 -- start start 2024-04-23T10:27:06.393 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.391+0000 7f8e2bfcc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e241021c0 0x7f8e24198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:06.393 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.391+0000 7f8e2bfcc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e24107290 0x7f8e24198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:06.393 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.391+0000 7f8e2bfcc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e2419cf70 0x7f8e2419d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:06.393 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.391+0000 7f8e2bfcc640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8e24112200 con 0x7f8e2419cf70 2024-04-23T10:27:06.393 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.391+0000 7f8e2bfcc640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8e24112080 con 0x7f8e24107290 2024-04-23T10:27:06.393 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.391+0000 7f8e2bfcc640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8e24112380 con 0x7f8e241021c0 2024-04-23T10:27:06.393 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.392+0000 7f8e2a569640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e2419cf70 0x7f8e2419d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:06.393 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.392+0000 7f8e2a569640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e2419cf70 0x7f8e2419d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52806/0 (socket says 172.21.15.97:52806) 2024-04-23T10:27:06.393 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.392+0000 7f8e2a569640 1 -- 172.21.15.97:0/3941085543 learned_addr learned my addr 172.21.15.97:0/3941085543 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:27:06.393 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.392+0000 7f8e29567640 1 --2- 172.21.15.97:0/3941085543 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e24107290 0x7f8e24198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:06.393 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.392+0000 7f8e29d68640 1 --2- 172.21.15.97:0/3941085543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e241021c0 0x7f8e24198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:06.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.392+0000 7f8e2a569640 1 -- 172.21.15.97:0/3941085543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e241021c0 msgr2=0x7f8e24198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:06.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.392+0000 7f8e2a569640 1 --2- 172.21.15.97:0/3941085543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e241021c0 0x7f8e24198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:06.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.392+0000 7f8e2a569640 1 -- 172.21.15.97:0/3941085543 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e24107290 msgr2=0x7f8e24198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:06.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.392+0000 7f8e2a569640 1 --2- 172.21.15.97:0/3941085543 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e24107290 0x7f8e24198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:06.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.392+0000 7f8e2a569640 1 -- 172.21.15.97:0/3941085543 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e2419da60 con 0x7f8e2419cf70 2024-04-23T10:27:06.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.392+0000 7f8e29d68640 1 --2- 172.21.15.97:0/3941085543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e241021c0 0x7f8e24198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:27:06.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.392+0000 7f8e2a569640 1 --2- 172.21.15.97:0/3941085543 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e2419cf70 0x7f8e2419d420 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c005cc0 tx=0x7f8e1c02f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:27:06.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.392+0000 7f8e1affd640 1 -- 172.21.15.97:0/3941085543 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c003bf0 con 0x7f8e2419cf70 2024-04-23T10:27:06.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.392+0000 7f8e1affd640 1 -- 172.21.15.97:0/3941085543 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e1c003d90 con 0x7f8e2419cf70 2024-04-23T10:27:06.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.392+0000 7f8e1affd640 1 -- 172.21.15.97:0/3941085543 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c02fbc0 con 0x7f8e2419cf70 2024-04-23T10:27:06.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.392+0000 7f8e2bfcc640 1 -- 172.21.15.97:0/3941085543 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e2419dcf0 con 0x7f8e2419cf70 2024-04-23T10:27:06.394 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.392+0000 7f8e2bfcc640 1 -- 172.21.15.97:0/3941085543 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e241a5510 con 0x7f8e2419cf70 2024-04-23T10:27:06.401 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.395+0000 7f8e2edb7740 1 Processor -- start 2024-04-23T10:27:06.401 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.395+0000 7f8e1affd640 1 -- 172.21.15.97:0/3941085543 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f8e1c02fd60 con 0x7f8e2419cf70 2024-04-23T10:27:06.401 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.395+0000 7f8e2edb7740 1 -- start start 2024-04-23T10:27:06.401 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.396+0000 7f8e2edb7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558ef693d540 0x558ef693d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:06.401 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.396+0000 7f8e2edb7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ef693de60 0x558ef694c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:06.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.396+0000 7f8e2edb7740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ef694cc10 0x558ef694f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:06.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.396+0000 7f8e2edb7740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x558ef68a2970 con 0x558ef694cc10 2024-04-23T10:27:06.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.396+0000 7f8e2edb7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558ef683f4a0 con 0x558ef693d540 2024-04-23T10:27:06.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.396+0000 7f8e2edb7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558ef689e930 con 0x558ef693de60 2024-04-23T10:27:06.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.396+0000 7f8e29567640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ef694cc10 0x558ef694f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:06.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.396+0000 7f8e29567640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ef694cc10 0x558ef694f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52822/0 (socket says 172.21.15.97:52822) 2024-04-23T10:27:06.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.396+0000 7f8e29567640 1 -- 172.21.15.97:0/2936866600 learned_addr learned my addr 172.21.15.97:0/2936866600 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:27:06.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.396+0000 7f8e29567640 1 --2- 172.21.15.97:0/2936866600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558ef693d540 0x558ef693d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:06.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.396+0000 7f8e1affd640 1 --2- 172.21.15.97:0/3941085543 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f8e100a3f70 0x7f8e100a6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:06.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.396+0000 7f8e29d68640 1 --2- 172.21.15.97:0/2936866600 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ef693de60 0x558ef694c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:06.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.396+0000 7f8e29567640 1 -- 172.21.15.97:0/2936866600 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ef693de60 msgr2=0x558ef694c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:27:06.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.396+0000 7f8e29567640 1 --2- 172.21.15.97:0/2936866600 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ef693de60 0x558ef694c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:06.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.396+0000 7f8e2a569640 1 --2- 172.21.15.97:0/3941085543 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f8e100a3f70 0x7f8e100a6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:06.403 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.396+0000 7f8e29567640 1 -- 172.21.15.97:0/2936866600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558ef693d540 msgr2=0x558ef693d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:27:06.403 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.396+0000 7f8e29567640 1 --2- 172.21.15.97:0/2936866600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558ef693d540 0x558ef693d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:06.403 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.396+0000 7f8e29567640 1 -- 172.21.15.97:0/2936866600 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ef65b5210 con 0x558ef694cc10 2024-04-23T10:27:06.403 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.396+0000 7f8e1affd640 1 -- 172.21.15.97:0/3941085543 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f8e1c0f20a0 con 0x7f8e2419cf70 2024-04-23T10:27:06.403 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.396+0000 7f8e29567640 1 --2- 172.21.15.97:0/2936866600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ef694cc10 0x558ef694f000 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f8e14012700 tx=0x7f8e1400b6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9e1d7c1acb42479 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:27:06.403 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.396+0000 7f8e0ffff640 1 -- 172.21.15.97:0/2936866600 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e14012c80 con 0x558ef694cc10 2024-04-23T10:27:06.403 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.396+0000 7f8e0ffff640 1 -- 172.21.15.97:0/2936866600 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e14012e20 con 0x558ef694cc10 2024-04-23T10:27:06.403 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.396+0000 7f8e0ffff640 1 -- 172.21.15.97:0/2936866600 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e14003850 con 0x558ef694cc10 2024-04-23T10:27:06.403 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.397+0000 7f8e2edb7740 1 -- 172.21.15.97:0/2936866600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ef694cc10 msgr2=0x558ef694f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:06.403 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.397+0000 7f8e2a569640 1 --2- 172.21.15.97:0/3941085543 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f8e100a3f70 0x7f8e100a6430 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c0ef770 tx=0x7f8e1c08f9a0 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:27:06.403 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.397+0000 7f8e2edb7740 1 --2- 172.21.15.97:0/2936866600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ef694cc10 0x558ef694f000 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f8e14012700 tx=0x7f8e1400b6e0 comp rx=0 tx=0).stop 2024-04-23T10:27:06.403 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.397+0000 7f8e2edb7740 1 -- 172.21.15.97:0/2936866600 shutdown_connections 2024-04-23T10:27:06.403 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.397+0000 7f8e2edb7740 1 --2- 172.21.15.97:0/2936866600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ef694cc10 0x558ef694f000 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:06.403 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.397+0000 7f8e2edb7740 1 --2- 172.21.15.97:0/2936866600 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ef693de60 0x558ef694c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:06.403 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.397+0000 7f8e2edb7740 1 --2- 172.21.15.97:0/2936866600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558ef693d540 0x558ef693d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.397+0000 7f8e2edb7740 1 -- 172.21.15.97:0/2936866600 >> 172.21.15.97:0/2936866600 conn(0x558ef6939b30 msgr2=0x558ef6939f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:27:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.397+0000 7f8e2edb7740 1 -- 172.21.15.97:0/2936866600 shutdown_connections 2024-04-23T10:27:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.397+0000 7f8e2edb7740 1 -- 172.21.15.97:0/2936866600 wait complete. 2024-04-23T10:27:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.398+0000 7f8e2edb7740 1 -- start start 2024-04-23T10:27:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.398+0000 7f8e2edb7740 1 Processor -- start 2024-04-23T10:27:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.398+0000 7f8e2edb7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ef693d540 0x558ef694c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.398+0000 7f8e2edb7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558ef693de60 0x558ef69498d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.398+0000 7f8e2edb7740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ef694cc10 0x558ef6949ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.398+0000 7f8e2edb7740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x558ef694f840 con 0x558ef694cc10 2024-04-23T10:27:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.398+0000 7f8e2edb7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558ef683d0b0 con 0x558ef693de60 2024-04-23T10:27:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.398+0000 7f8e2edb7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558ef694f9c0 con 0x558ef693d540 2024-04-23T10:27:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.398+0000 7f8e2edb7740 10 client.0 _ll_register_callbacks cb 0x558ef682ad30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:27:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.398+0000 7f8e29567640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ef694cc10 0x558ef6949ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:06.404 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.398+0000 7f8e29567640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ef694cc10 0x558ef6949ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52826/0 (socket says 172.21.15.97:52826) 2024-04-23T10:27:06.405 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.398+0000 7f8e29567640 1 -- 172.21.15.97:0/2631039484 learned_addr learned my addr 172.21.15.97:0/2631039484 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:27:06.405 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.398+0000 7f8e29d68640 1 --2- 172.21.15.97:0/2631039484 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558ef693de60 0x558ef69498d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:06.405 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.399+0000 7f8e29567640 1 -- 172.21.15.97:0/2631039484 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ef693d540 msgr2=0x558ef694c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:27:06.405 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.399+0000 7f8e29567640 1 --2- 172.21.15.97:0/2631039484 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ef693d540 0x558ef694c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:06.405 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.399+0000 7f8e29567640 1 -- 172.21.15.97:0/2631039484 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558ef693de60 msgr2=0x558ef69498d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:27:06.405 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.399+0000 7f8e29567640 1 --2- 172.21.15.97:0/2631039484 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558ef693de60 0x558ef69498d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:06.405 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.399+0000 7f8e29567640 1 -- 172.21.15.97:0/2631039484 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ef6681620 con 0x558ef694cc10 2024-04-23T10:27:06.405 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.399+0000 7f8e29d68640 1 --2- 172.21.15.97:0/2631039484 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558ef693de60 0x558ef69498d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:27:06.405 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.399+0000 7f8e29567640 1 --2- 172.21.15.97:0/2631039484 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ef694cc10 0x558ef6949ee0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f8e1400be80 tx=0x7f8e1400beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=900ff6ff500483ac server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:27:06.405 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.399+0000 7f8e0d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:27:06.405 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.399+0000 7f8e0d7fa640 1 -- 172.21.15.97:0/2631039484 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e14012c80 con 0x558ef694cc10 2024-04-23T10:27:06.405 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.399+0000 7f8e0d7fa640 1 -- 172.21.15.97:0/2631039484 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f8e14005460 con 0x558ef694cc10 2024-04-23T10:27:06.405 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.399+0000 7f8e2edb7740 1 -- 172.21.15.97:0/2631039484 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e14012e20 con 0x558ef694cc10 2024-04-23T10:27:06.405 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.399+0000 7f8e0d7fa640 1 -- 172.21.15.97:0/2631039484 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8e1402c020 con 0x558ef694cc10 2024-04-23T10:27:06.406 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.399+0000 7f8e2edb7740 10 client.15825 fetch_fsmap learned FSMap version 26 2024-04-23T10:27:06.406 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.399+0000 7f8e2edb7740 1 -- 172.21.15.97:0/2631039484 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558ef695b810 con 0x558ef694cc10 2024-04-23T10:27:06.406 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.399+0000 7f8e0d7fa640 1 -- 172.21.15.97:0/2631039484 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e14012e20 con 0x558ef694cc10 2024-04-23T10:27:06.406 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.399+0000 7f8e0d7fa640 1 -- 172.21.15.97:0/2631039484 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8e1401f4f0 con 0x558ef694cc10 2024-04-23T10:27:06.406 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.400+0000 7f8e2edb7740 10 client.15825 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:27:06.406 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.400+0000 7f8e2edb7740 10 client.15825 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:27:06.406 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.400+0000 7f8e2edb7740 20 client.15825 populate_metadata read hostname 'smithi097' 2024-04-23T10:27:06.406 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.400+0000 7f8e2edb7740 1 --2- 172.21.15.97:0/2631039484 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x558ef695cb60 0x558ef695ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:06.406 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.400+0000 7f8e2edb7740 4 client.15825 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:27:06.406 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.400+0000 7f8e2edb7740 1 -- 172.21.15.97:0/2631039484 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ef68b0390 con 0x558ef695cb60 2024-04-23T10:27:06.406 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.400+0000 7f8e29d68640 1 --2- 172.21.15.97:0/2631039484 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x558ef695cb60 0x558ef695ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:06.406 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.400+0000 7f8e29d68640 1 --2- 172.21.15.97:0/2631039484 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x558ef695cb60 0x558ef695ef50 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6457f924061a001 server_cookie=4ef44684d2b3c908 in_seq=0 out_seq=0 2024-04-23T10:27:06.406 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.400+0000 7f8e0d7fa640 10 client.15825 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:27:06.406 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.401+0000 7f8e0d7fa640 1 -- 172.21.15.97:0/2631039484 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558ef68b0390 con 0x558ef695cb60 2024-04-23T10:27:06.406 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.401+0000 7f8e0d7fa640 10 client.15825 handle_command_reply: tid=0 2024-04-23T10:27:06.407 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.401+0000 7f8e2edb7740 1 client.15825 shutdown 2024-04-23T10:27:06.407 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.401+0000 7f8e2edb7740 1 -- 172.21.15.97:0/2631039484 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ef694cc10 msgr2=0x558ef6949ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:06.407 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.401+0000 7f8e2edb7740 1 --2- 172.21.15.97:0/2631039484 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ef694cc10 0x558ef6949ee0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f8e1400be80 tx=0x7f8e1400beb0 comp rx=0 tx=0).stop 2024-04-23T10:27:06.407 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.401+0000 7f8e2edb7740 1 -- 172.21.15.97:0/2631039484 shutdown_connections 2024-04-23T10:27:06.407 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.401+0000 7f8e2edb7740 1 --2- 172.21.15.97:0/2631039484 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x558ef695cb60 0x558ef695ef50 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:06.407 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.401+0000 7f8e2edb7740 1 --2- 172.21.15.97:0/2631039484 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ef694cc10 0x558ef6949ee0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:06.407 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.401+0000 7f8e2edb7740 1 --2- 172.21.15.97:0/2631039484 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558ef693de60 0x558ef69498d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:06.407 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.401+0000 7f8e0d7fa640 0 client.15825 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:27:06.407 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.401+0000 7f8e2edb7740 1 --2- 172.21.15.97:0/2631039484 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ef693d540 0x558ef694c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:06.407 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.401+0000 7f8e2edb7740 1 -- 172.21.15.97:0/2631039484 >> 172.21.15.97:0/2631039484 conn(0x558ef6939b30 msgr2=0x558ef693f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:27:06.407 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.401+0000 7f8e2edb7740 1 -- 172.21.15.97:0/2631039484 shutdown_connections 2024-04-23T10:27:06.407 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.401+0000 7f8e2edb7740 1 -- 172.21.15.97:0/2631039484 wait complete. 2024-04-23T10:27:06.407 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.401+0000 7f8e2edb7740 20 client.15825 trim_cache size 0 max 16384 2024-04-23T10:27:06.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.416+0000 7f8e2edb7740 1 Processor -- start 2024-04-23T10:27:06.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.416+0000 7f8e2edb7740 1 -- start start 2024-04-23T10:27:06.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.416+0000 7f8e2edb7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558ef693d540 0x558ef6958700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:06.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.416+0000 7f8e2edb7740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ef693de60 0x558ef6938b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:06.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.416+0000 7f8e2edb7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ef694cc10 0x558ef694c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:06.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.416+0000 7f8e2edb7740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8e241ab7c0 con 0x558ef693de60 2024-04-23T10:27:06.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.416+0000 7f8e2edb7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8e241ad300 con 0x558ef693d540 2024-04-23T10:27:06.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.416+0000 7f8e2edb7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8e241abe90 con 0x558ef694cc10 2024-04-23T10:27:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.416+0000 7f8e29d68640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ef693de60 0x558ef6938b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.416+0000 7f8e29d68640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ef693de60 0x558ef6938b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52838/0 (socket says 172.21.15.97:52838) 2024-04-23T10:27:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.416+0000 7f8e29d68640 1 -- 172.21.15.97:0/3611195912 learned_addr learned my addr 172.21.15.97:0/3611195912 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:27:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.417+0000 7f8e29d68640 1 -- 172.21.15.97:0/3611195912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ef694cc10 msgr2=0x558ef694c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:27:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.417+0000 7f8e29d68640 1 --2- 172.21.15.97:0/3611195912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ef694cc10 0x558ef694c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.417+0000 7f8e29d68640 1 -- 172.21.15.97:0/3611195912 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558ef693d540 msgr2=0x558ef6958700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:27:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.417+0000 7f8e29d68640 1 --2- 172.21.15.97:0/3611195912 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558ef693d540 0x558ef6958700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.417+0000 7f8e29d68640 1 -- 172.21.15.97:0/3611195912 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ef6947e20 con 0x558ef693de60 2024-04-23T10:27:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.417+0000 7f8e29d68640 1 --2- 172.21.15.97:0/3611195912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ef693de60 0x558ef6938b30 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x558ef6949130 tx=0x7f8e2000a3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6ce3a185f5a1d49 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:27:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.417+0000 7f8e0f7fe640 1 -- 172.21.15.97:0/3611195912 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e20004220 con 0x558ef693de60 2024-04-23T10:27:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.417+0000 7f8e0f7fe640 1 -- 172.21.15.97:0/3611195912 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e200043c0 con 0x558ef693de60 2024-04-23T10:27:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.417+0000 7f8e0f7fe640 1 -- 172.21.15.97:0/3611195912 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e20004a00 con 0x558ef693de60 2024-04-23T10:27:06.421 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.418+0000 7f8e2edb7740 1 -- 172.21.15.97:0/3611195912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ef693de60 msgr2=0x558ef6938b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:06.421 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.418+0000 7f8e2edb7740 1 --2- 172.21.15.97:0/3611195912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ef693de60 0x558ef6938b30 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x558ef6949130 tx=0x7f8e2000a3a0 comp rx=0 tx=0).stop 2024-04-23T10:27:06.421 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.418+0000 7f8e2edb7740 1 -- 172.21.15.97:0/3611195912 shutdown_connections 2024-04-23T10:27:06.421 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.418+0000 7f8e2edb7740 1 --2- 172.21.15.97:0/3611195912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ef694cc10 0x558ef694c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:06.421 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.418+0000 7f8e2edb7740 1 --2- 172.21.15.97:0/3611195912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ef693de60 0x558ef6938b30 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:06.421 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.418+0000 7f8e2edb7740 1 --2- 172.21.15.97:0/3611195912 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558ef693d540 0x558ef6958700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:06.421 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.418+0000 7f8e2edb7740 1 -- 172.21.15.97:0/3611195912 >> 172.21.15.97:0/3611195912 conn(0x558ef6939b30 msgr2=0x558ef6938420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:27:06.421 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.418+0000 7f8e2edb7740 1 -- 172.21.15.97:0/3611195912 shutdown_connections 2024-04-23T10:27:06.421 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.418+0000 7f8e2edb7740 1 -- 172.21.15.97:0/3611195912 wait complete. 2024-04-23T10:27:06.421 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.418+0000 7f8e2edb7740 1 -- start start 2024-04-23T10:27:06.421 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.418+0000 7f8e2edb7740 1 Processor -- start 2024-04-23T10:27:06.422 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.418+0000 7f8e2edb7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558ef693d540 0x558ef6954c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:06.422 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.418+0000 7f8e2edb7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ef693de60 0x558ef6955190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:06.422 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.418+0000 7f8e2edb7740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ef694cc10 0x558ef6950380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:06.422 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.418+0000 7f8e2edb7740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8e241acc30 con 0x558ef694cc10 2024-04-23T10:27:06.422 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.418+0000 7f8e2edb7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8e241ac560 con 0x558ef693d540 2024-04-23T10:27:06.422 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.418+0000 7f8e2edb7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8e241a97e0 con 0x558ef693de60 2024-04-23T10:27:06.422 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.418+0000 7f8e2edb7740 10 client.0 _ll_register_callbacks cb 0x7f8e1400b390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:27:06.422 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.419+0000 7f8e29567640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ef694cc10 0x558ef6950380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:06.422 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.419+0000 7f8e29567640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ef694cc10 0x558ef6950380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52842/0 (socket says 172.21.15.97:52842) 2024-04-23T10:27:06.422 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.419+0000 7f8e29567640 1 -- 172.21.15.97:0/2854716354 learned_addr learned my addr 172.21.15.97:0/2854716354 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:27:06.422 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.419+0000 7f8e29d68640 1 --2- 172.21.15.97:0/2854716354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ef693de60 0x558ef6955190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:06.422 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.419+0000 7f8e29567640 1 -- 172.21.15.97:0/2854716354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ef693de60 msgr2=0x558ef6955190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:27:06.422 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.419+0000 7f8e29567640 1 --2- 172.21.15.97:0/2854716354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ef693de60 0x558ef6955190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:06.423 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.419+0000 7f8e29567640 1 -- 172.21.15.97:0/2854716354 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558ef693d540 msgr2=0x558ef6954c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:27:06.423 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.419+0000 7f8e29567640 1 --2- 172.21.15.97:0/2854716354 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558ef693d540 0x558ef6954c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:06.423 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.419+0000 7f8e29567640 1 -- 172.21.15.97:0/2854716354 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ef6950bb0 con 0x558ef694cc10 2024-04-23T10:27:06.423 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.419+0000 7f8e29d68640 1 --2- 172.21.15.97:0/2854716354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ef693de60 0x558ef6955190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:27:06.423 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.419+0000 7f8e29567640 1 --2- 172.21.15.97:0/2854716354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ef694cc10 0x558ef6950380 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f8e1400be80 tx=0x7f8e140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=979c6bb0de56e6d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:27:06.423 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.419+0000 7f8e0d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:27:06.423 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.419+0000 7f8e0d7fa640 1 -- 172.21.15.97:0/2854716354 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1400bbf0 con 0x558ef694cc10 2024-04-23T10:27:06.423 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.419+0000 7f8e0d7fa640 1 -- 172.21.15.97:0/2854716354 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f8e1401f460 con 0x558ef694cc10 2024-04-23T10:27:06.423 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.419+0000 7f8e2edb7740 1 -- 172.21.15.97:0/2854716354 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e200043c0 con 0x558ef694cc10 2024-04-23T10:27:06.423 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.419+0000 7f8e0d7fa640 1 -- 172.21.15.97:0/2854716354 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8e1401f6a0 con 0x558ef694cc10 2024-04-23T10:27:06.423 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.419+0000 7f8e2edb7740 10 client.15831 fetch_fsmap learned FSMap version 26 2024-04-23T10:27:06.423 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.419+0000 7f8e2edb7740 1 -- 172.21.15.97:0/2854716354 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558ef69c2b70 con 0x558ef694cc10 2024-04-23T10:27:06.423 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.419+0000 7f8e0d7fa640 1 -- 172.21.15.97:0/2854716354 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e200043c0 con 0x558ef694cc10 2024-04-23T10:27:06.423 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.420+0000 7f8e0d7fa640 1 -- 172.21.15.97:0/2854716354 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8e14003c70 con 0x558ef694cc10 2024-04-23T10:27:06.424 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.420+0000 7f8e2edb7740 10 client.15831 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:27:06.424 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.420+0000 7f8e2edb7740 10 client.15831 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:27:06.424 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.420+0000 7f8e2edb7740 20 client.15831 populate_metadata read hostname 'smithi097' 2024-04-23T10:27:06.424 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.420+0000 7f8e2edb7740 1 --2- 172.21.15.97:0/2854716354 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x558ef69c3230 0x558ef69c5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:06.424 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.420+0000 7f8e2edb7740 4 client.15831 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:27:06.424 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.420+0000 7f8e2edb7740 1 -- 172.21.15.97:0/2854716354 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558ef68aa200 con 0x558ef69c3230 2024-04-23T10:27:06.424 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.420+0000 7f8e29d68640 1 --2- 172.21.15.97:0/2854716354 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x558ef69c3230 0x558ef69c5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:06.424 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.420+0000 7f8e0d7fa640 10 client.15831 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:27:06.424 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.420+0000 7f8e29d68640 1 --2- 172.21.15.97:0/2854716354 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x558ef69c3230 0x558ef69c5650 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85ae5706daf2e3e server_cookie=929bfd2b33a6915b in_seq=0 out_seq=0 2024-04-23T10:27:06.426 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.426+0000 7f8e0d7fa640 1 -- 172.21.15.97:0/2854716354 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558ef68aa200 con 0x558ef69c3230 2024-04-23T10:27:06.426 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.426+0000 7f8e0d7fa640 10 client.15831 handle_command_reply: tid=0 2024-04-23T10:27:06.426 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.426+0000 7f8e2edb7740 1 client.15831 shutdown 2024-04-23T10:27:06.426 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.426+0000 7f8e2edb7740 1 -- 172.21.15.97:0/2854716354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ef694cc10 msgr2=0x558ef6950380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:06.426 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.426+0000 7f8e2edb7740 1 --2- 172.21.15.97:0/2854716354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ef694cc10 0x558ef6950380 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f8e1400be80 tx=0x7f8e140057f0 comp rx=0 tx=0).stop 2024-04-23T10:27:06.426 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.426+0000 7f8e2edb7740 1 -- 172.21.15.97:0/2854716354 shutdown_connections 2024-04-23T10:27:06.426 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.426+0000 7f8e2edb7740 1 --2- 172.21.15.97:0/2854716354 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x558ef69c3230 0x558ef69c5650 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:06.426 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.426+0000 7f8e2edb7740 1 --2- 172.21.15.97:0/2854716354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558ef694cc10 0x558ef6950380 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:06.426 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.426+0000 7f8e0d7fa640 0 client.15831 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:27:06.426 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.426+0000 7f8e2edb7740 1 --2- 172.21.15.97:0/2854716354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558ef693de60 0x558ef6955190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:06.426 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.426+0000 7f8e2edb7740 1 --2- 172.21.15.97:0/2854716354 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558ef693d540 0x558ef6954c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:06.427 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.426+0000 7f8e2edb7740 1 -- 172.21.15.97:0/2854716354 >> 172.21.15.97:0/2854716354 conn(0x558ef6939b30 msgr2=0x558ef6944f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:27:06.427 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.426+0000 7f8e2edb7740 1 -- 172.21.15.97:0/2854716354 shutdown_connections 2024-04-23T10:27:06.427 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.426+0000 7f8e2edb7740 1 -- 172.21.15.97:0/2854716354 wait complete. 2024-04-23T10:27:06.427 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.427+0000 7f8e2edb7740 20 client.15831 trim_cache size 0 max 16384 2024-04-23T10:27:06.427 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:27:06.427 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:27:06.427 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:27:06.427 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:27:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.427+0000 7f8e2bfcc640 1 -- 172.21.15.97:0/3941085543 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f8e100a3f70 msgr2=0x7f8e100a6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.427+0000 7f8e2bfcc640 1 --2- 172.21.15.97:0/3941085543 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f8e100a3f70 0x7f8e100a6430 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c0ef770 tx=0x7f8e1c08f9a0 comp rx=0 tx=0).stop 2024-04-23T10:27:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.427+0000 7f8e2bfcc640 1 -- 172.21.15.97:0/3941085543 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e2419cf70 msgr2=0x7f8e2419d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.427+0000 7f8e2bfcc640 1 --2- 172.21.15.97:0/3941085543 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e2419cf70 0x7f8e2419d420 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c005cc0 tx=0x7f8e1c02f380 comp rx=0 tx=0).stop 2024-04-23T10:27:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.427+0000 7f8e2bfcc640 1 -- 172.21.15.97:0/3941085543 shutdown_connections 2024-04-23T10:27:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.427+0000 7f8e2bfcc640 1 --2- 172.21.15.97:0/3941085543 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f8e100a3f70 0x7f8e100a6430 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.427+0000 7f8e2bfcc640 1 --2- 172.21.15.97:0/3941085543 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e2419cf70 0x7f8e2419d420 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.427+0000 7f8e2bfcc640 1 --2- 172.21.15.97:0/3941085543 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e24107290 0x7f8e24198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.427+0000 7f8e2bfcc640 1 --2- 172.21.15.97:0/3941085543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e241021c0 0x7f8e24198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.427+0000 7f8e2bfcc640 1 -- 172.21.15.97:0/3941085543 >> 172.21.15.97:0/3941085543 conn(0x7f8e240fda40 msgr2=0x7f8e240ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:27:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.427+0000 7f8e2bfcc640 1 -- 172.21.15.97:0/3941085543 shutdown_connections 2024-04-23T10:27:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:06.427+0000 7f8e2bfcc640 1 -- 172.21.15.97:0/3941085543 wait complete. 2024-04-23T10:27:06.613 INFO:tasks.workunit.client.0.smithi097.stderr:Note: switching to '0a6c3ed699031b80a2b419e7e795368719871394'. 2024-04-23T10:27:06.613 INFO:tasks.workunit.client.0.smithi097.stderr: 2024-04-23T10:27:06.613 INFO:tasks.workunit.client.0.smithi097.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-23T10:27:06.613 INFO:tasks.workunit.client.0.smithi097.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-23T10:27:06.613 INFO:tasks.workunit.client.0.smithi097.stderr:state without impacting any branches by switching back to a branch. 2024-04-23T10:27:06.613 INFO:tasks.workunit.client.0.smithi097.stderr: 2024-04-23T10:27:06.613 INFO:tasks.workunit.client.0.smithi097.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-23T10:27:06.613 INFO:tasks.workunit.client.0.smithi097.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-23T10:27:06.613 INFO:tasks.workunit.client.0.smithi097.stderr: 2024-04-23T10:27:06.614 INFO:tasks.workunit.client.0.smithi097.stderr: git switch -c 2024-04-23T10:27:06.614 INFO:tasks.workunit.client.0.smithi097.stderr: 2024-04-23T10:27:06.614 INFO:tasks.workunit.client.0.smithi097.stderr:Or undo this operation with: 2024-04-23T10:27:06.614 INFO:tasks.workunit.client.0.smithi097.stderr: 2024-04-23T10:27:06.614 INFO:tasks.workunit.client.0.smithi097.stderr: git switch - 2024-04-23T10:27:06.614 INFO:tasks.workunit.client.0.smithi097.stderr: 2024-04-23T10:27:06.614 INFO:tasks.workunit.client.0.smithi097.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-23T10:27:06.614 INFO:tasks.workunit.client.0.smithi097.stderr: 2024-04-23T10:27:06.614 INFO:tasks.workunit.client.0.smithi097.stderr:HEAD is now at 0a6c3ed6990 qa/suites/fs: add quiescer to the fs suite 2024-04-23T10:27:06.615 DEBUG:teuthology.orchestra.run.smithi097:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-23T10:27:06.639 INFO:tasks.workunit.client.0.smithi097.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-23T10:27:06.641 INFO:tasks.workunit.client.0.smithi097.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-23T10:27:06.641 INFO:tasks.workunit.client.0.smithi097.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-23T10:27:07.272 INFO:tasks.workunit.client.0.smithi097.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-23T10:27:07.324 INFO:tasks.workunit.client.0.smithi097.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-23T10:27:07.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:27:06 smithi196 ceph-mon[32919]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-23T10:27:07.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:27:06 smithi097 ceph-mon[29197]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-23T10:27:07.364 INFO:tasks.workunit.client.0.smithi097.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-23T10:27:07.366 INFO:tasks.workunit.client.0.smithi097.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-23T10:27:07.366 INFO:tasks.workunit.client.0.smithi097.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-23T10:27:07.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:27:06 smithi183 ceph-mon[32317]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-23T10:27:07.404 INFO:tasks.workunit.client.0.smithi097.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-23T10:27:07.407 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-23T10:27:07.407 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-23T10:27:07.463 INFO:tasks.workunit:Running workunits matching fs/test_o_trunc.sh on client.0... 2024-04-23T10:27:07.464 INFO:tasks.workunit:Running workunit fs/test_o_trunc.sh... 2024-04-23T10:27:07.465 DEBUG:teuthology.orchestra.run.smithi097:workunit test fs/test_o_trunc.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=0a6c3ed699031b80a2b419e7e795368719871394 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh 2024-04-23T10:27:09.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:27:08 smithi196 ceph-mon[32919]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-04-23T10:27:09.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:27:08 smithi097 ceph-mon[29197]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-04-23T10:27:09.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:27:08 smithi183 ceph-mon[32317]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-04-23T10:27:11.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:27:10 smithi196 ceph-mon[32919]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-04-23T10:27:11.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:27:10 smithi097 ceph-mon[29197]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-04-23T10:27:11.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:27:10 smithi183 ceph-mon[32317]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-04-23T10:27:13.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:27:13 smithi196 ceph-mon[32919]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-04-23T10:27:13.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:27:13 smithi097 ceph-mon[29197]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-04-23T10:27:13.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:27:13 smithi183 ceph-mon[32317]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-04-23T10:27:15.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:27:15 smithi196 ceph-mon[32919]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-04-23T10:27:15.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:27:15 smithi097 ceph-mon[29197]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-04-23T10:27:15.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:27:15 smithi183 ceph-mon[32317]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-04-23T10:27:17.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:27:17 smithi196 ceph-mon[32919]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-04-23T10:27:17.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:27:17 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:27:17.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:27:17 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:27:17.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:27:17 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:27:17.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:27:17 smithi097 ceph-mon[29197]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-04-23T10:27:17.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:27:17 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:27:17.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:27:17 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:27:17.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:27:17 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:27:17.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:27:17 smithi183 ceph-mon[32317]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-04-23T10:27:17.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:27:17 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:27:17.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:27:17 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:27:17.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:27:17 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:27:18.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:27:18 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:27:18.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:27:18 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:27:18.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:27:18 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:27:18.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:27:18 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:27:18.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:27:18 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:27:18.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:27:18 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:27:18.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:27:18 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:27:18.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:27:18 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:27:18.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:27:18 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:27:19.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:27:19 smithi196 ceph-mon[32919]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.1 KiB/s wr, 13 op/s 2024-04-23T10:27:19.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:27:19 smithi097 ceph-mon[29197]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.1 KiB/s wr, 13 op/s 2024-04-23T10:27:19.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:27:19 smithi183 ceph-mon[32317]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.1 KiB/s wr, 13 op/s 2024-04-23T10:27:19.525 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.523+0000 7efca8308640 1 -- 172.21.15.97:0/2873749494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efca0107330 msgr2=0x7efca010ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:19.525 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.523+0000 7efca8308640 1 --2- 172.21.15.97:0/2873749494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efca0107330 0x7efca010ee90 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7efc98009cd0 tx=0x7efc9802a8d0 comp rx=0 tx=0).stop 2024-04-23T10:27:19.525 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.523+0000 7efca8308640 1 -- 172.21.15.97:0/2873749494 shutdown_connections 2024-04-23T10:27:19.525 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.523+0000 7efca8308640 1 --2- 172.21.15.97:0/2873749494 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efca0107330 0x7efca010ee90 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:19.525 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.523+0000 7efca8308640 1 --2- 172.21.15.97:0/2873749494 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efca01010d0 0x7efca0106c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:19.525 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.523+0000 7efca8308640 1 --2- 172.21.15.97:0/2873749494 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efca0100720 0x7efca0100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:19.525 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.523+0000 7efca8308640 1 -- 172.21.15.97:0/2873749494 >> 172.21.15.97:0/2873749494 conn(0x7efca0077370 msgr2=0x7efca00fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:27:19.525 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.523+0000 7efca8308640 1 -- 172.21.15.97:0/2873749494 shutdown_connections 2024-04-23T10:27:19.525 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.523+0000 7efca8308640 1 -- 172.21.15.97:0/2873749494 wait complete. 2024-04-23T10:27:19.525 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.524+0000 7efca8308640 1 Processor -- start 2024-04-23T10:27:19.525 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.524+0000 7efca8308640 1 -- start start 2024-04-23T10:27:19.525 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.524+0000 7efca8308640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efca0100720 0x7efca01985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:19.525 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.524+0000 7efca8308640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efca0107330 0x7efca0198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:19.526 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.524+0000 7efca8308640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efca019ce80 0x7efca019d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:19.526 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.524+0000 7efca8308640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7efca01121e0 con 0x7efca0107330 2024-04-23T10:27:19.526 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.524+0000 7efca8308640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7efca0112060 con 0x7efca019ce80 2024-04-23T10:27:19.526 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.524+0000 7efca8308640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efca0112360 con 0x7efca0100720 2024-04-23T10:27:19.526 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.524+0000 7efca58a3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efca0107330 0x7efca0198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:19.526 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.524+0000 7efca58a3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efca0107330 0x7efca0198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39124/0 (socket says 172.21.15.97:39124) 2024-04-23T10:27:19.526 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.524+0000 7efca58a3640 1 -- 172.21.15.97:0/2303304109 learned_addr learned my addr 172.21.15.97:0/2303304109 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:27:19.526 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.524+0000 7efca60a4640 1 --2- 172.21.15.97:0/2303304109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efca0100720 0x7efca01985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:19.526 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.524+0000 7efca68a5640 1 --2- 172.21.15.97:0/2303304109 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efca019ce80 0x7efca019d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:19.526 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.525+0000 7efca58a3640 1 -- 172.21.15.97:0/2303304109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efca0100720 msgr2=0x7efca01985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:19.526 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.525+0000 7efca58a3640 1 --2- 172.21.15.97:0/2303304109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efca0100720 0x7efca01985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:19.526 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.525+0000 7efca58a3640 1 -- 172.21.15.97:0/2303304109 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efca019ce80 msgr2=0x7efca019d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:19.527 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.525+0000 7efca58a3640 1 --2- 172.21.15.97:0/2303304109 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efca019ce80 0x7efca019d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:19.527 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.525+0000 7efca58a3640 1 -- 172.21.15.97:0/2303304109 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efca019d970 con 0x7efca0107330 2024-04-23T10:27:19.527 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.525+0000 7efca58a3640 1 --2- 172.21.15.97:0/2303304109 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efca0107330 0x7efca0198b00 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7efc9000c970 tx=0x7efc9000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:27:19.527 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.525+0000 7efc977fe640 1 -- 172.21.15.97:0/2303304109 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc90014070 con 0x7efca0107330 2024-04-23T10:27:19.527 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.525+0000 7efca8308640 1 -- 172.21.15.97:0/2303304109 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efca019dc60 con 0x7efca0107330 2024-04-23T10:27:19.527 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.525+0000 7efc977fe640 1 -- 172.21.15.97:0/2303304109 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc90004440 con 0x7efca0107330 2024-04-23T10:27:19.527 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.525+0000 7efc977fe640 1 -- 172.21.15.97:0/2303304109 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc90003e50 con 0x7efca0107330 2024-04-23T10:27:19.527 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.525+0000 7efca8308640 1 -- 172.21.15.97:0/2303304109 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efca01a5450 con 0x7efca0107330 2024-04-23T10:27:19.529 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.528+0000 7efc977fe640 1 -- 172.21.15.97:0/2303304109 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7efc90017070 con 0x7efca0107330 2024-04-23T10:27:19.529 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.528+0000 7efcab025740 1 Processor -- start 2024-04-23T10:27:19.529 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.529+0000 7efcab025740 1 -- start start 2024-04-23T10:27:19.529 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.529+0000 7efcab025740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622c3cde620 0x5622c3cdea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:19.529 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.529+0000 7efcab025740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622c3cdef40 0x5622c3ce92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:19.529 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.529+0000 7efcab025740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622c3ce9800 0x5622c3cebbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:19.529 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.529+0000 7efcab025740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5622c3c43970 con 0x5622c3cde620 2024-04-23T10:27:19.529 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.529+0000 7efcab025740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5622c3be04a0 con 0x5622c3cdef40 2024-04-23T10:27:19.530 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.529+0000 7efcab025740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5622c3c3f930 con 0x5622c3ce9800 2024-04-23T10:27:19.530 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.529+0000 7efca68a5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622c3cde620 0x5622c3cdea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:19.530 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.529+0000 7efca68a5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622c3cde620 0x5622c3cdea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39138/0 (socket says 172.21.15.97:39138) 2024-04-23T10:27:19.530 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.529+0000 7efca68a5640 1 -- 172.21.15.97:0/1750046232 learned_addr learned my addr 172.21.15.97:0/1750046232 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:27:19.530 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.529+0000 7efca60a4640 1 --2- 172.21.15.97:0/1750046232 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622c3cdef40 0x5622c3ce92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:19.530 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.529+0000 7efca58a3640 1 --2- 172.21.15.97:0/1750046232 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622c3ce9800 0x5622c3cebbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:19.530 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.529+0000 7efc977fe640 1 --2- 172.21.15.97:0/2303304109 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7efc8c0a4110 0x7efc8c0a65d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:19.530 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.529+0000 7efca60a4640 1 -- 172.21.15.97:0/1750046232 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622c3ce9800 msgr2=0x5622c3cebbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:27:19.530 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.529+0000 7efca60a4640 1 --2- 172.21.15.97:0/1750046232 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622c3ce9800 0x5622c3cebbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:19.530 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.529+0000 7efca60a4640 1 -- 172.21.15.97:0/1750046232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622c3cde620 msgr2=0x5622c3cdea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:27:19.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.529+0000 7efca60a4640 1 --2- 172.21.15.97:0/1750046232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622c3cde620 0x5622c3cdea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:19.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.529+0000 7efca60a4640 1 -- 172.21.15.97:0/1750046232 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622c3956210 con 0x5622c3cdef40 2024-04-23T10:27:19.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.529+0000 7efca68a5640 1 --2- 172.21.15.97:0/1750046232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622c3cde620 0x5622c3cdea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T10:27:19.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.529+0000 7efca68a5640 1 --2- 172.21.15.97:0/2303304109 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7efc8c0a4110 0x7efc8c0a65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:19.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.530+0000 7efc977fe640 1 -- 172.21.15.97:0/2303304109 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7efc900d04e0 con 0x7efca0107330 2024-04-23T10:27:19.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.530+0000 7efca60a4640 1 --2- 172.21.15.97:0/1750046232 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622c3cdef40 0x5622c3ce92c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7efc9c005fd0 tx=0x7efc9c0107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51a6ea58842c616f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:27:19.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.530+0000 7efc94bfe640 1 -- 172.21.15.97:0/1750046232 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc9c01c680 con 0x5622c3cdef40 2024-04-23T10:27:19.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.530+0000 7efca68a5640 1 --2- 172.21.15.97:0/2303304109 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7efc8c0a4110 0x7efc8c0a65d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7efc98009430 tx=0x7efc980095c0 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:27:19.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.530+0000 7efc94bfe640 1 -- 172.21.15.97:0/1750046232 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc9c00be20 con 0x5622c3cdef40 2024-04-23T10:27:19.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.530+0000 7efc94bfe640 1 -- 172.21.15.97:0/1750046232 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc9c011ba0 con 0x5622c3cdef40 2024-04-23T10:27:19.535 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.531+0000 7efcab025740 1 -- 172.21.15.97:0/1750046232 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622c3cdef40 msgr2=0x5622c3ce92c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:19.535 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.531+0000 7efcab025740 1 --2- 172.21.15.97:0/1750046232 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622c3cdef40 0x5622c3ce92c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7efc9c005fd0 tx=0x7efc9c0107a0 comp rx=0 tx=0).stop 2024-04-23T10:27:19.535 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.531+0000 7efcab025740 1 -- 172.21.15.97:0/1750046232 shutdown_connections 2024-04-23T10:27:19.536 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.531+0000 7efcab025740 1 --2- 172.21.15.97:0/1750046232 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622c3ce9800 0x5622c3cebbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:19.536 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.531+0000 7efcab025740 1 --2- 172.21.15.97:0/1750046232 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622c3cdef40 0x5622c3ce92c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:19.536 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.531+0000 7efcab025740 1 --2- 172.21.15.97:0/1750046232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622c3cde620 0x5622c3cdea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:19.536 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.531+0000 7efcab025740 1 -- 172.21.15.97:0/1750046232 >> 172.21.15.97:0/1750046232 conn(0x5622c3cdab30 msgr2=0x5622c3cdaf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:27:19.536 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.531+0000 7efcab025740 1 -- 172.21.15.97:0/1750046232 shutdown_connections 2024-04-23T10:27:19.536 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.531+0000 7efcab025740 1 -- 172.21.15.97:0/1750046232 wait complete. 2024-04-23T10:27:19.536 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.531+0000 7efcab025740 1 -- start start 2024-04-23T10:27:19.536 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.531+0000 7efcab025740 1 Processor -- start 2024-04-23T10:27:19.536 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.531+0000 7efcab025740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622c3cde620 0x5622c3ce9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:19.536 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.531+0000 7efcab025740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622c3cdef40 0x5622c3ce6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:19.536 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.531+0000 7efcab025740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622c3ce9800 0x5622c3ce6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:19.536 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.531+0000 7efcab025740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5622c3cec430 con 0x5622c3ce9800 2024-04-23T10:27:19.536 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.531+0000 7efcab025740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5622c3bde0b0 con 0x5622c3cde620 2024-04-23T10:27:19.536 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.531+0000 7efcab025740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5622c3cec5b0 con 0x5622c3cdef40 2024-04-23T10:27:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.531+0000 7efcab025740 10 client.0 _ll_register_callbacks cb 0x5622c3bcbd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:27:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.532+0000 7efca68a5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622c3ce9800 0x5622c3ce6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.532+0000 7efca60a4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622c3cde620 0x5622c3ce9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.532+0000 7efca68a5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622c3ce9800 0x5622c3ce6960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39150/0 (socket says 172.21.15.97:39150) 2024-04-23T10:27:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.532+0000 7efca68a5640 1 -- 172.21.15.97:0/1401198799 learned_addr learned my addr 172.21.15.97:0/1401198799 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:27:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.532+0000 7efca58a3640 1 --2- 172.21.15.97:0/1401198799 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622c3cdef40 0x5622c3ce6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.532+0000 7efca68a5640 1 -- 172.21.15.97:0/1401198799 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622c3cdef40 msgr2=0x5622c3ce6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:27:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.532+0000 7efca68a5640 1 --2- 172.21.15.97:0/1401198799 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622c3cdef40 0x5622c3ce6420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.532+0000 7efca68a5640 1 -- 172.21.15.97:0/1401198799 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622c3cde620 msgr2=0x5622c3ce9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:27:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.532+0000 7efca68a5640 1 --2- 172.21.15.97:0/1401198799 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622c3cde620 0x5622c3ce9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.532+0000 7efca68a5640 1 -- 172.21.15.97:0/1401198799 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622c3a22620 con 0x5622c3ce9800 2024-04-23T10:27:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.532+0000 7efca60a4640 1 --2- 172.21.15.97:0/1401198799 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622c3cde620 0x5622c3ce9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T10:27:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.532+0000 7efca68a5640 1 --2- 172.21.15.97:0/1401198799 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622c3ce9800 0x5622c3ce6960 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7efc98006010 tx=0x7efc9802a8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0bdd4c15ac198fb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:27:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.532+0000 7efc89ffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:27:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.532+0000 7efc89ffb640 1 -- 172.21.15.97:0/1401198799 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc98042070 con 0x5622c3ce9800 2024-04-23T10:27:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.532+0000 7efc89ffb640 1 -- 172.21.15.97:0/1401198799 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7efc98004050 con 0x5622c3ce9800 2024-04-23T10:27:19.538 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.532+0000 7efcab025740 1 -- 172.21.15.97:0/1401198799 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc9c00be20 con 0x5622c3ce9800 2024-04-23T10:27:19.538 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.532+0000 7efc89ffb640 1 -- 172.21.15.97:0/1401198799 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7efc98057020 con 0x5622c3ce9800 2024-04-23T10:27:19.538 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.532+0000 7efcab025740 10 client.15843 fetch_fsmap learned FSMap version 26 2024-04-23T10:27:19.538 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.532+0000 7efcab025740 1 -- 172.21.15.97:0/1401198799 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5622c3cf8340 con 0x5622c3ce9800 2024-04-23T10:27:19.538 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.533+0000 7efc89ffb640 1 -- 172.21.15.97:0/1401198799 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc9c00be20 con 0x5622c3ce9800 2024-04-23T10:27:19.538 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.533+0000 7efc89ffb640 1 -- 172.21.15.97:0/1401198799 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7efc980309a0 con 0x5622c3ce9800 2024-04-23T10:27:19.538 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.533+0000 7efcab025740 10 client.15843 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:27:19.538 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.533+0000 7efcab025740 10 client.15843 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:27:19.538 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.533+0000 7efcab025740 20 client.15843 populate_metadata read hostname 'smithi097' 2024-04-23T10:27:19.538 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.533+0000 7efcab025740 1 --2- 172.21.15.97:0/1401198799 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5622c3cf9800 0x5622c3cfbc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:19.538 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.533+0000 7efcab025740 4 client.15843 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:27:19.538 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.533+0000 7efcab025740 1 -- 172.21.15.97:0/1401198799 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5622c3c51390 con 0x5622c3cf9800 2024-04-23T10:27:19.538 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.533+0000 7efca60a4640 1 --2- 172.21.15.97:0/1401198799 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5622c3cf9800 0x5622c3cfbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:19.539 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.534+0000 7efc89ffb640 10 client.15843 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:27:19.539 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.534+0000 7efca60a4640 1 --2- 172.21.15.97:0/1401198799 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5622c3cf9800 0x5622c3cfbc10 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63f80a57d2ff7294 server_cookie=d3351f7e3c7c3f34 in_seq=0 out_seq=0 2024-04-23T10:27:19.539 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.534+0000 7efc89ffb640 1 -- 172.21.15.97:0/1401198799 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5622c3c51390 con 0x5622c3cf9800 2024-04-23T10:27:19.539 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.534+0000 7efc89ffb640 10 client.15843 handle_command_reply: tid=0 2024-04-23T10:27:19.539 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.534+0000 7efcab025740 1 client.15843 shutdown 2024-04-23T10:27:19.539 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.535+0000 7efcab025740 1 -- 172.21.15.97:0/1401198799 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622c3ce9800 msgr2=0x5622c3ce6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:19.539 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.535+0000 7efcab025740 1 --2- 172.21.15.97:0/1401198799 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622c3ce9800 0x5622c3ce6960 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7efc98006010 tx=0x7efc9802a8d0 comp rx=0 tx=0).stop 2024-04-23T10:27:19.539 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.535+0000 7efcab025740 1 -- 172.21.15.97:0/1401198799 shutdown_connections 2024-04-23T10:27:19.539 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.535+0000 7efcab025740 1 --2- 172.21.15.97:0/1401198799 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5622c3cf9800 0x5622c3cfbc10 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:19.539 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.535+0000 7efcab025740 1 --2- 172.21.15.97:0/1401198799 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622c3ce9800 0x5622c3ce6960 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:19.539 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.535+0000 7efc89ffb640 0 client.15843 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:27:19.539 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.535+0000 7efcab025740 1 --2- 172.21.15.97:0/1401198799 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622c3cdef40 0x5622c3ce6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:19.539 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.535+0000 7efcab025740 1 --2- 172.21.15.97:0/1401198799 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622c3cde620 0x5622c3ce9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:19.539 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.535+0000 7efcab025740 1 -- 172.21.15.97:0/1401198799 >> 172.21.15.97:0/1401198799 conn(0x5622c3cdab30 msgr2=0x5622c3ce08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:27:19.540 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.535+0000 7efcab025740 1 -- 172.21.15.97:0/1401198799 shutdown_connections 2024-04-23T10:27:19.540 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.535+0000 7efcab025740 1 -- 172.21.15.97:0/1401198799 wait complete. 2024-04-23T10:27:19.540 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.535+0000 7efcab025740 20 client.15843 trim_cache size 0 max 16384 2024-04-23T10:27:19.553 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.549+0000 7efcab025740 1 Processor -- start 2024-04-23T10:27:19.553 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.550+0000 7efcab025740 1 -- start start 2024-04-23T10:27:19.553 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.550+0000 7efcab025740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622c3cde620 0x5622c3ce8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:19.553 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.550+0000 7efcab025740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622c3cdef40 0x5622c3d01520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:19.554 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.550+0000 7efcab025740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622c3ce9800 0x5622c3d01a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:19.554 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.550+0000 7efcab025740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7efca01ad9e0 con 0x5622c3cdef40 2024-04-23T10:27:19.554 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.550+0000 7efcab025740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7efca01af520 con 0x5622c3ce9800 2024-04-23T10:27:19.554 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.550+0000 7efcab025740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efca01ae0b0 con 0x5622c3cde620 2024-04-23T10:27:19.554 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.550+0000 7efca58a3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622c3cdef40 0x5622c3d01520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:19.554 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.550+0000 7efca58a3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622c3cdef40 0x5622c3d01520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39158/0 (socket says 172.21.15.97:39158) 2024-04-23T10:27:19.554 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.550+0000 7efca58a3640 1 -- 172.21.15.97:0/3674071017 learned_addr learned my addr 172.21.15.97:0/3674071017 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:27:19.554 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.550+0000 7efca58a3640 1 -- 172.21.15.97:0/3674071017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622c3cde620 msgr2=0x5622c3ce8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:27:19.554 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.550+0000 7efca58a3640 1 --2- 172.21.15.97:0/3674071017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622c3cde620 0x5622c3ce8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:19.554 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.550+0000 7efca58a3640 1 -- 172.21.15.97:0/3674071017 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622c3ce9800 msgr2=0x5622c3d01a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:27:19.554 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.550+0000 7efca68a5640 1 --2- 172.21.15.97:0/3674071017 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622c3ce9800 0x5622c3d01a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:19.554 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.550+0000 7efca58a3640 1 --2- 172.21.15.97:0/3674071017 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622c3ce9800 0x5622c3d01a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:19.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.550+0000 7efca58a3640 1 -- 172.21.15.97:0/3674071017 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622c3cdb700 con 0x5622c3cdef40 2024-04-23T10:27:19.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.550+0000 7efca68a5640 1 --2- 172.21.15.97:0/3674071017 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622c3ce9800 0x5622c3d01a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:27:19.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.550+0000 7efca58a3640 1 --2- 172.21.15.97:0/3674071017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622c3cdef40 0x5622c3d01520 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7efc900cdb80 tx=0x7efc9009c180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=337367dded83583f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:27:19.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.550+0000 7efc8bfff640 1 -- 172.21.15.97:0/3674071017 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc9009c290 con 0x5622c3cdef40 2024-04-23T10:27:19.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.550+0000 7efc8bfff640 1 -- 172.21.15.97:0/3674071017 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc900a0640 con 0x5622c3cdef40 2024-04-23T10:27:19.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.550+0000 7efc8bfff640 1 -- 172.21.15.97:0/3674071017 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc9009b940 con 0x5622c3cdef40 2024-04-23T10:27:19.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.551+0000 7efcab025740 1 -- 172.21.15.97:0/3674071017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622c3cdef40 msgr2=0x5622c3d01520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:19.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.551+0000 7efcab025740 1 --2- 172.21.15.97:0/3674071017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622c3cdef40 0x5622c3d01520 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7efc900cdb80 tx=0x7efc9009c180 comp rx=0 tx=0).stop 2024-04-23T10:27:19.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.551+0000 7efcab025740 1 -- 172.21.15.97:0/3674071017 shutdown_connections 2024-04-23T10:27:19.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.551+0000 7efcab025740 1 --2- 172.21.15.97:0/3674071017 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622c3ce9800 0x5622c3d01a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:19.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.551+0000 7efcab025740 1 --2- 172.21.15.97:0/3674071017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622c3cdef40 0x5622c3d01520 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:19.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.551+0000 7efcab025740 1 --2- 172.21.15.97:0/3674071017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622c3cde620 0x5622c3ce8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:19.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.551+0000 7efcab025740 1 -- 172.21.15.97:0/3674071017 >> 172.21.15.97:0/3674071017 conn(0x5622c3cdab30 msgr2=0x5622c3ce63e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:27:19.556 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.551+0000 7efcab025740 1 -- 172.21.15.97:0/3674071017 shutdown_connections 2024-04-23T10:27:19.556 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.551+0000 7efcab025740 1 -- 172.21.15.97:0/3674071017 wait complete. 2024-04-23T10:27:19.556 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.551+0000 7efcab025740 1 -- start start 2024-04-23T10:27:19.556 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.551+0000 7efcab025740 1 Processor -- start 2024-04-23T10:27:19.556 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.552+0000 7efcab025740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622c3cde620 0x5622c3cf1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:19.556 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.552+0000 7efcab025740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622c3cdef40 0x5622c3ceedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:19.556 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.552+0000 7efcab025740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622c3ce9800 0x5622c3cef310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:19.556 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.552+0000 7efcab025740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7efca01aee50 con 0x5622c3ce9800 2024-04-23T10:27:19.556 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.552+0000 7efcab025740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7efca01ae780 con 0x5622c3cde620 2024-04-23T10:27:19.556 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.552+0000 7efcab025740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7efca01ab9c0 con 0x5622c3cdef40 2024-04-23T10:27:19.556 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.552+0000 7efcab025740 10 client.0 _ll_register_callbacks cb 0x7efc98009800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:27:19.556 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.552+0000 7efca68a5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622c3ce9800 0x5622c3cef310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:19.556 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.552+0000 7efca68a5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622c3ce9800 0x5622c3cef310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39174/0 (socket says 172.21.15.97:39174) 2024-04-23T10:27:19.556 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.552+0000 7efca68a5640 1 -- 172.21.15.97:0/1981632220 learned_addr learned my addr 172.21.15.97:0/1981632220 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:27:19.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.552+0000 7efca60a4640 1 --2- 172.21.15.97:0/1981632220 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622c3cde620 0x5622c3cf1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:19.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.552+0000 7efca68a5640 1 -- 172.21.15.97:0/1981632220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622c3cdef40 msgr2=0x5622c3ceedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:27:19.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.552+0000 7efca58a3640 1 --2- 172.21.15.97:0/1981632220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622c3cdef40 0x5622c3ceedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:19.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.552+0000 7efca68a5640 1 --2- 172.21.15.97:0/1981632220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622c3cdef40 0x5622c3ceedd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:19.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.552+0000 7efca68a5640 1 -- 172.21.15.97:0/1981632220 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622c3cde620 msgr2=0x5622c3cf1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:27:19.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.552+0000 7efca68a5640 1 --2- 172.21.15.97:0/1981632220 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622c3cde620 0x5622c3cf1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:19.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.552+0000 7efca68a5640 1 -- 172.21.15.97:0/1981632220 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622c3cef980 con 0x5622c3ce9800 2024-04-23T10:27:19.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.552+0000 7efca68a5640 1 --2- 172.21.15.97:0/1981632220 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622c3ce9800 0x5622c3cef310 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7efc9802adf0 tx=0x7efc98003dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a56af9158e680d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:27:19.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.552+0000 7efc89ffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:27:19.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.552+0000 7efc89ffb640 1 -- 172.21.15.97:0/1981632220 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc98042040 con 0x5622c3ce9800 2024-04-23T10:27:19.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.552+0000 7efc89ffb640 1 -- 172.21.15.97:0/1981632220 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7efc980309a0 con 0x5622c3ce9800 2024-04-23T10:27:19.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.552+0000 7efcab025740 1 -- 172.21.15.97:0/1981632220 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc900a0640 con 0x5622c3ce9800 2024-04-23T10:27:19.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.553+0000 7efc89ffb640 1 -- 172.21.15.97:0/1981632220 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7efc98030ba0 con 0x5622c3ce9800 2024-04-23T10:27:19.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.553+0000 7efcab025740 10 client.15849 fetch_fsmap learned FSMap version 26 2024-04-23T10:27:19.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.553+0000 7efcab025740 1 -- 172.21.15.97:0/1981632220 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5622c3d5fbb0 con 0x5622c3ce9800 2024-04-23T10:27:19.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.553+0000 7efc89ffb640 1 -- 172.21.15.97:0/1981632220 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc900a0640 con 0x5622c3ce9800 2024-04-23T10:27:19.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.553+0000 7efc89ffb640 1 -- 172.21.15.97:0/1981632220 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7efc98064030 con 0x5622c3ce9800 2024-04-23T10:27:19.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.553+0000 7efcab025740 10 client.15849 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:27:19.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.553+0000 7efcab025740 10 client.15849 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:27:19.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.553+0000 7efcab025740 20 client.15849 populate_metadata read hostname 'smithi097' 2024-04-23T10:27:19.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.553+0000 7efcab025740 1 --2- 172.21.15.97:0/1981632220 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5622c3d60280 0x5622c3d626a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:19.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.553+0000 7efcab025740 4 client.15849 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:27:19.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.553+0000 7efcab025740 1 -- 172.21.15.97:0/1981632220 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5622c3c4b200 con 0x5622c3d60280 2024-04-23T10:27:19.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.553+0000 7efca60a4640 1 --2- 172.21.15.97:0/1981632220 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5622c3d60280 0x5622c3d626a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:19.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.554+0000 7efc89ffb640 10 client.15849 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:27:19.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.554+0000 7efca60a4640 1 --2- 172.21.15.97:0/1981632220 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5622c3d60280 0x5622c3d626a0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe98570b0c97e31d server_cookie=2fcb7b4270e55d0c in_seq=0 out_seq=0 2024-04-23T10:27:19.559 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.559+0000 7efc89ffb640 1 -- 172.21.15.97:0/1981632220 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5622c3c4b200 con 0x5622c3d60280 2024-04-23T10:27:19.559 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.559+0000 7efc89ffb640 10 client.15849 handle_command_reply: tid=0 2024-04-23T10:27:19.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.559+0000 7efcab025740 1 client.15849 shutdown 2024-04-23T10:27:19.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.560+0000 7efcab025740 1 -- 172.21.15.97:0/1981632220 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622c3ce9800 msgr2=0x5622c3cef310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:19.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.560+0000 7efcab025740 1 --2- 172.21.15.97:0/1981632220 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622c3ce9800 0x5622c3cef310 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7efc9802adf0 tx=0x7efc98003dc0 comp rx=0 tx=0).stop 2024-04-23T10:27:19.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.560+0000 7efcab025740 1 -- 172.21.15.97:0/1981632220 shutdown_connections 2024-04-23T10:27:19.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.560+0000 7efcab025740 1 --2- 172.21.15.97:0/1981632220 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5622c3d60280 0x5622c3d626a0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:19.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.560+0000 7efcab025740 1 --2- 172.21.15.97:0/1981632220 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5622c3ce9800 0x5622c3cef310 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:19.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.560+0000 7efcab025740 1 --2- 172.21.15.97:0/1981632220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622c3cdef40 0x5622c3ceedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:19.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.560+0000 7efcab025740 1 --2- 172.21.15.97:0/1981632220 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5622c3cde620 0x5622c3cf1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:19.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.560+0000 7efc89ffb640 0 client.15849 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:27:19.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.560+0000 7efcab025740 1 -- 172.21.15.97:0/1981632220 >> 172.21.15.97:0/1981632220 conn(0x5622c3cdab30 msgr2=0x5622c3cd9420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:27:19.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.560+0000 7efcab025740 1 -- 172.21.15.97:0/1981632220 shutdown_connections 2024-04-23T10:27:19.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.560+0000 7efcab025740 1 -- 172.21.15.97:0/1981632220 wait complete. 2024-04-23T10:27:19.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.560+0000 7efcab025740 20 client.15849 trim_cache size 0 max 16384 2024-04-23T10:27:19.561 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:27:19.561 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:27:19.561 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:27:19.561 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:27:19.562 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.560+0000 7efca8308640 1 -- 172.21.15.97:0/2303304109 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7efc8c0a4110 msgr2=0x7efc8c0a65d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:19.562 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.560+0000 7efca8308640 1 --2- 172.21.15.97:0/2303304109 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7efc8c0a4110 0x7efc8c0a65d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7efc98009430 tx=0x7efc980095c0 comp rx=0 tx=0).stop 2024-04-23T10:27:19.562 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.560+0000 7efca8308640 1 -- 172.21.15.97:0/2303304109 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efca0107330 msgr2=0x7efca0198b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:19.562 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.560+0000 7efca8308640 1 --2- 172.21.15.97:0/2303304109 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efca0107330 0x7efca0198b00 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7efc9000c970 tx=0x7efc9000ce30 comp rx=0 tx=0).stop 2024-04-23T10:27:19.562 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.560+0000 7efca8308640 1 -- 172.21.15.97:0/2303304109 shutdown_connections 2024-04-23T10:27:19.562 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.560+0000 7efca8308640 1 --2- 172.21.15.97:0/2303304109 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7efc8c0a4110 0x7efc8c0a65d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:19.562 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.560+0000 7efca8308640 1 --2- 172.21.15.97:0/2303304109 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efca019ce80 0x7efca019d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:19.562 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.560+0000 7efca8308640 1 --2- 172.21.15.97:0/2303304109 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efca0107330 0x7efca0198b00 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:19.562 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.560+0000 7efca8308640 1 --2- 172.21.15.97:0/2303304109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efca0100720 0x7efca01985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:19.562 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.560+0000 7efca8308640 1 -- 172.21.15.97:0/2303304109 >> 172.21.15.97:0/2303304109 conn(0x7efca0077370 msgr2=0x7efca00fc990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:27:19.562 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.560+0000 7efca8308640 1 -- 172.21.15.97:0/2303304109 shutdown_connections 2024-04-23T10:27:19.562 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:19.560+0000 7efca8308640 1 -- 172.21.15.97:0/2303304109 wait complete. 2024-04-23T10:27:21.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:27:21 smithi196 ceph-mon[32919]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-23T10:27:21.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:27:21 smithi097 ceph-mon[29197]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-23T10:27:21.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:27:21 smithi183 ceph-mon[32317]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-23T10:27:23.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:27:23 smithi196 ceph-mon[32919]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-23T10:27:23.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:27:23 smithi097 ceph-mon[29197]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-23T10:27:23.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:27:23 smithi183 ceph-mon[32317]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-23T10:27:23.724 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T10:27:23.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:23.837+0000 7f68b6610640 1 -- 172.21.15.97:0/3123135413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68b01031e0 msgr2=0x7f68b01035c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:23.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:23.837+0000 7f68b6610640 1 --2- 172.21.15.97:0/3123135413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68b01031e0 0x7f68b01035c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f68a0009900 tx=0x7f68a002a510 comp rx=0 tx=0).stop 2024-04-23T10:27:23.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:23.837+0000 7f68b6610640 1 -- 172.21.15.97:0/3123135413 shutdown_connections 2024-04-23T10:27:23.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:23.837+0000 7f68b6610640 1 --2- 172.21.15.97:0/3123135413 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68b010c520 0x7f68b010e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:23.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:23.837+0000 7f68b6610640 1 --2- 172.21.15.97:0/3123135413 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68b0103b00 0x7f68b010bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:23.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:23.837+0000 7f68b6610640 1 --2- 172.21.15.97:0/3123135413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68b01031e0 0x7f68b01035c0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:23.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:23.837+0000 7f68b6610640 1 -- 172.21.15.97:0/3123135413 >> 172.21.15.97:0/3123135413 conn(0x7f68b0077f50 msgr2=0x7f68b0078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:27:23.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:23.837+0000 7f68b6610640 1 -- 172.21.15.97:0/3123135413 shutdown_connections 2024-04-23T10:27:23.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:23.837+0000 7f68b6610640 1 -- 172.21.15.97:0/3123135413 wait complete. 2024-04-23T10:27:23.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:23.837+0000 7f68b6610640 1 Processor -- start 2024-04-23T10:27:23.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:23.837+0000 7f68b6610640 1 -- start start 2024-04-23T10:27:23.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:23.838+0000 7f68b6610640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68b01031e0 0x7f68b01986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:23.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:23.838+0000 7f68b6610640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68b0103b00 0x7f68b0198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:23.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:23.838+0000 7f68b6610640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68b010c520 0x7f68b019cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:23.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:23.838+0000 7f68b6610640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f68b01121b0 con 0x7f68b01031e0 2024-04-23T10:27:23.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:23.838+0000 7f68b6610640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f68b0112030 con 0x7f68b010c520 2024-04-23T10:27:23.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:23.838+0000 7f68b6610640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f68b0112330 con 0x7f68b0103b00 2024-04-23T10:27:23.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:23.838+0000 7f68affff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68b01031e0 0x7f68b01986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:23.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:23.838+0000 7f68affff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68b01031e0 0x7f68b01986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39180/0 (socket says 172.21.15.97:39180) 2024-04-23T10:27:23.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:23.838+0000 7f68affff640 1 -- 172.21.15.97:0/3574493696 learned_addr learned my addr 172.21.15.97:0/3574493696 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:27:23.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:23.838+0000 7f68af7fe640 1 --2- 172.21.15.97:0/3574493696 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68b0103b00 0x7f68b0198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:23.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:23.838+0000 7f68b4bad640 1 --2- 172.21.15.97:0/3574493696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68b010c520 0x7f68b019cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:23.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:23.838+0000 7f68affff640 1 -- 172.21.15.97:0/3574493696 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68b0103b00 msgr2=0x7f68b0198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:23.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:23.838+0000 7f68affff640 1 --2- 172.21.15.97:0/3574493696 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68b0103b00 0x7f68b0198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:23.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:23.838+0000 7f68affff640 1 -- 172.21.15.97:0/3574493696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68b010c520 msgr2=0x7f68b019cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:23.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:23.838+0000 7f68affff640 1 --2- 172.21.15.97:0/3574493696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68b010c520 0x7f68b019cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:23.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:23.838+0000 7f68affff640 1 -- 172.21.15.97:0/3574493696 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68b019d6c0 con 0x7f68b01031e0 2024-04-23T10:27:23.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:23.838+0000 7f68affff640 1 --2- 172.21.15.97:0/3574493696 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68b01031e0 0x7f68b01986d0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f68a0005940 tx=0x7f68a0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:27:23.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:23.838+0000 7f68ad7fa640 1 -- 172.21.15.97:0/3574493696 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a003f030 con 0x7f68b01031e0 2024-04-23T10:27:23.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:23.838+0000 7f68ad7fa640 1 -- 172.21.15.97:0/3574493696 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68a0003aa0 con 0x7f68b01031e0 2024-04-23T10:27:23.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:23.838+0000 7f68b6610640 1 -- 172.21.15.97:0/3574493696 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68b019d950 con 0x7f68b01031e0 2024-04-23T10:27:23.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:23.838+0000 7f68ad7fa640 1 -- 172.21.15.97:0/3574493696 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a0035070 con 0x7f68b01031e0 2024-04-23T10:27:23.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:23.838+0000 7f68b6610640 1 -- 172.21.15.97:0/3574493696 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68b01a5240 con 0x7f68b01031e0 2024-04-23T10:27:23.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:23.840+0000 7f68b6610640 1 -- 172.21.15.97:0/3574493696 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68b0068d70 con 0x7f68b01031e0 2024-04-23T10:27:23.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:23.841+0000 7f68ad7fa640 1 -- 172.21.15.97:0/3574493696 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f68a0042030 con 0x7f68b01031e0 2024-04-23T10:27:23.841 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:23.841+0000 7f68ad7fa640 1 --2- 172.21.15.97:0/3574493696 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f68980a3ed0 0x7f68980a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:23.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:23.841+0000 7f68af7fe640 1 --2- 172.21.15.97:0/3574493696 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f68980a3ed0 0x7f68980a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:23.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:23.842+0000 7f68ad7fa640 1 -- 172.21.15.97:0/3574493696 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f68a00f1090 con 0x7f68b01031e0 2024-04-23T10:27:23.842 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:23.842+0000 7f68af7fe640 1 --2- 172.21.15.97:0/3574493696 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f68980a3ed0 0x7f68980a6390 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f68b0100890 tx=0x7f68a400a2d0 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:27:23.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:23.846+0000 7f68ad7fa640 1 -- 172.21.15.97:0/3574493696 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f68a00bd540 con 0x7f68b01031e0 2024-04-23T10:27:24.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.062+0000 7f68b6610640 1 -- 172.21.15.97:0/3574493696 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f68b01076a0 con 0x7f68b01031e0 2024-04-23T10:27:24.061 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.062+0000 7f68ad7fa640 1 -- 172.21.15.97:0/3574493696 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11931 (secure 0 0 0) 0x7f68a00bce40 con 0x7f68b01031e0 2024-04-23T10:27:24.062 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:27:24.062 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3946878762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3946878762},{"type":"v1","addr":"172.21.15.97:6835","nonce":3946878762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/209800861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":209800861},{"type":"v1","addr":"172.21.15.97:6837","nonce":209800861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T10:24:06.910134+0000","modified":"2024-04-23T10:26:09.820985+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6839/418013575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":418013575},{"type":"v1","addr":"172.21.15.97:6839","nonce":418013575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6841/747242680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":747242680},{"type":"v1","addr":"172.21.15.97:6841","nonce":747242680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6833/4292847121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4292847121},{"type":"v1","addr":"172.21.15.183:6833","nonce":4292847121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6833/3885816565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3885816565},{"type":"v1","addr":"172.21.15.196:6833","nonce":3885816565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1897093105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1897093105},{"type":"v1","addr":"172.21.15.183:6835","nonce":1897093105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1171477563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1171477563},{"type":"v1","addr":"172.21.15.196:6835","nonce":1171477563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1898341911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1898341911},{"type":"v1","addr":"172.21.15.183:6837","nonce":1898341911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.196:6837/3094946502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3094946502},{"type":"v1","addr":"172.21.15.196:6837","nonce":3094946502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.196:6839/83843556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":83843556},{"type":"v1","addr":"172.21.15.196:6839","nonce":83843556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6839/1763841130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1763841130},{"type":"v1","addr":"172.21.15.183:6839","nonce":1763841130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24487,24403,24398,24469,24488]},"id":1}]} 2024-04-23T10:27:24.062 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 26 2024-04-23T10:27:24.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.064+0000 7f68b6610640 1 -- 172.21.15.97:0/3574493696 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f68980a3ed0 msgr2=0x7f68980a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:24.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.064+0000 7f68b6610640 1 --2- 172.21.15.97:0/3574493696 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f68980a3ed0 0x7f68980a6390 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f68b0100890 tx=0x7f68a400a2d0 comp rx=0 tx=0).stop 2024-04-23T10:27:24.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.064+0000 7f68b6610640 1 -- 172.21.15.97:0/3574493696 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68b01031e0 msgr2=0x7f68b01986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:24.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.064+0000 7f68b6610640 1 --2- 172.21.15.97:0/3574493696 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68b01031e0 0x7f68b01986d0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f68a0005940 tx=0x7f68a0004270 comp rx=0 tx=0).stop 2024-04-23T10:27:24.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.065+0000 7f68b6610640 1 -- 172.21.15.97:0/3574493696 shutdown_connections 2024-04-23T10:27:24.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.065+0000 7f68b6610640 1 --2- 172.21.15.97:0/3574493696 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f68980a3ed0 0x7f68980a6390 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:24.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.065+0000 7f68b6610640 1 --2- 172.21.15.97:0/3574493696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f68b010c520 0x7f68b019cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:24.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.065+0000 7f68b6610640 1 --2- 172.21.15.97:0/3574493696 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68b0103b00 0x7f68b0198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:24.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.065+0000 7f68b6610640 1 --2- 172.21.15.97:0/3574493696 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f68b01031e0 0x7f68b01986d0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:24.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.065+0000 7f68b6610640 1 -- 172.21.15.97:0/3574493696 >> 172.21.15.97:0/3574493696 conn(0x7f68b0077f50 msgr2=0x7f68b00fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:27:24.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.065+0000 7f68b6610640 1 -- 172.21.15.97:0/3574493696 shutdown_connections 2024-04-23T10:27:24.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.065+0000 7f68b6610640 1 -- 172.21.15.97:0/3574493696 wait complete. 2024-04-23T10:27:24.076 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --set-id c703558e --release --await' 2024-04-23T10:27:24.076 DEBUG:teuthology.orchestra.run.smithi097:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24488 quiesce db --set-id c703558e --release --await 2024-04-23T10:27:24.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.182+0000 7f40ecd29640 1 -- 172.21.15.97:0/589367354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40e8107260 msgr2=0x7f40e810ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:24.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.182+0000 7f40ecd29640 1 --2- 172.21.15.97:0/589367354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40e8107260 0x7f40e810ee30 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f40e0009ca0 tx=0x7f40e002a8c0 comp rx=0 tx=0).stop 2024-04-23T10:27:24.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.182+0000 7f40ecd29640 1 -- 172.21.15.97:0/589367354 shutdown_connections 2024-04-23T10:27:24.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.182+0000 7f40ecd29640 1 --2- 172.21.15.97:0/589367354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40e8107260 0x7f40e810ee30 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:24.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.182+0000 7f40ecd29640 1 --2- 172.21.15.97:0/589367354 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40e8102b40 0x7f40e8106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:24.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.182+0000 7f40ecd29640 1 --2- 172.21.15.97:0/589367354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40e8102190 0x7f40e8102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:24.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.182+0000 7f40ecd29640 1 -- 172.21.15.97:0/589367354 >> 172.21.15.97:0/589367354 conn(0x7f40e80793a0 msgr2=0x7f40e80797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:27:24.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.182+0000 7f40ecd29640 1 -- 172.21.15.97:0/589367354 shutdown_connections 2024-04-23T10:27:24.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.182+0000 7f40ecd29640 1 -- 172.21.15.97:0/589367354 wait complete. 2024-04-23T10:27:24.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.183+0000 7f40ecd29640 1 Processor -- start 2024-04-23T10:27:24.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.183+0000 7f40ecd29640 1 -- start start 2024-04-23T10:27:24.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.183+0000 7f40ecd29640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40e8102190 0x7f40e8198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:24.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.183+0000 7f40ecd29640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40e8107260 0x7f40e8198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:24.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.183+0000 7f40ecd29640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40e819cfe0 0x7f40e819d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:24.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.183+0000 7f40ecd29640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f40e81121d0 con 0x7f40e8107260 2024-04-23T10:27:24.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.183+0000 7f40ecd29640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f40e8112050 con 0x7f40e8102190 2024-04-23T10:27:24.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.183+0000 7f40ecd29640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f40e8112350 con 0x7f40e819cfe0 2024-04-23T10:27:24.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.183+0000 7f40dffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40e8107260 0x7f40e8198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:24.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.183+0000 7f40dffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40e8107260 0x7f40e8198cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39192/0 (socket says 172.21.15.97:39192) 2024-04-23T10:27:24.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.183+0000 7f40dffff640 1 -- 172.21.15.97:0/4078487122 learned_addr learned my addr 172.21.15.97:0/4078487122 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:27:24.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.183+0000 7f40e6d9d640 1 --2- 172.21.15.97:0/4078487122 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40e819cfe0 0x7f40e819d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:24.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.183+0000 7f40dffff640 1 -- 172.21.15.97:0/4078487122 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40e819cfe0 msgr2=0x7f40e819d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:24.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.183+0000 7f40dffff640 1 --2- 172.21.15.97:0/4078487122 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40e819cfe0 0x7f40e819d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:24.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.183+0000 7f40e659c640 1 --2- 172.21.15.97:0/4078487122 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40e8102190 0x7f40e8198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:24.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.183+0000 7f40dffff640 1 -- 172.21.15.97:0/4078487122 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40e8102190 msgr2=0x7f40e8198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:24.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.183+0000 7f40dffff640 1 --2- 172.21.15.97:0/4078487122 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40e8102190 0x7f40e8198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:24.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.183+0000 7f40dffff640 1 -- 172.21.15.97:0/4078487122 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40e819dad0 con 0x7f40e8107260 2024-04-23T10:27:24.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.183+0000 7f40e6d9d640 1 --2- 172.21.15.97:0/4078487122 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40e819cfe0 0x7f40e819d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:27:24.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.183+0000 7f40e659c640 1 --2- 172.21.15.97:0/4078487122 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40e8102190 0x7f40e8198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:27:24.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.184+0000 7f40dffff640 1 --2- 172.21.15.97:0/4078487122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40e8107260 0x7f40e8198cb0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f40d000c970 tx=0x7f40d000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:27:24.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.184+0000 7f40df7fe640 1 -- 172.21.15.97:0/4078487122 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d0007bb0 con 0x7f40e8107260 2024-04-23T10:27:24.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.184+0000 7f40df7fe640 1 -- 172.21.15.97:0/4078487122 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40d0007d50 con 0x7f40e8107260 2024-04-23T10:27:24.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.184+0000 7f40ecd29640 1 -- 172.21.15.97:0/4078487122 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40e819ddc0 con 0x7f40e8107260 2024-04-23T10:27:24.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.184+0000 7f40df7fe640 1 -- 172.21.15.97:0/4078487122 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d000f450 con 0x7f40e8107260 2024-04-23T10:27:24.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.184+0000 7f40ecd29640 1 -- 172.21.15.97:0/4078487122 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40e81a55b0 con 0x7f40e8107260 2024-04-23T10:27:24.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.186+0000 7f40df7fe640 1 -- 172.21.15.97:0/4078487122 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f40d0004ec0 con 0x7f40e8107260 2024-04-23T10:27:24.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.186+0000 7f40efa42740 1 Processor -- start 2024-04-23T10:27:24.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.186+0000 7f40efa42740 1 -- start start 2024-04-23T10:27:24.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.187+0000 7f40efa42740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55df3b766490 0x55df3b766870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:24.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.187+0000 7f40efa42740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55df3b766db0 0x55df3b770f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:24.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.187+0000 7f40efa42740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df3b771480 0x55df3b773870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:24.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.187+0000 7f40efa42740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55df3b6cb6d0 con 0x55df3b766db0 2024-04-23T10:27:24.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.187+0000 7f40efa42740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55df3b6681a0 con 0x55df3b766490 2024-04-23T10:27:24.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.187+0000 7f40efa42740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55df3b6c7690 con 0x55df3b771480 2024-04-23T10:27:24.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.187+0000 7f40df7fe640 1 --2- 172.21.15.97:0/4078487122 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f40d40a3ed0 0x7f40d40a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:24.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.187+0000 7f40e659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55df3b766db0 0x55df3b770f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:24.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.187+0000 7f40e6d9d640 1 --2- 172.21.15.97:0/4078487122 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f40d40a3ed0 0x7f40d40a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:24.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.187+0000 7f40e659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55df3b766db0 0x55df3b770f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39208/0 (socket says 172.21.15.97:39208) 2024-04-23T10:27:24.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.187+0000 7f40e659c640 1 -- 172.21.15.97:0/1816179373 learned_addr learned my addr 172.21.15.97:0/1816179373 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:27:24.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.187+0000 7f40e6d9d640 1 --2- 172.21.15.97:0/1816179373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55df3b766490 0x55df3b766870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:24.193 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.187+0000 7f40df7fe640 1 -- 172.21.15.97:0/4078487122 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f40d00d08c0 con 0x7f40e8107260 2024-04-23T10:27:24.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.187+0000 7f40dffff640 1 --2- 172.21.15.97:0/1816179373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df3b771480 0x55df3b773870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:24.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.187+0000 7f40e659c640 1 -- 172.21.15.97:0/1816179373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df3b771480 msgr2=0x55df3b773870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:27:24.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.187+0000 7f40e659c640 1 --2- 172.21.15.97:0/1816179373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df3b771480 0x55df3b773870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:24.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.187+0000 7f40e659c640 1 -- 172.21.15.97:0/1816179373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55df3b766490 msgr2=0x55df3b766870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:27:24.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.187+0000 7f40e659c640 1 --2- 172.21.15.97:0/1816179373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55df3b766490 0x55df3b766870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:24.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.187+0000 7f40e659c640 1 -- 172.21.15.97:0/1816179373 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df3b6725b0 con 0x55df3b766db0 2024-04-23T10:27:24.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.187+0000 7f40e6d9d640 1 --2- 172.21.15.97:0/4078487122 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f40d40a3ed0 0x7f40d40a6390 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f40e0009a60 tx=0x7f40e002f450 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:27:24.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.187+0000 7f40e659c640 1 --2- 172.21.15.97:0/1816179373 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55df3b766db0 0x55df3b770f40 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f40d8009c50 tx=0x7f40d800e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8cc523279236b70 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:27:24.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.188+0000 7f40dcff9640 1 -- 172.21.15.97:0/1816179373 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d8016070 con 0x55df3b766db0 2024-04-23T10:27:24.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.188+0000 7f40dcff9640 1 -- 172.21.15.97:0/1816179373 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40d8011920 con 0x55df3b766db0 2024-04-23T10:27:24.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.188+0000 7f40dcff9640 1 -- 172.21.15.97:0/1816179373 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d8010730 con 0x55df3b766db0 2024-04-23T10:27:24.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.188+0000 7f40efa42740 1 -- 172.21.15.97:0/1816179373 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55df3b766db0 msgr2=0x55df3b770f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:24.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.188+0000 7f40efa42740 1 --2- 172.21.15.97:0/1816179373 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55df3b766db0 0x55df3b770f40 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f40d8009c50 tx=0x7f40d800e6c0 comp rx=0 tx=0).stop 2024-04-23T10:27:24.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.188+0000 7f40efa42740 1 -- 172.21.15.97:0/1816179373 shutdown_connections 2024-04-23T10:27:24.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.188+0000 7f40efa42740 1 --2- 172.21.15.97:0/1816179373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df3b771480 0x55df3b773870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:24.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.188+0000 7f40efa42740 1 --2- 172.21.15.97:0/1816179373 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55df3b766db0 0x55df3b770f40 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:24.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.188+0000 7f40efa42740 1 --2- 172.21.15.97:0/1816179373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55df3b766490 0x55df3b766870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:24.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.188+0000 7f40efa42740 1 -- 172.21.15.97:0/1816179373 >> 172.21.15.97:0/1816179373 conn(0x55df3b7627b0 msgr2=0x55df3b762b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:27:24.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.188+0000 7f40efa42740 1 -- 172.21.15.97:0/1816179373 shutdown_connections 2024-04-23T10:27:24.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.188+0000 7f40efa42740 1 -- 172.21.15.97:0/1816179373 wait complete. 2024-04-23T10:27:24.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.189+0000 7f40efa42740 1 -- start start 2024-04-23T10:27:24.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.189+0000 7f40efa42740 1 Processor -- start 2024-04-23T10:27:24.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.189+0000 7f40efa42740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df3b766490 0x55df3b770e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:24.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.189+0000 7f40efa42740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55df3b766db0 0x55df3b76e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:24.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.189+0000 7f40efa42740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55df3b771480 0x55df3b76e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:24.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.189+0000 7f40efa42740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55df3b774310 con 0x55df3b766db0 2024-04-23T10:27:24.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.189+0000 7f40efa42740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55df3b665b20 con 0x55df3b771480 2024-04-23T10:27:24.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.189+0000 7f40efa42740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55df3b774490 con 0x55df3b766490 2024-04-23T10:27:24.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.189+0000 7f40efa42740 10 client.0 _ll_register_callbacks cb 0x55df3b665530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:27:24.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.189+0000 7f40dffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55df3b766db0 0x55df3b76e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:24.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.189+0000 7f40dffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55df3b766db0 0x55df3b76e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39210/0 (socket says 172.21.15.97:39210) 2024-04-23T10:27:24.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.189+0000 7f40dffff640 1 -- 172.21.15.97:0/2432328390 learned_addr learned my addr 172.21.15.97:0/2432328390 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:27:24.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.189+0000 7f40e6d9d640 1 --2- 172.21.15.97:0/2432328390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55df3b771480 0x55df3b76e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:24.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.189+0000 7f40e659c640 1 --2- 172.21.15.97:0/2432328390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df3b766490 0x55df3b770e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:24.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.189+0000 7f40dffff640 1 -- 172.21.15.97:0/2432328390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df3b766490 msgr2=0x55df3b770e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:27:24.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.189+0000 7f40dffff640 1 --2- 172.21.15.97:0/2432328390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df3b766490 0x55df3b770e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:24.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.189+0000 7f40dffff640 1 -- 172.21.15.97:0/2432328390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55df3b771480 msgr2=0x55df3b76e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:27:24.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.189+0000 7f40dffff640 1 --2- 172.21.15.97:0/2432328390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55df3b771480 0x55df3b76e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:24.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.189+0000 7f40dffff640 1 -- 172.21.15.97:0/2432328390 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df3b3dd920 con 0x55df3b766db0 2024-04-23T10:27:24.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.190+0000 7f40dffff640 1 --2- 172.21.15.97:0/2432328390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55df3b766db0 0x55df3b76e1c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f40d0098f60 tx=0x7f40d009bfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c9cb08b93a31be2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:27:24.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.190+0000 7f40cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:27:24.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.190+0000 7f40cdffb640 1 -- 172.21.15.97:0/2432328390 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d001d040 con 0x55df3b766db0 2024-04-23T10:27:24.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.190+0000 7f40cdffb640 1 -- 172.21.15.97:0/2432328390 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f40d00982e0 con 0x55df3b766db0 2024-04-23T10:27:24.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.190+0000 7f40efa42740 1 -- 172.21.15.97:0/2432328390 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40d8011920 con 0x55df3b766db0 2024-04-23T10:27:24.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.190+0000 7f40cdffb640 1 -- 172.21.15.97:0/2432328390 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f40d00a0940 con 0x55df3b766db0 2024-04-23T10:27:24.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.190+0000 7f40efa42740 10 client.15867 fetch_fsmap learned FSMap version 26 2024-04-23T10:27:24.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.190+0000 7f40efa42740 1 -- 172.21.15.97:0/2432328390 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55df3b4a9d30 con 0x55df3b766db0 2024-04-23T10:27:24.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.190+0000 7f40cdffb640 1 -- 172.21.15.97:0/2432328390 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40d8011920 con 0x55df3b766db0 2024-04-23T10:27:24.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.190+0000 7f40cdffb640 1 -- 172.21.15.97:0/2432328390 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f40d009b570 con 0x55df3b766db0 2024-04-23T10:27:24.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.190+0000 7f40efa42740 10 client.15867 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:27:24.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.190+0000 7f40efa42740 10 client.15867 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:27:24.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.190+0000 7f40efa42740 20 client.15867 populate_metadata read hostname 'smithi097' 2024-04-23T10:27:24.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.190+0000 7f40efa42740 1 --2- 172.21.15.97:0/2432328390 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55df3b781600 0x55df3b783a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:24.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.190+0000 7f40efa42740 4 client.15867 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:27:24.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.190+0000 7f40efa42740 1 -- 172.21.15.97:0/2432328390 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df3b689500 con 0x55df3b781600 2024-04-23T10:27:24.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.190+0000 7f40e659c640 1 --2- 172.21.15.97:0/2432328390 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55df3b781600 0x55df3b783a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:24.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.191+0000 7f40e659c640 1 --2- 172.21.15.97:0/2432328390 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55df3b781600 0x55df3b783a10 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae7593341852e92c server_cookie=8baa881aa2f341f7 in_seq=0 out_seq=0 2024-04-23T10:27:24.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.191+0000 7f40cdffb640 10 client.15867 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:27:24.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.191+0000 7f40cdffb640 1 -- 172.21.15.97:0/2432328390 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55df3b689500 con 0x55df3b781600 2024-04-23T10:27:24.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.191+0000 7f40cdffb640 10 client.15867 handle_command_reply: tid=0 2024-04-23T10:27:24.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.191+0000 7f40efa42740 1 client.15867 shutdown 2024-04-23T10:27:24.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.192+0000 7f40efa42740 1 -- 172.21.15.97:0/2432328390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55df3b766db0 msgr2=0x55df3b76e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:24.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.192+0000 7f40efa42740 1 --2- 172.21.15.97:0/2432328390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55df3b766db0 0x55df3b76e1c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f40d0098f60 tx=0x7f40d009bfb0 comp rx=0 tx=0).stop 2024-04-23T10:27:24.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.192+0000 7f40efa42740 1 -- 172.21.15.97:0/2432328390 shutdown_connections 2024-04-23T10:27:24.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.192+0000 7f40efa42740 1 --2- 172.21.15.97:0/2432328390 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55df3b781600 0x55df3b783a10 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:24.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.192+0000 7f40efa42740 1 --2- 172.21.15.97:0/2432328390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55df3b771480 0x55df3b76e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:24.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.192+0000 7f40efa42740 1 --2- 172.21.15.97:0/2432328390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55df3b766db0 0x55df3b76e1c0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:24.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.192+0000 7f40cdffb640 0 client.15867 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:27:24.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.192+0000 7f40efa42740 1 --2- 172.21.15.97:0/2432328390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df3b766490 0x55df3b770e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:24.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.192+0000 7f40efa42740 1 -- 172.21.15.97:0/2432328390 >> 172.21.15.97:0/2432328390 conn(0x55df3b7627b0 msgr2=0x55df3b7685b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:27:24.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.192+0000 7f40efa42740 1 -- 172.21.15.97:0/2432328390 shutdown_connections 2024-04-23T10:27:24.198 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.192+0000 7f40efa42740 1 -- 172.21.15.97:0/2432328390 wait complete. 2024-04-23T10:27:24.199 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.192+0000 7f40efa42740 20 client.15867 trim_cache size 0 max 16384 2024-04-23T10:27:24.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.208+0000 7f40efa42740 1 Processor -- start 2024-04-23T10:27:24.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.208+0000 7f40efa42740 1 -- start start 2024-04-23T10:27:24.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.209+0000 7f40efa42740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55df3b766490 0x55df3b768c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:24.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.209+0000 7f40efa42740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df3b766db0 0x55df3b77d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:24.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.209+0000 7f40efa42740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55df3b771480 0x55df3b76daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:24.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.209+0000 7f40efa42740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f40e81ada30 con 0x55df3b766490 2024-04-23T10:27:24.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.209+0000 7f40efa42740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f40e81af570 con 0x55df3b771480 2024-04-23T10:27:24.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.209+0000 7f40efa42740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f40e81ae100 con 0x55df3b766db0 2024-04-23T10:27:24.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.209+0000 7f40e659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55df3b766490 0x55df3b768c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:24.214 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.209+0000 7f40e659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55df3b766490 0x55df3b768c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39216/0 (socket says 172.21.15.97:39216) 2024-04-23T10:27:24.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.209+0000 7f40e659c640 1 -- 172.21.15.97:0/1022194927 learned_addr learned my addr 172.21.15.97:0/1022194927 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:27:24.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.209+0000 7f40e659c640 1 -- 172.21.15.97:0/1022194927 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df3b766db0 msgr2=0x55df3b77d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:27:24.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.209+0000 7f40e659c640 1 --2- 172.21.15.97:0/1022194927 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df3b766db0 0x55df3b77d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:24.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.209+0000 7f40e659c640 1 -- 172.21.15.97:0/1022194927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55df3b771480 msgr2=0x55df3b76daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:27:24.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.209+0000 7f40e659c640 1 --2- 172.21.15.97:0/1022194927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55df3b771480 0x55df3b76daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:24.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.209+0000 7f40e659c640 1 -- 172.21.15.97:0/1022194927 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df3b76f110 con 0x55df3b766490 2024-04-23T10:27:24.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.209+0000 7f40e659c640 1 --2- 172.21.15.97:0/1022194927 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55df3b766490 0x55df3b768c30 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f40d800ebc0 tx=0x7f40d8009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64bbc3a53365ac44 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:27:24.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.209+0000 7f40cffff640 1 -- 172.21.15.97:0/1022194927 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d802e070 con 0x55df3b766490 2024-04-23T10:27:24.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.209+0000 7f40cffff640 1 -- 172.21.15.97:0/1022194927 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40d8005040 con 0x55df3b766490 2024-04-23T10:27:24.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.209+0000 7f40cffff640 1 -- 172.21.15.97:0/1022194927 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d8016040 con 0x55df3b766490 2024-04-23T10:27:24.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.210+0000 7f40efa42740 1 -- 172.21.15.97:0/1022194927 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55df3b766490 msgr2=0x55df3b768c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:24.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.210+0000 7f40efa42740 1 --2- 172.21.15.97:0/1022194927 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55df3b766490 0x55df3b768c30 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f40d800ebc0 tx=0x7f40d8009ed0 comp rx=0 tx=0).stop 2024-04-23T10:27:24.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.210+0000 7f40efa42740 1 -- 172.21.15.97:0/1022194927 shutdown_connections 2024-04-23T10:27:24.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.210+0000 7f40efa42740 1 --2- 172.21.15.97:0/1022194927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55df3b771480 0x55df3b76daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:24.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.210+0000 7f40efa42740 1 --2- 172.21.15.97:0/1022194927 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df3b766db0 0x55df3b77d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:24.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.210+0000 7f40efa42740 1 --2- 172.21.15.97:0/1022194927 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55df3b766490 0x55df3b768c30 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:24.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.210+0000 7f40efa42740 1 -- 172.21.15.97:0/1022194927 >> 172.21.15.97:0/1022194927 conn(0x55df3b7627b0 msgr2=0x55df3b762b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:27:24.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.210+0000 7f40efa42740 1 -- 172.21.15.97:0/1022194927 shutdown_connections 2024-04-23T10:27:24.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.210+0000 7f40efa42740 1 -- 172.21.15.97:0/1022194927 wait complete. 2024-04-23T10:27:24.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.210+0000 7f40efa42740 1 -- start start 2024-04-23T10:27:24.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.210+0000 7f40efa42740 1 Processor -- start 2024-04-23T10:27:24.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.210+0000 7f40efa42740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55df3b766490 0x55df3b779bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:24.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.210+0000 7f40efa42740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55df3b766db0 0x55df3b76b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:24.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.210+0000 7f40efa42740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df3b771480 0x55df3b76bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:24.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.210+0000 7f40efa42740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f40e81aeea0 con 0x55df3b766db0 2024-04-23T10:27:24.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.210+0000 7f40efa42740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f40e81ae7d0 con 0x55df3b766490 2024-04-23T10:27:24.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.210+0000 7f40efa42740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f40e81aba10 con 0x55df3b771480 2024-04-23T10:27:24.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.210+0000 7f40efa42740 10 client.0 _ll_register_callbacks cb 0x7f40d009c0d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:27:24.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.210+0000 7f40dffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55df3b766db0 0x55df3b76b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:24.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.210+0000 7f40dffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55df3b766db0 0x55df3b76b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39218/0 (socket says 172.21.15.97:39218) 2024-04-23T10:27:24.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.210+0000 7f40dffff640 1 -- 172.21.15.97:0/1874165284 learned_addr learned my addr 172.21.15.97:0/1874165284 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:27:24.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.211+0000 7f40e6d9d640 1 --2- 172.21.15.97:0/1874165284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df3b771480 0x55df3b76bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:24.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.211+0000 7f40e659c640 1 --2- 172.21.15.97:0/1874165284 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55df3b766490 0x55df3b779bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:24.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.211+0000 7f40dffff640 1 -- 172.21.15.97:0/1874165284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df3b771480 msgr2=0x55df3b76bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:27:24.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.211+0000 7f40dffff640 1 --2- 172.21.15.97:0/1874165284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df3b771480 0x55df3b76bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:24.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.211+0000 7f40dffff640 1 -- 172.21.15.97:0/1874165284 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55df3b766490 msgr2=0x55df3b779bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:27:24.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.211+0000 7f40dffff640 1 --2- 172.21.15.97:0/1874165284 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55df3b766490 0x55df3b779bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:24.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.211+0000 7f40dffff640 1 -- 172.21.15.97:0/1874165284 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df3b7720a0 con 0x55df3b766db0 2024-04-23T10:27:24.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.211+0000 7f40e659c640 1 --2- 172.21.15.97:0/1874165284 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55df3b766490 0x55df3b779bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:27:24.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.211+0000 7f40dffff640 1 --2- 172.21.15.97:0/1874165284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55df3b766db0 0x55df3b76b4b0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f40d009c290 tx=0x7f40d009c180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93d13cf6ccb06791 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:27:24.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.211+0000 7f40cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:27:24.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.211+0000 7f40cdffb640 1 -- 172.21.15.97:0/1874165284 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d001d070 con 0x55df3b766db0 2024-04-23T10:27:24.217 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.211+0000 7f40cdffb640 1 -- 172.21.15.97:0/1874165284 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f40d0085ad0 con 0x55df3b766db0 2024-04-23T10:27:24.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.211+0000 7f40efa42740 1 -- 172.21.15.97:0/1874165284 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40d8005040 con 0x55df3b766db0 2024-04-23T10:27:24.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.211+0000 7f40cdffb640 1 -- 172.21.15.97:0/1874165284 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f40d00a5200 con 0x55df3b766db0 2024-04-23T10:27:24.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.211+0000 7f40efa42740 10 client.15873 fetch_fsmap learned FSMap version 26 2024-04-23T10:27:24.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.211+0000 7f40efa42740 1 -- 172.21.15.97:0/1874165284 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55df3b7e7d60 con 0x55df3b766db0 2024-04-23T10:27:24.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.211+0000 7f40cdffb640 1 -- 172.21.15.97:0/1874165284 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40d8005040 con 0x55df3b766db0 2024-04-23T10:27:24.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.211+0000 7f40cdffb640 1 -- 172.21.15.97:0/1874165284 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f40d00bb170 con 0x55df3b766db0 2024-04-23T10:27:24.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.211+0000 7f40efa42740 10 client.15873 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:27:24.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.211+0000 7f40efa42740 10 client.15873 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:27:24.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.211+0000 7f40efa42740 20 client.15873 populate_metadata read hostname 'smithi097' 2024-04-23T10:27:24.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.212+0000 7f40efa42740 1 --2- 172.21.15.97:0/1874165284 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55df3b7e83c0 0x55df3b7ea7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:24.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.212+0000 7f40efa42740 4 client.15873 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "c703558e", "release": true, "await": true}] 2024-04-23T10:27:24.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.212+0000 7f40efa42740 1 -- 172.21.15.97:0/1874165284 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "quiesce db", "set_id": "c703558e", "release": true, "await": true}) -- 0x55df3b6d90f0 con 0x55df3b7e83c0 2024-04-23T10:27:24.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.212+0000 7f40e659c640 1 --2- 172.21.15.97:0/1874165284 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55df3b7e83c0 0x55df3b7ea7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:24.218 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.212+0000 7f40e659c640 1 --2- 172.21.15.97:0/1874165284 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55df3b7e83c0 0x55df3b7ea7b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c377422c0758fdb5 server_cookie=adfbd9e8ab51b65c in_seq=0 out_seq=0 2024-04-23T10:27:24.219 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.212+0000 7f40cdffb640 10 client.15873 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:27:24.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.216+0000 7f40cdffb640 1 -- 172.21.15.97:0/1874165284 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x55df3b6d90f0 con 0x55df3b7e83c0 2024-04-23T10:27:24.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.216+0000 7f40cdffb640 10 client.15873 handle_command_reply: tid=0 2024-04-23T10:27:24.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.216+0000 7f40efa42740 1 client.15873 shutdown 2024-04-23T10:27:24.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.216+0000 7f40efa42740 1 -- 172.21.15.97:0/1874165284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55df3b766db0 msgr2=0x55df3b76b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:24.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.216+0000 7f40efa42740 1 --2- 172.21.15.97:0/1874165284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55df3b766db0 0x55df3b76b4b0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f40d009c290 tx=0x7f40d009c180 comp rx=0 tx=0).stop 2024-04-23T10:27:24.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.216+0000 7f40efa42740 1 -- 172.21.15.97:0/1874165284 shutdown_connections 2024-04-23T10:27:24.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.216+0000 7f40efa42740 1 --2- 172.21.15.97:0/1874165284 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55df3b7e83c0 0x55df3b7ea7b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:24.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.216+0000 7f40efa42740 1 --2- 172.21.15.97:0/1874165284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55df3b771480 0x55df3b76bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:24.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.216+0000 7f40efa42740 1 --2- 172.21.15.97:0/1874165284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55df3b766db0 0x55df3b76b4b0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:24.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.216+0000 7f40efa42740 1 --2- 172.21.15.97:0/1874165284 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55df3b766490 0x55df3b779bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:24.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.216+0000 7f40cdffb640 0 client.15873 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:27:24.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.216+0000 7f40efa42740 1 -- 172.21.15.97:0/1874165284 >> 172.21.15.97:0/1874165284 conn(0x55df3b7627b0 msgr2=0x55df3b76f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:27:24.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.217+0000 7f40efa42740 1 -- 172.21.15.97:0/1874165284 shutdown_connections 2024-04-23T10:27:24.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.217+0000 7f40efa42740 1 -- 172.21.15.97:0/1874165284 wait complete. 2024-04-23T10:27:24.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.217+0000 7f40efa42740 20 client.15873 trim_cache size 0 max 16384 2024-04-23T10:27:24.221 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-23T10:27:24.221 INFO:teuthology.orchestra.run.smithi097.stdout: "epoch": 26, 2024-04-23T10:27:24.221 INFO:teuthology.orchestra.run.smithi097.stdout: "leader": 24488, 2024-04-23T10:27:24.221 INFO:teuthology.orchestra.run.smithi097.stdout: "set_version": 8, 2024-04-23T10:27:24.221 INFO:teuthology.orchestra.run.smithi097.stdout: "sets": { 2024-04-23T10:27:24.221 INFO:teuthology.orchestra.run.smithi097.stdout: "c703558e": { 2024-04-23T10:27:24.221 INFO:teuthology.orchestra.run.smithi097.stdout: "version": 8, 2024-04-23T10:27:24.221 INFO:teuthology.orchestra.run.smithi097.stdout: "age_ref": 0.0, 2024-04-23T10:27:24.222 INFO:teuthology.orchestra.run.smithi097.stdout: "state": { 2024-04-23T10:27:24.222 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RELEASED", 2024-04-23T10:27:24.222 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0 2024-04-23T10:27:24.222 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:27:24.222 INFO:teuthology.orchestra.run.smithi097.stdout: "timeout": 60.0, 2024-04-23T10:27:24.222 INFO:teuthology.orchestra.run.smithi097.stdout: "expiration": 82.9, 2024-04-23T10:27:24.222 INFO:teuthology.orchestra.run.smithi097.stdout: "members": { 2024-04-23T10:27:24.222 INFO:teuthology.orchestra.run.smithi097.stdout: "file:/": { 2024-04-23T10:27:24.222 INFO:teuthology.orchestra.run.smithi097.stdout: "excluded": false, 2024-04-23T10:27:24.222 INFO:teuthology.orchestra.run.smithi097.stdout: "state": { 2024-04-23T10:27:24.222 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RELEASED", 2024-04-23T10:27:24.222 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0 2024-04-23T10:27:24.222 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:27:24.222 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:27:24.223 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:27:24.223 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:27:24.223 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:27:24.223 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-23T10:27:24.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.217+0000 7f40ecd29640 1 -- 172.21.15.97:0/4078487122 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f40d40a3ed0 msgr2=0x7f40d40a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:24.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.217+0000 7f40ecd29640 1 --2- 172.21.15.97:0/4078487122 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f40d40a3ed0 0x7f40d40a6390 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f40e0009a60 tx=0x7f40e002f450 comp rx=0 tx=0).stop 2024-04-23T10:27:24.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.217+0000 7f40ecd29640 1 -- 172.21.15.97:0/4078487122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40e8107260 msgr2=0x7f40e8198cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:24.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.217+0000 7f40ecd29640 1 --2- 172.21.15.97:0/4078487122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40e8107260 0x7f40e8198cb0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f40d000c970 tx=0x7f40d000ce30 comp rx=0 tx=0).stop 2024-04-23T10:27:24.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.217+0000 7f40ecd29640 1 -- 172.21.15.97:0/4078487122 shutdown_connections 2024-04-23T10:27:24.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.217+0000 7f40ecd29640 1 --2- 172.21.15.97:0/4078487122 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f40d40a3ed0 0x7f40d40a6390 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:24.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.217+0000 7f40ecd29640 1 --2- 172.21.15.97:0/4078487122 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40e819cfe0 0x7f40e819d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:24.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.217+0000 7f40ecd29640 1 --2- 172.21.15.97:0/4078487122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40e8107260 0x7f40e8198cb0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:24.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.217+0000 7f40ecd29640 1 --2- 172.21.15.97:0/4078487122 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40e8102190 0x7f40e8198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:24.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.217+0000 7f40ecd29640 1 -- 172.21.15.97:0/4078487122 >> 172.21.15.97:0/4078487122 conn(0x7f40e80793a0 msgr2=0x7f40e80fff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:27:24.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.217+0000 7f40ecd29640 1 -- 172.21.15.97:0/4078487122 shutdown_connections 2024-04-23T10:27:24.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:27:24.217+0000 7f40ecd29640 1 -- 172.21.15.97:0/4078487122 wait complete. 2024-04-23T10:27:24.225 INFO:tasks.workunit.client.0.smithi097.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh 2024-04-23T10:27:24.225 INFO:tasks.workunit.client.0.smithi097.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs 2024-04-23T10:27:24.225 INFO:tasks.workunit.client.0.smithi097.stderr:+ /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc trunc.foo 600 2024-04-23T10:27:24.227 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'c703558e', seconds elapsed: 24.2 2024-04-23T10:27:25.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:27:25 smithi196 ceph-mon[32919]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 853 B/s wr, 13 op/s 2024-04-23T10:27:25.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:27:25 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/3574493696' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:27:25.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:27:25 smithi097 ceph-mon[29197]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 853 B/s wr, 13 op/s 2024-04-23T10:27:25.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:27:25 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/3574493696' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:27:25.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:27:25 smithi183 ceph-mon[32317]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 853 B/s wr, 13 op/s 2024-04-23T10:27:25.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:27:25 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/3574493696' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:27:27.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:27:27 smithi196 ceph-mon[32919]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-23T10:27:27.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:27:27 smithi097 ceph-mon[29197]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-23T10:27:27.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:27:27 smithi183 ceph-mon[32317]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-23T10:27:29.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:27:29 smithi196 ceph-mon[32919]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 4.2 KiB/s wr, 14 op/s 2024-04-23T10:27:29.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:27:29 smithi097 ceph-mon[29197]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 4.2 KiB/s wr, 14 op/s 2024-04-23T10:27:29.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:27:29 smithi183 ceph-mon[32317]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 4.2 KiB/s wr, 14 op/s 2024-04-23T10:27:31.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:27:31 smithi196 ceph-mon[32919]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-04-23T10:27:31.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:27:31 smithi097 ceph-mon[29197]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-04-23T10:27:31.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:27:31 smithi183 ceph-mon[32317]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-04-23T10:27:32.657 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.656+0000 7f39e2187640 1 -- 172.21.15.97:0/1300868308 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39dc103b20 msgr2=0x7f39dc10c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:32.657 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.656+0000 7f39e2187640 1 --2- 172.21.15.97:0/1300868308 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39dc103b20 0x7f39dc10c000 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f39c4009900 tx=0x7f39c402a480 comp rx=0 tx=0).stop 2024-04-23T10:27:32.657 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.656+0000 7f39e2187640 1 -- 172.21.15.97:0/1300868308 shutdown_connections 2024-04-23T10:27:32.657 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.656+0000 7f39e2187640 1 --2- 172.21.15.97:0/1300868308 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f39dc10c540 0x7f39dc10e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:32.657 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.656+0000 7f39e2187640 1 --2- 172.21.15.97:0/1300868308 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39dc103b20 0x7f39dc10c000 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:32.657 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.656+0000 7f39e2187640 1 --2- 172.21.15.97:0/1300868308 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39dc103200 0x7f39dc1035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:32.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.656+0000 7f39e2187640 1 -- 172.21.15.97:0/1300868308 >> 172.21.15.97:0/1300868308 conn(0x7f39dc077fd0 msgr2=0x7f39dc0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:27:32.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.656+0000 7f39e2187640 1 -- 172.21.15.97:0/1300868308 shutdown_connections 2024-04-23T10:27:32.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.656+0000 7f39e2187640 1 -- 172.21.15.97:0/1300868308 wait complete. 2024-04-23T10:27:32.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.657+0000 7f39e2187640 1 Processor -- start 2024-04-23T10:27:32.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.657+0000 7f39e2187640 1 -- start start 2024-04-23T10:27:32.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.657+0000 7f39e2187640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f39dc103200 0x7f39dc1986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:32.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.657+0000 7f39e2187640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39dc103b20 0x7f39dc198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:32.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.657+0000 7f39e2187640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39dc10c540 0x7f39dc19cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:32.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.657+0000 7f39e2187640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f39dc1121a0 con 0x7f39dc103b20 2024-04-23T10:27:32.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.657+0000 7f39e2187640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f39dc112020 con 0x7f39dc103200 2024-04-23T10:27:32.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.657+0000 7f39e2187640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f39dc112320 con 0x7f39dc10c540 2024-04-23T10:27:32.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.657+0000 7f39d3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39dc103b20 0x7f39dc198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:32.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.657+0000 7f39d3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39dc103b20 0x7f39dc198be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50466/0 (socket says 172.21.15.97:50466) 2024-04-23T10:27:32.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.657+0000 7f39d3fff640 1 -- 172.21.15.97:0/2204105687 learned_addr learned my addr 172.21.15.97:0/2204105687 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:27:32.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.657+0000 7f39db7fe640 1 --2- 172.21.15.97:0/2204105687 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f39dc103200 0x7f39dc1986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:32.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.657+0000 7f39d3fff640 1 -- 172.21.15.97:0/2204105687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39dc10c540 msgr2=0x7f39dc19cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:32.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.657+0000 7f39dbfff640 1 --2- 172.21.15.97:0/2204105687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39dc10c540 0x7f39dc19cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:32.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.657+0000 7f39d3fff640 1 --2- 172.21.15.97:0/2204105687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39dc10c540 0x7f39dc19cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:32.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.657+0000 7f39d3fff640 1 -- 172.21.15.97:0/2204105687 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f39dc103200 msgr2=0x7f39dc1986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:32.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.657+0000 7f39d3fff640 1 --2- 172.21.15.97:0/2204105687 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f39dc103200 0x7f39dc1986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:32.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.657+0000 7f39d3fff640 1 -- 172.21.15.97:0/2204105687 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39dc19d690 con 0x7f39dc103b20 2024-04-23T10:27:32.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.657+0000 7f39d3fff640 1 --2- 172.21.15.97:0/2204105687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39dc103b20 0x7f39dc198be0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f39c4005940 tx=0x7f39c4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:27:32.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.658+0000 7f39d97fa640 1 -- 172.21.15.97:0/2204105687 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c403f030 con 0x7f39dc103b20 2024-04-23T10:27:32.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.658+0000 7f39d97fa640 1 -- 172.21.15.97:0/2204105687 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39c4003aa0 con 0x7f39dc103b20 2024-04-23T10:27:32.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.658+0000 7f39d97fa640 1 -- 172.21.15.97:0/2204105687 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c4035070 con 0x7f39dc103b20 2024-04-23T10:27:32.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.658+0000 7f39e2187640 1 -- 172.21.15.97:0/2204105687 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39dc19d920 con 0x7f39dc103b20 2024-04-23T10:27:32.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.658+0000 7f39e2187640 1 -- 172.21.15.97:0/2204105687 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39dc1a5210 con 0x7f39dc103b20 2024-04-23T10:27:32.665 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.660+0000 7f39d97fa640 1 -- 172.21.15.97:0/2204105687 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f39c4042030 con 0x7f39dc103b20 2024-04-23T10:27:32.665 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.660+0000 7f39e4f7b740 1 Processor -- start 2024-04-23T10:27:32.665 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.660+0000 7f39e4f7b740 1 -- start start 2024-04-23T10:27:32.665 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.660+0000 7f39e4f7b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564e2befd520 0x564e2befd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:32.666 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.660+0000 7f39e4f7b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564e2befde40 0x564e2bf0c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:32.666 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.660+0000 7f39e4f7b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e2bf0cc10 0x564e2bf0f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:32.666 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.660+0000 7f39e4f7b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x564e2be62970 con 0x564e2befde40 2024-04-23T10:27:32.666 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.660+0000 7f39e4f7b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564e2bdff4a0 con 0x564e2bf0cc10 2024-04-23T10:27:32.666 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.660+0000 7f39e4f7b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564e2be5e930 con 0x564e2befd520 2024-04-23T10:27:32.666 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.661+0000 7f39db7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564e2befde40 0x564e2bf0c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:32.666 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.661+0000 7f39db7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564e2befde40 0x564e2bf0c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50478/0 (socket says 172.21.15.97:50478) 2024-04-23T10:27:32.666 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.661+0000 7f39db7fe640 1 -- 172.21.15.97:0/1794017224 learned_addr learned my addr 172.21.15.97:0/1794017224 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:27:32.666 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.661+0000 7f39d97fa640 1 --2- 172.21.15.97:0/2204105687 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f39c80a3ff0 0x7f39c80a64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:32.666 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.661+0000 7f39d3fff640 1 --2- 172.21.15.97:0/1794017224 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e2bf0cc10 0x564e2bf0f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:32.666 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.661+0000 7f39dbfff640 1 --2- 172.21.15.97:0/1794017224 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564e2befd520 0x564e2befd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:32.666 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.661+0000 7f39dbfff640 1 --2- 172.21.15.97:0/2204105687 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f39c80a3ff0 0x7f39c80a64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:32.666 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.661+0000 7f39db7fe640 1 -- 172.21.15.97:0/1794017224 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564e2befd520 msgr2=0x564e2befd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:27:32.666 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.661+0000 7f39db7fe640 1 --2- 172.21.15.97:0/1794017224 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564e2befd520 0x564e2befd900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:32.666 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.661+0000 7f39db7fe640 1 -- 172.21.15.97:0/1794017224 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e2bf0cc10 msgr2=0x564e2bf0f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:27:32.667 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.661+0000 7f39db7fe640 1 --2- 172.21.15.97:0/1794017224 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e2bf0cc10 0x564e2bf0f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:32.667 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.661+0000 7f39db7fe640 1 -- 172.21.15.97:0/1794017224 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e2bb75210 con 0x564e2befde40 2024-04-23T10:27:32.667 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.661+0000 7f39d97fa640 1 -- 172.21.15.97:0/2204105687 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f39c40f1070 con 0x7f39dc103b20 2024-04-23T10:27:32.667 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.661+0000 7f39db7fe640 1 --2- 172.21.15.97:0/1794017224 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564e2befde40 0x564e2bf0c6d0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f39cc00b760 tx=0x7f39cc00bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4abc4fa9773416c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:27:32.667 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.661+0000 7f39dbfff640 1 --2- 172.21.15.97:0/2204105687 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f39c80a3ff0 0x7f39c80a64b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f39d4005fd0 tx=0x7f39d4009500 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:27:32.667 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.661+0000 7f39d1ffb640 1 -- 172.21.15.97:0/1794017224 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39cc003dd0 con 0x564e2befde40 2024-04-23T10:27:32.667 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.661+0000 7f39d1ffb640 1 -- 172.21.15.97:0/1794017224 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39cc015070 con 0x564e2befde40 2024-04-23T10:27:32.667 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.661+0000 7f39d1ffb640 1 -- 172.21.15.97:0/1794017224 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39cc004da0 con 0x564e2befde40 2024-04-23T10:27:32.667 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.662+0000 7f39e4f7b740 1 -- 172.21.15.97:0/1794017224 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564e2befde40 msgr2=0x564e2bf0c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:32.667 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.662+0000 7f39e4f7b740 1 --2- 172.21.15.97:0/1794017224 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564e2befde40 0x564e2bf0c6d0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f39cc00b760 tx=0x7f39cc00bc20 comp rx=0 tx=0).stop 2024-04-23T10:27:32.667 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.662+0000 7f39e4f7b740 1 -- 172.21.15.97:0/1794017224 shutdown_connections 2024-04-23T10:27:32.667 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.662+0000 7f39e4f7b740 1 --2- 172.21.15.97:0/1794017224 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e2bf0cc10 0x564e2bf0f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:32.667 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.662+0000 7f39e4f7b740 1 --2- 172.21.15.97:0/1794017224 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564e2befde40 0x564e2bf0c6d0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:32.667 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.662+0000 7f39e4f7b740 1 --2- 172.21.15.97:0/1794017224 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564e2befd520 0x564e2befd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:32.667 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.662+0000 7f39e4f7b740 1 -- 172.21.15.97:0/1794017224 >> 172.21.15.97:0/1794017224 conn(0x564e2bef9b30 msgr2=0x564e2bef9f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:27:32.667 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.662+0000 7f39e4f7b740 1 -- 172.21.15.97:0/1794017224 shutdown_connections 2024-04-23T10:27:32.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.662+0000 7f39e4f7b740 1 -- 172.21.15.97:0/1794017224 wait complete. 2024-04-23T10:27:32.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.662+0000 7f39e4f7b740 1 -- start start 2024-04-23T10:27:32.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.663+0000 7f39e4f7b740 1 Processor -- start 2024-04-23T10:27:32.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.663+0000 7f39e4f7b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564e2befd520 0x564e2bf0c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:32.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.663+0000 7f39e4f7b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564e2befde40 0x564e2bf097f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:32.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.663+0000 7f39e4f7b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e2bf0cc10 0x564e2bf09d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:32.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.663+0000 7f39e4f7b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x564e2bf0f840 con 0x564e2befde40 2024-04-23T10:27:32.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.663+0000 7f39e4f7b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564e2bdfd0b0 con 0x564e2bf0cc10 2024-04-23T10:27:32.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.663+0000 7f39e4f7b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564e2bf0f9c0 con 0x564e2befd520 2024-04-23T10:27:32.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.663+0000 7f39e4f7b740 10 client.0 _ll_register_callbacks cb 0x564e2bdead30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:27:32.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.663+0000 7f39d3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564e2befde40 0x564e2bf097f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:32.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.663+0000 7f39d3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564e2befde40 0x564e2bf097f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50482/0 (socket says 172.21.15.97:50482) 2024-04-23T10:27:32.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.663+0000 7f39d3fff640 1 -- 172.21.15.97:0/2737444315 learned_addr learned my addr 172.21.15.97:0/2737444315 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:27:32.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.663+0000 7f39db7fe640 1 --2- 172.21.15.97:0/2737444315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564e2befd520 0x564e2bf0c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:32.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.663+0000 7f39d3fff640 1 -- 172.21.15.97:0/2737444315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564e2befd520 msgr2=0x564e2bf0c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:27:32.669 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.663+0000 7f39d3fff640 1 --2- 172.21.15.97:0/2737444315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564e2befd520 0x564e2bf0c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:32.669 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.663+0000 7f39d3fff640 1 -- 172.21.15.97:0/2737444315 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e2bf0cc10 msgr2=0x564e2bf09d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:27:32.669 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.663+0000 7f39dbfff640 1 --2- 172.21.15.97:0/2737444315 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e2bf0cc10 0x564e2bf09d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:32.669 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.663+0000 7f39d3fff640 1 --2- 172.21.15.97:0/2737444315 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e2bf0cc10 0x564e2bf09d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:32.669 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.663+0000 7f39d3fff640 1 -- 172.21.15.97:0/2737444315 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e2bc41620 con 0x564e2befde40 2024-04-23T10:27:32.669 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.664+0000 7f39d3fff640 1 --2- 172.21.15.97:0/2737444315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564e2befde40 0x564e2bf097f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f39c40bc9b0 tx=0x7f39c403b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bc39e6c009f6a20 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:27:32.669 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.664+0000 7f39baffd640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:27:32.669 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.664+0000 7f39baffd640 1 -- 172.21.15.97:0/2737444315 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c4049040 con 0x564e2befde40 2024-04-23T10:27:32.669 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.664+0000 7f39baffd640 1 -- 172.21.15.97:0/2737444315 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f39c40dd0d0 con 0x564e2befde40 2024-04-23T10:27:32.669 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.664+0000 7f39e4f7b740 1 -- 172.21.15.97:0/2737444315 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39cc015070 con 0x564e2befde40 2024-04-23T10:27:32.669 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.664+0000 7f39baffd640 1 -- 172.21.15.97:0/2737444315 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f39c40bd3b0 con 0x564e2befde40 2024-04-23T10:27:32.669 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.664+0000 7f39e4f7b740 10 client.15885 fetch_fsmap learned FSMap version 26 2024-04-23T10:27:32.670 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.664+0000 7f39e4f7b740 1 -- 172.21.15.97:0/2737444315 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564e2bf1b790 con 0x564e2befde40 2024-04-23T10:27:32.670 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.664+0000 7f39baffd640 1 -- 172.21.15.97:0/2737444315 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39cc015070 con 0x564e2befde40 2024-04-23T10:27:32.670 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.664+0000 7f39baffd640 1 -- 172.21.15.97:0/2737444315 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f39c40d6080 con 0x564e2befde40 2024-04-23T10:27:32.670 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.664+0000 7f39e4f7b740 10 client.15885 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:27:32.670 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.664+0000 7f39e4f7b740 10 client.15885 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:27:32.670 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.664+0000 7f39e4f7b740 20 client.15885 populate_metadata read hostname 'smithi097' 2024-04-23T10:27:32.670 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.664+0000 7f39e4f7b740 1 --2- 172.21.15.97:0/2737444315 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x564e2bf1cbf0 0x564e2bf1f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:32.670 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.664+0000 7f39e4f7b740 4 client.15885 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:27:32.670 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.664+0000 7f39e4f7b740 1 -- 172.21.15.97:0/2737444315 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e2be70390 con 0x564e2bf1cbf0 2024-04-23T10:27:32.670 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.665+0000 7f39db7fe640 1 --2- 172.21.15.97:0/2737444315 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x564e2bf1cbf0 0x564e2bf1f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:32.670 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.665+0000 7f39baffd640 10 client.15885 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:27:32.670 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.665+0000 7f39db7fe640 1 --2- 172.21.15.97:0/2737444315 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x564e2bf1cbf0 0x564e2bf1f040 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b37bc4163745a31 server_cookie=ac6cabe8736468c3 in_seq=0 out_seq=0 2024-04-23T10:27:32.670 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.666+0000 7f39baffd640 1 -- 172.21.15.97:0/2737444315 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564e2be70390 con 0x564e2bf1cbf0 2024-04-23T10:27:32.670 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.666+0000 7f39baffd640 10 client.15885 handle_command_reply: tid=0 2024-04-23T10:27:32.670 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.666+0000 7f39e4f7b740 1 client.15885 shutdown 2024-04-23T10:27:32.670 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.666+0000 7f39e4f7b740 1 -- 172.21.15.97:0/2737444315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564e2befde40 msgr2=0x564e2bf097f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:32.671 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.666+0000 7f39e4f7b740 1 --2- 172.21.15.97:0/2737444315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564e2befde40 0x564e2bf097f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f39c40bc9b0 tx=0x7f39c403b460 comp rx=0 tx=0).stop 2024-04-23T10:27:32.671 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.666+0000 7f39e4f7b740 1 -- 172.21.15.97:0/2737444315 shutdown_connections 2024-04-23T10:27:32.671 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.666+0000 7f39e4f7b740 1 --2- 172.21.15.97:0/2737444315 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x564e2bf1cbf0 0x564e2bf1f040 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:32.671 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.666+0000 7f39e4f7b740 1 --2- 172.21.15.97:0/2737444315 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e2bf0cc10 0x564e2bf09d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:32.671 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.666+0000 7f39e4f7b740 1 --2- 172.21.15.97:0/2737444315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564e2befde40 0x564e2bf097f0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:32.671 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.666+0000 7f39baffd640 0 client.15885 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:27:32.671 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.666+0000 7f39e4f7b740 1 --2- 172.21.15.97:0/2737444315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564e2befd520 0x564e2bf0c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:32.671 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.666+0000 7f39e4f7b740 1 -- 172.21.15.97:0/2737444315 >> 172.21.15.97:0/2737444315 conn(0x564e2bef9b30 msgr2=0x564e2beff9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:27:32.671 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.666+0000 7f39e4f7b740 1 -- 172.21.15.97:0/2737444315 shutdown_connections 2024-04-23T10:27:32.671 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.666+0000 7f39e4f7b740 1 -- 172.21.15.97:0/2737444315 wait complete. 2024-04-23T10:27:32.671 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.666+0000 7f39e4f7b740 20 client.15885 trim_cache size 0 max 16384 2024-04-23T10:27:32.681 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.681+0000 7f39e4f7b740 1 Processor -- start 2024-04-23T10:27:32.681 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.681+0000 7f39e4f7b740 1 -- start start 2024-04-23T10:27:32.681 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.681+0000 7f39e4f7b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e2bef9b30 0x564e2bf18700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:32.681 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.681+0000 7f39e4f7b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564e2befd520 0x564e2bf04d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:32.681 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.681+0000 7f39e4f7b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564e2befde40 0x564e2bf0c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:32.681 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.681+0000 7f39e4f7b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f39dc1ab7f0 con 0x564e2befde40 2024-04-23T10:27:32.681 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.681+0000 7f39e4f7b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f39dc1ad330 con 0x564e2bef9b30 2024-04-23T10:27:32.681 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.681+0000 7f39e4f7b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f39dc1abec0 con 0x564e2befd520 2024-04-23T10:27:32.681 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.681+0000 7f39dbfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564e2befde40 0x564e2bf0c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:32.681 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.681+0000 7f39dbfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564e2befde40 0x564e2bf0c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50486/0 (socket says 172.21.15.97:50486) 2024-04-23T10:27:32.682 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.682+0000 7f39dbfff640 1 -- 172.21.15.97:0/199512176 learned_addr learned my addr 172.21.15.97:0/199512176 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:27:32.682 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.682+0000 7f39db7fe640 1 --2- 172.21.15.97:0/199512176 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e2bef9b30 0x564e2bf18700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:32.682 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.682+0000 7f39dbfff640 1 -- 172.21.15.97:0/199512176 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564e2befd520 msgr2=0x564e2bf04d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:27:32.682 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.682+0000 7f39dbfff640 1 --2- 172.21.15.97:0/199512176 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564e2befd520 0x564e2bf04d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:32.682 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.682+0000 7f39dbfff640 1 -- 172.21.15.97:0/199512176 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e2bef9b30 msgr2=0x564e2bf18700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:27:32.682 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.682+0000 7f39dbfff640 1 --2- 172.21.15.97:0/199512176 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e2bef9b30 0x564e2bf18700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:32.682 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.682+0000 7f39dbfff640 1 -- 172.21.15.97:0/199512176 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e2befe380 con 0x564e2befde40 2024-04-23T10:27:32.682 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.682+0000 7f39db7fe640 1 --2- 172.21.15.97:0/199512176 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e2bef9b30 0x564e2bf18700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:27:32.682 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.682+0000 7f39dbfff640 1 --2- 172.21.15.97:0/199512176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564e2befde40 0x564e2bf0c030 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f39d4009d00 tx=0x7f39d40098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd0d920817aa65aa server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:27:32.682 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.682+0000 7f39d17fa640 1 -- 172.21.15.97:0/199512176 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d4026070 con 0x564e2befde40 2024-04-23T10:27:32.682 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.682+0000 7f39d17fa640 1 -- 172.21.15.97:0/199512176 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39d40119d0 con 0x564e2befde40 2024-04-23T10:27:32.682 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.682+0000 7f39d17fa640 1 -- 172.21.15.97:0/199512176 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d4004630 con 0x564e2befde40 2024-04-23T10:27:32.685 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.683+0000 7f39e4f7b740 1 -- 172.21.15.97:0/199512176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564e2befde40 msgr2=0x564e2bf0c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:32.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.683+0000 7f39e4f7b740 1 --2- 172.21.15.97:0/199512176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564e2befde40 0x564e2bf0c030 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f39d4009d00 tx=0x7f39d40098e0 comp rx=0 tx=0).stop 2024-04-23T10:27:32.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.683+0000 7f39e4f7b740 1 -- 172.21.15.97:0/199512176 shutdown_connections 2024-04-23T10:27:32.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.683+0000 7f39e4f7b740 1 --2- 172.21.15.97:0/199512176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564e2befde40 0x564e2bf0c030 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:32.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.683+0000 7f39e4f7b740 1 --2- 172.21.15.97:0/199512176 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564e2befd520 0x564e2bf04d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:32.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.683+0000 7f39e4f7b740 1 --2- 172.21.15.97:0/199512176 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e2bef9b30 0x564e2bf18700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:32.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.683+0000 7f39e4f7b740 1 -- 172.21.15.97:0/199512176 >> 172.21.15.97:0/199512176 conn(0x564e2bf09e90 msgr2=0x564e2bf04670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:27:32.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.683+0000 7f39e4f7b740 1 -- 172.21.15.97:0/199512176 shutdown_connections 2024-04-23T10:27:32.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.683+0000 7f39e4f7b740 1 -- 172.21.15.97:0/199512176 wait complete. 2024-04-23T10:27:32.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.683+0000 7f39e4f7b740 1 -- start start 2024-04-23T10:27:32.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.683+0000 7f39e4f7b740 1 Processor -- start 2024-04-23T10:27:32.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.683+0000 7f39e4f7b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e2bef9b30 0x564e2bf15090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:32.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.683+0000 7f39e4f7b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564e2befd520 0x564e2bf10400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:32.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.683+0000 7f39e4f7b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564e2befde40 0x564e2bf109a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:32.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.683+0000 7f39e4f7b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f39dc1acc60 con 0x564e2befde40 2024-04-23T10:27:32.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.683+0000 7f39e4f7b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f39dc1ac590 con 0x564e2bef9b30 2024-04-23T10:27:32.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.683+0000 7f39e4f7b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f39dc1a97f0 con 0x564e2befd520 2024-04-23T10:27:32.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.684+0000 7f39e4f7b740 10 client.0 _ll_register_callbacks cb 0x7f39c4005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:27:32.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.684+0000 7f39dbfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564e2befde40 0x564e2bf109a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:32.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.684+0000 7f39dbfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564e2befde40 0x564e2bf109a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50496/0 (socket says 172.21.15.97:50496) 2024-04-23T10:27:32.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.684+0000 7f39dbfff640 1 -- 172.21.15.97:0/3257950737 learned_addr learned my addr 172.21.15.97:0/3257950737 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:27:32.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.684+0000 7f39d3fff640 1 --2- 172.21.15.97:0/3257950737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564e2befd520 0x564e2bf10400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:32.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.684+0000 7f39db7fe640 1 --2- 172.21.15.97:0/3257950737 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e2bef9b30 0x564e2bf15090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:32.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.684+0000 7f39dbfff640 1 -- 172.21.15.97:0/3257950737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564e2befd520 msgr2=0x564e2bf10400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:27:32.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.684+0000 7f39dbfff640 1 --2- 172.21.15.97:0/3257950737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564e2befd520 0x564e2bf10400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:32.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.684+0000 7f39dbfff640 1 -- 172.21.15.97:0/3257950737 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e2bef9b30 msgr2=0x564e2bf15090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:27:32.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.684+0000 7f39dbfff640 1 --2- 172.21.15.97:0/3257950737 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e2bef9b30 0x564e2bf15090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:32.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.684+0000 7f39dbfff640 1 -- 172.21.15.97:0/3257950737 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e2bef9960 con 0x564e2befde40 2024-04-23T10:27:32.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.684+0000 7f39db7fe640 1 --2- 172.21.15.97:0/3257950737 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e2bef9b30 0x564e2bf15090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:27:32.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.684+0000 7f39d3fff640 1 --2- 172.21.15.97:0/3257950737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564e2befd520 0x564e2bf10400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:27:32.688 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.684+0000 7f39dbfff640 1 --2- 172.21.15.97:0/3257950737 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564e2befde40 0x564e2bf109a0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f39d4009bc0 tx=0x7f39d4014850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecb68c05566f0f09 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:27:32.688 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.684+0000 7f39baffd640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:27:32.688 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.684+0000 7f39baffd640 1 -- 172.21.15.97:0/3257950737 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d4038060 con 0x564e2befde40 2024-04-23T10:27:32.688 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.684+0000 7f39baffd640 1 -- 172.21.15.97:0/3257950737 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f39d4004a40 con 0x564e2befde40 2024-04-23T10:27:32.688 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.684+0000 7f39e4f7b740 1 -- 172.21.15.97:0/3257950737 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39d40119d0 con 0x564e2befde40 2024-04-23T10:27:32.688 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.684+0000 7f39baffd640 1 -- 172.21.15.97:0/3257950737 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f39d4026050 con 0x564e2befde40 2024-04-23T10:27:32.688 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.684+0000 7f39e4f7b740 10 client.15891 fetch_fsmap learned FSMap version 26 2024-04-23T10:27:32.688 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.684+0000 7f39e4f7b740 1 -- 172.21.15.97:0/3257950737 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564e2bf83030 con 0x564e2befde40 2024-04-23T10:27:32.688 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.684+0000 7f39baffd640 1 -- 172.21.15.97:0/3257950737 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39d40119d0 con 0x564e2befde40 2024-04-23T10:27:32.688 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.685+0000 7f39baffd640 1 -- 172.21.15.97:0/3257950737 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f39d4020040 con 0x564e2befde40 2024-04-23T10:27:32.688 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.685+0000 7f39e4f7b740 10 client.15891 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:27:32.688 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.685+0000 7f39e4f7b740 10 client.15891 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:27:32.688 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.685+0000 7f39e4f7b740 20 client.15891 populate_metadata read hostname 'smithi097' 2024-04-23T10:27:32.688 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.685+0000 7f39e4f7b740 1 --2- 172.21.15.97:0/3257950737 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x564e2bf0cc10 0x564e2bf85760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:32.688 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.685+0000 7f39e4f7b740 4 client.15891 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:27:32.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.685+0000 7f39e4f7b740 1 -- 172.21.15.97:0/3257950737 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564e2be6a200 con 0x564e2bf0cc10 2024-04-23T10:27:32.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.685+0000 7f39db7fe640 1 --2- 172.21.15.97:0/3257950737 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x564e2bf0cc10 0x564e2bf85760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:32.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.685+0000 7f39db7fe640 1 --2- 172.21.15.97:0/3257950737 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x564e2bf0cc10 0x564e2bf85760 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f587af97681ec7c server_cookie=3f90e9e98799b0b in_seq=0 out_seq=0 2024-04-23T10:27:32.689 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.685+0000 7f39baffd640 10 client.15891 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:27:32.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.696+0000 7f39baffd640 1 -- 172.21.15.97:0/3257950737 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564e2be6a200 con 0x564e2bf0cc10 2024-04-23T10:27:32.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.696+0000 7f39baffd640 10 client.15891 handle_command_reply: tid=0 2024-04-23T10:27:32.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.696+0000 7f39e4f7b740 1 client.15891 shutdown 2024-04-23T10:27:32.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.697+0000 7f39e4f7b740 1 -- 172.21.15.97:0/3257950737 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564e2befde40 msgr2=0x564e2bf109a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:32.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.697+0000 7f39e4f7b740 1 --2- 172.21.15.97:0/3257950737 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564e2befde40 0x564e2bf109a0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f39d4009bc0 tx=0x7f39d4014850 comp rx=0 tx=0).stop 2024-04-23T10:27:32.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.697+0000 7f39e4f7b740 1 -- 172.21.15.97:0/3257950737 shutdown_connections 2024-04-23T10:27:32.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.697+0000 7f39e4f7b740 1 --2- 172.21.15.97:0/3257950737 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x564e2bf0cc10 0x564e2bf85760 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:32.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.697+0000 7f39e4f7b740 1 --2- 172.21.15.97:0/3257950737 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564e2befde40 0x564e2bf109a0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:32.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.697+0000 7f39baffd640 0 client.15891 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:27:32.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.697+0000 7f39e4f7b740 1 --2- 172.21.15.97:0/3257950737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564e2befd520 0x564e2bf10400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:32.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.697+0000 7f39e4f7b740 1 --2- 172.21.15.97:0/3257950737 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564e2bef9b30 0x564e2bf15090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:32.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.697+0000 7f39e4f7b740 1 -- 172.21.15.97:0/3257950737 >> 172.21.15.97:0/3257950737 conn(0x564e2bf09e90 msgr2=0x564e2bef8420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:27:32.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.697+0000 7f39e4f7b740 1 -- 172.21.15.97:0/3257950737 shutdown_connections 2024-04-23T10:27:32.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.697+0000 7f39e4f7b740 1 -- 172.21.15.97:0/3257950737 wait complete. 2024-04-23T10:27:32.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.697+0000 7f39e4f7b740 20 client.15891 trim_cache size 0 max 16384 2024-04-23T10:27:32.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:27:32.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:27:32.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:27:32.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:27:32.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.697+0000 7f39e2187640 1 -- 172.21.15.97:0/2204105687 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f39c80a3ff0 msgr2=0x7f39c80a64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:32.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.697+0000 7f39e2187640 1 --2- 172.21.15.97:0/2204105687 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f39c80a3ff0 0x7f39c80a64b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f39d4005fd0 tx=0x7f39d4009500 comp rx=0 tx=0).stop 2024-04-23T10:27:32.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.697+0000 7f39e2187640 1 -- 172.21.15.97:0/2204105687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39dc103b20 msgr2=0x7f39dc198be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:32.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.697+0000 7f39e2187640 1 --2- 172.21.15.97:0/2204105687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39dc103b20 0x7f39dc198be0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f39c4005940 tx=0x7f39c4004270 comp rx=0 tx=0).stop 2024-04-23T10:27:32.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.697+0000 7f39e2187640 1 -- 172.21.15.97:0/2204105687 shutdown_connections 2024-04-23T10:27:32.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.697+0000 7f39e2187640 1 --2- 172.21.15.97:0/2204105687 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f39c80a3ff0 0x7f39c80a64b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:32.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.697+0000 7f39e2187640 1 --2- 172.21.15.97:0/2204105687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39dc10c540 0x7f39dc19cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:32.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.697+0000 7f39e2187640 1 --2- 172.21.15.97:0/2204105687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f39dc103b20 0x7f39dc198be0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:32.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.697+0000 7f39e2187640 1 --2- 172.21.15.97:0/2204105687 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f39dc103200 0x7f39dc1986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:32.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.697+0000 7f39e2187640 1 -- 172.21.15.97:0/2204105687 >> 172.21.15.97:0/2204105687 conn(0x7f39dc077fd0 msgr2=0x7f39dc0fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:27:32.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.698+0000 7f39e2187640 1 -- 172.21.15.97:0/2204105687 shutdown_connections 2024-04-23T10:27:32.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:32.698+0000 7f39e2187640 1 -- 172.21.15.97:0/2204105687 wait complete. 2024-04-23T10:27:33.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:27:33 smithi196 ceph-mon[32919]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-04-23T10:27:33.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:27:33 smithi097 ceph-mon[29197]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-04-23T10:27:33.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:27:33 smithi183 ceph-mon[32317]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-04-23T10:27:35.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:27:35 smithi196 ceph-mon[32919]: pgmap v44: 97 pgs: 97 active+clean; 1.5 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-23T10:27:35.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:27:35 smithi097 ceph-mon[29197]: pgmap v44: 97 pgs: 97 active+clean; 1.5 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-23T10:27:35.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:27:35 smithi183 ceph-mon[32317]: pgmap v44: 97 pgs: 97 active+clean; 1.5 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-04-23T10:27:37.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:27:37 smithi196 ceph-mon[32919]: pgmap v45: 97 pgs: 97 active+clean; 1.5 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-23T10:27:37.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:27:37 smithi097 ceph-mon[29197]: pgmap v45: 97 pgs: 97 active+clean; 1.5 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-23T10:27:37.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:27:37 smithi183 ceph-mon[32317]: pgmap v45: 97 pgs: 97 active+clean; 1.5 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-23T10:27:39.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:27:39 smithi196 ceph-mon[32919]: pgmap v46: 97 pgs: 97 active+clean; 1.5 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-04-23T10:27:39.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:27:39 smithi097 ceph-mon[29197]: pgmap v46: 97 pgs: 97 active+clean; 1.5 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-04-23T10:27:39.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:27:39 smithi183 ceph-mon[32317]: pgmap v46: 97 pgs: 97 active+clean; 1.5 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-04-23T10:27:41.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:27:41 smithi196 ceph-mon[32919]: pgmap v47: 97 pgs: 97 active+clean; 1.5 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 15 op/s 2024-04-23T10:27:41.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:27:41 smithi097 ceph-mon[29197]: pgmap v47: 97 pgs: 97 active+clean; 1.5 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 15 op/s 2024-04-23T10:27:41.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:27:41 smithi183 ceph-mon[32317]: pgmap v47: 97 pgs: 97 active+clean; 1.5 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 15 op/s 2024-04-23T10:27:43.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:27:43 smithi196 ceph-mon[32919]: pgmap v48: 97 pgs: 97 active+clean; 1.5 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 15 op/s 2024-04-23T10:27:43.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:27:43 smithi097 ceph-mon[29197]: pgmap v48: 97 pgs: 97 active+clean; 1.5 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 15 op/s 2024-04-23T10:27:43.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:27:43 smithi183 ceph-mon[32317]: pgmap v48: 97 pgs: 97 active+clean; 1.5 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 15 op/s 2024-04-23T10:27:45.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:27:45 smithi196 ceph-mon[32919]: pgmap v49: 97 pgs: 97 active+clean; 1.6 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 25 KiB/s wr, 24 op/s 2024-04-23T10:27:45.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:27:45 smithi196 ceph-mon[32919]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-23T10:27:45.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:27:45 smithi097 ceph-mon[29197]: pgmap v49: 97 pgs: 97 active+clean; 1.6 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 25 KiB/s wr, 24 op/s 2024-04-23T10:27:45.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:27:45 smithi097 ceph-mon[29197]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-23T10:27:45.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:27:45 smithi183 ceph-mon[32317]: pgmap v49: 97 pgs: 97 active+clean; 1.6 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 25 KiB/s wr, 24 op/s 2024-04-23T10:27:45.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:27:45 smithi183 ceph-mon[32317]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-23T10:27:45.795 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.794+0000 7fe6967f7640 1 -- 172.21.15.97:0/4025969396 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe69010c520 msgr2=0x7fe69010e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:45.795 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.794+0000 7fe6967f7640 1 --2- 172.21.15.97:0/4025969396 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe69010c520 0x7fe69010e9c0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fe688009ca0 tx=0x7fe68802a9e0 comp rx=0 tx=0).stop 2024-04-23T10:27:45.795 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.794+0000 7fe6967f7640 1 -- 172.21.15.97:0/4025969396 shutdown_connections 2024-04-23T10:27:45.795 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.794+0000 7fe6967f7640 1 --2- 172.21.15.97:0/4025969396 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe69010c520 0x7fe69010e9c0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:45.795 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.794+0000 7fe6967f7640 1 --2- 172.21.15.97:0/4025969396 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe690103b00 0x7fe69010bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:45.795 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.794+0000 7fe6967f7640 1 --2- 172.21.15.97:0/4025969396 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6901031e0 0x7fe6901035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:45.795 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.794+0000 7fe6967f7640 1 -- 172.21.15.97:0/4025969396 >> 172.21.15.97:0/4025969396 conn(0x7fe690077fd0 msgr2=0x7fe6900783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:27:45.795 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.794+0000 7fe6967f7640 1 -- 172.21.15.97:0/4025969396 shutdown_connections 2024-04-23T10:27:45.795 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.794+0000 7fe6967f7640 1 -- 172.21.15.97:0/4025969396 wait complete. 2024-04-23T10:27:45.795 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.794+0000 7fe6967f7640 1 Processor -- start 2024-04-23T10:27:45.795 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.794+0000 7fe6967f7640 1 -- start start 2024-04-23T10:27:45.796 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.795+0000 7fe6967f7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe6901031e0 0x7fe690198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:45.796 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.795+0000 7fe6967f7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe690103b00 0x7fe690198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:45.796 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.795+0000 7fe6967f7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe69010c520 0x7fe69019cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:45.796 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.795+0000 7fe6967f7640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe6901121b0 con 0x7fe690103b00 2024-04-23T10:27:45.796 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.795+0000 7fe6967f7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe690112030 con 0x7fe6901031e0 2024-04-23T10:27:45.796 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.795+0000 7fe6967f7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe690112330 con 0x7fe69010c520 2024-04-23T10:27:45.796 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.795+0000 7fe68f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe690103b00 0x7fe690198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:45.796 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.795+0000 7fe68f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe690103b00 0x7fe690198bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54944/0 (socket says 172.21.15.97:54944) 2024-04-23T10:27:45.796 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.795+0000 7fe68f7fe640 1 -- 172.21.15.97:0/1217796617 learned_addr learned my addr 172.21.15.97:0/1217796617 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:27:45.796 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.795+0000 7fe68ffff640 1 --2- 172.21.15.97:0/1217796617 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe6901031e0 0x7fe690198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:45.796 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.795+0000 7fe694d94640 1 --2- 172.21.15.97:0/1217796617 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe69010c520 0x7fe69019cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:45.796 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.795+0000 7fe68f7fe640 1 -- 172.21.15.97:0/1217796617 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe69010c520 msgr2=0x7fe69019cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:45.796 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.795+0000 7fe68f7fe640 1 --2- 172.21.15.97:0/1217796617 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe69010c520 0x7fe69019cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:45.796 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.795+0000 7fe68f7fe640 1 -- 172.21.15.97:0/1217796617 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe6901031e0 msgr2=0x7fe690198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:45.796 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.795+0000 7fe68f7fe640 1 --2- 172.21.15.97:0/1217796617 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe6901031e0 0x7fe690198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:45.797 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.795+0000 7fe68f7fe640 1 -- 172.21.15.97:0/1217796617 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe69019d680 con 0x7fe690103b00 2024-04-23T10:27:45.797 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.795+0000 7fe68f7fe640 1 --2- 172.21.15.97:0/1217796617 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe690103b00 0x7fe690198bd0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fe68400c890 tx=0x7fe68400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:27:45.797 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.796+0000 7fe68d7fa640 1 -- 172.21.15.97:0/1217796617 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe684004220 con 0x7fe690103b00 2024-04-23T10:27:45.797 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.796+0000 7fe68d7fa640 1 -- 172.21.15.97:0/1217796617 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6840043c0 con 0x7fe690103b00 2024-04-23T10:27:45.797 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.796+0000 7fe6967f7640 1 -- 172.21.15.97:0/1217796617 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe69019d970 con 0x7fe690103b00 2024-04-23T10:27:45.797 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.796+0000 7fe68d7fa640 1 -- 172.21.15.97:0/1217796617 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe684004a00 con 0x7fe690103b00 2024-04-23T10:27:45.797 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.796+0000 7fe6967f7640 1 -- 172.21.15.97:0/1217796617 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6901a5200 con 0x7fe690103b00 2024-04-23T10:27:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.798+0000 7fe68d7fa640 1 -- 172.21.15.97:0/1217796617 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe6840079b0 con 0x7fe690103b00 2024-04-23T10:27:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.798+0000 7fe6995e4740 1 Processor -- start 2024-04-23T10:27:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.798+0000 7fe6995e4740 1 -- start start 2024-04-23T10:27:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.798+0000 7fe68d7fa640 1 --2- 172.21.15.97:0/1217796617 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fe6780a3dc0 0x7fe6780a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.799+0000 7fe6995e4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f3e85b620 0x560f3e85ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.799+0000 7fe6995e4740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f3e85bf40 0x560f3e8662c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.799+0000 7fe6995e4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f3e866800 0x560f3e868bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.799+0000 7fe6995e4740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560f3e7c0970 con 0x560f3e85bf40 2024-04-23T10:27:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.799+0000 7fe6995e4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560f3e75d4a0 con 0x560f3e866800 2024-04-23T10:27:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.799+0000 7fe6995e4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560f3e7bc930 con 0x560f3e85b620 2024-04-23T10:27:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.799+0000 7fe694d94640 1 --2- 172.21.15.97:0/1217796617 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fe6780a3dc0 0x7fe6780a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.799+0000 7fe68f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f3e85bf40 0x560f3e8662c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.799+0000 7fe68f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f3e85bf40 0x560f3e8662c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54954/0 (socket says 172.21.15.97:54954) 2024-04-23T10:27:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.799+0000 7fe68f7fe640 1 -- 172.21.15.97:0/828430165 learned_addr learned my addr 172.21.15.97:0/828430165 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:27:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.799+0000 7fe694d94640 1 --2- 172.21.15.97:0/828430165 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f3e866800 0x560f3e868bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.799+0000 7fe68d7fa640 1 -- 172.21.15.97:0/1217796617 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fe6840d03e0 con 0x7fe690103b00 2024-04-23T10:27:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.799+0000 7fe68ffff640 1 --2- 172.21.15.97:0/828430165 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f3e85b620 0x560f3e85ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.799+0000 7fe68f7fe640 1 -- 172.21.15.97:0/828430165 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f3e85b620 msgr2=0x560f3e85ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:27:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.799+0000 7fe68f7fe640 1 --2- 172.21.15.97:0/828430165 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f3e85b620 0x560f3e85ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.799+0000 7fe68f7fe640 1 -- 172.21.15.97:0/828430165 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f3e866800 msgr2=0x560f3e868bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:27:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.799+0000 7fe68f7fe640 1 --2- 172.21.15.97:0/828430165 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f3e866800 0x560f3e868bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.799+0000 7fe68f7fe640 1 -- 172.21.15.97:0/828430165 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f3e4d3210 con 0x560f3e85bf40 2024-04-23T10:27:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.799+0000 7fe694d94640 1 --2- 172.21.15.97:0/1217796617 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fe6780a3dc0 0x7fe6780a6280 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe688006010 tx=0x7fe68802ed90 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:27:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.799+0000 7fe68f7fe640 1 --2- 172.21.15.97:0/828430165 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f3e85bf40 0x560f3e8662c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fe68409c180 tx=0x7fe68401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cabc65c102ac66e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:27:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.799+0000 7fe6767fc640 1 -- 172.21.15.97:0/828430165 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68401d070 con 0x560f3e85bf40 2024-04-23T10:27:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.799+0000 7fe6767fc640 1 -- 172.21.15.97:0/828430165 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe684005290 con 0x560f3e85bf40 2024-04-23T10:27:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.799+0000 7fe6767fc640 1 -- 172.21.15.97:0/828430165 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6840a5040 con 0x560f3e85bf40 2024-04-23T10:27:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.800+0000 7fe6995e4740 1 -- 172.21.15.97:0/828430165 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f3e85bf40 msgr2=0x560f3e8662c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.800+0000 7fe6995e4740 1 --2- 172.21.15.97:0/828430165 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f3e85bf40 0x560f3e8662c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fe68409c180 tx=0x7fe68401b000 comp rx=0 tx=0).stop 2024-04-23T10:27:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.800+0000 7fe6995e4740 1 -- 172.21.15.97:0/828430165 shutdown_connections 2024-04-23T10:27:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.800+0000 7fe6995e4740 1 --2- 172.21.15.97:0/828430165 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f3e866800 0x560f3e868bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.800+0000 7fe6995e4740 1 --2- 172.21.15.97:0/828430165 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f3e85bf40 0x560f3e8662c0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.800+0000 7fe6995e4740 1 --2- 172.21.15.97:0/828430165 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f3e85b620 0x560f3e85ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.800+0000 7fe6995e4740 1 -- 172.21.15.97:0/828430165 >> 172.21.15.97:0/828430165 conn(0x560f3e857b30 msgr2=0x560f3e857f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:27:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.800+0000 7fe6995e4740 1 -- 172.21.15.97:0/828430165 shutdown_connections 2024-04-23T10:27:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.800+0000 7fe6995e4740 1 -- 172.21.15.97:0/828430165 wait complete. 2024-04-23T10:27:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.800+0000 7fe6995e4740 1 -- start start 2024-04-23T10:27:45.806 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.801+0000 7fe6995e4740 1 Processor -- start 2024-04-23T10:27:45.806 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.801+0000 7fe6995e4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f3e85b620 0x560f3e8661d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:45.806 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.801+0000 7fe6995e4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f3e85bf40 0x560f3e8634c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:45.806 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.801+0000 7fe6995e4740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f3e866800 0x560f3e863ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:45.806 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.801+0000 7fe6995e4740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560f3e869430 con 0x560f3e866800 2024-04-23T10:27:45.806 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.801+0000 7fe6995e4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560f3e75b0b0 con 0x560f3e85bf40 2024-04-23T10:27:45.806 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.801+0000 7fe6995e4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560f3e8695b0 con 0x560f3e85b620 2024-04-23T10:27:45.806 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.801+0000 7fe6995e4740 10 client.0 _ll_register_callbacks cb 0x560f3e748d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:27:45.806 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.801+0000 7fe694d94640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f3e866800 0x560f3e863ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:45.806 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.801+0000 7fe694d94640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f3e866800 0x560f3e863ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54956/0 (socket says 172.21.15.97:54956) 2024-04-23T10:27:45.806 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.801+0000 7fe694d94640 1 -- 172.21.15.97:0/3069293140 learned_addr learned my addr 172.21.15.97:0/3069293140 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:27:45.806 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.801+0000 7fe68ffff640 1 --2- 172.21.15.97:0/3069293140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f3e85b620 0x560f3e8661d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:45.806 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.801+0000 7fe68f7fe640 1 --2- 172.21.15.97:0/3069293140 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f3e85bf40 0x560f3e8634c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:45.806 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.801+0000 7fe694d94640 1 -- 172.21.15.97:0/3069293140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f3e85b620 msgr2=0x560f3e8661d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:27:45.806 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.801+0000 7fe694d94640 1 --2- 172.21.15.97:0/3069293140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f3e85b620 0x560f3e8661d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:45.807 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.801+0000 7fe694d94640 1 -- 172.21.15.97:0/3069293140 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f3e85bf40 msgr2=0x560f3e8634c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:27:45.807 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.801+0000 7fe694d94640 1 --2- 172.21.15.97:0/3069293140 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f3e85bf40 0x560f3e8634c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:45.807 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.801+0000 7fe694d94640 1 -- 172.21.15.97:0/3069293140 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f3e59f620 con 0x560f3e866800 2024-04-23T10:27:45.807 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.802+0000 7fe694d94640 1 --2- 172.21.15.97:0/3069293140 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f3e866800 0x560f3e863ad0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fe68802f040 tx=0x7fe68802a9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4dbca043fec801a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:27:45.807 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.802+0000 7fe66f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:27:45.807 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.802+0000 7fe66f7fe640 1 -- 172.21.15.97:0/3069293140 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe688033040 con 0x560f3e866800 2024-04-23T10:27:45.807 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.802+0000 7fe66f7fe640 1 -- 172.21.15.97:0/3069293140 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fe688038720 con 0x560f3e866800 2024-04-23T10:27:45.807 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.802+0000 7fe6995e4740 1 -- 172.21.15.97:0/3069293140 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560f3e7ce390 con 0x560f3e866800 2024-04-23T10:27:45.807 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.802+0000 7fe66f7fe640 1 -- 172.21.15.97:0/3069293140 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe68805f050 con 0x560f3e866800 2024-04-23T10:27:45.807 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.802+0000 7fe6995e4740 10 client.15903 fetch_fsmap learned FSMap version 26 2024-04-23T10:27:45.807 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.802+0000 7fe6995e4740 1 -- 172.21.15.97:0/3069293140 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560f3e875400 con 0x560f3e866800 2024-04-23T10:27:45.807 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.802+0000 7fe66f7fe640 1 -- 172.21.15.97:0/3069293140 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x560f3e7ce390 con 0x560f3e866800 2024-04-23T10:27:45.807 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.802+0000 7fe66f7fe640 1 -- 172.21.15.97:0/3069293140 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe688051a40 con 0x560f3e866800 2024-04-23T10:27:45.807 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.802+0000 7fe6995e4740 10 client.15903 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:27:45.807 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.802+0000 7fe6995e4740 10 client.15903 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:27:45.808 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.802+0000 7fe6995e4740 20 client.15903 populate_metadata read hostname 'smithi097' 2024-04-23T10:27:45.808 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.802+0000 7fe6995e4740 1 --2- 172.21.15.97:0/3069293140 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x560f3e8768b0 0x560f3e878ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:45.808 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.802+0000 7fe6995e4740 4 client.15903 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:27:45.808 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.802+0000 7fe6995e4740 1 -- 172.21.15.97:0/3069293140 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f3e7c8200 con 0x560f3e8768b0 2024-04-23T10:27:45.808 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.803+0000 7fe68ffff640 1 --2- 172.21.15.97:0/3069293140 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x560f3e8768b0 0x560f3e878ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:45.808 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.803+0000 7fe68ffff640 1 --2- 172.21.15.97:0/3069293140 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x560f3e8768b0 0x560f3e878ca0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=247090c5229f509f server_cookie=be9b521393e8c67d in_seq=0 out_seq=0 2024-04-23T10:27:45.808 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.803+0000 7fe66f7fe640 10 client.15903 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:27:45.808 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.804+0000 7fe66f7fe640 1 -- 172.21.15.97:0/3069293140 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560f3e7c8200 con 0x560f3e8768b0 2024-04-23T10:27:45.808 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.804+0000 7fe66f7fe640 10 client.15903 handle_command_reply: tid=0 2024-04-23T10:27:45.808 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.804+0000 7fe6995e4740 1 client.15903 shutdown 2024-04-23T10:27:45.808 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.804+0000 7fe6995e4740 1 -- 172.21.15.97:0/3069293140 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f3e866800 msgr2=0x560f3e863ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:45.808 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.804+0000 7fe6995e4740 1 --2- 172.21.15.97:0/3069293140 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f3e866800 0x560f3e863ad0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fe68802f040 tx=0x7fe68802a9e0 comp rx=0 tx=0).stop 2024-04-23T10:27:45.808 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.804+0000 7fe6995e4740 1 -- 172.21.15.97:0/3069293140 shutdown_connections 2024-04-23T10:27:45.808 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.804+0000 7fe6995e4740 1 --2- 172.21.15.97:0/3069293140 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x560f3e8768b0 0x560f3e878ca0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:45.809 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.804+0000 7fe6995e4740 1 --2- 172.21.15.97:0/3069293140 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f3e866800 0x560f3e863ad0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:45.809 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.804+0000 7fe6995e4740 1 --2- 172.21.15.97:0/3069293140 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f3e85bf40 0x560f3e8634c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:45.809 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.804+0000 7fe6995e4740 1 --2- 172.21.15.97:0/3069293140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f3e85b620 0x560f3e8661d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:45.809 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.804+0000 7fe6995e4740 1 -- 172.21.15.97:0/3069293140 >> 172.21.15.97:0/3069293140 conn(0x560f3e857b30 msgr2=0x560f3e85d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:27:45.809 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.804+0000 7fe66f7fe640 0 client.15903 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:27:45.809 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.804+0000 7fe6995e4740 1 -- 172.21.15.97:0/3069293140 shutdown_connections 2024-04-23T10:27:45.809 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.804+0000 7fe6995e4740 1 -- 172.21.15.97:0/3069293140 wait complete. 2024-04-23T10:27:45.809 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.804+0000 7fe6995e4740 20 client.15903 trim_cache size 0 max 16384 2024-04-23T10:27:45.822 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.819+0000 7fe6995e4740 1 Processor -- start 2024-04-23T10:27:45.822 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.819+0000 7fe6995e4740 1 -- start start 2024-04-23T10:27:45.823 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.819+0000 7fe6995e4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f3e85b620 0x560f3e865d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:45.823 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.819+0000 7fe6995e4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f3e85bf40 0x560f3e87e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:45.823 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.819+0000 7fe6995e4740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f3e866800 0x560f3e87ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:45.823 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.819+0000 7fe6995e4740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe6901ada70 con 0x560f3e866800 2024-04-23T10:27:45.823 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.819+0000 7fe6995e4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe6901af5b0 con 0x560f3e85bf40 2024-04-23T10:27:45.823 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.819+0000 7fe6995e4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe6901ae140 con 0x560f3e85b620 2024-04-23T10:27:45.823 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.819+0000 7fe694d94640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f3e866800 0x560f3e87ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:45.823 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.819+0000 7fe694d94640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f3e866800 0x560f3e87ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54960/0 (socket says 172.21.15.97:54960) 2024-04-23T10:27:45.823 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.819+0000 7fe694d94640 1 -- 172.21.15.97:0/3112765915 learned_addr learned my addr 172.21.15.97:0/3112765915 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:27:45.823 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.819+0000 7fe68ffff640 1 --2- 172.21.15.97:0/3112765915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f3e85b620 0x560f3e865d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:45.823 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.819+0000 7fe694d94640 1 -- 172.21.15.97:0/3112765915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f3e85b620 msgr2=0x560f3e865d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:27:45.823 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.819+0000 7fe694d94640 1 --2- 172.21.15.97:0/3112765915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f3e85b620 0x560f3e865d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:45.823 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.819+0000 7fe68f7fe640 1 --2- 172.21.15.97:0/3112765915 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f3e85bf40 0x560f3e87e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:45.823 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.819+0000 7fe694d94640 1 -- 172.21.15.97:0/3112765915 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f3e85bf40 msgr2=0x560f3e87e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:27:45.824 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.819+0000 7fe694d94640 1 --2- 172.21.15.97:0/3112765915 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f3e85bf40 0x560f3e87e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:45.824 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.819+0000 7fe694d94640 1 -- 172.21.15.97:0/3112765915 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f3e8640d0 con 0x560f3e866800 2024-04-23T10:27:45.824 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.819+0000 7fe68f7fe640 1 --2- 172.21.15.97:0/3112765915 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f3e85bf40 0x560f3e87e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:27:45.824 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.820+0000 7fe694d94640 1 --2- 172.21.15.97:0/3112765915 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f3e866800 0x560f3e87ead0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fe688009940 tx=0x7fe68802e7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b69d56194d896 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:27:45.824 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.820+0000 7fe675ffb640 1 -- 172.21.15.97:0/3112765915 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe688033040 con 0x560f3e866800 2024-04-23T10:27:45.824 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.820+0000 7fe675ffb640 1 -- 172.21.15.97:0/3112765915 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe688056070 con 0x560f3e866800 2024-04-23T10:27:45.824 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.820+0000 7fe675ffb640 1 -- 172.21.15.97:0/3112765915 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe688051920 con 0x560f3e866800 2024-04-23T10:27:45.824 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.820+0000 7fe6995e4740 1 -- 172.21.15.97:0/3112765915 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f3e866800 msgr2=0x560f3e87ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:45.824 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.820+0000 7fe6995e4740 1 --2- 172.21.15.97:0/3112765915 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f3e866800 0x560f3e87ead0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fe688009940 tx=0x7fe68802e7d0 comp rx=0 tx=0).stop 2024-04-23T10:27:45.824 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.820+0000 7fe6995e4740 1 -- 172.21.15.97:0/3112765915 shutdown_connections 2024-04-23T10:27:45.824 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.820+0000 7fe6995e4740 1 --2- 172.21.15.97:0/3112765915 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f3e866800 0x560f3e87ead0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:45.824 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.820+0000 7fe6995e4740 1 --2- 172.21.15.97:0/3112765915 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f3e85bf40 0x560f3e87e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:45.824 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.820+0000 7fe6995e4740 1 --2- 172.21.15.97:0/3112765915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f3e85b620 0x560f3e865d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:45.824 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.820+0000 7fe6995e4740 1 -- 172.21.15.97:0/3112765915 >> 172.21.15.97:0/3112765915 conn(0x560f3e857b30 msgr2=0x560f3e863480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:27:45.824 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.820+0000 7fe6995e4740 1 -- 172.21.15.97:0/3112765915 shutdown_connections 2024-04-23T10:27:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.820+0000 7fe6995e4740 1 -- 172.21.15.97:0/3112765915 wait complete. 2024-04-23T10:27:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.821+0000 7fe6995e4740 1 -- start start 2024-04-23T10:27:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.821+0000 7fe6995e4740 1 Processor -- start 2024-04-23T10:27:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.821+0000 7fe6995e4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f3e85b620 0x560f3e86e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.821+0000 7fe6995e4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f3e85bf40 0x560f3e86bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.821+0000 7fe6995e4740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f3e866800 0x560f3e86c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.821+0000 7fe6995e4740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe6901aeee0 con 0x560f3e866800 2024-04-23T10:27:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.821+0000 7fe6995e4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe6901ae810 con 0x560f3e85b620 2024-04-23T10:27:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.821+0000 7fe6995e4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe6901aba50 con 0x560f3e85bf40 2024-04-23T10:27:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.821+0000 7fe6995e4740 10 client.0 _ll_register_callbacks cb 0x7fe68802e840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:27:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.821+0000 7fe694d94640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f3e866800 0x560f3e86c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.821+0000 7fe694d94640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f3e866800 0x560f3e86c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54976/0 (socket says 172.21.15.97:54976) 2024-04-23T10:27:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.821+0000 7fe694d94640 1 -- 172.21.15.97:0/212324133 learned_addr learned my addr 172.21.15.97:0/212324133 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:27:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.821+0000 7fe68ffff640 1 --2- 172.21.15.97:0/212324133 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f3e85b620 0x560f3e86e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:45.826 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.821+0000 7fe68f7fe640 1 --2- 172.21.15.97:0/212324133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f3e85bf40 0x560f3e86bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:45.826 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.821+0000 7fe694d94640 1 -- 172.21.15.97:0/212324133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f3e85bf40 msgr2=0x560f3e86bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:27:45.826 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.821+0000 7fe694d94640 1 --2- 172.21.15.97:0/212324133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f3e85bf40 0x560f3e86bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:45.826 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.821+0000 7fe694d94640 1 -- 172.21.15.97:0/212324133 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f3e85b620 msgr2=0x560f3e86e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:27:45.826 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.821+0000 7fe694d94640 1 --2- 172.21.15.97:0/212324133 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f3e85b620 0x560f3e86e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:45.826 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.822+0000 7fe694d94640 1 -- 172.21.15.97:0/212324133 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f3e8532e0 con 0x560f3e866800 2024-04-23T10:27:45.826 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.822+0000 7fe68ffff640 1 --2- 172.21.15.97:0/212324133 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f3e85b620 0x560f3e86e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:27:45.826 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.822+0000 7fe694d94640 1 --2- 172.21.15.97:0/212324133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f3e866800 0x560f3e86c1e0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fe68802af00 tx=0x7fe688004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76a511ff43eeb44d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:27:45.826 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.822+0000 7fe66f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:27:45.826 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.822+0000 7fe66f7fe640 1 -- 172.21.15.97:0/212324133 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe688033040 con 0x560f3e866800 2024-04-23T10:27:45.826 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.822+0000 7fe66f7fe640 1 -- 172.21.15.97:0/212324133 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fe688038920 con 0x560f3e866800 2024-04-23T10:27:45.826 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.822+0000 7fe6995e4740 1 -- 172.21.15.97:0/212324133 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe688056070 con 0x560f3e866800 2024-04-23T10:27:45.826 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.822+0000 7fe66f7fe640 1 -- 172.21.15.97:0/212324133 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe68802fea0 con 0x560f3e866800 2024-04-23T10:27:45.826 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.822+0000 7fe6995e4740 10 client.15909 fetch_fsmap learned FSMap version 26 2024-04-23T10:27:45.827 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.822+0000 7fe6995e4740 1 -- 172.21.15.97:0/212324133 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560f3e8dc910 con 0x560f3e866800 2024-04-23T10:27:45.827 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.822+0000 7fe66f7fe640 1 -- 172.21.15.97:0/212324133 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe688056070 con 0x560f3e866800 2024-04-23T10:27:45.827 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.822+0000 7fe66f7fe640 1 -- 172.21.15.97:0/212324133 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe68805b410 con 0x560f3e866800 2024-04-23T10:27:45.827 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.822+0000 7fe6995e4740 10 client.15909 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:27:45.827 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.822+0000 7fe6995e4740 10 client.15909 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:27:45.827 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.822+0000 7fe6995e4740 20 client.15909 populate_metadata read hostname 'smithi097' 2024-04-23T10:27:45.827 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.822+0000 7fe6995e4740 1 --2- 172.21.15.97:0/212324133 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x560f3e8dcf20 0x560f3e8df340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:45.827 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.822+0000 7fe6995e4740 4 client.15909 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:27:45.827 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.822+0000 7fe6995e4740 1 -- 172.21.15.97:0/212324133 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560f3e7445f0 con 0x560f3e8dcf20 2024-04-23T10:27:45.827 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.823+0000 7fe68ffff640 1 --2- 172.21.15.97:0/212324133 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x560f3e8dcf20 0x560f3e8df340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:45.827 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.823+0000 7fe66f7fe640 10 client.15909 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:27:45.827 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.823+0000 7fe68ffff640 1 --2- 172.21.15.97:0/212324133 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x560f3e8dcf20 0x560f3e8df340 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=faa16fed0ef474ba server_cookie=abfeb7590f764857 in_seq=0 out_seq=0 2024-04-23T10:27:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.836+0000 7fe66f7fe640 1 -- 172.21.15.97:0/212324133 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560f3e7445f0 con 0x560f3e8dcf20 2024-04-23T10:27:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.836+0000 7fe66f7fe640 10 client.15909 handle_command_reply: tid=0 2024-04-23T10:27:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.836+0000 7fe6995e4740 1 client.15909 shutdown 2024-04-23T10:27:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.836+0000 7fe6995e4740 1 -- 172.21.15.97:0/212324133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f3e866800 msgr2=0x560f3e86c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.836+0000 7fe6995e4740 1 --2- 172.21.15.97:0/212324133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f3e866800 0x560f3e86c1e0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fe68802af00 tx=0x7fe688004ab0 comp rx=0 tx=0).stop 2024-04-23T10:27:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.836+0000 7fe6995e4740 1 -- 172.21.15.97:0/212324133 shutdown_connections 2024-04-23T10:27:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.836+0000 7fe6995e4740 1 --2- 172.21.15.97:0/212324133 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x560f3e8dcf20 0x560f3e8df340 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.836+0000 7fe6995e4740 1 --2- 172.21.15.97:0/212324133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560f3e866800 0x560f3e86c1e0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.836+0000 7fe6995e4740 1 --2- 172.21.15.97:0/212324133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f3e85bf40 0x560f3e86bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.836+0000 7fe66f7fe640 0 client.15909 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:27:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.836+0000 7fe6995e4740 1 --2- 172.21.15.97:0/212324133 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f3e85b620 0x560f3e86e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.836+0000 7fe6995e4740 1 -- 172.21.15.97:0/212324133 >> 172.21.15.97:0/212324133 conn(0x560f3e857b30 msgr2=0x560f3e856420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:27:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.836+0000 7fe6995e4740 1 -- 172.21.15.97:0/212324133 shutdown_connections 2024-04-23T10:27:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.836+0000 7fe6995e4740 1 -- 172.21.15.97:0/212324133 wait complete. 2024-04-23T10:27:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.837+0000 7fe6995e4740 20 client.15909 trim_cache size 0 max 16384 2024-04-23T10:27:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:27:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:27:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:27:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:27:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.837+0000 7fe6967f7640 1 -- 172.21.15.97:0/1217796617 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fe6780a3dc0 msgr2=0x7fe6780a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.837+0000 7fe6967f7640 1 --2- 172.21.15.97:0/1217796617 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fe6780a3dc0 0x7fe6780a6280 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe688006010 tx=0x7fe68802ed90 comp rx=0 tx=0).stop 2024-04-23T10:27:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.837+0000 7fe6967f7640 1 -- 172.21.15.97:0/1217796617 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe690103b00 msgr2=0x7fe690198bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.837+0000 7fe6967f7640 1 --2- 172.21.15.97:0/1217796617 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe690103b00 0x7fe690198bd0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fe68400c890 tx=0x7fe68400cd50 comp rx=0 tx=0).stop 2024-04-23T10:27:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.837+0000 7fe6967f7640 1 -- 172.21.15.97:0/1217796617 shutdown_connections 2024-04-23T10:27:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.837+0000 7fe6967f7640 1 --2- 172.21.15.97:0/1217796617 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fe6780a3dc0 0x7fe6780a6280 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.837+0000 7fe6967f7640 1 --2- 172.21.15.97:0/1217796617 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe69010c520 0x7fe69019cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.837+0000 7fe6967f7640 1 --2- 172.21.15.97:0/1217796617 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe690103b00 0x7fe690198bd0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.837+0000 7fe6967f7640 1 --2- 172.21.15.97:0/1217796617 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe6901031e0 0x7fe690198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.837+0000 7fe6967f7640 1 -- 172.21.15.97:0/1217796617 >> 172.21.15.97:0/1217796617 conn(0x7fe690077fd0 msgr2=0x7fe6900fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:27:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.837+0000 7fe6967f7640 1 -- 172.21.15.97:0/1217796617 shutdown_connections 2024-04-23T10:27:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:45.837+0000 7fe6967f7640 1 -- 172.21.15.97:0/1217796617 wait complete. 2024-04-23T10:27:46.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:27:46 smithi183 ceph-mon[32317]: pgmap v50: 97 pgs: 97 active+clean; 1.6 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-04-23T10:27:46.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:27:46 smithi196 ceph-mon[32919]: pgmap v50: 97 pgs: 97 active+clean; 1.6 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-04-23T10:27:46.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:27:46 smithi097 ceph-mon[29197]: pgmap v50: 97 pgs: 97 active+clean; 1.6 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-04-23T10:27:48.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:27:48 smithi196 ceph-mon[32919]: pgmap v51: 97 pgs: 97 active+clean; 1.7 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 28 KiB/s wr, 27 op/s 2024-04-23T10:27:48.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:27:48 smithi097 ceph-mon[29197]: pgmap v51: 97 pgs: 97 active+clean; 1.7 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 28 KiB/s wr, 27 op/s 2024-04-23T10:27:48.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:27:48 smithi183 ceph-mon[32317]: pgmap v51: 97 pgs: 97 active+clean; 1.7 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 28 KiB/s wr, 27 op/s 2024-04-23T10:27:50.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:27:50 smithi196 ceph-mon[32919]: pgmap v52: 97 pgs: 97 active+clean; 1.7 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-04-23T10:27:50.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:27:50 smithi097 ceph-mon[29197]: pgmap v52: 97 pgs: 97 active+clean; 1.7 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-04-23T10:27:50.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:27:50 smithi183 ceph-mon[32317]: pgmap v52: 97 pgs: 97 active+clean; 1.7 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-04-23T10:27:52.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:27:52 smithi196 ceph-mon[32919]: pgmap v53: 97 pgs: 97 active+clean; 1.7 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-04-23T10:27:52.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:27:52 smithi097 ceph-mon[29197]: pgmap v53: 97 pgs: 97 active+clean; 1.7 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-04-23T10:27:52.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:27:52 smithi183 ceph-mon[32317]: pgmap v53: 97 pgs: 97 active+clean; 1.7 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-04-23T10:27:54.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:27:54 smithi196 ceph-mon[32919]: pgmap v54: 97 pgs: 97 active+clean; 1.8 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 KiB/s wr, 24 op/s 2024-04-23T10:27:54.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:27:54 smithi097 ceph-mon[29197]: pgmap v54: 97 pgs: 97 active+clean; 1.8 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 KiB/s wr, 24 op/s 2024-04-23T10:27:54.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:27:54 smithi183 ceph-mon[32317]: pgmap v54: 97 pgs: 97 active+clean; 1.8 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 KiB/s wr, 24 op/s 2024-04-23T10:27:56.593 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:27:56 smithi196 ceph-mon[32919]: pgmap v55: 97 pgs: 97 active+clean; 1.8 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 15 op/s 2024-04-23T10:27:56.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:27:56 smithi097 ceph-mon[29197]: pgmap v55: 97 pgs: 97 active+clean; 1.8 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 15 op/s 2024-04-23T10:27:56.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:27:56 smithi183 ceph-mon[32317]: pgmap v55: 97 pgs: 97 active+clean; 1.8 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 15 op/s 2024-04-23T10:27:58.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:27:58 smithi196 ceph-mon[32919]: pgmap v56: 97 pgs: 97 active+clean; 1.8 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-04-23T10:27:58.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:27:58 smithi097 ceph-mon[29197]: pgmap v56: 97 pgs: 97 active+clean; 1.8 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-04-23T10:27:58.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:27:58 smithi183 ceph-mon[32317]: pgmap v56: 97 pgs: 97 active+clean; 1.8 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-04-23T10:27:58.935 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.932+0000 7f502ef9b640 1 -- 172.21.15.97:0/2797081840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50281010e0 msgr2=0x7f50281014c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:58.935 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.932+0000 7f502ef9b640 1 --2- 172.21.15.97:0/2797081840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50281010e0 0x7f50281014c0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f501c009900 tx=0x7f501c02a3f0 comp rx=0 tx=0).stop 2024-04-23T10:27:58.935 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.932+0000 7f502ef9b640 1 -- 172.21.15.97:0/2797081840 shutdown_connections 2024-04-23T10:27:58.936 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.932+0000 7f502ef9b640 1 --2- 172.21.15.97:0/2797081840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f50281061b0 0x7f502810dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:58.936 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.932+0000 7f502ef9b640 1 --2- 172.21.15.97:0/2797081840 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5028101a90 0x7f5028105ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:58.936 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.932+0000 7f502ef9b640 1 --2- 172.21.15.97:0/2797081840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50281010e0 0x7f50281014c0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:58.936 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.932+0000 7f502ef9b640 1 -- 172.21.15.97:0/2797081840 >> 172.21.15.97:0/2797081840 conn(0x7f50280771d0 msgr2=0x7f50280775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:27:58.936 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.932+0000 7f502ef9b640 1 -- 172.21.15.97:0/2797081840 shutdown_connections 2024-04-23T10:27:58.936 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.932+0000 7f502ef9b640 1 -- 172.21.15.97:0/2797081840 wait complete. 2024-04-23T10:27:58.936 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.933+0000 7f502ef9b640 1 Processor -- start 2024-04-23T10:27:58.936 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.933+0000 7f502ef9b640 1 -- start start 2024-04-23T10:27:58.936 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.933+0000 7f502ef9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50281010e0 0x7f5028198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:58.936 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.933+0000 7f502ef9b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50281061b0 0x7f5028198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:58.937 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.933+0000 7f502ef9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f502819d020 0x7f502819d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:58.937 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.933+0000 7f502ef9b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5028111170 con 0x7f50281010e0 2024-04-23T10:27:58.937 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.933+0000 7f502ef9b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5028110ff0 con 0x7f50281061b0 2024-04-23T10:27:58.937 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.933+0000 7f502ef9b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f50281112f0 con 0x7f502819d020 2024-04-23T10:27:58.937 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.933+0000 7f502cd37640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50281010e0 0x7f5028198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:58.937 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.933+0000 7f502cd37640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50281010e0 0x7f5028198760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44860/0 (socket says 172.21.15.97:44860) 2024-04-23T10:27:58.937 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.933+0000 7f502cd37640 1 -- 172.21.15.97:0/731436942 learned_addr learned my addr 172.21.15.97:0/731436942 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:27:58.937 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.934+0000 7f502d538640 1 --2- 172.21.15.97:0/731436942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f502819d020 0x7f502819d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:58.937 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.934+0000 7f5027fff640 1 --2- 172.21.15.97:0/731436942 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50281061b0 0x7f5028198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:58.938 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.934+0000 7f502cd37640 1 -- 172.21.15.97:0/731436942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f502819d020 msgr2=0x7f502819d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:58.938 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.934+0000 7f502cd37640 1 --2- 172.21.15.97:0/731436942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f502819d020 0x7f502819d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:58.938 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.934+0000 7f502cd37640 1 -- 172.21.15.97:0/731436942 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50281061b0 msgr2=0x7f5028198ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:58.938 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.934+0000 7f502cd37640 1 --2- 172.21.15.97:0/731436942 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50281061b0 0x7f5028198ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:58.938 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.934+0000 7f502cd37640 1 -- 172.21.15.97:0/731436942 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f502819db10 con 0x7f50281010e0 2024-04-23T10:27:58.938 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.934+0000 7f5027fff640 1 --2- 172.21.15.97:0/731436942 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50281061b0 0x7f5028198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:27:58.938 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.934+0000 7f502cd37640 1 --2- 172.21.15.97:0/731436942 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50281010e0 0x7f5028198760 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f501c005940 tx=0x7f501c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:27:58.938 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.934+0000 7f5025ffb640 1 -- 172.21.15.97:0/731436942 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f501c03f030 con 0x7f50281010e0 2024-04-23T10:27:58.938 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.934+0000 7f5025ffb640 1 -- 172.21.15.97:0/731436942 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f501c003cb0 con 0x7f50281010e0 2024-04-23T10:27:58.938 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.934+0000 7f5025ffb640 1 -- 172.21.15.97:0/731436942 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f501c035070 con 0x7f50281010e0 2024-04-23T10:27:58.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.934+0000 7f502ef9b640 1 -- 172.21.15.97:0/731436942 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f502819dda0 con 0x7f50281010e0 2024-04-23T10:27:58.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.934+0000 7f502ef9b640 1 -- 172.21.15.97:0/731436942 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50281a5560 con 0x7f50281010e0 2024-04-23T10:27:58.939 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.936+0000 7f5025ffb640 1 -- 172.21.15.97:0/731436942 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f501c042030 con 0x7f50281010e0 2024-04-23T10:27:58.948 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.937+0000 7f5031d90740 1 Processor -- start 2024-04-23T10:27:58.948 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.937+0000 7f5031d90740 1 -- start start 2024-04-23T10:27:58.948 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.937+0000 7f5025ffb640 1 --2- 172.21.15.97:0/731436942 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f50180a3dc0 0x7f50180a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:58.948 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.937+0000 7f5031d90740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d2a5a6520 0x560d2a5a6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:58.948 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.937+0000 7f5031d90740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d2a5a6e40 0x560d2a5b56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:58.948 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.937+0000 7f5031d90740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d2a5b5c10 0x560d2a5b8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:58.948 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.937+0000 7f5031d90740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560d2a50b970 con 0x560d2a5b5c10 2024-04-23T10:27:58.949 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.937+0000 7f5031d90740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560d2a4a84a0 con 0x560d2a5a6e40 2024-04-23T10:27:58.949 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.937+0000 7f5031d90740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560d2a507930 con 0x560d2a5a6520 2024-04-23T10:27:58.949 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.937+0000 7f502d538640 1 --2- 172.21.15.97:0/731436942 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f50180a3dc0 0x7f50180a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:58.949 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.937+0000 7f5025ffb640 1 -- 172.21.15.97:0/731436942 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f501c0f10e0 con 0x7f50281010e0 2024-04-23T10:27:58.949 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.937+0000 7f502d538640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d2a5b5c10 0x560d2a5b8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:58.949 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.937+0000 7f502d538640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d2a5b5c10 0x560d2a5b8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44864/0 (socket says 172.21.15.97:44864) 2024-04-23T10:27:58.949 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.937+0000 7f502d538640 1 -- 172.21.15.97:0/511851728 learned_addr learned my addr 172.21.15.97:0/511851728 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:27:58.949 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.937+0000 7f502cd37640 1 --2- 172.21.15.97:0/511851728 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d2a5a6520 0x560d2a5a6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:58.949 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.937+0000 7f5027fff640 1 --2- 172.21.15.97:0/511851728 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d2a5a6e40 0x560d2a5b56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:58.949 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.938+0000 7f502d538640 1 -- 172.21.15.97:0/511851728 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d2a5a6520 msgr2=0x560d2a5a6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:27:58.949 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.938+0000 7f502d538640 1 --2- 172.21.15.97:0/511851728 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d2a5a6520 0x560d2a5a6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:58.950 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.938+0000 7f502d538640 1 -- 172.21.15.97:0/511851728 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d2a5a6e40 msgr2=0x560d2a5b56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:27:58.950 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.938+0000 7f502d538640 1 --2- 172.21.15.97:0/511851728 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d2a5a6e40 0x560d2a5b56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:58.950 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.938+0000 7f502d538640 1 -- 172.21.15.97:0/511851728 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d2a21e210 con 0x560d2a5b5c10 2024-04-23T10:27:58.950 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.938+0000 7f502cd37640 1 --2- 172.21.15.97:0/511851728 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d2a5a6520 0x560d2a5a6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T10:27:58.950 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.938+0000 7f502d538640 1 --2- 172.21.15.97:0/731436942 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f50180a3dc0 0x7f50180a6280 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5020012be0 tx=0x7f5020009db0 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:27:58.950 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.938+0000 7f502d538640 1 --2- 172.21.15.97:0/511851728 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d2a5b5c10 0x560d2a5b8000 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f5020010800 tx=0x7f502001c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=188a4776da8ae839 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:27:58.950 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.938+0000 7f5012ffd640 1 -- 172.21.15.97:0/511851728 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5020004b20 con 0x560d2a5b5c10 2024-04-23T10:27:58.950 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.938+0000 7f5012ffd640 1 -- 172.21.15.97:0/511851728 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5020004cc0 con 0x560d2a5b5c10 2024-04-23T10:27:58.950 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.938+0000 7f5012ffd640 1 -- 172.21.15.97:0/511851728 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5020005200 con 0x560d2a5b5c10 2024-04-23T10:27:58.950 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.939+0000 7f5031d90740 1 -- 172.21.15.97:0/511851728 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d2a5b5c10 msgr2=0x560d2a5b8000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:58.951 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.939+0000 7f5031d90740 1 --2- 172.21.15.97:0/511851728 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d2a5b5c10 0x560d2a5b8000 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f5020010800 tx=0x7f502001c000 comp rx=0 tx=0).stop 2024-04-23T10:27:58.951 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.939+0000 7f5031d90740 1 -- 172.21.15.97:0/511851728 shutdown_connections 2024-04-23T10:27:58.951 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.939+0000 7f5031d90740 1 --2- 172.21.15.97:0/511851728 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d2a5b5c10 0x560d2a5b8000 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:58.951 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.939+0000 7f5031d90740 1 --2- 172.21.15.97:0/511851728 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d2a5a6e40 0x560d2a5b56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:58.951 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.939+0000 7f5031d90740 1 --2- 172.21.15.97:0/511851728 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d2a5a6520 0x560d2a5a6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:58.951 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.939+0000 7f5031d90740 1 -- 172.21.15.97:0/511851728 >> 172.21.15.97:0/511851728 conn(0x560d2a5a2b30 msgr2=0x560d2a5a2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:27:58.951 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.939+0000 7f5031d90740 1 -- 172.21.15.97:0/511851728 shutdown_connections 2024-04-23T10:27:58.951 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.939+0000 7f5031d90740 1 -- 172.21.15.97:0/511851728 wait complete. 2024-04-23T10:27:58.951 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.939+0000 7f5031d90740 1 -- start start 2024-04-23T10:27:58.951 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.939+0000 7f5031d90740 1 Processor -- start 2024-04-23T10:27:58.952 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.940+0000 7f5031d90740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d2a5a6520 0x560d2a5b5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:58.952 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.940+0000 7f5031d90740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d2a5a6e40 0x560d2a5b2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:58.952 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.940+0000 7f5031d90740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d2a5b5c10 0x560d2a5b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:58.952 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.940+0000 7f5031d90740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560d2a5b87e0 con 0x560d2a5a6e40 2024-04-23T10:27:58.952 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.940+0000 7f5031d90740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560d2a4a60b0 con 0x560d2a5b5c10 2024-04-23T10:27:58.952 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.940+0000 7f5031d90740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560d2a5b8960 con 0x560d2a5a6520 2024-04-23T10:27:58.952 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.940+0000 7f5031d90740 10 client.0 _ll_register_callbacks cb 0x560d2a493d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:27:58.952 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.940+0000 7f5027fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d2a5a6e40 0x560d2a5b2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:58.952 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.940+0000 7f5027fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d2a5a6e40 0x560d2a5b2850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44872/0 (socket says 172.21.15.97:44872) 2024-04-23T10:27:58.953 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.940+0000 7f5027fff640 1 -- 172.21.15.97:0/682669121 learned_addr learned my addr 172.21.15.97:0/682669121 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:27:58.953 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.940+0000 7f502cd37640 1 --2- 172.21.15.97:0/682669121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d2a5a6520 0x560d2a5b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:58.953 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.940+0000 7f502d538640 1 --2- 172.21.15.97:0/682669121 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d2a5b5c10 0x560d2a5b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:58.953 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.940+0000 7f5027fff640 1 -- 172.21.15.97:0/682669121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d2a5a6520 msgr2=0x560d2a5b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:27:58.953 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.940+0000 7f5027fff640 1 --2- 172.21.15.97:0/682669121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d2a5a6520 0x560d2a5b5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:58.953 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.940+0000 7f5027fff640 1 -- 172.21.15.97:0/682669121 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d2a5b5c10 msgr2=0x560d2a5b2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:27:58.953 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.940+0000 7f5027fff640 1 --2- 172.21.15.97:0/682669121 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d2a5b5c10 0x560d2a5b2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:58.953 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.940+0000 7f5027fff640 1 -- 172.21.15.97:0/682669121 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d2a2ea620 con 0x560d2a5a6e40 2024-04-23T10:27:58.953 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.940+0000 7f502cd37640 1 --2- 172.21.15.97:0/682669121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d2a5a6520 0x560d2a5b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:27:58.953 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.940+0000 7f502d538640 1 --2- 172.21.15.97:0/682669121 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d2a5b5c10 0x560d2a5b2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:27:58.953 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.940+0000 7f5027fff640 1 --2- 172.21.15.97:0/682669121 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d2a5a6e40 0x560d2a5b2850 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f501400f4f0 tx=0x7f501400f9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f04dde65cf54441d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:27:58.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.940+0000 7f5007fff640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:27:58.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.940+0000 7f5007fff640 1 -- 172.21.15.97:0/682669121 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5014013850 con 0x560d2a5a6e40 2024-04-23T10:27:58.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.941+0000 7f5007fff640 1 -- 172.21.15.97:0/682669121 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f50140139f0 con 0x560d2a5a6e40 2024-04-23T10:27:58.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.941+0000 7f5031d90740 1 -- 172.21.15.97:0/682669121 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5020004cc0 con 0x560d2a5a6e40 2024-04-23T10:27:58.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.941+0000 7f5007fff640 1 -- 172.21.15.97:0/682669121 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5014013bf0 con 0x560d2a5a6e40 2024-04-23T10:27:58.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.941+0000 7f5031d90740 10 client.15921 fetch_fsmap learned FSMap version 26 2024-04-23T10:27:58.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.941+0000 7f5031d90740 1 -- 172.21.15.97:0/682669121 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560d2a5c4830 con 0x560d2a5a6e40 2024-04-23T10:27:58.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.941+0000 7f5007fff640 1 -- 172.21.15.97:0/682669121 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5020004cc0 con 0x560d2a5a6e40 2024-04-23T10:27:58.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.941+0000 7f5007fff640 1 -- 172.21.15.97:0/682669121 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f50140065e0 con 0x560d2a5a6e40 2024-04-23T10:27:58.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.941+0000 7f5031d90740 10 client.15921 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:27:58.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.941+0000 7f5031d90740 10 client.15921 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:27:58.955 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.941+0000 7f5031d90740 20 client.15921 populate_metadata read hostname 'smithi097' 2024-04-23T10:27:58.955 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.941+0000 7f5031d90740 1 --2- 172.21.15.97:0/682669121 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x560d2a5c5d40 0x560d2a5c8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:58.955 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.941+0000 7f5031d90740 4 client.15921 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:27:58.955 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.941+0000 7f5031d90740 1 -- 172.21.15.97:0/682669121 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d2a519390 con 0x560d2a5c5d40 2024-04-23T10:27:58.955 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.942+0000 7f502cd37640 1 --2- 172.21.15.97:0/682669121 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x560d2a5c5d40 0x560d2a5c8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:58.955 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.942+0000 7f502cd37640 1 --2- 172.21.15.97:0/682669121 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x560d2a5c5d40 0x560d2a5c8150 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1304656ebf6a3af server_cookie=638cc1c4cdd93ea8 in_seq=0 out_seq=0 2024-04-23T10:27:58.955 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.942+0000 7f5007fff640 10 client.15921 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:27:58.955 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.943+0000 7f5007fff640 1 -- 172.21.15.97:0/682669121 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560d2a519390 con 0x560d2a5c5d40 2024-04-23T10:27:58.955 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.943+0000 7f5007fff640 10 client.15921 handle_command_reply: tid=0 2024-04-23T10:27:58.955 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.943+0000 7f5031d90740 1 client.15921 shutdown 2024-04-23T10:27:58.955 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.943+0000 7f5031d90740 1 -- 172.21.15.97:0/682669121 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d2a5a6e40 msgr2=0x560d2a5b2850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:58.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.943+0000 7f5031d90740 1 --2- 172.21.15.97:0/682669121 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d2a5a6e40 0x560d2a5b2850 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f501400f4f0 tx=0x7f501400f9b0 comp rx=0 tx=0).stop 2024-04-23T10:27:58.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.943+0000 7f5031d90740 1 -- 172.21.15.97:0/682669121 shutdown_connections 2024-04-23T10:27:58.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.943+0000 7f5031d90740 1 --2- 172.21.15.97:0/682669121 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x560d2a5c5d40 0x560d2a5c8150 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:58.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.943+0000 7f5031d90740 1 --2- 172.21.15.97:0/682669121 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d2a5b5c10 0x560d2a5b2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:58.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.943+0000 7f5031d90740 1 --2- 172.21.15.97:0/682669121 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d2a5a6e40 0x560d2a5b2850 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:58.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.943+0000 7f5007fff640 0 client.15921 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:27:58.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.943+0000 7f5031d90740 1 --2- 172.21.15.97:0/682669121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d2a5a6520 0x560d2a5b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:58.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.943+0000 7f5031d90740 1 -- 172.21.15.97:0/682669121 >> 172.21.15.97:0/682669121 conn(0x560d2a5a2b30 msgr2=0x560d2a5a89a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:27:58.957 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.943+0000 7f5031d90740 1 -- 172.21.15.97:0/682669121 shutdown_connections 2024-04-23T10:27:58.957 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.943+0000 7f5031d90740 1 -- 172.21.15.97:0/682669121 wait complete. 2024-04-23T10:27:58.957 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.943+0000 7f5031d90740 20 client.15921 trim_cache size 0 max 16384 2024-04-23T10:27:58.964 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.958+0000 7f5031d90740 1 Processor -- start 2024-04-23T10:27:58.965 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.958+0000 7f5031d90740 1 -- start start 2024-04-23T10:27:58.965 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.958+0000 7f5031d90740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d2a5a6520 0x560d2a5c1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:58.965 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.958+0000 7f5031d90740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d2a5a6e40 0x560d2a5b5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:58.965 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.958+0000 7f5031d90740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d2a5b5c10 0x560d2a5cda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:58.965 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.958+0000 7f5031d90740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f50281ab740 con 0x560d2a5b5c10 2024-04-23T10:27:58.965 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.958+0000 7f5031d90740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f50281ad280 con 0x560d2a5a6520 2024-04-23T10:27:58.965 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.958+0000 7f5031d90740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f50281abe10 con 0x560d2a5a6e40 2024-04-23T10:27:58.965 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.958+0000 7f502d538640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d2a5b5c10 0x560d2a5cda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:58.965 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.958+0000 7f502d538640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d2a5b5c10 0x560d2a5cda60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44884/0 (socket says 172.21.15.97:44884) 2024-04-23T10:27:58.965 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.958+0000 7f502d538640 1 -- 172.21.15.97:0/2864939687 learned_addr learned my addr 172.21.15.97:0/2864939687 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:27:58.966 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.958+0000 7f502cd37640 1 --2- 172.21.15.97:0/2864939687 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d2a5a6520 0x560d2a5c1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:58.966 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.958+0000 7f5027fff640 1 --2- 172.21.15.97:0/2864939687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d2a5a6e40 0x560d2a5b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:58.966 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.958+0000 7f502d538640 1 -- 172.21.15.97:0/2864939687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d2a5a6e40 msgr2=0x560d2a5b5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:27:58.966 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.958+0000 7f502d538640 1 --2- 172.21.15.97:0/2864939687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d2a5a6e40 0x560d2a5b5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:58.966 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.958+0000 7f502d538640 1 -- 172.21.15.97:0/2864939687 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d2a5a6520 msgr2=0x560d2a5c1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:27:58.966 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.958+0000 7f502d538640 1 --2- 172.21.15.97:0/2864939687 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d2a5a6520 0x560d2a5c1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:58.966 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.958+0000 7f502d538640 1 -- 172.21.15.97:0/2864939687 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d2a5a6c70 con 0x560d2a5b5c10 2024-04-23T10:27:58.966 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.958+0000 7f502cd37640 1 --2- 172.21.15.97:0/2864939687 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d2a5a6520 0x560d2a5c1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:27:58.966 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.958+0000 7f5027fff640 1 --2- 172.21.15.97:0/2864939687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d2a5a6e40 0x560d2a5b5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:27:58.966 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.958+0000 7f502d538640 1 --2- 172.21.15.97:0/2864939687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d2a5b5c10 0x560d2a5cda60 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f5020000f80 tx=0x7f5020004bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31102869f35aa244 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:27:58.966 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.959+0000 7f50127fc640 1 -- 172.21.15.97:0/2864939687 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f502002d020 con 0x560d2a5b5c10 2024-04-23T10:27:58.967 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.959+0000 7f50127fc640 1 -- 172.21.15.97:0/2864939687 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5020004de0 con 0x560d2a5b5c10 2024-04-23T10:27:58.967 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.959+0000 7f50127fc640 1 -- 172.21.15.97:0/2864939687 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5020005330 con 0x560d2a5b5c10 2024-04-23T10:27:58.967 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.959+0000 7f5031d90740 1 -- 172.21.15.97:0/2864939687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d2a5b5c10 msgr2=0x560d2a5cda60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:58.967 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.959+0000 7f5031d90740 1 --2- 172.21.15.97:0/2864939687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d2a5b5c10 0x560d2a5cda60 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f5020000f80 tx=0x7f5020004bf0 comp rx=0 tx=0).stop 2024-04-23T10:27:58.967 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.959+0000 7f5031d90740 1 -- 172.21.15.97:0/2864939687 shutdown_connections 2024-04-23T10:27:58.967 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.959+0000 7f5031d90740 1 --2- 172.21.15.97:0/2864939687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d2a5b5c10 0x560d2a5cda60 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:58.967 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.959+0000 7f5031d90740 1 --2- 172.21.15.97:0/2864939687 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d2a5a6e40 0x560d2a5b5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:58.967 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.959+0000 7f5031d90740 1 --2- 172.21.15.97:0/2864939687 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d2a5a6520 0x560d2a5c1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:58.967 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.959+0000 7f5031d90740 1 -- 172.21.15.97:0/2864939687 >> 172.21.15.97:0/2864939687 conn(0x560d2a5a2b30 msgr2=0x560d2a5adf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:27:58.967 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.959+0000 7f5031d90740 1 -- 172.21.15.97:0/2864939687 shutdown_connections 2024-04-23T10:27:58.967 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.959+0000 7f5031d90740 1 -- 172.21.15.97:0/2864939687 wait complete. 2024-04-23T10:27:58.968 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.960+0000 7f5031d90740 1 -- start start 2024-04-23T10:27:58.968 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.960+0000 7f5031d90740 1 Processor -- start 2024-04-23T10:27:58.968 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.960+0000 7f5031d90740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d2a5a6520 0x560d2a5af650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:58.968 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.960+0000 7f5031d90740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d2a5a6e40 0x560d2a5afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:58.968 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.960+0000 7f5031d90740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d2a5b5c10 0x560d2a5b00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:58.968 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.960+0000 7f5031d90740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f50281acbb0 con 0x560d2a5a6e40 2024-04-23T10:27:58.968 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.960+0000 7f5031d90740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f50281ac4e0 con 0x560d2a5b5c10 2024-04-23T10:27:58.968 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.960+0000 7f5031d90740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f50281a9740 con 0x560d2a5a6520 2024-04-23T10:27:58.968 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.960+0000 7f5031d90740 10 client.0 _ll_register_callbacks cb 0x7f501400add0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:27:58.968 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.960+0000 7f5027fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d2a5a6e40 0x560d2a5afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:58.968 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.960+0000 7f5027fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d2a5a6e40 0x560d2a5afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:44892/0 (socket says 172.21.15.97:44892) 2024-04-23T10:27:58.968 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.960+0000 7f5027fff640 1 -- 172.21.15.97:0/708541405 learned_addr learned my addr 172.21.15.97:0/708541405 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:27:58.969 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.960+0000 7f502cd37640 1 --2- 172.21.15.97:0/708541405 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d2a5a6520 0x560d2a5af650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:58.969 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.961+0000 7f5027fff640 1 -- 172.21.15.97:0/708541405 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d2a5a6520 msgr2=0x560d2a5af650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:27:58.969 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.961+0000 7f502d538640 1 --2- 172.21.15.97:0/708541405 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d2a5b5c10 0x560d2a5b00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:58.969 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.961+0000 7f5027fff640 1 --2- 172.21.15.97:0/708541405 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d2a5a6520 0x560d2a5af650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:58.969 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.961+0000 7f5027fff640 1 -- 172.21.15.97:0/708541405 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d2a5b5c10 msgr2=0x560d2a5b00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:27:58.969 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.961+0000 7f5027fff640 1 --2- 172.21.15.97:0/708541405 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d2a5b5c10 0x560d2a5b00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:58.969 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.961+0000 7f5027fff640 1 -- 172.21.15.97:0/708541405 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d2a62b430 con 0x560d2a5a6e40 2024-04-23T10:27:58.969 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.961+0000 7f502d538640 1 --2- 172.21.15.97:0/708541405 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d2a5b5c10 0x560d2a5b00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:27:58.969 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.961+0000 7f5027fff640 1 --2- 172.21.15.97:0/708541405 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d2a5a6e40 0x560d2a5afb90 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f501400f4c0 tx=0x7f5014013900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79603602d174ea9d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:27:58.969 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.961+0000 7f5007fff640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:27:58.969 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.961+0000 7f5007fff640 1 -- 172.21.15.97:0/708541405 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f501400ae80 con 0x560d2a5a6e40 2024-04-23T10:27:58.970 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.961+0000 7f5007fff640 1 -- 172.21.15.97:0/708541405 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f501401b400 con 0x560d2a5a6e40 2024-04-23T10:27:58.970 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.961+0000 7f5031d90740 1 -- 172.21.15.97:0/708541405 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5020004de0 con 0x560d2a5a6e40 2024-04-23T10:27:58.970 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.961+0000 7f5007fff640 1 -- 172.21.15.97:0/708541405 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5014024e40 con 0x560d2a5a6e40 2024-04-23T10:27:58.970 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.961+0000 7f5031d90740 10 client.15927 fetch_fsmap learned FSMap version 26 2024-04-23T10:27:58.970 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.961+0000 7f5031d90740 1 -- 172.21.15.97:0/708541405 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560d2a62c6f0 con 0x560d2a5a6e40 2024-04-23T10:27:58.970 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.961+0000 7f5007fff640 1 -- 172.21.15.97:0/708541405 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5020004de0 con 0x560d2a5a6e40 2024-04-23T10:27:58.970 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.961+0000 7f5007fff640 1 -- 172.21.15.97:0/708541405 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5014036440 con 0x560d2a5a6e40 2024-04-23T10:27:58.970 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.961+0000 7f5031d90740 10 client.15927 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:27:58.970 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.961+0000 7f5031d90740 10 client.15927 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:27:58.970 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.961+0000 7f5031d90740 20 client.15927 populate_metadata read hostname 'smithi097' 2024-04-23T10:27:58.970 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.961+0000 7f5031d90740 1 --2- 172.21.15.97:0/708541405 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x560d2a62cec0 0x560d2a62f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:27:58.971 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.961+0000 7f5031d90740 4 client.15927 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:27:58.971 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.961+0000 7f5031d90740 1 -- 172.21.15.97:0/708541405 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560d2a513200 con 0x560d2a62cec0 2024-04-23T10:27:58.971 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.962+0000 7f502cd37640 1 --2- 172.21.15.97:0/708541405 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x560d2a62cec0 0x560d2a62f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:27:58.971 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.963+0000 7f5007fff640 10 client.15927 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:27:58.971 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.963+0000 7f502cd37640 1 --2- 172.21.15.97:0/708541405 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x560d2a62cec0 0x560d2a62f2e0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1de9e962d80b7ff4 server_cookie=64fb60acd323e730 in_seq=0 out_seq=0 2024-04-23T10:27:58.976 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.976+0000 7f5007fff640 1 -- 172.21.15.97:0/708541405 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560d2a513200 con 0x560d2a62cec0 2024-04-23T10:27:58.976 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.976+0000 7f5007fff640 10 client.15927 handle_command_reply: tid=0 2024-04-23T10:27:58.976 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.976+0000 7f5031d90740 1 client.15927 shutdown 2024-04-23T10:27:58.976 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.976+0000 7f5031d90740 1 -- 172.21.15.97:0/708541405 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d2a5a6e40 msgr2=0x560d2a5afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:58.977 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.976+0000 7f5031d90740 1 --2- 172.21.15.97:0/708541405 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d2a5a6e40 0x560d2a5afb90 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f501400f4c0 tx=0x7f5014013900 comp rx=0 tx=0).stop 2024-04-23T10:27:58.977 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.976+0000 7f5031d90740 1 -- 172.21.15.97:0/708541405 shutdown_connections 2024-04-23T10:27:58.977 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.976+0000 7f5031d90740 1 --2- 172.21.15.97:0/708541405 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x560d2a62cec0 0x560d2a62f2e0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:58.977 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.976+0000 7f5031d90740 1 --2- 172.21.15.97:0/708541405 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560d2a5b5c10 0x560d2a5b00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:58.977 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.976+0000 7f5031d90740 1 --2- 172.21.15.97:0/708541405 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560d2a5a6e40 0x560d2a5afb90 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:58.977 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.976+0000 7f5007fff640 0 client.15927 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:27:58.977 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.976+0000 7f5031d90740 1 --2- 172.21.15.97:0/708541405 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d2a5a6520 0x560d2a5af650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:58.977 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.976+0000 7f5031d90740 1 -- 172.21.15.97:0/708541405 >> 172.21.15.97:0/708541405 conn(0x560d2a5a2b30 msgr2=0x560d2a5a1420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:27:58.977 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.977+0000 7f5031d90740 1 -- 172.21.15.97:0/708541405 shutdown_connections 2024-04-23T10:27:58.977 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.977+0000 7f5031d90740 1 -- 172.21.15.97:0/708541405 wait complete. 2024-04-23T10:27:58.977 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.977+0000 7f5031d90740 20 client.15927 trim_cache size 0 max 16384 2024-04-23T10:27:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:27:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:27:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:27:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:27:58.979 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.977+0000 7f502ef9b640 1 -- 172.21.15.97:0/731436942 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f50180a3dc0 msgr2=0x7f50180a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:58.979 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.977+0000 7f502ef9b640 1 --2- 172.21.15.97:0/731436942 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f50180a3dc0 0x7f50180a6280 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5020012be0 tx=0x7f5020009db0 comp rx=0 tx=0).stop 2024-04-23T10:27:58.979 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.977+0000 7f502ef9b640 1 -- 172.21.15.97:0/731436942 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50281010e0 msgr2=0x7f5028198760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:27:58.979 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.977+0000 7f502ef9b640 1 --2- 172.21.15.97:0/731436942 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50281010e0 0x7f5028198760 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f501c005940 tx=0x7f501c0042f0 comp rx=0 tx=0).stop 2024-04-23T10:27:58.979 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.977+0000 7f502ef9b640 1 -- 172.21.15.97:0/731436942 shutdown_connections 2024-04-23T10:27:58.979 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.977+0000 7f502ef9b640 1 --2- 172.21.15.97:0/731436942 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f50180a3dc0 0x7f50180a6280 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:58.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.977+0000 7f502ef9b640 1 --2- 172.21.15.97:0/731436942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f502819d020 0x7f502819d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:58.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.977+0000 7f502ef9b640 1 --2- 172.21.15.97:0/731436942 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50281061b0 0x7f5028198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:58.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.977+0000 7f502ef9b640 1 --2- 172.21.15.97:0/731436942 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50281010e0 0x7f5028198760 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:27:58.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.977+0000 7f502ef9b640 1 -- 172.21.15.97:0/731436942 >> 172.21.15.97:0/731436942 conn(0x7f50280771d0 msgr2=0x7f5028104e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:27:58.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.977+0000 7f502ef9b640 1 -- 172.21.15.97:0/731436942 shutdown_connections 2024-04-23T10:27:58.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:27:58.977+0000 7f502ef9b640 1 -- 172.21.15.97:0/731436942 wait complete. 2024-04-23T10:28:00.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:28:00 smithi196 ceph-mon[32919]: pgmap v57: 97 pgs: 97 active+clean; 1.8 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-23T10:28:00.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:28:00 smithi097 ceph-mon[29197]: pgmap v57: 97 pgs: 97 active+clean; 1.8 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-23T10:28:00.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:28:00 smithi183 ceph-mon[32317]: pgmap v57: 97 pgs: 97 active+clean; 1.8 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-23T10:28:02.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:28:02 smithi196 ceph-mon[32919]: pgmap v58: 97 pgs: 97 active+clean; 1.8 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-23T10:28:02.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:28:02 smithi097 ceph-mon[29197]: pgmap v58: 97 pgs: 97 active+clean; 1.8 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-23T10:28:02.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:28:02 smithi183 ceph-mon[32317]: pgmap v58: 97 pgs: 97 active+clean; 1.8 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 13 op/s 2024-04-23T10:28:04.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:28:04 smithi196 ceph-mon[32919]: pgmap v59: 97 pgs: 97 active+clean; 1.9 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 47 KiB/s wr, 36 op/s 2024-04-23T10:28:04.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:28:04 smithi097 ceph-mon[29197]: pgmap v59: 97 pgs: 97 active+clean; 1.9 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 47 KiB/s wr, 36 op/s 2024-04-23T10:28:04.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:28:04 smithi183 ceph-mon[32317]: pgmap v59: 97 pgs: 97 active+clean; 1.9 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 47 KiB/s wr, 36 op/s 2024-04-23T10:28:07.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:28:07 smithi196 ceph-mon[32919]: pgmap v60: 97 pgs: 97 active+clean; 1.9 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 40 KiB/s wr, 30 op/s 2024-04-23T10:28:07.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:28:07 smithi097 ceph-mon[29197]: pgmap v60: 97 pgs: 97 active+clean; 1.9 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 40 KiB/s wr, 30 op/s 2024-04-23T10:28:07.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:28:07 smithi183 ceph-mon[32317]: pgmap v60: 97 pgs: 97 active+clean; 1.9 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 40 KiB/s wr, 30 op/s 2024-04-23T10:28:09.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:28:09 smithi196 ceph-mon[32919]: pgmap v61: 97 pgs: 97 active+clean; 2.0 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 49 KiB/s wr, 37 op/s 2024-04-23T10:28:09.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:28:09 smithi097 ceph-mon[29197]: pgmap v61: 97 pgs: 97 active+clean; 2.0 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 49 KiB/s wr, 37 op/s 2024-04-23T10:28:09.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:28:09 smithi183 ceph-mon[32317]: pgmap v61: 97 pgs: 97 active+clean; 2.0 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 49 KiB/s wr, 37 op/s 2024-04-23T10:28:11.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:28:11 smithi196 ceph-mon[32919]: pgmap v62: 97 pgs: 97 active+clean; 2.0 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 42 KiB/s wr, 30 op/s 2024-04-23T10:28:11.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:28:11 smithi097 ceph-mon[29197]: pgmap v62: 97 pgs: 97 active+clean; 2.0 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 42 KiB/s wr, 30 op/s 2024-04-23T10:28:11.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:28:11 smithi183 ceph-mon[32317]: pgmap v62: 97 pgs: 97 active+clean; 2.0 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 42 KiB/s wr, 30 op/s 2024-04-23T10:28:12.076 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.075+0000 7fccf43e5640 1 -- 172.21.15.97:0/3761151329 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fccec1005e0 msgr2=0x7fccec100a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:28:12.076 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.075+0000 7fccf43e5640 1 --2- 172.21.15.97:0/3761151329 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fccec1005e0 0x7fccec100a60 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fccdc009900 tx=0x7fccdc02a410 comp rx=0 tx=0).stop 2024-04-23T10:28:12.077 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.075+0000 7fccf43e5640 1 -- 172.21.15.97:0/3761151329 shutdown_connections 2024-04-23T10:28:12.077 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.075+0000 7fccf43e5640 1 --2- 172.21.15.97:0/3761151329 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fccec100fa0 0x7fccec10d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:12.077 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.075+0000 7fccf43e5640 1 --2- 172.21.15.97:0/3761151329 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fccec1005e0 0x7fccec100a60 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:12.077 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.075+0000 7fccf43e5640 1 --2- 172.21.15.97:0/3761151329 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccec106680 0x7fccec106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:12.077 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.075+0000 7fccf43e5640 1 -- 172.21.15.97:0/3761151329 >> 172.21.15.97:0/3761151329 conn(0x7fccec077390 msgr2=0x7fccec0fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:28:12.077 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.075+0000 7fccf43e5640 1 -- 172.21.15.97:0/3761151329 shutdown_connections 2024-04-23T10:28:12.077 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.075+0000 7fccf43e5640 1 -- 172.21.15.97:0/3761151329 wait complete. 2024-04-23T10:28:12.077 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.076+0000 7fccf43e5640 1 Processor -- start 2024-04-23T10:28:12.077 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.076+0000 7fccf43e5640 1 -- start start 2024-04-23T10:28:12.077 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.076+0000 7fccf43e5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccec1005e0 0x7fccec198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:12.077 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.076+0000 7fccf43e5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fccec100fa0 0x7fccec198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:12.077 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.076+0000 7fccf43e5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fccec106680 0x7fccec19cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:12.077 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.076+0000 7fccf43e5640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fccec112250 con 0x7fccec100fa0 2024-04-23T10:28:12.077 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.076+0000 7fccf43e5640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fccec1120d0 con 0x7fccec106680 2024-04-23T10:28:12.078 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.076+0000 7fccf43e5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fccec1123d0 con 0x7fccec1005e0 2024-04-23T10:28:12.078 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.076+0000 7fccf1980640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fccec100fa0 0x7fccec198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:12.078 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.076+0000 7fccf1980640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fccec100fa0 0x7fccec198b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37140/0 (socket says 172.21.15.97:37140) 2024-04-23T10:28:12.078 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.076+0000 7fccf1980640 1 -- 172.21.15.97:0/1468099852 learned_addr learned my addr 172.21.15.97:0/1468099852 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:28:12.078 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.076+0000 7fccf2181640 1 --2- 172.21.15.97:0/1468099852 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccec1005e0 0x7fccec198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:12.078 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.076+0000 7fccf1980640 1 -- 172.21.15.97:0/1468099852 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccec1005e0 msgr2=0x7fccec198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:28:12.078 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.076+0000 7fccf1980640 1 --2- 172.21.15.97:0/1468099852 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccec1005e0 0x7fccec198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:12.078 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.076+0000 7fccf1980640 1 -- 172.21.15.97:0/1468099852 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fccec106680 msgr2=0x7fccec19cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:28:12.078 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.076+0000 7fccf1980640 1 --2- 172.21.15.97:0/1468099852 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fccec106680 0x7fccec19cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:12.078 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.076+0000 7fccf1980640 1 -- 172.21.15.97:0/1468099852 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccec19d640 con 0x7fccec100fa0 2024-04-23T10:28:12.078 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.077+0000 7fccf1980640 1 --2- 172.21.15.97:0/1468099852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fccec100fa0 0x7fccec198b90 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fccdc005940 tx=0x7fccdc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:28:12.078 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.077+0000 7fcce37fe640 1 -- 172.21.15.97:0/1468099852 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccdc03f030 con 0x7fccec100fa0 2024-04-23T10:28:12.078 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.077+0000 7fcce37fe640 1 -- 172.21.15.97:0/1468099852 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccdc003cb0 con 0x7fccec100fa0 2024-04-23T10:28:12.078 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.077+0000 7fccf43e5640 1 -- 172.21.15.97:0/1468099852 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccec19d8d0 con 0x7fccec100fa0 2024-04-23T10:28:12.078 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.077+0000 7fcce37fe640 1 -- 172.21.15.97:0/1468099852 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccdc035070 con 0x7fccec100fa0 2024-04-23T10:28:12.079 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.077+0000 7fccf43e5640 1 -- 172.21.15.97:0/1468099852 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccec1a51c0 con 0x7fccec100fa0 2024-04-23T10:28:12.086 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.079+0000 7fcce37fe640 1 -- 172.21.15.97:0/1468099852 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fccdc042030 con 0x7fccec100fa0 2024-04-23T10:28:12.086 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.080+0000 7fccf71da740 1 Processor -- start 2024-04-23T10:28:12.086 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.080+0000 7fccf71da740 1 -- start start 2024-04-23T10:28:12.086 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.080+0000 7fccf71da740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5558834d6620 0x5558834d6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:12.086 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.080+0000 7fccf71da740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5558834d6f40 0x5558834e12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:12.086 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.081+0000 7fccf71da740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5558834e1800 0x5558834e3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:12.086 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.081+0000 7fccf71da740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55588343b970 con 0x5558834e1800 2024-04-23T10:28:12.086 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.081+0000 7fccf71da740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5558833d84a0 con 0x5558834d6620 2024-04-23T10:28:12.086 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.081+0000 7fccf71da740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555883437930 con 0x5558834d6f40 2024-04-23T10:28:12.086 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.081+0000 7fcce37fe640 1 --2- 172.21.15.97:0/1468099852 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fccd80a3dc0 0x7fccd80a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:12.087 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.081+0000 7fccf1980640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5558834e1800 0x5558834e3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:12.087 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.081+0000 7fccf2181640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5558834d6f40 0x5558834e12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:12.087 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.081+0000 7fccf2982640 1 --2- 172.21.15.97:0/1468099852 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fccd80a3dc0 0x7fccd80a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:12.087 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.081+0000 7fccf2181640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5558834d6f40 0x5558834e12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.97:43054/0 (socket says 172.21.15.97:43054) 2024-04-23T10:28:12.087 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.081+0000 7fccf1980640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5558834e1800 0x5558834e3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37150/0 (socket says 172.21.15.97:37150) 2024-04-23T10:28:12.087 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.081+0000 7fccf2181640 1 -- 172.21.15.97:0/1513268263 learned_addr learned my addr 172.21.15.97:0/1513268263 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:28:12.087 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.081+0000 7fccf2982640 1 --2- 172.21.15.97:0/1513268263 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5558834d6620 0x5558834d6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:12.087 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.081+0000 7fcce37fe640 1 -- 172.21.15.97:0/1468099852 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fccdc0f1160 con 0x7fccec100fa0 2024-04-23T10:28:12.087 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.081+0000 7fccf2181640 1 -- 172.21.15.97:0/1513268263 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5558834d6620 msgr2=0x5558834d6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:28:12.087 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.081+0000 7fccf2181640 1 --2- 172.21.15.97:0/1513268263 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5558834d6620 0x5558834d6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:12.087 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.081+0000 7fccf2181640 1 -- 172.21.15.97:0/1513268263 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5558834e1800 msgr2=0x5558834e3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:28:12.087 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.081+0000 7fccf2181640 1 --2- 172.21.15.97:0/1513268263 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5558834e1800 0x5558834e3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:12.087 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.081+0000 7fccf2181640 1 -- 172.21.15.97:0/1513268263 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55588314e210 con 0x5558834d6f40 2024-04-23T10:28:12.087 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.081+0000 7fccf2982640 1 --2- 172.21.15.97:0/1468099852 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fccd80a3dc0 0x7fccd80a6280 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fcce4009270 tx=0x7fcce40109a0 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:28:12.087 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.081+0000 7fccf2181640 1 --2- 172.21.15.97:0/1513268263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5558834d6f40 0x5558834e12c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcce800bc80 tx=0x7fcce800f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e9e4959594a616e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:28:12.088 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.082+0000 7fcce0bfe640 1 -- 172.21.15.97:0/1513268263 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce8015070 con 0x5558834d6f40 2024-04-23T10:28:12.088 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.082+0000 7fcce0bfe640 1 -- 172.21.15.97:0/1513268263 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcce801a910 con 0x5558834d6f40 2024-04-23T10:28:12.088 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.082+0000 7fcce0bfe640 1 -- 172.21.15.97:0/1513268263 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce8011830 con 0x5558834d6f40 2024-04-23T10:28:12.088 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.082+0000 7fccf71da740 1 -- 172.21.15.97:0/1513268263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5558834d6f40 msgr2=0x5558834e12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:28:12.088 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.082+0000 7fccf71da740 1 --2- 172.21.15.97:0/1513268263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5558834d6f40 0x5558834e12c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcce800bc80 tx=0x7fcce800f730 comp rx=0 tx=0).stop 2024-04-23T10:28:12.088 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.082+0000 7fccf71da740 1 -- 172.21.15.97:0/1513268263 shutdown_connections 2024-04-23T10:28:12.088 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.082+0000 7fccf71da740 1 --2- 172.21.15.97:0/1513268263 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5558834e1800 0x5558834e3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:12.088 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.082+0000 7fccf71da740 1 --2- 172.21.15.97:0/1513268263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5558834d6f40 0x5558834e12c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:12.088 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.082+0000 7fccf71da740 1 --2- 172.21.15.97:0/1513268263 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5558834d6620 0x5558834d6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:12.088 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.082+0000 7fccf71da740 1 -- 172.21.15.97:0/1513268263 >> 172.21.15.97:0/1513268263 conn(0x5558834d2b30 msgr2=0x5558834d2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:28:12.088 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.082+0000 7fccf71da740 1 -- 172.21.15.97:0/1513268263 shutdown_connections 2024-04-23T10:28:12.088 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.083+0000 7fccf71da740 1 -- 172.21.15.97:0/1513268263 wait complete. 2024-04-23T10:28:12.088 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.083+0000 7fccf71da740 1 -- start start 2024-04-23T10:28:12.088 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.083+0000 7fccf71da740 1 Processor -- start 2024-04-23T10:28:12.088 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.083+0000 7fccf71da740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5558834d6620 0x5558834e11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:12.089 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.083+0000 7fccf71da740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5558834d6f40 0x5558834de4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:12.089 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.083+0000 7fccf71da740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5558834e1800 0x5558834dea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:12.089 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.083+0000 7fccf71da740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5558834e4430 con 0x5558834d6620 2024-04-23T10:28:12.089 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.083+0000 7fccf71da740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5558833d60b0 con 0x5558834e1800 2024-04-23T10:28:12.089 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.083+0000 7fccf71da740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5558834e45b0 con 0x5558834d6f40 2024-04-23T10:28:12.089 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.083+0000 7fccf71da740 10 client.0 _ll_register_callbacks cb 0x5558833c3d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:28:12.089 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.083+0000 7fccf2181640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5558834d6620 0x5558834e11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:12.089 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.083+0000 7fccf2181640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5558834d6620 0x5558834e11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37158/0 (socket says 172.21.15.97:37158) 2024-04-23T10:28:12.089 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.083+0000 7fccf2181640 1 -- 172.21.15.97:0/3979628166 learned_addr learned my addr 172.21.15.97:0/3979628166 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:28:12.089 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.083+0000 7fccf1980640 1 --2- 172.21.15.97:0/3979628166 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5558834d6f40 0x5558834de4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:12.089 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.084+0000 7fccf2982640 1 --2- 172.21.15.97:0/3979628166 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5558834e1800 0x5558834dea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:12.089 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.084+0000 7fccf2181640 1 -- 172.21.15.97:0/3979628166 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5558834d6f40 msgr2=0x5558834de4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:28:12.089 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.084+0000 7fccf2181640 1 --2- 172.21.15.97:0/3979628166 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5558834d6f40 0x5558834de4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:12.089 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.084+0000 7fccf2181640 1 -- 172.21.15.97:0/3979628166 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5558834e1800 msgr2=0x5558834dea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:28:12.089 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.084+0000 7fccf2181640 1 --2- 172.21.15.97:0/3979628166 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5558834e1800 0x5558834dea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:12.090 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.084+0000 7fccf2181640 1 -- 172.21.15.97:0/3979628166 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55588321a620 con 0x5558834d6620 2024-04-23T10:28:12.090 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.084+0000 7fccf1980640 1 --2- 172.21.15.97:0/3979628166 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5558834d6f40 0x5558834de4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T10:28:12.090 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.084+0000 7fccf2181640 1 --2- 172.21.15.97:0/3979628166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5558834d6620 0x5558834e11b0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fcce8006010 tx=0x7fcce800bfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfec1420888e2ab8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:28:12.090 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.084+0000 7fccd5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:28:12.090 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.084+0000 7fccd5ffb640 1 -- 172.21.15.97:0/3979628166 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce8015070 con 0x5558834d6620 2024-04-23T10:28:12.090 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.084+0000 7fccd5ffb640 1 -- 172.21.15.97:0/3979628166 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fcce801aac0 con 0x5558834d6620 2024-04-23T10:28:12.090 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.084+0000 7fccf71da740 1 -- 172.21.15.97:0/3979628166 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcce801a910 con 0x5558834d6620 2024-04-23T10:28:12.090 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.084+0000 7fccd5ffb640 1 -- 172.21.15.97:0/3979628166 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcce801acc0 con 0x5558834d6620 2024-04-23T10:28:12.090 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.084+0000 7fccf71da740 10 client.15939 fetch_fsmap learned FSMap version 26 2024-04-23T10:28:12.090 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.084+0000 7fccf71da740 1 -- 172.21.15.97:0/3979628166 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5558834f03a0 con 0x5558834d6620 2024-04-23T10:28:12.090 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.084+0000 7fccd5ffb640 1 -- 172.21.15.97:0/3979628166 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcce801a910 con 0x5558834d6620 2024-04-23T10:28:12.090 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.084+0000 7fccd5ffb640 1 -- 172.21.15.97:0/3979628166 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fcce80049e0 con 0x5558834d6620 2024-04-23T10:28:12.090 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.084+0000 7fccf71da740 10 client.15939 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:28:12.090 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.084+0000 7fccf71da740 10 client.15939 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:28:12.091 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.085+0000 7fccf71da740 20 client.15939 populate_metadata read hostname 'smithi097' 2024-04-23T10:28:12.091 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.085+0000 7fccf71da740 1 --2- 172.21.15.97:0/3979628166 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5558834f17a0 0x5558834f3b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:12.091 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.085+0000 7fccf71da740 4 client.15939 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:28:12.091 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.085+0000 7fccf71da740 1 -- 172.21.15.97:0/3979628166 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555883449390 con 0x5558834f17a0 2024-04-23T10:28:12.091 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.085+0000 7fccf1980640 1 --2- 172.21.15.97:0/3979628166 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5558834f17a0 0x5558834f3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:12.091 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.085+0000 7fccf1980640 1 --2- 172.21.15.97:0/3979628166 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5558834f17a0 0x5558834f3b90 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edba3b43042beec server_cookie=4770f89975dfb8c0 in_seq=0 out_seq=0 2024-04-23T10:28:12.091 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.085+0000 7fccd5ffb640 10 client.15939 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:28:12.091 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.086+0000 7fccd5ffb640 1 -- 172.21.15.97:0/3979628166 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555883449390 con 0x5558834f17a0 2024-04-23T10:28:12.091 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.086+0000 7fccd5ffb640 10 client.15939 handle_command_reply: tid=0 2024-04-23T10:28:12.091 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.086+0000 7fccf71da740 1 client.15939 shutdown 2024-04-23T10:28:12.091 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.086+0000 7fccf71da740 1 -- 172.21.15.97:0/3979628166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5558834d6620 msgr2=0x5558834e11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:28:12.091 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.086+0000 7fccf71da740 1 --2- 172.21.15.97:0/3979628166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5558834d6620 0x5558834e11b0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fcce8006010 tx=0x7fcce800bfa0 comp rx=0 tx=0).stop 2024-04-23T10:28:12.091 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.086+0000 7fccf71da740 1 -- 172.21.15.97:0/3979628166 shutdown_connections 2024-04-23T10:28:12.091 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.086+0000 7fccf71da740 1 --2- 172.21.15.97:0/3979628166 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5558834f17a0 0x5558834f3b90 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.086+0000 7fccf71da740 1 --2- 172.21.15.97:0/3979628166 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5558834e1800 0x5558834dea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.086+0000 7fccf71da740 1 --2- 172.21.15.97:0/3979628166 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5558834d6f40 0x5558834de4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.086+0000 7fccd5ffb640 0 client.15939 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:28:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.086+0000 7fccf71da740 1 --2- 172.21.15.97:0/3979628166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5558834d6620 0x5558834e11b0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.086+0000 7fccf71da740 1 -- 172.21.15.97:0/3979628166 >> 172.21.15.97:0/3979628166 conn(0x5558834d2b30 msgr2=0x5558834d8860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:28:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.086+0000 7fccf71da740 1 -- 172.21.15.97:0/3979628166 shutdown_connections 2024-04-23T10:28:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.086+0000 7fccf71da740 1 -- 172.21.15.97:0/3979628166 wait complete. 2024-04-23T10:28:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.086+0000 7fccf71da740 20 client.15939 trim_cache size 0 max 16384 2024-04-23T10:28:12.105 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.101+0000 7fccf71da740 1 Processor -- start 2024-04-23T10:28:12.105 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.101+0000 7fccf71da740 1 -- start start 2024-04-23T10:28:12.105 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.101+0000 7fccf71da740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5558834d6620 0x5558834ed2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:12.105 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.101+0000 7fccf71da740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5558834d6f40 0x5558834d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:12.105 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.101+0000 7fccf71da740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5558834e1800 0x5558834e0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:12.105 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.101+0000 7fccf71da740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fccec1ab7a0 con 0x5558834d6620 2024-04-23T10:28:12.105 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.101+0000 7fccf71da740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fccec1ad2e0 con 0x5558834e1800 2024-04-23T10:28:12.105 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.101+0000 7fccf71da740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fccec1abe70 con 0x5558834d6f40 2024-04-23T10:28:12.105 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.101+0000 7fccf2181640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5558834d6620 0x5558834ed2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.102+0000 7fccf2181640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5558834d6620 0x5558834ed2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37174/0 (socket says 172.21.15.97:37174) 2024-04-23T10:28:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.102+0000 7fccf2181640 1 -- 172.21.15.97:0/865869241 learned_addr learned my addr 172.21.15.97:0/865869241 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:28:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.102+0000 7fccf2181640 1 -- 172.21.15.97:0/865869241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5558834d6f40 msgr2=0x5558834d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:28:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.102+0000 7fccf1980640 1 --2- 172.21.15.97:0/865869241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5558834d6f40 0x5558834d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.102+0000 7fccf2181640 1 --2- 172.21.15.97:0/865869241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5558834d6f40 0x5558834d1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.102+0000 7fccf2982640 1 --2- 172.21.15.97:0/865869241 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5558834e1800 0x5558834e0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.102+0000 7fccf2181640 1 -- 172.21.15.97:0/865869241 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5558834e1800 msgr2=0x5558834e0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:28:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.102+0000 7fccf2181640 1 --2- 172.21.15.97:0/865869241 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5558834e1800 0x5558834e0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.102+0000 7fccf2181640 1 -- 172.21.15.97:0/865869241 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558834ede20 con 0x5558834d6620 2024-04-23T10:28:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.102+0000 7fccf2181640 1 --2- 172.21.15.97:0/865869241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5558834d6620 0x5558834ed2f0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fcce8004dd0 tx=0x7fcce800b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b83bbe13d3accb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:28:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.102+0000 7fccd7fff640 1 -- 172.21.15.97:0/865869241 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce8031070 con 0x5558834d6620 2024-04-23T10:28:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.102+0000 7fccd7fff640 1 -- 172.21.15.97:0/865869241 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcce8011600 con 0x5558834d6620 2024-04-23T10:28:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.102+0000 7fccd7fff640 1 -- 172.21.15.97:0/865869241 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce801ae60 con 0x5558834d6620 2024-04-23T10:28:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.103+0000 7fccf71da740 1 -- 172.21.15.97:0/865869241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5558834d6620 msgr2=0x5558834ed2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:28:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.103+0000 7fccf71da740 1 --2- 172.21.15.97:0/865869241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5558834d6620 0x5558834ed2f0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fcce8004dd0 tx=0x7fcce800b410 comp rx=0 tx=0).stop 2024-04-23T10:28:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.103+0000 7fccf71da740 1 -- 172.21.15.97:0/865869241 shutdown_connections 2024-04-23T10:28:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.103+0000 7fccf71da740 1 --2- 172.21.15.97:0/865869241 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5558834e1800 0x5558834e0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.103+0000 7fccf71da740 1 --2- 172.21.15.97:0/865869241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5558834d6f40 0x5558834d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.103+0000 7fccf71da740 1 --2- 172.21.15.97:0/865869241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5558834d6620 0x5558834ed2f0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.103+0000 7fccf71da740 1 -- 172.21.15.97:0/865869241 >> 172.21.15.97:0/865869241 conn(0x5558834d2b30 msgr2=0x5558834d1420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:28:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.103+0000 7fccf71da740 1 -- 172.21.15.97:0/865869241 shutdown_connections 2024-04-23T10:28:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.103+0000 7fccf71da740 1 -- 172.21.15.97:0/865869241 wait complete. 2024-04-23T10:28:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.103+0000 7fccf71da740 1 -- start start 2024-04-23T10:28:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.103+0000 7fccf71da740 1 Processor -- start 2024-04-23T10:28:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.103+0000 7fccf71da740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5558834d6620 0x5558834e7510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.103+0000 7fccf71da740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5558834d6f40 0x5558834e7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.103+0000 7fccf71da740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5558834e1800 0x5558834e7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.103+0000 7fccf71da740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fccec1acc10 con 0x5558834d6620 2024-04-23T10:28:12.108 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.103+0000 7fccf71da740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fccec1ac540 con 0x5558834d6f40 2024-04-23T10:28:12.108 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.103+0000 7fccf71da740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fccec1a9780 con 0x5558834e1800 2024-04-23T10:28:12.108 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.103+0000 7fccf71da740 10 client.0 _ll_register_callbacks cb 0x7fcce800b900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:28:12.108 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.103+0000 7fccf2181640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5558834d6620 0x5558834e7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:12.108 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.104+0000 7fccf2181640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5558834d6620 0x5558834e7510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37184/0 (socket says 172.21.15.97:37184) 2024-04-23T10:28:12.108 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.104+0000 7fccf2181640 1 -- 172.21.15.97:0/624793072 learned_addr learned my addr 172.21.15.97:0/624793072 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:28:12.108 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.104+0000 7fccf2181640 1 -- 172.21.15.97:0/624793072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5558834e1800 msgr2=0x5558834e7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:28:12.108 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.104+0000 7fccf2982640 1 --2- 172.21.15.97:0/624793072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5558834e1800 0x5558834e7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:12.108 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.104+0000 7fccf2181640 1 --2- 172.21.15.97:0/624793072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5558834e1800 0x5558834e7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:12.108 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.104+0000 7fccf1980640 1 --2- 172.21.15.97:0/624793072 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5558834d6f40 0x5558834e7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:12.108 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.104+0000 7fccf2181640 1 -- 172.21.15.97:0/624793072 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5558834d6f40 msgr2=0x5558834e7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:28:12.108 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.104+0000 7fccf2181640 1 --2- 172.21.15.97:0/624793072 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5558834d6f40 0x5558834e7a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:12.108 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.104+0000 7fccf2181640 1 -- 172.21.15.97:0/624793072 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555883556e50 con 0x5558834d6620 2024-04-23T10:28:12.108 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.104+0000 7fccf2181640 1 --2- 172.21.15.97:0/624793072 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5558834d6620 0x5558834e7510 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fcce800bf80 tx=0x7fcce800b6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=436c1c054f97f433 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:28:12.108 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.104+0000 7fccd5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:28:12.109 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.104+0000 7fccd5ffb640 1 -- 172.21.15.97:0/624793072 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce8039070 con 0x5558834d6620 2024-04-23T10:28:12.109 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.104+0000 7fccd5ffb640 1 -- 172.21.15.97:0/624793072 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fcce8004890 con 0x5558834d6620 2024-04-23T10:28:12.109 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.104+0000 7fccf71da740 1 -- 172.21.15.97:0/624793072 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcce8011600 con 0x5558834d6620 2024-04-23T10:28:12.109 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.104+0000 7fccd5ffb640 1 -- 172.21.15.97:0/624793072 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcce8045050 con 0x5558834d6620 2024-04-23T10:28:12.109 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.104+0000 7fccf71da740 10 client.15945 fetch_fsmap learned FSMap version 26 2024-04-23T10:28:12.109 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.104+0000 7fccf71da740 1 -- 172.21.15.97:0/624793072 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555883558160 con 0x5558834d6620 2024-04-23T10:28:12.109 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.104+0000 7fccd5ffb640 1 -- 172.21.15.97:0/624793072 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcce8011600 con 0x5558834d6620 2024-04-23T10:28:12.109 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.105+0000 7fccd5ffb640 1 -- 172.21.15.97:0/624793072 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fcce80117b0 con 0x5558834d6620 2024-04-23T10:28:12.109 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.105+0000 7fccf71da740 10 client.15945 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:28:12.109 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.105+0000 7fccf71da740 10 client.15945 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:28:12.109 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.105+0000 7fccf71da740 20 client.15945 populate_metadata read hostname 'smithi097' 2024-04-23T10:28:12.109 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.105+0000 7fccf71da740 1 --2- 172.21.15.97:0/624793072 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x555883558810 0x55588355ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:12.109 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.105+0000 7fccf71da740 4 client.15945 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:28:12.109 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.105+0000 7fccf71da740 1 -- 172.21.15.97:0/624793072 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555883443200 con 0x555883558810 2024-04-23T10:28:12.109 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.105+0000 7fccf1980640 1 --2- 172.21.15.97:0/624793072 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x555883558810 0x55588355ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:12.110 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.105+0000 7fccf1980640 1 --2- 172.21.15.97:0/624793072 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x555883558810 0x55588355ac30 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab6c96d71e965c2d server_cookie=f2eef439800fd072 in_seq=0 out_seq=0 2024-04-23T10:28:12.110 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.105+0000 7fccd5ffb640 10 client.15945 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:28:12.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.119+0000 7fccd5ffb640 1 -- 172.21.15.97:0/624793072 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555883443200 con 0x555883558810 2024-04-23T10:28:12.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.119+0000 7fccd5ffb640 10 client.15945 handle_command_reply: tid=0 2024-04-23T10:28:12.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.119+0000 7fccf71da740 1 client.15945 shutdown 2024-04-23T10:28:12.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.120+0000 7fccf71da740 1 -- 172.21.15.97:0/624793072 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5558834d6620 msgr2=0x5558834e7510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:28:12.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.120+0000 7fccf71da740 1 --2- 172.21.15.97:0/624793072 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5558834d6620 0x5558834e7510 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fcce800bf80 tx=0x7fcce800b6d0 comp rx=0 tx=0).stop 2024-04-23T10:28:12.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.120+0000 7fccf71da740 1 -- 172.21.15.97:0/624793072 shutdown_connections 2024-04-23T10:28:12.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.120+0000 7fccf71da740 1 --2- 172.21.15.97:0/624793072 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x555883558810 0x55588355ac30 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:12.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.120+0000 7fccf71da740 1 --2- 172.21.15.97:0/624793072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5558834e1800 0x5558834e7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:12.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.120+0000 7fccf71da740 1 --2- 172.21.15.97:0/624793072 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5558834d6f40 0x5558834e7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:12.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.120+0000 7fccd5ffb640 0 client.15945 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:28:12.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.120+0000 7fccf71da740 1 --2- 172.21.15.97:0/624793072 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5558834d6620 0x5558834e7510 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:12.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.120+0000 7fccf71da740 1 -- 172.21.15.97:0/624793072 >> 172.21.15.97:0/624793072 conn(0x5558834d2b30 msgr2=0x5558834d1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:28:12.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.120+0000 7fccf71da740 1 -- 172.21.15.97:0/624793072 shutdown_connections 2024-04-23T10:28:12.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.120+0000 7fccf71da740 1 -- 172.21.15.97:0/624793072 wait complete. 2024-04-23T10:28:12.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.120+0000 7fccf71da740 20 client.15945 trim_cache size 0 max 16384 2024-04-23T10:28:12.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:28:12.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:28:12.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:28:12.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:28:12.121 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.120+0000 7fccf43e5640 1 -- 172.21.15.97:0/1468099852 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fccd80a3dc0 msgr2=0x7fccd80a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:28:12.121 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.120+0000 7fccf43e5640 1 --2- 172.21.15.97:0/1468099852 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fccd80a3dc0 0x7fccd80a6280 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fcce4009270 tx=0x7fcce40109a0 comp rx=0 tx=0).stop 2024-04-23T10:28:12.121 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.120+0000 7fccf43e5640 1 -- 172.21.15.97:0/1468099852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fccec100fa0 msgr2=0x7fccec198b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:28:12.122 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.120+0000 7fccf43e5640 1 --2- 172.21.15.97:0/1468099852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fccec100fa0 0x7fccec198b90 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fccdc005940 tx=0x7fccdc0042f0 comp rx=0 tx=0).stop 2024-04-23T10:28:12.122 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.120+0000 7fccf43e5640 1 -- 172.21.15.97:0/1468099852 shutdown_connections 2024-04-23T10:28:12.122 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.120+0000 7fccf43e5640 1 --2- 172.21.15.97:0/1468099852 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fccd80a3dc0 0x7fccd80a6280 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:12.122 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.120+0000 7fccf43e5640 1 --2- 172.21.15.97:0/1468099852 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fccec106680 0x7fccec19cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:12.122 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.120+0000 7fccf43e5640 1 --2- 172.21.15.97:0/1468099852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fccec100fa0 0x7fccec198b90 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:12.122 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.120+0000 7fccf43e5640 1 --2- 172.21.15.97:0/1468099852 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fccec1005e0 0x7fccec198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:12.122 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.120+0000 7fccf43e5640 1 -- 172.21.15.97:0/1468099852 >> 172.21.15.97:0/1468099852 conn(0x7fccec077390 msgr2=0x7fccec0fe960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:28:12.122 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.120+0000 7fccf43e5640 1 -- 172.21.15.97:0/1468099852 shutdown_connections 2024-04-23T10:28:12.122 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:12.120+0000 7fccf43e5640 1 -- 172.21.15.97:0/1468099852 wait complete. 2024-04-23T10:28:13.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:28:13 smithi196 ceph-mon[32919]: pgmap v63: 97 pgs: 97 active+clean; 2.0 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 42 KiB/s wr, 30 op/s 2024-04-23T10:28:13.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:28:13 smithi097 ceph-mon[29197]: pgmap v63: 97 pgs: 97 active+clean; 2.0 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 42 KiB/s wr, 30 op/s 2024-04-23T10:28:13.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:28:13 smithi183 ceph-mon[32317]: pgmap v63: 97 pgs: 97 active+clean; 2.0 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 42 KiB/s wr, 30 op/s 2024-04-23T10:28:15.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:28:15 smithi196 ceph-mon[32919]: pgmap v64: 97 pgs: 97 active+clean; 2.2 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 55 KiB/s wr, 39 op/s 2024-04-23T10:28:15.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:28:15 smithi097 ceph-mon[29197]: pgmap v64: 97 pgs: 97 active+clean; 2.2 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 55 KiB/s wr, 39 op/s 2024-04-23T10:28:15.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:28:15 smithi183 ceph-mon[32317]: pgmap v64: 97 pgs: 97 active+clean; 2.2 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 55 KiB/s wr, 39 op/s 2024-04-23T10:28:17.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:28:17 smithi196 ceph-mon[32919]: pgmap v65: 97 pgs: 97 active+clean; 2.2 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-23T10:28:17.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:28:17 smithi097 ceph-mon[29197]: pgmap v65: 97 pgs: 97 active+clean; 2.2 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-23T10:28:17.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:28:17 smithi183 ceph-mon[32317]: pgmap v65: 97 pgs: 97 active+clean; 2.2 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-23T10:28:18.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:28:18 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:28:18.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:28:18 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:28:18.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:28:18 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:28:18.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:28:18 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:28:18.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:28:18 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:28:18.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:28:18 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:28:18.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:28:18 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:28:18.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:28:18 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:28:18.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:28:18 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:28:18.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:28:18 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:28:18.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:28:18 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:28:18.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:28:18 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:28:18.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:28:18 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:28:18.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:28:18 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:28:18.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:28:18 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:28:19.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:28:19 smithi196 ceph-mon[32919]: pgmap v66: 97 pgs: 97 active+clean; 2.2 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 30 KiB/s wr, 22 op/s 2024-04-23T10:28:19.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:28:19 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:28:19.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:28:19 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:28:19.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:28:19 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:28:19.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:28:19 smithi097 ceph-mon[29197]: pgmap v66: 97 pgs: 97 active+clean; 2.2 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 30 KiB/s wr, 22 op/s 2024-04-23T10:28:19.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:28:19 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:28:19.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:28:19 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:28:19.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:28:19 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:28:19.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:28:19 smithi183 ceph-mon[32317]: pgmap v66: 97 pgs: 97 active+clean; 2.2 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 30 KiB/s wr, 22 op/s 2024-04-23T10:28:19.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:28:19 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:28:19.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:28:19 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:28:19.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:28:19 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:28:21.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:28:21 smithi196 ceph-mon[32919]: pgmap v67: 97 pgs: 97 active+clean; 2.2 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-04-23T10:28:21.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:28:21 smithi097 ceph-mon[29197]: pgmap v67: 97 pgs: 97 active+clean; 2.2 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-04-23T10:28:21.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:28:21 smithi183 ceph-mon[32317]: pgmap v67: 97 pgs: 97 active+clean; 2.2 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-04-23T10:28:23.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:28:23 smithi196 ceph-mon[32919]: pgmap v68: 97 pgs: 97 active+clean; 2.2 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-04-23T10:28:23.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:28:23 smithi097 ceph-mon[29197]: pgmap v68: 97 pgs: 97 active+clean; 2.2 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-04-23T10:28:23.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:28:23 smithi183 ceph-mon[32317]: pgmap v68: 97 pgs: 97 active+clean; 2.2 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-04-23T10:28:25.217 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.216+0000 7f34b02cf640 1 -- 172.21.15.97:0/4016734080 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34a806b120 msgr2=0x7f34a810edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:28:25.217 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.216+0000 7f34b02cf640 1 --2- 172.21.15.97:0/4016734080 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34a806b120 0x7f34a810edf0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f34a0009ca0 tx=0x7f34a002a950 comp rx=0 tx=0).stop 2024-04-23T10:28:25.217 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.216+0000 7f34b02cf640 1 -- 172.21.15.97:0/4016734080 shutdown_connections 2024-04-23T10:28:25.217 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.216+0000 7f34b02cf640 1 --2- 172.21.15.97:0/4016734080 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34a806b120 0x7f34a810edf0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:25.218 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.216+0000 7f34b02cf640 1 --2- 172.21.15.97:0/4016734080 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34a806a780 0x7f34a806abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:25.218 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.216+0000 7f34b02cf640 1 --2- 172.21.15.97:0/4016734080 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f34a81080e0 0x7f34a81084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:25.218 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.216+0000 7f34b02cf640 1 -- 172.21.15.97:0/4016734080 >> 172.21.15.97:0/4016734080 conn(0x7f34a8076c50 msgr2=0x7f34a8077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:28:25.218 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.216+0000 7f34b02cf640 1 -- 172.21.15.97:0/4016734080 shutdown_connections 2024-04-23T10:28:25.218 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.216+0000 7f34b02cf640 1 -- 172.21.15.97:0/4016734080 wait complete. 2024-04-23T10:28:25.218 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.217+0000 7f34b02cf640 1 Processor -- start 2024-04-23T10:28:25.218 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.217+0000 7f34b02cf640 1 -- start start 2024-04-23T10:28:25.218 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.217+0000 7f34b02cf640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34a806a780 0x7f34a81985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:25.218 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.217+0000 7f34b02cf640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f34a806b120 0x7f34a8198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:25.218 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.217+0000 7f34b02cf640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34a81080e0 0x7f34a819ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:25.218 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.217+0000 7f34b02cf640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f34a81121b0 con 0x7f34a81080e0 2024-04-23T10:28:25.218 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.217+0000 7f34b02cf640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f34a8112030 con 0x7f34a806b120 2024-04-23T10:28:25.219 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.217+0000 7f34b02cf640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f34a8112330 con 0x7f34a806a780 2024-04-23T10:28:25.219 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.217+0000 7f34ae86c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34a81080e0 0x7f34a819ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:25.219 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.217+0000 7f34ae86c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34a81080e0 0x7f34a819ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57092/0 (socket says 172.21.15.97:57092) 2024-04-23T10:28:25.219 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.217+0000 7f34ae86c640 1 -- 172.21.15.97:0/1777481378 learned_addr learned my addr 172.21.15.97:0/1777481378 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:28:25.219 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.217+0000 7f34ae06b640 1 --2- 172.21.15.97:0/1777481378 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34a806a780 0x7f34a81985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:25.219 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.217+0000 7f34ae86c640 1 -- 172.21.15.97:0/1777481378 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34a806a780 msgr2=0x7f34a81985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:28:25.219 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.217+0000 7f34ae86c640 1 --2- 172.21.15.97:0/1777481378 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34a806a780 0x7f34a81985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:25.219 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.217+0000 7f34ae86c640 1 -- 172.21.15.97:0/1777481378 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f34a806b120 msgr2=0x7f34a8198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:28:25.219 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.217+0000 7f34ad86a640 1 --2- 172.21.15.97:0/1777481378 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f34a806b120 0x7f34a8198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:25.219 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.217+0000 7f34ae86c640 1 --2- 172.21.15.97:0/1777481378 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f34a806b120 0x7f34a8198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:25.219 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.217+0000 7f34ae86c640 1 -- 172.21.15.97:0/1777481378 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34a819d5c0 con 0x7f34a81080e0 2024-04-23T10:28:25.219 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.217+0000 7f34ad86a640 1 --2- 172.21.15.97:0/1777481378 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f34a806b120 0x7f34a8198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:28:25.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.217+0000 7f34ae86c640 1 --2- 172.21.15.97:0/1777481378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34a81080e0 0x7f34a819ce90 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f34a002ae70 tx=0x7f34a002f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:28:25.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.218+0000 7f349f7fe640 1 -- 172.21.15.97:0/1777481378 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a0034040 con 0x7f34a81080e0 2024-04-23T10:28:25.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.218+0000 7f349f7fe640 1 -- 172.21.15.97:0/1777481378 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34a00308f0 con 0x7f34a81080e0 2024-04-23T10:28:25.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.218+0000 7f349f7fe640 1 -- 172.21.15.97:0/1777481378 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a00047f0 con 0x7f34a81080e0 2024-04-23T10:28:25.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.218+0000 7f34b02cf640 1 -- 172.21.15.97:0/1777481378 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34a819d850 con 0x7f34a81080e0 2024-04-23T10:28:25.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.218+0000 7f34b02cf640 1 -- 172.21.15.97:0/1777481378 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34a81a5140 con 0x7f34a81080e0 2024-04-23T10:28:25.227 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.220+0000 7f349f7fe640 1 -- 172.21.15.97:0/1777481378 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f34a0037020 con 0x7f34a81080e0 2024-04-23T10:28:25.227 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.221+0000 7f34b2fec740 1 Processor -- start 2024-04-23T10:28:25.227 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.221+0000 7f34b2fec740 1 -- start start 2024-04-23T10:28:25.227 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.221+0000 7f34b2fec740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a7f5c9620 0x561a7f5c9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:25.227 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.221+0000 7f34b2fec740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561a7f5c9f40 0x561a7f5d42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:25.227 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.221+0000 7f34b2fec740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a7f5d4800 0x561a7f5d6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:25.227 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.221+0000 7f34b2fec740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x561a7f52e970 con 0x561a7f5c9620 2024-04-23T10:28:25.227 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.221+0000 7f34b2fec740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561a7f4cb4a0 con 0x561a7f5c9f40 2024-04-23T10:28:25.227 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.221+0000 7f34b2fec740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561a7f52a930 con 0x561a7f5d4800 2024-04-23T10:28:25.227 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.221+0000 7f349f7fe640 1 --2- 172.21.15.97:0/1777481378 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f34940a3dc0 0x7f34940a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:25.228 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.222+0000 7f34ad86a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a7f5c9620 0x561a7f5c9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:25.228 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.222+0000 7f34ad86a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a7f5c9620 0x561a7f5c9a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57098/0 (socket says 172.21.15.97:57098) 2024-04-23T10:28:25.228 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.222+0000 7f34ad86a640 1 -- 172.21.15.97:0/479926916 learned_addr learned my addr 172.21.15.97:0/479926916 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:28:25.228 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.222+0000 7f34ae86c640 1 --2- 172.21.15.97:0/1777481378 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f34940a3dc0 0x7f34940a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:25.228 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.222+0000 7f34ae06b640 1 --2- 172.21.15.97:0/479926916 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561a7f5c9f40 0x561a7f5d42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:25.228 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.222+0000 7f34ad86a640 1 --2- 172.21.15.97:0/479926916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a7f5d4800 0x561a7f5d6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:25.228 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.222+0000 7f349f7fe640 1 -- 172.21.15.97:0/1777481378 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f34a00f1730 con 0x7f34a81080e0 2024-04-23T10:28:25.228 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.222+0000 7f34ad86a640 1 -- 172.21.15.97:0/479926916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a7f5d4800 msgr2=0x561a7f5d6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:28:25.228 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.222+0000 7f34ad86a640 1 --2- 172.21.15.97:0/479926916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a7f5d4800 0x561a7f5d6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:25.228 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.222+0000 7f34ad86a640 1 -- 172.21.15.97:0/479926916 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561a7f5c9f40 msgr2=0x561a7f5d42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:28:25.228 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.222+0000 7f34ad86a640 1 --2- 172.21.15.97:0/479926916 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561a7f5c9f40 0x561a7f5d42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:25.228 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.222+0000 7f34ad86a640 1 -- 172.21.15.97:0/479926916 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a7f241210 con 0x561a7f5c9620 2024-04-23T10:28:25.228 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.222+0000 7f34ad86a640 1 --2- 172.21.15.97:0/479926916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a7f5c9620 0x561a7f5c9a00 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f34a4005fd0 tx=0x7f34a4009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc7f6296cc48e541 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:28:25.228 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.222+0000 7f34ae86c640 1 --2- 172.21.15.97:0/1777481378 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f34940a3dc0 0x7f34940a6280 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f34a00ef090 tx=0x7f34a008d110 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:28:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.222+0000 7f349cbfe640 1 -- 172.21.15.97:0/479926916 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a40104d0 con 0x561a7f5c9620 2024-04-23T10:28:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.222+0000 7f349cbfe640 1 -- 172.21.15.97:0/479926916 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34a401b070 con 0x561a7f5c9620 2024-04-23T10:28:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.222+0000 7f349cbfe640 1 -- 172.21.15.97:0/479926916 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a4013780 con 0x561a7f5c9620 2024-04-23T10:28:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.223+0000 7f34b2fec740 1 -- 172.21.15.97:0/479926916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a7f5c9620 msgr2=0x561a7f5c9a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:28:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.223+0000 7f34b2fec740 1 --2- 172.21.15.97:0/479926916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a7f5c9620 0x561a7f5c9a00 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f34a4005fd0 tx=0x7f34a4009650 comp rx=0 tx=0).stop 2024-04-23T10:28:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.223+0000 7f34b2fec740 1 -- 172.21.15.97:0/479926916 shutdown_connections 2024-04-23T10:28:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.223+0000 7f34b2fec740 1 --2- 172.21.15.97:0/479926916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a7f5d4800 0x561a7f5d6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.223+0000 7f34b2fec740 1 --2- 172.21.15.97:0/479926916 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561a7f5c9f40 0x561a7f5d42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.223+0000 7f34b2fec740 1 --2- 172.21.15.97:0/479926916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a7f5c9620 0x561a7f5c9a00 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.223+0000 7f34b2fec740 1 -- 172.21.15.97:0/479926916 >> 172.21.15.97:0/479926916 conn(0x561a7f5c5b30 msgr2=0x561a7f5c5f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:28:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.223+0000 7f34b2fec740 1 -- 172.21.15.97:0/479926916 shutdown_connections 2024-04-23T10:28:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.223+0000 7f34b2fec740 1 -- 172.21.15.97:0/479926916 wait complete. 2024-04-23T10:28:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.224+0000 7f34b2fec740 1 -- start start 2024-04-23T10:28:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.224+0000 7f34b2fec740 1 Processor -- start 2024-04-23T10:28:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.224+0000 7f34b2fec740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a7f5c9620 0x561a7f5de480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:25.230 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.224+0000 7f34b2fec740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a7f5c9f40 0x561a7f5de9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:25.230 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.224+0000 7f34b2fec740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561a7f5d4800 0x561a7f5d87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:25.230 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.224+0000 7f34b2fec740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x561a7f5d73d0 con 0x561a7f5c9620 2024-04-23T10:28:25.230 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.224+0000 7f34b2fec740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561a7f4c90b0 con 0x561a7f5d4800 2024-04-23T10:28:25.230 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.224+0000 7f34b2fec740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561a7f5d7550 con 0x561a7f5c9f40 2024-04-23T10:28:25.230 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.224+0000 7f34b2fec740 10 client.0 _ll_register_callbacks cb 0x561a7f4b6d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:28:25.230 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.224+0000 7f34ad86a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a7f5c9620 0x561a7f5de480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:25.230 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.224+0000 7f34ad86a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a7f5c9620 0x561a7f5de480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57100/0 (socket says 172.21.15.97:57100) 2024-04-23T10:28:25.230 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.224+0000 7f34ad86a640 1 -- 172.21.15.97:0/2611816937 learned_addr learned my addr 172.21.15.97:0/2611816937 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:28:25.230 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.224+0000 7f34ae06b640 1 --2- 172.21.15.97:0/2611816937 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a7f5c9f40 0x561a7f5de9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:25.230 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.224+0000 7f34ad86a640 1 --2- 172.21.15.97:0/2611816937 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561a7f5d4800 0x561a7f5d87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:25.231 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.224+0000 7f34ad86a640 1 -- 172.21.15.97:0/2611816937 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a7f5c9f40 msgr2=0x561a7f5de9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:28:25.231 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.224+0000 7f34ad86a640 1 --2- 172.21.15.97:0/2611816937 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a7f5c9f40 0x561a7f5de9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:25.231 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.224+0000 7f34ad86a640 1 -- 172.21.15.97:0/2611816937 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561a7f5d4800 msgr2=0x561a7f5d87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:28:25.231 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.224+0000 7f34ad86a640 1 --2- 172.21.15.97:0/2611816937 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561a7f5d4800 0x561a7f5d87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:25.231 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.224+0000 7f34ad86a640 1 -- 172.21.15.97:0/2611816937 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a7f30d620 con 0x561a7f5c9620 2024-04-23T10:28:25.231 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.225+0000 7f34ad86a640 1 --2- 172.21.15.97:0/2611816937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a7f5c9620 0x561a7f5de480 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f34a4009620 tx=0x7f34a40095b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3be9458db00685cd server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:28:25.231 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.225+0000 7f3491ffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:28:25.231 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.225+0000 7f3491ffb640 1 -- 172.21.15.97:0/2611816937 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a4005260 con 0x561a7f5c9620 2024-04-23T10:28:25.231 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.225+0000 7f3491ffb640 1 -- 172.21.15.97:0/2611816937 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f34a4005400 con 0x561a7f5c9620 2024-04-23T10:28:25.231 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.225+0000 7f34b2fec740 1 -- 172.21.15.97:0/2611816937 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34a401b070 con 0x561a7f5c9620 2024-04-23T10:28:25.231 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.225+0000 7f3491ffb640 1 -- 172.21.15.97:0/2611816937 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f34a4005600 con 0x561a7f5c9620 2024-04-23T10:28:25.231 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.225+0000 7f34b2fec740 10 client.15957 fetch_fsmap learned FSMap version 26 2024-04-23T10:28:25.231 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.225+0000 7f34b2fec740 1 -- 172.21.15.97:0/2611816937 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561a7f5e7700 con 0x561a7f5c9620 2024-04-23T10:28:25.232 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.225+0000 7f3491ffb640 1 -- 172.21.15.97:0/2611816937 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34a401b070 con 0x561a7f5c9620 2024-04-23T10:28:25.232 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.225+0000 7f3491ffb640 1 -- 172.21.15.97:0/2611816937 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f34a400c070 con 0x561a7f5c9620 2024-04-23T10:28:25.232 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.225+0000 7f34b2fec740 10 client.15957 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:28:25.232 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.225+0000 7f34b2fec740 10 client.15957 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:28:25.232 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.225+0000 7f34b2fec740 20 client.15957 populate_metadata read hostname 'smithi097' 2024-04-23T10:28:25.232 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.225+0000 7f34b2fec740 1 --2- 172.21.15.97:0/2611816937 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x561a7f5e8b00 0x561a7f5eaef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:25.232 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.225+0000 7f34b2fec740 4 client.15957 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:28:25.232 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.225+0000 7f34b2fec740 1 -- 172.21.15.97:0/2611816937 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a7f53c390 con 0x561a7f5e8b00 2024-04-23T10:28:25.232 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.226+0000 7f34ae06b640 1 --2- 172.21.15.97:0/2611816937 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x561a7f5e8b00 0x561a7f5eaef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:25.232 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.226+0000 7f34ae06b640 1 --2- 172.21.15.97:0/2611816937 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x561a7f5e8b00 0x561a7f5eaef0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d548cd2c680c7fa1 server_cookie=69402e42c43b1aed in_seq=0 out_seq=0 2024-04-23T10:28:25.232 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.226+0000 7f3491ffb640 10 client.15957 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:28:25.232 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.227+0000 7f3491ffb640 1 -- 172.21.15.97:0/2611816937 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561a7f53c390 con 0x561a7f5e8b00 2024-04-23T10:28:25.232 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.227+0000 7f3491ffb640 10 client.15957 handle_command_reply: tid=0 2024-04-23T10:28:25.232 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.227+0000 7f34b2fec740 1 client.15957 shutdown 2024-04-23T10:28:25.232 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.227+0000 7f34b2fec740 1 -- 172.21.15.97:0/2611816937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a7f5c9620 msgr2=0x561a7f5de480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:28:25.233 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.227+0000 7f34b2fec740 1 --2- 172.21.15.97:0/2611816937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a7f5c9620 0x561a7f5de480 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f34a4009620 tx=0x7f34a40095b0 comp rx=0 tx=0).stop 2024-04-23T10:28:25.233 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.227+0000 7f34b2fec740 1 -- 172.21.15.97:0/2611816937 shutdown_connections 2024-04-23T10:28:25.233 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.227+0000 7f34b2fec740 1 --2- 172.21.15.97:0/2611816937 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x561a7f5e8b00 0x561a7f5eaef0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:25.233 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.227+0000 7f34b2fec740 1 --2- 172.21.15.97:0/2611816937 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561a7f5d4800 0x561a7f5d87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:25.233 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.227+0000 7f34b2fec740 1 --2- 172.21.15.97:0/2611816937 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a7f5c9f40 0x561a7f5de9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:25.233 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.227+0000 7f34b2fec740 1 --2- 172.21.15.97:0/2611816937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a7f5c9620 0x561a7f5de480 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:25.233 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.227+0000 7f34b2fec740 1 -- 172.21.15.97:0/2611816937 >> 172.21.15.97:0/2611816937 conn(0x561a7f5c5b30 msgr2=0x561a7f5cb8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:28:25.233 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.227+0000 7f3491ffb640 0 client.15957 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:28:25.233 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.227+0000 7f34b2fec740 1 -- 172.21.15.97:0/2611816937 shutdown_connections 2024-04-23T10:28:25.233 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.227+0000 7f34b2fec740 1 -- 172.21.15.97:0/2611816937 wait complete. 2024-04-23T10:28:25.233 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.227+0000 7f34b2fec740 20 client.15957 trim_cache size 0 max 16384 2024-04-23T10:28:25.245 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.242+0000 7f34b2fec740 1 Processor -- start 2024-04-23T10:28:25.245 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.242+0000 7f34b2fec740 1 -- start start 2024-04-23T10:28:25.245 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.242+0000 7f34b2fec740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a7f5c5b30 0x561a7f5de750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:25.245 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.242+0000 7f34b2fec740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561a7f5c9620 0x561a7f5f05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:25.246 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.242+0000 7f34b2fec740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a7f5c9f40 0x561a7f5f0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:25.246 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.242+0000 7f34b2fec740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f34a81ab6b0 con 0x561a7f5c5b30 2024-04-23T10:28:25.246 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.242+0000 7f34b2fec740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f34a81ad1f0 con 0x561a7f5c9620 2024-04-23T10:28:25.246 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.242+0000 7f34b2fec740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f34a81abd80 con 0x561a7f5c9f40 2024-04-23T10:28:25.246 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.242+0000 7f34ad86a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a7f5c5b30 0x561a7f5de750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:25.246 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.242+0000 7f34ad86a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a7f5c5b30 0x561a7f5de750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57102/0 (socket says 172.21.15.97:57102) 2024-04-23T10:28:25.246 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.242+0000 7f34ad86a640 1 -- 172.21.15.97:0/4143599411 learned_addr learned my addr 172.21.15.97:0/4143599411 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:28:25.246 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.242+0000 7f34ae06b640 1 --2- 172.21.15.97:0/4143599411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561a7f5c9620 0x561a7f5f05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:25.246 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.243+0000 7f34ad86a640 1 -- 172.21.15.97:0/4143599411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a7f5c9f40 msgr2=0x561a7f5f0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:28:25.246 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.243+0000 7f34ad86a640 1 --2- 172.21.15.97:0/4143599411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a7f5c9f40 0x561a7f5f0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:25.246 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.243+0000 7f34ad86a640 1 -- 172.21.15.97:0/4143599411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561a7f5c9620 msgr2=0x561a7f5f05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:28:25.246 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.243+0000 7f34ad86a640 1 --2- 172.21.15.97:0/4143599411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561a7f5c9620 0x561a7f5f05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:25.247 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.243+0000 7f34ad86a640 1 -- 172.21.15.97:0/4143599411 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a7f5c5960 con 0x561a7f5c5b30 2024-04-23T10:28:25.247 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.243+0000 7f34ad86a640 1 --2- 172.21.15.97:0/4143599411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a7f5c5b30 0x561a7f5de750 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f34a40057a0 tx=0x7f34a4015300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45f25a433c3faeb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:28:25.247 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.243+0000 7f3493fff640 1 -- 172.21.15.97:0/4143599411 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a403c070 con 0x561a7f5c5b30 2024-04-23T10:28:25.247 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.243+0000 7f3493fff640 1 -- 172.21.15.97:0/4143599411 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34a401b040 con 0x561a7f5c5b30 2024-04-23T10:28:25.247 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.243+0000 7f3493fff640 1 -- 172.21.15.97:0/4143599411 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a4010e00 con 0x561a7f5c5b30 2024-04-23T10:28:25.247 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.243+0000 7f34b2fec740 1 -- 172.21.15.97:0/4143599411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a7f5c5b30 msgr2=0x561a7f5de750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:28:25.247 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.244+0000 7f34b2fec740 1 --2- 172.21.15.97:0/4143599411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a7f5c5b30 0x561a7f5de750 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f34a40057a0 tx=0x7f34a4015300 comp rx=0 tx=0).stop 2024-04-23T10:28:25.247 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.244+0000 7f34b2fec740 1 -- 172.21.15.97:0/4143599411 shutdown_connections 2024-04-23T10:28:25.247 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.244+0000 7f34b2fec740 1 --2- 172.21.15.97:0/4143599411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a7f5c9f40 0x561a7f5f0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:25.247 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.244+0000 7f34b2fec740 1 --2- 172.21.15.97:0/4143599411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561a7f5c9620 0x561a7f5f05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:25.247 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.244+0000 7f34b2fec740 1 --2- 172.21.15.97:0/4143599411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a7f5c5b30 0x561a7f5de750 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:25.247 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.244+0000 7f34b2fec740 1 -- 172.21.15.97:0/4143599411 >> 172.21.15.97:0/4143599411 conn(0x561a7f5d4420 msgr2=0x561a7f5de040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:28:25.247 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.244+0000 7f34b2fec740 1 -- 172.21.15.97:0/4143599411 shutdown_connections 2024-04-23T10:28:25.247 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.244+0000 7f34b2fec740 1 -- 172.21.15.97:0/4143599411 wait complete. 2024-04-23T10:28:25.247 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.244+0000 7f34b2fec740 1 -- start start 2024-04-23T10:28:25.247 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.244+0000 7f34b2fec740 1 Processor -- start 2024-04-23T10:28:25.248 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.244+0000 7f34b2fec740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561a7f5c5b30 0x561a7f5e4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:25.248 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.244+0000 7f34b2fec740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a7f5c9620 0x561a7f5e1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:25.248 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.244+0000 7f34b2fec740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a7f5c9f40 0x561a7f5e23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:25.248 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.244+0000 7f34b2fec740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f34a81acb20 con 0x561a7f5c9620 2024-04-23T10:28:25.248 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.244+0000 7f34b2fec740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f34a81ac450 con 0x561a7f5c5b30 2024-04-23T10:28:25.248 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.244+0000 7f34b2fec740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f34a81a96b0 con 0x561a7f5c9f40 2024-04-23T10:28:25.248 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.244+0000 7f34b2fec740 10 client.0 _ll_register_callbacks cb 0x7f34a4009330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:28:25.248 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.244+0000 7f34ae06b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a7f5c9620 0x561a7f5e1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:25.248 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.244+0000 7f34ae06b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a7f5c9620 0x561a7f5e1eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57114/0 (socket says 172.21.15.97:57114) 2024-04-23T10:28:25.248 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.244+0000 7f34ae06b640 1 -- 172.21.15.97:0/3664826139 learned_addr learned my addr 172.21.15.97:0/3664826139 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:28:25.248 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.244+0000 7f34ad86a640 1 --2- 172.21.15.97:0/3664826139 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561a7f5c5b30 0x561a7f5e4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:25.248 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.245+0000 7f34ae06b640 1 -- 172.21.15.97:0/3664826139 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a7f5c9f40 msgr2=0x561a7f5e23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:28:25.248 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.245+0000 7f34ae06b640 1 --2- 172.21.15.97:0/3664826139 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a7f5c9f40 0x561a7f5e23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:25.248 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.245+0000 7f34ae06b640 1 -- 172.21.15.97:0/3664826139 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561a7f5c5b30 msgr2=0x561a7f5e4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:28:25.248 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.245+0000 7f34ae06b640 1 --2- 172.21.15.97:0/3664826139 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561a7f5c5b30 0x561a7f5e4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:25.249 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.245+0000 7f34ae06b640 1 -- 172.21.15.97:0/3664826139 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a7f5e2930 con 0x561a7f5c9620 2024-04-23T10:28:25.249 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.245+0000 7f34ae06b640 1 --2- 172.21.15.97:0/3664826139 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a7f5c9620 0x561a7f5e1eb0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x561a7f5d8050 tx=0x7f3498004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9bbeb8439d95b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:28:25.249 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.245+0000 7f3491ffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:28:25.249 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.245+0000 7f3491ffb640 1 -- 172.21.15.97:0/3664826139 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3498021070 con 0x561a7f5c9620 2024-04-23T10:28:25.249 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.245+0000 7f3491ffb640 1 -- 172.21.15.97:0/3664826139 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f349801d380 con 0x561a7f5c9620 2024-04-23T10:28:25.249 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.245+0000 7f34b2fec740 1 -- 172.21.15.97:0/3664826139 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34a401b040 con 0x561a7f5c9620 2024-04-23T10:28:25.249 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.245+0000 7f3491ffb640 1 -- 172.21.15.97:0/3664826139 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f349802d050 con 0x561a7f5c9620 2024-04-23T10:28:25.249 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.245+0000 7f34b2fec740 10 client.15963 fetch_fsmap learned FSMap version 26 2024-04-23T10:28:25.249 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.245+0000 7f34b2fec740 1 -- 172.21.15.97:0/3664826139 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561a7f64ec10 con 0x561a7f5c9620 2024-04-23T10:28:25.249 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.245+0000 7f3491ffb640 1 -- 172.21.15.97:0/3664826139 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34a401b040 con 0x561a7f5c9620 2024-04-23T10:28:25.249 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.245+0000 7f3491ffb640 1 -- 172.21.15.97:0/3664826139 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3498011c70 con 0x561a7f5c9620 2024-04-23T10:28:25.249 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.245+0000 7f34b2fec740 10 client.15963 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:28:25.249 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.245+0000 7f34b2fec740 10 client.15963 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:28:25.249 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.245+0000 7f34b2fec740 20 client.15963 populate_metadata read hostname 'smithi097' 2024-04-23T10:28:25.249 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.245+0000 7f34b2fec740 1 --2- 172.21.15.97:0/3664826139 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x561a7f5d4800 0x561a7f6513e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:25.249 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.245+0000 7f34b2fec740 4 client.15963 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:28:25.250 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.245+0000 7f34b2fec740 1 -- 172.21.15.97:0/3664826139 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561a7f536200 con 0x561a7f5d4800 2024-04-23T10:28:25.250 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.246+0000 7f34ad86a640 1 --2- 172.21.15.97:0/3664826139 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x561a7f5d4800 0x561a7f6513e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:25.250 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.246+0000 7f34ad86a640 1 --2- 172.21.15.97:0/3664826139 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x561a7f5d4800 0x561a7f6513e0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f240a8fb1d381b1 server_cookie=d957bf0d3a2491d5 in_seq=0 out_seq=0 2024-04-23T10:28:25.250 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.246+0000 7f3491ffb640 10 client.15963 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:28:25.260 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.260+0000 7f3491ffb640 1 -- 172.21.15.97:0/3664826139 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561a7f536200 con 0x561a7f5d4800 2024-04-23T10:28:25.260 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.260+0000 7f3491ffb640 10 client.15963 handle_command_reply: tid=0 2024-04-23T10:28:25.260 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.260+0000 7f34b2fec740 1 client.15963 shutdown 2024-04-23T10:28:25.260 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.261+0000 7f34b2fec740 1 -- 172.21.15.97:0/3664826139 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a7f5c9620 msgr2=0x561a7f5e1eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:28:25.260 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.261+0000 7f34b2fec740 1 --2- 172.21.15.97:0/3664826139 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a7f5c9620 0x561a7f5e1eb0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x561a7f5d8050 tx=0x7f3498004410 comp rx=0 tx=0).stop 2024-04-23T10:28:25.260 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.261+0000 7f34b2fec740 1 -- 172.21.15.97:0/3664826139 shutdown_connections 2024-04-23T10:28:25.260 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.261+0000 7f34b2fec740 1 --2- 172.21.15.97:0/3664826139 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x561a7f5d4800 0x561a7f6513e0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:25.260 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.261+0000 7f34b2fec740 1 --2- 172.21.15.97:0/3664826139 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561a7f5c9f40 0x561a7f5e23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:25.260 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.261+0000 7f34b2fec740 1 --2- 172.21.15.97:0/3664826139 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a7f5c9620 0x561a7f5e1eb0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.261+0000 7f34b2fec740 1 --2- 172.21.15.97:0/3664826139 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561a7f5c5b30 0x561a7f5e4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.261+0000 7f3491ffb640 0 client.15963 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:28:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.261+0000 7f34b2fec740 1 -- 172.21.15.97:0/3664826139 >> 172.21.15.97:0/3664826139 conn(0x561a7f5d4420 msgr2=0x561a7f5d0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:28:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.261+0000 7f34b2fec740 1 -- 172.21.15.97:0/3664826139 shutdown_connections 2024-04-23T10:28:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.261+0000 7f34b2fec740 1 -- 172.21.15.97:0/3664826139 wait complete. 2024-04-23T10:28:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.261+0000 7f34b2fec740 20 client.15963 trim_cache size 0 max 16384 2024-04-23T10:28:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:28:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:28:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:28:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:28:25.262 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.261+0000 7f34b02cf640 1 -- 172.21.15.97:0/1777481378 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f34940a3dc0 msgr2=0x7f34940a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:28:25.262 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.261+0000 7f34b02cf640 1 --2- 172.21.15.97:0/1777481378 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f34940a3dc0 0x7f34940a6280 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f34a00ef090 tx=0x7f34a008d110 comp rx=0 tx=0).stop 2024-04-23T10:28:25.263 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.261+0000 7f34b02cf640 1 -- 172.21.15.97:0/1777481378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34a81080e0 msgr2=0x7f34a819ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:28:25.263 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.261+0000 7f34b02cf640 1 --2- 172.21.15.97:0/1777481378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34a81080e0 0x7f34a819ce90 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f34a002ae70 tx=0x7f34a002f5d0 comp rx=0 tx=0).stop 2024-04-23T10:28:25.263 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.261+0000 7f34b02cf640 1 -- 172.21.15.97:0/1777481378 shutdown_connections 2024-04-23T10:28:25.263 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.261+0000 7f34b02cf640 1 --2- 172.21.15.97:0/1777481378 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f34940a3dc0 0x7f34940a6280 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:25.263 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.261+0000 7f34b02cf640 1 --2- 172.21.15.97:0/1777481378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34a81080e0 0x7f34a819ce90 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:25.263 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.261+0000 7f34b02cf640 1 --2- 172.21.15.97:0/1777481378 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f34a806b120 0x7f34a8198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:25.263 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.261+0000 7f34b02cf640 1 --2- 172.21.15.97:0/1777481378 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34a806a780 0x7f34a81985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:25.263 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.261+0000 7f34b02cf640 1 -- 172.21.15.97:0/1777481378 >> 172.21.15.97:0/1777481378 conn(0x7f34a8076c50 msgr2=0x7f34a80ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:28:25.263 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.261+0000 7f34b02cf640 1 -- 172.21.15.97:0/1777481378 shutdown_connections 2024-04-23T10:28:25.263 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:25.262+0000 7f34b02cf640 1 -- 172.21.15.97:0/1777481378 wait complete. 2024-04-23T10:28:25.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:28:25 smithi196 ceph-mon[32919]: pgmap v69: 97 pgs: 97 active+clean; 2.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 30 KiB/s wr, 22 op/s 2024-04-23T10:28:25.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:28:25 smithi097 ceph-mon[29197]: pgmap v69: 97 pgs: 97 active+clean; 2.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 30 KiB/s wr, 22 op/s 2024-04-23T10:28:25.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:28:25 smithi183 ceph-mon[32317]: pgmap v69: 97 pgs: 97 active+clean; 2.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 30 KiB/s wr, 22 op/s 2024-04-23T10:28:27.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:28:27 smithi196 ceph-mon[32919]: pgmap v70: 97 pgs: 97 active+clean; 2.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-04-23T10:28:27.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:28:27 smithi097 ceph-mon[29197]: pgmap v70: 97 pgs: 97 active+clean; 2.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-04-23T10:28:27.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:28:27 smithi183 ceph-mon[32317]: pgmap v70: 97 pgs: 97 active+clean; 2.3 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-04-23T10:28:27.698 INFO:tasks.workunit.client.0.smithi097.stdout:1/600: open fd = 3 2024-04-23T10:28:27.698 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.698 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.698 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:28:27.698 INFO:tasks.workunit.client.0.smithi097.stdout:2/600: open fd = 3 2024-04-23T10:28:27.698 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.698 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.698 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:28:27.698 INFO:tasks.workunit.client.0.smithi097.stdout:3/600: open fd = 3 2024-04-23T10:28:27.699 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.699 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.699 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:28:27.699 INFO:tasks.workunit.client.0.smithi097.stdout:4/600: open fd = 3 2024-04-23T10:28:27.699 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.699 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.699 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:28:27.699 INFO:tasks.workunit.client.0.smithi097.stdout:5/600: open fd = 3 2024-04-23T10:28:27.699 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.699 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.699 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:28:27.699 INFO:tasks.workunit.client.0.smithi097.stdout:6/600: open fd = 3 2024-04-23T10:28:27.699 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.699 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.699 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:28:27.699 INFO:tasks.workunit.client.0.smithi097.stdout:7/600: open fd = 3 2024-04-23T10:28:27.699 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.700 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.700 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:28:27.700 INFO:tasks.workunit.client.0.smithi097.stdout:8/600: open fd = 3 2024-04-23T10:28:27.700 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.700 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.700 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:28:27.700 INFO:tasks.workunit.client.0.smithi097.stdout:9/600: open fd = 3 2024-04-23T10:28:27.700 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.700 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.700 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:28:27.700 INFO:tasks.workunit.client.0.smithi097.stdout:10/600: open fd = 3 2024-04-23T10:28:27.700 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.700 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.700 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:28:27.700 INFO:tasks.workunit.client.0.smithi097.stdout:11/600: open fd = 3 2024-04-23T10:28:27.700 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.700 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.701 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:28:27.701 INFO:tasks.workunit.client.0.smithi097.stdout:12/600: open fd = 3 2024-04-23T10:28:27.701 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.701 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.701 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:28:27.701 INFO:tasks.workunit.client.0.smithi097.stdout:13/600: open fd = 3 2024-04-23T10:28:27.701 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.701 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.701 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:28:27.701 INFO:tasks.workunit.client.0.smithi097.stdout:14/600: open fd = 3 2024-04-23T10:28:27.701 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.701 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.701 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:28:27.701 INFO:tasks.workunit.client.0.smithi097.stdout:15/600: open fd = 3 2024-04-23T10:28:27.701 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.702 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.702 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:28:27.702 INFO:tasks.workunit.client.0.smithi097.stdout:16/600: open fd = 3 2024-04-23T10:28:27.702 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.702 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.702 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:28:27.702 INFO:tasks.workunit.client.0.smithi097.stdout:17/600: open fd = 3 2024-04-23T10:28:27.702 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.702 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.702 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:28:27.702 INFO:tasks.workunit.client.0.smithi097.stdout:18/600: open fd = 3 2024-04-23T10:28:27.702 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.702 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.702 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:28:27.703 INFO:tasks.workunit.client.0.smithi097.stdout:19/600: open fd = 3 2024-04-23T10:28:27.703 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.703 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.703 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:28:27.703 INFO:tasks.workunit.client.0.smithi097.stdout:20/600: open fd = 3 2024-04-23T10:28:27.703 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.703 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.703 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:28:27.703 INFO:tasks.workunit.client.0.smithi097.stdout:21/600: open fd = 3 2024-04-23T10:28:27.703 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.703 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.703 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:28:27.703 INFO:tasks.workunit.client.0.smithi097.stdout:22/600: open fd = 3 2024-04-23T10:28:27.703 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.703 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.703 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:28:27.703 INFO:tasks.workunit.client.0.smithi097.stdout:23/600: open fd = 3 2024-04-23T10:28:27.704 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.704 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.704 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:28:27.704 INFO:tasks.workunit.client.0.smithi097.stdout:24/600: open fd = 3 2024-04-23T10:28:27.704 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.704 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.704 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:28:27.704 INFO:tasks.workunit.client.0.smithi097.stdout:25/600: open fd = 3 2024-04-23T10:28:27.704 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.704 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.704 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:28:27.704 INFO:tasks.workunit.client.0.smithi097.stdout:26/600: open fd = 3 2024-04-23T10:28:27.704 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.704 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.704 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:28:27.704 INFO:tasks.workunit.client.0.smithi097.stdout:27/600: open fd = 3 2024-04-23T10:28:27.705 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.705 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.705 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:28:27.705 INFO:tasks.workunit.client.0.smithi097.stdout:28/600: open fd = 3 2024-04-23T10:28:27.705 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.705 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.705 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:28:27.705 INFO:tasks.workunit.client.0.smithi097.stdout:29/600: open fd = 3 2024-04-23T10:28:27.705 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.705 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.705 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:28:27.705 INFO:tasks.workunit.client.0.smithi097.stdout:30/600: open fd = 3 2024-04-23T10:28:27.705 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.705 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.705 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:28:27.705 INFO:tasks.workunit.client.0.smithi097.stdout:31/600: open fd = 3 2024-04-23T10:28:27.705 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.706 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.706 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:28:27.706 INFO:tasks.workunit.client.0.smithi097.stdout:32/600: open fd = 3 2024-04-23T10:28:27.706 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.706 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.706 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:28:27.706 INFO:tasks.workunit.client.0.smithi097.stdout:33/600: open fd = 3 2024-04-23T10:28:27.706 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.706 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.706 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:28:27.706 INFO:tasks.workunit.client.0.smithi097.stdout:34/600: open fd = 3 2024-04-23T10:28:27.706 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.706 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.706 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:28:27.706 INFO:tasks.workunit.client.0.smithi097.stdout:35/600: open fd = 3 2024-04-23T10:28:27.706 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.707 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.707 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:28:27.707 INFO:tasks.workunit.client.0.smithi097.stdout:36/600: open fd = 3 2024-04-23T10:28:27.707 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.707 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.707 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:28:27.707 INFO:tasks.workunit.client.0.smithi097.stdout:37/600: open fd = 3 2024-04-23T10:28:27.707 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.707 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.707 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:28:27.707 INFO:tasks.workunit.client.0.smithi097.stdout:38/600: open fd = 3 2024-04-23T10:28:27.707 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.707 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.707 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:28:27.707 INFO:tasks.workunit.client.0.smithi097.stdout:39/600: open fd = 3 2024-04-23T10:28:27.707 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.707 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.708 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:28:27.708 INFO:tasks.workunit.client.0.smithi097.stdout:40/600: open fd = 3 2024-04-23T10:28:27.708 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.708 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.708 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:28:27.708 INFO:tasks.workunit.client.0.smithi097.stdout:41/600: open fd = 3 2024-04-23T10:28:27.708 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.708 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.708 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:28:27.708 INFO:tasks.workunit.client.0.smithi097.stdout:42/600: open fd = 3 2024-04-23T10:28:27.708 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.708 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.708 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:28:27.708 INFO:tasks.workunit.client.0.smithi097.stdout:43/600: open fd = 3 2024-04-23T10:28:27.708 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.709 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.709 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:28:27.709 INFO:tasks.workunit.client.0.smithi097.stdout:44/600: open fd = 3 2024-04-23T10:28:27.709 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.709 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.709 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:28:27.709 INFO:tasks.workunit.client.0.smithi097.stdout:45/600: open fd = 3 2024-04-23T10:28:27.709 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.709 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.709 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:28:27.709 INFO:tasks.workunit.client.0.smithi097.stdout:46/600: open fd = 3 2024-04-23T10:28:27.709 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.709 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.709 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:28:27.709 INFO:tasks.workunit.client.0.smithi097.stdout:47/600: open fd = 3 2024-04-23T10:28:27.709 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.710 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.710 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:28:27.710 INFO:tasks.workunit.client.0.smithi097.stdout:48/600: open fd = 3 2024-04-23T10:28:27.710 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.710 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.710 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:28:27.710 INFO:tasks.workunit.client.0.smithi097.stdout:49/600: open fd = 3 2024-04-23T10:28:27.710 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.710 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.710 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:28:27.710 INFO:tasks.workunit.client.0.smithi097.stdout:50/600: open fd = 3 2024-04-23T10:28:27.710 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.710 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.710 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:28:27.710 INFO:tasks.workunit.client.0.smithi097.stdout:51/600: open fd = 3 2024-04-23T10:28:27.710 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.711 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.711 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:28:27.711 INFO:tasks.workunit.client.0.smithi097.stdout:52/600: open fd = 3 2024-04-23T10:28:27.711 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.711 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.711 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:28:27.711 INFO:tasks.workunit.client.0.smithi097.stdout:53/600: open fd = 3 2024-04-23T10:28:27.711 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.711 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.711 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:28:27.711 INFO:tasks.workunit.client.0.smithi097.stdout:54/600: open fd = 3 2024-04-23T10:28:27.711 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.711 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.711 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:28:27.711 INFO:tasks.workunit.client.0.smithi097.stdout:55/600: open fd = 3 2024-04-23T10:28:27.712 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.712 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.712 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:28:27.712 INFO:tasks.workunit.client.0.smithi097.stdout:56/600: open fd = 3 2024-04-23T10:28:27.712 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.712 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.712 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:28:27.712 INFO:tasks.workunit.client.0.smithi097.stdout:57/600: open fd = 3 2024-04-23T10:28:27.712 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.712 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.712 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:28:27.712 INFO:tasks.workunit.client.0.smithi097.stdout:58/600: open fd = 3 2024-04-23T10:28:27.712 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.712 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.712 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:28:27.713 INFO:tasks.workunit.client.0.smithi097.stdout:59/600: open fd = 3 2024-04-23T10:28:27.713 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.713 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.713 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:28:27.713 INFO:tasks.workunit.client.0.smithi097.stdout:60/600: open fd = 3 2024-04-23T10:28:27.713 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.713 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.713 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:28:27.713 INFO:tasks.workunit.client.0.smithi097.stdout:61/600: open fd = 3 2024-04-23T10:28:27.713 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.713 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.713 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:28:27.713 INFO:tasks.workunit.client.0.smithi097.stdout:62/600: open fd = 3 2024-04-23T10:28:27.713 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.713 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.713 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:28:27.714 INFO:tasks.workunit.client.0.smithi097.stdout:63/600: open fd = 3 2024-04-23T10:28:27.714 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.714 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:28:27.714 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:28:29.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:28:29 smithi196 ceph-mon[32919]: pgmap v71: 97 pgs: 97 active+clean; 2.5 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 30 KiB/s wr, 22 op/s 2024-04-23T10:28:29.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:28:29 smithi097 ceph-mon[29197]: pgmap v71: 97 pgs: 97 active+clean; 2.5 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 30 KiB/s wr, 22 op/s 2024-04-23T10:28:29.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:28:29 smithi183 ceph-mon[32317]: pgmap v71: 97 pgs: 97 active+clean; 2.5 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 30 KiB/s wr, 22 op/s 2024-04-23T10:28:31.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:28:31 smithi196 ceph-mon[32919]: pgmap v72: 97 pgs: 97 active+clean; 2.5 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 KiB/s wr, 15 op/s 2024-04-23T10:28:31.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:28:31 smithi097 ceph-mon[29197]: pgmap v72: 97 pgs: 97 active+clean; 2.5 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 KiB/s wr, 15 op/s 2024-04-23T10:28:31.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:28:31 smithi183 ceph-mon[32317]: pgmap v72: 97 pgs: 97 active+clean; 2.5 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 KiB/s wr, 15 op/s 2024-04-23T10:28:33.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:28:33 smithi196 ceph-mon[32919]: pgmap v73: 97 pgs: 97 active+clean; 2.5 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 KiB/s wr, 15 op/s 2024-04-23T10:28:33.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:28:33 smithi097 ceph-mon[29197]: pgmap v73: 97 pgs: 97 active+clean; 2.5 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 KiB/s wr, 15 op/s 2024-04-23T10:28:33.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:28:33 smithi183 ceph-mon[32317]: pgmap v73: 97 pgs: 97 active+clean; 2.5 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 KiB/s wr, 15 op/s 2024-04-23T10:28:35.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:28:35 smithi196 ceph-mon[32919]: pgmap v74: 97 pgs: 97 active+clean; 2.6 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 22 op/s 2024-04-23T10:28:35.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:28:35 smithi097 ceph-mon[29197]: pgmap v74: 97 pgs: 97 active+clean; 2.6 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 22 op/s 2024-04-23T10:28:35.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:28:35 smithi183 ceph-mon[32317]: pgmap v74: 97 pgs: 97 active+clean; 2.6 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 22 op/s 2024-04-23T10:28:37.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:28:37 smithi196 ceph-mon[32919]: pgmap v75: 97 pgs: 97 active+clean; 2.6 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 KiB/s wr, 15 op/s 2024-04-23T10:28:37.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:28:37 smithi097 ceph-mon[29197]: pgmap v75: 97 pgs: 97 active+clean; 2.6 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 KiB/s wr, 15 op/s 2024-04-23T10:28:37.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:28:37 smithi183 ceph-mon[32317]: pgmap v75: 97 pgs: 97 active+clean; 2.6 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 KiB/s wr, 15 op/s 2024-04-23T10:28:38.358 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.358+0000 7f824a5e2640 1 -- 172.21.15.97:0/2037501369 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82441010b0 msgr2=0x7f8244101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:28:38.358 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.358+0000 7f824a5e2640 1 --2- 172.21.15.97:0/2037501369 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82441010b0 0x7f8244101490 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f8234009900 tx=0x7f823402a410 comp rx=0 tx=0).stop 2024-04-23T10:28:38.358 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.358+0000 7f824a5e2640 1 -- 172.21.15.97:0/2037501369 shutdown_connections 2024-04-23T10:28:38.358 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.358+0000 7f824a5e2640 1 --2- 172.21.15.97:0/2037501369 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8244106180 0x7f824410dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:38.358 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.358+0000 7f824a5e2640 1 --2- 172.21.15.97:0/2037501369 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8244101a60 0x7f8244105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:38.358 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.358+0000 7f824a5e2640 1 --2- 172.21.15.97:0/2037501369 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82441010b0 0x7f8244101490 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:38.358 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.358+0000 7f824a5e2640 1 -- 172.21.15.97:0/2037501369 >> 172.21.15.97:0/2037501369 conn(0x7f8244078000 msgr2=0x7f8244078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:28:38.358 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.358+0000 7f824a5e2640 1 -- 172.21.15.97:0/2037501369 shutdown_connections 2024-04-23T10:28:38.358 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.358+0000 7f824a5e2640 1 -- 172.21.15.97:0/2037501369 wait complete. 2024-04-23T10:28:38.358 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.359+0000 7f824a5e2640 1 Processor -- start 2024-04-23T10:28:38.359 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.359+0000 7f824a5e2640 1 -- start start 2024-04-23T10:28:38.360 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.359+0000 7f824a5e2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82441010b0 0x7f8244198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:38.360 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.359+0000 7f824a5e2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8244106180 0x7f8244198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:38.360 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.359+0000 7f824a5e2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f824419cfc0 0x7f824419d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:38.360 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.359+0000 7f824a5e2640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8244112200 con 0x7f824419cfc0 2024-04-23T10:28:38.361 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.359+0000 7f824a5e2640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8244112080 con 0x7f82441010b0 2024-04-23T10:28:38.361 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.359+0000 7f824a5e2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8244112380 con 0x7f8244106180 2024-04-23T10:28:38.361 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.359+0000 7f8248b7f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f824419cfc0 0x7f824419d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:38.361 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.359+0000 7f8248b7f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f824419cfc0 0x7f824419d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43344/0 (socket says 172.21.15.97:43344) 2024-04-23T10:28:38.361 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.360+0000 7f8248b7f640 1 -- 172.21.15.97:0/2982939360 learned_addr learned my addr 172.21.15.97:0/2982939360 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:28:38.361 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.360+0000 7f82437fe640 1 --2- 172.21.15.97:0/2982939360 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8244106180 0x7f8244198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:38.361 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.360+0000 7f8243fff640 1 --2- 172.21.15.97:0/2982939360 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82441010b0 0x7f8244198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:38.361 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.360+0000 7f8248b7f640 1 -- 172.21.15.97:0/2982939360 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8244106180 msgr2=0x7f8244198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:28:38.361 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.360+0000 7f8248b7f640 1 --2- 172.21.15.97:0/2982939360 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8244106180 0x7f8244198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:38.361 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.360+0000 7f8248b7f640 1 -- 172.21.15.97:0/2982939360 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82441010b0 msgr2=0x7f8244198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:28:38.361 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.360+0000 7f8248b7f640 1 --2- 172.21.15.97:0/2982939360 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82441010b0 0x7f8244198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:38.361 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.360+0000 7f8248b7f640 1 -- 172.21.15.97:0/2982939360 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f824419dab0 con 0x7f824419cfc0 2024-04-23T10:28:38.361 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.360+0000 7f8243fff640 1 --2- 172.21.15.97:0/2982939360 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82441010b0 0x7f8244198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:28:38.361 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.360+0000 7f8248b7f640 1 --2- 172.21.15.97:0/2982939360 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f824419cfc0 0x7f824419d470 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f823c00eb10 tx=0x7f823c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:28:38.361 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.360+0000 7f82417fa640 1 -- 172.21.15.97:0/2982939360 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823c004b20 con 0x7f824419cfc0 2024-04-23T10:28:38.362 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.360+0000 7f82417fa640 1 -- 172.21.15.97:0/2982939360 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f823c004cc0 con 0x7f824419cfc0 2024-04-23T10:28:38.362 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.360+0000 7f824a5e2640 1 -- 172.21.15.97:0/2982939360 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f824419dda0 con 0x7f824419cfc0 2024-04-23T10:28:38.362 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.360+0000 7f82417fa640 1 -- 172.21.15.97:0/2982939360 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823c010670 con 0x7f824419cfc0 2024-04-23T10:28:38.362 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.360+0000 7f824a5e2640 1 -- 172.21.15.97:0/2982939360 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82441a5540 con 0x7f824419cfc0 2024-04-23T10:28:38.364 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.363+0000 7f824d3d2740 1 Processor -- start 2024-04-23T10:28:38.364 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.363+0000 7f82417fa640 1 -- 172.21.15.97:0/2982939360 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f823c005020 con 0x7f824419cfc0 2024-04-23T10:28:38.364 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.363+0000 7f824d3d2740 1 -- start start 2024-04-23T10:28:38.364 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.363+0000 7f824d3d2740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e1a6288620 0x55e1a6288a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:38.364 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.363+0000 7f824d3d2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e1a6288f40 0x55e1a62932c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:38.365 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.363+0000 7f824d3d2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1a6293800 0x55e1a6295bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:38.365 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.363+0000 7f824d3d2740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e1a61ed970 con 0x55e1a6288620 2024-04-23T10:28:38.365 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.363+0000 7f824d3d2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e1a618a4a0 con 0x55e1a6293800 2024-04-23T10:28:38.365 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.363+0000 7f824d3d2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e1a61e9930 con 0x55e1a6288f40 2024-04-23T10:28:38.365 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.364+0000 7f82437fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e1a6288620 0x55e1a6288a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:38.365 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.364+0000 7f82437fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e1a6288620 0x55e1a6288a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43356/0 (socket says 172.21.15.97:43356) 2024-04-23T10:28:38.365 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.364+0000 7f82437fe640 1 -- 172.21.15.97:0/1608812951 learned_addr learned my addr 172.21.15.97:0/1608812951 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:28:38.365 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.364+0000 7f8243fff640 1 --2- 172.21.15.97:0/1608812951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e1a6288f40 0x55e1a62932c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:38.365 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.364+0000 7f82417fa640 1 --2- 172.21.15.97:0/2982939360 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f822c0a3dc0 0x7f822c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:38.365 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.364+0000 7f82437fe640 1 -- 172.21.15.97:0/1608812951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e1a6288f40 msgr2=0x55e1a62932c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:28:38.365 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.364+0000 7f82437fe640 1 --2- 172.21.15.97:0/1608812951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e1a6288f40 0x55e1a62932c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:38.365 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.364+0000 7f82437fe640 1 -- 172.21.15.97:0/1608812951 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1a6293800 msgr2=0x55e1a6295bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:28:38.365 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.364+0000 7f82437fe640 1 --2- 172.21.15.97:0/1608812951 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1a6293800 0x55e1a6295bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:38.365 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.364+0000 7f82437fe640 1 -- 172.21.15.97:0/1608812951 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1a5f00210 con 0x55e1a6288620 2024-04-23T10:28:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.364+0000 7f8248b7f640 1 --2- 172.21.15.97:0/2982939360 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f822c0a3dc0 0x7f822c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.364+0000 7f82437fe640 1 --2- 172.21.15.97:0/1608812951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e1a6288620 0x55e1a6288a00 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f8238009ce0 tx=0x7f8238013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faed13ccd9ad73c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:28:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.364+0000 7f82417fa640 1 -- 172.21.15.97:0/2982939360 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f823c0d0d60 con 0x7f824419cfc0 2024-04-23T10:28:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.364+0000 7f822a7fc640 1 -- 172.21.15.97:0/1608812951 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8238004050 con 0x55e1a6288620 2024-04-23T10:28:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.364+0000 7f822a7fc640 1 -- 172.21.15.97:0/1608812951 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8238017030 con 0x55e1a6288620 2024-04-23T10:28:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.364+0000 7f822a7fc640 1 -- 172.21.15.97:0/1608812951 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8238010430 con 0x55e1a6288620 2024-04-23T10:28:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.365+0000 7f824d3d2740 1 -- 172.21.15.97:0/1608812951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e1a6288620 msgr2=0x55e1a6288a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:28:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.365+0000 7f8248b7f640 1 --2- 172.21.15.97:0/2982939360 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f822c0a3dc0 0x7f822c0a6280 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f823c09cce0 tx=0x7f823c06f840 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:28:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.365+0000 7f824d3d2740 1 --2- 172.21.15.97:0/1608812951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e1a6288620 0x55e1a6288a00 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f8238009ce0 tx=0x7f8238013830 comp rx=0 tx=0).stop 2024-04-23T10:28:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.365+0000 7f824d3d2740 1 -- 172.21.15.97:0/1608812951 shutdown_connections 2024-04-23T10:28:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.365+0000 7f824d3d2740 1 --2- 172.21.15.97:0/1608812951 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1a6293800 0x55e1a6295bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.365+0000 7f824d3d2740 1 --2- 172.21.15.97:0/1608812951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e1a6288f40 0x55e1a62932c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.365+0000 7f824d3d2740 1 --2- 172.21.15.97:0/1608812951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e1a6288620 0x55e1a6288a00 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.365+0000 7f824d3d2740 1 -- 172.21.15.97:0/1608812951 >> 172.21.15.97:0/1608812951 conn(0x55e1a6284b30 msgr2=0x55e1a6284f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:28:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.365+0000 7f824d3d2740 1 -- 172.21.15.97:0/1608812951 shutdown_connections 2024-04-23T10:28:38.367 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.365+0000 7f824d3d2740 1 -- 172.21.15.97:0/1608812951 wait complete. 2024-04-23T10:28:38.367 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.365+0000 7f824d3d2740 1 -- start start 2024-04-23T10:28:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.366+0000 7f824d3d2740 1 Processor -- start 2024-04-23T10:28:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.366+0000 7f824d3d2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e1a6288620 0x55e1a62930c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.366+0000 7f824d3d2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1a6288f40 0x55e1a62903b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.366+0000 7f824d3d2740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e1a6293800 0x55e1a62908f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.366+0000 7f824d3d2740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e1a6296430 con 0x55e1a6293800 2024-04-23T10:28:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.366+0000 7f824d3d2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e1a61880b0 con 0x55e1a6288f40 2024-04-23T10:28:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.366+0000 7f824d3d2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e1a62965b0 con 0x55e1a6288620 2024-04-23T10:28:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.366+0000 7f824d3d2740 10 client.0 _ll_register_callbacks cb 0x55e1a6175d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:28:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.366+0000 7f82437fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e1a6293800 0x55e1a62908f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.366+0000 7f82437fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e1a6293800 0x55e1a62908f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43358/0 (socket says 172.21.15.97:43358) 2024-04-23T10:28:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.366+0000 7f82437fe640 1 -- 172.21.15.97:0/1842148783 learned_addr learned my addr 172.21.15.97:0/1842148783 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:28:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.366+0000 7f82437fe640 1 -- 172.21.15.97:0/1842148783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e1a6288620 msgr2=0x55e1a62930c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:28:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.366+0000 7f82437fe640 1 --2- 172.21.15.97:0/1842148783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e1a6288620 0x55e1a62930c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.366+0000 7f82437fe640 1 -- 172.21.15.97:0/1842148783 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1a6288f40 msgr2=0x55e1a62903b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:28:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.366+0000 7f82437fe640 1 --2- 172.21.15.97:0/1842148783 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1a6288f40 0x55e1a62903b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.366+0000 7f82437fe640 1 -- 172.21.15.97:0/1842148783 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1a5fcc620 con 0x55e1a6293800 2024-04-23T10:28:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.366+0000 7f82437fe640 1 --2- 172.21.15.97:0/1842148783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e1a6293800 0x55e1a62908f0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f8238003ba0 tx=0x7f8238003aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ff909c62dffd684 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:28:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.366+0000 7f82237fe640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:28:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.367+0000 7f82237fe640 1 -- 172.21.15.97:0/1842148783 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8238003ce0 con 0x55e1a6293800 2024-04-23T10:28:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.367+0000 7f82237fe640 1 -- 172.21.15.97:0/1842148783 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f8238003e80 con 0x55e1a6293800 2024-04-23T10:28:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.367+0000 7f824d3d2740 1 -- 172.21.15.97:0/1842148783 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8238017030 con 0x55e1a6293800 2024-04-23T10:28:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.367+0000 7f82237fe640 1 -- 172.21.15.97:0/1842148783 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f823802d030 con 0x55e1a6293800 2024-04-23T10:28:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.367+0000 7f824d3d2740 10 client.15975 fetch_fsmap learned FSMap version 26 2024-04-23T10:28:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.367+0000 7f824d3d2740 1 -- 172.21.15.97:0/1842148783 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e1a62a22c0 con 0x55e1a6293800 2024-04-23T10:28:38.372 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.367+0000 7f82237fe640 1 -- 172.21.15.97:0/1842148783 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8238017030 con 0x55e1a6293800 2024-04-23T10:28:38.372 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.367+0000 7f82237fe640 1 -- 172.21.15.97:0/1842148783 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f823802b4e0 con 0x55e1a6293800 2024-04-23T10:28:38.372 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.367+0000 7f824d3d2740 10 client.15975 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:28:38.372 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.367+0000 7f824d3d2740 10 client.15975 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:28:38.372 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.367+0000 7f824d3d2740 20 client.15975 populate_metadata read hostname 'smithi097' 2024-04-23T10:28:38.372 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.367+0000 7f824d3d2740 1 --2- 172.21.15.97:0/1842148783 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55e1a62a3670 0x55e1a62a5a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:38.372 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.367+0000 7f824d3d2740 4 client.15975 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:28:38.372 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.367+0000 7f824d3d2740 1 -- 172.21.15.97:0/1842148783 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e1a61fb390 con 0x55e1a62a3670 2024-04-23T10:28:38.372 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.368+0000 7f8243fff640 1 --2- 172.21.15.97:0/1842148783 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55e1a62a3670 0x55e1a62a5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:38.372 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.368+0000 7f82237fe640 10 client.15975 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:28:38.372 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.368+0000 7f8243fff640 1 --2- 172.21.15.97:0/1842148783 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55e1a62a3670 0x55e1a62a5a60 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69e18f2187733909 server_cookie=c87ac990c8a69366 in_seq=0 out_seq=0 2024-04-23T10:28:38.372 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.369+0000 7f82237fe640 1 -- 172.21.15.97:0/1842148783 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e1a61fb390 con 0x55e1a62a3670 2024-04-23T10:28:38.372 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.369+0000 7f82237fe640 10 client.15975 handle_command_reply: tid=0 2024-04-23T10:28:38.372 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.369+0000 7f824d3d2740 1 client.15975 shutdown 2024-04-23T10:28:38.372 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.369+0000 7f824d3d2740 1 -- 172.21.15.97:0/1842148783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e1a6293800 msgr2=0x55e1a62908f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:28:38.372 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.369+0000 7f824d3d2740 1 --2- 172.21.15.97:0/1842148783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e1a6293800 0x55e1a62908f0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f8238003ba0 tx=0x7f8238003aa0 comp rx=0 tx=0).stop 2024-04-23T10:28:38.372 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.369+0000 7f824d3d2740 1 -- 172.21.15.97:0/1842148783 shutdown_connections 2024-04-23T10:28:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.369+0000 7f824d3d2740 1 --2- 172.21.15.97:0/1842148783 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55e1a62a3670 0x55e1a62a5a60 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.369+0000 7f824d3d2740 1 --2- 172.21.15.97:0/1842148783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e1a6293800 0x55e1a62908f0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.369+0000 7f824d3d2740 1 --2- 172.21.15.97:0/1842148783 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1a6288f40 0x55e1a62903b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.369+0000 7f82237fe640 0 client.15975 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:28:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.369+0000 7f824d3d2740 1 --2- 172.21.15.97:0/1842148783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e1a6288620 0x55e1a62930c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.369+0000 7f824d3d2740 1 -- 172.21.15.97:0/1842148783 >> 172.21.15.97:0/1842148783 conn(0x55e1a6284b30 msgr2=0x55e1a628a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:28:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.369+0000 7f824d3d2740 1 -- 172.21.15.97:0/1842148783 shutdown_connections 2024-04-23T10:28:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.369+0000 7f824d3d2740 1 -- 172.21.15.97:0/1842148783 wait complete. 2024-04-23T10:28:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.369+0000 7f824d3d2740 20 client.15975 trim_cache size 0 max 16384 2024-04-23T10:28:38.387 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.384+0000 7f824d3d2740 1 Processor -- start 2024-04-23T10:28:38.388 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.384+0000 7f824d3d2740 1 -- start start 2024-04-23T10:28:38.388 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.384+0000 7f824d3d2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1a6284b30 0x55e1a629f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:38.388 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.384+0000 7f824d3d2740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e1a6288620 0x55e1a62ab1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:38.388 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.384+0000 7f824d3d2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e1a6288f40 0x55e1a62ab710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:38.388 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.384+0000 7f824d3d2740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f82441ada50 con 0x55e1a6288620 2024-04-23T10:28:38.388 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.384+0000 7f824d3d2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f82441af590 con 0x55e1a6284b30 2024-04-23T10:28:38.388 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.384+0000 7f824d3d2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f82441ae120 con 0x55e1a6288f40 2024-04-23T10:28:38.388 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.384+0000 7f8243fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e1a6288620 0x55e1a62ab1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:38.388 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.384+0000 7f8243fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e1a6288620 0x55e1a62ab1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43364/0 (socket says 172.21.15.97:43364) 2024-04-23T10:28:38.388 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.384+0000 7f8243fff640 1 -- 172.21.15.97:0/903373562 learned_addr learned my addr 172.21.15.97:0/903373562 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:28:38.388 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.385+0000 7f8243fff640 1 -- 172.21.15.97:0/903373562 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e1a6288f40 msgr2=0x55e1a62ab710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:28:38.388 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.385+0000 7f8243fff640 1 --2- 172.21.15.97:0/903373562 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e1a6288f40 0x55e1a62ab710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:38.388 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.385+0000 7f8243fff640 1 -- 172.21.15.97:0/903373562 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1a6284b30 msgr2=0x55e1a629f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:28:38.388 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.385+0000 7f82437fe640 1 --2- 172.21.15.97:0/903373562 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1a6284b30 0x55e1a629f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:38.388 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.385+0000 7f8243fff640 1 --2- 172.21.15.97:0/903373562 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1a6284b30 0x55e1a629f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:38.389 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.385+0000 7f8243fff640 1 -- 172.21.15.97:0/903373562 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1a6289a60 con 0x55e1a6288620 2024-04-23T10:28:38.389 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.385+0000 7f82437fe640 1 --2- 172.21.15.97:0/903373562 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1a6284b30 0x55e1a629f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:28:38.389 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.385+0000 7f8243fff640 1 --2- 172.21.15.97:0/903373562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e1a6288620 0x55e1a62ab1d0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f8234031440 tx=0x7f82340058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b11bfd7104cad886 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:28:38.389 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.385+0000 7f8229ffb640 1 -- 172.21.15.97:0/903373562 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8234035040 con 0x55e1a6288620 2024-04-23T10:28:38.389 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.385+0000 7f8229ffb640 1 -- 172.21.15.97:0/903373562 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82340042e0 con 0x55e1a6288620 2024-04-23T10:28:38.389 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.385+0000 7f8229ffb640 1 -- 172.21.15.97:0/903373562 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8234042250 con 0x55e1a6288620 2024-04-23T10:28:38.389 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.386+0000 7f824d3d2740 1 -- 172.21.15.97:0/903373562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e1a6288620 msgr2=0x55e1a62ab1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:28:38.389 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.386+0000 7f824d3d2740 1 --2- 172.21.15.97:0/903373562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e1a6288620 0x55e1a62ab1d0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f8234031440 tx=0x7f82340058b0 comp rx=0 tx=0).stop 2024-04-23T10:28:38.389 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.386+0000 7f824d3d2740 1 -- 172.21.15.97:0/903373562 shutdown_connections 2024-04-23T10:28:38.389 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.386+0000 7f824d3d2740 1 --2- 172.21.15.97:0/903373562 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e1a6288f40 0x55e1a62ab710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:38.389 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.386+0000 7f824d3d2740 1 --2- 172.21.15.97:0/903373562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e1a6288620 0x55e1a62ab1d0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:38.389 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.386+0000 7f824d3d2740 1 --2- 172.21.15.97:0/903373562 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1a6284b30 0x55e1a629f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:38.389 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.386+0000 7f824d3d2740 1 -- 172.21.15.97:0/903373562 >> 172.21.15.97:0/903373562 conn(0x55e1a6290a50 msgr2=0x55e1a6283420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:28:38.389 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.386+0000 7f824d3d2740 1 -- 172.21.15.97:0/903373562 shutdown_connections 2024-04-23T10:28:38.389 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.386+0000 7f824d3d2740 1 -- 172.21.15.97:0/903373562 wait complete. 2024-04-23T10:28:38.389 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.386+0000 7f824d3d2740 1 -- start start 2024-04-23T10:28:38.390 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.386+0000 7f824d3d2740 1 Processor -- start 2024-04-23T10:28:38.390 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.386+0000 7f824d3d2740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e1a6284b30 0x55e1a629ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:38.390 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.386+0000 7f824d3d2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e1a6288620 0x55e1a6298db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:38.390 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.386+0000 7f824d3d2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1a6288f40 0x55e1a62992f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:38.390 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.386+0000 7f824d3d2740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f82441aeec0 con 0x55e1a6284b30 2024-04-23T10:28:38.390 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.386+0000 7f824d3d2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f82441ae7f0 con 0x55e1a6288f40 2024-04-23T10:28:38.390 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.386+0000 7f824d3d2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f82441aba50 con 0x55e1a6288620 2024-04-23T10:28:38.390 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.387+0000 7f824d3d2740 10 client.0 _ll_register_callbacks cb 0x7f8238009eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:28:38.390 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.387+0000 7f82437fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e1a6284b30 0x55e1a629ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:38.390 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.387+0000 7f82437fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e1a6284b30 0x55e1a629ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43366/0 (socket says 172.21.15.97:43366) 2024-04-23T10:28:38.390 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.387+0000 7f82437fe640 1 -- 172.21.15.97:0/189510219 learned_addr learned my addr 172.21.15.97:0/189510219 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:28:38.390 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.387+0000 7f8243fff640 1 --2- 172.21.15.97:0/189510219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e1a6288620 0x55e1a6298db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:38.390 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.387+0000 7f82437fe640 1 --2- 172.21.15.97:0/189510219 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1a6288f40 0x55e1a62992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:38.390 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.387+0000 7f82437fe640 1 -- 172.21.15.97:0/189510219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e1a6288620 msgr2=0x55e1a6298db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:28:38.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.387+0000 7f82437fe640 1 --2- 172.21.15.97:0/189510219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e1a6288620 0x55e1a6298db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:38.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.387+0000 7f82437fe640 1 -- 172.21.15.97:0/189510219 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1a6288f40 msgr2=0x55e1a62992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:28:38.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.387+0000 7f82437fe640 1 --2- 172.21.15.97:0/189510219 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1a6288f40 0x55e1a62992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:38.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.387+0000 7f82437fe640 1 -- 172.21.15.97:0/189510219 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1a6299930 con 0x55e1a6284b30 2024-04-23T10:28:38.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.387+0000 7f82437fe640 1 --2- 172.21.15.97:0/189510219 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e1a6284b30 0x55e1a629ba20 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f8238013d30 tx=0x7f8238003b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82a8b50a996b061a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:28:38.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.387+0000 7f82237fe640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:28:38.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.387+0000 7f82237fe640 1 -- 172.21.15.97:0/189510219 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823802b0b0 con 0x55e1a6284b30 2024-04-23T10:28:38.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.387+0000 7f82237fe640 1 -- 172.21.15.97:0/189510219 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f823802b250 con 0x55e1a6284b30 2024-04-23T10:28:38.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.387+0000 7f824d3d2740 1 -- 172.21.15.97:0/189510219 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82340042e0 con 0x55e1a6284b30 2024-04-23T10:28:38.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.387+0000 7f82237fe640 1 -- 172.21.15.97:0/189510219 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f823802b660 con 0x55e1a6284b30 2024-04-23T10:28:38.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.387+0000 7f824d3d2740 10 client.15981 fetch_fsmap learned FSMap version 26 2024-04-23T10:28:38.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.387+0000 7f824d3d2740 1 -- 172.21.15.97:0/189510219 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e1a6309840 con 0x55e1a6284b30 2024-04-23T10:28:38.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.388+0000 7f82237fe640 1 -- 172.21.15.97:0/189510219 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82340042e0 con 0x55e1a6284b30 2024-04-23T10:28:38.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.388+0000 7f82237fe640 1 -- 172.21.15.97:0/189510219 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8238003e40 con 0x55e1a6284b30 2024-04-23T10:28:38.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.388+0000 7f824d3d2740 10 client.15981 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:28:38.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.388+0000 7f824d3d2740 10 client.15981 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:28:38.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.388+0000 7f824d3d2740 20 client.15981 populate_metadata read hostname 'smithi097' 2024-04-23T10:28:38.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.388+0000 7f824d3d2740 1 --2- 172.21.15.97:0/189510219 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55e1a6293800 0x55e1a630bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:38.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.388+0000 7f824d3d2740 4 client.15981 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:28:38.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.388+0000 7f824d3d2740 1 -- 172.21.15.97:0/189510219 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e1a61f5200 con 0x55e1a6293800 2024-04-23T10:28:38.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.388+0000 7f8243fff640 1 --2- 172.21.15.97:0/189510219 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55e1a6293800 0x55e1a630bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:38.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.389+0000 7f82237fe640 10 client.15981 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:28:38.392 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.389+0000 7f8243fff640 1 --2- 172.21.15.97:0/189510219 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55e1a6293800 0x55e1a630bfd0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d73940db82c07f34 server_cookie=30df972e8aff345f in_seq=0 out_seq=0 2024-04-23T10:28:38.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.403+0000 7f82237fe640 1 -- 172.21.15.97:0/189510219 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e1a61f5200 con 0x55e1a6293800 2024-04-23T10:28:38.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.403+0000 7f82237fe640 10 client.15981 handle_command_reply: tid=0 2024-04-23T10:28:38.403 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.403+0000 7f824d3d2740 1 client.15981 shutdown 2024-04-23T10:28:38.403 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.403+0000 7f824d3d2740 1 -- 172.21.15.97:0/189510219 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e1a6284b30 msgr2=0x55e1a629ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:28:38.403 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.403+0000 7f824d3d2740 1 --2- 172.21.15.97:0/189510219 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e1a6284b30 0x55e1a629ba20 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f8238013d30 tx=0x7f8238003b10 comp rx=0 tx=0).stop 2024-04-23T10:28:38.403 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.403+0000 7f824d3d2740 1 -- 172.21.15.97:0/189510219 shutdown_connections 2024-04-23T10:28:38.403 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.403+0000 7f824d3d2740 1 --2- 172.21.15.97:0/189510219 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55e1a6293800 0x55e1a630bfd0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:38.403 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.403+0000 7f824d3d2740 1 --2- 172.21.15.97:0/189510219 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1a6288f40 0x55e1a62992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:38.403 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.403+0000 7f824d3d2740 1 --2- 172.21.15.97:0/189510219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e1a6288620 0x55e1a6298db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:38.403 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.403+0000 7f82237fe640 0 client.15981 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:28:38.403 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.403+0000 7f824d3d2740 1 --2- 172.21.15.97:0/189510219 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e1a6284b30 0x55e1a629ba20 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:38.403 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.403+0000 7f824d3d2740 1 -- 172.21.15.97:0/189510219 >> 172.21.15.97:0/189510219 conn(0x55e1a6290a50 msgr2=0x55e1a6283cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:28:38.403 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.403+0000 7f824d3d2740 1 -- 172.21.15.97:0/189510219 shutdown_connections 2024-04-23T10:28:38.403 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.403+0000 7f824d3d2740 1 -- 172.21.15.97:0/189510219 wait complete. 2024-04-23T10:28:38.403 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.403+0000 7f824d3d2740 20 client.15981 trim_cache size 0 max 16384 2024-04-23T10:28:38.404 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:28:38.404 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:28:38.404 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:28:38.404 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:28:38.404 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.404+0000 7f824a5e2640 1 -- 172.21.15.97:0/2982939360 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f822c0a3dc0 msgr2=0x7f822c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:28:38.405 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.404+0000 7f824a5e2640 1 --2- 172.21.15.97:0/2982939360 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f822c0a3dc0 0x7f822c0a6280 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f823c09cce0 tx=0x7f823c06f840 comp rx=0 tx=0).stop 2024-04-23T10:28:38.405 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.404+0000 7f824a5e2640 1 -- 172.21.15.97:0/2982939360 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f824419cfc0 msgr2=0x7f824419d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:28:38.405 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.404+0000 7f824a5e2640 1 --2- 172.21.15.97:0/2982939360 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f824419cfc0 0x7f824419d470 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f823c00eb10 tx=0x7f823c00c2d0 comp rx=0 tx=0).stop 2024-04-23T10:28:38.405 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.404+0000 7f824a5e2640 1 -- 172.21.15.97:0/2982939360 shutdown_connections 2024-04-23T10:28:38.405 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.404+0000 7f824a5e2640 1 --2- 172.21.15.97:0/2982939360 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f822c0a3dc0 0x7f822c0a6280 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:38.405 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.404+0000 7f824a5e2640 1 --2- 172.21.15.97:0/2982939360 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f824419cfc0 0x7f824419d470 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:38.405 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.404+0000 7f824a5e2640 1 --2- 172.21.15.97:0/2982939360 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8244106180 0x7f8244198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:38.405 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.404+0000 7f824a5e2640 1 --2- 172.21.15.97:0/2982939360 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f82441010b0 0x7f8244198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:38.405 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.404+0000 7f824a5e2640 1 -- 172.21.15.97:0/2982939360 >> 172.21.15.97:0/2982939360 conn(0x7f8244078000 msgr2=0x7f82440fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:28:38.405 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.404+0000 7f824a5e2640 1 -- 172.21.15.97:0/2982939360 shutdown_connections 2024-04-23T10:28:38.405 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:38.404+0000 7f824a5e2640 1 -- 172.21.15.97:0/2982939360 wait complete. 2024-04-23T10:28:39.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:28:39 smithi196 ceph-mon[32919]: pgmap v76: 97 pgs: 97 active+clean; 2.7 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 31 KiB/s wr, 22 op/s 2024-04-23T10:28:39.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:28:39 smithi097 ceph-mon[29197]: pgmap v76: 97 pgs: 97 active+clean; 2.7 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 31 KiB/s wr, 22 op/s 2024-04-23T10:28:39.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:28:39 smithi183 ceph-mon[32317]: pgmap v76: 97 pgs: 97 active+clean; 2.7 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 31 KiB/s wr, 22 op/s 2024-04-23T10:28:41.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:28:41 smithi196 ceph-mon[32919]: pgmap v77: 97 pgs: 97 active+clean; 2.7 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-04-23T10:28:41.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:28:41 smithi097 ceph-mon[29197]: pgmap v77: 97 pgs: 97 active+clean; 2.7 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-04-23T10:28:41.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:28:41 smithi183 ceph-mon[32317]: pgmap v77: 97 pgs: 97 active+clean; 2.7 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-04-23T10:28:43.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:28:43 smithi196 ceph-mon[32919]: pgmap v78: 97 pgs: 97 active+clean; 2.7 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-04-23T10:28:43.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:28:43 smithi097 ceph-mon[29197]: pgmap v78: 97 pgs: 97 active+clean; 2.7 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-04-23T10:28:43.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:28:43 smithi183 ceph-mon[32317]: pgmap v78: 97 pgs: 97 active+clean; 2.7 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 13 op/s 2024-04-23T10:28:45.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:28:45 smithi196 ceph-mon[32919]: pgmap v79: 97 pgs: 97 active+clean; 2.8 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 31 KiB/s wr, 22 op/s 2024-04-23T10:28:45.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:28:45 smithi097 ceph-mon[29197]: pgmap v79: 97 pgs: 97 active+clean; 2.8 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 31 KiB/s wr, 22 op/s 2024-04-23T10:28:45.399 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:28:45 smithi183 ceph-mon[32317]: pgmap v79: 97 pgs: 97 active+clean; 2.8 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 31 KiB/s wr, 22 op/s 2024-04-23T10:28:47.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:28:47 smithi196 ceph-mon[32919]: pgmap v80: 97 pgs: 97 active+clean; 2.8 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-04-23T10:28:47.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:28:47 smithi097 ceph-mon[29197]: pgmap v80: 97 pgs: 97 active+clean; 2.8 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-04-23T10:28:47.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:28:47 smithi183 ceph-mon[32317]: pgmap v80: 97 pgs: 97 active+clean; 2.8 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-04-23T10:28:49.285 INFO:tasks.quiescer.fs.[cephfs]:Total duration (59.8) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-04-23T10:28:49.285 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 29.9 2024-04-23T10:28:49.286 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T10:28:49.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:28:49 smithi196 ceph-mon[32919]: pgmap v81: 97 pgs: 97 active+clean; 2.9 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 30 KiB/s wr, 22 op/s 2024-04-23T10:28:49.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:28:49 smithi097 ceph-mon[29197]: pgmap v81: 97 pgs: 97 active+clean; 2.9 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 30 KiB/s wr, 22 op/s 2024-04-23T10:28:49.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.399+0000 7f27c9fbf640 1 -- 172.21.15.97:0/1885880081 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27c4103230 msgr2=0x7f27c4103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:28:49.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.399+0000 7f27c9fbf640 1 --2- 172.21.15.97:0/1885880081 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27c4103230 0x7f27c4103610 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f27b4009900 tx=0x7f27b402a480 comp rx=0 tx=0).stop 2024-04-23T10:28:49.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.399+0000 7f27c9fbf640 1 -- 172.21.15.97:0/1885880081 shutdown_connections 2024-04-23T10:28:49.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.399+0000 7f27c9fbf640 1 --2- 172.21.15.97:0/1885880081 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27c410c570 0x7f27c410e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:49.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.399+0000 7f27c9fbf640 1 --2- 172.21.15.97:0/1885880081 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27c4103b50 0x7f27c410c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:49.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.399+0000 7f27c9fbf640 1 --2- 172.21.15.97:0/1885880081 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27c4103230 0x7f27c4103610 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:49.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.399+0000 7f27c9fbf640 1 -- 172.21.15.97:0/1885880081 >> 172.21.15.97:0/1885880081 conn(0x7f27c4077fb0 msgr2=0x7f27c40783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:28:49.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.399+0000 7f27c9fbf640 1 -- 172.21.15.97:0/1885880081 shutdown_connections 2024-04-23T10:28:49.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.400+0000 7f27c9fbf640 1 -- 172.21.15.97:0/1885880081 wait complete. 2024-04-23T10:28:49.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.400+0000 7f27c9fbf640 1 Processor -- start 2024-04-23T10:28:49.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.400+0000 7f27c9fbf640 1 -- start start 2024-04-23T10:28:49.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.400+0000 7f27c9fbf640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27c4103230 0x7f27c4198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:49.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.400+0000 7f27c9fbf640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27c4103b50 0x7f27c4198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:49.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.400+0000 7f27c9fbf640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27c410c570 0x7f27c419cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:49.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.400+0000 7f27c9fbf640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f27c4112230 con 0x7f27c4103230 2024-04-23T10:28:49.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.400+0000 7f27c9fbf640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f27c41120b0 con 0x7f27c4103b50 2024-04-23T10:28:49.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.400+0000 7f27c9fbf640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f27c41123b0 con 0x7f27c410c570 2024-04-23T10:28:49.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.400+0000 7f27c37fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27c4103230 0x7f27c4198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:49.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.400+0000 7f27c37fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27c4103230 0x7f27c4198770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56538/0 (socket says 172.21.15.97:56538) 2024-04-23T10:28:49.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.400+0000 7f27c37fe640 1 -- 172.21.15.97:0/4157449124 learned_addr learned my addr 172.21.15.97:0/4157449124 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:28:49.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.401+0000 7f27c3fff640 1 --2- 172.21.15.97:0/4157449124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27c410c570 0x7f27c419cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:49.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.401+0000 7f27bbfff640 1 --2- 172.21.15.97:0/4157449124 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27c4103b50 0x7f27c4198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:49.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.401+0000 7f27c37fe640 1 -- 172.21.15.97:0/4157449124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27c410c570 msgr2=0x7f27c419cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:28:49.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.401+0000 7f27c37fe640 1 --2- 172.21.15.97:0/4157449124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27c410c570 0x7f27c419cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:49.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.401+0000 7f27c37fe640 1 -- 172.21.15.97:0/4157449124 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27c4103b50 msgr2=0x7f27c4198cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:28:49.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.401+0000 7f27c37fe640 1 --2- 172.21.15.97:0/4157449124 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27c4103b50 0x7f27c4198cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:49.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.401+0000 7f27c37fe640 1 -- 172.21.15.97:0/4157449124 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27c419d710 con 0x7f27c4103230 2024-04-23T10:28:49.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.401+0000 7f27c3fff640 1 --2- 172.21.15.97:0/4157449124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27c410c570 0x7f27c419cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:28:49.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.401+0000 7f27bbfff640 1 --2- 172.21.15.97:0/4157449124 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27c4103b50 0x7f27c4198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:28:49.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.401+0000 7f27c37fe640 1 --2- 172.21.15.97:0/4157449124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27c4103230 0x7f27c4198770 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f27b4006010 tx=0x7f27b40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:28:49.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.401+0000 7f27c17fa640 1 -- 172.21.15.97:0/4157449124 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b403f030 con 0x7f27c4103230 2024-04-23T10:28:49.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.401+0000 7f27c9fbf640 1 -- 172.21.15.97:0/4157449124 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27c419d9a0 con 0x7f27c4103230 2024-04-23T10:28:49.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.401+0000 7f27c17fa640 1 -- 172.21.15.97:0/4157449124 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27b4003800 con 0x7f27c4103230 2024-04-23T10:28:49.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.401+0000 7f27c17fa640 1 -- 172.21.15.97:0/4157449124 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b4035070 con 0x7f27c4103230 2024-04-23T10:28:49.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.401+0000 7f27c9fbf640 1 -- 172.21.15.97:0/4157449124 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27c41a5290 con 0x7f27c4103230 2024-04-23T10:28:49.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.402+0000 7f27c9fbf640 1 -- 172.21.15.97:0/4157449124 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27c40691c0 con 0x7f27c4103230 2024-04-23T10:28:49.404 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:28:49 smithi183 ceph-mon[32317]: pgmap v81: 97 pgs: 97 active+clean; 2.9 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 30 KiB/s wr, 22 op/s 2024-04-23T10:28:49.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.403+0000 7f27c17fa640 1 -- 172.21.15.97:0/4157449124 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f27b4003cb0 con 0x7f27c4103230 2024-04-23T10:28:49.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.404+0000 7f27c17fa640 1 --2- 172.21.15.97:0/4157449124 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f27b00a3d20 0x7f27b00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:49.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.404+0000 7f27bbfff640 1 --2- 172.21.15.97:0/4157449124 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f27b00a3d20 0x7f27b00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:49.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.404+0000 7f27c17fa640 1 -- 172.21.15.97:0/4157449124 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f27b40f11f0 con 0x7f27c4103230 2024-04-23T10:28:49.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.404+0000 7f27bbfff640 1 --2- 172.21.15.97:0/4157449124 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f27b00a3d20 0x7f27b00a61e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f27c410e220 tx=0x7f27ac00a5c0 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:28:49.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.409+0000 7f27c17fa640 1 -- 172.21.15.97:0/4157449124 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f27b40bd8e0 con 0x7f27c4103230 2024-04-23T10:28:49.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.625+0000 7f27c9fbf640 1 -- 172.21.15.97:0/4157449124 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f27c41076f0 con 0x7f27c4103230 2024-04-23T10:28:49.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.625+0000 7f27c17fa640 1 -- 172.21.15.97:0/4157449124 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11931 (secure 0 0 0) 0x7f27b40bd1e0 con 0x7f27c4103230 2024-04-23T10:28:49.625 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:28:49.625 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3946878762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3946878762},{"type":"v1","addr":"172.21.15.97:6835","nonce":3946878762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/209800861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":209800861},{"type":"v1","addr":"172.21.15.97:6837","nonce":209800861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T10:24:06.910134+0000","modified":"2024-04-23T10:26:09.820985+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6839/418013575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":418013575},{"type":"v1","addr":"172.21.15.97:6839","nonce":418013575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6841/747242680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":747242680},{"type":"v1","addr":"172.21.15.97:6841","nonce":747242680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6833/4292847121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4292847121},{"type":"v1","addr":"172.21.15.183:6833","nonce":4292847121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6833/3885816565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3885816565},{"type":"v1","addr":"172.21.15.196:6833","nonce":3885816565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1897093105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1897093105},{"type":"v1","addr":"172.21.15.183:6835","nonce":1897093105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1171477563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1171477563},{"type":"v1","addr":"172.21.15.196:6835","nonce":1171477563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1898341911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1898341911},{"type":"v1","addr":"172.21.15.183:6837","nonce":1898341911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.196:6837/3094946502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3094946502},{"type":"v1","addr":"172.21.15.196:6837","nonce":3094946502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.196:6839/83843556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":83843556},{"type":"v1","addr":"172.21.15.196:6839","nonce":83843556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6839/1763841130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1763841130},{"type":"v1","addr":"172.21.15.183:6839","nonce":1763841130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24487,24403,24398,24469,24488]},"id":1}]} 2024-04-23T10:28:49.625 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 26 2024-04-23T10:28:49.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.627+0000 7f27c9fbf640 1 -- 172.21.15.97:0/4157449124 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f27b00a3d20 msgr2=0x7f27b00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:28:49.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.627+0000 7f27c9fbf640 1 --2- 172.21.15.97:0/4157449124 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f27b00a3d20 0x7f27b00a61e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f27c410e220 tx=0x7f27ac00a5c0 comp rx=0 tx=0).stop 2024-04-23T10:28:49.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.627+0000 7f27c9fbf640 1 -- 172.21.15.97:0/4157449124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27c4103230 msgr2=0x7f27c4198770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:28:49.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.627+0000 7f27c9fbf640 1 --2- 172.21.15.97:0/4157449124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27c4103230 0x7f27c4198770 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f27b4006010 tx=0x7f27b40042f0 comp rx=0 tx=0).stop 2024-04-23T10:28:49.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.627+0000 7f27c9fbf640 1 -- 172.21.15.97:0/4157449124 shutdown_connections 2024-04-23T10:28:49.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.627+0000 7f27c9fbf640 1 --2- 172.21.15.97:0/4157449124 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f27b00a3d20 0x7f27b00a61e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:49.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.627+0000 7f27c9fbf640 1 --2- 172.21.15.97:0/4157449124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27c410c570 0x7f27c419cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:49.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.627+0000 7f27c9fbf640 1 --2- 172.21.15.97:0/4157449124 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27c4103b50 0x7f27c4198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:49.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.627+0000 7f27c9fbf640 1 --2- 172.21.15.97:0/4157449124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27c4103230 0x7f27c4198770 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:49.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.627+0000 7f27c9fbf640 1 -- 172.21.15.97:0/4157449124 >> 172.21.15.97:0/4157449124 conn(0x7f27c4077fb0 msgr2=0x7f27c40fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:28:49.627 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.628+0000 7f27c9fbf640 1 -- 172.21.15.97:0/4157449124 shutdown_connections 2024-04-23T10:28:49.628 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.628+0000 7f27c9fbf640 1 -- 172.21.15.97:0/4157449124 wait complete. 2024-04-23T10:28:49.638 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db / --timeout 60 --expiration 89.9 --await-for 42.0' 2024-04-23T10:28:49.638 DEBUG:teuthology.orchestra.run.smithi097:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24488 quiesce db / --timeout 60 --expiration 89.9 --await-for 42.0 2024-04-23T10:28:49.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.747+0000 7f0b84920640 1 -- 172.21.15.97:0/3473354800 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b80106170 msgr2=0x7f0b8010dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:28:49.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.747+0000 7f0b84920640 1 --2- 172.21.15.97:0/3473354800 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b80106170 0x7f0b8010dd40 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f0b78009900 tx=0x7f0b7802a410 comp rx=0 tx=0).stop 2024-04-23T10:28:49.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.747+0000 7f0b84920640 1 -- 172.21.15.97:0/3473354800 shutdown_connections 2024-04-23T10:28:49.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.747+0000 7f0b84920640 1 --2- 172.21.15.97:0/3473354800 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b80106170 0x7f0b8010dd40 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:49.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.747+0000 7f0b84920640 1 --2- 172.21.15.97:0/3473354800 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b80101a50 0x7f0b80105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:49.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.747+0000 7f0b84920640 1 --2- 172.21.15.97:0/3473354800 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b801010a0 0x7f0b80101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:49.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.747+0000 7f0b84920640 1 -- 172.21.15.97:0/3473354800 >> 172.21.15.97:0/3473354800 conn(0x7f0b80077f50 msgr2=0x7f0b80078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:28:49.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.748+0000 7f0b84920640 1 -- 172.21.15.97:0/3473354800 shutdown_connections 2024-04-23T10:28:49.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.748+0000 7f0b84920640 1 -- 172.21.15.97:0/3473354800 wait complete. 2024-04-23T10:28:49.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.748+0000 7f0b84920640 1 Processor -- start 2024-04-23T10:28:49.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.748+0000 7f0b84920640 1 -- start start 2024-04-23T10:28:49.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.748+0000 7f0b84920640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b801010a0 0x7f0b80197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:49.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.748+0000 7f0b84920640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b80106170 0x7f0b80197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:49.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.748+0000 7f0b84920640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b8019bec0 0x7f0b8019c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:49.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.748+0000 7f0b84920640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0b801110e0 con 0x7f0b801010a0 2024-04-23T10:28:49.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.748+0000 7f0b84920640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0b80110f60 con 0x7f0b80106170 2024-04-23T10:28:49.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.748+0000 7f0b84920640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0b80111260 con 0x7f0b8019bec0 2024-04-23T10:28:49.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.749+0000 7f0b7e59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b801010a0 0x7f0b80197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:49.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.749+0000 7f0b7e59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b801010a0 0x7f0b80197600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56556/0 (socket says 172.21.15.97:56556) 2024-04-23T10:28:49.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.749+0000 7f0b7e59c640 1 -- 172.21.15.97:0/2519407813 learned_addr learned my addr 172.21.15.97:0/2519407813 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:28:49.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.749+0000 7f0b7ed9d640 1 --2- 172.21.15.97:0/2519407813 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b8019bec0 0x7f0b8019c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:49.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.749+0000 7f0b7dd9b640 1 --2- 172.21.15.97:0/2519407813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b80106170 0x7f0b80197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:49.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.749+0000 7f0b7e59c640 1 -- 172.21.15.97:0/2519407813 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b8019bec0 msgr2=0x7f0b8019c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:28:49.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.749+0000 7f0b7e59c640 1 --2- 172.21.15.97:0/2519407813 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b8019bec0 0x7f0b8019c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:49.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.749+0000 7f0b7e59c640 1 -- 172.21.15.97:0/2519407813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b80106170 msgr2=0x7f0b80197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:28:49.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.749+0000 7f0b7e59c640 1 --2- 172.21.15.97:0/2519407813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b80106170 0x7f0b80197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:49.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.749+0000 7f0b7e59c640 1 -- 172.21.15.97:0/2519407813 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b8019c9b0 con 0x7f0b801010a0 2024-04-23T10:28:49.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.749+0000 7f0b7e59c640 1 --2- 172.21.15.97:0/2519407813 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b801010a0 0x7f0b80197600 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f0b7000ea80 tx=0x7f0b7000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:28:49.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.749+0000 7f0b6f7fe640 1 -- 172.21.15.97:0/2519407813 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7000ce40 con 0x7f0b801010a0 2024-04-23T10:28:49.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.749+0000 7f0b6f7fe640 1 -- 172.21.15.97:0/2519407813 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b70005020 con 0x7f0b801010a0 2024-04-23T10:28:49.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.749+0000 7f0b84920640 1 -- 172.21.15.97:0/2519407813 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b8019cca0 con 0x7f0b801010a0 2024-04-23T10:28:49.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.749+0000 7f0b6f7fe640 1 -- 172.21.15.97:0/2519407813 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b700116d0 con 0x7f0b801010a0 2024-04-23T10:28:49.752 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.749+0000 7f0b84920640 1 -- 172.21.15.97:0/2519407813 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b801a4440 con 0x7f0b801010a0 2024-04-23T10:28:49.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.751+0000 7f0b6f7fe640 1 -- 172.21.15.97:0/2519407813 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0b700054d0 con 0x7f0b801010a0 2024-04-23T10:28:49.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.751+0000 7f0b87636740 1 Processor -- start 2024-04-23T10:28:49.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.752+0000 7f0b87636740 1 -- start start 2024-04-23T10:28:49.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.752+0000 7f0b87636740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7c0fba0a0 0x55b7c0fba480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:49.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.752+0000 7f0b87636740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b7c0fba9c0 0x55b7c0fc4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:49.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.752+0000 7f0b87636740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b7c0fc5090 0x55b7c0fc7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:49.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.752+0000 7f0b87636740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55b7c0f1f370 con 0x55b7c0fc5090 2024-04-23T10:28:49.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.752+0000 7f0b6f7fe640 1 --2- 172.21.15.97:0/2519407813 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f0b680a3d20 0x7f0b680a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:49.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.752+0000 7f0b87636740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b7c0ebc2e0 con 0x55b7c0fba9c0 2024-04-23T10:28:49.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.752+0000 7f0b87636740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b7c0f1b330 con 0x55b7c0fba0a0 2024-04-23T10:28:49.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.752+0000 7f0b7dd9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b7c0fc5090 0x55b7c0fc7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:49.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.752+0000 7f0b7ed9d640 1 --2- 172.21.15.97:0/2519407813 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f0b680a3d20 0x7f0b680a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:49.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.752+0000 7f0b7dd9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b7c0fc5090 0x55b7c0fc7480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56568/0 (socket says 172.21.15.97:56568) 2024-04-23T10:28:49.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.752+0000 7f0b7dd9b640 1 -- 172.21.15.97:0/3551202020 learned_addr learned my addr 172.21.15.97:0/3551202020 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:28:49.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.752+0000 7f0b7ed9d640 1 --2- 172.21.15.97:0/3551202020 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7c0fba0a0 0x55b7c0fba480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:49.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.752+0000 7f0b6f7fe640 1 -- 172.21.15.97:0/2519407813 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f0b700d0a90 con 0x7f0b801010a0 2024-04-23T10:28:49.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.752+0000 7f0b7e59c640 1 --2- 172.21.15.97:0/3551202020 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b7c0fba9c0 0x55b7c0fc4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:49.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.752+0000 7f0b7dd9b640 1 -- 172.21.15.97:0/3551202020 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7c0fba0a0 msgr2=0x55b7c0fba480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:28:49.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.752+0000 7f0b7dd9b640 1 --2- 172.21.15.97:0/3551202020 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7c0fba0a0 0x55b7c0fba480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:49.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.752+0000 7f0b7dd9b640 1 -- 172.21.15.97:0/3551202020 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b7c0fba9c0 msgr2=0x55b7c0fc4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:28:49.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.752+0000 7f0b7dd9b640 1 --2- 172.21.15.97:0/3551202020 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b7c0fba9c0 0x55b7c0fc4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:49.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.752+0000 7f0b7dd9b640 1 -- 172.21.15.97:0/3551202020 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7c0cfdd80 con 0x55b7c0fc5090 2024-04-23T10:28:49.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.752+0000 7f0b7ed9d640 1 --2- 172.21.15.97:0/3551202020 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7c0fba0a0 0x55b7c0fba480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:28:49.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.753+0000 7f0b7dd9b640 1 --2- 172.21.15.97:0/3551202020 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b7c0fc5090 0x55b7c0fc7480 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f0b7400bbe0 tx=0x7f0b7400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ba4c4cc2e57a1c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:28:49.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.753+0000 7f0b7ed9d640 1 --2- 172.21.15.97:0/2519407813 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f0b680a3d20 0x7f0b680a61e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0b78006010 tx=0x7f0b78031400 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:28:49.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.753+0000 7f0b6cff9640 1 -- 172.21.15.97:0/3551202020 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b74015070 con 0x55b7c0fc5090 2024-04-23T10:28:49.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.753+0000 7f0b6cff9640 1 -- 172.21.15.97:0/3551202020 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b740118f0 con 0x55b7c0fc5090 2024-04-23T10:28:49.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.753+0000 7f0b6cff9640 1 -- 172.21.15.97:0/3551202020 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b74010730 con 0x55b7c0fc5090 2024-04-23T10:28:49.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.753+0000 7f0b87636740 1 -- 172.21.15.97:0/3551202020 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b7c0fc5090 msgr2=0x55b7c0fc7480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:28:49.761 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.753+0000 7f0b87636740 1 --2- 172.21.15.97:0/3551202020 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b7c0fc5090 0x55b7c0fc7480 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f0b7400bbe0 tx=0x7f0b7400f730 comp rx=0 tx=0).stop 2024-04-23T10:28:49.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.753+0000 7f0b87636740 1 -- 172.21.15.97:0/3551202020 shutdown_connections 2024-04-23T10:28:49.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.753+0000 7f0b87636740 1 --2- 172.21.15.97:0/3551202020 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b7c0fc5090 0x55b7c0fc7480 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:49.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.753+0000 7f0b87636740 1 --2- 172.21.15.97:0/3551202020 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b7c0fba9c0 0x55b7c0fc4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:49.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.753+0000 7f0b87636740 1 --2- 172.21.15.97:0/3551202020 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7c0fba0a0 0x55b7c0fba480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:49.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.753+0000 7f0b87636740 1 -- 172.21.15.97:0/3551202020 >> 172.21.15.97:0/3551202020 conn(0x55b7c0fb63c0 msgr2=0x55b7c0fb67a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:28:49.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.754+0000 7f0b87636740 1 -- 172.21.15.97:0/3551202020 shutdown_connections 2024-04-23T10:28:49.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.754+0000 7f0b87636740 1 -- 172.21.15.97:0/3551202020 wait complete. 2024-04-23T10:28:49.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.754+0000 7f0b87636740 1 -- start start 2024-04-23T10:28:49.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.754+0000 7f0b87636740 1 Processor -- start 2024-04-23T10:28:49.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.754+0000 7f0b87636740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b7c0fba0a0 0x55b7c0fc8290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:49.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.754+0000 7f0b87636740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7c0fba9c0 0x55b7c0fc87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:49.762 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.754+0000 7f0b87636740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b7c0fc5090 0x55b7c0fc8d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:49.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.754+0000 7f0b87636740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55b7c0fc7e80 con 0x55b7c0fba0a0 2024-04-23T10:28:49.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.754+0000 7f0b87636740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b7c0eb9fb0 con 0x55b7c0fc5090 2024-04-23T10:28:49.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.754+0000 7f0b87636740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55b7c0fc8000 con 0x55b7c0fba9c0 2024-04-23T10:28:49.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.754+0000 7f0b87636740 10 client.0 _ll_register_callbacks cb 0x55b7c0eb99a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:28:49.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.754+0000 7f0b7e59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b7c0fba0a0 0x55b7c0fc8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:49.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.754+0000 7f0b7e59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b7c0fba0a0 0x55b7c0fc8290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56570/0 (socket says 172.21.15.97:56570) 2024-04-23T10:28:49.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.754+0000 7f0b7e59c640 1 -- 172.21.15.97:0/933057952 learned_addr learned my addr 172.21.15.97:0/933057952 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:28:49.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.755+0000 7f0b7e59c640 1 -- 172.21.15.97:0/933057952 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7c0fba9c0 msgr2=0x55b7c0fc87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:28:49.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.755+0000 7f0b7e59c640 1 --2- 172.21.15.97:0/933057952 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7c0fba9c0 0x55b7c0fc87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:49.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.755+0000 7f0b7e59c640 1 -- 172.21.15.97:0/933057952 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b7c0fc5090 msgr2=0x55b7c0fc8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:28:49.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.755+0000 7f0b7e59c640 1 --2- 172.21.15.97:0/933057952 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b7c0fc5090 0x55b7c0fc8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:49.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.755+0000 7f0b7e59c640 1 -- 172.21.15.97:0/933057952 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7c0d5e510 con 0x55b7c0fba0a0 2024-04-23T10:28:49.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.755+0000 7f0b7e59c640 1 --2- 172.21.15.97:0/933057952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b7c0fba0a0 0x55b7c0fc8290 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f0b70009270 tx=0x7f0b7009c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=945d24120c6406ee server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:28:49.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.755+0000 7f0b65ffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:28:49.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.755+0000 7f0b65ffb640 1 -- 172.21.15.97:0/933057952 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7001f050 con 0x55b7c0fba0a0 2024-04-23T10:28:49.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.755+0000 7f0b65ffb640 1 -- 172.21.15.97:0/933057952 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f0b7009bed0 con 0x55b7c0fba0a0 2024-04-23T10:28:49.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.755+0000 7f0b87636740 1 -- 172.21.15.97:0/933057952 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b740118f0 con 0x55b7c0fba0a0 2024-04-23T10:28:49.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.755+0000 7f0b65ffb640 1 -- 172.21.15.97:0/933057952 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0b700a0e30 con 0x55b7c0fba0a0 2024-04-23T10:28:49.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.755+0000 7f0b87636740 10 client.15999 fetch_fsmap learned FSMap version 26 2024-04-23T10:28:49.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.755+0000 7f0b87636740 1 -- 172.21.15.97:0/933057952 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b7c0bf1670 con 0x55b7c0fba0a0 2024-04-23T10:28:49.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.755+0000 7f0b65ffb640 1 -- 172.21.15.97:0/933057952 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b740118f0 con 0x55b7c0fba0a0 2024-04-23T10:28:49.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.755+0000 7f0b65ffb640 1 -- 172.21.15.97:0/933057952 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f0b7000f040 con 0x55b7c0fba0a0 2024-04-23T10:28:49.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.755+0000 7f0b87636740 10 client.15999 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:28:49.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.755+0000 7f0b87636740 10 client.15999 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:28:49.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.755+0000 7f0b87636740 20 client.15999 populate_metadata read hostname 'smithi097' 2024-04-23T10:28:49.764 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.755+0000 7f0b87636740 1 --2- 172.21.15.97:0/933057952 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55b7c0fd76a0 0x55b7c0fd9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:49.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.755+0000 7f0b87636740 4 client.15999 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:28:49.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.755+0000 7f0b87636740 1 -- 172.21.15.97:0/933057952 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b7c0f2cd90 con 0x55b7c0fd76a0 2024-04-23T10:28:49.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.756+0000 7f0b7dd9b640 1 --2- 172.21.15.97:0/933057952 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55b7c0fd76a0 0x55b7c0fd9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:49.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.756+0000 7f0b65ffb640 10 client.15999 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:28:49.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.756+0000 7f0b7dd9b640 1 --2- 172.21.15.97:0/933057952 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55b7c0fd76a0 0x55b7c0fd9af0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcbeb39da50c9478 server_cookie=2efd644fbf9460fa in_seq=0 out_seq=0 2024-04-23T10:28:49.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.757+0000 7f0b65ffb640 1 -- 172.21.15.97:0/933057952 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b7c0f2cd90 con 0x55b7c0fd76a0 2024-04-23T10:28:49.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.757+0000 7f0b65ffb640 10 client.15999 handle_command_reply: tid=0 2024-04-23T10:28:49.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.757+0000 7f0b87636740 1 client.15999 shutdown 2024-04-23T10:28:49.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.757+0000 7f0b87636740 1 -- 172.21.15.97:0/933057952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b7c0fba0a0 msgr2=0x55b7c0fc8290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:28:49.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.757+0000 7f0b87636740 1 --2- 172.21.15.97:0/933057952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b7c0fba0a0 0x55b7c0fc8290 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f0b70009270 tx=0x7f0b7009c4a0 comp rx=0 tx=0).stop 2024-04-23T10:28:49.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.757+0000 7f0b87636740 1 -- 172.21.15.97:0/933057952 shutdown_connections 2024-04-23T10:28:49.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.757+0000 7f0b87636740 1 --2- 172.21.15.97:0/933057952 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55b7c0fd76a0 0x55b7c0fd9af0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:49.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.757+0000 7f0b87636740 1 --2- 172.21.15.97:0/933057952 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b7c0fc5090 0x55b7c0fc8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:49.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.757+0000 7f0b87636740 1 --2- 172.21.15.97:0/933057952 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7c0fba9c0 0x55b7c0fc87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:49.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.757+0000 7f0b65ffb640 0 client.15999 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:28:49.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.757+0000 7f0b87636740 1 --2- 172.21.15.97:0/933057952 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b7c0fba0a0 0x55b7c0fc8290 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:49.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.757+0000 7f0b87636740 1 -- 172.21.15.97:0/933057952 >> 172.21.15.97:0/933057952 conn(0x55b7c0fb63c0 msgr2=0x55b7c0fbc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:28:49.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.757+0000 7f0b87636740 1 -- 172.21.15.97:0/933057952 shutdown_connections 2024-04-23T10:28:49.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.757+0000 7f0b87636740 1 -- 172.21.15.97:0/933057952 wait complete. 2024-04-23T10:28:49.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.757+0000 7f0b87636740 20 client.15999 trim_cache size 0 max 16384 2024-04-23T10:28:49.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.774+0000 7f0b87636740 1 Processor -- start 2024-04-23T10:28:49.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.774+0000 7f0b87636740 1 -- start start 2024-04-23T10:28:49.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.774+0000 7f0b87636740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b7c0fba0a0 0x55b7c0fd2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:49.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.774+0000 7f0b87636740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b7c0fba9c0 0x55b7c0fbc800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:49.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.774+0000 7f0b87636740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7c0fc5090 0x55b7c0fdf400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:49.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.774+0000 7f0b87636740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0b801ab800 con 0x55b7c0fba9c0 2024-04-23T10:28:49.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.774+0000 7f0b87636740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0b801ad340 con 0x55b7c0fba0a0 2024-04-23T10:28:49.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.774+0000 7f0b87636740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0b801abed0 con 0x55b7c0fc5090 2024-04-23T10:28:49.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.774+0000 7f0b7dd9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b7c0fba9c0 0x55b7c0fbc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:49.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.774+0000 7f0b7dd9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b7c0fba9c0 0x55b7c0fbc800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56582/0 (socket says 172.21.15.97:56582) 2024-04-23T10:28:49.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.774+0000 7f0b7dd9b640 1 -- 172.21.15.97:0/2487560432 learned_addr learned my addr 172.21.15.97:0/2487560432 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:28:49.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.774+0000 7f0b7dd9b640 1 -- 172.21.15.97:0/2487560432 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7c0fc5090 msgr2=0x55b7c0fdf400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:28:49.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.774+0000 7f0b7dd9b640 1 --2- 172.21.15.97:0/2487560432 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7c0fc5090 0x55b7c0fdf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:49.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.774+0000 7f0b7dd9b640 1 -- 172.21.15.97:0/2487560432 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b7c0fba0a0 msgr2=0x55b7c0fd2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:28:49.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.774+0000 7f0b7dd9b640 1 --2- 172.21.15.97:0/2487560432 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b7c0fba0a0 0x55b7c0fd2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:49.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.774+0000 7f0b7dd9b640 1 -- 172.21.15.97:0/2487560432 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7c0fc7480 con 0x55b7c0fba9c0 2024-04-23T10:28:49.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.775+0000 7f0b7dd9b640 1 --2- 172.21.15.97:0/2487560432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b7c0fba9c0 0x55b7c0fbc800 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f0b74000be0 tx=0x7f0b7400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9592c9525975ce51 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:28:49.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.775+0000 7f0b67fff640 1 -- 172.21.15.97:0/2487560432 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7402a030 con 0x55b7c0fba9c0 2024-04-23T10:28:49.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.775+0000 7f0b67fff640 1 -- 172.21.15.97:0/2487560432 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b74004020 con 0x55b7c0fba9c0 2024-04-23T10:28:49.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.775+0000 7f0b67fff640 1 -- 172.21.15.97:0/2487560432 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b74015040 con 0x55b7c0fba9c0 2024-04-23T10:28:49.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.775+0000 7f0b87636740 1 -- 172.21.15.97:0/2487560432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b7c0fba9c0 msgr2=0x55b7c0fbc800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:28:49.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.775+0000 7f0b87636740 1 --2- 172.21.15.97:0/2487560432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b7c0fba9c0 0x55b7c0fbc800 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f0b74000be0 tx=0x7f0b7400b840 comp rx=0 tx=0).stop 2024-04-23T10:28:49.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.775+0000 7f0b87636740 1 -- 172.21.15.97:0/2487560432 shutdown_connections 2024-04-23T10:28:49.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.775+0000 7f0b87636740 1 --2- 172.21.15.97:0/2487560432 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7c0fc5090 0x55b7c0fdf400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:49.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.775+0000 7f0b87636740 1 --2- 172.21.15.97:0/2487560432 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b7c0fba9c0 0x55b7c0fbc800 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:49.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.775+0000 7f0b87636740 1 --2- 172.21.15.97:0/2487560432 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b7c0fba0a0 0x55b7c0fd2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:49.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.775+0000 7f0b87636740 1 -- 172.21.15.97:0/2487560432 >> 172.21.15.97:0/2487560432 conn(0x55b7c0fb63c0 msgr2=0x55b7c0fb5600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:28:49.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.775+0000 7f0b87636740 1 -- 172.21.15.97:0/2487560432 shutdown_connections 2024-04-23T10:28:49.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.775+0000 7f0b87636740 1 -- 172.21.15.97:0/2487560432 wait complete. 2024-04-23T10:28:49.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.776+0000 7f0b87636740 1 -- start start 2024-04-23T10:28:49.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.776+0000 7f0b87636740 1 Processor -- start 2024-04-23T10:28:49.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.776+0000 7f0b87636740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b7c0fba0a0 0x55b7c1033c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:49.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.776+0000 7f0b87636740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b7c0fba9c0 0x55b7c1034160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:49.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.776+0000 7f0b87636740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7c0fc5090 0x55b7c10346a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:49.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.776+0000 7f0b87636740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0b801acc70 con 0x55b7c0fba0a0 2024-04-23T10:28:49.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.776+0000 7f0b87636740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0b801ac5a0 con 0x55b7c0fba9c0 2024-04-23T10:28:49.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.776+0000 7f0b87636740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0b801a97e0 con 0x55b7c0fc5090 2024-04-23T10:28:49.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.776+0000 7f0b87636740 10 client.0 _ll_register_callbacks cb 0x7f0b7009c8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:28:49.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.776+0000 7f0b7e59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b7c0fba0a0 0x55b7c1033c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:49.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.776+0000 7f0b7e59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b7c0fba0a0 0x55b7c1033c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56586/0 (socket says 172.21.15.97:56586) 2024-04-23T10:28:49.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.776+0000 7f0b7e59c640 1 -- 172.21.15.97:0/3079835314 learned_addr learned my addr 172.21.15.97:0/3079835314 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:28:49.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.776+0000 7f0b7dd9b640 1 --2- 172.21.15.97:0/3079835314 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b7c0fba9c0 0x55b7c1034160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:49.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.776+0000 7f0b7e59c640 1 -- 172.21.15.97:0/3079835314 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7c0fc5090 msgr2=0x55b7c10346a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:28:49.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.776+0000 7f0b7e59c640 1 --2- 172.21.15.97:0/3079835314 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7c0fc5090 0x55b7c10346a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:49.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.776+0000 7f0b7e59c640 1 -- 172.21.15.97:0/3079835314 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b7c0fba9c0 msgr2=0x55b7c1034160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:28:49.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.776+0000 7f0b7e59c640 1 --2- 172.21.15.97:0/3079835314 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b7c0fba9c0 0x55b7c1034160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:49.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.776+0000 7f0b7e59c640 1 -- 172.21.15.97:0/3079835314 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7c103ef40 con 0x55b7c0fba0a0 2024-04-23T10:28:49.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.776+0000 7f0b7dd9b640 1 --2- 172.21.15.97:0/3079835314 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b7c0fba9c0 0x55b7c1034160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:28:49.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.776+0000 7f0b7e59c640 1 --2- 172.21.15.97:0/3079835314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b7c0fba0a0 0x55b7c1033c20 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f0b70099450 tx=0x7f0b7009c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c7bd1a278121586 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:28:49.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.776+0000 7f0b65ffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:28:49.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.776+0000 7f0b65ffb640 1 -- 172.21.15.97:0/3079835314 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7007f070 con 0x55b7c0fba0a0 2024-04-23T10:28:49.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.776+0000 7f0b65ffb640 1 -- 172.21.15.97:0/3079835314 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f0b70087180 con 0x55b7c0fba0a0 2024-04-23T10:28:49.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.776+0000 7f0b87636740 1 -- 172.21.15.97:0/3079835314 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b74004020 con 0x55b7c0fba0a0 2024-04-23T10:28:49.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.777+0000 7f0b65ffb640 1 -- 172.21.15.97:0/3079835314 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0b7001f070 con 0x55b7c0fba0a0 2024-04-23T10:28:49.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.777+0000 7f0b87636740 10 client.16005 fetch_fsmap learned FSMap version 26 2024-04-23T10:28:49.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.777+0000 7f0b87636740 1 -- 172.21.15.97:0/3079835314 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b7c1040200 con 0x55b7c0fba0a0 2024-04-23T10:28:49.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.777+0000 7f0b65ffb640 1 -- 172.21.15.97:0/3079835314 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b74004020 con 0x55b7c0fba0a0 2024-04-23T10:28:49.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.777+0000 7f0b65ffb640 1 -- 172.21.15.97:0/3079835314 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f0b70082030 con 0x55b7c0fba0a0 2024-04-23T10:28:49.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.777+0000 7f0b87636740 10 client.16005 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:28:49.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.777+0000 7f0b87636740 10 client.16005 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:28:49.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.777+0000 7f0b87636740 20 client.16005 populate_metadata read hostname 'smithi097' 2024-04-23T10:28:49.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.777+0000 7f0b87636740 1 --2- 172.21.15.97:0/3079835314 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55b7c1040a10 0x55b7c1042e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:49.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.777+0000 7f0b87636740 4 client.16005 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 89.9, "await_for": 42.0}] 2024-04-23T10:28:49.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.777+0000 7f0b87636740 1 -- 172.21.15.97:0/3079835314 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 89.9, "await_for": 42.0}) -- 0x55b7c0f26c00 con 0x55b7c1040a10 2024-04-23T10:28:49.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.777+0000 7f0b7dd9b640 1 --2- 172.21.15.97:0/3079835314 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55b7c1040a10 0x55b7c1042e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:49.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.778+0000 7f0b65ffb640 10 client.16005 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:28:49.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.778+0000 7f0b7dd9b640 1 --2- 172.21.15.97:0/3079835314 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55b7c1040a10 0x55b7c1042e30 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46312abb73159b4c server_cookie=5d3d172fb687aa1d in_seq=0 out_seq=0 2024-04-23T10:28:49.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.792+0000 7f0b65ffb640 1 -- 172.21.15.97:0/3079835314 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55b7c0f26c00 con 0x55b7c1040a10 2024-04-23T10:28:49.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.792+0000 7f0b65ffb640 10 client.16005 handle_command_reply: tid=0 2024-04-23T10:28:49.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.792+0000 7f0b87636740 1 client.16005 shutdown 2024-04-23T10:28:49.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.793+0000 7f0b87636740 1 -- 172.21.15.97:0/3079835314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b7c0fba0a0 msgr2=0x55b7c1033c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:28:49.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.793+0000 7f0b87636740 1 --2- 172.21.15.97:0/3079835314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b7c0fba0a0 0x55b7c1033c20 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f0b70099450 tx=0x7f0b7009c670 comp rx=0 tx=0).stop 2024-04-23T10:28:49.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.793+0000 7f0b87636740 1 -- 172.21.15.97:0/3079835314 shutdown_connections 2024-04-23T10:28:49.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.793+0000 7f0b87636740 1 --2- 172.21.15.97:0/3079835314 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55b7c1040a10 0x55b7c1042e30 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:49.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.793+0000 7f0b87636740 1 --2- 172.21.15.97:0/3079835314 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7c0fc5090 0x55b7c10346a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:49.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.793+0000 7f0b87636740 1 --2- 172.21.15.97:0/3079835314 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b7c0fba9c0 0x55b7c1034160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:49.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.793+0000 7f0b65ffb640 0 client.16005 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:28:49.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.793+0000 7f0b87636740 1 --2- 172.21.15.97:0/3079835314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b7c0fba0a0 0x55b7c1033c20 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:49.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.793+0000 7f0b87636740 1 -- 172.21.15.97:0/3079835314 >> 172.21.15.97:0/3079835314 conn(0x55b7c0fb63c0 msgr2=0x55b7c0fd1d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:28:49.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.793+0000 7f0b87636740 1 -- 172.21.15.97:0/3079835314 shutdown_connections 2024-04-23T10:28:49.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.793+0000 7f0b87636740 1 -- 172.21.15.97:0/3079835314 wait complete. 2024-04-23T10:28:49.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.793+0000 7f0b87636740 20 client.16005 trim_cache size 0 max 16384 2024-04-23T10:28:49.793 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-23T10:28:49.793 INFO:teuthology.orchestra.run.smithi097.stdout: "epoch": 26, 2024-04-23T10:28:49.794 INFO:teuthology.orchestra.run.smithi097.stdout: "leader": 24488, 2024-04-23T10:28:49.794 INFO:teuthology.orchestra.run.smithi097.stdout: "set_version": 10, 2024-04-23T10:28:49.794 INFO:teuthology.orchestra.run.smithi097.stdout: "sets": { 2024-04-23T10:28:49.794 INFO:teuthology.orchestra.run.smithi097.stdout: "3f96bb3b": { 2024-04-23T10:28:49.794 INFO:teuthology.orchestra.run.smithi097.stdout: "version": 10, 2024-04-23T10:28:49.794 INFO:teuthology.orchestra.run.smithi097.stdout: "age_ref": 0.0, 2024-04-23T10:28:49.794 INFO:teuthology.orchestra.run.smithi097.stdout: "state": { 2024-04-23T10:28:49.794 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "QUIESCED", 2024-04-23T10:28:49.794 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0 2024-04-23T10:28:49.794 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:28:49.794 INFO:teuthology.orchestra.run.smithi097.stdout: "timeout": 60.0, 2024-04-23T10:28:49.794 INFO:teuthology.orchestra.run.smithi097.stdout: "expiration": 89.9, 2024-04-23T10:28:49.795 INFO:teuthology.orchestra.run.smithi097.stdout: "members": { 2024-04-23T10:28:49.795 INFO:teuthology.orchestra.run.smithi097.stdout: "file:/": { 2024-04-23T10:28:49.795 INFO:teuthology.orchestra.run.smithi097.stdout: "excluded": false, 2024-04-23T10:28:49.795 INFO:teuthology.orchestra.run.smithi097.stdout: "state": { 2024-04-23T10:28:49.795 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "QUIESCED", 2024-04-23T10:28:49.795 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0 2024-04-23T10:28:49.795 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:28:49.795 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:28:49.795 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:28:49.795 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:28:49.795 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:28:49.795 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-23T10:28:49.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.793+0000 7f0b84920640 1 -- 172.21.15.97:0/2519407813 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f0b680a3d20 msgr2=0x7f0b680a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:28:49.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.793+0000 7f0b84920640 1 --2- 172.21.15.97:0/2519407813 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f0b680a3d20 0x7f0b680a61e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0b78006010 tx=0x7f0b78031400 comp rx=0 tx=0).stop 2024-04-23T10:28:49.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.793+0000 7f0b84920640 1 -- 172.21.15.97:0/2519407813 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b801010a0 msgr2=0x7f0b80197600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:28:49.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.793+0000 7f0b84920640 1 --2- 172.21.15.97:0/2519407813 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b801010a0 0x7f0b80197600 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f0b7000ea80 tx=0x7f0b7000ef40 comp rx=0 tx=0).stop 2024-04-23T10:28:49.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.793+0000 7f0b84920640 1 -- 172.21.15.97:0/2519407813 shutdown_connections 2024-04-23T10:28:49.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.793+0000 7f0b84920640 1 --2- 172.21.15.97:0/2519407813 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f0b680a3d20 0x7f0b680a61e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:49.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.793+0000 7f0b84920640 1 --2- 172.21.15.97:0/2519407813 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b8019bec0 0x7f0b8019c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:49.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.793+0000 7f0b84920640 1 --2- 172.21.15.97:0/2519407813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b80106170 0x7f0b80197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:49.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.793+0000 7f0b84920640 1 --2- 172.21.15.97:0/2519407813 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b801010a0 0x7f0b80197600 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:49.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.793+0000 7f0b84920640 1 -- 172.21.15.97:0/2519407813 >> 172.21.15.97:0/2519407813 conn(0x7f0b80077f50 msgr2=0x7f0b800fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:28:49.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.793+0000 7f0b84920640 1 -- 172.21.15.97:0/2519407813 shutdown_connections 2024-04-23T10:28:49.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:28:49.793+0000 7f0b84920640 1 -- 172.21.15.97:0/2519407813 wait complete. 2024-04-23T10:28:49.803 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '3f96bb3b', quiesce took 0.5 seconds. Will release after: 29.9 2024-04-23T10:28:50.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:28:50 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/4157449124' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:28:50.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:28:50 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/4157449124' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:28:50.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:28:50 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/4157449124' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:28:51.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:28:51 smithi196 ceph-mon[32919]: pgmap v82: 97 pgs: 97 active+clean; 2.9 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-04-23T10:28:51.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:28:51 smithi097 ceph-mon[29197]: pgmap v82: 97 pgs: 97 active+clean; 2.9 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-04-23T10:28:51.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:28:51 smithi183 ceph-mon[32317]: pgmap v82: 97 pgs: 97 active+clean; 2.9 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-04-23T10:28:51.500 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.501+0000 7f7e292cf640 1 -- 172.21.15.97:0/3165538315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e241021d0 msgr2=0x7f7e241025b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:28:51.500 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.501+0000 7f7e292cf640 1 --2- 172.21.15.97:0/3165538315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e241021d0 0x7f7e241025b0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f7e14009ca0 tx=0x7f7e1402a8c0 comp rx=0 tx=0).stop 2024-04-23T10:28:51.500 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.501+0000 7f7e292cf640 1 -- 172.21.15.97:0/3165538315 shutdown_connections 2024-04-23T10:28:51.500 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.501+0000 7f7e292cf640 1 --2- 172.21.15.97:0/3165538315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e241072a0 0x7f7e2410ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:51.500 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.501+0000 7f7e292cf640 1 --2- 172.21.15.97:0/3165538315 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e24102b80 0x7f7e24106bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:51.500 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.501+0000 7f7e292cf640 1 --2- 172.21.15.97:0/3165538315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e241021d0 0x7f7e241025b0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:51.501 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.501+0000 7f7e292cf640 1 -- 172.21.15.97:0/3165538315 >> 172.21.15.97:0/3165538315 conn(0x7f7e240793e0 msgr2=0x7f7e240797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:28:51.501 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.501+0000 7f7e292cf640 1 -- 172.21.15.97:0/3165538315 shutdown_connections 2024-04-23T10:28:51.501 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.501+0000 7f7e292cf640 1 -- 172.21.15.97:0/3165538315 wait complete. 2024-04-23T10:28:51.501 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.501+0000 7f7e292cf640 1 Processor -- start 2024-04-23T10:28:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.502+0000 7f7e292cf640 1 -- start start 2024-04-23T10:28:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.502+0000 7f7e292cf640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e241021d0 0x7f7e24074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.502+0000 7f7e292cf640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e241072a0 0x7f7e24074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.502+0000 7f7e292cf640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e24074cd0 0x7f7e241a6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.502+0000 7f7e292cf640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7e241121e0 con 0x7f7e241072a0 2024-04-23T10:28:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.502+0000 7f7e292cf640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7e24112060 con 0x7f7e241021d0 2024-04-23T10:28:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.502+0000 7f7e292cf640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7e24112360 con 0x7f7e24074cd0 2024-04-23T10:28:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.502+0000 7f7e227fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e241072a0 0x7f7e24074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.502+0000 7f7e227fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e241072a0 0x7f7e24074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56600/0 (socket says 172.21.15.97:56600) 2024-04-23T10:28:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.502+0000 7f7e227fc640 1 -- 172.21.15.97:0/4064468544 learned_addr learned my addr 172.21.15.97:0/4064468544 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:28:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.502+0000 7f7e237fe640 1 --2- 172.21.15.97:0/4064468544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e24074cd0 0x7f7e241a6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.502+0000 7f7e22ffd640 1 --2- 172.21.15.97:0/4064468544 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e241021d0 0x7f7e24074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.502+0000 7f7e227fc640 1 -- 172.21.15.97:0/4064468544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e24074cd0 msgr2=0x7f7e241a6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:28:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.502+0000 7f7e227fc640 1 --2- 172.21.15.97:0/4064468544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e24074cd0 0x7f7e241a6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.502+0000 7f7e227fc640 1 -- 172.21.15.97:0/4064468544 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e241021d0 msgr2=0x7f7e24074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:28:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.502+0000 7f7e227fc640 1 --2- 172.21.15.97:0/4064468544 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e241021d0 0x7f7e24074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.502+0000 7f7e227fc640 1 -- 172.21.15.97:0/4064468544 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e241a6f70 con 0x7f7e241072a0 2024-04-23T10:28:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.502+0000 7f7e227fc640 1 --2- 172.21.15.97:0/4064468544 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e241072a0 0x7f7e24074790 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f7e1800d770 tx=0x7f7e1800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:28:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.503+0000 7f7e0bfff640 1 -- 172.21.15.97:0/4064468544 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e18017070 con 0x7f7e241072a0 2024-04-23T10:28:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.503+0000 7f7e0bfff640 1 -- 172.21.15.97:0/4064468544 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e18009c30 con 0x7f7e241072a0 2024-04-23T10:28:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.503+0000 7f7e292cf640 1 -- 172.21.15.97:0/4064468544 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e241a7260 con 0x7f7e241072a0 2024-04-23T10:28:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.503+0000 7f7e0bfff640 1 -- 172.21.15.97:0/4064468544 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e1800f620 con 0x7f7e241072a0 2024-04-23T10:28:51.504 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.503+0000 7f7e292cf640 1 -- 172.21.15.97:0/4064468544 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e241a77b0 con 0x7f7e241072a0 2024-04-23T10:28:51.510 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.505+0000 7f7e0bfff640 1 -- 172.21.15.97:0/4064468544 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7e1801a070 con 0x7f7e241072a0 2024-04-23T10:28:51.510 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.505+0000 7f7e2bfe8740 1 Processor -- start 2024-04-23T10:28:51.510 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.505+0000 7f7e2bfe8740 1 -- start start 2024-04-23T10:28:51.510 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.506+0000 7f7e0bfff640 1 --2- 172.21.15.97:0/4064468544 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f7e0c0a3df0 0x7f7e0c0a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:51.511 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.506+0000 7f7e2bfe8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed60dd4620 0x55ed60dd4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:51.511 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.506+0000 7f7e2bfe8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed60dd4f40 0x55ed60ddf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:51.511 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.506+0000 7f7e2bfe8740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed60ddf800 0x55ed60de1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:51.511 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.506+0000 7f7e2bfe8740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ed60d39970 con 0x55ed60ddf800 2024-04-23T10:28:51.511 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.506+0000 7f7e2bfe8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ed60cd64a0 con 0x55ed60dd4f40 2024-04-23T10:28:51.511 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.506+0000 7f7e2bfe8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ed60d35930 con 0x55ed60dd4620 2024-04-23T10:28:51.511 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.506+0000 7f7e237fe640 1 --2- 172.21.15.97:0/4064468544 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f7e0c0a3df0 0x7f7e0c0a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:51.511 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.506+0000 7f7e0bfff640 1 -- 172.21.15.97:0/4064468544 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f7e180d0360 con 0x7f7e241072a0 2024-04-23T10:28:51.511 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.506+0000 7f7e237fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed60ddf800 0x55ed60de1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:51.511 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.506+0000 7f7e227fc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed60dd4f40 0x55ed60ddf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:51.511 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.506+0000 7f7e237fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed60ddf800 0x55ed60de1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56602/0 (socket says 172.21.15.97:56602) 2024-04-23T10:28:51.511 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.506+0000 7f7e237fe640 1 -- 172.21.15.97:0/61154060 learned_addr learned my addr 172.21.15.97:0/61154060 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:28:51.511 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.506+0000 7f7e22ffd640 1 --2- 172.21.15.97:0/61154060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed60dd4620 0x55ed60dd4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:51.511 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.506+0000 7f7e237fe640 1 -- 172.21.15.97:0/61154060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed60dd4620 msgr2=0x55ed60dd4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:28:51.511 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.506+0000 7f7e237fe640 1 --2- 172.21.15.97:0/61154060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed60dd4620 0x55ed60dd4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:51.512 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.506+0000 7f7e237fe640 1 -- 172.21.15.97:0/61154060 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed60dd4f40 msgr2=0x55ed60ddf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:28:51.512 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.506+0000 7f7e237fe640 1 --2- 172.21.15.97:0/61154060 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed60dd4f40 0x55ed60ddf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:51.512 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.506+0000 7f7e237fe640 1 -- 172.21.15.97:0/61154060 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed60a4c210 con 0x55ed60ddf800 2024-04-23T10:28:51.512 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.506+0000 7f7e237fe640 1 --2- 172.21.15.97:0/61154060 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed60ddf800 0x55ed60de1bf0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c0117b0 tx=0x7f7e1c01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf4e2a9acc68733a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:28:51.512 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.506+0000 7f7e237fe640 1 --2- 172.21.15.97:0/4064468544 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f7e0c0a3df0 0x7f7e0c0a62b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c014910 tx=0x7f7e1c00cb80 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:28:51.512 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.507+0000 7f7e097fa640 1 -- 172.21.15.97:0/61154060 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e1c022070 con 0x55ed60ddf800 2024-04-23T10:28:51.512 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.507+0000 7f7e097fa640 1 -- 172.21.15.97:0/61154060 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e1c00c3a0 con 0x55ed60ddf800 2024-04-23T10:28:51.512 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.507+0000 7f7e097fa640 1 -- 172.21.15.97:0/61154060 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e1c00fd90 con 0x55ed60ddf800 2024-04-23T10:28:51.512 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.507+0000 7f7e2bfe8740 1 -- 172.21.15.97:0/61154060 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed60ddf800 msgr2=0x55ed60de1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:28:51.512 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.507+0000 7f7e2bfe8740 1 --2- 172.21.15.97:0/61154060 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed60ddf800 0x55ed60de1bf0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c0117b0 tx=0x7f7e1c01b000 comp rx=0 tx=0).stop 2024-04-23T10:28:51.512 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.507+0000 7f7e2bfe8740 1 -- 172.21.15.97:0/61154060 shutdown_connections 2024-04-23T10:28:51.512 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.507+0000 7f7e2bfe8740 1 --2- 172.21.15.97:0/61154060 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed60ddf800 0x55ed60de1bf0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:51.512 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.507+0000 7f7e2bfe8740 1 --2- 172.21.15.97:0/61154060 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed60dd4f40 0x55ed60ddf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:51.512 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.507+0000 7f7e2bfe8740 1 --2- 172.21.15.97:0/61154060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed60dd4620 0x55ed60dd4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:51.512 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.507+0000 7f7e2bfe8740 1 -- 172.21.15.97:0/61154060 >> 172.21.15.97:0/61154060 conn(0x55ed60dd0b30 msgr2=0x55ed60dd0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:28:51.513 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.507+0000 7f7e2bfe8740 1 -- 172.21.15.97:0/61154060 shutdown_connections 2024-04-23T10:28:51.513 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.507+0000 7f7e2bfe8740 1 -- 172.21.15.97:0/61154060 wait complete. 2024-04-23T10:28:51.513 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.508+0000 7f7e2bfe8740 1 -- start start 2024-04-23T10:28:51.513 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.508+0000 7f7e2bfe8740 1 Processor -- start 2024-04-23T10:28:51.513 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.508+0000 7f7e2bfe8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed60dd4620 0x55ed60ddf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:51.513 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.508+0000 7f7e2bfe8740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed60dd4f40 0x55ed60ddc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:51.513 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.508+0000 7f7e2bfe8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed60ddf800 0x55ed60ddca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:51.513 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.508+0000 7f7e2bfe8740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ed60de23d0 con 0x55ed60dd4f40 2024-04-23T10:28:51.513 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.508+0000 7f7e2bfe8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ed60cd40b0 con 0x55ed60dd4620 2024-04-23T10:28:51.513 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.508+0000 7f7e2bfe8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ed60de2550 con 0x55ed60ddf800 2024-04-23T10:28:51.513 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.508+0000 7f7e2bfe8740 10 client.0 _ll_register_callbacks cb 0x55ed60cc1d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:28:51.513 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.508+0000 7f7e227fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed60dd4f40 0x55ed60ddc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:51.513 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.508+0000 7f7e227fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed60dd4f40 0x55ed60ddc480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56614/0 (socket says 172.21.15.97:56614) 2024-04-23T10:28:51.513 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.508+0000 7f7e227fc640 1 -- 172.21.15.97:0/1500283083 learned_addr learned my addr 172.21.15.97:0/1500283083 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:28:51.513 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.508+0000 7f7e237fe640 1 --2- 172.21.15.97:0/1500283083 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed60ddf800 0x55ed60ddca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:51.513 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.508+0000 7f7e22ffd640 1 --2- 172.21.15.97:0/1500283083 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed60dd4620 0x55ed60ddf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:51.514 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.508+0000 7f7e227fc640 1 -- 172.21.15.97:0/1500283083 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed60ddf800 msgr2=0x55ed60ddca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:28:51.514 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.508+0000 7f7e227fc640 1 --2- 172.21.15.97:0/1500283083 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed60ddf800 0x55ed60ddca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:51.514 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.508+0000 7f7e227fc640 1 -- 172.21.15.97:0/1500283083 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed60dd4620 msgr2=0x55ed60ddf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:28:51.514 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.508+0000 7f7e227fc640 1 --2- 172.21.15.97:0/1500283083 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed60dd4620 0x55ed60ddf120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:51.514 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.508+0000 7f7e227fc640 1 -- 172.21.15.97:0/1500283083 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed60b18620 con 0x55ed60dd4f40 2024-04-23T10:28:51.514 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.509+0000 7f7e227fc640 1 --2- 172.21.15.97:0/1500283083 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed60dd4f40 0x55ed60ddc480 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f7e1800d740 tx=0x7f7e1809cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb87ecd3c37ec56d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:28:51.514 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.509+0000 7f7e06ff5640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:28:51.514 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.509+0000 7f7e06ff5640 1 -- 172.21.15.97:0/1500283083 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e18021040 con 0x55ed60dd4f40 2024-04-23T10:28:51.514 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.509+0000 7f7e06ff5640 1 -- 172.21.15.97:0/1500283083 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f7e1809a650 con 0x55ed60dd4f40 2024-04-23T10:28:51.514 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.509+0000 7f7e2bfe8740 1 -- 172.21.15.97:0/1500283083 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e1c00c3a0 con 0x55ed60dd4f40 2024-04-23T10:28:51.514 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.509+0000 7f7e06ff5640 1 -- 172.21.15.97:0/1500283083 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7e180cdd80 con 0x55ed60dd4f40 2024-04-23T10:28:51.514 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.509+0000 7f7e2bfe8740 10 client.16017 fetch_fsmap learned FSMap version 26 2024-04-23T10:28:51.514 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.509+0000 7f7e2bfe8740 1 -- 172.21.15.97:0/1500283083 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ed60dee3c0 con 0x55ed60dd4f40 2024-04-23T10:28:51.514 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.509+0000 7f7e06ff5640 1 -- 172.21.15.97:0/1500283083 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e1c00c3a0 con 0x55ed60dd4f40 2024-04-23T10:28:51.515 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.509+0000 7f7e06ff5640 1 -- 172.21.15.97:0/1500283083 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f7e180c4490 con 0x55ed60dd4f40 2024-04-23T10:28:51.515 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.509+0000 7f7e2bfe8740 10 client.16017 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:28:51.515 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.509+0000 7f7e2bfe8740 10 client.16017 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:28:51.515 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.509+0000 7f7e2bfe8740 20 client.16017 populate_metadata read hostname 'smithi097' 2024-04-23T10:28:51.515 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.509+0000 7f7e2bfe8740 1 --2- 172.21.15.97:0/1500283083 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55ed60def830 0x55ed60df1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:51.515 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.509+0000 7f7e2bfe8740 4 client.16017 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:28:51.515 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.509+0000 7f7e2bfe8740 1 -- 172.21.15.97:0/1500283083 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed60d47390 con 0x55ed60def830 2024-04-23T10:28:51.515 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.510+0000 7f7e22ffd640 1 --2- 172.21.15.97:0/1500283083 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55ed60def830 0x55ed60df1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:51.515 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.510+0000 7f7e22ffd640 1 --2- 172.21.15.97:0/1500283083 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55ed60def830 0x55ed60df1c20 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29950ffe4c46475f server_cookie=ef8fddffde33f3de in_seq=0 out_seq=0 2024-04-23T10:28:51.515 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.510+0000 7f7e06ff5640 10 client.16017 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:28:51.515 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.511+0000 7f7e06ff5640 1 -- 172.21.15.97:0/1500283083 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ed60d47390 con 0x55ed60def830 2024-04-23T10:28:51.515 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.511+0000 7f7e06ff5640 10 client.16017 handle_command_reply: tid=0 2024-04-23T10:28:51.515 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.511+0000 7f7e2bfe8740 1 client.16017 shutdown 2024-04-23T10:28:51.515 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.511+0000 7f7e2bfe8740 1 -- 172.21.15.97:0/1500283083 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed60dd4f40 msgr2=0x55ed60ddc480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:28:51.515 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.511+0000 7f7e2bfe8740 1 --2- 172.21.15.97:0/1500283083 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed60dd4f40 0x55ed60ddc480 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f7e1800d740 tx=0x7f7e1809cb90 comp rx=0 tx=0).stop 2024-04-23T10:28:51.515 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.511+0000 7f7e2bfe8740 1 -- 172.21.15.97:0/1500283083 shutdown_connections 2024-04-23T10:28:51.516 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.511+0000 7f7e2bfe8740 1 --2- 172.21.15.97:0/1500283083 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55ed60def830 0x55ed60df1c20 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:51.516 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.511+0000 7f7e2bfe8740 1 --2- 172.21.15.97:0/1500283083 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed60ddf800 0x55ed60ddca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:51.516 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.511+0000 7f7e06ff5640 0 client.16017 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:28:51.516 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.511+0000 7f7e2bfe8740 1 --2- 172.21.15.97:0/1500283083 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed60dd4f40 0x55ed60ddc480 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:51.516 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.511+0000 7f7e2bfe8740 1 --2- 172.21.15.97:0/1500283083 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed60dd4620 0x55ed60ddf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:51.516 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.511+0000 7f7e2bfe8740 1 -- 172.21.15.97:0/1500283083 >> 172.21.15.97:0/1500283083 conn(0x55ed60dd0b30 msgr2=0x55ed60dd68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:28:51.516 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.511+0000 7f7e2bfe8740 1 -- 172.21.15.97:0/1500283083 shutdown_connections 2024-04-23T10:28:51.516 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.511+0000 7f7e2bfe8740 1 -- 172.21.15.97:0/1500283083 wait complete. 2024-04-23T10:28:51.516 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.511+0000 7f7e2bfe8740 20 client.16017 trim_cache size 0 max 16384 2024-04-23T10:28:51.529 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.526+0000 7f7e2bfe8740 1 Processor -- start 2024-04-23T10:28:51.529 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.526+0000 7f7e2bfe8740 1 -- start start 2024-04-23T10:28:51.529 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.526+0000 7f7e2bfe8740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed60dd4620 0x55ed60deb290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:51.529 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.526+0000 7f7e2bfe8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed60dd4f40 0x55ed60dde3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:51.529 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.526+0000 7f7e2bfe8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed60ddf800 0x55ed60df7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:51.529 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.526+0000 7f7e2bfe8740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7e241afc30 con 0x55ed60dd4620 2024-04-23T10:28:51.529 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.526+0000 7f7e2bfe8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7e241b1770 con 0x55ed60ddf800 2024-04-23T10:28:51.529 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.526+0000 7f7e2bfe8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7e241b0300 con 0x55ed60dd4f40 2024-04-23T10:28:51.530 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.526+0000 7f7e22ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed60dd4620 0x55ed60deb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:51.530 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.526+0000 7f7e22ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed60dd4620 0x55ed60deb290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56616/0 (socket says 172.21.15.97:56616) 2024-04-23T10:28:51.530 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.526+0000 7f7e22ffd640 1 -- 172.21.15.97:0/3823252853 learned_addr learned my addr 172.21.15.97:0/3823252853 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:28:51.530 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.526+0000 7f7e22ffd640 1 -- 172.21.15.97:0/3823252853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed60dd4f40 msgr2=0x55ed60dde3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:28:51.530 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.526+0000 7f7e22ffd640 1 --2- 172.21.15.97:0/3823252853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed60dd4f40 0x55ed60dde3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:51.530 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.526+0000 7f7e22ffd640 1 -- 172.21.15.97:0/3823252853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed60ddf800 msgr2=0x55ed60df7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:28:51.530 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.526+0000 7f7e22ffd640 1 --2- 172.21.15.97:0/3823252853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed60ddf800 0x55ed60df7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:51.530 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.526+0000 7f7e22ffd640 1 -- 172.21.15.97:0/3823252853 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed60de1f60 con 0x55ed60dd4620 2024-04-23T10:28:51.530 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.526+0000 7f7e22ffd640 1 --2- 172.21.15.97:0/3823252853 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed60dd4620 0x55ed60deb290 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f7e14005e30 tx=0x7f7e14031d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d15b4be4bc3d866f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:28:51.530 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.526+0000 7f7e08ff9640 1 -- 172.21.15.97:0/3823252853 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e14031650 con 0x55ed60dd4620 2024-04-23T10:28:51.530 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.527+0000 7f7e08ff9640 1 -- 172.21.15.97:0/3823252853 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e1404d070 con 0x55ed60dd4620 2024-04-23T10:28:51.530 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.527+0000 7f7e08ff9640 1 -- 172.21.15.97:0/3823252853 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e14004570 con 0x55ed60dd4620 2024-04-23T10:28:51.530 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.527+0000 7f7e2bfe8740 1 -- 172.21.15.97:0/3823252853 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed60dd4620 msgr2=0x55ed60deb290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:28:51.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.527+0000 7f7e2bfe8740 1 --2- 172.21.15.97:0/3823252853 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed60dd4620 0x55ed60deb290 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f7e14005e30 tx=0x7f7e14031d80 comp rx=0 tx=0).stop 2024-04-23T10:28:51.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.527+0000 7f7e2bfe8740 1 -- 172.21.15.97:0/3823252853 shutdown_connections 2024-04-23T10:28:51.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.527+0000 7f7e2bfe8740 1 --2- 172.21.15.97:0/3823252853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed60ddf800 0x55ed60df7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:51.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.527+0000 7f7e2bfe8740 1 --2- 172.21.15.97:0/3823252853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed60dd4f40 0x55ed60dde3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:51.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.527+0000 7f7e2bfe8740 1 --2- 172.21.15.97:0/3823252853 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed60dd4620 0x55ed60deb290 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:51.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.527+0000 7f7e2bfe8740 1 -- 172.21.15.97:0/3823252853 >> 172.21.15.97:0/3823252853 conn(0x55ed60dd0b30 msgr2=0x55ed60dddcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:28:51.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.527+0000 7f7e2bfe8740 1 -- 172.21.15.97:0/3823252853 shutdown_connections 2024-04-23T10:28:51.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.527+0000 7f7e2bfe8740 1 -- 172.21.15.97:0/3823252853 wait complete. 2024-04-23T10:28:51.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.527+0000 7f7e2bfe8740 1 -- start start 2024-04-23T10:28:51.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.527+0000 7f7e2bfe8740 1 Processor -- start 2024-04-23T10:28:51.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.528+0000 7f7e2bfe8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed60dd4620 0x55ed60dd9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:51.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.528+0000 7f7e2bfe8740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed60dd4f40 0x55ed60dd9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:51.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.528+0000 7f7e2bfe8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed60ddf800 0x55ed60e54db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:51.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.528+0000 7f7e2bfe8740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7e241b10a0 con 0x55ed60dd4f40 2024-04-23T10:28:51.532 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.528+0000 7f7e2bfe8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7e241b09d0 con 0x55ed60dd4620 2024-04-23T10:28:51.532 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.528+0000 7f7e2bfe8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7e241adc30 con 0x55ed60ddf800 2024-04-23T10:28:51.532 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.528+0000 7f7e2bfe8740 10 client.0 _ll_register_callbacks cb 0x7f7e1809bcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:28:51.532 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.528+0000 7f7e227fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed60dd4f40 0x55ed60dd9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:51.532 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.528+0000 7f7e227fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed60dd4f40 0x55ed60dd9a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56618/0 (socket says 172.21.15.97:56618) 2024-04-23T10:28:51.532 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.528+0000 7f7e227fc640 1 -- 172.21.15.97:0/3869172232 learned_addr learned my addr 172.21.15.97:0/3869172232 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:28:51.532 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.528+0000 7f7e237fe640 1 --2- 172.21.15.97:0/3869172232 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed60ddf800 0x55ed60e54db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:51.532 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.528+0000 7f7e227fc640 1 -- 172.21.15.97:0/3869172232 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed60ddf800 msgr2=0x55ed60e54db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:28:51.532 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.528+0000 7f7e227fc640 1 --2- 172.21.15.97:0/3869172232 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed60ddf800 0x55ed60e54db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:51.532 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.528+0000 7f7e22ffd640 1 --2- 172.21.15.97:0/3869172232 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed60dd4620 0x55ed60dd9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:51.532 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.528+0000 7f7e227fc640 1 -- 172.21.15.97:0/3869172232 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed60dd4620 msgr2=0x55ed60dd9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:28:51.532 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.528+0000 7f7e227fc640 1 --2- 172.21.15.97:0/3869172232 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed60dd4620 0x55ed60dd9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:51.532 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.528+0000 7f7e227fc640 1 -- 172.21.15.97:0/3869172232 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed60df1f90 con 0x55ed60dd4f40 2024-04-23T10:28:51.532 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.528+0000 7f7e237fe640 1 --2- 172.21.15.97:0/3869172232 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed60ddf800 0x55ed60e54db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:28:51.533 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.528+0000 7f7e22ffd640 1 --2- 172.21.15.97:0/3869172232 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed60dd4620 0x55ed60dd9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:28:51.533 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.528+0000 7f7e227fc640 1 --2- 172.21.15.97:0/3869172232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed60dd4f40 0x55ed60dd9a70 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f7e18010fd0 tx=0x7f7e1809bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=100e75070a10bb25 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:28:51.533 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.528+0000 7f7e06ff5640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:28:51.533 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.528+0000 7f7e06ff5640 1 -- 172.21.15.97:0/3869172232 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e18021070 con 0x55ed60dd4f40 2024-04-23T10:28:51.533 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.528+0000 7f7e06ff5640 1 -- 172.21.15.97:0/3869172232 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f7e180bb100 con 0x55ed60dd4f40 2024-04-23T10:28:51.533 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.528+0000 7f7e2bfe8740 1 -- 172.21.15.97:0/3869172232 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e1404d070 con 0x55ed60dd4f40 2024-04-23T10:28:51.533 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.529+0000 7f7e06ff5640 1 -- 172.21.15.97:0/3869172232 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7e180b9040 con 0x55ed60dd4f40 2024-04-23T10:28:51.533 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.529+0000 7f7e2bfe8740 10 client.16023 fetch_fsmap learned FSMap version 26 2024-04-23T10:28:51.533 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.529+0000 7f7e2bfe8740 1 -- 172.21.15.97:0/3869172232 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ed60e56270 con 0x55ed60dd4f40 2024-04-23T10:28:51.533 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.529+0000 7f7e06ff5640 1 -- 172.21.15.97:0/3869172232 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e1404d070 con 0x55ed60dd4f40 2024-04-23T10:28:51.533 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.529+0000 7f7e06ff5640 1 -- 172.21.15.97:0/3869172232 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f7e180981e0 con 0x55ed60dd4f40 2024-04-23T10:28:51.533 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.529+0000 7f7e2bfe8740 10 client.16023 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:28:51.533 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.529+0000 7f7e2bfe8740 10 client.16023 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:28:51.533 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.529+0000 7f7e2bfe8740 20 client.16023 populate_metadata read hostname 'smithi097' 2024-04-23T10:28:51.533 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.529+0000 7f7e2bfe8740 1 --2- 172.21.15.97:0/3869172232 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55ed60e56990 0x55ed60e58d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:28:51.534 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.529+0000 7f7e2bfe8740 4 client.16023 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:28:51.534 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.529+0000 7f7e2bfe8740 1 -- 172.21.15.97:0/3869172232 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ed60d41200 con 0x55ed60e56990 2024-04-23T10:28:51.534 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.529+0000 7f7e22ffd640 1 --2- 172.21.15.97:0/3869172232 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55ed60e56990 0x55ed60e58d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:28:51.534 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.530+0000 7f7e06ff5640 10 client.16023 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:28:51.534 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.530+0000 7f7e22ffd640 1 --2- 172.21.15.97:0/3869172232 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55ed60e56990 0x55ed60e58d80 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2f4a439eac4f21d server_cookie=f5110e5691483604 in_seq=0 out_seq=0 2024-04-23T10:28:51.544 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.545+0000 7f7e06ff5640 1 -- 172.21.15.97:0/3869172232 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ed60d41200 con 0x55ed60e56990 2024-04-23T10:28:51.544 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.545+0000 7f7e06ff5640 10 client.16023 handle_command_reply: tid=0 2024-04-23T10:28:51.544 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.545+0000 7f7e2bfe8740 1 client.16023 shutdown 2024-04-23T10:28:51.544 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.545+0000 7f7e2bfe8740 1 -- 172.21.15.97:0/3869172232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed60dd4f40 msgr2=0x55ed60dd9a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:28:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.545+0000 7f7e2bfe8740 1 --2- 172.21.15.97:0/3869172232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed60dd4f40 0x55ed60dd9a70 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f7e18010fd0 tx=0x7f7e1809bd60 comp rx=0 tx=0).stop 2024-04-23T10:28:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.545+0000 7f7e2bfe8740 1 -- 172.21.15.97:0/3869172232 shutdown_connections 2024-04-23T10:28:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.545+0000 7f7e2bfe8740 1 --2- 172.21.15.97:0/3869172232 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55ed60e56990 0x55ed60e58d80 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.545+0000 7f7e2bfe8740 1 --2- 172.21.15.97:0/3869172232 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed60ddf800 0x55ed60e54db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.545+0000 7f7e2bfe8740 1 --2- 172.21.15.97:0/3869172232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed60dd4f40 0x55ed60dd9a70 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.545+0000 7f7e2bfe8740 1 --2- 172.21.15.97:0/3869172232 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed60dd4620 0x55ed60dd9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.545+0000 7f7e06ff5640 0 client.16023 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:28:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.545+0000 7f7e2bfe8740 1 -- 172.21.15.97:0/3869172232 >> 172.21.15.97:0/3869172232 conn(0x55ed60dd0b30 msgr2=0x55ed60dd68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:28:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.545+0000 7f7e2bfe8740 1 -- 172.21.15.97:0/3869172232 shutdown_connections 2024-04-23T10:28:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.545+0000 7f7e2bfe8740 1 -- 172.21.15.97:0/3869172232 wait complete. 2024-04-23T10:28:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.545+0000 7f7e2bfe8740 20 client.16023 trim_cache size 0 max 16384 2024-04-23T10:28:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:28:51.546 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:28:51.546 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:28:51.546 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:28:51.546 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.546+0000 7f7e292cf640 1 -- 172.21.15.97:0/4064468544 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f7e0c0a3df0 msgr2=0x7f7e0c0a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:28:51.547 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.546+0000 7f7e292cf640 1 --2- 172.21.15.97:0/4064468544 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f7e0c0a3df0 0x7f7e0c0a62b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c014910 tx=0x7f7e1c00cb80 comp rx=0 tx=0).stop 2024-04-23T10:28:51.547 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.546+0000 7f7e292cf640 1 -- 172.21.15.97:0/4064468544 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e241072a0 msgr2=0x7f7e24074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:28:51.547 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.546+0000 7f7e292cf640 1 --2- 172.21.15.97:0/4064468544 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e241072a0 0x7f7e24074790 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f7e1800d770 tx=0x7f7e1800dc30 comp rx=0 tx=0).stop 2024-04-23T10:28:51.547 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.546+0000 7f7e292cf640 1 -- 172.21.15.97:0/4064468544 shutdown_connections 2024-04-23T10:28:51.547 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.546+0000 7f7e292cf640 1 --2- 172.21.15.97:0/4064468544 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f7e0c0a3df0 0x7f7e0c0a62b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:51.547 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.546+0000 7f7e292cf640 1 --2- 172.21.15.97:0/4064468544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e24074cd0 0x7f7e241a6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:51.547 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.546+0000 7f7e292cf640 1 --2- 172.21.15.97:0/4064468544 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e241072a0 0x7f7e24074790 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:51.547 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.546+0000 7f7e292cf640 1 --2- 172.21.15.97:0/4064468544 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e241021d0 0x7f7e24074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:28:51.547 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.546+0000 7f7e292cf640 1 -- 172.21.15.97:0/4064468544 >> 172.21.15.97:0/4064468544 conn(0x7f7e240793e0 msgr2=0x7f7e240ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:28:51.547 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.546+0000 7f7e292cf640 1 -- 172.21.15.97:0/4064468544 shutdown_connections 2024-04-23T10:28:51.547 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:28:51.546+0000 7f7e292cf640 1 -- 172.21.15.97:0/4064468544 wait complete. 2024-04-23T10:28:53.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:28:53 smithi196 ceph-mon[32919]: pgmap v83: 97 pgs: 97 active+clean; 2.9 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-04-23T10:28:53.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:28:53 smithi097 ceph-mon[29197]: pgmap v83: 97 pgs: 97 active+clean; 2.9 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-04-23T10:28:53.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:28:53 smithi183 ceph-mon[32317]: pgmap v83: 97 pgs: 97 active+clean; 2.9 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-04-23T10:28:54.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:28:54 smithi196 ceph-mon[32919]: pgmap v84: 97 pgs: 97 active+clean; 2.9 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 29 KiB/s wr, 23 op/s 2024-04-23T10:28:54.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:28:54 smithi097 ceph-mon[29197]: pgmap v84: 97 pgs: 97 active+clean; 2.9 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 29 KiB/s wr, 23 op/s 2024-04-23T10:28:54.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:28:54 smithi183 ceph-mon[32317]: pgmap v84: 97 pgs: 97 active+clean; 2.9 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 29 KiB/s wr, 23 op/s 2024-04-23T10:28:56.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:28:56 smithi196 ceph-mon[32919]: pgmap v85: 97 pgs: 97 active+clean; 2.9 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-04-23T10:28:56.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:28:56 smithi097 ceph-mon[29197]: pgmap v85: 97 pgs: 97 active+clean; 2.9 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-04-23T10:28:56.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:28:56 smithi183 ceph-mon[32317]: pgmap v85: 97 pgs: 97 active+clean; 2.9 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-04-23T10:28:58.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:28:58 smithi196 ceph-mon[32919]: pgmap v86: 97 pgs: 97 active+clean; 2.9 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 18 op/s 2024-04-23T10:28:58.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:28:58 smithi097 ceph-mon[29197]: pgmap v86: 97 pgs: 97 active+clean; 2.9 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 18 op/s 2024-04-23T10:28:58.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:28:58 smithi183 ceph-mon[32317]: pgmap v86: 97 pgs: 97 active+clean; 2.9 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 18 op/s 2024-04-23T10:29:00.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:29:00 smithi196 ceph-mon[32919]: pgmap v87: 97 pgs: 97 active+clean; 2.9 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.8 KiB/s wr, 12 op/s 2024-04-23T10:29:00.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:29:00 smithi097 ceph-mon[29197]: pgmap v87: 97 pgs: 97 active+clean; 2.9 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.8 KiB/s wr, 12 op/s 2024-04-23T10:29:00.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:29:00 smithi183 ceph-mon[32317]: pgmap v87: 97 pgs: 97 active+clean; 2.9 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.8 KiB/s wr, 12 op/s 2024-04-23T10:29:02.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:29:02 smithi196 ceph-mon[32919]: pgmap v88: 97 pgs: 97 active+clean; 2.9 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.8 KiB/s wr, 12 op/s 2024-04-23T10:29:02.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:29:02 smithi097 ceph-mon[29197]: pgmap v88: 97 pgs: 97 active+clean; 2.9 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.8 KiB/s wr, 12 op/s 2024-04-23T10:29:02.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:29:02 smithi183 ceph-mon[32317]: pgmap v88: 97 pgs: 97 active+clean; 2.9 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.8 KiB/s wr, 12 op/s 2024-04-23T10:29:04.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:29:04 smithi196 ceph-mon[32919]: pgmap v89: 97 pgs: 97 active+clean; 2.9 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 31 KiB/s wr, 31 op/s 2024-04-23T10:29:04.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:29:04 smithi097 ceph-mon[29197]: pgmap v89: 97 pgs: 97 active+clean; 2.9 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 31 KiB/s wr, 31 op/s 2024-04-23T10:29:04.640 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.641+0000 7f680498e640 1 -- 172.21.15.97:0/2930220209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6800103af0 msgr2=0x7f680010bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:04.640 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.641+0000 7f680498e640 1 --2- 172.21.15.97:0/2930220209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6800103af0 0x7f680010bfd0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f67f4009900 tx=0x7f67f402a480 comp rx=0 tx=0).stop 2024-04-23T10:29:04.640 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.641+0000 7f680498e640 1 -- 172.21.15.97:0/2930220209 shutdown_connections 2024-04-23T10:29:04.641 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.641+0000 7f680498e640 1 --2- 172.21.15.97:0/2930220209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f680010c5a0 0x7f680010e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:04.641 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.641+0000 7f680498e640 1 --2- 172.21.15.97:0/2930220209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6800103af0 0x7f680010bfd0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:04.641 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.641+0000 7f680498e640 1 --2- 172.21.15.97:0/2930220209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68001031d0 0x7f68001035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:04.641 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.641+0000 7f680498e640 1 -- 172.21.15.97:0/2930220209 >> 172.21.15.97:0/2930220209 conn(0x7f6800077fa0 msgr2=0x7f68000783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:29:04.641 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.641+0000 7f680498e640 1 -- 172.21.15.97:0/2930220209 shutdown_connections 2024-04-23T10:29:04.641 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.641+0000 7f680498e640 1 -- 172.21.15.97:0/2930220209 wait complete. 2024-04-23T10:29:04.643 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.642+0000 7f680498e640 1 Processor -- start 2024-04-23T10:29:04.643 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.642+0000 7f680498e640 1 -- start start 2024-04-23T10:29:04.644 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.642+0000 7f680498e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68001031d0 0x7f6800100e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:04.644 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.642+0000 7f680498e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6800103af0 0x7f6800101360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:04.644 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.642+0000 7f680498e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f680010c5a0 0x7f68001018a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:04.644 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.642+0000 7f680498e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f68001121d0 con 0x7f680010c5a0 2024-04-23T10:29:04.644 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.642+0000 7f680498e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6800112050 con 0x7f6800103af0 2024-04-23T10:29:04.644 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.642+0000 7f680498e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6800112350 con 0x7f68001031d0 2024-04-23T10:29:04.644 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.642+0000 7f67fed9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f680010c5a0 0x7f68001018a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:04.644 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.642+0000 7f67fed9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f680010c5a0 0x7f68001018a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47400/0 (socket says 172.21.15.97:47400) 2024-04-23T10:29:04.644 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.642+0000 7f67fed9d640 1 -- 172.21.15.97:0/2220836207 learned_addr learned my addr 172.21.15.97:0/2220836207 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:29:04.644 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.642+0000 7f67fdd9b640 1 --2- 172.21.15.97:0/2220836207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6800103af0 0x7f6800101360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:04.644 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.643+0000 7f67fe59c640 1 --2- 172.21.15.97:0/2220836207 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68001031d0 0x7f6800100e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:04.645 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.643+0000 7f67fed9d640 1 -- 172.21.15.97:0/2220836207 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68001031d0 msgr2=0x7f6800100e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:04.645 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.643+0000 7f67fed9d640 1 --2- 172.21.15.97:0/2220836207 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68001031d0 0x7f6800100e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:04.645 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.643+0000 7f67fed9d640 1 -- 172.21.15.97:0/2220836207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6800103af0 msgr2=0x7f6800101360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:04.645 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.643+0000 7f67fed9d640 1 --2- 172.21.15.97:0/2220836207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6800103af0 0x7f6800101360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:04.645 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.643+0000 7f67fed9d640 1 -- 172.21.15.97:0/2220836207 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68001a6b50 con 0x7f680010c5a0 2024-04-23T10:29:04.645 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.643+0000 7f67fdd9b640 1 --2- 172.21.15.97:0/2220836207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6800103af0 0x7f6800101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:29:04.645 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.643+0000 7f67fe59c640 1 --2- 172.21.15.97:0/2220836207 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68001031d0 0x7f6800100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:29:04.645 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.643+0000 7f67fed9d640 1 --2- 172.21.15.97:0/2220836207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f680010c5a0 0x7f68001018a0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f67f800eb10 tx=0x7f67f800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:29:04.645 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.643+0000 7f67ef7fe640 1 -- 172.21.15.97:0/2220836207 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f8004b20 con 0x7f680010c5a0 2024-04-23T10:29:04.645 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.643+0000 7f67ef7fe640 1 -- 172.21.15.97:0/2220836207 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67f8004cc0 con 0x7f680010c5a0 2024-04-23T10:29:04.645 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.643+0000 7f680498e640 1 -- 172.21.15.97:0/2220836207 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68001a6e40 con 0x7f680010c5a0 2024-04-23T10:29:04.645 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.643+0000 7f67ef7fe640 1 -- 172.21.15.97:0/2220836207 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f8010670 con 0x7f680010c5a0 2024-04-23T10:29:04.645 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.643+0000 7f680498e640 1 -- 172.21.15.97:0/2220836207 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68001a7310 con 0x7f680010c5a0 2024-04-23T10:29:04.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:29:04 smithi183 ceph-mon[32317]: pgmap v89: 97 pgs: 97 active+clean; 2.9 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 31 KiB/s wr, 31 op/s 2024-04-23T10:29:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.645+0000 7f67ef7fe640 1 -- 172.21.15.97:0/2220836207 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f67f8005000 con 0x7f680010c5a0 2024-04-23T10:29:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.646+0000 7f680777e740 1 Processor -- start 2024-04-23T10:29:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.646+0000 7f680777e740 1 -- start start 2024-04-23T10:29:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.646+0000 7f680777e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5558a7f59620 0x5558a7f59a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.646+0000 7f680777e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5558a7f59f40 0x5558a7f642c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.646+0000 7f680777e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5558a7f64800 0x5558a7f66bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.646+0000 7f680777e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5558a7ebe970 con 0x5558a7f59f40 2024-04-23T10:29:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.646+0000 7f680777e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5558a7e5b4a0 con 0x5558a7f59620 2024-04-23T10:29:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.646+0000 7f680777e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5558a7eba930 con 0x5558a7f64800 2024-04-23T10:29:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.646+0000 7f67ef7fe640 1 --2- 172.21.15.97:0/2220836207 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f67e80a3df0 0x7f67e80a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.646+0000 7f67fe59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5558a7f59f40 0x5558a7f642c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.646+0000 7f67fe59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5558a7f59f40 0x5558a7f642c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47410/0 (socket says 172.21.15.97:47410) 2024-04-23T10:29:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.646+0000 7f67fe59c640 1 -- 172.21.15.97:0/1739624499 learned_addr learned my addr 172.21.15.97:0/1739624499 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:29:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.646+0000 7f67fed9d640 1 --2- 172.21.15.97:0/2220836207 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f67e80a3df0 0x7f67e80a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.647+0000 7f67fdd9b640 1 --2- 172.21.15.97:0/1739624499 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5558a7f64800 0x5558a7f66bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.647+0000 7f67fdd9b640 1 --2- 172.21.15.97:0/1739624499 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5558a7f59620 0x5558a7f59a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.647+0000 7f67ef7fe640 1 -- 172.21.15.97:0/2220836207 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f67f80d0db0 con 0x7f680010c5a0 2024-04-23T10:29:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.647+0000 7f67fe59c640 1 -- 172.21.15.97:0/1739624499 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5558a7f64800 msgr2=0x5558a7f66bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:29:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.647+0000 7f67fe59c640 1 --2- 172.21.15.97:0/1739624499 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5558a7f64800 0x5558a7f66bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:04.654 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.647+0000 7f67fe59c640 1 -- 172.21.15.97:0/1739624499 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5558a7f59620 msgr2=0x5558a7f59a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:29:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.647+0000 7f67fe59c640 1 --2- 172.21.15.97:0/1739624499 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5558a7f59620 0x5558a7f59a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.647+0000 7f67fe59c640 1 -- 172.21.15.97:0/1739624499 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558a7bd1210 con 0x5558a7f59f40 2024-04-23T10:29:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.647+0000 7f67fdd9b640 1 --2- 172.21.15.97:0/1739624499 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5558a7f59620 0x5558a7f59a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:29:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.647+0000 7f67fed9d640 1 --2- 172.21.15.97:0/2220836207 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f67e80a3df0 0x7f67e80a62b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f67f8099200 tx=0x7f67f80bd040 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:29:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.647+0000 7f67fe59c640 1 --2- 172.21.15.97:0/1739624499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5558a7f59f40 0x5558a7f642c0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f67f000bd40 tx=0x7f67f000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f2cad39df8f2265 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:29:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.647+0000 7f67ecff9640 1 -- 172.21.15.97:0/1739624499 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f0015070 con 0x5558a7f59f40 2024-04-23T10:29:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.647+0000 7f67ecff9640 1 -- 172.21.15.97:0/1739624499 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67f000be30 con 0x5558a7f59f40 2024-04-23T10:29:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.647+0000 7f67ecff9640 1 -- 172.21.15.97:0/1739624499 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f00106e0 con 0x5558a7f59f40 2024-04-23T10:29:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.648+0000 7f680777e740 1 -- 172.21.15.97:0/1739624499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5558a7f59f40 msgr2=0x5558a7f642c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.648+0000 7f680777e740 1 --2- 172.21.15.97:0/1739624499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5558a7f59f40 0x5558a7f642c0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f67f000bd40 tx=0x7f67f000f730 comp rx=0 tx=0).stop 2024-04-23T10:29:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.648+0000 7f680777e740 1 -- 172.21.15.97:0/1739624499 shutdown_connections 2024-04-23T10:29:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.648+0000 7f680777e740 1 --2- 172.21.15.97:0/1739624499 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5558a7f64800 0x5558a7f66bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.648+0000 7f680777e740 1 --2- 172.21.15.97:0/1739624499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5558a7f59f40 0x5558a7f642c0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:04.656 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.648+0000 7f680777e740 1 --2- 172.21.15.97:0/1739624499 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5558a7f59620 0x5558a7f59a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:04.656 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.648+0000 7f680777e740 1 -- 172.21.15.97:0/1739624499 >> 172.21.15.97:0/1739624499 conn(0x5558a7f55b30 msgr2=0x5558a7f55f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:29:04.656 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.648+0000 7f680777e740 1 -- 172.21.15.97:0/1739624499 shutdown_connections 2024-04-23T10:29:04.656 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.648+0000 7f680777e740 1 -- 172.21.15.97:0/1739624499 wait complete. 2024-04-23T10:29:04.656 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.648+0000 7f680777e740 1 -- start start 2024-04-23T10:29:04.656 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.648+0000 7f680777e740 1 Processor -- start 2024-04-23T10:29:04.656 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.649+0000 7f680777e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5558a7f59620 0x5558a7f640d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:04.656 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.649+0000 7f680777e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5558a7f59f40 0x5558a7f613c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:04.656 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.649+0000 7f680777e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5558a7f64800 0x5558a7f61900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:04.656 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.649+0000 7f680777e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5558a7f67430 con 0x5558a7f59620 2024-04-23T10:29:04.656 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.649+0000 7f680777e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5558a7e590b0 con 0x5558a7f59f40 2024-04-23T10:29:04.657 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.649+0000 7f680777e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5558a7f675b0 con 0x5558a7f64800 2024-04-23T10:29:04.657 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.649+0000 7f680777e740 10 client.0 _ll_register_callbacks cb 0x5558a7e46d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:29:04.657 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.649+0000 7f67fdd9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5558a7f59620 0x5558a7f640d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:04.657 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.649+0000 7f67fdd9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5558a7f59620 0x5558a7f640d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47420/0 (socket says 172.21.15.97:47420) 2024-04-23T10:29:04.657 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.649+0000 7f67fdd9b640 1 -- 172.21.15.97:0/3337313328 learned_addr learned my addr 172.21.15.97:0/3337313328 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:29:04.657 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.649+0000 7f67fe59c640 1 --2- 172.21.15.97:0/3337313328 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5558a7f59f40 0x5558a7f613c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:04.657 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.649+0000 7f67fdd9b640 1 -- 172.21.15.97:0/3337313328 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5558a7f64800 msgr2=0x5558a7f61900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:29:04.657 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.649+0000 7f67fdd9b640 1 --2- 172.21.15.97:0/3337313328 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5558a7f64800 0x5558a7f61900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:04.657 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.649+0000 7f67fdd9b640 1 -- 172.21.15.97:0/3337313328 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5558a7f59f40 msgr2=0x5558a7f613c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:29:04.657 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.649+0000 7f67fdd9b640 1 --2- 172.21.15.97:0/3337313328 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5558a7f59f40 0x5558a7f613c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:04.657 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.649+0000 7f67fdd9b640 1 -- 172.21.15.97:0/3337313328 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558a7c9d620 con 0x5558a7f59620 2024-04-23T10:29:04.657 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.649+0000 7f67fe59c640 1 --2- 172.21.15.97:0/3337313328 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5558a7f59f40 0x5558a7f613c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:29:04.657 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.649+0000 7f67fdd9b640 1 --2- 172.21.15.97:0/3337313328 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5558a7f59620 0x5558a7f640d0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f67f4005ab0 tx=0x7f67f4032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45ce190a2464a917 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:29:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.649+0000 7f67e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:29:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.649+0000 7f67e5ffb640 1 -- 172.21.15.97:0/3337313328 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f4044070 con 0x5558a7f59620 2024-04-23T10:29:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.649+0000 7f67e5ffb640 1 -- 172.21.15.97:0/3337313328 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f67f4004240 con 0x5558a7f59620 2024-04-23T10:29:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.649+0000 7f680777e740 1 -- 172.21.15.97:0/3337313328 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67f000be30 con 0x5558a7f59620 2024-04-23T10:29:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.650+0000 7f67e5ffb640 1 -- 172.21.15.97:0/3337313328 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f67f4050050 con 0x5558a7f59620 2024-04-23T10:29:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.650+0000 7f680777e740 10 client.16035 fetch_fsmap learned FSMap version 26 2024-04-23T10:29:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.650+0000 7f680777e740 1 -- 172.21.15.97:0/3337313328 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5558a7f73240 con 0x5558a7f59620 2024-04-23T10:29:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.650+0000 7f67e5ffb640 1 -- 172.21.15.97:0/3337313328 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67f000be30 con 0x5558a7f59620 2024-04-23T10:29:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.650+0000 7f67e5ffb640 1 -- 172.21.15.97:0/3337313328 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f67f403d460 con 0x5558a7f59620 2024-04-23T10:29:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.650+0000 7f680777e740 10 client.16035 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:29:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.650+0000 7f680777e740 10 client.16035 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:29:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.650+0000 7f680777e740 20 client.16035 populate_metadata read hostname 'smithi097' 2024-04-23T10:29:04.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.650+0000 7f680777e740 1 --2- 172.21.15.97:0/3337313328 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5558a7f746b0 0x5558a7f76ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:04.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.650+0000 7f680777e740 4 client.16035 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:29:04.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.650+0000 7f680777e740 1 -- 172.21.15.97:0/3337313328 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5558a7ecc390 con 0x5558a7f746b0 2024-04-23T10:29:04.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.650+0000 7f67fe59c640 1 --2- 172.21.15.97:0/3337313328 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5558a7f746b0 0x5558a7f76ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:04.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.651+0000 7f67e5ffb640 10 client.16035 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:29:04.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.651+0000 7f67fe59c640 1 --2- 172.21.15.97:0/3337313328 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5558a7f746b0 0x5558a7f76ae0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98648eff02403a92 server_cookie=d6194e31d6579c8e in_seq=0 out_seq=0 2024-04-23T10:29:04.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.652+0000 7f67e5ffb640 1 -- 172.21.15.97:0/3337313328 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5558a7ecc390 con 0x5558a7f746b0 2024-04-23T10:29:04.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.652+0000 7f67e5ffb640 10 client.16035 handle_command_reply: tid=0 2024-04-23T10:29:04.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.652+0000 7f680777e740 1 client.16035 shutdown 2024-04-23T10:29:04.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.652+0000 7f680777e740 1 -- 172.21.15.97:0/3337313328 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5558a7f59620 msgr2=0x5558a7f640d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:04.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.652+0000 7f680777e740 1 --2- 172.21.15.97:0/3337313328 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5558a7f59620 0x5558a7f640d0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f67f4005ab0 tx=0x7f67f4032f20 comp rx=0 tx=0).stop 2024-04-23T10:29:04.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.652+0000 7f680777e740 1 -- 172.21.15.97:0/3337313328 shutdown_connections 2024-04-23T10:29:04.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.652+0000 7f680777e740 1 --2- 172.21.15.97:0/3337313328 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5558a7f746b0 0x5558a7f76ae0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:04.660 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.652+0000 7f680777e740 1 --2- 172.21.15.97:0/3337313328 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5558a7f64800 0x5558a7f61900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:04.660 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.652+0000 7f680777e740 1 --2- 172.21.15.97:0/3337313328 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5558a7f59f40 0x5558a7f613c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:04.660 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.652+0000 7f67e5ffb640 0 client.16035 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:29:04.660 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.652+0000 7f680777e740 1 --2- 172.21.15.97:0/3337313328 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5558a7f59620 0x5558a7f640d0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:04.660 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.652+0000 7f680777e740 1 -- 172.21.15.97:0/3337313328 >> 172.21.15.97:0/3337313328 conn(0x5558a7f55b30 msgr2=0x5558a7f5b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:29:04.660 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.652+0000 7f680777e740 1 -- 172.21.15.97:0/3337313328 shutdown_connections 2024-04-23T10:29:04.660 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.652+0000 7f680777e740 1 -- 172.21.15.97:0/3337313328 wait complete. 2024-04-23T10:29:04.660 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.652+0000 7f680777e740 20 client.16035 trim_cache size 0 max 16384 2024-04-23T10:29:04.671 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.667+0000 7f680777e740 1 Processor -- start 2024-04-23T10:29:04.671 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.667+0000 7f680777e740 1 -- start start 2024-04-23T10:29:04.671 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.667+0000 7f680777e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5558a7f59620 0x5558a7f702f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:04.671 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.667+0000 7f680777e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5558a7f59f40 0x5558a7f5c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:04.671 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.667+0000 7f680777e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5558a7f64800 0x5558a7f63c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:04.672 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.667+0000 7f680777e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f68001afb20 con 0x5558a7f59f40 2024-04-23T10:29:04.672 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.667+0000 7f680777e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f68001b1660 con 0x5558a7f64800 2024-04-23T10:29:04.672 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.667+0000 7f680777e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f68001b01f0 con 0x5558a7f59620 2024-04-23T10:29:04.672 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.667+0000 7f67fe59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5558a7f59f40 0x5558a7f5c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:04.672 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.668+0000 7f67fe59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5558a7f59f40 0x5558a7f5c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47422/0 (socket says 172.21.15.97:47422) 2024-04-23T10:29:04.672 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.668+0000 7f67fe59c640 1 -- 172.21.15.97:0/815041883 learned_addr learned my addr 172.21.15.97:0/815041883 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:29:04.672 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.668+0000 7f67fe59c640 1 -- 172.21.15.97:0/815041883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5558a7f59620 msgr2=0x5558a7f702f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:29:04.672 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.668+0000 7f67fdd9b640 1 --2- 172.21.15.97:0/815041883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5558a7f59620 0x5558a7f702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:04.672 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.668+0000 7f67fe59c640 1 --2- 172.21.15.97:0/815041883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5558a7f59620 0x5558a7f702f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:04.672 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.668+0000 7f67fe59c640 1 -- 172.21.15.97:0/815041883 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5558a7f64800 msgr2=0x5558a7f63c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:29:04.672 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.668+0000 7f67fdd9b640 1 --2- 172.21.15.97:0/815041883 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5558a7f64800 0x5558a7f63c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:04.673 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.668+0000 7f67fe59c640 1 --2- 172.21.15.97:0/815041883 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5558a7f64800 0x5558a7f63c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:04.673 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.668+0000 7f67fe59c640 1 -- 172.21.15.97:0/815041883 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558a7f61c60 con 0x5558a7f59f40 2024-04-23T10:29:04.673 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.668+0000 7f67fe59c640 1 --2- 172.21.15.97:0/815041883 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5558a7f59f40 0x5558a7f5c140 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f67f0011f50 tx=0x7f67f0011ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1c50367b3b3b5f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:29:04.673 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.668+0000 7f67e7fff640 1 -- 172.21.15.97:0/815041883 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f0015040 con 0x5558a7f59f40 2024-04-23T10:29:04.673 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.668+0000 7f67e7fff640 1 -- 172.21.15.97:0/815041883 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67f002d070 con 0x5558a7f59f40 2024-04-23T10:29:04.673 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.668+0000 7f67e7fff640 1 -- 172.21.15.97:0/815041883 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f0011a20 con 0x5558a7f59f40 2024-04-23T10:29:04.673 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.669+0000 7f680777e740 1 -- 172.21.15.97:0/815041883 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5558a7f59f40 msgr2=0x5558a7f5c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:04.673 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.669+0000 7f680777e740 1 --2- 172.21.15.97:0/815041883 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5558a7f59f40 0x5558a7f5c140 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f67f0011f50 tx=0x7f67f0011ee0 comp rx=0 tx=0).stop 2024-04-23T10:29:04.673 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.669+0000 7f680777e740 1 -- 172.21.15.97:0/815041883 shutdown_connections 2024-04-23T10:29:04.673 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.669+0000 7f680777e740 1 --2- 172.21.15.97:0/815041883 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5558a7f64800 0x5558a7f63c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:04.673 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.669+0000 7f680777e740 1 --2- 172.21.15.97:0/815041883 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5558a7f59f40 0x5558a7f5c140 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:04.673 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.669+0000 7f680777e740 1 --2- 172.21.15.97:0/815041883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5558a7f59620 0x5558a7f702f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:04.673 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.669+0000 7f680777e740 1 -- 172.21.15.97:0/815041883 >> 172.21.15.97:0/815041883 conn(0x5558a7f55b30 msgr2=0x5558a7f5b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:29:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.669+0000 7f680777e740 1 -- 172.21.15.97:0/815041883 shutdown_connections 2024-04-23T10:29:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.669+0000 7f680777e740 1 -- 172.21.15.97:0/815041883 wait complete. 2024-04-23T10:29:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.669+0000 7f680777e740 1 -- start start 2024-04-23T10:29:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.669+0000 7f680777e740 1 Processor -- start 2024-04-23T10:29:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.669+0000 7f680777e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5558a7f59620 0x5558a7f6cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.669+0000 7f680777e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5558a7f59f40 0x5558a7f5ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.669+0000 7f680777e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5558a7f64800 0x5558a7f5e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.669+0000 7f680777e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f68001b0f90 con 0x5558a7f64800 2024-04-23T10:29:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.669+0000 7f680777e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f68001b08c0 con 0x5558a7f59620 2024-04-23T10:29:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.669+0000 7f680777e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f68001adb00 con 0x5558a7f59f40 2024-04-23T10:29:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.670+0000 7f680777e740 10 client.0 _ll_register_callbacks cb 0x7f67f4032bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:29:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.670+0000 7f67fdd9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5558a7f64800 0x5558a7f5e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.670+0000 7f67fdd9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5558a7f64800 0x5558a7f5e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47432/0 (socket says 172.21.15.97:47432) 2024-04-23T10:29:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.670+0000 7f67fdd9b640 1 -- 172.21.15.97:0/681663545 learned_addr learned my addr 172.21.15.97:0/681663545 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:29:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.670+0000 7f67fe59c640 1 --2- 172.21.15.97:0/681663545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5558a7f59f40 0x5558a7f5ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.670+0000 7f67fdd9b640 1 -- 172.21.15.97:0/681663545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5558a7f59f40 msgr2=0x5558a7f5ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:29:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.670+0000 7f67fdd9b640 1 --2- 172.21.15.97:0/681663545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5558a7f59f40 0x5558a7f5ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.670+0000 7f67fdd9b640 1 -- 172.21.15.97:0/681663545 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5558a7f59620 msgr2=0x5558a7f6cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:29:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.670+0000 7f67fdd9b640 1 --2- 172.21.15.97:0/681663545 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5558a7f59620 0x5558a7f6cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.670+0000 7f67fdd9b640 1 -- 172.21.15.97:0/681663545 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558a7f5eb20 con 0x5558a7f64800 2024-04-23T10:29:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.670+0000 7f67fe59c640 1 --2- 172.21.15.97:0/681663545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5558a7f59f40 0x5558a7f5ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:29:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.670+0000 7f67fdd9b640 1 --2- 172.21.15.97:0/681663545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5558a7f64800 0x5558a7f5e2f0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f67f402a4c0 tx=0x7f67f403df40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffe89bcc00fa8740 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:29:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.670+0000 7f67e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:29:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.670+0000 7f67e5ffb640 1 -- 172.21.15.97:0/681663545 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f4044040 con 0x5558a7f64800 2024-04-23T10:29:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.670+0000 7f67e5ffb640 1 -- 172.21.15.97:0/681663545 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f67f4005270 con 0x5558a7f64800 2024-04-23T10:29:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.670+0000 7f680777e740 1 -- 172.21.15.97:0/681663545 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67f002d070 con 0x5558a7f64800 2024-04-23T10:29:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.670+0000 7f67e5ffb640 1 -- 172.21.15.97:0/681663545 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f67f4035040 con 0x5558a7f64800 2024-04-23T10:29:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.670+0000 7f680777e740 10 client.16041 fetch_fsmap learned FSMap version 26 2024-04-23T10:29:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.670+0000 7f680777e740 1 -- 172.21.15.97:0/681663545 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5558a7fdad60 con 0x5558a7f64800 2024-04-23T10:29:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.671+0000 7f67e5ffb640 1 -- 172.21.15.97:0/681663545 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67f002d070 con 0x5558a7f64800 2024-04-23T10:29:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.671+0000 7f67e5ffb640 1 -- 172.21.15.97:0/681663545 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f67f40046c0 con 0x5558a7f64800 2024-04-23T10:29:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.671+0000 7f680777e740 10 client.16041 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:29:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.671+0000 7f680777e740 10 client.16041 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:29:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.671+0000 7f680777e740 20 client.16041 populate_metadata read hostname 'smithi097' 2024-04-23T10:29:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.671+0000 7f680777e740 1 --2- 172.21.15.97:0/681663545 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5558a7fdb4e0 0x5558a7fdd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.671+0000 7f680777e740 4 client.16041 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:29:04.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.671+0000 7f680777e740 1 -- 172.21.15.97:0/681663545 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5558a7ec6200 con 0x5558a7fdb4e0 2024-04-23T10:29:04.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.671+0000 7f67fe59c640 1 --2- 172.21.15.97:0/681663545 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5558a7fdb4e0 0x5558a7fdd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:04.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.672+0000 7f67e5ffb640 10 client.16041 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:29:04.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.672+0000 7f67fe59c640 1 --2- 172.21.15.97:0/681663545 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5558a7fdb4e0 0x5558a7fdd900 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6777a63a22db7bd server_cookie=49e1fbfa7f45584b in_seq=0 out_seq=0 2024-04-23T10:29:04.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.683+0000 7f67e5ffb640 1 -- 172.21.15.97:0/681663545 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5558a7ec6200 con 0x5558a7fdb4e0 2024-04-23T10:29:04.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.683+0000 7f67e5ffb640 10 client.16041 handle_command_reply: tid=0 2024-04-23T10:29:04.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.683+0000 7f680777e740 1 client.16041 shutdown 2024-04-23T10:29:04.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.684+0000 7f680777e740 1 -- 172.21.15.97:0/681663545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5558a7f64800 msgr2=0x5558a7f5e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:04.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.684+0000 7f680777e740 1 --2- 172.21.15.97:0/681663545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5558a7f64800 0x5558a7f5e2f0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f67f402a4c0 tx=0x7f67f403df40 comp rx=0 tx=0).stop 2024-04-23T10:29:04.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.684+0000 7f680777e740 1 -- 172.21.15.97:0/681663545 shutdown_connections 2024-04-23T10:29:04.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.684+0000 7f680777e740 1 --2- 172.21.15.97:0/681663545 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5558a7fdb4e0 0x5558a7fdd900 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:04.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.684+0000 7f680777e740 1 --2- 172.21.15.97:0/681663545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5558a7f64800 0x5558a7f5e2f0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:04.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.684+0000 7f680777e740 1 --2- 172.21.15.97:0/681663545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5558a7f59f40 0x5558a7f5ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:04.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.684+0000 7f680777e740 1 --2- 172.21.15.97:0/681663545 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5558a7f59620 0x5558a7f6cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:04.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.684+0000 7f67e5ffb640 0 client.16041 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:29:04.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.684+0000 7f680777e740 1 -- 172.21.15.97:0/681663545 >> 172.21.15.97:0/681663545 conn(0x5558a7f55b30 msgr2=0x5558a7f5b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:29:04.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.684+0000 7f680777e740 1 -- 172.21.15.97:0/681663545 shutdown_connections 2024-04-23T10:29:04.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.684+0000 7f680777e740 1 -- 172.21.15.97:0/681663545 wait complete. 2024-04-23T10:29:04.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.684+0000 7f680777e740 20 client.16041 trim_cache size 0 max 16384 2024-04-23T10:29:04.685 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:29:04.685 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:29:04.685 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:29:04.685 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:29:04.685 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.684+0000 7f680498e640 1 -- 172.21.15.97:0/2220836207 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f67e80a3df0 msgr2=0x7f67e80a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.684+0000 7f680498e640 1 --2- 172.21.15.97:0/2220836207 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f67e80a3df0 0x7f67e80a62b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f67f8099200 tx=0x7f67f80bd040 comp rx=0 tx=0).stop 2024-04-23T10:29:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.684+0000 7f680498e640 1 -- 172.21.15.97:0/2220836207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f680010c5a0 msgr2=0x7f68001018a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.684+0000 7f680498e640 1 --2- 172.21.15.97:0/2220836207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f680010c5a0 0x7f68001018a0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f67f800eb10 tx=0x7f67f800c2d0 comp rx=0 tx=0).stop 2024-04-23T10:29:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.684+0000 7f680498e640 1 -- 172.21.15.97:0/2220836207 shutdown_connections 2024-04-23T10:29:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.684+0000 7f680498e640 1 --2- 172.21.15.97:0/2220836207 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f67e80a3df0 0x7f67e80a62b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.684+0000 7f680498e640 1 --2- 172.21.15.97:0/2220836207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f680010c5a0 0x7f68001018a0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.684+0000 7f680498e640 1 --2- 172.21.15.97:0/2220836207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6800103af0 0x7f6800101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.684+0000 7f680498e640 1 --2- 172.21.15.97:0/2220836207 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68001031d0 0x7f6800100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.684+0000 7f680498e640 1 -- 172.21.15.97:0/2220836207 >> 172.21.15.97:0/2220836207 conn(0x7f6800077fa0 msgr2=0x7f68000fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:29:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.684+0000 7f680498e640 1 -- 172.21.15.97:0/2220836207 shutdown_connections 2024-04-23T10:29:04.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:04.685+0000 7f680498e640 1 -- 172.21.15.97:0/2220836207 wait complete. 2024-04-23T10:29:06.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:29:06 smithi196 ceph-mon[32919]: pgmap v90: 97 pgs: 97 active+clean; 2.9 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-04-23T10:29:06.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:29:06 smithi097 ceph-mon[29197]: pgmap v90: 97 pgs: 97 active+clean; 2.9 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-04-23T10:29:06.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:29:06 smithi183 ceph-mon[32317]: pgmap v90: 97 pgs: 97 active+clean; 2.9 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-04-23T10:29:08.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:29:08 smithi196 ceph-mon[32919]: pgmap v91: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 26 KiB/s wr, 30 op/s 2024-04-23T10:29:08.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:29:08 smithi097 ceph-mon[29197]: pgmap v91: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 26 KiB/s wr, 30 op/s 2024-04-23T10:29:08.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:29:08 smithi183 ceph-mon[32317]: pgmap v91: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 26 KiB/s wr, 30 op/s 2024-04-23T10:29:10.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:29:10 smithi196 ceph-mon[32919]: pgmap v92: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 26 KiB/s wr, 25 op/s 2024-04-23T10:29:10.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:29:10 smithi097 ceph-mon[29197]: pgmap v92: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 26 KiB/s wr, 25 op/s 2024-04-23T10:29:10.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:29:10 smithi183 ceph-mon[32317]: pgmap v92: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 26 KiB/s wr, 25 op/s 2024-04-23T10:29:12.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:29:12 smithi196 ceph-mon[32919]: pgmap v93: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 26 KiB/s wr, 25 op/s 2024-04-23T10:29:12.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:29:12 smithi097 ceph-mon[29197]: pgmap v93: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 26 KiB/s wr, 25 op/s 2024-04-23T10:29:12.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:29:12 smithi183 ceph-mon[32317]: pgmap v93: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 26 KiB/s wr, 25 op/s 2024-04-23T10:29:14.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:29:14 smithi196 ceph-mon[32919]: pgmap v94: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 27 KiB/s wr, 30 op/s 2024-04-23T10:29:14.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:29:14 smithi097 ceph-mon[29197]: pgmap v94: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 27 KiB/s wr, 30 op/s 2024-04-23T10:29:14.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:29:14 smithi183 ceph-mon[32317]: pgmap v94: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 27 KiB/s wr, 30 op/s 2024-04-23T10:29:16.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:29:16 smithi196 ceph-mon[32919]: pgmap v95: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.2 KiB/s wr, 10 op/s 2024-04-23T10:29:16.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:29:16 smithi097 ceph-mon[29197]: pgmap v95: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.2 KiB/s wr, 10 op/s 2024-04-23T10:29:16.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:29:16 smithi183 ceph-mon[32317]: pgmap v95: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.2 KiB/s wr, 10 op/s 2024-04-23T10:29:17.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.780+0000 7f56e370f640 1 -- 172.21.15.97:0/2255479986 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56dc074b70 msgr2=0x7f56dc10e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:17.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.780+0000 7f56e370f640 1 --2- 172.21.15.97:0/2255479986 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56dc074b70 0x7f56dc10e300 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f56cc009900 tx=0x7f56cc02a480 comp rx=0 tx=0).stop 2024-04-23T10:29:17.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.780+0000 7f56e370f640 1 -- 172.21.15.97:0/2255479986 shutdown_connections 2024-04-23T10:29:17.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.780+0000 7f56e370f640 1 --2- 172.21.15.97:0/2255479986 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f56dc10e840 0x7f56dc110c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:17.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.780+0000 7f56e370f640 1 --2- 172.21.15.97:0/2255479986 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56dc074b70 0x7f56dc10e300 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:17.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.780+0000 7f56e370f640 1 --2- 172.21.15.97:0/2255479986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56dc074250 0x7f56dc074630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:17.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.780+0000 7f56e370f640 1 -- 172.21.15.97:0/2255479986 >> 172.21.15.97:0/2255479986 conn(0x7f56dc0793e0 msgr2=0x7f56dc0797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:29:17.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.780+0000 7f56e370f640 1 -- 172.21.15.97:0/2255479986 shutdown_connections 2024-04-23T10:29:17.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.780+0000 7f56e370f640 1 -- 172.21.15.97:0/2255479986 wait complete. 2024-04-23T10:29:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.780+0000 7f56e370f640 1 Processor -- start 2024-04-23T10:29:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.780+0000 7f56e370f640 1 -- start start 2024-04-23T10:29:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.781+0000 7f56e370f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f56dc074250 0x7f56dc1a0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.781+0000 7f56e370f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56dc074b70 0x7f56dc1a0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.781+0000 7f56e370f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56dc10e840 0x7f56dc19a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.781+0000 7f56e370f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f56dc1143f0 con 0x7f56dc074b70 2024-04-23T10:29:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.781+0000 7f56e370f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f56dc114270 con 0x7f56dc074250 2024-04-23T10:29:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.781+0000 7f56e370f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f56dc114570 con 0x7f56dc10e840 2024-04-23T10:29:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.781+0000 7f56e0caa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56dc074b70 0x7f56dc1a0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.781+0000 7f56e0caa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56dc074b70 0x7f56dc1a0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40066/0 (socket says 172.21.15.97:40066) 2024-04-23T10:29:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.781+0000 7f56e0caa640 1 -- 172.21.15.97:0/2559584046 learned_addr learned my addr 172.21.15.97:0/2559584046 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:29:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.781+0000 7f56e14ab640 1 --2- 172.21.15.97:0/2559584046 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f56dc074250 0x7f56dc1a0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:17.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.781+0000 7f56e1cac640 1 --2- 172.21.15.97:0/2559584046 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56dc10e840 0x7f56dc19a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:17.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.781+0000 7f56e0caa640 1 -- 172.21.15.97:0/2559584046 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56dc10e840 msgr2=0x7f56dc19a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:17.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.781+0000 7f56e0caa640 1 --2- 172.21.15.97:0/2559584046 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56dc10e840 0x7f56dc19a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:17.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.781+0000 7f56e0caa640 1 -- 172.21.15.97:0/2559584046 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f56dc074250 msgr2=0x7f56dc1a0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:17.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.781+0000 7f56e0caa640 1 --2- 172.21.15.97:0/2559584046 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f56dc074250 0x7f56dc1a0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:17.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.781+0000 7f56e0caa640 1 -- 172.21.15.97:0/2559584046 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56dc19af10 con 0x7f56dc074b70 2024-04-23T10:29:17.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.781+0000 7f56e0caa640 1 --2- 172.21.15.97:0/2559584046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56dc074b70 0x7f56dc1a0ad0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f56cc006010 tx=0x7f56cc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:29:17.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.781+0000 7f56d27fc640 1 -- 172.21.15.97:0/2559584046 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56cc03f030 con 0x7f56dc074b70 2024-04-23T10:29:17.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.781+0000 7f56d27fc640 1 -- 172.21.15.97:0/2559584046 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56cc003800 con 0x7f56dc074b70 2024-04-23T10:29:17.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.781+0000 7f56e370f640 1 -- 172.21.15.97:0/2559584046 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56dc19b1a0 con 0x7f56dc074b70 2024-04-23T10:29:17.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.782+0000 7f56d27fc640 1 -- 172.21.15.97:0/2559584046 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56cc035070 con 0x7f56dc074b70 2024-04-23T10:29:17.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.782+0000 7f56e370f640 1 -- 172.21.15.97:0/2559584046 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56dc1a7510 con 0x7f56dc074b70 2024-04-23T10:29:17.790 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.784+0000 7f56d27fc640 1 -- 172.21.15.97:0/2559584046 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f56cc003cb0 con 0x7f56dc074b70 2024-04-23T10:29:17.790 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.784+0000 7f56e6422740 1 Processor -- start 2024-04-23T10:29:17.790 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.784+0000 7f56e6422740 1 -- start start 2024-04-23T10:29:17.790 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.784+0000 7f56e6422740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562a2108a620 0x562a2108aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:17.790 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.784+0000 7f56e6422740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562a2108af40 0x562a210952c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:17.790 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.784+0000 7f56e6422740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562a21095800 0x562a21097bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:17.790 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.784+0000 7f56e6422740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x562a20fef970 con 0x562a21095800 2024-04-23T10:29:17.791 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.784+0000 7f56e6422740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562a20f8c4a0 con 0x562a2108a620 2024-04-23T10:29:17.791 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.784+0000 7f56e6422740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562a20feb930 con 0x562a2108af40 2024-04-23T10:29:17.791 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.785+0000 7f56d27fc640 1 --2- 172.21.15.97:0/2559584046 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f56c80a3e40 0x7f56c80a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:17.791 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.785+0000 7f56e0caa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562a21095800 0x562a21097bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:17.791 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.785+0000 7f56e14ab640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562a2108af40 0x562a210952c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:17.791 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.785+0000 7f56e1cac640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562a2108a620 0x562a2108aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:17.791 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.785+0000 7f56e14ab640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562a2108af40 0x562a210952c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.97:43560/0 (socket says 172.21.15.97:43560) 2024-04-23T10:29:17.791 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.785+0000 7f56e0caa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562a21095800 0x562a21097bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40074/0 (socket says 172.21.15.97:40074) 2024-04-23T10:29:17.791 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.785+0000 7f56e14ab640 1 -- 172.21.15.97:0/1961849720 learned_addr learned my addr 172.21.15.97:0/1961849720 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:29:17.791 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.785+0000 7f56e1cac640 1 --2- 172.21.15.97:0/2559584046 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f56c80a3e40 0x7f56c80a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:17.791 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.785+0000 7f56d27fc640 1 -- 172.21.15.97:0/2559584046 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f56cc0f10d0 con 0x7f56dc074b70 2024-04-23T10:29:17.791 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.785+0000 7f56e1cac640 1 -- 172.21.15.97:0/1961849720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562a2108af40 msgr2=0x562a210952c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:29:17.791 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.785+0000 7f56e1cac640 1 --2- 172.21.15.97:0/1961849720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562a2108af40 0x562a210952c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:17.792 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.785+0000 7f56e1cac640 1 -- 172.21.15.97:0/1961849720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562a21095800 msgr2=0x562a21097bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:29:17.792 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.785+0000 7f56e1cac640 1 --2- 172.21.15.97:0/1961849720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562a21095800 0x562a21097bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:17.792 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.785+0000 7f56e1cac640 1 -- 172.21.15.97:0/1961849720 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a20d02210 con 0x562a2108a620 2024-04-23T10:29:17.792 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.785+0000 7f56e1cac640 1 --2- 172.21.15.97:0/2559584046 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f56c80a3e40 0x7f56c80a6300 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f56d4005d50 tx=0x7f56d4015000 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:29:17.792 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.785+0000 7f56e1cac640 1 --2- 172.21.15.97:0/1961849720 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562a2108a620 0x562a2108aa00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f56dc1017c0 tx=0x7f56d4013f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a35367c7e439cae3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:29:17.792 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.785+0000 7f56c77fe640 1 -- 172.21.15.97:0/1961849720 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d4021070 con 0x562a2108a620 2024-04-23T10:29:17.792 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.785+0000 7f56c77fe640 1 -- 172.21.15.97:0/1961849720 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56d4013460 con 0x562a2108a620 2024-04-23T10:29:17.792 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.786+0000 7f56c77fe640 1 -- 172.21.15.97:0/1961849720 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d400ade0 con 0x562a2108a620 2024-04-23T10:29:17.792 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.786+0000 7f56e6422740 1 -- 172.21.15.97:0/1961849720 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562a2108a620 msgr2=0x562a2108aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:17.792 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.786+0000 7f56e6422740 1 --2- 172.21.15.97:0/1961849720 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562a2108a620 0x562a2108aa00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f56dc1017c0 tx=0x7f56d4013f10 comp rx=0 tx=0).stop 2024-04-23T10:29:17.792 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.786+0000 7f56e6422740 1 -- 172.21.15.97:0/1961849720 shutdown_connections 2024-04-23T10:29:17.792 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.786+0000 7f56e6422740 1 --2- 172.21.15.97:0/1961849720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562a21095800 0x562a21097bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:17.792 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.786+0000 7f56e6422740 1 --2- 172.21.15.97:0/1961849720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562a2108af40 0x562a210952c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:17.793 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.786+0000 7f56e6422740 1 --2- 172.21.15.97:0/1961849720 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562a2108a620 0x562a2108aa00 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:17.793 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.786+0000 7f56e6422740 1 -- 172.21.15.97:0/1961849720 >> 172.21.15.97:0/1961849720 conn(0x562a21086b30 msgr2=0x562a21086f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:29:17.793 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.786+0000 7f56e6422740 1 -- 172.21.15.97:0/1961849720 shutdown_connections 2024-04-23T10:29:17.793 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.786+0000 7f56e6422740 1 -- 172.21.15.97:0/1961849720 wait complete. 2024-04-23T10:29:17.793 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.787+0000 7f56e6422740 1 -- start start 2024-04-23T10:29:17.793 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.787+0000 7f56e6422740 1 Processor -- start 2024-04-23T10:29:17.793 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.787+0000 7f56e6422740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562a2108a620 0x562a210951c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:17.793 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.787+0000 7f56e6422740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562a2108af40 0x562a210924b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:17.793 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.787+0000 7f56e6422740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562a21095800 0x562a21092ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:17.793 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.787+0000 7f56e6422740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x562a210983d0 con 0x562a2108a620 2024-04-23T10:29:17.793 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.787+0000 7f56e6422740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562a20f8a0b0 con 0x562a2108af40 2024-04-23T10:29:17.793 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.787+0000 7f56e6422740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562a21098550 con 0x562a21095800 2024-04-23T10:29:17.793 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.787+0000 7f56e6422740 10 client.0 _ll_register_callbacks cb 0x562a20f77d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:29:17.793 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.787+0000 7f56e14ab640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562a2108a620 0x562a210951c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:17.793 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.787+0000 7f56e14ab640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562a2108a620 0x562a210951c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40080/0 (socket says 172.21.15.97:40080) 2024-04-23T10:29:17.794 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.787+0000 7f56e14ab640 1 -- 172.21.15.97:0/955106005 learned_addr learned my addr 172.21.15.97:0/955106005 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:29:17.794 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.787+0000 7f56e0caa640 1 --2- 172.21.15.97:0/955106005 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562a2108af40 0x562a210924b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:17.794 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.787+0000 7f56e1cac640 1 --2- 172.21.15.97:0/955106005 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562a21095800 0x562a21092ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:17.794 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.788+0000 7f56e14ab640 1 -- 172.21.15.97:0/955106005 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562a21095800 msgr2=0x562a21092ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:29:17.794 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.788+0000 7f56e14ab640 1 --2- 172.21.15.97:0/955106005 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562a21095800 0x562a21092ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:17.794 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.788+0000 7f56e14ab640 1 -- 172.21.15.97:0/955106005 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562a2108af40 msgr2=0x562a210924b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:29:17.794 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.788+0000 7f56e14ab640 1 --2- 172.21.15.97:0/955106005 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562a2108af40 0x562a210924b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:17.794 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.788+0000 7f56e14ab640 1 -- 172.21.15.97:0/955106005 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a20dce620 con 0x562a2108a620 2024-04-23T10:29:17.794 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.788+0000 7f56e0caa640 1 --2- 172.21.15.97:0/955106005 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562a2108af40 0x562a210924b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T10:29:17.794 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.788+0000 7f56e14ab640 1 --2- 172.21.15.97:0/955106005 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562a2108a620 0x562a210951c0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f56d8010b70 tx=0x7f56d8010f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d8e0482375d7bc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:29:17.794 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.788+0000 7f56c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:29:17.794 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.788+0000 7f56c4ff9640 1 -- 172.21.15.97:0/955106005 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d801e060 con 0x562a2108a620 2024-04-23T10:29:17.794 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.788+0000 7f56c4ff9640 1 -- 172.21.15.97:0/955106005 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f56d800fd10 con 0x562a2108a620 2024-04-23T10:29:17.794 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.788+0000 7f56e6422740 1 -- 172.21.15.97:0/955106005 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56d4013460 con 0x562a2108a620 2024-04-23T10:29:17.795 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.788+0000 7f56c4ff9640 1 -- 172.21.15.97:0/955106005 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f56d8004b20 con 0x562a2108a620 2024-04-23T10:29:17.795 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.788+0000 7f56e6422740 10 client.16053 fetch_fsmap learned FSMap version 26 2024-04-23T10:29:17.795 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.788+0000 7f56e6422740 1 -- 172.21.15.97:0/955106005 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562a210a43c0 con 0x562a2108a620 2024-04-23T10:29:17.795 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.788+0000 7f56c4ff9640 1 -- 172.21.15.97:0/955106005 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56d4013460 con 0x562a2108a620 2024-04-23T10:29:17.795 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.788+0000 7f56c4ff9640 1 -- 172.21.15.97:0/955106005 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f56d8022410 con 0x562a2108a620 2024-04-23T10:29:17.795 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.789+0000 7f56e6422740 10 client.16053 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:29:17.795 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.789+0000 7f56e6422740 10 client.16053 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:29:17.795 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.789+0000 7f56e6422740 20 client.16053 populate_metadata read hostname 'smithi097' 2024-04-23T10:29:17.795 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.789+0000 7f56e6422740 1 --2- 172.21.15.97:0/955106005 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x562a210a5880 0x562a210a7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:17.795 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.789+0000 7f56e6422740 4 client.16053 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:29:17.795 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.789+0000 7f56e6422740 1 -- 172.21.15.97:0/955106005 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a20ffd390 con 0x562a210a5880 2024-04-23T10:29:17.795 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.789+0000 7f56e0caa640 1 --2- 172.21.15.97:0/955106005 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x562a210a5880 0x562a210a7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:17.795 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.789+0000 7f56e0caa640 1 --2- 172.21.15.97:0/955106005 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x562a210a5880 0x562a210a7c70 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=380c13ec61b084a7 server_cookie=c9460229015af9cc in_seq=0 out_seq=0 2024-04-23T10:29:17.795 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.789+0000 7f56c4ff9640 10 client.16053 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:29:17.795 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.790+0000 7f56c4ff9640 1 -- 172.21.15.97:0/955106005 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562a20ffd390 con 0x562a210a5880 2024-04-23T10:29:17.796 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.790+0000 7f56c4ff9640 10 client.16053 handle_command_reply: tid=0 2024-04-23T10:29:17.796 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.790+0000 7f56e6422740 1 client.16053 shutdown 2024-04-23T10:29:17.796 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.790+0000 7f56e6422740 1 -- 172.21.15.97:0/955106005 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562a2108a620 msgr2=0x562a210951c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:17.796 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.790+0000 7f56e6422740 1 --2- 172.21.15.97:0/955106005 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562a2108a620 0x562a210951c0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f56d8010b70 tx=0x7f56d8010f70 comp rx=0 tx=0).stop 2024-04-23T10:29:17.796 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.790+0000 7f56e6422740 1 -- 172.21.15.97:0/955106005 shutdown_connections 2024-04-23T10:29:17.796 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.790+0000 7f56e6422740 1 --2- 172.21.15.97:0/955106005 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x562a210a5880 0x562a210a7c70 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:17.796 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.790+0000 7f56e6422740 1 --2- 172.21.15.97:0/955106005 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562a21095800 0x562a21092ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:17.796 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.790+0000 7f56e6422740 1 --2- 172.21.15.97:0/955106005 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562a2108af40 0x562a210924b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:17.796 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.790+0000 7f56e6422740 1 --2- 172.21.15.97:0/955106005 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562a2108a620 0x562a210951c0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:17.796 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.790+0000 7f56c4ff9640 0 client.16053 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:29:17.796 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.790+0000 7f56e6422740 1 -- 172.21.15.97:0/955106005 >> 172.21.15.97:0/955106005 conn(0x562a21086b30 msgr2=0x562a2108c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:29:17.796 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.790+0000 7f56e6422740 1 -- 172.21.15.97:0/955106005 shutdown_connections 2024-04-23T10:29:17.796 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.790+0000 7f56e6422740 1 -- 172.21.15.97:0/955106005 wait complete. 2024-04-23T10:29:17.796 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.790+0000 7f56e6422740 20 client.16053 trim_cache size 0 max 16384 2024-04-23T10:29:17.809 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.805+0000 7f56e6422740 1 Processor -- start 2024-04-23T10:29:17.809 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.805+0000 7f56e6422740 1 -- start start 2024-04-23T10:29:17.809 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.805+0000 7f56e6422740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562a2108a620 0x562a210a1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:17.809 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.805+0000 7f56e6422740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562a2108af40 0x562a21091610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:17.809 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.805+0000 7f56e6422740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562a21095800 0x562a2108c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.805+0000 7f56e6422740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f56dc1afc20 con 0x562a2108af40 2024-04-23T10:29:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.805+0000 7f56e6422740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f56dc1b1760 con 0x562a2108a620 2024-04-23T10:29:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.805+0000 7f56e6422740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f56dc1b02f0 con 0x562a21095800 2024-04-23T10:29:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.805+0000 7f56e0caa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562a2108af40 0x562a21091610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.806+0000 7f56e0caa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562a2108af40 0x562a21091610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40096/0 (socket says 172.21.15.97:40096) 2024-04-23T10:29:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.806+0000 7f56e0caa640 1 -- 172.21.15.97:0/3270660680 learned_addr learned my addr 172.21.15.97:0/3270660680 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:29:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.806+0000 7f56e14ab640 1 --2- 172.21.15.97:0/3270660680 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562a2108a620 0x562a210a1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.806+0000 7f56e1cac640 1 --2- 172.21.15.97:0/3270660680 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562a21095800 0x562a2108c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.806+0000 7f56e0caa640 1 -- 172.21.15.97:0/3270660680 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562a21095800 msgr2=0x562a2108c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:29:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.806+0000 7f56e0caa640 1 --2- 172.21.15.97:0/3270660680 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562a21095800 0x562a2108c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.806+0000 7f56e0caa640 1 -- 172.21.15.97:0/3270660680 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562a2108a620 msgr2=0x562a210a1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:29:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.806+0000 7f56e0caa640 1 --2- 172.21.15.97:0/3270660680 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562a2108a620 0x562a210a1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.806+0000 7f56e0caa640 1 -- 172.21.15.97:0/3270660680 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a21090a00 con 0x562a2108af40 2024-04-23T10:29:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.806+0000 7f56e14ab640 1 --2- 172.21.15.97:0/3270660680 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562a2108a620 0x562a210a1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:29:17.811 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.806+0000 7f56e0caa640 1 --2- 172.21.15.97:0/3270660680 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562a2108af40 0x562a21091610 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f56cc004650 tx=0x7f56cc0b9490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=903e7540a81045a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:29:17.811 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.806+0000 7f56c6ffd640 1 -- 172.21.15.97:0/3270660680 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56cc049040 con 0x562a2108af40 2024-04-23T10:29:17.811 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.806+0000 7f56c6ffd640 1 -- 172.21.15.97:0/3270660680 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56cc0c5d30 con 0x562a2108af40 2024-04-23T10:29:17.811 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.806+0000 7f56c6ffd640 1 -- 172.21.15.97:0/3270660680 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56cc0b9730 con 0x562a2108af40 2024-04-23T10:29:17.811 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.807+0000 7f56e6422740 1 -- 172.21.15.97:0/3270660680 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562a2108af40 msgr2=0x562a21091610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:17.811 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.807+0000 7f56e6422740 1 --2- 172.21.15.97:0/3270660680 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562a2108af40 0x562a21091610 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f56cc004650 tx=0x7f56cc0b9490 comp rx=0 tx=0).stop 2024-04-23T10:29:17.811 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.807+0000 7f56e6422740 1 -- 172.21.15.97:0/3270660680 shutdown_connections 2024-04-23T10:29:17.811 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.807+0000 7f56e6422740 1 --2- 172.21.15.97:0/3270660680 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562a21095800 0x562a2108c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:17.811 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.807+0000 7f56e6422740 1 --2- 172.21.15.97:0/3270660680 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562a2108af40 0x562a21091610 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:17.811 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.807+0000 7f56e6422740 1 --2- 172.21.15.97:0/3270660680 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562a2108a620 0x562a210a1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:17.811 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.807+0000 7f56e6422740 1 -- 172.21.15.97:0/3270660680 >> 172.21.15.97:0/3270660680 conn(0x562a21086b30 msgr2=0x562a2108c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:29:17.811 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.807+0000 7f56e6422740 1 -- 172.21.15.97:0/3270660680 shutdown_connections 2024-04-23T10:29:17.811 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.807+0000 7f56e6422740 1 -- 172.21.15.97:0/3270660680 wait complete. 2024-04-23T10:29:17.811 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.807+0000 7f56e6422740 1 -- start start 2024-04-23T10:29:17.811 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.807+0000 7f56e6422740 1 Processor -- start 2024-04-23T10:29:17.811 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.808+0000 7f56e6422740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562a2108a620 0x562a2109db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:17.812 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.808+0000 7f56e6422740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562a2108af40 0x562a21098ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:17.812 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.808+0000 7f56e6422740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562a21095800 0x562a21099430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:17.812 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.808+0000 7f56e6422740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f56dc1b1090 con 0x562a2108af40 2024-04-23T10:29:17.812 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.808+0000 7f56e6422740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f56dc1b09c0 con 0x562a21095800 2024-04-23T10:29:17.812 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.808+0000 7f56e6422740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f56dc1adc00 con 0x562a2108a620 2024-04-23T10:29:17.812 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.808+0000 7f56e6422740 10 client.0 _ll_register_callbacks cb 0x7f56d8010780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:29:17.812 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.808+0000 7f56e0caa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562a2108af40 0x562a21098ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:17.812 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.808+0000 7f56e0caa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562a2108af40 0x562a21098ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40102/0 (socket says 172.21.15.97:40102) 2024-04-23T10:29:17.812 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.808+0000 7f56e0caa640 1 -- 172.21.15.97:0/4168092762 learned_addr learned my addr 172.21.15.97:0/4168092762 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:29:17.812 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.808+0000 7f56e14ab640 1 --2- 172.21.15.97:0/4168092762 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562a2108a620 0x562a2109db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:17.812 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.808+0000 7f56e1cac640 1 --2- 172.21.15.97:0/4168092762 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562a21095800 0x562a21099430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:17.812 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.808+0000 7f56e0caa640 1 -- 172.21.15.97:0/4168092762 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562a2108a620 msgr2=0x562a2109db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:29:17.812 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.808+0000 7f56e0caa640 1 --2- 172.21.15.97:0/4168092762 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562a2108a620 0x562a2109db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:17.812 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.808+0000 7f56e0caa640 1 -- 172.21.15.97:0/4168092762 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562a21095800 msgr2=0x562a21099430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:29:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.808+0000 7f56e0caa640 1 --2- 172.21.15.97:0/4168092762 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562a21095800 0x562a21099430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.808+0000 7f56e0caa640 1 -- 172.21.15.97:0/4168092762 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a21099c30 con 0x562a2108af40 2024-04-23T10:29:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.808+0000 7f56e1cac640 1 --2- 172.21.15.97:0/4168092762 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562a21095800 0x562a21099430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:29:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.808+0000 7f56e0caa640 1 --2- 172.21.15.97:0/4168092762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562a2108af40 0x562a21098ef0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f56cc0eee70 tx=0x7f56cc0c15f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fd7cf41437121cf server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:29:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.808+0000 7f56c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:29:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.808+0000 7f56c4ff9640 1 -- 172.21.15.97:0/4168092762 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56cc049040 con 0x562a2108af40 2024-04-23T10:29:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.808+0000 7f56c4ff9640 1 -- 172.21.15.97:0/4168092762 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f56cc0b9730 con 0x562a2108af40 2024-04-23T10:29:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.808+0000 7f56e6422740 1 -- 172.21.15.97:0/4168092762 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56cc0c5d30 con 0x562a2108af40 2024-04-23T10:29:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.808+0000 7f56c4ff9640 1 -- 172.21.15.97:0/4168092762 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f56cc0c1270 con 0x562a2108af40 2024-04-23T10:29:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.809+0000 7f56e6422740 10 client.16059 fetch_fsmap learned FSMap version 26 2024-04-23T10:29:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.809+0000 7f56e6422740 1 -- 172.21.15.97:0/4168092762 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562a2110be30 con 0x562a2108af40 2024-04-23T10:29:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.809+0000 7f56c4ff9640 1 -- 172.21.15.97:0/4168092762 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56cc0c5d30 con 0x562a2108af40 2024-04-23T10:29:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.809+0000 7f56c4ff9640 1 -- 172.21.15.97:0/4168092762 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f56cc0f3110 con 0x562a2108af40 2024-04-23T10:29:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.809+0000 7f56e6422740 10 client.16059 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:29:17.813 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.809+0000 7f56e6422740 10 client.16059 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:29:17.814 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.809+0000 7f56e6422740 20 client.16059 populate_metadata read hostname 'smithi097' 2024-04-23T10:29:17.814 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.809+0000 7f56e6422740 1 --2- 172.21.15.97:0/4168092762 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x562a2110c560 0x562a2110e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:17.814 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.809+0000 7f56e6422740 4 client.16059 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:29:17.814 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.809+0000 7f56e6422740 1 -- 172.21.15.97:0/4168092762 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562a20ff7200 con 0x562a2110c560 2024-04-23T10:29:17.814 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.809+0000 7f56e14ab640 1 --2- 172.21.15.97:0/4168092762 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x562a2110c560 0x562a2110e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:17.814 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.810+0000 7f56e14ab640 1 --2- 172.21.15.97:0/4168092762 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x562a2110c560 0x562a2110e980 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3463701007174da9 server_cookie=8cc6147f8dc1874c in_seq=0 out_seq=0 2024-04-23T10:29:17.814 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.810+0000 7f56c4ff9640 10 client.16059 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:29:17.817 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.818+0000 7f56c4ff9640 1 -- 172.21.15.97:0/4168092762 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562a20ff7200 con 0x562a2110c560 2024-04-23T10:29:17.817 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.818+0000 7f56c4ff9640 10 client.16059 handle_command_reply: tid=0 2024-04-23T10:29:17.817 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.818+0000 7f56e6422740 1 client.16059 shutdown 2024-04-23T10:29:17.818 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.818+0000 7f56e6422740 1 -- 172.21.15.97:0/4168092762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562a2108af40 msgr2=0x562a21098ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:17.818 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.818+0000 7f56e6422740 1 --2- 172.21.15.97:0/4168092762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562a2108af40 0x562a21098ef0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f56cc0eee70 tx=0x7f56cc0c15f0 comp rx=0 tx=0).stop 2024-04-23T10:29:17.818 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.818+0000 7f56e6422740 1 -- 172.21.15.97:0/4168092762 shutdown_connections 2024-04-23T10:29:17.818 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.818+0000 7f56e6422740 1 --2- 172.21.15.97:0/4168092762 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x562a2110c560 0x562a2110e980 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:17.818 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.818+0000 7f56e6422740 1 --2- 172.21.15.97:0/4168092762 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562a21095800 0x562a21099430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:17.818 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.818+0000 7f56e6422740 1 --2- 172.21.15.97:0/4168092762 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562a2108af40 0x562a21098ef0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:17.818 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.818+0000 7f56e6422740 1 --2- 172.21.15.97:0/4168092762 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562a2108a620 0x562a2109db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:17.818 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.818+0000 7f56c4ff9640 0 client.16059 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:29:17.818 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.818+0000 7f56e6422740 1 -- 172.21.15.97:0/4168092762 >> 172.21.15.97:0/4168092762 conn(0x562a21086b30 msgr2=0x562a21085cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:29:17.818 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.818+0000 7f56e6422740 1 -- 172.21.15.97:0/4168092762 shutdown_connections 2024-04-23T10:29:17.818 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.818+0000 7f56e6422740 1 -- 172.21.15.97:0/4168092762 wait complete. 2024-04-23T10:29:17.818 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.818+0000 7f56e6422740 20 client.16059 trim_cache size 0 max 16384 2024-04-23T10:29:17.819 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:29:17.819 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:29:17.819 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:29:17.819 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:29:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.819+0000 7f56e370f640 1 -- 172.21.15.97:0/2559584046 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f56c80a3e40 msgr2=0x7f56c80a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.819+0000 7f56e370f640 1 --2- 172.21.15.97:0/2559584046 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f56c80a3e40 0x7f56c80a6300 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f56d4005d50 tx=0x7f56d4015000 comp rx=0 tx=0).stop 2024-04-23T10:29:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.819+0000 7f56e370f640 1 -- 172.21.15.97:0/2559584046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56dc074b70 msgr2=0x7f56dc1a0ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.819+0000 7f56e370f640 1 --2- 172.21.15.97:0/2559584046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56dc074b70 0x7f56dc1a0ad0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f56cc006010 tx=0x7f56cc0042f0 comp rx=0 tx=0).stop 2024-04-23T10:29:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.819+0000 7f56e370f640 1 -- 172.21.15.97:0/2559584046 shutdown_connections 2024-04-23T10:29:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.819+0000 7f56e370f640 1 --2- 172.21.15.97:0/2559584046 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f56c80a3e40 0x7f56c80a6300 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.819+0000 7f56e370f640 1 --2- 172.21.15.97:0/2559584046 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56dc10e840 0x7f56dc19a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.819+0000 7f56e370f640 1 --2- 172.21.15.97:0/2559584046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56dc074b70 0x7f56dc1a0ad0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.819+0000 7f56e370f640 1 --2- 172.21.15.97:0/2559584046 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f56dc074250 0x7f56dc1a0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.819+0000 7f56e370f640 1 -- 172.21.15.97:0/2559584046 >> 172.21.15.97:0/2559584046 conn(0x7f56dc0793e0 msgr2=0x7f56dc0fee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:29:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.819+0000 7f56e370f640 1 -- 172.21.15.97:0/2559584046 shutdown_connections 2024-04-23T10:29:17.821 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:17.819+0000 7f56e370f640 1 -- 172.21.15.97:0/2559584046 wait complete. 2024-04-23T10:29:18.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:29:18 smithi196 ceph-mon[32919]: pgmap v96: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 3.2 KiB/s wr, 14 op/s 2024-04-23T10:29:18.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:29:18 smithi097 ceph-mon[29197]: pgmap v96: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 3.2 KiB/s wr, 14 op/s 2024-04-23T10:29:18.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:29:18 smithi183 ceph-mon[32317]: pgmap v96: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 3.2 KiB/s wr, 14 op/s 2024-04-23T10:29:19.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:29:19 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:29:19.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:29:19 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:29:19.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:29:19 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:29:19.704 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T10:29:19.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:19.807+0000 7f1b872a7640 1 -- 172.21.15.97:0/4247817982 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b8010c580 msgr2=0x7f1b8010e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:19.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:19.807+0000 7f1b872a7640 1 --2- 172.21.15.97:0/4247817982 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b8010c580 0x7f1b8010e9b0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f1b78009ca0 tx=0x7f1b7802a950 comp rx=0 tx=0).stop 2024-04-23T10:29:19.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:19.807+0000 7f1b872a7640 1 -- 172.21.15.97:0/4247817982 shutdown_connections 2024-04-23T10:29:19.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:19.807+0000 7f1b872a7640 1 --2- 172.21.15.97:0/4247817982 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b8010c580 0x7f1b8010e9b0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:19.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:19.807+0000 7f1b872a7640 1 --2- 172.21.15.97:0/4247817982 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b80103ad0 0x7f1b8010bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:19.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:19.807+0000 7f1b872a7640 1 --2- 172.21.15.97:0/4247817982 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b801031b0 0x7f1b80103590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:19.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:19.807+0000 7f1b872a7640 1 -- 172.21.15.97:0/4247817982 >> 172.21.15.97:0/4247817982 conn(0x7f1b800771d0 msgr2=0x7f1b800775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:29:19.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:19.807+0000 7f1b872a7640 1 -- 172.21.15.97:0/4247817982 shutdown_connections 2024-04-23T10:29:19.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:19.807+0000 7f1b872a7640 1 -- 172.21.15.97:0/4247817982 wait complete. 2024-04-23T10:29:19.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:19.807+0000 7f1b872a7640 1 Processor -- start 2024-04-23T10:29:19.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:19.807+0000 7f1b872a7640 1 -- start start 2024-04-23T10:29:19.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:19.808+0000 7f1b872a7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b801031b0 0x7f1b80198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:19.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:19.808+0000 7f1b872a7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b80103ad0 0x7f1b80198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:19.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:19.808+0000 7f1b872a7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b8010c580 0x7f1b8019cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:19.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:19.808+0000 7f1b872a7640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1b801121a0 con 0x7f1b80103ad0 2024-04-23T10:29:19.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:19.808+0000 7f1b872a7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1b80112020 con 0x7f1b801031b0 2024-04-23T10:29:19.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:19.808+0000 7f1b872a7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1b80112320 con 0x7f1b8010c580 2024-04-23T10:29:19.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:19.808+0000 7f1b84842640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b80103ad0 0x7f1b80198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:19.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:19.808+0000 7f1b84842640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b80103ad0 0x7f1b80198c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40128/0 (socket says 172.21.15.97:40128) 2024-04-23T10:29:19.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:19.808+0000 7f1b84842640 1 -- 172.21.15.97:0/3967909817 learned_addr learned my addr 172.21.15.97:0/3967909817 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:29:19.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:19.808+0000 7f1b85043640 1 --2- 172.21.15.97:0/3967909817 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b801031b0 0x7f1b80198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:19.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:19.808+0000 7f1b85844640 1 --2- 172.21.15.97:0/3967909817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b8010c580 0x7f1b8019cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:19.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:19.808+0000 7f1b84842640 1 -- 172.21.15.97:0/3967909817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b8010c580 msgr2=0x7f1b8019cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:19.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:19.808+0000 7f1b84842640 1 --2- 172.21.15.97:0/3967909817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b8010c580 0x7f1b8019cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:19.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:19.808+0000 7f1b84842640 1 -- 172.21.15.97:0/3967909817 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b801031b0 msgr2=0x7f1b80198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:19.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:19.808+0000 7f1b84842640 1 --2- 172.21.15.97:0/3967909817 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b801031b0 0x7f1b80198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:19.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:19.808+0000 7f1b84842640 1 -- 172.21.15.97:0/3967909817 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b8019d6f0 con 0x7f1b80103ad0 2024-04-23T10:29:19.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:19.808+0000 7f1b84842640 1 --2- 172.21.15.97:0/3967909817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b80103ad0 0x7f1b80198c40 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f1b7000d770 tx=0x7f1b7000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:29:19.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:19.808+0000 7f1b767fc640 1 -- 172.21.15.97:0/3967909817 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b70017070 con 0x7f1b80103ad0 2024-04-23T10:29:19.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:19.808+0000 7f1b767fc640 1 -- 172.21.15.97:0/3967909817 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b700054d0 con 0x7f1b80103ad0 2024-04-23T10:29:19.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:19.808+0000 7f1b767fc640 1 -- 172.21.15.97:0/3967909817 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b70004c30 con 0x7f1b80103ad0 2024-04-23T10:29:19.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:19.808+0000 7f1b872a7640 1 -- 172.21.15.97:0/3967909817 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b8019d9e0 con 0x7f1b80103ad0 2024-04-23T10:29:19.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:19.808+0000 7f1b872a7640 1 -- 172.21.15.97:0/3967909817 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b801a5270 con 0x7f1b80103ad0 2024-04-23T10:29:19.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:19.809+0000 7f1b872a7640 1 -- 172.21.15.97:0/3967909817 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b800691c0 con 0x7f1b80103ad0 2024-04-23T10:29:19.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:19.810+0000 7f1b767fc640 1 -- 172.21.15.97:0/3967909817 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1b70004dd0 con 0x7f1b80103ad0 2024-04-23T10:29:19.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:19.811+0000 7f1b767fc640 1 --2- 172.21.15.97:0/3967909817 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f1b6c0a3d20 0x7f1b6c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:19.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:19.811+0000 7f1b85043640 1 --2- 172.21.15.97:0/3967909817 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f1b6c0a3d20 0x7f1b6c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:19.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:19.811+0000 7f1b767fc640 1 -- 172.21.15.97:0/3967909817 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f1b700d05d0 con 0x7f1b80103ad0 2024-04-23T10:29:19.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:19.812+0000 7f1b85043640 1 --2- 172.21.15.97:0/3967909817 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f1b6c0a3d20 0x7f1b6c0a61e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c005d00 tx=0x7f1b7c00b410 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:29:19.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:19.815+0000 7f1b767fc640 1 -- 172.21.15.97:0/3967909817 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1b7009c610 con 0x7f1b80103ad0 2024-04-23T10:29:20.027 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.028+0000 7f1b872a7640 1 -- 172.21.15.97:0/3967909817 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1b80199a50 con 0x7f1b80103ad0 2024-04-23T10:29:20.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.029+0000 7f1b767fc640 1 -- 172.21.15.97:0/3967909817 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11931 (secure 0 0 0) 0x7f1b80199a50 con 0x7f1b80103ad0 2024-04-23T10:29:20.028 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:29:20.028 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3946878762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3946878762},{"type":"v1","addr":"172.21.15.97:6835","nonce":3946878762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/209800861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":209800861},{"type":"v1","addr":"172.21.15.97:6837","nonce":209800861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T10:24:06.910134+0000","modified":"2024-04-23T10:26:09.820985+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6839/418013575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":418013575},{"type":"v1","addr":"172.21.15.97:6839","nonce":418013575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6841/747242680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":747242680},{"type":"v1","addr":"172.21.15.97:6841","nonce":747242680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6833/4292847121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4292847121},{"type":"v1","addr":"172.21.15.183:6833","nonce":4292847121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6833/3885816565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3885816565},{"type":"v1","addr":"172.21.15.196:6833","nonce":3885816565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1897093105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1897093105},{"type":"v1","addr":"172.21.15.183:6835","nonce":1897093105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1171477563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1171477563},{"type":"v1","addr":"172.21.15.196:6835","nonce":1171477563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1898341911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1898341911},{"type":"v1","addr":"172.21.15.183:6837","nonce":1898341911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.196:6837/3094946502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3094946502},{"type":"v1","addr":"172.21.15.196:6837","nonce":3094946502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.196:6839/83843556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":83843556},{"type":"v1","addr":"172.21.15.196:6839","nonce":83843556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6839/1763841130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1763841130},{"type":"v1","addr":"172.21.15.183:6839","nonce":1763841130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24487,24403,24398,24469,24488]},"id":1}]} 2024-04-23T10:29:20.029 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 26 2024-04-23T10:29:20.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.031+0000 7f1b872a7640 1 -- 172.21.15.97:0/3967909817 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f1b6c0a3d20 msgr2=0x7f1b6c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:20.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.031+0000 7f1b872a7640 1 --2- 172.21.15.97:0/3967909817 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f1b6c0a3d20 0x7f1b6c0a61e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c005d00 tx=0x7f1b7c00b410 comp rx=0 tx=0).stop 2024-04-23T10:29:20.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.031+0000 7f1b872a7640 1 -- 172.21.15.97:0/3967909817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b80103ad0 msgr2=0x7f1b80198c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:20.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.031+0000 7f1b872a7640 1 --2- 172.21.15.97:0/3967909817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b80103ad0 0x7f1b80198c40 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f1b7000d770 tx=0x7f1b7000dc30 comp rx=0 tx=0).stop 2024-04-23T10:29:20.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.031+0000 7f1b872a7640 1 -- 172.21.15.97:0/3967909817 shutdown_connections 2024-04-23T10:29:20.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.031+0000 7f1b872a7640 1 --2- 172.21.15.97:0/3967909817 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f1b6c0a3d20 0x7f1b6c0a61e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.031+0000 7f1b872a7640 1 --2- 172.21.15.97:0/3967909817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b8010c580 0x7f1b8019cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.031+0000 7f1b872a7640 1 --2- 172.21.15.97:0/3967909817 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b80103ad0 0x7f1b80198c40 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.031+0000 7f1b872a7640 1 --2- 172.21.15.97:0/3967909817 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b801031b0 0x7f1b80198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.031+0000 7f1b872a7640 1 -- 172.21.15.97:0/3967909817 >> 172.21.15.97:0/3967909817 conn(0x7f1b800771d0 msgr2=0x7f1b800fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:29:20.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.031+0000 7f1b872a7640 1 -- 172.21.15.97:0/3967909817 shutdown_connections 2024-04-23T10:29:20.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.031+0000 7f1b872a7640 1 -- 172.21.15.97:0/3967909817 wait complete. 2024-04-23T10:29:20.041 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --set-id 3f96bb3b --release --await' 2024-04-23T10:29:20.041 DEBUG:teuthology.orchestra.run.smithi097:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24488 quiesce db --set-id 3f96bb3b --release --await 2024-04-23T10:29:20.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.148+0000 7f61df1f7640 1 -- 172.21.15.97:0/1947439315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61d8101470 msgr2=0x7f61d810d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:20.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.148+0000 7f61df1f7640 1 --2- 172.21.15.97:0/1947439315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61d8101470 0x7f61d810d430 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f61d0009ca0 tx=0x7f61d002a8c0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.148+0000 7f61df1f7640 1 -- 172.21.15.97:0/1947439315 shutdown_connections 2024-04-23T10:29:20.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.148+0000 7f61df1f7640 1 --2- 172.21.15.97:0/1947439315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61d8101470 0x7f61d810d430 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.148+0000 7f61df1f7640 1 --2- 172.21.15.97:0/1947439315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61d8100ab0 0x7f61d8100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.148+0000 7f61df1f7640 1 --2- 172.21.15.97:0/1947439315 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f61d8106670 0x7f61d8106a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.149 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.148+0000 7f61df1f7640 1 -- 172.21.15.97:0/1947439315 >> 172.21.15.97:0/1947439315 conn(0x7f61d8076150 msgr2=0x7f61d8076560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:29:20.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.148+0000 7f61df1f7640 1 -- 172.21.15.97:0/1947439315 shutdown_connections 2024-04-23T10:29:20.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.148+0000 7f61df1f7640 1 -- 172.21.15.97:0/1947439315 wait complete. 2024-04-23T10:29:20.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.149+0000 7f61df1f7640 1 Processor -- start 2024-04-23T10:29:20.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.149+0000 7f61df1f7640 1 -- start start 2024-04-23T10:29:20.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.149+0000 7f61df1f7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f61d8100ab0 0x7f61d8199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:20.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.149+0000 7f61df1f7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61d8101470 0x7f61d81999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:20.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.149+0000 7f61df1f7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61d8106670 0x7f61d81a4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:20.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.149+0000 7f61df1f7640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f61d8110860 con 0x7f61d8101470 2024-04-23T10:29:20.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.149+0000 7f61df1f7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f61d81106e0 con 0x7f61d8100ab0 2024-04-23T10:29:20.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.149+0000 7f61df1f7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f61d81109e0 con 0x7f61d8106670 2024-04-23T10:29:20.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.149+0000 7f61d7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61d8101470 0x7f61d81999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:20.150 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.149+0000 7f61d7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61d8101470 0x7f61d81999c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40156/0 (socket says 172.21.15.97:40156) 2024-04-23T10:29:20.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.149+0000 7f61d7fff640 1 -- 172.21.15.97:0/1142929177 learned_addr learned my addr 172.21.15.97:0/1142929177 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:29:20.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.149+0000 7f61dd794640 1 --2- 172.21.15.97:0/1142929177 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61d8106670 0x7f61d81a4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:20.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.149+0000 7f61dcf93640 1 --2- 172.21.15.97:0/1142929177 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f61d8100ab0 0x7f61d8199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:20.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.149+0000 7f61d7fff640 1 -- 172.21.15.97:0/1142929177 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61d8106670 msgr2=0x7f61d81a4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:20.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.149+0000 7f61d7fff640 1 --2- 172.21.15.97:0/1142929177 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61d8106670 0x7f61d81a4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.149+0000 7f61d7fff640 1 -- 172.21.15.97:0/1142929177 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f61d8100ab0 msgr2=0x7f61d8199480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:20.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.149+0000 7f61d7fff640 1 --2- 172.21.15.97:0/1142929177 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f61d8100ab0 0x7f61d8199480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.149+0000 7f61d7fff640 1 -- 172.21.15.97:0/1142929177 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61d81a4a40 con 0x7f61d8101470 2024-04-23T10:29:20.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.149+0000 7f61d7fff640 1 --2- 172.21.15.97:0/1142929177 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61d8101470 0x7f61d81999c0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f61c400c920 tx=0x7f61c400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:29:20.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.149+0000 7f61d5ffb640 1 -- 172.21.15.97:0/1142929177 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c4007c70 con 0x7f61d8101470 2024-04-23T10:29:20.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.149+0000 7f61d5ffb640 1 -- 172.21.15.97:0/1142929177 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61c4007e10 con 0x7f61d8101470 2024-04-23T10:29:20.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.150+0000 7f61df1f7640 1 -- 172.21.15.97:0/1142929177 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61d81a4c10 con 0x7f61d8101470 2024-04-23T10:29:20.151 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.150+0000 7f61d5ffb640 1 -- 172.21.15.97:0/1142929177 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c4018820 con 0x7f61d8101470 2024-04-23T10:29:20.152 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.150+0000 7f61df1f7640 1 -- 172.21.15.97:0/1142929177 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61d81a5100 con 0x7f61d8101470 2024-04-23T10:29:20.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.152+0000 7f61d5ffb640 1 -- 172.21.15.97:0/1142929177 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f61c400f450 con 0x7f61d8101470 2024-04-23T10:29:20.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.152+0000 7f61e1fe6740 1 Processor -- start 2024-04-23T10:29:20.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.152+0000 7f61e1fe6740 1 -- start start 2024-04-23T10:29:20.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.152+0000 7f61d5ffb640 1 --2- 172.21.15.97:0/1142929177 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f61c80a3d20 0x7f61c80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:20.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.152+0000 7f61e1fe6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d4624c5490 0x55d4624c5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:20.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.152+0000 7f61e1fe6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4624c5db0 0x55d4624cff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:20.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.152+0000 7f61e1fe6740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d4624d0480 0x55d4624d2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:20.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.152+0000 7f61e1fe6740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d46242a6d0 con 0x55d4624d0480 2024-04-23T10:29:20.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.152+0000 7f61e1fe6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d4623c71a0 con 0x55d4624c5db0 2024-04-23T10:29:20.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.152+0000 7f61e1fe6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d462426690 con 0x55d4624c5490 2024-04-23T10:29:20.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.152+0000 7f61dd794640 1 --2- 172.21.15.97:0/1142929177 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f61c80a3d20 0x7f61c80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:20.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.152+0000 7f61d5ffb640 1 -- 172.21.15.97:0/1142929177 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f61c40d0890 con 0x7f61d8101470 2024-04-23T10:29:20.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.153+0000 7f61dcf93640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d4624c5490 0x55d4624c5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:20.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.153+0000 7f61d7fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4624c5db0 0x55d4624cff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:20.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.153+0000 7f61dcf93640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d4624c5490 0x55d4624c5870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.97:43654/0 (socket says 172.21.15.97:43654) 2024-04-23T10:29:20.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.153+0000 7f61dcf93640 1 -- 172.21.15.97:0/2847421897 learned_addr learned my addr 172.21.15.97:0/2847421897 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:29:20.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.153+0000 7f61d7fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4624c5db0 0x55d4624cff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.97:56522/0 (socket says 172.21.15.97:56522) 2024-04-23T10:29:20.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.153+0000 7f61dd794640 1 --2- 172.21.15.97:0/2847421897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d4624d0480 0x55d4624d2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:20.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.153+0000 7f61d7fff640 1 -- 172.21.15.97:0/2847421897 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d4624c5490 msgr2=0x55d4624c5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:29:20.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.153+0000 7f61d7fff640 1 --2- 172.21.15.97:0/2847421897 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d4624c5490 0x55d4624c5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.153+0000 7f61d7fff640 1 -- 172.21.15.97:0/2847421897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d4624d0480 msgr2=0x55d4624d2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:29:20.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.153+0000 7f61d7fff640 1 --2- 172.21.15.97:0/2847421897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d4624d0480 0x55d4624d2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.153+0000 7f61d7fff640 1 -- 172.21.15.97:0/2847421897 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4623d15b0 con 0x55d4624c5db0 2024-04-23T10:29:20.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.153+0000 7f61dd794640 1 --2- 172.21.15.97:0/1142929177 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f61c80a3d20 0x7f61c80a61e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f61d0005fe0 tx=0x7f61d0031cb0 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:29:20.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.153+0000 7f61dcf93640 1 --2- 172.21.15.97:0/2847421897 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d4624c5490 0x55d4624c5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T10:29:20.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.153+0000 7f61d7fff640 1 --2- 172.21.15.97:0/2847421897 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4624c5db0 0x55d4624cff40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f61c40ce030 tx=0x7f61c409c780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec51ec8f3bbf52e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:29:20.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.153+0000 7f61c2ffd640 1 -- 172.21.15.97:0/2847421897 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c40a54f0 con 0x55d4624c5db0 2024-04-23T10:29:20.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.153+0000 7f61c2ffd640 1 -- 172.21.15.97:0/2847421897 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61c400fa90 con 0x55d4624c5db0 2024-04-23T10:29:20.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.153+0000 7f61c2ffd640 1 -- 172.21.15.97:0/2847421897 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c40870b0 con 0x55d4624c5db0 2024-04-23T10:29:20.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.154+0000 7f61e1fe6740 1 -- 172.21.15.97:0/2847421897 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4624c5db0 msgr2=0x55d4624cff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:20.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.154+0000 7f61e1fe6740 1 --2- 172.21.15.97:0/2847421897 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4624c5db0 0x55d4624cff40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f61c40ce030 tx=0x7f61c409c780 comp rx=0 tx=0).stop 2024-04-23T10:29:20.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.154+0000 7f61e1fe6740 1 -- 172.21.15.97:0/2847421897 shutdown_connections 2024-04-23T10:29:20.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.154+0000 7f61e1fe6740 1 --2- 172.21.15.97:0/2847421897 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d4624d0480 0x55d4624d2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.154+0000 7f61e1fe6740 1 --2- 172.21.15.97:0/2847421897 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4624c5db0 0x55d4624cff40 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.154+0000 7f61e1fe6740 1 --2- 172.21.15.97:0/2847421897 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d4624c5490 0x55d4624c5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.161 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.154+0000 7f61e1fe6740 1 -- 172.21.15.97:0/2847421897 >> 172.21.15.97:0/2847421897 conn(0x55d4624c17b0 msgr2=0x55d4624c1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:29:20.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.154+0000 7f61e1fe6740 1 -- 172.21.15.97:0/2847421897 shutdown_connections 2024-04-23T10:29:20.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.154+0000 7f61e1fe6740 1 -- 172.21.15.97:0/2847421897 wait complete. 2024-04-23T10:29:20.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.154+0000 7f61e1fe6740 1 -- start start 2024-04-23T10:29:20.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.154+0000 7f61e1fe6740 1 Processor -- start 2024-04-23T10:29:20.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.155+0000 7f61e1fe6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4624c5490 0x55d4624cfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:20.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.155+0000 7f61e1fe6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d4624c5db0 0x55d4624cd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:20.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.155+0000 7f61e1fe6740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d4624d0480 0x55d4624cd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:20.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.155+0000 7f61e1fe6740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d4624d3310 con 0x55d4624d0480 2024-04-23T10:29:20.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.155+0000 7f61e1fe6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d4623c4b20 con 0x55d4624c5490 2024-04-23T10:29:20.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.155+0000 7f61e1fe6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d4624d3490 con 0x55d4624c5db0 2024-04-23T10:29:20.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.155+0000 7f61e1fe6740 10 client.0 _ll_register_callbacks cb 0x55d4623c4530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:29:20.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.155+0000 7f61dd794640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d4624d0480 0x55d4624cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:20.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.155+0000 7f61dd794640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d4624d0480 0x55d4624cd900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40178/0 (socket says 172.21.15.97:40178) 2024-04-23T10:29:20.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.155+0000 7f61dd794640 1 -- 172.21.15.97:0/1805204501 learned_addr learned my addr 172.21.15.97:0/1805204501 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:29:20.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.155+0000 7f61dcf93640 1 --2- 172.21.15.97:0/1805204501 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4624c5490 0x55d4624cfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:20.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.155+0000 7f61d7fff640 1 --2- 172.21.15.97:0/1805204501 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d4624c5db0 0x55d4624cd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:20.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.155+0000 7f61dd794640 1 -- 172.21.15.97:0/1805204501 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d4624c5db0 msgr2=0x55d4624cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:29:20.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.155+0000 7f61dd794640 1 --2- 172.21.15.97:0/1805204501 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d4624c5db0 0x55d4624cd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.155+0000 7f61dd794640 1 -- 172.21.15.97:0/1805204501 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4624c5490 msgr2=0x55d4624cfed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:29:20.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.155+0000 7f61dd794640 1 --2- 172.21.15.97:0/1805204501 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4624c5490 0x55d4624cfed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.155+0000 7f61dd794640 1 -- 172.21.15.97:0/1805204501 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d46213c920 con 0x55d4624d0480 2024-04-23T10:29:20.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.155+0000 7f61d7fff640 1 --2- 172.21.15.97:0/1805204501 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d4624c5db0 0x55d4624cd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:29:20.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.155+0000 7f61dd794640 1 --2- 172.21.15.97:0/1805204501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d4624d0480 0x55d4624cd900 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f61d0035000 tx=0x7f61d002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc3e9cc574552fdf server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:29:20.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.155+0000 7f61b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:29:20.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.155+0000 7f61b7fff640 1 -- 172.21.15.97:0/1805204501 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d0004740 con 0x55d4624d0480 2024-04-23T10:29:20.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.155+0000 7f61b7fff640 1 -- 172.21.15.97:0/1805204501 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f61d002f660 con 0x55d4624d0480 2024-04-23T10:29:20.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.155+0000 7f61e1fe6740 1 -- 172.21.15.97:0/1805204501 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61c400fa90 con 0x55d4624d0480 2024-04-23T10:29:20.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.156+0000 7f61b7fff640 1 -- 172.21.15.97:0/1805204501 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f61d002f860 con 0x55d4624d0480 2024-04-23T10:29:20.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.156+0000 7f61e1fe6740 10 client.16077 fetch_fsmap learned FSMap version 26 2024-04-23T10:29:20.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.156+0000 7f61e1fe6740 1 -- 172.21.15.97:0/1805204501 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d462208d30 con 0x55d4624d0480 2024-04-23T10:29:20.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.156+0000 7f61b7fff640 1 -- 172.21.15.97:0/1805204501 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61c400fa90 con 0x55d4624d0480 2024-04-23T10:29:20.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.156+0000 7f61b7fff640 1 -- 172.21.15.97:0/1805204501 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f61d0030920 con 0x55d4624d0480 2024-04-23T10:29:20.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.156+0000 7f61e1fe6740 10 client.16077 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:29:20.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.156+0000 7f61e1fe6740 10 client.16077 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:29:20.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.156+0000 7f61e1fe6740 20 client.16077 populate_metadata read hostname 'smithi097' 2024-04-23T10:29:20.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.156+0000 7f61e1fe6740 1 --2- 172.21.15.97:0/1805204501 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55d4624e0580 0x55d4624e2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:20.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.156+0000 7f61e1fe6740 4 client.16077 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:29:20.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.156+0000 7f61e1fe6740 1 -- 172.21.15.97:0/1805204501 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d4623e8500 con 0x55d4624e0580 2024-04-23T10:29:20.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.156+0000 7f61dcf93640 1 --2- 172.21.15.97:0/1805204501 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55d4624e0580 0x55d4624e2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:20.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.157+0000 7f61b7fff640 10 client.16077 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:29:20.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.157+0000 7f61dcf93640 1 --2- 172.21.15.97:0/1805204501 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55d4624e0580 0x55d4624e2970 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48f44cc219e6c782 server_cookie=5db2f7b52a7ee1ad in_seq=0 out_seq=0 2024-04-23T10:29:20.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.157+0000 7f61b7fff640 1 -- 172.21.15.97:0/1805204501 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d4623e8500 con 0x55d4624e0580 2024-04-23T10:29:20.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.157+0000 7f61b7fff640 10 client.16077 handle_command_reply: tid=0 2024-04-23T10:29:20.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.157+0000 7f61e1fe6740 1 client.16077 shutdown 2024-04-23T10:29:20.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.157+0000 7f61e1fe6740 1 -- 172.21.15.97:0/1805204501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d4624d0480 msgr2=0x55d4624cd900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:20.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.157+0000 7f61e1fe6740 1 --2- 172.21.15.97:0/1805204501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d4624d0480 0x55d4624cd900 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f61d0035000 tx=0x7f61d002a8c0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.158+0000 7f61e1fe6740 1 -- 172.21.15.97:0/1805204501 shutdown_connections 2024-04-23T10:29:20.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.158+0000 7f61e1fe6740 1 --2- 172.21.15.97:0/1805204501 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55d4624e0580 0x55d4624e2970 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.158+0000 7f61e1fe6740 1 --2- 172.21.15.97:0/1805204501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d4624d0480 0x55d4624cd900 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.158+0000 7f61e1fe6740 1 --2- 172.21.15.97:0/1805204501 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d4624c5db0 0x55d4624cd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.158+0000 7f61b7fff640 0 client.16077 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:29:20.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.158+0000 7f61e1fe6740 1 --2- 172.21.15.97:0/1805204501 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4624c5490 0x55d4624cfed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.158+0000 7f61e1fe6740 1 -- 172.21.15.97:0/1805204501 >> 172.21.15.97:0/1805204501 conn(0x55d4624c17b0 msgr2=0x55d4624c75b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:29:20.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.158+0000 7f61e1fe6740 1 -- 172.21.15.97:0/1805204501 shutdown_connections 2024-04-23T10:29:20.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.158+0000 7f61e1fe6740 1 -- 172.21.15.97:0/1805204501 wait complete. 2024-04-23T10:29:20.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.158+0000 7f61e1fe6740 20 client.16077 trim_cache size 0 max 16384 2024-04-23T10:29:20.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.174+0000 7f61e1fe6740 1 Processor -- start 2024-04-23T10:29:20.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.174+0000 7f61e1fe6740 1 -- start start 2024-04-23T10:29:20.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.175+0000 7f61e1fe6740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d4624c5490 0x55d4624d8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:20.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.175+0000 7f61e1fe6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4624c5db0 0x55d4624cfa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:20.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.175+0000 7f61e1fe6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d4624d0480 0x55d4624cff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:20.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.175+0000 7f61e1fe6740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f61d81ad970 con 0x55d4624c5490 2024-04-23T10:29:20.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.175+0000 7f61e1fe6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f61d81af4b0 con 0x55d4624c5db0 2024-04-23T10:29:20.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.175+0000 7f61e1fe6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f61d81ae040 con 0x55d4624d0480 2024-04-23T10:29:20.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.175+0000 7f61dcf93640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d4624c5490 0x55d4624d8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:20.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.175+0000 7f61dcf93640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d4624c5490 0x55d4624d8ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40182/0 (socket says 172.21.15.97:40182) 2024-04-23T10:29:20.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.175+0000 7f61dcf93640 1 -- 172.21.15.97:0/155471921 learned_addr learned my addr 172.21.15.97:0/155471921 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:29:20.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.175+0000 7f61dcf93640 1 -- 172.21.15.97:0/155471921 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d4624d0480 msgr2=0x55d4624cff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:29:20.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.175+0000 7f61dcf93640 1 --2- 172.21.15.97:0/155471921 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d4624d0480 0x55d4624cff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.175+0000 7f61dcf93640 1 -- 172.21.15.97:0/155471921 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4624c5db0 msgr2=0x55d4624cfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:29:20.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.175+0000 7f61dcf93640 1 --2- 172.21.15.97:0/155471921 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4624c5db0 0x55d4624cfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.175+0000 7f61dcf93640 1 -- 172.21.15.97:0/155471921 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4624c5be0 con 0x55d4624c5490 2024-04-23T10:29:20.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.175+0000 7f61dcf93640 1 --2- 172.21.15.97:0/155471921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d4624c5490 0x55d4624d8ac0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f61cc00cf70 tx=0x7f61cc00c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=906ec1c3ed2dae5a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:29:20.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.175+0000 7f61c27fc640 1 -- 172.21.15.97:0/155471921 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61cc020070 con 0x55d4624c5490 2024-04-23T10:29:20.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.175+0000 7f61c27fc640 1 -- 172.21.15.97:0/155471921 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61cc005020 con 0x55d4624c5490 2024-04-23T10:29:20.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.175+0000 7f61c27fc640 1 -- 172.21.15.97:0/155471921 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61cc004b50 con 0x55d4624c5490 2024-04-23T10:29:20.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.176+0000 7f61e1fe6740 1 -- 172.21.15.97:0/155471921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d4624c5490 msgr2=0x55d4624d8ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:20.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.176+0000 7f61e1fe6740 1 --2- 172.21.15.97:0/155471921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d4624c5490 0x55d4624d8ac0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f61cc00cf70 tx=0x7f61cc00c480 comp rx=0 tx=0).stop 2024-04-23T10:29:20.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.176+0000 7f61e1fe6740 1 -- 172.21.15.97:0/155471921 shutdown_connections 2024-04-23T10:29:20.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.176+0000 7f61e1fe6740 1 --2- 172.21.15.97:0/155471921 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d4624d0480 0x55d4624cff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.176+0000 7f61e1fe6740 1 --2- 172.21.15.97:0/155471921 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4624c5db0 0x55d4624cfa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.176+0000 7f61e1fe6740 1 --2- 172.21.15.97:0/155471921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d4624c5490 0x55d4624d8ac0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.176+0000 7f61e1fe6740 1 -- 172.21.15.97:0/155471921 >> 172.21.15.97:0/155471921 conn(0x55d4624c17b0 msgr2=0x55d4624c7500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:29:20.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.176+0000 7f61e1fe6740 1 -- 172.21.15.97:0/155471921 shutdown_connections 2024-04-23T10:29:20.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.176+0000 7f61e1fe6740 1 -- 172.21.15.97:0/155471921 wait complete. 2024-04-23T10:29:20.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.176+0000 7f61e1fe6740 1 -- start start 2024-04-23T10:29:20.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.176+0000 7f61e1fe6740 1 Processor -- start 2024-04-23T10:29:20.180 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.177+0000 7f61e1fe6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4624c5490 0x55d4624d1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:20.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.177+0000 7f61e1fe6740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d4624c5db0 0x55d4624d21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:20.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.177+0000 7f61e1fe6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d4624d0480 0x55d4624ca4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:20.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.177+0000 7f61e1fe6740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f61d81aede0 con 0x55d4624c5db0 2024-04-23T10:29:20.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.177+0000 7f61e1fe6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f61d81ae710 con 0x55d4624c5490 2024-04-23T10:29:20.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.177+0000 7f61e1fe6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f61d81ab950 con 0x55d4624d0480 2024-04-23T10:29:20.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.177+0000 7f61e1fe6740 10 client.0 _ll_register_callbacks cb 0x7f61d0031760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:29:20.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.177+0000 7f61d7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d4624c5db0 0x55d4624d21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:20.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.177+0000 7f61d7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d4624c5db0 0x55d4624d21a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40186/0 (socket says 172.21.15.97:40186) 2024-04-23T10:29:20.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.177+0000 7f61d7fff640 1 -- 172.21.15.97:0/2965699070 learned_addr learned my addr 172.21.15.97:0/2965699070 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:29:20.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.177+0000 7f61dd794640 1 --2- 172.21.15.97:0/2965699070 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d4624d0480 0x55d4624ca4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:20.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.177+0000 7f61dcf93640 1 --2- 172.21.15.97:0/2965699070 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4624c5490 0x55d4624d1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:20.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.177+0000 7f61d7fff640 1 -- 172.21.15.97:0/2965699070 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d4624d0480 msgr2=0x55d4624ca4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:29:20.181 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.177+0000 7f61d7fff640 1 --2- 172.21.15.97:0/2965699070 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d4624d0480 0x55d4624ca4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.177+0000 7f61d7fff640 1 -- 172.21.15.97:0/2965699070 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4624c5490 msgr2=0x55d4624d1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:29:20.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.177+0000 7f61d7fff640 1 --2- 172.21.15.97:0/2965699070 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4624c5490 0x55d4624d1c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.177+0000 7f61d7fff640 1 -- 172.21.15.97:0/2965699070 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4624cacd0 con 0x55d4624c5db0 2024-04-23T10:29:20.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.177+0000 7f61d7fff640 1 --2- 172.21.15.97:0/2965699070 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d4624c5db0 0x55d4624d21a0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f61c4098ce0 tx=0x7f61c409c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4653f80a4c341fa0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:29:20.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.177+0000 7f61b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:29:20.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.177+0000 7f61b7fff640 1 -- 172.21.15.97:0/2965699070 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c4098ef0 con 0x55d4624c5db0 2024-04-23T10:29:20.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.177+0000 7f61b7fff640 1 -- 172.21.15.97:0/2965699070 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f61c40870b0 con 0x55d4624c5db0 2024-04-23T10:29:20.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.177+0000 7f61e1fe6740 1 -- 172.21.15.97:0/2965699070 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61cc005020 con 0x55d4624c5db0 2024-04-23T10:29:20.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.177+0000 7f61b7fff640 1 -- 172.21.15.97:0/2965699070 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f61c40ba070 con 0x55d4624c5db0 2024-04-23T10:29:20.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.177+0000 7f61e1fe6740 10 client.16083 fetch_fsmap learned FSMap version 26 2024-04-23T10:29:20.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.177+0000 7f61e1fe6740 1 -- 172.21.15.97:0/2965699070 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d4624d1440 con 0x55d4624c5db0 2024-04-23T10:29:20.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.178+0000 7f61b7fff640 1 -- 172.21.15.97:0/2965699070 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61cc005020 con 0x55d4624c5db0 2024-04-23T10:29:20.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.178+0000 7f61b7fff640 1 -- 172.21.15.97:0/2965699070 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f61c40a5060 con 0x55d4624c5db0 2024-04-23T10:29:20.182 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.178+0000 7f61e1fe6740 10 client.16083 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:29:20.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.178+0000 7f61e1fe6740 10 client.16083 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:29:20.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.178+0000 7f61e1fe6740 20 client.16083 populate_metadata read hostname 'smithi097' 2024-04-23T10:29:20.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.178+0000 7f61e1fe6740 1 --2- 172.21.15.97:0/2965699070 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55d46254baf0 0x55d46254dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:20.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.178+0000 7f61e1fe6740 4 client.16083 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "3f96bb3b", "release": true, "await": true}] 2024-04-23T10:29:20.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.178+0000 7f61e1fe6740 1 -- 172.21.15.97:0/2965699070 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "quiesce db", "set_id": "3f96bb3b", "release": true, "await": true}) -- 0x55d4624380f0 con 0x55d46254baf0 2024-04-23T10:29:20.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.178+0000 7f61dcf93640 1 --2- 172.21.15.97:0/2965699070 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55d46254baf0 0x55d46254dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:20.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.178+0000 7f61dcf93640 1 --2- 172.21.15.97:0/2965699070 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55d46254baf0 0x55d46254dee0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ad31ed1c9cdc277 server_cookie=561c72b872eef1c2 in_seq=0 out_seq=0 2024-04-23T10:29:20.183 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.178+0000 7f61b7fff640 10 client.16083 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:29:20.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.182+0000 7f61b7fff640 1 -- 172.21.15.97:0/2965699070 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55d4624380f0 con 0x55d46254baf0 2024-04-23T10:29:20.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.182+0000 7f61b7fff640 10 client.16083 handle_command_reply: tid=0 2024-04-23T10:29:20.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.182+0000 7f61e1fe6740 1 client.16083 shutdown 2024-04-23T10:29:20.184 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.182+0000 7f61e1fe6740 1 -- 172.21.15.97:0/2965699070 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d4624c5db0 msgr2=0x55d4624d21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:20.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.182+0000 7f61e1fe6740 1 --2- 172.21.15.97:0/2965699070 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d4624c5db0 0x55d4624d21a0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f61c4098ce0 tx=0x7f61c409c9c0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.182+0000 7f61e1fe6740 1 -- 172.21.15.97:0/2965699070 shutdown_connections 2024-04-23T10:29:20.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.182+0000 7f61e1fe6740 1 --2- 172.21.15.97:0/2965699070 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55d46254baf0 0x55d46254dee0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.182+0000 7f61e1fe6740 1 --2- 172.21.15.97:0/2965699070 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d4624d0480 0x55d4624ca4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.182+0000 7f61b7fff640 0 client.16083 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:29:20.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.182+0000 7f61e1fe6740 1 --2- 172.21.15.97:0/2965699070 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d4624c5db0 0x55d4624d21a0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.182+0000 7f61e1fe6740 1 --2- 172.21.15.97:0/2965699070 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4624c5490 0x55d4624d1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.182+0000 7f61e1fe6740 1 -- 172.21.15.97:0/2965699070 >> 172.21.15.97:0/2965699070 conn(0x55d4624c17b0 msgr2=0x55d4624c6be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:29:20.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.182+0000 7f61e1fe6740 1 -- 172.21.15.97:0/2965699070 shutdown_connections 2024-04-23T10:29:20.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.182+0000 7f61e1fe6740 1 -- 172.21.15.97:0/2965699070 wait complete. 2024-04-23T10:29:20.185 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.182+0000 7f61e1fe6740 20 client.16083 trim_cache size 0 max 16384 2024-04-23T10:29:20.185 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-23T10:29:20.186 INFO:teuthology.orchestra.run.smithi097.stdout: "epoch": 26, 2024-04-23T10:29:20.186 INFO:teuthology.orchestra.run.smithi097.stdout: "leader": 24488, 2024-04-23T10:29:20.186 INFO:teuthology.orchestra.run.smithi097.stdout: "set_version": 12, 2024-04-23T10:29:20.186 INFO:teuthology.orchestra.run.smithi097.stdout: "sets": { 2024-04-23T10:29:20.186 INFO:teuthology.orchestra.run.smithi097.stdout: "3f96bb3b": { 2024-04-23T10:29:20.186 INFO:teuthology.orchestra.run.smithi097.stdout: "version": 12, 2024-04-23T10:29:20.186 INFO:teuthology.orchestra.run.smithi097.stdout: "age_ref": 0.0, 2024-04-23T10:29:20.186 INFO:teuthology.orchestra.run.smithi097.stdout: "state": { 2024-04-23T10:29:20.186 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RELEASED", 2024-04-23T10:29:20.186 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0 2024-04-23T10:29:20.186 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:29:20.186 INFO:teuthology.orchestra.run.smithi097.stdout: "timeout": 60.0, 2024-04-23T10:29:20.186 INFO:teuthology.orchestra.run.smithi097.stdout: "expiration": 89.9, 2024-04-23T10:29:20.186 INFO:teuthology.orchestra.run.smithi097.stdout: "members": { 2024-04-23T10:29:20.187 INFO:teuthology.orchestra.run.smithi097.stdout: "file:/": { 2024-04-23T10:29:20.187 INFO:teuthology.orchestra.run.smithi097.stdout: "excluded": false, 2024-04-23T10:29:20.187 INFO:teuthology.orchestra.run.smithi097.stdout: "state": { 2024-04-23T10:29:20.187 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RELEASED", 2024-04-23T10:29:20.187 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0 2024-04-23T10:29:20.187 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:29:20.187 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:29:20.187 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:29:20.187 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:29:20.187 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:29:20.187 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-23T10:29:20.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.183+0000 7f61df1f7640 1 -- 172.21.15.97:0/1142929177 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f61c80a3d20 msgr2=0x7f61c80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:20.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.183+0000 7f61df1f7640 1 --2- 172.21.15.97:0/1142929177 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f61c80a3d20 0x7f61c80a61e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f61d0005fe0 tx=0x7f61d0031cb0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.183+0000 7f61df1f7640 1 -- 172.21.15.97:0/1142929177 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61d8101470 msgr2=0x7f61d81999c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:20.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.183+0000 7f61df1f7640 1 --2- 172.21.15.97:0/1142929177 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61d8101470 0x7f61d81999c0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f61c400c920 tx=0x7f61c400cde0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.183+0000 7f61df1f7640 1 -- 172.21.15.97:0/1142929177 shutdown_connections 2024-04-23T10:29:20.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.183+0000 7f61df1f7640 1 --2- 172.21.15.97:0/1142929177 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f61c80a3d20 0x7f61c80a61e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.183+0000 7f61df1f7640 1 --2- 172.21.15.97:0/1142929177 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61d8106670 0x7f61d81a4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.183+0000 7f61df1f7640 1 --2- 172.21.15.97:0/1142929177 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61d8101470 0x7f61d81999c0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.183+0000 7f61df1f7640 1 --2- 172.21.15.97:0/1142929177 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f61d8100ab0 0x7f61d8199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.183+0000 7f61df1f7640 1 -- 172.21.15.97:0/1142929177 >> 172.21.15.97:0/1142929177 conn(0x7f61d8076150 msgr2=0x7f61d8104c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:29:20.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.183+0000 7f61df1f7640 1 -- 172.21.15.97:0/1142929177 shutdown_connections 2024-04-23T10:29:20.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.183+0000 7f61df1f7640 1 -- 172.21.15.97:0/1142929177 wait complete. 2024-04-23T10:29:20.192 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '3f96bb3b', seconds elapsed: 30.9 2024-04-23T10:29:20.192 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 29.9 2024-04-23T10:29:20.192 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T10:29:20.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.292+0000 7f033a2ef640 1 -- 172.21.15.97:0/2824520352 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03341031b0 msgr2=0x7f0334103590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:20.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.292+0000 7f033a2ef640 1 --2- 172.21.15.97:0/2824520352 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03341031b0 0x7f0334103590 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f0324009900 tx=0x7f032402a480 comp rx=0 tx=0).stop 2024-04-23T10:29:20.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.292+0000 7f033a2ef640 1 -- 172.21.15.97:0/2824520352 shutdown_connections 2024-04-23T10:29:20.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.292+0000 7f033a2ef640 1 --2- 172.21.15.97:0/2824520352 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f033410c580 0x7f033410e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.292+0000 7f033a2ef640 1 --2- 172.21.15.97:0/2824520352 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0334103ad0 0x7f033410bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.292+0000 7f033a2ef640 1 --2- 172.21.15.97:0/2824520352 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03341031b0 0x7f0334103590 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.292+0000 7f033a2ef640 1 -- 172.21.15.97:0/2824520352 >> 172.21.15.97:0/2824520352 conn(0x7f03340771d0 msgr2=0x7f03340775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:29:20.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.292+0000 7f033a2ef640 1 -- 172.21.15.97:0/2824520352 shutdown_connections 2024-04-23T10:29:20.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.292+0000 7f033a2ef640 1 -- 172.21.15.97:0/2824520352 wait complete. 2024-04-23T10:29:20.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.293+0000 7f033a2ef640 1 Processor -- start 2024-04-23T10:29:20.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.293+0000 7f033a2ef640 1 -- start start 2024-04-23T10:29:20.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.293+0000 7f033a2ef640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03341031b0 0x7f0334198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:20.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.293+0000 7f033a2ef640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0334103ad0 0x7f0334198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:20.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.293+0000 7f033a2ef640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f033410c580 0x7f033419d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:20.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.293+0000 7f033a2ef640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f03341121b0 con 0x7f033410c580 2024-04-23T10:29:20.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.293+0000 7f033a2ef640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0334112030 con 0x7f0334103ad0 2024-04-23T10:29:20.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.293+0000 7f033a2ef640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0334112330 con 0x7f03341031b0 2024-04-23T10:29:20.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.293+0000 7f033888c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f033410c580 0x7f033419d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:20.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.293+0000 7f033888c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f033410c580 0x7f033419d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40198/0 (socket says 172.21.15.97:40198) 2024-04-23T10:29:20.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.293+0000 7f033888c640 1 -- 172.21.15.97:0/757144103 learned_addr learned my addr 172.21.15.97:0/757144103 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:29:20.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.293+0000 7f03337fe640 1 --2- 172.21.15.97:0/757144103 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0334103ad0 0x7f0334198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:20.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.294+0000 7f0333fff640 1 --2- 172.21.15.97:0/757144103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03341031b0 0x7f0334198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:20.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.294+0000 7f033888c640 1 -- 172.21.15.97:0/757144103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03341031b0 msgr2=0x7f0334198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:20.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.294+0000 7f033888c640 1 --2- 172.21.15.97:0/757144103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03341031b0 0x7f0334198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.294+0000 7f033888c640 1 -- 172.21.15.97:0/757144103 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0334103ad0 msgr2=0x7f0334198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:20.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.294+0000 7f033888c640 1 --2- 172.21.15.97:0/757144103 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0334103ad0 0x7f0334198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.294+0000 7f033888c640 1 -- 172.21.15.97:0/757144103 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f033419d740 con 0x7f033410c580 2024-04-23T10:29:20.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.294+0000 7f033888c640 1 --2- 172.21.15.97:0/757144103 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f033410c580 0x7f033419d010 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f032c00e9d0 tx=0x7f032c00ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:29:20.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.294+0000 7f03317fa640 1 -- 172.21.15.97:0/757144103 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f032c00cd80 con 0x7f033410c580 2024-04-23T10:29:20.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.294+0000 7f033a2ef640 1 -- 172.21.15.97:0/757144103 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f033419da30 con 0x7f033410c580 2024-04-23T10:29:20.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.294+0000 7f03317fa640 1 -- 172.21.15.97:0/757144103 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f032c005020 con 0x7f033410c580 2024-04-23T10:29:20.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.294+0000 7f03317fa640 1 -- 172.21.15.97:0/757144103 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f032c0116d0 con 0x7f033410c580 2024-04-23T10:29:20.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.294+0000 7f033a2ef640 1 -- 172.21.15.97:0/757144103 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03341a52c0 con 0x7f033410c580 2024-04-23T10:29:20.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.295+0000 7f033a2ef640 1 -- 172.21.15.97:0/757144103 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03340691c0 con 0x7f033410c580 2024-04-23T10:29:20.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.296+0000 7f03317fa640 1 -- 172.21.15.97:0/757144103 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f032c0051d0 con 0x7f033410c580 2024-04-23T10:29:20.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.297+0000 7f03317fa640 1 --2- 172.21.15.97:0/757144103 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f031c0a3d20 0x7f031c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:20.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.297+0000 7f0333fff640 1 --2- 172.21.15.97:0/757144103 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f031c0a3d20 0x7f031c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:20.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.297+0000 7f03317fa640 1 -- 172.21.15.97:0/757144103 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f032c0d0890 con 0x7f033410c580 2024-04-23T10:29:20.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.297+0000 7f0333fff640 1 --2- 172.21.15.97:0/757144103 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f031c0a3d20 0x7f031c0a61e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0324031440 tx=0x7f0324005b50 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:29:20.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.301+0000 7f03317fa640 1 -- 172.21.15.97:0/757144103 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f032c016030 con 0x7f033410c580 2024-04-23T10:29:20.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:29:20 smithi097 ceph-mon[29197]: pgmap v97: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-04-23T10:29:20.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:29:20 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:29:20.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:29:20 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:29:20.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:29:20 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:29:20.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:29:20 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/3967909817' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:29:20.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.520+0000 7f033a2ef640 1 -- 172.21.15.97:0/757144103 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f033402d5c0 con 0x7f033410c580 2024-04-23T10:29:20.519 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.521+0000 7f03317fa640 1 -- 172.21.15.97:0/757144103 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11931 (secure 0 0 0) 0x7f032c09c930 con 0x7f033410c580 2024-04-23T10:29:20.520 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:29:20.520 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3946878762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3946878762},{"type":"v1","addr":"172.21.15.97:6835","nonce":3946878762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/209800861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":209800861},{"type":"v1","addr":"172.21.15.97:6837","nonce":209800861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T10:24:06.910134+0000","modified":"2024-04-23T10:26:09.820985+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6839/418013575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":418013575},{"type":"v1","addr":"172.21.15.97:6839","nonce":418013575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6841/747242680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":747242680},{"type":"v1","addr":"172.21.15.97:6841","nonce":747242680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6833/4292847121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4292847121},{"type":"v1","addr":"172.21.15.183:6833","nonce":4292847121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6833/3885816565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3885816565},{"type":"v1","addr":"172.21.15.196:6833","nonce":3885816565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1897093105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1897093105},{"type":"v1","addr":"172.21.15.183:6835","nonce":1897093105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1171477563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1171477563},{"type":"v1","addr":"172.21.15.196:6835","nonce":1171477563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1898341911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1898341911},{"type":"v1","addr":"172.21.15.183:6837","nonce":1898341911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.196:6837/3094946502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3094946502},{"type":"v1","addr":"172.21.15.196:6837","nonce":3094946502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.196:6839/83843556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":83843556},{"type":"v1","addr":"172.21.15.196:6839","nonce":83843556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6839/1763841130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1763841130},{"type":"v1","addr":"172.21.15.183:6839","nonce":1763841130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24487,24403,24398,24469,24488]},"id":1}]} 2024-04-23T10:29:20.520 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 26 2024-04-23T10:29:20.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.523+0000 7f033a2ef640 1 -- 172.21.15.97:0/757144103 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f031c0a3d20 msgr2=0x7f031c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:20.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.523+0000 7f033a2ef640 1 --2- 172.21.15.97:0/757144103 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f031c0a3d20 0x7f031c0a61e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0324031440 tx=0x7f0324005b50 comp rx=0 tx=0).stop 2024-04-23T10:29:20.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.523+0000 7f033a2ef640 1 -- 172.21.15.97:0/757144103 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f033410c580 msgr2=0x7f033419d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:20.522 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.523+0000 7f033a2ef640 1 --2- 172.21.15.97:0/757144103 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f033410c580 0x7f033419d010 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f032c00e9d0 tx=0x7f032c00ee90 comp rx=0 tx=0).stop 2024-04-23T10:29:20.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.523+0000 7f033a2ef640 1 -- 172.21.15.97:0/757144103 shutdown_connections 2024-04-23T10:29:20.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.523+0000 7f033a2ef640 1 --2- 172.21.15.97:0/757144103 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f031c0a3d20 0x7f031c0a61e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.523+0000 7f033a2ef640 1 --2- 172.21.15.97:0/757144103 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f033410c580 0x7f033419d010 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.523+0000 7f033a2ef640 1 --2- 172.21.15.97:0/757144103 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0334103ad0 0x7f0334198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.523+0000 7f033a2ef640 1 --2- 172.21.15.97:0/757144103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03341031b0 0x7f0334198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.523+0000 7f033a2ef640 1 -- 172.21.15.97:0/757144103 >> 172.21.15.97:0/757144103 conn(0x7f03340771d0 msgr2=0x7f03340fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:29:20.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.523+0000 7f033a2ef640 1 -- 172.21.15.97:0/757144103 shutdown_connections 2024-04-23T10:29:20.523 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.523+0000 7f033a2ef640 1 -- 172.21.15.97:0/757144103 wait complete. 2024-04-23T10:29:20.533 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db / --timeout 60 --expiration 89.9 --await-for 42.0' 2024-04-23T10:29:20.533 DEBUG:teuthology.orchestra.run.smithi097:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24488 quiesce db / --timeout 60 --expiration 89.9 --await-for 42.0 2024-04-23T10:29:20.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:29:20 smithi196 ceph-mon[32919]: pgmap v97: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-04-23T10:29:20.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:29:20 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:29:20.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:29:20 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:29:20.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:29:20 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:29:20.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:29:20 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/3967909817' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:29:20.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.642+0000 7f2a179e2640 1 -- 172.21.15.97:0/354005534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a10107270 msgr2=0x7f2a1010ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:20.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.642+0000 7f2a179e2640 1 --2- 172.21.15.97:0/354005534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a10107270 0x7f2a1010ee40 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f2a08009ca0 tx=0x7f2a0802a8c0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.642+0000 7f2a179e2640 1 -- 172.21.15.97:0/354005534 shutdown_connections 2024-04-23T10:29:20.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.642+0000 7f2a179e2640 1 --2- 172.21.15.97:0/354005534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a10107270 0x7f2a1010ee40 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.642+0000 7f2a179e2640 1 --2- 172.21.15.97:0/354005534 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a1006b290 0x7f2a10106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.641 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.642+0000 7f2a179e2640 1 --2- 172.21.15.97:0/354005534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a1006a8e0 0x7f2a1006acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.642+0000 7f2a179e2640 1 -- 172.21.15.97:0/354005534 >> 172.21.15.97:0/354005534 conn(0x7f2a10077f20 msgr2=0x7f2a10078330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:29:20.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.642+0000 7f2a179e2640 1 -- 172.21.15.97:0/354005534 shutdown_connections 2024-04-23T10:29:20.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.642+0000 7f2a179e2640 1 -- 172.21.15.97:0/354005534 wait complete. 2024-04-23T10:29:20.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.643+0000 7f2a179e2640 1 Processor -- start 2024-04-23T10:29:20.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.643+0000 7f2a179e2640 1 -- start start 2024-04-23T10:29:20.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.643+0000 7f2a179e2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a1006a8e0 0x7f2a10198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:20.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.643+0000 7f2a179e2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a10107270 0x7f2a10198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:20.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.643+0000 7f2a179e2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a1019cfd0 0x7f2a1019d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:20.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.643+0000 7f2a179e2640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2a101121e0 con 0x7f2a1019cfd0 2024-04-23T10:29:20.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.643+0000 7f2a179e2640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2a10112060 con 0x7f2a1006a8e0 2024-04-23T10:29:20.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.643+0000 7f2a179e2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2a10112360 con 0x7f2a10107270 2024-04-23T10:29:20.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.643+0000 7f2a15f7f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a1019cfd0 0x7f2a1019d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:20.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.643+0000 7f2a15f7f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a1019cfd0 0x7f2a1019d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40212/0 (socket says 172.21.15.97:40212) 2024-04-23T10:29:20.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.643+0000 7f2a15f7f640 1 -- 172.21.15.97:0/1715556360 learned_addr learned my addr 172.21.15.97:0/1715556360 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:29:20.644 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.643+0000 7f2a1577e640 1 --2- 172.21.15.97:0/1715556360 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a1006a8e0 0x7f2a10198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:20.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.643+0000 7f2a14f7d640 1 --2- 172.21.15.97:0/1715556360 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a10107270 0x7f2a10198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:20.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.643+0000 7f2a15f7f640 1 -- 172.21.15.97:0/1715556360 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a10107270 msgr2=0x7f2a10198ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:20.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.643+0000 7f2a15f7f640 1 --2- 172.21.15.97:0/1715556360 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a10107270 0x7f2a10198ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.643+0000 7f2a15f7f640 1 -- 172.21.15.97:0/1715556360 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a1006a8e0 msgr2=0x7f2a10198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:20.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.643+0000 7f2a15f7f640 1 --2- 172.21.15.97:0/1715556360 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a1006a8e0 0x7f2a10198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.644+0000 7f2a15f7f640 1 -- 172.21.15.97:0/1715556360 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a1019dac0 con 0x7f2a1019cfd0 2024-04-23T10:29:20.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.644+0000 7f2a14f7d640 1 --2- 172.21.15.97:0/1715556360 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a10107270 0x7f2a10198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:29:20.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.644+0000 7f2a15f7f640 1 --2- 172.21.15.97:0/1715556360 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a1019cfd0 0x7f2a1019d480 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f2a08005cc0 tx=0x7f2a0802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:29:20.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.644+0000 7f2a067fc640 1 -- 172.21.15.97:0/1715556360 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a08003bf0 con 0x7f2a1019cfd0 2024-04-23T10:29:20.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.644+0000 7f2a179e2640 1 -- 172.21.15.97:0/1715556360 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a1019dd50 con 0x7f2a1019cfd0 2024-04-23T10:29:20.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.644+0000 7f2a067fc640 1 -- 172.21.15.97:0/1715556360 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a08003d90 con 0x7f2a1019cfd0 2024-04-23T10:29:20.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.644+0000 7f2a067fc640 1 -- 172.21.15.97:0/1715556360 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0802fbc0 con 0x7f2a1019cfd0 2024-04-23T10:29:20.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.644+0000 7f2a179e2640 1 -- 172.21.15.97:0/1715556360 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a101a5570 con 0x7f2a1019cfd0 2024-04-23T10:29:20.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:29:20 smithi183 ceph-mon[32317]: pgmap v97: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-04-23T10:29:20.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:29:20 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:29:20.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:29:20 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:29:20.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:29:20 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:29:20.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:29:20 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/3967909817' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:29:20.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.646+0000 7f2a067fc640 1 -- 172.21.15.97:0/1715556360 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2a0802fd60 con 0x7f2a1019cfd0 2024-04-23T10:29:20.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.646+0000 7f2a1a7d1740 1 Processor -- start 2024-04-23T10:29:20.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.646+0000 7f2a1a7d1740 1 -- start start 2024-04-23T10:29:20.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.647+0000 7f2a1a7d1740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56468af1b0a0 0x56468af1b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:20.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.647+0000 7f2a1a7d1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56468af1b9c0 0x56468af25b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:20.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.647+0000 7f2a1a7d1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56468af26090 0x56468af28480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:20.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.647+0000 7f2a1a7d1740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56468ae80370 con 0x56468af1b0a0 2024-04-23T10:29:20.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.647+0000 7f2a1a7d1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56468ae1d2e0 con 0x56468af26090 2024-04-23T10:29:20.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.647+0000 7f2a1a7d1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56468ae7c330 con 0x56468af1b9c0 2024-04-23T10:29:20.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.647+0000 7f2a14f7d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56468af1b0a0 0x56468af1b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:20.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.647+0000 7f2a14f7d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56468af1b0a0 0x56468af1b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40222/0 (socket says 172.21.15.97:40222) 2024-04-23T10:29:20.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.647+0000 7f2a14f7d640 1 -- 172.21.15.97:0/4192952654 learned_addr learned my addr 172.21.15.97:0/4192952654 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:29:20.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.647+0000 7f2a067fc640 1 --2- 172.21.15.97:0/1715556360 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f29fc0a3d20 0x7f29fc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:20.653 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.647+0000 7f2a1577e640 1 --2- 172.21.15.97:0/4192952654 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56468af1b9c0 0x56468af25b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:20.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.647+0000 7f2a14f7d640 1 --2- 172.21.15.97:0/4192952654 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56468af26090 0x56468af28480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:20.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.647+0000 7f2a14f7d640 1 -- 172.21.15.97:0/4192952654 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56468af1b9c0 msgr2=0x56468af25b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:29:20.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.647+0000 7f2a14f7d640 1 --2- 172.21.15.97:0/4192952654 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56468af1b9c0 0x56468af25b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.647+0000 7f2a14f7d640 1 -- 172.21.15.97:0/4192952654 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56468af26090 msgr2=0x56468af28480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:29:20.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.647+0000 7f2a14f7d640 1 --2- 172.21.15.97:0/4192952654 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56468af26090 0x56468af28480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.647+0000 7f2a15f7f640 1 --2- 172.21.15.97:0/1715556360 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f29fc0a3d20 0x7f29fc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:20.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.647+0000 7f2a14f7d640 1 -- 172.21.15.97:0/4192952654 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56468ac5ed80 con 0x56468af1b0a0 2024-04-23T10:29:20.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.647+0000 7f2a067fc640 1 -- 172.21.15.97:0/1715556360 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f2a080f1dc0 con 0x7f2a1019cfd0 2024-04-23T10:29:20.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.647+0000 7f2a14f7d640 1 --2- 172.21.15.97:0/4192952654 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56468af1b0a0 0x56468af1b480 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f2a0000bef0 tx=0x7f2a0000b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c6744394dc4ecf9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:29:20.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.647+0000 7f2a15f7f640 1 --2- 172.21.15.97:0/1715556360 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f29fc0a3d20 0x7f29fc0a61e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2a080ef720 tx=0x7f2a0808fb40 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:29:20.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.647+0000 7f29fb7fe640 1 -- 172.21.15.97:0/4192952654 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a00004030 con 0x56468af1b0a0 2024-04-23T10:29:20.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.647+0000 7f29fb7fe640 1 -- 172.21.15.97:0/4192952654 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a00019070 con 0x56468af1b0a0 2024-04-23T10:29:20.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.647+0000 7f29fb7fe640 1 -- 172.21.15.97:0/4192952654 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0000f3f0 con 0x56468af1b0a0 2024-04-23T10:29:20.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.648+0000 7f2a1a7d1740 1 -- 172.21.15.97:0/4192952654 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56468af1b0a0 msgr2=0x56468af1b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:20.654 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.648+0000 7f2a1a7d1740 1 --2- 172.21.15.97:0/4192952654 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56468af1b0a0 0x56468af1b480 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f2a0000bef0 tx=0x7f2a0000b670 comp rx=0 tx=0).stop 2024-04-23T10:29:20.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.648+0000 7f2a1a7d1740 1 -- 172.21.15.97:0/4192952654 shutdown_connections 2024-04-23T10:29:20.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.648+0000 7f2a1a7d1740 1 --2- 172.21.15.97:0/4192952654 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56468af26090 0x56468af28480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.648+0000 7f2a1a7d1740 1 --2- 172.21.15.97:0/4192952654 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56468af1b9c0 0x56468af25b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.648+0000 7f2a1a7d1740 1 --2- 172.21.15.97:0/4192952654 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56468af1b0a0 0x56468af1b480 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.648+0000 7f2a1a7d1740 1 -- 172.21.15.97:0/4192952654 >> 172.21.15.97:0/4192952654 conn(0x56468af173c0 msgr2=0x56468af177a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:29:20.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.648+0000 7f2a1a7d1740 1 -- 172.21.15.97:0/4192952654 shutdown_connections 2024-04-23T10:29:20.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.648+0000 7f2a1a7d1740 1 -- 172.21.15.97:0/4192952654 wait complete. 2024-04-23T10:29:20.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.648+0000 7f2a1a7d1740 1 -- start start 2024-04-23T10:29:20.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.649+0000 7f2a1a7d1740 1 Processor -- start 2024-04-23T10:29:20.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.649+0000 7f2a1a7d1740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56468af1b0a0 0x56468af23000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:20.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.649+0000 7f2a1a7d1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56468af1b9c0 0x56468af23540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:20.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.649+0000 7f2a1a7d1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56468af26090 0x56468af23a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:20.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.649+0000 7f2a1a7d1740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56468af28e80 con 0x56468af1b0a0 2024-04-23T10:29:20.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.649+0000 7f2a1a7d1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56468ae1afb0 con 0x56468af1b9c0 2024-04-23T10:29:20.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.649+0000 7f2a1a7d1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56468af29000 con 0x56468af26090 2024-04-23T10:29:20.655 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.649+0000 7f2a1a7d1740 10 client.0 _ll_register_callbacks cb 0x56468ae1a9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:29:20.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.649+0000 7f2a14f7d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56468af1b0a0 0x56468af23000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:20.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.649+0000 7f2a14f7d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56468af1b0a0 0x56468af23000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40236/0 (socket says 172.21.15.97:40236) 2024-04-23T10:29:20.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.649+0000 7f2a14f7d640 1 -- 172.21.15.97:0/2114207361 learned_addr learned my addr 172.21.15.97:0/2114207361 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:29:20.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.649+0000 7f2a14f7d640 1 --2- 172.21.15.97:0/2114207361 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56468af26090 0x56468af23a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:20.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.649+0000 7f2a1577e640 1 --2- 172.21.15.97:0/2114207361 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56468af1b9c0 0x56468af23540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:20.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.649+0000 7f2a14f7d640 1 -- 172.21.15.97:0/2114207361 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56468af26090 msgr2=0x56468af23a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:29:20.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.649+0000 7f2a14f7d640 1 --2- 172.21.15.97:0/2114207361 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56468af26090 0x56468af23a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.649+0000 7f2a14f7d640 1 -- 172.21.15.97:0/2114207361 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56468af1b9c0 msgr2=0x56468af23540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:29:20.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.649+0000 7f2a14f7d640 1 --2- 172.21.15.97:0/2114207361 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56468af1b9c0 0x56468af23540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.649+0000 7f2a14f7d640 1 -- 172.21.15.97:0/2114207361 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56468acbf510 con 0x56468af1b0a0 2024-04-23T10:29:20.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.650+0000 7f2a14f7d640 1 --2- 172.21.15.97:0/2114207361 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56468af1b0a0 0x56468af23000 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f2a0000b4a0 tx=0x7f2a0000b5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15fbcbd6341807b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:29:20.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.650+0000 7f29f8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:29:20.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.650+0000 7f29f8ff9640 1 -- 172.21.15.97:0/2114207361 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a00026020 con 0x56468af1b0a0 2024-04-23T10:29:20.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.650+0000 7f29f8ff9640 1 -- 172.21.15.97:0/2114207361 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f2a00004000 con 0x56468af1b0a0 2024-04-23T10:29:20.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.650+0000 7f2a1a7d1740 1 -- 172.21.15.97:0/2114207361 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a00019070 con 0x56468af1b0a0 2024-04-23T10:29:20.656 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.650+0000 7f29f8ff9640 1 -- 172.21.15.97:0/2114207361 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2a00004200 con 0x56468af1b0a0 2024-04-23T10:29:20.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.650+0000 7f2a1a7d1740 10 client.16101 fetch_fsmap learned FSMap version 26 2024-04-23T10:29:20.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.650+0000 7f2a1a7d1740 1 -- 172.21.15.97:0/2114207361 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56468ab52670 con 0x56468af1b0a0 2024-04-23T10:29:20.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.650+0000 7f29f8ff9640 1 -- 172.21.15.97:0/2114207361 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a00019070 con 0x56468af1b0a0 2024-04-23T10:29:20.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.650+0000 7f29f8ff9640 1 -- 172.21.15.97:0/2114207361 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f2a0001f410 con 0x56468af1b0a0 2024-04-23T10:29:20.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.650+0000 7f2a1a7d1740 10 client.16101 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:29:20.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.650+0000 7f2a1a7d1740 10 client.16101 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:29:20.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.650+0000 7f2a1a7d1740 20 client.16101 populate_metadata read hostname 'smithi097' 2024-04-23T10:29:20.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.650+0000 7f2a1a7d1740 1 --2- 172.21.15.97:0/2114207361 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x56468af363b0 0x56468af387a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:20.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.650+0000 7f2a1a7d1740 4 client.16101 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:29:20.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.650+0000 7f2a1a7d1740 1 -- 172.21.15.97:0/2114207361 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56468ae8dd90 con 0x56468af363b0 2024-04-23T10:29:20.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.651+0000 7f2a1577e640 1 --2- 172.21.15.97:0/2114207361 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x56468af363b0 0x56468af387a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:20.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.651+0000 7f2a1577e640 1 --2- 172.21.15.97:0/2114207361 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x56468af363b0 0x56468af387a0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c23fe852da7589d server_cookie=8f94f512ba79adab in_seq=0 out_seq=0 2024-04-23T10:29:20.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.651+0000 7f29f8ff9640 10 client.16101 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:29:20.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.652+0000 7f29f8ff9640 1 -- 172.21.15.97:0/2114207361 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56468ae8dd90 con 0x56468af363b0 2024-04-23T10:29:20.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.652+0000 7f29f8ff9640 10 client.16101 handle_command_reply: tid=0 2024-04-23T10:29:20.657 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.652+0000 7f2a1a7d1740 1 client.16101 shutdown 2024-04-23T10:29:20.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.652+0000 7f2a1a7d1740 1 -- 172.21.15.97:0/2114207361 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56468af1b0a0 msgr2=0x56468af23000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:20.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.652+0000 7f2a1a7d1740 1 --2- 172.21.15.97:0/2114207361 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56468af1b0a0 0x56468af23000 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f2a0000b4a0 tx=0x7f2a0000b5d0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.652+0000 7f2a1a7d1740 1 -- 172.21.15.97:0/2114207361 shutdown_connections 2024-04-23T10:29:20.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.652+0000 7f2a1a7d1740 1 --2- 172.21.15.97:0/2114207361 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x56468af363b0 0x56468af387a0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.652+0000 7f2a1a7d1740 1 --2- 172.21.15.97:0/2114207361 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56468af26090 0x56468af23a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.652+0000 7f2a1a7d1740 1 --2- 172.21.15.97:0/2114207361 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56468af1b9c0 0x56468af23540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.652+0000 7f29f8ff9640 0 client.16101 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:29:20.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.652+0000 7f2a1a7d1740 1 --2- 172.21.15.97:0/2114207361 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56468af1b0a0 0x56468af23000 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.652+0000 7f2a1a7d1740 1 -- 172.21.15.97:0/2114207361 >> 172.21.15.97:0/2114207361 conn(0x56468af173c0 msgr2=0x56468af1d0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:29:20.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.652+0000 7f2a1a7d1740 1 -- 172.21.15.97:0/2114207361 shutdown_connections 2024-04-23T10:29:20.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.652+0000 7f2a1a7d1740 1 -- 172.21.15.97:0/2114207361 wait complete. 2024-04-23T10:29:20.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.653+0000 7f2a1a7d1740 20 client.16101 trim_cache size 0 max 16384 2024-04-23T10:29:20.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.669+0000 7f2a1a7d1740 1 Processor -- start 2024-04-23T10:29:20.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.669+0000 7f2a1a7d1740 1 -- start start 2024-04-23T10:29:20.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.670+0000 7f2a1a7d1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56468af1b0a0 0x56468af31d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:20.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.670+0000 7f2a1a7d1740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56468af1b9c0 0x56468af225a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:20.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.670+0000 7f2a1a7d1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56468af26090 0x56468af3de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:20.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.670+0000 7f2a1a7d1740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2a101adb00 con 0x56468af1b9c0 2024-04-23T10:29:20.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.670+0000 7f2a1a7d1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2a101af640 con 0x56468af1b0a0 2024-04-23T10:29:20.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.670+0000 7f2a1a7d1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2a101ae1d0 con 0x56468af26090 2024-04-23T10:29:20.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.670+0000 7f2a1577e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56468af1b9c0 0x56468af225a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:20.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.670+0000 7f2a1577e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56468af1b9c0 0x56468af225a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40244/0 (socket says 172.21.15.97:40244) 2024-04-23T10:29:20.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.670+0000 7f2a1577e640 1 -- 172.21.15.97:0/3768732175 learned_addr learned my addr 172.21.15.97:0/3768732175 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:29:20.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.670+0000 7f2a14f7d640 1 --2- 172.21.15.97:0/3768732175 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56468af26090 0x56468af3de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:20.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.670+0000 7f2a1577e640 1 -- 172.21.15.97:0/3768732175 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56468af26090 msgr2=0x56468af3de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:29:20.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.670+0000 7f2a1577e640 1 --2- 172.21.15.97:0/3768732175 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56468af26090 0x56468af3de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.670+0000 7f2a1577e640 1 -- 172.21.15.97:0/3768732175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56468af1b0a0 msgr2=0x56468af31d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:29:20.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.670+0000 7f2a1577e640 1 --2- 172.21.15.97:0/3768732175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56468af1b0a0 0x56468af31d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.670+0000 7f2a1577e640 1 -- 172.21.15.97:0/3768732175 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56468af18120 con 0x56468af1b9c0 2024-04-23T10:29:20.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.670+0000 7f2a14f7d640 1 --2- 172.21.15.97:0/3768732175 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56468af26090 0x56468af3de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:29:20.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.670+0000 7f2a1577e640 1 --2- 172.21.15.97:0/3768732175 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56468af1b9c0 0x56468af225a0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c00b780 tx=0x7f2a0c004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43b8e70db1affe77 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:29:20.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.670+0000 7f29faffd640 1 -- 172.21.15.97:0/3768732175 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c021020 con 0x56468af1b9c0 2024-04-23T10:29:20.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.670+0000 7f29faffd640 1 -- 172.21.15.97:0/3768732175 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a0c01d8d0 con 0x56468af1b9c0 2024-04-23T10:29:20.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.670+0000 7f29faffd640 1 -- 172.21.15.97:0/3768732175 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c010bf0 con 0x56468af1b9c0 2024-04-23T10:29:20.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.671+0000 7f2a1a7d1740 1 -- 172.21.15.97:0/3768732175 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56468af1b9c0 msgr2=0x56468af225a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:20.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.671+0000 7f2a1a7d1740 1 --2- 172.21.15.97:0/3768732175 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56468af1b9c0 0x56468af225a0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c00b780 tx=0x7f2a0c004530 comp rx=0 tx=0).stop 2024-04-23T10:29:20.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.671+0000 7f2a1a7d1740 1 -- 172.21.15.97:0/3768732175 shutdown_connections 2024-04-23T10:29:20.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.671+0000 7f2a1a7d1740 1 --2- 172.21.15.97:0/3768732175 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56468af26090 0x56468af3de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.671+0000 7f2a1a7d1740 1 --2- 172.21.15.97:0/3768732175 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56468af1b9c0 0x56468af225a0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.671+0000 7f2a1a7d1740 1 --2- 172.21.15.97:0/3768732175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56468af1b0a0 0x56468af31d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.671+0000 7f2a1a7d1740 1 -- 172.21.15.97:0/3768732175 >> 172.21.15.97:0/3768732175 conn(0x56468af173c0 msgr2=0x56468af24830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:29:20.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.671+0000 7f2a1a7d1740 1 -- 172.21.15.97:0/3768732175 shutdown_connections 2024-04-23T10:29:20.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.671+0000 7f2a1a7d1740 1 -- 172.21.15.97:0/3768732175 wait complete. 2024-04-23T10:29:20.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.671+0000 7f2a1a7d1740 1 -- start start 2024-04-23T10:29:20.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.671+0000 7f2a1a7d1740 1 Processor -- start 2024-04-23T10:29:20.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.672+0000 7f2a1a7d1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56468af1b0a0 0x56468af20140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:20.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.672+0000 7f2a1a7d1740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56468af1b9c0 0x56468af20680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:20.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.672+0000 7f2a1a7d1740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56468af26090 0x56468af9b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:20.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.672+0000 7f2a1a7d1740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2a101aef70 con 0x56468af1b9c0 2024-04-23T10:29:20.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.672+0000 7f2a1a7d1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2a101ae8a0 con 0x56468af1b0a0 2024-04-23T10:29:20.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.672+0000 7f2a1a7d1740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2a101abb00 con 0x56468af26090 2024-04-23T10:29:20.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.672+0000 7f2a1a7d1740 10 client.0 _ll_register_callbacks cb 0x7f2a00011270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:29:20.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.672+0000 7f2a1577e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56468af1b9c0 0x56468af20680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:20.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.672+0000 7f2a1577e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56468af1b9c0 0x56468af20680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40248/0 (socket says 172.21.15.97:40248) 2024-04-23T10:29:20.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.672+0000 7f2a1577e640 1 -- 172.21.15.97:0/146943930 learned_addr learned my addr 172.21.15.97:0/146943930 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:29:20.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.672+0000 7f2a14f7d640 1 --2- 172.21.15.97:0/146943930 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56468af1b0a0 0x56468af20140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:20.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.672+0000 7f2a1577e640 1 -- 172.21.15.97:0/146943930 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56468af26090 msgr2=0x56468af9b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:29:20.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.672+0000 7f2a1577e640 1 --2- 172.21.15.97:0/146943930 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56468af26090 0x56468af9b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.672+0000 7f2a1577e640 1 -- 172.21.15.97:0/146943930 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56468af1b0a0 msgr2=0x56468af20140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:29:20.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.672+0000 7f2a1577e640 1 --2- 172.21.15.97:0/146943930 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56468af1b0a0 0x56468af20140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.672+0000 7f2a1577e640 1 -- 172.21.15.97:0/146943930 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56468af9bc00 con 0x56468af1b9c0 2024-04-23T10:29:20.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.672+0000 7f2a1577e640 1 --2- 172.21.15.97:0/146943930 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56468af1b9c0 0x56468af20680 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c005c20 tx=0x7f2a0c011e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4868ca108db22b24 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:29:20.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.672+0000 7f29f8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:29:20.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.672+0000 7f29f8ff9640 1 -- 172.21.15.97:0/146943930 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c021030 con 0x56468af1b9c0 2024-04-23T10:29:20.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.672+0000 7f29f8ff9640 1 -- 172.21.15.97:0/146943930 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f2a0c010630 con 0x56468af1b9c0 2024-04-23T10:29:20.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.672+0000 7f2a1a7d1740 1 -- 172.21.15.97:0/146943930 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a0c01d8d0 con 0x56468af1b9c0 2024-04-23T10:29:20.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.673+0000 7f29f8ff9640 1 -- 172.21.15.97:0/146943930 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2a0c010970 con 0x56468af1b9c0 2024-04-23T10:29:20.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.673+0000 7f2a1a7d1740 10 client.16107 fetch_fsmap learned FSMap version 26 2024-04-23T10:29:20.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.673+0000 7f2a1a7d1740 1 -- 172.21.15.97:0/146943930 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56468af9cd10 con 0x56468af1b9c0 2024-04-23T10:29:20.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.673+0000 7f29f8ff9640 1 -- 172.21.15.97:0/146943930 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a0c01d8d0 con 0x56468af1b9c0 2024-04-23T10:29:20.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.673+0000 7f29f8ff9640 1 -- 172.21.15.97:0/146943930 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f2a0c0046c0 con 0x56468af1b9c0 2024-04-23T10:29:20.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.673+0000 7f2a1a7d1740 10 client.16107 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:29:20.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.673+0000 7f2a1a7d1740 10 client.16107 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:29:20.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.673+0000 7f2a1a7d1740 20 client.16107 populate_metadata read hostname 'smithi097' 2024-04-23T10:29:20.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.673+0000 7f2a1a7d1740 1 --2- 172.21.15.97:0/146943930 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x56468af9d420 0x56468af9f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:20.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.673+0000 7f2a1a7d1740 4 client.16107 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 89.9, "await_for": 42.0}] 2024-04-23T10:29:20.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.673+0000 7f2a1a7d1740 1 -- 172.21.15.97:0/146943930 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 89.9, "await_for": 42.0}) -- 0x56468ae87c00 con 0x56468af9d420 2024-04-23T10:29:20.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.673+0000 7f2a14f7d640 1 --2- 172.21.15.97:0/146943930 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x56468af9d420 0x56468af9f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:20.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.674+0000 7f2a14f7d640 1 --2- 172.21.15.97:0/146943930 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x56468af9d420 0x56468af9f840 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76b1a1c9b524e463 server_cookie=728c163b4be09d71 in_seq=0 out_seq=0 2024-04-23T10:29:20.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.674+0000 7f29f8ff9640 10 client.16107 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:29:20.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.688+0000 7f29f8ff9640 1 -- 172.21.15.97:0/146943930 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x56468ae87c00 con 0x56468af9d420 2024-04-23T10:29:20.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.688+0000 7f29f8ff9640 10 client.16107 handle_command_reply: tid=0 2024-04-23T10:29:20.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.688+0000 7f2a1a7d1740 1 client.16107 shutdown 2024-04-23T10:29:20.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.688+0000 7f2a1a7d1740 1 -- 172.21.15.97:0/146943930 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56468af1b9c0 msgr2=0x56468af20680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:20.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.688+0000 7f2a1a7d1740 1 --2- 172.21.15.97:0/146943930 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56468af1b9c0 0x56468af20680 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c005c20 tx=0x7f2a0c011e30 comp rx=0 tx=0).stop 2024-04-23T10:29:20.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.688+0000 7f2a1a7d1740 1 -- 172.21.15.97:0/146943930 shutdown_connections 2024-04-23T10:29:20.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.688+0000 7f2a1a7d1740 1 --2- 172.21.15.97:0/146943930 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x56468af9d420 0x56468af9f840 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.688+0000 7f2a1a7d1740 1 --2- 172.21.15.97:0/146943930 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56468af26090 0x56468af9b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.688+0000 7f2a1a7d1740 1 --2- 172.21.15.97:0/146943930 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56468af1b9c0 0x56468af20680 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.688+0000 7f29f8ff9640 0 client.16107 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:29:20.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.688+0000 7f2a1a7d1740 1 --2- 172.21.15.97:0/146943930 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56468af1b0a0 0x56468af20140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.688+0000 7f2a1a7d1740 1 -- 172.21.15.97:0/146943930 >> 172.21.15.97:0/146943930 conn(0x56468af173c0 msgr2=0x56468af250c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:29:20.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.689+0000 7f2a1a7d1740 1 -- 172.21.15.97:0/146943930 shutdown_connections 2024-04-23T10:29:20.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.689+0000 7f2a1a7d1740 1 -- 172.21.15.97:0/146943930 wait complete. 2024-04-23T10:29:20.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.689+0000 7f2a1a7d1740 20 client.16107 trim_cache size 0 max 16384 2024-04-23T10:29:20.689 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-23T10:29:20.689 INFO:teuthology.orchestra.run.smithi097.stdout: "epoch": 26, 2024-04-23T10:29:20.689 INFO:teuthology.orchestra.run.smithi097.stdout: "leader": 24488, 2024-04-23T10:29:20.689 INFO:teuthology.orchestra.run.smithi097.stdout: "set_version": 14, 2024-04-23T10:29:20.689 INFO:teuthology.orchestra.run.smithi097.stdout: "sets": { 2024-04-23T10:29:20.689 INFO:teuthology.orchestra.run.smithi097.stdout: "2ee2b90": { 2024-04-23T10:29:20.689 INFO:teuthology.orchestra.run.smithi097.stdout: "version": 14, 2024-04-23T10:29:20.689 INFO:teuthology.orchestra.run.smithi097.stdout: "age_ref": 0.0, 2024-04-23T10:29:20.689 INFO:teuthology.orchestra.run.smithi097.stdout: "state": { 2024-04-23T10:29:20.689 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "QUIESCED", 2024-04-23T10:29:20.690 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0 2024-04-23T10:29:20.690 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:29:20.690 INFO:teuthology.orchestra.run.smithi097.stdout: "timeout": 60.0, 2024-04-23T10:29:20.690 INFO:teuthology.orchestra.run.smithi097.stdout: "expiration": 89.9, 2024-04-23T10:29:20.690 INFO:teuthology.orchestra.run.smithi097.stdout: "members": { 2024-04-23T10:29:20.690 INFO:teuthology.orchestra.run.smithi097.stdout: "file:/": { 2024-04-23T10:29:20.690 INFO:teuthology.orchestra.run.smithi097.stdout: "excluded": false, 2024-04-23T10:29:20.690 INFO:teuthology.orchestra.run.smithi097.stdout: "state": { 2024-04-23T10:29:20.690 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "QUIESCED", 2024-04-23T10:29:20.690 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0 2024-04-23T10:29:20.690 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:29:20.690 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:29:20.690 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:29:20.690 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:29:20.690 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:29:20.691 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-23T10:29:20.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.689+0000 7f2a179e2640 1 -- 172.21.15.97:0/1715556360 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f29fc0a3d20 msgr2=0x7f29fc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:20.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.689+0000 7f2a179e2640 1 --2- 172.21.15.97:0/1715556360 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f29fc0a3d20 0x7f29fc0a61e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2a080ef720 tx=0x7f2a0808fb40 comp rx=0 tx=0).stop 2024-04-23T10:29:20.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.689+0000 7f2a179e2640 1 -- 172.21.15.97:0/1715556360 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a1019cfd0 msgr2=0x7f2a1019d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:20.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.689+0000 7f2a179e2640 1 --2- 172.21.15.97:0/1715556360 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a1019cfd0 0x7f2a1019d480 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f2a08005cc0 tx=0x7f2a0802f380 comp rx=0 tx=0).stop 2024-04-23T10:29:20.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.689+0000 7f2a179e2640 1 -- 172.21.15.97:0/1715556360 shutdown_connections 2024-04-23T10:29:20.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.689+0000 7f2a179e2640 1 --2- 172.21.15.97:0/1715556360 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f29fc0a3d20 0x7f29fc0a61e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.689+0000 7f2a179e2640 1 --2- 172.21.15.97:0/1715556360 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a1019cfd0 0x7f2a1019d480 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.689+0000 7f2a179e2640 1 --2- 172.21.15.97:0/1715556360 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a10107270 0x7f2a10198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.689+0000 7f2a179e2640 1 --2- 172.21.15.97:0/1715556360 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a1006a8e0 0x7f2a10198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:20.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.689+0000 7f2a179e2640 1 -- 172.21.15.97:0/1715556360 >> 172.21.15.97:0/1715556360 conn(0x7f2a10077f20 msgr2=0x7f2a100fefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:29:20.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.689+0000 7f2a179e2640 1 -- 172.21.15.97:0/1715556360 shutdown_connections 2024-04-23T10:29:20.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:20.689+0000 7f2a179e2640 1 -- 172.21.15.97:0/1715556360 wait complete. 2024-04-23T10:29:20.699 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '2ee2b90', quiesce took 0.5 seconds. Will release after: 29.9 2024-04-23T10:29:21.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:29:21 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/757144103' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:29:21.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:29:21 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/757144103' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:29:21.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:29:21 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/757144103' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:29:22.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:29:22 smithi196 ceph-mon[32919]: pgmap v98: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-04-23T10:29:22.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:29:22 smithi097 ceph-mon[29197]: pgmap v98: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-04-23T10:29:22.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:29:22 smithi183 ceph-mon[32317]: pgmap v98: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-04-23T10:29:24.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:29:24 smithi196 ceph-mon[32919]: pgmap v99: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.6 KiB/s wr, 14 op/s 2024-04-23T10:29:24.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:29:24 smithi097 ceph-mon[29197]: pgmap v99: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.6 KiB/s wr, 14 op/s 2024-04-23T10:29:24.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:29:24 smithi183 ceph-mon[32317]: pgmap v99: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.6 KiB/s wr, 14 op/s 2024-04-23T10:29:26.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:29:26 smithi196 ceph-mon[32919]: pgmap v100: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 4.0 KiB/s wr, 10 op/s 2024-04-23T10:29:26.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:29:26 smithi097 ceph-mon[29197]: pgmap v100: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 4.0 KiB/s wr, 10 op/s 2024-04-23T10:29:26.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:29:26 smithi183 ceph-mon[32317]: pgmap v100: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 4.0 KiB/s wr, 10 op/s 2024-04-23T10:29:28.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:29:28 smithi196 ceph-mon[32919]: pgmap v101: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.3 KiB/s wr, 14 op/s 2024-04-23T10:29:28.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:29:28 smithi097 ceph-mon[29197]: pgmap v101: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.3 KiB/s wr, 14 op/s 2024-04-23T10:29:28.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:29:28 smithi183 ceph-mon[32317]: pgmap v101: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.3 KiB/s wr, 14 op/s 2024-04-23T10:29:30.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:29:30 smithi196 ceph-mon[32919]: pgmap v102: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 5.3 KiB/s wr, 10 op/s 2024-04-23T10:29:30.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:29:30 smithi097 ceph-mon[29197]: pgmap v102: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 5.3 KiB/s wr, 10 op/s 2024-04-23T10:29:30.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:29:30 smithi183 ceph-mon[32317]: pgmap v102: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 5.3 KiB/s wr, 10 op/s 2024-04-23T10:29:30.914 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.915+0000 7f520238a640 1 -- 172.21.15.97:0/424290059 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51fc101aa0 msgr2=0x7f51fc105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:30.914 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.915+0000 7f520238a640 1 --2- 172.21.15.97:0/424290059 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51fc101aa0 0x7f51fc105a90 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f51f0009900 tx=0x7f51f002a410 comp rx=0 tx=0).stop 2024-04-23T10:29:30.914 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.915+0000 7f520238a640 1 -- 172.21.15.97:0/424290059 shutdown_connections 2024-04-23T10:29:30.914 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.915+0000 7f520238a640 1 --2- 172.21.15.97:0/424290059 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51fc106170 0x7f51fc10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:30.914 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.915+0000 7f520238a640 1 --2- 172.21.15.97:0/424290059 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51fc101aa0 0x7f51fc105a90 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:30.915 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.915+0000 7f520238a640 1 --2- 172.21.15.97:0/424290059 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51fc1010f0 0x7f51fc1014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:30.915 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.915+0000 7f520238a640 1 -- 172.21.15.97:0/424290059 >> 172.21.15.97:0/424290059 conn(0x7f51fc077fd0 msgr2=0x7f51fc0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:29:30.915 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.915+0000 7f520238a640 1 -- 172.21.15.97:0/424290059 shutdown_connections 2024-04-23T10:29:30.915 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.915+0000 7f520238a640 1 -- 172.21.15.97:0/424290059 wait complete. 2024-04-23T10:29:30.915 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.915+0000 7f520238a640 1 Processor -- start 2024-04-23T10:29:30.915 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.915+0000 7f520238a640 1 -- start start 2024-04-23T10:29:30.917 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.916+0000 7f520238a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51fc1010f0 0x7f51fc1986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:30.917 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.916+0000 7f520238a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51fc106170 0x7f51fc198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:30.917 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.916+0000 7f520238a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51fc19cf60 0x7f51fc19d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:30.917 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.916+0000 7f520238a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f51fc111100 con 0x7f51fc1010f0 2024-04-23T10:29:30.917 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.916+0000 7f520238a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f51fc110f80 con 0x7f51fc106170 2024-04-23T10:29:30.917 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.916+0000 7f520238a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f51fc111280 con 0x7f51fc19cf60 2024-04-23T10:29:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.916+0000 7f51fbfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51fc1010f0 0x7f51fc1986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.916+0000 7f51fbfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51fc1010f0 0x7f51fc1986a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47676/0 (socket says 172.21.15.97:47676) 2024-04-23T10:29:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.916+0000 7f51fbfff640 1 -- 172.21.15.97:0/2006601768 learned_addr learned my addr 172.21.15.97:0/2006601768 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:29:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.916+0000 7f51fb7fe640 1 --2- 172.21.15.97:0/2006601768 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51fc106170 0x7f51fc198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.916+0000 7f5200927640 1 --2- 172.21.15.97:0/2006601768 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51fc19cf60 0x7f51fc19d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.916+0000 7f51fbfff640 1 -- 172.21.15.97:0/2006601768 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51fc19cf60 msgr2=0x7f51fc19d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.916+0000 7f51fbfff640 1 --2- 172.21.15.97:0/2006601768 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51fc19cf60 0x7f51fc19d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.916+0000 7f51fbfff640 1 -- 172.21.15.97:0/2006601768 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51fc106170 msgr2=0x7f51fc198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.916+0000 7f51fbfff640 1 --2- 172.21.15.97:0/2006601768 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51fc106170 0x7f51fc198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.916+0000 7f51fbfff640 1 -- 172.21.15.97:0/2006601768 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51fc19da50 con 0x7f51fc1010f0 2024-04-23T10:29:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.916+0000 7f51fb7fe640 1 --2- 172.21.15.97:0/2006601768 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51fc106170 0x7f51fc198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:29:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.917+0000 7f51fbfff640 1 --2- 172.21.15.97:0/2006601768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51fc1010f0 0x7f51fc1986a0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f51ec00c890 tx=0x7f51ec00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:29:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.917+0000 7f51f97fa640 1 -- 172.21.15.97:0/2006601768 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51ec004220 con 0x7f51fc1010f0 2024-04-23T10:29:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.917+0000 7f520238a640 1 -- 172.21.15.97:0/2006601768 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51fc19dd40 con 0x7f51fc1010f0 2024-04-23T10:29:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.917+0000 7f51f97fa640 1 -- 172.21.15.97:0/2006601768 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51ec0043c0 con 0x7f51fc1010f0 2024-04-23T10:29:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.917+0000 7f51f97fa640 1 -- 172.21.15.97:0/2006601768 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51ec004a00 con 0x7f51fc1010f0 2024-04-23T10:29:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.917+0000 7f520238a640 1 -- 172.21.15.97:0/2006601768 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51fc1a5550 con 0x7f51fc1010f0 2024-04-23T10:29:30.925 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.919+0000 7f51f97fa640 1 -- 172.21.15.97:0/2006601768 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f51ec0079b0 con 0x7f51fc1010f0 2024-04-23T10:29:30.925 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.919+0000 7f5205176740 1 Processor -- start 2024-04-23T10:29:30.925 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.919+0000 7f51f97fa640 1 --2- 172.21.15.97:0/2006601768 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f51e40a3dc0 0x7f51e40a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:30.925 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.919+0000 7f5205176740 1 -- start start 2024-04-23T10:29:30.926 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.920+0000 7f51f97fa640 1 -- 172.21.15.97:0/2006601768 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f51ec0d05b0 con 0x7f51fc1010f0 2024-04-23T10:29:30.926 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.920+0000 7f5200927640 1 --2- 172.21.15.97:0/2006601768 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f51e40a3dc0 0x7f51e40a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:30.926 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.920+0000 7f5205176740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563cea2d4620 0x563cea2d4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:30.926 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.920+0000 7f5205176740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563cea2d4f40 0x563cea2df2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:30.926 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.920+0000 7f5205176740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563cea2df800 0x563cea2e1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:30.926 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.920+0000 7f5205176740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x563cea239970 con 0x563cea2d4620 2024-04-23T10:29:30.926 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.920+0000 7f5205176740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563cea1d64a0 con 0x563cea2df800 2024-04-23T10:29:30.926 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.920+0000 7f5205176740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563cea235930 con 0x563cea2d4f40 2024-04-23T10:29:30.926 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.920+0000 7f51fbfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563cea2d4620 0x563cea2d4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:30.926 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.920+0000 7f51fbfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563cea2d4620 0x563cea2d4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47690/0 (socket says 172.21.15.97:47690) 2024-04-23T10:29:30.926 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.920+0000 7f51fbfff640 1 -- 172.21.15.97:0/2886043010 learned_addr learned my addr 172.21.15.97:0/2886043010 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:29:30.926 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.920+0000 7f5200927640 1 --2- 172.21.15.97:0/2886043010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563cea2df800 0x563cea2e1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:30.926 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.920+0000 7f51fb7fe640 1 --2- 172.21.15.97:0/2886043010 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563cea2d4f40 0x563cea2df2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:30.926 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.920+0000 7f5200927640 1 --2- 172.21.15.97:0/2006601768 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f51e40a3dc0 0x7f51e40a6280 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f51f4005fd0 tx=0x7f51f4010740 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:29:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.920+0000 7f51fbfff640 1 -- 172.21.15.97:0/2886043010 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563cea2d4f40 msgr2=0x563cea2df2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:29:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.920+0000 7f51fbfff640 1 --2- 172.21.15.97:0/2886043010 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563cea2d4f40 0x563cea2df2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.920+0000 7f51fbfff640 1 -- 172.21.15.97:0/2886043010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563cea2df800 msgr2=0x563cea2e1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:29:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.920+0000 7f51fbfff640 1 --2- 172.21.15.97:0/2886043010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563cea2df800 0x563cea2e1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.920+0000 7f51fbfff640 1 -- 172.21.15.97:0/2886043010 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ce9f4c210 con 0x563cea2d4620 2024-04-23T10:29:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.920+0000 7f5200927640 1 --2- 172.21.15.97:0/2886043010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563cea2df800 0x563cea2e1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:29:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.920+0000 7f51fbfff640 1 --2- 172.21.15.97:0/2886043010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563cea2d4620 0x563cea2d4a00 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f51ec098f70 tx=0x7f51ec005290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eb3b854d74935e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:29:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.920+0000 7f51e27fc640 1 -- 172.21.15.97:0/2886043010 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51ec01d040 con 0x563cea2d4620 2024-04-23T10:29:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.920+0000 7f51e27fc640 1 -- 172.21.15.97:0/2886043010 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51ec09c5a0 con 0x563cea2d4620 2024-04-23T10:29:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.920+0000 7f51e27fc640 1 -- 172.21.15.97:0/2886043010 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51ec098c10 con 0x563cea2d4620 2024-04-23T10:29:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.921+0000 7f5205176740 1 -- 172.21.15.97:0/2886043010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563cea2d4620 msgr2=0x563cea2d4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.921+0000 7f5205176740 1 --2- 172.21.15.97:0/2886043010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563cea2d4620 0x563cea2d4a00 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f51ec098f70 tx=0x7f51ec005290 comp rx=0 tx=0).stop 2024-04-23T10:29:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.921+0000 7f5205176740 1 -- 172.21.15.97:0/2886043010 shutdown_connections 2024-04-23T10:29:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.921+0000 7f5205176740 1 --2- 172.21.15.97:0/2886043010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563cea2df800 0x563cea2e1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.921+0000 7f5205176740 1 --2- 172.21.15.97:0/2886043010 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563cea2d4f40 0x563cea2df2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.921+0000 7f5205176740 1 --2- 172.21.15.97:0/2886043010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563cea2d4620 0x563cea2d4a00 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:30.928 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.921+0000 7f5205176740 1 -- 172.21.15.97:0/2886043010 >> 172.21.15.97:0/2886043010 conn(0x563cea2d0b30 msgr2=0x563cea2d0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:29:30.928 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.921+0000 7f5205176740 1 -- 172.21.15.97:0/2886043010 shutdown_connections 2024-04-23T10:29:30.928 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.921+0000 7f5205176740 1 -- 172.21.15.97:0/2886043010 wait complete. 2024-04-23T10:29:30.928 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.921+0000 7f5205176740 1 -- start start 2024-04-23T10:29:30.928 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.922+0000 7f5205176740 1 Processor -- start 2024-04-23T10:29:30.928 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.922+0000 7f5205176740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563cea2d4620 0x563cea2df150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:30.928 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.922+0000 7f5205176740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563cea2d4f40 0x563cea2dc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:30.928 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.922+0000 7f5205176740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563cea2df800 0x563cea2dc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:30.928 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.922+0000 7f5205176740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x563cea2e2430 con 0x563cea2d4620 2024-04-23T10:29:30.928 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.922+0000 7f5205176740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563cea1d40b0 con 0x563cea2d4f40 2024-04-23T10:29:30.928 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.922+0000 7f5205176740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563cea2e25b0 con 0x563cea2df800 2024-04-23T10:29:30.928 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.922+0000 7f5205176740 10 client.0 _ll_register_callbacks cb 0x563cea1c1d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:29:30.928 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.922+0000 7f51fbfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563cea2d4620 0x563cea2df150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:30.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.922+0000 7f51fbfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563cea2d4620 0x563cea2df150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47702/0 (socket says 172.21.15.97:47702) 2024-04-23T10:29:30.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.922+0000 7f51fbfff640 1 -- 172.21.15.97:0/2470757790 learned_addr learned my addr 172.21.15.97:0/2470757790 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:29:30.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.922+0000 7f5200927640 1 --2- 172.21.15.97:0/2470757790 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563cea2df800 0x563cea2dc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:30.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.922+0000 7f51fb7fe640 1 --2- 172.21.15.97:0/2470757790 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563cea2d4f40 0x563cea2dc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:30.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.922+0000 7f51fbfff640 1 -- 172.21.15.97:0/2470757790 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563cea2df800 msgr2=0x563cea2dc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:29:30.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.922+0000 7f51fbfff640 1 --2- 172.21.15.97:0/2470757790 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563cea2df800 0x563cea2dc9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:30.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.922+0000 7f51fbfff640 1 -- 172.21.15.97:0/2470757790 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563cea2d4f40 msgr2=0x563cea2dc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:29:30.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.922+0000 7f51fbfff640 1 --2- 172.21.15.97:0/2470757790 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563cea2d4f40 0x563cea2dc440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:30.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.922+0000 7f51fbfff640 1 -- 172.21.15.97:0/2470757790 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563cea018620 con 0x563cea2d4620 2024-04-23T10:29:30.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.923+0000 7f51fbfff640 1 --2- 172.21.15.97:0/2470757790 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563cea2d4620 0x563cea2df150 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f51ec09c530 tx=0x7f51ec0a09c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea36321eef541c57 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:29:30.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.923+0000 7f51db7fe640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:29:30.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.923+0000 7f51db7fe640 1 -- 172.21.15.97:0/2470757790 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51ec01d040 con 0x563cea2d4620 2024-04-23T10:29:30.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.923+0000 7f51db7fe640 1 -- 172.21.15.97:0/2470757790 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f51ec0bb2c0 con 0x563cea2d4620 2024-04-23T10:29:30.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.923+0000 7f5205176740 1 -- 172.21.15.97:0/2470757790 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51ec09c5a0 con 0x563cea2d4620 2024-04-23T10:29:30.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.923+0000 7f51db7fe640 1 -- 172.21.15.97:0/2470757790 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f51ec0bb4c0 con 0x563cea2d4620 2024-04-23T10:29:30.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.923+0000 7f5205176740 10 client.16119 fetch_fsmap learned FSMap version 26 2024-04-23T10:29:30.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.923+0000 7f5205176740 1 -- 172.21.15.97:0/2470757790 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563cea2ee360 con 0x563cea2d4620 2024-04-23T10:29:30.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.923+0000 7f51db7fe640 1 -- 172.21.15.97:0/2470757790 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51ec09c5a0 con 0x563cea2d4620 2024-04-23T10:29:30.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.923+0000 7f51db7fe640 1 -- 172.21.15.97:0/2470757790 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f51ec085c70 con 0x563cea2d4620 2024-04-23T10:29:30.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.923+0000 7f5205176740 10 client.16119 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:29:30.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.923+0000 7f5205176740 10 client.16119 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:29:30.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.923+0000 7f5205176740 20 client.16119 populate_metadata read hostname 'smithi097' 2024-04-23T10:29:30.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.923+0000 7f5205176740 1 --2- 172.21.15.97:0/2470757790 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x563cea2ef6b0 0x563cea2f1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:30.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.923+0000 7f5205176740 4 client.16119 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:29:30.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.923+0000 7f5205176740 1 -- 172.21.15.97:0/2470757790 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563cea247390 con 0x563cea2ef6b0 2024-04-23T10:29:30.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.924+0000 7f51fb7fe640 1 --2- 172.21.15.97:0/2470757790 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x563cea2ef6b0 0x563cea2f1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:30.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.924+0000 7f51db7fe640 10 client.16119 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:29:30.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.924+0000 7f51fb7fe640 1 --2- 172.21.15.97:0/2470757790 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x563cea2ef6b0 0x563cea2f1aa0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50c4f7ed23cf4af4 server_cookie=b48ff22dda8c51b3 in_seq=0 out_seq=0 2024-04-23T10:29:30.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.925+0000 7f51db7fe640 1 -- 172.21.15.97:0/2470757790 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563cea247390 con 0x563cea2ef6b0 2024-04-23T10:29:30.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.925+0000 7f51db7fe640 10 client.16119 handle_command_reply: tid=0 2024-04-23T10:29:30.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.925+0000 7f5205176740 1 client.16119 shutdown 2024-04-23T10:29:30.931 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.925+0000 7f5205176740 1 -- 172.21.15.97:0/2470757790 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563cea2d4620 msgr2=0x563cea2df150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:30.931 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.925+0000 7f5205176740 1 --2- 172.21.15.97:0/2470757790 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563cea2d4620 0x563cea2df150 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f51ec09c530 tx=0x7f51ec0a09c0 comp rx=0 tx=0).stop 2024-04-23T10:29:30.931 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.925+0000 7f5205176740 1 -- 172.21.15.97:0/2470757790 shutdown_connections 2024-04-23T10:29:30.931 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.925+0000 7f5205176740 1 --2- 172.21.15.97:0/2470757790 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x563cea2ef6b0 0x563cea2f1aa0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:30.931 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.925+0000 7f5205176740 1 --2- 172.21.15.97:0/2470757790 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563cea2df800 0x563cea2dc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:30.931 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.925+0000 7f5205176740 1 --2- 172.21.15.97:0/2470757790 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563cea2d4f40 0x563cea2dc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:30.931 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.925+0000 7f51db7fe640 0 client.16119 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:29:30.931 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.925+0000 7f5205176740 1 --2- 172.21.15.97:0/2470757790 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563cea2d4620 0x563cea2df150 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:30.931 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.925+0000 7f5205176740 1 -- 172.21.15.97:0/2470757790 >> 172.21.15.97:0/2470757790 conn(0x563cea2d0b30 msgr2=0x563cea2d6860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:29:30.931 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.925+0000 7f5205176740 1 -- 172.21.15.97:0/2470757790 shutdown_connections 2024-04-23T10:29:30.931 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.925+0000 7f5205176740 1 -- 172.21.15.97:0/2470757790 wait complete. 2024-04-23T10:29:30.931 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.925+0000 7f5205176740 20 client.16119 trim_cache size 0 max 16384 2024-04-23T10:29:30.943 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.940+0000 7f5205176740 1 Processor -- start 2024-04-23T10:29:30.943 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.940+0000 7f5205176740 1 -- start start 2024-04-23T10:29:30.943 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.940+0000 7f5205176740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563cea2d4620 0x563cea2eb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:30.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.940+0000 7f5205176740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563cea2d4f40 0x563cea2dec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:30.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.940+0000 7f5205176740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563cea2df800 0x563cea2f7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:30.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.940+0000 7f5205176740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f51fc1ada00 con 0x563cea2df800 2024-04-23T10:29:30.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.940+0000 7f5205176740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f51fc1af540 con 0x563cea2d4620 2024-04-23T10:29:30.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.940+0000 7f5205176740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f51fc1ae0d0 con 0x563cea2d4f40 2024-04-23T10:29:30.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.940+0000 7f5200927640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563cea2df800 0x563cea2f7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:30.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.940+0000 7f5200927640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563cea2df800 0x563cea2f7070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47704/0 (socket says 172.21.15.97:47704) 2024-04-23T10:29:30.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.940+0000 7f5200927640 1 -- 172.21.15.97:0/3532091806 learned_addr learned my addr 172.21.15.97:0/3532091806 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:29:30.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.940+0000 7f51fb7fe640 1 --2- 172.21.15.97:0/3532091806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563cea2d4f40 0x563cea2dec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:30.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.940+0000 7f5200927640 1 -- 172.21.15.97:0/3532091806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563cea2d4f40 msgr2=0x563cea2dec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:29:30.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.940+0000 7f5200927640 1 --2- 172.21.15.97:0/3532091806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563cea2d4f40 0x563cea2dec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:30.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.940+0000 7f5200927640 1 -- 172.21.15.97:0/3532091806 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563cea2d4620 msgr2=0x563cea2eb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:29:30.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.940+0000 7f5200927640 1 --2- 172.21.15.97:0/3532091806 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563cea2d4620 0x563cea2eb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:30.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.941+0000 7f5200927640 1 -- 172.21.15.97:0/3532091806 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563cea2d1470 con 0x563cea2df800 2024-04-23T10:29:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.941+0000 7f51fb7fe640 1 --2- 172.21.15.97:0/3532091806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563cea2d4f40 0x563cea2dec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:29:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.941+0000 7f5200927640 1 --2- 172.21.15.97:0/3532091806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563cea2df800 0x563cea2f7070 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f51f40125b0 tx=0x7f51f4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5659b3a8dd12961 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:29:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.941+0000 7f51e1ffb640 1 -- 172.21.15.97:0/3532091806 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f402c070 con 0x563cea2df800 2024-04-23T10:29:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.941+0000 7f51e1ffb640 1 -- 172.21.15.97:0/3532091806 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51f401fcc0 con 0x563cea2df800 2024-04-23T10:29:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.941+0000 7f51e1ffb640 1 -- 172.21.15.97:0/3532091806 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f4027d00 con 0x563cea2df800 2024-04-23T10:29:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.941+0000 7f5205176740 1 -- 172.21.15.97:0/3532091806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563cea2df800 msgr2=0x563cea2f7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.941+0000 7f5205176740 1 --2- 172.21.15.97:0/3532091806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563cea2df800 0x563cea2f7070 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f51f40125b0 tx=0x7f51f4015000 comp rx=0 tx=0).stop 2024-04-23T10:29:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.941+0000 7f5205176740 1 -- 172.21.15.97:0/3532091806 shutdown_connections 2024-04-23T10:29:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.941+0000 7f5205176740 1 --2- 172.21.15.97:0/3532091806 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563cea2df800 0x563cea2f7070 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.941+0000 7f5205176740 1 --2- 172.21.15.97:0/3532091806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563cea2d4f40 0x563cea2dec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.941+0000 7f5205176740 1 --2- 172.21.15.97:0/3532091806 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563cea2d4620 0x563cea2eb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.941+0000 7f5205176740 1 -- 172.21.15.97:0/3532091806 >> 172.21.15.97:0/3532091806 conn(0x563cea2d0b30 msgr2=0x563cea2d79d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:29:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.942+0000 7f5205176740 1 -- 172.21.15.97:0/3532091806 shutdown_connections 2024-04-23T10:29:30.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.942+0000 7f5205176740 1 -- 172.21.15.97:0/3532091806 wait complete. 2024-04-23T10:29:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.942+0000 7f5205176740 1 -- start start 2024-04-23T10:29:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.942+0000 7f5205176740 1 Processor -- start 2024-04-23T10:29:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.942+0000 7f5205176740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563cea2d4620 0x563cea2e3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.942+0000 7f5205176740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563cea2d4f40 0x563cea2e3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.942+0000 7f5205176740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563cea2df800 0x563cea2e3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.942+0000 7f5205176740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f51fc1aee70 con 0x563cea2df800 2024-04-23T10:29:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.942+0000 7f5205176740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f51fc1ae7a0 con 0x563cea2d4620 2024-04-23T10:29:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.942+0000 7f5205176740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f51fc1aba00 con 0x563cea2d4f40 2024-04-23T10:29:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.942+0000 7f5205176740 10 client.0 _ll_register_callbacks cb 0x7f51ec0cdcc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:29:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.942+0000 7f5200927640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563cea2df800 0x563cea2e3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.942+0000 7f5200927640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563cea2df800 0x563cea2e3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47720/0 (socket says 172.21.15.97:47720) 2024-04-23T10:29:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.942+0000 7f5200927640 1 -- 172.21.15.97:0/2207189995 learned_addr learned my addr 172.21.15.97:0/2207189995 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:29:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.943+0000 7f51fb7fe640 1 --2- 172.21.15.97:0/2207189995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563cea2d4f40 0x563cea2e3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.943+0000 7f51fbfff640 1 --2- 172.21.15.97:0/2207189995 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563cea2d4620 0x563cea2e3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.943+0000 7f5200927640 1 -- 172.21.15.97:0/2207189995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563cea2d4f40 msgr2=0x563cea2e3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:29:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.943+0000 7f5200927640 1 --2- 172.21.15.97:0/2207189995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563cea2d4f40 0x563cea2e3850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.943+0000 7f5200927640 1 -- 172.21.15.97:0/2207189995 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563cea2d4620 msgr2=0x563cea2e3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:29:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.943+0000 7f5200927640 1 --2- 172.21.15.97:0/2207189995 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563cea2d4620 0x563cea2e3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.943+0000 7f5200927640 1 -- 172.21.15.97:0/2207189995 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563cea354990 con 0x563cea2df800 2024-04-23T10:29:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.943+0000 7f51fbfff640 1 --2- 172.21.15.97:0/2207189995 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563cea2d4620 0x563cea2e3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:29:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.943+0000 7f5200927640 1 --2- 172.21.15.97:0/2207189995 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563cea2df800 0x563cea2e3d90 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f51f4012e40 tx=0x7f51f4027620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be4b28b577917559 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:29:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.943+0000 7f51db7fe640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:29:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.943+0000 7f51db7fe640 1 -- 172.21.15.97:0/2207189995 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f402c040 con 0x563cea2df800 2024-04-23T10:29:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.943+0000 7f51db7fe640 1 -- 172.21.15.97:0/2207189995 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f51f4004c40 con 0x563cea2df800 2024-04-23T10:29:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.943+0000 7f5205176740 1 -- 172.21.15.97:0/2207189995 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51f401fcc0 con 0x563cea2df800 2024-04-23T10:29:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.943+0000 7f51db7fe640 1 -- 172.21.15.97:0/2207189995 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f51f4004e40 con 0x563cea2df800 2024-04-23T10:29:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.943+0000 7f5205176740 10 client.16125 fetch_fsmap learned FSMap version 26 2024-04-23T10:29:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.943+0000 7f5205176740 1 -- 172.21.15.97:0/2207189995 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563cea355bf0 con 0x563cea2df800 2024-04-23T10:29:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.943+0000 7f51db7fe640 1 -- 172.21.15.97:0/2207189995 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51f401fcc0 con 0x563cea2df800 2024-04-23T10:29:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.944+0000 7f51db7fe640 1 -- 172.21.15.97:0/2207189995 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f51f4026030 con 0x563cea2df800 2024-04-23T10:29:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.944+0000 7f5205176740 10 client.16125 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:29:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.944+0000 7f5205176740 10 client.16125 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:29:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.944+0000 7f5205176740 20 client.16125 populate_metadata read hostname 'smithi097' 2024-04-23T10:29:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.944+0000 7f5205176740 1 --2- 172.21.15.97:0/2207189995 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x563cea356340 0x563cea358760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.944+0000 7f5205176740 4 client.16125 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:29:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.944+0000 7f5205176740 1 -- 172.21.15.97:0/2207189995 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563cea241200 con 0x563cea356340 2024-04-23T10:29:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.944+0000 7f51fbfff640 1 --2- 172.21.15.97:0/2207189995 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x563cea356340 0x563cea358760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.944+0000 7f51db7fe640 10 client.16125 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:29:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.944+0000 7f51fbfff640 1 --2- 172.21.15.97:0/2207189995 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x563cea356340 0x563cea358760 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ab1424b8c2b8202 server_cookie=2eeaf3a6ca9996b3 in_seq=0 out_seq=0 2024-04-23T10:29:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.952+0000 7f51db7fe640 1 -- 172.21.15.97:0/2207189995 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563cea241200 con 0x563cea356340 2024-04-23T10:29:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.952+0000 7f51db7fe640 10 client.16125 handle_command_reply: tid=0 2024-04-23T10:29:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.952+0000 7f5205176740 1 client.16125 shutdown 2024-04-23T10:29:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.952+0000 7f5205176740 1 -- 172.21.15.97:0/2207189995 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563cea2df800 msgr2=0x563cea2e3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.952+0000 7f5205176740 1 --2- 172.21.15.97:0/2207189995 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563cea2df800 0x563cea2e3d90 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f51f4012e40 tx=0x7f51f4027620 comp rx=0 tx=0).stop 2024-04-23T10:29:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.953+0000 7f5205176740 1 -- 172.21.15.97:0/2207189995 shutdown_connections 2024-04-23T10:29:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.953+0000 7f5205176740 1 --2- 172.21.15.97:0/2207189995 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x563cea356340 0x563cea358760 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.953+0000 7f5205176740 1 --2- 172.21.15.97:0/2207189995 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563cea2df800 0x563cea2e3d90 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.953+0000 7f51db7fe640 0 client.16125 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:29:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.953+0000 7f5205176740 1 --2- 172.21.15.97:0/2207189995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563cea2d4f40 0x563cea2e3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:30.952 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.953+0000 7f5205176740 1 --2- 172.21.15.97:0/2207189995 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563cea2d4620 0x563cea2e3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.953+0000 7f5205176740 1 -- 172.21.15.97:0/2207189995 >> 172.21.15.97:0/2207189995 conn(0x563cea2d0b30 msgr2=0x563cea2d5fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:29:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.953+0000 7f5205176740 1 -- 172.21.15.97:0/2207189995 shutdown_connections 2024-04-23T10:29:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.953+0000 7f5205176740 1 -- 172.21.15.97:0/2207189995 wait complete. 2024-04-23T10:29:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.953+0000 7f5205176740 20 client.16125 trim_cache size 0 max 16384 2024-04-23T10:29:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:29:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:29:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:29:30.953 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:29:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.953+0000 7f520238a640 1 -- 172.21.15.97:0/2006601768 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f51e40a3dc0 msgr2=0x7f51e40a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.953+0000 7f520238a640 1 --2- 172.21.15.97:0/2006601768 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f51e40a3dc0 0x7f51e40a6280 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f51f4005fd0 tx=0x7f51f4010740 comp rx=0 tx=0).stop 2024-04-23T10:29:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.953+0000 7f520238a640 1 -- 172.21.15.97:0/2006601768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51fc1010f0 msgr2=0x7f51fc1986a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.953+0000 7f520238a640 1 --2- 172.21.15.97:0/2006601768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51fc1010f0 0x7f51fc1986a0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f51ec00c890 tx=0x7f51ec00cd50 comp rx=0 tx=0).stop 2024-04-23T10:29:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.953+0000 7f520238a640 1 -- 172.21.15.97:0/2006601768 shutdown_connections 2024-04-23T10:29:30.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.953+0000 7f520238a640 1 --2- 172.21.15.97:0/2006601768 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f51e40a3dc0 0x7f51e40a6280 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:30.955 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.953+0000 7f520238a640 1 --2- 172.21.15.97:0/2006601768 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51fc19cf60 0x7f51fc19d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:30.955 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.953+0000 7f520238a640 1 --2- 172.21.15.97:0/2006601768 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51fc106170 0x7f51fc198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:30.955 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.953+0000 7f520238a640 1 --2- 172.21.15.97:0/2006601768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51fc1010f0 0x7f51fc1986a0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:30.955 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.953+0000 7f520238a640 1 -- 172.21.15.97:0/2006601768 >> 172.21.15.97:0/2006601768 conn(0x7f51fc077fd0 msgr2=0x7f51fc069970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:29:30.955 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.953+0000 7f520238a640 1 -- 172.21.15.97:0/2006601768 shutdown_connections 2024-04-23T10:29:30.955 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:30.953+0000 7f520238a640 1 -- 172.21.15.97:0/2006601768 wait complete. 2024-04-23T10:29:32.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:29:32 smithi196 ceph-mon[32919]: pgmap v103: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 5.3 KiB/s wr, 10 op/s 2024-04-23T10:29:32.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:29:32 smithi097 ceph-mon[29197]: pgmap v103: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 5.3 KiB/s wr, 10 op/s 2024-04-23T10:29:32.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:29:32 smithi183 ceph-mon[32317]: pgmap v103: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 5.3 KiB/s wr, 10 op/s 2024-04-23T10:29:34.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:29:34 smithi196 ceph-mon[32919]: pgmap v104: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.2 KiB/s wr, 16 op/s 2024-04-23T10:29:34.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:29:34 smithi097 ceph-mon[29197]: pgmap v104: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.2 KiB/s wr, 16 op/s 2024-04-23T10:29:34.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:29:34 smithi183 ceph-mon[32317]: pgmap v104: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.2 KiB/s wr, 16 op/s 2024-04-23T10:29:36.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:29:36 smithi196 ceph-mon[32919]: pgmap v105: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.2 KiB/s wr, 10 op/s 2024-04-23T10:29:36.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:29:36 smithi097 ceph-mon[29197]: pgmap v105: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.2 KiB/s wr, 10 op/s 2024-04-23T10:29:36.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:29:36 smithi183 ceph-mon[32317]: pgmap v105: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.2 KiB/s wr, 10 op/s 2024-04-23T10:29:38.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:29:38 smithi196 ceph-mon[32919]: pgmap v106: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 3.2 KiB/s wr, 15 op/s 2024-04-23T10:29:38.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:29:38 smithi097 ceph-mon[29197]: pgmap v106: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 3.2 KiB/s wr, 15 op/s 2024-04-23T10:29:38.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:29:38 smithi183 ceph-mon[32317]: pgmap v106: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 3.2 KiB/s wr, 15 op/s 2024-04-23T10:29:40.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:29:40 smithi196 ceph-mon[32919]: pgmap v107: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.8 KiB/s wr, 10 op/s 2024-04-23T10:29:40.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:29:40 smithi097 ceph-mon[29197]: pgmap v107: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.8 KiB/s wr, 10 op/s 2024-04-23T10:29:40.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:29:40 smithi183 ceph-mon[32317]: pgmap v107: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.8 KiB/s wr, 10 op/s 2024-04-23T10:29:42.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:29:42 smithi196 ceph-mon[32919]: pgmap v108: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.8 KiB/s wr, 10 op/s 2024-04-23T10:29:42.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:29:42 smithi097 ceph-mon[29197]: pgmap v108: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.8 KiB/s wr, 10 op/s 2024-04-23T10:29:42.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:29:42 smithi183 ceph-mon[32317]: pgmap v108: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.8 KiB/s wr, 10 op/s 2024-04-23T10:29:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.049+0000 7fedadabb640 1 -- 172.21.15.97:0/3420591114 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feda81021d0 msgr2=0x7feda81025b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.049+0000 7fedadabb640 1 --2- 172.21.15.97:0/3420591114 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feda81021d0 0x7feda81025b0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fed98009900 tx=0x7fed9802a410 comp rx=0 tx=0).stop 2024-04-23T10:29:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.049+0000 7fedadabb640 1 -- 172.21.15.97:0/3420591114 shutdown_connections 2024-04-23T10:29:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.049+0000 7fedadabb640 1 --2- 172.21.15.97:0/3420591114 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feda81072a0 0x7feda810ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.049+0000 7fedadabb640 1 --2- 172.21.15.97:0/3420591114 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feda8102b80 0x7feda8106bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.049+0000 7fedadabb640 1 --2- 172.21.15.97:0/3420591114 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feda81021d0 0x7feda81025b0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.049+0000 7fedadabb640 1 -- 172.21.15.97:0/3420591114 >> 172.21.15.97:0/3420591114 conn(0x7feda80793e0 msgr2=0x7feda80797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:29:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.049+0000 7fedadabb640 1 -- 172.21.15.97:0/3420591114 shutdown_connections 2024-04-23T10:29:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.049+0000 7fedadabb640 1 -- 172.21.15.97:0/3420591114 wait complete. 2024-04-23T10:29:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.050+0000 7fedadabb640 1 Processor -- start 2024-04-23T10:29:44.051 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.050+0000 7fedadabb640 1 -- start start 2024-04-23T10:29:44.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.050+0000 7fedadabb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feda81021d0 0x7feda8198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:44.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.050+0000 7fedadabb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feda81072a0 0x7feda8198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:44.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.050+0000 7fedadabb640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feda819cee0 0x7feda819d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:44.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.050+0000 7fedadabb640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7feda8112200 con 0x7feda81072a0 2024-04-23T10:29:44.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.050+0000 7fedadabb640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7feda8112080 con 0x7feda819cee0 2024-04-23T10:29:44.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.050+0000 7fedadabb640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7feda8112380 con 0x7feda81021d0 2024-04-23T10:29:44.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.050+0000 7feda6ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feda81072a0 0x7feda8198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:44.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.051+0000 7feda6ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feda81072a0 0x7feda8198b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46300/0 (socket says 172.21.15.97:46300) 2024-04-23T10:29:44.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.051+0000 7feda6ffd640 1 -- 172.21.15.97:0/1030317759 learned_addr learned my addr 172.21.15.97:0/1030317759 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:29:44.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.051+0000 7feda77fe640 1 --2- 172.21.15.97:0/1030317759 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feda81021d0 0x7feda8198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:44.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.051+0000 7feda7fff640 1 --2- 172.21.15.97:0/1030317759 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feda819cee0 0x7feda819d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:44.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.051+0000 7feda6ffd640 1 -- 172.21.15.97:0/1030317759 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feda81021d0 msgr2=0x7feda8198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:44.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.051+0000 7feda6ffd640 1 --2- 172.21.15.97:0/1030317759 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feda81021d0 0x7feda8198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:44.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.051+0000 7feda6ffd640 1 -- 172.21.15.97:0/1030317759 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feda819cee0 msgr2=0x7feda819d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:44.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.051+0000 7feda6ffd640 1 --2- 172.21.15.97:0/1030317759 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feda819cee0 0x7feda819d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:44.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.051+0000 7feda6ffd640 1 -- 172.21.15.97:0/1030317759 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feda819d9d0 con 0x7feda81072a0 2024-04-23T10:29:44.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.051+0000 7feda77fe640 1 --2- 172.21.15.97:0/1030317759 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feda81021d0 0x7feda8198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:29:44.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.051+0000 7feda6ffd640 1 --2- 172.21.15.97:0/1030317759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feda81072a0 0x7feda8198b60 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fed9c00c970 tx=0x7fed9c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:29:44.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.051+0000 7feda4ff9640 1 -- 172.21.15.97:0/1030317759 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed9c007bb0 con 0x7feda81072a0 2024-04-23T10:29:44.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.051+0000 7feda4ff9640 1 -- 172.21.15.97:0/1030317759 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed9c007d50 con 0x7feda81072a0 2024-04-23T10:29:44.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.051+0000 7fedadabb640 1 -- 172.21.15.97:0/1030317759 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feda819dcc0 con 0x7feda81072a0 2024-04-23T10:29:44.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.051+0000 7feda4ff9640 1 -- 172.21.15.97:0/1030317759 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed9c018730 con 0x7feda81072a0 2024-04-23T10:29:44.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.051+0000 7fedadabb640 1 -- 172.21.15.97:0/1030317759 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feda81a5460 con 0x7feda81072a0 2024-04-23T10:29:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.053+0000 7feda4ff9640 1 -- 172.21.15.97:0/1030317759 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fed9c00f990 con 0x7feda81072a0 2024-04-23T10:29:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.054+0000 7fedb07ce740 1 Processor -- start 2024-04-23T10:29:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.054+0000 7fedb07ce740 1 -- start start 2024-04-23T10:29:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.054+0000 7fedb07ce740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56300b906520 0x56300b906900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.054+0000 7feda4ff9640 1 --2- 172.21.15.97:0/1030317759 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fed900a3dc0 0x7fed900a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.054+0000 7fedb07ce740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56300b906e40 0x56300b9156d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.054+0000 7fedb07ce740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56300b915c10 0x56300b918000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.054+0000 7fedb07ce740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56300b86b970 con 0x56300b915c10 2024-04-23T10:29:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.054+0000 7fedb07ce740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56300b8084a0 con 0x56300b906520 2024-04-23T10:29:44.060 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.054+0000 7fedb07ce740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56300b867930 con 0x56300b906e40 2024-04-23T10:29:44.060 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.054+0000 7feda77fe640 1 --2- 172.21.15.97:0/1030317759 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fed900a3dc0 0x7fed900a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:44.060 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.054+0000 7feda7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56300b915c10 0x56300b918000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:44.060 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.054+0000 7feda7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56300b915c10 0x56300b918000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46304/0 (socket says 172.21.15.97:46304) 2024-04-23T10:29:44.060 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.054+0000 7feda6ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56300b906e40 0x56300b9156d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:44.060 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.054+0000 7feda7fff640 1 -- 172.21.15.97:0/3989593863 learned_addr learned my addr 172.21.15.97:0/3989593863 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:29:44.060 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.054+0000 7feda4ff9640 1 -- 172.21.15.97:0/1030317759 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fed9c0d08b0 con 0x7feda81072a0 2024-04-23T10:29:44.060 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.054+0000 7feda7fff640 1 --2- 172.21.15.97:0/3989593863 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56300b906520 0x56300b906900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:44.060 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.054+0000 7feda7fff640 1 -- 172.21.15.97:0/3989593863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56300b906e40 msgr2=0x56300b9156d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:29:44.060 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.054+0000 7feda7fff640 1 --2- 172.21.15.97:0/3989593863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56300b906e40 0x56300b9156d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:44.060 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.055+0000 7feda7fff640 1 -- 172.21.15.97:0/3989593863 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56300b906520 msgr2=0x56300b906900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:29:44.060 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.055+0000 7feda7fff640 1 --2- 172.21.15.97:0/3989593863 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56300b906520 0x56300b906900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:44.060 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.055+0000 7feda7fff640 1 -- 172.21.15.97:0/3989593863 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56300b57e210 con 0x56300b915c10 2024-04-23T10:29:44.060 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.055+0000 7feda77fe640 1 --2- 172.21.15.97:0/1030317759 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fed900a3dc0 0x7fed900a6280 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fed98031440 tx=0x7fed980095f0 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:29:44.060 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.055+0000 7feda7fff640 1 --2- 172.21.15.97:0/3989593863 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56300b915c10 0x56300b918000 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7feda0007f30 tx=0x7feda00077f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7866d152ead7bebd server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:29:44.061 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.055+0000 7fed8dffb640 1 -- 172.21.15.97:0/3989593863 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda0018030 con 0x56300b915c10 2024-04-23T10:29:44.061 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.055+0000 7fed8dffb640 1 -- 172.21.15.97:0/3989593863 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feda0012e20 con 0x56300b915c10 2024-04-23T10:29:44.061 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.055+0000 7fed8dffb640 1 -- 172.21.15.97:0/3989593863 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda00053a0 con 0x56300b915c10 2024-04-23T10:29:44.061 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.056+0000 7fedb07ce740 1 -- 172.21.15.97:0/3989593863 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56300b915c10 msgr2=0x56300b918000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:44.061 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.056+0000 7fedb07ce740 1 --2- 172.21.15.97:0/3989593863 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56300b915c10 0x56300b918000 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7feda0007f30 tx=0x7feda00077f0 comp rx=0 tx=0).stop 2024-04-23T10:29:44.061 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.056+0000 7fedb07ce740 1 -- 172.21.15.97:0/3989593863 shutdown_connections 2024-04-23T10:29:44.061 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.056+0000 7fedb07ce740 1 --2- 172.21.15.97:0/3989593863 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56300b915c10 0x56300b918000 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:44.061 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.056+0000 7fedb07ce740 1 --2- 172.21.15.97:0/3989593863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56300b906e40 0x56300b9156d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:44.061 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.056+0000 7fedb07ce740 1 --2- 172.21.15.97:0/3989593863 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56300b906520 0x56300b906900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:44.061 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.056+0000 7fedb07ce740 1 -- 172.21.15.97:0/3989593863 >> 172.21.15.97:0/3989593863 conn(0x56300b902b30 msgr2=0x56300b902f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:29:44.061 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.056+0000 7fedb07ce740 1 -- 172.21.15.97:0/3989593863 shutdown_connections 2024-04-23T10:29:44.061 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.056+0000 7fedb07ce740 1 -- 172.21.15.97:0/3989593863 wait complete. 2024-04-23T10:29:44.061 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.056+0000 7fedb07ce740 1 -- start start 2024-04-23T10:29:44.061 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.056+0000 7fedb07ce740 1 Processor -- start 2024-04-23T10:29:44.061 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.057+0000 7fedb07ce740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56300b906520 0x56300b915560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:44.061 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.057+0000 7fedb07ce740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56300b906e40 0x56300b912850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:44.062 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.057+0000 7fedb07ce740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56300b915c10 0x56300b912db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:44.062 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.057+0000 7fedb07ce740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56300b918840 con 0x56300b915c10 2024-04-23T10:29:44.062 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.057+0000 7fedb07ce740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56300b8060b0 con 0x56300b906e40 2024-04-23T10:29:44.062 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.057+0000 7fedb07ce740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56300b9189c0 con 0x56300b906520 2024-04-23T10:29:44.062 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.057+0000 7fedb07ce740 10 client.0 _ll_register_callbacks cb 0x56300b7f3d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:29:44.062 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.057+0000 7feda7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56300b915c10 0x56300b912db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:44.062 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.057+0000 7feda7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56300b915c10 0x56300b912db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46314/0 (socket says 172.21.15.97:46314) 2024-04-23T10:29:44.062 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.057+0000 7feda7fff640 1 -- 172.21.15.97:0/1408717870 learned_addr learned my addr 172.21.15.97:0/1408717870 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:29:44.062 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.057+0000 7feda77fe640 1 --2- 172.21.15.97:0/1408717870 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56300b906520 0x56300b915560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:44.062 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.057+0000 7feda6ffd640 1 --2- 172.21.15.97:0/1408717870 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56300b906e40 0x56300b912850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:44.062 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.057+0000 7feda7fff640 1 -- 172.21.15.97:0/1408717870 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56300b906520 msgr2=0x56300b915560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:29:44.062 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.057+0000 7feda7fff640 1 --2- 172.21.15.97:0/1408717870 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56300b906520 0x56300b915560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:44.062 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.057+0000 7feda7fff640 1 -- 172.21.15.97:0/1408717870 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56300b906e40 msgr2=0x56300b912850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:29:44.062 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.057+0000 7feda7fff640 1 --2- 172.21.15.97:0/1408717870 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56300b906e40 0x56300b912850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:44.062 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.057+0000 7feda7fff640 1 -- 172.21.15.97:0/1408717870 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56300b64a620 con 0x56300b915c10 2024-04-23T10:29:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.057+0000 7feda7fff640 1 --2- 172.21.15.97:0/1408717870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56300b915c10 0x56300b912db0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7feda0007430 tx=0x7feda0004c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faf44a2c8c9609ea server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:29:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.057+0000 7fed86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:29:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.057+0000 7fed86ffd640 1 -- 172.21.15.97:0/1408717870 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda0018040 con 0x56300b915c10 2024-04-23T10:29:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.057+0000 7fed86ffd640 1 -- 172.21.15.97:0/1408717870 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7feda0012040 con 0x56300b915c10 2024-04-23T10:29:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.057+0000 7fedb07ce740 1 -- 172.21.15.97:0/1408717870 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feda0012e20 con 0x56300b915c10 2024-04-23T10:29:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.057+0000 7fed86ffd640 1 -- 172.21.15.97:0/1408717870 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7feda002a020 con 0x56300b915c10 2024-04-23T10:29:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.058+0000 7fedb07ce740 10 client.16137 fetch_fsmap learned FSMap version 26 2024-04-23T10:29:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.058+0000 7fedb07ce740 1 -- 172.21.15.97:0/1408717870 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56300b9247f0 con 0x56300b915c10 2024-04-23T10:29:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.058+0000 7fed86ffd640 1 -- 172.21.15.97:0/1408717870 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feda0012e20 con 0x56300b915c10 2024-04-23T10:29:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.058+0000 7fed86ffd640 1 -- 172.21.15.97:0/1408717870 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7feda00056f0 con 0x56300b915c10 2024-04-23T10:29:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.058+0000 7fedb07ce740 10 client.16137 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:29:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.058+0000 7fedb07ce740 10 client.16137 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:29:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.058+0000 7fedb07ce740 20 client.16137 populate_metadata read hostname 'smithi097' 2024-04-23T10:29:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.058+0000 7fedb07ce740 1 --2- 172.21.15.97:0/1408717870 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x56300b925ba0 0x56300b927f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.058+0000 7fedb07ce740 4 client.16137 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:29:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.058+0000 7fedb07ce740 1 -- 172.21.15.97:0/1408717870 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56300b879390 con 0x56300b925ba0 2024-04-23T10:29:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.058+0000 7feda77fe640 1 --2- 172.21.15.97:0/1408717870 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x56300b925ba0 0x56300b927f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.059+0000 7feda77fe640 1 --2- 172.21.15.97:0/1408717870 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x56300b925ba0 0x56300b927f90 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab973da57e463f8a server_cookie=b168f3647ce58bb8 in_seq=0 out_seq=0 2024-04-23T10:29:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.059+0000 7fed86ffd640 10 client.16137 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:29:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.059+0000 7fed86ffd640 1 -- 172.21.15.97:0/1408717870 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56300b879390 con 0x56300b925ba0 2024-04-23T10:29:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.059+0000 7fed86ffd640 10 client.16137 handle_command_reply: tid=0 2024-04-23T10:29:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.059+0000 7fedb07ce740 1 client.16137 shutdown 2024-04-23T10:29:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.060+0000 7fedb07ce740 1 -- 172.21.15.97:0/1408717870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56300b915c10 msgr2=0x56300b912db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.060+0000 7fedb07ce740 1 --2- 172.21.15.97:0/1408717870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56300b915c10 0x56300b912db0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7feda0007430 tx=0x7feda0004c70 comp rx=0 tx=0).stop 2024-04-23T10:29:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.060+0000 7fedb07ce740 1 -- 172.21.15.97:0/1408717870 shutdown_connections 2024-04-23T10:29:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.060+0000 7fedb07ce740 1 --2- 172.21.15.97:0/1408717870 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x56300b925ba0 0x56300b927f90 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.060+0000 7fedb07ce740 1 --2- 172.21.15.97:0/1408717870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56300b915c10 0x56300b912db0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.060+0000 7fed86ffd640 0 client.16137 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:29:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.060+0000 7fedb07ce740 1 --2- 172.21.15.97:0/1408717870 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56300b906e40 0x56300b912850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.060+0000 7fedb07ce740 1 --2- 172.21.15.97:0/1408717870 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56300b906520 0x56300b915560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.060+0000 7fedb07ce740 1 -- 172.21.15.97:0/1408717870 >> 172.21.15.97:0/1408717870 conn(0x56300b902b30 msgr2=0x56300b9089a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:29:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.060+0000 7fedb07ce740 1 -- 172.21.15.97:0/1408717870 shutdown_connections 2024-04-23T10:29:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.060+0000 7fedb07ce740 1 -- 172.21.15.97:0/1408717870 wait complete. 2024-04-23T10:29:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.060+0000 7fedb07ce740 20 client.16137 trim_cache size 0 max 16384 2024-04-23T10:29:44.077 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.074+0000 7fedb07ce740 1 Processor -- start 2024-04-23T10:29:44.077 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.074+0000 7fedb07ce740 1 -- start start 2024-04-23T10:29:44.078 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.075+0000 7fedb07ce740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56300b906520 0x56300b921700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:44.078 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.075+0000 7fedb07ce740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56300b906e40 0x56300b901ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:44.078 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.075+0000 7fedb07ce740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56300b915c10 0x56300b92d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:44.078 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.075+0000 7fedb07ce740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7feda81ab740 con 0x56300b906e40 2024-04-23T10:29:44.078 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.075+0000 7fedb07ce740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7feda81ad280 con 0x56300b915c10 2024-04-23T10:29:44.078 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.075+0000 7fedb07ce740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7feda81abe10 con 0x56300b906520 2024-04-23T10:29:44.078 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.075+0000 7feda6ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56300b906e40 0x56300b901ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:44.078 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.075+0000 7feda6ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56300b906e40 0x56300b901ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46324/0 (socket says 172.21.15.97:46324) 2024-04-23T10:29:44.078 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.075+0000 7feda6ffd640 1 -- 172.21.15.97:0/1094398469 learned_addr learned my addr 172.21.15.97:0/1094398469 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:29:44.078 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.075+0000 7feda77fe640 1 --2- 172.21.15.97:0/1094398469 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56300b906520 0x56300b921700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:44.078 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.075+0000 7feda6ffd640 1 -- 172.21.15.97:0/1094398469 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56300b906520 msgr2=0x56300b921700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:29:44.078 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.075+0000 7feda6ffd640 1 --2- 172.21.15.97:0/1094398469 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56300b906520 0x56300b921700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:44.078 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.075+0000 7feda6ffd640 1 -- 172.21.15.97:0/1094398469 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56300b915c10 msgr2=0x56300b92d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:29:44.078 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.075+0000 7feda6ffd640 1 --2- 172.21.15.97:0/1094398469 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56300b915c10 0x56300b92d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:44.078 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.075+0000 7feda6ffd640 1 -- 172.21.15.97:0/1094398469 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56300b909010 con 0x56300b906e40 2024-04-23T10:29:44.079 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.075+0000 7feda6ffd640 1 --2- 172.21.15.97:0/1094398469 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56300b906e40 0x56300b901ef0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fed9c0ce100 tx=0x7fed9c01b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a4bfcf8dc6bddc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:29:44.079 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.075+0000 7fed8d7fa640 1 -- 172.21.15.97:0/1094398469 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed9c0a1de0 con 0x56300b906e40 2024-04-23T10:29:44.079 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.075+0000 7fed8d7fa640 1 -- 172.21.15.97:0/1094398469 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed9c09c950 con 0x56300b906e40 2024-04-23T10:29:44.079 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.075+0000 7fed8d7fa640 1 -- 172.21.15.97:0/1094398469 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed9c0a31f0 con 0x56300b906e40 2024-04-23T10:29:44.079 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.076+0000 7fedb07ce740 1 -- 172.21.15.97:0/1094398469 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56300b906e40 msgr2=0x56300b901ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:44.079 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.076+0000 7fedb07ce740 1 --2- 172.21.15.97:0/1094398469 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56300b906e40 0x56300b901ef0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fed9c0ce100 tx=0x7fed9c01b690 comp rx=0 tx=0).stop 2024-04-23T10:29:44.079 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.076+0000 7fedb07ce740 1 -- 172.21.15.97:0/1094398469 shutdown_connections 2024-04-23T10:29:44.079 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.076+0000 7fedb07ce740 1 --2- 172.21.15.97:0/1094398469 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56300b915c10 0x56300b92d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:44.079 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.076+0000 7fedb07ce740 1 --2- 172.21.15.97:0/1094398469 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56300b906e40 0x56300b901ef0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:44.079 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.076+0000 7fedb07ce740 1 --2- 172.21.15.97:0/1094398469 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56300b906520 0x56300b921700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:44.079 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.076+0000 7fedb07ce740 1 -- 172.21.15.97:0/1094398469 >> 172.21.15.97:0/1094398469 conn(0x56300b902b30 msgr2=0x56300b908900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:29:44.079 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.076+0000 7fedb07ce740 1 -- 172.21.15.97:0/1094398469 shutdown_connections 2024-04-23T10:29:44.079 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.076+0000 7fedb07ce740 1 -- 172.21.15.97:0/1094398469 wait complete. 2024-04-23T10:29:44.079 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.076+0000 7fedb07ce740 1 -- start start 2024-04-23T10:29:44.079 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.077+0000 7fedb07ce740 1 Processor -- start 2024-04-23T10:29:44.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.077+0000 7fedb07ce740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56300b906520 0x56300b91e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:44.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.077+0000 7fedb07ce740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56300b906e40 0x56300b91b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:44.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.077+0000 7fedb07ce740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56300b915c10 0x56300b91bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:44.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.077+0000 7fedb07ce740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7feda81acbb0 con 0x56300b906520 2024-04-23T10:29:44.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.077+0000 7fedb07ce740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7feda81ac4e0 con 0x56300b915c10 2024-04-23T10:29:44.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.077+0000 7fedb07ce740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7feda81a9740 con 0x56300b906e40 2024-04-23T10:29:44.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.077+0000 7fedb07ce740 10 client.0 _ll_register_callbacks cb 0x7feda0007e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:29:44.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.077+0000 7feda77fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56300b906520 0x56300b91e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:44.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.077+0000 7feda77fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56300b906520 0x56300b91e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46326/0 (socket says 172.21.15.97:46326) 2024-04-23T10:29:44.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.077+0000 7feda77fe640 1 -- 172.21.15.97:0/2721474767 learned_addr learned my addr 172.21.15.97:0/2721474767 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:29:44.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.077+0000 7feda6ffd640 1 --2- 172.21.15.97:0/2721474767 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56300b906e40 0x56300b91b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:44.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.077+0000 7feda77fe640 1 -- 172.21.15.97:0/2721474767 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56300b906e40 msgr2=0x56300b91b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:29:44.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.077+0000 7feda77fe640 1 --2- 172.21.15.97:0/2721474767 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56300b906e40 0x56300b91b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:44.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.077+0000 7feda7fff640 1 --2- 172.21.15.97:0/2721474767 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56300b915c10 0x56300b91bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:44.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.077+0000 7feda77fe640 1 -- 172.21.15.97:0/2721474767 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56300b915c10 msgr2=0x56300b91bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:29:44.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.077+0000 7feda77fe640 1 --2- 172.21.15.97:0/2721474767 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56300b915c10 0x56300b91bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:44.081 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.077+0000 7feda77fe640 1 -- 172.21.15.97:0/2721474767 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56300b98af50 con 0x56300b906520 2024-04-23T10:29:44.081 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.077+0000 7feda7fff640 1 --2- 172.21.15.97:0/2721474767 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56300b915c10 0x56300b91bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:29:44.081 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.077+0000 7feda77fe640 1 --2- 172.21.15.97:0/2721474767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56300b906520 0x56300b91e180 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fed980316f0 tx=0x7fed98009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa35c942598d0783 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:29:44.081 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.077+0000 7fed86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:29:44.081 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.077+0000 7fed86ffd640 1 -- 172.21.15.97:0/2721474767 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed98057060 con 0x56300b906520 2024-04-23T10:29:44.081 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.078+0000 7fed86ffd640 1 -- 172.21.15.97:0/2721474767 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fed98004310 con 0x56300b906520 2024-04-23T10:29:44.081 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.078+0000 7fedb07ce740 1 -- 172.21.15.97:0/2721474767 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed9c09c950 con 0x56300b906520 2024-04-23T10:29:44.081 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.078+0000 7fed86ffd640 1 -- 172.21.15.97:0/2721474767 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fed98035050 con 0x56300b906520 2024-04-23T10:29:44.081 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.078+0000 7fedb07ce740 10 client.16143 fetch_fsmap learned FSMap version 26 2024-04-23T10:29:44.081 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.078+0000 7fedb07ce740 1 -- 172.21.15.97:0/2721474767 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56300b98c160 con 0x56300b906520 2024-04-23T10:29:44.081 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.078+0000 7fed86ffd640 1 -- 172.21.15.97:0/2721474767 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed9c09c950 con 0x56300b906520 2024-04-23T10:29:44.081 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.078+0000 7fed86ffd640 1 -- 172.21.15.97:0/2721474767 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fed98051070 con 0x56300b906520 2024-04-23T10:29:44.081 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.078+0000 7fedb07ce740 10 client.16143 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:29:44.081 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.078+0000 7fedb07ce740 10 client.16143 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:29:44.081 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.078+0000 7fedb07ce740 20 client.16143 populate_metadata read hostname 'smithi097' 2024-04-23T10:29:44.082 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.078+0000 7fedb07ce740 1 --2- 172.21.15.97:0/2721474767 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x56300b98c890 0x56300b98ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:44.082 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.078+0000 7fedb07ce740 4 client.16143 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:29:44.082 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.078+0000 7fedb07ce740 1 -- 172.21.15.97:0/2721474767 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56300b873200 con 0x56300b98c890 2024-04-23T10:29:44.082 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.078+0000 7feda6ffd640 1 --2- 172.21.15.97:0/2721474767 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x56300b98c890 0x56300b98ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:44.082 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.079+0000 7fed86ffd640 10 client.16143 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:29:44.082 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.079+0000 7feda6ffd640 1 --2- 172.21.15.97:0/2721474767 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x56300b98c890 0x56300b98ecb0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32f1fbfb0de4ff1c server_cookie=9b4153bccebe566e in_seq=0 out_seq=0 2024-04-23T10:29:44.083 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.083+0000 7fed86ffd640 1 -- 172.21.15.97:0/2721474767 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56300b873200 con 0x56300b98c890 2024-04-23T10:29:44.083 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.083+0000 7fed86ffd640 10 client.16143 handle_command_reply: tid=0 2024-04-23T10:29:44.083 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.083+0000 7fedb07ce740 1 client.16143 shutdown 2024-04-23T10:29:44.083 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.083+0000 7fedb07ce740 1 -- 172.21.15.97:0/2721474767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56300b906520 msgr2=0x56300b91e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:44.083 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.083+0000 7fedb07ce740 1 --2- 172.21.15.97:0/2721474767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56300b906520 0x56300b91e180 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fed980316f0 tx=0x7fed98009260 comp rx=0 tx=0).stop 2024-04-23T10:29:44.083 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.083+0000 7fedb07ce740 1 -- 172.21.15.97:0/2721474767 shutdown_connections 2024-04-23T10:29:44.083 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.083+0000 7fedb07ce740 1 --2- 172.21.15.97:0/2721474767 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x56300b98c890 0x56300b98ecb0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:44.083 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.083+0000 7fedb07ce740 1 --2- 172.21.15.97:0/2721474767 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56300b915c10 0x56300b91bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:44.084 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.083+0000 7fed86ffd640 0 client.16143 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:29:44.084 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.083+0000 7fedb07ce740 1 --2- 172.21.15.97:0/2721474767 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56300b906e40 0x56300b91b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:44.084 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.083+0000 7fedb07ce740 1 --2- 172.21.15.97:0/2721474767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56300b906520 0x56300b91e180 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:44.084 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.083+0000 7fedb07ce740 1 -- 172.21.15.97:0/2721474767 >> 172.21.15.97:0/2721474767 conn(0x56300b902b30 msgr2=0x56300b9206f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:29:44.084 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.083+0000 7fedb07ce740 1 -- 172.21.15.97:0/2721474767 shutdown_connections 2024-04-23T10:29:44.084 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.083+0000 7fedb07ce740 1 -- 172.21.15.97:0/2721474767 wait complete. 2024-04-23T10:29:44.084 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.083+0000 7fedb07ce740 20 client.16143 trim_cache size 0 max 16384 2024-04-23T10:29:44.084 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:29:44.084 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:29:44.084 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:29:44.084 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:29:44.085 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.084+0000 7fedadabb640 1 -- 172.21.15.97:0/1030317759 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fed900a3dc0 msgr2=0x7fed900a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:44.085 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.084+0000 7fedadabb640 1 --2- 172.21.15.97:0/1030317759 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fed900a3dc0 0x7fed900a6280 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fed98031440 tx=0x7fed980095f0 comp rx=0 tx=0).stop 2024-04-23T10:29:44.085 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.084+0000 7fedadabb640 1 -- 172.21.15.97:0/1030317759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feda81072a0 msgr2=0x7feda8198b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:44.085 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.084+0000 7fedadabb640 1 --2- 172.21.15.97:0/1030317759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feda81072a0 0x7feda8198b60 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fed9c00c970 tx=0x7fed9c00ce30 comp rx=0 tx=0).stop 2024-04-23T10:29:44.085 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.084+0000 7fedadabb640 1 -- 172.21.15.97:0/1030317759 shutdown_connections 2024-04-23T10:29:44.086 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.084+0000 7fedadabb640 1 --2- 172.21.15.97:0/1030317759 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fed900a3dc0 0x7fed900a6280 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:44.086 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.084+0000 7fedadabb640 1 --2- 172.21.15.97:0/1030317759 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feda819cee0 0x7feda819d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:44.086 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.084+0000 7fedadabb640 1 --2- 172.21.15.97:0/1030317759 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feda81072a0 0x7feda8198b60 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:44.086 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.084+0000 7fedadabb640 1 --2- 172.21.15.97:0/1030317759 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feda81021d0 0x7feda8198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:44.086 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.084+0000 7fedadabb640 1 -- 172.21.15.97:0/1030317759 >> 172.21.15.97:0/1030317759 conn(0x7feda80793e0 msgr2=0x7feda80ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:29:44.086 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.084+0000 7fedadabb640 1 -- 172.21.15.97:0/1030317759 shutdown_connections 2024-04-23T10:29:44.086 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:44.084+0000 7fedadabb640 1 -- 172.21.15.97:0/1030317759 wait complete. 2024-04-23T10:29:44.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:29:44 smithi196 ceph-mon[32919]: pgmap v109: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.8 KiB/s wr, 14 op/s 2024-04-23T10:29:44.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:29:44 smithi097 ceph-mon[29197]: pgmap v109: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.8 KiB/s wr, 14 op/s 2024-04-23T10:29:44.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:29:44 smithi183 ceph-mon[32317]: pgmap v109: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.8 KiB/s wr, 14 op/s 2024-04-23T10:29:46.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:29:46 smithi196 ceph-mon[32919]: pgmap v110: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T10:29:46.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:29:46 smithi097 ceph-mon[29197]: pgmap v110: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T10:29:46.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:29:46 smithi183 ceph-mon[32317]: pgmap v110: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T10:29:48.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:29:48 smithi196 ceph-mon[32919]: pgmap v111: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T10:29:48.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:29:48 smithi097 ceph-mon[29197]: pgmap v111: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T10:29:48.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:29:48 smithi183 ceph-mon[32317]: pgmap v111: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T10:29:50.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:29:50 smithi196 ceph-mon[32919]: pgmap v112: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T10:29:50.600 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T10:29:50.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:29:50 smithi097 ceph-mon[29197]: pgmap v112: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T10:29:50.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:29:50 smithi183 ceph-mon[32317]: pgmap v112: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T10:29:50.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:50.712+0000 7f382db6f640 1 -- 172.21.15.97:0/1412311331 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3828102b40 msgr2=0x7f3828106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:50.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:50.712+0000 7f382db6f640 1 --2- 172.21.15.97:0/1412311331 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3828102b40 0x7f3828106b80 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f381c009900 tx=0x7f381c02a410 comp rx=0 tx=0).stop 2024-04-23T10:29:50.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:50.712+0000 7f382db6f640 1 -- 172.21.15.97:0/1412311331 shutdown_connections 2024-04-23T10:29:50.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:50.712+0000 7f382db6f640 1 --2- 172.21.15.97:0/1412311331 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3828107260 0x7f382810ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:50.711 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:50.712+0000 7f382db6f640 1 --2- 172.21.15.97:0/1412311331 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3828102b40 0x7f3828106b80 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:50.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:50.712+0000 7f382db6f640 1 --2- 172.21.15.97:0/1412311331 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3828102190 0x7f3828102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:50.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:50.712+0000 7f382db6f640 1 -- 172.21.15.97:0/1412311331 >> 172.21.15.97:0/1412311331 conn(0x7f3828079310 msgr2=0x7f3828079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:29:50.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:50.712+0000 7f382db6f640 1 -- 172.21.15.97:0/1412311331 shutdown_connections 2024-04-23T10:29:50.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:50.712+0000 7f382db6f640 1 -- 172.21.15.97:0/1412311331 wait complete. 2024-04-23T10:29:50.712 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:50.713+0000 7f382db6f640 1 Processor -- start 2024-04-23T10:29:50.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:50.713+0000 7f382db6f640 1 -- start start 2024-04-23T10:29:50.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:50.713+0000 7f382db6f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3828102190 0x7f3828198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:50.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:50.713+0000 7f382db6f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3828107260 0x7f3828198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:50.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:50.713+0000 7f382db6f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f382819d020 0x7f382819d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:50.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:50.713+0000 7f382db6f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f38281121a0 con 0x7f3828107260 2024-04-23T10:29:50.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:50.713+0000 7f382db6f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3828112020 con 0x7f382819d020 2024-04-23T10:29:50.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:50.713+0000 7f382db6f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3828112320 con 0x7f3828102190 2024-04-23T10:29:50.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:50.713+0000 7f3826ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3828107260 0x7f3828198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:50.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:50.713+0000 7f3826ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3828107260 0x7f3828198ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55610/0 (socket says 172.21.15.97:55610) 2024-04-23T10:29:50.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:50.713+0000 7f3826ffd640 1 -- 172.21.15.97:0/2170491711 learned_addr learned my addr 172.21.15.97:0/2170491711 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:29:50.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:50.713+0000 7f38277fe640 1 --2- 172.21.15.97:0/2170491711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3828102190 0x7f3828198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:50.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:50.713+0000 7f3826ffd640 1 -- 172.21.15.97:0/2170491711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3828102190 msgr2=0x7f3828198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:50.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:50.713+0000 7f3826ffd640 1 --2- 172.21.15.97:0/2170491711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3828102190 0x7f3828198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:50.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:50.713+0000 7f3826ffd640 1 -- 172.21.15.97:0/2170491711 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f382819d020 msgr2=0x7f382819d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:50.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:50.713+0000 7f3827fff640 1 --2- 172.21.15.97:0/2170491711 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f382819d020 0x7f382819d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:50.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:50.713+0000 7f3826ffd640 1 --2- 172.21.15.97:0/2170491711 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f382819d020 0x7f382819d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:50.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:50.713+0000 7f3826ffd640 1 -- 172.21.15.97:0/2170491711 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f382819db10 con 0x7f3828107260 2024-04-23T10:29:50.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:50.714+0000 7f3826ffd640 1 --2- 172.21.15.97:0/2170491711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3828107260 0x7f3828198ca0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f381c032fd0 tx=0x7f381c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:29:50.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:50.714+0000 7f3824ff9640 1 -- 172.21.15.97:0/2170491711 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f381c03f030 con 0x7f3828107260 2024-04-23T10:29:50.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:50.714+0000 7f3824ff9640 1 -- 172.21.15.97:0/2170491711 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f381c004490 con 0x7f3828107260 2024-04-23T10:29:50.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:50.714+0000 7f382db6f640 1 -- 172.21.15.97:0/2170491711 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f382819dda0 con 0x7f3828107260 2024-04-23T10:29:50.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:50.714+0000 7f3824ff9640 1 -- 172.21.15.97:0/2170491711 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f381c035070 con 0x7f3828107260 2024-04-23T10:29:50.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:50.714+0000 7f382db6f640 1 -- 172.21.15.97:0/2170491711 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38281a5560 con 0x7f3828107260 2024-04-23T10:29:50.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:50.715+0000 7f382db6f640 1 -- 172.21.15.97:0/2170491711 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38280691c0 con 0x7f3828107260 2024-04-23T10:29:50.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:50.716+0000 7f3824ff9640 1 -- 172.21.15.97:0/2170491711 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f381c042030 con 0x7f3828107260 2024-04-23T10:29:50.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:50.717+0000 7f3824ff9640 1 --2- 172.21.15.97:0/2170491711 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f38100a3d20 0x7f38100a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:50.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:50.717+0000 7f38277fe640 1 --2- 172.21.15.97:0/2170491711 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f38100a3d20 0x7f38100a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:50.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:50.717+0000 7f3824ff9640 1 -- 172.21.15.97:0/2170491711 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f381c0f0cc0 con 0x7f3828107260 2024-04-23T10:29:50.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:50.717+0000 7f38277fe640 1 --2- 172.21.15.97:0/2170491711 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f38100a3d20 0x7f38100a61e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3818005fd0 tx=0x7f381800b390 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:29:50.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:50.721+0000 7f3824ff9640 1 -- 172.21.15.97:0/2170491711 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f381c0bd3b0 con 0x7f3828107260 2024-04-23T10:29:50.936 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:50.937+0000 7f382db6f640 1 -- 172.21.15.97:0/2170491711 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3828102b40 con 0x7f3828107260 2024-04-23T10:29:50.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:50.938+0000 7f3824ff9640 1 -- 172.21.15.97:0/2170491711 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11931 (secure 0 0 0) 0x7f381c0bccb0 con 0x7f3828107260 2024-04-23T10:29:50.937 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:29:50.937 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3946878762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3946878762},{"type":"v1","addr":"172.21.15.97:6835","nonce":3946878762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/209800861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":209800861},{"type":"v1","addr":"172.21.15.97:6837","nonce":209800861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T10:24:06.910134+0000","modified":"2024-04-23T10:26:09.820985+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6839/418013575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":418013575},{"type":"v1","addr":"172.21.15.97:6839","nonce":418013575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6841/747242680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":747242680},{"type":"v1","addr":"172.21.15.97:6841","nonce":747242680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6833/4292847121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4292847121},{"type":"v1","addr":"172.21.15.183:6833","nonce":4292847121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6833/3885816565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3885816565},{"type":"v1","addr":"172.21.15.196:6833","nonce":3885816565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1897093105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1897093105},{"type":"v1","addr":"172.21.15.183:6835","nonce":1897093105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1171477563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1171477563},{"type":"v1","addr":"172.21.15.196:6835","nonce":1171477563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1898341911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1898341911},{"type":"v1","addr":"172.21.15.183:6837","nonce":1898341911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.196:6837/3094946502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3094946502},{"type":"v1","addr":"172.21.15.196:6837","nonce":3094946502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.196:6839/83843556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":83843556},{"type":"v1","addr":"172.21.15.196:6839","nonce":83843556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6839/1763841130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1763841130},{"type":"v1","addr":"172.21.15.183:6839","nonce":1763841130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24487,24403,24398,24469,24488]},"id":1}]} 2024-04-23T10:29:50.938 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 26 2024-04-23T10:29:50.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:50.940+0000 7f382db6f640 1 -- 172.21.15.97:0/2170491711 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f38100a3d20 msgr2=0x7f38100a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:50.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:50.940+0000 7f382db6f640 1 --2- 172.21.15.97:0/2170491711 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f38100a3d20 0x7f38100a61e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3818005fd0 tx=0x7f381800b390 comp rx=0 tx=0).stop 2024-04-23T10:29:50.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:50.940+0000 7f382db6f640 1 -- 172.21.15.97:0/2170491711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3828107260 msgr2=0x7f3828198ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:50.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:50.940+0000 7f382db6f640 1 --2- 172.21.15.97:0/2170491711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3828107260 0x7f3828198ca0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f381c032fd0 tx=0x7f381c004270 comp rx=0 tx=0).stop 2024-04-23T10:29:50.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:50.940+0000 7f382db6f640 1 -- 172.21.15.97:0/2170491711 shutdown_connections 2024-04-23T10:29:50.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:50.940+0000 7f382db6f640 1 --2- 172.21.15.97:0/2170491711 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f38100a3d20 0x7f38100a61e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:50.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:50.940+0000 7f382db6f640 1 --2- 172.21.15.97:0/2170491711 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f382819d020 0x7f382819d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:50.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:50.940+0000 7f382db6f640 1 --2- 172.21.15.97:0/2170491711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3828107260 0x7f3828198ca0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:50.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:50.940+0000 7f382db6f640 1 --2- 172.21.15.97:0/2170491711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3828102190 0x7f3828198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:50.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:50.940+0000 7f382db6f640 1 -- 172.21.15.97:0/2170491711 >> 172.21.15.97:0/2170491711 conn(0x7f3828079310 msgr2=0x7f38281002e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:29:50.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:50.940+0000 7f382db6f640 1 -- 172.21.15.97:0/2170491711 shutdown_connections 2024-04-23T10:29:50.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:50.940+0000 7f382db6f640 1 -- 172.21.15.97:0/2170491711 wait complete. 2024-04-23T10:29:50.950 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --set-id 2ee2b90 --release --await' 2024-04-23T10:29:50.950 DEBUG:teuthology.orchestra.run.smithi097:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24488 quiesce db --set-id 2ee2b90 --release --await 2024-04-23T10:29:51.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.100+0000 7f434575d640 1 -- 172.21.15.97:0/401172643 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4340107260 msgr2=0x7f434010ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:51.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.100+0000 7f434575d640 1 --2- 172.21.15.97:0/401172643 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4340107260 0x7f434010ee30 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f4338009ca0 tx=0x7f433802a8c0 comp rx=0 tx=0).stop 2024-04-23T10:29:51.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.100+0000 7f434575d640 1 -- 172.21.15.97:0/401172643 shutdown_connections 2024-04-23T10:29:51.099 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.100+0000 7f434575d640 1 --2- 172.21.15.97:0/401172643 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4340107260 0x7f434010ee30 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:51.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.100+0000 7f434575d640 1 --2- 172.21.15.97:0/401172643 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4340102b40 0x7f4340106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:51.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.100+0000 7f434575d640 1 --2- 172.21.15.97:0/401172643 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4340102190 0x7f4340102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:51.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.100+0000 7f434575d640 1 -- 172.21.15.97:0/401172643 >> 172.21.15.97:0/401172643 conn(0x7f43400793a0 msgr2=0x7f43400797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:29:51.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.100+0000 7f434575d640 1 -- 172.21.15.97:0/401172643 shutdown_connections 2024-04-23T10:29:51.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.100+0000 7f434575d640 1 -- 172.21.15.97:0/401172643 wait complete. 2024-04-23T10:29:51.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.101+0000 7f434575d640 1 Processor -- start 2024-04-23T10:29:51.100 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.101+0000 7f434575d640 1 -- start start 2024-04-23T10:29:51.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.101+0000 7f434575d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4340102190 0x7f4340198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:51.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.101+0000 7f434575d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4340107260 0x7f4340198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:51.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.101+0000 7f434575d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f434019d010 0x7f434019d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:51.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.101+0000 7f434575d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f43401121d0 con 0x7f4340102190 2024-04-23T10:29:51.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.101+0000 7f434575d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4340112050 con 0x7f434019d010 2024-04-23T10:29:51.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.101+0000 7f434575d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4340112350 con 0x7f4340107260 2024-04-23T10:29:51.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.101+0000 7f433effd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4340102190 0x7f4340198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:51.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.101+0000 7f433effd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4340102190 0x7f4340198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55626/0 (socket says 172.21.15.97:55626) 2024-04-23T10:29:51.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.101+0000 7f433effd640 1 -- 172.21.15.97:0/3465597999 learned_addr learned my addr 172.21.15.97:0/3465597999 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:29:51.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.101+0000 7f433e7fc640 1 --2- 172.21.15.97:0/3465597999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4340107260 0x7f4340198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:51.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.101+0000 7f433f7fe640 1 --2- 172.21.15.97:0/3465597999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f434019d010 0x7f434019d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:51.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.101+0000 7f433effd640 1 -- 172.21.15.97:0/3465597999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4340107260 msgr2=0x7f4340198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:51.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.101+0000 7f433effd640 1 --2- 172.21.15.97:0/3465597999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4340107260 0x7f4340198c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:51.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.101+0000 7f433effd640 1 -- 172.21.15.97:0/3465597999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f434019d010 msgr2=0x7f434019d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:51.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.101+0000 7f433effd640 1 --2- 172.21.15.97:0/3465597999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f434019d010 0x7f434019d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:51.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.101+0000 7f433effd640 1 -- 172.21.15.97:0/3465597999 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f434019db00 con 0x7f4340102190 2024-04-23T10:29:51.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.101+0000 7f433f7fe640 1 --2- 172.21.15.97:0/3465597999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f434019d010 0x7f434019d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:29:51.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.101+0000 7f433effd640 1 --2- 172.21.15.97:0/3465597999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4340102190 0x7f4340198750 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f433000d950 tx=0x7f433000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:29:51.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.102+0000 7f4327fff640 1 -- 172.21.15.97:0/3465597999 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4330017070 con 0x7f4340102190 2024-04-23T10:29:51.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.102+0000 7f434575d640 1 -- 172.21.15.97:0/3465597999 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f434019ddf0 con 0x7f4340102190 2024-04-23T10:29:51.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.102+0000 7f4327fff640 1 -- 172.21.15.97:0/3465597999 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4330009c30 con 0x7f4340102190 2024-04-23T10:29:51.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.102+0000 7f4327fff640 1 -- 172.21.15.97:0/3465597999 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433000f620 con 0x7f4340102190 2024-04-23T10:29:51.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.102+0000 7f434575d640 1 -- 172.21.15.97:0/3465597999 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43401a55e0 con 0x7f4340102190 2024-04-23T10:29:51.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.104+0000 7f4327fff640 1 -- 172.21.15.97:0/3465597999 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4330010460 con 0x7f4340102190 2024-04-23T10:29:51.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.104+0000 7f434847c740 1 Processor -- start 2024-04-23T10:29:51.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.104+0000 7f434847c740 1 -- start start 2024-04-23T10:29:51.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.104+0000 7f434847c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a6b86a6490 0x55a6b86a6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:51.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.104+0000 7f434847c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6b86a6db0 0x55a6b86b0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:51.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.104+0000 7f434847c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6b86b1480 0x55a6b86b3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:51.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.104+0000 7f434847c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55a6b860b6d0 con 0x55a6b86a6490 2024-04-23T10:29:51.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.104+0000 7f434847c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a6b85a81a0 con 0x55a6b86a6db0 2024-04-23T10:29:51.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.104+0000 7f434847c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a6b8607690 con 0x55a6b86b1480 2024-04-23T10:29:51.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.104+0000 7f433f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a6b86a6490 0x55a6b86a6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:51.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.104+0000 7f4327fff640 1 --2- 172.21.15.97:0/3465597999 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f43280a3d20 0x7f43280a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:51.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.104+0000 7f433f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a6b86a6490 0x55a6b86a6870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55636/0 (socket says 172.21.15.97:55636) 2024-04-23T10:29:51.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.104+0000 7f433f7fe640 1 -- 172.21.15.97:0/2544615681 learned_addr learned my addr 172.21.15.97:0/2544615681 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:29:51.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.104+0000 7f433effd640 1 --2- 172.21.15.97:0/2544615681 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6b86a6db0 0x55a6b86b0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:51.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.104+0000 7f433f7fe640 1 --2- 172.21.15.97:0/3465597999 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f43280a3d20 0x7f43280a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:51.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.104+0000 7f433e7fc640 1 --2- 172.21.15.97:0/2544615681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6b86b1480 0x55a6b86b3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:51.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.105+0000 7f4327fff640 1 -- 172.21.15.97:0/3465597999 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f43300d04a0 con 0x7f4340102190 2024-04-23T10:29:51.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.105+0000 7f433f7fe640 1 -- 172.21.15.97:0/2544615681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6b86b1480 msgr2=0x55a6b86b3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:29:51.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.105+0000 7f433f7fe640 1 --2- 172.21.15.97:0/2544615681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6b86b1480 0x55a6b86b3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:51.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.105+0000 7f433f7fe640 1 -- 172.21.15.97:0/2544615681 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6b86a6db0 msgr2=0x55a6b86b0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:29:51.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.105+0000 7f433f7fe640 1 --2- 172.21.15.97:0/2544615681 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6b86a6db0 0x55a6b86b0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:51.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.105+0000 7f433f7fe640 1 -- 172.21.15.97:0/2544615681 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6b85b25b0 con 0x55a6b86a6490 2024-04-23T10:29:51.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.105+0000 7f433effd640 1 --2- 172.21.15.97:0/2544615681 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6b86a6db0 0x55a6b86b0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:29:51.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.105+0000 7f433f7fe640 1 --2- 172.21.15.97:0/2544615681 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a6b86a6490 0x55a6b86a6870 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f4338031770 tx=0x7f43380317a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3a7245bf5ee5c10 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:29:51.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.105+0000 7f433f7fe640 1 --2- 172.21.15.97:0/3465597999 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f43280a3d20 0x7f43280a61e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4338031810 tx=0x7f4338047000 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:29:51.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.105+0000 7f43257fa640 1 -- 172.21.15.97:0/2544615681 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433802af40 con 0x55a6b86a6490 2024-04-23T10:29:51.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.105+0000 7f43257fa640 1 -- 172.21.15.97:0/2544615681 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4338036070 con 0x55a6b86a6490 2024-04-23T10:29:51.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.105+0000 7f43257fa640 1 -- 172.21.15.97:0/2544615681 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43380305c0 con 0x55a6b86a6490 2024-04-23T10:29:51.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.106+0000 7f434847c740 1 -- 172.21.15.97:0/2544615681 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a6b86a6490 msgr2=0x55a6b86a6870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:51.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.106+0000 7f434847c740 1 --2- 172.21.15.97:0/2544615681 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a6b86a6490 0x55a6b86a6870 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f4338031770 tx=0x7f43380317a0 comp rx=0 tx=0).stop 2024-04-23T10:29:51.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.106+0000 7f434847c740 1 -- 172.21.15.97:0/2544615681 shutdown_connections 2024-04-23T10:29:51.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.106+0000 7f434847c740 1 --2- 172.21.15.97:0/2544615681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6b86b1480 0x55a6b86b3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:51.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.106+0000 7f434847c740 1 --2- 172.21.15.97:0/2544615681 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6b86a6db0 0x55a6b86b0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:51.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.106+0000 7f434847c740 1 --2- 172.21.15.97:0/2544615681 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a6b86a6490 0x55a6b86a6870 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:51.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.106+0000 7f434847c740 1 -- 172.21.15.97:0/2544615681 >> 172.21.15.97:0/2544615681 conn(0x55a6b86a27b0 msgr2=0x55a6b86a2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:29:51.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.106+0000 7f434847c740 1 -- 172.21.15.97:0/2544615681 shutdown_connections 2024-04-23T10:29:51.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.106+0000 7f434847c740 1 -- 172.21.15.97:0/2544615681 wait complete. 2024-04-23T10:29:51.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.106+0000 7f434847c740 1 -- start start 2024-04-23T10:29:51.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.106+0000 7f434847c740 1 Processor -- start 2024-04-23T10:29:51.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.106+0000 7f434847c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6b86a6490 0x55a6b86b0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:51.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.106+0000 7f434847c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a6b86a6db0 0x55a6b86ae1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:51.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.106+0000 7f434847c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6b86b1480 0x55a6b86ae930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:51.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.106+0000 7f434847c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55a6b86b42b0 con 0x55a6b86a6db0 2024-04-23T10:29:51.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.106+0000 7f434847c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a6b85a5b20 con 0x55a6b86a6490 2024-04-23T10:29:51.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.106+0000 7f434847c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a6b86b4430 con 0x55a6b86b1480 2024-04-23T10:29:51.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.106+0000 7f434847c740 10 client.0 _ll_register_callbacks cb 0x55a6b85a5530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:29:51.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.106+0000 7f433e7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a6b86a6db0 0x55a6b86ae1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:51.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.107+0000 7f433e7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a6b86a6db0 0x55a6b86ae1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55646/0 (socket says 172.21.15.97:55646) 2024-04-23T10:29:51.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.107+0000 7f433e7fc640 1 -- 172.21.15.97:0/414207390 learned_addr learned my addr 172.21.15.97:0/414207390 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:29:51.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.107+0000 7f433f7fe640 1 --2- 172.21.15.97:0/414207390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6b86b1480 0x55a6b86ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:51.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.107+0000 7f433effd640 1 --2- 172.21.15.97:0/414207390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6b86a6490 0x55a6b86b0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:51.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.107+0000 7f433e7fc640 1 -- 172.21.15.97:0/414207390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6b86b1480 msgr2=0x55a6b86ae930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:29:51.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.107+0000 7f433e7fc640 1 --2- 172.21.15.97:0/414207390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6b86b1480 0x55a6b86ae930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:51.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.107+0000 7f433e7fc640 1 -- 172.21.15.97:0/414207390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6b86a6490 msgr2=0x55a6b86b0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:29:51.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.107+0000 7f433e7fc640 1 --2- 172.21.15.97:0/414207390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6b86a6490 0x55a6b86b0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:51.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.107+0000 7f433e7fc640 1 -- 172.21.15.97:0/414207390 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6b831d920 con 0x55a6b86a6db0 2024-04-23T10:29:51.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.107+0000 7f433effd640 1 --2- 172.21.15.97:0/414207390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6b86a6490 0x55a6b86b0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:29:51.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.107+0000 7f433e7fc640 1 --2- 172.21.15.97:0/414207390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a6b86a6db0 0x55a6b86ae1f0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f4334013460 tx=0x7f4334013920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fea66d17c5f2a848 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:29:51.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.107+0000 7f4322ff5640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:29:51.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.107+0000 7f4322ff5640 1 -- 172.21.15.97:0/414207390 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4334004240 con 0x55a6b86a6db0 2024-04-23T10:29:51.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.107+0000 7f4322ff5640 1 -- 172.21.15.97:0/414207390 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f43340043e0 con 0x55a6b86a6db0 2024-04-23T10:29:51.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.107+0000 7f434847c740 1 -- 172.21.15.97:0/414207390 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4338036070 con 0x55a6b86a6db0 2024-04-23T10:29:51.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.107+0000 7f4322ff5640 1 -- 172.21.15.97:0/414207390 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f43340035f0 con 0x55a6b86a6db0 2024-04-23T10:29:51.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.107+0000 7f434847c740 10 client.16161 fetch_fsmap learned FSMap version 26 2024-04-23T10:29:51.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.107+0000 7f434847c740 1 -- 172.21.15.97:0/414207390 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a6b83e9d30 con 0x55a6b86a6db0 2024-04-23T10:29:51.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.107+0000 7f4322ff5640 1 -- 172.21.15.97:0/414207390 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4338036070 con 0x55a6b86a6db0 2024-04-23T10:29:51.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.107+0000 7f4322ff5640 1 -- 172.21.15.97:0/414207390 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f433400f040 con 0x55a6b86a6db0 2024-04-23T10:29:51.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.108+0000 7f434847c740 10 client.16161 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:29:51.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.108+0000 7f434847c740 10 client.16161 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:29:51.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.108+0000 7f434847c740 20 client.16161 populate_metadata read hostname 'smithi097' 2024-04-23T10:29:51.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.108+0000 7f434847c740 1 --2- 172.21.15.97:0/414207390 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55a6b86c16e0 0x55a6b86c3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:51.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.108+0000 7f434847c740 4 client.16161 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:29:51.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.108+0000 7f434847c740 1 -- 172.21.15.97:0/414207390 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6b85c9500 con 0x55a6b86c16e0 2024-04-23T10:29:51.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.108+0000 7f433effd640 1 --2- 172.21.15.97:0/414207390 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55a6b86c16e0 0x55a6b86c3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:51.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.108+0000 7f433effd640 1 --2- 172.21.15.97:0/414207390 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55a6b86c16e0 0x55a6b86c3b30 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e862054b5519b01 server_cookie=42909c083da67d30 in_seq=0 out_seq=0 2024-04-23T10:29:51.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.108+0000 7f4322ff5640 10 client.16161 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:29:51.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.109+0000 7f4322ff5640 1 -- 172.21.15.97:0/414207390 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a6b85c9500 con 0x55a6b86c16e0 2024-04-23T10:29:51.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.109+0000 7f4322ff5640 10 client.16161 handle_command_reply: tid=0 2024-04-23T10:29:51.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.109+0000 7f434847c740 1 client.16161 shutdown 2024-04-23T10:29:51.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.109+0000 7f434847c740 1 -- 172.21.15.97:0/414207390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a6b86a6db0 msgr2=0x55a6b86ae1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:51.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.109+0000 7f434847c740 1 --2- 172.21.15.97:0/414207390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a6b86a6db0 0x55a6b86ae1f0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f4334013460 tx=0x7f4334013920 comp rx=0 tx=0).stop 2024-04-23T10:29:51.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.109+0000 7f434847c740 1 -- 172.21.15.97:0/414207390 shutdown_connections 2024-04-23T10:29:51.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.109+0000 7f434847c740 1 --2- 172.21.15.97:0/414207390 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55a6b86c16e0 0x55a6b86c3b30 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:51.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.109+0000 7f434847c740 1 --2- 172.21.15.97:0/414207390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6b86b1480 0x55a6b86ae930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:51.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.109+0000 7f434847c740 1 --2- 172.21.15.97:0/414207390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a6b86a6db0 0x55a6b86ae1f0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:51.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.109+0000 7f434847c740 1 --2- 172.21.15.97:0/414207390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6b86a6490 0x55a6b86b0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:51.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.109+0000 7f4322ff5640 0 client.16161 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:29:51.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.109+0000 7f434847c740 1 -- 172.21.15.97:0/414207390 >> 172.21.15.97:0/414207390 conn(0x55a6b86a27b0 msgr2=0x55a6b86a85b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:29:51.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.109+0000 7f434847c740 1 -- 172.21.15.97:0/414207390 shutdown_connections 2024-04-23T10:29:51.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.109+0000 7f434847c740 1 -- 172.21.15.97:0/414207390 wait complete. 2024-04-23T10:29:51.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.109+0000 7f434847c740 20 client.16161 trim_cache size 0 max 16384 2024-04-23T10:29:51.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.127+0000 7f434847c740 1 Processor -- start 2024-04-23T10:29:51.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.127+0000 7f434847c740 1 -- start start 2024-04-23T10:29:51.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.127+0000 7f434847c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a6b86a6490 0x55a6b86ad130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:51.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.127+0000 7f434847c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6b86a6db0 0x55a6b86b0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:51.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.127+0000 7f434847c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6b86b1480 0x55a6b86c9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:51.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.127+0000 7f434847c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f43401adad0 con 0x55a6b86a6490 2024-04-23T10:29:51.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.127+0000 7f434847c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f43401af610 con 0x55a6b86a6db0 2024-04-23T10:29:51.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.127+0000 7f434847c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f43401ae1a0 con 0x55a6b86b1480 2024-04-23T10:29:51.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.127+0000 7f433effd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a6b86a6490 0x55a6b86ad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:51.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.127+0000 7f433effd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a6b86a6490 0x55a6b86ad130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55660/0 (socket says 172.21.15.97:55660) 2024-04-23T10:29:51.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.127+0000 7f433effd640 1 -- 172.21.15.97:0/3480874422 learned_addr learned my addr 172.21.15.97:0/3480874422 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:29:51.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.127+0000 7f433effd640 1 -- 172.21.15.97:0/3480874422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6b86b1480 msgr2=0x55a6b86c9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:29:51.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.127+0000 7f433effd640 1 --2- 172.21.15.97:0/3480874422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6b86b1480 0x55a6b86c9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:51.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.127+0000 7f433effd640 1 -- 172.21.15.97:0/3480874422 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6b86a6db0 msgr2=0x55a6b86b0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:29:51.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.127+0000 7f433effd640 1 --2- 172.21.15.97:0/3480874422 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6b86a6db0 0x55a6b86b0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:51.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.127+0000 7f433effd640 1 -- 172.21.15.97:0/3480874422 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6b86aeca0 con 0x55a6b86a6490 2024-04-23T10:29:51.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.127+0000 7f433effd640 1 --2- 172.21.15.97:0/3480874422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a6b86a6490 0x55a6b86ad130 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f433009c420 tx=0x7f4330016500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8072f69ff573c87 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:29:51.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.127+0000 7f4324ff9640 1 -- 172.21.15.97:0/3480874422 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4330020040 con 0x55a6b86a6490 2024-04-23T10:29:51.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.128+0000 7f4324ff9640 1 -- 172.21.15.97:0/3480874422 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f433009c490 con 0x55a6b86a6490 2024-04-23T10:29:51.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.128+0000 7f4324ff9640 1 -- 172.21.15.97:0/3480874422 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4330016170 con 0x55a6b86a6490 2024-04-23T10:29:51.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.128+0000 7f434847c740 1 -- 172.21.15.97:0/3480874422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a6b86a6490 msgr2=0x55a6b86ad130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:51.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.128+0000 7f434847c740 1 --2- 172.21.15.97:0/3480874422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a6b86a6490 0x55a6b86ad130 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f433009c420 tx=0x7f4330016500 comp rx=0 tx=0).stop 2024-04-23T10:29:51.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.128+0000 7f434847c740 1 -- 172.21.15.97:0/3480874422 shutdown_connections 2024-04-23T10:29:51.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.128+0000 7f434847c740 1 --2- 172.21.15.97:0/3480874422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6b86b1480 0x55a6b86c9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:51.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.128+0000 7f434847c740 1 --2- 172.21.15.97:0/3480874422 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6b86a6db0 0x55a6b86b0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:51.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.128+0000 7f434847c740 1 --2- 172.21.15.97:0/3480874422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a6b86a6490 0x55a6b86ad130 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:51.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.128+0000 7f434847c740 1 -- 172.21.15.97:0/3480874422 >> 172.21.15.97:0/3480874422 conn(0x55a6b86a27b0 msgr2=0x55a6b86a19d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:29:51.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.128+0000 7f434847c740 1 -- 172.21.15.97:0/3480874422 shutdown_connections 2024-04-23T10:29:51.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.128+0000 7f434847c740 1 -- 172.21.15.97:0/3480874422 wait complete. 2024-04-23T10:29:51.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.128+0000 7f434847c740 1 -- start start 2024-04-23T10:29:51.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.128+0000 7f434847c740 1 Processor -- start 2024-04-23T10:29:51.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.129+0000 7f434847c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6b86a6490 0x55a6b86b9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:51.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.129+0000 7f434847c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a6b86a6db0 0x55a6b86b9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:51.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.129+0000 7f434847c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6b86b1480 0x55a6b86aa460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:51.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.129+0000 7f434847c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f43401aef40 con 0x55a6b86a6db0 2024-04-23T10:29:51.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.129+0000 7f434847c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f43401ae870 con 0x55a6b86b1480 2024-04-23T10:29:51.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.129+0000 7f434847c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f43401abad0 con 0x55a6b86a6490 2024-04-23T10:29:51.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.129+0000 7f434847c740 10 client.0 _ll_register_callbacks cb 0x7f4334009c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:29:51.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.129+0000 7f433e7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a6b86a6db0 0x55a6b86b9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:51.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.129+0000 7f433e7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a6b86a6db0 0x55a6b86b9bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55674/0 (socket says 172.21.15.97:55674) 2024-04-23T10:29:51.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.129+0000 7f433e7fc640 1 -- 172.21.15.97:0/1860843559 learned_addr learned my addr 172.21.15.97:0/1860843559 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:29:51.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.129+0000 7f433effd640 1 --2- 172.21.15.97:0/1860843559 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6b86a6490 0x55a6b86b9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:51.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.129+0000 7f433f7fe640 1 --2- 172.21.15.97:0/1860843559 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6b86b1480 0x55a6b86aa460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:51.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.129+0000 7f433e7fc640 1 -- 172.21.15.97:0/1860843559 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6b86a6490 msgr2=0x55a6b86b9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:29:51.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.129+0000 7f433e7fc640 1 --2- 172.21.15.97:0/1860843559 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6b86a6490 0x55a6b86b9680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:51.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.129+0000 7f433e7fc640 1 -- 172.21.15.97:0/1860843559 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6b86b1480 msgr2=0x55a6b86aa460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:29:51.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.129+0000 7f433e7fc640 1 --2- 172.21.15.97:0/1860843559 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6b86b1480 0x55a6b86aa460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:51.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.129+0000 7f433e7fc640 1 -- 172.21.15.97:0/1860843559 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6b86aac90 con 0x55a6b86a6db0 2024-04-23T10:29:51.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.129+0000 7f433effd640 1 --2- 172.21.15.97:0/1860843559 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6b86a6490 0x55a6b86b9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:29:51.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.129+0000 7f433f7fe640 1 --2- 172.21.15.97:0/1860843559 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6b86b1480 0x55a6b86aa460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:29:51.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.129+0000 7f433e7fc640 1 --2- 172.21.15.97:0/1860843559 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a6b86a6db0 0x55a6b86b9bc0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f4334011ca0 tx=0x7f4334011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b6368f7e973ef7e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:29:51.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.129+0000 7f4322ff5640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:29:51.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.129+0000 7f4322ff5640 1 -- 172.21.15.97:0/1860843559 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4334029070 con 0x55a6b86a6db0 2024-04-23T10:29:51.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.129+0000 7f4322ff5640 1 -- 172.21.15.97:0/1860843559 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f43340039a0 con 0x55a6b86a6db0 2024-04-23T10:29:51.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.129+0000 7f434847c740 1 -- 172.21.15.97:0/1860843559 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f433009c490 con 0x55a6b86a6db0 2024-04-23T10:29:51.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.130+0000 7f4322ff5640 1 -- 172.21.15.97:0/1860843559 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4334011a00 con 0x55a6b86a6db0 2024-04-23T10:29:51.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.130+0000 7f434847c740 10 client.16167 fetch_fsmap learned FSMap version 26 2024-04-23T10:29:51.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.130+0000 7f434847c740 1 -- 172.21.15.97:0/1860843559 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a6b86abfd0 con 0x55a6b86a6db0 2024-04-23T10:29:51.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.130+0000 7f4322ff5640 1 -- 172.21.15.97:0/1860843559 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f433009c490 con 0x55a6b86a6db0 2024-04-23T10:29:51.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.130+0000 7f4322ff5640 1 -- 172.21.15.97:0/1860843559 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f433401caf0 con 0x55a6b86a6db0 2024-04-23T10:29:51.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.130+0000 7f434847c740 10 client.16167 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:29:51.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.130+0000 7f434847c740 10 client.16167 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:29:51.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.130+0000 7f434847c740 20 client.16167 populate_metadata read hostname 'smithi097' 2024-04-23T10:29:51.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.130+0000 7f434847c740 1 --2- 172.21.15.97:0/1860843559 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55a6b8728ec0 0x55a6b872b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:51.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.130+0000 7f434847c740 4 client.16167 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "2ee2b90", "release": true, "await": true}] 2024-04-23T10:29:51.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.130+0000 7f434847c740 1 -- 172.21.15.97:0/1860843559 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "quiesce db", "set_id": "2ee2b90", "release": true, "await": true}) -- 0x55a6b86190f0 con 0x55a6b8728ec0 2024-04-23T10:29:51.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.130+0000 7f433effd640 1 --2- 172.21.15.97:0/1860843559 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55a6b8728ec0 0x55a6b872b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:51.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.131+0000 7f4322ff5640 10 client.16167 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:29:51.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.131+0000 7f433effd640 1 --2- 172.21.15.97:0/1860843559 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55a6b8728ec0 0x55a6b872b2b0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5834a5056faae52a server_cookie=12dcf62385d3a5c3 in_seq=0 out_seq=0 2024-04-23T10:29:51.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.137+0000 7f4322ff5640 1 -- 172.21.15.97:0/1860843559 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55a6b86190f0 con 0x55a6b8728ec0 2024-04-23T10:29:51.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.137+0000 7f4322ff5640 10 client.16167 handle_command_reply: tid=0 2024-04-23T10:29:51.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.137+0000 7f434847c740 1 client.16167 shutdown 2024-04-23T10:29:51.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.137+0000 7f434847c740 1 -- 172.21.15.97:0/1860843559 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a6b86a6db0 msgr2=0x55a6b86b9bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:51.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.137+0000 7f434847c740 1 --2- 172.21.15.97:0/1860843559 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a6b86a6db0 0x55a6b86b9bc0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f4334011ca0 tx=0x7f4334011810 comp rx=0 tx=0).stop 2024-04-23T10:29:51.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.137+0000 7f434847c740 1 -- 172.21.15.97:0/1860843559 shutdown_connections 2024-04-23T10:29:51.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.137+0000 7f434847c740 1 --2- 172.21.15.97:0/1860843559 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55a6b8728ec0 0x55a6b872b2b0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:51.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.137+0000 7f434847c740 1 --2- 172.21.15.97:0/1860843559 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6b86b1480 0x55a6b86aa460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:51.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.137+0000 7f4322ff5640 0 client.16167 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:29:51.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.137+0000 7f434847c740 1 --2- 172.21.15.97:0/1860843559 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a6b86a6db0 0x55a6b86b9bc0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:51.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.137+0000 7f434847c740 1 --2- 172.21.15.97:0/1860843559 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a6b86a6490 0x55a6b86b9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:51.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.137+0000 7f434847c740 1 -- 172.21.15.97:0/1860843559 >> 172.21.15.97:0/1860843559 conn(0x55a6b86a27b0 msgr2=0x55a6b86a7cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:29:51.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.137+0000 7f434847c740 1 -- 172.21.15.97:0/1860843559 shutdown_connections 2024-04-23T10:29:51.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.137+0000 7f434847c740 1 -- 172.21.15.97:0/1860843559 wait complete. 2024-04-23T10:29:51.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.137+0000 7f434847c740 20 client.16167 trim_cache size 0 max 16384 2024-04-23T10:29:51.137 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-23T10:29:51.138 INFO:teuthology.orchestra.run.smithi097.stdout: "epoch": 26, 2024-04-23T10:29:51.138 INFO:teuthology.orchestra.run.smithi097.stdout: "leader": 24488, 2024-04-23T10:29:51.138 INFO:teuthology.orchestra.run.smithi097.stdout: "set_version": 16, 2024-04-23T10:29:51.138 INFO:teuthology.orchestra.run.smithi097.stdout: "sets": { 2024-04-23T10:29:51.138 INFO:teuthology.orchestra.run.smithi097.stdout: "2ee2b90": { 2024-04-23T10:29:51.138 INFO:teuthology.orchestra.run.smithi097.stdout: "version": 16, 2024-04-23T10:29:51.138 INFO:teuthology.orchestra.run.smithi097.stdout: "age_ref": 0.0, 2024-04-23T10:29:51.138 INFO:teuthology.orchestra.run.smithi097.stdout: "state": { 2024-04-23T10:29:51.138 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RELEASED", 2024-04-23T10:29:51.138 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0 2024-04-23T10:29:51.138 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:29:51.138 INFO:teuthology.orchestra.run.smithi097.stdout: "timeout": 60.0, 2024-04-23T10:29:51.138 INFO:teuthology.orchestra.run.smithi097.stdout: "expiration": 89.9, 2024-04-23T10:29:51.138 INFO:teuthology.orchestra.run.smithi097.stdout: "members": { 2024-04-23T10:29:51.139 INFO:teuthology.orchestra.run.smithi097.stdout: "file:/": { 2024-04-23T10:29:51.139 INFO:teuthology.orchestra.run.smithi097.stdout: "excluded": false, 2024-04-23T10:29:51.139 INFO:teuthology.orchestra.run.smithi097.stdout: "state": { 2024-04-23T10:29:51.139 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RELEASED", 2024-04-23T10:29:51.139 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0 2024-04-23T10:29:51.139 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:29:51.139 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:29:51.139 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:29:51.139 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:29:51.139 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:29:51.139 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-23T10:29:51.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.137+0000 7f434575d640 1 -- 172.21.15.97:0/3465597999 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f43280a3d20 msgr2=0x7f43280a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:51.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.137+0000 7f434575d640 1 --2- 172.21.15.97:0/3465597999 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f43280a3d20 0x7f43280a61e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4338031810 tx=0x7f4338047000 comp rx=0 tx=0).stop 2024-04-23T10:29:51.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.137+0000 7f434575d640 1 -- 172.21.15.97:0/3465597999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4340102190 msgr2=0x7f4340198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:51.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.137+0000 7f434575d640 1 --2- 172.21.15.97:0/3465597999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4340102190 0x7f4340198750 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f433000d950 tx=0x7f433000de10 comp rx=0 tx=0).stop 2024-04-23T10:29:51.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.137+0000 7f434575d640 1 -- 172.21.15.97:0/3465597999 shutdown_connections 2024-04-23T10:29:51.140 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.137+0000 7f434575d640 1 --2- 172.21.15.97:0/3465597999 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f43280a3d20 0x7f43280a61e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:51.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.137+0000 7f434575d640 1 --2- 172.21.15.97:0/3465597999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f434019d010 0x7f434019d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:51.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.137+0000 7f434575d640 1 --2- 172.21.15.97:0/3465597999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4340107260 0x7f4340198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:51.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.137+0000 7f434575d640 1 --2- 172.21.15.97:0/3465597999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4340102190 0x7f4340198750 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:51.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.137+0000 7f434575d640 1 -- 172.21.15.97:0/3465597999 >> 172.21.15.97:0/3465597999 conn(0x7f43400793a0 msgr2=0x7f4340100030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:29:51.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.138+0000 7f434575d640 1 -- 172.21.15.97:0/3465597999 shutdown_connections 2024-04-23T10:29:51.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:29:51.138+0000 7f434575d640 1 -- 172.21.15.97:0/3465597999 wait complete. 2024-04-23T10:29:51.147 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '2ee2b90', seconds elapsed: 31.0 2024-04-23T10:29:51.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:29:51 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/2170491711' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:29:51.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:29:51 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/2170491711' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:29:51.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:29:51 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/2170491711' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:29:52.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:29:52 smithi196 ceph-mon[32919]: pgmap v113: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T10:29:52.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:29:52 smithi097 ceph-mon[29197]: pgmap v113: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T10:29:52.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:29:52 smithi183 ceph-mon[32317]: pgmap v113: 97 pgs: 97 active+clean; 3.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T10:29:54.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:29:54 smithi196 ceph-mon[32919]: pgmap v114: 97 pgs: 97 active+clean; 3.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.4 KiB/s wr, 13 op/s 2024-04-23T10:29:54.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:29:54 smithi097 ceph-mon[29197]: pgmap v114: 97 pgs: 97 active+clean; 3.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.4 KiB/s wr, 13 op/s 2024-04-23T10:29:54.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:29:54 smithi183 ceph-mon[32317]: pgmap v114: 97 pgs: 97 active+clean; 3.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.4 KiB/s wr, 13 op/s 2024-04-23T10:29:56.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:29:56 smithi196 ceph-mon[32919]: pgmap v115: 97 pgs: 97 active+clean; 3.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 4.4 KiB/s wr, 9 op/s 2024-04-23T10:29:56.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:29:56 smithi097 ceph-mon[29197]: pgmap v115: 97 pgs: 97 active+clean; 3.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 4.4 KiB/s wr, 9 op/s 2024-04-23T10:29:56.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:29:56 smithi183 ceph-mon[32317]: pgmap v115: 97 pgs: 97 active+clean; 3.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 4.4 KiB/s wr, 9 op/s 2024-04-23T10:29:57.181 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.182+0000 7feccdfed640 1 -- 172.21.15.97:0/3249879375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fecc81010f0 msgr2=0x7fecc81014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:57.181 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.182+0000 7feccdfed640 1 --2- 172.21.15.97:0/3249879375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fecc81010f0 0x7fecc81014d0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fecb8009900 tx=0x7fecb802a410 comp rx=0 tx=0).stop 2024-04-23T10:29:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.182+0000 7feccdfed640 1 -- 172.21.15.97:0/3249879375 shutdown_connections 2024-04-23T10:29:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.182+0000 7feccdfed640 1 --2- 172.21.15.97:0/3249879375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fecc8106170 0x7fecc810dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.182+0000 7feccdfed640 1 --2- 172.21.15.97:0/3249879375 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fecc8101aa0 0x7fecc8105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.182+0000 7feccdfed640 1 --2- 172.21.15.97:0/3249879375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fecc81010f0 0x7fecc81014d0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.182+0000 7feccdfed640 1 -- 172.21.15.97:0/3249879375 >> 172.21.15.97:0/3249879375 conn(0x7fecc8077fd0 msgr2=0x7fecc80783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:29:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.182+0000 7feccdfed640 1 -- 172.21.15.97:0/3249879375 shutdown_connections 2024-04-23T10:29:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.182+0000 7feccdfed640 1 -- 172.21.15.97:0/3249879375 wait complete. 2024-04-23T10:29:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.183+0000 7feccdfed640 1 Processor -- start 2024-04-23T10:29:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.183+0000 7feccdfed640 1 -- start start 2024-04-23T10:29:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.183+0000 7feccdfed640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fecc81010f0 0x7fecc8102b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.183+0000 7feccdfed640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fecc8106170 0x7fecc8103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.183+0000 7feccdfed640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fecc81035b0 0x7fecc81a68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.183+0000 7feccdfed640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fecc81110e0 con 0x7fecc81010f0 2024-04-23T10:29:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.183+0000 7feccdfed640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fecc8110f60 con 0x7fecc81035b0 2024-04-23T10:29:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.183+0000 7feccdfed640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fecc8111260 con 0x7fecc8106170 2024-04-23T10:29:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.183+0000 7fecc77fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fecc81010f0 0x7fecc8102b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.183+0000 7fecc77fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fecc81010f0 0x7fecc8102b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51568/0 (socket says 172.21.15.97:51568) 2024-04-23T10:29:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.183+0000 7fecc77fe640 1 -- 172.21.15.97:0/1949460711 learned_addr learned my addr 172.21.15.97:0/1949460711 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:29:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.183+0000 7fecc7fff640 1 --2- 172.21.15.97:0/1949460711 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fecc81035b0 0x7fecc81a68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:57.185 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.183+0000 7fecc6ffd640 1 --2- 172.21.15.97:0/1949460711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fecc8106170 0x7fecc8103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:57.185 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.184+0000 7fecc77fe640 1 -- 172.21.15.97:0/1949460711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fecc8106170 msgr2=0x7fecc8103070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:57.185 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.184+0000 7fecc77fe640 1 --2- 172.21.15.97:0/1949460711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fecc8106170 0x7fecc8103070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:57.185 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.184+0000 7fecc77fe640 1 -- 172.21.15.97:0/1949460711 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fecc81035b0 msgr2=0x7fecc81a68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:57.185 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.184+0000 7fecc77fe640 1 --2- 172.21.15.97:0/1949460711 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fecc81035b0 0x7fecc81a68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:57.185 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.184+0000 7fecc77fe640 1 -- 172.21.15.97:0/1949460711 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecc81a6fe0 con 0x7fecc81010f0 2024-04-23T10:29:57.185 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.184+0000 7fecc77fe640 1 --2- 172.21.15.97:0/1949460711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fecc81010f0 0x7fecc8102b30 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fecb8005940 tx=0x7fecb80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:29:57.185 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.184+0000 7fecc4ff9640 1 -- 172.21.15.97:0/1949460711 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb803f030 con 0x7fecc81010f0 2024-04-23T10:29:57.185 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.184+0000 7feccdfed640 1 -- 172.21.15.97:0/1949460711 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecc81a7270 con 0x7fecc81010f0 2024-04-23T10:29:57.185 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.184+0000 7fecc4ff9640 1 -- 172.21.15.97:0/1949460711 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecb8003cb0 con 0x7fecc81010f0 2024-04-23T10:29:57.185 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.184+0000 7fecc4ff9640 1 -- 172.21.15.97:0/1949460711 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb8035070 con 0x7fecc81010f0 2024-04-23T10:29:57.185 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.184+0000 7feccdfed640 1 -- 172.21.15.97:0/1949460711 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecc81a7760 con 0x7fecc81010f0 2024-04-23T10:29:57.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.186+0000 7fecc4ff9640 1 -- 172.21.15.97:0/1949460711 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fecb8042030 con 0x7fecc81010f0 2024-04-23T10:29:57.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.186+0000 7fecd0dd8740 1 Processor -- start 2024-04-23T10:29:57.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.186+0000 7fecd0dd8740 1 -- start start 2024-04-23T10:29:57.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.187+0000 7fecd0dd8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555d986d7620 0x555d986d7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:57.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.187+0000 7fecd0dd8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d986d7f40 0x555d986e22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:57.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.187+0000 7fecd0dd8740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d986e2800 0x555d986e4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:57.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.187+0000 7fecd0dd8740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x555d9863c970 con 0x555d986e2800 2024-04-23T10:29:57.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.187+0000 7fecd0dd8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555d985d94a0 con 0x555d986d7f40 2024-04-23T10:29:57.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.187+0000 7fecd0dd8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555d98638930 con 0x555d986d7620 2024-04-23T10:29:57.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.187+0000 7fecc4ff9640 1 --2- 172.21.15.97:0/1949460711 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fecb00a3df0 0x7fecb00a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:57.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.187+0000 7fecc6ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d986e2800 0x555d986e4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:57.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.187+0000 7fecc77fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d986d7f40 0x555d986e22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:57.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.187+0000 7fecc7fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555d986d7620 0x555d986d7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:57.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.187+0000 7fecc6ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d986e2800 0x555d986e4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51578/0 (socket says 172.21.15.97:51578) 2024-04-23T10:29:57.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.187+0000 7fecc6ffd640 1 -- 172.21.15.97:0/1621488117 learned_addr learned my addr 172.21.15.97:0/1621488117 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:29:57.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.187+0000 7fecc7fff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555d986d7620 0x555d986d7a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.97:49458/0 (socket says 172.21.15.97:49458) 2024-04-23T10:29:57.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.187+0000 7fecc7fff640 1 --2- 172.21.15.97:0/1949460711 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fecb00a3df0 0x7fecb00a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:57.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.187+0000 7fecc4ff9640 1 -- 172.21.15.97:0/1949460711 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fecb80f0c90 con 0x7fecc81010f0 2024-04-23T10:29:57.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.187+0000 7fecc6ffd640 1 -- 172.21.15.97:0/1621488117 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555d986d7620 msgr2=0x555d986d7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:29:57.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.187+0000 7fecc6ffd640 1 --2- 172.21.15.97:0/1621488117 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555d986d7620 0x555d986d7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:57.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.187+0000 7fecc6ffd640 1 -- 172.21.15.97:0/1621488117 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d986d7f40 msgr2=0x555d986e22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:29:57.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.187+0000 7fecc6ffd640 1 --2- 172.21.15.97:0/1621488117 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d986d7f40 0x555d986e22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:57.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.187+0000 7fecc6ffd640 1 -- 172.21.15.97:0/1621488117 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d9834f210 con 0x555d986e2800 2024-04-23T10:29:57.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.187+0000 7fecc6ffd640 1 --2- 172.21.15.97:0/1621488117 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d986e2800 0x555d986e4bf0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fecbc00bc40 tx=0x7fecbc00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beff43b85573cdf0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:29:57.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.188+0000 7fecc7fff640 1 --2- 172.21.15.97:0/1949460711 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fecb00a3df0 0x7fecb00a62b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fecc000a780 tx=0x7fecc000a9e0 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:29:57.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.188+0000 7fecadffb640 1 -- 172.21.15.97:0/1621488117 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecbc015070 con 0x555d986e2800 2024-04-23T10:29:57.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.188+0000 7fecadffb640 1 -- 172.21.15.97:0/1621488117 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecbc01a910 con 0x555d986e2800 2024-04-23T10:29:57.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.188+0000 7fecadffb640 1 -- 172.21.15.97:0/1621488117 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecbc011830 con 0x555d986e2800 2024-04-23T10:29:57.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.188+0000 7fecd0dd8740 1 -- 172.21.15.97:0/1621488117 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d986e2800 msgr2=0x555d986e4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:57.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.188+0000 7fecd0dd8740 1 --2- 172.21.15.97:0/1621488117 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d986e2800 0x555d986e4bf0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fecbc00bc40 tx=0x7fecbc00f730 comp rx=0 tx=0).stop 2024-04-23T10:29:57.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.188+0000 7fecd0dd8740 1 -- 172.21.15.97:0/1621488117 shutdown_connections 2024-04-23T10:29:57.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.188+0000 7fecd0dd8740 1 --2- 172.21.15.97:0/1621488117 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d986e2800 0x555d986e4bf0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:57.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.188+0000 7fecd0dd8740 1 --2- 172.21.15.97:0/1621488117 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d986d7f40 0x555d986e22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:57.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.189+0000 7fecd0dd8740 1 --2- 172.21.15.97:0/1621488117 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555d986d7620 0x555d986d7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:57.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.189+0000 7fecd0dd8740 1 -- 172.21.15.97:0/1621488117 >> 172.21.15.97:0/1621488117 conn(0x555d986d3b30 msgr2=0x555d986d3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:29:57.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.189+0000 7fecd0dd8740 1 -- 172.21.15.97:0/1621488117 shutdown_connections 2024-04-23T10:29:57.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.189+0000 7fecd0dd8740 1 -- 172.21.15.97:0/1621488117 wait complete. 2024-04-23T10:29:57.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.189+0000 7fecd0dd8740 1 -- start start 2024-04-23T10:29:57.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.189+0000 7fecd0dd8740 1 Processor -- start 2024-04-23T10:29:57.195 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.189+0000 7fecd0dd8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d986d7620 0x555d986e2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:57.195 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.189+0000 7fecd0dd8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555d986d7f40 0x555d986df370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:57.195 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.189+0000 7fecd0dd8740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d986e2800 0x555d986df8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:57.195 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.189+0000 7fecd0dd8740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x555d986e5430 con 0x555d986e2800 2024-04-23T10:29:57.195 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.189+0000 7fecd0dd8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555d985d70b0 con 0x555d986d7620 2024-04-23T10:29:57.195 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.189+0000 7fecd0dd8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555d986e55b0 con 0x555d986d7f40 2024-04-23T10:29:57.195 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.189+0000 7fecd0dd8740 10 client.0 _ll_register_callbacks cb 0x555d985c4d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:29:57.195 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.190+0000 7fecc7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d986e2800 0x555d986df8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:57.195 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.190+0000 7fecc6ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555d986d7f40 0x555d986df370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:57.195 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.190+0000 7fecc7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d986e2800 0x555d986df8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51586/0 (socket says 172.21.15.97:51586) 2024-04-23T10:29:57.195 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.190+0000 7fecc6ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555d986d7f40 0x555d986df370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.97:49472/0 (socket says 172.21.15.97:49472) 2024-04-23T10:29:57.195 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.190+0000 7fecc7fff640 1 -- 172.21.15.97:0/2750400070 learned_addr learned my addr 172.21.15.97:0/2750400070 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:29:57.195 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.190+0000 7fecc77fe640 1 --2- 172.21.15.97:0/2750400070 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d986d7620 0x555d986e2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:57.195 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.190+0000 7fecc7fff640 1 -- 172.21.15.97:0/2750400070 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555d986d7f40 msgr2=0x555d986df370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:29:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.190+0000 7fecc7fff640 1 --2- 172.21.15.97:0/2750400070 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555d986d7f40 0x555d986df370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.190+0000 7fecc7fff640 1 -- 172.21.15.97:0/2750400070 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d986d7620 msgr2=0x555d986e2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:29:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.190+0000 7fecc7fff640 1 --2- 172.21.15.97:0/2750400070 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d986d7620 0x555d986e2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.190+0000 7fecc7fff640 1 -- 172.21.15.97:0/2750400070 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d9841b620 con 0x555d986e2800 2024-04-23T10:29:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.190+0000 7fecc7fff640 1 --2- 172.21.15.97:0/2750400070 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d986e2800 0x555d986df8b0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fecc000f9b0 tx=0x7fecc000a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac5574190b1f1830 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:29:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.190+0000 7feca6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:29:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.190+0000 7feca6ffd640 1 -- 172.21.15.97:0/2750400070 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc0027070 con 0x555d986e2800 2024-04-23T10:29:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.190+0000 7feca6ffd640 1 -- 172.21.15.97:0/2750400070 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fecc000eb70 con 0x555d986e2800 2024-04-23T10:29:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.190+0000 7fecd0dd8740 1 -- 172.21.15.97:0/2750400070 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fecbc01a910 con 0x555d986e2800 2024-04-23T10:29:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.190+0000 7feca6ffd640 1 -- 172.21.15.97:0/2750400070 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fecc0031050 con 0x555d986e2800 2024-04-23T10:29:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.190+0000 7fecd0dd8740 10 client.16179 fetch_fsmap learned FSMap version 26 2024-04-23T10:29:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.190+0000 7fecd0dd8740 1 -- 172.21.15.97:0/2750400070 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555d986f12c0 con 0x555d986e2800 2024-04-23T10:29:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.190+0000 7feca6ffd640 1 -- 172.21.15.97:0/2750400070 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecbc01a910 con 0x555d986e2800 2024-04-23T10:29:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.191+0000 7feca6ffd640 1 -- 172.21.15.97:0/2750400070 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fecc0004020 con 0x555d986e2800 2024-04-23T10:29:57.196 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.191+0000 7fecd0dd8740 10 client.16179 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:29:57.197 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.191+0000 7fecd0dd8740 10 client.16179 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:29:57.197 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.191+0000 7fecd0dd8740 20 client.16179 populate_metadata read hostname 'smithi097' 2024-04-23T10:29:57.197 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.191+0000 7fecd0dd8740 1 --2- 172.21.15.97:0/2750400070 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x555d986f2780 0x555d986f4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:57.197 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.191+0000 7fecd0dd8740 4 client.16179 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:29:57.197 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.191+0000 7fecd0dd8740 1 -- 172.21.15.97:0/2750400070 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d9864a390 con 0x555d986f2780 2024-04-23T10:29:57.197 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.191+0000 7fecc77fe640 1 --2- 172.21.15.97:0/2750400070 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x555d986f2780 0x555d986f4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:57.197 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.191+0000 7feca6ffd640 10 client.16179 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:29:57.197 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.191+0000 7fecc77fe640 1 --2- 172.21.15.97:0/2750400070 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x555d986f2780 0x555d986f4bf0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94121d5d1245774f server_cookie=74a99e7c85234566 in_seq=0 out_seq=0 2024-04-23T10:29:57.197 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.192+0000 7feca6ffd640 1 -- 172.21.15.97:0/2750400070 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555d9864a390 con 0x555d986f2780 2024-04-23T10:29:57.197 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.192+0000 7feca6ffd640 10 client.16179 handle_command_reply: tid=0 2024-04-23T10:29:57.197 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.192+0000 7fecd0dd8740 1 client.16179 shutdown 2024-04-23T10:29:57.197 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.192+0000 7fecd0dd8740 1 -- 172.21.15.97:0/2750400070 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d986e2800 msgr2=0x555d986df8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:57.197 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.192+0000 7fecd0dd8740 1 --2- 172.21.15.97:0/2750400070 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d986e2800 0x555d986df8b0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fecc000f9b0 tx=0x7fecc000a890 comp rx=0 tx=0).stop 2024-04-23T10:29:57.197 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.193+0000 7fecd0dd8740 1 -- 172.21.15.97:0/2750400070 shutdown_connections 2024-04-23T10:29:57.197 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.193+0000 7fecd0dd8740 1 --2- 172.21.15.97:0/2750400070 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x555d986f2780 0x555d986f4bf0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:57.198 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.193+0000 7fecd0dd8740 1 --2- 172.21.15.97:0/2750400070 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d986e2800 0x555d986df8b0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:57.198 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.193+0000 7fecd0dd8740 1 --2- 172.21.15.97:0/2750400070 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555d986d7f40 0x555d986df370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:57.198 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.193+0000 7feca6ffd640 0 client.16179 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:29:57.198 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.193+0000 7fecd0dd8740 1 --2- 172.21.15.97:0/2750400070 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d986d7620 0x555d986e2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:57.198 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.193+0000 7fecd0dd8740 1 -- 172.21.15.97:0/2750400070 >> 172.21.15.97:0/2750400070 conn(0x555d986d3b30 msgr2=0x555d986d9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:29:57.198 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.193+0000 7fecd0dd8740 1 -- 172.21.15.97:0/2750400070 shutdown_connections 2024-04-23T10:29:57.198 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.193+0000 7fecd0dd8740 1 -- 172.21.15.97:0/2750400070 wait complete. 2024-04-23T10:29:57.198 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.193+0000 7fecd0dd8740 20 client.16179 trim_cache size 0 max 16384 2024-04-23T10:29:57.211 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.208+0000 7fecd0dd8740 1 Processor -- start 2024-04-23T10:29:57.211 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.208+0000 7fecd0dd8740 1 -- start start 2024-04-23T10:29:57.211 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.208+0000 7fecd0dd8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555d986d7620 0x555d986df890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:57.211 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.208+0000 7fecd0dd8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d986d7f40 0x555d986ee2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:57.211 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.208+0000 7fecd0dd8740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d986e2800 0x555d986fa500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:57.211 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.208+0000 7fecd0dd8740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fecc81afc30 con 0x555d986e2800 2024-04-23T10:29:57.211 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.208+0000 7fecd0dd8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fecc81b1770 con 0x555d986d7f40 2024-04-23T10:29:57.211 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.208+0000 7fecd0dd8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fecc81b0300 con 0x555d986d7620 2024-04-23T10:29:57.212 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.208+0000 7fecc7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d986e2800 0x555d986fa500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:57.212 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.208+0000 7fecc7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d986e2800 0x555d986fa500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51590/0 (socket says 172.21.15.97:51590) 2024-04-23T10:29:57.212 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.208+0000 7fecc7fff640 1 -- 172.21.15.97:0/859280073 learned_addr learned my addr 172.21.15.97:0/859280073 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:29:57.212 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.208+0000 7fecc6ffd640 1 --2- 172.21.15.97:0/859280073 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d986d7f40 0x555d986ee2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:57.212 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.208+0000 7fecc77fe640 1 --2- 172.21.15.97:0/859280073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555d986d7620 0x555d986df890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:57.212 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.208+0000 7fecc7fff640 1 -- 172.21.15.97:0/859280073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555d986d7620 msgr2=0x555d986df890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:29:57.212 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.208+0000 7fecc7fff640 1 --2- 172.21.15.97:0/859280073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555d986d7620 0x555d986df890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:57.212 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.208+0000 7fecc7fff640 1 -- 172.21.15.97:0/859280073 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d986d7f40 msgr2=0x555d986ee2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:29:57.212 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.208+0000 7fecc7fff640 1 --2- 172.21.15.97:0/859280073 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d986d7f40 0x555d986ee2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:57.212 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.208+0000 7fecc7fff640 1 -- 172.21.15.97:0/859280073 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d986d7d70 con 0x555d986e2800 2024-04-23T10:29:57.212 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.208+0000 7fecc6ffd640 1 --2- 172.21.15.97:0/859280073 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d986d7f40 0x555d986ee2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:29:57.212 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.208+0000 7fecc7fff640 1 --2- 172.21.15.97:0/859280073 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d986e2800 0x555d986fa500 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fecc0014fd0 tx=0x7fecc0005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b20d54994a27c4b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:29:57.212 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.208+0000 7fecad7fa640 1 -- 172.21.15.97:0/859280073 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc0027070 con 0x555d986e2800 2024-04-23T10:29:57.213 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.208+0000 7fecad7fa640 1 -- 172.21.15.97:0/859280073 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecc00046b0 con 0x555d986e2800 2024-04-23T10:29:57.213 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.208+0000 7fecad7fa640 1 -- 172.21.15.97:0/859280073 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc0004d10 con 0x555d986e2800 2024-04-23T10:29:57.213 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.209+0000 7fecd0dd8740 1 -- 172.21.15.97:0/859280073 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d986e2800 msgr2=0x555d986fa500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:57.213 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.209+0000 7fecd0dd8740 1 --2- 172.21.15.97:0/859280073 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d986e2800 0x555d986fa500 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fecc0014fd0 tx=0x7fecc0005fa0 comp rx=0 tx=0).stop 2024-04-23T10:29:57.213 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.209+0000 7fecd0dd8740 1 -- 172.21.15.97:0/859280073 shutdown_connections 2024-04-23T10:29:57.213 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.209+0000 7fecd0dd8740 1 --2- 172.21.15.97:0/859280073 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d986e2800 0x555d986fa500 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:57.213 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.209+0000 7fecd0dd8740 1 --2- 172.21.15.97:0/859280073 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d986d7f40 0x555d986ee2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:57.213 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.209+0000 7fecd0dd8740 1 --2- 172.21.15.97:0/859280073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555d986d7620 0x555d986df890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:57.213 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.209+0000 7fecd0dd8740 1 -- 172.21.15.97:0/859280073 >> 172.21.15.97:0/859280073 conn(0x555d986d3b30 msgr2=0x555d986d2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:29:57.213 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.209+0000 7fecd0dd8740 1 -- 172.21.15.97:0/859280073 shutdown_connections 2024-04-23T10:29:57.213 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.209+0000 7fecd0dd8740 1 -- 172.21.15.97:0/859280073 wait complete. 2024-04-23T10:29:57.213 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.209+0000 7fecd0dd8740 1 -- start start 2024-04-23T10:29:57.213 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.210+0000 7fecd0dd8740 1 Processor -- start 2024-04-23T10:29:57.213 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.210+0000 7fecd0dd8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d986d7620 0x555d986ead20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:57.213 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.210+0000 7fecd0dd8740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d986d7f40 0x555d986e80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:57.214 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.210+0000 7fecd0dd8740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555d986e2800 0x555d986e87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:57.214 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.210+0000 7fecd0dd8740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fecc81b10a0 con 0x555d986d7f40 2024-04-23T10:29:57.214 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.210+0000 7fecd0dd8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fecc81b09d0 con 0x555d986d7620 2024-04-23T10:29:57.214 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.210+0000 7fecd0dd8740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fecc81adc30 con 0x555d986e2800 2024-04-23T10:29:57.214 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.210+0000 7fecd0dd8740 10 client.0 _ll_register_callbacks cb 0x7fecc0011ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:29:57.214 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.210+0000 7fecc6ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d986d7f40 0x555d986e80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:57.214 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.210+0000 7fecc6ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d986d7f40 0x555d986e80b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51606/0 (socket says 172.21.15.97:51606) 2024-04-23T10:29:57.214 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.210+0000 7fecc6ffd640 1 -- 172.21.15.97:0/1892587529 learned_addr learned my addr 172.21.15.97:0/1892587529 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:29:57.214 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.210+0000 7fecc7fff640 1 --2- 172.21.15.97:0/1892587529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555d986e2800 0x555d986e87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:57.214 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.210+0000 7fecc77fe640 1 --2- 172.21.15.97:0/1892587529 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d986d7620 0x555d986ead20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:57.214 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.210+0000 7fecc6ffd640 1 -- 172.21.15.97:0/1892587529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555d986e2800 msgr2=0x555d986e87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:29:57.214 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.210+0000 7fecc6ffd640 1 --2- 172.21.15.97:0/1892587529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555d986e2800 0x555d986e87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:57.214 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.210+0000 7fecc6ffd640 1 -- 172.21.15.97:0/1892587529 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d986d7620 msgr2=0x555d986ead20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:29:57.214 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.210+0000 7fecc6ffd640 1 --2- 172.21.15.97:0/1892587529 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d986d7620 0x555d986ead20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:57.215 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.210+0000 7fecc6ffd640 1 -- 172.21.15.97:0/1892587529 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d98757d80 con 0x555d986d7f40 2024-04-23T10:29:57.215 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.210+0000 7fecc7fff640 1 --2- 172.21.15.97:0/1892587529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555d986e2800 0x555d986e87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:29:57.215 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.210+0000 7fecc6ffd640 1 --2- 172.21.15.97:0/1892587529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d986d7f40 0x555d986e80b0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fecbc00bc40 tx=0x7fecbc005f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faea3671e193e677 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:29:57.215 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.210+0000 7feca6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:29:57.215 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.211+0000 7feca6ffd640 1 -- 172.21.15.97:0/1892587529 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecbc02a060 con 0x555d986d7f40 2024-04-23T10:29:57.215 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.211+0000 7feca6ffd640 1 -- 172.21.15.97:0/1892587529 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fecbc004440 con 0x555d986d7f40 2024-04-23T10:29:57.215 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.211+0000 7fecd0dd8740 1 -- 172.21.15.97:0/1892587529 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fecc00046b0 con 0x555d986d7f40 2024-04-23T10:29:57.215 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.211+0000 7feca6ffd640 1 -- 172.21.15.97:0/1892587529 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fecbc015040 con 0x555d986d7f40 2024-04-23T10:29:57.215 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.211+0000 7fecd0dd8740 10 client.16185 fetch_fsmap learned FSMap version 26 2024-04-23T10:29:57.215 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.211+0000 7fecd0dd8740 1 -- 172.21.15.97:0/1892587529 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555d98758f20 con 0x555d986d7f40 2024-04-23T10:29:57.215 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.211+0000 7feca6ffd640 1 -- 172.21.15.97:0/1892587529 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecc00046b0 con 0x555d986d7f40 2024-04-23T10:29:57.215 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.211+0000 7feca6ffd640 1 -- 172.21.15.97:0/1892587529 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fecbc004a90 con 0x555d986d7f40 2024-04-23T10:29:57.215 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.211+0000 7fecd0dd8740 10 client.16185 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:29:57.215 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.211+0000 7fecd0dd8740 10 client.16185 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:29:57.215 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.211+0000 7fecd0dd8740 20 client.16185 populate_metadata read hostname 'smithi097' 2024-04-23T10:29:57.216 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.211+0000 7fecd0dd8740 1 --2- 172.21.15.97:0/1892587529 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x555d987593d0 0x555d9875b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:29:57.216 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.211+0000 7fecd0dd8740 4 client.16185 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:29:57.216 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.211+0000 7fecd0dd8740 1 -- 172.21.15.97:0/1892587529 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555d98644200 con 0x555d987593d0 2024-04-23T10:29:57.216 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.211+0000 7fecc77fe640 1 --2- 172.21.15.97:0/1892587529 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x555d987593d0 0x555d9875b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:29:57.216 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.212+0000 7fecc77fe640 1 --2- 172.21.15.97:0/1892587529 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x555d987593d0 0x555d9875b7f0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cef54ca85925c6fc server_cookie=8ee5b1dcd86bdf01 in_seq=0 out_seq=0 2024-04-23T10:29:57.216 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.212+0000 7feca6ffd640 10 client.16185 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:29:57.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.220+0000 7feca6ffd640 1 -- 172.21.15.97:0/1892587529 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555d98644200 con 0x555d987593d0 2024-04-23T10:29:57.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.220+0000 7feca6ffd640 10 client.16185 handle_command_reply: tid=0 2024-04-23T10:29:57.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.220+0000 7fecd0dd8740 1 client.16185 shutdown 2024-04-23T10:29:57.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.220+0000 7fecd0dd8740 1 -- 172.21.15.97:0/1892587529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d986d7f40 msgr2=0x555d986e80b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:57.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.220+0000 7fecd0dd8740 1 --2- 172.21.15.97:0/1892587529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d986d7f40 0x555d986e80b0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fecbc00bc40 tx=0x7fecbc005f20 comp rx=0 tx=0).stop 2024-04-23T10:29:57.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.221+0000 7fecd0dd8740 1 -- 172.21.15.97:0/1892587529 shutdown_connections 2024-04-23T10:29:57.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.221+0000 7fecd0dd8740 1 --2- 172.21.15.97:0/1892587529 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x555d987593d0 0x555d9875b7f0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:57.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.221+0000 7fecd0dd8740 1 --2- 172.21.15.97:0/1892587529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555d986e2800 0x555d986e87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:57.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.221+0000 7fecd0dd8740 1 --2- 172.21.15.97:0/1892587529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d986d7f40 0x555d986e80b0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:57.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.221+0000 7feca6ffd640 0 client.16185 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:29:57.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.221+0000 7fecd0dd8740 1 --2- 172.21.15.97:0/1892587529 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d986d7620 0x555d986ead20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.221+0000 7fecd0dd8740 1 -- 172.21.15.97:0/1892587529 >> 172.21.15.97:0/1892587529 conn(0x555d986d3b30 msgr2=0x555d986ed2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:29:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.221+0000 7fecd0dd8740 1 -- 172.21.15.97:0/1892587529 shutdown_connections 2024-04-23T10:29:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.221+0000 7fecd0dd8740 1 -- 172.21.15.97:0/1892587529 wait complete. 2024-04-23T10:29:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.221+0000 7fecd0dd8740 20 client.16185 trim_cache size 0 max 16384 2024-04-23T10:29:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:29:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:29:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:29:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:29:57.222 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.221+0000 7feccdfed640 1 -- 172.21.15.97:0/1949460711 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fecb00a3df0 msgr2=0x7fecb00a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:57.222 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.221+0000 7feccdfed640 1 --2- 172.21.15.97:0/1949460711 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fecb00a3df0 0x7fecb00a62b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fecc000a780 tx=0x7fecc000a9e0 comp rx=0 tx=0).stop 2024-04-23T10:29:57.222 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.221+0000 7feccdfed640 1 -- 172.21.15.97:0/1949460711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fecc81010f0 msgr2=0x7fecc8102b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:29:57.222 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.221+0000 7feccdfed640 1 --2- 172.21.15.97:0/1949460711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fecc81010f0 0x7fecc8102b30 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fecb8005940 tx=0x7fecb80042f0 comp rx=0 tx=0).stop 2024-04-23T10:29:57.222 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.221+0000 7feccdfed640 1 -- 172.21.15.97:0/1949460711 shutdown_connections 2024-04-23T10:29:57.222 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.221+0000 7feccdfed640 1 --2- 172.21.15.97:0/1949460711 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fecb00a3df0 0x7fecb00a62b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:57.222 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.221+0000 7feccdfed640 1 --2- 172.21.15.97:0/1949460711 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fecc81035b0 0x7fecc81a68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:57.223 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.221+0000 7feccdfed640 1 --2- 172.21.15.97:0/1949460711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fecc8106170 0x7fecc8103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:57.223 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.221+0000 7feccdfed640 1 --2- 172.21.15.97:0/1949460711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fecc81010f0 0x7fecc8102b30 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:29:57.223 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.221+0000 7feccdfed640 1 -- 172.21.15.97:0/1949460711 >> 172.21.15.97:0/1949460711 conn(0x7fecc8077fd0 msgr2=0x7fecc80699b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:29:57.223 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.221+0000 7feccdfed640 1 -- 172.21.15.97:0/1949460711 shutdown_connections 2024-04-23T10:29:57.223 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:29:57.221+0000 7feccdfed640 1 -- 172.21.15.97:0/1949460711 wait complete. 2024-04-23T10:29:58.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:29:58 smithi196 ceph-mon[32919]: pgmap v116: 97 pgs: 97 active+clean; 3.2 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-04-23T10:29:58.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:29:58 smithi097 ceph-mon[29197]: pgmap v116: 97 pgs: 97 active+clean; 3.2 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-04-23T10:29:58.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:29:58 smithi183 ceph-mon[32317]: pgmap v116: 97 pgs: 97 active+clean; 3.2 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-04-23T10:30:00.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:30:00 smithi196 ceph-mon[32919]: pgmap v117: 97 pgs: 97 active+clean; 3.2 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-04-23T10:30:00.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:30:00 smithi196 ceph-mon[32919]: overall HEALTH_OK 2024-04-23T10:30:00.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:30:00 smithi097 ceph-mon[29197]: pgmap v117: 97 pgs: 97 active+clean; 3.2 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-04-23T10:30:00.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:30:00 smithi097 ceph-mon[29197]: overall HEALTH_OK 2024-04-23T10:30:00.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:30:00 smithi183 ceph-mon[32317]: pgmap v117: 97 pgs: 97 active+clean; 3.2 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-04-23T10:30:00.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:30:00 smithi183 ceph-mon[32317]: overall HEALTH_OK 2024-04-23T10:30:02.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:30:02 smithi196 ceph-mon[32919]: pgmap v118: 97 pgs: 97 active+clean; 3.2 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-04-23T10:30:02.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:30:02 smithi097 ceph-mon[29197]: pgmap v118: 97 pgs: 97 active+clean; 3.2 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-04-23T10:30:02.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:30:02 smithi183 ceph-mon[32317]: pgmap v118: 97 pgs: 97 active+clean; 3.2 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-04-23T10:30:04.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:30:04 smithi097 ceph-mon[29197]: pgmap v119: 97 pgs: 97 active+clean; 3.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 48 KiB/s wr, 34 op/s 2024-04-23T10:30:04.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:30:04 smithi183 ceph-mon[32317]: pgmap v119: 97 pgs: 97 active+clean; 3.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 48 KiB/s wr, 34 op/s 2024-04-23T10:30:04.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:30:04 smithi196 ceph-mon[32919]: pgmap v119: 97 pgs: 97 active+clean; 3.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 48 KiB/s wr, 34 op/s 2024-04-23T10:30:06.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:30:06 smithi097 ceph-mon[29197]: pgmap v120: 97 pgs: 97 active+clean; 3.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T10:30:06.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:30:06 smithi183 ceph-mon[32317]: pgmap v120: 97 pgs: 97 active+clean; 3.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T10:30:06.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:30:06 smithi196 ceph-mon[32919]: pgmap v120: 97 pgs: 97 active+clean; 3.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T10:30:08.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:30:08 smithi097 ceph-mon[29197]: pgmap v121: 97 pgs: 97 active+clean; 3.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 57 KiB/s wr, 36 op/s 2024-04-23T10:30:08.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:30:08 smithi183 ceph-mon[32317]: pgmap v121: 97 pgs: 97 active+clean; 3.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 57 KiB/s wr, 36 op/s 2024-04-23T10:30:08.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:30:08 smithi196 ceph-mon[32919]: pgmap v121: 97 pgs: 97 active+clean; 3.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 57 KiB/s wr, 36 op/s 2024-04-23T10:30:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.316+0000 7f200b744640 1 -- 172.21.15.97:0/1663695837 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f200410c550 msgr2=0x7f200410e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:30:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.316+0000 7f200b744640 1 --2- 172.21.15.97:0/1663695837 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f200410c550 0x7f200410e9d0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc009900 tx=0x7f1ffc02a480 comp rx=0 tx=0).stop 2024-04-23T10:30:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.317+0000 7f200b744640 1 -- 172.21.15.97:0/1663695837 shutdown_connections 2024-04-23T10:30:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.317+0000 7f200b744640 1 --2- 172.21.15.97:0/1663695837 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f200410c550 0x7f200410e9d0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:10.319 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.317+0000 7f200b744640 1 --2- 172.21.15.97:0/1663695837 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2004103aa0 0x7f200410bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:10.319 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.317+0000 7f200b744640 1 --2- 172.21.15.97:0/1663695837 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2004103180 0x7f2004103560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:10.319 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.317+0000 7f200b744640 1 -- 172.21.15.97:0/1663695837 >> 172.21.15.97:0/1663695837 conn(0x7f2004077fd0 msgr2=0x7f20040783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:30:10.319 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.317+0000 7f200b744640 1 -- 172.21.15.97:0/1663695837 shutdown_connections 2024-04-23T10:30:10.319 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.317+0000 7f200b744640 1 -- 172.21.15.97:0/1663695837 wait complete. 2024-04-23T10:30:10.319 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.317+0000 7f200b744640 1 Processor -- start 2024-04-23T10:30:10.319 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.317+0000 7f200b744640 1 -- start start 2024-04-23T10:30:10.319 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.317+0000 7f200b744640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2004103180 0x7f2004100dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:10.319 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.317+0000 7f200b744640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2004103aa0 0x7f2004101310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:10.319 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.317+0000 7f200b744640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f200410c550 0x7f2004101850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:10.319 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.317+0000 7f200b744640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f20041121a0 con 0x7f200410c550 2024-04-23T10:30:10.319 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.317+0000 7f200b744640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2004112020 con 0x7f2004103aa0 2024-04-23T10:30:10.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.317+0000 7f200b744640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2004112320 con 0x7f2004103180 2024-04-23T10:30:10.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.318+0000 7f2009ce1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f200410c550 0x7f2004101850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:10.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.318+0000 7f2009ce1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f200410c550 0x7f2004101850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34804/0 (socket says 172.21.15.97:34804) 2024-04-23T10:30:10.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.318+0000 7f2009ce1640 1 -- 172.21.15.97:0/3813959231 learned_addr learned my addr 172.21.15.97:0/3813959231 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:30:10.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.318+0000 7f2008cdf640 1 --2- 172.21.15.97:0/3813959231 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2004103aa0 0x7f2004101310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:10.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.318+0000 7f20094e0640 1 --2- 172.21.15.97:0/3813959231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2004103180 0x7f2004100dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:10.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.318+0000 7f2009ce1640 1 -- 172.21.15.97:0/3813959231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2004103180 msgr2=0x7f2004100dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:30:10.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.318+0000 7f2009ce1640 1 --2- 172.21.15.97:0/3813959231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2004103180 0x7f2004100dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:10.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.318+0000 7f2009ce1640 1 -- 172.21.15.97:0/3813959231 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2004103aa0 msgr2=0x7f2004101310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:30:10.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.318+0000 7f2009ce1640 1 --2- 172.21.15.97:0/3813959231 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2004103aa0 0x7f2004101310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:10.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.318+0000 7f2009ce1640 1 -- 172.21.15.97:0/3813959231 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20041a6ae0 con 0x7f200410c550 2024-04-23T10:30:10.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.318+0000 7f2009ce1640 1 --2- 172.21.15.97:0/3813959231 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f200410c550 0x7f2004101850 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc004770 tx=0x7f1ffc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:30:10.321 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.318+0000 7f1ffa7fc640 1 -- 172.21.15.97:0/3813959231 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc03f030 con 0x7f200410c550 2024-04-23T10:30:10.321 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.318+0000 7f1ffa7fc640 1 -- 172.21.15.97:0/3813959231 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ffc003aa0 con 0x7f200410c550 2024-04-23T10:30:10.321 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.318+0000 7f200b744640 1 -- 172.21.15.97:0/3813959231 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20041a6d70 con 0x7f200410c550 2024-04-23T10:30:10.321 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.318+0000 7f1ffa7fc640 1 -- 172.21.15.97:0/3813959231 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc035070 con 0x7f200410c550 2024-04-23T10:30:10.321 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.318+0000 7f200b744640 1 -- 172.21.15.97:0/3813959231 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20041a7260 con 0x7f200410c550 2024-04-23T10:30:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.320+0000 7f1ffa7fc640 1 -- 172.21.15.97:0/3813959231 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1ffc042030 con 0x7f200410c550 2024-04-23T10:30:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.321+0000 7f200e469740 1 Processor -- start 2024-04-23T10:30:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.321+0000 7f200e469740 1 -- start start 2024-04-23T10:30:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.321+0000 7f200e469740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9dadbc620 0x55a9dadbca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.321+0000 7f200e469740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9dadbcf40 0x55a9dadc72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.321+0000 7f1ffa7fc640 1 --2- 172.21.15.97:0/3813959231 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f1ff00a3df0 0x7f1ff00a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.321+0000 7f200e469740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a9dadc7800 0x55a9dadc9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:10.329 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.321+0000 7f200e469740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55a9dad21970 con 0x55a9dadc7800 2024-04-23T10:30:10.329 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.321+0000 7f200e469740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a9dacbe4a0 con 0x55a9dadbc620 2024-04-23T10:30:10.329 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.321+0000 7f200e469740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a9dad1d930 con 0x55a9dadbcf40 2024-04-23T10:30:10.329 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.322+0000 7f2009ce1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a9dadc7800 0x55a9dadc9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:10.329 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.322+0000 7f2008cdf640 1 --2- 172.21.15.97:0/3813959231 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f1ff00a3df0 0x7f1ff00a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:10.329 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.322+0000 7f2008cdf640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9dadbc620 0x55a9dadbca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:10.329 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.322+0000 7f2008cdf640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9dadbc620 0x55a9dadbca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.97:41608/0 (socket says 172.21.15.97:41608) 2024-04-23T10:30:10.329 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.322+0000 7f20094e0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9dadbcf40 0x55a9dadc72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:10.329 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.322+0000 7f2009ce1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a9dadc7800 0x55a9dadc9bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34816/0 (socket says 172.21.15.97:34816) 2024-04-23T10:30:10.329 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.322+0000 7f2008cdf640 1 -- 172.21.15.97:0/1164942790 learned_addr learned my addr 172.21.15.97:0/1164942790 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:30:10.329 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.322+0000 7f1ffa7fc640 1 -- 172.21.15.97:0/3813959231 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f1ffc0f0eb0 con 0x7f200410c550 2024-04-23T10:30:10.329 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.322+0000 7f2009ce1640 1 -- 172.21.15.97:0/1164942790 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9dadbcf40 msgr2=0x55a9dadc72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:30:10.329 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.322+0000 7f2009ce1640 1 --2- 172.21.15.97:0/1164942790 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9dadbcf40 0x55a9dadc72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:10.330 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.322+0000 7f2009ce1640 1 -- 172.21.15.97:0/1164942790 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9dadbc620 msgr2=0x55a9dadbca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:30:10.330 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.322+0000 7f2009ce1640 1 --2- 172.21.15.97:0/1164942790 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9dadbc620 0x55a9dadbca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:10.330 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.322+0000 7f2009ce1640 1 -- 172.21.15.97:0/1164942790 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9daa34210 con 0x55a9dadc7800 2024-04-23T10:30:10.330 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.322+0000 7f2008cdf640 1 --2- 172.21.15.97:0/1164942790 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9dadbc620 0x55a9dadbca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T10:30:10.330 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.322+0000 7f2009ce1640 1 --2- 172.21.15.97:0/1164942790 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a9dadc7800 0x55a9dadc9bf0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc0ee650 tx=0x7f1ffc08b850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49165cca11f381f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:30:10.330 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.322+0000 7f2008cdf640 1 --2- 172.21.15.97:0/3813959231 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f1ff00a3df0 0x7f1ff00a62b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1ff400bfd0 tx=0x7f1ff4004730 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:30:10.330 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.322+0000 7f1fef7fe640 1 -- 172.21.15.97:0/1164942790 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc049070 con 0x55a9dadc7800 2024-04-23T10:30:10.330 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.322+0000 7f1fef7fe640 1 -- 172.21.15.97:0/1164942790 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ffc0bd5a0 con 0x55a9dadc7800 2024-04-23T10:30:10.330 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.322+0000 7f1fef7fe640 1 -- 172.21.15.97:0/1164942790 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc0c1250 con 0x55a9dadc7800 2024-04-23T10:30:10.330 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.323+0000 7f200e469740 1 -- 172.21.15.97:0/1164942790 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a9dadc7800 msgr2=0x55a9dadc9bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:30:10.330 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.323+0000 7f200e469740 1 --2- 172.21.15.97:0/1164942790 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a9dadc7800 0x55a9dadc9bf0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc0ee650 tx=0x7f1ffc08b850 comp rx=0 tx=0).stop 2024-04-23T10:30:10.330 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.323+0000 7f200e469740 1 -- 172.21.15.97:0/1164942790 shutdown_connections 2024-04-23T10:30:10.330 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.323+0000 7f200e469740 1 --2- 172.21.15.97:0/1164942790 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a9dadc7800 0x55a9dadc9bf0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:10.330 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.323+0000 7f200e469740 1 --2- 172.21.15.97:0/1164942790 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9dadbcf40 0x55a9dadc72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.323+0000 7f200e469740 1 --2- 172.21.15.97:0/1164942790 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9dadbc620 0x55a9dadbca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.323+0000 7f200e469740 1 -- 172.21.15.97:0/1164942790 >> 172.21.15.97:0/1164942790 conn(0x55a9dadb8b30 msgr2=0x55a9dadb8f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:30:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.323+0000 7f200e469740 1 -- 172.21.15.97:0/1164942790 shutdown_connections 2024-04-23T10:30:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.323+0000 7f200e469740 1 -- 172.21.15.97:0/1164942790 wait complete. 2024-04-23T10:30:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.323+0000 7f200e469740 1 -- start start 2024-04-23T10:30:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.323+0000 7f200e469740 1 Processor -- start 2024-04-23T10:30:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.324+0000 7f200e469740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9dadbc620 0x55a9dadc7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.324+0000 7f200e469740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a9dadbcf40 0x55a9dadc4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.324+0000 7f200e469740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9dadc7800 0x55a9dadc49b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.324+0000 7f200e469740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55a9dadca1f0 con 0x55a9dadbcf40 2024-04-23T10:30:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.324+0000 7f200e469740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a9dacbc0b0 con 0x55a9dadc7800 2024-04-23T10:30:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.324+0000 7f200e469740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a9dadca370 con 0x55a9dadbc620 2024-04-23T10:30:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.324+0000 7f200e469740 10 client.0 _ll_register_callbacks cb 0x55a9daca9d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:30:10.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.324+0000 7f2008cdf640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a9dadbcf40 0x55a9dadc4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:10.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.324+0000 7f2008cdf640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a9dadbcf40 0x55a9dadc4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34824/0 (socket says 172.21.15.97:34824) 2024-04-23T10:30:10.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.324+0000 7f2008cdf640 1 -- 172.21.15.97:0/1381044590 learned_addr learned my addr 172.21.15.97:0/1381044590 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:30:10.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.324+0000 7f20094e0640 1 --2- 172.21.15.97:0/1381044590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9dadbc620 0x55a9dadc7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:10.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.324+0000 7f2009ce1640 1 --2- 172.21.15.97:0/1381044590 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9dadc7800 0x55a9dadc49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:10.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.324+0000 7f2008cdf640 1 -- 172.21.15.97:0/1381044590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9dadbc620 msgr2=0x55a9dadc7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:30:10.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.324+0000 7f2008cdf640 1 --2- 172.21.15.97:0/1381044590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9dadbc620 0x55a9dadc7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:10.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.324+0000 7f2008cdf640 1 -- 172.21.15.97:0/1381044590 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9dadc7800 msgr2=0x55a9dadc49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:30:10.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.324+0000 7f2008cdf640 1 --2- 172.21.15.97:0/1381044590 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9dadc7800 0x55a9dadc49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:10.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.324+0000 7f2008cdf640 1 -- 172.21.15.97:0/1381044590 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9dab00620 con 0x55a9dadbcf40 2024-04-23T10:30:10.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.324+0000 7f2009ce1640 1 --2- 172.21.15.97:0/1381044590 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9dadc7800 0x55a9dadc49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:30:10.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.325+0000 7f2008cdf640 1 --2- 172.21.15.97:0/1381044590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a9dadbcf40 0x55a9dadc4450 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f1ff400da00 tx=0x7f1ff40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9b261d635a47b4c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:30:10.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.325+0000 7f1fecff9640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:30:10.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.325+0000 7f1fecff9640 1 -- 172.21.15.97:0/1381044590 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff402e060 con 0x55a9dadbcf40 2024-04-23T10:30:10.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.325+0000 7f1fecff9640 1 -- 172.21.15.97:0/1381044590 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f1ff401f9b0 con 0x55a9dadbcf40 2024-04-23T10:30:10.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.325+0000 7f200e469740 1 -- 172.21.15.97:0/1381044590 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ffc0bd5a0 con 0x55a9dadbcf40 2024-04-23T10:30:10.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.325+0000 7f1fecff9640 1 -- 172.21.15.97:0/1381044590 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1ff401fbb0 con 0x55a9dadbcf40 2024-04-23T10:30:10.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.325+0000 7f200e469740 10 client.16197 fetch_fsmap learned FSMap version 26 2024-04-23T10:30:10.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.325+0000 7f200e469740 1 -- 172.21.15.97:0/1381044590 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a9dadd60f0 con 0x55a9dadbcf40 2024-04-23T10:30:10.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.325+0000 7f1fecff9640 1 -- 172.21.15.97:0/1381044590 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ffc0bd5a0 con 0x55a9dadbcf40 2024-04-23T10:30:10.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.325+0000 7f1fecff9640 1 -- 172.21.15.97:0/1381044590 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1ff4029b70 con 0x55a9dadbcf40 2024-04-23T10:30:10.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.325+0000 7f200e469740 10 client.16197 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:30:10.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.325+0000 7f200e469740 10 client.16197 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:30:10.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.325+0000 7f200e469740 20 client.16197 populate_metadata read hostname 'smithi097' 2024-04-23T10:30:10.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.325+0000 7f200e469740 1 --2- 172.21.15.97:0/1381044590 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55a9dadd75b0 0x55a9dadd99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:10.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.325+0000 7f200e469740 4 client.16197 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:30:10.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.325+0000 7f200e469740 1 -- 172.21.15.97:0/1381044590 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a9dad2f390 con 0x55a9dadd75b0 2024-04-23T10:30:10.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.326+0000 7f20094e0640 1 --2- 172.21.15.97:0/1381044590 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55a9dadd75b0 0x55a9dadd99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:10.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.326+0000 7f20094e0640 1 --2- 172.21.15.97:0/1381044590 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55a9dadd75b0 0x55a9dadd99e0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4acf10e695822976 server_cookie=b7ab700b51e70f6a in_seq=0 out_seq=0 2024-04-23T10:30:10.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.326+0000 7f1fecff9640 10 client.16197 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:30:10.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.327+0000 7f1fecff9640 1 -- 172.21.15.97:0/1381044590 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a9dad2f390 con 0x55a9dadd75b0 2024-04-23T10:30:10.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.327+0000 7f1fecff9640 10 client.16197 handle_command_reply: tid=0 2024-04-23T10:30:10.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.327+0000 7f200e469740 1 client.16197 shutdown 2024-04-23T10:30:10.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.327+0000 7f200e469740 1 -- 172.21.15.97:0/1381044590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a9dadbcf40 msgr2=0x55a9dadc4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:30:10.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.327+0000 7f200e469740 1 --2- 172.21.15.97:0/1381044590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a9dadbcf40 0x55a9dadc4450 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f1ff400da00 tx=0x7f1ff40046c0 comp rx=0 tx=0).stop 2024-04-23T10:30:10.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.327+0000 7f200e469740 1 -- 172.21.15.97:0/1381044590 shutdown_connections 2024-04-23T10:30:10.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.327+0000 7f200e469740 1 --2- 172.21.15.97:0/1381044590 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55a9dadd75b0 0x55a9dadd99e0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:10.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.327+0000 7f200e469740 1 --2- 172.21.15.97:0/1381044590 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9dadc7800 0x55a9dadc49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:10.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.327+0000 7f200e469740 1 --2- 172.21.15.97:0/1381044590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a9dadbcf40 0x55a9dadc4450 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:10.335 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.327+0000 7f1fecff9640 0 client.16197 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:30:10.335 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.327+0000 7f200e469740 1 --2- 172.21.15.97:0/1381044590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9dadbc620 0x55a9dadc7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:10.335 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.327+0000 7f200e469740 1 -- 172.21.15.97:0/1381044590 >> 172.21.15.97:0/1381044590 conn(0x55a9dadb8b30 msgr2=0x55a9dadbe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:30:10.335 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.327+0000 7f200e469740 1 -- 172.21.15.97:0/1381044590 shutdown_connections 2024-04-23T10:30:10.335 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.327+0000 7f200e469740 1 -- 172.21.15.97:0/1381044590 wait complete. 2024-04-23T10:30:10.335 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.327+0000 7f200e469740 20 client.16197 trim_cache size 0 max 16384 2024-04-23T10:30:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.342+0000 7f200e469740 1 Processor -- start 2024-04-23T10:30:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.342+0000 7f200e469740 1 -- start start 2024-04-23T10:30:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.342+0000 7f200e469740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a9dadbc620 0x55a9dadd3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.342+0000 7f200e469740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9dadbcf40 0x55a9dadb7b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:10.346 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.342+0000 7f200e469740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9dadc7800 0x55a9daddf2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:10.346 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.342+0000 7f200e469740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f20041afbc0 con 0x55a9dadbc620 2024-04-23T10:30:10.346 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.342+0000 7f200e469740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f20041b1700 con 0x55a9dadc7800 2024-04-23T10:30:10.346 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.342+0000 7f200e469740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f20041b0290 con 0x55a9dadbcf40 2024-04-23T10:30:10.346 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.342+0000 7f20094e0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a9dadbc620 0x55a9dadd3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:10.346 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.342+0000 7f20094e0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a9dadbc620 0x55a9dadd3020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34836/0 (socket says 172.21.15.97:34836) 2024-04-23T10:30:10.346 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.342+0000 7f20094e0640 1 -- 172.21.15.97:0/2369577071 learned_addr learned my addr 172.21.15.97:0/2369577071 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:30:10.346 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.342+0000 7f20094e0640 1 -- 172.21.15.97:0/2369577071 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9dadbcf40 msgr2=0x55a9dadb7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:30:10.346 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.342+0000 7f2008cdf640 1 --2- 172.21.15.97:0/2369577071 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9dadbcf40 0x55a9dadb7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:10.346 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.342+0000 7f2009ce1640 1 --2- 172.21.15.97:0/2369577071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9dadc7800 0x55a9daddf2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:10.346 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.342+0000 7f20094e0640 1 --2- 172.21.15.97:0/2369577071 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9dadbcf40 0x55a9dadb7b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:10.346 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.342+0000 7f20094e0640 1 -- 172.21.15.97:0/2369577071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9dadc7800 msgr2=0x55a9daddf2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:30:10.346 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.342+0000 7f20094e0640 1 --2- 172.21.15.97:0/2369577071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9dadc7800 0x55a9daddf2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.343+0000 7f20094e0640 1 -- 172.21.15.97:0/2369577071 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9dadbd9f0 con 0x55a9dadbc620 2024-04-23T10:30:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.343+0000 7f2008cdf640 1 --2- 172.21.15.97:0/2369577071 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9dadbcf40 0x55a9dadb7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:30:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.343+0000 7f2009ce1640 1 --2- 172.21.15.97:0/2369577071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9dadc7800 0x55a9daddf2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:30:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.343+0000 7f20094e0640 1 --2- 172.21.15.97:0/2369577071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a9dadbc620 0x55a9dadd3020 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f2000003b00 tx=0x7f2000012bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9ff7fe1e24f5f7d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:30:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.343+0000 7f1feeffd640 1 -- 172.21.15.97:0/2369577071 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2000022020 con 0x55a9dadbc620 2024-04-23T10:30:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.343+0000 7f1feeffd640 1 -- 172.21.15.97:0/2369577071 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f200001e8d0 con 0x55a9dadbc620 2024-04-23T10:30:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.343+0000 7f1feeffd640 1 -- 172.21.15.97:0/2369577071 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20000107d0 con 0x55a9dadbc620 2024-04-23T10:30:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.343+0000 7f200e469740 1 -- 172.21.15.97:0/2369577071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a9dadbc620 msgr2=0x55a9dadd3020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:30:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.343+0000 7f200e469740 1 --2- 172.21.15.97:0/2369577071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a9dadbc620 0x55a9dadd3020 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f2000003b00 tx=0x7f2000012bb0 comp rx=0 tx=0).stop 2024-04-23T10:30:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.343+0000 7f200e469740 1 -- 172.21.15.97:0/2369577071 shutdown_connections 2024-04-23T10:30:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.343+0000 7f200e469740 1 --2- 172.21.15.97:0/2369577071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9dadc7800 0x55a9daddf2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.343+0000 7f200e469740 1 --2- 172.21.15.97:0/2369577071 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9dadbcf40 0x55a9dadb7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.343+0000 7f200e469740 1 --2- 172.21.15.97:0/2369577071 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a9dadbc620 0x55a9dadd3020 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.344+0000 7f200e469740 1 -- 172.21.15.97:0/2369577071 >> 172.21.15.97:0/2369577071 conn(0x55a9dadb8b30 msgr2=0x55a9dadb7420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:30:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.344+0000 7f200e469740 1 -- 172.21.15.97:0/2369577071 shutdown_connections 2024-04-23T10:30:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.344+0000 7f200e469740 1 -- 172.21.15.97:0/2369577071 wait complete. 2024-04-23T10:30:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.344+0000 7f200e469740 1 -- start start 2024-04-23T10:30:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.344+0000 7f200e469740 1 Processor -- start 2024-04-23T10:30:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.344+0000 7f200e469740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9dadbc620 0x55a9dadcd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.344+0000 7f200e469740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9dadbcf40 0x55a9dadcd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.344+0000 7f200e469740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a9dadc7800 0x55a9dadcda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.344+0000 7f200e469740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f20041b1030 con 0x55a9dadc7800 2024-04-23T10:30:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.344+0000 7f200e469740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f20041b0960 con 0x55a9dadbc620 2024-04-23T10:30:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.344+0000 7f200e469740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f20041adbc0 con 0x55a9dadbcf40 2024-04-23T10:30:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.344+0000 7f200e469740 10 client.0 _ll_register_callbacks cb 0x7f1ff4012e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:30:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.344+0000 7f2009ce1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a9dadc7800 0x55a9dadcda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.344+0000 7f2009ce1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a9dadc7800 0x55a9dadcda90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34844/0 (socket says 172.21.15.97:34844) 2024-04-23T10:30:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.344+0000 7f2009ce1640 1 -- 172.21.15.97:0/379024077 learned_addr learned my addr 172.21.15.97:0/379024077 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:30:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.345+0000 7f2008cdf640 1 --2- 172.21.15.97:0/379024077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9dadbcf40 0x55a9dadcd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.345+0000 7f20094e0640 1 --2- 172.21.15.97:0/379024077 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9dadbc620 0x55a9dadcd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.345+0000 7f2009ce1640 1 -- 172.21.15.97:0/379024077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9dadbcf40 msgr2=0x55a9dadcd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:30:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.345+0000 7f2009ce1640 1 --2- 172.21.15.97:0/379024077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9dadbcf40 0x55a9dadcd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.345+0000 7f2009ce1640 1 -- 172.21.15.97:0/379024077 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9dadbc620 msgr2=0x55a9dadcd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:30:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.345+0000 7f2009ce1640 1 --2- 172.21.15.97:0/379024077 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9dadbc620 0x55a9dadcd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.345+0000 7f2009ce1640 1 -- 172.21.15.97:0/379024077 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9dae3cc20 con 0x55a9dadc7800 2024-04-23T10:30:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.345+0000 7f20094e0640 1 --2- 172.21.15.97:0/379024077 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9dadbc620 0x55a9dadcd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:30:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.345+0000 7f2009ce1640 1 --2- 172.21.15.97:0/379024077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a9dadc7800 0x55a9dadcda90 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc0bcba0 tx=0x7f1ffc005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=202d8abc9b29344b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:30:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.345+0000 7f1fecff9640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:30:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.345+0000 7f1fecff9640 1 -- 172.21.15.97:0/379024077 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc049070 con 0x55a9dadc7800 2024-04-23T10:30:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.345+0000 7f200e469740 1 -- 172.21.15.97:0/379024077 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f200001e8d0 con 0x55a9dadc7800 2024-04-23T10:30:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.345+0000 7f1fecff9640 1 -- 172.21.15.97:0/379024077 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f1ffc03bd20 con 0x55a9dadc7800 2024-04-23T10:30:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.345+0000 7f1fecff9640 1 -- 172.21.15.97:0/379024077 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1ffc0bc9e0 con 0x55a9dadc7800 2024-04-23T10:30:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.345+0000 7f200e469740 10 client.16203 fetch_fsmap learned FSMap version 26 2024-04-23T10:30:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.345+0000 7f200e469740 1 -- 172.21.15.97:0/379024077 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a9dae3de80 con 0x55a9dadc7800 2024-04-23T10:30:10.349 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.345+0000 7f1fecff9640 1 -- 172.21.15.97:0/379024077 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f200001e8d0 con 0x55a9dadc7800 2024-04-23T10:30:10.350 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.345+0000 7f1fecff9640 1 -- 172.21.15.97:0/379024077 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1ffc0ee8d0 con 0x55a9dadc7800 2024-04-23T10:30:10.350 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.345+0000 7f200e469740 10 client.16203 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:30:10.350 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.345+0000 7f200e469740 10 client.16203 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:30:10.350 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.345+0000 7f200e469740 20 client.16203 populate_metadata read hostname 'smithi097' 2024-04-23T10:30:10.350 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.346+0000 7f200e469740 1 --2- 172.21.15.97:0/379024077 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55a9dae3e440 0x55a9dae40860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:10.350 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.346+0000 7f200e469740 4 client.16203 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:30:10.350 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.346+0000 7f200e469740 1 -- 172.21.15.97:0/379024077 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a9dad29200 con 0x55a9dae3e440 2024-04-23T10:30:10.350 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.346+0000 7f20094e0640 1 --2- 172.21.15.97:0/379024077 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55a9dae3e440 0x55a9dae40860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:10.350 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.346+0000 7f20094e0640 1 --2- 172.21.15.97:0/379024077 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55a9dae3e440 0x55a9dae40860 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2cedc680a98b942 server_cookie=dbebcf1abd2820e1 in_seq=0 out_seq=0 2024-04-23T10:30:10.350 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.346+0000 7f1fecff9640 10 client.16203 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:30:10.361 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.362+0000 7f1fecff9640 1 -- 172.21.15.97:0/379024077 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a9dad29200 con 0x55a9dae3e440 2024-04-23T10:30:10.361 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.362+0000 7f1fecff9640 10 client.16203 handle_command_reply: tid=0 2024-04-23T10:30:10.361 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.363+0000 7f200e469740 1 client.16203 shutdown 2024-04-23T10:30:10.361 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.363+0000 7f200e469740 1 -- 172.21.15.97:0/379024077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a9dadc7800 msgr2=0x55a9dadcda90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:30:10.361 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.363+0000 7f200e469740 1 --2- 172.21.15.97:0/379024077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a9dadc7800 0x55a9dadcda90 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc0bcba0 tx=0x7f1ffc005be0 comp rx=0 tx=0).stop 2024-04-23T10:30:10.361 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.363+0000 7f200e469740 1 -- 172.21.15.97:0/379024077 shutdown_connections 2024-04-23T10:30:10.362 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.363+0000 7f200e469740 1 --2- 172.21.15.97:0/379024077 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55a9dae3e440 0x55a9dae40860 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:10.362 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.363+0000 7f200e469740 1 --2- 172.21.15.97:0/379024077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a9dadc7800 0x55a9dadcda90 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:10.362 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.363+0000 7f1fecff9640 0 client.16203 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:30:10.362 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.363+0000 7f200e469740 1 --2- 172.21.15.97:0/379024077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9dadbcf40 0x55a9dadcd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:10.362 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.363+0000 7f200e469740 1 --2- 172.21.15.97:0/379024077 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a9dadbc620 0x55a9dadcd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:10.362 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.363+0000 7f200e469740 1 -- 172.21.15.97:0/379024077 >> 172.21.15.97:0/379024077 conn(0x55a9dadb8b30 msgr2=0x55a9dadb7cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:30:10.362 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.363+0000 7f200e469740 1 -- 172.21.15.97:0/379024077 shutdown_connections 2024-04-23T10:30:10.362 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.363+0000 7f200e469740 1 -- 172.21.15.97:0/379024077 wait complete. 2024-04-23T10:30:10.362 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.363+0000 7f200e469740 20 client.16203 trim_cache size 0 max 16384 2024-04-23T10:30:10.362 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:30:10.362 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:30:10.363 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:30:10.363 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:30:10.363 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.363+0000 7f200b744640 1 -- 172.21.15.97:0/3813959231 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f1ff00a3df0 msgr2=0x7f1ff00a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:30:10.364 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.363+0000 7f200b744640 1 --2- 172.21.15.97:0/3813959231 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f1ff00a3df0 0x7f1ff00a62b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1ff400bfd0 tx=0x7f1ff4004730 comp rx=0 tx=0).stop 2024-04-23T10:30:10.364 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.363+0000 7f200b744640 1 -- 172.21.15.97:0/3813959231 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f200410c550 msgr2=0x7f2004101850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:30:10.364 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.363+0000 7f200b744640 1 --2- 172.21.15.97:0/3813959231 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f200410c550 0x7f2004101850 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc004770 tx=0x7f1ffc004270 comp rx=0 tx=0).stop 2024-04-23T10:30:10.364 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.363+0000 7f200b744640 1 -- 172.21.15.97:0/3813959231 shutdown_connections 2024-04-23T10:30:10.364 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.363+0000 7f200b744640 1 --2- 172.21.15.97:0/3813959231 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f1ff00a3df0 0x7f1ff00a62b0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:10.364 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.363+0000 7f200b744640 1 --2- 172.21.15.97:0/3813959231 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f200410c550 0x7f2004101850 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:10.364 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.363+0000 7f200b744640 1 --2- 172.21.15.97:0/3813959231 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2004103aa0 0x7f2004101310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:10.364 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.363+0000 7f200b744640 1 --2- 172.21.15.97:0/3813959231 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2004103180 0x7f2004100dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:10.364 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.363+0000 7f200b744640 1 -- 172.21.15.97:0/3813959231 >> 172.21.15.97:0/3813959231 conn(0x7f2004077fd0 msgr2=0x7f20040fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:30:10.364 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.363+0000 7f200b744640 1 -- 172.21.15.97:0/3813959231 shutdown_connections 2024-04-23T10:30:10.364 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:10.363+0000 7f200b744640 1 -- 172.21.15.97:0/3813959231 wait complete. 2024-04-23T10:30:10.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:30:10 smithi097 ceph-mon[29197]: pgmap v122: 97 pgs: 97 active+clean; 3.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 48 KiB/s wr, 30 op/s 2024-04-23T10:30:10.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:30:10 smithi183 ceph-mon[32317]: pgmap v122: 97 pgs: 97 active+clean; 3.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 48 KiB/s wr, 30 op/s 2024-04-23T10:30:10.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:30:10 smithi196 ceph-mon[32919]: pgmap v122: 97 pgs: 97 active+clean; 3.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 48 KiB/s wr, 30 op/s 2024-04-23T10:30:12.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:30:12 smithi097 ceph-mon[29197]: pgmap v123: 97 pgs: 97 active+clean; 3.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 48 KiB/s wr, 30 op/s 2024-04-23T10:30:12.644 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:30:12 smithi196 ceph-mon[32919]: pgmap v123: 97 pgs: 97 active+clean; 3.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 48 KiB/s wr, 30 op/s 2024-04-23T10:30:12.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:30:12 smithi183 ceph-mon[32317]: pgmap v123: 97 pgs: 97 active+clean; 3.4 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 48 KiB/s wr, 30 op/s 2024-04-23T10:30:14.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:30:14 smithi097 ceph-mon[29197]: pgmap v124: 97 pgs: 97 active+clean; 3.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 66 KiB/s wr, 39 op/s 2024-04-23T10:30:14.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:30:14 smithi183 ceph-mon[32317]: pgmap v124: 97 pgs: 97 active+clean; 3.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 66 KiB/s wr, 39 op/s 2024-04-23T10:30:14.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:30:14 smithi196 ceph-mon[32919]: pgmap v124: 97 pgs: 97 active+clean; 3.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 66 KiB/s wr, 39 op/s 2024-04-23T10:30:16.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:30:16 smithi097 ceph-mon[29197]: pgmap v125: 97 pgs: 97 active+clean; 3.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-04-23T10:30:16.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:30:16 smithi183 ceph-mon[32317]: pgmap v125: 97 pgs: 97 active+clean; 3.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-04-23T10:30:16.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:30:16 smithi196 ceph-mon[32919]: pgmap v125: 97 pgs: 97 active+clean; 3.6 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-04-23T10:30:18.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:30:18 smithi183 ceph-mon[32317]: pgmap v126: 97 pgs: 97 active+clean; 3.7 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-23T10:30:18.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:30:18 smithi196 ceph-mon[32919]: pgmap v126: 97 pgs: 97 active+clean; 3.7 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-23T10:30:18.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:30:18 smithi097 ceph-mon[29197]: pgmap v126: 97 pgs: 97 active+clean; 3.7 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-23T10:30:20.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:30:20 smithi183 ceph-mon[32317]: pgmap v127: 97 pgs: 97 active+clean; 3.7 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-04-23T10:30:20.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:30:20 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:30:20.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:30:20 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:30:20.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:30:20 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:30:20.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:30:20 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:30:20.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:30:20 smithi196 ceph-mon[32919]: pgmap v127: 97 pgs: 97 active+clean; 3.7 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-04-23T10:30:20.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:30:20 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:30:20.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:30:20 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:30:20.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:30:20 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:30:20.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:30:20 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:30:20.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:30:20 smithi097 ceph-mon[29197]: pgmap v127: 97 pgs: 97 active+clean; 3.7 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-04-23T10:30:20.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:30:20 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:30:20.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:30:20 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:30:20.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:30:20 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:30:20.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:30:20 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:30:22.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:30:22 smithi183 ceph-mon[32317]: pgmap v128: 97 pgs: 97 active+clean; 3.7 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-04-23T10:30:22.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:30:22 smithi196 ceph-mon[32919]: pgmap v128: 97 pgs: 97 active+clean; 3.7 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-04-23T10:30:22.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:30:22 smithi097 ceph-mon[29197]: pgmap v128: 97 pgs: 97 active+clean; 3.7 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-04-23T10:30:23.459 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.457+0000 7ff174a10640 1 -- 172.21.15.97:0/4080539425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff17010c520 msgr2=0x7ff17010e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:30:23.459 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.457+0000 7ff174a10640 1 --2- 172.21.15.97:0/4080539425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff17010c520 0x7ff17010e9c0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7ff168009ca0 tx=0x7ff16802a9e0 comp rx=0 tx=0).stop 2024-04-23T10:30:23.459 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.457+0000 7ff174a10640 1 -- 172.21.15.97:0/4080539425 shutdown_connections 2024-04-23T10:30:23.459 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.457+0000 7ff174a10640 1 --2- 172.21.15.97:0/4080539425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff17010c520 0x7ff17010e9c0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:23.459 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.457+0000 7ff174a10640 1 --2- 172.21.15.97:0/4080539425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff170103b00 0x7ff17010bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:23.459 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.457+0000 7ff174a10640 1 --2- 172.21.15.97:0/4080539425 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff1701031e0 0x7ff1701035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:23.459 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.457+0000 7ff174a10640 1 -- 172.21.15.97:0/4080539425 >> 172.21.15.97:0/4080539425 conn(0x7ff170077fd0 msgr2=0x7ff1700783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:30:23.459 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.457+0000 7ff174a10640 1 -- 172.21.15.97:0/4080539425 shutdown_connections 2024-04-23T10:30:23.459 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.457+0000 7ff174a10640 1 -- 172.21.15.97:0/4080539425 wait complete. 2024-04-23T10:30:23.459 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.458+0000 7ff174a10640 1 Processor -- start 2024-04-23T10:30:23.460 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.458+0000 7ff174a10640 1 -- start start 2024-04-23T10:30:23.460 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.458+0000 7ff174a10640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff1701031e0 0x7ff1701986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:23.460 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.458+0000 7ff174a10640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff170103b00 0x7ff170198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:23.460 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.458+0000 7ff174a10640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff17010c520 0x7ff17019cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:23.460 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.458+0000 7ff174a10640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff170112230 con 0x7ff170103b00 2024-04-23T10:30:23.460 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.458+0000 7ff174a10640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff1701120b0 con 0x7ff1701031e0 2024-04-23T10:30:23.460 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.458+0000 7ff174a10640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff1701123b0 con 0x7ff17010c520 2024-04-23T10:30:23.460 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.458+0000 7ff167fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff170103b00 0x7ff170198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:23.460 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.458+0000 7ff167fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff170103b00 0x7ff170198c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60544/0 (socket says 172.21.15.97:60544) 2024-04-23T10:30:23.460 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.458+0000 7ff167fff640 1 -- 172.21.15.97:0/4105597996 learned_addr learned my addr 172.21.15.97:0/4105597996 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:30:23.460 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.459+0000 7ff16ed9d640 1 --2- 172.21.15.97:0/4105597996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff17010c520 0x7ff17019cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:23.460 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.459+0000 7ff16e59c640 1 --2- 172.21.15.97:0/4105597996 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff1701031e0 0x7ff1701986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:23.460 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.459+0000 7ff167fff640 1 -- 172.21.15.97:0/4105597996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff17010c520 msgr2=0x7ff17019cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:30:23.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.459+0000 7ff167fff640 1 --2- 172.21.15.97:0/4105597996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff17010c520 0x7ff17019cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:23.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.459+0000 7ff167fff640 1 -- 172.21.15.97:0/4105597996 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff1701031e0 msgr2=0x7ff1701986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:30:23.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.459+0000 7ff167fff640 1 --2- 172.21.15.97:0/4105597996 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff1701031e0 0x7ff1701986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:23.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.459+0000 7ff167fff640 1 -- 172.21.15.97:0/4105597996 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff17019d6c0 con 0x7ff170103b00 2024-04-23T10:30:23.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.459+0000 7ff16e59c640 1 --2- 172.21.15.97:0/4105597996 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff1701031e0 0x7ff1701986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:30:23.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.459+0000 7ff167fff640 1 --2- 172.21.15.97:0/4105597996 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff170103b00 0x7ff170198c10 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7ff15800d770 tx=0x7ff15800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:30:23.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.459+0000 7ff1677fe640 1 -- 172.21.15.97:0/4105597996 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff158017070 con 0x7ff170103b00 2024-04-23T10:30:23.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.459+0000 7ff1677fe640 1 -- 172.21.15.97:0/4105597996 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1580054d0 con 0x7ff170103b00 2024-04-23T10:30:23.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.459+0000 7ff174a10640 1 -- 172.21.15.97:0/4105597996 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff17019d9b0 con 0x7ff170103b00 2024-04-23T10:30:23.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.459+0000 7ff1677fe640 1 -- 172.21.15.97:0/4105597996 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff158004c30 con 0x7ff170103b00 2024-04-23T10:30:23.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.459+0000 7ff174a10640 1 -- 172.21.15.97:0/4105597996 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1701a5240 con 0x7ff170103b00 2024-04-23T10:30:23.468 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.462+0000 7ff1677fe640 1 -- 172.21.15.97:0/4105597996 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff158004e10 con 0x7ff170103b00 2024-04-23T10:30:23.468 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.462+0000 7ff177801740 1 Processor -- start 2024-04-23T10:30:23.468 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.462+0000 7ff177801740 1 -- start start 2024-04-23T10:30:23.468 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.462+0000 7ff177801740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562687eec620 0x562687eeca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:23.468 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.462+0000 7ff177801740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562687eecf40 0x562687ef72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:23.469 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.462+0000 7ff177801740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562687ef7800 0x562687ef9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:23.469 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.462+0000 7ff177801740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x562687e51970 con 0x562687eecf40 2024-04-23T10:30:23.469 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.462+0000 7ff177801740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562687dee4a0 con 0x562687eec620 2024-04-23T10:30:23.469 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.462+0000 7ff177801740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562687e4d930 con 0x562687ef7800 2024-04-23T10:30:23.469 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.462+0000 7ff1677fe640 1 --2- 172.21.15.97:0/4105597996 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7ff15c0a3dc0 0x7ff15c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:23.469 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.462+0000 7ff16e59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562687eecf40 0x562687ef72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:23.469 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.462+0000 7ff16e59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562687eecf40 0x562687ef72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60560/0 (socket says 172.21.15.97:60560) 2024-04-23T10:30:23.469 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.462+0000 7ff16e59c640 1 -- 172.21.15.97:0/639524455 learned_addr learned my addr 172.21.15.97:0/639524455 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:30:23.469 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.463+0000 7ff16ed9d640 1 --2- 172.21.15.97:0/4105597996 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7ff15c0a3dc0 0x7ff15c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:23.469 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.463+0000 7ff167fff640 1 --2- 172.21.15.97:0/639524455 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562687ef7800 0x562687ef9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:23.469 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.463+0000 7ff1677fe640 1 -- 172.21.15.97:0/4105597996 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7ff1580d0280 con 0x7ff170103b00 2024-04-23T10:30:23.469 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.463+0000 7ff16ed9d640 1 --2- 172.21.15.97:0/639524455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562687eec620 0x562687eeca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:23.469 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.463+0000 7ff16e59c640 1 -- 172.21.15.97:0/639524455 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562687ef7800 msgr2=0x562687ef9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:30:23.469 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.463+0000 7ff16e59c640 1 --2- 172.21.15.97:0/639524455 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562687ef7800 0x562687ef9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:23.470 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.463+0000 7ff16e59c640 1 -- 172.21.15.97:0/639524455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562687eec620 msgr2=0x562687eeca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:30:23.470 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.463+0000 7ff16e59c640 1 --2- 172.21.15.97:0/639524455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562687eec620 0x562687eeca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:23.470 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.463+0000 7ff16e59c640 1 -- 172.21.15.97:0/639524455 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562687b64210 con 0x562687eecf40 2024-04-23T10:30:23.470 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.463+0000 7ff167fff640 1 --2- 172.21.15.97:0/639524455 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562687ef7800 0x562687ef9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:30:23.470 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.463+0000 7ff16e59c640 1 --2- 172.21.15.97:0/639524455 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562687eecf40 0x562687ef72c0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7ff160007850 tx=0x7ff160007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dc141571e4d1a61 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:30:23.470 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.463+0000 7ff16ed9d640 1 --2- 172.21.15.97:0/4105597996 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7ff15c0a3dc0 0x7ff15c0a6280 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff168009a60 tx=0x7ff168004a60 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:30:23.470 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.463+0000 7ff164ff9640 1 -- 172.21.15.97:0/639524455 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16000d950 con 0x562687eecf40 2024-04-23T10:30:23.470 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.463+0000 7ff164ff9640 1 -- 172.21.15.97:0/639524455 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff160018070 con 0x562687eecf40 2024-04-23T10:30:23.470 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.463+0000 7ff164ff9640 1 -- 172.21.15.97:0/639524455 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff160013e10 con 0x562687eecf40 2024-04-23T10:30:23.470 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.464+0000 7ff177801740 1 -- 172.21.15.97:0/639524455 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562687eecf40 msgr2=0x562687ef72c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:30:23.470 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.464+0000 7ff177801740 1 --2- 172.21.15.97:0/639524455 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562687eecf40 0x562687ef72c0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7ff160007850 tx=0x7ff160007d10 comp rx=0 tx=0).stop 2024-04-23T10:30:23.470 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.464+0000 7ff177801740 1 -- 172.21.15.97:0/639524455 shutdown_connections 2024-04-23T10:30:23.470 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.464+0000 7ff177801740 1 --2- 172.21.15.97:0/639524455 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562687ef7800 0x562687ef9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:23.470 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.464+0000 7ff177801740 1 --2- 172.21.15.97:0/639524455 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562687eecf40 0x562687ef72c0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:23.471 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.464+0000 7ff177801740 1 --2- 172.21.15.97:0/639524455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562687eec620 0x562687eeca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:23.471 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.464+0000 7ff177801740 1 -- 172.21.15.97:0/639524455 >> 172.21.15.97:0/639524455 conn(0x562687ee8b30 msgr2=0x562687ee8f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:30:23.471 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.464+0000 7ff177801740 1 -- 172.21.15.97:0/639524455 shutdown_connections 2024-04-23T10:30:23.471 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.464+0000 7ff177801740 1 -- 172.21.15.97:0/639524455 wait complete. 2024-04-23T10:30:23.471 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.464+0000 7ff177801740 1 -- start start 2024-04-23T10:30:23.471 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.465+0000 7ff177801740 1 Processor -- start 2024-04-23T10:30:23.471 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.465+0000 7ff177801740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562687eec620 0x562687ef7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:23.471 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.465+0000 7ff177801740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562687eecf40 0x562687ef4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:23.471 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.465+0000 7ff177801740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562687ef7800 0x562687ef49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:23.471 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.465+0000 7ff177801740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x562687efa430 con 0x562687eec620 2024-04-23T10:30:23.471 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.465+0000 7ff177801740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562687dec0b0 con 0x562687eecf40 2024-04-23T10:30:23.471 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.465+0000 7ff177801740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562687efa5b0 con 0x562687ef7800 2024-04-23T10:30:23.471 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.465+0000 7ff177801740 10 client.0 _ll_register_callbacks cb 0x562687dd9d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:30:23.471 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.465+0000 7ff16e59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562687eec620 0x562687ef7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:23.472 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.465+0000 7ff16e59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562687eec620 0x562687ef7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60574/0 (socket says 172.21.15.97:60574) 2024-04-23T10:30:23.472 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.465+0000 7ff16e59c640 1 -- 172.21.15.97:0/3446844542 learned_addr learned my addr 172.21.15.97:0/3446844542 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:30:23.472 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.465+0000 7ff167fff640 1 --2- 172.21.15.97:0/3446844542 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562687eecf40 0x562687ef4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:23.472 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.465+0000 7ff16ed9d640 1 --2- 172.21.15.97:0/3446844542 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562687ef7800 0x562687ef49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:23.472 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.465+0000 7ff16e59c640 1 -- 172.21.15.97:0/3446844542 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562687ef7800 msgr2=0x562687ef49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:30:23.472 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.465+0000 7ff16e59c640 1 --2- 172.21.15.97:0/3446844542 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562687ef7800 0x562687ef49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:23.472 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.465+0000 7ff16e59c640 1 -- 172.21.15.97:0/3446844542 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562687eecf40 msgr2=0x562687ef4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:30:23.472 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.465+0000 7ff16e59c640 1 --2- 172.21.15.97:0/3446844542 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562687eecf40 0x562687ef4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:23.472 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.465+0000 7ff16e59c640 1 -- 172.21.15.97:0/3446844542 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562687c30620 con 0x562687eec620 2024-04-23T10:30:23.472 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.465+0000 7ff16ed9d640 1 --2- 172.21.15.97:0/3446844542 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562687ef7800 0x562687ef49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:30:23.472 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.465+0000 7ff16e59c640 1 --2- 172.21.15.97:0/3446844542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562687eec620 0x562687ef7180 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7ff160004700 tx=0x7ff160013dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82b4b64ac18722a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:30:23.472 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.465+0000 7ff155ffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:30:23.472 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.466+0000 7ff155ffb640 1 -- 172.21.15.97:0/3446844542 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16000d950 con 0x562687eec620 2024-04-23T10:30:23.472 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.466+0000 7ff155ffb640 1 -- 172.21.15.97:0/3446844542 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7ff16000daf0 con 0x562687eec620 2024-04-23T10:30:23.472 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.466+0000 7ff177801740 1 -- 172.21.15.97:0/3446844542 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff160018070 con 0x562687eec620 2024-04-23T10:30:23.473 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.466+0000 7ff155ffb640 1 -- 172.21.15.97:0/3446844542 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff16000dcf0 con 0x562687eec620 2024-04-23T10:30:23.473 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.466+0000 7ff177801740 10 client.16215 fetch_fsmap learned FSMap version 26 2024-04-23T10:30:23.473 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.466+0000 7ff177801740 1 -- 172.21.15.97:0/3446844542 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562687f062f0 con 0x562687eec620 2024-04-23T10:30:23.473 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.466+0000 7ff155ffb640 1 -- 172.21.15.97:0/3446844542 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff160018070 con 0x562687eec620 2024-04-23T10:30:23.473 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.466+0000 7ff155ffb640 1 -- 172.21.15.97:0/3446844542 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff160027410 con 0x562687eec620 2024-04-23T10:30:23.473 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.466+0000 7ff177801740 10 client.16215 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:30:23.473 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.466+0000 7ff177801740 10 client.16215 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:30:23.473 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.466+0000 7ff177801740 20 client.16215 populate_metadata read hostname 'smithi097' 2024-04-23T10:30:23.473 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.466+0000 7ff177801740 1 --2- 172.21.15.97:0/3446844542 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x562687f07700 0x562687f09af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:23.473 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.466+0000 7ff177801740 4 client.16215 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:30:23.473 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.466+0000 7ff177801740 1 -- 172.21.15.97:0/3446844542 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562687e5f390 con 0x562687f07700 2024-04-23T10:30:23.473 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.466+0000 7ff167fff640 1 --2- 172.21.15.97:0/3446844542 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x562687f07700 0x562687f09af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:23.473 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.467+0000 7ff167fff640 1 --2- 172.21.15.97:0/3446844542 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x562687f07700 0x562687f09af0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8dc2cc47a8da001f server_cookie=f1d31d50ef0ab69a in_seq=0 out_seq=0 2024-04-23T10:30:23.473 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.467+0000 7ff155ffb640 10 client.16215 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:30:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.467+0000 7ff155ffb640 1 -- 172.21.15.97:0/3446844542 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562687e5f390 con 0x562687f07700 2024-04-23T10:30:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.467+0000 7ff155ffb640 10 client.16215 handle_command_reply: tid=0 2024-04-23T10:30:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.468+0000 7ff177801740 1 client.16215 shutdown 2024-04-23T10:30:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.468+0000 7ff177801740 1 -- 172.21.15.97:0/3446844542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562687eec620 msgr2=0x562687ef7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:30:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.468+0000 7ff177801740 1 --2- 172.21.15.97:0/3446844542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562687eec620 0x562687ef7180 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7ff160004700 tx=0x7ff160013dc0 comp rx=0 tx=0).stop 2024-04-23T10:30:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.468+0000 7ff177801740 1 -- 172.21.15.97:0/3446844542 shutdown_connections 2024-04-23T10:30:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.468+0000 7ff177801740 1 --2- 172.21.15.97:0/3446844542 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x562687f07700 0x562687f09af0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.468+0000 7ff177801740 1 --2- 172.21.15.97:0/3446844542 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562687ef7800 0x562687ef49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.468+0000 7ff177801740 1 --2- 172.21.15.97:0/3446844542 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562687eecf40 0x562687ef4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.468+0000 7ff177801740 1 --2- 172.21.15.97:0/3446844542 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562687eec620 0x562687ef7180 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.468+0000 7ff155ffb640 0 client.16215 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:30:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.468+0000 7ff177801740 1 -- 172.21.15.97:0/3446844542 >> 172.21.15.97:0/3446844542 conn(0x562687ee8b30 msgr2=0x562687eee8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:30:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.468+0000 7ff177801740 1 -- 172.21.15.97:0/3446844542 shutdown_connections 2024-04-23T10:30:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.468+0000 7ff177801740 1 -- 172.21.15.97:0/3446844542 wait complete. 2024-04-23T10:30:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.468+0000 7ff177801740 20 client.16215 trim_cache size 0 max 16384 2024-04-23T10:30:23.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.482+0000 7ff177801740 1 Processor -- start 2024-04-23T10:30:23.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.482+0000 7ff177801740 1 -- start start 2024-04-23T10:30:23.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.483+0000 7ff177801740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562687eec620 0x562687f02f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:23.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.483+0000 7ff177801740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562687eecf40 0x562687f0cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:23.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.483+0000 7ff177801740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562687ef7800 0x562687f0d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:23.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.483+0000 7ff177801740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff1701ada60 con 0x562687ef7800 2024-04-23T10:30:23.487 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.483+0000 7ff177801740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff1701af5a0 con 0x562687eec620 2024-04-23T10:30:23.487 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.483+0000 7ff177801740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff1701ae130 con 0x562687eecf40 2024-04-23T10:30:23.487 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.483+0000 7ff16ed9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562687ef7800 0x562687f0d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:23.487 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.483+0000 7ff16ed9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562687ef7800 0x562687f0d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60578/0 (socket says 172.21.15.97:60578) 2024-04-23T10:30:23.487 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.483+0000 7ff16ed9d640 1 -- 172.21.15.97:0/1389073766 learned_addr learned my addr 172.21.15.97:0/1389073766 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:30:23.487 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.483+0000 7ff167fff640 1 --2- 172.21.15.97:0/1389073766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562687eecf40 0x562687f0cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:23.487 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.483+0000 7ff16ed9d640 1 -- 172.21.15.97:0/1389073766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562687eecf40 msgr2=0x562687f0cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:30:23.487 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.483+0000 7ff16e59c640 1 --2- 172.21.15.97:0/1389073766 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562687eec620 0x562687f02f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:23.487 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.483+0000 7ff16ed9d640 1 --2- 172.21.15.97:0/1389073766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562687eecf40 0x562687f0cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:23.487 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.483+0000 7ff16ed9d640 1 -- 172.21.15.97:0/1389073766 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562687eec620 msgr2=0x562687f02f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:30:23.487 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.483+0000 7ff16ed9d640 1 --2- 172.21.15.97:0/1389073766 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562687eec620 0x562687f02f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:23.487 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.483+0000 7ff16ed9d640 1 -- 172.21.15.97:0/1389073766 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562687ef4d60 con 0x562687ef7800 2024-04-23T10:30:23.487 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.483+0000 7ff16ed9d640 1 --2- 172.21.15.97:0/1389073766 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562687ef7800 0x562687f0d060 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7ff16802af00 tx=0x7ff168041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffaadcacd6542965 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:30:23.487 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.483+0000 7ff157fff640 1 -- 172.21.15.97:0/1389073766 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff168056070 con 0x562687ef7800 2024-04-23T10:30:23.487 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.483+0000 7ff157fff640 1 -- 172.21.15.97:0/1389073766 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff16802fc70 con 0x562687ef7800 2024-04-23T10:30:23.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.483+0000 7ff157fff640 1 -- 172.21.15.97:0/1389073766 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff168042040 con 0x562687ef7800 2024-04-23T10:30:23.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.484+0000 7ff177801740 1 -- 172.21.15.97:0/1389073766 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562687ef7800 msgr2=0x562687f0d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:30:23.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.484+0000 7ff177801740 1 --2- 172.21.15.97:0/1389073766 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562687ef7800 0x562687f0d060 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7ff16802af00 tx=0x7ff168041000 comp rx=0 tx=0).stop 2024-04-23T10:30:23.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.484+0000 7ff177801740 1 -- 172.21.15.97:0/1389073766 shutdown_connections 2024-04-23T10:30:23.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.484+0000 7ff177801740 1 --2- 172.21.15.97:0/1389073766 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562687ef7800 0x562687f0d060 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:23.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.484+0000 7ff177801740 1 --2- 172.21.15.97:0/1389073766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562687eecf40 0x562687f0cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:23.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.484+0000 7ff177801740 1 --2- 172.21.15.97:0/1389073766 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562687eec620 0x562687f02f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:23.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.484+0000 7ff177801740 1 -- 172.21.15.97:0/1389073766 >> 172.21.15.97:0/1389073766 conn(0x562687ee8b30 msgr2=0x562687f02800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:30:23.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.484+0000 7ff177801740 1 -- 172.21.15.97:0/1389073766 shutdown_connections 2024-04-23T10:30:23.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.484+0000 7ff177801740 1 -- 172.21.15.97:0/1389073766 wait complete. 2024-04-23T10:30:23.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.484+0000 7ff177801740 1 -- start start 2024-04-23T10:30:23.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.484+0000 7ff177801740 1 Processor -- start 2024-04-23T10:30:23.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.485+0000 7ff177801740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562687eec620 0x562687f65fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:23.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.485+0000 7ff177801740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562687eecf40 0x562687f664f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:23.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.485+0000 7ff177801740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562687ef7800 0x562687f0c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:23.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.485+0000 7ff177801740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff1701aeed0 con 0x562687ef7800 2024-04-23T10:30:23.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.485+0000 7ff177801740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff1701ae800 con 0x562687eec620 2024-04-23T10:30:23.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.485+0000 7ff177801740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff1701aba40 con 0x562687eecf40 2024-04-23T10:30:23.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.485+0000 7ff177801740 10 client.0 _ll_register_callbacks cb 0x7ff1600139f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:30:23.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.485+0000 7ff16ed9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562687ef7800 0x562687f0c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:23.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.485+0000 7ff16ed9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562687ef7800 0x562687f0c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60588/0 (socket says 172.21.15.97:60588) 2024-04-23T10:30:23.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.485+0000 7ff16ed9d640 1 -- 172.21.15.97:0/1477960531 learned_addr learned my addr 172.21.15.97:0/1477960531 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:30:23.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.485+0000 7ff167fff640 1 --2- 172.21.15.97:0/1477960531 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562687eecf40 0x562687f664f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:23.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.485+0000 7ff16e59c640 1 --2- 172.21.15.97:0/1477960531 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562687eec620 0x562687f65fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:23.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.485+0000 7ff16ed9d640 1 -- 172.21.15.97:0/1477960531 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562687eecf40 msgr2=0x562687f664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:30:23.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.485+0000 7ff16ed9d640 1 --2- 172.21.15.97:0/1477960531 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562687eecf40 0x562687f664f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:23.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.485+0000 7ff16ed9d640 1 -- 172.21.15.97:0/1477960531 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562687eec620 msgr2=0x562687f65fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:30:23.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.485+0000 7ff16ed9d640 1 --2- 172.21.15.97:0/1477960531 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562687eec620 0x562687f65fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:23.490 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.485+0000 7ff16ed9d640 1 -- 172.21.15.97:0/1477960531 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562687f0c8b0 con 0x562687ef7800 2024-04-23T10:30:23.490 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.485+0000 7ff16e59c640 1 --2- 172.21.15.97:0/1477960531 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562687eec620 0x562687f65fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:30:23.490 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.485+0000 7ff16ed9d640 1 --2- 172.21.15.97:0/1477960531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562687ef7800 0x562687f0c170 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7ff16802b1a0 tx=0x7ff16802f9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7967db4f3d7b502 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:30:23.490 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.485+0000 7ff155ffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:30:23.490 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.485+0000 7ff155ffb640 1 -- 172.21.15.97:0/1477960531 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff168056070 con 0x562687ef7800 2024-04-23T10:30:23.490 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.485+0000 7ff155ffb640 1 -- 172.21.15.97:0/1477960531 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7ff168038cf0 con 0x562687ef7800 2024-04-23T10:30:23.490 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.485+0000 7ff177801740 1 -- 172.21.15.97:0/1477960531 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff16802fc70 con 0x562687ef7800 2024-04-23T10:30:23.490 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.485+0000 7ff155ffb640 1 -- 172.21.15.97:0/1477960531 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff168033040 con 0x562687ef7800 2024-04-23T10:30:23.490 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.485+0000 7ff177801740 10 client.16221 fetch_fsmap learned FSMap version 26 2024-04-23T10:30:23.490 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.485+0000 7ff177801740 1 -- 172.21.15.97:0/1477960531 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562687f6d7e0 con 0x562687ef7800 2024-04-23T10:30:23.490 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.486+0000 7ff155ffb640 1 -- 172.21.15.97:0/1477960531 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff16802fc70 con 0x562687ef7800 2024-04-23T10:30:23.490 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.486+0000 7ff155ffb640 1 -- 172.21.15.97:0/1477960531 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff168050070 con 0x562687ef7800 2024-04-23T10:30:23.490 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.486+0000 7ff177801740 10 client.16221 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:30:23.491 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.486+0000 7ff177801740 10 client.16221 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:30:23.491 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.486+0000 7ff177801740 20 client.16221 populate_metadata read hostname 'smithi097' 2024-04-23T10:30:23.491 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.486+0000 7ff177801740 1 --2- 172.21.15.97:0/1477960531 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x562687f6ded0 0x562687f702f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:23.491 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.486+0000 7ff177801740 4 client.16221 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:30:23.491 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.486+0000 7ff177801740 1 -- 172.21.15.97:0/1477960531 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562687e59200 con 0x562687f6ded0 2024-04-23T10:30:23.491 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.486+0000 7ff16e59c640 1 --2- 172.21.15.97:0/1477960531 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x562687f6ded0 0x562687f702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:23.491 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.487+0000 7ff16e59c640 1 --2- 172.21.15.97:0/1477960531 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x562687f6ded0 0x562687f702f0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a3007b345d487e8 server_cookie=3338d4b4e346d329 in_seq=0 out_seq=0 2024-04-23T10:30:23.491 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.487+0000 7ff155ffb640 10 client.16221 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:30:23.498 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.499+0000 7ff155ffb640 1 -- 172.21.15.97:0/1477960531 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562687e59200 con 0x562687f6ded0 2024-04-23T10:30:23.498 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.499+0000 7ff155ffb640 10 client.16221 handle_command_reply: tid=0 2024-04-23T10:30:23.498 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.499+0000 7ff177801740 1 client.16221 shutdown 2024-04-23T10:30:23.499 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.499+0000 7ff177801740 1 -- 172.21.15.97:0/1477960531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562687ef7800 msgr2=0x562687f0c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:30:23.499 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.499+0000 7ff177801740 1 --2- 172.21.15.97:0/1477960531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562687ef7800 0x562687f0c170 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7ff16802b1a0 tx=0x7ff16802f9f0 comp rx=0 tx=0).stop 2024-04-23T10:30:23.499 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.499+0000 7ff177801740 1 -- 172.21.15.97:0/1477960531 shutdown_connections 2024-04-23T10:30:23.499 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.499+0000 7ff177801740 1 --2- 172.21.15.97:0/1477960531 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x562687f6ded0 0x562687f702f0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:23.499 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.499+0000 7ff177801740 1 --2- 172.21.15.97:0/1477960531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562687ef7800 0x562687f0c170 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:23.499 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.499+0000 7ff177801740 1 --2- 172.21.15.97:0/1477960531 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562687eecf40 0x562687f664f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:23.499 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.499+0000 7ff177801740 1 --2- 172.21.15.97:0/1477960531 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562687eec620 0x562687f65fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:23.499 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.499+0000 7ff155ffb640 0 client.16221 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:30:23.499 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.499+0000 7ff177801740 1 -- 172.21.15.97:0/1477960531 >> 172.21.15.97:0/1477960531 conn(0x562687ee8b30 msgr2=0x562687ee7cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:30:23.499 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.499+0000 7ff177801740 1 -- 172.21.15.97:0/1477960531 shutdown_connections 2024-04-23T10:30:23.499 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.499+0000 7ff177801740 1 -- 172.21.15.97:0/1477960531 wait complete. 2024-04-23T10:30:23.499 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.499+0000 7ff177801740 20 client.16221 trim_cache size 0 max 16384 2024-04-23T10:30:23.500 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:30:23.500 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:30:23.500 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:30:23.500 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:30:23.501 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.500+0000 7ff174a10640 1 -- 172.21.15.97:0/4105597996 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7ff15c0a3dc0 msgr2=0x7ff15c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:30:23.501 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.500+0000 7ff174a10640 1 --2- 172.21.15.97:0/4105597996 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7ff15c0a3dc0 0x7ff15c0a6280 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff168009a60 tx=0x7ff168004a60 comp rx=0 tx=0).stop 2024-04-23T10:30:23.501 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.500+0000 7ff174a10640 1 -- 172.21.15.97:0/4105597996 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff170103b00 msgr2=0x7ff170198c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:30:23.501 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.500+0000 7ff174a10640 1 --2- 172.21.15.97:0/4105597996 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff170103b00 0x7ff170198c10 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7ff15800d770 tx=0x7ff15800dc30 comp rx=0 tx=0).stop 2024-04-23T10:30:23.501 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.500+0000 7ff174a10640 1 -- 172.21.15.97:0/4105597996 shutdown_connections 2024-04-23T10:30:23.501 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.500+0000 7ff174a10640 1 --2- 172.21.15.97:0/4105597996 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7ff15c0a3dc0 0x7ff15c0a6280 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:23.501 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.500+0000 7ff174a10640 1 --2- 172.21.15.97:0/4105597996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff17010c520 0x7ff17019cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:23.501 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.500+0000 7ff174a10640 1 --2- 172.21.15.97:0/4105597996 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff170103b00 0x7ff170198c10 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:23.501 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.500+0000 7ff174a10640 1 --2- 172.21.15.97:0/4105597996 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff1701031e0 0x7ff1701986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:23.501 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.500+0000 7ff174a10640 1 -- 172.21.15.97:0/4105597996 >> 172.21.15.97:0/4105597996 conn(0x7ff170077fd0 msgr2=0x7ff1700fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:30:23.502 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.500+0000 7ff174a10640 1 -- 172.21.15.97:0/4105597996 shutdown_connections 2024-04-23T10:30:23.502 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:23.500+0000 7ff174a10640 1 -- 172.21.15.97:0/4105597996 wait complete. 2024-04-23T10:30:24.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:30:24 smithi183 ceph-mon[32317]: pgmap v129: 97 pgs: 97 active+clean; 3.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 43 KiB/s wr, 22 op/s 2024-04-23T10:30:24.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:30:24 smithi196 ceph-mon[32919]: pgmap v129: 97 pgs: 97 active+clean; 3.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 43 KiB/s wr, 22 op/s 2024-04-23T10:30:24.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:30:24 smithi097 ceph-mon[29197]: pgmap v129: 97 pgs: 97 active+clean; 3.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 43 KiB/s wr, 22 op/s 2024-04-23T10:30:26.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:30:26 smithi183 ceph-mon[32317]: pgmap v130: 97 pgs: 97 active+clean; 3.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-04-23T10:30:26.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:30:26 smithi196 ceph-mon[32919]: pgmap v130: 97 pgs: 97 active+clean; 3.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-04-23T10:30:26.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:30:26 smithi097 ceph-mon[29197]: pgmap v130: 97 pgs: 97 active+clean; 3.9 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-04-23T10:30:28.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:30:28 smithi183 ceph-mon[32317]: pgmap v131: 97 pgs: 97 active+clean; 4.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 42 KiB/s wr, 22 op/s 2024-04-23T10:30:28.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:30:28 smithi196 ceph-mon[32919]: pgmap v131: 97 pgs: 97 active+clean; 4.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 42 KiB/s wr, 22 op/s 2024-04-23T10:30:28.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:30:28 smithi097 ceph-mon[29197]: pgmap v131: 97 pgs: 97 active+clean; 4.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 42 KiB/s wr, 22 op/s 2024-04-23T10:30:30.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:30:30 smithi183 ceph-mon[32317]: pgmap v132: 97 pgs: 97 active+clean; 4.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 29 KiB/s wr, 15 op/s 2024-04-23T10:30:30.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:30:30 smithi196 ceph-mon[32919]: pgmap v132: 97 pgs: 97 active+clean; 4.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 29 KiB/s wr, 15 op/s 2024-04-23T10:30:30.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:30:30 smithi097 ceph-mon[29197]: pgmap v132: 97 pgs: 97 active+clean; 4.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 29 KiB/s wr, 15 op/s 2024-04-23T10:30:31.466 INFO:tasks.workunit.client.0.smithi097.stdout:64/600: open fd = 3 2024-04-23T10:30:31.466 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.466 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.466 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:30:31.466 INFO:tasks.workunit.client.0.smithi097.stdout:65/600: open fd = 3 2024-04-23T10:30:31.466 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.466 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.466 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:30:31.466 INFO:tasks.workunit.client.0.smithi097.stdout:66/600: open fd = 3 2024-04-23T10:30:31.466 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.466 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.466 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:30:31.467 INFO:tasks.workunit.client.0.smithi097.stdout:67/600: open fd = 3 2024-04-23T10:30:31.467 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.467 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.467 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:30:31.467 INFO:tasks.workunit.client.0.smithi097.stdout:68/600: open fd = 3 2024-04-23T10:30:31.467 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.467 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.467 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:30:31.467 INFO:tasks.workunit.client.0.smithi097.stdout:69/600: open fd = 3 2024-04-23T10:30:31.467 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.467 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.467 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:30:31.467 INFO:tasks.workunit.client.0.smithi097.stdout:70/600: open fd = 3 2024-04-23T10:30:31.467 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.467 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.467 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:30:31.468 INFO:tasks.workunit.client.0.smithi097.stdout:71/600: open fd = 3 2024-04-23T10:30:31.468 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.468 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.468 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:30:31.468 INFO:tasks.workunit.client.0.smithi097.stdout:72/600: open fd = 3 2024-04-23T10:30:31.468 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.468 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.468 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:30:31.468 INFO:tasks.workunit.client.0.smithi097.stdout:73/600: open fd = 3 2024-04-23T10:30:31.468 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.468 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.468 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:30:31.468 INFO:tasks.workunit.client.0.smithi097.stdout:74/600: open fd = 3 2024-04-23T10:30:31.468 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.468 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.468 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:30:31.469 INFO:tasks.workunit.client.0.smithi097.stdout:75/600: open fd = 3 2024-04-23T10:30:31.469 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.469 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.469 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:30:31.469 INFO:tasks.workunit.client.0.smithi097.stdout:76/600: open fd = 3 2024-04-23T10:30:31.469 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.469 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.469 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:30:31.469 INFO:tasks.workunit.client.0.smithi097.stdout:77/600: open fd = 3 2024-04-23T10:30:31.469 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.469 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.469 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:30:31.469 INFO:tasks.workunit.client.0.smithi097.stdout:78/600: open fd = 3 2024-04-23T10:30:31.469 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.469 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.469 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:30:31.470 INFO:tasks.workunit.client.0.smithi097.stdout:79/600: open fd = 3 2024-04-23T10:30:31.470 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.470 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.470 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:30:31.470 INFO:tasks.workunit.client.0.smithi097.stdout:80/600: open fd = 3 2024-04-23T10:30:31.470 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.470 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.470 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:30:31.470 INFO:tasks.workunit.client.0.smithi097.stdout:81/600: open fd = 3 2024-04-23T10:30:31.470 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.470 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.470 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:30:31.470 INFO:tasks.workunit.client.0.smithi097.stdout:82/600: open fd = 3 2024-04-23T10:30:31.470 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.470 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.470 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:30:31.471 INFO:tasks.workunit.client.0.smithi097.stdout:83/600: open fd = 3 2024-04-23T10:30:31.471 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.471 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.471 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:30:31.471 INFO:tasks.workunit.client.0.smithi097.stdout:84/600: open fd = 3 2024-04-23T10:30:31.471 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.471 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.471 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:30:31.471 INFO:tasks.workunit.client.0.smithi097.stdout:85/600: open fd = 3 2024-04-23T10:30:31.471 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.471 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.471 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:30:31.471 INFO:tasks.workunit.client.0.smithi097.stdout:86/600: open fd = 3 2024-04-23T10:30:31.471 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.471 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.471 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:30:31.472 INFO:tasks.workunit.client.0.smithi097.stdout:87/600: open fd = 3 2024-04-23T10:30:31.472 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.472 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.472 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:30:31.472 INFO:tasks.workunit.client.0.smithi097.stdout:88/600: open fd = 3 2024-04-23T10:30:31.472 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.472 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.472 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:30:31.472 INFO:tasks.workunit.client.0.smithi097.stdout:89/600: open fd = 3 2024-04-23T10:30:31.472 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.472 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.472 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:30:31.472 INFO:tasks.workunit.client.0.smithi097.stdout:90/600: open fd = 3 2024-04-23T10:30:31.472 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.472 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.472 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:30:31.473 INFO:tasks.workunit.client.0.smithi097.stdout:91/600: open fd = 3 2024-04-23T10:30:31.473 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.473 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.473 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:30:31.473 INFO:tasks.workunit.client.0.smithi097.stdout:92/600: open fd = 3 2024-04-23T10:30:31.473 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.473 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.473 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:30:31.473 INFO:tasks.workunit.client.0.smithi097.stdout:93/600: open fd = 3 2024-04-23T10:30:31.473 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.473 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.473 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:30:31.473 INFO:tasks.workunit.client.0.smithi097.stdout:94/600: open fd = 3 2024-04-23T10:30:31.473 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.474 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.474 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:30:31.474 INFO:tasks.workunit.client.0.smithi097.stdout:95/600: open fd = 3 2024-04-23T10:30:31.474 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.474 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.474 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:30:31.474 INFO:tasks.workunit.client.0.smithi097.stdout:96/600: open fd = 3 2024-04-23T10:30:31.474 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.474 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.474 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:30:31.474 INFO:tasks.workunit.client.0.smithi097.stdout:97/600: open fd = 3 2024-04-23T10:30:31.474 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.474 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.474 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:30:31.474 INFO:tasks.workunit.client.0.smithi097.stdout:98/600: open fd = 3 2024-04-23T10:30:31.475 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.475 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.475 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:30:31.475 INFO:tasks.workunit.client.0.smithi097.stdout:99/600: open fd = 3 2024-04-23T10:30:31.475 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.475 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.475 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:30:31.475 INFO:tasks.workunit.client.0.smithi097.stdout:100/600: open fd = 3 2024-04-23T10:30:31.475 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.475 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.475 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:30:31.475 INFO:tasks.workunit.client.0.smithi097.stdout:101/600: open fd = 3 2024-04-23T10:30:31.475 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.475 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.475 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:30:31.475 INFO:tasks.workunit.client.0.smithi097.stdout:102/600: open fd = 3 2024-04-23T10:30:31.475 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.476 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.476 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:30:31.476 INFO:tasks.workunit.client.0.smithi097.stdout:103/600: open fd = 3 2024-04-23T10:30:31.476 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.476 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.476 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:30:31.476 INFO:tasks.workunit.client.0.smithi097.stdout:104/600: open fd = 3 2024-04-23T10:30:31.476 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.476 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.476 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:30:31.476 INFO:tasks.workunit.client.0.smithi097.stdout:105/600: open fd = 3 2024-04-23T10:30:31.476 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.476 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.476 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:30:31.476 INFO:tasks.workunit.client.0.smithi097.stdout:106/600: open fd = 3 2024-04-23T10:30:31.476 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.476 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.477 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:30:31.477 INFO:tasks.workunit.client.0.smithi097.stdout:107/600: open fd = 3 2024-04-23T10:30:31.477 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.477 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.477 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:30:31.477 INFO:tasks.workunit.client.0.smithi097.stdout:108/600: open fd = 3 2024-04-23T10:30:31.477 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.477 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.477 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:30:31.477 INFO:tasks.workunit.client.0.smithi097.stdout:109/600: open fd = 3 2024-04-23T10:30:31.477 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.477 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.477 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:30:31.477 INFO:tasks.workunit.client.0.smithi097.stdout:110/600: open fd = 3 2024-04-23T10:30:31.477 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.477 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.477 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:30:31.478 INFO:tasks.workunit.client.0.smithi097.stdout:111/600: open fd = 3 2024-04-23T10:30:31.478 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.478 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.478 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:30:31.478 INFO:tasks.workunit.client.0.smithi097.stdout:112/600: open fd = 3 2024-04-23T10:30:31.478 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.478 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.478 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:30:31.478 INFO:tasks.workunit.client.0.smithi097.stdout:113/600: open fd = 3 2024-04-23T10:30:31.478 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.478 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.478 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:30:31.478 INFO:tasks.workunit.client.0.smithi097.stdout:114/600: open fd = 3 2024-04-23T10:30:31.478 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.478 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.478 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:30:31.479 INFO:tasks.workunit.client.0.smithi097.stdout:115/600: open fd = 3 2024-04-23T10:30:31.479 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.479 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.479 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:30:31.479 INFO:tasks.workunit.client.0.smithi097.stdout:116/600: open fd = 3 2024-04-23T10:30:31.479 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.479 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.479 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:30:31.479 INFO:tasks.workunit.client.0.smithi097.stdout:117/600: open fd = 3 2024-04-23T10:30:31.479 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.479 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.479 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:30:31.479 INFO:tasks.workunit.client.0.smithi097.stdout:118/600: open fd = 3 2024-04-23T10:30:31.479 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.479 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.479 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:30:31.480 INFO:tasks.workunit.client.0.smithi097.stdout:119/600: open fd = 3 2024-04-23T10:30:31.480 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.480 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.480 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:30:31.480 INFO:tasks.workunit.client.0.smithi097.stdout:120/600: open fd = 3 2024-04-23T10:30:31.480 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.480 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.480 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:30:31.480 INFO:tasks.workunit.client.0.smithi097.stdout:121/600: open fd = 3 2024-04-23T10:30:31.480 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.480 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.480 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:30:31.480 INFO:tasks.workunit.client.0.smithi097.stdout:122/600: open fd = 3 2024-04-23T10:30:31.480 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.480 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.481 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:30:31.481 INFO:tasks.workunit.client.0.smithi097.stdout:123/600: open fd = 3 2024-04-23T10:30:31.481 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.481 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.481 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:30:31.481 INFO:tasks.workunit.client.0.smithi097.stdout:124/600: open fd = 3 2024-04-23T10:30:31.481 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.481 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.481 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:30:31.481 INFO:tasks.workunit.client.0.smithi097.stdout:125/600: open fd = 3 2024-04-23T10:30:31.481 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.481 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:31.481 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:30:31.481 INFO:tasks.workunit.client.0.smithi097.stdout:126/600: open fd = 3 2024-04-23T10:30:31.481 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:30:32.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:30:32 smithi183 ceph-mon[32317]: pgmap v133: 97 pgs: 97 active+clean; 4.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 29 KiB/s wr, 15 op/s 2024-04-23T10:30:32.663 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:30:32 smithi097 ceph-mon[29197]: pgmap v133: 97 pgs: 97 active+clean; 4.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 29 KiB/s wr, 15 op/s 2024-04-23T10:30:32.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:30:32 smithi196 ceph-mon[32919]: pgmap v133: 97 pgs: 97 active+clean; 4.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 29 KiB/s wr, 15 op/s 2024-04-23T10:30:34.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:30:34 smithi183 ceph-mon[32317]: pgmap v134: 97 pgs: 97 active+clean; 4.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 43 KiB/s wr, 22 op/s 2024-04-23T10:30:34.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:30:34 smithi196 ceph-mon[32919]: pgmap v134: 97 pgs: 97 active+clean; 4.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 43 KiB/s wr, 22 op/s 2024-04-23T10:30:34.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:30:34 smithi097 ceph-mon[29197]: pgmap v134: 97 pgs: 97 active+clean; 4.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 43 KiB/s wr, 22 op/s 2024-04-23T10:30:36.595 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.596+0000 7f032e410640 1 -- 172.21.15.97:0/1402703567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03281061b0 msgr2=0x7f032810dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:30:36.595 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.596+0000 7f032e410640 1 --2- 172.21.15.97:0/1402703567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03281061b0 0x7f032810dd80 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f0320009ca0 tx=0x7f032002a8c0 comp rx=0 tx=0).stop 2024-04-23T10:30:36.595 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.596+0000 7f032e410640 1 -- 172.21.15.97:0/1402703567 shutdown_connections 2024-04-23T10:30:36.596 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.596+0000 7f032e410640 1 --2- 172.21.15.97:0/1402703567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03281061b0 0x7f032810dd80 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:36.596 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.596+0000 7f032e410640 1 --2- 172.21.15.97:0/1402703567 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0328101a90 0x7f0328105ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:36.596 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.596+0000 7f032e410640 1 --2- 172.21.15.97:0/1402703567 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03281010e0 0x7f03281014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:36.596 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.596+0000 7f032e410640 1 -- 172.21.15.97:0/1402703567 >> 172.21.15.97:0/1402703567 conn(0x7f03280771d0 msgr2=0x7f03280775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:30:36.596 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.596+0000 7f032e410640 1 -- 172.21.15.97:0/1402703567 shutdown_connections 2024-04-23T10:30:36.596 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.596+0000 7f032e410640 1 -- 172.21.15.97:0/1402703567 wait complete. 2024-04-23T10:30:36.596 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.596+0000 7f032e410640 1 Processor -- start 2024-04-23T10:30:36.596 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.596+0000 7f032e410640 1 -- start start 2024-04-23T10:30:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.597+0000 7f032e410640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03281010e0 0x7f03281986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.597+0000 7f032e410640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03281061b0 0x7f0328198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.597+0000 7f032e410640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f032819cf20 0x7f032819d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.597+0000 7f032e410640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0328111120 con 0x7f03281061b0 2024-04-23T10:30:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.597+0000 7f032e410640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0328110fa0 con 0x7f032819cf20 2024-04-23T10:30:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.597+0000 7f032e410640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f03281112a0 con 0x7f03281010e0 2024-04-23T10:30:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.597+0000 7f03277fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03281061b0 0x7f0328198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.597+0000 7f03277fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03281061b0 0x7f0328198bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54708/0 (socket says 172.21.15.97:54708) 2024-04-23T10:30:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.597+0000 7f03277fe640 1 -- 172.21.15.97:0/1904194113 learned_addr learned my addr 172.21.15.97:0/1904194113 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:30:36.599 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.597+0000 7f032c9ad640 1 --2- 172.21.15.97:0/1904194113 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f032819cf20 0x7f032819d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:36.599 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.597+0000 7f0327fff640 1 --2- 172.21.15.97:0/1904194113 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03281010e0 0x7f03281986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:36.599 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.597+0000 7f03277fe640 1 -- 172.21.15.97:0/1904194113 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03281010e0 msgr2=0x7f03281986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:30:36.599 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.597+0000 7f03277fe640 1 --2- 172.21.15.97:0/1904194113 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03281010e0 0x7f03281986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:36.599 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.597+0000 7f03277fe640 1 -- 172.21.15.97:0/1904194113 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f032819cf20 msgr2=0x7f032819d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:30:36.599 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.597+0000 7f03277fe640 1 --2- 172.21.15.97:0/1904194113 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f032819cf20 0x7f032819d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:36.599 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.597+0000 7f03277fe640 1 -- 172.21.15.97:0/1904194113 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f032819da10 con 0x7f03281061b0 2024-04-23T10:30:36.599 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.597+0000 7f03277fe640 1 --2- 172.21.15.97:0/1904194113 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03281061b0 0x7f0328198bf0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f031c00d740 tx=0x7f031c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:30:36.599 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.598+0000 7f03257fa640 1 -- 172.21.15.97:0/1904194113 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031c017070 con 0x7f03281061b0 2024-04-23T10:30:36.599 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.598+0000 7f03257fa640 1 -- 172.21.15.97:0/1904194113 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f031c00de50 con 0x7f03281061b0 2024-04-23T10:30:36.599 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.598+0000 7f032e410640 1 -- 172.21.15.97:0/1904194113 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f032819dd00 con 0x7f03281061b0 2024-04-23T10:30:36.599 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.598+0000 7f03257fa640 1 -- 172.21.15.97:0/1904194113 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031c004ce0 con 0x7f03281061b0 2024-04-23T10:30:36.599 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.598+0000 7f032e410640 1 -- 172.21.15.97:0/1904194113 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03281a54f0 con 0x7f03281061b0 2024-04-23T10:30:36.606 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.600+0000 7f03257fa640 1 -- 172.21.15.97:0/1904194113 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f031c004e80 con 0x7f03281061b0 2024-04-23T10:30:36.606 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.600+0000 7f03311fe740 1 Processor -- start 2024-04-23T10:30:36.606 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.600+0000 7f03311fe740 1 -- start start 2024-04-23T10:30:36.606 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.600+0000 7f03311fe740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c6e7f4520 0x563c6e7f4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:36.606 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.600+0000 7f03311fe740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c6e7f4e40 0x563c6e8036d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:36.606 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.600+0000 7f03311fe740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563c6e803c10 0x563c6e806000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:36.606 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.601+0000 7f03311fe740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x563c6e759970 con 0x563c6e803c10 2024-04-23T10:30:36.606 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.601+0000 7f03311fe740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563c6e6f64a0 con 0x563c6e7f4520 2024-04-23T10:30:36.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.601+0000 7f03311fe740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563c6e755930 con 0x563c6e7f4e40 2024-04-23T10:30:36.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.601+0000 7f03257fa640 1 --2- 172.21.15.97:0/1904194113 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f03100a3e40 0x7f03100a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:36.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.601+0000 7f03277fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563c6e803c10 0x563c6e806000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:36.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.601+0000 7f03277fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563c6e803c10 0x563c6e806000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54722/0 (socket says 172.21.15.97:54722) 2024-04-23T10:30:36.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.601+0000 7f03277fe640 1 -- 172.21.15.97:0/4269309889 learned_addr learned my addr 172.21.15.97:0/4269309889 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:30:36.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.601+0000 7f032c9ad640 1 --2- 172.21.15.97:0/4269309889 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c6e7f4520 0x563c6e7f4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:36.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.601+0000 7f0327fff640 1 --2- 172.21.15.97:0/4269309889 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c6e7f4e40 0x563c6e8036d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:36.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.601+0000 7f032c9ad640 1 --2- 172.21.15.97:0/1904194113 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f03100a3e40 0x7f03100a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:36.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.601+0000 7f03257fa640 1 -- 172.21.15.97:0/1904194113 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f031c0d0050 con 0x7f03281061b0 2024-04-23T10:30:36.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.601+0000 7f03277fe640 1 -- 172.21.15.97:0/4269309889 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c6e7f4e40 msgr2=0x563c6e8036d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:30:36.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.601+0000 7f03277fe640 1 --2- 172.21.15.97:0/4269309889 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c6e7f4e40 0x563c6e8036d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:36.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.601+0000 7f03277fe640 1 -- 172.21.15.97:0/4269309889 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c6e7f4520 msgr2=0x563c6e7f4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:30:36.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.601+0000 7f03277fe640 1 --2- 172.21.15.97:0/4269309889 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c6e7f4520 0x563c6e7f4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:36.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.601+0000 7f03277fe640 1 -- 172.21.15.97:0/4269309889 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c6e46c210 con 0x563c6e803c10 2024-04-23T10:30:36.608 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.601+0000 7f03277fe640 1 --2- 172.21.15.97:0/4269309889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563c6e803c10 0x563c6e806000 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f031c009610 tx=0x7f031c01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a084df7064db3d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:30:36.608 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.601+0000 7f032c9ad640 1 --2- 172.21.15.97:0/1904194113 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f03100a3e40 0x7f03100a6300 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0320009a60 tx=0x7f032002a8c0 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:30:36.608 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.601+0000 7f030e7fc640 1 -- 172.21.15.97:0/4269309889 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031c020070 con 0x563c6e803c10 2024-04-23T10:30:36.608 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.601+0000 7f030e7fc640 1 -- 172.21.15.97:0/4269309889 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f031c0a4cb0 con 0x563c6e803c10 2024-04-23T10:30:36.608 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.601+0000 7f030e7fc640 1 -- 172.21.15.97:0/4269309889 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031c096bb0 con 0x563c6e803c10 2024-04-23T10:30:36.608 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.602+0000 7f03311fe740 1 -- 172.21.15.97:0/4269309889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563c6e803c10 msgr2=0x563c6e806000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:30:36.608 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.602+0000 7f03311fe740 1 --2- 172.21.15.97:0/4269309889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563c6e803c10 0x563c6e806000 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f031c009610 tx=0x7f031c01e000 comp rx=0 tx=0).stop 2024-04-23T10:30:36.608 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.602+0000 7f03311fe740 1 -- 172.21.15.97:0/4269309889 shutdown_connections 2024-04-23T10:30:36.608 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.602+0000 7f03311fe740 1 --2- 172.21.15.97:0/4269309889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563c6e803c10 0x563c6e806000 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:36.608 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.602+0000 7f03311fe740 1 --2- 172.21.15.97:0/4269309889 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c6e7f4e40 0x563c6e8036d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:36.608 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.602+0000 7f03311fe740 1 --2- 172.21.15.97:0/4269309889 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c6e7f4520 0x563c6e7f4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:36.608 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.602+0000 7f03311fe740 1 -- 172.21.15.97:0/4269309889 >> 172.21.15.97:0/4269309889 conn(0x563c6e7f0b30 msgr2=0x563c6e7f0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:30:36.608 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.602+0000 7f03311fe740 1 -- 172.21.15.97:0/4269309889 shutdown_connections 2024-04-23T10:30:36.608 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.602+0000 7f03311fe740 1 -- 172.21.15.97:0/4269309889 wait complete. 2024-04-23T10:30:36.608 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.603+0000 7f03311fe740 1 -- start start 2024-04-23T10:30:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.603+0000 7f03311fe740 1 Processor -- start 2024-04-23T10:30:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.603+0000 7f03311fe740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563c6e7f4520 0x563c6e8035e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.603+0000 7f03311fe740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c6e7f4e40 0x563c6e8008d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.603+0000 7f03311fe740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c6e803c10 0x563c6e800ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.603+0000 7f03311fe740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x563c6e8067e0 con 0x563c6e7f4520 2024-04-23T10:30:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.603+0000 7f03311fe740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563c6e6f40b0 con 0x563c6e7f4e40 2024-04-23T10:30:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.603+0000 7f03311fe740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563c6e806960 con 0x563c6e803c10 2024-04-23T10:30:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.603+0000 7f03311fe740 10 client.0 _ll_register_callbacks cb 0x563c6e6e1d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:30:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.603+0000 7f0327fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563c6e7f4520 0x563c6e8035e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.603+0000 7f0327fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563c6e7f4520 0x563c6e8035e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54726/0 (socket says 172.21.15.97:54726) 2024-04-23T10:30:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.603+0000 7f0327fff640 1 -- 172.21.15.97:0/4067777947 learned_addr learned my addr 172.21.15.97:0/4067777947 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:30:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.603+0000 7f032c9ad640 1 --2- 172.21.15.97:0/4067777947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c6e803c10 0x563c6e800ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.603+0000 7f03277fe640 1 --2- 172.21.15.97:0/4067777947 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c6e7f4e40 0x563c6e8008d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.604+0000 7f0327fff640 1 -- 172.21.15.97:0/4067777947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c6e803c10 msgr2=0x563c6e800ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:30:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.604+0000 7f0327fff640 1 --2- 172.21.15.97:0/4067777947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c6e803c10 0x563c6e800ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.604+0000 7f0327fff640 1 -- 172.21.15.97:0/4067777947 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c6e7f4e40 msgr2=0x563c6e8008d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:30:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.604+0000 7f0327fff640 1 --2- 172.21.15.97:0/4067777947 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c6e7f4e40 0x563c6e8008d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.604+0000 7f0327fff640 1 -- 172.21.15.97:0/4067777947 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c6e538620 con 0x563c6e7f4520 2024-04-23T10:30:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.604+0000 7f03277fe640 1 --2- 172.21.15.97:0/4067777947 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c6e7f4e40 0x563c6e8008d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:30:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.604+0000 7f0327fff640 1 --2- 172.21.15.97:0/4067777947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563c6e7f4520 0x563c6e8035e0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f0318012700 tx=0x7f0318012b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=390aeb3c9388b2cc server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:30:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.604+0000 7f03077fe640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:30:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.604+0000 7f03077fe640 1 -- 172.21.15.97:0/4067777947 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0318004320 con 0x563c6e7f4520 2024-04-23T10:30:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.604+0000 7f03077fe640 1 -- 172.21.15.97:0/4067777947 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f03180035f0 con 0x563c6e7f4520 2024-04-23T10:30:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.604+0000 7f03311fe740 1 -- 172.21.15.97:0/4067777947 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f031c0a4cb0 con 0x563c6e7f4520 2024-04-23T10:30:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.604+0000 7f03077fe640 1 -- 172.21.15.97:0/4067777947 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f03180044c0 con 0x563c6e7f4520 2024-04-23T10:30:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.604+0000 7f03311fe740 10 client.16233 fetch_fsmap learned FSMap version 26 2024-04-23T10:30:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.604+0000 7f03311fe740 1 -- 172.21.15.97:0/4067777947 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563c6e812850 con 0x563c6e7f4520 2024-04-23T10:30:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.604+0000 7f03077fe640 1 -- 172.21.15.97:0/4067777947 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f031c0a4cb0 con 0x563c6e7f4520 2024-04-23T10:30:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.604+0000 7f03077fe640 1 -- 172.21.15.97:0/4067777947 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f0318011b50 con 0x563c6e7f4520 2024-04-23T10:30:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.604+0000 7f03311fe740 10 client.16233 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:30:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.604+0000 7f03311fe740 10 client.16233 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:30:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.604+0000 7f03311fe740 20 client.16233 populate_metadata read hostname 'smithi097' 2024-04-23T10:30:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.604+0000 7f03311fe740 1 --2- 172.21.15.97:0/4067777947 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x563c6e813d10 0x563c6e816120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.604+0000 7f03311fe740 4 client.16233 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:30:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.604+0000 7f03311fe740 1 -- 172.21.15.97:0/4067777947 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c6e767390 con 0x563c6e813d10 2024-04-23T10:30:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.605+0000 7f03277fe640 1 --2- 172.21.15.97:0/4067777947 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x563c6e813d10 0x563c6e816120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.605+0000 7f03277fe640 1 --2- 172.21.15.97:0/4067777947 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x563c6e813d10 0x563c6e816120 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aefd919d56d270b7 server_cookie=4ee4b637575c6a11 in_seq=0 out_seq=0 2024-04-23T10:30:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.605+0000 7f03077fe640 10 client.16233 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:30:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.606+0000 7f03077fe640 1 -- 172.21.15.97:0/4067777947 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563c6e767390 con 0x563c6e813d10 2024-04-23T10:30:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.606+0000 7f03077fe640 10 client.16233 handle_command_reply: tid=0 2024-04-23T10:30:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.606+0000 7f03311fe740 1 client.16233 shutdown 2024-04-23T10:30:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.606+0000 7f03311fe740 1 -- 172.21.15.97:0/4067777947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563c6e7f4520 msgr2=0x563c6e8035e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:30:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.606+0000 7f03311fe740 1 --2- 172.21.15.97:0/4067777947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563c6e7f4520 0x563c6e8035e0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f0318012700 tx=0x7f0318012b00 comp rx=0 tx=0).stop 2024-04-23T10:30:36.612 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.606+0000 7f03311fe740 1 -- 172.21.15.97:0/4067777947 shutdown_connections 2024-04-23T10:30:36.612 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.606+0000 7f03311fe740 1 --2- 172.21.15.97:0/4067777947 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x563c6e813d10 0x563c6e816120 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:36.612 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.606+0000 7f03311fe740 1 --2- 172.21.15.97:0/4067777947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c6e803c10 0x563c6e800ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:36.612 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.606+0000 7f03311fe740 1 --2- 172.21.15.97:0/4067777947 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c6e7f4e40 0x563c6e8008d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:36.612 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.606+0000 7f03077fe640 0 client.16233 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:30:36.612 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.606+0000 7f03311fe740 1 --2- 172.21.15.97:0/4067777947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563c6e7f4520 0x563c6e8035e0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:36.612 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.606+0000 7f03311fe740 1 -- 172.21.15.97:0/4067777947 >> 172.21.15.97:0/4067777947 conn(0x563c6e7f0b30 msgr2=0x563c6e7f69a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:30:36.612 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.606+0000 7f03311fe740 1 -- 172.21.15.97:0/4067777947 shutdown_connections 2024-04-23T10:30:36.612 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.606+0000 7f03311fe740 1 -- 172.21.15.97:0/4067777947 wait complete. 2024-04-23T10:30:36.612 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.606+0000 7f03311fe740 20 client.16233 trim_cache size 0 max 16384 2024-04-23T10:30:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.621+0000 7f03311fe740 1 Processor -- start 2024-04-23T10:30:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.621+0000 7f03311fe740 1 -- start start 2024-04-23T10:30:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.621+0000 7f03311fe740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563c6e7f4520 0x563c6e80f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.621+0000 7f03311fe740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c6e7f4e40 0x563c6e7f0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.621+0000 7f03311fe740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c6e803c10 0x563c6e803110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.621+0000 7f03311fe740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f03281ab7e0 con 0x563c6e7f4520 2024-04-23T10:30:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.621+0000 7f03311fe740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f03281ad320 con 0x563c6e7f4e40 2024-04-23T10:30:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.621+0000 7f03311fe740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f03281abeb0 con 0x563c6e803c10 2024-04-23T10:30:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.621+0000 7f0327fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563c6e7f4520 0x563c6e80f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.621+0000 7f0327fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563c6e7f4520 0x563c6e80f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54742/0 (socket says 172.21.15.97:54742) 2024-04-23T10:30:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.622+0000 7f0327fff640 1 -- 172.21.15.97:0/2563068833 learned_addr learned my addr 172.21.15.97:0/2563068833 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:30:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.622+0000 7f03277fe640 1 --2- 172.21.15.97:0/2563068833 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c6e7f4e40 0x563c6e7f0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.622+0000 7f032c9ad640 1 --2- 172.21.15.97:0/2563068833 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c6e803c10 0x563c6e803110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.622+0000 7f0327fff640 1 -- 172.21.15.97:0/2563068833 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c6e803c10 msgr2=0x563c6e803110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:30:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.622+0000 7f0327fff640 1 --2- 172.21.15.97:0/2563068833 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c6e803c10 0x563c6e803110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.622+0000 7f0327fff640 1 -- 172.21.15.97:0/2563068833 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c6e7f4e40 msgr2=0x563c6e7f0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:30:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.622+0000 7f0327fff640 1 --2- 172.21.15.97:0/2563068833 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c6e7f4e40 0x563c6e7f0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.622+0000 7f0327fff640 1 -- 172.21.15.97:0/2563068833 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c6e801250 con 0x563c6e7f4520 2024-04-23T10:30:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.622+0000 7f0327fff640 1 --2- 172.21.15.97:0/2563068833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563c6e7f4520 0x563c6e80f6e0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f0318005c20 tx=0x7f0318009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26c898835c07c9f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:30:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.622+0000 7f030dffb640 1 -- 172.21.15.97:0/2563068833 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031800f630 con 0x563c6e7f4520 2024-04-23T10:30:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.622+0000 7f030dffb640 1 -- 172.21.15.97:0/2563068833 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f031800fe10 con 0x563c6e7f4520 2024-04-23T10:30:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.622+0000 7f030dffb640 1 -- 172.21.15.97:0/2563068833 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0318011820 con 0x563c6e7f4520 2024-04-23T10:30:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.623+0000 7f03311fe740 1 -- 172.21.15.97:0/2563068833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563c6e7f4520 msgr2=0x563c6e80f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:30:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.623+0000 7f03311fe740 1 --2- 172.21.15.97:0/2563068833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563c6e7f4520 0x563c6e80f6e0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f0318005c20 tx=0x7f0318009640 comp rx=0 tx=0).stop 2024-04-23T10:30:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.623+0000 7f03311fe740 1 -- 172.21.15.97:0/2563068833 shutdown_connections 2024-04-23T10:30:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.623+0000 7f03311fe740 1 --2- 172.21.15.97:0/2563068833 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c6e803c10 0x563c6e803110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.623+0000 7f03311fe740 1 --2- 172.21.15.97:0/2563068833 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c6e7f4e40 0x563c6e7f0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.623+0000 7f03311fe740 1 --2- 172.21.15.97:0/2563068833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563c6e7f4520 0x563c6e80f6e0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.623+0000 7f03311fe740 1 -- 172.21.15.97:0/2563068833 >> 172.21.15.97:0/2563068833 conn(0x563c6e7f0b30 msgr2=0x563c6e7fbf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:30:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.623+0000 7f03311fe740 1 -- 172.21.15.97:0/2563068833 shutdown_connections 2024-04-23T10:30:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.623+0000 7f03311fe740 1 -- 172.21.15.97:0/2563068833 wait complete. 2024-04-23T10:30:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.623+0000 7f03311fe740 1 -- start start 2024-04-23T10:30:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.623+0000 7f03311fe740 1 Processor -- start 2024-04-23T10:30:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.624+0000 7f03311fe740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563c6e7f4520 0x563c6e8076b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.624+0000 7f03311fe740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c6e7f4e40 0x563c6e807bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.624+0000 7f03311fe740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c6e803c10 0x563c6e808130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.624+0000 7f03311fe740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f03281acc50 con 0x563c6e7f4520 2024-04-23T10:30:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.624+0000 7f03311fe740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f03281ac580 con 0x563c6e803c10 2024-04-23T10:30:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.624+0000 7f03311fe740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f03281a97e0 con 0x563c6e7f4e40 2024-04-23T10:30:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.624+0000 7f03311fe740 10 client.0 _ll_register_callbacks cb 0x7f0318009d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:30:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.624+0000 7f0327fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563c6e7f4520 0x563c6e8076b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.624+0000 7f0327fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563c6e7f4520 0x563c6e8076b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54758/0 (socket says 172.21.15.97:54758) 2024-04-23T10:30:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.624+0000 7f0327fff640 1 -- 172.21.15.97:0/3889006119 learned_addr learned my addr 172.21.15.97:0/3889006119 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:30:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.624+0000 7f03277fe640 1 --2- 172.21.15.97:0/3889006119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c6e7f4e40 0x563c6e807bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.624+0000 7f032c9ad640 1 --2- 172.21.15.97:0/3889006119 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c6e803c10 0x563c6e808130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.624+0000 7f0327fff640 1 -- 172.21.15.97:0/3889006119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c6e7f4e40 msgr2=0x563c6e807bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:30:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.624+0000 7f0327fff640 1 --2- 172.21.15.97:0/3889006119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c6e7f4e40 0x563c6e807bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.624+0000 7f0327fff640 1 -- 172.21.15.97:0/3889006119 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c6e803c10 msgr2=0x563c6e808130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:30:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.624+0000 7f0327fff640 1 --2- 172.21.15.97:0/3889006119 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c6e803c10 0x563c6e808130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.624+0000 7f0327fff640 1 -- 172.21.15.97:0/3889006119 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c6e7f4c70 con 0x563c6e7f4520 2024-04-23T10:30:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.624+0000 7f0327fff640 1 --2- 172.21.15.97:0/3889006119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563c6e7f4520 0x563c6e8076b0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f0318012700 tx=0x7f03180117f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68cc2355c118aaf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:30:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.624+0000 7f03077fe640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:30:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.624+0000 7f03077fe640 1 -- 172.21.15.97:0/3889006119 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031800fc50 con 0x563c6e7f4520 2024-04-23T10:30:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.624+0000 7f03077fe640 1 -- 172.21.15.97:0/3889006119 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f0318011b70 con 0x563c6e7f4520 2024-04-23T10:30:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.624+0000 7f03311fe740 1 -- 172.21.15.97:0/3889006119 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f031800fe10 con 0x563c6e7f4520 2024-04-23T10:30:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.624+0000 7f03077fe640 1 -- 172.21.15.97:0/3889006119 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f031803d050 con 0x563c6e7f4520 2024-04-23T10:30:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.625+0000 7f03311fe740 10 client.16239 fetch_fsmap learned FSMap version 26 2024-04-23T10:30:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.625+0000 7f03311fe740 1 -- 172.21.15.97:0/3889006119 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563c6e87a470 con 0x563c6e7f4520 2024-04-23T10:30:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.625+0000 7f03077fe640 1 -- 172.21.15.97:0/3889006119 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f031800fe10 con 0x563c6e7f4520 2024-04-23T10:30:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.625+0000 7f03077fe640 1 -- 172.21.15.97:0/3889006119 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f031802d030 con 0x563c6e7f4520 2024-04-23T10:30:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.625+0000 7f03311fe740 10 client.16239 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:30:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.625+0000 7f03311fe740 10 client.16239 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:30:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.625+0000 7f03311fe740 20 client.16239 populate_metadata read hostname 'smithi097' 2024-04-23T10:30:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.625+0000 7f03311fe740 1 --2- 172.21.15.97:0/3889006119 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x563c6e87aad0 0x563c6e87cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.625+0000 7f03311fe740 4 client.16239 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:30:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.625+0000 7f03311fe740 1 -- 172.21.15.97:0/3889006119 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563c6e761200 con 0x563c6e87aad0 2024-04-23T10:30:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.625+0000 7f03277fe640 1 --2- 172.21.15.97:0/3889006119 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x563c6e87aad0 0x563c6e87cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.625+0000 7f03277fe640 1 --2- 172.21.15.97:0/3889006119 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x563c6e87aad0 0x563c6e87cef0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e372402f35e5283 server_cookie=590456ebf3bd4620 in_seq=0 out_seq=0 2024-04-23T10:30:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.625+0000 7f03077fe640 10 client.16239 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:30:36.640 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.640+0000 7f03077fe640 1 -- 172.21.15.97:0/3889006119 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563c6e761200 con 0x563c6e87aad0 2024-04-23T10:30:36.640 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.640+0000 7f03077fe640 10 client.16239 handle_command_reply: tid=0 2024-04-23T10:30:36.640 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.640+0000 7f03311fe740 1 client.16239 shutdown 2024-04-23T10:30:36.640 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.641+0000 7f03311fe740 1 -- 172.21.15.97:0/3889006119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563c6e7f4520 msgr2=0x563c6e8076b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:30:36.640 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.641+0000 7f03311fe740 1 --2- 172.21.15.97:0/3889006119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563c6e7f4520 0x563c6e8076b0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f0318012700 tx=0x7f03180117f0 comp rx=0 tx=0).stop 2024-04-23T10:30:36.640 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.641+0000 7f03311fe740 1 -- 172.21.15.97:0/3889006119 shutdown_connections 2024-04-23T10:30:36.640 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.641+0000 7f03311fe740 1 --2- 172.21.15.97:0/3889006119 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x563c6e87aad0 0x563c6e87cef0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:36.640 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.641+0000 7f03311fe740 1 --2- 172.21.15.97:0/3889006119 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563c6e803c10 0x563c6e808130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:36.640 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.641+0000 7f03311fe740 1 --2- 172.21.15.97:0/3889006119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563c6e7f4e40 0x563c6e807bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:36.641 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.641+0000 7f03077fe640 0 client.16239 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:30:36.641 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.641+0000 7f03311fe740 1 --2- 172.21.15.97:0/3889006119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563c6e7f4520 0x563c6e8076b0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:36.641 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.641+0000 7f03311fe740 1 -- 172.21.15.97:0/3889006119 >> 172.21.15.97:0/3889006119 conn(0x563c6e7f0b30 msgr2=0x563c6e7fbf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:30:36.641 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.641+0000 7f03311fe740 1 -- 172.21.15.97:0/3889006119 shutdown_connections 2024-04-23T10:30:36.641 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.641+0000 7f03311fe740 1 -- 172.21.15.97:0/3889006119 wait complete. 2024-04-23T10:30:36.641 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.641+0000 7f03311fe740 20 client.16239 trim_cache size 0 max 16384 2024-04-23T10:30:36.641 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:30:36.641 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:30:36.641 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:30:36.641 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:30:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.641+0000 7f032e410640 1 -- 172.21.15.97:0/1904194113 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f03100a3e40 msgr2=0x7f03100a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:30:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.641+0000 7f032e410640 1 --2- 172.21.15.97:0/1904194113 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f03100a3e40 0x7f03100a6300 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0320009a60 tx=0x7f032002a8c0 comp rx=0 tx=0).stop 2024-04-23T10:30:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.641+0000 7f032e410640 1 -- 172.21.15.97:0/1904194113 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03281061b0 msgr2=0x7f0328198bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:30:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.641+0000 7f032e410640 1 --2- 172.21.15.97:0/1904194113 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03281061b0 0x7f0328198bf0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f031c00d740 tx=0x7f031c00dc00 comp rx=0 tx=0).stop 2024-04-23T10:30:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.641+0000 7f032e410640 1 -- 172.21.15.97:0/1904194113 shutdown_connections 2024-04-23T10:30:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.642+0000 7f032e410640 1 --2- 172.21.15.97:0/1904194113 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f03100a3e40 0x7f03100a6300 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.642+0000 7f032e410640 1 --2- 172.21.15.97:0/1904194113 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f032819cf20 0x7f032819d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.642+0000 7f032e410640 1 --2- 172.21.15.97:0/1904194113 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03281061b0 0x7f0328198bf0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.642+0000 7f032e410640 1 --2- 172.21.15.97:0/1904194113 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f03281010e0 0x7f03281986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.642+0000 7f032e410640 1 -- 172.21.15.97:0/1904194113 >> 172.21.15.97:0/1904194113 conn(0x7f03280771d0 msgr2=0x7f0328104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:30:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.642+0000 7f032e410640 1 -- 172.21.15.97:0/1904194113 shutdown_connections 2024-04-23T10:30:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:36.642+0000 7f032e410640 1 -- 172.21.15.97:0/1904194113 wait complete. 2024-04-23T10:30:36.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:30:36 smithi183 ceph-mon[32317]: pgmap v135: 97 pgs: 97 active+clean; 4.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 30 KiB/s wr, 15 op/s 2024-04-23T10:30:36.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:30:36 smithi196 ceph-mon[32919]: pgmap v135: 97 pgs: 97 active+clean; 4.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 30 KiB/s wr, 15 op/s 2024-04-23T10:30:36.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:30:36 smithi097 ceph-mon[29197]: pgmap v135: 97 pgs: 97 active+clean; 4.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 30 KiB/s wr, 15 op/s 2024-04-23T10:30:38.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:30:38 smithi183 ceph-mon[32317]: pgmap v136: 97 pgs: 97 active+clean; 4.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 48 KiB/s wr, 24 op/s 2024-04-23T10:30:38.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:30:38 smithi196 ceph-mon[32919]: pgmap v136: 97 pgs: 97 active+clean; 4.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 48 KiB/s wr, 24 op/s 2024-04-23T10:30:38.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:30:38 smithi097 ceph-mon[29197]: pgmap v136: 97 pgs: 97 active+clean; 4.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 48 KiB/s wr, 24 op/s 2024-04-23T10:30:40.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:30:40 smithi183 ceph-mon[32317]: pgmap v137: 97 pgs: 97 active+clean; 4.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-04-23T10:30:40.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:30:40 smithi196 ceph-mon[32919]: pgmap v137: 97 pgs: 97 active+clean; 4.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-04-23T10:30:40.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:30:40 smithi097 ceph-mon[29197]: pgmap v137: 97 pgs: 97 active+clean; 4.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-04-23T10:30:42.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:30:42 smithi183 ceph-mon[32317]: pgmap v138: 97 pgs: 97 active+clean; 4.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-04-23T10:30:42.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:30:42 smithi196 ceph-mon[32919]: pgmap v138: 97 pgs: 97 active+clean; 4.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-04-23T10:30:42.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:30:42 smithi097 ceph-mon[29197]: pgmap v138: 97 pgs: 97 active+clean; 4.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-04-23T10:30:44.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:30:44 smithi196 ceph-mon[32919]: pgmap v139: 97 pgs: 97 active+clean; 4.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-23T10:30:44.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:30:44 smithi097 ceph-mon[29197]: pgmap v139: 97 pgs: 97 active+clean; 4.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-23T10:30:44.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:30:44 smithi183 ceph-mon[32317]: pgmap v139: 97 pgs: 97 active+clean; 4.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-23T10:30:46.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:30:46 smithi196 ceph-mon[32919]: pgmap v140: 97 pgs: 97 active+clean; 4.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 30 KiB/s wr, 15 op/s 2024-04-23T10:30:46.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:30:46 smithi097 ceph-mon[29197]: pgmap v140: 97 pgs: 97 active+clean; 4.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 30 KiB/s wr, 15 op/s 2024-04-23T10:30:46.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:30:46 smithi183 ceph-mon[32317]: pgmap v140: 97 pgs: 97 active+clean; 4.5 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 30 KiB/s wr, 15 op/s 2024-04-23T10:30:48.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:30:48 smithi196 ceph-mon[32919]: pgmap v141: 97 pgs: 97 active+clean; 4.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 43 KiB/s wr, 22 op/s 2024-04-23T10:30:48.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:30:48 smithi097 ceph-mon[29197]: pgmap v141: 97 pgs: 97 active+clean; 4.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 43 KiB/s wr, 22 op/s 2024-04-23T10:30:48.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:30:48 smithi183 ceph-mon[32317]: pgmap v141: 97 pgs: 97 active+clean; 4.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 43 KiB/s wr, 22 op/s 2024-04-23T10:30:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.737+0000 7f4c45b9b640 1 -- 172.21.15.97:0/3692608773 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c40107290 msgr2=0x7f4c4010ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:30:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.737+0000 7f4c45b9b640 1 --2- 172.21.15.97:0/3692608773 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c40107290 0x7f4c4010ee60 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f4c38009ca0 tx=0x7f4c3802a8c0 comp rx=0 tx=0).stop 2024-04-23T10:30:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.737+0000 7f4c45b9b640 1 -- 172.21.15.97:0/3692608773 shutdown_connections 2024-04-23T10:30:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.737+0000 7f4c45b9b640 1 --2- 172.21.15.97:0/3692608773 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c40107290 0x7f4c4010ee60 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.737+0000 7f4c45b9b640 1 --2- 172.21.15.97:0/3692608773 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c40102b70 0x7f4c40106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.737+0000 7f4c45b9b640 1 --2- 172.21.15.97:0/3692608773 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c401021c0 0x7f4c401025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.737+0000 7f4c45b9b640 1 -- 172.21.15.97:0/3692608773 >> 172.21.15.97:0/3692608773 conn(0x7f4c400fda20 msgr2=0x7f4c400ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:30:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.737+0000 7f4c45b9b640 1 -- 172.21.15.97:0/3692608773 shutdown_connections 2024-04-23T10:30:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.737+0000 7f4c45b9b640 1 -- 172.21.15.97:0/3692608773 wait complete. 2024-04-23T10:30:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.738+0000 7f4c45b9b640 1 Processor -- start 2024-04-23T10:30:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.738+0000 7f4c45b9b640 1 -- start start 2024-04-23T10:30:49.739 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.738+0000 7f4c45b9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c401021c0 0x7f4c40198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:49.739 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.738+0000 7f4c45b9b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c40107290 0x7f4c40198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:49.740 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.738+0000 7f4c45b9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c4019cfd0 0x7f4c4019d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:49.740 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.738+0000 7f4c45b9b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4c40112220 con 0x7f4c401021c0 2024-04-23T10:30:49.740 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.738+0000 7f4c45b9b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4c401120a0 con 0x7f4c40107290 2024-04-23T10:30:49.740 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.738+0000 7f4c45b9b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4c401123a0 con 0x7f4c4019cfd0 2024-04-23T10:30:49.740 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.738+0000 7f4c3f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c401021c0 0x7f4c40198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:49.740 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.738+0000 7f4c3f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c401021c0 0x7f4c40198760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60550/0 (socket says 172.21.15.97:60550) 2024-04-23T10:30:49.740 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.738+0000 7f4c3f7fe640 1 -- 172.21.15.97:0/3474690258 learned_addr learned my addr 172.21.15.97:0/3474690258 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:30:49.740 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.739+0000 7f4c3effd640 1 --2- 172.21.15.97:0/3474690258 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c40107290 0x7f4c40198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:49.740 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.739+0000 7f4c3ffff640 1 --2- 172.21.15.97:0/3474690258 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c4019cfd0 0x7f4c4019d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:49.740 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.739+0000 7f4c3f7fe640 1 -- 172.21.15.97:0/3474690258 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c4019cfd0 msgr2=0x7f4c4019d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:30:49.740 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.739+0000 7f4c3f7fe640 1 --2- 172.21.15.97:0/3474690258 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c4019cfd0 0x7f4c4019d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:49.740 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.739+0000 7f4c3f7fe640 1 -- 172.21.15.97:0/3474690258 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c40107290 msgr2=0x7f4c40198ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:30:49.740 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.739+0000 7f4c3f7fe640 1 --2- 172.21.15.97:0/3474690258 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c40107290 0x7f4c40198ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:49.740 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.739+0000 7f4c3f7fe640 1 -- 172.21.15.97:0/3474690258 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c4019dac0 con 0x7f4c401021c0 2024-04-23T10:30:49.740 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.739+0000 7f4c3f7fe640 1 --2- 172.21.15.97:0/3474690258 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c401021c0 0x7f4c40198760 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f4c3000d770 tx=0x7f4c3000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:30:49.741 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.739+0000 7f4c3cff9640 1 -- 172.21.15.97:0/3474690258 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c30017070 con 0x7f4c401021c0 2024-04-23T10:30:49.741 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.739+0000 7f4c45b9b640 1 -- 172.21.15.97:0/3474690258 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c4019ddb0 con 0x7f4c401021c0 2024-04-23T10:30:49.741 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.739+0000 7f4c3cff9640 1 -- 172.21.15.97:0/3474690258 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c300054d0 con 0x7f4c401021c0 2024-04-23T10:30:49.741 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.739+0000 7f4c3cff9640 1 -- 172.21.15.97:0/3474690258 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c30004c30 con 0x7f4c401021c0 2024-04-23T10:30:49.741 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.739+0000 7f4c45b9b640 1 -- 172.21.15.97:0/3474690258 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c401a55a0 con 0x7f4c401021c0 2024-04-23T10:30:49.747 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.742+0000 7f4c3cff9640 1 -- 172.21.15.97:0/3474690258 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4c30004e10 con 0x7f4c401021c0 2024-04-23T10:30:49.747 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.742+0000 7f4c48992740 1 Processor -- start 2024-04-23T10:30:49.748 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.742+0000 7f4c48992740 1 -- start start 2024-04-23T10:30:49.748 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.742+0000 7f4c48992740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e0a260620 0x561e0a260a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:49.748 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.742+0000 7f4c48992740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e0a260f40 0x561e0a26b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:49.748 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.742+0000 7f4c3cff9640 1 --2- 172.21.15.97:0/3474690258 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f4c280a3dc0 0x7f4c280a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:49.748 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.742+0000 7f4c48992740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e0a26b800 0x561e0a26dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:49.748 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.742+0000 7f4c48992740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x561e0a1c5970 con 0x561e0a260f40 2024-04-23T10:30:49.748 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.742+0000 7f4c48992740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561e0a1624a0 con 0x561e0a260620 2024-04-23T10:30:49.748 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.742+0000 7f4c48992740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561e0a1c1930 con 0x561e0a26b800 2024-04-23T10:30:49.748 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.742+0000 7f4c3effd640 1 --2- 172.21.15.97:0/3474690258 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f4c280a3dc0 0x7f4c280a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:49.748 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.742+0000 7f4c3f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e0a260f40 0x561e0a26b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:49.748 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.742+0000 7f4c3f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e0a260f40 0x561e0a26b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60562/0 (socket says 172.21.15.97:60562) 2024-04-23T10:30:49.748 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.742+0000 7f4c3f7fe640 1 -- 172.21.15.97:0/1460507458 learned_addr learned my addr 172.21.15.97:0/1460507458 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:30:49.748 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.742+0000 7f4c3cff9640 1 -- 172.21.15.97:0/3474690258 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f4c300d0280 con 0x7f4c401021c0 2024-04-23T10:30:49.748 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.743+0000 7f4c3ffff640 1 --2- 172.21.15.97:0/1460507458 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e0a26b800 0x561e0a26dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:49.748 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.743+0000 7f4c3ffff640 1 --2- 172.21.15.97:0/1460507458 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e0a260620 0x561e0a260a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:49.749 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.743+0000 7f4c3f7fe640 1 -- 172.21.15.97:0/1460507458 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e0a26b800 msgr2=0x561e0a26dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:30:49.749 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.743+0000 7f4c3f7fe640 1 --2- 172.21.15.97:0/1460507458 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e0a26b800 0x561e0a26dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:49.749 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.743+0000 7f4c3f7fe640 1 -- 172.21.15.97:0/1460507458 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e0a260620 msgr2=0x561e0a260a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:30:49.749 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.743+0000 7f4c3f7fe640 1 --2- 172.21.15.97:0/1460507458 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e0a260620 0x561e0a260a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:49.749 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.743+0000 7f4c3f7fe640 1 -- 172.21.15.97:0/1460507458 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e09ed8210 con 0x561e0a260f40 2024-04-23T10:30:49.749 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.743+0000 7f4c3ffff640 1 --2- 172.21.15.97:0/1460507458 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e0a260620 0x561e0a260a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:30:49.749 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.743+0000 7f4c3effd640 1 --2- 172.21.15.97:0/3474690258 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f4c280a3dc0 0x7f4c280a6280 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4c40108890 tx=0x7f4c3400b390 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:30:49.749 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.743+0000 7f4c3f7fe640 1 --2- 172.21.15.97:0/1460507458 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e0a260f40 0x561e0a26b2c0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f4c3009c200 tx=0x7f4c3006bb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ed3f0035d158cc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:30:49.749 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.743+0000 7f4c25ffb640 1 -- 172.21.15.97:0/1460507458 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c300cf070 con 0x561e0a260f40 2024-04-23T10:30:49.749 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.743+0000 7f4c25ffb640 1 -- 172.21.15.97:0/1460507458 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c3009c970 con 0x561e0a260f40 2024-04-23T10:30:49.749 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.743+0000 7f4c25ffb640 1 -- 172.21.15.97:0/1460507458 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c30020070 con 0x561e0a260f40 2024-04-23T10:30:49.749 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.744+0000 7f4c48992740 1 -- 172.21.15.97:0/1460507458 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e0a260f40 msgr2=0x561e0a26b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:30:49.749 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.744+0000 7f4c48992740 1 --2- 172.21.15.97:0/1460507458 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e0a260f40 0x561e0a26b2c0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f4c3009c200 tx=0x7f4c3006bb50 comp rx=0 tx=0).stop 2024-04-23T10:30:49.749 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.744+0000 7f4c48992740 1 -- 172.21.15.97:0/1460507458 shutdown_connections 2024-04-23T10:30:49.749 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.744+0000 7f4c48992740 1 --2- 172.21.15.97:0/1460507458 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e0a26b800 0x561e0a26dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:49.749 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.744+0000 7f4c48992740 1 --2- 172.21.15.97:0/1460507458 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e0a260f40 0x561e0a26b2c0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:49.750 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.744+0000 7f4c48992740 1 --2- 172.21.15.97:0/1460507458 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e0a260620 0x561e0a260a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:49.750 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.744+0000 7f4c48992740 1 -- 172.21.15.97:0/1460507458 >> 172.21.15.97:0/1460507458 conn(0x561e0a25cb30 msgr2=0x561e0a25cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:30:49.750 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.744+0000 7f4c48992740 1 -- 172.21.15.97:0/1460507458 shutdown_connections 2024-04-23T10:30:49.750 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.744+0000 7f4c48992740 1 -- 172.21.15.97:0/1460507458 wait complete. 2024-04-23T10:30:49.750 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.744+0000 7f4c48992740 1 -- start start 2024-04-23T10:30:49.750 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.744+0000 7f4c48992740 1 Processor -- start 2024-04-23T10:30:49.750 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.745+0000 7f4c48992740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e0a260620 0x561e0a26b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:49.750 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.745+0000 7f4c48992740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e0a260f40 0x561e0a2684e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:49.750 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.745+0000 7f4c48992740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e0a26b800 0x561e0a268af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:49.750 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.745+0000 7f4c48992740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x561e0a26e430 con 0x561e0a260620 2024-04-23T10:30:49.750 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.745+0000 7f4c48992740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561e0a1600b0 con 0x561e0a26b800 2024-04-23T10:30:49.750 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.745+0000 7f4c48992740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561e0a26e5b0 con 0x561e0a260f40 2024-04-23T10:30:49.750 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.745+0000 7f4c48992740 10 client.0 _ll_register_callbacks cb 0x561e0a14dd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:30:49.750 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.745+0000 7f4c3f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e0a260620 0x561e0a26b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:49.751 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.745+0000 7f4c3f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e0a260620 0x561e0a26b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60576/0 (socket says 172.21.15.97:60576) 2024-04-23T10:30:49.751 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.745+0000 7f4c3f7fe640 1 -- 172.21.15.97:0/3896148954 learned_addr learned my addr 172.21.15.97:0/3896148954 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:30:49.751 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.745+0000 7f4c3ffff640 1 --2- 172.21.15.97:0/3896148954 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e0a26b800 0x561e0a268af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:49.751 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.745+0000 7f4c3effd640 1 --2- 172.21.15.97:0/3896148954 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e0a260f40 0x561e0a2684e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:49.751 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.745+0000 7f4c3f7fe640 1 -- 172.21.15.97:0/3896148954 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e0a260f40 msgr2=0x561e0a2684e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:30:49.751 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.745+0000 7f4c3f7fe640 1 --2- 172.21.15.97:0/3896148954 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e0a260f40 0x561e0a2684e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:49.751 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.745+0000 7f4c3f7fe640 1 -- 172.21.15.97:0/3896148954 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e0a26b800 msgr2=0x561e0a268af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:30:49.751 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.745+0000 7f4c3f7fe640 1 --2- 172.21.15.97:0/3896148954 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e0a26b800 0x561e0a268af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:49.751 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.745+0000 7f4c3f7fe640 1 -- 172.21.15.97:0/3896148954 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e09fa4620 con 0x561e0a260620 2024-04-23T10:30:49.751 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.745+0000 7f4c3ffff640 1 --2- 172.21.15.97:0/3896148954 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e0a26b800 0x561e0a268af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:30:49.751 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.745+0000 7f4c3f7fe640 1 --2- 172.21.15.97:0/3896148954 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e0a260620 0x561e0a26b1f0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f4c3009bf70 tx=0x7f4c30095370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa39d6c94da3ace6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:30:49.751 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.745+0000 7f4c1effd640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:30:49.751 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.745+0000 7f4c1effd640 1 -- 172.21.15.97:0/3896148954 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c300cf070 con 0x561e0a260620 2024-04-23T10:30:49.751 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.745+0000 7f4c1effd640 1 -- 172.21.15.97:0/3896148954 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f4c300988e0 con 0x561e0a260620 2024-04-23T10:30:49.751 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.745+0000 7f4c48992740 1 -- 172.21.15.97:0/3896148954 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c3009c970 con 0x561e0a260620 2024-04-23T10:30:49.752 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.746+0000 7f4c1effd640 1 -- 172.21.15.97:0/3896148954 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4c30020050 con 0x561e0a260620 2024-04-23T10:30:49.752 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.746+0000 7f4c48992740 10 client.16251 fetch_fsmap learned FSMap version 26 2024-04-23T10:30:49.752 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.746+0000 7f4c48992740 1 -- 172.21.15.97:0/3896148954 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561e0a27a3e0 con 0x561e0a260620 2024-04-23T10:30:49.752 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.746+0000 7f4c1effd640 1 -- 172.21.15.97:0/3896148954 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c3009c970 con 0x561e0a260620 2024-04-23T10:30:49.752 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.746+0000 7f4c1effd640 1 -- 172.21.15.97:0/3896148954 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4c3006bbc0 con 0x561e0a260620 2024-04-23T10:30:49.752 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.746+0000 7f4c48992740 10 client.16251 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:30:49.752 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.746+0000 7f4c48992740 10 client.16251 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:30:49.752 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.746+0000 7f4c48992740 20 client.16251 populate_metadata read hostname 'smithi097' 2024-04-23T10:30:49.752 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.746+0000 7f4c48992740 1 --2- 172.21.15.97:0/3896148954 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x561e0a27b730 0x561e0a27db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:49.752 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.746+0000 7f4c48992740 4 client.16251 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:30:49.752 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.746+0000 7f4c48992740 1 -- 172.21.15.97:0/3896148954 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e0a1d3390 con 0x561e0a27b730 2024-04-23T10:30:49.752 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.746+0000 7f4c3effd640 1 --2- 172.21.15.97:0/3896148954 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x561e0a27b730 0x561e0a27db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:49.752 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.747+0000 7f4c1effd640 10 client.16251 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:30:49.752 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.747+0000 7f4c3effd640 1 --2- 172.21.15.97:0/3896148954 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x561e0a27b730 0x561e0a27db20 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7bc4818d555206b server_cookie=867a955599dc40b3 in_seq=0 out_seq=0 2024-04-23T10:30:49.752 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.748+0000 7f4c1effd640 1 -- 172.21.15.97:0/3896148954 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561e0a1d3390 con 0x561e0a27b730 2024-04-23T10:30:49.753 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.748+0000 7f4c1effd640 10 client.16251 handle_command_reply: tid=0 2024-04-23T10:30:49.753 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.748+0000 7f4c48992740 1 client.16251 shutdown 2024-04-23T10:30:49.753 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.748+0000 7f4c48992740 1 -- 172.21.15.97:0/3896148954 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e0a260620 msgr2=0x561e0a26b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:30:49.753 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.748+0000 7f4c48992740 1 --2- 172.21.15.97:0/3896148954 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e0a260620 0x561e0a26b1f0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f4c3009bf70 tx=0x7f4c30095370 comp rx=0 tx=0).stop 2024-04-23T10:30:49.753 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.748+0000 7f4c48992740 1 -- 172.21.15.97:0/3896148954 shutdown_connections 2024-04-23T10:30:49.753 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.748+0000 7f4c48992740 1 --2- 172.21.15.97:0/3896148954 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x561e0a27b730 0x561e0a27db20 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:49.753 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.748+0000 7f4c48992740 1 --2- 172.21.15.97:0/3896148954 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e0a26b800 0x561e0a268af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:49.753 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.748+0000 7f4c48992740 1 --2- 172.21.15.97:0/3896148954 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e0a260f40 0x561e0a2684e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:49.753 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.748+0000 7f4c1effd640 0 client.16251 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:30:49.753 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.748+0000 7f4c48992740 1 --2- 172.21.15.97:0/3896148954 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e0a260620 0x561e0a26b1f0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:49.753 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.748+0000 7f4c48992740 1 -- 172.21.15.97:0/3896148954 >> 172.21.15.97:0/3896148954 conn(0x561e0a25cb30 msgr2=0x561e0a2628b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:30:49.753 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.748+0000 7f4c48992740 1 -- 172.21.15.97:0/3896148954 shutdown_connections 2024-04-23T10:30:49.753 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.748+0000 7f4c48992740 1 -- 172.21.15.97:0/3896148954 wait complete. 2024-04-23T10:30:49.753 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.748+0000 7f4c48992740 20 client.16251 trim_cache size 0 max 16384 2024-04-23T10:30:49.767 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.763+0000 7f4c48992740 1 Processor -- start 2024-04-23T10:30:49.767 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.763+0000 7f4c48992740 1 -- start start 2024-04-23T10:30:49.767 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.763+0000 7f4c48992740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e0a260620 0x561e0a2772f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:49.767 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.763+0000 7f4c48992740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e0a260f40 0x561e0a25c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:49.767 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.763+0000 7f4c48992740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e0a26b800 0x561e0a26ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:49.767 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.763+0000 7f4c48992740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4c401ada50 con 0x561e0a26b800 2024-04-23T10:30:49.767 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.763+0000 7f4c48992740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4c401af590 con 0x561e0a260620 2024-04-23T10:30:49.767 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.763+0000 7f4c48992740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4c401ae120 con 0x561e0a260f40 2024-04-23T10:30:49.767 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.763+0000 7f4c3ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e0a26b800 0x561e0a26ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:49.767 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.763+0000 7f4c3ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e0a26b800 0x561e0a26ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60580/0 (socket says 172.21.15.97:60580) 2024-04-23T10:30:49.767 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.763+0000 7f4c3ffff640 1 -- 172.21.15.97:0/945313127 learned_addr learned my addr 172.21.15.97:0/945313127 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:30:49.767 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.764+0000 7f4c3effd640 1 --2- 172.21.15.97:0/945313127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e0a260f40 0x561e0a25c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:49.767 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.764+0000 7f4c3ffff640 1 -- 172.21.15.97:0/945313127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e0a260f40 msgr2=0x561e0a25c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:30:49.767 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.764+0000 7f4c3ffff640 1 --2- 172.21.15.97:0/945313127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e0a260f40 0x561e0a25c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.764+0000 7f4c3ffff640 1 -- 172.21.15.97:0/945313127 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e0a260620 msgr2=0x561e0a2772f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:30:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.764+0000 7f4c3ffff640 1 --2- 172.21.15.97:0/945313127 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e0a260620 0x561e0a2772f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.764+0000 7f4c3ffff640 1 -- 172.21.15.97:0/945313127 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e0a258500 con 0x561e0a26b800 2024-04-23T10:30:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.764+0000 7f4c3effd640 1 --2- 172.21.15.97:0/945313127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e0a260f40 0x561e0a25c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:30:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.764+0000 7f4c3ffff640 1 --2- 172.21.15.97:0/945313127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e0a26b800 0x561e0a26ad20 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f4c38009ca0 tx=0x7f4c380098b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd6d3ad866b33fb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:30:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.764+0000 7f4c257fa640 1 -- 172.21.15.97:0/945313127 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c38044070 con 0x561e0a26b800 2024-04-23T10:30:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.764+0000 7f4c257fa640 1 -- 172.21.15.97:0/945313127 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c38036040 con 0x561e0a26b800 2024-04-23T10:30:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.764+0000 7f4c257fa640 1 -- 172.21.15.97:0/945313127 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c380042a0 con 0x561e0a26b800 2024-04-23T10:30:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.764+0000 7f4c48992740 1 -- 172.21.15.97:0/945313127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e0a26b800 msgr2=0x561e0a26ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:30:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.764+0000 7f4c48992740 1 --2- 172.21.15.97:0/945313127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e0a26b800 0x561e0a26ad20 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f4c38009ca0 tx=0x7f4c380098b0 comp rx=0 tx=0).stop 2024-04-23T10:30:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.765+0000 7f4c48992740 1 -- 172.21.15.97:0/945313127 shutdown_connections 2024-04-23T10:30:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.765+0000 7f4c48992740 1 --2- 172.21.15.97:0/945313127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e0a26b800 0x561e0a26ad20 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.765+0000 7f4c48992740 1 --2- 172.21.15.97:0/945313127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e0a260f40 0x561e0a25c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.765+0000 7f4c48992740 1 --2- 172.21.15.97:0/945313127 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e0a260620 0x561e0a2772f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.765+0000 7f4c48992740 1 -- 172.21.15.97:0/945313127 >> 172.21.15.97:0/945313127 conn(0x561e0a25cb30 msgr2=0x561e0a267570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:30:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.765+0000 7f4c48992740 1 -- 172.21.15.97:0/945313127 shutdown_connections 2024-04-23T10:30:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.765+0000 7f4c48992740 1 -- 172.21.15.97:0/945313127 wait complete. 2024-04-23T10:30:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.765+0000 7f4c48992740 1 -- start start 2024-04-23T10:30:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.765+0000 7f4c48992740 1 Processor -- start 2024-04-23T10:30:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.765+0000 7f4c48992740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e0a260620 0x561e0a273ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.765+0000 7f4c48992740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e0a260f40 0x561e0a264ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.765+0000 7f4c48992740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e0a26b800 0x561e0a265430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.765+0000 7f4c48992740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4c401aeec0 con 0x561e0a260620 2024-04-23T10:30:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.765+0000 7f4c48992740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4c401ae7f0 con 0x561e0a26b800 2024-04-23T10:30:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.765+0000 7f4c48992740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4c401aba50 con 0x561e0a260f40 2024-04-23T10:30:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.765+0000 7f4c48992740 10 client.0 _ll_register_callbacks cb 0x7f4c300a4ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:30:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.765+0000 7f4c3f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e0a260620 0x561e0a273ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.766+0000 7f4c3f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e0a260620 0x561e0a273ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60590/0 (socket says 172.21.15.97:60590) 2024-04-23T10:30:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.766+0000 7f4c3f7fe640 1 -- 172.21.15.97:0/2413777541 learned_addr learned my addr 172.21.15.97:0/2413777541 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:30:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.766+0000 7f4c3effd640 1 --2- 172.21.15.97:0/2413777541 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e0a260f40 0x561e0a264ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.766+0000 7f4c3ffff640 1 --2- 172.21.15.97:0/2413777541 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e0a26b800 0x561e0a265430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:49.770 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.766+0000 7f4c3f7fe640 1 -- 172.21.15.97:0/2413777541 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e0a260f40 msgr2=0x561e0a264ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:30:49.770 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.766+0000 7f4c3f7fe640 1 --2- 172.21.15.97:0/2413777541 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e0a260f40 0x561e0a264ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:49.770 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.766+0000 7f4c3f7fe640 1 -- 172.21.15.97:0/2413777541 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e0a26b800 msgr2=0x561e0a265430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:30:49.770 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.766+0000 7f4c3f7fe640 1 --2- 172.21.15.97:0/2413777541 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e0a26b800 0x561e0a265430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:49.770 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.766+0000 7f4c3f7fe640 1 -- 172.21.15.97:0/2413777541 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e0a265b70 con 0x561e0a260620 2024-04-23T10:30:49.770 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.766+0000 7f4c3ffff640 1 --2- 172.21.15.97:0/2413777541 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e0a26b800 0x561e0a265430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:30:49.770 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.766+0000 7f4c3f7fe640 1 --2- 172.21.15.97:0/2413777541 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e0a260620 0x561e0a273ad0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f4c3009c200 tx=0x7f4c3000def0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a030b52b3acc10ef server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:30:49.770 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.766+0000 7f4c1effd640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:30:49.770 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.766+0000 7f4c1effd640 1 -- 172.21.15.97:0/2413777541 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c300ba060 con 0x561e0a260620 2024-04-23T10:30:49.770 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.766+0000 7f4c1effd640 1 -- 172.21.15.97:0/2413777541 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f4c300a4a50 con 0x561e0a260620 2024-04-23T10:30:49.770 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.766+0000 7f4c48992740 1 -- 172.21.15.97:0/2413777541 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c38036040 con 0x561e0a260620 2024-04-23T10:30:49.770 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.766+0000 7f4c1effd640 1 -- 172.21.15.97:0/2413777541 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4c300cf070 con 0x561e0a260620 2024-04-23T10:30:49.770 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.766+0000 7f4c48992740 10 client.16257 fetch_fsmap learned FSMap version 26 2024-04-23T10:30:49.770 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.766+0000 7f4c48992740 1 -- 172.21.15.97:0/2413777541 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561e0a2e1890 con 0x561e0a260620 2024-04-23T10:30:49.770 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.766+0000 7f4c1effd640 1 -- 172.21.15.97:0/2413777541 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c38036040 con 0x561e0a260620 2024-04-23T10:30:49.771 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.767+0000 7f4c1effd640 1 -- 172.21.15.97:0/2413777541 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4c300c65e0 con 0x561e0a260620 2024-04-23T10:30:49.771 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.767+0000 7f4c48992740 10 client.16257 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:30:49.771 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.767+0000 7f4c48992740 10 client.16257 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:30:49.771 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.767+0000 7f4c48992740 20 client.16257 populate_metadata read hostname 'smithi097' 2024-04-23T10:30:49.771 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.767+0000 7f4c48992740 1 --2- 172.21.15.97:0/2413777541 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x561e0a2e1c80 0x561e0a2e40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:30:49.771 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.767+0000 7f4c48992740 4 client.16257 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:30:49.771 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.767+0000 7f4c48992740 1 -- 172.21.15.97:0/2413777541 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561e0a1cd200 con 0x561e0a2e1c80 2024-04-23T10:30:49.771 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.767+0000 7f4c3effd640 1 --2- 172.21.15.97:0/2413777541 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x561e0a2e1c80 0x561e0a2e40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:30:49.771 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.767+0000 7f4c1effd640 10 client.16257 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:30:49.771 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.767+0000 7f4c3effd640 1 --2- 172.21.15.97:0/2413777541 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x561e0a2e1c80 0x561e0a2e40a0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf37c208ce6d0ce server_cookie=627c4d4b937da797 in_seq=0 out_seq=0 2024-04-23T10:30:49.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.782+0000 7f4c1effd640 1 -- 172.21.15.97:0/2413777541 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561e0a1cd200 con 0x561e0a2e1c80 2024-04-23T10:30:49.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.782+0000 7f4c1effd640 10 client.16257 handle_command_reply: tid=0 2024-04-23T10:30:49.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.782+0000 7f4c48992740 1 client.16257 shutdown 2024-04-23T10:30:49.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.783+0000 7f4c48992740 1 -- 172.21.15.97:0/2413777541 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e0a260620 msgr2=0x561e0a273ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:30:49.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.783+0000 7f4c48992740 1 --2- 172.21.15.97:0/2413777541 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e0a260620 0x561e0a273ad0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f4c3009c200 tx=0x7f4c3000def0 comp rx=0 tx=0).stop 2024-04-23T10:30:49.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.783+0000 7f4c48992740 1 -- 172.21.15.97:0/2413777541 shutdown_connections 2024-04-23T10:30:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.783+0000 7f4c48992740 1 --2- 172.21.15.97:0/2413777541 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x561e0a2e1c80 0x561e0a2e40a0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.783+0000 7f4c48992740 1 --2- 172.21.15.97:0/2413777541 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561e0a26b800 0x561e0a265430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.783+0000 7f4c48992740 1 --2- 172.21.15.97:0/2413777541 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561e0a260f40 0x561e0a264ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.783+0000 7f4c1effd640 0 client.16257 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:30:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.783+0000 7f4c48992740 1 --2- 172.21.15.97:0/2413777541 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561e0a260620 0x561e0a273ad0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.783+0000 7f4c48992740 1 -- 172.21.15.97:0/2413777541 >> 172.21.15.97:0/2413777541 conn(0x561e0a25cb30 msgr2=0x561e0a266ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:30:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.783+0000 7f4c48992740 1 -- 172.21.15.97:0/2413777541 shutdown_connections 2024-04-23T10:30:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.783+0000 7f4c48992740 1 -- 172.21.15.97:0/2413777541 wait complete. 2024-04-23T10:30:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.783+0000 7f4c48992740 20 client.16257 trim_cache size 0 max 16384 2024-04-23T10:30:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:30:49.784 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:30:49.784 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:30:49.784 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:30:49.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.783+0000 7f4c45b9b640 1 -- 172.21.15.97:0/3474690258 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f4c280a3dc0 msgr2=0x7f4c280a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:30:49.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.783+0000 7f4c45b9b640 1 --2- 172.21.15.97:0/3474690258 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f4c280a3dc0 0x7f4c280a6280 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4c40108890 tx=0x7f4c3400b390 comp rx=0 tx=0).stop 2024-04-23T10:30:49.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.783+0000 7f4c45b9b640 1 -- 172.21.15.97:0/3474690258 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c401021c0 msgr2=0x7f4c40198760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:30:49.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.783+0000 7f4c45b9b640 1 --2- 172.21.15.97:0/3474690258 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c401021c0 0x7f4c40198760 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f4c3000d770 tx=0x7f4c3000dc30 comp rx=0 tx=0).stop 2024-04-23T10:30:49.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.783+0000 7f4c45b9b640 1 -- 172.21.15.97:0/3474690258 shutdown_connections 2024-04-23T10:30:49.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.783+0000 7f4c45b9b640 1 --2- 172.21.15.97:0/3474690258 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f4c280a3dc0 0x7f4c280a6280 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:49.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.783+0000 7f4c45b9b640 1 --2- 172.21.15.97:0/3474690258 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c4019cfd0 0x7f4c4019d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:49.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.783+0000 7f4c45b9b640 1 --2- 172.21.15.97:0/3474690258 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c40107290 0x7f4c40198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:49.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.783+0000 7f4c45b9b640 1 --2- 172.21.15.97:0/3474690258 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c401021c0 0x7f4c40198760 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:30:49.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.783+0000 7f4c45b9b640 1 -- 172.21.15.97:0/3474690258 >> 172.21.15.97:0/3474690258 conn(0x7f4c400fda20 msgr2=0x7f4c400ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:30:49.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.784+0000 7f4c45b9b640 1 -- 172.21.15.97:0/3474690258 shutdown_connections 2024-04-23T10:30:49.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:30:49.784+0000 7f4c45b9b640 1 -- 172.21.15.97:0/3474690258 wait complete. 2024-04-23T10:30:50.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:30:50 smithi196 ceph-mon[32919]: pgmap v142: 97 pgs: 97 active+clean; 4.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 KiB/s wr, 13 op/s 2024-04-23T10:30:50.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:30:50 smithi097 ceph-mon[29197]: pgmap v142: 97 pgs: 97 active+clean; 4.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 KiB/s wr, 13 op/s 2024-04-23T10:30:50.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:30:50 smithi183 ceph-mon[32317]: pgmap v142: 97 pgs: 97 active+clean; 4.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 KiB/s wr, 13 op/s 2024-04-23T10:30:52.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:30:52 smithi196 ceph-mon[32919]: pgmap v143: 97 pgs: 97 active+clean; 4.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 KiB/s wr, 13 op/s 2024-04-23T10:30:52.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:30:52 smithi097 ceph-mon[29197]: pgmap v143: 97 pgs: 97 active+clean; 4.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 KiB/s wr, 13 op/s 2024-04-23T10:30:52.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:30:52 smithi183 ceph-mon[32317]: pgmap v143: 97 pgs: 97 active+clean; 4.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 KiB/s wr, 13 op/s 2024-04-23T10:30:54.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:30:54 smithi196 ceph-mon[32919]: pgmap v144: 97 pgs: 97 active+clean; 4.8 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-23T10:30:54.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:30:54 smithi097 ceph-mon[29197]: pgmap v144: 97 pgs: 97 active+clean; 4.8 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-23T10:30:54.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:30:54 smithi183 ceph-mon[32317]: pgmap v144: 97 pgs: 97 active+clean; 4.8 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-23T10:30:56.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:30:56 smithi196 ceph-mon[32919]: pgmap v145: 97 pgs: 97 active+clean; 4.8 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-04-23T10:30:56.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:30:56 smithi097 ceph-mon[29197]: pgmap v145: 97 pgs: 97 active+clean; 4.8 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-04-23T10:30:56.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:30:56 smithi183 ceph-mon[32317]: pgmap v145: 97 pgs: 97 active+clean; 4.8 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-04-23T10:30:58.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:30:58 smithi196 ceph-mon[32919]: pgmap v146: 97 pgs: 97 active+clean; 4.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-23T10:30:58.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:30:58 smithi097 ceph-mon[29197]: pgmap v146: 97 pgs: 97 active+clean; 4.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-23T10:30:58.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:30:58 smithi183 ceph-mon[32317]: pgmap v146: 97 pgs: 97 active+clean; 4.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-23T10:31:00.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:31:00 smithi196 ceph-mon[32919]: pgmap v147: 97 pgs: 97 active+clean; 4.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-04-23T10:31:00.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:31:00 smithi097 ceph-mon[29197]: pgmap v147: 97 pgs: 97 active+clean; 4.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-04-23T10:31:00.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:31:00 smithi183 ceph-mon[32317]: pgmap v147: 97 pgs: 97 active+clean; 4.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-04-23T10:31:02.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:31:02 smithi196 ceph-mon[32919]: pgmap v148: 97 pgs: 97 active+clean; 4.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-04-23T10:31:02.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:31:02 smithi097 ceph-mon[29197]: pgmap v148: 97 pgs: 97 active+clean; 4.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-04-23T10:31:02.878 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.879+0000 7fe8a1729640 1 -- 172.21.15.97:0/2646908352 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe89c103250 msgr2=0x7fe89c100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:31:02.878 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.879+0000 7fe8a1729640 1 --2- 172.21.15.97:0/2646908352 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe89c103250 0x7fe89c100990 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fe890009900 tx=0x7fe89002a480 comp rx=0 tx=0).stop 2024-04-23T10:31:02.878 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.879+0000 7fe8a1729640 1 -- 172.21.15.97:0/2646908352 shutdown_connections 2024-04-23T10:31:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.879+0000 7fe8a1729640 1 --2- 172.21.15.97:0/2646908352 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe89c100fb0 0x7fe89c110be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.879+0000 7fe8a1729640 1 --2- 172.21.15.97:0/2646908352 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe89c103250 0x7fe89c100990 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.879+0000 7fe8a1729640 1 --2- 172.21.15.97:0/2646908352 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe89c102e70 0x7fe89c100450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.879+0000 7fe8a1729640 1 -- 172.21.15.97:0/2646908352 >> 172.21.15.97:0/2646908352 conn(0x7fe89c077390 msgr2=0x7fe89c0fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:31:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.879+0000 7fe8a1729640 1 -- 172.21.15.97:0/2646908352 shutdown_connections 2024-04-23T10:31:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.879+0000 7fe8a1729640 1 -- 172.21.15.97:0/2646908352 wait complete. 2024-04-23T10:31:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.879+0000 7fe8a1729640 1 Processor -- start 2024-04-23T10:31:02.879 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.880+0000 7fe8a1729640 1 -- start start 2024-04-23T10:31:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.880+0000 7fe8a1729640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe89c100fb0 0x7fe89c1a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.880+0000 7fe8a1729640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe89c102e70 0x7fe89c1a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.880+0000 7fe8a1729640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe89c103250 0x7fe89c19a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:02.890 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.880+0000 7fe8a1729640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe89c1143b0 con 0x7fe89c103250 2024-04-23T10:31:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.880+0000 7fe8a1729640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe89c114230 con 0x7fe89c102e70 2024-04-23T10:31:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.880+0000 7fe8a1729640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe89c114530 con 0x7fe89c100fb0 2024-04-23T10:31:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.880+0000 7fe89b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe89c103250 0x7fe89c19a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.880+0000 7fe89b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe89c103250 0x7fe89c19a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59798/0 (socket says 172.21.15.97:59798) 2024-04-23T10:31:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.880+0000 7fe89b7fe640 1 -- 172.21.15.97:0/3540914750 learned_addr learned my addr 172.21.15.97:0/3540914750 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:31:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.880+0000 7fe89affd640 1 --2- 172.21.15.97:0/3540914750 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe89c100fb0 0x7fe89c1a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.880+0000 7fe89a7fc640 1 --2- 172.21.15.97:0/3540914750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe89c102e70 0x7fe89c1a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.880+0000 7fe89b7fe640 1 -- 172.21.15.97:0/3540914750 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe89c100fb0 msgr2=0x7fe89c1a05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:31:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.880+0000 7fe89b7fe640 1 --2- 172.21.15.97:0/3540914750 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe89c100fb0 0x7fe89c1a05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:02.891 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.880+0000 7fe89b7fe640 1 -- 172.21.15.97:0/3540914750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe89c102e70 msgr2=0x7fe89c1a0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:31:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.880+0000 7fe89b7fe640 1 --2- 172.21.15.97:0/3540914750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe89c102e70 0x7fe89c1a0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.880+0000 7fe89b7fe640 1 -- 172.21.15.97:0/3540914750 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe89c19af50 con 0x7fe89c103250 2024-04-23T10:31:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.880+0000 7fe89a7fc640 1 --2- 172.21.15.97:0/3540914750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe89c102e70 0x7fe89c1a0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:31:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.880+0000 7fe89b7fe640 1 --2- 172.21.15.97:0/3540914750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe89c103250 0x7fe89c19a690 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fe89400cce0 tx=0x7fe89400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:31:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.881+0000 7fe883fff640 1 -- 172.21.15.97:0/3540914750 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe89400f450 con 0x7fe89c103250 2024-04-23T10:31:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.881+0000 7fe883fff640 1 -- 172.21.15.97:0/3540914750 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe89400faa0 con 0x7fe89c103250 2024-04-23T10:31:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.881+0000 7fe8a1729640 1 -- 172.21.15.97:0/3540914750 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe89c19b240 con 0x7fe89c103250 2024-04-23T10:31:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.881+0000 7fe883fff640 1 -- 172.21.15.97:0/3540914750 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe894018b70 con 0x7fe89c103250 2024-04-23T10:31:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.881+0000 7fe8a1729640 1 -- 172.21.15.97:0/3540914750 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe89c1a7550 con 0x7fe89c103250 2024-04-23T10:31:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.883+0000 7fe883fff640 1 -- 172.21.15.97:0/3540914750 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe894018d10 con 0x7fe89c103250 2024-04-23T10:31:02.892 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.883+0000 7fe8a4440740 1 Processor -- start 2024-04-23T10:31:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.883+0000 7fe883fff640 1 --2- 172.21.15.97:0/3540914750 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fe8840a3dc0 0x7fe8840a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.883+0000 7fe8a4440740 1 -- start start 2024-04-23T10:31:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.884+0000 7fe89a7fc640 1 --2- 172.21.15.97:0/3540914750 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fe8840a3dc0 0x7fe8840a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.884+0000 7fe883fff640 1 -- 172.21.15.97:0/3540914750 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fe8940d0c10 con 0x7fe89c103250 2024-04-23T10:31:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.884+0000 7fe8a4440740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562870e00620 0x562870e00a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.884+0000 7fe8a4440740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562870e00f40 0x562870e0b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.884+0000 7fe8a4440740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562870e0b800 0x562870e0dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.884+0000 7fe8a4440740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x562870d65970 con 0x562870e00620 2024-04-23T10:31:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.884+0000 7fe8a4440740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562870d024a0 con 0x562870e0b800 2024-04-23T10:31:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.884+0000 7fe8a4440740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562870d61930 con 0x562870e00f40 2024-04-23T10:31:02.893 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.884+0000 7fe89affd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562870e00620 0x562870e00a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:02.894 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.884+0000 7fe89affd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562870e00620 0x562870e00a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59804/0 (socket says 172.21.15.97:59804) 2024-04-23T10:31:02.894 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.884+0000 7fe89affd640 1 -- 172.21.15.97:0/1058116894 learned_addr learned my addr 172.21.15.97:0/1058116894 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:31:02.894 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.884+0000 7fe89a7fc640 1 --2- 172.21.15.97:0/1058116894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562870e00f40 0x562870e0b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:02.894 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.884+0000 7fe89b7fe640 1 --2- 172.21.15.97:0/1058116894 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562870e0b800 0x562870e0dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:02.894 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.884+0000 7fe89affd640 1 -- 172.21.15.97:0/1058116894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562870e00f40 msgr2=0x562870e0b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:31:02.894 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.884+0000 7fe89affd640 1 --2- 172.21.15.97:0/1058116894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562870e00f40 0x562870e0b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:02.894 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.884+0000 7fe89affd640 1 -- 172.21.15.97:0/1058116894 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562870e0b800 msgr2=0x562870e0dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:31:02.894 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.884+0000 7fe89affd640 1 --2- 172.21.15.97:0/1058116894 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562870e0b800 0x562870e0dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:02.894 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.884+0000 7fe89affd640 1 -- 172.21.15.97:0/1058116894 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562870a78210 con 0x562870e00620 2024-04-23T10:31:02.894 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.884+0000 7fe89a7fc640 1 --2- 172.21.15.97:0/3540914750 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fe8840a3dc0 0x7fe8840a6280 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe890031440 tx=0x7fe890032d40 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:31:02.895 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.884+0000 7fe89affd640 1 --2- 172.21.15.97:0/1058116894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562870e00620 0x562870e00a00 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fe88c00b710 tx=0x7fe88c004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc5709c0fb29632b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:31:02.895 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.884+0000 7fe8817fa640 1 -- 172.21.15.97:0/1058116894 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe88c003bd0 con 0x562870e00620 2024-04-23T10:31:02.895 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.885+0000 7fe8817fa640 1 -- 172.21.15.97:0/1058116894 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe88c015070 con 0x562870e00620 2024-04-23T10:31:02.895 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.885+0000 7fe8817fa640 1 -- 172.21.15.97:0/1058116894 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe88c004ce0 con 0x562870e00620 2024-04-23T10:31:02.895 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.885+0000 7fe8a4440740 1 -- 172.21.15.97:0/1058116894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562870e00620 msgr2=0x562870e00a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:31:02.895 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.885+0000 7fe8a4440740 1 --2- 172.21.15.97:0/1058116894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562870e00620 0x562870e00a00 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fe88c00b710 tx=0x7fe88c004610 comp rx=0 tx=0).stop 2024-04-23T10:31:02.895 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.885+0000 7fe8a4440740 1 -- 172.21.15.97:0/1058116894 shutdown_connections 2024-04-23T10:31:02.895 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.885+0000 7fe8a4440740 1 --2- 172.21.15.97:0/1058116894 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562870e0b800 0x562870e0dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:02.895 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.885+0000 7fe8a4440740 1 --2- 172.21.15.97:0/1058116894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562870e00f40 0x562870e0b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:02.895 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.885+0000 7fe8a4440740 1 --2- 172.21.15.97:0/1058116894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562870e00620 0x562870e00a00 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:02.895 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.885+0000 7fe8a4440740 1 -- 172.21.15.97:0/1058116894 >> 172.21.15.97:0/1058116894 conn(0x562870dfcb30 msgr2=0x562870dfcf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:31:02.896 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.885+0000 7fe8a4440740 1 -- 172.21.15.97:0/1058116894 shutdown_connections 2024-04-23T10:31:02.896 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.885+0000 7fe8a4440740 1 -- 172.21.15.97:0/1058116894 wait complete. 2024-04-23T10:31:02.896 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.886+0000 7fe8a4440740 1 -- start start 2024-04-23T10:31:02.896 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.886+0000 7fe8a4440740 1 Processor -- start 2024-04-23T10:31:02.896 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.886+0000 7fe8a4440740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562870e00620 0x562870e0b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:02.896 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.886+0000 7fe8a4440740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562870e00f40 0x562870e08460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:02.896 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.886+0000 7fe8a4440740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562870e0b800 0x562870e089c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:02.896 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.886+0000 7fe8a4440740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x562870e0e430 con 0x562870e00620 2024-04-23T10:31:02.896 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.886+0000 7fe8a4440740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562870d000b0 con 0x562870e0b800 2024-04-23T10:31:02.896 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.886+0000 7fe8a4440740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x562870e0e5b0 con 0x562870e00f40 2024-04-23T10:31:02.896 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.886+0000 7fe8a4440740 10 client.0 _ll_register_callbacks cb 0x562870cedd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:31:02.897 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.886+0000 7fe89affd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562870e00620 0x562870e0b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:02.897 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.886+0000 7fe89affd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562870e00620 0x562870e0b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59806/0 (socket says 172.21.15.97:59806) 2024-04-23T10:31:02.897 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.886+0000 7fe89affd640 1 -- 172.21.15.97:0/117734931 learned_addr learned my addr 172.21.15.97:0/117734931 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:31:02.897 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.886+0000 7fe89b7fe640 1 --2- 172.21.15.97:0/117734931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562870e0b800 0x562870e089c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:02.897 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.886+0000 7fe89affd640 1 -- 172.21.15.97:0/117734931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562870e00f40 msgr2=0x562870e08460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:31:02.897 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.886+0000 7fe89affd640 1 --2- 172.21.15.97:0/117734931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562870e00f40 0x562870e08460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:02.897 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.886+0000 7fe89affd640 1 -- 172.21.15.97:0/117734931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562870e0b800 msgr2=0x562870e089c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:31:02.897 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.886+0000 7fe89affd640 1 --2- 172.21.15.97:0/117734931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562870e0b800 0x562870e089c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:02.897 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.886+0000 7fe89affd640 1 -- 172.21.15.97:0/117734931 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562870b44620 con 0x562870e00620 2024-04-23T10:31:02.897 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.886+0000 7fe89b7fe640 1 --2- 172.21.15.97:0/117734931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562870e0b800 0x562870e089c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:31:02.897 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.886+0000 7fe89affd640 1 --2- 172.21.15.97:0/117734931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562870e00620 0x562870e0b170 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fe88c0039c0 tx=0x7fe88c003bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18c81b720fe3a006 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:31:02.897 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.886+0000 7fe87eff5640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:31:02.898 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.887+0000 7fe87eff5640 1 -- 172.21.15.97:0/117734931 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe88c028060 con 0x562870e00620 2024-04-23T10:31:02.898 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.887+0000 7fe87eff5640 1 -- 172.21.15.97:0/117734931 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fe88c011640 con 0x562870e00620 2024-04-23T10:31:02.898 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.887+0000 7fe8a4440740 1 -- 172.21.15.97:0/117734931 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe88c015070 con 0x562870e00620 2024-04-23T10:31:02.898 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.887+0000 7fe87eff5640 1 -- 172.21.15.97:0/117734931 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe88c011840 con 0x562870e00620 2024-04-23T10:31:02.898 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.887+0000 7fe8a4440740 10 client.16269 fetch_fsmap learned FSMap version 26 2024-04-23T10:31:02.898 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.887+0000 7fe8a4440740 1 -- 172.21.15.97:0/117734931 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562870e1a360 con 0x562870e00620 2024-04-23T10:31:02.898 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.887+0000 7fe87eff5640 1 -- 172.21.15.97:0/117734931 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe88c015070 con 0x562870e00620 2024-04-23T10:31:02.898 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.887+0000 7fe87eff5640 1 -- 172.21.15.97:0/117734931 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe88c022070 con 0x562870e00620 2024-04-23T10:31:02.898 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.887+0000 7fe8a4440740 10 client.16269 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:31:02.898 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.887+0000 7fe8a4440740 10 client.16269 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:31:02.899 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.887+0000 7fe8a4440740 20 client.16269 populate_metadata read hostname 'smithi097' 2024-04-23T10:31:02.899 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.887+0000 7fe8a4440740 1 --2- 172.21.15.97:0/117734931 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x562870e1b760 0x562870e1db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:02.899 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.887+0000 7fe8a4440740 4 client.16269 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:31:02.899 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.887+0000 7fe8a4440740 1 -- 172.21.15.97:0/117734931 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562870d73390 con 0x562870e1b760 2024-04-23T10:31:02.899 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.888+0000 7fe89a7fc640 1 --2- 172.21.15.97:0/117734931 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x562870e1b760 0x562870e1db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:02.899 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.888+0000 7fe87eff5640 10 client.16269 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:31:02.899 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.888+0000 7fe89a7fc640 1 --2- 172.21.15.97:0/117734931 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x562870e1b760 0x562870e1db50 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b3dba89364d0850 server_cookie=cc7a7bc748c1fec1 in_seq=0 out_seq=0 2024-04-23T10:31:02.899 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.889+0000 7fe87eff5640 1 -- 172.21.15.97:0/117734931 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562870d73390 con 0x562870e1b760 2024-04-23T10:31:02.899 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.889+0000 7fe87eff5640 10 client.16269 handle_command_reply: tid=0 2024-04-23T10:31:02.899 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.889+0000 7fe8a4440740 1 client.16269 shutdown 2024-04-23T10:31:02.899 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.889+0000 7fe8a4440740 1 -- 172.21.15.97:0/117734931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562870e00620 msgr2=0x562870e0b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:31:02.899 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.889+0000 7fe8a4440740 1 --2- 172.21.15.97:0/117734931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562870e00620 0x562870e0b170 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fe88c0039c0 tx=0x7fe88c003bd0 comp rx=0 tx=0).stop 2024-04-23T10:31:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.889+0000 7fe8a4440740 1 -- 172.21.15.97:0/117734931 shutdown_connections 2024-04-23T10:31:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.889+0000 7fe8a4440740 1 --2- 172.21.15.97:0/117734931 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x562870e1b760 0x562870e1db50 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.889+0000 7fe8a4440740 1 --2- 172.21.15.97:0/117734931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562870e0b800 0x562870e089c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.889+0000 7fe8a4440740 1 --2- 172.21.15.97:0/117734931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562870e00f40 0x562870e08460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.889+0000 7fe87eff5640 0 client.16269 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:31:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.889+0000 7fe8a4440740 1 --2- 172.21.15.97:0/117734931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562870e00620 0x562870e0b170 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.889+0000 7fe8a4440740 1 -- 172.21.15.97:0/117734931 >> 172.21.15.97:0/117734931 conn(0x562870dfcb30 msgr2=0x562870e02860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:31:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.889+0000 7fe8a4440740 1 -- 172.21.15.97:0/117734931 shutdown_connections 2024-04-23T10:31:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.889+0000 7fe8a4440740 1 -- 172.21.15.97:0/117734931 wait complete. 2024-04-23T10:31:02.900 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.889+0000 7fe8a4440740 20 client.16269 trim_cache size 0 max 16384 2024-04-23T10:31:02.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:31:02 smithi183 ceph-mon[32317]: pgmap v148: 97 pgs: 97 active+clean; 4.9 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 31 KiB/s wr, 15 op/s 2024-04-23T10:31:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.904+0000 7fe8a4440740 1 Processor -- start 2024-04-23T10:31:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.904+0000 7fe8a4440740 1 -- start start 2024-04-23T10:31:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.904+0000 7fe8a4440740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562870e00620 0x562870e172f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.904+0000 7fe8a4440740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562870e00f40 0x562870e0aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.904+0000 7fe8a4440740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562870e0b800 0x562870e23150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.904+0000 7fe8a4440740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe89c1afbb0 con 0x562870e0b800 2024-04-23T10:31:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.904+0000 7fe8a4440740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe89c1b16f0 con 0x562870e00620 2024-04-23T10:31:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.904+0000 7fe8a4440740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe89c1b0280 con 0x562870e00f40 2024-04-23T10:31:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.904+0000 7fe89b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562870e0b800 0x562870e23150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.904+0000 7fe89b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562870e0b800 0x562870e23150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59812/0 (socket says 172.21.15.97:59812) 2024-04-23T10:31:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.904+0000 7fe89b7fe640 1 -- 172.21.15.97:0/1868843117 learned_addr learned my addr 172.21.15.97:0/1868843117 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:31:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.904+0000 7fe89a7fc640 1 --2- 172.21.15.97:0/1868843117 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562870e00f40 0x562870e0aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.904+0000 7fe89affd640 1 --2- 172.21.15.97:0/1868843117 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562870e00620 0x562870e172f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.904+0000 7fe89b7fe640 1 -- 172.21.15.97:0/1868843117 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562870e00f40 msgr2=0x562870e0aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:31:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.904+0000 7fe89b7fe640 1 --2- 172.21.15.97:0/1868843117 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562870e00f40 0x562870e0aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.904+0000 7fe89b7fe640 1 -- 172.21.15.97:0/1868843117 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562870e00620 msgr2=0x562870e172f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:31:02.909 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.904+0000 7fe89b7fe640 1 --2- 172.21.15.97:0/1868843117 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562870e00620 0x562870e172f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.904+0000 7fe89b7fe640 1 -- 172.21.15.97:0/1868843117 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562870e00d70 con 0x562870e0b800 2024-04-23T10:31:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.905+0000 7fe89b7fe640 1 --2- 172.21.15.97:0/1868843117 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562870e0b800 0x562870e23150 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fe8940ce9b0 tx=0x7fe89400af30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5083b30daab30396 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:31:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.905+0000 7fe880ff9640 1 -- 172.21.15.97:0/1868843117 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8940b6070 con 0x562870e0b800 2024-04-23T10:31:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.905+0000 7fe880ff9640 1 -- 172.21.15.97:0/1868843117 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8940a0fb0 con 0x562870e0b800 2024-04-23T10:31:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.905+0000 7fe880ff9640 1 -- 172.21.15.97:0/1868843117 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8940bd0b0 con 0x562870e0b800 2024-04-23T10:31:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.905+0000 7fe8a4440740 1 -- 172.21.15.97:0/1868843117 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562870e0b800 msgr2=0x562870e23150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:31:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.905+0000 7fe8a4440740 1 --2- 172.21.15.97:0/1868843117 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562870e0b800 0x562870e23150 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fe8940ce9b0 tx=0x7fe89400af30 comp rx=0 tx=0).stop 2024-04-23T10:31:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.905+0000 7fe8a4440740 1 -- 172.21.15.97:0/1868843117 shutdown_connections 2024-04-23T10:31:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.905+0000 7fe8a4440740 1 --2- 172.21.15.97:0/1868843117 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562870e0b800 0x562870e23150 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.905+0000 7fe8a4440740 1 --2- 172.21.15.97:0/1868843117 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562870e00f40 0x562870e0aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.905+0000 7fe8a4440740 1 --2- 172.21.15.97:0/1868843117 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562870e00620 0x562870e172f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:02.910 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.905+0000 7fe8a4440740 1 -- 172.21.15.97:0/1868843117 >> 172.21.15.97:0/1868843117 conn(0x562870dfcb30 msgr2=0x562870e03af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:31:02.911 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.905+0000 7fe8a4440740 1 -- 172.21.15.97:0/1868843117 shutdown_connections 2024-04-23T10:31:02.911 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.905+0000 7fe8a4440740 1 -- 172.21.15.97:0/1868843117 wait complete. 2024-04-23T10:31:02.911 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.906+0000 7fe8a4440740 1 -- start start 2024-04-23T10:31:02.911 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.906+0000 7fe8a4440740 1 Processor -- start 2024-04-23T10:31:02.911 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.906+0000 7fe8a4440740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562870e00620 0x562870e13720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:02.911 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.906+0000 7fe8a4440740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562870e00f40 0x562870e13c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:02.911 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.906+0000 7fe8a4440740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562870e0b800 0x562870e0ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:02.911 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.906+0000 7fe8a4440740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe89c1b1020 con 0x562870e00f40 2024-04-23T10:31:02.911 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.906+0000 7fe8a4440740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe89c1b0950 con 0x562870e00620 2024-04-23T10:31:02.911 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.906+0000 7fe8a4440740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe89c1adbb0 con 0x562870e0b800 2024-04-23T10:31:02.911 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.906+0000 7fe8a4440740 10 client.0 _ll_register_callbacks cb 0x7fe88c00be40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:31:02.911 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.906+0000 7fe89a7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562870e00f40 0x562870e13c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:02.912 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.906+0000 7fe89a7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562870e00f40 0x562870e13c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59818/0 (socket says 172.21.15.97:59818) 2024-04-23T10:31:02.912 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.906+0000 7fe89a7fc640 1 -- 172.21.15.97:0/2665234737 learned_addr learned my addr 172.21.15.97:0/2665234737 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:31:02.912 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.906+0000 7fe89affd640 1 --2- 172.21.15.97:0/2665234737 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562870e00620 0x562870e13720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:02.912 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.906+0000 7fe89b7fe640 1 --2- 172.21.15.97:0/2665234737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562870e0b800 0x562870e0ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:02.912 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.906+0000 7fe89a7fc640 1 -- 172.21.15.97:0/2665234737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562870e0b800 msgr2=0x562870e0ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:31:02.912 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.906+0000 7fe89a7fc640 1 --2- 172.21.15.97:0/2665234737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562870e0b800 0x562870e0ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:02.912 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.906+0000 7fe89a7fc640 1 -- 172.21.15.97:0/2665234737 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562870e00620 msgr2=0x562870e13720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:31:02.912 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.906+0000 7fe89a7fc640 1 --2- 172.21.15.97:0/2665234737 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562870e00620 0x562870e13720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:02.912 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.906+0000 7fe89a7fc640 1 -- 172.21.15.97:0/2665234737 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562870e0f400 con 0x562870e00f40 2024-04-23T10:31:02.912 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.906+0000 7fe89b7fe640 1 --2- 172.21.15.97:0/2665234737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562870e0b800 0x562870e0ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:31:02.912 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.906+0000 7fe89affd640 1 --2- 172.21.15.97:0/2665234737 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562870e00620 0x562870e13720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:31:02.912 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.907+0000 7fe89a7fc640 1 --2- 172.21.15.97:0/2665234737 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562870e00f40 0x562870e13c60 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fe890009900 tx=0x7fe890009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b4b5a2592eaf5a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:31:02.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.907+0000 7fe87eff5640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:31:02.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.907+0000 7fe87eff5640 1 -- 172.21.15.97:0/2665234737 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe890035050 con 0x562870e00f40 2024-04-23T10:31:02.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.907+0000 7fe87eff5640 1 -- 172.21.15.97:0/2665234737 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fe8900043d0 con 0x562870e00f40 2024-04-23T10:31:02.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.907+0000 7fe8a4440740 1 -- 172.21.15.97:0/2665234737 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8940a0fb0 con 0x562870e00f40 2024-04-23T10:31:02.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.907+0000 7fe87eff5640 1 -- 172.21.15.97:0/2665234737 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe890005040 con 0x562870e00f40 2024-04-23T10:31:02.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.907+0000 7fe8a4440740 10 client.16275 fetch_fsmap learned FSMap version 26 2024-04-23T10:31:02.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.907+0000 7fe8a4440740 1 -- 172.21.15.97:0/2665234737 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562870e81530 con 0x562870e00f40 2024-04-23T10:31:02.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.907+0000 7fe87eff5640 1 -- 172.21.15.97:0/2665234737 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8940a0fb0 con 0x562870e00f40 2024-04-23T10:31:02.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.907+0000 7fe87eff5640 1 -- 172.21.15.97:0/2665234737 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe8900035f0 con 0x562870e00f40 2024-04-23T10:31:02.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.907+0000 7fe8a4440740 10 client.16275 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:31:02.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.908+0000 7fe8a4440740 10 client.16275 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:31:02.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.908+0000 7fe8a4440740 20 client.16275 populate_metadata read hostname 'smithi097' 2024-04-23T10:31:02.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.908+0000 7fe8a4440740 1 --2- 172.21.15.97:0/2665234737 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x562870e81ce0 0x562870e84100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:02.914 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.908+0000 7fe8a4440740 4 client.16275 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:31:02.914 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.908+0000 7fe8a4440740 1 -- 172.21.15.97:0/2665234737 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562870d6d200 con 0x562870e81ce0 2024-04-23T10:31:02.914 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.908+0000 7fe89affd640 1 --2- 172.21.15.97:0/2665234737 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x562870e81ce0 0x562870e84100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:02.914 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.908+0000 7fe87eff5640 10 client.16275 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:31:02.914 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.908+0000 7fe89affd640 1 --2- 172.21.15.97:0/2665234737 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x562870e81ce0 0x562870e84100 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15a35631d652d84d server_cookie=785547feec801063 in_seq=0 out_seq=0 2024-04-23T10:31:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.925+0000 7fe87eff5640 1 -- 172.21.15.97:0/2665234737 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562870d6d200 con 0x562870e81ce0 2024-04-23T10:31:02.924 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.925+0000 7fe87eff5640 10 client.16275 handle_command_reply: tid=0 2024-04-23T10:31:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.925+0000 7fe8a4440740 1 client.16275 shutdown 2024-04-23T10:31:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.925+0000 7fe8a4440740 1 -- 172.21.15.97:0/2665234737 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562870e00f40 msgr2=0x562870e13c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:31:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.925+0000 7fe8a4440740 1 --2- 172.21.15.97:0/2665234737 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562870e00f40 0x562870e13c60 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fe890009900 tx=0x7fe890009260 comp rx=0 tx=0).stop 2024-04-23T10:31:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.925+0000 7fe8a4440740 1 -- 172.21.15.97:0/2665234737 shutdown_connections 2024-04-23T10:31:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.925+0000 7fe8a4440740 1 --2- 172.21.15.97:0/2665234737 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x562870e81ce0 0x562870e84100 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.925+0000 7fe8a4440740 1 --2- 172.21.15.97:0/2665234737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562870e0b800 0x562870e0ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.925+0000 7fe8a4440740 1 --2- 172.21.15.97:0/2665234737 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562870e00f40 0x562870e13c60 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.925+0000 7fe87eff5640 0 client.16275 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:31:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.925+0000 7fe8a4440740 1 --2- 172.21.15.97:0/2665234737 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562870e00620 0x562870e13720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.925+0000 7fe8a4440740 1 -- 172.21.15.97:0/2665234737 >> 172.21.15.97:0/2665234737 conn(0x562870dfcb30 msgr2=0x562870e01fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:31:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.925+0000 7fe8a4440740 1 -- 172.21.15.97:0/2665234737 shutdown_connections 2024-04-23T10:31:02.925 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.925+0000 7fe8a4440740 1 -- 172.21.15.97:0/2665234737 wait complete. 2024-04-23T10:31:02.926 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.925+0000 7fe8a4440740 20 client.16275 trim_cache size 0 max 16384 2024-04-23T10:31:02.926 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:31:02.926 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:31:02.926 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:31:02.926 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:31:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.926+0000 7fe8a1729640 1 -- 172.21.15.97:0/3540914750 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fe8840a3dc0 msgr2=0x7fe8840a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:31:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.926+0000 7fe8a1729640 1 --2- 172.21.15.97:0/3540914750 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fe8840a3dc0 0x7fe8840a6280 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe890031440 tx=0x7fe890032d40 comp rx=0 tx=0).stop 2024-04-23T10:31:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.926+0000 7fe8a1729640 1 -- 172.21.15.97:0/3540914750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe89c103250 msgr2=0x7fe89c19a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:31:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.926+0000 7fe8a1729640 1 --2- 172.21.15.97:0/3540914750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe89c103250 0x7fe89c19a690 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fe89400cce0 tx=0x7fe89400a620 comp rx=0 tx=0).stop 2024-04-23T10:31:02.927 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.926+0000 7fe8a1729640 1 -- 172.21.15.97:0/3540914750 shutdown_connections 2024-04-23T10:31:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.926+0000 7fe8a1729640 1 --2- 172.21.15.97:0/3540914750 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fe8840a3dc0 0x7fe8840a6280 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.926+0000 7fe8a1729640 1 --2- 172.21.15.97:0/3540914750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe89c103250 0x7fe89c19a690 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.926+0000 7fe8a1729640 1 --2- 172.21.15.97:0/3540914750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe89c102e70 0x7fe89c1a0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.926+0000 7fe8a1729640 1 --2- 172.21.15.97:0/3540914750 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe89c100fb0 0x7fe89c1a05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.926+0000 7fe8a1729640 1 -- 172.21.15.97:0/3540914750 >> 172.21.15.97:0/3540914750 conn(0x7fe89c077390 msgr2=0x7fe89c0fd9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:31:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.926+0000 7fe8a1729640 1 -- 172.21.15.97:0/3540914750 shutdown_connections 2024-04-23T10:31:02.928 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:02.926+0000 7fe8a1729640 1 -- 172.21.15.97:0/3540914750 wait complete. 2024-04-23T10:31:04.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:31:04 smithi196 ceph-mon[32919]: pgmap v149: 97 pgs: 97 active+clean; 5.1 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 67 KiB/s wr, 37 op/s 2024-04-23T10:31:04.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:31:04 smithi097 ceph-mon[29197]: pgmap v149: 97 pgs: 97 active+clean; 5.1 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 67 KiB/s wr, 37 op/s 2024-04-23T10:31:04.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:31:04 smithi183 ceph-mon[32317]: pgmap v149: 97 pgs: 97 active+clean; 5.1 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 67 KiB/s wr, 37 op/s 2024-04-23T10:31:06.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:31:06 smithi196 ceph-mon[32919]: pgmap v150: 97 pgs: 97 active+clean; 5.1 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 50 KiB/s wr, 27 op/s 2024-04-23T10:31:06.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:31:06 smithi097 ceph-mon[29197]: pgmap v150: 97 pgs: 97 active+clean; 5.1 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 50 KiB/s wr, 27 op/s 2024-04-23T10:31:06.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:31:06 smithi183 ceph-mon[32317]: pgmap v150: 97 pgs: 97 active+clean; 5.1 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 50 KiB/s wr, 27 op/s 2024-04-23T10:31:08.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:31:08 smithi196 ceph-mon[32919]: pgmap v151: 97 pgs: 97 active+clean; 5.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 71 KiB/s wr, 37 op/s 2024-04-23T10:31:08.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:31:08 smithi097 ceph-mon[29197]: pgmap v151: 97 pgs: 97 active+clean; 5.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 71 KiB/s wr, 37 op/s 2024-04-23T10:31:08.899 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:31:08 smithi183 ceph-mon[32317]: pgmap v151: 97 pgs: 97 active+clean; 5.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 71 KiB/s wr, 37 op/s 2024-04-23T10:31:10.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:31:10 smithi196 ceph-mon[32919]: pgmap v152: 97 pgs: 97 active+clean; 5.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 57 KiB/s wr, 30 op/s 2024-04-23T10:31:10.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:31:10 smithi097 ceph-mon[29197]: pgmap v152: 97 pgs: 97 active+clean; 5.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 57 KiB/s wr, 30 op/s 2024-04-23T10:31:10.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:31:10 smithi183 ceph-mon[32317]: pgmap v152: 97 pgs: 97 active+clean; 5.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 57 KiB/s wr, 30 op/s 2024-04-23T10:31:12.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:31:12 smithi196 ceph-mon[32919]: pgmap v153: 97 pgs: 97 active+clean; 5.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 57 KiB/s wr, 30 op/s 2024-04-23T10:31:12.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:31:12 smithi097 ceph-mon[29197]: pgmap v153: 97 pgs: 97 active+clean; 5.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 57 KiB/s wr, 30 op/s 2024-04-23T10:31:12.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:31:12 smithi183 ceph-mon[32317]: pgmap v153: 97 pgs: 97 active+clean; 5.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 57 KiB/s wr, 30 op/s 2024-04-23T10:31:14.702 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:31:14 smithi183 ceph-mon[32317]: pgmap v154: 97 pgs: 97 active+clean; 5.5 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 72 KiB/s wr, 37 op/s 2024-04-23T10:31:14.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:31:14 smithi196 ceph-mon[32919]: pgmap v154: 97 pgs: 97 active+clean; 5.5 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 72 KiB/s wr, 37 op/s 2024-04-23T10:31:14.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:31:14 smithi097 ceph-mon[29197]: pgmap v154: 97 pgs: 97 active+clean; 5.5 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 72 KiB/s wr, 37 op/s 2024-04-23T10:31:16.022 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.021+0000 7f13b7e4e640 1 -- 172.21.15.97:0/3936113569 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13b01021d0 msgr2=0x7f13b01025b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:31:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.021+0000 7f13b7e4e640 1 --2- 172.21.15.97:0/3936113569 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13b01021d0 0x7f13b01025b0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f13ac009900 tx=0x7f13ac02a410 comp rx=0 tx=0).stop 2024-04-23T10:31:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.022+0000 7f13b7e4e640 1 -- 172.21.15.97:0/3936113569 shutdown_connections 2024-04-23T10:31:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.022+0000 7f13b7e4e640 1 --2- 172.21.15.97:0/3936113569 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13b01072a0 0x7f13b010ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.022+0000 7f13b7e4e640 1 --2- 172.21.15.97:0/3936113569 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f13b0102b80 0x7f13b0106bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.022+0000 7f13b7e4e640 1 --2- 172.21.15.97:0/3936113569 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13b01021d0 0x7f13b01025b0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.022+0000 7f13b7e4e640 1 -- 172.21.15.97:0/3936113569 >> 172.21.15.97:0/3936113569 conn(0x7f13b00793e0 msgr2=0x7f13b00797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:31:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.022+0000 7f13b7e4e640 1 -- 172.21.15.97:0/3936113569 shutdown_connections 2024-04-23T10:31:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.022+0000 7f13b7e4e640 1 -- 172.21.15.97:0/3936113569 wait complete. 2024-04-23T10:31:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.022+0000 7f13b7e4e640 1 Processor -- start 2024-04-23T10:31:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.022+0000 7f13b7e4e640 1 -- start start 2024-04-23T10:31:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.022+0000 7f13b7e4e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13b01021d0 0x7f13b01986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.022+0000 7f13b7e4e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f13b01072a0 0x7f13b0198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.022+0000 7f13b7e4e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13b019cf50 0x7f13b019d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.022+0000 7f13b7e4e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f13b01121e0 con 0x7f13b019cf50 2024-04-23T10:31:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.022+0000 7f13b7e4e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f13b0112060 con 0x7f13b01021d0 2024-04-23T10:31:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.022+0000 7f13b7e4e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f13b0112360 con 0x7f13b01072a0 2024-04-23T10:31:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.023+0000 7f13b63eb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13b019cf50 0x7f13b019d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.023+0000 7f13b5bea640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13b01021d0 0x7f13b01986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.023+0000 7f13b5bea640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13b01021d0 0x7f13b01986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.97:57244/0 (socket says 172.21.15.97:57244) 2024-04-23T10:31:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.023+0000 7f13b63eb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13b019cf50 0x7f13b019d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57988/0 (socket says 172.21.15.97:57988) 2024-04-23T10:31:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.023+0000 7f13b5bea640 1 -- 172.21.15.97:0/4087685457 learned_addr learned my addr 172.21.15.97:0/4087685457 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:31:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.023+0000 7f13b53e9640 1 --2- 172.21.15.97:0/4087685457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f13b01072a0 0x7f13b0198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.023+0000 7f13b63eb640 1 -- 172.21.15.97:0/4087685457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f13b01072a0 msgr2=0x7f13b0198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:31:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.023+0000 7f13b63eb640 1 --2- 172.21.15.97:0/4087685457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f13b01072a0 0x7f13b0198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.023+0000 7f13b63eb640 1 -- 172.21.15.97:0/4087685457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13b01021d0 msgr2=0x7f13b01986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:31:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.023+0000 7f13b63eb640 1 --2- 172.21.15.97:0/4087685457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13b01021d0 0x7f13b01986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.023+0000 7f13b63eb640 1 -- 172.21.15.97:0/4087685457 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13b019da40 con 0x7f13b019cf50 2024-04-23T10:31:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.023+0000 7f13b63eb640 1 --2- 172.21.15.97:0/4087685457 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13b019cf50 0x7f13b019d400 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f13a800d9c0 tx=0x7f13a800de80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:31:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.023+0000 7f13a6ffd640 1 -- 172.21.15.97:0/4087685457 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a800bd10 con 0x7f13b019cf50 2024-04-23T10:31:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.023+0000 7f13b7e4e640 1 -- 172.21.15.97:0/4087685457 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13b019dd30 con 0x7f13b019cf50 2024-04-23T10:31:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.023+0000 7f13a6ffd640 1 -- 172.21.15.97:0/4087685457 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13a8003e20 con 0x7f13b019cf50 2024-04-23T10:31:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.023+0000 7f13a6ffd640 1 -- 172.21.15.97:0/4087685457 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a80106c0 con 0x7f13b019cf50 2024-04-23T10:31:16.025 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.023+0000 7f13b7e4e640 1 -- 172.21.15.97:0/4087685457 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13b01a54d0 con 0x7f13b019cf50 2024-04-23T10:31:16.031 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.025+0000 7f13a6ffd640 1 -- 172.21.15.97:0/4087685457 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f13a8003970 con 0x7f13b019cf50 2024-04-23T10:31:16.031 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.026+0000 7f13bab65740 1 Processor -- start 2024-04-23T10:31:16.031 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.027+0000 7f13a6ffd640 1 --2- 172.21.15.97:0/4087685457 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f139c0a3dc0 0x7f139c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:16.031 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.027+0000 7f13bab65740 1 -- start start 2024-04-23T10:31:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.027+0000 7f13b53e9640 1 --2- 172.21.15.97:0/4087685457 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f139c0a3dc0 0x7f139c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.027+0000 7f13bab65740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccb9cb3620 0x55ccb9cb3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.027+0000 7f13bab65740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ccb9cb3f40 0x55ccb9cbe2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.027+0000 7f13bab65740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccb9cbe800 0x55ccb9cc0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.027+0000 7f13a6ffd640 1 -- 172.21.15.97:0/4087685457 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f13a80d0990 con 0x7f13b019cf50 2024-04-23T10:31:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.027+0000 7f13bab65740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ccb9c18970 con 0x55ccb9cbe800 2024-04-23T10:31:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.027+0000 7f13bab65740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ccb9bb54a0 con 0x55ccb9cb3f40 2024-04-23T10:31:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.027+0000 7f13bab65740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ccb9c14930 con 0x55ccb9cb3620 2024-04-23T10:31:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.027+0000 7f13b63eb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccb9cbe800 0x55ccb9cc0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.027+0000 7f13b63eb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccb9cbe800 0x55ccb9cc0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57998/0 (socket says 172.21.15.97:57998) 2024-04-23T10:31:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.027+0000 7f13b63eb640 1 -- 172.21.15.97:0/2152273386 learned_addr learned my addr 172.21.15.97:0/2152273386 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:31:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.027+0000 7f13b5bea640 1 --2- 172.21.15.97:0/2152273386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccb9cb3620 0x55ccb9cb3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.027+0000 7f13b53e9640 1 --2- 172.21.15.97:0/2152273386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ccb9cb3f40 0x55ccb9cbe2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.027+0000 7f13b63eb640 1 -- 172.21.15.97:0/2152273386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccb9cb3620 msgr2=0x55ccb9cb3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:31:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.027+0000 7f13b63eb640 1 --2- 172.21.15.97:0/2152273386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccb9cb3620 0x55ccb9cb3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.027+0000 7f13b63eb640 1 -- 172.21.15.97:0/2152273386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ccb9cb3f40 msgr2=0x55ccb9cbe2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:31:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.027+0000 7f13b63eb640 1 --2- 172.21.15.97:0/2152273386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ccb9cb3f40 0x55ccb9cbe2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.027+0000 7f13b63eb640 1 -- 172.21.15.97:0/2152273386 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccb992b210 con 0x55ccb9cbe800 2024-04-23T10:31:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.027+0000 7f13b53e9640 1 --2- 172.21.15.97:0/2152273386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ccb9cb3f40 0x55ccb9cbe2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:31:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.027+0000 7f13b53e9640 1 --2- 172.21.15.97:0/4087685457 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f139c0a3dc0 0x7f139c0a6280 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f13b0101810 tx=0x7f13a00107a0 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:31:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.027+0000 7f13b63eb640 1 --2- 172.21.15.97:0/2152273386 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccb9cbe800 0x55ccb9cc0bf0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f13a800dfd0 tx=0x7f13a809c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5ac4ef390392a9c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:31:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.028+0000 7f139bfff640 1 -- 172.21.15.97:0/2152273386 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a8003c60 con 0x55ccb9cbe800 2024-04-23T10:31:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.028+0000 7f139bfff640 1 -- 172.21.15.97:0/2152273386 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13a809d080 con 0x55ccb9cbe800 2024-04-23T10:31:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.028+0000 7f139bfff640 1 -- 172.21.15.97:0/2152273386 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a80042d0 con 0x55ccb9cbe800 2024-04-23T10:31:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.028+0000 7f13bab65740 1 -- 172.21.15.97:0/2152273386 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccb9cbe800 msgr2=0x55ccb9cc0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:31:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.028+0000 7f13bab65740 1 --2- 172.21.15.97:0/2152273386 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccb9cbe800 0x55ccb9cc0bf0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f13a800dfd0 tx=0x7f13a809c750 comp rx=0 tx=0).stop 2024-04-23T10:31:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.028+0000 7f13bab65740 1 -- 172.21.15.97:0/2152273386 shutdown_connections 2024-04-23T10:31:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.028+0000 7f13bab65740 1 --2- 172.21.15.97:0/2152273386 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccb9cbe800 0x55ccb9cc0bf0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:16.033 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.028+0000 7f13bab65740 1 --2- 172.21.15.97:0/2152273386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ccb9cb3f40 0x55ccb9cbe2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:16.034 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.028+0000 7f13bab65740 1 --2- 172.21.15.97:0/2152273386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccb9cb3620 0x55ccb9cb3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:16.034 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.028+0000 7f13bab65740 1 -- 172.21.15.97:0/2152273386 >> 172.21.15.97:0/2152273386 conn(0x55ccb9cafb30 msgr2=0x55ccb9caff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:31:16.034 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.029+0000 7f13bab65740 1 -- 172.21.15.97:0/2152273386 shutdown_connections 2024-04-23T10:31:16.034 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.029+0000 7f13bab65740 1 -- 172.21.15.97:0/2152273386 wait complete. 2024-04-23T10:31:16.034 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.029+0000 7f13bab65740 1 -- start start 2024-04-23T10:31:16.034 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.029+0000 7f13bab65740 1 Processor -- start 2024-04-23T10:31:16.034 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.029+0000 7f13bab65740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ccb9cb3620 0x55ccb9cbe280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:16.034 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.029+0000 7f13bab65740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccb9cb3f40 0x55ccb9cbb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:16.034 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.029+0000 7f13bab65740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccb9cbe800 0x55ccb9cbbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:16.034 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.029+0000 7f13bab65740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ccb9cc1430 con 0x55ccb9cbe800 2024-04-23T10:31:16.034 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.029+0000 7f13bab65740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ccb9bb30b0 con 0x55ccb9cb3620 2024-04-23T10:31:16.034 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.029+0000 7f13bab65740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ccb9cc15b0 con 0x55ccb9cb3f40 2024-04-23T10:31:16.034 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.029+0000 7f13bab65740 10 client.0 _ll_register_callbacks cb 0x55ccb9ba0d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:31:16.034 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.029+0000 7f13b63eb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccb9cbe800 0x55ccb9cbbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:16.035 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.030+0000 7f13b63eb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccb9cbe800 0x55ccb9cbbcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58002/0 (socket says 172.21.15.97:58002) 2024-04-23T10:31:16.035 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.030+0000 7f13b63eb640 1 -- 172.21.15.97:0/3799144409 learned_addr learned my addr 172.21.15.97:0/3799144409 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:31:16.035 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.030+0000 7f13b53e9640 1 --2- 172.21.15.97:0/3799144409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccb9cb3f40 0x55ccb9cbb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:16.035 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.030+0000 7f13b5bea640 1 --2- 172.21.15.97:0/3799144409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ccb9cb3620 0x55ccb9cbe280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:16.035 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.030+0000 7f13b63eb640 1 -- 172.21.15.97:0/3799144409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccb9cb3f40 msgr2=0x55ccb9cbb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:31:16.035 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.030+0000 7f13b63eb640 1 --2- 172.21.15.97:0/3799144409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccb9cb3f40 0x55ccb9cbb570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:16.035 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.030+0000 7f13b63eb640 1 -- 172.21.15.97:0/3799144409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ccb9cb3620 msgr2=0x55ccb9cbe280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:31:16.035 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.030+0000 7f13b63eb640 1 --2- 172.21.15.97:0/3799144409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ccb9cb3620 0x55ccb9cbe280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:16.035 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.030+0000 7f13b63eb640 1 -- 172.21.15.97:0/3799144409 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccb99f7620 con 0x55ccb9cbe800 2024-04-23T10:31:16.035 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.030+0000 7f13b5bea640 1 --2- 172.21.15.97:0/3799144409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ccb9cb3620 0x55ccb9cbe280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:31:16.035 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.030+0000 7f13b63eb640 1 --2- 172.21.15.97:0/3799144409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccb9cbe800 0x55ccb9cbbcb0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f13a80ce730 tx=0x7f13a8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1664d951145b9d09 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:31:16.035 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.030+0000 7f13997fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:31:16.035 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.030+0000 7f13997fa640 1 -- 172.21.15.97:0/3799144409 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a809c980 con 0x55ccb9cbe800 2024-04-23T10:31:16.035 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.030+0000 7f13997fa640 1 -- 172.21.15.97:0/3799144409 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f13a8098ff0 con 0x55ccb9cbe800 2024-04-23T10:31:16.035 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.030+0000 7f13bab65740 1 -- 172.21.15.97:0/3799144409 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13a809d080 con 0x55ccb9cbe800 2024-04-23T10:31:16.036 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.030+0000 7f13997fa640 1 -- 172.21.15.97:0/3799144409 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f13a80bb020 con 0x55ccb9cbe800 2024-04-23T10:31:16.036 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.030+0000 7f13bab65740 10 client.16287 fetch_fsmap learned FSMap version 26 2024-04-23T10:31:16.036 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.030+0000 7f13bab65740 1 -- 172.21.15.97:0/3799144409 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ccb9cc1430 con 0x55ccb9cbe800 2024-04-23T10:31:16.036 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.031+0000 7f13997fa640 1 -- 172.21.15.97:0/3799144409 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13a809d080 con 0x55ccb9cbe800 2024-04-23T10:31:16.036 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.031+0000 7f13997fa640 1 -- 172.21.15.97:0/3799144409 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f13a8024060 con 0x55ccb9cbe800 2024-04-23T10:31:16.036 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.031+0000 7f13bab65740 10 client.16287 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:31:16.036 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.031+0000 7f13bab65740 10 client.16287 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:31:16.036 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.031+0000 7f13bab65740 20 client.16287 populate_metadata read hostname 'smithi097' 2024-04-23T10:31:16.036 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.031+0000 7f13bab65740 1 --2- 172.21.15.97:0/3799144409 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55ccb9cce600 0x55ccb9cd09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:16.036 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.031+0000 7f13bab65740 4 client.16287 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:31:16.036 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.031+0000 7f13bab65740 1 -- 172.21.15.97:0/3799144409 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ccb9c26390 con 0x55ccb9cce600 2024-04-23T10:31:16.036 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.031+0000 7f13b5bea640 1 --2- 172.21.15.97:0/3799144409 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55ccb9cce600 0x55ccb9cd09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:16.036 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.031+0000 7f13997fa640 10 client.16287 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:31:16.036 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.031+0000 7f13b5bea640 1 --2- 172.21.15.97:0/3799144409 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55ccb9cce600 0x55ccb9cd09f0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e61ff49adf76cae server_cookie=4c964f400e01fda8 in_seq=0 out_seq=0 2024-04-23T10:31:16.036 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.032+0000 7f13997fa640 1 -- 172.21.15.97:0/3799144409 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ccb9c26390 con 0x55ccb9cce600 2024-04-23T10:31:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.032+0000 7f13997fa640 10 client.16287 handle_command_reply: tid=0 2024-04-23T10:31:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.032+0000 7f13bab65740 1 client.16287 shutdown 2024-04-23T10:31:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.032+0000 7f13bab65740 1 -- 172.21.15.97:0/3799144409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccb9cbe800 msgr2=0x55ccb9cbbcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:31:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.032+0000 7f13bab65740 1 --2- 172.21.15.97:0/3799144409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccb9cbe800 0x55ccb9cbbcb0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f13a80ce730 tx=0x7f13a8004a40 comp rx=0 tx=0).stop 2024-04-23T10:31:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.032+0000 7f13bab65740 1 -- 172.21.15.97:0/3799144409 shutdown_connections 2024-04-23T10:31:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.032+0000 7f13bab65740 1 --2- 172.21.15.97:0/3799144409 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55ccb9cce600 0x55ccb9cd09f0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.032+0000 7f13bab65740 1 --2- 172.21.15.97:0/3799144409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccb9cbe800 0x55ccb9cbbcb0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.032+0000 7f13bab65740 1 --2- 172.21.15.97:0/3799144409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccb9cb3f40 0x55ccb9cbb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.032+0000 7f13bab65740 1 --2- 172.21.15.97:0/3799144409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ccb9cb3620 0x55ccb9cbe280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.032+0000 7f13997fa640 0 client.16287 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:31:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.032+0000 7f13bab65740 1 -- 172.21.15.97:0/3799144409 >> 172.21.15.97:0/3799144409 conn(0x55ccb9cafb30 msgr2=0x55ccb9caecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:31:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.032+0000 7f13bab65740 1 -- 172.21.15.97:0/3799144409 shutdown_connections 2024-04-23T10:31:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.032+0000 7f13bab65740 1 -- 172.21.15.97:0/3799144409 wait complete. 2024-04-23T10:31:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.033+0000 7f13bab65740 20 client.16287 trim_cache size 0 max 16384 2024-04-23T10:31:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.047+0000 7f13bab65740 1 Processor -- start 2024-04-23T10:31:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.047+0000 7f13bab65740 1 -- start start 2024-04-23T10:31:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.047+0000 7f13bab65740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccb9cb3620 0x55ccb9cd3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.048+0000 7f13bab65740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ccb9cb3f40 0x55ccb9cbddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.048+0000 7f13bab65740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccb9cbe800 0x55ccb9cb8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.048+0000 7f13bab65740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f13b01ad990 con 0x55ccb9cb3620 2024-04-23T10:31:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.048+0000 7f13bab65740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f13b01af4d0 con 0x55ccb9cb3f40 2024-04-23T10:31:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.048+0000 7f13bab65740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f13b01ae060 con 0x55ccb9cbe800 2024-04-23T10:31:16.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.048+0000 7f13b5bea640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccb9cb3620 0x55ccb9cd3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:16.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.048+0000 7f13b5bea640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccb9cb3620 0x55ccb9cd3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58018/0 (socket says 172.21.15.97:58018) 2024-04-23T10:31:16.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.048+0000 7f13b5bea640 1 -- 172.21.15.97:0/4115105986 learned_addr learned my addr 172.21.15.97:0/4115105986 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:31:16.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.048+0000 7f13b5bea640 1 -- 172.21.15.97:0/4115105986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccb9cbe800 msgr2=0x55ccb9cb8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:31:16.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.048+0000 7f13b5bea640 1 --2- 172.21.15.97:0/4115105986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccb9cbe800 0x55ccb9cb8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:16.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.048+0000 7f13b5bea640 1 -- 172.21.15.97:0/4115105986 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ccb9cb3f40 msgr2=0x55ccb9cbddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:31:16.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.048+0000 7f13b5bea640 1 --2- 172.21.15.97:0/4115105986 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ccb9cb3f40 0x55ccb9cbddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:16.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.048+0000 7f13b5bea640 1 -- 172.21.15.97:0/4115105986 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccb9caf930 con 0x55ccb9cb3620 2024-04-23T10:31:16.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.048+0000 7f13b5bea640 1 --2- 172.21.15.97:0/4115105986 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccb9cb3620 0x55ccb9cd3e00 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f13ac000c00 tx=0x7f13ac009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad6017a3c9e23eca server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:31:16.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.048+0000 7f139b7fe640 1 -- 172.21.15.97:0/4115105986 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ac049070 con 0x55ccb9cb3620 2024-04-23T10:31:16.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.048+0000 7f139b7fe640 1 -- 172.21.15.97:0/4115105986 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13ac031950 con 0x55ccb9cb3620 2024-04-23T10:31:16.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.048+0000 7f139b7fe640 1 -- 172.21.15.97:0/4115105986 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ac035040 con 0x55ccb9cb3620 2024-04-23T10:31:16.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.049+0000 7f13bab65740 1 -- 172.21.15.97:0/4115105986 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccb9cb3620 msgr2=0x55ccb9cd3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:31:16.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.049+0000 7f13bab65740 1 --2- 172.21.15.97:0/4115105986 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccb9cb3620 0x55ccb9cd3e00 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f13ac000c00 tx=0x7f13ac009260 comp rx=0 tx=0).stop 2024-04-23T10:31:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.049+0000 7f13bab65740 1 -- 172.21.15.97:0/4115105986 shutdown_connections 2024-04-23T10:31:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.049+0000 7f13bab65740 1 --2- 172.21.15.97:0/4115105986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccb9cbe800 0x55ccb9cb8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.049+0000 7f13bab65740 1 --2- 172.21.15.97:0/4115105986 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ccb9cb3f40 0x55ccb9cbddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.049+0000 7f13bab65740 1 --2- 172.21.15.97:0/4115105986 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccb9cb3620 0x55ccb9cd3e00 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.049+0000 7f13bab65740 1 -- 172.21.15.97:0/4115105986 >> 172.21.15.97:0/4115105986 conn(0x55ccb9cafb30 msgr2=0x55ccb9caecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:31:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.049+0000 7f13bab65740 1 -- 172.21.15.97:0/4115105986 shutdown_connections 2024-04-23T10:31:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.049+0000 7f13bab65740 1 -- 172.21.15.97:0/4115105986 wait complete. 2024-04-23T10:31:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.049+0000 7f13bab65740 1 -- start start 2024-04-23T10:31:16.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.049+0000 7f13bab65740 1 Processor -- start 2024-04-23T10:31:16.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.050+0000 7f13bab65740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccb9cb3620 0x55ccb9d312a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:16.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.050+0000 7f13bab65740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccb9cb3f40 0x55ccb9d317e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:16.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.050+0000 7f13bab65740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ccb9cbe800 0x55ccb9d2b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:16.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.050+0000 7f13bab65740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f13b01aee00 con 0x55ccb9cb3f40 2024-04-23T10:31:16.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.050+0000 7f13bab65740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f13b01ae730 con 0x55ccb9cbe800 2024-04-23T10:31:16.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.050+0000 7f13bab65740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f13b01ab970 con 0x55ccb9cb3620 2024-04-23T10:31:16.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.050+0000 7f13bab65740 10 client.0 _ll_register_callbacks cb 0x7f13a8098520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:31:16.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.050+0000 7f13b53e9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccb9cb3f40 0x55ccb9d317e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:16.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.050+0000 7f13b53e9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccb9cb3f40 0x55ccb9d317e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58028/0 (socket says 172.21.15.97:58028) 2024-04-23T10:31:16.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.050+0000 7f13b53e9640 1 -- 172.21.15.97:0/2424514704 learned_addr learned my addr 172.21.15.97:0/2424514704 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:31:16.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.050+0000 7f13b63eb640 1 --2- 172.21.15.97:0/2424514704 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ccb9cbe800 0x55ccb9d2b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:16.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.050+0000 7f13b5bea640 1 --2- 172.21.15.97:0/2424514704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccb9cb3620 0x55ccb9d312a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:16.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.050+0000 7f13b53e9640 1 -- 172.21.15.97:0/2424514704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccb9cb3620 msgr2=0x55ccb9d312a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:31:16.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.050+0000 7f13b53e9640 1 --2- 172.21.15.97:0/2424514704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccb9cb3620 0x55ccb9d312a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:16.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.050+0000 7f13b53e9640 1 -- 172.21.15.97:0/2424514704 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ccb9cbe800 msgr2=0x55ccb9d2b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:31:16.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.050+0000 7f13b53e9640 1 --2- 172.21.15.97:0/2424514704 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ccb9cbe800 0x55ccb9d2b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:16.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.050+0000 7f13b53e9640 1 -- 172.21.15.97:0/2424514704 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccb9d2bda0 con 0x55ccb9cb3f40 2024-04-23T10:31:16.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.050+0000 7f13b53e9640 1 --2- 172.21.15.97:0/2424514704 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccb9cb3f40 0x55ccb9d317e0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f13a0005fd0 tx=0x7f13a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dbd1b191d88a480 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:31:16.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.050+0000 7f13997fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:31:16.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.050+0000 7f13997fa640 1 -- 172.21.15.97:0/2424514704 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a0020700 con 0x55ccb9cb3f40 2024-04-23T10:31:16.055 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.050+0000 7f13997fa640 1 -- 172.21.15.97:0/2424514704 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f13a00208a0 con 0x55ccb9cb3f40 2024-04-23T10:31:16.055 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.050+0000 7f13bab65740 1 -- 172.21.15.97:0/2424514704 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ccb9c20200 con 0x55ccb9cb3f40 2024-04-23T10:31:16.055 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.051+0000 7f13997fa640 1 -- 172.21.15.97:0/2424514704 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f13a0020aa0 con 0x55ccb9cb3f40 2024-04-23T10:31:16.055 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.051+0000 7f13bab65740 10 client.16293 fetch_fsmap learned FSMap version 26 2024-04-23T10:31:16.055 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.051+0000 7f13bab65740 1 -- 172.21.15.97:0/2424514704 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ccb9d38b40 con 0x55ccb9cb3f40 2024-04-23T10:31:16.055 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.051+0000 7f13997fa640 1 -- 172.21.15.97:0/2424514704 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55ccb9c20200 con 0x55ccb9cb3f40 2024-04-23T10:31:16.055 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.051+0000 7f13997fa640 1 -- 172.21.15.97:0/2424514704 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f13a0027070 con 0x55ccb9cb3f40 2024-04-23T10:31:16.055 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.051+0000 7f13bab65740 10 client.16293 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:31:16.055 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.051+0000 7f13bab65740 10 client.16293 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:31:16.055 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.051+0000 7f13bab65740 20 client.16293 populate_metadata read hostname 'smithi097' 2024-04-23T10:31:16.055 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.051+0000 7f13bab65740 1 --2- 172.21.15.97:0/2424514704 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55ccb9d39370 0x55ccb9d3b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:16.055 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.051+0000 7f13bab65740 4 client.16293 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:31:16.055 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.051+0000 7f13bab65740 1 -- 172.21.15.97:0/2424514704 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ccb9b9c5f0 con 0x55ccb9d39370 2024-04-23T10:31:16.055 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.051+0000 7f13b5bea640 1 --2- 172.21.15.97:0/2424514704 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55ccb9d39370 0x55ccb9d3b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.052+0000 7f13997fa640 10 client.16293 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:31:16.056 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.052+0000 7f13b5bea640 1 --2- 172.21.15.97:0/2424514704 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55ccb9d39370 0x55ccb9d3b790 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4215f238d54c3837 server_cookie=4ab628b2cf7c6e54 in_seq=0 out_seq=0 2024-04-23T10:31:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.065+0000 7f13997fa640 1 -- 172.21.15.97:0/2424514704 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ccb9b9c5f0 con 0x55ccb9d39370 2024-04-23T10:31:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.065+0000 7f13997fa640 10 client.16293 handle_command_reply: tid=0 2024-04-23T10:31:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.065+0000 7f13bab65740 1 client.16293 shutdown 2024-04-23T10:31:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.066+0000 7f13bab65740 1 -- 172.21.15.97:0/2424514704 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccb9cb3f40 msgr2=0x55ccb9d317e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:31:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.066+0000 7f13bab65740 1 --2- 172.21.15.97:0/2424514704 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccb9cb3f40 0x55ccb9d317e0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f13a0005fd0 tx=0x7f13a00046c0 comp rx=0 tx=0).stop 2024-04-23T10:31:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.066+0000 7f13bab65740 1 -- 172.21.15.97:0/2424514704 shutdown_connections 2024-04-23T10:31:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.066+0000 7f13bab65740 1 --2- 172.21.15.97:0/2424514704 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55ccb9d39370 0x55ccb9d3b790 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.066+0000 7f13bab65740 1 --2- 172.21.15.97:0/2424514704 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ccb9cbe800 0x55ccb9d2b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.066+0000 7f13bab65740 1 --2- 172.21.15.97:0/2424514704 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ccb9cb3f40 0x55ccb9d317e0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.066+0000 7f13bab65740 1 --2- 172.21.15.97:0/2424514704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ccb9cb3620 0x55ccb9d312a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.066+0000 7f13997fa640 0 client.16293 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:31:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.066+0000 7f13bab65740 1 -- 172.21.15.97:0/2424514704 >> 172.21.15.97:0/2424514704 conn(0x55ccb9cafb30 msgr2=0x55ccb9cb9d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:31:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.066+0000 7f13bab65740 1 -- 172.21.15.97:0/2424514704 shutdown_connections 2024-04-23T10:31:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.066+0000 7f13bab65740 1 -- 172.21.15.97:0/2424514704 wait complete. 2024-04-23T10:31:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.066+0000 7f13bab65740 20 client.16293 trim_cache size 0 max 16384 2024-04-23T10:31:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:31:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:31:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:31:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:31:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.066+0000 7f13b7e4e640 1 -- 172.21.15.97:0/4087685457 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f139c0a3dc0 msgr2=0x7f139c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:31:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.066+0000 7f13b7e4e640 1 --2- 172.21.15.97:0/4087685457 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f139c0a3dc0 0x7f139c0a6280 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f13b0101810 tx=0x7f13a00107a0 comp rx=0 tx=0).stop 2024-04-23T10:31:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.066+0000 7f13b7e4e640 1 -- 172.21.15.97:0/4087685457 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13b019cf50 msgr2=0x7f13b019d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:31:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.066+0000 7f13b7e4e640 1 --2- 172.21.15.97:0/4087685457 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13b019cf50 0x7f13b019d400 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f13a800d9c0 tx=0x7f13a800de80 comp rx=0 tx=0).stop 2024-04-23T10:31:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.066+0000 7f13b7e4e640 1 -- 172.21.15.97:0/4087685457 shutdown_connections 2024-04-23T10:31:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.066+0000 7f13b7e4e640 1 --2- 172.21.15.97:0/4087685457 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f139c0a3dc0 0x7f139c0a6280 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.066+0000 7f13b7e4e640 1 --2- 172.21.15.97:0/4087685457 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13b019cf50 0x7f13b019d400 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.066+0000 7f13b7e4e640 1 --2- 172.21.15.97:0/4087685457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f13b01072a0 0x7f13b0198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.066+0000 7f13b7e4e640 1 --2- 172.21.15.97:0/4087685457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13b01021d0 0x7f13b01986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.066+0000 7f13b7e4e640 1 -- 172.21.15.97:0/4087685457 >> 172.21.15.97:0/4087685457 conn(0x7f13b00793e0 msgr2=0x7f13b00ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:31:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.067+0000 7f13b7e4e640 1 -- 172.21.15.97:0/4087685457 shutdown_connections 2024-04-23T10:31:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:16.067+0000 7f13b7e4e640 1 -- 172.21.15.97:0/4087685457 wait complete. 2024-04-23T10:31:16.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:31:16 smithi196 ceph-mon[32919]: pgmap v155: 97 pgs: 97 active+clean; 5.5 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 36 KiB/s wr, 16 op/s 2024-04-23T10:31:16.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:31:16 smithi097 ceph-mon[29197]: pgmap v155: 97 pgs: 97 active+clean; 5.5 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 36 KiB/s wr, 16 op/s 2024-04-23T10:31:16.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:31:16 smithi183 ceph-mon[32317]: pgmap v155: 97 pgs: 97 active+clean; 5.5 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 36 KiB/s wr, 16 op/s 2024-04-23T10:31:18.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:31:18 smithi196 ceph-mon[32919]: pgmap v156: 97 pgs: 97 active+clean; 1.7 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 26 op/s 2024-04-23T10:31:18.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:31:18 smithi097 ceph-mon[29197]: pgmap v156: 97 pgs: 97 active+clean; 1.7 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 26 op/s 2024-04-23T10:31:18.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:31:18 smithi183 ceph-mon[32317]: pgmap v156: 97 pgs: 97 active+clean; 1.7 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 56 KiB/s wr, 26 op/s 2024-04-23T10:31:20.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:31:20 smithi196 ceph-mon[32919]: pgmap v157: 97 pgs: 97 active+clean; 1.7 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 KiB/s wr, 16 op/s 2024-04-23T10:31:20.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:31:20 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:31:20.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:31:20 smithi097 ceph-mon[29197]: pgmap v157: 97 pgs: 97 active+clean; 1.7 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 KiB/s wr, 16 op/s 2024-04-23T10:31:20.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:31:20 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:31:20.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:31:20 smithi183 ceph-mon[32317]: pgmap v157: 97 pgs: 97 active+clean; 1.7 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 KiB/s wr, 16 op/s 2024-04-23T10:31:20.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:31:20 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:31:21.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:31:21 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:31:21.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:31:21 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:31:21.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:31:21 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:31:21.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:31:21 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:31:21.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:31:21 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:31:21.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:31:21 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:31:21.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:31:21 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:31:21.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:31:21 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:31:21.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:31:21 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:31:23.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:31:22 smithi196 ceph-mon[32919]: pgmap v158: 97 pgs: 97 active+clean; 1.7 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 KiB/s wr, 16 op/s 2024-04-23T10:31:23.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:31:22 smithi097 ceph-mon[29197]: pgmap v158: 97 pgs: 97 active+clean; 1.7 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 KiB/s wr, 16 op/s 2024-04-23T10:31:23.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:31:22 smithi183 ceph-mon[32317]: pgmap v158: 97 pgs: 97 active+clean; 1.7 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 KiB/s wr, 16 op/s 2024-04-23T10:31:25.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:31:24 smithi196 ceph-mon[32919]: pgmap v159: 97 pgs: 97 active+clean; 1.8 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 50 KiB/s wr, 22 op/s 2024-04-23T10:31:25.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:31:24 smithi097 ceph-mon[29197]: pgmap v159: 97 pgs: 97 active+clean; 1.8 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 50 KiB/s wr, 22 op/s 2024-04-23T10:31:25.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:31:24 smithi183 ceph-mon[32317]: pgmap v159: 97 pgs: 97 active+clean; 1.8 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 50 KiB/s wr, 22 op/s 2024-04-23T10:31:27.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:31:26 smithi196 ceph-mon[32919]: pgmap v160: 97 pgs: 97 active+clean; 1.8 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 KiB/s wr, 15 op/s 2024-04-23T10:31:27.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:31:26 smithi097 ceph-mon[29197]: pgmap v160: 97 pgs: 97 active+clean; 1.8 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 KiB/s wr, 15 op/s 2024-04-23T10:31:27.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:31:26 smithi183 ceph-mon[32317]: pgmap v160: 97 pgs: 97 active+clean; 1.8 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 35 KiB/s wr, 15 op/s 2024-04-23T10:31:29.162 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.162+0000 7fbfbdfdb640 1 -- 172.21.15.97:0/1209751381 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbfb81021c0 msgr2=0x7fbfb81025a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:31:29.162 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.162+0000 7fbfbdfdb640 1 --2- 172.21.15.97:0/1209751381 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbfb81021c0 0x7fbfb81025a0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fbfac009900 tx=0x7fbfac02a410 comp rx=0 tx=0).stop 2024-04-23T10:31:29.162 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.162+0000 7fbfbdfdb640 1 -- 172.21.15.97:0/1209751381 shutdown_connections 2024-04-23T10:31:29.162 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.162+0000 7fbfbdfdb640 1 --2- 172.21.15.97:0/1209751381 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbfb8107290 0x7fbfb810ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:29.162 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.162+0000 7fbfbdfdb640 1 --2- 172.21.15.97:0/1209751381 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbfb8102b70 0x7fbfb8106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:29.162 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.162+0000 7fbfbdfdb640 1 --2- 172.21.15.97:0/1209751381 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbfb81021c0 0x7fbfb81025a0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:29.162 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.162+0000 7fbfbdfdb640 1 -- 172.21.15.97:0/1209751381 >> 172.21.15.97:0/1209751381 conn(0x7fbfb80fda20 msgr2=0x7fbfb80ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:31:29.163 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.162+0000 7fbfbdfdb640 1 -- 172.21.15.97:0/1209751381 shutdown_connections 2024-04-23T10:31:29.163 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.163+0000 7fbfbdfdb640 1 -- 172.21.15.97:0/1209751381 wait complete. 2024-04-23T10:31:29.163 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.163+0000 7fbfbdfdb640 1 Processor -- start 2024-04-23T10:31:29.163 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.163+0000 7fbfbdfdb640 1 -- start start 2024-04-23T10:31:29.165 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.164+0000 7fbfbdfdb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbfb81021c0 0x7fbfb8198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:29.165 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.164+0000 7fbfbdfdb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbfb8107290 0x7fbfb8198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:29.165 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.164+0000 7fbfbdfdb640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbfb819cfa0 0x7fbfb819d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:29.165 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.164+0000 7fbfbdfdb640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb81121f0 con 0x7fbfb8107290 2024-04-23T10:31:29.165 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.164+0000 7fbfbdfdb640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb8112070 con 0x7fbfb819cfa0 2024-04-23T10:31:29.165 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.164+0000 7fbfbdfdb640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb8112370 con 0x7fbfb81021c0 2024-04-23T10:31:29.165 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.164+0000 7fbfb6ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbfb8107290 0x7fbfb8198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:29.165 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.164+0000 7fbfb6ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbfb8107290 0x7fbfb8198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58698/0 (socket says 172.21.15.97:58698) 2024-04-23T10:31:29.165 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.164+0000 7fbfb6ffd640 1 -- 172.21.15.97:0/893399250 learned_addr learned my addr 172.21.15.97:0/893399250 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:31:29.165 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.164+0000 7fbfb77fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbfb81021c0 0x7fbfb8198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:29.165 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.164+0000 7fbfb7fff640 1 --2- 172.21.15.97:0/893399250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbfb819cfa0 0x7fbfb819d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:29.165 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.164+0000 7fbfb6ffd640 1 -- 172.21.15.97:0/893399250 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbfb81021c0 msgr2=0x7fbfb8198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:31:29.165 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.164+0000 7fbfb6ffd640 1 --2- 172.21.15.97:0/893399250 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbfb81021c0 0x7fbfb8198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:29.166 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.164+0000 7fbfb6ffd640 1 -- 172.21.15.97:0/893399250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbfb819cfa0 msgr2=0x7fbfb819d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:31:29.166 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.164+0000 7fbfb6ffd640 1 --2- 172.21.15.97:0/893399250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbfb819cfa0 0x7fbfb819d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:29.166 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.164+0000 7fbfb6ffd640 1 -- 172.21.15.97:0/893399250 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfb819da90 con 0x7fbfb8107290 2024-04-23T10:31:29.166 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.164+0000 7fbfb6ffd640 1 --2- 172.21.15.97:0/893399250 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbfb8107290 0x7fbfb8198c70 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fbfa800d740 tx=0x7fbfa800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:31:29.166 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.164+0000 7fbfb4ff9640 1 -- 172.21.15.97:0/893399250 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa8017070 con 0x7fbfb8107290 2024-04-23T10:31:29.166 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.164+0000 7fbfb4ff9640 1 -- 172.21.15.97:0/893399250 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfa800de50 con 0x7fbfb8107290 2024-04-23T10:31:29.166 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.164+0000 7fbfbdfdb640 1 -- 172.21.15.97:0/893399250 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfb819dd80 con 0x7fbfb8107290 2024-04-23T10:31:29.166 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.164+0000 7fbfb4ff9640 1 -- 172.21.15.97:0/893399250 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa8004ce0 con 0x7fbfb8107290 2024-04-23T10:31:29.166 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.164+0000 7fbfbdfdb640 1 -- 172.21.15.97:0/893399250 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfb81a5520 con 0x7fbfb8107290 2024-04-23T10:31:29.173 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.167+0000 7fbfb4ff9640 1 -- 172.21.15.97:0/893399250 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbfa8004e80 con 0x7fbfb8107290 2024-04-23T10:31:29.173 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.167+0000 7fbfc0dcd740 1 Processor -- start 2024-04-23T10:31:29.173 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.167+0000 7fbfc0dcd740 1 -- start start 2024-04-23T10:31:29.173 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.167+0000 7fbfc0dcd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f81b79c620 0x55f81b79ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:29.173 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.167+0000 7fbfc0dcd740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f81b79cf40 0x55f81b7a72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:29.173 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.167+0000 7fbfc0dcd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f81b7a7800 0x55f81b7a9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:29.173 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.167+0000 7fbfc0dcd740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f81b701970 con 0x55f81b79cf40 2024-04-23T10:31:29.174 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.167+0000 7fbfc0dcd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f81b69e4a0 con 0x55f81b7a7800 2024-04-23T10:31:29.174 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.167+0000 7fbfc0dcd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f81b6fd930 con 0x55f81b79c620 2024-04-23T10:31:29.174 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.167+0000 7fbfb4ff9640 1 --2- 172.21.15.97:0/893399250 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fbfa00a3dc0 0x7fbfa00a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:29.174 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.167+0000 7fbfb77fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f81b79cf40 0x55f81b7a72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:29.174 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.167+0000 7fbfb77fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f81b79cf40 0x55f81b7a72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58712/0 (socket says 172.21.15.97:58712) 2024-04-23T10:31:29.174 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.167+0000 7fbfb77fe640 1 -- 172.21.15.97:0/2425548568 learned_addr learned my addr 172.21.15.97:0/2425548568 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:31:29.174 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.167+0000 7fbfb7fff640 1 --2- 172.21.15.97:0/893399250 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fbfa00a3dc0 0x7fbfa00a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:29.174 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.168+0000 7fbfb7fff640 1 --2- 172.21.15.97:0/2425548568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f81b79c620 0x55f81b79ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:29.174 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.168+0000 7fbfb4ff9640 1 -- 172.21.15.97:0/893399250 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fbfa80d00b0 con 0x7fbfb8107290 2024-04-23T10:31:29.174 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.168+0000 7fbfb6ffd640 1 --2- 172.21.15.97:0/2425548568 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f81b7a7800 0x55f81b7a9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:29.174 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.168+0000 7fbfb77fe640 1 -- 172.21.15.97:0/2425548568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f81b79c620 msgr2=0x55f81b79ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:31:29.174 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.168+0000 7fbfb77fe640 1 --2- 172.21.15.97:0/2425548568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f81b79c620 0x55f81b79ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:29.174 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.168+0000 7fbfb77fe640 1 -- 172.21.15.97:0/2425548568 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f81b7a7800 msgr2=0x55f81b7a9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:31:29.174 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.168+0000 7fbfb77fe640 1 --2- 172.21.15.97:0/2425548568 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f81b7a7800 0x55f81b7a9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:29.175 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.168+0000 7fbfb77fe640 1 -- 172.21.15.97:0/2425548568 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f81b414210 con 0x55f81b79cf40 2024-04-23T10:31:29.175 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.168+0000 7fbfb7fff640 1 --2- 172.21.15.97:0/2425548568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f81b79c620 0x55f81b79ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:31:29.175 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.168+0000 7fbfb6ffd640 1 --2- 172.21.15.97:0/2425548568 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f81b7a7800 0x55f81b7a9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:31:29.175 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.168+0000 7fbfb77fe640 1 --2- 172.21.15.97:0/2425548568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f81b79cf40 0x55f81b7a72c0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fbfac009900 tx=0x7fbfac032bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d52acb7c84a821b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:31:29.175 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.168+0000 7fbfb7fff640 1 --2- 172.21.15.97:0/893399250 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fbfa00a3dc0 0x7fbfa00a6280 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0009fd0 tx=0x7fbfb0009ad0 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:31:29.175 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.168+0000 7fbf9dffb640 1 -- 172.21.15.97:0/2425548568 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfac040070 con 0x55f81b79cf40 2024-04-23T10:31:29.175 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.168+0000 7fbf9dffb640 1 -- 172.21.15.97:0/2425548568 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfac004030 con 0x55f81b79cf40 2024-04-23T10:31:29.175 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.168+0000 7fbf9dffb640 1 -- 172.21.15.97:0/2425548568 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfac035040 con 0x55f81b79cf40 2024-04-23T10:31:29.175 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.169+0000 7fbfc0dcd740 1 -- 172.21.15.97:0/2425548568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f81b79cf40 msgr2=0x55f81b7a72c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:31:29.175 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.169+0000 7fbfc0dcd740 1 --2- 172.21.15.97:0/2425548568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f81b79cf40 0x55f81b7a72c0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fbfac009900 tx=0x7fbfac032bd0 comp rx=0 tx=0).stop 2024-04-23T10:31:29.175 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.169+0000 7fbfc0dcd740 1 -- 172.21.15.97:0/2425548568 shutdown_connections 2024-04-23T10:31:29.175 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.169+0000 7fbfc0dcd740 1 --2- 172.21.15.97:0/2425548568 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f81b7a7800 0x55f81b7a9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:29.175 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.169+0000 7fbfc0dcd740 1 --2- 172.21.15.97:0/2425548568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f81b79cf40 0x55f81b7a72c0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:29.175 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.169+0000 7fbfc0dcd740 1 --2- 172.21.15.97:0/2425548568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f81b79c620 0x55f81b79ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:29.175 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.169+0000 7fbfc0dcd740 1 -- 172.21.15.97:0/2425548568 >> 172.21.15.97:0/2425548568 conn(0x55f81b798b30 msgr2=0x55f81b798f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:31:29.176 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.169+0000 7fbfc0dcd740 1 -- 172.21.15.97:0/2425548568 shutdown_connections 2024-04-23T10:31:29.176 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.169+0000 7fbfc0dcd740 1 -- 172.21.15.97:0/2425548568 wait complete. 2024-04-23T10:31:29.176 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.169+0000 7fbfc0dcd740 1 -- start start 2024-04-23T10:31:29.176 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.170+0000 7fbfc0dcd740 1 Processor -- start 2024-04-23T10:31:29.176 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.170+0000 7fbfc0dcd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f81b79c620 0x55f81b7a7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:29.176 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.170+0000 7fbfc0dcd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f81b79cf40 0x55f81b7a4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:29.176 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.170+0000 7fbfc0dcd740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f81b7a7800 0x55f81b7a49b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:29.176 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.170+0000 7fbfc0dcd740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f81b7aa3d0 con 0x55f81b7a7800 2024-04-23T10:31:29.176 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.170+0000 7fbfc0dcd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f81b69c0b0 con 0x55f81b79c620 2024-04-23T10:31:29.176 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.170+0000 7fbfc0dcd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f81b7aa550 con 0x55f81b79cf40 2024-04-23T10:31:29.176 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.170+0000 7fbfc0dcd740 10 client.0 _ll_register_callbacks cb 0x55f81b689d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:31:29.176 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.170+0000 7fbfb7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f81b7a7800 0x55f81b7a49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:29.176 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.170+0000 7fbfb7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f81b7a7800 0x55f81b7a49b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58724/0 (socket says 172.21.15.97:58724) 2024-04-23T10:31:29.176 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.170+0000 7fbfb7fff640 1 -- 172.21.15.97:0/3532324879 learned_addr learned my addr 172.21.15.97:0/3532324879 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:31:29.176 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.170+0000 7fbfb77fe640 1 --2- 172.21.15.97:0/3532324879 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f81b79c620 0x55f81b7a7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:29.177 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.170+0000 7fbfb6ffd640 1 --2- 172.21.15.97:0/3532324879 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f81b79cf40 0x55f81b7a4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:29.177 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.170+0000 7fbfb7fff640 1 -- 172.21.15.97:0/3532324879 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f81b79cf40 msgr2=0x55f81b7a4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:31:29.177 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.170+0000 7fbfb7fff640 1 --2- 172.21.15.97:0/3532324879 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f81b79cf40 0x55f81b7a4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:29.177 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.170+0000 7fbfb7fff640 1 -- 172.21.15.97:0/3532324879 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f81b79c620 msgr2=0x55f81b7a7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:31:29.177 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.170+0000 7fbfb7fff640 1 --2- 172.21.15.97:0/3532324879 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f81b79c620 0x55f81b7a7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:29.177 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.170+0000 7fbfb7fff640 1 -- 172.21.15.97:0/3532324879 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f81b4e0620 con 0x55f81b7a7800 2024-04-23T10:31:29.177 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.171+0000 7fbfb7fff640 1 --2- 172.21.15.97:0/3532324879 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f81b7a7800 0x55f81b7a49b0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0010ae0 tx=0x7fbfb0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fe19c2aea21ea41 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:31:29.177 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.171+0000 7fbf96ffd640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:31:29.177 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.171+0000 7fbf96ffd640 1 -- 172.21.15.97:0/3532324879 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb0025070 con 0x55f81b7a7800 2024-04-23T10:31:29.177 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.171+0000 7fbf96ffd640 1 -- 172.21.15.97:0/3532324879 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fbfb001edb0 con 0x55f81b7a7800 2024-04-23T10:31:29.177 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.171+0000 7fbfc0dcd740 1 -- 172.21.15.97:0/3532324879 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfac004030 con 0x55f81b7a7800 2024-04-23T10:31:29.177 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.171+0000 7fbf96ffd640 1 -- 172.21.15.97:0/3532324879 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbfb001f450 con 0x55f81b7a7800 2024-04-23T10:31:29.177 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.171+0000 7fbfc0dcd740 10 client.16305 fetch_fsmap learned FSMap version 26 2024-04-23T10:31:29.177 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.171+0000 7fbfc0dcd740 1 -- 172.21.15.97:0/3532324879 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f81b7b6320 con 0x55f81b7a7800 2024-04-23T10:31:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.171+0000 7fbf96ffd640 1 -- 172.21.15.97:0/3532324879 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfac004030 con 0x55f81b7a7800 2024-04-23T10:31:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.171+0000 7fbf96ffd640 1 -- 172.21.15.97:0/3532324879 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fbfb001c040 con 0x55f81b7a7800 2024-04-23T10:31:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.171+0000 7fbfc0dcd740 10 client.16305 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:31:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.171+0000 7fbfc0dcd740 10 client.16305 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:31:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.171+0000 7fbfc0dcd740 20 client.16305 populate_metadata read hostname 'smithi097' 2024-04-23T10:31:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.171+0000 7fbfc0dcd740 1 --2- 172.21.15.97:0/3532324879 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55f81b7b7830 0x55f81b7b9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.171+0000 7fbfc0dcd740 4 client.16305 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:31:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.171+0000 7fbfc0dcd740 1 -- 172.21.15.97:0/3532324879 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f81b70f390 con 0x55f81b7b7830 2024-04-23T10:31:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.172+0000 7fbfb77fe640 1 --2- 172.21.15.97:0/3532324879 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55f81b7b7830 0x55f81b7b9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.172+0000 7fbf96ffd640 10 client.16305 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:31:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.172+0000 7fbfb77fe640 1 --2- 172.21.15.97:0/3532324879 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55f81b7b7830 0x55f81b7b9c40 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0862005c92b680f server_cookie=8bf3fd9ccf50bfd6 in_seq=0 out_seq=0 2024-04-23T10:31:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.173+0000 7fbf96ffd640 1 -- 172.21.15.97:0/3532324879 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f81b70f390 con 0x55f81b7b7830 2024-04-23T10:31:29.178 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.173+0000 7fbf96ffd640 10 client.16305 handle_command_reply: tid=0 2024-04-23T10:31:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.173+0000 7fbfc0dcd740 1 client.16305 shutdown 2024-04-23T10:31:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.173+0000 7fbfc0dcd740 1 -- 172.21.15.97:0/3532324879 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f81b7a7800 msgr2=0x55f81b7a49b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:31:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.173+0000 7fbfc0dcd740 1 --2- 172.21.15.97:0/3532324879 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f81b7a7800 0x55f81b7a49b0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0010ae0 tx=0x7fbfb0004ab0 comp rx=0 tx=0).stop 2024-04-23T10:31:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.173+0000 7fbfc0dcd740 1 -- 172.21.15.97:0/3532324879 shutdown_connections 2024-04-23T10:31:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.173+0000 7fbfc0dcd740 1 --2- 172.21.15.97:0/3532324879 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55f81b7b7830 0x55f81b7b9c40 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.173+0000 7fbfc0dcd740 1 --2- 172.21.15.97:0/3532324879 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f81b7a7800 0x55f81b7a49b0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.173+0000 7fbfc0dcd740 1 --2- 172.21.15.97:0/3532324879 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f81b79cf40 0x55f81b7a4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.173+0000 7fbfc0dcd740 1 --2- 172.21.15.97:0/3532324879 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f81b79c620 0x55f81b7a7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.173+0000 7fbf96ffd640 0 client.16305 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:31:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.173+0000 7fbfc0dcd740 1 -- 172.21.15.97:0/3532324879 >> 172.21.15.97:0/3532324879 conn(0x55f81b798b30 msgr2=0x55f81b79e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:31:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.173+0000 7fbfc0dcd740 1 -- 172.21.15.97:0/3532324879 shutdown_connections 2024-04-23T10:31:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.173+0000 7fbfc0dcd740 1 -- 172.21.15.97:0/3532324879 wait complete. 2024-04-23T10:31:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.173+0000 7fbfc0dcd740 20 client.16305 trim_cache size 0 max 16384 2024-04-23T10:31:29.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.188+0000 7fbfc0dcd740 1 Processor -- start 2024-04-23T10:31:29.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.188+0000 7fbfc0dcd740 1 -- start start 2024-04-23T10:31:29.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.188+0000 7fbfc0dcd740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f81b79c620 0x55f81b7b3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:29.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.188+0000 7fbfc0dcd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f81b79cf40 0x55f81b797b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:29.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.188+0000 7fbfc0dcd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f81b7a7800 0x55f81b7bf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:29.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.188+0000 7fbfc0dcd740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb81ad9b0 con 0x55f81b79c620 2024-04-23T10:31:29.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.188+0000 7fbfc0dcd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb81af4f0 con 0x55f81b79cf40 2024-04-23T10:31:29.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.188+0000 7fbfc0dcd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb81ae080 con 0x55f81b7a7800 2024-04-23T10:31:29.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.188+0000 7fbfb77fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f81b79c620 0x55f81b7b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:29.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.188+0000 7fbfb77fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f81b79c620 0x55f81b7b3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58738/0 (socket says 172.21.15.97:58738) 2024-04-23T10:31:29.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.188+0000 7fbfb77fe640 1 -- 172.21.15.97:0/2923950262 learned_addr learned my addr 172.21.15.97:0/2923950262 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:31:29.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.188+0000 7fbfb7fff640 1 --2- 172.21.15.97:0/2923950262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f81b7a7800 0x55f81b7bf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:29.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.188+0000 7fbfb6ffd640 1 --2- 172.21.15.97:0/2923950262 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f81b79cf40 0x55f81b797b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:29.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.188+0000 7fbfb77fe640 1 -- 172.21.15.97:0/2923950262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f81b7a7800 msgr2=0x55f81b7bf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:31:29.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.188+0000 7fbfb77fe640 1 --2- 172.21.15.97:0/2923950262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f81b7a7800 0x55f81b7bf550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:29.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.188+0000 7fbfb77fe640 1 -- 172.21.15.97:0/2923950262 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f81b79cf40 msgr2=0x55f81b797b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:31:29.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.188+0000 7fbfb77fe640 1 --2- 172.21.15.97:0/2923950262 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f81b79cf40 0x55f81b797b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:29.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.188+0000 7fbfb77fe640 1 -- 172.21.15.97:0/2923950262 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f81b799140 con 0x55f81b79c620 2024-04-23T10:31:29.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.189+0000 7fbfb7fff640 1 --2- 172.21.15.97:0/2923950262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f81b7a7800 0x55f81b7bf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:31:29.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.189+0000 7fbfb6ffd640 1 --2- 172.21.15.97:0/2923950262 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f81b79cf40 0x55f81b797b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:31:29.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.189+0000 7fbfb77fe640 1 --2- 172.21.15.97:0/2923950262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f81b79c620 0x55f81b7b3290 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fbfac0051f0 tx=0x7fbfac031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68e731a6a13e417a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:31:29.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.189+0000 7fbf9d7fa640 1 -- 172.21.15.97:0/2923950262 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfac040040 con 0x55f81b79c620 2024-04-23T10:31:29.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.189+0000 7fbf9d7fa640 1 -- 172.21.15.97:0/2923950262 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfac005490 con 0x55f81b79c620 2024-04-23T10:31:29.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.189+0000 7fbf9d7fa640 1 -- 172.21.15.97:0/2923950262 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfac035040 con 0x55f81b79c620 2024-04-23T10:31:29.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.189+0000 7fbfc0dcd740 1 -- 172.21.15.97:0/2923950262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f81b79c620 msgr2=0x55f81b7b3290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:31:29.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.189+0000 7fbfc0dcd740 1 --2- 172.21.15.97:0/2923950262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f81b79c620 0x55f81b7b3290 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fbfac0051f0 tx=0x7fbfac031400 comp rx=0 tx=0).stop 2024-04-23T10:31:29.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.189+0000 7fbfc0dcd740 1 -- 172.21.15.97:0/2923950262 shutdown_connections 2024-04-23T10:31:29.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.189+0000 7fbfc0dcd740 1 --2- 172.21.15.97:0/2923950262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f81b7a7800 0x55f81b7bf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:29.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.189+0000 7fbfc0dcd740 1 --2- 172.21.15.97:0/2923950262 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f81b79cf40 0x55f81b797b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:29.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.189+0000 7fbfc0dcd740 1 --2- 172.21.15.97:0/2923950262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f81b79c620 0x55f81b7b3290 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:29.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.189+0000 7fbfc0dcd740 1 -- 172.21.15.97:0/2923950262 >> 172.21.15.97:0/2923950262 conn(0x55f81b798b30 msgr2=0x55f81b797420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:31:29.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.190+0000 7fbfc0dcd740 1 -- 172.21.15.97:0/2923950262 shutdown_connections 2024-04-23T10:31:29.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.190+0000 7fbfc0dcd740 1 -- 172.21.15.97:0/2923950262 wait complete. 2024-04-23T10:31:29.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.190+0000 7fbfc0dcd740 1 -- start start 2024-04-23T10:31:29.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.190+0000 7fbfc0dcd740 1 Processor -- start 2024-04-23T10:31:29.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.190+0000 7fbfc0dcd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f81b79c620 0x55f81b7af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:29.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.190+0000 7fbfc0dcd740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f81b79cf40 0x55f81b7acc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:29.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.190+0000 7fbfc0dcd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f81b7a7800 0x55f81b7ad180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:29.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.190+0000 7fbfc0dcd740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb81aee20 con 0x55f81b79cf40 2024-04-23T10:31:29.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.190+0000 7fbfc0dcd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb81ae750 con 0x55f81b79c620 2024-04-23T10:31:29.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.190+0000 7fbfc0dcd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb81ab9b0 con 0x55f81b7a7800 2024-04-23T10:31:29.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.190+0000 7fbfc0dcd740 10 client.0 _ll_register_callbacks cb 0x7fbfb0012f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:31:29.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.190+0000 7fbfb6ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f81b79cf40 0x55f81b7acc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:29.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.190+0000 7fbfb6ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f81b79cf40 0x55f81b7acc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58742/0 (socket says 172.21.15.97:58742) 2024-04-23T10:31:29.195 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.190+0000 7fbfb6ffd640 1 -- 172.21.15.97:0/1098490639 learned_addr learned my addr 172.21.15.97:0/1098490639 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:31:29.195 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.191+0000 7fbfb77fe640 1 --2- 172.21.15.97:0/1098490639 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f81b79c620 0x55f81b7af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:29.195 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.191+0000 7fbfb6ffd640 1 -- 172.21.15.97:0/1098490639 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f81b7a7800 msgr2=0x55f81b7ad180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:31:29.195 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.191+0000 7fbfb6ffd640 1 --2- 172.21.15.97:0/1098490639 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f81b7a7800 0x55f81b7ad180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:29.195 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.191+0000 7fbfb6ffd640 1 -- 172.21.15.97:0/1098490639 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f81b79c620 msgr2=0x55f81b7af8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:31:29.195 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.191+0000 7fbfb6ffd640 1 --2- 172.21.15.97:0/1098490639 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f81b79c620 0x55f81b7af8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:29.195 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.191+0000 7fbfb6ffd640 1 -- 172.21.15.97:0/1098490639 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f81b7ad6c0 con 0x55f81b79cf40 2024-04-23T10:31:29.195 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.191+0000 7fbfb77fe640 1 --2- 172.21.15.97:0/1098490639 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f81b79c620 0x55f81b7af8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:31:29.195 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.191+0000 7fbfb6ffd640 1 --2- 172.21.15.97:0/1098490639 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f81b79cf40 0x55f81b7acc40 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8098b20 tx=0x7fbfa809bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9be491bd86a884ba server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:31:29.195 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.191+0000 7fbf96ffd640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:31:29.195 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.191+0000 7fbf96ffd640 1 -- 172.21.15.97:0/1098490639 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa80b4020 con 0x55f81b79cf40 2024-04-23T10:31:29.195 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.191+0000 7fbf96ffd640 1 -- 172.21.15.97:0/1098490639 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fbfa80987c0 con 0x55f81b79cf40 2024-04-23T10:31:29.195 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.191+0000 7fbfc0dcd740 1 -- 172.21.15.97:0/1098490639 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfac005490 con 0x55f81b79cf40 2024-04-23T10:31:29.195 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.191+0000 7fbf96ffd640 1 -- 172.21.15.97:0/1098490639 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbfa807f050 con 0x55f81b79cf40 2024-04-23T10:31:29.196 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.191+0000 7fbfc0dcd740 10 client.16311 fetch_fsmap learned FSMap version 26 2024-04-23T10:31:29.196 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.191+0000 7fbfc0dcd740 1 -- 172.21.15.97:0/1098490639 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f81b81dad0 con 0x55f81b79cf40 2024-04-23T10:31:29.196 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.191+0000 7fbf96ffd640 1 -- 172.21.15.97:0/1098490639 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfac005490 con 0x55f81b79cf40 2024-04-23T10:31:29.196 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.191+0000 7fbf96ffd640 1 -- 172.21.15.97:0/1098490639 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fbfa8097ea0 con 0x55f81b79cf40 2024-04-23T10:31:29.196 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.192+0000 7fbfc0dcd740 10 client.16311 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:31:29.196 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.192+0000 7fbfc0dcd740 10 client.16311 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:31:29.196 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.192+0000 7fbfc0dcd740 20 client.16311 populate_metadata read hostname 'smithi097' 2024-04-23T10:31:29.196 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.192+0000 7fbfc0dcd740 1 --2- 172.21.15.97:0/1098490639 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55f81b81e2d0 0x55f81b8206f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:29.196 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.192+0000 7fbfc0dcd740 4 client.16311 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:31:29.196 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.192+0000 7fbfc0dcd740 1 -- 172.21.15.97:0/1098490639 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f81b709200 con 0x55f81b81e2d0 2024-04-23T10:31:29.196 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.192+0000 7fbfb77fe640 1 --2- 172.21.15.97:0/1098490639 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55f81b81e2d0 0x55f81b8206f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:29.196 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.192+0000 7fbfb77fe640 1 --2- 172.21.15.97:0/1098490639 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55f81b81e2d0 0x55f81b8206f0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=306991b08412237a server_cookie=87594ea65312045c in_seq=0 out_seq=0 2024-04-23T10:31:29.196 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.192+0000 7fbf96ffd640 10 client.16311 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:31:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.206+0000 7fbf96ffd640 1 -- 172.21.15.97:0/1098490639 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f81b709200 con 0x55f81b81e2d0 2024-04-23T10:31:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.206+0000 7fbf96ffd640 10 client.16311 handle_command_reply: tid=0 2024-04-23T10:31:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.206+0000 7fbfc0dcd740 1 client.16311 shutdown 2024-04-23T10:31:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.206+0000 7fbfc0dcd740 1 -- 172.21.15.97:0/1098490639 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f81b79cf40 msgr2=0x55f81b7acc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:31:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.206+0000 7fbfc0dcd740 1 --2- 172.21.15.97:0/1098490639 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f81b79cf40 0x55f81b7acc40 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8098b20 tx=0x7fbfa809bab0 comp rx=0 tx=0).stop 2024-04-23T10:31:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.206+0000 7fbfc0dcd740 1 -- 172.21.15.97:0/1098490639 shutdown_connections 2024-04-23T10:31:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.206+0000 7fbfc0dcd740 1 --2- 172.21.15.97:0/1098490639 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55f81b81e2d0 0x55f81b8206f0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.206+0000 7fbfc0dcd740 1 --2- 172.21.15.97:0/1098490639 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f81b7a7800 0x55f81b7ad180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.206+0000 7fbfc0dcd740 1 --2- 172.21.15.97:0/1098490639 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f81b79cf40 0x55f81b7acc40 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.206+0000 7fbf96ffd640 0 client.16311 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:31:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.206+0000 7fbfc0dcd740 1 --2- 172.21.15.97:0/1098490639 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f81b79c620 0x55f81b7af8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:29.207 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.206+0000 7fbfc0dcd740 1 -- 172.21.15.97:0/1098490639 >> 172.21.15.97:0/1098490639 conn(0x55f81b798b30 msgr2=0x55f81b797cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:31:29.207 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.207+0000 7fbfc0dcd740 1 -- 172.21.15.97:0/1098490639 shutdown_connections 2024-04-23T10:31:29.207 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.207+0000 7fbfc0dcd740 1 -- 172.21.15.97:0/1098490639 wait complete. 2024-04-23T10:31:29.207 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.207+0000 7fbfc0dcd740 20 client.16311 trim_cache size 0 max 16384 2024-04-23T10:31:29.207 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:31:29.207 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:31:29.207 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:31:29.207 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:31:29.208 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.207+0000 7fbfbdfdb640 1 -- 172.21.15.97:0/893399250 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fbfa00a3dc0 msgr2=0x7fbfa00a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:31:29.208 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.207+0000 7fbfbdfdb640 1 --2- 172.21.15.97:0/893399250 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fbfa00a3dc0 0x7fbfa00a6280 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0009fd0 tx=0x7fbfb0009ad0 comp rx=0 tx=0).stop 2024-04-23T10:31:29.208 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.207+0000 7fbfbdfdb640 1 -- 172.21.15.97:0/893399250 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbfb8107290 msgr2=0x7fbfb8198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:31:29.208 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.207+0000 7fbfbdfdb640 1 --2- 172.21.15.97:0/893399250 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbfb8107290 0x7fbfb8198c70 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fbfa800d740 tx=0x7fbfa800dc00 comp rx=0 tx=0).stop 2024-04-23T10:31:29.208 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.207+0000 7fbfbdfdb640 1 -- 172.21.15.97:0/893399250 shutdown_connections 2024-04-23T10:31:29.208 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.207+0000 7fbfbdfdb640 1 --2- 172.21.15.97:0/893399250 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fbfa00a3dc0 0x7fbfa00a6280 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:29.209 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.207+0000 7fbfbdfdb640 1 --2- 172.21.15.97:0/893399250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbfb819cfa0 0x7fbfb819d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:29.209 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.207+0000 7fbfbdfdb640 1 --2- 172.21.15.97:0/893399250 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbfb8107290 0x7fbfb8198c70 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:29.209 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.207+0000 7fbfbdfdb640 1 --2- 172.21.15.97:0/893399250 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbfb81021c0 0x7fbfb8198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:29.209 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.207+0000 7fbfbdfdb640 1 -- 172.21.15.97:0/893399250 >> 172.21.15.97:0/893399250 conn(0x7fbfb80fda20 msgr2=0x7fbfb80fee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:31:29.209 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.207+0000 7fbfbdfdb640 1 -- 172.21.15.97:0/893399250 shutdown_connections 2024-04-23T10:31:29.209 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:29.207+0000 7fbfbdfdb640 1 -- 172.21.15.97:0/893399250 wait complete. 2024-04-23T10:31:29.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:31:28 smithi196 ceph-mon[32919]: pgmap v161: 97 pgs: 97 active+clean; 2.0 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 51 KiB/s wr, 22 op/s 2024-04-23T10:31:29.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:31:28 smithi097 ceph-mon[29197]: pgmap v161: 97 pgs: 97 active+clean; 2.0 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 51 KiB/s wr, 22 op/s 2024-04-23T10:31:29.399 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:31:28 smithi183 ceph-mon[32317]: pgmap v161: 97 pgs: 97 active+clean; 2.0 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 51 KiB/s wr, 22 op/s 2024-04-23T10:31:31.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:31:30 smithi196 ceph-mon[32919]: pgmap v162: 97 pgs: 97 active+clean; 2.0 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-04-23T10:31:31.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:31:30 smithi097 ceph-mon[29197]: pgmap v162: 97 pgs: 97 active+clean; 2.0 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-04-23T10:31:31.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:31:30 smithi183 ceph-mon[32317]: pgmap v162: 97 pgs: 97 active+clean; 2.0 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-04-23T10:31:33.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:31:32 smithi196 ceph-mon[32919]: pgmap v163: 97 pgs: 97 active+clean; 2.0 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-04-23T10:31:33.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:31:32 smithi097 ceph-mon[29197]: pgmap v163: 97 pgs: 97 active+clean; 2.0 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-04-23T10:31:33.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:31:32 smithi183 ceph-mon[32317]: pgmap v163: 97 pgs: 97 active+clean; 2.0 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-04-23T10:31:33.989 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.989 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:31:33.989 INFO:tasks.workunit.client.0.smithi097.stdout:127/600: open fd = 3 2024-04-23T10:31:33.989 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.989 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.989 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:31:33.989 INFO:tasks.workunit.client.0.smithi097.stdout:128/600: open fd = 3 2024-04-23T10:31:33.989 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.989 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.989 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:31:33.989 INFO:tasks.workunit.client.0.smithi097.stdout:129/600: open fd = 3 2024-04-23T10:31:33.989 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.990 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.990 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:31:33.990 INFO:tasks.workunit.client.0.smithi097.stdout:130/600: open fd = 3 2024-04-23T10:31:33.990 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.990 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.990 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:31:33.990 INFO:tasks.workunit.client.0.smithi097.stdout:131/600: open fd = 3 2024-04-23T10:31:33.990 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.990 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.990 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:31:33.990 INFO:tasks.workunit.client.0.smithi097.stdout:132/600: open fd = 3 2024-04-23T10:31:33.990 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.990 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.990 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:31:33.991 INFO:tasks.workunit.client.0.smithi097.stdout:133/600: open fd = 3 2024-04-23T10:31:33.991 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.991 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.991 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:31:33.991 INFO:tasks.workunit.client.0.smithi097.stdout:134/600: open fd = 3 2024-04-23T10:31:33.991 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.991 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.991 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:31:33.991 INFO:tasks.workunit.client.0.smithi097.stdout:135/600: open fd = 3 2024-04-23T10:31:33.991 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.991 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.991 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:31:33.991 INFO:tasks.workunit.client.0.smithi097.stdout:136/600: open fd = 3 2024-04-23T10:31:33.991 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.991 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.992 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:31:33.992 INFO:tasks.workunit.client.0.smithi097.stdout:137/600: open fd = 3 2024-04-23T10:31:33.992 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.992 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.992 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:31:33.992 INFO:tasks.workunit.client.0.smithi097.stdout:138/600: open fd = 3 2024-04-23T10:31:33.992 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.992 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.992 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:31:33.992 INFO:tasks.workunit.client.0.smithi097.stdout:139/600: open fd = 3 2024-04-23T10:31:33.992 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.992 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.992 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:31:33.992 INFO:tasks.workunit.client.0.smithi097.stdout:140/600: open fd = 3 2024-04-23T10:31:33.992 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.993 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.993 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:31:33.993 INFO:tasks.workunit.client.0.smithi097.stdout:141/600: open fd = 3 2024-04-23T10:31:33.993 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.993 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.993 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:31:33.993 INFO:tasks.workunit.client.0.smithi097.stdout:142/600: open fd = 3 2024-04-23T10:31:33.993 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.993 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.993 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:31:33.993 INFO:tasks.workunit.client.0.smithi097.stdout:143/600: open fd = 3 2024-04-23T10:31:33.993 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.993 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.993 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:31:33.993 INFO:tasks.workunit.client.0.smithi097.stdout:144/600: open fd = 3 2024-04-23T10:31:33.993 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.994 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.994 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:31:33.994 INFO:tasks.workunit.client.0.smithi097.stdout:145/600: open fd = 3 2024-04-23T10:31:33.994 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.994 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.994 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:31:33.994 INFO:tasks.workunit.client.0.smithi097.stdout:146/600: open fd = 3 2024-04-23T10:31:33.994 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.994 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.994 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:31:33.994 INFO:tasks.workunit.client.0.smithi097.stdout:147/600: open fd = 3 2024-04-23T10:31:33.994 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.994 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.994 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:31:33.994 INFO:tasks.workunit.client.0.smithi097.stdout:148/600: open fd = 3 2024-04-23T10:31:33.994 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.995 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.995 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:31:33.995 INFO:tasks.workunit.client.0.smithi097.stdout:149/600: open fd = 3 2024-04-23T10:31:33.995 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.995 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.995 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:31:33.995 INFO:tasks.workunit.client.0.smithi097.stdout:150/600: open fd = 3 2024-04-23T10:31:33.995 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.995 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.995 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:31:33.995 INFO:tasks.workunit.client.0.smithi097.stdout:151/600: open fd = 3 2024-04-23T10:31:33.995 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.995 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.995 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:31:33.995 INFO:tasks.workunit.client.0.smithi097.stdout:152/600: open fd = 3 2024-04-23T10:31:33.995 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.996 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.996 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:31:33.996 INFO:tasks.workunit.client.0.smithi097.stdout:153/600: open fd = 3 2024-04-23T10:31:33.996 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.996 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.996 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:31:33.996 INFO:tasks.workunit.client.0.smithi097.stdout:154/600: open fd = 3 2024-04-23T10:31:33.996 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.996 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.996 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:31:33.996 INFO:tasks.workunit.client.0.smithi097.stdout:155/600: open fd = 3 2024-04-23T10:31:33.996 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.996 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.996 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:31:33.996 INFO:tasks.workunit.client.0.smithi097.stdout:156/600: open fd = 3 2024-04-23T10:31:33.996 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.997 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.997 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:31:33.997 INFO:tasks.workunit.client.0.smithi097.stdout:157/600: open fd = 3 2024-04-23T10:31:33.997 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.997 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.997 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:31:33.997 INFO:tasks.workunit.client.0.smithi097.stdout:158/600: open fd = 3 2024-04-23T10:31:33.997 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.997 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.997 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:31:33.997 INFO:tasks.workunit.client.0.smithi097.stdout:159/600: open fd = 3 2024-04-23T10:31:33.997 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.997 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.997 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:31:33.997 INFO:tasks.workunit.client.0.smithi097.stdout:160/600: open fd = 3 2024-04-23T10:31:33.998 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.998 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.998 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:31:33.998 INFO:tasks.workunit.client.0.smithi097.stdout:161/600: open fd = 3 2024-04-23T10:31:33.998 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.998 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.998 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:31:33.998 INFO:tasks.workunit.client.0.smithi097.stdout:162/600: open fd = 3 2024-04-23T10:31:33.998 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.998 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.998 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:31:33.998 INFO:tasks.workunit.client.0.smithi097.stdout:163/600: open fd = 3 2024-04-23T10:31:33.998 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.998 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.998 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:31:33.999 INFO:tasks.workunit.client.0.smithi097.stdout:164/600: open fd = 3 2024-04-23T10:31:33.999 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.999 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.999 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:31:33.999 INFO:tasks.workunit.client.0.smithi097.stdout:165/600: open fd = 3 2024-04-23T10:31:33.999 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.999 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.999 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:31:33.999 INFO:tasks.workunit.client.0.smithi097.stdout:166/600: open fd = 3 2024-04-23T10:31:33.999 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.999 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:33.999 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:31:33.999 INFO:tasks.workunit.client.0.smithi097.stdout:167/600: open fd = 3 2024-04-23T10:31:33.999 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:34.000 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:34.000 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:31:34.000 INFO:tasks.workunit.client.0.smithi097.stdout:168/600: open fd = 3 2024-04-23T10:31:34.000 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:34.000 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:34.000 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:31:34.000 INFO:tasks.workunit.client.0.smithi097.stdout:169/600: open fd = 3 2024-04-23T10:31:34.000 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:34.000 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:34.000 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:31:34.000 INFO:tasks.workunit.client.0.smithi097.stdout:170/600: open fd = 3 2024-04-23T10:31:34.000 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:34.000 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:34.000 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:31:34.000 INFO:tasks.workunit.client.0.smithi097.stdout:171/600: open fd = 3 2024-04-23T10:31:34.000 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:34.001 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:34.001 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:31:34.001 INFO:tasks.workunit.client.0.smithi097.stdout:172/600: open fd = 3 2024-04-23T10:31:34.001 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:34.001 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:34.001 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:31:34.001 INFO:tasks.workunit.client.0.smithi097.stdout:173/600: open fd = 3 2024-04-23T10:31:34.001 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:34.001 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:34.001 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:31:34.001 INFO:tasks.workunit.client.0.smithi097.stdout:174/600: open fd = 3 2024-04-23T10:31:34.001 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:34.001 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:34.001 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:31:34.002 INFO:tasks.workunit.client.0.smithi097.stdout:175/600: open fd = 3 2024-04-23T10:31:34.002 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:34.002 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:34.002 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:31:34.002 INFO:tasks.workunit.client.0.smithi097.stdout:176/600: open fd = 3 2024-04-23T10:31:34.002 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:34.002 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:34.002 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:31:34.002 INFO:tasks.workunit.client.0.smithi097.stdout:177/600: open fd = 3 2024-04-23T10:31:34.002 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:34.002 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:34.002 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:31:34.002 INFO:tasks.workunit.client.0.smithi097.stdout:178/600: open fd = 3 2024-04-23T10:31:34.002 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:34.003 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:34.003 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:31:34.003 INFO:tasks.workunit.client.0.smithi097.stdout:179/600: open fd = 3 2024-04-23T10:31:34.003 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:34.003 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:34.003 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:31:34.003 INFO:tasks.workunit.client.0.smithi097.stdout:180/600: open fd = 3 2024-04-23T10:31:34.003 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:34.003 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:34.003 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:31:34.003 INFO:tasks.workunit.client.0.smithi097.stdout:181/600: open fd = 3 2024-04-23T10:31:34.003 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:34.003 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:34.003 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:31:34.003 INFO:tasks.workunit.client.0.smithi097.stdout:182/600: open fd = 3 2024-04-23T10:31:34.004 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:34.004 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:34.004 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:31:34.004 INFO:tasks.workunit.client.0.smithi097.stdout:183/600: open fd = 3 2024-04-23T10:31:34.004 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:34.004 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:34.004 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:31:34.004 INFO:tasks.workunit.client.0.smithi097.stdout:184/600: open fd = 3 2024-04-23T10:31:34.004 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:34.004 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:34.004 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:31:34.004 INFO:tasks.workunit.client.0.smithi097.stdout:185/600: open fd = 3 2024-04-23T10:31:34.004 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:34.004 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:34.004 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:31:34.005 INFO:tasks.workunit.client.0.smithi097.stdout:186/600: open fd = 3 2024-04-23T10:31:34.005 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:34.005 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:34.005 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:31:34.005 INFO:tasks.workunit.client.0.smithi097.stdout:187/600: open fd = 3 2024-04-23T10:31:34.005 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:34.005 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:34.005 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:31:34.005 INFO:tasks.workunit.client.0.smithi097.stdout:188/600: open fd = 3 2024-04-23T10:31:34.005 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:34.005 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:31:35.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:31:34 smithi196 ceph-mon[32919]: pgmap v164: 97 pgs: 97 active+clean; 2.2 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 51 KiB/s wr, 22 op/s 2024-04-23T10:31:35.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:31:34 smithi097 ceph-mon[29197]: pgmap v164: 97 pgs: 97 active+clean; 2.2 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 51 KiB/s wr, 22 op/s 2024-04-23T10:31:35.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:31:34 smithi183 ceph-mon[32317]: pgmap v164: 97 pgs: 97 active+clean; 2.2 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 51 KiB/s wr, 22 op/s 2024-04-23T10:31:37.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:31:36 smithi196 ceph-mon[32919]: pgmap v165: 97 pgs: 97 active+clean; 2.2 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 37 KiB/s wr, 15 op/s 2024-04-23T10:31:37.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:31:36 smithi097 ceph-mon[29197]: pgmap v165: 97 pgs: 97 active+clean; 2.2 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 37 KiB/s wr, 15 op/s 2024-04-23T10:31:37.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:31:36 smithi183 ceph-mon[32317]: pgmap v165: 97 pgs: 97 active+clean; 2.2 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 37 KiB/s wr, 15 op/s 2024-04-23T10:31:39.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:31:38 smithi196 ceph-mon[32919]: pgmap v166: 97 pgs: 97 active+clean; 2.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-23T10:31:39.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:31:38 smithi097 ceph-mon[29197]: pgmap v166: 97 pgs: 97 active+clean; 2.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-23T10:31:39.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:31:38 smithi183 ceph-mon[32317]: pgmap v166: 97 pgs: 97 active+clean; 2.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-23T10:31:41.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:31:40 smithi196 ceph-mon[32919]: pgmap v167: 97 pgs: 97 active+clean; 2.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-23T10:31:41.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:31:40 smithi097 ceph-mon[29197]: pgmap v167: 97 pgs: 97 active+clean; 2.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-23T10:31:41.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:31:40 smithi183 ceph-mon[32317]: pgmap v167: 97 pgs: 97 active+clean; 2.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-23T10:31:42.205 INFO:tasks.quiescer.fs.[cephfs]:Total duration (38.9) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-04-23T10:31:42.205 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.45 2024-04-23T10:31:42.206 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T10:31:42.306 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.305+0000 7f3c585a1640 1 -- 172.21.15.97:0/3974859838 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c50106170 msgr2=0x7f3c5010dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:31:42.306 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.305+0000 7f3c585a1640 1 --2- 172.21.15.97:0/3974859838 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c50106170 0x7f3c5010dd40 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f3c48009ca0 tx=0x7f3c4802a8c0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.306 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.305+0000 7f3c585a1640 1 -- 172.21.15.97:0/3974859838 shutdown_connections 2024-04-23T10:31:42.306 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.305+0000 7f3c585a1640 1 --2- 172.21.15.97:0/3974859838 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c50106170 0x7f3c5010dd40 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.306 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.305+0000 7f3c585a1640 1 --2- 172.21.15.97:0/3974859838 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c50101aa0 0x7f3c50105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.306 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.305+0000 7f3c585a1640 1 --2- 172.21.15.97:0/3974859838 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c501010f0 0x7f3c501014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.306 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.305+0000 7f3c585a1640 1 -- 172.21.15.97:0/3974859838 >> 172.21.15.97:0/3974859838 conn(0x7f3c50077fd0 msgr2=0x7f3c500783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:31:42.306 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.305+0000 7f3c585a1640 1 -- 172.21.15.97:0/3974859838 shutdown_connections 2024-04-23T10:31:42.306 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.305+0000 7f3c585a1640 1 -- 172.21.15.97:0/3974859838 wait complete. 2024-04-23T10:31:42.307 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.306+0000 7f3c585a1640 1 Processor -- start 2024-04-23T10:31:42.307 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.306+0000 7f3c585a1640 1 -- start start 2024-04-23T10:31:42.307 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.306+0000 7f3c585a1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c501010f0 0x7f3c50102b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:42.307 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.306+0000 7f3c585a1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c50106170 0x7f3c50103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:42.307 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.306+0000 7f3c585a1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c501035b0 0x7f3c501a6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:42.307 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.306+0000 7f3c585a1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3c501110e0 con 0x7f3c501035b0 2024-04-23T10:31:42.307 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.306+0000 7f3c585a1640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3c50110f60 con 0x7f3c501010f0 2024-04-23T10:31:42.307 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.306+0000 7f3c585a1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3c50111260 con 0x7f3c50106170 2024-04-23T10:31:42.307 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.306+0000 7f3c56b3e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c501035b0 0x7f3c501a6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:42.307 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.306+0000 7f3c56b3e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c501035b0 0x7f3c501a6890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51820/0 (socket says 172.21.15.97:51820) 2024-04-23T10:31:42.307 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.306+0000 7f3c56b3e640 1 -- 172.21.15.97:0/3231584318 learned_addr learned my addr 172.21.15.97:0/3231584318 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:31:42.307 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.306+0000 7f3c5633d640 1 --2- 172.21.15.97:0/3231584318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c501010f0 0x7f3c50102b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:42.307 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.306+0000 7f3c55b3c640 1 --2- 172.21.15.97:0/3231584318 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c50106170 0x7f3c50103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:42.307 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.306+0000 7f3c56b3e640 1 -- 172.21.15.97:0/3231584318 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c50106170 msgr2=0x7f3c50103070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:31:42.307 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.306+0000 7f3c56b3e640 1 --2- 172.21.15.97:0/3231584318 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c50106170 0x7f3c50103070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.307 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.306+0000 7f3c56b3e640 1 -- 172.21.15.97:0/3231584318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c501010f0 msgr2=0x7f3c50102b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:31:42.308 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.306+0000 7f3c56b3e640 1 --2- 172.21.15.97:0/3231584318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c501010f0 0x7f3c50102b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.308 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.306+0000 7f3c56b3e640 1 -- 172.21.15.97:0/3231584318 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c501a6f30 con 0x7f3c501035b0 2024-04-23T10:31:42.308 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.306+0000 7f3c5633d640 1 --2- 172.21.15.97:0/3231584318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c501010f0 0x7f3c50102b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:31:42.308 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.307+0000 7f3c56b3e640 1 --2- 172.21.15.97:0/3231584318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c501035b0 0x7f3c501a6890 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f3c480098d0 tx=0x7f3c480317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:31:42.308 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.307+0000 7f3c477fe640 1 -- 172.21.15.97:0/3231584318 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c48003b70 con 0x7f3c501035b0 2024-04-23T10:31:42.308 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.307+0000 7f3c585a1640 1 -- 172.21.15.97:0/3231584318 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c501a71c0 con 0x7f3c501035b0 2024-04-23T10:31:42.308 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.307+0000 7f3c477fe640 1 -- 172.21.15.97:0/3231584318 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c48003d10 con 0x7f3c501035b0 2024-04-23T10:31:42.308 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.307+0000 7f3c477fe640 1 -- 172.21.15.97:0/3231584318 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c4802fa20 con 0x7f3c501035b0 2024-04-23T10:31:42.308 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.307+0000 7f3c585a1640 1 -- 172.21.15.97:0/3231584318 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c501a7760 con 0x7f3c501035b0 2024-04-23T10:31:42.311 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.309+0000 7f3c477fe640 1 -- 172.21.15.97:0/3231584318 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3c4802fc40 con 0x7f3c501035b0 2024-04-23T10:31:42.311 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.310+0000 7f3c5b38a740 1 Processor -- start 2024-04-23T10:31:42.311 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.310+0000 7f3c5b38a740 1 -- start start 2024-04-23T10:31:42.311 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.310+0000 7f3c5b38a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56240266a620 0x56240266aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:42.311 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.310+0000 7f3c5b38a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56240266af40 0x5624026752c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.310+0000 7f3c5b38a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562402675800 0x562402677bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.310+0000 7f3c5b38a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5624025cf970 con 0x56240266a620 2024-04-23T10:31:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.310+0000 7f3c5b38a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56240256c4a0 con 0x562402675800 2024-04-23T10:31:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.310+0000 7f3c5b38a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5624025cb930 con 0x56240266af40 2024-04-23T10:31:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.310+0000 7f3c477fe640 1 --2- 172.21.15.97:0/3231584318 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f3c3c0a3dc0 0x7f3c3c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.311+0000 7f3c55b3c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56240266a620 0x56240266aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.311+0000 7f3c55b3c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56240266a620 0x56240266aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51834/0 (socket says 172.21.15.97:51834) 2024-04-23T10:31:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.311+0000 7f3c55b3c640 1 -- 172.21.15.97:0/1830268082 learned_addr learned my addr 172.21.15.97:0/1830268082 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:31:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.311+0000 7f3c56b3e640 1 --2- 172.21.15.97:0/3231584318 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f3c3c0a3dc0 0x7f3c3c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.311+0000 7f3c5633d640 1 --2- 172.21.15.97:0/1830268082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56240266af40 0x5624026752c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.311+0000 7f3c55b3c640 1 --2- 172.21.15.97:0/1830268082 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562402675800 0x562402677bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.311+0000 7f3c477fe640 1 -- 172.21.15.97:0/3231584318 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f3c480f1ca0 con 0x7f3c501035b0 2024-04-23T10:31:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.311+0000 7f3c55b3c640 1 -- 172.21.15.97:0/1830268082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56240266af40 msgr2=0x5624026752c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:31:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.311+0000 7f3c55b3c640 1 --2- 172.21.15.97:0/1830268082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56240266af40 0x5624026752c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.311+0000 7f3c55b3c640 1 -- 172.21.15.97:0/1830268082 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562402675800 msgr2=0x562402677bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:31:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.311+0000 7f3c55b3c640 1 --2- 172.21.15.97:0/1830268082 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562402675800 0x562402677bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.311+0000 7f3c55b3c640 1 -- 172.21.15.97:0/1830268082 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624022e2210 con 0x56240266a620 2024-04-23T10:31:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.311+0000 7f3c55b3c640 1 --2- 172.21.15.97:0/1830268082 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56240266a620 0x56240266aa00 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f3c40013480 tx=0x7f3c400096e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c96f47472ba74ea2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:31:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.311+0000 7f3c56b3e640 1 --2- 172.21.15.97:0/3231584318 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f3c3c0a3dc0 0x7f3c3c0a6280 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3c480ba1a0 tx=0x7f3c4802a8c0 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:31:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.311+0000 7f3c44bfe640 1 -- 172.21.15.97:0/1830268082 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c40011470 con 0x56240266a620 2024-04-23T10:31:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.311+0000 7f3c44bfe640 1 -- 172.21.15.97:0/1830268082 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c40011ac0 con 0x56240266a620 2024-04-23T10:31:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.311+0000 7f3c44bfe640 1 -- 172.21.15.97:0/1830268082 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c4000f7e0 con 0x56240266a620 2024-04-23T10:31:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.312+0000 7f3c5b38a740 1 -- 172.21.15.97:0/1830268082 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56240266a620 msgr2=0x56240266aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:31:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.312+0000 7f3c5b38a740 1 --2- 172.21.15.97:0/1830268082 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56240266a620 0x56240266aa00 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f3c40013480 tx=0x7f3c400096e0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.312+0000 7f3c5b38a740 1 -- 172.21.15.97:0/1830268082 shutdown_connections 2024-04-23T10:31:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.312+0000 7f3c5b38a740 1 --2- 172.21.15.97:0/1830268082 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562402675800 0x562402677bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.312+0000 7f3c5b38a740 1 --2- 172.21.15.97:0/1830268082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56240266af40 0x5624026752c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.312+0000 7f3c5b38a740 1 --2- 172.21.15.97:0/1830268082 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56240266a620 0x56240266aa00 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.312+0000 7f3c5b38a740 1 -- 172.21.15.97:0/1830268082 >> 172.21.15.97:0/1830268082 conn(0x562402666b30 msgr2=0x562402666f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:31:42.313 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.312+0000 7f3c5b38a740 1 -- 172.21.15.97:0/1830268082 shutdown_connections 2024-04-23T10:31:42.314 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.312+0000 7f3c5b38a740 1 -- 172.21.15.97:0/1830268082 wait complete. 2024-04-23T10:31:42.314 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.313+0000 7f3c5b38a740 1 -- start start 2024-04-23T10:31:42.314 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.313+0000 7f3c5b38a740 1 Processor -- start 2024-04-23T10:31:42.317 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.313+0000 7f3c5b38a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56240266a620 0x5624026750c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:42.317 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.313+0000 7f3c5b38a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56240266af40 0x5624026723b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:42.317 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.313+0000 7f3c5b38a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562402675800 0x5624026728f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:42.317 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.313+0000 7f3c5b38a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x562402678430 con 0x56240266af40 2024-04-23T10:31:42.317 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.313+0000 7f3c5b38a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56240256a0b0 con 0x562402675800 2024-04-23T10:31:42.317 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.313+0000 7f3c5b38a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5624026785b0 con 0x56240266a620 2024-04-23T10:31:42.317 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.313+0000 7f3c5b38a740 10 client.0 _ll_register_callbacks cb 0x562402557d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:31:42.318 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.313+0000 7f3c5633d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56240266af40 0x5624026723b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:42.318 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.313+0000 7f3c5633d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56240266af40 0x5624026723b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51838/0 (socket says 172.21.15.97:51838) 2024-04-23T10:31:42.318 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.313+0000 7f3c5633d640 1 -- 172.21.15.97:0/1006773969 learned_addr learned my addr 172.21.15.97:0/1006773969 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:31:42.318 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.313+0000 7f3c55b3c640 1 --2- 172.21.15.97:0/1006773969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56240266a620 0x5624026750c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:42.318 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.313+0000 7f3c55b3c640 1 --2- 172.21.15.97:0/1006773969 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562402675800 0x5624026728f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:42.318 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.313+0000 7f3c5633d640 1 -- 172.21.15.97:0/1006773969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56240266a620 msgr2=0x5624026750c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:31:42.318 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.313+0000 7f3c5633d640 1 --2- 172.21.15.97:0/1006773969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56240266a620 0x5624026750c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.318 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.313+0000 7f3c5633d640 1 -- 172.21.15.97:0/1006773969 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562402675800 msgr2=0x5624026728f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:31:42.318 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.313+0000 7f3c5633d640 1 --2- 172.21.15.97:0/1006773969 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562402675800 0x5624026728f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.318 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.313+0000 7f3c5633d640 1 -- 172.21.15.97:0/1006773969 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624023ae620 con 0x56240266af40 2024-04-23T10:31:42.318 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.313+0000 7f3c55b3c640 1 --2- 172.21.15.97:0/1006773969 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562402675800 0x5624026728f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:31:42.318 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.314+0000 7f3c5633d640 1 --2- 172.21.15.97:0/1006773969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56240266af40 0x5624026723b0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c00bb40 tx=0x7f3c4c0044b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9211fcff6de73ce server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:31:42.318 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.314+0000 7f3c39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:31:42.318 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.314+0000 7f3c39ffb640 1 -- 172.21.15.97:0/1006773969 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c005220 con 0x56240266af40 2024-04-23T10:31:42.318 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.314+0000 7f3c39ffb640 1 -- 172.21.15.97:0/1006773969 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f3c4c0053c0 con 0x56240266af40 2024-04-23T10:31:42.319 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.314+0000 7f3c5b38a740 1 -- 172.21.15.97:0/1006773969 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c40011ac0 con 0x56240266af40 2024-04-23T10:31:42.319 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.314+0000 7f3c39ffb640 1 -- 172.21.15.97:0/1006773969 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3c4c0055c0 con 0x56240266af40 2024-04-23T10:31:42.319 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.314+0000 7f3c5b38a740 10 client.16323 fetch_fsmap learned FSMap version 26 2024-04-23T10:31:42.319 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.314+0000 7f3c5b38a740 1 -- 172.21.15.97:0/1006773969 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562402684320 con 0x56240266af40 2024-04-23T10:31:42.319 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.314+0000 7f3c39ffb640 1 -- 172.21.15.97:0/1006773969 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c40011ac0 con 0x56240266af40 2024-04-23T10:31:42.319 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.314+0000 7f3c39ffb640 1 -- 172.21.15.97:0/1006773969 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3c4c019030 con 0x56240266af40 2024-04-23T10:31:42.319 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.314+0000 7f3c5b38a740 10 client.16323 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:31:42.319 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.315+0000 7f3c5b38a740 10 client.16323 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:31:42.319 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.315+0000 7f3c5b38a740 20 client.16323 populate_metadata read hostname 'smithi097' 2024-04-23T10:31:42.319 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.315+0000 7f3c5b38a740 1 --2- 172.21.15.97:0/1006773969 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5624026857e0 0x562402687bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:42.319 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.315+0000 7f3c5b38a740 4 client.16323 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:31:42.319 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.315+0000 7f3c5b38a740 1 -- 172.21.15.97:0/1006773969 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5624025dd390 con 0x5624026857e0 2024-04-23T10:31:42.319 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.315+0000 7f3c55b3c640 1 --2- 172.21.15.97:0/1006773969 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5624026857e0 0x562402687bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:42.319 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.315+0000 7f3c39ffb640 10 client.16323 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:31:42.319 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.315+0000 7f3c55b3c640 1 --2- 172.21.15.97:0/1006773969 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5624026857e0 0x562402687bd0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12c3020036652a8b server_cookie=6cfd431e1a8f78b6 in_seq=0 out_seq=0 2024-04-23T10:31:42.319 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.316+0000 7f3c39ffb640 1 -- 172.21.15.97:0/1006773969 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5624025dd390 con 0x5624026857e0 2024-04-23T10:31:42.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.316+0000 7f3c39ffb640 10 client.16323 handle_command_reply: tid=0 2024-04-23T10:31:42.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.316+0000 7f3c5b38a740 1 client.16323 shutdown 2024-04-23T10:31:42.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.316+0000 7f3c5b38a740 1 -- 172.21.15.97:0/1006773969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56240266af40 msgr2=0x5624026723b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:31:42.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.316+0000 7f3c5b38a740 1 --2- 172.21.15.97:0/1006773969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56240266af40 0x5624026723b0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c00bb40 tx=0x7f3c4c0044b0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.316+0000 7f3c5b38a740 1 -- 172.21.15.97:0/1006773969 shutdown_connections 2024-04-23T10:31:42.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.316+0000 7f3c5b38a740 1 --2- 172.21.15.97:0/1006773969 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5624026857e0 0x562402687bd0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.316+0000 7f3c39ffb640 0 client.16323 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:31:42.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.316+0000 7f3c5b38a740 1 --2- 172.21.15.97:0/1006773969 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562402675800 0x5624026728f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.316+0000 7f3c5b38a740 1 --2- 172.21.15.97:0/1006773969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56240266af40 0x5624026723b0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.316+0000 7f3c5b38a740 1 --2- 172.21.15.97:0/1006773969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56240266a620 0x5624026750c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.316+0000 7f3c5b38a740 1 -- 172.21.15.97:0/1006773969 >> 172.21.15.97:0/1006773969 conn(0x562402666b30 msgr2=0x56240266c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:31:42.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.316+0000 7f3c5b38a740 1 -- 172.21.15.97:0/1006773969 shutdown_connections 2024-04-23T10:31:42.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.316+0000 7f3c5b38a740 1 -- 172.21.15.97:0/1006773969 wait complete. 2024-04-23T10:31:42.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.316+0000 7f3c5b38a740 20 client.16323 trim_cache size 0 max 16384 2024-04-23T10:31:42.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.323+0000 7fa9aaecf640 1 -- 172.21.15.97:0/1442394321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9a4102b40 msgr2=0x7fa9a4106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:31:42.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.323+0000 7fa9aaecf640 1 --2- 172.21.15.97:0/1442394321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9a4102b40 0x7fa9a4106b80 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fa990009900 tx=0x7fa99002a410 comp rx=0 tx=0).stop 2024-04-23T10:31:42.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.323+0000 7fa9aaecf640 1 -- 172.21.15.97:0/1442394321 shutdown_connections 2024-04-23T10:31:42.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.323+0000 7fa9aaecf640 1 --2- 172.21.15.97:0/1442394321 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa9a4107260 0x7fa9a410ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.323+0000 7fa9aaecf640 1 --2- 172.21.15.97:0/1442394321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9a4102b40 0x7fa9a4106b80 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.323+0000 7fa9aaecf640 1 --2- 172.21.15.97:0/1442394321 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9a4102190 0x7fa9a4102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.323+0000 7fa9aaecf640 1 -- 172.21.15.97:0/1442394321 >> 172.21.15.97:0/1442394321 conn(0x7fa9a4079310 msgr2=0x7fa9a4079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:31:42.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.323+0000 7fa9aaecf640 1 -- 172.21.15.97:0/1442394321 shutdown_connections 2024-04-23T10:31:42.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.323+0000 7fa9aaecf640 1 -- 172.21.15.97:0/1442394321 wait complete. 2024-04-23T10:31:42.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.324+0000 7fa9aaecf640 1 Processor -- start 2024-04-23T10:31:42.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.324+0000 7fa9aaecf640 1 -- start start 2024-04-23T10:31:42.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.324+0000 7fa9aaecf640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9a4102190 0x7fa9a4074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:42.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.324+0000 7fa9aaecf640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9a4107260 0x7fa9a4074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:42.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.324+0000 7fa9aaecf640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa9a4074cd0 0x7fa9a41a6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:42.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.324+0000 7fa9aaecf640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a41121a0 con 0x7fa9a4107260 2024-04-23T10:31:42.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.324+0000 7fa9aaecf640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a4112020 con 0x7fa9a4074cd0 2024-04-23T10:31:42.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.324+0000 7fa9aaecf640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a4112320 con 0x7fa9a4102190 2024-04-23T10:31:42.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.324+0000 7fa9a3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9a4107260 0x7fa9a4074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:42.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.324+0000 7fa9a3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9a4107260 0x7fa9a4074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51852/0 (socket says 172.21.15.97:51852) 2024-04-23T10:31:42.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.324+0000 7fa9a3fff640 1 -- 172.21.15.97:0/729785032 learned_addr learned my addr 172.21.15.97:0/729785032 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:31:42.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.324+0000 7fa9a8c6b640 1 --2- 172.21.15.97:0/729785032 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9a4102190 0x7fa9a4074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:42.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.324+0000 7fa9a946c640 1 --2- 172.21.15.97:0/729785032 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa9a4074cd0 0x7fa9a41a6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:42.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.324+0000 7fa9a3fff640 1 -- 172.21.15.97:0/729785032 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9a4102190 msgr2=0x7fa9a4074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:31:42.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.324+0000 7fa9a3fff640 1 --2- 172.21.15.97:0/729785032 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9a4102190 0x7fa9a4074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.324+0000 7fa9a3fff640 1 -- 172.21.15.97:0/729785032 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa9a4074cd0 msgr2=0x7fa9a41a6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:31:42.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.324+0000 7fa9a3fff640 1 --2- 172.21.15.97:0/729785032 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa9a4074cd0 0x7fa9a41a6930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.324+0000 7fa9a3fff640 1 -- 172.21.15.97:0/729785032 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9a41a7030 con 0x7fa9a4107260 2024-04-23T10:31:42.325 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.324+0000 7fa9a3fff640 1 --2- 172.21.15.97:0/729785032 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9a4107260 0x7fa9a4074790 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fa990006010 tx=0x7fa990004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:31:42.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.325+0000 7fa9a1ffb640 1 -- 172.21.15.97:0/729785032 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa99003f030 con 0x7fa9a4107260 2024-04-23T10:31:42.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.325+0000 7fa9a1ffb640 1 -- 172.21.15.97:0/729785032 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa990003aa0 con 0x7fa9a4107260 2024-04-23T10:31:42.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.325+0000 7fa9aaecf640 1 -- 172.21.15.97:0/729785032 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9a41a72c0 con 0x7fa9a4107260 2024-04-23T10:31:42.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.325+0000 7fa9a1ffb640 1 -- 172.21.15.97:0/729785032 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa990035070 con 0x7fa9a4107260 2024-04-23T10:31:42.326 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.325+0000 7fa9aaecf640 1 -- 172.21.15.97:0/729785032 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9a41a7800 con 0x7fa9a4107260 2024-04-23T10:31:42.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.326+0000 7fa9aaecf640 1 -- 172.21.15.97:0/729785032 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9a40691c0 con 0x7fa9a4107260 2024-04-23T10:31:42.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.327+0000 7fa9a1ffb640 1 -- 172.21.15.97:0/729785032 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa990042030 con 0x7fa9a4107260 2024-04-23T10:31:42.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.328+0000 7fa9a1ffb640 1 --2- 172.21.15.97:0/729785032 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fa9940a3dd0 0x7fa9940a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:42.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.328+0000 7fa9a8c6b640 1 --2- 172.21.15.97:0/729785032 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fa9940a3dd0 0x7fa9940a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:42.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.328+0000 7fa9a1ffb640 1 -- 172.21.15.97:0/729785032 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fa9900f0f10 con 0x7fa9a4107260 2024-04-23T10:31:42.327 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.328+0000 7fa9a8c6b640 1 --2- 172.21.15.97:0/729785032 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fa9940a3dd0 0x7fa9940a6290 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa998009bd0 tx=0x7fa998005f90 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:31:42.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.336+0000 7fa9a1ffb640 1 -- 172.21.15.97:0/729785032 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa9900bd600 con 0x7fa9a4107260 2024-04-23T10:31:42.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.337+0000 7f3c5b38a740 1 Processor -- start 2024-04-23T10:31:42.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.337+0000 7f3c5b38a740 1 -- start start 2024-04-23T10:31:42.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.337+0000 7f3c5b38a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562402666b30 0x5624026812f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:42.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.337+0000 7f3c5b38a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56240266a620 0x56240266d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:42.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.337+0000 7f3c5b38a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56240266af40 0x562402674bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:42.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.337+0000 7f3c5b38a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3c501afbf0 con 0x56240266a620 2024-04-23T10:31:42.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.337+0000 7f3c5b38a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3c501b1730 con 0x56240266af40 2024-04-23T10:31:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.337+0000 7f3c5b38a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3c501b02c0 con 0x562402666b30 2024-04-23T10:31:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.338+0000 7f3c5633d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56240266a620 0x56240266d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.338+0000 7f3c5633d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56240266a620 0x56240266d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51858/0 (socket says 172.21.15.97:51858) 2024-04-23T10:31:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.338+0000 7f3c5633d640 1 -- 172.21.15.97:0/1530311345 learned_addr learned my addr 172.21.15.97:0/1530311345 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:31:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.338+0000 7f3c55b3c640 1 --2- 172.21.15.97:0/1530311345 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562402666b30 0x5624026812f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.338+0000 7f3c55b3c640 1 --2- 172.21.15.97:0/1530311345 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56240266af40 0x562402674bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.338+0000 7f3c5633d640 1 -- 172.21.15.97:0/1530311345 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562402666b30 msgr2=0x5624026812f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:31:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.338+0000 7f3c5633d640 1 --2- 172.21.15.97:0/1530311345 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562402666b30 0x5624026812f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.338+0000 7f3c5633d640 1 -- 172.21.15.97:0/1530311345 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56240266af40 msgr2=0x562402674bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:31:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.338+0000 7f3c5633d640 1 --2- 172.21.15.97:0/1530311345 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56240266af40 0x562402674bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.338+0000 7f3c5633d640 1 -- 172.21.15.97:0/1530311345 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562402662660 con 0x56240266a620 2024-04-23T10:31:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.338+0000 7f3c5633d640 1 --2- 172.21.15.97:0/1530311345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56240266a620 0x56240266d970 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c024990 tx=0x7f3c4c003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=268a442164d53f5e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:31:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.338+0000 7f3c3bfff640 1 -- 172.21.15.97:0/1530311345 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c02a070 con 0x56240266a620 2024-04-23T10:31:42.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.338+0000 7f3c3bfff640 1 -- 172.21.15.97:0/1530311345 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c4c024a10 con 0x56240266a620 2024-04-23T10:31:42.339 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.338+0000 7f3c3bfff640 1 -- 172.21.15.97:0/1530311345 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c01a650 con 0x56240266a620 2024-04-23T10:31:42.342 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.339+0000 7f3c5b38a740 1 -- 172.21.15.97:0/1530311345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56240266a620 msgr2=0x56240266d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:31:42.342 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.339+0000 7f3c5b38a740 1 --2- 172.21.15.97:0/1530311345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56240266a620 0x56240266d970 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c024990 tx=0x7f3c4c003f60 comp rx=0 tx=0).stop 2024-04-23T10:31:42.342 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.339+0000 7f3c5b38a740 1 -- 172.21.15.97:0/1530311345 shutdown_connections 2024-04-23T10:31:42.342 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.339+0000 7f3c5b38a740 1 --2- 172.21.15.97:0/1530311345 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56240266af40 0x562402674bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.342 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.339+0000 7f3c5b38a740 1 --2- 172.21.15.97:0/1530311345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56240266a620 0x56240266d970 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.342 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.339+0000 7f3c5b38a740 1 --2- 172.21.15.97:0/1530311345 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562402666b30 0x5624026812f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.342 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.339+0000 7f3c5b38a740 1 -- 172.21.15.97:0/1530311345 >> 172.21.15.97:0/1530311345 conn(0x562402672a50 msgr2=0x56240266d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:31:42.342 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.339+0000 7f3c5b38a740 1 -- 172.21.15.97:0/1530311345 shutdown_connections 2024-04-23T10:31:42.342 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.339+0000 7f3c5b38a740 1 -- 172.21.15.97:0/1530311345 wait complete. 2024-04-23T10:31:42.342 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.339+0000 7f3c5b38a740 1 -- start start 2024-04-23T10:31:42.342 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.339+0000 7f3c5b38a740 1 Processor -- start 2024-04-23T10:31:42.342 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.340+0000 7f3c5b38a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562402666b30 0x56240267dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:42.343 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.340+0000 7f3c5b38a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56240266a620 0x56240266ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:42.343 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.340+0000 7f3c5b38a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56240266af40 0x56240266f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:42.343 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.340+0000 7f3c5b38a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3c501b1060 con 0x56240266a620 2024-04-23T10:31:42.343 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.340+0000 7f3c5b38a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3c501b0990 con 0x56240266af40 2024-04-23T10:31:42.343 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.340+0000 7f3c5b38a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3c501adbf0 con 0x562402666b30 2024-04-23T10:31:42.343 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.340+0000 7f3c5b38a740 10 client.0 _ll_register_callbacks cb 0x7f3c4c004230 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:31:42.343 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.340+0000 7f3c5633d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56240266a620 0x56240266ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:42.343 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.340+0000 7f3c5633d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56240266a620 0x56240266ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51874/0 (socket says 172.21.15.97:51874) 2024-04-23T10:31:42.343 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.340+0000 7f3c5633d640 1 -- 172.21.15.97:0/3501737705 learned_addr learned my addr 172.21.15.97:0/3501737705 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:31:42.343 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.340+0000 7f3c55b3c640 1 --2- 172.21.15.97:0/3501737705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56240266af40 0x56240266f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:42.343 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.340+0000 7f3c55b3c640 1 --2- 172.21.15.97:0/3501737705 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562402666b30 0x56240267dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:42.343 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.340+0000 7f3c5633d640 1 -- 172.21.15.97:0/3501737705 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562402666b30 msgr2=0x56240267dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:31:42.343 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.340+0000 7f3c5633d640 1 --2- 172.21.15.97:0/3501737705 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562402666b30 0x56240267dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.343 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.340+0000 7f3c5633d640 1 -- 172.21.15.97:0/3501737705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56240266af40 msgr2=0x56240266f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:31:42.343 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.340+0000 7f3c5633d640 1 --2- 172.21.15.97:0/3501737705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56240266af40 0x56240266f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.340+0000 7f3c5633d640 1 -- 172.21.15.97:0/3501737705 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624026ead10 con 0x56240266a620 2024-04-23T10:31:42.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.340+0000 7f3c55b3c640 1 --2- 172.21.15.97:0/3501737705 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562402666b30 0x56240267dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:31:42.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.340+0000 7f3c5633d640 1 --2- 172.21.15.97:0/3501737705 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56240266a620 0x56240266ef70 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c0248c0 tx=0x7f3c4c011bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c5d4a83501b3009 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:31:42.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.340+0000 7f3c39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:31:42.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.340+0000 7f3c39ffb640 1 -- 172.21.15.97:0/3501737705 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c02a070 con 0x56240266a620 2024-04-23T10:31:42.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.340+0000 7f3c39ffb640 1 -- 172.21.15.97:0/3501737705 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f3c4c02e400 con 0x56240266a620 2024-04-23T10:31:42.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.340+0000 7f3c5b38a740 1 -- 172.21.15.97:0/3501737705 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c4c024a10 con 0x56240266a620 2024-04-23T10:31:42.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.341+0000 7f3c39ffb640 1 -- 172.21.15.97:0/3501737705 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3c4c02e600 con 0x56240266a620 2024-04-23T10:31:42.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.341+0000 7f3c5b38a740 10 client.16335 fetch_fsmap learned FSMap version 26 2024-04-23T10:31:42.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.341+0000 7f3c5b38a740 1 -- 172.21.15.97:0/3501737705 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5624026ebe90 con 0x56240266a620 2024-04-23T10:31:42.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.341+0000 7f3c39ffb640 1 -- 172.21.15.97:0/3501737705 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c4c024a10 con 0x56240266a620 2024-04-23T10:31:42.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.341+0000 7f3c39ffb640 1 -- 172.21.15.97:0/3501737705 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3c4c019070 con 0x56240266a620 2024-04-23T10:31:42.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.341+0000 7f3c5b38a740 10 client.16335 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:31:42.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.341+0000 7f3c5b38a740 10 client.16335 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:31:42.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.341+0000 7f3c5b38a740 20 client.16335 populate_metadata read hostname 'smithi097' 2024-04-23T10:31:42.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.341+0000 7f3c5b38a740 1 --2- 172.21.15.97:0/3501737705 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x562402675800 0x5624026ee4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:42.345 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.341+0000 7f3c5b38a740 4 client.16335 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:31:42.345 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.341+0000 7f3c5b38a740 1 -- 172.21.15.97:0/3501737705 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5624025d7200 con 0x562402675800 2024-04-23T10:31:42.345 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.341+0000 7f3c55b3c640 1 --2- 172.21.15.97:0/3501737705 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x562402675800 0x5624026ee4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:42.345 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.342+0000 7f3c55b3c640 1 --2- 172.21.15.97:0/3501737705 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x562402675800 0x5624026ee4b0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4301468905a1550 server_cookie=ca5d9421e1dd97e5 in_seq=0 out_seq=0 2024-04-23T10:31:42.345 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.342+0000 7f3c39ffb640 10 client.16335 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:31:42.354 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.354+0000 7f3c39ffb640 1 -- 172.21.15.97:0/3501737705 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5624025d7200 con 0x562402675800 2024-04-23T10:31:42.354 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.354+0000 7f3c39ffb640 10 client.16335 handle_command_reply: tid=0 2024-04-23T10:31:42.354 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.355+0000 7f3c5b38a740 1 client.16335 shutdown 2024-04-23T10:31:42.354 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.355+0000 7f3c5b38a740 1 -- 172.21.15.97:0/3501737705 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56240266a620 msgr2=0x56240266ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:31:42.354 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.355+0000 7f3c5b38a740 1 --2- 172.21.15.97:0/3501737705 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56240266a620 0x56240266ef70 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c0248c0 tx=0x7f3c4c011bd0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.354 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.355+0000 7f3c5b38a740 1 -- 172.21.15.97:0/3501737705 shutdown_connections 2024-04-23T10:31:42.354 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.355+0000 7f3c5b38a740 1 --2- 172.21.15.97:0/3501737705 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x562402675800 0x5624026ee4b0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.355 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.355+0000 7f3c5b38a740 1 --2- 172.21.15.97:0/3501737705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56240266af40 0x56240266f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.355 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.355+0000 7f3c5b38a740 1 --2- 172.21.15.97:0/3501737705 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56240266a620 0x56240266ef70 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.355 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.355+0000 7f3c5b38a740 1 --2- 172.21.15.97:0/3501737705 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562402666b30 0x56240267dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.355 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.355+0000 7f3c5b38a740 1 -- 172.21.15.97:0/3501737705 >> 172.21.15.97:0/3501737705 conn(0x562402672a50 msgr2=0x56240266bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:31:42.355 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.355+0000 7f3c39ffb640 0 client.16335 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:31:42.355 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.355+0000 7f3c5b38a740 1 -- 172.21.15.97:0/3501737705 shutdown_connections 2024-04-23T10:31:42.355 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.355+0000 7f3c5b38a740 1 -- 172.21.15.97:0/3501737705 wait complete. 2024-04-23T10:31:42.355 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.355+0000 7f3c5b38a740 20 client.16335 trim_cache size 0 max 16384 2024-04-23T10:31:42.355 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:31:42.355 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:31:42.355 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:31:42.355 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:31:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.356+0000 7f3c585a1640 1 -- 172.21.15.97:0/3231584318 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f3c3c0a3dc0 msgr2=0x7f3c3c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:31:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.356+0000 7f3c585a1640 1 --2- 172.21.15.97:0/3231584318 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f3c3c0a3dc0 0x7f3c3c0a6280 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3c480ba1a0 tx=0x7f3c4802a8c0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.356+0000 7f3c585a1640 1 -- 172.21.15.97:0/3231584318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c501035b0 msgr2=0x7f3c501a6890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:31:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.356+0000 7f3c585a1640 1 --2- 172.21.15.97:0/3231584318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c501035b0 0x7f3c501a6890 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f3c480098d0 tx=0x7f3c480317b0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.356+0000 7f3c585a1640 1 -- 172.21.15.97:0/3231584318 shutdown_connections 2024-04-23T10:31:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.356+0000 7f3c585a1640 1 --2- 172.21.15.97:0/3231584318 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f3c3c0a3dc0 0x7f3c3c0a6280 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.356+0000 7f3c585a1640 1 --2- 172.21.15.97:0/3231584318 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c501035b0 0x7f3c501a6890 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.356+0000 7f3c585a1640 1 --2- 172.21.15.97:0/3231584318 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c50106170 0x7f3c50103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.356+0000 7f3c585a1640 1 --2- 172.21.15.97:0/3231584318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3c501010f0 0x7f3c50102b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.356+0000 7f3c585a1640 1 -- 172.21.15.97:0/3231584318 >> 172.21.15.97:0/3231584318 conn(0x7f3c50077fd0 msgr2=0x7f3c500698e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:31:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.356+0000 7f3c585a1640 1 -- 172.21.15.97:0/3231584318 shutdown_connections 2024-04-23T10:31:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:42.356+0000 7f3c585a1640 1 -- 172.21.15.97:0/3231584318 wait complete. 2024-04-23T10:31:42.563 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.564+0000 7fa9aaecf640 1 -- 172.21.15.97:0/729785032 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa9a410a490 con 0x7fa9a4107260 2024-04-23T10:31:42.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.565+0000 7fa9a1ffb640 1 -- 172.21.15.97:0/729785032 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11931 (secure 0 0 0) 0x7fa9900bcf00 con 0x7fa9a4107260 2024-04-23T10:31:42.564 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:31:42.564 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3946878762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3946878762},{"type":"v1","addr":"172.21.15.97:6835","nonce":3946878762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/209800861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":209800861},{"type":"v1","addr":"172.21.15.97:6837","nonce":209800861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T10:24:06.910134+0000","modified":"2024-04-23T10:26:09.820985+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6839/418013575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":418013575},{"type":"v1","addr":"172.21.15.97:6839","nonce":418013575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6841/747242680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":747242680},{"type":"v1","addr":"172.21.15.97:6841","nonce":747242680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6833/4292847121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4292847121},{"type":"v1","addr":"172.21.15.183:6833","nonce":4292847121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6833/3885816565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3885816565},{"type":"v1","addr":"172.21.15.196:6833","nonce":3885816565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1897093105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1897093105},{"type":"v1","addr":"172.21.15.183:6835","nonce":1897093105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1171477563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1171477563},{"type":"v1","addr":"172.21.15.196:6835","nonce":1171477563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1898341911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1898341911},{"type":"v1","addr":"172.21.15.183:6837","nonce":1898341911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.196:6837/3094946502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3094946502},{"type":"v1","addr":"172.21.15.196:6837","nonce":3094946502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.196:6839/83843556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":83843556},{"type":"v1","addr":"172.21.15.196:6839","nonce":83843556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6839/1763841130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1763841130},{"type":"v1","addr":"172.21.15.183:6839","nonce":1763841130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24487,24403,24398,24469,24488]},"id":1}]} 2024-04-23T10:31:42.564 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 26 2024-04-23T10:31:42.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.567+0000 7fa9aaecf640 1 -- 172.21.15.97:0/729785032 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fa9940a3dd0 msgr2=0x7fa9940a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:31:42.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.567+0000 7fa9aaecf640 1 --2- 172.21.15.97:0/729785032 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fa9940a3dd0 0x7fa9940a6290 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa998009bd0 tx=0x7fa998005f90 comp rx=0 tx=0).stop 2024-04-23T10:31:42.566 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.567+0000 7fa9aaecf640 1 -- 172.21.15.97:0/729785032 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9a4107260 msgr2=0x7fa9a4074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:31:42.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.567+0000 7fa9aaecf640 1 --2- 172.21.15.97:0/729785032 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9a4107260 0x7fa9a4074790 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fa990006010 tx=0x7fa990004060 comp rx=0 tx=0).stop 2024-04-23T10:31:42.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.567+0000 7fa9aaecf640 1 -- 172.21.15.97:0/729785032 shutdown_connections 2024-04-23T10:31:42.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.567+0000 7fa9aaecf640 1 --2- 172.21.15.97:0/729785032 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fa9940a3dd0 0x7fa9940a6290 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.567+0000 7fa9aaecf640 1 --2- 172.21.15.97:0/729785032 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa9a4074cd0 0x7fa9a41a6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.567+0000 7fa9aaecf640 1 --2- 172.21.15.97:0/729785032 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9a4107260 0x7fa9a4074790 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.567+0000 7fa9aaecf640 1 --2- 172.21.15.97:0/729785032 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9a4102190 0x7fa9a4074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.567+0000 7fa9aaecf640 1 -- 172.21.15.97:0/729785032 >> 172.21.15.97:0/729785032 conn(0x7fa9a4079310 msgr2=0x7fa9a41002e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:31:42.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.567+0000 7fa9aaecf640 1 -- 172.21.15.97:0/729785032 shutdown_connections 2024-04-23T10:31:42.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.567+0000 7fa9aaecf640 1 -- 172.21.15.97:0/729785032 wait complete. 2024-04-23T10:31:42.577 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db / --timeout 60 --expiration 79.45 --await-for 42.0' 2024-04-23T10:31:42.577 DEBUG:teuthology.orchestra.run.smithi097:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24488 quiesce db / --timeout 60 --expiration 79.45 --await-for 42.0 2024-04-23T10:31:42.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.686+0000 7fb740595640 1 -- 172.21.15.97:0/2250112949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb738103230 msgr2=0x7fb738103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:31:42.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.686+0000 7fb740595640 1 --2- 172.21.15.97:0/2250112949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb738103230 0x7fb738103610 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fb734009900 tx=0x7fb73402a480 comp rx=0 tx=0).stop 2024-04-23T10:31:42.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.687+0000 7fb740595640 1 -- 172.21.15.97:0/2250112949 shutdown_connections 2024-04-23T10:31:42.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.687+0000 7fb740595640 1 --2- 172.21.15.97:0/2250112949 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb73810c570 0x7fb73810e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.687+0000 7fb740595640 1 --2- 172.21.15.97:0/2250112949 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb738103b50 0x7fb73810c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.687+0000 7fb740595640 1 --2- 172.21.15.97:0/2250112949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb738103230 0x7fb738103610 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.687+0000 7fb740595640 1 -- 172.21.15.97:0/2250112949 >> 172.21.15.97:0/2250112949 conn(0x7fb738077fb0 msgr2=0x7fb7380783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:31:42.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.687+0000 7fb740595640 1 -- 172.21.15.97:0/2250112949 shutdown_connections 2024-04-23T10:31:42.686 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.687+0000 7fb740595640 1 -- 172.21.15.97:0/2250112949 wait complete. 2024-04-23T10:31:42.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.687+0000 7fb740595640 1 Processor -- start 2024-04-23T10:31:42.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.687+0000 7fb740595640 1 -- start start 2024-04-23T10:31:42.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.687+0000 7fb740595640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb738103230 0x7fb7381a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:42.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.687+0000 7fb740595640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb738103b50 0x7fb7381a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:42.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.687+0000 7fb740595640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb73810c570 0x7fb73819a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:42.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.687+0000 7fb740595640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb738112230 con 0x7fb73810c570 2024-04-23T10:31:42.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.687+0000 7fb740595640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb7381120b0 con 0x7fb738103b50 2024-04-23T10:31:42.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.687+0000 7fb740595640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb7381123b0 con 0x7fb738103230 2024-04-23T10:31:42.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.688+0000 7fb73eb32640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb73810c570 0x7fb73819a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:42.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.688+0000 7fb73eb32640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb73810c570 0x7fb73819a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51892/0 (socket says 172.21.15.97:51892) 2024-04-23T10:31:42.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.688+0000 7fb73eb32640 1 -- 172.21.15.97:0/1154369961 learned_addr learned my addr 172.21.15.97:0/1154369961 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:31:42.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.688+0000 7fb73e331640 1 --2- 172.21.15.97:0/1154369961 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb738103230 0x7fb7381a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:42.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.688+0000 7fb73db30640 1 --2- 172.21.15.97:0/1154369961 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb738103b50 0x7fb7381a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:42.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.688+0000 7fb73eb32640 1 -- 172.21.15.97:0/1154369961 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb738103230 msgr2=0x7fb7381a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:31:42.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.688+0000 7fb73eb32640 1 --2- 172.21.15.97:0/1154369961 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb738103230 0x7fb7381a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.688+0000 7fb73eb32640 1 -- 172.21.15.97:0/1154369961 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb738103b50 msgr2=0x7fb7381a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:31:42.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.688+0000 7fb73eb32640 1 --2- 172.21.15.97:0/1154369961 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb738103b50 0x7fb7381a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.688+0000 7fb73eb32640 1 -- 172.21.15.97:0/1154369961 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb73819afb0 con 0x7fb73810c570 2024-04-23T10:31:42.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.688+0000 7fb73db30640 1 --2- 172.21.15.97:0/1154369961 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb738103b50 0x7fb7381a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:31:42.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.688+0000 7fb73eb32640 1 --2- 172.21.15.97:0/1154369961 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb73810c570 0x7fb73819a6f0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fb73000b9e0 tx=0x7fb73000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:31:42.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.688+0000 7fb72f7fe640 1 -- 172.21.15.97:0/1154369961 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb730019070 con 0x7fb73810c570 2024-04-23T10:31:42.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.688+0000 7fb740595640 1 -- 172.21.15.97:0/1154369961 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb73819b2a0 con 0x7fb73810c570 2024-04-23T10:31:42.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.688+0000 7fb72f7fe640 1 -- 172.21.15.97:0/1154369961 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb730014440 con 0x7fb73810c570 2024-04-23T10:31:42.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.688+0000 7fb72f7fe640 1 -- 172.21.15.97:0/1154369961 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7300075d0 con 0x7fb73810c570 2024-04-23T10:31:42.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.688+0000 7fb740595640 1 -- 172.21.15.97:0/1154369961 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7381a75b0 con 0x7fb73810c570 2024-04-23T10:31:42.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.690+0000 7fb72f7fe640 1 -- 172.21.15.97:0/1154369961 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb73001c070 con 0x7fb73810c570 2024-04-23T10:31:42.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.691+0000 7fb743386740 1 Processor -- start 2024-04-23T10:31:42.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.691+0000 7fb743386740 1 -- start start 2024-04-23T10:31:42.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.691+0000 7fb743386740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646f36d80a0 0x5646f36d8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:42.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.691+0000 7fb72f7fe640 1 --2- 172.21.15.97:0/1154369961 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fb7240a3d20 0x7fb7240a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:42.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.691+0000 7fb743386740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646f36d89c0 0x5646f36e2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:42.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.691+0000 7fb743386740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646f36e3090 0x5646f36e5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:42.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.691+0000 7fb743386740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5646f363d370 con 0x5646f36d80a0 2024-04-23T10:31:42.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.691+0000 7fb743386740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5646f35da2e0 con 0x5646f36d89c0 2024-04-23T10:31:42.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.691+0000 7fb743386740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5646f3639330 con 0x5646f36e3090 2024-04-23T10:31:42.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.691+0000 7fb73db30640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646f36d80a0 0x5646f36d8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:42.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.691+0000 7fb73e331640 1 --2- 172.21.15.97:0/1154369961 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fb7240a3d20 0x7fb7240a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:42.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.691+0000 7fb73db30640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646f36d80a0 0x5646f36d8480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51900/0 (socket says 172.21.15.97:51900) 2024-04-23T10:31:42.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.691+0000 7fb73db30640 1 -- 172.21.15.97:0/3102410567 learned_addr learned my addr 172.21.15.97:0/3102410567 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:31:42.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.691+0000 7fb72f7fe640 1 -- 172.21.15.97:0/1154369961 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fb7300cfc50 con 0x7fb73810c570 2024-04-23T10:31:42.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.691+0000 7fb73eb32640 1 --2- 172.21.15.97:0/3102410567 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646f36e3090 0x5646f36e5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:42.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.692+0000 7fb73db30640 1 -- 172.21.15.97:0/3102410567 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646f36e3090 msgr2=0x5646f36e5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:31:42.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.692+0000 7fb73db30640 1 --2- 172.21.15.97:0/3102410567 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646f36e3090 0x5646f36e5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.692+0000 7fb73db30640 1 -- 172.21.15.97:0/3102410567 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646f36d89c0 msgr2=0x5646f36e2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:31:42.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.692+0000 7fb73db30640 1 --2- 172.21.15.97:0/3102410567 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646f36d89c0 0x5646f36e2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.692+0000 7fb73db30640 1 -- 172.21.15.97:0/3102410567 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646f341bd80 con 0x5646f36d80a0 2024-04-23T10:31:42.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.692+0000 7fb73e331640 1 --2- 172.21.15.97:0/1154369961 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fb7240a3d20 0x7fb7240a61e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb734031440 tx=0x7fb734005e80 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:31:42.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.692+0000 7fb73db30640 1 --2- 172.21.15.97:0/3102410567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646f36d80a0 0x5646f36d8480 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fb728008cc0 tx=0x7fb728010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=719db26c09e49401 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:31:42.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.692+0000 7fb72cbfe640 1 -- 172.21.15.97:0/3102410567 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb728016070 con 0x5646f36d80a0 2024-04-23T10:31:42.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.692+0000 7fb72cbfe640 1 -- 172.21.15.97:0/3102410567 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7280056d0 con 0x5646f36d80a0 2024-04-23T10:31:42.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.692+0000 7fb72cbfe640 1 -- 172.21.15.97:0/3102410567 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7280126e0 con 0x5646f36d80a0 2024-04-23T10:31:42.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.693+0000 7fb743386740 1 -- 172.21.15.97:0/3102410567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646f36d80a0 msgr2=0x5646f36d8480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:31:42.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.693+0000 7fb743386740 1 --2- 172.21.15.97:0/3102410567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646f36d80a0 0x5646f36d8480 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fb728008cc0 tx=0x7fb728010740 comp rx=0 tx=0).stop 2024-04-23T10:31:42.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.693+0000 7fb743386740 1 -- 172.21.15.97:0/3102410567 shutdown_connections 2024-04-23T10:31:42.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.693+0000 7fb743386740 1 --2- 172.21.15.97:0/3102410567 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646f36e3090 0x5646f36e5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.693+0000 7fb743386740 1 --2- 172.21.15.97:0/3102410567 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646f36d89c0 0x5646f36e2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.693+0000 7fb743386740 1 --2- 172.21.15.97:0/3102410567 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646f36d80a0 0x5646f36d8480 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.693+0000 7fb743386740 1 -- 172.21.15.97:0/3102410567 >> 172.21.15.97:0/3102410567 conn(0x5646f36d43c0 msgr2=0x5646f36d47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:31:42.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.693+0000 7fb743386740 1 -- 172.21.15.97:0/3102410567 shutdown_connections 2024-04-23T10:31:42.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.693+0000 7fb743386740 1 -- 172.21.15.97:0/3102410567 wait complete. 2024-04-23T10:31:42.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.693+0000 7fb743386740 1 -- start start 2024-04-23T10:31:42.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.693+0000 7fb743386740 1 Processor -- start 2024-04-23T10:31:42.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.693+0000 7fb743386740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646f36d80a0 0x5646f36e0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:42.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.693+0000 7fb743386740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646f36d89c0 0x5646f36e0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:42.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.693+0000 7fb743386740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646f36e3090 0x5646f36e0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:42.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.693+0000 7fb743386740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5646f36e5e80 con 0x5646f36d80a0 2024-04-23T10:31:42.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.693+0000 7fb743386740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5646f35d7fb0 con 0x5646f36d89c0 2024-04-23T10:31:42.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.693+0000 7fb743386740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5646f36e6000 con 0x5646f36e3090 2024-04-23T10:31:42.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.694+0000 7fb743386740 10 client.0 _ll_register_callbacks cb 0x5646f35d79a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:31:42.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.694+0000 7fb73e331640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646f36d80a0 0x5646f36e0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:42.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.694+0000 7fb73e331640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646f36d80a0 0x5646f36e0050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51904/0 (socket says 172.21.15.97:51904) 2024-04-23T10:31:42.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.694+0000 7fb73e331640 1 -- 172.21.15.97:0/1080737255 learned_addr learned my addr 172.21.15.97:0/1080737255 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:31:42.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.694+0000 7fb73eb32640 1 --2- 172.21.15.97:0/1080737255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646f36e3090 0x5646f36e0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:42.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.694+0000 7fb73db30640 1 --2- 172.21.15.97:0/1080737255 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646f36d89c0 0x5646f36e0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:42.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.694+0000 7fb73e331640 1 -- 172.21.15.97:0/1080737255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646f36e3090 msgr2=0x5646f36e0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:31:42.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.694+0000 7fb73e331640 1 --2- 172.21.15.97:0/1080737255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646f36e3090 0x5646f36e0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.694+0000 7fb73e331640 1 -- 172.21.15.97:0/1080737255 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646f36d89c0 msgr2=0x5646f36e0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:31:42.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.694+0000 7fb73e331640 1 --2- 172.21.15.97:0/1080737255 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646f36d89c0 0x5646f36e0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.694+0000 7fb73e331640 1 -- 172.21.15.97:0/1080737255 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646f347c510 con 0x5646f36d80a0 2024-04-23T10:31:42.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.694+0000 7fb73e331640 1 --2- 172.21.15.97:0/1080737255 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646f36d80a0 0x5646f36e0050 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fb734009900 tx=0x7fb734009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f1a87f0513d27a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:31:42.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.694+0000 7fb721ffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:31:42.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.694+0000 7fb721ffb640 1 -- 172.21.15.97:0/1080737255 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb73404c070 con 0x5646f36d80a0 2024-04-23T10:31:42.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.694+0000 7fb721ffb640 1 -- 172.21.15.97:0/1080737255 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fb7340035f0 con 0x5646f36d80a0 2024-04-23T10:31:42.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.694+0000 7fb743386740 1 -- 172.21.15.97:0/1080737255 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7280056d0 con 0x5646f36d80a0 2024-04-23T10:31:42.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.694+0000 7fb721ffb640 1 -- 172.21.15.97:0/1080737255 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb734058050 con 0x5646f36d80a0 2024-04-23T10:31:42.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.694+0000 7fb743386740 10 client.16347 fetch_fsmap learned FSMap version 26 2024-04-23T10:31:42.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.694+0000 7fb743386740 1 -- 172.21.15.97:0/1080737255 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5646f330f670 con 0x5646f36d80a0 2024-04-23T10:31:42.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.694+0000 7fb721ffb640 1 -- 172.21.15.97:0/1080737255 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7280056d0 con 0x5646f36d80a0 2024-04-23T10:31:42.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.695+0000 7fb721ffb640 1 -- 172.21.15.97:0/1080737255 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb73403c850 con 0x5646f36d80a0 2024-04-23T10:31:42.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.695+0000 7fb743386740 10 client.16347 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:31:42.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.695+0000 7fb743386740 10 client.16347 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:31:42.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.695+0000 7fb743386740 20 client.16347 populate_metadata read hostname 'smithi097' 2024-04-23T10:31:42.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.695+0000 7fb743386740 1 --2- 172.21.15.97:0/1080737255 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5646f36f3430 0x5646f36f58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:42.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.695+0000 7fb743386740 4 client.16347 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:31:42.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.695+0000 7fb743386740 1 -- 172.21.15.97:0/1080737255 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646f364ad90 con 0x5646f36f3430 2024-04-23T10:31:42.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.695+0000 7fb73db30640 1 --2- 172.21.15.97:0/1080737255 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5646f36f3430 0x5646f36f58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:42.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.695+0000 7fb721ffb640 10 client.16347 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:31:42.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.695+0000 7fb73db30640 1 --2- 172.21.15.97:0/1080737255 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5646f36f3430 0x5646f36f58b0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55654c5d055d0813 server_cookie=49a65b396ac61d44 in_seq=0 out_seq=0 2024-04-23T10:31:42.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.696+0000 7fb721ffb640 1 -- 172.21.15.97:0/1080737255 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5646f364ad90 con 0x5646f36f3430 2024-04-23T10:31:42.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.696+0000 7fb721ffb640 10 client.16347 handle_command_reply: tid=0 2024-04-23T10:31:42.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.696+0000 7fb743386740 1 client.16347 shutdown 2024-04-23T10:31:42.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.696+0000 7fb743386740 1 -- 172.21.15.97:0/1080737255 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646f36d80a0 msgr2=0x5646f36e0050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:31:42.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.696+0000 7fb743386740 1 --2- 172.21.15.97:0/1080737255 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646f36d80a0 0x5646f36e0050 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fb734009900 tx=0x7fb734009260 comp rx=0 tx=0).stop 2024-04-23T10:31:42.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.696+0000 7fb743386740 1 -- 172.21.15.97:0/1080737255 shutdown_connections 2024-04-23T10:31:42.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.696+0000 7fb743386740 1 --2- 172.21.15.97:0/1080737255 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5646f36f3430 0x5646f36f58b0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.696+0000 7fb743386740 1 --2- 172.21.15.97:0/1080737255 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646f36e3090 0x5646f36e0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.696+0000 7fb743386740 1 --2- 172.21.15.97:0/1080737255 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646f36d89c0 0x5646f36e0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.696+0000 7fb721ffb640 0 client.16347 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:31:42.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.696+0000 7fb743386740 1 --2- 172.21.15.97:0/1080737255 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646f36d80a0 0x5646f36e0050 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.696+0000 7fb743386740 1 -- 172.21.15.97:0/1080737255 >> 172.21.15.97:0/1080737255 conn(0x5646f36d43c0 msgr2=0x5646f36d3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:31:42.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.696+0000 7fb743386740 1 -- 172.21.15.97:0/1080737255 shutdown_connections 2024-04-23T10:31:42.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.696+0000 7fb743386740 1 -- 172.21.15.97:0/1080737255 wait complete. 2024-04-23T10:31:42.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.696+0000 7fb743386740 20 client.16347 trim_cache size 0 max 16384 2024-04-23T10:31:42.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.713+0000 7fb743386740 1 Processor -- start 2024-04-23T10:31:42.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.713+0000 7fb743386740 1 -- start start 2024-04-23T10:31:42.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.713+0000 7fb743386740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646f36d80a0 0x5646f36e1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:42.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.713+0000 7fb743386740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646f36d89c0 0x5646f36da800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:42.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.713+0000 7fb743386740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646f36e3090 0x5646f36fb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:42.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.713+0000 7fb743386740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb7381afbe0 con 0x5646f36d89c0 2024-04-23T10:31:42.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.713+0000 7fb743386740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb7381b1720 con 0x5646f36d80a0 2024-04-23T10:31:42.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.713+0000 7fb743386740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb7381b02b0 con 0x5646f36e3090 2024-04-23T10:31:42.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.713+0000 7fb73db30640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646f36d89c0 0x5646f36da800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:42.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.714+0000 7fb73db30640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646f36d89c0 0x5646f36da800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51920/0 (socket says 172.21.15.97:51920) 2024-04-23T10:31:42.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.714+0000 7fb73db30640 1 -- 172.21.15.97:0/464710240 learned_addr learned my addr 172.21.15.97:0/464710240 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:31:42.716 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.714+0000 7fb73db30640 1 -- 172.21.15.97:0/464710240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646f36e3090 msgr2=0x5646f36fb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:31:42.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.714+0000 7fb73eb32640 1 --2- 172.21.15.97:0/464710240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646f36e3090 0x5646f36fb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:42.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.714+0000 7fb73db30640 1 --2- 172.21.15.97:0/464710240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646f36e3090 0x5646f36fb180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.714+0000 7fb73db30640 1 -- 172.21.15.97:0/464710240 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646f36d80a0 msgr2=0x5646f36e1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:31:42.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.714+0000 7fb73e331640 1 --2- 172.21.15.97:0/464710240 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646f36d80a0 0x5646f36e1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:42.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.714+0000 7fb73db30640 1 --2- 172.21.15.97:0/464710240 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646f36d80a0 0x5646f36e1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.714+0000 7fb73db30640 1 -- 172.21.15.97:0/464710240 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646f36d4f90 con 0x5646f36d89c0 2024-04-23T10:31:42.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.714+0000 7fb73e331640 1 --2- 172.21.15.97:0/464710240 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646f36d80a0 0x5646f36e1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:31:42.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.714+0000 7fb73db30640 1 --2- 172.21.15.97:0/464710240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646f36d89c0 0x5646f36da800 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fb728008380 tx=0x7fb728008640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b194b9f0703ff79 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:31:42.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.714+0000 7fb723fff640 1 -- 172.21.15.97:0/464710240 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb728016040 con 0x5646f36d89c0 2024-04-23T10:31:42.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.714+0000 7fb723fff640 1 -- 172.21.15.97:0/464710240 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb728004ad0 con 0x5646f36d89c0 2024-04-23T10:31:42.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.714+0000 7fb723fff640 1 -- 172.21.15.97:0/464710240 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7280284b0 con 0x5646f36d89c0 2024-04-23T10:31:42.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.715+0000 7fb743386740 1 -- 172.21.15.97:0/464710240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646f36d89c0 msgr2=0x5646f36da800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:31:42.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.715+0000 7fb743386740 1 --2- 172.21.15.97:0/464710240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646f36d89c0 0x5646f36da800 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fb728008380 tx=0x7fb728008640 comp rx=0 tx=0).stop 2024-04-23T10:31:42.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.715+0000 7fb743386740 1 -- 172.21.15.97:0/464710240 shutdown_connections 2024-04-23T10:31:42.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.715+0000 7fb743386740 1 --2- 172.21.15.97:0/464710240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646f36e3090 0x5646f36fb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.715+0000 7fb743386740 1 --2- 172.21.15.97:0/464710240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646f36d89c0 0x5646f36da800 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.715+0000 7fb743386740 1 --2- 172.21.15.97:0/464710240 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646f36d80a0 0x5646f36e1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.715+0000 7fb743386740 1 -- 172.21.15.97:0/464710240 >> 172.21.15.97:0/464710240 conn(0x5646f36d43c0 msgr2=0x5646f36e08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:31:42.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.715+0000 7fb743386740 1 -- 172.21.15.97:0/464710240 shutdown_connections 2024-04-23T10:31:42.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.715+0000 7fb743386740 1 -- 172.21.15.97:0/464710240 wait complete. 2024-04-23T10:31:42.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.715+0000 7fb743386740 1 -- start start 2024-04-23T10:31:42.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.715+0000 7fb743386740 1 Processor -- start 2024-04-23T10:31:42.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.715+0000 7fb743386740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646f36d80a0 0x5646f36eb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:42.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.715+0000 7fb743386740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646f36d89c0 0x5646f36e8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:42.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.715+0000 7fb743386740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646f36e3090 0x5646f36e8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:42.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.715+0000 7fb743386740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb7381b1050 con 0x5646f36d80a0 2024-04-23T10:31:42.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.715+0000 7fb743386740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb7381b0980 con 0x5646f36d89c0 2024-04-23T10:31:42.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.715+0000 7fb743386740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb7381adbc0 con 0x5646f36e3090 2024-04-23T10:31:42.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.715+0000 7fb743386740 10 client.0 _ll_register_callbacks cb 0x7fb734032f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:31:42.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.715+0000 7fb73e331640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646f36d80a0 0x5646f36eb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:42.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.715+0000 7fb73e331640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646f36d80a0 0x5646f36eb4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:51930/0 (socket says 172.21.15.97:51930) 2024-04-23T10:31:42.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.715+0000 7fb73e331640 1 -- 172.21.15.97:0/1717261136 learned_addr learned my addr 172.21.15.97:0/1717261136 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:31:42.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.716+0000 7fb73e331640 1 -- 172.21.15.97:0/1717261136 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646f36e3090 msgr2=0x5646f36e8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:31:42.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.716+0000 7fb73e331640 1 --2- 172.21.15.97:0/1717261136 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646f36e3090 0x5646f36e8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.716+0000 7fb73e331640 1 -- 172.21.15.97:0/1717261136 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646f36d89c0 msgr2=0x5646f36e8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:31:42.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.716+0000 7fb73e331640 1 --2- 172.21.15.97:0/1717261136 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646f36d89c0 0x5646f36e8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.716+0000 7fb73e331640 1 -- 172.21.15.97:0/1717261136 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646f36e8e00 con 0x5646f36d80a0 2024-04-23T10:31:42.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.716+0000 7fb73e331640 1 --2- 172.21.15.97:0/1717261136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646f36d80a0 0x5646f36eb4b0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fb73402a4c0 tx=0x7fb734031e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4405c0111af38449 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:31:42.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.716+0000 7fb721ffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:31:42.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.716+0000 7fb721ffb640 1 -- 172.21.15.97:0/1717261136 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb73404c040 con 0x5646f36d80a0 2024-04-23T10:31:42.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.716+0000 7fb721ffb640 1 -- 172.21.15.97:0/1717261136 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fb73403c910 con 0x5646f36d80a0 2024-04-23T10:31:42.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.716+0000 7fb743386740 1 -- 172.21.15.97:0/1717261136 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb728004ad0 con 0x5646f36d80a0 2024-04-23T10:31:42.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.716+0000 7fb721ffb640 1 -- 172.21.15.97:0/1717261136 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb734035040 con 0x5646f36d80a0 2024-04-23T10:31:42.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.716+0000 7fb743386740 10 client.16353 fetch_fsmap learned FSMap version 26 2024-04-23T10:31:42.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.716+0000 7fb743386740 1 -- 172.21.15.97:0/1717261136 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5646f375a7e0 con 0x5646f36d80a0 2024-04-23T10:31:42.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.716+0000 7fb721ffb640 1 -- 172.21.15.97:0/1717261136 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb728004ad0 con 0x5646f36d80a0 2024-04-23T10:31:42.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.716+0000 7fb721ffb640 1 -- 172.21.15.97:0/1717261136 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb7340486b0 con 0x5646f36d80a0 2024-04-23T10:31:42.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.716+0000 7fb743386740 10 client.16353 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:31:42.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.716+0000 7fb743386740 10 client.16353 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:31:42.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.716+0000 7fb743386740 20 client.16353 populate_metadata read hostname 'smithi097' 2024-04-23T10:31:42.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.716+0000 7fb743386740 1 --2- 172.21.15.97:0/1717261136 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5646f375af70 0x5646f375d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:42.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.716+0000 7fb743386740 4 client.16353 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 79.45, "await_for": 42.0}] 2024-04-23T10:31:42.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.716+0000 7fb743386740 1 -- 172.21.15.97:0/1717261136 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 79.45, "await_for": 42.0}) -- 0x5646f3644c00 con 0x5646f375af70 2024-04-23T10:31:42.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.717+0000 7fb73db30640 1 --2- 172.21.15.97:0/1717261136 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5646f375af70 0x5646f375d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:42.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.717+0000 7fb721ffb640 10 client.16353 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:31:42.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.717+0000 7fb73db30640 1 --2- 172.21.15.97:0/1717261136 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5646f375af70 0x5646f375d430 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9679ad39c310f3f server_cookie=327e432634dc9213 in_seq=0 out_seq=0 2024-04-23T10:31:42.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.731+0000 7fb721ffb640 1 -- 172.21.15.97:0/1717261136 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5646f3644c00 con 0x5646f375af70 2024-04-23T10:31:42.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.731+0000 7fb721ffb640 10 client.16353 handle_command_reply: tid=0 2024-04-23T10:31:42.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.731+0000 7fb743386740 1 client.16353 shutdown 2024-04-23T10:31:42.730 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.731+0000 7fb743386740 1 -- 172.21.15.97:0/1717261136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646f36d80a0 msgr2=0x5646f36eb4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:31:42.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.731+0000 7fb743386740 1 --2- 172.21.15.97:0/1717261136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646f36d80a0 0x5646f36eb4b0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fb73402a4c0 tx=0x7fb734031e80 comp rx=0 tx=0).stop 2024-04-23T10:31:42.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.731+0000 7fb743386740 1 -- 172.21.15.97:0/1717261136 shutdown_connections 2024-04-23T10:31:42.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.731+0000 7fb743386740 1 --2- 172.21.15.97:0/1717261136 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5646f375af70 0x5646f375d430 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.731+0000 7fb743386740 1 --2- 172.21.15.97:0/1717261136 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646f36e3090 0x5646f36e8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.731+0000 7fb721ffb640 0 client.16353 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:31:42.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.731+0000 7fb743386740 1 --2- 172.21.15.97:0/1717261136 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646f36d89c0 0x5646f36e8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.731+0000 7fb743386740 1 --2- 172.21.15.97:0/1717261136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646f36d80a0 0x5646f36eb4b0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.731+0000 7fb743386740 1 -- 172.21.15.97:0/1717261136 >> 172.21.15.97:0/1717261136 conn(0x5646f36d43c0 msgr2=0x5646f36dfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:31:42.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.731+0000 7fb743386740 1 -- 172.21.15.97:0/1717261136 shutdown_connections 2024-04-23T10:31:42.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.731+0000 7fb743386740 1 -- 172.21.15.97:0/1717261136 wait complete. 2024-04-23T10:31:42.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.731+0000 7fb743386740 20 client.16353 trim_cache size 0 max 16384 2024-04-23T10:31:42.731 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-23T10:31:42.732 INFO:teuthology.orchestra.run.smithi097.stdout: "epoch": 26, 2024-04-23T10:31:42.732 INFO:teuthology.orchestra.run.smithi097.stdout: "leader": 24488, 2024-04-23T10:31:42.732 INFO:teuthology.orchestra.run.smithi097.stdout: "set_version": 18, 2024-04-23T10:31:42.732 INFO:teuthology.orchestra.run.smithi097.stdout: "sets": { 2024-04-23T10:31:42.732 INFO:teuthology.orchestra.run.smithi097.stdout: "48fb0165": { 2024-04-23T10:31:42.732 INFO:teuthology.orchestra.run.smithi097.stdout: "version": 18, 2024-04-23T10:31:42.732 INFO:teuthology.orchestra.run.smithi097.stdout: "age_ref": 0.0, 2024-04-23T10:31:42.732 INFO:teuthology.orchestra.run.smithi097.stdout: "state": { 2024-04-23T10:31:42.732 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "QUIESCED", 2024-04-23T10:31:42.732 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0 2024-04-23T10:31:42.732 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:31:42.732 INFO:teuthology.orchestra.run.smithi097.stdout: "timeout": 60.0, 2024-04-23T10:31:42.732 INFO:teuthology.orchestra.run.smithi097.stdout: "expiration": 79.5, 2024-04-23T10:31:42.733 INFO:teuthology.orchestra.run.smithi097.stdout: "members": { 2024-04-23T10:31:42.733 INFO:teuthology.orchestra.run.smithi097.stdout: "file:/": { 2024-04-23T10:31:42.733 INFO:teuthology.orchestra.run.smithi097.stdout: "excluded": false, 2024-04-23T10:31:42.733 INFO:teuthology.orchestra.run.smithi097.stdout: "state": { 2024-04-23T10:31:42.733 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "QUIESCED", 2024-04-23T10:31:42.733 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0 2024-04-23T10:31:42.733 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:31:42.733 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:31:42.733 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:31:42.733 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:31:42.733 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:31:42.733 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-23T10:31:42.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.732+0000 7fb740595640 1 -- 172.21.15.97:0/1154369961 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fb7240a3d20 msgr2=0x7fb7240a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:31:42.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.732+0000 7fb740595640 1 --2- 172.21.15.97:0/1154369961 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fb7240a3d20 0x7fb7240a61e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb734031440 tx=0x7fb734005e80 comp rx=0 tx=0).stop 2024-04-23T10:31:42.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.732+0000 7fb740595640 1 -- 172.21.15.97:0/1154369961 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb73810c570 msgr2=0x7fb73819a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:31:42.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.732+0000 7fb740595640 1 --2- 172.21.15.97:0/1154369961 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb73810c570 0x7fb73819a6f0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fb73000b9e0 tx=0x7fb73000bea0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.732+0000 7fb740595640 1 -- 172.21.15.97:0/1154369961 shutdown_connections 2024-04-23T10:31:42.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.732+0000 7fb740595640 1 --2- 172.21.15.97:0/1154369961 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fb7240a3d20 0x7fb7240a61e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.732+0000 7fb740595640 1 --2- 172.21.15.97:0/1154369961 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb73810c570 0x7fb73819a6f0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.732+0000 7fb740595640 1 --2- 172.21.15.97:0/1154369961 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb738103b50 0x7fb7381a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.732+0000 7fb740595640 1 --2- 172.21.15.97:0/1154369961 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb738103230 0x7fb7381a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:42.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.732+0000 7fb740595640 1 -- 172.21.15.97:0/1154369961 >> 172.21.15.97:0/1154369961 conn(0x7fb738077fb0 msgr2=0x7fb7381083e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:31:42.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.732+0000 7fb740595640 1 -- 172.21.15.97:0/1154369961 shutdown_connections 2024-04-23T10:31:42.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:31:42.732+0000 7fb740595640 1 -- 172.21.15.97:0/1154369961 wait complete. 2024-04-23T10:31:42.741 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '48fb0165', quiesce took 0.5 seconds. Will release after: 19.45 2024-04-23T10:31:43.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:31:42 smithi196 ceph-mon[32919]: pgmap v168: 97 pgs: 97 active+clean; 2.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-23T10:31:43.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:31:42 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/729785032' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:31:43.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:31:42 smithi097 ceph-mon[29197]: pgmap v168: 97 pgs: 97 active+clean; 2.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-23T10:31:43.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:31:42 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/729785032' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:31:43.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:31:42 smithi183 ceph-mon[32317]: pgmap v168: 97 pgs: 97 active+clean; 2.4 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-23T10:31:43.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:31:42 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/729785032' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:31:45.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:31:44 smithi196 ceph-mon[32919]: pgmap v169: 97 pgs: 97 active+clean; 2.5 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 52 KiB/s wr, 22 op/s 2024-04-23T10:31:45.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:31:44 smithi097 ceph-mon[29197]: pgmap v169: 97 pgs: 97 active+clean; 2.5 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 52 KiB/s wr, 22 op/s 2024-04-23T10:31:45.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:31:44 smithi183 ceph-mon[32317]: pgmap v169: 97 pgs: 97 active+clean; 2.5 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 52 KiB/s wr, 22 op/s 2024-04-23T10:31:47.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:31:46 smithi196 ceph-mon[32919]: pgmap v170: 97 pgs: 97 active+clean; 2.5 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 12 op/s 2024-04-23T10:31:47.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:31:46 smithi097 ceph-mon[29197]: pgmap v170: 97 pgs: 97 active+clean; 2.5 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 12 op/s 2024-04-23T10:31:47.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:31:46 smithi183 ceph-mon[32317]: pgmap v170: 97 pgs: 97 active+clean; 2.5 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 12 op/s 2024-04-23T10:31:49.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:31:48 smithi196 ceph-mon[32919]: pgmap v171: 97 pgs: 97 active+clean; 2.6 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 20 op/s 2024-04-23T10:31:49.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:31:48 smithi097 ceph-mon[29197]: pgmap v171: 97 pgs: 97 active+clean; 2.6 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 20 op/s 2024-04-23T10:31:49.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:31:48 smithi183 ceph-mon[32317]: pgmap v171: 97 pgs: 97 active+clean; 2.6 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 20 op/s 2024-04-23T10:31:51.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:31:50 smithi196 ceph-mon[32919]: pgmap v172: 97 pgs: 97 active+clean; 2.6 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 13 op/s 2024-04-23T10:31:51.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:31:50 smithi097 ceph-mon[29197]: pgmap v172: 97 pgs: 97 active+clean; 2.6 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 13 op/s 2024-04-23T10:31:51.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:31:50 smithi183 ceph-mon[32317]: pgmap v172: 97 pgs: 97 active+clean; 2.6 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 13 op/s 2024-04-23T10:31:53.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:31:52 smithi196 ceph-mon[32919]: pgmap v173: 97 pgs: 97 active+clean; 2.6 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 13 op/s 2024-04-23T10:31:53.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:31:52 smithi097 ceph-mon[29197]: pgmap v173: 97 pgs: 97 active+clean; 2.6 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 13 op/s 2024-04-23T10:31:53.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:31:52 smithi183 ceph-mon[32317]: pgmap v173: 97 pgs: 97 active+clean; 2.6 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 13 op/s 2024-04-23T10:31:55.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:31:54 smithi196 ceph-mon[32919]: pgmap v174: 97 pgs: 97 active+clean; 2.6 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 23 KiB/s wr, 17 op/s 2024-04-23T10:31:55.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:31:54 smithi097 ceph-mon[29197]: pgmap v174: 97 pgs: 97 active+clean; 2.6 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 23 KiB/s wr, 17 op/s 2024-04-23T10:31:55.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:31:54 smithi183 ceph-mon[32317]: pgmap v174: 97 pgs: 97 active+clean; 2.6 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 23 KiB/s wr, 17 op/s 2024-04-23T10:31:55.456 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.454+0000 7fa5a4937640 1 -- 172.21.15.97:0/4282229025 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5a0101070 msgr2=0x7fa5a0106ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:31:55.456 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.454+0000 7fa5a4937640 1 --2- 172.21.15.97:0/4282229025 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5a0101070 0x7fa5a0106ba0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fa588009900 tx=0x7fa58802a410 comp rx=0 tx=0).stop 2024-04-23T10:31:55.456 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.454+0000 7fa5a4937640 1 -- 172.21.15.97:0/4282229025 shutdown_connections 2024-04-23T10:31:55.456 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.454+0000 7fa5a4937640 1 --2- 172.21.15.97:0/4282229025 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5a0107280 0x7fa5a010ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:55.456 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.454+0000 7fa5a4937640 1 --2- 172.21.15.97:0/4282229025 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5a0101070 0x7fa5a0106ba0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:55.456 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.454+0000 7fa5a4937640 1 --2- 172.21.15.97:0/4282229025 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5a01006c0 0x7fa5a0100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:55.456 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.455+0000 7fa5a4937640 1 -- 172.21.15.97:0/4282229025 >> 172.21.15.97:0/4282229025 conn(0x7fa5a0077390 msgr2=0x7fa5a00fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:31:55.456 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.455+0000 7fa5a4937640 1 -- 172.21.15.97:0/4282229025 shutdown_connections 2024-04-23T10:31:55.456 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.455+0000 7fa5a4937640 1 -- 172.21.15.97:0/4282229025 wait complete. 2024-04-23T10:31:55.456 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.455+0000 7fa5a4937640 1 Processor -- start 2024-04-23T10:31:55.456 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.455+0000 7fa5a4937640 1 -- start start 2024-04-23T10:31:55.456 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.456+0000 7fa5a4937640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5a01006c0 0x7fa5a01986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:55.457 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.456+0000 7fa5a4937640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5a0107280 0x7fa5a0198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:55.457 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.456+0000 7fa5a4937640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5a019cf60 0x7fa5a019d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:55.457 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.456+0000 7fa5a4937640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a01121a0 con 0x7fa5a01006c0 2024-04-23T10:31:55.457 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.456+0000 7fa5a4937640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a0112020 con 0x7fa5a0107280 2024-04-23T10:31:55.457 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.456+0000 7fa5a4937640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a0112320 con 0x7fa5a019cf60 2024-04-23T10:31:55.457 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.456+0000 7fa59e59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5a01006c0 0x7fa5a01986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:55.457 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.456+0000 7fa59e59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5a01006c0 0x7fa5a01986a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43446/0 (socket says 172.21.15.97:43446) 2024-04-23T10:31:55.457 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.456+0000 7fa59e59c640 1 -- 172.21.15.97:0/2320966248 learned_addr learned my addr 172.21.15.97:0/2320966248 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:31:55.457 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.456+0000 7fa597fff640 1 --2- 172.21.15.97:0/2320966248 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5a0107280 0x7fa5a0198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:55.457 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.456+0000 7fa59ed9d640 1 --2- 172.21.15.97:0/2320966248 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5a019cf60 0x7fa5a019d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:55.457 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.456+0000 7fa59e59c640 1 -- 172.21.15.97:0/2320966248 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5a019cf60 msgr2=0x7fa5a019d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:31:55.457 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.456+0000 7fa59e59c640 1 --2- 172.21.15.97:0/2320966248 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5a019cf60 0x7fa5a019d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:55.457 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.456+0000 7fa59e59c640 1 -- 172.21.15.97:0/2320966248 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5a0107280 msgr2=0x7fa5a0198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:31:55.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.456+0000 7fa59e59c640 1 --2- 172.21.15.97:0/2320966248 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5a0107280 0x7fa5a0198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:55.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.456+0000 7fa59e59c640 1 -- 172.21.15.97:0/2320966248 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5a019da50 con 0x7fa5a01006c0 2024-04-23T10:31:55.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.456+0000 7fa59ed9d640 1 --2- 172.21.15.97:0/2320966248 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5a019cf60 0x7fa5a019d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:31:55.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.456+0000 7fa59e59c640 1 --2- 172.21.15.97:0/2320966248 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5a01006c0 0x7fa5a01986a0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fa59000d950 tx=0x7fa59000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:31:55.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.456+0000 7fa5977fe640 1 -- 172.21.15.97:0/2320966248 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa590017070 con 0x7fa5a01006c0 2024-04-23T10:31:55.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.456+0000 7fa5977fe640 1 -- 172.21.15.97:0/2320966248 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa590009c30 con 0x7fa5a01006c0 2024-04-23T10:31:55.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.456+0000 7fa5a4937640 1 -- 172.21.15.97:0/2320966248 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5a019dd40 con 0x7fa5a01006c0 2024-04-23T10:31:55.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.456+0000 7fa5977fe640 1 -- 172.21.15.97:0/2320966248 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59000f620 con 0x7fa5a01006c0 2024-04-23T10:31:55.458 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.456+0000 7fa5a4937640 1 -- 172.21.15.97:0/2320966248 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5a01a5500 con 0x7fa5a01006c0 2024-04-23T10:31:55.465 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.459+0000 7fa5977fe640 1 -- 172.21.15.97:0/2320966248 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa590010460 con 0x7fa5a01006c0 2024-04-23T10:31:55.465 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.459+0000 7fa5a764f740 1 Processor -- start 2024-04-23T10:31:55.465 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.459+0000 7fa5a764f740 1 -- start start 2024-04-23T10:31:55.465 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.459+0000 7fa5977fe640 1 --2- 172.21.15.97:0/2320966248 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fa58c0a3dc0 0x7fa58c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:55.465 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.459+0000 7fa5a764f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564cb8229520 0x564cb8229900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:55.465 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.459+0000 7fa5a764f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564cb8229e40 0x564cb82386d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:55.465 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.459+0000 7fa5a764f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564cb8238c10 0x564cb823b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:55.465 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.460+0000 7fa5a764f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x564cb818e970 con 0x564cb8229e40 2024-04-23T10:31:55.465 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.460+0000 7fa5a764f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564cb812b4a0 con 0x564cb8238c10 2024-04-23T10:31:55.465 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.460+0000 7fa5a764f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564cb818a930 con 0x564cb8229520 2024-04-23T10:31:55.465 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.460+0000 7fa59ed9d640 1 --2- 172.21.15.97:0/2320966248 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fa58c0a3dc0 0x7fa58c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:55.465 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.460+0000 7fa597fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564cb8229e40 0x564cb82386d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:55.466 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.460+0000 7fa59e59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564cb8229520 0x564cb8229900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:55.466 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.460+0000 7fa597fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564cb8229e40 0x564cb82386d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46212/0 (socket says 172.21.15.97:46212) 2024-04-23T10:31:55.466 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.460+0000 7fa597fff640 1 -- 172.21.15.97:0/595045426 learned_addr learned my addr 172.21.15.97:0/595045426 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:31:55.466 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.460+0000 7fa5977fe640 1 -- 172.21.15.97:0/2320966248 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fa5900d03f0 con 0x7fa5a01006c0 2024-04-23T10:31:55.466 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.460+0000 7fa59ed9d640 1 --2- 172.21.15.97:0/595045426 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564cb8238c10 0x564cb823b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:55.466 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.460+0000 7fa597fff640 1 -- 172.21.15.97:0/595045426 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564cb8229520 msgr2=0x564cb8229900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:31:55.466 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.460+0000 7fa597fff640 1 --2- 172.21.15.97:0/595045426 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564cb8229520 0x564cb8229900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:55.466 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.460+0000 7fa597fff640 1 -- 172.21.15.97:0/595045426 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564cb8238c10 msgr2=0x564cb823b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:31:55.466 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.460+0000 7fa597fff640 1 --2- 172.21.15.97:0/595045426 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564cb8238c10 0x564cb823b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:55.466 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.460+0000 7fa597fff640 1 -- 172.21.15.97:0/595045426 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564cb7ea1210 con 0x564cb8229e40 2024-04-23T10:31:55.466 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.460+0000 7fa59ed9d640 1 --2- 172.21.15.97:0/2320966248 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fa58c0a3dc0 0x7fa58c0a6280 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa598005fd0 tx=0x7fa598015a80 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:31:55.466 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.460+0000 7fa597fff640 1 --2- 172.21.15.97:0/595045426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564cb8229e40 0x564cb82386d0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fa588009900 tx=0x7fa588005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f486ed97babb5e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:31:55.466 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.460+0000 7fa594ff9640 1 -- 172.21.15.97:0/595045426 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa588040070 con 0x564cb8229e40 2024-04-23T10:31:55.466 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.460+0000 7fa594ff9640 1 -- 172.21.15.97:0/595045426 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa588004030 con 0x564cb8229e40 2024-04-23T10:31:55.467 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.460+0000 7fa594ff9640 1 -- 172.21.15.97:0/595045426 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa588035040 con 0x564cb8229e40 2024-04-23T10:31:55.467 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.461+0000 7fa5a764f740 1 -- 172.21.15.97:0/595045426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564cb8229e40 msgr2=0x564cb82386d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:31:55.467 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.461+0000 7fa5a764f740 1 --2- 172.21.15.97:0/595045426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564cb8229e40 0x564cb82386d0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fa588009900 tx=0x7fa588005c00 comp rx=0 tx=0).stop 2024-04-23T10:31:55.467 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.461+0000 7fa5a764f740 1 -- 172.21.15.97:0/595045426 shutdown_connections 2024-04-23T10:31:55.467 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.461+0000 7fa5a764f740 1 --2- 172.21.15.97:0/595045426 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564cb8238c10 0x564cb823b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:55.467 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.461+0000 7fa5a764f740 1 --2- 172.21.15.97:0/595045426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564cb8229e40 0x564cb82386d0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:55.467 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.461+0000 7fa5a764f740 1 --2- 172.21.15.97:0/595045426 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564cb8229520 0x564cb8229900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:55.467 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.461+0000 7fa5a764f740 1 -- 172.21.15.97:0/595045426 >> 172.21.15.97:0/595045426 conn(0x564cb8225b30 msgr2=0x564cb8225f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:31:55.467 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.461+0000 7fa5a764f740 1 -- 172.21.15.97:0/595045426 shutdown_connections 2024-04-23T10:31:55.467 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.461+0000 7fa5a764f740 1 -- 172.21.15.97:0/595045426 wait complete. 2024-04-23T10:31:55.467 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.462+0000 7fa5a764f740 1 -- start start 2024-04-23T10:31:55.467 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.462+0000 7fa5a764f740 1 Processor -- start 2024-04-23T10:31:55.467 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.462+0000 7fa5a764f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564cb8229520 0x564cb8238530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:55.467 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.462+0000 7fa5a764f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564cb8229e40 0x564cb8235820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:55.467 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.462+0000 7fa5a764f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564cb8238c10 0x564cb8235d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:55.467 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.462+0000 7fa5a764f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x564cb823b7e0 con 0x564cb8229e40 2024-04-23T10:31:55.468 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.462+0000 7fa5a764f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564cb81290b0 con 0x564cb8229520 2024-04-23T10:31:55.468 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.462+0000 7fa5a764f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564cb823b960 con 0x564cb8238c10 2024-04-23T10:31:55.468 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.462+0000 7fa5a764f740 10 client.0 _ll_register_callbacks cb 0x564cb8116d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:31:55.468 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.462+0000 7fa597fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564cb8229e40 0x564cb8235820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:55.468 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.462+0000 7fa597fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564cb8229e40 0x564cb8235820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46220/0 (socket says 172.21.15.97:46220) 2024-04-23T10:31:55.468 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.462+0000 7fa597fff640 1 -- 172.21.15.97:0/1216753153 learned_addr learned my addr 172.21.15.97:0/1216753153 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:31:55.468 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.462+0000 7fa59ed9d640 1 --2- 172.21.15.97:0/1216753153 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564cb8238c10 0x564cb8235d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:55.468 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.462+0000 7fa59e59c640 1 --2- 172.21.15.97:0/1216753153 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564cb8229520 0x564cb8238530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:55.468 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.462+0000 7fa597fff640 1 -- 172.21.15.97:0/1216753153 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564cb8238c10 msgr2=0x564cb8235d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:31:55.468 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.462+0000 7fa597fff640 1 --2- 172.21.15.97:0/1216753153 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564cb8238c10 0x564cb8235d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:55.468 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.462+0000 7fa597fff640 1 -- 172.21.15.97:0/1216753153 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564cb8229520 msgr2=0x564cb8238530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:31:55.468 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.462+0000 7fa597fff640 1 --2- 172.21.15.97:0/1216753153 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564cb8229520 0x564cb8238530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:55.468 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.462+0000 7fa597fff640 1 -- 172.21.15.97:0/1216753153 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564cb7f6d620 con 0x564cb8229e40 2024-04-23T10:31:55.468 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.462+0000 7fa59e59c640 1 --2- 172.21.15.97:0/1216753153 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564cb8229520 0x564cb8238530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:31:55.469 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.462+0000 7fa597fff640 1 --2- 172.21.15.97:0/1216753153 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564cb8229e40 0x564cb8235820 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fa588009a20 tx=0x7fa588004490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38c5334d5242ff7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:31:55.469 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.463+0000 7fa585ffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:31:55.469 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.463+0000 7fa585ffb640 1 -- 172.21.15.97:0/1216753153 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa588040070 con 0x564cb8229e40 2024-04-23T10:31:55.469 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.463+0000 7fa585ffb640 1 -- 172.21.15.97:0/1216753153 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fa5880318b0 con 0x564cb8229e40 2024-04-23T10:31:55.469 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.463+0000 7fa5a764f740 1 -- 172.21.15.97:0/1216753153 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa588004030 con 0x564cb8229e40 2024-04-23T10:31:55.469 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.463+0000 7fa585ffb640 1 -- 172.21.15.97:0/1216753153 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa588056050 con 0x564cb8229e40 2024-04-23T10:31:55.469 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.463+0000 7fa5a764f740 10 client.16365 fetch_fsmap learned FSMap version 26 2024-04-23T10:31:55.469 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.463+0000 7fa5a764f740 1 -- 172.21.15.97:0/1216753153 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564cb8247740 con 0x564cb8229e40 2024-04-23T10:31:55.469 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.463+0000 7fa585ffb640 1 -- 172.21.15.97:0/1216753153 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa588004030 con 0x564cb8229e40 2024-04-23T10:31:55.469 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.463+0000 7fa585ffb640 1 -- 172.21.15.97:0/1216753153 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa588004cf0 con 0x564cb8229e40 2024-04-23T10:31:55.469 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.463+0000 7fa5a764f740 10 client.16365 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:31:55.469 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.463+0000 7fa5a764f740 10 client.16365 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:31:55.469 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.463+0000 7fa5a764f740 20 client.16365 populate_metadata read hostname 'smithi097' 2024-04-23T10:31:55.469 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.463+0000 7fa5a764f740 1 --2- 172.21.15.97:0/1216753153 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x564cb8248a90 0x564cb824ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:55.469 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.463+0000 7fa5a764f740 4 client.16365 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:31:55.470 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.463+0000 7fa5a764f740 1 -- 172.21.15.97:0/1216753153 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564cb819c390 con 0x564cb8248a90 2024-04-23T10:31:55.470 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.464+0000 7fa59e59c640 1 --2- 172.21.15.97:0/1216753153 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x564cb8248a90 0x564cb824ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:55.470 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.464+0000 7fa59e59c640 1 --2- 172.21.15.97:0/1216753153 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x564cb8248a90 0x564cb824ae80 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1b94eb912dd2626 server_cookie=b77cab2ed5a551fd in_seq=0 out_seq=0 2024-04-23T10:31:55.470 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.464+0000 7fa585ffb640 10 client.16365 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:31:55.470 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.465+0000 7fa585ffb640 1 -- 172.21.15.97:0/1216753153 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564cb819c390 con 0x564cb8248a90 2024-04-23T10:31:55.470 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.465+0000 7fa585ffb640 10 client.16365 handle_command_reply: tid=0 2024-04-23T10:31:55.470 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.465+0000 7fa5a764f740 1 client.16365 shutdown 2024-04-23T10:31:55.470 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.465+0000 7fa5a764f740 1 -- 172.21.15.97:0/1216753153 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564cb8229e40 msgr2=0x564cb8235820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:31:55.470 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.465+0000 7fa5a764f740 1 --2- 172.21.15.97:0/1216753153 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564cb8229e40 0x564cb8235820 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fa588009a20 tx=0x7fa588004490 comp rx=0 tx=0).stop 2024-04-23T10:31:55.470 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.465+0000 7fa5a764f740 1 -- 172.21.15.97:0/1216753153 shutdown_connections 2024-04-23T10:31:55.470 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.465+0000 7fa5a764f740 1 --2- 172.21.15.97:0/1216753153 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x564cb8248a90 0x564cb824ae80 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:55.470 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.465+0000 7fa5a764f740 1 --2- 172.21.15.97:0/1216753153 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564cb8238c10 0x564cb8235d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:55.470 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.465+0000 7fa5a764f740 1 --2- 172.21.15.97:0/1216753153 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564cb8229e40 0x564cb8235820 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:55.470 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.465+0000 7fa585ffb640 0 client.16365 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:31:55.471 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.465+0000 7fa5a764f740 1 --2- 172.21.15.97:0/1216753153 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564cb8229520 0x564cb8238530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:55.471 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.465+0000 7fa5a764f740 1 -- 172.21.15.97:0/1216753153 >> 172.21.15.97:0/1216753153 conn(0x564cb8225b30 msgr2=0x564cb822b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:31:55.471 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.465+0000 7fa5a764f740 1 -- 172.21.15.97:0/1216753153 shutdown_connections 2024-04-23T10:31:55.471 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.465+0000 7fa5a764f740 1 -- 172.21.15.97:0/1216753153 wait complete. 2024-04-23T10:31:55.471 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.465+0000 7fa5a764f740 20 client.16365 trim_cache size 0 max 16384 2024-04-23T10:31:55.484 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.480+0000 7fa5a764f740 1 Processor -- start 2024-04-23T10:31:55.484 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.480+0000 7fa5a764f740 1 -- start start 2024-04-23T10:31:55.484 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.480+0000 7fa5a764f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564cb8229520 0x564cb82446a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:55.484 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.480+0000 7fa5a764f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564cb8229e40 0x564cb8230f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:55.484 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.480+0000 7fa5a764f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564cb8238c10 0x564cb8234800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:55.484 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.480+0000 7fa5a764f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a01ab730 con 0x564cb8229520 2024-04-23T10:31:55.484 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.480+0000 7fa5a764f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a01ad270 con 0x564cb8229e40 2024-04-23T10:31:55.484 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.480+0000 7fa5a764f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a01abe00 con 0x564cb8238c10 2024-04-23T10:31:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.480+0000 7fa59e59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564cb8229520 0x564cb82446a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.480+0000 7fa59e59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564cb8229520 0x564cb82446a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46230/0 (socket says 172.21.15.97:46230) 2024-04-23T10:31:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.480+0000 7fa59e59c640 1 -- 172.21.15.97:0/3539771047 learned_addr learned my addr 172.21.15.97:0/3539771047 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:31:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.481+0000 7fa59ed9d640 1 --2- 172.21.15.97:0/3539771047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564cb8238c10 0x564cb8234800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.481+0000 7fa59e59c640 1 -- 172.21.15.97:0/3539771047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564cb8238c10 msgr2=0x564cb8234800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:31:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.481+0000 7fa59e59c640 1 --2- 172.21.15.97:0/3539771047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564cb8238c10 0x564cb8234800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.481+0000 7fa597fff640 1 --2- 172.21.15.97:0/3539771047 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564cb8229e40 0x564cb8230f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.481+0000 7fa59e59c640 1 -- 172.21.15.97:0/3539771047 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564cb8229e40 msgr2=0x564cb8230f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:31:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.481+0000 7fa59e59c640 1 --2- 172.21.15.97:0/3539771047 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564cb8229e40 0x564cb8230f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.481+0000 7fa59e59c640 1 -- 172.21.15.97:0/3539771047 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564cb822a380 con 0x564cb8229520 2024-04-23T10:31:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.481+0000 7fa597fff640 1 --2- 172.21.15.97:0/3539771047 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564cb8229e40 0x564cb8230f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:31:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.481+0000 7fa59e59c640 1 --2- 172.21.15.97:0/3539771047 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564cb8229520 0x564cb82446a0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fa590098db0 tx=0x7fa5900515d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=245bb91d723f08a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:31:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.481+0000 7fa587fff640 1 -- 172.21.15.97:0/3539771047 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa590020040 con 0x564cb8229520 2024-04-23T10:31:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.481+0000 7fa587fff640 1 -- 172.21.15.97:0/3539771047 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5900a0790 con 0x564cb8229520 2024-04-23T10:31:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.481+0000 7fa587fff640 1 -- 172.21.15.97:0/3539771047 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa590098130 con 0x564cb8229520 2024-04-23T10:31:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.482+0000 7fa5a764f740 1 -- 172.21.15.97:0/3539771047 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564cb8229520 msgr2=0x564cb82446a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:31:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.482+0000 7fa5a764f740 1 --2- 172.21.15.97:0/3539771047 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564cb8229520 0x564cb82446a0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fa590098db0 tx=0x7fa5900515d0 comp rx=0 tx=0).stop 2024-04-23T10:31:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.482+0000 7fa5a764f740 1 -- 172.21.15.97:0/3539771047 shutdown_connections 2024-04-23T10:31:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.482+0000 7fa5a764f740 1 --2- 172.21.15.97:0/3539771047 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564cb8238c10 0x564cb8234800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.482+0000 7fa5a764f740 1 --2- 172.21.15.97:0/3539771047 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564cb8229e40 0x564cb8230f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.482+0000 7fa5a764f740 1 --2- 172.21.15.97:0/3539771047 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564cb8229520 0x564cb82446a0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.482+0000 7fa5a764f740 1 -- 172.21.15.97:0/3539771047 >> 172.21.15.97:0/3539771047 conn(0x564cb8225b30 msgr2=0x564cb82340f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:31:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.482+0000 7fa5a764f740 1 -- 172.21.15.97:0/3539771047 shutdown_connections 2024-04-23T10:31:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.482+0000 7fa5a764f740 1 -- 172.21.15.97:0/3539771047 wait complete. 2024-04-23T10:31:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.482+0000 7fa5a764f740 1 -- start start 2024-04-23T10:31:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.482+0000 7fa5a764f740 1 Processor -- start 2024-04-23T10:31:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.482+0000 7fa5a764f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564cb8229520 0x564cb823be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.482+0000 7fa5a764f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564cb8229e40 0x564cb823c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.482+0000 7fa5a764f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564cb8238c10 0x564cb82a6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.482+0000 7fa5a764f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a01acba0 con 0x564cb8229520 2024-04-23T10:31:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.482+0000 7fa5a764f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a01ac4d0 con 0x564cb8238c10 2024-04-23T10:31:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.482+0000 7fa5a764f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a01a9750 con 0x564cb8229e40 2024-04-23T10:31:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.482+0000 7fa5a764f740 10 client.0 _ll_register_callbacks cb 0x7fa5880095c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:31:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.482+0000 7fa59e59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564cb8229520 0x564cb823be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.482+0000 7fa59e59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564cb8229520 0x564cb823be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46238/0 (socket says 172.21.15.97:46238) 2024-04-23T10:31:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.482+0000 7fa59e59c640 1 -- 172.21.15.97:0/2476964691 learned_addr learned my addr 172.21.15.97:0/2476964691 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:31:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.483+0000 7fa597fff640 1 --2- 172.21.15.97:0/2476964691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564cb8229e40 0x564cb823c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.483+0000 7fa59e59c640 1 -- 172.21.15.97:0/2476964691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564cb8229e40 msgr2=0x564cb823c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:31:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.483+0000 7fa59e59c640 1 --2- 172.21.15.97:0/2476964691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564cb8229e40 0x564cb823c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.483+0000 7fa59e59c640 1 -- 172.21.15.97:0/2476964691 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564cb8238c10 msgr2=0x564cb82a6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:31:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.483+0000 7fa59e59c640 1 --2- 172.21.15.97:0/2476964691 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564cb8238c10 0x564cb82a6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.483+0000 7fa59e59c640 1 -- 172.21.15.97:0/2476964691 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564cb8225960 con 0x564cb8229520 2024-04-23T10:31:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.483+0000 7fa59e59c640 1 --2- 172.21.15.97:0/2476964691 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564cb8229520 0x564cb823be00 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fa590076a80 tx=0x7fa5900a5120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6203e38c52c418c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:31:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.483+0000 7fa585ffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:31:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.483+0000 7fa585ffb640 1 -- 172.21.15.97:0/2476964691 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa590020070 con 0x564cb8229520 2024-04-23T10:31:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.483+0000 7fa585ffb640 1 -- 172.21.15.97:0/2476964691 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fa59009b830 con 0x564cb8229520 2024-04-23T10:31:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.483+0000 7fa5a764f740 1 -- 172.21.15.97:0/2476964691 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5900a0790 con 0x564cb8229520 2024-04-23T10:31:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.483+0000 7fa585ffb640 1 -- 172.21.15.97:0/2476964691 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa59009ba30 con 0x564cb8229520 2024-04-23T10:31:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.483+0000 7fa5a764f740 10 client.16371 fetch_fsmap learned FSMap version 26 2024-04-23T10:31:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.483+0000 7fa5a764f740 1 -- 172.21.15.97:0/2476964691 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564cb82ae3c0 con 0x564cb8229520 2024-04-23T10:31:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.483+0000 7fa585ffb640 1 -- 172.21.15.97:0/2476964691 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5900a0790 con 0x564cb8229520 2024-04-23T10:31:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.483+0000 7fa585ffb640 1 -- 172.21.15.97:0/2476964691 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa590010750 con 0x564cb8229520 2024-04-23T10:31:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.483+0000 7fa5a764f740 10 client.16371 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:31:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.483+0000 7fa5a764f740 10 client.16371 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:31:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.483+0000 7fa5a764f740 20 client.16371 populate_metadata read hostname 'smithi097' 2024-04-23T10:31:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.483+0000 7fa5a764f740 1 --2- 172.21.15.97:0/2476964691 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x564cb82aeae0 0x564cb82b0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:31:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.483+0000 7fa5a764f740 4 client.16371 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:31:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.483+0000 7fa5a764f740 1 -- 172.21.15.97:0/2476964691 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564cb8196200 con 0x564cb82aeae0 2024-04-23T10:31:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.484+0000 7fa597fff640 1 --2- 172.21.15.97:0/2476964691 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x564cb82aeae0 0x564cb82b0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:31:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.484+0000 7fa585ffb640 10 client.16371 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:31:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.484+0000 7fa597fff640 1 --2- 172.21.15.97:0/2476964691 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x564cb82aeae0 0x564cb82b0ed0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=276b79bce45d935e server_cookie=97cd2ea0e02e44cb in_seq=0 out_seq=0 2024-04-23T10:31:55.496 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.496+0000 7fa585ffb640 1 -- 172.21.15.97:0/2476964691 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564cb8196200 con 0x564cb82aeae0 2024-04-23T10:31:55.496 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.496+0000 7fa585ffb640 10 client.16371 handle_command_reply: tid=0 2024-04-23T10:31:55.496 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.496+0000 7fa5a764f740 1 client.16371 shutdown 2024-04-23T10:31:55.496 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.496+0000 7fa5a764f740 1 -- 172.21.15.97:0/2476964691 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564cb8229520 msgr2=0x564cb823be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:31:55.496 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.496+0000 7fa5a764f740 1 --2- 172.21.15.97:0/2476964691 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564cb8229520 0x564cb823be00 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fa590076a80 tx=0x7fa5900a5120 comp rx=0 tx=0).stop 2024-04-23T10:31:55.496 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.496+0000 7fa5a764f740 1 -- 172.21.15.97:0/2476964691 shutdown_connections 2024-04-23T10:31:55.496 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.496+0000 7fa5a764f740 1 --2- 172.21.15.97:0/2476964691 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x564cb82aeae0 0x564cb82b0ed0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:55.496 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.496+0000 7fa5a764f740 1 --2- 172.21.15.97:0/2476964691 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564cb8238c10 0x564cb82a6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:55.496 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.496+0000 7fa5a764f740 1 --2- 172.21.15.97:0/2476964691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564cb8229e40 0x564cb823c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:55.496 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.496+0000 7fa585ffb640 0 client.16371 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:31:55.496 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.496+0000 7fa5a764f740 1 --2- 172.21.15.97:0/2476964691 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564cb8229520 0x564cb823be00 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:55.496 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.496+0000 7fa5a764f740 1 -- 172.21.15.97:0/2476964691 >> 172.21.15.97:0/2476964691 conn(0x564cb8225b30 msgr2=0x564cb8230d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:31:55.496 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.496+0000 7fa5a764f740 1 -- 172.21.15.97:0/2476964691 shutdown_connections 2024-04-23T10:31:55.496 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.496+0000 7fa5a764f740 1 -- 172.21.15.97:0/2476964691 wait complete. 2024-04-23T10:31:55.497 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.496+0000 7fa5a764f740 20 client.16371 trim_cache size 0 max 16384 2024-04-23T10:31:55.497 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:31:55.497 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:31:55.497 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:31:55.497 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:31:55.498 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.497+0000 7fa5a4937640 1 -- 172.21.15.97:0/2320966248 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fa58c0a3dc0 msgr2=0x7fa58c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:31:55.498 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.497+0000 7fa5a4937640 1 --2- 172.21.15.97:0/2320966248 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fa58c0a3dc0 0x7fa58c0a6280 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa598005fd0 tx=0x7fa598015a80 comp rx=0 tx=0).stop 2024-04-23T10:31:55.498 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.497+0000 7fa5a4937640 1 -- 172.21.15.97:0/2320966248 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5a01006c0 msgr2=0x7fa5a01986a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:31:55.498 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.497+0000 7fa5a4937640 1 --2- 172.21.15.97:0/2320966248 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5a01006c0 0x7fa5a01986a0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fa59000d950 tx=0x7fa59000de10 comp rx=0 tx=0).stop 2024-04-23T10:31:55.498 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.497+0000 7fa5a4937640 1 -- 172.21.15.97:0/2320966248 shutdown_connections 2024-04-23T10:31:55.498 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.497+0000 7fa5a4937640 1 --2- 172.21.15.97:0/2320966248 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fa58c0a3dc0 0x7fa58c0a6280 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:55.498 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.497+0000 7fa5a4937640 1 --2- 172.21.15.97:0/2320966248 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5a019cf60 0x7fa5a019d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:55.498 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.497+0000 7fa5a4937640 1 --2- 172.21.15.97:0/2320966248 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5a0107280 0x7fa5a0198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:55.498 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.497+0000 7fa5a4937640 1 --2- 172.21.15.97:0/2320966248 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5a01006c0 0x7fa5a01986a0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:31:55.498 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.497+0000 7fa5a4937640 1 -- 172.21.15.97:0/2320966248 >> 172.21.15.97:0/2320966248 conn(0x7fa5a0077390 msgr2=0x7fa5a0105460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:31:55.498 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.497+0000 7fa5a4937640 1 -- 172.21.15.97:0/2320966248 shutdown_connections 2024-04-23T10:31:55.499 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:31:55.497+0000 7fa5a4937640 1 -- 172.21.15.97:0/2320966248 wait complete. 2024-04-23T10:31:57.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:31:56 smithi196 ceph-mon[32919]: pgmap v175: 97 pgs: 97 active+clean; 2.6 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.4 KiB/s wr, 11 op/s 2024-04-23T10:31:57.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:31:56 smithi097 ceph-mon[29197]: pgmap v175: 97 pgs: 97 active+clean; 2.6 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.4 KiB/s wr, 11 op/s 2024-04-23T10:31:57.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:31:56 smithi183 ceph-mon[32317]: pgmap v175: 97 pgs: 97 active+clean; 2.6 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.4 KiB/s wr, 11 op/s 2024-04-23T10:31:59.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:31:58 smithi196 ceph-mon[32919]: pgmap v176: 97 pgs: 97 active+clean; 2.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-04-23T10:31:59.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:31:58 smithi097 ceph-mon[29197]: pgmap v176: 97 pgs: 97 active+clean; 2.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-04-23T10:31:59.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:31:58 smithi183 ceph-mon[32317]: pgmap v176: 97 pgs: 97 active+clean; 2.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-04-23T10:32:01.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:32:00 smithi196 ceph-mon[32919]: pgmap v177: 97 pgs: 97 active+clean; 2.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.1 KiB/s wr, 11 op/s 2024-04-23T10:32:01.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:32:00 smithi097 ceph-mon[29197]: pgmap v177: 97 pgs: 97 active+clean; 2.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.1 KiB/s wr, 11 op/s 2024-04-23T10:32:01.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:32:00 smithi183 ceph-mon[32317]: pgmap v177: 97 pgs: 97 active+clean; 2.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.1 KiB/s wr, 11 op/s 2024-04-23T10:32:02.192 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T10:32:02.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.304+0000 7fc333920640 1 -- 172.21.15.97:0/1507524190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc32c107300 msgr2=0x7fc32c10eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:02.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.304+0000 7fc333920640 1 --2- 172.21.15.97:0/1507524190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc32c107300 0x7fc32c10eeb0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fc324009ca0 tx=0x7fc32402a8c0 comp rx=0 tx=0).stop 2024-04-23T10:32:02.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.304+0000 7fc333920640 1 -- 172.21.15.97:0/1507524190 shutdown_connections 2024-04-23T10:32:02.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.304+0000 7fc333920640 1 --2- 172.21.15.97:0/1507524190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc32c107300 0x7fc32c10eeb0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:02.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.305+0000 7fc333920640 1 --2- 172.21.15.97:0/1507524190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc32c06b290 0x7fc32c106c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:02.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.305+0000 7fc333920640 1 --2- 172.21.15.97:0/1507524190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc32c06a8e0 0x7fc32c06acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:02.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.305+0000 7fc333920640 1 -- 172.21.15.97:0/1507524190 >> 172.21.15.97:0/1507524190 conn(0x7fc32c077f20 msgr2=0x7fc32c078330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:32:02.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.305+0000 7fc333920640 1 -- 172.21.15.97:0/1507524190 shutdown_connections 2024-04-23T10:32:02.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.305+0000 7fc333920640 1 -- 172.21.15.97:0/1507524190 wait complete. 2024-04-23T10:32:02.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.305+0000 7fc333920640 1 Processor -- start 2024-04-23T10:32:02.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.305+0000 7fc333920640 1 -- start start 2024-04-23T10:32:02.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.305+0000 7fc333920640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc32c06a8e0 0x7fc32c1986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:02.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.305+0000 7fc333920640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc32c107300 0x7fc32c198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:02.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.305+0000 7fc333920640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc32c19cf90 0x7fc32c19d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:02.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.305+0000 7fc333920640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc32c1121f0 con 0x7fc32c06a8e0 2024-04-23T10:32:02.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.305+0000 7fc333920640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc32c112070 con 0x7fc32c107300 2024-04-23T10:32:02.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.305+0000 7fc333920640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc32c112370 con 0x7fc32c19cf90 2024-04-23T10:32:02.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.306+0000 7fc3316bc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc32c06a8e0 0x7fc32c1986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:02.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.306+0000 7fc3316bc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc32c06a8e0 0x7fc32c1986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46256/0 (socket says 172.21.15.97:46256) 2024-04-23T10:32:02.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.306+0000 7fc3316bc640 1 -- 172.21.15.97:0/3797018571 learned_addr learned my addr 172.21.15.97:0/3797018571 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:32:02.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.306+0000 7fc331ebd640 1 --2- 172.21.15.97:0/3797018571 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc32c19cf90 0x7fc32c19d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:02.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.306+0000 7fc330ebb640 1 --2- 172.21.15.97:0/3797018571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc32c107300 0x7fc32c198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:02.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.306+0000 7fc3316bc640 1 -- 172.21.15.97:0/3797018571 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc32c19cf90 msgr2=0x7fc32c19d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:02.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.306+0000 7fc3316bc640 1 --2- 172.21.15.97:0/3797018571 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc32c19cf90 0x7fc32c19d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:02.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.306+0000 7fc3316bc640 1 -- 172.21.15.97:0/3797018571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc32c107300 msgr2=0x7fc32c198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:02.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.306+0000 7fc3316bc640 1 --2- 172.21.15.97:0/3797018571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc32c107300 0x7fc32c198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:02.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.306+0000 7fc3316bc640 1 -- 172.21.15.97:0/3797018571 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc32c19da80 con 0x7fc32c06a8e0 2024-04-23T10:32:02.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.306+0000 7fc330ebb640 1 --2- 172.21.15.97:0/3797018571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc32c107300 0x7fc32c198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:32:02.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.306+0000 7fc3316bc640 1 --2- 172.21.15.97:0/3797018571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc32c06a8e0 0x7fc32c1986d0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fc31c00d950 tx=0x7fc31c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:32:02.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.306+0000 7fc3227fc640 1 -- 172.21.15.97:0/3797018571 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc31c017030 con 0x7fc32c06a8e0 2024-04-23T10:32:02.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.306+0000 7fc3227fc640 1 -- 172.21.15.97:0/3797018571 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc31c004020 con 0x7fc32c06a8e0 2024-04-23T10:32:02.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.306+0000 7fc333920640 1 -- 172.21.15.97:0/3797018571 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc32c19dd70 con 0x7fc32c06a8e0 2024-04-23T10:32:02.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.306+0000 7fc3227fc640 1 -- 172.21.15.97:0/3797018571 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc31c0047a0 con 0x7fc32c06a8e0 2024-04-23T10:32:02.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.306+0000 7fc333920640 1 -- 172.21.15.97:0/3797018571 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc32c1a5560 con 0x7fc32c06a8e0 2024-04-23T10:32:02.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.307+0000 7fc333920640 1 -- 172.21.15.97:0/3797018571 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc32c0691c0 con 0x7fc32c06a8e0 2024-04-23T10:32:02.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.308+0000 7fc3227fc640 1 -- 172.21.15.97:0/3797018571 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc31c009a30 con 0x7fc32c06a8e0 2024-04-23T10:32:02.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.309+0000 7fc3227fc640 1 --2- 172.21.15.97:0/3797018571 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fc3180a3d20 0x7fc3180a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:02.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.309+0000 7fc330ebb640 1 --2- 172.21.15.97:0/3797018571 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fc3180a3d20 0x7fc3180a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:02.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.309+0000 7fc3227fc640 1 -- 172.21.15.97:0/3797018571 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fc31c0d0280 con 0x7fc32c06a8e0 2024-04-23T10:32:02.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.310+0000 7fc330ebb640 1 --2- 172.21.15.97:0/3797018571 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fc3180a3d20 0x7fc3180a61e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc328005fd0 tx=0x7fc328005850 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:32:02.312 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.314+0000 7fc3227fc640 1 -- 172.21.15.97:0/3797018571 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc31c09c970 con 0x7fc32c06a8e0 2024-04-23T10:32:02.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.529+0000 7fc333920640 1 -- 172.21.15.97:0/3797018571 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc32c199a20 con 0x7fc32c06a8e0 2024-04-23T10:32:02.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.530+0000 7fc3227fc640 1 -- 172.21.15.97:0/3797018571 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11931 (secure 0 0 0) 0x7fc31c09c270 con 0x7fc32c06a8e0 2024-04-23T10:32:02.529 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:32:02.529 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3946878762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3946878762},{"type":"v1","addr":"172.21.15.97:6835","nonce":3946878762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/209800861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":209800861},{"type":"v1","addr":"172.21.15.97:6837","nonce":209800861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T10:24:06.910134+0000","modified":"2024-04-23T10:26:09.820985+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6839/418013575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":418013575},{"type":"v1","addr":"172.21.15.97:6839","nonce":418013575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6841/747242680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":747242680},{"type":"v1","addr":"172.21.15.97:6841","nonce":747242680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6833/4292847121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4292847121},{"type":"v1","addr":"172.21.15.183:6833","nonce":4292847121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6833/3885816565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3885816565},{"type":"v1","addr":"172.21.15.196:6833","nonce":3885816565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1897093105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1897093105},{"type":"v1","addr":"172.21.15.183:6835","nonce":1897093105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1171477563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1171477563},{"type":"v1","addr":"172.21.15.196:6835","nonce":1171477563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1898341911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1898341911},{"type":"v1","addr":"172.21.15.183:6837","nonce":1898341911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.196:6837/3094946502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3094946502},{"type":"v1","addr":"172.21.15.196:6837","nonce":3094946502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.196:6839/83843556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":83843556},{"type":"v1","addr":"172.21.15.196:6839","nonce":83843556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6839/1763841130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1763841130},{"type":"v1","addr":"172.21.15.183:6839","nonce":1763841130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24487,24403,24398,24469,24488]},"id":1}]} 2024-04-23T10:32:02.529 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 26 2024-04-23T10:32:02.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.531+0000 7fc333920640 1 -- 172.21.15.97:0/3797018571 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fc3180a3d20 msgr2=0x7fc3180a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:02.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.531+0000 7fc333920640 1 --2- 172.21.15.97:0/3797018571 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fc3180a3d20 0x7fc3180a61e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc328005fd0 tx=0x7fc328005850 comp rx=0 tx=0).stop 2024-04-23T10:32:02.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.532+0000 7fc333920640 1 -- 172.21.15.97:0/3797018571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc32c06a8e0 msgr2=0x7fc32c1986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:02.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.532+0000 7fc333920640 1 --2- 172.21.15.97:0/3797018571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc32c06a8e0 0x7fc32c1986d0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fc31c00d950 tx=0x7fc31c00de10 comp rx=0 tx=0).stop 2024-04-23T10:32:02.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.532+0000 7fc333920640 1 -- 172.21.15.97:0/3797018571 shutdown_connections 2024-04-23T10:32:02.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.532+0000 7fc333920640 1 --2- 172.21.15.97:0/3797018571 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fc3180a3d20 0x7fc3180a61e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:02.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.532+0000 7fc333920640 1 --2- 172.21.15.97:0/3797018571 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc32c19cf90 0x7fc32c19d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:02.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.532+0000 7fc333920640 1 --2- 172.21.15.97:0/3797018571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc32c107300 0x7fc32c198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:02.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.532+0000 7fc333920640 1 --2- 172.21.15.97:0/3797018571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc32c06a8e0 0x7fc32c1986d0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:02.531 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.532+0000 7fc333920640 1 -- 172.21.15.97:0/3797018571 >> 172.21.15.97:0/3797018571 conn(0x7fc32c077f20 msgr2=0x7fc32c0ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:32:02.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.532+0000 7fc333920640 1 -- 172.21.15.97:0/3797018571 shutdown_connections 2024-04-23T10:32:02.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.532+0000 7fc333920640 1 -- 172.21.15.97:0/3797018571 wait complete. 2024-04-23T10:32:02.541 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --set-id 48fb0165 --release --await' 2024-04-23T10:32:02.541 DEBUG:teuthology.orchestra.run.smithi097:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24488 quiesce db --set-id 48fb0165 --release --await 2024-04-23T10:32:02.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.647+0000 7fdff9659640 1 -- 172.21.15.97:0/935555713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdff4101a40 msgr2=0x7fdff4105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:02.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.647+0000 7fdff9659640 1 --2- 172.21.15.97:0/935555713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdff4101a40 0x7fdff4105a80 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fdfe8009900 tx=0x7fdfe802a410 comp rx=0 tx=0).stop 2024-04-23T10:32:02.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.648+0000 7fdff9659640 1 -- 172.21.15.97:0/935555713 shutdown_connections 2024-04-23T10:32:02.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.648+0000 7fdff9659640 1 --2- 172.21.15.97:0/935555713 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdff4106160 0x7fdff410dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:02.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.648+0000 7fdff9659640 1 --2- 172.21.15.97:0/935555713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdff4101a40 0x7fdff4105a80 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:02.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.648+0000 7fdff9659640 1 --2- 172.21.15.97:0/935555713 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdff4101090 0x7fdff4101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:02.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.648+0000 7fdff9659640 1 -- 172.21.15.97:0/935555713 >> 172.21.15.97:0/935555713 conn(0x7fdff4077f50 msgr2=0x7fdff4078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:32:02.649 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.648+0000 7fdff9659640 1 -- 172.21.15.97:0/935555713 shutdown_connections 2024-04-23T10:32:02.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.648+0000 7fdff9659640 1 -- 172.21.15.97:0/935555713 wait complete. 2024-04-23T10:32:02.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.648+0000 7fdff9659640 1 Processor -- start 2024-04-23T10:32:02.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.648+0000 7fdff9659640 1 -- start start 2024-04-23T10:32:02.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.648+0000 7fdff9659640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdff4101090 0x7fdff4198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:02.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.648+0000 7fdff9659640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdff4106160 0x7fdff4198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:02.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.648+0000 7fdff9659640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdff419d000 0x7fdff419d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:02.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.648+0000 7fdff9659640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdff41121a0 con 0x7fdff419d000 2024-04-23T10:32:02.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.648+0000 7fdff9659640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdff4112020 con 0x7fdff4101090 2024-04-23T10:32:02.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.648+0000 7fdff9659640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdff4112320 con 0x7fdff4106160 2024-04-23T10:32:02.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.649+0000 7fdff37fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdff419d000 0x7fdff419d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:02.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.649+0000 7fdff37fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdff419d000 0x7fdff419d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46282/0 (socket says 172.21.15.97:46282) 2024-04-23T10:32:02.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.649+0000 7fdff37fe640 1 -- 172.21.15.97:0/3572407310 learned_addr learned my addr 172.21.15.97:0/3572407310 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:32:02.650 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.649+0000 7fdff27fc640 1 --2- 172.21.15.97:0/3572407310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdff4106160 0x7fdff4198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:02.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.649+0000 7fdff2ffd640 1 --2- 172.21.15.97:0/3572407310 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdff4101090 0x7fdff4198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:02.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.649+0000 7fdff37fe640 1 -- 172.21.15.97:0/3572407310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdff4106160 msgr2=0x7fdff4198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:02.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.649+0000 7fdff37fe640 1 --2- 172.21.15.97:0/3572407310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdff4106160 0x7fdff4198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:02.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.649+0000 7fdff37fe640 1 -- 172.21.15.97:0/3572407310 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdff4101090 msgr2=0x7fdff4198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:02.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.649+0000 7fdff37fe640 1 --2- 172.21.15.97:0/3572407310 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdff4101090 0x7fdff4198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:02.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.649+0000 7fdff37fe640 1 -- 172.21.15.97:0/3572407310 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdff419daf0 con 0x7fdff419d000 2024-04-23T10:32:02.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.649+0000 7fdff27fc640 1 --2- 172.21.15.97:0/3572407310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdff4106160 0x7fdff4198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:32:02.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.649+0000 7fdff37fe640 1 --2- 172.21.15.97:0/3572407310 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdff419d000 0x7fdff419d4b0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fdfec00c970 tx=0x7fdfec00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:32:02.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.649+0000 7fdfdbfff640 1 -- 172.21.15.97:0/3572407310 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfec007bb0 con 0x7fdff419d000 2024-04-23T10:32:02.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.649+0000 7fdff9659640 1 -- 172.21.15.97:0/3572407310 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdff419dde0 con 0x7fdff419d000 2024-04-23T10:32:02.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.649+0000 7fdfdbfff640 1 -- 172.21.15.97:0/3572407310 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfec007d50 con 0x7fdff419d000 2024-04-23T10:32:02.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.649+0000 7fdfdbfff640 1 -- 172.21.15.97:0/3572407310 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfec004e60 con 0x7fdff419d000 2024-04-23T10:32:02.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.649+0000 7fdff9659640 1 -- 172.21.15.97:0/3572407310 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdff41a5550 con 0x7fdff419d000 2024-04-23T10:32:02.658 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.652+0000 7fdfdbfff640 1 -- 172.21.15.97:0/3572407310 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fdfec005020 con 0x7fdff419d000 2024-04-23T10:32:02.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.652+0000 7fdffc36a740 1 Processor -- start 2024-04-23T10:32:02.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.652+0000 7fdffc36a740 1 -- start start 2024-04-23T10:32:02.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.652+0000 7fdffc36a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55837bafe370 0x55837bafe750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:02.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.652+0000 7fdffc36a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55837bafec90 0x55837bb0d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:02.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.652+0000 7fdffc36a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55837bb0d890 0x55837bb0fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:02.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.652+0000 7fdffc36a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55837ba636d0 con 0x55837bafe370 2024-04-23T10:32:02.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.652+0000 7fdffc36a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55837ba001a0 con 0x55837bafec90 2024-04-23T10:32:02.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.652+0000 7fdffc36a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55837ba5f690 con 0x55837bb0d890 2024-04-23T10:32:02.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.652+0000 7fdff27fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55837bafe370 0x55837bafe750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:02.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.652+0000 7fdff27fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55837bafe370 0x55837bafe750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46286/0 (socket says 172.21.15.97:46286) 2024-04-23T10:32:02.659 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.652+0000 7fdff27fc640 1 -- 172.21.15.97:0/1725091496 learned_addr learned my addr 172.21.15.97:0/1725091496 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:32:02.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.652+0000 7fdff2ffd640 1 --2- 172.21.15.97:0/1725091496 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55837bafec90 0x55837bb0d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:02.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.652+0000 7fdfdbfff640 1 --2- 172.21.15.97:0/3572407310 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fdfdc0a3d20 0x7fdfdc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:02.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.653+0000 7fdff27fc640 1 -- 172.21.15.97:0/1725091496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55837bb0d890 msgr2=0x55837bb0fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:32:02.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.653+0000 7fdff27fc640 1 --2- 172.21.15.97:0/1725091496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55837bb0d890 0x55837bb0fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:02.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.653+0000 7fdff27fc640 1 -- 172.21.15.97:0/1725091496 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55837bafec90 msgr2=0x55837bb0d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:32:02.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.653+0000 7fdff27fc640 1 --2- 172.21.15.97:0/1725091496 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55837bafec90 0x55837bb0d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:02.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.653+0000 7fdff37fe640 1 --2- 172.21.15.97:0/3572407310 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fdfdc0a3d20 0x7fdfdc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:02.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.653+0000 7fdff27fc640 1 -- 172.21.15.97:0/1725091496 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55837ba0a5b0 con 0x55837bafe370 2024-04-23T10:32:02.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.653+0000 7fdfdbfff640 1 -- 172.21.15.97:0/3572407310 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fdfec0d04e0 con 0x7fdff419d000 2024-04-23T10:32:02.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.653+0000 7fdff27fc640 1 --2- 172.21.15.97:0/1725091496 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55837bafe370 0x55837bafe750 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fdfe8031440 tx=0x7fdfe8032e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd19b42770a1acca server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:32:02.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.653+0000 7fdfd97fa640 1 -- 172.21.15.97:0/1725091496 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe8035050 con 0x55837bafe370 2024-04-23T10:32:02.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.653+0000 7fdff37fe640 1 --2- 172.21.15.97:0/3572407310 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fdfdc0a3d20 0x7fdfdc0a61e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fdfec0ce280 tx=0x7fdfec06df10 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:32:02.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.653+0000 7fdfd97fa640 1 -- 172.21.15.97:0/1725091496 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfe80316a0 con 0x55837bafe370 2024-04-23T10:32:02.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.653+0000 7fdfd97fa640 1 -- 172.21.15.97:0/1725091496 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe80046c0 con 0x55837bafe370 2024-04-23T10:32:02.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.654+0000 7fdffc36a740 1 -- 172.21.15.97:0/1725091496 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55837bafe370 msgr2=0x55837bafe750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:02.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.654+0000 7fdffc36a740 1 --2- 172.21.15.97:0/1725091496 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55837bafe370 0x55837bafe750 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fdfe8031440 tx=0x7fdfe8032e10 comp rx=0 tx=0).stop 2024-04-23T10:32:02.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.654+0000 7fdffc36a740 1 -- 172.21.15.97:0/1725091496 shutdown_connections 2024-04-23T10:32:02.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.654+0000 7fdffc36a740 1 --2- 172.21.15.97:0/1725091496 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55837bb0d890 0x55837bb0fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:02.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.654+0000 7fdffc36a740 1 --2- 172.21.15.97:0/1725091496 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55837bafec90 0x55837bb0d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:02.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.654+0000 7fdffc36a740 1 --2- 172.21.15.97:0/1725091496 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55837bafe370 0x55837bafe750 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:02.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.654+0000 7fdffc36a740 1 -- 172.21.15.97:0/1725091496 >> 172.21.15.97:0/1725091496 conn(0x55837bafa7b0 msgr2=0x55837bafab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:32:02.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.654+0000 7fdffc36a740 1 -- 172.21.15.97:0/1725091496 shutdown_connections 2024-04-23T10:32:02.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.654+0000 7fdffc36a740 1 -- 172.21.15.97:0/1725091496 wait complete. 2024-04-23T10:32:02.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.654+0000 7fdffc36a740 1 -- start start 2024-04-23T10:32:02.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.654+0000 7fdffc36a740 1 Processor -- start 2024-04-23T10:32:02.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.654+0000 7fdffc36a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55837bafe370 0x55837bb0d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:02.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.654+0000 7fdffc36a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55837bafec90 0x55837bb0a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:02.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.655+0000 7fdffc36a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55837bb0d890 0x55837bb0aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:02.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.655+0000 7fdffc36a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55837bb10710 con 0x55837bafe370 2024-04-23T10:32:02.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.655+0000 7fdffc36a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55837b9fdb20 con 0x55837bafec90 2024-04-23T10:32:02.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.655+0000 7fdffc36a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55837bb10890 con 0x55837bb0d890 2024-04-23T10:32:02.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.655+0000 7fdff27fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55837bafe370 0x55837bb0d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:02.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.655+0000 7fdffc36a740 10 client.0 _ll_register_callbacks cb 0x55837b9fd530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:32:02.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.655+0000 7fdff27fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55837bafe370 0x55837bb0d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46302/0 (socket says 172.21.15.97:46302) 2024-04-23T10:32:02.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.655+0000 7fdff27fc640 1 -- 172.21.15.97:0/175934812 learned_addr learned my addr 172.21.15.97:0/175934812 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:32:02.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.655+0000 7fdff2ffd640 1 --2- 172.21.15.97:0/175934812 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55837bafec90 0x55837bb0a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:02.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.655+0000 7fdff27fc640 1 --2- 172.21.15.97:0/175934812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55837bb0d890 0x55837bb0aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:02.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.655+0000 7fdff27fc640 1 -- 172.21.15.97:0/175934812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55837bb0d890 msgr2=0x55837bb0aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:32:02.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.655+0000 7fdff27fc640 1 --2- 172.21.15.97:0/175934812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55837bb0d890 0x55837bb0aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:02.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.655+0000 7fdff27fc640 1 -- 172.21.15.97:0/175934812 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55837bafec90 msgr2=0x55837bb0a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:32:02.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.655+0000 7fdff27fc640 1 --2- 172.21.15.97:0/175934812 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55837bafec90 0x55837bb0a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:02.662 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.655+0000 7fdff27fc640 1 -- 172.21.15.97:0/175934812 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55837b775920 con 0x55837bafe370 2024-04-23T10:32:02.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.655+0000 7fdff27fc640 1 --2- 172.21.15.97:0/175934812 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55837bafe370 0x55837bb0d240 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fdfe802a450 tx=0x7fdfe8005420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=237642f1c47195fb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:32:02.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.655+0000 7fdfd6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:32:02.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.655+0000 7fdfd6ff5640 1 -- 172.21.15.97:0/175934812 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe8052060 con 0x55837bafe370 2024-04-23T10:32:02.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.655+0000 7fdfd6ff5640 1 -- 172.21.15.97:0/175934812 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fdfe8004030 con 0x55837bafe370 2024-04-23T10:32:02.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.655+0000 7fdffc36a740 1 -- 172.21.15.97:0/175934812 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfe80316a0 con 0x55837bafe370 2024-04-23T10:32:02.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.655+0000 7fdfd6ff5640 1 -- 172.21.15.97:0/175934812 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdfe8035040 con 0x55837bafe370 2024-04-23T10:32:02.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.655+0000 7fdffc36a740 10 client.16389 fetch_fsmap learned FSMap version 26 2024-04-23T10:32:02.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.655+0000 7fdffc36a740 1 -- 172.21.15.97:0/175934812 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55837b841d30 con 0x55837bafe370 2024-04-23T10:32:02.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.655+0000 7fdfd6ff5640 1 -- 172.21.15.97:0/175934812 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfe80316a0 con 0x55837bafe370 2024-04-23T10:32:02.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.656+0000 7fdfd6ff5640 1 -- 172.21.15.97:0/175934812 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fdfe8003aa0 con 0x55837bafe370 2024-04-23T10:32:02.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.656+0000 7fdffc36a740 10 client.16389 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:32:02.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.656+0000 7fdffc36a740 10 client.16389 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:32:02.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.656+0000 7fdffc36a740 20 client.16389 populate_metadata read hostname 'smithi097' 2024-04-23T10:32:02.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.656+0000 7fdffc36a740 1 --2- 172.21.15.97:0/175934812 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55837bb1d8b0 0x55837bb1fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:02.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.656+0000 7fdffc36a740 4 client.16389 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:32:02.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.656+0000 7fdffc36a740 1 -- 172.21.15.97:0/175934812 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55837ba21500 con 0x55837bb1d8b0 2024-04-23T10:32:02.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.656+0000 7fdff2ffd640 1 --2- 172.21.15.97:0/175934812 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55837bb1d8b0 0x55837bb1fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:02.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.656+0000 7fdff2ffd640 1 --2- 172.21.15.97:0/175934812 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55837bb1d8b0 0x55837bb1fca0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98e897b238de997d server_cookie=32e5bdfcc6a219a4 in_seq=0 out_seq=0 2024-04-23T10:32:02.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.656+0000 7fdfd6ff5640 10 client.16389 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:32:02.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.657+0000 7fdfd6ff5640 1 -- 172.21.15.97:0/175934812 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55837ba21500 con 0x55837bb1d8b0 2024-04-23T10:32:02.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.657+0000 7fdfd6ff5640 10 client.16389 handle_command_reply: tid=0 2024-04-23T10:32:02.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.657+0000 7fdffc36a740 1 client.16389 shutdown 2024-04-23T10:32:02.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.657+0000 7fdffc36a740 1 -- 172.21.15.97:0/175934812 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55837bafe370 msgr2=0x55837bb0d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:02.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.657+0000 7fdffc36a740 1 --2- 172.21.15.97:0/175934812 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55837bafe370 0x55837bb0d240 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fdfe802a450 tx=0x7fdfe8005420 comp rx=0 tx=0).stop 2024-04-23T10:32:02.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.658+0000 7fdffc36a740 1 -- 172.21.15.97:0/175934812 shutdown_connections 2024-04-23T10:32:02.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.658+0000 7fdffc36a740 1 --2- 172.21.15.97:0/175934812 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55837bb1d8b0 0x55837bb1fca0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:02.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.658+0000 7fdffc36a740 1 --2- 172.21.15.97:0/175934812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55837bb0d890 0x55837bb0aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:02.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.658+0000 7fdffc36a740 1 --2- 172.21.15.97:0/175934812 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55837bafec90 0x55837bb0a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:02.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.658+0000 7fdffc36a740 1 --2- 172.21.15.97:0/175934812 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55837bafe370 0x55837bb0d240 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:02.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.658+0000 7fdfd6ff5640 0 client.16389 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:32:02.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.658+0000 7fdffc36a740 1 -- 172.21.15.97:0/175934812 >> 172.21.15.97:0/175934812 conn(0x55837bafa7b0 msgr2=0x55837bb00560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:32:02.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.658+0000 7fdffc36a740 1 -- 172.21.15.97:0/175934812 shutdown_connections 2024-04-23T10:32:02.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.658+0000 7fdffc36a740 1 -- 172.21.15.97:0/175934812 wait complete. 2024-04-23T10:32:02.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.658+0000 7fdffc36a740 20 client.16389 trim_cache size 0 max 16384 2024-04-23T10:32:02.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.675+0000 7fdffc36a740 1 Processor -- start 2024-04-23T10:32:02.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.675+0000 7fdffc36a740 1 -- start start 2024-04-23T10:32:02.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.675+0000 7fdffc36a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55837bafe370 0x55837bb0cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:02.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.675+0000 7fdffc36a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55837bafec90 0x55837bb22cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:02.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.675+0000 7fdffc36a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55837bb0d890 0x55837bb23200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:02.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.675+0000 7fdffc36a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdff41ab7a0 con 0x55837bb0d890 2024-04-23T10:32:02.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.675+0000 7fdffc36a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdff41ad2e0 con 0x55837bafe370 2024-04-23T10:32:02.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.675+0000 7fdffc36a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdff41abe70 con 0x55837bafec90 2024-04-23T10:32:02.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.675+0000 7fdff27fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55837bb0d890 0x55837bb23200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:02.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.675+0000 7fdff27fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55837bb0d890 0x55837bb23200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46306/0 (socket says 172.21.15.97:46306) 2024-04-23T10:32:02.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.675+0000 7fdff27fc640 1 -- 172.21.15.97:0/1183214701 learned_addr learned my addr 172.21.15.97:0/1183214701 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:32:02.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.675+0000 7fdff27fc640 1 -- 172.21.15.97:0/1183214701 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55837bafec90 msgr2=0x55837bb22cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:32:02.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.675+0000 7fdff27fc640 1 --2- 172.21.15.97:0/1183214701 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55837bafec90 0x55837bb22cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:02.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.675+0000 7fdff27fc640 1 -- 172.21.15.97:0/1183214701 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55837bafe370 msgr2=0x55837bb0cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:32:02.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.675+0000 7fdff27fc640 1 --2- 172.21.15.97:0/1183214701 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55837bafe370 0x55837bb0cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:02.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.675+0000 7fdff27fc640 1 -- 172.21.15.97:0/1183214701 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55837bb0ae40 con 0x55837bb0d890 2024-04-23T10:32:02.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.675+0000 7fdff27fc640 1 --2- 172.21.15.97:0/1183214701 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55837bb0d890 0x55837bb23200 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fdfe8000c00 tx=0x7fdfe8005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48d9933e653edce4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:32:02.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.675+0000 7fdfd8ff9640 1 -- 172.21.15.97:0/1183214701 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe8049020 con 0x55837bb0d890 2024-04-23T10:32:02.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.675+0000 7fdfd8ff9640 1 -- 172.21.15.97:0/1183214701 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfe803a660 con 0x55837bb0d890 2024-04-23T10:32:02.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.675+0000 7fdfd8ff9640 1 -- 172.21.15.97:0/1183214701 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe8003ab0 con 0x55837bb0d890 2024-04-23T10:32:02.679 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.676+0000 7fdffc36a740 1 -- 172.21.15.97:0/1183214701 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55837bb0d890 msgr2=0x55837bb23200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:02.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.676+0000 7fdffc36a740 1 --2- 172.21.15.97:0/1183214701 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55837bb0d890 0x55837bb23200 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fdfe8000c00 tx=0x7fdfe8005c90 comp rx=0 tx=0).stop 2024-04-23T10:32:02.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.676+0000 7fdffc36a740 1 -- 172.21.15.97:0/1183214701 shutdown_connections 2024-04-23T10:32:02.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.676+0000 7fdffc36a740 1 --2- 172.21.15.97:0/1183214701 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55837bb0d890 0x55837bb23200 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:02.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.676+0000 7fdffc36a740 1 --2- 172.21.15.97:0/1183214701 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55837bafec90 0x55837bb22cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:02.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.676+0000 7fdffc36a740 1 --2- 172.21.15.97:0/1183214701 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55837bafe370 0x55837bb0cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:02.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.676+0000 7fdffc36a740 1 -- 172.21.15.97:0/1183214701 >> 172.21.15.97:0/1183214701 conn(0x55837bafa7b0 msgr2=0x55837bb08e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:32:02.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.676+0000 7fdffc36a740 1 -- 172.21.15.97:0/1183214701 shutdown_connections 2024-04-23T10:32:02.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.676+0000 7fdffc36a740 1 -- 172.21.15.97:0/1183214701 wait complete. 2024-04-23T10:32:02.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.676+0000 7fdffc36a740 1 -- start start 2024-04-23T10:32:02.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.676+0000 7fdffc36a740 1 Processor -- start 2024-04-23T10:32:02.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.676+0000 7fdffc36a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55837bafe370 0x55837bb22cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:02.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.676+0000 7fdffc36a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55837bafec90 0x55837bb23200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:02.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.676+0000 7fdffc36a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55837bb0d890 0x55837bb1f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:02.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.676+0000 7fdffc36a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdff41acc10 con 0x55837bafec90 2024-04-23T10:32:02.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.676+0000 7fdffc36a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdff41ac540 con 0x55837bafe370 2024-04-23T10:32:02.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.676+0000 7fdffc36a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdff41a97a0 con 0x55837bb0d890 2024-04-23T10:32:02.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.677+0000 7fdffc36a740 10 client.0 _ll_register_callbacks cb 0x7fdfe803e920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:32:02.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.677+0000 7fdff2ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55837bafec90 0x55837bb23200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:02.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.677+0000 7fdff2ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55837bafec90 0x55837bb23200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46320/0 (socket says 172.21.15.97:46320) 2024-04-23T10:32:02.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.677+0000 7fdff2ffd640 1 -- 172.21.15.97:0/2283518921 learned_addr learned my addr 172.21.15.97:0/2283518921 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:32:02.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.677+0000 7fdff27fc640 1 --2- 172.21.15.97:0/2283518921 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55837bb0d890 0x55837bb1f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:02.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.677+0000 7fdff27fc640 1 --2- 172.21.15.97:0/2283518921 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55837bafe370 0x55837bb22cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:02.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.677+0000 7fdff2ffd640 1 -- 172.21.15.97:0/2283518921 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55837bb0d890 msgr2=0x55837bb1f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:32:02.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.677+0000 7fdff2ffd640 1 --2- 172.21.15.97:0/2283518921 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55837bb0d890 0x55837bb1f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:02.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.677+0000 7fdff2ffd640 1 -- 172.21.15.97:0/2283518921 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55837bafe370 msgr2=0x55837bb22cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:32:02.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.677+0000 7fdff2ffd640 1 --2- 172.21.15.97:0/2283518921 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55837bafe370 0x55837bb22cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:02.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.677+0000 7fdff2ffd640 1 -- 172.21.15.97:0/2283518921 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55837bb1fa90 con 0x55837bafec90 2024-04-23T10:32:02.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.677+0000 7fdff2ffd640 1 --2- 172.21.15.97:0/2283518921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55837bafec90 0x55837bb23200 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4003a80 tx=0x7fdfe40128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ea6939d19b2b24f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:32:02.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.677+0000 7fdfd6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:32:02.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.677+0000 7fdfd6ff5640 1 -- 172.21.15.97:0/2283518921 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe401d880 con 0x55837bafec90 2024-04-23T10:32:02.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.677+0000 7fdfd6ff5640 1 -- 172.21.15.97:0/2283518921 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fdfe40113e0 con 0x55837bafec90 2024-04-23T10:32:02.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.677+0000 7fdffc36a740 1 -- 172.21.15.97:0/2283518921 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfe803a660 con 0x55837bafec90 2024-04-23T10:32:02.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.677+0000 7fdfd6ff5640 1 -- 172.21.15.97:0/2283518921 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdfe402d050 con 0x55837bafec90 2024-04-23T10:32:02.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.677+0000 7fdffc36a740 10 client.16395 fetch_fsmap learned FSMap version 26 2024-04-23T10:32:02.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.677+0000 7fdffc36a740 1 -- 172.21.15.97:0/2283518921 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55837bb847c0 con 0x55837bafec90 2024-04-23T10:32:02.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.677+0000 7fdfd6ff5640 1 -- 172.21.15.97:0/2283518921 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfe803a660 con 0x55837bafec90 2024-04-23T10:32:02.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.678+0000 7fdfd6ff5640 1 -- 172.21.15.97:0/2283518921 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fdfe400f040 con 0x55837bafec90 2024-04-23T10:32:02.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.678+0000 7fdffc36a740 10 client.16395 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:32:02.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.678+0000 7fdffc36a740 10 client.16395 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:32:02.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.678+0000 7fdffc36a740 20 client.16395 populate_metadata read hostname 'smithi097' 2024-04-23T10:32:02.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.678+0000 7fdffc36a740 1 --2- 172.21.15.97:0/2283518921 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55837bb84ec0 0x55837bb872e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:02.682 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.678+0000 7fdffc36a740 4 client.16395 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "48fb0165", "release": true, "await": true}] 2024-04-23T10:32:02.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.678+0000 7fdffc36a740 1 -- 172.21.15.97:0/2283518921 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "quiesce db", "set_id": "48fb0165", "release": true, "await": true}) -- 0x55837ba710f0 con 0x55837bb84ec0 2024-04-23T10:32:02.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.678+0000 7fdff27fc640 1 --2- 172.21.15.97:0/2283518921 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55837bb84ec0 0x55837bb872e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:02.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.678+0000 7fdfd6ff5640 10 client.16395 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:32:02.683 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.678+0000 7fdff27fc640 1 --2- 172.21.15.97:0/2283518921 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55837bb84ec0 0x55837bb872e0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19845f8c9d292196 server_cookie=f76c0bf5cf5e270a in_seq=0 out_seq=0 2024-04-23T10:32:02.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.681+0000 7fdfd6ff5640 1 -- 172.21.15.97:0/2283518921 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55837ba710f0 con 0x55837bb84ec0 2024-04-23T10:32:02.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.681+0000 7fdfd6ff5640 10 client.16395 handle_command_reply: tid=0 2024-04-23T10:32:02.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.681+0000 7fdffc36a740 1 client.16395 shutdown 2024-04-23T10:32:02.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.681+0000 7fdffc36a740 1 -- 172.21.15.97:0/2283518921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55837bafec90 msgr2=0x55837bb23200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:02.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.681+0000 7fdffc36a740 1 --2- 172.21.15.97:0/2283518921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55837bafec90 0x55837bb23200 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4003a80 tx=0x7fdfe40128e0 comp rx=0 tx=0).stop 2024-04-23T10:32:02.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.681+0000 7fdffc36a740 1 -- 172.21.15.97:0/2283518921 shutdown_connections 2024-04-23T10:32:02.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.682+0000 7fdffc36a740 1 --2- 172.21.15.97:0/2283518921 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55837bb84ec0 0x55837bb872e0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:02.684 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.682+0000 7fdffc36a740 1 --2- 172.21.15.97:0/2283518921 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55837bb0d890 0x55837bb1f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:02.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.682+0000 7fdffc36a740 1 --2- 172.21.15.97:0/2283518921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55837bafec90 0x55837bb23200 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:02.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.682+0000 7fdfd6ff5640 0 client.16395 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:32:02.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.682+0000 7fdffc36a740 1 --2- 172.21.15.97:0/2283518921 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55837bafe370 0x55837bb22cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:02.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.682+0000 7fdffc36a740 1 -- 172.21.15.97:0/2283518921 >> 172.21.15.97:0/2283518921 conn(0x55837bafa7b0 msgr2=0x55837baf9140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:32:02.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.682+0000 7fdffc36a740 1 -- 172.21.15.97:0/2283518921 shutdown_connections 2024-04-23T10:32:02.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.682+0000 7fdffc36a740 1 -- 172.21.15.97:0/2283518921 wait complete. 2024-04-23T10:32:02.685 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.682+0000 7fdffc36a740 20 client.16395 trim_cache size 0 max 16384 2024-04-23T10:32:02.685 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-23T10:32:02.685 INFO:teuthology.orchestra.run.smithi097.stdout: "epoch": 26, 2024-04-23T10:32:02.685 INFO:teuthology.orchestra.run.smithi097.stdout: "leader": 24488, 2024-04-23T10:32:02.686 INFO:teuthology.orchestra.run.smithi097.stdout: "set_version": 20, 2024-04-23T10:32:02.686 INFO:teuthology.orchestra.run.smithi097.stdout: "sets": { 2024-04-23T10:32:02.686 INFO:teuthology.orchestra.run.smithi097.stdout: "48fb0165": { 2024-04-23T10:32:02.686 INFO:teuthology.orchestra.run.smithi097.stdout: "version": 20, 2024-04-23T10:32:02.686 INFO:teuthology.orchestra.run.smithi097.stdout: "age_ref": 0.0, 2024-04-23T10:32:02.686 INFO:teuthology.orchestra.run.smithi097.stdout: "state": { 2024-04-23T10:32:02.686 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RELEASED", 2024-04-23T10:32:02.686 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0 2024-04-23T10:32:02.686 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:32:02.686 INFO:teuthology.orchestra.run.smithi097.stdout: "timeout": 60.0, 2024-04-23T10:32:02.686 INFO:teuthology.orchestra.run.smithi097.stdout: "expiration": 79.5, 2024-04-23T10:32:02.686 INFO:teuthology.orchestra.run.smithi097.stdout: "members": { 2024-04-23T10:32:02.686 INFO:teuthology.orchestra.run.smithi097.stdout: "file:/": { 2024-04-23T10:32:02.686 INFO:teuthology.orchestra.run.smithi097.stdout: "excluded": false, 2024-04-23T10:32:02.687 INFO:teuthology.orchestra.run.smithi097.stdout: "state": { 2024-04-23T10:32:02.687 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RELEASED", 2024-04-23T10:32:02.687 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0 2024-04-23T10:32:02.687 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:32:02.687 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:32:02.687 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:32:02.687 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:32:02.687 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:32:02.687 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-23T10:32:02.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.682+0000 7fdff9659640 1 -- 172.21.15.97:0/3572407310 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fdfdc0a3d20 msgr2=0x7fdfdc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:02.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.682+0000 7fdff9659640 1 --2- 172.21.15.97:0/3572407310 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fdfdc0a3d20 0x7fdfdc0a61e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fdfec0ce280 tx=0x7fdfec06df10 comp rx=0 tx=0).stop 2024-04-23T10:32:02.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.682+0000 7fdff9659640 1 -- 172.21.15.97:0/3572407310 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdff419d000 msgr2=0x7fdff419d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:02.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.682+0000 7fdff9659640 1 --2- 172.21.15.97:0/3572407310 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdff419d000 0x7fdff419d4b0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fdfec00c970 tx=0x7fdfec00ce30 comp rx=0 tx=0).stop 2024-04-23T10:32:02.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.682+0000 7fdff9659640 1 -- 172.21.15.97:0/3572407310 shutdown_connections 2024-04-23T10:32:02.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.682+0000 7fdff9659640 1 --2- 172.21.15.97:0/3572407310 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fdfdc0a3d20 0x7fdfdc0a61e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:02.688 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.682+0000 7fdff9659640 1 --2- 172.21.15.97:0/3572407310 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdff419d000 0x7fdff419d4b0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:02.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.682+0000 7fdff9659640 1 --2- 172.21.15.97:0/3572407310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdff4106160 0x7fdff4198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:02.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.682+0000 7fdff9659640 1 --2- 172.21.15.97:0/3572407310 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdff4101090 0x7fdff4198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:02.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.682+0000 7fdff9659640 1 -- 172.21.15.97:0/3572407310 >> 172.21.15.97:0/3572407310 conn(0x7fdff4077f50 msgr2=0x7fdff40ff120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:32:02.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.682+0000 7fdff9659640 1 -- 172.21.15.97:0/3572407310 shutdown_connections 2024-04-23T10:32:02.689 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.682+0000 7fdff9659640 1 -- 172.21.15.97:0/3572407310 wait complete. 2024-04-23T10:32:02.692 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '48fb0165', seconds elapsed: 20.5 2024-04-23T10:32:02.692 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.45 2024-04-23T10:32:02.692 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T10:32:02.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.791+0000 7f1595108640 1 -- 172.21.15.97:0/1260292842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f159010c580 msgr2=0x7f159010e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:02.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.791+0000 7f1595108640 1 --2- 172.21.15.97:0/1260292842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f159010c580 0x7f159010e9b0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f1588009ca0 tx=0x7f158802a950 comp rx=0 tx=0).stop 2024-04-23T10:32:02.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.791+0000 7f1595108640 1 -- 172.21.15.97:0/1260292842 shutdown_connections 2024-04-23T10:32:02.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.791+0000 7f1595108640 1 --2- 172.21.15.97:0/1260292842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f159010c580 0x7f159010e9b0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:02.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.791+0000 7f1595108640 1 --2- 172.21.15.97:0/1260292842 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1590103ad0 0x7f159010bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:02.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.791+0000 7f1595108640 1 --2- 172.21.15.97:0/1260292842 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f15901031b0 0x7f1590103590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:02.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.791+0000 7f1595108640 1 -- 172.21.15.97:0/1260292842 >> 172.21.15.97:0/1260292842 conn(0x7f15900771d0 msgr2=0x7f15900775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:32:02.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.791+0000 7f1595108640 1 -- 172.21.15.97:0/1260292842 shutdown_connections 2024-04-23T10:32:02.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.791+0000 7f1595108640 1 -- 172.21.15.97:0/1260292842 wait complete. 2024-04-23T10:32:02.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.792+0000 7f1595108640 1 Processor -- start 2024-04-23T10:32:02.793 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.792+0000 7f1595108640 1 -- start start 2024-04-23T10:32:02.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.792+0000 7f1595108640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15901031b0 0x7f1590198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:02.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.792+0000 7f1595108640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1590103ad0 0x7f1590198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:02.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.792+0000 7f1595108640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f159010c580 0x7f159019cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:02.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.792+0000 7f1595108640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1590112180 con 0x7f159010c580 2024-04-23T10:32:02.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.792+0000 7f1595108640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1590112000 con 0x7f15901031b0 2024-04-23T10:32:02.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.792+0000 7f1595108640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1590112300 con 0x7f1590103ad0 2024-04-23T10:32:02.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.793+0000 7f158f59e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f159010c580 0x7f159019cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:02.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.793+0000 7f158f59e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f159010c580 0x7f159019cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46342/0 (socket says 172.21.15.97:46342) 2024-04-23T10:32:02.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.793+0000 7f158f59e640 1 -- 172.21.15.97:0/2397955236 learned_addr learned my addr 172.21.15.97:0/2397955236 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:32:02.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.793+0000 7f158ed9d640 1 --2- 172.21.15.97:0/2397955236 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15901031b0 0x7f1590198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:02.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.793+0000 7f158e59c640 1 --2- 172.21.15.97:0/2397955236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1590103ad0 0x7f1590198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:02.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.793+0000 7f158f59e640 1 -- 172.21.15.97:0/2397955236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1590103ad0 msgr2=0x7f1590198ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:02.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.793+0000 7f158f59e640 1 --2- 172.21.15.97:0/2397955236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1590103ad0 0x7f1590198ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:02.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.793+0000 7f158f59e640 1 -- 172.21.15.97:0/2397955236 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15901031b0 msgr2=0x7f1590198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:02.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.793+0000 7f158f59e640 1 --2- 172.21.15.97:0/2397955236 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15901031b0 0x7f1590198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:02.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.793+0000 7f158f59e640 1 -- 172.21.15.97:0/2397955236 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f159019d650 con 0x7f159010c580 2024-04-23T10:32:02.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.793+0000 7f158f59e640 1 --2- 172.21.15.97:0/2397955236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f159010c580 0x7f159019cf20 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f158802ae70 tx=0x7f158802f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:32:02.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.793+0000 7f1577fff640 1 -- 172.21.15.97:0/2397955236 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1588034040 con 0x7f159010c580 2024-04-23T10:32:02.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.793+0000 7f1595108640 1 -- 172.21.15.97:0/2397955236 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f159019d8e0 con 0x7f159010c580 2024-04-23T10:32:02.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.793+0000 7f1577fff640 1 -- 172.21.15.97:0/2397955236 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15880308f0 con 0x7f159010c580 2024-04-23T10:32:02.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.793+0000 7f1595108640 1 -- 172.21.15.97:0/2397955236 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15901a51d0 con 0x7f159010c580 2024-04-23T10:32:02.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.793+0000 7f1577fff640 1 -- 172.21.15.97:0/2397955236 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1588004690 con 0x7f159010c580 2024-04-23T10:32:02.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.794+0000 7f1595108640 1 -- 172.21.15.97:0/2397955236 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15900691c0 con 0x7f159010c580 2024-04-23T10:32:02.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.795+0000 7f1577fff640 1 -- 172.21.15.97:0/2397955236 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1588037020 con 0x7f159010c580 2024-04-23T10:32:02.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.796+0000 7f1577fff640 1 --2- 172.21.15.97:0/2397955236 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f15780a3d20 0x7f15780a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:02.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.796+0000 7f158ed9d640 1 --2- 172.21.15.97:0/2397955236 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f15780a3d20 0x7f15780a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:02.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.796+0000 7f1577fff640 1 -- 172.21.15.97:0/2397955236 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f15880f16e0 con 0x7f159010c580 2024-04-23T10:32:02.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.797+0000 7f158ed9d640 1 --2- 172.21.15.97:0/2397955236 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f15780a3d20 0x7f15780a61e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f158000abe0 tx=0x7f158000a2d0 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:32:02.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:02.800+0000 7f1577fff640 1 -- 172.21.15.97:0/2397955236 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1588080ff0 con 0x7f159010c580 2024-04-23T10:32:03.021 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.022+0000 7f1595108640 1 -- 172.21.15.97:0/2397955236 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f15901076d0 con 0x7f159010c580 2024-04-23T10:32:03.022 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.023+0000 7f1577fff640 1 -- 172.21.15.97:0/2397955236 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11931 (secure 0 0 0) 0x7f15880bdee0 con 0x7f159010c580 2024-04-23T10:32:03.022 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:32:03.022 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3946878762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3946878762},{"type":"v1","addr":"172.21.15.97:6835","nonce":3946878762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/209800861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":209800861},{"type":"v1","addr":"172.21.15.97:6837","nonce":209800861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T10:24:06.910134+0000","modified":"2024-04-23T10:26:09.820985+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6839/418013575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":418013575},{"type":"v1","addr":"172.21.15.97:6839","nonce":418013575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6841/747242680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":747242680},{"type":"v1","addr":"172.21.15.97:6841","nonce":747242680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6833/4292847121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4292847121},{"type":"v1","addr":"172.21.15.183:6833","nonce":4292847121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6833/3885816565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3885816565},{"type":"v1","addr":"172.21.15.196:6833","nonce":3885816565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1897093105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1897093105},{"type":"v1","addr":"172.21.15.183:6835","nonce":1897093105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1171477563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1171477563},{"type":"v1","addr":"172.21.15.196:6835","nonce":1171477563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1898341911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1898341911},{"type":"v1","addr":"172.21.15.183:6837","nonce":1898341911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.196:6837/3094946502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3094946502},{"type":"v1","addr":"172.21.15.196:6837","nonce":3094946502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.196:6839/83843556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":83843556},{"type":"v1","addr":"172.21.15.196:6839","nonce":83843556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6839/1763841130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1763841130},{"type":"v1","addr":"172.21.15.183:6839","nonce":1763841130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24487,24403,24398,24469,24488]},"id":1}]} 2024-04-23T10:32:03.022 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 26 2024-04-23T10:32:03.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.025+0000 7f1595108640 1 -- 172.21.15.97:0/2397955236 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f15780a3d20 msgr2=0x7f15780a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:03.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.025+0000 7f1595108640 1 --2- 172.21.15.97:0/2397955236 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f15780a3d20 0x7f15780a61e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f158000abe0 tx=0x7f158000a2d0 comp rx=0 tx=0).stop 2024-04-23T10:32:03.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.025+0000 7f1595108640 1 -- 172.21.15.97:0/2397955236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f159010c580 msgr2=0x7f159019cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:03.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.025+0000 7f1595108640 1 --2- 172.21.15.97:0/2397955236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f159010c580 0x7f159019cf20 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f158802ae70 tx=0x7f158802f5d0 comp rx=0 tx=0).stop 2024-04-23T10:32:03.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.025+0000 7f1595108640 1 -- 172.21.15.97:0/2397955236 shutdown_connections 2024-04-23T10:32:03.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.025+0000 7f1595108640 1 --2- 172.21.15.97:0/2397955236 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f15780a3d20 0x7f15780a61e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:03.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.025+0000 7f1595108640 1 --2- 172.21.15.97:0/2397955236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f159010c580 0x7f159019cf20 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:03.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.025+0000 7f1595108640 1 --2- 172.21.15.97:0/2397955236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1590103ad0 0x7f1590198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:03.024 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.025+0000 7f1595108640 1 --2- 172.21.15.97:0/2397955236 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15901031b0 0x7f1590198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:03.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.025+0000 7f1595108640 1 -- 172.21.15.97:0/2397955236 >> 172.21.15.97:0/2397955236 conn(0x7f15900771d0 msgr2=0x7f15900fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:32:03.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.025+0000 7f1595108640 1 -- 172.21.15.97:0/2397955236 shutdown_connections 2024-04-23T10:32:03.025 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.025+0000 7f1595108640 1 -- 172.21.15.97:0/2397955236 wait complete. 2024-04-23T10:32:03.035 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db / --timeout 60 --expiration 79.45 --await-for 42.0' 2024-04-23T10:32:03.035 DEBUG:teuthology.orchestra.run.smithi097:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24488 quiesce db / --timeout 60 --expiration 79.45 --await-for 42.0 2024-04-23T10:32:03.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.144+0000 7f434e3cc640 1 -- 172.21.15.97:0/2003567303 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4348101110 msgr2=0x7f43481014f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:03.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.144+0000 7f434e3cc640 1 --2- 172.21.15.97:0/2003567303 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4348101110 0x7f43481014f0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f4338009900 tx=0x7f433802a410 comp rx=0 tx=0).stop 2024-04-23T10:32:03.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.144+0000 7f434e3cc640 1 -- 172.21.15.97:0/2003567303 shutdown_connections 2024-04-23T10:32:03.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.144+0000 7f434e3cc640 1 --2- 172.21.15.97:0/2003567303 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4348106190 0x7f434810dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:03.145 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.144+0000 7f434e3cc640 1 --2- 172.21.15.97:0/2003567303 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4348101ac0 0x7f4348105ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:03.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.144+0000 7f434e3cc640 1 --2- 172.21.15.97:0/2003567303 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4348101110 0x7f43481014f0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:03.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.144+0000 7f434e3cc640 1 -- 172.21.15.97:0/2003567303 >> 172.21.15.97:0/2003567303 conn(0x7f4348077fb0 msgr2=0x7f43480783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:32:03.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.144+0000 7f434e3cc640 1 -- 172.21.15.97:0/2003567303 shutdown_connections 2024-04-23T10:32:03.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.144+0000 7f434e3cc640 1 -- 172.21.15.97:0/2003567303 wait complete. 2024-04-23T10:32:03.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.145+0000 7f434e3cc640 1 Processor -- start 2024-04-23T10:32:03.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.145+0000 7f434e3cc640 1 -- start start 2024-04-23T10:32:03.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.145+0000 7f434e3cc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4348101110 0x7f4348199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:03.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.145+0000 7f434e3cc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4348106190 0x7f4348199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:03.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.145+0000 7f434e3cc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f434819a1f0 0x7f43481a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:03.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.145+0000 7f434e3cc640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f43481111d0 con 0x7f4348101110 2024-04-23T10:32:03.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.145+0000 7f434e3cc640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4348111050 con 0x7f434819a1f0 2024-04-23T10:32:03.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.145+0000 7f434e3cc640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4348111350 con 0x7f4348106190 2024-04-23T10:32:03.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.145+0000 7f4347fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4348101110 0x7f4348199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:03.146 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.145+0000 7f4347fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4348101110 0x7f4348199770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46352/0 (socket says 172.21.15.97:46352) 2024-04-23T10:32:03.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.145+0000 7f4347fff640 1 -- 172.21.15.97:0/3285476600 learned_addr learned my addr 172.21.15.97:0/3285476600 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:32:03.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.145+0000 7f43477fe640 1 --2- 172.21.15.97:0/3285476600 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4348106190 0x7f4348199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:03.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.145+0000 7f434c969640 1 --2- 172.21.15.97:0/3285476600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f434819a1f0 0x7f43481a47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:03.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.145+0000 7f4347fff640 1 -- 172.21.15.97:0/3285476600 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4348106190 msgr2=0x7f4348199cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:03.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.145+0000 7f4347fff640 1 --2- 172.21.15.97:0/3285476600 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4348106190 0x7f4348199cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:03.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.145+0000 7f4347fff640 1 -- 172.21.15.97:0/3285476600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f434819a1f0 msgr2=0x7f43481a47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:03.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.145+0000 7f4347fff640 1 --2- 172.21.15.97:0/3285476600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f434819a1f0 0x7f43481a47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:03.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.145+0000 7f4347fff640 1 -- 172.21.15.97:0/3285476600 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43481a4f70 con 0x7f4348101110 2024-04-23T10:32:03.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.145+0000 7f43477fe640 1 --2- 172.21.15.97:0/3285476600 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4348106190 0x7f4348199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:32:03.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.145+0000 7f434c969640 1 --2- 172.21.15.97:0/3285476600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f434819a1f0 0x7f43481a47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:32:03.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.146+0000 7f4347fff640 1 --2- 172.21.15.97:0/3285476600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4348101110 0x7f4348199770 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f4338006010 tx=0x7f43380042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:32:03.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.146+0000 7f43457fa640 1 -- 172.21.15.97:0/3285476600 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433803f030 con 0x7f4348101110 2024-04-23T10:32:03.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.146+0000 7f43457fa640 1 -- 172.21.15.97:0/3285476600 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4338003800 con 0x7f4348101110 2024-04-23T10:32:03.147 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.146+0000 7f434e3cc640 1 -- 172.21.15.97:0/3285476600 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43481a5200 con 0x7f4348101110 2024-04-23T10:32:03.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.146+0000 7f43457fa640 1 -- 172.21.15.97:0/3285476600 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4338035070 con 0x7f4348101110 2024-04-23T10:32:03.148 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.146+0000 7f434e3cc640 1 -- 172.21.15.97:0/3285476600 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43481a56f0 con 0x7f4348101110 2024-04-23T10:32:03.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.148+0000 7f43457fa640 1 -- 172.21.15.97:0/3285476600 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4338003cb0 con 0x7f4348101110 2024-04-23T10:32:03.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.148+0000 7f43511b6740 1 Processor -- start 2024-04-23T10:32:03.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.148+0000 7f43511b6740 1 -- start start 2024-04-23T10:32:03.154 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.148+0000 7f43457fa640 1 --2- 172.21.15.97:0/3285476600 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f43340a3d20 0x7f43340a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:03.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.148+0000 7f43511b6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d0db3a30a0 0x55d0db3a3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:03.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.148+0000 7f43511b6740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0db3a39c0 0x55d0db3adb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:03.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.149+0000 7f43511b6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0db3ae090 0x55d0db3b0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:03.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.149+0000 7f43511b6740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d0db308370 con 0x55d0db3a39c0 2024-04-23T10:32:03.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.149+0000 7f43511b6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d0db2a52e0 con 0x55d0db3ae090 2024-04-23T10:32:03.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.149+0000 7f43511b6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d0db304330 con 0x55d0db3a30a0 2024-04-23T10:32:03.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.149+0000 7f434c969640 1 --2- 172.21.15.97:0/3285476600 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f43340a3d20 0x7f43340a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:03.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.149+0000 7f43477fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0db3a39c0 0x55d0db3adb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:03.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.149+0000 7f43477fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0db3a39c0 0x55d0db3adb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46354/0 (socket says 172.21.15.97:46354) 2024-04-23T10:32:03.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.149+0000 7f43477fe640 1 -- 172.21.15.97:0/2591279638 learned_addr learned my addr 172.21.15.97:0/2591279638 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:32:03.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.149+0000 7f43457fa640 1 -- 172.21.15.97:0/3285476600 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f43380f1260 con 0x7f4348101110 2024-04-23T10:32:03.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.149+0000 7f4347fff640 1 --2- 172.21.15.97:0/2591279638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d0db3a30a0 0x55d0db3a3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:03.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.149+0000 7f434c969640 1 --2- 172.21.15.97:0/2591279638 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0db3ae090 0x55d0db3b0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:03.155 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.149+0000 7f43477fe640 1 -- 172.21.15.97:0/2591279638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d0db3a30a0 msgr2=0x55d0db3a3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:32:03.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.149+0000 7f43477fe640 1 --2- 172.21.15.97:0/2591279638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d0db3a30a0 0x55d0db3a3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:03.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.149+0000 7f43477fe640 1 -- 172.21.15.97:0/2591279638 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0db3ae090 msgr2=0x55d0db3b0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:32:03.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.149+0000 7f43477fe640 1 --2- 172.21.15.97:0/2591279638 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0db3ae090 0x55d0db3b0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:03.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.149+0000 7f43477fe640 1 -- 172.21.15.97:0/2591279638 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0db0e6d80 con 0x55d0db3a39c0 2024-04-23T10:32:03.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.149+0000 7f434c969640 1 --2- 172.21.15.97:0/3285476600 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f43340a3d20 0x7f43340a61e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4340007fd0 tx=0x7f43400108e0 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:32:03.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.149+0000 7f43477fe640 1 --2- 172.21.15.97:0/2591279638 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0db3a39c0 0x55d0db3adb50 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f433000a9a0 tx=0x7f433000ae60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92cd09f15b5f606 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:32:03.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.149+0000 7f433e7fc640 1 -- 172.21.15.97:0/2591279638 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4330015070 con 0x55d0db3a39c0 2024-04-23T10:32:03.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.149+0000 7f433e7fc640 1 -- 172.21.15.97:0/2591279638 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f433000cc00 con 0x55d0db3a39c0 2024-04-23T10:32:03.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.149+0000 7f433e7fc640 1 -- 172.21.15.97:0/2591279638 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43300185a0 con 0x55d0db3a39c0 2024-04-23T10:32:03.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.150+0000 7f43511b6740 1 -- 172.21.15.97:0/2591279638 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0db3a39c0 msgr2=0x55d0db3adb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:03.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.150+0000 7f43511b6740 1 --2- 172.21.15.97:0/2591279638 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0db3a39c0 0x55d0db3adb50 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f433000a9a0 tx=0x7f433000ae60 comp rx=0 tx=0).stop 2024-04-23T10:32:03.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.150+0000 7f43511b6740 1 -- 172.21.15.97:0/2591279638 shutdown_connections 2024-04-23T10:32:03.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.150+0000 7f43511b6740 1 --2- 172.21.15.97:0/2591279638 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0db3ae090 0x55d0db3b0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:03.156 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.150+0000 7f43511b6740 1 --2- 172.21.15.97:0/2591279638 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0db3a39c0 0x55d0db3adb50 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:03.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.150+0000 7f43511b6740 1 --2- 172.21.15.97:0/2591279638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d0db3a30a0 0x55d0db3a3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:03.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.150+0000 7f43511b6740 1 -- 172.21.15.97:0/2591279638 >> 172.21.15.97:0/2591279638 conn(0x55d0db39f3c0 msgr2=0x55d0db39f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:32:03.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.150+0000 7f43511b6740 1 -- 172.21.15.97:0/2591279638 shutdown_connections 2024-04-23T10:32:03.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.150+0000 7f43511b6740 1 -- 172.21.15.97:0/2591279638 wait complete. 2024-04-23T10:32:03.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.150+0000 7f43511b6740 1 -- start start 2024-04-23T10:32:03.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.151+0000 7f43511b6740 1 Processor -- start 2024-04-23T10:32:03.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.151+0000 7f43511b6740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0db3a30a0 0x55d0db3ab010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:03.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.151+0000 7f43511b6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d0db3a39c0 0x55d0db3ab550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:03.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.151+0000 7f43511b6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0db3ae090 0x55d0db3aba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:03.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.151+0000 7f43511b6740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d0db3b0e80 con 0x55d0db3a30a0 2024-04-23T10:32:03.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.151+0000 7f43511b6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d0db2a2fb0 con 0x55d0db3ae090 2024-04-23T10:32:03.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.151+0000 7f43511b6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d0db3b1000 con 0x55d0db3a39c0 2024-04-23T10:32:03.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.151+0000 7f43511b6740 10 client.0 _ll_register_callbacks cb 0x55d0db2a29a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:32:03.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.151+0000 7f4347fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0db3a30a0 0x55d0db3ab010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:03.157 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.151+0000 7f4347fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0db3a30a0 0x55d0db3ab010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46364/0 (socket says 172.21.15.97:46364) 2024-04-23T10:32:03.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.151+0000 7f4347fff640 1 -- 172.21.15.97:0/980885755 learned_addr learned my addr 172.21.15.97:0/980885755 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:32:03.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.151+0000 7f43477fe640 1 --2- 172.21.15.97:0/980885755 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d0db3a39c0 0x55d0db3ab550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:03.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.151+0000 7f4347fff640 1 -- 172.21.15.97:0/980885755 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d0db3a39c0 msgr2=0x55d0db3ab550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:32:03.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.151+0000 7f4347fff640 1 --2- 172.21.15.97:0/980885755 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d0db3a39c0 0x55d0db3ab550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:03.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.151+0000 7f434c969640 1 --2- 172.21.15.97:0/980885755 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0db3ae090 0x55d0db3aba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:03.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.151+0000 7f4347fff640 1 -- 172.21.15.97:0/980885755 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0db3ae090 msgr2=0x55d0db3aba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:32:03.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.151+0000 7f4347fff640 1 --2- 172.21.15.97:0/980885755 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0db3ae090 0x55d0db3aba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:03.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.151+0000 7f4347fff640 1 -- 172.21.15.97:0/980885755 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0db147510 con 0x55d0db3a30a0 2024-04-23T10:32:03.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.151+0000 7f4347fff640 1 --2- 172.21.15.97:0/980885755 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0db3a30a0 0x55d0db3ab010 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f4338005ec0 tx=0x7f43380bd9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2966dc521a7a5378 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:32:03.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.151+0000 7f43277fe640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:32:03.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.151+0000 7f43277fe640 1 -- 172.21.15.97:0/980885755 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4338049040 con 0x55d0db3a30a0 2024-04-23T10:32:03.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.151+0000 7f43277fe640 1 -- 172.21.15.97:0/980885755 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f43380c6430 con 0x55d0db3a30a0 2024-04-23T10:32:03.158 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.151+0000 7f43511b6740 1 -- 172.21.15.97:0/980885755 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f433000cc00 con 0x55d0db3a30a0 2024-04-23T10:32:03.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.152+0000 7f43277fe640 1 -- 172.21.15.97:0/980885755 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4338083020 con 0x55d0db3a30a0 2024-04-23T10:32:03.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.152+0000 7f43511b6740 10 client.16413 fetch_fsmap learned FSMap version 26 2024-04-23T10:32:03.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.152+0000 7f43511b6740 1 -- 172.21.15.97:0/980885755 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d0dafda670 con 0x55d0db3a30a0 2024-04-23T10:32:03.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.152+0000 7f43277fe640 1 -- 172.21.15.97:0/980885755 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f433000cc00 con 0x55d0db3a30a0 2024-04-23T10:32:03.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.152+0000 7f43277fe640 1 -- 172.21.15.97:0/980885755 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f43380eec20 con 0x55d0db3a30a0 2024-04-23T10:32:03.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.152+0000 7f43511b6740 10 client.16413 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:32:03.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.152+0000 7f43511b6740 10 client.16413 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:32:03.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.152+0000 7f43511b6740 20 client.16413 populate_metadata read hostname 'smithi097' 2024-04-23T10:32:03.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.152+0000 7f43511b6740 1 --2- 172.21.15.97:0/980885755 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55d0db3be430 0x55d0db3c0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:03.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.152+0000 7f43511b6740 4 client.16413 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:32:03.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.152+0000 7f43511b6740 1 -- 172.21.15.97:0/980885755 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0db315d90 con 0x55d0db3be430 2024-04-23T10:32:03.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.152+0000 7f43477fe640 1 --2- 172.21.15.97:0/980885755 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55d0db3be430 0x55d0db3c0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:03.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.153+0000 7f43277fe640 10 client.16413 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:32:03.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.153+0000 7f43477fe640 1 --2- 172.21.15.97:0/980885755 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55d0db3be430 0x55d0db3c0880 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc1b481a15cbd96d server_cookie=2e5fb5c1e1ad82f2 in_seq=0 out_seq=0 2024-04-23T10:32:03.159 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.153+0000 7f43277fe640 1 -- 172.21.15.97:0/980885755 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d0db315d90 con 0x55d0db3be430 2024-04-23T10:32:03.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.153+0000 7f43277fe640 10 client.16413 handle_command_reply: tid=0 2024-04-23T10:32:03.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.153+0000 7f43511b6740 1 client.16413 shutdown 2024-04-23T10:32:03.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.154+0000 7f43511b6740 1 -- 172.21.15.97:0/980885755 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0db3a30a0 msgr2=0x55d0db3ab010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:03.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.154+0000 7f43511b6740 1 --2- 172.21.15.97:0/980885755 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0db3a30a0 0x55d0db3ab010 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f4338005ec0 tx=0x7f43380bd9b0 comp rx=0 tx=0).stop 2024-04-23T10:32:03.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.154+0000 7f43511b6740 1 -- 172.21.15.97:0/980885755 shutdown_connections 2024-04-23T10:32:03.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.154+0000 7f43511b6740 1 --2- 172.21.15.97:0/980885755 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55d0db3be430 0x55d0db3c0880 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:03.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.154+0000 7f43511b6740 1 --2- 172.21.15.97:0/980885755 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0db3ae090 0x55d0db3aba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:03.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.154+0000 7f43511b6740 1 --2- 172.21.15.97:0/980885755 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d0db3a39c0 0x55d0db3ab550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:03.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.154+0000 7f43511b6740 1 --2- 172.21.15.97:0/980885755 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0db3a30a0 0x55d0db3ab010 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:03.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.154+0000 7f43277fe640 0 client.16413 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:32:03.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.154+0000 7f43511b6740 1 -- 172.21.15.97:0/980885755 >> 172.21.15.97:0/980885755 conn(0x55d0db39f3c0 msgr2=0x55d0db3a51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:32:03.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.154+0000 7f43511b6740 1 -- 172.21.15.97:0/980885755 shutdown_connections 2024-04-23T10:32:03.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.154+0000 7f43511b6740 1 -- 172.21.15.97:0/980885755 wait complete. 2024-04-23T10:32:03.160 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.154+0000 7f43511b6740 20 client.16413 trim_cache size 0 max 16384 2024-04-23T10:32:03.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.171+0000 7f43511b6740 1 Processor -- start 2024-04-23T10:32:03.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.171+0000 7f43511b6740 1 -- start start 2024-04-23T10:32:03.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.171+0000 7f43511b6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d0db3a30a0 0x55d0db3ad7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:03.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.171+0000 7f43511b6740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0db3a39c0 0x55d0db39ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:03.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.171+0000 7f43511b6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0db3ae090 0x55d0db3a5910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:03.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.171+0000 7f43511b6740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f43481adb70 con 0x55d0db3a39c0 2024-04-23T10:32:03.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.171+0000 7f43511b6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f43481af6b0 con 0x55d0db3ae090 2024-04-23T10:32:03.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.171+0000 7f43511b6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f43481ae240 con 0x55d0db3a30a0 2024-04-23T10:32:03.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.171+0000 7f43477fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0db3a39c0 0x55d0db39ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:03.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.171+0000 7f43477fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0db3a39c0 0x55d0db39ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46374/0 (socket says 172.21.15.97:46374) 2024-04-23T10:32:03.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.171+0000 7f43477fe640 1 -- 172.21.15.97:0/3764748472 learned_addr learned my addr 172.21.15.97:0/3764748472 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:32:03.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.171+0000 7f4347fff640 1 --2- 172.21.15.97:0/3764748472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d0db3a30a0 0x55d0db3ad7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:03.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.171+0000 7f43477fe640 1 -- 172.21.15.97:0/3764748472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d0db3a30a0 msgr2=0x55d0db3ad7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:32:03.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.171+0000 7f43477fe640 1 --2- 172.21.15.97:0/3764748472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d0db3a30a0 0x55d0db3ad7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:03.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.171+0000 7f43477fe640 1 -- 172.21.15.97:0/3764748472 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0db3ae090 msgr2=0x55d0db3a5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:32:03.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.171+0000 7f43477fe640 1 --2- 172.21.15.97:0/3764748472 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0db3ae090 0x55d0db3a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:03.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.171+0000 7f43477fe640 1 -- 172.21.15.97:0/3764748472 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0db3a5e50 con 0x55d0db3a39c0 2024-04-23T10:32:03.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.171+0000 7f43477fe640 1 --2- 172.21.15.97:0/3764748472 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0db3a39c0 0x55d0db39ed80 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f433000a730 tx=0x7f433001c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3a9ecda83559fda server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:32:03.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.172+0000 7f433dffb640 1 -- 172.21.15.97:0/3764748472 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4330015040 con 0x55d0db3a39c0 2024-04-23T10:32:03.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.172+0000 7f433dffb640 1 -- 172.21.15.97:0/3764748472 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f433002a030 con 0x55d0db3a39c0 2024-04-23T10:32:03.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.172+0000 7f433dffb640 1 -- 172.21.15.97:0/3764748472 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43300266b0 con 0x55d0db3a39c0 2024-04-23T10:32:03.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.172+0000 7f43511b6740 1 -- 172.21.15.97:0/3764748472 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0db3a39c0 msgr2=0x55d0db39ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:03.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.172+0000 7f43511b6740 1 --2- 172.21.15.97:0/3764748472 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0db3a39c0 0x55d0db39ed80 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f433000a730 tx=0x7f433001c040 comp rx=0 tx=0).stop 2024-04-23T10:32:03.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.172+0000 7f43511b6740 1 -- 172.21.15.97:0/3764748472 shutdown_connections 2024-04-23T10:32:03.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.172+0000 7f43511b6740 1 --2- 172.21.15.97:0/3764748472 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0db3ae090 0x55d0db3a5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:03.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.172+0000 7f43511b6740 1 --2- 172.21.15.97:0/3764748472 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0db3a39c0 0x55d0db39ed80 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:03.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.172+0000 7f43511b6740 1 --2- 172.21.15.97:0/3764748472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d0db3a30a0 0x55d0db3ad7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:03.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.172+0000 7f43511b6740 1 -- 172.21.15.97:0/3764748472 >> 172.21.15.97:0/3764748472 conn(0x55d0db39f3c0 msgr2=0x55d0db3ad0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:32:03.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.172+0000 7f43511b6740 1 -- 172.21.15.97:0/3764748472 shutdown_connections 2024-04-23T10:32:03.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.172+0000 7f43511b6740 1 -- 172.21.15.97:0/3764748472 wait complete. 2024-04-23T10:32:03.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.172+0000 7f43511b6740 1 -- start start 2024-04-23T10:32:03.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.173+0000 7f43511b6740 1 Processor -- start 2024-04-23T10:32:03.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.173+0000 7f43511b6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0db3a30a0 0x55d0db3b6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:03.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.173+0000 7f43511b6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d0db3a39c0 0x55d0db3b6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:03.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.173+0000 7f43511b6740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0db3ae090 0x55d0db3b3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:03.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.173+0000 7f43511b6740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f43481aefe0 con 0x55d0db3ae090 2024-04-23T10:32:03.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.173+0000 7f43511b6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f43481ae910 con 0x55d0db3a30a0 2024-04-23T10:32:03.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.173+0000 7f43511b6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f43481abb70 con 0x55d0db3a39c0 2024-04-23T10:32:03.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.173+0000 7f43511b6740 10 client.0 _ll_register_callbacks cb 0x7f43380be320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:32:03.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.173+0000 7f434c969640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0db3ae090 0x55d0db3b3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:03.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.173+0000 7f434c969640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0db3ae090 0x55d0db3b3220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46376/0 (socket says 172.21.15.97:46376) 2024-04-23T10:32:03.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.173+0000 7f434c969640 1 -- 172.21.15.97:0/266237406 learned_addr learned my addr 172.21.15.97:0/266237406 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:32:03.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.173+0000 7f434c969640 1 -- 172.21.15.97:0/266237406 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d0db3a39c0 msgr2=0x55d0db3b6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:32:03.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.173+0000 7f43477fe640 1 --2- 172.21.15.97:0/266237406 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d0db3a39c0 0x55d0db3b6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:03.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.173+0000 7f434c969640 1 --2- 172.21.15.97:0/266237406 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d0db3a39c0 0x55d0db3b6790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:03.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.173+0000 7f434c969640 1 -- 172.21.15.97:0/266237406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0db3a30a0 msgr2=0x55d0db3b6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:32:03.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.173+0000 7f4347fff640 1 --2- 172.21.15.97:0/266237406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0db3a30a0 0x55d0db3b6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:03.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.173+0000 7f434c969640 1 --2- 172.21.15.97:0/266237406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0db3a30a0 0x55d0db3b6250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:03.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.173+0000 7f434c969640 1 -- 172.21.15.97:0/266237406 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0db3b3a20 con 0x55d0db3ae090 2024-04-23T10:32:03.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.173+0000 7f434c969640 1 --2- 172.21.15.97:0/266237406 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0db3ae090 0x55d0db3b3220 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f434000eab0 tx=0x7f4340004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dec94c69684d8802 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:32:03.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.173+0000 7f43277fe640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:32:03.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.174+0000 7f43277fe640 1 -- 172.21.15.97:0/266237406 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434001be00 con 0x55d0db3ae090 2024-04-23T10:32:03.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.174+0000 7f43277fe640 1 -- 172.21.15.97:0/266237406 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f434001d450 con 0x55d0db3ae090 2024-04-23T10:32:03.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.174+0000 7f43511b6740 1 -- 172.21.15.97:0/266237406 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f433002a030 con 0x55d0db3ae090 2024-04-23T10:32:03.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.174+0000 7f43277fe640 1 -- 172.21.15.97:0/266237406 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f434001d650 con 0x55d0db3ae090 2024-04-23T10:32:03.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.174+0000 7f43511b6740 10 client.16419 fetch_fsmap learned FSMap version 26 2024-04-23T10:32:03.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.174+0000 7f43511b6740 1 -- 172.21.15.97:0/266237406 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d0db425490 con 0x55d0db3ae090 2024-04-23T10:32:03.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.174+0000 7f43277fe640 1 -- 172.21.15.97:0/266237406 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f433002a030 con 0x55d0db3ae090 2024-04-23T10:32:03.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.174+0000 7f43277fe640 1 -- 172.21.15.97:0/266237406 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f434001a040 con 0x55d0db3ae090 2024-04-23T10:32:03.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.174+0000 7f43511b6740 10 client.16419 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:32:03.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.174+0000 7f43511b6740 10 client.16419 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:32:03.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.174+0000 7f43511b6740 20 client.16419 populate_metadata read hostname 'smithi097' 2024-04-23T10:32:03.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.174+0000 7f43511b6740 1 --2- 172.21.15.97:0/266237406 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55d0db425c60 0x55d0db428080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:03.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.174+0000 7f43511b6740 4 client.16419 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 79.45, "await_for": 42.0}] 2024-04-23T10:32:03.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.174+0000 7f43511b6740 1 -- 172.21.15.97:0/266237406 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 79.45, "await_for": 42.0}) -- 0x55d0db30fc00 con 0x55d0db425c60 2024-04-23T10:32:03.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.174+0000 7f4347fff640 1 --2- 172.21.15.97:0/266237406 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55d0db425c60 0x55d0db428080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:03.178 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.175+0000 7f43277fe640 10 client.16419 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:32:03.179 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.175+0000 7f4347fff640 1 --2- 172.21.15.97:0/266237406 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55d0db425c60 0x55d0db428080 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2d614893e681779 server_cookie=a6908db01e4cbceb in_seq=0 out_seq=0 2024-04-23T10:32:03.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.188+0000 7f43277fe640 1 -- 172.21.15.97:0/266237406 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55d0db30fc00 con 0x55d0db425c60 2024-04-23T10:32:03.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.188+0000 7f43277fe640 10 client.16419 handle_command_reply: tid=0 2024-04-23T10:32:03.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.188+0000 7f43511b6740 1 client.16419 shutdown 2024-04-23T10:32:03.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.188+0000 7f43511b6740 1 -- 172.21.15.97:0/266237406 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0db3ae090 msgr2=0x55d0db3b3220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:03.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.188+0000 7f43511b6740 1 --2- 172.21.15.97:0/266237406 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0db3ae090 0x55d0db3b3220 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f434000eab0 tx=0x7f4340004b50 comp rx=0 tx=0).stop 2024-04-23T10:32:03.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.188+0000 7f43511b6740 1 -- 172.21.15.97:0/266237406 shutdown_connections 2024-04-23T10:32:03.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.188+0000 7f43511b6740 1 --2- 172.21.15.97:0/266237406 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55d0db425c60 0x55d0db428080 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:03.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.188+0000 7f43511b6740 1 --2- 172.21.15.97:0/266237406 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d0db3ae090 0x55d0db3b3220 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:03.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.188+0000 7f43511b6740 1 --2- 172.21.15.97:0/266237406 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d0db3a39c0 0x55d0db3b6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:03.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.188+0000 7f43277fe640 0 client.16419 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:32:03.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.188+0000 7f43511b6740 1 --2- 172.21.15.97:0/266237406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0db3a30a0 0x55d0db3b6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:03.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.188+0000 7f43511b6740 1 -- 172.21.15.97:0/266237406 >> 172.21.15.97:0/266237406 conn(0x55d0db39f3c0 msgr2=0x55d0db39dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:32:03.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.189+0000 7f43511b6740 1 -- 172.21.15.97:0/266237406 shutdown_connections 2024-04-23T10:32:03.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.189+0000 7f43511b6740 1 -- 172.21.15.97:0/266237406 wait complete. 2024-04-23T10:32:03.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.189+0000 7f43511b6740 20 client.16419 trim_cache size 0 max 16384 2024-04-23T10:32:03.189 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-23T10:32:03.189 INFO:teuthology.orchestra.run.smithi097.stdout: "epoch": 26, 2024-04-23T10:32:03.189 INFO:teuthology.orchestra.run.smithi097.stdout: "leader": 24488, 2024-04-23T10:32:03.189 INFO:teuthology.orchestra.run.smithi097.stdout: "set_version": 22, 2024-04-23T10:32:03.189 INFO:teuthology.orchestra.run.smithi097.stdout: "sets": { 2024-04-23T10:32:03.189 INFO:teuthology.orchestra.run.smithi097.stdout: "702905e7": { 2024-04-23T10:32:03.190 INFO:teuthology.orchestra.run.smithi097.stdout: "version": 22, 2024-04-23T10:32:03.190 INFO:teuthology.orchestra.run.smithi097.stdout: "age_ref": 0.0, 2024-04-23T10:32:03.190 INFO:teuthology.orchestra.run.smithi097.stdout: "state": { 2024-04-23T10:32:03.190 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "QUIESCED", 2024-04-23T10:32:03.190 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0 2024-04-23T10:32:03.190 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:32:03.190 INFO:teuthology.orchestra.run.smithi097.stdout: "timeout": 60.0, 2024-04-23T10:32:03.190 INFO:teuthology.orchestra.run.smithi097.stdout: "expiration": 79.5, 2024-04-23T10:32:03.190 INFO:teuthology.orchestra.run.smithi097.stdout: "members": { 2024-04-23T10:32:03.190 INFO:teuthology.orchestra.run.smithi097.stdout: "file:/": { 2024-04-23T10:32:03.190 INFO:teuthology.orchestra.run.smithi097.stdout: "excluded": false, 2024-04-23T10:32:03.190 INFO:teuthology.orchestra.run.smithi097.stdout: "state": { 2024-04-23T10:32:03.190 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "QUIESCED", 2024-04-23T10:32:03.190 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0 2024-04-23T10:32:03.191 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:32:03.191 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:32:03.191 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:32:03.191 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:32:03.191 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:32:03.191 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-23T10:32:03.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.189+0000 7f434e3cc640 1 -- 172.21.15.97:0/3285476600 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f43340a3d20 msgr2=0x7f43340a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:03.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.189+0000 7f434e3cc640 1 --2- 172.21.15.97:0/3285476600 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f43340a3d20 0x7f43340a61e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4340007fd0 tx=0x7f43400108e0 comp rx=0 tx=0).stop 2024-04-23T10:32:03.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.189+0000 7f434e3cc640 1 -- 172.21.15.97:0/3285476600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4348101110 msgr2=0x7f4348199770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:03.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.189+0000 7f434e3cc640 1 --2- 172.21.15.97:0/3285476600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4348101110 0x7f4348199770 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f4338006010 tx=0x7f43380042f0 comp rx=0 tx=0).stop 2024-04-23T10:32:03.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.189+0000 7f434e3cc640 1 -- 172.21.15.97:0/3285476600 shutdown_connections 2024-04-23T10:32:03.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.189+0000 7f434e3cc640 1 --2- 172.21.15.97:0/3285476600 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f43340a3d20 0x7f43340a61e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:03.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.189+0000 7f434e3cc640 1 --2- 172.21.15.97:0/3285476600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f434819a1f0 0x7f43481a47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:03.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.189+0000 7f434e3cc640 1 --2- 172.21.15.97:0/3285476600 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4348106190 0x7f4348199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:03.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.189+0000 7f434e3cc640 1 --2- 172.21.15.97:0/3285476600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4348101110 0x7f4348199770 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:03.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.189+0000 7f434e3cc640 1 -- 172.21.15.97:0/3285476600 >> 172.21.15.97:0/3285476600 conn(0x7f4348077fb0 msgr2=0x7f43480ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:32:03.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.189+0000 7f434e3cc640 1 -- 172.21.15.97:0/3285476600 shutdown_connections 2024-04-23T10:32:03.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:03.189+0000 7f434e3cc640 1 -- 172.21.15.97:0/3285476600 wait complete. 2024-04-23T10:32:03.199 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '702905e7', quiesce took 0.5 seconds. Will release after: 19.45 2024-04-23T10:32:03.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:32:02 smithi196 ceph-mon[32919]: pgmap v178: 97 pgs: 97 active+clean; 2.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.1 KiB/s wr, 11 op/s 2024-04-23T10:32:03.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:32:02 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/3797018571' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:32:03.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:32:02 smithi097 ceph-mon[29197]: pgmap v178: 97 pgs: 97 active+clean; 2.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.1 KiB/s wr, 11 op/s 2024-04-23T10:32:03.404 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:32:02 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/3797018571' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:32:03.405 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:32:02 smithi183 ceph-mon[32317]: pgmap v178: 97 pgs: 97 active+clean; 2.6 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.1 KiB/s wr, 11 op/s 2024-04-23T10:32:03.405 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:32:02 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/3797018571' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:32:04.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:32:03 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/2397955236' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:32:04.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:32:03 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/2397955236' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:32:04.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:32:03 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/2397955236' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:32:05.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:32:04 smithi196 ceph-mon[32919]: pgmap v179: 97 pgs: 97 active+clean; 2.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 30 KiB/s wr, 30 op/s 2024-04-23T10:32:05.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:32:04 smithi097 ceph-mon[29197]: pgmap v179: 97 pgs: 97 active+clean; 2.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 30 KiB/s wr, 30 op/s 2024-04-23T10:32:05.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:32:04 smithi183 ceph-mon[32317]: pgmap v179: 97 pgs: 97 active+clean; 2.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 30 KiB/s wr, 30 op/s 2024-04-23T10:32:07.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:32:06 smithi196 ceph-mon[32919]: pgmap v180: 97 pgs: 97 active+clean; 2.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 28 KiB/s wr, 26 op/s 2024-04-23T10:32:07.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:32:06 smithi097 ceph-mon[29197]: pgmap v180: 97 pgs: 97 active+clean; 2.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 28 KiB/s wr, 26 op/s 2024-04-23T10:32:07.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:32:06 smithi183 ceph-mon[32317]: pgmap v180: 97 pgs: 97 active+clean; 2.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 28 KiB/s wr, 26 op/s 2024-04-23T10:32:08.595 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.593+0000 7f1c7c8cf640 1 -- 172.21.15.97:0/102681724 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c78107290 msgr2=0x7f1c7810ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:08.595 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.593+0000 7f1c7c8cf640 1 --2- 172.21.15.97:0/102681724 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c78107290 0x7f1c7810ee60 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f1c70009ca0 tx=0x7f1c7002a8c0 comp rx=0 tx=0).stop 2024-04-23T10:32:08.595 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.593+0000 7f1c7c8cf640 1 -- 172.21.15.97:0/102681724 shutdown_connections 2024-04-23T10:32:08.595 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.593+0000 7f1c7c8cf640 1 --2- 172.21.15.97:0/102681724 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c78107290 0x7f1c7810ee60 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:08.595 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.593+0000 7f1c7c8cf640 1 --2- 172.21.15.97:0/102681724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c78102b70 0x7f1c78106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:08.595 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.593+0000 7f1c7c8cf640 1 --2- 172.21.15.97:0/102681724 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c781021c0 0x7f1c781025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:08.595 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.593+0000 7f1c7c8cf640 1 -- 172.21.15.97:0/102681724 >> 172.21.15.97:0/102681724 conn(0x7f1c780fda20 msgr2=0x7f1c780ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:32:08.596 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.593+0000 7f1c7c8cf640 1 -- 172.21.15.97:0/102681724 shutdown_connections 2024-04-23T10:32:08.596 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.593+0000 7f1c7c8cf640 1 -- 172.21.15.97:0/102681724 wait complete. 2024-04-23T10:32:08.596 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.594+0000 7f1c7c8cf640 1 Processor -- start 2024-04-23T10:32:08.596 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.594+0000 7f1c7c8cf640 1 -- start start 2024-04-23T10:32:08.596 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.594+0000 7f1c7c8cf640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c781021c0 0x7f1c781985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:08.596 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.594+0000 7f1c7c8cf640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c78107290 0x7f1c78198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:08.596 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.594+0000 7f1c7c8cf640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c7819cea0 0x7f1c7819d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:08.596 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.594+0000 7f1c7c8cf640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1c78112200 con 0x7f1c781021c0 2024-04-23T10:32:08.597 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.594+0000 7f1c7c8cf640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1c78112080 con 0x7f1c7819cea0 2024-04-23T10:32:08.597 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.594+0000 7f1c7c8cf640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1c78112380 con 0x7f1c78107290 2024-04-23T10:32:08.597 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.594+0000 7f1c7659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c781021c0 0x7f1c781985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:08.597 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.594+0000 7f1c7659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c781021c0 0x7f1c781985e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47754/0 (socket says 172.21.15.97:47754) 2024-04-23T10:32:08.597 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.594+0000 7f1c7659c640 1 -- 172.21.15.97:0/695123903 learned_addr learned my addr 172.21.15.97:0/695123903 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:32:08.597 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.594+0000 7f1c75d9b640 1 --2- 172.21.15.97:0/695123903 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c78107290 0x7f1c78198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:08.597 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.594+0000 7f1c76d9d640 1 --2- 172.21.15.97:0/695123903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c7819cea0 0x7f1c7819d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:08.597 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.594+0000 7f1c7659c640 1 -- 172.21.15.97:0/695123903 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c78107290 msgr2=0x7f1c78198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:08.597 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.594+0000 7f1c7659c640 1 --2- 172.21.15.97:0/695123903 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c78107290 0x7f1c78198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:08.597 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.594+0000 7f1c7659c640 1 -- 172.21.15.97:0/695123903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c7819cea0 msgr2=0x7f1c7819d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:08.598 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.594+0000 7f1c7659c640 1 --2- 172.21.15.97:0/695123903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c7819cea0 0x7f1c7819d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:08.598 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.594+0000 7f1c7659c640 1 -- 172.21.15.97:0/695123903 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c7819d990 con 0x7f1c781021c0 2024-04-23T10:32:08.598 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.595+0000 7f1c7659c640 1 --2- 172.21.15.97:0/695123903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c781021c0 0x7f1c781985e0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f1c6000c920 tx=0x7f1c6000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:32:08.598 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.595+0000 7f1c6f7fe640 1 -- 172.21.15.97:0/695123903 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c60007c70 con 0x7f1c781021c0 2024-04-23T10:32:08.598 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.595+0000 7f1c7c8cf640 1 -- 172.21.15.97:0/695123903 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c7819dc80 con 0x7f1c781021c0 2024-04-23T10:32:08.598 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.595+0000 7f1c6f7fe640 1 -- 172.21.15.97:0/695123903 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c60007e10 con 0x7f1c781021c0 2024-04-23T10:32:08.598 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.595+0000 7f1c6f7fe640 1 -- 172.21.15.97:0/695123903 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c60018820 con 0x7f1c781021c0 2024-04-23T10:32:08.598 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.595+0000 7f1c7c8cf640 1 -- 172.21.15.97:0/695123903 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c781a5470 con 0x7f1c781021c0 2024-04-23T10:32:08.604 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.597+0000 7f1c6f7fe640 1 -- 172.21.15.97:0/695123903 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1c6000f450 con 0x7f1c781021c0 2024-04-23T10:32:08.604 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.597+0000 7f1c7f5e6740 1 Processor -- start 2024-04-23T10:32:08.604 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.598+0000 7f1c7f5e6740 1 -- start start 2024-04-23T10:32:08.604 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.598+0000 7f1c7f5e6740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fc4d858520 0x55fc4d858900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:08.605 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.598+0000 7f1c7f5e6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc4d858e40 0x55fc4d8676d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:08.605 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.598+0000 7f1c7f5e6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fc4d867c10 0x55fc4d86a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:08.605 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.598+0000 7f1c7f5e6740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55fc4d7bd970 con 0x55fc4d858520 2024-04-23T10:32:08.605 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.598+0000 7f1c7f5e6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55fc4d75a4a0 con 0x55fc4d858e40 2024-04-23T10:32:08.605 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.598+0000 7f1c7f5e6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55fc4d7b9930 con 0x55fc4d867c10 2024-04-23T10:32:08.605 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.598+0000 7f1c6f7fe640 1 --2- 172.21.15.97:0/695123903 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f1c640a3dc0 0x7f1c640a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:08.605 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.598+0000 7f1c76d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fc4d858520 0x55fc4d858900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:08.605 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.598+0000 7f1c76d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fc4d858520 0x55fc4d858900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47760/0 (socket says 172.21.15.97:47760) 2024-04-23T10:32:08.605 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.598+0000 7f1c76d9d640 1 -- 172.21.15.97:0/2489776354 learned_addr learned my addr 172.21.15.97:0/2489776354 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:32:08.605 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.598+0000 7f1c76d9d640 1 --2- 172.21.15.97:0/695123903 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f1c640a3dc0 0x7f1c640a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:08.605 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.598+0000 7f1c7659c640 1 --2- 172.21.15.97:0/2489776354 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc4d858e40 0x55fc4d8676d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:08.605 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.598+0000 7f1c75d9b640 1 --2- 172.21.15.97:0/2489776354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fc4d867c10 0x55fc4d86a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:08.605 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.598+0000 7f1c6f7fe640 1 -- 172.21.15.97:0/695123903 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f1c600d08d0 con 0x7f1c781021c0 2024-04-23T10:32:08.605 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.599+0000 7f1c76d9d640 1 -- 172.21.15.97:0/2489776354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fc4d867c10 msgr2=0x55fc4d86a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:32:08.606 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.599+0000 7f1c76d9d640 1 --2- 172.21.15.97:0/2489776354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fc4d867c10 0x55fc4d86a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:08.606 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.599+0000 7f1c76d9d640 1 -- 172.21.15.97:0/2489776354 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc4d858e40 msgr2=0x55fc4d8676d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:32:08.606 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.599+0000 7f1c76d9d640 1 --2- 172.21.15.97:0/2489776354 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc4d858e40 0x55fc4d8676d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:08.606 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.599+0000 7f1c76d9d640 1 -- 172.21.15.97:0/2489776354 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc4d4d0210 con 0x55fc4d858520 2024-04-23T10:32:08.606 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.599+0000 7f1c76d9d640 1 --2- 172.21.15.97:0/2489776354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fc4d858520 0x55fc4d858900 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f1c70033040 tx=0x7f1c70031600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f3741550c68675c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:32:08.606 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.599+0000 7f1c76d9d640 1 --2- 172.21.15.97:0/695123903 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f1c640a3dc0 0x7f1c640a6280 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1c70031670 tx=0x7f1c70043000 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:32:08.606 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.599+0000 7f1c6cff9640 1 -- 172.21.15.97:0/2489776354 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c7004f070 con 0x55fc4d858520 2024-04-23T10:32:08.606 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.599+0000 7f1c6cff9640 1 -- 172.21.15.97:0/2489776354 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c70036040 con 0x55fc4d858520 2024-04-23T10:32:08.606 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.599+0000 7f1c6cff9640 1 -- 172.21.15.97:0/2489776354 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c70004640 con 0x55fc4d858520 2024-04-23T10:32:08.606 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.600+0000 7f1c7f5e6740 1 -- 172.21.15.97:0/2489776354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fc4d858520 msgr2=0x55fc4d858900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:08.606 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.600+0000 7f1c7f5e6740 1 --2- 172.21.15.97:0/2489776354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fc4d858520 0x55fc4d858900 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f1c70033040 tx=0x7f1c70031600 comp rx=0 tx=0).stop 2024-04-23T10:32:08.606 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.600+0000 7f1c7f5e6740 1 -- 172.21.15.97:0/2489776354 shutdown_connections 2024-04-23T10:32:08.606 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.600+0000 7f1c7f5e6740 1 --2- 172.21.15.97:0/2489776354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fc4d867c10 0x55fc4d86a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:08.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.600+0000 7f1c7f5e6740 1 --2- 172.21.15.97:0/2489776354 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc4d858e40 0x55fc4d8676d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:08.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.600+0000 7f1c7f5e6740 1 --2- 172.21.15.97:0/2489776354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fc4d858520 0x55fc4d858900 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:08.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.600+0000 7f1c7f5e6740 1 -- 172.21.15.97:0/2489776354 >> 172.21.15.97:0/2489776354 conn(0x55fc4d854b30 msgr2=0x55fc4d854f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:32:08.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.600+0000 7f1c7f5e6740 1 -- 172.21.15.97:0/2489776354 shutdown_connections 2024-04-23T10:32:08.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.600+0000 7f1c7f5e6740 1 -- 172.21.15.97:0/2489776354 wait complete. 2024-04-23T10:32:08.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.600+0000 7f1c7f5e6740 1 -- start start 2024-04-23T10:32:08.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.600+0000 7f1c7f5e6740 1 Processor -- start 2024-04-23T10:32:08.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.601+0000 7f1c7f5e6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc4d858520 0x55fc4d867640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:08.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.601+0000 7f1c7f5e6740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fc4d858e40 0x55fc4d864930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:08.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.601+0000 7f1c7f5e6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fc4d867c10 0x55fc4d865070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:08.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.601+0000 7f1c7f5e6740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55fc4d86a7e0 con 0x55fc4d858e40 2024-04-23T10:32:08.608 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.601+0000 7f1c7f5e6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55fc4d7580b0 con 0x55fc4d858520 2024-04-23T10:32:08.608 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.601+0000 7f1c7f5e6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55fc4d86a960 con 0x55fc4d867c10 2024-04-23T10:32:08.608 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.601+0000 7f1c7f5e6740 10 client.0 _ll_register_callbacks cb 0x55fc4d745d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:32:08.608 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.601+0000 7f1c75d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fc4d858e40 0x55fc4d864930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:08.608 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.601+0000 7f1c7659c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc4d858520 0x55fc4d867640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:08.608 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.601+0000 7f1c75d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fc4d858e40 0x55fc4d864930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47768/0 (socket says 172.21.15.97:47768) 2024-04-23T10:32:08.608 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.601+0000 7f1c75d9b640 1 -- 172.21.15.97:0/3964440709 learned_addr learned my addr 172.21.15.97:0/3964440709 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:32:08.608 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.601+0000 7f1c76d9d640 1 --2- 172.21.15.97:0/3964440709 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fc4d867c10 0x55fc4d865070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:08.608 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.601+0000 7f1c75d9b640 1 -- 172.21.15.97:0/3964440709 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fc4d867c10 msgr2=0x55fc4d865070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:32:08.608 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.601+0000 7f1c75d9b640 1 --2- 172.21.15.97:0/3964440709 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fc4d867c10 0x55fc4d865070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:08.608 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.601+0000 7f1c75d9b640 1 -- 172.21.15.97:0/3964440709 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc4d858520 msgr2=0x55fc4d867640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:32:08.608 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.601+0000 7f1c75d9b640 1 --2- 172.21.15.97:0/3964440709 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc4d858520 0x55fc4d867640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:08.608 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.601+0000 7f1c75d9b640 1 -- 172.21.15.97:0/3964440709 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc4d59c620 con 0x55fc4d858e40 2024-04-23T10:32:08.608 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.601+0000 7f1c7659c640 1 --2- 172.21.15.97:0/3964440709 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc4d858520 0x55fc4d867640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T10:32:08.608 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.601+0000 7f1c75d9b640 1 --2- 172.21.15.97:0/3964440709 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fc4d858e40 0x55fc4d864930 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f1c6800b8f0 tx=0x7f1c6800bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bfe0a7176c86ddb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:32:08.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.601+0000 7f1c5dffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:32:08.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.601+0000 7f1c5dffb640 1 -- 172.21.15.97:0/3964440709 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c68003e20 con 0x55fc4d858e40 2024-04-23T10:32:08.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.601+0000 7f1c5dffb640 1 -- 172.21.15.97:0/3964440709 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f1c68005020 con 0x55fc4d858e40 2024-04-23T10:32:08.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.601+0000 7f1c7f5e6740 1 -- 172.21.15.97:0/3964440709 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c70036040 con 0x55fc4d858e40 2024-04-23T10:32:08.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.602+0000 7f1c5dffb640 1 -- 172.21.15.97:0/3964440709 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1c68005220 con 0x55fc4d858e40 2024-04-23T10:32:08.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.602+0000 7f1c7f5e6740 10 client.16431 fetch_fsmap learned FSMap version 26 2024-04-23T10:32:08.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.602+0000 7f1c7f5e6740 1 -- 172.21.15.97:0/3964440709 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fc4d8768a0 con 0x55fc4d858e40 2024-04-23T10:32:08.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.602+0000 7f1c5dffb640 1 -- 172.21.15.97:0/3964440709 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c70036040 con 0x55fc4d858e40 2024-04-23T10:32:08.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.602+0000 7f1c5dffb640 1 -- 172.21.15.97:0/3964440709 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1c68025480 con 0x55fc4d858e40 2024-04-23T10:32:08.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.602+0000 7f1c7f5e6740 10 client.16431 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:32:08.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.602+0000 7f1c7f5e6740 10 client.16431 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:32:08.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.602+0000 7f1c7f5e6740 20 client.16431 populate_metadata read hostname 'smithi097' 2024-04-23T10:32:08.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.602+0000 7f1c7f5e6740 1 --2- 172.21.15.97:0/3964440709 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55fc4d877d10 0x55fc4d87a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:08.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.602+0000 7f1c7f5e6740 4 client.16431 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:32:08.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.602+0000 7f1c7f5e6740 1 -- 172.21.15.97:0/3964440709 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc4d7cb390 con 0x55fc4d877d10 2024-04-23T10:32:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.602+0000 7f1c7659c640 1 --2- 172.21.15.97:0/3964440709 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55fc4d877d10 0x55fc4d87a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.603+0000 7f1c5dffb640 10 client.16431 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:32:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.603+0000 7f1c7659c640 1 --2- 172.21.15.97:0/3964440709 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55fc4d877d10 0x55fc4d87a120 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca40dfd1294c055d server_cookie=59447b7058f7a22f in_seq=0 out_seq=0 2024-04-23T10:32:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.603+0000 7f1c5dffb640 1 -- 172.21.15.97:0/3964440709 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fc4d7cb390 con 0x55fc4d877d10 2024-04-23T10:32:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.603+0000 7f1c5dffb640 10 client.16431 handle_command_reply: tid=0 2024-04-23T10:32:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.603+0000 7f1c7f5e6740 1 client.16431 shutdown 2024-04-23T10:32:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.604+0000 7f1c7f5e6740 1 -- 172.21.15.97:0/3964440709 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fc4d858e40 msgr2=0x55fc4d864930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.604+0000 7f1c7f5e6740 1 --2- 172.21.15.97:0/3964440709 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fc4d858e40 0x55fc4d864930 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f1c6800b8f0 tx=0x7f1c6800bdb0 comp rx=0 tx=0).stop 2024-04-23T10:32:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.604+0000 7f1c7f5e6740 1 -- 172.21.15.97:0/3964440709 shutdown_connections 2024-04-23T10:32:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.604+0000 7f1c7f5e6740 1 --2- 172.21.15.97:0/3964440709 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55fc4d877d10 0x55fc4d87a120 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.604+0000 7f1c7f5e6740 1 --2- 172.21.15.97:0/3964440709 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fc4d867c10 0x55fc4d865070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.604+0000 7f1c7f5e6740 1 --2- 172.21.15.97:0/3964440709 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fc4d858e40 0x55fc4d864930 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.604+0000 7f1c5dffb640 0 client.16431 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:32:08.611 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.604+0000 7f1c7f5e6740 1 --2- 172.21.15.97:0/3964440709 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc4d858520 0x55fc4d867640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:08.611 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.604+0000 7f1c7f5e6740 1 -- 172.21.15.97:0/3964440709 >> 172.21.15.97:0/3964440709 conn(0x55fc4d854b30 msgr2=0x55fc4d85a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:32:08.611 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.604+0000 7f1c7f5e6740 1 -- 172.21.15.97:0/3964440709 shutdown_connections 2024-04-23T10:32:08.611 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.604+0000 7f1c7f5e6740 1 -- 172.21.15.97:0/3964440709 wait complete. 2024-04-23T10:32:08.611 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.604+0000 7f1c7f5e6740 20 client.16431 trim_cache size 0 max 16384 2024-04-23T10:32:08.623 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.618+0000 7f1c7f5e6740 1 Processor -- start 2024-04-23T10:32:08.623 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.618+0000 7f1c7f5e6740 1 -- start start 2024-04-23T10:32:08.623 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.619+0000 7f1c7f5e6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc4d858520 0x55fc4d8736a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:08.623 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.619+0000 7f1c7f5e6740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fc4d858e40 0x55fc4d85fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:08.623 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.619+0000 7f1c7f5e6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fc4d867c10 0x55fc4d867170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:08.623 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.619+0000 7f1c7f5e6740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1c781ab7a0 con 0x55fc4d858e40 2024-04-23T10:32:08.623 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.619+0000 7f1c7f5e6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1c781ad2e0 con 0x55fc4d858520 2024-04-23T10:32:08.623 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.619+0000 7f1c7f5e6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1c781abe70 con 0x55fc4d867c10 2024-04-23T10:32:08.623 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.619+0000 7f1c75d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fc4d858e40 0x55fc4d85fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:08.623 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.619+0000 7f1c75d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fc4d858e40 0x55fc4d85fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47770/0 (socket says 172.21.15.97:47770) 2024-04-23T10:32:08.623 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.619+0000 7f1c75d9b640 1 -- 172.21.15.97:0/434239211 learned_addr learned my addr 172.21.15.97:0/434239211 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:32:08.624 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.619+0000 7f1c76d9d640 1 --2- 172.21.15.97:0/434239211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fc4d867c10 0x55fc4d867170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:08.624 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.619+0000 7f1c7659c640 1 --2- 172.21.15.97:0/434239211 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc4d858520 0x55fc4d8736a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:08.624 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.619+0000 7f1c75d9b640 1 -- 172.21.15.97:0/434239211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fc4d867c10 msgr2=0x55fc4d867170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:32:08.624 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.619+0000 7f1c75d9b640 1 --2- 172.21.15.97:0/434239211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fc4d867c10 0x55fc4d867170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:08.624 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.619+0000 7f1c75d9b640 1 -- 172.21.15.97:0/434239211 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc4d858520 msgr2=0x55fc4d8736a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:32:08.624 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.619+0000 7f1c75d9b640 1 --2- 172.21.15.97:0/434239211 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc4d858520 0x55fc4d8736a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:08.624 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.619+0000 7f1c75d9b640 1 -- 172.21.15.97:0/434239211 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc4d858c70 con 0x55fc4d858e40 2024-04-23T10:32:08.624 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.619+0000 7f1c75d9b640 1 --2- 172.21.15.97:0/434239211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fc4d858e40 0x55fc4d85fd80 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f1c68000c00 tx=0x7f1c680040f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24b7295057168a15 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:32:08.624 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.619+0000 7f1c5ffff640 1 -- 172.21.15.97:0/434239211 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c68025480 con 0x55fc4d858e40 2024-04-23T10:32:08.624 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.619+0000 7f1c5ffff640 1 -- 172.21.15.97:0/434239211 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c68025620 con 0x55fc4d858e40 2024-04-23T10:32:08.625 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.619+0000 7f1c5ffff640 1 -- 172.21.15.97:0/434239211 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6802fb90 con 0x55fc4d858e40 2024-04-23T10:32:08.625 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.620+0000 7f1c7f5e6740 1 -- 172.21.15.97:0/434239211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fc4d858e40 msgr2=0x55fc4d85fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:08.625 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.620+0000 7f1c7f5e6740 1 --2- 172.21.15.97:0/434239211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fc4d858e40 0x55fc4d85fd80 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f1c68000c00 tx=0x7f1c680040f0 comp rx=0 tx=0).stop 2024-04-23T10:32:08.625 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.620+0000 7f1c7f5e6740 1 -- 172.21.15.97:0/434239211 shutdown_connections 2024-04-23T10:32:08.625 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.620+0000 7f1c7f5e6740 1 --2- 172.21.15.97:0/434239211 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fc4d867c10 0x55fc4d867170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:08.625 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.620+0000 7f1c7f5e6740 1 --2- 172.21.15.97:0/434239211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fc4d858e40 0x55fc4d85fd80 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:08.625 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.620+0000 7f1c7f5e6740 1 --2- 172.21.15.97:0/434239211 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc4d858520 0x55fc4d8736a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:08.625 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.620+0000 7f1c7f5e6740 1 -- 172.21.15.97:0/434239211 >> 172.21.15.97:0/434239211 conn(0x55fc4d854b30 msgr2=0x55fc4d85f670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:32:08.625 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.620+0000 7f1c7f5e6740 1 -- 172.21.15.97:0/434239211 shutdown_connections 2024-04-23T10:32:08.625 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.620+0000 7f1c7f5e6740 1 -- 172.21.15.97:0/434239211 wait complete. 2024-04-23T10:32:08.625 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.620+0000 7f1c7f5e6740 1 -- start start 2024-04-23T10:32:08.625 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.620+0000 7f1c7f5e6740 1 Processor -- start 2024-04-23T10:32:08.626 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.621+0000 7f1c7f5e6740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fc4d858520 0x55fc4d8700f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:08.626 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.621+0000 7f1c7f5e6740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fc4d858e40 0x55fc4d8614d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:08.626 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.621+0000 7f1c7f5e6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc4d867c10 0x55fc4d861c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:08.626 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.621+0000 7f1c7f5e6740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1c781acc10 con 0x55fc4d858e40 2024-04-23T10:32:08.626 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.621+0000 7f1c7f5e6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1c781ac540 con 0x55fc4d867c10 2024-04-23T10:32:08.626 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.621+0000 7f1c7f5e6740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1c781a97a0 con 0x55fc4d858520 2024-04-23T10:32:08.626 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.621+0000 7f1c7f5e6740 10 client.0 _ll_register_callbacks cb 0x7f1c68004210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:32:08.626 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.621+0000 7f1c75d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fc4d858e40 0x55fc4d8614d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:08.626 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.621+0000 7f1c75d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fc4d858e40 0x55fc4d8614d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47784/0 (socket says 172.21.15.97:47784) 2024-04-23T10:32:08.626 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.621+0000 7f1c75d9b640 1 -- 172.21.15.97:0/2441525964 learned_addr learned my addr 172.21.15.97:0/2441525964 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:32:08.626 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.621+0000 7f1c76d9d640 1 --2- 172.21.15.97:0/2441525964 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc4d867c10 0x55fc4d861c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:08.626 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.621+0000 7f1c7659c640 1 --2- 172.21.15.97:0/2441525964 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fc4d858520 0x55fc4d8700f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.621+0000 7f1c75d9b640 1 -- 172.21.15.97:0/2441525964 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fc4d858520 msgr2=0x55fc4d8700f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:32:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.621+0000 7f1c75d9b640 1 --2- 172.21.15.97:0/2441525964 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fc4d858520 0x55fc4d8700f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.621+0000 7f1c75d9b640 1 -- 172.21.15.97:0/2441525964 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc4d867c10 msgr2=0x55fc4d861c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:32:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.621+0000 7f1c75d9b640 1 --2- 172.21.15.97:0/2441525964 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc4d867c10 0x55fc4d861c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.621+0000 7f1c75d9b640 1 -- 172.21.15.97:0/2441525964 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc4d8dd2a0 con 0x55fc4d858e40 2024-04-23T10:32:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.621+0000 7f1c7659c640 1 --2- 172.21.15.97:0/2441525964 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fc4d858520 0x55fc4d8700f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:32:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.621+0000 7f1c75d9b640 1 --2- 172.21.15.97:0/2441525964 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fc4d858e40 0x55fc4d8614d0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f1c68003ac0 tx=0x7f1c6800b5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=443749a682acca33 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:32:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.621+0000 7f1c5dffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:32:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.621+0000 7f1c5dffb640 1 -- 172.21.15.97:0/2441525964 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6802f690 con 0x55fc4d858e40 2024-04-23T10:32:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.621+0000 7f1c5dffb640 1 -- 172.21.15.97:0/2441525964 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f1c6802f830 con 0x55fc4d858e40 2024-04-23T10:32:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.621+0000 7f1c7f5e6740 1 -- 172.21.15.97:0/2441525964 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c68025620 con 0x55fc4d858e40 2024-04-23T10:32:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.622+0000 7f1c5dffb640 1 -- 172.21.15.97:0/2441525964 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1c6803c020 con 0x55fc4d858e40 2024-04-23T10:32:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.622+0000 7f1c7f5e6740 10 client.16437 fetch_fsmap learned FSMap version 26 2024-04-23T10:32:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.622+0000 7f1c7f5e6740 1 -- 172.21.15.97:0/2441525964 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fc4d8de450 con 0x55fc4d858e40 2024-04-23T10:32:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.622+0000 7f1c5dffb640 1 -- 172.21.15.97:0/2441525964 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c68025620 con 0x55fc4d858e40 2024-04-23T10:32:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.622+0000 7f1c5dffb640 1 -- 172.21.15.97:0/2441525964 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1c6801bb40 con 0x55fc4d858e40 2024-04-23T10:32:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.622+0000 7f1c7f5e6740 10 client.16437 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:32:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.622+0000 7f1c7f5e6740 10 client.16437 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:32:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.622+0000 7f1c7f5e6740 20 client.16437 populate_metadata read hostname 'smithi097' 2024-04-23T10:32:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.622+0000 7f1c7f5e6740 1 --2- 172.21.15.97:0/2441525964 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55fc4d8deb10 0x55fc4d8e0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:08.629 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.622+0000 7f1c7f5e6740 4 client.16437 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:32:08.629 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.622+0000 7f1c7f5e6740 1 -- 172.21.15.97:0/2441525964 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fc4d7c5200 con 0x55fc4d8deb10 2024-04-23T10:32:08.629 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.622+0000 7f1c7659c640 1 --2- 172.21.15.97:0/2441525964 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55fc4d8deb10 0x55fc4d8e0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:08.629 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.622+0000 7f1c5dffb640 10 client.16437 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:32:08.629 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.622+0000 7f1c7659c640 1 --2- 172.21.15.97:0/2441525964 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55fc4d8deb10 0x55fc4d8e0f30 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e22e661e9faeda75 server_cookie=e04a8711f8acb7c2 in_seq=0 out_seq=0 2024-04-23T10:32:08.634 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.634+0000 7f1c5dffb640 1 -- 172.21.15.97:0/2441525964 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fc4d7c5200 con 0x55fc4d8deb10 2024-04-23T10:32:08.634 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.634+0000 7f1c5dffb640 10 client.16437 handle_command_reply: tid=0 2024-04-23T10:32:08.634 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.634+0000 7f1c7f5e6740 1 client.16437 shutdown 2024-04-23T10:32:08.634 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.635+0000 7f1c7f5e6740 1 -- 172.21.15.97:0/2441525964 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fc4d858e40 msgr2=0x55fc4d8614d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:08.634 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.635+0000 7f1c7f5e6740 1 --2- 172.21.15.97:0/2441525964 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fc4d858e40 0x55fc4d8614d0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f1c68003ac0 tx=0x7f1c6800b5e0 comp rx=0 tx=0).stop 2024-04-23T10:32:08.634 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.635+0000 7f1c7f5e6740 1 -- 172.21.15.97:0/2441525964 shutdown_connections 2024-04-23T10:32:08.634 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.635+0000 7f1c7f5e6740 1 --2- 172.21.15.97:0/2441525964 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55fc4d8deb10 0x55fc4d8e0f30 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:08.634 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.635+0000 7f1c7f5e6740 1 --2- 172.21.15.97:0/2441525964 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fc4d867c10 0x55fc4d861c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:08.634 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.635+0000 7f1c5dffb640 0 client.16437 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:32:08.635 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.635+0000 7f1c7f5e6740 1 --2- 172.21.15.97:0/2441525964 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55fc4d858e40 0x55fc4d8614d0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:08.635 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.635+0000 7f1c7f5e6740 1 --2- 172.21.15.97:0/2441525964 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fc4d858520 0x55fc4d8700f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:08.635 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.635+0000 7f1c7f5e6740 1 -- 172.21.15.97:0/2441525964 >> 172.21.15.97:0/2441525964 conn(0x55fc4d854b30 msgr2=0x55fc4d85ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:32:08.635 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.635+0000 7f1c7f5e6740 1 -- 172.21.15.97:0/2441525964 shutdown_connections 2024-04-23T10:32:08.635 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.635+0000 7f1c7f5e6740 1 -- 172.21.15.97:0/2441525964 wait complete. 2024-04-23T10:32:08.635 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.635+0000 7f1c7f5e6740 20 client.16437 trim_cache size 0 max 16384 2024-04-23T10:32:08.635 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:32:08.635 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:32:08.635 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:32:08.635 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:32:08.636 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.635+0000 7f1c7c8cf640 1 -- 172.21.15.97:0/695123903 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f1c640a3dc0 msgr2=0x7f1c640a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:08.636 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.635+0000 7f1c7c8cf640 1 --2- 172.21.15.97:0/695123903 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f1c640a3dc0 0x7f1c640a6280 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1c70031670 tx=0x7f1c70043000 comp rx=0 tx=0).stop 2024-04-23T10:32:08.636 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.635+0000 7f1c7c8cf640 1 -- 172.21.15.97:0/695123903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c781021c0 msgr2=0x7f1c781985e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:08.636 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.635+0000 7f1c7c8cf640 1 --2- 172.21.15.97:0/695123903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c781021c0 0x7f1c781985e0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f1c6000c920 tx=0x7f1c6000cde0 comp rx=0 tx=0).stop 2024-04-23T10:32:08.636 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.635+0000 7f1c7c8cf640 1 -- 172.21.15.97:0/695123903 shutdown_connections 2024-04-23T10:32:08.637 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.635+0000 7f1c7c8cf640 1 --2- 172.21.15.97:0/695123903 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f1c640a3dc0 0x7f1c640a6280 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:08.637 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.635+0000 7f1c7c8cf640 1 --2- 172.21.15.97:0/695123903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c7819cea0 0x7f1c7819d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:08.637 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.635+0000 7f1c7c8cf640 1 --2- 172.21.15.97:0/695123903 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1c78107290 0x7f1c78198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:08.637 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.635+0000 7f1c7c8cf640 1 --2- 172.21.15.97:0/695123903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c781021c0 0x7f1c781985e0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:08.637 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.635+0000 7f1c7c8cf640 1 -- 172.21.15.97:0/695123903 >> 172.21.15.97:0/695123903 conn(0x7f1c780fda20 msgr2=0x7f1c780fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:32:08.637 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.636+0000 7f1c7c8cf640 1 -- 172.21.15.97:0/695123903 shutdown_connections 2024-04-23T10:32:08.637 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:08.636+0000 7f1c7c8cf640 1 -- 172.21.15.97:0/695123903 wait complete. 2024-04-23T10:32:09.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:32:08 smithi196 ceph-mon[32919]: pgmap v181: 97 pgs: 97 active+clean; 2.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 32 KiB/s wr, 30 op/s 2024-04-23T10:32:09.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:32:08 smithi097 ceph-mon[29197]: pgmap v181: 97 pgs: 97 active+clean; 2.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 32 KiB/s wr, 30 op/s 2024-04-23T10:32:09.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:32:08 smithi183 ceph-mon[32317]: pgmap v181: 97 pgs: 97 active+clean; 2.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 32 KiB/s wr, 30 op/s 2024-04-23T10:32:11.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:32:10 smithi196 ceph-mon[32919]: pgmap v182: 97 pgs: 97 active+clean; 2.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 27 KiB/s wr, 23 op/s 2024-04-23T10:32:11.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:32:10 smithi097 ceph-mon[29197]: pgmap v182: 97 pgs: 97 active+clean; 2.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 27 KiB/s wr, 23 op/s 2024-04-23T10:32:11.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:32:10 smithi183 ceph-mon[32317]: pgmap v182: 97 pgs: 97 active+clean; 2.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 27 KiB/s wr, 23 op/s 2024-04-23T10:32:13.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:32:13 smithi196 ceph-mon[32919]: pgmap v183: 97 pgs: 97 active+clean; 2.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 27 KiB/s wr, 23 op/s 2024-04-23T10:32:13.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:32:13 smithi097 ceph-mon[29197]: pgmap v183: 97 pgs: 97 active+clean; 2.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 27 KiB/s wr, 23 op/s 2024-04-23T10:32:13.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:32:13 smithi183 ceph-mon[32317]: pgmap v183: 97 pgs: 97 active+clean; 2.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 27 KiB/s wr, 23 op/s 2024-04-23T10:32:15.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:32:15 smithi196 ceph-mon[32919]: pgmap v184: 97 pgs: 97 active+clean; 2.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 34 KiB/s wr, 30 op/s 2024-04-23T10:32:15.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:32:15 smithi097 ceph-mon[29197]: pgmap v184: 97 pgs: 97 active+clean; 2.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 34 KiB/s wr, 30 op/s 2024-04-23T10:32:15.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:32:15 smithi183 ceph-mon[32317]: pgmap v184: 97 pgs: 97 active+clean; 2.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 34 KiB/s wr, 30 op/s 2024-04-23T10:32:17.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:32:17 smithi196 ceph-mon[32919]: pgmap v185: 97 pgs: 97 active+clean; 2.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-04-23T10:32:17.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:32:17 smithi097 ceph-mon[29197]: pgmap v185: 97 pgs: 97 active+clean; 2.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-04-23T10:32:17.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:32:17 smithi183 ceph-mon[32317]: pgmap v185: 97 pgs: 97 active+clean; 2.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-04-23T10:32:19.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:32:19 smithi196 ceph-mon[32919]: pgmap v186: 97 pgs: 97 active+clean; 2.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-04-23T10:32:19.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:32:19 smithi097 ceph-mon[29197]: pgmap v186: 97 pgs: 97 active+clean; 2.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-04-23T10:32:19.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:32:19 smithi183 ceph-mon[32317]: pgmap v186: 97 pgs: 97 active+clean; 2.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-04-23T10:32:21.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:32:21 smithi196 ceph-mon[32919]: pgmap v187: 97 pgs: 97 active+clean; 2.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.3 KiB/s wr, 11 op/s 2024-04-23T10:32:21.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:32:21 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:32:21.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:32:21 smithi097 ceph-mon[29197]: pgmap v187: 97 pgs: 97 active+clean; 2.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.3 KiB/s wr, 11 op/s 2024-04-23T10:32:21.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:32:21 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:32:21.399 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:32:21 smithi183 ceph-mon[32317]: pgmap v187: 97 pgs: 97 active+clean; 2.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.3 KiB/s wr, 11 op/s 2024-04-23T10:32:21.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:32:21 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:32:21.723 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.724+0000 7f3767cfd640 1 -- 172.21.15.97:0/2176614878 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37601010f0 msgr2=0x7f37601014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:21.723 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.724+0000 7f3767cfd640 1 --2- 172.21.15.97:0/2176614878 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37601010f0 0x7f37601014d0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f3750009900 tx=0x7f375002a410 comp rx=0 tx=0).stop 2024-04-23T10:32:21.723 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.724+0000 7f3767cfd640 1 -- 172.21.15.97:0/2176614878 shutdown_connections 2024-04-23T10:32:21.723 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.724+0000 7f3767cfd640 1 --2- 172.21.15.97:0/2176614878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3760106170 0x7f376010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:21.723 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.724+0000 7f3767cfd640 1 --2- 172.21.15.97:0/2176614878 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3760101aa0 0x7f3760105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:21.723 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.724+0000 7f3767cfd640 1 --2- 172.21.15.97:0/2176614878 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37601010f0 0x7f37601014d0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:21.723 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.724+0000 7f3767cfd640 1 -- 172.21.15.97:0/2176614878 >> 172.21.15.97:0/2176614878 conn(0x7f3760077fd0 msgr2=0x7f37600783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:32:21.723 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.724+0000 7f3767cfd640 1 -- 172.21.15.97:0/2176614878 shutdown_connections 2024-04-23T10:32:21.724 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.724+0000 7f3767cfd640 1 -- 172.21.15.97:0/2176614878 wait complete. 2024-04-23T10:32:21.724 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.724+0000 7f3767cfd640 1 Processor -- start 2024-04-23T10:32:21.724 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.724+0000 7f3767cfd640 1 -- start start 2024-04-23T10:32:21.725 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.725+0000 7f3767cfd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37601010f0 0x7f3760102b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:21.726 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.725+0000 7f3767cfd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3760106170 0x7f3760103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:21.726 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.725+0000 7f3767cfd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37601035b0 0x7f37601a6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:21.726 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.725+0000 7f3767cfd640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f37601110b0 con 0x7f3760106170 2024-04-23T10:32:21.726 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.725+0000 7f3767cfd640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3760110f30 con 0x7f37601035b0 2024-04-23T10:32:21.726 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.725+0000 7f3767cfd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3760111230 con 0x7f37601010f0 2024-04-23T10:32:21.726 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.725+0000 7f3765298640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3760106170 0x7f3760103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:21.726 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.725+0000 7f3765298640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3760106170 0x7f3760103070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60378/0 (socket says 172.21.15.97:60378) 2024-04-23T10:32:21.726 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.725+0000 7f3765298640 1 -- 172.21.15.97:0/644208265 learned_addr learned my addr 172.21.15.97:0/644208265 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:32:21.726 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.725+0000 7f376629a640 1 --2- 172.21.15.97:0/644208265 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37601035b0 0x7f37601a6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:21.726 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.725+0000 7f3765a99640 1 --2- 172.21.15.97:0/644208265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37601010f0 0x7f3760102b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:21.726 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.725+0000 7f3765298640 1 -- 172.21.15.97:0/644208265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37601010f0 msgr2=0x7f3760102b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:21.726 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.725+0000 7f3765298640 1 --2- 172.21.15.97:0/644208265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37601010f0 0x7f3760102b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:21.726 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.725+0000 7f3765298640 1 -- 172.21.15.97:0/644208265 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37601035b0 msgr2=0x7f37601a6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:21.727 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.725+0000 7f3765298640 1 --2- 172.21.15.97:0/644208265 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37601035b0 0x7f37601a6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:21.727 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.725+0000 7f3765298640 1 -- 172.21.15.97:0/644208265 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37601a6fd0 con 0x7f3760106170 2024-04-23T10:32:21.727 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.725+0000 7f3765a99640 1 --2- 172.21.15.97:0/644208265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37601010f0 0x7f3760102b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:32:21.727 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.725+0000 7f3765298640 1 --2- 172.21.15.97:0/644208265 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3760106170 0x7f3760103070 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f375c00c970 tx=0x7f375c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:32:21.727 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.725+0000 7f3756ffd640 1 -- 172.21.15.97:0/644208265 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375c007bb0 con 0x7f3760106170 2024-04-23T10:32:21.727 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.725+0000 7f3756ffd640 1 -- 172.21.15.97:0/644208265 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f375c007d50 con 0x7f3760106170 2024-04-23T10:32:21.727 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.725+0000 7f3767cfd640 1 -- 172.21.15.97:0/644208265 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37601a72c0 con 0x7f3760106170 2024-04-23T10:32:21.727 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.725+0000 7f3756ffd640 1 -- 172.21.15.97:0/644208265 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375c00f450 con 0x7f3760106170 2024-04-23T10:32:21.727 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.725+0000 7f3767cfd640 1 -- 172.21.15.97:0/644208265 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37601a7860 con 0x7f3760106170 2024-04-23T10:32:21.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.728+0000 7f3756ffd640 1 -- 172.21.15.97:0/644208265 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f375c004f20 con 0x7f3760106170 2024-04-23T10:32:21.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.728+0000 7f376aa10740 1 Processor -- start 2024-04-23T10:32:21.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.728+0000 7f376aa10740 1 -- start start 2024-04-23T10:32:21.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.729+0000 7f376aa10740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed73c4e620 0x55ed73c4ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:21.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.729+0000 7f376aa10740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed73c4ef40 0x55ed73c592c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:21.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.729+0000 7f376aa10740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed73c59800 0x55ed73c5bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:21.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.729+0000 7f376aa10740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ed73bb3970 con 0x55ed73c59800 2024-04-23T10:32:21.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.729+0000 7f376aa10740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ed73b504a0 con 0x55ed73c4ef40 2024-04-23T10:32:21.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.729+0000 7f376aa10740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ed73baf930 con 0x55ed73c4e620 2024-04-23T10:32:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.729+0000 7f3756ffd640 1 --2- 172.21.15.97:0/644208265 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f374c0a3e40 0x7f374c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.729+0000 7f3765a99640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed73c4ef40 0x55ed73c592c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.729+0000 7f3765298640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed73c59800 0x55ed73c5bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.729+0000 7f3765a99640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed73c4ef40 0x55ed73c592c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.97:44920/0 (socket says 172.21.15.97:44920) 2024-04-23T10:32:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.729+0000 7f3765a99640 1 -- 172.21.15.97:0/2952454730 learned_addr learned my addr 172.21.15.97:0/2952454730 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:32:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.729+0000 7f376629a640 1 --2- 172.21.15.97:0/2952454730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed73c4e620 0x55ed73c4ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.729+0000 7f376629a640 1 --2- 172.21.15.97:0/644208265 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f374c0a3e40 0x7f374c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.729+0000 7f3756ffd640 1 -- 172.21.15.97:0/644208265 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f375c0d07d0 con 0x7f3760106170 2024-04-23T10:32:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.729+0000 7f3765a99640 1 -- 172.21.15.97:0/2952454730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed73c4e620 msgr2=0x55ed73c4ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:32:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.729+0000 7f3765a99640 1 --2- 172.21.15.97:0/2952454730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed73c4e620 0x55ed73c4ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.729+0000 7f3765a99640 1 -- 172.21.15.97:0/2952454730 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed73c59800 msgr2=0x55ed73c5bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:32:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.729+0000 7f3765a99640 1 --2- 172.21.15.97:0/2952454730 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed73c59800 0x55ed73c5bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.729+0000 7f3765a99640 1 -- 172.21.15.97:0/2952454730 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed738c6210 con 0x55ed73c4ef40 2024-04-23T10:32:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.729+0000 7f3765298640 1 --2- 172.21.15.97:0/2952454730 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed73c59800 0x55ed73c5bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T10:32:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.729+0000 7f376629a640 1 --2- 172.21.15.97:0/2952454730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed73c4e620 0x55ed73c4ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T10:32:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.730+0000 7f376629a640 1 --2- 172.21.15.97:0/644208265 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f374c0a3e40 0x7f374c0a6300 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3758009fd0 tx=0x7f3758009600 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:32:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.730+0000 7f3765a99640 1 --2- 172.21.15.97:0/2952454730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed73c4ef40 0x55ed73c592c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f375002a930 tx=0x7f3750032e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=172387c15ebd9bd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:32:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.730+0000 7f374bfff640 1 -- 172.21.15.97:0/2952454730 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375003f070 con 0x55ed73c4ef40 2024-04-23T10:32:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.730+0000 7f374bfff640 1 -- 172.21.15.97:0/2952454730 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3750031760 con 0x55ed73c4ef40 2024-04-23T10:32:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.730+0000 7f374bfff640 1 -- 172.21.15.97:0/2952454730 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3750035070 con 0x55ed73c4ef40 2024-04-23T10:32:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.731+0000 7f376aa10740 1 -- 172.21.15.97:0/2952454730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed73c4ef40 msgr2=0x55ed73c592c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.731+0000 7f376aa10740 1 --2- 172.21.15.97:0/2952454730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed73c4ef40 0x55ed73c592c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f375002a930 tx=0x7f3750032e70 comp rx=0 tx=0).stop 2024-04-23T10:32:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.731+0000 7f376aa10740 1 -- 172.21.15.97:0/2952454730 shutdown_connections 2024-04-23T10:32:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.731+0000 7f376aa10740 1 --2- 172.21.15.97:0/2952454730 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed73c59800 0x55ed73c5bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.731+0000 7f376aa10740 1 --2- 172.21.15.97:0/2952454730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed73c4ef40 0x55ed73c592c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.731+0000 7f376aa10740 1 --2- 172.21.15.97:0/2952454730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed73c4e620 0x55ed73c4ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.731+0000 7f376aa10740 1 -- 172.21.15.97:0/2952454730 >> 172.21.15.97:0/2952454730 conn(0x55ed73c4ab30 msgr2=0x55ed73c4af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:32:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.731+0000 7f376aa10740 1 -- 172.21.15.97:0/2952454730 shutdown_connections 2024-04-23T10:32:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.731+0000 7f376aa10740 1 -- 172.21.15.97:0/2952454730 wait complete. 2024-04-23T10:32:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.731+0000 7f376aa10740 1 -- start start 2024-04-23T10:32:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.731+0000 7f376aa10740 1 Processor -- start 2024-04-23T10:32:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.731+0000 7f376aa10740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed73c4e620 0x55ed73c591a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.732+0000 7f376aa10740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed73c4ef40 0x55ed73c56490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.732+0000 7f376aa10740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed73c59800 0x55ed73c56a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.732+0000 7f376aa10740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ed73c5c3d0 con 0x55ed73c4e620 2024-04-23T10:32:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.732+0000 7f376aa10740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ed73b4e0b0 con 0x55ed73c4ef40 2024-04-23T10:32:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.732+0000 7f376aa10740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ed73c5c550 con 0x55ed73c59800 2024-04-23T10:32:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.732+0000 7f376aa10740 10 client.0 _ll_register_callbacks cb 0x55ed73b3bd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:32:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.732+0000 7f3765a99640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed73c4e620 0x55ed73c591a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.732+0000 7f3765a99640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed73c4e620 0x55ed73c591a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60398/0 (socket says 172.21.15.97:60398) 2024-04-23T10:32:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.732+0000 7f3765a99640 1 -- 172.21.15.97:0/1911235869 learned_addr learned my addr 172.21.15.97:0/1911235869 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:32:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.732+0000 7f3765298640 1 --2- 172.21.15.97:0/1911235869 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed73c4ef40 0x55ed73c56490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.732+0000 7f376629a640 1 --2- 172.21.15.97:0/1911235869 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed73c59800 0x55ed73c56a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.732+0000 7f3765a99640 1 -- 172.21.15.97:0/1911235869 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed73c59800 msgr2=0x55ed73c56a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:32:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.732+0000 7f3765a99640 1 --2- 172.21.15.97:0/1911235869 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed73c59800 0x55ed73c56a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.732+0000 7f3765a99640 1 -- 172.21.15.97:0/1911235869 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed73c4ef40 msgr2=0x55ed73c56490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:32:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.732+0000 7f3765a99640 1 --2- 172.21.15.97:0/1911235869 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed73c4ef40 0x55ed73c56490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.732+0000 7f3765a99640 1 -- 172.21.15.97:0/1911235869 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed73992620 con 0x55ed73c4e620 2024-04-23T10:32:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.732+0000 7f3765298640 1 --2- 172.21.15.97:0/1911235869 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed73c4ef40 0x55ed73c56490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T10:32:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.732+0000 7f3765a99640 1 --2- 172.21.15.97:0/1911235869 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed73c4e620 0x55ed73c591a0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f3750006010 tx=0x7f375002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e145cb0ad516c9be server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:32:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.732+0000 7f37497fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:32:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.732+0000 7f37497fa640 1 -- 172.21.15.97:0/1911235869 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375003f070 con 0x55ed73c4e620 2024-04-23T10:32:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.732+0000 7f37497fa640 1 -- 172.21.15.97:0/1911235869 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f375003cbf0 con 0x55ed73c4e620 2024-04-23T10:32:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.732+0000 7f376aa10740 1 -- 172.21.15.97:0/1911235869 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3750031760 con 0x55ed73c4e620 2024-04-23T10:32:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.732+0000 7f37497fa640 1 -- 172.21.15.97:0/1911235869 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3750054050 con 0x55ed73c4e620 2024-04-23T10:32:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.733+0000 7f376aa10740 10 client.16449 fetch_fsmap learned FSMap version 26 2024-04-23T10:32:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.733+0000 7f376aa10740 1 -- 172.21.15.97:0/1911235869 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ed73c68340 con 0x55ed73c4e620 2024-04-23T10:32:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.733+0000 7f37497fa640 1 -- 172.21.15.97:0/1911235869 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3750031760 con 0x55ed73c4e620 2024-04-23T10:32:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.733+0000 7f37497fa640 1 -- 172.21.15.97:0/1911235869 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f37500035f0 con 0x55ed73c4e620 2024-04-23T10:32:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.733+0000 7f376aa10740 10 client.16449 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:32:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.733+0000 7f376aa10740 10 client.16449 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:32:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.733+0000 7f376aa10740 20 client.16449 populate_metadata read hostname 'smithi097' 2024-04-23T10:32:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.733+0000 7f376aa10740 1 --2- 172.21.15.97:0/1911235869 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55ed73c69690 0x55ed73c6ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.733+0000 7f376aa10740 4 client.16449 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:32:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.733+0000 7f376aa10740 1 -- 172.21.15.97:0/1911235869 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed73bc1390 con 0x55ed73c69690 2024-04-23T10:32:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.733+0000 7f3765298640 1 --2- 172.21.15.97:0/1911235869 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55ed73c69690 0x55ed73c6ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.733+0000 7f37497fa640 10 client.16449 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:32:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.734+0000 7f3765298640 1 --2- 172.21.15.97:0/1911235869 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55ed73c69690 0x55ed73c6ba80 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e106d24b899ec99 server_cookie=cdab53ea1f7ae67 in_seq=0 out_seq=0 2024-04-23T10:32:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.734+0000 7f37497fa640 1 -- 172.21.15.97:0/1911235869 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ed73bc1390 con 0x55ed73c69690 2024-04-23T10:32:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.734+0000 7f37497fa640 10 client.16449 handle_command_reply: tid=0 2024-04-23T10:32:21.740 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.734+0000 7f376aa10740 1 client.16449 shutdown 2024-04-23T10:32:21.740 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.734+0000 7f376aa10740 1 -- 172.21.15.97:0/1911235869 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed73c4e620 msgr2=0x55ed73c591a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:21.740 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.734+0000 7f376aa10740 1 --2- 172.21.15.97:0/1911235869 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed73c4e620 0x55ed73c591a0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f3750006010 tx=0x7f375002a410 comp rx=0 tx=0).stop 2024-04-23T10:32:21.740 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.734+0000 7f376aa10740 1 -- 172.21.15.97:0/1911235869 shutdown_connections 2024-04-23T10:32:21.740 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.734+0000 7f376aa10740 1 --2- 172.21.15.97:0/1911235869 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55ed73c69690 0x55ed73c6ba80 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:21.740 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.734+0000 7f376aa10740 1 --2- 172.21.15.97:0/1911235869 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed73c59800 0x55ed73c56a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:21.740 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.734+0000 7f37497fa640 0 client.16449 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:32:21.740 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.734+0000 7f376aa10740 1 --2- 172.21.15.97:0/1911235869 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed73c4ef40 0x55ed73c56490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:21.740 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.734+0000 7f376aa10740 1 --2- 172.21.15.97:0/1911235869 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed73c4e620 0x55ed73c591a0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:21.740 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.734+0000 7f376aa10740 1 -- 172.21.15.97:0/1911235869 >> 172.21.15.97:0/1911235869 conn(0x55ed73c4ab30 msgr2=0x55ed73c50860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:32:21.740 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.734+0000 7f376aa10740 1 -- 172.21.15.97:0/1911235869 shutdown_connections 2024-04-23T10:32:21.740 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.734+0000 7f376aa10740 1 -- 172.21.15.97:0/1911235869 wait complete. 2024-04-23T10:32:21.740 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.734+0000 7f376aa10740 20 client.16449 trim_cache size 0 max 16384 2024-04-23T10:32:21.752 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.749+0000 7f376aa10740 1 Processor -- start 2024-04-23T10:32:21.752 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.749+0000 7f376aa10740 1 -- start start 2024-04-23T10:32:21.752 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.749+0000 7f376aa10740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed73c4e620 0x55ed73c58cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:21.752 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.749+0000 7f376aa10740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed73c4ef40 0x55ed73c536f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.749+0000 7f376aa10740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed73c59800 0x55ed73c53c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.749+0000 7f376aa10740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f37601afc60 con 0x55ed73c4ef40 2024-04-23T10:32:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.749+0000 7f376aa10740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f37601b17a0 con 0x55ed73c59800 2024-04-23T10:32:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.749+0000 7f376aa10740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f37601b0330 con 0x55ed73c4e620 2024-04-23T10:32:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.749+0000 7f3765298640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed73c4ef40 0x55ed73c536f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.749+0000 7f3765298640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed73c4ef40 0x55ed73c536f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60414/0 (socket says 172.21.15.97:60414) 2024-04-23T10:32:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.749+0000 7f3765298640 1 -- 172.21.15.97:0/2737592342 learned_addr learned my addr 172.21.15.97:0/2737592342 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:32:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.750+0000 7f3765a99640 1 --2- 172.21.15.97:0/2737592342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed73c4e620 0x55ed73c58cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.750+0000 7f3765298640 1 -- 172.21.15.97:0/2737592342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed73c4e620 msgr2=0x55ed73c58cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:32:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.750+0000 7f3765298640 1 --2- 172.21.15.97:0/2737592342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed73c4e620 0x55ed73c58cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.750+0000 7f3765298640 1 -- 172.21.15.97:0/2737592342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed73c59800 msgr2=0x55ed73c53c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:32:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.750+0000 7f3765298640 1 --2- 172.21.15.97:0/2737592342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed73c59800 0x55ed73c53c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.750+0000 7f3765298640 1 -- 172.21.15.97:0/2737592342 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed73c4ed70 con 0x55ed73c4ef40 2024-04-23T10:32:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.750+0000 7f3765298640 1 --2- 172.21.15.97:0/2737592342 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed73c4ef40 0x55ed73c536f0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f375c0cdf70 tx=0x7f375c0d2c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cbe43aa4ba144ed server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:32:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.750+0000 7f374b7fe640 1 -- 172.21.15.97:0/2737592342 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375c01d040 con 0x55ed73c4ef40 2024-04-23T10:32:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.750+0000 7f374b7fe640 1 -- 172.21.15.97:0/2737592342 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f375c0870b0 con 0x55ed73c4ef40 2024-04-23T10:32:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.750+0000 7f374b7fe640 1 -- 172.21.15.97:0/2737592342 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375c0ce190 con 0x55ed73c4ef40 2024-04-23T10:32:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.750+0000 7f376aa10740 1 -- 172.21.15.97:0/2737592342 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed73c4ef40 msgr2=0x55ed73c536f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.750+0000 7f376aa10740 1 --2- 172.21.15.97:0/2737592342 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed73c4ef40 0x55ed73c536f0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f375c0cdf70 tx=0x7f375c0d2c20 comp rx=0 tx=0).stop 2024-04-23T10:32:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.751+0000 7f376aa10740 1 -- 172.21.15.97:0/2737592342 shutdown_connections 2024-04-23T10:32:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.751+0000 7f376aa10740 1 --2- 172.21.15.97:0/2737592342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed73c59800 0x55ed73c53c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.751+0000 7f376aa10740 1 --2- 172.21.15.97:0/2737592342 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed73c4ef40 0x55ed73c536f0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.751+0000 7f376aa10740 1 --2- 172.21.15.97:0/2737592342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed73c4e620 0x55ed73c58cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.751+0000 7f376aa10740 1 -- 172.21.15.97:0/2737592342 >> 172.21.15.97:0/2737592342 conn(0x55ed73c4ab30 msgr2=0x55ed73c56260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:32:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.751+0000 7f376aa10740 1 -- 172.21.15.97:0/2737592342 shutdown_connections 2024-04-23T10:32:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.751+0000 7f376aa10740 1 -- 172.21.15.97:0/2737592342 wait complete. 2024-04-23T10:32:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.751+0000 7f376aa10740 1 -- start start 2024-04-23T10:32:21.754 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.751+0000 7f376aa10740 1 Processor -- start 2024-04-23T10:32:21.755 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.751+0000 7f376aa10740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed73c4e620 0x55ed73cc7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:21.755 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.751+0000 7f376aa10740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed73c4ef40 0x55ed73cc8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:21.755 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.751+0000 7f376aa10740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed73c59800 0x55ed73c6e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:21.755 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.751+0000 7f376aa10740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f37601b10d0 con 0x55ed73c4ef40 2024-04-23T10:32:21.755 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.751+0000 7f376aa10740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f37601b0a00 con 0x55ed73c59800 2024-04-23T10:32:21.755 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.751+0000 7f376aa10740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f37601adc60 con 0x55ed73c4e620 2024-04-23T10:32:21.755 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.751+0000 7f376aa10740 10 client.0 _ll_register_callbacks cb 0x7f3750032d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:32:21.755 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.751+0000 7f3765298640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed73c4ef40 0x55ed73cc8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:21.755 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.751+0000 7f3765298640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed73c4ef40 0x55ed73cc8520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60418/0 (socket says 172.21.15.97:60418) 2024-04-23T10:32:21.755 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.751+0000 7f3765298640 1 -- 172.21.15.97:0/748934126 learned_addr learned my addr 172.21.15.97:0/748934126 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:32:21.755 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.752+0000 7f3765a99640 1 --2- 172.21.15.97:0/748934126 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed73c4e620 0x55ed73cc7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:21.755 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.752+0000 7f3765298640 1 -- 172.21.15.97:0/748934126 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed73c4e620 msgr2=0x55ed73cc7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:32:21.755 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.752+0000 7f3765298640 1 --2- 172.21.15.97:0/748934126 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed73c4e620 0x55ed73cc7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:21.755 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.752+0000 7f3765298640 1 -- 172.21.15.97:0/748934126 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed73c59800 msgr2=0x55ed73c6e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:32:21.755 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.752+0000 7f3765298640 1 --2- 172.21.15.97:0/748934126 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed73c59800 0x55ed73c6e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:21.756 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.752+0000 7f3765298640 1 -- 172.21.15.97:0/748934126 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed73c6e8e0 con 0x55ed73c4ef40 2024-04-23T10:32:21.756 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.752+0000 7f3765298640 1 --2- 172.21.15.97:0/748934126 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed73c4ef40 0x55ed73cc8520 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f375c09c6f0 tx=0x7f375c0a0c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db491fface8420fa server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:32:21.756 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.752+0000 7f37497fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:32:21.756 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.752+0000 7f37497fa640 1 -- 172.21.15.97:0/748934126 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375c01d040 con 0x55ed73c4ef40 2024-04-23T10:32:21.756 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.752+0000 7f37497fa640 1 -- 172.21.15.97:0/748934126 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f375c005210 con 0x55ed73c4ef40 2024-04-23T10:32:21.756 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.752+0000 7f376aa10740 1 -- 172.21.15.97:0/748934126 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f375c0870b0 con 0x55ed73c4ef40 2024-04-23T10:32:21.756 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.752+0000 7f37497fa640 1 -- 172.21.15.97:0/748934126 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f375c005410 con 0x55ed73c4ef40 2024-04-23T10:32:21.756 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.752+0000 7f376aa10740 10 client.16455 fetch_fsmap learned FSMap version 26 2024-04-23T10:32:21.756 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.752+0000 7f376aa10740 1 -- 172.21.15.97:0/748934126 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ed73ccf810 con 0x55ed73c4ef40 2024-04-23T10:32:21.756 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.752+0000 7f37497fa640 1 -- 172.21.15.97:0/748934126 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f375c0870b0 con 0x55ed73c4ef40 2024-04-23T10:32:21.756 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.752+0000 7f37497fa640 1 -- 172.21.15.97:0/748934126 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f375c0d0bc0 con 0x55ed73c4ef40 2024-04-23T10:32:21.756 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.752+0000 7f376aa10740 10 client.16455 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:32:21.756 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.752+0000 7f376aa10740 10 client.16455 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:32:21.756 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.752+0000 7f376aa10740 20 client.16455 populate_metadata read hostname 'smithi097' 2024-04-23T10:32:21.756 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.752+0000 7f376aa10740 1 --2- 172.21.15.97:0/748934126 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55ed73ccffb0 0x55ed73cd23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:21.756 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.753+0000 7f376aa10740 4 client.16455 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:32:21.757 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.753+0000 7f376aa10740 1 -- 172.21.15.97:0/748934126 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ed73bbb200 con 0x55ed73ccffb0 2024-04-23T10:32:21.757 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.753+0000 7f3765a99640 1 --2- 172.21.15.97:0/748934126 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55ed73ccffb0 0x55ed73cd23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:21.757 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.753+0000 7f3765a99640 1 --2- 172.21.15.97:0/748934126 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55ed73ccffb0 0x55ed73cd23d0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5a106bca701d39b server_cookie=90e9705c57fc306d in_seq=0 out_seq=0 2024-04-23T10:32:21.757 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.753+0000 7f37497fa640 10 client.16455 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:32:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.761+0000 7f37497fa640 1 -- 172.21.15.97:0/748934126 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ed73bbb200 con 0x55ed73ccffb0 2024-04-23T10:32:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.761+0000 7f37497fa640 10 client.16455 handle_command_reply: tid=0 2024-04-23T10:32:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.761+0000 7f376aa10740 1 client.16455 shutdown 2024-04-23T10:32:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.761+0000 7f376aa10740 1 -- 172.21.15.97:0/748934126 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed73c4ef40 msgr2=0x55ed73cc8520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.761+0000 7f376aa10740 1 --2- 172.21.15.97:0/748934126 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed73c4ef40 0x55ed73cc8520 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f375c09c6f0 tx=0x7f375c0a0c90 comp rx=0 tx=0).stop 2024-04-23T10:32:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.761+0000 7f376aa10740 1 -- 172.21.15.97:0/748934126 shutdown_connections 2024-04-23T10:32:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.761+0000 7f376aa10740 1 --2- 172.21.15.97:0/748934126 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55ed73ccffb0 0x55ed73cd23d0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.761+0000 7f376aa10740 1 --2- 172.21.15.97:0/748934126 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ed73c59800 0x55ed73c6e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.761+0000 7f376aa10740 1 --2- 172.21.15.97:0/748934126 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ed73c4ef40 0x55ed73cc8520 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.761+0000 7f37497fa640 0 client.16455 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:32:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.761+0000 7f376aa10740 1 --2- 172.21.15.97:0/748934126 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed73c4e620 0x55ed73cc7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.761+0000 7f376aa10740 1 -- 172.21.15.97:0/748934126 >> 172.21.15.97:0/748934126 conn(0x55ed73c4ab30 msgr2=0x55ed73c559d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:32:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.762+0000 7f376aa10740 1 -- 172.21.15.97:0/748934126 shutdown_connections 2024-04-23T10:32:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.762+0000 7f376aa10740 1 -- 172.21.15.97:0/748934126 wait complete. 2024-04-23T10:32:21.762 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.762+0000 7f376aa10740 20 client.16455 trim_cache size 0 max 16384 2024-04-23T10:32:21.762 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:32:21.762 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:32:21.762 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:32:21.762 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:32:21.763 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.762+0000 7f3767cfd640 1 -- 172.21.15.97:0/644208265 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f374c0a3e40 msgr2=0x7f374c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:21.763 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.762+0000 7f3767cfd640 1 --2- 172.21.15.97:0/644208265 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f374c0a3e40 0x7f374c0a6300 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3758009fd0 tx=0x7f3758009600 comp rx=0 tx=0).stop 2024-04-23T10:32:21.763 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.762+0000 7f3767cfd640 1 -- 172.21.15.97:0/644208265 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3760106170 msgr2=0x7f3760103070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:21.763 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.762+0000 7f3767cfd640 1 --2- 172.21.15.97:0/644208265 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3760106170 0x7f3760103070 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f375c00c970 tx=0x7f375c00ce30 comp rx=0 tx=0).stop 2024-04-23T10:32:21.763 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.762+0000 7f3767cfd640 1 -- 172.21.15.97:0/644208265 shutdown_connections 2024-04-23T10:32:21.763 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.762+0000 7f3767cfd640 1 --2- 172.21.15.97:0/644208265 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f374c0a3e40 0x7f374c0a6300 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:21.763 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.762+0000 7f3767cfd640 1 --2- 172.21.15.97:0/644208265 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37601035b0 0x7f37601a6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:21.764 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.762+0000 7f3767cfd640 1 --2- 172.21.15.97:0/644208265 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3760106170 0x7f3760103070 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:21.764 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.762+0000 7f3767cfd640 1 --2- 172.21.15.97:0/644208265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37601010f0 0x7f3760102b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:21.764 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.762+0000 7f3767cfd640 1 -- 172.21.15.97:0/644208265 >> 172.21.15.97:0/644208265 conn(0x7f3760077fd0 msgr2=0x7f3760069a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:32:21.764 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.762+0000 7f3767cfd640 1 -- 172.21.15.97:0/644208265 shutdown_connections 2024-04-23T10:32:21.764 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:21.762+0000 7f3767cfd640 1 -- 172.21.15.97:0/644208265 wait complete. 2024-04-23T10:32:22.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:32:22 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:32:22.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:32:22 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:32:22.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:32:22 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:32:22.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:32:22 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:32:22.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:32:22 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:32:22.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:32:22 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:32:22.399 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:32:22 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:32:22.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:32:22 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:32:22.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:32:22 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:32:22.650 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T10:32:22.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:22.755+0000 7f50287a2640 1 -- 172.21.15.97:0/778140058 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5020101a50 msgr2=0x7f5020105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:22.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:22.755+0000 7f50287a2640 1 --2- 172.21.15.97:0/778140058 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5020101a50 0x7f5020105a90 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f5010009900 tx=0x7f501002a410 comp rx=0 tx=0).stop 2024-04-23T10:32:22.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:22.756+0000 7f50287a2640 1 -- 172.21.15.97:0/778140058 shutdown_connections 2024-04-23T10:32:22.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:22.756+0000 7f50287a2640 1 --2- 172.21.15.97:0/778140058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5020106170 0x7f502010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:22.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:22.756+0000 7f50287a2640 1 --2- 172.21.15.97:0/778140058 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5020101a50 0x7f5020105a90 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:22.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:22.756+0000 7f50287a2640 1 --2- 172.21.15.97:0/778140058 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f50201010a0 0x7f5020101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:22.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:22.756+0000 7f50287a2640 1 -- 172.21.15.97:0/778140058 >> 172.21.15.97:0/778140058 conn(0x7f5020077f50 msgr2=0x7f5020078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:32:22.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:22.756+0000 7f50287a2640 1 -- 172.21.15.97:0/778140058 shutdown_connections 2024-04-23T10:32:22.755 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:22.756+0000 7f50287a2640 1 -- 172.21.15.97:0/778140058 wait complete. 2024-04-23T10:32:22.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:22.756+0000 7f50287a2640 1 Processor -- start 2024-04-23T10:32:22.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:22.756+0000 7f50287a2640 1 -- start start 2024-04-23T10:32:22.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:22.756+0000 7f50287a2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50201010a0 0x7f502019e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:22.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:22.756+0000 7f50287a2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5020106170 0x7f502019e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:22.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:22.756+0000 7f50287a2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5020198600 0x7f5020198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:22.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:22.756+0000 7f50287a2640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5020111130 con 0x7f5020198600 2024-04-23T10:32:22.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:22.756+0000 7f50287a2640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5020110fb0 con 0x7f50201010a0 2024-04-23T10:32:22.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:22.756+0000 7f50287a2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f50201112b0 con 0x7f5020106170 2024-04-23T10:32:22.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:22.757+0000 7f5026d3f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5020198600 0x7f5020198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:22.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:22.757+0000 7f5026d3f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5020198600 0x7f5020198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60424/0 (socket says 172.21.15.97:60424) 2024-04-23T10:32:22.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:22.757+0000 7f5026d3f640 1 -- 172.21.15.97:0/3563700448 learned_addr learned my addr 172.21.15.97:0/3563700448 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:32:22.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:22.757+0000 7f502653e640 1 --2- 172.21.15.97:0/3563700448 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50201010a0 0x7f502019e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:22.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:22.757+0000 7f5025d3d640 1 --2- 172.21.15.97:0/3563700448 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5020106170 0x7f502019e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:22.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:22.757+0000 7f5026d3f640 1 -- 172.21.15.97:0/3563700448 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5020106170 msgr2=0x7f502019e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:22.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:22.757+0000 7f5026d3f640 1 --2- 172.21.15.97:0/3563700448 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5020106170 0x7f502019e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:22.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:22.757+0000 7f5026d3f640 1 -- 172.21.15.97:0/3563700448 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50201010a0 msgr2=0x7f502019e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:22.758 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:22.757+0000 7f5026d3f640 1 --2- 172.21.15.97:0/3563700448 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50201010a0 0x7f502019e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:22.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:22.757+0000 7f5026d3f640 1 -- 172.21.15.97:0/3563700448 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5020199120 con 0x7f5020198600 2024-04-23T10:32:22.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:22.757+0000 7f5026d3f640 1 --2- 172.21.15.97:0/3563700448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5020198600 0x7f5020198ab0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f501c009fd0 tx=0x7f501c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:32:22.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:22.757+0000 7f50177fe640 1 -- 172.21.15.97:0/3563700448 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f501c011830 con 0x7f5020198600 2024-04-23T10:32:22.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:22.757+0000 7f50177fe640 1 -- 172.21.15.97:0/3563700448 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f501c016070 con 0x7f5020198600 2024-04-23T10:32:22.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:22.757+0000 7f50177fe640 1 -- 172.21.15.97:0/3563700448 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f501c010de0 con 0x7f5020198600 2024-04-23T10:32:22.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:22.757+0000 7f50287a2640 1 -- 172.21.15.97:0/3563700448 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5020199410 con 0x7f5020198600 2024-04-23T10:32:22.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:22.757+0000 7f50287a2640 1 -- 172.21.15.97:0/3563700448 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50201a5660 con 0x7f5020198600 2024-04-23T10:32:22.759 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:22.758+0000 7f50287a2640 1 -- 172.21.15.97:0/3563700448 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5020068d70 con 0x7f5020198600 2024-04-23T10:32:22.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:22.759+0000 7f50177fe640 1 -- 172.21.15.97:0/3563700448 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f501c0119d0 con 0x7f5020198600 2024-04-23T10:32:22.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:22.760+0000 7f50177fe640 1 --2- 172.21.15.97:0/3563700448 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f500c0a3da0 0x7f500c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:22.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:22.760+0000 7f502653e640 1 --2- 172.21.15.97:0/3563700448 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f500c0a3da0 0x7f500c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:22.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:22.760+0000 7f50177fe640 1 -- 172.21.15.97:0/3563700448 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f501c0cfd20 con 0x7f5020198600 2024-04-23T10:32:22.760 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:22.760+0000 7f502653e640 1 --2- 172.21.15.97:0/3563700448 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f500c0a3da0 0x7f500c0a6260 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5018005fd0 tx=0x7f5018009500 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:32:22.763 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:22.764+0000 7f50177fe640 1 -- 172.21.15.97:0/3563700448 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f501c09c520 con 0x7f5020198600 2024-04-23T10:32:22.977 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:22.979+0000 7f50287a2640 1 -- 172.21.15.97:0/3563700448 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f502010a440 con 0x7f5020198600 2024-04-23T10:32:22.978 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:22.979+0000 7f50177fe640 1 -- 172.21.15.97:0/3563700448 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11931 (secure 0 0 0) 0x7f501c09be20 con 0x7f5020198600 2024-04-23T10:32:22.978 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:32:22.979 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3946878762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3946878762},{"type":"v1","addr":"172.21.15.97:6835","nonce":3946878762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/209800861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":209800861},{"type":"v1","addr":"172.21.15.97:6837","nonce":209800861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T10:24:06.910134+0000","modified":"2024-04-23T10:26:09.820985+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6839/418013575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":418013575},{"type":"v1","addr":"172.21.15.97:6839","nonce":418013575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6841/747242680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":747242680},{"type":"v1","addr":"172.21.15.97:6841","nonce":747242680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6833/4292847121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4292847121},{"type":"v1","addr":"172.21.15.183:6833","nonce":4292847121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6833/3885816565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3885816565},{"type":"v1","addr":"172.21.15.196:6833","nonce":3885816565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1897093105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1897093105},{"type":"v1","addr":"172.21.15.183:6835","nonce":1897093105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1171477563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1171477563},{"type":"v1","addr":"172.21.15.196:6835","nonce":1171477563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1898341911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1898341911},{"type":"v1","addr":"172.21.15.183:6837","nonce":1898341911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.196:6837/3094946502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3094946502},{"type":"v1","addr":"172.21.15.196:6837","nonce":3094946502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.196:6839/83843556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":83843556},{"type":"v1","addr":"172.21.15.196:6839","nonce":83843556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6839/1763841130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1763841130},{"type":"v1","addr":"172.21.15.183:6839","nonce":1763841130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24487,24403,24398,24469,24488]},"id":1}]} 2024-04-23T10:32:22.979 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 26 2024-04-23T10:32:22.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:22.981+0000 7f50287a2640 1 -- 172.21.15.97:0/3563700448 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f500c0a3da0 msgr2=0x7f500c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:22.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:22.981+0000 7f50287a2640 1 --2- 172.21.15.97:0/3563700448 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f500c0a3da0 0x7f500c0a6260 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5018005fd0 tx=0x7f5018009500 comp rx=0 tx=0).stop 2024-04-23T10:32:22.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:22.981+0000 7f50287a2640 1 -- 172.21.15.97:0/3563700448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5020198600 msgr2=0x7f5020198ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:22.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:22.981+0000 7f50287a2640 1 --2- 172.21.15.97:0/3563700448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5020198600 0x7f5020198ab0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f501c009fd0 tx=0x7f501c00b8e0 comp rx=0 tx=0).stop 2024-04-23T10:32:22.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:22.981+0000 7f50287a2640 1 -- 172.21.15.97:0/3563700448 shutdown_connections 2024-04-23T10:32:22.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:22.981+0000 7f50287a2640 1 --2- 172.21.15.97:0/3563700448 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f500c0a3da0 0x7f500c0a6260 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:22.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:22.981+0000 7f50287a2640 1 --2- 172.21.15.97:0/3563700448 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5020198600 0x7f5020198ab0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:22.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:22.981+0000 7f50287a2640 1 --2- 172.21.15.97:0/3563700448 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5020106170 0x7f502019e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:22.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:22.981+0000 7f50287a2640 1 --2- 172.21.15.97:0/3563700448 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50201010a0 0x7f502019e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:22.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:22.981+0000 7f50287a2640 1 -- 172.21.15.97:0/3563700448 >> 172.21.15.97:0/3563700448 conn(0x7f5020077f50 msgr2=0x7f5020069480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:32:22.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:22.982+0000 7f50287a2640 1 -- 172.21.15.97:0/3563700448 shutdown_connections 2024-04-23T10:32:22.981 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:22.982+0000 7f50287a2640 1 -- 172.21.15.97:0/3563700448 wait complete. 2024-04-23T10:32:22.991 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --set-id 702905e7 --release --await' 2024-04-23T10:32:22.991 DEBUG:teuthology.orchestra.run.smithi097:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24488 quiesce db --set-id 702905e7 --release --await 2024-04-23T10:32:23.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.101+0000 7f9af6b8e640 1 -- 172.21.15.97:0/1471063400 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9af006aed0 msgr2=0x7f9af010edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:23.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.101+0000 7f9af6b8e640 1 --2- 172.21.15.97:0/1471063400 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9af006aed0 0x7f9af010edf0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f9ae8009cd0 tx=0x7f9ae802a8e0 comp rx=0 tx=0).stop 2024-04-23T10:32:23.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.101+0000 7f9af6b8e640 1 -- 172.21.15.97:0/1471063400 shutdown_connections 2024-04-23T10:32:23.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.101+0000 7f9af6b8e640 1 --2- 172.21.15.97:0/1471063400 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9af006aed0 0x7f9af010edf0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:23.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.101+0000 7f9af6b8e640 1 --2- 172.21.15.97:0/1471063400 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9af006a550 0x7f9af006a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:23.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.101+0000 7f9af6b8e640 1 --2- 172.21.15.97:0/1471063400 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9af0107fc0 0x7f9af01083a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:23.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.101+0000 7f9af6b8e640 1 -- 172.21.15.97:0/1471063400 >> 172.21.15.97:0/1471063400 conn(0x7f9af0076c50 msgr2=0x7f9af0077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:32:23.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.101+0000 7f9af6b8e640 1 -- 172.21.15.97:0/1471063400 shutdown_connections 2024-04-23T10:32:23.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.101+0000 7f9af6b8e640 1 -- 172.21.15.97:0/1471063400 wait complete. 2024-04-23T10:32:23.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.102+0000 7f9af6b8e640 1 Processor -- start 2024-04-23T10:32:23.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.102+0000 7f9af6b8e640 1 -- start start 2024-04-23T10:32:23.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.102+0000 7f9af6b8e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9af006a550 0x7f9af0101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:23.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.102+0000 7f9af6b8e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9af006aed0 0x7f9af01017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:23.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.102+0000 7f9af6b8e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9af0107fc0 0x7f9af0101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:23.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.102+0000 7f9af6b8e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9af0112190 con 0x7f9af006aed0 2024-04-23T10:32:23.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.102+0000 7f9af6b8e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9af0112010 con 0x7f9af006a550 2024-04-23T10:32:23.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.102+0000 7f9af6b8e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9af0112310 con 0x7f9af0107fc0 2024-04-23T10:32:23.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.102+0000 7f9aeffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9af006aed0 0x7f9af01017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:23.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.102+0000 7f9aeffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9af006aed0 0x7f9af01017b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60434/0 (socket says 172.21.15.97:60434) 2024-04-23T10:32:23.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.102+0000 7f9aeffff640 1 -- 172.21.15.97:0/1750436260 learned_addr learned my addr 172.21.15.97:0/1750436260 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:32:23.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.102+0000 7f9aeffff640 1 -- 172.21.15.97:0/1750436260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9af0107fc0 msgr2=0x7f9af0101cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:23.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.103+0000 7f9aeffff640 1 --2- 172.21.15.97:0/1750436260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9af0107fc0 0x7f9af0101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:23.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.103+0000 7f9aeffff640 1 -- 172.21.15.97:0/1750436260 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9af006a550 msgr2=0x7f9af0101270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:23.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.102+0000 7f9af492a640 1 --2- 172.21.15.97:0/1750436260 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9af006a550 0x7f9af0101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:23.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.103+0000 7f9aeffff640 1 --2- 172.21.15.97:0/1750436260 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9af006a550 0x7f9af0101270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:23.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.103+0000 7f9aeffff640 1 -- 172.21.15.97:0/1750436260 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9af01a6c50 con 0x7f9af006aed0 2024-04-23T10:32:23.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.103+0000 7f9af492a640 1 --2- 172.21.15.97:0/1750436260 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9af006a550 0x7f9af0101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:32:23.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.103+0000 7f9aeffff640 1 --2- 172.21.15.97:0/1750436260 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9af006aed0 0x7f9af01017b0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f9ae400d950 tx=0x7f9ae400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:32:23.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.103+0000 7f9aedffb640 1 -- 172.21.15.97:0/1750436260 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae4017030 con 0x7f9af006aed0 2024-04-23T10:32:23.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.103+0000 7f9aedffb640 1 -- 172.21.15.97:0/1750436260 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ae4004020 con 0x7f9af006aed0 2024-04-23T10:32:23.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.103+0000 7f9aedffb640 1 -- 172.21.15.97:0/1750436260 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae40047a0 con 0x7f9af006aed0 2024-04-23T10:32:23.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.103+0000 7f9af6b8e640 1 -- 172.21.15.97:0/1750436260 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9af01a6f40 con 0x7f9af006aed0 2024-04-23T10:32:23.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.103+0000 7f9af6b8e640 1 -- 172.21.15.97:0/1750436260 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9af01a7450 con 0x7f9af006aed0 2024-04-23T10:32:23.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.105+0000 7f9aedffb640 1 -- 172.21.15.97:0/1750436260 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9ae4009a30 con 0x7f9af006aed0 2024-04-23T10:32:23.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.106+0000 7f9af997d740 1 Processor -- start 2024-04-23T10:32:23.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.106+0000 7f9aedffb640 1 --2- 172.21.15.97:0/1750436260 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f9ad80a3da0 0x7f9ad80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:23.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.106+0000 7f9af997d740 1 -- start start 2024-04-23T10:32:23.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.106+0000 7f9af512b640 1 --2- 172.21.15.97:0/1750436260 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f9ad80a3da0 0x7f9ad80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:23.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.106+0000 7f9aedffb640 1 -- 172.21.15.97:0/1750436260 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f9ae40d0250 con 0x7f9af006aed0 2024-04-23T10:32:23.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.106+0000 7f9af997d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ca51d1490 0x557ca51d1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:23.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.106+0000 7f9af997d740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ca51d1db0 0x557ca51dbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:23.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.106+0000 7f9af997d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ca51dc480 0x557ca51de870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:23.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.106+0000 7f9af997d740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x557ca51366d0 con 0x557ca51d1db0 2024-04-23T10:32:23.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.106+0000 7f9af997d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557ca50d31a0 con 0x557ca51dc480 2024-04-23T10:32:23.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.106+0000 7f9af997d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557ca5132690 con 0x557ca51d1490 2024-04-23T10:32:23.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.106+0000 7f9aeffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ca51d1db0 0x557ca51dbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:23.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.106+0000 7f9aeffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ca51d1db0 0x557ca51dbf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60448/0 (socket says 172.21.15.97:60448) 2024-04-23T10:32:23.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.106+0000 7f9aeffff640 1 -- 172.21.15.97:0/3808849809 learned_addr learned my addr 172.21.15.97:0/3808849809 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:32:23.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.106+0000 7f9af492a640 1 --2- 172.21.15.97:0/3808849809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ca51d1490 0x557ca51d1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:23.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.106+0000 7f9aeffff640 1 -- 172.21.15.97:0/3808849809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ca51d1490 msgr2=0x557ca51d1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:32:23.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.106+0000 7f9aeffff640 1 --2- 172.21.15.97:0/3808849809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ca51d1490 0x557ca51d1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:23.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.106+0000 7f9af512b640 1 --2- 172.21.15.97:0/1750436260 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f9ad80a3da0 0x7f9ad80a6260 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9ae8004af0 tx=0x7f9ae8009360 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:32:23.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.106+0000 7f9aeffff640 1 -- 172.21.15.97:0/3808849809 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ca51dc480 msgr2=0x557ca51de870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:32:23.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.106+0000 7f9aeffff640 1 --2- 172.21.15.97:0/3808849809 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ca51dc480 0x557ca51de870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:23.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.106+0000 7f9aeffff640 1 -- 172.21.15.97:0/3808849809 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ca50dd5b0 con 0x557ca51d1db0 2024-04-23T10:32:23.113 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.106+0000 7f9af492a640 1 --2- 172.21.15.97:0/3808849809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ca51d1490 0x557ca51d1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:32:23.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.106+0000 7f9aeffff640 1 --2- 172.21.15.97:0/3808849809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ca51d1db0 0x557ca51dbf40 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f9ae40a4980 tx=0x7f9ae4005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1b9a1ecdd87d900 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:32:23.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.107+0000 7f9ad6ffd640 1 -- 172.21.15.97:0/3808849809 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae40b5070 con 0x557ca51d1db0 2024-04-23T10:32:23.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.107+0000 7f9ad6ffd640 1 -- 172.21.15.97:0/3808849809 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ae409c240 con 0x557ca51d1db0 2024-04-23T10:32:23.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.107+0000 7f9ad6ffd640 1 -- 172.21.15.97:0/3808849809 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae40042f0 con 0x557ca51d1db0 2024-04-23T10:32:23.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.107+0000 7f9af997d740 1 -- 172.21.15.97:0/3808849809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ca51d1db0 msgr2=0x557ca51dbf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:23.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.107+0000 7f9af997d740 1 --2- 172.21.15.97:0/3808849809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ca51d1db0 0x557ca51dbf40 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f9ae40a4980 tx=0x7f9ae4005760 comp rx=0 tx=0).stop 2024-04-23T10:32:23.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.107+0000 7f9af997d740 1 -- 172.21.15.97:0/3808849809 shutdown_connections 2024-04-23T10:32:23.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.107+0000 7f9af997d740 1 --2- 172.21.15.97:0/3808849809 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ca51dc480 0x557ca51de870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:23.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.107+0000 7f9af997d740 1 --2- 172.21.15.97:0/3808849809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ca51d1db0 0x557ca51dbf40 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:23.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.107+0000 7f9af997d740 1 --2- 172.21.15.97:0/3808849809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ca51d1490 0x557ca51d1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:23.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.107+0000 7f9af997d740 1 -- 172.21.15.97:0/3808849809 >> 172.21.15.97:0/3808849809 conn(0x557ca51cd7b0 msgr2=0x557ca51cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:32:23.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.107+0000 7f9af997d740 1 -- 172.21.15.97:0/3808849809 shutdown_connections 2024-04-23T10:32:23.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.107+0000 7f9af997d740 1 -- 172.21.15.97:0/3808849809 wait complete. 2024-04-23T10:32:23.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.108+0000 7f9af997d740 1 -- start start 2024-04-23T10:32:23.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.108+0000 7f9af997d740 1 Processor -- start 2024-04-23T10:32:23.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.108+0000 7f9af997d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ca51d1490 0x557ca51d92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:23.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.108+0000 7f9af997d740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ca51d1db0 0x557ca51d97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:23.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.108+0000 7f9af997d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ca51dc480 0x557ca51d9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:23.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.108+0000 7f9af997d740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x557ca51df310 con 0x557ca51d1db0 2024-04-23T10:32:23.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.108+0000 7f9af997d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557ca50d0b20 con 0x557ca51dc480 2024-04-23T10:32:23.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.108+0000 7f9af997d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557ca51df490 con 0x557ca51d1490 2024-04-23T10:32:23.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.108+0000 7f9af997d740 10 client.0 _ll_register_callbacks cb 0x557ca50d0530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:32:23.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.108+0000 7f9aeffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ca51d1db0 0x557ca51d97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:23.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.108+0000 7f9aeffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ca51d1db0 0x557ca51d97f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60458/0 (socket says 172.21.15.97:60458) 2024-04-23T10:32:23.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.108+0000 7f9aeffff640 1 -- 172.21.15.97:0/2216741313 learned_addr learned my addr 172.21.15.97:0/2216741313 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:32:23.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.108+0000 7f9af492a640 1 --2- 172.21.15.97:0/2216741313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ca51d1490 0x557ca51d92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:23.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.108+0000 7f9aeffff640 1 -- 172.21.15.97:0/2216741313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ca51d1490 msgr2=0x557ca51d92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:32:23.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.108+0000 7f9aeffff640 1 --2- 172.21.15.97:0/2216741313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ca51d1490 0x557ca51d92b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:23.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.108+0000 7f9af512b640 1 --2- 172.21.15.97:0/2216741313 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ca51dc480 0x557ca51d9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:23.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.108+0000 7f9aeffff640 1 -- 172.21.15.97:0/2216741313 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ca51dc480 msgr2=0x557ca51d9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:32:23.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.109+0000 7f9aeffff640 1 --2- 172.21.15.97:0/2216741313 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ca51dc480 0x557ca51d9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:23.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.109+0000 7f9aeffff640 1 -- 172.21.15.97:0/2216741313 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ca4e48920 con 0x557ca51d1db0 2024-04-23T10:32:23.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.109+0000 7f9aeffff640 1 --2- 172.21.15.97:0/2216741313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ca51d1db0 0x557ca51d97f0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f9ae40cdff0 tx=0x7f9ae4009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b7dde2a9b140059 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:32:23.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.109+0000 7f9acffff640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:32:23.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.109+0000 7f9acffff640 1 -- 172.21.15.97:0/2216741313 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae40b9070 con 0x557ca51d1db0 2024-04-23T10:32:23.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.109+0000 7f9acffff640 1 -- 172.21.15.97:0/2216741313 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f9ae40988b0 con 0x557ca51d1db0 2024-04-23T10:32:23.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.109+0000 7f9af997d740 1 -- 172.21.15.97:0/2216741313 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ae409c240 con 0x557ca51d1db0 2024-04-23T10:32:23.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.109+0000 7f9acffff640 1 -- 172.21.15.97:0/2216741313 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9ae40b5050 con 0x557ca51d1db0 2024-04-23T10:32:23.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.109+0000 7f9af997d740 10 client.16473 fetch_fsmap learned FSMap version 26 2024-04-23T10:32:23.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.109+0000 7f9af997d740 1 -- 172.21.15.97:0/2216741313 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557ca4f14d30 con 0x557ca51d1db0 2024-04-23T10:32:23.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.109+0000 7f9acffff640 1 -- 172.21.15.97:0/2216741313 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ae409c240 con 0x557ca51d1db0 2024-04-23T10:32:23.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.109+0000 7f9acffff640 1 -- 172.21.15.97:0/2216741313 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f9ae40b0ce0 con 0x557ca51d1db0 2024-04-23T10:32:23.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.109+0000 7f9af997d740 10 client.16473 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:32:23.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.109+0000 7f9af997d740 10 client.16473 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:32:23.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.109+0000 7f9af997d740 20 client.16473 populate_metadata read hostname 'smithi097' 2024-04-23T10:32:23.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.109+0000 7f9af997d740 1 --2- 172.21.15.97:0/2216741313 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x557ca51ec530 0x557ca51ee920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:23.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.109+0000 7f9af997d740 4 client.16473 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:32:23.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.109+0000 7f9af997d740 1 -- 172.21.15.97:0/2216741313 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ca50f4500 con 0x557ca51ec530 2024-04-23T10:32:23.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.110+0000 7f9af492a640 1 --2- 172.21.15.97:0/2216741313 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x557ca51ec530 0x557ca51ee920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:23.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.110+0000 7f9acffff640 10 client.16473 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:32:23.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.110+0000 7f9af492a640 1 --2- 172.21.15.97:0/2216741313 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x557ca51ec530 0x557ca51ee920 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30a9a3c25fd692f0 server_cookie=6f7fc7139dd040c0 in_seq=0 out_seq=0 2024-04-23T10:32:23.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.111+0000 7f9acffff640 1 -- 172.21.15.97:0/2216741313 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557ca50f4500 con 0x557ca51ec530 2024-04-23T10:32:23.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.111+0000 7f9acffff640 10 client.16473 handle_command_reply: tid=0 2024-04-23T10:32:23.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.111+0000 7f9af997d740 1 client.16473 shutdown 2024-04-23T10:32:23.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.111+0000 7f9af997d740 1 -- 172.21.15.97:0/2216741313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ca51d1db0 msgr2=0x557ca51d97f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:23.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.111+0000 7f9af997d740 1 --2- 172.21.15.97:0/2216741313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ca51d1db0 0x557ca51d97f0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f9ae40cdff0 tx=0x7f9ae4009f50 comp rx=0 tx=0).stop 2024-04-23T10:32:23.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.111+0000 7f9af997d740 1 -- 172.21.15.97:0/2216741313 shutdown_connections 2024-04-23T10:32:23.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.111+0000 7f9af997d740 1 --2- 172.21.15.97:0/2216741313 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x557ca51ec530 0x557ca51ee920 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:23.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.111+0000 7f9af997d740 1 --2- 172.21.15.97:0/2216741313 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ca51dc480 0x557ca51d9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:23.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.111+0000 7f9acffff640 0 client.16473 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:32:23.117 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.111+0000 7f9af997d740 1 --2- 172.21.15.97:0/2216741313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ca51d1db0 0x557ca51d97f0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:23.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.111+0000 7f9af997d740 1 --2- 172.21.15.97:0/2216741313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ca51d1490 0x557ca51d92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:23.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.111+0000 7f9af997d740 1 -- 172.21.15.97:0/2216741313 >> 172.21.15.97:0/2216741313 conn(0x557ca51cd7b0 msgr2=0x557ca51d35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:32:23.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.111+0000 7f9af997d740 1 -- 172.21.15.97:0/2216741313 shutdown_connections 2024-04-23T10:32:23.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.111+0000 7f9af997d740 1 -- 172.21.15.97:0/2216741313 wait complete. 2024-04-23T10:32:23.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.111+0000 7f9af997d740 20 client.16473 trim_cache size 0 max 16384 2024-04-23T10:32:23.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.128+0000 7f9af997d740 1 Processor -- start 2024-04-23T10:32:23.131 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.128+0000 7f9af997d740 1 -- start start 2024-04-23T10:32:23.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.128+0000 7f9af997d740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ca51d1490 0x557ca51cd150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:23.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.128+0000 7f9af997d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ca51d1db0 0x557ca51d9980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:23.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.128+0000 7f9af997d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ca51dc480 0x557ca51d6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:23.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.128+0000 7f9af997d740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9af01afdf0 con 0x557ca51d1490 2024-04-23T10:32:23.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.128+0000 7f9af997d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9af01b1930 con 0x557ca51dc480 2024-04-23T10:32:23.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.128+0000 7f9af997d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9af01b04c0 con 0x557ca51d1db0 2024-04-23T10:32:23.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.129+0000 7f9af492a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ca51d1490 0x557ca51cd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:23.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.129+0000 7f9af492a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ca51d1490 0x557ca51cd150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60470/0 (socket says 172.21.15.97:60470) 2024-04-23T10:32:23.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.129+0000 7f9af492a640 1 -- 172.21.15.97:0/3266742161 learned_addr learned my addr 172.21.15.97:0/3266742161 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:32:23.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.129+0000 7f9af492a640 1 -- 172.21.15.97:0/3266742161 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ca51d1db0 msgr2=0x557ca51d9980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:32:23.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.129+0000 7f9af492a640 1 --2- 172.21.15.97:0/3266742161 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ca51d1db0 0x557ca51d9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:23.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.129+0000 7f9af492a640 1 -- 172.21.15.97:0/3266742161 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ca51dc480 msgr2=0x557ca51d6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:32:23.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.129+0000 7f9af492a640 1 --2- 172.21.15.97:0/3266742161 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ca51dc480 0x557ca51d6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:23.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.129+0000 7f9af492a640 1 -- 172.21.15.97:0/3266742161 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ca51c9170 con 0x557ca51d1490 2024-04-23T10:32:23.132 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.129+0000 7f9af492a640 1 --2- 172.21.15.97:0/3266742161 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ca51d1490 0x557ca51cd150 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x557ca51df580 tx=0x7f9ae000deb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd3922a9ef43d26d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:32:23.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.129+0000 7f9ad67fc640 1 -- 172.21.15.97:0/3266742161 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae001b070 con 0x557ca51d1490 2024-04-23T10:32:23.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.129+0000 7f9ad67fc640 1 -- 172.21.15.97:0/3266742161 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ae000ee30 con 0x557ca51d1490 2024-04-23T10:32:23.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.129+0000 7f9ad67fc640 1 -- 172.21.15.97:0/3266742161 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae000c660 con 0x557ca51d1490 2024-04-23T10:32:23.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.130+0000 7f9af997d740 1 -- 172.21.15.97:0/3266742161 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ca51d1490 msgr2=0x557ca51cd150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:23.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.130+0000 7f9af997d740 1 --2- 172.21.15.97:0/3266742161 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ca51d1490 0x557ca51cd150 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x557ca51df580 tx=0x7f9ae000deb0 comp rx=0 tx=0).stop 2024-04-23T10:32:23.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.130+0000 7f9af997d740 1 -- 172.21.15.97:0/3266742161 shutdown_connections 2024-04-23T10:32:23.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.130+0000 7f9af997d740 1 --2- 172.21.15.97:0/3266742161 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ca51dc480 0x557ca51d6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:23.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.130+0000 7f9af997d740 1 --2- 172.21.15.97:0/3266742161 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ca51d1db0 0x557ca51d9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:23.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.130+0000 7f9af997d740 1 --2- 172.21.15.97:0/3266742161 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ca51d1490 0x557ca51cd150 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:23.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.130+0000 7f9af997d740 1 -- 172.21.15.97:0/3266742161 >> 172.21.15.97:0/3266742161 conn(0x557ca51cd7b0 msgr2=0x557ca51d9270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:32:23.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.130+0000 7f9af997d740 1 -- 172.21.15.97:0/3266742161 shutdown_connections 2024-04-23T10:32:23.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.130+0000 7f9af997d740 1 -- 172.21.15.97:0/3266742161 wait complete. 2024-04-23T10:32:23.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.130+0000 7f9af997d740 1 -- start start 2024-04-23T10:32:23.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.130+0000 7f9af997d740 1 Processor -- start 2024-04-23T10:32:23.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.130+0000 7f9af997d740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ca51d1490 0x557ca51ef6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:23.133 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.130+0000 7f9af997d740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ca51d1db0 0x557ca51efc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:23.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.130+0000 7f9af997d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ca51dc480 0x557ca51f0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:23.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.130+0000 7f9af997d740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9af01b1260 con 0x557ca51d1db0 2024-04-23T10:32:23.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.130+0000 7f9af997d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9af01b0b90 con 0x557ca51dc480 2024-04-23T10:32:23.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.130+0000 7f9af997d740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9af01addd0 con 0x557ca51d1490 2024-04-23T10:32:23.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.130+0000 7f9af997d740 10 client.0 _ll_register_callbacks cb 0x7f9ae409bba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:32:23.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.130+0000 7f9aeffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ca51d1db0 0x557ca51efc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:23.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.131+0000 7f9aeffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ca51d1db0 0x557ca51efc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60472/0 (socket says 172.21.15.97:60472) 2024-04-23T10:32:23.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.131+0000 7f9aeffff640 1 -- 172.21.15.97:0/3140310799 learned_addr learned my addr 172.21.15.97:0/3140310799 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:32:23.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.131+0000 7f9af492a640 1 --2- 172.21.15.97:0/3140310799 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ca51d1490 0x557ca51ef6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:23.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.131+0000 7f9af512b640 1 --2- 172.21.15.97:0/3140310799 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ca51dc480 0x557ca51f0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:23.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.131+0000 7f9aeffff640 1 -- 172.21.15.97:0/3140310799 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ca51d1490 msgr2=0x557ca51ef6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:32:23.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.131+0000 7f9aeffff640 1 --2- 172.21.15.97:0/3140310799 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ca51d1490 0x557ca51ef6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:23.134 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.131+0000 7f9aeffff640 1 -- 172.21.15.97:0/3140310799 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ca51dc480 msgr2=0x557ca51f0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:32:23.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.131+0000 7f9aeffff640 1 --2- 172.21.15.97:0/3140310799 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ca51dc480 0x557ca51f0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:23.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.131+0000 7f9aeffff640 1 -- 172.21.15.97:0/3140310799 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ca5251a00 con 0x557ca51d1db0 2024-04-23T10:32:23.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.131+0000 7f9af492a640 1 --2- 172.21.15.97:0/3140310799 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ca51d1490 0x557ca51ef6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:32:23.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.131+0000 7f9aeffff640 1 --2- 172.21.15.97:0/3140310799 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ca51d1db0 0x557ca51efc10 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f9ae409c180 tx=0x7f9ae4005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73a3781c93b90c99 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:32:23.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.131+0000 7f9acffff640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:32:23.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.131+0000 7f9acffff640 1 -- 172.21.15.97:0/3140310799 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae40b9040 con 0x557ca51d1db0 2024-04-23T10:32:23.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.131+0000 7f9acffff640 1 -- 172.21.15.97:0/3140310799 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f9ae40a4a20 con 0x557ca51d1db0 2024-04-23T10:32:23.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.131+0000 7f9af997d740 1 -- 172.21.15.97:0/3140310799 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ae000ee30 con 0x557ca51d1db0 2024-04-23T10:32:23.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.131+0000 7f9acffff640 1 -- 172.21.15.97:0/3140310799 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9ae40984e0 con 0x557ca51d1db0 2024-04-23T10:32:23.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.131+0000 7f9af997d740 10 client.16479 fetch_fsmap learned FSMap version 26 2024-04-23T10:32:23.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.131+0000 7f9af997d740 1 -- 172.21.15.97:0/3140310799 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557ca5252d40 con 0x557ca51d1db0 2024-04-23T10:32:23.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.131+0000 7f9acffff640 1 -- 172.21.15.97:0/3140310799 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ae40a7080 con 0x557ca51d1db0 2024-04-23T10:32:23.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.132+0000 7f9acffff640 1 -- 172.21.15.97:0/3140310799 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f9ae40b0ce0 con 0x557ca51d1db0 2024-04-23T10:32:23.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.132+0000 7f9af997d740 10 client.16479 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:32:23.135 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.132+0000 7f9af997d740 10 client.16479 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:32:23.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.132+0000 7f9af997d740 20 client.16479 populate_metadata read hostname 'smithi097' 2024-04-23T10:32:23.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.132+0000 7f9af997d740 1 --2- 172.21.15.97:0/3140310799 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x557ca5253300 0x557ca5255720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:23.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.132+0000 7f9af997d740 4 client.16479 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "702905e7", "release": true, "await": true}] 2024-04-23T10:32:23.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.132+0000 7f9af997d740 1 -- 172.21.15.97:0/3140310799 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "quiesce db", "set_id": "702905e7", "release": true, "await": true}) -- 0x557ca51440f0 con 0x557ca5253300 2024-04-23T10:32:23.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.132+0000 7f9af492a640 1 --2- 172.21.15.97:0/3140310799 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x557ca5253300 0x557ca5255720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:23.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.132+0000 7f9acffff640 10 client.16479 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:32:23.136 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.132+0000 7f9af492a640 1 --2- 172.21.15.97:0/3140310799 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x557ca5253300 0x557ca5255720 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2217aeb57940073c server_cookie=20c5e366bc9215c in_seq=0 out_seq=0 2024-04-23T10:32:23.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.135+0000 7f9acffff640 1 -- 172.21.15.97:0/3140310799 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x557ca51440f0 con 0x557ca5253300 2024-04-23T10:32:23.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.135+0000 7f9acffff640 10 client.16479 handle_command_reply: tid=0 2024-04-23T10:32:23.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.135+0000 7f9af997d740 1 client.16479 shutdown 2024-04-23T10:32:23.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.136+0000 7f9af997d740 1 -- 172.21.15.97:0/3140310799 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ca51d1db0 msgr2=0x557ca51efc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:23.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.136+0000 7f9af997d740 1 --2- 172.21.15.97:0/3140310799 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ca51d1db0 0x557ca51efc10 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f9ae409c180 tx=0x7f9ae4005d30 comp rx=0 tx=0).stop 2024-04-23T10:32:23.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.136+0000 7f9af997d740 1 -- 172.21.15.97:0/3140310799 shutdown_connections 2024-04-23T10:32:23.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.136+0000 7f9af997d740 1 --2- 172.21.15.97:0/3140310799 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x557ca5253300 0x557ca5255720 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:23.137 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.136+0000 7f9af997d740 1 --2- 172.21.15.97:0/3140310799 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557ca51dc480 0x557ca51f0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:23.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.136+0000 7f9acffff640 0 client.16479 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:32:23.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.136+0000 7f9af997d740 1 --2- 172.21.15.97:0/3140310799 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557ca51d1db0 0x557ca51efc10 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:23.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.136+0000 7f9af997d740 1 --2- 172.21.15.97:0/3140310799 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557ca51d1490 0x557ca51ef6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:23.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.136+0000 7f9af997d740 1 -- 172.21.15.97:0/3140310799 >> 172.21.15.97:0/3140310799 conn(0x557ca51cd7b0 msgr2=0x557ca51d9ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:32:23.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.136+0000 7f9af997d740 1 -- 172.21.15.97:0/3140310799 shutdown_connections 2024-04-23T10:32:23.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.136+0000 7f9af997d740 1 -- 172.21.15.97:0/3140310799 wait complete. 2024-04-23T10:32:23.138 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.136+0000 7f9af997d740 20 client.16479 trim_cache size 0 max 16384 2024-04-23T10:32:23.138 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-23T10:32:23.138 INFO:teuthology.orchestra.run.smithi097.stdout: "epoch": 26, 2024-04-23T10:32:23.138 INFO:teuthology.orchestra.run.smithi097.stdout: "leader": 24488, 2024-04-23T10:32:23.138 INFO:teuthology.orchestra.run.smithi097.stdout: "set_version": 24, 2024-04-23T10:32:23.138 INFO:teuthology.orchestra.run.smithi097.stdout: "sets": { 2024-04-23T10:32:23.139 INFO:teuthology.orchestra.run.smithi097.stdout: "702905e7": { 2024-04-23T10:32:23.139 INFO:teuthology.orchestra.run.smithi097.stdout: "version": 24, 2024-04-23T10:32:23.139 INFO:teuthology.orchestra.run.smithi097.stdout: "age_ref": 0.0, 2024-04-23T10:32:23.139 INFO:teuthology.orchestra.run.smithi097.stdout: "state": { 2024-04-23T10:32:23.139 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RELEASED", 2024-04-23T10:32:23.139 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0 2024-04-23T10:32:23.139 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:32:23.139 INFO:teuthology.orchestra.run.smithi097.stdout: "timeout": 60.0, 2024-04-23T10:32:23.139 INFO:teuthology.orchestra.run.smithi097.stdout: "expiration": 79.5, 2024-04-23T10:32:23.139 INFO:teuthology.orchestra.run.smithi097.stdout: "members": { 2024-04-23T10:32:23.139 INFO:teuthology.orchestra.run.smithi097.stdout: "file:/": { 2024-04-23T10:32:23.139 INFO:teuthology.orchestra.run.smithi097.stdout: "excluded": false, 2024-04-23T10:32:23.139 INFO:teuthology.orchestra.run.smithi097.stdout: "state": { 2024-04-23T10:32:23.139 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RELEASED", 2024-04-23T10:32:23.139 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0 2024-04-23T10:32:23.140 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:32:23.140 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:32:23.140 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:32:23.140 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:32:23.140 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:32:23.140 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-23T10:32:23.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.136+0000 7f9af6b8e640 1 -- 172.21.15.97:0/1750436260 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f9ad80a3da0 msgr2=0x7f9ad80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:23.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.136+0000 7f9af6b8e640 1 --2- 172.21.15.97:0/1750436260 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f9ad80a3da0 0x7f9ad80a6260 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9ae8004af0 tx=0x7f9ae8009360 comp rx=0 tx=0).stop 2024-04-23T10:32:23.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.136+0000 7f9af6b8e640 1 -- 172.21.15.97:0/1750436260 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9af006aed0 msgr2=0x7f9af01017b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:23.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.136+0000 7f9af6b8e640 1 --2- 172.21.15.97:0/1750436260 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9af006aed0 0x7f9af01017b0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f9ae400d950 tx=0x7f9ae400de10 comp rx=0 tx=0).stop 2024-04-23T10:32:23.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.136+0000 7f9af6b8e640 1 -- 172.21.15.97:0/1750436260 shutdown_connections 2024-04-23T10:32:23.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.136+0000 7f9af6b8e640 1 --2- 172.21.15.97:0/1750436260 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f9ad80a3da0 0x7f9ad80a6260 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:23.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.136+0000 7f9af6b8e640 1 --2- 172.21.15.97:0/1750436260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9af0107fc0 0x7f9af0101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:23.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.136+0000 7f9af6b8e640 1 --2- 172.21.15.97:0/1750436260 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9af006aed0 0x7f9af01017b0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:23.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.136+0000 7f9af6b8e640 1 --2- 172.21.15.97:0/1750436260 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9af006a550 0x7f9af0101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:23.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.136+0000 7f9af6b8e640 1 -- 172.21.15.97:0/1750436260 >> 172.21.15.97:0/1750436260 conn(0x7f9af0076c50 msgr2=0x7f9af00fff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:32:23.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.136+0000 7f9af6b8e640 1 -- 172.21.15.97:0/1750436260 shutdown_connections 2024-04-23T10:32:23.141 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:32:23.137+0000 7f9af6b8e640 1 -- 172.21.15.97:0/1750436260 wait complete. 2024-04-23T10:32:23.146 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '702905e7', seconds elapsed: 20.5 2024-04-23T10:32:23.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:32:23 smithi196 ceph-mon[32919]: pgmap v188: 97 pgs: 97 active+clean; 2.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.3 KiB/s wr, 11 op/s 2024-04-23T10:32:23.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:32:23 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/3563700448' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:32:23.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:32:23 smithi097 ceph-mon[29197]: pgmap v188: 97 pgs: 97 active+clean; 2.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.3 KiB/s wr, 11 op/s 2024-04-23T10:32:23.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:32:23 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/3563700448' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:32:23.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:32:23 smithi183 ceph-mon[32317]: pgmap v188: 97 pgs: 97 active+clean; 2.7 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.3 KiB/s wr, 11 op/s 2024-04-23T10:32:23.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:32:23 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/3563700448' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:32:25.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:32:25 smithi196 ceph-mon[32919]: pgmap v189: 97 pgs: 97 active+clean; 2.8 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.9 KiB/s wr, 17 op/s 2024-04-23T10:32:25.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:32:25 smithi097 ceph-mon[29197]: pgmap v189: 97 pgs: 97 active+clean; 2.8 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.9 KiB/s wr, 17 op/s 2024-04-23T10:32:25.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:32:25 smithi183 ceph-mon[32317]: pgmap v189: 97 pgs: 97 active+clean; 2.8 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.9 KiB/s wr, 17 op/s 2024-04-23T10:32:27.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:32:27 smithi196 ceph-mon[32919]: pgmap v190: 97 pgs: 97 active+clean; 2.8 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.6 KiB/s wr, 10 op/s 2024-04-23T10:32:27.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:32:27 smithi097 ceph-mon[29197]: pgmap v190: 97 pgs: 97 active+clean; 2.8 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.6 KiB/s wr, 10 op/s 2024-04-23T10:32:27.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:32:27 smithi183 ceph-mon[32317]: pgmap v190: 97 pgs: 97 active+clean; 2.8 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.6 KiB/s wr, 10 op/s 2024-04-23T10:32:29.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:32:29 smithi196 ceph-mon[32919]: pgmap v191: 97 pgs: 97 active+clean; 2.9 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-04-23T10:32:29.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:32:29 smithi097 ceph-mon[29197]: pgmap v191: 97 pgs: 97 active+clean; 2.9 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-04-23T10:32:29.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:32:29 smithi183 ceph-mon[32317]: pgmap v191: 97 pgs: 97 active+clean; 2.9 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-04-23T10:32:31.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:32:31 smithi196 ceph-mon[32919]: pgmap v192: 97 pgs: 97 active+clean; 2.9 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-04-23T10:32:31.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:32:31 smithi097 ceph-mon[29197]: pgmap v192: 97 pgs: 97 active+clean; 2.9 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-04-23T10:32:31.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:32:31 smithi183 ceph-mon[32317]: pgmap v192: 97 pgs: 97 active+clean; 2.9 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-04-23T10:32:33.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:32:33 smithi196 ceph-mon[32919]: pgmap v193: 97 pgs: 97 active+clean; 2.9 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-04-23T10:32:33.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:32:33 smithi097 ceph-mon[29197]: pgmap v193: 97 pgs: 97 active+clean; 2.9 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-04-23T10:32:33.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:32:33 smithi183 ceph-mon[32317]: pgmap v193: 97 pgs: 97 active+clean; 2.9 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-04-23T10:32:34.860 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.859+0000 7f036678a640 1 -- 172.21.15.97:0/75631271 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03601006c0 msgr2=0x7f0360100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:34.860 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.859+0000 7f036678a640 1 --2- 172.21.15.97:0/75631271 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03601006c0 0x7f0360100aa0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f0350009900 tx=0x7f035002a410 comp rx=0 tx=0).stop 2024-04-23T10:32:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.859+0000 7f036678a640 1 -- 172.21.15.97:0/75631271 shutdown_connections 2024-04-23T10:32:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.859+0000 7f036678a640 1 --2- 172.21.15.97:0/75631271 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0360107280 0x7f036010ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.859+0000 7f036678a640 1 --2- 172.21.15.97:0/75631271 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0360101070 0x7f0360106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.859+0000 7f036678a640 1 --2- 172.21.15.97:0/75631271 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03601006c0 0x7f0360100aa0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.859+0000 7f036678a640 1 -- 172.21.15.97:0/75631271 >> 172.21.15.97:0/75631271 conn(0x7f0360077370 msgr2=0x7f03600fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:32:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.859+0000 7f036678a640 1 -- 172.21.15.97:0/75631271 shutdown_connections 2024-04-23T10:32:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.859+0000 7f036678a640 1 -- 172.21.15.97:0/75631271 wait complete. 2024-04-23T10:32:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.859+0000 7f036678a640 1 Processor -- start 2024-04-23T10:32:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.860+0000 7f036678a640 1 -- start start 2024-04-23T10:32:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.860+0000 7f036678a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03601006c0 0x7f0360198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.860+0000 7f036678a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0360107280 0x7f0360198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.860+0000 7f036678a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f036019d000 0x7f036019d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.860+0000 7f036678a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f03601121c0 con 0x7f0360107280 2024-04-23T10:32:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.860+0000 7f036678a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0360112040 con 0x7f03601006c0 2024-04-23T10:32:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.860+0000 7f036678a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0360112340 con 0x7f036019d000 2024-04-23T10:32:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.860+0000 7f035f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0360107280 0x7f0360198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.860+0000 7f035f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0360107280 0x7f0360198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57196/0 (socket says 172.21.15.97:57196) 2024-04-23T10:32:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.860+0000 7f035f7fe640 1 -- 172.21.15.97:0/244702043 learned_addr learned my addr 172.21.15.97:0/244702043 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:32:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.860+0000 7f0364d27640 1 --2- 172.21.15.97:0/244702043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f036019d000 0x7f036019d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.860+0000 7f035ffff640 1 --2- 172.21.15.97:0/244702043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03601006c0 0x7f0360198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.860+0000 7f035f7fe640 1 -- 172.21.15.97:0/244702043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f036019d000 msgr2=0x7f036019d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.860+0000 7f035f7fe640 1 --2- 172.21.15.97:0/244702043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f036019d000 0x7f036019d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.860+0000 7f035f7fe640 1 -- 172.21.15.97:0/244702043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03601006c0 msgr2=0x7f0360198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.860+0000 7f035f7fe640 1 --2- 172.21.15.97:0/244702043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03601006c0 0x7f0360198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.860+0000 7f035f7fe640 1 -- 172.21.15.97:0/244702043 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f036019daf0 con 0x7f0360107280 2024-04-23T10:32:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.860+0000 7f0364d27640 1 --2- 172.21.15.97:0/244702043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f036019d000 0x7f036019d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:32:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.861+0000 7f035f7fe640 1 --2- 172.21.15.97:0/244702043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0360107280 0x7f0360198c80 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f035400c970 tx=0x7f035400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:32:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.861+0000 7f035d7fa640 1 -- 172.21.15.97:0/244702043 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0354007bb0 con 0x7f0360107280 2024-04-23T10:32:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.861+0000 7f035d7fa640 1 -- 172.21.15.97:0/244702043 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0354007d50 con 0x7f0360107280 2024-04-23T10:32:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.861+0000 7f036678a640 1 -- 172.21.15.97:0/244702043 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f036019dde0 con 0x7f0360107280 2024-04-23T10:32:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.861+0000 7f035d7fa640 1 -- 172.21.15.97:0/244702043 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f035400f450 con 0x7f0360107280 2024-04-23T10:32:34.863 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.861+0000 7f036678a640 1 -- 172.21.15.97:0/244702043 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03601a55d0 con 0x7f0360107280 2024-04-23T10:32:34.870 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.863+0000 7f035d7fa640 1 -- 172.21.15.97:0/244702043 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0354004f20 con 0x7f0360107280 2024-04-23T10:32:34.870 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.864+0000 7f0369577740 1 Processor -- start 2024-04-23T10:32:34.870 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.864+0000 7f0369577740 1 -- start start 2024-04-23T10:32:34.870 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.864+0000 7f035d7fa640 1 --2- 172.21.15.97:0/244702043 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f03480a3dc0 0x7f03480a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:34.870 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.864+0000 7f0369577740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559668e8a620 0x559668e8aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:34.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.864+0000 7f0369577740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559668e8af40 0x559668e952c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:34.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.864+0000 7f0369577740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559668e95800 0x559668e97bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:34.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.864+0000 7f0369577740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559668def970 con 0x559668e95800 2024-04-23T10:32:34.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.864+0000 7f0369577740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559668d8c4a0 con 0x559668e8af40 2024-04-23T10:32:34.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.864+0000 7f0369577740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559668deb930 con 0x559668e8a620 2024-04-23T10:32:34.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.864+0000 7f0364d27640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559668e95800 0x559668e97bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:34.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.864+0000 7f0364d27640 1 --2- 172.21.15.97:0/244702043 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f03480a3dc0 0x7f03480a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:34.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.864+0000 7f035d7fa640 1 -- 172.21.15.97:0/244702043 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f03540d0890 con 0x7f0360107280 2024-04-23T10:32:34.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.864+0000 7f0364d27640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559668e95800 0x559668e97bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57206/0 (socket says 172.21.15.97:57206) 2024-04-23T10:32:34.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.864+0000 7f0364d27640 1 -- 172.21.15.97:0/1401201655 learned_addr learned my addr 172.21.15.97:0/1401201655 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:32:34.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.864+0000 7f035ffff640 1 --2- 172.21.15.97:0/1401201655 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559668e8a620 0x559668e8aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:34.872 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.864+0000 7f035f7fe640 1 --2- 172.21.15.97:0/1401201655 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559668e8af40 0x559668e952c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:34.872 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.864+0000 7f0364d27640 1 -- 172.21.15.97:0/1401201655 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559668e8a620 msgr2=0x559668e8aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:32:34.872 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.864+0000 7f0364d27640 1 --2- 172.21.15.97:0/1401201655 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559668e8a620 0x559668e8aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:34.872 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.864+0000 7f0364d27640 1 -- 172.21.15.97:0/1401201655 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559668e8af40 msgr2=0x559668e952c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:32:34.872 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.864+0000 7f0364d27640 1 --2- 172.21.15.97:0/1401201655 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559668e8af40 0x559668e952c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:34.872 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.865+0000 7f0364d27640 1 -- 172.21.15.97:0/1401201655 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559668b02210 con 0x559668e95800 2024-04-23T10:32:34.872 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.865+0000 7f0364d27640 1 --2- 172.21.15.97:0/244702043 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f03480a3dc0 0x7f03480a6280 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f035800afd0 tx=0x7f035800a560 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:32:34.872 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.865+0000 7f0364d27640 1 --2- 172.21.15.97:0/1401201655 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559668e95800 0x559668e97bf0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f0358010d90 tx=0x7f035801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcd07a8d02fd06bf server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:32:34.872 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.865+0000 7f03467fc640 1 -- 172.21.15.97:0/1401201655 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0358013270 con 0x559668e95800 2024-04-23T10:32:34.872 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.865+0000 7f03467fc640 1 -- 172.21.15.97:0/1401201655 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f035800de50 con 0x559668e95800 2024-04-23T10:32:34.872 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.865+0000 7f03467fc640 1 -- 172.21.15.97:0/1401201655 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0358005710 con 0x559668e95800 2024-04-23T10:32:34.872 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.865+0000 7f0369577740 1 -- 172.21.15.97:0/1401201655 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559668e95800 msgr2=0x559668e97bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:34.873 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.865+0000 7f0369577740 1 --2- 172.21.15.97:0/1401201655 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559668e95800 0x559668e97bf0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f0358010d90 tx=0x7f035801a000 comp rx=0 tx=0).stop 2024-04-23T10:32:34.873 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.866+0000 7f0369577740 1 -- 172.21.15.97:0/1401201655 shutdown_connections 2024-04-23T10:32:34.873 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.866+0000 7f0369577740 1 --2- 172.21.15.97:0/1401201655 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559668e95800 0x559668e97bf0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:34.873 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.866+0000 7f0369577740 1 --2- 172.21.15.97:0/1401201655 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559668e8af40 0x559668e952c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:34.873 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.866+0000 7f0369577740 1 --2- 172.21.15.97:0/1401201655 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559668e8a620 0x559668e8aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:34.873 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.866+0000 7f0369577740 1 -- 172.21.15.97:0/1401201655 >> 172.21.15.97:0/1401201655 conn(0x559668e86b30 msgr2=0x559668e86f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:32:34.873 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.866+0000 7f0369577740 1 -- 172.21.15.97:0/1401201655 shutdown_connections 2024-04-23T10:32:34.873 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.866+0000 7f0369577740 1 -- 172.21.15.97:0/1401201655 wait complete. 2024-04-23T10:32:34.873 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.866+0000 7f0369577740 1 -- start start 2024-04-23T10:32:34.873 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.866+0000 7f0369577740 1 Processor -- start 2024-04-23T10:32:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.866+0000 7f0369577740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559668e8a620 0x559668e95170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.866+0000 7f0369577740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559668e8af40 0x559668e92460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.866+0000 7f0369577740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559668e95800 0x559668e929c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.866+0000 7f0369577740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559668e983d0 con 0x559668e8a620 2024-04-23T10:32:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.866+0000 7f0369577740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559668d8a0b0 con 0x559668e95800 2024-04-23T10:32:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.866+0000 7f0369577740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559668e98550 con 0x559668e8af40 2024-04-23T10:32:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.866+0000 7f035ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559668e8a620 0x559668e95170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.866+0000 7f0369577740 10 client.0 _ll_register_callbacks cb 0x559668d77d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:32:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.867+0000 7f035ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559668e8a620 0x559668e95170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57218/0 (socket says 172.21.15.97:57218) 2024-04-23T10:32:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.867+0000 7f035ffff640 1 -- 172.21.15.97:0/3692969270 learned_addr learned my addr 172.21.15.97:0/3692969270 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:32:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.867+0000 7f035f7fe640 1 --2- 172.21.15.97:0/3692969270 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559668e8af40 0x559668e92460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.867+0000 7f0364d27640 1 --2- 172.21.15.97:0/3692969270 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559668e95800 0x559668e929c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.867+0000 7f035ffff640 1 -- 172.21.15.97:0/3692969270 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559668e8af40 msgr2=0x559668e92460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:32:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.867+0000 7f035ffff640 1 --2- 172.21.15.97:0/3692969270 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559668e8af40 0x559668e92460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.867+0000 7f035ffff640 1 -- 172.21.15.97:0/3692969270 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559668e95800 msgr2=0x559668e929c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:32:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.867+0000 7f035ffff640 1 --2- 172.21.15.97:0/3692969270 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559668e95800 0x559668e929c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.867+0000 7f035ffff640 1 -- 172.21.15.97:0/3692969270 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559668bce620 con 0x559668e8a620 2024-04-23T10:32:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.867+0000 7f035ffff640 1 --2- 172.21.15.97:0/3692969270 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559668e8a620 0x559668e95170 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f035002a930 tx=0x7f0350031810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eddee453d153460 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:32:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.867+0000 7f033f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:32:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.867+0000 7f033f7fe640 1 -- 172.21.15.97:0/3692969270 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0350035040 con 0x559668e8a620 2024-04-23T10:32:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.867+0000 7f033f7fe640 1 -- 172.21.15.97:0/3692969270 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f0350004240 con 0x559668e8a620 2024-04-23T10:32:34.876 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.867+0000 7f0369577740 1 -- 172.21.15.97:0/3692969270 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f035800de50 con 0x559668e8a620 2024-04-23T10:32:34.876 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.867+0000 7f033f7fe640 1 -- 172.21.15.97:0/3692969270 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0350004440 con 0x559668e8a620 2024-04-23T10:32:34.876 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.867+0000 7f0369577740 10 client.16491 fetch_fsmap learned FSMap version 26 2024-04-23T10:32:34.876 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.867+0000 7f0369577740 1 -- 172.21.15.97:0/3692969270 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559668ea4340 con 0x559668e8a620 2024-04-23T10:32:34.876 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.867+0000 7f033f7fe640 1 -- 172.21.15.97:0/3692969270 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f035800de50 con 0x559668e8a620 2024-04-23T10:32:34.876 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.868+0000 7f033f7fe640 1 -- 172.21.15.97:0/3692969270 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f035004f6b0 con 0x559668e8a620 2024-04-23T10:32:34.876 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.868+0000 7f0369577740 10 client.16491 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:32:34.876 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.868+0000 7f0369577740 10 client.16491 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:32:34.876 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.868+0000 7f0369577740 20 client.16491 populate_metadata read hostname 'smithi097' 2024-04-23T10:32:34.876 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.868+0000 7f0369577740 1 --2- 172.21.15.97:0/3692969270 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x559668ea57b0 0x559668ea7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:34.877 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.868+0000 7f0369577740 4 client.16491 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:32:34.877 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.868+0000 7f0369577740 1 -- 172.21.15.97:0/3692969270 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559668dfd390 con 0x559668ea57b0 2024-04-23T10:32:34.877 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.868+0000 7f035f7fe640 1 --2- 172.21.15.97:0/3692969270 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x559668ea57b0 0x559668ea7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:34.877 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.869+0000 7f033f7fe640 10 client.16491 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:32:34.877 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.869+0000 7f035f7fe640 1 --2- 172.21.15.97:0/3692969270 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x559668ea57b0 0x559668ea7ba0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=812f1ab6724b2aa9 server_cookie=964a314968ad4173 in_seq=0 out_seq=0 2024-04-23T10:32:34.877 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.869+0000 7f033f7fe640 1 -- 172.21.15.97:0/3692969270 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559668dfd390 con 0x559668ea57b0 2024-04-23T10:32:34.877 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.869+0000 7f033f7fe640 10 client.16491 handle_command_reply: tid=0 2024-04-23T10:32:34.877 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.869+0000 7f0369577740 1 client.16491 shutdown 2024-04-23T10:32:34.877 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.870+0000 7f0369577740 1 -- 172.21.15.97:0/3692969270 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559668e8a620 msgr2=0x559668e95170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:34.877 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.870+0000 7f0369577740 1 --2- 172.21.15.97:0/3692969270 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559668e8a620 0x559668e95170 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f035002a930 tx=0x7f0350031810 comp rx=0 tx=0).stop 2024-04-23T10:32:34.877 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.870+0000 7f0369577740 1 -- 172.21.15.97:0/3692969270 shutdown_connections 2024-04-23T10:32:34.877 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.870+0000 7f0369577740 1 --2- 172.21.15.97:0/3692969270 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x559668ea57b0 0x559668ea7ba0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:34.877 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.870+0000 7f0369577740 1 --2- 172.21.15.97:0/3692969270 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559668e95800 0x559668e929c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:34.877 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.870+0000 7f0369577740 1 --2- 172.21.15.97:0/3692969270 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559668e8af40 0x559668e92460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:34.878 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.870+0000 7f033f7fe640 0 client.16491 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:32:34.878 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.870+0000 7f0369577740 1 --2- 172.21.15.97:0/3692969270 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559668e8a620 0x559668e95170 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:34.878 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.870+0000 7f0369577740 1 -- 172.21.15.97:0/3692969270 >> 172.21.15.97:0/3692969270 conn(0x559668e86b30 msgr2=0x559668e8c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:32:34.878 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.870+0000 7f0369577740 1 -- 172.21.15.97:0/3692969270 shutdown_connections 2024-04-23T10:32:34.878 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.870+0000 7f0369577740 1 -- 172.21.15.97:0/3692969270 wait complete. 2024-04-23T10:32:34.878 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.870+0000 7f0369577740 20 client.16491 trim_cache size 0 max 16384 2024-04-23T10:32:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.885+0000 7f0369577740 1 Processor -- start 2024-04-23T10:32:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.885+0000 7f0369577740 1 -- start start 2024-04-23T10:32:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.885+0000 7f0369577740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559668e8a620 0x559668ea1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.885+0000 7f0369577740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559668e8af40 0x559668ead4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.885+0000 7f0369577740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559668e95800 0x559668ead9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.885+0000 7f0369577740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f03601adaf0 con 0x559668e8a620 2024-04-23T10:32:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.885+0000 7f0369577740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f03601af630 con 0x559668e8af40 2024-04-23T10:32:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.885+0000 7f0369577740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f03601ae1c0 con 0x559668e95800 2024-04-23T10:32:34.890 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.885+0000 7f035ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559668e8a620 0x559668ea1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:34.890 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.885+0000 7f035ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559668e8a620 0x559668ea1290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57220/0 (socket says 172.21.15.97:57220) 2024-04-23T10:32:34.890 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.885+0000 7f035ffff640 1 -- 172.21.15.97:0/3443481467 learned_addr learned my addr 172.21.15.97:0/3443481467 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:32:34.890 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.885+0000 7f035ffff640 1 -- 172.21.15.97:0/3443481467 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559668e95800 msgr2=0x559668ead9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:32:34.890 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.885+0000 7f035ffff640 1 --2- 172.21.15.97:0/3443481467 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559668e95800 0x559668ead9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:34.890 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.885+0000 7f035ffff640 1 -- 172.21.15.97:0/3443481467 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559668e8af40 msgr2=0x559668ead4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:32:34.890 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.885+0000 7f035f7fe640 1 --2- 172.21.15.97:0/3443481467 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559668e8af40 0x559668ead4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:34.890 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.885+0000 7f035ffff640 1 --2- 172.21.15.97:0/3443481467 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559668e8af40 0x559668ead4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:34.890 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.885+0000 7f035ffff640 1 -- 172.21.15.97:0/3443481467 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559668e92d30 con 0x559668e8a620 2024-04-23T10:32:34.890 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.885+0000 7f035f7fe640 1 --2- 172.21.15.97:0/3443481467 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559668e8af40 0x559668ead4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:32:34.891 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.885+0000 7f035ffff640 1 --2- 172.21.15.97:0/3443481467 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559668e8a620 0x559668ea1290 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f035004fe60 tx=0x7f035002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33b51a0d9eaf28bd server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:32:34.891 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.885+0000 7f0345ffb640 1 -- 172.21.15.97:0/3443481467 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0350035070 con 0x559668e8a620 2024-04-23T10:32:34.891 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.886+0000 7f0345ffb640 1 -- 172.21.15.97:0/3443481467 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0350004200 con 0x559668e8a620 2024-04-23T10:32:34.891 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.886+0000 7f0345ffb640 1 -- 172.21.15.97:0/3443481467 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0350003b40 con 0x559668e8a620 2024-04-23T10:32:34.891 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.886+0000 7f0369577740 1 -- 172.21.15.97:0/3443481467 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559668e8a620 msgr2=0x559668ea1290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:34.891 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.886+0000 7f0369577740 1 --2- 172.21.15.97:0/3443481467 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559668e8a620 0x559668ea1290 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f035004fe60 tx=0x7f035002a410 comp rx=0 tx=0).stop 2024-04-23T10:32:34.891 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.886+0000 7f0369577740 1 -- 172.21.15.97:0/3443481467 shutdown_connections 2024-04-23T10:32:34.891 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.886+0000 7f0369577740 1 --2- 172.21.15.97:0/3443481467 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559668e95800 0x559668ead9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:34.891 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.886+0000 7f0369577740 1 --2- 172.21.15.97:0/3443481467 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559668e8af40 0x559668ead4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:34.891 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.886+0000 7f0369577740 1 --2- 172.21.15.97:0/3443481467 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559668e8a620 0x559668ea1290 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:34.891 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.886+0000 7f0369577740 1 -- 172.21.15.97:0/3443481467 >> 172.21.15.97:0/3443481467 conn(0x559668e86b30 msgr2=0x559668e85cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:32:34.891 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.886+0000 7f0369577740 1 -- 172.21.15.97:0/3443481467 shutdown_connections 2024-04-23T10:32:34.891 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.886+0000 7f0369577740 1 -- 172.21.15.97:0/3443481467 wait complete. 2024-04-23T10:32:34.892 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.886+0000 7f0369577740 1 -- start start 2024-04-23T10:32:34.892 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.887+0000 7f0369577740 1 Processor -- start 2024-04-23T10:32:34.892 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.887+0000 7f0369577740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559668e8a620 0x559668e9b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:34.892 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.887+0000 7f0369577740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559668e8af40 0x559668e9b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:34.892 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.887+0000 7f0369577740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559668e95800 0x559668e9bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:34.892 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.887+0000 7f0369577740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f03601aef60 con 0x559668e8af40 2024-04-23T10:32:34.892 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.887+0000 7f0369577740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f03601ae890 con 0x559668e8a620 2024-04-23T10:32:34.892 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.887+0000 7f0369577740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f03601abaf0 con 0x559668e95800 2024-04-23T10:32:34.892 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.887+0000 7f0369577740 10 client.0 _ll_register_callbacks cb 0x7f0350031eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:32:34.892 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.887+0000 7f035f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559668e8af40 0x559668e9b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:34.892 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.887+0000 7f035f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559668e8af40 0x559668e9b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57236/0 (socket says 172.21.15.97:57236) 2024-04-23T10:32:34.892 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.887+0000 7f035f7fe640 1 -- 172.21.15.97:0/2267578295 learned_addr learned my addr 172.21.15.97:0/2267578295 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:32:34.892 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.887+0000 7f035ffff640 1 --2- 172.21.15.97:0/2267578295 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559668e8a620 0x559668e9b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:34.892 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.887+0000 7f035f7fe640 1 -- 172.21.15.97:0/2267578295 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559668e95800 msgr2=0x559668e9bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:32:34.893 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.887+0000 7f035f7fe640 1 --2- 172.21.15.97:0/2267578295 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559668e95800 0x559668e9bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:34.893 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.887+0000 7f035f7fe640 1 -- 172.21.15.97:0/2267578295 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559668e8a620 msgr2=0x559668e9b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:32:34.893 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.887+0000 7f035f7fe640 1 --2- 172.21.15.97:0/2267578295 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559668e8a620 0x559668e9b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:34.893 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.887+0000 7f035f7fe640 1 -- 172.21.15.97:0/2267578295 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559668f0ad30 con 0x559668e8af40 2024-04-23T10:32:34.893 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.887+0000 7f035ffff640 1 --2- 172.21.15.97:0/2267578295 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559668e8a620 0x559668e9b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:32:34.893 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.887+0000 7f035f7fe640 1 --2- 172.21.15.97:0/2267578295 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559668e8af40 0x559668e9b670 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f03540ce0e0 tx=0x7f0354087040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d03110c149c5427a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:32:34.893 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.887+0000 7f033f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:32:34.893 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.888+0000 7f033f7fe640 1 -- 172.21.15.97:0/2267578295 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f035401d040 con 0x559668e8af40 2024-04-23T10:32:34.893 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.888+0000 7f033f7fe640 1 -- 172.21.15.97:0/2267578295 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f03540055d0 con 0x559668e8af40 2024-04-23T10:32:34.893 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.888+0000 7f0369577740 1 -- 172.21.15.97:0/2267578295 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0350004200 con 0x559668e8af40 2024-04-23T10:32:34.893 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.888+0000 7f033f7fe640 1 -- 172.21.15.97:0/2267578295 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f03540a54f0 con 0x559668e8af40 2024-04-23T10:32:34.893 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.888+0000 7f0369577740 10 client.16497 fetch_fsmap learned FSMap version 26 2024-04-23T10:32:34.893 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.888+0000 7f0369577740 1 -- 172.21.15.97:0/2267578295 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559668f0bf10 con 0x559668e8af40 2024-04-23T10:32:34.893 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.888+0000 7f033f7fe640 1 -- 172.21.15.97:0/2267578295 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0350004200 con 0x559668e8af40 2024-04-23T10:32:34.893 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.888+0000 7f033f7fe640 1 -- 172.21.15.97:0/2267578295 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f03540870b0 con 0x559668e8af40 2024-04-23T10:32:34.894 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.888+0000 7f0369577740 10 client.16497 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:32:34.894 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.888+0000 7f0369577740 10 client.16497 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:32:34.894 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.888+0000 7f0369577740 20 client.16497 populate_metadata read hostname 'smithi097' 2024-04-23T10:32:34.894 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.888+0000 7f0369577740 1 --2- 172.21.15.97:0/2267578295 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x559668f0c690 0x559668f0eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:34.894 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.888+0000 7f0369577740 4 client.16497 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:32:34.894 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.888+0000 7f0369577740 1 -- 172.21.15.97:0/2267578295 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559668df7200 con 0x559668f0c690 2024-04-23T10:32:34.894 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.888+0000 7f035ffff640 1 --2- 172.21.15.97:0/2267578295 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x559668f0c690 0x559668f0eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:34.894 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.889+0000 7f035ffff640 1 --2- 172.21.15.97:0/2267578295 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x559668f0c690 0x559668f0eab0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9745a8c25fbe421 server_cookie=8d9dd440333e774e in_seq=0 out_seq=0 2024-04-23T10:32:34.894 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.889+0000 7f033f7fe640 10 client.16497 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:32:34.902 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.901+0000 7f033f7fe640 1 -- 172.21.15.97:0/2267578295 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559668df7200 con 0x559668f0c690 2024-04-23T10:32:34.902 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.901+0000 7f033f7fe640 10 client.16497 handle_command_reply: tid=0 2024-04-23T10:32:34.902 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.901+0000 7f0369577740 1 client.16497 shutdown 2024-04-23T10:32:34.902 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.901+0000 7f0369577740 1 -- 172.21.15.97:0/2267578295 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559668e8af40 msgr2=0x559668e9b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:34.902 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.901+0000 7f0369577740 1 --2- 172.21.15.97:0/2267578295 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559668e8af40 0x559668e9b670 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f03540ce0e0 tx=0x7f0354087040 comp rx=0 tx=0).stop 2024-04-23T10:32:34.902 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.901+0000 7f0369577740 1 -- 172.21.15.97:0/2267578295 shutdown_connections 2024-04-23T10:32:34.903 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.901+0000 7f0369577740 1 --2- 172.21.15.97:0/2267578295 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x559668f0c690 0x559668f0eab0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:34.903 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.901+0000 7f0369577740 1 --2- 172.21.15.97:0/2267578295 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559668e95800 0x559668e9bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:34.903 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.901+0000 7f0369577740 1 --2- 172.21.15.97:0/2267578295 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559668e8af40 0x559668e9b670 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:34.903 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.901+0000 7f033f7fe640 0 client.16497 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:32:34.903 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.901+0000 7f0369577740 1 --2- 172.21.15.97:0/2267578295 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559668e8a620 0x559668e9b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:34.903 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.901+0000 7f0369577740 1 -- 172.21.15.97:0/2267578295 >> 172.21.15.97:0/2267578295 conn(0x559668e86b30 msgr2=0x559668ea0280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:32:34.903 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.901+0000 7f0369577740 1 -- 172.21.15.97:0/2267578295 shutdown_connections 2024-04-23T10:32:34.903 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.901+0000 7f0369577740 1 -- 172.21.15.97:0/2267578295 wait complete. 2024-04-23T10:32:34.903 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.901+0000 7f0369577740 20 client.16497 trim_cache size 0 max 16384 2024-04-23T10:32:34.903 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:32:34.903 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:32:34.904 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:32:34.904 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:32:34.904 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.902+0000 7f036678a640 1 -- 172.21.15.97:0/244702043 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f03480a3dc0 msgr2=0x7f03480a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:34.904 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.902+0000 7f036678a640 1 --2- 172.21.15.97:0/244702043 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f03480a3dc0 0x7f03480a6280 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f035800afd0 tx=0x7f035800a560 comp rx=0 tx=0).stop 2024-04-23T10:32:34.905 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.902+0000 7f036678a640 1 -- 172.21.15.97:0/244702043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0360107280 msgr2=0x7f0360198c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:34.905 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.902+0000 7f036678a640 1 --2- 172.21.15.97:0/244702043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0360107280 0x7f0360198c80 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f035400c970 tx=0x7f035400ce30 comp rx=0 tx=0).stop 2024-04-23T10:32:34.905 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.902+0000 7f036678a640 1 -- 172.21.15.97:0/244702043 shutdown_connections 2024-04-23T10:32:34.905 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.902+0000 7f036678a640 1 --2- 172.21.15.97:0/244702043 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f03480a3dc0 0x7f03480a6280 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:34.905 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.902+0000 7f036678a640 1 --2- 172.21.15.97:0/244702043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f036019d000 0x7f036019d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:34.905 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.902+0000 7f036678a640 1 --2- 172.21.15.97:0/244702043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0360107280 0x7f0360198c80 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:34.905 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.902+0000 7f036678a640 1 --2- 172.21.15.97:0/244702043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03601006c0 0x7f0360198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:34.905 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.902+0000 7f036678a640 1 -- 172.21.15.97:0/244702043 >> 172.21.15.97:0/244702043 conn(0x7f0360077370 msgr2=0x7f03600fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:32:34.905 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.902+0000 7f036678a640 1 -- 172.21.15.97:0/244702043 shutdown_connections 2024-04-23T10:32:34.905 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:34.902+0000 7f036678a640 1 -- 172.21.15.97:0/244702043 wait complete. 2024-04-23T10:32:35.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:32:35 smithi196 ceph-mon[32919]: pgmap v194: 97 pgs: 97 active+clean; 3.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 32 KiB/s wr, 18 op/s 2024-04-23T10:32:35.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:32:35 smithi097 ceph-mon[29197]: pgmap v194: 97 pgs: 97 active+clean; 3.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 32 KiB/s wr, 18 op/s 2024-04-23T10:32:35.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:32:35 smithi183 ceph-mon[32317]: pgmap v194: 97 pgs: 97 active+clean; 3.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 32 KiB/s wr, 18 op/s 2024-04-23T10:32:37.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:32:37 smithi196 ceph-mon[32919]: pgmap v195: 97 pgs: 97 active+clean; 3.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 12 op/s 2024-04-23T10:32:37.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:32:37 smithi097 ceph-mon[29197]: pgmap v195: 97 pgs: 97 active+clean; 3.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 12 op/s 2024-04-23T10:32:37.399 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:32:37 smithi183 ceph-mon[32317]: pgmap v195: 97 pgs: 97 active+clean; 3.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 12 op/s 2024-04-23T10:32:39.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:32:39 smithi097 ceph-mon[29197]: pgmap v196: 97 pgs: 97 active+clean; 3.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 50 KiB/s wr, 21 op/s 2024-04-23T10:32:39.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:32:39 smithi183 ceph-mon[32317]: pgmap v196: 97 pgs: 97 active+clean; 3.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 50 KiB/s wr, 21 op/s 2024-04-23T10:32:39.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:32:39 smithi196 ceph-mon[32919]: pgmap v196: 97 pgs: 97 active+clean; 3.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 50 KiB/s wr, 21 op/s 2024-04-23T10:32:41.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:32:41 smithi097 ceph-mon[29197]: pgmap v197: 97 pgs: 97 active+clean; 3.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 37 KiB/s wr, 15 op/s 2024-04-23T10:32:41.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:32:41 smithi183 ceph-mon[32317]: pgmap v197: 97 pgs: 97 active+clean; 3.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 37 KiB/s wr, 15 op/s 2024-04-23T10:32:41.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:32:41 smithi196 ceph-mon[32919]: pgmap v197: 97 pgs: 97 active+clean; 3.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 37 KiB/s wr, 15 op/s 2024-04-23T10:32:43.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:32:43 smithi097 ceph-mon[29197]: pgmap v198: 97 pgs: 97 active+clean; 3.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 37 KiB/s wr, 15 op/s 2024-04-23T10:32:43.394 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:32:43 smithi196 ceph-mon[32919]: pgmap v198: 97 pgs: 97 active+clean; 3.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 37 KiB/s wr, 15 op/s 2024-04-23T10:32:43.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:32:43 smithi183 ceph-mon[32317]: pgmap v198: 97 pgs: 97 active+clean; 3.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 37 KiB/s wr, 15 op/s 2024-04-23T10:32:45.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:32:45 smithi097 ceph-mon[29197]: pgmap v199: 97 pgs: 97 active+clean; 3.5 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-04-23T10:32:45.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:32:45 smithi183 ceph-mon[32317]: pgmap v199: 97 pgs: 97 active+clean; 3.5 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-04-23T10:32:45.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:32:45 smithi196 ceph-mon[32919]: pgmap v199: 97 pgs: 97 active+clean; 3.5 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-04-23T10:32:47.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:32:47 smithi097 ceph-mon[29197]: pgmap v200: 97 pgs: 97 active+clean; 3.5 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-23T10:32:47.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:32:47 smithi183 ceph-mon[32317]: pgmap v200: 97 pgs: 97 active+clean; 3.5 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-23T10:32:47.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:32:47 smithi196 ceph-mon[32919]: pgmap v200: 97 pgs: 97 active+clean; 3.5 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-23T10:32:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:47.997+0000 7fcdc1968640 1 -- 172.21.15.97:0/2626185347 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdbc1021d0 msgr2=0x7fcdbc1025b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:47.997+0000 7fcdc1968640 1 --2- 172.21.15.97:0/2626185347 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdbc1021d0 0x7fcdbc1025b0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fcdac009900 tx=0x7fcdac02a410 comp rx=0 tx=0).stop 2024-04-23T10:32:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:47.998+0000 7fcdc1968640 1 -- 172.21.15.97:0/2626185347 shutdown_connections 2024-04-23T10:32:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:47.998+0000 7fcdc1968640 1 --2- 172.21.15.97:0/2626185347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcdbc1072a0 0x7fcdbc10ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:47.998+0000 7fcdc1968640 1 --2- 172.21.15.97:0/2626185347 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcdbc102b80 0x7fcdbc106bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:47.998+0000 7fcdc1968640 1 --2- 172.21.15.97:0/2626185347 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdbc1021d0 0x7fcdbc1025b0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:47.998+0000 7fcdc1968640 1 -- 172.21.15.97:0/2626185347 >> 172.21.15.97:0/2626185347 conn(0x7fcdbc0793e0 msgr2=0x7fcdbc0797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:32:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:47.998+0000 7fcdc1968640 1 -- 172.21.15.97:0/2626185347 shutdown_connections 2024-04-23T10:32:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:47.998+0000 7fcdc1968640 1 -- 172.21.15.97:0/2626185347 wait complete. 2024-04-23T10:32:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:47.998+0000 7fcdc1968640 1 Processor -- start 2024-04-23T10:32:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:47.998+0000 7fcdc1968640 1 -- start start 2024-04-23T10:32:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:47.998+0000 7fcdc1968640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdbc1021d0 0x7fcdbc1a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:47.998+0000 7fcdc1968640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcdbc1072a0 0x7fcdbc1a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:47.998+0000 7fcdc1968640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcdbc19a6b0 0x7fcdbc19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:47.998+0000 7fcdc1968640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcdbc112200 con 0x7fcdbc1021d0 2024-04-23T10:32:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:47.998+0000 7fcdc1968640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcdbc112080 con 0x7fcdbc1072a0 2024-04-23T10:32:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:47.998+0000 7fcdc1968640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcdbc112380 con 0x7fcdbc19a6b0 2024-04-23T10:32:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:47.999+0000 7fcdbaffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdbc1021d0 0x7fcdbc1a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:47.999+0000 7fcdbaffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdbc1021d0 0x7fcdbc1a05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39538/0 (socket says 172.21.15.97:39538) 2024-04-23T10:32:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:47.999+0000 7fcdbaffd640 1 -- 172.21.15.97:0/2044729991 learned_addr learned my addr 172.21.15.97:0/2044729991 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:32:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:47.999+0000 7fcdbb7fe640 1 --2- 172.21.15.97:0/2044729991 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcdbc19a6b0 0x7fcdbc19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:47.999+0000 7fcdb3fff640 1 --2- 172.21.15.97:0/2044729991 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcdbc1072a0 0x7fcdbc1a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:47.999+0000 7fcdbaffd640 1 -- 172.21.15.97:0/2044729991 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcdbc19a6b0 msgr2=0x7fcdbc19ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:47.999+0000 7fcdbaffd640 1 --2- 172.21.15.97:0/2044729991 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcdbc19a6b0 0x7fcdbc19ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:47.999+0000 7fcdbaffd640 1 -- 172.21.15.97:0/2044729991 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcdbc1072a0 msgr2=0x7fcdbc1a0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:47.999+0000 7fcdbaffd640 1 --2- 172.21.15.97:0/2044729991 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcdbc1072a0 0x7fcdbc1a0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:47.999+0000 7fcdbaffd640 1 -- 172.21.15.97:0/2044729991 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdbc19b2b0 con 0x7fcdbc1021d0 2024-04-23T10:32:48.001 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:47.999+0000 7fcdb3fff640 1 --2- 172.21.15.97:0/2044729991 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcdbc1072a0 0x7fcdbc1a0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:32:48.001 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:47.999+0000 7fcdbaffd640 1 --2- 172.21.15.97:0/2044729991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdbc1021d0 0x7fcdbc1a05f0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fcdac005940 tx=0x7fcdac004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:32:48.001 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:47.999+0000 7fcdc0966640 1 -- 172.21.15.97:0/2044729991 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdac03f030 con 0x7fcdbc1021d0 2024-04-23T10:32:48.001 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:47.999+0000 7fcdc0966640 1 -- 172.21.15.97:0/2044729991 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdac003aa0 con 0x7fcdbc1021d0 2024-04-23T10:32:48.001 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:47.999+0000 7fcdc1968640 1 -- 172.21.15.97:0/2044729991 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdbc19b540 con 0x7fcdbc1021d0 2024-04-23T10:32:48.001 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:47.999+0000 7fcdc0966640 1 -- 172.21.15.97:0/2044729991 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdac035070 con 0x7fcdbc1021d0 2024-04-23T10:32:48.001 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:47.999+0000 7fcdc1968640 1 -- 172.21.15.97:0/2044729991 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdbc1a7680 con 0x7fcdbc1021d0 2024-04-23T10:32:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.002+0000 7fcdc0966640 1 -- 172.21.15.97:0/2044729991 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fcdac042030 con 0x7fcdbc1021d0 2024-04-23T10:32:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.002+0000 7fcdc4682740 1 Processor -- start 2024-04-23T10:32:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.002+0000 7fcdc4682740 1 -- start start 2024-04-23T10:32:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.002+0000 7fcdc4682740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c42f989620 0x55c42f989a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.002+0000 7fcdc4682740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c42f989f40 0x55c42f9942c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.002+0000 7fcdc4682740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c42f994800 0x55c42f996bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.002+0000 7fcdc4682740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c42f8ee970 con 0x55c42f994800 2024-04-23T10:32:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.002+0000 7fcdc4682740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c42f88b4a0 con 0x55c42f989620 2024-04-23T10:32:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.002+0000 7fcdc4682740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c42f8ea930 con 0x55c42f989f40 2024-04-23T10:32:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.002+0000 7fcdc0966640 1 --2- 172.21.15.97:0/2044729991 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fcda80a3dc0 0x7fcda80a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.003+0000 7fcdb3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c42f994800 0x55c42f996bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:48.009 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.003+0000 7fcdbb7fe640 1 --2- 172.21.15.97:0/2044729991 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fcda80a3dc0 0x7fcda80a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:48.009 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.003+0000 7fcdbaffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c42f989f40 0x55c42f9942c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:48.009 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.003+0000 7fcdbaffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c42f989f40 0x55c42f9942c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.97:48744/0 (socket says 172.21.15.97:48744) 2024-04-23T10:32:48.009 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.003+0000 7fcdbaffd640 1 -- 172.21.15.97:0/3278423531 learned_addr learned my addr 172.21.15.97:0/3278423531 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:32:48.009 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.003+0000 7fcdbb7fe640 1 --2- 172.21.15.97:0/3278423531 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c42f989620 0x55c42f989a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:48.009 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.003+0000 7fcdc0966640 1 -- 172.21.15.97:0/2044729991 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fcdac0f0f00 con 0x7fcdbc1021d0 2024-04-23T10:32:48.009 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.003+0000 7fcdbaffd640 1 -- 172.21.15.97:0/3278423531 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c42f989620 msgr2=0x55c42f989a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:32:48.009 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.003+0000 7fcdbaffd640 1 --2- 172.21.15.97:0/3278423531 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c42f989620 0x55c42f989a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:48.009 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.003+0000 7fcdbaffd640 1 -- 172.21.15.97:0/3278423531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c42f994800 msgr2=0x55c42f996bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:32:48.009 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.003+0000 7fcdbaffd640 1 --2- 172.21.15.97:0/3278423531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c42f994800 0x55c42f996bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:48.009 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.003+0000 7fcdbaffd640 1 -- 172.21.15.97:0/3278423531 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c42f601210 con 0x55c42f989f40 2024-04-23T10:32:48.009 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.003+0000 7fcdb3fff640 1 --2- 172.21.15.97:0/3278423531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c42f994800 0x55c42f996bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T10:32:48.009 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.003+0000 7fcdbb7fe640 1 --2- 172.21.15.97:0/2044729991 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fcda80a3dc0 0x7fcda80a6280 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fcdb4007310 tx=0x7fcdb4007d50 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:32:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.003+0000 7fcdbaffd640 1 --2- 172.21.15.97:0/3278423531 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c42f989f40 0x55c42f9942c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcdac006010 tx=0x7fcdac0e7fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e878b72cf8adb45 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:32:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.003+0000 7fcdb1ffb640 1 -- 172.21.15.97:0/3278423531 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdac049070 con 0x55c42f989f40 2024-04-23T10:32:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.004+0000 7fcdb1ffb640 1 -- 172.21.15.97:0/3278423531 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdac0bce90 con 0x55c42f989f40 2024-04-23T10:32:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.004+0000 7fcdb1ffb640 1 -- 172.21.15.97:0/3278423531 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdac03bcc0 con 0x55c42f989f40 2024-04-23T10:32:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.004+0000 7fcdc4682740 1 -- 172.21.15.97:0/3278423531 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c42f989f40 msgr2=0x55c42f9942c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.004+0000 7fcdc4682740 1 --2- 172.21.15.97:0/3278423531 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c42f989f40 0x55c42f9942c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcdac006010 tx=0x7fcdac0e7fa0 comp rx=0 tx=0).stop 2024-04-23T10:32:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.004+0000 7fcdc4682740 1 -- 172.21.15.97:0/3278423531 shutdown_connections 2024-04-23T10:32:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.004+0000 7fcdc4682740 1 --2- 172.21.15.97:0/3278423531 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c42f994800 0x55c42f996bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.004+0000 7fcdc4682740 1 --2- 172.21.15.97:0/3278423531 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c42f989f40 0x55c42f9942c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.004+0000 7fcdc4682740 1 --2- 172.21.15.97:0/3278423531 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c42f989620 0x55c42f989a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.004+0000 7fcdc4682740 1 -- 172.21.15.97:0/3278423531 >> 172.21.15.97:0/3278423531 conn(0x55c42f985b30 msgr2=0x55c42f985f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:32:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.004+0000 7fcdc4682740 1 -- 172.21.15.97:0/3278423531 shutdown_connections 2024-04-23T10:32:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.004+0000 7fcdc4682740 1 -- 172.21.15.97:0/3278423531 wait complete. 2024-04-23T10:32:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.005+0000 7fcdc4682740 1 -- start start 2024-04-23T10:32:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.005+0000 7fcdc4682740 1 Processor -- start 2024-04-23T10:32:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.005+0000 7fcdc4682740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c42f989620 0x55c42f9941b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.005+0000 7fcdc4682740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c42f989f40 0x55c42f9914a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.005+0000 7fcdc4682740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c42f994800 0x55c42f991a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.005+0000 7fcdc4682740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c42f8ea930 con 0x55c42f989f40 2024-04-23T10:32:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.005+0000 7fcdc4682740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c42f8890b0 con 0x55c42f994800 2024-04-23T10:32:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.005+0000 7fcdc4682740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c42f9971f0 con 0x55c42f989620 2024-04-23T10:32:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.005+0000 7fcdc4682740 10 client.0 _ll_register_callbacks cb 0x55c42f876d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:32:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.005+0000 7fcdb3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c42f989f40 0x55c42f9914a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.005+0000 7fcdb3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c42f989f40 0x55c42f9914a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39560/0 (socket says 172.21.15.97:39560) 2024-04-23T10:32:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.005+0000 7fcdb3fff640 1 -- 172.21.15.97:0/212501399 learned_addr learned my addr 172.21.15.97:0/212501399 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:32:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.005+0000 7fcdbaffd640 1 --2- 172.21.15.97:0/212501399 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c42f989620 0x55c42f9941b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.005+0000 7fcdbb7fe640 1 --2- 172.21.15.97:0/212501399 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c42f994800 0x55c42f991a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.005+0000 7fcdb3fff640 1 -- 172.21.15.97:0/212501399 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c42f989620 msgr2=0x55c42f9941b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:32:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.005+0000 7fcdb3fff640 1 --2- 172.21.15.97:0/212501399 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c42f989620 0x55c42f9941b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.005+0000 7fcdb3fff640 1 -- 172.21.15.97:0/212501399 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c42f994800 msgr2=0x55c42f991a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:32:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.005+0000 7fcdb3fff640 1 --2- 172.21.15.97:0/212501399 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c42f994800 0x55c42f991a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.006+0000 7fcdb3fff640 1 -- 172.21.15.97:0/212501399 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c42f6cd620 con 0x55c42f989f40 2024-04-23T10:32:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.006+0000 7fcdbb7fe640 1 --2- 172.21.15.97:0/212501399 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c42f994800 0x55c42f991a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:32:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.006+0000 7fcdb3fff640 1 --2- 172.21.15.97:0/212501399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c42f989f40 0x55c42f9914a0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fcda4012700 tx=0x7fcda4012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f44b9b3b4e4e1e6c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:32:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.006+0000 7fcd9affd640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:32:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.006+0000 7fcd9affd640 1 -- 172.21.15.97:0/212501399 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda4011c70 con 0x55c42f989f40 2024-04-23T10:32:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.006+0000 7fcd9affd640 1 -- 172.21.15.97:0/212501399 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fcda4011e10 con 0x55c42f989f40 2024-04-23T10:32:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.006+0000 7fcdc4682740 1 -- 172.21.15.97:0/212501399 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdac0bce90 con 0x55c42f989f40 2024-04-23T10:32:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.006+0000 7fcd9affd640 1 -- 172.21.15.97:0/212501399 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcda4005020 con 0x55c42f989f40 2024-04-23T10:32:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.006+0000 7fcdc4682740 10 client.16509 fetch_fsmap learned FSMap version 26 2024-04-23T10:32:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.006+0000 7fcdc4682740 1 -- 172.21.15.97:0/212501399 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c42f9a3080 con 0x55c42f989f40 2024-04-23T10:32:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.006+0000 7fcd9affd640 1 -- 172.21.15.97:0/212501399 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdac0bce90 con 0x55c42f989f40 2024-04-23T10:32:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.006+0000 7fcd9affd640 1 -- 172.21.15.97:0/212501399 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fcda401c470 con 0x55c42f989f40 2024-04-23T10:32:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.006+0000 7fcdc4682740 10 client.16509 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:32:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.006+0000 7fcdc4682740 10 client.16509 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:32:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.006+0000 7fcdc4682740 20 client.16509 populate_metadata read hostname 'smithi097' 2024-04-23T10:32:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.006+0000 7fcdc4682740 1 --2- 172.21.15.97:0/212501399 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55c42f9a4590 0x55c42f9a69b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.006+0000 7fcdc4682740 4 client.16509 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:32:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.006+0000 7fcdc4682740 1 -- 172.21.15.97:0/212501399 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c42f8fc390 con 0x55c42f9a4590 2024-04-23T10:32:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.007+0000 7fcdbaffd640 1 --2- 172.21.15.97:0/212501399 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55c42f9a4590 0x55c42f9a69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.007+0000 7fcd9affd640 10 client.16509 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:32:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.007+0000 7fcdbaffd640 1 --2- 172.21.15.97:0/212501399 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55c42f9a4590 0x55c42f9a69b0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=858ddeff3421473f server_cookie=4ca85d49d0692a9f in_seq=0 out_seq=0 2024-04-23T10:32:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.008+0000 7fcd9affd640 1 -- 172.21.15.97:0/212501399 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c42f8fc390 con 0x55c42f9a4590 2024-04-23T10:32:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.008+0000 7fcd9affd640 10 client.16509 handle_command_reply: tid=0 2024-04-23T10:32:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.008+0000 7fcdc4682740 1 client.16509 shutdown 2024-04-23T10:32:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.008+0000 7fcdc4682740 1 -- 172.21.15.97:0/212501399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c42f989f40 msgr2=0x55c42f9914a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:48.014 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.008+0000 7fcdc4682740 1 --2- 172.21.15.97:0/212501399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c42f989f40 0x55c42f9914a0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fcda4012700 tx=0x7fcda4012bc0 comp rx=0 tx=0).stop 2024-04-23T10:32:48.014 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.008+0000 7fcdc4682740 1 -- 172.21.15.97:0/212501399 shutdown_connections 2024-04-23T10:32:48.014 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.008+0000 7fcdc4682740 1 --2- 172.21.15.97:0/212501399 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55c42f9a4590 0x55c42f9a69b0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:48.014 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.008+0000 7fcdc4682740 1 --2- 172.21.15.97:0/212501399 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c42f994800 0x55c42f991a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:48.014 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.008+0000 7fcdc4682740 1 --2- 172.21.15.97:0/212501399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c42f989f40 0x55c42f9914a0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:48.014 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.008+0000 7fcd9affd640 0 client.16509 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:32:48.014 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.008+0000 7fcdc4682740 1 --2- 172.21.15.97:0/212501399 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c42f989620 0x55c42f9941b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:48.014 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.008+0000 7fcdc4682740 1 -- 172.21.15.97:0/212501399 >> 172.21.15.97:0/212501399 conn(0x55c42f985b30 msgr2=0x55c42f98b860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:32:48.014 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.009+0000 7fcdc4682740 1 -- 172.21.15.97:0/212501399 shutdown_connections 2024-04-23T10:32:48.014 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.009+0000 7fcdc4682740 1 -- 172.21.15.97:0/212501399 wait complete. 2024-04-23T10:32:48.014 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.009+0000 7fcdc4682740 20 client.16509 trim_cache size 0 max 16384 2024-04-23T10:32:48.027 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.023+0000 7fcdc4682740 1 Processor -- start 2024-04-23T10:32:48.027 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.024+0000 7fcdc4682740 1 -- start start 2024-04-23T10:32:48.027 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.024+0000 7fcdc4682740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c42f989620 0x55c42f99ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:48.027 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.024+0000 7fcdc4682740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c42f989f40 0x55c42f990600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:48.027 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.024+0000 7fcdc4682740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c42f994800 0x55c42f984b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:48.027 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.024+0000 7fcdc4682740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcdbc1afba0 con 0x55c42f989f40 2024-04-23T10:32:48.027 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.024+0000 7fcdc4682740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcdbc1b16e0 con 0x55c42f994800 2024-04-23T10:32:48.028 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.024+0000 7fcdc4682740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcdbc1b0270 con 0x55c42f989620 2024-04-23T10:32:48.028 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.024+0000 7fcdb3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c42f989f40 0x55c42f990600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:48.028 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.024+0000 7fcdb3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c42f989f40 0x55c42f990600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39570/0 (socket says 172.21.15.97:39570) 2024-04-23T10:32:48.028 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.024+0000 7fcdb3fff640 1 -- 172.21.15.97:0/3226029587 learned_addr learned my addr 172.21.15.97:0/3226029587 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:32:48.028 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.024+0000 7fcdbb7fe640 1 --2- 172.21.15.97:0/3226029587 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c42f994800 0x55c42f984b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:48.028 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.024+0000 7fcdbaffd640 1 --2- 172.21.15.97:0/3226029587 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c42f989620 0x55c42f99ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:48.028 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.024+0000 7fcdb3fff640 1 -- 172.21.15.97:0/3226029587 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c42f989620 msgr2=0x55c42f99ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:32:48.028 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.024+0000 7fcdb3fff640 1 --2- 172.21.15.97:0/3226029587 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c42f989620 0x55c42f99ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:48.028 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.024+0000 7fcdb3fff640 1 -- 172.21.15.97:0/3226029587 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c42f994800 msgr2=0x55c42f984b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:32:48.028 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.024+0000 7fcdb3fff640 1 --2- 172.21.15.97:0/3226029587 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c42f994800 0x55c42f984b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:48.028 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.024+0000 7fcdb3fff640 1 -- 172.21.15.97:0/3226029587 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c42f98f9f0 con 0x55c42f989f40 2024-04-23T10:32:48.028 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.024+0000 7fcdb3fff640 1 --2- 172.21.15.97:0/3226029587 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c42f989f40 0x55c42f990600 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fcda401c8c0 tx=0x7fcda401de90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=724a43d50a2f580c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:32:48.028 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.024+0000 7fcdb17fa640 1 -- 172.21.15.97:0/3226029587 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda401c470 con 0x55c42f989f40 2024-04-23T10:32:48.028 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.025+0000 7fcdb17fa640 1 -- 172.21.15.97:0/3226029587 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcda401c610 con 0x55c42f989f40 2024-04-23T10:32:48.028 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.025+0000 7fcdb17fa640 1 -- 172.21.15.97:0/3226029587 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda4004660 con 0x55c42f989f40 2024-04-23T10:32:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.025+0000 7fcdc4682740 1 -- 172.21.15.97:0/3226029587 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c42f989f40 msgr2=0x55c42f990600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.025+0000 7fcdc4682740 1 --2- 172.21.15.97:0/3226029587 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c42f989f40 0x55c42f990600 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fcda401c8c0 tx=0x7fcda401de90 comp rx=0 tx=0).stop 2024-04-23T10:32:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.025+0000 7fcdc4682740 1 -- 172.21.15.97:0/3226029587 shutdown_connections 2024-04-23T10:32:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.025+0000 7fcdc4682740 1 --2- 172.21.15.97:0/3226029587 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c42f994800 0x55c42f984b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.025+0000 7fcdc4682740 1 --2- 172.21.15.97:0/3226029587 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c42f989f40 0x55c42f990600 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.025+0000 7fcdc4682740 1 --2- 172.21.15.97:0/3226029587 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c42f989620 0x55c42f99ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.025+0000 7fcdc4682740 1 -- 172.21.15.97:0/3226029587 >> 172.21.15.97:0/3226029587 conn(0x55c42f985b30 msgr2=0x55c42f984420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:32:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.025+0000 7fcdc4682740 1 -- 172.21.15.97:0/3226029587 shutdown_connections 2024-04-23T10:32:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.025+0000 7fcdc4682740 1 -- 172.21.15.97:0/3226029587 wait complete. 2024-04-23T10:32:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.025+0000 7fcdc4682740 1 -- start start 2024-04-23T10:32:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.026+0000 7fcdc4682740 1 Processor -- start 2024-04-23T10:32:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.026+0000 7fcdc4682740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c42f989620 0x55c42f998080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.026+0000 7fcdc4682740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c42f989f40 0x55c42f9985c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.026+0000 7fcdc4682740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c42f994800 0x55c42f998b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.026+0000 7fcdc4682740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcdbc1b1010 con 0x55c42f989620 2024-04-23T10:32:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.026+0000 7fcdc4682740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcdbc1b0940 con 0x55c42f994800 2024-04-23T10:32:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.026+0000 7fcdc4682740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcdbc1adb80 con 0x55c42f989f40 2024-04-23T10:32:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.026+0000 7fcdc4682740 10 client.0 _ll_register_callbacks cb 0x7fcda4009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:32:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.026+0000 7fcdbaffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c42f989620 0x55c42f998080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.026+0000 7fcdbaffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c42f989620 0x55c42f998080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39586/0 (socket says 172.21.15.97:39586) 2024-04-23T10:32:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.026+0000 7fcdbaffd640 1 -- 172.21.15.97:0/596161879 learned_addr learned my addr 172.21.15.97:0/596161879 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:32:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.026+0000 7fcdbb7fe640 1 --2- 172.21.15.97:0/596161879 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c42f994800 0x55c42f998b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.026+0000 7fcdb3fff640 1 --2- 172.21.15.97:0/596161879 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c42f989f40 0x55c42f9985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.026+0000 7fcdbaffd640 1 -- 172.21.15.97:0/596161879 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c42f989f40 msgr2=0x55c42f9985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:32:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.026+0000 7fcdbaffd640 1 --2- 172.21.15.97:0/596161879 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c42f989f40 0x55c42f9985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.026+0000 7fcdbaffd640 1 -- 172.21.15.97:0/596161879 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c42f994800 msgr2=0x55c42f998b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:32:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.026+0000 7fcdbaffd640 1 --2- 172.21.15.97:0/596161879 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c42f994800 0x55c42f998b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.026+0000 7fcdbaffd640 1 -- 172.21.15.97:0/596161879 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c42fa09d40 con 0x55c42f989620 2024-04-23T10:32:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.026+0000 7fcdbb7fe640 1 --2- 172.21.15.97:0/596161879 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c42f994800 0x55c42f998b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:32:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.026+0000 7fcdbaffd640 1 --2- 172.21.15.97:0/596161879 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c42f989620 0x55c42f998080 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fcdac006010 tx=0x7fcdac003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=840b8aa5134ad5af server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:32:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.026+0000 7fcd9affd640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:32:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.027+0000 7fcd9affd640 1 -- 172.21.15.97:0/596161879 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdac049040 con 0x55c42f989620 2024-04-23T10:32:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.027+0000 7fcd9affd640 1 -- 172.21.15.97:0/596161879 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7fcdac0b9500 con 0x55c42f989620 2024-04-23T10:32:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.027+0000 7fcdc4682740 1 -- 172.21.15.97:0/596161879 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcda401c610 con 0x55c42f989620 2024-04-23T10:32:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.027+0000 7fcd9affd640 1 -- 172.21.15.97:0/596161879 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcdac0c1240 con 0x55c42f989620 2024-04-23T10:32:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.027+0000 7fcdc4682740 10 client.16515 fetch_fsmap learned FSMap version 26 2024-04-23T10:32:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.027+0000 7fcdc4682740 1 -- 172.21.15.97:0/596161879 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c42fa0b010 con 0x55c42f989620 2024-04-23T10:32:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.027+0000 7fcd9affd640 1 -- 172.21.15.97:0/596161879 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcda401c610 con 0x55c42f989620 2024-04-23T10:32:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.027+0000 7fcd9affd640 1 -- 172.21.15.97:0/596161879 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fcdac0ee8c0 con 0x55c42f989620 2024-04-23T10:32:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.027+0000 7fcdc4682740 10 client.16515 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:32:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.027+0000 7fcdc4682740 10 client.16515 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:32:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.027+0000 7fcdc4682740 20 client.16515 populate_metadata read hostname 'smithi097' 2024-04-23T10:32:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.027+0000 7fcdc4682740 1 --2- 172.21.15.97:0/596161879 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55c42fa0b650 0x55c42fa0da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:32:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.027+0000 7fcdc4682740 4 client.16515 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:32:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.027+0000 7fcdc4682740 1 -- 172.21.15.97:0/596161879 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c42f8f6200 con 0x55c42fa0b650 2024-04-23T10:32:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.028+0000 7fcdb3fff640 1 --2- 172.21.15.97:0/596161879 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55c42fa0b650 0x55c42fa0da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:32:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.028+0000 7fcd9affd640 10 client.16515 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:32:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.028+0000 7fcdb3fff640 1 --2- 172.21.15.97:0/596161879 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55c42fa0b650 0x55c42fa0da70 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebf07f09448a17c0 server_cookie=44a66b9c812d7dc1 in_seq=0 out_seq=0 2024-04-23T10:32:48.040 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.041+0000 7fcd9affd640 1 -- 172.21.15.97:0/596161879 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c42f8f6200 con 0x55c42fa0b650 2024-04-23T10:32:48.040 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.041+0000 7fcd9affd640 10 client.16515 handle_command_reply: tid=0 2024-04-23T10:32:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.041+0000 7fcdc4682740 1 client.16515 shutdown 2024-04-23T10:32:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.041+0000 7fcdc4682740 1 -- 172.21.15.97:0/596161879 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c42f989620 msgr2=0x55c42f998080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.041+0000 7fcdc4682740 1 --2- 172.21.15.97:0/596161879 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c42f989620 0x55c42f998080 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fcdac006010 tx=0x7fcdac003f10 comp rx=0 tx=0).stop 2024-04-23T10:32:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.041+0000 7fcdc4682740 1 -- 172.21.15.97:0/596161879 shutdown_connections 2024-04-23T10:32:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.041+0000 7fcdc4682740 1 --2- 172.21.15.97:0/596161879 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55c42fa0b650 0x55c42fa0da70 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.041+0000 7fcdc4682740 1 --2- 172.21.15.97:0/596161879 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c42f994800 0x55c42f998b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.041+0000 7fcdc4682740 1 --2- 172.21.15.97:0/596161879 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c42f989f40 0x55c42f9985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.041+0000 7fcd9affd640 0 client.16515 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:32:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.041+0000 7fcdc4682740 1 --2- 172.21.15.97:0/596161879 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c42f989620 0x55c42f998080 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.041+0000 7fcdc4682740 1 -- 172.21.15.97:0/596161879 >> 172.21.15.97:0/596161879 conn(0x55c42f985b30 msgr2=0x55c42f984cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:32:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.041+0000 7fcdc4682740 1 -- 172.21.15.97:0/596161879 shutdown_connections 2024-04-23T10:32:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.041+0000 7fcdc4682740 1 -- 172.21.15.97:0/596161879 wait complete. 2024-04-23T10:32:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.042+0000 7fcdc4682740 20 client.16515 trim_cache size 0 max 16384 2024-04-23T10:32:48.042 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:32:48.042 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:32:48.042 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:32:48.042 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:32:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.042+0000 7fcdc1968640 1 -- 172.21.15.97:0/2044729991 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fcda80a3dc0 msgr2=0x7fcda80a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.042+0000 7fcdc1968640 1 --2- 172.21.15.97:0/2044729991 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fcda80a3dc0 0x7fcda80a6280 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fcdb4007310 tx=0x7fcdb4007d50 comp rx=0 tx=0).stop 2024-04-23T10:32:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.042+0000 7fcdc1968640 1 -- 172.21.15.97:0/2044729991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdbc1021d0 msgr2=0x7fcdbc1a05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:32:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.042+0000 7fcdc1968640 1 --2- 172.21.15.97:0/2044729991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdbc1021d0 0x7fcdbc1a05f0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fcdac005940 tx=0x7fcdac004270 comp rx=0 tx=0).stop 2024-04-23T10:32:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.042+0000 7fcdc1968640 1 -- 172.21.15.97:0/2044729991 shutdown_connections 2024-04-23T10:32:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.042+0000 7fcdc1968640 1 --2- 172.21.15.97:0/2044729991 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fcda80a3dc0 0x7fcda80a6280 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.042+0000 7fcdc1968640 1 --2- 172.21.15.97:0/2044729991 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcdbc19a6b0 0x7fcdbc19ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.042+0000 7fcdc1968640 1 --2- 172.21.15.97:0/2044729991 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcdbc1072a0 0x7fcdbc1a0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.042+0000 7fcdc1968640 1 --2- 172.21.15.97:0/2044729991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdbc1021d0 0x7fcdbc1a05f0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:32:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.042+0000 7fcdc1968640 1 -- 172.21.15.97:0/2044729991 >> 172.21.15.97:0/2044729991 conn(0x7fcdbc0793e0 msgr2=0x7fcdbc0ffe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:32:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.042+0000 7fcdc1968640 1 -- 172.21.15.97:0/2044729991 shutdown_connections 2024-04-23T10:32:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:32:48.042+0000 7fcdc1968640 1 -- 172.21.15.97:0/2044729991 wait complete. 2024-04-23T10:32:49.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:32:49 smithi097 ceph-mon[29197]: pgmap v201: 97 pgs: 97 active+clean; 3.7 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 55 KiB/s wr, 22 op/s 2024-04-23T10:32:49.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:32:49 smithi183 ceph-mon[32317]: pgmap v201: 97 pgs: 97 active+clean; 3.7 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 55 KiB/s wr, 22 op/s 2024-04-23T10:32:49.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:32:49 smithi196 ceph-mon[32919]: pgmap v201: 97 pgs: 97 active+clean; 3.7 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 55 KiB/s wr, 22 op/s 2024-04-23T10:32:51.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:32:51 smithi183 ceph-mon[32317]: pgmap v202: 97 pgs: 97 active+clean; 3.7 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-04-23T10:32:51.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:32:51 smithi196 ceph-mon[32919]: pgmap v202: 97 pgs: 97 active+clean; 3.7 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-04-23T10:32:51.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:32:51 smithi097 ceph-mon[29197]: pgmap v202: 97 pgs: 97 active+clean; 3.7 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-04-23T10:32:53.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:32:53 smithi183 ceph-mon[32317]: pgmap v203: 97 pgs: 97 active+clean; 3.7 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-04-23T10:32:53.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:32:53 smithi196 ceph-mon[32919]: pgmap v203: 97 pgs: 97 active+clean; 3.7 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-04-23T10:32:53.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:32:53 smithi097 ceph-mon[29197]: pgmap v203: 97 pgs: 97 active+clean; 3.7 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-04-23T10:32:54.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:32:54 smithi183 ceph-mon[32317]: pgmap v204: 97 pgs: 97 active+clean; 3.9 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 56 KiB/s wr, 22 op/s 2024-04-23T10:32:54.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:32:54 smithi196 ceph-mon[32919]: pgmap v204: 97 pgs: 97 active+clean; 3.9 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 56 KiB/s wr, 22 op/s 2024-04-23T10:32:54.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:32:54 smithi097 ceph-mon[29197]: pgmap v204: 97 pgs: 97 active+clean; 3.9 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 56 KiB/s wr, 22 op/s 2024-04-23T10:32:56.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:32:56 smithi196 ceph-mon[32919]: pgmap v205: 97 pgs: 97 active+clean; 3.9 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-23T10:32:56.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:32:56 smithi097 ceph-mon[29197]: pgmap v205: 97 pgs: 97 active+clean; 3.9 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-23T10:32:56.649 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:32:56 smithi183 ceph-mon[32317]: pgmap v205: 97 pgs: 97 active+clean; 3.9 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-23T10:32:58.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:32:58 smithi196 ceph-mon[32919]: pgmap v206: 97 pgs: 97 active+clean; 4.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 56 KiB/s wr, 21 op/s 2024-04-23T10:32:58.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:32:58 smithi097 ceph-mon[29197]: pgmap v206: 97 pgs: 97 active+clean; 4.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 56 KiB/s wr, 21 op/s 2024-04-23T10:32:58.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:32:58 smithi183 ceph-mon[32317]: pgmap v206: 97 pgs: 97 active+clean; 4.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 56 KiB/s wr, 21 op/s 2024-04-23T10:33:00.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:33:00 smithi196 ceph-mon[32919]: pgmap v207: 97 pgs: 97 active+clean; 4.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-23T10:33:00.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:33:00 smithi097 ceph-mon[29197]: pgmap v207: 97 pgs: 97 active+clean; 4.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-23T10:33:00.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:33:00 smithi183 ceph-mon[32317]: pgmap v207: 97 pgs: 97 active+clean; 4.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-23T10:33:01.138 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.137+0000 7f4a7a679640 1 -- 172.21.15.97:0/2593564106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a74107290 msgr2=0x7f4a7410ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:01.138 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.137+0000 7f4a7a679640 1 --2- 172.21.15.97:0/2593564106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a74107290 0x7f4a7410ee60 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c009ca0 tx=0x7f4a6c02a8c0 comp rx=0 tx=0).stop 2024-04-23T10:33:01.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.137+0000 7f4a7a679640 1 -- 172.21.15.97:0/2593564106 shutdown_connections 2024-04-23T10:33:01.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.137+0000 7f4a7a679640 1 --2- 172.21.15.97:0/2593564106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a74107290 0x7f4a7410ee60 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:01.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.137+0000 7f4a7a679640 1 --2- 172.21.15.97:0/2593564106 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a74102b70 0x7f4a74106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:01.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.137+0000 7f4a7a679640 1 --2- 172.21.15.97:0/2593564106 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a741021c0 0x7f4a741025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:01.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.137+0000 7f4a7a679640 1 -- 172.21.15.97:0/2593564106 >> 172.21.15.97:0/2593564106 conn(0x7f4a740fda20 msgr2=0x7f4a740ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:33:01.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.137+0000 7f4a7a679640 1 -- 172.21.15.97:0/2593564106 shutdown_connections 2024-04-23T10:33:01.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.137+0000 7f4a7a679640 1 -- 172.21.15.97:0/2593564106 wait complete. 2024-04-23T10:33:01.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.138+0000 7f4a7a679640 1 Processor -- start 2024-04-23T10:33:01.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.138+0000 7f4a7a679640 1 -- start start 2024-04-23T10:33:01.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.138+0000 7f4a7a679640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a741021c0 0x7f4a741a0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:01.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.138+0000 7f4a7a679640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a74107290 0x7f4a741a09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:01.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.138+0000 7f4a7a679640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a7419a620 0x7f4a7419aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:01.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.138+0000 7f4a7a679640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4a74112200 con 0x7f4a741021c0 2024-04-23T10:33:01.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.138+0000 7f4a7a679640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4a74112080 con 0x7f4a7419a620 2024-04-23T10:33:01.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.138+0000 7f4a7a679640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4a74112380 con 0x7f4a74107290 2024-04-23T10:33:01.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.138+0000 7f4a73fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a741021c0 0x7f4a741a0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:01.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.138+0000 7f4a73fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a741021c0 0x7f4a741a0470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39260/0 (socket says 172.21.15.97:39260) 2024-04-23T10:33:01.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.139+0000 7f4a73fff640 1 -- 172.21.15.97:0/792698068 learned_addr learned my addr 172.21.15.97:0/792698068 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:33:01.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.139+0000 7f4a78c16640 1 --2- 172.21.15.97:0/792698068 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a7419a620 0x7f4a7419aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:01.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.139+0000 7f4a737fe640 1 --2- 172.21.15.97:0/792698068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a74107290 0x7f4a741a09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:01.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.139+0000 7f4a73fff640 1 -- 172.21.15.97:0/792698068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a74107290 msgr2=0x7f4a741a09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:01.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.139+0000 7f4a73fff640 1 --2- 172.21.15.97:0/792698068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a74107290 0x7f4a741a09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:01.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.139+0000 7f4a73fff640 1 -- 172.21.15.97:0/792698068 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a7419a620 msgr2=0x7f4a7419aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:01.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.139+0000 7f4a73fff640 1 --2- 172.21.15.97:0/792698068 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a7419a620 0x7f4a7419aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:01.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.139+0000 7f4a73fff640 1 -- 172.21.15.97:0/792698068 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a7419b140 con 0x7f4a741021c0 2024-04-23T10:33:01.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.139+0000 7f4a737fe640 1 --2- 172.21.15.97:0/792698068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a74107290 0x7f4a741a09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:33:01.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.139+0000 7f4a73fff640 1 --2- 172.21.15.97:0/792698068 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a741021c0 0x7f4a741a0470 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f4a6400c920 tx=0x7f4a6400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:33:01.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.139+0000 7f4a717fa640 1 -- 172.21.15.97:0/792698068 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a64007c70 con 0x7f4a741021c0 2024-04-23T10:33:01.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.139+0000 7f4a7a679640 1 -- 172.21.15.97:0/792698068 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a7419b430 con 0x7f4a741021c0 2024-04-23T10:33:01.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.139+0000 7f4a717fa640 1 -- 172.21.15.97:0/792698068 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a64007e10 con 0x7f4a741021c0 2024-04-23T10:33:01.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.139+0000 7f4a717fa640 1 -- 172.21.15.97:0/792698068 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a64018820 con 0x7f4a741021c0 2024-04-23T10:33:01.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.139+0000 7f4a7a679640 1 -- 172.21.15.97:0/792698068 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a741a7680 con 0x7f4a741021c0 2024-04-23T10:33:01.148 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.141+0000 7f4a717fa640 1 -- 172.21.15.97:0/792698068 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4a6400f450 con 0x7f4a741021c0 2024-04-23T10:33:01.148 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.142+0000 7f4a7d38e740 1 Processor -- start 2024-04-23T10:33:01.148 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.142+0000 7f4a7d38e740 1 -- start start 2024-04-23T10:33:01.148 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.142+0000 7f4a717fa640 1 --2- 172.21.15.97:0/792698068 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f4a5c0a3dc0 0x7f4a5c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:01.148 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.142+0000 7f4a7d38e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b7f3c0620 0x557b7f3c0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:01.148 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.142+0000 7f4a7d38e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b7f3c0f40 0x557b7f3cb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:01.148 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.142+0000 7f4a7d38e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b7f3cb800 0x557b7f3cdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:01.148 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.142+0000 7f4a7d38e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x557b7f325970 con 0x557b7f3c0f40 2024-04-23T10:33:01.148 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.142+0000 7f4a7d38e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557b7f2c24a0 con 0x557b7f3c0620 2024-04-23T10:33:01.148 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.142+0000 7f4a717fa640 1 -- 172.21.15.97:0/792698068 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f4a640d08d0 con 0x7f4a741021c0 2024-04-23T10:33:01.148 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.142+0000 7f4a7d38e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557b7f321930 con 0x557b7f3cb800 2024-04-23T10:33:01.148 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.142+0000 7f4a78c16640 1 --2- 172.21.15.97:0/792698068 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f4a5c0a3dc0 0x7f4a5c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:01.148 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.142+0000 7f4a737fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b7f3c0f40 0x557b7f3cb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:01.149 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.142+0000 7f4a737fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b7f3c0f40 0x557b7f3cb2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39262/0 (socket says 172.21.15.97:39262) 2024-04-23T10:33:01.149 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.142+0000 7f4a737fe640 1 -- 172.21.15.97:0/4107948510 learned_addr learned my addr 172.21.15.97:0/4107948510 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:33:01.149 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.142+0000 7f4a78c16640 1 --2- 172.21.15.97:0/4107948510 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b7f3cb800 0x557b7f3cdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:01.149 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.142+0000 7f4a73fff640 1 --2- 172.21.15.97:0/4107948510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b7f3c0620 0x557b7f3c0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:01.149 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.143+0000 7f4a737fe640 1 -- 172.21.15.97:0/4107948510 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b7f3cb800 msgr2=0x557b7f3cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:33:01.149 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.143+0000 7f4a737fe640 1 --2- 172.21.15.97:0/4107948510 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b7f3cb800 0x557b7f3cdbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:01.149 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.143+0000 7f4a737fe640 1 -- 172.21.15.97:0/4107948510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b7f3c0620 msgr2=0x557b7f3c0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:33:01.149 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.143+0000 7f4a737fe640 1 --2- 172.21.15.97:0/4107948510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b7f3c0620 0x557b7f3c0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:01.149 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.143+0000 7f4a737fe640 1 -- 172.21.15.97:0/4107948510 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b7f038210 con 0x557b7f3c0f40 2024-04-23T10:33:01.149 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.143+0000 7f4a78c16640 1 --2- 172.21.15.97:0/792698068 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f4a5c0a3dc0 0x7f4a5c0a6280 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c006010 tx=0x7f4a6c031bd0 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:33:01.149 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.143+0000 7f4a737fe640 1 --2- 172.21.15.97:0/4107948510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b7f3c0f40 0x557b7f3cb2c0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f4a6800bc60 tx=0x7f4a6800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34b9ce36523c38a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:33:01.149 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.143+0000 7f4a5a7fc640 1 -- 172.21.15.97:0/4107948510 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a68015070 con 0x557b7f3c0f40 2024-04-23T10:33:01.149 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.143+0000 7f4a5a7fc640 1 -- 172.21.15.97:0/4107948510 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a680054d0 con 0x557b7f3c0f40 2024-04-23T10:33:01.149 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.143+0000 7f4a5a7fc640 1 -- 172.21.15.97:0/4107948510 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a68011440 con 0x557b7f3c0f40 2024-04-23T10:33:01.150 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.144+0000 7f4a7d38e740 1 -- 172.21.15.97:0/4107948510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b7f3c0f40 msgr2=0x557b7f3cb2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:01.150 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.144+0000 7f4a7d38e740 1 --2- 172.21.15.97:0/4107948510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b7f3c0f40 0x557b7f3cb2c0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f4a6800bc60 tx=0x7f4a6800f730 comp rx=0 tx=0).stop 2024-04-23T10:33:01.150 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.144+0000 7f4a7d38e740 1 -- 172.21.15.97:0/4107948510 shutdown_connections 2024-04-23T10:33:01.150 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.144+0000 7f4a7d38e740 1 --2- 172.21.15.97:0/4107948510 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b7f3cb800 0x557b7f3cdbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:01.150 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.144+0000 7f4a7d38e740 1 --2- 172.21.15.97:0/4107948510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b7f3c0f40 0x557b7f3cb2c0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:01.150 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.144+0000 7f4a7d38e740 1 --2- 172.21.15.97:0/4107948510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b7f3c0620 0x557b7f3c0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:01.150 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.144+0000 7f4a7d38e740 1 -- 172.21.15.97:0/4107948510 >> 172.21.15.97:0/4107948510 conn(0x557b7f3bcb30 msgr2=0x557b7f3bcf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:33:01.150 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.144+0000 7f4a7d38e740 1 -- 172.21.15.97:0/4107948510 shutdown_connections 2024-04-23T10:33:01.150 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.144+0000 7f4a7d38e740 1 -- 172.21.15.97:0/4107948510 wait complete. 2024-04-23T10:33:01.150 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.144+0000 7f4a7d38e740 1 -- start start 2024-04-23T10:33:01.150 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.144+0000 7f4a7d38e740 1 Processor -- start 2024-04-23T10:33:01.150 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.145+0000 7f4a7d38e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b7f3c0620 0x557b7f3cb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:01.150 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.145+0000 7f4a7d38e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b7f3c0f40 0x557b7f3c84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:01.150 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.145+0000 7f4a7d38e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b7f3cb800 0x557b7f3c8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:01.150 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.145+0000 7f4a7d38e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x557b7f3ce430 con 0x557b7f3c0f40 2024-04-23T10:33:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.145+0000 7f4a7d38e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557b7f2c00b0 con 0x557b7f3cb800 2024-04-23T10:33:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.145+0000 7f4a7d38e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557b7f3ce5b0 con 0x557b7f3c0620 2024-04-23T10:33:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.145+0000 7f4a7d38e740 10 client.0 _ll_register_callbacks cb 0x557b7f2add30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:33:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.145+0000 7f4a737fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b7f3c0f40 0x557b7f3c84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.145+0000 7f4a737fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b7f3c0f40 0x557b7f3c84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39274/0 (socket says 172.21.15.97:39274) 2024-04-23T10:33:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.145+0000 7f4a737fe640 1 -- 172.21.15.97:0/413048540 learned_addr learned my addr 172.21.15.97:0/413048540 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:33:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.145+0000 7f4a73fff640 1 --2- 172.21.15.97:0/413048540 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b7f3c0620 0x557b7f3cb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.145+0000 7f4a78c16640 1 --2- 172.21.15.97:0/413048540 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b7f3cb800 0x557b7f3c8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.145+0000 7f4a737fe640 1 -- 172.21.15.97:0/413048540 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b7f3c0620 msgr2=0x557b7f3cb1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:33:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.145+0000 7f4a737fe640 1 --2- 172.21.15.97:0/413048540 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b7f3c0620 0x557b7f3cb1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.145+0000 7f4a737fe640 1 -- 172.21.15.97:0/413048540 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b7f3cb800 msgr2=0x557b7f3c8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:33:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.145+0000 7f4a737fe640 1 --2- 172.21.15.97:0/413048540 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b7f3cb800 0x557b7f3c8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.145+0000 7f4a737fe640 1 -- 172.21.15.97:0/413048540 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b7f104620 con 0x557b7f3c0f40 2024-04-23T10:33:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.145+0000 7f4a737fe640 1 --2- 172.21.15.97:0/413048540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b7f3c0f40 0x557b7f3c84c0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f4a68004ae0 tx=0x7f4a680049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd9cd645d0ff360b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:33:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.145+0000 7f4a537fe640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:33:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.145+0000 7f4a537fe640 1 -- 172.21.15.97:0/413048540 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a68015070 con 0x557b7f3c0f40 2024-04-23T10:33:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.145+0000 7f4a537fe640 1 -- 172.21.15.97:0/413048540 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f4a68004ce0 con 0x557b7f3c0f40 2024-04-23T10:33:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.145+0000 7f4a7d38e740 1 -- 172.21.15.97:0/413048540 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a680054d0 con 0x557b7f3c0f40 2024-04-23T10:33:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.146+0000 7f4a537fe640 1 -- 172.21.15.97:0/413048540 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4a68004f00 con 0x557b7f3c0f40 2024-04-23T10:33:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.146+0000 7f4a7d38e740 10 client.16527 fetch_fsmap learned FSMap version 26 2024-04-23T10:33:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.146+0000 7f4a7d38e740 1 -- 172.21.15.97:0/413048540 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557b7f3da3c0 con 0x557b7f3c0f40 2024-04-23T10:33:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.146+0000 7f4a537fe640 1 -- 172.21.15.97:0/413048540 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a680054d0 con 0x557b7f3c0f40 2024-04-23T10:33:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.146+0000 7f4a537fe640 1 -- 172.21.15.97:0/413048540 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4a68019410 con 0x557b7f3c0f40 2024-04-23T10:33:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.146+0000 7f4a7d38e740 10 client.16527 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:33:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.146+0000 7f4a7d38e740 10 client.16527 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:33:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.146+0000 7f4a7d38e740 20 client.16527 populate_metadata read hostname 'smithi097' 2024-04-23T10:33:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.146+0000 7f4a7d38e740 1 --2- 172.21.15.97:0/413048540 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x557b7f3db7c0 0x557b7f3ddbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.146+0000 7f4a7d38e740 4 client.16527 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:33:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.146+0000 7f4a7d38e740 1 -- 172.21.15.97:0/413048540 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b7f333390 con 0x557b7f3db7c0 2024-04-23T10:33:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.146+0000 7f4a73fff640 1 --2- 172.21.15.97:0/413048540 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x557b7f3db7c0 0x557b7f3ddbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:01.153 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.147+0000 7f4a537fe640 10 client.16527 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:33:01.153 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.147+0000 7f4a73fff640 1 --2- 172.21.15.97:0/413048540 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x557b7f3db7c0 0x557b7f3ddbb0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1be7b967a4dc2e57 server_cookie=eb5a8742db6625eb in_seq=0 out_seq=0 2024-04-23T10:33:01.153 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.147+0000 7f4a537fe640 1 -- 172.21.15.97:0/413048540 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557b7f333390 con 0x557b7f3db7c0 2024-04-23T10:33:01.153 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.147+0000 7f4a537fe640 10 client.16527 handle_command_reply: tid=0 2024-04-23T10:33:01.153 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.147+0000 7f4a7d38e740 1 client.16527 shutdown 2024-04-23T10:33:01.153 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.147+0000 7f4a7d38e740 1 -- 172.21.15.97:0/413048540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b7f3c0f40 msgr2=0x557b7f3c84c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:01.153 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.147+0000 7f4a7d38e740 1 --2- 172.21.15.97:0/413048540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b7f3c0f40 0x557b7f3c84c0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f4a68004ae0 tx=0x7f4a680049d0 comp rx=0 tx=0).stop 2024-04-23T10:33:01.153 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.148+0000 7f4a7d38e740 1 -- 172.21.15.97:0/413048540 shutdown_connections 2024-04-23T10:33:01.153 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.148+0000 7f4a7d38e740 1 --2- 172.21.15.97:0/413048540 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x557b7f3db7c0 0x557b7f3ddbb0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:01.153 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.148+0000 7f4a7d38e740 1 --2- 172.21.15.97:0/413048540 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b7f3cb800 0x557b7f3c8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:01.153 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.148+0000 7f4a7d38e740 1 --2- 172.21.15.97:0/413048540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b7f3c0f40 0x557b7f3c84c0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:01.153 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.148+0000 7f4a537fe640 0 client.16527 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:33:01.153 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.148+0000 7f4a7d38e740 1 --2- 172.21.15.97:0/413048540 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b7f3c0620 0x557b7f3cb1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:01.154 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.148+0000 7f4a7d38e740 1 -- 172.21.15.97:0/413048540 >> 172.21.15.97:0/413048540 conn(0x557b7f3bcb30 msgr2=0x557b7f3c28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:33:01.154 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.148+0000 7f4a7d38e740 1 -- 172.21.15.97:0/413048540 shutdown_connections 2024-04-23T10:33:01.154 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.148+0000 7f4a7d38e740 1 -- 172.21.15.97:0/413048540 wait complete. 2024-04-23T10:33:01.154 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.148+0000 7f4a7d38e740 20 client.16527 trim_cache size 0 max 16384 2024-04-23T10:33:01.166 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.163+0000 7f4a7d38e740 1 Processor -- start 2024-04-23T10:33:01.166 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.163+0000 7f4a7d38e740 1 -- start start 2024-04-23T10:33:01.166 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.163+0000 7f4a7d38e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b7f3c0620 0x557b7f3d72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:01.166 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.163+0000 7f4a7d38e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b7f3c0f40 0x557b7f3cad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:01.166 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.163+0000 7f4a7d38e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b7f3cb800 0x557b7f3e32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:01.166 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.163+0000 7f4a7d38e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4a741afb50 con 0x557b7f3cb800 2024-04-23T10:33:01.166 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.163+0000 7f4a7d38e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4a741b1690 con 0x557b7f3c0f40 2024-04-23T10:33:01.166 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.163+0000 7f4a7d38e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4a741b0220 con 0x557b7f3c0620 2024-04-23T10:33:01.166 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.163+0000 7f4a78c16640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b7f3cb800 0x557b7f3e32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:01.166 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.163+0000 7f4a78c16640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b7f3cb800 0x557b7f3e32f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39280/0 (socket says 172.21.15.97:39280) 2024-04-23T10:33:01.167 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.163+0000 7f4a78c16640 1 -- 172.21.15.97:0/1224953800 learned_addr learned my addr 172.21.15.97:0/1224953800 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:33:01.167 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.163+0000 7f4a737fe640 1 --2- 172.21.15.97:0/1224953800 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b7f3c0f40 0x557b7f3cad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:01.167 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.163+0000 7f4a73fff640 1 --2- 172.21.15.97:0/1224953800 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b7f3c0620 0x557b7f3d72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:01.167 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.163+0000 7f4a78c16640 1 -- 172.21.15.97:0/1224953800 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b7f3c0620 msgr2=0x557b7f3d72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:33:01.167 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.163+0000 7f4a78c16640 1 --2- 172.21.15.97:0/1224953800 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b7f3c0620 0x557b7f3d72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:01.167 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.163+0000 7f4a78c16640 1 -- 172.21.15.97:0/1224953800 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b7f3c0f40 msgr2=0x557b7f3cad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:33:01.167 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.163+0000 7f4a78c16640 1 --2- 172.21.15.97:0/1224953800 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b7f3c0f40 0x557b7f3cad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:01.167 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.163+0000 7f4a78c16640 1 -- 172.21.15.97:0/1224953800 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b7f3c90d0 con 0x557b7f3cb800 2024-04-23T10:33:01.167 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.163+0000 7f4a737fe640 1 --2- 172.21.15.97:0/1224953800 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b7f3c0f40 0x557b7f3cad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:33:01.167 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.163+0000 7f4a78c16640 1 --2- 172.21.15.97:0/1224953800 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b7f3cb800 0x557b7f3e32f0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c009ca0 tx=0x7f4a6c049000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0d8bec30847aff5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:33:01.167 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.164+0000 7f4a59ffb640 1 -- 172.21.15.97:0/1224953800 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6c04f070 con 0x557b7f3cb800 2024-04-23T10:33:01.167 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.164+0000 7f4a59ffb640 1 -- 172.21.15.97:0/1224953800 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a6c036070 con 0x557b7f3cb800 2024-04-23T10:33:01.167 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.164+0000 7f4a59ffb640 1 -- 172.21.15.97:0/1224953800 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6c02f8c0 con 0x557b7f3cb800 2024-04-23T10:33:01.167 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.164+0000 7f4a7d38e740 1 -- 172.21.15.97:0/1224953800 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b7f3cb800 msgr2=0x557b7f3e32f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:01.168 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.164+0000 7f4a7d38e740 1 --2- 172.21.15.97:0/1224953800 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b7f3cb800 0x557b7f3e32f0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c009ca0 tx=0x7f4a6c049000 comp rx=0 tx=0).stop 2024-04-23T10:33:01.168 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.164+0000 7f4a7d38e740 1 -- 172.21.15.97:0/1224953800 shutdown_connections 2024-04-23T10:33:01.168 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.164+0000 7f4a7d38e740 1 --2- 172.21.15.97:0/1224953800 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b7f3cb800 0x557b7f3e32f0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:01.168 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.164+0000 7f4a7d38e740 1 --2- 172.21.15.97:0/1224953800 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b7f3c0f40 0x557b7f3cad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:01.168 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.164+0000 7f4a7d38e740 1 --2- 172.21.15.97:0/1224953800 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b7f3c0620 0x557b7f3d72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:01.168 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.164+0000 7f4a7d38e740 1 -- 172.21.15.97:0/1224953800 >> 172.21.15.97:0/1224953800 conn(0x557b7f3bcb30 msgr2=0x557b7f3c6d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:33:01.168 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.164+0000 7f4a7d38e740 1 -- 172.21.15.97:0/1224953800 shutdown_connections 2024-04-23T10:33:01.168 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.164+0000 7f4a7d38e740 1 -- 172.21.15.97:0/1224953800 wait complete. 2024-04-23T10:33:01.168 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.165+0000 7f4a7d38e740 1 -- start start 2024-04-23T10:33:01.168 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.165+0000 7f4a7d38e740 1 Processor -- start 2024-04-23T10:33:01.168 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.165+0000 7f4a7d38e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b7f3c0620 0x557b7f3d3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:01.168 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.165+0000 7f4a7d38e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b7f3c0f40 0x557b7f3cedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:01.168 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.165+0000 7f4a7d38e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b7f3cb800 0x557b7f3cf300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:01.168 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.165+0000 7f4a7d38e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4a741b0fc0 con 0x557b7f3cb800 2024-04-23T10:33:01.168 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.165+0000 7f4a7d38e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4a741b08f0 con 0x557b7f3c0f40 2024-04-23T10:33:01.169 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.165+0000 7f4a7d38e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4a741adb30 con 0x557b7f3c0620 2024-04-23T10:33:01.169 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.165+0000 7f4a7d38e740 10 client.0 _ll_register_callbacks cb 0x7f4a68005680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:33:01.169 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.165+0000 7f4a78c16640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b7f3cb800 0x557b7f3cf300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:01.169 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.165+0000 7f4a78c16640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b7f3cb800 0x557b7f3cf300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39292/0 (socket says 172.21.15.97:39292) 2024-04-23T10:33:01.169 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.165+0000 7f4a78c16640 1 -- 172.21.15.97:0/2506064191 learned_addr learned my addr 172.21.15.97:0/2506064191 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:33:01.169 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.165+0000 7f4a737fe640 1 --2- 172.21.15.97:0/2506064191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b7f3c0f40 0x557b7f3cedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:01.169 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.165+0000 7f4a73fff640 1 --2- 172.21.15.97:0/2506064191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b7f3c0620 0x557b7f3d3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:01.169 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.165+0000 7f4a78c16640 1 -- 172.21.15.97:0/2506064191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b7f3c0620 msgr2=0x557b7f3d3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:33:01.169 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.165+0000 7f4a78c16640 1 --2- 172.21.15.97:0/2506064191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b7f3c0620 0x557b7f3d3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:01.169 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.165+0000 7f4a78c16640 1 -- 172.21.15.97:0/2506064191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b7f3c0f40 msgr2=0x557b7f3cedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:33:01.169 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.165+0000 7f4a78c16640 1 --2- 172.21.15.97:0/2506064191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b7f3c0f40 0x557b7f3cedc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:01.169 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.165+0000 7f4a78c16640 1 -- 172.21.15.97:0/2506064191 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b7f3cdbf0 con 0x557b7f3cb800 2024-04-23T10:33:01.169 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.165+0000 7f4a73fff640 1 --2- 172.21.15.97:0/2506064191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b7f3c0620 0x557b7f3d3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:33:01.169 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.166+0000 7f4a78c16640 1 --2- 172.21.15.97:0/2506064191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b7f3cb800 0x557b7f3cf300 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c02ade0 tx=0x7f4a6c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fd35a2fba40bf4d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:33:01.170 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.166+0000 7f4a537fe640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:33:01.170 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.166+0000 7f4a537fe640 1 -- 172.21.15.97:0/2506064191 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6c058070 con 0x557b7f3cb800 2024-04-23T10:33:01.170 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.166+0000 7f4a7d38e740 1 -- 172.21.15.97:0/2506064191 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a6c036070 con 0x557b7f3cb800 2024-04-23T10:33:01.170 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.166+0000 7f4a537fe640 1 -- 172.21.15.97:0/2506064191 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8440+0+0 (secure 0 0 0) 0x7f4a6c004440 con 0x557b7f3cb800 2024-04-23T10:33:01.170 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.166+0000 7f4a537fe640 1 -- 172.21.15.97:0/2506064191 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4a6c04f050 con 0x557b7f3cb800 2024-04-23T10:33:01.170 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.166+0000 7f4a7d38e740 10 client.16533 fetch_fsmap learned FSMap version 26 2024-04-23T10:33:01.170 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.166+0000 7f4a7d38e740 1 -- 172.21.15.97:0/2506064191 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557b7f4417d0 con 0x557b7f3cb800 2024-04-23T10:33:01.170 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.166+0000 7f4a537fe640 1 -- 172.21.15.97:0/2506064191 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a6c036070 con 0x557b7f3cb800 2024-04-23T10:33:01.170 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.166+0000 7f4a537fe640 1 -- 172.21.15.97:0/2506064191 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4a6c030820 con 0x557b7f3cb800 2024-04-23T10:33:01.170 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.166+0000 7f4a7d38e740 10 client.16533 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:33:01.170 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.166+0000 7f4a7d38e740 10 client.16533 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:33:01.170 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.166+0000 7f4a7d38e740 20 client.16533 populate_metadata read hostname 'smithi097' 2024-04-23T10:33:01.170 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.166+0000 7f4a7d38e740 1 --2- 172.21.15.97:0/2506064191 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x557b7f441ea0 0x557b7f444290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:01.171 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.166+0000 7f4a7d38e740 4 client.16533 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:33:01.171 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.166+0000 7f4a7d38e740 1 -- 172.21.15.97:0/2506064191 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557b7f32d200 con 0x557b7f441ea0 2024-04-23T10:33:01.171 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.167+0000 7f4a73fff640 1 --2- 172.21.15.97:0/2506064191 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x557b7f441ea0 0x557b7f444290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:01.171 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.167+0000 7f4a73fff640 1 --2- 172.21.15.97:0/2506064191 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x557b7f441ea0 0x557b7f444290 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66c7b59967a9bc8b server_cookie=694a81270fa1fe85 in_seq=0 out_seq=0 2024-04-23T10:33:01.171 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.167+0000 7f4a537fe640 10 client.16533 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:33:01.180 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.181+0000 7f4a537fe640 1 -- 172.21.15.97:0/2506064191 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557b7f32d200 con 0x557b7f441ea0 2024-04-23T10:33:01.180 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.181+0000 7f4a537fe640 10 client.16533 handle_command_reply: tid=0 2024-04-23T10:33:01.180 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.181+0000 7f4a7d38e740 1 client.16533 shutdown 2024-04-23T10:33:01.181 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.181+0000 7f4a7d38e740 1 -- 172.21.15.97:0/2506064191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b7f3cb800 msgr2=0x557b7f3cf300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:01.181 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.181+0000 7f4a7d38e740 1 --2- 172.21.15.97:0/2506064191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b7f3cb800 0x557b7f3cf300 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c02ade0 tx=0x7f4a6c0095c0 comp rx=0 tx=0).stop 2024-04-23T10:33:01.181 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.181+0000 7f4a7d38e740 1 -- 172.21.15.97:0/2506064191 shutdown_connections 2024-04-23T10:33:01.181 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.181+0000 7f4a7d38e740 1 --2- 172.21.15.97:0/2506064191 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x557b7f441ea0 0x557b7f444290 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:01.181 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.181+0000 7f4a7d38e740 1 --2- 172.21.15.97:0/2506064191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b7f3cb800 0x557b7f3cf300 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:01.181 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.181+0000 7f4a7d38e740 1 --2- 172.21.15.97:0/2506064191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b7f3c0f40 0x557b7f3cedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:01.181 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.181+0000 7f4a7d38e740 1 --2- 172.21.15.97:0/2506064191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b7f3c0620 0x557b7f3d3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:01.181 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.181+0000 7f4a537fe640 0 client.16533 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:33:01.181 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.181+0000 7f4a7d38e740 1 -- 172.21.15.97:0/2506064191 >> 172.21.15.97:0/2506064191 conn(0x557b7f3bcb30 msgr2=0x557b7f3bb420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:33:01.181 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.181+0000 7f4a7d38e740 1 -- 172.21.15.97:0/2506064191 shutdown_connections 2024-04-23T10:33:01.181 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.181+0000 7f4a7d38e740 1 -- 172.21.15.97:0/2506064191 wait complete. 2024-04-23T10:33:01.181 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.181+0000 7f4a7d38e740 20 client.16533 trim_cache size 0 max 16384 2024-04-23T10:33:01.181 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:33:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:33:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:33:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:33:01.183 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.182+0000 7f4a7a679640 1 -- 172.21.15.97:0/792698068 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f4a5c0a3dc0 msgr2=0x7f4a5c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:01.183 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.182+0000 7f4a7a679640 1 --2- 172.21.15.97:0/792698068 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f4a5c0a3dc0 0x7f4a5c0a6280 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c006010 tx=0x7f4a6c031bd0 comp rx=0 tx=0).stop 2024-04-23T10:33:01.183 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.182+0000 7f4a7a679640 1 -- 172.21.15.97:0/792698068 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a741021c0 msgr2=0x7f4a741a0470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:01.183 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.182+0000 7f4a7a679640 1 --2- 172.21.15.97:0/792698068 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a741021c0 0x7f4a741a0470 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f4a6400c920 tx=0x7f4a6400cde0 comp rx=0 tx=0).stop 2024-04-23T10:33:01.183 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.182+0000 7f4a7a679640 1 -- 172.21.15.97:0/792698068 shutdown_connections 2024-04-23T10:33:01.183 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.182+0000 7f4a7a679640 1 --2- 172.21.15.97:0/792698068 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f4a5c0a3dc0 0x7f4a5c0a6280 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:01.183 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.182+0000 7f4a7a679640 1 --2- 172.21.15.97:0/792698068 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a7419a620 0x7f4a7419aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:01.183 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.182+0000 7f4a7a679640 1 --2- 172.21.15.97:0/792698068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4a74107290 0x7f4a741a09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:01.183 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.182+0000 7f4a7a679640 1 --2- 172.21.15.97:0/792698068 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a741021c0 0x7f4a741a0470 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:01.183 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.182+0000 7f4a7a679640 1 -- 172.21.15.97:0/792698068 >> 172.21.15.97:0/792698068 conn(0x7f4a740fda20 msgr2=0x7f4a7410d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:33:01.183 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.182+0000 7f4a7a679640 1 -- 172.21.15.97:0/792698068 shutdown_connections 2024-04-23T10:33:01.183 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:01.182+0000 7f4a7a679640 1 -- 172.21.15.97:0/792698068 wait complete. 2024-04-23T10:33:02.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:33:02 smithi196 ceph-mon[32919]: pgmap v208: 97 pgs: 97 active+clean; 4.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-23T10:33:02.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:33:02 smithi097 ceph-mon[29197]: pgmap v208: 97 pgs: 97 active+clean; 4.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-23T10:33:02.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:33:02 smithi183 ceph-mon[32317]: pgmap v208: 97 pgs: 97 active+clean; 4.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-23T10:33:04.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:33:04 smithi196 ceph-mon[32919]: pgmap v209: 97 pgs: 97 active+clean; 4.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 108 KiB/s wr, 41 op/s 2024-04-23T10:33:04.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:33:04 smithi097 ceph-mon[29197]: pgmap v209: 97 pgs: 97 active+clean; 4.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 108 KiB/s wr, 41 op/s 2024-04-23T10:33:04.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:33:04 smithi183 ceph-mon[32317]: pgmap v209: 97 pgs: 97 active+clean; 4.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 108 KiB/s wr, 41 op/s 2024-04-23T10:33:06.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:33:06 smithi196 ceph-mon[32919]: pgmap v210: 97 pgs: 97 active+clean; 4.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 85 KiB/s wr, 32 op/s 2024-04-23T10:33:06.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:33:06 smithi097 ceph-mon[29197]: pgmap v210: 97 pgs: 97 active+clean; 4.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 85 KiB/s wr, 32 op/s 2024-04-23T10:33:06.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:33:06 smithi183 ceph-mon[32317]: pgmap v210: 97 pgs: 97 active+clean; 4.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 85 KiB/s wr, 32 op/s 2024-04-23T10:33:07.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:33:07 smithi196 ceph-mon[32919]: osdmap e90: 12 total, 12 up, 12 in 2024-04-23T10:33:07.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:33:07 smithi097 ceph-mon[29197]: osdmap e90: 12 total, 12 up, 12 in 2024-04-23T10:33:07.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:33:07 smithi183 ceph-mon[32317]: osdmap e90: 12 total, 12 up, 12 in 2024-04-23T10:33:08.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:33:08 smithi196 ceph-mon[32919]: pgmap v212: 97 pgs: 97 active+clean; 4.5 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 103 KiB/s wr, 39 op/s 2024-04-23T10:33:08.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:33:08 smithi097 ceph-mon[29197]: pgmap v212: 97 pgs: 97 active+clean; 4.5 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 103 KiB/s wr, 39 op/s 2024-04-23T10:33:08.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:33:08 smithi183 ceph-mon[32317]: pgmap v212: 97 pgs: 97 active+clean; 4.5 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 103 KiB/s wr, 39 op/s 2024-04-23T10:33:10.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:33:10 smithi196 ceph-mon[32919]: pgmap v213: 97 pgs: 97 active+clean; 4.5 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 103 KiB/s wr, 39 op/s 2024-04-23T10:33:10.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:33:10 smithi097 ceph-mon[29197]: pgmap v213: 97 pgs: 97 active+clean; 4.5 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 103 KiB/s wr, 39 op/s 2024-04-23T10:33:10.649 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:33:10 smithi183 ceph-mon[32317]: pgmap v213: 97 pgs: 97 active+clean; 4.5 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 103 KiB/s wr, 39 op/s 2024-04-23T10:33:12.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:33:12 smithi196 ceph-mon[32919]: osdmap e91: 12 total, 12 up, 12 in 2024-04-23T10:33:12.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:33:12 smithi097 ceph-mon[29197]: osdmap e91: 12 total, 12 up, 12 in 2024-04-23T10:33:12.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:33:12 smithi183 ceph-mon[32317]: osdmap e91: 12 total, 12 up, 12 in 2024-04-23T10:33:13.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:33:13 smithi196 ceph-mon[32919]: pgmap v215: 97 pgs: 97 active+clean; 4.5 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 26 KiB/s wr, 9 op/s 2024-04-23T10:33:13.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:33:13 smithi097 ceph-mon[29197]: pgmap v215: 97 pgs: 97 active+clean; 4.5 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 26 KiB/s wr, 9 op/s 2024-04-23T10:33:13.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:33:13 smithi183 ceph-mon[32317]: pgmap v215: 97 pgs: 97 active+clean; 4.5 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 26 KiB/s wr, 9 op/s 2024-04-23T10:33:14.280 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.278+0000 7f4ffe3fe640 1 -- 172.21.15.97:0/2413941245 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ff8102b70 msgr2=0x7f4ff8106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:14.281 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.278+0000 7f4ffe3fe640 1 --2- 172.21.15.97:0/2413941245 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ff8102b70 0x7f4ff8106bb0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f4fec009900 tx=0x7f4fec02a410 comp rx=0 tx=0).stop 2024-04-23T10:33:14.281 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.278+0000 7f4ffe3fe640 1 -- 172.21.15.97:0/2413941245 shutdown_connections 2024-04-23T10:33:14.281 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.278+0000 7f4ffe3fe640 1 --2- 172.21.15.97:0/2413941245 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ff8107290 0x7f4ff810ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:14.281 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.278+0000 7f4ffe3fe640 1 --2- 172.21.15.97:0/2413941245 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ff8102b70 0x7f4ff8106bb0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:14.281 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.278+0000 7f4ffe3fe640 1 --2- 172.21.15.97:0/2413941245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ff81021c0 0x7f4ff81025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:14.281 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.278+0000 7f4ffe3fe640 1 -- 172.21.15.97:0/2413941245 >> 172.21.15.97:0/2413941245 conn(0x7f4ff80fda20 msgr2=0x7f4ff80ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:33:14.281 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.278+0000 7f4ffe3fe640 1 -- 172.21.15.97:0/2413941245 shutdown_connections 2024-04-23T10:33:14.281 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.278+0000 7f4ffe3fe640 1 -- 172.21.15.97:0/2413941245 wait complete. 2024-04-23T10:33:14.282 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.279+0000 7f4ffe3fe640 1 Processor -- start 2024-04-23T10:33:14.282 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.279+0000 7f4ffe3fe640 1 -- start start 2024-04-23T10:33:14.282 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.279+0000 7f4ffe3fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ff81021c0 0x7f4ff81986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:14.282 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.279+0000 7f4ffe3fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ff8107290 0x7f4ff8198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:14.282 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.279+0000 7f4ffe3fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ff819cfa0 0x7f4ff819d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:14.282 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.279+0000 7f4ffe3fe640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff8112220 con 0x7f4ff819cfa0 2024-04-23T10:33:14.282 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.279+0000 7f4ffe3fe640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff81120a0 con 0x7f4ff8107290 2024-04-23T10:33:14.282 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.279+0000 7f4ffe3fe640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff81123a0 con 0x7f4ff81021c0 2024-04-23T10:33:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.279+0000 7f4ffc99b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ff819cfa0 0x7f4ff819d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.279+0000 7f4ffc99b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ff819cfa0 0x7f4ff819d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57610/0 (socket says 172.21.15.97:57610) 2024-04-23T10:33:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.279+0000 7f4ffc99b640 1 -- 172.21.15.97:0/2476212140 learned_addr learned my addr 172.21.15.97:0/2476212140 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:33:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.279+0000 7f4ff7fff640 1 --2- 172.21.15.97:0/2476212140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ff81021c0 0x7f4ff81986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.280+0000 7f4ff77fe640 1 --2- 172.21.15.97:0/2476212140 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ff8107290 0x7f4ff8198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.280+0000 7f4ffc99b640 1 -- 172.21.15.97:0/2476212140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ff81021c0 msgr2=0x7f4ff81986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.280+0000 7f4ffc99b640 1 --2- 172.21.15.97:0/2476212140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ff81021c0 0x7f4ff81986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.280+0000 7f4ffc99b640 1 -- 172.21.15.97:0/2476212140 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ff8107290 msgr2=0x7f4ff8198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.280+0000 7f4ffc99b640 1 --2- 172.21.15.97:0/2476212140 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ff8107290 0x7f4ff8198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.280+0000 7f4ffc99b640 1 -- 172.21.15.97:0/2476212140 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ff819da90 con 0x7f4ff819cfa0 2024-04-23T10:33:14.283 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.280+0000 7f4ffc99b640 1 --2- 172.21.15.97:0/2476212140 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ff819cfa0 0x7f4ff819d450 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f4ff0009fd0 tx=0x7f4ff000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:33:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.280+0000 7f4ff57fa640 1 -- 172.21.15.97:0/2476212140 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff0017070 con 0x7f4ff819cfa0 2024-04-23T10:33:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.280+0000 7f4ff57fa640 1 -- 172.21.15.97:0/2476212140 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ff0003e20 con 0x7f4ff819cfa0 2024-04-23T10:33:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.280+0000 7f4ffe3fe640 1 -- 172.21.15.97:0/2476212140 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ff819dd80 con 0x7f4ff819cfa0 2024-04-23T10:33:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.280+0000 7f4ff57fa640 1 -- 172.21.15.97:0/2476212140 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff0010460 con 0x7f4ff819cfa0 2024-04-23T10:33:14.284 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.280+0000 7f4ffe3fe640 1 -- 172.21.15.97:0/2476212140 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ff81a54f0 con 0x7f4ff819cfa0 2024-04-23T10:33:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.282+0000 7f4ff57fa640 1 -- 172.21.15.97:0/2476212140 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4ff0003970 con 0x7f4ff819cfa0 2024-04-23T10:33:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.283+0000 7f50011ed740 1 Processor -- start 2024-04-23T10:33:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.283+0000 7f50011ed740 1 -- start start 2024-04-23T10:33:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.283+0000 7f50011ed740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560a7ecae620 0x560a7ecaea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:14.292 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.283+0000 7f50011ed740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a7ecaef40 0x560a7ecb92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.283+0000 7f50011ed740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a7ecb9800 0x560a7ecbbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.283+0000 7f50011ed740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560a7ec13970 con 0x560a7ecae620 2024-04-23T10:33:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.283+0000 7f4ff57fa640 1 --2- 172.21.15.97:0/2476212140 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f4fe00a3dc0 0x7f4fe00a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.283+0000 7f50011ed740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560a7ebb04a0 con 0x560a7ecaef40 2024-04-23T10:33:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.283+0000 7f50011ed740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560a7ec0f930 con 0x560a7ecb9800 2024-04-23T10:33:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.283+0000 7f4ff77fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560a7ecae620 0x560a7ecaea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.284+0000 7f4ff77fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560a7ecae620 0x560a7ecaea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57624/0 (socket says 172.21.15.97:57624) 2024-04-23T10:33:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.284+0000 7f4ff77fe640 1 -- 172.21.15.97:0/272054511 learned_addr learned my addr 172.21.15.97:0/272054511 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:33:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.284+0000 7f4ffc99b640 1 --2- 172.21.15.97:0/2476212140 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f4fe00a3dc0 0x7f4fe00a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.284+0000 7f4ff7fff640 1 --2- 172.21.15.97:0/272054511 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a7ecaef40 0x560a7ecb92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.284+0000 7f4ff57fa640 1 -- 172.21.15.97:0/2476212140 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7f4ff00d0580 con 0x7f4ff819cfa0 2024-04-23T10:33:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.284+0000 7f4ff77fe640 1 --2- 172.21.15.97:0/272054511 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a7ecb9800 0x560a7ecbbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.284+0000 7f4ff77fe640 1 -- 172.21.15.97:0/272054511 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a7ecb9800 msgr2=0x560a7ecbbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:33:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.284+0000 7f4ff77fe640 1 --2- 172.21.15.97:0/272054511 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a7ecb9800 0x560a7ecbbbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.284+0000 7f4ff77fe640 1 -- 172.21.15.97:0/272054511 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a7ecaef40 msgr2=0x560a7ecb92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:33:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.284+0000 7f4ff77fe640 1 --2- 172.21.15.97:0/272054511 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a7ecaef40 0x560a7ecb92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.284+0000 7f4ff77fe640 1 -- 172.21.15.97:0/272054511 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a7e926210 con 0x560a7ecae620 2024-04-23T10:33:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.284+0000 7f4ff77fe640 1 --2- 172.21.15.97:0/272054511 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560a7ecae620 0x560a7ecaea00 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f4fec005c40 tx=0x7f4fec005bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73d5390b97d51ee2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:33:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.284+0000 7f4ffc99b640 1 --2- 172.21.15.97:0/2476212140 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f4fe00a3dc0 0x7f4fe00a6280 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4ff009c470 tx=0x7f4ff006df20 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:33:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.284+0000 7f4fde7fc640 1 -- 172.21.15.97:0/272054511 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fec043020 con 0x560a7ecae620 2024-04-23T10:33:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.284+0000 7f4fde7fc640 1 -- 172.21.15.97:0/272054511 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fec0046c0 con 0x560a7ecae620 2024-04-23T10:33:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.284+0000 7f4fde7fc640 1 -- 172.21.15.97:0/272054511 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fec035070 con 0x560a7ecae620 2024-04-23T10:33:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.285+0000 7f50011ed740 1 -- 172.21.15.97:0/272054511 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560a7ecae620 msgr2=0x560a7ecaea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.285+0000 7f50011ed740 1 --2- 172.21.15.97:0/272054511 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560a7ecae620 0x560a7ecaea00 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f4fec005c40 tx=0x7f4fec005bb0 comp rx=0 tx=0).stop 2024-04-23T10:33:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.285+0000 7f50011ed740 1 -- 172.21.15.97:0/272054511 shutdown_connections 2024-04-23T10:33:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.285+0000 7f50011ed740 1 --2- 172.21.15.97:0/272054511 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a7ecb9800 0x560a7ecbbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.285+0000 7f50011ed740 1 --2- 172.21.15.97:0/272054511 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a7ecaef40 0x560a7ecb92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.285+0000 7f50011ed740 1 --2- 172.21.15.97:0/272054511 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560a7ecae620 0x560a7ecaea00 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.285+0000 7f50011ed740 1 -- 172.21.15.97:0/272054511 >> 172.21.15.97:0/272054511 conn(0x560a7ecaab30 msgr2=0x560a7ecaaf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:33:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.285+0000 7f50011ed740 1 -- 172.21.15.97:0/272054511 shutdown_connections 2024-04-23T10:33:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.285+0000 7f50011ed740 1 -- 172.21.15.97:0/272054511 wait complete. 2024-04-23T10:33:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.285+0000 7f50011ed740 1 -- start start 2024-04-23T10:33:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.286+0000 7f50011ed740 1 Processor -- start 2024-04-23T10:33:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.286+0000 7f50011ed740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a7ecae620 0x560a7ecb9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.286+0000 7f50011ed740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a7ecaef40 0x560a7ecb6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.286+0000 7f50011ed740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560a7ecb9800 0x560a7ecb69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.286+0000 7f50011ed740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560a7ecbc3d0 con 0x560a7ecb9800 2024-04-23T10:33:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.286+0000 7f50011ed740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560a7ebae0b0 con 0x560a7ecaef40 2024-04-23T10:33:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.286+0000 7f50011ed740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560a7ecbc550 con 0x560a7ecae620 2024-04-23T10:33:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.286+0000 7f50011ed740 10 client.0 _ll_register_callbacks cb 0x560a7eb9bd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:33:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.286+0000 7f4ff77fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560a7ecb9800 0x560a7ecb69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.286+0000 7f4ff77fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560a7ecb9800 0x560a7ecb69a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57630/0 (socket says 172.21.15.97:57630) 2024-04-23T10:33:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.286+0000 7f4ff77fe640 1 -- 172.21.15.97:0/2845447022 learned_addr learned my addr 172.21.15.97:0/2845447022 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:33:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.286+0000 7f4ff7fff640 1 --2- 172.21.15.97:0/2845447022 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a7ecaef40 0x560a7ecb6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.286+0000 7f4ff77fe640 1 --2- 172.21.15.97:0/2845447022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a7ecae620 0x560a7ecb9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.286+0000 7f4ff77fe640 1 -- 172.21.15.97:0/2845447022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a7ecae620 msgr2=0x560a7ecb9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:33:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.286+0000 7f4ff77fe640 1 --2- 172.21.15.97:0/2845447022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a7ecae620 0x560a7ecb9150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.286+0000 7f4ff77fe640 1 -- 172.21.15.97:0/2845447022 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a7ecaef40 msgr2=0x560a7ecb6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:33:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.286+0000 7f4ff77fe640 1 --2- 172.21.15.97:0/2845447022 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a7ecaef40 0x560a7ecb6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.286+0000 7f4ff77fe640 1 -- 172.21.15.97:0/2845447022 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a7e9f2620 con 0x560a7ecb9800 2024-04-23T10:33:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.286+0000 7f4ff77fe640 1 --2- 172.21.15.97:0/2845447022 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560a7ecb9800 0x560a7ecb69a0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f4fec005b80 tx=0x7f4fec02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a743095227286076 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:33:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.286+0000 7f4fd77fe640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:33:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.287+0000 7f4fd77fe640 1 -- 172.21.15.97:0/2845447022 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fec035070 con 0x560a7ecb9800 2024-04-23T10:33:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.287+0000 7f4fd77fe640 1 -- 172.21.15.97:0/2845447022 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7f4fec0035f0 con 0x560a7ecb9800 2024-04-23T10:33:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.287+0000 7f50011ed740 1 -- 172.21.15.97:0/2845447022 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fec0046c0 con 0x560a7ecb9800 2024-04-23T10:33:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.287+0000 7f4fd77fe640 1 -- 172.21.15.97:0/2845447022 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4fec043030 con 0x560a7ecb9800 2024-04-23T10:33:14.298 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.287+0000 7f50011ed740 10 client.16545 fetch_fsmap learned FSMap version 26 2024-04-23T10:33:14.298 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.287+0000 7f50011ed740 1 -- 172.21.15.97:0/2845447022 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560a7ecc8300 con 0x560a7ecb9800 2024-04-23T10:33:14.298 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.287+0000 7f4fd77fe640 1 -- 172.21.15.97:0/2845447022 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fec0046c0 con 0x560a7ecb9800 2024-04-23T10:33:14.298 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.287+0000 7f4fd77fe640 1 -- 172.21.15.97:0/2845447022 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4fec056070 con 0x560a7ecb9800 2024-04-23T10:33:14.298 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.287+0000 7f50011ed740 10 client.16545 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:33:14.298 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.287+0000 7f50011ed740 10 client.16545 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:33:14.298 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.287+0000 7f50011ed740 20 client.16545 populate_metadata read hostname 'smithi097' 2024-04-23T10:33:14.298 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.287+0000 7f50011ed740 1 --2- 172.21.15.97:0/2845447022 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x560a7ecc9650 0x560a7eccba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:14.298 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.287+0000 7f50011ed740 4 client.16545 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:33:14.298 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.287+0000 7f50011ed740 1 -- 172.21.15.97:0/2845447022 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a7ec21390 con 0x560a7ecc9650 2024-04-23T10:33:14.299 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.288+0000 7f4ff7fff640 1 --2- 172.21.15.97:0/2845447022 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x560a7ecc9650 0x560a7eccba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:14.299 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.288+0000 7f4fd77fe640 10 client.16545 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:33:14.299 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.288+0000 7f4ff7fff640 1 --2- 172.21.15.97:0/2845447022 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x560a7ecc9650 0x560a7eccba40 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8dc112e8c6a8dc0 server_cookie=8328d5bc4f2f4e6c in_seq=0 out_seq=0 2024-04-23T10:33:14.299 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.289+0000 7f4fd77fe640 1 -- 172.21.15.97:0/2845447022 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560a7ec21390 con 0x560a7ecc9650 2024-04-23T10:33:14.299 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.289+0000 7f4fd77fe640 10 client.16545 handle_command_reply: tid=0 2024-04-23T10:33:14.299 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.289+0000 7f50011ed740 1 client.16545 shutdown 2024-04-23T10:33:14.299 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.289+0000 7f50011ed740 1 -- 172.21.15.97:0/2845447022 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560a7ecb9800 msgr2=0x560a7ecb69a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:14.299 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.289+0000 7f50011ed740 1 --2- 172.21.15.97:0/2845447022 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560a7ecb9800 0x560a7ecb69a0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f4fec005b80 tx=0x7f4fec02a410 comp rx=0 tx=0).stop 2024-04-23T10:33:14.299 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.289+0000 7f50011ed740 1 -- 172.21.15.97:0/2845447022 shutdown_connections 2024-04-23T10:33:14.299 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.289+0000 7f50011ed740 1 --2- 172.21.15.97:0/2845447022 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x560a7ecc9650 0x560a7eccba40 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:14.299 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.289+0000 7f50011ed740 1 --2- 172.21.15.97:0/2845447022 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560a7ecb9800 0x560a7ecb69a0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.289+0000 7f50011ed740 1 --2- 172.21.15.97:0/2845447022 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a7ecaef40 0x560a7ecb6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.289+0000 7f4fd77fe640 0 client.16545 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:33:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.289+0000 7f50011ed740 1 --2- 172.21.15.97:0/2845447022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a7ecae620 0x560a7ecb9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.289+0000 7f50011ed740 1 -- 172.21.15.97:0/2845447022 >> 172.21.15.97:0/2845447022 conn(0x560a7ecaab30 msgr2=0x560a7ecb08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:33:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.289+0000 7f50011ed740 1 -- 172.21.15.97:0/2845447022 shutdown_connections 2024-04-23T10:33:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.289+0000 7f50011ed740 1 -- 172.21.15.97:0/2845447022 wait complete. 2024-04-23T10:33:14.300 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.289+0000 7f50011ed740 20 client.16545 trim_cache size 0 max 16384 2024-04-23T10:33:14.309 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.304+0000 7f50011ed740 1 Processor -- start 2024-04-23T10:33:14.309 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.304+0000 7f50011ed740 1 -- start start 2024-04-23T10:33:14.309 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.304+0000 7f50011ed740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560a7ecae620 0x560a7ecc5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:14.310 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.304+0000 7f50011ed740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a7ecaef40 0x560a7ecd1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:14.310 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.304+0000 7f50011ed740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a7ecb9800 0x560a7ecd1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:14.310 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.304+0000 7f50011ed740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff81ad970 con 0x560a7ecae620 2024-04-23T10:33:14.310 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.304+0000 7f50011ed740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff81af4b0 con 0x560a7ecb9800 2024-04-23T10:33:14.310 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.304+0000 7f50011ed740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff81ae040 con 0x560a7ecaef40 2024-04-23T10:33:14.310 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.304+0000 7f4ff77fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560a7ecae620 0x560a7ecc5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:14.310 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.304+0000 7f4ff77fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560a7ecae620 0x560a7ecc5290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57632/0 (socket says 172.21.15.97:57632) 2024-04-23T10:33:14.310 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.304+0000 7f4ff77fe640 1 -- 172.21.15.97:0/3774251154 learned_addr learned my addr 172.21.15.97:0/3774251154 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:33:14.310 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.304+0000 7f4ff7fff640 1 --2- 172.21.15.97:0/3774251154 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a7ecaef40 0x560a7ecd1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:14.311 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.304+0000 7f4ff77fe640 1 -- 172.21.15.97:0/3774251154 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a7ecaef40 msgr2=0x560a7ecd1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:33:14.311 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.304+0000 7f4ff77fe640 1 --2- 172.21.15.97:0/3774251154 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a7ecaef40 0x560a7ecd1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:14.311 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.304+0000 7f4ff77fe640 1 -- 172.21.15.97:0/3774251154 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a7ecb9800 msgr2=0x560a7ecd1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:33:14.311 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.304+0000 7f4ff77fe640 1 --2- 172.21.15.97:0/3774251154 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a7ecb9800 0x560a7ecd1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:14.311 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.305+0000 7f4ff77fe640 1 -- 172.21.15.97:0/3774251154 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a7ecb6d10 con 0x560a7ecae620 2024-04-23T10:33:14.311 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.305+0000 7f4ff77fe640 1 --2- 172.21.15.97:0/3774251154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560a7ecae620 0x560a7ecc5290 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f4fec031640 tx=0x7f4fec005bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46f6264078bb5f88 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:33:14.311 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.305+0000 7f4fddffb640 1 -- 172.21.15.97:0/3774251154 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fec05c070 con 0x560a7ecae620 2024-04-23T10:33:14.311 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.305+0000 7f4fddffb640 1 -- 172.21.15.97:0/3774251154 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fec0046c0 con 0x560a7ecae620 2024-04-23T10:33:14.311 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.305+0000 7f4fddffb640 1 -- 172.21.15.97:0/3774251154 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fec035040 con 0x560a7ecae620 2024-04-23T10:33:14.311 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.305+0000 7f50011ed740 1 -- 172.21.15.97:0/3774251154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560a7ecae620 msgr2=0x560a7ecc5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:14.312 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.305+0000 7f50011ed740 1 --2- 172.21.15.97:0/3774251154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560a7ecae620 0x560a7ecc5290 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f4fec031640 tx=0x7f4fec005bb0 comp rx=0 tx=0).stop 2024-04-23T10:33:14.312 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.305+0000 7f50011ed740 1 -- 172.21.15.97:0/3774251154 shutdown_connections 2024-04-23T10:33:14.312 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.305+0000 7f50011ed740 1 --2- 172.21.15.97:0/3774251154 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a7ecb9800 0x560a7ecd1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:14.312 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.305+0000 7f50011ed740 1 --2- 172.21.15.97:0/3774251154 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a7ecaef40 0x560a7ecd1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:14.312 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.305+0000 7f50011ed740 1 --2- 172.21.15.97:0/3774251154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560a7ecae620 0x560a7ecc5290 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:14.312 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.305+0000 7f50011ed740 1 -- 172.21.15.97:0/3774251154 >> 172.21.15.97:0/3774251154 conn(0x560a7ecaab30 msgr2=0x560a7eca9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:33:14.312 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.305+0000 7f50011ed740 1 -- 172.21.15.97:0/3774251154 shutdown_connections 2024-04-23T10:33:14.312 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.306+0000 7f50011ed740 1 -- 172.21.15.97:0/3774251154 wait complete. 2024-04-23T10:33:14.312 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.306+0000 7f50011ed740 1 -- start start 2024-04-23T10:33:14.312 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.306+0000 7f50011ed740 1 Processor -- start 2024-04-23T10:33:14.313 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.306+0000 7f50011ed740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560a7ecae620 0x560a7ecbf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:14.313 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.306+0000 7f50011ed740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a7ecaef40 0x560a7ecbf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:14.313 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.306+0000 7f50011ed740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a7ecb9800 0x560a7ecbfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:14.313 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.306+0000 7f50011ed740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff81aede0 con 0x560a7ecae620 2024-04-23T10:33:14.313 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.306+0000 7f50011ed740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff81ae710 con 0x560a7ecb9800 2024-04-23T10:33:14.313 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.306+0000 7f50011ed740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff81ab970 con 0x560a7ecaef40 2024-04-23T10:33:14.313 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.306+0000 7f50011ed740 10 client.0 _ll_register_callbacks cb 0x7f4fec005480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:33:14.313 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.306+0000 7f4ff77fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560a7ecae620 0x560a7ecbf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:14.313 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.306+0000 7f4ff77fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560a7ecae620 0x560a7ecbf190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57644/0 (socket says 172.21.15.97:57644) 2024-04-23T10:33:14.313 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.306+0000 7f4ff77fe640 1 -- 172.21.15.97:0/2519987926 learned_addr learned my addr 172.21.15.97:0/2519987926 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:33:14.313 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.307+0000 7f4ff77fe640 1 -- 172.21.15.97:0/2519987926 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a7ecaef40 msgr2=0x560a7ecbf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:33:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.307+0000 7f4ff7fff640 1 --2- 172.21.15.97:0/2519987926 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a7ecaef40 0x560a7ecbf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.307+0000 7f4ff77fe640 1 --2- 172.21.15.97:0/2519987926 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a7ecaef40 0x560a7ecbf6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.307+0000 7f4ff77fe640 1 -- 172.21.15.97:0/2519987926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a7ecb9800 msgr2=0x560a7ecbfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:33:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.307+0000 7f4ff77fe640 1 --2- 172.21.15.97:0/2519987926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a7ecb9800 0x560a7ecbfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.307+0000 7f4ff77fe640 1 -- 172.21.15.97:0/2519987926 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a7ecaed70 con 0x560a7ecae620 2024-04-23T10:33:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.307+0000 7f4ff77fe640 1 --2- 172.21.15.97:0/2519987926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560a7ecae620 0x560a7ecbf190 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f4fec04b040 tx=0x7f4fec009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f1051515456a4c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:33:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.307+0000 7f4fd77fe640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:33:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.307+0000 7f4fd77fe640 1 -- 172.21.15.97:0/2519987926 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fec05c070 con 0x560a7ecae620 2024-04-23T10:33:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.307+0000 7f4fd77fe640 1 -- 172.21.15.97:0/2519987926 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7f4fec0037f0 con 0x560a7ecae620 2024-04-23T10:33:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.307+0000 7f50011ed740 1 -- 172.21.15.97:0/2519987926 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fec0046c0 con 0x560a7ecae620 2024-04-23T10:33:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.307+0000 7f4fd77fe640 1 -- 172.21.15.97:0/2519987926 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4fec035040 con 0x560a7ecae620 2024-04-23T10:33:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.307+0000 7f50011ed740 10 client.16551 fetch_fsmap learned FSMap version 26 2024-04-23T10:33:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.307+0000 7f50011ed740 1 -- 172.21.15.97:0/2519987926 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560a7ed2f930 con 0x560a7ecae620 2024-04-23T10:33:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.307+0000 7f4fd77fe640 1 -- 172.21.15.97:0/2519987926 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fec0046c0 con 0x560a7ecae620 2024-04-23T10:33:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.307+0000 7f4fd77fe640 1 -- 172.21.15.97:0/2519987926 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4fec056070 con 0x560a7ecae620 2024-04-23T10:33:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.307+0000 7f50011ed740 10 client.16551 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:33:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.307+0000 7f50011ed740 10 client.16551 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:33:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.307+0000 7f50011ed740 20 client.16551 populate_metadata read hostname 'smithi097' 2024-04-23T10:33:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.307+0000 7f50011ed740 1 --2- 172.21.15.97:0/2519987926 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x560a7ed2ff40 0x560a7ed32360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.308+0000 7f50011ed740 4 client.16551 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:33:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.308+0000 7f50011ed740 1 -- 172.21.15.97:0/2519987926 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560a7ec1b200 con 0x560a7ed2ff40 2024-04-23T10:33:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.308+0000 7f4ff7fff640 1 --2- 172.21.15.97:0/2519987926 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x560a7ed2ff40 0x560a7ed32360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.308+0000 7f4fd77fe640 10 client.16551 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:33:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.308+0000 7f4ff7fff640 1 --2- 172.21.15.97:0/2519987926 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x560a7ed2ff40 0x560a7ed32360 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14e052f6534c75d2 server_cookie=dc392f1beba6647a in_seq=0 out_seq=0 2024-04-23T10:33:14.323 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.323+0000 7f4fd77fe640 1 -- 172.21.15.97:0/2519987926 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560a7ec1b200 con 0x560a7ed2ff40 2024-04-23T10:33:14.323 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.323+0000 7f4fd77fe640 10 client.16551 handle_command_reply: tid=0 2024-04-23T10:33:14.323 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.323+0000 7f50011ed740 1 client.16551 shutdown 2024-04-23T10:33:14.323 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.323+0000 7f50011ed740 1 -- 172.21.15.97:0/2519987926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560a7ecae620 msgr2=0x560a7ecbf190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:14.323 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.323+0000 7f50011ed740 1 --2- 172.21.15.97:0/2519987926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560a7ecae620 0x560a7ecbf190 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f4fec04b040 tx=0x7f4fec009800 comp rx=0 tx=0).stop 2024-04-23T10:33:14.323 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.323+0000 7f50011ed740 1 -- 172.21.15.97:0/2519987926 shutdown_connections 2024-04-23T10:33:14.323 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.323+0000 7f50011ed740 1 --2- 172.21.15.97:0/2519987926 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x560a7ed2ff40 0x560a7ed32360 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:14.323 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.323+0000 7f50011ed740 1 --2- 172.21.15.97:0/2519987926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a7ecb9800 0x560a7ecbfc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:14.323 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.323+0000 7f4fd77fe640 0 client.16551 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:33:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.323+0000 7f50011ed740 1 --2- 172.21.15.97:0/2519987926 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560a7ecaef40 0x560a7ecbf6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.323+0000 7f50011ed740 1 --2- 172.21.15.97:0/2519987926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560a7ecae620 0x560a7ecbf190 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.323+0000 7f50011ed740 1 -- 172.21.15.97:0/2519987926 >> 172.21.15.97:0/2519987926 conn(0x560a7ecaab30 msgr2=0x560a7ecc4280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:33:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.323+0000 7f50011ed740 1 -- 172.21.15.97:0/2519987926 shutdown_connections 2024-04-23T10:33:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.323+0000 7f50011ed740 1 -- 172.21.15.97:0/2519987926 wait complete. 2024-04-23T10:33:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.323+0000 7f50011ed740 20 client.16551 trim_cache size 0 max 16384 2024-04-23T10:33:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:33:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:33:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:33:14.325 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:33:14.325 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.324+0000 7f4ffe3fe640 1 -- 172.21.15.97:0/2476212140 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f4fe00a3dc0 msgr2=0x7f4fe00a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.324+0000 7f4ffe3fe640 1 --2- 172.21.15.97:0/2476212140 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f4fe00a3dc0 0x7f4fe00a6280 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4ff009c470 tx=0x7f4ff006df20 comp rx=0 tx=0).stop 2024-04-23T10:33:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.324+0000 7f4ffe3fe640 1 -- 172.21.15.97:0/2476212140 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ff819cfa0 msgr2=0x7f4ff819d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.324+0000 7f4ffe3fe640 1 --2- 172.21.15.97:0/2476212140 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ff819cfa0 0x7f4ff819d450 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f4ff0009fd0 tx=0x7f4ff000b260 comp rx=0 tx=0).stop 2024-04-23T10:33:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.324+0000 7f4ffe3fe640 1 -- 172.21.15.97:0/2476212140 shutdown_connections 2024-04-23T10:33:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.324+0000 7f4ffe3fe640 1 --2- 172.21.15.97:0/2476212140 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f4fe00a3dc0 0x7f4fe00a6280 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.324+0000 7f4ffe3fe640 1 --2- 172.21.15.97:0/2476212140 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ff819cfa0 0x7f4ff819d450 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.324+0000 7f4ffe3fe640 1 --2- 172.21.15.97:0/2476212140 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ff8107290 0x7f4ff8198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.324+0000 7f4ffe3fe640 1 --2- 172.21.15.97:0/2476212140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ff81021c0 0x7f4ff81986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.324+0000 7f4ffe3fe640 1 -- 172.21.15.97:0/2476212140 >> 172.21.15.97:0/2476212140 conn(0x7f4ff80fda20 msgr2=0x7f4ff80ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:33:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.324+0000 7f4ffe3fe640 1 -- 172.21.15.97:0/2476212140 shutdown_connections 2024-04-23T10:33:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:14.324+0000 7f4ffe3fe640 1 -- 172.21.15.97:0/2476212140 wait complete. 2024-04-23T10:33:15.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:33:15 smithi196 ceph-mon[32919]: pgmap v216: 97 pgs: 97 active+clean; 4.7 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 54 KiB/s wr, 19 op/s 2024-04-23T10:33:15.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:33:15 smithi097 ceph-mon[29197]: pgmap v216: 97 pgs: 97 active+clean; 4.7 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 54 KiB/s wr, 19 op/s 2024-04-23T10:33:15.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:33:15 smithi183 ceph-mon[32317]: pgmap v216: 97 pgs: 97 active+clean; 4.7 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 54 KiB/s wr, 19 op/s 2024-04-23T10:33:15.561 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:33:15.561 INFO:tasks.workunit.client.0.smithi097.stdout:189/600: open fd = 3 2024-04-23T10:33:15.562 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.562 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.562 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:33:15.562 INFO:tasks.workunit.client.0.smithi097.stdout:190/600: open fd = 3 2024-04-23T10:33:15.562 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.562 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.562 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:33:15.562 INFO:tasks.workunit.client.0.smithi097.stdout:191/600: open fd = 3 2024-04-23T10:33:15.562 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.562 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.562 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:33:15.562 INFO:tasks.workunit.client.0.smithi097.stdout:192/600: open fd = 3 2024-04-23T10:33:15.562 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.562 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.562 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:33:15.563 INFO:tasks.workunit.client.0.smithi097.stdout:193/600: open fd = 3 2024-04-23T10:33:15.563 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.563 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.563 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:33:15.563 INFO:tasks.workunit.client.0.smithi097.stdout:194/600: open fd = 3 2024-04-23T10:33:15.563 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.563 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.563 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:33:15.563 INFO:tasks.workunit.client.0.smithi097.stdout:195/600: open fd = 3 2024-04-23T10:33:15.563 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.563 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.563 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:33:15.563 INFO:tasks.workunit.client.0.smithi097.stdout:196/600: open fd = 3 2024-04-23T10:33:15.563 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.563 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.563 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:33:15.564 INFO:tasks.workunit.client.0.smithi097.stdout:197/600: open fd = 3 2024-04-23T10:33:15.564 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.564 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.564 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:33:15.564 INFO:tasks.workunit.client.0.smithi097.stdout:198/600: open fd = 3 2024-04-23T10:33:15.564 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.564 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.564 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:33:15.564 INFO:tasks.workunit.client.0.smithi097.stdout:199/600: open fd = 3 2024-04-23T10:33:15.564 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.564 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.564 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:33:15.564 INFO:tasks.workunit.client.0.smithi097.stdout:200/600: open fd = 3 2024-04-23T10:33:15.564 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.564 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.564 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:33:15.565 INFO:tasks.workunit.client.0.smithi097.stdout:201/600: open fd = 3 2024-04-23T10:33:15.565 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.565 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.565 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:33:15.565 INFO:tasks.workunit.client.0.smithi097.stdout:202/600: open fd = 3 2024-04-23T10:33:15.565 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.565 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.565 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:33:15.565 INFO:tasks.workunit.client.0.smithi097.stdout:203/600: open fd = 3 2024-04-23T10:33:15.565 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.565 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.565 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:33:15.565 INFO:tasks.workunit.client.0.smithi097.stdout:204/600: open fd = 3 2024-04-23T10:33:15.565 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.565 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.566 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:33:15.566 INFO:tasks.workunit.client.0.smithi097.stdout:205/600: open fd = 3 2024-04-23T10:33:15.566 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.566 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.566 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:33:15.566 INFO:tasks.workunit.client.0.smithi097.stdout:206/600: open fd = 3 2024-04-23T10:33:15.566 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.566 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.566 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:33:15.566 INFO:tasks.workunit.client.0.smithi097.stdout:207/600: open fd = 3 2024-04-23T10:33:15.566 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.566 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.566 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:33:15.566 INFO:tasks.workunit.client.0.smithi097.stdout:208/600: open fd = 3 2024-04-23T10:33:15.566 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.567 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.567 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:33:15.567 INFO:tasks.workunit.client.0.smithi097.stdout:209/600: open fd = 3 2024-04-23T10:33:15.567 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.567 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.567 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:33:15.567 INFO:tasks.workunit.client.0.smithi097.stdout:210/600: open fd = 3 2024-04-23T10:33:15.567 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.567 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.567 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:33:15.567 INFO:tasks.workunit.client.0.smithi097.stdout:211/600: open fd = 3 2024-04-23T10:33:15.567 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.567 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.567 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:33:15.567 INFO:tasks.workunit.client.0.smithi097.stdout:212/600: open fd = 3 2024-04-23T10:33:15.568 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.568 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.568 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:33:15.568 INFO:tasks.workunit.client.0.smithi097.stdout:213/600: open fd = 3 2024-04-23T10:33:15.568 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.568 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.568 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:33:15.568 INFO:tasks.workunit.client.0.smithi097.stdout:214/600: open fd = 3 2024-04-23T10:33:15.568 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.568 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.568 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:33:15.568 INFO:tasks.workunit.client.0.smithi097.stdout:215/600: open fd = 3 2024-04-23T10:33:15.568 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.568 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.568 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:33:15.568 INFO:tasks.workunit.client.0.smithi097.stdout:216/600: open fd = 3 2024-04-23T10:33:15.568 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.569 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.569 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:33:15.569 INFO:tasks.workunit.client.0.smithi097.stdout:217/600: open fd = 3 2024-04-23T10:33:15.569 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.569 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.569 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:33:15.569 INFO:tasks.workunit.client.0.smithi097.stdout:218/600: open fd = 3 2024-04-23T10:33:15.569 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.569 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.569 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:33:15.569 INFO:tasks.workunit.client.0.smithi097.stdout:219/600: open fd = 3 2024-04-23T10:33:15.569 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.569 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.569 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:33:15.570 INFO:tasks.workunit.client.0.smithi097.stdout:220/600: open fd = 3 2024-04-23T10:33:15.570 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.570 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.570 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:33:15.570 INFO:tasks.workunit.client.0.smithi097.stdout:221/600: open fd = 3 2024-04-23T10:33:15.570 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.570 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.570 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:33:15.570 INFO:tasks.workunit.client.0.smithi097.stdout:222/600: open fd = 3 2024-04-23T10:33:15.570 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.570 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.570 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:33:15.570 INFO:tasks.workunit.client.0.smithi097.stdout:223/600: open fd = 3 2024-04-23T10:33:15.570 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.570 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.571 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:33:15.571 INFO:tasks.workunit.client.0.smithi097.stdout:224/600: open fd = 3 2024-04-23T10:33:15.571 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.571 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.571 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:33:15.571 INFO:tasks.workunit.client.0.smithi097.stdout:225/600: open fd = 3 2024-04-23T10:33:15.571 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.571 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.571 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:33:15.571 INFO:tasks.workunit.client.0.smithi097.stdout:226/600: open fd = 3 2024-04-23T10:33:15.571 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.571 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.571 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:33:15.571 INFO:tasks.workunit.client.0.smithi097.stdout:227/600: open fd = 3 2024-04-23T10:33:15.571 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.572 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.572 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:33:15.572 INFO:tasks.workunit.client.0.smithi097.stdout:228/600: open fd = 3 2024-04-23T10:33:15.572 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.572 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.572 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:33:15.572 INFO:tasks.workunit.client.0.smithi097.stdout:229/600: open fd = 3 2024-04-23T10:33:15.572 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.572 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.572 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:33:15.572 INFO:tasks.workunit.client.0.smithi097.stdout:230/600: open fd = 3 2024-04-23T10:33:15.572 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.572 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.572 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:33:15.572 INFO:tasks.workunit.client.0.smithi097.stdout:231/600: open fd = 3 2024-04-23T10:33:15.572 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.573 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.573 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:33:15.573 INFO:tasks.workunit.client.0.smithi097.stdout:232/600: open fd = 3 2024-04-23T10:33:15.573 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.573 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.573 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:33:15.573 INFO:tasks.workunit.client.0.smithi097.stdout:233/600: open fd = 3 2024-04-23T10:33:15.573 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.573 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.573 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:33:15.573 INFO:tasks.workunit.client.0.smithi097.stdout:234/600: open fd = 3 2024-04-23T10:33:15.573 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.573 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.573 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:33:15.573 INFO:tasks.workunit.client.0.smithi097.stdout:235/600: open fd = 3 2024-04-23T10:33:15.573 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.573 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.574 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:33:15.574 INFO:tasks.workunit.client.0.smithi097.stdout:236/600: open fd = 3 2024-04-23T10:33:15.574 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.574 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.574 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:33:15.574 INFO:tasks.workunit.client.0.smithi097.stdout:237/600: open fd = 3 2024-04-23T10:33:15.574 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.574 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.574 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:33:15.574 INFO:tasks.workunit.client.0.smithi097.stdout:238/600: open fd = 3 2024-04-23T10:33:15.574 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.574 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.574 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:33:15.574 INFO:tasks.workunit.client.0.smithi097.stdout:239/600: open fd = 3 2024-04-23T10:33:15.574 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.574 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.575 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:33:15.575 INFO:tasks.workunit.client.0.smithi097.stdout:240/600: open fd = 3 2024-04-23T10:33:15.575 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.575 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.575 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:33:15.575 INFO:tasks.workunit.client.0.smithi097.stdout:241/600: open fd = 3 2024-04-23T10:33:15.575 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.575 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.575 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:33:15.575 INFO:tasks.workunit.client.0.smithi097.stdout:242/600: open fd = 3 2024-04-23T10:33:15.575 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.575 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.575 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:33:15.575 INFO:tasks.workunit.client.0.smithi097.stdout:243/600: open fd = 3 2024-04-23T10:33:15.575 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.575 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.576 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:33:15.576 INFO:tasks.workunit.client.0.smithi097.stdout:244/600: open fd = 3 2024-04-23T10:33:15.576 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.576 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.576 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:33:15.576 INFO:tasks.workunit.client.0.smithi097.stdout:245/600: open fd = 3 2024-04-23T10:33:15.576 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.576 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.576 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:33:15.576 INFO:tasks.workunit.client.0.smithi097.stdout:246/600: open fd = 3 2024-04-23T10:33:15.576 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.576 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.576 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:33:15.576 INFO:tasks.workunit.client.0.smithi097.stdout:247/600: open fd = 3 2024-04-23T10:33:15.576 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.577 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.577 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:33:15.577 INFO:tasks.workunit.client.0.smithi097.stdout:248/600: open fd = 3 2024-04-23T10:33:15.577 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.577 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.577 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:33:15.577 INFO:tasks.workunit.client.0.smithi097.stdout:249/600: open fd = 3 2024-04-23T10:33:15.577 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.577 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.577 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:33:15.577 INFO:tasks.workunit.client.0.smithi097.stdout:250/600: open fd = 3 2024-04-23T10:33:15.577 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:15.577 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:33:17.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:33:17 smithi196 ceph-mon[32919]: pgmap v217: 97 pgs: 97 active+clean; 4.7 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 48 KiB/s wr, 17 op/s 2024-04-23T10:33:17.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:33:17 smithi097 ceph-mon[29197]: pgmap v217: 97 pgs: 97 active+clean; 4.7 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 48 KiB/s wr, 17 op/s 2024-04-23T10:33:17.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:33:17 smithi183 ceph-mon[32317]: pgmap v217: 97 pgs: 97 active+clean; 4.7 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 48 KiB/s wr, 17 op/s 2024-04-23T10:33:19.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:33:19 smithi196 ceph-mon[32919]: pgmap v218: 97 pgs: 97 active+clean; 4.9 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 52 KiB/s wr, 19 op/s 2024-04-23T10:33:19.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:33:19 smithi097 ceph-mon[29197]: pgmap v218: 97 pgs: 97 active+clean; 4.9 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 52 KiB/s wr, 19 op/s 2024-04-23T10:33:19.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:33:19 smithi183 ceph-mon[32317]: pgmap v218: 97 pgs: 97 active+clean; 4.9 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 52 KiB/s wr, 19 op/s 2024-04-23T10:33:21.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:33:21 smithi196 ceph-mon[32919]: pgmap v219: 97 pgs: 97 active+clean; 4.9 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 52 KiB/s wr, 19 op/s 2024-04-23T10:33:21.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:33:21 smithi097 ceph-mon[29197]: pgmap v219: 97 pgs: 97 active+clean; 4.9 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 52 KiB/s wr, 19 op/s 2024-04-23T10:33:21.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:33:21 smithi183 ceph-mon[32317]: pgmap v219: 97 pgs: 97 active+clean; 4.9 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 52 KiB/s wr, 19 op/s 2024-04-23T10:33:22.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:33:22 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:33:22.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:33:22 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:33:22.399 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:33:22 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:33:23.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:33:23 smithi196 ceph-mon[32919]: pgmap v220: 97 pgs: 97 active+clean; 4.9 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 51 KiB/s wr, 18 op/s 2024-04-23T10:33:23.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:33:23 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:33:23.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:33:23 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:33:23.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:33:23 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:33:23.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:33:23 smithi097 ceph-mon[29197]: pgmap v220: 97 pgs: 97 active+clean; 4.9 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 51 KiB/s wr, 18 op/s 2024-04-23T10:33:23.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:33:23 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:33:23.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:33:23 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:33:23.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:33:23 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:33:23.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:33:23 smithi183 ceph-mon[32317]: pgmap v220: 97 pgs: 97 active+clean; 4.9 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 51 KiB/s wr, 18 op/s 2024-04-23T10:33:23.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:33:23 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:33:23.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:33:23 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:33:23.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:33:23 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:33:25.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:33:25 smithi196 ceph-mon[32919]: pgmap v221: 97 pgs: 97 active+clean; 5.1 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 61 KiB/s wr, 22 op/s 2024-04-23T10:33:25.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:33:25 smithi097 ceph-mon[29197]: pgmap v221: 97 pgs: 97 active+clean; 5.1 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 61 KiB/s wr, 22 op/s 2024-04-23T10:33:25.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:33:25 smithi183 ceph-mon[32317]: pgmap v221: 97 pgs: 97 active+clean; 5.1 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 61 KiB/s wr, 22 op/s 2024-04-23T10:33:27.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:33:27 smithi097 ceph-mon[29197]: pgmap v222: 97 pgs: 97 active+clean; 5.1 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-23T10:33:27.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:33:27 smithi183 ceph-mon[32317]: pgmap v222: 97 pgs: 97 active+clean; 5.1 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-23T10:33:27.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.420+0000 7fba25b9b640 1 -- 172.21.15.97:0/380125406 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba20106f90 msgr2=0x7fba20107370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:27.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.420+0000 7fba25b9b640 1 --2- 172.21.15.97:0/380125406 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba20106f90 0x7fba20107370 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fba10009930 tx=0x7fba1002a420 comp rx=0 tx=0).stop 2024-04-23T10:33:27.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.420+0000 7fba25b9b640 1 -- 172.21.15.97:0/380125406 shutdown_connections 2024-04-23T10:33:27.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.420+0000 7fba25b9b640 1 --2- 172.21.15.97:0/380125406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fba20101af0 0x7fba2010dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:27.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.420+0000 7fba25b9b640 1 --2- 172.21.15.97:0/380125406 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba20101130 0x7fba201015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:27.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.420+0000 7fba25b9b640 1 --2- 172.21.15.97:0/380125406 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba20106f90 0x7fba20107370 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:27.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.420+0000 7fba25b9b640 1 -- 172.21.15.97:0/380125406 >> 172.21.15.97:0/380125406 conn(0x7fba20077c30 msgr2=0x7fba20078040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:33:27.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.420+0000 7fba25b9b640 1 -- 172.21.15.97:0/380125406 shutdown_connections 2024-04-23T10:33:27.420 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.420+0000 7fba25b9b640 1 -- 172.21.15.97:0/380125406 wait complete. 2024-04-23T10:33:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.421+0000 7fba25b9b640 1 Processor -- start 2024-04-23T10:33:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.421+0000 7fba25b9b640 1 -- start start 2024-04-23T10:33:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.421+0000 7fba25b9b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fba20101130 0x7fba201a05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.421+0000 7fba25b9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba20101af0 0x7fba201a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.421+0000 7fba25b9b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba20106f90 0x7fba2019a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.421+0000 7fba25b9b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fba20111070 con 0x7fba20101af0 2024-04-23T10:33:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.421+0000 7fba25b9b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fba20110ef0 con 0x7fba20101130 2024-04-23T10:33:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.421+0000 7fba25b9b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fba201111f0 con 0x7fba20106f90 2024-04-23T10:33:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.421+0000 7fba1effd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba20101af0 0x7fba201a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.421+0000 7fba1effd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba20101af0 0x7fba201a0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60818/0 (socket says 172.21.15.97:60818) 2024-04-23T10:33:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.421+0000 7fba1effd640 1 -- 172.21.15.97:0/733399777 learned_addr learned my addr 172.21.15.97:0/733399777 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:33:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.421+0000 7fba1f7fe640 1 --2- 172.21.15.97:0/733399777 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fba20101130 0x7fba201a05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.421+0000 7fba1ffff640 1 --2- 172.21.15.97:0/733399777 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba20106f90 0x7fba2019a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.422+0000 7fba1effd640 1 -- 172.21.15.97:0/733399777 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba20106f90 msgr2=0x7fba2019a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.422+0000 7fba1effd640 1 --2- 172.21.15.97:0/733399777 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba20106f90 0x7fba2019a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.422+0000 7fba1effd640 1 -- 172.21.15.97:0/733399777 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fba20101130 msgr2=0x7fba201a05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.422+0000 7fba1effd640 1 --2- 172.21.15.97:0/733399777 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fba20101130 0x7fba201a05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.422+0000 7fba1effd640 1 -- 172.21.15.97:0/733399777 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba2019af30 con 0x7fba20101af0 2024-04-23T10:33:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.422+0000 7fba1effd640 1 --2- 172.21.15.97:0/733399777 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba20101af0 0x7fba201a0af0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fba1400d950 tx=0x7fba1400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:33:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.422+0000 7fba1cff9640 1 -- 172.21.15.97:0/733399777 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba14017030 con 0x7fba20101af0 2024-04-23T10:33:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.422+0000 7fba1cff9640 1 -- 172.21.15.97:0/733399777 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba14004020 con 0x7fba20101af0 2024-04-23T10:33:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.422+0000 7fba1cff9640 1 -- 172.21.15.97:0/733399777 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba140047a0 con 0x7fba20101af0 2024-04-23T10:33:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.422+0000 7fba25b9b640 1 -- 172.21.15.97:0/733399777 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba2019b220 con 0x7fba20101af0 2024-04-23T10:33:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.422+0000 7fba25b9b640 1 -- 172.21.15.97:0/733399777 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba201a7530 con 0x7fba20101af0 2024-04-23T10:33:27.430 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.424+0000 7fba1cff9640 1 -- 172.21.15.97:0/733399777 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fba14009a30 con 0x7fba20101af0 2024-04-23T10:33:27.430 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.424+0000 7fba28993740 1 Processor -- start 2024-04-23T10:33:27.430 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.425+0000 7fba28993740 1 -- start start 2024-04-23T10:33:27.430 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.425+0000 7fba1cff9640 1 --2- 172.21.15.97:0/733399777 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fba080a3dc0 0x7fba080a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:27.430 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.425+0000 7fba28993740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644ae170620 0x5644ae170a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:27.430 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.425+0000 7fba28993740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644ae170f40 0x5644ae17b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:27.430 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.425+0000 7fba28993740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644ae17b800 0x5644ae17dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:27.430 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.425+0000 7fba28993740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5644ae0d5970 con 0x5644ae170620 2024-04-23T10:33:27.430 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.425+0000 7fba28993740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5644ae0724a0 con 0x5644ae170f40 2024-04-23T10:33:27.430 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.425+0000 7fba28993740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5644ae0d1930 con 0x5644ae17b800 2024-04-23T10:33:27.431 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.425+0000 7fba1f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644ae170620 0x5644ae170a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:27.431 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.425+0000 7fba1f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644ae170620 0x5644ae170a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60826/0 (socket says 172.21.15.97:60826) 2024-04-23T10:33:27.431 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.425+0000 7fba1f7fe640 1 -- 172.21.15.97:0/4087634830 learned_addr learned my addr 172.21.15.97:0/4087634830 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:33:27.431 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.425+0000 7fba1ffff640 1 --2- 172.21.15.97:0/733399777 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fba080a3dc0 0x7fba080a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:27.431 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.425+0000 7fba1cff9640 1 -- 172.21.15.97:0/733399777 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7fba140d02a0 con 0x7fba20101af0 2024-04-23T10:33:27.431 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.425+0000 7fba1effd640 1 --2- 172.21.15.97:0/4087634830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644ae170f40 0x5644ae17b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:27.431 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.425+0000 7fba1ffff640 1 --2- 172.21.15.97:0/4087634830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644ae17b800 0x5644ae17dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:27.431 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.425+0000 7fba1f7fe640 1 -- 172.21.15.97:0/4087634830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644ae17b800 msgr2=0x5644ae17dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:33:27.431 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.425+0000 7fba1f7fe640 1 --2- 172.21.15.97:0/4087634830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644ae17b800 0x5644ae17dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:27.431 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.425+0000 7fba1f7fe640 1 -- 172.21.15.97:0/4087634830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644ae170f40 msgr2=0x5644ae17b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:33:27.431 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.425+0000 7fba1f7fe640 1 --2- 172.21.15.97:0/4087634830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644ae170f40 0x5644ae17b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:27.431 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.425+0000 7fba1f7fe640 1 -- 172.21.15.97:0/4087634830 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644adde8210 con 0x5644ae170620 2024-04-23T10:33:27.431 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.425+0000 7fba1effd640 1 --2- 172.21.15.97:0/4087634830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644ae170f40 0x5644ae17b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:33:27.432 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.426+0000 7fba1ffff640 1 --2- 172.21.15.97:0/733399777 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fba080a3dc0 0x7fba080a6280 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fba18005cc0 tx=0x7fba18012960 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:33:27.432 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.426+0000 7fba1f7fe640 1 --2- 172.21.15.97:0/4087634830 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644ae170620 0x5644ae170a00 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fba10009930 tx=0x7fba10005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6a7c3e105be80c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:33:27.432 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.426+0000 7fba05ffb640 1 -- 172.21.15.97:0/4087634830 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba10040070 con 0x5644ae170620 2024-04-23T10:33:27.432 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.426+0000 7fba05ffb640 1 -- 172.21.15.97:0/4087634830 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba100316a0 con 0x5644ae170620 2024-04-23T10:33:27.432 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.426+0000 7fba05ffb640 1 -- 172.21.15.97:0/4087634830 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba10035040 con 0x5644ae170620 2024-04-23T10:33:27.432 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.426+0000 7fba28993740 1 -- 172.21.15.97:0/4087634830 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644ae170620 msgr2=0x5644ae170a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:27.432 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.426+0000 7fba28993740 1 --2- 172.21.15.97:0/4087634830 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644ae170620 0x5644ae170a00 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fba10009930 tx=0x7fba10005fd0 comp rx=0 tx=0).stop 2024-04-23T10:33:27.432 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.427+0000 7fba28993740 1 -- 172.21.15.97:0/4087634830 shutdown_connections 2024-04-23T10:33:27.432 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.427+0000 7fba28993740 1 --2- 172.21.15.97:0/4087634830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644ae17b800 0x5644ae17dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:27.432 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.427+0000 7fba28993740 1 --2- 172.21.15.97:0/4087634830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644ae170f40 0x5644ae17b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:27.432 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.427+0000 7fba28993740 1 --2- 172.21.15.97:0/4087634830 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644ae170620 0x5644ae170a00 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:27.432 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.427+0000 7fba28993740 1 -- 172.21.15.97:0/4087634830 >> 172.21.15.97:0/4087634830 conn(0x5644ae16cb30 msgr2=0x5644ae16cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:33:27.432 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.427+0000 7fba28993740 1 -- 172.21.15.97:0/4087634830 shutdown_connections 2024-04-23T10:33:27.432 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.427+0000 7fba28993740 1 -- 172.21.15.97:0/4087634830 wait complete. 2024-04-23T10:33:27.432 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.427+0000 7fba28993740 1 -- start start 2024-04-23T10:33:27.432 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.427+0000 7fba28993740 1 Processor -- start 2024-04-23T10:33:27.433 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.427+0000 7fba28993740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644ae170620 0x5644ae17b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:27.433 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.427+0000 7fba28993740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644ae170f40 0x5644ae178370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:27.433 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.427+0000 7fba28993740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644ae17b800 0x5644ae1788b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:27.433 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.427+0000 7fba28993740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5644ae17e3d0 con 0x5644ae170f40 2024-04-23T10:33:27.433 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.427+0000 7fba28993740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5644ae0700b0 con 0x5644ae170620 2024-04-23T10:33:27.433 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.427+0000 7fba28993740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5644ae17e550 con 0x5644ae17b800 2024-04-23T10:33:27.433 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.427+0000 7fba28993740 10 client.0 _ll_register_callbacks cb 0x5644ae05dd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:33:27.433 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.428+0000 7fba1effd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644ae170f40 0x5644ae178370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:27.433 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.428+0000 7fba1effd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644ae170f40 0x5644ae178370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60838/0 (socket says 172.21.15.97:60838) 2024-04-23T10:33:27.433 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.428+0000 7fba1effd640 1 -- 172.21.15.97:0/3014444358 learned_addr learned my addr 172.21.15.97:0/3014444358 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:33:27.433 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.428+0000 7fba1f7fe640 1 --2- 172.21.15.97:0/3014444358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644ae170620 0x5644ae17b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:27.433 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.428+0000 7fba1ffff640 1 --2- 172.21.15.97:0/3014444358 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644ae17b800 0x5644ae1788b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:27.433 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.428+0000 7fba1effd640 1 -- 172.21.15.97:0/3014444358 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644ae17b800 msgr2=0x5644ae1788b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:33:27.433 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.428+0000 7fba1effd640 1 --2- 172.21.15.97:0/3014444358 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644ae17b800 0x5644ae1788b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:27.433 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.428+0000 7fba1effd640 1 -- 172.21.15.97:0/3014444358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644ae170620 msgr2=0x5644ae17b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:33:27.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.428+0000 7fba1effd640 1 --2- 172.21.15.97:0/3014444358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644ae170620 0x5644ae17b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:27.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.428+0000 7fba1effd640 1 -- 172.21.15.97:0/3014444358 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644adeb4620 con 0x5644ae170f40 2024-04-23T10:33:27.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.428+0000 7fba1effd640 1 --2- 172.21.15.97:0/3014444358 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644ae170f40 0x5644ae178370 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fba14098c30 tx=0x7fba140b59e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adfb25a6cee016ed server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:33:27.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.428+0000 7fb9feffd640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:33:27.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.428+0000 7fb9feffd640 1 -- 172.21.15.97:0/3014444358 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba140b3e10 con 0x5644ae170f40 2024-04-23T10:33:27.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.428+0000 7fb9feffd640 1 -- 172.21.15.97:0/3014444358 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7fba140cdc90 con 0x5644ae170f40 2024-04-23T10:33:27.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.428+0000 7fba28993740 1 -- 172.21.15.97:0/3014444358 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba100316a0 con 0x5644ae170f40 2024-04-23T10:33:27.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.428+0000 7fb9feffd640 1 -- 172.21.15.97:0/3014444358 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fba1409c260 con 0x5644ae170f40 2024-04-23T10:33:27.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.428+0000 7fba28993740 10 client.16563 fetch_fsmap learned FSMap version 26 2024-04-23T10:33:27.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.428+0000 7fba28993740 1 -- 172.21.15.97:0/3014444358 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5644ae18a220 con 0x5644ae170f40 2024-04-23T10:33:27.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.428+0000 7fb9feffd640 1 -- 172.21.15.97:0/3014444358 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba100316a0 con 0x5644ae170f40 2024-04-23T10:33:27.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.429+0000 7fb9feffd640 1 -- 172.21.15.97:0/3014444358 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fba140ba070 con 0x5644ae170f40 2024-04-23T10:33:27.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.429+0000 7fba28993740 10 client.16563 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:33:27.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.429+0000 7fba28993740 10 client.16563 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:33:27.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.429+0000 7fba28993740 20 client.16563 populate_metadata read hostname 'smithi097' 2024-04-23T10:33:27.435 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.429+0000 7fba28993740 1 --2- 172.21.15.97:0/3014444358 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5644ae18b6e0 0x5644ae18db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:27.435 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.429+0000 7fba28993740 4 client.16563 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:33:27.435 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.429+0000 7fba28993740 1 -- 172.21.15.97:0/3014444358 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644ae0e3390 con 0x5644ae18b6e0 2024-04-23T10:33:27.435 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.429+0000 7fba1f7fe640 1 --2- 172.21.15.97:0/3014444358 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5644ae18b6e0 0x5644ae18db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:27.435 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.429+0000 7fba1f7fe640 1 --2- 172.21.15.97:0/3014444358 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5644ae18b6e0 0x5644ae18db10 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcb6209a3f58519 server_cookie=4f210573c557474c in_seq=0 out_seq=0 2024-04-23T10:33:27.435 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.429+0000 7fb9feffd640 10 client.16563 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:33:27.435 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.430+0000 7fb9feffd640 1 -- 172.21.15.97:0/3014444358 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5644ae0e3390 con 0x5644ae18b6e0 2024-04-23T10:33:27.435 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.430+0000 7fb9feffd640 10 client.16563 handle_command_reply: tid=0 2024-04-23T10:33:27.435 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.430+0000 7fba28993740 1 client.16563 shutdown 2024-04-23T10:33:27.435 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.430+0000 7fba28993740 1 -- 172.21.15.97:0/3014444358 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644ae170f40 msgr2=0x5644ae178370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:27.435 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.430+0000 7fba28993740 1 --2- 172.21.15.97:0/3014444358 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644ae170f40 0x5644ae178370 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fba14098c30 tx=0x7fba140b59e0 comp rx=0 tx=0).stop 2024-04-23T10:33:27.435 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.430+0000 7fba28993740 1 -- 172.21.15.97:0/3014444358 shutdown_connections 2024-04-23T10:33:27.435 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.430+0000 7fba28993740 1 --2- 172.21.15.97:0/3014444358 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5644ae18b6e0 0x5644ae18db10 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:27.435 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.430+0000 7fba28993740 1 --2- 172.21.15.97:0/3014444358 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644ae17b800 0x5644ae1788b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:27.435 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.430+0000 7fba28993740 1 --2- 172.21.15.97:0/3014444358 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644ae170f40 0x5644ae178370 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:27.435 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.430+0000 7fba28993740 1 --2- 172.21.15.97:0/3014444358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644ae170620 0x5644ae17b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:27.436 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.430+0000 7fb9feffd640 0 client.16563 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:33:27.436 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.430+0000 7fba28993740 1 -- 172.21.15.97:0/3014444358 >> 172.21.15.97:0/3014444358 conn(0x5644ae16cb30 msgr2=0x5644ae1728b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:33:27.436 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.430+0000 7fba28993740 1 -- 172.21.15.97:0/3014444358 shutdown_connections 2024-04-23T10:33:27.436 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.430+0000 7fba28993740 1 -- 172.21.15.97:0/3014444358 wait complete. 2024-04-23T10:33:27.436 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.431+0000 7fba28993740 20 client.16563 trim_cache size 0 max 16384 2024-04-23T10:33:27.449 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.446+0000 7fba28993740 1 Processor -- start 2024-04-23T10:33:27.449 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.446+0000 7fba28993740 1 -- start start 2024-04-23T10:33:27.449 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.446+0000 7fba28993740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644ae170620 0x5644ae1774d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:27.449 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.446+0000 7fba28993740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644ae170f40 0x5644ae17abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:27.449 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.446+0000 7fba28993740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644ae17b800 0x5644ae193390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:27.450 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.446+0000 7fba28993740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fba201afc90 con 0x5644ae170f40 2024-04-23T10:33:27.450 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.446+0000 7fba28993740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fba201b17d0 con 0x5644ae17b800 2024-04-23T10:33:27.450 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.446+0000 7fba28993740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fba201b0360 con 0x5644ae170620 2024-04-23T10:33:27.450 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.446+0000 7fba1effd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644ae170f40 0x5644ae17abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:27.450 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.446+0000 7fba1effd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644ae170f40 0x5644ae17abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60844/0 (socket says 172.21.15.97:60844) 2024-04-23T10:33:27.450 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.446+0000 7fba1effd640 1 -- 172.21.15.97:0/2359599462 learned_addr learned my addr 172.21.15.97:0/2359599462 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:33:27.450 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.446+0000 7fba1effd640 1 -- 172.21.15.97:0/2359599462 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644ae170620 msgr2=0x5644ae1774d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:33:27.450 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.446+0000 7fba1f7fe640 1 --2- 172.21.15.97:0/2359599462 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644ae170620 0x5644ae1774d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:27.450 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.446+0000 7fba1effd640 1 --2- 172.21.15.97:0/2359599462 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644ae170620 0x5644ae1774d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:27.450 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.446+0000 7fba1effd640 1 -- 172.21.15.97:0/2359599462 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644ae17b800 msgr2=0x5644ae193390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:33:27.450 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.446+0000 7fba1effd640 1 --2- 172.21.15.97:0/2359599462 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644ae17b800 0x5644ae193390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:27.450 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.446+0000 7fba1effd640 1 -- 172.21.15.97:0/2359599462 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644ae1768c0 con 0x5644ae170f40 2024-04-23T10:33:27.450 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.446+0000 7fba1f7fe640 1 --2- 172.21.15.97:0/2359599462 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644ae170620 0x5644ae1774d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:33:27.450 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.446+0000 7fba1effd640 1 --2- 172.21.15.97:0/2359599462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644ae170f40 0x5644ae17abb0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fba140986c0 tx=0x7fba1409bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d9570db65f54eb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:33:27.450 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.446+0000 7fba057fa640 1 -- 172.21.15.97:0/2359599462 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1409bda0 con 0x5644ae170f40 2024-04-23T10:33:27.451 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.446+0000 7fba057fa640 1 -- 172.21.15.97:0/2359599462 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba140b5d00 con 0x5644ae170f40 2024-04-23T10:33:27.451 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.446+0000 7fba057fa640 1 -- 172.21.15.97:0/2359599462 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba14098500 con 0x5644ae170f40 2024-04-23T10:33:27.451 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.447+0000 7fba28993740 1 -- 172.21.15.97:0/2359599462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644ae170f40 msgr2=0x5644ae17abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:27.451 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.447+0000 7fba28993740 1 --2- 172.21.15.97:0/2359599462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644ae170f40 0x5644ae17abb0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fba140986c0 tx=0x7fba1409bcd0 comp rx=0 tx=0).stop 2024-04-23T10:33:27.451 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.447+0000 7fba28993740 1 -- 172.21.15.97:0/2359599462 shutdown_connections 2024-04-23T10:33:27.451 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.447+0000 7fba28993740 1 --2- 172.21.15.97:0/2359599462 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644ae17b800 0x5644ae193390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:27.451 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.447+0000 7fba28993740 1 --2- 172.21.15.97:0/2359599462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644ae170f40 0x5644ae17abb0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:27.451 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.447+0000 7fba28993740 1 --2- 172.21.15.97:0/2359599462 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644ae170620 0x5644ae1774d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:27.451 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.447+0000 7fba28993740 1 -- 172.21.15.97:0/2359599462 >> 172.21.15.97:0/2359599462 conn(0x5644ae16cb30 msgr2=0x5644ae171fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:33:27.451 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.447+0000 7fba28993740 1 -- 172.21.15.97:0/2359599462 shutdown_connections 2024-04-23T10:33:27.451 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.447+0000 7fba28993740 1 -- 172.21.15.97:0/2359599462 wait complete. 2024-04-23T10:33:27.451 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.447+0000 7fba28993740 1 -- start start 2024-04-23T10:33:27.451 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.448+0000 7fba28993740 1 Processor -- start 2024-04-23T10:33:27.451 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.448+0000 7fba28993740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644ae170620 0x5644ae183c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:27.451 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.448+0000 7fba28993740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644ae170f40 0x5644ae17c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:27.451 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.448+0000 7fba28993740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644ae17b800 0x5644ae17c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:27.452 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.448+0000 7fba28993740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fba201b1100 con 0x5644ae170f40 2024-04-23T10:33:27.452 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.448+0000 7fba28993740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fba201b0a30 con 0x5644ae17b800 2024-04-23T10:33:27.452 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.448+0000 7fba28993740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fba201adc90 con 0x5644ae170620 2024-04-23T10:33:27.452 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.448+0000 7fba28993740 10 client.0 _ll_register_callbacks cb 0x7fba140b5bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:33:27.452 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.448+0000 7fba1effd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644ae170f40 0x5644ae17c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:27.452 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.448+0000 7fba1effd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644ae170f40 0x5644ae17c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60852/0 (socket says 172.21.15.97:60852) 2024-04-23T10:33:27.452 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.448+0000 7fba1effd640 1 -- 172.21.15.97:0/434017460 learned_addr learned my addr 172.21.15.97:0/434017460 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:33:27.452 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.448+0000 7fba1f7fe640 1 --2- 172.21.15.97:0/434017460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644ae170620 0x5644ae183c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:27.452 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.448+0000 7fba1ffff640 1 --2- 172.21.15.97:0/434017460 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644ae17b800 0x5644ae17c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:27.452 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.448+0000 7fba1effd640 1 -- 172.21.15.97:0/434017460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644ae170620 msgr2=0x5644ae183c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:33:27.452 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.448+0000 7fba1effd640 1 --2- 172.21.15.97:0/434017460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644ae170620 0x5644ae183c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:27.452 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.448+0000 7fba1effd640 1 -- 172.21.15.97:0/434017460 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644ae17b800 msgr2=0x5644ae17c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:33:27.452 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.448+0000 7fba1effd640 1 --2- 172.21.15.97:0/434017460 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644ae17b800 0x5644ae17c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:27.452 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.448+0000 7fba1effd640 1 -- 172.21.15.97:0/434017460 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644ae17d0d0 con 0x5644ae170f40 2024-04-23T10:33:27.452 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.448+0000 7fba1ffff640 1 --2- 172.21.15.97:0/434017460 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644ae17b800 0x5644ae17c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:33:27.453 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.448+0000 7fba1effd640 1 --2- 172.21.15.97:0/434017460 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644ae170f40 0x5644ae17c360 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fba14005fd0 tx=0x7fba14009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=871859e5830cc2f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:33:27.453 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.448+0000 7fb9feffd640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:33:27.453 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.449+0000 7fb9feffd640 1 -- 172.21.15.97:0/434017460 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba140a0610 con 0x5644ae170f40 2024-04-23T10:33:27.453 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.449+0000 7fb9feffd640 1 -- 172.21.15.97:0/434017460 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7fba140988d0 con 0x5644ae170f40 2024-04-23T10:33:27.453 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.449+0000 7fba28993740 1 -- 172.21.15.97:0/434017460 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba140b5d00 con 0x5644ae170f40 2024-04-23T10:33:27.453 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.449+0000 7fb9feffd640 1 -- 172.21.15.97:0/434017460 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fba140a4ed0 con 0x5644ae170f40 2024-04-23T10:33:27.453 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.449+0000 7fba28993740 10 client.16569 fetch_fsmap learned FSMap version 26 2024-04-23T10:33:27.453 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.449+0000 7fba28993740 1 -- 172.21.15.97:0/434017460 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5644ae1f1c00 con 0x5644ae170f40 2024-04-23T10:33:27.453 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.449+0000 7fb9feffd640 1 -- 172.21.15.97:0/434017460 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba140b5d00 con 0x5644ae170f40 2024-04-23T10:33:27.453 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.449+0000 7fb9feffd640 1 -- 172.21.15.97:0/434017460 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fba140aa070 con 0x5644ae170f40 2024-04-23T10:33:27.453 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.449+0000 7fba28993740 10 client.16569 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:33:27.453 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.449+0000 7fba28993740 10 client.16569 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:33:27.453 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.449+0000 7fba28993740 20 client.16569 populate_metadata read hostname 'smithi097' 2024-04-23T10:33:27.453 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.449+0000 7fba28993740 1 --2- 172.21.15.97:0/434017460 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5644ae1f20d0 0x5644ae1f44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:27.453 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.449+0000 7fba28993740 4 client.16569 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:33:27.453 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.449+0000 7fba28993740 1 -- 172.21.15.97:0/434017460 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5644ae0dd200 con 0x5644ae1f20d0 2024-04-23T10:33:27.454 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.449+0000 7fba1f7fe640 1 --2- 172.21.15.97:0/434017460 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5644ae1f20d0 0x5644ae1f44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:27.454 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.450+0000 7fba1f7fe640 1 --2- 172.21.15.97:0/434017460 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5644ae1f20d0 0x5644ae1f44c0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b01271c4558f3219 server_cookie=176a66e7b76ad0bc in_seq=0 out_seq=0 2024-04-23T10:33:27.454 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.450+0000 7fb9feffd640 10 client.16569 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:33:27.463 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.464+0000 7fb9feffd640 1 -- 172.21.15.97:0/434017460 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5644ae0dd200 con 0x5644ae1f20d0 2024-04-23T10:33:27.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.464+0000 7fb9feffd640 10 client.16569 handle_command_reply: tid=0 2024-04-23T10:33:27.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.464+0000 7fba28993740 1 client.16569 shutdown 2024-04-23T10:33:27.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.464+0000 7fba28993740 1 -- 172.21.15.97:0/434017460 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644ae170f40 msgr2=0x5644ae17c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:27.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.464+0000 7fba28993740 1 --2- 172.21.15.97:0/434017460 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644ae170f40 0x5644ae17c360 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fba14005fd0 tx=0x7fba14009f30 comp rx=0 tx=0).stop 2024-04-23T10:33:27.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.464+0000 7fba28993740 1 -- 172.21.15.97:0/434017460 shutdown_connections 2024-04-23T10:33:27.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.464+0000 7fba28993740 1 --2- 172.21.15.97:0/434017460 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5644ae1f20d0 0x5644ae1f44c0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:27.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.464+0000 7fba28993740 1 --2- 172.21.15.97:0/434017460 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5644ae17b800 0x5644ae17c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:27.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.464+0000 7fba28993740 1 --2- 172.21.15.97:0/434017460 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5644ae170f40 0x5644ae17c360 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:27.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.464+0000 7fb9feffd640 0 client.16569 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:33:27.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.464+0000 7fba28993740 1 --2- 172.21.15.97:0/434017460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5644ae170620 0x5644ae183c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:27.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.464+0000 7fba28993740 1 -- 172.21.15.97:0/434017460 >> 172.21.15.97:0/434017460 conn(0x5644ae16cb30 msgr2=0x5644ae16b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:33:27.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.464+0000 7fba28993740 1 -- 172.21.15.97:0/434017460 shutdown_connections 2024-04-23T10:33:27.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.464+0000 7fba28993740 1 -- 172.21.15.97:0/434017460 wait complete. 2024-04-23T10:33:27.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.465+0000 7fba28993740 20 client.16569 trim_cache size 0 max 16384 2024-04-23T10:33:27.465 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:33:27.465 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:33:27.465 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:33:27.465 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:33:27.466 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.465+0000 7fba25b9b640 1 -- 172.21.15.97:0/733399777 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fba080a3dc0 msgr2=0x7fba080a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:27.466 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.465+0000 7fba25b9b640 1 --2- 172.21.15.97:0/733399777 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fba080a3dc0 0x7fba080a6280 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fba18005cc0 tx=0x7fba18012960 comp rx=0 tx=0).stop 2024-04-23T10:33:27.466 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.465+0000 7fba25b9b640 1 -- 172.21.15.97:0/733399777 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba20101af0 msgr2=0x7fba201a0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:27.466 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.465+0000 7fba25b9b640 1 --2- 172.21.15.97:0/733399777 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba20101af0 0x7fba201a0af0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fba1400d950 tx=0x7fba1400de10 comp rx=0 tx=0).stop 2024-04-23T10:33:27.466 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.465+0000 7fba25b9b640 1 -- 172.21.15.97:0/733399777 shutdown_connections 2024-04-23T10:33:27.466 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.465+0000 7fba25b9b640 1 --2- 172.21.15.97:0/733399777 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fba080a3dc0 0x7fba080a6280 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:27.466 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.465+0000 7fba25b9b640 1 --2- 172.21.15.97:0/733399777 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba20106f90 0x7fba2019a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:27.466 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.465+0000 7fba25b9b640 1 --2- 172.21.15.97:0/733399777 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba20101af0 0x7fba201a0af0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:27.466 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.465+0000 7fba25b9b640 1 --2- 172.21.15.97:0/733399777 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fba20101130 0x7fba201a05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:27.466 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.465+0000 7fba25b9b640 1 -- 172.21.15.97:0/733399777 >> 172.21.15.97:0/733399777 conn(0x7fba20077c30 msgr2=0x7fba2006a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:33:27.466 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.465+0000 7fba25b9b640 1 -- 172.21.15.97:0/733399777 shutdown_connections 2024-04-23T10:33:27.467 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:27.465+0000 7fba25b9b640 1 -- 172.21.15.97:0/733399777 wait complete. 2024-04-23T10:33:27.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:33:27 smithi196 ceph-mon[32919]: pgmap v222: 97 pgs: 97 active+clean; 5.1 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-23T10:33:29.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:33:29 smithi097 ceph-mon[29197]: pgmap v223: 97 pgs: 97 active+clean; 5.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-23T10:33:29.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:33:29 smithi183 ceph-mon[32317]: pgmap v223: 97 pgs: 97 active+clean; 5.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-23T10:33:29.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:33:29 smithi196 ceph-mon[32919]: pgmap v223: 97 pgs: 97 active+clean; 5.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-23T10:33:31.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:33:31 smithi097 ceph-mon[29197]: pgmap v224: 97 pgs: 97 active+clean; 5.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-23T10:33:31.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:33:31 smithi183 ceph-mon[32317]: pgmap v224: 97 pgs: 97 active+clean; 5.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-23T10:33:31.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:33:31 smithi196 ceph-mon[32919]: pgmap v224: 97 pgs: 97 active+clean; 5.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-23T10:33:33.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:33:33 smithi097 ceph-mon[29197]: pgmap v225: 97 pgs: 97 active+clean; 5.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-23T10:33:33.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:33:33 smithi183 ceph-mon[32317]: pgmap v225: 97 pgs: 97 active+clean; 5.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-23T10:33:33.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:33:33 smithi196 ceph-mon[32919]: pgmap v225: 97 pgs: 97 active+clean; 5.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-23T10:33:35.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:33:35 smithi097 ceph-mon[29197]: pgmap v226: 97 pgs: 97 active+clean; 1.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-23T10:33:35.390 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 15.3 2024-04-23T10:33:35.390 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T10:33:35.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:33:35 smithi183 ceph-mon[32317]: pgmap v226: 97 pgs: 97 active+clean; 1.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-23T10:33:35.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.502+0000 7fb9e7929640 1 -- 172.21.15.97:0/3297574617 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9e0107280 msgr2=0x7fb9e010ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:35.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.502+0000 7fb9e7929640 1 --2- 172.21.15.97:0/3297574617 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9e0107280 0x7fb9e010ee50 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fb9d8009ca0 tx=0x7fb9d802a8c0 comp rx=0 tx=0).stop 2024-04-23T10:33:35.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.502+0000 7fb9e7929640 1 -- 172.21.15.97:0/3297574617 shutdown_connections 2024-04-23T10:33:35.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.502+0000 7fb9e7929640 1 --2- 172.21.15.97:0/3297574617 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9e0107280 0x7fb9e010ee50 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:35.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.502+0000 7fb9e7929640 1 --2- 172.21.15.97:0/3297574617 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9e0102bf0 0x7fb9e0106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:35.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.502+0000 7fb9e7929640 1 --2- 172.21.15.97:0/3297574617 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9e0102240 0x7fb9e0102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:35.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.502+0000 7fb9e7929640 1 -- 172.21.15.97:0/3297574617 >> 172.21.15.97:0/3297574617 conn(0x7fb9e00790e0 msgr2=0x7fb9e00794f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:33:35.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.502+0000 7fb9e7929640 1 -- 172.21.15.97:0/3297574617 shutdown_connections 2024-04-23T10:33:35.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.502+0000 7fb9e7929640 1 -- 172.21.15.97:0/3297574617 wait complete. 2024-04-23T10:33:35.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.503+0000 7fb9e7929640 1 Processor -- start 2024-04-23T10:33:35.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.503+0000 7fb9e7929640 1 -- start start 2024-04-23T10:33:35.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.503+0000 7fb9e7929640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9e0102240 0x7fb9e0198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:35.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.503+0000 7fb9e7929640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9e0107280 0x7fb9e0198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:35.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.503+0000 7fb9e7929640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9e019cfe0 0x7fb9e019d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:35.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.503+0000 7fb9e7929640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e0112200 con 0x7fb9e019cfe0 2024-04-23T10:33:35.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.503+0000 7fb9e7929640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e0112080 con 0x7fb9e0107280 2024-04-23T10:33:35.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.503+0000 7fb9e7929640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e0112380 con 0x7fb9e0102240 2024-04-23T10:33:35.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.503+0000 7fb9e5ec6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9e019cfe0 0x7fb9e019d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:35.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.503+0000 7fb9e5ec6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9e019cfe0 0x7fb9e019d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56382/0 (socket says 172.21.15.97:56382) 2024-04-23T10:33:35.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.503+0000 7fb9e5ec6640 1 -- 172.21.15.97:0/4022632932 learned_addr learned my addr 172.21.15.97:0/4022632932 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:33:35.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.503+0000 7fb9e4ec4640 1 --2- 172.21.15.97:0/4022632932 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9e0107280 0x7fb9e0198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:35.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.503+0000 7fb9e56c5640 1 --2- 172.21.15.97:0/4022632932 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9e0102240 0x7fb9e0198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:35.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.503+0000 7fb9e5ec6640 1 -- 172.21.15.97:0/4022632932 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9e0102240 msgr2=0x7fb9e0198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:35.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.503+0000 7fb9e5ec6640 1 --2- 172.21.15.97:0/4022632932 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9e0102240 0x7fb9e0198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:35.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.503+0000 7fb9e5ec6640 1 -- 172.21.15.97:0/4022632932 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9e0107280 msgr2=0x7fb9e0198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:35.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.503+0000 7fb9e5ec6640 1 --2- 172.21.15.97:0/4022632932 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9e0107280 0x7fb9e0198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:35.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.503+0000 7fb9e5ec6640 1 -- 172.21.15.97:0/4022632932 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9e019dad0 con 0x7fb9e019cfe0 2024-04-23T10:33:35.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.503+0000 7fb9e4ec4640 1 --2- 172.21.15.97:0/4022632932 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9e0107280 0x7fb9e0198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:33:35.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.504+0000 7fb9e5ec6640 1 --2- 172.21.15.97:0/4022632932 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9e019cfe0 0x7fb9e019d490 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fb9d8005cc0 tx=0x7fb9d802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:33:35.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.504+0000 7fb9d67fc640 1 -- 172.21.15.97:0/4022632932 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d8003bf0 con 0x7fb9e019cfe0 2024-04-23T10:33:35.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.504+0000 7fb9e7929640 1 -- 172.21.15.97:0/4022632932 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9e019dd60 con 0x7fb9e019cfe0 2024-04-23T10:33:35.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.504+0000 7fb9d67fc640 1 -- 172.21.15.97:0/4022632932 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9d8003d90 con 0x7fb9e019cfe0 2024-04-23T10:33:35.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.504+0000 7fb9d67fc640 1 -- 172.21.15.97:0/4022632932 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d802fbc0 con 0x7fb9e019cfe0 2024-04-23T10:33:35.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.504+0000 7fb9e7929640 1 -- 172.21.15.97:0/4022632932 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9e01a5580 con 0x7fb9e019cfe0 2024-04-23T10:33:35.505 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.505+0000 7fb9e7929640 1 -- 172.21.15.97:0/4022632932 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9e0199a70 con 0x7fb9e019cfe0 2024-04-23T10:33:35.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.507+0000 7fb9d67fc640 1 -- 172.21.15.97:0/4022632932 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb9d802fd60 con 0x7fb9e019cfe0 2024-04-23T10:33:35.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.507+0000 7fb9d67fc640 1 --2- 172.21.15.97:0/4022632932 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fb9cc0a3d20 0x7fb9cc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:35.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.507+0000 7fb9e56c5640 1 --2- 172.21.15.97:0/4022632932 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fb9cc0a3d20 0x7fb9cc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:35.506 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.507+0000 7fb9d67fc640 1 -- 172.21.15.97:0/4022632932 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7fb9d80f2090 con 0x7fb9e019cfe0 2024-04-23T10:33:35.507 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.508+0000 7fb9e56c5640 1 --2- 172.21.15.97:0/4022632932 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fb9cc0a3d20 0x7fb9cc0a61e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc005fd0 tx=0x7fb9dc005850 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:33:35.510 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.511+0000 7fb9d67fc640 1 -- 172.21.15.97:0/4022632932 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb9d807fff0 con 0x7fb9e019cfe0 2024-04-23T10:33:35.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:33:35 smithi196 ceph-mon[32919]: pgmap v226: 97 pgs: 97 active+clean; 1.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-23T10:33:35.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.726+0000 7fb9e7929640 1 -- 172.21.15.97:0/4022632932 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb9e00691a0 con 0x7fb9e019cfe0 2024-04-23T10:33:35.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.727+0000 7fb9d67fc640 1 -- 172.21.15.97:0/4022632932 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11931 (secure 0 0 0) 0x7fb9e00691a0 con 0x7fb9e019cfe0 2024-04-23T10:33:35.726 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:33:35.726 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3946878762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3946878762},{"type":"v1","addr":"172.21.15.97:6835","nonce":3946878762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/209800861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":209800861},{"type":"v1","addr":"172.21.15.97:6837","nonce":209800861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T10:24:06.910134+0000","modified":"2024-04-23T10:26:09.820985+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6839/418013575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":418013575},{"type":"v1","addr":"172.21.15.97:6839","nonce":418013575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6841/747242680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":747242680},{"type":"v1","addr":"172.21.15.97:6841","nonce":747242680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6833/4292847121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4292847121},{"type":"v1","addr":"172.21.15.183:6833","nonce":4292847121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6833/3885816565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3885816565},{"type":"v1","addr":"172.21.15.196:6833","nonce":3885816565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1897093105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1897093105},{"type":"v1","addr":"172.21.15.183:6835","nonce":1897093105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1171477563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1171477563},{"type":"v1","addr":"172.21.15.196:6835","nonce":1171477563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1898341911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1898341911},{"type":"v1","addr":"172.21.15.183:6837","nonce":1898341911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.196:6837/3094946502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3094946502},{"type":"v1","addr":"172.21.15.196:6837","nonce":3094946502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.196:6839/83843556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":83843556},{"type":"v1","addr":"172.21.15.196:6839","nonce":83843556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6839/1763841130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1763841130},{"type":"v1","addr":"172.21.15.183:6839","nonce":1763841130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24487,24403,24398,24469,24488]},"id":1}]} 2024-04-23T10:33:35.726 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 26 2024-04-23T10:33:35.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.729+0000 7fb9e7929640 1 -- 172.21.15.97:0/4022632932 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fb9cc0a3d20 msgr2=0x7fb9cc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:35.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.729+0000 7fb9e7929640 1 --2- 172.21.15.97:0/4022632932 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fb9cc0a3d20 0x7fb9cc0a61e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc005fd0 tx=0x7fb9dc005850 comp rx=0 tx=0).stop 2024-04-23T10:33:35.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.729+0000 7fb9e7929640 1 -- 172.21.15.97:0/4022632932 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9e019cfe0 msgr2=0x7fb9e019d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:35.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.729+0000 7fb9e7929640 1 --2- 172.21.15.97:0/4022632932 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9e019cfe0 0x7fb9e019d490 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fb9d8005cc0 tx=0x7fb9d802f380 comp rx=0 tx=0).stop 2024-04-23T10:33:35.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.729+0000 7fb9e7929640 1 -- 172.21.15.97:0/4022632932 shutdown_connections 2024-04-23T10:33:35.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.729+0000 7fb9e7929640 1 --2- 172.21.15.97:0/4022632932 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fb9cc0a3d20 0x7fb9cc0a61e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:35.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.729+0000 7fb9e7929640 1 --2- 172.21.15.97:0/4022632932 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9e019cfe0 0x7fb9e019d490 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:35.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.729+0000 7fb9e7929640 1 --2- 172.21.15.97:0/4022632932 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9e0107280 0x7fb9e0198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:35.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.729+0000 7fb9e7929640 1 --2- 172.21.15.97:0/4022632932 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9e0102240 0x7fb9e0198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:35.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.729+0000 7fb9e7929640 1 -- 172.21.15.97:0/4022632932 >> 172.21.15.97:0/4022632932 conn(0x7fb9e00790e0 msgr2=0x7fb9e01002e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:33:35.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.729+0000 7fb9e7929640 1 -- 172.21.15.97:0/4022632932 shutdown_connections 2024-04-23T10:33:35.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.729+0000 7fb9e7929640 1 -- 172.21.15.97:0/4022632932 wait complete. 2024-04-23T10:33:35.740 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db / --timeout 60 --expiration 75.3 --await-for 42.0' 2024-04-23T10:33:35.740 DEBUG:teuthology.orchestra.run.smithi097:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24488 quiesce db / --timeout 60 --expiration 75.3 --await-for 42.0 2024-04-23T10:33:35.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.849+0000 7fdc222bb640 1 -- 172.21.15.97:0/3189815620 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc1c107280 msgr2=0x7fdc1c10ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:35.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.849+0000 7fdc222bb640 1 --2- 172.21.15.97:0/3189815620 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc1c107280 0x7fdc1c10ee50 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fdc14009ca0 tx=0x7fdc1402a8e0 comp rx=0 tx=0).stop 2024-04-23T10:33:35.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.849+0000 7fdc222bb640 1 -- 172.21.15.97:0/3189815620 shutdown_connections 2024-04-23T10:33:35.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.849+0000 7fdc222bb640 1 --2- 172.21.15.97:0/3189815620 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc1c107280 0x7fdc1c10ee50 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:35.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.849+0000 7fdc222bb640 1 --2- 172.21.15.97:0/3189815620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdc1c102bf0 0x7fdc1c106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:35.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.849+0000 7fdc222bb640 1 --2- 172.21.15.97:0/3189815620 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc1c102240 0x7fdc1c102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:35.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.849+0000 7fdc222bb640 1 -- 172.21.15.97:0/3189815620 >> 172.21.15.97:0/3189815620 conn(0x7fdc1c0790e0 msgr2=0x7fdc1c0794f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:33:35.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.849+0000 7fdc222bb640 1 -- 172.21.15.97:0/3189815620 shutdown_connections 2024-04-23T10:33:35.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.849+0000 7fdc222bb640 1 -- 172.21.15.97:0/3189815620 wait complete. 2024-04-23T10:33:35.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.850+0000 7fdc222bb640 1 Processor -- start 2024-04-23T10:33:35.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.850+0000 7fdc222bb640 1 -- start start 2024-04-23T10:33:35.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.850+0000 7fdc222bb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc1c102240 0x7fdc1c074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:35.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.850+0000 7fdc222bb640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdc1c107280 0x7fdc1c074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:35.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.850+0000 7fdc222bb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc1c074cd0 0x7fdc1c1a6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:35.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.850+0000 7fdc222bb640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1c112260 con 0x7fdc1c074cd0 2024-04-23T10:33:35.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.850+0000 7fdc222bb640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1c1120e0 con 0x7fdc1c107280 2024-04-23T10:33:35.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.850+0000 7fdc222bb640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1c1123e0 con 0x7fdc1c102240 2024-04-23T10:33:35.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.850+0000 7fdc20858640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc1c074cd0 0x7fdc1c1a6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:35.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.850+0000 7fdc20858640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc1c074cd0 0x7fdc1c1a6930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56388/0 (socket says 172.21.15.97:56388) 2024-04-23T10:33:35.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.850+0000 7fdc20858640 1 -- 172.21.15.97:0/359546645 learned_addr learned my addr 172.21.15.97:0/359546645 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:33:35.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.850+0000 7fdc1bfff640 1 --2- 172.21.15.97:0/359546645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc1c102240 0x7fdc1c074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:35.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.850+0000 7fdc1b7fe640 1 --2- 172.21.15.97:0/359546645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdc1c107280 0x7fdc1c074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:35.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.850+0000 7fdc20858640 1 -- 172.21.15.97:0/359546645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc1c102240 msgr2=0x7fdc1c074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:35.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.850+0000 7fdc20858640 1 --2- 172.21.15.97:0/359546645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc1c102240 0x7fdc1c074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:35.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.850+0000 7fdc20858640 1 -- 172.21.15.97:0/359546645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdc1c107280 msgr2=0x7fdc1c074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:35.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.850+0000 7fdc20858640 1 --2- 172.21.15.97:0/359546645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdc1c107280 0x7fdc1c074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:35.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.850+0000 7fdc20858640 1 -- 172.21.15.97:0/359546645 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc1c1a7030 con 0x7fdc1c074cd0 2024-04-23T10:33:35.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.851+0000 7fdc20858640 1 --2- 172.21.15.97:0/359546645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc1c074cd0 0x7fdc1c1a6930 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fdc14005cc0 tx=0x7fdc140039f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:33:35.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.851+0000 7fdc197fa640 1 -- 172.21.15.97:0/359546645 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc14003de0 con 0x7fdc1c074cd0 2024-04-23T10:33:35.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.851+0000 7fdc197fa640 1 -- 172.21.15.97:0/359546645 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc140044f0 con 0x7fdc1c074cd0 2024-04-23T10:33:35.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.851+0000 7fdc222bb640 1 -- 172.21.15.97:0/359546645 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc1c1a72c0 con 0x7fdc1c074cd0 2024-04-23T10:33:35.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.851+0000 7fdc197fa640 1 -- 172.21.15.97:0/359546645 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc14030d90 con 0x7fdc1c074cd0 2024-04-23T10:33:35.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.851+0000 7fdc222bb640 1 -- 172.21.15.97:0/359546645 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc1c1a7810 con 0x7fdc1c074cd0 2024-04-23T10:33:35.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.853+0000 7fdc197fa640 1 -- 172.21.15.97:0/359546645 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fdc14004040 con 0x7fdc1c074cd0 2024-04-23T10:33:35.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.853+0000 7fdc24fd0740 1 Processor -- start 2024-04-23T10:33:35.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.853+0000 7fdc24fd0740 1 -- start start 2024-04-23T10:33:35.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.854+0000 7fdc24fd0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560827d590a0 0x560827d59480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:35.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.854+0000 7fdc24fd0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560827d599c0 0x560827d63b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:35.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.854+0000 7fdc24fd0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560827d64090 0x560827d66480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:35.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.854+0000 7fdc24fd0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560827cbe370 con 0x560827d599c0 2024-04-23T10:33:35.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.854+0000 7fdc24fd0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560827c5b2e0 con 0x560827d64090 2024-04-23T10:33:35.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.854+0000 7fdc24fd0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560827cba330 con 0x560827d590a0 2024-04-23T10:33:35.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.854+0000 7fdc197fa640 1 --2- 172.21.15.97:0/359546645 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fdc040a3d50 0x7fdc040a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:35.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.854+0000 7fdc1bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560827d599c0 0x560827d63b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:35.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.854+0000 7fdc1bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560827d599c0 0x560827d63b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56402/0 (socket says 172.21.15.97:56402) 2024-04-23T10:33:35.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.854+0000 7fdc1bfff640 1 -- 172.21.15.97:0/3814845490 learned_addr learned my addr 172.21.15.97:0/3814845490 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:33:35.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.854+0000 7fdc20858640 1 --2- 172.21.15.97:0/359546645 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fdc040a3d50 0x7fdc040a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:35.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.854+0000 7fdc1b7fe640 1 --2- 172.21.15.97:0/3814845490 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560827d64090 0x560827d66480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:35.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.854+0000 7fdc1b7fe640 1 --2- 172.21.15.97:0/3814845490 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560827d590a0 0x560827d59480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:35.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.854+0000 7fdc197fa640 1 -- 172.21.15.97:0/359546645 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7fdc140f1aa0 con 0x7fdc1c074cd0 2024-04-23T10:33:35.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.854+0000 7fdc1bfff640 1 -- 172.21.15.97:0/3814845490 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560827d590a0 msgr2=0x560827d59480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:33:35.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.854+0000 7fdc1bfff640 1 --2- 172.21.15.97:0/3814845490 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560827d590a0 0x560827d59480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:35.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.854+0000 7fdc1bfff640 1 -- 172.21.15.97:0/3814845490 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560827d64090 msgr2=0x560827d66480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:33:35.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.854+0000 7fdc1bfff640 1 --2- 172.21.15.97:0/3814845490 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560827d64090 0x560827d66480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:35.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.854+0000 7fdc1bfff640 1 -- 172.21.15.97:0/3814845490 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560827a9cd80 con 0x560827d599c0 2024-04-23T10:33:35.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.854+0000 7fdc1b7fe640 1 --2- 172.21.15.97:0/3814845490 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560827d590a0 0x560827d59480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:33:35.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.854+0000 7fdc20858640 1 --2- 172.21.15.97:0/359546645 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fdc040a3d50 0x7fdc040a6210 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fdc140bdaa0 tx=0x7fdc14090600 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:33:35.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.854+0000 7fdc1bfff640 1 --2- 172.21.15.97:0/3814845490 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560827d599c0 0x560827d63b50 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c00b760 tx=0x7fdc0c00bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbd69b4a3d8322f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:33:35.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.855+0000 7fdc027fc640 1 -- 172.21.15.97:0/3814845490 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c003dd0 con 0x560827d599c0 2024-04-23T10:33:35.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.855+0000 7fdc027fc640 1 -- 172.21.15.97:0/3814845490 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc0c015070 con 0x560827d599c0 2024-04-23T10:33:35.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.855+0000 7fdc027fc640 1 -- 172.21.15.97:0/3814845490 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c004d90 con 0x560827d599c0 2024-04-23T10:33:35.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.855+0000 7fdc24fd0740 1 -- 172.21.15.97:0/3814845490 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560827d599c0 msgr2=0x560827d63b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:35.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.855+0000 7fdc24fd0740 1 --2- 172.21.15.97:0/3814845490 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560827d599c0 0x560827d63b50 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c00b760 tx=0x7fdc0c00bc20 comp rx=0 tx=0).stop 2024-04-23T10:33:35.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.855+0000 7fdc24fd0740 1 -- 172.21.15.97:0/3814845490 shutdown_connections 2024-04-23T10:33:35.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.855+0000 7fdc24fd0740 1 --2- 172.21.15.97:0/3814845490 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560827d64090 0x560827d66480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:35.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.855+0000 7fdc24fd0740 1 --2- 172.21.15.97:0/3814845490 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560827d599c0 0x560827d63b50 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:35.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.855+0000 7fdc24fd0740 1 --2- 172.21.15.97:0/3814845490 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560827d590a0 0x560827d59480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:35.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.855+0000 7fdc24fd0740 1 -- 172.21.15.97:0/3814845490 >> 172.21.15.97:0/3814845490 conn(0x560827d553c0 msgr2=0x560827d557a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:33:35.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.855+0000 7fdc24fd0740 1 -- 172.21.15.97:0/3814845490 shutdown_connections 2024-04-23T10:33:35.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.855+0000 7fdc24fd0740 1 -- 172.21.15.97:0/3814845490 wait complete. 2024-04-23T10:33:35.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.856+0000 7fdc24fd0740 1 -- start start 2024-04-23T10:33:35.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.856+0000 7fdc24fd0740 1 Processor -- start 2024-04-23T10:33:35.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.856+0000 7fdc24fd0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560827d590a0 0x560827d61030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:35.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.856+0000 7fdc24fd0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560827d599c0 0x560827d61570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:35.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.856+0000 7fdc24fd0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560827d64090 0x560827d61ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:35.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.856+0000 7fdc24fd0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560827d66e80 con 0x560827d590a0 2024-04-23T10:33:35.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.856+0000 7fdc24fd0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560827c58fb0 con 0x560827d64090 2024-04-23T10:33:35.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.856+0000 7fdc24fd0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560827d67000 con 0x560827d599c0 2024-04-23T10:33:35.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.856+0000 7fdc24fd0740 10 client.0 _ll_register_callbacks cb 0x560827c589a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:33:35.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.856+0000 7fdc1b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560827d590a0 0x560827d61030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:35.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.856+0000 7fdc1b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560827d590a0 0x560827d61030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56414/0 (socket says 172.21.15.97:56414) 2024-04-23T10:33:35.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.856+0000 7fdc1b7fe640 1 -- 172.21.15.97:0/1231148348 learned_addr learned my addr 172.21.15.97:0/1231148348 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:33:35.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.856+0000 7fdc1bfff640 1 --2- 172.21.15.97:0/1231148348 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560827d599c0 0x560827d61570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:35.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.856+0000 7fdc1b7fe640 1 -- 172.21.15.97:0/1231148348 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560827d599c0 msgr2=0x560827d61570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:33:35.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.856+0000 7fdc1b7fe640 1 --2- 172.21.15.97:0/1231148348 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560827d599c0 0x560827d61570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:35.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.856+0000 7fdc1b7fe640 1 -- 172.21.15.97:0/1231148348 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560827d64090 msgr2=0x560827d61ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:33:35.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.856+0000 7fdc1b7fe640 1 --2- 172.21.15.97:0/1231148348 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560827d64090 0x560827d61ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:35.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.856+0000 7fdc1b7fe640 1 -- 172.21.15.97:0/1231148348 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560827afd510 con 0x560827d590a0 2024-04-23T10:33:35.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.857+0000 7fdc1b7fe640 1 --2- 172.21.15.97:0/1231148348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560827d590a0 0x560827d61030 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fdc1000afd0 tx=0x7fdc100147e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95a743e7da1e0d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:33:35.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.857+0000 7fdbfb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:33:35.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.857+0000 7fdbfb7fe640 1 -- 172.21.15.97:0/1231148348 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc100134f0 con 0x560827d590a0 2024-04-23T10:33:35.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.857+0000 7fdbfb7fe640 1 -- 172.21.15.97:0/1231148348 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7fdc10013690 con 0x560827d590a0 2024-04-23T10:33:35.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.857+0000 7fdc24fd0740 1 -- 172.21.15.97:0/1231148348 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc0c015070 con 0x560827d590a0 2024-04-23T10:33:35.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.857+0000 7fdbfb7fe640 1 -- 172.21.15.97:0/1231148348 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdc10024050 con 0x560827d590a0 2024-04-23T10:33:35.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.857+0000 7fdc24fd0740 10 client.16587 fetch_fsmap learned FSMap version 26 2024-04-23T10:33:35.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.857+0000 7fdc24fd0740 1 -- 172.21.15.97:0/1231148348 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560827990670 con 0x560827d590a0 2024-04-23T10:33:35.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.857+0000 7fdbfb7fe640 1 -- 172.21.15.97:0/1231148348 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc0c015070 con 0x560827d590a0 2024-04-23T10:33:35.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.857+0000 7fdbfb7fe640 1 -- 172.21.15.97:0/1231148348 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fdc10015040 con 0x560827d590a0 2024-04-23T10:33:35.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.857+0000 7fdc24fd0740 10 client.16587 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:33:35.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.857+0000 7fdc24fd0740 10 client.16587 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:33:35.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.857+0000 7fdc24fd0740 20 client.16587 populate_metadata read hostname 'smithi097' 2024-04-23T10:33:35.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.857+0000 7fdc24fd0740 1 --2- 172.21.15.97:0/1231148348 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x560827d744d0 0x560827d768c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:35.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.857+0000 7fdc24fd0740 4 client.16587 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:33:35.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.857+0000 7fdc24fd0740 1 -- 172.21.15.97:0/1231148348 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560827ccbd90 con 0x560827d744d0 2024-04-23T10:33:35.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.858+0000 7fdc1bfff640 1 --2- 172.21.15.97:0/1231148348 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x560827d744d0 0x560827d768c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:35.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.858+0000 7fdbfb7fe640 10 client.16587 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:33:35.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.858+0000 7fdc1bfff640 1 --2- 172.21.15.97:0/1231148348 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x560827d744d0 0x560827d768c0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8c12e81c2a6c248 server_cookie=7492948634618233 in_seq=0 out_seq=0 2024-04-23T10:33:35.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.859+0000 7fdbfb7fe640 1 -- 172.21.15.97:0/1231148348 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560827ccbd90 con 0x560827d744d0 2024-04-23T10:33:35.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.859+0000 7fdbfb7fe640 10 client.16587 handle_command_reply: tid=0 2024-04-23T10:33:35.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.859+0000 7fdc24fd0740 1 client.16587 shutdown 2024-04-23T10:33:35.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.859+0000 7fdc24fd0740 1 -- 172.21.15.97:0/1231148348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560827d590a0 msgr2=0x560827d61030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:35.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.859+0000 7fdc24fd0740 1 --2- 172.21.15.97:0/1231148348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560827d590a0 0x560827d61030 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fdc1000afd0 tx=0x7fdc100147e0 comp rx=0 tx=0).stop 2024-04-23T10:33:35.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.859+0000 7fdc24fd0740 1 -- 172.21.15.97:0/1231148348 shutdown_connections 2024-04-23T10:33:35.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.859+0000 7fdc24fd0740 1 --2- 172.21.15.97:0/1231148348 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x560827d744d0 0x560827d768c0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:35.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.859+0000 7fdc24fd0740 1 --2- 172.21.15.97:0/1231148348 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560827d64090 0x560827d61ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:35.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.859+0000 7fdc24fd0740 1 --2- 172.21.15.97:0/1231148348 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560827d599c0 0x560827d61570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:35.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.859+0000 7fdc24fd0740 1 --2- 172.21.15.97:0/1231148348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560827d590a0 0x560827d61030 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:35.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.859+0000 7fdc24fd0740 1 -- 172.21.15.97:0/1231148348 >> 172.21.15.97:0/1231148348 conn(0x560827d553c0 msgr2=0x560827d5b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:33:35.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.859+0000 7fdbfb7fe640 0 client.16587 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:33:35.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.859+0000 7fdc24fd0740 1 -- 172.21.15.97:0/1231148348 shutdown_connections 2024-04-23T10:33:35.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.859+0000 7fdc24fd0740 1 -- 172.21.15.97:0/1231148348 wait complete. 2024-04-23T10:33:35.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.859+0000 7fdc24fd0740 20 client.16587 trim_cache size 0 max 16384 2024-04-23T10:33:35.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.876+0000 7fdc24fd0740 1 Processor -- start 2024-04-23T10:33:35.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.876+0000 7fdc24fd0740 1 -- start start 2024-04-23T10:33:35.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.876+0000 7fdc24fd0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560827d590a0 0x560827d5c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:35.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.876+0000 7fdc24fd0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560827d599c0 0x560827d54d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:35.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.876+0000 7fdc24fd0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560827d64090 0x560827d5b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:35.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.876+0000 7fdc24fd0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1c1afd20 con 0x560827d599c0 2024-04-23T10:33:35.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.876+0000 7fdc24fd0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1c1b1860 con 0x560827d64090 2024-04-23T10:33:35.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.876+0000 7fdc24fd0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1c1b03f0 con 0x560827d590a0 2024-04-23T10:33:35.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.876+0000 7fdc1bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560827d599c0 0x560827d54d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:35.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.876+0000 7fdc1bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560827d599c0 0x560827d54d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56426/0 (socket says 172.21.15.97:56426) 2024-04-23T10:33:35.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.876+0000 7fdc1bfff640 1 -- 172.21.15.97:0/3757612367 learned_addr learned my addr 172.21.15.97:0/3757612367 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:33:35.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.876+0000 7fdc1b7fe640 1 --2- 172.21.15.97:0/3757612367 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560827d590a0 0x560827d5c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:35.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.876+0000 7fdc1bfff640 1 -- 172.21.15.97:0/3757612367 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560827d590a0 msgr2=0x560827d5c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:33:35.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.876+0000 7fdc1bfff640 1 --2- 172.21.15.97:0/3757612367 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560827d590a0 0x560827d5c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:35.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.876+0000 7fdc1b7fe640 1 --2- 172.21.15.97:0/3757612367 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560827d64090 0x560827d5b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:35.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.876+0000 7fdc1bfff640 1 -- 172.21.15.97:0/3757612367 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560827d64090 msgr2=0x560827d5b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:33:35.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.876+0000 7fdc1bfff640 1 --2- 172.21.15.97:0/3757612367 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560827d64090 0x560827d5b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:35.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.876+0000 7fdc1bfff640 1 -- 172.21.15.97:0/3757612367 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560827d597f0 con 0x560827d599c0 2024-04-23T10:33:35.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.877+0000 7fdc1bfff640 1 --2- 172.21.15.97:0/3757612367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560827d599c0 0x560827d54d80 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c00b760 tx=0x7fdc0c004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=229bb53faee31c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:33:35.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.877+0000 7fdc01ffb640 1 -- 172.21.15.97:0/3757612367 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c02c070 con 0x560827d599c0 2024-04-23T10:33:35.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.877+0000 7fdc01ffb640 1 -- 172.21.15.97:0/3757612367 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc0c015030 con 0x560827d599c0 2024-04-23T10:33:35.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.877+0000 7fdc01ffb640 1 -- 172.21.15.97:0/3757612367 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c011440 con 0x560827d599c0 2024-04-23T10:33:35.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.877+0000 7fdc24fd0740 1 -- 172.21.15.97:0/3757612367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560827d599c0 msgr2=0x560827d54d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:35.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.877+0000 7fdc24fd0740 1 --2- 172.21.15.97:0/3757612367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560827d599c0 0x560827d54d80 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c00b760 tx=0x7fdc0c004970 comp rx=0 tx=0).stop 2024-04-23T10:33:35.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.877+0000 7fdc24fd0740 1 -- 172.21.15.97:0/3757612367 shutdown_connections 2024-04-23T10:33:35.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.877+0000 7fdc24fd0740 1 --2- 172.21.15.97:0/3757612367 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560827d64090 0x560827d5b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:35.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.877+0000 7fdc24fd0740 1 --2- 172.21.15.97:0/3757612367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560827d599c0 0x560827d54d80 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:35.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.877+0000 7fdc24fd0740 1 --2- 172.21.15.97:0/3757612367 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560827d590a0 0x560827d5c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:35.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.877+0000 7fdc24fd0740 1 -- 172.21.15.97:0/3757612367 >> 172.21.15.97:0/3757612367 conn(0x560827d553c0 msgr2=0x560827d62860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:33:35.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.877+0000 7fdc24fd0740 1 -- 172.21.15.97:0/3757612367 shutdown_connections 2024-04-23T10:33:35.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.877+0000 7fdc24fd0740 1 -- 172.21.15.97:0/3757612367 wait complete. 2024-04-23T10:33:35.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.878+0000 7fdc24fd0740 1 -- start start 2024-04-23T10:33:35.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.878+0000 7fdc24fd0740 1 Processor -- start 2024-04-23T10:33:35.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.878+0000 7fdc24fd0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560827d590a0 0x560827d5def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:35.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.878+0000 7fdc24fd0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560827d599c0 0x560827d5e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:35.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.878+0000 7fdc24fd0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560827d64090 0x560827d5e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:35.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.878+0000 7fdc24fd0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1c1b1190 con 0x560827d599c0 2024-04-23T10:33:35.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.878+0000 7fdc24fd0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1c1b0ac0 con 0x560827d590a0 2024-04-23T10:33:35.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.878+0000 7fdc24fd0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1c1add20 con 0x560827d64090 2024-04-23T10:33:35.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.878+0000 7fdc24fd0740 10 client.0 _ll_register_callbacks cb 0x7fdc1000ac60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:33:35.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.878+0000 7fdc1bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560827d599c0 0x560827d5e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:35.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.878+0000 7fdc1bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560827d599c0 0x560827d5e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56438/0 (socket says 172.21.15.97:56438) 2024-04-23T10:33:35.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.878+0000 7fdc1bfff640 1 -- 172.21.15.97:0/2542218155 learned_addr learned my addr 172.21.15.97:0/2542218155 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:33:35.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.878+0000 7fdc1b7fe640 1 --2- 172.21.15.97:0/2542218155 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560827d64090 0x560827d5e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:35.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.879+0000 7fdc1b7fe640 1 --2- 172.21.15.97:0/2542218155 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560827d590a0 0x560827d5def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:35.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.879+0000 7fdc1bfff640 1 -- 172.21.15.97:0/2542218155 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560827d64090 msgr2=0x560827d5e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:33:35.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.879+0000 7fdc1bfff640 1 --2- 172.21.15.97:0/2542218155 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560827d64090 0x560827d5e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:35.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.879+0000 7fdc1bfff640 1 -- 172.21.15.97:0/2542218155 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560827d590a0 msgr2=0x560827d5def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:33:35.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.879+0000 7fdc1bfff640 1 --2- 172.21.15.97:0/2542218155 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560827d590a0 0x560827d5def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:35.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.879+0000 7fdc1bfff640 1 -- 172.21.15.97:0/2542218155 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560827dd9c00 con 0x560827d599c0 2024-04-23T10:33:35.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.879+0000 7fdc1b7fe640 1 --2- 172.21.15.97:0/2542218155 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560827d590a0 0x560827d5def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:33:35.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.879+0000 7fdc1bfff640 1 --2- 172.21.15.97:0/2542218155 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560827d599c0 0x560827d5e430 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c006010 tx=0x7fdc0c0042d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c42de1b62feb247 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:33:35.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.879+0000 7fdbfb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:33:35.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.879+0000 7fdbfb7fe640 1 -- 172.21.15.97:0/2542218155 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c035060 con 0x560827d599c0 2024-04-23T10:33:35.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.879+0000 7fdbfb7fe640 1 -- 172.21.15.97:0/2542218155 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7fdc0c0278b0 con 0x560827d599c0 2024-04-23T10:33:35.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.879+0000 7fdc24fd0740 1 -- 172.21.15.97:0/2542218155 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc0c015030 con 0x560827d599c0 2024-04-23T10:33:35.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.879+0000 7fdbfb7fe640 1 -- 172.21.15.97:0/2542218155 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdc0c02c040 con 0x560827d599c0 2024-04-23T10:33:35.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.879+0000 7fdc24fd0740 10 client.16593 fetch_fsmap learned FSMap version 26 2024-04-23T10:33:35.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.879+0000 7fdc24fd0740 1 -- 172.21.15.97:0/2542218155 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560827ddaea0 con 0x560827d599c0 2024-04-23T10:33:35.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.879+0000 7fdbfb7fe640 1 -- 172.21.15.97:0/2542218155 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc0c015030 con 0x560827d599c0 2024-04-23T10:33:35.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.879+0000 7fdbfb7fe640 1 -- 172.21.15.97:0/2542218155 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fdc0c0396b0 con 0x560827d599c0 2024-04-23T10:33:35.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.879+0000 7fdc24fd0740 10 client.16593 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:33:35.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.879+0000 7fdc24fd0740 10 client.16593 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:33:35.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.879+0000 7fdc24fd0740 20 client.16593 populate_metadata read hostname 'smithi097' 2024-04-23T10:33:35.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.879+0000 7fdc24fd0740 1 --2- 172.21.15.97:0/2542218155 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x560827ddb4a0 0x560827ddd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:35.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.879+0000 7fdc24fd0740 4 client.16593 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 75.3, "await_for": 42.0}] 2024-04-23T10:33:35.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.879+0000 7fdc24fd0740 1 -- 172.21.15.97:0/2542218155 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 75.3, "await_for": 42.0}) -- 0x560827cc5c00 con 0x560827ddb4a0 2024-04-23T10:33:35.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.880+0000 7fdc1b7fe640 1 --2- 172.21.15.97:0/2542218155 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x560827ddb4a0 0x560827ddd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:35.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.880+0000 7fdbfb7fe640 10 client.16593 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:33:35.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.880+0000 7fdc1b7fe640 1 --2- 172.21.15.97:0/2542218155 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x560827ddb4a0 0x560827ddd8c0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f52ec0d84c3655b7 server_cookie=af182b7bfb3fc6b in_seq=0 out_seq=0 2024-04-23T10:33:35.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.893+0000 7fdbfb7fe640 1 -- 172.21.15.97:0/2542218155 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x560827cc5c00 con 0x560827ddb4a0 2024-04-23T10:33:35.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.893+0000 7fdbfb7fe640 10 client.16593 handle_command_reply: tid=0 2024-04-23T10:33:35.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.893+0000 7fdc24fd0740 1 client.16593 shutdown 2024-04-23T10:33:35.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.893+0000 7fdc24fd0740 1 -- 172.21.15.97:0/2542218155 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560827d599c0 msgr2=0x560827d5e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:35.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.893+0000 7fdc24fd0740 1 --2- 172.21.15.97:0/2542218155 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560827d599c0 0x560827d5e430 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c006010 tx=0x7fdc0c0042d0 comp rx=0 tx=0).stop 2024-04-23T10:33:35.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.893+0000 7fdc24fd0740 1 -- 172.21.15.97:0/2542218155 shutdown_connections 2024-04-23T10:33:35.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.893+0000 7fdc24fd0740 1 --2- 172.21.15.97:0/2542218155 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x560827ddb4a0 0x560827ddd8c0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:35.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.893+0000 7fdc24fd0740 1 --2- 172.21.15.97:0/2542218155 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560827d64090 0x560827d5e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:35.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.893+0000 7fdc24fd0740 1 --2- 172.21.15.97:0/2542218155 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560827d599c0 0x560827d5e430 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:35.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.893+0000 7fdc24fd0740 1 --2- 172.21.15.97:0/2542218155 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560827d590a0 0x560827d5def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:35.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.893+0000 7fdbfb7fe640 0 client.16593 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:33:35.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.893+0000 7fdc24fd0740 1 -- 172.21.15.97:0/2542218155 >> 172.21.15.97:0/2542218155 conn(0x560827d553c0 msgr2=0x560827d630f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:33:35.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.894+0000 7fdc24fd0740 1 -- 172.21.15.97:0/2542218155 shutdown_connections 2024-04-23T10:33:35.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.894+0000 7fdc24fd0740 1 -- 172.21.15.97:0/2542218155 wait complete. 2024-04-23T10:33:35.893 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.894+0000 7fdc24fd0740 20 client.16593 trim_cache size 0 max 16384 2024-04-23T10:33:35.894 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-23T10:33:35.894 INFO:teuthology.orchestra.run.smithi097.stdout: "epoch": 26, 2024-04-23T10:33:35.894 INFO:teuthology.orchestra.run.smithi097.stdout: "leader": 24488, 2024-04-23T10:33:35.894 INFO:teuthology.orchestra.run.smithi097.stdout: "set_version": 26, 2024-04-23T10:33:35.894 INFO:teuthology.orchestra.run.smithi097.stdout: "sets": { 2024-04-23T10:33:35.894 INFO:teuthology.orchestra.run.smithi097.stdout: "af277cb5": { 2024-04-23T10:33:35.894 INFO:teuthology.orchestra.run.smithi097.stdout: "version": 26, 2024-04-23T10:33:35.894 INFO:teuthology.orchestra.run.smithi097.stdout: "age_ref": 0.0, 2024-04-23T10:33:35.894 INFO:teuthology.orchestra.run.smithi097.stdout: "state": { 2024-04-23T10:33:35.894 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "QUIESCED", 2024-04-23T10:33:35.894 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0 2024-04-23T10:33:35.895 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:33:35.895 INFO:teuthology.orchestra.run.smithi097.stdout: "timeout": 60.0, 2024-04-23T10:33:35.895 INFO:teuthology.orchestra.run.smithi097.stdout: "expiration": 75.3, 2024-04-23T10:33:35.895 INFO:teuthology.orchestra.run.smithi097.stdout: "members": { 2024-04-23T10:33:35.895 INFO:teuthology.orchestra.run.smithi097.stdout: "file:/": { 2024-04-23T10:33:35.895 INFO:teuthology.orchestra.run.smithi097.stdout: "excluded": false, 2024-04-23T10:33:35.895 INFO:teuthology.orchestra.run.smithi097.stdout: "state": { 2024-04-23T10:33:35.895 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "QUIESCED", 2024-04-23T10:33:35.895 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0 2024-04-23T10:33:35.895 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:33:35.895 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:33:35.895 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:33:35.895 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:33:35.895 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:33:35.895 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-23T10:33:35.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.894+0000 7fdc222bb640 1 -- 172.21.15.97:0/359546645 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fdc040a3d50 msgr2=0x7fdc040a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:35.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.894+0000 7fdc222bb640 1 --2- 172.21.15.97:0/359546645 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fdc040a3d50 0x7fdc040a6210 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fdc140bdaa0 tx=0x7fdc14090600 comp rx=0 tx=0).stop 2024-04-23T10:33:35.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.894+0000 7fdc222bb640 1 -- 172.21.15.97:0/359546645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc1c074cd0 msgr2=0x7fdc1c1a6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:35.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.894+0000 7fdc222bb640 1 --2- 172.21.15.97:0/359546645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc1c074cd0 0x7fdc1c1a6930 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fdc14005cc0 tx=0x7fdc140039f0 comp rx=0 tx=0).stop 2024-04-23T10:33:35.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.894+0000 7fdc222bb640 1 -- 172.21.15.97:0/359546645 shutdown_connections 2024-04-23T10:33:35.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.894+0000 7fdc222bb640 1 --2- 172.21.15.97:0/359546645 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fdc040a3d50 0x7fdc040a6210 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:35.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.894+0000 7fdc222bb640 1 --2- 172.21.15.97:0/359546645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc1c074cd0 0x7fdc1c1a6930 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:35.896 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.894+0000 7fdc222bb640 1 --2- 172.21.15.97:0/359546645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdc1c107280 0x7fdc1c074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:35.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.894+0000 7fdc222bb640 1 --2- 172.21.15.97:0/359546645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc1c102240 0x7fdc1c074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:35.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.894+0000 7fdc222bb640 1 -- 172.21.15.97:0/359546645 >> 172.21.15.97:0/359546645 conn(0x7fdc1c0790e0 msgr2=0x7fdc1c100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:33:35.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.894+0000 7fdc222bb640 1 -- 172.21.15.97:0/359546645 shutdown_connections 2024-04-23T10:33:35.897 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:35.894+0000 7fdc222bb640 1 -- 172.21.15.97:0/359546645 wait complete. 2024-04-23T10:33:35.903 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'af277cb5', quiesce took 0.5 seconds. Will release after: 15.3 2024-04-23T10:33:36.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:33:36 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/4022632932' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:33:36.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:33:36 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/4022632932' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:33:36.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:33:36 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/4022632932' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:33:37.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:33:37 smithi183 ceph-mon[32317]: pgmap v227: 97 pgs: 97 active+clean; 1.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:33:37.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:33:37 smithi196 ceph-mon[32919]: pgmap v227: 97 pgs: 97 active+clean; 1.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:33:37.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:33:37 smithi097 ceph-mon[29197]: pgmap v227: 97 pgs: 97 active+clean; 1.6 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:33:39.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:33:39 smithi183 ceph-mon[32317]: pgmap v228: 97 pgs: 97 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 22 op/s 2024-04-23T10:33:39.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:33:39 smithi196 ceph-mon[32919]: pgmap v228: 97 pgs: 97 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 22 op/s 2024-04-23T10:33:39.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:33:39 smithi097 ceph-mon[29197]: pgmap v228: 97 pgs: 97 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 22 op/s 2024-04-23T10:33:40.563 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.561+0000 7ffa0f1a2640 1 -- 172.21.15.97:0/3688974165 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa081021d0 msgr2=0x7ffa081025b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:40.563 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.561+0000 7ffa0f1a2640 1 --2- 172.21.15.97:0/3688974165 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa081021d0 0x7ffa081025b0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc009900 tx=0x7ff9fc02a410 comp rx=0 tx=0).stop 2024-04-23T10:33:40.563 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.561+0000 7ffa0f1a2640 1 -- 172.21.15.97:0/3688974165 shutdown_connections 2024-04-23T10:33:40.563 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.561+0000 7ffa0f1a2640 1 --2- 172.21.15.97:0/3688974165 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa081072a0 0x7ffa0810ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:40.563 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.561+0000 7ffa0f1a2640 1 --2- 172.21.15.97:0/3688974165 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa08102b80 0x7ffa08106bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:40.563 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.561+0000 7ffa0f1a2640 1 --2- 172.21.15.97:0/3688974165 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa081021d0 0x7ffa081025b0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:40.563 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.561+0000 7ffa0f1a2640 1 -- 172.21.15.97:0/3688974165 >> 172.21.15.97:0/3688974165 conn(0x7ffa080793e0 msgr2=0x7ffa080797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:33:40.563 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.561+0000 7ffa0f1a2640 1 -- 172.21.15.97:0/3688974165 shutdown_connections 2024-04-23T10:33:40.563 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.561+0000 7ffa0f1a2640 1 -- 172.21.15.97:0/3688974165 wait complete. 2024-04-23T10:33:40.563 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.562+0000 7ffa0f1a2640 1 Processor -- start 2024-04-23T10:33:40.563 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.562+0000 7ffa0f1a2640 1 -- start start 2024-04-23T10:33:40.563 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.562+0000 7ffa0f1a2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa081021d0 0x7ffa081986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.562+0000 7ffa0f1a2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa081072a0 0x7ffa08198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.562+0000 7ffa0f1a2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa0819cf70 0x7ffa0819d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.562+0000 7ffa0f1a2640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ffa08112210 con 0x7ffa0819cf70 2024-04-23T10:33:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.562+0000 7ffa0f1a2640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ffa08112090 con 0x7ffa081021d0 2024-04-23T10:33:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.562+0000 7ffa0f1a2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ffa08112390 con 0x7ffa081072a0 2024-04-23T10:33:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.562+0000 7ffa0d73f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa0819cf70 0x7ffa0819d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.562+0000 7ffa0d73f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa0819cf70 0x7ffa0819d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56454/0 (socket says 172.21.15.97:56454) 2024-04-23T10:33:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.563+0000 7ffa0d73f640 1 -- 172.21.15.97:0/530777862 learned_addr learned my addr 172.21.15.97:0/530777862 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:33:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.563+0000 7ffa07fff640 1 --2- 172.21.15.97:0/530777862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa081072a0 0x7ffa08198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.563+0000 7ffa0cf3e640 1 --2- 172.21.15.97:0/530777862 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa081021d0 0x7ffa081986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.563+0000 7ffa0d73f640 1 -- 172.21.15.97:0/530777862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa081072a0 msgr2=0x7ffa08198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.563+0000 7ffa0d73f640 1 --2- 172.21.15.97:0/530777862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa081072a0 0x7ffa08198bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.563+0000 7ffa0d73f640 1 -- 172.21.15.97:0/530777862 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa081021d0 msgr2=0x7ffa081986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.563+0000 7ffa0d73f640 1 --2- 172.21.15.97:0/530777862 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa081021d0 0x7ffa081986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.563+0000 7ffa0d73f640 1 -- 172.21.15.97:0/530777862 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa0819da60 con 0x7ffa0819cf70 2024-04-23T10:33:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.563+0000 7ffa0d73f640 1 --2- 172.21.15.97:0/530777862 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa0819cf70 0x7ffa0819d420 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7ffa0000ea60 tx=0x7ffa0000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:33:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.563+0000 7ffa05ffb640 1 -- 172.21.15.97:0/530777862 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0000ce40 con 0x7ffa0819cf70 2024-04-23T10:33:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.563+0000 7ffa05ffb640 1 -- 172.21.15.97:0/530777862 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa00005020 con 0x7ffa0819cf70 2024-04-23T10:33:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.563+0000 7ffa0f1a2640 1 -- 172.21.15.97:0/530777862 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa0819dd50 con 0x7ffa0819cf70 2024-04-23T10:33:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.563+0000 7ffa05ffb640 1 -- 172.21.15.97:0/530777862 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa00010670 con 0x7ffa0819cf70 2024-04-23T10:33:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.563+0000 7ffa0f1a2640 1 -- 172.21.15.97:0/530777862 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa081a5540 con 0x7ffa0819cf70 2024-04-23T10:33:40.572 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.565+0000 7ffa05ffb640 1 -- 172.21.15.97:0/530777862 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ffa000054b0 con 0x7ffa0819cf70 2024-04-23T10:33:40.572 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.566+0000 7ffa11f96740 1 Processor -- start 2024-04-23T10:33:40.572 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.566+0000 7ffa05ffb640 1 --2- 172.21.15.97:0/530777862 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7ff9f80a3dc0 0x7ff9f80a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:40.572 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.566+0000 7ffa11f96740 1 -- start start 2024-04-23T10:33:40.572 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.566+0000 7ffa07fff640 1 --2- 172.21.15.97:0/530777862 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7ff9f80a3dc0 0x7ff9f80a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:40.572 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.566+0000 7ffa11f96740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560c91b44520 0x560c91b44900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:40.573 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.566+0000 7ffa11f96740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c91b44e40 0x560c91b536d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:40.573 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.566+0000 7ffa05ffb640 1 -- 172.21.15.97:0/530777862 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7ffa000d0c00 con 0x7ffa0819cf70 2024-04-23T10:33:40.573 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.566+0000 7ffa11f96740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c91b53c10 0x560c91b56000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:40.573 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.566+0000 7ffa11f96740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560c91aa9970 con 0x560c91b44e40 2024-04-23T10:33:40.573 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.566+0000 7ffa11f96740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560c91a464a0 con 0x560c91b44520 2024-04-23T10:33:40.573 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.566+0000 7ffa11f96740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560c91aa5930 con 0x560c91b53c10 2024-04-23T10:33:40.573 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.566+0000 7ffa07fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c91b44e40 0x560c91b536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:40.573 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.566+0000 7ffa07fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c91b44e40 0x560c91b536d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56456/0 (socket says 172.21.15.97:56456) 2024-04-23T10:33:40.573 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.566+0000 7ffa07fff640 1 -- 172.21.15.97:0/545220426 learned_addr learned my addr 172.21.15.97:0/545220426 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:33:40.573 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.566+0000 7ffa0cf3e640 1 --2- 172.21.15.97:0/545220426 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560c91b44520 0x560c91b44900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:40.573 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.567+0000 7ffa0d73f640 1 --2- 172.21.15.97:0/545220426 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c91b53c10 0x560c91b56000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:40.573 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.567+0000 7ffa07fff640 1 --2- 172.21.15.97:0/530777862 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7ff9f80a3dc0 0x7ff9f80a6280 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4012500 tx=0x7ff9f4012900 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:33:40.573 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.567+0000 7ffa07fff640 1 -- 172.21.15.97:0/545220426 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c91b53c10 msgr2=0x560c91b56000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:33:40.573 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.567+0000 7ffa07fff640 1 --2- 172.21.15.97:0/545220426 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c91b53c10 0x560c91b56000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:40.573 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.567+0000 7ffa07fff640 1 -- 172.21.15.97:0/545220426 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560c91b44520 msgr2=0x560c91b44900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:33:40.574 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.567+0000 7ffa07fff640 1 --2- 172.21.15.97:0/545220426 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560c91b44520 0x560c91b44900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:40.574 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.567+0000 7ffa07fff640 1 -- 172.21.15.97:0/545220426 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c917bc210 con 0x560c91b44e40 2024-04-23T10:33:40.574 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.567+0000 7ffa0cf3e640 1 --2- 172.21.15.97:0/545220426 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560c91b44520 0x560c91b44900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T10:33:40.574 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.567+0000 7ffa07fff640 1 --2- 172.21.15.97:0/545220426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c91b44e40 0x560c91b536d0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7ff9f400e7d0 tx=0x7ff9f401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5384c891173f7dc server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:33:40.574 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.567+0000 7ff9f2ffd640 1 -- 172.21.15.97:0/545220426 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f400ee10 con 0x560c91b44e40 2024-04-23T10:33:40.574 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.567+0000 7ff9f2ffd640 1 -- 172.21.15.97:0/545220426 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9f4023020 con 0x560c91b44e40 2024-04-23T10:33:40.574 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.567+0000 7ff9f2ffd640 1 -- 172.21.15.97:0/545220426 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f401fad0 con 0x560c91b44e40 2024-04-23T10:33:40.574 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.568+0000 7ffa11f96740 1 -- 172.21.15.97:0/545220426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c91b44e40 msgr2=0x560c91b536d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:40.574 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.568+0000 7ffa11f96740 1 --2- 172.21.15.97:0/545220426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c91b44e40 0x560c91b536d0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7ff9f400e7d0 tx=0x7ff9f401e000 comp rx=0 tx=0).stop 2024-04-23T10:33:40.574 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.568+0000 7ffa11f96740 1 -- 172.21.15.97:0/545220426 shutdown_connections 2024-04-23T10:33:40.574 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.568+0000 7ffa11f96740 1 --2- 172.21.15.97:0/545220426 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c91b53c10 0x560c91b56000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:40.574 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.568+0000 7ffa11f96740 1 --2- 172.21.15.97:0/545220426 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c91b44e40 0x560c91b536d0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:40.574 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.568+0000 7ffa11f96740 1 --2- 172.21.15.97:0/545220426 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560c91b44520 0x560c91b44900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:40.574 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.568+0000 7ffa11f96740 1 -- 172.21.15.97:0/545220426 >> 172.21.15.97:0/545220426 conn(0x560c91b40b30 msgr2=0x560c91b40f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:33:40.574 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.568+0000 7ffa11f96740 1 -- 172.21.15.97:0/545220426 shutdown_connections 2024-04-23T10:33:40.574 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.568+0000 7ffa11f96740 1 -- 172.21.15.97:0/545220426 wait complete. 2024-04-23T10:33:40.575 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.568+0000 7ffa11f96740 1 -- start start 2024-04-23T10:33:40.575 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.568+0000 7ffa11f96740 1 Processor -- start 2024-04-23T10:33:40.575 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.569+0000 7ffa11f96740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c91b44520 0x560c91b535b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:40.575 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.569+0000 7ffa11f96740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560c91b44e40 0x560c91b508a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:40.575 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.569+0000 7ffa11f96740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c91b53c10 0x560c91b50e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:40.575 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.569+0000 7ffa11f96740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x560c91b567e0 con 0x560c91b44520 2024-04-23T10:33:40.575 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.569+0000 7ffa11f96740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560c91a440b0 con 0x560c91b44e40 2024-04-23T10:33:40.575 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.569+0000 7ffa11f96740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x560c91b56960 con 0x560c91b53c10 2024-04-23T10:33:40.575 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.569+0000 7ffa11f96740 10 client.0 _ll_register_callbacks cb 0x560c91a31d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:33:40.575 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.569+0000 7ffa0cf3e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c91b44520 0x560c91b535b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:40.575 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.569+0000 7ffa0cf3e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c91b44520 0x560c91b535b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56460/0 (socket says 172.21.15.97:56460) 2024-04-23T10:33:40.575 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.569+0000 7ffa0cf3e640 1 -- 172.21.15.97:0/689263744 learned_addr learned my addr 172.21.15.97:0/689263744 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:33:40.575 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.569+0000 7ffa0d73f640 1 --2- 172.21.15.97:0/689263744 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c91b53c10 0x560c91b50e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:40.576 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.569+0000 7ffa07fff640 1 --2- 172.21.15.97:0/689263744 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560c91b44e40 0x560c91b508a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:40.576 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.569+0000 7ffa0cf3e640 1 -- 172.21.15.97:0/689263744 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c91b53c10 msgr2=0x560c91b50e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:33:40.576 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.569+0000 7ffa0cf3e640 1 --2- 172.21.15.97:0/689263744 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c91b53c10 0x560c91b50e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:40.576 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.569+0000 7ffa0cf3e640 1 -- 172.21.15.97:0/689263744 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560c91b44e40 msgr2=0x560c91b508a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:33:40.576 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.569+0000 7ffa0cf3e640 1 --2- 172.21.15.97:0/689263744 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560c91b44e40 0x560c91b508a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:40.576 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.569+0000 7ffa0cf3e640 1 -- 172.21.15.97:0/689263744 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c91888620 con 0x560c91b44520 2024-04-23T10:33:40.576 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.569+0000 7ffa07fff640 1 --2- 172.21.15.97:0/689263744 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560c91b44e40 0x560c91b508a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:33:40.576 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.569+0000 7ffa0cf3e640 1 --2- 172.21.15.97:0/689263744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c91b44520 0x560c91b535b0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc009900 tx=0x7ff9fc0314a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba6a3722bceda586 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:33:40.576 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.569+0000 7ff9e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:33:40.576 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.569+0000 7ff9e7fff640 1 -- 172.21.15.97:0/689263744 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc035040 con 0x560c91b44520 2024-04-23T10:33:40.576 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.570+0000 7ff9e7fff640 1 -- 172.21.15.97:0/689263744 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7ff9fc031810 con 0x560c91b44520 2024-04-23T10:33:40.576 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.570+0000 7ffa11f96740 1 -- 172.21.15.97:0/689263744 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9f4023020 con 0x560c91b44520 2024-04-23T10:33:40.576 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.570+0000 7ff9e7fff640 1 -- 172.21.15.97:0/689263744 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff9fc004040 con 0x560c91b44520 2024-04-23T10:33:40.576 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.570+0000 7ffa11f96740 10 client.16605 fetch_fsmap learned FSMap version 26 2024-04-23T10:33:40.576 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.570+0000 7ffa11f96740 1 -- 172.21.15.97:0/689263744 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560c91b627d0 con 0x560c91b44520 2024-04-23T10:33:40.577 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.570+0000 7ff9e7fff640 1 -- 172.21.15.97:0/689263744 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9f4023020 con 0x560c91b44520 2024-04-23T10:33:40.577 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.570+0000 7ff9e7fff640 1 -- 172.21.15.97:0/689263744 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff9fc04d020 con 0x560c91b44520 2024-04-23T10:33:40.577 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.570+0000 7ffa11f96740 10 client.16605 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:33:40.577 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.570+0000 7ffa11f96740 10 client.16605 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:33:40.577 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.570+0000 7ffa11f96740 20 client.16605 populate_metadata read hostname 'smithi097' 2024-04-23T10:33:40.577 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.570+0000 7ffa11f96740 1 --2- 172.21.15.97:0/689263744 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x560c91b63c40 0x560c91b66070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:40.577 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.570+0000 7ffa11f96740 4 client.16605 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:33:40.577 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.570+0000 7ffa11f96740 1 -- 172.21.15.97:0/689263744 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c91ab7390 con 0x560c91b63c40 2024-04-23T10:33:40.577 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.571+0000 7ffa07fff640 1 --2- 172.21.15.97:0/689263744 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x560c91b63c40 0x560c91b66070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:40.577 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.571+0000 7ff9e7fff640 10 client.16605 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:33:40.577 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.571+0000 7ffa07fff640 1 --2- 172.21.15.97:0/689263744 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x560c91b63c40 0x560c91b66070 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77394838c685ce04 server_cookie=4584411ec58e7aa in_seq=0 out_seq=0 2024-04-23T10:33:40.577 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.572+0000 7ff9e7fff640 1 -- 172.21.15.97:0/689263744 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560c91ab7390 con 0x560c91b63c40 2024-04-23T10:33:40.577 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.572+0000 7ff9e7fff640 10 client.16605 handle_command_reply: tid=0 2024-04-23T10:33:40.577 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.572+0000 7ffa11f96740 1 client.16605 shutdown 2024-04-23T10:33:40.577 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.572+0000 7ffa11f96740 1 -- 172.21.15.97:0/689263744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c91b44520 msgr2=0x560c91b535b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:40.577 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.572+0000 7ffa11f96740 1 --2- 172.21.15.97:0/689263744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c91b44520 0x560c91b535b0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc009900 tx=0x7ff9fc0314a0 comp rx=0 tx=0).stop 2024-04-23T10:33:40.578 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.572+0000 7ffa11f96740 1 -- 172.21.15.97:0/689263744 shutdown_connections 2024-04-23T10:33:40.578 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.572+0000 7ffa11f96740 1 --2- 172.21.15.97:0/689263744 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x560c91b63c40 0x560c91b66070 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:40.578 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.572+0000 7ffa11f96740 1 --2- 172.21.15.97:0/689263744 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c91b53c10 0x560c91b50e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:40.578 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.572+0000 7ff9e7fff640 0 client.16605 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:33:40.578 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.572+0000 7ffa11f96740 1 --2- 172.21.15.97:0/689263744 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560c91b44e40 0x560c91b508a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:40.578 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.572+0000 7ffa11f96740 1 --2- 172.21.15.97:0/689263744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c91b44520 0x560c91b535b0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:40.578 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.572+0000 7ffa11f96740 1 -- 172.21.15.97:0/689263744 >> 172.21.15.97:0/689263744 conn(0x560c91b40b30 msgr2=0x560c91b469a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:33:40.578 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.572+0000 7ffa11f96740 1 -- 172.21.15.97:0/689263744 shutdown_connections 2024-04-23T10:33:40.578 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.572+0000 7ffa11f96740 1 -- 172.21.15.97:0/689263744 wait complete. 2024-04-23T10:33:40.578 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.572+0000 7ffa11f96740 20 client.16605 trim_cache size 0 max 16384 2024-04-23T10:33:40.587 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.587+0000 7ffa11f96740 1 Processor -- start 2024-04-23T10:33:40.587 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.587+0000 7ffa11f96740 1 -- start start 2024-04-23T10:33:40.587 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.587+0000 7ffa11f96740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c91b44520 0x560c91b5f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:40.588 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.587+0000 7ffa11f96740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560c91b44e40 0x560c91b40f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:40.588 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.587+0000 7ffa11f96740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c91b53c10 0x560c91b530e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:40.588 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.587+0000 7ffa11f96740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ffa081ab7f0 con 0x560c91b53c10 2024-04-23T10:33:40.588 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.587+0000 7ffa11f96740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ffa081ad330 con 0x560c91b44e40 2024-04-23T10:33:40.588 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.587+0000 7ffa11f96740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ffa081abec0 con 0x560c91b44520 2024-04-23T10:33:40.588 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.587+0000 7ffa0d73f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c91b53c10 0x560c91b530e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:40.588 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.587+0000 7ffa0d73f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c91b53c10 0x560c91b530e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56470/0 (socket says 172.21.15.97:56470) 2024-04-23T10:33:40.588 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.587+0000 7ffa0d73f640 1 -- 172.21.15.97:0/1206943795 learned_addr learned my addr 172.21.15.97:0/1206943795 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:33:40.588 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.587+0000 7ffa0cf3e640 1 --2- 172.21.15.97:0/1206943795 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c91b44520 0x560c91b5f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:40.588 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.587+0000 7ffa07fff640 1 --2- 172.21.15.97:0/1206943795 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560c91b44e40 0x560c91b40f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:40.588 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.587+0000 7ffa0d73f640 1 -- 172.21.15.97:0/1206943795 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c91b44520 msgr2=0x560c91b5f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:33:40.588 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.587+0000 7ffa0d73f640 1 --2- 172.21.15.97:0/1206943795 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c91b44520 0x560c91b5f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:40.588 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.587+0000 7ffa0d73f640 1 -- 172.21.15.97:0/1206943795 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560c91b44e40 msgr2=0x560c91b40f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:33:40.588 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.587+0000 7ffa0d73f640 1 --2- 172.21.15.97:0/1206943795 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560c91b44e40 0x560c91b40f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.587+0000 7ffa0d73f640 1 -- 172.21.15.97:0/1206943795 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c91b44c70 con 0x560c91b53c10 2024-04-23T10:33:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.587+0000 7ffa07fff640 1 --2- 172.21.15.97:0/1206943795 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560c91b44e40 0x560c91b40f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:33:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.588+0000 7ffa0d73f640 1 --2- 172.21.15.97:0/1206943795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c91b53c10 0x560c91b530e0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7ffa000ce970 tx=0x7ffa000bd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5011ebecd0e9d23d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:33:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.588+0000 7ff9f27fc640 1 -- 172.21.15.97:0/1206943795 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa000a5830 con 0x560c91b53c10 2024-04-23T10:33:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.588+0000 7ff9f27fc640 1 -- 172.21.15.97:0/1206943795 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa000051d0 con 0x560c91b53c10 2024-04-23T10:33:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.588+0000 7ff9f27fc640 1 -- 172.21.15.97:0/1206943795 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa000bd240 con 0x560c91b53c10 2024-04-23T10:33:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.588+0000 7ffa11f96740 1 -- 172.21.15.97:0/1206943795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c91b53c10 msgr2=0x560c91b530e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.588+0000 7ffa11f96740 1 --2- 172.21.15.97:0/1206943795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c91b53c10 0x560c91b530e0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7ffa000ce970 tx=0x7ffa000bd040 comp rx=0 tx=0).stop 2024-04-23T10:33:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.588+0000 7ffa11f96740 1 -- 172.21.15.97:0/1206943795 shutdown_connections 2024-04-23T10:33:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.588+0000 7ffa11f96740 1 --2- 172.21.15.97:0/1206943795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c91b53c10 0x560c91b530e0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.588+0000 7ffa11f96740 1 --2- 172.21.15.97:0/1206943795 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560c91b44e40 0x560c91b40f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.588+0000 7ffa11f96740 1 --2- 172.21.15.97:0/1206943795 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c91b44520 0x560c91b5f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.588+0000 7ffa11f96740 1 -- 172.21.15.97:0/1206943795 >> 172.21.15.97:0/1206943795 conn(0x560c91b40b30 msgr2=0x560c91b4bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:33:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.588+0000 7ffa11f96740 1 -- 172.21.15.97:0/1206943795 shutdown_connections 2024-04-23T10:33:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.588+0000 7ffa11f96740 1 -- 172.21.15.97:0/1206943795 wait complete. 2024-04-23T10:33:40.590 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.589+0000 7ffa11f96740 1 -- start start 2024-04-23T10:33:40.590 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.589+0000 7ffa11f96740 1 Processor -- start 2024-04-23T10:33:40.592 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.589+0000 7ffa11f96740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560c91b44520 0x560c91b57950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:40.592 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.589+0000 7ffa11f96740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c91b44e40 0x560c91b57e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:40.592 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.589+0000 7ffa11f96740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c91b53c10 0x560c91bc91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:40.592 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.589+0000 7ffa11f96740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ffa081acc60 con 0x560c91b44e40 2024-04-23T10:33:40.593 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.589+0000 7ffa11f96740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ffa081ac590 con 0x560c91b44520 2024-04-23T10:33:40.593 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.589+0000 7ffa11f96740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ffa081a97f0 con 0x560c91b53c10 2024-04-23T10:33:40.593 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.589+0000 7ffa11f96740 10 client.0 _ll_register_callbacks cb 0x7ff9fc032c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:33:40.593 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.589+0000 7ffa07fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c91b44e40 0x560c91b57e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:40.593 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.589+0000 7ffa07fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c91b44e40 0x560c91b57e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56480/0 (socket says 172.21.15.97:56480) 2024-04-23T10:33:40.593 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.589+0000 7ffa07fff640 1 -- 172.21.15.97:0/662230326 learned_addr learned my addr 172.21.15.97:0/662230326 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:33:40.593 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.589+0000 7ffa0cf3e640 1 --2- 172.21.15.97:0/662230326 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560c91b44520 0x560c91b57950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:40.593 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.589+0000 7ffa0d73f640 1 --2- 172.21.15.97:0/662230326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c91b53c10 0x560c91bc91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:40.593 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.590+0000 7ffa07fff640 1 -- 172.21.15.97:0/662230326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c91b53c10 msgr2=0x560c91bc91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:33:40.593 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.590+0000 7ffa07fff640 1 --2- 172.21.15.97:0/662230326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c91b53c10 0x560c91bc91f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:40.593 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.590+0000 7ffa07fff640 1 -- 172.21.15.97:0/662230326 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560c91b44520 msgr2=0x560c91b57950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:33:40.593 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.590+0000 7ffa07fff640 1 --2- 172.21.15.97:0/662230326 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560c91b44520 0x560c91b57950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:40.593 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.590+0000 7ffa07fff640 1 -- 172.21.15.97:0/662230326 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c91b58790 con 0x560c91b44e40 2024-04-23T10:33:40.593 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.590+0000 7ffa0d73f640 1 --2- 172.21.15.97:0/662230326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c91b53c10 0x560c91bc91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:33:40.593 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.590+0000 7ffa07fff640 1 --2- 172.21.15.97:0/662230326 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c91b44e40 0x560c91b57e90 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4012fd0 tx=0x7ff9f400ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9c303bafd4a90ca server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:33:40.594 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.590+0000 7ff9e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:33:40.594 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.590+0000 7ff9e7fff640 1 -- 172.21.15.97:0/662230326 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f4037060 con 0x560c91b44e40 2024-04-23T10:33:40.594 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.590+0000 7ff9e7fff640 1 -- 172.21.15.97:0/662230326 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7ff9f40327c0 con 0x560c91b44e40 2024-04-23T10:33:40.594 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.590+0000 7ffa11f96740 1 -- 172.21.15.97:0/662230326 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa000051d0 con 0x560c91b44e40 2024-04-23T10:33:40.594 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.590+0000 7ff9e7fff640 1 -- 172.21.15.97:0/662230326 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff9f401e780 con 0x560c91b44e40 2024-04-23T10:33:40.594 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.590+0000 7ffa11f96740 10 client.16611 fetch_fsmap learned FSMap version 26 2024-04-23T10:33:40.594 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.590+0000 7ffa11f96740 1 -- 172.21.15.97:0/662230326 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560c91bca7e0 con 0x560c91b44e40 2024-04-23T10:33:40.594 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.590+0000 7ff9e7fff640 1 -- 172.21.15.97:0/662230326 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa000051d0 con 0x560c91b44e40 2024-04-23T10:33:40.594 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.590+0000 7ff9e7fff640 1 -- 172.21.15.97:0/662230326 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff9f401f5e0 con 0x560c91b44e40 2024-04-23T10:33:40.594 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.590+0000 7ffa11f96740 10 client.16611 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:33:40.594 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.590+0000 7ffa11f96740 10 client.16611 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:33:40.594 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.590+0000 7ffa11f96740 20 client.16611 populate_metadata read hostname 'smithi097' 2024-04-23T10:33:40.594 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.591+0000 7ffa11f96740 1 --2- 172.21.15.97:0/662230326 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x560c91bcacb0 0x560c91bcd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:40.594 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.591+0000 7ffa11f96740 4 client.16611 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:33:40.595 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.591+0000 7ffa11f96740 1 -- 172.21.15.97:0/662230326 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560c91ab1200 con 0x560c91bcacb0 2024-04-23T10:33:40.595 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.591+0000 7ffa0cf3e640 1 --2- 172.21.15.97:0/662230326 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x560c91bcacb0 0x560c91bcd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:40.595 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.591+0000 7ff9e7fff640 10 client.16611 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:33:40.595 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.591+0000 7ffa0cf3e640 1 --2- 172.21.15.97:0/662230326 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x560c91bcacb0 0x560c91bcd0d0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8dad841f3ec66ad4 server_cookie=a8b43a597df7649c in_seq=0 out_seq=0 2024-04-23T10:33:40.604 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.605+0000 7ff9e7fff640 1 -- 172.21.15.97:0/662230326 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560c91ab1200 con 0x560c91bcacb0 2024-04-23T10:33:40.604 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.605+0000 7ff9e7fff640 10 client.16611 handle_command_reply: tid=0 2024-04-23T10:33:40.605 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.605+0000 7ffa11f96740 1 client.16611 shutdown 2024-04-23T10:33:40.605 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.605+0000 7ffa11f96740 1 -- 172.21.15.97:0/662230326 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c91b44e40 msgr2=0x560c91b57e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:40.605 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.605+0000 7ffa11f96740 1 --2- 172.21.15.97:0/662230326 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c91b44e40 0x560c91b57e90 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4012fd0 tx=0x7ff9f400ba30 comp rx=0 tx=0).stop 2024-04-23T10:33:40.605 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.605+0000 7ffa11f96740 1 -- 172.21.15.97:0/662230326 shutdown_connections 2024-04-23T10:33:40.605 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.605+0000 7ffa11f96740 1 --2- 172.21.15.97:0/662230326 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x560c91bcacb0 0x560c91bcd0d0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:40.605 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.605+0000 7ffa11f96740 1 --2- 172.21.15.97:0/662230326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c91b53c10 0x560c91bc91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:40.605 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.605+0000 7ff9e7fff640 0 client.16611 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:33:40.605 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.605+0000 7ffa11f96740 1 --2- 172.21.15.97:0/662230326 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x560c91b44e40 0x560c91b57e90 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:40.605 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.605+0000 7ffa11f96740 1 --2- 172.21.15.97:0/662230326 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560c91b44520 0x560c91b57950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:40.605 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.605+0000 7ffa11f96740 1 -- 172.21.15.97:0/662230326 >> 172.21.15.97:0/662230326 conn(0x560c91b40b30 msgr2=0x560c91b4bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:33:40.605 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.605+0000 7ffa11f96740 1 -- 172.21.15.97:0/662230326 shutdown_connections 2024-04-23T10:33:40.605 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.605+0000 7ffa11f96740 1 -- 172.21.15.97:0/662230326 wait complete. 2024-04-23T10:33:40.605 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.605+0000 7ffa11f96740 20 client.16611 trim_cache size 0 max 16384 2024-04-23T10:33:40.606 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:33:40.606 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:33:40.606 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:33:40.606 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:33:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.606+0000 7ffa0f1a2640 1 -- 172.21.15.97:0/530777862 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7ff9f80a3dc0 msgr2=0x7ff9f80a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.606+0000 7ffa0f1a2640 1 --2- 172.21.15.97:0/530777862 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7ff9f80a3dc0 0x7ff9f80a6280 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4012500 tx=0x7ff9f4012900 comp rx=0 tx=0).stop 2024-04-23T10:33:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.606+0000 7ffa0f1a2640 1 -- 172.21.15.97:0/530777862 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa0819cf70 msgr2=0x7ffa0819d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.606+0000 7ffa0f1a2640 1 --2- 172.21.15.97:0/530777862 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa0819cf70 0x7ffa0819d420 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7ffa0000ea60 tx=0x7ffa0000ef20 comp rx=0 tx=0).stop 2024-04-23T10:33:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.606+0000 7ffa0f1a2640 1 -- 172.21.15.97:0/530777862 shutdown_connections 2024-04-23T10:33:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.606+0000 7ffa0f1a2640 1 --2- 172.21.15.97:0/530777862 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7ff9f80a3dc0 0x7ff9f80a6280 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.606+0000 7ffa0f1a2640 1 --2- 172.21.15.97:0/530777862 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa0819cf70 0x7ffa0819d420 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.606+0000 7ffa0f1a2640 1 --2- 172.21.15.97:0/530777862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa081072a0 0x7ffa08198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.606+0000 7ffa0f1a2640 1 --2- 172.21.15.97:0/530777862 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa081021d0 0x7ffa081986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.606+0000 7ffa0f1a2640 1 -- 172.21.15.97:0/530777862 >> 172.21.15.97:0/530777862 conn(0x7ffa080793e0 msgr2=0x7ffa080ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:33:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.606+0000 7ffa0f1a2640 1 -- 172.21.15.97:0/530777862 shutdown_connections 2024-04-23T10:33:40.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:40.606+0000 7ffa0f1a2640 1 -- 172.21.15.97:0/530777862 wait complete. 2024-04-23T10:33:41.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:33:41 smithi183 ceph-mon[32317]: pgmap v229: 97 pgs: 97 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-23T10:33:41.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:33:41 smithi196 ceph-mon[32919]: pgmap v229: 97 pgs: 97 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-23T10:33:41.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:33:41 smithi097 ceph-mon[29197]: pgmap v229: 97 pgs: 97 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-23T10:33:42.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:33:42 smithi196 ceph-mon[32919]: pgmap v230: 97 pgs: 97 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-23T10:33:42.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:33:42 smithi097 ceph-mon[29197]: pgmap v230: 97 pgs: 97 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-23T10:33:42.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:33:42 smithi183 ceph-mon[32317]: pgmap v230: 97 pgs: 97 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-23T10:33:44.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:33:44 smithi196 ceph-mon[32919]: pgmap v231: 97 pgs: 97 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 48 KiB/s wr, 22 op/s 2024-04-23T10:33:44.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:33:44 smithi097 ceph-mon[29197]: pgmap v231: 97 pgs: 97 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 48 KiB/s wr, 22 op/s 2024-04-23T10:33:44.649 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:33:44 smithi183 ceph-mon[32317]: pgmap v231: 97 pgs: 97 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 48 KiB/s wr, 22 op/s 2024-04-23T10:33:46.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:33:46 smithi196 ceph-mon[32919]: pgmap v232: 97 pgs: 97 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 KiB/s wr, 13 op/s 2024-04-23T10:33:46.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:33:46 smithi097 ceph-mon[29197]: pgmap v232: 97 pgs: 97 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 KiB/s wr, 13 op/s 2024-04-23T10:33:46.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:33:46 smithi183 ceph-mon[32317]: pgmap v232: 97 pgs: 97 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 KiB/s wr, 13 op/s 2024-04-23T10:33:48.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:33:48 smithi196 ceph-mon[32919]: pgmap v233: 97 pgs: 97 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 KiB/s wr, 17 op/s 2024-04-23T10:33:48.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:33:48 smithi097 ceph-mon[29197]: pgmap v233: 97 pgs: 97 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 KiB/s wr, 17 op/s 2024-04-23T10:33:48.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:33:48 smithi183 ceph-mon[32317]: pgmap v233: 97 pgs: 97 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 KiB/s wr, 17 op/s 2024-04-23T10:33:50.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:33:50 smithi196 ceph-mon[32919]: pgmap v234: 97 pgs: 97 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.6 KiB/s wr, 11 op/s 2024-04-23T10:33:50.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:33:50 smithi097 ceph-mon[29197]: pgmap v234: 97 pgs: 97 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.6 KiB/s wr, 11 op/s 2024-04-23T10:33:50.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:33:50 smithi183 ceph-mon[32317]: pgmap v234: 97 pgs: 97 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.6 KiB/s wr, 11 op/s 2024-04-23T10:33:51.204 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T10:33:51.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.315+0000 7f4b6c7c6640 1 -- 172.21.15.97:0/3568749556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b64103b00 msgr2=0x7f4b6410bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:51.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.315+0000 7f4b6c7c6640 1 --2- 172.21.15.97:0/3568749556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b64103b00 0x7f4b6410bfe0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f4b60009900 tx=0x7f4b6002a510 comp rx=0 tx=0).stop 2024-04-23T10:33:51.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.315+0000 7f4b6c7c6640 1 -- 172.21.15.97:0/3568749556 shutdown_connections 2024-04-23T10:33:51.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.315+0000 7f4b6c7c6640 1 --2- 172.21.15.97:0/3568749556 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b6410c520 0x7f4b6410e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:51.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.315+0000 7f4b6c7c6640 1 --2- 172.21.15.97:0/3568749556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b64103b00 0x7f4b6410bfe0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:51.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.315+0000 7f4b6c7c6640 1 --2- 172.21.15.97:0/3568749556 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b641031e0 0x7f4b641035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:51.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.315+0000 7f4b6c7c6640 1 -- 172.21.15.97:0/3568749556 >> 172.21.15.97:0/3568749556 conn(0x7f4b64077f50 msgr2=0x7f4b64078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:33:51.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.315+0000 7f4b6c7c6640 1 -- 172.21.15.97:0/3568749556 shutdown_connections 2024-04-23T10:33:51.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.315+0000 7f4b6c7c6640 1 -- 172.21.15.97:0/3568749556 wait complete. 2024-04-23T10:33:51.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.315+0000 7f4b6c7c6640 1 Processor -- start 2024-04-23T10:33:51.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.315+0000 7f4b6c7c6640 1 -- start start 2024-04-23T10:33:51.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.316+0000 7f4b6c7c6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b641031e0 0x7f4b641a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:51.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.316+0000 7f4b6c7c6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b64103b00 0x7f4b641a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:51.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.316+0000 7f4b6c7c6640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b6410c520 0x7f4b6419a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:51.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.316+0000 7f4b6c7c6640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4b641121b0 con 0x7f4b64103b00 2024-04-23T10:33:51.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.316+0000 7f4b6c7c6640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4b64112030 con 0x7f4b6410c520 2024-04-23T10:33:51.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.316+0000 7f4b6c7c6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4b64112330 con 0x7f4b641031e0 2024-04-23T10:33:51.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.316+0000 7f4b69d61640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b64103b00 0x7f4b641a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:51.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.316+0000 7f4b69d61640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b64103b00 0x7f4b641a0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36694/0 (socket says 172.21.15.97:36694) 2024-04-23T10:33:51.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.316+0000 7f4b69d61640 1 -- 172.21.15.97:0/3781035693 learned_addr learned my addr 172.21.15.97:0/3781035693 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:33:51.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.316+0000 7f4b6ad63640 1 --2- 172.21.15.97:0/3781035693 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b6410c520 0x7f4b6419a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:51.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.316+0000 7f4b6a562640 1 --2- 172.21.15.97:0/3781035693 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b641031e0 0x7f4b641a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:51.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.316+0000 7f4b69d61640 1 -- 172.21.15.97:0/3781035693 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b641031e0 msgr2=0x7f4b641a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:51.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.316+0000 7f4b69d61640 1 --2- 172.21.15.97:0/3781035693 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b641031e0 0x7f4b641a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:51.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.316+0000 7f4b69d61640 1 -- 172.21.15.97:0/3781035693 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b6410c520 msgr2=0x7f4b6419a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:51.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.316+0000 7f4b69d61640 1 --2- 172.21.15.97:0/3781035693 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b6410c520 0x7f4b6419a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:51.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.316+0000 7f4b69d61640 1 -- 172.21.15.97:0/3781035693 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b6419afa0 con 0x7f4b64103b00 2024-04-23T10:33:51.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.316+0000 7f4b6a562640 1 --2- 172.21.15.97:0/3781035693 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b641031e0 0x7f4b641a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:33:51.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.316+0000 7f4b69d61640 1 --2- 172.21.15.97:0/3781035693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b64103b00 0x7f4b641a0b50 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f4b60005940 tx=0x7f4b600040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:33:51.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.316+0000 7f4b577fe640 1 -- 172.21.15.97:0/3781035693 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b6003f030 con 0x7f4b64103b00 2024-04-23T10:33:51.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.316+0000 7f4b577fe640 1 -- 172.21.15.97:0/3781035693 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b600044c0 con 0x7f4b64103b00 2024-04-23T10:33:51.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.316+0000 7f4b6c7c6640 1 -- 172.21.15.97:0/3781035693 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b6419b230 con 0x7f4b64103b00 2024-04-23T10:33:51.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.316+0000 7f4b577fe640 1 -- 172.21.15.97:0/3781035693 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b60035070 con 0x7f4b64103b00 2024-04-23T10:33:51.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.316+0000 7f4b6c7c6640 1 -- 172.21.15.97:0/3781035693 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b641a7460 con 0x7f4b64103b00 2024-04-23T10:33:51.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.318+0000 7f4b6c7c6640 1 -- 172.21.15.97:0/3781035693 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b640698e0 con 0x7f4b64103b00 2024-04-23T10:33:51.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.319+0000 7f4b577fe640 1 -- 172.21.15.97:0/3781035693 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4b60042030 con 0x7f4b64103b00 2024-04-23T10:33:51.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.319+0000 7f4b577fe640 1 --2- 172.21.15.97:0/3781035693 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f4b500a3d20 0x7f4b500a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:51.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.319+0000 7f4b6a562640 1 --2- 172.21.15.97:0/3781035693 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f4b500a3d20 0x7f4b500a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:51.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.319+0000 7f4b577fe640 1 -- 172.21.15.97:0/3781035693 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7f4b600f0bb0 con 0x7f4b64103b00 2024-04-23T10:33:51.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.319+0000 7f4b6a562640 1 --2- 172.21.15.97:0/3781035693 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f4b500a3d20 0x7f4b500a61e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4b58009c20 tx=0x7f4b58005f90 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:33:51.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.325+0000 7f4b577fe640 1 -- 172.21.15.97:0/3781035693 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4b600bd270 con 0x7f4b64103b00 2024-04-23T10:33:51.542 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.543+0000 7f4b6c7c6640 1 -- 172.21.15.97:0/3781035693 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4b641a7630 con 0x7f4b64103b00 2024-04-23T10:33:51.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.544+0000 7f4b577fe640 1 -- 172.21.15.97:0/3781035693 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11931 (secure 0 0 0) 0x7f4b600bcb70 con 0x7f4b64103b00 2024-04-23T10:33:51.543 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:33:51.543 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3946878762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3946878762},{"type":"v1","addr":"172.21.15.97:6835","nonce":3946878762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/209800861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":209800861},{"type":"v1","addr":"172.21.15.97:6837","nonce":209800861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T10:24:06.910134+0000","modified":"2024-04-23T10:26:09.820985+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6839/418013575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":418013575},{"type":"v1","addr":"172.21.15.97:6839","nonce":418013575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6841/747242680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":747242680},{"type":"v1","addr":"172.21.15.97:6841","nonce":747242680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6833/4292847121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4292847121},{"type":"v1","addr":"172.21.15.183:6833","nonce":4292847121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6833/3885816565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3885816565},{"type":"v1","addr":"172.21.15.196:6833","nonce":3885816565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1897093105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1897093105},{"type":"v1","addr":"172.21.15.183:6835","nonce":1897093105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1171477563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1171477563},{"type":"v1","addr":"172.21.15.196:6835","nonce":1171477563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1898341911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1898341911},{"type":"v1","addr":"172.21.15.183:6837","nonce":1898341911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.196:6837/3094946502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3094946502},{"type":"v1","addr":"172.21.15.196:6837","nonce":3094946502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.196:6839/83843556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":83843556},{"type":"v1","addr":"172.21.15.196:6839","nonce":83843556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6839/1763841130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1763841130},{"type":"v1","addr":"172.21.15.183:6839","nonce":1763841130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24487,24403,24398,24469,24488]},"id":1}]} 2024-04-23T10:33:51.544 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 26 2024-04-23T10:33:51.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.547+0000 7f4b6c7c6640 1 -- 172.21.15.97:0/3781035693 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f4b500a3d20 msgr2=0x7f4b500a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:51.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.547+0000 7f4b6c7c6640 1 --2- 172.21.15.97:0/3781035693 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f4b500a3d20 0x7f4b500a61e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4b58009c20 tx=0x7f4b58005f90 comp rx=0 tx=0).stop 2024-04-23T10:33:51.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.547+0000 7f4b6c7c6640 1 -- 172.21.15.97:0/3781035693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b64103b00 msgr2=0x7f4b641a0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:51.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.547+0000 7f4b6c7c6640 1 --2- 172.21.15.97:0/3781035693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b64103b00 0x7f4b641a0b50 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f4b60005940 tx=0x7f4b600040c0 comp rx=0 tx=0).stop 2024-04-23T10:33:51.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.547+0000 7f4b6c7c6640 1 -- 172.21.15.97:0/3781035693 shutdown_connections 2024-04-23T10:33:51.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.547+0000 7f4b6c7c6640 1 --2- 172.21.15.97:0/3781035693 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f4b500a3d20 0x7f4b500a61e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:51.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.547+0000 7f4b6c7c6640 1 --2- 172.21.15.97:0/3781035693 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b6410c520 0x7f4b6419a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:51.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.547+0000 7f4b6c7c6640 1 --2- 172.21.15.97:0/3781035693 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b64103b00 0x7f4b641a0b50 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:51.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.547+0000 7f4b6c7c6640 1 --2- 172.21.15.97:0/3781035693 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b641031e0 0x7f4b641a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:51.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.547+0000 7f4b6c7c6640 1 -- 172.21.15.97:0/3781035693 >> 172.21.15.97:0/3781035693 conn(0x7f4b64077f50 msgr2=0x7f4b641084d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:33:51.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.547+0000 7f4b6c7c6640 1 -- 172.21.15.97:0/3781035693 shutdown_connections 2024-04-23T10:33:51.546 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.547+0000 7f4b6c7c6640 1 -- 172.21.15.97:0/3781035693 wait complete. 2024-04-23T10:33:51.556 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --set-id af277cb5 --release --await' 2024-04-23T10:33:51.556 DEBUG:teuthology.orchestra.run.smithi097:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24488 quiesce db --set-id af277cb5 --release --await 2024-04-23T10:33:51.660 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.661+0000 7f341ccbb640 1 -- 172.21.15.97:0/996563860 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3418107260 msgr2=0x7f341810ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:51.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.661+0000 7f341ccbb640 1 --2- 172.21.15.97:0/996563860 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3418107260 0x7f341810ee30 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f3410009ca0 tx=0x7f341002a8c0 comp rx=0 tx=0).stop 2024-04-23T10:33:51.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.662+0000 7f341ccbb640 1 -- 172.21.15.97:0/996563860 shutdown_connections 2024-04-23T10:33:51.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.662+0000 7f341ccbb640 1 --2- 172.21.15.97:0/996563860 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3418107260 0x7f341810ee30 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:51.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.662+0000 7f341ccbb640 1 --2- 172.21.15.97:0/996563860 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3418102b40 0x7f3418106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:51.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.662+0000 7f341ccbb640 1 --2- 172.21.15.97:0/996563860 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3418102190 0x7f3418102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:51.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.662+0000 7f341ccbb640 1 -- 172.21.15.97:0/996563860 >> 172.21.15.97:0/996563860 conn(0x7f34180793a0 msgr2=0x7f34180797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:33:51.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.662+0000 7f341ccbb640 1 -- 172.21.15.97:0/996563860 shutdown_connections 2024-04-23T10:33:51.661 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.662+0000 7f341ccbb640 1 -- 172.21.15.97:0/996563860 wait complete. 2024-04-23T10:33:51.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.662+0000 7f341ccbb640 1 Processor -- start 2024-04-23T10:33:51.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.662+0000 7f341ccbb640 1 -- start start 2024-04-23T10:33:51.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.663+0000 7f341ccbb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3418102190 0x7f341819e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:51.663 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.663+0000 7f341ccbb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3418107260 0x7f341819eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:51.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.663+0000 7f341ccbb640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3418198720 0x7f3418198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:51.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.663+0000 7f341ccbb640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f34181122a0 con 0x7f3418102190 2024-04-23T10:33:51.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.663+0000 7f341ccbb640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3418112120 con 0x7f3418198720 2024-04-23T10:33:51.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.663+0000 7f341ccbb640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3418112420 con 0x7f3418107260 2024-04-23T10:33:51.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.663+0000 7f341659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3418102190 0x7f341819e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:51.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.663+0000 7f341659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3418102190 0x7f341819e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36720/0 (socket says 172.21.15.97:36720) 2024-04-23T10:33:51.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.663+0000 7f341659c640 1 -- 172.21.15.97:0/3007345440 learned_addr learned my addr 172.21.15.97:0/3007345440 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:33:51.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.663+0000 7f3415d9b640 1 --2- 172.21.15.97:0/3007345440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3418107260 0x7f341819eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:51.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.663+0000 7f3416d9d640 1 --2- 172.21.15.97:0/3007345440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3418198720 0x7f3418198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:51.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.663+0000 7f341659c640 1 -- 172.21.15.97:0/3007345440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3418107260 msgr2=0x7f341819eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:51.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.663+0000 7f341659c640 1 --2- 172.21.15.97:0/3007345440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3418107260 0x7f341819eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:51.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.663+0000 7f341659c640 1 -- 172.21.15.97:0/3007345440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3418198720 msgr2=0x7f3418198ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:51.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.663+0000 7f341659c640 1 --2- 172.21.15.97:0/3007345440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3418198720 0x7f3418198ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:51.664 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.663+0000 7f341659c640 1 -- 172.21.15.97:0/3007345440 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3418199320 con 0x7f3418102190 2024-04-23T10:33:51.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.663+0000 7f341659c640 1 --2- 172.21.15.97:0/3007345440 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3418102190 0x7f341819e610 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f340800b4a0 tx=0x7f340800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:33:51.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.663+0000 7f340f7fe640 1 -- 172.21.15.97:0/3007345440 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3408015020 con 0x7f3418102190 2024-04-23T10:33:51.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.663+0000 7f340f7fe640 1 -- 172.21.15.97:0/3007345440 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3408011ce0 con 0x7f3418102190 2024-04-23T10:33:51.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.663+0000 7f341ccbb640 1 -- 172.21.15.97:0/3007345440 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34181a53b0 con 0x7f3418102190 2024-04-23T10:33:51.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.663+0000 7f340f7fe640 1 -- 172.21.15.97:0/3007345440 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3408010dc0 con 0x7f3418102190 2024-04-23T10:33:51.665 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.663+0000 7f341ccbb640 1 -- 172.21.15.97:0/3007345440 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34181a5810 con 0x7f3418102190 2024-04-23T10:33:51.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.666+0000 7f340f7fe640 1 -- 172.21.15.97:0/3007345440 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3408018020 con 0x7f3418102190 2024-04-23T10:33:51.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.666+0000 7f341f9ce740 1 Processor -- start 2024-04-23T10:33:51.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.666+0000 7f341f9ce740 1 -- start start 2024-04-23T10:33:51.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.666+0000 7f341f9ce740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556dc9c09370 0x556dc9c09750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:51.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.666+0000 7f341f9ce740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556dc9c09c90 0x556dc9c18350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:51.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.666+0000 7f341f9ce740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556dc9c18890 0x556dc9c1ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:51.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.666+0000 7f341f9ce740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x556dc9b6e6d0 con 0x556dc9c09370 2024-04-23T10:33:51.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.666+0000 7f341f9ce740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556dc9b0b1a0 con 0x556dc9c09c90 2024-04-23T10:33:51.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.666+0000 7f341f9ce740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556dc9b6a690 con 0x556dc9c18890 2024-04-23T10:33:51.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.666+0000 7f3416d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556dc9c09370 0x556dc9c09750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:51.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.666+0000 7f3416d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556dc9c09370 0x556dc9c09750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36734/0 (socket says 172.21.15.97:36734) 2024-04-23T10:33:51.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.666+0000 7f3416d9d640 1 -- 172.21.15.97:0/3866572436 learned_addr learned my addr 172.21.15.97:0/3866572436 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:33:51.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.666+0000 7f340f7fe640 1 --2- 172.21.15.97:0/3007345440 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f34040a3da0 0x7f34040a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:51.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.666+0000 7f3415d9b640 1 --2- 172.21.15.97:0/3866572436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556dc9c18890 0x556dc9c1ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:51.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.666+0000 7f341659c640 1 --2- 172.21.15.97:0/3866572436 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556dc9c09c90 0x556dc9c18350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:51.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.666+0000 7f3416d9d640 1 -- 172.21.15.97:0/3866572436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556dc9c18890 msgr2=0x556dc9c1ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:33:51.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.666+0000 7f3416d9d640 1 --2- 172.21.15.97:0/3866572436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556dc9c18890 0x556dc9c1ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:51.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.667+0000 7f3416d9d640 1 -- 172.21.15.97:0/3866572436 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556dc9c09c90 msgr2=0x556dc9c18350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:33:51.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.667+0000 7f340f7fe640 1 -- 172.21.15.97:0/3007345440 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7f34080cff30 con 0x7f3418102190 2024-04-23T10:33:51.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.667+0000 7f3416d9d640 1 --2- 172.21.15.97:0/3866572436 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556dc9c09c90 0x556dc9c18350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:51.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.667+0000 7f3416d9d640 1 -- 172.21.15.97:0/3866572436 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556dc9b155b0 con 0x556dc9c09370 2024-04-23T10:33:51.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.667+0000 7f341659c640 1 --2- 172.21.15.97:0/3866572436 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556dc9c09c90 0x556dc9c18350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:33:51.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.667+0000 7f3416d9d640 1 --2- 172.21.15.97:0/3007345440 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f34040a3da0 0x7f34040a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:51.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.667+0000 7f3416d9d640 1 --2- 172.21.15.97:0/3866572436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556dc9c09370 0x556dc9c09750 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f3410005cc0 tx=0x7f3410031af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2faa04fdc7db9d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:33:51.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.667+0000 7f340cff9640 1 -- 172.21.15.97:0/3866572436 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3410003a30 con 0x556dc9c09370 2024-04-23T10:33:51.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.667+0000 7f340cff9640 1 -- 172.21.15.97:0/3866572436 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3410036040 con 0x556dc9c09370 2024-04-23T10:33:51.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.667+0000 7f340cff9640 1 -- 172.21.15.97:0/3866572436 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341002f660 con 0x556dc9c09370 2024-04-23T10:33:51.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.668+0000 7f341f9ce740 1 -- 172.21.15.97:0/3866572436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556dc9c09370 msgr2=0x556dc9c09750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:51.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.668+0000 7f3416d9d640 1 --2- 172.21.15.97:0/3007345440 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f34040a3da0 0x7f34040a6260 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f341004c000 tx=0x7f3410049000 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:33:51.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.668+0000 7f341f9ce740 1 --2- 172.21.15.97:0/3866572436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556dc9c09370 0x556dc9c09750 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f3410005cc0 tx=0x7f3410031af0 comp rx=0 tx=0).stop 2024-04-23T10:33:51.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.668+0000 7f341f9ce740 1 -- 172.21.15.97:0/3866572436 shutdown_connections 2024-04-23T10:33:51.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.668+0000 7f341f9ce740 1 --2- 172.21.15.97:0/3866572436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556dc9c18890 0x556dc9c1ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:51.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.668+0000 7f341f9ce740 1 --2- 172.21.15.97:0/3866572436 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556dc9c09c90 0x556dc9c18350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:51.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.668+0000 7f341f9ce740 1 --2- 172.21.15.97:0/3866572436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556dc9c09370 0x556dc9c09750 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:51.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.668+0000 7f341f9ce740 1 -- 172.21.15.97:0/3866572436 >> 172.21.15.97:0/3866572436 conn(0x556dc9c057b0 msgr2=0x556dc9c05b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:33:51.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.668+0000 7f341f9ce740 1 -- 172.21.15.97:0/3866572436 shutdown_connections 2024-04-23T10:33:51.673 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.668+0000 7f341f9ce740 1 -- 172.21.15.97:0/3866572436 wait complete. 2024-04-23T10:33:51.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.668+0000 7f341f9ce740 1 -- start start 2024-04-23T10:33:51.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.668+0000 7f341f9ce740 1 Processor -- start 2024-04-23T10:33:51.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.668+0000 7f341f9ce740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556dc9c09370 0x556dc9c182b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:51.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.668+0000 7f341f9ce740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556dc9c09c90 0x556dc9c155a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:51.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.668+0000 7f341f9ce740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556dc9c18890 0x556dc9c15bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:51.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.668+0000 7f341f9ce740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x556dc9c1b5f0 con 0x556dc9c18890 2024-04-23T10:33:51.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.668+0000 7f341f9ce740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556dc9b08b20 con 0x556dc9c09370 2024-04-23T10:33:51.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.668+0000 7f341f9ce740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556dc9c1b770 con 0x556dc9c09c90 2024-04-23T10:33:51.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.669+0000 7f341f9ce740 10 client.0 _ll_register_callbacks cb 0x556dc9b08530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:33:51.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.669+0000 7f3416d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556dc9c18890 0x556dc9c15bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:51.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.669+0000 7f3416d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556dc9c18890 0x556dc9c15bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36748/0 (socket says 172.21.15.97:36748) 2024-04-23T10:33:51.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.669+0000 7f3416d9d640 1 -- 172.21.15.97:0/10688548 learned_addr learned my addr 172.21.15.97:0/10688548 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:33:51.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.669+0000 7f3415d9b640 1 --2- 172.21.15.97:0/10688548 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556dc9c09c90 0x556dc9c155a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:51.674 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.669+0000 7f341659c640 1 --2- 172.21.15.97:0/10688548 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556dc9c09370 0x556dc9c182b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:51.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.669+0000 7f3416d9d640 1 -- 172.21.15.97:0/10688548 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556dc9c09c90 msgr2=0x556dc9c155a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:33:51.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.669+0000 7f3416d9d640 1 --2- 172.21.15.97:0/10688548 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556dc9c09c90 0x556dc9c155a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:51.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.669+0000 7f3416d9d640 1 -- 172.21.15.97:0/10688548 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556dc9c09370 msgr2=0x556dc9c182b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:33:51.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.669+0000 7f3416d9d640 1 --2- 172.21.15.97:0/10688548 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556dc9c09370 0x556dc9c182b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:51.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.669+0000 7f3416d9d640 1 -- 172.21.15.97:0/10688548 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556dc9880920 con 0x556dc9c18890 2024-04-23T10:33:51.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.669+0000 7f341659c640 1 --2- 172.21.15.97:0/10688548 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556dc9c09370 0x556dc9c182b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:33:51.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.669+0000 7f3416d9d640 1 --2- 172.21.15.97:0/10688548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556dc9c18890 0x556dc9c15bb0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f3410005c70 tx=0x7f3410031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87f8e99b2d802ed2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:33:51.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.669+0000 7f33fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:33:51.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.669+0000 7f33fdffb640 1 -- 172.21.15.97:0/10688548 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3410004040 con 0x556dc9c18890 2024-04-23T10:33:51.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.669+0000 7f33fdffb640 1 -- 172.21.15.97:0/10688548 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7f341004e030 con 0x556dc9c18890 2024-04-23T10:33:51.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.669+0000 7f341f9ce740 1 -- 172.21.15.97:0/10688548 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3410036040 con 0x556dc9c18890 2024-04-23T10:33:51.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.669+0000 7f33fdffb640 1 -- 172.21.15.97:0/10688548 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f34100041e0 con 0x556dc9c18890 2024-04-23T10:33:51.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.669+0000 7f341f9ce740 10 client.16629 fetch_fsmap learned FSMap version 26 2024-04-23T10:33:51.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.669+0000 7f341f9ce740 1 -- 172.21.15.97:0/10688548 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556dc994cd30 con 0x556dc9c18890 2024-04-23T10:33:51.675 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.669+0000 7f33fdffb640 1 -- 172.21.15.97:0/10688548 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3410036040 con 0x556dc9c18890 2024-04-23T10:33:51.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.670+0000 7f33fdffb640 1 -- 172.21.15.97:0/10688548 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3410030650 con 0x556dc9c18890 2024-04-23T10:33:51.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.670+0000 7f341f9ce740 10 client.16629 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:33:51.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.670+0000 7f341f9ce740 10 client.16629 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:33:51.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.670+0000 7f341f9ce740 20 client.16629 populate_metadata read hostname 'smithi097' 2024-04-23T10:33:51.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.670+0000 7f341f9ce740 1 --2- 172.21.15.97:0/10688548 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x556dc9c287b0 0x556dc9c2aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:51.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.670+0000 7f341f9ce740 4 client.16629 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:33:51.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.670+0000 7f341f9ce740 1 -- 172.21.15.97:0/10688548 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556dc9b2c500 con 0x556dc9c287b0 2024-04-23T10:33:51.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.670+0000 7f341659c640 1 --2- 172.21.15.97:0/10688548 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x556dc9c287b0 0x556dc9c2aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:51.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.670+0000 7f341659c640 1 --2- 172.21.15.97:0/10688548 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x556dc9c287b0 0x556dc9c2aba0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8630c24dbd882817 server_cookie=a04c0c7f40634b53 in_seq=0 out_seq=0 2024-04-23T10:33:51.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.670+0000 7f33fdffb640 10 client.16629 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:33:51.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.671+0000 7f33fdffb640 1 -- 172.21.15.97:0/10688548 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556dc9b2c500 con 0x556dc9c287b0 2024-04-23T10:33:51.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.671+0000 7f33fdffb640 10 client.16629 handle_command_reply: tid=0 2024-04-23T10:33:51.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.671+0000 7f341f9ce740 1 client.16629 shutdown 2024-04-23T10:33:51.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.671+0000 7f341f9ce740 1 -- 172.21.15.97:0/10688548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556dc9c18890 msgr2=0x556dc9c15bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:51.676 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.671+0000 7f341f9ce740 1 --2- 172.21.15.97:0/10688548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556dc9c18890 0x556dc9c15bb0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f3410005c70 tx=0x7f3410031a20 comp rx=0 tx=0).stop 2024-04-23T10:33:51.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.671+0000 7f341f9ce740 1 -- 172.21.15.97:0/10688548 shutdown_connections 2024-04-23T10:33:51.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.671+0000 7f341f9ce740 1 --2- 172.21.15.97:0/10688548 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x556dc9c287b0 0x556dc9c2aba0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:51.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.671+0000 7f341f9ce740 1 --2- 172.21.15.97:0/10688548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556dc9c18890 0x556dc9c15bb0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:51.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.671+0000 7f33fdffb640 0 client.16629 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:33:51.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.671+0000 7f341f9ce740 1 --2- 172.21.15.97:0/10688548 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556dc9c09c90 0x556dc9c155a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:51.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.671+0000 7f341f9ce740 1 --2- 172.21.15.97:0/10688548 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556dc9c09370 0x556dc9c182b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:51.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.671+0000 7f341f9ce740 1 -- 172.21.15.97:0/10688548 >> 172.21.15.97:0/10688548 conn(0x556dc9c057b0 msgr2=0x556dc9c0b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:33:51.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.671+0000 7f341f9ce740 1 -- 172.21.15.97:0/10688548 shutdown_connections 2024-04-23T10:33:51.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.671+0000 7f341f9ce740 1 -- 172.21.15.97:0/10688548 wait complete. 2024-04-23T10:33:51.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.671+0000 7f341f9ce740 20 client.16629 trim_cache size 0 max 16384 2024-04-23T10:33:51.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.688+0000 7f341f9ce740 1 Processor -- start 2024-04-23T10:33:51.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.689+0000 7f341f9ce740 1 -- start start 2024-04-23T10:33:51.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.689+0000 7f341f9ce740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556dc9c09370 0x556dc9c15790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:51.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.689+0000 7f341f9ce740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556dc9c09c90 0x556dc9c17de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:51.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.689+0000 7f341f9ce740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556dc9c18890 0x556dc9c04850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:51.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.689+0000 7f341f9ce740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f34181abb60 con 0x556dc9c18890 2024-04-23T10:33:51.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.689+0000 7f341f9ce740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f34181ad6a0 con 0x556dc9c09370 2024-04-23T10:33:51.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.689+0000 7f341f9ce740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f34181ac230 con 0x556dc9c09c90 2024-04-23T10:33:51.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.689+0000 7f3416d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556dc9c18890 0x556dc9c04850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:51.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.689+0000 7f3416d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556dc9c18890 0x556dc9c04850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36758/0 (socket says 172.21.15.97:36758) 2024-04-23T10:33:51.692 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.689+0000 7f3416d9d640 1 -- 172.21.15.97:0/4149943194 learned_addr learned my addr 172.21.15.97:0/4149943194 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:33:51.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.689+0000 7f341659c640 1 --2- 172.21.15.97:0/4149943194 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556dc9c09370 0x556dc9c15790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:51.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.689+0000 7f3416d9d640 1 -- 172.21.15.97:0/4149943194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556dc9c09c90 msgr2=0x556dc9c17de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:33:51.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.689+0000 7f3415d9b640 1 --2- 172.21.15.97:0/4149943194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556dc9c09c90 0x556dc9c17de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:51.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.689+0000 7f3416d9d640 1 --2- 172.21.15.97:0/4149943194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556dc9c09c90 0x556dc9c17de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:51.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.689+0000 7f3416d9d640 1 -- 172.21.15.97:0/4149943194 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556dc9c09370 msgr2=0x556dc9c15790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:33:51.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.689+0000 7f3416d9d640 1 --2- 172.21.15.97:0/4149943194 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556dc9c09370 0x556dc9c15790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:51.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.689+0000 7f3416d9d640 1 -- 172.21.15.97:0/4149943194 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556dc9c11180 con 0x556dc9c18890 2024-04-23T10:33:51.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.689+0000 7f341659c640 1 --2- 172.21.15.97:0/4149943194 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556dc9c09370 0x556dc9c15790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:33:51.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.689+0000 7f3415d9b640 1 --2- 172.21.15.97:0/4149943194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556dc9c09c90 0x556dc9c17de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:33:51.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.689+0000 7f3416d9d640 1 --2- 172.21.15.97:0/4149943194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556dc9c18890 0x556dc9c04850 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f3410005cc0 tx=0x7f341002f540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ee437ff36a1564a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:33:51.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.689+0000 7f33fffff640 1 -- 172.21.15.97:0/4149943194 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341006a020 con 0x556dc9c18890 2024-04-23T10:33:51.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.689+0000 7f33fffff640 1 -- 172.21.15.97:0/4149943194 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3410040070 con 0x556dc9c18890 2024-04-23T10:33:51.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.689+0000 7f33fffff640 1 -- 172.21.15.97:0/4149943194 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3410004470 con 0x556dc9c18890 2024-04-23T10:33:51.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.690+0000 7f341f9ce740 1 -- 172.21.15.97:0/4149943194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556dc9c18890 msgr2=0x556dc9c04850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:51.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.690+0000 7f341f9ce740 1 --2- 172.21.15.97:0/4149943194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556dc9c18890 0x556dc9c04850 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f3410005cc0 tx=0x7f341002f540 comp rx=0 tx=0).stop 2024-04-23T10:33:51.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.690+0000 7f341f9ce740 1 -- 172.21.15.97:0/4149943194 shutdown_connections 2024-04-23T10:33:51.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.690+0000 7f341f9ce740 1 --2- 172.21.15.97:0/4149943194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556dc9c18890 0x556dc9c04850 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:51.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.690+0000 7f341f9ce740 1 --2- 172.21.15.97:0/4149943194 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556dc9c09c90 0x556dc9c17de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:51.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.690+0000 7f341f9ce740 1 --2- 172.21.15.97:0/4149943194 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556dc9c09370 0x556dc9c15790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:51.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.690+0000 7f341f9ce740 1 -- 172.21.15.97:0/4149943194 >> 172.21.15.97:0/4149943194 conn(0x556dc9c057b0 msgr2=0x556dc9c04140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:33:51.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.690+0000 7f341f9ce740 1 -- 172.21.15.97:0/4149943194 shutdown_connections 2024-04-23T10:33:51.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.690+0000 7f341f9ce740 1 -- 172.21.15.97:0/4149943194 wait complete. 2024-04-23T10:33:51.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.690+0000 7f341f9ce740 1 -- start start 2024-04-23T10:33:51.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.690+0000 7f341f9ce740 1 Processor -- start 2024-04-23T10:33:51.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.690+0000 7f341f9ce740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556dc9c09370 0x556dc9c132d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:51.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.690+0000 7f341f9ce740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556dc9c09c90 0x556dc9c20a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:51.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.690+0000 7f341f9ce740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556dc9c18890 0x556dc9c19a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:51.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.690+0000 7f341f9ce740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f34181acfd0 con 0x556dc9c18890 2024-04-23T10:33:51.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.690+0000 7f341f9ce740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f34181ac900 con 0x556dc9c09c90 2024-04-23T10:33:51.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.690+0000 7f341f9ce740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f34181a9b60 con 0x556dc9c09370 2024-04-23T10:33:51.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.691+0000 7f341f9ce740 10 client.0 _ll_register_callbacks cb 0x7f34100098f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:33:51.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.691+0000 7f3416d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556dc9c18890 0x556dc9c19a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:51.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.691+0000 7f3416d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556dc9c18890 0x556dc9c19a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36768/0 (socket says 172.21.15.97:36768) 2024-04-23T10:33:51.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.691+0000 7f3416d9d640 1 -- 172.21.15.97:0/3720547411 learned_addr learned my addr 172.21.15.97:0/3720547411 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:33:51.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.691+0000 7f341659c640 1 --2- 172.21.15.97:0/3720547411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556dc9c09370 0x556dc9c132d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:51.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.691+0000 7f3416d9d640 1 -- 172.21.15.97:0/3720547411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556dc9c09370 msgr2=0x556dc9c132d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:33:51.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.691+0000 7f3415d9b640 1 --2- 172.21.15.97:0/3720547411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556dc9c09c90 0x556dc9c20a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:51.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.691+0000 7f3416d9d640 1 --2- 172.21.15.97:0/3720547411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556dc9c09370 0x556dc9c132d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:51.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.691+0000 7f3416d9d640 1 -- 172.21.15.97:0/3720547411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556dc9c09c90 msgr2=0x556dc9c20a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:33:51.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.691+0000 7f3416d9d640 1 --2- 172.21.15.97:0/3720547411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556dc9c09c90 0x556dc9c20a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:51.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.691+0000 7f3416d9d640 1 -- 172.21.15.97:0/3720547411 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556dc9c19f80 con 0x556dc9c18890 2024-04-23T10:33:51.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.691+0000 7f3415d9b640 1 --2- 172.21.15.97:0/3720547411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556dc9c09c90 0x556dc9c20a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:33:51.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.691+0000 7f3416d9d640 1 --2- 172.21.15.97:0/3720547411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556dc9c18890 0x556dc9c19a40 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f3410030af0 tx=0x7f341003d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=472a4050b121185d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:33:51.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.691+0000 7f33fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:33:51.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.691+0000 7f33fdffb640 1 -- 172.21.15.97:0/3720547411 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3410074060 con 0x556dc9c18890 2024-04-23T10:33:51.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.691+0000 7f33fdffb640 1 -- 172.21.15.97:0/3720547411 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7f3410058040 con 0x556dc9c18890 2024-04-23T10:33:51.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.691+0000 7f341f9ce740 1 -- 172.21.15.97:0/3720547411 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3410040070 con 0x556dc9c18890 2024-04-23T10:33:51.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.691+0000 7f33fdffb640 1 -- 172.21.15.97:0/3720547411 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f341006a030 con 0x556dc9c18890 2024-04-23T10:33:51.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.691+0000 7f341f9ce740 10 client.16635 fetch_fsmap learned FSMap version 26 2024-04-23T10:33:51.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.691+0000 7f341f9ce740 1 -- 172.21.15.97:0/3720547411 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556dc9c8f160 con 0x556dc9c18890 2024-04-23T10:33:51.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.691+0000 7f33fdffb640 1 -- 172.21.15.97:0/3720547411 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3410040070 con 0x556dc9c18890 2024-04-23T10:33:51.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.692+0000 7f33fdffb640 1 -- 172.21.15.97:0/3720547411 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3410052480 con 0x556dc9c18890 2024-04-23T10:33:51.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.692+0000 7f341f9ce740 10 client.16635 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:33:51.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.692+0000 7f341f9ce740 10 client.16635 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:33:51.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.692+0000 7f341f9ce740 20 client.16635 populate_metadata read hostname 'smithi097' 2024-04-23T10:33:51.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.692+0000 7f341f9ce740 1 --2- 172.21.15.97:0/3720547411 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x556dc9c8f710 0x556dc9c91b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:51.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.692+0000 7f341f9ce740 4 client.16635 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "af277cb5", "release": true, "await": true}] 2024-04-23T10:33:51.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.692+0000 7f341f9ce740 1 -- 172.21.15.97:0/3720547411 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "quiesce db", "set_id": "af277cb5", "release": true, "await": true}) -- 0x556dc9b7c0f0 con 0x556dc9c8f710 2024-04-23T10:33:51.696 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.692+0000 7f341659c640 1 --2- 172.21.15.97:0/3720547411 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x556dc9c8f710 0x556dc9c91b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:51.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.692+0000 7f33fdffb640 10 client.16635 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:33:51.697 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.692+0000 7f341659c640 1 --2- 172.21.15.97:0/3720547411 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x556dc9c8f710 0x556dc9c91b00 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef2f4349d31c18df server_cookie=83a2ad99cbde0d70 in_seq=0 out_seq=0 2024-04-23T10:33:51.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.696+0000 7f33fdffb640 1 -- 172.21.15.97:0/3720547411 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x556dc9b7c0f0 con 0x556dc9c8f710 2024-04-23T10:33:51.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.696+0000 7f33fdffb640 10 client.16635 handle_command_reply: tid=0 2024-04-23T10:33:51.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.696+0000 7f341f9ce740 1 client.16635 shutdown 2024-04-23T10:33:51.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.696+0000 7f341f9ce740 1 -- 172.21.15.97:0/3720547411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556dc9c18890 msgr2=0x556dc9c19a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:51.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.696+0000 7f341f9ce740 1 --2- 172.21.15.97:0/3720547411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556dc9c18890 0x556dc9c19a40 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f3410030af0 tx=0x7f341003d000 comp rx=0 tx=0).stop 2024-04-23T10:33:51.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.696+0000 7f341f9ce740 1 -- 172.21.15.97:0/3720547411 shutdown_connections 2024-04-23T10:33:51.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.696+0000 7f341f9ce740 1 --2- 172.21.15.97:0/3720547411 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x556dc9c8f710 0x556dc9c91b00 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:51.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.696+0000 7f341f9ce740 1 --2- 172.21.15.97:0/3720547411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556dc9c18890 0x556dc9c19a40 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:51.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.696+0000 7f33fdffb640 0 client.16635 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:33:51.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.696+0000 7f341f9ce740 1 --2- 172.21.15.97:0/3720547411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556dc9c09c90 0x556dc9c20a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:51.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.696+0000 7f341f9ce740 1 --2- 172.21.15.97:0/3720547411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556dc9c09370 0x556dc9c132d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:51.698 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.696+0000 7f341f9ce740 1 -- 172.21.15.97:0/3720547411 >> 172.21.15.97:0/3720547411 conn(0x556dc9c057b0 msgr2=0x556dc9c049d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:33:51.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.696+0000 7f341f9ce740 1 -- 172.21.15.97:0/3720547411 shutdown_connections 2024-04-23T10:33:51.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.696+0000 7f341f9ce740 1 -- 172.21.15.97:0/3720547411 wait complete. 2024-04-23T10:33:51.699 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.696+0000 7f341f9ce740 20 client.16635 trim_cache size 0 max 16384 2024-04-23T10:33:51.699 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-23T10:33:51.699 INFO:teuthology.orchestra.run.smithi097.stdout: "epoch": 26, 2024-04-23T10:33:51.699 INFO:teuthology.orchestra.run.smithi097.stdout: "leader": 24488, 2024-04-23T10:33:51.699 INFO:teuthology.orchestra.run.smithi097.stdout: "set_version": 28, 2024-04-23T10:33:51.699 INFO:teuthology.orchestra.run.smithi097.stdout: "sets": { 2024-04-23T10:33:51.699 INFO:teuthology.orchestra.run.smithi097.stdout: "af277cb5": { 2024-04-23T10:33:51.699 INFO:teuthology.orchestra.run.smithi097.stdout: "version": 28, 2024-04-23T10:33:51.699 INFO:teuthology.orchestra.run.smithi097.stdout: "age_ref": 0.0, 2024-04-23T10:33:51.699 INFO:teuthology.orchestra.run.smithi097.stdout: "state": { 2024-04-23T10:33:51.700 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RELEASED", 2024-04-23T10:33:51.700 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0 2024-04-23T10:33:51.700 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:33:51.700 INFO:teuthology.orchestra.run.smithi097.stdout: "timeout": 60.0, 2024-04-23T10:33:51.700 INFO:teuthology.orchestra.run.smithi097.stdout: "expiration": 75.3, 2024-04-23T10:33:51.700 INFO:teuthology.orchestra.run.smithi097.stdout: "members": { 2024-04-23T10:33:51.700 INFO:teuthology.orchestra.run.smithi097.stdout: "file:/": { 2024-04-23T10:33:51.700 INFO:teuthology.orchestra.run.smithi097.stdout: "excluded": false, 2024-04-23T10:33:51.700 INFO:teuthology.orchestra.run.smithi097.stdout: "state": { 2024-04-23T10:33:51.700 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RELEASED", 2024-04-23T10:33:51.700 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0 2024-04-23T10:33:51.700 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:33:51.700 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:33:51.700 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:33:51.700 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:33:51.701 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:33:51.701 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-23T10:33:51.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.696+0000 7f341ccbb640 1 -- 172.21.15.97:0/3007345440 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f34040a3da0 msgr2=0x7f34040a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:51.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.696+0000 7f341ccbb640 1 --2- 172.21.15.97:0/3007345440 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f34040a3da0 0x7f34040a6260 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f341004c000 tx=0x7f3410049000 comp rx=0 tx=0).stop 2024-04-23T10:33:51.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.696+0000 7f341ccbb640 1 -- 172.21.15.97:0/3007345440 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3418102190 msgr2=0x7f341819e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:51.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.696+0000 7f341ccbb640 1 --2- 172.21.15.97:0/3007345440 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3418102190 0x7f341819e610 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f340800b4a0 tx=0x7f340800b8a0 comp rx=0 tx=0).stop 2024-04-23T10:33:51.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.696+0000 7f341ccbb640 1 -- 172.21.15.97:0/3007345440 shutdown_connections 2024-04-23T10:33:51.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.696+0000 7f341ccbb640 1 --2- 172.21.15.97:0/3007345440 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f34040a3da0 0x7f34040a6260 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:51.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.696+0000 7f341ccbb640 1 --2- 172.21.15.97:0/3007345440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3418198720 0x7f3418198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:51.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.696+0000 7f341ccbb640 1 --2- 172.21.15.97:0/3007345440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3418107260 0x7f341819eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:51.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.696+0000 7f341ccbb640 1 --2- 172.21.15.97:0/3007345440 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3418102190 0x7f341819e610 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:51.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.696+0000 7f341ccbb640 1 -- 172.21.15.97:0/3007345440 >> 172.21.15.97:0/3007345440 conn(0x7f34180793a0 msgr2=0x7f3418100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:33:51.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.696+0000 7f341ccbb640 1 -- 172.21.15.97:0/3007345440 shutdown_connections 2024-04-23T10:33:51.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:33:51.696+0000 7f341ccbb640 1 -- 172.21.15.97:0/3007345440 wait complete. 2024-04-23T10:33:51.706 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'af277cb5', seconds elapsed: 16.3 2024-04-23T10:33:52.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:33:52 smithi196 ceph-mon[32919]: pgmap v235: 97 pgs: 97 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.6 KiB/s wr, 11 op/s 2024-04-23T10:33:52.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:33:52 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/3781035693' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:33:52.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:33:52 smithi097 ceph-mon[29197]: pgmap v235: 97 pgs: 97 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.6 KiB/s wr, 11 op/s 2024-04-23T10:33:52.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:33:52 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/3781035693' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:33:52.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:33:52 smithi183 ceph-mon[32317]: pgmap v235: 97 pgs: 97 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 7.6 KiB/s wr, 11 op/s 2024-04-23T10:33:52.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:33:52 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/3781035693' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:33:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.700+0000 7f81954cb640 1 -- 172.21.15.97:0/283954315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f819010c520 msgr2=0x7f819010e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:53.702 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.700+0000 7f81954cb640 1 --2- 172.21.15.97:0/283954315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f819010c520 0x7f819010e9c0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f8188009ca0 tx=0x7f818802a9e0 comp rx=0 tx=0).stop 2024-04-23T10:33:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.700+0000 7f81954cb640 1 -- 172.21.15.97:0/283954315 shutdown_connections 2024-04-23T10:33:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.700+0000 7f81954cb640 1 --2- 172.21.15.97:0/283954315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f819010c520 0x7f819010e9c0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.700+0000 7f81954cb640 1 --2- 172.21.15.97:0/283954315 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8190103b00 0x7f819010bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.700+0000 7f81954cb640 1 --2- 172.21.15.97:0/283954315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f81901031e0 0x7f81901035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.700+0000 7f81954cb640 1 -- 172.21.15.97:0/283954315 >> 172.21.15.97:0/283954315 conn(0x7f8190077fd0 msgr2=0x7f81900783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:33:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.701+0000 7f81954cb640 1 -- 172.21.15.97:0/283954315 shutdown_connections 2024-04-23T10:33:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.701+0000 7f81954cb640 1 -- 172.21.15.97:0/283954315 wait complete. 2024-04-23T10:33:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.701+0000 7f81954cb640 1 Processor -- start 2024-04-23T10:33:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.701+0000 7f81954cb640 1 -- start start 2024-04-23T10:33:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.701+0000 7f81954cb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81901031e0 0x7f8190198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:53.703 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.701+0000 7f81954cb640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8190103b00 0x7f8190198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.701+0000 7f81954cb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f819010c520 0x7f819019cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.701+0000 7f81954cb640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f81901121b0 con 0x7f81901031e0 2024-04-23T10:33:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.701+0000 7f81954cb640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8190112030 con 0x7f8190103b00 2024-04-23T10:33:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.701+0000 7f81954cb640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8190112330 con 0x7f819010c520 2024-04-23T10:33:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.701+0000 7f818effd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81901031e0 0x7f8190198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.702+0000 7f818effd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81901031e0 0x7f8190198690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36790/0 (socket says 172.21.15.97:36790) 2024-04-23T10:33:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.702+0000 7f818effd640 1 -- 172.21.15.97:0/4072097900 learned_addr learned my addr 172.21.15.97:0/4072097900 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:33:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.702+0000 7f8187fff640 1 --2- 172.21.15.97:0/4072097900 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8190103b00 0x7f8190198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.702+0000 7f818f7fe640 1 --2- 172.21.15.97:0/4072097900 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f819010c520 0x7f819019cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.702+0000 7f818effd640 1 -- 172.21.15.97:0/4072097900 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f819010c520 msgr2=0x7f819019cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.702+0000 7f818effd640 1 --2- 172.21.15.97:0/4072097900 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f819010c520 0x7f819019cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:53.704 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.702+0000 7f818effd640 1 -- 172.21.15.97:0/4072097900 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8190103b00 msgr2=0x7f8190198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.702+0000 7f818effd640 1 --2- 172.21.15.97:0/4072097900 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8190103b00 0x7f8190198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.702+0000 7f818effd640 1 -- 172.21.15.97:0/4072097900 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f819019d680 con 0x7f81901031e0 2024-04-23T10:33:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.702+0000 7f818effd640 1 --2- 172.21.15.97:0/4072097900 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81901031e0 0x7f8190198690 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f818000e9f0 tx=0x7f818000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:33:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.702+0000 7f818cff9640 1 -- 172.21.15.97:0/4072097900 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f818000cd80 con 0x7f81901031e0 2024-04-23T10:33:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.702+0000 7f818cff9640 1 -- 172.21.15.97:0/4072097900 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81800054d0 con 0x7f81901031e0 2024-04-23T10:33:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.702+0000 7f81954cb640 1 -- 172.21.15.97:0/4072097900 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f819019d970 con 0x7f81901031e0 2024-04-23T10:33:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.702+0000 7f818cff9640 1 -- 172.21.15.97:0/4072097900 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8180010690 con 0x7f81901031e0 2024-04-23T10:33:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.702+0000 7f81954cb640 1 -- 172.21.15.97:0/4072097900 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81901a5200 con 0x7f81901031e0 2024-04-23T10:33:53.713 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.704+0000 7f818cff9640 1 -- 172.21.15.97:0/4072097900 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8180004e90 con 0x7f81901031e0 2024-04-23T10:33:53.713 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.705+0000 7f81981db740 1 Processor -- start 2024-04-23T10:33:53.713 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.705+0000 7f81981db740 1 -- start start 2024-04-23T10:33:53.713 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.705+0000 7f818cff9640 1 --2- 172.21.15.97:0/4072097900 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f817c0a3dc0 0x7f817c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:53.713 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.705+0000 7f818f7fe640 1 --2- 172.21.15.97:0/4072097900 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f817c0a3dc0 0x7f817c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:53.713 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.705+0000 7f818cff9640 1 -- 172.21.15.97:0/4072097900 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7f81800d0c80 con 0x7f81901031e0 2024-04-23T10:33:53.713 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.705+0000 7f81981db740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f52ae87520 0x55f52ae87900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:53.713 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.705+0000 7f81981db740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f52ae87e40 0x55f52ae966d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:53.714 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.705+0000 7f81981db740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f52ae96c10 0x55f52ae99000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:53.714 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.705+0000 7f81981db740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f52adec970 con 0x55f52ae96c10 2024-04-23T10:33:53.714 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.705+0000 7f81981db740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f52ad894a0 con 0x55f52ae87520 2024-04-23T10:33:53.714 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.705+0000 7f81981db740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f52ade8930 con 0x55f52ae87e40 2024-04-23T10:33:53.714 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.706+0000 7f818f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f52ae96c10 0x55f52ae99000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:53.714 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.706+0000 7f818f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f52ae96c10 0x55f52ae99000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36800/0 (socket says 172.21.15.97:36800) 2024-04-23T10:33:53.714 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.706+0000 7f818f7fe640 1 -- 172.21.15.97:0/3119562683 learned_addr learned my addr 172.21.15.97:0/3119562683 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:33:53.714 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.706+0000 7f8187fff640 1 --2- 172.21.15.97:0/3119562683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f52ae87e40 0x55f52ae966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:53.714 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.706+0000 7f818effd640 1 --2- 172.21.15.97:0/3119562683 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f52ae87520 0x55f52ae87900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:53.714 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.706+0000 7f818f7fe640 1 --2- 172.21.15.97:0/4072097900 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f817c0a3dc0 0x7f817c0a6280 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f818802af00 tx=0x7f818802ee70 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:33:53.714 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.706+0000 7f818f7fe640 1 -- 172.21.15.97:0/3119562683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f52ae87e40 msgr2=0x55f52ae966d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:33:53.715 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.706+0000 7f818f7fe640 1 --2- 172.21.15.97:0/3119562683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f52ae87e40 0x55f52ae966d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:53.715 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.706+0000 7f818f7fe640 1 -- 172.21.15.97:0/3119562683 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f52ae87520 msgr2=0x55f52ae87900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:33:53.715 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.706+0000 7f818f7fe640 1 --2- 172.21.15.97:0/3119562683 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f52ae87520 0x55f52ae87900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:53.715 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.706+0000 7f818f7fe640 1 -- 172.21.15.97:0/3119562683 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f52aaff210 con 0x55f52ae96c10 2024-04-23T10:33:53.715 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.706+0000 7f8187fff640 1 --2- 172.21.15.97:0/3119562683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f52ae87e40 0x55f52ae966d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T10:33:53.715 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.706+0000 7f818effd640 1 --2- 172.21.15.97:0/3119562683 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f52ae87520 0x55f52ae87900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T10:33:53.715 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.706+0000 7f818f7fe640 1 --2- 172.21.15.97:0/3119562683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f52ae96c10 0x55f52ae99000 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f8188009ca0 tx=0x7f8188049000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46844e26963d998f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:33:53.715 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.706+0000 7f81857fa640 1 -- 172.21.15.97:0/3119562683 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8188033040 con 0x55f52ae96c10 2024-04-23T10:33:53.715 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.706+0000 7f81857fa640 1 -- 172.21.15.97:0/3119562683 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f818804d070 con 0x55f52ae96c10 2024-04-23T10:33:53.715 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.706+0000 7f81857fa640 1 -- 172.21.15.97:0/3119562683 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8188004580 con 0x55f52ae96c10 2024-04-23T10:33:53.715 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.707+0000 7f81981db740 1 -- 172.21.15.97:0/3119562683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f52ae96c10 msgr2=0x55f52ae99000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:53.715 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.707+0000 7f81981db740 1 --2- 172.21.15.97:0/3119562683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f52ae96c10 0x55f52ae99000 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f8188009ca0 tx=0x7f8188049000 comp rx=0 tx=0).stop 2024-04-23T10:33:53.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.707+0000 7f81981db740 1 -- 172.21.15.97:0/3119562683 shutdown_connections 2024-04-23T10:33:53.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.707+0000 7f81981db740 1 --2- 172.21.15.97:0/3119562683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f52ae96c10 0x55f52ae99000 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:53.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.707+0000 7f81981db740 1 --2- 172.21.15.97:0/3119562683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f52ae87e40 0x55f52ae966d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:53.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.707+0000 7f81981db740 1 --2- 172.21.15.97:0/3119562683 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f52ae87520 0x55f52ae87900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:53.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.707+0000 7f81981db740 1 -- 172.21.15.97:0/3119562683 >> 172.21.15.97:0/3119562683 conn(0x55f52ae83b30 msgr2=0x55f52ae83f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:33:53.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.707+0000 7f81981db740 1 -- 172.21.15.97:0/3119562683 shutdown_connections 2024-04-23T10:33:53.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.707+0000 7f81981db740 1 -- 172.21.15.97:0/3119562683 wait complete. 2024-04-23T10:33:53.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.707+0000 7f81981db740 1 -- start start 2024-04-23T10:33:53.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.708+0000 7f81981db740 1 Processor -- start 2024-04-23T10:33:53.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.708+0000 7f81981db740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f52ae87520 0x55f52ae964a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:53.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.708+0000 7f81981db740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f52ae87e40 0x55f52ae93790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:53.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.708+0000 7f81981db740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f52ae96c10 0x55f52ae93cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.708+0000 7f81981db740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f52ae997e0 con 0x55f52ae87520 2024-04-23T10:33:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.708+0000 7f81981db740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f52ad870b0 con 0x55f52ae87e40 2024-04-23T10:33:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.708+0000 7f81981db740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f52ae99960 con 0x55f52ae96c10 2024-04-23T10:33:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.708+0000 7f81981db740 10 client.0 _ll_register_callbacks cb 0x55f52ad74d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:33:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.708+0000 7f818effd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f52ae87520 0x55f52ae964a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.708+0000 7f818effd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f52ae87520 0x55f52ae964a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36806/0 (socket says 172.21.15.97:36806) 2024-04-23T10:33:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.708+0000 7f818effd640 1 -- 172.21.15.97:0/3617665254 learned_addr learned my addr 172.21.15.97:0/3617665254 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:33:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.708+0000 7f8187fff640 1 --2- 172.21.15.97:0/3617665254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f52ae87e40 0x55f52ae93790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.708+0000 7f818f7fe640 1 --2- 172.21.15.97:0/3617665254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f52ae96c10 0x55f52ae93cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.708+0000 7f818effd640 1 -- 172.21.15.97:0/3617665254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f52ae96c10 msgr2=0x55f52ae93cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:33:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.708+0000 7f818effd640 1 --2- 172.21.15.97:0/3617665254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f52ae96c10 0x55f52ae93cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.708+0000 7f818effd640 1 -- 172.21.15.97:0/3617665254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f52ae87e40 msgr2=0x55f52ae93790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:33:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.708+0000 7f818effd640 1 --2- 172.21.15.97:0/3617665254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f52ae87e40 0x55f52ae93790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.708+0000 7f818effd640 1 -- 172.21.15.97:0/3617665254 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f52abcb620 con 0x55f52ae87520 2024-04-23T10:33:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.708+0000 7f8187fff640 1 --2- 172.21.15.97:0/3617665254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f52ae87e40 0x55f52ae93790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:33:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.708+0000 7f818effd640 1 --2- 172.21.15.97:0/3617665254 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f52ae87520 0x55f52ae964a0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f818000e9c0 tx=0x7f818009d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=518f2129d1596e66 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:33:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.708+0000 7f81767fc640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:33:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.709+0000 7f81767fc640 1 -- 172.21.15.97:0/3617665254 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81800b5020 con 0x55f52ae87520 2024-04-23T10:33:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.709+0000 7f81767fc640 1 -- 172.21.15.97:0/3617665254 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7f81800992b0 con 0x55f52ae87520 2024-04-23T10:33:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.709+0000 7f81981db740 1 -- 172.21.15.97:0/3617665254 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f818804d070 con 0x55f52ae87520 2024-04-23T10:33:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.709+0000 7f81767fc640 1 -- 172.21.15.97:0/3617665254 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8180066050 con 0x55f52ae87520 2024-04-23T10:33:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.709+0000 7f81981db740 10 client.16647 fetch_fsmap learned FSMap version 26 2024-04-23T10:33:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.709+0000 7f81981db740 1 -- 172.21.15.97:0/3617665254 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f52aea5700 con 0x55f52ae87520 2024-04-23T10:33:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.709+0000 7f81767fc640 1 -- 172.21.15.97:0/3617665254 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f818804d070 con 0x55f52ae87520 2024-04-23T10:33:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.709+0000 7f81767fc640 1 -- 172.21.15.97:0/3617665254 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f81800a5420 con 0x55f52ae87520 2024-04-23T10:33:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.709+0000 7f81981db740 10 client.16647 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:33:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.709+0000 7f81981db740 10 client.16647 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:33:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.709+0000 7f81981db740 20 client.16647 populate_metadata read hostname 'smithi097' 2024-04-23T10:33:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.709+0000 7f81981db740 1 --2- 172.21.15.97:0/3617665254 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55f52aea6bc0 0x55f52aea8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.709+0000 7f81981db740 4 client.16647 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:33:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.709+0000 7f81981db740 1 -- 172.21.15.97:0/3617665254 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f52adfa390 con 0x55f52aea6bc0 2024-04-23T10:33:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.709+0000 7f8187fff640 1 --2- 172.21.15.97:0/3617665254 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55f52aea6bc0 0x55f52aea8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.710+0000 7f81767fc640 10 client.16647 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:33:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.710+0000 7f8187fff640 1 --2- 172.21.15.97:0/3617665254 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55f52aea6bc0 0x55f52aea8fb0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f649d52d7971b66b server_cookie=3c8ca10d774b6f in_seq=0 out_seq=0 2024-04-23T10:33:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.710+0000 7f81767fc640 1 -- 172.21.15.97:0/3617665254 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f52adfa390 con 0x55f52aea6bc0 2024-04-23T10:33:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.710+0000 7f81767fc640 10 client.16647 handle_command_reply: tid=0 2024-04-23T10:33:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.711+0000 7f81981db740 1 client.16647 shutdown 2024-04-23T10:33:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.711+0000 7f81981db740 1 -- 172.21.15.97:0/3617665254 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f52ae87520 msgr2=0x55f52ae964a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.711+0000 7f81981db740 1 --2- 172.21.15.97:0/3617665254 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f52ae87520 0x55f52ae964a0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f818000e9c0 tx=0x7f818009d340 comp rx=0 tx=0).stop 2024-04-23T10:33:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.711+0000 7f81981db740 1 -- 172.21.15.97:0/3617665254 shutdown_connections 2024-04-23T10:33:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.711+0000 7f81981db740 1 --2- 172.21.15.97:0/3617665254 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55f52aea6bc0 0x55f52aea8fb0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.711+0000 7f81981db740 1 --2- 172.21.15.97:0/3617665254 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f52ae96c10 0x55f52ae93cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.711+0000 7f81981db740 1 --2- 172.21.15.97:0/3617665254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f52ae87e40 0x55f52ae93790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.711+0000 7f81767fc640 0 client.16647 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:33:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.711+0000 7f81981db740 1 --2- 172.21.15.97:0/3617665254 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f52ae87520 0x55f52ae964a0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.711+0000 7f81981db740 1 -- 172.21.15.97:0/3617665254 >> 172.21.15.97:0/3617665254 conn(0x55f52ae83b30 msgr2=0x55f52ae899a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:33:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.711+0000 7f81981db740 1 -- 172.21.15.97:0/3617665254 shutdown_connections 2024-04-23T10:33:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.711+0000 7f81981db740 1 -- 172.21.15.97:0/3617665254 wait complete. 2024-04-23T10:33:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.711+0000 7f81981db740 20 client.16647 trim_cache size 0 max 16384 2024-04-23T10:33:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.726+0000 7f81981db740 1 Processor -- start 2024-04-23T10:33:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.726+0000 7f81981db740 1 -- start start 2024-04-23T10:33:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.726+0000 7f81981db740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f52ae87520 0x55f52aea26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.726+0000 7f81981db740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f52ae87e40 0x55f52ae8ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.726+0000 7f81981db740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f52ae96c10 0x55f52ae95fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.726+0000 7f81981db740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f81901ab8b0 con 0x55f52ae96c10 2024-04-23T10:33:53.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.726+0000 7f81981db740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f81901ad3f0 con 0x55f52ae87520 2024-04-23T10:33:53.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.726+0000 7f81981db740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f81901abf80 con 0x55f52ae87e40 2024-04-23T10:33:53.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.726+0000 7f818f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f52ae96c10 0x55f52ae95fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:53.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.726+0000 7f818f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f52ae96c10 0x55f52ae95fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36818/0 (socket says 172.21.15.97:36818) 2024-04-23T10:33:53.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.726+0000 7f818f7fe640 1 -- 172.21.15.97:0/2858807700 learned_addr learned my addr 172.21.15.97:0/2858807700 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:33:53.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.727+0000 7f818effd640 1 --2- 172.21.15.97:0/2858807700 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f52ae87520 0x55f52aea26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:53.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.727+0000 7f8187fff640 1 --2- 172.21.15.97:0/2858807700 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f52ae87e40 0x55f52ae8ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:53.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.727+0000 7f818f7fe640 1 -- 172.21.15.97:0/2858807700 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f52ae87e40 msgr2=0x55f52ae8ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:33:53.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.727+0000 7f818f7fe640 1 --2- 172.21.15.97:0/2858807700 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f52ae87e40 0x55f52ae8ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:53.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.727+0000 7f818f7fe640 1 -- 172.21.15.97:0/2858807700 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f52ae87520 msgr2=0x55f52aea26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:33:53.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.727+0000 7f818f7fe640 1 --2- 172.21.15.97:0/2858807700 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f52ae87520 0x55f52aea26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:53.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.727+0000 7f818f7fe640 1 -- 172.21.15.97:0/2858807700 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f52aea9590 con 0x55f52ae96c10 2024-04-23T10:33:53.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.727+0000 7f8187fff640 1 --2- 172.21.15.97:0/2858807700 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f52ae87e40 0x55f52ae8ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:33:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.727+0000 7f818f7fe640 1 --2- 172.21.15.97:0/2858807700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f52ae96c10 0x55f52ae95fd0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f818802f040 tx=0x7f81880095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=144f44776ef34787 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:33:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.727+0000 7f8184ff9640 1 -- 172.21.15.97:0/2858807700 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8188060070 con 0x55f52ae96c10 2024-04-23T10:33:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.727+0000 7f8184ff9640 1 -- 172.21.15.97:0/2858807700 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f818804d050 con 0x55f52ae96c10 2024-04-23T10:33:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.727+0000 7f8184ff9640 1 -- 172.21.15.97:0/2858807700 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8188004230 con 0x55f52ae96c10 2024-04-23T10:33:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.728+0000 7f81981db740 1 -- 172.21.15.97:0/2858807700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f52ae96c10 msgr2=0x55f52ae95fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.728+0000 7f81981db740 1 --2- 172.21.15.97:0/2858807700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f52ae96c10 0x55f52ae95fd0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f818802f040 tx=0x7f81880095a0 comp rx=0 tx=0).stop 2024-04-23T10:33:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.728+0000 7f81981db740 1 -- 172.21.15.97:0/2858807700 shutdown_connections 2024-04-23T10:33:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.728+0000 7f81981db740 1 --2- 172.21.15.97:0/2858807700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f52ae96c10 0x55f52ae95fd0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.728+0000 7f81981db740 1 --2- 172.21.15.97:0/2858807700 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f52ae87e40 0x55f52ae8ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.728+0000 7f81981db740 1 --2- 172.21.15.97:0/2858807700 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f52ae87520 0x55f52aea26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.728+0000 7f81981db740 1 -- 172.21.15.97:0/2858807700 >> 172.21.15.97:0/2858807700 conn(0x55f52ae83b30 msgr2=0x55f52ae8e670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:33:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.728+0000 7f81981db740 1 -- 172.21.15.97:0/2858807700 shutdown_connections 2024-04-23T10:33:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.728+0000 7f81981db740 1 -- 172.21.15.97:0/2858807700 wait complete. 2024-04-23T10:33:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.728+0000 7f81981db740 1 -- start start 2024-04-23T10:33:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.728+0000 7f81981db740 1 Processor -- start 2024-04-23T10:33:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.728+0000 7f81981db740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f52ae87520 0x55f52ae9ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.728+0000 7f81981db740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f52ae87e40 0x55f52ae90040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.728+0000 7f81981db740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f52ae96c10 0x55f52ae90580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.728+0000 7f81981db740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f81901acd20 con 0x55f52ae87e40 2024-04-23T10:33:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.728+0000 7f81981db740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f81901ac650 con 0x55f52ae96c10 2024-04-23T10:33:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.728+0000 7f81981db740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f81901a98b0 con 0x55f52ae87520 2024-04-23T10:33:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.728+0000 7f81981db740 10 client.0 _ll_register_callbacks cb 0x7f818009c5a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:33:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.729+0000 7f8187fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f52ae87e40 0x55f52ae90040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.729+0000 7f8187fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f52ae87e40 0x55f52ae90040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36820/0 (socket says 172.21.15.97:36820) 2024-04-23T10:33:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.729+0000 7f8187fff640 1 -- 172.21.15.97:0/2010869208 learned_addr learned my addr 172.21.15.97:0/2010869208 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:33:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.729+0000 7f818effd640 1 --2- 172.21.15.97:0/2010869208 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f52ae87520 0x55f52ae9ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.729+0000 7f818f7fe640 1 --2- 172.21.15.97:0/2010869208 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f52ae96c10 0x55f52ae90580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.729+0000 7f8187fff640 1 -- 172.21.15.97:0/2010869208 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f52ae87520 msgr2=0x55f52ae9ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:33:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.729+0000 7f8187fff640 1 --2- 172.21.15.97:0/2010869208 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f52ae87520 0x55f52ae9ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.729+0000 7f8187fff640 1 -- 172.21.15.97:0/2010869208 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f52ae96c10 msgr2=0x55f52ae90580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:33:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.729+0000 7f8187fff640 1 --2- 172.21.15.97:0/2010869208 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f52ae96c10 0x55f52ae90580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.729+0000 7f8187fff640 1 -- 172.21.15.97:0/2010869208 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f52ae83960 con 0x55f52ae87e40 2024-04-23T10:33:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.729+0000 7f818f7fe640 1 --2- 172.21.15.97:0/2010869208 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f52ae96c10 0x55f52ae90580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:33:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.729+0000 7f8187fff640 1 --2- 172.21.15.97:0/2010869208 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f52ae87e40 0x55f52ae90040 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x55f52ae92ff0 tx=0x7f8178012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f591a2238214d6ca server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:33:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.729+0000 7f81767fc640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:33:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.729+0000 7f81767fc640 1 -- 172.21.15.97:0/2010869208 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8178004250 con 0x55f52ae87e40 2024-04-23T10:33:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.729+0000 7f81767fc640 1 -- 172.21.15.97:0/2010869208 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8484+0+0 (secure 0 0 0) 0x7f81780043f0 con 0x55f52ae87e40 2024-04-23T10:33:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.729+0000 7f81981db740 1 -- 172.21.15.97:0/2010869208 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f818804d050 con 0x55f52ae87e40 2024-04-23T10:33:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.729+0000 7f81767fc640 1 -- 172.21.15.97:0/2010869208 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f817802d050 con 0x55f52ae87e40 2024-04-23T10:33:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.729+0000 7f81981db740 10 client.16653 fetch_fsmap learned FSMap version 26 2024-04-23T10:33:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.729+0000 7f81981db740 1 -- 172.21.15.97:0/2010869208 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f52af0cc80 con 0x55f52ae87e40 2024-04-23T10:33:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.730+0000 7f81767fc640 1 -- 172.21.15.97:0/2010869208 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f818804d050 con 0x55f52ae87e40 2024-04-23T10:33:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.730+0000 7f81767fc640 1 -- 172.21.15.97:0/2010869208 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8178003b60 con 0x55f52ae87e40 2024-04-23T10:33:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.730+0000 7f81981db740 10 client.16653 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:33:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.730+0000 7f81981db740 10 client.16653 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:33:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.730+0000 7f81981db740 20 client.16653 populate_metadata read hostname 'smithi097' 2024-04-23T10:33:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.730+0000 7f81981db740 1 --2- 172.21.15.97:0/2010869208 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55f52aea6bc0 0x55f52af0f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:33:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.730+0000 7f81981db740 4 client.16653 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:33:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.730+0000 7f81981db740 1 -- 172.21.15.97:0/2010869208 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f52ae94060 con 0x55f52aea6bc0 2024-04-23T10:33:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.730+0000 7f818effd640 1 --2- 172.21.15.97:0/2010869208 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55f52aea6bc0 0x55f52af0f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:33:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.730+0000 7f818effd640 1 --2- 172.21.15.97:0/2010869208 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55f52aea6bc0 0x55f52af0f4c0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f04503a844d610cd server_cookie=796ff73de5e3f6bb in_seq=0 out_seq=0 2024-04-23T10:33:53.736 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.730+0000 7f81767fc640 10 client.16653 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:33:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.742+0000 7f81767fc640 1 -- 172.21.15.97:0/2010869208 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f52ae94060 con 0x55f52aea6bc0 2024-04-23T10:33:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.742+0000 7f81767fc640 10 client.16653 handle_command_reply: tid=0 2024-04-23T10:33:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.742+0000 7f81981db740 1 client.16653 shutdown 2024-04-23T10:33:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.742+0000 7f81981db740 1 -- 172.21.15.97:0/2010869208 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f52ae87e40 msgr2=0x55f52ae90040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.742+0000 7f81981db740 1 --2- 172.21.15.97:0/2010869208 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f52ae87e40 0x55f52ae90040 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x55f52ae92ff0 tx=0x7f8178012820 comp rx=0 tx=0).stop 2024-04-23T10:33:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.742+0000 7f81981db740 1 -- 172.21.15.97:0/2010869208 shutdown_connections 2024-04-23T10:33:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.742+0000 7f81981db740 1 --2- 172.21.15.97:0/2010869208 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55f52aea6bc0 0x55f52af0f4c0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.742+0000 7f81981db740 1 --2- 172.21.15.97:0/2010869208 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f52ae96c10 0x55f52ae90580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.742+0000 7f81981db740 1 --2- 172.21.15.97:0/2010869208 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f52ae87e40 0x55f52ae90040 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.742+0000 7f81981db740 1 --2- 172.21.15.97:0/2010869208 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f52ae87520 0x55f52ae9ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.742+0000 7f81767fc640 0 client.16653 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:33:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.742+0000 7f81981db740 1 -- 172.21.15.97:0/2010869208 >> 172.21.15.97:0/2010869208 conn(0x55f52ae83b30 msgr2=0x55f52ae89070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:33:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.742+0000 7f81981db740 1 -- 172.21.15.97:0/2010869208 shutdown_connections 2024-04-23T10:33:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.742+0000 7f81981db740 1 -- 172.21.15.97:0/2010869208 wait complete. 2024-04-23T10:33:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.742+0000 7f81981db740 20 client.16653 trim_cache size 0 max 16384 2024-04-23T10:33:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:33:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:33:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:33:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:33:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.743+0000 7f81954cb640 1 -- 172.21.15.97:0/4072097900 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f817c0a3dc0 msgr2=0x7f817c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.743+0000 7f81954cb640 1 --2- 172.21.15.97:0/4072097900 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f817c0a3dc0 0x7f817c0a6280 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f818802af00 tx=0x7f818802ee70 comp rx=0 tx=0).stop 2024-04-23T10:33:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.743+0000 7f81954cb640 1 -- 172.21.15.97:0/4072097900 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81901031e0 msgr2=0x7f8190198690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:33:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.743+0000 7f81954cb640 1 --2- 172.21.15.97:0/4072097900 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81901031e0 0x7f8190198690 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f818000e9f0 tx=0x7f818000edf0 comp rx=0 tx=0).stop 2024-04-23T10:33:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.743+0000 7f81954cb640 1 -- 172.21.15.97:0/4072097900 shutdown_connections 2024-04-23T10:33:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.743+0000 7f81954cb640 1 --2- 172.21.15.97:0/4072097900 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f817c0a3dc0 0x7f817c0a6280 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.743+0000 7f81954cb640 1 --2- 172.21.15.97:0/4072097900 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f819010c520 0x7f819019cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.743+0000 7f81954cb640 1 --2- 172.21.15.97:0/4072097900 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8190103b00 0x7f8190198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.743+0000 7f81954cb640 1 --2- 172.21.15.97:0/4072097900 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81901031e0 0x7f8190198690 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:33:53.745 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.743+0000 7f81954cb640 1 -- 172.21.15.97:0/4072097900 >> 172.21.15.97:0/4072097900 conn(0x7f8190077fd0 msgr2=0x7f81900fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:33:53.745 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.743+0000 7f81954cb640 1 -- 172.21.15.97:0/4072097900 shutdown_connections 2024-04-23T10:33:53.745 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:33:53.743+0000 7f81954cb640 1 -- 172.21.15.97:0/4072097900 wait complete. 2024-04-23T10:33:54.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:33:54 smithi196 ceph-mon[32919]: pgmap v236: 97 pgs: 97 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-23T10:33:54.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:33:54 smithi097 ceph-mon[29197]: pgmap v236: 97 pgs: 97 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-23T10:33:54.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:33:54 smithi183 ceph-mon[32317]: pgmap v236: 97 pgs: 97 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-23T10:33:56.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:33:56 smithi196 ceph-mon[32919]: pgmap v237: 97 pgs: 97 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 4.3 KiB/s wr, 8 op/s 2024-04-23T10:33:56.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:33:56 smithi097 ceph-mon[29197]: pgmap v237: 97 pgs: 97 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 4.3 KiB/s wr, 8 op/s 2024-04-23T10:33:56.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:33:56 smithi183 ceph-mon[32317]: pgmap v237: 97 pgs: 97 active+clean; 1.8 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 4.3 KiB/s wr, 8 op/s 2024-04-23T10:33:58.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:33:58 smithi196 ceph-mon[32919]: pgmap v238: 97 pgs: 97 active+clean; 2.1 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 17 op/s 2024-04-23T10:33:58.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:33:58 smithi097 ceph-mon[29197]: pgmap v238: 97 pgs: 97 active+clean; 2.1 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 17 op/s 2024-04-23T10:33:58.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:33:58 smithi183 ceph-mon[32317]: pgmap v238: 97 pgs: 97 active+clean; 2.1 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 17 op/s 2024-04-23T10:34:00.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:34:00 smithi196 ceph-mon[32919]: pgmap v239: 97 pgs: 97 active+clean; 2.1 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 26 KiB/s wr, 13 op/s 2024-04-23T10:34:00.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:34:00 smithi097 ceph-mon[29197]: pgmap v239: 97 pgs: 97 active+clean; 2.1 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 26 KiB/s wr, 13 op/s 2024-04-23T10:34:00.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:34:00 smithi183 ceph-mon[32317]: pgmap v239: 97 pgs: 97 active+clean; 2.1 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 26 KiB/s wr, 13 op/s 2024-04-23T10:34:02.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:34:02 smithi196 ceph-mon[32919]: pgmap v240: 97 pgs: 97 active+clean; 2.1 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 26 KiB/s wr, 13 op/s 2024-04-23T10:34:02.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:34:02 smithi196 ceph-mon[32919]: osdmap e92: 12 total, 12 up, 12 in 2024-04-23T10:34:02.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:34:02 smithi097 ceph-mon[29197]: pgmap v240: 97 pgs: 97 active+clean; 2.1 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 26 KiB/s wr, 13 op/s 2024-04-23T10:34:02.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:34:02 smithi097 ceph-mon[29197]: osdmap e92: 12 total, 12 up, 12 in 2024-04-23T10:34:02.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:34:02 smithi183 ceph-mon[32317]: pgmap v240: 97 pgs: 97 active+clean; 2.1 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 26 KiB/s wr, 13 op/s 2024-04-23T10:34:02.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:34:02 smithi183 ceph-mon[32317]: osdmap e92: 12 total, 12 up, 12 in 2024-04-23T10:34:04.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:34:04 smithi196 ceph-mon[32919]: pgmap v242: 97 pgs: 97 active+clean; 2.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 103 KiB/s wr, 39 op/s 2024-04-23T10:34:04.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:34:04 smithi097 ceph-mon[29197]: pgmap v242: 97 pgs: 97 active+clean; 2.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 103 KiB/s wr, 39 op/s 2024-04-23T10:34:04.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:34:04 smithi183 ceph-mon[32317]: pgmap v242: 97 pgs: 97 active+clean; 2.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 103 KiB/s wr, 39 op/s 2024-04-23T10:34:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.839+0000 7f1d652ae640 1 -- 172.21.15.97:0/694077265 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d60103b00 msgr2=0x7f1d6010bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.839+0000 7f1d652ae640 1 --2- 172.21.15.97:0/694077265 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d60103b00 0x7f1d6010bfe0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f1d54009900 tx=0x7f1d5402a510 comp rx=0 tx=0).stop 2024-04-23T10:34:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.839+0000 7f1d652ae640 1 -- 172.21.15.97:0/694077265 shutdown_connections 2024-04-23T10:34:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.839+0000 7f1d652ae640 1 --2- 172.21.15.97:0/694077265 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d6010c520 0x7f1d6010e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:06.841 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.839+0000 7f1d652ae640 1 --2- 172.21.15.97:0/694077265 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d60103b00 0x7f1d6010bfe0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.839+0000 7f1d652ae640 1 --2- 172.21.15.97:0/694077265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d601031e0 0x7f1d601035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.839+0000 7f1d652ae640 1 -- 172.21.15.97:0/694077265 >> 172.21.15.97:0/694077265 conn(0x7f1d60077fd0 msgr2=0x7f1d600783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:34:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.839+0000 7f1d652ae640 1 -- 172.21.15.97:0/694077265 shutdown_connections 2024-04-23T10:34:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.839+0000 7f1d652ae640 1 -- 172.21.15.97:0/694077265 wait complete. 2024-04-23T10:34:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.840+0000 7f1d652ae640 1 Processor -- start 2024-04-23T10:34:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.840+0000 7f1d652ae640 1 -- start start 2024-04-23T10:34:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.840+0000 7f1d652ae640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d601031e0 0x7f1d60198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.840+0000 7f1d652ae640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d60103b00 0x7f1d60198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.840+0000 7f1d652ae640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d6010c520 0x7f1d6019cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.840+0000 7f1d652ae640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1d601121b0 con 0x7f1d6010c520 2024-04-23T10:34:06.842 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.840+0000 7f1d652ae640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1d60112030 con 0x7f1d601031e0 2024-04-23T10:34:06.843 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.840+0000 7f1d652ae640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1d60112330 con 0x7f1d60103b00 2024-04-23T10:34:06.843 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.841+0000 7f1d5f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d6010c520 0x7f1d6019cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:06.843 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.841+0000 7f1d5f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d6010c520 0x7f1d6019cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57756/0 (socket says 172.21.15.97:57756) 2024-04-23T10:34:06.843 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.841+0000 7f1d5f7fe640 1 -- 172.21.15.97:0/294020398 learned_addr learned my addr 172.21.15.97:0/294020398 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:34:06.843 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.841+0000 7f1d5e7fc640 1 --2- 172.21.15.97:0/294020398 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d60103b00 0x7f1d60198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:06.843 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.841+0000 7f1d5effd640 1 --2- 172.21.15.97:0/294020398 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d601031e0 0x7f1d60198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:06.843 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.841+0000 7f1d5f7fe640 1 -- 172.21.15.97:0/294020398 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d60103b00 msgr2=0x7f1d60198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:06.843 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.841+0000 7f1d5f7fe640 1 --2- 172.21.15.97:0/294020398 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d60103b00 0x7f1d60198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:06.843 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.841+0000 7f1d5f7fe640 1 -- 172.21.15.97:0/294020398 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d601031e0 msgr2=0x7f1d60198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:06.843 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.841+0000 7f1d5f7fe640 1 --2- 172.21.15.97:0/294020398 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d601031e0 0x7f1d60198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:06.843 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.841+0000 7f1d5f7fe640 1 -- 172.21.15.97:0/294020398 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d6019d6f0 con 0x7f1d6010c520 2024-04-23T10:34:06.843 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.841+0000 7f1d5effd640 1 --2- 172.21.15.97:0/294020398 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d601031e0 0x7f1d60198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:34:06.843 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.841+0000 7f1d5f7fe640 1 --2- 172.21.15.97:0/294020398 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d6010c520 0x7f1d6019cfc0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f1d5000cad0 tx=0x7f1d5000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:34:06.844 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.841+0000 7f1d47fff640 1 -- 172.21.15.97:0/294020398 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5000ae20 con 0x7f1d6010c520 2024-04-23T10:34:06.844 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.841+0000 7f1d47fff640 1 -- 172.21.15.97:0/294020398 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d50005020 con 0x7f1d6010c520 2024-04-23T10:34:06.844 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.841+0000 7f1d652ae640 1 -- 172.21.15.97:0/294020398 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d6019d9e0 con 0x7f1d6010c520 2024-04-23T10:34:06.844 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.841+0000 7f1d47fff640 1 -- 172.21.15.97:0/294020398 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5000f6b0 con 0x7f1d6010c520 2024-04-23T10:34:06.844 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.841+0000 7f1d652ae640 1 -- 172.21.15.97:0/294020398 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d601a5270 con 0x7f1d6010c520 2024-04-23T10:34:06.851 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.844+0000 7f1d47fff640 1 -- 172.21.15.97:0/294020398 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1d500051d0 con 0x7f1d6010c520 2024-04-23T10:34:06.851 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.844+0000 7f1d67fca740 1 Processor -- start 2024-04-23T10:34:06.852 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.844+0000 7f1d67fca740 1 -- start start 2024-04-23T10:34:06.852 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.845+0000 7f1d67fca740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5568ddb27620 0x5568ddb27a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:06.852 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.845+0000 7f1d67fca740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568ddb27f40 0x5568ddb322c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:06.852 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.845+0000 7f1d67fca740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568ddb32800 0x5568ddb34bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:06.852 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.845+0000 7f1d67fca740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5568dda8c970 con 0x5568ddb32800 2024-04-23T10:34:06.852 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.845+0000 7f1d67fca740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5568dda294a0 con 0x5568ddb27620 2024-04-23T10:34:06.852 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.845+0000 7f1d67fca740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5568dda88930 con 0x5568ddb27f40 2024-04-23T10:34:06.852 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.845+0000 7f1d47fff640 1 --2- 172.21.15.97:0/294020398 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f1d480a3dc0 0x7f1d480a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:06.852 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.845+0000 7f1d5e7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568ddb32800 0x5568ddb34bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:06.852 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.845+0000 7f1d5e7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568ddb32800 0x5568ddb34bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57764/0 (socket says 172.21.15.97:57764) 2024-04-23T10:34:06.852 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.845+0000 7f1d5e7fc640 1 -- 172.21.15.97:0/4046349948 learned_addr learned my addr 172.21.15.97:0/4046349948 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:34:06.852 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.845+0000 7f1d5effd640 1 --2- 172.21.15.97:0/4046349948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568ddb27f40 0x5568ddb322c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:06.853 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.845+0000 7f1d5f7fe640 1 --2- 172.21.15.97:0/294020398 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f1d480a3dc0 0x7f1d480a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:06.853 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.845+0000 7f1d5e7fc640 1 --2- 172.21.15.97:0/4046349948 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5568ddb27620 0x5568ddb27a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:06.853 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.845+0000 7f1d5e7fc640 1 -- 172.21.15.97:0/4046349948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568ddb27f40 msgr2=0x5568ddb322c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:34:06.853 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.845+0000 7f1d5e7fc640 1 --2- 172.21.15.97:0/4046349948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568ddb27f40 0x5568ddb322c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:06.853 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.845+0000 7f1d47fff640 1 -- 172.21.15.97:0/294020398 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7f1d500d0820 con 0x7f1d6010c520 2024-04-23T10:34:06.853 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.845+0000 7f1d5e7fc640 1 -- 172.21.15.97:0/4046349948 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5568ddb27620 msgr2=0x5568ddb27a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:34:06.853 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.845+0000 7f1d5e7fc640 1 --2- 172.21.15.97:0/4046349948 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5568ddb27620 0x5568ddb27a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:06.853 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.845+0000 7f1d5e7fc640 1 -- 172.21.15.97:0/4046349948 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568dd79f210 con 0x5568ddb32800 2024-04-23T10:34:06.853 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.845+0000 7f1d5e7fc640 1 --2- 172.21.15.97:0/4046349948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568ddb32800 0x5568ddb34bf0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f1d54009900 tx=0x7f1d54005e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b981d36e8ebb74ea server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:34:06.853 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.845+0000 7f1d5f7fe640 1 --2- 172.21.15.97:0/294020398 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f1d480a3dc0 0x7f1d480a6280 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1d500ce590 tx=0x7f1d5006c0c0 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:34:06.853 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.845+0000 7f1d457fa640 1 -- 172.21.15.97:0/4046349948 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d54035070 con 0x5568ddb32800 2024-04-23T10:34:06.853 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.846+0000 7f1d457fa640 1 -- 172.21.15.97:0/4046349948 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d54031620 con 0x5568ddb32800 2024-04-23T10:34:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.846+0000 7f1d457fa640 1 -- 172.21.15.97:0/4046349948 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d54003c30 con 0x5568ddb32800 2024-04-23T10:34:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.846+0000 7f1d67fca740 1 -- 172.21.15.97:0/4046349948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568ddb32800 msgr2=0x5568ddb34bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.846+0000 7f1d67fca740 1 --2- 172.21.15.97:0/4046349948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568ddb32800 0x5568ddb34bf0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f1d54009900 tx=0x7f1d54005e60 comp rx=0 tx=0).stop 2024-04-23T10:34:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.846+0000 7f1d67fca740 1 -- 172.21.15.97:0/4046349948 shutdown_connections 2024-04-23T10:34:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.846+0000 7f1d67fca740 1 --2- 172.21.15.97:0/4046349948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568ddb32800 0x5568ddb34bf0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.846+0000 7f1d67fca740 1 --2- 172.21.15.97:0/4046349948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568ddb27f40 0x5568ddb322c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.846+0000 7f1d67fca740 1 --2- 172.21.15.97:0/4046349948 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5568ddb27620 0x5568ddb27a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.846+0000 7f1d67fca740 1 -- 172.21.15.97:0/4046349948 >> 172.21.15.97:0/4046349948 conn(0x5568ddb23b30 msgr2=0x5568ddb23f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:34:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.846+0000 7f1d67fca740 1 -- 172.21.15.97:0/4046349948 shutdown_connections 2024-04-23T10:34:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.846+0000 7f1d67fca740 1 -- 172.21.15.97:0/4046349948 wait complete. 2024-04-23T10:34:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.847+0000 7f1d67fca740 1 -- start start 2024-04-23T10:34:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.847+0000 7f1d67fca740 1 Processor -- start 2024-04-23T10:34:06.855 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.847+0000 7f1d67fca740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568ddb27620 0x5568ddb32130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:06.855 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.847+0000 7f1d67fca740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5568ddb27f40 0x5568ddb2f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:06.855 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.847+0000 7f1d67fca740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568ddb32800 0x5568ddb2f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:06.855 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.847+0000 7f1d67fca740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5568ddb353d0 con 0x5568ddb32800 2024-04-23T10:34:06.855 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.847+0000 7f1d67fca740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5568dda270b0 con 0x5568ddb27f40 2024-04-23T10:34:06.855 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.847+0000 7f1d67fca740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5568ddb35550 con 0x5568ddb27620 2024-04-23T10:34:06.855 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.847+0000 7f1d67fca740 10 client.0 _ll_register_callbacks cb 0x5568dda14d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:34:06.855 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.847+0000 7f1d5e7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568ddb32800 0x5568ddb2f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:06.855 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.847+0000 7f1d5e7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568ddb32800 0x5568ddb2f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57774/0 (socket says 172.21.15.97:57774) 2024-04-23T10:34:06.855 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.847+0000 7f1d5e7fc640 1 -- 172.21.15.97:0/273918784 learned_addr learned my addr 172.21.15.97:0/273918784 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:34:06.855 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.847+0000 7f1d5e7fc640 1 --2- 172.21.15.97:0/273918784 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568ddb27620 0x5568ddb32130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:06.855 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.847+0000 7f1d5effd640 1 --2- 172.21.15.97:0/273918784 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5568ddb27f40 0x5568ddb2f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:06.855 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.847+0000 7f1d5e7fc640 1 -- 172.21.15.97:0/273918784 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568ddb27620 msgr2=0x5568ddb32130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:34:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.847+0000 7f1d5e7fc640 1 --2- 172.21.15.97:0/273918784 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568ddb27620 0x5568ddb32130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.847+0000 7f1d5e7fc640 1 -- 172.21.15.97:0/273918784 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5568ddb27f40 msgr2=0x5568ddb2f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:34:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.847+0000 7f1d5e7fc640 1 --2- 172.21.15.97:0/273918784 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5568ddb27f40 0x5568ddb2f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.847+0000 7f1d5e7fc640 1 -- 172.21.15.97:0/273918784 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568dd86b620 con 0x5568ddb32800 2024-04-23T10:34:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.848+0000 7f1d5e7fc640 1 --2- 172.21.15.97:0/273918784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568ddb32800 0x5568ddb2f960 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f1d54000c00 tx=0x7f1d54005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eca07dd486cffc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:34:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.848+0000 7f1d42ff5640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:34:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.848+0000 7f1d42ff5640 1 -- 172.21.15.97:0/273918784 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d54035070 con 0x5568ddb32800 2024-04-23T10:34:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.848+0000 7f1d42ff5640 1 -- 172.21.15.97:0/273918784 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7f1d54005280 con 0x5568ddb32800 2024-04-23T10:34:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.848+0000 7f1d67fca740 1 -- 172.21.15.97:0/273918784 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d54031620 con 0x5568ddb32800 2024-04-23T10:34:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.848+0000 7f1d42ff5640 1 -- 172.21.15.97:0/273918784 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1d54005480 con 0x5568ddb32800 2024-04-23T10:34:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.848+0000 7f1d67fca740 10 client.16665 fetch_fsmap learned FSMap version 26 2024-04-23T10:34:06.856 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.848+0000 7f1d67fca740 1 -- 172.21.15.97:0/273918784 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5568ddb41320 con 0x5568ddb32800 2024-04-23T10:34:06.857 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.848+0000 7f1d42ff5640 1 -- 172.21.15.97:0/273918784 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d54031620 con 0x5568ddb32800 2024-04-23T10:34:06.857 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.848+0000 7f1d42ff5640 1 -- 172.21.15.97:0/273918784 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1d54003c70 con 0x5568ddb32800 2024-04-23T10:34:06.857 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.848+0000 7f1d67fca740 10 client.16665 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:34:06.857 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.848+0000 7f1d67fca740 10 client.16665 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:34:06.857 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.848+0000 7f1d67fca740 20 client.16665 populate_metadata read hostname 'smithi097' 2024-04-23T10:34:06.857 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.848+0000 7f1d67fca740 1 --2- 172.21.15.97:0/273918784 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5568ddb42670 0x5568ddb44a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:06.857 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.848+0000 7f1d67fca740 4 client.16665 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:34:06.857 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.848+0000 7f1d67fca740 1 -- 172.21.15.97:0/273918784 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568dda9a390 con 0x5568ddb42670 2024-04-23T10:34:06.857 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.849+0000 7f1d5effd640 1 --2- 172.21.15.97:0/273918784 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5568ddb42670 0x5568ddb44a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:06.857 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.849+0000 7f1d42ff5640 10 client.16665 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:34:06.857 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.849+0000 7f1d5effd640 1 --2- 172.21.15.97:0/273918784 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5568ddb42670 0x5568ddb44a60 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64c66ff32ca764db server_cookie=6aa5d08eada8ea54 in_seq=0 out_seq=0 2024-04-23T10:34:06.857 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.850+0000 7f1d42ff5640 1 -- 172.21.15.97:0/273918784 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5568dda9a390 con 0x5568ddb42670 2024-04-23T10:34:06.857 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.850+0000 7f1d42ff5640 10 client.16665 handle_command_reply: tid=0 2024-04-23T10:34:06.858 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.850+0000 7f1d67fca740 1 client.16665 shutdown 2024-04-23T10:34:06.858 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.850+0000 7f1d67fca740 1 -- 172.21.15.97:0/273918784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568ddb32800 msgr2=0x5568ddb2f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:06.858 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.850+0000 7f1d67fca740 1 --2- 172.21.15.97:0/273918784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568ddb32800 0x5568ddb2f960 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f1d54000c00 tx=0x7f1d54005fd0 comp rx=0 tx=0).stop 2024-04-23T10:34:06.858 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.850+0000 7f1d67fca740 1 -- 172.21.15.97:0/273918784 shutdown_connections 2024-04-23T10:34:06.858 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.850+0000 7f1d67fca740 1 --2- 172.21.15.97:0/273918784 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5568ddb42670 0x5568ddb44a60 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:06.858 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.850+0000 7f1d67fca740 1 --2- 172.21.15.97:0/273918784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568ddb32800 0x5568ddb2f960 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:06.858 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.850+0000 7f1d42ff5640 0 client.16665 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:34:06.858 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.850+0000 7f1d67fca740 1 --2- 172.21.15.97:0/273918784 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5568ddb27f40 0x5568ddb2f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:06.858 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.850+0000 7f1d67fca740 1 --2- 172.21.15.97:0/273918784 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568ddb27620 0x5568ddb32130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:06.858 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.850+0000 7f1d67fca740 1 -- 172.21.15.97:0/273918784 >> 172.21.15.97:0/273918784 conn(0x5568ddb23b30 msgr2=0x5568ddb29860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:34:06.858 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.850+0000 7f1d67fca740 1 -- 172.21.15.97:0/273918784 shutdown_connections 2024-04-23T10:34:06.858 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.850+0000 7f1d67fca740 1 -- 172.21.15.97:0/273918784 wait complete. 2024-04-23T10:34:06.859 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.850+0000 7f1d67fca740 20 client.16665 trim_cache size 0 max 16384 2024-04-23T10:34:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.865+0000 7f1d67fca740 1 Processor -- start 2024-04-23T10:34:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.865+0000 7f1d67fca740 1 -- start start 2024-04-23T10:34:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.865+0000 7f1d67fca740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568ddb27620 0x5568ddb3deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.865+0000 7f1d67fca740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568ddb27f40 0x5568ddb31c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.865+0000 7f1d67fca740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5568ddb32800 0x5568ddb2c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.865+0000 7f1d67fca740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1d601adad0 con 0x5568ddb27f40 2024-04-23T10:34:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.865+0000 7f1d67fca740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1d601af610 con 0x5568ddb32800 2024-04-23T10:34:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.865+0000 7f1d67fca740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1d601ae1a0 con 0x5568ddb27620 2024-04-23T10:34:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.865+0000 7f1d5effd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568ddb27f40 0x5568ddb31c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.865+0000 7f1d5effd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568ddb27f40 0x5568ddb31c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57788/0 (socket says 172.21.15.97:57788) 2024-04-23T10:34:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.865+0000 7f1d5effd640 1 -- 172.21.15.97:0/990241657 learned_addr learned my addr 172.21.15.97:0/990241657 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:34:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.865+0000 7f1d5effd640 1 -- 172.21.15.97:0/990241657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568ddb27620 msgr2=0x5568ddb3deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:34:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.865+0000 7f1d5effd640 1 --2- 172.21.15.97:0/990241657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568ddb27620 0x5568ddb3deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.865+0000 7f1d5effd640 1 -- 172.21.15.97:0/990241657 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5568ddb32800 msgr2=0x5568ddb2c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:34:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.865+0000 7f1d5effd640 1 --2- 172.21.15.97:0/990241657 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5568ddb32800 0x5568ddb2c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.865+0000 7f1d5effd640 1 -- 172.21.15.97:0/990241657 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568ddb27d70 con 0x5568ddb27f40 2024-04-23T10:34:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.866+0000 7f1d5effd640 1 --2- 172.21.15.97:0/990241657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568ddb27f40 0x5568ddb31c60 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x5568ddb2ec80 tx=0x7f1d58005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=863054d205624fe4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:34:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.866+0000 7f1d44ff9640 1 -- 172.21.15.97:0/990241657 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d58014e90 con 0x5568ddb27f40 2024-04-23T10:34:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.866+0000 7f1d44ff9640 1 -- 172.21.15.97:0/990241657 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d5801b7e0 con 0x5568ddb27f40 2024-04-23T10:34:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.866+0000 7f1d44ff9640 1 -- 172.21.15.97:0/990241657 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d58012a10 con 0x5568ddb27f40 2024-04-23T10:34:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.866+0000 7f1d67fca740 1 -- 172.21.15.97:0/990241657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568ddb27f40 msgr2=0x5568ddb31c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.866+0000 7f1d67fca740 1 --2- 172.21.15.97:0/990241657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568ddb27f40 0x5568ddb31c60 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x5568ddb2ec80 tx=0x7f1d58005850 comp rx=0 tx=0).stop 2024-04-23T10:34:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.866+0000 7f1d67fca740 1 -- 172.21.15.97:0/990241657 shutdown_connections 2024-04-23T10:34:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.866+0000 7f1d67fca740 1 --2- 172.21.15.97:0/990241657 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5568ddb32800 0x5568ddb2c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.866+0000 7f1d67fca740 1 --2- 172.21.15.97:0/990241657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568ddb27f40 0x5568ddb31c60 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.866+0000 7f1d67fca740 1 --2- 172.21.15.97:0/990241657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568ddb27620 0x5568ddb3deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.866+0000 7f1d67fca740 1 -- 172.21.15.97:0/990241657 >> 172.21.15.97:0/990241657 conn(0x5568ddb23b30 msgr2=0x5568ddb3d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:34:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.866+0000 7f1d67fca740 1 -- 172.21.15.97:0/990241657 shutdown_connections 2024-04-23T10:34:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.866+0000 7f1d67fca740 1 -- 172.21.15.97:0/990241657 wait complete. 2024-04-23T10:34:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.867+0000 7f1d67fca740 1 -- start start 2024-04-23T10:34:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.867+0000 7f1d67fca740 1 Processor -- start 2024-04-23T10:34:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.867+0000 7f1d67fca740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568ddb27620 0x5568ddba55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.867+0000 7f1d67fca740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568ddb27f40 0x5568ddba5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.867+0000 7f1d67fca740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5568ddb32800 0x5568ddb9f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.867+0000 7f1d67fca740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1d601aef40 con 0x5568ddb27620 2024-04-23T10:34:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.867+0000 7f1d67fca740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1d601ae870 con 0x5568ddb32800 2024-04-23T10:34:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.867+0000 7f1d67fca740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1d601abad0 con 0x5568ddb27f40 2024-04-23T10:34:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.867+0000 7f1d67fca740 10 client.0 _ll_register_callbacks cb 0x7f1d540317d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:34:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.867+0000 7f1d5e7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568ddb27620 0x5568ddba55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.867+0000 7f1d5e7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568ddb27620 0x5568ddba55a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57800/0 (socket says 172.21.15.97:57800) 2024-04-23T10:34:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.867+0000 7f1d5e7fc640 1 -- 172.21.15.97:0/2145747032 learned_addr learned my addr 172.21.15.97:0/2145747032 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:34:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.867+0000 7f1d5e7fc640 1 -- 172.21.15.97:0/2145747032 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568ddb27f40 msgr2=0x5568ddba5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:34:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.867+0000 7f1d5effd640 1 --2- 172.21.15.97:0/2145747032 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568ddb27f40 0x5568ddba5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.867+0000 7f1d5e7fc640 1 --2- 172.21.15.97:0/2145747032 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568ddb27f40 0x5568ddba5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.867+0000 7f1d5e7fc640 1 -- 172.21.15.97:0/2145747032 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5568ddb32800 msgr2=0x5568ddb9f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:34:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.867+0000 7f1d5e7fc640 1 --2- 172.21.15.97:0/2145747032 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5568ddb32800 0x5568ddb9f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.867+0000 7f1d5e7fc640 1 -- 172.21.15.97:0/2145747032 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568ddba00a0 con 0x5568ddb27620 2024-04-23T10:34:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.867+0000 7f1d5e7fc640 1 --2- 172.21.15.97:0/2145747032 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568ddb27620 0x5568ddba55a0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f1d54033000 tx=0x7f1d5402ac30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d937715fa90e550d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:34:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.867+0000 7f1d42ff5640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:34:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.868+0000 7f1d42ff5640 1 -- 172.21.15.97:0/2145747032 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d54060060 con 0x5568ddb27620 2024-04-23T10:34:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.868+0000 7f1d42ff5640 1 -- 172.21.15.97:0/2145747032 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7f1d54005280 con 0x5568ddb27620 2024-04-23T10:34:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.868+0000 7f1d67fca740 1 -- 172.21.15.97:0/2145747032 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d5801b7e0 con 0x5568ddb27620 2024-04-23T10:34:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.868+0000 7f1d42ff5640 1 -- 172.21.15.97:0/2145747032 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1d54035040 con 0x5568ddb27620 2024-04-23T10:34:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.868+0000 7f1d67fca740 10 client.16671 fetch_fsmap learned FSMap version 26 2024-04-23T10:34:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.868+0000 7f1d67fca740 1 -- 172.21.15.97:0/2145747032 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5568ddbacdb0 con 0x5568ddb27620 2024-04-23T10:34:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.868+0000 7f1d42ff5640 1 -- 172.21.15.97:0/2145747032 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d5801b7e0 con 0x5568ddb27620 2024-04-23T10:34:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.868+0000 7f1d42ff5640 1 -- 172.21.15.97:0/2145747032 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1d54004d00 con 0x5568ddb27620 2024-04-23T10:34:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.868+0000 7f1d67fca740 10 client.16671 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:34:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.868+0000 7f1d67fca740 10 client.16671 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:34:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.868+0000 7f1d67fca740 20 client.16671 populate_metadata read hostname 'smithi097' 2024-04-23T10:34:06.874 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.868+0000 7f1d67fca740 1 --2- 172.21.15.97:0/2145747032 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5568ddbad4a0 0x5568ddbaf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:06.874 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.868+0000 7f1d67fca740 4 client.16671 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:34:06.874 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.868+0000 7f1d67fca740 1 -- 172.21.15.97:0/2145747032 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5568ddb2fcf0 con 0x5568ddbad4a0 2024-04-23T10:34:06.874 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.868+0000 7f1d5effd640 1 --2- 172.21.15.97:0/2145747032 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5568ddbad4a0 0x5568ddbaf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:06.874 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.869+0000 7f1d42ff5640 10 client.16671 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:34:06.874 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.869+0000 7f1d5effd640 1 --2- 172.21.15.97:0/2145747032 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5568ddbad4a0 0x5568ddbaf8c0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91faa8e376e39d8b server_cookie=d6eb9c6331258c35 in_seq=0 out_seq=0 2024-04-23T10:34:06.881 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.882+0000 7f1d42ff5640 1 -- 172.21.15.97:0/2145747032 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5568ddb2fcf0 con 0x5568ddbad4a0 2024-04-23T10:34:06.881 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.882+0000 7f1d42ff5640 10 client.16671 handle_command_reply: tid=0 2024-04-23T10:34:06.882 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.882+0000 7f1d67fca740 1 client.16671 shutdown 2024-04-23T10:34:06.882 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.882+0000 7f1d67fca740 1 -- 172.21.15.97:0/2145747032 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568ddb27620 msgr2=0x5568ddba55a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:06.882 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.882+0000 7f1d67fca740 1 --2- 172.21.15.97:0/2145747032 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568ddb27620 0x5568ddba55a0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f1d54033000 tx=0x7f1d5402ac30 comp rx=0 tx=0).stop 2024-04-23T10:34:06.882 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.882+0000 7f1d67fca740 1 -- 172.21.15.97:0/2145747032 shutdown_connections 2024-04-23T10:34:06.882 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.882+0000 7f1d67fca740 1 --2- 172.21.15.97:0/2145747032 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5568ddbad4a0 0x5568ddbaf8c0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:06.882 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.882+0000 7f1d67fca740 1 --2- 172.21.15.97:0/2145747032 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5568ddb32800 0x5568ddb9f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:06.882 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.882+0000 7f1d42ff5640 0 client.16671 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:34:06.882 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.882+0000 7f1d67fca740 1 --2- 172.21.15.97:0/2145747032 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5568ddb27f40 0x5568ddba5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:06.882 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.882+0000 7f1d67fca740 1 --2- 172.21.15.97:0/2145747032 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5568ddb27620 0x5568ddba55a0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:06.882 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.882+0000 7f1d67fca740 1 -- 172.21.15.97:0/2145747032 >> 172.21.15.97:0/2145747032 conn(0x5568ddb23b30 msgr2=0x5568ddb22cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:34:06.883 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.882+0000 7f1d67fca740 1 -- 172.21.15.97:0/2145747032 shutdown_connections 2024-04-23T10:34:06.883 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.882+0000 7f1d67fca740 1 -- 172.21.15.97:0/2145747032 wait complete. 2024-04-23T10:34:06.883 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.882+0000 7f1d67fca740 20 client.16671 trim_cache size 0 max 16384 2024-04-23T10:34:06.883 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:34:06.883 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:34:06.883 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:34:06.883 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:34:06.884 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.883+0000 7f1d652ae640 1 -- 172.21.15.97:0/294020398 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f1d480a3dc0 msgr2=0x7f1d480a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:06.884 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.883+0000 7f1d652ae640 1 --2- 172.21.15.97:0/294020398 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f1d480a3dc0 0x7f1d480a6280 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1d500ce590 tx=0x7f1d5006c0c0 comp rx=0 tx=0).stop 2024-04-23T10:34:06.884 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.883+0000 7f1d652ae640 1 -- 172.21.15.97:0/294020398 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d6010c520 msgr2=0x7f1d6019cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:06.884 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.883+0000 7f1d652ae640 1 --2- 172.21.15.97:0/294020398 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d6010c520 0x7f1d6019cfc0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f1d5000cad0 tx=0x7f1d5000cf90 comp rx=0 tx=0).stop 2024-04-23T10:34:06.885 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.883+0000 7f1d652ae640 1 -- 172.21.15.97:0/294020398 shutdown_connections 2024-04-23T10:34:06.885 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.883+0000 7f1d652ae640 1 --2- 172.21.15.97:0/294020398 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f1d480a3dc0 0x7f1d480a6280 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:06.885 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.883+0000 7f1d652ae640 1 --2- 172.21.15.97:0/294020398 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d6010c520 0x7f1d6019cfc0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:06.885 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.883+0000 7f1d652ae640 1 --2- 172.21.15.97:0/294020398 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d60103b00 0x7f1d60198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:06.885 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.883+0000 7f1d652ae640 1 --2- 172.21.15.97:0/294020398 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d601031e0 0x7f1d60198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:06.885 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.883+0000 7f1d652ae640 1 -- 172.21.15.97:0/294020398 >> 172.21.15.97:0/294020398 conn(0x7f1d60077fd0 msgr2=0x7f1d600fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:34:06.885 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.883+0000 7f1d652ae640 1 -- 172.21.15.97:0/294020398 shutdown_connections 2024-04-23T10:34:06.885 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:06.883+0000 7f1d652ae640 1 -- 172.21.15.97:0/294020398 wait complete. 2024-04-23T10:34:07.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:34:07 smithi196 ceph-mon[32919]: pgmap v243: 97 pgs: 97 active+clean; 2.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 103 KiB/s wr, 39 op/s 2024-04-23T10:34:07.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:34:07 smithi196 ceph-mon[32919]: osdmap e93: 12 total, 12 up, 12 in 2024-04-23T10:34:07.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:34:07 smithi097 ceph-mon[29197]: pgmap v243: 97 pgs: 97 active+clean; 2.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 103 KiB/s wr, 39 op/s 2024-04-23T10:34:07.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:34:07 smithi097 ceph-mon[29197]: osdmap e93: 12 total, 12 up, 12 in 2024-04-23T10:34:07.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:34:07 smithi183 ceph-mon[32317]: pgmap v243: 97 pgs: 97 active+clean; 2.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 103 KiB/s wr, 39 op/s 2024-04-23T10:34:07.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:34:07 smithi183 ceph-mon[32317]: osdmap e93: 12 total, 12 up, 12 in 2024-04-23T10:34:09.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:34:09 smithi196 ceph-mon[32919]: pgmap v245: 97 pgs: 97 active+clean; 2.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 133 KiB/s wr, 48 op/s 2024-04-23T10:34:09.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:34:09 smithi097 ceph-mon[29197]: pgmap v245: 97 pgs: 97 active+clean; 2.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 133 KiB/s wr, 48 op/s 2024-04-23T10:34:09.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:34:09 smithi183 ceph-mon[32317]: pgmap v245: 97 pgs: 97 active+clean; 2.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 133 KiB/s wr, 48 op/s 2024-04-23T10:34:11.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:34:11 smithi196 ceph-mon[32919]: pgmap v246: 97 pgs: 97 active+clean; 2.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 133 KiB/s wr, 48 op/s 2024-04-23T10:34:11.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:34:11 smithi097 ceph-mon[29197]: pgmap v246: 97 pgs: 97 active+clean; 2.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 133 KiB/s wr, 48 op/s 2024-04-23T10:34:11.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:34:11 smithi183 ceph-mon[32317]: pgmap v246: 97 pgs: 97 active+clean; 2.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 133 KiB/s wr, 48 op/s 2024-04-23T10:34:13.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:34:13 smithi196 ceph-mon[32919]: pgmap v247: 97 pgs: 97 active+clean; 2.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 107 KiB/s wr, 39 op/s 2024-04-23T10:34:13.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:34:13 smithi097 ceph-mon[29197]: pgmap v247: 97 pgs: 97 active+clean; 2.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 107 KiB/s wr, 39 op/s 2024-04-23T10:34:13.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:34:13 smithi183 ceph-mon[32317]: pgmap v247: 97 pgs: 97 active+clean; 2.5 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 107 KiB/s wr, 39 op/s 2024-04-23T10:34:15.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:34:15 smithi196 ceph-mon[32919]: pgmap v248: 97 pgs: 97 active+clean; 2.7 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-23T10:34:15.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:34:15 smithi097 ceph-mon[29197]: pgmap v248: 97 pgs: 97 active+clean; 2.7 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-23T10:34:15.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:34:15 smithi183 ceph-mon[32317]: pgmap v248: 97 pgs: 97 active+clean; 2.7 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-23T10:34:17.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:34:17 smithi196 ceph-mon[32919]: pgmap v249: 97 pgs: 97 active+clean; 2.7 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-23T10:34:17.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:34:17 smithi097 ceph-mon[29197]: pgmap v249: 97 pgs: 97 active+clean; 2.7 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-23T10:34:17.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:34:17 smithi183 ceph-mon[32317]: pgmap v249: 97 pgs: 97 active+clean; 2.7 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-23T10:34:19.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:34:19 smithi097 ceph-mon[29197]: pgmap v250: 97 pgs: 97 active+clean; 2.9 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 64 KiB/s wr, 23 op/s 2024-04-23T10:34:19.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:34:19 smithi183 ceph-mon[32317]: pgmap v250: 97 pgs: 97 active+clean; 2.9 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 64 KiB/s wr, 23 op/s 2024-04-23T10:34:19.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:34:19 smithi196 ceph-mon[32919]: pgmap v250: 97 pgs: 97 active+clean; 2.9 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 64 KiB/s wr, 23 op/s 2024-04-23T10:34:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.979+0000 7f10c053d640 1 -- 172.21.15.97:0/3528478985 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10b81005c0 msgr2=0x7f10b8100a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.979+0000 7f10c053d640 1 --2- 172.21.15.97:0/3528478985 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10b81005c0 0x7f10b8100a40 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f10a8009930 tx=0x7f10a802a420 comp rx=0 tx=0).stop 2024-04-23T10:34:19.979 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.979+0000 7f10c053d640 1 -- 172.21.15.97:0/3528478985 shutdown_connections 2024-04-23T10:34:19.979 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.979+0000 7f10c053d640 1 --2- 172.21.15.97:0/3528478985 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10b8100f80 0x7f10b810d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:19.979 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.979+0000 7f10c053d640 1 --2- 172.21.15.97:0/3528478985 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10b81005c0 0x7f10b8100a40 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:19.979 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.979+0000 7f10c053d640 1 --2- 172.21.15.97:0/3528478985 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10b8106660 0x7f10b8106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:19.979 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.979+0000 7f10c053d640 1 -- 172.21.15.97:0/3528478985 >> 172.21.15.97:0/3528478985 conn(0x7f10b8077390 msgr2=0x7f10b80fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:34:19.979 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.979+0000 7f10c053d640 1 -- 172.21.15.97:0/3528478985 shutdown_connections 2024-04-23T10:34:19.979 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.979+0000 7f10c053d640 1 -- 172.21.15.97:0/3528478985 wait complete. 2024-04-23T10:34:19.979 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.980+0000 7f10c053d640 1 Processor -- start 2024-04-23T10:34:19.979 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.980+0000 7f10c053d640 1 -- start start 2024-04-23T10:34:19.981 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.980+0000 7f10c053d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10b81005c0 0x7f10b819c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:19.981 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.980+0000 7f10c053d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10b8100f80 0x7f10b819c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:19.981 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.980+0000 7f10c053d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10b8106660 0x7f10b8197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:19.981 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.980+0000 7f10c053d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f10b81108b0 con 0x7f10b8106660 2024-04-23T10:34:19.981 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.980+0000 7f10c053d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f10b8110730 con 0x7f10b8100f80 2024-04-23T10:34:19.981 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.980+0000 7f10c053d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f10b8110a30 con 0x7f10b81005c0 2024-04-23T10:34:19.981 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.980+0000 7f10beada640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10b8106660 0x7f10b8197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:19.981 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.980+0000 7f10beada640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10b8106660 0x7f10b8197b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58060/0 (socket says 172.21.15.97:58060) 2024-04-23T10:34:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.980+0000 7f10beada640 1 -- 172.21.15.97:0/1735420562 learned_addr learned my addr 172.21.15.97:0/1735420562 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:34:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.980+0000 7f10bdad8640 1 --2- 172.21.15.97:0/1735420562 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10b8100f80 0x7f10b819c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.980+0000 7f10be2d9640 1 --2- 172.21.15.97:0/1735420562 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10b81005c0 0x7f10b819c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.981+0000 7f10beada640 1 -- 172.21.15.97:0/1735420562 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10b81005c0 msgr2=0x7f10b819c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.981+0000 7f10beada640 1 --2- 172.21.15.97:0/1735420562 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10b81005c0 0x7f10b819c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.981+0000 7f10beada640 1 -- 172.21.15.97:0/1735420562 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10b8100f80 msgr2=0x7f10b819c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.981+0000 7f10beada640 1 --2- 172.21.15.97:0/1735420562 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10b8100f80 0x7f10b819c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.981+0000 7f10beada640 1 -- 172.21.15.97:0/1735420562 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10b81983f0 con 0x7f10b8106660 2024-04-23T10:34:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.981+0000 7f10bdad8640 1 --2- 172.21.15.97:0/1735420562 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10b8100f80 0x7f10b819c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:34:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.981+0000 7f10beada640 1 --2- 172.21.15.97:0/1735420562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10b8106660 0x7f10b8197b30 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f10b000b970 tx=0x7f10b000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:34:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.981+0000 7f10af7fe640 1 -- 172.21.15.97:0/1735420562 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b0019070 con 0x7f10b8106660 2024-04-23T10:34:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.981+0000 7f10af7fe640 1 -- 172.21.15.97:0/1735420562 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10b0014440 con 0x7f10b8106660 2024-04-23T10:34:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.981+0000 7f10c053d640 1 -- 172.21.15.97:0/1735420562 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10b8198680 con 0x7f10b8106660 2024-04-23T10:34:19.982 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.981+0000 7f10af7fe640 1 -- 172.21.15.97:0/1735420562 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b00075d0 con 0x7f10b8106660 2024-04-23T10:34:19.983 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.981+0000 7f10c053d640 1 -- 172.21.15.97:0/1735420562 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10b81a42c0 con 0x7f10b8106660 2024-04-23T10:34:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.983+0000 7f10af7fe640 1 -- 172.21.15.97:0/1735420562 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f10b001c070 con 0x7f10b8106660 2024-04-23T10:34:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.984+0000 7f10c324b740 1 Processor -- start 2024-04-23T10:34:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.984+0000 7f10c324b740 1 -- start start 2024-04-23T10:34:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.984+0000 7f10c324b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556565550520 0x556565550900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.984+0000 7f10c324b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556565550e40 0x55656555d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.984+0000 7f10c324b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55656555da10 0x55656555fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.984+0000 7f10c324b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5565654b5970 con 0x556565550520 2024-04-23T10:34:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.984+0000 7f10c324b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5565654524a0 con 0x556565550e40 2024-04-23T10:34:19.990 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.984+0000 7f10c324b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5565654b1930 con 0x55656555da10 2024-04-23T10:34:19.990 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.984+0000 7f10af7fe640 1 --2- 172.21.15.97:0/1735420562 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f10a40a3dc0 0x7f10a40a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:19.990 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.984+0000 7f10bdad8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556565550520 0x556565550900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:19.990 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.984+0000 7f10bdad8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556565550520 0x556565550900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58076/0 (socket says 172.21.15.97:58076) 2024-04-23T10:34:19.990 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.984+0000 7f10bdad8640 1 -- 172.21.15.97:0/486739699 learned_addr learned my addr 172.21.15.97:0/486739699 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:34:19.990 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.984+0000 7f10beada640 1 --2- 172.21.15.97:0/1735420562 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f10a40a3dc0 0x7f10a40a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:19.990 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.984+0000 7f10be2d9640 1 --2- 172.21.15.97:0/486739699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556565550e40 0x55656555d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:19.990 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.985+0000 7f10bdad8640 1 --2- 172.21.15.97:0/486739699 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55656555da10 0x55656555fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:19.990 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.985+0000 7f10af7fe640 1 -- 172.21.15.97:0/1735420562 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7f10b00cfd10 con 0x7f10b8106660 2024-04-23T10:34:19.990 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.985+0000 7f10bdad8640 1 -- 172.21.15.97:0/486739699 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55656555da10 msgr2=0x55656555fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:34:19.990 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.985+0000 7f10bdad8640 1 --2- 172.21.15.97:0/486739699 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55656555da10 0x55656555fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:19.990 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.985+0000 7f10bdad8640 1 -- 172.21.15.97:0/486739699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556565550e40 msgr2=0x55656555d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:34:19.990 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.985+0000 7f10bdad8640 1 --2- 172.21.15.97:0/486739699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556565550e40 0x55656555d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:19.990 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.985+0000 7f10bdad8640 1 -- 172.21.15.97:0/486739699 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565651c8210 con 0x556565550520 2024-04-23T10:34:19.990 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.985+0000 7f10be2d9640 1 --2- 172.21.15.97:0/486739699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556565550e40 0x55656555d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:34:19.991 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.985+0000 7f10beada640 1 --2- 172.21.15.97:0/1735420562 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f10a40a3dc0 0x7f10a40a6280 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f10b00cda80 tx=0x7f10b0098100 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:34:19.991 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.985+0000 7f10bdad8640 1 --2- 172.21.15.97:0/486739699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556565550520 0x556565550900 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f10a8000c00 tx=0x7f10a8009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d8f6e99f1102d15 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:34:19.991 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.985+0000 7f10acbfe640 1 -- 172.21.15.97:0/486739699 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a8035050 con 0x556565550520 2024-04-23T10:34:19.991 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.985+0000 7f10acbfe640 1 -- 172.21.15.97:0/486739699 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10a80044d0 con 0x556565550520 2024-04-23T10:34:19.991 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.985+0000 7f10acbfe640 1 -- 172.21.15.97:0/486739699 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a8004920 con 0x556565550520 2024-04-23T10:34:19.991 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.986+0000 7f10c324b740 1 -- 172.21.15.97:0/486739699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556565550520 msgr2=0x556565550900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:19.991 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.986+0000 7f10c324b740 1 --2- 172.21.15.97:0/486739699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556565550520 0x556565550900 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f10a8000c00 tx=0x7f10a8009560 comp rx=0 tx=0).stop 2024-04-23T10:34:19.991 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.986+0000 7f10c324b740 1 -- 172.21.15.97:0/486739699 shutdown_connections 2024-04-23T10:34:19.991 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.986+0000 7f10c324b740 1 --2- 172.21.15.97:0/486739699 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55656555da10 0x55656555fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:19.991 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.986+0000 7f10c324b740 1 --2- 172.21.15.97:0/486739699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556565550e40 0x55656555d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:19.991 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.986+0000 7f10c324b740 1 --2- 172.21.15.97:0/486739699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556565550520 0x556565550900 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:19.991 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.986+0000 7f10c324b740 1 -- 172.21.15.97:0/486739699 >> 172.21.15.97:0/486739699 conn(0x55656554cb30 msgr2=0x55656554cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:34:19.991 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.986+0000 7f10c324b740 1 -- 172.21.15.97:0/486739699 shutdown_connections 2024-04-23T10:34:19.991 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.986+0000 7f10c324b740 1 -- 172.21.15.97:0/486739699 wait complete. 2024-04-23T10:34:19.991 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.986+0000 7f10c324b740 1 -- start start 2024-04-23T10:34:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.986+0000 7f10c324b740 1 Processor -- start 2024-04-23T10:34:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.987+0000 7f10c324b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556565550520 0x55656555d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.987+0000 7f10c324b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556565550e40 0x55656555a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.987+0000 7f10c324b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55656555da10 0x55656555aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.987+0000 7f10c324b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5565655605e0 con 0x556565550e40 2024-04-23T10:34:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.987+0000 7f10c324b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5565654500b0 con 0x556565550520 2024-04-23T10:34:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.987+0000 7f10c324b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556565560760 con 0x55656555da10 2024-04-23T10:34:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.987+0000 7f10c324b740 10 client.0 _ll_register_callbacks cb 0x55656543dd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:34:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.987+0000 7f10be2d9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556565550e40 0x55656555a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.987+0000 7f10bdad8640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556565550520 0x55656555d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.987+0000 7f10be2d9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556565550e40 0x55656555a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58088/0 (socket says 172.21.15.97:58088) 2024-04-23T10:34:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.987+0000 7f10be2d9640 1 -- 172.21.15.97:0/1190514598 learned_addr learned my addr 172.21.15.97:0/1190514598 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:34:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.987+0000 7f10bdad8640 1 --2- 172.21.15.97:0/1190514598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55656555da10 0x55656555aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.987+0000 7f10be2d9640 1 -- 172.21.15.97:0/1190514598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55656555da10 msgr2=0x55656555aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:34:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.987+0000 7f10be2d9640 1 --2- 172.21.15.97:0/1190514598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55656555da10 0x55656555aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.987+0000 7f10be2d9640 1 -- 172.21.15.97:0/1190514598 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556565550520 msgr2=0x55656555d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:34:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.987+0000 7f10be2d9640 1 --2- 172.21.15.97:0/1190514598 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556565550520 0x55656555d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.987+0000 7f10be2d9640 1 -- 172.21.15.97:0/1190514598 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556565294620 con 0x556565550e40 2024-04-23T10:34:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.987+0000 7f10be2d9640 1 --2- 172.21.15.97:0/1190514598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556565550e40 0x55656555a5b0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f10b400f6c0 tx=0x7f10b400fb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ffa700205b721af server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:34:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.987+0000 7f10a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:34:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.987+0000 7f10a1ffb640 1 -- 172.21.15.97:0/1190514598 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b4016070 con 0x556565550e40 2024-04-23T10:34:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.987+0000 7f10a1ffb640 1 -- 172.21.15.97:0/1190514598 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7f10b400bb00 con 0x556565550e40 2024-04-23T10:34:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.987+0000 7f10c324b740 1 -- 172.21.15.97:0/1190514598 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10a80044d0 con 0x556565550e40 2024-04-23T10:34:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.988+0000 7f10a1ffb640 1 -- 172.21.15.97:0/1190514598 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f10b400bd00 con 0x556565550e40 2024-04-23T10:34:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.988+0000 7f10c324b740 10 client.16683 fetch_fsmap learned FSMap version 26 2024-04-23T10:34:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.988+0000 7f10c324b740 1 -- 172.21.15.97:0/1190514598 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55656556c510 con 0x556565550e40 2024-04-23T10:34:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.988+0000 7f10a1ffb640 1 -- 172.21.15.97:0/1190514598 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10a80044d0 con 0x556565550e40 2024-04-23T10:34:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.988+0000 7f10a1ffb640 1 -- 172.21.15.97:0/1190514598 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f10b4021070 con 0x556565550e40 2024-04-23T10:34:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.988+0000 7f10c324b740 10 client.16683 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:34:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.988+0000 7f10c324b740 10 client.16683 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:34:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.988+0000 7f10c324b740 20 client.16683 populate_metadata read hostname 'smithi097' 2024-04-23T10:34:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.988+0000 7f10c324b740 1 --2- 172.21.15.97:0/1190514598 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55656556d9d0 0x55656556fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.988+0000 7f10c324b740 4 client.16683 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:34:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.988+0000 7f10c324b740 1 -- 172.21.15.97:0/1190514598 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565654c3390 con 0x55656556d9d0 2024-04-23T10:34:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.988+0000 7f10bdad8640 1 --2- 172.21.15.97:0/1190514598 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55656556d9d0 0x55656556fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.989+0000 7f10a1ffb640 10 client.16683 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:34:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.989+0000 7f10bdad8640 1 --2- 172.21.15.97:0/1190514598 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55656556d9d0 0x55656556fe40 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=781fb924f4eb3728 server_cookie=5be666f3f74a0365 in_seq=0 out_seq=0 2024-04-23T10:34:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.989+0000 7f10a1ffb640 1 -- 172.21.15.97:0/1190514598 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5565654c3390 con 0x55656556d9d0 2024-04-23T10:34:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.989+0000 7f10a1ffb640 10 client.16683 handle_command_reply: tid=0 2024-04-23T10:34:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.989+0000 7f10c324b740 1 client.16683 shutdown 2024-04-23T10:34:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.990+0000 7f10c324b740 1 -- 172.21.15.97:0/1190514598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556565550e40 msgr2=0x55656555a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.990+0000 7f10c324b740 1 --2- 172.21.15.97:0/1190514598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556565550e40 0x55656555a5b0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f10b400f6c0 tx=0x7f10b400fb80 comp rx=0 tx=0).stop 2024-04-23T10:34:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.990+0000 7f10c324b740 1 -- 172.21.15.97:0/1190514598 shutdown_connections 2024-04-23T10:34:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.990+0000 7f10c324b740 1 --2- 172.21.15.97:0/1190514598 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55656556d9d0 0x55656556fe40 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:19.994 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.990+0000 7f10c324b740 1 --2- 172.21.15.97:0/1190514598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55656555da10 0x55656555aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.990+0000 7f10c324b740 1 --2- 172.21.15.97:0/1190514598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556565550e40 0x55656555a5b0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.990+0000 7f10a1ffb640 0 client.16683 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:34:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.990+0000 7f10c324b740 1 --2- 172.21.15.97:0/1190514598 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556565550520 0x55656555d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.990+0000 7f10c324b740 1 -- 172.21.15.97:0/1190514598 >> 172.21.15.97:0/1190514598 conn(0x55656554cb30 msgr2=0x556565552970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:34:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.990+0000 7f10c324b740 1 -- 172.21.15.97:0/1190514598 shutdown_connections 2024-04-23T10:34:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.990+0000 7f10c324b740 1 -- 172.21.15.97:0/1190514598 wait complete. 2024-04-23T10:34:19.995 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:19.990+0000 7f10c324b740 20 client.16683 trim_cache size 0 max 16384 2024-04-23T10:34:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.004+0000 7f10c324b740 1 Processor -- start 2024-04-23T10:34:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.004+0000 7f10c324b740 1 -- start start 2024-04-23T10:34:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.005+0000 7f10c324b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556565550520 0x5565655694e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.005+0000 7f10c324b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556565550e40 0x55656555cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.005+0000 7f10c324b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55656555da10 0x556565575750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.005+0000 7f10c324b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f10b81ab8b0 con 0x55656555da10 2024-04-23T10:34:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.005+0000 7f10c324b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f10b81ad3f0 con 0x556565550e40 2024-04-23T10:34:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.005+0000 7f10c324b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f10b81abf80 con 0x556565550520 2024-04-23T10:34:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.005+0000 7f10bdad8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55656555da10 0x556565575750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.005+0000 7f10bdad8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55656555da10 0x556565575750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58090/0 (socket says 172.21.15.97:58090) 2024-04-23T10:34:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.005+0000 7f10bdad8640 1 -- 172.21.15.97:0/1029782793 learned_addr learned my addr 172.21.15.97:0/1029782793 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:34:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.005+0000 7f10bdad8640 1 --2- 172.21.15.97:0/1029782793 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556565550520 0x5565655694e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.005+0000 7f10bdad8640 1 -- 172.21.15.97:0/1029782793 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556565550520 msgr2=0x5565655694e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:34:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.005+0000 7f10bdad8640 1 --2- 172.21.15.97:0/1029782793 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556565550520 0x5565655694e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.005+0000 7f10bdad8640 1 -- 172.21.15.97:0/1029782793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556565550e40 msgr2=0x55656555cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:34:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.005+0000 7f10be2d9640 1 --2- 172.21.15.97:0/1029782793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556565550e40 0x55656555cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.005+0000 7f10bdad8640 1 --2- 172.21.15.97:0/1029782793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556565550e40 0x55656555cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.005+0000 7f10bdad8640 1 -- 172.21.15.97:0/1029782793 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55656554c570 con 0x55656555da10 2024-04-23T10:34:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.005+0000 7f10be2d9640 1 --2- 172.21.15.97:0/1029782793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556565550e40 0x55656555cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:34:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.005+0000 7f10bdad8640 1 --2- 172.21.15.97:0/1029782793 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55656555da10 0x556565575750 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f10a80055c0 tx=0x7f10a8032d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9faea926090e07f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:34:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.005+0000 7f10a3fff640 1 -- 172.21.15.97:0/1029782793 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a8035040 con 0x55656555da10 2024-04-23T10:34:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.005+0000 7f10a3fff640 1 -- 172.21.15.97:0/1029782793 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10a8056410 con 0x55656555da10 2024-04-23T10:34:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.005+0000 7f10a3fff640 1 -- 172.21.15.97:0/1029782793 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a80047b0 con 0x55656555da10 2024-04-23T10:34:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.006+0000 7f10c324b740 1 -- 172.21.15.97:0/1029782793 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55656555da10 msgr2=0x556565575750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.006+0000 7f10c324b740 1 --2- 172.21.15.97:0/1029782793 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55656555da10 0x556565575750 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f10a80055c0 tx=0x7f10a8032d80 comp rx=0 tx=0).stop 2024-04-23T10:34:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.006+0000 7f10c324b740 1 -- 172.21.15.97:0/1029782793 shutdown_connections 2024-04-23T10:34:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.006+0000 7f10c324b740 1 --2- 172.21.15.97:0/1029782793 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55656555da10 0x556565575750 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.006+0000 7f10c324b740 1 --2- 172.21.15.97:0/1029782793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556565550e40 0x55656555cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.006+0000 7f10c324b740 1 --2- 172.21.15.97:0/1029782793 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556565550520 0x5565655694e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.006+0000 7f10c324b740 1 -- 172.21.15.97:0/1029782793 >> 172.21.15.97:0/1029782793 conn(0x55656554cb30 msgr2=0x55656555a570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:34:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.006+0000 7f10c324b740 1 -- 172.21.15.97:0/1029782793 shutdown_connections 2024-04-23T10:34:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.006+0000 7f10c324b740 1 -- 172.21.15.97:0/1029782793 wait complete. 2024-04-23T10:34:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.006+0000 7f10c324b740 1 -- start start 2024-04-23T10:34:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.006+0000 7f10c324b740 1 Processor -- start 2024-04-23T10:34:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.007+0000 7f10c324b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556565550520 0x556565565b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.007+0000 7f10c324b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556565550e40 0x556565560fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.007+0000 7f10c324b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55656555da10 0x5565655614f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.007+0000 7f10c324b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f10b81acd20 con 0x556565550520 2024-04-23T10:34:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.007+0000 7f10c324b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f10b81ac650 con 0x556565550e40 2024-04-23T10:34:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.007+0000 7f10c324b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f10b81a98b0 con 0x55656555da10 2024-04-23T10:34:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.007+0000 7f10c324b740 10 client.0 _ll_register_callbacks cb 0x7f10b400de10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:34:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.007+0000 7f10bdad8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556565550520 0x556565565b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.007+0000 7f10bdad8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556565550520 0x556565565b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58098/0 (socket says 172.21.15.97:58098) 2024-04-23T10:34:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.007+0000 7f10bdad8640 1 -- 172.21.15.97:0/3927425996 learned_addr learned my addr 172.21.15.97:0/3927425996 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:34:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.007+0000 7f10be2d9640 1 --2- 172.21.15.97:0/3927425996 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556565550e40 0x556565560fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.007+0000 7f10bdad8640 1 -- 172.21.15.97:0/3927425996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55656555da10 msgr2=0x5565655614f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:34:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.007+0000 7f10bdad8640 1 --2- 172.21.15.97:0/3927425996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55656555da10 0x5565655614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.007+0000 7f10bdad8640 1 -- 172.21.15.97:0/3927425996 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556565550e40 msgr2=0x556565560fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:34:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.007+0000 7f10bdad8640 1 --2- 172.21.15.97:0/3927425996 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556565550e40 0x556565560fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.007+0000 7f10bdad8640 1 -- 172.21.15.97:0/3927425996 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556565561ab0 con 0x556565550520 2024-04-23T10:34:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.007+0000 7f10be2d9640 1 --2- 172.21.15.97:0/3927425996 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556565550e40 0x556565560fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:34:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.007+0000 7f10bdad8640 1 --2- 172.21.15.97:0/3927425996 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556565550520 0x556565565b50 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f10a8005ec0 tx=0x7f10a8009660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e74aeed233f910d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:34:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.007+0000 7f10a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:34:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.007+0000 7f10a1ffb640 1 -- 172.21.15.97:0/3927425996 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a805a020 con 0x556565550520 2024-04-23T10:34:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.007+0000 7f10a1ffb640 1 -- 172.21.15.97:0/3927425996 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7f10a8004040 con 0x556565550520 2024-04-23T10:34:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.007+0000 7f10c324b740 1 -- 172.21.15.97:0/3927425996 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10a8056410 con 0x556565550520 2024-04-23T10:34:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.007+0000 7f10a1ffb640 1 -- 172.21.15.97:0/3927425996 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f10a8035040 con 0x556565550520 2024-04-23T10:34:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.007+0000 7f10c324b740 10 client.16689 fetch_fsmap learned FSMap version 26 2024-04-23T10:34:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.007+0000 7f10c324b740 1 -- 172.21.15.97:0/3927425996 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5565655d3cd0 con 0x556565550520 2024-04-23T10:34:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.008+0000 7f10a1ffb640 1 -- 172.21.15.97:0/3927425996 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10a8056410 con 0x556565550520 2024-04-23T10:34:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.008+0000 7f10a1ffb640 1 -- 172.21.15.97:0/3927425996 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f10a8056ce0 con 0x556565550520 2024-04-23T10:34:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.008+0000 7f10c324b740 10 client.16689 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:34:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.008+0000 7f10c324b740 10 client.16689 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:34:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.008+0000 7f10c324b740 20 client.16689 populate_metadata read hostname 'smithi097' 2024-04-23T10:34:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.008+0000 7f10c324b740 1 --2- 172.21.15.97:0/3927425996 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5565655d3f50 0x5565655d6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.008+0000 7f10c324b740 4 client.16689 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:34:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.008+0000 7f10c324b740 1 -- 172.21.15.97:0/3927425996 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5565654bd200 con 0x5565655d3f50 2024-04-23T10:34:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.008+0000 7f10be2d9640 1 --2- 172.21.15.97:0/3927425996 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5565655d3f50 0x5565655d6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.008+0000 7f10a1ffb640 10 client.16689 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:34:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.008+0000 7f10be2d9640 1 --2- 172.21.15.97:0/3927425996 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5565655d3f50 0x5565655d6340 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=865ffd127641f43a server_cookie=5ca63b9058898ee7 in_seq=0 out_seq=0 2024-04-23T10:34:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.022+0000 7f10a1ffb640 1 -- 172.21.15.97:0/3927425996 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5565654bd200 con 0x5565655d3f50 2024-04-23T10:34:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.022+0000 7f10a1ffb640 10 client.16689 handle_command_reply: tid=0 2024-04-23T10:34:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.022+0000 7f10c324b740 1 client.16689 shutdown 2024-04-23T10:34:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.022+0000 7f10c324b740 1 -- 172.21.15.97:0/3927425996 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556565550520 msgr2=0x556565565b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.022+0000 7f10c324b740 1 --2- 172.21.15.97:0/3927425996 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556565550520 0x556565565b50 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f10a8005ec0 tx=0x7f10a8009660 comp rx=0 tx=0).stop 2024-04-23T10:34:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.022+0000 7f10c324b740 1 -- 172.21.15.97:0/3927425996 shutdown_connections 2024-04-23T10:34:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.022+0000 7f10c324b740 1 --2- 172.21.15.97:0/3927425996 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5565655d3f50 0x5565655d6340 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.022+0000 7f10c324b740 1 --2- 172.21.15.97:0/3927425996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55656555da10 0x5565655614f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.022+0000 7f10c324b740 1 --2- 172.21.15.97:0/3927425996 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556565550e40 0x556565560fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.022+0000 7f10a1ffb640 0 client.16689 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:34:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.022+0000 7f10c324b740 1 --2- 172.21.15.97:0/3927425996 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556565550520 0x556565565b50 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.022+0000 7f10c324b740 1 -- 172.21.15.97:0/3927425996 >> 172.21.15.97:0/3927425996 conn(0x55656554cb30 msgr2=0x556565555470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:34:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.022+0000 7f10c324b740 1 -- 172.21.15.97:0/3927425996 shutdown_connections 2024-04-23T10:34:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.022+0000 7f10c324b740 1 -- 172.21.15.97:0/3927425996 wait complete. 2024-04-23T10:34:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.022+0000 7f10c324b740 20 client.16689 trim_cache size 0 max 16384 2024-04-23T10:34:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:34:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:34:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:34:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:34:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.023+0000 7f10c053d640 1 -- 172.21.15.97:0/1735420562 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f10a40a3dc0 msgr2=0x7f10a40a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.023+0000 7f10c053d640 1 --2- 172.21.15.97:0/1735420562 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f10a40a3dc0 0x7f10a40a6280 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f10b00cda80 tx=0x7f10b0098100 comp rx=0 tx=0).stop 2024-04-23T10:34:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.023+0000 7f10c053d640 1 -- 172.21.15.97:0/1735420562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10b8106660 msgr2=0x7f10b8197b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.023+0000 7f10c053d640 1 --2- 172.21.15.97:0/1735420562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10b8106660 0x7f10b8197b30 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f10b000b970 tx=0x7f10b000be30 comp rx=0 tx=0).stop 2024-04-23T10:34:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.023+0000 7f10c053d640 1 -- 172.21.15.97:0/1735420562 shutdown_connections 2024-04-23T10:34:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.023+0000 7f10c053d640 1 --2- 172.21.15.97:0/1735420562 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f10a40a3dc0 0x7f10a40a6280 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.023+0000 7f10c053d640 1 --2- 172.21.15.97:0/1735420562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10b8106660 0x7f10b8197b30 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.023+0000 7f10c053d640 1 --2- 172.21.15.97:0/1735420562 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f10b8100f80 0x7f10b819c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.023+0000 7f10c053d640 1 --2- 172.21.15.97:0/1735420562 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10b81005c0 0x7f10b819c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.023+0000 7f10c053d640 1 -- 172.21.15.97:0/1735420562 >> 172.21.15.97:0/1735420562 conn(0x7f10b8077390 msgr2=0x7f10b80fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:34:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.023+0000 7f10c053d640 1 -- 172.21.15.97:0/1735420562 shutdown_connections 2024-04-23T10:34:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:20.023+0000 7f10c053d640 1 -- 172.21.15.97:0/1735420562 wait complete. 2024-04-23T10:34:20.121 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 10 2024-04-23T10:34:20.121 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T10:34:20.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.222+0000 7f3719576640 1 -- 172.21.15.97:0/1442459754 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3714100880 msgr2=0x7f3714100d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:20.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.222+0000 7f3719576640 1 --2- 172.21.15.97:0/1442459754 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3714100880 0x7f3714100d00 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f3708009900 tx=0x7f370802a410 comp rx=0 tx=0).stop 2024-04-23T10:34:20.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.222+0000 7f3719576640 1 -- 172.21.15.97:0/1442459754 shutdown_connections 2024-04-23T10:34:20.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.222+0000 7f3719576640 1 --2- 172.21.15.97:0/1442459754 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3714101240 0x7f371410ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:20.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.222+0000 7f3719576640 1 --2- 172.21.15.97:0/1442459754 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3714100880 0x7f3714100d00 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:20.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.222+0000 7f3719576640 1 --2- 172.21.15.97:0/1442459754 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3714106f80 0x7f3714107360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:20.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.222+0000 7f3719576640 1 -- 172.21.15.97:0/1442459754 >> 172.21.15.97:0/1442459754 conn(0x7f37140761d0 msgr2=0x7f37140765e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:34:20.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.222+0000 7f3719576640 1 -- 172.21.15.97:0/1442459754 shutdown_connections 2024-04-23T10:34:20.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.222+0000 7f3719576640 1 -- 172.21.15.97:0/1442459754 wait complete. 2024-04-23T10:34:20.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.223+0000 7f3719576640 1 Processor -- start 2024-04-23T10:34:20.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.223+0000 7f3719576640 1 -- start start 2024-04-23T10:34:20.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.223+0000 7f3719576640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3714100880 0x7f3714198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:20.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.223+0000 7f3719576640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3714101240 0x7f3714198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:20.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.223+0000 7f3719576640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3714106f80 0x7f371419cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:20.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.223+0000 7f3719576640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3714112260 con 0x7f3714100880 2024-04-23T10:34:20.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.223+0000 7f3719576640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f37141120e0 con 0x7f3714101240 2024-04-23T10:34:20.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.223+0000 7f3719576640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f37141123e0 con 0x7f3714106f80 2024-04-23T10:34:20.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.223+0000 7f3712ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3714100880 0x7f3714198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:20.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.223+0000 7f3712ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3714100880 0x7f3714198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58114/0 (socket says 172.21.15.97:58114) 2024-04-23T10:34:20.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.223+0000 7f3712ffd640 1 -- 172.21.15.97:0/4048653326 learned_addr learned my addr 172.21.15.97:0/4048653326 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:34:20.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.223+0000 7f37127fc640 1 --2- 172.21.15.97:0/4048653326 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3714101240 0x7f3714198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:20.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.223+0000 7f37137fe640 1 --2- 172.21.15.97:0/4048653326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3714106f80 0x7f371419cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:20.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.223+0000 7f3712ffd640 1 -- 172.21.15.97:0/4048653326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3714106f80 msgr2=0x7f371419cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:20.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.223+0000 7f3712ffd640 1 --2- 172.21.15.97:0/4048653326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3714106f80 0x7f371419cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:20.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.223+0000 7f3712ffd640 1 -- 172.21.15.97:0/4048653326 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3714101240 msgr2=0x7f3714198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:20.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.223+0000 7f3712ffd640 1 --2- 172.21.15.97:0/4048653326 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3714101240 0x7f3714198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:20.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.223+0000 7f3712ffd640 1 -- 172.21.15.97:0/4048653326 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f371419d6f0 con 0x7f3714100880 2024-04-23T10:34:20.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.224+0000 7f37137fe640 1 --2- 172.21.15.97:0/4048653326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3714106f80 0x7f371419cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:34:20.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.224+0000 7f3712ffd640 1 --2- 172.21.15.97:0/4048653326 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3714100880 0x7f3714198700 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f370400ba50 tx=0x7f370400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:34:20.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.224+0000 7f36fbfff640 1 -- 172.21.15.97:0/4048653326 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3704019070 con 0x7f3714100880 2024-04-23T10:34:20.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.224+0000 7f36fbfff640 1 -- 172.21.15.97:0/4048653326 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f370400cbe0 con 0x7f3714100880 2024-04-23T10:34:20.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.224+0000 7f3719576640 1 -- 172.21.15.97:0/4048653326 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f371419d9e0 con 0x7f3714100880 2024-04-23T10:34:20.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.224+0000 7f36fbfff640 1 -- 172.21.15.97:0/4048653326 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37040075d0 con 0x7f3714100880 2024-04-23T10:34:20.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.224+0000 7f3719576640 1 -- 172.21.15.97:0/4048653326 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37141a5270 con 0x7f3714100880 2024-04-23T10:34:20.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.225+0000 7f3719576640 1 -- 172.21.15.97:0/4048653326 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f371406a080 con 0x7f3714100880 2024-04-23T10:34:20.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.226+0000 7f36fbfff640 1 -- 172.21.15.97:0/4048653326 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f370401c070 con 0x7f3714100880 2024-04-23T10:34:20.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.226+0000 7f36fbfff640 1 --2- 172.21.15.97:0/4048653326 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f36fc0a3d20 0x7f36fc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:20.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.226+0000 7f37127fc640 1 --2- 172.21.15.97:0/4048653326 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f36fc0a3d20 0x7f36fc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:20.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.227+0000 7f36fbfff640 1 -- 172.21.15.97:0/4048653326 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7f37040cfce0 con 0x7f3714100880 2024-04-23T10:34:20.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.227+0000 7f37127fc640 1 --2- 172.21.15.97:0/4048653326 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f36fc0a3d20 0x7f36fc0a61e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3708000c00 tx=0x7f3708005e80 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:34:20.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.231+0000 7f36fbfff640 1 -- 172.21.15.97:0/4048653326 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f370409be60 con 0x7f3714100880 2024-04-23T10:34:20.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.445+0000 7f3719576640 1 -- 172.21.15.97:0/4048653326 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3714199a50 con 0x7f3714100880 2024-04-23T10:34:20.445 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.446+0000 7f36fbfff640 1 -- 172.21.15.97:0/4048653326 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11931 (secure 0 0 0) 0x7f3714199a50 con 0x7f3714100880 2024-04-23T10:34:20.445 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:34:20.445 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3946878762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3946878762},{"type":"v1","addr":"172.21.15.97:6835","nonce":3946878762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/209800861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":209800861},{"type":"v1","addr":"172.21.15.97:6837","nonce":209800861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T10:24:06.910134+0000","modified":"2024-04-23T10:26:09.820985+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6839/418013575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":418013575},{"type":"v1","addr":"172.21.15.97:6839","nonce":418013575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6841/747242680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":747242680},{"type":"v1","addr":"172.21.15.97:6841","nonce":747242680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6833/4292847121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4292847121},{"type":"v1","addr":"172.21.15.183:6833","nonce":4292847121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6833/3885816565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3885816565},{"type":"v1","addr":"172.21.15.196:6833","nonce":3885816565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1897093105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1897093105},{"type":"v1","addr":"172.21.15.183:6835","nonce":1897093105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1171477563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1171477563},{"type":"v1","addr":"172.21.15.196:6835","nonce":1171477563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1898341911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1898341911},{"type":"v1","addr":"172.21.15.183:6837","nonce":1898341911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.196:6837/3094946502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3094946502},{"type":"v1","addr":"172.21.15.196:6837","nonce":3094946502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.196:6839/83843556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":83843556},{"type":"v1","addr":"172.21.15.196:6839","nonce":83843556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6839/1763841130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1763841130},{"type":"v1","addr":"172.21.15.183:6839","nonce":1763841130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24487,24403,24398,24469,24488]},"id":1}]} 2024-04-23T10:34:20.445 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 26 2024-04-23T10:34:20.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.448+0000 7f3719576640 1 -- 172.21.15.97:0/4048653326 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f36fc0a3d20 msgr2=0x7f36fc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:20.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.448+0000 7f3719576640 1 --2- 172.21.15.97:0/4048653326 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f36fc0a3d20 0x7f36fc0a61e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3708000c00 tx=0x7f3708005e80 comp rx=0 tx=0).stop 2024-04-23T10:34:20.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.448+0000 7f3719576640 1 -- 172.21.15.97:0/4048653326 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3714100880 msgr2=0x7f3714198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:20.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.448+0000 7f3719576640 1 --2- 172.21.15.97:0/4048653326 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3714100880 0x7f3714198700 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f370400ba50 tx=0x7f370400bf10 comp rx=0 tx=0).stop 2024-04-23T10:34:20.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.448+0000 7f3719576640 1 -- 172.21.15.97:0/4048653326 shutdown_connections 2024-04-23T10:34:20.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.448+0000 7f3719576640 1 --2- 172.21.15.97:0/4048653326 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f36fc0a3d20 0x7f36fc0a61e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:20.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.448+0000 7f3719576640 1 --2- 172.21.15.97:0/4048653326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3714106f80 0x7f371419cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:20.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.448+0000 7f3719576640 1 --2- 172.21.15.97:0/4048653326 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3714101240 0x7f3714198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:20.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.448+0000 7f3719576640 1 --2- 172.21.15.97:0/4048653326 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3714100880 0x7f3714198700 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:20.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.448+0000 7f3719576640 1 -- 172.21.15.97:0/4048653326 >> 172.21.15.97:0/4048653326 conn(0x7f37140761d0 msgr2=0x7f37140fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:34:20.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.448+0000 7f3719576640 1 -- 172.21.15.97:0/4048653326 shutdown_connections 2024-04-23T10:34:20.448 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.448+0000 7f3719576640 1 -- 172.21.15.97:0/4048653326 wait complete. 2024-04-23T10:34:20.458 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db / --timeout 60 --expiration 70 --await-for 42.0' 2024-04-23T10:34:20.458 DEBUG:teuthology.orchestra.run.smithi097:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24488 quiesce db / --timeout 60 --expiration 70 --await-for 42.0 2024-04-23T10:34:20.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.567+0000 7f18569ef640 1 -- 172.21.15.97:0/1397277931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1850101a50 msgr2=0x7f1850105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:20.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.567+0000 7f18569ef640 1 --2- 172.21.15.97:0/1397277931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1850101a50 0x7f1850105a90 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f1844009900 tx=0x7f184402a410 comp rx=0 tx=0).stop 2024-04-23T10:34:20.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.567+0000 7f18569ef640 1 -- 172.21.15.97:0/1397277931 shutdown_connections 2024-04-23T10:34:20.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.567+0000 7f18569ef640 1 --2- 172.21.15.97:0/1397277931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1850106170 0x7f185010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:20.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.567+0000 7f18569ef640 1 --2- 172.21.15.97:0/1397277931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1850101a50 0x7f1850105a90 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:20.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.567+0000 7f18569ef640 1 --2- 172.21.15.97:0/1397277931 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f18501010a0 0x7f1850101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:20.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.567+0000 7f18569ef640 1 -- 172.21.15.97:0/1397277931 >> 172.21.15.97:0/1397277931 conn(0x7f1850077f50 msgr2=0x7f1850078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:34:20.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.568+0000 7f18569ef640 1 -- 172.21.15.97:0/1397277931 shutdown_connections 2024-04-23T10:34:20.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.568+0000 7f18569ef640 1 -- 172.21.15.97:0/1397277931 wait complete. 2024-04-23T10:34:20.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.568+0000 7f18569ef640 1 Processor -- start 2024-04-23T10:34:20.567 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.568+0000 7f18569ef640 1 -- start start 2024-04-23T10:34:20.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.568+0000 7f18569ef640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18501010a0 0x7f1850197630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:20.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.568+0000 7f18569ef640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1850106170 0x7f1850197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:20.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.568+0000 7f18569ef640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f185019bef0 0x7f185019c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:20.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.568+0000 7f18569ef640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f18501110b0 con 0x7f18501010a0 2024-04-23T10:34:20.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.568+0000 7f18569ef640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1850110f30 con 0x7f1850106170 2024-04-23T10:34:20.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.568+0000 7f18569ef640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1850111230 con 0x7f185019bef0 2024-04-23T10:34:20.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.569+0000 7f184ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18501010a0 0x7f1850197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:20.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.569+0000 7f184ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18501010a0 0x7f1850197630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58130/0 (socket says 172.21.15.97:58130) 2024-04-23T10:34:20.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.569+0000 7f184ffff640 1 -- 172.21.15.97:0/4257275620 learned_addr learned my addr 172.21.15.97:0/4257275620 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:34:20.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.569+0000 7f184f7fe640 1 --2- 172.21.15.97:0/4257275620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1850106170 0x7f1850197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:20.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.569+0000 7f1854f8c640 1 --2- 172.21.15.97:0/4257275620 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f185019bef0 0x7f185019c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:20.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.569+0000 7f184ffff640 1 -- 172.21.15.97:0/4257275620 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f185019bef0 msgr2=0x7f185019c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:20.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.569+0000 7f184ffff640 1 --2- 172.21.15.97:0/4257275620 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f185019bef0 0x7f185019c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:20.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.569+0000 7f184ffff640 1 -- 172.21.15.97:0/4257275620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1850106170 msgr2=0x7f1850197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:20.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.569+0000 7f184ffff640 1 --2- 172.21.15.97:0/4257275620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1850106170 0x7f1850197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:20.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.569+0000 7f184ffff640 1 -- 172.21.15.97:0/4257275620 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f185019c9e0 con 0x7f18501010a0 2024-04-23T10:34:20.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.569+0000 7f184f7fe640 1 --2- 172.21.15.97:0/4257275620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1850106170 0x7f1850197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:34:20.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.569+0000 7f1854f8c640 1 --2- 172.21.15.97:0/4257275620 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f185019bef0 0x7f185019c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:34:20.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.569+0000 7f184ffff640 1 --2- 172.21.15.97:0/4257275620 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18501010a0 0x7f1850197630 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f184000d770 tx=0x7f184000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:34:20.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.569+0000 7f184d7fa640 1 -- 172.21.15.97:0/4257275620 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1840017070 con 0x7f18501010a0 2024-04-23T10:34:20.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.569+0000 7f18569ef640 1 -- 172.21.15.97:0/4257275620 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f185019ccd0 con 0x7f18501010a0 2024-04-23T10:34:20.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.569+0000 7f184d7fa640 1 -- 172.21.15.97:0/4257275620 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18400054d0 con 0x7f18501010a0 2024-04-23T10:34:20.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.569+0000 7f184d7fa640 1 -- 172.21.15.97:0/4257275620 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1840004c30 con 0x7f18501010a0 2024-04-23T10:34:20.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.569+0000 7f18569ef640 1 -- 172.21.15.97:0/4257275620 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18501a4490 con 0x7f18501010a0 2024-04-23T10:34:20.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.571+0000 7f184d7fa640 1 -- 172.21.15.97:0/4257275620 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1840004e10 con 0x7f18501010a0 2024-04-23T10:34:20.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.572+0000 7f18597d7740 1 Processor -- start 2024-04-23T10:34:20.576 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.572+0000 7f18597d7740 1 -- start start 2024-04-23T10:34:20.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.572+0000 7f18597d7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f9625b7fa0 0x55f9625b8380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:20.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.572+0000 7f18597d7740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9625b88c0 0x55f9625c4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:20.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.572+0000 7f18597d7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f9625c52a0 0x55f9625c7690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:20.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.572+0000 7f18597d7740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f96251d370 con 0x55f9625b88c0 2024-04-23T10:34:20.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.572+0000 7f18597d7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f9624ba2e0 con 0x55f9625b7fa0 2024-04-23T10:34:20.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.572+0000 7f184d7fa640 1 --2- 172.21.15.97:0/4257275620 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f18380a3d20 0x7f18380a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:20.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.572+0000 7f18597d7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f962519330 con 0x55f9625c52a0 2024-04-23T10:34:20.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.572+0000 7f184ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9625b88c0 0x55f9625c4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:20.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.572+0000 7f1854f8c640 1 --2- 172.21.15.97:0/4257275620 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f18380a3d20 0x7f18380a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:20.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.572+0000 7f184ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9625b88c0 0x55f9625c4d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58134/0 (socket says 172.21.15.97:58134) 2024-04-23T10:34:20.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.572+0000 7f184ffff640 1 -- 172.21.15.97:0/4265749517 learned_addr learned my addr 172.21.15.97:0/4265749517 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:34:20.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.572+0000 7f1854f8c640 1 --2- 172.21.15.97:0/4265749517 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f9625b7fa0 0x55f9625b8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:20.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.572+0000 7f184f7fe640 1 --2- 172.21.15.97:0/4265749517 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f9625c52a0 0x55f9625c7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:20.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.572+0000 7f184d7fa640 1 -- 172.21.15.97:0/4257275620 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7f18400d0360 con 0x7f18501010a0 2024-04-23T10:34:20.577 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.572+0000 7f184ffff640 1 -- 172.21.15.97:0/4265749517 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f9625c52a0 msgr2=0x55f9625c7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:34:20.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.572+0000 7f184ffff640 1 --2- 172.21.15.97:0/4265749517 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f9625c52a0 0x55f9625c7690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:20.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.572+0000 7f184ffff640 1 -- 172.21.15.97:0/4265749517 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f9625b7fa0 msgr2=0x55f9625b8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:34:20.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.572+0000 7f184ffff640 1 --2- 172.21.15.97:0/4265749517 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f9625b7fa0 0x55f9625b8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:20.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.572+0000 7f184ffff640 1 -- 172.21.15.97:0/4265749517 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9622fbd80 con 0x55f9625b88c0 2024-04-23T10:34:20.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.573+0000 7f184ffff640 1 --2- 172.21.15.97:0/4265749517 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9625b88c0 0x55f9625c4d60 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f184000d740 tx=0x7f184006dd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d97a2d6c6293166e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:34:20.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.573+0000 7f1854f8c640 1 --2- 172.21.15.97:0/4257275620 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f18380a3d20 0x7f18380a61e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1848009fd0 tx=0x7f1848009dc0 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:34:20.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.573+0000 7f18367fc640 1 -- 172.21.15.97:0/4265749517 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1840020040 con 0x55f9625b88c0 2024-04-23T10:34:20.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.573+0000 7f18367fc640 1 -- 172.21.15.97:0/4265749517 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1840005680 con 0x55f9625b88c0 2024-04-23T10:34:20.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.573+0000 7f18367fc640 1 -- 172.21.15.97:0/4265749517 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18400a4f90 con 0x55f9625b88c0 2024-04-23T10:34:20.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.573+0000 7f18597d7740 1 -- 172.21.15.97:0/4265749517 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9625b88c0 msgr2=0x55f9625c4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:20.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.573+0000 7f18597d7740 1 --2- 172.21.15.97:0/4265749517 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9625b88c0 0x55f9625c4d60 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f184000d740 tx=0x7f184006dd60 comp rx=0 tx=0).stop 2024-04-23T10:34:20.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.573+0000 7f18597d7740 1 -- 172.21.15.97:0/4265749517 shutdown_connections 2024-04-23T10:34:20.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.573+0000 7f18597d7740 1 --2- 172.21.15.97:0/4265749517 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f9625c52a0 0x55f9625c7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:20.578 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.573+0000 7f18597d7740 1 --2- 172.21.15.97:0/4265749517 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9625b88c0 0x55f9625c4d60 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:20.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.573+0000 7f18597d7740 1 --2- 172.21.15.97:0/4265749517 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f9625b7fa0 0x55f9625b8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:20.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.573+0000 7f18597d7740 1 -- 172.21.15.97:0/4265749517 >> 172.21.15.97:0/4265749517 conn(0x55f9625b43c0 msgr2=0x55f9625b47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:34:20.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.573+0000 7f18597d7740 1 -- 172.21.15.97:0/4265749517 shutdown_connections 2024-04-23T10:34:20.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.573+0000 7f18597d7740 1 -- 172.21.15.97:0/4265749517 wait complete. 2024-04-23T10:34:20.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.574+0000 7f18597d7740 1 -- start start 2024-04-23T10:34:20.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.574+0000 7f18597d7740 1 Processor -- start 2024-04-23T10:34:20.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.574+0000 7f18597d7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f9625b7fa0 0x55f9625c0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:20.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.574+0000 7f18597d7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f9625b88c0 0x55f9625c0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:20.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.574+0000 7f18597d7740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9625c52a0 0x55f9625cee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:20.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.574+0000 7f18597d7740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f9625bad30 con 0x55f9625c52a0 2024-04-23T10:34:20.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.574+0000 7f18597d7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f9624b7fb0 con 0x55f9625b7fa0 2024-04-23T10:34:20.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.574+0000 7f18597d7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f9625baeb0 con 0x55f9625b88c0 2024-04-23T10:34:20.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.574+0000 7f18597d7740 10 client.0 _ll_register_callbacks cb 0x55f9624b79a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:34:20.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.574+0000 7f1854f8c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9625c52a0 0x55f9625cee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:20.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.574+0000 7f1854f8c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9625c52a0 0x55f9625cee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58140/0 (socket says 172.21.15.97:58140) 2024-04-23T10:34:20.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.574+0000 7f1854f8c640 1 -- 172.21.15.97:0/372955809 learned_addr learned my addr 172.21.15.97:0/372955809 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:34:20.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.574+0000 7f184ffff640 1 --2- 172.21.15.97:0/372955809 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f9625b7fa0 0x55f9625c0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:20.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.574+0000 7f184f7fe640 1 --2- 172.21.15.97:0/372955809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f9625b88c0 0x55f9625c0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:20.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.575+0000 7f1854f8c640 1 -- 172.21.15.97:0/372955809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f9625b88c0 msgr2=0x55f9625c0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:34:20.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.575+0000 7f1854f8c640 1 --2- 172.21.15.97:0/372955809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f9625b88c0 0x55f9625c0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:20.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.575+0000 7f1854f8c640 1 -- 172.21.15.97:0/372955809 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f9625b7fa0 msgr2=0x55f9625c0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:34:20.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.575+0000 7f1854f8c640 1 --2- 172.21.15.97:0/372955809 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f9625b7fa0 0x55f9625c0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:20.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.575+0000 7f1854f8c640 1 -- 172.21.15.97:0/372955809 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f96235c510 con 0x55f9625c52a0 2024-04-23T10:34:20.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.575+0000 7f1854f8c640 1 --2- 172.21.15.97:0/372955809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9625c52a0 0x55f9625cee60 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f1848003bd0 tx=0x7f1848009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7491b0d23f9a942a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:34:20.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.575+0000 7f182f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:34:20.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.575+0000 7f182f7fe640 1 -- 172.21.15.97:0/372955809 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1848025070 con 0x55f9625c52a0 2024-04-23T10:34:20.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.575+0000 7f182f7fe640 1 -- 172.21.15.97:0/372955809 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7f1848004700 con 0x55f9625c52a0 2024-04-23T10:34:20.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.575+0000 7f18597d7740 1 -- 172.21.15.97:0/372955809 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1840005680 con 0x55f9625c52a0 2024-04-23T10:34:20.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.575+0000 7f182f7fe640 1 -- 172.21.15.97:0/372955809 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1848031050 con 0x55f9625c52a0 2024-04-23T10:34:20.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.575+0000 7f18597d7740 10 client.16707 fetch_fsmap learned FSMap version 26 2024-04-23T10:34:20.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.575+0000 7f18597d7740 1 -- 172.21.15.97:0/372955809 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f9621ef670 con 0x55f9625c52a0 2024-04-23T10:34:20.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.575+0000 7f182f7fe640 1 -- 172.21.15.97:0/372955809 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1840005680 con 0x55f9625c52a0 2024-04-23T10:34:20.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.575+0000 7f182f7fe640 1 -- 172.21.15.97:0/372955809 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1848012450 con 0x55f9625c52a0 2024-04-23T10:34:20.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.575+0000 7f18597d7740 10 client.16707 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:34:20.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.575+0000 7f18597d7740 10 client.16707 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:34:20.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.575+0000 7f18597d7740 20 client.16707 populate_metadata read hostname 'smithi097' 2024-04-23T10:34:20.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.575+0000 7f18597d7740 1 --2- 172.21.15.97:0/372955809 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55f9625d33b0 0x55f9625d5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:20.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.575+0000 7f18597d7740 4 client.16707 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:34:20.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.575+0000 7f18597d7740 1 -- 172.21.15.97:0/372955809 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f96252ad90 con 0x55f9625d33b0 2024-04-23T10:34:20.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.576+0000 7f184ffff640 1 --2- 172.21.15.97:0/372955809 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55f9625d33b0 0x55f9625d5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:20.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.576+0000 7f184ffff640 1 --2- 172.21.15.97:0/372955809 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55f9625d33b0 0x55f9625d5820 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f8efba0ca1a8ec0 server_cookie=84aa0c9e5d9ed34b in_seq=0 out_seq=0 2024-04-23T10:34:20.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.576+0000 7f182f7fe640 10 client.16707 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:34:20.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.577+0000 7f182f7fe640 1 -- 172.21.15.97:0/372955809 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f96252ad90 con 0x55f9625d33b0 2024-04-23T10:34:20.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.577+0000 7f182f7fe640 10 client.16707 handle_command_reply: tid=0 2024-04-23T10:34:20.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.577+0000 7f18597d7740 1 client.16707 shutdown 2024-04-23T10:34:20.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.577+0000 7f18597d7740 1 -- 172.21.15.97:0/372955809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9625c52a0 msgr2=0x55f9625cee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:20.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.577+0000 7f18597d7740 1 --2- 172.21.15.97:0/372955809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9625c52a0 0x55f9625cee60 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f1848003bd0 tx=0x7f1848009500 comp rx=0 tx=0).stop 2024-04-23T10:34:20.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.577+0000 7f18597d7740 1 -- 172.21.15.97:0/372955809 shutdown_connections 2024-04-23T10:34:20.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.577+0000 7f18597d7740 1 --2- 172.21.15.97:0/372955809 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55f9625d33b0 0x55f9625d5820 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:20.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.577+0000 7f18597d7740 1 --2- 172.21.15.97:0/372955809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9625c52a0 0x55f9625cee60 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:20.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.577+0000 7f182f7fe640 0 client.16707 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:34:20.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.577+0000 7f18597d7740 1 --2- 172.21.15.97:0/372955809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f9625b88c0 0x55f9625c0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:20.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.577+0000 7f18597d7740 1 --2- 172.21.15.97:0/372955809 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f9625b7fa0 0x55f9625c0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:20.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.577+0000 7f18597d7740 1 -- 172.21.15.97:0/372955809 >> 172.21.15.97:0/372955809 conn(0x55f9625b43c0 msgr2=0x55f9625ba1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:34:20.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.577+0000 7f18597d7740 1 -- 172.21.15.97:0/372955809 shutdown_connections 2024-04-23T10:34:20.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.577+0000 7f18597d7740 1 -- 172.21.15.97:0/372955809 wait complete. 2024-04-23T10:34:20.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.577+0000 7f18597d7740 20 client.16707 trim_cache size 0 max 16384 2024-04-23T10:34:20.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.594+0000 7f18597d7740 1 Processor -- start 2024-04-23T10:34:20.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.594+0000 7f18597d7740 1 -- start start 2024-04-23T10:34:20.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.594+0000 7f18597d7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f9625b7fa0 0x55f9625db0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:20.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.594+0000 7f18597d7740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9625b88c0 0x55f9625cee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:20.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.594+0000 7f18597d7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f9625c52a0 0x55f9625db5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:20.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.594+0000 7f18597d7740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f18501ab8b0 con 0x55f9625b88c0 2024-04-23T10:34:20.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.594+0000 7f18597d7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f18501ad3f0 con 0x55f9625b7fa0 2024-04-23T10:34:20.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.594+0000 7f18597d7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f18501abf80 con 0x55f9625c52a0 2024-04-23T10:34:20.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.594+0000 7f184f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9625b88c0 0x55f9625cee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:20.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.594+0000 7f184f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9625b88c0 0x55f9625cee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58144/0 (socket says 172.21.15.97:58144) 2024-04-23T10:34:20.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.594+0000 7f184f7fe640 1 -- 172.21.15.97:0/3105106012 learned_addr learned my addr 172.21.15.97:0/3105106012 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:34:20.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.595+0000 7f184ffff640 1 --2- 172.21.15.97:0/3105106012 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f9625b7fa0 0x55f9625db0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:20.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.595+0000 7f184f7fe640 1 -- 172.21.15.97:0/3105106012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f9625c52a0 msgr2=0x55f9625db5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:34:20.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.595+0000 7f1854f8c640 1 --2- 172.21.15.97:0/3105106012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f9625c52a0 0x55f9625db5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:20.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.595+0000 7f184f7fe640 1 --2- 172.21.15.97:0/3105106012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f9625c52a0 0x55f9625db5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:20.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.595+0000 7f184f7fe640 1 -- 172.21.15.97:0/3105106012 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f9625b7fa0 msgr2=0x55f9625db0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:34:20.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.595+0000 7f184f7fe640 1 --2- 172.21.15.97:0/3105106012 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f9625b7fa0 0x55f9625db0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:20.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.595+0000 7f184f7fe640 1 -- 172.21.15.97:0/3105106012 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9625c4fb0 con 0x55f9625b88c0 2024-04-23T10:34:20.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.595+0000 7f184ffff640 1 --2- 172.21.15.97:0/3105106012 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f9625b7fa0 0x55f9625db0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:34:20.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.595+0000 7f184f7fe640 1 --2- 172.21.15.97:0/3105106012 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9625b88c0 0x55f9625cee60 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f1844031440 tx=0x7f1844005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa3fc95f7a7fe6ef server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:34:20.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.595+0000 7f1835ffb640 1 -- 172.21.15.97:0/3105106012 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1844035040 con 0x55f9625b88c0 2024-04-23T10:34:20.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.595+0000 7f1835ffb640 1 -- 172.21.15.97:0/3105106012 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18440316a0 con 0x55f9625b88c0 2024-04-23T10:34:20.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.595+0000 7f1835ffb640 1 -- 172.21.15.97:0/3105106012 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184403b6e0 con 0x55f9625b88c0 2024-04-23T10:34:20.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.595+0000 7f18597d7740 1 -- 172.21.15.97:0/3105106012 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9625b88c0 msgr2=0x55f9625cee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:20.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.595+0000 7f18597d7740 1 --2- 172.21.15.97:0/3105106012 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9625b88c0 0x55f9625cee60 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f1844031440 tx=0x7f1844005b50 comp rx=0 tx=0).stop 2024-04-23T10:34:20.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.596+0000 7f18597d7740 1 -- 172.21.15.97:0/3105106012 shutdown_connections 2024-04-23T10:34:20.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.596+0000 7f18597d7740 1 --2- 172.21.15.97:0/3105106012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f9625c52a0 0x55f9625db5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:20.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.596+0000 7f18597d7740 1 --2- 172.21.15.97:0/3105106012 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9625b88c0 0x55f9625cee60 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:20.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.596+0000 7f18597d7740 1 --2- 172.21.15.97:0/3105106012 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f9625b7fa0 0x55f9625db0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:20.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.596+0000 7f18597d7740 1 -- 172.21.15.97:0/3105106012 >> 172.21.15.97:0/3105106012 conn(0x55f9625b43c0 msgr2=0x55f9625af490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:34:20.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.596+0000 7f18597d7740 1 -- 172.21.15.97:0/3105106012 shutdown_connections 2024-04-23T10:34:20.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.596+0000 7f18597d7740 1 -- 172.21.15.97:0/3105106012 wait complete. 2024-04-23T10:34:20.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.596+0000 7f18597d7740 1 -- start start 2024-04-23T10:34:20.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.596+0000 7f18597d7740 1 Processor -- start 2024-04-23T10:34:20.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.596+0000 7f18597d7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f9625b7fa0 0x55f9625cee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:20.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.596+0000 7f18597d7740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9625b88c0 0x55f9625ca220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:20.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.596+0000 7f18597d7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f9625c52a0 0x55f9625ca960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:20.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.596+0000 7f18597d7740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f18501acd20 con 0x55f9625b88c0 2024-04-23T10:34:20.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.596+0000 7f18597d7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f18501ac650 con 0x55f9625c52a0 2024-04-23T10:34:20.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.596+0000 7f18597d7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f18501a98b0 con 0x55f9625b7fa0 2024-04-23T10:34:20.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.596+0000 7f18597d7740 10 client.0 _ll_register_callbacks cb 0x55f9624b79a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:34:20.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.596+0000 7f184f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9625b88c0 0x55f9625ca220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:20.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.596+0000 7f184f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9625b88c0 0x55f9625ca220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58152/0 (socket says 172.21.15.97:58152) 2024-04-23T10:34:20.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.596+0000 7f184f7fe640 1 -- 172.21.15.97:0/2963911298 learned_addr learned my addr 172.21.15.97:0/2963911298 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:34:20.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.597+0000 7f184ffff640 1 --2- 172.21.15.97:0/2963911298 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f9625b7fa0 0x55f9625cee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:20.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.597+0000 7f184f7fe640 1 -- 172.21.15.97:0/2963911298 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f9625b7fa0 msgr2=0x55f9625cee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:34:20.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.597+0000 7f1854f8c640 1 --2- 172.21.15.97:0/2963911298 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f9625c52a0 0x55f9625ca960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:20.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.597+0000 7f184f7fe640 1 --2- 172.21.15.97:0/2963911298 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f9625b7fa0 0x55f9625cee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:20.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.597+0000 7f184f7fe640 1 -- 172.21.15.97:0/2963911298 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f9625c52a0 msgr2=0x55f9625ca960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:34:20.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.597+0000 7f184f7fe640 1 --2- 172.21.15.97:0/2963911298 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f9625c52a0 0x55f9625ca960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:20.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.597+0000 7f184f7fe640 1 -- 172.21.15.97:0/2963911298 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f962638910 con 0x55f9625b88c0 2024-04-23T10:34:20.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.597+0000 7f184f7fe640 1 --2- 172.21.15.97:0/2963911298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9625b88c0 0x55f9625ca220 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f1844000c00 tx=0x7f1844032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e7eb3ad0692ce1f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:34:20.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.597+0000 7f182f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:34:20.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.597+0000 7f182f7fe640 1 -- 172.21.15.97:0/2963911298 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1844035040 con 0x55f9625b88c0 2024-04-23T10:34:20.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.597+0000 7f182f7fe640 1 -- 172.21.15.97:0/2963911298 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7f1844004460 con 0x55f9625b88c0 2024-04-23T10:34:20.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.597+0000 7f18597d7740 1 -- 172.21.15.97:0/2963911298 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18440316a0 con 0x55f9625b88c0 2024-04-23T10:34:20.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.597+0000 7f182f7fe640 1 -- 172.21.15.97:0/2963911298 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f184405d050 con 0x55f9625b88c0 2024-04-23T10:34:20.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.597+0000 7f18597d7740 10 client.16713 fetch_fsmap learned FSMap version 26 2024-04-23T10:34:20.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.597+0000 7f18597d7740 1 -- 172.21.15.97:0/2963911298 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f962639a00 con 0x55f9625b88c0 2024-04-23T10:34:20.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.597+0000 7f182f7fe640 1 -- 172.21.15.97:0/2963911298 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18440316a0 con 0x55f9625b88c0 2024-04-23T10:34:20.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.597+0000 7f182f7fe640 1 -- 172.21.15.97:0/2963911298 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1844003710 con 0x55f9625b88c0 2024-04-23T10:34:20.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.598+0000 7f18597d7740 10 client.16713 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:34:20.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.598+0000 7f18597d7740 10 client.16713 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:34:20.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.598+0000 7f18597d7740 20 client.16713 populate_metadata read hostname 'smithi097' 2024-04-23T10:34:20.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.598+0000 7f18597d7740 1 --2- 172.21.15.97:0/2963911298 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55f96263a1c0 0x55f96263c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:20.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.598+0000 7f18597d7740 4 client.16713 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 70.0, "await_for": 42.0}] 2024-04-23T10:34:20.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.598+0000 7f18597d7740 1 -- 172.21.15.97:0/2963911298 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 70.0, "await_for": 42.0}) -- 0x55f962524c00 con 0x55f96263a1c0 2024-04-23T10:34:20.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.598+0000 7f184ffff640 1 --2- 172.21.15.97:0/2963911298 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55f96263a1c0 0x55f96263c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:20.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.598+0000 7f182f7fe640 10 client.16713 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:34:20.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.598+0000 7f184ffff640 1 --2- 172.21.15.97:0/2963911298 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55f96263a1c0 0x55f96263c5e0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0c97ff5b6bd6696 server_cookie=b0519e9dbbc53413 in_seq=0 out_seq=0 2024-04-23T10:34:20.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.616+0000 7f182f7fe640 1 -- 172.21.15.97:0/2963911298 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55f962524c00 con 0x55f96263a1c0 2024-04-23T10:34:20.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.616+0000 7f182f7fe640 10 client.16713 handle_command_reply: tid=0 2024-04-23T10:34:20.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.616+0000 7f18597d7740 1 client.16713 shutdown 2024-04-23T10:34:20.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.617+0000 7f18597d7740 1 -- 172.21.15.97:0/2963911298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9625b88c0 msgr2=0x55f9625ca220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:20.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.617+0000 7f18597d7740 1 --2- 172.21.15.97:0/2963911298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9625b88c0 0x55f9625ca220 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f1844000c00 tx=0x7f1844032c10 comp rx=0 tx=0).stop 2024-04-23T10:34:20.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.617+0000 7f18597d7740 1 -- 172.21.15.97:0/2963911298 shutdown_connections 2024-04-23T10:34:20.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.617+0000 7f18597d7740 1 --2- 172.21.15.97:0/2963911298 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55f96263a1c0 0x55f96263c5e0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:20.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.617+0000 7f18597d7740 1 --2- 172.21.15.97:0/2963911298 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f9625c52a0 0x55f9625ca960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:20.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.617+0000 7f18597d7740 1 --2- 172.21.15.97:0/2963911298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9625b88c0 0x55f9625ca220 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:20.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.617+0000 7f182f7fe640 0 client.16713 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:34:20.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.617+0000 7f18597d7740 1 --2- 172.21.15.97:0/2963911298 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f9625b7fa0 0x55f9625cee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:20.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.617+0000 7f18597d7740 1 -- 172.21.15.97:0/2963911298 >> 172.21.15.97:0/2963911298 conn(0x55f9625b43c0 msgr2=0x55f9625bef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:34:20.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.617+0000 7f18597d7740 1 -- 172.21.15.97:0/2963911298 shutdown_connections 2024-04-23T10:34:20.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.617+0000 7f18597d7740 1 -- 172.21.15.97:0/2963911298 wait complete. 2024-04-23T10:34:20.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.617+0000 7f18597d7740 20 client.16713 trim_cache size 0 max 16384 2024-04-23T10:34:20.617 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-23T10:34:20.617 INFO:teuthology.orchestra.run.smithi097.stdout: "epoch": 26, 2024-04-23T10:34:20.617 INFO:teuthology.orchestra.run.smithi097.stdout: "leader": 24488, 2024-04-23T10:34:20.617 INFO:teuthology.orchestra.run.smithi097.stdout: "set_version": 30, 2024-04-23T10:34:20.617 INFO:teuthology.orchestra.run.smithi097.stdout: "sets": { 2024-04-23T10:34:20.617 INFO:teuthology.orchestra.run.smithi097.stdout: "b6473fa6": { 2024-04-23T10:34:20.618 INFO:teuthology.orchestra.run.smithi097.stdout: "version": 30, 2024-04-23T10:34:20.618 INFO:teuthology.orchestra.run.smithi097.stdout: "age_ref": 0.0, 2024-04-23T10:34:20.618 INFO:teuthology.orchestra.run.smithi097.stdout: "state": { 2024-04-23T10:34:20.618 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "QUIESCED", 2024-04-23T10:34:20.618 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0 2024-04-23T10:34:20.618 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:34:20.618 INFO:teuthology.orchestra.run.smithi097.stdout: "timeout": 60.0, 2024-04-23T10:34:20.618 INFO:teuthology.orchestra.run.smithi097.stdout: "expiration": 70.0, 2024-04-23T10:34:20.618 INFO:teuthology.orchestra.run.smithi097.stdout: "members": { 2024-04-23T10:34:20.618 INFO:teuthology.orchestra.run.smithi097.stdout: "file:/": { 2024-04-23T10:34:20.618 INFO:teuthology.orchestra.run.smithi097.stdout: "excluded": false, 2024-04-23T10:34:20.618 INFO:teuthology.orchestra.run.smithi097.stdout: "state": { 2024-04-23T10:34:20.618 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "QUIESCED", 2024-04-23T10:34:20.618 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0 2024-04-23T10:34:20.618 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:34:20.618 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:34:20.619 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:34:20.619 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:34:20.619 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:34:20.619 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-23T10:34:20.619 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.617+0000 7f18569ef640 1 -- 172.21.15.97:0/4257275620 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f18380a3d20 msgr2=0x7f18380a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:20.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.617+0000 7f18569ef640 1 --2- 172.21.15.97:0/4257275620 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f18380a3d20 0x7f18380a61e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1848009fd0 tx=0x7f1848009dc0 comp rx=0 tx=0).stop 2024-04-23T10:34:20.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.617+0000 7f18569ef640 1 -- 172.21.15.97:0/4257275620 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18501010a0 msgr2=0x7f1850197630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:20.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.617+0000 7f18569ef640 1 --2- 172.21.15.97:0/4257275620 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18501010a0 0x7f1850197630 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f184000d770 tx=0x7f184000dc30 comp rx=0 tx=0).stop 2024-04-23T10:34:20.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.617+0000 7f18569ef640 1 -- 172.21.15.97:0/4257275620 shutdown_connections 2024-04-23T10:34:20.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.617+0000 7f18569ef640 1 --2- 172.21.15.97:0/4257275620 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f18380a3d20 0x7f18380a61e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:20.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.617+0000 7f18569ef640 1 --2- 172.21.15.97:0/4257275620 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f185019bef0 0x7f185019c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:20.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.617+0000 7f18569ef640 1 --2- 172.21.15.97:0/4257275620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1850106170 0x7f1850197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:20.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.617+0000 7f18569ef640 1 --2- 172.21.15.97:0/4257275620 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f18501010a0 0x7f1850197630 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:20.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.617+0000 7f18569ef640 1 -- 172.21.15.97:0/4257275620 >> 172.21.15.97:0/4257275620 conn(0x7f1850077f50 msgr2=0x7f18500ff090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:34:20.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.618+0000 7f18569ef640 1 -- 172.21.15.97:0/4257275620 shutdown_connections 2024-04-23T10:34:20.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:20.618+0000 7f18569ef640 1 -- 172.21.15.97:0/4257275620 wait complete. 2024-04-23T10:34:20.627 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'b6473fa6', quiesce took 0.5 seconds. Will release after: 10 2024-04-23T10:34:21.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:34:21 smithi097 ceph-mon[29197]: pgmap v251: 97 pgs: 97 active+clean; 2.9 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-23T10:34:21.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:34:21 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/4048653326' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:34:21.399 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:34:21 smithi183 ceph-mon[32317]: pgmap v251: 97 pgs: 97 active+clean; 2.9 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-23T10:34:21.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:34:21 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/4048653326' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:34:21.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:34:21 smithi196 ceph-mon[32919]: pgmap v251: 97 pgs: 97 active+clean; 2.9 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-23T10:34:21.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:34:21 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/4048653326' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:34:23.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:34:23 smithi097 ceph-mon[29197]: pgmap v252: 97 pgs: 97 active+clean; 2.9 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-23T10:34:23.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:34:23 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:34:23.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:34:23 smithi183 ceph-mon[32317]: pgmap v252: 97 pgs: 97 active+clean; 2.9 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-23T10:34:23.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:34:23 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:34:23.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:34:23 smithi196 ceph-mon[32919]: pgmap v252: 97 pgs: 97 active+clean; 2.9 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-23T10:34:23.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:34:23 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:34:24.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:34:24 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:34:24.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:34:24 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:34:24.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:34:24 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:34:24.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:34:24 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:34:24.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:34:24 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:34:24.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:34:24 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:34:24.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:34:24 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:34:24.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:34:24 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:34:24.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:34:24 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:34:25.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:34:25 smithi097 ceph-mon[29197]: pgmap v253: 97 pgs: 97 active+clean; 3.1 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 52 KiB/s wr, 21 op/s 2024-04-23T10:34:25.403 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:34:25 smithi183 ceph-mon[32317]: pgmap v253: 97 pgs: 97 active+clean; 3.1 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 52 KiB/s wr, 21 op/s 2024-04-23T10:34:25.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:34:25 smithi196 ceph-mon[32919]: pgmap v253: 97 pgs: 97 active+clean; 3.1 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 52 KiB/s wr, 21 op/s 2024-04-23T10:34:27.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:34:27 smithi097 ceph-mon[29197]: pgmap v254: 97 pgs: 97 active+clean; 3.1 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 36 KiB/s wr, 15 op/s 2024-04-23T10:34:27.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:34:27 smithi183 ceph-mon[32317]: pgmap v254: 97 pgs: 97 active+clean; 3.1 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 36 KiB/s wr, 15 op/s 2024-04-23T10:34:27.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:34:27 smithi196 ceph-mon[32919]: pgmap v254: 97 pgs: 97 active+clean; 3.1 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 36 KiB/s wr, 15 op/s 2024-04-23T10:34:29.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:34:29 smithi183 ceph-mon[32317]: pgmap v255: 97 pgs: 97 active+clean; 3.1 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 38 KiB/s wr, 19 op/s 2024-04-23T10:34:29.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:34:29 smithi196 ceph-mon[32919]: pgmap v255: 97 pgs: 97 active+clean; 3.1 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 38 KiB/s wr, 19 op/s 2024-04-23T10:34:29.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:34:29 smithi097 ceph-mon[29197]: pgmap v255: 97 pgs: 97 active+clean; 3.1 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 38 KiB/s wr, 19 op/s 2024-04-23T10:34:30.627 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T10:34:30.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:30.739+0000 7f2eb7e91640 1 -- 172.21.15.97:0/908324865 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2eb01010a0 msgr2=0x7f2eb0101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:30.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:30.739+0000 7f2eb7e91640 1 --2- 172.21.15.97:0/908324865 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2eb01010a0 0x7f2eb0101480 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f2eac009900 tx=0x7f2eac02a410 comp rx=0 tx=0).stop 2024-04-23T10:34:30.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:30.739+0000 7f2eb7e91640 1 -- 172.21.15.97:0/908324865 shutdown_connections 2024-04-23T10:34:30.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:30.739+0000 7f2eb7e91640 1 --2- 172.21.15.97:0/908324865 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2eb0106170 0x7f2eb010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:30.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:30.739+0000 7f2eb7e91640 1 --2- 172.21.15.97:0/908324865 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2eb0101a50 0x7f2eb0105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:30.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:30.739+0000 7f2eb7e91640 1 --2- 172.21.15.97:0/908324865 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2eb01010a0 0x7f2eb0101480 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:30.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:30.739+0000 7f2eb7e91640 1 -- 172.21.15.97:0/908324865 >> 172.21.15.97:0/908324865 conn(0x7f2eb0077f50 msgr2=0x7f2eb0078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:34:30.740 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:30.739+0000 7f2eb7e91640 1 -- 172.21.15.97:0/908324865 shutdown_connections 2024-04-23T10:34:30.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:30.739+0000 7f2eb7e91640 1 -- 172.21.15.97:0/908324865 wait complete. 2024-04-23T10:34:30.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:30.740+0000 7f2eb7e91640 1 Processor -- start 2024-04-23T10:34:30.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:30.740+0000 7f2eb7e91640 1 -- start start 2024-04-23T10:34:30.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:30.740+0000 7f2eb7e91640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2eb01010a0 0x7f2eb0198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:30.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:30.740+0000 7f2eb7e91640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2eb0106170 0x7f2eb0198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:30.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:30.740+0000 7f2eb7e91640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2eb019d020 0x7f2eb019d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:30.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:30.740+0000 7f2eb7e91640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb01110b0 con 0x7f2eb019d020 2024-04-23T10:34:30.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:30.740+0000 7f2eb7e91640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb0110f30 con 0x7f2eb01010a0 2024-04-23T10:34:30.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:30.740+0000 7f2eb7e91640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb0111230 con 0x7f2eb0106170 2024-04-23T10:34:30.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:30.740+0000 7f2eb642e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2eb019d020 0x7f2eb019d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:30.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:30.740+0000 7f2eb642e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2eb019d020 0x7f2eb019d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38034/0 (socket says 172.21.15.97:38034) 2024-04-23T10:34:30.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:30.740+0000 7f2eb542c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2eb0106170 0x7f2eb0198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:30.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:30.740+0000 7f2eb642e640 1 -- 172.21.15.97:0/3645590900 learned_addr learned my addr 172.21.15.97:0/3645590900 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:34:30.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:30.740+0000 7f2eb5c2d640 1 --2- 172.21.15.97:0/3645590900 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2eb01010a0 0x7f2eb0198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:30.741 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:30.741+0000 7f2eb642e640 1 -- 172.21.15.97:0/3645590900 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2eb0106170 msgr2=0x7f2eb0198ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:30.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:30.741+0000 7f2eb642e640 1 --2- 172.21.15.97:0/3645590900 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2eb0106170 0x7f2eb0198ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:30.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:30.741+0000 7f2eb642e640 1 -- 172.21.15.97:0/3645590900 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2eb01010a0 msgr2=0x7f2eb0198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:30.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:30.741+0000 7f2eb642e640 1 --2- 172.21.15.97:0/3645590900 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2eb01010a0 0x7f2eb0198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:30.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:30.741+0000 7f2eb642e640 1 -- 172.21.15.97:0/3645590900 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2eb019db10 con 0x7f2eb019d020 2024-04-23T10:34:30.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:30.741+0000 7f2eb642e640 1 --2- 172.21.15.97:0/3645590900 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2eb019d020 0x7f2eb019d4d0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f2ea800cad0 tx=0x7f2ea800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:34:30.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:30.741+0000 7f2ea6ffd640 1 -- 172.21.15.97:0/3645590900 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea800ae20 con 0x7f2eb019d020 2024-04-23T10:34:30.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:30.741+0000 7f2ea6ffd640 1 -- 172.21.15.97:0/3645590900 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ea8005020 con 0x7f2eb019d020 2024-04-23T10:34:30.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:30.741+0000 7f2eb7e91640 1 -- 172.21.15.97:0/3645590900 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2eb019de00 con 0x7f2eb019d020 2024-04-23T10:34:30.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:30.741+0000 7f2ea6ffd640 1 -- 172.21.15.97:0/3645590900 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea800f6b0 con 0x7f2eb019d020 2024-04-23T10:34:30.742 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:30.741+0000 7f2eb7e91640 1 -- 172.21.15.97:0/3645590900 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2eb01a55a0 con 0x7f2eb019d020 2024-04-23T10:34:30.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:30.742+0000 7f2eb7e91640 1 -- 172.21.15.97:0/3645590900 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2eb0068d70 con 0x7f2eb019d020 2024-04-23T10:34:30.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:30.743+0000 7f2ea6ffd640 1 -- 172.21.15.97:0/3645590900 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2ea80051d0 con 0x7f2eb019d020 2024-04-23T10:34:30.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:30.744+0000 7f2ea6ffd640 1 --2- 172.21.15.97:0/3645590900 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f2e9c0a3d20 0x7f2e9c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:30.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:30.744+0000 7f2eb5c2d640 1 --2- 172.21.15.97:0/3645590900 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f2e9c0a3d20 0x7f2e9c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:30.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:30.744+0000 7f2ea6ffd640 1 -- 172.21.15.97:0/3645590900 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7f2ea80d0790 con 0x7f2eb019d020 2024-04-23T10:34:30.743 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:30.744+0000 7f2eb5c2d640 1 --2- 172.21.15.97:0/3645590900 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f2e9c0a3d20 0x7f2e9c0a61e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2eac031440 tx=0x7f2eac005b50 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:34:30.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:30.748+0000 7f2ea6ffd640 1 -- 172.21.15.97:0/3645590900 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2ea8014030 con 0x7f2eb019d020 2024-04-23T10:34:30.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:30.966+0000 7f2eb7e91640 1 -- 172.21.15.97:0/3645590900 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2eb0101a50 con 0x7f2eb019d020 2024-04-23T10:34:30.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:30.967+0000 7f2ea6ffd640 1 -- 172.21.15.97:0/3645590900 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11931 (secure 0 0 0) 0x7f2ea809c800 con 0x7f2eb019d020 2024-04-23T10:34:30.966 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:34:30.966 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3946878762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3946878762},{"type":"v1","addr":"172.21.15.97:6835","nonce":3946878762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/209800861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":209800861},{"type":"v1","addr":"172.21.15.97:6837","nonce":209800861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T10:24:06.910134+0000","modified":"2024-04-23T10:26:09.820985+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6839/418013575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":418013575},{"type":"v1","addr":"172.21.15.97:6839","nonce":418013575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6841/747242680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":747242680},{"type":"v1","addr":"172.21.15.97:6841","nonce":747242680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6833/4292847121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4292847121},{"type":"v1","addr":"172.21.15.183:6833","nonce":4292847121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6833/3885816565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3885816565},{"type":"v1","addr":"172.21.15.196:6833","nonce":3885816565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1897093105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1897093105},{"type":"v1","addr":"172.21.15.183:6835","nonce":1897093105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1171477563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1171477563},{"type":"v1","addr":"172.21.15.196:6835","nonce":1171477563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1898341911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1898341911},{"type":"v1","addr":"172.21.15.183:6837","nonce":1898341911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.196:6837/3094946502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3094946502},{"type":"v1","addr":"172.21.15.196:6837","nonce":3094946502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.196:6839/83843556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":83843556},{"type":"v1","addr":"172.21.15.196:6839","nonce":83843556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6839/1763841130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1763841130},{"type":"v1","addr":"172.21.15.183:6839","nonce":1763841130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24487,24403,24398,24469,24488]},"id":1}]} 2024-04-23T10:34:30.966 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 26 2024-04-23T10:34:30.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:30.969+0000 7f2eb7e91640 1 -- 172.21.15.97:0/3645590900 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f2e9c0a3d20 msgr2=0x7f2e9c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:30.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:30.969+0000 7f2eb7e91640 1 --2- 172.21.15.97:0/3645590900 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f2e9c0a3d20 0x7f2e9c0a61e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2eac031440 tx=0x7f2eac005b50 comp rx=0 tx=0).stop 2024-04-23T10:34:30.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:30.969+0000 7f2eb7e91640 1 -- 172.21.15.97:0/3645590900 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2eb019d020 msgr2=0x7f2eb019d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:30.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:30.969+0000 7f2eb7e91640 1 --2- 172.21.15.97:0/3645590900 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2eb019d020 0x7f2eb019d4d0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f2ea800cad0 tx=0x7f2ea800cf90 comp rx=0 tx=0).stop 2024-04-23T10:34:30.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:30.969+0000 7f2eb7e91640 1 -- 172.21.15.97:0/3645590900 shutdown_connections 2024-04-23T10:34:30.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:30.969+0000 7f2eb7e91640 1 --2- 172.21.15.97:0/3645590900 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f2e9c0a3d20 0x7f2e9c0a61e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:30.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:30.969+0000 7f2eb7e91640 1 --2- 172.21.15.97:0/3645590900 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2eb019d020 0x7f2eb019d4d0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:30.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:30.969+0000 7f2eb7e91640 1 --2- 172.21.15.97:0/3645590900 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2eb0106170 0x7f2eb0198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:30.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:30.969+0000 7f2eb7e91640 1 --2- 172.21.15.97:0/3645590900 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2eb01010a0 0x7f2eb0198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:30.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:30.969+0000 7f2eb7e91640 1 -- 172.21.15.97:0/3645590900 >> 172.21.15.97:0/3645590900 conn(0x7f2eb0077f50 msgr2=0x7f2eb0069540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:34:30.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:30.970+0000 7f2eb7e91640 1 -- 172.21.15.97:0/3645590900 shutdown_connections 2024-04-23T10:34:30.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:30.970+0000 7f2eb7e91640 1 -- 172.21.15.97:0/3645590900 wait complete. 2024-04-23T10:34:30.979 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --set-id b6473fa6 --release --await' 2024-04-23T10:34:30.979 DEBUG:teuthology.orchestra.run.smithi097:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24488 quiesce db --set-id b6473fa6 --release --await 2024-04-23T10:34:31.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.089+0000 7fb0f3944640 1 -- 172.21.15.97:0/4219298701 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0ec102190 msgr2=0x7fb0ec102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:31.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.089+0000 7fb0f3944640 1 --2- 172.21.15.97:0/4219298701 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0ec102190 0x7fb0ec102570 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc009900 tx=0x7fb0dc02a410 comp rx=0 tx=0).stop 2024-04-23T10:34:31.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.089+0000 7fb0f3944640 1 -- 172.21.15.97:0/4219298701 shutdown_connections 2024-04-23T10:34:31.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.089+0000 7fb0f3944640 1 --2- 172.21.15.97:0/4219298701 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0ec107260 0x7fb0ec10ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:31.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.089+0000 7fb0f3944640 1 --2- 172.21.15.97:0/4219298701 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0ec102b40 0x7fb0ec106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:31.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.089+0000 7fb0f3944640 1 --2- 172.21.15.97:0/4219298701 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0ec102190 0x7fb0ec102570 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:31.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.089+0000 7fb0f3944640 1 -- 172.21.15.97:0/4219298701 >> 172.21.15.97:0/4219298701 conn(0x7fb0ec0793a0 msgr2=0x7fb0ec0797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:34:31.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.089+0000 7fb0f3944640 1 -- 172.21.15.97:0/4219298701 shutdown_connections 2024-04-23T10:34:31.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.089+0000 7fb0f3944640 1 -- 172.21.15.97:0/4219298701 wait complete. 2024-04-23T10:34:31.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.089+0000 7fb0f3944640 1 Processor -- start 2024-04-23T10:34:31.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.089+0000 7fb0f3944640 1 -- start start 2024-04-23T10:34:31.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.089+0000 7fb0f3944640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0ec102190 0x7fb0ec19e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:31.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.089+0000 7fb0f3944640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0ec107260 0x7fb0ec19e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:31.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.089+0000 7fb0f3944640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0ec198590 0x7fb0ec198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:31.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.090+0000 7fb0f3944640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ec112210 con 0x7fb0ec198590 2024-04-23T10:34:31.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.090+0000 7fb0f3944640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ec112090 con 0x7fb0ec102190 2024-04-23T10:34:31.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.090+0000 7fb0f3944640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ec112390 con 0x7fb0ec107260 2024-04-23T10:34:31.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.090+0000 7fb0f1ee1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0ec198590 0x7fb0ec198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:31.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.090+0000 7fb0f1ee1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0ec198590 0x7fb0ec198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38056/0 (socket says 172.21.15.97:38056) 2024-04-23T10:34:31.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.090+0000 7fb0f1ee1640 1 -- 172.21.15.97:0/2875322344 learned_addr learned my addr 172.21.15.97:0/2875322344 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:34:31.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.090+0000 7fb0f16e0640 1 --2- 172.21.15.97:0/2875322344 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0ec102190 0x7fb0ec19e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:31.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.090+0000 7fb0f0edf640 1 --2- 172.21.15.97:0/2875322344 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0ec107260 0x7fb0ec19e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:31.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.090+0000 7fb0f1ee1640 1 -- 172.21.15.97:0/2875322344 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0ec107260 msgr2=0x7fb0ec19e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:31.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.090+0000 7fb0f1ee1640 1 --2- 172.21.15.97:0/2875322344 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0ec107260 0x7fb0ec19e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:31.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.090+0000 7fb0f1ee1640 1 -- 172.21.15.97:0/2875322344 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0ec102190 msgr2=0x7fb0ec19e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:31.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.090+0000 7fb0f1ee1640 1 --2- 172.21.15.97:0/2875322344 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0ec102190 0x7fb0ec19e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:31.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.090+0000 7fb0f1ee1640 1 -- 172.21.15.97:0/2875322344 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0ec1990b0 con 0x7fb0ec198590 2024-04-23T10:34:31.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.090+0000 7fb0f1ee1640 1 --2- 172.21.15.97:0/2875322344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0ec198590 0x7fb0ec198a40 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fb0e400eb10 tx=0x7fb0e400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:34:31.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.090+0000 7fb0e27fc640 1 -- 172.21.15.97:0/2875322344 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e4004b20 con 0x7fb0ec198590 2024-04-23T10:34:31.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.090+0000 7fb0f3944640 1 -- 172.21.15.97:0/2875322344 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0ec1993a0 con 0x7fb0ec198590 2024-04-23T10:34:31.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.090+0000 7fb0e27fc640 1 -- 172.21.15.97:0/2875322344 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0e4004cc0 con 0x7fb0ec198590 2024-04-23T10:34:31.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.090+0000 7fb0e27fc640 1 -- 172.21.15.97:0/2875322344 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e4010670 con 0x7fb0ec198590 2024-04-23T10:34:31.092 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.090+0000 7fb0f3944640 1 -- 172.21.15.97:0/2875322344 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0ec1a55a0 con 0x7fb0ec198590 2024-04-23T10:34:31.094 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.093+0000 7fb0e27fc640 1 -- 172.21.15.97:0/2875322344 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb0e4005020 con 0x7fb0ec198590 2024-04-23T10:34:31.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.093+0000 7fb0f6668740 1 Processor -- start 2024-04-23T10:34:31.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.093+0000 7fb0f6668740 1 -- start start 2024-04-23T10:34:31.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.094+0000 7fb0f6668740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5567f9fed490 0x5567f9fed870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:31.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.094+0000 7fb0f6668740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5567f9feddb0 0x5567f9ff7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:31.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.094+0000 7fb0f6668740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5567f9ff8480 0x5567f9ffa870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:31.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.094+0000 7fb0f6668740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5567f9f526d0 con 0x5567f9ff8480 2024-04-23T10:34:31.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.094+0000 7fb0f6668740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5567f9eef1a0 con 0x5567f9feddb0 2024-04-23T10:34:31.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.094+0000 7fb0f6668740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5567f9f4e690 con 0x5567f9fed490 2024-04-23T10:34:31.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.094+0000 7fb0e27fc640 1 --2- 172.21.15.97:0/2875322344 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fb0d80a3d20 0x7fb0d80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:31.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.094+0000 7fb0f0edf640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5567f9ff8480 0x5567f9ffa870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:31.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.094+0000 7fb0f1ee1640 1 --2- 172.21.15.97:0/2875322344 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fb0d80a3d20 0x7fb0d80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:31.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.094+0000 7fb0f0edf640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5567f9ff8480 0x5567f9ffa870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38070/0 (socket says 172.21.15.97:38070) 2024-04-23T10:34:31.095 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.094+0000 7fb0f0edf640 1 -- 172.21.15.97:0/798980200 learned_addr learned my addr 172.21.15.97:0/798980200 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:34:31.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.094+0000 7fb0f16e0640 1 --2- 172.21.15.97:0/798980200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5567f9feddb0 0x5567f9ff7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:31.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.094+0000 7fb0f0edf640 1 --2- 172.21.15.97:0/798980200 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5567f9fed490 0x5567f9fed870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:31.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.094+0000 7fb0e27fc640 1 -- 172.21.15.97:0/2875322344 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7fb0e40d0ce0 con 0x7fb0ec198590 2024-04-23T10:34:31.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.094+0000 7fb0f0edf640 1 -- 172.21.15.97:0/798980200 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5567f9fed490 msgr2=0x5567f9fed870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:34:31.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.094+0000 7fb0f0edf640 1 --2- 172.21.15.97:0/798980200 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5567f9fed490 0x5567f9fed870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:31.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.094+0000 7fb0f0edf640 1 -- 172.21.15.97:0/798980200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5567f9feddb0 msgr2=0x5567f9ff7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:34:31.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.094+0000 7fb0f0edf640 1 --2- 172.21.15.97:0/798980200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5567f9feddb0 0x5567f9ff7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:31.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.094+0000 7fb0f0edf640 1 -- 172.21.15.97:0/798980200 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567f9ef95b0 con 0x5567f9ff8480 2024-04-23T10:34:31.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.094+0000 7fb0f1ee1640 1 --2- 172.21.15.97:0/2875322344 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fb0d80a3d20 0x7fb0d80a61e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb0e40ce500 tx=0x7fb0e406e6e0 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:34:31.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.094+0000 7fb0f0edf640 1 --2- 172.21.15.97:0/798980200 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5567f9ff8480 0x5567f9ffa870 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fb0e80058c0 tx=0x7fb0e8005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b44446550bbc99ab server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:34:31.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.095+0000 7fb0d77fe640 1 -- 172.21.15.97:0/798980200 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e8005020 con 0x5567f9ff8480 2024-04-23T10:34:31.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.095+0000 7fb0d77fe640 1 -- 172.21.15.97:0/798980200 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0e80047a0 con 0x5567f9ff8480 2024-04-23T10:34:31.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.095+0000 7fb0d77fe640 1 -- 172.21.15.97:0/798980200 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e800fb20 con 0x5567f9ff8480 2024-04-23T10:34:31.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.095+0000 7fb0f6668740 1 -- 172.21.15.97:0/798980200 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5567f9ff8480 msgr2=0x5567f9ffa870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:31.096 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.095+0000 7fb0f6668740 1 --2- 172.21.15.97:0/798980200 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5567f9ff8480 0x5567f9ffa870 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fb0e80058c0 tx=0x7fb0e8005850 comp rx=0 tx=0).stop 2024-04-23T10:34:31.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.095+0000 7fb0f6668740 1 -- 172.21.15.97:0/798980200 shutdown_connections 2024-04-23T10:34:31.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.095+0000 7fb0f6668740 1 --2- 172.21.15.97:0/798980200 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5567f9ff8480 0x5567f9ffa870 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:31.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.095+0000 7fb0f6668740 1 --2- 172.21.15.97:0/798980200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5567f9feddb0 0x5567f9ff7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:31.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.095+0000 7fb0f6668740 1 --2- 172.21.15.97:0/798980200 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5567f9fed490 0x5567f9fed870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:31.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.096+0000 7fb0f6668740 1 -- 172.21.15.97:0/798980200 >> 172.21.15.97:0/798980200 conn(0x5567f9fe97b0 msgr2=0x5567f9fe9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:34:31.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.096+0000 7fb0f6668740 1 -- 172.21.15.97:0/798980200 shutdown_connections 2024-04-23T10:34:31.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.096+0000 7fb0f6668740 1 -- 172.21.15.97:0/798980200 wait complete. 2024-04-23T10:34:31.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.096+0000 7fb0f6668740 1 -- start start 2024-04-23T10:34:31.097 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.096+0000 7fb0f6668740 1 Processor -- start 2024-04-23T10:34:31.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.096+0000 7fb0f6668740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5567f9fed490 0x5567f9ff5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:31.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.096+0000 7fb0f6668740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5567f9feddb0 0x5567f9ff57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:31.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.096+0000 7fb0f6668740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5567f9ff8480 0x5567f9ff5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:31.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.096+0000 7fb0f6668740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5567f9ffb310 con 0x5567f9feddb0 2024-04-23T10:34:31.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.096+0000 7fb0f6668740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5567f9eecb20 con 0x5567f9ff8480 2024-04-23T10:34:31.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.096+0000 7fb0f6668740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5567f9ffb490 con 0x5567f9fed490 2024-04-23T10:34:31.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.096+0000 7fb0f6668740 10 client.0 _ll_register_callbacks cb 0x5567f9eec530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:34:31.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.096+0000 7fb0f16e0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5567f9feddb0 0x5567f9ff57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:31.101 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.097+0000 7fb0f16e0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5567f9feddb0 0x5567f9ff57a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38078/0 (socket says 172.21.15.97:38078) 2024-04-23T10:34:31.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.097+0000 7fb0f16e0640 1 -- 172.21.15.97:0/1964561327 learned_addr learned my addr 172.21.15.97:0/1964561327 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:34:31.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.097+0000 7fb0f0edf640 1 --2- 172.21.15.97:0/1964561327 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5567f9fed490 0x5567f9ff5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:31.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.097+0000 7fb0f0edf640 1 --2- 172.21.15.97:0/1964561327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5567f9ff8480 0x5567f9ff5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:31.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.097+0000 7fb0f16e0640 1 -- 172.21.15.97:0/1964561327 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5567f9fed490 msgr2=0x5567f9ff5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:34:31.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.097+0000 7fb0f16e0640 1 --2- 172.21.15.97:0/1964561327 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5567f9fed490 0x5567f9ff5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:31.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.097+0000 7fb0f16e0640 1 -- 172.21.15.97:0/1964561327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5567f9ff8480 msgr2=0x5567f9ff5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:34:31.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.097+0000 7fb0f16e0640 1 --2- 172.21.15.97:0/1964561327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5567f9ff8480 0x5567f9ff5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:31.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.097+0000 7fb0f16e0640 1 -- 172.21.15.97:0/1964561327 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567f9c64920 con 0x5567f9feddb0 2024-04-23T10:34:31.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.097+0000 7fb0f0edf640 1 --2- 172.21.15.97:0/1964561327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5567f9ff8480 0x5567f9ff5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:34:31.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.097+0000 7fb0f16e0640 1 --2- 172.21.15.97:0/1964561327 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5567f9feddb0 0x5567f9ff57a0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc009900 tx=0x7fb0dc004440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc88056f9a1cb0f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:34:31.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.097+0000 7fb0d4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:34:31.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.097+0000 7fb0d4ff9640 1 -- 172.21.15.97:0/1964561327 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc035040 con 0x5567f9feddb0 2024-04-23T10:34:31.102 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.097+0000 7fb0d4ff9640 1 -- 172.21.15.97:0/1964561327 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7fb0dc005510 con 0x5567f9feddb0 2024-04-23T10:34:31.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.097+0000 7fb0f6668740 1 -- 172.21.15.97:0/1964561327 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0e80047a0 con 0x5567f9feddb0 2024-04-23T10:34:31.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.097+0000 7fb0d4ff9640 1 -- 172.21.15.97:0/1964561327 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb0dc004770 con 0x5567f9feddb0 2024-04-23T10:34:31.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.097+0000 7fb0f6668740 10 client.16731 fetch_fsmap learned FSMap version 26 2024-04-23T10:34:31.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.097+0000 7fb0f6668740 1 -- 172.21.15.97:0/1964561327 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5567f9d30d30 con 0x5567f9feddb0 2024-04-23T10:34:31.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.097+0000 7fb0d4ff9640 1 -- 172.21.15.97:0/1964561327 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0e80047a0 con 0x5567f9feddb0 2024-04-23T10:34:31.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.098+0000 7fb0d4ff9640 1 -- 172.21.15.97:0/1964561327 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb0dc04d020 con 0x5567f9feddb0 2024-04-23T10:34:31.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.098+0000 7fb0f6668740 10 client.16731 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:34:31.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.098+0000 7fb0f6668740 10 client.16731 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:34:31.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.098+0000 7fb0f6668740 20 client.16731 populate_metadata read hostname 'smithi097' 2024-04-23T10:34:31.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.098+0000 7fb0f6668740 1 --2- 172.21.15.97:0/1964561327 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5567fa008640 0x5567fa00aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:31.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.098+0000 7fb0f6668740 4 client.16731 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:34:31.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.098+0000 7fb0f6668740 1 -- 172.21.15.97:0/1964561327 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5567f9f10500 con 0x5567fa008640 2024-04-23T10:34:31.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.098+0000 7fb0f0edf640 1 --2- 172.21.15.97:0/1964561327 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5567fa008640 0x5567fa00aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:31.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.098+0000 7fb0d4ff9640 10 client.16731 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:34:31.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.098+0000 7fb0f0edf640 1 --2- 172.21.15.97:0/1964561327 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5567fa008640 0x5567fa00aa60 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42cd563ae27c2787 server_cookie=670b2a888002677 in_seq=0 out_seq=0 2024-04-23T10:34:31.103 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.099+0000 7fb0d4ff9640 1 -- 172.21.15.97:0/1964561327 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5567f9f10500 con 0x5567fa008640 2024-04-23T10:34:31.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.099+0000 7fb0d4ff9640 10 client.16731 handle_command_reply: tid=0 2024-04-23T10:34:31.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.099+0000 7fb0f6668740 1 client.16731 shutdown 2024-04-23T10:34:31.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.099+0000 7fb0f6668740 1 -- 172.21.15.97:0/1964561327 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5567f9feddb0 msgr2=0x5567f9ff57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:31.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.099+0000 7fb0f6668740 1 --2- 172.21.15.97:0/1964561327 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5567f9feddb0 0x5567f9ff57a0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc009900 tx=0x7fb0dc004440 comp rx=0 tx=0).stop 2024-04-23T10:34:31.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.099+0000 7fb0f6668740 1 -- 172.21.15.97:0/1964561327 shutdown_connections 2024-04-23T10:34:31.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.099+0000 7fb0f6668740 1 --2- 172.21.15.97:0/1964561327 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5567fa008640 0x5567fa00aa60 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:31.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.099+0000 7fb0f6668740 1 --2- 172.21.15.97:0/1964561327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5567f9ff8480 0x5567f9ff5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:31.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.099+0000 7fb0d4ff9640 0 client.16731 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:34:31.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.099+0000 7fb0f6668740 1 --2- 172.21.15.97:0/1964561327 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5567f9feddb0 0x5567f9ff57a0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:31.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.099+0000 7fb0f6668740 1 --2- 172.21.15.97:0/1964561327 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5567f9fed490 0x5567f9ff5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:31.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.099+0000 7fb0f6668740 1 -- 172.21.15.97:0/1964561327 >> 172.21.15.97:0/1964561327 conn(0x5567f9fe97b0 msgr2=0x5567f9fef5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:34:31.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.099+0000 7fb0f6668740 1 -- 172.21.15.97:0/1964561327 shutdown_connections 2024-04-23T10:34:31.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.099+0000 7fb0f6668740 1 -- 172.21.15.97:0/1964561327 wait complete. 2024-04-23T10:34:31.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.100+0000 7fb0f6668740 20 client.16731 trim_cache size 0 max 16384 2024-04-23T10:34:31.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.117+0000 7fb0f6668740 1 Processor -- start 2024-04-23T10:34:31.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.117+0000 7fb0f6668740 1 -- start start 2024-04-23T10:34:31.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.117+0000 7fb0f6668740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5567f9fed490 0x5567f9fefc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:31.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.117+0000 7fb0f6668740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5567f9feddb0 0x5567f9ff5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:31.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.117+0000 7fb0f6668740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5567f9ff8480 0x5567fa010330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:31.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.117+0000 7fb0f6668740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ec1adb80 con 0x5567f9feddb0 2024-04-23T10:34:31.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.117+0000 7fb0f6668740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ec1af6c0 con 0x5567f9ff8480 2024-04-23T10:34:31.120 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.117+0000 7fb0f6668740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ec1ae250 con 0x5567f9fed490 2024-04-23T10:34:31.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.117+0000 7fb0f16e0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5567f9feddb0 0x5567f9ff5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:31.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.117+0000 7fb0f16e0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5567f9feddb0 0x5567f9ff5b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38090/0 (socket says 172.21.15.97:38090) 2024-04-23T10:34:31.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.117+0000 7fb0f16e0640 1 -- 172.21.15.97:0/2513629128 learned_addr learned my addr 172.21.15.97:0/2513629128 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:34:31.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.117+0000 7fb0f0edf640 1 --2- 172.21.15.97:0/2513629128 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5567f9ff8480 0x5567fa010330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:31.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.117+0000 7fb0f0edf640 1 --2- 172.21.15.97:0/2513629128 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5567f9fed490 0x5567f9fefc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:31.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.117+0000 7fb0f16e0640 1 -- 172.21.15.97:0/2513629128 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5567f9fed490 msgr2=0x5567f9fefc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:34:31.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.117+0000 7fb0f16e0640 1 --2- 172.21.15.97:0/2513629128 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5567f9fed490 0x5567f9fefc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:31.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.117+0000 7fb0f16e0640 1 -- 172.21.15.97:0/2513629128 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5567f9ff8480 msgr2=0x5567fa010330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:34:31.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.117+0000 7fb0f16e0640 1 --2- 172.21.15.97:0/2513629128 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5567f9ff8480 0x5567fa010330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:31.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.117+0000 7fb0f16e0640 1 -- 172.21.15.97:0/2513629128 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567f9fedbe0 con 0x5567f9feddb0 2024-04-23T10:34:31.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.117+0000 7fb0f0edf640 1 --2- 172.21.15.97:0/2513629128 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5567f9ff8480 0x5567fa010330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:34:31.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.118+0000 7fb0f16e0640 1 --2- 172.21.15.97:0/2513629128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5567f9feddb0 0x5567f9ff5b00 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc003c90 tx=0x7fb0dc02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afe73acf2bf1ae75 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:34:31.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.118+0000 7fb0d6ffd640 1 -- 172.21.15.97:0/2513629128 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc035070 con 0x5567f9feddb0 2024-04-23T10:34:31.121 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.118+0000 7fb0d6ffd640 1 -- 172.21.15.97:0/2513629128 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0dc041070 con 0x5567f9feddb0 2024-04-23T10:34:31.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.118+0000 7fb0d6ffd640 1 -- 172.21.15.97:0/2513629128 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc004b70 con 0x5567f9feddb0 2024-04-23T10:34:31.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.118+0000 7fb0f6668740 1 -- 172.21.15.97:0/2513629128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5567f9feddb0 msgr2=0x5567f9ff5b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:31.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.118+0000 7fb0f6668740 1 --2- 172.21.15.97:0/2513629128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5567f9feddb0 0x5567f9ff5b00 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc003c90 tx=0x7fb0dc02a410 comp rx=0 tx=0).stop 2024-04-23T10:34:31.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.119+0000 7fb0f6668740 1 -- 172.21.15.97:0/2513629128 shutdown_connections 2024-04-23T10:34:31.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.119+0000 7fb0f6668740 1 --2- 172.21.15.97:0/2513629128 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5567f9ff8480 0x5567fa010330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:31.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.119+0000 7fb0f6668740 1 --2- 172.21.15.97:0/2513629128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5567f9feddb0 0x5567f9ff5b00 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:31.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.119+0000 7fb0f6668740 1 --2- 172.21.15.97:0/2513629128 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5567f9fed490 0x5567f9fefc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:31.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.119+0000 7fb0f6668740 1 -- 172.21.15.97:0/2513629128 >> 172.21.15.97:0/2513629128 conn(0x5567f9fe97b0 msgr2=0x5567fa0031c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:34:31.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.119+0000 7fb0f6668740 1 -- 172.21.15.97:0/2513629128 shutdown_connections 2024-04-23T10:34:31.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.119+0000 7fb0f6668740 1 -- 172.21.15.97:0/2513629128 wait complete. 2024-04-23T10:34:31.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.119+0000 7fb0f6668740 1 -- start start 2024-04-23T10:34:31.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.119+0000 7fb0f6668740 1 Processor -- start 2024-04-23T10:34:31.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.119+0000 7fb0f6668740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5567f9fed490 0x5567fa0006f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:31.122 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.119+0000 7fb0f6668740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5567f9feddb0 0x5567fa000c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:31.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.119+0000 7fb0f6668740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5567f9ff8480 0x5567f9ffb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:31.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.119+0000 7fb0f6668740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ec1aeff0 con 0x5567f9feddb0 2024-04-23T10:34:31.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.119+0000 7fb0f6668740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ec1ae920 con 0x5567f9ff8480 2024-04-23T10:34:31.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.119+0000 7fb0f6668740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ec1abb80 con 0x5567f9fed490 2024-04-23T10:34:31.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.119+0000 7fb0f6668740 10 client.0 _ll_register_callbacks cb 0x7fb0dc0040d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:34:31.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.119+0000 7fb0f16e0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5567f9feddb0 0x5567fa000c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:31.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.119+0000 7fb0f16e0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5567f9feddb0 0x5567fa000c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38104/0 (socket says 172.21.15.97:38104) 2024-04-23T10:34:31.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.119+0000 7fb0f16e0640 1 -- 172.21.15.97:0/2402688745 learned_addr learned my addr 172.21.15.97:0/2402688745 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:34:31.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.120+0000 7fb0f0edf640 1 --2- 172.21.15.97:0/2402688745 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5567f9fed490 0x5567fa0006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:31.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.120+0000 7fb0f0edf640 1 --2- 172.21.15.97:0/2402688745 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5567f9ff8480 0x5567f9ffb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:31.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.120+0000 7fb0f16e0640 1 -- 172.21.15.97:0/2402688745 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5567f9fed490 msgr2=0x5567fa0006f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:34:31.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.120+0000 7fb0f16e0640 1 --2- 172.21.15.97:0/2402688745 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5567f9fed490 0x5567fa0006f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:31.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.120+0000 7fb0f16e0640 1 -- 172.21.15.97:0/2402688745 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5567f9ff8480 msgr2=0x5567f9ffb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:34:31.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.120+0000 7fb0f16e0640 1 --2- 172.21.15.97:0/2402688745 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5567f9ff8480 0x5567f9ffb6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:31.123 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.120+0000 7fb0f16e0640 1 -- 172.21.15.97:0/2402688745 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567fa00add0 con 0x5567f9feddb0 2024-04-23T10:34:31.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.120+0000 7fb0f16e0640 1 --2- 172.21.15.97:0/2402688745 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5567f9feddb0 0x5567fa000c30 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc003b20 tx=0x7fb0dc031540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3033a3345b71bf6b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:34:31.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.120+0000 7fb0d4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:34:31.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.120+0000 7fb0d4ff9640 1 -- 172.21.15.97:0/2402688745 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc035040 con 0x5567f9feddb0 2024-04-23T10:34:31.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.120+0000 7fb0d4ff9640 1 -- 172.21.15.97:0/2402688745 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7fb0dc003e40 con 0x5567f9feddb0 2024-04-23T10:34:31.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.120+0000 7fb0f6668740 1 -- 172.21.15.97:0/2402688745 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0dc041070 con 0x5567f9feddb0 2024-04-23T10:34:31.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.120+0000 7fb0d4ff9640 1 -- 172.21.15.97:0/2402688745 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb0dc005520 con 0x5567f9feddb0 2024-04-23T10:34:31.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.120+0000 7fb0f6668740 10 client.16737 fetch_fsmap learned FSMap version 26 2024-04-23T10:34:31.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.120+0000 7fb0f6668740 1 -- 172.21.15.97:0/2402688745 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5567fa06f3c0 con 0x5567f9feddb0 2024-04-23T10:34:31.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.120+0000 7fb0d4ff9640 1 -- 172.21.15.97:0/2402688745 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0dc041070 con 0x5567f9feddb0 2024-04-23T10:34:31.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.120+0000 7fb0d4ff9640 1 -- 172.21.15.97:0/2402688745 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb0dc04d040 con 0x5567f9feddb0 2024-04-23T10:34:31.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.120+0000 7fb0f6668740 10 client.16737 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:34:31.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.120+0000 7fb0f6668740 10 client.16737 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:34:31.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.120+0000 7fb0f6668740 20 client.16737 populate_metadata read hostname 'smithi097' 2024-04-23T10:34:31.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.120+0000 7fb0f6668740 1 --2- 172.21.15.97:0/2402688745 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5567fa008640 0x5567fa0719d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:31.124 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.120+0000 7fb0f6668740 4 client.16737 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "b6473fa6", "release": true, "await": true}] 2024-04-23T10:34:31.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.120+0000 7fb0f6668740 1 -- 172.21.15.97:0/2402688745 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "quiesce db", "set_id": "b6473fa6", "release": true, "await": true}) -- 0x5567f9f600f0 con 0x5567fa008640 2024-04-23T10:34:31.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.121+0000 7fb0f0edf640 1 --2- 172.21.15.97:0/2402688745 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5567fa008640 0x5567fa0719d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:31.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.121+0000 7fb0d4ff9640 10 client.16737 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:34:31.125 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.121+0000 7fb0f0edf640 1 --2- 172.21.15.97:0/2402688745 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5567fa008640 0x5567fa0719d0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fde662e7c93925a3 server_cookie=93dc11145a5cd0b in_seq=0 out_seq=0 2024-04-23T10:34:31.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.124+0000 7fb0d4ff9640 1 -- 172.21.15.97:0/2402688745 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5567f9f600f0 con 0x5567fa008640 2024-04-23T10:34:31.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.124+0000 7fb0d4ff9640 10 client.16737 handle_command_reply: tid=0 2024-04-23T10:34:31.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.124+0000 7fb0f6668740 1 client.16737 shutdown 2024-04-23T10:34:31.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.124+0000 7fb0f6668740 1 -- 172.21.15.97:0/2402688745 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5567f9feddb0 msgr2=0x5567fa000c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:31.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.124+0000 7fb0f6668740 1 --2- 172.21.15.97:0/2402688745 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5567f9feddb0 0x5567fa000c30 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc003b20 tx=0x7fb0dc031540 comp rx=0 tx=0).stop 2024-04-23T10:34:31.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.124+0000 7fb0f6668740 1 -- 172.21.15.97:0/2402688745 shutdown_connections 2024-04-23T10:34:31.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.124+0000 7fb0f6668740 1 --2- 172.21.15.97:0/2402688745 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5567fa008640 0x5567fa0719d0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:31.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.124+0000 7fb0f6668740 1 --2- 172.21.15.97:0/2402688745 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5567f9ff8480 0x5567f9ffb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:31.126 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.124+0000 7fb0f6668740 1 --2- 172.21.15.97:0/2402688745 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5567f9feddb0 0x5567fa000c30 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:31.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.124+0000 7fb0d4ff9640 0 client.16737 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:34:31.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.124+0000 7fb0f6668740 1 --2- 172.21.15.97:0/2402688745 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5567f9fed490 0x5567fa0006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:31.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.124+0000 7fb0f6668740 1 -- 172.21.15.97:0/2402688745 >> 172.21.15.97:0/2402688745 conn(0x5567f9fe97b0 msgr2=0x5567f9fef470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:34:31.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.124+0000 7fb0f6668740 1 -- 172.21.15.97:0/2402688745 shutdown_connections 2024-04-23T10:34:31.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.124+0000 7fb0f6668740 1 -- 172.21.15.97:0/2402688745 wait complete. 2024-04-23T10:34:31.127 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.124+0000 7fb0f6668740 20 client.16737 trim_cache size 0 max 16384 2024-04-23T10:34:31.127 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-23T10:34:31.127 INFO:teuthology.orchestra.run.smithi097.stdout: "epoch": 26, 2024-04-23T10:34:31.127 INFO:teuthology.orchestra.run.smithi097.stdout: "leader": 24488, 2024-04-23T10:34:31.127 INFO:teuthology.orchestra.run.smithi097.stdout: "set_version": 32, 2024-04-23T10:34:31.127 INFO:teuthology.orchestra.run.smithi097.stdout: "sets": { 2024-04-23T10:34:31.128 INFO:teuthology.orchestra.run.smithi097.stdout: "b6473fa6": { 2024-04-23T10:34:31.128 INFO:teuthology.orchestra.run.smithi097.stdout: "version": 32, 2024-04-23T10:34:31.128 INFO:teuthology.orchestra.run.smithi097.stdout: "age_ref": 0.0, 2024-04-23T10:34:31.128 INFO:teuthology.orchestra.run.smithi097.stdout: "state": { 2024-04-23T10:34:31.128 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RELEASED", 2024-04-23T10:34:31.128 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0 2024-04-23T10:34:31.128 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:34:31.128 INFO:teuthology.orchestra.run.smithi097.stdout: "timeout": 60.0, 2024-04-23T10:34:31.128 INFO:teuthology.orchestra.run.smithi097.stdout: "expiration": 70.0, 2024-04-23T10:34:31.128 INFO:teuthology.orchestra.run.smithi097.stdout: "members": { 2024-04-23T10:34:31.128 INFO:teuthology.orchestra.run.smithi097.stdout: "file:/": { 2024-04-23T10:34:31.128 INFO:teuthology.orchestra.run.smithi097.stdout: "excluded": false, 2024-04-23T10:34:31.128 INFO:teuthology.orchestra.run.smithi097.stdout: "state": { 2024-04-23T10:34:31.128 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RELEASED", 2024-04-23T10:34:31.129 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0 2024-04-23T10:34:31.129 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:34:31.129 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:34:31.129 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:34:31.129 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:34:31.129 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:34:31.129 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-23T10:34:31.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.125+0000 7fb0f3944640 1 -- 172.21.15.97:0/2875322344 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fb0d80a3d20 msgr2=0x7fb0d80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:31.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.125+0000 7fb0f3944640 1 --2- 172.21.15.97:0/2875322344 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fb0d80a3d20 0x7fb0d80a61e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb0e40ce500 tx=0x7fb0e406e6e0 comp rx=0 tx=0).stop 2024-04-23T10:34:31.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.125+0000 7fb0f3944640 1 -- 172.21.15.97:0/2875322344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0ec198590 msgr2=0x7fb0ec198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:31.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.125+0000 7fb0f3944640 1 --2- 172.21.15.97:0/2875322344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0ec198590 0x7fb0ec198a40 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fb0e400eb10 tx=0x7fb0e400c2d0 comp rx=0 tx=0).stop 2024-04-23T10:34:31.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.125+0000 7fb0f3944640 1 -- 172.21.15.97:0/2875322344 shutdown_connections 2024-04-23T10:34:31.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.125+0000 7fb0f3944640 1 --2- 172.21.15.97:0/2875322344 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fb0d80a3d20 0x7fb0d80a61e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:31.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.125+0000 7fb0f3944640 1 --2- 172.21.15.97:0/2875322344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0ec198590 0x7fb0ec198a40 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:31.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.125+0000 7fb0f3944640 1 --2- 172.21.15.97:0/2875322344 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0ec107260 0x7fb0ec19e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:31.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.125+0000 7fb0f3944640 1 --2- 172.21.15.97:0/2875322344 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0ec102190 0x7fb0ec19e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:31.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.125+0000 7fb0f3944640 1 -- 172.21.15.97:0/2875322344 >> 172.21.15.97:0/2875322344 conn(0x7fb0ec0793a0 msgr2=0x7fb0ec0fff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:34:31.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.125+0000 7fb0f3944640 1 -- 172.21.15.97:0/2875322344 shutdown_connections 2024-04-23T10:34:31.130 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:31.125+0000 7fb0f3944640 1 -- 172.21.15.97:0/2875322344 wait complete. 2024-04-23T10:34:31.134 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'b6473fa6', seconds elapsed: 11.0 2024-04-23T10:34:31.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:34:31 smithi097 ceph-mon[29197]: pgmap v256: 97 pgs: 97 active+clean; 3.1 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 12 op/s 2024-04-23T10:34:31.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:34:31 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/3645590900' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:34:31.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:34:31 smithi183 ceph-mon[32317]: pgmap v256: 97 pgs: 97 active+clean; 3.1 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 12 op/s 2024-04-23T10:34:31.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:34:31 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/3645590900' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:34:31.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:34:31 smithi196 ceph-mon[32919]: pgmap v256: 97 pgs: 97 active+clean; 3.1 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 12 op/s 2024-04-23T10:34:31.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:34:31 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/3645590900' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:34:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.121+0000 7fc803d6f640 1 -- 172.21.15.97:0/706639309 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7fc106170 msgr2=0x7fc7fc10dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.121+0000 7fc803d6f640 1 --2- 172.21.15.97:0/706639309 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7fc106170 0x7fc7fc10dd40 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fc7f4009ca0 tx=0x7fc7f402a8c0 comp rx=0 tx=0).stop 2024-04-23T10:34:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.121+0000 7fc803d6f640 1 -- 172.21.15.97:0/706639309 shutdown_connections 2024-04-23T10:34:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.121+0000 7fc803d6f640 1 --2- 172.21.15.97:0/706639309 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7fc106170 0x7fc7fc10dd40 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.121+0000 7fc803d6f640 1 --2- 172.21.15.97:0/706639309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7fc101aa0 0x7fc7fc105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.121+0000 7fc803d6f640 1 --2- 172.21.15.97:0/706639309 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7fc1010f0 0x7fc7fc1014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.121+0000 7fc803d6f640 1 -- 172.21.15.97:0/706639309 >> 172.21.15.97:0/706639309 conn(0x7fc7fc077fd0 msgr2=0x7fc7fc0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:34:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.121+0000 7fc803d6f640 1 -- 172.21.15.97:0/706639309 shutdown_connections 2024-04-23T10:34:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.121+0000 7fc803d6f640 1 -- 172.21.15.97:0/706639309 wait complete. 2024-04-23T10:34:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.122+0000 7fc803d6f640 1 Processor -- start 2024-04-23T10:34:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.122+0000 7fc803d6f640 1 -- start start 2024-04-23T10:34:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.122+0000 7fc803d6f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7fc1010f0 0x7fc7fc1986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.122+0000 7fc803d6f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7fc106170 0x7fc7fc198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.122+0000 7fc803d6f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7fc19cfa0 0x7fc7fc19d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.122+0000 7fc803d6f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc7fc112220 con 0x7fc7fc19cfa0 2024-04-23T10:34:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.122+0000 7fc803d6f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc7fc1120a0 con 0x7fc7fc106170 2024-04-23T10:34:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.122+0000 7fc803d6f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc7fc1123a0 con 0x7fc7fc1010f0 2024-04-23T10:34:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.122+0000 7fc80230c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7fc19cfa0 0x7fc7fc19d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.122+0000 7fc80230c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7fc19cfa0 0x7fc7fc19d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38120/0 (socket says 172.21.15.97:38120) 2024-04-23T10:34:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.123+0000 7fc80230c640 1 -- 172.21.15.97:0/2521459632 learned_addr learned my addr 172.21.15.97:0/2521459632 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:34:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.123+0000 7fc80130a640 1 --2- 172.21.15.97:0/2521459632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7fc106170 0x7fc7fc198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.123+0000 7fc801b0b640 1 --2- 172.21.15.97:0/2521459632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7fc1010f0 0x7fc7fc1986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.123+0000 7fc80230c640 1 -- 172.21.15.97:0/2521459632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7fc1010f0 msgr2=0x7fc7fc1986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.123+0000 7fc80230c640 1 --2- 172.21.15.97:0/2521459632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7fc1010f0 0x7fc7fc1986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.123+0000 7fc80230c640 1 -- 172.21.15.97:0/2521459632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7fc106170 msgr2=0x7fc7fc198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.123+0000 7fc80230c640 1 --2- 172.21.15.97:0/2521459632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7fc106170 0x7fc7fc198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.123+0000 7fc80230c640 1 -- 172.21.15.97:0/2521459632 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7fc19da90 con 0x7fc7fc19cfa0 2024-04-23T10:34:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.123+0000 7fc80230c640 1 --2- 172.21.15.97:0/2521459632 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7fc19cfa0 0x7fc7fc19d450 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fc7f402f540 tx=0x7fc7f40316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:34:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.123+0000 7fc7f2ffd640 1 -- 172.21.15.97:0/2521459632 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f4003b70 con 0x7fc7fc19cfa0 2024-04-23T10:34:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.123+0000 7fc7f2ffd640 1 -- 172.21.15.97:0/2521459632 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7f4003d10 con 0x7fc7fc19cfa0 2024-04-23T10:34:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.123+0000 7fc803d6f640 1 -- 172.21.15.97:0/2521459632 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7fc19dd20 con 0x7fc7fc19cfa0 2024-04-23T10:34:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.123+0000 7fc7f2ffd640 1 -- 172.21.15.97:0/2521459632 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f402f990 con 0x7fc7fc19cfa0 2024-04-23T10:34:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.123+0000 7fc803d6f640 1 -- 172.21.15.97:0/2521459632 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7fc1a5570 con 0x7fc7fc19cfa0 2024-04-23T10:34:33.130 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.125+0000 7fc7f2ffd640 1 -- 172.21.15.97:0/2521459632 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc7f402fb30 con 0x7fc7fc19cfa0 2024-04-23T10:34:33.130 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.126+0000 7fc806b67740 1 Processor -- start 2024-04-23T10:34:33.130 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.126+0000 7fc806b67740 1 -- start start 2024-04-23T10:34:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.126+0000 7fc7f2ffd640 1 --2- 172.21.15.97:0/2521459632 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fc7e80a3e40 0x7fc7e80a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.126+0000 7fc806b67740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556aaf8b3500 0x556aaf8b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.126+0000 7fc806b67740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556aaf8b3e20 0x556aaf8c26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.126+0000 7fc806b67740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556aaf8c2c10 0x556aaf8c5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.126+0000 7fc806b67740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x556aaf818970 con 0x556aaf8b3e20 2024-04-23T10:34:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.126+0000 7fc806b67740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556aaf7b54a0 con 0x556aaf8b3500 2024-04-23T10:34:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.126+0000 7fc806b67740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556aaf814930 con 0x556aaf8c2c10 2024-04-23T10:34:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.126+0000 7fc80130a640 1 --2- 172.21.15.97:0/2521459632 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fc7e80a3e40 0x7fc7e80a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.127+0000 7fc7f2ffd640 1 -- 172.21.15.97:0/2521459632 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7fc7f40f19d0 con 0x7fc7fc19cfa0 2024-04-23T10:34:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.127+0000 7fc80130a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556aaf8b3e20 0x556aaf8c26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.127+0000 7fc80130a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556aaf8b3e20 0x556aaf8c26d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38126/0 (socket says 172.21.15.97:38126) 2024-04-23T10:34:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.127+0000 7fc80130a640 1 -- 172.21.15.97:0/4280591943 learned_addr learned my addr 172.21.15.97:0/4280591943 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:34:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.127+0000 7fc801b0b640 1 --2- 172.21.15.97:0/4280591943 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556aaf8b3500 0x556aaf8b38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.127+0000 7fc80230c640 1 --2- 172.21.15.97:0/4280591943 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556aaf8c2c10 0x556aaf8c5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.127+0000 7fc801b0b640 1 -- 172.21.15.97:0/4280591943 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556aaf8c2c10 msgr2=0x556aaf8c5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:34:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.127+0000 7fc801b0b640 1 --2- 172.21.15.97:0/4280591943 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556aaf8c2c10 0x556aaf8c5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.127+0000 7fc801b0b640 1 -- 172.21.15.97:0/4280591943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556aaf8b3e20 msgr2=0x556aaf8c26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:34:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.127+0000 7fc801b0b640 1 --2- 172.21.15.97:0/4280591943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556aaf8b3e20 0x556aaf8c26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.127+0000 7fc801b0b640 1 -- 172.21.15.97:0/4280591943 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556aaf52b210 con 0x556aaf8b3500 2024-04-23T10:34:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.127+0000 7fc80130a640 1 --2- 172.21.15.97:0/4280591943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556aaf8b3e20 0x556aaf8c26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T10:34:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.127+0000 7fc80130a640 1 --2- 172.21.15.97:0/2521459632 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fc7e80a3e40 0x7fc7e80a6300 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec0118f0 tx=0x7fc7ec009b20 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:34:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.127+0000 7fc801b0b640 1 --2- 172.21.15.97:0/4280591943 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556aaf8b3500 0x556aaf8b38e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc7f8009c50 tx=0x7fc7f8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7fae6a8428ee6304 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:34:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.127+0000 7fc7e7fff640 1 -- 172.21.15.97:0/4280591943 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f8004050 con 0x556aaf8b3500 2024-04-23T10:34:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.127+0000 7fc7e7fff640 1 -- 172.21.15.97:0/4280591943 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7f80054d0 con 0x556aaf8b3500 2024-04-23T10:34:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.127+0000 7fc7e7fff640 1 -- 172.21.15.97:0/4280591943 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f8004c60 con 0x556aaf8b3500 2024-04-23T10:34:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.128+0000 7fc806b67740 1 -- 172.21.15.97:0/4280591943 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556aaf8b3500 msgr2=0x556aaf8b38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.128+0000 7fc806b67740 1 --2- 172.21.15.97:0/4280591943 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556aaf8b3500 0x556aaf8b38e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc7f8009c50 tx=0x7fc7f8011810 comp rx=0 tx=0).stop 2024-04-23T10:34:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.128+0000 7fc806b67740 1 -- 172.21.15.97:0/4280591943 shutdown_connections 2024-04-23T10:34:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.128+0000 7fc806b67740 1 --2- 172.21.15.97:0/4280591943 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556aaf8c2c10 0x556aaf8c5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.128+0000 7fc806b67740 1 --2- 172.21.15.97:0/4280591943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556aaf8b3e20 0x556aaf8c26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.128+0000 7fc806b67740 1 --2- 172.21.15.97:0/4280591943 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556aaf8b3500 0x556aaf8b38e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.128+0000 7fc806b67740 1 -- 172.21.15.97:0/4280591943 >> 172.21.15.97:0/4280591943 conn(0x556aaf8afb30 msgr2=0x556aaf8aff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:34:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.128+0000 7fc806b67740 1 -- 172.21.15.97:0/4280591943 shutdown_connections 2024-04-23T10:34:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.128+0000 7fc806b67740 1 -- 172.21.15.97:0/4280591943 wait complete. 2024-04-23T10:34:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.129+0000 7fc806b67740 1 -- start start 2024-04-23T10:34:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.129+0000 7fc806b67740 1 Processor -- start 2024-04-23T10:34:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.129+0000 7fc806b67740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556aaf8b3500 0x556aaf8c2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.129+0000 7fc806b67740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556aaf8b3e20 0x556aaf8bf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.129+0000 7fc806b67740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556aaf8c2c10 0x556aaf8bfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.129+0000 7fc806b67740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x556aaf8c5840 con 0x556aaf8b3500 2024-04-23T10:34:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.129+0000 7fc806b67740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556aaf7b30b0 con 0x556aaf8c2c10 2024-04-23T10:34:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.129+0000 7fc806b67740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556aaf8c59c0 con 0x556aaf8b3e20 2024-04-23T10:34:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.129+0000 7fc806b67740 10 client.0 _ll_register_callbacks cb 0x556aaf7a0d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:34:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.129+0000 7fc801b0b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556aaf8b3500 0x556aaf8c2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.129+0000 7fc801b0b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556aaf8b3500 0x556aaf8c2470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38140/0 (socket says 172.21.15.97:38140) 2024-04-23T10:34:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.129+0000 7fc801b0b640 1 -- 172.21.15.97:0/4170195225 learned_addr learned my addr 172.21.15.97:0/4170195225 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:34:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.129+0000 7fc80130a640 1 --2- 172.21.15.97:0/4170195225 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556aaf8b3e20 0x556aaf8bf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.129+0000 7fc80230c640 1 --2- 172.21.15.97:0/4170195225 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556aaf8c2c10 0x556aaf8bfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.130+0000 7fc801b0b640 1 -- 172.21.15.97:0/4170195225 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556aaf8b3e20 msgr2=0x556aaf8bf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:34:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.130+0000 7fc801b0b640 1 --2- 172.21.15.97:0/4170195225 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556aaf8b3e20 0x556aaf8bf760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.130+0000 7fc801b0b640 1 -- 172.21.15.97:0/4170195225 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556aaf8c2c10 msgr2=0x556aaf8bfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:34:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.130+0000 7fc801b0b640 1 --2- 172.21.15.97:0/4170195225 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556aaf8c2c10 0x556aaf8bfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.130+0000 7fc801b0b640 1 -- 172.21.15.97:0/4170195225 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556aaf5f7620 con 0x556aaf8b3500 2024-04-23T10:34:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.130+0000 7fc801b0b640 1 --2- 172.21.15.97:0/4170195225 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556aaf8b3500 0x556aaf8c2470 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fc7f8005df0 tx=0x7fc7f80044f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26192d2e656643ac server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:34:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.130+0000 7fc7e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:34:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.130+0000 7fc7e57fa640 1 -- 172.21.15.97:0/4170195225 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f8027070 con 0x556aaf8b3500 2024-04-23T10:34:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.130+0000 7fc7e57fa640 1 -- 172.21.15.97:0/4170195225 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7fc7f8003a00 con 0x556aaf8b3500 2024-04-23T10:34:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.130+0000 7fc806b67740 1 -- 172.21.15.97:0/4170195225 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7f80054d0 con 0x556aaf8b3500 2024-04-23T10:34:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.130+0000 7fc7e57fa640 1 -- 172.21.15.97:0/4170195225 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc7f8033050 con 0x556aaf8b3500 2024-04-23T10:34:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.130+0000 7fc806b67740 10 client.16749 fetch_fsmap learned FSMap version 26 2024-04-23T10:34:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.130+0000 7fc806b67740 1 -- 172.21.15.97:0/4170195225 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556aaf8d1760 con 0x556aaf8b3500 2024-04-23T10:34:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.130+0000 7fc7e57fa640 1 -- 172.21.15.97:0/4170195225 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7f80054d0 con 0x556aaf8b3500 2024-04-23T10:34:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.131+0000 7fc7e57fa640 1 -- 172.21.15.97:0/4170195225 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc7f80050f0 con 0x556aaf8b3500 2024-04-23T10:34:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.131+0000 7fc806b67740 10 client.16749 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:34:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.131+0000 7fc806b67740 10 client.16749 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:34:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.131+0000 7fc806b67740 20 client.16749 populate_metadata read hostname 'smithi097' 2024-04-23T10:34:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.131+0000 7fc806b67740 1 --2- 172.21.15.97:0/4170195225 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x556aaf8d2bd0 0x556aaf8d4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.131+0000 7fc806b67740 4 client.16749 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:34:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.131+0000 7fc806b67740 1 -- 172.21.15.97:0/4170195225 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556aaf826390 con 0x556aaf8d2bd0 2024-04-23T10:34:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.131+0000 7fc80130a640 1 --2- 172.21.15.97:0/4170195225 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x556aaf8d2bd0 0x556aaf8d4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.131+0000 7fc7e57fa640 10 client.16749 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:34:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.131+0000 7fc80130a640 1 --2- 172.21.15.97:0/4170195225 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x556aaf8d2bd0 0x556aaf8d4fc0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7b20c03b5a04ec6 server_cookie=ed8e7d60155c238f in_seq=0 out_seq=0 2024-04-23T10:34:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.132+0000 7fc7e57fa640 1 -- 172.21.15.97:0/4170195225 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556aaf826390 con 0x556aaf8d2bd0 2024-04-23T10:34:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.132+0000 7fc7e57fa640 10 client.16749 handle_command_reply: tid=0 2024-04-23T10:34:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.132+0000 7fc806b67740 1 client.16749 shutdown 2024-04-23T10:34:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.132+0000 7fc806b67740 1 -- 172.21.15.97:0/4170195225 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556aaf8b3500 msgr2=0x556aaf8c2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.132+0000 7fc806b67740 1 --2- 172.21.15.97:0/4170195225 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556aaf8b3500 0x556aaf8c2470 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fc7f8005df0 tx=0x7fc7f80044f0 comp rx=0 tx=0).stop 2024-04-23T10:34:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.132+0000 7fc806b67740 1 -- 172.21.15.97:0/4170195225 shutdown_connections 2024-04-23T10:34:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.132+0000 7fc806b67740 1 --2- 172.21.15.97:0/4170195225 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x556aaf8d2bd0 0x556aaf8d4fc0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.132+0000 7fc806b67740 1 --2- 172.21.15.97:0/4170195225 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556aaf8c2c10 0x556aaf8bfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.132+0000 7fc7e57fa640 0 client.16749 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:34:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.132+0000 7fc806b67740 1 --2- 172.21.15.97:0/4170195225 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556aaf8b3e20 0x556aaf8bf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.132+0000 7fc806b67740 1 --2- 172.21.15.97:0/4170195225 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556aaf8b3500 0x556aaf8c2470 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.132+0000 7fc806b67740 1 -- 172.21.15.97:0/4170195225 >> 172.21.15.97:0/4170195225 conn(0x556aaf8afb30 msgr2=0x556aaf8b5940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:34:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.132+0000 7fc806b67740 1 -- 172.21.15.97:0/4170195225 shutdown_connections 2024-04-23T10:34:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.133+0000 7fc806b67740 1 -- 172.21.15.97:0/4170195225 wait complete. 2024-04-23T10:34:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.133+0000 7fc806b67740 20 client.16749 trim_cache size 0 max 16384 2024-04-23T10:34:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.148+0000 7fc806b67740 1 Processor -- start 2024-04-23T10:34:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.148+0000 7fc806b67740 1 -- start start 2024-04-23T10:34:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.148+0000 7fc806b67740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556aaf8b3500 0x556aaf8ce770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.148+0000 7fc806b67740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556aaf8b3e20 0x556aaf8bad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.148+0000 7fc806b67740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556aaf8c2c10 0x556aaf8c1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.148+0000 7fc806b67740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc7fc1ab880 con 0x556aaf8b3500 2024-04-23T10:34:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.148+0000 7fc806b67740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc7fc1ad3c0 con 0x556aaf8c2c10 2024-04-23T10:34:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.148+0000 7fc806b67740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc7fc1abf50 con 0x556aaf8b3e20 2024-04-23T10:34:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.148+0000 7fc801b0b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556aaf8b3500 0x556aaf8ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.148+0000 7fc801b0b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556aaf8b3500 0x556aaf8ce770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38150/0 (socket says 172.21.15.97:38150) 2024-04-23T10:34:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.148+0000 7fc801b0b640 1 -- 172.21.15.97:0/1197160947 learned_addr learned my addr 172.21.15.97:0/1197160947 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:34:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.148+0000 7fc80130a640 1 --2- 172.21.15.97:0/1197160947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556aaf8b3e20 0x556aaf8bad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.148+0000 7fc80230c640 1 --2- 172.21.15.97:0/1197160947 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556aaf8c2c10 0x556aaf8c1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.148+0000 7fc801b0b640 1 -- 172.21.15.97:0/1197160947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556aaf8b3e20 msgr2=0x556aaf8bad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:34:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.148+0000 7fc801b0b640 1 --2- 172.21.15.97:0/1197160947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556aaf8b3e20 0x556aaf8bad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.148+0000 7fc801b0b640 1 -- 172.21.15.97:0/1197160947 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556aaf8c2c10 msgr2=0x556aaf8c1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:34:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.148+0000 7fc801b0b640 1 --2- 172.21.15.97:0/1197160947 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556aaf8c2c10 0x556aaf8c1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.148+0000 7fc801b0b640 1 -- 172.21.15.97:0/1197160947 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556aaf8af960 con 0x556aaf8b3500 2024-04-23T10:34:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.148+0000 7fc801b0b640 1 --2- 172.21.15.97:0/1197160947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556aaf8b3500 0x556aaf8ce770 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fc7f8009710 tx=0x7fc7f80236a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f96dee375ac92f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:34:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.148+0000 7fc7e77fe640 1 -- 172.21.15.97:0/1197160947 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f8027070 con 0x556aaf8b3500 2024-04-23T10:34:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.149+0000 7fc7e77fe640 1 -- 172.21.15.97:0/1197160947 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7f8023740 con 0x556aaf8b3500 2024-04-23T10:34:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.149+0000 7fc7e77fe640 1 -- 172.21.15.97:0/1197160947 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f8003d70 con 0x556aaf8b3500 2024-04-23T10:34:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.149+0000 7fc806b67740 1 -- 172.21.15.97:0/1197160947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556aaf8b3500 msgr2=0x556aaf8ce770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.149+0000 7fc806b67740 1 --2- 172.21.15.97:0/1197160947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556aaf8b3500 0x556aaf8ce770 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fc7f8009710 tx=0x7fc7f80236a0 comp rx=0 tx=0).stop 2024-04-23T10:34:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.149+0000 7fc806b67740 1 -- 172.21.15.97:0/1197160947 shutdown_connections 2024-04-23T10:34:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.149+0000 7fc806b67740 1 --2- 172.21.15.97:0/1197160947 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556aaf8c2c10 0x556aaf8c1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.149+0000 7fc806b67740 1 --2- 172.21.15.97:0/1197160947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556aaf8b3e20 0x556aaf8bad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.149+0000 7fc806b67740 1 --2- 172.21.15.97:0/1197160947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556aaf8b3500 0x556aaf8ce770 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.149+0000 7fc806b67740 1 -- 172.21.15.97:0/1197160947 >> 172.21.15.97:0/1197160947 conn(0x556aaf8afb30 msgr2=0x556aaf8ba670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:34:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.149+0000 7fc806b67740 1 -- 172.21.15.97:0/1197160947 shutdown_connections 2024-04-23T10:34:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.149+0000 7fc806b67740 1 -- 172.21.15.97:0/1197160947 wait complete. 2024-04-23T10:34:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.150+0000 7fc806b67740 1 -- start start 2024-04-23T10:34:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.150+0000 7fc806b67740 1 Processor -- start 2024-04-23T10:34:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.150+0000 7fc806b67740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556aaf8b3500 0x556aaf8c6280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.150+0000 7fc806b67740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556aaf8b3e20 0x556aaf8c67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.150+0000 7fc806b67740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556aaf8c2c10 0x556aaf8ca680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.150+0000 7fc806b67740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc7fc1accf0 con 0x556aaf8b3500 2024-04-23T10:34:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.150+0000 7fc806b67740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc7fc1ac620 con 0x556aaf8c2c10 2024-04-23T10:34:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.150+0000 7fc806b67740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc7fc1a9880 con 0x556aaf8b3e20 2024-04-23T10:34:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.150+0000 7fc806b67740 10 client.0 _ll_register_callbacks cb 0x7fc7f8009a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:34:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.150+0000 7fc801b0b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556aaf8b3500 0x556aaf8c6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.150+0000 7fc801b0b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556aaf8b3500 0x556aaf8c6280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38162/0 (socket says 172.21.15.97:38162) 2024-04-23T10:34:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.150+0000 7fc801b0b640 1 -- 172.21.15.97:0/1167402534 learned_addr learned my addr 172.21.15.97:0/1167402534 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:34:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.150+0000 7fc80130a640 1 --2- 172.21.15.97:0/1167402534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556aaf8b3e20 0x556aaf8c67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.150+0000 7fc80230c640 1 --2- 172.21.15.97:0/1167402534 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556aaf8c2c10 0x556aaf8ca680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.150+0000 7fc801b0b640 1 -- 172.21.15.97:0/1167402534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556aaf8b3e20 msgr2=0x556aaf8c67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:34:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.150+0000 7fc801b0b640 1 --2- 172.21.15.97:0/1167402534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556aaf8b3e20 0x556aaf8c67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.150+0000 7fc801b0b640 1 -- 172.21.15.97:0/1167402534 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556aaf8c2c10 msgr2=0x556aaf8ca680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:34:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.150+0000 7fc801b0b640 1 --2- 172.21.15.97:0/1167402534 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556aaf8c2c10 0x556aaf8ca680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.150+0000 7fc801b0b640 1 -- 172.21.15.97:0/1167402534 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556aaf8cacf0 con 0x556aaf8b3500 2024-04-23T10:34:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.151+0000 7fc801b0b640 1 --2- 172.21.15.97:0/1167402534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556aaf8b3500 0x556aaf8c6280 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fc7f80234b0 tx=0x7fc7f8009420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b906489232f6bcd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:34:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.151+0000 7fc7e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:34:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.151+0000 7fc7e57fa640 1 -- 172.21.15.97:0/1167402534 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f8027040 con 0x556aaf8b3500 2024-04-23T10:34:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.151+0000 7fc7e57fa640 1 -- 172.21.15.97:0/1167402534 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7fc7f8023d50 con 0x556aaf8b3500 2024-04-23T10:34:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.151+0000 7fc806b67740 1 -- 172.21.15.97:0/1167402534 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7f8023740 con 0x556aaf8b3500 2024-04-23T10:34:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.151+0000 7fc7e57fa640 1 -- 172.21.15.97:0/1167402534 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc7f8047050 con 0x556aaf8b3500 2024-04-23T10:34:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.151+0000 7fc806b67740 10 client.16755 fetch_fsmap learned FSMap version 26 2024-04-23T10:34:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.151+0000 7fc806b67740 1 -- 172.21.15.97:0/1167402534 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556aaf938b50 con 0x556aaf8b3500 2024-04-23T10:34:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.151+0000 7fc7e57fa640 1 -- 172.21.15.97:0/1167402534 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7f8023740 con 0x556aaf8b3500 2024-04-23T10:34:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.151+0000 7fc7e57fa640 1 -- 172.21.15.97:0/1167402534 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc7f802c930 con 0x556aaf8b3500 2024-04-23T10:34:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.151+0000 7fc806b67740 10 client.16755 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:34:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.151+0000 7fc806b67740 10 client.16755 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:34:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.151+0000 7fc806b67740 20 client.16755 populate_metadata read hostname 'smithi097' 2024-04-23T10:34:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.151+0000 7fc806b67740 1 --2- 172.21.15.97:0/1167402534 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x556aaf8d2bd0 0x556aaf93b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.151+0000 7fc806b67740 4 client.16755 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:34:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.151+0000 7fc806b67740 1 -- 172.21.15.97:0/1167402534 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556aaf8c0030 con 0x556aaf8d2bd0 2024-04-23T10:34:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.152+0000 7fc80130a640 1 --2- 172.21.15.97:0/1167402534 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x556aaf8d2bd0 0x556aaf93b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.152+0000 7fc7e57fa640 10 client.16755 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:34:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.152+0000 7fc80130a640 1 --2- 172.21.15.97:0/1167402534 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x556aaf8d2bd0 0x556aaf93b190 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ffe7aee787a0419 server_cookie=d8f09929d3f03328 in_seq=0 out_seq=0 2024-04-23T10:34:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.164+0000 7fc7e57fa640 1 -- 172.21.15.97:0/1167402534 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556aaf8c0030 con 0x556aaf8d2bd0 2024-04-23T10:34:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.164+0000 7fc7e57fa640 10 client.16755 handle_command_reply: tid=0 2024-04-23T10:34:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.164+0000 7fc806b67740 1 client.16755 shutdown 2024-04-23T10:34:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.164+0000 7fc806b67740 1 -- 172.21.15.97:0/1167402534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556aaf8b3500 msgr2=0x556aaf8c6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.164+0000 7fc806b67740 1 --2- 172.21.15.97:0/1167402534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556aaf8b3500 0x556aaf8c6280 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fc7f80234b0 tx=0x7fc7f8009420 comp rx=0 tx=0).stop 2024-04-23T10:34:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.164+0000 7fc806b67740 1 -- 172.21.15.97:0/1167402534 shutdown_connections 2024-04-23T10:34:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.164+0000 7fc806b67740 1 --2- 172.21.15.97:0/1167402534 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x556aaf8d2bd0 0x556aaf93b190 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.164+0000 7fc806b67740 1 --2- 172.21.15.97:0/1167402534 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556aaf8c2c10 0x556aaf8ca680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.164+0000 7fc806b67740 1 --2- 172.21.15.97:0/1167402534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556aaf8b3e20 0x556aaf8c67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.164+0000 7fc7e57fa640 0 client.16755 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:34:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.164+0000 7fc806b67740 1 --2- 172.21.15.97:0/1167402534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556aaf8b3500 0x556aaf8c6280 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.164+0000 7fc806b67740 1 -- 172.21.15.97:0/1167402534 >> 172.21.15.97:0/1167402534 conn(0x556aaf8afb30 msgr2=0x556aaf8ae420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:34:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.164+0000 7fc806b67740 1 -- 172.21.15.97:0/1167402534 shutdown_connections 2024-04-23T10:34:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.164+0000 7fc806b67740 1 -- 172.21.15.97:0/1167402534 wait complete. 2024-04-23T10:34:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.165+0000 7fc806b67740 20 client.16755 trim_cache size 0 max 16384 2024-04-23T10:34:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:34:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:34:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:34:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:34:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.165+0000 7fc803d6f640 1 -- 172.21.15.97:0/2521459632 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fc7e80a3e40 msgr2=0x7fc7e80a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.165+0000 7fc803d6f640 1 --2- 172.21.15.97:0/2521459632 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fc7e80a3e40 0x7fc7e80a6300 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec0118f0 tx=0x7fc7ec009b20 comp rx=0 tx=0).stop 2024-04-23T10:34:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.165+0000 7fc803d6f640 1 -- 172.21.15.97:0/2521459632 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7fc19cfa0 msgr2=0x7fc7fc19d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.165+0000 7fc803d6f640 1 --2- 172.21.15.97:0/2521459632 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7fc19cfa0 0x7fc7fc19d450 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fc7f402f540 tx=0x7fc7f40316f0 comp rx=0 tx=0).stop 2024-04-23T10:34:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.165+0000 7fc803d6f640 1 -- 172.21.15.97:0/2521459632 shutdown_connections 2024-04-23T10:34:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.165+0000 7fc803d6f640 1 --2- 172.21.15.97:0/2521459632 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fc7e80a3e40 0x7fc7e80a6300 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.165+0000 7fc803d6f640 1 --2- 172.21.15.97:0/2521459632 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7fc19cfa0 0x7fc7fc19d450 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.165+0000 7fc803d6f640 1 --2- 172.21.15.97:0/2521459632 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7fc106170 0x7fc7fc198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.165+0000 7fc803d6f640 1 --2- 172.21.15.97:0/2521459632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7fc1010f0 0x7fc7fc1986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.165+0000 7fc803d6f640 1 -- 172.21.15.97:0/2521459632 >> 172.21.15.97:0/2521459632 conn(0x7fc7fc077fd0 msgr2=0x7fc7fc0fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:34:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.165+0000 7fc803d6f640 1 -- 172.21.15.97:0/2521459632 shutdown_connections 2024-04-23T10:34:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:33.165+0000 7fc803d6f640 1 -- 172.21.15.97:0/2521459632 wait complete. 2024-04-23T10:34:33.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:34:33 smithi183 ceph-mon[32317]: pgmap v257: 97 pgs: 97 active+clean; 3.1 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 12 op/s 2024-04-23T10:34:33.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:34:33 smithi196 ceph-mon[32919]: pgmap v257: 97 pgs: 97 active+clean; 3.1 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 12 op/s 2024-04-23T10:34:33.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:34:33 smithi097 ceph-mon[29197]: pgmap v257: 97 pgs: 97 active+clean; 3.1 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 12 op/s 2024-04-23T10:34:35.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:34:35 smithi183 ceph-mon[32317]: pgmap v258: 97 pgs: 97 active+clean; 3.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 27 KiB/s wr, 17 op/s 2024-04-23T10:34:35.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:34:35 smithi196 ceph-mon[32919]: pgmap v258: 97 pgs: 97 active+clean; 3.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 27 KiB/s wr, 17 op/s 2024-04-23T10:34:35.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:34:35 smithi097 ceph-mon[29197]: pgmap v258: 97 pgs: 97 active+clean; 3.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 27 KiB/s wr, 17 op/s 2024-04-23T10:34:37.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:34:37 smithi183 ceph-mon[32317]: pgmap v259: 97 pgs: 97 active+clean; 3.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 9.0 KiB/s wr, 9 op/s 2024-04-23T10:34:37.412 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:34:37 smithi097 ceph-mon[29197]: pgmap v259: 97 pgs: 97 active+clean; 3.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 9.0 KiB/s wr, 9 op/s 2024-04-23T10:34:37.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:34:37 smithi196 ceph-mon[32919]: pgmap v259: 97 pgs: 97 active+clean; 3.2 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 9.0 KiB/s wr, 9 op/s 2024-04-23T10:34:39.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:34:39 smithi183 ceph-mon[32317]: pgmap v260: 97 pgs: 97 active+clean; 3.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 32 KiB/s wr, 18 op/s 2024-04-23T10:34:39.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:34:39 smithi196 ceph-mon[32919]: pgmap v260: 97 pgs: 97 active+clean; 3.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 32 KiB/s wr, 18 op/s 2024-04-23T10:34:39.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:34:39 smithi097 ceph-mon[29197]: pgmap v260: 97 pgs: 97 active+clean; 3.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 32 KiB/s wr, 18 op/s 2024-04-23T10:34:41.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:34:41 smithi183 ceph-mon[32317]: pgmap v261: 97 pgs: 97 active+clean; 3.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-04-23T10:34:41.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:34:41 smithi196 ceph-mon[32919]: pgmap v261: 97 pgs: 97 active+clean; 3.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-04-23T10:34:41.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:34:41 smithi097 ceph-mon[29197]: pgmap v261: 97 pgs: 97 active+clean; 3.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-04-23T10:34:42.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:34:42 smithi196 ceph-mon[32919]: pgmap v262: 97 pgs: 97 active+clean; 3.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-04-23T10:34:42.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:34:42 smithi097 ceph-mon[29197]: pgmap v262: 97 pgs: 97 active+clean; 3.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-04-23T10:34:42.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:34:42 smithi183 ceph-mon[32317]: pgmap v262: 97 pgs: 97 active+clean; 3.4 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-04-23T10:34:43.218 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:34:43.218 INFO:tasks.workunit.client.0.smithi097.stdout:251/600: open fd = 3 2024-04-23T10:34:43.219 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.219 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.219 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:34:43.219 INFO:tasks.workunit.client.0.smithi097.stdout:252/600: open fd = 3 2024-04-23T10:34:43.219 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.219 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.219 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:34:43.219 INFO:tasks.workunit.client.0.smithi097.stdout:253/600: open fd = 3 2024-04-23T10:34:43.219 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.219 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.219 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:34:43.219 INFO:tasks.workunit.client.0.smithi097.stdout:254/600: open fd = 3 2024-04-23T10:34:43.219 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.220 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.220 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:34:43.220 INFO:tasks.workunit.client.0.smithi097.stdout:255/600: open fd = 3 2024-04-23T10:34:43.220 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.220 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.220 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:34:43.220 INFO:tasks.workunit.client.0.smithi097.stdout:256/600: open fd = 3 2024-04-23T10:34:43.220 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.220 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.220 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:34:43.220 INFO:tasks.workunit.client.0.smithi097.stdout:257/600: open fd = 3 2024-04-23T10:34:43.220 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.220 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.220 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:34:43.220 INFO:tasks.workunit.client.0.smithi097.stdout:258/600: open fd = 3 2024-04-23T10:34:43.221 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.221 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.221 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:34:43.221 INFO:tasks.workunit.client.0.smithi097.stdout:259/600: open fd = 3 2024-04-23T10:34:43.221 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.221 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.221 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:34:43.221 INFO:tasks.workunit.client.0.smithi097.stdout:260/600: open fd = 3 2024-04-23T10:34:43.221 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.221 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.221 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:34:43.221 INFO:tasks.workunit.client.0.smithi097.stdout:261/600: open fd = 3 2024-04-23T10:34:43.221 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.221 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.221 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:34:43.222 INFO:tasks.workunit.client.0.smithi097.stdout:262/600: open fd = 3 2024-04-23T10:34:43.222 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.222 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.222 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:34:43.222 INFO:tasks.workunit.client.0.smithi097.stdout:263/600: open fd = 3 2024-04-23T10:34:43.222 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.222 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.222 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:34:43.222 INFO:tasks.workunit.client.0.smithi097.stdout:264/600: open fd = 3 2024-04-23T10:34:43.222 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.222 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.222 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:34:43.222 INFO:tasks.workunit.client.0.smithi097.stdout:265/600: open fd = 3 2024-04-23T10:34:43.222 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.223 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.223 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:34:43.223 INFO:tasks.workunit.client.0.smithi097.stdout:266/600: open fd = 3 2024-04-23T10:34:43.223 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.223 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.223 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:34:43.223 INFO:tasks.workunit.client.0.smithi097.stdout:267/600: open fd = 3 2024-04-23T10:34:43.223 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.223 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.223 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:34:43.223 INFO:tasks.workunit.client.0.smithi097.stdout:268/600: open fd = 3 2024-04-23T10:34:43.223 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.223 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.223 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:34:43.223 INFO:tasks.workunit.client.0.smithi097.stdout:269/600: open fd = 3 2024-04-23T10:34:43.224 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.224 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.224 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:34:43.224 INFO:tasks.workunit.client.0.smithi097.stdout:270/600: open fd = 3 2024-04-23T10:34:43.224 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.224 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.224 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:34:43.224 INFO:tasks.workunit.client.0.smithi097.stdout:271/600: open fd = 3 2024-04-23T10:34:43.224 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.224 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.224 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:34:43.224 INFO:tasks.workunit.client.0.smithi097.stdout:272/600: open fd = 3 2024-04-23T10:34:43.224 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.224 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.224 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:34:43.225 INFO:tasks.workunit.client.0.smithi097.stdout:273/600: open fd = 3 2024-04-23T10:34:43.225 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.225 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.225 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:34:43.225 INFO:tasks.workunit.client.0.smithi097.stdout:274/600: open fd = 3 2024-04-23T10:34:43.225 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.225 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.225 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:34:43.225 INFO:tasks.workunit.client.0.smithi097.stdout:275/600: open fd = 3 2024-04-23T10:34:43.225 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.225 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.225 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:34:43.225 INFO:tasks.workunit.client.0.smithi097.stdout:276/600: open fd = 3 2024-04-23T10:34:43.225 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.226 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.226 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:34:43.226 INFO:tasks.workunit.client.0.smithi097.stdout:277/600: open fd = 3 2024-04-23T10:34:43.226 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.226 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.226 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:34:43.226 INFO:tasks.workunit.client.0.smithi097.stdout:278/600: open fd = 3 2024-04-23T10:34:43.226 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.226 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.226 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:34:43.226 INFO:tasks.workunit.client.0.smithi097.stdout:279/600: open fd = 3 2024-04-23T10:34:43.226 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.226 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.226 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:34:43.226 INFO:tasks.workunit.client.0.smithi097.stdout:280/600: open fd = 3 2024-04-23T10:34:43.227 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.227 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.227 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:34:43.227 INFO:tasks.workunit.client.0.smithi097.stdout:281/600: open fd = 3 2024-04-23T10:34:43.227 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.227 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.227 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:34:43.227 INFO:tasks.workunit.client.0.smithi097.stdout:282/600: open fd = 3 2024-04-23T10:34:43.227 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.227 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.227 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:34:43.227 INFO:tasks.workunit.client.0.smithi097.stdout:283/600: open fd = 3 2024-04-23T10:34:43.227 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.227 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.227 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:34:43.228 INFO:tasks.workunit.client.0.smithi097.stdout:284/600: open fd = 3 2024-04-23T10:34:43.228 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.228 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.228 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:34:43.228 INFO:tasks.workunit.client.0.smithi097.stdout:285/600: open fd = 3 2024-04-23T10:34:43.228 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.228 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.228 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:34:43.228 INFO:tasks.workunit.client.0.smithi097.stdout:286/600: open fd = 3 2024-04-23T10:34:43.228 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.228 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.228 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:34:43.228 INFO:tasks.workunit.client.0.smithi097.stdout:287/600: open fd = 3 2024-04-23T10:34:43.228 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.229 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.229 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:34:43.229 INFO:tasks.workunit.client.0.smithi097.stdout:288/600: open fd = 3 2024-04-23T10:34:43.229 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.229 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.229 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:34:43.229 INFO:tasks.workunit.client.0.smithi097.stdout:289/600: open fd = 3 2024-04-23T10:34:43.229 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.229 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.229 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:34:43.229 INFO:tasks.workunit.client.0.smithi097.stdout:290/600: open fd = 3 2024-04-23T10:34:43.229 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.229 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.229 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:34:43.229 INFO:tasks.workunit.client.0.smithi097.stdout:291/600: open fd = 3 2024-04-23T10:34:43.230 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.230 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.230 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:34:43.230 INFO:tasks.workunit.client.0.smithi097.stdout:292/600: open fd = 3 2024-04-23T10:34:43.230 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.230 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.230 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:34:43.230 INFO:tasks.workunit.client.0.smithi097.stdout:293/600: open fd = 3 2024-04-23T10:34:43.230 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.230 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.230 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:34:43.230 INFO:tasks.workunit.client.0.smithi097.stdout:294/600: open fd = 3 2024-04-23T10:34:43.230 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.230 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.230 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:34:43.231 INFO:tasks.workunit.client.0.smithi097.stdout:295/600: open fd = 3 2024-04-23T10:34:43.231 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.231 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.231 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:34:43.231 INFO:tasks.workunit.client.0.smithi097.stdout:296/600: open fd = 3 2024-04-23T10:34:43.231 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.231 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.231 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:34:43.231 INFO:tasks.workunit.client.0.smithi097.stdout:297/600: open fd = 3 2024-04-23T10:34:43.231 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.231 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.231 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:34:43.231 INFO:tasks.workunit.client.0.smithi097.stdout:298/600: open fd = 3 2024-04-23T10:34:43.231 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.231 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.232 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:34:43.232 INFO:tasks.workunit.client.0.smithi097.stdout:299/600: open fd = 3 2024-04-23T10:34:43.232 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.232 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.232 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:34:43.232 INFO:tasks.workunit.client.0.smithi097.stdout:300/600: open fd = 3 2024-04-23T10:34:43.232 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.232 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.232 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:34:43.232 INFO:tasks.workunit.client.0.smithi097.stdout:301/600: open fd = 3 2024-04-23T10:34:43.232 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.232 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.232 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:34:43.232 INFO:tasks.workunit.client.0.smithi097.stdout:302/600: open fd = 3 2024-04-23T10:34:43.233 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.233 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.233 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:34:43.233 INFO:tasks.workunit.client.0.smithi097.stdout:303/600: open fd = 3 2024-04-23T10:34:43.233 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.233 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.233 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:34:43.233 INFO:tasks.workunit.client.0.smithi097.stdout:304/600: open fd = 3 2024-04-23T10:34:43.233 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.233 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.233 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:34:43.233 INFO:tasks.workunit.client.0.smithi097.stdout:305/600: open fd = 3 2024-04-23T10:34:43.233 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.233 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.233 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:34:43.234 INFO:tasks.workunit.client.0.smithi097.stdout:306/600: open fd = 3 2024-04-23T10:34:43.234 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.234 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.234 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:34:43.234 INFO:tasks.workunit.client.0.smithi097.stdout:307/600: open fd = 3 2024-04-23T10:34:43.234 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.234 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.234 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:34:43.234 INFO:tasks.workunit.client.0.smithi097.stdout:308/600: open fd = 3 2024-04-23T10:34:43.234 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.234 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.234 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:34:43.234 INFO:tasks.workunit.client.0.smithi097.stdout:309/600: open fd = 3 2024-04-23T10:34:43.234 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.235 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.235 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:34:43.235 INFO:tasks.workunit.client.0.smithi097.stdout:310/600: open fd = 3 2024-04-23T10:34:43.235 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.235 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.235 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:34:43.235 INFO:tasks.workunit.client.0.smithi097.stdout:311/600: open fd = 3 2024-04-23T10:34:43.235 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.235 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.235 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:34:43.235 INFO:tasks.workunit.client.0.smithi097.stdout:312/600: open fd = 3 2024-04-23T10:34:43.235 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:43.235 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:34:44.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:34:44 smithi196 ceph-mon[32919]: pgmap v263: 97 pgs: 97 active+clean; 3.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 44 KiB/s wr, 20 op/s 2024-04-23T10:34:44.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:34:44 smithi097 ceph-mon[29197]: pgmap v263: 97 pgs: 97 active+clean; 3.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 44 KiB/s wr, 20 op/s 2024-04-23T10:34:44.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:34:44 smithi183 ceph-mon[32317]: pgmap v263: 97 pgs: 97 active+clean; 3.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 44 KiB/s wr, 20 op/s 2024-04-23T10:34:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.261+0000 7f6bf0860640 1 -- 172.21.15.97:0/2586133148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6bec1021d0 msgr2=0x7f6bec1025b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.261+0000 7f6bf0860640 1 --2- 172.21.15.97:0/2586133148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6bec1021d0 0x7f6bec1025b0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc009900 tx=0x7f6bdc02a410 comp rx=0 tx=0).stop 2024-04-23T10:34:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.261+0000 7f6bf0860640 1 -- 172.21.15.97:0/2586133148 shutdown_connections 2024-04-23T10:34:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.261+0000 7f6bf0860640 1 --2- 172.21.15.97:0/2586133148 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6bec1072a0 0x7f6bec10ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.261+0000 7f6bf0860640 1 --2- 172.21.15.97:0/2586133148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6bec102b80 0x7f6bec106bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.261+0000 7f6bf0860640 1 --2- 172.21.15.97:0/2586133148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6bec1021d0 0x7f6bec1025b0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.261+0000 7f6bf0860640 1 -- 172.21.15.97:0/2586133148 >> 172.21.15.97:0/2586133148 conn(0x7f6bec0793e0 msgr2=0x7f6bec0797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:34:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.261+0000 7f6bf0860640 1 -- 172.21.15.97:0/2586133148 shutdown_connections 2024-04-23T10:34:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.261+0000 7f6bf0860640 1 -- 172.21.15.97:0/2586133148 wait complete. 2024-04-23T10:34:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.262+0000 7f6bf0860640 1 Processor -- start 2024-04-23T10:34:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.262+0000 7f6bf0860640 1 -- start start 2024-04-23T10:34:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.262+0000 7f6bf0860640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6bec1021d0 0x7f6bec198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.262+0000 7f6bf0860640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6bec1072a0 0x7f6bec198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.262+0000 7f6bf0860640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6bec19cf40 0x7f6bec19d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.262+0000 7f6bf0860640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6bec1121e0 con 0x7f6bec1021d0 2024-04-23T10:34:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.262+0000 7f6bf0860640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6bec112060 con 0x7f6bec1072a0 2024-04-23T10:34:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.262+0000 7f6bf0860640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6bec112360 con 0x7f6bec19cf40 2024-04-23T10:34:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.262+0000 7f6bea59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6bec1021d0 0x7f6bec198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.262+0000 7f6bea59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6bec1021d0 0x7f6bec198680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52002/0 (socket says 172.21.15.97:52002) 2024-04-23T10:34:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.262+0000 7f6bea59c640 1 -- 172.21.15.97:0/2051536324 learned_addr learned my addr 172.21.15.97:0/2051536324 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:34:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.263+0000 7f6bead9d640 1 --2- 172.21.15.97:0/2051536324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6bec19cf40 0x7f6bec19d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.263+0000 7f6be3fff640 1 --2- 172.21.15.97:0/2051536324 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6bec1072a0 0x7f6bec198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.263+0000 7f6bea59c640 1 -- 172.21.15.97:0/2051536324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6bec19cf40 msgr2=0x7f6bec19d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.263+0000 7f6bea59c640 1 --2- 172.21.15.97:0/2051536324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6bec19cf40 0x7f6bec19d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.263+0000 7f6bea59c640 1 -- 172.21.15.97:0/2051536324 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6bec1072a0 msgr2=0x7f6bec198bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.263+0000 7f6bea59c640 1 --2- 172.21.15.97:0/2051536324 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6bec1072a0 0x7f6bec198bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.263+0000 7f6bea59c640 1 -- 172.21.15.97:0/2051536324 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bec19da30 con 0x7f6bec1021d0 2024-04-23T10:34:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.263+0000 7f6be3fff640 1 --2- 172.21.15.97:0/2051536324 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6bec1072a0 0x7f6bec198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:34:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.263+0000 7f6bea59c640 1 --2- 172.21.15.97:0/2051536324 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6bec1021d0 0x7f6bec198680 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc005940 tx=0x7f6bdc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:34:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.263+0000 7f6be37fe640 1 -- 172.21.15.97:0/2051536324 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc03f030 con 0x7f6bec1021d0 2024-04-23T10:34:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.263+0000 7f6be37fe640 1 -- 172.21.15.97:0/2051536324 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6bdc003cb0 con 0x7f6bec1021d0 2024-04-23T10:34:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.263+0000 7f6bf0860640 1 -- 172.21.15.97:0/2051536324 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bec19dcc0 con 0x7f6bec1021d0 2024-04-23T10:34:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.263+0000 7f6be37fe640 1 -- 172.21.15.97:0/2051536324 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc035070 con 0x7f6bec1021d0 2024-04-23T10:34:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.263+0000 7f6bf0860640 1 -- 172.21.15.97:0/2051536324 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bec1a5480 con 0x7f6bec1021d0 2024-04-23T10:34:46.272 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.266+0000 7f6be37fe640 1 -- 172.21.15.97:0/2051536324 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6bdc042030 con 0x7f6bec1021d0 2024-04-23T10:34:46.272 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.266+0000 7f6bf3573740 1 Processor -- start 2024-04-23T10:34:46.273 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.266+0000 7f6bf3573740 1 -- start start 2024-04-23T10:34:46.273 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.266+0000 7f6be37fe640 1 --2- 172.21.15.97:0/2051536324 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f6bd80a3dc0 0x7f6bd80a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:46.273 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.266+0000 7f6bf3573740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e2a9e4620 0x555e2a9e4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:46.273 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.266+0000 7f6bf3573740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e2a9e4f40 0x555e2a9ef2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:46.273 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.266+0000 7f6bf3573740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e2a9ef800 0x555e2a9f1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:46.273 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.266+0000 7f6bf3573740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x555e2a949970 con 0x555e2a9e4f40 2024-04-23T10:34:46.273 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.266+0000 7f6bf3573740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555e2a8e64a0 con 0x555e2a9e4620 2024-04-23T10:34:46.273 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.266+0000 7f6bf3573740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555e2a945930 con 0x555e2a9ef800 2024-04-23T10:34:46.273 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.267+0000 7f6bead9d640 1 --2- 172.21.15.97:0/2051536324 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f6bd80a3dc0 0x7f6bd80a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:46.273 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.267+0000 7f6be3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e2a9e4f40 0x555e2a9ef2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:46.273 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.267+0000 7f6be37fe640 1 -- 172.21.15.97:0/2051536324 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7f6bdc0f1060 con 0x7f6bec1021d0 2024-04-23T10:34:46.273 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.267+0000 7f6be3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e2a9e4f40 0x555e2a9ef2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52006/0 (socket says 172.21.15.97:52006) 2024-04-23T10:34:46.273 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.267+0000 7f6be3fff640 1 -- 172.21.15.97:0/1829756643 learned_addr learned my addr 172.21.15.97:0/1829756643 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:34:46.273 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.267+0000 7f6bea59c640 1 --2- 172.21.15.97:0/1829756643 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e2a9e4620 0x555e2a9e4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:46.274 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.267+0000 7f6bead9d640 1 --2- 172.21.15.97:0/1829756643 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e2a9ef800 0x555e2a9f1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:46.274 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.267+0000 7f6be3fff640 1 -- 172.21.15.97:0/1829756643 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e2a9ef800 msgr2=0x555e2a9f1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:34:46.274 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.267+0000 7f6be3fff640 1 --2- 172.21.15.97:0/1829756643 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e2a9ef800 0x555e2a9f1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:46.274 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.267+0000 7f6be3fff640 1 -- 172.21.15.97:0/1829756643 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e2a9e4620 msgr2=0x555e2a9e4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:34:46.274 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.267+0000 7f6be3fff640 1 --2- 172.21.15.97:0/1829756643 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e2a9e4620 0x555e2a9e4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:46.274 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.267+0000 7f6be3fff640 1 -- 172.21.15.97:0/1829756643 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e2a65c210 con 0x555e2a9e4f40 2024-04-23T10:34:46.274 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.267+0000 7f6bead9d640 1 --2- 172.21.15.97:0/2051536324 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f6bd80a3dc0 0x7f6bd80a6280 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6be4005fd0 tx=0x7f6be4014b70 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:34:46.274 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.267+0000 7f6be3fff640 1 --2- 172.21.15.97:0/1829756643 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e2a9e4f40 0x555e2a9ef2c0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f6bd400cc40 tx=0x7f6bd400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13f118b4a355d7c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:34:46.274 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.267+0000 7f6be0ff9640 1 -- 172.21.15.97:0/1829756643 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd4014070 con 0x555e2a9e4f40 2024-04-23T10:34:46.274 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.267+0000 7f6be0ff9640 1 -- 172.21.15.97:0/1829756643 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6bd40047a0 con 0x555e2a9e4f40 2024-04-23T10:34:46.274 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.267+0000 7f6be0ff9640 1 -- 172.21.15.97:0/1829756643 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd4010650 con 0x555e2a9e4f40 2024-04-23T10:34:46.274 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.268+0000 7f6bf3573740 1 -- 172.21.15.97:0/1829756643 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e2a9e4f40 msgr2=0x555e2a9ef2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:46.274 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.268+0000 7f6bf3573740 1 --2- 172.21.15.97:0/1829756643 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e2a9e4f40 0x555e2a9ef2c0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f6bd400cc40 tx=0x7f6bd400e720 comp rx=0 tx=0).stop 2024-04-23T10:34:46.274 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.268+0000 7f6bf3573740 1 -- 172.21.15.97:0/1829756643 shutdown_connections 2024-04-23T10:34:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.268+0000 7f6bf3573740 1 --2- 172.21.15.97:0/1829756643 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e2a9ef800 0x555e2a9f1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.268+0000 7f6bf3573740 1 --2- 172.21.15.97:0/1829756643 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e2a9e4f40 0x555e2a9ef2c0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.268+0000 7f6bf3573740 1 --2- 172.21.15.97:0/1829756643 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e2a9e4620 0x555e2a9e4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.268+0000 7f6bf3573740 1 -- 172.21.15.97:0/1829756643 >> 172.21.15.97:0/1829756643 conn(0x555e2a9e0b30 msgr2=0x555e2a9e0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:34:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.268+0000 7f6bf3573740 1 -- 172.21.15.97:0/1829756643 shutdown_connections 2024-04-23T10:34:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.268+0000 7f6bf3573740 1 -- 172.21.15.97:0/1829756643 wait complete. 2024-04-23T10:34:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.269+0000 7f6bf3573740 1 -- start start 2024-04-23T10:34:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.269+0000 7f6bf3573740 1 Processor -- start 2024-04-23T10:34:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.269+0000 7f6bf3573740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e2a9e4620 0x555e2a9ef150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.269+0000 7f6bf3573740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e2a9e4f40 0x555e2a9ec440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.269+0000 7f6bf3573740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e2a9ef800 0x555e2a9ec9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.269+0000 7f6bf3573740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x555e2a9f2430 con 0x555e2a9e4f40 2024-04-23T10:34:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.269+0000 7f6bf3573740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555e2a8e40b0 con 0x555e2a9ef800 2024-04-23T10:34:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.269+0000 7f6bf3573740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555e2a9f25b0 con 0x555e2a9e4620 2024-04-23T10:34:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.269+0000 7f6bf3573740 10 client.0 _ll_register_callbacks cb 0x555e2a8d1d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:34:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.269+0000 7f6be3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e2a9e4f40 0x555e2a9ec440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.269+0000 7f6be3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e2a9e4f40 0x555e2a9ec440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52014/0 (socket says 172.21.15.97:52014) 2024-04-23T10:34:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.269+0000 7f6be3fff640 1 -- 172.21.15.97:0/561151976 learned_addr learned my addr 172.21.15.97:0/561151976 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:34:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.269+0000 7f6bead9d640 1 --2- 172.21.15.97:0/561151976 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e2a9ef800 0x555e2a9ec9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.269+0000 7f6bea59c640 1 --2- 172.21.15.97:0/561151976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e2a9e4620 0x555e2a9ef150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.269+0000 7f6be3fff640 1 -- 172.21.15.97:0/561151976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e2a9e4620 msgr2=0x555e2a9ef150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:34:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.269+0000 7f6be3fff640 1 --2- 172.21.15.97:0/561151976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e2a9e4620 0x555e2a9ef150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.269+0000 7f6be3fff640 1 -- 172.21.15.97:0/561151976 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e2a9ef800 msgr2=0x555e2a9ec9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:34:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.269+0000 7f6be3fff640 1 --2- 172.21.15.97:0/561151976 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e2a9ef800 0x555e2a9ec9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.269+0000 7f6be3fff640 1 -- 172.21.15.97:0/561151976 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e2a728620 con 0x555e2a9e4f40 2024-04-23T10:34:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.269+0000 7f6bea59c640 1 --2- 172.21.15.97:0/561151976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e2a9e4620 0x555e2a9ef150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:34:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.269+0000 7f6be3fff640 1 --2- 172.21.15.97:0/561151976 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e2a9e4f40 0x555e2a9ec440 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f6bd40058c0 tx=0x7f6bd4005230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48013e2ff32fda55 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:34:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.269+0000 7f6bd1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:34:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.270+0000 7f6bd1ffb640 1 -- 172.21.15.97:0/561151976 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd4014070 con 0x555e2a9e4f40 2024-04-23T10:34:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.270+0000 7f6bd1ffb640 1 -- 172.21.15.97:0/561151976 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7f6bd4004020 con 0x555e2a9e4f40 2024-04-23T10:34:46.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.270+0000 7f6bf3573740 1 -- 172.21.15.97:0/561151976 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6bd40047a0 con 0x555e2a9e4f40 2024-04-23T10:34:46.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.270+0000 7f6bd1ffb640 1 -- 172.21.15.97:0/561151976 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6bd400c810 con 0x555e2a9e4f40 2024-04-23T10:34:46.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.270+0000 7f6bf3573740 10 client.16767 fetch_fsmap learned FSMap version 26 2024-04-23T10:34:46.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.270+0000 7f6bf3573740 1 -- 172.21.15.97:0/561151976 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555e2a9fe3a0 con 0x555e2a9e4f40 2024-04-23T10:34:46.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.270+0000 7f6bd1ffb640 1 -- 172.21.15.97:0/561151976 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6bd40047a0 con 0x555e2a9e4f40 2024-04-23T10:34:46.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.270+0000 7f6bd1ffb640 1 -- 172.21.15.97:0/561151976 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f6bd4029020 con 0x555e2a9e4f40 2024-04-23T10:34:46.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.270+0000 7f6bf3573740 10 client.16767 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:34:46.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.270+0000 7f6bf3573740 10 client.16767 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:34:46.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.270+0000 7f6bf3573740 20 client.16767 populate_metadata read hostname 'smithi097' 2024-04-23T10:34:46.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.270+0000 7f6bf3573740 1 --2- 172.21.15.97:0/561151976 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x555e2a9ff7a0 0x555e2aa01b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:46.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.270+0000 7f6bf3573740 4 client.16767 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:34:46.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.270+0000 7f6bf3573740 1 -- 172.21.15.97:0/561151976 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e2a957390 con 0x555e2a9ff7a0 2024-04-23T10:34:46.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.270+0000 7f6bea59c640 1 --2- 172.21.15.97:0/561151976 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x555e2a9ff7a0 0x555e2aa01b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:46.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.271+0000 7f6bd1ffb640 10 client.16767 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:34:46.277 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.271+0000 7f6bea59c640 1 --2- 172.21.15.97:0/561151976 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x555e2a9ff7a0 0x555e2aa01b90 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d25acc77c4e3eaf3 server_cookie=4f622ca72f3bb54c in_seq=0 out_seq=0 2024-04-23T10:34:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.272+0000 7f6bd1ffb640 1 -- 172.21.15.97:0/561151976 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555e2a957390 con 0x555e2a9ff7a0 2024-04-23T10:34:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.272+0000 7f6bd1ffb640 10 client.16767 handle_command_reply: tid=0 2024-04-23T10:34:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.272+0000 7f6bf3573740 1 client.16767 shutdown 2024-04-23T10:34:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.272+0000 7f6bf3573740 1 -- 172.21.15.97:0/561151976 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e2a9e4f40 msgr2=0x555e2a9ec440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.272+0000 7f6bf3573740 1 --2- 172.21.15.97:0/561151976 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e2a9e4f40 0x555e2a9ec440 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f6bd40058c0 tx=0x7f6bd4005230 comp rx=0 tx=0).stop 2024-04-23T10:34:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.272+0000 7f6bf3573740 1 -- 172.21.15.97:0/561151976 shutdown_connections 2024-04-23T10:34:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.272+0000 7f6bf3573740 1 --2- 172.21.15.97:0/561151976 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x555e2a9ff7a0 0x555e2aa01b90 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.272+0000 7f6bf3573740 1 --2- 172.21.15.97:0/561151976 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e2a9ef800 0x555e2a9ec9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.272+0000 7f6bf3573740 1 --2- 172.21.15.97:0/561151976 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e2a9e4f40 0x555e2a9ec440 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.272+0000 7f6bd1ffb640 0 client.16767 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:34:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.272+0000 7f6bf3573740 1 --2- 172.21.15.97:0/561151976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e2a9e4620 0x555e2a9ef150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.272+0000 7f6bf3573740 1 -- 172.21.15.97:0/561151976 >> 172.21.15.97:0/561151976 conn(0x555e2a9e0b30 msgr2=0x555e2a9e68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:34:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.272+0000 7f6bf3573740 1 -- 172.21.15.97:0/561151976 shutdown_connections 2024-04-23T10:34:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.272+0000 7f6bf3573740 1 -- 172.21.15.97:0/561151976 wait complete. 2024-04-23T10:34:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.272+0000 7f6bf3573740 20 client.16767 trim_cache size 0 max 16384 2024-04-23T10:34:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.287+0000 7f6bf3573740 1 Processor -- start 2024-04-23T10:34:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.287+0000 7f6bf3573740 1 -- start start 2024-04-23T10:34:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.287+0000 7f6bf3573740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e2a9e4620 0x555e2a9fb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.287+0000 7f6bf3573740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e2a9e4f40 0x555e2a9dfb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.287+0000 7f6bf3573740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e2a9ef800 0x555e2aa071c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.287+0000 7f6bf3573740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6bec1ada20 con 0x555e2a9e4f40 2024-04-23T10:34:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.287+0000 7f6bf3573740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6bec1af560 con 0x555e2a9e4620 2024-04-23T10:34:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.287+0000 7f6bf3573740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6bec1ae0f0 con 0x555e2a9ef800 2024-04-23T10:34:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.287+0000 7f6be3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e2a9e4f40 0x555e2a9dfb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.287+0000 7f6be3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e2a9e4f40 0x555e2a9dfb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52016/0 (socket says 172.21.15.97:52016) 2024-04-23T10:34:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.287+0000 7f6be3fff640 1 -- 172.21.15.97:0/626592182 learned_addr learned my addr 172.21.15.97:0/626592182 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:34:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.287+0000 7f6be3fff640 1 -- 172.21.15.97:0/626592182 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e2a9ef800 msgr2=0x555e2aa071c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:34:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.287+0000 7f6be3fff640 1 --2- 172.21.15.97:0/626592182 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e2a9ef800 0x555e2aa071c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.287+0000 7f6be3fff640 1 -- 172.21.15.97:0/626592182 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e2a9e4620 msgr2=0x555e2a9fb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:34:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.287+0000 7f6be3fff640 1 --2- 172.21.15.97:0/626592182 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e2a9e4620 0x555e2a9fb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.287+0000 7f6be3fff640 1 -- 172.21.15.97:0/626592182 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e2a9f1fb0 con 0x555e2a9e4f40 2024-04-23T10:34:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.287+0000 7f6be3fff640 1 --2- 172.21.15.97:0/626592182 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e2a9e4f40 0x555e2a9dfb30 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f6bd4003d80 tx=0x7f6bd4005a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c07253b421f9ab2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:34:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.288+0000 7f6bd3fff640 1 -- 172.21.15.97:0/626592182 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd4014070 con 0x555e2a9e4f40 2024-04-23T10:34:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.288+0000 7f6bd3fff640 1 -- 172.21.15.97:0/626592182 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6bd4010430 con 0x555e2a9e4f40 2024-04-23T10:34:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.288+0000 7f6bd3fff640 1 -- 172.21.15.97:0/626592182 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd40041e0 con 0x555e2a9e4f40 2024-04-23T10:34:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.288+0000 7f6bf3573740 1 -- 172.21.15.97:0/626592182 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e2a9e4f40 msgr2=0x555e2a9dfb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.288+0000 7f6bf3573740 1 --2- 172.21.15.97:0/626592182 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e2a9e4f40 0x555e2a9dfb30 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f6bd4003d80 tx=0x7f6bd4005a10 comp rx=0 tx=0).stop 2024-04-23T10:34:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.288+0000 7f6bf3573740 1 -- 172.21.15.97:0/626592182 shutdown_connections 2024-04-23T10:34:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.288+0000 7f6bf3573740 1 --2- 172.21.15.97:0/626592182 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e2a9ef800 0x555e2aa071c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.288+0000 7f6bf3573740 1 --2- 172.21.15.97:0/626592182 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e2a9e4f40 0x555e2a9dfb30 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.288+0000 7f6bf3573740 1 --2- 172.21.15.97:0/626592182 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e2a9e4620 0x555e2a9fb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.288+0000 7f6bf3573740 1 -- 172.21.15.97:0/626592182 >> 172.21.15.97:0/626592182 conn(0x555e2a9e0b30 msgr2=0x555e2a9df420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:34:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.288+0000 7f6bf3573740 1 -- 172.21.15.97:0/626592182 shutdown_connections 2024-04-23T10:34:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.288+0000 7f6bf3573740 1 -- 172.21.15.97:0/626592182 wait complete. 2024-04-23T10:34:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.288+0000 7f6bf3573740 1 -- start start 2024-04-23T10:34:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.289+0000 7f6bf3573740 1 Processor -- start 2024-04-23T10:34:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.289+0000 7f6bf3573740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e2a9e4620 0x555e2a9f79b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.289+0000 7f6bf3573740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e2a9e4f40 0x555e2a9f4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.289+0000 7f6bf3573740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e2a9ef800 0x555e2a9f5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.289+0000 7f6bf3573740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6bec1aee90 con 0x555e2a9e4f40 2024-04-23T10:34:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.289+0000 7f6bf3573740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6bec1ae7c0 con 0x555e2a9e4620 2024-04-23T10:34:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.289+0000 7f6bf3573740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6bec1aba20 con 0x555e2a9ef800 2024-04-23T10:34:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.289+0000 7f6bf3573740 10 client.0 _ll_register_callbacks cb 0x7f6bd4004cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:34:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.289+0000 7f6be3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e2a9e4f40 0x555e2a9f4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.289+0000 7f6be3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e2a9e4f40 0x555e2a9f4dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52020/0 (socket says 172.21.15.97:52020) 2024-04-23T10:34:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.289+0000 7f6be3fff640 1 -- 172.21.15.97:0/3837493779 learned_addr learned my addr 172.21.15.97:0/3837493779 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:34:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.289+0000 7f6bea59c640 1 --2- 172.21.15.97:0/3837493779 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e2a9e4620 0x555e2a9f79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.289+0000 7f6bead9d640 1 --2- 172.21.15.97:0/3837493779 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e2a9ef800 0x555e2a9f5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.289+0000 7f6be3fff640 1 -- 172.21.15.97:0/3837493779 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e2a9ef800 msgr2=0x555e2a9f5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:34:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.289+0000 7f6be3fff640 1 --2- 172.21.15.97:0/3837493779 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e2a9ef800 0x555e2a9f5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.289+0000 7f6be3fff640 1 -- 172.21.15.97:0/3837493779 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e2a9e4620 msgr2=0x555e2a9f79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:34:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.289+0000 7f6be3fff640 1 --2- 172.21.15.97:0/3837493779 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e2a9e4620 0x555e2a9f79b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.289+0000 7f6be3fff640 1 -- 172.21.15.97:0/3837493779 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e2a9f5900 con 0x555e2a9e4f40 2024-04-23T10:34:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.289+0000 7f6bea59c640 1 --2- 172.21.15.97:0/3837493779 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e2a9e4620 0x555e2a9f79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:34:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.289+0000 7f6be3fff640 1 --2- 172.21.15.97:0/3837493779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e2a9e4f40 0x555e2a9f4dd0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f6bd40058c0 tx=0x7f6bd400cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e3158231a6064b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:34:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.289+0000 7f6bd1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:34:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.289+0000 7f6bd1ffb640 1 -- 172.21.15.97:0/3837493779 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd4014040 con 0x555e2a9e4f40 2024-04-23T10:34:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.289+0000 7f6bd1ffb640 1 -- 172.21.15.97:0/3837493779 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7f6bd4003f80 con 0x555e2a9e4f40 2024-04-23T10:34:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.289+0000 7f6bf3573740 1 -- 172.21.15.97:0/3837493779 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6bd4010430 con 0x555e2a9e4f40 2024-04-23T10:34:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.290+0000 7f6bd1ffb640 1 -- 172.21.15.97:0/3837493779 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6bd4042050 con 0x555e2a9e4f40 2024-04-23T10:34:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.290+0000 7f6bf3573740 10 client.16773 fetch_fsmap learned FSMap version 26 2024-04-23T10:34:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.290+0000 7f6bf3573740 1 -- 172.21.15.97:0/3837493779 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555e2aa65830 con 0x555e2a9e4f40 2024-04-23T10:34:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.290+0000 7f6bd1ffb640 1 -- 172.21.15.97:0/3837493779 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6bd4010430 con 0x555e2a9e4f40 2024-04-23T10:34:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.290+0000 7f6bd1ffb640 1 -- 172.21.15.97:0/3837493779 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f6bd4010a30 con 0x555e2a9e4f40 2024-04-23T10:34:46.295 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.290+0000 7f6bf3573740 10 client.16773 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:34:46.295 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.290+0000 7f6bf3573740 10 client.16773 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:34:46.295 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.290+0000 7f6bf3573740 20 client.16773 populate_metadata read hostname 'smithi097' 2024-04-23T10:34:46.295 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.290+0000 7f6bf3573740 1 --2- 172.21.15.97:0/3837493779 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x555e2aa65e40 0x555e2aa68260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:46.295 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.290+0000 7f6bf3573740 4 client.16773 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:34:46.295 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.290+0000 7f6bf3573740 1 -- 172.21.15.97:0/3837493779 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555e2a951200 con 0x555e2aa65e40 2024-04-23T10:34:46.295 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.290+0000 7f6bea59c640 1 --2- 172.21.15.97:0/3837493779 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x555e2aa65e40 0x555e2aa68260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:46.295 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.291+0000 7f6bea59c640 1 --2- 172.21.15.97:0/3837493779 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x555e2aa65e40 0x555e2aa68260 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=248237292fe1a97 server_cookie=c903f24820223f7e in_seq=0 out_seq=0 2024-04-23T10:34:46.295 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.291+0000 7f6bd1ffb640 10 client.16773 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:34:46.305 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.305+0000 7f6bd1ffb640 1 -- 172.21.15.97:0/3837493779 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555e2a951200 con 0x555e2aa65e40 2024-04-23T10:34:46.305 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.305+0000 7f6bd1ffb640 10 client.16773 handle_command_reply: tid=0 2024-04-23T10:34:46.305 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.305+0000 7f6bf3573740 1 client.16773 shutdown 2024-04-23T10:34:46.305 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.305+0000 7f6bf3573740 1 -- 172.21.15.97:0/3837493779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e2a9e4f40 msgr2=0x555e2a9f4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:46.305 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.305+0000 7f6bf3573740 1 --2- 172.21.15.97:0/3837493779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e2a9e4f40 0x555e2a9f4dd0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f6bd40058c0 tx=0x7f6bd400cb50 comp rx=0 tx=0).stop 2024-04-23T10:34:46.305 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.305+0000 7f6bf3573740 1 -- 172.21.15.97:0/3837493779 shutdown_connections 2024-04-23T10:34:46.305 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.305+0000 7f6bf3573740 1 --2- 172.21.15.97:0/3837493779 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x555e2aa65e40 0x555e2aa68260 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:46.305 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.305+0000 7f6bf3573740 1 --2- 172.21.15.97:0/3837493779 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e2a9ef800 0x555e2a9f5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:46.305 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.305+0000 7f6bd1ffb640 0 client.16773 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:34:46.305 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.305+0000 7f6bf3573740 1 --2- 172.21.15.97:0/3837493779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e2a9e4f40 0x555e2a9f4dd0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:46.305 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.305+0000 7f6bf3573740 1 --2- 172.21.15.97:0/3837493779 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e2a9e4620 0x555e2a9f79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:46.305 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.305+0000 7f6bf3573740 1 -- 172.21.15.97:0/3837493779 >> 172.21.15.97:0/3837493779 conn(0x555e2a9e0b30 msgr2=0x555e2a9dfcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:34:46.305 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.305+0000 7f6bf3573740 1 -- 172.21.15.97:0/3837493779 shutdown_connections 2024-04-23T10:34:46.305 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.305+0000 7f6bf3573740 1 -- 172.21.15.97:0/3837493779 wait complete. 2024-04-23T10:34:46.306 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.305+0000 7f6bf3573740 20 client.16773 trim_cache size 0 max 16384 2024-04-23T10:34:46.306 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:34:46.306 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:34:46.306 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:34:46.306 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:34:46.307 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.305+0000 7f6bf0860640 1 -- 172.21.15.97:0/2051536324 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f6bd80a3dc0 msgr2=0x7f6bd80a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:46.307 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.305+0000 7f6bf0860640 1 --2- 172.21.15.97:0/2051536324 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f6bd80a3dc0 0x7f6bd80a6280 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6be4005fd0 tx=0x7f6be4014b70 comp rx=0 tx=0).stop 2024-04-23T10:34:46.307 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.305+0000 7f6bf0860640 1 -- 172.21.15.97:0/2051536324 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6bec1021d0 msgr2=0x7f6bec198680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:46.307 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.305+0000 7f6bf0860640 1 --2- 172.21.15.97:0/2051536324 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6bec1021d0 0x7f6bec198680 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc005940 tx=0x7f6bdc0042f0 comp rx=0 tx=0).stop 2024-04-23T10:34:46.307 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.305+0000 7f6bf0860640 1 -- 172.21.15.97:0/2051536324 shutdown_connections 2024-04-23T10:34:46.307 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.305+0000 7f6bf0860640 1 --2- 172.21.15.97:0/2051536324 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f6bd80a3dc0 0x7f6bd80a6280 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:46.307 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.305+0000 7f6bf0860640 1 --2- 172.21.15.97:0/2051536324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6bec19cf40 0x7f6bec19d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:46.307 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.305+0000 7f6bf0860640 1 --2- 172.21.15.97:0/2051536324 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6bec1072a0 0x7f6bec198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:46.307 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.305+0000 7f6bf0860640 1 --2- 172.21.15.97:0/2051536324 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6bec1021d0 0x7f6bec198680 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:46.307 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.305+0000 7f6bf0860640 1 -- 172.21.15.97:0/2051536324 >> 172.21.15.97:0/2051536324 conn(0x7f6bec0793e0 msgr2=0x7f6bec0ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:34:46.307 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.305+0000 7f6bf0860640 1 -- 172.21.15.97:0/2051536324 shutdown_connections 2024-04-23T10:34:46.307 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:46.305+0000 7f6bf0860640 1 -- 172.21.15.97:0/2051536324 wait complete. 2024-04-23T10:34:46.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:34:46 smithi196 ceph-mon[32919]: pgmap v264: 97 pgs: 97 active+clean; 3.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 KiB/s wr, 15 op/s 2024-04-23T10:34:46.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:34:46 smithi097 ceph-mon[29197]: pgmap v264: 97 pgs: 97 active+clean; 3.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 KiB/s wr, 15 op/s 2024-04-23T10:34:46.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:34:46 smithi183 ceph-mon[32317]: pgmap v264: 97 pgs: 97 active+clean; 3.6 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 KiB/s wr, 15 op/s 2024-04-23T10:34:48.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:34:48 smithi196 ceph-mon[32919]: pgmap v265: 97 pgs: 97 active+clean; 3.8 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 61 KiB/s wr, 24 op/s 2024-04-23T10:34:48.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:34:48 smithi097 ceph-mon[29197]: pgmap v265: 97 pgs: 97 active+clean; 3.8 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 61 KiB/s wr, 24 op/s 2024-04-23T10:34:48.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:34:48 smithi183 ceph-mon[32317]: pgmap v265: 97 pgs: 97 active+clean; 3.8 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 61 KiB/s wr, 24 op/s 2024-04-23T10:34:49.706 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 10 2024-04-23T10:34:49.706 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T10:34:49.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:49.818+0000 7febc3033640 1 -- 172.21.15.97:0/87134637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febbc101090 msgr2=0x7febbc101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:49.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:49.818+0000 7febc3033640 1 --2- 172.21.15.97:0/87134637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febbc101090 0x7febbc101470 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7febb0009900 tx=0x7febb002a410 comp rx=0 tx=0).stop 2024-04-23T10:34:49.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:49.818+0000 7febc3033640 1 -- 172.21.15.97:0/87134637 shutdown_connections 2024-04-23T10:34:49.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:49.818+0000 7febc3033640 1 --2- 172.21.15.97:0/87134637 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febbc106160 0x7febbc10dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:49.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:49.818+0000 7febc3033640 1 --2- 172.21.15.97:0/87134637 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7febbc101a40 0x7febbc105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:49.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:49.818+0000 7febc3033640 1 --2- 172.21.15.97:0/87134637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febbc101090 0x7febbc101470 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:49.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:49.818+0000 7febc3033640 1 -- 172.21.15.97:0/87134637 >> 172.21.15.97:0/87134637 conn(0x7febbc0771d0 msgr2=0x7febbc0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:34:49.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:49.818+0000 7febc3033640 1 -- 172.21.15.97:0/87134637 shutdown_connections 2024-04-23T10:34:49.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:49.818+0000 7febc3033640 1 -- 172.21.15.97:0/87134637 wait complete. 2024-04-23T10:34:49.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:49.819+0000 7febc3033640 1 Processor -- start 2024-04-23T10:34:49.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:49.819+0000 7febc3033640 1 -- start start 2024-04-23T10:34:49.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:49.819+0000 7febc3033640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febbc101090 0x7febbc10e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:49.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:49.819+0000 7febc3033640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febbc106160 0x7febbc10ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:49.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:49.819+0000 7febc3033640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7febbc10f140 0x7febbc1a6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:49.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:49.819+0000 7febc3033640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7febbc1121e0 con 0x7febbc106160 2024-04-23T10:34:49.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:49.819+0000 7febc3033640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7febbc112060 con 0x7febbc101090 2024-04-23T10:34:49.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:49.819+0000 7febc3033640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7febbc112360 con 0x7febbc10f140 2024-04-23T10:34:49.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:49.819+0000 7febbbfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febbc106160 0x7febbc10ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:49.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:49.819+0000 7febbbfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febbc106160 0x7febbc10ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52028/0 (socket says 172.21.15.97:52028) 2024-04-23T10:34:49.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:49.819+0000 7febbbfff640 1 -- 172.21.15.97:0/4098857036 learned_addr learned my addr 172.21.15.97:0/4098857036 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:34:49.820 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:49.819+0000 7febc15d0640 1 --2- 172.21.15.97:0/4098857036 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7febbc10f140 0x7febbc1a6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:49.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:49.819+0000 7febc0dcf640 1 --2- 172.21.15.97:0/4098857036 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febbc101090 0x7febbc10e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:49.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:49.819+0000 7febbbfff640 1 -- 172.21.15.97:0/4098857036 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7febbc10f140 msgr2=0x7febbc1a6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:49.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:49.819+0000 7febbbfff640 1 --2- 172.21.15.97:0/4098857036 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7febbc10f140 0x7febbc1a6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:49.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:49.819+0000 7febbbfff640 1 -- 172.21.15.97:0/4098857036 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febbc101090 msgr2=0x7febbc10e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:49.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:49.819+0000 7febbbfff640 1 --2- 172.21.15.97:0/4098857036 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febbc101090 0x7febbc10e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:49.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:49.819+0000 7febbbfff640 1 -- 172.21.15.97:0/4098857036 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febbc1a7050 con 0x7febbc106160 2024-04-23T10:34:49.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:49.819+0000 7febbbfff640 1 --2- 172.21.15.97:0/4098857036 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febbc106160 0x7febbc10ec00 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7feba800d740 tx=0x7feba800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:34:49.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:49.820+0000 7febb9ffb640 1 -- 172.21.15.97:0/4098857036 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba8017070 con 0x7febbc106160 2024-04-23T10:34:49.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:49.820+0000 7febb9ffb640 1 -- 172.21.15.97:0/4098857036 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feba800de50 con 0x7febbc106160 2024-04-23T10:34:49.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:49.820+0000 7febc3033640 1 -- 172.21.15.97:0/4098857036 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febbc1a7340 con 0x7febbc106160 2024-04-23T10:34:49.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:49.820+0000 7febb9ffb640 1 -- 172.21.15.97:0/4098857036 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba8004ce0 con 0x7febbc106160 2024-04-23T10:34:49.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:49.820+0000 7febc3033640 1 -- 172.21.15.97:0/4098857036 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febbc1a7840 con 0x7febbc106160 2024-04-23T10:34:49.821 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:49.821+0000 7febc3033640 1 -- 172.21.15.97:0/4098857036 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febbc102560 con 0x7febbc106160 2024-04-23T10:34:49.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:49.822+0000 7febb9ffb640 1 -- 172.21.15.97:0/4098857036 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7feba8004e80 con 0x7febbc106160 2024-04-23T10:34:49.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:49.822+0000 7febb9ffb640 1 --2- 172.21.15.97:0/4098857036 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7febac0a3d50 0x7febac0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:49.822 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:49.822+0000 7febc0dcf640 1 --2- 172.21.15.97:0/4098857036 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7febac0a3d50 0x7febac0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:49.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:49.822+0000 7febb9ffb640 1 -- 172.21.15.97:0/4098857036 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7feba80d00b0 con 0x7febbc106160 2024-04-23T10:34:49.823 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:49.823+0000 7febc0dcf640 1 --2- 172.21.15.97:0/4098857036 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7febac0a3d50 0x7febac0a6210 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7febb0031440 tx=0x7febb0005b50 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:34:49.825 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:49.827+0000 7febb9ffb640 1 -- 172.21.15.97:0/4098857036 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feba809c120 con 0x7febbc106160 2024-04-23T10:34:50.039 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.040+0000 7febc3033640 1 -- 172.21.15.97:0/4098857036 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7febbc109390 con 0x7febbc106160 2024-04-23T10:34:50.040 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.041+0000 7febb9ffb640 1 -- 172.21.15.97:0/4098857036 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11931 (secure 0 0 0) 0x7feba80a0420 con 0x7febbc106160 2024-04-23T10:34:50.040 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:34:50.040 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3946878762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3946878762},{"type":"v1","addr":"172.21.15.97:6835","nonce":3946878762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/209800861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":209800861},{"type":"v1","addr":"172.21.15.97:6837","nonce":209800861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T10:24:06.910134+0000","modified":"2024-04-23T10:26:09.820985+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6839/418013575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":418013575},{"type":"v1","addr":"172.21.15.97:6839","nonce":418013575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6841/747242680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":747242680},{"type":"v1","addr":"172.21.15.97:6841","nonce":747242680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6833/4292847121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4292847121},{"type":"v1","addr":"172.21.15.183:6833","nonce":4292847121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6833/3885816565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3885816565},{"type":"v1","addr":"172.21.15.196:6833","nonce":3885816565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1897093105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1897093105},{"type":"v1","addr":"172.21.15.183:6835","nonce":1897093105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1171477563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1171477563},{"type":"v1","addr":"172.21.15.196:6835","nonce":1171477563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1898341911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1898341911},{"type":"v1","addr":"172.21.15.183:6837","nonce":1898341911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.196:6837/3094946502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3094946502},{"type":"v1","addr":"172.21.15.196:6837","nonce":3094946502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.196:6839/83843556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":83843556},{"type":"v1","addr":"172.21.15.196:6839","nonce":83843556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6839/1763841130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1763841130},{"type":"v1","addr":"172.21.15.183:6839","nonce":1763841130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24487,24403,24398,24469,24488]},"id":1}]} 2024-04-23T10:34:50.040 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 26 2024-04-23T10:34:50.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.043+0000 7febc3033640 1 -- 172.21.15.97:0/4098857036 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7febac0a3d50 msgr2=0x7febac0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:50.042 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.043+0000 7febc3033640 1 --2- 172.21.15.97:0/4098857036 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7febac0a3d50 0x7febac0a6210 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7febb0031440 tx=0x7febb0005b50 comp rx=0 tx=0).stop 2024-04-23T10:34:50.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.043+0000 7febc3033640 1 -- 172.21.15.97:0/4098857036 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febbc106160 msgr2=0x7febbc10ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:50.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.043+0000 7febc3033640 1 --2- 172.21.15.97:0/4098857036 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febbc106160 0x7febbc10ec00 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7feba800d740 tx=0x7feba800dc00 comp rx=0 tx=0).stop 2024-04-23T10:34:50.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.043+0000 7febc3033640 1 -- 172.21.15.97:0/4098857036 shutdown_connections 2024-04-23T10:34:50.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.043+0000 7febc3033640 1 --2- 172.21.15.97:0/4098857036 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7febac0a3d50 0x7febac0a6210 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:50.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.043+0000 7febc3033640 1 --2- 172.21.15.97:0/4098857036 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7febbc10f140 0x7febbc1a6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:50.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.043+0000 7febc3033640 1 --2- 172.21.15.97:0/4098857036 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7febbc106160 0x7febbc10ec00 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:50.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.043+0000 7febc3033640 1 --2- 172.21.15.97:0/4098857036 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febbc101090 0x7febbc10e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:50.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.043+0000 7febc3033640 1 -- 172.21.15.97:0/4098857036 >> 172.21.15.97:0/4098857036 conn(0x7febbc0771d0 msgr2=0x7febbc0fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:34:50.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.043+0000 7febc3033640 1 -- 172.21.15.97:0/4098857036 shutdown_connections 2024-04-23T10:34:50.043 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.043+0000 7febc3033640 1 -- 172.21.15.97:0/4098857036 wait complete. 2024-04-23T10:34:50.053 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db / --timeout 60 --expiration 70 --await-for 42.0' 2024-04-23T10:34:50.053 DEBUG:teuthology.orchestra.run.smithi097:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24488 quiesce db / --timeout 60 --expiration 70 --await-for 42.0 2024-04-23T10:34:50.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.163+0000 7fd9ab2bb640 1 -- 172.21.15.97:0/4022007673 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9a4106160 msgr2=0x7fd9a410dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:50.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.163+0000 7fd9ab2bb640 1 --2- 172.21.15.97:0/4022007673 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9a4106160 0x7fd9a410dd30 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fd99c009ca0 tx=0x7fd99c02a8c0 comp rx=0 tx=0).stop 2024-04-23T10:34:50.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.163+0000 7fd9ab2bb640 1 -- 172.21.15.97:0/4022007673 shutdown_connections 2024-04-23T10:34:50.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.163+0000 7fd9ab2bb640 1 --2- 172.21.15.97:0/4022007673 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9a4106160 0x7fd9a410dd30 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:50.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.163+0000 7fd9ab2bb640 1 --2- 172.21.15.97:0/4022007673 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9a4101a40 0x7fd9a4105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:50.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.163+0000 7fd9ab2bb640 1 --2- 172.21.15.97:0/4022007673 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9a4101090 0x7fd9a4101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:50.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.163+0000 7fd9ab2bb640 1 -- 172.21.15.97:0/4022007673 >> 172.21.15.97:0/4022007673 conn(0x7fd9a40771d0 msgr2=0x7fd9a40775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:34:50.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.163+0000 7fd9ab2bb640 1 -- 172.21.15.97:0/4022007673 shutdown_connections 2024-04-23T10:34:50.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.163+0000 7fd9ab2bb640 1 -- 172.21.15.97:0/4022007673 wait complete. 2024-04-23T10:34:50.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.163+0000 7fd9ab2bb640 1 Processor -- start 2024-04-23T10:34:50.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.163+0000 7fd9ab2bb640 1 -- start start 2024-04-23T10:34:50.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.163+0000 7fd9ab2bb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9a4101090 0x7fd9a419e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:50.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.163+0000 7fd9ab2bb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9a4106160 0x7fd9a419e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:50.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.163+0000 7fd9ab2bb640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9a4198640 0x7fd9a4198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:50.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.163+0000 7fd9ab2bb640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a41110d0 con 0x7fd9a4101090 2024-04-23T10:34:50.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.163+0000 7fd9ab2bb640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a4110f50 con 0x7fd9a4198640 2024-04-23T10:34:50.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.163+0000 7fd9ab2bb640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a4111250 con 0x7fd9a4106160 2024-04-23T10:34:50.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.164+0000 7fd9a9057640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9a4101090 0x7fd9a419e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:50.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.164+0000 7fd9a9057640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9a4101090 0x7fd9a419e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52034/0 (socket says 172.21.15.97:52034) 2024-04-23T10:34:50.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.164+0000 7fd9a9057640 1 -- 172.21.15.97:0/3351785993 learned_addr learned my addr 172.21.15.97:0/3351785993 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:34:50.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.164+0000 7fd9a8856640 1 --2- 172.21.15.97:0/3351785993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9a4106160 0x7fd9a419e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:50.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.164+0000 7fd9a9057640 1 -- 172.21.15.97:0/3351785993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9a4106160 msgr2=0x7fd9a419e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:50.164 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.164+0000 7fd9a9057640 1 --2- 172.21.15.97:0/3351785993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9a4106160 0x7fd9a419e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:50.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.164+0000 7fd9a9057640 1 -- 172.21.15.97:0/3351785993 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9a4198640 msgr2=0x7fd9a4198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:50.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.164+0000 7fd9a9057640 1 --2- 172.21.15.97:0/3351785993 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9a4198640 0x7fd9a4198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:50.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.164+0000 7fd9a9057640 1 -- 172.21.15.97:0/3351785993 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9a4199160 con 0x7fd9a4101090 2024-04-23T10:34:50.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.164+0000 7fd9a9057640 1 --2- 172.21.15.97:0/3351785993 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9a4101090 0x7fd9a419e440 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fd9a000d770 tx=0x7fd9a000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:34:50.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.164+0000 7fd99a7fc640 1 -- 172.21.15.97:0/3351785993 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a0017070 con 0x7fd9a4101090 2024-04-23T10:34:50.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.164+0000 7fd99a7fc640 1 -- 172.21.15.97:0/3351785993 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9a00054d0 con 0x7fd9a4101090 2024-04-23T10:34:50.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.164+0000 7fd9ab2bb640 1 -- 172.21.15.97:0/3351785993 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9a4199450 con 0x7fd9a4101090 2024-04-23T10:34:50.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.164+0000 7fd99a7fc640 1 -- 172.21.15.97:0/3351785993 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a0004c30 con 0x7fd9a4101090 2024-04-23T10:34:50.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.164+0000 7fd9ab2bb640 1 -- 172.21.15.97:0/3351785993 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9a41a5600 con 0x7fd9a4101090 2024-04-23T10:34:50.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.166+0000 7fd99a7fc640 1 -- 172.21.15.97:0/3351785993 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd9a0004e10 con 0x7fd9a4101090 2024-04-23T10:34:50.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.167+0000 7fd9adfcf740 1 Processor -- start 2024-04-23T10:34:50.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.167+0000 7fd9adfcf740 1 -- start start 2024-04-23T10:34:50.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.167+0000 7fd9adfcf740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558cdea180a0 0x558cdea18480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:50.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.167+0000 7fd9adfcf740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558cdea189c0 0x558cdea22b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:50.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.167+0000 7fd9adfcf740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558cdea23090 0x558cdea25480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:50.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.167+0000 7fd9adfcf740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x558cde97d370 con 0x558cdea189c0 2024-04-23T10:34:50.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.167+0000 7fd9adfcf740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558cde91a2e0 con 0x558cdea180a0 2024-04-23T10:34:50.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.167+0000 7fd9adfcf740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558cde979330 con 0x558cdea23090 2024-04-23T10:34:50.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.167+0000 7fd99a7fc640 1 --2- 172.21.15.97:0/3351785993 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fd9900a3d20 0x7fd9900a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:50.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.167+0000 7fd9a9057640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558cdea189c0 0x558cdea22b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:50.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.167+0000 7fd9a9057640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558cdea189c0 0x558cdea22b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52038/0 (socket says 172.21.15.97:52038) 2024-04-23T10:34:50.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.167+0000 7fd9a9057640 1 -- 172.21.15.97:0/3085322600 learned_addr learned my addr 172.21.15.97:0/3085322600 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:34:50.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.167+0000 7fd9a9858640 1 --2- 172.21.15.97:0/3351785993 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fd9900a3d20 0x7fd9900a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:50.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.167+0000 7fd9a8856640 1 --2- 172.21.15.97:0/3085322600 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558cdea23090 0x558cdea25480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:50.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.167+0000 7fd9a9858640 1 --2- 172.21.15.97:0/3085322600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558cdea180a0 0x558cdea18480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:50.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.167+0000 7fd99a7fc640 1 -- 172.21.15.97:0/3351785993 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7fd9a00d02b0 con 0x7fd9a4101090 2024-04-23T10:34:50.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.167+0000 7fd9a9057640 1 -- 172.21.15.97:0/3085322600 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558cdea23090 msgr2=0x558cdea25480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:34:50.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.167+0000 7fd9a9057640 1 --2- 172.21.15.97:0/3085322600 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558cdea23090 0x558cdea25480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:50.171 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.167+0000 7fd9a9057640 1 -- 172.21.15.97:0/3085322600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558cdea180a0 msgr2=0x558cdea18480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:34:50.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.167+0000 7fd9a9057640 1 --2- 172.21.15.97:0/3085322600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558cdea180a0 0x558cdea18480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:50.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.167+0000 7fd9a9057640 1 -- 172.21.15.97:0/3085322600 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cde75bd80 con 0x558cdea189c0 2024-04-23T10:34:50.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.168+0000 7fd9a9858640 1 --2- 172.21.15.97:0/3351785993 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fd9900a3d20 0x7fd9900a61e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd99c005cc0 tx=0x7fd99c02f3a0 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:34:50.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.168+0000 7fd9a9057640 1 --2- 172.21.15.97:0/3085322600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558cdea189c0 0x558cdea22b50 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fd9a009c200 tx=0x7fd9a0022f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b718f256b15ebeb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:34:50.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.168+0000 7fd98f7fe640 1 -- 172.21.15.97:0/3085322600 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a00cf070 con 0x558cdea189c0 2024-04-23T10:34:50.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.168+0000 7fd98f7fe640 1 -- 172.21.15.97:0/3085322600 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9a009c270 con 0x558cdea189c0 2024-04-23T10:34:50.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.168+0000 7fd98f7fe640 1 -- 172.21.15.97:0/3085322600 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a0020040 con 0x558cdea189c0 2024-04-23T10:34:50.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.168+0000 7fd9adfcf740 1 -- 172.21.15.97:0/3085322600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558cdea189c0 msgr2=0x558cdea22b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:50.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.168+0000 7fd9adfcf740 1 --2- 172.21.15.97:0/3085322600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558cdea189c0 0x558cdea22b50 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fd9a009c200 tx=0x7fd9a0022f00 comp rx=0 tx=0).stop 2024-04-23T10:34:50.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.168+0000 7fd9adfcf740 1 -- 172.21.15.97:0/3085322600 shutdown_connections 2024-04-23T10:34:50.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.168+0000 7fd9adfcf740 1 --2- 172.21.15.97:0/3085322600 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558cdea23090 0x558cdea25480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:50.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.168+0000 7fd9adfcf740 1 --2- 172.21.15.97:0/3085322600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558cdea189c0 0x558cdea22b50 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:50.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.168+0000 7fd9adfcf740 1 --2- 172.21.15.97:0/3085322600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558cdea180a0 0x558cdea18480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:50.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.168+0000 7fd9adfcf740 1 -- 172.21.15.97:0/3085322600 >> 172.21.15.97:0/3085322600 conn(0x558cdea143c0 msgr2=0x558cdea147a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:34:50.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.168+0000 7fd9adfcf740 1 -- 172.21.15.97:0/3085322600 shutdown_connections 2024-04-23T10:34:50.172 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.168+0000 7fd9adfcf740 1 -- 172.21.15.97:0/3085322600 wait complete. 2024-04-23T10:34:50.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.169+0000 7fd9adfcf740 1 -- start start 2024-04-23T10:34:50.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.169+0000 7fd9adfcf740 1 Processor -- start 2024-04-23T10:34:50.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.169+0000 7fd9adfcf740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558cdea180a0 0x558cdea1ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:50.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.169+0000 7fd9adfcf740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558cdea189c0 0x558cdea20500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:50.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.169+0000 7fd9adfcf740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558cdea23090 0x558cdea20a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:50.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.169+0000 7fd9adfcf740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x558cdea25e20 con 0x558cdea189c0 2024-04-23T10:34:50.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.169+0000 7fd9adfcf740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558cde917fb0 con 0x558cdea23090 2024-04-23T10:34:50.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.169+0000 7fd9adfcf740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558cdea25fa0 con 0x558cdea180a0 2024-04-23T10:34:50.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.169+0000 7fd9a8856640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558cdea189c0 0x558cdea20500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:50.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.169+0000 7fd9adfcf740 10 client.0 _ll_register_callbacks cb 0x558cde9179a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:34:50.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.169+0000 7fd9a8856640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558cdea189c0 0x558cdea20500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52040/0 (socket says 172.21.15.97:52040) 2024-04-23T10:34:50.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.169+0000 7fd9a8856640 1 -- 172.21.15.97:0/2108403010 learned_addr learned my addr 172.21.15.97:0/2108403010 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:34:50.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.169+0000 7fd9a9858640 1 --2- 172.21.15.97:0/2108403010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558cdea23090 0x558cdea20a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:50.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.169+0000 7fd9a9057640 1 --2- 172.21.15.97:0/2108403010 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558cdea180a0 0x558cdea1ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:50.173 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.170+0000 7fd9a8856640 1 -- 172.21.15.97:0/2108403010 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558cdea180a0 msgr2=0x558cdea1ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:34:50.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.170+0000 7fd9a8856640 1 --2- 172.21.15.97:0/2108403010 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558cdea180a0 0x558cdea1ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:50.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.170+0000 7fd9a8856640 1 -- 172.21.15.97:0/2108403010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558cdea23090 msgr2=0x558cdea20a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:34:50.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.170+0000 7fd9a8856640 1 --2- 172.21.15.97:0/2108403010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558cdea23090 0x558cdea20a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:50.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.170+0000 7fd9a8856640 1 -- 172.21.15.97:0/2108403010 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cde7bc510 con 0x558cdea189c0 2024-04-23T10:34:50.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.170+0000 7fd9a9057640 1 --2- 172.21.15.97:0/2108403010 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558cdea180a0 0x558cdea1ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:34:50.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.170+0000 7fd9a8856640 1 --2- 172.21.15.97:0/2108403010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558cdea189c0 0x558cdea20500 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fd99400bed0 tx=0x7fd994011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd4096c901d7e48a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:34:50.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.170+0000 7fd98cff9640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:34:50.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.170+0000 7fd98cff9640 1 -- 172.21.15.97:0/2108403010 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd994004020 con 0x558cdea189c0 2024-04-23T10:34:50.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.170+0000 7fd98cff9640 1 -- 172.21.15.97:0/2108403010 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7fd9940041c0 con 0x558cdea189c0 2024-04-23T10:34:50.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.170+0000 7fd9adfcf740 1 -- 172.21.15.97:0/2108403010 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9a009c270 con 0x558cdea189c0 2024-04-23T10:34:50.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.170+0000 7fd98cff9640 1 -- 172.21.15.97:0/2108403010 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd9940043c0 con 0x558cdea189c0 2024-04-23T10:34:50.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.170+0000 7fd9adfcf740 10 client.16791 fetch_fsmap learned FSMap version 26 2024-04-23T10:34:50.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.170+0000 7fd9adfcf740 1 -- 172.21.15.97:0/2108403010 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558cde64f670 con 0x558cdea189c0 2024-04-23T10:34:50.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.170+0000 7fd98cff9640 1 -- 172.21.15.97:0/2108403010 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9a009c270 con 0x558cdea189c0 2024-04-23T10:34:50.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.170+0000 7fd98cff9640 1 -- 172.21.15.97:0/2108403010 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd994021030 con 0x558cdea189c0 2024-04-23T10:34:50.174 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.170+0000 7fd9adfcf740 10 client.16791 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:34:50.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.170+0000 7fd9adfcf740 10 client.16791 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:34:50.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.170+0000 7fd9adfcf740 20 client.16791 populate_metadata read hostname 'smithi097' 2024-04-23T10:34:50.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.170+0000 7fd9adfcf740 1 --2- 172.21.15.97:0/2108403010 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x558cdea33350 0x558cdea357f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:50.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.170+0000 7fd9adfcf740 4 client.16791 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:34:50.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.170+0000 7fd9adfcf740 1 -- 172.21.15.97:0/2108403010 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558cde98ad90 con 0x558cdea33350 2024-04-23T10:34:50.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.171+0000 7fd9a9057640 1 --2- 172.21.15.97:0/2108403010 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x558cdea33350 0x558cdea357f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:50.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.171+0000 7fd98cff9640 10 client.16791 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:34:50.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.171+0000 7fd9a9057640 1 --2- 172.21.15.97:0/2108403010 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x558cdea33350 0x558cdea357f0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f1e29c56e198bc1 server_cookie=78128812b0fed1e7 in_seq=0 out_seq=0 2024-04-23T10:34:50.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.172+0000 7fd98cff9640 1 -- 172.21.15.97:0/2108403010 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558cde98ad90 con 0x558cdea33350 2024-04-23T10:34:50.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.172+0000 7fd98cff9640 10 client.16791 handle_command_reply: tid=0 2024-04-23T10:34:50.175 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.172+0000 7fd9adfcf740 1 client.16791 shutdown 2024-04-23T10:34:50.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.172+0000 7fd9adfcf740 1 -- 172.21.15.97:0/2108403010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558cdea189c0 msgr2=0x558cdea20500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:50.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.172+0000 7fd9adfcf740 1 --2- 172.21.15.97:0/2108403010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558cdea189c0 0x558cdea20500 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fd99400bed0 tx=0x7fd994011810 comp rx=0 tx=0).stop 2024-04-23T10:34:50.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.172+0000 7fd9adfcf740 1 -- 172.21.15.97:0/2108403010 shutdown_connections 2024-04-23T10:34:50.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.172+0000 7fd9adfcf740 1 --2- 172.21.15.97:0/2108403010 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x558cdea33350 0x558cdea357f0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:50.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.172+0000 7fd9adfcf740 1 --2- 172.21.15.97:0/2108403010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558cdea23090 0x558cdea20a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:50.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.172+0000 7fd98cff9640 0 client.16791 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:34:50.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.172+0000 7fd9adfcf740 1 --2- 172.21.15.97:0/2108403010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558cdea189c0 0x558cdea20500 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:50.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.172+0000 7fd9adfcf740 1 --2- 172.21.15.97:0/2108403010 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558cdea180a0 0x558cdea1ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:50.176 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.172+0000 7fd9adfcf740 1 -- 172.21.15.97:0/2108403010 >> 172.21.15.97:0/2108403010 conn(0x558cdea143c0 msgr2=0x558cdea1a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:34:50.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.172+0000 7fd9adfcf740 1 -- 172.21.15.97:0/2108403010 shutdown_connections 2024-04-23T10:34:50.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.172+0000 7fd9adfcf740 1 -- 172.21.15.97:0/2108403010 wait complete. 2024-04-23T10:34:50.177 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.172+0000 7fd9adfcf740 20 client.16791 trim_cache size 0 max 16384 2024-04-23T10:34:50.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.189+0000 7fd9adfcf740 1 Processor -- start 2024-04-23T10:34:50.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.189+0000 7fd9adfcf740 1 -- start start 2024-04-23T10:34:50.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.190+0000 7fd9adfcf740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558cdea180a0 0x558cdea13d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:50.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.190+0000 7fd9adfcf740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558cdea189c0 0x558cdea2b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:50.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.190+0000 7fd9adfcf740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558cdea23090 0x558cdea20690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:50.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.190+0000 7fd9adfcf740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a41adc00 con 0x558cdea23090 2024-04-23T10:34:50.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.190+0000 7fd9adfcf740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a41af740 con 0x558cdea180a0 2024-04-23T10:34:50.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.190+0000 7fd9adfcf740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a41ae2d0 con 0x558cdea189c0 2024-04-23T10:34:50.190 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.190+0000 7fd9a9858640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558cdea23090 0x558cdea20690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:50.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.190+0000 7fd9a9858640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558cdea23090 0x558cdea20690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52042/0 (socket says 172.21.15.97:52042) 2024-04-23T10:34:50.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.190+0000 7fd9a9858640 1 -- 172.21.15.97:0/3003879970 learned_addr learned my addr 172.21.15.97:0/3003879970 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:34:50.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.190+0000 7fd9a9057640 1 --2- 172.21.15.97:0/3003879970 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558cdea180a0 0x558cdea13d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:50.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.190+0000 7fd9a8856640 1 --2- 172.21.15.97:0/3003879970 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558cdea189c0 0x558cdea2b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:50.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.190+0000 7fd9a9858640 1 -- 172.21.15.97:0/3003879970 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558cdea189c0 msgr2=0x558cdea2b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:34:50.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.190+0000 7fd9a9858640 1 --2- 172.21.15.97:0/3003879970 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558cdea189c0 0x558cdea2b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:50.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.190+0000 7fd9a9858640 1 -- 172.21.15.97:0/3003879970 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558cdea180a0 msgr2=0x558cdea13d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:34:50.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.190+0000 7fd9a9858640 1 --2- 172.21.15.97:0/3003879970 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558cdea180a0 0x558cdea13d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:50.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.190+0000 7fd9a9858640 1 -- 172.21.15.97:0/3003879970 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cdea187f0 con 0x558cdea23090 2024-04-23T10:34:50.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.190+0000 7fd9a9858640 1 --2- 172.21.15.97:0/3003879970 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558cdea23090 0x558cdea20690 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fd99c02b070 tx=0x7fd99c049000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85297e88b9c0c117 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:34:50.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.190+0000 7fd98effd640 1 -- 172.21.15.97:0/3003879970 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99c0318d0 con 0x558cdea23090 2024-04-23T10:34:50.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.190+0000 7fd98effd640 1 -- 172.21.15.97:0/3003879970 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd99c036040 con 0x558cdea23090 2024-04-23T10:34:50.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.190+0000 7fd98effd640 1 -- 172.21.15.97:0/3003879970 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99c030a50 con 0x558cdea23090 2024-04-23T10:34:50.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.191+0000 7fd9adfcf740 1 -- 172.21.15.97:0/3003879970 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558cdea23090 msgr2=0x558cdea20690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:50.191 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.191+0000 7fd9adfcf740 1 --2- 172.21.15.97:0/3003879970 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558cdea23090 0x558cdea20690 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fd99c02b070 tx=0x7fd99c049000 comp rx=0 tx=0).stop 2024-04-23T10:34:50.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.191+0000 7fd9adfcf740 1 -- 172.21.15.97:0/3003879970 shutdown_connections 2024-04-23T10:34:50.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.191+0000 7fd9adfcf740 1 --2- 172.21.15.97:0/3003879970 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558cdea23090 0x558cdea20690 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:50.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.191+0000 7fd9adfcf740 1 --2- 172.21.15.97:0/3003879970 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558cdea189c0 0x558cdea2b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:50.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.191+0000 7fd9adfcf740 1 --2- 172.21.15.97:0/3003879970 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558cdea180a0 0x558cdea13d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:50.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.191+0000 7fd9adfcf740 1 -- 172.21.15.97:0/3003879970 >> 172.21.15.97:0/3003879970 conn(0x558cdea143c0 msgr2=0x558cdea1ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:34:50.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.191+0000 7fd9adfcf740 1 -- 172.21.15.97:0/3003879970 shutdown_connections 2024-04-23T10:34:50.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.191+0000 7fd9adfcf740 1 -- 172.21.15.97:0/3003879970 wait complete. 2024-04-23T10:34:50.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.191+0000 7fd9adfcf740 1 -- start start 2024-04-23T10:34:50.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.191+0000 7fd9adfcf740 1 Processor -- start 2024-04-23T10:34:50.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.192+0000 7fd9adfcf740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558cdea180a0 0x558cdea1d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:50.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.192+0000 7fd9adfcf740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558cdea189c0 0x558cdea1c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:50.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.192+0000 7fd9adfcf740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558cdea23090 0x558cdea1c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:50.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.192+0000 7fd9adfcf740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a41af070 con 0x558cdea189c0 2024-04-23T10:34:50.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.192+0000 7fd9adfcf740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a41ae9a0 con 0x558cdea23090 2024-04-23T10:34:50.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.192+0000 7fd9adfcf740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a41abc00 con 0x558cdea180a0 2024-04-23T10:34:50.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.192+0000 7fd9adfcf740 10 client.0 _ll_register_callbacks cb 0x558cde9179a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:34:50.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.192+0000 7fd9a8856640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558cdea189c0 0x558cdea1c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:50.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.192+0000 7fd9a8856640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558cdea189c0 0x558cdea1c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52052/0 (socket says 172.21.15.97:52052) 2024-04-23T10:34:50.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.192+0000 7fd9a8856640 1 -- 172.21.15.97:0/2122964191 learned_addr learned my addr 172.21.15.97:0/2122964191 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:34:50.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.192+0000 7fd9a9057640 1 --2- 172.21.15.97:0/2122964191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558cdea180a0 0x558cdea1d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:50.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.192+0000 7fd9a9858640 1 --2- 172.21.15.97:0/2122964191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558cdea23090 0x558cdea1c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:50.195 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.192+0000 7fd9a8856640 1 -- 172.21.15.97:0/2122964191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558cdea180a0 msgr2=0x558cdea1d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:34:50.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.192+0000 7fd9a8856640 1 --2- 172.21.15.97:0/2122964191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558cdea180a0 0x558cdea1d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:50.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.192+0000 7fd9a8856640 1 -- 172.21.15.97:0/2122964191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558cdea23090 msgr2=0x558cdea1c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:34:50.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.192+0000 7fd9a8856640 1 --2- 172.21.15.97:0/2122964191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558cdea23090 0x558cdea1c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:50.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.192+0000 7fd9a8856640 1 -- 172.21.15.97:0/2122964191 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cdea1cdf0 con 0x558cdea189c0 2024-04-23T10:34:50.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.192+0000 7fd9a9057640 1 --2- 172.21.15.97:0/2122964191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558cdea180a0 0x558cdea1d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:34:50.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.192+0000 7fd9a8856640 1 --2- 172.21.15.97:0/2122964191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558cdea189c0 0x558cdea1c270 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x558cdea21590 tx=0x7fd9940038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bdc2a87776968f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:34:50.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.192+0000 7fd98cff9640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:34:50.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.192+0000 7fd98cff9640 1 -- 172.21.15.97:0/2122964191 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd994004210 con 0x558cdea189c0 2024-04-23T10:34:50.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.192+0000 7fd98cff9640 1 -- 172.21.15.97:0/2122964191 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7fd994010700 con 0x558cdea189c0 2024-04-23T10:34:50.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.192+0000 7fd9adfcf740 1 -- 172.21.15.97:0/2122964191 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd99c036040 con 0x558cdea189c0 2024-04-23T10:34:50.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.193+0000 7fd98cff9640 1 -- 172.21.15.97:0/2122964191 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd994010900 con 0x558cdea189c0 2024-04-23T10:34:50.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.193+0000 7fd9adfcf740 10 client.16797 fetch_fsmap learned FSMap version 26 2024-04-23T10:34:50.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.193+0000 7fd9adfcf740 1 -- 172.21.15.97:0/2122964191 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558cdea9e760 con 0x558cdea189c0 2024-04-23T10:34:50.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.193+0000 7fd98cff9640 1 -- 172.21.15.97:0/2122964191 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd99c036040 con 0x558cdea189c0 2024-04-23T10:34:50.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.193+0000 7fd98cff9640 1 -- 172.21.15.97:0/2122964191 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd994032030 con 0x558cdea189c0 2024-04-23T10:34:50.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.193+0000 7fd9adfcf740 10 client.16797 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:34:50.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.193+0000 7fd9adfcf740 10 client.16797 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:34:50.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.193+0000 7fd9adfcf740 20 client.16797 populate_metadata read hostname 'smithi097' 2024-04-23T10:34:50.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.193+0000 7fd9adfcf740 1 --2- 172.21.15.97:0/2122964191 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x558cdea9ef40 0x558cdeaa1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:50.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.193+0000 7fd9adfcf740 4 client.16797 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 70.0, "await_for": 42.0}] 2024-04-23T10:34:50.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.193+0000 7fd9adfcf740 1 -- 172.21.15.97:0/2122964191 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 70.0, "await_for": 42.0}) -- 0x558cde984c00 con 0x558cdea9ef40 2024-04-23T10:34:50.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.193+0000 7fd9a9057640 1 --2- 172.21.15.97:0/2122964191 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x558cdea9ef40 0x558cdeaa1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:50.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.193+0000 7fd98cff9640 10 client.16797 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:34:50.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.193+0000 7fd9a9057640 1 --2- 172.21.15.97:0/2122964191 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x558cdea9ef40 0x558cdeaa1360 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a1f110072af8eaf server_cookie=ad3c7ee8087025f3 in_seq=0 out_seq=0 2024-04-23T10:34:50.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.212+0000 7fd98cff9640 1 -- 172.21.15.97:0/2122964191 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x558cde984c00 con 0x558cdea9ef40 2024-04-23T10:34:50.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.212+0000 7fd98cff9640 10 client.16797 handle_command_reply: tid=0 2024-04-23T10:34:50.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.212+0000 7fd9adfcf740 1 client.16797 shutdown 2024-04-23T10:34:50.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.213+0000 7fd9adfcf740 1 -- 172.21.15.97:0/2122964191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558cdea189c0 msgr2=0x558cdea1c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:50.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.213+0000 7fd9adfcf740 1 --2- 172.21.15.97:0/2122964191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558cdea189c0 0x558cdea1c270 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x558cdea21590 tx=0x7fd9940038a0 comp rx=0 tx=0).stop 2024-04-23T10:34:50.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.213+0000 7fd9adfcf740 1 -- 172.21.15.97:0/2122964191 shutdown_connections 2024-04-23T10:34:50.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.213+0000 7fd9adfcf740 1 --2- 172.21.15.97:0/2122964191 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x558cdea9ef40 0x558cdeaa1360 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:50.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.213+0000 7fd9adfcf740 1 --2- 172.21.15.97:0/2122964191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558cdea23090 0x558cdea1c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:50.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.213+0000 7fd98cff9640 0 client.16797 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:34:50.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.213+0000 7fd9adfcf740 1 --2- 172.21.15.97:0/2122964191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558cdea189c0 0x558cdea1c270 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:50.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.213+0000 7fd9adfcf740 1 --2- 172.21.15.97:0/2122964191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558cdea180a0 0x558cdea1d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:50.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.213+0000 7fd9adfcf740 1 -- 172.21.15.97:0/2122964191 >> 172.21.15.97:0/2122964191 conn(0x558cdea143c0 msgr2=0x558cdea1ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:34:50.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.213+0000 7fd9adfcf740 1 -- 172.21.15.97:0/2122964191 shutdown_connections 2024-04-23T10:34:50.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.213+0000 7fd9adfcf740 1 -- 172.21.15.97:0/2122964191 wait complete. 2024-04-23T10:34:50.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.213+0000 7fd9adfcf740 20 client.16797 trim_cache size 0 max 16384 2024-04-23T10:34:50.213 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-23T10:34:50.213 INFO:teuthology.orchestra.run.smithi097.stdout: "epoch": 26, 2024-04-23T10:34:50.213 INFO:teuthology.orchestra.run.smithi097.stdout: "leader": 24488, 2024-04-23T10:34:50.213 INFO:teuthology.orchestra.run.smithi097.stdout: "set_version": 34, 2024-04-23T10:34:50.213 INFO:teuthology.orchestra.run.smithi097.stdout: "sets": { 2024-04-23T10:34:50.213 INFO:teuthology.orchestra.run.smithi097.stdout: "c24eb2b3": { 2024-04-23T10:34:50.213 INFO:teuthology.orchestra.run.smithi097.stdout: "version": 34, 2024-04-23T10:34:50.213 INFO:teuthology.orchestra.run.smithi097.stdout: "age_ref": 0.0, 2024-04-23T10:34:50.213 INFO:teuthology.orchestra.run.smithi097.stdout: "state": { 2024-04-23T10:34:50.213 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "QUIESCED", 2024-04-23T10:34:50.214 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0 2024-04-23T10:34:50.214 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:34:50.214 INFO:teuthology.orchestra.run.smithi097.stdout: "timeout": 60.0, 2024-04-23T10:34:50.214 INFO:teuthology.orchestra.run.smithi097.stdout: "expiration": 70.0, 2024-04-23T10:34:50.214 INFO:teuthology.orchestra.run.smithi097.stdout: "members": { 2024-04-23T10:34:50.214 INFO:teuthology.orchestra.run.smithi097.stdout: "file:/": { 2024-04-23T10:34:50.214 INFO:teuthology.orchestra.run.smithi097.stdout: "excluded": false, 2024-04-23T10:34:50.214 INFO:teuthology.orchestra.run.smithi097.stdout: "state": { 2024-04-23T10:34:50.214 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "QUIESCED", 2024-04-23T10:34:50.214 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0 2024-04-23T10:34:50.214 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:34:50.214 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:34:50.214 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:34:50.214 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:34:50.214 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:34:50.214 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-23T10:34:50.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.213+0000 7fd9ab2bb640 1 -- 172.21.15.97:0/3351785993 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fd9900a3d20 msgr2=0x7fd9900a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:50.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.213+0000 7fd9ab2bb640 1 --2- 172.21.15.97:0/3351785993 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fd9900a3d20 0x7fd9900a61e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd99c005cc0 tx=0x7fd99c02f3a0 comp rx=0 tx=0).stop 2024-04-23T10:34:50.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.213+0000 7fd9ab2bb640 1 -- 172.21.15.97:0/3351785993 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9a4101090 msgr2=0x7fd9a419e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:50.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.213+0000 7fd9ab2bb640 1 --2- 172.21.15.97:0/3351785993 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9a4101090 0x7fd9a419e440 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fd9a000d770 tx=0x7fd9a000dc30 comp rx=0 tx=0).stop 2024-04-23T10:34:50.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.213+0000 7fd9ab2bb640 1 -- 172.21.15.97:0/3351785993 shutdown_connections 2024-04-23T10:34:50.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.213+0000 7fd9ab2bb640 1 --2- 172.21.15.97:0/3351785993 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fd9900a3d20 0x7fd9900a61e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:50.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.213+0000 7fd9ab2bb640 1 --2- 172.21.15.97:0/3351785993 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9a4198640 0x7fd9a4198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:50.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.213+0000 7fd9ab2bb640 1 --2- 172.21.15.97:0/3351785993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9a4106160 0x7fd9a419e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:50.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.213+0000 7fd9ab2bb640 1 --2- 172.21.15.97:0/3351785993 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd9a4101090 0x7fd9a419e440 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:50.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.213+0000 7fd9ab2bb640 1 -- 172.21.15.97:0/3351785993 >> 172.21.15.97:0/3351785993 conn(0x7fd9a40771d0 msgr2=0x7fd9a4104e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:34:50.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.214+0000 7fd9ab2bb640 1 -- 172.21.15.97:0/3351785993 shutdown_connections 2024-04-23T10:34:50.216 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:34:50.214+0000 7fd9ab2bb640 1 -- 172.21.15.97:0/3351785993 wait complete. 2024-04-23T10:34:50.223 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'c24eb2b3', quiesce took 0.5 seconds. Will release after: 10 2024-04-23T10:34:50.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:34:50 smithi196 ceph-mon[32919]: pgmap v266: 97 pgs: 97 active+clean; 3.8 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-23T10:34:50.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:34:50 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/4098857036' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:34:50.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:34:50 smithi097 ceph-mon[29197]: pgmap v266: 97 pgs: 97 active+clean; 3.8 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-23T10:34:50.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:34:50 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/4098857036' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:34:50.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:34:50 smithi183 ceph-mon[32317]: pgmap v266: 97 pgs: 97 active+clean; 3.8 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-23T10:34:50.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:34:50 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/4098857036' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:34:52.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:34:52 smithi196 ceph-mon[32919]: pgmap v267: 97 pgs: 97 active+clean; 3.8 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-23T10:34:52.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:34:52 smithi097 ceph-mon[29197]: pgmap v267: 97 pgs: 97 active+clean; 3.8 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-23T10:34:52.649 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:34:52 smithi183 ceph-mon[32317]: pgmap v267: 97 pgs: 97 active+clean; 3.8 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-23T10:34:54.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:34:54 smithi196 ceph-mon[32919]: pgmap v268: 97 pgs: 97 active+clean; 3.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 48 KiB/s wr, 21 op/s 2024-04-23T10:34:54.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:34:54 smithi097 ceph-mon[29197]: pgmap v268: 97 pgs: 97 active+clean; 3.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 48 KiB/s wr, 21 op/s 2024-04-23T10:34:54.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:34:54 smithi183 ceph-mon[32317]: pgmap v268: 97 pgs: 97 active+clean; 3.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 48 KiB/s wr, 21 op/s 2024-04-23T10:34:56.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:34:56 smithi196 ceph-mon[32919]: pgmap v269: 97 pgs: 97 active+clean; 3.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 34 KiB/s wr, 14 op/s 2024-04-23T10:34:56.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:34:56 smithi097 ceph-mon[29197]: pgmap v269: 97 pgs: 97 active+clean; 3.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 34 KiB/s wr, 14 op/s 2024-04-23T10:34:56.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:34:56 smithi183 ceph-mon[32317]: pgmap v269: 97 pgs: 97 active+clean; 3.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 34 KiB/s wr, 14 op/s 2024-04-23T10:34:58.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:34:58 smithi196 ceph-mon[32919]: pgmap v270: 97 pgs: 97 active+clean; 3.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 37 KiB/s wr, 19 op/s 2024-04-23T10:34:58.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:34:58 smithi097 ceph-mon[29197]: pgmap v270: 97 pgs: 97 active+clean; 3.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 37 KiB/s wr, 19 op/s 2024-04-23T10:34:58.649 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:34:58 smithi183 ceph-mon[32317]: pgmap v270: 97 pgs: 97 active+clean; 3.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 37 KiB/s wr, 19 op/s 2024-04-23T10:34:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.404+0000 7fca5863c640 1 -- 172.21.15.97:0/460844947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca50103200 msgr2=0x7fca501035e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.404+0000 7fca5863c640 1 --2- 172.21.15.97:0/460844947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca50103200 0x7fca501035e0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fca4c009ca0 tx=0x7fca4c02a950 comp rx=0 tx=0).stop 2024-04-23T10:34:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.404+0000 7fca5863c640 1 -- 172.21.15.97:0/460844947 shutdown_connections 2024-04-23T10:34:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.404+0000 7fca5863c640 1 --2- 172.21.15.97:0/460844947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca5010c5d0 0x7fca5010ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.404+0000 7fca5863c640 1 --2- 172.21.15.97:0/460844947 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca50103b20 0x7fca5010c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.404+0000 7fca5863c640 1 --2- 172.21.15.97:0/460844947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca50103200 0x7fca501035e0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.404+0000 7fca5863c640 1 -- 172.21.15.97:0/460844947 >> 172.21.15.97:0/460844947 conn(0x7fca500771d0 msgr2=0x7fca500775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:34:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.404+0000 7fca5863c640 1 -- 172.21.15.97:0/460844947 shutdown_connections 2024-04-23T10:34:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.404+0000 7fca5863c640 1 -- 172.21.15.97:0/460844947 wait complete. 2024-04-23T10:34:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.404+0000 7fca5863c640 1 Processor -- start 2024-04-23T10:34:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.404+0000 7fca5863c640 1 -- start start 2024-04-23T10:34:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.405+0000 7fca5863c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca50103200 0x7fca50198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.405+0000 7fca5863c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca50103b20 0x7fca50198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.405+0000 7fca5863c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca5010c5d0 0x7fca5019cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.405+0000 7fca5863c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fca50112220 con 0x7fca5010c5d0 2024-04-23T10:34:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.405+0000 7fca5863c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fca501120a0 con 0x7fca50103200 2024-04-23T10:34:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.405+0000 7fca5863c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fca501123a0 con 0x7fca50103b20 2024-04-23T10:34:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.405+0000 7fca56bd9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca5010c5d0 0x7fca5019cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.405+0000 7fca56bd9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca5010c5d0 0x7fca5019cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33454/0 (socket says 172.21.15.97:33454) 2024-04-23T10:34:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.405+0000 7fca56bd9640 1 -- 172.21.15.97:0/1781995290 learned_addr learned my addr 172.21.15.97:0/1781995290 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:34:59.407 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.405+0000 7fca563d8640 1 --2- 172.21.15.97:0/1781995290 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca50103200 0x7fca50198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:59.407 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.405+0000 7fca55bd7640 1 --2- 172.21.15.97:0/1781995290 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca50103b20 0x7fca50198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:59.407 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.405+0000 7fca56bd9640 1 -- 172.21.15.97:0/1781995290 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca50103b20 msgr2=0x7fca50198ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:59.407 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.405+0000 7fca56bd9640 1 --2- 172.21.15.97:0/1781995290 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca50103b20 0x7fca50198ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:59.407 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.405+0000 7fca56bd9640 1 -- 172.21.15.97:0/1781995290 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca50103200 msgr2=0x7fca50198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:59.407 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.405+0000 7fca56bd9640 1 --2- 172.21.15.97:0/1781995290 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca50103200 0x7fca50198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:59.407 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.405+0000 7fca56bd9640 1 -- 172.21.15.97:0/1781995290 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca5019d650 con 0x7fca5010c5d0 2024-04-23T10:34:59.407 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.405+0000 7fca563d8640 1 --2- 172.21.15.97:0/1781995290 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca50103200 0x7fca50198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T10:34:59.407 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.405+0000 7fca56bd9640 1 --2- 172.21.15.97:0/1781995290 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca5010c5d0 0x7fca5019cf20 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fca4800e970 tx=0x7fca4800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:34:59.407 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.405+0000 7fca477fe640 1 -- 172.21.15.97:0/1781995290 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca4800eea0 con 0x7fca5010c5d0 2024-04-23T10:34:59.407 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.406+0000 7fca5863c640 1 -- 172.21.15.97:0/1781995290 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca5019d940 con 0x7fca5010c5d0 2024-04-23T10:34:59.407 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.406+0000 7fca477fe640 1 -- 172.21.15.97:0/1781995290 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca480054d0 con 0x7fca5010c5d0 2024-04-23T10:34:59.407 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.406+0000 7fca477fe640 1 -- 172.21.15.97:0/1781995290 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca48004e60 con 0x7fca5010c5d0 2024-04-23T10:34:59.408 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.406+0000 7fca5863c640 1 -- 172.21.15.97:0/1781995290 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca501a51d0 con 0x7fca5010c5d0 2024-04-23T10:34:59.414 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.408+0000 7fca477fe640 1 -- 172.21.15.97:0/1781995290 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fca48005020 con 0x7fca5010c5d0 2024-04-23T10:34:59.414 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.408+0000 7fca5b432740 1 Processor -- start 2024-04-23T10:34:59.414 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.409+0000 7fca5b432740 1 -- start start 2024-04-23T10:34:59.414 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.409+0000 7fca5b432740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563e55acb520 0x563e55acb900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:59.414 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.409+0000 7fca5b432740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e55acbe40 0x563e55ada6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:59.414 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.409+0000 7fca5b432740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e55adac10 0x563e55add000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:59.414 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.409+0000 7fca5b432740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x563e55a30970 con 0x563e55acb520 2024-04-23T10:34:59.414 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.409+0000 7fca5b432740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563e559cd4a0 con 0x563e55adac10 2024-04-23T10:34:59.414 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.409+0000 7fca5b432740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563e55a2c930 con 0x563e55acbe40 2024-04-23T10:34:59.414 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.409+0000 7fca477fe640 1 --2- 172.21.15.97:0/1781995290 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fca3c0a3dc0 0x7fca3c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:59.414 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.409+0000 7fca55bd7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563e55acb520 0x563e55acb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:59.414 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.409+0000 7fca55bd7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563e55acb520 0x563e55acb900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33462/0 (socket says 172.21.15.97:33462) 2024-04-23T10:34:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.409+0000 7fca55bd7640 1 -- 172.21.15.97:0/3355058774 learned_addr learned my addr 172.21.15.97:0/3355058774 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:34:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.409+0000 7fca56bd9640 1 --2- 172.21.15.97:0/1781995290 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fca3c0a3dc0 0x7fca3c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.409+0000 7fca563d8640 1 --2- 172.21.15.97:0/3355058774 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e55acbe40 0x563e55ada6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.409+0000 7fca55bd7640 1 --2- 172.21.15.97:0/3355058774 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e55adac10 0x563e55add000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.409+0000 7fca477fe640 1 -- 172.21.15.97:0/1781995290 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7fca480d0a20 con 0x7fca5010c5d0 2024-04-23T10:34:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.409+0000 7fca55bd7640 1 -- 172.21.15.97:0/3355058774 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e55acbe40 msgr2=0x563e55ada6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:34:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.409+0000 7fca55bd7640 1 --2- 172.21.15.97:0/3355058774 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e55acbe40 0x563e55ada6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.409+0000 7fca55bd7640 1 -- 172.21.15.97:0/3355058774 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e55adac10 msgr2=0x563e55add000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:34:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.409+0000 7fca55bd7640 1 --2- 172.21.15.97:0/3355058774 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e55adac10 0x563e55add000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.409+0000 7fca55bd7640 1 -- 172.21.15.97:0/3355058774 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e55743210 con 0x563e55acb520 2024-04-23T10:34:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.410+0000 7fca55bd7640 1 --2- 172.21.15.97:0/3355058774 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563e55acb520 0x563e55acb900 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fca40013610 tx=0x7fca4000a800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7650e04f962b4716 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:34:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.410+0000 7fca56bd9640 1 --2- 172.21.15.97:0/1781995290 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fca3c0a3dc0 0x7fca3c0a6280 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fca480ce240 tx=0x7fca4806f4d0 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:34:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.410+0000 7fca44bfe640 1 -- 172.21.15.97:0/3355058774 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca40019030 con 0x563e55acb520 2024-04-23T10:34:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.410+0000 7fca44bfe640 1 -- 172.21.15.97:0/3355058774 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca400110d0 con 0x563e55acb520 2024-04-23T10:34:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.410+0000 7fca44bfe640 1 -- 172.21.15.97:0/3355058774 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca4000f730 con 0x563e55acb520 2024-04-23T10:34:59.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.410+0000 7fca5b432740 1 -- 172.21.15.97:0/3355058774 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563e55acb520 msgr2=0x563e55acb900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:59.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.410+0000 7fca5b432740 1 --2- 172.21.15.97:0/3355058774 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563e55acb520 0x563e55acb900 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fca40013610 tx=0x7fca4000a800 comp rx=0 tx=0).stop 2024-04-23T10:34:59.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.411+0000 7fca5b432740 1 -- 172.21.15.97:0/3355058774 shutdown_connections 2024-04-23T10:34:59.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.411+0000 7fca5b432740 1 --2- 172.21.15.97:0/3355058774 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e55adac10 0x563e55add000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:59.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.411+0000 7fca5b432740 1 --2- 172.21.15.97:0/3355058774 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e55acbe40 0x563e55ada6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:59.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.411+0000 7fca5b432740 1 --2- 172.21.15.97:0/3355058774 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563e55acb520 0x563e55acb900 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:59.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.411+0000 7fca5b432740 1 -- 172.21.15.97:0/3355058774 >> 172.21.15.97:0/3355058774 conn(0x563e55ac7b30 msgr2=0x563e55ac7f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:34:59.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.411+0000 7fca5b432740 1 -- 172.21.15.97:0/3355058774 shutdown_connections 2024-04-23T10:34:59.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.411+0000 7fca5b432740 1 -- 172.21.15.97:0/3355058774 wait complete. 2024-04-23T10:34:59.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.411+0000 7fca5b432740 1 -- start start 2024-04-23T10:34:59.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.411+0000 7fca5b432740 1 Processor -- start 2024-04-23T10:34:59.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.411+0000 7fca5b432740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563e55acb520 0x563e55ada470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:59.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.411+0000 7fca5b432740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e55acbe40 0x563e55ad7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:59.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.411+0000 7fca5b432740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e55adac10 0x563e55ad7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:59.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.411+0000 7fca5b432740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x563e55add840 con 0x563e55acb520 2024-04-23T10:34:59.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.411+0000 7fca5b432740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563e559cb0b0 con 0x563e55acbe40 2024-04-23T10:34:59.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.411+0000 7fca5b432740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563e55add9c0 con 0x563e55adac10 2024-04-23T10:34:59.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.411+0000 7fca5b432740 10 client.0 _ll_register_callbacks cb 0x563e559b8d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:34:59.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.411+0000 7fca55bd7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563e55acb520 0x563e55ada470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:59.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.411+0000 7fca55bd7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563e55acb520 0x563e55ada470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33472/0 (socket says 172.21.15.97:33472) 2024-04-23T10:34:59.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.411+0000 7fca55bd7640 1 -- 172.21.15.97:0/833075133 learned_addr learned my addr 172.21.15.97:0/833075133 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:34:59.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.412+0000 7fca55bd7640 1 --2- 172.21.15.97:0/833075133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e55adac10 0x563e55ad7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:59.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.412+0000 7fca55bd7640 1 -- 172.21.15.97:0/833075133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e55adac10 msgr2=0x563e55ad7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:34:59.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.412+0000 7fca55bd7640 1 --2- 172.21.15.97:0/833075133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e55adac10 0x563e55ad7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:59.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.412+0000 7fca55bd7640 1 -- 172.21.15.97:0/833075133 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e55acbe40 msgr2=0x563e55ad7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:34:59.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.412+0000 7fca563d8640 1 --2- 172.21.15.97:0/833075133 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e55acbe40 0x563e55ad7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:59.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.412+0000 7fca55bd7640 1 --2- 172.21.15.97:0/833075133 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e55acbe40 0x563e55ad7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:59.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.412+0000 7fca55bd7640 1 -- 172.21.15.97:0/833075133 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e5580f620 con 0x563e55acb520 2024-04-23T10:34:59.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.412+0000 7fca55bd7640 1 --2- 172.21.15.97:0/833075133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563e55acb520 0x563e55ada470 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fca40005bd0 tx=0x7fca40011280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2091b21ff86f13a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:34:59.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.412+0000 7fca39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:34:59.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.412+0000 7fca39ffb640 1 -- 172.21.15.97:0/833075133 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca40019070 con 0x563e55acb520 2024-04-23T10:34:59.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.412+0000 7fca39ffb640 1 -- 172.21.15.97:0/833075133 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7fca40005450 con 0x563e55acb520 2024-04-23T10:34:59.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.412+0000 7fca5b432740 1 -- 172.21.15.97:0/833075133 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca400110d0 con 0x563e55acb520 2024-04-23T10:34:59.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.412+0000 7fca39ffb640 1 -- 172.21.15.97:0/833075133 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fca40005650 con 0x563e55acb520 2024-04-23T10:34:59.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.412+0000 7fca5b432740 10 client.16809 fetch_fsmap learned FSMap version 26 2024-04-23T10:34:59.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.412+0000 7fca5b432740 1 -- 172.21.15.97:0/833075133 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563e55ae9710 con 0x563e55acb520 2024-04-23T10:34:59.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.412+0000 7fca39ffb640 1 -- 172.21.15.97:0/833075133 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca400110d0 con 0x563e55acb520 2024-04-23T10:34:59.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.412+0000 7fca39ffb640 1 -- 172.21.15.97:0/833075133 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fca40031050 con 0x563e55acb520 2024-04-23T10:34:59.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.413+0000 7fca5b432740 10 client.16809 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:34:59.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.413+0000 7fca5b432740 10 client.16809 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:34:59.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.413+0000 7fca5b432740 20 client.16809 populate_metadata read hostname 'smithi097' 2024-04-23T10:34:59.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.413+0000 7fca5b432740 1 --2- 172.21.15.97:0/833075133 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x563e55aeaac0 0x563e55aeceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:59.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.413+0000 7fca5b432740 4 client.16809 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:34:59.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.413+0000 7fca5b432740 1 -- 172.21.15.97:0/833075133 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e55a3e390 con 0x563e55aeaac0 2024-04-23T10:34:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.413+0000 7fca563d8640 1 --2- 172.21.15.97:0/833075133 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x563e55aeaac0 0x563e55aeceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.413+0000 7fca39ffb640 10 client.16809 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:34:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.413+0000 7fca563d8640 1 --2- 172.21.15.97:0/833075133 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x563e55aeaac0 0x563e55aeceb0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3621e8c77fec1afe server_cookie=eea94ca59f2f2fd in_seq=0 out_seq=0 2024-04-23T10:34:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.414+0000 7fca39ffb640 1 -- 172.21.15.97:0/833075133 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563e55a3e390 con 0x563e55aeaac0 2024-04-23T10:34:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.414+0000 7fca39ffb640 10 client.16809 handle_command_reply: tid=0 2024-04-23T10:34:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.414+0000 7fca5b432740 1 client.16809 shutdown 2024-04-23T10:34:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.414+0000 7fca5b432740 1 -- 172.21.15.97:0/833075133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563e55acb520 msgr2=0x563e55ada470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.414+0000 7fca5b432740 1 --2- 172.21.15.97:0/833075133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563e55acb520 0x563e55ada470 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fca40005bd0 tx=0x7fca40011280 comp rx=0 tx=0).stop 2024-04-23T10:34:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.414+0000 7fca5b432740 1 -- 172.21.15.97:0/833075133 shutdown_connections 2024-04-23T10:34:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.414+0000 7fca5b432740 1 --2- 172.21.15.97:0/833075133 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x563e55aeaac0 0x563e55aeceb0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.414+0000 7fca5b432740 1 --2- 172.21.15.97:0/833075133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e55adac10 0x563e55ad7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.414+0000 7fca39ffb640 0 client.16809 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:34:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.414+0000 7fca5b432740 1 --2- 172.21.15.97:0/833075133 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e55acbe40 0x563e55ad7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.414+0000 7fca5b432740 1 --2- 172.21.15.97:0/833075133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563e55acb520 0x563e55ada470 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.414+0000 7fca5b432740 1 -- 172.21.15.97:0/833075133 >> 172.21.15.97:0/833075133 conn(0x563e55ac7b30 msgr2=0x563e55acd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:34:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.414+0000 7fca5b432740 1 -- 172.21.15.97:0/833075133 shutdown_connections 2024-04-23T10:34:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.414+0000 7fca5b432740 1 -- 172.21.15.97:0/833075133 wait complete. 2024-04-23T10:34:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.414+0000 7fca5b432740 20 client.16809 trim_cache size 0 max 16384 2024-04-23T10:34:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.429+0000 7fca5b432740 1 Processor -- start 2024-04-23T10:34:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.429+0000 7fca5b432740 1 -- start start 2024-04-23T10:34:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.429+0000 7fca5b432740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e55acb520 0x563e55ae6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.429+0000 7fca5b432740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563e55acbe40 0x563e55ad2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.429+0000 7fca5b432740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e55adac10 0x563e55ad9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.429+0000 7fca5b432740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fca501ab910 con 0x563e55acbe40 2024-04-23T10:34:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.429+0000 7fca5b432740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fca501ad450 con 0x563e55acb520 2024-04-23T10:34:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.429+0000 7fca5b432740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fca501abfe0 con 0x563e55adac10 2024-04-23T10:34:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.429+0000 7fca563d8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563e55acbe40 0x563e55ad2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.430+0000 7fca563d8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563e55acbe40 0x563e55ad2d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33484/0 (socket says 172.21.15.97:33484) 2024-04-23T10:34:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.430+0000 7fca563d8640 1 -- 172.21.15.97:0/3232195993 learned_addr learned my addr 172.21.15.97:0/3232195993 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:34:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.430+0000 7fca55bd7640 1 --2- 172.21.15.97:0/3232195993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e55adac10 0x563e55ad9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.430+0000 7fca55bd7640 1 --2- 172.21.15.97:0/3232195993 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e55acb520 0x563e55ae6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.430+0000 7fca563d8640 1 -- 172.21.15.97:0/3232195993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e55adac10 msgr2=0x563e55ad9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:34:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.430+0000 7fca563d8640 1 --2- 172.21.15.97:0/3232195993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e55adac10 0x563e55ad9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.430+0000 7fca563d8640 1 -- 172.21.15.97:0/3232195993 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e55acb520 msgr2=0x563e55ae6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:34:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.430+0000 7fca563d8640 1 --2- 172.21.15.97:0/3232195993 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e55acb520 0x563e55ae6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.430+0000 7fca563d8640 1 -- 172.21.15.97:0/3232195993 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e55ac7960 con 0x563e55acbe40 2024-04-23T10:34:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.430+0000 7fca55bd7640 1 --2- 172.21.15.97:0/3232195993 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e55acb520 0x563e55ae6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:34:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.430+0000 7fca563d8640 1 --2- 172.21.15.97:0/3232195993 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563e55acbe40 0x563e55ad2d80 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fca4c009ca0 tx=0x7fca4c009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d945eaa6954c76fc server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:34:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.430+0000 7fca3bfff640 1 -- 172.21.15.97:0/3232195993 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca4c034040 con 0x563e55acbe40 2024-04-23T10:34:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.430+0000 7fca3bfff640 1 -- 172.21.15.97:0/3232195993 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca4c04c070 con 0x563e55acbe40 2024-04-23T10:34:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.430+0000 7fca3bfff640 1 -- 172.21.15.97:0/3232195993 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca4c004040 con 0x563e55acbe40 2024-04-23T10:34:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.431+0000 7fca5b432740 1 -- 172.21.15.97:0/3232195993 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563e55acbe40 msgr2=0x563e55ad2d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.431+0000 7fca5b432740 1 --2- 172.21.15.97:0/3232195993 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563e55acbe40 0x563e55ad2d80 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fca4c009ca0 tx=0x7fca4c009360 comp rx=0 tx=0).stop 2024-04-23T10:34:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.431+0000 7fca5b432740 1 -- 172.21.15.97:0/3232195993 shutdown_connections 2024-04-23T10:34:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.431+0000 7fca5b432740 1 --2- 172.21.15.97:0/3232195993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e55adac10 0x563e55ad9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.431+0000 7fca5b432740 1 --2- 172.21.15.97:0/3232195993 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563e55acbe40 0x563e55ad2d80 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.431+0000 7fca5b432740 1 --2- 172.21.15.97:0/3232195993 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e55acb520 0x563e55ae6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.431+0000 7fca5b432740 1 -- 172.21.15.97:0/3232195993 >> 172.21.15.97:0/3232195993 conn(0x563e55ac7b30 msgr2=0x563e55ad2670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:34:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.431+0000 7fca5b432740 1 -- 172.21.15.97:0/3232195993 shutdown_connections 2024-04-23T10:34:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.431+0000 7fca5b432740 1 -- 172.21.15.97:0/3232195993 wait complete. 2024-04-23T10:34:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.431+0000 7fca5b432740 1 -- start start 2024-04-23T10:34:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.431+0000 7fca5b432740 1 Processor -- start 2024-04-23T10:34:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.431+0000 7fca5b432740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563e55acb520 0x563e55ae2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.431+0000 7fca5b432740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e55acbe40 0x563e55ade260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.431+0000 7fca5b432740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e55adac10 0x563e55ade7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.431+0000 7fca5b432740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fca501acd80 con 0x563e55acb520 2024-04-23T10:34:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.431+0000 7fca5b432740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fca501ac6b0 con 0x563e55adac10 2024-04-23T10:34:59.436 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.431+0000 7fca5b432740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fca501a9930 con 0x563e55acbe40 2024-04-23T10:34:59.436 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.431+0000 7fca5b432740 10 client.0 _ll_register_callbacks cb 0x7fca40011c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:34:59.436 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.431+0000 7fca55bd7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563e55acb520 0x563e55ae2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:59.436 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.431+0000 7fca55bd7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563e55acb520 0x563e55ae2ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33500/0 (socket says 172.21.15.97:33500) 2024-04-23T10:34:59.436 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.432+0000 7fca55bd7640 1 -- 172.21.15.97:0/2527695030 learned_addr learned my addr 172.21.15.97:0/2527695030 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:34:59.436 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.432+0000 7fca563d8640 1 --2- 172.21.15.97:0/2527695030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e55acbe40 0x563e55ade260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:59.436 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.432+0000 7fca55bd7640 1 --2- 172.21.15.97:0/2527695030 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e55adac10 0x563e55ade7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:59.436 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.432+0000 7fca55bd7640 1 -- 172.21.15.97:0/2527695030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e55acbe40 msgr2=0x563e55ade260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:34:59.436 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.432+0000 7fca55bd7640 1 --2- 172.21.15.97:0/2527695030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e55acbe40 0x563e55ade260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:59.436 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.432+0000 7fca55bd7640 1 -- 172.21.15.97:0/2527695030 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e55adac10 msgr2=0x563e55ade7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:34:59.436 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.432+0000 7fca55bd7640 1 --2- 172.21.15.97:0/2527695030 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e55adac10 0x563e55ade7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:59.436 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.432+0000 7fca55bd7640 1 -- 172.21.15.97:0/2527695030 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e55adede0 con 0x563e55acb520 2024-04-23T10:34:59.436 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.432+0000 7fca55bd7640 1 --2- 172.21.15.97:0/2527695030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563e55acb520 0x563e55ae2ea0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fca40011bf0 tx=0x7fca40011280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c69a3cfc55cdf10c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:34:59.436 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.432+0000 7fca39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:34:59.436 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.432+0000 7fca39ffb640 1 -- 172.21.15.97:0/2527695030 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca40019040 con 0x563e55acb520 2024-04-23T10:34:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.432+0000 7fca39ffb640 1 -- 172.21.15.97:0/2527695030 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7fca40005450 con 0x563e55acb520 2024-04-23T10:34:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.432+0000 7fca5b432740 1 -- 172.21.15.97:0/2527695030 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca4c04c070 con 0x563e55acb520 2024-04-23T10:34:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.432+0000 7fca39ffb640 1 -- 172.21.15.97:0/2527695030 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fca4000ae70 con 0x563e55acb520 2024-04-23T10:34:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.432+0000 7fca5b432740 10 client.16815 fetch_fsmap learned FSMap version 26 2024-04-23T10:34:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.432+0000 7fca5b432740 1 -- 172.21.15.97:0/2527695030 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563e55b50c80 con 0x563e55acb520 2024-04-23T10:34:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.432+0000 7fca39ffb640 1 -- 172.21.15.97:0/2527695030 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca4c04c070 con 0x563e55acb520 2024-04-23T10:34:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.433+0000 7fca39ffb640 1 -- 172.21.15.97:0/2527695030 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fca4003c070 con 0x563e55acb520 2024-04-23T10:34:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.433+0000 7fca5b432740 10 client.16815 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:34:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.433+0000 7fca5b432740 10 client.16815 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:34:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.433+0000 7fca5b432740 20 client.16815 populate_metadata read hostname 'smithi097' 2024-04-23T10:34:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.433+0000 7fca5b432740 1 --2- 172.21.15.97:0/2527695030 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x563e55b51370 0x563e55b53790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:34:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.433+0000 7fca5b432740 4 client.16815 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:34:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.433+0000 7fca5b432740 1 -- 172.21.15.97:0/2527695030 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563e55ad8030 con 0x563e55b51370 2024-04-23T10:34:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.433+0000 7fca563d8640 1 --2- 172.21.15.97:0/2527695030 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x563e55b51370 0x563e55b53790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:34:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.433+0000 7fca39ffb640 10 client.16815 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:34:59.438 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.433+0000 7fca563d8640 1 --2- 172.21.15.97:0/2527695030 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x563e55b51370 0x563e55b53790 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b25ac9f803bdee1 server_cookie=932db302edd36f6 in_seq=0 out_seq=0 2024-04-23T10:34:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.448+0000 7fca39ffb640 1 -- 172.21.15.97:0/2527695030 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563e55ad8030 con 0x563e55b51370 2024-04-23T10:34:59.447 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.448+0000 7fca39ffb640 10 client.16815 handle_command_reply: tid=0 2024-04-23T10:34:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.448+0000 7fca5b432740 1 client.16815 shutdown 2024-04-23T10:34:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.448+0000 7fca5b432740 1 -- 172.21.15.97:0/2527695030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563e55acb520 msgr2=0x563e55ae2ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.448+0000 7fca5b432740 1 --2- 172.21.15.97:0/2527695030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563e55acb520 0x563e55ae2ea0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fca40011bf0 tx=0x7fca40011280 comp rx=0 tx=0).stop 2024-04-23T10:34:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.448+0000 7fca5b432740 1 -- 172.21.15.97:0/2527695030 shutdown_connections 2024-04-23T10:34:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.448+0000 7fca5b432740 1 --2- 172.21.15.97:0/2527695030 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x563e55b51370 0x563e55b53790 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.448+0000 7fca5b432740 1 --2- 172.21.15.97:0/2527695030 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e55adac10 0x563e55ade7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.448+0000 7fca39ffb640 0 client.16815 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:34:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.448+0000 7fca5b432740 1 --2- 172.21.15.97:0/2527695030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563e55acbe40 0x563e55ade260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.448+0000 7fca5b432740 1 --2- 172.21.15.97:0/2527695030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563e55acb520 0x563e55ae2ea0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.448+0000 7fca5b432740 1 -- 172.21.15.97:0/2527695030 >> 172.21.15.97:0/2527695030 conn(0x563e55ac7b30 msgr2=0x563e55acd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:34:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.448+0000 7fca5b432740 1 -- 172.21.15.97:0/2527695030 shutdown_connections 2024-04-23T10:34:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.448+0000 7fca5b432740 1 -- 172.21.15.97:0/2527695030 wait complete. 2024-04-23T10:34:59.448 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.448+0000 7fca5b432740 20 client.16815 trim_cache size 0 max 16384 2024-04-23T10:34:59.449 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:34:59.449 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:34:59.449 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:34:59.449 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:34:59.449 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.449+0000 7fca5863c640 1 -- 172.21.15.97:0/1781995290 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fca3c0a3dc0 msgr2=0x7fca3c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:59.449 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.449+0000 7fca5863c640 1 --2- 172.21.15.97:0/1781995290 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fca3c0a3dc0 0x7fca3c0a6280 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fca480ce240 tx=0x7fca4806f4d0 comp rx=0 tx=0).stop 2024-04-23T10:34:59.450 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.449+0000 7fca5863c640 1 -- 172.21.15.97:0/1781995290 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca5010c5d0 msgr2=0x7fca5019cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:34:59.450 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.449+0000 7fca5863c640 1 --2- 172.21.15.97:0/1781995290 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca5010c5d0 0x7fca5019cf20 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fca4800e970 tx=0x7fca4800ed70 comp rx=0 tx=0).stop 2024-04-23T10:34:59.450 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.449+0000 7fca5863c640 1 -- 172.21.15.97:0/1781995290 shutdown_connections 2024-04-23T10:34:59.450 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.449+0000 7fca5863c640 1 --2- 172.21.15.97:0/1781995290 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fca3c0a3dc0 0x7fca3c0a6280 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:59.450 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.449+0000 7fca5863c640 1 --2- 172.21.15.97:0/1781995290 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fca5010c5d0 0x7fca5019cf20 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:59.450 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.449+0000 7fca5863c640 1 --2- 172.21.15.97:0/1781995290 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca50103b20 0x7fca50198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:59.450 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.449+0000 7fca5863c640 1 --2- 172.21.15.97:0/1781995290 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fca50103200 0x7fca50198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:34:59.450 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.449+0000 7fca5863c640 1 -- 172.21.15.97:0/1781995290 >> 172.21.15.97:0/1781995290 conn(0x7fca500771d0 msgr2=0x7fca500fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:34:59.450 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.449+0000 7fca5863c640 1 -- 172.21.15.97:0/1781995290 shutdown_connections 2024-04-23T10:34:59.450 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:34:59.449+0000 7fca5863c640 1 -- 172.21.15.97:0/1781995290 wait complete. 2024-04-23T10:35:00.224 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T10:35:00.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.336+0000 7f004e405640 1 -- 172.21.15.97:0/1539032233 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00481031b0 msgr2=0x7f0048103590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:00.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.336+0000 7f004e405640 1 --2- 172.21.15.97:0/1539032233 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00481031b0 0x7f0048103590 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f0038009900 tx=0x7f003802a480 comp rx=0 tx=0).stop 2024-04-23T10:35:00.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.336+0000 7f004e405640 1 -- 172.21.15.97:0/1539032233 shutdown_connections 2024-04-23T10:35:00.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.336+0000 7f004e405640 1 --2- 172.21.15.97:0/1539032233 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f004810c580 0x7f004810e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:00.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.336+0000 7f004e405640 1 --2- 172.21.15.97:0/1539032233 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0048103ad0 0x7f004810bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:00.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.336+0000 7f004e405640 1 --2- 172.21.15.97:0/1539032233 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00481031b0 0x7f0048103590 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:00.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.336+0000 7f004e405640 1 -- 172.21.15.97:0/1539032233 >> 172.21.15.97:0/1539032233 conn(0x7f00480771d0 msgr2=0x7f00480775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:35:00.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.336+0000 7f004e405640 1 -- 172.21.15.97:0/1539032233 shutdown_connections 2024-04-23T10:35:00.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.336+0000 7f004e405640 1 -- 172.21.15.97:0/1539032233 wait complete. 2024-04-23T10:35:00.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.337+0000 7f004e405640 1 Processor -- start 2024-04-23T10:35:00.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.337+0000 7f004e405640 1 -- start start 2024-04-23T10:35:00.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.337+0000 7f004e405640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00481031b0 0x7f0048198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:00.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.337+0000 7f004e405640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0048103ad0 0x7f0048198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:00.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.337+0000 7f004e405640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f004810c580 0x7f004819d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:00.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.337+0000 7f004e405640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f00481121b0 con 0x7f004810c580 2024-04-23T10:35:00.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.337+0000 7f004e405640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0048112030 con 0x7f0048103ad0 2024-04-23T10:35:00.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.337+0000 7f004e405640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0048112330 con 0x7f00481031b0 2024-04-23T10:35:00.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.337+0000 7f004c9a2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f004810c580 0x7f004819d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:00.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.337+0000 7f004c9a2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f004810c580 0x7f004819d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33516/0 (socket says 172.21.15.97:33516) 2024-04-23T10:35:00.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.337+0000 7f004c9a2640 1 -- 172.21.15.97:0/2137324332 learned_addr learned my addr 172.21.15.97:0/2137324332 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:35:00.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.337+0000 7f0047fff640 1 --2- 172.21.15.97:0/2137324332 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00481031b0 0x7f0048198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:00.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.337+0000 7f00477fe640 1 --2- 172.21.15.97:0/2137324332 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0048103ad0 0x7f0048198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:00.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.337+0000 7f004c9a2640 1 -- 172.21.15.97:0/2137324332 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00481031b0 msgr2=0x7f0048198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:00.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.337+0000 7f004c9a2640 1 --2- 172.21.15.97:0/2137324332 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00481031b0 0x7f0048198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:00.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.337+0000 7f004c9a2640 1 -- 172.21.15.97:0/2137324332 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0048103ad0 msgr2=0x7f0048198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:00.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.337+0000 7f004c9a2640 1 --2- 172.21.15.97:0/2137324332 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0048103ad0 0x7f0048198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:00.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.337+0000 7f004c9a2640 1 -- 172.21.15.97:0/2137324332 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f004819d740 con 0x7f004810c580 2024-04-23T10:35:00.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.337+0000 7f0047fff640 1 --2- 172.21.15.97:0/2137324332 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00481031b0 0x7f0048198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:35:00.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.337+0000 7f00477fe640 1 --2- 172.21.15.97:0/2137324332 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0048103ad0 0x7f0048198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:35:00.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.338+0000 7f004c9a2640 1 --2- 172.21.15.97:0/2137324332 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f004810c580 0x7f004819d010 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f004000e9d0 tx=0x7f004000ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:35:00.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.338+0000 7f00457fa640 1 -- 172.21.15.97:0/2137324332 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004000cd80 con 0x7f004810c580 2024-04-23T10:35:00.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.338+0000 7f00457fa640 1 -- 172.21.15.97:0/2137324332 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0040005020 con 0x7f004810c580 2024-04-23T10:35:00.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.338+0000 7f004e405640 1 -- 172.21.15.97:0/2137324332 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f004819da30 con 0x7f004810c580 2024-04-23T10:35:00.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.338+0000 7f00457fa640 1 -- 172.21.15.97:0/2137324332 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00400116d0 con 0x7f004810c580 2024-04-23T10:35:00.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.338+0000 7f004e405640 1 -- 172.21.15.97:0/2137324332 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00481a52c0 con 0x7f004810c580 2024-04-23T10:35:00.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.339+0000 7f004e405640 1 -- 172.21.15.97:0/2137324332 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f004806add0 con 0x7f004810c580 2024-04-23T10:35:00.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.340+0000 7f00457fa640 1 -- 172.21.15.97:0/2137324332 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f00400051d0 con 0x7f004810c580 2024-04-23T10:35:00.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.341+0000 7f00457fa640 1 --2- 172.21.15.97:0/2137324332 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f00300a3d20 0x7f00300a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:00.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.341+0000 7f0047fff640 1 --2- 172.21.15.97:0/2137324332 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f00300a3d20 0x7f00300a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:00.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.341+0000 7f00457fa640 1 -- 172.21.15.97:0/2137324332 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7f00400d0a20 con 0x7f004810c580 2024-04-23T10:35:00.341 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.341+0000 7f0047fff640 1 --2- 172.21.15.97:0/2137324332 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f00300a3d20 0x7f00300a61e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0038000c00 tx=0x7f0038031400 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:35:00.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.345+0000 7f00457fa640 1 -- 172.21.15.97:0/2137324332 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0040016030 con 0x7f004810c580 2024-04-23T10:35:00.561 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.563+0000 7f004e405640 1 -- 172.21.15.97:0/2137324332 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0048199aa0 con 0x7f004810c580 2024-04-23T10:35:00.562 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.563+0000 7f00457fa640 1 -- 172.21.15.97:0/2137324332 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11931 (secure 0 0 0) 0x7f004009d0e0 con 0x7f004810c580 2024-04-23T10:35:00.562 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:35:00.562 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3946878762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3946878762},{"type":"v1","addr":"172.21.15.97:6835","nonce":3946878762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/209800861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":209800861},{"type":"v1","addr":"172.21.15.97:6837","nonce":209800861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T10:24:06.910134+0000","modified":"2024-04-23T10:26:09.820985+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6839/418013575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":418013575},{"type":"v1","addr":"172.21.15.97:6839","nonce":418013575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6841/747242680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":747242680},{"type":"v1","addr":"172.21.15.97:6841","nonce":747242680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6833/4292847121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4292847121},{"type":"v1","addr":"172.21.15.183:6833","nonce":4292847121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6833/3885816565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3885816565},{"type":"v1","addr":"172.21.15.196:6833","nonce":3885816565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1897093105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1897093105},{"type":"v1","addr":"172.21.15.183:6835","nonce":1897093105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1171477563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1171477563},{"type":"v1","addr":"172.21.15.196:6835","nonce":1171477563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1898341911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1898341911},{"type":"v1","addr":"172.21.15.183:6837","nonce":1898341911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.196:6837/3094946502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3094946502},{"type":"v1","addr":"172.21.15.196:6837","nonce":3094946502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.196:6839/83843556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":83843556},{"type":"v1","addr":"172.21.15.196:6839","nonce":83843556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6839/1763841130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1763841130},{"type":"v1","addr":"172.21.15.183:6839","nonce":1763841130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24487,24403,24398,24469,24488]},"id":1}]} 2024-04-23T10:35:00.563 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 26 2024-04-23T10:35:00.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.565+0000 7f004e405640 1 -- 172.21.15.97:0/2137324332 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f00300a3d20 msgr2=0x7f00300a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:00.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.565+0000 7f004e405640 1 --2- 172.21.15.97:0/2137324332 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f00300a3d20 0x7f00300a61e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0038000c00 tx=0x7f0038031400 comp rx=0 tx=0).stop 2024-04-23T10:35:00.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.565+0000 7f004e405640 1 -- 172.21.15.97:0/2137324332 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f004810c580 msgr2=0x7f004819d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:00.564 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.565+0000 7f004e405640 1 --2- 172.21.15.97:0/2137324332 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f004810c580 0x7f004819d010 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f004000e9d0 tx=0x7f004000ee90 comp rx=0 tx=0).stop 2024-04-23T10:35:00.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.565+0000 7f004e405640 1 -- 172.21.15.97:0/2137324332 shutdown_connections 2024-04-23T10:35:00.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.565+0000 7f004e405640 1 --2- 172.21.15.97:0/2137324332 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f00300a3d20 0x7f00300a61e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:00.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.565+0000 7f004e405640 1 --2- 172.21.15.97:0/2137324332 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f004810c580 0x7f004819d010 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:00.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.565+0000 7f004e405640 1 --2- 172.21.15.97:0/2137324332 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0048103ad0 0x7f0048198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:00.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.565+0000 7f004e405640 1 --2- 172.21.15.97:0/2137324332 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f00481031b0 0x7f0048198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:00.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.565+0000 7f004e405640 1 -- 172.21.15.97:0/2137324332 >> 172.21.15.97:0/2137324332 conn(0x7f00480771d0 msgr2=0x7f00480fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:35:00.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.565+0000 7f004e405640 1 -- 172.21.15.97:0/2137324332 shutdown_connections 2024-04-23T10:35:00.565 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.565+0000 7f004e405640 1 -- 172.21.15.97:0/2137324332 wait complete. 2024-04-23T10:35:00.574 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:35:00 smithi097 ceph-mon[29197]: pgmap v271: 97 pgs: 97 active+clean; 3.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-04-23T10:35:00.575 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --set-id c24eb2b3 --release --await' 2024-04-23T10:35:00.575 DEBUG:teuthology.orchestra.run.smithi097:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24488 quiesce db --set-id c24eb2b3 --release --await 2024-04-23T10:35:00.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:35:00 smithi183 ceph-mon[32317]: pgmap v271: 97 pgs: 97 active+clean; 3.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-04-23T10:35:00.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.691+0000 7fb08adbb640 1 -- 172.21.15.97:0/3566193995 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb084106160 msgr2=0x7fb08410dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:00.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.691+0000 7fb08adbb640 1 --2- 172.21.15.97:0/3566193995 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb084106160 0x7fb08410dd30 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fb07c009ca0 tx=0x7fb07c02a8c0 comp rx=0 tx=0).stop 2024-04-23T10:35:00.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.691+0000 7fb08adbb640 1 -- 172.21.15.97:0/3566193995 shutdown_connections 2024-04-23T10:35:00.690 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.691+0000 7fb08adbb640 1 --2- 172.21.15.97:0/3566193995 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb084106160 0x7fb08410dd30 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:00.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.691+0000 7fb08adbb640 1 --2- 172.21.15.97:0/3566193995 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb084101a40 0x7fb084105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:00.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.691+0000 7fb08adbb640 1 --2- 172.21.15.97:0/3566193995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb084101090 0x7fb084101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:00.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.691+0000 7fb08adbb640 1 -- 172.21.15.97:0/3566193995 >> 172.21.15.97:0/3566193995 conn(0x7fb084077fb0 msgr2=0x7fb0840783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:35:00.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.691+0000 7fb08adbb640 1 -- 172.21.15.97:0/3566193995 shutdown_connections 2024-04-23T10:35:00.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.691+0000 7fb08adbb640 1 -- 172.21.15.97:0/3566193995 wait complete. 2024-04-23T10:35:00.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.692+0000 7fb08adbb640 1 Processor -- start 2024-04-23T10:35:00.691 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.692+0000 7fb08adbb640 1 -- start start 2024-04-23T10:35:00.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.692+0000 7fb08adbb640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb084101090 0x7fb084199530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:00.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.692+0000 7fb08adbb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb084106160 0x7fb084199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:00.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.692+0000 7fb08adbb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb08419a000 0x7fb0841a45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:00.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.692+0000 7fb08adbb640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb0841110d0 con 0x7fb08419a000 2024-04-23T10:35:00.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.692+0000 7fb08adbb640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb084110f50 con 0x7fb084101090 2024-04-23T10:35:00.693 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.692+0000 7fb08adbb640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb084111250 con 0x7fb084106160 2024-04-23T10:35:00.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.692+0000 7fb089358640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb08419a000 0x7fb0841a45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:00.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.692+0000 7fb089358640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb08419a000 0x7fb0841a45b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33534/0 (socket says 172.21.15.97:33534) 2024-04-23T10:35:00.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.692+0000 7fb089358640 1 -- 172.21.15.97:0/1719079249 learned_addr learned my addr 172.21.15.97:0/1719079249 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:35:00.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.692+0000 7fb083fff640 1 --2- 172.21.15.97:0/1719079249 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb084106160 0x7fb084199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:00.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.692+0000 7fb088b57640 1 --2- 172.21.15.97:0/1719079249 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb084101090 0x7fb084199530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:00.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.693+0000 7fb089358640 1 -- 172.21.15.97:0/1719079249 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb084106160 msgr2=0x7fb084199a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:00.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.693+0000 7fb089358640 1 --2- 172.21.15.97:0/1719079249 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb084106160 0x7fb084199a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:00.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.693+0000 7fb089358640 1 -- 172.21.15.97:0/1719079249 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb084101090 msgr2=0x7fb084199530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:00.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.693+0000 7fb089358640 1 --2- 172.21.15.97:0/1719079249 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb084101090 0x7fb084199530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:00.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.693+0000 7fb089358640 1 -- 172.21.15.97:0/1719079249 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0841a4d80 con 0x7fb08419a000 2024-04-23T10:35:00.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.693+0000 7fb083fff640 1 --2- 172.21.15.97:0/1719079249 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb084106160 0x7fb084199a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T10:35:00.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.693+0000 7fb089358640 1 --2- 172.21.15.97:0/1719079249 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb08419a000 0x7fb0841a45b0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fb07c005cc0 tx=0x7fb07c031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:35:00.694 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.693+0000 7fb081ffb640 1 -- 172.21.15.97:0/1719079249 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb07c003b70 con 0x7fb08419a000 2024-04-23T10:35:00.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.693+0000 7fb08adbb640 1 -- 172.21.15.97:0/1719079249 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0841a5010 con 0x7fb08419a000 2024-04-23T10:35:00.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.693+0000 7fb081ffb640 1 -- 172.21.15.97:0/1719079249 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb07c0318f0 con 0x7fb08419a000 2024-04-23T10:35:00.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.693+0000 7fb08adbb640 1 -- 172.21.15.97:0/1719079249 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0841a55b0 con 0x7fb08419a000 2024-04-23T10:35:00.695 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.693+0000 7fb081ffb640 1 -- 172.21.15.97:0/1719079249 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb07c02f930 con 0x7fb08419a000 2024-04-23T10:35:00.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.695+0000 7fb081ffb640 1 -- 172.21.15.97:0/1719079249 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb07c02fb10 con 0x7fb08419a000 2024-04-23T10:35:00.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.695+0000 7fb08dbae740 1 Processor -- start 2024-04-23T10:35:00.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.696+0000 7fb08dbae740 1 -- start start 2024-04-23T10:35:00.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.696+0000 7fb08dbae740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55614feed1c0 0x55614feed5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:00.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.696+0000 7fb08dbae740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55614feedb70 0x55614fefc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:00.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.696+0000 7fb08dbae740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55614fefc890 0x55614fefec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:00.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.696+0000 7fb08dbae740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55614fe526d0 con 0x55614fefc890 2024-04-23T10:35:00.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.696+0000 7fb08dbae740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55614fdef1a0 con 0x55614feedb70 2024-04-23T10:35:00.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.696+0000 7fb08dbae740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55614fe4e690 con 0x55614feed1c0 2024-04-23T10:35:00.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.696+0000 7fb081ffb640 1 --2- 172.21.15.97:0/1719079249 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fb0740a3d20 0x7fb0740a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:00.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.696+0000 7fb083fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55614fefc890 0x55614fefec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:00.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.696+0000 7fb083fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55614fefc890 0x55614fefec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33538/0 (socket says 172.21.15.97:33538) 2024-04-23T10:35:00.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.696+0000 7fb083fff640 1 -- 172.21.15.97:0/2724694303 learned_addr learned my addr 172.21.15.97:0/2724694303 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:35:00.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.696+0000 7fb083fff640 1 --2- 172.21.15.97:0/2724694303 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55614feed1c0 0x55614feed5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:00.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.696+0000 7fb089358640 1 --2- 172.21.15.97:0/1719079249 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fb0740a3d20 0x7fb0740a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:00.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.696+0000 7fb088b57640 1 --2- 172.21.15.97:0/2724694303 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55614feedb70 0x55614fefc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:00.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.696+0000 7fb081ffb640 1 -- 172.21.15.97:0/1719079249 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7fb07c0f1b00 con 0x7fb08419a000 2024-04-23T10:35:00.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.696+0000 7fb083fff640 1 -- 172.21.15.97:0/2724694303 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55614feed1c0 msgr2=0x55614feed5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:35:00.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.696+0000 7fb083fff640 1 --2- 172.21.15.97:0/2724694303 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55614feed1c0 0x55614feed5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:00.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.696+0000 7fb083fff640 1 -- 172.21.15.97:0/2724694303 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55614feedb70 msgr2=0x55614fefc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:35:00.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.696+0000 7fb083fff640 1 --2- 172.21.15.97:0/2724694303 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55614feedb70 0x55614fefc350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:00.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.696+0000 7fb083fff640 1 -- 172.21.15.97:0/2724694303 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55614fdf95b0 con 0x55614fefc890 2024-04-23T10:35:00.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.696+0000 7fb083fff640 1 --2- 172.21.15.97:0/2724694303 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55614fefc890 0x55614fefec80 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fb0700128b0 tx=0x7fb070005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9be72d75dd18f192 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:35:00.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.696+0000 7fb089358640 1 --2- 172.21.15.97:0/1719079249 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fb0740a3d20 0x7fb0740a61e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb07c0ef870 tx=0x7fb07c0de040 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:35:00.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.697+0000 7fb06effd640 1 -- 172.21.15.97:0/2724694303 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb070004220 con 0x55614fefc890 2024-04-23T10:35:00.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.697+0000 7fb06effd640 1 -- 172.21.15.97:0/2724694303 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0700043c0 con 0x55614fefc890 2024-04-23T10:35:00.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.697+0000 7fb06effd640 1 -- 172.21.15.97:0/2724694303 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0700047a0 con 0x55614fefc890 2024-04-23T10:35:00.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.697+0000 7fb08dbae740 1 -- 172.21.15.97:0/2724694303 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55614fefc890 msgr2=0x55614fefec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:00.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.697+0000 7fb08dbae740 1 --2- 172.21.15.97:0/2724694303 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55614fefc890 0x55614fefec80 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fb0700128b0 tx=0x7fb070005d90 comp rx=0 tx=0).stop 2024-04-23T10:35:00.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.697+0000 7fb08dbae740 1 -- 172.21.15.97:0/2724694303 shutdown_connections 2024-04-23T10:35:00.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.697+0000 7fb08dbae740 1 --2- 172.21.15.97:0/2724694303 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55614fefc890 0x55614fefec80 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:00.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.697+0000 7fb08dbae740 1 --2- 172.21.15.97:0/2724694303 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55614feedb70 0x55614fefc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:00.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.697+0000 7fb08dbae740 1 --2- 172.21.15.97:0/2724694303 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55614feed1c0 0x55614feed5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:00.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.697+0000 7fb08dbae740 1 -- 172.21.15.97:0/2724694303 >> 172.21.15.97:0/2724694303 conn(0x55614fee97b0 msgr2=0x55614fee9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:35:00.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.697+0000 7fb08dbae740 1 -- 172.21.15.97:0/2724694303 shutdown_connections 2024-04-23T10:35:00.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.697+0000 7fb08dbae740 1 -- 172.21.15.97:0/2724694303 wait complete. 2024-04-23T10:35:00.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.698+0000 7fb08dbae740 1 -- start start 2024-04-23T10:35:00.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.698+0000 7fb08dbae740 1 Processor -- start 2024-04-23T10:35:00.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.698+0000 7fb08dbae740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55614feed1c0 0x55614fefc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:00.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.698+0000 7fb08dbae740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55614feedb70 0x55614fef9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:00.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.698+0000 7fb08dbae740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55614fefc890 0x55614fef9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:00.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.698+0000 7fb08dbae740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55614feff6f0 con 0x55614feedb70 2024-04-23T10:35:00.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.698+0000 7fb08dbae740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55614fdecb20 con 0x55614fefc890 2024-04-23T10:35:00.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.698+0000 7fb08dbae740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55614feff870 con 0x55614feed1c0 2024-04-23T10:35:00.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.698+0000 7fb08dbae740 10 client.0 _ll_register_callbacks cb 0x55614fdec530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:35:00.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.698+0000 7fb088b57640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55614feedb70 0x55614fef9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:00.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.698+0000 7fb088b57640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55614feedb70 0x55614fef9620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33546/0 (socket says 172.21.15.97:33546) 2024-04-23T10:35:00.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.698+0000 7fb088b57640 1 -- 172.21.15.97:0/836719645 learned_addr learned my addr 172.21.15.97:0/836719645 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:35:00.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.698+0000 7fb083fff640 1 --2- 172.21.15.97:0/836719645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55614fefc890 0x55614fef9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:00.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.698+0000 7fb083fff640 1 --2- 172.21.15.97:0/836719645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55614feed1c0 0x55614fefc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:00.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.698+0000 7fb088b57640 1 -- 172.21.15.97:0/836719645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55614feed1c0 msgr2=0x55614fefc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:35:00.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.698+0000 7fb088b57640 1 --2- 172.21.15.97:0/836719645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55614feed1c0 0x55614fefc330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:00.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.698+0000 7fb088b57640 1 -- 172.21.15.97:0/836719645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55614fefc890 msgr2=0x55614fef9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:35:00.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.698+0000 7fb088b57640 1 --2- 172.21.15.97:0/836719645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55614fefc890 0x55614fef9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:00.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.698+0000 7fb088b57640 1 -- 172.21.15.97:0/836719645 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55614fb64920 con 0x55614feedb70 2024-04-23T10:35:00.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.698+0000 7fb083fff640 1 --2- 172.21.15.97:0/836719645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55614fefc890 0x55614fef9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:35:00.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.699+0000 7fb088b57640 1 --2- 172.21.15.97:0/836719645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55614feedb70 0x55614fef9620 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fb078012700 tx=0x7fb078012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=826eaec3576e9eb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:35:00.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.699+0000 7fb063fff640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:35:00.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.699+0000 7fb063fff640 1 -- 172.21.15.97:0/836719645 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb078004050 con 0x55614feedb70 2024-04-23T10:35:00.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.699+0000 7fb08dbae740 1 -- 172.21.15.97:0/836719645 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0700043c0 con 0x55614feedb70 2024-04-23T10:35:00.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.699+0000 7fb063fff640 1 -- 172.21.15.97:0/836719645 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7fb0780041f0 con 0x55614feedb70 2024-04-23T10:35:00.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.699+0000 7fb063fff640 1 -- 172.21.15.97:0/836719645 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb0780043f0 con 0x55614feedb70 2024-04-23T10:35:00.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.699+0000 7fb08dbae740 10 client.16833 fetch_fsmap learned FSMap version 26 2024-04-23T10:35:00.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.699+0000 7fb08dbae740 1 -- 172.21.15.97:0/836719645 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55614fc30d30 con 0x55614feedb70 2024-04-23T10:35:00.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.699+0000 7fb063fff640 1 -- 172.21.15.97:0/836719645 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0700043c0 con 0x55614feedb70 2024-04-23T10:35:00.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.699+0000 7fb063fff640 1 -- 172.21.15.97:0/836719645 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb078023020 con 0x55614feedb70 2024-04-23T10:35:00.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.699+0000 7fb08dbae740 10 client.16833 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:35:00.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.699+0000 7fb08dbae740 10 client.16833 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:35:00.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.699+0000 7fb08dbae740 20 client.16833 populate_metadata read hostname 'smithi097' 2024-04-23T10:35:00.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.699+0000 7fb08dbae740 1 --2- 172.21.15.97:0/836719645 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55614ff0cab0 0x55614ff0eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:00.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.699+0000 7fb08dbae740 4 client.16833 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:35:00.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.699+0000 7fb08dbae740 1 -- 172.21.15.97:0/836719645 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55614fe10500 con 0x55614ff0cab0 2024-04-23T10:35:00.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.700+0000 7fb083fff640 1 --2- 172.21.15.97:0/836719645 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55614ff0cab0 0x55614ff0eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:00.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.700+0000 7fb083fff640 1 --2- 172.21.15.97:0/836719645 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55614ff0cab0 0x55614ff0eed0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=412ac7213a44bbfb server_cookie=ddff399b4644ada7 in_seq=0 out_seq=0 2024-04-23T10:35:00.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.700+0000 7fb063fff640 10 client.16833 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:35:00.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.701+0000 7fb063fff640 1 -- 172.21.15.97:0/836719645 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55614fe10500 con 0x55614ff0cab0 2024-04-23T10:35:00.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.701+0000 7fb063fff640 10 client.16833 handle_command_reply: tid=0 2024-04-23T10:35:00.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.701+0000 7fb08dbae740 1 client.16833 shutdown 2024-04-23T10:35:00.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.701+0000 7fb08dbae740 1 -- 172.21.15.97:0/836719645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55614feedb70 msgr2=0x55614fef9620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:00.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.701+0000 7fb08dbae740 1 --2- 172.21.15.97:0/836719645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55614feedb70 0x55614fef9620 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fb078012700 tx=0x7fb078012bc0 comp rx=0 tx=0).stop 2024-04-23T10:35:00.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.701+0000 7fb08dbae740 1 -- 172.21.15.97:0/836719645 shutdown_connections 2024-04-23T10:35:00.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.701+0000 7fb08dbae740 1 --2- 172.21.15.97:0/836719645 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55614ff0cab0 0x55614ff0eed0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:00.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.701+0000 7fb08dbae740 1 --2- 172.21.15.97:0/836719645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55614fefc890 0x55614fef9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:00.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.701+0000 7fb063fff640 0 client.16833 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:35:00.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.701+0000 7fb08dbae740 1 --2- 172.21.15.97:0/836719645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55614feedb70 0x55614fef9620 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:00.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.701+0000 7fb08dbae740 1 --2- 172.21.15.97:0/836719645 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55614feed1c0 0x55614fefc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:00.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.701+0000 7fb08dbae740 1 -- 172.21.15.97:0/836719645 >> 172.21.15.97:0/836719645 conn(0x55614fee97b0 msgr2=0x55614feef620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:35:00.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.701+0000 7fb08dbae740 1 -- 172.21.15.97:0/836719645 shutdown_connections 2024-04-23T10:35:00.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.701+0000 7fb08dbae740 1 -- 172.21.15.97:0/836719645 wait complete. 2024-04-23T10:35:00.708 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.701+0000 7fb08dbae740 20 client.16833 trim_cache size 0 max 16384 2024-04-23T10:35:00.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.718+0000 7fb08dbae740 1 Processor -- start 2024-04-23T10:35:00.721 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.718+0000 7fb08dbae740 1 -- start start 2024-04-23T10:35:00.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.718+0000 7fb08dbae740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55614feed1c0 0x55614fef8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:00.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.718+0000 7fb08dbae740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55614feedb70 0x55614fefbe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:00.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.718+0000 7fb08dbae740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55614fefc890 0x55614ff147a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:00.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.718+0000 7fb08dbae740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb0841ab8d0 con 0x55614fefc890 2024-04-23T10:35:00.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.718+0000 7fb08dbae740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb0841ad410 con 0x55614feedb70 2024-04-23T10:35:00.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.718+0000 7fb08dbae740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb0841abfa0 con 0x55614feed1c0 2024-04-23T10:35:00.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.718+0000 7fb083fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55614fefc890 0x55614ff147a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:00.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.718+0000 7fb083fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55614fefc890 0x55614ff147a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33552/0 (socket says 172.21.15.97:33552) 2024-04-23T10:35:00.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.719+0000 7fb083fff640 1 -- 172.21.15.97:0/2883905172 learned_addr learned my addr 172.21.15.97:0/2883905172 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:35:00.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.719+0000 7fb083fff640 1 --2- 172.21.15.97:0/2883905172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55614feed1c0 0x55614fef8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:00.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.719+0000 7fb083fff640 1 -- 172.21.15.97:0/2883905172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55614feed1c0 msgr2=0x55614fef8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:35:00.722 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.719+0000 7fb083fff640 1 --2- 172.21.15.97:0/2883905172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55614feed1c0 0x55614fef8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:00.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.719+0000 7fb083fff640 1 -- 172.21.15.97:0/2883905172 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55614feedb70 msgr2=0x55614fefbe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:35:00.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.719+0000 7fb088b57640 1 --2- 172.21.15.97:0/2883905172 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55614feedb70 0x55614fefbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:00.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.719+0000 7fb083fff640 1 --2- 172.21.15.97:0/2883905172 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55614feedb70 0x55614fefbe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:00.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.719+0000 7fb083fff640 1 -- 172.21.15.97:0/2883905172 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55614feee2c0 con 0x55614fefc890 2024-04-23T10:35:00.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.719+0000 7fb083fff640 1 --2- 172.21.15.97:0/2883905172 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55614fefc890 0x55614ff147a0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fb070010ba0 tx=0x7fb07000bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9fe291cd2858bf3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:35:00.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.719+0000 7fb06e7fc640 1 -- 172.21.15.97:0/2883905172 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb070032020 con 0x55614fefc890 2024-04-23T10:35:00.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.719+0000 7fb06e7fc640 1 -- 172.21.15.97:0/2883905172 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb070005040 con 0x55614fefc890 2024-04-23T10:35:00.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.719+0000 7fb06e7fc640 1 -- 172.21.15.97:0/2883905172 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0700049c0 con 0x55614fefc890 2024-04-23T10:35:00.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.720+0000 7fb08dbae740 1 -- 172.21.15.97:0/2883905172 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55614fefc890 msgr2=0x55614ff147a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:00.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.720+0000 7fb08dbae740 1 --2- 172.21.15.97:0/2883905172 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55614fefc890 0x55614ff147a0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fb070010ba0 tx=0x7fb07000bc20 comp rx=0 tx=0).stop 2024-04-23T10:35:00.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.720+0000 7fb08dbae740 1 -- 172.21.15.97:0/2883905172 shutdown_connections 2024-04-23T10:35:00.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.720+0000 7fb08dbae740 1 --2- 172.21.15.97:0/2883905172 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55614fefc890 0x55614ff147a0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:00.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.720+0000 7fb08dbae740 1 --2- 172.21.15.97:0/2883905172 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55614feedb70 0x55614fefbe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:00.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.720+0000 7fb08dbae740 1 --2- 172.21.15.97:0/2883905172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55614feed1c0 0x55614fef8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:00.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.720+0000 7fb08dbae740 1 -- 172.21.15.97:0/2883905172 >> 172.21.15.97:0/2883905172 conn(0x55614fee97b0 msgr2=0x55614fee89d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:35:00.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.720+0000 7fb08dbae740 1 -- 172.21.15.97:0/2883905172 shutdown_connections 2024-04-23T10:35:00.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.720+0000 7fb08dbae740 1 -- 172.21.15.97:0/2883905172 wait complete. 2024-04-23T10:35:00.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.720+0000 7fb08dbae740 1 -- start start 2024-04-23T10:35:00.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.720+0000 7fb08dbae740 1 Processor -- start 2024-04-23T10:35:00.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.720+0000 7fb08dbae740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55614feed1c0 0x55614ff04dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:00.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.720+0000 7fb08dbae740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55614feedb70 0x55614feffaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:00.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.720+0000 7fb08dbae740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55614fefc890 0x55614fefffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:00.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.720+0000 7fb08dbae740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb0841acd40 con 0x55614fefc890 2024-04-23T10:35:00.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.720+0000 7fb08dbae740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb0841ac670 con 0x55614feed1c0 2024-04-23T10:35:00.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.720+0000 7fb08dbae740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb0841a98d0 con 0x55614feedb70 2024-04-23T10:35:00.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.720+0000 7fb08dbae740 10 client.0 _ll_register_callbacks cb 0x7fb078009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:35:00.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.721+0000 7fb083fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55614fefc890 0x55614fefffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:00.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.721+0000 7fb083fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55614fefc890 0x55614fefffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33554/0 (socket says 172.21.15.97:33554) 2024-04-23T10:35:00.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.721+0000 7fb083fff640 1 -- 172.21.15.97:0/3011130174 learned_addr learned my addr 172.21.15.97:0/3011130174 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:35:00.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.721+0000 7fb088b57640 1 --2- 172.21.15.97:0/3011130174 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55614feedb70 0x55614feffaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:00.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.721+0000 7fb083fff640 1 --2- 172.21.15.97:0/3011130174 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55614feed1c0 0x55614ff04dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:00.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.721+0000 7fb083fff640 1 -- 172.21.15.97:0/3011130174 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55614feedb70 msgr2=0x55614feffaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:35:00.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.721+0000 7fb083fff640 1 --2- 172.21.15.97:0/3011130174 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55614feedb70 0x55614feffaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:00.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.721+0000 7fb083fff640 1 -- 172.21.15.97:0/3011130174 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55614feed1c0 msgr2=0x55614ff04dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:35:00.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.721+0000 7fb083fff640 1 --2- 172.21.15.97:0/3011130174 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55614feed1c0 0x55614ff04dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:00.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.721+0000 7fb083fff640 1 -- 172.21.15.97:0/3011130174 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55614ff006a0 con 0x55614fefc890 2024-04-23T10:35:00.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.721+0000 7fb083fff640 1 --2- 172.21.15.97:0/3011130174 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55614fefc890 0x55614fefffe0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fb0700107c0 tx=0x7fb070005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1238021ded83950 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:35:00.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.721+0000 7fb063fff640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:35:00.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.721+0000 7fb063fff640 1 -- 172.21.15.97:0/3011130174 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb070032070 con 0x55614fefc890 2024-04-23T10:35:00.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.721+0000 7fb063fff640 1 -- 172.21.15.97:0/3011130174 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8484+0+0 (secure 0 0 0) 0x7fb07002c620 con 0x55614fefc890 2024-04-23T10:35:00.725 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.721+0000 7fb08dbae740 1 -- 172.21.15.97:0/3011130174 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb070005040 con 0x55614fefc890 2024-04-23T10:35:00.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.721+0000 7fb063fff640 1 -- 172.21.15.97:0/3011130174 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb070005640 con 0x55614fefc890 2024-04-23T10:35:00.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.721+0000 7fb08dbae740 10 client.16839 fetch_fsmap learned FSMap version 26 2024-04-23T10:35:00.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.721+0000 7fb08dbae740 1 -- 172.21.15.97:0/3011130174 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55614ff73f00 con 0x55614fefc890 2024-04-23T10:35:00.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.721+0000 7fb063fff640 1 -- 172.21.15.97:0/3011130174 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb070005040 con 0x55614fefc890 2024-04-23T10:35:00.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.722+0000 7fb063fff640 1 -- 172.21.15.97:0/3011130174 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb070041050 con 0x55614fefc890 2024-04-23T10:35:00.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.722+0000 7fb08dbae740 10 client.16839 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:35:00.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.722+0000 7fb08dbae740 10 client.16839 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:35:00.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.722+0000 7fb08dbae740 20 client.16839 populate_metadata read hostname 'smithi097' 2024-04-23T10:35:00.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.722+0000 7fb08dbae740 1 --2- 172.21.15.97:0/3011130174 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55614ff74370 0x55614ff76790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:00.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.722+0000 7fb08dbae740 4 client.16839 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "c24eb2b3", "release": true, "await": true}] 2024-04-23T10:35:00.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.722+0000 7fb08dbae740 1 -- 172.21.15.97:0/3011130174 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "quiesce db", "set_id": "c24eb2b3", "release": true, "await": true}) -- 0x55614fe600f0 con 0x55614ff74370 2024-04-23T10:35:00.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.722+0000 7fb088b57640 1 --2- 172.21.15.97:0/3011130174 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55614ff74370 0x55614ff76790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:00.726 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.722+0000 7fb088b57640 1 --2- 172.21.15.97:0/3011130174 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55614ff74370 0x55614ff76790 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5283c66d48a2bf64 server_cookie=b18928ec81999d29 in_seq=0 out_seq=0 2024-04-23T10:35:00.727 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.722+0000 7fb063fff640 10 client.16839 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:35:00.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.725+0000 7fb063fff640 1 -- 172.21.15.97:0/3011130174 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55614fe600f0 con 0x55614ff74370 2024-04-23T10:35:00.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.725+0000 7fb063fff640 10 client.16839 handle_command_reply: tid=0 2024-04-23T10:35:00.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.725+0000 7fb08dbae740 1 client.16839 shutdown 2024-04-23T10:35:00.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.725+0000 7fb08dbae740 1 -- 172.21.15.97:0/3011130174 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55614fefc890 msgr2=0x55614fefffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:00.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.725+0000 7fb08dbae740 1 --2- 172.21.15.97:0/3011130174 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55614fefc890 0x55614fefffe0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fb0700107c0 tx=0x7fb070005fb0 comp rx=0 tx=0).stop 2024-04-23T10:35:00.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.725+0000 7fb08dbae740 1 -- 172.21.15.97:0/3011130174 shutdown_connections 2024-04-23T10:35:00.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.725+0000 7fb08dbae740 1 --2- 172.21.15.97:0/3011130174 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55614ff74370 0x55614ff76790 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:00.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.725+0000 7fb08dbae740 1 --2- 172.21.15.97:0/3011130174 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55614fefc890 0x55614fefffe0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:00.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.725+0000 7fb063fff640 0 client.16839 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:35:00.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.725+0000 7fb08dbae740 1 --2- 172.21.15.97:0/3011130174 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55614feedb70 0x55614feffaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:00.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.725+0000 7fb08dbae740 1 --2- 172.21.15.97:0/3011130174 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55614feed1c0 0x55614ff04dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:00.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.725+0000 7fb08dbae740 1 -- 172.21.15.97:0/3011130174 >> 172.21.15.97:0/3011130174 conn(0x55614fee97b0 msgr2=0x55614feee990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:35:00.728 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.725+0000 7fb08dbae740 1 -- 172.21.15.97:0/3011130174 shutdown_connections 2024-04-23T10:35:00.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.725+0000 7fb08dbae740 1 -- 172.21.15.97:0/3011130174 wait complete. 2024-04-23T10:35:00.729 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.725+0000 7fb08dbae740 20 client.16839 trim_cache size 0 max 16384 2024-04-23T10:35:00.729 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-23T10:35:00.729 INFO:teuthology.orchestra.run.smithi097.stdout: "epoch": 26, 2024-04-23T10:35:00.729 INFO:teuthology.orchestra.run.smithi097.stdout: "leader": 24488, 2024-04-23T10:35:00.729 INFO:teuthology.orchestra.run.smithi097.stdout: "set_version": 36, 2024-04-23T10:35:00.729 INFO:teuthology.orchestra.run.smithi097.stdout: "sets": { 2024-04-23T10:35:00.729 INFO:teuthology.orchestra.run.smithi097.stdout: "c24eb2b3": { 2024-04-23T10:35:00.729 INFO:teuthology.orchestra.run.smithi097.stdout: "version": 36, 2024-04-23T10:35:00.729 INFO:teuthology.orchestra.run.smithi097.stdout: "age_ref": 0.0, 2024-04-23T10:35:00.729 INFO:teuthology.orchestra.run.smithi097.stdout: "state": { 2024-04-23T10:35:00.729 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RELEASED", 2024-04-23T10:35:00.730 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0 2024-04-23T10:35:00.730 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:35:00.730 INFO:teuthology.orchestra.run.smithi097.stdout: "timeout": 60.0, 2024-04-23T10:35:00.730 INFO:teuthology.orchestra.run.smithi097.stdout: "expiration": 70.0, 2024-04-23T10:35:00.730 INFO:teuthology.orchestra.run.smithi097.stdout: "members": { 2024-04-23T10:35:00.730 INFO:teuthology.orchestra.run.smithi097.stdout: "file:/": { 2024-04-23T10:35:00.730 INFO:teuthology.orchestra.run.smithi097.stdout: "excluded": false, 2024-04-23T10:35:00.730 INFO:teuthology.orchestra.run.smithi097.stdout: "state": { 2024-04-23T10:35:00.730 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RELEASED", 2024-04-23T10:35:00.730 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0 2024-04-23T10:35:00.730 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:35:00.730 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:35:00.730 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:35:00.730 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:35:00.731 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:35:00.731 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-23T10:35:00.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.726+0000 7fb08adbb640 1 -- 172.21.15.97:0/1719079249 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fb0740a3d20 msgr2=0x7fb0740a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:00.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.726+0000 7fb08adbb640 1 --2- 172.21.15.97:0/1719079249 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fb0740a3d20 0x7fb0740a61e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb07c0ef870 tx=0x7fb07c0de040 comp rx=0 tx=0).stop 2024-04-23T10:35:00.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.726+0000 7fb08adbb640 1 -- 172.21.15.97:0/1719079249 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb08419a000 msgr2=0x7fb0841a45b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:00.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.726+0000 7fb08adbb640 1 --2- 172.21.15.97:0/1719079249 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb08419a000 0x7fb0841a45b0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fb07c005cc0 tx=0x7fb07c031700 comp rx=0 tx=0).stop 2024-04-23T10:35:00.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.726+0000 7fb08adbb640 1 -- 172.21.15.97:0/1719079249 shutdown_connections 2024-04-23T10:35:00.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.726+0000 7fb08adbb640 1 --2- 172.21.15.97:0/1719079249 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fb0740a3d20 0x7fb0740a61e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:00.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.726+0000 7fb08adbb640 1 --2- 172.21.15.97:0/1719079249 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb08419a000 0x7fb0841a45b0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:00.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.726+0000 7fb08adbb640 1 --2- 172.21.15.97:0/1719079249 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb084106160 0x7fb084199a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:00.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.726+0000 7fb08adbb640 1 --2- 172.21.15.97:0/1719079249 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb084101090 0x7fb084199530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:00.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.726+0000 7fb08adbb640 1 -- 172.21.15.97:0/1719079249 >> 172.21.15.97:0/1719079249 conn(0x7fb084077fb0 msgr2=0x7fb0840fefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:35:00.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.726+0000 7fb08adbb640 1 -- 172.21.15.97:0/1719079249 shutdown_connections 2024-04-23T10:35:00.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:00.726+0000 7fb08adbb640 1 -- 172.21.15.97:0/1719079249 wait complete. 2024-04-23T10:35:00.735 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'c24eb2b3', seconds elapsed: 11.0 2024-04-23T10:35:00.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:35:00 smithi196 ceph-mon[32919]: pgmap v271: 97 pgs: 97 active+clean; 3.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-04-23T10:35:01.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:35:01 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/2137324332' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:35:01.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:35:01 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/2137324332' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:35:01.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:35:01 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/2137324332' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:35:02.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:35:02 smithi097 ceph-mon[29197]: pgmap v272: 97 pgs: 97 active+clean; 3.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-04-23T10:35:02.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:35:02 smithi097 ceph-mon[29197]: osdmap e94: 12 total, 12 up, 12 in 2024-04-23T10:35:02.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:35:02 smithi183 ceph-mon[32317]: pgmap v272: 97 pgs: 97 active+clean; 3.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-04-23T10:35:02.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:35:02 smithi183 ceph-mon[32317]: osdmap e94: 12 total, 12 up, 12 in 2024-04-23T10:35:02.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:35:02 smithi196 ceph-mon[32919]: pgmap v272: 97 pgs: 97 active+clean; 3.9 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-04-23T10:35:02.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:35:02 smithi196 ceph-mon[32919]: osdmap e94: 12 total, 12 up, 12 in 2024-04-23T10:35:04.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:35:04 smithi097 ceph-mon[29197]: pgmap v274: 97 pgs: 97 active+clean; 4.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 74 KiB/s wr, 35 op/s 2024-04-23T10:35:04.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:35:04 smithi183 ceph-mon[32317]: pgmap v274: 97 pgs: 97 active+clean; 4.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 74 KiB/s wr, 35 op/s 2024-04-23T10:35:04.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:35:04 smithi196 ceph-mon[32919]: pgmap v274: 97 pgs: 97 active+clean; 4.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 74 KiB/s wr, 35 op/s 2024-04-23T10:35:07.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:35:07 smithi196 ceph-mon[32919]: pgmap v275: 97 pgs: 97 active+clean; 4.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 74 KiB/s wr, 35 op/s 2024-04-23T10:35:07.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:35:07 smithi196 ceph-mon[32919]: osdmap e95: 12 total, 12 up, 12 in 2024-04-23T10:35:07.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:35:07 smithi097 ceph-mon[29197]: pgmap v275: 97 pgs: 97 active+clean; 4.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 74 KiB/s wr, 35 op/s 2024-04-23T10:35:07.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:35:07 smithi097 ceph-mon[29197]: osdmap e95: 12 total, 12 up, 12 in 2024-04-23T10:35:07.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:35:07 smithi183 ceph-mon[32317]: pgmap v275: 97 pgs: 97 active+clean; 4.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 74 KiB/s wr, 35 op/s 2024-04-23T10:35:07.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:35:07 smithi183 ceph-mon[32317]: osdmap e95: 12 total, 12 up, 12 in 2024-04-23T10:35:09.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:35:09 smithi097 ceph-mon[29197]: pgmap v277: 97 pgs: 97 active+clean; 4.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 116 KiB/s wr, 46 op/s 2024-04-23T10:35:09.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:35:09 smithi183 ceph-mon[32317]: pgmap v277: 97 pgs: 97 active+clean; 4.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 116 KiB/s wr, 46 op/s 2024-04-23T10:35:09.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:35:09 smithi196 ceph-mon[32919]: pgmap v277: 97 pgs: 97 active+clean; 4.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 116 KiB/s wr, 46 op/s 2024-04-23T10:35:11.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:35:11 smithi097 ceph-mon[29197]: pgmap v278: 97 pgs: 97 active+clean; 4.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 116 KiB/s wr, 46 op/s 2024-04-23T10:35:11.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:35:11 smithi183 ceph-mon[32317]: pgmap v278: 97 pgs: 97 active+clean; 4.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 116 KiB/s wr, 46 op/s 2024-04-23T10:35:11.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:35:11 smithi196 ceph-mon[32919]: pgmap v278: 97 pgs: 97 active+clean; 4.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 116 KiB/s wr, 46 op/s 2024-04-23T10:35:12.545 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.544+0000 7fc5ac6d9640 1 -- 172.21.15.97:0/3346305285 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5a41010f0 msgr2=0x7fc5a41014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:12.545 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.544+0000 7fc5ac6d9640 1 --2- 172.21.15.97:0/3346305285 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5a41010f0 0x7fc5a41014d0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0009900 tx=0x7fc5a002a410 comp rx=0 tx=0).stop 2024-04-23T10:35:12.545 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.544+0000 7fc5ac6d9640 1 -- 172.21.15.97:0/3346305285 shutdown_connections 2024-04-23T10:35:12.545 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.544+0000 7fc5ac6d9640 1 --2- 172.21.15.97:0/3346305285 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5a4106170 0x7fc5a410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:12.545 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.544+0000 7fc5ac6d9640 1 --2- 172.21.15.97:0/3346305285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc5a4101aa0 0x7fc5a4105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:12.545 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.544+0000 7fc5ac6d9640 1 --2- 172.21.15.97:0/3346305285 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5a41010f0 0x7fc5a41014d0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:12.545 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.544+0000 7fc5ac6d9640 1 -- 172.21.15.97:0/3346305285 >> 172.21.15.97:0/3346305285 conn(0x7fc5a4077fd0 msgr2=0x7fc5a40783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:35:12.545 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.544+0000 7fc5ac6d9640 1 -- 172.21.15.97:0/3346305285 shutdown_connections 2024-04-23T10:35:12.546 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.544+0000 7fc5ac6d9640 1 -- 172.21.15.97:0/3346305285 wait complete. 2024-04-23T10:35:12.546 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.544+0000 7fc5ac6d9640 1 Processor -- start 2024-04-23T10:35:12.546 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.545+0000 7fc5ac6d9640 1 -- start start 2024-04-23T10:35:12.546 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.545+0000 7fc5ac6d9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5a41010f0 0x7fc5a41a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:12.546 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.545+0000 7fc5ac6d9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5a4106170 0x7fc5a41a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:12.546 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.545+0000 7fc5ac6d9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc5a419a690 0x7fc5a419ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:12.546 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.545+0000 7fc5ac6d9640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a41121f0 con 0x7fc5a4106170 2024-04-23T10:35:12.546 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.545+0000 7fc5ac6d9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a4112070 con 0x7fc5a419a690 2024-04-23T10:35:12.546 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.545+0000 7fc5ac6d9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a4112370 con 0x7fc5a41010f0 2024-04-23T10:35:12.546 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.545+0000 7fc5a9c74640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5a4106170 0x7fc5a41a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:12.546 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.545+0000 7fc5a9c74640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5a4106170 0x7fc5a41a0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48180/0 (socket says 172.21.15.97:48180) 2024-04-23T10:35:12.546 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.545+0000 7fc5a9c74640 1 -- 172.21.15.97:0/4178321001 learned_addr learned my addr 172.21.15.97:0/4178321001 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:35:12.547 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.545+0000 7fc5aa475640 1 --2- 172.21.15.97:0/4178321001 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5a41010f0 0x7fc5a41a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:12.547 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.545+0000 7fc5aac76640 1 --2- 172.21.15.97:0/4178321001 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc5a419a690 0x7fc5a419ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:12.547 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.545+0000 7fc5a9c74640 1 -- 172.21.15.97:0/4178321001 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5a41010f0 msgr2=0x7fc5a41a05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:12.547 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.545+0000 7fc5a9c74640 1 --2- 172.21.15.97:0/4178321001 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5a41010f0 0x7fc5a41a05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:12.547 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.545+0000 7fc5a9c74640 1 -- 172.21.15.97:0/4178321001 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc5a419a690 msgr2=0x7fc5a419ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:12.547 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.545+0000 7fc5a9c74640 1 --2- 172.21.15.97:0/4178321001 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc5a419a690 0x7fc5a419ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:12.547 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.545+0000 7fc5a9c74640 1 -- 172.21.15.97:0/4178321001 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5a419b290 con 0x7fc5a4106170 2024-04-23T10:35:12.547 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.545+0000 7fc5a9c74640 1 --2- 172.21.15.97:0/4178321001 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5a4106170 0x7fc5a41a0b10 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fc59400d770 tx=0x7fc59400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:35:12.547 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.545+0000 7fc59b7fe640 1 -- 172.21.15.97:0/4178321001 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc594017070 con 0x7fc5a4106170 2024-04-23T10:35:12.547 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.546+0000 7fc5ac6d9640 1 -- 172.21.15.97:0/4178321001 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5a41a7370 con 0x7fc5a4106170 2024-04-23T10:35:12.547 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.546+0000 7fc59b7fe640 1 -- 172.21.15.97:0/4178321001 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc594009c30 con 0x7fc5a4106170 2024-04-23T10:35:12.547 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.546+0000 7fc59b7fe640 1 -- 172.21.15.97:0/4178321001 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59400f620 con 0x7fc5a4106170 2024-04-23T10:35:12.547 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.546+0000 7fc5ac6d9640 1 -- 172.21.15.97:0/4178321001 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5a41a7770 con 0x7fc5a4106170 2024-04-23T10:35:12.554 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.548+0000 7fc59b7fe640 1 -- 172.21.15.97:0/4178321001 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc59401a070 con 0x7fc5a4106170 2024-04-23T10:35:12.554 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.548+0000 7fc5af3df740 1 Processor -- start 2024-04-23T10:35:12.554 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.548+0000 7fc5af3df740 1 -- start start 2024-04-23T10:35:12.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.548+0000 7fc59b7fe640 1 --2- 172.21.15.97:0/4178321001 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fc5900a3dc0 0x7fc5900a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:12.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.549+0000 7fc5af3df740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615396bf620 0x5615396bfa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:12.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.549+0000 7fc5af3df740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615396bff40 0x5615396ca2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:12.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.549+0000 7fc5af3df740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615396ca800 0x5615396ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:12.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.549+0000 7fc5af3df740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x561539624970 con 0x5615396bff40 2024-04-23T10:35:12.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.549+0000 7fc5af3df740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5615395c14a0 con 0x5615396ca800 2024-04-23T10:35:12.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.549+0000 7fc59b7fe640 1 -- 172.21.15.97:0/4178321001 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8484+0+0 (secure 0 0 0) 0x7fc5940d03e0 con 0x7fc5a4106170 2024-04-23T10:35:12.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.549+0000 7fc5af3df740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x561539620930 con 0x5615396bf620 2024-04-23T10:35:12.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.549+0000 7fc5aac76640 1 --2- 172.21.15.97:0/4178321001 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fc5900a3dc0 0x7fc5900a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:12.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.549+0000 7fc5a9c74640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615396bff40 0x5615396ca2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:12.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.549+0000 7fc5a9c74640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615396bff40 0x5615396ca2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48182/0 (socket says 172.21.15.97:48182) 2024-04-23T10:35:12.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.549+0000 7fc5a9c74640 1 -- 172.21.15.97:0/3087171871 learned_addr learned my addr 172.21.15.97:0/3087171871 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:35:12.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.549+0000 7fc5aa475640 1 --2- 172.21.15.97:0/3087171871 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615396bf620 0x5615396bfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:12.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.549+0000 7fc5aac76640 1 --2- 172.21.15.97:0/3087171871 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615396ca800 0x5615396ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:12.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.549+0000 7fc5a9c74640 1 -- 172.21.15.97:0/3087171871 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615396bf620 msgr2=0x5615396bfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:35:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.549+0000 7fc5a9c74640 1 --2- 172.21.15.97:0/3087171871 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615396bf620 0x5615396bfa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.549+0000 7fc5a9c74640 1 -- 172.21.15.97:0/3087171871 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615396ca800 msgr2=0x5615396ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:35:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.549+0000 7fc5a9c74640 1 --2- 172.21.15.97:0/3087171871 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615396ca800 0x5615396ccbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.549+0000 7fc5a9c74640 1 -- 172.21.15.97:0/3087171871 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561539337210 con 0x5615396bff40 2024-04-23T10:35:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.549+0000 7fc5aac76640 1 --2- 172.21.15.97:0/3087171871 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615396ca800 0x5615396ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:35:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.549+0000 7fc5aac76640 1 --2- 172.21.15.97:0/4178321001 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fc5900a3dc0 0x7fc5900a6280 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc59c00acf0 tx=0x7fc59c00e720 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:35:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.549+0000 7fc5a9c74640 1 --2- 172.21.15.97:0/3087171871 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615396bff40 0x5615396ca2c0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fc5940ce150 tx=0x7fc594010c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d89fb23e566c6d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:35:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.549+0000 7fc598ff9640 1 -- 172.21.15.97:0/3087171871 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc594021070 con 0x5615396bff40 2024-04-23T10:35:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.550+0000 7fc598ff9640 1 -- 172.21.15.97:0/3087171871 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5940a0750 con 0x5615396bff40 2024-04-23T10:35:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.550+0000 7fc598ff9640 1 -- 172.21.15.97:0/3087171871 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc594098350 con 0x5615396bff40 2024-04-23T10:35:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.550+0000 7fc5af3df740 1 -- 172.21.15.97:0/3087171871 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615396bff40 msgr2=0x5615396ca2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.550+0000 7fc5af3df740 1 --2- 172.21.15.97:0/3087171871 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615396bff40 0x5615396ca2c0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fc5940ce150 tx=0x7fc594010c00 comp rx=0 tx=0).stop 2024-04-23T10:35:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.550+0000 7fc5af3df740 1 -- 172.21.15.97:0/3087171871 shutdown_connections 2024-04-23T10:35:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.550+0000 7fc5af3df740 1 --2- 172.21.15.97:0/3087171871 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615396ca800 0x5615396ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.550+0000 7fc5af3df740 1 --2- 172.21.15.97:0/3087171871 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615396bff40 0x5615396ca2c0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.550+0000 7fc5af3df740 1 --2- 172.21.15.97:0/3087171871 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615396bf620 0x5615396bfa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.550+0000 7fc5af3df740 1 -- 172.21.15.97:0/3087171871 >> 172.21.15.97:0/3087171871 conn(0x5615396bbb30 msgr2=0x5615396bbf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:35:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.550+0000 7fc5af3df740 1 -- 172.21.15.97:0/3087171871 shutdown_connections 2024-04-23T10:35:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.550+0000 7fc5af3df740 1 -- 172.21.15.97:0/3087171871 wait complete. 2024-04-23T10:35:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.551+0000 7fc5af3df740 1 -- start start 2024-04-23T10:35:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.551+0000 7fc5af3df740 1 Processor -- start 2024-04-23T10:35:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.551+0000 7fc5af3df740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615396bf620 0x5615396ca180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.551+0000 7fc5af3df740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615396bff40 0x5615396c7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.551+0000 7fc5af3df740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615396ca800 0x5615396c79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.551+0000 7fc5af3df740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5615396cd3d0 con 0x5615396bff40 2024-04-23T10:35:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.551+0000 7fc5af3df740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5615395bf0b0 con 0x5615396bf620 2024-04-23T10:35:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.551+0000 7fc5af3df740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5615396cd550 con 0x5615396ca800 2024-04-23T10:35:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.551+0000 7fc5af3df740 10 client.0 _ll_register_callbacks cb 0x5615395acd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:35:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.551+0000 7fc5a9c74640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615396bff40 0x5615396c7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:12.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.551+0000 7fc5a9c74640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615396bff40 0x5615396c7470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48190/0 (socket says 172.21.15.97:48190) 2024-04-23T10:35:12.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.551+0000 7fc5a9c74640 1 -- 172.21.15.97:0/362309272 learned_addr learned my addr 172.21.15.97:0/362309272 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:35:12.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.551+0000 7fc5aa475640 1 --2- 172.21.15.97:0/362309272 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615396bf620 0x5615396ca180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:12.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.551+0000 7fc5a9c74640 1 -- 172.21.15.97:0/362309272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615396ca800 msgr2=0x5615396c79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:35:12.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.551+0000 7fc5a9c74640 1 --2- 172.21.15.97:0/362309272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615396ca800 0x5615396c79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:12.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.551+0000 7fc5a9c74640 1 -- 172.21.15.97:0/362309272 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615396bf620 msgr2=0x5615396ca180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:35:12.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.551+0000 7fc5a9c74640 1 --2- 172.21.15.97:0/362309272 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615396bf620 0x5615396ca180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:12.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.551+0000 7fc5a9c74640 1 -- 172.21.15.97:0/362309272 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561539403620 con 0x5615396bff40 2024-04-23T10:35:12.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.552+0000 7fc5a9c74640 1 --2- 172.21.15.97:0/362309272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615396bff40 0x5615396c7470 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fc59409c330 tx=0x7fc5940ce080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faa64071870c888c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:35:12.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.552+0000 7fc58dffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:35:12.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.552+0000 7fc58dffb640 1 -- 172.21.15.97:0/362309272 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5940b9020 con 0x5615396bff40 2024-04-23T10:35:12.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.552+0000 7fc58dffb640 1 -- 172.21.15.97:0/362309272 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8484+0+0 (secure 0 0 0) 0x7fc5940bb2c0 con 0x5615396bff40 2024-04-23T10:35:12.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.552+0000 7fc5af3df740 1 -- 172.21.15.97:0/362309272 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5940a0750 con 0x5615396bff40 2024-04-23T10:35:12.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.552+0000 7fc58dffb640 1 -- 172.21.15.97:0/362309272 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc594021070 con 0x5615396bff40 2024-04-23T10:35:12.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.552+0000 7fc5af3df740 10 client.16851 fetch_fsmap learned FSMap version 26 2024-04-23T10:35:12.559 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.552+0000 7fc5af3df740 1 -- 172.21.15.97:0/362309272 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5615396d92d0 con 0x5615396bff40 2024-04-23T10:35:12.559 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.552+0000 7fc58dffb640 1 -- 172.21.15.97:0/362309272 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5940a0750 con 0x5615396bff40 2024-04-23T10:35:12.559 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.552+0000 7fc58dffb640 1 -- 172.21.15.97:0/362309272 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc59409b260 con 0x5615396bff40 2024-04-23T10:35:12.559 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.552+0000 7fc5af3df740 10 client.16851 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:35:12.559 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.552+0000 7fc5af3df740 10 client.16851 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:35:12.559 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.552+0000 7fc5af3df740 20 client.16851 populate_metadata read hostname 'smithi097' 2024-04-23T10:35:12.559 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.552+0000 7fc5af3df740 1 --2- 172.21.15.97:0/362309272 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5615396da620 0x5615396dca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:12.559 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.552+0000 7fc5af3df740 4 client.16851 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:35:12.559 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.552+0000 7fc5af3df740 1 -- 172.21.15.97:0/362309272 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561539632390 con 0x5615396da620 2024-04-23T10:35:12.559 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.553+0000 7fc5aa475640 1 --2- 172.21.15.97:0/362309272 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5615396da620 0x5615396dca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:12.559 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.553+0000 7fc58dffb640 10 client.16851 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:35:12.559 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.553+0000 7fc5aa475640 1 --2- 172.21.15.97:0/362309272 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5615396da620 0x5615396dca10 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38f7d765cade3338 server_cookie=5ed66a64a26dd813 in_seq=0 out_seq=0 2024-04-23T10:35:12.559 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.554+0000 7fc58dffb640 1 -- 172.21.15.97:0/362309272 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561539632390 con 0x5615396da620 2024-04-23T10:35:12.559 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.554+0000 7fc58dffb640 10 client.16851 handle_command_reply: tid=0 2024-04-23T10:35:12.559 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.554+0000 7fc5af3df740 1 client.16851 shutdown 2024-04-23T10:35:12.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.554+0000 7fc5af3df740 1 -- 172.21.15.97:0/362309272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615396bff40 msgr2=0x5615396c7470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:12.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.554+0000 7fc5af3df740 1 --2- 172.21.15.97:0/362309272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615396bff40 0x5615396c7470 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fc59409c330 tx=0x7fc5940ce080 comp rx=0 tx=0).stop 2024-04-23T10:35:12.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.554+0000 7fc5af3df740 1 -- 172.21.15.97:0/362309272 shutdown_connections 2024-04-23T10:35:12.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.554+0000 7fc5af3df740 1 --2- 172.21.15.97:0/362309272 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5615396da620 0x5615396dca10 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:12.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.554+0000 7fc5af3df740 1 --2- 172.21.15.97:0/362309272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615396ca800 0x5615396c79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:12.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.554+0000 7fc5af3df740 1 --2- 172.21.15.97:0/362309272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615396bff40 0x5615396c7470 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:12.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.554+0000 7fc58dffb640 0 client.16851 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:35:12.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.554+0000 7fc5af3df740 1 --2- 172.21.15.97:0/362309272 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615396bf620 0x5615396ca180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:12.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.554+0000 7fc5af3df740 1 -- 172.21.15.97:0/362309272 >> 172.21.15.97:0/362309272 conn(0x5615396bbb30 msgr2=0x5615396bacb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:35:12.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.554+0000 7fc5af3df740 1 -- 172.21.15.97:0/362309272 shutdown_connections 2024-04-23T10:35:12.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.554+0000 7fc5af3df740 1 -- 172.21.15.97:0/362309272 wait complete. 2024-04-23T10:35:12.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.554+0000 7fc5af3df740 20 client.16851 trim_cache size 0 max 16384 2024-04-23T10:35:12.572 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.569+0000 7fc5af3df740 1 Processor -- start 2024-04-23T10:35:12.572 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.569+0000 7fc5af3df740 1 -- start start 2024-04-23T10:35:12.572 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.569+0000 7fc5af3df740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615396bf620 0x5615396d6290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:12.572 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.569+0000 7fc5af3df740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615396bff40 0x5615396bab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:12.572 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.569+0000 7fc5af3df740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615396ca800 0x5615396c9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:12.572 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.569+0000 7fc5af3df740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a41afcb0 con 0x5615396bf620 2024-04-23T10:35:12.573 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.569+0000 7fc5af3df740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a41b17f0 con 0x5615396ca800 2024-04-23T10:35:12.573 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.569+0000 7fc5af3df740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a41b0380 con 0x5615396bff40 2024-04-23T10:35:12.573 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.569+0000 7fc5aa475640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615396bf620 0x5615396d6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:12.573 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.569+0000 7fc5aa475640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615396bf620 0x5615396d6290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48202/0 (socket says 172.21.15.97:48202) 2024-04-23T10:35:12.573 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.569+0000 7fc5aa475640 1 -- 172.21.15.97:0/3061842082 learned_addr learned my addr 172.21.15.97:0/3061842082 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:35:12.573 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.569+0000 7fc5a9c74640 1 --2- 172.21.15.97:0/3061842082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615396bff40 0x5615396bab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:12.573 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.569+0000 7fc5aa475640 1 -- 172.21.15.97:0/3061842082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615396bff40 msgr2=0x5615396bab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:35:12.573 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.569+0000 7fc5aa475640 1 --2- 172.21.15.97:0/3061842082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615396bff40 0x5615396bab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:12.573 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.569+0000 7fc5aa475640 1 -- 172.21.15.97:0/3061842082 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615396ca800 msgr2=0x5615396c9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:35:12.573 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.569+0000 7fc5aa475640 1 --2- 172.21.15.97:0/3061842082 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615396ca800 0x5615396c9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:12.573 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.569+0000 7fc5aa475640 1 -- 172.21.15.97:0/3061842082 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615396b7500 con 0x5615396bf620 2024-04-23T10:35:12.573 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.570+0000 7fc5aa475640 1 --2- 172.21.15.97:0/3061842082 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615396bf620 0x5615396d6290 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fc5a002a930 tx=0x7fc5a0005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bf14290a8cee63c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:35:12.573 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.570+0000 7fc58ffff640 1 -- 172.21.15.97:0/3061842082 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a0035040 con 0x5615396bf620 2024-04-23T10:35:12.573 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.570+0000 7fc58ffff640 1 -- 172.21.15.97:0/3061842082 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5a0048070 con 0x5615396bf620 2024-04-23T10:35:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.570+0000 7fc58ffff640 1 -- 172.21.15.97:0/3061842082 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a0005500 con 0x5615396bf620 2024-04-23T10:35:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.570+0000 7fc5af3df740 1 -- 172.21.15.97:0/3061842082 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615396bf620 msgr2=0x5615396d6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.570+0000 7fc5af3df740 1 --2- 172.21.15.97:0/3061842082 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615396bf620 0x5615396d6290 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fc5a002a930 tx=0x7fc5a0005a20 comp rx=0 tx=0).stop 2024-04-23T10:35:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.570+0000 7fc5af3df740 1 -- 172.21.15.97:0/3061842082 shutdown_connections 2024-04-23T10:35:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.570+0000 7fc5af3df740 1 --2- 172.21.15.97:0/3061842082 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615396ca800 0x5615396c9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.570+0000 7fc5af3df740 1 --2- 172.21.15.97:0/3061842082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615396bff40 0x5615396bab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.570+0000 7fc5af3df740 1 --2- 172.21.15.97:0/3061842082 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615396bf620 0x5615396d6290 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.570+0000 7fc5af3df740 1 -- 172.21.15.97:0/3061842082 >> 172.21.15.97:0/3061842082 conn(0x5615396bbb30 msgr2=0x5615396ba420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:35:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.570+0000 7fc5af3df740 1 -- 172.21.15.97:0/3061842082 shutdown_connections 2024-04-23T10:35:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.570+0000 7fc5af3df740 1 -- 172.21.15.97:0/3061842082 wait complete. 2024-04-23T10:35:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.571+0000 7fc5af3df740 1 -- start start 2024-04-23T10:35:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.571+0000 7fc5af3df740 1 Processor -- start 2024-04-23T10:35:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.571+0000 7fc5af3df740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615396bf620 0x5615396c41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.571+0000 7fc5af3df740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615396bff40 0x5615396c46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.571+0000 7fc5af3df740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615396ca800 0x5615396c4c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:12.575 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.571+0000 7fc5af3df740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a41b1120 con 0x5615396bf620 2024-04-23T10:35:12.575 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.571+0000 7fc5af3df740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a41b0a50 con 0x5615396ca800 2024-04-23T10:35:12.575 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.571+0000 7fc5af3df740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a41adcb0 con 0x5615396bff40 2024-04-23T10:35:12.575 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.571+0000 7fc5af3df740 10 client.0 _ll_register_callbacks cb 0x7fc594005b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:35:12.575 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.571+0000 7fc5aa475640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615396bf620 0x5615396c41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:12.575 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.571+0000 7fc5aa475640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615396bf620 0x5615396c41a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48214/0 (socket says 172.21.15.97:48214) 2024-04-23T10:35:12.575 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.571+0000 7fc5aa475640 1 -- 172.21.15.97:0/647352164 learned_addr learned my addr 172.21.15.97:0/647352164 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:35:12.575 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.571+0000 7fc5aac76640 1 --2- 172.21.15.97:0/647352164 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615396ca800 0x5615396c4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:12.575 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.571+0000 7fc5aa475640 1 -- 172.21.15.97:0/647352164 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615396bff40 msgr2=0x5615396c46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:35:12.575 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.571+0000 7fc5aa475640 1 --2- 172.21.15.97:0/647352164 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615396bff40 0x5615396c46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:12.575 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.571+0000 7fc5aa475640 1 -- 172.21.15.97:0/647352164 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615396ca800 msgr2=0x5615396c4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:35:12.575 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.571+0000 7fc5aa475640 1 --2- 172.21.15.97:0/647352164 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615396ca800 0x5615396c4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:12.575 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.571+0000 7fc5aa475640 1 -- 172.21.15.97:0/647352164 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56153973f8d0 con 0x5615396bf620 2024-04-23T10:35:12.575 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.571+0000 7fc5aac76640 1 --2- 172.21.15.97:0/647352164 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615396ca800 0x5615396c4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:35:12.576 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.571+0000 7fc5aa475640 1 --2- 172.21.15.97:0/647352164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615396bf620 0x5615396c41a0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fc5a00041d0 tx=0x7fc5a0004200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=241bf36cbda6d38 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:35:12.576 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.571+0000 7fc58dffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:35:12.576 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.572+0000 7fc58dffb640 1 -- 172.21.15.97:0/647352164 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a0035030 con 0x5615396bf620 2024-04-23T10:35:12.576 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.572+0000 7fc58dffb640 1 -- 172.21.15.97:0/647352164 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8484+0+0 (secure 0 0 0) 0x7fc5a0042070 con 0x5615396bf620 2024-04-23T10:35:12.576 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.572+0000 7fc5af3df740 1 -- 172.21.15.97:0/647352164 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5a0048070 con 0x5615396bf620 2024-04-23T10:35:12.576 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.572+0000 7fc58dffb640 1 -- 172.21.15.97:0/647352164 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc5a0005180 con 0x5615396bf620 2024-04-23T10:35:12.576 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.572+0000 7fc5af3df740 10 client.16857 fetch_fsmap learned FSMap version 26 2024-04-23T10:35:12.576 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.572+0000 7fc5af3df740 1 -- 172.21.15.97:0/647352164 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561539740be0 con 0x5615396bf620 2024-04-23T10:35:12.576 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.572+0000 7fc58dffb640 1 -- 172.21.15.97:0/647352164 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5a0048070 con 0x5615396bf620 2024-04-23T10:35:12.576 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.572+0000 7fc58dffb640 1 -- 172.21.15.97:0/647352164 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc5a00038c0 con 0x5615396bf620 2024-04-23T10:35:12.576 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.572+0000 7fc5af3df740 10 client.16857 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:35:12.576 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.572+0000 7fc5af3df740 10 client.16857 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:35:12.576 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.572+0000 7fc5af3df740 20 client.16857 populate_metadata read hostname 'smithi097' 2024-04-23T10:35:12.576 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.572+0000 7fc5af3df740 1 --2- 172.21.15.97:0/647352164 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x561539741290 0x5615397436a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:12.576 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.572+0000 7fc5af3df740 4 client.16857 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:35:12.577 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.572+0000 7fc5af3df740 1 -- 172.21.15.97:0/647352164 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56153962c200 con 0x561539741290 2024-04-23T10:35:12.577 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.572+0000 7fc5a9c74640 1 --2- 172.21.15.97:0/647352164 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x561539741290 0x5615397436a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:12.577 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.573+0000 7fc5a9c74640 1 --2- 172.21.15.97:0/647352164 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x561539741290 0x5615397436a0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cb1f55f5c0231c9 server_cookie=2a874bb886dbfbc1 in_seq=0 out_seq=0 2024-04-23T10:35:12.577 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.573+0000 7fc58dffb640 10 client.16857 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:35:12.586 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.587+0000 7fc58dffb640 1 -- 172.21.15.97:0/647352164 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56153962c200 con 0x561539741290 2024-04-23T10:35:12.586 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.587+0000 7fc58dffb640 10 client.16857 handle_command_reply: tid=0 2024-04-23T10:35:12.586 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.587+0000 7fc5af3df740 1 client.16857 shutdown 2024-04-23T10:35:12.587 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.587+0000 7fc5af3df740 1 -- 172.21.15.97:0/647352164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615396bf620 msgr2=0x5615396c41a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:12.587 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.587+0000 7fc5af3df740 1 --2- 172.21.15.97:0/647352164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615396bf620 0x5615396c41a0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fc5a00041d0 tx=0x7fc5a0004200 comp rx=0 tx=0).stop 2024-04-23T10:35:12.587 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.587+0000 7fc5af3df740 1 -- 172.21.15.97:0/647352164 shutdown_connections 2024-04-23T10:35:12.587 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.587+0000 7fc5af3df740 1 --2- 172.21.15.97:0/647352164 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x561539741290 0x5615397436a0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:12.587 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.587+0000 7fc5af3df740 1 --2- 172.21.15.97:0/647352164 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615396ca800 0x5615396c4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:12.587 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.587+0000 7fc58dffb640 0 client.16857 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:35:12.587 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.587+0000 7fc5af3df740 1 --2- 172.21.15.97:0/647352164 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5615396bff40 0x5615396c46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:12.587 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.587+0000 7fc5af3df740 1 --2- 172.21.15.97:0/647352164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5615396bf620 0x5615396c41a0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:12.587 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.587+0000 7fc5af3df740 1 -- 172.21.15.97:0/647352164 >> 172.21.15.97:0/647352164 conn(0x5615396bbb30 msgr2=0x5615396bacb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:35:12.587 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.587+0000 7fc5af3df740 1 -- 172.21.15.97:0/647352164 shutdown_connections 2024-04-23T10:35:12.587 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.587+0000 7fc5af3df740 1 -- 172.21.15.97:0/647352164 wait complete. 2024-04-23T10:35:12.587 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.587+0000 7fc5af3df740 20 client.16857 trim_cache size 0 max 16384 2024-04-23T10:35:12.588 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:35:12.588 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:35:12.588 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:35:12.588 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:35:12.589 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.588+0000 7fc5ac6d9640 1 -- 172.21.15.97:0/4178321001 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fc5900a3dc0 msgr2=0x7fc5900a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:12.589 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.588+0000 7fc5ac6d9640 1 --2- 172.21.15.97:0/4178321001 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fc5900a3dc0 0x7fc5900a6280 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc59c00acf0 tx=0x7fc59c00e720 comp rx=0 tx=0).stop 2024-04-23T10:35:12.589 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.588+0000 7fc5ac6d9640 1 -- 172.21.15.97:0/4178321001 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5a4106170 msgr2=0x7fc5a41a0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:12.589 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.588+0000 7fc5ac6d9640 1 --2- 172.21.15.97:0/4178321001 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5a4106170 0x7fc5a41a0b10 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fc59400d770 tx=0x7fc59400dc30 comp rx=0 tx=0).stop 2024-04-23T10:35:12.589 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.588+0000 7fc5ac6d9640 1 -- 172.21.15.97:0/4178321001 shutdown_connections 2024-04-23T10:35:12.589 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.588+0000 7fc5ac6d9640 1 --2- 172.21.15.97:0/4178321001 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fc5900a3dc0 0x7fc5900a6280 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:12.589 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.588+0000 7fc5ac6d9640 1 --2- 172.21.15.97:0/4178321001 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc5a419a690 0x7fc5a419ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:12.589 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.588+0000 7fc5ac6d9640 1 --2- 172.21.15.97:0/4178321001 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5a4106170 0x7fc5a41a0b10 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:12.589 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.588+0000 7fc5ac6d9640 1 --2- 172.21.15.97:0/4178321001 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5a41010f0 0x7fc5a41a05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:12.589 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.588+0000 7fc5ac6d9640 1 -- 172.21.15.97:0/4178321001 >> 172.21.15.97:0/4178321001 conn(0x7fc5a4077fd0 msgr2=0x7fc5a40ff0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:35:12.589 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.588+0000 7fc5ac6d9640 1 -- 172.21.15.97:0/4178321001 shutdown_connections 2024-04-23T10:35:12.589 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:12.588+0000 7fc5ac6d9640 1 -- 172.21.15.97:0/4178321001 wait complete. 2024-04-23T10:35:13.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:35:13 smithi097 ceph-mon[29197]: pgmap v279: 97 pgs: 97 active+clean; 4.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 94 KiB/s wr, 37 op/s 2024-04-23T10:35:13.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:35:13 smithi183 ceph-mon[32317]: pgmap v279: 97 pgs: 97 active+clean; 4.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 94 KiB/s wr, 37 op/s 2024-04-23T10:35:13.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:35:13 smithi196 ceph-mon[32919]: pgmap v279: 97 pgs: 97 active+clean; 4.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 94 KiB/s wr, 37 op/s 2024-04-23T10:35:15.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:35:15 smithi097 ceph-mon[29197]: pgmap v280: 97 pgs: 97 active+clean; 4.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 44 KiB/s wr, 15 op/s 2024-04-23T10:35:15.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:35:15 smithi183 ceph-mon[32317]: pgmap v280: 97 pgs: 97 active+clean; 4.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 44 KiB/s wr, 15 op/s 2024-04-23T10:35:15.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:35:15 smithi196 ceph-mon[32919]: pgmap v280: 97 pgs: 97 active+clean; 4.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 44 KiB/s wr, 15 op/s 2024-04-23T10:35:17.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:35:17 smithi097 ceph-mon[29197]: pgmap v281: 97 pgs: 97 active+clean; 4.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 44 KiB/s wr, 15 op/s 2024-04-23T10:35:17.393 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:35:17 smithi196 ceph-mon[32919]: pgmap v281: 97 pgs: 97 active+clean; 4.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 44 KiB/s wr, 15 op/s 2024-04-23T10:35:17.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:35:17 smithi183 ceph-mon[32317]: pgmap v281: 97 pgs: 97 active+clean; 4.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 44 KiB/s wr, 15 op/s 2024-04-23T10:35:19.307 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 10 2024-04-23T10:35:19.308 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T10:35:19.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:35:19 smithi097 ceph-mon[29197]: pgmap v282: 97 pgs: 97 active+clean; 4.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 64 KiB/s wr, 23 op/s 2024-04-23T10:35:19.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:35:19 smithi183 ceph-mon[32317]: pgmap v282: 97 pgs: 97 active+clean; 4.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 64 KiB/s wr, 23 op/s 2024-04-23T10:35:19.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.420+0000 7fa82ce85640 1 -- 172.21.15.97:0/3202414118 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa828101a40 msgr2=0x7fa828105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:19.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.420+0000 7fa82ce85640 1 --2- 172.21.15.97:0/3202414118 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa828101a40 0x7fa828105a80 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fa810009900 tx=0x7fa81002a410 comp rx=0 tx=0).stop 2024-04-23T10:35:19.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.420+0000 7fa82ce85640 1 -- 172.21.15.97:0/3202414118 shutdown_connections 2024-04-23T10:35:19.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.420+0000 7fa82ce85640 1 --2- 172.21.15.97:0/3202414118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa828106160 0x7fa82810dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:19.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.420+0000 7fa82ce85640 1 --2- 172.21.15.97:0/3202414118 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa828101a40 0x7fa828105a80 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:19.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.420+0000 7fa82ce85640 1 --2- 172.21.15.97:0/3202414118 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa828101090 0x7fa828101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:19.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.420+0000 7fa82ce85640 1 -- 172.21.15.97:0/3202414118 >> 172.21.15.97:0/3202414118 conn(0x7fa8280771d0 msgr2=0x7fa8280775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:35:19.419 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.420+0000 7fa82ce85640 1 -- 172.21.15.97:0/3202414118 shutdown_connections 2024-04-23T10:35:19.420 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.420+0000 7fa82ce85640 1 -- 172.21.15.97:0/3202414118 wait complete. 2024-04-23T10:35:19.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.421+0000 7fa82ce85640 1 Processor -- start 2024-04-23T10:35:19.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.421+0000 7fa82ce85640 1 -- start start 2024-04-23T10:35:19.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.421+0000 7fa82ce85640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa828101090 0x7fa828198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:19.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.421+0000 7fa82ce85640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa828106160 0x7fa828198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:19.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.421+0000 7fa82ce85640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa82819cfe0 0x7fa82819d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:19.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.421+0000 7fa82ce85640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa828112200 con 0x7fa828101090 2024-04-23T10:35:19.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.421+0000 7fa82ce85640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa828112080 con 0x7fa82819cfe0 2024-04-23T10:35:19.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.421+0000 7fa82ce85640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa828112380 con 0x7fa828106160 2024-04-23T10:35:19.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.421+0000 7fa82659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa828101090 0x7fa828198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:19.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.421+0000 7fa82659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa828101090 0x7fa828198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56976/0 (socket says 172.21.15.97:56976) 2024-04-23T10:35:19.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.421+0000 7fa82659c640 1 -- 172.21.15.97:0/2408734947 learned_addr learned my addr 172.21.15.97:0/2408734947 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:35:19.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.421+0000 7fa81ffff640 1 --2- 172.21.15.97:0/2408734947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa828106160 0x7fa828198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:19.422 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.421+0000 7fa826d9d640 1 --2- 172.21.15.97:0/2408734947 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa82819cfe0 0x7fa82819d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:19.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.422+0000 7fa82659c640 1 -- 172.21.15.97:0/2408734947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa828106160 msgr2=0x7fa828198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:19.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.422+0000 7fa82659c640 1 --2- 172.21.15.97:0/2408734947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa828106160 0x7fa828198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:19.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.422+0000 7fa82659c640 1 -- 172.21.15.97:0/2408734947 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa82819cfe0 msgr2=0x7fa82819d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:19.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.422+0000 7fa82659c640 1 --2- 172.21.15.97:0/2408734947 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa82819cfe0 0x7fa82819d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:19.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.422+0000 7fa82659c640 1 -- 172.21.15.97:0/2408734947 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa82819dad0 con 0x7fa828101090 2024-04-23T10:35:19.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.422+0000 7fa826d9d640 1 --2- 172.21.15.97:0/2408734947 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa82819cfe0 0x7fa82819d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:35:19.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.422+0000 7fa82659c640 1 --2- 172.21.15.97:0/2408734947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa828101090 0x7fa828198720 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fa81800b4a0 tx=0x7fa81800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:35:19.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.422+0000 7fa81f7fe640 1 -- 172.21.15.97:0/2408734947 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa818014070 con 0x7fa828101090 2024-04-23T10:35:19.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.422+0000 7fa81f7fe640 1 -- 172.21.15.97:0/2408734947 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa818010ac0 con 0x7fa828101090 2024-04-23T10:35:19.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.422+0000 7fa82ce85640 1 -- 172.21.15.97:0/2408734947 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa82819ddc0 con 0x7fa828101090 2024-04-23T10:35:19.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.422+0000 7fa81f7fe640 1 -- 172.21.15.97:0/2408734947 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa818017c30 con 0x7fa828101090 2024-04-23T10:35:19.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.422+0000 7fa82ce85640 1 -- 172.21.15.97:0/2408734947 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8281a5580 con 0x7fa828101090 2024-04-23T10:35:19.423 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.423+0000 7fa82ce85640 1 -- 172.21.15.97:0/2408734947 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8280691c0 con 0x7fa828101090 2024-04-23T10:35:19.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.424+0000 7fa81f7fe640 1 -- 172.21.15.97:0/2408734947 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa818017070 con 0x7fa828101090 2024-04-23T10:35:19.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.425+0000 7fa81f7fe640 1 --2- 172.21.15.97:0/2408734947 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fa8140a3d20 0x7fa8140a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:19.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.425+0000 7fa81ffff640 1 --2- 172.21.15.97:0/2408734947 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fa8140a3d20 0x7fa8140a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:19.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.425+0000 7fa81f7fe640 1 -- 172.21.15.97:0/2408734947 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8484+0+0 (secure 0 0 0) 0x7fa8180cfe10 con 0x7fa828101090 2024-04-23T10:35:19.424 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.425+0000 7fa81ffff640 1 --2- 172.21.15.97:0/2408734947 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fa8140a3d20 0x7fa8140a61e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa810000c00 tx=0x7fa810005e80 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:35:19.427 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.428+0000 7fa81f7fe640 1 -- 172.21.15.97:0/2408734947 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa81809c5e0 con 0x7fa828101090 2024-04-23T10:35:19.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:35:19 smithi196 ceph-mon[32919]: pgmap v282: 97 pgs: 97 active+clean; 4.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 64 KiB/s wr, 23 op/s 2024-04-23T10:35:19.642 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.644+0000 7fa82ce85640 1 -- 172.21.15.97:0/2408734947 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa8281093f0 con 0x7fa828101090 2024-04-23T10:35:19.643 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.644+0000 7fa81f7fe640 1 -- 172.21.15.97:0/2408734947 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11931 (secure 0 0 0) 0x7fa81809bee0 con 0x7fa828101090 2024-04-23T10:35:19.643 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:35:19.643 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3946878762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3946878762},{"type":"v1","addr":"172.21.15.97:6835","nonce":3946878762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/209800861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":209800861},{"type":"v1","addr":"172.21.15.97:6837","nonce":209800861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T10:24:06.910134+0000","modified":"2024-04-23T10:26:09.820985+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6839/418013575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":418013575},{"type":"v1","addr":"172.21.15.97:6839","nonce":418013575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6841/747242680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":747242680},{"type":"v1","addr":"172.21.15.97:6841","nonce":747242680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6833/4292847121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4292847121},{"type":"v1","addr":"172.21.15.183:6833","nonce":4292847121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6833/3885816565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3885816565},{"type":"v1","addr":"172.21.15.196:6833","nonce":3885816565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1897093105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1897093105},{"type":"v1","addr":"172.21.15.183:6835","nonce":1897093105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1171477563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1171477563},{"type":"v1","addr":"172.21.15.196:6835","nonce":1171477563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1898341911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1898341911},{"type":"v1","addr":"172.21.15.183:6837","nonce":1898341911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.196:6837/3094946502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3094946502},{"type":"v1","addr":"172.21.15.196:6837","nonce":3094946502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.196:6839/83843556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":83843556},{"type":"v1","addr":"172.21.15.196:6839","nonce":83843556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6839/1763841130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1763841130},{"type":"v1","addr":"172.21.15.183:6839","nonce":1763841130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24487,24403,24398,24469,24488]},"id":1}]} 2024-04-23T10:35:19.644 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 26 2024-04-23T10:35:19.645 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.646+0000 7fa82ce85640 1 -- 172.21.15.97:0/2408734947 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fa8140a3d20 msgr2=0x7fa8140a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:19.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.646+0000 7fa82ce85640 1 --2- 172.21.15.97:0/2408734947 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fa8140a3d20 0x7fa8140a61e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa810000c00 tx=0x7fa810005e80 comp rx=0 tx=0).stop 2024-04-23T10:35:19.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.646+0000 7fa82ce85640 1 -- 172.21.15.97:0/2408734947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa828101090 msgr2=0x7fa828198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:19.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.646+0000 7fa82ce85640 1 --2- 172.21.15.97:0/2408734947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa828101090 0x7fa828198720 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fa81800b4a0 tx=0x7fa81800b960 comp rx=0 tx=0).stop 2024-04-23T10:35:19.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.646+0000 7fa82ce85640 1 -- 172.21.15.97:0/2408734947 shutdown_connections 2024-04-23T10:35:19.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.646+0000 7fa82ce85640 1 --2- 172.21.15.97:0/2408734947 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fa8140a3d20 0x7fa8140a61e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:19.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.646+0000 7fa82ce85640 1 --2- 172.21.15.97:0/2408734947 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa82819cfe0 0x7fa82819d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:19.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.646+0000 7fa82ce85640 1 --2- 172.21.15.97:0/2408734947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa828106160 0x7fa828198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:19.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.646+0000 7fa82ce85640 1 --2- 172.21.15.97:0/2408734947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa828101090 0x7fa828198720 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:19.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.646+0000 7fa82ce85640 1 -- 172.21.15.97:0/2408734947 >> 172.21.15.97:0/2408734947 conn(0x7fa8280771d0 msgr2=0x7fa8280fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:35:19.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.646+0000 7fa82ce85640 1 -- 172.21.15.97:0/2408734947 shutdown_connections 2024-04-23T10:35:19.646 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.646+0000 7fa82ce85640 1 -- 172.21.15.97:0/2408734947 wait complete. 2024-04-23T10:35:19.656 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db / --timeout 60 --expiration 70 --await-for 42.0' 2024-04-23T10:35:19.656 DEBUG:teuthology.orchestra.run.smithi097:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24488 quiesce db / --timeout 60 --expiration 70 --await-for 42.0 2024-04-23T10:35:19.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.765+0000 7f733ea7c640 1 -- 172.21.15.97:0/1884107265 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73381021a0 msgr2=0x7f7338102580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:19.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.765+0000 7f733ea7c640 1 --2- 172.21.15.97:0/1884107265 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73381021a0 0x7f7338102580 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f7330009ca0 tx=0x7f733002a8c0 comp rx=0 tx=0).stop 2024-04-23T10:35:19.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.765+0000 7f733ea7c640 1 -- 172.21.15.97:0/1884107265 shutdown_connections 2024-04-23T10:35:19.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.765+0000 7f733ea7c640 1 --2- 172.21.15.97:0/1884107265 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7338107270 0x7f733810ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:19.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.765+0000 7f733ea7c640 1 --2- 172.21.15.97:0/1884107265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7338102b50 0x7f7338106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:19.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.765+0000 7f733ea7c640 1 --2- 172.21.15.97:0/1884107265 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73381021a0 0x7f7338102580 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:19.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.765+0000 7f733ea7c640 1 -- 172.21.15.97:0/1884107265 >> 172.21.15.97:0/1884107265 conn(0x7f7338079350 msgr2=0x7f7338079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:35:19.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.765+0000 7f733ea7c640 1 -- 172.21.15.97:0/1884107265 shutdown_connections 2024-04-23T10:35:19.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.765+0000 7f733ea7c640 1 -- 172.21.15.97:0/1884107265 wait complete. 2024-04-23T10:35:19.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.766+0000 7f733ea7c640 1 Processor -- start 2024-04-23T10:35:19.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.766+0000 7f733ea7c640 1 -- start start 2024-04-23T10:35:19.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.766+0000 7f733ea7c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73381021a0 0x7f733819e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:19.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.766+0000 7f733ea7c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7338107270 0x7f733819e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:19.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.766+0000 7f733ea7c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7338198600 0x7f7338198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:19.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.766+0000 7f733ea7c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f73381121b0 con 0x7f73381021a0 2024-04-23T10:35:19.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.766+0000 7f733ea7c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7338112030 con 0x7f7338198600 2024-04-23T10:35:19.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.766+0000 7f733ea7c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7338112330 con 0x7f7338107270 2024-04-23T10:35:19.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.766+0000 7f733c818640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73381021a0 0x7f733819e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:19.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.766+0000 7f733c818640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73381021a0 0x7f733819e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:56992/0 (socket says 172.21.15.97:56992) 2024-04-23T10:35:19.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.766+0000 7f733c818640 1 -- 172.21.15.97:0/3240804528 learned_addr learned my addr 172.21.15.97:0/3240804528 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:35:19.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.767+0000 7f7337fff640 1 --2- 172.21.15.97:0/3240804528 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7338107270 0x7f733819e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:19.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.767+0000 7f733c818640 1 -- 172.21.15.97:0/3240804528 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7338107270 msgr2=0x7f733819e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:19.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.767+0000 7f733d019640 1 --2- 172.21.15.97:0/3240804528 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7338198600 0x7f7338198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:19.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.767+0000 7f733c818640 1 --2- 172.21.15.97:0/3240804528 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7338107270 0x7f733819e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:19.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.767+0000 7f733c818640 1 -- 172.21.15.97:0/3240804528 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7338198600 msgr2=0x7f7338198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:19.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.767+0000 7f733c818640 1 --2- 172.21.15.97:0/3240804528 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7338198600 0x7f7338198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:19.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.767+0000 7f733c818640 1 -- 172.21.15.97:0/3240804528 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7338199120 con 0x7f73381021a0 2024-04-23T10:35:19.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.767+0000 7f7337fff640 1 --2- 172.21.15.97:0/3240804528 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7338107270 0x7f733819e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:35:19.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.767+0000 7f733c818640 1 --2- 172.21.15.97:0/3240804528 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73381021a0 0x7f733819e400 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f7330006010 tx=0x7f73300319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:35:19.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.767+0000 7f7335ffb640 1 -- 172.21.15.97:0/3240804528 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7330003bf0 con 0x7f73381021a0 2024-04-23T10:35:19.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.767+0000 7f7335ffb640 1 -- 172.21.15.97:0/3240804528 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7330003d90 con 0x7f73381021a0 2024-04-23T10:35:19.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.767+0000 7f733ea7c640 1 -- 172.21.15.97:0/3240804528 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73381993b0 con 0x7f73381021a0 2024-04-23T10:35:19.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.767+0000 7f7335ffb640 1 -- 172.21.15.97:0/3240804528 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733002fc20 con 0x7f73381021a0 2024-04-23T10:35:19.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.767+0000 7f733ea7c640 1 -- 172.21.15.97:0/3240804528 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73381a55b0 con 0x7f73381021a0 2024-04-23T10:35:19.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.769+0000 7f7335ffb640 1 -- 172.21.15.97:0/3240804528 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f73300046a0 con 0x7f73381021a0 2024-04-23T10:35:19.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.769+0000 7f7341794740 1 Processor -- start 2024-04-23T10:35:19.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.769+0000 7f7341794740 1 -- start start 2024-04-23T10:35:19.775 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.770+0000 7f7341794740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572efdbffa0 0x5572efdc0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:19.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.770+0000 7f7335ffb640 1 --2- 172.21.15.97:0/3240804528 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f73240a3d20 0x7f73240a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:19.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.770+0000 7f7341794740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572efdc08c0 0x5572efdcef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:19.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.770+0000 7f7341794740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572efdcf4a0 0x5572efdd1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:19.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.770+0000 7f7341794740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5572efd25370 con 0x5572efdbffa0 2024-04-23T10:35:19.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.770+0000 7f7341794740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5572efcc22e0 con 0x5572efdcf4a0 2024-04-23T10:35:19.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.770+0000 7f7341794740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5572efd21330 con 0x5572efdc08c0 2024-04-23T10:35:19.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.770+0000 7f733d019640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572efdbffa0 0x5572efdc0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:19.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.770+0000 7f733c818640 1 --2- 172.21.15.97:0/3240804528 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f73240a3d20 0x7f73240a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:19.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.770+0000 7f733d019640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572efdbffa0 0x5572efdc0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57002/0 (socket says 172.21.15.97:57002) 2024-04-23T10:35:19.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.770+0000 7f733d019640 1 -- 172.21.15.97:0/254665285 learned_addr learned my addr 172.21.15.97:0/254665285 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:35:19.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.770+0000 7f7337fff640 1 --2- 172.21.15.97:0/254665285 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572efdc08c0 0x5572efdcef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:19.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.770+0000 7f7335ffb640 1 -- 172.21.15.97:0/3240804528 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8484+0+0 (secure 0 0 0) 0x7f73300f1d10 con 0x7f73381021a0 2024-04-23T10:35:19.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.770+0000 7f733d019640 1 --2- 172.21.15.97:0/254665285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572efdcf4a0 0x5572efdd1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:19.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.770+0000 7f733d019640 1 -- 172.21.15.97:0/254665285 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572efdc08c0 msgr2=0x5572efdcef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:35:19.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.770+0000 7f733d019640 1 --2- 172.21.15.97:0/254665285 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572efdc08c0 0x5572efdcef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:19.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.770+0000 7f733d019640 1 -- 172.21.15.97:0/254665285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572efdcf4a0 msgr2=0x5572efdd1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:35:19.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.770+0000 7f733d019640 1 --2- 172.21.15.97:0/254665285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572efdcf4a0 0x5572efdd1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:19.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.770+0000 7f733d019640 1 -- 172.21.15.97:0/254665285 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572efb03d80 con 0x5572efdbffa0 2024-04-23T10:35:19.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.770+0000 7f7337fff640 1 --2- 172.21.15.97:0/254665285 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572efdc08c0 0x5572efdcef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:35:19.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.770+0000 7f733c818640 1 --2- 172.21.15.97:0/3240804528 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f73240a3d20 0x7f73240a61e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f73300ef640 tx=0x7f73300095c0 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:35:19.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.770+0000 7f733d019640 1 --2- 172.21.15.97:0/254665285 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572efdbffa0 0x5572efdc0380 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f7320010340 tx=0x7f7320009480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40aedecaae8f4efb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:35:19.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.770+0000 7f732effd640 1 -- 172.21.15.97:0/254665285 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f732000d610 con 0x5572efdbffa0 2024-04-23T10:35:19.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.771+0000 7f732effd640 1 -- 172.21.15.97:0/254665285 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f732001b070 con 0x5572efdbffa0 2024-04-23T10:35:19.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.771+0000 7f732effd640 1 -- 172.21.15.97:0/254665285 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7320013b00 con 0x5572efdbffa0 2024-04-23T10:35:19.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.771+0000 7f7341794740 1 -- 172.21.15.97:0/254665285 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572efdbffa0 msgr2=0x5572efdc0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:19.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.771+0000 7f7341794740 1 --2- 172.21.15.97:0/254665285 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572efdbffa0 0x5572efdc0380 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f7320010340 tx=0x7f7320009480 comp rx=0 tx=0).stop 2024-04-23T10:35:19.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.771+0000 7f7341794740 1 -- 172.21.15.97:0/254665285 shutdown_connections 2024-04-23T10:35:19.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.771+0000 7f7341794740 1 --2- 172.21.15.97:0/254665285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572efdcf4a0 0x5572efdd1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:19.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.771+0000 7f7341794740 1 --2- 172.21.15.97:0/254665285 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572efdc08c0 0x5572efdcef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:19.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.771+0000 7f7341794740 1 --2- 172.21.15.97:0/254665285 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572efdbffa0 0x5572efdc0380 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:19.777 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.771+0000 7f7341794740 1 -- 172.21.15.97:0/254665285 >> 172.21.15.97:0/254665285 conn(0x5572efdbc3c0 msgr2=0x5572efdbc7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:35:19.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.771+0000 7f7341794740 1 -- 172.21.15.97:0/254665285 shutdown_connections 2024-04-23T10:35:19.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.771+0000 7f7341794740 1 -- 172.21.15.97:0/254665285 wait complete. 2024-04-23T10:35:19.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.771+0000 7f7341794740 1 -- start start 2024-04-23T10:35:19.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.772+0000 7f7341794740 1 Processor -- start 2024-04-23T10:35:19.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.772+0000 7f7341794740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572efdbffa0 0x5572efdcc420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:19.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.772+0000 7f7341794740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572efdc08c0 0x5572efdcc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:19.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.772+0000 7f7341794740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572efdcf4a0 0x5572efdccea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:19.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.772+0000 7f7341794740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5572efdd2200 con 0x5572efdbffa0 2024-04-23T10:35:19.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.772+0000 7f7341794740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5572efcbffb0 con 0x5572efdc08c0 2024-04-23T10:35:19.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.772+0000 7f7341794740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5572efdd2380 con 0x5572efdcf4a0 2024-04-23T10:35:19.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.772+0000 7f7341794740 10 client.0 _ll_register_callbacks cb 0x5572efcbf9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:35:19.778 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.772+0000 7f733d019640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572efdbffa0 0x5572efdcc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:19.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.772+0000 7f733d019640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572efdbffa0 0x5572efdcc420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57006/0 (socket says 172.21.15.97:57006) 2024-04-23T10:35:19.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.772+0000 7f733d019640 1 -- 172.21.15.97:0/1935840073 learned_addr learned my addr 172.21.15.97:0/1935840073 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:35:19.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.772+0000 7f733d019640 1 -- 172.21.15.97:0/1935840073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572efdcf4a0 msgr2=0x5572efdccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:35:19.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.772+0000 7f733d019640 1 --2- 172.21.15.97:0/1935840073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572efdcf4a0 0x5572efdccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:19.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.772+0000 7f733d019640 1 -- 172.21.15.97:0/1935840073 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572efdc08c0 msgr2=0x5572efdcc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:35:19.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.772+0000 7f7337fff640 1 --2- 172.21.15.97:0/1935840073 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572efdc08c0 0x5572efdcc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:19.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.772+0000 7f733d019640 1 --2- 172.21.15.97:0/1935840073 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572efdc08c0 0x5572efdcc960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:19.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.772+0000 7f733d019640 1 -- 172.21.15.97:0/1935840073 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572efb64510 con 0x5572efdbffa0 2024-04-23T10:35:19.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.773+0000 7f733d019640 1 --2- 172.21.15.97:0/1935840073 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572efdbffa0 0x5572efdcc420 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f7320010310 tx=0x7f73200134a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b70baad961cf7852 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:35:19.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.773+0000 7f7317fff640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:35:19.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.773+0000 7f7317fff640 1 -- 172.21.15.97:0/1935840073 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7320025070 con 0x5572efdbffa0 2024-04-23T10:35:19.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.773+0000 7f7317fff640 1 -- 172.21.15.97:0/1935840073 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8484+0+0 (secure 0 0 0) 0x7f7320005680 con 0x5572efdbffa0 2024-04-23T10:35:19.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.773+0000 7f7341794740 1 -- 172.21.15.97:0/1935840073 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f732001b070 con 0x5572efdbffa0 2024-04-23T10:35:19.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.773+0000 7f7317fff640 1 -- 172.21.15.97:0/1935840073 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7320010e80 con 0x5572efdbffa0 2024-04-23T10:35:19.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.773+0000 7f7341794740 10 client.16875 fetch_fsmap learned FSMap version 26 2024-04-23T10:35:19.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.773+0000 7f7341794740 1 -- 172.21.15.97:0/1935840073 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5572ef9f7670 con 0x5572efdbffa0 2024-04-23T10:35:19.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.773+0000 7f7317fff640 1 -- 172.21.15.97:0/1935840073 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f732001b070 con 0x5572efdbffa0 2024-04-23T10:35:19.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.773+0000 7f7317fff640 1 -- 172.21.15.97:0/1935840073 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f732000c050 con 0x5572efdbffa0 2024-04-23T10:35:19.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.773+0000 7f7341794740 10 client.16875 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:35:19.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.773+0000 7f7341794740 10 client.16875 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:35:19.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.773+0000 7f7341794740 20 client.16875 populate_metadata read hostname 'smithi097' 2024-04-23T10:35:19.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.773+0000 7f7341794740 1 --2- 172.21.15.97:0/1935840073 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5572efddf6a0 0x5572efde1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:19.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.773+0000 7f7341794740 4 client.16875 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:35:19.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.773+0000 7f7341794740 1 -- 172.21.15.97:0/1935840073 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5572efd32d90 con 0x5572efddf6a0 2024-04-23T10:35:19.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.774+0000 7f7337fff640 1 --2- 172.21.15.97:0/1935840073 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5572efddf6a0 0x5572efde1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:19.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.774+0000 7f7337fff640 1 --2- 172.21.15.97:0/1935840073 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5572efddf6a0 0x5572efde1a90 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fd867d891b3890d server_cookie=854cd1e0d79ce0ab in_seq=0 out_seq=0 2024-04-23T10:35:19.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.774+0000 7f7317fff640 10 client.16875 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:35:19.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.775+0000 7f7317fff640 1 -- 172.21.15.97:0/1935840073 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5572efd32d90 con 0x5572efddf6a0 2024-04-23T10:35:19.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.775+0000 7f7317fff640 10 client.16875 handle_command_reply: tid=0 2024-04-23T10:35:19.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.775+0000 7f7341794740 1 client.16875 shutdown 2024-04-23T10:35:19.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.775+0000 7f7341794740 1 -- 172.21.15.97:0/1935840073 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572efdbffa0 msgr2=0x5572efdcc420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:19.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.775+0000 7f7341794740 1 --2- 172.21.15.97:0/1935840073 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572efdbffa0 0x5572efdcc420 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f7320010310 tx=0x7f73200134a0 comp rx=0 tx=0).stop 2024-04-23T10:35:19.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.775+0000 7f7341794740 1 -- 172.21.15.97:0/1935840073 shutdown_connections 2024-04-23T10:35:19.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.775+0000 7f7341794740 1 --2- 172.21.15.97:0/1935840073 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5572efddf6a0 0x5572efde1a90 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:19.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.775+0000 7f7341794740 1 --2- 172.21.15.97:0/1935840073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572efdcf4a0 0x5572efdccea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:19.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.775+0000 7f7341794740 1 --2- 172.21.15.97:0/1935840073 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572efdc08c0 0x5572efdcc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:19.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.775+0000 7f7317fff640 0 client.16875 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:35:19.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.775+0000 7f7341794740 1 --2- 172.21.15.97:0/1935840073 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572efdbffa0 0x5572efdcc420 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:19.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.775+0000 7f7341794740 1 -- 172.21.15.97:0/1935840073 >> 172.21.15.97:0/1935840073 conn(0x5572efdbc3c0 msgr2=0x5572efdc21f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:35:19.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.775+0000 7f7341794740 1 -- 172.21.15.97:0/1935840073 shutdown_connections 2024-04-23T10:35:19.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.775+0000 7f7341794740 1 -- 172.21.15.97:0/1935840073 wait complete. 2024-04-23T10:35:19.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.775+0000 7f7341794740 20 client.16875 trim_cache size 0 max 16384 2024-04-23T10:35:19.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.792+0000 7f7341794740 1 Processor -- start 2024-04-23T10:35:19.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.792+0000 7f7341794740 1 -- start start 2024-04-23T10:35:19.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.792+0000 7f7341794740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572efdbffa0 0x5572efdbbd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:19.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.792+0000 7f7341794740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572efdc08c0 0x5572efddb100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:19.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.792+0000 7f7341794740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572efdcf4a0 0x5572efdcb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:19.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.792+0000 7f7341794740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f73381aba00 con 0x5572efdbffa0 2024-04-23T10:35:19.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.792+0000 7f7341794740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f73381ad540 con 0x5572efdc08c0 2024-04-23T10:35:19.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.792+0000 7f7341794740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f73381ac0d0 con 0x5572efdcf4a0 2024-04-23T10:35:19.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.792+0000 7f733d019640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572efdbffa0 0x5572efdbbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:19.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.792+0000 7f733d019640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572efdbffa0 0x5572efdbbd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57016/0 (socket says 172.21.15.97:57016) 2024-04-23T10:35:19.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.792+0000 7f733d019640 1 -- 172.21.15.97:0/1595884890 learned_addr learned my addr 172.21.15.97:0/1595884890 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:35:19.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.792+0000 7f7337fff640 1 --2- 172.21.15.97:0/1595884890 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572efdc08c0 0x5572efddb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:19.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.792+0000 7f733d019640 1 -- 172.21.15.97:0/1595884890 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572efdcf4a0 msgr2=0x5572efdcb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:35:19.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.792+0000 7f733d019640 1 --2- 172.21.15.97:0/1595884890 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572efdcf4a0 0x5572efdcb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:19.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.792+0000 7f733d019640 1 -- 172.21.15.97:0/1595884890 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572efdc08c0 msgr2=0x5572efddb100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:35:19.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.792+0000 7f733d019640 1 --2- 172.21.15.97:0/1595884890 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572efdc08c0 0x5572efddb100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:19.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.792+0000 7f733d019640 1 -- 172.21.15.97:0/1595884890 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572efdca970 con 0x5572efdbffa0 2024-04-23T10:35:19.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.793+0000 7f733d019640 1 --2- 172.21.15.97:0/1595884890 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572efdbffa0 0x5572efdbbd80 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f7320005f80 tx=0x7f7320005910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94ab408f9b8c7c05 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:35:19.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.793+0000 7f732e7fc640 1 -- 172.21.15.97:0/1595884890 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7320025070 con 0x5572efdbffa0 2024-04-23T10:35:19.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.793+0000 7f732e7fc640 1 -- 172.21.15.97:0/1595884890 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f732001b040 con 0x5572efdbffa0 2024-04-23T10:35:19.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.793+0000 7f732e7fc640 1 -- 172.21.15.97:0/1595884890 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7320005480 con 0x5572efdbffa0 2024-04-23T10:35:19.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.793+0000 7f7341794740 1 -- 172.21.15.97:0/1595884890 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572efdbffa0 msgr2=0x5572efdbbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:19.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.793+0000 7f7341794740 1 --2- 172.21.15.97:0/1595884890 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572efdbffa0 0x5572efdbbd80 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f7320005f80 tx=0x7f7320005910 comp rx=0 tx=0).stop 2024-04-23T10:35:19.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.793+0000 7f7341794740 1 -- 172.21.15.97:0/1595884890 shutdown_connections 2024-04-23T10:35:19.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.793+0000 7f7341794740 1 --2- 172.21.15.97:0/1595884890 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572efdcf4a0 0x5572efdcb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:19.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.793+0000 7f7341794740 1 --2- 172.21.15.97:0/1595884890 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572efdc08c0 0x5572efddb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:19.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.793+0000 7f7341794740 1 --2- 172.21.15.97:0/1595884890 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572efdbffa0 0x5572efdbbd80 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:19.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.793+0000 7f7341794740 1 -- 172.21.15.97:0/1595884890 >> 172.21.15.97:0/1595884890 conn(0x5572efdbc3c0 msgr2=0x5572efdcabc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:35:19.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.793+0000 7f7341794740 1 -- 172.21.15.97:0/1595884890 shutdown_connections 2024-04-23T10:35:19.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.793+0000 7f7341794740 1 -- 172.21.15.97:0/1595884890 wait complete. 2024-04-23T10:35:19.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.794+0000 7f7341794740 1 -- start start 2024-04-23T10:35:19.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.794+0000 7f7341794740 1 Processor -- start 2024-04-23T10:35:19.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.794+0000 7f7341794740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572efdbffa0 0x5572efdd30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:19.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.794+0000 7f7341794740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572efdc08c0 0x5572efdd3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:19.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.794+0000 7f7341794740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572efdcf4a0 0x5572efdd3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:19.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.794+0000 7f7341794740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f73381ace70 con 0x5572efdcf4a0 2024-04-23T10:35:19.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.794+0000 7f7341794740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f73381ac7a0 con 0x5572efdbffa0 2024-04-23T10:35:19.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.794+0000 7f7341794740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f73381a9a00 con 0x5572efdc08c0 2024-04-23T10:35:19.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.794+0000 7f7341794740 10 client.0 _ll_register_callbacks cb 0x7f7320014d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:35:19.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.794+0000 7f733d019640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572efdcf4a0 0x5572efdd3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:19.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.794+0000 7f733d019640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572efdcf4a0 0x5572efdd3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57026/0 (socket says 172.21.15.97:57026) 2024-04-23T10:35:19.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.794+0000 7f733d019640 1 -- 172.21.15.97:0/2063738717 learned_addr learned my addr 172.21.15.97:0/2063738717 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:35:19.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.794+0000 7f733d019640 1 --2- 172.21.15.97:0/2063738717 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572efdbffa0 0x5572efdd30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:19.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.794+0000 7f7337fff640 1 --2- 172.21.15.97:0/2063738717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572efdc08c0 0x5572efdd3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:19.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.795+0000 7f733d019640 1 -- 172.21.15.97:0/2063738717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572efdc08c0 msgr2=0x5572efdd3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:35:19.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.795+0000 7f733d019640 1 --2- 172.21.15.97:0/2063738717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572efdc08c0 0x5572efdd3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:19.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.795+0000 7f733d019640 1 -- 172.21.15.97:0/2063738717 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572efdbffa0 msgr2=0x5572efdd30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:35:19.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.795+0000 7f733d019640 1 --2- 172.21.15.97:0/2063738717 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572efdbffa0 0x5572efdd30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:19.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.795+0000 7f733d019640 1 -- 172.21.15.97:0/2063738717 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572efe44a40 con 0x5572efdcf4a0 2024-04-23T10:35:19.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.795+0000 7f733d019640 1 --2- 172.21.15.97:0/2063738717 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572efdcf4a0 0x5572efdd3b40 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f7320010d90 tx=0x7f73200147e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c3366da947f41e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:35:19.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.795+0000 7f7317fff640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:35:19.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.795+0000 7f7317fff640 1 -- 172.21.15.97:0/2063738717 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7320025070 con 0x5572efdcf4a0 2024-04-23T10:35:19.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.795+0000 7f7341794740 1 -- 172.21.15.97:0/2063738717 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f732001b040 con 0x5572efdcf4a0 2024-04-23T10:35:19.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.795+0000 7f7317fff640 1 -- 172.21.15.97:0/2063738717 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8484+0+0 (secure 0 0 0) 0x7f7320014040 con 0x5572efdcf4a0 2024-04-23T10:35:19.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.795+0000 7f7317fff640 1 -- 172.21.15.97:0/2063738717 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f732004c050 con 0x5572efdcf4a0 2024-04-23T10:35:19.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.795+0000 7f7341794740 10 client.16881 fetch_fsmap learned FSMap version 26 2024-04-23T10:35:19.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.795+0000 7f7341794740 1 -- 172.21.15.97:0/2063738717 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5572efe45ce0 con 0x5572efdcf4a0 2024-04-23T10:35:19.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.795+0000 7f7317fff640 1 -- 172.21.15.97:0/2063738717 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f732001b040 con 0x5572efdcf4a0 2024-04-23T10:35:19.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.795+0000 7f7317fff640 1 -- 172.21.15.97:0/2063738717 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f732000c040 con 0x5572efdcf4a0 2024-04-23T10:35:19.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.795+0000 7f7341794740 10 client.16881 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:35:19.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.795+0000 7f7341794740 10 client.16881 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:35:19.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.795+0000 7f7341794740 20 client.16881 populate_metadata read hostname 'smithi097' 2024-04-23T10:35:19.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.795+0000 7f7341794740 1 --2- 172.21.15.97:0/2063738717 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5572efe46300 0x5572efe486f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:19.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.795+0000 7f7341794740 4 client.16881 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 70.0, "await_for": 42.0}] 2024-04-23T10:35:19.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.795+0000 7f7341794740 1 -- 172.21.15.97:0/2063738717 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 70.0, "await_for": 42.0}) -- 0x5572efd2cc00 con 0x5572efe46300 2024-04-23T10:35:19.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.796+0000 7f7337fff640 1 --2- 172.21.15.97:0/2063738717 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5572efe46300 0x5572efe486f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:19.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.796+0000 7f7317fff640 10 client.16881 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:35:19.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.796+0000 7f7337fff640 1 --2- 172.21.15.97:0/2063738717 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5572efe46300 0x5572efe486f0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2520f51f18f66158 server_cookie=c9d7ff7e6030f599 in_seq=0 out_seq=0 2024-04-23T10:35:19.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.813+0000 7f7317fff640 1 -- 172.21.15.97:0/2063738717 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5572efd2cc00 con 0x5572efe46300 2024-04-23T10:35:19.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.813+0000 7f7317fff640 10 client.16881 handle_command_reply: tid=0 2024-04-23T10:35:19.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.813+0000 7f7341794740 1 client.16881 shutdown 2024-04-23T10:35:19.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.813+0000 7f7341794740 1 -- 172.21.15.97:0/2063738717 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572efdcf4a0 msgr2=0x5572efdd3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:19.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.813+0000 7f7341794740 1 --2- 172.21.15.97:0/2063738717 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572efdcf4a0 0x5572efdd3b40 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f7320010d90 tx=0x7f73200147e0 comp rx=0 tx=0).stop 2024-04-23T10:35:19.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.813+0000 7f7341794740 1 -- 172.21.15.97:0/2063738717 shutdown_connections 2024-04-23T10:35:19.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.813+0000 7f7341794740 1 --2- 172.21.15.97:0/2063738717 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5572efe46300 0x5572efe486f0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:19.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.813+0000 7f7341794740 1 --2- 172.21.15.97:0/2063738717 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5572efdcf4a0 0x5572efdd3b40 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:19.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.813+0000 7f7341794740 1 --2- 172.21.15.97:0/2063738717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5572efdc08c0 0x5572efdd3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:19.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.813+0000 7f7341794740 1 --2- 172.21.15.97:0/2063738717 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572efdbffa0 0x5572efdd30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:19.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.813+0000 7f7317fff640 0 client.16881 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:35:19.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.813+0000 7f7341794740 1 -- 172.21.15.97:0/2063738717 >> 172.21.15.97:0/2063738717 conn(0x5572efdbc3c0 msgr2=0x5572efdcb450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:35:19.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.813+0000 7f7341794740 1 -- 172.21.15.97:0/2063738717 shutdown_connections 2024-04-23T10:35:19.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.813+0000 7f7341794740 1 -- 172.21.15.97:0/2063738717 wait complete. 2024-04-23T10:35:19.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.813+0000 7f7341794740 20 client.16881 trim_cache size 0 max 16384 2024-04-23T10:35:19.813 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-23T10:35:19.813 INFO:teuthology.orchestra.run.smithi097.stdout: "epoch": 26, 2024-04-23T10:35:19.813 INFO:teuthology.orchestra.run.smithi097.stdout: "leader": 24488, 2024-04-23T10:35:19.813 INFO:teuthology.orchestra.run.smithi097.stdout: "set_version": 38, 2024-04-23T10:35:19.814 INFO:teuthology.orchestra.run.smithi097.stdout: "sets": { 2024-04-23T10:35:19.814 INFO:teuthology.orchestra.run.smithi097.stdout: "13110ad2": { 2024-04-23T10:35:19.814 INFO:teuthology.orchestra.run.smithi097.stdout: "version": 38, 2024-04-23T10:35:19.814 INFO:teuthology.orchestra.run.smithi097.stdout: "age_ref": 0.0, 2024-04-23T10:35:19.814 INFO:teuthology.orchestra.run.smithi097.stdout: "state": { 2024-04-23T10:35:19.814 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "QUIESCED", 2024-04-23T10:35:19.814 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0 2024-04-23T10:35:19.814 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:35:19.814 INFO:teuthology.orchestra.run.smithi097.stdout: "timeout": 60.0, 2024-04-23T10:35:19.814 INFO:teuthology.orchestra.run.smithi097.stdout: "expiration": 70.0, 2024-04-23T10:35:19.814 INFO:teuthology.orchestra.run.smithi097.stdout: "members": { 2024-04-23T10:35:19.814 INFO:teuthology.orchestra.run.smithi097.stdout: "file:/": { 2024-04-23T10:35:19.814 INFO:teuthology.orchestra.run.smithi097.stdout: "excluded": false, 2024-04-23T10:35:19.814 INFO:teuthology.orchestra.run.smithi097.stdout: "state": { 2024-04-23T10:35:19.814 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "QUIESCED", 2024-04-23T10:35:19.815 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0 2024-04-23T10:35:19.815 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:35:19.815 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:35:19.815 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:35:19.815 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:35:19.815 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:35:19.815 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-23T10:35:19.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.814+0000 7f733ea7c640 1 -- 172.21.15.97:0/3240804528 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f73240a3d20 msgr2=0x7f73240a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:19.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.814+0000 7f733ea7c640 1 --2- 172.21.15.97:0/3240804528 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f73240a3d20 0x7f73240a61e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f73300ef640 tx=0x7f73300095c0 comp rx=0 tx=0).stop 2024-04-23T10:35:19.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.814+0000 7f733ea7c640 1 -- 172.21.15.97:0/3240804528 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73381021a0 msgr2=0x7f733819e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:19.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.814+0000 7f733ea7c640 1 --2- 172.21.15.97:0/3240804528 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73381021a0 0x7f733819e400 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f7330006010 tx=0x7f73300319f0 comp rx=0 tx=0).stop 2024-04-23T10:35:19.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.814+0000 7f733ea7c640 1 -- 172.21.15.97:0/3240804528 shutdown_connections 2024-04-23T10:35:19.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.814+0000 7f733ea7c640 1 --2- 172.21.15.97:0/3240804528 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f73240a3d20 0x7f73240a61e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:19.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.814+0000 7f733ea7c640 1 --2- 172.21.15.97:0/3240804528 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7338198600 0x7f7338198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:19.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.814+0000 7f733ea7c640 1 --2- 172.21.15.97:0/3240804528 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7338107270 0x7f733819e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:19.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.814+0000 7f733ea7c640 1 --2- 172.21.15.97:0/3240804528 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73381021a0 0x7f733819e400 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:19.816 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.814+0000 7f733ea7c640 1 -- 172.21.15.97:0/3240804528 >> 172.21.15.97:0/3240804528 conn(0x7f7338079350 msgr2=0x7f7338100260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:35:19.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.814+0000 7f733ea7c640 1 -- 172.21.15.97:0/3240804528 shutdown_connections 2024-04-23T10:35:19.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:19.814+0000 7f733ea7c640 1 -- 172.21.15.97:0/3240804528 wait complete. 2024-04-23T10:35:19.823 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '13110ad2', quiesce took 0.5 seconds. Will release after: 10 2024-04-23T10:35:20.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:35:20 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/2408734947' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:35:20.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:35:20 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/2408734947' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:35:20.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:35:20 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/2408734947' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:35:21.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:35:21 smithi183 ceph-mon[32317]: pgmap v283: 97 pgs: 97 active+clean; 4.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:35:21.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:35:21 smithi196 ceph-mon[32919]: pgmap v283: 97 pgs: 97 active+clean; 4.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:35:21.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:35:21 smithi097 ceph-mon[29197]: pgmap v283: 97 pgs: 97 active+clean; 4.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:35:23.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:35:23 smithi183 ceph-mon[32317]: pgmap v284: 97 pgs: 97 active+clean; 4.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:35:23.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:35:23 smithi196 ceph-mon[32919]: pgmap v284: 97 pgs: 97 active+clean; 4.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:35:23.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:35:23 smithi097 ceph-mon[29197]: pgmap v284: 97 pgs: 97 active+clean; 4.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:35:24.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:35:24 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:35:24.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:35:24 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:35:24.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:35:24 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:35:24.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:35:24 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:35:24.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:35:24 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:35:24.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:35:24 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:35:24.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:35:24 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:35:24.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:35:24 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:35:24.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:35:24 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:35:24.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:35:24 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:35:24.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:35:24 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:35:24.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:35:24 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:35:25.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:35:25 smithi183 ceph-mon[32317]: pgmap v285: 97 pgs: 97 active+clean; 4.8 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 52 KiB/s wr, 21 op/s 2024-04-23T10:35:25.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:35:25 smithi196 ceph-mon[32919]: pgmap v285: 97 pgs: 97 active+clean; 4.8 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 52 KiB/s wr, 21 op/s 2024-04-23T10:35:25.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:35:25 smithi097 ceph-mon[29197]: pgmap v285: 97 pgs: 97 active+clean; 4.8 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 52 KiB/s wr, 21 op/s 2024-04-23T10:35:25.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.684+0000 7f4ad41f9640 1 -- 172.21.15.97:0/3742337447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4acc10c5d0 msgr2=0x7f4acc10ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:25.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.684+0000 7f4ad41f9640 1 --2- 172.21.15.97:0/3742337447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4acc10c5d0 0x7f4acc10ea00 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f4ac4009ca0 tx=0x7f4ac402a950 comp rx=0 tx=0).stop 2024-04-23T10:35:25.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.684+0000 7f4ad41f9640 1 -- 172.21.15.97:0/3742337447 shutdown_connections 2024-04-23T10:35:25.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.684+0000 7f4ad41f9640 1 --2- 172.21.15.97:0/3742337447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4acc10c5d0 0x7f4acc10ea00 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:25.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.684+0000 7f4ad41f9640 1 --2- 172.21.15.97:0/3742337447 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4acc103b20 0x7f4acc10c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:25.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.684+0000 7f4ad41f9640 1 --2- 172.21.15.97:0/3742337447 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4acc103200 0x7f4acc1035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:25.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.684+0000 7f4ad41f9640 1 -- 172.21.15.97:0/3742337447 >> 172.21.15.97:0/3742337447 conn(0x7f4acc0771d0 msgr2=0x7f4acc0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:35:25.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.684+0000 7f4ad41f9640 1 -- 172.21.15.97:0/3742337447 shutdown_connections 2024-04-23T10:35:25.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.684+0000 7f4ad41f9640 1 -- 172.21.15.97:0/3742337447 wait complete. 2024-04-23T10:35:25.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.685+0000 7f4ad41f9640 1 Processor -- start 2024-04-23T10:35:25.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.685+0000 7f4ad41f9640 1 -- start start 2024-04-23T10:35:25.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.685+0000 7f4ad41f9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4acc103200 0x7f4acc198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:25.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.685+0000 7f4ad41f9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4acc103b20 0x7f4acc198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:25.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.685+0000 7f4ad41f9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4acc10c5d0 0x7f4acc19cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:25.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.685+0000 7f4ad41f9640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4acc1121f0 con 0x7f4acc10c5d0 2024-04-23T10:35:25.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.685+0000 7f4ad41f9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4acc112070 con 0x7f4acc103200 2024-04-23T10:35:25.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.685+0000 7f4ad41f9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4acc112370 con 0x7f4acc103b20 2024-04-23T10:35:25.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.685+0000 7f4ad2796640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4acc10c5d0 0x7f4acc19cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:25.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.685+0000 7f4ad2796640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4acc10c5d0 0x7f4acc19cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42798/0 (socket says 172.21.15.97:42798) 2024-04-23T10:35:25.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.685+0000 7f4ad2796640 1 -- 172.21.15.97:0/2964712407 learned_addr learned my addr 172.21.15.97:0/2964712407 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:35:25.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.685+0000 7f4ad1f95640 1 --2- 172.21.15.97:0/2964712407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4acc103200 0x7f4acc198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:25.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.685+0000 7f4ad1794640 1 --2- 172.21.15.97:0/2964712407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4acc103b20 0x7f4acc198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:25.686 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.685+0000 7f4ad2796640 1 -- 172.21.15.97:0/2964712407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4acc103b20 msgr2=0x7f4acc198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:25.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.685+0000 7f4ad2796640 1 --2- 172.21.15.97:0/2964712407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4acc103b20 0x7f4acc198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:25.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.685+0000 7f4ad2796640 1 -- 172.21.15.97:0/2964712407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4acc103200 msgr2=0x7f4acc198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:25.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.685+0000 7f4ad2796640 1 --2- 172.21.15.97:0/2964712407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4acc103200 0x7f4acc198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:25.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.685+0000 7f4ad2796640 1 -- 172.21.15.97:0/2964712407 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4acc19d630 con 0x7f4acc10c5d0 2024-04-23T10:35:25.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.685+0000 7f4ad1794640 1 --2- 172.21.15.97:0/2964712407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4acc103b20 0x7f4acc198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:35:25.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.685+0000 7f4ad2796640 1 --2- 172.21.15.97:0/2964712407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4acc10c5d0 0x7f4acc19cf00 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f4ac402ae70 tx=0x7f4ac402f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:35:25.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.686+0000 7f4ac2ffd640 1 -- 172.21.15.97:0/2964712407 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac4034040 con 0x7f4acc10c5d0 2024-04-23T10:35:25.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.686+0000 7f4ac2ffd640 1 -- 172.21.15.97:0/2964712407 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ac40308f0 con 0x7f4acc10c5d0 2024-04-23T10:35:25.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.686+0000 7f4ad41f9640 1 -- 172.21.15.97:0/2964712407 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4acc19d8c0 con 0x7f4acc10c5d0 2024-04-23T10:35:25.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.686+0000 7f4ac2ffd640 1 -- 172.21.15.97:0/2964712407 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac40047c0 con 0x7f4acc10c5d0 2024-04-23T10:35:25.687 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.686+0000 7f4ad41f9640 1 -- 172.21.15.97:0/2964712407 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4acc1a51b0 con 0x7f4acc10c5d0 2024-04-23T10:35:25.693 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.688+0000 7f4ac2ffd640 1 -- 172.21.15.97:0/2964712407 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4ac4037020 con 0x7f4acc10c5d0 2024-04-23T10:35:25.693 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.688+0000 7f4ad6fe0740 1 Processor -- start 2024-04-23T10:35:25.693 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.689+0000 7f4ad6fe0740 1 -- start start 2024-04-23T10:35:25.693 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.689+0000 7f4ad6fe0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d9e108a620 0x55d9e108aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:25.693 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.689+0000 7f4ad6fe0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9e108af40 0x55d9e10952c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:25.693 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.689+0000 7f4ad6fe0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9e1095800 0x55d9e1097bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:25.693 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.689+0000 7f4ad6fe0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d9e0fef970 con 0x55d9e1095800 2024-04-23T10:35:25.693 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.689+0000 7f4ad6fe0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d9e0f8c4a0 con 0x55d9e108a620 2024-04-23T10:35:25.693 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.689+0000 7f4ad6fe0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d9e0feb930 con 0x55d9e108af40 2024-04-23T10:35:25.693 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.689+0000 7f4ac2ffd640 1 --2- 172.21.15.97:0/2964712407 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f4ab80a3e40 0x7f4ab80a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:25.694 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.689+0000 7f4ad1794640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9e1095800 0x55d9e1097bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:25.694 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.689+0000 7f4ad1794640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9e1095800 0x55d9e1097bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42812/0 (socket says 172.21.15.97:42812) 2024-04-23T10:35:25.694 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.689+0000 7f4ad2796640 1 --2- 172.21.15.97:0/2964712407 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f4ab80a3e40 0x7f4ab80a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:25.694 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.689+0000 7f4ad1794640 1 -- 172.21.15.97:0/3647535246 learned_addr learned my addr 172.21.15.97:0/3647535246 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:35:25.694 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.689+0000 7f4ad1f95640 1 --2- 172.21.15.97:0/3647535246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9e108af40 0x55d9e10952c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:25.694 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.689+0000 7f4ad1794640 1 --2- 172.21.15.97:0/3647535246 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d9e108a620 0x55d9e108aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:25.694 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.689+0000 7f4ac2ffd640 1 -- 172.21.15.97:0/2964712407 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8484+0+0 (secure 0 0 0) 0x7f4ac40f16b0 con 0x7f4acc10c5d0 2024-04-23T10:35:25.694 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.690+0000 7f4ad1794640 1 -- 172.21.15.97:0/3647535246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9e108af40 msgr2=0x55d9e10952c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:35:25.694 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.690+0000 7f4ad1794640 1 --2- 172.21.15.97:0/3647535246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9e108af40 0x55d9e10952c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:25.694 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.690+0000 7f4ad1794640 1 -- 172.21.15.97:0/3647535246 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d9e108a620 msgr2=0x55d9e108aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:35:25.694 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.690+0000 7f4ad1794640 1 --2- 172.21.15.97:0/3647535246 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d9e108a620 0x55d9e108aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:25.694 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.690+0000 7f4ad1794640 1 -- 172.21.15.97:0/3647535246 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9e0d02210 con 0x55d9e1095800 2024-04-23T10:35:25.694 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.690+0000 7f4ad2796640 1 --2- 172.21.15.97:0/2964712407 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f4ab80a3e40 0x7f4ab80a6300 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4ac4083080 tx=0x7f4ac408c0d0 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:35:25.694 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.690+0000 7f4ad1794640 1 --2- 172.21.15.97:0/3647535246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9e1095800 0x55d9e1097bf0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8005e00 tx=0x7f4ac800b4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84ce4791f242b9f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:35:25.694 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.690+0000 7f4ab7fff640 1 -- 172.21.15.97:0/3647535246 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac8004030 con 0x55d9e1095800 2024-04-23T10:35:25.695 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.690+0000 7f4ab7fff640 1 -- 172.21.15.97:0/3647535246 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ac8005020 con 0x55d9e1095800 2024-04-23T10:35:25.695 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.690+0000 7f4ab7fff640 1 -- 172.21.15.97:0/3647535246 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac8004a00 con 0x55d9e1095800 2024-04-23T10:35:25.695 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.691+0000 7f4ad6fe0740 1 -- 172.21.15.97:0/3647535246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9e1095800 msgr2=0x55d9e1097bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:25.695 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.691+0000 7f4ad6fe0740 1 --2- 172.21.15.97:0/3647535246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9e1095800 0x55d9e1097bf0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8005e00 tx=0x7f4ac800b4c0 comp rx=0 tx=0).stop 2024-04-23T10:35:25.695 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.691+0000 7f4ad6fe0740 1 -- 172.21.15.97:0/3647535246 shutdown_connections 2024-04-23T10:35:25.695 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.691+0000 7f4ad6fe0740 1 --2- 172.21.15.97:0/3647535246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9e1095800 0x55d9e1097bf0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:25.695 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.691+0000 7f4ad6fe0740 1 --2- 172.21.15.97:0/3647535246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9e108af40 0x55d9e10952c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:25.695 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.691+0000 7f4ad6fe0740 1 --2- 172.21.15.97:0/3647535246 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d9e108a620 0x55d9e108aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:25.695 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.691+0000 7f4ad6fe0740 1 -- 172.21.15.97:0/3647535246 >> 172.21.15.97:0/3647535246 conn(0x55d9e1086b30 msgr2=0x55d9e1086f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:35:25.695 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.691+0000 7f4ad6fe0740 1 -- 172.21.15.97:0/3647535246 shutdown_connections 2024-04-23T10:35:25.695 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.691+0000 7f4ad6fe0740 1 -- 172.21.15.97:0/3647535246 wait complete. 2024-04-23T10:35:25.695 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.691+0000 7f4ad6fe0740 1 -- start start 2024-04-23T10:35:25.695 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.691+0000 7f4ad6fe0740 1 Processor -- start 2024-04-23T10:35:25.695 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.692+0000 7f4ad6fe0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d9e108a620 0x55d9e1095130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:25.695 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.692+0000 7f4ad6fe0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9e108af40 0x55d9e1092420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.692+0000 7f4ad6fe0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9e1095800 0x55d9e1092960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.692+0000 7f4ad6fe0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d9e1098430 con 0x55d9e1095800 2024-04-23T10:35:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.692+0000 7f4ad6fe0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d9e0f8a0b0 con 0x55d9e108a620 2024-04-23T10:35:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.692+0000 7f4ad6fe0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d9e10985b0 con 0x55d9e108af40 2024-04-23T10:35:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.692+0000 7f4ad6fe0740 10 client.0 _ll_register_callbacks cb 0x55d9e0f77d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:35:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.692+0000 7f4ad1794640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9e1095800 0x55d9e1092960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.692+0000 7f4ad1794640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9e1095800 0x55d9e1092960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42814/0 (socket says 172.21.15.97:42814) 2024-04-23T10:35:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.692+0000 7f4ad1f95640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9e108af40 0x55d9e1092420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.692+0000 7f4ad1794640 1 -- 172.21.15.97:0/510967650 learned_addr learned my addr 172.21.15.97:0/510967650 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:35:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.692+0000 7f4ad1794640 1 --2- 172.21.15.97:0/510967650 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d9e108a620 0x55d9e1095130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.692+0000 7f4ad1f95640 1 -- 172.21.15.97:0/510967650 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d9e108a620 msgr2=0x55d9e1095130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:35:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.692+0000 7f4ad1f95640 1 --2- 172.21.15.97:0/510967650 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d9e108a620 0x55d9e1095130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.692+0000 7f4ad1f95640 1 -- 172.21.15.97:0/510967650 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9e1095800 msgr2=0x55d9e1092960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:35:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.692+0000 7f4ad1f95640 1 --2- 172.21.15.97:0/510967650 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9e1095800 0x55d9e1092960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.692+0000 7f4ad1f95640 1 -- 172.21.15.97:0/510967650 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9e0dce620 con 0x55d9e108af40 2024-04-23T10:35:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.692+0000 7f4ad1794640 1 --2- 172.21.15.97:0/510967650 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9e1095800 0x55d9e1092960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T10:35:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.692+0000 7f4ad1f95640 1 --2- 172.21.15.97:0/510967650 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9e108af40 0x55d9e1092420 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4abc005fd0 tx=0x7f4abc00bdf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7864380d95a8e023 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:35:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.692+0000 7f4ab57fa640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-04-23T10:35:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.693+0000 7f4ab57fa640 1 -- 172.21.15.97:0/510967650 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4abc005020 con 0x55d9e108af40 2024-04-23T10:35:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.693+0000 7f4ad6fe0740 1 -- 172.21.15.97:0/510967650 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ac8005020 con 0x55d9e108af40 2024-04-23T10:35:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.693+0000 7f4ab57fa640 1 -- 172.21.15.97:0/510967650 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8484+0+0 (secure 0 0 0) 0x7f4abc0051c0 con 0x55d9e108af40 2024-04-23T10:35:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.693+0000 7f4ab57fa640 1 -- 172.21.15.97:0/510967650 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4abc0053c0 con 0x55d9e108af40 2024-04-23T10:35:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.693+0000 7f4ad6fe0740 10 client.26032 fetch_fsmap learned FSMap version 26 2024-04-23T10:35:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.693+0000 7f4ad6fe0740 1 -- 172.21.15.97:0/510967650 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d9e10a4430 con 0x55d9e108af40 2024-04-23T10:35:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.693+0000 7f4ab57fa640 1 -- 172.21.15.97:0/510967650 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ac8005020 con 0x55d9e108af40 2024-04-23T10:35:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.693+0000 7f4ab57fa640 1 -- 172.21.15.97:0/510967650 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4abc022020 con 0x55d9e108af40 2024-04-23T10:35:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.693+0000 7f4ad6fe0740 10 client.26032 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:35:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.693+0000 7f4ad6fe0740 10 client.26032 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:35:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.693+0000 7f4ad6fe0740 20 client.26032 populate_metadata read hostname 'smithi097' 2024-04-23T10:35:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.693+0000 7f4ad6fe0740 1 --2- 172.21.15.97:0/510967650 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55d9e10a58a0 0x55d9e10a7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:25.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.693+0000 7f4ad6fe0740 4 client.26032 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:35:25.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.693+0000 7f4ad6fe0740 1 -- 172.21.15.97:0/510967650 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9e0ffd390 con 0x55d9e10a58a0 2024-04-23T10:35:25.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.694+0000 7f4ad1794640 1 --2- 172.21.15.97:0/510967650 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55d9e10a58a0 0x55d9e10a7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:25.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.694+0000 7f4ab57fa640 10 client.26032 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:35:25.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.694+0000 7f4ad1794640 1 --2- 172.21.15.97:0/510967650 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55d9e10a58a0 0x55d9e10a7c90 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a74097596621451f server_cookie=874813767ccd9a10 in_seq=0 out_seq=0 2024-04-23T10:35:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.695+0000 7f4ab57fa640 1 -- 172.21.15.97:0/510967650 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d9e0ffd390 con 0x55d9e10a58a0 2024-04-23T10:35:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.695+0000 7f4ab57fa640 10 client.26032 handle_command_reply: tid=0 2024-04-23T10:35:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.695+0000 7f4ad6fe0740 1 client.26032 shutdown 2024-04-23T10:35:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.695+0000 7f4ad6fe0740 1 -- 172.21.15.97:0/510967650 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9e108af40 msgr2=0x55d9e1092420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.695+0000 7f4ad6fe0740 1 --2- 172.21.15.97:0/510967650 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9e108af40 0x55d9e1092420 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4abc005fd0 tx=0x7f4abc00bdf0 comp rx=0 tx=0).stop 2024-04-23T10:35:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.695+0000 7f4ad6fe0740 1 -- 172.21.15.97:0/510967650 shutdown_connections 2024-04-23T10:35:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.695+0000 7f4ad6fe0740 1 --2- 172.21.15.97:0/510967650 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55d9e10a58a0 0x55d9e10a7c90 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.695+0000 7f4ad6fe0740 1 --2- 172.21.15.97:0/510967650 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9e1095800 0x55d9e1092960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.695+0000 7f4ad6fe0740 1 --2- 172.21.15.97:0/510967650 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9e108af40 0x55d9e1092420 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.695+0000 7f4ab57fa640 0 client.26032 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:35:25.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.695+0000 7f4ad6fe0740 1 --2- 172.21.15.97:0/510967650 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d9e108a620 0x55d9e1095130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:25.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.695+0000 7f4ad6fe0740 1 -- 172.21.15.97:0/510967650 >> 172.21.15.97:0/510967650 conn(0x55d9e1086b30 msgr2=0x55d9e108c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:35:25.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.695+0000 7f4ad6fe0740 1 -- 172.21.15.97:0/510967650 shutdown_connections 2024-04-23T10:35:25.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.695+0000 7f4ad6fe0740 1 -- 172.21.15.97:0/510967650 wait complete. 2024-04-23T10:35:25.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.695+0000 7f4ad6fe0740 20 client.26032 trim_cache size 0 max 16384 2024-04-23T10:35:25.710 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.710+0000 7f4ad6fe0740 1 Processor -- start 2024-04-23T10:35:25.710 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.710+0000 7f4ad6fe0740 1 -- start start 2024-04-23T10:35:25.711 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.710+0000 7f4ad6fe0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9e1086b30 0x55d9e10a12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:25.711 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.710+0000 7f4ad6fe0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9e108a620 0x55d9e1085b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:25.711 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.710+0000 7f4ad6fe0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d9e108af40 0x55d9e10ad550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:25.711 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.710+0000 7f4ad6fe0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4acc1ad9b0 con 0x55d9e108a620 2024-04-23T10:35:25.711 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.710+0000 7f4ad6fe0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4acc1af4f0 con 0x55d9e108af40 2024-04-23T10:35:25.711 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.710+0000 7f4ad6fe0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4acc1ae080 con 0x55d9e1086b30 2024-04-23T10:35:25.711 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.710+0000 7f4ad1f95640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9e108a620 0x55d9e1085b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:25.711 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.710+0000 7f4ad1f95640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9e108a620 0x55d9e1085b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42818/0 (socket says 172.21.15.97:42818) 2024-04-23T10:35:25.711 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.710+0000 7f4ad1f95640 1 -- 172.21.15.97:0/2082663604 learned_addr learned my addr 172.21.15.97:0/2082663604 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:35:25.711 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.710+0000 7f4ad1794640 1 --2- 172.21.15.97:0/2082663604 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d9e108af40 0x55d9e10ad550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:25.711 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.710+0000 7f4ad1f95640 1 -- 172.21.15.97:0/2082663604 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9e1086b30 msgr2=0x55d9e10a12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:35:25.711 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.710+0000 7f4ad1f95640 1 --2- 172.21.15.97:0/2082663604 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9e1086b30 0x55d9e10a12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:25.711 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.710+0000 7f4ad1f95640 1 -- 172.21.15.97:0/2082663604 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d9e108af40 msgr2=0x55d9e10ad550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:35:25.711 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.710+0000 7f4ad1f95640 1 --2- 172.21.15.97:0/2082663604 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d9e108af40 0x55d9e10ad550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.710+0000 7f4ad1f95640 1 -- 172.21.15.97:0/2082663604 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9e108ad30 con 0x55d9e108a620 2024-04-23T10:35:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.710+0000 7f4ad1794640 1 --2- 172.21.15.97:0/2082663604 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d9e108af40 0x55d9e10ad550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:35:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.711+0000 7f4ad1f95640 1 --2- 172.21.15.97:0/2082663604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9e108a620 0x55d9e1085b30 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f4abc01b710 tx=0x7f4abc00bfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d172f236dd2f903b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:35:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.711+0000 7f4ab77fe640 1 -- 172.21.15.97:0/2082663604 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4abc005560 con 0x55d9e108a620 2024-04-23T10:35:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.711+0000 7f4ab77fe640 1 -- 172.21.15.97:0/2082663604 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4abc01bc40 con 0x55d9e108a620 2024-04-23T10:35:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.711+0000 7f4ab77fe640 1 -- 172.21.15.97:0/2082663604 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4abc024ac0 con 0x55d9e108a620 2024-04-23T10:35:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.711+0000 7f4ad6fe0740 1 -- 172.21.15.97:0/2082663604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9e108a620 msgr2=0x55d9e1085b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.711+0000 7f4ad6fe0740 1 --2- 172.21.15.97:0/2082663604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9e108a620 0x55d9e1085b30 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f4abc01b710 tx=0x7f4abc00bfa0 comp rx=0 tx=0).stop 2024-04-23T10:35:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.711+0000 7f4ad6fe0740 1 -- 172.21.15.97:0/2082663604 shutdown_connections 2024-04-23T10:35:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.711+0000 7f4ad6fe0740 1 --2- 172.21.15.97:0/2082663604 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d9e108af40 0x55d9e10ad550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.711+0000 7f4ad6fe0740 1 --2- 172.21.15.97:0/2082663604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9e108a620 0x55d9e1085b30 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.711+0000 7f4ad6fe0740 1 --2- 172.21.15.97:0/2082663604 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9e1086b30 0x55d9e10a12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.711+0000 7f4ad6fe0740 1 -- 172.21.15.97:0/2082663604 >> 172.21.15.97:0/2082663604 conn(0x55d9e1092ac0 msgr2=0x55d9e1085420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:35:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.712+0000 7f4ad6fe0740 1 -- 172.21.15.97:0/2082663604 shutdown_connections 2024-04-23T10:35:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.712+0000 7f4ad6fe0740 1 -- 172.21.15.97:0/2082663604 wait complete. 2024-04-23T10:35:25.713 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.712+0000 7f4ad6fe0740 1 -- start start 2024-04-23T10:35:25.713 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.712+0000 7f4ad6fe0740 1 Processor -- start 2024-04-23T10:35:25.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.712+0000 7f4ad6fe0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9e1086b30 0x55d9e109b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:25.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.712+0000 7f4ad6fe0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9e108a620 0x55d9e109be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:25.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.712+0000 7f4ad6fe0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d9e108af40 0x55d9e110add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:25.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.712+0000 7f4ad6fe0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4acc1aee20 con 0x55d9e1086b30 2024-04-23T10:35:25.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.712+0000 7f4ad6fe0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4acc1ae750 con 0x55d9e108af40 2024-04-23T10:35:25.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.712+0000 7f4ad6fe0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4acc1ab9b0 con 0x55d9e108a620 2024-04-23T10:35:25.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.712+0000 7f4ad6fe0740 10 client.0 _ll_register_callbacks cb 0x7f4abc0045b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:35:25.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.712+0000 7f4ad1794640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9e1086b30 0x55d9e109b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:25.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.712+0000 7f4ad1794640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9e1086b30 0x55d9e109b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42822/0 (socket says 172.21.15.97:42822) 2024-04-23T10:35:25.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.712+0000 7f4ad1794640 1 -- 172.21.15.97:0/2498919936 learned_addr learned my addr 172.21.15.97:0/2498919936 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:35:25.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.713+0000 7f4ad1f95640 1 --2- 172.21.15.97:0/2498919936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9e108a620 0x55d9e109be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:25.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.713+0000 7f4ad1794640 1 --2- 172.21.15.97:0/2498919936 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d9e108af40 0x55d9e110add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:25.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.713+0000 7f4ad1794640 1 -- 172.21.15.97:0/2498919936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9e108a620 msgr2=0x55d9e109be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:35:25.716 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.713+0000 7f4ad1794640 1 --2- 172.21.15.97:0/2498919936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9e108a620 0x55d9e109be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:25.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.713+0000 7f4ad1794640 1 -- 172.21.15.97:0/2498919936 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d9e108af40 msgr2=0x55d9e110add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:35:25.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.713+0000 7f4ad1794640 1 --2- 172.21.15.97:0/2498919936 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d9e108af40 0x55d9e110add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:25.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.713+0000 7f4ad1794640 1 -- 172.21.15.97:0/2498919936 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9e110b310 con 0x55d9e1086b30 2024-04-23T10:35:25.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.713+0000 7f4ad1f95640 1 --2- 172.21.15.97:0/2498919936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9e108a620 0x55d9e109be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:35:25.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.713+0000 7f4ad1794640 1 --2- 172.21.15.97:0/2498919936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9e1086b30 0x55d9e109b8f0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8005bc0 tx=0x7f4ac800bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=162bf60e679ee416 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:35:25.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.713+0000 7f4ab57fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:35:25.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.713+0000 7f4ab57fa640 1 -- 172.21.15.97:0/2498919936 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac80143a0 con 0x55d9e1086b30 2024-04-23T10:35:25.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.713+0000 7f4ab57fa640 1 -- 172.21.15.97:0/2498919936 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8484+0+0 (secure 0 0 0) 0x7f4ac80052a0 con 0x55d9e1086b30 2024-04-23T10:35:25.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.713+0000 7f4ad6fe0740 1 -- 172.21.15.97:0/2498919936 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4abc01bc40 con 0x55d9e1086b30 2024-04-23T10:35:25.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.713+0000 7f4ab57fa640 1 -- 172.21.15.97:0/2498919936 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4ac80054a0 con 0x55d9e1086b30 2024-04-23T10:35:25.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.713+0000 7f4ad6fe0740 10 client.16899 fetch_fsmap learned FSMap version 26 2024-04-23T10:35:25.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.713+0000 7f4ad6fe0740 1 -- 172.21.15.97:0/2498919936 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d9e110c500 con 0x55d9e1086b30 2024-04-23T10:35:25.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.713+0000 7f4ab57fa640 1 -- 172.21.15.97:0/2498919936 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4abc01bc40 con 0x55d9e1086b30 2024-04-23T10:35:25.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.714+0000 7f4ab57fa640 1 -- 172.21.15.97:0/2498919936 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4ac802f040 con 0x55d9e1086b30 2024-04-23T10:35:25.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.714+0000 7f4ad6fe0740 10 client.16899 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:35:25.717 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.714+0000 7f4ad6fe0740 10 client.16899 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:35:25.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.714+0000 7f4ad6fe0740 20 client.16899 populate_metadata read hostname 'smithi097' 2024-04-23T10:35:25.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.714+0000 7f4ad6fe0740 1 --2- 172.21.15.97:0/2498919936 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55d9e1095800 0x55d9e110eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:25.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.714+0000 7f4ad6fe0740 4 client.16899 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:35:25.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.714+0000 7f4ad6fe0740 1 -- 172.21.15.97:0/2498919936 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d9e0ff7200 con 0x55d9e1095800 2024-04-23T10:35:25.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.714+0000 7f4ad1f95640 1 --2- 172.21.15.97:0/2498919936 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55d9e1095800 0x55d9e110eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:25.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.714+0000 7f4ab57fa640 10 client.16899 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:35:25.718 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.714+0000 7f4ad1f95640 1 --2- 172.21.15.97:0/2498919936 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55d9e1095800 0x55d9e110eae0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c5667b3c80c2949 server_cookie=eb9ca8b945a3344a in_seq=0 out_seq=0 2024-04-23T10:35:25.728 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.729+0000 7f4ab57fa640 1 -- 172.21.15.97:0/2498919936 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d9e0ff7200 con 0x55d9e1095800 2024-04-23T10:35:25.728 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.729+0000 7f4ab57fa640 10 client.16899 handle_command_reply: tid=0 2024-04-23T10:35:25.728 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.729+0000 7f4ad6fe0740 1 client.16899 shutdown 2024-04-23T10:35:25.728 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.729+0000 7f4ad6fe0740 1 -- 172.21.15.97:0/2498919936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9e1086b30 msgr2=0x55d9e109b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:25.728 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.729+0000 7f4ad6fe0740 1 --2- 172.21.15.97:0/2498919936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9e1086b30 0x55d9e109b8f0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8005bc0 tx=0x7f4ac800bcd0 comp rx=0 tx=0).stop 2024-04-23T10:35:25.728 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.729+0000 7f4ad6fe0740 1 -- 172.21.15.97:0/2498919936 shutdown_connections 2024-04-23T10:35:25.729 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.729+0000 7f4ad6fe0740 1 --2- 172.21.15.97:0/2498919936 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55d9e1095800 0x55d9e110eae0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:25.729 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.729+0000 7f4ad6fe0740 1 --2- 172.21.15.97:0/2498919936 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d9e108af40 0x55d9e110add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:25.729 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.729+0000 7f4ad6fe0740 1 --2- 172.21.15.97:0/2498919936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d9e108a620 0x55d9e109be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:25.729 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.729+0000 7f4ad6fe0740 1 --2- 172.21.15.97:0/2498919936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9e1086b30 0x55d9e109b8f0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:25.729 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.729+0000 7f4ab57fa640 0 client.16899 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:35:25.729 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.729+0000 7f4ad6fe0740 1 -- 172.21.15.97:0/2498919936 >> 172.21.15.97:0/2498919936 conn(0x55d9e1092ac0 msgr2=0x55d9e1085cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:35:25.729 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.729+0000 7f4ad6fe0740 1 -- 172.21.15.97:0/2498919936 shutdown_connections 2024-04-23T10:35:25.729 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.729+0000 7f4ad6fe0740 1 -- 172.21.15.97:0/2498919936 wait complete. 2024-04-23T10:35:25.729 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.729+0000 7f4ad6fe0740 20 client.16899 trim_cache size 0 max 16384 2024-04-23T10:35:25.729 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:35:25.729 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:35:25.729 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:35:25.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:35:25.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.730+0000 7f4ad41f9640 1 -- 172.21.15.97:0/2964712407 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f4ab80a3e40 msgr2=0x7f4ab80a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:25.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.730+0000 7f4ad41f9640 1 --2- 172.21.15.97:0/2964712407 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f4ab80a3e40 0x7f4ab80a6300 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4ac4083080 tx=0x7f4ac408c0d0 comp rx=0 tx=0).stop 2024-04-23T10:35:25.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.730+0000 7f4ad41f9640 1 -- 172.21.15.97:0/2964712407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4acc10c5d0 msgr2=0x7f4acc19cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:25.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.730+0000 7f4ad41f9640 1 --2- 172.21.15.97:0/2964712407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4acc10c5d0 0x7f4acc19cf00 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f4ac402ae70 tx=0x7f4ac402f5d0 comp rx=0 tx=0).stop 2024-04-23T10:35:25.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.730+0000 7f4ad41f9640 1 -- 172.21.15.97:0/2964712407 shutdown_connections 2024-04-23T10:35:25.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.730+0000 7f4ad41f9640 1 --2- 172.21.15.97:0/2964712407 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f4ab80a3e40 0x7f4ab80a6300 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:25.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.730+0000 7f4ad41f9640 1 --2- 172.21.15.97:0/2964712407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4acc10c5d0 0x7f4acc19cf00 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:25.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.730+0000 7f4ad41f9640 1 --2- 172.21.15.97:0/2964712407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4acc103b20 0x7f4acc198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:25.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.730+0000 7f4ad41f9640 1 --2- 172.21.15.97:0/2964712407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4acc103200 0x7f4acc198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:25.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.730+0000 7f4ad41f9640 1 -- 172.21.15.97:0/2964712407 >> 172.21.15.97:0/2964712407 conn(0x7f4acc0771d0 msgr2=0x7f4acc0fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:35:25.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.730+0000 7f4ad41f9640 1 -- 172.21.15.97:0/2964712407 shutdown_connections 2024-04-23T10:35:25.731 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:25.730+0000 7f4ad41f9640 1 -- 172.21.15.97:0/2964712407 wait complete. 2024-04-23T10:35:27.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:35:27 smithi183 ceph-mon[32317]: pgmap v286: 97 pgs: 97 active+clean; 4.8 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 15 op/s 2024-04-23T10:35:27.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:35:27 smithi196 ceph-mon[32919]: pgmap v286: 97 pgs: 97 active+clean; 4.8 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 15 op/s 2024-04-23T10:35:27.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:35:27 smithi097 ceph-mon[29197]: pgmap v286: 97 pgs: 97 active+clean; 4.8 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 15 op/s 2024-04-23T10:35:29.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:35:29 smithi183 ceph-mon[32317]: pgmap v287: 97 pgs: 97 active+clean; 4.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 41 KiB/s wr, 22 op/s 2024-04-23T10:35:29.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:35:29 smithi196 ceph-mon[32919]: pgmap v287: 97 pgs: 97 active+clean; 4.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 41 KiB/s wr, 22 op/s 2024-04-23T10:35:29.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:35:29 smithi097 ceph-mon[29197]: pgmap v287: 97 pgs: 97 active+clean; 4.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 41 KiB/s wr, 22 op/s 2024-04-23T10:35:29.824 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T10:35:29.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:29.936+0000 7f722fed9640 1 -- 172.21.15.97:0/2905125653 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7228100fa0 msgr2=0x7f7228101380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:29.937 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:29.936+0000 7f722fed9640 1 --2- 172.21.15.97:0/2905125653 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7228100fa0 0x7f7228101380 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f7224009930 tx=0x7f722402a4d0 comp rx=0 tx=0).stop 2024-04-23T10:35:29.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:29.936+0000 7f722fed9640 1 -- 172.21.15.97:0/2905125653 shutdown_connections 2024-04-23T10:35:29.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:29.936+0000 7f722fed9640 1 --2- 172.21.15.97:0/2905125653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f72281063a0 0x7f7228106850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:29.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:29.936+0000 7f722fed9640 1 --2- 172.21.15.97:0/2905125653 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72281018c0 0x7f7228105d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:29.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:29.936+0000 7f722fed9640 1 --2- 172.21.15.97:0/2905125653 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7228100fa0 0x7f7228101380 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:29.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:29.936+0000 7f722fed9640 1 -- 172.21.15.97:0/2905125653 >> 172.21.15.97:0/2905125653 conn(0x7f72280771d0 msgr2=0x7f72280775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:35:29.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:29.937+0000 7f722fed9640 1 -- 172.21.15.97:0/2905125653 shutdown_connections 2024-04-23T10:35:29.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:29.937+0000 7f722fed9640 1 -- 172.21.15.97:0/2905125653 wait complete. 2024-04-23T10:35:29.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:29.937+0000 7f722fed9640 1 Processor -- start 2024-04-23T10:35:29.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:29.937+0000 7f722fed9640 1 -- start start 2024-04-23T10:35:29.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:29.937+0000 7f722fed9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7228100fa0 0x7f722819edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:29.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:29.937+0000 7f722fed9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72281018c0 0x7f722819f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:29.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:29.937+0000 7f722fed9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72281063a0 0x7f72281a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:29.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:29.937+0000 7f722fed9640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7228118780 con 0x7f72281063a0 2024-04-23T10:35:29.938 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:29.937+0000 7f722fed9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7228118600 con 0x7f72281018c0 2024-04-23T10:35:29.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:29.937+0000 7f722fed9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7228118900 con 0x7f7228100fa0 2024-04-23T10:35:29.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:29.937+0000 7f722e476640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72281063a0 0x7f72281a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:29.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:29.938+0000 7f722e476640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72281063a0 0x7f72281a3680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42830/0 (socket says 172.21.15.97:42830) 2024-04-23T10:35:29.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:29.938+0000 7f722e476640 1 -- 172.21.15.97:0/4049579552 learned_addr learned my addr 172.21.15.97:0/4049579552 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:35:29.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:29.938+0000 7f722dc75640 1 --2- 172.21.15.97:0/4049579552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7228100fa0 0x7f722819edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:29.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:29.938+0000 7f722d474640 1 --2- 172.21.15.97:0/4049579552 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72281018c0 0x7f722819f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:29.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:29.938+0000 7f722e476640 1 -- 172.21.15.97:0/4049579552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7228100fa0 msgr2=0x7f722819edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:29.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:29.938+0000 7f722e476640 1 --2- 172.21.15.97:0/4049579552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7228100fa0 0x7f722819edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:29.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:29.938+0000 7f722e476640 1 -- 172.21.15.97:0/4049579552 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72281018c0 msgr2=0x7f722819f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:29.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:29.938+0000 7f722e476640 1 --2- 172.21.15.97:0/4049579552 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72281018c0 0x7f722819f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:29.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:29.938+0000 7f722e476640 1 -- 172.21.15.97:0/4049579552 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72281a3db0 con 0x7f72281063a0 2024-04-23T10:35:29.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:29.938+0000 7f722e476640 1 --2- 172.21.15.97:0/4049579552 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72281063a0 0x7f72281a3680 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f7220009fd0 tx=0x7f722000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:35:29.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:29.938+0000 7f721effd640 1 -- 172.21.15.97:0/4049579552 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7220017070 con 0x7f72281063a0 2024-04-23T10:35:29.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:29.938+0000 7f721effd640 1 -- 172.21.15.97:0/4049579552 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7220003e20 con 0x7f72281063a0 2024-04-23T10:35:29.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:29.938+0000 7f722fed9640 1 -- 172.21.15.97:0/4049579552 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72281a40a0 con 0x7f72281063a0 2024-04-23T10:35:29.939 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:29.938+0000 7f721effd640 1 -- 172.21.15.97:0/4049579552 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7220010460 con 0x7f72281063a0 2024-04-23T10:35:29.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:29.938+0000 7f722fed9640 1 -- 172.21.15.97:0/4049579552 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72281ab930 con 0x7f72281063a0 2024-04-23T10:35:29.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:29.939+0000 7f722fed9640 1 -- 172.21.15.97:0/4049579552 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f722806add0 con 0x7f72281063a0 2024-04-23T10:35:29.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:29.940+0000 7f721effd640 1 -- 172.21.15.97:0/4049579552 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7220003970 con 0x7f72281063a0 2024-04-23T10:35:29.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:29.941+0000 7f721effd640 1 --2- 172.21.15.97:0/4049579552 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f72140a3d20 0x7f72140a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:29.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:29.941+0000 7f722dc75640 1 --2- 172.21.15.97:0/4049579552 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f72140a3d20 0x7f72140a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:29.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:29.941+0000 7f721effd640 1 -- 172.21.15.97:0/4049579552 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8484+0+0 (secure 0 0 0) 0x7f72200d0500 con 0x7f72281063a0 2024-04-23T10:35:29.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:29.941+0000 7f722dc75640 1 --2- 172.21.15.97:0/4049579552 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f72140a3d20 0x7f72140a61e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7224009900 tx=0x7f7224000bc0 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:35:29.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:29.945+0000 7f721effd640 1 -- 172.21.15.97:0/4049579552 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f722009cbc0 con 0x7f72281063a0 2024-04-23T10:35:30.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.163+0000 7f722fed9640 1 -- 172.21.15.97:0/4049579552 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f72281a0110 con 0x7f72281063a0 2024-04-23T10:35:30.162 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.164+0000 7f721effd640 1 -- 172.21.15.97:0/4049579552 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11931 (secure 0 0 0) 0x7f722009c4c0 con 0x7f72281063a0 2024-04-23T10:35:30.163 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:35:30.163 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3946878762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3946878762},{"type":"v1","addr":"172.21.15.97:6835","nonce":3946878762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/209800861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":209800861},{"type":"v1","addr":"172.21.15.97:6837","nonce":209800861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T10:24:06.910134+0000","modified":"2024-04-23T10:26:09.820985+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6839/418013575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":418013575},{"type":"v1","addr":"172.21.15.97:6839","nonce":418013575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6841/747242680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":747242680},{"type":"v1","addr":"172.21.15.97:6841","nonce":747242680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6833/4292847121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4292847121},{"type":"v1","addr":"172.21.15.183:6833","nonce":4292847121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6833/3885816565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3885816565},{"type":"v1","addr":"172.21.15.196:6833","nonce":3885816565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1897093105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1897093105},{"type":"v1","addr":"172.21.15.183:6835","nonce":1897093105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1171477563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1171477563},{"type":"v1","addr":"172.21.15.196:6835","nonce":1171477563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1898341911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1898341911},{"type":"v1","addr":"172.21.15.183:6837","nonce":1898341911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.196:6837/3094946502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3094946502},{"type":"v1","addr":"172.21.15.196:6837","nonce":3094946502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.196:6839/83843556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":83843556},{"type":"v1","addr":"172.21.15.196:6839","nonce":83843556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6839/1763841130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1763841130},{"type":"v1","addr":"172.21.15.183:6839","nonce":1763841130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24487,24403,24398,24469,24488]},"id":1}]} 2024-04-23T10:35:30.163 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 26 2024-04-23T10:35:30.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.166+0000 7f722fed9640 1 -- 172.21.15.97:0/4049579552 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f72140a3d20 msgr2=0x7f72140a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:30.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.166+0000 7f722fed9640 1 --2- 172.21.15.97:0/4049579552 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f72140a3d20 0x7f72140a61e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7224009900 tx=0x7f7224000bc0 comp rx=0 tx=0).stop 2024-04-23T10:35:30.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.166+0000 7f722fed9640 1 -- 172.21.15.97:0/4049579552 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72281063a0 msgr2=0x7f72281a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:30.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.166+0000 7f722fed9640 1 --2- 172.21.15.97:0/4049579552 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72281063a0 0x7f72281a3680 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f7220009fd0 tx=0x7f722000b260 comp rx=0 tx=0).stop 2024-04-23T10:35:30.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.166+0000 7f722fed9640 1 -- 172.21.15.97:0/4049579552 shutdown_connections 2024-04-23T10:35:30.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.166+0000 7f722fed9640 1 --2- 172.21.15.97:0/4049579552 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f72140a3d20 0x7f72140a61e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:30.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.166+0000 7f722fed9640 1 --2- 172.21.15.97:0/4049579552 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72281063a0 0x7f72281a3680 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:30.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.166+0000 7f722fed9640 1 --2- 172.21.15.97:0/4049579552 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72281018c0 0x7f722819f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:30.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.166+0000 7f722fed9640 1 --2- 172.21.15.97:0/4049579552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7228100fa0 0x7f722819edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:30.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.166+0000 7f722fed9640 1 -- 172.21.15.97:0/4049579552 >> 172.21.15.97:0/4049579552 conn(0x7f72280771d0 msgr2=0x7f7228103d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:35:30.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.166+0000 7f722fed9640 1 -- 172.21.15.97:0/4049579552 shutdown_connections 2024-04-23T10:35:30.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.166+0000 7f722fed9640 1 -- 172.21.15.97:0/4049579552 wait complete. 2024-04-23T10:35:30.176 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --set-id 13110ad2 --release --await' 2024-04-23T10:35:30.176 DEBUG:teuthology.orchestra.run.smithi097:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24488 quiesce db --set-id 13110ad2 --release --await 2024-04-23T10:35:30.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.282+0000 7f431fa4e640 1 -- 172.21.15.97:0/224268917 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4318101ad0 msgr2=0x7f431810dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:30.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.282+0000 7f431fa4e640 1 --2- 172.21.15.97:0/224268917 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4318101ad0 0x7f431810dcf0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f4310009ca0 tx=0x7f431002a8b0 comp rx=0 tx=0).stop 2024-04-23T10:35:30.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.282+0000 7f431fa4e640 1 -- 172.21.15.97:0/224268917 shutdown_connections 2024-04-23T10:35:30.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.282+0000 7f431fa4e640 1 --2- 172.21.15.97:0/224268917 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4318101ad0 0x7f431810dcf0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:30.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.282+0000 7f431fa4e640 1 --2- 172.21.15.97:0/224268917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4318101130 0x7f4318101590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:30.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.282+0000 7f431fa4e640 1 --2- 172.21.15.97:0/224268917 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4318105ea0 0x7f4318106280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:30.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.282+0000 7f431fa4e640 1 -- 172.21.15.97:0/224268917 >> 172.21.15.97:0/224268917 conn(0x7f4318077e30 msgr2=0x7f4318078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:35:30.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.282+0000 7f431fa4e640 1 -- 172.21.15.97:0/224268917 shutdown_connections 2024-04-23T10:35:30.281 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.282+0000 7f431fa4e640 1 -- 172.21.15.97:0/224268917 wait complete. 2024-04-23T10:35:30.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.282+0000 7f431fa4e640 1 Processor -- start 2024-04-23T10:35:30.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.283+0000 7f431fa4e640 1 -- start start 2024-04-23T10:35:30.283 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.283+0000 7f431fa4e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4318101130 0x7f431819e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:30.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.283+0000 7f431fa4e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4318101ad0 0x7f431819e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:30.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.283+0000 7f431fa4e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4318105ea0 0x7f43181985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:30.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.283+0000 7f431fa4e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f43181111f0 con 0x7f4318101130 2024-04-23T10:35:30.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.283+0000 7f431fa4e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4318111070 con 0x7f4318101ad0 2024-04-23T10:35:30.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.283+0000 7f431fa4e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4318111370 con 0x7f4318105ea0 2024-04-23T10:35:30.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.283+0000 7f431d7ea640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4318101130 0x7f431819e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:30.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.283+0000 7f431d7ea640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4318101130 0x7f431819e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42862/0 (socket says 172.21.15.97:42862) 2024-04-23T10:35:30.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.283+0000 7f431d7ea640 1 -- 172.21.15.97:0/42598998 learned_addr learned my addr 172.21.15.97:0/42598998 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:35:30.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.283+0000 7f431cfe9640 1 --2- 172.21.15.97:0/42598998 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4318101ad0 0x7f431819e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:30.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.283+0000 7f431d7ea640 1 -- 172.21.15.97:0/42598998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4318105ea0 msgr2=0x7f43181985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:30.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.283+0000 7f431dfeb640 1 --2- 172.21.15.97:0/42598998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4318105ea0 0x7f43181985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:30.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.283+0000 7f431d7ea640 1 --2- 172.21.15.97:0/42598998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4318105ea0 0x7f43181985d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:30.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.283+0000 7f431d7ea640 1 -- 172.21.15.97:0/42598998 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4318101ad0 msgr2=0x7f431819e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:30.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.283+0000 7f431d7ea640 1 --2- 172.21.15.97:0/42598998 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4318101ad0 0x7f431819e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:30.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.283+0000 7f431d7ea640 1 -- 172.21.15.97:0/42598998 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4318198d50 con 0x7f4318101130 2024-04-23T10:35:30.284 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.283+0000 7f431cfe9640 1 --2- 172.21.15.97:0/42598998 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4318101ad0 0x7f431819e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:35:30.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.283+0000 7f431d7ea640 1 --2- 172.21.15.97:0/42598998 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4318101130 0x7f431819e420 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f430c00e9f0 tx=0x7f430c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:35:30.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.283+0000 7f430a7fc640 1 -- 172.21.15.97:0/42598998 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430c00cd80 con 0x7f4318101130 2024-04-23T10:35:30.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.283+0000 7f430a7fc640 1 -- 172.21.15.97:0/42598998 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f430c0108e0 con 0x7f4318101130 2024-04-23T10:35:30.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.283+0000 7f430a7fc640 1 -- 172.21.15.97:0/42598998 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430c018980 con 0x7f4318101130 2024-04-23T10:35:30.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.283+0000 7f431fa4e640 1 -- 172.21.15.97:0/42598998 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4318199040 con 0x7f4318101130 2024-04-23T10:35:30.285 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.283+0000 7f431fa4e640 1 -- 172.21.15.97:0/42598998 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43181a52e0 con 0x7f4318101130 2024-04-23T10:35:30.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.285+0000 7f430a7fc640 1 -- 172.21.15.97:0/42598998 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f430c010430 con 0x7f4318101130 2024-04-23T10:35:30.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.286+0000 7f4322766740 1 Processor -- start 2024-04-23T10:35:30.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.286+0000 7f4322766740 1 -- start start 2024-04-23T10:35:30.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.286+0000 7f4322766740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8a93c8490 0x55e8a93c8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:30.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.286+0000 7f430a7fc640 1 --2- 172.21.15.97:0/42598998 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f43040a3d20 0x7f43040a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:30.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.286+0000 7f4322766740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8a93c8db0 0x55e8a93d2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:30.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.286+0000 7f4322766740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8a93d3480 0x55e8a93d5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:30.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.286+0000 7f4322766740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e8a932d6d0 con 0x55e8a93c8db0 2024-04-23T10:35:30.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.286+0000 7f4322766740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e8a92ca1a0 con 0x55e8a93d3480 2024-04-23T10:35:30.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.286+0000 7f4322766740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e8a9329690 con 0x55e8a93c8490 2024-04-23T10:35:30.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.286+0000 7f431cfe9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8a93c8db0 0x55e8a93d2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:30.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.286+0000 7f431cfe9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8a93c8db0 0x55e8a93d2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42866/0 (socket says 172.21.15.97:42866) 2024-04-23T10:35:30.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.286+0000 7f430a7fc640 1 -- 172.21.15.97:0/42598998 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8484+0+0 (secure 0 0 0) 0x7f430c0d0c00 con 0x7f4318101130 2024-04-23T10:35:30.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.286+0000 7f431cfe9640 1 -- 172.21.15.97:0/4280940021 learned_addr learned my addr 172.21.15.97:0/4280940021 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:35:30.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.286+0000 7f431d7ea640 1 --2- 172.21.15.97:0/42598998 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f43040a3d20 0x7f43040a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:30.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.287+0000 7f431dfeb640 1 --2- 172.21.15.97:0/4280940021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8a93c8490 0x55e8a93c8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:30.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.287+0000 7f431dfeb640 1 --2- 172.21.15.97:0/4280940021 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8a93d3480 0x55e8a93d5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:30.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.287+0000 7f431cfe9640 1 -- 172.21.15.97:0/4280940021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8a93c8490 msgr2=0x55e8a93c8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:35:30.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.287+0000 7f431cfe9640 1 --2- 172.21.15.97:0/4280940021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8a93c8490 0x55e8a93c8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:30.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.287+0000 7f431cfe9640 1 -- 172.21.15.97:0/4280940021 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8a93d3480 msgr2=0x55e8a93d5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:35:30.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.287+0000 7f431cfe9640 1 --2- 172.21.15.97:0/4280940021 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8a93d3480 0x55e8a93d5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:30.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.287+0000 7f431cfe9640 1 -- 172.21.15.97:0/4280940021 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8a92d45b0 con 0x55e8a93c8db0 2024-04-23T10:35:30.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.287+0000 7f431dfeb640 1 --2- 172.21.15.97:0/4280940021 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8a93d3480 0x55e8a93d5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:35:30.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.287+0000 7f431d7ea640 1 --2- 172.21.15.97:0/42598998 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f43040a3d20 0x7f43040a61e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f430c099590 tx=0x7f430c00cf20 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:35:30.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.287+0000 7f431cfe9640 1 --2- 172.21.15.97:0/4280940021 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8a93c8db0 0x55e8a93d2f40 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f4314009cb0 tx=0x7f43140117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b87e6505823dce86 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:35:30.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.287+0000 7f43037fe640 1 -- 172.21.15.97:0/4280940021 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4314012450 con 0x55e8a93c8db0 2024-04-23T10:35:30.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.287+0000 7f43037fe640 1 -- 172.21.15.97:0/4280940021 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4314012aa0 con 0x55e8a93c8db0 2024-04-23T10:35:30.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.287+0000 7f43037fe640 1 -- 172.21.15.97:0/4280940021 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431400fc00 con 0x55e8a93c8db0 2024-04-23T10:35:30.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.288+0000 7f4322766740 1 -- 172.21.15.97:0/4280940021 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8a93c8db0 msgr2=0x55e8a93d2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:30.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.288+0000 7f4322766740 1 --2- 172.21.15.97:0/4280940021 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8a93c8db0 0x55e8a93d2f40 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f4314009cb0 tx=0x7f43140117b0 comp rx=0 tx=0).stop 2024-04-23T10:35:30.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.288+0000 7f4322766740 1 -- 172.21.15.97:0/4280940021 shutdown_connections 2024-04-23T10:35:30.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.288+0000 7f4322766740 1 --2- 172.21.15.97:0/4280940021 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8a93d3480 0x55e8a93d5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:30.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.288+0000 7f4322766740 1 --2- 172.21.15.97:0/4280940021 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8a93c8db0 0x55e8a93d2f40 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:30.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.288+0000 7f4322766740 1 --2- 172.21.15.97:0/4280940021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8a93c8490 0x55e8a93c8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:30.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.288+0000 7f4322766740 1 -- 172.21.15.97:0/4280940021 >> 172.21.15.97:0/4280940021 conn(0x55e8a93c47b0 msgr2=0x55e8a93c4b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:35:30.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.288+0000 7f4322766740 1 -- 172.21.15.97:0/4280940021 shutdown_connections 2024-04-23T10:35:30.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.288+0000 7f4322766740 1 -- 172.21.15.97:0/4280940021 wait complete. 2024-04-23T10:35:30.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.288+0000 7f4322766740 1 -- start start 2024-04-23T10:35:30.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.288+0000 7f4322766740 1 Processor -- start 2024-04-23T10:35:30.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.288+0000 7f4322766740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8a93c8490 0x55e8a93d2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:30.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.289+0000 7f4322766740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8a93c8db0 0x55e8a93d01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:30.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.289+0000 7f4322766740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8a93d3480 0x55e8a93d0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:30.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.289+0000 7f4322766740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e8a93d6310 con 0x55e8a93c8490 2024-04-23T10:35:30.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.289+0000 7f4322766740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e8a92c7b20 con 0x55e8a93c8db0 2024-04-23T10:35:30.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.289+0000 7f4322766740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e8a93d6490 con 0x55e8a93d3480 2024-04-23T10:35:30.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.289+0000 7f4322766740 10 client.0 _ll_register_callbacks cb 0x55e8a92c7530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:35:30.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.289+0000 7f431dfeb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8a93c8490 0x55e8a93d2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:30.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.289+0000 7f431dfeb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8a93c8490 0x55e8a93d2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42882/0 (socket says 172.21.15.97:42882) 2024-04-23T10:35:30.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.289+0000 7f431dfeb640 1 -- 172.21.15.97:0/3957589083 learned_addr learned my addr 172.21.15.97:0/3957589083 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:35:30.294 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.289+0000 7f431dfeb640 1 --2- 172.21.15.97:0/3957589083 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8a93d3480 0x55e8a93d0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:30.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.289+0000 7f431cfe9640 1 --2- 172.21.15.97:0/3957589083 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8a93c8db0 0x55e8a93d01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:30.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.289+0000 7f431dfeb640 1 -- 172.21.15.97:0/3957589083 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8a93d3480 msgr2=0x55e8a93d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:35:30.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.289+0000 7f431dfeb640 1 --2- 172.21.15.97:0/3957589083 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8a93d3480 0x55e8a93d0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:30.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.289+0000 7f431dfeb640 1 -- 172.21.15.97:0/3957589083 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8a93c8db0 msgr2=0x55e8a93d01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:35:30.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.289+0000 7f431dfeb640 1 --2- 172.21.15.97:0/3957589083 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8a93c8db0 0x55e8a93d01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:30.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.289+0000 7f431dfeb640 1 -- 172.21.15.97:0/3957589083 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8a903f920 con 0x55e8a93c8490 2024-04-23T10:35:30.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.289+0000 7f431cfe9640 1 --2- 172.21.15.97:0/3957589083 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8a93c8db0 0x55e8a93d01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:35:30.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.289+0000 7f431dfeb640 1 --2- 172.21.15.97:0/3957589083 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8a93c8490 0x55e8a93d2f00 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f4310004af0 tx=0x7f4310004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68f1d45a159323e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:35:30.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.289+0000 7f4300ff9640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:35:30.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.289+0000 7f4300ff9640 1 -- 172.21.15.97:0/3957589083 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4310003b70 con 0x55e8a93c8490 2024-04-23T10:35:30.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.289+0000 7f4300ff9640 1 -- 172.21.15.97:0/3957589083 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8484+0+0 (secure 0 0 0) 0x7f4310003d10 con 0x55e8a93c8490 2024-04-23T10:35:30.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.289+0000 7f4322766740 1 -- 172.21.15.97:0/3957589083 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4314012aa0 con 0x55e8a93c8490 2024-04-23T10:35:30.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.289+0000 7f4300ff9640 1 -- 172.21.15.97:0/3957589083 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4310004040 con 0x55e8a93c8490 2024-04-23T10:35:30.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.289+0000 7f4322766740 10 client.16917 fetch_fsmap learned FSMap version 26 2024-04-23T10:35:30.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.290+0000 7f4322766740 1 -- 172.21.15.97:0/3957589083 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e8a910bd30 con 0x55e8a93c8490 2024-04-23T10:35:30.295 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.290+0000 7f4300ff9640 1 -- 172.21.15.97:0/3957589083 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4314012aa0 con 0x55e8a93c8490 2024-04-23T10:35:30.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.290+0000 7f4300ff9640 1 -- 172.21.15.97:0/3957589083 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f43100317b0 con 0x55e8a93c8490 2024-04-23T10:35:30.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.290+0000 7f4322766740 10 client.16917 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:35:30.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.290+0000 7f4322766740 10 client.16917 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:35:30.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.290+0000 7f4322766740 20 client.16917 populate_metadata read hostname 'smithi097' 2024-04-23T10:35:30.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.290+0000 7f4322766740 1 --2- 172.21.15.97:0/3957589083 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55e8a93e3550 0x55e8a93e5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:30.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.290+0000 7f4322766740 4 client.16917 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:35:30.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.290+0000 7f4322766740 1 -- 172.21.15.97:0/3957589083 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e8a92eb500 con 0x55e8a93e3550 2024-04-23T10:35:30.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.290+0000 7f431cfe9640 1 --2- 172.21.15.97:0/3957589083 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55e8a93e3550 0x55e8a93e5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:30.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.291+0000 7f4300ff9640 10 client.16917 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:35:30.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.291+0000 7f431cfe9640 1 --2- 172.21.15.97:0/3957589083 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55e8a93e3550 0x55e8a93e5980 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=898567df04b97c11 server_cookie=bb601e28593a3e99 in_seq=0 out_seq=0 2024-04-23T10:35:30.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.291+0000 7f4300ff9640 1 -- 172.21.15.97:0/3957589083 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e8a92eb500 con 0x55e8a93e3550 2024-04-23T10:35:30.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.291+0000 7f4300ff9640 10 client.16917 handle_command_reply: tid=0 2024-04-23T10:35:30.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.291+0000 7f4322766740 1 client.16917 shutdown 2024-04-23T10:35:30.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.291+0000 7f4322766740 1 -- 172.21.15.97:0/3957589083 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8a93c8490 msgr2=0x55e8a93d2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:30.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.291+0000 7f4322766740 1 --2- 172.21.15.97:0/3957589083 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8a93c8490 0x55e8a93d2f00 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f4310004af0 tx=0x7f4310004a40 comp rx=0 tx=0).stop 2024-04-23T10:35:30.296 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.291+0000 7f4322766740 1 -- 172.21.15.97:0/3957589083 shutdown_connections 2024-04-23T10:35:30.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.291+0000 7f4322766740 1 --2- 172.21.15.97:0/3957589083 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55e8a93e3550 0x55e8a93e5980 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:30.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.291+0000 7f4322766740 1 --2- 172.21.15.97:0/3957589083 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8a93d3480 0x55e8a93d0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:30.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.291+0000 7f4322766740 1 --2- 172.21.15.97:0/3957589083 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8a93c8db0 0x55e8a93d01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:30.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.291+0000 7f4300ff9640 0 client.16917 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:35:30.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.291+0000 7f4322766740 1 --2- 172.21.15.97:0/3957589083 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8a93c8490 0x55e8a93d2f00 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:30.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.291+0000 7f4322766740 1 -- 172.21.15.97:0/3957589083 >> 172.21.15.97:0/3957589083 conn(0x55e8a93c47b0 msgr2=0x55e8a93ca5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:35:30.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.292+0000 7f4322766740 1 -- 172.21.15.97:0/3957589083 shutdown_connections 2024-04-23T10:35:30.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.292+0000 7f4322766740 1 -- 172.21.15.97:0/3957589083 wait complete. 2024-04-23T10:35:30.297 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.292+0000 7f4322766740 20 client.16917 trim_cache size 0 max 16384 2024-04-23T10:35:30.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.308+0000 7f4322766740 1 Processor -- start 2024-04-23T10:35:30.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.308+0000 7f4322766740 1 -- start start 2024-04-23T10:35:30.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.309+0000 7f4322766740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8a93c8490 0x55e8a93df1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:30.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.309+0000 7f4322766740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8a93c8db0 0x55e8a93d2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:30.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.309+0000 7f4322766740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8a93d3480 0x55e8a93ca330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:30.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.309+0000 7f4322766740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f43181ada60 con 0x55e8a93d3480 2024-04-23T10:35:30.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.309+0000 7f4322766740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f43181af5a0 con 0x55e8a93c8db0 2024-04-23T10:35:30.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.309+0000 7f4322766740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f43181ae130 con 0x55e8a93c8490 2024-04-23T10:35:30.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.309+0000 7f431dfeb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8a93d3480 0x55e8a93ca330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:30.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.309+0000 7f431dfeb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8a93d3480 0x55e8a93ca330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42888/0 (socket says 172.21.15.97:42888) 2024-04-23T10:35:30.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.309+0000 7f431dfeb640 1 -- 172.21.15.97:0/3954980401 learned_addr learned my addr 172.21.15.97:0/3954980401 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:35:30.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.309+0000 7f431dfeb640 1 --2- 172.21.15.97:0/3954980401 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8a93c8490 0x55e8a93df1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:30.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.309+0000 7f431dfeb640 1 -- 172.21.15.97:0/3954980401 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8a93c8490 msgr2=0x55e8a93df1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:35:30.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.309+0000 7f431dfeb640 1 --2- 172.21.15.97:0/3954980401 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8a93c8490 0x55e8a93df1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:30.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.309+0000 7f431dfeb640 1 -- 172.21.15.97:0/3954980401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8a93c8db0 msgr2=0x55e8a93d2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:35:30.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.309+0000 7f431cfe9640 1 --2- 172.21.15.97:0/3954980401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8a93c8db0 0x55e8a93d2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:30.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.309+0000 7f431dfeb640 1 --2- 172.21.15.97:0/3954980401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8a93c8db0 0x55e8a93d2a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:30.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.309+0000 7f431dfeb640 1 -- 172.21.15.97:0/3954980401 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8a93cbdd0 con 0x55e8a93d3480 2024-04-23T10:35:30.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.309+0000 7f431dfeb640 1 --2- 172.21.15.97:0/3954980401 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8a93d3480 0x55e8a93ca330 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f4310031040 tx=0x7f431002a8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aea326b1f80f96e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:35:30.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.309+0000 7f4302ffd640 1 -- 172.21.15.97:0/3954980401 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4310003b70 con 0x55e8a93d3480 2024-04-23T10:35:30.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.309+0000 7f4302ffd640 1 -- 172.21.15.97:0/3954980401 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4310054040 con 0x55e8a93d3480 2024-04-23T10:35:30.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.309+0000 7f4302ffd640 1 -- 172.21.15.97:0/3954980401 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4310004640 con 0x55e8a93d3480 2024-04-23T10:35:30.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.310+0000 7f4322766740 1 -- 172.21.15.97:0/3954980401 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8a93d3480 msgr2=0x55e8a93ca330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:30.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.310+0000 7f4322766740 1 --2- 172.21.15.97:0/3954980401 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8a93d3480 0x55e8a93ca330 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f4310031040 tx=0x7f431002a8b0 comp rx=0 tx=0).stop 2024-04-23T10:35:30.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.310+0000 7f4322766740 1 -- 172.21.15.97:0/3954980401 shutdown_connections 2024-04-23T10:35:30.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.310+0000 7f4322766740 1 --2- 172.21.15.97:0/3954980401 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8a93d3480 0x55e8a93ca330 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:30.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.310+0000 7f4322766740 1 --2- 172.21.15.97:0/3954980401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8a93c8db0 0x55e8a93d2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:30.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.310+0000 7f4322766740 1 --2- 172.21.15.97:0/3954980401 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8a93c8490 0x55e8a93df1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:30.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.310+0000 7f4322766740 1 -- 172.21.15.97:0/3954980401 >> 172.21.15.97:0/3954980401 conn(0x55e8a93c47b0 msgr2=0x55e8a93c9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:35:30.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.310+0000 7f4322766740 1 -- 172.21.15.97:0/3954980401 shutdown_connections 2024-04-23T10:35:30.311 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.310+0000 7f4322766740 1 -- 172.21.15.97:0/3954980401 wait complete. 2024-04-23T10:35:30.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.310+0000 7f4322766740 1 -- start start 2024-04-23T10:35:30.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.311+0000 7f4322766740 1 Processor -- start 2024-04-23T10:35:30.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.311+0000 7f4322766740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8a93c8490 0x55e8a93dbb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:30.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.311+0000 7f4322766740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8a93c8db0 0x55e8a93cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:30.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.311+0000 7f4322766740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8a93d3480 0x55e8a93cdec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:30.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.311+0000 7f4322766740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f43181aeed0 con 0x55e8a93c8db0 2024-04-23T10:35:30.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.311+0000 7f4322766740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f43181ae800 con 0x55e8a93d3480 2024-04-23T10:35:30.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.311+0000 7f4322766740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f43181aba60 con 0x55e8a93c8490 2024-04-23T10:35:30.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.311+0000 7f4322766740 10 client.0 _ll_register_callbacks cb 0x7f4310009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:35:30.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.311+0000 7f431cfe9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8a93c8db0 0x55e8a93cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:30.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.311+0000 7f431cfe9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8a93c8db0 0x55e8a93cd8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:42890/0 (socket says 172.21.15.97:42890) 2024-04-23T10:35:30.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.311+0000 7f431cfe9640 1 -- 172.21.15.97:0/322606657 learned_addr learned my addr 172.21.15.97:0/322606657 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:35:30.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.311+0000 7f431dfeb640 1 --2- 172.21.15.97:0/322606657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8a93c8490 0x55e8a93dbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:30.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.311+0000 7f431dfeb640 1 --2- 172.21.15.97:0/322606657 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8a93d3480 0x55e8a93cdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:30.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.311+0000 7f431cfe9640 1 -- 172.21.15.97:0/322606657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8a93c8490 msgr2=0x55e8a93dbb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:35:30.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.311+0000 7f431cfe9640 1 --2- 172.21.15.97:0/322606657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8a93c8490 0x55e8a93dbb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:30.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.311+0000 7f431cfe9640 1 -- 172.21.15.97:0/322606657 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8a93d3480 msgr2=0x55e8a93cdec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:35:30.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.311+0000 7f431cfe9640 1 --2- 172.21.15.97:0/322606657 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8a93d3480 0x55e8a93cdec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:30.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.311+0000 7f431cfe9640 1 -- 172.21.15.97:0/322606657 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8a93d4800 con 0x55e8a93c8db0 2024-04-23T10:35:30.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.311+0000 7f431dfeb640 1 --2- 172.21.15.97:0/322606657 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8a93d3480 0x55e8a93cdec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:35:30.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.312+0000 7f431cfe9640 1 --2- 172.21.15.97:0/322606657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8a93c8db0 0x55e8a93cd8b0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f4314009840 tx=0x7f4314009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e1b95fbd1b9af61 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:35:30.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.312+0000 7f4300ff9640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:35:30.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.312+0000 7f4300ff9640 1 -- 172.21.15.97:0/322606657 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4314005040 con 0x55e8a93c8db0 2024-04-23T10:35:30.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.312+0000 7f4300ff9640 1 -- 172.21.15.97:0/322606657 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8484+0+0 (secure 0 0 0) 0x7f43140051e0 con 0x55e8a93c8db0 2024-04-23T10:35:30.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.312+0000 7f4322766740 1 -- 172.21.15.97:0/322606657 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4310054040 con 0x55e8a93c8db0 2024-04-23T10:35:30.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.312+0000 7f4300ff9640 1 -- 172.21.15.97:0/322606657 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4314005420 con 0x55e8a93c8db0 2024-04-23T10:35:30.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.312+0000 7f4322766740 10 client.16923 fetch_fsmap learned FSMap version 26 2024-04-23T10:35:30.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.312+0000 7f4322766740 1 -- 172.21.15.97:0/322606657 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e8a944a080 con 0x55e8a93c8db0 2024-04-23T10:35:30.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.312+0000 7f4300ff9640 1 -- 172.21.15.97:0/322606657 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4310054040 con 0x55e8a93c8db0 2024-04-23T10:35:30.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.312+0000 7f4300ff9640 1 -- 172.21.15.97:0/322606657 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f431402e070 con 0x55e8a93c8db0 2024-04-23T10:35:30.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.312+0000 7f4322766740 10 client.16923 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:35:30.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.312+0000 7f4322766740 10 client.16923 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:35:30.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.312+0000 7f4322766740 20 client.16923 populate_metadata read hostname 'smithi097' 2024-04-23T10:35:30.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.312+0000 7f4322766740 1 --2- 172.21.15.97:0/322606657 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55e8a944a520 0x55e8a944c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:30.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.312+0000 7f4322766740 4 client.16923 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "13110ad2", "release": true, "await": true}] 2024-04-23T10:35:30.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.312+0000 7f4322766740 1 -- 172.21.15.97:0/322606657 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "quiesce db", "set_id": "13110ad2", "release": true, "await": true}) -- 0x55e8a933b0f0 con 0x55e8a944a520 2024-04-23T10:35:30.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.313+0000 7f431dfeb640 1 --2- 172.21.15.97:0/322606657 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55e8a944a520 0x55e8a944c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:30.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.313+0000 7f4300ff9640 10 client.16923 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:35:30.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.313+0000 7f431dfeb640 1 --2- 172.21.15.97:0/322606657 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55e8a944a520 0x55e8a944c910 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3261bdf2147474ee server_cookie=3823f0cb9e9c7a35 in_seq=0 out_seq=0 2024-04-23T10:35:30.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.317+0000 7f4300ff9640 1 -- 172.21.15.97:0/322606657 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55e8a933b0f0 con 0x55e8a944a520 2024-04-23T10:35:30.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.317+0000 7f4300ff9640 10 client.16923 handle_command_reply: tid=0 2024-04-23T10:35:30.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.317+0000 7f4322766740 1 client.16923 shutdown 2024-04-23T10:35:30.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.317+0000 7f4322766740 1 -- 172.21.15.97:0/322606657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8a93c8db0 msgr2=0x55e8a93cd8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:30.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.317+0000 7f4322766740 1 --2- 172.21.15.97:0/322606657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8a93c8db0 0x55e8a93cd8b0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f4314009840 tx=0x7f4314009650 comp rx=0 tx=0).stop 2024-04-23T10:35:30.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.317+0000 7f4322766740 1 -- 172.21.15.97:0/322606657 shutdown_connections 2024-04-23T10:35:30.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.317+0000 7f4322766740 1 --2- 172.21.15.97:0/322606657 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55e8a944a520 0x55e8a944c910 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:30.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.317+0000 7f4322766740 1 --2- 172.21.15.97:0/322606657 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8a93d3480 0x55e8a93cdec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:30.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.317+0000 7f4322766740 1 --2- 172.21.15.97:0/322606657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e8a93c8db0 0x55e8a93cd8b0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:30.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.317+0000 7f4300ff9640 0 client.16923 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:35:30.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.317+0000 7f4322766740 1 --2- 172.21.15.97:0/322606657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e8a93c8490 0x55e8a93dbb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:30.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.317+0000 7f4322766740 1 -- 172.21.15.97:0/322606657 >> 172.21.15.97:0/322606657 conn(0x55e8a93c47b0 msgr2=0x55e8a93ca470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:35:30.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.317+0000 7f4322766740 1 -- 172.21.15.97:0/322606657 shutdown_connections 2024-04-23T10:35:30.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.317+0000 7f4322766740 1 -- 172.21.15.97:0/322606657 wait complete. 2024-04-23T10:35:30.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.317+0000 7f4322766740 20 client.16923 trim_cache size 0 max 16384 2024-04-23T10:35:30.318 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-23T10:35:30.318 INFO:teuthology.orchestra.run.smithi097.stdout: "epoch": 26, 2024-04-23T10:35:30.318 INFO:teuthology.orchestra.run.smithi097.stdout: "leader": 24488, 2024-04-23T10:35:30.318 INFO:teuthology.orchestra.run.smithi097.stdout: "set_version": 40, 2024-04-23T10:35:30.318 INFO:teuthology.orchestra.run.smithi097.stdout: "sets": { 2024-04-23T10:35:30.318 INFO:teuthology.orchestra.run.smithi097.stdout: "13110ad2": { 2024-04-23T10:35:30.318 INFO:teuthology.orchestra.run.smithi097.stdout: "version": 40, 2024-04-23T10:35:30.318 INFO:teuthology.orchestra.run.smithi097.stdout: "age_ref": 0.0, 2024-04-23T10:35:30.318 INFO:teuthology.orchestra.run.smithi097.stdout: "state": { 2024-04-23T10:35:30.319 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RELEASED", 2024-04-23T10:35:30.319 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0 2024-04-23T10:35:30.319 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:35:30.319 INFO:teuthology.orchestra.run.smithi097.stdout: "timeout": 60.0, 2024-04-23T10:35:30.319 INFO:teuthology.orchestra.run.smithi097.stdout: "expiration": 70.0, 2024-04-23T10:35:30.319 INFO:teuthology.orchestra.run.smithi097.stdout: "members": { 2024-04-23T10:35:30.319 INFO:teuthology.orchestra.run.smithi097.stdout: "file:/": { 2024-04-23T10:35:30.319 INFO:teuthology.orchestra.run.smithi097.stdout: "excluded": false, 2024-04-23T10:35:30.319 INFO:teuthology.orchestra.run.smithi097.stdout: "state": { 2024-04-23T10:35:30.319 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RELEASED", 2024-04-23T10:35:30.319 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0 2024-04-23T10:35:30.319 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:35:30.319 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:35:30.319 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:35:30.319 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:35:30.319 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:35:30.320 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-23T10:35:30.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.318+0000 7f431fa4e640 1 -- 172.21.15.97:0/42598998 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f43040a3d20 msgr2=0x7f43040a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:30.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.318+0000 7f431fa4e640 1 --2- 172.21.15.97:0/42598998 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f43040a3d20 0x7f43040a61e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f430c099590 tx=0x7f430c00cf20 comp rx=0 tx=0).stop 2024-04-23T10:35:30.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.318+0000 7f431fa4e640 1 -- 172.21.15.97:0/42598998 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4318101130 msgr2=0x7f431819e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:30.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.318+0000 7f431fa4e640 1 --2- 172.21.15.97:0/42598998 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4318101130 0x7f431819e420 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f430c00e9f0 tx=0x7f430c00eeb0 comp rx=0 tx=0).stop 2024-04-23T10:35:30.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.318+0000 7f431fa4e640 1 -- 172.21.15.97:0/42598998 shutdown_connections 2024-04-23T10:35:30.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.318+0000 7f431fa4e640 1 --2- 172.21.15.97:0/42598998 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f43040a3d20 0x7f43040a61e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:30.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.318+0000 7f431fa4e640 1 --2- 172.21.15.97:0/42598998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4318105ea0 0x7f43181985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:30.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.318+0000 7f431fa4e640 1 --2- 172.21.15.97:0/42598998 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4318101ad0 0x7f431819e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:30.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.318+0000 7f431fa4e640 1 --2- 172.21.15.97:0/42598998 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4318101130 0x7f431819e420 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:30.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.318+0000 7f431fa4e640 1 -- 172.21.15.97:0/42598998 >> 172.21.15.97:0/42598998 conn(0x7f4318077e30 msgr2=0x7f431806a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:35:30.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.318+0000 7f431fa4e640 1 -- 172.21.15.97:0/42598998 shutdown_connections 2024-04-23T10:35:30.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:30.318+0000 7f431fa4e640 1 -- 172.21.15.97:0/42598998 wait complete. 2024-04-23T10:35:30.327 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '13110ad2', seconds elapsed: 11.0 2024-04-23T10:35:31.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:35:31 smithi183 ceph-mon[32317]: pgmap v288: 97 pgs: 97 active+clean; 4.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-04-23T10:35:31.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:35:31 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/4049579552' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:35:31.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:35:31 smithi196 ceph-mon[32919]: pgmap v288: 97 pgs: 97 active+clean; 4.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-04-23T10:35:31.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:35:31 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/4049579552' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:35:31.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:35:31 smithi097 ceph-mon[29197]: pgmap v288: 97 pgs: 97 active+clean; 4.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-04-23T10:35:31.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:35:31 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/4049579552' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:35:33.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:35:33 smithi183 ceph-mon[32317]: pgmap v289: 97 pgs: 97 active+clean; 4.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-04-23T10:35:33.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:35:33 smithi196 ceph-mon[32919]: pgmap v289: 97 pgs: 97 active+clean; 4.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-04-23T10:35:33.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:35:33 smithi097 ceph-mon[29197]: pgmap v289: 97 pgs: 97 active+clean; 4.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-04-23T10:35:34.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:35:34 smithi183 ceph-mon[32317]: pgmap v290: 97 pgs: 97 active+clean; 4.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 24 KiB/s wr, 17 op/s 2024-04-23T10:35:34.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:35:34 smithi196 ceph-mon[32919]: pgmap v290: 97 pgs: 97 active+clean; 4.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 24 KiB/s wr, 17 op/s 2024-04-23T10:35:34.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:35:34 smithi097 ceph-mon[29197]: pgmap v290: 97 pgs: 97 active+clean; 4.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 24 KiB/s wr, 17 op/s 2024-04-23T10:35:36.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:35:36 smithi196 ceph-mon[32919]: pgmap v291: 97 pgs: 97 active+clean; 4.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-23T10:35:36.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:35:36 smithi097 ceph-mon[29197]: pgmap v291: 97 pgs: 97 active+clean; 4.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-23T10:35:36.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:35:36 smithi183 ceph-mon[32317]: pgmap v291: 97 pgs: 97 active+clean; 4.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-23T10:35:38.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:35:38 smithi196 ceph-mon[32919]: pgmap v292: 97 pgs: 97 active+clean; 5.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 KiB/s wr, 18 op/s 2024-04-23T10:35:38.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:35:38 smithi097 ceph-mon[29197]: pgmap v292: 97 pgs: 97 active+clean; 5.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 KiB/s wr, 18 op/s 2024-04-23T10:35:38.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:35:38 smithi183 ceph-mon[32317]: pgmap v292: 97 pgs: 97 active+clean; 5.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 KiB/s wr, 18 op/s 2024-04-23T10:35:38.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.826+0000 7f536414a640 1 -- 172.21.15.97:0/1779433467 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f535c1011a0 msgr2=0x7f535c101620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:38.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.826+0000 7f536414a640 1 --2- 172.21.15.97:0/1779433467 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f535c1011a0 0x7f535c101620 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f534c009900 tx=0x7f534c02a400 comp rx=0 tx=0).stop 2024-04-23T10:35:38.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.826+0000 7f536414a640 1 -- 172.21.15.97:0/1779433467 shutdown_connections 2024-04-23T10:35:38.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.826+0000 7f536414a640 1 --2- 172.21.15.97:0/1779433467 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f535c101b60 0x7f535c10dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:38.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.826+0000 7f536414a640 1 --2- 172.21.15.97:0/1779433467 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f535c1011a0 0x7f535c101620 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:38.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.826+0000 7f536414a640 1 --2- 172.21.15.97:0/1779433467 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f535c106fb0 0x7f535c107390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:38.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.826+0000 7f536414a640 1 -- 172.21.15.97:0/1779433467 >> 172.21.15.97:0/1779433467 conn(0x7f535c076c50 msgr2=0x7f535c077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:35:38.825 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.826+0000 7f536414a640 1 -- 172.21.15.97:0/1779433467 shutdown_connections 2024-04-23T10:35:38.826 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.826+0000 7f536414a640 1 -- 172.21.15.97:0/1779433467 wait complete. 2024-04-23T10:35:38.827 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.827+0000 7f536414a640 1 Processor -- start 2024-04-23T10:35:38.828 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.827+0000 7f536414a640 1 -- start start 2024-04-23T10:35:38.828 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.827+0000 7f536414a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f535c1011a0 0x7f535c1a0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:38.828 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.827+0000 7f536414a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f535c101b60 0x7f535c1a0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:38.828 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.827+0000 7f536414a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f535c106fb0 0x7f535c19a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:38.828 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.827+0000 7f536414a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f535c111080 con 0x7f535c1011a0 2024-04-23T10:35:38.828 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.827+0000 7f536414a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f535c110f00 con 0x7f535c101b60 2024-04-23T10:35:38.828 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.827+0000 7f536414a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f535c111200 con 0x7f535c106fb0 2024-04-23T10:35:38.828 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.827+0000 7f5361ee6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f535c1011a0 0x7f535c1a0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:38.828 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.827+0000 7f5361ee6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f535c1011a0 0x7f535c1a0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47228/0 (socket says 172.21.15.97:47228) 2024-04-23T10:35:38.828 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.827+0000 7f5361ee6640 1 -- 172.21.15.97:0/1387165747 learned_addr learned my addr 172.21.15.97:0/1387165747 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:35:38.828 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.827+0000 7f53626e7640 1 --2- 172.21.15.97:0/1387165747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f535c106fb0 0x7f535c19a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:38.828 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.827+0000 7f53616e5640 1 --2- 172.21.15.97:0/1387165747 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f535c101b60 0x7f535c1a0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:38.828 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.828+0000 7f5361ee6640 1 -- 172.21.15.97:0/1387165747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f535c106fb0 msgr2=0x7f535c19a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:38.829 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.828+0000 7f5361ee6640 1 --2- 172.21.15.97:0/1387165747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f535c106fb0 0x7f535c19a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:38.829 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.828+0000 7f5361ee6640 1 -- 172.21.15.97:0/1387165747 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f535c101b60 msgr2=0x7f535c1a0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:38.829 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.828+0000 7f5361ee6640 1 --2- 172.21.15.97:0/1387165747 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f535c101b60 0x7f535c1a0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:38.829 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.828+0000 7f5361ee6640 1 -- 172.21.15.97:0/1387165747 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f535c19ad80 con 0x7f535c1011a0 2024-04-23T10:35:38.829 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.828+0000 7f5361ee6640 1 --2- 172.21.15.97:0/1387165747 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f535c1011a0 0x7f535c1a0450 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f535800d770 tx=0x7f535800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:35:38.829 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.828+0000 7f5352ffd640 1 -- 172.21.15.97:0/1387165747 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5358017070 con 0x7f535c1011a0 2024-04-23T10:35:38.829 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.828+0000 7f536414a640 1 -- 172.21.15.97:0/1387165747 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f535c19b070 con 0x7f535c1011a0 2024-04-23T10:35:38.829 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.828+0000 7f5352ffd640 1 -- 172.21.15.97:0/1387165747 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53580054d0 con 0x7f535c1011a0 2024-04-23T10:35:38.829 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.828+0000 7f5352ffd640 1 -- 172.21.15.97:0/1387165747 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5358004c30 con 0x7f535c1011a0 2024-04-23T10:35:38.829 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.828+0000 7f536414a640 1 -- 172.21.15.97:0/1387165747 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f535c1a7310 con 0x7f535c1011a0 2024-04-23T10:35:38.834 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.830+0000 7f5352ffd640 1 -- 172.21.15.97:0/1387165747 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5358004e10 con 0x7f535c1011a0 2024-04-23T10:35:38.834 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.830+0000 7f5366e5c740 1 Processor -- start 2024-04-23T10:35:38.834 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.831+0000 7f5366e5c740 1 -- start start 2024-04-23T10:35:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.831+0000 7f5366e5c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f9d4a70620 0x55f9d4a70a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.831+0000 7f5352ffd640 1 --2- 172.21.15.97:0/1387165747 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f53480a3dc0 0x7f53480a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.831+0000 7f5366e5c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9d4a70f40 0x55f9d4a7b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.831+0000 7f5366e5c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f9d4a7b800 0x55f9d4a7dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.831+0000 7f5366e5c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f9d49d5970 con 0x55f9d4a70f40 2024-04-23T10:35:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.831+0000 7f5366e5c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f9d49724a0 con 0x55f9d4a7b800 2024-04-23T10:35:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.831+0000 7f5366e5c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f9d49d1930 con 0x55f9d4a70620 2024-04-23T10:35:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.831+0000 7f5361ee6640 1 --2- 172.21.15.97:0/1387165747 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f53480a3dc0 0x7f53480a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.831+0000 7f53616e5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9d4a70f40 0x55f9d4a7b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.831+0000 7f53616e5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9d4a70f40 0x55f9d4a7b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47242/0 (socket says 172.21.15.97:47242) 2024-04-23T10:35:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.831+0000 7f53616e5640 1 -- 172.21.15.97:0/2477265191 learned_addr learned my addr 172.21.15.97:0/2477265191 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:35:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.831+0000 7f53626e7640 1 --2- 172.21.15.97:0/2477265191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f9d4a7b800 0x55f9d4a7dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.831+0000 7f5352ffd640 1 -- 172.21.15.97:0/1387165747 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8484+0+0 (secure 0 0 0) 0x7f53580d02b0 con 0x7f535c1011a0 2024-04-23T10:35:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.831+0000 7f53626e7640 1 --2- 172.21.15.97:0/2477265191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f9d4a70620 0x55f9d4a70a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.831+0000 7f53616e5640 1 -- 172.21.15.97:0/2477265191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f9d4a70620 msgr2=0x55f9d4a70a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:35:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.831+0000 7f53616e5640 1 --2- 172.21.15.97:0/2477265191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f9d4a70620 0x55f9d4a70a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.831+0000 7f53616e5640 1 -- 172.21.15.97:0/2477265191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f9d4a7b800 msgr2=0x55f9d4a7dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:35:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.831+0000 7f53616e5640 1 --2- 172.21.15.97:0/2477265191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f9d4a7b800 0x55f9d4a7dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.831+0000 7f53616e5640 1 -- 172.21.15.97:0/2477265191 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9d46e8210 con 0x55f9d4a70f40 2024-04-23T10:35:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.832+0000 7f5361ee6640 1 --2- 172.21.15.97:0/1387165747 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f53480a3dc0 0x7f53480a6280 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5358098c40 tx=0x7f5358064040 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:35:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.832+0000 7f53616e5640 1 --2- 172.21.15.97:0/2477265191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9d4a70f40 0x55f9d4a7b2c0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f534c02a920 tx=0x7f534c032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abdf25ec5eebb95a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:35:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.832+0000 7f5347fff640 1 -- 172.21.15.97:0/2477265191 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534c03f070 con 0x55f9d4a70f40 2024-04-23T10:35:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.832+0000 7f5347fff640 1 -- 172.21.15.97:0/2477265191 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f534c032e50 con 0x55f9d4a70f40 2024-04-23T10:35:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.832+0000 7f5347fff640 1 -- 172.21.15.97:0/2477265191 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534c035070 con 0x55f9d4a70f40 2024-04-23T10:35:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.832+0000 7f5366e5c740 1 -- 172.21.15.97:0/2477265191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9d4a70f40 msgr2=0x55f9d4a7b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.832+0000 7f5366e5c740 1 --2- 172.21.15.97:0/2477265191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9d4a70f40 0x55f9d4a7b2c0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f534c02a920 tx=0x7f534c032ce0 comp rx=0 tx=0).stop 2024-04-23T10:35:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.833+0000 7f5366e5c740 1 -- 172.21.15.97:0/2477265191 shutdown_connections 2024-04-23T10:35:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.833+0000 7f5366e5c740 1 --2- 172.21.15.97:0/2477265191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f9d4a7b800 0x55f9d4a7dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.833+0000 7f5366e5c740 1 --2- 172.21.15.97:0/2477265191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9d4a70f40 0x55f9d4a7b2c0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.833+0000 7f5366e5c740 1 --2- 172.21.15.97:0/2477265191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f9d4a70620 0x55f9d4a70a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.833+0000 7f5366e5c740 1 -- 172.21.15.97:0/2477265191 >> 172.21.15.97:0/2477265191 conn(0x55f9d4a6cb30 msgr2=0x55f9d4a6cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:35:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.833+0000 7f5366e5c740 1 -- 172.21.15.97:0/2477265191 shutdown_connections 2024-04-23T10:35:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.833+0000 7f5366e5c740 1 -- 172.21.15.97:0/2477265191 wait complete. 2024-04-23T10:35:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.833+0000 7f5366e5c740 1 -- start start 2024-04-23T10:35:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.833+0000 7f5366e5c740 1 Processor -- start 2024-04-23T10:35:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.833+0000 7f5366e5c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9d4a70620 0x55f9d4a7b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.833+0000 7f5366e5c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f9d4a70f40 0x55f9d4a78490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.833+0000 7f5366e5c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f9d4a7b800 0x55f9d4a78a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.834+0000 7f5366e5c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f9d4a7e3d0 con 0x55f9d4a70620 2024-04-23T10:35:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.834+0000 7f5366e5c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f9d49700b0 con 0x55f9d4a70f40 2024-04-23T10:35:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.834+0000 7f5366e5c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f9d4a7e550 con 0x55f9d4a7b800 2024-04-23T10:35:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.834+0000 7f5366e5c740 10 client.0 _ll_register_callbacks cb 0x55f9d495dd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:35:38.838 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.834+0000 7f53626e7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9d4a70620 0x55f9d4a7b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:38.838 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.834+0000 7f53626e7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9d4a70620 0x55f9d4a7b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47250/0 (socket says 172.21.15.97:47250) 2024-04-23T10:35:38.838 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.834+0000 7f53626e7640 1 -- 172.21.15.97:0/3261723307 learned_addr learned my addr 172.21.15.97:0/3261723307 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:35:38.838 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.834+0000 7f53616e5640 1 --2- 172.21.15.97:0/3261723307 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f9d4a70f40 0x55f9d4a78490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:38.838 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.834+0000 7f53626e7640 1 -- 172.21.15.97:0/3261723307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f9d4a7b800 msgr2=0x55f9d4a78a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:35:38.838 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.834+0000 7f53626e7640 1 --2- 172.21.15.97:0/3261723307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f9d4a7b800 0x55f9d4a78a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:38.838 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.834+0000 7f53626e7640 1 -- 172.21.15.97:0/3261723307 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f9d4a70f40 msgr2=0x55f9d4a78490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:35:38.838 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.834+0000 7f53626e7640 1 --2- 172.21.15.97:0/3261723307 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f9d4a70f40 0x55f9d4a78490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:38.838 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.834+0000 7f53626e7640 1 -- 172.21.15.97:0/3261723307 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9d47b4620 con 0x55f9d4a70620 2024-04-23T10:35:38.838 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.834+0000 7f53626e7640 1 --2- 172.21.15.97:0/3261723307 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9d4a70620 0x55f9d4a7b1a0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f5354014eb0 tx=0x7f5354011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=171a6c3c33069857 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:35:38.838 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.834+0000 7f53457fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:35:38.838 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.834+0000 7f53457fa640 1 -- 172.21.15.97:0/3261723307 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5354003b70 con 0x55f9d4a70620 2024-04-23T10:35:38.838 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.834+0000 7f53457fa640 1 -- 172.21.15.97:0/3261723307 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8484+0+0 (secure 0 0 0) 0x7f5354003d10 con 0x55f9d4a70620 2024-04-23T10:35:38.838 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.834+0000 7f5366e5c740 1 -- 172.21.15.97:0/3261723307 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f534c032e50 con 0x55f9d4a70620 2024-04-23T10:35:38.839 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.834+0000 7f53457fa640 1 -- 172.21.15.97:0/3261723307 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5354025050 con 0x55f9d4a70620 2024-04-23T10:35:38.839 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.834+0000 7f5366e5c740 10 client.16935 fetch_fsmap learned FSMap version 26 2024-04-23T10:35:38.839 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.834+0000 7f5366e5c740 1 -- 172.21.15.97:0/3261723307 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f9d4a8a3a0 con 0x55f9d4a70620 2024-04-23T10:35:38.839 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.835+0000 7f53457fa640 1 -- 172.21.15.97:0/3261723307 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f534c032e50 con 0x55f9d4a70620 2024-04-23T10:35:38.839 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.835+0000 7f53457fa640 1 -- 172.21.15.97:0/3261723307 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f53540124c0 con 0x55f9d4a70620 2024-04-23T10:35:38.839 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.835+0000 7f5366e5c740 10 client.16935 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:35:38.839 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.835+0000 7f5366e5c740 10 client.16935 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:35:38.839 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.835+0000 7f5366e5c740 20 client.16935 populate_metadata read hostname 'smithi097' 2024-04-23T10:35:38.839 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.835+0000 7f5366e5c740 1 --2- 172.21.15.97:0/3261723307 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55f9d4a8b810 0x55f9d4a8dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:38.839 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.835+0000 7f5366e5c740 4 client.16935 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:35:38.839 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.835+0000 7f5366e5c740 1 -- 172.21.15.97:0/3261723307 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9d49e3390 con 0x55f9d4a8b810 2024-04-23T10:35:38.839 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.835+0000 7f53616e5640 1 --2- 172.21.15.97:0/3261723307 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55f9d4a8b810 0x55f9d4a8dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:38.839 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.835+0000 7f53457fa640 10 client.16935 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:35:38.839 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.835+0000 7f53616e5640 1 --2- 172.21.15.97:0/3261723307 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55f9d4a8b810 0x55f9d4a8dc00 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23e9756a5891d6c6 server_cookie=78db8720459c751c in_seq=0 out_seq=0 2024-04-23T10:35:38.840 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.836+0000 7f53457fa640 1 -- 172.21.15.97:0/3261723307 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f9d49e3390 con 0x55f9d4a8b810 2024-04-23T10:35:38.841 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.836+0000 7f53457fa640 10 client.16935 handle_command_reply: tid=0 2024-04-23T10:35:38.841 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.836+0000 7f5366e5c740 1 client.16935 shutdown 2024-04-23T10:35:38.841 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.836+0000 7f5366e5c740 1 -- 172.21.15.97:0/3261723307 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9d4a70620 msgr2=0x55f9d4a7b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:38.841 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.836+0000 7f5366e5c740 1 --2- 172.21.15.97:0/3261723307 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9d4a70620 0x55f9d4a7b1a0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f5354014eb0 tx=0x7f5354011810 comp rx=0 tx=0).stop 2024-04-23T10:35:38.841 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.836+0000 7f5366e5c740 1 -- 172.21.15.97:0/3261723307 shutdown_connections 2024-04-23T10:35:38.841 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.836+0000 7f5366e5c740 1 --2- 172.21.15.97:0/3261723307 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55f9d4a8b810 0x55f9d4a8dc00 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:38.841 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.836+0000 7f5366e5c740 1 --2- 172.21.15.97:0/3261723307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f9d4a7b800 0x55f9d4a78a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:38.841 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.836+0000 7f5366e5c740 1 --2- 172.21.15.97:0/3261723307 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f9d4a70f40 0x55f9d4a78490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:38.841 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.836+0000 7f53457fa640 0 client.16935 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:35:38.841 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.836+0000 7f5366e5c740 1 --2- 172.21.15.97:0/3261723307 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9d4a70620 0x55f9d4a7b1a0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:38.841 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.836+0000 7f5366e5c740 1 -- 172.21.15.97:0/3261723307 >> 172.21.15.97:0/3261723307 conn(0x55f9d4a6cb30 msgr2=0x55f9d4a6bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:35:38.841 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.836+0000 7f5366e5c740 1 -- 172.21.15.97:0/3261723307 shutdown_connections 2024-04-23T10:35:38.841 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.837+0000 7f5366e5c740 1 -- 172.21.15.97:0/3261723307 wait complete. 2024-04-23T10:35:38.841 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.837+0000 7f5366e5c740 20 client.16935 trim_cache size 0 max 16384 2024-04-23T10:35:38.852 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.851+0000 7f5366e5c740 1 Processor -- start 2024-04-23T10:35:38.852 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.851+0000 7f5366e5c740 1 -- start start 2024-04-23T10:35:38.852 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.851+0000 7f5366e5c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9d4a70620 0x55f9d4a87290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:38.852 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.851+0000 7f5366e5c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f9d4a70f40 0x55f9d4a780e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:38.852 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.851+0000 7f5366e5c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f9d4a7b800 0x55f9d4a7acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:38.852 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.851+0000 7f5366e5c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f535c1afb10 con 0x55f9d4a70620 2024-04-23T10:35:38.852 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.851+0000 7f5366e5c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f535c1b1650 con 0x55f9d4a70f40 2024-04-23T10:35:38.853 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.851+0000 7f5366e5c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f535c1b01e0 con 0x55f9d4a7b800 2024-04-23T10:35:38.853 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.852+0000 7f53626e7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9d4a70620 0x55f9d4a87290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:38.853 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.852+0000 7f53626e7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9d4a70620 0x55f9d4a87290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47260/0 (socket says 172.21.15.97:47260) 2024-04-23T10:35:38.853 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.852+0000 7f53626e7640 1 -- 172.21.15.97:0/2163690754 learned_addr learned my addr 172.21.15.97:0/2163690754 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:35:38.853 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.852+0000 7f53626e7640 1 --2- 172.21.15.97:0/2163690754 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f9d4a7b800 0x55f9d4a7acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:38.853 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.852+0000 7f53626e7640 1 -- 172.21.15.97:0/2163690754 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f9d4a7b800 msgr2=0x55f9d4a7acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:35:38.853 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.852+0000 7f53626e7640 1 --2- 172.21.15.97:0/2163690754 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f9d4a7b800 0x55f9d4a7acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:38.853 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.852+0000 7f53626e7640 1 -- 172.21.15.97:0/2163690754 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f9d4a70f40 msgr2=0x55f9d4a780e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:35:38.853 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.852+0000 7f53616e5640 1 --2- 172.21.15.97:0/2163690754 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f9d4a70f40 0x55f9d4a780e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:38.853 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.852+0000 7f53626e7640 1 --2- 172.21.15.97:0/2163690754 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f9d4a70f40 0x55f9d4a780e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:38.853 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.852+0000 7f53626e7640 1 -- 172.21.15.97:0/2163690754 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9d4a78da0 con 0x55f9d4a70620 2024-04-23T10:35:38.853 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.852+0000 7f53626e7640 1 --2- 172.21.15.97:0/2163690754 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9d4a70620 0x55f9d4a87290 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x55f9d4a1b560 tx=0x7f5354007520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=474d860d4802b67e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:35:38.853 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.852+0000 7f53477fe640 1 -- 172.21.15.97:0/2163690754 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535402f070 con 0x55f9d4a70620 2024-04-23T10:35:38.854 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.852+0000 7f53477fe640 1 -- 172.21.15.97:0/2163690754 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5354012650 con 0x55f9d4a70620 2024-04-23T10:35:38.854 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.852+0000 7f53477fe640 1 -- 172.21.15.97:0/2163690754 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5354029c30 con 0x55f9d4a70620 2024-04-23T10:35:38.854 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.853+0000 7f5366e5c740 1 -- 172.21.15.97:0/2163690754 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9d4a70620 msgr2=0x55f9d4a87290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:38.854 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.853+0000 7f5366e5c740 1 --2- 172.21.15.97:0/2163690754 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9d4a70620 0x55f9d4a87290 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x55f9d4a1b560 tx=0x7f5354007520 comp rx=0 tx=0).stop 2024-04-23T10:35:38.854 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.853+0000 7f5366e5c740 1 -- 172.21.15.97:0/2163690754 shutdown_connections 2024-04-23T10:35:38.854 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.853+0000 7f5366e5c740 1 --2- 172.21.15.97:0/2163690754 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f9d4a7b800 0x55f9d4a7acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:38.854 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.853+0000 7f5366e5c740 1 --2- 172.21.15.97:0/2163690754 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f9d4a70f40 0x55f9d4a780e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:38.854 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.853+0000 7f5366e5c740 1 --2- 172.21.15.97:0/2163690754 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9d4a70620 0x55f9d4a87290 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:38.854 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.853+0000 7f5366e5c740 1 -- 172.21.15.97:0/2163690754 >> 172.21.15.97:0/2163690754 conn(0x55f9d4a6cb30 msgr2=0x55f9d4a779d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:35:38.854 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.853+0000 7f5366e5c740 1 -- 172.21.15.97:0/2163690754 shutdown_connections 2024-04-23T10:35:38.854 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.853+0000 7f5366e5c740 1 -- 172.21.15.97:0/2163690754 wait complete. 2024-04-23T10:35:38.854 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.853+0000 7f5366e5c740 1 -- start start 2024-04-23T10:35:38.854 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.853+0000 7f5366e5c740 1 Processor -- start 2024-04-23T10:35:38.857 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.854+0000 7f5366e5c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f9d4a70620 0x55f9d4a7f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:38.858 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.854+0000 7f5366e5c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f9d4a70f40 0x55f9d4a7f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:38.858 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.854+0000 7f5366e5c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9d4a7b800 0x55f9d4a7fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:38.858 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.854+0000 7f5366e5c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f535c1b0f80 con 0x55f9d4a7b800 2024-04-23T10:35:38.858 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.854+0000 7f5366e5c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f535c1b08b0 con 0x55f9d4a70620 2024-04-23T10:35:38.858 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.854+0000 7f5366e5c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f535c1adb10 con 0x55f9d4a70f40 2024-04-23T10:35:38.858 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.854+0000 7f5366e5c740 10 client.0 _ll_register_callbacks cb 0x7f53540147e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:35:38.858 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.854+0000 7f53626e7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9d4a7b800 0x55f9d4a7fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:38.858 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.854+0000 7f53626e7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9d4a7b800 0x55f9d4a7fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:47264/0 (socket says 172.21.15.97:47264) 2024-04-23T10:35:38.858 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.854+0000 7f53626e7640 1 -- 172.21.15.97:0/3855535746 learned_addr learned my addr 172.21.15.97:0/3855535746 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:35:38.858 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.854+0000 7f53626e7640 1 --2- 172.21.15.97:0/3855535746 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f9d4a70620 0x55f9d4a7f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:38.858 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.854+0000 7f53616e5640 1 --2- 172.21.15.97:0/3855535746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f9d4a70f40 0x55f9d4a7f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:38.858 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.854+0000 7f53626e7640 1 -- 172.21.15.97:0/3855535746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f9d4a70f40 msgr2=0x55f9d4a7f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:35:38.858 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.854+0000 7f53626e7640 1 --2- 172.21.15.97:0/3855535746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f9d4a70f40 0x55f9d4a7f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:38.858 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.854+0000 7f53626e7640 1 -- 172.21.15.97:0/3855535746 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f9d4a70620 msgr2=0x55f9d4a7f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:35:38.859 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.854+0000 7f53626e7640 1 --2- 172.21.15.97:0/3855535746 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f9d4a70620 0x55f9d4a7f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:38.859 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.854+0000 7f53626e7640 1 -- 172.21.15.97:0/3855535746 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9d4af0d80 con 0x55f9d4a7b800 2024-04-23T10:35:38.859 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.854+0000 7f53616e5640 1 --2- 172.21.15.97:0/3855535746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f9d4a70f40 0x55f9d4a7f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:35:38.859 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.854+0000 7f53626e7640 1 --2- 172.21.15.97:0/3855535746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9d4a7b800 0x55f9d4a7fba0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f5354014eb0 tx=0x7f5354012e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d42c48c9fe1576d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:35:38.859 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.854+0000 7f53457fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:35:38.859 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.854+0000 7f53457fa640 1 -- 172.21.15.97:0/3855535746 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535402f040 con 0x55f9d4a7b800 2024-04-23T10:35:38.859 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.854+0000 7f53457fa640 1 -- 172.21.15.97:0/3855535746 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8484+0+0 (secure 0 0 0) 0x7f53540299b0 con 0x55f9d4a7b800 2024-04-23T10:35:38.859 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.854+0000 7f5366e5c740 1 -- 172.21.15.97:0/3855535746 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5354012650 con 0x55f9d4a7b800 2024-04-23T10:35:38.859 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.855+0000 7f53457fa640 1 -- 172.21.15.97:0/3855535746 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5354029bb0 con 0x55f9d4a7b800 2024-04-23T10:35:38.859 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.855+0000 7f5366e5c740 10 client.16941 fetch_fsmap learned FSMap version 26 2024-04-23T10:35:38.859 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.855+0000 7f5366e5c740 1 -- 172.21.15.97:0/3855535746 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f9d4af1fd0 con 0x55f9d4a7b800 2024-04-23T10:35:38.859 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.855+0000 7f53457fa640 1 -- 172.21.15.97:0/3855535746 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5354012650 con 0x55f9d4a7b800 2024-04-23T10:35:38.859 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.855+0000 7f53457fa640 1 -- 172.21.15.97:0/3855535746 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5354033950 con 0x55f9d4a7b800 2024-04-23T10:35:38.859 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.855+0000 7f5366e5c740 10 client.16941 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:35:38.860 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.855+0000 7f5366e5c740 10 client.16941 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:35:38.860 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.855+0000 7f5366e5c740 20 client.16941 populate_metadata read hostname 'smithi097' 2024-04-23T10:35:38.860 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.855+0000 7f5366e5c740 1 --2- 172.21.15.97:0/3855535746 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55f9d4af2600 0x55f9d4af4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:38.860 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.855+0000 7f5366e5c740 4 client.16941 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:35:38.860 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.855+0000 7f5366e5c740 1 -- 172.21.15.97:0/3855535746 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f9d49dd200 con 0x55f9d4af2600 2024-04-23T10:35:38.860 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.855+0000 7f53616e5640 1 --2- 172.21.15.97:0/3855535746 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55f9d4af2600 0x55f9d4af4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:38.860 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.856+0000 7f53457fa640 10 client.16941 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:35:38.860 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.856+0000 7f53616e5640 1 --2- 172.21.15.97:0/3855535746 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55f9d4af2600 0x55f9d4af4a20 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76680c04783b8880 server_cookie=772b35dc0f0ee813 in_seq=0 out_seq=0 2024-04-23T10:35:38.870 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.871+0000 7f53457fa640 1 -- 172.21.15.97:0/3855535746 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f9d49dd200 con 0x55f9d4af2600 2024-04-23T10:35:38.870 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.871+0000 7f53457fa640 10 client.16941 handle_command_reply: tid=0 2024-04-23T10:35:38.870 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.871+0000 7f5366e5c740 1 client.16941 shutdown 2024-04-23T10:35:38.870 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.871+0000 7f5366e5c740 1 -- 172.21.15.97:0/3855535746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9d4a7b800 msgr2=0x55f9d4a7fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:38.870 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.871+0000 7f5366e5c740 1 --2- 172.21.15.97:0/3855535746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9d4a7b800 0x55f9d4a7fba0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f5354014eb0 tx=0x7f5354012e10 comp rx=0 tx=0).stop 2024-04-23T10:35:38.870 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.871+0000 7f5366e5c740 1 -- 172.21.15.97:0/3855535746 shutdown_connections 2024-04-23T10:35:38.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.871+0000 7f5366e5c740 1 --2- 172.21.15.97:0/3855535746 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55f9d4af2600 0x55f9d4af4a20 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:38.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.871+0000 7f5366e5c740 1 --2- 172.21.15.97:0/3855535746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f9d4a7b800 0x55f9d4a7fba0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:38.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.871+0000 7f5366e5c740 1 --2- 172.21.15.97:0/3855535746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f9d4a70f40 0x55f9d4a7f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:38.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.871+0000 7f53457fa640 0 client.16941 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:35:38.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.871+0000 7f5366e5c740 1 --2- 172.21.15.97:0/3855535746 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f9d4a70620 0x55f9d4a7f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:38.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.871+0000 7f5366e5c740 1 -- 172.21.15.97:0/3855535746 >> 172.21.15.97:0/3855535746 conn(0x55f9d4a6cb30 msgr2=0x55f9d4a78260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:35:38.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.871+0000 7f5366e5c740 1 -- 172.21.15.97:0/3855535746 shutdown_connections 2024-04-23T10:35:38.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.871+0000 7f5366e5c740 1 -- 172.21.15.97:0/3855535746 wait complete. 2024-04-23T10:35:38.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.871+0000 7f5366e5c740 20 client.16941 trim_cache size 0 max 16384 2024-04-23T10:35:38.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:35:38.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:35:38.872 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:35:38.872 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:35:38.872 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.872+0000 7f536414a640 1 -- 172.21.15.97:0/1387165747 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f53480a3dc0 msgr2=0x7f53480a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:38.872 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.872+0000 7f536414a640 1 --2- 172.21.15.97:0/1387165747 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f53480a3dc0 0x7f53480a6280 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5358098c40 tx=0x7f5358064040 comp rx=0 tx=0).stop 2024-04-23T10:35:38.872 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.872+0000 7f536414a640 1 -- 172.21.15.97:0/1387165747 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f535c1011a0 msgr2=0x7f535c1a0450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:38.873 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.872+0000 7f536414a640 1 --2- 172.21.15.97:0/1387165747 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f535c1011a0 0x7f535c1a0450 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f535800d770 tx=0x7f535800dc30 comp rx=0 tx=0).stop 2024-04-23T10:35:38.873 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.872+0000 7f536414a640 1 -- 172.21.15.97:0/1387165747 shutdown_connections 2024-04-23T10:35:38.873 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.872+0000 7f536414a640 1 --2- 172.21.15.97:0/1387165747 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f53480a3dc0 0x7f53480a6280 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:38.873 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.872+0000 7f536414a640 1 --2- 172.21.15.97:0/1387165747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f535c106fb0 0x7f535c19a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:38.873 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.872+0000 7f536414a640 1 --2- 172.21.15.97:0/1387165747 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f535c101b60 0x7f535c1a0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:38.873 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.872+0000 7f536414a640 1 --2- 172.21.15.97:0/1387165747 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f535c1011a0 0x7f535c1a0450 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:38.873 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.872+0000 7f536414a640 1 -- 172.21.15.97:0/1387165747 >> 172.21.15.97:0/1387165747 conn(0x7f535c076c50 msgr2=0x7f535c06a780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:35:38.873 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.872+0000 7f536414a640 1 -- 172.21.15.97:0/1387165747 shutdown_connections 2024-04-23T10:35:38.873 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:38.872+0000 7f536414a640 1 -- 172.21.15.97:0/1387165747 wait complete. 2024-04-23T10:35:40.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:35:40 smithi196 ceph-mon[32919]: pgmap v293: 97 pgs: 97 active+clean; 5.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 11 op/s 2024-04-23T10:35:40.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:35:40 smithi097 ceph-mon[29197]: pgmap v293: 97 pgs: 97 active+clean; 5.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 11 op/s 2024-04-23T10:35:40.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:35:40 smithi183 ceph-mon[32317]: pgmap v293: 97 pgs: 97 active+clean; 5.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 11 op/s 2024-04-23T10:35:42.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:35:42 smithi196 ceph-mon[32919]: pgmap v294: 97 pgs: 97 active+clean; 5.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 11 op/s 2024-04-23T10:35:42.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:35:42 smithi097 ceph-mon[29197]: pgmap v294: 97 pgs: 97 active+clean; 5.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 11 op/s 2024-04-23T10:35:42.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:35:42 smithi183 ceph-mon[32317]: pgmap v294: 97 pgs: 97 active+clean; 5.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 23 KiB/s wr, 11 op/s 2024-04-23T10:35:44.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:35:44 smithi097 ceph-mon[29197]: pgmap v295: 97 pgs: 97 active+clean; 5.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 47 KiB/s wr, 21 op/s 2024-04-23T10:35:44.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:35:44 smithi183 ceph-mon[32317]: pgmap v295: 97 pgs: 97 active+clean; 5.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 47 KiB/s wr, 21 op/s 2024-04-23T10:35:44.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:35:44 smithi196 ceph-mon[32919]: pgmap v295: 97 pgs: 97 active+clean; 5.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 47 KiB/s wr, 21 op/s 2024-04-23T10:35:46.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:35:46 smithi097 ceph-mon[29197]: pgmap v296: 97 pgs: 97 active+clean; 5.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 40 KiB/s wr, 16 op/s 2024-04-23T10:35:46.649 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:35:46 smithi183 ceph-mon[32317]: pgmap v296: 97 pgs: 97 active+clean; 5.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 40 KiB/s wr, 16 op/s 2024-04-23T10:35:46.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:35:46 smithi196 ceph-mon[32919]: pgmap v296: 97 pgs: 97 active+clean; 5.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 40 KiB/s wr, 16 op/s 2024-04-23T10:35:48.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:35:48 smithi097 ceph-mon[29197]: pgmap v297: 97 pgs: 97 active+clean; 5.6 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 59 KiB/s wr, 23 op/s 2024-04-23T10:35:48.644 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:35:48 smithi196 ceph-mon[32919]: pgmap v297: 97 pgs: 97 active+clean; 5.6 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 59 KiB/s wr, 23 op/s 2024-04-23T10:35:48.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:35:48 smithi183 ceph-mon[32317]: pgmap v297: 97 pgs: 97 active+clean; 5.6 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 59 KiB/s wr, 23 op/s 2024-04-23T10:35:48.900 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 17.0 2024-04-23T10:35:48.900 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T10:35:49.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.012+0000 7f97cda91640 1 -- 172.21.15.97:0/3959612404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97c810c580 msgr2=0x7f97c810e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:49.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.012+0000 7f97cda91640 1 --2- 172.21.15.97:0/3959612404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97c810c580 0x7f97c810e9b0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f97c0009ca0 tx=0x7f97c002a950 comp rx=0 tx=0).stop 2024-04-23T10:35:49.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.012+0000 7f97cda91640 1 -- 172.21.15.97:0/3959612404 shutdown_connections 2024-04-23T10:35:49.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.012+0000 7f97cda91640 1 --2- 172.21.15.97:0/3959612404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97c810c580 0x7f97c810e9b0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:49.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.012+0000 7f97cda91640 1 --2- 172.21.15.97:0/3959612404 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f97c8103ad0 0x7f97c810bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:49.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.012+0000 7f97cda91640 1 --2- 172.21.15.97:0/3959612404 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97c81031b0 0x7f97c8103590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:49.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.012+0000 7f97cda91640 1 -- 172.21.15.97:0/3959612404 >> 172.21.15.97:0/3959612404 conn(0x7f97c80771d0 msgr2=0x7f97c80775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:35:49.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.012+0000 7f97cda91640 1 -- 172.21.15.97:0/3959612404 shutdown_connections 2024-04-23T10:35:49.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.012+0000 7f97cda91640 1 -- 172.21.15.97:0/3959612404 wait complete. 2024-04-23T10:35:49.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.013+0000 7f97cda91640 1 Processor -- start 2024-04-23T10:35:49.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.013+0000 7f97cda91640 1 -- start start 2024-04-23T10:35:49.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.013+0000 7f97cda91640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97c81031b0 0x7f97c81986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:49.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.013+0000 7f97cda91640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f97c8103ad0 0x7f97c8198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:49.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.013+0000 7f97cda91640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97c810c580 0x7f97c819cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:49.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.013+0000 7f97cda91640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f97c8112180 con 0x7f97c81031b0 2024-04-23T10:35:49.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.013+0000 7f97cda91640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f97c8112000 con 0x7f97c8103ad0 2024-04-23T10:35:49.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.013+0000 7f97cda91640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f97c8112300 con 0x7f97c810c580 2024-04-23T10:35:49.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.013+0000 7f97c77fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97c81031b0 0x7f97c81986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:49.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.013+0000 7f97c77fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97c81031b0 0x7f97c81986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52402/0 (socket says 172.21.15.97:52402) 2024-04-23T10:35:49.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.013+0000 7f97c77fe640 1 -- 172.21.15.97:0/434188820 learned_addr learned my addr 172.21.15.97:0/434188820 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:35:49.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.013+0000 7f97c6ffd640 1 --2- 172.21.15.97:0/434188820 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f97c8103ad0 0x7f97c8198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:49.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.013+0000 7f97c7fff640 1 --2- 172.21.15.97:0/434188820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97c810c580 0x7f97c819cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:49.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.013+0000 7f97c77fe640 1 -- 172.21.15.97:0/434188820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97c810c580 msgr2=0x7f97c819cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:49.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.013+0000 7f97c77fe640 1 --2- 172.21.15.97:0/434188820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97c810c580 0x7f97c819cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:49.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.013+0000 7f97c77fe640 1 -- 172.21.15.97:0/434188820 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f97c8103ad0 msgr2=0x7f97c8198c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:49.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.013+0000 7f97c77fe640 1 --2- 172.21.15.97:0/434188820 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f97c8103ad0 0x7f97c8198c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:49.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.013+0000 7f97c77fe640 1 -- 172.21.15.97:0/434188820 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97c819d660 con 0x7f97c81031b0 2024-04-23T10:35:49.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.013+0000 7f97c7fff640 1 --2- 172.21.15.97:0/434188820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97c810c580 0x7f97c819cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:35:49.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.014+0000 7f97c77fe640 1 --2- 172.21.15.97:0/434188820 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97c81031b0 0x7f97c81986c0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f97b800c920 tx=0x7f97b800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:35:49.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.014+0000 7f97c4ff9640 1 -- 172.21.15.97:0/434188820 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b8007c70 con 0x7f97c81031b0 2024-04-23T10:35:49.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.014+0000 7f97c4ff9640 1 -- 172.21.15.97:0/434188820 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97b8007e10 con 0x7f97c81031b0 2024-04-23T10:35:49.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.014+0000 7f97cda91640 1 -- 172.21.15.97:0/434188820 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97c819d950 con 0x7f97c81031b0 2024-04-23T10:35:49.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.014+0000 7f97c4ff9640 1 -- 172.21.15.97:0/434188820 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b8018820 con 0x7f97c81031b0 2024-04-23T10:35:49.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.014+0000 7f97cda91640 1 -- 172.21.15.97:0/434188820 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97c81a51e0 con 0x7f97c81031b0 2024-04-23T10:35:49.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.015+0000 7f97cda91640 1 -- 172.21.15.97:0/434188820 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97c806add0 con 0x7f97c81031b0 2024-04-23T10:35:49.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.016+0000 7f97c4ff9640 1 -- 172.21.15.97:0/434188820 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f97b800f450 con 0x7f97c81031b0 2024-04-23T10:35:49.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.017+0000 7f97c4ff9640 1 --2- 172.21.15.97:0/434188820 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f97b00a3d20 0x7f97b00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:49.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.017+0000 7f97c6ffd640 1 --2- 172.21.15.97:0/434188820 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f97b00a3d20 0x7f97b00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:49.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.017+0000 7f97c4ff9640 1 -- 172.21.15.97:0/434188820 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8484+0+0 (secure 0 0 0) 0x7f97b80d0bb0 con 0x7f97c81031b0 2024-04-23T10:35:49.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.017+0000 7f97c6ffd640 1 --2- 172.21.15.97:0/434188820 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f97b00a3d20 0x7f97b00a61e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f97c8100650 tx=0x7f97bc00b390 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:35:49.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.020+0000 7f97c4ff9640 1 -- 172.21.15.97:0/434188820 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f97b8014030 con 0x7f97c81031b0 2024-04-23T10:35:49.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.236+0000 7f97cda91640 1 -- 172.21.15.97:0/434188820 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f97c80691c0 con 0x7f97c81031b0 2024-04-23T10:35:49.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.237+0000 7f97c4ff9640 1 -- 172.21.15.97:0/434188820 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11931 (secure 0 0 0) 0x7f97b809d300 con 0x7f97c81031b0 2024-04-23T10:35:49.236 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:35:49.236 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3946878762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3946878762},{"type":"v1","addr":"172.21.15.97:6835","nonce":3946878762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/209800861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":209800861},{"type":"v1","addr":"172.21.15.97:6837","nonce":209800861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T10:24:06.910134+0000","modified":"2024-04-23T10:26:09.820985+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6839/418013575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":418013575},{"type":"v1","addr":"172.21.15.97:6839","nonce":418013575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6841/747242680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":747242680},{"type":"v1","addr":"172.21.15.97:6841","nonce":747242680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6833/4292847121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4292847121},{"type":"v1","addr":"172.21.15.183:6833","nonce":4292847121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6833/3885816565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3885816565},{"type":"v1","addr":"172.21.15.196:6833","nonce":3885816565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1897093105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1897093105},{"type":"v1","addr":"172.21.15.183:6835","nonce":1897093105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1171477563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1171477563},{"type":"v1","addr":"172.21.15.196:6835","nonce":1171477563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1898341911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1898341911},{"type":"v1","addr":"172.21.15.183:6837","nonce":1898341911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.196:6837/3094946502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3094946502},{"type":"v1","addr":"172.21.15.196:6837","nonce":3094946502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.196:6839/83843556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":83843556},{"type":"v1","addr":"172.21.15.196:6839","nonce":83843556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6839/1763841130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1763841130},{"type":"v1","addr":"172.21.15.183:6839","nonce":1763841130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24487,24403,24398,24469,24488]},"id":1}]} 2024-04-23T10:35:49.236 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 26 2024-04-23T10:35:49.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.239+0000 7f97cda91640 1 -- 172.21.15.97:0/434188820 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f97b00a3d20 msgr2=0x7f97b00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:49.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.239+0000 7f97cda91640 1 --2- 172.21.15.97:0/434188820 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f97b00a3d20 0x7f97b00a61e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f97c8100650 tx=0x7f97bc00b390 comp rx=0 tx=0).stop 2024-04-23T10:35:49.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.239+0000 7f97cda91640 1 -- 172.21.15.97:0/434188820 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97c81031b0 msgr2=0x7f97c81986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:49.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.239+0000 7f97cda91640 1 --2- 172.21.15.97:0/434188820 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97c81031b0 0x7f97c81986c0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f97b800c920 tx=0x7f97b800cde0 comp rx=0 tx=0).stop 2024-04-23T10:35:49.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.239+0000 7f97cda91640 1 -- 172.21.15.97:0/434188820 shutdown_connections 2024-04-23T10:35:49.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.239+0000 7f97cda91640 1 --2- 172.21.15.97:0/434188820 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f97b00a3d20 0x7f97b00a61e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:49.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.239+0000 7f97cda91640 1 --2- 172.21.15.97:0/434188820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97c810c580 0x7f97c819cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:49.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.239+0000 7f97cda91640 1 --2- 172.21.15.97:0/434188820 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f97c8103ad0 0x7f97c8198c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:49.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.239+0000 7f97cda91640 1 --2- 172.21.15.97:0/434188820 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97c81031b0 0x7f97c81986c0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:49.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.239+0000 7f97cda91640 1 -- 172.21.15.97:0/434188820 >> 172.21.15.97:0/434188820 conn(0x7f97c80771d0 msgr2=0x7f97c80fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:35:49.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.239+0000 7f97cda91640 1 -- 172.21.15.97:0/434188820 shutdown_connections 2024-04-23T10:35:49.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.239+0000 7f97cda91640 1 -- 172.21.15.97:0/434188820 wait complete. 2024-04-23T10:35:49.249 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db / --timeout 60 --expiration 77.0 --await-for 42.0' 2024-04-23T10:35:49.249 DEBUG:teuthology.orchestra.run.smithi097:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24488 quiesce db / --timeout 60 --expiration 77.0 --await-for 42.0 2024-04-23T10:35:49.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.360+0000 7fd4bdfe5640 1 -- 172.21.15.97:0/1675454636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4b81021b0 msgr2=0x7fd4b8102590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:49.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.360+0000 7fd4bdfe5640 1 --2- 172.21.15.97:0/1675454636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4b81021b0 0x7fd4b8102590 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fd4a8009900 tx=0x7fd4a802a410 comp rx=0 tx=0).stop 2024-04-23T10:35:49.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.360+0000 7fd4bdfe5640 1 -- 172.21.15.97:0/1675454636 shutdown_connections 2024-04-23T10:35:49.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.360+0000 7fd4bdfe5640 1 --2- 172.21.15.97:0/1675454636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4b8107280 0x7fd4b810ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:49.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.360+0000 7fd4bdfe5640 1 --2- 172.21.15.97:0/1675454636 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd4b8102b60 0x7fd4b8106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:49.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.360+0000 7fd4bdfe5640 1 --2- 172.21.15.97:0/1675454636 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4b81021b0 0x7fd4b8102590 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:49.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.360+0000 7fd4bdfe5640 1 -- 172.21.15.97:0/1675454636 >> 172.21.15.97:0/1675454636 conn(0x7fd4b80fd9f0 msgr2=0x7fd4b80ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:35:49.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.360+0000 7fd4bdfe5640 1 -- 172.21.15.97:0/1675454636 shutdown_connections 2024-04-23T10:35:49.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.360+0000 7fd4bdfe5640 1 -- 172.21.15.97:0/1675454636 wait complete. 2024-04-23T10:35:49.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.360+0000 7fd4bdfe5640 1 Processor -- start 2024-04-23T10:35:49.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.360+0000 7fd4bdfe5640 1 -- start start 2024-04-23T10:35:49.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.361+0000 7fd4bdfe5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4b81021b0 0x7fd4b819e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:49.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.361+0000 7fd4bdfe5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd4b8107280 0x7fd4b819e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:49.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.361+0000 7fd4bdfe5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4b8198610 0x7fd4b8198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:49.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.361+0000 7fd4bdfe5640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b8112210 con 0x7fd4b81021b0 2024-04-23T10:35:49.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.361+0000 7fd4bdfe5640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b8112090 con 0x7fd4b8107280 2024-04-23T10:35:49.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.361+0000 7fd4bdfe5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b8112390 con 0x7fd4b8198610 2024-04-23T10:35:49.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.361+0000 7fd4b77fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4b81021b0 0x7fd4b819e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:49.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.361+0000 7fd4b77fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4b81021b0 0x7fd4b819e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52420/0 (socket says 172.21.15.97:52420) 2024-04-23T10:35:49.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.361+0000 7fd4b77fe640 1 -- 172.21.15.97:0/1029648857 learned_addr learned my addr 172.21.15.97:0/1029648857 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:35:49.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.361+0000 7fd4b6ffd640 1 --2- 172.21.15.97:0/1029648857 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd4b8107280 0x7fd4b819e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:49.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.361+0000 7fd4b7fff640 1 --2- 172.21.15.97:0/1029648857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4b8198610 0x7fd4b8198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:49.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.361+0000 7fd4b77fe640 1 -- 172.21.15.97:0/1029648857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4b8198610 msgr2=0x7fd4b8198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:49.362 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.361+0000 7fd4b77fe640 1 --2- 172.21.15.97:0/1029648857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4b8198610 0x7fd4b8198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:49.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.361+0000 7fd4b77fe640 1 -- 172.21.15.97:0/1029648857 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd4b8107280 msgr2=0x7fd4b819e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:49.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.361+0000 7fd4b77fe640 1 --2- 172.21.15.97:0/1029648857 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd4b8107280 0x7fd4b819e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:49.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.361+0000 7fd4b77fe640 1 -- 172.21.15.97:0/1029648857 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4b8199130 con 0x7fd4b81021b0 2024-04-23T10:35:49.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.361+0000 7fd4b6ffd640 1 --2- 172.21.15.97:0/1029648857 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd4b8107280 0x7fd4b819e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T10:35:49.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.361+0000 7fd4b77fe640 1 --2- 172.21.15.97:0/1029648857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4b81021b0 0x7fd4b819e410 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fd4a8005ab0 tx=0x7fd4a8003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:35:49.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.361+0000 7fd4b4ff9640 1 -- 172.21.15.97:0/1029648857 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a803f070 con 0x7fd4b81021b0 2024-04-23T10:35:49.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.361+0000 7fd4b4ff9640 1 -- 172.21.15.97:0/1029648857 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4a8031d60 con 0x7fd4b81021b0 2024-04-23T10:35:49.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.361+0000 7fd4bdfe5640 1 -- 172.21.15.97:0/1029648857 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4b81993c0 con 0x7fd4b81021b0 2024-04-23T10:35:49.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.361+0000 7fd4b4ff9640 1 -- 172.21.15.97:0/1029648857 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a8035030 con 0x7fd4b81021b0 2024-04-23T10:35:49.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.362+0000 7fd4bdfe5640 1 -- 172.21.15.97:0/1029648857 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4b81a5560 con 0x7fd4b81021b0 2024-04-23T10:35:49.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.364+0000 7fd4b4ff9640 1 -- 172.21.15.97:0/1029648857 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd4a8042070 con 0x7fd4b81021b0 2024-04-23T10:35:49.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.364+0000 7fd4c0ddc740 1 Processor -- start 2024-04-23T10:35:49.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.364+0000 7fd4b4ff9640 1 --2- 172.21.15.97:0/1029648857 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fd4a00a3d20 0x7fd4a00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:49.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.364+0000 7fd4c0ddc740 1 -- start start 2024-04-23T10:35:49.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.364+0000 7fd4b7fff640 1 --2- 172.21.15.97:0/1029648857 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fd4a00a3d20 0x7fd4a00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:49.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.364+0000 7fd4c0ddc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5633a4d32dd0 0x5633a4d331b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:49.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.364+0000 7fd4b4ff9640 1 -- 172.21.15.97:0/1029648857 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8484+0+0 (secure 0 0 0) 0x7fd4a80f0f30 con 0x7fd4b81021b0 2024-04-23T10:35:49.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.364+0000 7fd4c0ddc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5633a4d33780 0x5633a4d41f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:49.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.364+0000 7fd4c0ddc740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5633a4d424a0 0x5633a4d44890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:49.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.364+0000 7fd4c0ddc740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5633a4c98370 con 0x5633a4d424a0 2024-04-23T10:35:49.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.364+0000 7fd4c0ddc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5633a4c352e0 con 0x5633a4d32dd0 2024-04-23T10:35:49.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.364+0000 7fd4c0ddc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5633a4c94330 con 0x5633a4d33780 2024-04-23T10:35:49.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.364+0000 7fd4b7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5633a4d424a0 0x5633a4d44890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:49.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.364+0000 7fd4b7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5633a4d424a0 0x5633a4d44890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52434/0 (socket says 172.21.15.97:52434) 2024-04-23T10:35:49.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.364+0000 7fd4b7fff640 1 -- 172.21.15.97:0/103856380 learned_addr learned my addr 172.21.15.97:0/103856380 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:35:49.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.365+0000 7fd4b6ffd640 1 --2- 172.21.15.97:0/103856380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5633a4d33780 0x5633a4d41f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:49.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.365+0000 7fd4b77fe640 1 --2- 172.21.15.97:0/103856380 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5633a4d32dd0 0x5633a4d331b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:49.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.365+0000 7fd4b7fff640 1 --2- 172.21.15.97:0/1029648857 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fd4a00a3d20 0x7fd4a00a61e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd4b000af90 tx=0x7fd4b0013830 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:35:49.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.365+0000 7fd4b7fff640 1 -- 172.21.15.97:0/103856380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5633a4d33780 msgr2=0x5633a4d41f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:35:49.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.365+0000 7fd4b7fff640 1 --2- 172.21.15.97:0/103856380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5633a4d33780 0x5633a4d41f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:49.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.365+0000 7fd4b7fff640 1 -- 172.21.15.97:0/103856380 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5633a4d32dd0 msgr2=0x5633a4d331b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:35:49.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.365+0000 7fd4b7fff640 1 --2- 172.21.15.97:0/103856380 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5633a4d32dd0 0x5633a4d331b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:49.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.365+0000 7fd4b7fff640 1 -- 172.21.15.97:0/103856380 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633a4a76d80 con 0x5633a4d424a0 2024-04-23T10:35:49.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.365+0000 7fd4b6ffd640 1 --2- 172.21.15.97:0/103856380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5633a4d33780 0x5633a4d41f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T10:35:49.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.365+0000 7fd4b7fff640 1 --2- 172.21.15.97:0/103856380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5633a4d424a0 0x5633a4d44890 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fd4b000a910 tx=0x7fd4b001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71d6caf9d1f822e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:35:49.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.365+0000 7fd49dffb640 1 -- 172.21.15.97:0/103856380 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b001f600 con 0x5633a4d424a0 2024-04-23T10:35:49.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.365+0000 7fd49dffb640 1 -- 172.21.15.97:0/103856380 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4b0023020 con 0x5633a4d424a0 2024-04-23T10:35:49.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.365+0000 7fd49dffb640 1 -- 172.21.15.97:0/103856380 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b000eab0 con 0x5633a4d424a0 2024-04-23T10:35:49.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.366+0000 7fd4c0ddc740 1 -- 172.21.15.97:0/103856380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5633a4d424a0 msgr2=0x5633a4d44890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:49.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.366+0000 7fd4c0ddc740 1 --2- 172.21.15.97:0/103856380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5633a4d424a0 0x5633a4d44890 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fd4b000a910 tx=0x7fd4b001e000 comp rx=0 tx=0).stop 2024-04-23T10:35:49.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.366+0000 7fd4c0ddc740 1 -- 172.21.15.97:0/103856380 shutdown_connections 2024-04-23T10:35:49.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.366+0000 7fd4c0ddc740 1 --2- 172.21.15.97:0/103856380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5633a4d424a0 0x5633a4d44890 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:49.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.366+0000 7fd4c0ddc740 1 --2- 172.21.15.97:0/103856380 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5633a4d33780 0x5633a4d41f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:49.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.366+0000 7fd4c0ddc740 1 --2- 172.21.15.97:0/103856380 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5633a4d32dd0 0x5633a4d331b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:49.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.366+0000 7fd4c0ddc740 1 -- 172.21.15.97:0/103856380 >> 172.21.15.97:0/103856380 conn(0x5633a4d2f3c0 msgr2=0x5633a4d2f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:35:49.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.366+0000 7fd4c0ddc740 1 -- 172.21.15.97:0/103856380 shutdown_connections 2024-04-23T10:35:49.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.366+0000 7fd4c0ddc740 1 -- 172.21.15.97:0/103856380 wait complete. 2024-04-23T10:35:49.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.366+0000 7fd4c0ddc740 1 -- start start 2024-04-23T10:35:49.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.366+0000 7fd4c0ddc740 1 Processor -- start 2024-04-23T10:35:49.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.367+0000 7fd4c0ddc740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5633a4d32dd0 0x5633a4d3f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:49.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.367+0000 7fd4c0ddc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5633a4d33780 0x5633a4d3f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:49.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.367+0000 7fd4c0ddc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5633a4d424a0 0x5633a4d3fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:49.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.367+0000 7fd4c0ddc740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5633a4d45200 con 0x5633a4d32dd0 2024-04-23T10:35:49.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.367+0000 7fd4c0ddc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5633a4c32fb0 con 0x5633a4d424a0 2024-04-23T10:35:49.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.367+0000 7fd4c0ddc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5633a4d45380 con 0x5633a4d33780 2024-04-23T10:35:49.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.367+0000 7fd4c0ddc740 10 client.0 _ll_register_callbacks cb 0x5633a4c329a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:35:49.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.367+0000 7fd4b77fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5633a4d32dd0 0x5633a4d3f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:49.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.367+0000 7fd4b77fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5633a4d32dd0 0x5633a4d3f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52444/0 (socket says 172.21.15.97:52444) 2024-04-23T10:35:49.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.367+0000 7fd4b77fe640 1 -- 172.21.15.97:0/1966787042 learned_addr learned my addr 172.21.15.97:0/1966787042 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:35:49.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.367+0000 7fd4b7fff640 1 --2- 172.21.15.97:0/1966787042 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5633a4d424a0 0x5633a4d3fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:49.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.367+0000 7fd4b6ffd640 1 --2- 172.21.15.97:0/1966787042 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5633a4d33780 0x5633a4d3f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:49.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.367+0000 7fd4b77fe640 1 -- 172.21.15.97:0/1966787042 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5633a4d33780 msgr2=0x5633a4d3f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:35:49.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.367+0000 7fd4b77fe640 1 --2- 172.21.15.97:0/1966787042 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5633a4d33780 0x5633a4d3f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:49.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.367+0000 7fd4b77fe640 1 -- 172.21.15.97:0/1966787042 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5633a4d424a0 msgr2=0x5633a4d3fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:35:49.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.367+0000 7fd4b77fe640 1 --2- 172.21.15.97:0/1966787042 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5633a4d424a0 0x5633a4d3fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:49.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.367+0000 7fd4b77fe640 1 -- 172.21.15.97:0/1966787042 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633a4ad7510 con 0x5633a4d32dd0 2024-04-23T10:35:49.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.367+0000 7fd4b77fe640 1 --2- 172.21.15.97:0/1966787042 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5633a4d32dd0 0x5633a4d3f420 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fd4a8005ec0 tx=0x7fd4a8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57900a1504cd097c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:35:49.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.367+0000 7fd496ffd640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:35:49.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.367+0000 7fd496ffd640 1 -- 172.21.15.97:0/1966787042 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a80db060 con 0x5633a4d32dd0 2024-04-23T10:35:49.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.367+0000 7fd496ffd640 1 -- 172.21.15.97:0/1966787042 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8484+0+0 (secure 0 0 0) 0x7fd4a80b9610 con 0x5633a4d32dd0 2024-04-23T10:35:49.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.367+0000 7fd4c0ddc740 1 -- 172.21.15.97:0/1966787042 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4b0023020 con 0x5633a4d32dd0 2024-04-23T10:35:49.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.367+0000 7fd496ffd640 1 -- 172.21.15.97:0/1966787042 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd4a80a2050 con 0x5633a4d32dd0 2024-04-23T10:35:49.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.367+0000 7fd4c0ddc740 10 client.16959 fetch_fsmap learned FSMap version 26 2024-04-23T10:35:49.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.367+0000 7fd4c0ddc740 1 -- 172.21.15.97:0/1966787042 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5633a496a670 con 0x5633a4d32dd0 2024-04-23T10:35:49.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.368+0000 7fd496ffd640 1 -- 172.21.15.97:0/1966787042 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4b0023020 con 0x5633a4d32dd0 2024-04-23T10:35:49.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.368+0000 7fd496ffd640 1 -- 172.21.15.97:0/1966787042 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd4a80d6210 con 0x5633a4d32dd0 2024-04-23T10:35:49.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.368+0000 7fd4c0ddc740 10 client.16959 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:35:49.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.368+0000 7fd4c0ddc740 10 client.16959 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:35:49.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.368+0000 7fd4c0ddc740 20 client.16959 populate_metadata read hostname 'smithi097' 2024-04-23T10:35:49.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.368+0000 7fd4c0ddc740 1 --2- 172.21.15.97:0/1966787042 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5633a4d52730 0x5633a4d54b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:49.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.368+0000 7fd4c0ddc740 4 client.16959 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:35:49.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.368+0000 7fd4c0ddc740 1 -- 172.21.15.97:0/1966787042 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5633a4ca5d90 con 0x5633a4d52730 2024-04-23T10:35:49.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.368+0000 7fd4b6ffd640 1 --2- 172.21.15.97:0/1966787042 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5633a4d52730 0x5633a4d54b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:49.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.369+0000 7fd496ffd640 10 client.16959 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:35:49.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.369+0000 7fd4b6ffd640 1 --2- 172.21.15.97:0/1966787042 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5633a4d52730 0x5633a4d54b60 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f733936ed980d96 server_cookie=1087267e6fc42eac in_seq=0 out_seq=0 2024-04-23T10:35:49.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.369+0000 7fd496ffd640 1 -- 172.21.15.97:0/1966787042 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5633a4ca5d90 con 0x5633a4d52730 2024-04-23T10:35:49.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.369+0000 7fd496ffd640 10 client.16959 handle_command_reply: tid=0 2024-04-23T10:35:49.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.369+0000 7fd4c0ddc740 1 client.16959 shutdown 2024-04-23T10:35:49.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.370+0000 7fd4c0ddc740 1 -- 172.21.15.97:0/1966787042 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5633a4d32dd0 msgr2=0x5633a4d3f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:49.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.370+0000 7fd4c0ddc740 1 --2- 172.21.15.97:0/1966787042 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5633a4d32dd0 0x5633a4d3f420 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fd4a8005ec0 tx=0x7fd4a8009260 comp rx=0 tx=0).stop 2024-04-23T10:35:49.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.370+0000 7fd4c0ddc740 1 -- 172.21.15.97:0/1966787042 shutdown_connections 2024-04-23T10:35:49.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.370+0000 7fd4c0ddc740 1 --2- 172.21.15.97:0/1966787042 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5633a4d52730 0x5633a4d54b60 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:49.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.370+0000 7fd4c0ddc740 1 --2- 172.21.15.97:0/1966787042 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5633a4d424a0 0x5633a4d3fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:49.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.370+0000 7fd4c0ddc740 1 --2- 172.21.15.97:0/1966787042 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5633a4d33780 0x5633a4d3f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:49.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.370+0000 7fd496ffd640 0 client.16959 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:35:49.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.370+0000 7fd4c0ddc740 1 --2- 172.21.15.97:0/1966787042 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5633a4d32dd0 0x5633a4d3f420 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:49.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.370+0000 7fd4c0ddc740 1 -- 172.21.15.97:0/1966787042 >> 172.21.15.97:0/1966787042 conn(0x5633a4d2f3c0 msgr2=0x5633a4d35230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:35:49.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.370+0000 7fd4c0ddc740 1 -- 172.21.15.97:0/1966787042 shutdown_connections 2024-04-23T10:35:49.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.370+0000 7fd4c0ddc740 1 -- 172.21.15.97:0/1966787042 wait complete. 2024-04-23T10:35:49.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.370+0000 7fd4c0ddc740 20 client.16959 trim_cache size 0 max 16384 2024-04-23T10:35:49.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.387+0000 7fd4c0ddc740 1 Processor -- start 2024-04-23T10:35:49.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.387+0000 7fd4c0ddc740 1 -- start start 2024-04-23T10:35:49.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.387+0000 7fd4c0ddc740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5633a4d32dd0 0x5633a4d2ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:49.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.387+0000 7fd4c0ddc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5633a4d33780 0x5633a4d35090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:49.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.387+0000 7fd4c0ddc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5633a4d424a0 0x5633a4d5a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:49.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.387+0000 7fd4c0ddc740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b81ab9b0 con 0x5633a4d32dd0 2024-04-23T10:35:49.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.387+0000 7fd4c0ddc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b81ad4f0 con 0x5633a4d424a0 2024-04-23T10:35:49.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.387+0000 7fd4c0ddc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b81ac080 con 0x5633a4d33780 2024-04-23T10:35:49.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.387+0000 7fd4b77fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5633a4d32dd0 0x5633a4d2ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:49.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.387+0000 7fd4b77fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5633a4d32dd0 0x5633a4d2ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52456/0 (socket says 172.21.15.97:52456) 2024-04-23T10:35:49.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.387+0000 7fd4b77fe640 1 -- 172.21.15.97:0/2727710315 learned_addr learned my addr 172.21.15.97:0/2727710315 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:35:49.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.387+0000 7fd4b7fff640 1 --2- 172.21.15.97:0/2727710315 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5633a4d424a0 0x5633a4d5a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:49.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.387+0000 7fd4b6ffd640 1 --2- 172.21.15.97:0/2727710315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5633a4d33780 0x5633a4d35090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:49.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.387+0000 7fd4b77fe640 1 -- 172.21.15.97:0/2727710315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5633a4d33780 msgr2=0x5633a4d35090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:35:49.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.387+0000 7fd4b77fe640 1 --2- 172.21.15.97:0/2727710315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5633a4d33780 0x5633a4d35090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:49.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.387+0000 7fd4b77fe640 1 -- 172.21.15.97:0/2727710315 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5633a4d424a0 msgr2=0x5633a4d5a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:35:49.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.387+0000 7fd4b77fe640 1 --2- 172.21.15.97:0/2727710315 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5633a4d424a0 0x5633a4d5a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:49.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.387+0000 7fd4b77fe640 1 -- 172.21.15.97:0/2727710315 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633a4d345c0 con 0x5633a4d32dd0 2024-04-23T10:35:49.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.387+0000 7fd4b6ffd640 1 --2- 172.21.15.97:0/2727710315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5633a4d33780 0x5633a4d35090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:35:49.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.387+0000 7fd4b77fe640 1 --2- 172.21.15.97:0/2727710315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5633a4d32dd0 0x5633a4d2ed80 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fd4a80bcca0 tx=0x7fd4a80bd120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d4947209694e2a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:35:49.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.388+0000 7fd49d7fa640 1 -- 172.21.15.97:0/2727710315 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a80d6070 con 0x5633a4d32dd0 2024-04-23T10:35:49.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.388+0000 7fd49d7fa640 1 -- 172.21.15.97:0/2727710315 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4a80c12a0 con 0x5633a4d32dd0 2024-04-23T10:35:49.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.388+0000 7fd49d7fa640 1 -- 172.21.15.97:0/2727710315 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a8049040 con 0x5633a4d32dd0 2024-04-23T10:35:49.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.388+0000 7fd4c0ddc740 1 -- 172.21.15.97:0/2727710315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5633a4d32dd0 msgr2=0x5633a4d2ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:49.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.388+0000 7fd4c0ddc740 1 --2- 172.21.15.97:0/2727710315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5633a4d32dd0 0x5633a4d2ed80 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fd4a80bcca0 tx=0x7fd4a80bd120 comp rx=0 tx=0).stop 2024-04-23T10:35:49.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.388+0000 7fd4c0ddc740 1 -- 172.21.15.97:0/2727710315 shutdown_connections 2024-04-23T10:35:49.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.388+0000 7fd4c0ddc740 1 --2- 172.21.15.97:0/2727710315 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5633a4d424a0 0x5633a4d5a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:49.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.388+0000 7fd4c0ddc740 1 --2- 172.21.15.97:0/2727710315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5633a4d33780 0x5633a4d35090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:49.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.388+0000 7fd4c0ddc740 1 --2- 172.21.15.97:0/2727710315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5633a4d32dd0 0x5633a4d2ed80 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:49.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.388+0000 7fd4c0ddc740 1 -- 172.21.15.97:0/2727710315 >> 172.21.15.97:0/2727710315 conn(0x5633a4d2f3c0 msgr2=0x5633a4d34980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:35:49.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.388+0000 7fd4c0ddc740 1 -- 172.21.15.97:0/2727710315 shutdown_connections 2024-04-23T10:35:49.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.388+0000 7fd4c0ddc740 1 -- 172.21.15.97:0/2727710315 wait complete. 2024-04-23T10:35:49.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.389+0000 7fd4c0ddc740 1 -- start start 2024-04-23T10:35:49.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.389+0000 7fd4c0ddc740 1 Processor -- start 2024-04-23T10:35:49.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.389+0000 7fd4c0ddc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5633a4d32dd0 0x5633a4d4a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:49.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.389+0000 7fd4c0ddc740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5633a4d33780 0x5633a4d4ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:49.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.389+0000 7fd4c0ddc740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5633a4d424a0 0x5633a4d475e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:49.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.389+0000 7fd4c0ddc740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b81ace20 con 0x5633a4d424a0 2024-04-23T10:35:49.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.389+0000 7fd4c0ddc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b81ac750 con 0x5633a4d32dd0 2024-04-23T10:35:49.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.389+0000 7fd4c0ddc740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b81a99b0 con 0x5633a4d33780 2024-04-23T10:35:49.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.389+0000 7fd4c0ddc740 10 client.0 _ll_register_callbacks cb 0x7fd4a80bcd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:35:49.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.389+0000 7fd4b7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5633a4d424a0 0x5633a4d475e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:49.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.389+0000 7fd4b7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5633a4d424a0 0x5633a4d475e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52464/0 (socket says 172.21.15.97:52464) 2024-04-23T10:35:49.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.389+0000 7fd4b7fff640 1 -- 172.21.15.97:0/4003419247 learned_addr learned my addr 172.21.15.97:0/4003419247 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:35:49.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.389+0000 7fd4b6ffd640 1 --2- 172.21.15.97:0/4003419247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5633a4d33780 0x5633a4d4ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:49.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.389+0000 7fd4b77fe640 1 --2- 172.21.15.97:0/4003419247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5633a4d32dd0 0x5633a4d4a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:49.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.389+0000 7fd4b7fff640 1 -- 172.21.15.97:0/4003419247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5633a4d33780 msgr2=0x5633a4d4ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:35:49.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.389+0000 7fd4b7fff640 1 --2- 172.21.15.97:0/4003419247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5633a4d33780 0x5633a4d4ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:49.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.389+0000 7fd4b7fff640 1 -- 172.21.15.97:0/4003419247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5633a4d32dd0 msgr2=0x5633a4d4a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:35:49.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.389+0000 7fd4b7fff640 1 --2- 172.21.15.97:0/4003419247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5633a4d32dd0 0x5633a4d4a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:49.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.389+0000 7fd4b7fff640 1 -- 172.21.15.97:0/4003419247 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633a4d47de0 con 0x5633a4d424a0 2024-04-23T10:35:49.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.389+0000 7fd4b6ffd640 1 --2- 172.21.15.97:0/4003419247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5633a4d33780 0x5633a4d4ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:35:49.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.389+0000 7fd4b77fe640 1 --2- 172.21.15.97:0/4003419247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5633a4d32dd0 0x5633a4d4a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:35:49.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.390+0000 7fd4b7fff640 1 --2- 172.21.15.97:0/4003419247 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5633a4d424a0 0x5633a4d475e0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fd4b0013dd0 tx=0x7fd4b000e5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bdb8da1a6f29e1f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:35:49.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.390+0000 7fd496ffd640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:35:49.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.390+0000 7fd496ffd640 1 -- 172.21.15.97:0/4003419247 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b0036060 con 0x5633a4d424a0 2024-04-23T10:35:49.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.390+0000 7fd496ffd640 1 -- 172.21.15.97:0/4003419247 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8484+0+0 (secure 0 0 0) 0x7fd4b000e910 con 0x5633a4d424a0 2024-04-23T10:35:49.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.390+0000 7fd4c0ddc740 1 -- 172.21.15.97:0/4003419247 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4a80c12a0 con 0x5633a4d424a0 2024-04-23T10:35:49.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.390+0000 7fd496ffd640 1 -- 172.21.15.97:0/4003419247 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd4b000e420 con 0x5633a4d424a0 2024-04-23T10:35:49.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.390+0000 7fd4c0ddc740 10 client.16965 fetch_fsmap learned FSMap version 26 2024-04-23T10:35:49.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.390+0000 7fd4c0ddc740 1 -- 172.21.15.97:0/4003419247 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5633a4db96a0 con 0x5633a4d424a0 2024-04-23T10:35:49.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.390+0000 7fd496ffd640 1 -- 172.21.15.97:0/4003419247 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4a80c12a0 con 0x5633a4d424a0 2024-04-23T10:35:49.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.390+0000 7fd496ffd640 1 -- 172.21.15.97:0/4003419247 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd4b0030070 con 0x5633a4d424a0 2024-04-23T10:35:49.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.390+0000 7fd4c0ddc740 10 client.16965 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:35:49.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.390+0000 7fd4c0ddc740 10 client.16965 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:35:49.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.390+0000 7fd4c0ddc740 20 client.16965 populate_metadata read hostname 'smithi097' 2024-04-23T10:35:49.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.390+0000 7fd4c0ddc740 1 --2- 172.21.15.97:0/4003419247 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5633a4db9b50 0x5633a4dbbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:49.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.390+0000 7fd4c0ddc740 4 client.16965 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 77.0, "await_for": 42.0}] 2024-04-23T10:35:49.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.390+0000 7fd4c0ddc740 1 -- 172.21.15.97:0/4003419247 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 77.0, "await_for": 42.0}) -- 0x5633a4c9fc00 con 0x5633a4db9b50 2024-04-23T10:35:49.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.390+0000 7fd4b77fe640 1 --2- 172.21.15.97:0/4003419247 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5633a4db9b50 0x5633a4dbbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:49.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.391+0000 7fd4b77fe640 1 --2- 172.21.15.97:0/4003419247 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5633a4db9b50 0x5633a4dbbf70 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a123ea67f5704ad server_cookie=41b30a0e9be79dae in_seq=0 out_seq=0 2024-04-23T10:35:49.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.391+0000 7fd496ffd640 10 client.16965 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:35:49.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.404+0000 7fd496ffd640 1 -- 172.21.15.97:0/4003419247 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5633a4c9fc00 con 0x5633a4db9b50 2024-04-23T10:35:49.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.404+0000 7fd496ffd640 10 client.16965 handle_command_reply: tid=0 2024-04-23T10:35:49.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.404+0000 7fd4c0ddc740 1 client.16965 shutdown 2024-04-23T10:35:49.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.404+0000 7fd4c0ddc740 1 -- 172.21.15.97:0/4003419247 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5633a4d424a0 msgr2=0x5633a4d475e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:49.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.404+0000 7fd4c0ddc740 1 --2- 172.21.15.97:0/4003419247 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5633a4d424a0 0x5633a4d475e0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fd4b0013dd0 tx=0x7fd4b000e5f0 comp rx=0 tx=0).stop 2024-04-23T10:35:49.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.404+0000 7fd4c0ddc740 1 -- 172.21.15.97:0/4003419247 shutdown_connections 2024-04-23T10:35:49.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.404+0000 7fd4c0ddc740 1 --2- 172.21.15.97:0/4003419247 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5633a4db9b50 0x5633a4dbbf70 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:49.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.404+0000 7fd4c0ddc740 1 --2- 172.21.15.97:0/4003419247 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5633a4d424a0 0x5633a4d475e0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:49.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.404+0000 7fd496ffd640 0 client.16965 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:35:49.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.404+0000 7fd4c0ddc740 1 --2- 172.21.15.97:0/4003419247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5633a4d33780 0x5633a4d4ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:49.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.404+0000 7fd4c0ddc740 1 --2- 172.21.15.97:0/4003419247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5633a4d32dd0 0x5633a4d4a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:49.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.404+0000 7fd4c0ddc740 1 -- 172.21.15.97:0/4003419247 >> 172.21.15.97:0/4003419247 conn(0x5633a4d2f3c0 msgr2=0x5633a4d35210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:35:49.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.404+0000 7fd4c0ddc740 1 -- 172.21.15.97:0/4003419247 shutdown_connections 2024-04-23T10:35:49.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.404+0000 7fd4c0ddc740 1 -- 172.21.15.97:0/4003419247 wait complete. 2024-04-23T10:35:49.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.404+0000 7fd4c0ddc740 20 client.16965 trim_cache size 0 max 16384 2024-04-23T10:35:49.405 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-23T10:35:49.405 INFO:teuthology.orchestra.run.smithi097.stdout: "epoch": 26, 2024-04-23T10:35:49.405 INFO:teuthology.orchestra.run.smithi097.stdout: "leader": 24488, 2024-04-23T10:35:49.405 INFO:teuthology.orchestra.run.smithi097.stdout: "set_version": 42, 2024-04-23T10:35:49.405 INFO:teuthology.orchestra.run.smithi097.stdout: "sets": { 2024-04-23T10:35:49.405 INFO:teuthology.orchestra.run.smithi097.stdout: "32aeb117": { 2024-04-23T10:35:49.405 INFO:teuthology.orchestra.run.smithi097.stdout: "version": 42, 2024-04-23T10:35:49.405 INFO:teuthology.orchestra.run.smithi097.stdout: "age_ref": 0.0, 2024-04-23T10:35:49.406 INFO:teuthology.orchestra.run.smithi097.stdout: "state": { 2024-04-23T10:35:49.406 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "QUIESCED", 2024-04-23T10:35:49.406 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0 2024-04-23T10:35:49.406 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:35:49.406 INFO:teuthology.orchestra.run.smithi097.stdout: "timeout": 60.0, 2024-04-23T10:35:49.406 INFO:teuthology.orchestra.run.smithi097.stdout: "expiration": 77.0, 2024-04-23T10:35:49.406 INFO:teuthology.orchestra.run.smithi097.stdout: "members": { 2024-04-23T10:35:49.406 INFO:teuthology.orchestra.run.smithi097.stdout: "file:/": { 2024-04-23T10:35:49.406 INFO:teuthology.orchestra.run.smithi097.stdout: "excluded": false, 2024-04-23T10:35:49.406 INFO:teuthology.orchestra.run.smithi097.stdout: "state": { 2024-04-23T10:35:49.406 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "QUIESCED", 2024-04-23T10:35:49.406 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0 2024-04-23T10:35:49.406 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:35:49.406 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:35:49.406 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:35:49.406 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:35:49.407 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:35:49.407 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-23T10:35:49.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.405+0000 7fd4bdfe5640 1 -- 172.21.15.97:0/1029648857 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fd4a00a3d20 msgr2=0x7fd4a00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:49.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.405+0000 7fd4bdfe5640 1 --2- 172.21.15.97:0/1029648857 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fd4a00a3d20 0x7fd4a00a61e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd4b000af90 tx=0x7fd4b0013830 comp rx=0 tx=0).stop 2024-04-23T10:35:49.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.405+0000 7fd4bdfe5640 1 -- 172.21.15.97:0/1029648857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4b81021b0 msgr2=0x7fd4b819e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:49.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.405+0000 7fd4bdfe5640 1 --2- 172.21.15.97:0/1029648857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4b81021b0 0x7fd4b819e410 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fd4a8005ab0 tx=0x7fd4a8003c80 comp rx=0 tx=0).stop 2024-04-23T10:35:49.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.405+0000 7fd4bdfe5640 1 -- 172.21.15.97:0/1029648857 shutdown_connections 2024-04-23T10:35:49.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.405+0000 7fd4bdfe5640 1 --2- 172.21.15.97:0/1029648857 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fd4a00a3d20 0x7fd4a00a61e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:49.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.405+0000 7fd4bdfe5640 1 --2- 172.21.15.97:0/1029648857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4b8198610 0x7fd4b8198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:49.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.405+0000 7fd4bdfe5640 1 --2- 172.21.15.97:0/1029648857 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd4b8107280 0x7fd4b819e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:49.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.405+0000 7fd4bdfe5640 1 --2- 172.21.15.97:0/1029648857 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd4b81021b0 0x7fd4b819e410 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:49.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.405+0000 7fd4bdfe5640 1 -- 172.21.15.97:0/1029648857 >> 172.21.15.97:0/1029648857 conn(0x7fd4b80fd9f0 msgr2=0x7fd4b80fee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:35:49.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.405+0000 7fd4bdfe5640 1 -- 172.21.15.97:0/1029648857 shutdown_connections 2024-04-23T10:35:49.408 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:35:49.405+0000 7fd4bdfe5640 1 -- 172.21.15.97:0/1029648857 wait complete. 2024-04-23T10:35:49.414 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '32aeb117', quiesce took 0.5 seconds. Will release after: 17.0 2024-04-23T10:35:49.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:35:49 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/434188820' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:35:49.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:35:49 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/434188820' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:35:49.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:35:49 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/434188820' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:35:50.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:35:50 smithi097 ceph-mon[29197]: pgmap v298: 97 pgs: 97 active+clean; 5.6 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-04-23T10:35:50.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:35:50 smithi183 ceph-mon[32317]: pgmap v298: 97 pgs: 97 active+clean; 5.6 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-04-23T10:35:50.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:35:50 smithi196 ceph-mon[32919]: pgmap v298: 97 pgs: 97 active+clean; 5.6 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-04-23T10:35:51.970 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.968+0000 7f6bb8375640 1 -- 172.21.15.97:0/2108433726 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6bb0103b20 msgr2=0x7f6bb010c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:51.970 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.968+0000 7f6bb8375640 1 --2- 172.21.15.97:0/2108433726 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6bb0103b20 0x7f6bb010c000 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0009900 tx=0x7f6ba002a480 comp rx=0 tx=0).stop 2024-04-23T10:35:51.970 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.968+0000 7f6bb8375640 1 -- 172.21.15.97:0/2108433726 shutdown_connections 2024-04-23T10:35:51.970 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.968+0000 7f6bb8375640 1 --2- 172.21.15.97:0/2108433726 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6bb010c540 0x7f6bb010e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:51.970 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.968+0000 7f6bb8375640 1 --2- 172.21.15.97:0/2108433726 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6bb0103b20 0x7f6bb010c000 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:51.970 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.968+0000 7f6bb8375640 1 --2- 172.21.15.97:0/2108433726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6bb0103200 0x7f6bb01035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:51.970 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.968+0000 7f6bb8375640 1 -- 172.21.15.97:0/2108433726 >> 172.21.15.97:0/2108433726 conn(0x7f6bb0077fd0 msgr2=0x7f6bb00783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:35:51.970 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.968+0000 7f6bb8375640 1 -- 172.21.15.97:0/2108433726 shutdown_connections 2024-04-23T10:35:51.970 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.968+0000 7f6bb8375640 1 -- 172.21.15.97:0/2108433726 wait complete. 2024-04-23T10:35:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.968+0000 7f6bb8375640 1 Processor -- start 2024-04-23T10:35:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.968+0000 7f6bb8375640 1 -- start start 2024-04-23T10:35:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.969+0000 7f6bb8375640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6bb0103200 0x7f6bb0198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.969+0000 7f6bb8375640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6bb0103b20 0x7f6bb0198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.969+0000 7f6bb8375640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6bb010c540 0x7f6bb019cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.969+0000 7f6bb8375640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb0112180 con 0x7f6bb010c540 2024-04-23T10:35:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.969+0000 7f6bb8375640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb0112000 con 0x7f6bb0103200 2024-04-23T10:35:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.969+0000 7f6bb8375640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb0112300 con 0x7f6bb0103b20 2024-04-23T10:35:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.969+0000 7f6bb6912640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6bb010c540 0x7f6bb019cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.969+0000 7f6bb6912640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6bb010c540 0x7f6bb019cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52492/0 (socket says 172.21.15.97:52492) 2024-04-23T10:35:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.969+0000 7f6bb6912640 1 -- 172.21.15.97:0/1131776658 learned_addr learned my addr 172.21.15.97:0/1131776658 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:35:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.969+0000 7f6bb6111640 1 --2- 172.21.15.97:0/1131776658 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6bb0103200 0x7f6bb0198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.969+0000 7f6bb5910640 1 --2- 172.21.15.97:0/1131776658 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6bb0103b20 0x7f6bb0198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.969+0000 7f6bb6912640 1 -- 172.21.15.97:0/1131776658 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6bb0103b20 msgr2=0x7f6bb0198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.969+0000 7f6bb6912640 1 --2- 172.21.15.97:0/1131776658 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6bb0103b20 0x7f6bb0198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:51.972 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.969+0000 7f6bb6912640 1 -- 172.21.15.97:0/1131776658 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6bb0103200 msgr2=0x7f6bb0198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:51.972 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.969+0000 7f6bb6912640 1 --2- 172.21.15.97:0/1131776658 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6bb0103200 0x7f6bb0198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:51.972 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.969+0000 7f6bb6912640 1 -- 172.21.15.97:0/1131776658 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bb019d6c0 con 0x7f6bb010c540 2024-04-23T10:35:51.972 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.969+0000 7f6bb5910640 1 --2- 172.21.15.97:0/1131776658 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6bb0103b20 0x7f6bb0198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:35:51.972 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.969+0000 7f6bb6912640 1 --2- 172.21.15.97:0/1131776658 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6bb010c540 0x7f6bb019cf90 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f6ba800eaa0 tx=0x7f6ba800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:35:51.972 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.969+0000 7f6ba77fe640 1 -- 172.21.15.97:0/1131776658 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba8004d20 con 0x7f6bb010c540 2024-04-23T10:35:51.972 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.969+0000 7f6ba77fe640 1 -- 172.21.15.97:0/1131776658 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ba8005020 con 0x7f6bb010c540 2024-04-23T10:35:51.972 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.969+0000 7f6bb8375640 1 -- 172.21.15.97:0/1131776658 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bb019d9b0 con 0x7f6bb010c540 2024-04-23T10:35:51.972 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.969+0000 7f6ba77fe640 1 -- 172.21.15.97:0/1131776658 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba80116b0 con 0x7f6bb010c540 2024-04-23T10:35:51.972 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.969+0000 7f6bb8375640 1 -- 172.21.15.97:0/1131776658 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bb01a5240 con 0x7f6bb010c540 2024-04-23T10:35:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.973+0000 7f6bbb085740 1 Processor -- start 2024-04-23T10:35:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.973+0000 7f6ba77fe640 1 -- 172.21.15.97:0/1131776658 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6ba80051d0 con 0x7f6bb010c540 2024-04-23T10:35:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.973+0000 7f6bbb085740 1 -- start start 2024-04-23T10:35:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.973+0000 7f6bbb085740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ebf9868620 0x55ebf9868a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.973+0000 7f6bbb085740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ebf9868f40 0x55ebf98732c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.973+0000 7f6bbb085740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ebf9873800 0x55ebf9875bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.973+0000 7f6bbb085740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ebf97cd970 con 0x55ebf9868620 2024-04-23T10:35:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.973+0000 7f6bbb085740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ebf976a4a0 con 0x55ebf9873800 2024-04-23T10:35:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.973+0000 7f6bbb085740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ebf97c9930 con 0x55ebf9868f40 2024-04-23T10:35:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.973+0000 7f6bb5910640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ebf9868620 0x55ebf9868a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:51.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.973+0000 7f6bb5910640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ebf9868620 0x55ebf9868a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52506/0 (socket says 172.21.15.97:52506) 2024-04-23T10:35:51.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.973+0000 7f6bb5910640 1 -- 172.21.15.97:0/2242578858 learned_addr learned my addr 172.21.15.97:0/2242578858 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:35:51.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.973+0000 7f6ba77fe640 1 --2- 172.21.15.97:0/1131776658 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f6b9c0a3e40 0x7f6b9c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:51.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.973+0000 7f6bb6111640 1 --2- 172.21.15.97:0/2242578858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ebf9868f40 0x55ebf98732c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:51.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.974+0000 7f6bb5910640 1 --2- 172.21.15.97:0/2242578858 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ebf9873800 0x55ebf9875bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:51.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.974+0000 7f6bb5910640 1 -- 172.21.15.97:0/2242578858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ebf9868f40 msgr2=0x55ebf98732c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:35:51.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.974+0000 7f6bb5910640 1 --2- 172.21.15.97:0/2242578858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ebf9868f40 0x55ebf98732c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:51.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.974+0000 7f6bb5910640 1 -- 172.21.15.97:0/2242578858 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ebf9873800 msgr2=0x55ebf9875bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:35:51.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.974+0000 7f6bb5910640 1 --2- 172.21.15.97:0/2242578858 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ebf9873800 0x55ebf9875bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:51.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.974+0000 7f6bb5910640 1 -- 172.21.15.97:0/2242578858 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebf94e0210 con 0x55ebf9868620 2024-04-23T10:35:51.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.974+0000 7f6bb6912640 1 --2- 172.21.15.97:0/1131776658 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f6b9c0a3e40 0x7f6b9c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:51.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.974+0000 7f6ba77fe640 1 -- 172.21.15.97:0/1131776658 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8484+0+0 (secure 0 0 0) 0x7f6ba80d0940 con 0x7f6bb010c540 2024-04-23T10:35:51.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.974+0000 7f6bb5910640 1 --2- 172.21.15.97:0/2242578858 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ebf9868620 0x55ebf9868a00 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0005ab0 tx=0x7f6ba0005b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fdf6e82ac247438 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:35:51.980 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.974+0000 7f6ba4bfe640 1 -- 172.21.15.97:0/2242578858 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba0035050 con 0x55ebf9868620 2024-04-23T10:35:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.974+0000 7f6bb6912640 1 --2- 172.21.15.97:0/1131776658 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f6b9c0a3e40 0x7f6b9c0a6300 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6ba809c600 tx=0x7f6ba8087040 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:35:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.974+0000 7f6ba4bfe640 1 -- 172.21.15.97:0/2242578858 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ba00317e0 con 0x55ebf9868620 2024-04-23T10:35:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.974+0000 7f6ba4bfe640 1 -- 172.21.15.97:0/2242578858 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba0005650 con 0x55ebf9868620 2024-04-23T10:35:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.975+0000 7f6bbb085740 1 -- 172.21.15.97:0/2242578858 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ebf9868620 msgr2=0x55ebf9868a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.975+0000 7f6bbb085740 1 --2- 172.21.15.97:0/2242578858 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ebf9868620 0x55ebf9868a00 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0005ab0 tx=0x7f6ba0005b70 comp rx=0 tx=0).stop 2024-04-23T10:35:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.975+0000 7f6bbb085740 1 -- 172.21.15.97:0/2242578858 shutdown_connections 2024-04-23T10:35:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.975+0000 7f6bbb085740 1 --2- 172.21.15.97:0/2242578858 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ebf9873800 0x55ebf9875bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.975+0000 7f6bbb085740 1 --2- 172.21.15.97:0/2242578858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ebf9868f40 0x55ebf98732c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.975+0000 7f6bbb085740 1 --2- 172.21.15.97:0/2242578858 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ebf9868620 0x55ebf9868a00 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.975+0000 7f6bbb085740 1 -- 172.21.15.97:0/2242578858 >> 172.21.15.97:0/2242578858 conn(0x55ebf9864b30 msgr2=0x55ebf9864f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:35:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.975+0000 7f6bbb085740 1 -- 172.21.15.97:0/2242578858 shutdown_connections 2024-04-23T10:35:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.975+0000 7f6bbb085740 1 -- 172.21.15.97:0/2242578858 wait complete. 2024-04-23T10:35:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.975+0000 7f6bbb085740 1 -- start start 2024-04-23T10:35:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.975+0000 7f6bbb085740 1 Processor -- start 2024-04-23T10:35:51.982 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.976+0000 7f6bbb085740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ebf9868620 0x55ebf9873130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:51.982 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.976+0000 7f6bbb085740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ebf9868f40 0x55ebf9870420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:51.982 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.976+0000 7f6bbb085740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ebf9873800 0x55ebf9870960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:51.982 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.976+0000 7f6bbb085740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ebf98763d0 con 0x55ebf9873800 2024-04-23T10:35:51.982 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.976+0000 7f6bbb085740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ebf97680b0 con 0x55ebf9868620 2024-04-23T10:35:51.982 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.976+0000 7f6bbb085740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ebf9876550 con 0x55ebf9868f40 2024-04-23T10:35:51.982 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.976+0000 7f6bbb085740 10 client.0 _ll_register_callbacks cb 0x55ebf9755d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:35:51.982 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.976+0000 7f6bb5910640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ebf9873800 0x55ebf9870960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:51.982 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.976+0000 7f6bb5910640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ebf9873800 0x55ebf9870960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52508/0 (socket says 172.21.15.97:52508) 2024-04-23T10:35:51.982 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.976+0000 7f6bb5910640 1 -- 172.21.15.97:0/271069710 learned_addr learned my addr 172.21.15.97:0/271069710 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:35:51.982 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.976+0000 7f6bb5910640 1 --2- 172.21.15.97:0/271069710 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ebf9868620 0x55ebf9873130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:51.982 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.976+0000 7f6bb6111640 1 --2- 172.21.15.97:0/271069710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ebf9868f40 0x55ebf9870420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:51.982 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.976+0000 7f6bb5910640 1 -- 172.21.15.97:0/271069710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ebf9868f40 msgr2=0x55ebf9870420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:35:51.983 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.976+0000 7f6bb5910640 1 --2- 172.21.15.97:0/271069710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ebf9868f40 0x55ebf9870420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:51.983 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.976+0000 7f6bb5910640 1 -- 172.21.15.97:0/271069710 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ebf9868620 msgr2=0x55ebf9873130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:35:51.983 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.976+0000 7f6bb5910640 1 --2- 172.21.15.97:0/271069710 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ebf9868620 0x55ebf9873130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:51.983 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.976+0000 7f6bb5910640 1 -- 172.21.15.97:0/271069710 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebf95ac620 con 0x55ebf9873800 2024-04-23T10:35:51.983 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.976+0000 7f6bb5910640 1 --2- 172.21.15.97:0/271069710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ebf9873800 0x55ebf9870960 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0031a90 tx=0x7f6ba0005570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f59ddf04c8c5d486 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:35:51.983 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.976+0000 7f6b99ffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:35:51.983 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.976+0000 7f6b99ffb640 1 -- 172.21.15.97:0/271069710 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba0035040 con 0x55ebf9873800 2024-04-23T10:35:51.983 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.976+0000 7f6b99ffb640 1 -- 172.21.15.97:0/271069710 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8484+0+0 (secure 0 0 0) 0x7f6ba00042d0 con 0x55ebf9873800 2024-04-23T10:35:51.983 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.977+0000 7f6bbb085740 1 -- 172.21.15.97:0/271069710 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ba00317e0 con 0x55ebf9873800 2024-04-23T10:35:51.983 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.977+0000 7f6b99ffb640 1 -- 172.21.15.97:0/271069710 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6ba005c050 con 0x55ebf9873800 2024-04-23T10:35:51.983 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.977+0000 7f6bbb085740 10 client.16977 fetch_fsmap learned FSMap version 26 2024-04-23T10:35:51.983 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.977+0000 7f6bbb085740 1 -- 172.21.15.97:0/271069710 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ebf9882320 con 0x55ebf9873800 2024-04-23T10:35:51.983 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.977+0000 7f6b99ffb640 1 -- 172.21.15.97:0/271069710 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ba00317e0 con 0x55ebf9873800 2024-04-23T10:35:51.983 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.977+0000 7f6b99ffb640 1 -- 172.21.15.97:0/271069710 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f6ba0003680 con 0x55ebf9873800 2024-04-23T10:35:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.977+0000 7f6bbb085740 10 client.16977 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:35:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.977+0000 7f6bbb085740 10 client.16977 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:35:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.977+0000 7f6bbb085740 20 client.16977 populate_metadata read hostname 'smithi097' 2024-04-23T10:35:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.977+0000 7f6bbb085740 1 --2- 172.21.15.97:0/271069710 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55ebf9883670 0x55ebf9885a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.977+0000 7f6bbb085740 4 client.16977 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:35:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.977+0000 7f6bbb085740 1 -- 172.21.15.97:0/271069710 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ebf97db390 con 0x55ebf9883670 2024-04-23T10:35:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.977+0000 7f6bb6111640 1 --2- 172.21.15.97:0/271069710 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55ebf9883670 0x55ebf9885a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.978+0000 7f6bb6111640 1 --2- 172.21.15.97:0/271069710 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55ebf9883670 0x55ebf9885a60 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9af8db06a4064f88 server_cookie=8ef6ac350d6f4482 in_seq=0 out_seq=0 2024-04-23T10:35:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.978+0000 7f6b99ffb640 10 client.16977 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:35:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.978+0000 7f6b99ffb640 1 -- 172.21.15.97:0/271069710 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ebf97db390 con 0x55ebf9883670 2024-04-23T10:35:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.978+0000 7f6b99ffb640 10 client.16977 handle_command_reply: tid=0 2024-04-23T10:35:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.979+0000 7f6bbb085740 1 client.16977 shutdown 2024-04-23T10:35:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.979+0000 7f6bbb085740 1 -- 172.21.15.97:0/271069710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ebf9873800 msgr2=0x55ebf9870960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.979+0000 7f6bbb085740 1 --2- 172.21.15.97:0/271069710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ebf9873800 0x55ebf9870960 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0031a90 tx=0x7f6ba0005570 comp rx=0 tx=0).stop 2024-04-23T10:35:51.985 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.979+0000 7f6bbb085740 1 -- 172.21.15.97:0/271069710 shutdown_connections 2024-04-23T10:35:51.985 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.979+0000 7f6bbb085740 1 --2- 172.21.15.97:0/271069710 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55ebf9883670 0x55ebf9885a60 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:51.985 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.979+0000 7f6bbb085740 1 --2- 172.21.15.97:0/271069710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ebf9873800 0x55ebf9870960 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:51.985 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.979+0000 7f6bbb085740 1 --2- 172.21.15.97:0/271069710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ebf9868f40 0x55ebf9870420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:51.985 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.979+0000 7f6b99ffb640 0 client.16977 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:35:51.985 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.979+0000 7f6bbb085740 1 --2- 172.21.15.97:0/271069710 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ebf9868620 0x55ebf9873130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:51.985 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.979+0000 7f6bbb085740 1 -- 172.21.15.97:0/271069710 >> 172.21.15.97:0/271069710 conn(0x55ebf9864b30 msgr2=0x55ebf986a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:35:51.985 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.979+0000 7f6bbb085740 1 -- 172.21.15.97:0/271069710 shutdown_connections 2024-04-23T10:35:51.985 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.979+0000 7f6bbb085740 1 -- 172.21.15.97:0/271069710 wait complete. 2024-04-23T10:35:51.985 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.979+0000 7f6bbb085740 20 client.16977 trim_cache size 0 max 16384 2024-04-23T10:35:51.998 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.994+0000 7f6bbb085740 1 Processor -- start 2024-04-23T10:35:51.998 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.994+0000 7f6bbb085740 1 -- start start 2024-04-23T10:35:51.998 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.994+0000 7f6bbb085740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ebf9864b30 0x55ebf9888e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:51.998 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.994+0000 7f6bbb085740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ebf9868620 0x55ebf987eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:51.998 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.994+0000 7f6bbb085740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ebf9868f40 0x55ebf9872c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:51.998 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.994+0000 7f6bbb085740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb01ad9f0 con 0x55ebf9868f40 2024-04-23T10:35:51.998 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.994+0000 7f6bbb085740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb01af530 con 0x55ebf9864b30 2024-04-23T10:35:51.998 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.994+0000 7f6bbb085740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb01ae0c0 con 0x55ebf9868620 2024-04-23T10:35:51.998 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.994+0000 7f6bb5910640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ebf9868f40 0x55ebf9872c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:51.998 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.994+0000 7f6bb5910640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ebf9868f40 0x55ebf9872c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52518/0 (socket says 172.21.15.97:52518) 2024-04-23T10:35:51.998 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.994+0000 7f6bb5910640 1 -- 172.21.15.97:0/1586979756 learned_addr learned my addr 172.21.15.97:0/1586979756 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:35:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.994+0000 7f6bb5910640 1 --2- 172.21.15.97:0/1586979756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ebf9864b30 0x55ebf9888e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.994+0000 7f6bb6111640 1 --2- 172.21.15.97:0/1586979756 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ebf9868620 0x55ebf987eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.994+0000 7f6bb5910640 1 -- 172.21.15.97:0/1586979756 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ebf9868620 msgr2=0x55ebf987eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:35:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.994+0000 7f6bb5910640 1 --2- 172.21.15.97:0/1586979756 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ebf9868620 0x55ebf987eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.994+0000 7f6bb5910640 1 -- 172.21.15.97:0/1586979756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ebf9864b30 msgr2=0x55ebf9888e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:35:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.994+0000 7f6bb5910640 1 --2- 172.21.15.97:0/1586979756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ebf9864b30 0x55ebf9888e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.994+0000 7f6bb5910640 1 -- 172.21.15.97:0/1586979756 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebf9868d70 con 0x55ebf9868f40 2024-04-23T10:35:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.995+0000 7f6bb5910640 1 --2- 172.21.15.97:0/1586979756 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ebf9868f40 0x55ebf9872c60 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0009900 tx=0x7f6ba0031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1c5c006346feafc server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:35:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.995+0000 7f6b9bfff640 1 -- 172.21.15.97:0/1586979756 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba0035070 con 0x55ebf9868f40 2024-04-23T10:35:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.995+0000 7f6b9bfff640 1 -- 172.21.15.97:0/1586979756 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ba00317e0 con 0x55ebf9868f40 2024-04-23T10:35:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.995+0000 7f6b9bfff640 1 -- 172.21.15.97:0/1586979756 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba00035f0 con 0x55ebf9868f40 2024-04-23T10:35:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.995+0000 7f6bbb085740 1 -- 172.21.15.97:0/1586979756 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ebf9868f40 msgr2=0x55ebf9872c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.995+0000 7f6bbb085740 1 --2- 172.21.15.97:0/1586979756 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ebf9868f40 0x55ebf9872c60 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0009900 tx=0x7f6ba0031400 comp rx=0 tx=0).stop 2024-04-23T10:35:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.995+0000 7f6bbb085740 1 -- 172.21.15.97:0/1586979756 shutdown_connections 2024-04-23T10:35:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.995+0000 7f6bbb085740 1 --2- 172.21.15.97:0/1586979756 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ebf9868f40 0x55ebf9872c60 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.995+0000 7f6bbb085740 1 --2- 172.21.15.97:0/1586979756 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ebf9868620 0x55ebf987eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.995+0000 7f6bbb085740 1 --2- 172.21.15.97:0/1586979756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ebf9864b30 0x55ebf9888e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.995+0000 7f6bbb085740 1 -- 172.21.15.97:0/1586979756 >> 172.21.15.97:0/1586979756 conn(0x55ebf9870ac0 msgr2=0x55ebf987e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:35:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.995+0000 7f6bbb085740 1 -- 172.21.15.97:0/1586979756 shutdown_connections 2024-04-23T10:35:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.995+0000 7f6bbb085740 1 -- 172.21.15.97:0/1586979756 wait complete. 2024-04-23T10:35:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.996+0000 7f6bbb085740 1 -- start start 2024-04-23T10:35:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.996+0000 7f6bbb085740 1 Processor -- start 2024-04-23T10:35:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.996+0000 7f6bbb085740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ebf9864b30 0x55ebf98e19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.996+0000 7f6bbb085740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ebf9868620 0x55ebf98e1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.996+0000 7f6bbb085740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ebf9868f40 0x55ebf98ec880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.996+0000 7f6bbb085740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb01aee60 con 0x55ebf9864b30 2024-04-23T10:35:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.996+0000 7f6bbb085740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb01ae790 con 0x55ebf9868f40 2024-04-23T10:35:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.996+0000 7f6bbb085740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb01ab9f0 con 0x55ebf9868620 2024-04-23T10:35:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.996+0000 7f6bbb085740 10 client.0 _ll_register_callbacks cb 0x7f6ba00316d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:35:52.001 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.996+0000 7f6bb5910640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ebf9864b30 0x55ebf98e19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:52.001 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.996+0000 7f6bb5910640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ebf9864b30 0x55ebf98e19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52528/0 (socket says 172.21.15.97:52528) 2024-04-23T10:35:52.001 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.996+0000 7f6bb5910640 1 -- 172.21.15.97:0/629904187 learned_addr learned my addr 172.21.15.97:0/629904187 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:35:52.001 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.996+0000 7f6bb6111640 1 --2- 172.21.15.97:0/629904187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ebf9868620 0x55ebf98e1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:52.001 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.996+0000 7f6bb5910640 1 -- 172.21.15.97:0/629904187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ebf9868620 msgr2=0x55ebf98e1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:35:52.001 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.996+0000 7f6bb5910640 1 --2- 172.21.15.97:0/629904187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ebf9868620 0x55ebf98e1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:52.001 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.996+0000 7f6bb5910640 1 -- 172.21.15.97:0/629904187 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ebf9868f40 msgr2=0x55ebf98ec880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:35:52.001 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.996+0000 7f6bb5910640 1 --2- 172.21.15.97:0/629904187 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ebf9868f40 0x55ebf98ec880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:52.001 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.996+0000 7f6bb5910640 1 -- 172.21.15.97:0/629904187 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebf98ece40 con 0x55ebf9864b30 2024-04-23T10:35:52.001 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.996+0000 7f6bb6111640 1 --2- 172.21.15.97:0/629904187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ebf9868620 0x55ebf98e1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:35:52.001 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.996+0000 7f6bb5910640 1 --2- 172.21.15.97:0/629904187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ebf9864b30 0x55ebf98e19f0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0033000 tx=0x7f6ba00329a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66b709105a8d803a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:35:52.001 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.997+0000 7f6b99ffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:35:52.001 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.997+0000 7f6b99ffb640 1 -- 172.21.15.97:0/629904187 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba0035040 con 0x55ebf9864b30 2024-04-23T10:35:52.002 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.997+0000 7f6b99ffb640 1 -- 172.21.15.97:0/629904187 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8484+0+0 (secure 0 0 0) 0x7f6ba00035f0 con 0x55ebf9864b30 2024-04-23T10:35:52.002 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.997+0000 7f6bbb085740 1 -- 172.21.15.97:0/629904187 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ba00317e0 con 0x55ebf9864b30 2024-04-23T10:35:52.002 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.997+0000 7f6b99ffb640 1 -- 172.21.15.97:0/629904187 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6ba00044e0 con 0x55ebf9864b30 2024-04-23T10:35:52.002 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.997+0000 7f6bbb085740 10 client.16983 fetch_fsmap learned FSMap version 26 2024-04-23T10:35:52.002 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.997+0000 7f6bbb085740 1 -- 172.21.15.97:0/629904187 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ebf98ee130 con 0x55ebf9864b30 2024-04-23T10:35:52.002 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.997+0000 7f6b99ffb640 1 -- 172.21.15.97:0/629904187 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ba00317e0 con 0x55ebf9864b30 2024-04-23T10:35:52.002 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.997+0000 7f6b99ffb640 1 -- 172.21.15.97:0/629904187 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f6ba0070050 con 0x55ebf9864b30 2024-04-23T10:35:52.002 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.997+0000 7f6bbb085740 10 client.16983 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:35:52.002 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.997+0000 7f6bbb085740 10 client.16983 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:35:52.002 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.997+0000 7f6bbb085740 20 client.16983 populate_metadata read hostname 'smithi097' 2024-04-23T10:35:52.002 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.997+0000 7f6bbb085740 1 --2- 172.21.15.97:0/629904187 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55ebf9873800 0x55ebf98f0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:35:52.002 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.997+0000 7f6bbb085740 4 client.16983 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:35:52.002 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.997+0000 7f6bbb085740 1 -- 172.21.15.97:0/629904187 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ebf97d5200 con 0x55ebf9873800 2024-04-23T10:35:52.002 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.998+0000 7f6bb6111640 1 --2- 172.21.15.97:0/629904187 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55ebf9873800 0x55ebf98f0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:35:52.003 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.998+0000 7f6b99ffb640 10 client.16983 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:35:52.003 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:51.998+0000 7f6bb6111640 1 --2- 172.21.15.97:0/629904187 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55ebf9873800 0x55ebf98f0620 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89b73f391454ca48 server_cookie=233798c6a140644d in_seq=0 out_seq=0 2024-04-23T10:35:52.013 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:52.013+0000 7f6b99ffb640 1 -- 172.21.15.97:0/629904187 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ebf97d5200 con 0x55ebf9873800 2024-04-23T10:35:52.013 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:52.013+0000 7f6b99ffb640 10 client.16983 handle_command_reply: tid=0 2024-04-23T10:35:52.013 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:52.014+0000 7f6bbb085740 1 client.16983 shutdown 2024-04-23T10:35:52.013 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:52.014+0000 7f6bbb085740 1 -- 172.21.15.97:0/629904187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ebf9864b30 msgr2=0x55ebf98e19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:52.013 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:52.014+0000 7f6bbb085740 1 --2- 172.21.15.97:0/629904187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ebf9864b30 0x55ebf98e19f0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0033000 tx=0x7f6ba00329a0 comp rx=0 tx=0).stop 2024-04-23T10:35:52.013 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:52.014+0000 7f6bbb085740 1 -- 172.21.15.97:0/629904187 shutdown_connections 2024-04-23T10:35:52.013 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:52.014+0000 7f6bbb085740 1 --2- 172.21.15.97:0/629904187 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55ebf9873800 0x55ebf98f0620 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:52.013 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:52.014+0000 7f6bbb085740 1 --2- 172.21.15.97:0/629904187 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ebf9868f40 0x55ebf98ec880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:52.013 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:52.014+0000 7f6bbb085740 1 --2- 172.21.15.97:0/629904187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ebf9868620 0x55ebf98e1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:52.014 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:52.014+0000 7f6b99ffb640 0 client.16983 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:35:52.014 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:52.014+0000 7f6bbb085740 1 --2- 172.21.15.97:0/629904187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ebf9864b30 0x55ebf98e19f0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:52.014 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:52.014+0000 7f6bbb085740 1 -- 172.21.15.97:0/629904187 >> 172.21.15.97:0/629904187 conn(0x55ebf9870ac0 msgr2=0x55ebf987f030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:35:52.014 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:52.014+0000 7f6bbb085740 1 -- 172.21.15.97:0/629904187 shutdown_connections 2024-04-23T10:35:52.014 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:52.014+0000 7f6bbb085740 1 -- 172.21.15.97:0/629904187 wait complete. 2024-04-23T10:35:52.014 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:52.014+0000 7f6bbb085740 20 client.16983 trim_cache size 0 max 16384 2024-04-23T10:35:52.014 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:35:52.014 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:35:52.014 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:35:52.014 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:35:52.015 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:52.014+0000 7f6bb8375640 1 -- 172.21.15.97:0/1131776658 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f6b9c0a3e40 msgr2=0x7f6b9c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:52.015 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:52.014+0000 7f6bb8375640 1 --2- 172.21.15.97:0/1131776658 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f6b9c0a3e40 0x7f6b9c0a6300 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6ba809c600 tx=0x7f6ba8087040 comp rx=0 tx=0).stop 2024-04-23T10:35:52.015 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:52.014+0000 7f6bb8375640 1 -- 172.21.15.97:0/1131776658 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6bb010c540 msgr2=0x7f6bb019cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:35:52.015 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:52.014+0000 7f6bb8375640 1 --2- 172.21.15.97:0/1131776658 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6bb010c540 0x7f6bb019cf90 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f6ba800eaa0 tx=0x7f6ba800eea0 comp rx=0 tx=0).stop 2024-04-23T10:35:52.015 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:52.015+0000 7f6bb8375640 1 -- 172.21.15.97:0/1131776658 shutdown_connections 2024-04-23T10:35:52.016 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:52.015+0000 7f6bb8375640 1 --2- 172.21.15.97:0/1131776658 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f6b9c0a3e40 0x7f6b9c0a6300 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:52.016 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:52.015+0000 7f6bb8375640 1 --2- 172.21.15.97:0/1131776658 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6bb010c540 0x7f6bb019cf90 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:52.016 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:52.015+0000 7f6bb8375640 1 --2- 172.21.15.97:0/1131776658 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6bb0103b20 0x7f6bb0198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:52.016 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:52.015+0000 7f6bb8375640 1 --2- 172.21.15.97:0/1131776658 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6bb0103200 0x7f6bb0198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:35:52.016 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:52.015+0000 7f6bb8375640 1 -- 172.21.15.97:0/1131776658 >> 172.21.15.97:0/1131776658 conn(0x7f6bb0077fd0 msgr2=0x7f6bb00ff0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:35:52.016 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:52.015+0000 7f6bb8375640 1 -- 172.21.15.97:0/1131776658 shutdown_connections 2024-04-23T10:35:52.016 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:35:52.015+0000 7f6bb8375640 1 -- 172.21.15.97:0/1131776658 wait complete. 2024-04-23T10:35:52.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:35:52 smithi097 ceph-mon[29197]: pgmap v299: 97 pgs: 97 active+clean; 5.6 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-04-23T10:35:52.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:35:52 smithi183 ceph-mon[32317]: pgmap v299: 97 pgs: 97 active+clean; 5.6 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-04-23T10:35:52.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:35:52 smithi196 ceph-mon[32919]: pgmap v299: 97 pgs: 97 active+clean; 5.6 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-04-23T10:35:54.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:35:54 smithi183 ceph-mon[32317]: pgmap v300: 97 pgs: 97 active+clean; 1.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 56 KiB/s wr, 25 op/s 2024-04-23T10:35:54.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:35:54 smithi196 ceph-mon[32919]: pgmap v300: 97 pgs: 97 active+clean; 1.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 56 KiB/s wr, 25 op/s 2024-04-23T10:35:54.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:35:54 smithi097 ceph-mon[29197]: pgmap v300: 97 pgs: 97 active+clean; 1.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 56 KiB/s wr, 25 op/s 2024-04-23T10:35:56.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:35:56 smithi183 ceph-mon[32317]: pgmap v301: 97 pgs: 97 active+clean; 1.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-04-23T10:35:56.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:35:56 smithi196 ceph-mon[32919]: pgmap v301: 97 pgs: 97 active+clean; 1.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-04-23T10:35:56.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:35:56 smithi097 ceph-mon[29197]: pgmap v301: 97 pgs: 97 active+clean; 1.7 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-04-23T10:35:58.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:35:58 smithi183 ceph-mon[32317]: pgmap v302: 97 pgs: 97 active+clean; 1.7 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 33 KiB/s wr, 19 op/s 2024-04-23T10:35:58.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:35:58 smithi196 ceph-mon[32919]: pgmap v302: 97 pgs: 97 active+clean; 1.7 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 33 KiB/s wr, 19 op/s 2024-04-23T10:35:58.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:35:58 smithi097 ceph-mon[29197]: pgmap v302: 97 pgs: 97 active+clean; 1.7 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 33 KiB/s wr, 19 op/s 2024-04-23T10:36:00.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:36:00 smithi183 ceph-mon[32317]: pgmap v303: 97 pgs: 97 active+clean; 1.7 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-23T10:36:00.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:36:00 smithi196 ceph-mon[32919]: pgmap v303: 97 pgs: 97 active+clean; 1.7 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-23T10:36:00.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:36:00 smithi097 ceph-mon[29197]: pgmap v303: 97 pgs: 97 active+clean; 1.7 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-23T10:36:02.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:36:02 smithi183 ceph-mon[32317]: pgmap v304: 97 pgs: 97 active+clean; 1.7 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-23T10:36:02.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:36:02 smithi196 ceph-mon[32919]: pgmap v304: 97 pgs: 97 active+clean; 1.7 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-23T10:36:02.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:36:02 smithi097 ceph-mon[29197]: pgmap v304: 97 pgs: 97 active+clean; 1.7 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-23T10:36:04.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:36:04 smithi183 ceph-mon[32317]: pgmap v305: 97 pgs: 97 active+clean; 1.7 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 61 KiB/s wr, 34 op/s 2024-04-23T10:36:04.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:36:04 smithi196 ceph-mon[32919]: pgmap v305: 97 pgs: 97 active+clean; 1.7 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 61 KiB/s wr, 34 op/s 2024-04-23T10:36:04.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:36:04 smithi097 ceph-mon[29197]: pgmap v305: 97 pgs: 97 active+clean; 1.7 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 61 KiB/s wr, 34 op/s 2024-04-23T10:36:05.111 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.110+0000 7f2e2c107640 1 -- 172.21.15.97:0/2472210179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e241062f0 msgr2=0x7f2e24106780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:05.111 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.110+0000 7f2e2c107640 1 --2- 172.21.15.97:0/2472210179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e241062f0 0x7f2e24106780 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c009ca0 tx=0x7f2e1c02a9e0 comp rx=0 tx=0).stop 2024-04-23T10:36:05.111 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.110+0000 7f2e2c107640 1 -- 172.21.15.97:0/2472210179 shutdown_connections 2024-04-23T10:36:05.111 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.110+0000 7f2e2c107640 1 --2- 172.21.15.97:0/2472210179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e241062f0 0x7f2e24106780 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:05.111 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.110+0000 7f2e2c107640 1 --2- 172.21.15.97:0/2472210179 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e241018f0 0x7f2e24105db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:05.111 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.110+0000 7f2e2c107640 1 --2- 172.21.15.97:0/2472210179 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e24100fd0 0x7f2e241013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:05.111 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.110+0000 7f2e2c107640 1 -- 172.21.15.97:0/2472210179 >> 172.21.15.97:0/2472210179 conn(0x7f2e24077fd0 msgr2=0x7f2e240783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:36:05.111 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.110+0000 7f2e2c107640 1 -- 172.21.15.97:0/2472210179 shutdown_connections 2024-04-23T10:36:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.110+0000 7f2e2c107640 1 -- 172.21.15.97:0/2472210179 wait complete. 2024-04-23T10:36:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.111+0000 7f2e2c107640 1 Processor -- start 2024-04-23T10:36:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.111+0000 7f2e2c107640 1 -- start start 2024-04-23T10:36:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.111+0000 7f2e2c107640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e24100fd0 0x7f2e241a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.111+0000 7f2e2c107640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e241018f0 0x7f2e241a4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.111+0000 7f2e2c107640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e241062f0 0x7f2e2419ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.111+0000 7f2e2c107640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2e241187b0 con 0x7f2e241062f0 2024-04-23T10:36:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.111+0000 7f2e2c107640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2e24118630 con 0x7f2e24100fd0 2024-04-23T10:36:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.111+0000 7f2e2c107640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2e24118930 con 0x7f2e241018f0 2024-04-23T10:36:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.111+0000 7f2e2a6a4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e241062f0 0x7f2e2419ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.111+0000 7f2e2a6a4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e241062f0 0x7f2e2419ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35822/0 (socket says 172.21.15.97:35822) 2024-04-23T10:36:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.111+0000 7f2e2a6a4640 1 -- 172.21.15.97:0/3227788054 learned_addr learned my addr 172.21.15.97:0/3227788054 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:36:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.111+0000 7f2e296a2640 1 --2- 172.21.15.97:0/3227788054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e241018f0 0x7f2e241a4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.111+0000 7f2e29ea3640 1 --2- 172.21.15.97:0/3227788054 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e24100fd0 0x7f2e241a48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.111+0000 7f2e2a6a4640 1 -- 172.21.15.97:0/3227788054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e241018f0 msgr2=0x7f2e241a4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.111+0000 7f2e2a6a4640 1 --2- 172.21.15.97:0/3227788054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e241018f0 0x7f2e241a4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.111+0000 7f2e2a6a4640 1 -- 172.21.15.97:0/3227788054 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e24100fd0 msgr2=0x7f2e241a48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.111+0000 7f2e2a6a4640 1 --2- 172.21.15.97:0/3227788054 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e24100fd0 0x7f2e241a48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.111+0000 7f2e2a6a4640 1 -- 172.21.15.97:0/3227788054 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e2419f1f0 con 0x7f2e241062f0 2024-04-23T10:36:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.112+0000 7f2e2a6a4640 1 --2- 172.21.15.97:0/3227788054 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e241062f0 0x7f2e2419ea70 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c006010 tx=0x7f2e1c02fbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:36:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.112+0000 7f2e1affd640 1 -- 172.21.15.97:0/3227788054 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c033040 con 0x7f2e241062f0 2024-04-23T10:36:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.112+0000 7f2e1affd640 1 -- 172.21.15.97:0/3227788054 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e1c02e7a0 con 0x7f2e241062f0 2024-04-23T10:36:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.112+0000 7f2e2c107640 1 -- 172.21.15.97:0/3227788054 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e2419f480 con 0x7f2e241062f0 2024-04-23T10:36:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.112+0000 7f2e1affd640 1 -- 172.21.15.97:0/3227788054 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c043720 con 0x7f2e241062f0 2024-04-23T10:36:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.112+0000 7f2e2c107640 1 -- 172.21.15.97:0/3227788054 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e241ab780 con 0x7f2e241062f0 2024-04-23T10:36:05.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.114+0000 7f2e1affd640 1 -- 172.21.15.97:0/3227788054 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2e1c036070 con 0x7f2e241062f0 2024-04-23T10:36:05.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.114+0000 7f2e2ee1b740 1 Processor -- start 2024-04-23T10:36:05.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.114+0000 7f2e2ee1b740 1 -- start start 2024-04-23T10:36:05.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.115+0000 7f2e2ee1b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5605f108a620 0x5605f108aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:05.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.115+0000 7f2e2ee1b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605f108af40 0x5605f10952c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:05.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.115+0000 7f2e1affd640 1 --2- 172.21.15.97:0/3227788054 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f2e100a3ec0 0x7f2e100a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:05.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.115+0000 7f2e2ee1b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605f1095800 0x5605f1097bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:05.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.115+0000 7f2e2ee1b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5605f0fef970 con 0x5605f1095800 2024-04-23T10:36:05.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.115+0000 7f2e2ee1b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5605f0f8c4a0 con 0x5605f108af40 2024-04-23T10:36:05.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.115+0000 7f2e2ee1b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5605f0feb930 con 0x5605f108a620 2024-04-23T10:36:05.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.115+0000 7f2e296a2640 1 --2- 172.21.15.97:0/3227788054 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f2e100a3ec0 0x7f2e100a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:05.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.115+0000 7f2e2a6a4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605f1095800 0x5605f1097bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:05.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.115+0000 7f2e2a6a4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605f1095800 0x5605f1097bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35836/0 (socket says 172.21.15.97:35836) 2024-04-23T10:36:05.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.115+0000 7f2e2a6a4640 1 -- 172.21.15.97:0/1641662209 learned_addr learned my addr 172.21.15.97:0/1641662209 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:36:05.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.115+0000 7f2e296a2640 1 --2- 172.21.15.97:0/1641662209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5605f108a620 0x5605f108aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:05.121 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.115+0000 7f2e29ea3640 1 --2- 172.21.15.97:0/1641662209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605f108af40 0x5605f10952c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:05.121 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.115+0000 7f2e1affd640 1 -- 172.21.15.97:0/3227788054 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8484+0+0 (secure 0 0 0) 0x7f2e1c0f14c0 con 0x7f2e241062f0 2024-04-23T10:36:05.121 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.115+0000 7f2e2a6a4640 1 -- 172.21.15.97:0/1641662209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5605f108a620 msgr2=0x5605f108aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:36:05.121 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.115+0000 7f2e2a6a4640 1 --2- 172.21.15.97:0/1641662209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5605f108a620 0x5605f108aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:05.121 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.115+0000 7f2e2a6a4640 1 -- 172.21.15.97:0/1641662209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605f108af40 msgr2=0x5605f10952c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:36:05.121 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.115+0000 7f2e2a6a4640 1 --2- 172.21.15.97:0/1641662209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605f108af40 0x5605f10952c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:05.121 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.115+0000 7f2e2a6a4640 1 -- 172.21.15.97:0/1641662209 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605f0d02210 con 0x5605f1095800 2024-04-23T10:36:05.121 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.115+0000 7f2e296a2640 1 --2- 172.21.15.97:0/3227788054 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f2e100a3ec0 0x7f2e100a6380 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2e240787b0 tx=0x7f2e20009ad0 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:36:05.121 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.116+0000 7f2e2a6a4640 1 --2- 172.21.15.97:0/1641662209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605f1095800 0x5605f1097bf0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c0b9930 tx=0x7f2e1c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b926e1a2eab66d3a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:36:05.121 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.116+0000 7f2e0ffff640 1 -- 172.21.15.97:0/1641662209 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c0bd010 con 0x5605f1095800 2024-04-23T10:36:05.121 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.116+0000 7f2e0ffff640 1 -- 172.21.15.97:0/1641662209 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e1c0bd4d0 con 0x5605f1095800 2024-04-23T10:36:05.121 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.116+0000 7f2e0ffff640 1 -- 172.21.15.97:0/1641662209 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c086260 con 0x5605f1095800 2024-04-23T10:36:05.121 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.116+0000 7f2e2ee1b740 1 -- 172.21.15.97:0/1641662209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605f1095800 msgr2=0x5605f1097bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:05.121 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.116+0000 7f2e2ee1b740 1 --2- 172.21.15.97:0/1641662209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605f1095800 0x5605f1097bf0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c0b9930 tx=0x7f2e1c0095c0 comp rx=0 tx=0).stop 2024-04-23T10:36:05.122 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.116+0000 7f2e2ee1b740 1 -- 172.21.15.97:0/1641662209 shutdown_connections 2024-04-23T10:36:05.122 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.116+0000 7f2e2ee1b740 1 --2- 172.21.15.97:0/1641662209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605f1095800 0x5605f1097bf0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:05.122 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.116+0000 7f2e2ee1b740 1 --2- 172.21.15.97:0/1641662209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605f108af40 0x5605f10952c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:05.122 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.116+0000 7f2e2ee1b740 1 --2- 172.21.15.97:0/1641662209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5605f108a620 0x5605f108aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:05.122 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.116+0000 7f2e2ee1b740 1 -- 172.21.15.97:0/1641662209 >> 172.21.15.97:0/1641662209 conn(0x5605f1086b30 msgr2=0x5605f1086f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:36:05.122 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.117+0000 7f2e2ee1b740 1 -- 172.21.15.97:0/1641662209 shutdown_connections 2024-04-23T10:36:05.122 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.117+0000 7f2e2ee1b740 1 -- 172.21.15.97:0/1641662209 wait complete. 2024-04-23T10:36:05.122 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.117+0000 7f2e2ee1b740 1 -- start start 2024-04-23T10:36:05.122 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.117+0000 7f2e2ee1b740 1 Processor -- start 2024-04-23T10:36:05.122 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.117+0000 7f2e2ee1b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605f108a620 0x5605f1095180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:05.122 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.117+0000 7f2e2ee1b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5605f108af40 0x5605f1092470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:05.122 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.117+0000 7f2e2ee1b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605f1095800 0x5605f10929f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:05.122 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.117+0000 7f2e2ee1b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5605f0fef970 con 0x5605f108a620 2024-04-23T10:36:05.122 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.117+0000 7f2e2ee1b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5605f0f8a0b0 con 0x5605f1095800 2024-04-23T10:36:05.122 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.117+0000 7f2e2ee1b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5605f10981f0 con 0x5605f108af40 2024-04-23T10:36:05.123 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.117+0000 7f2e2ee1b740 10 client.0 _ll_register_callbacks cb 0x5605f0f77d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:36:05.123 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.118+0000 7f2e29ea3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605f108a620 0x5605f1095180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:05.123 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.118+0000 7f2e29ea3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605f108a620 0x5605f1095180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35844/0 (socket says 172.21.15.97:35844) 2024-04-23T10:36:05.123 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.118+0000 7f2e29ea3640 1 -- 172.21.15.97:0/3984965862 learned_addr learned my addr 172.21.15.97:0/3984965862 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:36:05.123 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.118+0000 7f2e296a2640 1 --2- 172.21.15.97:0/3984965862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5605f108af40 0x5605f1092470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:05.123 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.118+0000 7f2e2a6a4640 1 --2- 172.21.15.97:0/3984965862 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605f1095800 0x5605f10929f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:05.123 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.118+0000 7f2e29ea3640 1 -- 172.21.15.97:0/3984965862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5605f108af40 msgr2=0x5605f1092470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:36:05.123 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.118+0000 7f2e29ea3640 1 --2- 172.21.15.97:0/3984965862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5605f108af40 0x5605f1092470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:05.123 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.118+0000 7f2e29ea3640 1 -- 172.21.15.97:0/3984965862 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605f1095800 msgr2=0x5605f10929f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:36:05.123 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.118+0000 7f2e29ea3640 1 --2- 172.21.15.97:0/3984965862 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605f1095800 0x5605f10929f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:05.123 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.118+0000 7f2e29ea3640 1 -- 172.21.15.97:0/3984965862 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605f0dce620 con 0x5605f108a620 2024-04-23T10:36:05.123 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.118+0000 7f2e29ea3640 1 --2- 172.21.15.97:0/3984965862 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605f108a620 0x5605f1095180 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f2e1400bad0 tx=0x7f2e1400bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35c393e41f37265a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:36:05.123 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.118+0000 7f2e0d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:36:05.123 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.118+0000 7f2e0d7fa640 1 -- 172.21.15.97:0/3984965862 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e14005020 con 0x5605f108a620 2024-04-23T10:36:05.123 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.118+0000 7f2e0d7fa640 1 -- 172.21.15.97:0/3984965862 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8484+0+0 (secure 0 0 0) 0x7f2e140051c0 con 0x5605f108a620 2024-04-23T10:36:05.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.118+0000 7f2e2ee1b740 1 -- 172.21.15.97:0/3984965862 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e1c0bd4d0 con 0x5605f108a620 2024-04-23T10:36:05.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.118+0000 7f2e0d7fa640 1 -- 172.21.15.97:0/3984965862 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2e140053c0 con 0x5605f108a620 2024-04-23T10:36:05.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.118+0000 7f2e2ee1b740 10 client.16995 fetch_fsmap learned FSMap version 26 2024-04-23T10:36:05.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.118+0000 7f2e2ee1b740 1 -- 172.21.15.97:0/3984965862 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5605f10a3f50 con 0x5605f108a620 2024-04-23T10:36:05.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.119+0000 7f2e0d7fa640 1 -- 172.21.15.97:0/3984965862 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e1c0bd4d0 con 0x5605f108a620 2024-04-23T10:36:05.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.119+0000 7f2e0d7fa640 1 -- 172.21.15.97:0/3984965862 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f2e14022020 con 0x5605f108a620 2024-04-23T10:36:05.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.119+0000 7f2e2ee1b740 10 client.16995 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:36:05.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.119+0000 7f2e2ee1b740 10 client.16995 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:36:05.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.119+0000 7f2e2ee1b740 20 client.16995 populate_metadata read hostname 'smithi097' 2024-04-23T10:36:05.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.119+0000 7f2e2ee1b740 1 --2- 172.21.15.97:0/3984965862 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5605f10a5410 0x5605f10a7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:05.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.119+0000 7f2e2ee1b740 4 client.16995 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:36:05.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.119+0000 7f2e2ee1b740 1 -- 172.21.15.97:0/3984965862 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5605f0ffd390 con 0x5605f10a5410 2024-04-23T10:36:05.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.119+0000 7f2e296a2640 1 --2- 172.21.15.97:0/3984965862 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5605f10a5410 0x5605f10a7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:05.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.120+0000 7f2e0d7fa640 10 client.16995 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:36:05.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.120+0000 7f2e296a2640 1 --2- 172.21.15.97:0/3984965862 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5605f10a5410 0x5605f10a7800 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c90b498ce3ede2e server_cookie=ef64e8b75644f33d in_seq=0 out_seq=0 2024-04-23T10:36:05.124 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.120+0000 7f2e0d7fa640 1 -- 172.21.15.97:0/3984965862 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5605f0ffd390 con 0x5605f10a5410 2024-04-23T10:36:05.125 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.120+0000 7f2e0d7fa640 10 client.16995 handle_command_reply: tid=0 2024-04-23T10:36:05.125 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.120+0000 7f2e2ee1b740 1 client.16995 shutdown 2024-04-23T10:36:05.125 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.120+0000 7f2e2ee1b740 1 -- 172.21.15.97:0/3984965862 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605f108a620 msgr2=0x5605f1095180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:05.125 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.120+0000 7f2e2ee1b740 1 --2- 172.21.15.97:0/3984965862 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605f108a620 0x5605f1095180 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f2e1400bad0 tx=0x7f2e1400bf90 comp rx=0 tx=0).stop 2024-04-23T10:36:05.125 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.120+0000 7f2e2ee1b740 1 -- 172.21.15.97:0/3984965862 shutdown_connections 2024-04-23T10:36:05.125 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.120+0000 7f2e2ee1b740 1 --2- 172.21.15.97:0/3984965862 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5605f10a5410 0x5605f10a7800 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:05.125 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.120+0000 7f2e2ee1b740 1 --2- 172.21.15.97:0/3984965862 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605f1095800 0x5605f10929f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:05.125 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.121+0000 7f2e0d7fa640 0 client.16995 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:36:05.125 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.120+0000 7f2e2ee1b740 1 --2- 172.21.15.97:0/3984965862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5605f108af40 0x5605f1092470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:05.125 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.121+0000 7f2e2ee1b740 1 --2- 172.21.15.97:0/3984965862 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605f108a620 0x5605f1095180 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:05.125 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.121+0000 7f2e2ee1b740 1 -- 172.21.15.97:0/3984965862 >> 172.21.15.97:0/3984965862 conn(0x5605f1086b30 msgr2=0x5605f108c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:36:05.125 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.121+0000 7f2e2ee1b740 1 -- 172.21.15.97:0/3984965862 shutdown_connections 2024-04-23T10:36:05.125 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.121+0000 7f2e2ee1b740 1 -- 172.21.15.97:0/3984965862 wait complete. 2024-04-23T10:36:05.125 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.121+0000 7f2e2ee1b740 20 client.16995 trim_cache size 0 max 16384 2024-04-23T10:36:05.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.135+0000 7f2e2ee1b740 1 Processor -- start 2024-04-23T10:36:05.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.136+0000 7f2e2ee1b740 1 -- start start 2024-04-23T10:36:05.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.136+0000 7f2e2ee1b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605f108a620 0x5605f1085b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:05.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.136+0000 7f2e2ee1b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605f108af40 0x5605f10ad0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:05.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.136+0000 7f2e2ee1b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5605f1095800 0x5605f10ad630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:05.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.136+0000 7f2e2ee1b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2e241b40d0 con 0x5605f108a620 2024-04-23T10:36:05.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.136+0000 7f2e2ee1b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2e241b5c10 con 0x5605f108af40 2024-04-23T10:36:05.139 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.136+0000 7f2e2ee1b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2e241b47a0 con 0x5605f1095800 2024-04-23T10:36:05.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.136+0000 7f2e29ea3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605f108a620 0x5605f1085b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:05.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.136+0000 7f2e29ea3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605f108a620 0x5605f1085b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35846/0 (socket says 172.21.15.97:35846) 2024-04-23T10:36:05.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.136+0000 7f2e29ea3640 1 -- 172.21.15.97:0/896446472 learned_addr learned my addr 172.21.15.97:0/896446472 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:36:05.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.136+0000 7f2e2a6a4640 1 --2- 172.21.15.97:0/896446472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5605f1095800 0x5605f10ad630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:05.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.136+0000 7f2e296a2640 1 --2- 172.21.15.97:0/896446472 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605f108af40 0x5605f10ad0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:05.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.136+0000 7f2e29ea3640 1 -- 172.21.15.97:0/896446472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5605f1095800 msgr2=0x5605f10ad630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:36:05.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.136+0000 7f2e29ea3640 1 --2- 172.21.15.97:0/896446472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5605f1095800 0x5605f10ad630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:05.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.136+0000 7f2e29ea3640 1 -- 172.21.15.97:0/896446472 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605f108af40 msgr2=0x5605f10ad0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:36:05.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.136+0000 7f2e29ea3640 1 --2- 172.21.15.97:0/896446472 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605f108af40 0x5605f10ad0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:05.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.136+0000 7f2e29ea3640 1 -- 172.21.15.97:0/896446472 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605f10a7e60 con 0x5605f108a620 2024-04-23T10:36:05.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.136+0000 7f2e296a2640 1 --2- 172.21.15.97:0/896446472 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605f108af40 0x5605f10ad0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:36:05.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.136+0000 7f2e29ea3640 1 --2- 172.21.15.97:0/896446472 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605f108a620 0x5605f1085b30 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f2e14004240 tx=0x7f2e140058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9be0364d4a9bc656 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:36:05.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.136+0000 7f2e0f7fe640 1 -- 172.21.15.97:0/896446472 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e14005560 con 0x5605f108a620 2024-04-23T10:36:05.140 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.136+0000 7f2e0f7fe640 1 -- 172.21.15.97:0/896446472 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e1401bd00 con 0x5605f108a620 2024-04-23T10:36:05.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.136+0000 7f2e0f7fe640 1 -- 172.21.15.97:0/896446472 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e14024aa0 con 0x5605f108a620 2024-04-23T10:36:05.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.137+0000 7f2e2ee1b740 1 -- 172.21.15.97:0/896446472 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605f108a620 msgr2=0x5605f1085b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:05.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.137+0000 7f2e2ee1b740 1 --2- 172.21.15.97:0/896446472 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605f108a620 0x5605f1085b30 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f2e14004240 tx=0x7f2e140058d0 comp rx=0 tx=0).stop 2024-04-23T10:36:05.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.137+0000 7f2e2ee1b740 1 -- 172.21.15.97:0/896446472 shutdown_connections 2024-04-23T10:36:05.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.137+0000 7f2e2ee1b740 1 --2- 172.21.15.97:0/896446472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5605f1095800 0x5605f10ad630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:05.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.137+0000 7f2e2ee1b740 1 --2- 172.21.15.97:0/896446472 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605f108af40 0x5605f10ad0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:05.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.137+0000 7f2e2ee1b740 1 --2- 172.21.15.97:0/896446472 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605f108a620 0x5605f1085b30 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:05.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.137+0000 7f2e2ee1b740 1 -- 172.21.15.97:0/896446472 >> 172.21.15.97:0/896446472 conn(0x5605f1086b30 msgr2=0x5605f1085420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:36:05.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.137+0000 7f2e2ee1b740 1 -- 172.21.15.97:0/896446472 shutdown_connections 2024-04-23T10:36:05.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.137+0000 7f2e2ee1b740 1 -- 172.21.15.97:0/896446472 wait complete. 2024-04-23T10:36:05.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.137+0000 7f2e2ee1b740 1 -- start start 2024-04-23T10:36:05.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.137+0000 7f2e2ee1b740 1 Processor -- start 2024-04-23T10:36:05.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.138+0000 7f2e2ee1b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605f108a620 0x5605f109d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:05.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.138+0000 7f2e2ee1b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605f108af40 0x5605f109a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:05.141 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.138+0000 7f2e2ee1b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5605f1095800 0x5605f109aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:05.142 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.138+0000 7f2e2ee1b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2e241b5540 con 0x5605f108a620 2024-04-23T10:36:05.142 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.138+0000 7f2e2ee1b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2e241b4e70 con 0x5605f108af40 2024-04-23T10:36:05.142 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.138+0000 7f2e2ee1b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f2e241b20d0 con 0x5605f1095800 2024-04-23T10:36:05.142 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.138+0000 7f2e2ee1b740 10 client.0 _ll_register_callbacks cb 0x7f2e14004440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:36:05.142 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.138+0000 7f2e29ea3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605f108a620 0x5605f109d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:05.142 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.138+0000 7f2e29ea3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605f108a620 0x5605f109d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:35856/0 (socket says 172.21.15.97:35856) 2024-04-23T10:36:05.142 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.138+0000 7f2e29ea3640 1 -- 172.21.15.97:0/84492471 learned_addr learned my addr 172.21.15.97:0/84492471 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:36:05.142 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.138+0000 7f2e296a2640 1 --2- 172.21.15.97:0/84492471 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605f108af40 0x5605f109a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:05.142 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.138+0000 7f2e29ea3640 1 -- 172.21.15.97:0/84492471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5605f1095800 msgr2=0x5605f109aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:36:05.142 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.138+0000 7f2e2a6a4640 1 --2- 172.21.15.97:0/84492471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5605f1095800 0x5605f109aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:05.142 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.138+0000 7f2e29ea3640 1 --2- 172.21.15.97:0/84492471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5605f1095800 0x5605f109aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:05.142 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.138+0000 7f2e29ea3640 1 -- 172.21.15.97:0/84492471 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605f108af40 msgr2=0x5605f109a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:36:05.142 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.138+0000 7f2e29ea3640 1 --2- 172.21.15.97:0/84492471 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605f108af40 0x5605f109a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:05.142 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.138+0000 7f2e29ea3640 1 -- 172.21.15.97:0/84492471 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605f109b4d0 con 0x5605f108a620 2024-04-23T10:36:05.143 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.138+0000 7f2e2a6a4640 1 --2- 172.21.15.97:0/84492471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5605f1095800 0x5605f109aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:36:05.143 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.138+0000 7f2e296a2640 1 --2- 172.21.15.97:0/84492471 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605f108af40 0x5605f109a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:36:05.143 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.138+0000 7f2e29ea3640 1 --2- 172.21.15.97:0/84492471 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605f108a620 0x5605f109d570 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f2e1401cf40 tx=0x7f2e1401cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f972ce125aa6dadb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:36:05.143 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.138+0000 7f2e0d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:36:05.143 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.139+0000 7f2e0d7fa640 1 -- 172.21.15.97:0/84492471 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e14005620 con 0x5605f108a620 2024-04-23T10:36:05.143 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.139+0000 7f2e0d7fa640 1 -- 172.21.15.97:0/84492471 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8484+0+0 (secure 0 0 0) 0x7f2e1401b7d0 con 0x5605f108a620 2024-04-23T10:36:05.143 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.139+0000 7f2e2ee1b740 1 -- 172.21.15.97:0/84492471 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e1401bd00 con 0x5605f108a620 2024-04-23T10:36:05.143 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.139+0000 7f2e0d7fa640 1 -- 172.21.15.97:0/84492471 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2e1401b9d0 con 0x5605f108a620 2024-04-23T10:36:05.143 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.139+0000 7f2e2ee1b740 10 client.17001 fetch_fsmap learned FSMap version 26 2024-04-23T10:36:05.143 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.139+0000 7f2e2ee1b740 1 -- 172.21.15.97:0/84492471 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5605f110b6f0 con 0x5605f108a620 2024-04-23T10:36:05.143 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.139+0000 7f2e0d7fa640 1 -- 172.21.15.97:0/84492471 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e1401bd00 con 0x5605f108a620 2024-04-23T10:36:05.143 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.139+0000 7f2e0d7fa640 1 -- 172.21.15.97:0/84492471 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f2e1403e070 con 0x5605f108a620 2024-04-23T10:36:05.143 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.139+0000 7f2e2ee1b740 10 client.17001 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:36:05.143 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.139+0000 7f2e2ee1b740 10 client.17001 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:36:05.143 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.139+0000 7f2e2ee1b740 20 client.17001 populate_metadata read hostname 'smithi097' 2024-04-23T10:36:05.143 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.139+0000 7f2e2ee1b740 1 --2- 172.21.15.97:0/84492471 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5605f110bdc0 0x5605f110e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.139+0000 7f2e2ee1b740 4 client.17001 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:36:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.139+0000 7f2e2ee1b740 1 -- 172.21.15.97:0/84492471 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5605f0ff7200 con 0x5605f110bdc0 2024-04-23T10:36:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.140+0000 7f2e296a2640 1 --2- 172.21.15.97:0/84492471 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5605f110bdc0 0x5605f110e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.140+0000 7f2e0d7fa640 10 client.17001 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:36:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.140+0000 7f2e296a2640 1 --2- 172.21.15.97:0/84492471 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5605f110bdc0 0x5605f110e1e0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f503d04bc6ba4a1 server_cookie=7354e272466bcb25 in_seq=0 out_seq=0 2024-04-23T10:36:05.153 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.154+0000 7f2e0d7fa640 1 -- 172.21.15.97:0/84492471 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5605f0ff7200 con 0x5605f110bdc0 2024-04-23T10:36:05.153 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.154+0000 7f2e0d7fa640 10 client.17001 handle_command_reply: tid=0 2024-04-23T10:36:05.153 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.154+0000 7f2e2ee1b740 1 client.17001 shutdown 2024-04-23T10:36:05.153 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.154+0000 7f2e2ee1b740 1 -- 172.21.15.97:0/84492471 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605f108a620 msgr2=0x5605f109d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:05.153 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.154+0000 7f2e2ee1b740 1 --2- 172.21.15.97:0/84492471 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605f108a620 0x5605f109d570 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f2e1401cf40 tx=0x7f2e1401cf70 comp rx=0 tx=0).stop 2024-04-23T10:36:05.153 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.154+0000 7f2e2ee1b740 1 -- 172.21.15.97:0/84492471 shutdown_connections 2024-04-23T10:36:05.154 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.154+0000 7f2e2ee1b740 1 --2- 172.21.15.97:0/84492471 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5605f110bdc0 0x5605f110e1e0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:05.154 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.154+0000 7f2e2ee1b740 1 --2- 172.21.15.97:0/84492471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5605f1095800 0x5605f109aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:05.154 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.154+0000 7f2e2ee1b740 1 --2- 172.21.15.97:0/84492471 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5605f108af40 0x5605f109a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:05.154 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.154+0000 7f2e0d7fa640 0 client.17001 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:36:05.154 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.154+0000 7f2e2ee1b740 1 --2- 172.21.15.97:0/84492471 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5605f108a620 0x5605f109d570 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:05.154 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.154+0000 7f2e2ee1b740 1 -- 172.21.15.97:0/84492471 >> 172.21.15.97:0/84492471 conn(0x5605f1086b30 msgr2=0x5605f1085cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:36:05.154 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.154+0000 7f2e2ee1b740 1 -- 172.21.15.97:0/84492471 shutdown_connections 2024-04-23T10:36:05.154 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.154+0000 7f2e2ee1b740 1 -- 172.21.15.97:0/84492471 wait complete. 2024-04-23T10:36:05.154 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.154+0000 7f2e2ee1b740 20 client.17001 trim_cache size 0 max 16384 2024-04-23T10:36:05.154 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:36:05.155 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:36:05.155 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:36:05.155 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:36:05.156 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.155+0000 7f2e2c107640 1 -- 172.21.15.97:0/3227788054 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f2e100a3ec0 msgr2=0x7f2e100a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:05.156 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.155+0000 7f2e2c107640 1 --2- 172.21.15.97:0/3227788054 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f2e100a3ec0 0x7f2e100a6380 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2e240787b0 tx=0x7f2e20009ad0 comp rx=0 tx=0).stop 2024-04-23T10:36:05.156 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.155+0000 7f2e2c107640 1 -- 172.21.15.97:0/3227788054 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e241062f0 msgr2=0x7f2e2419ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:05.156 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.155+0000 7f2e2c107640 1 --2- 172.21.15.97:0/3227788054 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e241062f0 0x7f2e2419ea70 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c006010 tx=0x7f2e1c02fbe0 comp rx=0 tx=0).stop 2024-04-23T10:36:05.156 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.155+0000 7f2e2c107640 1 -- 172.21.15.97:0/3227788054 shutdown_connections 2024-04-23T10:36:05.156 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.155+0000 7f2e2c107640 1 --2- 172.21.15.97:0/3227788054 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f2e100a3ec0 0x7f2e100a6380 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:05.156 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.155+0000 7f2e2c107640 1 --2- 172.21.15.97:0/3227788054 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e241062f0 0x7f2e2419ea70 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:05.156 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.155+0000 7f2e2c107640 1 --2- 172.21.15.97:0/3227788054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e241018f0 0x7f2e241a4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:05.156 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.155+0000 7f2e2c107640 1 --2- 172.21.15.97:0/3227788054 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e24100fd0 0x7f2e241a48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:05.156 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.155+0000 7f2e2c107640 1 -- 172.21.15.97:0/3227788054 >> 172.21.15.97:0/3227788054 conn(0x7f2e24077fd0 msgr2=0x7f2e24103d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:36:05.156 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.155+0000 7f2e2c107640 1 -- 172.21.15.97:0/3227788054 shutdown_connections 2024-04-23T10:36:05.156 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:05.155+0000 7f2e2c107640 1 -- 172.21.15.97:0/3227788054 wait complete. 2024-04-23T10:36:06.415 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T10:36:06.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.526+0000 7f7a6aaef640 1 -- 172.21.15.97:0/1237711132 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a64102bf0 msgr2=0x7f7a64106ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:06.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.526+0000 7f7a6aaef640 1 --2- 172.21.15.97:0/1237711132 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a64102bf0 0x7f7a64106ba0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f7a50009900 tx=0x7f7a5002a410 comp rx=0 tx=0).stop 2024-04-23T10:36:06.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.526+0000 7f7a6aaef640 1 -- 172.21.15.97:0/1237711132 shutdown_connections 2024-04-23T10:36:06.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.526+0000 7f7a6aaef640 1 --2- 172.21.15.97:0/1237711132 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a64107280 0x7f7a6410ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:06.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.526+0000 7f7a6aaef640 1 --2- 172.21.15.97:0/1237711132 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a64102bf0 0x7f7a64106ba0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:06.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.526+0000 7f7a6aaef640 1 --2- 172.21.15.97:0/1237711132 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a64102240 0x7f7a64102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:06.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.526+0000 7f7a6aaef640 1 -- 172.21.15.97:0/1237711132 >> 172.21.15.97:0/1237711132 conn(0x7f7a640790e0 msgr2=0x7f7a640794f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:36:06.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.526+0000 7f7a6aaef640 1 -- 172.21.15.97:0/1237711132 shutdown_connections 2024-04-23T10:36:06.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.526+0000 7f7a6aaef640 1 -- 172.21.15.97:0/1237711132 wait complete. 2024-04-23T10:36:06.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.526+0000 7f7a6aaef640 1 Processor -- start 2024-04-23T10:36:06.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.526+0000 7f7a6aaef640 1 -- start start 2024-04-23T10:36:06.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.526+0000 7f7a6aaef640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a64102240 0x7f7a64198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:06.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.526+0000 7f7a6aaef640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a64107280 0x7f7a64198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:06.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.526+0000 7f7a6aaef640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a6419d020 0x7f7a6419d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:06.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.526+0000 7f7a6aaef640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7a641121c0 con 0x7f7a6419d020 2024-04-23T10:36:06.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.526+0000 7f7a6aaef640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7a64112040 con 0x7f7a64102240 2024-04-23T10:36:06.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.526+0000 7f7a6aaef640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7a64112340 con 0x7f7a64107280 2024-04-23T10:36:06.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.527+0000 7f7a6908c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a6419d020 0x7f7a6419d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:06.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.527+0000 7f7a6908c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a6419d020 0x7f7a6419d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39490/0 (socket says 172.21.15.97:39490) 2024-04-23T10:36:06.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.527+0000 7f7a6908c640 1 -- 172.21.15.97:0/789373665 learned_addr learned my addr 172.21.15.97:0/789373665 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:36:06.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.527+0000 7f7a63fff640 1 --2- 172.21.15.97:0/789373665 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a64107280 0x7f7a64198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:06.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.527+0000 7f7a6888b640 1 --2- 172.21.15.97:0/789373665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a64102240 0x7f7a64198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:06.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.527+0000 7f7a6908c640 1 -- 172.21.15.97:0/789373665 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a64107280 msgr2=0x7f7a64198d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:06.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.527+0000 7f7a6908c640 1 --2- 172.21.15.97:0/789373665 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a64107280 0x7f7a64198d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:06.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.527+0000 7f7a6908c640 1 -- 172.21.15.97:0/789373665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a64102240 msgr2=0x7f7a64198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:06.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.527+0000 7f7a6908c640 1 --2- 172.21.15.97:0/789373665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a64102240 0x7f7a64198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:06.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.527+0000 7f7a6908c640 1 -- 172.21.15.97:0/789373665 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a6419db10 con 0x7f7a6419d020 2024-04-23T10:36:06.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.527+0000 7f7a6888b640 1 --2- 172.21.15.97:0/789373665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a64102240 0x7f7a64198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:36:06.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.527+0000 7f7a6908c640 1 --2- 172.21.15.97:0/789373665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a6419d020 0x7f7a6419d4d0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c00cad0 tx=0x7f7a5c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:36:06.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.527+0000 7f7a61ffb640 1 -- 172.21.15.97:0/789373665 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c00ae20 con 0x7f7a6419d020 2024-04-23T10:36:06.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.527+0000 7f7a61ffb640 1 -- 172.21.15.97:0/789373665 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a5c005020 con 0x7f7a6419d020 2024-04-23T10:36:06.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.527+0000 7f7a6aaef640 1 -- 172.21.15.97:0/789373665 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a6419de00 con 0x7f7a6419d020 2024-04-23T10:36:06.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.527+0000 7f7a61ffb640 1 -- 172.21.15.97:0/789373665 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c00f6b0 con 0x7f7a6419d020 2024-04-23T10:36:06.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.528+0000 7f7a6aaef640 1 -- 172.21.15.97:0/789373665 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a641a55c0 con 0x7f7a6419d020 2024-04-23T10:36:06.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.528+0000 7f7a6aaef640 1 -- 172.21.15.97:0/789373665 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a640691c0 con 0x7f7a6419d020 2024-04-23T10:36:06.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.529+0000 7f7a61ffb640 1 -- 172.21.15.97:0/789373665 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7a5c0051d0 con 0x7f7a6419d020 2024-04-23T10:36:06.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.530+0000 7f7a61ffb640 1 --2- 172.21.15.97:0/789373665 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f7a540a3d20 0x7f7a540a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:06.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.530+0000 7f7a6888b640 1 --2- 172.21.15.97:0/789373665 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f7a540a3d20 0x7f7a540a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:06.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.530+0000 7f7a61ffb640 1 -- 172.21.15.97:0/789373665 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8528+0+0 (secure 0 0 0) 0x7f7a5c0d07e0 con 0x7f7a6419d020 2024-04-23T10:36:06.530 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.530+0000 7f7a6888b640 1 --2- 172.21.15.97:0/789373665 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f7a540a3d20 0x7f7a540a61e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7a58005fd0 tx=0x7f7a5800b5f0 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:36:06.533 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.535+0000 7f7a61ffb640 1 -- 172.21.15.97:0/789373665 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7a5c014030 con 0x7f7a6419d020 2024-04-23T10:36:06.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:36:06 smithi183 ceph-mon[32317]: pgmap v306: 97 pgs: 97 active+clean; 1.7 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 47 KiB/s wr, 25 op/s 2024-04-23T10:36:06.753 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.754+0000 7f7a6aaef640 1 -- 172.21.15.97:0/789373665 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7a64102bf0 con 0x7f7a6419d020 2024-04-23T10:36:06.754 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.755+0000 7f7a61ffb640 1 -- 172.21.15.97:0/789373665 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11931 (secure 0 0 0) 0x7f7a5c09ce80 con 0x7f7a6419d020 2024-04-23T10:36:06.754 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:36:06.754 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3946878762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3946878762},{"type":"v1","addr":"172.21.15.97:6835","nonce":3946878762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/209800861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":209800861},{"type":"v1","addr":"172.21.15.97:6837","nonce":209800861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T10:24:06.910134+0000","modified":"2024-04-23T10:26:09.820985+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6839/418013575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":418013575},{"type":"v1","addr":"172.21.15.97:6839","nonce":418013575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6841/747242680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":747242680},{"type":"v1","addr":"172.21.15.97:6841","nonce":747242680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6833/4292847121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4292847121},{"type":"v1","addr":"172.21.15.183:6833","nonce":4292847121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6833/3885816565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3885816565},{"type":"v1","addr":"172.21.15.196:6833","nonce":3885816565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1897093105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1897093105},{"type":"v1","addr":"172.21.15.183:6835","nonce":1897093105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1171477563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1171477563},{"type":"v1","addr":"172.21.15.196:6835","nonce":1171477563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1898341911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1898341911},{"type":"v1","addr":"172.21.15.183:6837","nonce":1898341911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.196:6837/3094946502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3094946502},{"type":"v1","addr":"172.21.15.196:6837","nonce":3094946502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.196:6839/83843556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":83843556},{"type":"v1","addr":"172.21.15.196:6839","nonce":83843556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6839/1763841130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1763841130},{"type":"v1","addr":"172.21.15.183:6839","nonce":1763841130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24487,24403,24398,24469,24488]},"id":1}]} 2024-04-23T10:36:06.755 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 26 2024-04-23T10:36:06.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.757+0000 7f7a6aaef640 1 -- 172.21.15.97:0/789373665 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f7a540a3d20 msgr2=0x7f7a540a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:06.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.757+0000 7f7a6aaef640 1 --2- 172.21.15.97:0/789373665 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f7a540a3d20 0x7f7a540a61e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7a58005fd0 tx=0x7f7a5800b5f0 comp rx=0 tx=0).stop 2024-04-23T10:36:06.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.757+0000 7f7a6aaef640 1 -- 172.21.15.97:0/789373665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a6419d020 msgr2=0x7f7a6419d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:06.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.757+0000 7f7a6aaef640 1 --2- 172.21.15.97:0/789373665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a6419d020 0x7f7a6419d4d0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c00cad0 tx=0x7f7a5c00cf90 comp rx=0 tx=0).stop 2024-04-23T10:36:06.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.757+0000 7f7a6aaef640 1 -- 172.21.15.97:0/789373665 shutdown_connections 2024-04-23T10:36:06.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.757+0000 7f7a6aaef640 1 --2- 172.21.15.97:0/789373665 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f7a540a3d20 0x7f7a540a61e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:06.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.757+0000 7f7a6aaef640 1 --2- 172.21.15.97:0/789373665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a6419d020 0x7f7a6419d4d0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:06.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.757+0000 7f7a6aaef640 1 --2- 172.21.15.97:0/789373665 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a64107280 0x7f7a64198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:06.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.757+0000 7f7a6aaef640 1 --2- 172.21.15.97:0/789373665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a64102240 0x7f7a64198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:06.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.757+0000 7f7a6aaef640 1 -- 172.21.15.97:0/789373665 >> 172.21.15.97:0/789373665 conn(0x7f7a640790e0 msgr2=0x7f7a64100400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:36:06.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.757+0000 7f7a6aaef640 1 -- 172.21.15.97:0/789373665 shutdown_connections 2024-04-23T10:36:06.757 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.757+0000 7f7a6aaef640 1 -- 172.21.15.97:0/789373665 wait complete. 2024-04-23T10:36:06.765 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:36:06 smithi097 ceph-mon[29197]: pgmap v306: 97 pgs: 97 active+clean; 1.7 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 47 KiB/s wr, 25 op/s 2024-04-23T10:36:06.767 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --set-id 32aeb117 --release --await' 2024-04-23T10:36:06.767 DEBUG:teuthology.orchestra.run.smithi097:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24488 quiesce db --set-id 32aeb117 --release --await 2024-04-23T10:36:06.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:36:06 smithi196 ceph-mon[32919]: pgmap v306: 97 pgs: 97 active+clean; 1.7 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 47 KiB/s wr, 25 op/s 2024-04-23T10:36:06.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.873+0000 7f9daad68640 1 -- 172.21.15.97:0/1436356261 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9da4103af0 msgr2=0x7f9da410bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:06.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.873+0000 7f9daad68640 1 --2- 172.21.15.97:0/1436356261 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9da4103af0 0x7f9da410bfd0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f9d90009900 tx=0x7f9d9002a510 comp rx=0 tx=0).stop 2024-04-23T10:36:06.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.873+0000 7f9daad68640 1 -- 172.21.15.97:0/1436356261 shutdown_connections 2024-04-23T10:36:06.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.873+0000 7f9daad68640 1 --2- 172.21.15.97:0/1436356261 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9da410c510 0x7f9da410e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:06.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.873+0000 7f9daad68640 1 --2- 172.21.15.97:0/1436356261 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9da4103af0 0x7f9da410bfd0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:06.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.873+0000 7f9daad68640 1 --2- 172.21.15.97:0/1436356261 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9da41031d0 0x7f9da41035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:06.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.873+0000 7f9daad68640 1 -- 172.21.15.97:0/1436356261 >> 172.21.15.97:0/1436356261 conn(0x7f9da4077f50 msgr2=0x7f9da4078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:36:06.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.873+0000 7f9daad68640 1 -- 172.21.15.97:0/1436356261 shutdown_connections 2024-04-23T10:36:06.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.873+0000 7f9daad68640 1 -- 172.21.15.97:0/1436356261 wait complete. 2024-04-23T10:36:06.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.873+0000 7f9daad68640 1 Processor -- start 2024-04-23T10:36:06.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.873+0000 7f9daad68640 1 -- start start 2024-04-23T10:36:06.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.874+0000 7f9daad68640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9da41031d0 0x7f9da419e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:06.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.874+0000 7f9daad68640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9da4103af0 0x7f9da419eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:06.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.874+0000 7f9daad68640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9da410c510 0x7f9da4198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:06.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.874+0000 7f9daad68640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9da4112270 con 0x7f9da410c510 2024-04-23T10:36:06.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.874+0000 7f9daad68640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9da41120f0 con 0x7f9da41031d0 2024-04-23T10:36:06.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.874+0000 7f9daad68640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9da41123f0 con 0x7f9da4103af0 2024-04-23T10:36:06.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.874+0000 7f9da9305640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9da410c510 0x7f9da4198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:06.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.874+0000 7f9da9305640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9da410c510 0x7f9da4198670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39508/0 (socket says 172.21.15.97:39508) 2024-04-23T10:36:06.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.874+0000 7f9da9305640 1 -- 172.21.15.97:0/1292208832 learned_addr learned my addr 172.21.15.97:0/1292208832 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:36:06.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.874+0000 7f9da3fff640 1 --2- 172.21.15.97:0/1292208832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9da4103af0 0x7f9da419eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:06.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.874+0000 7f9da8b04640 1 --2- 172.21.15.97:0/1292208832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9da41031d0 0x7f9da419e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:06.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.874+0000 7f9da9305640 1 -- 172.21.15.97:0/1292208832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9da4103af0 msgr2=0x7f9da419eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:06.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.874+0000 7f9da9305640 1 --2- 172.21.15.97:0/1292208832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9da4103af0 0x7f9da419eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:06.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.874+0000 7f9da9305640 1 -- 172.21.15.97:0/1292208832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9da41031d0 msgr2=0x7f9da419e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:06.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.874+0000 7f9da9305640 1 --2- 172.21.15.97:0/1292208832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9da41031d0 0x7f9da419e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:06.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.874+0000 7f9da9305640 1 -- 172.21.15.97:0/1292208832 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9da4198f30 con 0x7f9da410c510 2024-04-23T10:36:06.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.874+0000 7f9da3fff640 1 --2- 172.21.15.97:0/1292208832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9da4103af0 0x7f9da419eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:36:06.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.874+0000 7f9da9305640 1 --2- 172.21.15.97:0/1292208832 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9da410c510 0x7f9da4198670 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c00b970 tx=0x7f9d9c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:36:06.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.874+0000 7f9da1ffb640 1 -- 172.21.15.97:0/1292208832 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c019070 con 0x7f9da410c510 2024-04-23T10:36:06.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.874+0000 7f9daad68640 1 -- 172.21.15.97:0/1292208832 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9da4199220 con 0x7f9da410c510 2024-04-23T10:36:06.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.874+0000 7f9da1ffb640 1 -- 172.21.15.97:0/1292208832 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d9c014440 con 0x7f9da410c510 2024-04-23T10:36:06.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.874+0000 7f9da1ffb640 1 -- 172.21.15.97:0/1292208832 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c0075d0 con 0x7f9da410c510 2024-04-23T10:36:06.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.875+0000 7f9daad68640 1 -- 172.21.15.97:0/1292208832 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9da41a5530 con 0x7f9da410c510 2024-04-23T10:36:06.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.877+0000 7f9da1ffb640 1 -- 172.21.15.97:0/1292208832 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9d9c01c070 con 0x7f9da410c510 2024-04-23T10:36:06.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.877+0000 7f9dada82740 1 Processor -- start 2024-04-23T10:36:06.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.877+0000 7f9dada82740 1 -- start start 2024-04-23T10:36:06.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.877+0000 7f9da1ffb640 1 --2- 172.21.15.97:0/1292208832 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f9d940a3d20 0x7f9d940a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:06.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.878+0000 7f9da3fff640 1 --2- 172.21.15.97:0/1292208832 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f9d940a3d20 0x7f9d940a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:06.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.878+0000 7f9dada82740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5627c6e3f490 0x5627c6e3f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:06.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.878+0000 7f9dada82740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5627c6e3fdb0 0x5627c6e49f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:06.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.878+0000 7f9dada82740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5627c6e4a480 0x5627c6e4c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:06.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.878+0000 7f9dada82740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5627c6da46d0 con 0x5627c6e3fdb0 2024-04-23T10:36:06.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.878+0000 7f9dada82740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5627c6d411a0 con 0x5627c6e3f490 2024-04-23T10:36:06.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.878+0000 7f9dada82740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5627c6da0690 con 0x5627c6e4a480 2024-04-23T10:36:06.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.878+0000 7f9da1ffb640 1 -- 172.21.15.97:0/1292208832 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8528+0+0 (secure 0 0 0) 0x7f9d9c0cfd90 con 0x7f9da410c510 2024-04-23T10:36:06.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.878+0000 7f9da3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5627c6e3fdb0 0x5627c6e49f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:06.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.878+0000 7f9da3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5627c6e3fdb0 0x5627c6e49f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39524/0 (socket says 172.21.15.97:39524) 2024-04-23T10:36:06.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.878+0000 7f9da3fff640 1 -- 172.21.15.97:0/2408365778 learned_addr learned my addr 172.21.15.97:0/2408365778 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:36:06.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.878+0000 7f9da8b04640 1 --2- 172.21.15.97:0/2408365778 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5627c6e3f490 0x5627c6e3f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:06.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.878+0000 7f9da9305640 1 --2- 172.21.15.97:0/2408365778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5627c6e4a480 0x5627c6e4c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:06.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.878+0000 7f9da3fff640 1 -- 172.21.15.97:0/2408365778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5627c6e4a480 msgr2=0x5627c6e4c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:36:06.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.878+0000 7f9da3fff640 1 --2- 172.21.15.97:0/2408365778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5627c6e4a480 0x5627c6e4c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:06.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.878+0000 7f9da3fff640 1 -- 172.21.15.97:0/2408365778 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5627c6e3f490 msgr2=0x5627c6e3f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:36:06.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.878+0000 7f9da3fff640 1 --2- 172.21.15.97:0/2408365778 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5627c6e3f490 0x5627c6e3f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:06.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.878+0000 7f9da3fff640 1 -- 172.21.15.97:0/2408365778 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627c6d4b5b0 con 0x5627c6e3fdb0 2024-04-23T10:36:06.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.878+0000 7f9da3fff640 1 --2- 172.21.15.97:0/1292208832 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f9d940a3d20 0x7f9d940a61e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9d90009230 tx=0x7f9d900035f0 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:36:06.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.878+0000 7f9da3fff640 1 --2- 172.21.15.97:0/2408365778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5627c6e3fdb0 0x5627c6e49f40 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f9d90005ec0 tx=0x7f9d90047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=485b287b90137f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:36:06.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.878+0000 7f9d8effd640 1 -- 172.21.15.97:0/2408365778 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d90035050 con 0x5627c6e3fdb0 2024-04-23T10:36:06.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.879+0000 7f9d8effd640 1 -- 172.21.15.97:0/2408365778 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d90031cd0 con 0x5627c6e3fdb0 2024-04-23T10:36:06.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.879+0000 7f9d8effd640 1 -- 172.21.15.97:0/2408365778 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9003db70 con 0x5627c6e3fdb0 2024-04-23T10:36:06.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.879+0000 7f9dada82740 1 -- 172.21.15.97:0/2408365778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5627c6e3fdb0 msgr2=0x5627c6e49f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:06.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.879+0000 7f9dada82740 1 --2- 172.21.15.97:0/2408365778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5627c6e3fdb0 0x5627c6e49f40 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f9d90005ec0 tx=0x7f9d90047000 comp rx=0 tx=0).stop 2024-04-23T10:36:06.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.879+0000 7f9dada82740 1 -- 172.21.15.97:0/2408365778 shutdown_connections 2024-04-23T10:36:06.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.879+0000 7f9dada82740 1 --2- 172.21.15.97:0/2408365778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5627c6e4a480 0x5627c6e4c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:06.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.879+0000 7f9dada82740 1 --2- 172.21.15.97:0/2408365778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5627c6e3fdb0 0x5627c6e49f40 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:06.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.879+0000 7f9dada82740 1 --2- 172.21.15.97:0/2408365778 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5627c6e3f490 0x5627c6e3f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:06.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.879+0000 7f9dada82740 1 -- 172.21.15.97:0/2408365778 >> 172.21.15.97:0/2408365778 conn(0x5627c6e3b7b0 msgr2=0x5627c6e3bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:36:06.885 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.879+0000 7f9dada82740 1 -- 172.21.15.97:0/2408365778 shutdown_connections 2024-04-23T10:36:06.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.879+0000 7f9dada82740 1 -- 172.21.15.97:0/2408365778 wait complete. 2024-04-23T10:36:06.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.880+0000 7f9dada82740 1 -- start start 2024-04-23T10:36:06.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.880+0000 7f9dada82740 1 Processor -- start 2024-04-23T10:36:06.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.880+0000 7f9dada82740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5627c6e3f490 0x5627c6e49e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:06.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.880+0000 7f9dada82740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5627c6e3fdb0 0x5627c6e47150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:06.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.880+0000 7f9dada82740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5627c6e4a480 0x5627c6e47760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:06.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.880+0000 7f9dada82740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5627c6e4d2b0 con 0x5627c6e3f490 2024-04-23T10:36:06.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.880+0000 7f9dada82740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5627c6d3eb20 con 0x5627c6e3fdb0 2024-04-23T10:36:06.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.880+0000 7f9dada82740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5627c6e4d430 con 0x5627c6e4a480 2024-04-23T10:36:06.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.880+0000 7f9dada82740 10 client.0 _ll_register_callbacks cb 0x5627c6d3e530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:36:06.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.880+0000 7f9da8b04640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5627c6e3f490 0x5627c6e49e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:06.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.880+0000 7f9da8b04640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5627c6e3f490 0x5627c6e49e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39536/0 (socket says 172.21.15.97:39536) 2024-04-23T10:36:06.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.880+0000 7f9da8b04640 1 -- 172.21.15.97:0/248141596 learned_addr learned my addr 172.21.15.97:0/248141596 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:36:06.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.880+0000 7f9da8b04640 1 -- 172.21.15.97:0/248141596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5627c6e4a480 msgr2=0x5627c6e47760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:36:06.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.880+0000 7f9da8b04640 1 --2- 172.21.15.97:0/248141596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5627c6e4a480 0x5627c6e47760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:06.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.880+0000 7f9da8b04640 1 -- 172.21.15.97:0/248141596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5627c6e3fdb0 msgr2=0x5627c6e47150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:36:06.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.880+0000 7f9da3fff640 1 --2- 172.21.15.97:0/248141596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5627c6e3fdb0 0x5627c6e47150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:06.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.880+0000 7f9da8b04640 1 --2- 172.21.15.97:0/248141596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5627c6e3fdb0 0x5627c6e47150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:06.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.880+0000 7f9da8b04640 1 -- 172.21.15.97:0/248141596 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627c6ab6920 con 0x5627c6e3f490 2024-04-23T10:36:06.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.881+0000 7f9da8b04640 1 --2- 172.21.15.97:0/248141596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5627c6e3f490 0x5627c6e49e60 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f9d980117b0 tx=0x7f9d98011c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76fec2314305391d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:36:06.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.881+0000 7f9d83fff640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:36:06.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.881+0000 7f9d83fff640 1 -- 172.21.15.97:0/248141596 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9801e060 con 0x5627c6e3f490 2024-04-23T10:36:06.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.881+0000 7f9d83fff640 1 -- 172.21.15.97:0/248141596 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8528+0+0 (secure 0 0 0) 0x7f9d9800fdb0 con 0x5627c6e3f490 2024-04-23T10:36:06.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.881+0000 7f9dada82740 1 -- 172.21.15.97:0/248141596 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d90031cd0 con 0x5627c6e3f490 2024-04-23T10:36:06.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.881+0000 7f9d83fff640 1 -- 172.21.15.97:0/248141596 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9d98022410 con 0x5627c6e3f490 2024-04-23T10:36:06.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.881+0000 7f9dada82740 10 client.17019 fetch_fsmap learned FSMap version 26 2024-04-23T10:36:06.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.881+0000 7f9dada82740 1 -- 172.21.15.97:0/248141596 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5627c6b82d30 con 0x5627c6e3f490 2024-04-23T10:36:06.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.881+0000 7f9d83fff640 1 -- 172.21.15.97:0/248141596 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d90031cd0 con 0x5627c6e3f490 2024-04-23T10:36:06.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.881+0000 7f9d83fff640 1 -- 172.21.15.97:0/248141596 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f9d98018030 con 0x5627c6e3f490 2024-04-23T10:36:06.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.881+0000 7f9dada82740 10 client.17019 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:36:06.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.881+0000 7f9dada82740 10 client.17019 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:36:06.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.881+0000 7f9dada82740 20 client.17019 populate_metadata read hostname 'smithi097' 2024-04-23T10:36:06.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.881+0000 7f9dada82740 1 --2- 172.21.15.97:0/248141596 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5627c6e5a690 0x5627c6e5cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:06.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.881+0000 7f9dada82740 4 client.17019 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:36:06.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.881+0000 7f9dada82740 1 -- 172.21.15.97:0/248141596 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5627c6d62500 con 0x5627c6e5a690 2024-04-23T10:36:06.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.882+0000 7f9da3fff640 1 --2- 172.21.15.97:0/248141596 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5627c6e5a690 0x5627c6e5cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:06.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.882+0000 7f9da3fff640 1 --2- 172.21.15.97:0/248141596 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5627c6e5a690 0x5627c6e5cb10 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bea786ded6767ac server_cookie=54f1f5585e7e92d0 in_seq=0 out_seq=0 2024-04-23T10:36:06.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.882+0000 7f9d83fff640 10 client.17019 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:36:06.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.883+0000 7f9d83fff640 1 -- 172.21.15.97:0/248141596 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5627c6d62500 con 0x5627c6e5a690 2024-04-23T10:36:06.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.883+0000 7f9d83fff640 10 client.17019 handle_command_reply: tid=0 2024-04-23T10:36:06.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.883+0000 7f9dada82740 1 client.17019 shutdown 2024-04-23T10:36:06.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.883+0000 7f9dada82740 1 -- 172.21.15.97:0/248141596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5627c6e3f490 msgr2=0x5627c6e49e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:06.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.883+0000 7f9dada82740 1 --2- 172.21.15.97:0/248141596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5627c6e3f490 0x5627c6e49e60 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f9d980117b0 tx=0x7f9d98011c70 comp rx=0 tx=0).stop 2024-04-23T10:36:06.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.883+0000 7f9dada82740 1 -- 172.21.15.97:0/248141596 shutdown_connections 2024-04-23T10:36:06.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.883+0000 7f9dada82740 1 --2- 172.21.15.97:0/248141596 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5627c6e5a690 0x5627c6e5cb10 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:06.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.883+0000 7f9dada82740 1 --2- 172.21.15.97:0/248141596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5627c6e4a480 0x5627c6e47760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:06.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.883+0000 7f9d83fff640 0 client.17019 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:36:06.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.883+0000 7f9dada82740 1 --2- 172.21.15.97:0/248141596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5627c6e3fdb0 0x5627c6e47150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:06.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.883+0000 7f9dada82740 1 --2- 172.21.15.97:0/248141596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5627c6e3f490 0x5627c6e49e60 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:06.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.883+0000 7f9dada82740 1 -- 172.21.15.97:0/248141596 >> 172.21.15.97:0/248141596 conn(0x5627c6e3b7b0 msgr2=0x5627c6e3a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:36:06.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.883+0000 7f9dada82740 1 -- 172.21.15.97:0/248141596 shutdown_connections 2024-04-23T10:36:06.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.883+0000 7f9dada82740 1 -- 172.21.15.97:0/248141596 wait complete. 2024-04-23T10:36:06.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.883+0000 7f9dada82740 20 client.17019 trim_cache size 0 max 16384 2024-04-23T10:36:06.903 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.900+0000 7f9dada82740 1 Processor -- start 2024-04-23T10:36:06.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.900+0000 7f9dada82740 1 -- start start 2024-04-23T10:36:06.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.900+0000 7f9dada82740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5627c6e3f490 0x5627c6e561c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:06.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.900+0000 7f9dada82740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5627c6e3fdb0 0x5627c6e46090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:06.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.900+0000 7f9dada82740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5627c6e4a480 0x5627c6e49990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:06.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.900+0000 7f9dada82740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9da41adc80 con 0x5627c6e3f490 2024-04-23T10:36:06.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.900+0000 7f9dada82740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9da41af7c0 con 0x5627c6e4a480 2024-04-23T10:36:06.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.900+0000 7f9dada82740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9da41ae350 con 0x5627c6e3fdb0 2024-04-23T10:36:06.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.900+0000 7f9da8b04640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5627c6e3f490 0x5627c6e561c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:06.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.900+0000 7f9da8b04640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5627c6e3f490 0x5627c6e561c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39544/0 (socket says 172.21.15.97:39544) 2024-04-23T10:36:06.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.900+0000 7f9da8b04640 1 -- 172.21.15.97:0/1984554917 learned_addr learned my addr 172.21.15.97:0/1984554917 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:36:06.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.901+0000 7f9da8b04640 1 -- 172.21.15.97:0/1984554917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5627c6e3fdb0 msgr2=0x5627c6e46090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:36:06.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.901+0000 7f9da3fff640 1 --2- 172.21.15.97:0/1984554917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5627c6e3fdb0 0x5627c6e46090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:06.904 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.901+0000 7f9da8b04640 1 --2- 172.21.15.97:0/1984554917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5627c6e3fdb0 0x5627c6e46090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:06.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.901+0000 7f9da8b04640 1 -- 172.21.15.97:0/1984554917 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5627c6e4a480 msgr2=0x5627c6e49990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:36:06.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.901+0000 7f9da8b04640 1 --2- 172.21.15.97:0/1984554917 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5627c6e4a480 0x5627c6e49990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:06.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.901+0000 7f9da8b04640 1 -- 172.21.15.97:0/1984554917 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627c6e47ad0 con 0x5627c6e3f490 2024-04-23T10:36:06.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.901+0000 7f9da3fff640 1 --2- 172.21.15.97:0/1984554917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5627c6e3fdb0 0x5627c6e46090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:36:06.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.901+0000 7f9da8b04640 1 --2- 172.21.15.97:0/1984554917 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5627c6e3f490 0x5627c6e561c0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f9d98011550 tx=0x7f9d980213b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fae1a6effdb35b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:36:06.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.901+0000 7f9d8e7fc640 1 -- 172.21.15.97:0/1984554917 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d98030070 con 0x5627c6e3f490 2024-04-23T10:36:06.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.901+0000 7f9d8e7fc640 1 -- 172.21.15.97:0/1984554917 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d98021a10 con 0x5627c6e3f490 2024-04-23T10:36:06.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.901+0000 7f9d8e7fc640 1 -- 172.21.15.97:0/1984554917 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d98000e60 con 0x5627c6e3f490 2024-04-23T10:36:06.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.901+0000 7f9dada82740 1 -- 172.21.15.97:0/1984554917 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5627c6e3f490 msgr2=0x5627c6e561c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:06.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.901+0000 7f9dada82740 1 --2- 172.21.15.97:0/1984554917 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5627c6e3f490 0x5627c6e561c0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f9d98011550 tx=0x7f9d980213b0 comp rx=0 tx=0).stop 2024-04-23T10:36:06.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.902+0000 7f9dada82740 1 -- 172.21.15.97:0/1984554917 shutdown_connections 2024-04-23T10:36:06.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.902+0000 7f9dada82740 1 --2- 172.21.15.97:0/1984554917 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5627c6e4a480 0x5627c6e49990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:06.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.902+0000 7f9dada82740 1 --2- 172.21.15.97:0/1984554917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5627c6e3fdb0 0x5627c6e46090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:06.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.902+0000 7f9dada82740 1 --2- 172.21.15.97:0/1984554917 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5627c6e3f490 0x5627c6e561c0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:06.905 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.902+0000 7f9dada82740 1 -- 172.21.15.97:0/1984554917 >> 172.21.15.97:0/1984554917 conn(0x5627c6e3b7b0 msgr2=0x5627c6e41470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:36:06.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.902+0000 7f9dada82740 1 -- 172.21.15.97:0/1984554917 shutdown_connections 2024-04-23T10:36:06.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.902+0000 7f9dada82740 1 -- 172.21.15.97:0/1984554917 wait complete. 2024-04-23T10:36:06.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.902+0000 7f9dada82740 1 -- start start 2024-04-23T10:36:06.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.902+0000 7f9dada82740 1 Processor -- start 2024-04-23T10:36:06.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.902+0000 7f9dada82740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5627c6e3f490 0x5627c6e526d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:06.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.902+0000 7f9dada82740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5627c6e3fdb0 0x5627c6e52c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:06.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.902+0000 7f9dada82740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5627c6e4a480 0x5627c6e43440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:06.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.902+0000 7f9dada82740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9da41af0f0 con 0x5627c6e3fdb0 2024-04-23T10:36:06.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.902+0000 7f9dada82740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9da41aea20 con 0x5627c6e4a480 2024-04-23T10:36:06.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.902+0000 7f9dada82740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9da41abc60 con 0x5627c6e3f490 2024-04-23T10:36:06.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.902+0000 7f9dada82740 10 client.0 _ll_register_callbacks cb 0x7f9d98011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:36:06.906 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.902+0000 7f9da3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5627c6e3fdb0 0x5627c6e52c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:06.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.903+0000 7f9da3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5627c6e3fdb0 0x5627c6e52c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39556/0 (socket says 172.21.15.97:39556) 2024-04-23T10:36:06.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.903+0000 7f9da3fff640 1 -- 172.21.15.97:0/4117733770 learned_addr learned my addr 172.21.15.97:0/4117733770 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:36:06.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.903+0000 7f9da8b04640 1 --2- 172.21.15.97:0/4117733770 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5627c6e3f490 0x5627c6e526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:06.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.903+0000 7f9da3fff640 1 -- 172.21.15.97:0/4117733770 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5627c6e3f490 msgr2=0x5627c6e526d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:36:06.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.903+0000 7f9da3fff640 1 --2- 172.21.15.97:0/4117733770 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5627c6e3f490 0x5627c6e526d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:06.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.903+0000 7f9da3fff640 1 -- 172.21.15.97:0/4117733770 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5627c6e4a480 msgr2=0x5627c6e43440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:36:06.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.903+0000 7f9da3fff640 1 --2- 172.21.15.97:0/4117733770 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5627c6e4a480 0x5627c6e43440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:06.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.903+0000 7f9da3fff640 1 -- 172.21.15.97:0/4117733770 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627c6e43c70 con 0x5627c6e3fdb0 2024-04-23T10:36:06.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.903+0000 7f9da8b04640 1 --2- 172.21.15.97:0/4117733770 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5627c6e3f490 0x5627c6e526d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:36:06.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.903+0000 7f9da3fff640 1 --2- 172.21.15.97:0/4117733770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5627c6e3fdb0 0x5627c6e52c10 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f9d9002aa30 tx=0x7f9d90009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64ef1d0488f96094 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:36:06.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.903+0000 7f9d83fff640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:36:06.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.903+0000 7f9d83fff640 1 -- 172.21.15.97:0/4117733770 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9005c070 con 0x5627c6e3fdb0 2024-04-23T10:36:06.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.903+0000 7f9d83fff640 1 -- 172.21.15.97:0/4117733770 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8528+0+0 (secure 0 0 0) 0x7f9d90031820 con 0x5627c6e3fdb0 2024-04-23T10:36:06.907 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.903+0000 7f9dada82740 1 -- 172.21.15.97:0/4117733770 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d98021a10 con 0x5627c6e3fdb0 2024-04-23T10:36:06.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.903+0000 7f9d83fff640 1 -- 172.21.15.97:0/4117733770 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9d90068050 con 0x5627c6e3fdb0 2024-04-23T10:36:06.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.903+0000 7f9dada82740 10 client.17025 fetch_fsmap learned FSMap version 26 2024-04-23T10:36:06.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.903+0000 7f9dada82740 1 -- 172.21.15.97:0/4117733770 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5627c6e44f80 con 0x5627c6e3fdb0 2024-04-23T10:36:06.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.903+0000 7f9d83fff640 1 -- 172.21.15.97:0/4117733770 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d98021a10 con 0x5627c6e3fdb0 2024-04-23T10:36:06.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.904+0000 7f9d83fff640 1 -- 172.21.15.97:0/4117733770 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f9d90031a20 con 0x5627c6e3fdb0 2024-04-23T10:36:06.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.904+0000 7f9dada82740 10 client.17025 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:36:06.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.904+0000 7f9dada82740 10 client.17025 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:36:06.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.904+0000 7f9dada82740 20 client.17025 populate_metadata read hostname 'smithi097' 2024-04-23T10:36:06.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.904+0000 7f9dada82740 1 --2- 172.21.15.97:0/4117733770 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5627c6ec1c60 0x5627c6ec4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:06.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.904+0000 7f9dada82740 4 client.17025 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "32aeb117", "release": true, "await": true}] 2024-04-23T10:36:06.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.904+0000 7f9dada82740 1 -- 172.21.15.97:0/4117733770 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "quiesce db", "set_id": "32aeb117", "release": true, "await": true}) -- 0x5627c6db20f0 con 0x5627c6ec1c60 2024-04-23T10:36:06.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.904+0000 7f9da8b04640 1 --2- 172.21.15.97:0/4117733770 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5627c6ec1c60 0x5627c6ec4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:06.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.904+0000 7f9da8b04640 1 --2- 172.21.15.97:0/4117733770 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5627c6ec1c60 0x5627c6ec4050 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a859dab6a4ea285 server_cookie=65d2a459b8feff0f in_seq=0 out_seq=0 2024-04-23T10:36:06.908 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.904+0000 7f9d83fff640 10 client.17025 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:36:06.909 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.907+0000 7f9d83fff640 1 -- 172.21.15.97:0/4117733770 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5627c6db20f0 con 0x5627c6ec1c60 2024-04-23T10:36:06.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.907+0000 7f9d83fff640 10 client.17025 handle_command_reply: tid=0 2024-04-23T10:36:06.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.907+0000 7f9dada82740 1 client.17025 shutdown 2024-04-23T10:36:06.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.908+0000 7f9dada82740 1 -- 172.21.15.97:0/4117733770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5627c6e3fdb0 msgr2=0x5627c6e52c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:06.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.908+0000 7f9dada82740 1 --2- 172.21.15.97:0/4117733770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5627c6e3fdb0 0x5627c6e52c10 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f9d9002aa30 tx=0x7f9d90009260 comp rx=0 tx=0).stop 2024-04-23T10:36:06.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.908+0000 7f9dada82740 1 -- 172.21.15.97:0/4117733770 shutdown_connections 2024-04-23T10:36:06.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.908+0000 7f9dada82740 1 --2- 172.21.15.97:0/4117733770 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5627c6ec1c60 0x5627c6ec4050 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:06.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.908+0000 7f9dada82740 1 --2- 172.21.15.97:0/4117733770 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5627c6e4a480 0x5627c6e43440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:06.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.908+0000 7f9dada82740 1 --2- 172.21.15.97:0/4117733770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5627c6e3fdb0 0x5627c6e52c10 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:06.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.908+0000 7f9d83fff640 0 client.17025 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:36:06.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.908+0000 7f9dada82740 1 --2- 172.21.15.97:0/4117733770 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5627c6e3f490 0x5627c6e526d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:06.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.908+0000 7f9dada82740 1 -- 172.21.15.97:0/4117733770 >> 172.21.15.97:0/4117733770 conn(0x5627c6e3b7b0 msgr2=0x5627c6e551b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:36:06.910 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.908+0000 7f9dada82740 1 -- 172.21.15.97:0/4117733770 shutdown_connections 2024-04-23T10:36:06.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.908+0000 7f9dada82740 1 -- 172.21.15.97:0/4117733770 wait complete. 2024-04-23T10:36:06.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.908+0000 7f9dada82740 20 client.17025 trim_cache size 0 max 16384 2024-04-23T10:36:06.911 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-23T10:36:06.911 INFO:teuthology.orchestra.run.smithi097.stdout: "epoch": 26, 2024-04-23T10:36:06.911 INFO:teuthology.orchestra.run.smithi097.stdout: "leader": 24488, 2024-04-23T10:36:06.911 INFO:teuthology.orchestra.run.smithi097.stdout: "set_version": 44, 2024-04-23T10:36:06.911 INFO:teuthology.orchestra.run.smithi097.stdout: "sets": { 2024-04-23T10:36:06.911 INFO:teuthology.orchestra.run.smithi097.stdout: "32aeb117": { 2024-04-23T10:36:06.911 INFO:teuthology.orchestra.run.smithi097.stdout: "version": 44, 2024-04-23T10:36:06.911 INFO:teuthology.orchestra.run.smithi097.stdout: "age_ref": 0.0, 2024-04-23T10:36:06.911 INFO:teuthology.orchestra.run.smithi097.stdout: "state": { 2024-04-23T10:36:06.912 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RELEASED", 2024-04-23T10:36:06.912 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0 2024-04-23T10:36:06.912 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:36:06.912 INFO:teuthology.orchestra.run.smithi097.stdout: "timeout": 60.0, 2024-04-23T10:36:06.912 INFO:teuthology.orchestra.run.smithi097.stdout: "expiration": 77.0, 2024-04-23T10:36:06.912 INFO:teuthology.orchestra.run.smithi097.stdout: "members": { 2024-04-23T10:36:06.912 INFO:teuthology.orchestra.run.smithi097.stdout: "file:/": { 2024-04-23T10:36:06.912 INFO:teuthology.orchestra.run.smithi097.stdout: "excluded": false, 2024-04-23T10:36:06.912 INFO:teuthology.orchestra.run.smithi097.stdout: "state": { 2024-04-23T10:36:06.912 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RELEASED", 2024-04-23T10:36:06.912 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0 2024-04-23T10:36:06.912 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:36:06.912 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:36:06.912 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:36:06.912 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:36:06.913 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:36:06.913 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-23T10:36:06.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.908+0000 7f9daad68640 1 -- 172.21.15.97:0/1292208832 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f9d940a3d20 msgr2=0x7f9d940a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:06.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.908+0000 7f9daad68640 1 --2- 172.21.15.97:0/1292208832 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f9d940a3d20 0x7f9d940a61e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9d90009230 tx=0x7f9d900035f0 comp rx=0 tx=0).stop 2024-04-23T10:36:06.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.908+0000 7f9daad68640 1 -- 172.21.15.97:0/1292208832 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9da410c510 msgr2=0x7f9da4198670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:06.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.908+0000 7f9daad68640 1 --2- 172.21.15.97:0/1292208832 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9da410c510 0x7f9da4198670 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c00b970 tx=0x7f9d9c00be30 comp rx=0 tx=0).stop 2024-04-23T10:36:06.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.908+0000 7f9daad68640 1 -- 172.21.15.97:0/1292208832 shutdown_connections 2024-04-23T10:36:06.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.908+0000 7f9daad68640 1 --2- 172.21.15.97:0/1292208832 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f9d940a3d20 0x7f9d940a61e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:06.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.908+0000 7f9daad68640 1 --2- 172.21.15.97:0/1292208832 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9da410c510 0x7f9da4198670 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:06.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.908+0000 7f9daad68640 1 --2- 172.21.15.97:0/1292208832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9da4103af0 0x7f9da419eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:06.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.908+0000 7f9daad68640 1 --2- 172.21.15.97:0/1292208832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9da41031d0 0x7f9da419e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:06.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.908+0000 7f9daad68640 1 -- 172.21.15.97:0/1292208832 >> 172.21.15.97:0/1292208832 conn(0x7f9da4077f50 msgr2=0x7f9da40ff270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:36:06.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.908+0000 7f9daad68640 1 -- 172.21.15.97:0/1292208832 shutdown_connections 2024-04-23T10:36:06.914 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:06.908+0000 7f9daad68640 1 -- 172.21.15.97:0/1292208832 wait complete. 2024-04-23T10:36:06.918 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '32aeb117', seconds elapsed: 18.0 2024-04-23T10:36:07.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:36:07 smithi183 ceph-mon[32317]: osdmap e96: 12 total, 12 up, 12 in 2024-04-23T10:36:07.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:36:07 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/789373665' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:36:07.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:36:07 smithi196 ceph-mon[32919]: osdmap e96: 12 total, 12 up, 12 in 2024-04-23T10:36:07.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:36:07 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/789373665' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:36:07.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:36:07 smithi097 ceph-mon[29197]: osdmap e96: 12 total, 12 up, 12 in 2024-04-23T10:36:07.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:36:07 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/789373665' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:36:08.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:36:08 smithi183 ceph-mon[32317]: pgmap v308: 97 pgs: 97 active+clean; 1.8 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 68 KiB/s wr, 34 op/s 2024-04-23T10:36:08.762 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:36:08 smithi097 ceph-mon[29197]: pgmap v308: 97 pgs: 97 active+clean; 1.8 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 68 KiB/s wr, 34 op/s 2024-04-23T10:36:08.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:36:08 smithi196 ceph-mon[32919]: pgmap v308: 97 pgs: 97 active+clean; 1.8 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 68 KiB/s wr, 34 op/s 2024-04-23T10:36:10.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:36:10 smithi183 ceph-mon[32317]: pgmap v309: 97 pgs: 97 active+clean; 1.8 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 68 KiB/s wr, 34 op/s 2024-04-23T10:36:10.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:36:10 smithi196 ceph-mon[32919]: pgmap v309: 97 pgs: 97 active+clean; 1.8 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 68 KiB/s wr, 34 op/s 2024-04-23T10:36:10.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:36:10 smithi097 ceph-mon[29197]: pgmap v309: 97 pgs: 97 active+clean; 1.8 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 68 KiB/s wr, 34 op/s 2024-04-23T10:36:12.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:36:12 smithi097 ceph-mon[29197]: osdmap e97: 12 total, 12 up, 12 in 2024-04-23T10:36:12.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:36:12 smithi183 ceph-mon[32317]: osdmap e97: 12 total, 12 up, 12 in 2024-04-23T10:36:12.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:36:12 smithi196 ceph-mon[32919]: osdmap e97: 12 total, 12 up, 12 in 2024-04-23T10:36:13.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:36:13 smithi183 ceph-mon[32317]: pgmap v311: 97 pgs: 97 active+clean; 1.8 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 14 KiB/s wr, 10 op/s 2024-04-23T10:36:13.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:36:13 smithi196 ceph-mon[32919]: pgmap v311: 97 pgs: 97 active+clean; 1.8 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 14 KiB/s wr, 10 op/s 2024-04-23T10:36:13.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:36:13 smithi097 ceph-mon[29197]: pgmap v311: 97 pgs: 97 active+clean; 1.8 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 14 KiB/s wr, 10 op/s 2024-04-23T10:36:15.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:36:15 smithi183 ceph-mon[32317]: pgmap v312: 97 pgs: 97 active+clean; 2.0 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 41 KiB/s wr, 19 op/s 2024-04-23T10:36:15.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:36:15 smithi196 ceph-mon[32919]: pgmap v312: 97 pgs: 97 active+clean; 2.0 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 41 KiB/s wr, 19 op/s 2024-04-23T10:36:15.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:36:15 smithi097 ceph-mon[29197]: pgmap v312: 97 pgs: 97 active+clean; 2.0 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 41 KiB/s wr, 19 op/s 2024-04-23T10:36:17.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:36:17 smithi183 ceph-mon[32317]: pgmap v313: 97 pgs: 97 active+clean; 2.0 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 37 KiB/s wr, 17 op/s 2024-04-23T10:36:17.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:36:17 smithi196 ceph-mon[32919]: pgmap v313: 97 pgs: 97 active+clean; 2.0 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 37 KiB/s wr, 17 op/s 2024-04-23T10:36:17.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:36:17 smithi097 ceph-mon[29197]: pgmap v313: 97 pgs: 97 active+clean; 2.0 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 37 KiB/s wr, 17 op/s 2024-04-23T10:36:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.247+0000 7fac9fae1640 1 -- 172.21.15.97:0/2815997606 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac98102b80 msgr2=0x7fac98106bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.247+0000 7fac9fae1640 1 --2- 172.21.15.97:0/2815997606 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac98102b80 0x7fac98106bc0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fac94009900 tx=0x7fac9402a410 comp rx=0 tx=0).stop 2024-04-23T10:36:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.247+0000 7fac9fae1640 1 -- 172.21.15.97:0/2815997606 shutdown_connections 2024-04-23T10:36:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.247+0000 7fac9fae1640 1 --2- 172.21.15.97:0/2815997606 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fac981072a0 0x7fac9810ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.247+0000 7fac9fae1640 1 --2- 172.21.15.97:0/2815997606 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac98102b80 0x7fac98106bc0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.247+0000 7fac9fae1640 1 --2- 172.21.15.97:0/2815997606 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac981021d0 0x7fac981025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.247+0000 7fac9fae1640 1 -- 172.21.15.97:0/2815997606 >> 172.21.15.97:0/2815997606 conn(0x7fac980fda50 msgr2=0x7fac980ffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:36:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.247+0000 7fac9fae1640 1 -- 172.21.15.97:0/2815997606 shutdown_connections 2024-04-23T10:36:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.247+0000 7fac9fae1640 1 -- 172.21.15.97:0/2815997606 wait complete. 2024-04-23T10:36:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.248+0000 7fac9fae1640 1 Processor -- start 2024-04-23T10:36:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.248+0000 7fac9fae1640 1 -- start start 2024-04-23T10:36:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.248+0000 7fac9fae1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac981021d0 0x7fac98198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.248+0000 7fac9fae1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac981072a0 0x7fac98198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.248+0000 7fac9fae1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fac9819ce70 0x7fac9819d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.248+0000 7fac9fae1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fac981121c0 con 0x7fac981072a0 2024-04-23T10:36:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.248+0000 7fac9fae1640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fac98112040 con 0x7fac9819ce70 2024-04-23T10:36:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.248+0000 7fac9fae1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fac98112340 con 0x7fac981021d0 2024-04-23T10:36:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.248+0000 7fac9d07c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac981072a0 0x7fac98198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.248+0000 7fac9d07c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac981072a0 0x7fac98198b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34890/0 (socket says 172.21.15.97:34890) 2024-04-23T10:36:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.248+0000 7fac9d07c640 1 -- 172.21.15.97:0/3921345968 learned_addr learned my addr 172.21.15.97:0/3921345968 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:36:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.248+0000 7fac9d87d640 1 --2- 172.21.15.97:0/3921345968 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac981021d0 0x7fac98198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.248+0000 7fac9e07e640 1 --2- 172.21.15.97:0/3921345968 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fac9819ce70 0x7fac9819d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.248+0000 7fac9d07c640 1 -- 172.21.15.97:0/3921345968 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac981021d0 msgr2=0x7fac98198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:18.250 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.248+0000 7fac9d07c640 1 --2- 172.21.15.97:0/3921345968 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac981021d0 0x7fac98198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:18.250 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.248+0000 7fac9d07c640 1 -- 172.21.15.97:0/3921345968 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fac9819ce70 msgr2=0x7fac9819d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:18.250 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.248+0000 7fac9d07c640 1 --2- 172.21.15.97:0/3921345968 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fac9819ce70 0x7fac9819d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:18.250 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.248+0000 7fac9d07c640 1 -- 172.21.15.97:0/3921345968 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac9819d960 con 0x7fac981072a0 2024-04-23T10:36:18.250 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.249+0000 7fac9d07c640 1 --2- 172.21.15.97:0/3921345968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac981072a0 0x7fac98198b40 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fac94031a90 tx=0x7fac94004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:36:18.250 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.249+0000 7fac8effd640 1 -- 172.21.15.97:0/3921345968 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9403f030 con 0x7fac981072a0 2024-04-23T10:36:18.250 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.249+0000 7fac9fae1640 1 -- 172.21.15.97:0/3921345968 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac9819dbf0 con 0x7fac981072a0 2024-04-23T10:36:18.250 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.249+0000 7fac8effd640 1 -- 172.21.15.97:0/3921345968 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac94004490 con 0x7fac981072a0 2024-04-23T10:36:18.250 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.249+0000 7fac8effd640 1 -- 172.21.15.97:0/3921345968 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac94035070 con 0x7fac981072a0 2024-04-23T10:36:18.250 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.249+0000 7fac9fae1640 1 -- 172.21.15.97:0/3921345968 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac981a53b0 con 0x7fac981072a0 2024-04-23T10:36:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.251+0000 7fac8effd640 1 -- 172.21.15.97:0/3921345968 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fac94042030 con 0x7fac981072a0 2024-04-23T10:36:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.252+0000 7faca27f7740 1 Processor -- start 2024-04-23T10:36:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.252+0000 7faca27f7740 1 -- start start 2024-04-23T10:36:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.252+0000 7faca27f7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ba2ca71620 0x55ba2ca71a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.252+0000 7faca27f7740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba2ca71f40 0x55ba2ca7c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.252+0000 7faca27f7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba2ca7c800 0x55ba2ca7ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.252+0000 7faca27f7740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ba2c9d6970 con 0x55ba2ca71f40 2024-04-23T10:36:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.252+0000 7fac8effd640 1 --2- 172.21.15.97:0/3921345968 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fac840a3dc0 0x7fac840a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.252+0000 7faca27f7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ba2c9734a0 con 0x55ba2ca71620 2024-04-23T10:36:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.252+0000 7faca27f7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ba2c9d2930 con 0x55ba2ca7c800 2024-04-23T10:36:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.252+0000 7fac9d87d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba2ca71f40 0x55ba2ca7c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.252+0000 7fac9e07e640 1 --2- 172.21.15.97:0/3921345968 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fac840a3dc0 0x7fac840a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.252+0000 7fac9d87d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba2ca71f40 0x55ba2ca7c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34906/0 (socket says 172.21.15.97:34906) 2024-04-23T10:36:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.252+0000 7fac9d87d640 1 -- 172.21.15.97:0/422638723 learned_addr learned my addr 172.21.15.97:0/422638723 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:36:18.258 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.252+0000 7fac8effd640 1 -- 172.21.15.97:0/3921345968 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8484+0+0 (secure 0 0 0) 0x7fac940f0d80 con 0x7fac981072a0 2024-04-23T10:36:18.258 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.252+0000 7fac9e07e640 1 --2- 172.21.15.97:0/422638723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ba2ca71620 0x55ba2ca71a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:18.258 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.252+0000 7fac9d07c640 1 --2- 172.21.15.97:0/422638723 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba2ca7c800 0x55ba2ca7ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:18.258 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.253+0000 7fac9d87d640 1 -- 172.21.15.97:0/422638723 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba2ca7c800 msgr2=0x55ba2ca7ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:36:18.258 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.253+0000 7fac9d87d640 1 --2- 172.21.15.97:0/422638723 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba2ca7c800 0x55ba2ca7ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:18.258 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.253+0000 7fac9d87d640 1 -- 172.21.15.97:0/422638723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ba2ca71620 msgr2=0x55ba2ca71a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:36:18.258 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.253+0000 7fac9d87d640 1 --2- 172.21.15.97:0/422638723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ba2ca71620 0x55ba2ca71a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:18.258 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.253+0000 7fac9d87d640 1 -- 172.21.15.97:0/422638723 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba2c6e9210 con 0x55ba2ca71f40 2024-04-23T10:36:18.258 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.253+0000 7fac9d07c640 1 --2- 172.21.15.97:0/422638723 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba2ca7c800 0x55ba2ca7ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:36:18.258 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.253+0000 7fac9e07e640 1 --2- 172.21.15.97:0/3921345968 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fac840a3dc0 0x7fac840a6280 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fac90014580 tx=0x7fac90014980 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:36:18.258 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.253+0000 7fac9d87d640 1 --2- 172.21.15.97:0/422638723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba2ca71f40 0x55ba2ca7c2c0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fac8800ad00 tx=0x7fac8800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=834f696817e7024c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:36:18.258 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.253+0000 7fac83fff640 1 -- 172.21.15.97:0/422638723 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac8801d070 con 0x55ba2ca71f40 2024-04-23T10:36:18.258 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.253+0000 7fac83fff640 1 -- 172.21.15.97:0/422638723 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac88018af0 con 0x55ba2ca71f40 2024-04-23T10:36:18.258 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.253+0000 7fac83fff640 1 -- 172.21.15.97:0/422638723 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac88010a70 con 0x55ba2ca71f40 2024-04-23T10:36:18.259 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.254+0000 7faca27f7740 1 -- 172.21.15.97:0/422638723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba2ca71f40 msgr2=0x55ba2ca7c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:18.259 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.254+0000 7faca27f7740 1 --2- 172.21.15.97:0/422638723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba2ca71f40 0x55ba2ca7c2c0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fac8800ad00 tx=0x7fac8800f790 comp rx=0 tx=0).stop 2024-04-23T10:36:18.259 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.254+0000 7faca27f7740 1 -- 172.21.15.97:0/422638723 shutdown_connections 2024-04-23T10:36:18.259 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.254+0000 7faca27f7740 1 --2- 172.21.15.97:0/422638723 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba2ca7c800 0x55ba2ca7ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:18.259 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.254+0000 7faca27f7740 1 --2- 172.21.15.97:0/422638723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba2ca71f40 0x55ba2ca7c2c0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:18.259 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.254+0000 7faca27f7740 1 --2- 172.21.15.97:0/422638723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ba2ca71620 0x55ba2ca71a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:18.259 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.254+0000 7faca27f7740 1 -- 172.21.15.97:0/422638723 >> 172.21.15.97:0/422638723 conn(0x55ba2ca6db30 msgr2=0x55ba2ca6df10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:36:18.259 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.254+0000 7faca27f7740 1 -- 172.21.15.97:0/422638723 shutdown_connections 2024-04-23T10:36:18.259 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.254+0000 7faca27f7740 1 -- 172.21.15.97:0/422638723 wait complete. 2024-04-23T10:36:18.259 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.254+0000 7faca27f7740 1 -- start start 2024-04-23T10:36:18.259 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.254+0000 7faca27f7740 1 Processor -- start 2024-04-23T10:36:18.259 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.255+0000 7faca27f7740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba2ca71620 0x55ba2ca7c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:18.259 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.255+0000 7faca27f7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ba2ca71f40 0x55ba2ca793d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:18.259 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.255+0000 7faca27f7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba2ca7c800 0x55ba2ca79910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:18.259 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.255+0000 7faca27f7740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ba2ca7f430 con 0x55ba2ca71620 2024-04-23T10:36:18.260 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.255+0000 7faca27f7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ba2c9710b0 con 0x55ba2ca71f40 2024-04-23T10:36:18.260 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.255+0000 7faca27f7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55ba2ca7f5b0 con 0x55ba2ca7c800 2024-04-23T10:36:18.260 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.255+0000 7faca27f7740 10 client.0 _ll_register_callbacks cb 0x55ba2c95ed30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:36:18.260 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.255+0000 7fac9d87d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba2ca71620 0x55ba2ca7c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:18.260 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.255+0000 7fac9d87d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba2ca71620 0x55ba2ca7c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34910/0 (socket says 172.21.15.97:34910) 2024-04-23T10:36:18.260 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.255+0000 7fac9d87d640 1 -- 172.21.15.97:0/1426067661 learned_addr learned my addr 172.21.15.97:0/1426067661 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:36:18.260 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.255+0000 7fac9d07c640 1 --2- 172.21.15.97:0/1426067661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ba2ca71f40 0x55ba2ca793d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:18.260 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.255+0000 7fac9e07e640 1 --2- 172.21.15.97:0/1426067661 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba2ca7c800 0x55ba2ca79910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:18.260 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.255+0000 7fac9d87d640 1 -- 172.21.15.97:0/1426067661 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba2ca7c800 msgr2=0x55ba2ca79910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:36:18.260 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.255+0000 7fac9d87d640 1 --2- 172.21.15.97:0/1426067661 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba2ca7c800 0x55ba2ca79910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:18.260 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.255+0000 7fac9d87d640 1 -- 172.21.15.97:0/1426067661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ba2ca71f40 msgr2=0x55ba2ca793d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:36:18.260 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.255+0000 7fac9d87d640 1 --2- 172.21.15.97:0/1426067661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ba2ca71f40 0x55ba2ca793d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:18.260 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.255+0000 7fac9d87d640 1 -- 172.21.15.97:0/1426067661 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba2c7b5620 con 0x55ba2ca71620 2024-04-23T10:36:18.260 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.255+0000 7fac9d07c640 1 --2- 172.21.15.97:0/1426067661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ba2ca71f40 0x55ba2ca793d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:36:18.260 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.255+0000 7fac9d87d640 1 --2- 172.21.15.97:0/1426067661 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba2ca71620 0x55ba2ca7c0e0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fac880105f0 tx=0x7fac88018670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6cd314a178c8a36 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:36:18.260 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.255+0000 7fac817fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:36:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.255+0000 7fac817fa640 1 -- 172.21.15.97:0/1426067661 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac8801d040 con 0x55ba2ca71620 2024-04-23T10:36:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.255+0000 7fac817fa640 1 -- 172.21.15.97:0/1426067661 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8484+0+0 (secure 0 0 0) 0x7fac88005040 con 0x55ba2ca71620 2024-04-23T10:36:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.255+0000 7faca27f7740 1 -- 172.21.15.97:0/1426067661 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac88018af0 con 0x55ba2ca71620 2024-04-23T10:36:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.256+0000 7fac817fa640 1 -- 172.21.15.97:0/1426067661 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fac88031050 con 0x55ba2ca71620 2024-04-23T10:36:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.256+0000 7faca27f7740 10 client.17037 fetch_fsmap learned FSMap version 26 2024-04-23T10:36:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.256+0000 7faca27f7740 1 -- 172.21.15.97:0/1426067661 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ba2ca8b240 con 0x55ba2ca71620 2024-04-23T10:36:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.256+0000 7fac817fa640 1 -- 172.21.15.97:0/1426067661 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac88018af0 con 0x55ba2ca71620 2024-04-23T10:36:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.256+0000 7fac817fa640 1 -- 172.21.15.97:0/1426067661 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fac88020030 con 0x55ba2ca71620 2024-04-23T10:36:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.256+0000 7faca27f7740 10 client.17037 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:36:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.256+0000 7faca27f7740 10 client.17037 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:36:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.256+0000 7faca27f7740 20 client.17037 populate_metadata read hostname 'smithi097' 2024-04-23T10:36:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.256+0000 7faca27f7740 1 --2- 172.21.15.97:0/1426067661 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55ba2ca8c640 0x55ba2ca8ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.256+0000 7faca27f7740 4 client.17037 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:36:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.256+0000 7faca27f7740 1 -- 172.21.15.97:0/1426067661 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba2c9e4390 con 0x55ba2ca8c640 2024-04-23T10:36:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.256+0000 7fac9d07c640 1 --2- 172.21.15.97:0/1426067661 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55ba2ca8c640 0x55ba2ca8ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.257+0000 7fac817fa640 10 client.17037 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:36:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.257+0000 7fac9d07c640 1 --2- 172.21.15.97:0/1426067661 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55ba2ca8c640 0x55ba2ca8ea30 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a8124853f064286 server_cookie=20af39e71d489699 in_seq=0 out_seq=0 2024-04-23T10:36:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.257+0000 7fac817fa640 1 -- 172.21.15.97:0/1426067661 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ba2c9e4390 con 0x55ba2ca8c640 2024-04-23T10:36:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.257+0000 7fac817fa640 10 client.17037 handle_command_reply: tid=0 2024-04-23T10:36:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.258+0000 7faca27f7740 1 client.17037 shutdown 2024-04-23T10:36:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.258+0000 7faca27f7740 1 -- 172.21.15.97:0/1426067661 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba2ca71620 msgr2=0x55ba2ca7c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.258+0000 7faca27f7740 1 --2- 172.21.15.97:0/1426067661 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba2ca71620 0x55ba2ca7c0e0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fac880105f0 tx=0x7fac88018670 comp rx=0 tx=0).stop 2024-04-23T10:36:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.258+0000 7faca27f7740 1 -- 172.21.15.97:0/1426067661 shutdown_connections 2024-04-23T10:36:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.258+0000 7faca27f7740 1 --2- 172.21.15.97:0/1426067661 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55ba2ca8c640 0x55ba2ca8ea30 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.258+0000 7faca27f7740 1 --2- 172.21.15.97:0/1426067661 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba2ca7c800 0x55ba2ca79910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.258+0000 7faca27f7740 1 --2- 172.21.15.97:0/1426067661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ba2ca71f40 0x55ba2ca793d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.258+0000 7faca27f7740 1 --2- 172.21.15.97:0/1426067661 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba2ca71620 0x55ba2ca7c0e0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.258+0000 7fac817fa640 0 client.17037 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:36:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.258+0000 7faca27f7740 1 -- 172.21.15.97:0/1426067661 >> 172.21.15.97:0/1426067661 conn(0x55ba2ca6db30 msgr2=0x55ba2ca738b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:36:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.258+0000 7faca27f7740 1 -- 172.21.15.97:0/1426067661 shutdown_connections 2024-04-23T10:36:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.258+0000 7faca27f7740 1 -- 172.21.15.97:0/1426067661 wait complete. 2024-04-23T10:36:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.258+0000 7faca27f7740 20 client.17037 trim_cache size 0 max 16384 2024-04-23T10:36:18.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.273+0000 7faca27f7740 1 Processor -- start 2024-04-23T10:36:18.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.273+0000 7faca27f7740 1 -- start start 2024-04-23T10:36:18.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.273+0000 7faca27f7740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba2ca71620 0x55ba2ca882f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:18.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.273+0000 7faca27f7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ba2ca71f40 0x55ba2ca74af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:18.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.273+0000 7faca27f7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba2ca7c800 0x55ba2ca84be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:18.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.273+0000 7faca27f7740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fac981ad990 con 0x55ba2ca71620 2024-04-23T10:36:18.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.273+0000 7faca27f7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fac981af4d0 con 0x55ba2ca71f40 2024-04-23T10:36:18.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.273+0000 7faca27f7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fac981ae060 con 0x55ba2ca7c800 2024-04-23T10:36:18.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.273+0000 7fac9d87d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba2ca71620 0x55ba2ca882f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:18.278 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.273+0000 7fac9d87d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba2ca71620 0x55ba2ca882f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34922/0 (socket says 172.21.15.97:34922) 2024-04-23T10:36:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.273+0000 7fac9d87d640 1 -- 172.21.15.97:0/4140757550 learned_addr learned my addr 172.21.15.97:0/4140757550 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:36:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.273+0000 7fac9d07c640 1 --2- 172.21.15.97:0/4140757550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ba2ca71f40 0x55ba2ca74af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.273+0000 7fac9d87d640 1 -- 172.21.15.97:0/4140757550 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba2ca7c800 msgr2=0x55ba2ca84be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:36:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.273+0000 7fac9e07e640 1 --2- 172.21.15.97:0/4140757550 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba2ca7c800 0x55ba2ca84be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.273+0000 7fac9d87d640 1 --2- 172.21.15.97:0/4140757550 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba2ca7c800 0x55ba2ca84be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.273+0000 7fac9d87d640 1 -- 172.21.15.97:0/4140757550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ba2ca71f40 msgr2=0x55ba2ca74af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:36:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.273+0000 7fac9d87d640 1 --2- 172.21.15.97:0/4140757550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ba2ca71f40 0x55ba2ca74af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.273+0000 7fac9d87d640 1 -- 172.21.15.97:0/4140757550 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba2ca77920 con 0x55ba2ca71620 2024-04-23T10:36:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.273+0000 7fac9d07c640 1 --2- 172.21.15.97:0/4140757550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ba2ca71f40 0x55ba2ca74af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:36:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.273+0000 7fac9d87d640 1 --2- 172.21.15.97:0/4140757550 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba2ca71620 0x55ba2ca882f0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fac88001eb0 tx=0x7fac8800a3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdcfec5c64110bd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:36:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.274+0000 7fac837fe640 1 -- 172.21.15.97:0/4140757550 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac8803a070 con 0x55ba2ca71620 2024-04-23T10:36:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.274+0000 7fac837fe640 1 -- 172.21.15.97:0/4140757550 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac88028070 con 0x55ba2ca71620 2024-04-23T10:36:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.274+0000 7fac837fe640 1 -- 172.21.15.97:0/4140757550 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac8801d040 con 0x55ba2ca71620 2024-04-23T10:36:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.274+0000 7faca27f7740 1 -- 172.21.15.97:0/4140757550 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba2ca71620 msgr2=0x55ba2ca882f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.274+0000 7faca27f7740 1 --2- 172.21.15.97:0/4140757550 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba2ca71620 0x55ba2ca882f0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fac88001eb0 tx=0x7fac8800a3e0 comp rx=0 tx=0).stop 2024-04-23T10:36:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.274+0000 7faca27f7740 1 -- 172.21.15.97:0/4140757550 shutdown_connections 2024-04-23T10:36:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.274+0000 7faca27f7740 1 --2- 172.21.15.97:0/4140757550 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba2ca7c800 0x55ba2ca84be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.274+0000 7faca27f7740 1 --2- 172.21.15.97:0/4140757550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ba2ca71f40 0x55ba2ca74af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.274+0000 7faca27f7740 1 --2- 172.21.15.97:0/4140757550 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba2ca71620 0x55ba2ca882f0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.274+0000 7faca27f7740 1 -- 172.21.15.97:0/4140757550 >> 172.21.15.97:0/4140757550 conn(0x55ba2ca6db30 msgr2=0x55ba2ca78c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:36:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.274+0000 7faca27f7740 1 -- 172.21.15.97:0/4140757550 shutdown_connections 2024-04-23T10:36:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.274+0000 7faca27f7740 1 -- 172.21.15.97:0/4140757550 wait complete. 2024-04-23T10:36:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.274+0000 7faca27f7740 1 -- start start 2024-04-23T10:36:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.275+0000 7faca27f7740 1 Processor -- start 2024-04-23T10:36:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.275+0000 7faca27f7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba2ca71620 0x55ba2caef330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.275+0000 7faca27f7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ba2ca71f40 0x55ba2caef870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.275+0000 7faca27f7740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba2ca7c800 0x55ba2ca76540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.275+0000 7faca27f7740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fac981aee00 con 0x55ba2ca7c800 2024-04-23T10:36:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.275+0000 7faca27f7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fac981ae730 con 0x55ba2ca71f40 2024-04-23T10:36:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.275+0000 7faca27f7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fac981ab990 con 0x55ba2ca71620 2024-04-23T10:36:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.275+0000 7faca27f7740 10 client.0 _ll_register_callbacks cb 0x7fac88010430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:36:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.275+0000 7fac9e07e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba2ca7c800 0x55ba2ca76540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.275+0000 7fac9e07e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba2ca7c800 0x55ba2ca76540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34932/0 (socket says 172.21.15.97:34932) 2024-04-23T10:36:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.275+0000 7fac9e07e640 1 -- 172.21.15.97:0/2979865210 learned_addr learned my addr 172.21.15.97:0/2979865210 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:36:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.275+0000 7fac9d07c640 1 --2- 172.21.15.97:0/2979865210 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ba2ca71f40 0x55ba2caef870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.275+0000 7fac9d87d640 1 --2- 172.21.15.97:0/2979865210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba2ca71620 0x55ba2caef330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.275+0000 7fac9e07e640 1 -- 172.21.15.97:0/2979865210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba2ca71620 msgr2=0x55ba2caef330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:36:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.275+0000 7fac9e07e640 1 --2- 172.21.15.97:0/2979865210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba2ca71620 0x55ba2caef330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.275+0000 7fac9e07e640 1 -- 172.21.15.97:0/2979865210 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ba2ca71f40 msgr2=0x55ba2caef870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:36:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.275+0000 7fac9e07e640 1 --2- 172.21.15.97:0/2979865210 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ba2ca71f40 0x55ba2caef870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.275+0000 7fac9e07e640 1 -- 172.21.15.97:0/2979865210 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba2ca76c30 con 0x55ba2ca7c800 2024-04-23T10:36:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.275+0000 7fac9d87d640 1 --2- 172.21.15.97:0/2979865210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba2ca71620 0x55ba2caef330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:36:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.276+0000 7fac9e07e640 1 --2- 172.21.15.97:0/2979865210 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba2ca7c800 0x55ba2ca76540 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fac900118c0 tx=0x7fac90016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=167152d97a27980b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:36:18.282 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.276+0000 7fac817fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:36:18.282 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.276+0000 7fac817fa640 1 -- 172.21.15.97:0/2979865210 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac90004d20 con 0x55ba2ca7c800 2024-04-23T10:36:18.282 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.276+0000 7fac817fa640 1 -- 172.21.15.97:0/2979865210 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8484+0+0 (secure 0 0 0) 0x7fac900129d0 con 0x55ba2ca7c800 2024-04-23T10:36:18.282 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.276+0000 7faca27f7740 1 -- 172.21.15.97:0/2979865210 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac88028070 con 0x55ba2ca7c800 2024-04-23T10:36:18.282 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.276+0000 7fac817fa640 1 -- 172.21.15.97:0/2979865210 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fac90012bd0 con 0x55ba2ca7c800 2024-04-23T10:36:18.282 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.276+0000 7faca27f7740 10 client.17043 fetch_fsmap learned FSMap version 26 2024-04-23T10:36:18.282 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.276+0000 7faca27f7740 1 -- 172.21.15.97:0/2979865210 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ba2caf6a70 con 0x55ba2ca7c800 2024-04-23T10:36:18.282 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.276+0000 7fac817fa640 1 -- 172.21.15.97:0/2979865210 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac88028070 con 0x55ba2ca7c800 2024-04-23T10:36:18.282 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.276+0000 7fac817fa640 1 -- 172.21.15.97:0/2979865210 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fac90031020 con 0x55ba2ca7c800 2024-04-23T10:36:18.282 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.276+0000 7faca27f7740 10 client.17043 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:36:18.282 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.276+0000 7faca27f7740 10 client.17043 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:36:18.282 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.276+0000 7faca27f7740 20 client.17043 populate_metadata read hostname 'smithi097' 2024-04-23T10:36:18.282 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.276+0000 7faca27f7740 1 --2- 172.21.15.97:0/2979865210 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55ba2caf7250 0x55ba2caf9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:18.283 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.276+0000 7faca27f7740 4 client.17043 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:36:18.283 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.276+0000 7faca27f7740 1 -- 172.21.15.97:0/2979865210 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ba2c9de200 con 0x55ba2caf7250 2024-04-23T10:36:18.283 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.277+0000 7fac9d87d640 1 --2- 172.21.15.97:0/2979865210 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55ba2caf7250 0x55ba2caf9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:18.283 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.277+0000 7fac817fa640 10 client.17043 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:36:18.283 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.277+0000 7fac9d87d640 1 --2- 172.21.15.97:0/2979865210 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55ba2caf7250 0x55ba2caf9670 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd46a7197df1bcec server_cookie=acfa47b8930ee374 in_seq=0 out_seq=0 2024-04-23T10:36:18.290 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.291+0000 7fac817fa640 1 -- 172.21.15.97:0/2979865210 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ba2c9de200 con 0x55ba2caf7250 2024-04-23T10:36:18.290 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.291+0000 7fac817fa640 10 client.17043 handle_command_reply: tid=0 2024-04-23T10:36:18.291 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.291+0000 7faca27f7740 1 client.17043 shutdown 2024-04-23T10:36:18.291 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.291+0000 7faca27f7740 1 -- 172.21.15.97:0/2979865210 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba2ca7c800 msgr2=0x55ba2ca76540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:18.291 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.291+0000 7faca27f7740 1 --2- 172.21.15.97:0/2979865210 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba2ca7c800 0x55ba2ca76540 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fac900118c0 tx=0x7fac90016040 comp rx=0 tx=0).stop 2024-04-23T10:36:18.291 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.291+0000 7faca27f7740 1 -- 172.21.15.97:0/2979865210 shutdown_connections 2024-04-23T10:36:18.291 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.291+0000 7faca27f7740 1 --2- 172.21.15.97:0/2979865210 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55ba2caf7250 0x55ba2caf9670 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:18.291 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.291+0000 7faca27f7740 1 --2- 172.21.15.97:0/2979865210 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ba2ca7c800 0x55ba2ca76540 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:18.291 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.291+0000 7faca27f7740 1 --2- 172.21.15.97:0/2979865210 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ba2ca71f40 0x55ba2caef870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:18.291 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.291+0000 7fac817fa640 0 client.17043 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:36:18.291 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.291+0000 7faca27f7740 1 --2- 172.21.15.97:0/2979865210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ba2ca71620 0x55ba2caef330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:18.291 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.291+0000 7faca27f7740 1 -- 172.21.15.97:0/2979865210 >> 172.21.15.97:0/2979865210 conn(0x55ba2ca6db30 msgr2=0x55ba2ca6c420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:36:18.291 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.291+0000 7faca27f7740 1 -- 172.21.15.97:0/2979865210 shutdown_connections 2024-04-23T10:36:18.291 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.291+0000 7faca27f7740 1 -- 172.21.15.97:0/2979865210 wait complete. 2024-04-23T10:36:18.291 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.291+0000 7faca27f7740 20 client.17043 trim_cache size 0 max 16384 2024-04-23T10:36:18.292 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:36:18.292 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:36:18.292 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:36:18.292 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:36:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.292+0000 7fac9fae1640 1 -- 172.21.15.97:0/3921345968 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fac840a3dc0 msgr2=0x7fac840a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.292+0000 7fac9fae1640 1 --2- 172.21.15.97:0/3921345968 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fac840a3dc0 0x7fac840a6280 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fac90014580 tx=0x7fac90014980 comp rx=0 tx=0).stop 2024-04-23T10:36:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.292+0000 7fac9fae1640 1 -- 172.21.15.97:0/3921345968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac981072a0 msgr2=0x7fac98198b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.292+0000 7fac9fae1640 1 --2- 172.21.15.97:0/3921345968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac981072a0 0x7fac98198b40 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fac94031a90 tx=0x7fac94004270 comp rx=0 tx=0).stop 2024-04-23T10:36:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.292+0000 7fac9fae1640 1 -- 172.21.15.97:0/3921345968 shutdown_connections 2024-04-23T10:36:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.292+0000 7fac9fae1640 1 --2- 172.21.15.97:0/3921345968 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fac840a3dc0 0x7fac840a6280 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.292+0000 7fac9fae1640 1 --2- 172.21.15.97:0/3921345968 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fac9819ce70 0x7fac9819d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.292+0000 7fac9fae1640 1 --2- 172.21.15.97:0/3921345968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac981072a0 0x7fac98198b40 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.292+0000 7fac9fae1640 1 --2- 172.21.15.97:0/3921345968 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac981021d0 0x7fac98198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.292+0000 7fac9fae1640 1 -- 172.21.15.97:0/3921345968 >> 172.21.15.97:0/3921345968 conn(0x7fac980fda50 msgr2=0x7fac980fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:36:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.292+0000 7fac9fae1640 1 -- 172.21.15.97:0/3921345968 shutdown_connections 2024-04-23T10:36:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:18.292+0000 7fac9fae1640 1 -- 172.21.15.97:0/3921345968 wait complete. 2024-04-23T10:36:19.394 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:36:19 smithi196 ceph-mon[32919]: pgmap v314: 97 pgs: 97 active+clean; 2.2 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:36:19.399 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:36:19 smithi183 ceph-mon[32317]: pgmap v314: 97 pgs: 97 active+clean; 2.2 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:36:19.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:36:19 smithi097 ceph-mon[29197]: pgmap v314: 97 pgs: 97 active+clean; 2.2 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:36:21.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:36:21 smithi183 ceph-mon[32317]: pgmap v315: 97 pgs: 97 active+clean; 2.2 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:36:21.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:36:21 smithi196 ceph-mon[32919]: pgmap v315: 97 pgs: 97 active+clean; 2.2 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:36:21.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:36:21 smithi097 ceph-mon[29197]: pgmap v315: 97 pgs: 97 active+clean; 2.2 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:36:23.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:36:23 smithi183 ceph-mon[32317]: pgmap v316: 97 pgs: 97 active+clean; 2.2 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-23T10:36:23.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:36:23 smithi196 ceph-mon[32919]: pgmap v316: 97 pgs: 97 active+clean; 2.2 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-23T10:36:23.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:36:23 smithi097 ceph-mon[29197]: pgmap v316: 97 pgs: 97 active+clean; 2.2 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-23T10:36:24.053 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:36:24.054 INFO:tasks.workunit.client.0.smithi097.stdout:313/600: open fd = 3 2024-04-23T10:36:24.054 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.054 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.054 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:36:24.054 INFO:tasks.workunit.client.0.smithi097.stdout:314/600: open fd = 3 2024-04-23T10:36:24.054 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.054 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.054 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:36:24.054 INFO:tasks.workunit.client.0.smithi097.stdout:315/600: open fd = 3 2024-04-23T10:36:24.054 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.054 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.055 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:36:24.055 INFO:tasks.workunit.client.0.smithi097.stdout:316/600: open fd = 3 2024-04-23T10:36:24.055 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.055 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.055 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:36:24.055 INFO:tasks.workunit.client.0.smithi097.stdout:317/600: open fd = 3 2024-04-23T10:36:24.055 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.055 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.055 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:36:24.055 INFO:tasks.workunit.client.0.smithi097.stdout:318/600: open fd = 3 2024-04-23T10:36:24.055 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.055 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.055 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:36:24.055 INFO:tasks.workunit.client.0.smithi097.stdout:319/600: open fd = 3 2024-04-23T10:36:24.056 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.056 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.056 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:36:24.056 INFO:tasks.workunit.client.0.smithi097.stdout:320/600: open fd = 3 2024-04-23T10:36:24.056 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.056 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.056 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:36:24.056 INFO:tasks.workunit.client.0.smithi097.stdout:321/600: open fd = 3 2024-04-23T10:36:24.056 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.056 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.056 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:36:24.056 INFO:tasks.workunit.client.0.smithi097.stdout:322/600: open fd = 3 2024-04-23T10:36:24.056 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.056 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.057 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:36:24.057 INFO:tasks.workunit.client.0.smithi097.stdout:323/600: open fd = 3 2024-04-23T10:36:24.057 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.057 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.057 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:36:24.057 INFO:tasks.workunit.client.0.smithi097.stdout:324/600: open fd = 3 2024-04-23T10:36:24.057 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.057 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.057 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:36:24.057 INFO:tasks.workunit.client.0.smithi097.stdout:325/600: open fd = 3 2024-04-23T10:36:24.057 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.057 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.057 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:36:24.057 INFO:tasks.workunit.client.0.smithi097.stdout:326/600: open fd = 3 2024-04-23T10:36:24.057 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.058 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.058 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:36:24.058 INFO:tasks.workunit.client.0.smithi097.stdout:327/600: open fd = 3 2024-04-23T10:36:24.058 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.058 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.058 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:36:24.058 INFO:tasks.workunit.client.0.smithi097.stdout:328/600: open fd = 3 2024-04-23T10:36:24.058 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.058 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.058 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:36:24.058 INFO:tasks.workunit.client.0.smithi097.stdout:329/600: open fd = 3 2024-04-23T10:36:24.058 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.058 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.059 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:36:24.059 INFO:tasks.workunit.client.0.smithi097.stdout:330/600: open fd = 3 2024-04-23T10:36:24.059 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.059 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.059 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:36:24.059 INFO:tasks.workunit.client.0.smithi097.stdout:331/600: open fd = 3 2024-04-23T10:36:24.059 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.059 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.059 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:36:24.059 INFO:tasks.workunit.client.0.smithi097.stdout:332/600: open fd = 3 2024-04-23T10:36:24.059 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.059 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.059 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:36:24.059 INFO:tasks.workunit.client.0.smithi097.stdout:333/600: open fd = 3 2024-04-23T10:36:24.059 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.060 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.060 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:36:24.060 INFO:tasks.workunit.client.0.smithi097.stdout:334/600: open fd = 3 2024-04-23T10:36:24.060 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.060 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.060 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:36:24.060 INFO:tasks.workunit.client.0.smithi097.stdout:335/600: open fd = 3 2024-04-23T10:36:24.060 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.060 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.060 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:36:24.060 INFO:tasks.workunit.client.0.smithi097.stdout:336/600: open fd = 3 2024-04-23T10:36:24.060 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.060 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.060 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:36:24.060 INFO:tasks.workunit.client.0.smithi097.stdout:337/600: open fd = 3 2024-04-23T10:36:24.061 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.061 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.061 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:36:24.061 INFO:tasks.workunit.client.0.smithi097.stdout:338/600: open fd = 3 2024-04-23T10:36:24.061 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.061 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.061 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:36:24.061 INFO:tasks.workunit.client.0.smithi097.stdout:339/600: open fd = 3 2024-04-23T10:36:24.061 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.061 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.061 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:36:24.061 INFO:tasks.workunit.client.0.smithi097.stdout:340/600: open fd = 3 2024-04-23T10:36:24.061 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.061 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.062 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:36:24.062 INFO:tasks.workunit.client.0.smithi097.stdout:341/600: open fd = 3 2024-04-23T10:36:24.062 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.062 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.062 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:36:24.062 INFO:tasks.workunit.client.0.smithi097.stdout:342/600: open fd = 3 2024-04-23T10:36:24.062 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.062 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.062 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:36:24.062 INFO:tasks.workunit.client.0.smithi097.stdout:343/600: open fd = 3 2024-04-23T10:36:24.062 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.062 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.062 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:36:24.062 INFO:tasks.workunit.client.0.smithi097.stdout:344/600: open fd = 3 2024-04-23T10:36:24.062 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.063 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.063 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:36:24.063 INFO:tasks.workunit.client.0.smithi097.stdout:345/600: open fd = 3 2024-04-23T10:36:24.063 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.063 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.063 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:36:24.063 INFO:tasks.workunit.client.0.smithi097.stdout:346/600: open fd = 3 2024-04-23T10:36:24.063 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.063 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.063 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:36:24.063 INFO:tasks.workunit.client.0.smithi097.stdout:347/600: open fd = 3 2024-04-23T10:36:24.063 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.063 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.063 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:36:24.063 INFO:tasks.workunit.client.0.smithi097.stdout:348/600: open fd = 3 2024-04-23T10:36:24.064 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.064 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.064 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:36:24.064 INFO:tasks.workunit.client.0.smithi097.stdout:349/600: open fd = 3 2024-04-23T10:36:24.064 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.064 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.064 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:36:24.064 INFO:tasks.workunit.client.0.smithi097.stdout:350/600: open fd = 3 2024-04-23T10:36:24.064 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.064 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.064 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:36:24.064 INFO:tasks.workunit.client.0.smithi097.stdout:351/600: open fd = 3 2024-04-23T10:36:24.064 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.064 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.064 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:36:24.065 INFO:tasks.workunit.client.0.smithi097.stdout:352/600: open fd = 3 2024-04-23T10:36:24.065 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.065 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.065 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:36:24.065 INFO:tasks.workunit.client.0.smithi097.stdout:353/600: open fd = 3 2024-04-23T10:36:24.065 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.065 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.065 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:36:24.065 INFO:tasks.workunit.client.0.smithi097.stdout:354/600: open fd = 3 2024-04-23T10:36:24.065 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.065 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.065 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:36:24.065 INFO:tasks.workunit.client.0.smithi097.stdout:355/600: open fd = 3 2024-04-23T10:36:24.065 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.065 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.065 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:36:24.066 INFO:tasks.workunit.client.0.smithi097.stdout:356/600: open fd = 3 2024-04-23T10:36:24.066 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.066 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.066 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:36:24.066 INFO:tasks.workunit.client.0.smithi097.stdout:357/600: open fd = 3 2024-04-23T10:36:24.066 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.066 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.066 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:36:24.066 INFO:tasks.workunit.client.0.smithi097.stdout:358/600: open fd = 3 2024-04-23T10:36:24.066 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.066 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.066 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:36:24.066 INFO:tasks.workunit.client.0.smithi097.stdout:359/600: open fd = 3 2024-04-23T10:36:24.066 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.066 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.067 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:36:24.067 INFO:tasks.workunit.client.0.smithi097.stdout:360/600: open fd = 3 2024-04-23T10:36:24.067 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.067 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.067 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:36:24.067 INFO:tasks.workunit.client.0.smithi097.stdout:361/600: open fd = 3 2024-04-23T10:36:24.067 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.067 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.067 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:36:24.067 INFO:tasks.workunit.client.0.smithi097.stdout:362/600: open fd = 3 2024-04-23T10:36:24.067 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.067 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.067 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:36:24.067 INFO:tasks.workunit.client.0.smithi097.stdout:363/600: open fd = 3 2024-04-23T10:36:24.067 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.068 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.068 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:36:24.068 INFO:tasks.workunit.client.0.smithi097.stdout:364/600: open fd = 3 2024-04-23T10:36:24.068 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.068 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.068 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:36:24.068 INFO:tasks.workunit.client.0.smithi097.stdout:365/600: open fd = 3 2024-04-23T10:36:24.068 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.068 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.068 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:36:24.068 INFO:tasks.workunit.client.0.smithi097.stdout:366/600: open fd = 3 2024-04-23T10:36:24.068 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.068 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.068 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:36:24.068 INFO:tasks.workunit.client.0.smithi097.stdout:367/600: open fd = 3 2024-04-23T10:36:24.068 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.069 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.069 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:36:24.069 INFO:tasks.workunit.client.0.smithi097.stdout:368/600: open fd = 3 2024-04-23T10:36:24.069 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.069 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.069 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:36:24.069 INFO:tasks.workunit.client.0.smithi097.stdout:369/600: open fd = 3 2024-04-23T10:36:24.069 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.069 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.069 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:36:24.069 INFO:tasks.workunit.client.0.smithi097.stdout:370/600: open fd = 3 2024-04-23T10:36:24.069 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.069 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.069 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:36:24.069 INFO:tasks.workunit.client.0.smithi097.stdout:371/600: open fd = 3 2024-04-23T10:36:24.069 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.070 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.070 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:36:24.070 INFO:tasks.workunit.client.0.smithi097.stdout:372/600: open fd = 3 2024-04-23T10:36:24.070 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.070 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.070 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:36:24.070 INFO:tasks.workunit.client.0.smithi097.stdout:373/600: open fd = 3 2024-04-23T10:36:24.070 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.070 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.070 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:36:24.070 INFO:tasks.workunit.client.0.smithi097.stdout:374/600: open fd = 3 2024-04-23T10:36:24.070 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.070 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:36:24.070 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:36:25.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:36:25 smithi097 ceph-mon[29197]: pgmap v317: 97 pgs: 97 active+clean; 2.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-23T10:36:25.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:36:25 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:36:25.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:36:25 smithi183 ceph-mon[32317]: pgmap v317: 97 pgs: 97 active+clean; 2.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-23T10:36:25.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:36:25 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:36:25.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:36:25 smithi196 ceph-mon[32919]: pgmap v317: 97 pgs: 97 active+clean; 2.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-23T10:36:25.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:36:25 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:36:27.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:36:26 smithi196 ceph-mon[32919]: pgmap v318: 97 pgs: 97 active+clean; 2.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-23T10:36:27.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:36:26 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:36:27.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:36:26 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:36:27.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:36:26 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:36:27.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:36:26 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:36:27.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:36:26 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:36:27.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:36:26 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:36:27.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:36:26 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:36:27.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:36:26 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:36:27.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:36:26 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:36:27.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:36:26 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:36:27.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:36:26 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:36:27.343 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:36:26 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:36:27.344 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:36:26 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:36:27.344 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:36:26 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:36:27.344 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:36:26 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:36:27.344 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:36:26 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:36:27.344 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:36:26 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:36:27.344 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:36:26 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:36:27.344 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:36:26 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:36:27.344 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:36:26 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:36:27.345 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:36:26 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:36:27.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:36:26 smithi097 ceph-mon[29197]: pgmap v318: 97 pgs: 97 active+clean; 2.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-23T10:36:27.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:36:26 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:36:27.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:36:26 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:36:27.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:36:26 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:36:27.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:36:26 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:36:27.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:36:26 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:36:27.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:36:26 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:36:27.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:36:26 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:36:27.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:36:26 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:36:27.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:36:26 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:36:27.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:36:26 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:36:27.362 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:36:26 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:36:27.362 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:36:26 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:36:27.362 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:36:26 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:36:27.362 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:36:26 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:36:27.362 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:36:26 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:36:27.362 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:36:26 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:36:27.362 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:36:26 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:36:27.362 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:36:26 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:36:27.362 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:36:26 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:36:27.362 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:36:26 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:36:27.362 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:36:26 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:36:27.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:36:26 smithi183 ceph-mon[32317]: pgmap v318: 97 pgs: 97 active+clean; 2.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-23T10:36:27.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:36:26 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:36:27.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:36:26 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:36:27.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:36:26 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:36:27.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:36:26 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:36:27.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:36:26 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:36:27.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:36:26 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:36:27.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:36:26 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:36:27.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:36:26 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:36:27.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:36:26 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:36:27.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:36:26 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:36:27.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:36:26 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:36:27.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:36:26 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:36:27.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:36:26 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:36:27.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:36:26 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:36:27.401 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:36:26 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:36:27.402 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:36:26 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:36:27.402 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:36:26 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:36:27.402 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:36:26 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T10:36:27.402 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:36:26 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:36:27.402 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:36:26 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:36:27.402 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:36:26 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:36:28.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:36:27 smithi196 ceph-mon[32919]: Adjusting osd_memory_target on smithi097 to 176.2M 2024-04-23T10:36:28.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:36:27 smithi196 ceph-mon[32919]: Adjusting osd_memory_target on smithi183 to 176.2M 2024-04-23T10:36:28.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:36:27 smithi196 ceph-mon[32919]: Unable to set osd_memory_target on smithi097 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-04-23T10:36:28.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:36:27 smithi196 ceph-mon[32919]: Unable to set osd_memory_target on smithi183 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-23T10:36:28.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:36:27 smithi196 ceph-mon[32919]: Adjusting osd_memory_target on smithi196 to 176.2M 2024-04-23T10:36:28.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:36:27 smithi196 ceph-mon[32919]: Unable to set osd_memory_target on smithi196 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-04-23T10:36:28.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:36:27 smithi097 ceph-mon[29197]: Adjusting osd_memory_target on smithi097 to 176.2M 2024-04-23T10:36:28.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:36:27 smithi097 ceph-mon[29197]: Adjusting osd_memory_target on smithi183 to 176.2M 2024-04-23T10:36:28.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:36:27 smithi097 ceph-mon[29197]: Unable to set osd_memory_target on smithi097 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-04-23T10:36:28.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:36:27 smithi097 ceph-mon[29197]: Unable to set osd_memory_target on smithi183 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-23T10:36:28.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:36:27 smithi097 ceph-mon[29197]: Adjusting osd_memory_target on smithi196 to 176.2M 2024-04-23T10:36:28.361 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:36:27 smithi097 ceph-mon[29197]: Unable to set osd_memory_target on smithi196 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-04-23T10:36:28.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:36:27 smithi183 ceph-mon[32317]: Adjusting osd_memory_target on smithi097 to 176.2M 2024-04-23T10:36:28.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:36:27 smithi183 ceph-mon[32317]: Adjusting osd_memory_target on smithi183 to 176.2M 2024-04-23T10:36:28.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:36:27 smithi183 ceph-mon[32317]: Unable to set osd_memory_target on smithi097 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-04-23T10:36:28.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:36:27 smithi183 ceph-mon[32317]: Unable to set osd_memory_target on smithi183 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-23T10:36:28.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:36:27 smithi183 ceph-mon[32317]: Adjusting osd_memory_target on smithi196 to 176.2M 2024-04-23T10:36:28.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:36:27 smithi183 ceph-mon[32317]: Unable to set osd_memory_target on smithi196 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-04-23T10:36:29.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:36:28 smithi196 ceph-mon[32919]: pgmap v319: 97 pgs: 97 active+clean; 2.6 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-23T10:36:29.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:36:28 smithi097 ceph-mon[29197]: pgmap v319: 97 pgs: 97 active+clean; 2.6 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-23T10:36:29.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:36:28 smithi183 ceph-mon[32317]: pgmap v319: 97 pgs: 97 active+clean; 2.6 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-23T10:36:31.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:36:30 smithi196 ceph-mon[32919]: pgmap v320: 97 pgs: 97 active+clean; 2.6 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:36:31.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:36:30 smithi097 ceph-mon[29197]: pgmap v320: 97 pgs: 97 active+clean; 2.6 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:36:31.386 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.387+0000 7f5ab229e640 1 -- 172.21.15.97:0/4102287510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5aac107290 msgr2=0x7f5aac10ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:31.387 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.387+0000 7f5ab229e640 1 --2- 172.21.15.97:0/4102287510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5aac107290 0x7f5aac10ee60 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4009ca0 tx=0x7f5aa402a8c0 comp rx=0 tx=0).stop 2024-04-23T10:36:31.387 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.387+0000 7f5ab229e640 1 -- 172.21.15.97:0/4102287510 shutdown_connections 2024-04-23T10:36:31.387 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.387+0000 7f5ab229e640 1 --2- 172.21.15.97:0/4102287510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5aac107290 0x7f5aac10ee60 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:31.387 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.387+0000 7f5ab229e640 1 --2- 172.21.15.97:0/4102287510 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5aac102b70 0x7f5aac106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:31.387 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.387+0000 7f5ab229e640 1 --2- 172.21.15.97:0/4102287510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5aac1021c0 0x7f5aac1025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:31.387 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.387+0000 7f5ab229e640 1 -- 172.21.15.97:0/4102287510 >> 172.21.15.97:0/4102287510 conn(0x7f5aac0fda20 msgr2=0x7f5aac0ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:36:31.387 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.387+0000 7f5ab229e640 1 -- 172.21.15.97:0/4102287510 shutdown_connections 2024-04-23T10:36:31.387 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.387+0000 7f5ab229e640 1 -- 172.21.15.97:0/4102287510 wait complete. 2024-04-23T10:36:31.387 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.388+0000 7f5ab229e640 1 Processor -- start 2024-04-23T10:36:31.387 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.388+0000 7f5ab229e640 1 -- start start 2024-04-23T10:36:31.389 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.388+0000 7f5ab229e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5aac1021c0 0x7f5aac1a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:31.389 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.388+0000 7f5ab229e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5aac107290 0x7f5aac1a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:31.389 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.388+0000 7f5ab229e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5aac19a690 0x7f5aac19ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:31.389 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.388+0000 7f5ab229e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5aac112200 con 0x7f5aac19a690 2024-04-23T10:36:31.389 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.388+0000 7f5ab229e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5aac112080 con 0x7f5aac107290 2024-04-23T10:36:31.389 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.388+0000 7f5ab229e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5aac112380 con 0x7f5aac1021c0 2024-04-23T10:36:31.390 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.389+0000 7f5ab083b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5aac19a690 0x7f5aac19ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:31.390 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.389+0000 7f5aabfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5aac1021c0 0x7f5aac1a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:31.390 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.389+0000 7f5ab083b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5aac19a690 0x7f5aac19ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36672/0 (socket says 172.21.15.97:36672) 2024-04-23T10:36:31.390 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.389+0000 7f5ab083b640 1 -- 172.21.15.97:0/1827090459 learned_addr learned my addr 172.21.15.97:0/1827090459 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:36:31.390 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.389+0000 7f5aab7fe640 1 --2- 172.21.15.97:0/1827090459 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5aac107290 0x7f5aac1a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:31.390 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.389+0000 7f5ab083b640 1 -- 172.21.15.97:0/1827090459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5aac1021c0 msgr2=0x7f5aac1a05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:31.390 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.389+0000 7f5ab083b640 1 --2- 172.21.15.97:0/1827090459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5aac1021c0 0x7f5aac1a05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:31.390 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.389+0000 7f5ab083b640 1 -- 172.21.15.97:0/1827090459 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5aac107290 msgr2=0x7f5aac1a0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:31.390 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.389+0000 7f5ab083b640 1 --2- 172.21.15.97:0/1827090459 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5aac107290 0x7f5aac1a0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:31.390 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.389+0000 7f5ab083b640 1 -- 172.21.15.97:0/1827090459 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5aac19b290 con 0x7f5aac19a690 2024-04-23T10:36:31.390 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.389+0000 7f5aabfff640 1 --2- 172.21.15.97:0/1827090459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5aac1021c0 0x7f5aac1a05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T10:36:31.390 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.389+0000 7f5ab083b640 1 --2- 172.21.15.97:0/1827090459 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5aac19a690 0x7f5aac19ab10 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4005cc0 tx=0x7f5aa402f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:36:31.390 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.389+0000 7f5aa97fa640 1 -- 172.21.15.97:0/1827090459 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa4003bf0 con 0x7f5aac19a690 2024-04-23T10:36:31.390 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.389+0000 7f5ab229e640 1 -- 172.21.15.97:0/1827090459 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5aac19b520 con 0x7f5aac19a690 2024-04-23T10:36:31.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.389+0000 7f5aa97fa640 1 -- 172.21.15.97:0/1827090459 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5aa4003d90 con 0x7f5aac19a690 2024-04-23T10:36:31.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.389+0000 7f5aa97fa640 1 -- 172.21.15.97:0/1827090459 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa402fbc0 con 0x7f5aac19a690 2024-04-23T10:36:31.391 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.389+0000 7f5ab229e640 1 -- 172.21.15.97:0/1827090459 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5aac1a76c0 con 0x7f5aac19a690 2024-04-23T10:36:31.397 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.392+0000 7f5aa97fa640 1 -- 172.21.15.97:0/1827090459 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5aa402fd60 con 0x7f5aac19a690 2024-04-23T10:36:31.397 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.392+0000 7f5ab4fb0740 1 Processor -- start 2024-04-23T10:36:31.397 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.392+0000 7f5ab4fb0740 1 -- start start 2024-04-23T10:36:31.397 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.392+0000 7f5aa97fa640 1 --2- 172.21.15.97:0/1827090459 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f5a940a3dc0 0x7f5a940a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:31.397 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.392+0000 7f5ab4fb0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a2bb6b620 0x558a2bb6ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:31.398 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.392+0000 7f5ab4fb0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a2bb6bf40 0x558a2bb762c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:31.398 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.392+0000 7f5ab4fb0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558a2bb76800 0x558a2bb78bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:31.398 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.392+0000 7f5ab4fb0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x558a2bad0970 con 0x558a2bb76800 2024-04-23T10:36:31.398 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.392+0000 7f5ab4fb0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558a2ba6d4a0 con 0x558a2bb6b620 2024-04-23T10:36:31.398 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.392+0000 7f5ab4fb0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558a2bacc930 con 0x558a2bb6bf40 2024-04-23T10:36:31.398 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.392+0000 7f5aab7fe640 1 --2- 172.21.15.97:0/1827090459 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f5a940a3dc0 0x7f5a940a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:31.398 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.392+0000 7f5ab083b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558a2bb76800 0x558a2bb78bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:31.398 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.393+0000 7f5ab083b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558a2bb76800 0x558a2bb78bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36688/0 (socket says 172.21.15.97:36688) 2024-04-23T10:36:31.398 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.393+0000 7f5ab083b640 1 -- 172.21.15.97:0/2639708243 learned_addr learned my addr 172.21.15.97:0/2639708243 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:36:31.398 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.393+0000 7f5aa97fa640 1 -- 172.21.15.97:0/1827090459 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8484+0+0 (secure 0 0 0) 0x7f5aa40f1d40 con 0x7f5aac19a690 2024-04-23T10:36:31.398 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.393+0000 7f5aabfff640 1 --2- 172.21.15.97:0/2639708243 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a2bb6b620 0x558a2bb6ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:31.398 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.393+0000 7f5aab7fe640 1 --2- 172.21.15.97:0/2639708243 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a2bb6bf40 0x558a2bb762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:31.398 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.393+0000 7f5ab083b640 1 -- 172.21.15.97:0/2639708243 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a2bb6bf40 msgr2=0x558a2bb762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:36:31.398 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.393+0000 7f5ab083b640 1 --2- 172.21.15.97:0/2639708243 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a2bb6bf40 0x558a2bb762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:31.399 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.393+0000 7f5ab083b640 1 -- 172.21.15.97:0/2639708243 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a2bb6b620 msgr2=0x558a2bb6ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:36:31.399 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.393+0000 7f5ab083b640 1 --2- 172.21.15.97:0/2639708243 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a2bb6b620 0x558a2bb6ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:31.399 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.393+0000 7f5ab083b640 1 -- 172.21.15.97:0/2639708243 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a2b7e3210 con 0x558a2bb76800 2024-04-23T10:36:31.399 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.393+0000 7f5aabfff640 1 --2- 172.21.15.97:0/2639708243 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a2bb6b620 0x558a2bb6ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:36:31.399 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.393+0000 7f5aab7fe640 1 --2- 172.21.15.97:0/1827090459 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f5a940a3dc0 0x7f5a940a6280 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5aac19bee0 tx=0x7f5aa000bdf0 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:36:31.399 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.393+0000 7f5ab083b640 1 --2- 172.21.15.97:0/2639708243 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558a2bb76800 0x558a2bb78bf0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4005e30 tx=0x7f5aa4003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c2c2cb3ffa5c90d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:36:31.399 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.393+0000 7f5a927fc640 1 -- 172.21.15.97:0/2639708243 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa40422e0 con 0x558a2bb76800 2024-04-23T10:36:31.399 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.393+0000 7f5a927fc640 1 -- 172.21.15.97:0/2639708243 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5aa4036070 con 0x558a2bb76800 2024-04-23T10:36:31.399 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.393+0000 7f5a927fc640 1 -- 172.21.15.97:0/2639708243 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa40bc110 con 0x558a2bb76800 2024-04-23T10:36:31.399 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.394+0000 7f5ab4fb0740 1 -- 172.21.15.97:0/2639708243 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558a2bb76800 msgr2=0x558a2bb78bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:31.399 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.394+0000 7f5ab4fb0740 1 --2- 172.21.15.97:0/2639708243 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558a2bb76800 0x558a2bb78bf0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4005e30 tx=0x7f5aa4003f40 comp rx=0 tx=0).stop 2024-04-23T10:36:31.399 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.394+0000 7f5ab4fb0740 1 -- 172.21.15.97:0/2639708243 shutdown_connections 2024-04-23T10:36:31.399 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.394+0000 7f5ab4fb0740 1 --2- 172.21.15.97:0/2639708243 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558a2bb76800 0x558a2bb78bf0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:31.399 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.394+0000 7f5ab4fb0740 1 --2- 172.21.15.97:0/2639708243 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a2bb6bf40 0x558a2bb762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.394+0000 7f5ab4fb0740 1 --2- 172.21.15.97:0/2639708243 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a2bb6b620 0x558a2bb6ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.394+0000 7f5ab4fb0740 1 -- 172.21.15.97:0/2639708243 >> 172.21.15.97:0/2639708243 conn(0x558a2bb67b30 msgr2=0x558a2bb67f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:36:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.394+0000 7f5ab4fb0740 1 -- 172.21.15.97:0/2639708243 shutdown_connections 2024-04-23T10:36:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.394+0000 7f5ab4fb0740 1 -- 172.21.15.97:0/2639708243 wait complete. 2024-04-23T10:36:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.394+0000 7f5ab4fb0740 1 -- start start 2024-04-23T10:36:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.395+0000 7f5ab4fb0740 1 Processor -- start 2024-04-23T10:36:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.395+0000 7f5ab4fb0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a2bb6b620 0x558a2bb76170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.395+0000 7f5ab4fb0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558a2bb6bf40 0x558a2bb73460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.395+0000 7f5ab4fb0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a2bb76800 0x558a2bb739c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.395+0000 7f5ab4fb0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x558a2bad0970 con 0x558a2bb6bf40 2024-04-23T10:36:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.395+0000 7f5ab4fb0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558a2ba6b0b0 con 0x558a2bb6b620 2024-04-23T10:36:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.395+0000 7f5ab4fb0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558a2bb791f0 con 0x558a2bb76800 2024-04-23T10:36:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.395+0000 7f5ab4fb0740 10 client.0 _ll_register_callbacks cb 0x558a2ba58d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:36:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.395+0000 7f5aab7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558a2bb6bf40 0x558a2bb73460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:31.400 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.395+0000 7f5aab7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558a2bb6bf40 0x558a2bb73460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36690/0 (socket says 172.21.15.97:36690) 2024-04-23T10:36:31.401 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.395+0000 7f5aab7fe640 1 -- 172.21.15.97:0/4073447576 learned_addr learned my addr 172.21.15.97:0/4073447576 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:36:31.401 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.395+0000 7f5ab083b640 1 --2- 172.21.15.97:0/4073447576 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a2bb76800 0x558a2bb739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:31.401 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.395+0000 7f5aabfff640 1 --2- 172.21.15.97:0/4073447576 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a2bb6b620 0x558a2bb76170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:31.401 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.395+0000 7f5aab7fe640 1 -- 172.21.15.97:0/4073447576 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a2bb76800 msgr2=0x558a2bb739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:36:31.401 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.395+0000 7f5aab7fe640 1 --2- 172.21.15.97:0/4073447576 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a2bb76800 0x558a2bb739c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:31.401 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.395+0000 7f5aab7fe640 1 -- 172.21.15.97:0/4073447576 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a2bb6b620 msgr2=0x558a2bb76170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:36:31.401 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.395+0000 7f5aab7fe640 1 --2- 172.21.15.97:0/4073447576 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a2bb6b620 0x558a2bb76170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:31.401 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.395+0000 7f5aab7fe640 1 -- 172.21.15.97:0/4073447576 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a2b8af620 con 0x558a2bb6bf40 2024-04-23T10:36:31.401 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.395+0000 7f5aab7fe640 1 --2- 172.21.15.97:0/4073447576 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558a2bb6bf40 0x558a2bb73460 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f5aa00126a0 tx=0x7f5aa0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c85a3ed818421070 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:36:31.401 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.395+0000 7f5a8b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:36:31.401 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.396+0000 7f5a8b7fe640 1 -- 172.21.15.97:0/4073447576 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa0005220 con 0x558a2bb6bf40 2024-04-23T10:36:31.401 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.396+0000 7f5a8b7fe640 1 -- 172.21.15.97:0/4073447576 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8484+0+0 (secure 0 0 0) 0x7f5aa00053c0 con 0x558a2bb6bf40 2024-04-23T10:36:31.401 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.396+0000 7f5ab4fb0740 1 -- 172.21.15.97:0/4073447576 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5aa4036070 con 0x558a2bb6bf40 2024-04-23T10:36:31.401 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.396+0000 7f5a8b7fe640 1 -- 172.21.15.97:0/4073447576 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5aa0005600 con 0x558a2bb6bf40 2024-04-23T10:36:31.401 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.396+0000 7f5ab4fb0740 10 client.17055 fetch_fsmap learned FSMap version 26 2024-04-23T10:36:31.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.396+0000 7f5ab4fb0740 1 -- 172.21.15.97:0/4073447576 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558a2bb84f10 con 0x558a2bb6bf40 2024-04-23T10:36:31.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.396+0000 7f5a8b7fe640 1 -- 172.21.15.97:0/4073447576 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5aa4036070 con 0x558a2bb6bf40 2024-04-23T10:36:31.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.396+0000 7f5a8b7fe640 1 -- 172.21.15.97:0/4073447576 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5aa002e020 con 0x558a2bb6bf40 2024-04-23T10:36:31.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.396+0000 7f5ab4fb0740 10 client.17055 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:36:31.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.396+0000 7f5ab4fb0740 10 client.17055 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:36:31.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.396+0000 7f5ab4fb0740 20 client.17055 populate_metadata read hostname 'smithi097' 2024-04-23T10:36:31.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.396+0000 7f5ab4fb0740 1 --2- 172.21.15.97:0/4073447576 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x558a2bb86440 0x558a2bb88830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:31.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.396+0000 7f5ab4fb0740 4 client.17055 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:36:31.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.396+0000 7f5ab4fb0740 1 -- 172.21.15.97:0/4073447576 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a2bade390 con 0x558a2bb86440 2024-04-23T10:36:31.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.396+0000 7f5aabfff640 1 --2- 172.21.15.97:0/4073447576 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x558a2bb86440 0x558a2bb88830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:31.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.397+0000 7f5aabfff640 1 --2- 172.21.15.97:0/4073447576 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x558a2bb86440 0x558a2bb88830 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45c5bf6d5c1f6aac server_cookie=82cce1faa6ded39f in_seq=0 out_seq=0 2024-04-23T10:36:31.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.397+0000 7f5a8b7fe640 10 client.17055 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:36:31.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.397+0000 7f5a8b7fe640 1 -- 172.21.15.97:0/4073447576 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558a2bade390 con 0x558a2bb86440 2024-04-23T10:36:31.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.397+0000 7f5a8b7fe640 10 client.17055 handle_command_reply: tid=0 2024-04-23T10:36:31.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.398+0000 7f5ab4fb0740 1 client.17055 shutdown 2024-04-23T10:36:31.403 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.398+0000 7f5ab4fb0740 1 -- 172.21.15.97:0/4073447576 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558a2bb6bf40 msgr2=0x558a2bb73460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:31.403 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.398+0000 7f5ab4fb0740 1 --2- 172.21.15.97:0/4073447576 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558a2bb6bf40 0x558a2bb73460 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f5aa00126a0 tx=0x7f5aa0004730 comp rx=0 tx=0).stop 2024-04-23T10:36:31.403 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.398+0000 7f5ab4fb0740 1 -- 172.21.15.97:0/4073447576 shutdown_connections 2024-04-23T10:36:31.403 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.398+0000 7f5ab4fb0740 1 --2- 172.21.15.97:0/4073447576 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x558a2bb86440 0x558a2bb88830 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:31.403 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.398+0000 7f5ab4fb0740 1 --2- 172.21.15.97:0/4073447576 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a2bb76800 0x558a2bb739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:31.403 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.398+0000 7f5ab4fb0740 1 --2- 172.21.15.97:0/4073447576 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558a2bb6bf40 0x558a2bb73460 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:31.403 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.398+0000 7f5ab4fb0740 1 --2- 172.21.15.97:0/4073447576 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a2bb6b620 0x558a2bb76170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:31.403 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.398+0000 7f5a8b7fe640 0 client.17055 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:36:31.403 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.398+0000 7f5ab4fb0740 1 -- 172.21.15.97:0/4073447576 >> 172.21.15.97:0/4073447576 conn(0x558a2bb67b30 msgr2=0x558a2bb6d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:36:31.403 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.398+0000 7f5ab4fb0740 1 -- 172.21.15.97:0/4073447576 shutdown_connections 2024-04-23T10:36:31.403 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.398+0000 7f5ab4fb0740 1 -- 172.21.15.97:0/4073447576 wait complete. 2024-04-23T10:36:31.403 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.398+0000 7f5ab4fb0740 20 client.17055 trim_cache size 0 max 16384 2024-04-23T10:36:31.404 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:36:30 smithi183 ceph-mon[32317]: pgmap v320: 97 pgs: 97 active+clean; 2.6 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:36:31.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.413+0000 7f5ab4fb0740 1 Processor -- start 2024-04-23T10:36:31.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.413+0000 7f5ab4fb0740 1 -- start start 2024-04-23T10:36:31.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.413+0000 7f5ab4fb0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a2bb6b620 0x558a2bb81ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:31.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.413+0000 7f5ab4fb0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558a2bb6bf40 0x558a2bb6e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:31.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.413+0000 7f5ab4fb0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a2bb76800 0x558a2bb75ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:31.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.413+0000 7f5ab4fb0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5aac1afc00 con 0x558a2bb6bf40 2024-04-23T10:36:31.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.413+0000 7f5ab4fb0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5aac1b1740 con 0x558a2bb76800 2024-04-23T10:36:31.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.413+0000 7f5ab4fb0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5aac1b02d0 con 0x558a2bb6b620 2024-04-23T10:36:31.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.413+0000 7f5aab7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558a2bb6bf40 0x558a2bb6e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:31.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.413+0000 7f5aab7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558a2bb6bf40 0x558a2bb6e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36698/0 (socket says 172.21.15.97:36698) 2024-04-23T10:36:31.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.413+0000 7f5aab7fe640 1 -- 172.21.15.97:0/1806551575 learned_addr learned my addr 172.21.15.97:0/1806551575 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:36:31.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.413+0000 7f5aabfff640 1 --2- 172.21.15.97:0/1806551575 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a2bb6b620 0x558a2bb81ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:31.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.413+0000 7f5aab7fe640 1 -- 172.21.15.97:0/1806551575 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a2bb6b620 msgr2=0x558a2bb81ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:36:31.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.413+0000 7f5aab7fe640 1 --2- 172.21.15.97:0/1806551575 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a2bb6b620 0x558a2bb81ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:31.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.413+0000 7f5aab7fe640 1 -- 172.21.15.97:0/1806551575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a2bb76800 msgr2=0x558a2bb75ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:36:31.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.413+0000 7f5aab7fe640 1 --2- 172.21.15.97:0/1806551575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a2bb76800 0x558a2bb75ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:31.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.413+0000 7f5aab7fe640 1 -- 172.21.15.97:0/1806551575 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a2bb73d30 con 0x558a2bb6bf40 2024-04-23T10:36:31.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.413+0000 7f5aab7fe640 1 --2- 172.21.15.97:0/1806551575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558a2bb6bf40 0x558a2bb6e850 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f5aa00057c0 tx=0x7f5aa0003a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f115c70ed52c00c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:36:31.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.414+0000 7f5a91ffb640 1 -- 172.21.15.97:0/1806551575 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa00292f0 con 0x558a2bb6bf40 2024-04-23T10:36:31.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.414+0000 7f5a91ffb640 1 -- 172.21.15.97:0/1806551575 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5aa0029680 con 0x558a2bb6bf40 2024-04-23T10:36:31.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.414+0000 7f5a91ffb640 1 -- 172.21.15.97:0/1806551575 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa00306c0 con 0x558a2bb6bf40 2024-04-23T10:36:31.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.414+0000 7f5ab4fb0740 1 -- 172.21.15.97:0/1806551575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558a2bb6bf40 msgr2=0x558a2bb6e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:31.417 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.414+0000 7f5ab4fb0740 1 --2- 172.21.15.97:0/1806551575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558a2bb6bf40 0x558a2bb6e850 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f5aa00057c0 tx=0x7f5aa0003a10 comp rx=0 tx=0).stop 2024-04-23T10:36:31.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.414+0000 7f5ab4fb0740 1 -- 172.21.15.97:0/1806551575 shutdown_connections 2024-04-23T10:36:31.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.414+0000 7f5ab4fb0740 1 --2- 172.21.15.97:0/1806551575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a2bb76800 0x558a2bb75ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:31.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.414+0000 7f5ab4fb0740 1 --2- 172.21.15.97:0/1806551575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558a2bb6bf40 0x558a2bb6e850 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:31.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.414+0000 7f5ab4fb0740 1 --2- 172.21.15.97:0/1806551575 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a2bb6b620 0x558a2bb81ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:31.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.414+0000 7f5ab4fb0740 1 -- 172.21.15.97:0/1806551575 >> 172.21.15.97:0/1806551575 conn(0x558a2bb67b30 msgr2=0x558a2bb6e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:36:31.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.414+0000 7f5ab4fb0740 1 -- 172.21.15.97:0/1806551575 shutdown_connections 2024-04-23T10:36:31.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.414+0000 7f5ab4fb0740 1 -- 172.21.15.97:0/1806551575 wait complete. 2024-04-23T10:36:31.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.415+0000 7f5ab4fb0740 1 -- start start 2024-04-23T10:36:31.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.415+0000 7f5ab4fb0740 1 Processor -- start 2024-04-23T10:36:31.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.415+0000 7f5ab4fb0740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558a2bb6b620 0x558a2bb7e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:31.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.415+0000 7f5ab4fb0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a2bb6bf40 0x558a2bb6fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:31.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.415+0000 7f5ab4fb0740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a2bb76800 0x558a2bb70310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:31.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.415+0000 7f5ab4fb0740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5aac1b1070 con 0x558a2bb6b620 2024-04-23T10:36:31.418 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.415+0000 7f5ab4fb0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5aac1b09a0 con 0x558a2bb6bf40 2024-04-23T10:36:31.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.415+0000 7f5ab4fb0740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5aac1adc00 con 0x558a2bb76800 2024-04-23T10:36:31.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.415+0000 7f5ab4fb0740 10 client.0 _ll_register_callbacks cb 0x7f5aa0003d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:36:31.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.415+0000 7f5aabfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558a2bb6b620 0x558a2bb7e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:31.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.415+0000 7f5aabfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558a2bb6b620 0x558a2bb7e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36708/0 (socket says 172.21.15.97:36708) 2024-04-23T10:36:31.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.415+0000 7f5aabfff640 1 -- 172.21.15.97:0/288223263 learned_addr learned my addr 172.21.15.97:0/288223263 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:36:31.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.415+0000 7f5aab7fe640 1 --2- 172.21.15.97:0/288223263 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a2bb6bf40 0x558a2bb6fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:31.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.415+0000 7f5ab083b640 1 --2- 172.21.15.97:0/288223263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a2bb76800 0x558a2bb70310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:31.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.415+0000 7f5aabfff640 1 -- 172.21.15.97:0/288223263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a2bb76800 msgr2=0x558a2bb70310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:36:31.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.416+0000 7f5aabfff640 1 --2- 172.21.15.97:0/288223263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a2bb76800 0x558a2bb70310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:31.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.416+0000 7f5aabfff640 1 -- 172.21.15.97:0/288223263 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a2bb6bf40 msgr2=0x558a2bb6fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:36:31.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.416+0000 7f5aabfff640 1 --2- 172.21.15.97:0/288223263 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a2bb6bf40 0x558a2bb6fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:31.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.416+0000 7f5aabfff640 1 -- 172.21.15.97:0/288223263 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a2bb70a50 con 0x558a2bb6b620 2024-04-23T10:36:31.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.416+0000 7f5ab083b640 1 --2- 172.21.15.97:0/288223263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a2bb76800 0x558a2bb70310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:36:31.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.416+0000 7f5aab7fe640 1 --2- 172.21.15.97:0/288223263 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a2bb6bf40 0x558a2bb6fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:36:31.419 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.416+0000 7f5aabfff640 1 --2- 172.21.15.97:0/288223263 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558a2bb6b620 0x558a2bb7e6a0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x558a2bb89020 tx=0x7f5a9c012cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=645f96586d922ca3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:36:31.420 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.416+0000 7f5a8b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:36:31.420 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.416+0000 7f5a8b7fe640 1 -- 172.21.15.97:0/288223263 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c004220 con 0x558a2bb6b620 2024-04-23T10:36:31.420 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.416+0000 7f5ab4fb0740 1 -- 172.21.15.97:0/288223263 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5aa0029680 con 0x558a2bb6b620 2024-04-23T10:36:31.420 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.416+0000 7f5a8b7fe640 1 -- 172.21.15.97:0/288223263 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8484+0+0 (secure 0 0 0) 0x7f5a9c0043c0 con 0x558a2bb6b620 2024-04-23T10:36:31.420 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.416+0000 7f5a8b7fe640 1 -- 172.21.15.97:0/288223263 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5a9c010900 con 0x558a2bb6b620 2024-04-23T10:36:31.420 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.416+0000 7f5ab4fb0740 10 client.17061 fetch_fsmap learned FSMap version 26 2024-04-23T10:36:31.420 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.416+0000 7f5ab4fb0740 1 -- 172.21.15.97:0/288223263 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558a2bbec8e0 con 0x558a2bb6b620 2024-04-23T10:36:31.420 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.416+0000 7f5a8b7fe640 1 -- 172.21.15.97:0/288223263 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5aa0029680 con 0x558a2bb6b620 2024-04-23T10:36:31.420 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.416+0000 7f5a8b7fe640 1 -- 172.21.15.97:0/288223263 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5a9c02f020 con 0x558a2bb6b620 2024-04-23T10:36:31.420 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.416+0000 7f5ab4fb0740 10 client.17061 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:36:31.420 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.416+0000 7f5ab4fb0740 10 client.17061 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:36:31.420 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.416+0000 7f5ab4fb0740 20 client.17061 populate_metadata read hostname 'smithi097' 2024-04-23T10:36:31.420 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.416+0000 7f5ab4fb0740 1 --2- 172.21.15.97:0/288223263 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x558a2bbecfb0 0x558a2bbef3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:31.420 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.416+0000 7f5ab4fb0740 4 client.17061 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:36:31.420 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.416+0000 7f5ab4fb0740 1 -- 172.21.15.97:0/288223263 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558a2bad8200 con 0x558a2bbecfb0 2024-04-23T10:36:31.421 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.417+0000 7f5aab7fe640 1 --2- 172.21.15.97:0/288223263 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x558a2bbecfb0 0x558a2bbef3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:31.421 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.417+0000 7f5a8b7fe640 10 client.17061 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:36:31.421 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.417+0000 7f5aab7fe640 1 --2- 172.21.15.97:0/288223263 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x558a2bbecfb0 0x558a2bbef3a0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbbc759e54ad7cca server_cookie=f6e99351e5eb5b1c in_seq=0 out_seq=0 2024-04-23T10:36:31.431 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.432+0000 7f5a8b7fe640 1 -- 172.21.15.97:0/288223263 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558a2bad8200 con 0x558a2bbecfb0 2024-04-23T10:36:31.431 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.432+0000 7f5a8b7fe640 10 client.17061 handle_command_reply: tid=0 2024-04-23T10:36:31.431 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.432+0000 7f5ab4fb0740 1 client.17061 shutdown 2024-04-23T10:36:31.431 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.432+0000 7f5ab4fb0740 1 -- 172.21.15.97:0/288223263 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558a2bb6b620 msgr2=0x558a2bb7e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:31.431 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.432+0000 7f5ab4fb0740 1 --2- 172.21.15.97:0/288223263 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558a2bb6b620 0x558a2bb7e6a0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x558a2bb89020 tx=0x7f5a9c012cd0 comp rx=0 tx=0).stop 2024-04-23T10:36:31.431 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.432+0000 7f5ab4fb0740 1 -- 172.21.15.97:0/288223263 shutdown_connections 2024-04-23T10:36:31.431 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.432+0000 7f5ab4fb0740 1 --2- 172.21.15.97:0/288223263 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x558a2bbecfb0 0x558a2bbef3a0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.432+0000 7f5ab4fb0740 1 --2- 172.21.15.97:0/288223263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a2bb76800 0x558a2bb70310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.432+0000 7f5a8b7fe640 0 client.17061 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:36:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.432+0000 7f5ab4fb0740 1 --2- 172.21.15.97:0/288223263 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a2bb6bf40 0x558a2bb6fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.432+0000 7f5ab4fb0740 1 --2- 172.21.15.97:0/288223263 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558a2bb6b620 0x558a2bb7e6a0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.432+0000 7f5ab4fb0740 1 -- 172.21.15.97:0/288223263 >> 172.21.15.97:0/288223263 conn(0x558a2bb67b30 msgr2=0x558a2bb6d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:36:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.432+0000 7f5ab4fb0740 1 -- 172.21.15.97:0/288223263 shutdown_connections 2024-04-23T10:36:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.432+0000 7f5ab4fb0740 1 -- 172.21.15.97:0/288223263 wait complete. 2024-04-23T10:36:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.432+0000 7f5ab4fb0740 20 client.17061 trim_cache size 0 max 16384 2024-04-23T10:36:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:36:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:36:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:36:31.433 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:36:31.433 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.433+0000 7f5ab229e640 1 -- 172.21.15.97:0/1827090459 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f5a940a3dc0 msgr2=0x7f5a940a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:31.433 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.433+0000 7f5ab229e640 1 --2- 172.21.15.97:0/1827090459 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f5a940a3dc0 0x7f5a940a6280 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5aac19bee0 tx=0x7f5aa000bdf0 comp rx=0 tx=0).stop 2024-04-23T10:36:31.433 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.433+0000 7f5ab229e640 1 -- 172.21.15.97:0/1827090459 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5aac19a690 msgr2=0x7f5aac19ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:31.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.433+0000 7f5ab229e640 1 --2- 172.21.15.97:0/1827090459 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5aac19a690 0x7f5aac19ab10 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4005cc0 tx=0x7f5aa402f380 comp rx=0 tx=0).stop 2024-04-23T10:36:31.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.433+0000 7f5ab229e640 1 -- 172.21.15.97:0/1827090459 shutdown_connections 2024-04-23T10:36:31.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.433+0000 7f5ab229e640 1 --2- 172.21.15.97:0/1827090459 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f5a940a3dc0 0x7f5a940a6280 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:31.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.433+0000 7f5ab229e640 1 --2- 172.21.15.97:0/1827090459 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5aac19a690 0x7f5aac19ab10 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:31.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.433+0000 7f5ab229e640 1 --2- 172.21.15.97:0/1827090459 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5aac107290 0x7f5aac1a0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:31.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.433+0000 7f5ab229e640 1 --2- 172.21.15.97:0/1827090459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5aac1021c0 0x7f5aac1a05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:31.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.433+0000 7f5ab229e640 1 -- 172.21.15.97:0/1827090459 >> 172.21.15.97:0/1827090459 conn(0x7f5aac0fda20 msgr2=0x7f5aac0fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:36:31.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.433+0000 7f5ab229e640 1 -- 172.21.15.97:0/1827090459 shutdown_connections 2024-04-23T10:36:31.434 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:31.433+0000 7f5ab229e640 1 -- 172.21.15.97:0/1827090459 wait complete. 2024-04-23T10:36:33.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:36:32 smithi196 ceph-mon[32919]: pgmap v321: 97 pgs: 97 active+clean; 2.6 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:36:33.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:36:32 smithi097 ceph-mon[29197]: pgmap v321: 97 pgs: 97 active+clean; 2.6 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:36:33.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:36:32 smithi183 ceph-mon[32317]: pgmap v321: 97 pgs: 97 active+clean; 2.6 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:36:35.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:36:34 smithi196 ceph-mon[32919]: pgmap v322: 97 pgs: 97 active+clean; 2.8 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 65 KiB/s wr, 24 op/s 2024-04-23T10:36:35.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:36:34 smithi097 ceph-mon[29197]: pgmap v322: 97 pgs: 97 active+clean; 2.8 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 65 KiB/s wr, 24 op/s 2024-04-23T10:36:35.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:36:34 smithi183 ceph-mon[32317]: pgmap v322: 97 pgs: 97 active+clean; 2.8 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 65 KiB/s wr, 24 op/s 2024-04-23T10:36:37.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:36:36 smithi196 ceph-mon[32919]: pgmap v323: 97 pgs: 97 active+clean; 2.8 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-23T10:36:37.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:36:36 smithi097 ceph-mon[29197]: pgmap v323: 97 pgs: 97 active+clean; 2.8 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-23T10:36:37.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:36:36 smithi183 ceph-mon[32317]: pgmap v323: 97 pgs: 97 active+clean; 2.8 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-23T10:36:38.490 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 26.9 2024-04-23T10:36:38.490 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T10:36:38.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.605+0000 7f83fb6d0640 1 -- 172.21.15.97:0/1662501633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f83f41072a0 msgr2=0x7f83f410ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:38.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.605+0000 7f83fb6d0640 1 --2- 172.21.15.97:0/1662501633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f83f41072a0 0x7f83f410ee70 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f83ec009ca0 tx=0x7f83ec02a8c0 comp rx=0 tx=0).stop 2024-04-23T10:36:38.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.605+0000 7f83fb6d0640 1 -- 172.21.15.97:0/1662501633 shutdown_connections 2024-04-23T10:36:38.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.605+0000 7f83fb6d0640 1 --2- 172.21.15.97:0/1662501633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f83f41072a0 0x7f83f410ee70 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:38.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.605+0000 7f83fb6d0640 1 --2- 172.21.15.97:0/1662501633 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83f4102b80 0x7f83f4106bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:38.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.605+0000 7f83fb6d0640 1 --2- 172.21.15.97:0/1662501633 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f83f41021d0 0x7f83f41025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:38.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.605+0000 7f83fb6d0640 1 -- 172.21.15.97:0/1662501633 >> 172.21.15.97:0/1662501633 conn(0x7f83f40793e0 msgr2=0x7f83f40797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:36:38.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.605+0000 7f83fb6d0640 1 -- 172.21.15.97:0/1662501633 shutdown_connections 2024-04-23T10:36:38.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.605+0000 7f83fb6d0640 1 -- 172.21.15.97:0/1662501633 wait complete. 2024-04-23T10:36:38.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.606+0000 7f83fb6d0640 1 Processor -- start 2024-04-23T10:36:38.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.606+0000 7f83fb6d0640 1 -- start start 2024-04-23T10:36:38.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.606+0000 7f83fb6d0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f83f41021d0 0x7f83f419e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:38.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.606+0000 7f83fb6d0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f83f41072a0 0x7f83f419e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:38.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.606+0000 7f83fb6d0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83f41985f0 0x7f83f4198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:38.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.606+0000 7f83fb6d0640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f83f4112210 con 0x7f83f41021d0 2024-04-23T10:36:38.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.606+0000 7f83fb6d0640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f83f4112090 con 0x7f83f41072a0 2024-04-23T10:36:38.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.606+0000 7f83fb6d0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f83f4112390 con 0x7f83f41985f0 2024-04-23T10:36:38.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.606+0000 7f83f946c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f83f41021d0 0x7f83f419e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:38.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.606+0000 7f83f946c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f83f41021d0 0x7f83f419e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59910/0 (socket says 172.21.15.97:59910) 2024-04-23T10:36:38.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.606+0000 7f83f946c640 1 -- 172.21.15.97:0/1018464264 learned_addr learned my addr 172.21.15.97:0/1018464264 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:36:38.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.607+0000 7f83f9c6d640 1 --2- 172.21.15.97:0/1018464264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83f41985f0 0x7f83f4198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:38.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.607+0000 7f83f8c6b640 1 --2- 172.21.15.97:0/1018464264 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f83f41072a0 0x7f83f419e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:38.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.607+0000 7f83f946c640 1 -- 172.21.15.97:0/1018464264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83f41985f0 msgr2=0x7f83f4198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:38.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.607+0000 7f83f946c640 1 --2- 172.21.15.97:0/1018464264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83f41985f0 0x7f83f4198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:38.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.607+0000 7f83f946c640 1 -- 172.21.15.97:0/1018464264 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f83f41072a0 msgr2=0x7f83f419e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:38.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.607+0000 7f83f946c640 1 --2- 172.21.15.97:0/1018464264 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f83f41072a0 0x7f83f419e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:38.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.607+0000 7f83f946c640 1 -- 172.21.15.97:0/1018464264 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83f4199110 con 0x7f83f41021d0 2024-04-23T10:36:38.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.607+0000 7f83f946c640 1 --2- 172.21.15.97:0/1018464264 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f83f41021d0 0x7f83f419e3f0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f83f000e9f0 tx=0x7f83f000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:36:38.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.607+0000 7f83ea7fc640 1 -- 172.21.15.97:0/1018464264 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f000cd80 con 0x7f83f41021d0 2024-04-23T10:36:38.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.607+0000 7f83ea7fc640 1 -- 172.21.15.97:0/1018464264 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83f00108e0 con 0x7f83f41021d0 2024-04-23T10:36:38.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.607+0000 7f83fb6d0640 1 -- 172.21.15.97:0/1018464264 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83f4199400 con 0x7f83f41021d0 2024-04-23T10:36:38.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.607+0000 7f83ea7fc640 1 -- 172.21.15.97:0/1018464264 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f0018980 con 0x7f83f41021d0 2024-04-23T10:36:38.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.607+0000 7f83fb6d0640 1 -- 172.21.15.97:0/1018464264 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83f41a5600 con 0x7f83f41021d0 2024-04-23T10:36:38.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.608+0000 7f83fb6d0640 1 -- 172.21.15.97:0/1018464264 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83f41036a0 con 0x7f83f41021d0 2024-04-23T10:36:38.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.609+0000 7f83ea7fc640 1 -- 172.21.15.97:0/1018464264 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f83f0010430 con 0x7f83f41021d0 2024-04-23T10:36:38.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.610+0000 7f83ea7fc640 1 --2- 172.21.15.97:0/1018464264 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f83e00a3d20 0x7f83e00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:38.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.610+0000 7f83f8c6b640 1 --2- 172.21.15.97:0/1018464264 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f83e00a3d20 0x7f83e00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:38.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.610+0000 7f83ea7fc640 1 -- 172.21.15.97:0/1018464264 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8484+0+0 (secure 0 0 0) 0x7f83f00d0c20 con 0x7f83f41021d0 2024-04-23T10:36:38.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.610+0000 7f83f8c6b640 1 --2- 172.21.15.97:0/1018464264 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f83e00a3d20 0x7f83e00a61e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f83f4101820 tx=0x7f83e4005f90 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:36:38.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.614+0000 7f83ea7fc640 1 -- 172.21.15.97:0/1018464264 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f83f009d2e0 con 0x7f83f41021d0 2024-04-23T10:36:38.829 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.830+0000 7f83fb6d0640 1 -- 172.21.15.97:0/1018464264 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f83f410a530 con 0x7f83f41021d0 2024-04-23T10:36:38.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.831+0000 7f83ea7fc640 1 -- 172.21.15.97:0/1018464264 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11931 (secure 0 0 0) 0x7f83f009cbe0 con 0x7f83f41021d0 2024-04-23T10:36:38.830 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:36:38.830 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3946878762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3946878762},{"type":"v1","addr":"172.21.15.97:6835","nonce":3946878762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/209800861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":209800861},{"type":"v1","addr":"172.21.15.97:6837","nonce":209800861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T10:24:06.910134+0000","modified":"2024-04-23T10:26:09.820985+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6839/418013575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":418013575},{"type":"v1","addr":"172.21.15.97:6839","nonce":418013575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6841/747242680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":747242680},{"type":"v1","addr":"172.21.15.97:6841","nonce":747242680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6833/4292847121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4292847121},{"type":"v1","addr":"172.21.15.183:6833","nonce":4292847121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6833/3885816565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3885816565},{"type":"v1","addr":"172.21.15.196:6833","nonce":3885816565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1897093105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1897093105},{"type":"v1","addr":"172.21.15.183:6835","nonce":1897093105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1171477563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1171477563},{"type":"v1","addr":"172.21.15.196:6835","nonce":1171477563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1898341911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1898341911},{"type":"v1","addr":"172.21.15.183:6837","nonce":1898341911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.196:6837/3094946502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3094946502},{"type":"v1","addr":"172.21.15.196:6837","nonce":3094946502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.196:6839/83843556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":83843556},{"type":"v1","addr":"172.21.15.196:6839","nonce":83843556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6839/1763841130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1763841130},{"type":"v1","addr":"172.21.15.183:6839","nonce":1763841130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24487,24403,24398,24469,24488]},"id":1}]} 2024-04-23T10:36:38.830 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 26 2024-04-23T10:36:38.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.833+0000 7f83fb6d0640 1 -- 172.21.15.97:0/1018464264 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f83e00a3d20 msgr2=0x7f83e00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:38.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.833+0000 7f83fb6d0640 1 --2- 172.21.15.97:0/1018464264 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f83e00a3d20 0x7f83e00a61e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f83f4101820 tx=0x7f83e4005f90 comp rx=0 tx=0).stop 2024-04-23T10:36:38.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.833+0000 7f83fb6d0640 1 -- 172.21.15.97:0/1018464264 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f83f41021d0 msgr2=0x7f83f419e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:38.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.833+0000 7f83fb6d0640 1 --2- 172.21.15.97:0/1018464264 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f83f41021d0 0x7f83f419e3f0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f83f000e9f0 tx=0x7f83f000eeb0 comp rx=0 tx=0).stop 2024-04-23T10:36:38.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.833+0000 7f83fb6d0640 1 -- 172.21.15.97:0/1018464264 shutdown_connections 2024-04-23T10:36:38.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.833+0000 7f83fb6d0640 1 --2- 172.21.15.97:0/1018464264 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f83e00a3d20 0x7f83e00a61e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:38.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.833+0000 7f83fb6d0640 1 --2- 172.21.15.97:0/1018464264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83f41985f0 0x7f83f4198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:38.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.833+0000 7f83fb6d0640 1 --2- 172.21.15.97:0/1018464264 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f83f41072a0 0x7f83f419e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:38.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.833+0000 7f83fb6d0640 1 --2- 172.21.15.97:0/1018464264 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f83f41021d0 0x7f83f419e3f0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:38.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.833+0000 7f83fb6d0640 1 -- 172.21.15.97:0/1018464264 >> 172.21.15.97:0/1018464264 conn(0x7f83f40793e0 msgr2=0x7f83f40ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:36:38.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.833+0000 7f83fb6d0640 1 -- 172.21.15.97:0/1018464264 shutdown_connections 2024-04-23T10:36:38.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.833+0000 7f83fb6d0640 1 -- 172.21.15.97:0/1018464264 wait complete. 2024-04-23T10:36:38.843 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db / --timeout 60 --expiration 86.9 --await-for 42.0' 2024-04-23T10:36:38.843 DEBUG:teuthology.orchestra.run.smithi097:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24488 quiesce db / --timeout 60 --expiration 86.9 --await-for 42.0 2024-04-23T10:36:38.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.955+0000 7fe224901640 1 -- 172.21.15.97:0/2519445599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe220106170 msgr2=0x7fe22010dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:38.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.955+0000 7fe224901640 1 --2- 172.21.15.97:0/2519445599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe220106170 0x7fe22010dd40 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fe210009900 tx=0x7fe21002a410 comp rx=0 tx=0).stop 2024-04-23T10:36:38.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.955+0000 7fe224901640 1 -- 172.21.15.97:0/2519445599 shutdown_connections 2024-04-23T10:36:38.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.955+0000 7fe224901640 1 --2- 172.21.15.97:0/2519445599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe220106170 0x7fe22010dd40 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:38.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.955+0000 7fe224901640 1 --2- 172.21.15.97:0/2519445599 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe220101a50 0x7fe220105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:38.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.955+0000 7fe224901640 1 --2- 172.21.15.97:0/2519445599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2201010a0 0x7fe220101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:38.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.955+0000 7fe224901640 1 -- 172.21.15.97:0/2519445599 >> 172.21.15.97:0/2519445599 conn(0x7fe220077f50 msgr2=0x7fe220078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:36:38.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.955+0000 7fe224901640 1 -- 172.21.15.97:0/2519445599 shutdown_connections 2024-04-23T10:36:38.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.955+0000 7fe224901640 1 -- 172.21.15.97:0/2519445599 wait complete. 2024-04-23T10:36:38.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.956+0000 7fe224901640 1 Processor -- start 2024-04-23T10:36:38.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.956+0000 7fe224901640 1 -- start start 2024-04-23T10:36:38.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.956+0000 7fe224901640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2201010a0 0x7fe220197540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:38.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.956+0000 7fe224901640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe220106170 0x7fe220197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:38.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.956+0000 7fe224901640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe22019be00 0x7fe22019c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:38.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.956+0000 7fe224901640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe2201110c0 con 0x7fe22019be00 2024-04-23T10:36:38.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.956+0000 7fe224901640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe220110f40 con 0x7fe220106170 2024-04-23T10:36:38.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.956+0000 7fe224901640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe220111240 con 0x7fe2201010a0 2024-04-23T10:36:38.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.956+0000 7fe21ed9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe22019be00 0x7fe22019c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:38.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.956+0000 7fe21ed9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe22019be00 0x7fe22019c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59924/0 (socket says 172.21.15.97:59924) 2024-04-23T10:36:38.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.956+0000 7fe21ed9d640 1 -- 172.21.15.97:0/14557736 learned_addr learned my addr 172.21.15.97:0/14557736 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:36:38.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.956+0000 7fe21dd9b640 1 --2- 172.21.15.97:0/14557736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe220106170 0x7fe220197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:38.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.956+0000 7fe21e59c640 1 --2- 172.21.15.97:0/14557736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2201010a0 0x7fe220197540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:38.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.956+0000 7fe21ed9d640 1 -- 172.21.15.97:0/14557736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2201010a0 msgr2=0x7fe220197540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:38.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.956+0000 7fe21ed9d640 1 --2- 172.21.15.97:0/14557736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2201010a0 0x7fe220197540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:38.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.956+0000 7fe21ed9d640 1 -- 172.21.15.97:0/14557736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe220106170 msgr2=0x7fe220197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:38.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.956+0000 7fe21ed9d640 1 --2- 172.21.15.97:0/14557736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe220106170 0x7fe220197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:38.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.956+0000 7fe21ed9d640 1 -- 172.21.15.97:0/14557736 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe22019c8f0 con 0x7fe22019be00 2024-04-23T10:36:38.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.956+0000 7fe21ed9d640 1 --2- 172.21.15.97:0/14557736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe22019be00 0x7fe22019c2b0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fe210004770 tx=0x7fe210004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:36:38.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.957+0000 7fe20f7fe640 1 -- 172.21.15.97:0/14557736 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe21003f030 con 0x7fe22019be00 2024-04-23T10:36:38.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.957+0000 7fe20f7fe640 1 -- 172.21.15.97:0/14557736 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe210003aa0 con 0x7fe22019be00 2024-04-23T10:36:38.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.957+0000 7fe20f7fe640 1 -- 172.21.15.97:0/14557736 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe210035070 con 0x7fe22019be00 2024-04-23T10:36:38.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.957+0000 7fe224901640 1 -- 172.21.15.97:0/14557736 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe22019cb80 con 0x7fe22019be00 2024-04-23T10:36:38.958 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.957+0000 7fe224901640 1 -- 172.21.15.97:0/14557736 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2201a43a0 con 0x7fe22019be00 2024-04-23T10:36:38.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.959+0000 7fe20f7fe640 1 -- 172.21.15.97:0/14557736 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe210042030 con 0x7fe22019be00 2024-04-23T10:36:38.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.959+0000 7fe227618740 1 Processor -- start 2024-04-23T10:36:38.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.959+0000 7fe227618740 1 -- start start 2024-04-23T10:36:38.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.960+0000 7fe227618740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557faa9fa0a0 0x557faa9fa480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:38.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.960+0000 7fe227618740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557faa9fa9c0 0x557faaa04b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:38.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.960+0000 7fe227618740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557faaa05090 0x557faaa07480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:38.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.960+0000 7fe227618740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x557faa95f370 con 0x557faa9fa0a0 2024-04-23T10:36:38.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.960+0000 7fe227618740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557faa8fc2e0 con 0x557faa9fa9c0 2024-04-23T10:36:38.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.960+0000 7fe227618740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557faa95b330 con 0x557faaa05090 2024-04-23T10:36:38.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.960+0000 7fe21dd9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557faa9fa0a0 0x557faa9fa480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:38.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.960+0000 7fe20f7fe640 1 --2- 172.21.15.97:0/14557736 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fe2080a3da0 0x7fe2080a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:38.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.960+0000 7fe21dd9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557faa9fa0a0 0x557faa9fa480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59934/0 (socket says 172.21.15.97:59934) 2024-04-23T10:36:38.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.960+0000 7fe21dd9b640 1 -- 172.21.15.97:0/3948284927 learned_addr learned my addr 172.21.15.97:0/3948284927 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:36:38.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.960+0000 7fe21ed9d640 1 --2- 172.21.15.97:0/14557736 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fe2080a3da0 0x7fe2080a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:38.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.960+0000 7fe21e59c640 1 --2- 172.21.15.97:0/3948284927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557faa9fa9c0 0x557faaa04b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:38.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.960+0000 7fe20f7fe640 1 -- 172.21.15.97:0/14557736 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8484+0+0 (secure 0 0 0) 0x7fe2100f0ea0 con 0x7fe22019be00 2024-04-23T10:36:38.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.960+0000 7fe21dd9b640 1 --2- 172.21.15.97:0/3948284927 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557faaa05090 0x557faaa07480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:38.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.960+0000 7fe21dd9b640 1 -- 172.21.15.97:0/3948284927 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557faaa05090 msgr2=0x557faaa07480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:36:38.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.960+0000 7fe21dd9b640 1 --2- 172.21.15.97:0/3948284927 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557faaa05090 0x557faaa07480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:38.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.960+0000 7fe21dd9b640 1 -- 172.21.15.97:0/3948284927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557faa9fa9c0 msgr2=0x557faaa04b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:36:38.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.960+0000 7fe21dd9b640 1 --2- 172.21.15.97:0/3948284927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557faa9fa9c0 0x557faaa04b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:38.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.960+0000 7fe21dd9b640 1 -- 172.21.15.97:0/3948284927 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557faa73dd80 con 0x557faa9fa0a0 2024-04-23T10:36:38.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.961+0000 7fe21ed9d640 1 --2- 172.21.15.97:0/14557736 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fe2080a3da0 0x7fe2080a6260 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe2100ee6c0 tx=0x7fe21008e8a0 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:36:38.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.961+0000 7fe21dd9b640 1 --2- 172.21.15.97:0/3948284927 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557faa9fa0a0 0x557faa9fa480 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fe21400bf60 tx=0x7fe21400b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5cc8e635a012cff server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:36:38.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.961+0000 7fe20cff9640 1 -- 172.21.15.97:0/3948284927 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe214004020 con 0x557faa9fa0a0 2024-04-23T10:36:38.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.961+0000 7fe20cff9640 1 -- 172.21.15.97:0/3948284927 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe214004c50 con 0x557faa9fa0a0 2024-04-23T10:36:38.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.961+0000 7fe20cff9640 1 -- 172.21.15.97:0/3948284927 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe214005220 con 0x557faa9fa0a0 2024-04-23T10:36:38.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.961+0000 7fe227618740 1 -- 172.21.15.97:0/3948284927 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557faa9fa0a0 msgr2=0x557faa9fa480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:38.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.961+0000 7fe227618740 1 --2- 172.21.15.97:0/3948284927 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557faa9fa0a0 0x557faa9fa480 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fe21400bf60 tx=0x7fe21400b650 comp rx=0 tx=0).stop 2024-04-23T10:36:38.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.961+0000 7fe227618740 1 -- 172.21.15.97:0/3948284927 shutdown_connections 2024-04-23T10:36:38.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.961+0000 7fe227618740 1 --2- 172.21.15.97:0/3948284927 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557faaa05090 0x557faaa07480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:38.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.961+0000 7fe227618740 1 --2- 172.21.15.97:0/3948284927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557faa9fa9c0 0x557faaa04b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:38.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.961+0000 7fe227618740 1 --2- 172.21.15.97:0/3948284927 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557faa9fa0a0 0x557faa9fa480 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:38.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.961+0000 7fe227618740 1 -- 172.21.15.97:0/3948284927 >> 172.21.15.97:0/3948284927 conn(0x557faa9f63c0 msgr2=0x557faa9f67a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:36:38.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.962+0000 7fe227618740 1 -- 172.21.15.97:0/3948284927 shutdown_connections 2024-04-23T10:36:38.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.962+0000 7fe227618740 1 -- 172.21.15.97:0/3948284927 wait complete. 2024-04-23T10:36:38.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.962+0000 7fe227618740 1 -- start start 2024-04-23T10:36:38.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.962+0000 7fe227618740 1 Processor -- start 2024-04-23T10:36:38.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.962+0000 7fe227618740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557faa9fa0a0 0x557faaa01ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:38.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.962+0000 7fe227618740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557faa9fa9c0 0x557faaa02530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:38.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.962+0000 7fe227618740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557faaa05090 0x557faaa02a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:38.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.962+0000 7fe227618740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x557faaa07e80 con 0x557faa9fa9c0 2024-04-23T10:36:38.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.962+0000 7fe227618740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557faa8f9fb0 con 0x557faa9fa0a0 2024-04-23T10:36:38.967 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.962+0000 7fe227618740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557faaa08000 con 0x557faaa05090 2024-04-23T10:36:38.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.962+0000 7fe227618740 10 client.0 _ll_register_callbacks cb 0x557faa8f99a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:36:38.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.962+0000 7fe21e59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557faa9fa9c0 0x557faaa02530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:38.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.962+0000 7fe21e59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557faa9fa9c0 0x557faaa02530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59950/0 (socket says 172.21.15.97:59950) 2024-04-23T10:36:38.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.962+0000 7fe21e59c640 1 -- 172.21.15.97:0/4172194091 learned_addr learned my addr 172.21.15.97:0/4172194091 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:36:38.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.963+0000 7fe21dd9b640 1 --2- 172.21.15.97:0/4172194091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557faa9fa0a0 0x557faaa01ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:38.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.963+0000 7fe21e59c640 1 -- 172.21.15.97:0/4172194091 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557faaa05090 msgr2=0x557faaa02a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:36:38.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.963+0000 7fe21e59c640 1 --2- 172.21.15.97:0/4172194091 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557faaa05090 0x557faaa02a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:38.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.963+0000 7fe21e59c640 1 -- 172.21.15.97:0/4172194091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557faa9fa0a0 msgr2=0x557faaa01ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:36:38.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.963+0000 7fe21e59c640 1 --2- 172.21.15.97:0/4172194091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557faa9fa0a0 0x557faaa01ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:38.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.963+0000 7fe21e59c640 1 -- 172.21.15.97:0/4172194091 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557faa79e510 con 0x557faa9fa9c0 2024-04-23T10:36:38.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.963+0000 7fe21dd9b640 1 --2- 172.21.15.97:0/4172194091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557faa9fa0a0 0x557faaa01ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:36:38.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.963+0000 7fe21e59c640 1 --2- 172.21.15.97:0/4172194091 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557faa9fa9c0 0x557faaa02530 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fe218011a80 tx=0x7fe218011f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ef0bbec01c16dca server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:36:38.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.963+0000 7fe205ffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:36:38.968 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.963+0000 7fe205ffb640 1 -- 172.21.15.97:0/4172194091 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe218003970 con 0x557faa9fa9c0 2024-04-23T10:36:38.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.963+0000 7fe205ffb640 1 -- 172.21.15.97:0/4172194091 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8484+0+0 (secure 0 0 0) 0x7fe218003b10 con 0x557faa9fa9c0 2024-04-23T10:36:38.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.963+0000 7fe227618740 1 -- 172.21.15.97:0/4172194091 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe214004c50 con 0x557faa9fa9c0 2024-04-23T10:36:38.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.963+0000 7fe205ffb640 1 -- 172.21.15.97:0/4172194091 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe218003d10 con 0x557faa9fa9c0 2024-04-23T10:36:38.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.963+0000 7fe227618740 10 client.17079 fetch_fsmap learned FSMap version 26 2024-04-23T10:36:38.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.963+0000 7fe227618740 1 -- 172.21.15.97:0/4172194091 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557faa631670 con 0x557faa9fa9c0 2024-04-23T10:36:38.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.963+0000 7fe205ffb640 1 -- 172.21.15.97:0/4172194091 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe214004c50 con 0x557faa9fa9c0 2024-04-23T10:36:38.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.963+0000 7fe205ffb640 1 -- 172.21.15.97:0/4172194091 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe218020030 con 0x557faa9fa9c0 2024-04-23T10:36:38.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.963+0000 7fe227618740 10 client.17079 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:36:38.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.963+0000 7fe227618740 10 client.17079 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:36:38.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.963+0000 7fe227618740 20 client.17079 populate_metadata read hostname 'smithi097' 2024-04-23T10:36:38.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.963+0000 7fe227618740 1 --2- 172.21.15.97:0/4172194091 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x557faaa154b0 0x557faaa17910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:38.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.963+0000 7fe227618740 4 client.17079 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:36:38.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.963+0000 7fe227618740 1 -- 172.21.15.97:0/4172194091 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557faa96cd90 con 0x557faaa154b0 2024-04-23T10:36:38.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.964+0000 7fe21dd9b640 1 --2- 172.21.15.97:0/4172194091 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x557faaa154b0 0x557faaa17910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:38.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.964+0000 7fe205ffb640 10 client.17079 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:36:38.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.964+0000 7fe21dd9b640 1 --2- 172.21.15.97:0/4172194091 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x557faaa154b0 0x557faaa17910 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbdfd67d597a0d46 server_cookie=991aad06e843019d in_seq=0 out_seq=0 2024-04-23T10:36:38.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.965+0000 7fe205ffb640 1 -- 172.21.15.97:0/4172194091 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557faa96cd90 con 0x557faaa154b0 2024-04-23T10:36:38.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.965+0000 7fe205ffb640 10 client.17079 handle_command_reply: tid=0 2024-04-23T10:36:38.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.965+0000 7fe227618740 1 client.17079 shutdown 2024-04-23T10:36:38.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.965+0000 7fe227618740 1 -- 172.21.15.97:0/4172194091 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557faa9fa9c0 msgr2=0x557faaa02530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:38.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.965+0000 7fe227618740 1 --2- 172.21.15.97:0/4172194091 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557faa9fa9c0 0x557faaa02530 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fe218011a80 tx=0x7fe218011f40 comp rx=0 tx=0).stop 2024-04-23T10:36:38.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.965+0000 7fe227618740 1 -- 172.21.15.97:0/4172194091 shutdown_connections 2024-04-23T10:36:38.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.965+0000 7fe227618740 1 --2- 172.21.15.97:0/4172194091 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x557faaa154b0 0x557faaa17910 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:38.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.965+0000 7fe227618740 1 --2- 172.21.15.97:0/4172194091 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557faaa05090 0x557faaa02a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:38.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.965+0000 7fe227618740 1 --2- 172.21.15.97:0/4172194091 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557faa9fa9c0 0x557faaa02530 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:38.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.965+0000 7fe205ffb640 0 client.17079 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:36:38.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.965+0000 7fe227618740 1 --2- 172.21.15.97:0/4172194091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557faa9fa0a0 0x557faaa01ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:38.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.965+0000 7fe227618740 1 -- 172.21.15.97:0/4172194091 >> 172.21.15.97:0/4172194091 conn(0x557faa9f63c0 msgr2=0x557faa9fc0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:36:38.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.965+0000 7fe227618740 1 -- 172.21.15.97:0/4172194091 shutdown_connections 2024-04-23T10:36:38.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.965+0000 7fe227618740 1 -- 172.21.15.97:0/4172194091 wait complete. 2024-04-23T10:36:38.971 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.965+0000 7fe227618740 20 client.17079 trim_cache size 0 max 16384 2024-04-23T10:36:38.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.982+0000 7fe227618740 1 Processor -- start 2024-04-23T10:36:38.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.982+0000 7fe227618740 1 -- start start 2024-04-23T10:36:38.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.982+0000 7fe227618740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557faa9fa0a0 0x557faaa047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:38.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.982+0000 7fe227618740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557faa9fa9c0 0x557faaa02620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:38.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.982+0000 7fe227618740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557faaa05090 0x557faaa10d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:38.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.982+0000 7fe227618740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe2201ac930 con 0x557faaa05090 2024-04-23T10:36:38.982 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.982+0000 7fe227618740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe2201ae470 con 0x557faa9fa0a0 2024-04-23T10:36:38.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.982+0000 7fe227618740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe2201ad000 con 0x557faa9fa9c0 2024-04-23T10:36:38.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.983+0000 7fe21dd9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557faaa05090 0x557faaa10d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:38.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.983+0000 7fe21dd9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557faaa05090 0x557faaa10d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59954/0 (socket says 172.21.15.97:59954) 2024-04-23T10:36:38.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.983+0000 7fe21dd9b640 1 -- 172.21.15.97:0/2937430130 learned_addr learned my addr 172.21.15.97:0/2937430130 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:36:38.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.983+0000 7fe21dd9b640 1 --2- 172.21.15.97:0/2937430130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557faa9fa0a0 0x557faaa047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:38.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.983+0000 7fe21e59c640 1 --2- 172.21.15.97:0/2937430130 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557faa9fa9c0 0x557faaa02620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:38.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.983+0000 7fe21dd9b640 1 -- 172.21.15.97:0/2937430130 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557faa9fa9c0 msgr2=0x557faaa02620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:36:38.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.983+0000 7fe21dd9b640 1 --2- 172.21.15.97:0/2937430130 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557faa9fa9c0 0x557faaa02620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:38.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.983+0000 7fe21dd9b640 1 -- 172.21.15.97:0/2937430130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557faa9fa0a0 msgr2=0x557faaa047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:36:38.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.983+0000 7fe21dd9b640 1 --2- 172.21.15.97:0/2937430130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557faa9fa0a0 0x557faaa047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:38.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.983+0000 7fe21dd9b640 1 -- 172.21.15.97:0/2937430130 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557faa9f0820 con 0x557faaa05090 2024-04-23T10:36:38.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.983+0000 7fe21dd9b640 1 --2- 172.21.15.97:0/2937430130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557faaa05090 0x557faaa10d70 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fe214004420 tx=0x7fe2140043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e066a143624f0f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:36:38.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.983+0000 7fe207fff640 1 -- 172.21.15.97:0/2937430130 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe214028370 con 0x557faaa05090 2024-04-23T10:36:38.983 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.983+0000 7fe207fff640 1 -- 172.21.15.97:0/2937430130 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe214004660 con 0x557faaa05090 2024-04-23T10:36:38.984 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.983+0000 7fe207fff640 1 -- 172.21.15.97:0/2937430130 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2140051f0 con 0x557faaa05090 2024-04-23T10:36:38.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.984+0000 7fe227618740 1 -- 172.21.15.97:0/2937430130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557faaa05090 msgr2=0x557faaa10d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:38.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.984+0000 7fe227618740 1 --2- 172.21.15.97:0/2937430130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557faaa05090 0x557faaa10d70 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fe214004420 tx=0x7fe2140043b0 comp rx=0 tx=0).stop 2024-04-23T10:36:38.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.984+0000 7fe227618740 1 -- 172.21.15.97:0/2937430130 shutdown_connections 2024-04-23T10:36:38.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.984+0000 7fe227618740 1 --2- 172.21.15.97:0/2937430130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557faaa05090 0x557faaa10d70 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:38.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.984+0000 7fe227618740 1 --2- 172.21.15.97:0/2937430130 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557faa9fa9c0 0x557faaa02620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:38.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.984+0000 7fe227618740 1 --2- 172.21.15.97:0/2937430130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557faa9fa0a0 0x557faaa047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:38.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.984+0000 7fe227618740 1 -- 172.21.15.97:0/2937430130 >> 172.21.15.97:0/2937430130 conn(0x557faa9f63c0 msgr2=0x557faaa040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:36:38.987 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.984+0000 7fe227618740 1 -- 172.21.15.97:0/2937430130 shutdown_connections 2024-04-23T10:36:38.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.984+0000 7fe227618740 1 -- 172.21.15.97:0/2937430130 wait complete. 2024-04-23T10:36:38.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.984+0000 7fe227618740 1 -- start start 2024-04-23T10:36:38.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.984+0000 7fe227618740 1 Processor -- start 2024-04-23T10:36:38.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.985+0000 7fe227618740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557faa9fa0a0 0x557faa9fe280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:38.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.985+0000 7fe227618740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557faa9fa9c0 0x557faa9fe7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:38.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.985+0000 7fe227618740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557faaa05090 0x557faa9fed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:38.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.985+0000 7fe227618740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe2201adda0 con 0x557faaa05090 2024-04-23T10:36:38.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.985+0000 7fe227618740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe2201ad6d0 con 0x557faa9fa0a0 2024-04-23T10:36:38.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.985+0000 7fe227618740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe2201aa930 con 0x557faa9fa9c0 2024-04-23T10:36:38.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.985+0000 7fe227618740 10 client.0 _ll_register_callbacks cb 0x7fe2180116c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:36:38.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.985+0000 7fe21dd9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557faaa05090 0x557faa9fed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:38.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.985+0000 7fe21dd9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557faaa05090 0x557faa9fed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59958/0 (socket says 172.21.15.97:59958) 2024-04-23T10:36:38.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.985+0000 7fe21dd9b640 1 -- 172.21.15.97:0/3204882683 learned_addr learned my addr 172.21.15.97:0/3204882683 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:36:38.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.985+0000 7fe21e59c640 1 --2- 172.21.15.97:0/3204882683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557faa9fa9c0 0x557faa9fe7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:38.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.985+0000 7fe21dd9b640 1 --2- 172.21.15.97:0/3204882683 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557faa9fa0a0 0x557faa9fe280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:38.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.985+0000 7fe21e59c640 1 -- 172.21.15.97:0/3204882683 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557faa9fa0a0 msgr2=0x557faa9fe280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:36:38.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.985+0000 7fe21e59c640 1 --2- 172.21.15.97:0/3204882683 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557faa9fa0a0 0x557faa9fe280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:38.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.985+0000 7fe21e59c640 1 -- 172.21.15.97:0/3204882683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557faaa05090 msgr2=0x557faa9fed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:36:38.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.985+0000 7fe21e59c640 1 --2- 172.21.15.97:0/3204882683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557faaa05090 0x557faa9fed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:38.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.985+0000 7fe21e59c640 1 -- 172.21.15.97:0/3204882683 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557faaa7db00 con 0x557faa9fa9c0 2024-04-23T10:36:38.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.985+0000 7fe21dd9b640 1 --2- 172.21.15.97:0/3204882683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557faaa05090 0x557faa9fed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T10:36:38.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.985+0000 7fe21e59c640 1 --2- 172.21.15.97:0/3204882683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557faa9fa9c0 0x557faa9fe7c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe2180196c0 tx=0x7fe21800f970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2cf8614119911137 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:36:38.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.985+0000 7fe205ffb640 10 client.0 ms_handle_connect on v2:172.21.15.196:3300/0 2024-04-23T10:36:38.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.985+0000 7fe205ffb640 1 -- 172.21.15.97:0/3204882683 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2180198b0 con 0x557faa9fa9c0 2024-04-23T10:36:38.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.985+0000 7fe227618740 1 -- 172.21.15.97:0/3204882683 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe214004660 con 0x557faa9fa9c0 2024-04-23T10:36:38.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.985+0000 7fe205ffb640 1 -- 172.21.15.97:0/3204882683 <== mon.1 v2:172.21.15.196:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8484+0+0 (secure 0 0 0) 0x7fe218019a50 con 0x557faa9fa9c0 2024-04-23T10:36:38.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.986+0000 7fe205ffb640 1 -- 172.21.15.97:0/3204882683 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe218019cb0 con 0x557faa9fa9c0 2024-04-23T10:36:38.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.986+0000 7fe227618740 10 client.26170 fetch_fsmap learned FSMap version 26 2024-04-23T10:36:38.989 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.986+0000 7fe227618740 1 -- 172.21.15.97:0/3204882683 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557faaa7ed80 con 0x557faa9fa9c0 2024-04-23T10:36:38.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.986+0000 7fe205ffb640 1 -- 172.21.15.97:0/3204882683 <== mon.1 v2:172.21.15.196:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe214004660 con 0x557faa9fa9c0 2024-04-23T10:36:38.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.986+0000 7fe205ffb640 1 -- 172.21.15.97:0/3204882683 <== mon.1 v2:172.21.15.196:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe218031070 con 0x557faa9fa9c0 2024-04-23T10:36:38.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.986+0000 7fe227618740 10 client.26170 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:36:38.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.986+0000 7fe227618740 10 client.26170 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:36:38.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.986+0000 7fe227618740 20 client.26170 populate_metadata read hostname 'smithi097' 2024-04-23T10:36:38.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.986+0000 7fe227618740 1 --2- 172.21.15.97:0/3204882683 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x557faaa7f570 0x557faaa81990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:38.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.986+0000 7fe227618740 4 client.26170 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 86.9, "await_for": 42.0}] 2024-04-23T10:36:38.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.986+0000 7fe227618740 1 -- 172.21.15.97:0/3204882683 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 86.9, "await_for": 42.0}) -- 0x557faa966c00 con 0x557faaa7f570 2024-04-23T10:36:38.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.986+0000 7fe21dd9b640 1 --2- 172.21.15.97:0/3204882683 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x557faaa7f570 0x557faaa81990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:38.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.987+0000 7fe21dd9b640 1 --2- 172.21.15.97:0/3204882683 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x557faaa7f570 0x557faaa81990 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=981b5997f6c31581 server_cookie=4a10f08f94004a8d in_seq=0 out_seq=0 2024-04-23T10:36:38.990 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:38.987+0000 7fe205ffb640 10 client.26170 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:36:39.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:39.000+0000 7fe205ffb640 1 -- 172.21.15.97:0/3204882683 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x557faa966c00 con 0x557faaa7f570 2024-04-23T10:36:39.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:39.000+0000 7fe205ffb640 10 client.26170 handle_command_reply: tid=0 2024-04-23T10:36:39.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:39.000+0000 7fe227618740 1 client.26170 shutdown 2024-04-23T10:36:39.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:39.001+0000 7fe227618740 1 -- 172.21.15.97:0/3204882683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557faa9fa9c0 msgr2=0x557faa9fe7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:39.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:39.001+0000 7fe227618740 1 --2- 172.21.15.97:0/3204882683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557faa9fa9c0 0x557faa9fe7c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe2180196c0 tx=0x7fe21800f970 comp rx=0 tx=0).stop 2024-04-23T10:36:39.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:39.001+0000 7fe227618740 1 -- 172.21.15.97:0/3204882683 shutdown_connections 2024-04-23T10:36:39.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:39.001+0000 7fe227618740 1 --2- 172.21.15.97:0/3204882683 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x557faaa7f570 0x557faaa81990 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:39.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:39.001+0000 7fe227618740 1 --2- 172.21.15.97:0/3204882683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557faaa05090 0x557faa9fed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:39.000 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:39.001+0000 7fe205ffb640 0 client.26170 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:36:39.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:39.001+0000 7fe227618740 1 --2- 172.21.15.97:0/3204882683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557faa9fa9c0 0x557faa9fe7c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:39.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:39.001+0000 7fe227618740 1 --2- 172.21.15.97:0/3204882683 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557faa9fa0a0 0x557faa9fe280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:39.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:39.001+0000 7fe227618740 1 -- 172.21.15.97:0/3204882683 >> 172.21.15.97:0/3204882683 conn(0x557faa9f63c0 msgr2=0x557faaa0fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:36:39.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:39.001+0000 7fe227618740 1 -- 172.21.15.97:0/3204882683 shutdown_connections 2024-04-23T10:36:39.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:39.001+0000 7fe227618740 1 -- 172.21.15.97:0/3204882683 wait complete. 2024-04-23T10:36:39.001 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:39.001+0000 7fe227618740 20 client.26170 trim_cache size 0 max 16384 2024-04-23T10:36:39.001 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-23T10:36:39.001 INFO:teuthology.orchestra.run.smithi097.stdout: "epoch": 26, 2024-04-23T10:36:39.001 INFO:teuthology.orchestra.run.smithi097.stdout: "leader": 24488, 2024-04-23T10:36:39.001 INFO:teuthology.orchestra.run.smithi097.stdout: "set_version": 46, 2024-04-23T10:36:39.001 INFO:teuthology.orchestra.run.smithi097.stdout: "sets": { 2024-04-23T10:36:39.001 INFO:teuthology.orchestra.run.smithi097.stdout: "f5a8cd2e": { 2024-04-23T10:36:39.002 INFO:teuthology.orchestra.run.smithi097.stdout: "version": 46, 2024-04-23T10:36:39.002 INFO:teuthology.orchestra.run.smithi097.stdout: "age_ref": 0.0, 2024-04-23T10:36:39.002 INFO:teuthology.orchestra.run.smithi097.stdout: "state": { 2024-04-23T10:36:39.002 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "QUIESCED", 2024-04-23T10:36:39.002 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0 2024-04-23T10:36:39.002 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:36:39.002 INFO:teuthology.orchestra.run.smithi097.stdout: "timeout": 60.0, 2024-04-23T10:36:39.002 INFO:teuthology.orchestra.run.smithi097.stdout: "expiration": 86.9, 2024-04-23T10:36:39.002 INFO:teuthology.orchestra.run.smithi097.stdout: "members": { 2024-04-23T10:36:39.002 INFO:teuthology.orchestra.run.smithi097.stdout: "file:/": { 2024-04-23T10:36:39.002 INFO:teuthology.orchestra.run.smithi097.stdout: "excluded": false, 2024-04-23T10:36:39.002 INFO:teuthology.orchestra.run.smithi097.stdout: "state": { 2024-04-23T10:36:39.002 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "QUIESCED", 2024-04-23T10:36:39.002 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0 2024-04-23T10:36:39.003 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:36:39.003 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:36:39.003 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:36:39.003 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:36:39.003 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:36:39.003 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-23T10:36:39.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:39.001+0000 7fe224901640 1 -- 172.21.15.97:0/14557736 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fe2080a3da0 msgr2=0x7fe2080a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:39.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:39.001+0000 7fe224901640 1 --2- 172.21.15.97:0/14557736 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fe2080a3da0 0x7fe2080a6260 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe2100ee6c0 tx=0x7fe21008e8a0 comp rx=0 tx=0).stop 2024-04-23T10:36:39.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:39.001+0000 7fe224901640 1 -- 172.21.15.97:0/14557736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe22019be00 msgr2=0x7fe22019c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:39.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:39.001+0000 7fe224901640 1 --2- 172.21.15.97:0/14557736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe22019be00 0x7fe22019c2b0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fe210004770 tx=0x7fe210004270 comp rx=0 tx=0).stop 2024-04-23T10:36:39.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:39.001+0000 7fe224901640 1 -- 172.21.15.97:0/14557736 shutdown_connections 2024-04-23T10:36:39.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:39.001+0000 7fe224901640 1 --2- 172.21.15.97:0/14557736 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fe2080a3da0 0x7fe2080a6260 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:39.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:39.001+0000 7fe224901640 1 --2- 172.21.15.97:0/14557736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe22019be00 0x7fe22019c2b0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:39.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:39.001+0000 7fe224901640 1 --2- 172.21.15.97:0/14557736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe220106170 0x7fe220197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:39.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:39.001+0000 7fe224901640 1 --2- 172.21.15.97:0/14557736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2201010a0 0x7fe220197540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:39.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:39.001+0000 7fe224901640 1 -- 172.21.15.97:0/14557736 >> 172.21.15.97:0/14557736 conn(0x7fe220077f50 msgr2=0x7fe2200feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:36:39.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:39.001+0000 7fe224901640 1 -- 172.21.15.97:0/14557736 shutdown_connections 2024-04-23T10:36:39.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:36:39.002+0000 7fe224901640 1 -- 172.21.15.97:0/14557736 wait complete. 2024-04-23T10:36:39.011 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'f5a8cd2e', quiesce took 0.5 seconds. Will release after: 26.9 2024-04-23T10:36:39.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:36:38 smithi097 ceph-mon[29197]: pgmap v324: 97 pgs: 97 active+clean; 3.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-23T10:36:39.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:36:38 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/1018464264' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:36:39.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:36:38 smithi196 ceph-mon[32919]: pgmap v324: 97 pgs: 97 active+clean; 3.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-23T10:36:39.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:36:38 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/1018464264' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:36:39.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:36:38 smithi183 ceph-mon[32317]: pgmap v324: 97 pgs: 97 active+clean; 3.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-23T10:36:39.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:36:38 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/1018464264' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:36:41.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:36:40 smithi196 ceph-mon[32919]: pgmap v325: 97 pgs: 97 active+clean; 3.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:36:41.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:36:40 smithi097 ceph-mon[29197]: pgmap v325: 97 pgs: 97 active+clean; 3.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:36:41.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:36:40 smithi183 ceph-mon[32317]: pgmap v325: 97 pgs: 97 active+clean; 3.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:36:43.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:36:42 smithi196 ceph-mon[32919]: pgmap v326: 97 pgs: 97 active+clean; 3.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:36:43.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:36:42 smithi097 ceph-mon[29197]: pgmap v326: 97 pgs: 97 active+clean; 3.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:36:43.399 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:36:42 smithi183 ceph-mon[32317]: pgmap v326: 97 pgs: 97 active+clean; 3.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:36:44.529 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.528+0000 7f221477c640 1 -- 172.21.15.97:0/2425723695 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f220c1031e0 msgr2=0x7f220c1035c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:44.529 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.528+0000 7f221477c640 1 --2- 172.21.15.97:0/2425723695 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f220c1031e0 0x7f220c1035c0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f2208009900 tx=0x7f220802a510 comp rx=0 tx=0).stop 2024-04-23T10:36:44.529 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.528+0000 7f221477c640 1 -- 172.21.15.97:0/2425723695 shutdown_connections 2024-04-23T10:36:44.529 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.528+0000 7f221477c640 1 --2- 172.21.15.97:0/2425723695 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f220c10c520 0x7f220c10e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:44.530 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.528+0000 7f221477c640 1 --2- 172.21.15.97:0/2425723695 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f220c103b00 0x7f220c10bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:44.530 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.528+0000 7f221477c640 1 --2- 172.21.15.97:0/2425723695 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f220c1031e0 0x7f220c1035c0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:44.530 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.528+0000 7f221477c640 1 -- 172.21.15.97:0/2425723695 >> 172.21.15.97:0/2425723695 conn(0x7f220c077fd0 msgr2=0x7f220c0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:36:44.530 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.528+0000 7f221477c640 1 -- 172.21.15.97:0/2425723695 shutdown_connections 2024-04-23T10:36:44.530 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.528+0000 7f221477c640 1 -- 172.21.15.97:0/2425723695 wait complete. 2024-04-23T10:36:44.530 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.529+0000 7f221477c640 1 Processor -- start 2024-04-23T10:36:44.530 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.529+0000 7f221477c640 1 -- start start 2024-04-23T10:36:44.530 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.529+0000 7f221477c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f220c1031e0 0x7f220c100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:44.530 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.529+0000 7f221477c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f220c103b00 0x7f220c101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:44.530 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.529+0000 7f221477c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f220c10c520 0x7f220c1018b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:44.530 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.529+0000 7f221477c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f220c112190 con 0x7f220c1031e0 2024-04-23T10:36:44.530 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.529+0000 7f221477c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f220c112010 con 0x7f220c10c520 2024-04-23T10:36:44.530 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.529+0000 7f221477c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f220c112310 con 0x7f220c103b00 2024-04-23T10:36:44.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.529+0000 7f2212518640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f220c1031e0 0x7f220c100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:44.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.529+0000 7f2212518640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f220c1031e0 0x7f220c100e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59978/0 (socket says 172.21.15.97:59978) 2024-04-23T10:36:44.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.529+0000 7f2212518640 1 -- 172.21.15.97:0/3351251937 learned_addr learned my addr 172.21.15.97:0/3351251937 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:36:44.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.529+0000 7f2212d19640 1 --2- 172.21.15.97:0/3351251937 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f220c10c520 0x7f220c1018b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:44.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.529+0000 7f2211d17640 1 --2- 172.21.15.97:0/3351251937 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f220c103b00 0x7f220c101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:44.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.530+0000 7f2212518640 1 -- 172.21.15.97:0/3351251937 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f220c103b00 msgr2=0x7f220c101370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:44.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.530+0000 7f2212518640 1 --2- 172.21.15.97:0/3351251937 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f220c103b00 0x7f220c101370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:44.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.530+0000 7f2212518640 1 -- 172.21.15.97:0/3351251937 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f220c10c520 msgr2=0x7f220c1018b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:44.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.530+0000 7f2212518640 1 --2- 172.21.15.97:0/3351251937 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f220c10c520 0x7f220c1018b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:44.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.530+0000 7f2212518640 1 -- 172.21.15.97:0/3351251937 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f220c1a6b10 con 0x7f220c1031e0 2024-04-23T10:36:44.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.530+0000 7f2212518640 1 --2- 172.21.15.97:0/3351251937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f220c1031e0 0x7f220c100e30 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f2208005940 tx=0x7f22080042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:36:44.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.530+0000 7f22037fe640 1 -- 172.21.15.97:0/3351251937 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f220803f030 con 0x7f220c1031e0 2024-04-23T10:36:44.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.530+0000 7f221477c640 1 -- 172.21.15.97:0/3351251937 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f220c1a6da0 con 0x7f220c1031e0 2024-04-23T10:36:44.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.530+0000 7f22037fe640 1 -- 172.21.15.97:0/3351251937 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2208003cb0 con 0x7f220c1031e0 2024-04-23T10:36:44.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.530+0000 7f22037fe640 1 -- 172.21.15.97:0/3351251937 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2208035070 con 0x7f220c1031e0 2024-04-23T10:36:44.532 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.530+0000 7f221477c640 1 -- 172.21.15.97:0/3351251937 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f220c1a7230 con 0x7f220c1031e0 2024-04-23T10:36:44.538 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.532+0000 7f22037fe640 1 -- 172.21.15.97:0/3351251937 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2208042030 con 0x7f220c1031e0 2024-04-23T10:36:44.538 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.532+0000 7f2217568740 1 Processor -- start 2024-04-23T10:36:44.538 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.533+0000 7f2217568740 1 -- start start 2024-04-23T10:36:44.538 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.533+0000 7f2217568740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e442da0620 0x55e442da0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:44.538 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.533+0000 7f2217568740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e442da0f40 0x55e442dab2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:44.538 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.533+0000 7f2217568740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e442dab800 0x55e442dadbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:44.538 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.533+0000 7f2217568740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e442d05970 con 0x55e442da0620 2024-04-23T10:36:44.538 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.533+0000 7f2217568740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e442ca24a0 con 0x55e442da0f40 2024-04-23T10:36:44.538 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.533+0000 7f22037fe640 1 --2- 172.21.15.97:0/3351251937 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f21f80a3df0 0x7f21f80a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:44.538 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.533+0000 7f2217568740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e442d01930 con 0x55e442dab800 2024-04-23T10:36:44.538 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.533+0000 7f2212d19640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e442da0620 0x55e442da0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:44.538 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.533+0000 7f2212d19640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e442da0620 0x55e442da0a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59980/0 (socket says 172.21.15.97:59980) 2024-04-23T10:36:44.539 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.533+0000 7f2212d19640 1 -- 172.21.15.97:0/912901793 learned_addr learned my addr 172.21.15.97:0/912901793 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:36:44.539 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.533+0000 7f2212d19640 1 --2- 172.21.15.97:0/3351251937 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f21f80a3df0 0x7f21f80a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:44.539 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.533+0000 7f2212518640 1 --2- 172.21.15.97:0/912901793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e442da0f40 0x55e442dab2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:44.539 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.533+0000 7f2211d17640 1 --2- 172.21.15.97:0/912901793 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e442dab800 0x55e442dadbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:44.539 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.533+0000 7f22037fe640 1 -- 172.21.15.97:0/3351251937 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8484+0+0 (secure 0 0 0) 0x7f22080f0cb0 con 0x7f220c1031e0 2024-04-23T10:36:44.539 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.533+0000 7f2212d19640 1 -- 172.21.15.97:0/912901793 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e442dab800 msgr2=0x55e442dadbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:36:44.539 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.533+0000 7f2212d19640 1 --2- 172.21.15.97:0/912901793 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e442dab800 0x55e442dadbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:44.539 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.533+0000 7f2212d19640 1 -- 172.21.15.97:0/912901793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e442da0f40 msgr2=0x55e442dab2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:36:44.539 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.533+0000 7f2212d19640 1 --2- 172.21.15.97:0/912901793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e442da0f40 0x55e442dab2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:44.539 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.533+0000 7f2212d19640 1 -- 172.21.15.97:0/912901793 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e442a18210 con 0x55e442da0620 2024-04-23T10:36:44.539 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.534+0000 7f2212d19640 1 --2- 172.21.15.97:0/912901793 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e442da0620 0x55e442da0a00 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f2204012d00 tx=0x7f2204010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=366872b5ace505b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:36:44.539 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.534+0000 7f2212d19640 1 --2- 172.21.15.97:0/3351251937 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f21f80a3df0 0x7f21f80a62b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2204010c40 tx=0x7f220400c000 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:36:44.539 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.534+0000 7f2200ff9640 1 -- 172.21.15.97:0/912901793 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2204024070 con 0x55e442da0620 2024-04-23T10:36:44.539 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.534+0000 7f2200ff9640 1 -- 172.21.15.97:0/912901793 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2204009d20 con 0x55e442da0620 2024-04-23T10:36:44.540 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.534+0000 7f2200ff9640 1 -- 172.21.15.97:0/912901793 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2204005260 con 0x55e442da0620 2024-04-23T10:36:44.540 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.535+0000 7f2217568740 1 -- 172.21.15.97:0/912901793 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e442da0620 msgr2=0x55e442da0a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:44.540 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.535+0000 7f2217568740 1 --2- 172.21.15.97:0/912901793 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e442da0620 0x55e442da0a00 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f2204012d00 tx=0x7f2204010740 comp rx=0 tx=0).stop 2024-04-23T10:36:44.540 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.535+0000 7f2217568740 1 -- 172.21.15.97:0/912901793 shutdown_connections 2024-04-23T10:36:44.540 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.535+0000 7f2217568740 1 --2- 172.21.15.97:0/912901793 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e442dab800 0x55e442dadbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:44.540 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.535+0000 7f2217568740 1 --2- 172.21.15.97:0/912901793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e442da0f40 0x55e442dab2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:44.540 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.535+0000 7f2217568740 1 --2- 172.21.15.97:0/912901793 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e442da0620 0x55e442da0a00 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:44.540 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.535+0000 7f2217568740 1 -- 172.21.15.97:0/912901793 >> 172.21.15.97:0/912901793 conn(0x55e442d9cb30 msgr2=0x55e442d9cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:36:44.540 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.535+0000 7f2217568740 1 -- 172.21.15.97:0/912901793 shutdown_connections 2024-04-23T10:36:44.540 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.535+0000 7f2217568740 1 -- 172.21.15.97:0/912901793 wait complete. 2024-04-23T10:36:44.540 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.535+0000 7f2217568740 1 -- start start 2024-04-23T10:36:44.540 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.535+0000 7f2217568740 1 Processor -- start 2024-04-23T10:36:44.540 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.535+0000 7f2217568740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e442da0620 0x55e442dab1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:44.540 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.535+0000 7f2217568740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e442da0f40 0x55e442da84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:44.540 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.535+0000 7f2217568740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e442dab800 0x55e442da8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.535+0000 7f2217568740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e442dae3d0 con 0x55e442da0620 2024-04-23T10:36:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.535+0000 7f2217568740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e442ca00b0 con 0x55e442dab800 2024-04-23T10:36:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.535+0000 7f2217568740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e442dae550 con 0x55e442da0f40 2024-04-23T10:36:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.535+0000 7f2217568740 10 client.0 _ll_register_callbacks cb 0x55e442c8dd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:36:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.535+0000 7f2212518640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e442da0620 0x55e442dab1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.535+0000 7f2212518640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e442da0620 0x55e442dab1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59986/0 (socket says 172.21.15.97:59986) 2024-04-23T10:36:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.535+0000 7f2212518640 1 -- 172.21.15.97:0/1308503333 learned_addr learned my addr 172.21.15.97:0/1308503333 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:36:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.536+0000 7f2211d17640 1 --2- 172.21.15.97:0/1308503333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e442da0f40 0x55e442da84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.536+0000 7f2212d19640 1 --2- 172.21.15.97:0/1308503333 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e442dab800 0x55e442da8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.536+0000 7f2212518640 1 -- 172.21.15.97:0/1308503333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e442da0f40 msgr2=0x55e442da84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:36:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.536+0000 7f2212518640 1 --2- 172.21.15.97:0/1308503333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e442da0f40 0x55e442da84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.536+0000 7f2212518640 1 -- 172.21.15.97:0/1308503333 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e442dab800 msgr2=0x55e442da8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:36:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.536+0000 7f2212518640 1 --2- 172.21.15.97:0/1308503333 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e442dab800 0x55e442da8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.536+0000 7f2212518640 1 -- 172.21.15.97:0/1308503333 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e442ae4620 con 0x55e442da0620 2024-04-23T10:36:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.536+0000 7f2212d19640 1 --2- 172.21.15.97:0/1308503333 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e442dab800 0x55e442da8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:36:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.536+0000 7f2211d17640 1 --2- 172.21.15.97:0/1308503333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e442da0f40 0x55e442da84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:36:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.536+0000 7f2212518640 1 --2- 172.21.15.97:0/1308503333 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e442da0620 0x55e442dab1e0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f22080ee4d0 tx=0x7f220803b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abef9d4d50c4de75 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:36:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.536+0000 7f21f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:36:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.536+0000 7f21f5ffb640 1 -- 172.21.15.97:0/1308503333 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2208049040 con 0x55e442da0620 2024-04-23T10:36:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.536+0000 7f21f5ffb640 1 -- 172.21.15.97:0/1308503333 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8484+0+0 (secure 0 0 0) 0x7f22080b92e0 con 0x55e442da0620 2024-04-23T10:36:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.536+0000 7f2217568740 1 -- 172.21.15.97:0/1308503333 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e442d13390 con 0x55e442da0620 2024-04-23T10:36:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.536+0000 7f21f5ffb640 1 -- 172.21.15.97:0/1308503333 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2208005d50 con 0x55e442da0620 2024-04-23T10:36:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.536+0000 7f2217568740 10 client.17097 fetch_fsmap learned FSMap version 26 2024-04-23T10:36:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.536+0000 7f2217568740 1 -- 172.21.15.97:0/1308503333 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e442dba380 con 0x55e442da0620 2024-04-23T10:36:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.536+0000 7f21f5ffb640 1 -- 172.21.15.97:0/1308503333 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55e442d13390 con 0x55e442da0620 2024-04-23T10:36:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.537+0000 7f21f5ffb640 1 -- 172.21.15.97:0/1308503333 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f22080dd0b0 con 0x55e442da0620 2024-04-23T10:36:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.537+0000 7f2217568740 10 client.17097 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:36:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.537+0000 7f2217568740 10 client.17097 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:36:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.537+0000 7f2217568740 20 client.17097 populate_metadata read hostname 'smithi097' 2024-04-23T10:36:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.537+0000 7f2217568740 1 --2- 172.21.15.97:0/1308503333 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55e442dbb830 0x55e442dbdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.537+0000 7f2217568740 4 client.17097 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:36:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.537+0000 7f2217568740 1 -- 172.21.15.97:0/1308503333 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e442d0d200 con 0x55e442dbb830 2024-04-23T10:36:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.537+0000 7f2211d17640 1 --2- 172.21.15.97:0/1308503333 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55e442dbb830 0x55e442dbdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.538+0000 7f21f5ffb640 10 client.17097 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:36:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.538+0000 7f2211d17640 1 --2- 172.21.15.97:0/1308503333 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55e442dbb830 0x55e442dbdc60 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=641e889f90becb8c server_cookie=29f729fe1befcbc4 in_seq=0 out_seq=0 2024-04-23T10:36:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.538+0000 7f21f5ffb640 1 -- 172.21.15.97:0/1308503333 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e442d0d200 con 0x55e442dbb830 2024-04-23T10:36:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.538+0000 7f21f5ffb640 10 client.17097 handle_command_reply: tid=0 2024-04-23T10:36:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.538+0000 7f2217568740 1 client.17097 shutdown 2024-04-23T10:36:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.538+0000 7f2217568740 1 -- 172.21.15.97:0/1308503333 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e442da0620 msgr2=0x55e442dab1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.538+0000 7f2217568740 1 --2- 172.21.15.97:0/1308503333 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e442da0620 0x55e442dab1e0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f22080ee4d0 tx=0x7f220803b000 comp rx=0 tx=0).stop 2024-04-23T10:36:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.538+0000 7f2217568740 1 -- 172.21.15.97:0/1308503333 shutdown_connections 2024-04-23T10:36:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.538+0000 7f2217568740 1 --2- 172.21.15.97:0/1308503333 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55e442dbb830 0x55e442dbdc60 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.538+0000 7f2217568740 1 --2- 172.21.15.97:0/1308503333 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e442dab800 0x55e442da8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:44.543 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.538+0000 7f2217568740 1 --2- 172.21.15.97:0/1308503333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e442da0f40 0x55e442da84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.538+0000 7f21f5ffb640 0 client.17097 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:36:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.538+0000 7f2217568740 1 --2- 172.21.15.97:0/1308503333 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e442da0620 0x55e442dab1e0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.538+0000 7f2217568740 1 -- 172.21.15.97:0/1308503333 >> 172.21.15.97:0/1308503333 conn(0x55e442d9cb30 msgr2=0x55e442da2860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:36:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.538+0000 7f2217568740 1 -- 172.21.15.97:0/1308503333 shutdown_connections 2024-04-23T10:36:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.539+0000 7f2217568740 1 -- 172.21.15.97:0/1308503333 wait complete. 2024-04-23T10:36:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.539+0000 7f2217568740 20 client.17097 trim_cache size 0 max 16384 2024-04-23T10:36:44.556 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.553+0000 7f2217568740 1 Processor -- start 2024-04-23T10:36:44.556 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.553+0000 7f2217568740 1 -- start start 2024-04-23T10:36:44.556 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.553+0000 7f2217568740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e442da0620 0x55e442db7290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:44.556 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.553+0000 7f2217568740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e442da0f40 0x55e442daad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:44.556 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.553+0000 7f2217568740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e442dab800 0x55e442dc3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.553+0000 7f2217568740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f220c1afbb0 con 0x55e442dab800 2024-04-23T10:36:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.553+0000 7f2217568740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f220c1b16f0 con 0x55e442da0620 2024-04-23T10:36:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.553+0000 7f2217568740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f220c1b0280 con 0x55e442da0f40 2024-04-23T10:36:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.554+0000 7f2212d19640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e442dab800 0x55e442dc3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.554+0000 7f2212d19640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e442dab800 0x55e442dc3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59996/0 (socket says 172.21.15.97:59996) 2024-04-23T10:36:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.554+0000 7f2212d19640 1 -- 172.21.15.97:0/2032089353 learned_addr learned my addr 172.21.15.97:0/2032089353 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:36:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.554+0000 7f2212d19640 1 -- 172.21.15.97:0/2032089353 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e442da0f40 msgr2=0x55e442daad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:36:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.554+0000 7f2212d19640 1 --2- 172.21.15.97:0/2032089353 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e442da0f40 0x55e442daad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.554+0000 7f2212d19640 1 -- 172.21.15.97:0/2032089353 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e442da0620 msgr2=0x55e442db7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:36:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.554+0000 7f2212d19640 1 --2- 172.21.15.97:0/2032089353 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e442da0620 0x55e442db7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.554+0000 7f2212d19640 1 -- 172.21.15.97:0/2032089353 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e442da6a20 con 0x55e442dab800 2024-04-23T10:36:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.554+0000 7f2212d19640 1 --2- 172.21.15.97:0/2032089353 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e442dab800 0x55e442dc3570 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f22040096e0 tx=0x7f2204005c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2677a4bcd7e2bebe server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:36:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.554+0000 7f21f7fff640 1 -- 172.21.15.97:0/2032089353 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2204024040 con 0x55e442dab800 2024-04-23T10:36:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.554+0000 7f21f7fff640 1 -- 172.21.15.97:0/2032089353 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22040059c0 con 0x55e442dab800 2024-04-23T10:36:44.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.554+0000 7f21f7fff640 1 -- 172.21.15.97:0/2032089353 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2204004c60 con 0x55e442dab800 2024-04-23T10:36:44.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.555+0000 7f2217568740 1 -- 172.21.15.97:0/2032089353 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e442dab800 msgr2=0x55e442dc3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:44.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.555+0000 7f2217568740 1 --2- 172.21.15.97:0/2032089353 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e442dab800 0x55e442dc3570 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f22040096e0 tx=0x7f2204005c30 comp rx=0 tx=0).stop 2024-04-23T10:36:44.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.555+0000 7f2217568740 1 -- 172.21.15.97:0/2032089353 shutdown_connections 2024-04-23T10:36:44.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.555+0000 7f2217568740 1 --2- 172.21.15.97:0/2032089353 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e442dab800 0x55e442dc3570 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:44.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.555+0000 7f2217568740 1 --2- 172.21.15.97:0/2032089353 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e442da0f40 0x55e442daad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:44.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.555+0000 7f2217568740 1 --2- 172.21.15.97:0/2032089353 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e442da0620 0x55e442db7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:44.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.555+0000 7f2217568740 1 -- 172.21.15.97:0/2032089353 >> 172.21.15.97:0/2032089353 conn(0x55e442d9cb30 msgr2=0x55e442da2860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:36:44.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.555+0000 7f2217568740 1 -- 172.21.15.97:0/2032089353 shutdown_connections 2024-04-23T10:36:44.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.555+0000 7f2217568740 1 -- 172.21.15.97:0/2032089353 wait complete. 2024-04-23T10:36:44.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.555+0000 7f2217568740 1 -- start start 2024-04-23T10:36:44.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.555+0000 7f2217568740 1 Processor -- start 2024-04-23T10:36:44.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.555+0000 7f2217568740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e442da0620 0x55e442da5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:44.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.555+0000 7f2217568740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e442da0f40 0x55e442da57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:44.559 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.555+0000 7f2217568740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e442dab800 0x55e442da5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:44.559 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.555+0000 7f2217568740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f220c1b1020 con 0x55e442da0f40 2024-04-23T10:36:44.559 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.555+0000 7f2217568740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f220c1b0950 con 0x55e442da0620 2024-04-23T10:36:44.559 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.555+0000 7f2217568740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f220c1adbb0 con 0x55e442dab800 2024-04-23T10:36:44.559 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.555+0000 7f2217568740 10 client.0 _ll_register_callbacks cb 0x7f22080b9000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:36:44.559 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.555+0000 7f2211d17640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e442da0f40 0x55e442da57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:44.559 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.556+0000 7f2211d17640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e442da0f40 0x55e442da57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59998/0 (socket says 172.21.15.97:59998) 2024-04-23T10:36:44.559 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.556+0000 7f2211d17640 1 -- 172.21.15.97:0/2224461596 learned_addr learned my addr 172.21.15.97:0/2224461596 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:36:44.559 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.556+0000 7f2212518640 1 --2- 172.21.15.97:0/2224461596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e442da0620 0x55e442da5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:44.559 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.556+0000 7f2212d19640 1 --2- 172.21.15.97:0/2224461596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e442dab800 0x55e442da5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:44.559 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.556+0000 7f2211d17640 1 -- 172.21.15.97:0/2224461596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e442dab800 msgr2=0x55e442da5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:36:44.559 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.556+0000 7f2211d17640 1 --2- 172.21.15.97:0/2224461596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e442dab800 0x55e442da5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:44.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.556+0000 7f2211d17640 1 -- 172.21.15.97:0/2224461596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e442da0620 msgr2=0x55e442da5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:36:44.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.556+0000 7f2211d17640 1 --2- 172.21.15.97:0/2224461596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e442da0620 0x55e442da5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:44.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.556+0000 7f2211d17640 1 -- 172.21.15.97:0/2224461596 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e442e20eb0 con 0x55e442da0f40 2024-04-23T10:36:44.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.556+0000 7f2211d17640 1 --2- 172.21.15.97:0/2224461596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e442da0f40 0x55e442da57c0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x55e442d9dcc0 tx=0x7f21fc0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79ecbcfee326e405 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:36:44.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.556+0000 7f21f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:36:44.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.556+0000 7f21f5ffb640 1 -- 172.21.15.97:0/2224461596 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21fc020020 con 0x55e442da0f40 2024-04-23T10:36:44.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.556+0000 7f21f5ffb640 1 -- 172.21.15.97:0/2224461596 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8484+0+0 (secure 0 0 0) 0x7f21fc01c620 con 0x55e442da0f40 2024-04-23T10:36:44.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.556+0000 7f2217568740 1 -- 172.21.15.97:0/2224461596 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22040059c0 con 0x55e442da0f40 2024-04-23T10:36:44.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.556+0000 7f21f5ffb640 1 -- 172.21.15.97:0/2224461596 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f21fc02b050 con 0x55e442da0f40 2024-04-23T10:36:44.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.556+0000 7f2217568740 10 client.17103 fetch_fsmap learned FSMap version 26 2024-04-23T10:36:44.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.556+0000 7f2217568740 1 -- 172.21.15.97:0/2224461596 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e442e22250 con 0x55e442da0f40 2024-04-23T10:36:44.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.556+0000 7f21f5ffb640 1 -- 172.21.15.97:0/2224461596 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22040059c0 con 0x55e442da0f40 2024-04-23T10:36:44.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.557+0000 7f21f5ffb640 1 -- 172.21.15.97:0/2224461596 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f21fc00f3c0 con 0x55e442da0f40 2024-04-23T10:36:44.560 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.557+0000 7f2217568740 10 client.17103 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:36:44.561 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.557+0000 7f2217568740 10 client.17103 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:36:44.561 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.557+0000 7f2217568740 20 client.17103 populate_metadata read hostname 'smithi097' 2024-04-23T10:36:44.561 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.557+0000 7f2217568740 1 --2- 172.21.15.97:0/2224461596 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55e442e22a50 0x55e442e24e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:44.561 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.557+0000 7f2217568740 4 client.17103 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:36:44.561 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.557+0000 7f2217568740 1 -- 172.21.15.97:0/2224461596 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e442c895f0 con 0x55e442e22a50 2024-04-23T10:36:44.561 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.557+0000 7f2212518640 1 --2- 172.21.15.97:0/2224461596 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55e442e22a50 0x55e442e24e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:44.561 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.557+0000 7f21f5ffb640 10 client.17103 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:36:44.561 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.557+0000 7f2212518640 1 --2- 172.21.15.97:0/2224461596 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55e442e22a50 0x55e442e24e70 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42fc0ad1d665a05d server_cookie=67b884ac673c5200 in_seq=0 out_seq=0 2024-04-23T10:36:44.569 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.570+0000 7f21f5ffb640 1 -- 172.21.15.97:0/2224461596 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e442c895f0 con 0x55e442e22a50 2024-04-23T10:36:44.569 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.570+0000 7f21f5ffb640 10 client.17103 handle_command_reply: tid=0 2024-04-23T10:36:44.569 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.570+0000 7f2217568740 1 client.17103 shutdown 2024-04-23T10:36:44.569 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.570+0000 7f2217568740 1 -- 172.21.15.97:0/2224461596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e442da0f40 msgr2=0x55e442da57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:44.569 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.570+0000 7f2217568740 1 --2- 172.21.15.97:0/2224461596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e442da0f40 0x55e442da57c0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x55e442d9dcc0 tx=0x7f21fc0039c0 comp rx=0 tx=0).stop 2024-04-23T10:36:44.569 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.570+0000 7f2217568740 1 -- 172.21.15.97:0/2224461596 shutdown_connections 2024-04-23T10:36:44.570 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.570+0000 7f2217568740 1 --2- 172.21.15.97:0/2224461596 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55e442e22a50 0x55e442e24e70 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:44.570 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.570+0000 7f2217568740 1 --2- 172.21.15.97:0/2224461596 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e442dab800 0x55e442da5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:44.570 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.570+0000 7f21f5ffb640 0 client.17103 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:36:44.570 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.570+0000 7f2217568740 1 --2- 172.21.15.97:0/2224461596 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e442da0f40 0x55e442da57c0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:44.570 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.570+0000 7f2217568740 1 --2- 172.21.15.97:0/2224461596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e442da0620 0x55e442da5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:44.570 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.570+0000 7f2217568740 1 -- 172.21.15.97:0/2224461596 >> 172.21.15.97:0/2224461596 conn(0x55e442d9cb30 msgr2=0x55e442da1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:36:44.570 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.570+0000 7f2217568740 1 -- 172.21.15.97:0/2224461596 shutdown_connections 2024-04-23T10:36:44.570 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.570+0000 7f2217568740 1 -- 172.21.15.97:0/2224461596 wait complete. 2024-04-23T10:36:44.570 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.570+0000 7f2217568740 20 client.17103 trim_cache size 0 max 16384 2024-04-23T10:36:44.570 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:36:44.571 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:36:44.571 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:36:44.571 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:36:44.571 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.571+0000 7f221477c640 1 -- 172.21.15.97:0/3351251937 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f21f80a3df0 msgr2=0x7f21f80a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.571+0000 7f221477c640 1 --2- 172.21.15.97:0/3351251937 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f21f80a3df0 0x7f21f80a62b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2204010c40 tx=0x7f220400c000 comp rx=0 tx=0).stop 2024-04-23T10:36:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.571+0000 7f221477c640 1 -- 172.21.15.97:0/3351251937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f220c1031e0 msgr2=0x7f220c100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.571+0000 7f221477c640 1 --2- 172.21.15.97:0/3351251937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f220c1031e0 0x7f220c100e30 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f2208005940 tx=0x7f22080042f0 comp rx=0 tx=0).stop 2024-04-23T10:36:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.571+0000 7f221477c640 1 -- 172.21.15.97:0/3351251937 shutdown_connections 2024-04-23T10:36:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.571+0000 7f221477c640 1 --2- 172.21.15.97:0/3351251937 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f21f80a3df0 0x7f21f80a62b0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.571+0000 7f221477c640 1 --2- 172.21.15.97:0/3351251937 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f220c10c520 0x7f220c1018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.571+0000 7f221477c640 1 --2- 172.21.15.97:0/3351251937 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f220c103b00 0x7f220c101370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.571+0000 7f221477c640 1 --2- 172.21.15.97:0/3351251937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f220c1031e0 0x7f220c100e30 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.571+0000 7f221477c640 1 -- 172.21.15.97:0/3351251937 >> 172.21.15.97:0/3351251937 conn(0x7f220c077fd0 msgr2=0x7f220c0fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:36:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.571+0000 7f221477c640 1 -- 172.21.15.97:0/3351251937 shutdown_connections 2024-04-23T10:36:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:44.571+0000 7f221477c640 1 -- 172.21.15.97:0/3351251937 wait complete. 2024-04-23T10:36:45.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:36:44 smithi196 ceph-mon[32919]: pgmap v327: 97 pgs: 97 active+clean; 3.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 47 KiB/s wr, 20 op/s 2024-04-23T10:36:45.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:36:44 smithi097 ceph-mon[29197]: pgmap v327: 97 pgs: 97 active+clean; 3.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 47 KiB/s wr, 20 op/s 2024-04-23T10:36:45.399 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:36:44 smithi183 ceph-mon[32317]: pgmap v327: 97 pgs: 97 active+clean; 3.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 47 KiB/s wr, 20 op/s 2024-04-23T10:36:47.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:36:46 smithi196 ceph-mon[32919]: pgmap v328: 97 pgs: 97 active+clean; 3.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 23 KiB/s wr, 11 op/s 2024-04-23T10:36:47.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:36:46 smithi097 ceph-mon[29197]: pgmap v328: 97 pgs: 97 active+clean; 3.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 23 KiB/s wr, 11 op/s 2024-04-23T10:36:47.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:36:46 smithi183 ceph-mon[32317]: pgmap v328: 97 pgs: 97 active+clean; 3.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 23 KiB/s wr, 11 op/s 2024-04-23T10:36:49.202 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:36:48 smithi183 ceph-mon[32317]: pgmap v329: 97 pgs: 97 active+clean; 3.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 29 KiB/s wr, 18 op/s 2024-04-23T10:36:49.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:36:48 smithi196 ceph-mon[32919]: pgmap v329: 97 pgs: 97 active+clean; 3.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 29 KiB/s wr, 18 op/s 2024-04-23T10:36:49.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:36:48 smithi097 ceph-mon[29197]: pgmap v329: 97 pgs: 97 active+clean; 3.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 29 KiB/s wr, 18 op/s 2024-04-23T10:36:51.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:36:50 smithi196 ceph-mon[32919]: pgmap v330: 97 pgs: 97 active+clean; 3.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-23T10:36:51.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:36:50 smithi097 ceph-mon[29197]: pgmap v330: 97 pgs: 97 active+clean; 3.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-23T10:36:51.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:36:50 smithi183 ceph-mon[32317]: pgmap v330: 97 pgs: 97 active+clean; 3.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-23T10:36:53.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:36:52 smithi196 ceph-mon[32919]: pgmap v331: 97 pgs: 97 active+clean; 3.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-23T10:36:53.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:36:52 smithi097 ceph-mon[29197]: pgmap v331: 97 pgs: 97 active+clean; 3.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-23T10:36:53.399 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:36:52 smithi183 ceph-mon[32317]: pgmap v331: 97 pgs: 97 active+clean; 3.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-23T10:36:55.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:36:54 smithi196 ceph-mon[32919]: pgmap v332: 97 pgs: 97 active+clean; 3.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-04-23T10:36:55.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:36:54 smithi097 ceph-mon[29197]: pgmap v332: 97 pgs: 97 active+clean; 3.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-04-23T10:36:55.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:36:54 smithi183 ceph-mon[32317]: pgmap v332: 97 pgs: 97 active+clean; 3.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-04-23T10:36:57.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:36:56 smithi196 ceph-mon[32919]: pgmap v333: 97 pgs: 97 active+clean; 3.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 8.0 KiB/s wr, 11 op/s 2024-04-23T10:36:57.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:36:56 smithi097 ceph-mon[29197]: pgmap v333: 97 pgs: 97 active+clean; 3.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 8.0 KiB/s wr, 11 op/s 2024-04-23T10:36:57.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:36:56 smithi183 ceph-mon[32317]: pgmap v333: 97 pgs: 97 active+clean; 3.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 8.0 KiB/s wr, 11 op/s 2024-04-23T10:36:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.667+0000 7fb7fe67c640 1 -- 172.21.15.97:0/347500296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7f8103b20 msgr2=0x7fb7f810c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.667+0000 7fb7fe67c640 1 --2- 172.21.15.97:0/347500296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7f8103b20 0x7fb7f810c000 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec009900 tx=0x7fb7ec02a480 comp rx=0 tx=0).stop 2024-04-23T10:36:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.667+0000 7fb7fe67c640 1 -- 172.21.15.97:0/347500296 shutdown_connections 2024-04-23T10:36:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.667+0000 7fb7fe67c640 1 --2- 172.21.15.97:0/347500296 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb7f810c5d0 0x7fb7f810ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.667+0000 7fb7fe67c640 1 --2- 172.21.15.97:0/347500296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7f8103b20 0x7fb7f810c000 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.667+0000 7fb7fe67c640 1 --2- 172.21.15.97:0/347500296 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb7f8103200 0x7fb7f81035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.667+0000 7fb7fe67c640 1 -- 172.21.15.97:0/347500296 >> 172.21.15.97:0/347500296 conn(0x7fb7f80771d0 msgr2=0x7fb7f80775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:36:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.667+0000 7fb7fe67c640 1 -- 172.21.15.97:0/347500296 shutdown_connections 2024-04-23T10:36:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.667+0000 7fb7fe67c640 1 -- 172.21.15.97:0/347500296 wait complete. 2024-04-23T10:36:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.668+0000 7fb7fe67c640 1 Processor -- start 2024-04-23T10:36:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.668+0000 7fb7fe67c640 1 -- start start 2024-04-23T10:36:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.668+0000 7fb7fe67c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb7f8103200 0x7fb7f8198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.668+0000 7fb7fe67c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7f8103b20 0x7fb7f8198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.668+0000 7fb7fe67c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb7f810c5d0 0x7fb7f819ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.668+0000 7fb7fe67c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f81121f0 con 0x7fb7f8103b20 2024-04-23T10:36:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.668+0000 7fb7fe67c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f8112070 con 0x7fb7f810c5d0 2024-04-23T10:36:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.668+0000 7fb7fe67c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f8112370 con 0x7fb7f8103200 2024-04-23T10:36:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.668+0000 7fb7f77fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7f8103b20 0x7fb7f8198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.668+0000 7fb7f77fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7f8103b20 0x7fb7f8198b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40980/0 (socket says 172.21.15.97:40980) 2024-04-23T10:36:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.668+0000 7fb7f77fe640 1 -- 172.21.15.97:0/1836385113 learned_addr learned my addr 172.21.15.97:0/1836385113 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:36:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.669+0000 7fb7f7fff640 1 --2- 172.21.15.97:0/1836385113 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb7f8103200 0x7fb7f8198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.669+0000 7fb7fcc19640 1 --2- 172.21.15.97:0/1836385113 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb7f810c5d0 0x7fb7f819ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.669+0000 7fb7f77fe640 1 -- 172.21.15.97:0/1836385113 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb7f8103200 msgr2=0x7fb7f8198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.669+0000 7fb7f77fe640 1 --2- 172.21.15.97:0/1836385113 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb7f8103200 0x7fb7f8198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.669+0000 7fb7f77fe640 1 -- 172.21.15.97:0/1836385113 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb7f810c5d0 msgr2=0x7fb7f819ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.669+0000 7fb7f77fe640 1 --2- 172.21.15.97:0/1836385113 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb7f810c5d0 0x7fb7f819ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.669+0000 7fb7f77fe640 1 -- 172.21.15.97:0/1836385113 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7f819d5b0 con 0x7fb7f8103b20 2024-04-23T10:36:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.669+0000 7fb7f7fff640 1 --2- 172.21.15.97:0/1836385113 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb7f8103200 0x7fb7f8198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:36:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.669+0000 7fb7f77fe640 1 --2- 172.21.15.97:0/1836385113 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7f8103b20 0x7fb7f8198b50 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec006010 tx=0x7fb7ec0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:36:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.669+0000 7fb7f57fa640 1 -- 172.21.15.97:0/1836385113 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec03f030 con 0x7fb7f8103b20 2024-04-23T10:36:57.671 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.669+0000 7fb7f57fa640 1 -- 172.21.15.97:0/1836385113 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7ec003800 con 0x7fb7f8103b20 2024-04-23T10:36:57.671 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.669+0000 7fb7fe67c640 1 -- 172.21.15.97:0/1836385113 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7f819d840 con 0x7fb7f8103b20 2024-04-23T10:36:57.671 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.669+0000 7fb7f57fa640 1 -- 172.21.15.97:0/1836385113 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec035070 con 0x7fb7f8103b20 2024-04-23T10:36:57.671 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.669+0000 7fb7fe67c640 1 -- 172.21.15.97:0/1836385113 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7f81a5130 con 0x7fb7f8103b20 2024-04-23T10:36:57.678 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.672+0000 7fb7f57fa640 1 -- 172.21.15.97:0/1836385113 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb7ec003cb0 con 0x7fb7f8103b20 2024-04-23T10:36:57.678 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.672+0000 7fb801396740 1 Processor -- start 2024-04-23T10:36:57.678 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.672+0000 7fb801396740 1 -- start start 2024-04-23T10:36:57.678 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.672+0000 7fb7f57fa640 1 --2- 172.21.15.97:0/1836385113 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fb7e00a3dc0 0x7fb7e00a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:57.678 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.672+0000 7fb801396740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dabe77a620 0x55dabe77aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:57.678 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.672+0000 7fb801396740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dabe77af40 0x55dabe7852c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:57.678 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.672+0000 7fb801396740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dabe785800 0x55dabe787bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:57.678 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.672+0000 7fb801396740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55dabe6df970 con 0x55dabe77af40 2024-04-23T10:36:57.678 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.672+0000 7fb801396740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55dabe67c4a0 con 0x55dabe785800 2024-04-23T10:36:57.678 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.672+0000 7fb801396740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55dabe6db930 con 0x55dabe77a620 2024-04-23T10:36:57.678 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.672+0000 7fb7fcc19640 1 --2- 172.21.15.97:0/1836385113 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fb7e00a3dc0 0x7fb7e00a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:57.678 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.672+0000 7fb7f77fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dabe77af40 0x55dabe7852c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:57.679 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.672+0000 7fb7f77fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dabe77af40 0x55dabe7852c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40996/0 (socket says 172.21.15.97:40996) 2024-04-23T10:36:57.679 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.672+0000 7fb7f77fe640 1 -- 172.21.15.97:0/2394712749 learned_addr learned my addr 172.21.15.97:0/2394712749 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:36:57.679 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.673+0000 7fb7f7fff640 1 --2- 172.21.15.97:0/2394712749 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dabe77a620 0x55dabe77aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:57.679 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.673+0000 7fb7f57fa640 1 -- 172.21.15.97:0/1836385113 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8484+0+0 (secure 0 0 0) 0x7fb7ec0f1220 con 0x7fb7f8103b20 2024-04-23T10:36:57.679 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.673+0000 7fb7fcc19640 1 --2- 172.21.15.97:0/2394712749 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dabe785800 0x55dabe787bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:57.679 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.673+0000 7fb7f77fe640 1 -- 172.21.15.97:0/2394712749 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dabe77a620 msgr2=0x55dabe77aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:36:57.679 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.673+0000 7fb7f77fe640 1 --2- 172.21.15.97:0/2394712749 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dabe77a620 0x55dabe77aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:57.679 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.673+0000 7fb7f77fe640 1 -- 172.21.15.97:0/2394712749 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dabe785800 msgr2=0x55dabe787bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:36:57.679 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.673+0000 7fb7f77fe640 1 --2- 172.21.15.97:0/2394712749 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dabe785800 0x55dabe787bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:57.679 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.673+0000 7fb7f77fe640 1 -- 172.21.15.97:0/2394712749 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dabe3f2210 con 0x55dabe77af40 2024-04-23T10:36:57.679 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.673+0000 7fb7fcc19640 1 --2- 172.21.15.97:0/2394712749 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dabe785800 0x55dabe787bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:36:57.679 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.673+0000 7fb7fcc19640 1 --2- 172.21.15.97:0/1836385113 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fb7e00a3dc0 0x7fb7e00a6280 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb7f806b860 tx=0x7fb7f0012960 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:36:57.679 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.673+0000 7fb7f77fe640 1 --2- 172.21.15.97:0/2394712749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dabe77af40 0x55dabe7852c0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec0bd170 tx=0x7fb7ec02a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9ef84b9f02aeb64 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:36:57.680 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.673+0000 7fb7de7fc640 1 -- 172.21.15.97:0/2394712749 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec049070 con 0x55dabe77af40 2024-04-23T10:36:57.680 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.673+0000 7fb7de7fc640 1 -- 172.21.15.97:0/2394712749 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7ec0f1020 con 0x55dabe77af40 2024-04-23T10:36:57.680 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.673+0000 7fb7de7fc640 1 -- 172.21.15.97:0/2394712749 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec003a40 con 0x55dabe77af40 2024-04-23T10:36:57.680 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.674+0000 7fb801396740 1 -- 172.21.15.97:0/2394712749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dabe77af40 msgr2=0x55dabe7852c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:57.680 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.674+0000 7fb801396740 1 --2- 172.21.15.97:0/2394712749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dabe77af40 0x55dabe7852c0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec0bd170 tx=0x7fb7ec02a480 comp rx=0 tx=0).stop 2024-04-23T10:36:57.680 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.674+0000 7fb801396740 1 -- 172.21.15.97:0/2394712749 shutdown_connections 2024-04-23T10:36:57.680 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.674+0000 7fb801396740 1 --2- 172.21.15.97:0/2394712749 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dabe785800 0x55dabe787bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:57.680 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.674+0000 7fb801396740 1 --2- 172.21.15.97:0/2394712749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dabe77af40 0x55dabe7852c0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:57.680 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.674+0000 7fb801396740 1 --2- 172.21.15.97:0/2394712749 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dabe77a620 0x55dabe77aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:57.680 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.674+0000 7fb801396740 1 -- 172.21.15.97:0/2394712749 >> 172.21.15.97:0/2394712749 conn(0x55dabe776b30 msgr2=0x55dabe776f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:36:57.680 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.674+0000 7fb801396740 1 -- 172.21.15.97:0/2394712749 shutdown_connections 2024-04-23T10:36:57.680 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.674+0000 7fb801396740 1 -- 172.21.15.97:0/2394712749 wait complete. 2024-04-23T10:36:57.680 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.674+0000 7fb801396740 1 -- start start 2024-04-23T10:36:57.680 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.675+0000 7fb801396740 1 Processor -- start 2024-04-23T10:36:57.681 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.675+0000 7fb801396740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dabe77a620 0x55dabe785190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:57.681 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.675+0000 7fb801396740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dabe77af40 0x55dabe782480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:57.681 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.675+0000 7fb801396740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dabe785800 0x55dabe782a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:57.681 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.675+0000 7fb801396740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55dabe7881f0 con 0x55dabe77a620 2024-04-23T10:36:57.681 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.675+0000 7fb801396740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55dabe67a0b0 con 0x55dabe785800 2024-04-23T10:36:57.681 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.675+0000 7fb801396740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55dabe788370 con 0x55dabe77af40 2024-04-23T10:36:57.681 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.675+0000 7fb801396740 10 client.0 _ll_register_callbacks cb 0x55dabe667d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:36:57.681 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.675+0000 7fb7f7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dabe77a620 0x55dabe785190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:57.681 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.675+0000 7fb7f7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dabe77a620 0x55dabe785190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41008/0 (socket says 172.21.15.97:41008) 2024-04-23T10:36:57.681 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.675+0000 7fb7f7fff640 1 -- 172.21.15.97:0/3734617522 learned_addr learned my addr 172.21.15.97:0/3734617522 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:36:57.681 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.675+0000 7fb7f77fe640 1 --2- 172.21.15.97:0/3734617522 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dabe77af40 0x55dabe782480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:57.681 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.675+0000 7fb7fcc19640 1 --2- 172.21.15.97:0/3734617522 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dabe785800 0x55dabe782a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:57.681 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.675+0000 7fb7f7fff640 1 -- 172.21.15.97:0/3734617522 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dabe77af40 msgr2=0x55dabe782480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:36:57.682 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.675+0000 7fb7f7fff640 1 --2- 172.21.15.97:0/3734617522 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dabe77af40 0x55dabe782480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:57.682 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.675+0000 7fb7f7fff640 1 -- 172.21.15.97:0/3734617522 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dabe785800 msgr2=0x55dabe782a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:36:57.682 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.675+0000 7fb7f7fff640 1 --2- 172.21.15.97:0/3734617522 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dabe785800 0x55dabe782a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:57.682 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.675+0000 7fb7f7fff640 1 -- 172.21.15.97:0/3734617522 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dabe4be620 con 0x55dabe77a620 2024-04-23T10:36:57.682 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.675+0000 7fb7f7fff640 1 --2- 172.21.15.97:0/3734617522 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dabe77a620 0x55dabe785190 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8011a30 tx=0x7fb7e8011ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c57e6a15c6d705c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:36:57.682 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.675+0000 7fb7d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:36:57.682 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.676+0000 7fb7d77fe640 1 -- 172.21.15.97:0/3734617522 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e8004460 con 0x55dabe77a620 2024-04-23T10:36:57.682 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.676+0000 7fb7d77fe640 1 -- 172.21.15.97:0/3734617522 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8484+0+0 (secure 0 0 0) 0x7fb7e800f9a0 con 0x55dabe77a620 2024-04-23T10:36:57.682 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.676+0000 7fb801396740 1 -- 172.21.15.97:0/3734617522 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7ec0f1020 con 0x55dabe77a620 2024-04-23T10:36:57.682 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.676+0000 7fb7d77fe640 1 -- 172.21.15.97:0/3734617522 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb7e800fba0 con 0x55dabe77a620 2024-04-23T10:36:57.682 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.676+0000 7fb801396740 10 client.17115 fetch_fsmap learned FSMap version 26 2024-04-23T10:36:57.682 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.676+0000 7fb801396740 1 -- 172.21.15.97:0/3734617522 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55dabe7940d0 con 0x55dabe77a620 2024-04-23T10:36:57.682 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.676+0000 7fb7d77fe640 1 -- 172.21.15.97:0/3734617522 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7ec0f1020 con 0x55dabe77a620 2024-04-23T10:36:57.682 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.676+0000 7fb7d77fe640 1 -- 172.21.15.97:0/3734617522 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb7e8020070 con 0x55dabe77a620 2024-04-23T10:36:57.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.676+0000 7fb801396740 10 client.17115 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:36:57.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.676+0000 7fb801396740 10 client.17115 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:36:57.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.676+0000 7fb801396740 20 client.17115 populate_metadata read hostname 'smithi097' 2024-04-23T10:36:57.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.676+0000 7fb801396740 1 --2- 172.21.15.97:0/3734617522 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55dabe7955e0 0x55dabe7979d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:57.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.676+0000 7fb801396740 4 client.17115 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:36:57.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.676+0000 7fb801396740 1 -- 172.21.15.97:0/3734617522 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dabe6ed390 con 0x55dabe7955e0 2024-04-23T10:36:57.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.677+0000 7fb7f77fe640 1 --2- 172.21.15.97:0/3734617522 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55dabe7955e0 0x55dabe7979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:57.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.677+0000 7fb7d77fe640 10 client.17115 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:36:57.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.677+0000 7fb7f77fe640 1 --2- 172.21.15.97:0/3734617522 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55dabe7955e0 0x55dabe7979d0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66db90a0967ba1a5 server_cookie=ac0645b2ef7d7c63 in_seq=0 out_seq=0 2024-04-23T10:36:57.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.678+0000 7fb7d77fe640 1 -- 172.21.15.97:0/3734617522 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dabe6ed390 con 0x55dabe7955e0 2024-04-23T10:36:57.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.678+0000 7fb7d77fe640 10 client.17115 handle_command_reply: tid=0 2024-04-23T10:36:57.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.678+0000 7fb801396740 1 client.17115 shutdown 2024-04-23T10:36:57.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.678+0000 7fb801396740 1 -- 172.21.15.97:0/3734617522 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dabe77a620 msgr2=0x55dabe785190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:57.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.678+0000 7fb801396740 1 --2- 172.21.15.97:0/3734617522 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dabe77a620 0x55dabe785190 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8011a30 tx=0x7fb7e8011ef0 comp rx=0 tx=0).stop 2024-04-23T10:36:57.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.678+0000 7fb801396740 1 -- 172.21.15.97:0/3734617522 shutdown_connections 2024-04-23T10:36:57.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.678+0000 7fb801396740 1 --2- 172.21.15.97:0/3734617522 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55dabe7955e0 0x55dabe7979d0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:57.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.678+0000 7fb801396740 1 --2- 172.21.15.97:0/3734617522 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dabe785800 0x55dabe782a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:57.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.678+0000 7fb7d77fe640 0 client.17115 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:36:57.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.678+0000 7fb801396740 1 --2- 172.21.15.97:0/3734617522 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dabe77af40 0x55dabe782480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:57.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.678+0000 7fb801396740 1 --2- 172.21.15.97:0/3734617522 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dabe77a620 0x55dabe785190 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:57.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.678+0000 7fb801396740 1 -- 172.21.15.97:0/3734617522 >> 172.21.15.97:0/3734617522 conn(0x55dabe776b30 msgr2=0x55dabe77c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:36:57.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.678+0000 7fb801396740 1 -- 172.21.15.97:0/3734617522 shutdown_connections 2024-04-23T10:36:57.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.678+0000 7fb801396740 1 -- 172.21.15.97:0/3734617522 wait complete. 2024-04-23T10:36:57.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.678+0000 7fb801396740 20 client.17115 trim_cache size 0 max 16384 2024-04-23T10:36:57.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.693+0000 7fb801396740 1 Processor -- start 2024-04-23T10:36:57.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.693+0000 7fb801396740 1 -- start start 2024-04-23T10:36:57.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.693+0000 7fb801396740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dabe77a620 0x55dabe790f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:57.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.693+0000 7fb801396740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dabe77af40 0x55dabe775b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:57.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.693+0000 7fb801396740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dabe785800 0x55dabe79d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:57.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.693+0000 7fb801396740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f81ad9d0 con 0x55dabe77a620 2024-04-23T10:36:57.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.693+0000 7fb801396740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f81af510 con 0x55dabe785800 2024-04-23T10:36:57.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.693+0000 7fb801396740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f81ae0a0 con 0x55dabe77af40 2024-04-23T10:36:57.696 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.693+0000 7fb7f7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dabe77a620 0x55dabe790f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:57.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.693+0000 7fb7f7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dabe77a620 0x55dabe790f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41012/0 (socket says 172.21.15.97:41012) 2024-04-23T10:36:57.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.693+0000 7fb7f7fff640 1 -- 172.21.15.97:0/553300124 learned_addr learned my addr 172.21.15.97:0/553300124 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:36:57.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.693+0000 7fb7f7fff640 1 -- 172.21.15.97:0/553300124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dabe77af40 msgr2=0x55dabe775b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:36:57.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.693+0000 7fb7f7fff640 1 --2- 172.21.15.97:0/553300124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dabe77af40 0x55dabe775b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:57.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.693+0000 7fb7f7fff640 1 -- 172.21.15.97:0/553300124 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dabe785800 msgr2=0x55dabe79d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:36:57.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.693+0000 7fb7f7fff640 1 --2- 172.21.15.97:0/553300124 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dabe785800 0x55dabe79d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:57.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.693+0000 7fb7f7fff640 1 -- 172.21.15.97:0/553300124 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dabe781430 con 0x55dabe77a620 2024-04-23T10:36:57.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.694+0000 7fb7f7fff640 1 --2- 172.21.15.97:0/553300124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dabe77a620 0x55dabe790f80 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8006010 tx=0x7fb7e800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fefab05f7cbbb1c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:36:57.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.694+0000 7fb7ddffb640 1 -- 172.21.15.97:0/553300124 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e802f070 con 0x55dabe77a620 2024-04-23T10:36:57.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.694+0000 7fb7ddffb640 1 -- 172.21.15.97:0/553300124 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7e800fd40 con 0x55dabe77a620 2024-04-23T10:36:57.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.694+0000 7fb7ddffb640 1 -- 172.21.15.97:0/553300124 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e8026070 con 0x55dabe77a620 2024-04-23T10:36:57.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.694+0000 7fb801396740 1 -- 172.21.15.97:0/553300124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dabe77a620 msgr2=0x55dabe790f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:57.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.694+0000 7fb801396740 1 --2- 172.21.15.97:0/553300124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dabe77a620 0x55dabe790f80 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8006010 tx=0x7fb7e800f790 comp rx=0 tx=0).stop 2024-04-23T10:36:57.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.694+0000 7fb801396740 1 -- 172.21.15.97:0/553300124 shutdown_connections 2024-04-23T10:36:57.697 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.694+0000 7fb801396740 1 --2- 172.21.15.97:0/553300124 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dabe785800 0x55dabe79d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.694+0000 7fb801396740 1 --2- 172.21.15.97:0/553300124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dabe77af40 0x55dabe775b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.694+0000 7fb801396740 1 --2- 172.21.15.97:0/553300124 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dabe77a620 0x55dabe790f80 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.694+0000 7fb801396740 1 -- 172.21.15.97:0/553300124 >> 172.21.15.97:0/553300124 conn(0x55dabe776b30 msgr2=0x55dabe775420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:36:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.694+0000 7fb801396740 1 -- 172.21.15.97:0/553300124 shutdown_connections 2024-04-23T10:36:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.694+0000 7fb801396740 1 -- 172.21.15.97:0/553300124 wait complete. 2024-04-23T10:36:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.695+0000 7fb801396740 1 -- start start 2024-04-23T10:36:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.695+0000 7fb801396740 1 Processor -- start 2024-04-23T10:36:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.695+0000 7fb801396740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dabe77a620 0x55dabe78d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.695+0000 7fb801396740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dabe77af40 0x55dabe78ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.695+0000 7fb801396740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dabe785800 0x55dabe78b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.695+0000 7fb801396740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f81aee40 con 0x55dabe77a620 2024-04-23T10:36:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.695+0000 7fb801396740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f81ae770 con 0x55dabe77af40 2024-04-23T10:36:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.695+0000 7fb801396740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f81ab9d0 con 0x55dabe785800 2024-04-23T10:36:57.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.695+0000 7fb801396740 10 client.0 _ll_register_callbacks cb 0x7fb7e8008d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:36:57.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.695+0000 7fb7f7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dabe77a620 0x55dabe78d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:57.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.695+0000 7fb7f7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dabe77a620 0x55dabe78d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41014/0 (socket says 172.21.15.97:41014) 2024-04-23T10:36:57.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.695+0000 7fb7f7fff640 1 -- 172.21.15.97:0/4148809365 learned_addr learned my addr 172.21.15.97:0/4148809365 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:36:57.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.695+0000 7fb7f7fff640 1 -- 172.21.15.97:0/4148809365 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dabe785800 msgr2=0x55dabe78b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:36:57.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.695+0000 7fb7f7fff640 1 --2- 172.21.15.97:0/4148809365 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dabe785800 0x55dabe78b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:57.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.695+0000 7fb7f7fff640 1 -- 172.21.15.97:0/4148809365 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dabe77af40 msgr2=0x55dabe78ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:36:57.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.695+0000 7fb7f7fff640 1 --2- 172.21.15.97:0/4148809365 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dabe77af40 0x55dabe78ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:57.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.695+0000 7fb7f7fff640 1 -- 172.21.15.97:0/4148809365 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dabe782d70 con 0x55dabe77a620 2024-04-23T10:36:57.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.696+0000 7fb7f7fff640 1 --2- 172.21.15.97:0/4148809365 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dabe77a620 0x55dabe78d8e0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8008500 tx=0x7fb7e8005510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7260401ed0a42387 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:36:57.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.696+0000 7fb7d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:36:57.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.696+0000 7fb7d77fe640 1 -- 172.21.15.97:0/4148809365 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e8038070 con 0x55dabe77a620 2024-04-23T10:36:57.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.696+0000 7fb7d77fe640 1 -- 172.21.15.97:0/4148809365 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8484+0+0 (secure 0 0 0) 0x7fb7e8003ab0 con 0x55dabe77a620 2024-04-23T10:36:57.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.696+0000 7fb801396740 1 -- 172.21.15.97:0/4148809365 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7e800fd40 con 0x55dabe77a620 2024-04-23T10:36:57.699 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.696+0000 7fb7d77fe640 1 -- 172.21.15.97:0/4148809365 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb7e8041050 con 0x55dabe77a620 2024-04-23T10:36:57.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.696+0000 7fb801396740 10 client.17121 fetch_fsmap learned FSMap version 26 2024-04-23T10:36:57.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.696+0000 7fb801396740 1 -- 172.21.15.97:0/4148809365 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55dabe7ffe30 con 0x55dabe77a620 2024-04-23T10:36:57.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.696+0000 7fb7d77fe640 1 -- 172.21.15.97:0/4148809365 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7e800fd40 con 0x55dabe77a620 2024-04-23T10:36:57.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.696+0000 7fb7d77fe640 1 -- 172.21.15.97:0/4148809365 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb7e803b020 con 0x55dabe77a620 2024-04-23T10:36:57.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.696+0000 7fb801396740 10 client.17121 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:36:57.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.696+0000 7fb801396740 10 client.17121 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:36:57.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.696+0000 7fb801396740 20 client.17121 populate_metadata read hostname 'smithi097' 2024-04-23T10:36:57.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.696+0000 7fb801396740 1 --2- 172.21.15.97:0/4148809365 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55dabe800530 0x55dabe802940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:36:57.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.696+0000 7fb801396740 4 client.17121 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:36:57.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.696+0000 7fb801396740 1 -- 172.21.15.97:0/4148809365 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dabe6e7200 con 0x55dabe800530 2024-04-23T10:36:57.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.696+0000 7fb7f77fe640 1 --2- 172.21.15.97:0/4148809365 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55dabe800530 0x55dabe802940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:36:57.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.697+0000 7fb7d77fe640 10 client.17121 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:36:57.700 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.697+0000 7fb7f77fe640 1 --2- 172.21.15.97:0/4148809365 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55dabe800530 0x55dabe802940 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21e3901fbe886daa server_cookie=fff499a52757f306 in_seq=0 out_seq=0 2024-04-23T10:36:57.708 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.709+0000 7fb7d77fe640 1 -- 172.21.15.97:0/4148809365 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dabe6e7200 con 0x55dabe800530 2024-04-23T10:36:57.708 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.709+0000 7fb7d77fe640 10 client.17121 handle_command_reply: tid=0 2024-04-23T10:36:57.708 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.709+0000 7fb801396740 1 client.17121 shutdown 2024-04-23T10:36:57.708 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.709+0000 7fb801396740 1 -- 172.21.15.97:0/4148809365 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dabe77a620 msgr2=0x55dabe78d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.709+0000 7fb801396740 1 --2- 172.21.15.97:0/4148809365 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dabe77a620 0x55dabe78d8e0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8008500 tx=0x7fb7e8005510 comp rx=0 tx=0).stop 2024-04-23T10:36:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.709+0000 7fb801396740 1 -- 172.21.15.97:0/4148809365 shutdown_connections 2024-04-23T10:36:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.709+0000 7fb801396740 1 --2- 172.21.15.97:0/4148809365 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55dabe800530 0x55dabe802940 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.709+0000 7fb801396740 1 --2- 172.21.15.97:0/4148809365 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dabe785800 0x55dabe78b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.709+0000 7fb7d77fe640 0 client.17121 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:36:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.709+0000 7fb801396740 1 --2- 172.21.15.97:0/4148809365 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dabe77af40 0x55dabe78ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.709+0000 7fb801396740 1 --2- 172.21.15.97:0/4148809365 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dabe77a620 0x55dabe78d8e0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.709+0000 7fb801396740 1 -- 172.21.15.97:0/4148809365 >> 172.21.15.97:0/4148809365 conn(0x55dabe776b30 msgr2=0x55dabe775cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:36:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.709+0000 7fb801396740 1 -- 172.21.15.97:0/4148809365 shutdown_connections 2024-04-23T10:36:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.709+0000 7fb801396740 1 -- 172.21.15.97:0/4148809365 wait complete. 2024-04-23T10:36:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.709+0000 7fb801396740 20 client.17121 trim_cache size 0 max 16384 2024-04-23T10:36:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:36:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:36:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:36:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:36:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.710+0000 7fb7fe67c640 1 -- 172.21.15.97:0/1836385113 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fb7e00a3dc0 msgr2=0x7fb7e00a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.710+0000 7fb7fe67c640 1 --2- 172.21.15.97:0/1836385113 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fb7e00a3dc0 0x7fb7e00a6280 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb7f806b860 tx=0x7fb7f0012960 comp rx=0 tx=0).stop 2024-04-23T10:36:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.710+0000 7fb7fe67c640 1 -- 172.21.15.97:0/1836385113 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7f8103b20 msgr2=0x7fb7f8198b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:36:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.710+0000 7fb7fe67c640 1 --2- 172.21.15.97:0/1836385113 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7f8103b20 0x7fb7f8198b50 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec006010 tx=0x7fb7ec0042f0 comp rx=0 tx=0).stop 2024-04-23T10:36:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.710+0000 7fb7fe67c640 1 -- 172.21.15.97:0/1836385113 shutdown_connections 2024-04-23T10:36:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.710+0000 7fb7fe67c640 1 --2- 172.21.15.97:0/1836385113 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fb7e00a3dc0 0x7fb7e00a6280 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.710+0000 7fb7fe67c640 1 --2- 172.21.15.97:0/1836385113 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb7f810c5d0 0x7fb7f819ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.710+0000 7fb7fe67c640 1 --2- 172.21.15.97:0/1836385113 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb7f8103b20 0x7fb7f8198b50 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.710+0000 7fb7fe67c640 1 --2- 172.21.15.97:0/1836385113 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb7f8103200 0x7fb7f8198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:36:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.710+0000 7fb7fe67c640 1 -- 172.21.15.97:0/1836385113 >> 172.21.15.97:0/1836385113 conn(0x7fb7f80771d0 msgr2=0x7fb7f80fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:36:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.710+0000 7fb7fe67c640 1 -- 172.21.15.97:0/1836385113 shutdown_connections 2024-04-23T10:36:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:36:57.710+0000 7fb7fe67c640 1 -- 172.21.15.97:0/1836385113 wait complete. 2024-04-23T10:36:59.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:36:58 smithi196 ceph-mon[32919]: pgmap v334: 97 pgs: 97 active+clean; 3.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.0 KiB/s wr, 15 op/s 2024-04-23T10:36:59.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:36:58 smithi097 ceph-mon[29197]: pgmap v334: 97 pgs: 97 active+clean; 3.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.0 KiB/s wr, 15 op/s 2024-04-23T10:36:59.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:36:58 smithi183 ceph-mon[32317]: pgmap v334: 97 pgs: 97 active+clean; 3.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.0 KiB/s wr, 15 op/s 2024-04-23T10:37:01.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:37:00 smithi183 ceph-mon[32317]: pgmap v335: 97 pgs: 97 active+clean; 3.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-23T10:37:01.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:37:00 smithi196 ceph-mon[32919]: pgmap v335: 97 pgs: 97 active+clean; 3.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-23T10:37:01.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:37:00 smithi097 ceph-mon[29197]: pgmap v335: 97 pgs: 97 active+clean; 3.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-23T10:37:03.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:37:02 smithi196 ceph-mon[32919]: pgmap v336: 97 pgs: 97 active+clean; 3.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-23T10:37:03.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:37:02 smithi097 ceph-mon[29197]: pgmap v336: 97 pgs: 97 active+clean; 3.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-23T10:37:03.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:37:02 smithi183 ceph-mon[32317]: pgmap v336: 97 pgs: 97 active+clean; 3.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-23T10:37:05.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:37:04 smithi196 ceph-mon[32919]: pgmap v337: 97 pgs: 97 active+clean; 3.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 51 KiB/s wr, 31 op/s 2024-04-23T10:37:05.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:37:04 smithi097 ceph-mon[29197]: pgmap v337: 97 pgs: 97 active+clean; 3.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 51 KiB/s wr, 31 op/s 2024-04-23T10:37:05.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:37:04 smithi183 ceph-mon[32317]: pgmap v337: 97 pgs: 97 active+clean; 3.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 51 KiB/s wr, 31 op/s 2024-04-23T10:37:05.912 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T10:37:06.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.027+0000 7f9ab4005640 1 -- 172.21.15.97:0/3581120374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9aac1018f0 msgr2=0x7f9aac105db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:37:06.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.027+0000 7f9ab4005640 1 --2- 172.21.15.97:0/3581120374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9aac1018f0 0x7f9aac105db0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f9aa8009900 tx=0x7f9aa802a510 comp rx=0 tx=0).stop 2024-04-23T10:37:06.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.027+0000 7f9ab4005640 1 -- 172.21.15.97:0/3581120374 shutdown_connections 2024-04-23T10:37:06.028 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.027+0000 7f9ab4005640 1 --2- 172.21.15.97:0/3581120374 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9aac1062f0 0x7f9aac106780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:06.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.027+0000 7f9ab4005640 1 --2- 172.21.15.97:0/3581120374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9aac1018f0 0x7f9aac105db0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:06.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.027+0000 7f9ab4005640 1 --2- 172.21.15.97:0/3581120374 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9aac100fd0 0x7f9aac1013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:06.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.027+0000 7f9ab4005640 1 -- 172.21.15.97:0/3581120374 >> 172.21.15.97:0/3581120374 conn(0x7f9aac077f50 msgr2=0x7f9aac078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:37:06.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.027+0000 7f9ab4005640 1 -- 172.21.15.97:0/3581120374 shutdown_connections 2024-04-23T10:37:06.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.027+0000 7f9ab4005640 1 -- 172.21.15.97:0/3581120374 wait complete. 2024-04-23T10:37:06.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.028+0000 7f9ab4005640 1 Processor -- start 2024-04-23T10:37:06.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.028+0000 7f9ab4005640 1 -- start start 2024-04-23T10:37:06.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.028+0000 7f9ab4005640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9aac100fd0 0x7f9aac19ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:06.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.028+0000 7f9ab4005640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9aac1018f0 0x7f9aac19f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:06.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.028+0000 7f9ab4005640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9aac1062f0 0x7f9aac1a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:06.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.028+0000 7f9ab4005640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9aac118790 con 0x7f9aac1062f0 2024-04-23T10:37:06.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.028+0000 7f9ab4005640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9aac118610 con 0x7f9aac1018f0 2024-04-23T10:37:06.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.028+0000 7f9ab4005640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9aac118910 con 0x7f9aac100fd0 2024-04-23T10:37:06.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.028+0000 7f9ab25a2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9aac1062f0 0x7f9aac1a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:06.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.028+0000 7f9ab25a2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9aac1062f0 0x7f9aac1a3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39796/0 (socket says 172.21.15.97:39796) 2024-04-23T10:37:06.029 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.028+0000 7f9ab25a2640 1 -- 172.21.15.97:0/4280034998 learned_addr learned my addr 172.21.15.97:0/4280034998 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:37:06.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.028+0000 7f9ab15a0640 1 --2- 172.21.15.97:0/4280034998 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9aac1018f0 0x7f9aac19f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:06.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.029+0000 7f9ab1da1640 1 --2- 172.21.15.97:0/4280034998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9aac100fd0 0x7f9aac19ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:06.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.029+0000 7f9ab25a2640 1 -- 172.21.15.97:0/4280034998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9aac100fd0 msgr2=0x7f9aac19ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:37:06.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.029+0000 7f9ab25a2640 1 --2- 172.21.15.97:0/4280034998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9aac100fd0 0x7f9aac19ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:06.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.029+0000 7f9ab25a2640 1 -- 172.21.15.97:0/4280034998 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9aac1018f0 msgr2=0x7f9aac19f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:37:06.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.029+0000 7f9ab25a2640 1 --2- 172.21.15.97:0/4280034998 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9aac1018f0 0x7f9aac19f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:06.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.029+0000 7f9ab25a2640 1 -- 172.21.15.97:0/4280034998 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9aac1a3d60 con 0x7f9aac1062f0 2024-04-23T10:37:06.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.029+0000 7f9ab1da1640 1 --2- 172.21.15.97:0/4280034998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9aac100fd0 0x7f9aac19ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:37:06.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.029+0000 7f9ab15a0640 1 --2- 172.21.15.97:0/4280034998 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9aac1018f0 0x7f9aac19f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:37:06.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.029+0000 7f9ab25a2640 1 --2- 172.21.15.97:0/4280034998 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9aac1062f0 0x7f9aac1a3630 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4009fd0 tx=0x7f9aa400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:37:06.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.029+0000 7f9aa2ffd640 1 -- 172.21.15.97:0/4280034998 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa4017070 con 0x7f9aac1062f0 2024-04-23T10:37:06.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.029+0000 7f9aa2ffd640 1 -- 172.21.15.97:0/4280034998 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9aa4003e20 con 0x7f9aac1062f0 2024-04-23T10:37:06.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.029+0000 7f9ab4005640 1 -- 172.21.15.97:0/4280034998 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9aac1a4050 con 0x7f9aac1062f0 2024-04-23T10:37:06.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.029+0000 7f9aa2ffd640 1 -- 172.21.15.97:0/4280034998 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa4010460 con 0x7f9aac1062f0 2024-04-23T10:37:06.030 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.029+0000 7f9ab4005640 1 -- 172.21.15.97:0/4280034998 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9aac1ab8e0 con 0x7f9aac1062f0 2024-04-23T10:37:06.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.030+0000 7f9ab4005640 1 -- 172.21.15.97:0/4280034998 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9aac0698e0 con 0x7f9aac1062f0 2024-04-23T10:37:06.031 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.031+0000 7f9aa2ffd640 1 -- 172.21.15.97:0/4280034998 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9aa4003970 con 0x7f9aac1062f0 2024-04-23T10:37:06.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.032+0000 7f9aa2ffd640 1 --2- 172.21.15.97:0/4280034998 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f9a980a3da0 0x7f9a980a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:06.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.032+0000 7f9ab1da1640 1 --2- 172.21.15.97:0/4280034998 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f9a980a3da0 0x7f9a980a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:06.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.032+0000 7f9aa2ffd640 1 -- 172.21.15.97:0/4280034998 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8484+0+0 (secure 0 0 0) 0x7f9aa40d08d0 con 0x7f9aac1062f0 2024-04-23T10:37:06.032 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.032+0000 7f9ab1da1640 1 --2- 172.21.15.97:0/4280034998 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f9a980a3da0 0x7f9a980a6260 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c009b50 tx=0x7f9a9c005f90 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:37:06.034 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.036+0000 7f9aa2ffd640 1 -- 172.21.15.97:0/4280034998 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9aa409d010 con 0x7f9aac1062f0 2024-04-23T10:37:06.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.253+0000 7f9ab4005640 1 -- 172.21.15.97:0/4280034998 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9aac1abb10 con 0x7f9aac1062f0 2024-04-23T10:37:06.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.254+0000 7f9aa2ffd640 1 -- 172.21.15.97:0/4280034998 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11931 (secure 0 0 0) 0x7f9aa409c910 con 0x7f9aac1062f0 2024-04-23T10:37:06.253 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:37:06.253 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3946878762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3946878762},{"type":"v1","addr":"172.21.15.97:6835","nonce":3946878762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/209800861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":209800861},{"type":"v1","addr":"172.21.15.97:6837","nonce":209800861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T10:24:06.910134+0000","modified":"2024-04-23T10:26:09.820985+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6839/418013575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":418013575},{"type":"v1","addr":"172.21.15.97:6839","nonce":418013575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6841/747242680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":747242680},{"type":"v1","addr":"172.21.15.97:6841","nonce":747242680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6833/4292847121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4292847121},{"type":"v1","addr":"172.21.15.183:6833","nonce":4292847121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6833/3885816565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3885816565},{"type":"v1","addr":"172.21.15.196:6833","nonce":3885816565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1897093105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1897093105},{"type":"v1","addr":"172.21.15.183:6835","nonce":1897093105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1171477563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1171477563},{"type":"v1","addr":"172.21.15.196:6835","nonce":1171477563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1898341911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1898341911},{"type":"v1","addr":"172.21.15.183:6837","nonce":1898341911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.196:6837/3094946502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3094946502},{"type":"v1","addr":"172.21.15.196:6837","nonce":3094946502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.196:6839/83843556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":83843556},{"type":"v1","addr":"172.21.15.196:6839","nonce":83843556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6839/1763841130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1763841130},{"type":"v1","addr":"172.21.15.183:6839","nonce":1763841130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24487,24403,24398,24469,24488]},"id":1}]} 2024-04-23T10:37:06.254 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 26 2024-04-23T10:37:06.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.256+0000 7f9ab4005640 1 -- 172.21.15.97:0/4280034998 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f9a980a3da0 msgr2=0x7f9a980a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:37:06.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.256+0000 7f9ab4005640 1 --2- 172.21.15.97:0/4280034998 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f9a980a3da0 0x7f9a980a6260 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c009b50 tx=0x7f9a9c005f90 comp rx=0 tx=0).stop 2024-04-23T10:37:06.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.256+0000 7f9ab4005640 1 -- 172.21.15.97:0/4280034998 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9aac1062f0 msgr2=0x7f9aac1a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:37:06.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.256+0000 7f9ab4005640 1 --2- 172.21.15.97:0/4280034998 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9aac1062f0 0x7f9aac1a3630 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4009fd0 tx=0x7f9aa400b260 comp rx=0 tx=0).stop 2024-04-23T10:37:06.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.256+0000 7f9ab4005640 1 -- 172.21.15.97:0/4280034998 shutdown_connections 2024-04-23T10:37:06.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.256+0000 7f9ab4005640 1 --2- 172.21.15.97:0/4280034998 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f9a980a3da0 0x7f9a980a6260 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:06.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.256+0000 7f9ab4005640 1 --2- 172.21.15.97:0/4280034998 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9aac1062f0 0x7f9aac1a3630 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:06.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.256+0000 7f9ab4005640 1 --2- 172.21.15.97:0/4280034998 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9aac1018f0 0x7f9aac19f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:06.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.256+0000 7f9ab4005640 1 --2- 172.21.15.97:0/4280034998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9aac100fd0 0x7f9aac19ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:06.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.256+0000 7f9ab4005640 1 -- 172.21.15.97:0/4280034998 >> 172.21.15.97:0/4280034998 conn(0x7f9aac077f50 msgr2=0x7f9aac103d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:37:06.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.256+0000 7f9ab4005640 1 -- 172.21.15.97:0/4280034998 shutdown_connections 2024-04-23T10:37:06.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.256+0000 7f9ab4005640 1 -- 172.21.15.97:0/4280034998 wait complete. 2024-04-23T10:37:06.266 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --set-id f5a8cd2e --release --await' 2024-04-23T10:37:06.266 DEBUG:teuthology.orchestra.run.smithi097:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24488 quiesce db --set-id f5a8cd2e --release --await 2024-04-23T10:37:06.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.372+0000 7f0b39787640 1 -- 172.21.15.97:0/709209948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b3406a840 msgr2=0x7f0b3406ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:37:06.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.372+0000 7f0b39787640 1 --2- 172.21.15.97:0/709209948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b3406a840 0x7f0b3406ac20 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c009900 tx=0x7f0b2c02a410 comp rx=0 tx=0).stop 2024-04-23T10:37:06.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.372+0000 7f0b39787640 1 -- 172.21.15.97:0/709209948 shutdown_connections 2024-04-23T10:37:06.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.372+0000 7f0b39787640 1 --2- 172.21.15.97:0/709209948 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b34107270 0x7f0b3410ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:06.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.372+0000 7f0b39787640 1 --2- 172.21.15.97:0/709209948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b3406b1f0 0x7f0b34106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:06.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.372+0000 7f0b39787640 1 --2- 172.21.15.97:0/709209948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b3406a840 0x7f0b3406ac20 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:06.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.372+0000 7f0b39787640 1 -- 172.21.15.97:0/709209948 >> 172.21.15.97:0/709209948 conn(0x7f0b34077eb0 msgr2=0x7f0b340782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:37:06.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.372+0000 7f0b39787640 1 -- 172.21.15.97:0/709209948 shutdown_connections 2024-04-23T10:37:06.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.372+0000 7f0b39787640 1 -- 172.21.15.97:0/709209948 wait complete. 2024-04-23T10:37:06.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.373+0000 7f0b39787640 1 Processor -- start 2024-04-23T10:37:06.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.373+0000 7f0b39787640 1 -- start start 2024-04-23T10:37:06.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.373+0000 7f0b39787640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b3406a840 0x7f0b34198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:06.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.373+0000 7f0b39787640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b34107270 0x7f0b34198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:06.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.373+0000 7f0b39787640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b3419cff0 0x7f0b3419d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:06.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.373+0000 7f0b39787640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0b341121b0 con 0x7f0b3406a840 2024-04-23T10:37:06.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.373+0000 7f0b39787640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0b34112030 con 0x7f0b3419cff0 2024-04-23T10:37:06.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.373+0000 7f0b39787640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0b34112330 con 0x7f0b34107270 2024-04-23T10:37:06.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.373+0000 7f0b32ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b3406a840 0x7f0b34198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:06.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.373+0000 7f0b32ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b3406a840 0x7f0b34198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39804/0 (socket says 172.21.15.97:39804) 2024-04-23T10:37:06.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.373+0000 7f0b32ffd640 1 -- 172.21.15.97:0/1468853591 learned_addr learned my addr 172.21.15.97:0/1468853591 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:37:06.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.374+0000 7f0b32ffd640 1 -- 172.21.15.97:0/1468853591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b34107270 msgr2=0x7f0b34198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:37:06.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.374+0000 7f0b327fc640 1 --2- 172.21.15.97:0/1468853591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b34107270 0x7f0b34198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:06.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.374+0000 7f0b32ffd640 1 --2- 172.21.15.97:0/1468853591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b34107270 0x7f0b34198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:06.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.374+0000 7f0b32ffd640 1 -- 172.21.15.97:0/1468853591 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b3419cff0 msgr2=0x7f0b3419d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:37:06.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.374+0000 7f0b337fe640 1 --2- 172.21.15.97:0/1468853591 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b3419cff0 0x7f0b3419d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:06.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.374+0000 7f0b32ffd640 1 --2- 172.21.15.97:0/1468853591 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b3419cff0 0x7f0b3419d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:06.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.374+0000 7f0b32ffd640 1 -- 172.21.15.97:0/1468853591 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b3419dae0 con 0x7f0b3406a840 2024-04-23T10:37:06.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.374+0000 7f0b327fc640 1 --2- 172.21.15.97:0/1468853591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b34107270 0x7f0b34198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:37:06.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.374+0000 7f0b32ffd640 1 --2- 172.21.15.97:0/1468853591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b3406a840 0x7f0b34198730 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c005940 tx=0x7f0b2c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:37:06.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.374+0000 7f0b1bfff640 1 -- 172.21.15.97:0/1468853591 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b2c03f030 con 0x7f0b3406a840 2024-04-23T10:37:06.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.374+0000 7f0b39787640 1 -- 172.21.15.97:0/1468853591 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b3419dd70 con 0x7f0b3406a840 2024-04-23T10:37:06.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.374+0000 7f0b1bfff640 1 -- 172.21.15.97:0/1468853591 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b2c003cb0 con 0x7f0b3406a840 2024-04-23T10:37:06.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.374+0000 7f0b1bfff640 1 -- 172.21.15.97:0/1468853591 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b2c035070 con 0x7f0b3406a840 2024-04-23T10:37:06.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.374+0000 7f0b39787640 1 -- 172.21.15.97:0/1468853591 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b341a5590 con 0x7f0b3406a840 2024-04-23T10:37:06.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.376+0000 7f0b1bfff640 1 -- 172.21.15.97:0/1468853591 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0b2c042030 con 0x7f0b3406a840 2024-04-23T10:37:06.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.376+0000 7f0b3c57a740 1 Processor -- start 2024-04-23T10:37:06.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.376+0000 7f0b3c57a740 1 -- start start 2024-04-23T10:37:06.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.377+0000 7f0b3c57a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f3bc561c0 0x557f3bc565a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:06.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.377+0000 7f0b3c57a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f3bc56b70 0x557f3bc65350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:06.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.377+0000 7f0b3c57a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f3bc65890 0x557f3bc67c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:06.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.377+0000 7f0b1bfff640 1 --2- 172.21.15.97:0/1468853591 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f0b1c0a3d20 0x7f0b1c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:06.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.377+0000 7f0b3c57a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x557f3bbbb6d0 con 0x557f3bc65890 2024-04-23T10:37:06.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.377+0000 7f0b3c57a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557f3bb581a0 con 0x557f3bc56b70 2024-04-23T10:37:06.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.377+0000 7f0b3c57a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557f3bbb7690 con 0x557f3bc561c0 2024-04-23T10:37:06.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.377+0000 7f0b337fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f3bc65890 0x557f3bc67c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:06.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.377+0000 7f0b327fc640 1 --2- 172.21.15.97:0/1468853591 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f0b1c0a3d20 0x7f0b1c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:06.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.377+0000 7f0b337fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f3bc65890 0x557f3bc67c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39814/0 (socket says 172.21.15.97:39814) 2024-04-23T10:37:06.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.377+0000 7f0b337fe640 1 -- 172.21.15.97:0/2473638812 learned_addr learned my addr 172.21.15.97:0/2473638812 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:37:06.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.377+0000 7f0b337fe640 1 --2- 172.21.15.97:0/2473638812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f3bc561c0 0x557f3bc565a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:06.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.377+0000 7f0b1bfff640 1 -- 172.21.15.97:0/1468853591 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8484+0+0 (secure 0 0 0) 0x7f0b2c0f1140 con 0x7f0b3406a840 2024-04-23T10:37:06.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.377+0000 7f0b32ffd640 1 --2- 172.21.15.97:0/2473638812 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f3bc56b70 0x557f3bc65350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:06.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.377+0000 7f0b337fe640 1 -- 172.21.15.97:0/2473638812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f3bc561c0 msgr2=0x557f3bc565a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:37:06.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.377+0000 7f0b337fe640 1 --2- 172.21.15.97:0/2473638812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f3bc561c0 0x557f3bc565a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:06.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.377+0000 7f0b337fe640 1 -- 172.21.15.97:0/2473638812 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f3bc56b70 msgr2=0x557f3bc65350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:37:06.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.377+0000 7f0b337fe640 1 --2- 172.21.15.97:0/2473638812 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f3bc56b70 0x557f3bc65350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:06.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.377+0000 7f0b337fe640 1 -- 172.21.15.97:0/2473638812 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f3bb625b0 con 0x557f3bc65890 2024-04-23T10:37:06.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.377+0000 7f0b32ffd640 1 --2- 172.21.15.97:0/2473638812 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f3bc56b70 0x557f3bc65350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:37:06.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.377+0000 7f0b327fc640 1 --2- 172.21.15.97:0/1468853591 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f0b1c0a3d20 0x7f0b1c0a61e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0b341009f0 tx=0x7f0b28005d30 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:37:06.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.377+0000 7f0b337fe640 1 --2- 172.21.15.97:0/2473638812 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f3bc65890 0x557f3bc67c80 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f0b24013a30 tx=0x7f0b24009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86c63f7067c73dcf server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:37:06.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.377+0000 7f0b197fa640 1 -- 172.21.15.97:0/2473638812 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b24004d20 con 0x557f3bc65890 2024-04-23T10:37:06.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.377+0000 7f0b197fa640 1 -- 172.21.15.97:0/2473638812 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b2400eba0 con 0x557f3bc65890 2024-04-23T10:37:06.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.377+0000 7f0b197fa640 1 -- 172.21.15.97:0/2473638812 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b24005680 con 0x557f3bc65890 2024-04-23T10:37:06.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.378+0000 7f0b3c57a740 1 -- 172.21.15.97:0/2473638812 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f3bc65890 msgr2=0x557f3bc67c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:37:06.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.378+0000 7f0b3c57a740 1 --2- 172.21.15.97:0/2473638812 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f3bc65890 0x557f3bc67c80 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f0b24013a30 tx=0x7f0b24009640 comp rx=0 tx=0).stop 2024-04-23T10:37:06.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.378+0000 7f0b3c57a740 1 -- 172.21.15.97:0/2473638812 shutdown_connections 2024-04-23T10:37:06.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.378+0000 7f0b3c57a740 1 --2- 172.21.15.97:0/2473638812 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f3bc65890 0x557f3bc67c80 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:06.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.378+0000 7f0b3c57a740 1 --2- 172.21.15.97:0/2473638812 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f3bc56b70 0x557f3bc65350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:06.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.378+0000 7f0b3c57a740 1 --2- 172.21.15.97:0/2473638812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f3bc561c0 0x557f3bc565a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:06.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.378+0000 7f0b3c57a740 1 -- 172.21.15.97:0/2473638812 >> 172.21.15.97:0/2473638812 conn(0x557f3bc527b0 msgr2=0x557f3bc52b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:37:06.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.378+0000 7f0b3c57a740 1 -- 172.21.15.97:0/2473638812 shutdown_connections 2024-04-23T10:37:06.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.378+0000 7f0b3c57a740 1 -- 172.21.15.97:0/2473638812 wait complete. 2024-04-23T10:37:06.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.378+0000 7f0b3c57a740 1 -- start start 2024-04-23T10:37:06.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.378+0000 7f0b3c57a740 1 Processor -- start 2024-04-23T10:37:06.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.379+0000 7f0b3c57a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f3bc561c0 0x557f3bc652d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:06.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.379+0000 7f0b3c57a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f3bc56b70 0x557f3bc625c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:06.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.379+0000 7f0b3c57a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f3bc65890 0x557f3bc62d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:06.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.379+0000 7f0b3c57a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x557f3bc686f0 con 0x557f3bc56b70 2024-04-23T10:37:06.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.379+0000 7f0b3c57a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557f3bb55b20 con 0x557f3bc561c0 2024-04-23T10:37:06.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.379+0000 7f0b3c57a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557f3bc68870 con 0x557f3bc65890 2024-04-23T10:37:06.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.379+0000 7f0b3c57a740 10 client.0 _ll_register_callbacks cb 0x557f3bb55530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:37:06.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.379+0000 7f0b327fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f3bc56b70 0x557f3bc625c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:06.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.379+0000 7f0b327fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f3bc56b70 0x557f3bc625c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39824/0 (socket says 172.21.15.97:39824) 2024-04-23T10:37:06.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.379+0000 7f0b327fc640 1 -- 172.21.15.97:0/2724261812 learned_addr learned my addr 172.21.15.97:0/2724261812 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:37:06.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.379+0000 7f0b32ffd640 1 --2- 172.21.15.97:0/2724261812 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f3bc561c0 0x557f3bc652d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:06.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.379+0000 7f0b337fe640 1 --2- 172.21.15.97:0/2724261812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f3bc65890 0x557f3bc62d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:06.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.379+0000 7f0b327fc640 1 -- 172.21.15.97:0/2724261812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f3bc65890 msgr2=0x557f3bc62d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:37:06.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.379+0000 7f0b327fc640 1 --2- 172.21.15.97:0/2724261812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f3bc65890 0x557f3bc62d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:06.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.379+0000 7f0b327fc640 1 -- 172.21.15.97:0/2724261812 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f3bc561c0 msgr2=0x557f3bc652d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:37:06.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.379+0000 7f0b327fc640 1 --2- 172.21.15.97:0/2724261812 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f3bc561c0 0x557f3bc652d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:06.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.379+0000 7f0b327fc640 1 -- 172.21.15.97:0/2724261812 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f3b8cd920 con 0x557f3bc56b70 2024-04-23T10:37:06.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.379+0000 7f0b337fe640 1 --2- 172.21.15.97:0/2724261812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f3bc65890 0x557f3bc62d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:37:06.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.379+0000 7f0b327fc640 1 --2- 172.21.15.97:0/2724261812 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f3bc56b70 0x557f3bc625c0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f0b2800f9a0 tx=0x7f0b280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=263a897c0146dc5b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:37:06.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.379+0000 7f0b16ff5640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:37:06.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.379+0000 7f0b16ff5640 1 -- 172.21.15.97:0/2724261812 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b28020070 con 0x557f3bc56b70 2024-04-23T10:37:06.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.379+0000 7f0b16ff5640 1 -- 172.21.15.97:0/2724261812 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8484+0+0 (secure 0 0 0) 0x7f0b2800ecc0 con 0x557f3bc56b70 2024-04-23T10:37:06.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.379+0000 7f0b3c57a740 1 -- 172.21.15.97:0/2724261812 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b2400eba0 con 0x557f3bc56b70 2024-04-23T10:37:06.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.380+0000 7f0b16ff5640 1 -- 172.21.15.97:0/2724261812 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0b2802c050 con 0x557f3bc56b70 2024-04-23T10:37:06.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.380+0000 7f0b3c57a740 10 client.17139 fetch_fsmap learned FSMap version 26 2024-04-23T10:37:06.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.380+0000 7f0b3c57a740 1 -- 172.21.15.97:0/2724261812 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557f3b999d30 con 0x557f3bc56b70 2024-04-23T10:37:06.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.380+0000 7f0b16ff5640 1 -- 172.21.15.97:0/2724261812 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b2400eba0 con 0x557f3bc56b70 2024-04-23T10:37:06.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.380+0000 7f0b16ff5640 1 -- 172.21.15.97:0/2724261812 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f0b280243b0 con 0x557f3bc56b70 2024-04-23T10:37:06.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.380+0000 7f0b3c57a740 10 client.17139 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:37:06.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.380+0000 7f0b3c57a740 10 client.17139 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:37:06.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.380+0000 7f0b3c57a740 20 client.17139 populate_metadata read hostname 'smithi097' 2024-04-23T10:37:06.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.380+0000 7f0b3c57a740 1 --2- 172.21.15.97:0/2724261812 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x557f3bc75ad0 0x557f3bc77f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:06.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.380+0000 7f0b3c57a740 4 client.17139 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:37:06.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.380+0000 7f0b3c57a740 1 -- 172.21.15.97:0/2724261812 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f3bb79500 con 0x557f3bc75ad0 2024-04-23T10:37:06.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.380+0000 7f0b32ffd640 1 --2- 172.21.15.97:0/2724261812 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x557f3bc75ad0 0x557f3bc77f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:06.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.381+0000 7f0b16ff5640 10 client.17139 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:37:06.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.381+0000 7f0b32ffd640 1 --2- 172.21.15.97:0/2724261812 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x557f3bc75ad0 0x557f3bc77f30 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae781527e18c55b7 server_cookie=c838e2847c5ea5ab in_seq=0 out_seq=0 2024-04-23T10:37:06.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.381+0000 7f0b16ff5640 1 -- 172.21.15.97:0/2724261812 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557f3bb79500 con 0x557f3bc75ad0 2024-04-23T10:37:06.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.381+0000 7f0b16ff5640 10 client.17139 handle_command_reply: tid=0 2024-04-23T10:37:06.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.381+0000 7f0b3c57a740 1 client.17139 shutdown 2024-04-23T10:37:06.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.382+0000 7f0b3c57a740 1 -- 172.21.15.97:0/2724261812 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f3bc56b70 msgr2=0x557f3bc625c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:37:06.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.382+0000 7f0b3c57a740 1 --2- 172.21.15.97:0/2724261812 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f3bc56b70 0x557f3bc625c0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f0b2800f9a0 tx=0x7f0b280046c0 comp rx=0 tx=0).stop 2024-04-23T10:37:06.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.382+0000 7f0b3c57a740 1 -- 172.21.15.97:0/2724261812 shutdown_connections 2024-04-23T10:37:06.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.382+0000 7f0b3c57a740 1 --2- 172.21.15.97:0/2724261812 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x557f3bc75ad0 0x557f3bc77f30 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:06.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.382+0000 7f0b3c57a740 1 --2- 172.21.15.97:0/2724261812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f3bc65890 0x557f3bc62d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:06.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.382+0000 7f0b3c57a740 1 --2- 172.21.15.97:0/2724261812 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f3bc56b70 0x557f3bc625c0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:06.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.382+0000 7f0b16ff5640 0 client.17139 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:37:06.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.382+0000 7f0b3c57a740 1 --2- 172.21.15.97:0/2724261812 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f3bc561c0 0x557f3bc652d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:06.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.382+0000 7f0b3c57a740 1 -- 172.21.15.97:0/2724261812 >> 172.21.15.97:0/2724261812 conn(0x557f3bc527b0 msgr2=0x557f3bc58600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:37:06.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.382+0000 7f0b3c57a740 1 -- 172.21.15.97:0/2724261812 shutdown_connections 2024-04-23T10:37:06.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.382+0000 7f0b3c57a740 1 -- 172.21.15.97:0/2724261812 wait complete. 2024-04-23T10:37:06.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.382+0000 7f0b3c57a740 20 client.17139 trim_cache size 0 max 16384 2024-04-23T10:37:06.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.399+0000 7f0b3c57a740 1 Processor -- start 2024-04-23T10:37:06.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.399+0000 7f0b3c57a740 1 -- start start 2024-04-23T10:37:06.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.399+0000 7f0b3c57a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f3bc561c0 0x557f3bc71620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:06.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.399+0000 7f0b3c57a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f3bc56b70 0x557f3bc64e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:06.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.399+0000 7f0b3c57a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f3bc65890 0x557f3bc65340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:06.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.399+0000 7f0b3c57a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0b341ab950 con 0x557f3bc561c0 2024-04-23T10:37:06.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.399+0000 7f0b3c57a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0b341ad490 con 0x557f3bc65890 2024-04-23T10:37:06.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.399+0000 7f0b3c57a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0b341ac020 con 0x557f3bc56b70 2024-04-23T10:37:06.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.399+0000 7f0b32ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f3bc561c0 0x557f3bc71620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:06.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.399+0000 7f0b32ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f3bc561c0 0x557f3bc71620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39828/0 (socket says 172.21.15.97:39828) 2024-04-23T10:37:06.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.399+0000 7f0b32ffd640 1 -- 172.21.15.97:0/4054935522 learned_addr learned my addr 172.21.15.97:0/4054935522 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:37:06.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.399+0000 7f0b327fc640 1 --2- 172.21.15.97:0/4054935522 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f3bc56b70 0x557f3bc64e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:06.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.399+0000 7f0b32ffd640 1 -- 172.21.15.97:0/4054935522 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f3bc56b70 msgr2=0x557f3bc64e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:37:06.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.399+0000 7f0b32ffd640 1 --2- 172.21.15.97:0/4054935522 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f3bc56b70 0x557f3bc64e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:06.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.399+0000 7f0b32ffd640 1 -- 172.21.15.97:0/4054935522 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f3bc65890 msgr2=0x557f3bc65340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:37:06.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.399+0000 7f0b32ffd640 1 --2- 172.21.15.97:0/4054935522 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f3bc65890 0x557f3bc65340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:06.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.399+0000 7f0b32ffd640 1 -- 172.21.15.97:0/4054935522 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f3bc63070 con 0x557f3bc561c0 2024-04-23T10:37:06.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.399+0000 7f0b32ffd640 1 --2- 172.21.15.97:0/4054935522 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f3bc561c0 0x557f3bc71620 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c0b9500 tx=0x7f0b2c0f1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b06c568e6fd8ea server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:37:06.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.399+0000 7f0b18ff9640 1 -- 172.21.15.97:0/4054935522 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b2c049040 con 0x557f3bc561c0 2024-04-23T10:37:06.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.399+0000 7f0b18ff9640 1 -- 172.21.15.97:0/4054935522 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b2c0bd0a0 con 0x557f3bc561c0 2024-04-23T10:37:06.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.400+0000 7f0b18ff9640 1 -- 172.21.15.97:0/4054935522 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b2c0b9710 con 0x557f3bc561c0 2024-04-23T10:37:06.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.400+0000 7f0b3c57a740 1 -- 172.21.15.97:0/4054935522 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f3bc561c0 msgr2=0x557f3bc71620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:37:06.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.400+0000 7f0b3c57a740 1 --2- 172.21.15.97:0/4054935522 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f3bc561c0 0x557f3bc71620 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c0b9500 tx=0x7f0b2c0f1040 comp rx=0 tx=0).stop 2024-04-23T10:37:06.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.400+0000 7f0b3c57a740 1 -- 172.21.15.97:0/4054935522 shutdown_connections 2024-04-23T10:37:06.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.400+0000 7f0b3c57a740 1 --2- 172.21.15.97:0/4054935522 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f3bc65890 0x557f3bc65340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:06.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.400+0000 7f0b3c57a740 1 --2- 172.21.15.97:0/4054935522 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f3bc56b70 0x557f3bc64e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:06.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.400+0000 7f0b3c57a740 1 --2- 172.21.15.97:0/4054935522 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f3bc561c0 0x557f3bc71620 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:06.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.400+0000 7f0b3c57a740 1 -- 172.21.15.97:0/4054935522 >> 172.21.15.97:0/4054935522 conn(0x557f3bc527b0 msgr2=0x557f3bc5d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:37:06.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.400+0000 7f0b3c57a740 1 -- 172.21.15.97:0/4054935522 shutdown_connections 2024-04-23T10:37:06.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.400+0000 7f0b3c57a740 1 -- 172.21.15.97:0/4054935522 wait complete. 2024-04-23T10:37:06.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.400+0000 7f0b3c57a740 1 -- start start 2024-04-23T10:37:06.404 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.400+0000 7f0b3c57a740 1 Processor -- start 2024-04-23T10:37:06.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.401+0000 7f0b3c57a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f3bc561c0 0x557f3bc602a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:06.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.401+0000 7f0b3c57a740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f3bc56b70 0x557f3bc6db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:06.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.401+0000 7f0b3c57a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f3bc65890 0x557f3bc66960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:06.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.401+0000 7f0b3c57a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0b341acdc0 con 0x557f3bc65890 2024-04-23T10:37:06.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.401+0000 7f0b3c57a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0b341ac6f0 con 0x557f3bc561c0 2024-04-23T10:37:06.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.401+0000 7f0b3c57a740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0b341a9950 con 0x557f3bc56b70 2024-04-23T10:37:06.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.401+0000 7f0b3c57a740 10 client.0 _ll_register_callbacks cb 0x7f0b2800f650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:37:06.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.401+0000 7f0b337fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f3bc65890 0x557f3bc66960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:06.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.401+0000 7f0b337fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f3bc65890 0x557f3bc66960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39842/0 (socket says 172.21.15.97:39842) 2024-04-23T10:37:06.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.401+0000 7f0b337fe640 1 -- 172.21.15.97:0/453208527 learned_addr learned my addr 172.21.15.97:0/453208527 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:37:06.405 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.401+0000 7f0b327fc640 1 --2- 172.21.15.97:0/453208527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f3bc56b70 0x557f3bc6db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:06.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.401+0000 7f0b32ffd640 1 --2- 172.21.15.97:0/453208527 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f3bc561c0 0x557f3bc602a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:06.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.401+0000 7f0b337fe640 1 -- 172.21.15.97:0/453208527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f3bc56b70 msgr2=0x557f3bc6db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:37:06.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.401+0000 7f0b337fe640 1 --2- 172.21.15.97:0/453208527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f3bc56b70 0x557f3bc6db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:06.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.401+0000 7f0b337fe640 1 -- 172.21.15.97:0/453208527 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f3bc561c0 msgr2=0x557f3bc602a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:37:06.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.401+0000 7f0b337fe640 1 --2- 172.21.15.97:0/453208527 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f3bc561c0 0x557f3bc602a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:06.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.401+0000 7f0b337fe640 1 -- 172.21.15.97:0/453208527 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f3bc66ea0 con 0x557f3bc65890 2024-04-23T10:37:06.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.401+0000 7f0b32ffd640 1 --2- 172.21.15.97:0/453208527 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f3bc561c0 0x557f3bc602a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:37:06.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.401+0000 7f0b337fe640 1 --2- 172.21.15.97:0/453208527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f3bc65890 0x557f3bc66960 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f0b240092d0 tx=0x7f0b24009300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a724f023b7924d66 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:37:06.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.401+0000 7f0b16ff5640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:37:06.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.401+0000 7f0b16ff5640 1 -- 172.21.15.97:0/453208527 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b240049a0 con 0x557f3bc65890 2024-04-23T10:37:06.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.401+0000 7f0b16ff5640 1 -- 172.21.15.97:0/453208527 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8484+0+0 (secure 0 0 0) 0x7f0b24002030 con 0x557f3bc65890 2024-04-23T10:37:06.406 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.401+0000 7f0b3c57a740 1 -- 172.21.15.97:0/453208527 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b2c0bd0a0 con 0x557f3bc65890 2024-04-23T10:37:06.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.401+0000 7f0b16ff5640 1 -- 172.21.15.97:0/453208527 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0b24004b40 con 0x557f3bc65890 2024-04-23T10:37:06.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.401+0000 7f0b3c57a740 10 client.17145 fetch_fsmap learned FSMap version 26 2024-04-23T10:37:06.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.401+0000 7f0b3c57a740 1 -- 172.21.15.97:0/453208527 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557f3bcdc700 con 0x557f3bc65890 2024-04-23T10:37:06.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.402+0000 7f0b16ff5640 1 -- 172.21.15.97:0/453208527 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b2c0bd0a0 con 0x557f3bc65890 2024-04-23T10:37:06.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.402+0000 7f0b16ff5640 1 -- 172.21.15.97:0/453208527 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f0b2400f040 con 0x557f3bc65890 2024-04-23T10:37:06.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.402+0000 7f0b3c57a740 10 client.17145 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:37:06.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.402+0000 7f0b3c57a740 10 client.17145 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:37:06.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.402+0000 7f0b3c57a740 20 client.17145 populate_metadata read hostname 'smithi097' 2024-04-23T10:37:06.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.402+0000 7f0b3c57a740 1 --2- 172.21.15.97:0/453208527 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x557f3bcdcb70 0x557f3bcdef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:06.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.402+0000 7f0b3c57a740 4 client.17145 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "f5a8cd2e", "release": true, "await": true}] 2024-04-23T10:37:06.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.402+0000 7f0b3c57a740 1 -- 172.21.15.97:0/453208527 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "quiesce db", "set_id": "f5a8cd2e", "release": true, "await": true}) -- 0x557f3bbc90f0 con 0x557f3bcdcb70 2024-04-23T10:37:06.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.402+0000 7f0b32ffd640 1 --2- 172.21.15.97:0/453208527 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x557f3bcdcb70 0x557f3bcdef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:06.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.402+0000 7f0b16ff5640 10 client.17145 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:37:06.407 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.402+0000 7f0b32ffd640 1 --2- 172.21.15.97:0/453208527 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x557f3bcdcb70 0x557f3bcdef90 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=248f76db7b4b56c0 server_cookie=c800e22343fca4c2 in_seq=0 out_seq=0 2024-04-23T10:37:06.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.406+0000 7f0b16ff5640 1 -- 172.21.15.97:0/453208527 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x557f3bbc90f0 con 0x557f3bcdcb70 2024-04-23T10:37:06.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.406+0000 7f0b16ff5640 10 client.17145 handle_command_reply: tid=0 2024-04-23T10:37:06.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.406+0000 7f0b3c57a740 1 client.17145 shutdown 2024-04-23T10:37:06.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.407+0000 7f0b3c57a740 1 -- 172.21.15.97:0/453208527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f3bc65890 msgr2=0x557f3bc66960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:37:06.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.407+0000 7f0b3c57a740 1 --2- 172.21.15.97:0/453208527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f3bc65890 0x557f3bc66960 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f0b240092d0 tx=0x7f0b24009300 comp rx=0 tx=0).stop 2024-04-23T10:37:06.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.407+0000 7f0b3c57a740 1 -- 172.21.15.97:0/453208527 shutdown_connections 2024-04-23T10:37:06.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.407+0000 7f0b3c57a740 1 --2- 172.21.15.97:0/453208527 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x557f3bcdcb70 0x557f3bcdef90 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:06.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.407+0000 7f0b3c57a740 1 --2- 172.21.15.97:0/453208527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f3bc65890 0x557f3bc66960 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:06.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.407+0000 7f0b3c57a740 1 --2- 172.21.15.97:0/453208527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557f3bc56b70 0x557f3bc6db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:06.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.407+0000 7f0b16ff5640 0 client.17145 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:37:06.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.407+0000 7f0b3c57a740 1 --2- 172.21.15.97:0/453208527 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557f3bc561c0 0x557f3bc602a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:06.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.407+0000 7f0b3c57a740 1 -- 172.21.15.97:0/453208527 >> 172.21.15.97:0/453208527 conn(0x557f3bc527b0 msgr2=0x557f3bc51140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:37:06.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.407+0000 7f0b3c57a740 1 -- 172.21.15.97:0/453208527 shutdown_connections 2024-04-23T10:37:06.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.407+0000 7f0b3c57a740 1 -- 172.21.15.97:0/453208527 wait complete. 2024-04-23T10:37:06.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.407+0000 7f0b3c57a740 20 client.17145 trim_cache size 0 max 16384 2024-04-23T10:37:06.410 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-23T10:37:06.410 INFO:teuthology.orchestra.run.smithi097.stdout: "epoch": 26, 2024-04-23T10:37:06.410 INFO:teuthology.orchestra.run.smithi097.stdout: "leader": 24488, 2024-04-23T10:37:06.410 INFO:teuthology.orchestra.run.smithi097.stdout: "set_version": 48, 2024-04-23T10:37:06.410 INFO:teuthology.orchestra.run.smithi097.stdout: "sets": { 2024-04-23T10:37:06.410 INFO:teuthology.orchestra.run.smithi097.stdout: "f5a8cd2e": { 2024-04-23T10:37:06.411 INFO:teuthology.orchestra.run.smithi097.stdout: "version": 48, 2024-04-23T10:37:06.411 INFO:teuthology.orchestra.run.smithi097.stdout: "age_ref": 0.0, 2024-04-23T10:37:06.411 INFO:teuthology.orchestra.run.smithi097.stdout: "state": { 2024-04-23T10:37:06.411 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RELEASED", 2024-04-23T10:37:06.411 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0 2024-04-23T10:37:06.411 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:37:06.411 INFO:teuthology.orchestra.run.smithi097.stdout: "timeout": 60.0, 2024-04-23T10:37:06.411 INFO:teuthology.orchestra.run.smithi097.stdout: "expiration": 86.9, 2024-04-23T10:37:06.411 INFO:teuthology.orchestra.run.smithi097.stdout: "members": { 2024-04-23T10:37:06.411 INFO:teuthology.orchestra.run.smithi097.stdout: "file:/": { 2024-04-23T10:37:06.411 INFO:teuthology.orchestra.run.smithi097.stdout: "excluded": false, 2024-04-23T10:37:06.411 INFO:teuthology.orchestra.run.smithi097.stdout: "state": { 2024-04-23T10:37:06.411 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RELEASED", 2024-04-23T10:37:06.411 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0 2024-04-23T10:37:06.411 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:37:06.412 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:37:06.412 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:37:06.412 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:37:06.412 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:37:06.412 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-23T10:37:06.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.407+0000 7f0b39787640 1 -- 172.21.15.97:0/1468853591 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f0b1c0a3d20 msgr2=0x7f0b1c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:37:06.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.407+0000 7f0b39787640 1 --2- 172.21.15.97:0/1468853591 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f0b1c0a3d20 0x7f0b1c0a61e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0b341009f0 tx=0x7f0b28005d30 comp rx=0 tx=0).stop 2024-04-23T10:37:06.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.407+0000 7f0b39787640 1 -- 172.21.15.97:0/1468853591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b3406a840 msgr2=0x7f0b34198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:37:06.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.407+0000 7f0b39787640 1 --2- 172.21.15.97:0/1468853591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b3406a840 0x7f0b34198730 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c005940 tx=0x7f0b2c0042f0 comp rx=0 tx=0).stop 2024-04-23T10:37:06.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.407+0000 7f0b39787640 1 -- 172.21.15.97:0/1468853591 shutdown_connections 2024-04-23T10:37:06.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.407+0000 7f0b39787640 1 --2- 172.21.15.97:0/1468853591 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f0b1c0a3d20 0x7f0b1c0a61e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:06.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.407+0000 7f0b39787640 1 --2- 172.21.15.97:0/1468853591 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0b3419cff0 0x7f0b3419d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:06.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.407+0000 7f0b39787640 1 --2- 172.21.15.97:0/1468853591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b34107270 0x7f0b34198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:06.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.407+0000 7f0b39787640 1 --2- 172.21.15.97:0/1468853591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b3406a840 0x7f0b34198730 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:06.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.407+0000 7f0b39787640 1 -- 172.21.15.97:0/1468853591 >> 172.21.15.97:0/1468853591 conn(0x7f0b34077eb0 msgr2=0x7f0b34108440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:37:06.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.408+0000 7f0b39787640 1 -- 172.21.15.97:0/1468853591 shutdown_connections 2024-04-23T10:37:06.414 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:06.408+0000 7f0b39787640 1 -- 172.21.15.97:0/1468853591 wait complete. 2024-04-23T10:37:06.417 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'f5a8cd2e', seconds elapsed: 27.9 2024-04-23T10:37:07.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:37:06 smithi196 ceph-mon[32919]: pgmap v338: 97 pgs: 97 active+clean; 3.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 49 KiB/s wr, 27 op/s 2024-04-23T10:37:07.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:37:06 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/4280034998' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:37:07.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:37:06 smithi097 ceph-mon[29197]: pgmap v338: 97 pgs: 97 active+clean; 3.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 49 KiB/s wr, 27 op/s 2024-04-23T10:37:07.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:37:06 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/4280034998' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:37:07.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:37:06 smithi183 ceph-mon[32317]: pgmap v338: 97 pgs: 97 active+clean; 3.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 49 KiB/s wr, 27 op/s 2024-04-23T10:37:07.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:37:06 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/4280034998' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:37:08.262 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:37:07 smithi097 ceph-mon[29197]: osdmap e98: 12 total, 12 up, 12 in 2024-04-23T10:37:08.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:37:07 smithi196 ceph-mon[32919]: osdmap e98: 12 total, 12 up, 12 in 2024-04-23T10:37:08.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:37:07 smithi183 ceph-mon[32317]: osdmap e98: 12 total, 12 up, 12 in 2024-04-23T10:37:09.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:37:08 smithi196 ceph-mon[32919]: pgmap v340: 97 pgs: 97 active+clean; 3.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 64 KiB/s wr, 33 op/s 2024-04-23T10:37:09.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:37:08 smithi097 ceph-mon[29197]: pgmap v340: 97 pgs: 97 active+clean; 3.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 64 KiB/s wr, 33 op/s 2024-04-23T10:37:09.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:37:08 smithi183 ceph-mon[32317]: pgmap v340: 97 pgs: 97 active+clean; 3.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 64 KiB/s wr, 33 op/s 2024-04-23T10:37:10.808 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.806+0000 7f672ce79640 1 -- 172.21.15.97:0/1692794529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6728101aa0 msgr2=0x7f6728105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:37:10.808 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.806+0000 7f672ce79640 1 --2- 172.21.15.97:0/1692794529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6728101aa0 0x7f6728105a90 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f6710009900 tx=0x7f671002a410 comp rx=0 tx=0).stop 2024-04-23T10:37:10.808 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.806+0000 7f672ce79640 1 -- 172.21.15.97:0/1692794529 shutdown_connections 2024-04-23T10:37:10.808 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.806+0000 7f672ce79640 1 --2- 172.21.15.97:0/1692794529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6728106170 0x7f672810dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:10.808 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.806+0000 7f672ce79640 1 --2- 172.21.15.97:0/1692794529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6728101aa0 0x7f6728105a90 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:10.808 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.806+0000 7f672ce79640 1 --2- 172.21.15.97:0/1692794529 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f67281010f0 0x7f67281014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:10.808 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.806+0000 7f672ce79640 1 -- 172.21.15.97:0/1692794529 >> 172.21.15.97:0/1692794529 conn(0x7f6728077fd0 msgr2=0x7f67280783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:37:10.808 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.806+0000 7f672ce79640 1 -- 172.21.15.97:0/1692794529 shutdown_connections 2024-04-23T10:37:10.808 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.806+0000 7f672ce79640 1 -- 172.21.15.97:0/1692794529 wait complete. 2024-04-23T10:37:10.808 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.807+0000 7f672ce79640 1 Processor -- start 2024-04-23T10:37:10.808 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.807+0000 7f672ce79640 1 -- start start 2024-04-23T10:37:10.808 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.807+0000 7f672ce79640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67281010f0 0x7f67281994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:10.809 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.807+0000 7f672ce79640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6728106170 0x7f6728199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:10.809 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.807+0000 7f672ce79640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6728199fa0 0x7f67281a4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:10.809 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.807+0000 7f672ce79640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f67281110b0 con 0x7f6728106170 2024-04-23T10:37:10.809 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.807+0000 7f672ce79640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6728110f30 con 0x7f6728199fa0 2024-04-23T10:37:10.809 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.807+0000 7f672ce79640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6728111230 con 0x7f67281010f0 2024-04-23T10:37:10.809 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.807+0000 7f671ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6728106170 0x7f6728199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:10.809 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.807+0000 7f671ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6728106170 0x7f6728199a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39862/0 (socket says 172.21.15.97:39862) 2024-04-23T10:37:10.809 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.807+0000 7f671ffff640 1 -- 172.21.15.97:0/1491613415 learned_addr learned my addr 172.21.15.97:0/1491613415 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:37:10.809 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.808+0000 7f672659c640 1 --2- 172.21.15.97:0/1491613415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67281010f0 0x7f67281994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:10.809 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.808+0000 7f6726d9d640 1 --2- 172.21.15.97:0/1491613415 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6728199fa0 0x7f67281a4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:10.809 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.808+0000 7f671ffff640 1 -- 172.21.15.97:0/1491613415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67281010f0 msgr2=0x7f67281994d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:37:10.809 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.808+0000 7f671ffff640 1 --2- 172.21.15.97:0/1491613415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67281010f0 0x7f67281994d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:10.809 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.808+0000 7f671ffff640 1 -- 172.21.15.97:0/1491613415 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6728199fa0 msgr2=0x7f67281a4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:37:10.809 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.808+0000 7f671ffff640 1 --2- 172.21.15.97:0/1491613415 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6728199fa0 0x7f67281a4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:10.809 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.808+0000 7f671ffff640 1 -- 172.21.15.97:0/1491613415 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67281a4d50 con 0x7f6728106170 2024-04-23T10:37:10.810 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.808+0000 7f671ffff640 1 --2- 172.21.15.97:0/1491613415 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6728106170 0x7f6728199a10 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f6710031a90 tx=0x7f6710004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:37:10.810 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.808+0000 7f671f7fe640 1 -- 172.21.15.97:0/1491613415 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f671003f030 con 0x7f6728106170 2024-04-23T10:37:10.810 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.808+0000 7f671f7fe640 1 -- 172.21.15.97:0/1491613415 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6710003cb0 con 0x7f6728106170 2024-04-23T10:37:10.810 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.808+0000 7f672ce79640 1 -- 172.21.15.97:0/1491613415 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67281a4fe0 con 0x7f6728106170 2024-04-23T10:37:10.810 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.808+0000 7f671f7fe640 1 -- 172.21.15.97:0/1491613415 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6710035070 con 0x7f6728106170 2024-04-23T10:37:10.810 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.808+0000 7f672ce79640 1 -- 172.21.15.97:0/1491613415 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67281a5440 con 0x7f6728106170 2024-04-23T10:37:10.817 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.810+0000 7f671f7fe640 1 -- 172.21.15.97:0/1491613415 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6710042030 con 0x7f6728106170 2024-04-23T10:37:10.817 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.811+0000 7f672fb8e740 1 Processor -- start 2024-04-23T10:37:10.817 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.811+0000 7f672fb8e740 1 -- start start 2024-04-23T10:37:10.817 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.811+0000 7f671f7fe640 1 --2- 172.21.15.97:0/1491613415 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f67140a3dc0 0x7f67140a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:10.817 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.811+0000 7f672fb8e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e469e6f620 0x55e469e6fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:10.818 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.811+0000 7f672fb8e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e469e6ff40 0x55e469e7a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:10.818 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.811+0000 7f672fb8e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e469e7a800 0x55e469e7cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:10.818 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.811+0000 7f672fb8e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e469dd4970 con 0x55e469e6ff40 2024-04-23T10:37:10.818 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.811+0000 7f672fb8e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e469d714a0 con 0x55e469e7a800 2024-04-23T10:37:10.818 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.811+0000 7f672fb8e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e469dd0930 con 0x55e469e6f620 2024-04-23T10:37:10.818 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.811+0000 7f6726d9d640 1 --2- 172.21.15.97:0/1491613415 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f67140a3dc0 0x7f67140a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:10.818 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.811+0000 7f671f7fe640 1 -- 172.21.15.97:0/1491613415 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8528+0+0 (secure 0 0 0) 0x7f67100f0db0 con 0x7f6728106170 2024-04-23T10:37:10.818 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.812+0000 7f671ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e469e6ff40 0x55e469e7a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:10.818 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.812+0000 7f671ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e469e6ff40 0x55e469e7a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39872/0 (socket says 172.21.15.97:39872) 2024-04-23T10:37:10.818 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.812+0000 7f671ffff640 1 -- 172.21.15.97:0/2129399495 learned_addr learned my addr 172.21.15.97:0/2129399495 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:37:10.818 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.812+0000 7f672659c640 1 --2- 172.21.15.97:0/2129399495 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e469e6f620 0x55e469e6fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:10.818 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.812+0000 7f671ffff640 1 -- 172.21.15.97:0/2129399495 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e469e6f620 msgr2=0x55e469e6fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:37:10.818 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.812+0000 7f671ffff640 1 --2- 172.21.15.97:0/2129399495 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e469e6f620 0x55e469e6fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:10.818 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.812+0000 7f6726d9d640 1 --2- 172.21.15.97:0/2129399495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e469e7a800 0x55e469e7cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:10.819 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.812+0000 7f671ffff640 1 -- 172.21.15.97:0/2129399495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e469e7a800 msgr2=0x55e469e7cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:37:10.819 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.812+0000 7f671ffff640 1 --2- 172.21.15.97:0/2129399495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e469e7a800 0x55e469e7cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:10.819 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.812+0000 7f671ffff640 1 -- 172.21.15.97:0/2129399495 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e469ae7210 con 0x55e469e6ff40 2024-04-23T10:37:10.819 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.812+0000 7f672659c640 1 --2- 172.21.15.97:0/2129399495 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e469e6f620 0x55e469e6fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:37:10.819 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.812+0000 7f6726d9d640 1 --2- 172.21.15.97:0/1491613415 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f67140a3dc0 0x7f67140a6280 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f672000acd0 tx=0x7f672000e720 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:37:10.819 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.812+0000 7f671ffff640 1 --2- 172.21.15.97:0/2129399495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e469e6ff40 0x55e469e7a2c0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f67100b91b0 tx=0x7f67100b90e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f744e2f8ba12d6c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:37:10.819 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.812+0000 7f671cff9640 1 -- 172.21.15.97:0/2129399495 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6710049040 con 0x55e469e6ff40 2024-04-23T10:37:10.819 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.812+0000 7f671cff9640 1 -- 172.21.15.97:0/2129399495 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67100bd450 con 0x55e469e6ff40 2024-04-23T10:37:10.819 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.812+0000 7f671cff9640 1 -- 172.21.15.97:0/2129399495 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67100b93c0 con 0x55e469e6ff40 2024-04-23T10:37:10.819 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.813+0000 7f672fb8e740 1 -- 172.21.15.97:0/2129399495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e469e6ff40 msgr2=0x55e469e7a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:37:10.819 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.813+0000 7f672fb8e740 1 --2- 172.21.15.97:0/2129399495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e469e6ff40 0x55e469e7a2c0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f67100b91b0 tx=0x7f67100b90e0 comp rx=0 tx=0).stop 2024-04-23T10:37:10.819 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.813+0000 7f672fb8e740 1 -- 172.21.15.97:0/2129399495 shutdown_connections 2024-04-23T10:37:10.819 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.813+0000 7f672fb8e740 1 --2- 172.21.15.97:0/2129399495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e469e7a800 0x55e469e7cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:10.819 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.813+0000 7f672fb8e740 1 --2- 172.21.15.97:0/2129399495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e469e6ff40 0x55e469e7a2c0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:10.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.813+0000 7f672fb8e740 1 --2- 172.21.15.97:0/2129399495 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e469e6f620 0x55e469e6fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:10.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.813+0000 7f672fb8e740 1 -- 172.21.15.97:0/2129399495 >> 172.21.15.97:0/2129399495 conn(0x55e469e6bb30 msgr2=0x55e469e6bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:37:10.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.813+0000 7f672fb8e740 1 -- 172.21.15.97:0/2129399495 shutdown_connections 2024-04-23T10:37:10.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.813+0000 7f672fb8e740 1 -- 172.21.15.97:0/2129399495 wait complete. 2024-04-23T10:37:10.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.813+0000 7f672fb8e740 1 -- start start 2024-04-23T10:37:10.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.813+0000 7f672fb8e740 1 Processor -- start 2024-04-23T10:37:10.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.814+0000 7f672fb8e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e469e6f620 0x55e469e7a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:10.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.814+0000 7f672fb8e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e469e6ff40 0x55e469e77490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:10.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.814+0000 7f672fb8e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e469e7a800 0x55e469e77a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:10.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.814+0000 7f672fb8e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e469e7d1f0 con 0x55e469e6ff40 2024-04-23T10:37:10.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.814+0000 7f672fb8e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e469d6f0b0 con 0x55e469e7a800 2024-04-23T10:37:10.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.814+0000 7f672fb8e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e469e7d370 con 0x55e469e6f620 2024-04-23T10:37:10.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.814+0000 7f672fb8e740 10 client.0 _ll_register_callbacks cb 0x55e469d5cd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:37:10.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.814+0000 7f671ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e469e6ff40 0x55e469e77490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:10.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.814+0000 7f671ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e469e6ff40 0x55e469e77490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39878/0 (socket says 172.21.15.97:39878) 2024-04-23T10:37:10.821 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.814+0000 7f671ffff640 1 -- 172.21.15.97:0/939692962 learned_addr learned my addr 172.21.15.97:0/939692962 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:37:10.821 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.814+0000 7f6726d9d640 1 --2- 172.21.15.97:0/939692962 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e469e7a800 0x55e469e77a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:10.821 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.814+0000 7f672659c640 1 --2- 172.21.15.97:0/939692962 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e469e6f620 0x55e469e7a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:10.821 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.814+0000 7f671ffff640 1 -- 172.21.15.97:0/939692962 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e469e6f620 msgr2=0x55e469e7a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:37:10.821 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.814+0000 7f671ffff640 1 --2- 172.21.15.97:0/939692962 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e469e6f620 0x55e469e7a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:10.821 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.814+0000 7f671ffff640 1 -- 172.21.15.97:0/939692962 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e469e7a800 msgr2=0x55e469e77a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:37:10.821 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.814+0000 7f671ffff640 1 --2- 172.21.15.97:0/939692962 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e469e7a800 0x55e469e77a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:10.821 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.814+0000 7f671ffff640 1 -- 172.21.15.97:0/939692962 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e469bb3620 con 0x55e469e6ff40 2024-04-23T10:37:10.821 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.814+0000 7f671ffff640 1 --2- 172.21.15.97:0/939692962 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e469e6ff40 0x55e469e77490 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f6710006010 tx=0x7f67100b9640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5395f54be60d4a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:37:10.821 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.814+0000 7f670dffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:37:10.821 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.814+0000 7f670dffb640 1 -- 172.21.15.97:0/939692962 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67100da070 con 0x55e469e6ff40 2024-04-23T10:37:10.821 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.814+0000 7f670dffb640 1 -- 172.21.15.97:0/939692962 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8528+0+0 (secure 0 0 0) 0x7f67100ee780 con 0x55e469e6ff40 2024-04-23T10:37:10.821 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.814+0000 7f672fb8e740 1 -- 172.21.15.97:0/939692962 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67100bd450 con 0x55e469e6ff40 2024-04-23T10:37:10.821 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.815+0000 7f670dffb640 1 -- 172.21.15.97:0/939692962 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6710049070 con 0x55e469e6ff40 2024-04-23T10:37:10.821 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.815+0000 7f672fb8e740 10 client.17157 fetch_fsmap learned FSMap version 26 2024-04-23T10:37:10.822 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.815+0000 7f672fb8e740 1 -- 172.21.15.97:0/939692962 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e469e7d1f0 con 0x55e469e6ff40 2024-04-23T10:37:10.822 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.815+0000 7f670dffb640 1 -- 172.21.15.97:0/939692962 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67100bd450 con 0x55e469e6ff40 2024-04-23T10:37:10.822 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.815+0000 7f670dffb640 1 -- 172.21.15.97:0/939692962 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f67100bd040 con 0x55e469e6ff40 2024-04-23T10:37:10.822 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.815+0000 7f672fb8e740 10 client.17157 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:37:10.822 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.815+0000 7f672fb8e740 10 client.17157 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:37:10.822 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.815+0000 7f672fb8e740 20 client.17157 populate_metadata read hostname 'smithi097' 2024-04-23T10:37:10.822 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.815+0000 7f672fb8e740 1 --2- 172.21.15.97:0/939692962 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55e469e8a320 0x55e469e8c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:10.822 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.815+0000 7f672fb8e740 4 client.17157 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:37:10.822 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.815+0000 7f672fb8e740 1 -- 172.21.15.97:0/939692962 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e469de2390 con 0x55e469e8a320 2024-04-23T10:37:10.822 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.815+0000 7f672659c640 1 --2- 172.21.15.97:0/939692962 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55e469e8a320 0x55e469e8c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:10.822 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.816+0000 7f672659c640 1 --2- 172.21.15.97:0/939692962 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55e469e8a320 0x55e469e8c710 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=280713b83123878b server_cookie=a26a3cd9cfdb9728 in_seq=0 out_seq=0 2024-04-23T10:37:10.822 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.816+0000 7f670dffb640 10 client.17157 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:37:10.822 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.817+0000 7f670dffb640 1 -- 172.21.15.97:0/939692962 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e469de2390 con 0x55e469e8a320 2024-04-23T10:37:10.822 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.817+0000 7f670dffb640 10 client.17157 handle_command_reply: tid=0 2024-04-23T10:37:10.823 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.817+0000 7f672fb8e740 1 client.17157 shutdown 2024-04-23T10:37:10.823 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.817+0000 7f672fb8e740 1 -- 172.21.15.97:0/939692962 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e469e6ff40 msgr2=0x55e469e77490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:37:10.823 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.817+0000 7f672fb8e740 1 --2- 172.21.15.97:0/939692962 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e469e6ff40 0x55e469e77490 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f6710006010 tx=0x7f67100b9640 comp rx=0 tx=0).stop 2024-04-23T10:37:10.823 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.817+0000 7f672fb8e740 1 -- 172.21.15.97:0/939692962 shutdown_connections 2024-04-23T10:37:10.823 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.817+0000 7f672fb8e740 1 --2- 172.21.15.97:0/939692962 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55e469e8a320 0x55e469e8c710 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:10.823 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.817+0000 7f672fb8e740 1 --2- 172.21.15.97:0/939692962 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e469e7a800 0x55e469e77a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:10.823 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.817+0000 7f670dffb640 0 client.17157 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:37:10.823 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.817+0000 7f672fb8e740 1 --2- 172.21.15.97:0/939692962 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e469e6ff40 0x55e469e77490 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:10.823 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.817+0000 7f672fb8e740 1 --2- 172.21.15.97:0/939692962 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e469e6f620 0x55e469e7a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:10.823 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.817+0000 7f672fb8e740 1 -- 172.21.15.97:0/939692962 >> 172.21.15.97:0/939692962 conn(0x55e469e6bb30 msgr2=0x55e469e71860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:37:10.823 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.817+0000 7f672fb8e740 1 -- 172.21.15.97:0/939692962 shutdown_connections 2024-04-23T10:37:10.823 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.817+0000 7f672fb8e740 1 -- 172.21.15.97:0/939692962 wait complete. 2024-04-23T10:37:10.823 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.817+0000 7f672fb8e740 20 client.17157 trim_cache size 0 max 16384 2024-04-23T10:37:10.835 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.832+0000 7f672fb8e740 1 Processor -- start 2024-04-23T10:37:10.836 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.832+0000 7f672fb8e740 1 -- start start 2024-04-23T10:37:10.836 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.832+0000 7f672fb8e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e469e6f620 0x55e469e79cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:10.836 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.832+0000 7f672fb8e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e469e6ff40 0x55e469e91f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:10.836 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.832+0000 7f672fb8e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e469e7a800 0x55e469e924d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:10.836 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.832+0000 7f672fb8e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f67281adae0 con 0x55e469e6f620 2024-04-23T10:37:10.836 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.832+0000 7f672fb8e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f67281af620 con 0x55e469e7a800 2024-04-23T10:37:10.836 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.832+0000 7f672fb8e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f67281ae1b0 con 0x55e469e6ff40 2024-04-23T10:37:10.836 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.833+0000 7f672659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e469e6f620 0x55e469e79cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:10.836 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.833+0000 7f672659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e469e6f620 0x55e469e79cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39894/0 (socket says 172.21.15.97:39894) 2024-04-23T10:37:10.836 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.833+0000 7f672659c640 1 -- 172.21.15.97:0/1729163440 learned_addr learned my addr 172.21.15.97:0/1729163440 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:37:10.836 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.833+0000 7f6726d9d640 1 --2- 172.21.15.97:0/1729163440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e469e7a800 0x55e469e924d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:10.836 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.833+0000 7f672659c640 1 -- 172.21.15.97:0/1729163440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e469e6ff40 msgr2=0x55e469e91f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:37:10.836 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.833+0000 7f672659c640 1 --2- 172.21.15.97:0/1729163440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e469e6ff40 0x55e469e91f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:10.837 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.833+0000 7f672659c640 1 -- 172.21.15.97:0/1729163440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e469e7a800 msgr2=0x55e469e924d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:37:10.837 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.833+0000 7f672659c640 1 --2- 172.21.15.97:0/1729163440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e469e7a800 0x55e469e924d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:10.837 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.833+0000 7f672659c640 1 -- 172.21.15.97:0/1729163440 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e469e86b10 con 0x55e469e6f620 2024-04-23T10:37:10.837 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.833+0000 7f6726d9d640 1 --2- 172.21.15.97:0/1729163440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e469e7a800 0x55e469e924d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:37:10.837 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.833+0000 7f672659c640 1 --2- 172.21.15.97:0/1729163440 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e469e6f620 0x55e469e79cd0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x55e469e8cf30 tx=0x7f6718013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=876ca393bd1e1997 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:37:10.837 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.833+0000 7f670ffff640 1 -- 172.21.15.97:0/1729163440 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6718004020 con 0x55e469e6f620 2024-04-23T10:37:10.837 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.833+0000 7f670ffff640 1 -- 172.21.15.97:0/1729163440 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f671801d8e0 con 0x55e469e6f620 2024-04-23T10:37:10.837 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.833+0000 7f670ffff640 1 -- 172.21.15.97:0/1729163440 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6718005440 con 0x55e469e6f620 2024-04-23T10:37:10.837 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.834+0000 7f672fb8e740 1 -- 172.21.15.97:0/1729163440 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e469e6f620 msgr2=0x55e469e79cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:37:10.837 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.834+0000 7f672fb8e740 1 --2- 172.21.15.97:0/1729163440 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e469e6f620 0x55e469e79cd0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x55e469e8cf30 tx=0x7f6718013830 comp rx=0 tx=0).stop 2024-04-23T10:37:10.837 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.834+0000 7f672fb8e740 1 -- 172.21.15.97:0/1729163440 shutdown_connections 2024-04-23T10:37:10.838 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.834+0000 7f672fb8e740 1 --2- 172.21.15.97:0/1729163440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e469e7a800 0x55e469e924d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:10.838 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.834+0000 7f672fb8e740 1 --2- 172.21.15.97:0/1729163440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e469e6ff40 0x55e469e91f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:10.838 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.834+0000 7f672fb8e740 1 --2- 172.21.15.97:0/1729163440 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e469e6f620 0x55e469e79cd0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:10.838 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.834+0000 7f672fb8e740 1 -- 172.21.15.97:0/1729163440 >> 172.21.15.97:0/1729163440 conn(0x55e469e6bb30 msgr2=0x55e469e6acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:37:10.838 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.834+0000 7f672fb8e740 1 -- 172.21.15.97:0/1729163440 shutdown_connections 2024-04-23T10:37:10.838 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.834+0000 7f672fb8e740 1 -- 172.21.15.97:0/1729163440 wait complete. 2024-04-23T10:37:10.838 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.834+0000 7f672fb8e740 1 -- start start 2024-04-23T10:37:10.838 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.834+0000 7f672fb8e740 1 Processor -- start 2024-04-23T10:37:10.838 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.834+0000 7f672fb8e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e469e6f620 0x55e469e82510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:10.838 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.834+0000 7f672fb8e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e469e6ff40 0x55e469e82a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:10.838 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.834+0000 7f672fb8e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e469e7a800 0x55e469e7f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:10.838 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.834+0000 7f672fb8e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f67281aef50 con 0x55e469e7a800 2024-04-23T10:37:10.838 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.834+0000 7f672fb8e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f67281ae880 con 0x55e469e6ff40 2024-04-23T10:37:10.839 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.834+0000 7f672fb8e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f67281abac0 con 0x55e469e6f620 2024-04-23T10:37:10.839 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.834+0000 7f672fb8e740 10 client.0 _ll_register_callbacks cb 0x7f67100ee4a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:37:10.839 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.835+0000 7f6726d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e469e7a800 0x55e469e7f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:10.839 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.835+0000 7f6726d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e469e7a800 0x55e469e7f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39906/0 (socket says 172.21.15.97:39906) 2024-04-23T10:37:10.839 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.835+0000 7f6726d9d640 1 -- 172.21.15.97:0/4122813743 learned_addr learned my addr 172.21.15.97:0/4122813743 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:37:10.839 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.835+0000 7f672659c640 1 --2- 172.21.15.97:0/4122813743 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e469e6f620 0x55e469e82510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:10.839 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.835+0000 7f6726d9d640 1 -- 172.21.15.97:0/4122813743 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e469e6f620 msgr2=0x55e469e82510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:37:10.839 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.835+0000 7f6726d9d640 1 --2- 172.21.15.97:0/4122813743 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e469e6f620 0x55e469e82510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:10.839 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.835+0000 7f6726d9d640 1 -- 172.21.15.97:0/4122813743 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e469e6ff40 msgr2=0x55e469e82a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:37:10.839 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.835+0000 7f6726d9d640 1 --2- 172.21.15.97:0/4122813743 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e469e6ff40 0x55e469e82a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:10.839 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.835+0000 7f6726d9d640 1 -- 172.21.15.97:0/4122813743 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e469e7fcc0 con 0x55e469e7a800 2024-04-23T10:37:10.839 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.835+0000 7f6726d9d640 1 --2- 172.21.15.97:0/4122813743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e469e7a800 0x55e469e7f4e0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f6720015000 tx=0x7f6720004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=defb8450296ce5fc server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:37:10.839 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.835+0000 7f670dffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:37:10.839 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.835+0000 7f670dffb640 1 -- 172.21.15.97:0/4122813743 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6720010d90 con 0x55e469e7a800 2024-04-23T10:37:10.840 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.835+0000 7f670dffb640 1 -- 172.21.15.97:0/4122813743 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8528+0+0 (secure 0 0 0) 0x7f6720003970 con 0x55e469e7a800 2024-04-23T10:37:10.840 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.835+0000 7f672fb8e740 1 -- 172.21.15.97:0/4122813743 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f671801d8e0 con 0x55e469e7a800 2024-04-23T10:37:10.840 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.835+0000 7f670dffb640 1 -- 172.21.15.97:0/4122813743 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6720003b70 con 0x55e469e7a800 2024-04-23T10:37:10.840 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.835+0000 7f672fb8e740 10 client.17163 fetch_fsmap learned FSMap version 26 2024-04-23T10:37:10.840 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.835+0000 7f672fb8e740 1 -- 172.21.15.97:0/4122813743 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e469ef1300 con 0x55e469e7a800 2024-04-23T10:37:10.840 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.835+0000 7f670dffb640 1 -- 172.21.15.97:0/4122813743 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f671801d8e0 con 0x55e469e7a800 2024-04-23T10:37:10.840 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.836+0000 7f670dffb640 1 -- 172.21.15.97:0/4122813743 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f672001c040 con 0x55e469e7a800 2024-04-23T10:37:10.840 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.836+0000 7f672fb8e740 10 client.17163 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:37:10.840 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.836+0000 7f672fb8e740 10 client.17163 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:37:10.840 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.836+0000 7f672fb8e740 20 client.17163 populate_metadata read hostname 'smithi097' 2024-04-23T10:37:10.840 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.836+0000 7f672fb8e740 1 --2- 172.21.15.97:0/4122813743 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55e469ef1ae0 0x55e469ef3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:10.840 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.836+0000 7f672fb8e740 4 client.17163 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:37:10.840 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.836+0000 7f672fb8e740 1 -- 172.21.15.97:0/4122813743 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e469ddc200 con 0x55e469ef1ae0 2024-04-23T10:37:10.840 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.836+0000 7f672659c640 1 --2- 172.21.15.97:0/4122813743 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55e469ef1ae0 0x55e469ef3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:10.841 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.836+0000 7f672659c640 1 --2- 172.21.15.97:0/4122813743 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55e469ef1ae0 0x55e469ef3f20 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8044edde59a7ab49 server_cookie=1a01a69cdcbcdcb6 in_seq=0 out_seq=0 2024-04-23T10:37:10.841 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.836+0000 7f670dffb640 10 client.17163 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:37:10.848 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.849+0000 7f670dffb640 1 -- 172.21.15.97:0/4122813743 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e469ddc200 con 0x55e469ef1ae0 2024-04-23T10:37:10.848 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.849+0000 7f670dffb640 10 client.17163 handle_command_reply: tid=0 2024-04-23T10:37:10.848 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.849+0000 7f672fb8e740 1 client.17163 shutdown 2024-04-23T10:37:10.848 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.849+0000 7f672fb8e740 1 -- 172.21.15.97:0/4122813743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e469e7a800 msgr2=0x55e469e7f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:37:10.848 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.849+0000 7f672fb8e740 1 --2- 172.21.15.97:0/4122813743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e469e7a800 0x55e469e7f4e0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f6720015000 tx=0x7f6720004ab0 comp rx=0 tx=0).stop 2024-04-23T10:37:10.848 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.849+0000 7f672fb8e740 1 -- 172.21.15.97:0/4122813743 shutdown_connections 2024-04-23T10:37:10.848 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.849+0000 7f672fb8e740 1 --2- 172.21.15.97:0/4122813743 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55e469ef1ae0 0x55e469ef3f20 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:10.848 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.849+0000 7f672fb8e740 1 --2- 172.21.15.97:0/4122813743 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e469e7a800 0x55e469e7f4e0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:10.849 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.849+0000 7f672fb8e740 1 --2- 172.21.15.97:0/4122813743 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e469e6ff40 0x55e469e82a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:10.849 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.849+0000 7f670dffb640 0 client.17163 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:37:10.849 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.849+0000 7f672fb8e740 1 --2- 172.21.15.97:0/4122813743 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e469e6f620 0x55e469e82510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:10.849 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.849+0000 7f672fb8e740 1 -- 172.21.15.97:0/4122813743 >> 172.21.15.97:0/4122813743 conn(0x55e469e6bb30 msgr2=0x55e469e6a420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:37:10.849 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.849+0000 7f672fb8e740 1 -- 172.21.15.97:0/4122813743 shutdown_connections 2024-04-23T10:37:10.849 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.849+0000 7f672fb8e740 1 -- 172.21.15.97:0/4122813743 wait complete. 2024-04-23T10:37:10.849 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.849+0000 7f672fb8e740 20 client.17163 trim_cache size 0 max 16384 2024-04-23T10:37:10.849 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:37:10.849 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:37:10.849 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:37:10.849 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:37:10.850 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.850+0000 7f672ce79640 1 -- 172.21.15.97:0/1491613415 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f67140a3dc0 msgr2=0x7f67140a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:37:10.850 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.850+0000 7f672ce79640 1 --2- 172.21.15.97:0/1491613415 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f67140a3dc0 0x7f67140a6280 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f672000acd0 tx=0x7f672000e720 comp rx=0 tx=0).stop 2024-04-23T10:37:10.850 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.850+0000 7f672ce79640 1 -- 172.21.15.97:0/1491613415 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6728106170 msgr2=0x7f6728199a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:37:10.851 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.850+0000 7f672ce79640 1 --2- 172.21.15.97:0/1491613415 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6728106170 0x7f6728199a10 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f6710031a90 tx=0x7f6710004270 comp rx=0 tx=0).stop 2024-04-23T10:37:10.851 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.850+0000 7f672ce79640 1 -- 172.21.15.97:0/1491613415 shutdown_connections 2024-04-23T10:37:10.851 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.850+0000 7f672ce79640 1 --2- 172.21.15.97:0/1491613415 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f67140a3dc0 0x7f67140a6280 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:10.851 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.850+0000 7f672ce79640 1 --2- 172.21.15.97:0/1491613415 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6728199fa0 0x7f67281a4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:10.851 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.850+0000 7f672ce79640 1 --2- 172.21.15.97:0/1491613415 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6728106170 0x7f6728199a10 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:10.851 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.850+0000 7f672ce79640 1 --2- 172.21.15.97:0/1491613415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67281010f0 0x7f67281994d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:10.851 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.850+0000 7f672ce79640 1 -- 172.21.15.97:0/1491613415 >> 172.21.15.97:0/1491613415 conn(0x7f6728077fd0 msgr2=0x7f67280fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:37:10.851 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.850+0000 7f672ce79640 1 -- 172.21.15.97:0/1491613415 shutdown_connections 2024-04-23T10:37:10.851 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:10.850+0000 7f672ce79640 1 -- 172.21.15.97:0/1491613415 wait complete. 2024-04-23T10:37:11.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:37:10 smithi196 ceph-mon[32919]: pgmap v341: 97 pgs: 97 active+clean; 3.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 64 KiB/s wr, 33 op/s 2024-04-23T10:37:11.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:37:10 smithi097 ceph-mon[29197]: pgmap v341: 97 pgs: 97 active+clean; 3.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 64 KiB/s wr, 33 op/s 2024-04-23T10:37:11.399 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:37:10 smithi183 ceph-mon[32317]: pgmap v341: 97 pgs: 97 active+clean; 3.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 64 KiB/s wr, 33 op/s 2024-04-23T10:37:13.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:37:12 smithi196 ceph-mon[32919]: pgmap v342: 97 pgs: 97 active+clean; 3.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 64 KiB/s wr, 33 op/s 2024-04-23T10:37:13.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:37:12 smithi097 ceph-mon[29197]: pgmap v342: 97 pgs: 97 active+clean; 3.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 64 KiB/s wr, 33 op/s 2024-04-23T10:37:13.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:37:12 smithi183 ceph-mon[32317]: pgmap v342: 97 pgs: 97 active+clean; 3.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 64 KiB/s wr, 33 op/s 2024-04-23T10:37:15.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:37:14 smithi196 ceph-mon[32919]: pgmap v343: 97 pgs: 97 active+clean; 3.4 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 31 KiB/s wr, 16 op/s 2024-04-23T10:37:15.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:37:14 smithi097 ceph-mon[29197]: pgmap v343: 97 pgs: 97 active+clean; 3.4 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 31 KiB/s wr, 16 op/s 2024-04-23T10:37:15.399 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:37:14 smithi183 ceph-mon[32317]: pgmap v343: 97 pgs: 97 active+clean; 3.4 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 31 KiB/s wr, 16 op/s 2024-04-23T10:37:17.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:37:17 smithi183 ceph-mon[32317]: pgmap v344: 97 pgs: 97 active+clean; 3.4 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 31 KiB/s wr, 16 op/s 2024-04-23T10:37:17.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:37:17 smithi183 ceph-mon[32317]: osdmap e99: 12 total, 12 up, 12 in 2024-04-23T10:37:17.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:37:17 smithi196 ceph-mon[32919]: pgmap v344: 97 pgs: 97 active+clean; 3.4 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 31 KiB/s wr, 16 op/s 2024-04-23T10:37:17.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:37:17 smithi196 ceph-mon[32919]: osdmap e99: 12 total, 12 up, 12 in 2024-04-23T10:37:17.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:37:17 smithi097 ceph-mon[29197]: pgmap v344: 97 pgs: 97 active+clean; 3.4 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 31 KiB/s wr, 16 op/s 2024-04-23T10:37:17.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:37:17 smithi097 ceph-mon[29197]: osdmap e99: 12 total, 12 up, 12 in 2024-04-23T10:37:19.394 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:37:19 smithi196 ceph-mon[32919]: pgmap v346: 97 pgs: 97 active+clean; 3.6 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 46 KiB/s wr, 17 op/s 2024-04-23T10:37:19.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:37:19 smithi183 ceph-mon[32317]: pgmap v346: 97 pgs: 97 active+clean; 3.6 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 46 KiB/s wr, 17 op/s 2024-04-23T10:37:19.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:37:19 smithi097 ceph-mon[29197]: pgmap v346: 97 pgs: 97 active+clean; 3.6 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 46 KiB/s wr, 17 op/s 2024-04-23T10:37:21.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:37:21 smithi183 ceph-mon[32317]: pgmap v347: 97 pgs: 97 active+clean; 3.6 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 46 KiB/s wr, 17 op/s 2024-04-23T10:37:21.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:37:21 smithi196 ceph-mon[32919]: pgmap v347: 97 pgs: 97 active+clean; 3.6 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 46 KiB/s wr, 17 op/s 2024-04-23T10:37:21.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:37:21 smithi097 ceph-mon[29197]: pgmap v347: 97 pgs: 97 active+clean; 3.6 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 46 KiB/s wr, 17 op/s 2024-04-23T10:37:23.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:37:23 smithi183 ceph-mon[32317]: pgmap v348: 97 pgs: 97 active+clean; 3.6 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 46 KiB/s wr, 17 op/s 2024-04-23T10:37:23.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:37:23 smithi196 ceph-mon[32919]: pgmap v348: 97 pgs: 97 active+clean; 3.6 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 46 KiB/s wr, 17 op/s 2024-04-23T10:37:23.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:37:23 smithi097 ceph-mon[29197]: pgmap v348: 97 pgs: 97 active+clean; 3.6 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 46 KiB/s wr, 17 op/s 2024-04-23T10:37:23.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.945+0000 7f988e66b640 1 -- 172.21.15.97:0/785478820 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9888103200 msgr2=0x7f98881035e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:37:23.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.945+0000 7f988e66b640 1 --2- 172.21.15.97:0/785478820 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9888103200 0x7f98881035e0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f9878009900 tx=0x7f987802a480 comp rx=0 tx=0).stop 2024-04-23T10:37:23.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.945+0000 7f988e66b640 1 -- 172.21.15.97:0/785478820 shutdown_connections 2024-04-23T10:37:23.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.945+0000 7f988e66b640 1 --2- 172.21.15.97:0/785478820 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f988810c5d0 0x7f988810ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:23.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.945+0000 7f988e66b640 1 --2- 172.21.15.97:0/785478820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9888103b20 0x7f988810c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:23.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.945+0000 7f988e66b640 1 --2- 172.21.15.97:0/785478820 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9888103200 0x7f98881035e0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:23.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.945+0000 7f988e66b640 1 -- 172.21.15.97:0/785478820 >> 172.21.15.97:0/785478820 conn(0x7f98880771d0 msgr2=0x7f98880775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:37:23.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.945+0000 7f988e66b640 1 -- 172.21.15.97:0/785478820 shutdown_connections 2024-04-23T10:37:23.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.945+0000 7f988e66b640 1 -- 172.21.15.97:0/785478820 wait complete. 2024-04-23T10:37:23.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.946+0000 7f988e66b640 1 Processor -- start 2024-04-23T10:37:23.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.946+0000 7f988e66b640 1 -- start start 2024-04-23T10:37:23.947 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.946+0000 7f988e66b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9888103200 0x7f9888198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:23.948 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.946+0000 7f988e66b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9888103b20 0x7f9888198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:23.948 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.946+0000 7f988e66b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f988810c5d0 0x7f988819cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:23.948 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.946+0000 7f988e66b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9888112220 con 0x7f988810c5d0 2024-04-23T10:37:23.948 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.947+0000 7f988e66b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f98881120a0 con 0x7f9888103200 2024-04-23T10:37:23.948 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.947+0000 7f988e66b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f98881123a0 con 0x7f9888103b20 2024-04-23T10:37:23.948 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.947+0000 7f988cc08640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f988810c5d0 0x7f988819cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:23.948 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.947+0000 7f98877fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9888103b20 0x7f9888198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:23.948 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.947+0000 7f988cc08640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f988810c5d0 0x7f988819cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55378/0 (socket says 172.21.15.97:55378) 2024-04-23T10:37:23.948 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.947+0000 7f988cc08640 1 -- 172.21.15.97:0/2679629828 learned_addr learned my addr 172.21.15.97:0/2679629828 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:37:23.948 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.947+0000 7f9887fff640 1 --2- 172.21.15.97:0/2679629828 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9888103200 0x7f9888198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:23.948 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.947+0000 7f988cc08640 1 -- 172.21.15.97:0/2679629828 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9888103b20 msgr2=0x7f9888198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:37:23.948 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.947+0000 7f988cc08640 1 --2- 172.21.15.97:0/2679629828 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9888103b20 0x7f9888198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:23.948 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.947+0000 7f988cc08640 1 -- 172.21.15.97:0/2679629828 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9888103200 msgr2=0x7f9888198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:37:23.948 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.947+0000 7f988cc08640 1 --2- 172.21.15.97:0/2679629828 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9888103200 0x7f9888198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:23.949 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.947+0000 7f988cc08640 1 -- 172.21.15.97:0/2679629828 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f988819d6a0 con 0x7f988810c5d0 2024-04-23T10:37:23.949 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.947+0000 7f988cc08640 1 --2- 172.21.15.97:0/2679629828 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f988810c5d0 0x7f988819cf70 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f988000cab0 tx=0x7f988000cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:37:23.949 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.947+0000 7f98857fa640 1 -- 172.21.15.97:0/2679629828 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988000ae20 con 0x7f988810c5d0 2024-04-23T10:37:23.949 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.947+0000 7f988e66b640 1 -- 172.21.15.97:0/2679629828 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f988819d990 con 0x7f988810c5d0 2024-04-23T10:37:23.949 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.947+0000 7f98857fa640 1 -- 172.21.15.97:0/2679629828 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f988000f900 con 0x7f988810c5d0 2024-04-23T10:37:23.949 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.947+0000 7f98857fa640 1 -- 172.21.15.97:0/2679629828 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9880018a50 con 0x7f988810c5d0 2024-04-23T10:37:23.949 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.948+0000 7f988e66b640 1 -- 172.21.15.97:0/2679629828 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98881a5220 con 0x7f988810c5d0 2024-04-23T10:37:23.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.950+0000 7f98857fa640 1 -- 172.21.15.97:0/2679629828 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f988000f450 con 0x7f988810c5d0 2024-04-23T10:37:23.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.950+0000 7f989137c740 1 Processor -- start 2024-04-23T10:37:23.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.950+0000 7f989137c740 1 -- start start 2024-04-23T10:37:23.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.950+0000 7f98857fa640 1 --2- 172.21.15.97:0/2679629828 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f98700a3dc0 0x7f98700a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:23.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.950+0000 7f989137c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e70f60620 0x555e70f60a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:23.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.950+0000 7f989137c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e70f60f40 0x555e70f6b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:23.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.950+0000 7f989137c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e70f6b800 0x555e70f6dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:23.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.950+0000 7f989137c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x555e70ec5970 con 0x555e70f6b800 2024-04-23T10:37:23.957 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.950+0000 7f989137c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555e70e624a0 con 0x555e70f60620 2024-04-23T10:37:23.957 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.950+0000 7f989137c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555e70ec1930 con 0x555e70f60f40 2024-04-23T10:37:23.957 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.951+0000 7f98877fe640 1 --2- 172.21.15.97:0/2679629828 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f98700a3dc0 0x7f98700a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:23.957 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.951+0000 7f988cc08640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e70f6b800 0x555e70f6dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:23.957 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.951+0000 7f988cc08640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e70f6b800 0x555e70f6dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55386/0 (socket says 172.21.15.97:55386) 2024-04-23T10:37:23.957 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.951+0000 7f988cc08640 1 -- 172.21.15.97:0/1038909917 learned_addr learned my addr 172.21.15.97:0/1038909917 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:37:23.957 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.951+0000 7f98857fa640 1 -- 172.21.15.97:0/2679629828 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8484+0+0 (secure 0 0 0) 0x7f98800d0f20 con 0x7f988810c5d0 2024-04-23T10:37:23.957 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.951+0000 7f9887fff640 1 --2- 172.21.15.97:0/1038909917 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e70f60620 0x555e70f60a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:23.957 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.951+0000 7f98877fe640 1 --2- 172.21.15.97:0/1038909917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e70f60f40 0x555e70f6b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:23.957 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.951+0000 7f988cc08640 1 -- 172.21.15.97:0/1038909917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e70f60f40 msgr2=0x555e70f6b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:37:23.957 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.951+0000 7f988cc08640 1 --2- 172.21.15.97:0/1038909917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e70f60f40 0x555e70f6b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:23.957 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.951+0000 7f988cc08640 1 -- 172.21.15.97:0/1038909917 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e70f60620 msgr2=0x555e70f60a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:37:23.957 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.951+0000 7f988cc08640 1 --2- 172.21.15.97:0/1038909917 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e70f60620 0x555e70f60a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:23.957 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.951+0000 7f988cc08640 1 -- 172.21.15.97:0/1038909917 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e70bd8210 con 0x555e70f6b800 2024-04-23T10:37:23.958 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.951+0000 7f98877fe640 1 --2- 172.21.15.97:0/2679629828 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f98700a3dc0 0x7f98700a6280 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f98881006e0 tx=0x7f987c00be70 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:37:23.958 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.951+0000 7f988cc08640 1 --2- 172.21.15.97:0/1038909917 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e70f6b800 0x555e70f6dbf0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f988000a6e0 tx=0x7f988001c590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cac5b26847742a8b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:37:23.958 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.951+0000 7f986e7fc640 1 -- 172.21.15.97:0/1038909917 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988000f740 con 0x555e70f6b800 2024-04-23T10:37:23.958 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.951+0000 7f986e7fc640 1 -- 172.21.15.97:0/1038909917 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f988009d5e0 con 0x555e70f6b800 2024-04-23T10:37:23.958 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.951+0000 7f986e7fc640 1 -- 172.21.15.97:0/1038909917 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988000fab0 con 0x555e70f6b800 2024-04-23T10:37:23.958 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.952+0000 7f989137c740 1 -- 172.21.15.97:0/1038909917 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e70f6b800 msgr2=0x555e70f6dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:37:23.958 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.952+0000 7f989137c740 1 --2- 172.21.15.97:0/1038909917 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e70f6b800 0x555e70f6dbf0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f988000a6e0 tx=0x7f988001c590 comp rx=0 tx=0).stop 2024-04-23T10:37:23.958 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.952+0000 7f989137c740 1 -- 172.21.15.97:0/1038909917 shutdown_connections 2024-04-23T10:37:23.958 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.952+0000 7f989137c740 1 --2- 172.21.15.97:0/1038909917 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e70f6b800 0x555e70f6dbf0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:23.958 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.952+0000 7f989137c740 1 --2- 172.21.15.97:0/1038909917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e70f60f40 0x555e70f6b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:23.958 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.952+0000 7f989137c740 1 --2- 172.21.15.97:0/1038909917 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e70f60620 0x555e70f60a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:23.958 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.952+0000 7f989137c740 1 -- 172.21.15.97:0/1038909917 >> 172.21.15.97:0/1038909917 conn(0x555e70f5cb30 msgr2=0x555e70f5cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:37:23.958 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.952+0000 7f989137c740 1 -- 172.21.15.97:0/1038909917 shutdown_connections 2024-04-23T10:37:23.959 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.952+0000 7f989137c740 1 -- 172.21.15.97:0/1038909917 wait complete. 2024-04-23T10:37:23.959 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.952+0000 7f989137c740 1 -- start start 2024-04-23T10:37:23.959 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.953+0000 7f989137c740 1 Processor -- start 2024-04-23T10:37:23.959 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.953+0000 7f989137c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e70f60620 0x555e70f6b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:23.959 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.953+0000 7f989137c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e70f60f40 0x555e70f68340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:23.959 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.953+0000 7f989137c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e70f6b800 0x555e70f68880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:23.959 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.953+0000 7f989137c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x555e70f6e430 con 0x555e70f60f40 2024-04-23T10:37:23.959 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.953+0000 7f989137c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555e70e600b0 con 0x555e70f6b800 2024-04-23T10:37:23.959 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.953+0000 7f989137c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555e70f6e5b0 con 0x555e70f60620 2024-04-23T10:37:23.959 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.953+0000 7f989137c740 10 client.0 _ll_register_callbacks cb 0x555e70e4dd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:37:23.959 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.953+0000 7f98877fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e70f60f40 0x555e70f68340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:23.959 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.953+0000 7f98877fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e70f60f40 0x555e70f68340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55398/0 (socket says 172.21.15.97:55398) 2024-04-23T10:37:23.959 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.953+0000 7f98877fe640 1 -- 172.21.15.97:0/1722054919 learned_addr learned my addr 172.21.15.97:0/1722054919 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:37:23.959 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.953+0000 7f9887fff640 1 --2- 172.21.15.97:0/1722054919 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e70f60620 0x555e70f6b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:23.960 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.953+0000 7f988cc08640 1 --2- 172.21.15.97:0/1722054919 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e70f6b800 0x555e70f68880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:23.960 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.953+0000 7f98877fe640 1 -- 172.21.15.97:0/1722054919 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e70f60620 msgr2=0x555e70f6b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:37:23.960 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.953+0000 7f98877fe640 1 --2- 172.21.15.97:0/1722054919 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e70f60620 0x555e70f6b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:23.960 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.953+0000 7f98877fe640 1 -- 172.21.15.97:0/1722054919 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e70f6b800 msgr2=0x555e70f68880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:37:23.960 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.953+0000 7f98877fe640 1 --2- 172.21.15.97:0/1722054919 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e70f6b800 0x555e70f68880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:23.960 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.953+0000 7f98877fe640 1 -- 172.21.15.97:0/1722054919 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e70ca4620 con 0x555e70f60f40 2024-04-23T10:37:23.960 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.953+0000 7f9887fff640 1 --2- 172.21.15.97:0/1722054919 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e70f60620 0x555e70f6b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:37:23.960 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.954+0000 7f98877fe640 1 --2- 172.21.15.97:0/1722054919 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e70f60f40 0x555e70f68340 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f987c012670 tx=0x7f987c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b567cd9c0e3b0a5c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:37:23.960 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.954+0000 7f98677fe640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:37:23.960 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.954+0000 7f98677fe640 1 -- 172.21.15.97:0/1722054919 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f987c005220 con 0x555e70f60f40 2024-04-23T10:37:23.960 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.954+0000 7f98677fe640 1 -- 172.21.15.97:0/1722054919 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8484+0+0 (secure 0 0 0) 0x7f987c0053c0 con 0x555e70f60f40 2024-04-23T10:37:23.960 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.954+0000 7f989137c740 1 -- 172.21.15.97:0/1722054919 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f988009d5e0 con 0x555e70f60f40 2024-04-23T10:37:23.960 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.954+0000 7f98677fe640 1 -- 172.21.15.97:0/1722054919 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f987c0055c0 con 0x555e70f60f40 2024-04-23T10:37:23.960 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.954+0000 7f989137c740 10 client.17175 fetch_fsmap learned FSMap version 26 2024-04-23T10:37:23.961 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.954+0000 7f989137c740 1 -- 172.21.15.97:0/1722054919 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555e70f7a240 con 0x555e70f60f40 2024-04-23T10:37:23.961 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.954+0000 7f98677fe640 1 -- 172.21.15.97:0/1722054919 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f987c004c50 con 0x555e70f60f40 2024-04-23T10:37:23.961 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.954+0000 7f98677fe640 1 -- 172.21.15.97:0/1722054919 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f987c02e020 con 0x555e70f60f40 2024-04-23T10:37:23.961 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.954+0000 7f989137c740 10 client.17175 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:37:23.961 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.954+0000 7f989137c740 10 client.17175 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:37:23.961 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.954+0000 7f989137c740 20 client.17175 populate_metadata read hostname 'smithi097' 2024-04-23T10:37:23.961 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.954+0000 7f989137c740 1 --2- 172.21.15.97:0/1722054919 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x555e70f7b6b0 0x555e70f7db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:23.961 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.954+0000 7f989137c740 4 client.17175 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:37:23.961 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.954+0000 7f989137c740 1 -- 172.21.15.97:0/1722054919 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f988009d5e0 con 0x555e70f7b6b0 2024-04-23T10:37:23.961 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.955+0000 7f9887fff640 1 --2- 172.21.15.97:0/1722054919 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x555e70f7b6b0 0x555e70f7db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:23.961 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.955+0000 7f98677fe640 10 client.17175 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:37:23.961 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.955+0000 7f9887fff640 1 --2- 172.21.15.97:0/1722054919 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x555e70f7b6b0 0x555e70f7db00 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c40e906ea021a509 server_cookie=aaf1ace415576271 in_seq=0 out_seq=0 2024-04-23T10:37:23.961 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.956+0000 7f98677fe640 1 -- 172.21.15.97:0/1722054919 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f988009d5e0 con 0x555e70f7b6b0 2024-04-23T10:37:23.961 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.956+0000 7f98677fe640 10 client.17175 handle_command_reply: tid=0 2024-04-23T10:37:23.961 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.956+0000 7f989137c740 1 client.17175 shutdown 2024-04-23T10:37:23.962 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.956+0000 7f989137c740 1 -- 172.21.15.97:0/1722054919 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e70f60f40 msgr2=0x555e70f68340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:37:23.962 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.956+0000 7f989137c740 1 --2- 172.21.15.97:0/1722054919 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e70f60f40 0x555e70f68340 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f987c012670 tx=0x7f987c004730 comp rx=0 tx=0).stop 2024-04-23T10:37:23.962 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.956+0000 7f989137c740 1 -- 172.21.15.97:0/1722054919 shutdown_connections 2024-04-23T10:37:23.962 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.956+0000 7f989137c740 1 --2- 172.21.15.97:0/1722054919 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x555e70f7b6b0 0x555e70f7db00 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:23.962 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.956+0000 7f989137c740 1 --2- 172.21.15.97:0/1722054919 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e70f6b800 0x555e70f68880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:23.962 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.956+0000 7f98677fe640 0 client.17175 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:37:23.962 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.956+0000 7f989137c740 1 --2- 172.21.15.97:0/1722054919 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e70f60f40 0x555e70f68340 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:23.962 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.956+0000 7f989137c740 1 --2- 172.21.15.97:0/1722054919 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e70f60620 0x555e70f6b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:23.962 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.956+0000 7f989137c740 1 -- 172.21.15.97:0/1722054919 >> 172.21.15.97:0/1722054919 conn(0x555e70f5cb30 msgr2=0x555e70f628b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:37:23.962 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.956+0000 7f989137c740 1 -- 172.21.15.97:0/1722054919 shutdown_connections 2024-04-23T10:37:23.962 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.956+0000 7f989137c740 1 -- 172.21.15.97:0/1722054919 wait complete. 2024-04-23T10:37:23.962 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.956+0000 7f989137c740 20 client.17175 trim_cache size 0 max 16384 2024-04-23T10:37:23.971 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.971+0000 7f989137c740 1 Processor -- start 2024-04-23T10:37:23.971 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.971+0000 7f989137c740 1 -- start start 2024-04-23T10:37:23.971 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.971+0000 7f989137c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e70f5cb30 0x555e70f772f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:23.971 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.971+0000 7f989137c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e70f60620 0x555e70f6ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.971+0000 7f989137c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e70f60f40 0x555e70f83410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.971+0000 7f989137c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f98881ada40 con 0x555e70f5cb30 2024-04-23T10:37:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.971+0000 7f989137c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f98881af580 con 0x555e70f60f40 2024-04-23T10:37:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.971+0000 7f989137c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f98881ae110 con 0x555e70f60620 2024-04-23T10:37:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.971+0000 7f9887fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e70f5cb30 0x555e70f772f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.971+0000 7f9887fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e70f5cb30 0x555e70f772f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55406/0 (socket says 172.21.15.97:55406) 2024-04-23T10:37:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.971+0000 7f9887fff640 1 -- 172.21.15.97:0/1329935753 learned_addr learned my addr 172.21.15.97:0/1329935753 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:37:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.971+0000 7f9887fff640 1 -- 172.21.15.97:0/1329935753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e70f60620 msgr2=0x555e70f6ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:37:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.971+0000 7f9887fff640 1 --2- 172.21.15.97:0/1329935753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e70f60620 0x555e70f6ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.971+0000 7f9887fff640 1 -- 172.21.15.97:0/1329935753 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e70f60f40 msgr2=0x555e70f83410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:37:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.971+0000 7f9887fff640 1 --2- 172.21.15.97:0/1329935753 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e70f60f40 0x555e70f83410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.971+0000 7f9887fff640 1 -- 172.21.15.97:0/1329935753 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e70f66890 con 0x555e70f5cb30 2024-04-23T10:37:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.971+0000 7f9887fff640 1 --2- 172.21.15.97:0/1329935753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e70f5cb30 0x555e70f772f0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f9878009900 tx=0x7f9878005e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52f17c818c859d9c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:37:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.971+0000 7f986dffb640 1 -- 172.21.15.97:0/1329935753 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f987804a070 con 0x555e70f5cb30 2024-04-23T10:37:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.972+0000 7f986dffb640 1 -- 172.21.15.97:0/1329935753 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98780044c0 con 0x555e70f5cb30 2024-04-23T10:37:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.972+0000 7f986dffb640 1 -- 172.21.15.97:0/1329935753 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9878035040 con 0x555e70f5cb30 2024-04-23T10:37:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.972+0000 7f989137c740 1 -- 172.21.15.97:0/1329935753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e70f5cb30 msgr2=0x555e70f772f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:37:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.972+0000 7f989137c740 1 --2- 172.21.15.97:0/1329935753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e70f5cb30 0x555e70f772f0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f9878009900 tx=0x7f9878005e60 comp rx=0 tx=0).stop 2024-04-23T10:37:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.972+0000 7f989137c740 1 -- 172.21.15.97:0/1329935753 shutdown_connections 2024-04-23T10:37:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.972+0000 7f989137c740 1 --2- 172.21.15.97:0/1329935753 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e70f60f40 0x555e70f83410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.972+0000 7f989137c740 1 --2- 172.21.15.97:0/1329935753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e70f60620 0x555e70f6ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.972+0000 7f989137c740 1 --2- 172.21.15.97:0/1329935753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e70f5cb30 0x555e70f772f0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.972+0000 7f989137c740 1 -- 172.21.15.97:0/1329935753 >> 172.21.15.97:0/1329935753 conn(0x555e70f689e0 msgr2=0x555e70f5b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:37:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.972+0000 7f989137c740 1 -- 172.21.15.97:0/1329935753 shutdown_connections 2024-04-23T10:37:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.972+0000 7f989137c740 1 -- 172.21.15.97:0/1329935753 wait complete. 2024-04-23T10:37:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.972+0000 7f989137c740 1 -- start start 2024-04-23T10:37:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.973+0000 7f989137c740 1 Processor -- start 2024-04-23T10:37:23.976 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.973+0000 7f989137c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e70f5cb30 0x555e70fd9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:23.976 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.973+0000 7f989137c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e70f60620 0x555e70fda180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.973+0000 7f989137c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e70f60f40 0x555e70fe4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.973+0000 7f989137c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f98881aeeb0 con 0x555e70f60620 2024-04-23T10:37:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.973+0000 7f989137c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f98881ae7e0 con 0x555e70f5cb30 2024-04-23T10:37:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.973+0000 7f989137c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f98881aba40 con 0x555e70f60f40 2024-04-23T10:37:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.973+0000 7f989137c740 10 client.0 _ll_register_callbacks cb 0x7f987c003d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:37:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.973+0000 7f98877fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e70f60620 0x555e70fda180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.973+0000 7f98877fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e70f60620 0x555e70fda180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:55422/0 (socket says 172.21.15.97:55422) 2024-04-23T10:37:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.973+0000 7f98877fe640 1 -- 172.21.15.97:0/738328996 learned_addr learned my addr 172.21.15.97:0/738328996 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:37:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.973+0000 7f988cc08640 1 --2- 172.21.15.97:0/738328996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e70f60f40 0x555e70fe4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.973+0000 7f98877fe640 1 -- 172.21.15.97:0/738328996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e70f60f40 msgr2=0x555e70fe4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:37:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.973+0000 7f98877fe640 1 --2- 172.21.15.97:0/738328996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e70f60f40 0x555e70fe4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.973+0000 7f98877fe640 1 -- 172.21.15.97:0/738328996 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e70f5cb30 msgr2=0x555e70fd9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:37:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.973+0000 7f9887fff640 1 --2- 172.21.15.97:0/738328996 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e70f5cb30 0x555e70fd9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.973+0000 7f98877fe640 1 --2- 172.21.15.97:0/738328996 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e70f5cb30 0x555e70fd9c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.973+0000 7f98877fe640 1 -- 172.21.15.97:0/738328996 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e70fe5200 con 0x555e70f60620 2024-04-23T10:37:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.974+0000 7f98877fe640 1 --2- 172.21.15.97:0/738328996 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e70f60620 0x555e70fda180 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f987c003a30 tx=0x7f987c02aab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57ab50e681ecb6a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:37:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.974+0000 7f98677fe640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:37:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.974+0000 7f98677fe640 1 -- 172.21.15.97:0/738328996 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f987c005420 con 0x555e70f60620 2024-04-23T10:37:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.974+0000 7f98677fe640 1 -- 172.21.15.97:0/738328996 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8484+0+0 (secure 0 0 0) 0x7f987c02a330 con 0x555e70f60620 2024-04-23T10:37:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.974+0000 7f989137c740 1 -- 172.21.15.97:0/738328996 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555e70ed3390 con 0x555e70f60620 2024-04-23T10:37:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.974+0000 7f98677fe640 1 -- 172.21.15.97:0/738328996 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f987c040050 con 0x555e70f60620 2024-04-23T10:37:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.974+0000 7f989137c740 10 client.17181 fetch_fsmap learned FSMap version 26 2024-04-23T10:37:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.974+0000 7f989137c740 1 -- 172.21.15.97:0/738328996 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555e70fe6440 con 0x555e70f60620 2024-04-23T10:37:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.974+0000 7f98677fe640 1 -- 172.21.15.97:0/738328996 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x555e70ed3390 con 0x555e70f60620 2024-04-23T10:37:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.974+0000 7f98677fe640 1 -- 172.21.15.97:0/738328996 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f987c02e040 con 0x555e70f60620 2024-04-23T10:37:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.974+0000 7f989137c740 10 client.17181 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:37:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.974+0000 7f989137c740 10 client.17181 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:37:23.978 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.974+0000 7f989137c740 20 client.17181 populate_metadata read hostname 'smithi097' 2024-04-23T10:37:23.979 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.974+0000 7f989137c740 1 --2- 172.21.15.97:0/738328996 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x555e70f6b800 0x555e70fe8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:23.979 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.974+0000 7f989137c740 4 client.17181 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:37:23.979 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.974+0000 7f989137c740 1 -- 172.21.15.97:0/738328996 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555e70ecd200 con 0x555e70f6b800 2024-04-23T10:37:23.979 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.975+0000 7f9887fff640 1 --2- 172.21.15.97:0/738328996 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x555e70f6b800 0x555e70fe8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:23.979 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.975+0000 7f98677fe640 10 client.17181 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:37:23.979 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.975+0000 7f9887fff640 1 --2- 172.21.15.97:0/738328996 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x555e70f6b800 0x555e70fe8ba0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f078816d2d38a51 server_cookie=6209feaff4391a09 in_seq=0 out_seq=0 2024-04-23T10:37:23.988 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.989+0000 7f98677fe640 1 -- 172.21.15.97:0/738328996 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555e70ecd200 con 0x555e70f6b800 2024-04-23T10:37:23.988 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.989+0000 7f98677fe640 10 client.17181 handle_command_reply: tid=0 2024-04-23T10:37:23.988 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.989+0000 7f989137c740 1 client.17181 shutdown 2024-04-23T10:37:23.988 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.989+0000 7f989137c740 1 -- 172.21.15.97:0/738328996 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e70f60620 msgr2=0x555e70fda180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:37:23.989 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.989+0000 7f989137c740 1 --2- 172.21.15.97:0/738328996 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e70f60620 0x555e70fda180 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f987c003a30 tx=0x7f987c02aab0 comp rx=0 tx=0).stop 2024-04-23T10:37:23.989 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.989+0000 7f989137c740 1 -- 172.21.15.97:0/738328996 shutdown_connections 2024-04-23T10:37:23.989 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.989+0000 7f989137c740 1 --2- 172.21.15.97:0/738328996 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x555e70f6b800 0x555e70fe8ba0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:23.989 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.989+0000 7f989137c740 1 --2- 172.21.15.97:0/738328996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e70f60f40 0x555e70fe4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:23.989 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.989+0000 7f989137c740 1 --2- 172.21.15.97:0/738328996 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e70f60620 0x555e70fda180 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:23.989 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.989+0000 7f989137c740 1 --2- 172.21.15.97:0/738328996 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e70f5cb30 0x555e70fd9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:23.989 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.989+0000 7f98677fe640 0 client.17181 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:37:23.989 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.989+0000 7f989137c740 1 -- 172.21.15.97:0/738328996 >> 172.21.15.97:0/738328996 conn(0x555e70f689e0 msgr2=0x555e70f5bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:37:23.989 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.989+0000 7f989137c740 1 -- 172.21.15.97:0/738328996 shutdown_connections 2024-04-23T10:37:23.989 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.989+0000 7f989137c740 1 -- 172.21.15.97:0/738328996 wait complete. 2024-04-23T10:37:23.989 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.989+0000 7f989137c740 20 client.17181 trim_cache size 0 max 16384 2024-04-23T10:37:23.989 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:37:23.990 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:37:23.990 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:37:23.990 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:37:23.990 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.990+0000 7f988e66b640 1 -- 172.21.15.97:0/2679629828 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f98700a3dc0 msgr2=0x7f98700a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:37:23.991 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.990+0000 7f988e66b640 1 --2- 172.21.15.97:0/2679629828 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f98700a3dc0 0x7f98700a6280 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f98881006e0 tx=0x7f987c00be70 comp rx=0 tx=0).stop 2024-04-23T10:37:23.991 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.990+0000 7f988e66b640 1 -- 172.21.15.97:0/2679629828 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f988810c5d0 msgr2=0x7f988819cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:37:23.991 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.990+0000 7f988e66b640 1 --2- 172.21.15.97:0/2679629828 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f988810c5d0 0x7f988819cf70 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f988000cab0 tx=0x7f988000cf70 comp rx=0 tx=0).stop 2024-04-23T10:37:23.991 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.990+0000 7f988e66b640 1 -- 172.21.15.97:0/2679629828 shutdown_connections 2024-04-23T10:37:23.991 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.990+0000 7f988e66b640 1 --2- 172.21.15.97:0/2679629828 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f98700a3dc0 0x7f98700a6280 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:23.991 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.990+0000 7f988e66b640 1 --2- 172.21.15.97:0/2679629828 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f988810c5d0 0x7f988819cf70 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:23.991 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.990+0000 7f988e66b640 1 --2- 172.21.15.97:0/2679629828 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9888103b20 0x7f9888198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:23.991 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.990+0000 7f988e66b640 1 --2- 172.21.15.97:0/2679629828 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9888103200 0x7f9888198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:23.991 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.990+0000 7f988e66b640 1 -- 172.21.15.97:0/2679629828 >> 172.21.15.97:0/2679629828 conn(0x7f98880771d0 msgr2=0x7f98880fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:37:23.991 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.990+0000 7f988e66b640 1 -- 172.21.15.97:0/2679629828 shutdown_connections 2024-04-23T10:37:23.991 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:23.990+0000 7f988e66b640 1 -- 172.21.15.97:0/2679629828 wait complete. 2024-04-23T10:37:25.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:37:25 smithi183 ceph-mon[32317]: pgmap v349: 97 pgs: 97 active+clean; 3.8 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 40 KiB/s wr, 14 op/s 2024-04-23T10:37:25.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:37:25 smithi196 ceph-mon[32919]: pgmap v349: 97 pgs: 97 active+clean; 3.8 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 40 KiB/s wr, 14 op/s 2024-04-23T10:37:25.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:37:25 smithi097 ceph-mon[29197]: pgmap v349: 97 pgs: 97 active+clean; 3.8 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 40 KiB/s wr, 14 op/s 2024-04-23T10:37:27.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:37:27 smithi183 ceph-mon[32317]: pgmap v350: 97 pgs: 97 active+clean; 3.8 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 40 KiB/s wr, 14 op/s 2024-04-23T10:37:27.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:37:27 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:37:27.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:37:27 smithi196 ceph-mon[32919]: pgmap v350: 97 pgs: 97 active+clean; 3.8 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 40 KiB/s wr, 14 op/s 2024-04-23T10:37:27.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:37:27 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:37:27.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:37:27 smithi097 ceph-mon[29197]: pgmap v350: 97 pgs: 97 active+clean; 3.8 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 40 KiB/s wr, 14 op/s 2024-04-23T10:37:27.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:37:27 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:37:28.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:37:28 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:37:28.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:37:28 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:37:28.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:37:28 smithi196 ceph-mon[32919]: pgmap v351: 97 pgs: 97 active+clean; 4.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 61 KiB/s wr, 22 op/s 2024-04-23T10:37:28.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:37:28 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:37:28.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:37:28 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:37:28.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:37:28 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:37:28.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:37:28 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:37:28.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:37:28 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:37:28.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:37:28 smithi097 ceph-mon[29197]: pgmap v351: 97 pgs: 97 active+clean; 4.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 61 KiB/s wr, 22 op/s 2024-04-23T10:37:28.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:37:28 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:37:28.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:37:28 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:37:28.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:37:28 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:37:28.649 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:37:28 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:37:28.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:37:28 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:37:28.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:37:28 smithi183 ceph-mon[32317]: pgmap v351: 97 pgs: 97 active+clean; 4.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 61 KiB/s wr, 22 op/s 2024-04-23T10:37:28.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:37:28 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:37:28.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:37:28 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:37:28.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:37:28 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:37:30.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:37:30 smithi097 ceph-mon[29197]: pgmap v352: 97 pgs: 97 active+clean; 4.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-23T10:37:30.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:37:30 smithi183 ceph-mon[32317]: pgmap v352: 97 pgs: 97 active+clean; 4.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-23T10:37:30.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:37:30 smithi196 ceph-mon[32919]: pgmap v352: 97 pgs: 97 active+clean; 4.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-23T10:37:32.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:37:32 smithi183 ceph-mon[32317]: pgmap v353: 97 pgs: 97 active+clean; 4.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-23T10:37:32.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:37:32 smithi196 ceph-mon[32919]: pgmap v353: 97 pgs: 97 active+clean; 4.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-23T10:37:32.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:37:32 smithi097 ceph-mon[29197]: pgmap v353: 97 pgs: 97 active+clean; 4.0 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-23T10:37:34.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:37:34 smithi183 ceph-mon[32317]: pgmap v354: 97 pgs: 97 active+clean; 4.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-23T10:37:34.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:37:34 smithi196 ceph-mon[32919]: pgmap v354: 97 pgs: 97 active+clean; 4.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-23T10:37:34.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:37:34 smithi097 ceph-mon[29197]: pgmap v354: 97 pgs: 97 active+clean; 4.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-23T10:37:36.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:37:36 smithi183 ceph-mon[32317]: pgmap v355: 97 pgs: 97 active+clean; 4.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:37:36.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:37:36 smithi196 ceph-mon[32919]: pgmap v355: 97 pgs: 97 active+clean; 4.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:37:36.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:37:36 smithi097 ceph-mon[29197]: pgmap v355: 97 pgs: 97 active+clean; 4.2 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:37:37.087 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.086+0000 7fcf52405640 1 -- 172.21.15.97:0/2026003905 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf4c10e930 msgr2=0x7fcf4c110d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:37:37.087 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.086+0000 7fcf52405640 1 --2- 172.21.15.97:0/2026003905 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf4c10e930 0x7fcf4c110d20 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fcf44009ca0 tx=0x7fcf4402a8c0 comp rx=0 tx=0).stop 2024-04-23T10:37:37.087 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.086+0000 7fcf52405640 1 -- 172.21.15.97:0/2026003905 shutdown_connections 2024-04-23T10:37:37.088 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.086+0000 7fcf52405640 1 --2- 172.21.15.97:0/2026003905 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf4c10e930 0x7fcf4c110d20 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:37.088 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.086+0000 7fcf52405640 1 --2- 172.21.15.97:0/2026003905 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf4c1016a0 0x7fcf4c10e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:37.088 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.086+0000 7fcf52405640 1 --2- 172.21.15.97:0/2026003905 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf4c100d80 0x7fcf4c101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:37.088 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.086+0000 7fcf52405640 1 -- 172.21.15.97:0/2026003905 >> 172.21.15.97:0/2026003905 conn(0x7fcf4c077fd0 msgr2=0x7fcf4c0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:37:37.088 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.086+0000 7fcf52405640 1 -- 172.21.15.97:0/2026003905 shutdown_connections 2024-04-23T10:37:37.088 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.086+0000 7fcf52405640 1 -- 172.21.15.97:0/2026003905 wait complete. 2024-04-23T10:37:37.088 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.086+0000 7fcf52405640 1 Processor -- start 2024-04-23T10:37:37.088 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.086+0000 7fcf52405640 1 -- start start 2024-04-23T10:37:37.088 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.087+0000 7fcf52405640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf4c100d80 0x7fcf4c1a0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:37.088 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.087+0000 7fcf52405640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf4c1016a0 0x7fcf4c1a0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:37.088 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.087+0000 7fcf52405640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf4c10e930 0x7fcf4c19a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:37.088 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.087+0000 7fcf52405640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4c114390 con 0x7fcf4c100d80 2024-04-23T10:37:37.088 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.087+0000 7fcf52405640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4c114210 con 0x7fcf4c1016a0 2024-04-23T10:37:37.089 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.087+0000 7fcf52405640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4c114510 con 0x7fcf4c10e930 2024-04-23T10:37:37.089 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.087+0000 7fcf4bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf4c100d80 0x7fcf4c1a0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:37.089 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.087+0000 7fcf4bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf4c100d80 0x7fcf4c1a0440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41252/0 (socket says 172.21.15.97:41252) 2024-04-23T10:37:37.089 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.087+0000 7fcf4bfff640 1 -- 172.21.15.97:0/1776926715 learned_addr learned my addr 172.21.15.97:0/1776926715 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:37:37.089 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.087+0000 7fcf4b7fe640 1 --2- 172.21.15.97:0/1776926715 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf4c1016a0 0x7fcf4c1a0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:37.089 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.087+0000 7fcf509a2640 1 --2- 172.21.15.97:0/1776926715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf4c10e930 0x7fcf4c19a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:37.089 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.087+0000 7fcf4bfff640 1 -- 172.21.15.97:0/1776926715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf4c10e930 msgr2=0x7fcf4c19a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:37:37.089 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.087+0000 7fcf4bfff640 1 --2- 172.21.15.97:0/1776926715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf4c10e930 0x7fcf4c19a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:37.089 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.087+0000 7fcf4bfff640 1 -- 172.21.15.97:0/1776926715 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf4c1016a0 msgr2=0x7fcf4c1a0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:37:37.089 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.087+0000 7fcf4bfff640 1 --2- 172.21.15.97:0/1776926715 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf4c1016a0 0x7fcf4c1a0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:37.090 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.087+0000 7fcf4bfff640 1 -- 172.21.15.97:0/1776926715 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf4c19ad70 con 0x7fcf4c100d80 2024-04-23T10:37:37.090 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.088+0000 7fcf4bfff640 1 --2- 172.21.15.97:0/1776926715 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf4c100d80 0x7fcf4c1a0440 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fcf3400c920 tx=0x7fcf3400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:37:37.090 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.088+0000 7fcf497fa640 1 -- 172.21.15.97:0/1776926715 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf34007c70 con 0x7fcf4c100d80 2024-04-23T10:37:37.090 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.088+0000 7fcf497fa640 1 -- 172.21.15.97:0/1776926715 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf34007e10 con 0x7fcf4c100d80 2024-04-23T10:37:37.090 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.088+0000 7fcf497fa640 1 -- 172.21.15.97:0/1776926715 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf34018820 con 0x7fcf4c100d80 2024-04-23T10:37:37.090 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.088+0000 7fcf52405640 1 -- 172.21.15.97:0/1776926715 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf4c19b060 con 0x7fcf4c100d80 2024-04-23T10:37:37.090 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.088+0000 7fcf52405640 1 -- 172.21.15.97:0/1776926715 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf4c1a7300 con 0x7fcf4c100d80 2024-04-23T10:37:37.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.090+0000 7fcf497fa640 1 -- 172.21.15.97:0/1776926715 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fcf3400f450 con 0x7fcf4c100d80 2024-04-23T10:37:37.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.090+0000 7fcf551fa740 1 Processor -- start 2024-04-23T10:37:37.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.090+0000 7fcf551fa740 1 -- start start 2024-04-23T10:37:37.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.091+0000 7fcf551fa740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e501814620 0x55e501814a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:37.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.091+0000 7fcf551fa740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e501814f40 0x55e50181f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:37.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.091+0000 7fcf551fa740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e50181f800 0x55e501821bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:37.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.091+0000 7fcf551fa740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e501779970 con 0x55e50181f800 2024-04-23T10:37:37.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.091+0000 7fcf551fa740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e5017164a0 con 0x55e501814f40 2024-04-23T10:37:37.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.091+0000 7fcf551fa740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e501775930 con 0x55e501814620 2024-04-23T10:37:37.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.091+0000 7fcf497fa640 1 --2- 172.21.15.97:0/1776926715 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fcf380a3dc0 0x7fcf380a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:37.098 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.091+0000 7fcf4b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e50181f800 0x55e501821bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.091+0000 7fcf4b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e50181f800 0x55e501821bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41256/0 (socket says 172.21.15.97:41256) 2024-04-23T10:37:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.091+0000 7fcf4b7fe640 1 -- 172.21.15.97:0/2890346395 learned_addr learned my addr 172.21.15.97:0/2890346395 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:37:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.091+0000 7fcf509a2640 1 --2- 172.21.15.97:0/1776926715 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fcf380a3dc0 0x7fcf380a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.091+0000 7fcf4bfff640 1 --2- 172.21.15.97:0/2890346395 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e501814f40 0x55e50181f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.091+0000 7fcf509a2640 1 --2- 172.21.15.97:0/2890346395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e501814620 0x55e501814a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.091+0000 7fcf497fa640 1 -- 172.21.15.97:0/1776926715 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8484+0+0 (secure 0 0 0) 0x7fcf340d0a10 con 0x7fcf4c100d80 2024-04-23T10:37:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.091+0000 7fcf4b7fe640 1 -- 172.21.15.97:0/2890346395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e501814620 msgr2=0x55e501814a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:37:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.091+0000 7fcf4b7fe640 1 --2- 172.21.15.97:0/2890346395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e501814620 0x55e501814a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.091+0000 7fcf4b7fe640 1 -- 172.21.15.97:0/2890346395 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e501814f40 msgr2=0x55e50181f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:37:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.091+0000 7fcf4b7fe640 1 --2- 172.21.15.97:0/2890346395 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e501814f40 0x55e50181f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.091+0000 7fcf4b7fe640 1 -- 172.21.15.97:0/2890346395 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e50148c210 con 0x55e50181f800 2024-04-23T10:37:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.091+0000 7fcf4bfff640 1 --2- 172.21.15.97:0/2890346395 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e501814f40 0x55e50181f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:37:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.091+0000 7fcf509a2640 1 --2- 172.21.15.97:0/1776926715 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fcf380a3dc0 0x7fcf380a6280 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fcf44005fc0 tx=0x7fcf44005c30 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:37:37.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.091+0000 7fcf4b7fe640 1 --2- 172.21.15.97:0/2890346395 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e50181f800 0x55e501821bf0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c00bbe0 tx=0x7fcf3c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9ffbec90291c343 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:37:37.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.092+0000 7fcf427fc640 1 -- 172.21.15.97:0/2890346395 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c015070 con 0x55e50181f800 2024-04-23T10:37:37.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.092+0000 7fcf427fc640 1 -- 172.21.15.97:0/2890346395 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf3c0118f0 con 0x55e50181f800 2024-04-23T10:37:37.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.092+0000 7fcf427fc640 1 -- 172.21.15.97:0/2890346395 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c010790 con 0x55e50181f800 2024-04-23T10:37:37.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.092+0000 7fcf551fa740 1 -- 172.21.15.97:0/2890346395 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e50181f800 msgr2=0x55e501821bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:37:37.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.092+0000 7fcf551fa740 1 --2- 172.21.15.97:0/2890346395 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e50181f800 0x55e501821bf0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c00bbe0 tx=0x7fcf3c00f730 comp rx=0 tx=0).stop 2024-04-23T10:37:37.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.092+0000 7fcf551fa740 1 -- 172.21.15.97:0/2890346395 shutdown_connections 2024-04-23T10:37:37.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.092+0000 7fcf551fa740 1 --2- 172.21.15.97:0/2890346395 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e50181f800 0x55e501821bf0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:37.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.092+0000 7fcf551fa740 1 --2- 172.21.15.97:0/2890346395 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e501814f40 0x55e50181f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:37.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.092+0000 7fcf551fa740 1 --2- 172.21.15.97:0/2890346395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e501814620 0x55e501814a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:37.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.092+0000 7fcf551fa740 1 -- 172.21.15.97:0/2890346395 >> 172.21.15.97:0/2890346395 conn(0x55e501810b30 msgr2=0x55e501810f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:37:37.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.092+0000 7fcf551fa740 1 -- 172.21.15.97:0/2890346395 shutdown_connections 2024-04-23T10:37:37.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.092+0000 7fcf551fa740 1 -- 172.21.15.97:0/2890346395 wait complete. 2024-04-23T10:37:37.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.093+0000 7fcf551fa740 1 -- start start 2024-04-23T10:37:37.100 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.093+0000 7fcf551fa740 1 Processor -- start 2024-04-23T10:37:37.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.093+0000 7fcf551fa740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e501814620 0x55e50181f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:37.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.093+0000 7fcf551fa740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e501814f40 0x55e50181c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:37.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.093+0000 7fcf551fa740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e50181f800 0x55e50181c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:37.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.093+0000 7fcf551fa740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e501822430 con 0x55e501814f40 2024-04-23T10:37:37.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.093+0000 7fcf551fa740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e5017140b0 con 0x55e501814620 2024-04-23T10:37:37.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.093+0000 7fcf551fa740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e5018225b0 con 0x55e50181f800 2024-04-23T10:37:37.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.093+0000 7fcf551fa740 10 client.0 _ll_register_callbacks cb 0x55e501701d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:37:37.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.093+0000 7fcf4b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e501814f40 0x55e50181c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:37.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.093+0000 7fcf4b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e501814f40 0x55e50181c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41258/0 (socket says 172.21.15.97:41258) 2024-04-23T10:37:37.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.093+0000 7fcf4b7fe640 1 -- 172.21.15.97:0/3652225575 learned_addr learned my addr 172.21.15.97:0/3652225575 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:37:37.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.093+0000 7fcf4bfff640 1 --2- 172.21.15.97:0/3652225575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e501814620 0x55e50181f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:37.101 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.094+0000 7fcf509a2640 1 --2- 172.21.15.97:0/3652225575 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e50181f800 0x55e50181c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:37.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.094+0000 7fcf4b7fe640 1 -- 172.21.15.97:0/3652225575 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e50181f800 msgr2=0x55e50181c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:37:37.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.094+0000 7fcf4b7fe640 1 --2- 172.21.15.97:0/3652225575 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e50181f800 0x55e50181c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:37.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.094+0000 7fcf4b7fe640 1 -- 172.21.15.97:0/3652225575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e501814620 msgr2=0x55e50181f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:37:37.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.094+0000 7fcf4b7fe640 1 --2- 172.21.15.97:0/3652225575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e501814620 0x55e50181f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:37.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.094+0000 7fcf4b7fe640 1 -- 172.21.15.97:0/3652225575 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e501558620 con 0x55e501814f40 2024-04-23T10:37:37.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.094+0000 7fcf4bfff640 1 --2- 172.21.15.97:0/3652225575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e501814620 0x55e50181f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:37:37.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.094+0000 7fcf4b7fe640 1 --2- 172.21.15.97:0/3652225575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e501814f40 0x55e50181c440 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c00b950 tx=0x7fcf3c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a5c6402e5e2ce8f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:37:37.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.094+0000 7fcf2b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:37:37.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.094+0000 7fcf2b7fe640 1 -- 172.21.15.97:0/3652225575 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c015050 con 0x55e501814f40 2024-04-23T10:37:37.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.094+0000 7fcf2b7fe640 1 -- 172.21.15.97:0/3652225575 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8484+0+0 (secure 0 0 0) 0x7fcf3c011aa0 con 0x55e501814f40 2024-04-23T10:37:37.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.094+0000 7fcf551fa740 1 -- 172.21.15.97:0/3652225575 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf3c0118f0 con 0x55e501814f40 2024-04-23T10:37:37.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.094+0000 7fcf2b7fe640 1 -- 172.21.15.97:0/3652225575 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcf3c011ca0 con 0x55e501814f40 2024-04-23T10:37:37.102 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.094+0000 7fcf551fa740 10 client.17193 fetch_fsmap learned FSMap version 26 2024-04-23T10:37:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.094+0000 7fcf551fa740 1 -- 172.21.15.97:0/3652225575 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e50182e360 con 0x55e501814f40 2024-04-23T10:37:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.094+0000 7fcf2b7fe640 1 -- 172.21.15.97:0/3652225575 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf3c0118f0 con 0x55e501814f40 2024-04-23T10:37:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.094+0000 7fcf2b7fe640 1 -- 172.21.15.97:0/3652225575 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fcf3c02a030 con 0x55e501814f40 2024-04-23T10:37:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.095+0000 7fcf551fa740 10 client.17193 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:37:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.095+0000 7fcf551fa740 10 client.17193 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:37:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.095+0000 7fcf551fa740 20 client.17193 populate_metadata read hostname 'smithi097' 2024-04-23T10:37:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.095+0000 7fcf551fa740 1 --2- 172.21.15.97:0/3652225575 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55e50182f770 0x55e501831b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.095+0000 7fcf551fa740 4 client.17193 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:37:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.095+0000 7fcf551fa740 1 -- 172.21.15.97:0/3652225575 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e501787390 con 0x55e50182f770 2024-04-23T10:37:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.095+0000 7fcf4bfff640 1 --2- 172.21.15.97:0/3652225575 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55e50182f770 0x55e501831b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.095+0000 7fcf2b7fe640 10 client.17193 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:37:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.095+0000 7fcf4bfff640 1 --2- 172.21.15.97:0/3652225575 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55e50182f770 0x55e501831b60 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e7e0512cd079804 server_cookie=aceaaa90911ffdd4 in_seq=0 out_seq=0 2024-04-23T10:37:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.096+0000 7fcf2b7fe640 1 -- 172.21.15.97:0/3652225575 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e501787390 con 0x55e50182f770 2024-04-23T10:37:37.103 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.096+0000 7fcf2b7fe640 10 client.17193 handle_command_reply: tid=0 2024-04-23T10:37:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.096+0000 7fcf551fa740 1 client.17193 shutdown 2024-04-23T10:37:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.096+0000 7fcf551fa740 1 -- 172.21.15.97:0/3652225575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e501814f40 msgr2=0x55e50181c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:37:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.096+0000 7fcf551fa740 1 --2- 172.21.15.97:0/3652225575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e501814f40 0x55e50181c440 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c00b950 tx=0x7fcf3c00bea0 comp rx=0 tx=0).stop 2024-04-23T10:37:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.096+0000 7fcf551fa740 1 -- 172.21.15.97:0/3652225575 shutdown_connections 2024-04-23T10:37:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.096+0000 7fcf551fa740 1 --2- 172.21.15.97:0/3652225575 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55e50182f770 0x55e501831b60 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.096+0000 7fcf551fa740 1 --2- 172.21.15.97:0/3652225575 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e50181f800 0x55e50181c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.096+0000 7fcf551fa740 1 --2- 172.21.15.97:0/3652225575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e501814f40 0x55e50181c440 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.096+0000 7fcf551fa740 1 --2- 172.21.15.97:0/3652225575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e501814620 0x55e50181f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.096+0000 7fcf2b7fe640 0 client.17193 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:37:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.096+0000 7fcf551fa740 1 -- 172.21.15.97:0/3652225575 >> 172.21.15.97:0/3652225575 conn(0x55e501810b30 msgr2=0x55e50180fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:37:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.096+0000 7fcf551fa740 1 -- 172.21.15.97:0/3652225575 shutdown_connections 2024-04-23T10:37:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.096+0000 7fcf551fa740 1 -- 172.21.15.97:0/3652225575 wait complete. 2024-04-23T10:37:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.096+0000 7fcf551fa740 20 client.17193 trim_cache size 0 max 16384 2024-04-23T10:37:37.115 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.111+0000 7fcf551fa740 1 Processor -- start 2024-04-23T10:37:37.115 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.111+0000 7fcf551fa740 1 -- start start 2024-04-23T10:37:37.115 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.111+0000 7fcf551fa740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e501814620 0x55e50182b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:37.115 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.111+0000 7fcf551fa740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e501814f40 0x55e50180fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:37.115 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.111+0000 7fcf551fa740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e50181f800 0x55e501837190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:37.115 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.111+0000 7fcf551fa740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4c1afc20 con 0x55e501814f40 2024-04-23T10:37:37.115 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.111+0000 7fcf551fa740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4c1b1760 con 0x55e501814620 2024-04-23T10:37:37.115 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.111+0000 7fcf551fa740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4c1b02f0 con 0x55e50181f800 2024-04-23T10:37:37.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.111+0000 7fcf4b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e501814f40 0x55e50180fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:37.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.111+0000 7fcf4b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e501814f40 0x55e50180fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41266/0 (socket says 172.21.15.97:41266) 2024-04-23T10:37:37.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.111+0000 7fcf4b7fe640 1 -- 172.21.15.97:0/2801165228 learned_addr learned my addr 172.21.15.97:0/2801165228 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:37:37.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.112+0000 7fcf509a2640 1 --2- 172.21.15.97:0/2801165228 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e50181f800 0x55e501837190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:37.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.112+0000 7fcf4b7fe640 1 -- 172.21.15.97:0/2801165228 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e50181f800 msgr2=0x55e501837190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:37:37.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.112+0000 7fcf4b7fe640 1 --2- 172.21.15.97:0/2801165228 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e50181f800 0x55e501837190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:37.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.112+0000 7fcf4b7fe640 1 -- 172.21.15.97:0/2801165228 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e501814620 msgr2=0x55e50182b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:37:37.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.112+0000 7fcf4b7fe640 1 --2- 172.21.15.97:0/2801165228 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e501814620 0x55e50182b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:37.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.112+0000 7fcf4b7fe640 1 -- 172.21.15.97:0/2801165228 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e50181d2c0 con 0x55e501814f40 2024-04-23T10:37:37.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.112+0000 7fcf4b7fe640 1 --2- 172.21.15.97:0/2801165228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e501814f40 0x55e50180fb30 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c00fc30 tx=0x7fcf3c00ba60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4533bcfd7aaba75c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:37:37.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.112+0000 7fcf41ffb640 1 -- 172.21.15.97:0/2801165228 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c015070 con 0x55e501814f40 2024-04-23T10:37:37.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.112+0000 7fcf41ffb640 1 -- 172.21.15.97:0/2801165228 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf3c012e50 con 0x55e501814f40 2024-04-23T10:37:37.116 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.112+0000 7fcf41ffb640 1 -- 172.21.15.97:0/2801165228 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c004b70 con 0x55e501814f40 2024-04-23T10:37:37.117 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.113+0000 7fcf551fa740 1 -- 172.21.15.97:0/2801165228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e501814f40 msgr2=0x55e50180fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:37:37.117 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.113+0000 7fcf551fa740 1 --2- 172.21.15.97:0/2801165228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e501814f40 0x55e50180fb30 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c00fc30 tx=0x7fcf3c00ba60 comp rx=0 tx=0).stop 2024-04-23T10:37:37.117 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.113+0000 7fcf551fa740 1 -- 172.21.15.97:0/2801165228 shutdown_connections 2024-04-23T10:37:37.117 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.113+0000 7fcf551fa740 1 --2- 172.21.15.97:0/2801165228 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e50181f800 0x55e501837190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:37.117 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.113+0000 7fcf551fa740 1 --2- 172.21.15.97:0/2801165228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e501814f40 0x55e50180fb30 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:37.117 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.113+0000 7fcf551fa740 1 --2- 172.21.15.97:0/2801165228 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e501814620 0x55e50182b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:37.117 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.113+0000 7fcf551fa740 1 -- 172.21.15.97:0/2801165228 >> 172.21.15.97:0/2801165228 conn(0x55e501810b30 msgr2=0x55e50180f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:37:37.117 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.113+0000 7fcf551fa740 1 -- 172.21.15.97:0/2801165228 shutdown_connections 2024-04-23T10:37:37.117 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.113+0000 7fcf551fa740 1 -- 172.21.15.97:0/2801165228 wait complete. 2024-04-23T10:37:37.117 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.113+0000 7fcf551fa740 1 -- start start 2024-04-23T10:37:37.117 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.113+0000 7fcf551fa740 1 Processor -- start 2024-04-23T10:37:37.117 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.113+0000 7fcf551fa740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e501814620 0x55e501825250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:37.117 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.114+0000 7fcf551fa740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e501814f40 0x55e501825790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:37.118 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.114+0000 7fcf551fa740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e50181f800 0x55e501825cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:37.118 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.114+0000 7fcf551fa740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4c1b1090 con 0x55e50181f800 2024-04-23T10:37:37.118 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.114+0000 7fcf551fa740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4c1b09c0 con 0x55e501814f40 2024-04-23T10:37:37.118 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.114+0000 7fcf551fa740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4c1adc20 con 0x55e501814620 2024-04-23T10:37:37.118 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.114+0000 7fcf551fa740 10 client.0 _ll_register_callbacks cb 0x7fcf3c00bd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:37:37.118 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.114+0000 7fcf509a2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e50181f800 0x55e501825cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:37.118 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.114+0000 7fcf509a2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e50181f800 0x55e501825cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41282/0 (socket says 172.21.15.97:41282) 2024-04-23T10:37:37.118 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.114+0000 7fcf509a2640 1 -- 172.21.15.97:0/671743985 learned_addr learned my addr 172.21.15.97:0/671743985 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:37:37.118 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.114+0000 7fcf4bfff640 1 --2- 172.21.15.97:0/671743985 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e501814620 0x55e501825250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:37.118 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.114+0000 7fcf4b7fe640 1 --2- 172.21.15.97:0/671743985 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e501814f40 0x55e501825790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:37.118 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.114+0000 7fcf509a2640 1 -- 172.21.15.97:0/671743985 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e501814620 msgr2=0x55e501825250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:37:37.118 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.114+0000 7fcf509a2640 1 --2- 172.21.15.97:0/671743985 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e501814620 0x55e501825250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:37.118 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.114+0000 7fcf509a2640 1 -- 172.21.15.97:0/671743985 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e501814f40 msgr2=0x55e501825790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:37:37.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.114+0000 7fcf509a2640 1 --2- 172.21.15.97:0/671743985 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e501814f40 0x55e501825790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:37.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.114+0000 7fcf509a2640 1 -- 172.21.15.97:0/671743985 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e501894a40 con 0x55e50181f800 2024-04-23T10:37:37.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.114+0000 7fcf509a2640 1 --2- 172.21.15.97:0/671743985 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e50181f800 0x55e501825cd0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fcf44009ca0 tx=0x7fcf4402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3388144987e1fea server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:37:37.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.114+0000 7fcf2b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:37:37.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.114+0000 7fcf2b7fe640 1 -- 172.21.15.97:0/671743985 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf44003c40 con 0x55e50181f800 2024-04-23T10:37:37.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.114+0000 7fcf2b7fe640 1 -- 172.21.15.97:0/671743985 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8484+0+0 (secure 0 0 0) 0x7fcf44003de0 con 0x55e50181f800 2024-04-23T10:37:37.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.114+0000 7fcf551fa740 1 -- 172.21.15.97:0/671743985 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf3c012e50 con 0x55e50181f800 2024-04-23T10:37:37.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.114+0000 7fcf2b7fe640 1 -- 172.21.15.97:0/671743985 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcf44004040 con 0x55e50181f800 2024-04-23T10:37:37.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.114+0000 7fcf551fa740 10 client.17199 fetch_fsmap learned FSMap version 26 2024-04-23T10:37:37.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.114+0000 7fcf551fa740 1 -- 172.21.15.97:0/671743985 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e501895d10 con 0x55e50181f800 2024-04-23T10:37:37.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.115+0000 7fcf2b7fe640 1 -- 172.21.15.97:0/671743985 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf3c012e50 con 0x55e50181f800 2024-04-23T10:37:37.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.115+0000 7fcf2b7fe640 1 -- 172.21.15.97:0/671743985 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fcf4405e020 con 0x55e50181f800 2024-04-23T10:37:37.119 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.115+0000 7fcf551fa740 10 client.17199 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:37:37.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.115+0000 7fcf551fa740 10 client.17199 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:37:37.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.115+0000 7fcf551fa740 20 client.17199 populate_metadata read hostname 'smithi097' 2024-04-23T10:37:37.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.115+0000 7fcf551fa740 1 --2- 172.21.15.97:0/671743985 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55e501896440 0x55e501898860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:37.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.115+0000 7fcf551fa740 4 client.17199 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:37:37.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.115+0000 7fcf551fa740 1 -- 172.21.15.97:0/671743985 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e501781200 con 0x55e501896440 2024-04-23T10:37:37.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.115+0000 7fcf4bfff640 1 --2- 172.21.15.97:0/671743985 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55e501896440 0x55e501898860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:37.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.115+0000 7fcf2b7fe640 10 client.17199 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:37:37.120 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.115+0000 7fcf4bfff640 1 --2- 172.21.15.97:0/671743985 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55e501896440 0x55e501898860 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61f67aec7227f8f2 server_cookie=e055feca5a40bca3 in_seq=0 out_seq=0 2024-04-23T10:37:37.129 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.130+0000 7fcf2b7fe640 1 -- 172.21.15.97:0/671743985 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e501781200 con 0x55e501896440 2024-04-23T10:37:37.129 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.130+0000 7fcf2b7fe640 10 client.17199 handle_command_reply: tid=0 2024-04-23T10:37:37.129 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.130+0000 7fcf551fa740 1 client.17199 shutdown 2024-04-23T10:37:37.129 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.130+0000 7fcf551fa740 1 -- 172.21.15.97:0/671743985 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e50181f800 msgr2=0x55e501825cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:37:37.129 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.130+0000 7fcf551fa740 1 --2- 172.21.15.97:0/671743985 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e50181f800 0x55e501825cd0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fcf44009ca0 tx=0x7fcf4402a8c0 comp rx=0 tx=0).stop 2024-04-23T10:37:37.129 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.130+0000 7fcf551fa740 1 -- 172.21.15.97:0/671743985 shutdown_connections 2024-04-23T10:37:37.129 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.130+0000 7fcf551fa740 1 --2- 172.21.15.97:0/671743985 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55e501896440 0x55e501898860 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:37.130 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.130+0000 7fcf551fa740 1 --2- 172.21.15.97:0/671743985 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e50181f800 0x55e501825cd0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:37.130 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.130+0000 7fcf2b7fe640 0 client.17199 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:37:37.130 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.130+0000 7fcf551fa740 1 --2- 172.21.15.97:0/671743985 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e501814f40 0x55e501825790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:37.130 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.130+0000 7fcf551fa740 1 --2- 172.21.15.97:0/671743985 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e501814620 0x55e501825250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:37.130 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.130+0000 7fcf551fa740 1 -- 172.21.15.97:0/671743985 >> 172.21.15.97:0/671743985 conn(0x55e501810b30 msgr2=0x55e50180fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:37:37.130 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.130+0000 7fcf551fa740 1 -- 172.21.15.97:0/671743985 shutdown_connections 2024-04-23T10:37:37.130 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.130+0000 7fcf551fa740 1 -- 172.21.15.97:0/671743985 wait complete. 2024-04-23T10:37:37.130 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.130+0000 7fcf551fa740 20 client.17199 trim_cache size 0 max 16384 2024-04-23T10:37:37.130 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:37:37.131 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:37:37.131 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:37:37.131 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:37:37.131 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.131+0000 7fcf52405640 1 -- 172.21.15.97:0/1776926715 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fcf380a3dc0 msgr2=0x7fcf380a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:37:37.132 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.131+0000 7fcf52405640 1 --2- 172.21.15.97:0/1776926715 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fcf380a3dc0 0x7fcf380a6280 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fcf44005fc0 tx=0x7fcf44005c30 comp rx=0 tx=0).stop 2024-04-23T10:37:37.132 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.131+0000 7fcf52405640 1 -- 172.21.15.97:0/1776926715 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf4c100d80 msgr2=0x7fcf4c1a0440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:37:37.132 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.131+0000 7fcf52405640 1 --2- 172.21.15.97:0/1776926715 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf4c100d80 0x7fcf4c1a0440 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fcf3400c920 tx=0x7fcf3400cde0 comp rx=0 tx=0).stop 2024-04-23T10:37:37.132 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.131+0000 7fcf52405640 1 -- 172.21.15.97:0/1776926715 shutdown_connections 2024-04-23T10:37:37.132 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.131+0000 7fcf52405640 1 --2- 172.21.15.97:0/1776926715 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fcf380a3dc0 0x7fcf380a6280 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:37.132 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.131+0000 7fcf52405640 1 --2- 172.21.15.97:0/1776926715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf4c10e930 0x7fcf4c19a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:37.132 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.131+0000 7fcf52405640 1 --2- 172.21.15.97:0/1776926715 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf4c1016a0 0x7fcf4c1a0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:37.132 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.131+0000 7fcf52405640 1 --2- 172.21.15.97:0/1776926715 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf4c100d80 0x7fcf4c1a0440 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:37.132 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.131+0000 7fcf52405640 1 -- 172.21.15.97:0/1776926715 >> 172.21.15.97:0/1776926715 conn(0x7fcf4c077fd0 msgr2=0x7fcf4c10fca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:37:37.132 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.131+0000 7fcf52405640 1 -- 172.21.15.97:0/1776926715 shutdown_connections 2024-04-23T10:37:37.132 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:37.131+0000 7fcf52405640 1 -- 172.21.15.97:0/1776926715 wait complete. 2024-04-23T10:37:38.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:37:38 smithi183 ceph-mon[32317]: pgmap v356: 97 pgs: 97 active+clean; 4.5 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 65 KiB/s wr, 24 op/s 2024-04-23T10:37:38.662 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:37:38 smithi097 ceph-mon[29197]: pgmap v356: 97 pgs: 97 active+clean; 4.5 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 65 KiB/s wr, 24 op/s 2024-04-23T10:37:38.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:37:38 smithi196 ceph-mon[32919]: pgmap v356: 97 pgs: 97 active+clean; 4.5 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 65 KiB/s wr, 24 op/s 2024-04-23T10:37:40.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:37:40 smithi183 ceph-mon[32317]: pgmap v357: 97 pgs: 97 active+clean; 4.5 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:37:40.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:37:40 smithi196 ceph-mon[32919]: pgmap v357: 97 pgs: 97 active+clean; 4.5 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:37:40.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:37:40 smithi097 ceph-mon[29197]: pgmap v357: 97 pgs: 97 active+clean; 4.5 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:37:42.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:37:42 smithi183 ceph-mon[32317]: pgmap v358: 97 pgs: 97 active+clean; 4.5 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:37:42.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:37:42 smithi196 ceph-mon[32919]: pgmap v358: 97 pgs: 97 active+clean; 4.5 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:37:42.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:37:42 smithi097 ceph-mon[29197]: pgmap v358: 97 pgs: 97 active+clean; 4.5 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:37:44.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:37:44 smithi183 ceph-mon[32317]: pgmap v359: 97 pgs: 97 active+clean; 4.6 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-23T10:37:44.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:37:44 smithi196 ceph-mon[32919]: pgmap v359: 97 pgs: 97 active+clean; 4.6 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-23T10:37:44.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:37:44 smithi097 ceph-mon[29197]: pgmap v359: 97 pgs: 97 active+clean; 4.6 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-23T10:37:46.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:37:46 smithi183 ceph-mon[32317]: pgmap v360: 97 pgs: 97 active+clean; 4.6 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-23T10:37:46.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:37:46 smithi196 ceph-mon[32919]: pgmap v360: 97 pgs: 97 active+clean; 4.6 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-23T10:37:46.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:37:46 smithi097 ceph-mon[29197]: pgmap v360: 97 pgs: 97 active+clean; 4.6 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-23T10:37:48.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:37:48 smithi183 ceph-mon[32317]: pgmap v361: 97 pgs: 97 active+clean; 4.8 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-23T10:37:48.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:37:48 smithi196 ceph-mon[32919]: pgmap v361: 97 pgs: 97 active+clean; 4.8 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-23T10:37:48.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:37:48 smithi097 ceph-mon[29197]: pgmap v361: 97 pgs: 97 active+clean; 4.8 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-23T10:37:50.229 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.226+0000 7f351f507640 1 -- 172.21.15.97:0/2358865504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3518101aa0 msgr2=0x7f3518105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:37:50.229 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.226+0000 7f351f507640 1 --2- 172.21.15.97:0/2358865504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3518101aa0 0x7f3518105a90 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f3508009900 tx=0x7f350802a410 comp rx=0 tx=0).stop 2024-04-23T10:37:50.229 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.226+0000 7f351f507640 1 -- 172.21.15.97:0/2358865504 shutdown_connections 2024-04-23T10:37:50.229 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.226+0000 7f351f507640 1 --2- 172.21.15.97:0/2358865504 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3518106170 0x7f351810dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:50.229 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.226+0000 7f351f507640 1 --2- 172.21.15.97:0/2358865504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3518101aa0 0x7f3518105a90 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:50.229 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.226+0000 7f351f507640 1 --2- 172.21.15.97:0/2358865504 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35181010f0 0x7f35181014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:50.229 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.226+0000 7f351f507640 1 -- 172.21.15.97:0/2358865504 >> 172.21.15.97:0/2358865504 conn(0x7f3518077fd0 msgr2=0x7f35180783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:37:50.229 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.226+0000 7f351f507640 1 -- 172.21.15.97:0/2358865504 shutdown_connections 2024-04-23T10:37:50.230 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.226+0000 7f351f507640 1 -- 172.21.15.97:0/2358865504 wait complete. 2024-04-23T10:37:50.230 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.227+0000 7f351f507640 1 Processor -- start 2024-04-23T10:37:50.230 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.227+0000 7f351f507640 1 -- start start 2024-04-23T10:37:50.230 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.227+0000 7f351f507640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35181010f0 0x7f3518198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:50.230 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.227+0000 7f351f507640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3518106170 0x7f3518198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:50.230 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.227+0000 7f351f507640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f351819cf50 0x7f351819d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:50.230 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.227+0000 7f351f507640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f35181121f0 con 0x7f3518106170 2024-04-23T10:37:50.230 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.227+0000 7f351f507640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3518112070 con 0x7f351819cf50 2024-04-23T10:37:50.230 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.227+0000 7f351f507640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3518112370 con 0x7f35181010f0 2024-04-23T10:37:50.230 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.227+0000 7f351caa2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3518106170 0x7f3518198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:50.230 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.227+0000 7f351caa2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3518106170 0x7f3518198bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38394/0 (socket says 172.21.15.97:38394) 2024-04-23T10:37:50.231 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.227+0000 7f351caa2640 1 -- 172.21.15.97:0/566019886 learned_addr learned my addr 172.21.15.97:0/566019886 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:37:50.231 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.227+0000 7f351d2a3640 1 --2- 172.21.15.97:0/566019886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35181010f0 0x7f3518198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:50.231 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.227+0000 7f351daa4640 1 --2- 172.21.15.97:0/566019886 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f351819cf50 0x7f351819d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:50.231 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.228+0000 7f351caa2640 1 -- 172.21.15.97:0/566019886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35181010f0 msgr2=0x7f3518198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:37:50.231 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.228+0000 7f351caa2640 1 --2- 172.21.15.97:0/566019886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35181010f0 0x7f3518198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:50.231 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.228+0000 7f351caa2640 1 -- 172.21.15.97:0/566019886 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f351819cf50 msgr2=0x7f351819d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:37:50.231 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.228+0000 7f351caa2640 1 --2- 172.21.15.97:0/566019886 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f351819cf50 0x7f351819d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:50.231 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.228+0000 7f351caa2640 1 -- 172.21.15.97:0/566019886 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f351819da40 con 0x7f3518106170 2024-04-23T10:37:50.231 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.228+0000 7f351caa2640 1 --2- 172.21.15.97:0/566019886 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3518106170 0x7f3518198bd0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f3508006010 tx=0x7f3508004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:37:50.231 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.228+0000 7f350e7fc640 1 -- 172.21.15.97:0/566019886 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f350803f030 con 0x7f3518106170 2024-04-23T10:37:50.232 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.228+0000 7f351f507640 1 -- 172.21.15.97:0/566019886 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f351819dcd0 con 0x7f3518106170 2024-04-23T10:37:50.232 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.228+0000 7f350e7fc640 1 -- 172.21.15.97:0/566019886 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3508003aa0 con 0x7f3518106170 2024-04-23T10:37:50.232 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.228+0000 7f350e7fc640 1 -- 172.21.15.97:0/566019886 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3508035070 con 0x7f3518106170 2024-04-23T10:37:50.232 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.228+0000 7f351f507640 1 -- 172.21.15.97:0/566019886 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35181a5490 con 0x7f3518106170 2024-04-23T10:37:50.240 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.230+0000 7f350e7fc640 1 -- 172.21.15.97:0/566019886 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3508042030 con 0x7f3518106170 2024-04-23T10:37:50.240 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.231+0000 7f3522216740 1 Processor -- start 2024-04-23T10:37:50.240 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.231+0000 7f3522216740 1 -- start start 2024-04-23T10:37:50.240 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.231+0000 7f3522216740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a176c30620 0x55a176c30a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:50.240 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.231+0000 7f3522216740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a176c30f40 0x55a176c3b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:50.240 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.231+0000 7f3522216740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a176c3b800 0x55a176c3dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:50.240 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.231+0000 7f3522216740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55a176b95970 con 0x55a176c30620 2024-04-23T10:37:50.241 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.231+0000 7f3522216740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a176b324a0 con 0x55a176c30f40 2024-04-23T10:37:50.241 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.231+0000 7f3522216740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a176b91930 con 0x55a176c3b800 2024-04-23T10:37:50.241 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.231+0000 7f350e7fc640 1 --2- 172.21.15.97:0/566019886 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f35040a3dc0 0x7f35040a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:50.241 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.231+0000 7f351daa4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a176c30620 0x55a176c30a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:50.241 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.231+0000 7f351daa4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a176c30620 0x55a176c30a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38410/0 (socket says 172.21.15.97:38410) 2024-04-23T10:37:50.241 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.231+0000 7f351daa4640 1 -- 172.21.15.97:0/1234241134 learned_addr learned my addr 172.21.15.97:0/1234241134 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:37:50.241 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.232+0000 7f351daa4640 1 --2- 172.21.15.97:0/566019886 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f35040a3dc0 0x7f35040a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:50.241 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.232+0000 7f351d2a3640 1 --2- 172.21.15.97:0/1234241134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a176c30f40 0x55a176c3b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:50.241 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.232+0000 7f350e7fc640 1 -- 172.21.15.97:0/566019886 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8484+0+0 (secure 0 0 0) 0x7f35080f0c90 con 0x7f3518106170 2024-04-23T10:37:50.241 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.232+0000 7f351caa2640 1 --2- 172.21.15.97:0/1234241134 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a176c3b800 0x55a176c3dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.232+0000 7f351daa4640 1 -- 172.21.15.97:0/1234241134 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a176c3b800 msgr2=0x55a176c3dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:37:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.232+0000 7f351daa4640 1 --2- 172.21.15.97:0/1234241134 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a176c3b800 0x55a176c3dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.232+0000 7f351daa4640 1 -- 172.21.15.97:0/1234241134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a176c30f40 msgr2=0x55a176c3b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:37:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.232+0000 7f351daa4640 1 --2- 172.21.15.97:0/1234241134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a176c30f40 0x55a176c3b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.232+0000 7f351daa4640 1 -- 172.21.15.97:0/1234241134 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1768a8210 con 0x55a176c30620 2024-04-23T10:37:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.232+0000 7f351caa2640 1 --2- 172.21.15.97:0/1234241134 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a176c3b800 0x55a176c3dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:37:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.232+0000 7f351d2a3640 1 --2- 172.21.15.97:0/1234241134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a176c30f40 0x55a176c3b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:37:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.232+0000 7f351daa4640 1 --2- 172.21.15.97:0/1234241134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a176c30620 0x55a176c30a00 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f35100094d0 tx=0x7f35100107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e85a980d86029ce0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:37:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.232+0000 7f351daa4640 1 --2- 172.21.15.97:0/566019886 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f35040a3dc0 0x7f35040a6280 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3510005c40 tx=0x7f351001d000 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:37:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.232+0000 7f35037fe640 1 -- 172.21.15.97:0/1234241134 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3510012840 con 0x55a176c30620 2024-04-23T10:37:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.232+0000 7f35037fe640 1 -- 172.21.15.97:0/1234241134 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3510009d60 con 0x55a176c30620 2024-04-23T10:37:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.232+0000 7f35037fe640 1 -- 172.21.15.97:0/1234241134 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f351000dde0 con 0x55a176c30620 2024-04-23T10:37:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.233+0000 7f3522216740 1 -- 172.21.15.97:0/1234241134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a176c30620 msgr2=0x55a176c30a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:37:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.233+0000 7f3522216740 1 --2- 172.21.15.97:0/1234241134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a176c30620 0x55a176c30a00 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f35100094d0 tx=0x7f35100107a0 comp rx=0 tx=0).stop 2024-04-23T10:37:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.233+0000 7f3522216740 1 -- 172.21.15.97:0/1234241134 shutdown_connections 2024-04-23T10:37:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.233+0000 7f3522216740 1 --2- 172.21.15.97:0/1234241134 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a176c3b800 0x55a176c3dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.233+0000 7f3522216740 1 --2- 172.21.15.97:0/1234241134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a176c30f40 0x55a176c3b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.233+0000 7f3522216740 1 --2- 172.21.15.97:0/1234241134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a176c30620 0x55a176c30a00 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.233+0000 7f3522216740 1 -- 172.21.15.97:0/1234241134 >> 172.21.15.97:0/1234241134 conn(0x55a176c2cb30 msgr2=0x55a176c2cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:37:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.233+0000 7f3522216740 1 -- 172.21.15.97:0/1234241134 shutdown_connections 2024-04-23T10:37:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.233+0000 7f3522216740 1 -- 172.21.15.97:0/1234241134 wait complete. 2024-04-23T10:37:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.233+0000 7f3522216740 1 -- start start 2024-04-23T10:37:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.233+0000 7f3522216740 1 Processor -- start 2024-04-23T10:37:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.234+0000 7f3522216740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a176c30620 0x55a176c3b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.234+0000 7f3522216740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a176c30f40 0x55a176c384a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.234+0000 7f3522216740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a176c3b800 0x55a176c38a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.234+0000 7f3522216740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55a176c3e3d0 con 0x55a176c30f40 2024-04-23T10:37:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.234+0000 7f3522216740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a176b300b0 con 0x55a176c30620 2024-04-23T10:37:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.234+0000 7f3522216740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55a176c3e550 con 0x55a176c3b800 2024-04-23T10:37:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.234+0000 7f3522216740 10 client.0 _ll_register_callbacks cb 0x55a176b1dd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:37:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.234+0000 7f351caa2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a176c30f40 0x55a176c384a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.234+0000 7f351caa2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a176c30f40 0x55a176c384a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38424/0 (socket says 172.21.15.97:38424) 2024-04-23T10:37:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.234+0000 7f351caa2640 1 -- 172.21.15.97:0/405447599 learned_addr learned my addr 172.21.15.97:0/405447599 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:37:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.234+0000 7f351daa4640 1 --2- 172.21.15.97:0/405447599 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a176c3b800 0x55a176c38a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:50.245 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.234+0000 7f351d2a3640 1 --2- 172.21.15.97:0/405447599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a176c30620 0x55a176c3b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:50.245 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.234+0000 7f351caa2640 1 -- 172.21.15.97:0/405447599 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a176c3b800 msgr2=0x55a176c38a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:37:50.245 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.234+0000 7f351caa2640 1 --2- 172.21.15.97:0/405447599 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a176c3b800 0x55a176c38a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:50.245 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.234+0000 7f351caa2640 1 -- 172.21.15.97:0/405447599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a176c30620 msgr2=0x55a176c3b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:37:50.245 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.234+0000 7f351caa2640 1 --2- 172.21.15.97:0/405447599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a176c30620 0x55a176c3b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:50.245 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.234+0000 7f351caa2640 1 -- 172.21.15.97:0/405447599 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a176974620 con 0x55a176c30f40 2024-04-23T10:37:50.245 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.234+0000 7f351daa4640 1 --2- 172.21.15.97:0/405447599 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a176c3b800 0x55a176c38a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:37:50.245 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.234+0000 7f351d2a3640 1 --2- 172.21.15.97:0/405447599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a176c30620 0x55a176c3b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:37:50.245 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.235+0000 7f351caa2640 1 --2- 172.21.15.97:0/405447599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a176c30f40 0x55a176c384a0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f3508031440 tx=0x7f3508009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=803306502afd122b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:37:50.245 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.235+0000 7f3500ff9640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:37:50.245 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.235+0000 7f3500ff9640 1 -- 172.21.15.97:0/405447599 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3508049040 con 0x55a176c30f40 2024-04-23T10:37:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.235+0000 7f3500ff9640 1 -- 172.21.15.97:0/405447599 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8484+0+0 (secure 0 0 0) 0x7f35080dd0b0 con 0x55a176c30f40 2024-04-23T10:37:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.235+0000 7f3522216740 1 -- 172.21.15.97:0/405447599 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a176ba3390 con 0x55a176c30f40 2024-04-23T10:37:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.235+0000 7f3500ff9640 1 -- 172.21.15.97:0/405447599 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f350803b460 con 0x55a176c30f40 2024-04-23T10:37:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.235+0000 7f3522216740 10 client.17211 fetch_fsmap learned FSMap version 26 2024-04-23T10:37:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.235+0000 7f3522216740 1 -- 172.21.15.97:0/405447599 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a176c4a340 con 0x55a176c30f40 2024-04-23T10:37:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.235+0000 7f3500ff9640 1 -- 172.21.15.97:0/405447599 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55a176ba3390 con 0x55a176c30f40 2024-04-23T10:37:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.235+0000 7f3500ff9640 1 -- 172.21.15.97:0/405447599 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f35080ee620 con 0x55a176c30f40 2024-04-23T10:37:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.235+0000 7f3522216740 10 client.17211 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:37:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.235+0000 7f3522216740 10 client.17211 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:37:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.235+0000 7f3522216740 20 client.17211 populate_metadata read hostname 'smithi097' 2024-04-23T10:37:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.235+0000 7f3522216740 1 --2- 172.21.15.97:0/405447599 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55a176c4b820 0x55a176c4dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.235+0000 7f3522216740 4 client.17211 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:37:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.235+0000 7f3522216740 1 -- 172.21.15.97:0/405447599 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a176b9d200 con 0x55a176c4b820 2024-04-23T10:37:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.235+0000 7f351d2a3640 1 --2- 172.21.15.97:0/405447599 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55a176c4b820 0x55a176c4dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.236+0000 7f3500ff9640 10 client.17211 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:37:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.236+0000 7f351d2a3640 1 --2- 172.21.15.97:0/405447599 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55a176c4b820 0x55a176c4dc10 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f03998f5e9efe1a8 server_cookie=fef50e211915953 in_seq=0 out_seq=0 2024-04-23T10:37:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.237+0000 7f3500ff9640 1 -- 172.21.15.97:0/405447599 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a176b9d200 con 0x55a176c4b820 2024-04-23T10:37:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.237+0000 7f3500ff9640 10 client.17211 handle_command_reply: tid=0 2024-04-23T10:37:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.237+0000 7f3522216740 1 client.17211 shutdown 2024-04-23T10:37:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.237+0000 7f3522216740 1 -- 172.21.15.97:0/405447599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a176c30f40 msgr2=0x55a176c384a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:37:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.237+0000 7f3522216740 1 --2- 172.21.15.97:0/405447599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a176c30f40 0x55a176c384a0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f3508031440 tx=0x7f3508009260 comp rx=0 tx=0).stop 2024-04-23T10:37:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.237+0000 7f3522216740 1 -- 172.21.15.97:0/405447599 shutdown_connections 2024-04-23T10:37:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.237+0000 7f3522216740 1 --2- 172.21.15.97:0/405447599 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55a176c4b820 0x55a176c4dc10 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.237+0000 7f3522216740 1 --2- 172.21.15.97:0/405447599 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a176c3b800 0x55a176c38a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.237+0000 7f3522216740 1 --2- 172.21.15.97:0/405447599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a176c30f40 0x55a176c384a0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.237+0000 7f3500ff9640 0 client.17211 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:37:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.237+0000 7f3522216740 1 --2- 172.21.15.97:0/405447599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a176c30620 0x55a176c3b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.237+0000 7f3522216740 1 -- 172.21.15.97:0/405447599 >> 172.21.15.97:0/405447599 conn(0x55a176c2cb30 msgr2=0x55a176c32860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:37:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.237+0000 7f3522216740 1 -- 172.21.15.97:0/405447599 shutdown_connections 2024-04-23T10:37:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.237+0000 7f3522216740 1 -- 172.21.15.97:0/405447599 wait complete. 2024-04-23T10:37:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.237+0000 7f3522216740 20 client.17211 trim_cache size 0 max 16384 2024-04-23T10:37:50.257 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.252+0000 7f3522216740 1 Processor -- start 2024-04-23T10:37:50.257 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.252+0000 7f3522216740 1 -- start start 2024-04-23T10:37:50.257 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.252+0000 7f3522216740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a176c30620 0x55a176c3ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:50.257 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.252+0000 7f3522216740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a176c30f40 0x55a176c2bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:50.257 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.252+0000 7f3522216740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a176c3b800 0x55a176c53520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:50.257 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.252+0000 7f3522216740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f35181ada10 con 0x55a176c30620 2024-04-23T10:37:50.258 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.252+0000 7f3522216740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f35181af550 con 0x55a176c30f40 2024-04-23T10:37:50.258 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.252+0000 7f3522216740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f35181ae0e0 con 0x55a176c3b800 2024-04-23T10:37:50.258 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.252+0000 7f351d2a3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a176c30620 0x55a176c3ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:50.258 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.252+0000 7f351d2a3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a176c30620 0x55a176c3ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38426/0 (socket says 172.21.15.97:38426) 2024-04-23T10:37:50.258 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.252+0000 7f351d2a3640 1 -- 172.21.15.97:0/471573903 learned_addr learned my addr 172.21.15.97:0/471573903 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:37:50.258 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.252+0000 7f351caa2640 1 --2- 172.21.15.97:0/471573903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a176c30f40 0x55a176c2bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:50.258 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.252+0000 7f351daa4640 1 --2- 172.21.15.97:0/471573903 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a176c3b800 0x55a176c53520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:50.258 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.252+0000 7f351d2a3640 1 -- 172.21.15.97:0/471573903 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a176c3b800 msgr2=0x55a176c53520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:37:50.258 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.252+0000 7f351d2a3640 1 --2- 172.21.15.97:0/471573903 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a176c3b800 0x55a176c53520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:50.258 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.252+0000 7f351d2a3640 1 -- 172.21.15.97:0/471573903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a176c30f40 msgr2=0x55a176c2bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:37:50.258 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.252+0000 7f351d2a3640 1 --2- 172.21.15.97:0/471573903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a176c30f40 0x55a176c2bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:50.258 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.252+0000 7f351d2a3640 1 -- 172.21.15.97:0/471573903 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a176c30d70 con 0x55a176c30620 2024-04-23T10:37:50.259 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.252+0000 7f351caa2640 1 --2- 172.21.15.97:0/471573903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a176c30f40 0x55a176c2bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:37:50.259 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.252+0000 7f351daa4640 1 --2- 172.21.15.97:0/471573903 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a176c3b800 0x55a176c53520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:37:50.259 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.253+0000 7f351d2a3640 1 --2- 172.21.15.97:0/471573903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a176c30620 0x55a176c3ace0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x55a176c4e4c0 tx=0x7f351400b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5894ad9cafd2e7d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:37:50.259 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.253+0000 7f3502ffd640 1 -- 172.21.15.97:0/471573903 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3514003ac0 con 0x55a176c30620 2024-04-23T10:37:50.259 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.253+0000 7f3502ffd640 1 -- 172.21.15.97:0/471573903 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f351401b7e0 con 0x55a176c30620 2024-04-23T10:37:50.259 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.253+0000 7f3502ffd640 1 -- 172.21.15.97:0/471573903 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35140127a0 con 0x55a176c30620 2024-04-23T10:37:50.259 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.253+0000 7f3522216740 1 -- 172.21.15.97:0/471573903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a176c30620 msgr2=0x55a176c3ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:37:50.259 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.253+0000 7f3522216740 1 --2- 172.21.15.97:0/471573903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a176c30620 0x55a176c3ace0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x55a176c4e4c0 tx=0x7f351400b390 comp rx=0 tx=0).stop 2024-04-23T10:37:50.259 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.254+0000 7f3522216740 1 -- 172.21.15.97:0/471573903 shutdown_connections 2024-04-23T10:37:50.259 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.254+0000 7f3522216740 1 --2- 172.21.15.97:0/471573903 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a176c3b800 0x55a176c53520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:50.259 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.254+0000 7f3522216740 1 --2- 172.21.15.97:0/471573903 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a176c30f40 0x55a176c2bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:50.259 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.254+0000 7f3522216740 1 --2- 172.21.15.97:0/471573903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a176c30620 0x55a176c3ace0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:50.259 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.254+0000 7f3522216740 1 -- 172.21.15.97:0/471573903 >> 172.21.15.97:0/471573903 conn(0x55a176c2cb30 msgr2=0x55a176c2b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:37:50.260 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.254+0000 7f3522216740 1 -- 172.21.15.97:0/471573903 shutdown_connections 2024-04-23T10:37:50.260 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.254+0000 7f3522216740 1 -- 172.21.15.97:0/471573903 wait complete. 2024-04-23T10:37:50.260 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.254+0000 7f3522216740 1 -- start start 2024-04-23T10:37:50.260 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.254+0000 7f3522216740 1 Processor -- start 2024-04-23T10:37:50.260 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.254+0000 7f3522216740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a176c30620 0x55a176c35210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:50.260 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.254+0000 7f3522216740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a176c30f40 0x55a176c35750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:50.260 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.254+0000 7f3522216740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a176c3b800 0x55a176c35c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:50.260 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.254+0000 7f3522216740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f35181aee80 con 0x55a176c30f40 2024-04-23T10:37:50.260 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.254+0000 7f3522216740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f35181ae7b0 con 0x55a176c30620 2024-04-23T10:37:50.260 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.254+0000 7f3522216740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f35181aba10 con 0x55a176c3b800 2024-04-23T10:37:50.260 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.254+0000 7f3522216740 10 client.0 _ll_register_callbacks cb 0x7f35080b8f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:37:50.260 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.255+0000 7f351caa2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a176c30f40 0x55a176c35750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:50.261 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.255+0000 7f351caa2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a176c30f40 0x55a176c35750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38442/0 (socket says 172.21.15.97:38442) 2024-04-23T10:37:50.261 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.255+0000 7f351caa2640 1 -- 172.21.15.97:0/2393663552 learned_addr learned my addr 172.21.15.97:0/2393663552 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:37:50.261 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.255+0000 7f351d2a3640 1 --2- 172.21.15.97:0/2393663552 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a176c30620 0x55a176c35210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:50.261 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.255+0000 7f351daa4640 1 --2- 172.21.15.97:0/2393663552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a176c3b800 0x55a176c35c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:50.261 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.255+0000 7f351caa2640 1 -- 172.21.15.97:0/2393663552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a176c3b800 msgr2=0x55a176c35c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:37:50.261 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.255+0000 7f351caa2640 1 --2- 172.21.15.97:0/2393663552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a176c3b800 0x55a176c35c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:50.261 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.255+0000 7f351caa2640 1 -- 172.21.15.97:0/2393663552 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a176c30620 msgr2=0x55a176c35210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:37:50.261 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.255+0000 7f351caa2640 1 --2- 172.21.15.97:0/2393663552 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a176c30620 0x55a176c35210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:50.261 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.255+0000 7f351caa2640 1 -- 172.21.15.97:0/2393663552 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a176cb5250 con 0x55a176c30f40 2024-04-23T10:37:50.261 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.255+0000 7f351caa2640 1 --2- 172.21.15.97:0/2393663552 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a176c30f40 0x55a176c35750 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f35080baea0 tx=0x7f350803b640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a9263e2776f1a10 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:37:50.261 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.255+0000 7f3500ff9640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:37:50.262 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.255+0000 7f3500ff9640 1 -- 172.21.15.97:0/2393663552 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35080a0060 con 0x55a176c30f40 2024-04-23T10:37:50.262 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.255+0000 7f3500ff9640 1 -- 172.21.15.97:0/2393663552 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8484+0+0 (secure 0 0 0) 0x7f35080c55d0 con 0x55a176c30f40 2024-04-23T10:37:50.262 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.255+0000 7f3522216740 1 -- 172.21.15.97:0/2393663552 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f351401b7e0 con 0x55a176c30f40 2024-04-23T10:37:50.262 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.255+0000 7f3500ff9640 1 -- 172.21.15.97:0/2393663552 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3508049070 con 0x55a176c30f40 2024-04-23T10:37:50.262 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.255+0000 7f3522216740 10 client.17217 fetch_fsmap learned FSMap version 26 2024-04-23T10:37:50.262 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.255+0000 7f3522216740 1 -- 172.21.15.97:0/2393663552 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a176cb6540 con 0x55a176c30f40 2024-04-23T10:37:50.262 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.255+0000 7f3500ff9640 1 -- 172.21.15.97:0/2393663552 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f351401b7e0 con 0x55a176c30f40 2024-04-23T10:37:50.262 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.256+0000 7f3500ff9640 1 -- 172.21.15.97:0/2393663552 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f350803bb40 con 0x55a176c30f40 2024-04-23T10:37:50.262 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.256+0000 7f3522216740 10 client.17217 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:37:50.262 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.256+0000 7f3522216740 10 client.17217 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:37:50.262 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.256+0000 7f3522216740 20 client.17217 populate_metadata read hostname 'smithi097' 2024-04-23T10:37:50.263 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.256+0000 7f3522216740 1 --2- 172.21.15.97:0/2393663552 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55a176cb6ad0 0x55a176cb8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:50.263 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.256+0000 7f3522216740 4 client.17217 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:37:50.263 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.256+0000 7f3522216740 1 -- 172.21.15.97:0/2393663552 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a176b195f0 con 0x55a176cb6ad0 2024-04-23T10:37:50.263 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.256+0000 7f351d2a3640 1 --2- 172.21.15.97:0/2393663552 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55a176cb6ad0 0x55a176cb8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:50.263 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.256+0000 7f3500ff9640 10 client.17217 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:37:50.263 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.256+0000 7f351d2a3640 1 --2- 172.21.15.97:0/2393663552 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55a176cb6ad0 0x55a176cb8ef0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9204e8bcb8bc7ad server_cookie=1a279851d2645a2f in_seq=0 out_seq=0 2024-04-23T10:37:50.270 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.271+0000 7f3500ff9640 1 -- 172.21.15.97:0/2393663552 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a176b195f0 con 0x55a176cb6ad0 2024-04-23T10:37:50.271 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.271+0000 7f3500ff9640 10 client.17217 handle_command_reply: tid=0 2024-04-23T10:37:50.271 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.271+0000 7f3522216740 1 client.17217 shutdown 2024-04-23T10:37:50.271 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.271+0000 7f3522216740 1 -- 172.21.15.97:0/2393663552 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a176c30f40 msgr2=0x55a176c35750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:37:50.271 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.271+0000 7f3522216740 1 --2- 172.21.15.97:0/2393663552 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a176c30f40 0x55a176c35750 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f35080baea0 tx=0x7f350803b640 comp rx=0 tx=0).stop 2024-04-23T10:37:50.271 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.271+0000 7f3522216740 1 -- 172.21.15.97:0/2393663552 shutdown_connections 2024-04-23T10:37:50.271 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.271+0000 7f3522216740 1 --2- 172.21.15.97:0/2393663552 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55a176cb6ad0 0x55a176cb8ef0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:50.271 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.271+0000 7f3522216740 1 --2- 172.21.15.97:0/2393663552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a176c3b800 0x55a176c35c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:50.271 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.271+0000 7f3522216740 1 --2- 172.21.15.97:0/2393663552 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a176c30f40 0x55a176c35750 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:50.271 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.271+0000 7f3500ff9640 0 client.17217 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:37:50.271 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.271+0000 7f3522216740 1 --2- 172.21.15.97:0/2393663552 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a176c30620 0x55a176c35210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:50.271 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.271+0000 7f3522216740 1 -- 172.21.15.97:0/2393663552 >> 172.21.15.97:0/2393663552 conn(0x55a176c2cb30 msgr2=0x55a176c31fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:37:50.271 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.271+0000 7f3522216740 1 -- 172.21.15.97:0/2393663552 shutdown_connections 2024-04-23T10:37:50.271 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.271+0000 7f3522216740 1 -- 172.21.15.97:0/2393663552 wait complete. 2024-04-23T10:37:50.272 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.271+0000 7f3522216740 20 client.17217 trim_cache size 0 max 16384 2024-04-23T10:37:50.272 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:37:50.272 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:37:50.272 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:37:50.272 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:37:50.273 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.272+0000 7f351f507640 1 -- 172.21.15.97:0/566019886 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f35040a3dc0 msgr2=0x7f35040a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:37:50.273 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.272+0000 7f351f507640 1 --2- 172.21.15.97:0/566019886 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f35040a3dc0 0x7f35040a6280 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3510005c40 tx=0x7f351001d000 comp rx=0 tx=0).stop 2024-04-23T10:37:50.273 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.272+0000 7f351f507640 1 -- 172.21.15.97:0/566019886 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3518106170 msgr2=0x7f3518198bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:37:50.273 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.272+0000 7f351f507640 1 --2- 172.21.15.97:0/566019886 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3518106170 0x7f3518198bd0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f3508006010 tx=0x7f3508004060 comp rx=0 tx=0).stop 2024-04-23T10:37:50.273 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.272+0000 7f351f507640 1 -- 172.21.15.97:0/566019886 shutdown_connections 2024-04-23T10:37:50.273 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.272+0000 7f351f507640 1 --2- 172.21.15.97:0/566019886 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f35040a3dc0 0x7f35040a6280 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:50.273 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.272+0000 7f351f507640 1 --2- 172.21.15.97:0/566019886 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f351819cf50 0x7f351819d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:50.274 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.272+0000 7f351f507640 1 --2- 172.21.15.97:0/566019886 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3518106170 0x7f3518198bd0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:50.274 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.272+0000 7f351f507640 1 --2- 172.21.15.97:0/566019886 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35181010f0 0x7f3518198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:50.274 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.272+0000 7f351f507640 1 -- 172.21.15.97:0/566019886 >> 172.21.15.97:0/566019886 conn(0x7f3518077fd0 msgr2=0x7f35180fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:37:50.274 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.272+0000 7f351f507640 1 -- 172.21.15.97:0/566019886 shutdown_connections 2024-04-23T10:37:50.274 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:37:50.272+0000 7f351f507640 1 -- 172.21.15.97:0/566019886 wait complete. 2024-04-23T10:37:50.644 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:37:50 smithi196 ceph-mon[32919]: pgmap v362: 97 pgs: 97 active+clean; 4.8 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 34 KiB/s wr, 12 op/s 2024-04-23T10:37:50.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:37:50 smithi183 ceph-mon[32317]: pgmap v362: 97 pgs: 97 active+clean; 4.8 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 34 KiB/s wr, 12 op/s 2024-04-23T10:37:50.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:37:50 smithi097 ceph-mon[29197]: pgmap v362: 97 pgs: 97 active+clean; 4.8 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 34 KiB/s wr, 12 op/s 2024-04-23T10:37:52.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:37:52 smithi183 ceph-mon[32317]: pgmap v363: 97 pgs: 97 active+clean; 4.8 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 34 KiB/s wr, 12 op/s 2024-04-23T10:37:52.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:37:52 smithi196 ceph-mon[32919]: pgmap v363: 97 pgs: 97 active+clean; 4.8 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 34 KiB/s wr, 12 op/s 2024-04-23T10:37:52.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:37:52 smithi097 ceph-mon[29197]: pgmap v363: 97 pgs: 97 active+clean; 4.8 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 34 KiB/s wr, 12 op/s 2024-04-23T10:37:53.816 INFO:tasks.workunit.client.0.smithi097.stdout:375/600: open fd = 3 2024-04-23T10:37:53.816 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.816 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.816 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:37:53.816 INFO:tasks.workunit.client.0.smithi097.stdout:376/600: open fd = 3 2024-04-23T10:37:53.816 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.816 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.817 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:37:53.817 INFO:tasks.workunit.client.0.smithi097.stdout:377/600: open fd = 3 2024-04-23T10:37:53.817 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.817 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.817 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:37:53.817 INFO:tasks.workunit.client.0.smithi097.stdout:378/600: open fd = 3 2024-04-23T10:37:53.817 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.817 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.817 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:37:53.817 INFO:tasks.workunit.client.0.smithi097.stdout:379/600: open fd = 3 2024-04-23T10:37:53.817 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.817 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.817 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:37:53.817 INFO:tasks.workunit.client.0.smithi097.stdout:380/600: open fd = 3 2024-04-23T10:37:53.817 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.817 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.818 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:37:53.818 INFO:tasks.workunit.client.0.smithi097.stdout:381/600: open fd = 3 2024-04-23T10:37:53.818 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.818 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.818 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:37:53.818 INFO:tasks.workunit.client.0.smithi097.stdout:382/600: open fd = 3 2024-04-23T10:37:53.818 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.818 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.818 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:37:53.818 INFO:tasks.workunit.client.0.smithi097.stdout:383/600: open fd = 3 2024-04-23T10:37:53.818 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.818 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.818 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:37:53.818 INFO:tasks.workunit.client.0.smithi097.stdout:384/600: open fd = 3 2024-04-23T10:37:53.818 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.818 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.819 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:37:53.819 INFO:tasks.workunit.client.0.smithi097.stdout:385/600: open fd = 3 2024-04-23T10:37:53.819 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.819 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.819 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:37:53.819 INFO:tasks.workunit.client.0.smithi097.stdout:386/600: open fd = 3 2024-04-23T10:37:53.819 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.819 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.819 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:37:53.819 INFO:tasks.workunit.client.0.smithi097.stdout:387/600: open fd = 3 2024-04-23T10:37:53.819 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.819 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.819 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:37:53.819 INFO:tasks.workunit.client.0.smithi097.stdout:388/600: open fd = 3 2024-04-23T10:37:53.819 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.820 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.820 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:37:53.820 INFO:tasks.workunit.client.0.smithi097.stdout:389/600: open fd = 3 2024-04-23T10:37:53.820 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.820 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.820 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:37:53.820 INFO:tasks.workunit.client.0.smithi097.stdout:390/600: open fd = 3 2024-04-23T10:37:53.820 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.820 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.820 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:37:53.820 INFO:tasks.workunit.client.0.smithi097.stdout:391/600: open fd = 3 2024-04-23T10:37:53.820 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.820 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.820 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:37:53.820 INFO:tasks.workunit.client.0.smithi097.stdout:392/600: open fd = 3 2024-04-23T10:37:53.820 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.820 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.821 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:37:53.821 INFO:tasks.workunit.client.0.smithi097.stdout:393/600: open fd = 3 2024-04-23T10:37:53.821 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.821 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.821 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:37:53.821 INFO:tasks.workunit.client.0.smithi097.stdout:394/600: open fd = 3 2024-04-23T10:37:53.821 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.821 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.821 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:37:53.821 INFO:tasks.workunit.client.0.smithi097.stdout:395/600: open fd = 3 2024-04-23T10:37:53.821 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.821 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.821 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:37:53.821 INFO:tasks.workunit.client.0.smithi097.stdout:396/600: open fd = 3 2024-04-23T10:37:53.821 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.821 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.822 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:37:53.822 INFO:tasks.workunit.client.0.smithi097.stdout:397/600: open fd = 3 2024-04-23T10:37:53.822 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.822 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.822 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:37:53.822 INFO:tasks.workunit.client.0.smithi097.stdout:398/600: open fd = 3 2024-04-23T10:37:53.822 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.822 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.822 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:37:53.822 INFO:tasks.workunit.client.0.smithi097.stdout:399/600: open fd = 3 2024-04-23T10:37:53.822 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.822 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.822 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:37:53.822 INFO:tasks.workunit.client.0.smithi097.stdout:400/600: open fd = 3 2024-04-23T10:37:53.823 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.823 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.823 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:37:53.823 INFO:tasks.workunit.client.0.smithi097.stdout:401/600: open fd = 3 2024-04-23T10:37:53.823 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.823 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.823 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:37:53.823 INFO:tasks.workunit.client.0.smithi097.stdout:402/600: open fd = 3 2024-04-23T10:37:53.823 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.823 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.823 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:37:53.823 INFO:tasks.workunit.client.0.smithi097.stdout:403/600: open fd = 3 2024-04-23T10:37:53.823 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.823 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.823 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:37:53.823 INFO:tasks.workunit.client.0.smithi097.stdout:404/600: open fd = 3 2024-04-23T10:37:53.823 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.824 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.824 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:37:53.824 INFO:tasks.workunit.client.0.smithi097.stdout:405/600: open fd = 3 2024-04-23T10:37:53.824 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.824 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.824 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:37:53.824 INFO:tasks.workunit.client.0.smithi097.stdout:406/600: open fd = 3 2024-04-23T10:37:53.824 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.824 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.824 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:37:53.824 INFO:tasks.workunit.client.0.smithi097.stdout:407/600: open fd = 3 2024-04-23T10:37:53.824 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.824 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.824 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:37:53.824 INFO:tasks.workunit.client.0.smithi097.stdout:408/600: open fd = 3 2024-04-23T10:37:53.824 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.824 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.825 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:37:53.825 INFO:tasks.workunit.client.0.smithi097.stdout:409/600: open fd = 3 2024-04-23T10:37:53.825 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.825 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.825 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:37:53.825 INFO:tasks.workunit.client.0.smithi097.stdout:410/600: open fd = 3 2024-04-23T10:37:53.825 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.825 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.825 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:37:53.825 INFO:tasks.workunit.client.0.smithi097.stdout:411/600: open fd = 3 2024-04-23T10:37:53.825 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.825 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.825 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:37:53.825 INFO:tasks.workunit.client.0.smithi097.stdout:412/600: open fd = 3 2024-04-23T10:37:53.825 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.825 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.826 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:37:53.826 INFO:tasks.workunit.client.0.smithi097.stdout:413/600: open fd = 3 2024-04-23T10:37:53.826 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.826 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.826 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:37:53.826 INFO:tasks.workunit.client.0.smithi097.stdout:414/600: open fd = 3 2024-04-23T10:37:53.826 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.826 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.826 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:37:53.826 INFO:tasks.workunit.client.0.smithi097.stdout:415/600: open fd = 3 2024-04-23T10:37:53.826 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.826 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.826 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:37:53.826 INFO:tasks.workunit.client.0.smithi097.stdout:416/600: open fd = 3 2024-04-23T10:37:53.827 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.827 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.827 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:37:53.827 INFO:tasks.workunit.client.0.smithi097.stdout:417/600: open fd = 3 2024-04-23T10:37:53.827 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.827 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.827 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:37:53.827 INFO:tasks.workunit.client.0.smithi097.stdout:418/600: open fd = 3 2024-04-23T10:37:53.827 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.827 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.827 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:37:53.827 INFO:tasks.workunit.client.0.smithi097.stdout:419/600: open fd = 3 2024-04-23T10:37:53.827 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.827 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.827 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:37:53.827 INFO:tasks.workunit.client.0.smithi097.stdout:420/600: open fd = 3 2024-04-23T10:37:53.827 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.828 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.828 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:37:53.828 INFO:tasks.workunit.client.0.smithi097.stdout:421/600: open fd = 3 2024-04-23T10:37:53.828 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.828 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.828 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:37:53.828 INFO:tasks.workunit.client.0.smithi097.stdout:422/600: open fd = 3 2024-04-23T10:37:53.828 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.828 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.828 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:37:53.828 INFO:tasks.workunit.client.0.smithi097.stdout:423/600: open fd = 3 2024-04-23T10:37:53.828 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.828 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.828 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:37:53.828 INFO:tasks.workunit.client.0.smithi097.stdout:424/600: open fd = 3 2024-04-23T10:37:53.829 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.829 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.829 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:37:53.829 INFO:tasks.workunit.client.0.smithi097.stdout:425/600: open fd = 3 2024-04-23T10:37:53.829 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.829 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.829 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:37:53.829 INFO:tasks.workunit.client.0.smithi097.stdout:426/600: open fd = 3 2024-04-23T10:37:53.829 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.829 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.829 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:37:53.829 INFO:tasks.workunit.client.0.smithi097.stdout:427/600: open fd = 3 2024-04-23T10:37:53.829 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.829 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.829 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:37:53.829 INFO:tasks.workunit.client.0.smithi097.stdout:428/600: open fd = 3 2024-04-23T10:37:53.829 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.830 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.830 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:37:53.830 INFO:tasks.workunit.client.0.smithi097.stdout:429/600: open fd = 3 2024-04-23T10:37:53.830 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.830 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.830 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:37:53.830 INFO:tasks.workunit.client.0.smithi097.stdout:430/600: open fd = 3 2024-04-23T10:37:53.830 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.830 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.830 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:37:53.830 INFO:tasks.workunit.client.0.smithi097.stdout:431/600: open fd = 3 2024-04-23T10:37:53.830 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.830 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.830 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:37:53.830 INFO:tasks.workunit.client.0.smithi097.stdout:432/600: open fd = 3 2024-04-23T10:37:53.830 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.831 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.831 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:37:53.831 INFO:tasks.workunit.client.0.smithi097.stdout:433/600: open fd = 3 2024-04-23T10:37:53.831 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.831 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.831 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:37:53.831 INFO:tasks.workunit.client.0.smithi097.stdout:434/600: open fd = 3 2024-04-23T10:37:53.831 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.831 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.831 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:37:53.831 INFO:tasks.workunit.client.0.smithi097.stdout:435/600: open fd = 3 2024-04-23T10:37:53.831 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.831 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.831 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:37:53.832 INFO:tasks.workunit.client.0.smithi097.stdout:436/600: open fd = 3 2024-04-23T10:37:53.832 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.832 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:37:53.832 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:37:54.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:37:54 smithi183 ceph-mon[32317]: pgmap v364: 97 pgs: 97 active+clean; 5.1 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-23T10:37:54.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:37:54 smithi196 ceph-mon[32919]: pgmap v364: 97 pgs: 97 active+clean; 5.1 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-23T10:37:54.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:37:54 smithi097 ceph-mon[29197]: pgmap v364: 97 pgs: 97 active+clean; 5.1 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-23T10:37:56.375 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-04-23T10:37:56.376 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-04-23T10:37:56.376 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T10:37:56.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.489+0000 7fce4595d640 1 -- 172.21.15.97:0/178111259 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce40102190 msgr2=0x7fce40102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:37:56.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.489+0000 7fce4595d640 1 --2- 172.21.15.97:0/178111259 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce40102190 0x7fce40102570 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fce34009900 tx=0x7fce3402a410 comp rx=0 tx=0).stop 2024-04-23T10:37:56.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.489+0000 7fce4595d640 1 -- 172.21.15.97:0/178111259 shutdown_connections 2024-04-23T10:37:56.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.489+0000 7fce4595d640 1 --2- 172.21.15.97:0/178111259 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce40107260 0x7fce4010ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:56.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.489+0000 7fce4595d640 1 --2- 172.21.15.97:0/178111259 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce40102b40 0x7fce40106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:56.488 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.489+0000 7fce4595d640 1 --2- 172.21.15.97:0/178111259 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce40102190 0x7fce40102570 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:56.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.489+0000 7fce4595d640 1 -- 172.21.15.97:0/178111259 >> 172.21.15.97:0/178111259 conn(0x7fce40079310 msgr2=0x7fce40079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:37:56.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.489+0000 7fce4595d640 1 -- 172.21.15.97:0/178111259 shutdown_connections 2024-04-23T10:37:56.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.489+0000 7fce4595d640 1 -- 172.21.15.97:0/178111259 wait complete. 2024-04-23T10:37:56.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.489+0000 7fce4595d640 1 Processor -- start 2024-04-23T10:37:56.489 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.490+0000 7fce4595d640 1 -- start start 2024-04-23T10:37:56.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.490+0000 7fce4595d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce40102190 0x7fce4019e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:56.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.490+0000 7fce4595d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce40107260 0x7fce4019e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:56.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.490+0000 7fce4595d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce401985c0 0x7fce40198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:56.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.490+0000 7fce4595d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fce401121c0 con 0x7fce40107260 2024-04-23T10:37:56.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.490+0000 7fce4595d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fce40112040 con 0x7fce40102190 2024-04-23T10:37:56.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.490+0000 7fce4595d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fce40112340 con 0x7fce401985c0 2024-04-23T10:37:56.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.490+0000 7fce3e7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce40107260 0x7fce4019e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:56.491 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.490+0000 7fce3e7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce40107260 0x7fce4019e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38000/0 (socket says 172.21.15.97:38000) 2024-04-23T10:37:56.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.490+0000 7fce3e7fc640 1 -- 172.21.15.97:0/2604532463 learned_addr learned my addr 172.21.15.97:0/2604532463 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:37:56.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.490+0000 7fce3effd640 1 --2- 172.21.15.97:0/2604532463 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce40102190 0x7fce4019e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:56.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.490+0000 7fce3f7fe640 1 --2- 172.21.15.97:0/2604532463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce401985c0 0x7fce40198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:56.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.490+0000 7fce3e7fc640 1 -- 172.21.15.97:0/2604532463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce401985c0 msgr2=0x7fce40198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:37:56.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.490+0000 7fce3e7fc640 1 --2- 172.21.15.97:0/2604532463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce401985c0 0x7fce40198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:56.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.490+0000 7fce3e7fc640 1 -- 172.21.15.97:0/2604532463 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce40102190 msgr2=0x7fce4019e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:37:56.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.490+0000 7fce3e7fc640 1 --2- 172.21.15.97:0/2604532463 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce40102190 0x7fce4019e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:56.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.490+0000 7fce3e7fc640 1 -- 172.21.15.97:0/2604532463 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce401990e0 con 0x7fce40107260 2024-04-23T10:37:56.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.490+0000 7fce3effd640 1 --2- 172.21.15.97:0/2604532463 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce40102190 0x7fce4019e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:37:56.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.490+0000 7fce3f7fe640 1 --2- 172.21.15.97:0/2604532463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce401985c0 0x7fce40198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:37:56.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.490+0000 7fce3e7fc640 1 --2- 172.21.15.97:0/2604532463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce40107260 0x7fce4019e950 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fce3000d740 tx=0x7fce3000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:37:56.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.491+0000 7fce4495b640 1 -- 172.21.15.97:0/2604532463 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce30017070 con 0x7fce40107260 2024-04-23T10:37:56.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.491+0000 7fce4595d640 1 -- 172.21.15.97:0/2604532463 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce401993d0 con 0x7fce40107260 2024-04-23T10:37:56.492 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.491+0000 7fce4495b640 1 -- 172.21.15.97:0/2604532463 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce3000de50 con 0x7fce40107260 2024-04-23T10:37:56.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.491+0000 7fce4495b640 1 -- 172.21.15.97:0/2604532463 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce30004ce0 con 0x7fce40107260 2024-04-23T10:37:56.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.491+0000 7fce4595d640 1 -- 172.21.15.97:0/2604532463 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce401a5620 con 0x7fce40107260 2024-04-23T10:37:56.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.492+0000 7fce4595d640 1 -- 172.21.15.97:0/2604532463 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce400691c0 con 0x7fce40107260 2024-04-23T10:37:56.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.493+0000 7fce4495b640 1 -- 172.21.15.97:0/2604532463 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fce30004e80 con 0x7fce40107260 2024-04-23T10:37:56.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.493+0000 7fce4495b640 1 --2- 172.21.15.97:0/2604532463 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fce280a3d20 0x7fce280a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:56.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.493+0000 7fce3effd640 1 --2- 172.21.15.97:0/2604532463 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fce280a3d20 0x7fce280a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:56.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.493+0000 7fce4495b640 1 -- 172.21.15.97:0/2604532463 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8484+0+0 (secure 0 0 0) 0x7fce300d0240 con 0x7fce40107260 2024-04-23T10:37:56.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.494+0000 7fce3effd640 1 --2- 172.21.15.97:0/2604532463 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fce280a3d20 0x7fce280a61e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fce34031440 tx=0x7fce34005a20 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:37:56.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.498+0000 7fce4495b640 1 -- 172.21.15.97:0/2604532463 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fce3009c900 con 0x7fce40107260 2024-04-23T10:37:56.714 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.715+0000 7fce4595d640 1 -- 172.21.15.97:0/2604532463 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fce4010a4f0 con 0x7fce40107260 2024-04-23T10:37:56.715 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.716+0000 7fce4495b640 1 -- 172.21.15.97:0/2604532463 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11931 (secure 0 0 0) 0x7fce3009c200 con 0x7fce40107260 2024-04-23T10:37:56.715 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:37:56.715 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3946878762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3946878762},{"type":"v1","addr":"172.21.15.97:6835","nonce":3946878762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/209800861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":209800861},{"type":"v1","addr":"172.21.15.97:6837","nonce":209800861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T10:24:06.910134+0000","modified":"2024-04-23T10:26:09.820985+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6839/418013575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":418013575},{"type":"v1","addr":"172.21.15.97:6839","nonce":418013575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6841/747242680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":747242680},{"type":"v1","addr":"172.21.15.97:6841","nonce":747242680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6833/4292847121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4292847121},{"type":"v1","addr":"172.21.15.183:6833","nonce":4292847121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6833/3885816565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3885816565},{"type":"v1","addr":"172.21.15.196:6833","nonce":3885816565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1897093105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1897093105},{"type":"v1","addr":"172.21.15.183:6835","nonce":1897093105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1171477563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1171477563},{"type":"v1","addr":"172.21.15.196:6835","nonce":1171477563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1898341911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1898341911},{"type":"v1","addr":"172.21.15.183:6837","nonce":1898341911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.196:6837/3094946502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3094946502},{"type":"v1","addr":"172.21.15.196:6837","nonce":3094946502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.196:6839/83843556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":83843556},{"type":"v1","addr":"172.21.15.196:6839","nonce":83843556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6839/1763841130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1763841130},{"type":"v1","addr":"172.21.15.183:6839","nonce":1763841130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24487,24403,24398,24469,24488]},"id":1}]} 2024-04-23T10:37:56.716 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 26 2024-04-23T10:37:56.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.718+0000 7fce4595d640 1 -- 172.21.15.97:0/2604532463 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fce280a3d20 msgr2=0x7fce280a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:37:56.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.718+0000 7fce4595d640 1 --2- 172.21.15.97:0/2604532463 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fce280a3d20 0x7fce280a61e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fce34031440 tx=0x7fce34005a20 comp rx=0 tx=0).stop 2024-04-23T10:37:56.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.718+0000 7fce4595d640 1 -- 172.21.15.97:0/2604532463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce40107260 msgr2=0x7fce4019e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:37:56.717 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.718+0000 7fce4595d640 1 --2- 172.21.15.97:0/2604532463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce40107260 0x7fce4019e950 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fce3000d740 tx=0x7fce3000dc00 comp rx=0 tx=0).stop 2024-04-23T10:37:56.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.718+0000 7fce4595d640 1 -- 172.21.15.97:0/2604532463 shutdown_connections 2024-04-23T10:37:56.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.718+0000 7fce4595d640 1 --2- 172.21.15.97:0/2604532463 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fce280a3d20 0x7fce280a61e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:56.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.718+0000 7fce4595d640 1 --2- 172.21.15.97:0/2604532463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce401985c0 0x7fce40198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:56.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.718+0000 7fce4595d640 1 --2- 172.21.15.97:0/2604532463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce40107260 0x7fce4019e950 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:56.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.718+0000 7fce4595d640 1 --2- 172.21.15.97:0/2604532463 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce40102190 0x7fce4019e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:56.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.718+0000 7fce4595d640 1 -- 172.21.15.97:0/2604532463 >> 172.21.15.97:0/2604532463 conn(0x7fce40079310 msgr2=0x7fce40100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:37:56.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.718+0000 7fce4595d640 1 -- 172.21.15.97:0/2604532463 shutdown_connections 2024-04-23T10:37:56.718 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.718+0000 7fce4595d640 1 -- 172.21.15.97:0/2604532463 wait complete. 2024-04-23T10:37:56.726 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:37:56 smithi097 ceph-mon[29197]: pgmap v365: 97 pgs: 97 active+clean; 5.1 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-23T10:37:56.728 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db / --timeout 60 --expiration 90.0 --await-for 42.0' 2024-04-23T10:37:56.728 DEBUG:teuthology.orchestra.run.smithi097:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24488 quiesce db / --timeout 60 --expiration 90.0 --await-for 42.0 2024-04-23T10:37:56.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.837+0000 7f59240dc640 1 -- 172.21.15.97:0/314329826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f591c102b40 msgr2=0x7f591c106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:37:56.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.837+0000 7f59240dc640 1 --2- 172.21.15.97:0/314329826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f591c102b40 0x7f591c106b80 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f590c009900 tx=0x7f590c02a410 comp rx=0 tx=0).stop 2024-04-23T10:37:56.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.837+0000 7f59240dc640 1 -- 172.21.15.97:0/314329826 shutdown_connections 2024-04-23T10:37:56.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.837+0000 7f59240dc640 1 --2- 172.21.15.97:0/314329826 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f591c107260 0x7f591c10ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:56.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.837+0000 7f59240dc640 1 --2- 172.21.15.97:0/314329826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f591c102b40 0x7f591c106b80 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:56.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.837+0000 7f59240dc640 1 --2- 172.21.15.97:0/314329826 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f591c102190 0x7f591c102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:56.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.837+0000 7f59240dc640 1 -- 172.21.15.97:0/314329826 >> 172.21.15.97:0/314329826 conn(0x7f591c079310 msgr2=0x7f591c079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:37:56.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.837+0000 7f59240dc640 1 -- 172.21.15.97:0/314329826 shutdown_connections 2024-04-23T10:37:56.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.837+0000 7f59240dc640 1 -- 172.21.15.97:0/314329826 wait complete. 2024-04-23T10:37:56.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.837+0000 7f59240dc640 1 Processor -- start 2024-04-23T10:37:56.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.837+0000 7f59240dc640 1 -- start start 2024-04-23T10:37:56.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.837+0000 7f59240dc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f591c102190 0x7f591c1986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:56.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.838+0000 7f59240dc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f591c107260 0x7f591c198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:56.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.838+0000 7f59240dc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f591c19cfa0 0x7f591c19d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:56.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.838+0000 7f59240dc640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f591c1121a0 con 0x7f591c19cfa0 2024-04-23T10:37:56.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.838+0000 7f59240dc640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f591c112020 con 0x7f591c102190 2024-04-23T10:37:56.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.838+0000 7f59240dc640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f591c112320 con 0x7f591c107260 2024-04-23T10:37:56.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.838+0000 7f5922679640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f591c19cfa0 0x7f591c19d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:56.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.838+0000 7f5922679640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f591c19cfa0 0x7f591c19d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38032/0 (socket says 172.21.15.97:38032) 2024-04-23T10:37:56.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.838+0000 7f5921677640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f591c107260 0x7f591c198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:56.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.838+0000 7f5922679640 1 -- 172.21.15.97:0/3542898849 learned_addr learned my addr 172.21.15.97:0/3542898849 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:37:56.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.838+0000 7f5921e78640 1 --2- 172.21.15.97:0/3542898849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f591c102190 0x7f591c1986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:56.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.838+0000 7f5922679640 1 -- 172.21.15.97:0/3542898849 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f591c107260 msgr2=0x7f591c198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:37:56.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.838+0000 7f5922679640 1 --2- 172.21.15.97:0/3542898849 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f591c107260 0x7f591c198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:56.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.838+0000 7f5922679640 1 -- 172.21.15.97:0/3542898849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f591c102190 msgr2=0x7f591c1986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:37:56.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.838+0000 7f5922679640 1 --2- 172.21.15.97:0/3542898849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f591c102190 0x7f591c1986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:56.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.838+0000 7f5922679640 1 -- 172.21.15.97:0/3542898849 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f591c19da90 con 0x7f591c19cfa0 2024-04-23T10:37:56.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.838+0000 7f5922679640 1 --2- 172.21.15.97:0/3542898849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f591c19cfa0 0x7f591c19d450 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f591400cad0 tx=0x7f591400ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:37:56.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.838+0000 7f5912ffd640 1 -- 172.21.15.97:0/3542898849 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f591400ad60 con 0x7f591c19cfa0 2024-04-23T10:37:56.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.838+0000 7f59240dc640 1 -- 172.21.15.97:0/3542898849 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f591c19dd80 con 0x7f591c19cfa0 2024-04-23T10:37:56.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.838+0000 7f5912ffd640 1 -- 172.21.15.97:0/3542898849 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5914005020 con 0x7f591c19cfa0 2024-04-23T10:37:56.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.838+0000 7f5912ffd640 1 -- 172.21.15.97:0/3542898849 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f591400f6b0 con 0x7f591c19cfa0 2024-04-23T10:37:56.840 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.838+0000 7f59240dc640 1 -- 172.21.15.97:0/3542898849 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f591c1a5590 con 0x7f591c19cfa0 2024-04-23T10:37:56.847 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:37:56 smithi196 ceph-mon[32919]: pgmap v365: 97 pgs: 97 active+clean; 5.1 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-23T10:37:56.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.840+0000 7f5912ffd640 1 -- 172.21.15.97:0/3542898849 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f59140051d0 con 0x7f591c19cfa0 2024-04-23T10:37:56.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.841+0000 7f5926dea740 1 Processor -- start 2024-04-23T10:37:56.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.841+0000 7f5912ffd640 1 --2- 172.21.15.97:0/3542898849 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f59080a3d20 0x7f59080a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:56.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.841+0000 7f5926dea740 1 -- start start 2024-04-23T10:37:56.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.841+0000 7f5926dea740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5610c747b0a0 0x5610c747b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:56.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.841+0000 7f5921677640 1 --2- 172.21.15.97:0/3542898849 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f59080a3d20 0x7f59080a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:56.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.841+0000 7f5926dea740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610c747b9c0 0x5610c7485b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:56.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.841+0000 7f5912ffd640 1 -- 172.21.15.97:0/3542898849 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8484+0+0 (secure 0 0 0) 0x7f59140d06e0 con 0x7f591c19cfa0 2024-04-23T10:37:56.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.841+0000 7f5926dea740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5610c7486090 0x5610c7488480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:56.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.842+0000 7f5926dea740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5610c73e0370 con 0x5610c747b0a0 2024-04-23T10:37:56.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.842+0000 7f5926dea740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5610c737d2e0 con 0x5610c747b9c0 2024-04-23T10:37:56.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.842+0000 7f5926dea740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5610c73dc330 con 0x5610c7486090 2024-04-23T10:37:56.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.842+0000 7f5921e78640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5610c747b0a0 0x5610c747b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:56.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.842+0000 7f5921e78640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5610c747b0a0 0x5610c747b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38040/0 (socket says 172.21.15.97:38040) 2024-04-23T10:37:56.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.842+0000 7f5921e78640 1 -- 172.21.15.97:0/709916132 learned_addr learned my addr 172.21.15.97:0/709916132 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:37:56.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.842+0000 7f5921677640 1 --2- 172.21.15.97:0/709916132 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610c747b9c0 0x5610c7485b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:56.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.842+0000 7f5922679640 1 --2- 172.21.15.97:0/709916132 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5610c7486090 0x5610c7488480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:56.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.842+0000 7f5921e78640 1 -- 172.21.15.97:0/709916132 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5610c7486090 msgr2=0x5610c7488480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:37:56.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.842+0000 7f5921e78640 1 --2- 172.21.15.97:0/709916132 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5610c7486090 0x5610c7488480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:56.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.842+0000 7f5921e78640 1 -- 172.21.15.97:0/709916132 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610c747b9c0 msgr2=0x5610c7485b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:37:56.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.842+0000 7f5921e78640 1 --2- 172.21.15.97:0/709916132 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610c747b9c0 0x5610c7485b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:56.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.842+0000 7f5921e78640 1 -- 172.21.15.97:0/709916132 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610c71bed80 con 0x5610c747b0a0 2024-04-23T10:37:56.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.842+0000 7f5921677640 1 --2- 172.21.15.97:0/3542898849 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f59080a3d20 0x7f59080a61e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f590c000c00 tx=0x7f590c031d90 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:37:56.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.842+0000 7f5921e78640 1 --2- 172.21.15.97:0/709916132 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5610c747b0a0 0x5610c747b480 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f591800ad00 tx=0x7f591800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff024f2fe53593e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:37:56.849 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.842+0000 7f5907fff640 1 -- 172.21.15.97:0/709916132 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f591801d070 con 0x5610c747b0a0 2024-04-23T10:37:56.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.842+0000 7f5907fff640 1 -- 172.21.15.97:0/709916132 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5918018af0 con 0x5610c747b0a0 2024-04-23T10:37:56.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.842+0000 7f5907fff640 1 -- 172.21.15.97:0/709916132 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5918010a70 con 0x5610c747b0a0 2024-04-23T10:37:56.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.843+0000 7f5926dea740 1 -- 172.21.15.97:0/709916132 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5610c747b0a0 msgr2=0x5610c747b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:37:56.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.843+0000 7f5926dea740 1 --2- 172.21.15.97:0/709916132 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5610c747b0a0 0x5610c747b480 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f591800ad00 tx=0x7f591800f790 comp rx=0 tx=0).stop 2024-04-23T10:37:56.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.843+0000 7f5926dea740 1 -- 172.21.15.97:0/709916132 shutdown_connections 2024-04-23T10:37:56.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.843+0000 7f5926dea740 1 --2- 172.21.15.97:0/709916132 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5610c7486090 0x5610c7488480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:56.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.843+0000 7f5926dea740 1 --2- 172.21.15.97:0/709916132 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610c747b9c0 0x5610c7485b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:56.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.843+0000 7f5926dea740 1 --2- 172.21.15.97:0/709916132 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5610c747b0a0 0x5610c747b480 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:56.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.843+0000 7f5926dea740 1 -- 172.21.15.97:0/709916132 >> 172.21.15.97:0/709916132 conn(0x5610c74773c0 msgr2=0x5610c74777a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:37:56.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.843+0000 7f5926dea740 1 -- 172.21.15.97:0/709916132 shutdown_connections 2024-04-23T10:37:56.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.843+0000 7f5926dea740 1 -- 172.21.15.97:0/709916132 wait complete. 2024-04-23T10:37:56.850 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.844+0000 7f5926dea740 1 -- start start 2024-04-23T10:37:56.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.844+0000 7f5926dea740 1 Processor -- start 2024-04-23T10:37:56.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.844+0000 7f5926dea740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610c747b0a0 0x5610c7483000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:56.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.844+0000 7f5926dea740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5610c747b9c0 0x5610c7483540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:56.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.844+0000 7f5926dea740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5610c7486090 0x5610c7483a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:56.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.844+0000 7f5926dea740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5610c748d290 con 0x5610c7486090 2024-04-23T10:37:56.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.844+0000 7f5926dea740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5610c737afb0 con 0x5610c747b0a0 2024-04-23T10:37:56.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.844+0000 7f5926dea740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5610c748d410 con 0x5610c747b9c0 2024-04-23T10:37:56.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.844+0000 7f5926dea740 10 client.0 _ll_register_callbacks cb 0x5610c737a9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:37:56.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.844+0000 7f5922679640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5610c7486090 0x5610c7483a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:56.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.844+0000 7f5921677640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5610c747b9c0 0x5610c7483540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:56.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.844+0000 7f5922679640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5610c7486090 0x5610c7483a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38046/0 (socket says 172.21.15.97:38046) 2024-04-23T10:37:56.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.844+0000 7f5922679640 1 -- 172.21.15.97:0/204566108 learned_addr learned my addr 172.21.15.97:0/204566108 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:37:56.851 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.844+0000 7f5921e78640 1 --2- 172.21.15.97:0/204566108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610c747b0a0 0x5610c7483000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:56.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.845+0000 7f5922679640 1 -- 172.21.15.97:0/204566108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5610c747b9c0 msgr2=0x5610c7483540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:37:56.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.845+0000 7f5922679640 1 --2- 172.21.15.97:0/204566108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5610c747b9c0 0x5610c7483540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:56.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.845+0000 7f5922679640 1 -- 172.21.15.97:0/204566108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610c747b0a0 msgr2=0x5610c7483000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:37:56.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.845+0000 7f5922679640 1 --2- 172.21.15.97:0/204566108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610c747b0a0 0x5610c7483000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:56.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.845+0000 7f5922679640 1 -- 172.21.15.97:0/204566108 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610c721f510 con 0x5610c7486090 2024-04-23T10:37:56.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.845+0000 7f5922679640 1 --2- 172.21.15.97:0/204566108 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5610c7486090 0x5610c7483a80 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f5914005cc0 tx=0x7f591409c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fed4d3dc920c8fa server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:37:56.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.845+0000 7f59057fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:37:56.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.845+0000 7f59057fa640 1 -- 172.21.15.97:0/204566108 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f591401d040 con 0x5610c7486090 2024-04-23T10:37:56.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.845+0000 7f59057fa640 1 -- 172.21.15.97:0/204566108 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8484+0+0 (secure 0 0 0) 0x7f59140a4e80 con 0x5610c7486090 2024-04-23T10:37:56.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.845+0000 7f5926dea740 1 -- 172.21.15.97:0/204566108 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5918018af0 con 0x5610c7486090 2024-04-23T10:37:56.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.845+0000 7f59057fa640 1 -- 172.21.15.97:0/204566108 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f59140a0a50 con 0x5610c7486090 2024-04-23T10:37:56.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.845+0000 7f5926dea740 10 client.17235 fetch_fsmap learned FSMap version 26 2024-04-23T10:37:56.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.845+0000 7f5926dea740 1 -- 172.21.15.97:0/204566108 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5610c70b2670 con 0x5610c7486090 2024-04-23T10:37:56.852 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.845+0000 7f59057fa640 1 -- 172.21.15.97:0/204566108 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5918018af0 con 0x5610c7486090 2024-04-23T10:37:56.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.845+0000 7f59057fa640 1 -- 172.21.15.97:0/204566108 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f591409bc80 con 0x5610c7486090 2024-04-23T10:37:56.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.846+0000 7f5926dea740 10 client.17235 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:37:56.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.846+0000 7f5926dea740 10 client.17235 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:37:56.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.846+0000 7f5926dea740 20 client.17235 populate_metadata read hostname 'smithi097' 2024-04-23T10:37:56.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.846+0000 7f5926dea740 1 --2- 172.21.15.97:0/204566108 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5610c749a860 0x5610c749ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:56.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.846+0000 7f5926dea740 4 client.17235 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:37:56.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.846+0000 7f5926dea740 1 -- 172.21.15.97:0/204566108 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5610c73edd90 con 0x5610c749a860 2024-04-23T10:37:56.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.846+0000 7f5921e78640 1 --2- 172.21.15.97:0/204566108 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5610c749a860 0x5610c749ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:56.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.846+0000 7f5921e78640 1 --2- 172.21.15.97:0/204566108 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5610c749a860 0x5610c749ccb0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36469bcf5224f374 server_cookie=6ca44616775b7db0 in_seq=0 out_seq=0 2024-04-23T10:37:56.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.846+0000 7f59057fa640 10 client.17235 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:37:56.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.847+0000 7f59057fa640 1 -- 172.21.15.97:0/204566108 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5610c73edd90 con 0x5610c749a860 2024-04-23T10:37:56.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.847+0000 7f59057fa640 10 client.17235 handle_command_reply: tid=0 2024-04-23T10:37:56.853 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.847+0000 7f5926dea740 1 client.17235 shutdown 2024-04-23T10:37:56.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.847+0000 7f5926dea740 1 -- 172.21.15.97:0/204566108 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5610c7486090 msgr2=0x5610c7483a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:37:56.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.847+0000 7f5926dea740 1 --2- 172.21.15.97:0/204566108 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5610c7486090 0x5610c7483a80 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f5914005cc0 tx=0x7f591409c2d0 comp rx=0 tx=0).stop 2024-04-23T10:37:56.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.847+0000 7f5926dea740 1 -- 172.21.15.97:0/204566108 shutdown_connections 2024-04-23T10:37:56.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.847+0000 7f5926dea740 1 --2- 172.21.15.97:0/204566108 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5610c749a860 0x5610c749ccb0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:56.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.847+0000 7f5926dea740 1 --2- 172.21.15.97:0/204566108 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5610c7486090 0x5610c7483a80 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:56.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.847+0000 7f5926dea740 1 --2- 172.21.15.97:0/204566108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5610c747b9c0 0x5610c7483540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:56.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.847+0000 7f59057fa640 0 client.17235 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:37:56.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.847+0000 7f5926dea740 1 --2- 172.21.15.97:0/204566108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610c747b0a0 0x5610c7483000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:56.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.847+0000 7f5926dea740 1 -- 172.21.15.97:0/204566108 >> 172.21.15.97:0/204566108 conn(0x5610c74773c0 msgr2=0x5610c747d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:37:56.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.847+0000 7f5926dea740 1 -- 172.21.15.97:0/204566108 shutdown_connections 2024-04-23T10:37:56.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.847+0000 7f5926dea740 1 -- 172.21.15.97:0/204566108 wait complete. 2024-04-23T10:37:56.854 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.848+0000 7f5926dea740 20 client.17235 trim_cache size 0 max 16384 2024-04-23T10:37:56.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.865+0000 7f5926dea740 1 Processor -- start 2024-04-23T10:37:56.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.865+0000 7f5926dea740 1 -- start start 2024-04-23T10:37:56.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.865+0000 7f5926dea740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5610c747b0a0 0x5610c7496150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:56.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.865+0000 7f5926dea740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5610c747b9c0 0x5610c747d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:56.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.865+0000 7f5926dea740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610c7486090 0x5610c74a25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:56.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.865+0000 7f5926dea740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f591c1ab9c0 con 0x5610c747b0a0 2024-04-23T10:37:56.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.865+0000 7f5926dea740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f591c1ad500 con 0x5610c7486090 2024-04-23T10:37:56.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.865+0000 7f5926dea740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f591c1ac090 con 0x5610c747b9c0 2024-04-23T10:37:56.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.865+0000 7f5921e78640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5610c747b0a0 0x5610c7496150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:56.869 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.865+0000 7f5921e78640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5610c747b0a0 0x5610c7496150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38052/0 (socket says 172.21.15.97:38052) 2024-04-23T10:37:56.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.865+0000 7f5921e78640 1 -- 172.21.15.97:0/3080164409 learned_addr learned my addr 172.21.15.97:0/3080164409 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:37:56.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.865+0000 7f5921e78640 1 -- 172.21.15.97:0/3080164409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5610c747b9c0 msgr2=0x5610c747d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:37:56.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.865+0000 7f5921e78640 1 --2- 172.21.15.97:0/3080164409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5610c747b9c0 0x5610c747d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:56.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.865+0000 7f5921e78640 1 -- 172.21.15.97:0/3080164409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610c7486090 msgr2=0x5610c74a25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:37:56.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.865+0000 7f5921e78640 1 --2- 172.21.15.97:0/3080164409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610c7486090 0x5610c74a25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:56.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.865+0000 7f5921e78640 1 -- 172.21.15.97:0/3080164409 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610c7481550 con 0x5610c747b0a0 2024-04-23T10:37:56.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.865+0000 7f5921e78640 1 --2- 172.21.15.97:0/3080164409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5610c747b0a0 0x5610c7496150 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f5918000c00 tx=0x7f5918010670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=889ea209aac80aed server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:37:56.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.865+0000 7f59077fe640 1 -- 172.21.15.97:0/3080164409 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f591802c020 con 0x5610c747b0a0 2024-04-23T10:37:56.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.865+0000 7f59077fe640 1 -- 172.21.15.97:0/3080164409 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f591801e040 con 0x5610c747b0a0 2024-04-23T10:37:56.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.865+0000 7f59077fe640 1 -- 172.21.15.97:0/3080164409 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5918028e60 con 0x5610c747b0a0 2024-04-23T10:37:56.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.866+0000 7f5926dea740 1 -- 172.21.15.97:0/3080164409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5610c747b0a0 msgr2=0x5610c7496150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:37:56.870 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.866+0000 7f5926dea740 1 --2- 172.21.15.97:0/3080164409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5610c747b0a0 0x5610c7496150 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f5918000c00 tx=0x7f5918010670 comp rx=0 tx=0).stop 2024-04-23T10:37:56.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.866+0000 7f5926dea740 1 -- 172.21.15.97:0/3080164409 shutdown_connections 2024-04-23T10:37:56.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.866+0000 7f5926dea740 1 --2- 172.21.15.97:0/3080164409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610c7486090 0x5610c74a25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:56.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.866+0000 7f5926dea740 1 --2- 172.21.15.97:0/3080164409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5610c747b9c0 0x5610c747d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:56.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.866+0000 7f5926dea740 1 --2- 172.21.15.97:0/3080164409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5610c747b0a0 0x5610c7496150 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:56.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.866+0000 7f5926dea740 1 -- 172.21.15.97:0/3080164409 >> 172.21.15.97:0/3080164409 conn(0x5610c74773c0 msgr2=0x5610c74850c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:37:56.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.866+0000 7f5926dea740 1 -- 172.21.15.97:0/3080164409 shutdown_connections 2024-04-23T10:37:56.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.866+0000 7f5926dea740 1 -- 172.21.15.97:0/3080164409 wait complete. 2024-04-23T10:37:56.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.866+0000 7f5926dea740 1 -- start start 2024-04-23T10:37:56.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.867+0000 7f5926dea740 1 Processor -- start 2024-04-23T10:37:56.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.867+0000 7f5926dea740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610c747b0a0 0x5610c7480370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:56.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.867+0000 7f5926dea740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5610c747b9c0 0x5610c74808b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:56.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.867+0000 7f5926dea740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5610c7486090 0x5610c74ffe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:56.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.867+0000 7f5926dea740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f591c1ace30 con 0x5610c747b9c0 2024-04-23T10:37:56.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.867+0000 7f5926dea740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f591c1ac760 con 0x5610c747b0a0 2024-04-23T10:37:56.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.867+0000 7f5926dea740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f591c1a99c0 con 0x5610c7486090 2024-04-23T10:37:56.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.867+0000 7f5926dea740 10 client.0 _ll_register_callbacks cb 0x7f591409c4e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:37:56.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.867+0000 7f5921677640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5610c747b9c0 0x5610c74808b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:56.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.867+0000 7f5921677640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5610c747b9c0 0x5610c74808b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38060/0 (socket says 172.21.15.97:38060) 2024-04-23T10:37:56.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.867+0000 7f5921677640 1 -- 172.21.15.97:0/257839523 learned_addr learned my addr 172.21.15.97:0/257839523 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:37:56.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.867+0000 7f5921e78640 1 --2- 172.21.15.97:0/257839523 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610c747b0a0 0x5610c7480370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:56.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.867+0000 7f5922679640 1 --2- 172.21.15.97:0/257839523 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5610c7486090 0x5610c74ffe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:56.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.867+0000 7f5921677640 1 -- 172.21.15.97:0/257839523 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5610c7486090 msgr2=0x5610c74ffe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:37:56.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.867+0000 7f5921677640 1 --2- 172.21.15.97:0/257839523 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5610c7486090 0x5610c74ffe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:56.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.867+0000 7f5921677640 1 -- 172.21.15.97:0/257839523 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610c747b0a0 msgr2=0x5610c7480370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:37:56.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.867+0000 7f5921677640 1 --2- 172.21.15.97:0/257839523 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610c747b0a0 0x5610c7480370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:56.872 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.867+0000 7f5921677640 1 -- 172.21.15.97:0/257839523 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610c7500380 con 0x5610c747b9c0 2024-04-23T10:37:56.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.867+0000 7f5921e78640 1 --2- 172.21.15.97:0/257839523 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610c747b0a0 0x5610c7480370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:37:56.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.868+0000 7f5921677640 1 --2- 172.21.15.97:0/257839523 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5610c747b9c0 0x5610c74808b0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f591c06b670 tx=0x7f590c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6af972e591e85cd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:37:56.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.868+0000 7f59057fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:37:56.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.868+0000 7f59057fa640 1 -- 172.21.15.97:0/257839523 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f590c035040 con 0x5610c747b9c0 2024-04-23T10:37:56.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.868+0000 7f59057fa640 1 -- 172.21.15.97:0/257839523 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8484+0+0 (secure 0 0 0) 0x7f590c0035f0 con 0x5610c747b9c0 2024-04-23T10:37:56.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.868+0000 7f5926dea740 1 -- 172.21.15.97:0/257839523 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f591801e040 con 0x5610c747b9c0 2024-04-23T10:37:56.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.868+0000 7f59057fa640 1 -- 172.21.15.97:0/257839523 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f590c05a050 con 0x5610c747b9c0 2024-04-23T10:37:56.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.868+0000 7f5926dea740 10 client.17241 fetch_fsmap learned FSMap version 26 2024-04-23T10:37:56.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.868+0000 7f5926dea740 1 -- 172.21.15.97:0/257839523 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5610c7501600 con 0x5610c747b9c0 2024-04-23T10:37:56.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.868+0000 7f59057fa640 1 -- 172.21.15.97:0/257839523 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f591801e040 con 0x5610c747b9c0 2024-04-23T10:37:56.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.868+0000 7f59057fa640 1 -- 172.21.15.97:0/257839523 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f590c048050 con 0x5610c747b9c0 2024-04-23T10:37:56.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.868+0000 7f5926dea740 10 client.17241 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:37:56.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.868+0000 7f5926dea740 10 client.17241 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:37:56.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.868+0000 7f5926dea740 20 client.17241 populate_metadata read hostname 'smithi097' 2024-04-23T10:37:56.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.868+0000 7f5926dea740 1 --2- 172.21.15.97:0/257839523 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5610c7501e70 0x5610c7504290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:37:56.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.868+0000 7f5926dea740 4 client.17241 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 90.0, "await_for": 42.0}] 2024-04-23T10:37:56.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.868+0000 7f5926dea740 1 -- 172.21.15.97:0/257839523 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 90.0, "await_for": 42.0}) -- 0x5610c73e7c00 con 0x5610c7501e70 2024-04-23T10:37:56.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.868+0000 7f5921e78640 1 --2- 172.21.15.97:0/257839523 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5610c7501e70 0x5610c7504290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:37:56.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.869+0000 7f59057fa640 10 client.17241 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:37:56.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.869+0000 7f5921e78640 1 --2- 172.21.15.97:0/257839523 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5610c7501e70 0x5610c7504290 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1de1dd1b37a2a68 server_cookie=e231ef8cf007de44 in_seq=0 out_seq=0 2024-04-23T10:37:56.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.883+0000 7f59057fa640 1 -- 172.21.15.97:0/257839523 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x5610c73e7c00 con 0x5610c7501e70 2024-04-23T10:37:56.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.883+0000 7f59057fa640 10 client.17241 handle_command_reply: tid=0 2024-04-23T10:37:56.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.883+0000 7f5926dea740 1 client.17241 shutdown 2024-04-23T10:37:56.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.883+0000 7f5926dea740 1 -- 172.21.15.97:0/257839523 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5610c747b9c0 msgr2=0x5610c74808b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:37:56.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.883+0000 7f5926dea740 1 --2- 172.21.15.97:0/257839523 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5610c747b9c0 0x5610c74808b0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f591c06b670 tx=0x7f590c009260 comp rx=0 tx=0).stop 2024-04-23T10:37:56.882 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.883+0000 7f5926dea740 1 -- 172.21.15.97:0/257839523 shutdown_connections 2024-04-23T10:37:56.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.883+0000 7f5926dea740 1 --2- 172.21.15.97:0/257839523 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5610c7501e70 0x5610c7504290 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:56.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.883+0000 7f5926dea740 1 --2- 172.21.15.97:0/257839523 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5610c7486090 0x5610c74ffe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:56.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.883+0000 7f5926dea740 1 --2- 172.21.15.97:0/257839523 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5610c747b9c0 0x5610c74808b0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:56.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.883+0000 7f5926dea740 1 --2- 172.21.15.97:0/257839523 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5610c747b0a0 0x5610c7480370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:56.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.883+0000 7f59057fa640 0 client.17241 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:37:56.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.883+0000 7f5926dea740 1 -- 172.21.15.97:0/257839523 >> 172.21.15.97:0/257839523 conn(0x5610c74773c0 msgr2=0x5610c7475d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:37:56.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.883+0000 7f5926dea740 1 -- 172.21.15.97:0/257839523 shutdown_connections 2024-04-23T10:37:56.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.883+0000 7f5926dea740 1 -- 172.21.15.97:0/257839523 wait complete. 2024-04-23T10:37:56.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.883+0000 7f5926dea740 20 client.17241 trim_cache size 0 max 16384 2024-04-23T10:37:56.883 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-23T10:37:56.883 INFO:teuthology.orchestra.run.smithi097.stdout: "epoch": 26, 2024-04-23T10:37:56.884 INFO:teuthology.orchestra.run.smithi097.stdout: "leader": 24488, 2024-04-23T10:37:56.884 INFO:teuthology.orchestra.run.smithi097.stdout: "set_version": 50, 2024-04-23T10:37:56.884 INFO:teuthology.orchestra.run.smithi097.stdout: "sets": { 2024-04-23T10:37:56.884 INFO:teuthology.orchestra.run.smithi097.stdout: "4280baf": { 2024-04-23T10:37:56.884 INFO:teuthology.orchestra.run.smithi097.stdout: "version": 50, 2024-04-23T10:37:56.884 INFO:teuthology.orchestra.run.smithi097.stdout: "age_ref": 0.0, 2024-04-23T10:37:56.884 INFO:teuthology.orchestra.run.smithi097.stdout: "state": { 2024-04-23T10:37:56.884 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "QUIESCED", 2024-04-23T10:37:56.884 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0 2024-04-23T10:37:56.884 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:37:56.884 INFO:teuthology.orchestra.run.smithi097.stdout: "timeout": 60.0, 2024-04-23T10:37:56.884 INFO:teuthology.orchestra.run.smithi097.stdout: "expiration": 90.0, 2024-04-23T10:37:56.884 INFO:teuthology.orchestra.run.smithi097.stdout: "members": { 2024-04-23T10:37:56.884 INFO:teuthology.orchestra.run.smithi097.stdout: "file:/": { 2024-04-23T10:37:56.885 INFO:teuthology.orchestra.run.smithi097.stdout: "excluded": false, 2024-04-23T10:37:56.885 INFO:teuthology.orchestra.run.smithi097.stdout: "state": { 2024-04-23T10:37:56.885 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "QUIESCED", 2024-04-23T10:37:56.885 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0 2024-04-23T10:37:56.885 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:37:56.885 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:37:56.885 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:37:56.885 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:37:56.885 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:37:56.885 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-23T10:37:56.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.884+0000 7f59240dc640 1 -- 172.21.15.97:0/3542898849 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f59080a3d20 msgr2=0x7f59080a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:37:56.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.884+0000 7f59240dc640 1 --2- 172.21.15.97:0/3542898849 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f59080a3d20 0x7f59080a61e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f590c000c00 tx=0x7f590c031d90 comp rx=0 tx=0).stop 2024-04-23T10:37:56.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.884+0000 7f59240dc640 1 -- 172.21.15.97:0/3542898849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f591c19cfa0 msgr2=0x7f591c19d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:37:56.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.884+0000 7f59240dc640 1 --2- 172.21.15.97:0/3542898849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f591c19cfa0 0x7f591c19d450 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f591400cad0 tx=0x7f591400ced0 comp rx=0 tx=0).stop 2024-04-23T10:37:56.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.884+0000 7f59240dc640 1 -- 172.21.15.97:0/3542898849 shutdown_connections 2024-04-23T10:37:56.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.884+0000 7f59240dc640 1 --2- 172.21.15.97:0/3542898849 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f59080a3d20 0x7f59080a61e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:56.886 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.884+0000 7f59240dc640 1 --2- 172.21.15.97:0/3542898849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f591c19cfa0 0x7f591c19d450 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:56.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.884+0000 7f59240dc640 1 --2- 172.21.15.97:0/3542898849 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f591c107260 0x7f591c198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:56.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.884+0000 7f59240dc640 1 --2- 172.21.15.97:0/3542898849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f591c102190 0x7f591c1986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:37:56.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.884+0000 7f59240dc640 1 -- 172.21.15.97:0/3542898849 >> 172.21.15.97:0/3542898849 conn(0x7f591c079310 msgr2=0x7f591c1001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:37:56.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.884+0000 7f59240dc640 1 -- 172.21.15.97:0/3542898849 shutdown_connections 2024-04-23T10:37:56.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:37:56.884+0000 7f59240dc640 1 -- 172.21.15.97:0/3542898849 wait complete. 2024-04-23T10:37:56.893 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '4280baf', quiesce took 0.5 seconds. Will release after: 30.0 2024-04-23T10:37:56.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:37:56 smithi183 ceph-mon[32317]: pgmap v365: 97 pgs: 97 active+clean; 5.1 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-23T10:37:57.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:37:57 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/2604532463' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:37:57.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:37:57 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/2604532463' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:37:57.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:37:57 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/2604532463' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:37:58.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:37:58 smithi196 ceph-mon[32919]: pgmap v366: 97 pgs: 97 active+clean; 5.3 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-23T10:37:58.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:37:58 smithi097 ceph-mon[29197]: pgmap v366: 97 pgs: 97 active+clean; 5.3 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-23T10:37:58.899 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:37:58 smithi183 ceph-mon[32317]: pgmap v366: 97 pgs: 97 active+clean; 5.3 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-23T10:38:00.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:38:00 smithi196 ceph-mon[32919]: pgmap v367: 97 pgs: 97 active+clean; 5.3 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-23T10:38:00.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:38:00 smithi097 ceph-mon[29197]: pgmap v367: 97 pgs: 97 active+clean; 5.3 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-23T10:38:00.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:38:00 smithi183 ceph-mon[32317]: pgmap v367: 97 pgs: 97 active+clean; 5.3 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-23T10:38:02.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:38:02 smithi196 ceph-mon[32919]: pgmap v368: 97 pgs: 97 active+clean; 5.3 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-23T10:38:02.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:38:02 smithi097 ceph-mon[29197]: pgmap v368: 97 pgs: 97 active+clean; 5.3 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-23T10:38:02.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:38:02 smithi183 ceph-mon[32317]: pgmap v368: 97 pgs: 97 active+clean; 5.3 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-23T10:38:03.371 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.369+0000 7f88b4649640 1 -- 172.21.15.97:0/1037017996 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88ac10c5b0 msgr2=0x7f88ac10e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:03.371 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.369+0000 7f88b4649640 1 --2- 172.21.15.97:0/1037017996 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88ac10c5b0 0x7f88ac10e9e0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f88a4009ca0 tx=0x7f88a402a950 comp rx=0 tx=0).stop 2024-04-23T10:38:03.371 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.369+0000 7f88b4649640 1 -- 172.21.15.97:0/1037017996 shutdown_connections 2024-04-23T10:38:03.371 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.369+0000 7f88b4649640 1 --2- 172.21.15.97:0/1037017996 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88ac10c5b0 0x7f88ac10e9e0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:03.371 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.369+0000 7f88b4649640 1 --2- 172.21.15.97:0/1037017996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88ac103b00 0x7f88ac10bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:03.371 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.369+0000 7f88b4649640 1 --2- 172.21.15.97:0/1037017996 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88ac1031e0 0x7f88ac1035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:03.371 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.369+0000 7f88b4649640 1 -- 172.21.15.97:0/1037017996 >> 172.21.15.97:0/1037017996 conn(0x7f88ac077f60 msgr2=0x7f88ac078370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:38:03.371 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.369+0000 7f88b4649640 1 -- 172.21.15.97:0/1037017996 shutdown_connections 2024-04-23T10:38:03.371 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.369+0000 7f88b4649640 1 -- 172.21.15.97:0/1037017996 wait complete. 2024-04-23T10:38:03.372 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.370+0000 7f88b4649640 1 Processor -- start 2024-04-23T10:38:03.372 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.370+0000 7f88b4649640 1 -- start start 2024-04-23T10:38:03.372 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.370+0000 7f88b4649640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88ac1031e0 0x7f88ac198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:03.372 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.370+0000 7f88b4649640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88ac103b00 0x7f88ac198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:03.372 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.370+0000 7f88b4649640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88ac10c5b0 0x7f88ac19cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:03.372 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.370+0000 7f88b4649640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f88ac1121b0 con 0x7f88ac1031e0 2024-04-23T10:38:03.372 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.370+0000 7f88b4649640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f88ac112030 con 0x7f88ac103b00 2024-04-23T10:38:03.372 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.370+0000 7f88b4649640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f88ac112330 con 0x7f88ac10c5b0 2024-04-23T10:38:03.372 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.370+0000 7f88b23e5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88ac1031e0 0x7f88ac198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:03.372 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.370+0000 7f88b23e5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88ac1031e0 0x7f88ac198690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38084/0 (socket says 172.21.15.97:38084) 2024-04-23T10:38:03.372 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.370+0000 7f88b23e5640 1 -- 172.21.15.97:0/2461560197 learned_addr learned my addr 172.21.15.97:0/2461560197 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:38:03.373 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.370+0000 7f88b1be4640 1 --2- 172.21.15.97:0/2461560197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88ac103b00 0x7f88ac198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:03.373 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.370+0000 7f88b2be6640 1 --2- 172.21.15.97:0/2461560197 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88ac10c5b0 0x7f88ac19cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:03.373 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.370+0000 7f88b23e5640 1 -- 172.21.15.97:0/2461560197 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88ac10c5b0 msgr2=0x7f88ac19cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:03.373 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.370+0000 7f88b23e5640 1 --2- 172.21.15.97:0/2461560197 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88ac10c5b0 0x7f88ac19cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:03.373 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.370+0000 7f88b23e5640 1 -- 172.21.15.97:0/2461560197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88ac103b00 msgr2=0x7f88ac198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:03.373 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.370+0000 7f88b23e5640 1 --2- 172.21.15.97:0/2461560197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88ac103b00 0x7f88ac198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:03.373 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.370+0000 7f88b23e5640 1 -- 172.21.15.97:0/2461560197 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88ac19d680 con 0x7f88ac1031e0 2024-04-23T10:38:03.373 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.370+0000 7f88b1be4640 1 --2- 172.21.15.97:0/2461560197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88ac103b00 0x7f88ac198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:38:03.373 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.371+0000 7f88b23e5640 1 --2- 172.21.15.97:0/2461560197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88ac1031e0 0x7f88ac198690 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f88a000c890 tx=0x7f88a000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:38:03.373 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.371+0000 7f889f7fe640 1 -- 172.21.15.97:0/2461560197 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a0004220 con 0x7f88ac1031e0 2024-04-23T10:38:03.374 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.371+0000 7f889f7fe640 1 -- 172.21.15.97:0/2461560197 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88a00043c0 con 0x7f88ac1031e0 2024-04-23T10:38:03.374 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.371+0000 7f88b4649640 1 -- 172.21.15.97:0/2461560197 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88ac19d970 con 0x7f88ac1031e0 2024-04-23T10:38:03.374 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.371+0000 7f889f7fe640 1 -- 172.21.15.97:0/2461560197 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a0004a00 con 0x7f88ac1031e0 2024-04-23T10:38:03.374 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.371+0000 7f88b4649640 1 -- 172.21.15.97:0/2461560197 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88ac1a5200 con 0x7f88ac1031e0 2024-04-23T10:38:03.381 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.373+0000 7f889f7fe640 1 -- 172.21.15.97:0/2461560197 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f88a00079b0 con 0x7f88ac1031e0 2024-04-23T10:38:03.381 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.374+0000 7f88b7433740 1 Processor -- start 2024-04-23T10:38:03.381 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.374+0000 7f88b7433740 1 -- start start 2024-04-23T10:38:03.381 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.374+0000 7f889f7fe640 1 --2- 172.21.15.97:0/2461560197 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f88980a3e40 0x7f88980a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:03.381 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.374+0000 7f88b7433740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559674304620 0x559674304a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:03.381 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.374+0000 7f88b7433740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559674304f40 0x55967430f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:03.382 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.374+0000 7f88b7433740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55967430f800 0x559674311bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:03.382 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.374+0000 7f88b7433740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559674269970 con 0x559674304f40 2024-04-23T10:38:03.382 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.374+0000 7f88b7433740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5596742064a0 con 0x559674304620 2024-04-23T10:38:03.382 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.374+0000 7f88b7433740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x559674265930 con 0x55967430f800 2024-04-23T10:38:03.382 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.374+0000 7f88b2be6640 1 --2- 172.21.15.97:0/2461560197 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f88980a3e40 0x7f88980a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:03.382 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.374+0000 7f88b1be4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559674304f40 0x55967430f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:03.382 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.374+0000 7f889f7fe640 1 -- 172.21.15.97:0/2461560197 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8484+0+0 (secure 0 0 0) 0x7f88a00d0560 con 0x7f88ac1031e0 2024-04-23T10:38:03.382 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.374+0000 7f88b1be4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559674304f40 0x55967430f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38100/0 (socket says 172.21.15.97:38100) 2024-04-23T10:38:03.382 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.374+0000 7f88b1be4640 1 -- 172.21.15.97:0/1951402411 learned_addr learned my addr 172.21.15.97:0/1951402411 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:38:03.382 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.374+0000 7f88b23e5640 1 --2- 172.21.15.97:0/1951402411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559674304620 0x559674304a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:03.382 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.375+0000 7f88b2be6640 1 --2- 172.21.15.97:0/1951402411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55967430f800 0x559674311bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:03.382 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.375+0000 7f88b1be4640 1 -- 172.21.15.97:0/1951402411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55967430f800 msgr2=0x559674311bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:38:03.382 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.375+0000 7f88b1be4640 1 --2- 172.21.15.97:0/1951402411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55967430f800 0x559674311bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:03.383 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.375+0000 7f88b1be4640 1 -- 172.21.15.97:0/1951402411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559674304620 msgr2=0x559674304a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:38:03.383 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.375+0000 7f88b1be4640 1 --2- 172.21.15.97:0/1951402411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559674304620 0x559674304a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:03.383 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.375+0000 7f88b1be4640 1 -- 172.21.15.97:0/1951402411 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559673f7c210 con 0x559674304f40 2024-04-23T10:38:03.383 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.375+0000 7f88b2be6640 1 --2- 172.21.15.97:0/1951402411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55967430f800 0x559674311bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:38:03.383 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.375+0000 7f88b23e5640 1 --2- 172.21.15.97:0/1951402411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559674304620 0x559674304a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:38:03.383 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.375+0000 7f88b1be4640 1 --2- 172.21.15.97:0/1951402411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559674304f40 0x55967430f2c0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f88a800bcb0 tx=0x7f88a800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=463e033089900c2b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:38:03.383 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.375+0000 7f88b2be6640 1 --2- 172.21.15.97:0/2461560197 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f88980a3e40 0x7f88980a6300 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x559674301f50 tx=0x7f88a4005f80 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:38:03.383 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.375+0000 7f889cbfe640 1 -- 172.21.15.97:0/1951402411 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a8015070 con 0x559674304f40 2024-04-23T10:38:03.383 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.375+0000 7f889cbfe640 1 -- 172.21.15.97:0/1951402411 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88a80118f0 con 0x559674304f40 2024-04-23T10:38:03.383 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.375+0000 7f889cbfe640 1 -- 172.21.15.97:0/1951402411 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a80108b0 con 0x559674304f40 2024-04-23T10:38:03.383 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.376+0000 7f88b7433740 1 -- 172.21.15.97:0/1951402411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559674304f40 msgr2=0x55967430f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:03.383 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.376+0000 7f88b7433740 1 --2- 172.21.15.97:0/1951402411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559674304f40 0x55967430f2c0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f88a800bcb0 tx=0x7f88a800f730 comp rx=0 tx=0).stop 2024-04-23T10:38:03.383 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.376+0000 7f88b7433740 1 -- 172.21.15.97:0/1951402411 shutdown_connections 2024-04-23T10:38:03.383 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.376+0000 7f88b7433740 1 --2- 172.21.15.97:0/1951402411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55967430f800 0x559674311bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:03.384 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.376+0000 7f88b7433740 1 --2- 172.21.15.97:0/1951402411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559674304f40 0x55967430f2c0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:03.384 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.376+0000 7f88b7433740 1 --2- 172.21.15.97:0/1951402411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559674304620 0x559674304a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:03.384 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.376+0000 7f88b7433740 1 -- 172.21.15.97:0/1951402411 >> 172.21.15.97:0/1951402411 conn(0x559674300b30 msgr2=0x559674300f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:38:03.384 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.376+0000 7f88b7433740 1 -- 172.21.15.97:0/1951402411 shutdown_connections 2024-04-23T10:38:03.384 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.376+0000 7f88b7433740 1 -- 172.21.15.97:0/1951402411 wait complete. 2024-04-23T10:38:03.384 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.376+0000 7f88b7433740 1 -- start start 2024-04-23T10:38:03.384 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.376+0000 7f88b7433740 1 Processor -- start 2024-04-23T10:38:03.384 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.377+0000 7f88b7433740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559674304620 0x55967430f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:03.384 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.377+0000 7f88b7433740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559674304f40 0x55967430c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:03.384 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.377+0000 7f88b7433740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55967430f800 0x55967430c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:03.384 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.377+0000 7f88b7433740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559674312430 con 0x559674304620 2024-04-23T10:38:03.384 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.377+0000 7f88b7433740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5596742040b0 con 0x559674304f40 2024-04-23T10:38:03.384 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.377+0000 7f88b7433740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5596743125b0 con 0x55967430f800 2024-04-23T10:38:03.385 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.377+0000 7f88b7433740 10 client.0 _ll_register_callbacks cb 0x5596741f1d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:38:03.385 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.377+0000 7f88b23e5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559674304620 0x55967430f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:03.385 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.377+0000 7f88b23e5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559674304620 0x55967430f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38104/0 (socket says 172.21.15.97:38104) 2024-04-23T10:38:03.385 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.377+0000 7f88b23e5640 1 -- 172.21.15.97:0/2706353486 learned_addr learned my addr 172.21.15.97:0/2706353486 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:38:03.385 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.377+0000 7f88b1be4640 1 --2- 172.21.15.97:0/2706353486 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559674304f40 0x55967430c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:03.385 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.377+0000 7f88b2be6640 1 --2- 172.21.15.97:0/2706353486 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55967430f800 0x55967430c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:03.385 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.377+0000 7f88b23e5640 1 -- 172.21.15.97:0/2706353486 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55967430f800 msgr2=0x55967430c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:38:03.385 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.377+0000 7f88b23e5640 1 --2- 172.21.15.97:0/2706353486 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55967430f800 0x55967430c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:03.385 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.377+0000 7f88b23e5640 1 -- 172.21.15.97:0/2706353486 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559674304f40 msgr2=0x55967430c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:38:03.385 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.377+0000 7f88b23e5640 1 --2- 172.21.15.97:0/2706353486 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559674304f40 0x55967430c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:03.385 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.377+0000 7f88b23e5640 1 -- 172.21.15.97:0/2706353486 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559674048620 con 0x559674304620 2024-04-23T10:38:03.385 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.377+0000 7f88b2be6640 1 --2- 172.21.15.97:0/2706353486 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55967430f800 0x55967430c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:38:03.385 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.377+0000 7f88b23e5640 1 --2- 172.21.15.97:0/2706353486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559674304620 0x55967430f170 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f88a0098980 tx=0x7f88a009c0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c18f947dec807bcf server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:38:03.385 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.377+0000 7f8895ffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:38:03.386 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.377+0000 7f8895ffb640 1 -- 172.21.15.97:0/2706353486 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a001d040 con 0x559674304620 2024-04-23T10:38:03.386 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.377+0000 7f8895ffb640 1 -- 172.21.15.97:0/2706353486 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8484+0+0 (secure 0 0 0) 0x7f88a00d00d0 con 0x559674304620 2024-04-23T10:38:03.386 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.377+0000 7f88b7433740 1 -- 172.21.15.97:0/2706353486 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88a80118f0 con 0x559674304620 2024-04-23T10:38:03.386 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.377+0000 7f8895ffb640 1 -- 172.21.15.97:0/2706353486 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f88a0005290 con 0x559674304620 2024-04-23T10:38:03.386 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.378+0000 7f88b7433740 10 client.17253 fetch_fsmap learned FSMap version 26 2024-04-23T10:38:03.386 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.378+0000 7f88b7433740 1 -- 172.21.15.97:0/2706353486 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55967431e370 con 0x559674304620 2024-04-23T10:38:03.386 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.378+0000 7f8895ffb640 1 -- 172.21.15.97:0/2706353486 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88a80118f0 con 0x559674304620 2024-04-23T10:38:03.386 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.378+0000 7f8895ffb640 1 -- 172.21.15.97:0/2706353486 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f88a0098270 con 0x559674304620 2024-04-23T10:38:03.386 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.378+0000 7f88b7433740 10 client.17253 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:38:03.386 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.378+0000 7f88b7433740 10 client.17253 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:38:03.386 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.378+0000 7f88b7433740 20 client.17253 populate_metadata read hostname 'smithi097' 2024-04-23T10:38:03.386 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.378+0000 7f88b7433740 1 --2- 172.21.15.97:0/2706353486 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55967431f850 0x559674321c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:03.386 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.378+0000 7f88b7433740 4 client.17253 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:38:03.386 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.378+0000 7f88b7433740 1 -- 172.21.15.97:0/2706353486 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559674277390 con 0x55967431f850 2024-04-23T10:38:03.387 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.378+0000 7f88b1be4640 1 --2- 172.21.15.97:0/2706353486 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55967431f850 0x559674321c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:03.387 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.379+0000 7f8895ffb640 10 client.17253 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:38:03.387 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.379+0000 7f88b1be4640 1 --2- 172.21.15.97:0/2706353486 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55967431f850 0x559674321c60 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7bbd417f9d704b7 server_cookie=359233464c80a021 in_seq=0 out_seq=0 2024-04-23T10:38:03.387 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.379+0000 7f8895ffb640 1 -- 172.21.15.97:0/2706353486 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559674277390 con 0x55967431f850 2024-04-23T10:38:03.387 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.379+0000 7f8895ffb640 10 client.17253 handle_command_reply: tid=0 2024-04-23T10:38:03.387 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.379+0000 7f88b7433740 1 client.17253 shutdown 2024-04-23T10:38:03.387 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.379+0000 7f88b7433740 1 -- 172.21.15.97:0/2706353486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559674304620 msgr2=0x55967430f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:03.387 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.379+0000 7f88b7433740 1 --2- 172.21.15.97:0/2706353486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559674304620 0x55967430f170 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f88a0098980 tx=0x7f88a009c0e0 comp rx=0 tx=0).stop 2024-04-23T10:38:03.387 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.379+0000 7f88b7433740 1 -- 172.21.15.97:0/2706353486 shutdown_connections 2024-04-23T10:38:03.387 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.379+0000 7f88b7433740 1 --2- 172.21.15.97:0/2706353486 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55967431f850 0x559674321c60 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:03.387 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.379+0000 7f88b7433740 1 --2- 172.21.15.97:0/2706353486 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55967430f800 0x55967430c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:03.387 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.379+0000 7f88b7433740 1 --2- 172.21.15.97:0/2706353486 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559674304f40 0x55967430c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:03.387 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.379+0000 7f88b7433740 1 --2- 172.21.15.97:0/2706353486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559674304620 0x55967430f170 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:03.387 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.379+0000 7f88b7433740 1 -- 172.21.15.97:0/2706353486 >> 172.21.15.97:0/2706353486 conn(0x559674300b30 msgr2=0x5596743068b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:38:03.388 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.379+0000 7f8895ffb640 0 client.17253 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:38:03.388 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.380+0000 7f88b7433740 1 -- 172.21.15.97:0/2706353486 shutdown_connections 2024-04-23T10:38:03.388 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.380+0000 7f88b7433740 1 -- 172.21.15.97:0/2706353486 wait complete. 2024-04-23T10:38:03.388 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.380+0000 7f88b7433740 20 client.17253 trim_cache size 0 max 16384 2024-04-23T10:38:03.398 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.394+0000 7f88b7433740 1 Processor -- start 2024-04-23T10:38:03.398 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.394+0000 7f88b7433740 1 -- start start 2024-04-23T10:38:03.398 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.394+0000 7f88b7433740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559674304620 0x55967431b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:03.398 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.394+0000 7f88b7433740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559674304f40 0x55967430eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:03.398 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.394+0000 7f88b7433740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55967430f800 0x559674327540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:03.398 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.395+0000 7f88b7433740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f88ac1adac0 con 0x559674304620 2024-04-23T10:38:03.398 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.395+0000 7f88b7433740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f88ac1af600 con 0x55967430f800 2024-04-23T10:38:03.398 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.395+0000 7f88b7433740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f88ac1ae190 con 0x559674304f40 2024-04-23T10:38:03.398 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.395+0000 7f88b23e5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559674304620 0x55967431b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:03.398 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.395+0000 7f88b23e5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559674304620 0x55967431b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38120/0 (socket says 172.21.15.97:38120) 2024-04-23T10:38:03.399 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.395+0000 7f88b23e5640 1 -- 172.21.15.97:0/3116047749 learned_addr learned my addr 172.21.15.97:0/3116047749 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:38:03.399 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.395+0000 7f88b23e5640 1 -- 172.21.15.97:0/3116047749 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559674304f40 msgr2=0x55967430eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:38:03.399 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.395+0000 7f88b23e5640 1 --2- 172.21.15.97:0/3116047749 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559674304f40 0x55967430eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:03.399 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.395+0000 7f88b23e5640 1 -- 172.21.15.97:0/3116047749 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55967430f800 msgr2=0x559674327540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:38:03.399 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.395+0000 7f88b23e5640 1 --2- 172.21.15.97:0/3116047749 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55967430f800 0x559674327540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:03.399 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.395+0000 7f88b23e5640 1 -- 172.21.15.97:0/3116047749 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55967430cd30 con 0x559674304620 2024-04-23T10:38:03.399 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.395+0000 7f88b23e5640 1 --2- 172.21.15.97:0/3116047749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559674304620 0x55967431b2f0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f88a0098ef0 tx=0x7f88a0087050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e813ecfbc80af363 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:38:03.399 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.395+0000 7f8897fff640 1 -- 172.21.15.97:0/3116047749 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a001d070 con 0x559674304620 2024-04-23T10:38:03.399 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.395+0000 7f8897fff640 1 -- 172.21.15.97:0/3116047749 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88a00a08d0 con 0x559674304620 2024-04-23T10:38:03.399 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.395+0000 7f8897fff640 1 -- 172.21.15.97:0/3116047749 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a0098b90 con 0x559674304620 2024-04-23T10:38:03.399 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.396+0000 7f88b7433740 1 -- 172.21.15.97:0/3116047749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559674304620 msgr2=0x55967431b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:03.399 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.396+0000 7f88b7433740 1 --2- 172.21.15.97:0/3116047749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559674304620 0x55967431b2f0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f88a0098ef0 tx=0x7f88a0087050 comp rx=0 tx=0).stop 2024-04-23T10:38:03.399 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.396+0000 7f88b7433740 1 -- 172.21.15.97:0/3116047749 shutdown_connections 2024-04-23T10:38:03.399 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.396+0000 7f88b7433740 1 --2- 172.21.15.97:0/3116047749 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55967430f800 0x559674327540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:03.399 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.396+0000 7f88b7433740 1 --2- 172.21.15.97:0/3116047749 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559674304f40 0x55967430eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:03.400 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.396+0000 7f88b7433740 1 --2- 172.21.15.97:0/3116047749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559674304620 0x55967431b2f0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:03.400 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.396+0000 7f88b7433740 1 -- 172.21.15.97:0/3116047749 >> 172.21.15.97:0/3116047749 conn(0x559674300b30 msgr2=0x55967430c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:38:03.400 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.396+0000 7f88b7433740 1 -- 172.21.15.97:0/3116047749 shutdown_connections 2024-04-23T10:38:03.400 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.396+0000 7f88b7433740 1 -- 172.21.15.97:0/3116047749 wait complete. 2024-04-23T10:38:03.400 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.396+0000 7f88b7433740 1 -- start start 2024-04-23T10:38:03.400 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.396+0000 7f88b7433740 1 Processor -- start 2024-04-23T10:38:03.400 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.396+0000 7f88b7433740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559674304620 0x559674317800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:03.400 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.396+0000 7f88b7433740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559674304f40 0x559674317d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:03.400 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.396+0000 7f88b7433740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55967430f800 0x559674308c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:03.400 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.396+0000 7f88b7433740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f88ac1aef30 con 0x559674304f40 2024-04-23T10:38:03.400 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.396+0000 7f88b7433740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f88ac1ae860 con 0x559674304620 2024-04-23T10:38:03.400 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.396+0000 7f88b7433740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f88ac1abac0 con 0x55967430f800 2024-04-23T10:38:03.400 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.396+0000 7f88b7433740 10 client.0 _ll_register_callbacks cb 0x7f88a009c2f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:38:03.400 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.397+0000 7f88b1be4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559674304f40 0x559674317d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:03.401 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.397+0000 7f88b1be4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559674304f40 0x559674317d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38124/0 (socket says 172.21.15.97:38124) 2024-04-23T10:38:03.401 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.397+0000 7f88b1be4640 1 -- 172.21.15.97:0/2120150980 learned_addr learned my addr 172.21.15.97:0/2120150980 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:38:03.401 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.397+0000 7f88b2be6640 1 --2- 172.21.15.97:0/2120150980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55967430f800 0x559674308c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:03.401 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.397+0000 7f88b23e5640 1 --2- 172.21.15.97:0/2120150980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559674304620 0x559674317800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:03.401 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.397+0000 7f88b1be4640 1 -- 172.21.15.97:0/2120150980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55967430f800 msgr2=0x559674308c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:38:03.401 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.397+0000 7f88b1be4640 1 --2- 172.21.15.97:0/2120150980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55967430f800 0x559674308c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:03.401 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.397+0000 7f88b1be4640 1 -- 172.21.15.97:0/2120150980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559674304620 msgr2=0x559674317800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:38:03.401 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.397+0000 7f88b1be4640 1 --2- 172.21.15.97:0/2120150980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559674304620 0x559674317800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:03.401 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.397+0000 7f88b1be4640 1 -- 172.21.15.97:0/2120150980 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559674304d70 con 0x559674304f40 2024-04-23T10:38:03.401 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.397+0000 7f88b2be6640 1 --2- 172.21.15.97:0/2120150980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55967430f800 0x559674308c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:38:03.401 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.397+0000 7f88b23e5640 1 --2- 172.21.15.97:0/2120150980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559674304620 0x559674317800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:38:03.401 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.397+0000 7f88b1be4640 1 --2- 172.21.15.97:0/2120150980 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559674304f40 0x559674317d40 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f88a8010440 tx=0x7f88a800b3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51490c96e09042a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:38:03.401 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.397+0000 7f8895ffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:38:03.401 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.397+0000 7f8895ffb640 1 -- 172.21.15.97:0/2120150980 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a8015040 con 0x559674304f40 2024-04-23T10:38:03.401 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.397+0000 7f8895ffb640 1 -- 172.21.15.97:0/2120150980 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8484+0+0 (secure 0 0 0) 0x7f88a8004450 con 0x559674304f40 2024-04-23T10:38:03.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.397+0000 7f88b7433740 1 -- 172.21.15.97:0/2120150980 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88a00a08d0 con 0x559674304f40 2024-04-23T10:38:03.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.397+0000 7f8895ffb640 1 -- 172.21.15.97:0/2120150980 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f88a800b8c0 con 0x559674304f40 2024-04-23T10:38:03.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.397+0000 7f88b7433740 10 client.17259 fetch_fsmap learned FSMap version 26 2024-04-23T10:38:03.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.397+0000 7f88b7433740 1 -- 172.21.15.97:0/2120150980 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5596743857e0 con 0x559674304f40 2024-04-23T10:38:03.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.397+0000 7f8895ffb640 1 -- 172.21.15.97:0/2120150980 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88a00a08d0 con 0x559674304f40 2024-04-23T10:38:03.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.398+0000 7f8895ffb640 1 -- 172.21.15.97:0/2120150980 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f88a802e070 con 0x559674304f40 2024-04-23T10:38:03.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.398+0000 7f88b7433740 10 client.17259 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:38:03.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.398+0000 7f88b7433740 10 client.17259 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:38:03.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.398+0000 7f88b7433740 20 client.17259 populate_metadata read hostname 'smithi097' 2024-04-23T10:38:03.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.398+0000 7f88b7433740 1 --2- 172.21.15.97:0/2120150980 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x559674385e40 0x559674388260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:03.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.398+0000 7f88b7433740 4 client.17259 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:38:03.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.398+0000 7f88b7433740 1 -- 172.21.15.97:0/2120150980 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559674271200 con 0x559674385e40 2024-04-23T10:38:03.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.398+0000 7f88b23e5640 1 --2- 172.21.15.97:0/2120150980 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x559674385e40 0x559674388260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:03.402 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.398+0000 7f88b23e5640 1 --2- 172.21.15.97:0/2120150980 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x559674385e40 0x559674388260 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1733897b944049a6 server_cookie=c3f88767d32473ec in_seq=0 out_seq=0 2024-04-23T10:38:03.403 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.398+0000 7f8895ffb640 10 client.17259 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:38:03.413 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.414+0000 7f8895ffb640 1 -- 172.21.15.97:0/2120150980 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559674271200 con 0x559674385e40 2024-04-23T10:38:03.413 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.414+0000 7f8895ffb640 10 client.17259 handle_command_reply: tid=0 2024-04-23T10:38:03.413 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.414+0000 7f88b7433740 1 client.17259 shutdown 2024-04-23T10:38:03.413 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.414+0000 7f88b7433740 1 -- 172.21.15.97:0/2120150980 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559674304f40 msgr2=0x559674317d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:03.413 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.414+0000 7f88b7433740 1 --2- 172.21.15.97:0/2120150980 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559674304f40 0x559674317d40 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f88a8010440 tx=0x7f88a800b3a0 comp rx=0 tx=0).stop 2024-04-23T10:38:03.413 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.414+0000 7f88b7433740 1 -- 172.21.15.97:0/2120150980 shutdown_connections 2024-04-23T10:38:03.414 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.414+0000 7f88b7433740 1 --2- 172.21.15.97:0/2120150980 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x559674385e40 0x559674388260 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:03.414 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.414+0000 7f88b7433740 1 --2- 172.21.15.97:0/2120150980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55967430f800 0x559674308c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:03.414 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.414+0000 7f88b7433740 1 --2- 172.21.15.97:0/2120150980 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559674304f40 0x559674317d40 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:03.414 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.414+0000 7f88b7433740 1 --2- 172.21.15.97:0/2120150980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559674304620 0x559674317800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:03.414 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.414+0000 7f88b7433740 1 -- 172.21.15.97:0/2120150980 >> 172.21.15.97:0/2120150980 conn(0x559674300b30 msgr2=0x5596742ff420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:38:03.414 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.414+0000 7f8895ffb640 0 client.17259 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:38:03.414 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.414+0000 7f88b7433740 1 -- 172.21.15.97:0/2120150980 shutdown_connections 2024-04-23T10:38:03.414 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.414+0000 7f88b7433740 1 -- 172.21.15.97:0/2120150980 wait complete. 2024-04-23T10:38:03.414 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.414+0000 7f88b7433740 20 client.17259 trim_cache size 0 max 16384 2024-04-23T10:38:03.414 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:38:03.415 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:38:03.415 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:38:03.415 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:38:03.415 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.415+0000 7f88b4649640 1 -- 172.21.15.97:0/2461560197 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f88980a3e40 msgr2=0x7f88980a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:03.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.415+0000 7f88b4649640 1 --2- 172.21.15.97:0/2461560197 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f88980a3e40 0x7f88980a6300 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x559674301f50 tx=0x7f88a4005f80 comp rx=0 tx=0).stop 2024-04-23T10:38:03.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.415+0000 7f88b4649640 1 -- 172.21.15.97:0/2461560197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88ac1031e0 msgr2=0x7f88ac198690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:03.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.415+0000 7f88b4649640 1 --2- 172.21.15.97:0/2461560197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88ac1031e0 0x7f88ac198690 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f88a000c890 tx=0x7f88a000cd50 comp rx=0 tx=0).stop 2024-04-23T10:38:03.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.415+0000 7f88b4649640 1 -- 172.21.15.97:0/2461560197 shutdown_connections 2024-04-23T10:38:03.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.415+0000 7f88b4649640 1 --2- 172.21.15.97:0/2461560197 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f88980a3e40 0x7f88980a6300 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:03.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.415+0000 7f88b4649640 1 --2- 172.21.15.97:0/2461560197 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88ac10c5b0 0x7f88ac19cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:03.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.415+0000 7f88b4649640 1 --2- 172.21.15.97:0/2461560197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88ac103b00 0x7f88ac198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:03.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.415+0000 7f88b4649640 1 --2- 172.21.15.97:0/2461560197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88ac1031e0 0x7f88ac198690 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:03.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.415+0000 7f88b4649640 1 -- 172.21.15.97:0/2461560197 >> 172.21.15.97:0/2461560197 conn(0x7f88ac077f60 msgr2=0x7f88ac0fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:38:03.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.415+0000 7f88b4649640 1 -- 172.21.15.97:0/2461560197 shutdown_connections 2024-04-23T10:38:03.416 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:03.415+0000 7f88b4649640 1 -- 172.21.15.97:0/2461560197 wait complete. 2024-04-23T10:38:04.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:38:04 smithi196 ceph-mon[32919]: pgmap v369: 97 pgs: 97 active+clean; 5.3 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 87 KiB/s wr, 37 op/s 2024-04-23T10:38:04.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:38:04 smithi097 ceph-mon[29197]: pgmap v369: 97 pgs: 97 active+clean; 5.3 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 87 KiB/s wr, 37 op/s 2024-04-23T10:38:04.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:38:04 smithi183 ceph-mon[32317]: pgmap v369: 97 pgs: 97 active+clean; 5.3 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 87 KiB/s wr, 37 op/s 2024-04-23T10:38:07.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:38:07 smithi183 ceph-mon[32317]: pgmap v370: 97 pgs: 97 active+clean; 5.3 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 63 KiB/s wr, 28 op/s 2024-04-23T10:38:07.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:38:07 smithi183 ceph-mon[32317]: osdmap e100: 12 total, 12 up, 12 in 2024-04-23T10:38:07.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:38:07 smithi196 ceph-mon[32919]: pgmap v370: 97 pgs: 97 active+clean; 5.3 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 63 KiB/s wr, 28 op/s 2024-04-23T10:38:07.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:38:07 smithi196 ceph-mon[32919]: osdmap e100: 12 total, 12 up, 12 in 2024-04-23T10:38:07.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:38:07 smithi097 ceph-mon[29197]: pgmap v370: 97 pgs: 97 active+clean; 5.3 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 63 KiB/s wr, 28 op/s 2024-04-23T10:38:07.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:38:07 smithi097 ceph-mon[29197]: osdmap e100: 12 total, 12 up, 12 in 2024-04-23T10:38:09.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:38:09 smithi183 ceph-mon[32317]: pgmap v372: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 62 KiB/s wr, 35 op/s 2024-04-23T10:38:09.413 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:38:09 smithi097 ceph-mon[29197]: pgmap v372: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 62 KiB/s wr, 35 op/s 2024-04-23T10:38:09.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:38:09 smithi196 ceph-mon[32919]: pgmap v372: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 62 KiB/s wr, 35 op/s 2024-04-23T10:38:11.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:38:11 smithi183 ceph-mon[32317]: pgmap v373: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 62 KiB/s wr, 35 op/s 2024-04-23T10:38:11.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:38:11 smithi196 ceph-mon[32919]: pgmap v373: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 62 KiB/s wr, 35 op/s 2024-04-23T10:38:11.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:38:11 smithi097 ceph-mon[29197]: pgmap v373: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 62 KiB/s wr, 35 op/s 2024-04-23T10:38:12.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:38:12 smithi183 ceph-mon[32317]: osdmap e101: 12 total, 12 up, 12 in 2024-04-23T10:38:12.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:38:12 smithi196 ceph-mon[32919]: osdmap e101: 12 total, 12 up, 12 in 2024-04-23T10:38:12.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:38:12 smithi097 ceph-mon[29197]: osdmap e101: 12 total, 12 up, 12 in 2024-04-23T10:38:13.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:38:13 smithi183 ceph-mon[32317]: pgmap v375: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-04-23T10:38:13.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:38:13 smithi196 ceph-mon[32919]: pgmap v375: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-04-23T10:38:13.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:38:13 smithi097 ceph-mon[29197]: pgmap v375: 97 pgs: 97 active+clean; 1.3 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-04-23T10:38:15.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:38:15 smithi196 ceph-mon[32919]: pgmap v376: 97 pgs: 97 active+clean; 1.4 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-04-23T10:38:15.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:38:15 smithi097 ceph-mon[29197]: pgmap v376: 97 pgs: 97 active+clean; 1.4 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-04-23T10:38:15.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:38:15 smithi183 ceph-mon[32317]: pgmap v376: 97 pgs: 97 active+clean; 1.4 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 18 op/s 2024-04-23T10:38:16.513 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.512+0000 7f66fab0f640 1 -- 172.21.15.97:0/1946534167 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66f41021c0 msgr2=0x7f66f41025a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:16.514 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.512+0000 7f66fab0f640 1 --2- 172.21.15.97:0/1946534167 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66f41021c0 0x7f66f41025a0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f66e8009900 tx=0x7f66e802a410 comp rx=0 tx=0).stop 2024-04-23T10:38:16.514 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.512+0000 7f66fab0f640 1 -- 172.21.15.97:0/1946534167 shutdown_connections 2024-04-23T10:38:16.514 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.512+0000 7f66fab0f640 1 --2- 172.21.15.97:0/1946534167 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f66f4107290 0x7f66f410ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:16.514 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.512+0000 7f66fab0f640 1 --2- 172.21.15.97:0/1946534167 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66f4102b70 0x7f66f4106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:16.514 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.512+0000 7f66fab0f640 1 --2- 172.21.15.97:0/1946534167 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66f41021c0 0x7f66f41025a0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:16.514 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.512+0000 7f66fab0f640 1 -- 172.21.15.97:0/1946534167 >> 172.21.15.97:0/1946534167 conn(0x7f66f40fda20 msgr2=0x7f66f40ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:38:16.514 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.512+0000 7f66fab0f640 1 -- 172.21.15.97:0/1946534167 shutdown_connections 2024-04-23T10:38:16.514 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.512+0000 7f66fab0f640 1 -- 172.21.15.97:0/1946534167 wait complete. 2024-04-23T10:38:16.514 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.513+0000 7f66fab0f640 1 Processor -- start 2024-04-23T10:38:16.514 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.513+0000 7f66fab0f640 1 -- start start 2024-04-23T10:38:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.513+0000 7f66fab0f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66f41021c0 0x7f66f4198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.513+0000 7f66fab0f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66f4107290 0x7f66f4198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.513+0000 7f66fab0f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f66f419cfa0 0x7f66f419d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.513+0000 7f66fab0f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f66f41121f0 con 0x7f66f41021c0 2024-04-23T10:38:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.513+0000 7f66fab0f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f66f4112070 con 0x7f66f419cfa0 2024-04-23T10:38:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.513+0000 7f66fab0f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f66f4112370 con 0x7f66f4107290 2024-04-23T10:38:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.513+0000 7f66f88ab640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66f41021c0 0x7f66f4198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.513+0000 7f66f88ab640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66f41021c0 0x7f66f4198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40236/0 (socket says 172.21.15.97:40236) 2024-04-23T10:38:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.513+0000 7f66f88ab640 1 -- 172.21.15.97:0/2829113574 learned_addr learned my addr 172.21.15.97:0/2829113574 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:38:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.513+0000 7f66f3fff640 1 --2- 172.21.15.97:0/2829113574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66f4107290 0x7f66f4198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.513+0000 7f66f90ac640 1 --2- 172.21.15.97:0/2829113574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f66f419cfa0 0x7f66f419d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.513+0000 7f66f88ab640 1 -- 172.21.15.97:0/2829113574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66f4107290 msgr2=0x7f66f4198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.513+0000 7f66f88ab640 1 --2- 172.21.15.97:0/2829113574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66f4107290 0x7f66f4198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.513+0000 7f66f88ab640 1 -- 172.21.15.97:0/2829113574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f66f419cfa0 msgr2=0x7f66f419d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.513+0000 7f66f88ab640 1 --2- 172.21.15.97:0/2829113574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f66f419cfa0 0x7f66f419d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.513+0000 7f66f88ab640 1 -- 172.21.15.97:0/2829113574 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66f419da90 con 0x7f66f41021c0 2024-04-23T10:38:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.514+0000 7f66f88ab640 1 --2- 172.21.15.97:0/2829113574 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66f41021c0 0x7f66f4198730 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f66e8005940 tx=0x7f66e80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:38:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.514+0000 7f66f1ffb640 1 -- 172.21.15.97:0/2829113574 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e803f030 con 0x7f66f41021c0 2024-04-23T10:38:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.514+0000 7f66f1ffb640 1 -- 172.21.15.97:0/2829113574 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66e8003cb0 con 0x7f66f41021c0 2024-04-23T10:38:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.514+0000 7f66fab0f640 1 -- 172.21.15.97:0/2829113574 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66f419dd20 con 0x7f66f41021c0 2024-04-23T10:38:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.514+0000 7f66f1ffb640 1 -- 172.21.15.97:0/2829113574 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e8035070 con 0x7f66f41021c0 2024-04-23T10:38:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.514+0000 7f66fab0f640 1 -- 172.21.15.97:0/2829113574 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66f41a5540 con 0x7f66f41021c0 2024-04-23T10:38:16.524 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.516+0000 7f66f1ffb640 1 -- 172.21.15.97:0/2829113574 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f66e8042030 con 0x7f66f41021c0 2024-04-23T10:38:16.524 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.517+0000 7f66fd822740 1 Processor -- start 2024-04-23T10:38:16.524 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.517+0000 7f66fd822740 1 -- start start 2024-04-23T10:38:16.524 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.517+0000 7f66fd822740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5565a8959620 0x5565a8959a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:16.524 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.517+0000 7f66fd822740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565a8959f40 0x5565a89642c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:16.524 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.517+0000 7f66fd822740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565a8964800 0x5565a8966bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.517+0000 7f66fd822740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5565a88be970 con 0x5565a8959f40 2024-04-23T10:38:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.517+0000 7f66fd822740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5565a885b4a0 con 0x5565a8959620 2024-04-23T10:38:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.517+0000 7f66fd822740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5565a88ba930 con 0x5565a8964800 2024-04-23T10:38:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.517+0000 7f66f1ffb640 1 --2- 172.21.15.97:0/2829113574 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f66e40a3e40 0x7f66e40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.517+0000 7f66f88ab640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565a8959f40 0x5565a89642c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.517+0000 7f66f88ab640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565a8959f40 0x5565a89642c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40252/0 (socket says 172.21.15.97:40252) 2024-04-23T10:38:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.517+0000 7f66f88ab640 1 -- 172.21.15.97:0/484700067 learned_addr learned my addr 172.21.15.97:0/484700067 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:38:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.517+0000 7f66f90ac640 1 --2- 172.21.15.97:0/2829113574 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f66e40a3e40 0x7f66e40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.518+0000 7f66f90ac640 1 --2- 172.21.15.97:0/484700067 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5565a8959620 0x5565a8959a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.518+0000 7f66f3fff640 1 --2- 172.21.15.97:0/484700067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565a8964800 0x5565a8966bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.518+0000 7f66f1ffb640 1 -- 172.21.15.97:0/2829113574 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8484+0+0 (secure 0 0 0) 0x7f66e80f0f80 con 0x7f66f41021c0 2024-04-23T10:38:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.518+0000 7f66f88ab640 1 -- 172.21.15.97:0/484700067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565a8964800 msgr2=0x5565a8966bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:38:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.518+0000 7f66f88ab640 1 --2- 172.21.15.97:0/484700067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565a8964800 0x5565a8966bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.518+0000 7f66f88ab640 1 -- 172.21.15.97:0/484700067 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5565a8959620 msgr2=0x5565a8959a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:38:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.518+0000 7f66f88ab640 1 --2- 172.21.15.97:0/484700067 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5565a8959620 0x5565a8959a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.518+0000 7f66f88ab640 1 -- 172.21.15.97:0/484700067 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565a85d1210 con 0x5565a8959f40 2024-04-23T10:38:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.518+0000 7f66f3fff640 1 --2- 172.21.15.97:0/484700067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565a8964800 0x5565a8966bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:38:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.518+0000 7f66f90ac640 1 --2- 172.21.15.97:0/2829113574 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f66e40a3e40 0x7f66e40a6300 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f66ec005cc0 tx=0x7f66ec00aa00 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:38:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.518+0000 7f66f88ab640 1 --2- 172.21.15.97:0/484700067 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565a8959f40 0x5565a89642c0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f66e803f000 tx=0x7f66e802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a84ebf24f295e987 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:38:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.518+0000 7f66deffd640 1 -- 172.21.15.97:0/484700067 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e8049070 con 0x5565a8959f40 2024-04-23T10:38:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.518+0000 7f66deffd640 1 -- 172.21.15.97:0/484700067 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66e80bcf90 con 0x5565a8959f40 2024-04-23T10:38:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.518+0000 7f66deffd640 1 -- 172.21.15.97:0/484700067 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e803bbf0 con 0x5565a8959f40 2024-04-23T10:38:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.519+0000 7f66fd822740 1 -- 172.21.15.97:0/484700067 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565a8959f40 msgr2=0x5565a89642c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.519+0000 7f66fd822740 1 --2- 172.21.15.97:0/484700067 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565a8959f40 0x5565a89642c0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f66e803f000 tx=0x7f66e802a410 comp rx=0 tx=0).stop 2024-04-23T10:38:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.519+0000 7f66fd822740 1 -- 172.21.15.97:0/484700067 shutdown_connections 2024-04-23T10:38:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.519+0000 7f66fd822740 1 --2- 172.21.15.97:0/484700067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565a8964800 0x5565a8966bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.519+0000 7f66fd822740 1 --2- 172.21.15.97:0/484700067 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565a8959f40 0x5565a89642c0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.519+0000 7f66fd822740 1 --2- 172.21.15.97:0/484700067 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5565a8959620 0x5565a8959a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.519+0000 7f66fd822740 1 -- 172.21.15.97:0/484700067 >> 172.21.15.97:0/484700067 conn(0x5565a8955b30 msgr2=0x5565a8955f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:38:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.519+0000 7f66fd822740 1 -- 172.21.15.97:0/484700067 shutdown_connections 2024-04-23T10:38:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.519+0000 7f66fd822740 1 -- 172.21.15.97:0/484700067 wait complete. 2024-04-23T10:38:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.519+0000 7f66fd822740 1 -- start start 2024-04-23T10:38:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.519+0000 7f66fd822740 1 Processor -- start 2024-04-23T10:38:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.520+0000 7f66fd822740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565a8959620 0x5565a8964180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.520+0000 7f66fd822740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5565a8959f40 0x5565a8961470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.520+0000 7f66fd822740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565a8964800 0x5565a89619f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.520+0000 7f66fd822740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5565a89671f0 con 0x5565a8964800 2024-04-23T10:38:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.520+0000 7f66fd822740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5565a88590b0 con 0x5565a8959f40 2024-04-23T10:38:16.528 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.520+0000 7f66fd822740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5565a8967370 con 0x5565a8959620 2024-04-23T10:38:16.528 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.520+0000 7f66fd822740 10 client.0 _ll_register_callbacks cb 0x5565a8846d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:38:16.528 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.520+0000 7f66f90ac640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565a8964800 0x5565a89619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:16.528 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.520+0000 7f66f90ac640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565a8964800 0x5565a89619f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40254/0 (socket says 172.21.15.97:40254) 2024-04-23T10:38:16.528 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.520+0000 7f66f90ac640 1 -- 172.21.15.97:0/2440220229 learned_addr learned my addr 172.21.15.97:0/2440220229 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:38:16.528 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.520+0000 7f66f88ab640 1 --2- 172.21.15.97:0/2440220229 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565a8959620 0x5565a8964180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:16.528 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.520+0000 7f66f3fff640 1 --2- 172.21.15.97:0/2440220229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5565a8959f40 0x5565a8961470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:16.528 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.520+0000 7f66f90ac640 1 -- 172.21.15.97:0/2440220229 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565a8959620 msgr2=0x5565a8964180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:38:16.528 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.520+0000 7f66f90ac640 1 --2- 172.21.15.97:0/2440220229 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565a8959620 0x5565a8964180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:16.528 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.520+0000 7f66f90ac640 1 -- 172.21.15.97:0/2440220229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5565a8959f40 msgr2=0x5565a8961470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:38:16.528 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.520+0000 7f66f90ac640 1 --2- 172.21.15.97:0/2440220229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5565a8959f40 0x5565a8961470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:16.528 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.520+0000 7f66f90ac640 1 -- 172.21.15.97:0/2440220229 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565a869d620 con 0x5565a8964800 2024-04-23T10:38:16.528 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.520+0000 7f66f90ac640 1 --2- 172.21.15.97:0/2440220229 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565a8964800 0x5565a89619f0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f66ec00f630 tx=0x7f66ec00a4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28c49db1658816ed server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:38:16.529 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.520+0000 7f66d3fff640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:38:16.529 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.521+0000 7f66d3fff640 1 -- 172.21.15.97:0/2440220229 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ec024070 con 0x5565a8964800 2024-04-23T10:38:16.529 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.521+0000 7f66d3fff640 1 -- 172.21.15.97:0/2440220229 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8484+0+0 (secure 0 0 0) 0x7f66ec01f910 con 0x5565a8964800 2024-04-23T10:38:16.529 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.521+0000 7f66fd822740 1 -- 172.21.15.97:0/2440220229 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66e80bcf90 con 0x5565a8964800 2024-04-23T10:38:16.529 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.521+0000 7f66d3fff640 1 -- 172.21.15.97:0/2440220229 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f66ec004b20 con 0x5565a8964800 2024-04-23T10:38:16.529 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.521+0000 7f66fd822740 10 client.17271 fetch_fsmap learned FSMap version 26 2024-04-23T10:38:16.529 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.521+0000 7f66fd822740 1 -- 172.21.15.97:0/2440220229 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5565a89730b0 con 0x5565a8964800 2024-04-23T10:38:16.529 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.521+0000 7f66d3fff640 1 -- 172.21.15.97:0/2440220229 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66e80bcf90 con 0x5565a8964800 2024-04-23T10:38:16.529 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.521+0000 7f66d3fff640 1 -- 172.21.15.97:0/2440220229 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f66ec0054d0 con 0x5565a8964800 2024-04-23T10:38:16.529 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.521+0000 7f66fd822740 10 client.17271 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:38:16.529 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.521+0000 7f66fd822740 10 client.17271 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:38:16.529 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.521+0000 7f66fd822740 20 client.17271 populate_metadata read hostname 'smithi097' 2024-04-23T10:38:16.529 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.521+0000 7f66fd822740 1 --2- 172.21.15.97:0/2440220229 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5565a89745e0 0x5565a89769d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:16.530 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.521+0000 7f66fd822740 4 client.17271 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:38:16.530 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.521+0000 7f66fd822740 1 -- 172.21.15.97:0/2440220229 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565a88cc390 con 0x5565a89745e0 2024-04-23T10:38:16.530 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.521+0000 7f66f88ab640 1 --2- 172.21.15.97:0/2440220229 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5565a89745e0 0x5565a89769d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:16.530 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.522+0000 7f66f88ab640 1 --2- 172.21.15.97:0/2440220229 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5565a89745e0 0x5565a89769d0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=572796ce739ea330 server_cookie=ab4acc79ca440483 in_seq=0 out_seq=0 2024-04-23T10:38:16.530 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.522+0000 7f66d3fff640 10 client.17271 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:38:16.530 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.522+0000 7f66d3fff640 1 -- 172.21.15.97:0/2440220229 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5565a88cc390 con 0x5565a89745e0 2024-04-23T10:38:16.530 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.522+0000 7f66d3fff640 10 client.17271 handle_command_reply: tid=0 2024-04-23T10:38:16.530 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.522+0000 7f66fd822740 1 client.17271 shutdown 2024-04-23T10:38:16.530 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.523+0000 7f66fd822740 1 -- 172.21.15.97:0/2440220229 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565a8964800 msgr2=0x5565a89619f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:16.530 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.523+0000 7f66fd822740 1 --2- 172.21.15.97:0/2440220229 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565a8964800 0x5565a89619f0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f66ec00f630 tx=0x7f66ec00a4d0 comp rx=0 tx=0).stop 2024-04-23T10:38:16.530 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.523+0000 7f66fd822740 1 -- 172.21.15.97:0/2440220229 shutdown_connections 2024-04-23T10:38:16.530 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.523+0000 7f66fd822740 1 --2- 172.21.15.97:0/2440220229 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5565a89745e0 0x5565a89769d0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:16.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.523+0000 7f66fd822740 1 --2- 172.21.15.97:0/2440220229 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565a8964800 0x5565a89619f0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:16.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.523+0000 7f66fd822740 1 --2- 172.21.15.97:0/2440220229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5565a8959f40 0x5565a8961470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:16.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.523+0000 7f66fd822740 1 --2- 172.21.15.97:0/2440220229 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565a8959620 0x5565a8964180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:16.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.523+0000 7f66fd822740 1 -- 172.21.15.97:0/2440220229 >> 172.21.15.97:0/2440220229 conn(0x5565a8955b30 msgr2=0x5565a895b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:38:16.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.523+0000 7f66d3fff640 0 client.17271 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:38:16.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.523+0000 7f66fd822740 1 -- 172.21.15.97:0/2440220229 shutdown_connections 2024-04-23T10:38:16.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.523+0000 7f66fd822740 1 -- 172.21.15.97:0/2440220229 wait complete. 2024-04-23T10:38:16.531 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.523+0000 7f66fd822740 20 client.17271 trim_cache size 0 max 16384 2024-04-23T10:38:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.538+0000 7f66fd822740 1 Processor -- start 2024-04-23T10:38:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.538+0000 7f66fd822740 1 -- start start 2024-04-23T10:38:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.538+0000 7f66fd822740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565a8959620 0x5565a8963cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.538+0000 7f66fd822740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5565a8959f40 0x5565a897c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.538+0000 7f66fd822740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565a8964800 0x5565a897c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.538+0000 7f66fd822740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f66f41adaf0 con 0x5565a8964800 2024-04-23T10:38:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.538+0000 7f66fd822740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f66f41af630 con 0x5565a8959f40 2024-04-23T10:38:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.538+0000 7f66fd822740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f66f41ae1c0 con 0x5565a8959620 2024-04-23T10:38:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.538+0000 7f66f90ac640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565a8964800 0x5565a897c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.538+0000 7f66f90ac640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565a8964800 0x5565a897c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40262/0 (socket says 172.21.15.97:40262) 2024-04-23T10:38:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.538+0000 7f66f90ac640 1 -- 172.21.15.97:0/152783819 learned_addr learned my addr 172.21.15.97:0/152783819 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:38:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.538+0000 7f66f88ab640 1 --2- 172.21.15.97:0/152783819 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565a8959620 0x5565a8963cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.538+0000 7f66f90ac640 1 -- 172.21.15.97:0/152783819 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565a8959620 msgr2=0x5565a8963cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:38:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.538+0000 7f66f90ac640 1 --2- 172.21.15.97:0/152783819 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565a8959620 0x5565a8963cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.538+0000 7f66f90ac640 1 -- 172.21.15.97:0/152783819 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5565a8959f40 msgr2=0x5565a897c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:38:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.538+0000 7f66f90ac640 1 --2- 172.21.15.97:0/152783819 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5565a8959f40 0x5565a897c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.538+0000 7f66f90ac640 1 -- 172.21.15.97:0/152783819 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565a8966bf0 con 0x5565a8964800 2024-04-23T10:38:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.539+0000 7f66f90ac640 1 --2- 172.21.15.97:0/152783819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565a8964800 0x5565a897c820 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f66ec016040 tx=0x7f66ec01e5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c90165d2d1858504 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:38:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.539+0000 7f66de7fc640 1 -- 172.21.15.97:0/152783819 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ec038020 con 0x5565a8964800 2024-04-23T10:38:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.539+0000 7f66de7fc640 1 -- 172.21.15.97:0/152783819 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66ec01fbc0 con 0x5565a8964800 2024-04-23T10:38:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.539+0000 7f66de7fc640 1 -- 172.21.15.97:0/152783819 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ec024040 con 0x5565a8964800 2024-04-23T10:38:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.539+0000 7f66fd822740 1 -- 172.21.15.97:0/152783819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565a8964800 msgr2=0x5565a897c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.539+0000 7f66fd822740 1 --2- 172.21.15.97:0/152783819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565a8964800 0x5565a897c820 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f66ec016040 tx=0x7f66ec01e5f0 comp rx=0 tx=0).stop 2024-04-23T10:38:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.539+0000 7f66fd822740 1 -- 172.21.15.97:0/152783819 shutdown_connections 2024-04-23T10:38:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.539+0000 7f66fd822740 1 --2- 172.21.15.97:0/152783819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565a8964800 0x5565a897c820 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.539+0000 7f66fd822740 1 --2- 172.21.15.97:0/152783819 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5565a8959f40 0x5565a897c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.539+0000 7f66fd822740 1 --2- 172.21.15.97:0/152783819 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565a8959620 0x5565a8963cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.539+0000 7f66fd822740 1 -- 172.21.15.97:0/152783819 >> 172.21.15.97:0/152783819 conn(0x5565a8955b30 msgr2=0x5565a8961430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:38:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.540+0000 7f66fd822740 1 -- 172.21.15.97:0/152783819 shutdown_connections 2024-04-23T10:38:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.540+0000 7f66fd822740 1 -- 172.21.15.97:0/152783819 wait complete. 2024-04-23T10:38:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.540+0000 7f66fd822740 1 -- start start 2024-04-23T10:38:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.540+0000 7f66fd822740 1 Processor -- start 2024-04-23T10:38:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.540+0000 7f66fd822740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565a8959620 0x5565a8969a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.540+0000 7f66fd822740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5565a8959f40 0x5565a8969fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.540+0000 7f66fd822740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565a8964800 0x5565a89d2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.540+0000 7f66fd822740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f66f41aef60 con 0x5565a8959620 2024-04-23T10:38:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.540+0000 7f66fd822740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f66f41ae890 con 0x5565a8959f40 2024-04-23T10:38:16.545 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.540+0000 7f66fd822740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f66f41abaf0 con 0x5565a8964800 2024-04-23T10:38:16.545 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.540+0000 7f66fd822740 10 client.0 _ll_register_callbacks cb 0x7f66ec011d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:38:16.545 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.540+0000 7f66f88ab640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565a8959620 0x5565a8969a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:16.545 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.540+0000 7f66f88ab640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565a8959620 0x5565a8969a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40276/0 (socket says 172.21.15.97:40276) 2024-04-23T10:38:16.545 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.540+0000 7f66f88ab640 1 -- 172.21.15.97:0/2728317848 learned_addr learned my addr 172.21.15.97:0/2728317848 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:38:16.545 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.541+0000 7f66f3fff640 1 --2- 172.21.15.97:0/2728317848 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5565a8959f40 0x5565a8969fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:16.545 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.541+0000 7f66f90ac640 1 --2- 172.21.15.97:0/2728317848 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565a8964800 0x5565a89d2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:16.545 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.541+0000 7f66f88ab640 1 -- 172.21.15.97:0/2728317848 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565a8964800 msgr2=0x5565a89d2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:38:16.545 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.541+0000 7f66f88ab640 1 --2- 172.21.15.97:0/2728317848 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565a8964800 0x5565a89d2b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:16.545 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.541+0000 7f66f88ab640 1 -- 172.21.15.97:0/2728317848 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5565a8959f40 msgr2=0x5565a8969fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:38:16.545 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.541+0000 7f66f88ab640 1 --2- 172.21.15.97:0/2728317848 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5565a8959f40 0x5565a8969fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:16.545 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.541+0000 7f66f88ab640 1 -- 172.21.15.97:0/2728317848 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565a89d3190 con 0x5565a8959620 2024-04-23T10:38:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.541+0000 7f66f3fff640 1 --2- 172.21.15.97:0/2728317848 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5565a8959f40 0x5565a8969fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:38:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.541+0000 7f66f88ab640 1 --2- 172.21.15.97:0/2728317848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565a8959620 0x5565a8969a60 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f66e80bcf20 tx=0x7f66e8005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c420452534ccc2b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:38:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.541+0000 7f66d3fff640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:38:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.541+0000 7f66d3fff640 1 -- 172.21.15.97:0/2728317848 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e80da070 con 0x5565a8959620 2024-04-23T10:38:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.541+0000 7f66d3fff640 1 -- 172.21.15.97:0/2728317848 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8484+0+0 (secure 0 0 0) 0x7f66e80ee910 con 0x5565a8959620 2024-04-23T10:38:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.541+0000 7f66fd822740 1 -- 172.21.15.97:0/2728317848 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66ec01fbc0 con 0x5565a8959620 2024-04-23T10:38:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.541+0000 7f66d3fff640 1 -- 172.21.15.97:0/2728317848 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f66e8049040 con 0x5565a8959620 2024-04-23T10:38:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.541+0000 7f66fd822740 10 client.17277 fetch_fsmap learned FSMap version 26 2024-04-23T10:38:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.541+0000 7f66fd822740 1 -- 172.21.15.97:0/2728317848 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5565a89da650 con 0x5565a8959620 2024-04-23T10:38:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.541+0000 7f66d3fff640 1 -- 172.21.15.97:0/2728317848 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66ec01fbc0 con 0x5565a8959620 2024-04-23T10:38:16.546 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.541+0000 7f66d3fff640 1 -- 172.21.15.97:0/2728317848 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f66e8099020 con 0x5565a8959620 2024-04-23T10:38:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.542+0000 7f66fd822740 10 client.17277 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:38:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.542+0000 7f66fd822740 10 client.17277 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:38:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.542+0000 7f66fd822740 20 client.17277 populate_metadata read hostname 'smithi097' 2024-04-23T10:38:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.542+0000 7f66fd822740 1 --2- 172.21.15.97:0/2728317848 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5565a89dabc0 0x5565a89dcfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.542+0000 7f66fd822740 4 client.17277 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:38:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.542+0000 7f66fd822740 1 -- 172.21.15.97:0/2728317848 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5565a88c6200 con 0x5565a89dabc0 2024-04-23T10:38:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.542+0000 7f66f3fff640 1 --2- 172.21.15.97:0/2728317848 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5565a89dabc0 0x5565a89dcfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.542+0000 7f66f3fff640 1 --2- 172.21.15.97:0/2728317848 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5565a89dabc0 0x5565a89dcfe0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d444750e63b3abea server_cookie=72f00dfc908dfea1 in_seq=0 out_seq=0 2024-04-23T10:38:16.547 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.542+0000 7f66d3fff640 10 client.17277 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:38:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.555+0000 7f66d3fff640 1 -- 172.21.15.97:0/2728317848 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5565a88c6200 con 0x5565a89dabc0 2024-04-23T10:38:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.555+0000 7f66d3fff640 10 client.17277 handle_command_reply: tid=0 2024-04-23T10:38:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.555+0000 7f66fd822740 1 client.17277 shutdown 2024-04-23T10:38:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.555+0000 7f66fd822740 1 -- 172.21.15.97:0/2728317848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565a8959620 msgr2=0x5565a8969a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.555+0000 7f66fd822740 1 --2- 172.21.15.97:0/2728317848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565a8959620 0x5565a8969a60 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f66e80bcf20 tx=0x7f66e8005c00 comp rx=0 tx=0).stop 2024-04-23T10:38:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.555+0000 7f66fd822740 1 -- 172.21.15.97:0/2728317848 shutdown_connections 2024-04-23T10:38:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.555+0000 7f66fd822740 1 --2- 172.21.15.97:0/2728317848 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5565a89dabc0 0x5565a89dcfe0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.555+0000 7f66fd822740 1 --2- 172.21.15.97:0/2728317848 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565a8964800 0x5565a89d2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.555+0000 7f66fd822740 1 --2- 172.21.15.97:0/2728317848 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5565a8959f40 0x5565a8969fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.555+0000 7f66d3fff640 0 client.17277 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:38:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.555+0000 7f66fd822740 1 --2- 172.21.15.97:0/2728317848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5565a8959620 0x5565a8969a60 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.555+0000 7f66fd822740 1 -- 172.21.15.97:0/2728317848 >> 172.21.15.97:0/2728317848 conn(0x5565a8955b30 msgr2=0x5565a8954420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:38:16.556 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.555+0000 7f66fd822740 1 -- 172.21.15.97:0/2728317848 shutdown_connections 2024-04-23T10:38:16.556 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.555+0000 7f66fd822740 1 -- 172.21.15.97:0/2728317848 wait complete. 2024-04-23T10:38:16.556 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.555+0000 7f66fd822740 20 client.17277 trim_cache size 0 max 16384 2024-04-23T10:38:16.556 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:38:16.556 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:38:16.556 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:38:16.556 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:38:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.556+0000 7f66fab0f640 1 -- 172.21.15.97:0/2829113574 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f66e40a3e40 msgr2=0x7f66e40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.556+0000 7f66fab0f640 1 --2- 172.21.15.97:0/2829113574 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f66e40a3e40 0x7f66e40a6300 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f66ec005cc0 tx=0x7f66ec00aa00 comp rx=0 tx=0).stop 2024-04-23T10:38:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.556+0000 7f66fab0f640 1 -- 172.21.15.97:0/2829113574 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66f41021c0 msgr2=0x7f66f4198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.556+0000 7f66fab0f640 1 --2- 172.21.15.97:0/2829113574 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66f41021c0 0x7f66f4198730 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f66e8005940 tx=0x7f66e80042f0 comp rx=0 tx=0).stop 2024-04-23T10:38:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.556+0000 7f66fab0f640 1 -- 172.21.15.97:0/2829113574 shutdown_connections 2024-04-23T10:38:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.556+0000 7f66fab0f640 1 --2- 172.21.15.97:0/2829113574 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f66e40a3e40 0x7f66e40a6300 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.556+0000 7f66fab0f640 1 --2- 172.21.15.97:0/2829113574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f66f419cfa0 0x7f66f419d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.556+0000 7f66fab0f640 1 --2- 172.21.15.97:0/2829113574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66f4107290 0x7f66f4198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.556+0000 7f66fab0f640 1 --2- 172.21.15.97:0/2829113574 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66f41021c0 0x7f66f4198730 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.556+0000 7f66fab0f640 1 -- 172.21.15.97:0/2829113574 >> 172.21.15.97:0/2829113574 conn(0x7f66f40fda20 msgr2=0x7f66f40fee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:38:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.556+0000 7f66fab0f640 1 -- 172.21.15.97:0/2829113574 shutdown_connections 2024-04-23T10:38:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:16.556+0000 7f66fab0f640 1 -- 172.21.15.97:0/2829113574 wait complete. 2024-04-23T10:38:17.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:38:17 smithi196 ceph-mon[32919]: pgmap v377: 97 pgs: 97 active+clean; 1.4 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-04-23T10:38:17.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:38:17 smithi097 ceph-mon[29197]: pgmap v377: 97 pgs: 97 active+clean; 1.4 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-04-23T10:38:17.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:38:17 smithi183 ceph-mon[32317]: pgmap v377: 97 pgs: 97 active+clean; 1.4 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-04-23T10:38:19.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:38:19 smithi196 ceph-mon[32919]: pgmap v378: 97 pgs: 97 active+clean; 1.4 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.5 KiB/s wr, 13 op/s 2024-04-23T10:38:19.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:38:19 smithi097 ceph-mon[29197]: pgmap v378: 97 pgs: 97 active+clean; 1.4 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.5 KiB/s wr, 13 op/s 2024-04-23T10:38:19.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:38:19 smithi183 ceph-mon[32317]: pgmap v378: 97 pgs: 97 active+clean; 1.4 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.5 KiB/s wr, 13 op/s 2024-04-23T10:38:21.452 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:38:21 smithi183 ceph-mon[32317]: pgmap v379: 97 pgs: 97 active+clean; 1.4 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.5 KiB/s wr, 13 op/s 2024-04-23T10:38:21.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:38:21 smithi196 ceph-mon[32919]: pgmap v379: 97 pgs: 97 active+clean; 1.4 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.5 KiB/s wr, 13 op/s 2024-04-23T10:38:21.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:38:21 smithi097 ceph-mon[29197]: pgmap v379: 97 pgs: 97 active+clean; 1.4 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.5 KiB/s wr, 13 op/s 2024-04-23T10:38:23.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:38:23 smithi196 ceph-mon[32919]: pgmap v380: 97 pgs: 97 active+clean; 1.4 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.3 KiB/s wr, 13 op/s 2024-04-23T10:38:23.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:38:23 smithi097 ceph-mon[29197]: pgmap v380: 97 pgs: 97 active+clean; 1.4 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.3 KiB/s wr, 13 op/s 2024-04-23T10:38:23.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:38:23 smithi183 ceph-mon[32317]: pgmap v380: 97 pgs: 97 active+clean; 1.4 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.3 KiB/s wr, 13 op/s 2024-04-23T10:38:24.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:38:24 smithi196 ceph-mon[32919]: pgmap v381: 97 pgs: 97 active+clean; 1.4 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.9 KiB/s wr, 15 op/s 2024-04-23T10:38:24.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:38:24 smithi097 ceph-mon[29197]: pgmap v381: 97 pgs: 97 active+clean; 1.4 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.9 KiB/s wr, 15 op/s 2024-04-23T10:38:24.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:38:24 smithi183 ceph-mon[32317]: pgmap v381: 97 pgs: 97 active+clean; 1.4 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.9 KiB/s wr, 15 op/s 2024-04-23T10:38:26.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:38:26 smithi183 ceph-mon[32317]: pgmap v382: 97 pgs: 97 active+clean; 1.4 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.4 KiB/s wr, 10 op/s 2024-04-23T10:38:26.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:38:26 smithi196 ceph-mon[32919]: pgmap v382: 97 pgs: 97 active+clean; 1.4 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.4 KiB/s wr, 10 op/s 2024-04-23T10:38:26.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:38:26 smithi097 ceph-mon[29197]: pgmap v382: 97 pgs: 97 active+clean; 1.4 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.4 KiB/s wr, 10 op/s 2024-04-23T10:38:26.894 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T10:38:27.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.005+0000 7f8280508640 1 -- 172.21.15.97:0/3918368584 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8278101ac0 msgr2=0x7f8278105ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:27.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.005+0000 7f8280508640 1 --2- 172.21.15.97:0/3918368584 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8278101ac0 0x7f8278105ab0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f8268009900 tx=0x7f826802a410 comp rx=0 tx=0).stop 2024-04-23T10:38:27.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.005+0000 7f8280508640 1 -- 172.21.15.97:0/3918368584 shutdown_connections 2024-04-23T10:38:27.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.005+0000 7f8280508640 1 --2- 172.21.15.97:0/3918368584 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8278106190 0x7f827810dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.005+0000 7f8280508640 1 --2- 172.21.15.97:0/3918368584 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8278101ac0 0x7f8278105ab0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.005+0000 7f8280508640 1 --2- 172.21.15.97:0/3918368584 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8278101110 0x7f82781014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.005+0000 7f8280508640 1 -- 172.21.15.97:0/3918368584 >> 172.21.15.97:0/3918368584 conn(0x7f8278077fb0 msgr2=0x7f82780783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:38:27.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.005+0000 7f8280508640 1 -- 172.21.15.97:0/3918368584 shutdown_connections 2024-04-23T10:38:27.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.005+0000 7f8280508640 1 -- 172.21.15.97:0/3918368584 wait complete. 2024-04-23T10:38:27.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.006+0000 7f8280508640 1 Processor -- start 2024-04-23T10:38:27.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.006+0000 7f8280508640 1 -- start start 2024-04-23T10:38:27.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.006+0000 7f8280508640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8278101110 0x7f8278198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:27.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.006+0000 7f8280508640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8278106190 0x7f8278198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:27.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.006+0000 7f8280508640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f827819cfa0 0x7f827819d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:27.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.006+0000 7f8280508640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8278112210 con 0x7f8278101110 2024-04-23T10:38:27.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.006+0000 7f8280508640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8278112090 con 0x7f8278106190 2024-04-23T10:38:27.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.006+0000 7f8280508640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8278112390 con 0x7f827819cfa0 2024-04-23T10:38:27.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.006+0000 7f827e2a4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8278101110 0x7f8278198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:27.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.006+0000 7f827e2a4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8278101110 0x7f8278198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52226/0 (socket says 172.21.15.97:52226) 2024-04-23T10:38:27.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.006+0000 7f827e2a4640 1 -- 172.21.15.97:0/172996079 learned_addr learned my addr 172.21.15.97:0/172996079 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:38:27.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.006+0000 7f827eaa5640 1 --2- 172.21.15.97:0/172996079 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f827819cfa0 0x7f827819d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:27.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.006+0000 7f827e2a4640 1 -- 172.21.15.97:0/172996079 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f827819cfa0 msgr2=0x7f827819d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:27.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.006+0000 7f827daa3640 1 --2- 172.21.15.97:0/172996079 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8278106190 0x7f8278198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:27.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.006+0000 7f827e2a4640 1 --2- 172.21.15.97:0/172996079 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f827819cfa0 0x7f827819d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.006+0000 7f827e2a4640 1 -- 172.21.15.97:0/172996079 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8278106190 msgr2=0x7f8278198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:27.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.006+0000 7f827e2a4640 1 --2- 172.21.15.97:0/172996079 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8278106190 0x7f8278198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.006+0000 7f827e2a4640 1 -- 172.21.15.97:0/172996079 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f827819da90 con 0x7f8278101110 2024-04-23T10:38:27.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.007+0000 7f827e2a4640 1 --2- 172.21.15.97:0/172996079 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8278101110 0x7f8278198730 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f827000c920 tx=0x7f827000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:38:27.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.007+0000 7f826f7fe640 1 -- 172.21.15.97:0/172996079 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8270007c70 con 0x7f8278101110 2024-04-23T10:38:27.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.007+0000 7f8280508640 1 -- 172.21.15.97:0/172996079 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f827819dd80 con 0x7f8278101110 2024-04-23T10:38:27.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.007+0000 7f826f7fe640 1 -- 172.21.15.97:0/172996079 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8270007e10 con 0x7f8278101110 2024-04-23T10:38:27.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.007+0000 7f826f7fe640 1 -- 172.21.15.97:0/172996079 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8270018820 con 0x7f8278101110 2024-04-23T10:38:27.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.007+0000 7f8280508640 1 -- 172.21.15.97:0/172996079 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82781a55c0 con 0x7f8278101110 2024-04-23T10:38:27.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.008+0000 7f8280508640 1 -- 172.21.15.97:0/172996079 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82780691c0 con 0x7f8278101110 2024-04-23T10:38:27.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.009+0000 7f826f7fe640 1 -- 172.21.15.97:0/172996079 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f827000f450 con 0x7f8278101110 2024-04-23T10:38:27.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.009+0000 7f826f7fe640 1 --2- 172.21.15.97:0/172996079 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f82640a3d20 0x7f82640a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:27.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.009+0000 7f827daa3640 1 --2- 172.21.15.97:0/172996079 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f82640a3d20 0x7f82640a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:27.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.010+0000 7f826f7fe640 1 -- 172.21.15.97:0/172996079 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8484+0+0 (secure 0 0 0) 0x7f82700d0ca0 con 0x7f8278101110 2024-04-23T10:38:27.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.010+0000 7f827daa3640 1 --2- 172.21.15.97:0/172996079 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f82640a3d20 0x7f82640a61e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8268006010 tx=0x7f8268000bc0 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:38:27.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.013+0000 7f826f7fe640 1 -- 172.21.15.97:0/172996079 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8270014030 con 0x7f8278101110 2024-04-23T10:38:27.228 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.229+0000 7f8280508640 1 -- 172.21.15.97:0/172996079 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8278109420 con 0x7f8278101110 2024-04-23T10:38:27.229 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.230+0000 7f826f7fe640 1 -- 172.21.15.97:0/172996079 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11931 (secure 0 0 0) 0x7f82700bbe20 con 0x7f8278101110 2024-04-23T10:38:27.229 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:38:27.229 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3946878762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3946878762},{"type":"v1","addr":"172.21.15.97:6835","nonce":3946878762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/209800861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":209800861},{"type":"v1","addr":"172.21.15.97:6837","nonce":209800861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T10:24:06.910134+0000","modified":"2024-04-23T10:26:09.820985+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6839/418013575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":418013575},{"type":"v1","addr":"172.21.15.97:6839","nonce":418013575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6841/747242680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":747242680},{"type":"v1","addr":"172.21.15.97:6841","nonce":747242680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6833/4292847121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4292847121},{"type":"v1","addr":"172.21.15.183:6833","nonce":4292847121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6833/3885816565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3885816565},{"type":"v1","addr":"172.21.15.196:6833","nonce":3885816565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1897093105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1897093105},{"type":"v1","addr":"172.21.15.183:6835","nonce":1897093105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1171477563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1171477563},{"type":"v1","addr":"172.21.15.196:6835","nonce":1171477563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1898341911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1898341911},{"type":"v1","addr":"172.21.15.183:6837","nonce":1898341911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.196:6837/3094946502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3094946502},{"type":"v1","addr":"172.21.15.196:6837","nonce":3094946502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.196:6839/83843556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":83843556},{"type":"v1","addr":"172.21.15.196:6839","nonce":83843556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6839/1763841130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1763841130},{"type":"v1","addr":"172.21.15.183:6839","nonce":1763841130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24487,24403,24398,24469,24488]},"id":1}]} 2024-04-23T10:38:27.229 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 26 2024-04-23T10:38:27.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.232+0000 7f8280508640 1 -- 172.21.15.97:0/172996079 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f82640a3d20 msgr2=0x7f82640a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:27.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.232+0000 7f8280508640 1 --2- 172.21.15.97:0/172996079 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f82640a3d20 0x7f82640a61e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8268006010 tx=0x7f8268000bc0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.232+0000 7f8280508640 1 -- 172.21.15.97:0/172996079 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8278101110 msgr2=0x7f8278198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:27.231 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.232+0000 7f8280508640 1 --2- 172.21.15.97:0/172996079 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8278101110 0x7f8278198730 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f827000c920 tx=0x7f827000cde0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.232+0000 7f8280508640 1 -- 172.21.15.97:0/172996079 shutdown_connections 2024-04-23T10:38:27.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.232+0000 7f8280508640 1 --2- 172.21.15.97:0/172996079 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f82640a3d20 0x7f82640a61e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.232+0000 7f8280508640 1 --2- 172.21.15.97:0/172996079 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f827819cfa0 0x7f827819d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.232+0000 7f8280508640 1 --2- 172.21.15.97:0/172996079 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8278106190 0x7f8278198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.232+0000 7f8280508640 1 --2- 172.21.15.97:0/172996079 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8278101110 0x7f8278198730 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.232+0000 7f8280508640 1 -- 172.21.15.97:0/172996079 >> 172.21.15.97:0/172996079 conn(0x7f8278077fb0 msgr2=0x7f82780fefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:38:27.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.232+0000 7f8280508640 1 -- 172.21.15.97:0/172996079 shutdown_connections 2024-04-23T10:38:27.232 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.232+0000 7f8280508640 1 -- 172.21.15.97:0/172996079 wait complete. 2024-04-23T10:38:27.242 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --set-id 4280baf --release --await' 2024-04-23T10:38:27.242 DEBUG:teuthology.orchestra.run.smithi097:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24488 quiesce db --set-id 4280baf --release --await 2024-04-23T10:38:27.351 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.349+0000 7feebf9bf640 1 -- 172.21.15.97:0/3573509529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feeb8102b40 msgr2=0x7feeb8106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:27.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.349+0000 7feebf9bf640 1 --2- 172.21.15.97:0/3573509529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feeb8102b40 0x7feeb8106b80 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7feeb4009900 tx=0x7feeb402a410 comp rx=0 tx=0).stop 2024-04-23T10:38:27.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.350+0000 7feebf9bf640 1 -- 172.21.15.97:0/3573509529 shutdown_connections 2024-04-23T10:38:27.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.350+0000 7feebf9bf640 1 --2- 172.21.15.97:0/3573509529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feeb8107260 0x7feeb810ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.350+0000 7feebf9bf640 1 --2- 172.21.15.97:0/3573509529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feeb8102b40 0x7feeb8106b80 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.350+0000 7feebf9bf640 1 --2- 172.21.15.97:0/3573509529 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feeb8102190 0x7feeb8102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.350+0000 7feebf9bf640 1 -- 172.21.15.97:0/3573509529 >> 172.21.15.97:0/3573509529 conn(0x7feeb80fd990 msgr2=0x7feeb80ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:38:27.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.350+0000 7feebf9bf640 1 -- 172.21.15.97:0/3573509529 shutdown_connections 2024-04-23T10:38:27.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.350+0000 7feebf9bf640 1 -- 172.21.15.97:0/3573509529 wait complete. 2024-04-23T10:38:27.352 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.350+0000 7feebf9bf640 1 Processor -- start 2024-04-23T10:38:27.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.350+0000 7feebf9bf640 1 -- start start 2024-04-23T10:38:27.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.350+0000 7feebf9bf640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feeb8102190 0x7feeb819e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:27.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.350+0000 7feebf9bf640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feeb8107260 0x7feeb819e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:27.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.350+0000 7feebf9bf640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feeb8198630 0x7feeb8198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:27.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.350+0000 7feebf9bf640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7feeb81121a0 con 0x7feeb8198630 2024-04-23T10:38:27.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.350+0000 7feebf9bf640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7feeb8112020 con 0x7feeb8107260 2024-04-23T10:38:27.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.350+0000 7feebf9bf640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7feeb8112320 con 0x7feeb8102190 2024-04-23T10:38:27.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.351+0000 7feebdf5c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feeb8198630 0x7feeb8198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:27.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.351+0000 7feebdf5c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feeb8198630 0x7feeb8198ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52242/0 (socket says 172.21.15.97:52242) 2024-04-23T10:38:27.353 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.351+0000 7feebdf5c640 1 -- 172.21.15.97:0/2218544527 learned_addr learned my addr 172.21.15.97:0/2218544527 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:38:27.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.351+0000 7feebcf5a640 1 --2- 172.21.15.97:0/2218544527 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feeb8107260 0x7feeb819e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:27.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.351+0000 7feebd75b640 1 --2- 172.21.15.97:0/2218544527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feeb8102190 0x7feeb819e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:27.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.351+0000 7feebdf5c640 1 -- 172.21.15.97:0/2218544527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feeb8102190 msgr2=0x7feeb819e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:27.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.351+0000 7feebdf5c640 1 --2- 172.21.15.97:0/2218544527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feeb8102190 0x7feeb819e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.351+0000 7feebdf5c640 1 -- 172.21.15.97:0/2218544527 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feeb8107260 msgr2=0x7feeb819e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:27.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.351+0000 7feebdf5c640 1 --2- 172.21.15.97:0/2218544527 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feeb8107260 0x7feeb819e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.351+0000 7feebdf5c640 1 -- 172.21.15.97:0/2218544527 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feeb8199150 con 0x7feeb8198630 2024-04-23T10:38:27.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.351+0000 7feebdf5c640 1 --2- 172.21.15.97:0/2218544527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feeb8198630 0x7feeb8198ae0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7feeb000eaa0 tx=0x7feeb000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:38:27.354 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.351+0000 7feeae7fc640 1 -- 172.21.15.97:0/2218544527 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb000cdb0 con 0x7feeb8198630 2024-04-23T10:38:27.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.351+0000 7feeae7fc640 1 -- 172.21.15.97:0/2218544527 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feeb0005020 con 0x7feeb8198630 2024-04-23T10:38:27.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.351+0000 7feeae7fc640 1 -- 172.21.15.97:0/2218544527 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb00116d0 con 0x7feeb8198630 2024-04-23T10:38:27.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.351+0000 7feebf9bf640 1 -- 172.21.15.97:0/2218544527 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feeb8199440 con 0x7feeb8198630 2024-04-23T10:38:27.355 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.351+0000 7feebf9bf640 1 -- 172.21.15.97:0/2218544527 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feeb81a5640 con 0x7feeb8198630 2024-04-23T10:38:27.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.353+0000 7feeae7fc640 1 -- 172.21.15.97:0/2218544527 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7feeb00051d0 con 0x7feeb8198630 2024-04-23T10:38:27.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.354+0000 7feec27b2740 1 Processor -- start 2024-04-23T10:38:27.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.354+0000 7feec27b2740 1 -- start start 2024-04-23T10:38:27.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.354+0000 7feec27b2740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55692f41a490 0x55692f41a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:27.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.354+0000 7feec27b2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55692f41adb0 0x55692f424f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:27.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.354+0000 7feec27b2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55692f425480 0x55692f427870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:27.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.354+0000 7feec27b2740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55692f37f6d0 con 0x55692f41a490 2024-04-23T10:38:27.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.354+0000 7feec27b2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55692f31c1a0 con 0x55692f425480 2024-04-23T10:38:27.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.354+0000 7feec27b2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55692f37b690 con 0x55692f41adb0 2024-04-23T10:38:27.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.354+0000 7feeae7fc640 1 --2- 172.21.15.97:0/2218544527 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7feea40a3d20 0x7feea40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:27.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.354+0000 7feebcf5a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55692f41a490 0x55692f41a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:27.364 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.354+0000 7feebcf5a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55692f41a490 0x55692f41a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52256/0 (socket says 172.21.15.97:52256) 2024-04-23T10:38:27.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.354+0000 7feebcf5a640 1 -- 172.21.15.97:0/842019789 learned_addr learned my addr 172.21.15.97:0/842019789 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:38:27.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.354+0000 7feebdf5c640 1 --2- 172.21.15.97:0/2218544527 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7feea40a3d20 0x7feea40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:27.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.354+0000 7feebd75b640 1 --2- 172.21.15.97:0/842019789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55692f41adb0 0x55692f424f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:27.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.354+0000 7feeae7fc640 1 -- 172.21.15.97:0/2218544527 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8484+0+0 (secure 0 0 0) 0x7feeb00d0920 con 0x7feeb8198630 2024-04-23T10:38:27.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.354+0000 7feebcf5a640 1 --2- 172.21.15.97:0/842019789 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55692f425480 0x55692f427870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:27.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.354+0000 7feebcf5a640 1 -- 172.21.15.97:0/842019789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55692f41adb0 msgr2=0x55692f424f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:38:27.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.354+0000 7feebcf5a640 1 --2- 172.21.15.97:0/842019789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55692f41adb0 0x55692f424f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.354+0000 7feebcf5a640 1 -- 172.21.15.97:0/842019789 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55692f425480 msgr2=0x55692f427870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:38:27.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.354+0000 7feebcf5a640 1 --2- 172.21.15.97:0/842019789 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55692f425480 0x55692f427870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.354+0000 7feebcf5a640 1 -- 172.21.15.97:0/842019789 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55692f3265b0 con 0x55692f41a490 2024-04-23T10:38:27.365 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.355+0000 7feebcf5a640 1 --2- 172.21.15.97:0/842019789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55692f41a490 0x55692f41a870 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7feeb40096c0 tx=0x7feeb40095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6beb33888d60067 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:38:27.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.355+0000 7feebdf5c640 1 --2- 172.21.15.97:0/2218544527 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7feea40a3d20 0x7feea40a61e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7feeb00a5020 tx=0x7feeb0087040 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:38:27.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.355+0000 7feea37fe640 1 -- 172.21.15.97:0/842019789 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb4044020 con 0x55692f41a490 2024-04-23T10:38:27.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.355+0000 7feea37fe640 1 -- 172.21.15.97:0/842019789 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feeb40317a0 con 0x55692f41a490 2024-04-23T10:38:27.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.355+0000 7feea37fe640 1 -- 172.21.15.97:0/842019789 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb4035040 con 0x55692f41a490 2024-04-23T10:38:27.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.355+0000 7feec27b2740 1 -- 172.21.15.97:0/842019789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55692f41a490 msgr2=0x55692f41a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:27.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.355+0000 7feec27b2740 1 --2- 172.21.15.97:0/842019789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55692f41a490 0x55692f41a870 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7feeb40096c0 tx=0x7feeb40095a0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.355+0000 7feec27b2740 1 -- 172.21.15.97:0/842019789 shutdown_connections 2024-04-23T10:38:27.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.355+0000 7feec27b2740 1 --2- 172.21.15.97:0/842019789 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55692f425480 0x55692f427870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.366 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.355+0000 7feec27b2740 1 --2- 172.21.15.97:0/842019789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55692f41adb0 0x55692f424f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.355+0000 7feec27b2740 1 --2- 172.21.15.97:0/842019789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55692f41a490 0x55692f41a870 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.355+0000 7feec27b2740 1 -- 172.21.15.97:0/842019789 >> 172.21.15.97:0/842019789 conn(0x55692f4167b0 msgr2=0x55692f416b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:38:27.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.355+0000 7feec27b2740 1 -- 172.21.15.97:0/842019789 shutdown_connections 2024-04-23T10:38:27.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.355+0000 7feec27b2740 1 -- 172.21.15.97:0/842019789 wait complete. 2024-04-23T10:38:27.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.356+0000 7feec27b2740 1 -- start start 2024-04-23T10:38:27.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.356+0000 7feec27b2740 1 Processor -- start 2024-04-23T10:38:27.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.356+0000 7feec27b2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55692f41a490 0x55692f424eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:27.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.356+0000 7feec27b2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55692f41adb0 0x55692f4221a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:27.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.356+0000 7feec27b2740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55692f425480 0x55692f4228e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:27.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.356+0000 7feec27b2740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55692f4282b0 con 0x55692f425480 2024-04-23T10:38:27.367 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.356+0000 7feec27b2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55692f319b20 con 0x55692f41a490 2024-04-23T10:38:27.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.356+0000 7feec27b2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55692f428430 con 0x55692f41adb0 2024-04-23T10:38:27.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.356+0000 7feec27b2740 10 client.0 _ll_register_callbacks cb 0x55692f319530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:38:27.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.356+0000 7feebcf5a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55692f425480 0x55692f4228e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:27.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.356+0000 7feebcf5a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55692f425480 0x55692f4228e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52268/0 (socket says 172.21.15.97:52268) 2024-04-23T10:38:27.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.356+0000 7feebcf5a640 1 -- 172.21.15.97:0/1444615783 learned_addr learned my addr 172.21.15.97:0/1444615783 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:38:27.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.356+0000 7feebd75b640 1 --2- 172.21.15.97:0/1444615783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55692f41adb0 0x55692f4221a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:27.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.356+0000 7feebcf5a640 1 --2- 172.21.15.97:0/1444615783 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55692f41a490 0x55692f424eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:27.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.357+0000 7feebcf5a640 1 -- 172.21.15.97:0/1444615783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55692f41adb0 msgr2=0x55692f4221a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:38:27.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.357+0000 7feebcf5a640 1 --2- 172.21.15.97:0/1444615783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55692f41adb0 0x55692f4221a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.357+0000 7feebcf5a640 1 -- 172.21.15.97:0/1444615783 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55692f41a490 msgr2=0x55692f424eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:38:27.368 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.357+0000 7feebcf5a640 1 --2- 172.21.15.97:0/1444615783 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55692f41a490 0x55692f424eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.357+0000 7feebcf5a640 1 -- 172.21.15.97:0/1444615783 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55692f091920 con 0x55692f425480 2024-04-23T10:38:27.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.357+0000 7feebcf5a640 1 --2- 172.21.15.97:0/1444615783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55692f425480 0x55692f4228e0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7feeb4009a20 tx=0x7feeb4040040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=621405a898b0890e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:38:27.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.357+0000 7feea0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:38:27.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.357+0000 7feea0ff9640 1 -- 172.21.15.97:0/1444615783 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb4035070 con 0x55692f425480 2024-04-23T10:38:27.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.357+0000 7feea0ff9640 1 -- 172.21.15.97:0/1444615783 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8484+0+0 (secure 0 0 0) 0x7feeb4005250 con 0x55692f425480 2024-04-23T10:38:27.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.357+0000 7feec27b2740 1 -- 172.21.15.97:0/1444615783 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feeb40317a0 con 0x55692f425480 2024-04-23T10:38:27.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.357+0000 7feea0ff9640 1 -- 172.21.15.97:0/1444615783 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7feeb405a050 con 0x55692f425480 2024-04-23T10:38:27.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.357+0000 7feec27b2740 10 client.17295 fetch_fsmap learned FSMap version 26 2024-04-23T10:38:27.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.357+0000 7feec27b2740 1 -- 172.21.15.97:0/1444615783 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55692f15dd30 con 0x55692f425480 2024-04-23T10:38:27.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.357+0000 7feea0ff9640 1 -- 172.21.15.97:0/1444615783 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feeb40317a0 con 0x55692f425480 2024-04-23T10:38:27.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.357+0000 7feea0ff9640 1 -- 172.21.15.97:0/1444615783 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7feeb4004ca0 con 0x55692f425480 2024-04-23T10:38:27.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.357+0000 7feec27b2740 10 client.17295 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:38:27.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.357+0000 7feec27b2740 10 client.17295 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:38:27.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.357+0000 7feec27b2740 20 client.17295 populate_metadata read hostname 'smithi097' 2024-04-23T10:38:27.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.357+0000 7feec27b2740 1 --2- 172.21.15.97:0/1444615783 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55692f435480 0x55692f437870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:27.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.357+0000 7feec27b2740 4 client.17295 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:38:27.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.357+0000 7feec27b2740 1 -- 172.21.15.97:0/1444615783 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55692f33d500 con 0x55692f435480 2024-04-23T10:38:27.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.358+0000 7feebd75b640 1 --2- 172.21.15.97:0/1444615783 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55692f435480 0x55692f437870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:27.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.358+0000 7feebd75b640 1 --2- 172.21.15.97:0/1444615783 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55692f435480 0x55692f437870 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c1ff6bcbc80ca9b server_cookie=2f052a0f3c0488dd in_seq=0 out_seq=0 2024-04-23T10:38:27.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.358+0000 7feea0ff9640 10 client.17295 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:38:27.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.359+0000 7feea0ff9640 1 -- 172.21.15.97:0/1444615783 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55692f33d500 con 0x55692f435480 2024-04-23T10:38:27.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.359+0000 7feea0ff9640 10 client.17295 handle_command_reply: tid=0 2024-04-23T10:38:27.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.359+0000 7feec27b2740 1 client.17295 shutdown 2024-04-23T10:38:27.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.359+0000 7feec27b2740 1 -- 172.21.15.97:0/1444615783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55692f425480 msgr2=0x55692f4228e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:27.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.359+0000 7feec27b2740 1 --2- 172.21.15.97:0/1444615783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55692f425480 0x55692f4228e0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7feeb4009a20 tx=0x7feeb4040040 comp rx=0 tx=0).stop 2024-04-23T10:38:27.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.359+0000 7feec27b2740 1 -- 172.21.15.97:0/1444615783 shutdown_connections 2024-04-23T10:38:27.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.359+0000 7feec27b2740 1 --2- 172.21.15.97:0/1444615783 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55692f435480 0x55692f437870 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.359+0000 7feec27b2740 1 --2- 172.21.15.97:0/1444615783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55692f425480 0x55692f4228e0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.359+0000 7feec27b2740 1 --2- 172.21.15.97:0/1444615783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55692f41adb0 0x55692f4221a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.359+0000 7feec27b2740 1 --2- 172.21.15.97:0/1444615783 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55692f41a490 0x55692f424eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.359+0000 7feea0ff9640 0 client.17295 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:38:27.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.359+0000 7feec27b2740 1 -- 172.21.15.97:0/1444615783 >> 172.21.15.97:0/1444615783 conn(0x55692f4167b0 msgr2=0x55692f41c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:38:27.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.360+0000 7feec27b2740 1 -- 172.21.15.97:0/1444615783 shutdown_connections 2024-04-23T10:38:27.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.360+0000 7feec27b2740 1 -- 172.21.15.97:0/1444615783 wait complete. 2024-04-23T10:38:27.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.360+0000 7feec27b2740 20 client.17295 trim_cache size 0 max 16384 2024-04-23T10:38:27.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.376+0000 7feec27b2740 1 Processor -- start 2024-04-23T10:38:27.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.377+0000 7feec27b2740 1 -- start start 2024-04-23T10:38:27.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.377+0000 7feec27b2740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55692f41a490 0x55692f41cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:27.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.377+0000 7feec27b2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55692f41adb0 0x55692f4249e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:27.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.377+0000 7feec27b2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55692f425480 0x55692f424f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:27.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.377+0000 7feec27b2740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7feeb81adba0 con 0x55692f41a490 2024-04-23T10:38:27.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.377+0000 7feec27b2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7feeb81af6e0 con 0x55692f41adb0 2024-04-23T10:38:27.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.377+0000 7feec27b2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7feeb81ae270 con 0x55692f425480 2024-04-23T10:38:27.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.377+0000 7feebcf5a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55692f41a490 0x55692f41cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:27.381 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.377+0000 7feebcf5a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55692f41a490 0x55692f41cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52278/0 (socket says 172.21.15.97:52278) 2024-04-23T10:38:27.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.377+0000 7feebcf5a640 1 -- 172.21.15.97:0/727730716 learned_addr learned my addr 172.21.15.97:0/727730716 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:38:27.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.377+0000 7feebd75b640 1 --2- 172.21.15.97:0/727730716 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55692f41adb0 0x55692f4249e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:27.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.377+0000 7feebcf5a640 1 --2- 172.21.15.97:0/727730716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55692f425480 0x55692f424f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:27.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.377+0000 7feebcf5a640 1 -- 172.21.15.97:0/727730716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55692f425480 msgr2=0x55692f424f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:38:27.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.377+0000 7feebcf5a640 1 --2- 172.21.15.97:0/727730716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55692f425480 0x55692f424f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.377+0000 7feebcf5a640 1 -- 172.21.15.97:0/727730716 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55692f41adb0 msgr2=0x55692f4249e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:38:27.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.377+0000 7feebcf5a640 1 --2- 172.21.15.97:0/727730716 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55692f41adb0 0x55692f4249e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.377+0000 7feebcf5a640 1 -- 172.21.15.97:0/727730716 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55692f41abe0 con 0x55692f41a490 2024-04-23T10:38:27.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.377+0000 7feebcf5a640 1 --2- 172.21.15.97:0/727730716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55692f41a490 0x55692f41cbf0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7feeb4000c00 tx=0x7feeb40095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57f3ad669e2cf3f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:38:27.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.377+0000 7feea2ffd640 1 -- 172.21.15.97:0/727730716 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb4035050 con 0x55692f41a490 2024-04-23T10:38:27.382 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.377+0000 7feea2ffd640 1 -- 172.21.15.97:0/727730716 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feeb4005310 con 0x55692f41a490 2024-04-23T10:38:27.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.377+0000 7feea2ffd640 1 -- 172.21.15.97:0/727730716 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb4044030 con 0x55692f41a490 2024-04-23T10:38:27.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.378+0000 7feec27b2740 1 -- 172.21.15.97:0/727730716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55692f41a490 msgr2=0x55692f41cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:27.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.378+0000 7feec27b2740 1 --2- 172.21.15.97:0/727730716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55692f41a490 0x55692f41cbf0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7feeb4000c00 tx=0x7feeb40095a0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.378+0000 7feec27b2740 1 -- 172.21.15.97:0/727730716 shutdown_connections 2024-04-23T10:38:27.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.378+0000 7feec27b2740 1 --2- 172.21.15.97:0/727730716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55692f425480 0x55692f424f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.378+0000 7feec27b2740 1 --2- 172.21.15.97:0/727730716 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55692f41adb0 0x55692f4249e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.378+0000 7feec27b2740 1 --2- 172.21.15.97:0/727730716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55692f41a490 0x55692f41cbf0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.378+0000 7feec27b2740 1 -- 172.21.15.97:0/727730716 >> 172.21.15.97:0/727730716 conn(0x55692f4167b0 msgr2=0x55692f4159d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:38:27.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.378+0000 7feec27b2740 1 -- 172.21.15.97:0/727730716 shutdown_connections 2024-04-23T10:38:27.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.378+0000 7feec27b2740 1 -- 172.21.15.97:0/727730716 wait complete. 2024-04-23T10:38:27.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.378+0000 7feec27b2740 1 -- start start 2024-04-23T10:38:27.383 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.379+0000 7feec27b2740 1 Processor -- start 2024-04-23T10:38:27.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.379+0000 7feec27b2740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55692f41a490 0x55692f42d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:27.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.379+0000 7feec27b2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55692f41adb0 0x55692f428670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:27.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.379+0000 7feec27b2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55692f425480 0x55692f428bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:27.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.379+0000 7feec27b2740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7feeb81af010 con 0x55692f41a490 2024-04-23T10:38:27.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.379+0000 7feec27b2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7feeb81ae940 con 0x55692f425480 2024-04-23T10:38:27.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.379+0000 7feec27b2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7feeb81abba0 con 0x55692f41adb0 2024-04-23T10:38:27.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.379+0000 7feec27b2740 10 client.0 _ll_register_callbacks cb 0x7feeb4032cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:38:27.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.379+0000 7feebcf5a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55692f41a490 0x55692f42d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:27.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.379+0000 7feebcf5a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55692f41a490 0x55692f42d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52288/0 (socket says 172.21.15.97:52288) 2024-04-23T10:38:27.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.379+0000 7feebcf5a640 1 -- 172.21.15.97:0/1193845683 learned_addr learned my addr 172.21.15.97:0/1193845683 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:38:27.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.379+0000 7feebd75b640 1 --2- 172.21.15.97:0/1193845683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55692f41adb0 0x55692f428670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:27.384 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.379+0000 7feebcf5a640 1 --2- 172.21.15.97:0/1193845683 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55692f425480 0x55692f428bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:27.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.379+0000 7feebcf5a640 1 -- 172.21.15.97:0/1193845683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55692f41adb0 msgr2=0x55692f428670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:38:27.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.379+0000 7feebcf5a640 1 --2- 172.21.15.97:0/1193845683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55692f41adb0 0x55692f428670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.379+0000 7feebcf5a640 1 -- 172.21.15.97:0/1193845683 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55692f425480 msgr2=0x55692f428bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:38:27.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.379+0000 7feebcf5a640 1 --2- 172.21.15.97:0/1193845683 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55692f425480 0x55692f428bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.379+0000 7feebcf5a640 1 -- 172.21.15.97:0/1193845683 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55692f429220 con 0x55692f41a490 2024-04-23T10:38:27.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.380+0000 7feebcf5a640 1 --2- 172.21.15.97:0/1193845683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55692f41a490 0x55692f42d920 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7feeb40059d0 tx=0x7feeb40048b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8806d852d2e1599 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:38:27.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.380+0000 7feea0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:38:27.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.380+0000 7feea0ff9640 1 -- 172.21.15.97:0/1193845683 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb4067070 con 0x55692f41a490 2024-04-23T10:38:27.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.380+0000 7feea0ff9640 1 -- 172.21.15.97:0/1193845683 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8484+0+0 (secure 0 0 0) 0x7feeb4004950 con 0x55692f41a490 2024-04-23T10:38:27.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.380+0000 7feec27b2740 1 -- 172.21.15.97:0/1193845683 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feeb4005310 con 0x55692f41a490 2024-04-23T10:38:27.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.380+0000 7feea0ff9640 1 -- 172.21.15.97:0/1193845683 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7feeb4035040 con 0x55692f41a490 2024-04-23T10:38:27.385 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.381+0000 7feec27b2740 10 client.17301 fetch_fsmap learned FSMap version 26 2024-04-23T10:38:27.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.381+0000 7feec27b2740 1 -- 172.21.15.97:0/1193845683 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55692f49c550 con 0x55692f41a490 2024-04-23T10:38:27.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.381+0000 7feea0ff9640 1 -- 172.21.15.97:0/1193845683 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feeb4005310 con 0x55692f41a490 2024-04-23T10:38:27.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.381+0000 7feea0ff9640 1 -- 172.21.15.97:0/1193845683 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7feeb406a020 con 0x55692f41a490 2024-04-23T10:38:27.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.381+0000 7feec27b2740 10 client.17301 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:38:27.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.381+0000 7feec27b2740 10 client.17301 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:38:27.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.381+0000 7feec27b2740 20 client.17301 populate_metadata read hostname 'smithi097' 2024-04-23T10:38:27.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.381+0000 7feec27b2740 1 --2- 172.21.15.97:0/1193845683 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55692f49c8f0 0x55692f49ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:27.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.381+0000 7feec27b2740 4 client.17301 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "4280baf", "release": true, "await": true}] 2024-04-23T10:38:27.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.381+0000 7feec27b2740 1 -- 172.21.15.97:0/1193845683 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "quiesce db", "set_id": "4280baf", "release": true, "await": true}) -- 0x55692f38d0f0 con 0x55692f49c8f0 2024-04-23T10:38:27.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.381+0000 7feebd75b640 1 --2- 172.21.15.97:0/1193845683 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55692f49c8f0 0x55692f49ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:27.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.382+0000 7feea0ff9640 10 client.17301 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:38:27.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.382+0000 7feebd75b640 1 --2- 172.21.15.97:0/1193845683 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55692f49c8f0 0x55692f49ed10 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61d07c2a75cf0eb7 server_cookie=889abff8acf9feff in_seq=0 out_seq=0 2024-04-23T10:38:27.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.385+0000 7feea0ff9640 1 -- 172.21.15.97:0/1193845683 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55692f38d0f0 con 0x55692f49c8f0 2024-04-23T10:38:27.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.385+0000 7feea0ff9640 10 client.17301 handle_command_reply: tid=0 2024-04-23T10:38:27.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.385+0000 7feec27b2740 1 client.17301 shutdown 2024-04-23T10:38:27.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.385+0000 7feec27b2740 1 -- 172.21.15.97:0/1193845683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55692f41a490 msgr2=0x55692f42d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:27.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.385+0000 7feec27b2740 1 --2- 172.21.15.97:0/1193845683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55692f41a490 0x55692f42d920 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7feeb40059d0 tx=0x7feeb40048b0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.385+0000 7feec27b2740 1 -- 172.21.15.97:0/1193845683 shutdown_connections 2024-04-23T10:38:27.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.385+0000 7feec27b2740 1 --2- 172.21.15.97:0/1193845683 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55692f49c8f0 0x55692f49ed10 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.385+0000 7feec27b2740 1 --2- 172.21.15.97:0/1193845683 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55692f425480 0x55692f428bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.385+0000 7feea0ff9640 0 client.17301 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:38:27.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.385+0000 7feec27b2740 1 --2- 172.21.15.97:0/1193845683 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55692f41adb0 0x55692f428670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.385+0000 7feec27b2740 1 --2- 172.21.15.97:0/1193845683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55692f41a490 0x55692f42d920 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.385+0000 7feec27b2740 1 -- 172.21.15.97:0/1193845683 >> 172.21.15.97:0/1193845683 conn(0x55692f4167b0 msgr2=0x55692f430190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:38:27.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.385+0000 7feec27b2740 1 -- 172.21.15.97:0/1193845683 shutdown_connections 2024-04-23T10:38:27.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.385+0000 7feec27b2740 1 -- 172.21.15.97:0/1193845683 wait complete. 2024-04-23T10:38:27.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.385+0000 7feec27b2740 20 client.17301 trim_cache size 0 max 16384 2024-04-23T10:38:27.390 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-23T10:38:27.390 INFO:teuthology.orchestra.run.smithi097.stdout: "epoch": 26, 2024-04-23T10:38:27.390 INFO:teuthology.orchestra.run.smithi097.stdout: "leader": 24488, 2024-04-23T10:38:27.390 INFO:teuthology.orchestra.run.smithi097.stdout: "set_version": 52, 2024-04-23T10:38:27.390 INFO:teuthology.orchestra.run.smithi097.stdout: "sets": { 2024-04-23T10:38:27.390 INFO:teuthology.orchestra.run.smithi097.stdout: "4280baf": { 2024-04-23T10:38:27.390 INFO:teuthology.orchestra.run.smithi097.stdout: "version": 52, 2024-04-23T10:38:27.390 INFO:teuthology.orchestra.run.smithi097.stdout: "age_ref": 0.0, 2024-04-23T10:38:27.390 INFO:teuthology.orchestra.run.smithi097.stdout: "state": { 2024-04-23T10:38:27.390 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RELEASED", 2024-04-23T10:38:27.390 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0 2024-04-23T10:38:27.390 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:38:27.391 INFO:teuthology.orchestra.run.smithi097.stdout: "timeout": 60.0, 2024-04-23T10:38:27.391 INFO:teuthology.orchestra.run.smithi097.stdout: "expiration": 90.0, 2024-04-23T10:38:27.391 INFO:teuthology.orchestra.run.smithi097.stdout: "members": { 2024-04-23T10:38:27.391 INFO:teuthology.orchestra.run.smithi097.stdout: "file:/": { 2024-04-23T10:38:27.391 INFO:teuthology.orchestra.run.smithi097.stdout: "excluded": false, 2024-04-23T10:38:27.391 INFO:teuthology.orchestra.run.smithi097.stdout: "state": { 2024-04-23T10:38:27.391 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RELEASED", 2024-04-23T10:38:27.391 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0 2024-04-23T10:38:27.391 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:38:27.391 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:38:27.391 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:38:27.391 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:38:27.392 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:38:27.392 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-23T10:38:27.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.386+0000 7feebf9bf640 1 -- 172.21.15.97:0/2218544527 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7feea40a3d20 msgr2=0x7feea40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:27.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.386+0000 7feebf9bf640 1 --2- 172.21.15.97:0/2218544527 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7feea40a3d20 0x7feea40a61e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7feeb00a5020 tx=0x7feeb0087040 comp rx=0 tx=0).stop 2024-04-23T10:38:27.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.386+0000 7feebf9bf640 1 -- 172.21.15.97:0/2218544527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feeb8198630 msgr2=0x7feeb8198ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:27.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.386+0000 7feebf9bf640 1 --2- 172.21.15.97:0/2218544527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feeb8198630 0x7feeb8198ae0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7feeb000eaa0 tx=0x7feeb000ef60 comp rx=0 tx=0).stop 2024-04-23T10:38:27.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.386+0000 7feebf9bf640 1 -- 172.21.15.97:0/2218544527 shutdown_connections 2024-04-23T10:38:27.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.386+0000 7feebf9bf640 1 --2- 172.21.15.97:0/2218544527 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7feea40a3d20 0x7feea40a61e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.386+0000 7feebf9bf640 1 --2- 172.21.15.97:0/2218544527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feeb8198630 0x7feeb8198ae0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.386+0000 7feebf9bf640 1 --2- 172.21.15.97:0/2218544527 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feeb8107260 0x7feeb819e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.386+0000 7feebf9bf640 1 --2- 172.21.15.97:0/2218544527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feeb8102190 0x7feeb819e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.386+0000 7feebf9bf640 1 -- 172.21.15.97:0/2218544527 >> 172.21.15.97:0/2218544527 conn(0x7feeb80fd990 msgr2=0x7feeb80fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:38:27.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.386+0000 7feebf9bf640 1 -- 172.21.15.97:0/2218544527 shutdown_connections 2024-04-23T10:38:27.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.386+0000 7feebf9bf640 1 -- 172.21.15.97:0/2218544527 wait complete. 2024-04-23T10:38:27.395 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '4280baf', seconds elapsed: 31.0 2024-04-23T10:38:27.395 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-04-23T10:38:27.396 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T10:38:27.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.495+0000 7fea4d285640 1 -- 172.21.15.97:0/4198478827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea481010a0 msgr2=0x7fea48101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:27.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.495+0000 7fea4d285640 1 --2- 172.21.15.97:0/4198478827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea481010a0 0x7fea48101480 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fea40009900 tx=0x7fea4002a410 comp rx=0 tx=0).stop 2024-04-23T10:38:27.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.495+0000 7fea4d285640 1 -- 172.21.15.97:0/4198478827 shutdown_connections 2024-04-23T10:38:27.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.495+0000 7fea4d285640 1 --2- 172.21.15.97:0/4198478827 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea48106170 0x7fea4810dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.495+0000 7fea4d285640 1 --2- 172.21.15.97:0/4198478827 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea48101a50 0x7fea48105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.495+0000 7fea4d285640 1 --2- 172.21.15.97:0/4198478827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea481010a0 0x7fea48101480 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.495+0000 7fea4d285640 1 -- 172.21.15.97:0/4198478827 >> 172.21.15.97:0/4198478827 conn(0x7fea48077f50 msgr2=0x7fea48078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:38:27.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.495+0000 7fea4d285640 1 -- 172.21.15.97:0/4198478827 shutdown_connections 2024-04-23T10:38:27.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.495+0000 7fea4d285640 1 -- 172.21.15.97:0/4198478827 wait complete. 2024-04-23T10:38:27.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.496+0000 7fea4d285640 1 Processor -- start 2024-04-23T10:38:27.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.496+0000 7fea4d285640 1 -- start start 2024-04-23T10:38:27.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.496+0000 7fea4d285640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea481010a0 0x7fea481994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:27.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.496+0000 7fea4d285640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea48106170 0x7fea48199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:27.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.496+0000 7fea4d285640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea48199fc0 0x7fea481a4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:27.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.496+0000 7fea4d285640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fea481110b0 con 0x7fea481010a0 2024-04-23T10:38:27.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.496+0000 7fea4d285640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fea48110f30 con 0x7fea48106170 2024-04-23T10:38:27.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.496+0000 7fea4d285640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fea48111230 con 0x7fea48199fc0 2024-04-23T10:38:27.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.496+0000 7fea46ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea481010a0 0x7fea481994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:27.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.496+0000 7fea46ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea481010a0 0x7fea481994f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52312/0 (socket says 172.21.15.97:52312) 2024-04-23T10:38:27.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.496+0000 7fea46ffd640 1 -- 172.21.15.97:0/3682236568 learned_addr learned my addr 172.21.15.97:0/3682236568 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:38:27.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.496+0000 7fea467fc640 1 --2- 172.21.15.97:0/3682236568 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea48106170 0x7fea48199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:27.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.497+0000 7fea477fe640 1 --2- 172.21.15.97:0/3682236568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea48199fc0 0x7fea481a4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:27.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.497+0000 7fea46ffd640 1 -- 172.21.15.97:0/3682236568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea48199fc0 msgr2=0x7fea481a4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:27.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.497+0000 7fea46ffd640 1 --2- 172.21.15.97:0/3682236568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea48199fc0 0x7fea481a4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.497+0000 7fea46ffd640 1 -- 172.21.15.97:0/3682236568 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea48106170 msgr2=0x7fea48199a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:27.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.497+0000 7fea46ffd640 1 --2- 172.21.15.97:0/3682236568 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea48106170 0x7fea48199a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.497+0000 7fea46ffd640 1 -- 172.21.15.97:0/3682236568 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea481a4d40 con 0x7fea481010a0 2024-04-23T10:38:27.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.497+0000 7fea46ffd640 1 --2- 172.21.15.97:0/3682236568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea481010a0 0x7fea481994f0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fea40006010 tx=0x7fea400042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:38:27.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.497+0000 7fea2ffff640 1 -- 172.21.15.97:0/3682236568 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4003f030 con 0x7fea481010a0 2024-04-23T10:38:27.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.497+0000 7fea2ffff640 1 -- 172.21.15.97:0/3682236568 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea40003800 con 0x7fea481010a0 2024-04-23T10:38:27.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.497+0000 7fea4d285640 1 -- 172.21.15.97:0/3682236568 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea481a4fd0 con 0x7fea481010a0 2024-04-23T10:38:27.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.497+0000 7fea2ffff640 1 -- 172.21.15.97:0/3682236568 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea40035070 con 0x7fea481010a0 2024-04-23T10:38:27.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.497+0000 7fea4d285640 1 -- 172.21.15.97:0/3682236568 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea481a5430 con 0x7fea481010a0 2024-04-23T10:38:27.500 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.498+0000 7fea4d285640 1 -- 172.21.15.97:0/3682236568 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea48068d70 con 0x7fea481010a0 2024-04-23T10:38:27.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.499+0000 7fea2ffff640 1 -- 172.21.15.97:0/3682236568 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fea40003cb0 con 0x7fea481010a0 2024-04-23T10:38:27.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.500+0000 7fea2ffff640 1 --2- 172.21.15.97:0/3682236568 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fea300a3d20 0x7fea300a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:27.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.500+0000 7fea467fc640 1 --2- 172.21.15.97:0/3682236568 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fea300a3d20 0x7fea300a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:27.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.500+0000 7fea2ffff640 1 -- 172.21.15.97:0/3682236568 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8484+0+0 (secure 0 0 0) 0x7fea400f10d0 con 0x7fea481010a0 2024-04-23T10:38:27.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.500+0000 7fea467fc640 1 --2- 172.21.15.97:0/3682236568 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fea300a3d20 0x7fea300a61e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fea48069c40 tx=0x7fea3c00a590 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:38:27.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.504+0000 7fea2ffff640 1 -- 172.21.15.97:0/3682236568 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fea400bd830 con 0x7fea481010a0 2024-04-23T10:38:27.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:38:27 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/172996079' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:38:27.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:38:27 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/172996079' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:38:27.719 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.721+0000 7fea4d285640 1 -- 172.21.15.97:0/3682236568 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fea48101a50 con 0x7fea481010a0 2024-04-23T10:38:27.720 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.721+0000 7fea2ffff640 1 -- 172.21.15.97:0/3682236568 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11931 (secure 0 0 0) 0x7fea400bd130 con 0x7fea481010a0 2024-04-23T10:38:27.720 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:38:27.721 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3946878762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3946878762},{"type":"v1","addr":"172.21.15.97:6835","nonce":3946878762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/209800861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":209800861},{"type":"v1","addr":"172.21.15.97:6837","nonce":209800861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T10:24:06.910134+0000","modified":"2024-04-23T10:26:09.820985+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6839/418013575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":418013575},{"type":"v1","addr":"172.21.15.97:6839","nonce":418013575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6841/747242680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":747242680},{"type":"v1","addr":"172.21.15.97:6841","nonce":747242680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6833/4292847121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4292847121},{"type":"v1","addr":"172.21.15.183:6833","nonce":4292847121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6833/3885816565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3885816565},{"type":"v1","addr":"172.21.15.196:6833","nonce":3885816565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1897093105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1897093105},{"type":"v1","addr":"172.21.15.183:6835","nonce":1897093105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1171477563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1171477563},{"type":"v1","addr":"172.21.15.196:6835","nonce":1171477563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1898341911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1898341911},{"type":"v1","addr":"172.21.15.183:6837","nonce":1898341911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.196:6837/3094946502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3094946502},{"type":"v1","addr":"172.21.15.196:6837","nonce":3094946502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.196:6839/83843556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":83843556},{"type":"v1","addr":"172.21.15.196:6839","nonce":83843556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6839/1763841130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1763841130},{"type":"v1","addr":"172.21.15.183:6839","nonce":1763841130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24487,24403,24398,24469,24488]},"id":1}]} 2024-04-23T10:38:27.721 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 26 2024-04-23T10:38:27.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.723+0000 7fea4d285640 1 -- 172.21.15.97:0/3682236568 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fea300a3d20 msgr2=0x7fea300a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:27.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.723+0000 7fea4d285640 1 --2- 172.21.15.97:0/3682236568 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fea300a3d20 0x7fea300a61e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fea48069c40 tx=0x7fea3c00a590 comp rx=0 tx=0).stop 2024-04-23T10:38:27.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.723+0000 7fea4d285640 1 -- 172.21.15.97:0/3682236568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea481010a0 msgr2=0x7fea481994f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:27.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.723+0000 7fea4d285640 1 --2- 172.21.15.97:0/3682236568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea481010a0 0x7fea481994f0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fea40006010 tx=0x7fea400042f0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.723+0000 7fea4d285640 1 -- 172.21.15.97:0/3682236568 shutdown_connections 2024-04-23T10:38:27.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.723+0000 7fea4d285640 1 --2- 172.21.15.97:0/3682236568 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fea300a3d20 0x7fea300a61e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.723+0000 7fea4d285640 1 --2- 172.21.15.97:0/3682236568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea48199fc0 0x7fea481a4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.723+0000 7fea4d285640 1 --2- 172.21.15.97:0/3682236568 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea48106170 0x7fea48199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.723 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.723+0000 7fea4d285640 1 --2- 172.21.15.97:0/3682236568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea481010a0 0x7fea481994f0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.723+0000 7fea4d285640 1 -- 172.21.15.97:0/3682236568 >> 172.21.15.97:0/3682236568 conn(0x7fea48077f50 msgr2=0x7fea480fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:38:27.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.723+0000 7fea4d285640 1 -- 172.21.15.97:0/3682236568 shutdown_connections 2024-04-23T10:38:27.724 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.723+0000 7fea4d285640 1 -- 172.21.15.97:0/3682236568 wait complete. 2024-04-23T10:38:27.733 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db / --timeout 60 --expiration 90.0 --await-for 42.0' 2024-04-23T10:38:27.733 DEBUG:teuthology.orchestra.run.smithi097:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24488 quiesce db / --timeout 60 --expiration 90.0 --await-for 42.0 2024-04-23T10:38:27.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.842+0000 7f21618ae640 1 -- 172.21.15.97:0/372693561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f215c103230 msgr2=0x7f215c103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:27.844 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.842+0000 7f21618ae640 1 --2- 172.21.15.97:0/372693561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f215c103230 0x7f215c103610 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f2154009900 tx=0x7f215402a480 comp rx=0 tx=0).stop 2024-04-23T10:38:27.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.842+0000 7f21618ae640 1 -- 172.21.15.97:0/372693561 shutdown_connections 2024-04-23T10:38:27.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.842+0000 7f21618ae640 1 --2- 172.21.15.97:0/372693561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f215c10c570 0x7f215c10e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.842+0000 7f21618ae640 1 --2- 172.21.15.97:0/372693561 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f215c103b50 0x7f215c10c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.842+0000 7f21618ae640 1 --2- 172.21.15.97:0/372693561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f215c103230 0x7f215c103610 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.842+0000 7f21618ae640 1 -- 172.21.15.97:0/372693561 >> 172.21.15.97:0/372693561 conn(0x7f215c077fb0 msgr2=0x7f215c0783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:38:27.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.843+0000 7f21618ae640 1 -- 172.21.15.97:0/372693561 shutdown_connections 2024-04-23T10:38:27.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.843+0000 7f21618ae640 1 -- 172.21.15.97:0/372693561 wait complete. 2024-04-23T10:38:27.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.843+0000 7f21618ae640 1 Processor -- start 2024-04-23T10:38:27.845 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.843+0000 7f21618ae640 1 -- start start 2024-04-23T10:38:27.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.843+0000 7f21618ae640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f215c103230 0x7f215c19e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:27.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.843+0000 7f21618ae640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f215c103b50 0x7f215c19e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:27.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.843+0000 7f21618ae640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f215c10c570 0x7f215c1985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:27.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.843+0000 7f21618ae640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f215c112250 con 0x7f215c103b50 2024-04-23T10:38:27.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.843+0000 7f21618ae640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f215c1120d0 con 0x7f215c10c570 2024-04-23T10:38:27.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.843+0000 7f21618ae640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f215c1123d0 con 0x7f215c103230 2024-04-23T10:38:27.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.844+0000 7f215a7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f215c103b50 0x7f215c19e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:27.846 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.844+0000 7f215a7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f215c103b50 0x7f215c19e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52328/0 (socket says 172.21.15.97:52328) 2024-04-23T10:38:27.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.844+0000 7f215a7fc640 1 -- 172.21.15.97:0/942683846 learned_addr learned my addr 172.21.15.97:0/942683846 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:38:27.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.844+0000 7f215affd640 1 --2- 172.21.15.97:0/942683846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f215c103230 0x7f215c19e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:27.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.844+0000 7f215b7fe640 1 --2- 172.21.15.97:0/942683846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f215c10c570 0x7f215c1985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:27.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.844+0000 7f215a7fc640 1 -- 172.21.15.97:0/942683846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f215c103230 msgr2=0x7f215c19e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:27.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.844+0000 7f215a7fc640 1 --2- 172.21.15.97:0/942683846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f215c103230 0x7f215c19e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.844+0000 7f215a7fc640 1 -- 172.21.15.97:0/942683846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f215c10c570 msgr2=0x7f215c1985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:27.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.844+0000 7f215a7fc640 1 --2- 172.21.15.97:0/942683846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f215c10c570 0x7f215c1985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.844+0000 7f215a7fc640 1 -- 172.21.15.97:0/942683846 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f215c198d70 con 0x7f215c103b50 2024-04-23T10:38:27.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.844+0000 7f215a7fc640 1 --2- 172.21.15.97:0/942683846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f215c103b50 0x7f215c19e980 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f215000b4a0 tx=0x7f215000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:38:27.847 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.844+0000 7f21608ac640 1 -- 172.21.15.97:0/942683846 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2150014070 con 0x7f215c103b50 2024-04-23T10:38:27.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.844+0000 7f21618ae640 1 -- 172.21.15.97:0/942683846 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f215c199060 con 0x7f215c103b50 2024-04-23T10:38:27.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.844+0000 7f21608ac640 1 -- 172.21.15.97:0/942683846 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2150010ac0 con 0x7f215c103b50 2024-04-23T10:38:27.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.844+0000 7f21608ac640 1 -- 172.21.15.97:0/942683846 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2150017c30 con 0x7f215c103b50 2024-04-23T10:38:27.848 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.844+0000 7f21618ae640 1 -- 172.21.15.97:0/942683846 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f215c1a5300 con 0x7f215c103b50 2024-04-23T10:38:27.849 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:38:27 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/172996079' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:38:27.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.846+0000 7f21608ac640 1 -- 172.21.15.97:0/942683846 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2150017070 con 0x7f215c103b50 2024-04-23T10:38:27.857 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.847+0000 7f21645c9740 1 Processor -- start 2024-04-23T10:38:27.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.847+0000 7f21645c9740 1 -- start start 2024-04-23T10:38:27.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.847+0000 7f21645c9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f5f005b0a0 0x55f5f005b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:27.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.847+0000 7f21645c9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5f005b9c0 0x55f5f0065b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:27.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.847+0000 7f21645c9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f5f0066090 0x55f5f0068480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:27.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.847+0000 7f21645c9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f5effc0370 con 0x55f5f0066090 2024-04-23T10:38:27.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.847+0000 7f21645c9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f5eff5d2e0 con 0x55f5f005b0a0 2024-04-23T10:38:27.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.847+0000 7f21645c9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f5effbc330 con 0x55f5f005b9c0 2024-04-23T10:38:27.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.847+0000 7f215a7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f5f0066090 0x55f5f0068480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:27.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.847+0000 7f215a7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f5f0066090 0x55f5f0068480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52330/0 (socket says 172.21.15.97:52330) 2024-04-23T10:38:27.858 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.847+0000 7f21608ac640 1 --2- 172.21.15.97:0/942683846 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f21440a3d20 0x7f21440a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:27.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.847+0000 7f215a7fc640 1 -- 172.21.15.97:0/4235016176 learned_addr learned my addr 172.21.15.97:0/4235016176 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:38:27.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.847+0000 7f215affd640 1 --2- 172.21.15.97:0/4235016176 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5f005b9c0 0x55f5f0065b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:27.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.847+0000 7f215b7fe640 1 --2- 172.21.15.97:0/942683846 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f21440a3d20 0x7f21440a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:27.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.847+0000 7f215b7fe640 1 --2- 172.21.15.97:0/4235016176 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f5f005b0a0 0x55f5f005b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:27.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.847+0000 7f21608ac640 1 -- 172.21.15.97:0/942683846 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8484+0+0 (secure 0 0 0) 0x7f21500cfe80 con 0x7f215c103b50 2024-04-23T10:38:27.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.847+0000 7f215a7fc640 1 -- 172.21.15.97:0/4235016176 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5f005b9c0 msgr2=0x55f5f0065b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:38:27.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.847+0000 7f215a7fc640 1 --2- 172.21.15.97:0/4235016176 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5f005b9c0 0x55f5f0065b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.847+0000 7f215a7fc640 1 -- 172.21.15.97:0/4235016176 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f5f005b0a0 msgr2=0x55f5f005b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:38:27.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.847+0000 7f215a7fc640 1 --2- 172.21.15.97:0/4235016176 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f5f005b0a0 0x55f5f005b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.859 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.847+0000 7f215a7fc640 1 -- 172.21.15.97:0/4235016176 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5efd9ed80 con 0x55f5f0066090 2024-04-23T10:38:27.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.847+0000 7f215b7fe640 1 --2- 172.21.15.97:0/4235016176 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f5f005b0a0 0x55f5f005b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:38:27.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.848+0000 7f215a7fc640 1 --2- 172.21.15.97:0/4235016176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f5f0066090 0x55f5f0068480 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f215009bc50 tx=0x7f2150000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=819f919e1fa6422d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:38:27.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.848+0000 7f215b7fe640 1 --2- 172.21.15.97:0/942683846 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f21440a3d20 0x7f21440a61e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f214c00cfd0 tx=0x7f214c00a250 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:38:27.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.848+0000 7f2141ffb640 1 -- 172.21.15.97:0/4235016176 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f215001e070 con 0x55f5f0066090 2024-04-23T10:38:27.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.848+0000 7f2141ffb640 1 -- 172.21.15.97:0/4235016176 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f215009c650 con 0x55f5f0066090 2024-04-23T10:38:27.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.848+0000 7f2141ffb640 1 -- 172.21.15.97:0/4235016176 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21500cd870 con 0x55f5f0066090 2024-04-23T10:38:27.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.848+0000 7f21645c9740 1 -- 172.21.15.97:0/4235016176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f5f0066090 msgr2=0x55f5f0068480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:27.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.848+0000 7f21645c9740 1 --2- 172.21.15.97:0/4235016176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f5f0066090 0x55f5f0068480 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f215009bc50 tx=0x7f2150000bc0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.848+0000 7f21645c9740 1 -- 172.21.15.97:0/4235016176 shutdown_connections 2024-04-23T10:38:27.860 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.848+0000 7f21645c9740 1 --2- 172.21.15.97:0/4235016176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f5f0066090 0x55f5f0068480 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.849+0000 7f21645c9740 1 --2- 172.21.15.97:0/4235016176 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5f005b9c0 0x55f5f0065b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.849+0000 7f21645c9740 1 --2- 172.21.15.97:0/4235016176 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f5f005b0a0 0x55f5f005b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.849+0000 7f21645c9740 1 -- 172.21.15.97:0/4235016176 >> 172.21.15.97:0/4235016176 conn(0x55f5f00573c0 msgr2=0x55f5f00577a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:38:27.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.849+0000 7f21645c9740 1 -- 172.21.15.97:0/4235016176 shutdown_connections 2024-04-23T10:38:27.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.849+0000 7f21645c9740 1 -- 172.21.15.97:0/4235016176 wait complete. 2024-04-23T10:38:27.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.849+0000 7f21645c9740 1 -- start start 2024-04-23T10:38:27.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.849+0000 7f21645c9740 1 Processor -- start 2024-04-23T10:38:27.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.849+0000 7f21645c9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f5f005b0a0 0x55f5f0062fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:27.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.849+0000 7f21645c9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f5f005b9c0 0x55f5f0063520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:27.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.849+0000 7f21645c9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5f0066090 0x55f5f0063a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:27.861 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.849+0000 7f21645c9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f5f0068e20 con 0x55f5f005b9c0 2024-04-23T10:38:27.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.849+0000 7f21645c9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f5eff5afb0 con 0x55f5f005b0a0 2024-04-23T10:38:27.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.849+0000 7f21645c9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55f5f0068fa0 con 0x55f5f0066090 2024-04-23T10:38:27.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.849+0000 7f21645c9740 10 client.0 _ll_register_callbacks cb 0x55f5eff5a9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:38:27.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.849+0000 7f215a7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f5f005b9c0 0x55f5f0063520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:27.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.849+0000 7f215a7fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f5f005b9c0 0x55f5f0063520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52336/0 (socket says 172.21.15.97:52336) 2024-04-23T10:38:27.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.849+0000 7f215a7fc640 1 -- 172.21.15.97:0/311892489 learned_addr learned my addr 172.21.15.97:0/311892489 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:38:27.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.850+0000 7f215b7fe640 1 --2- 172.21.15.97:0/311892489 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5f0066090 0x55f5f0063a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:27.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.850+0000 7f215a7fc640 1 -- 172.21.15.97:0/311892489 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5f0066090 msgr2=0x55f5f0063a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:38:27.862 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.850+0000 7f215a7fc640 1 --2- 172.21.15.97:0/311892489 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5f0066090 0x55f5f0063a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.850+0000 7f215a7fc640 1 -- 172.21.15.97:0/311892489 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f5f005b0a0 msgr2=0x55f5f0062fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:38:27.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.850+0000 7f215a7fc640 1 --2- 172.21.15.97:0/311892489 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f5f005b0a0 0x55f5f0062fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.850+0000 7f215a7fc640 1 -- 172.21.15.97:0/311892489 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5efdff510 con 0x55f5f005b9c0 2024-04-23T10:38:27.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.850+0000 7f215a7fc640 1 --2- 172.21.15.97:0/311892489 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f5f005b9c0 0x55f5f0063520 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f215009bee0 tx=0x7f21500047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6d82609b5f589c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:38:27.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.850+0000 7f213affd640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:38:27.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.850+0000 7f213affd640 1 -- 172.21.15.97:0/311892489 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f215001e050 con 0x55f5f005b9c0 2024-04-23T10:38:27.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.850+0000 7f213affd640 1 -- 172.21.15.97:0/311892489 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8484+0+0 (secure 0 0 0) 0x7f21500985c0 con 0x55f5f005b9c0 2024-04-23T10:38:27.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.850+0000 7f21645c9740 1 -- 172.21.15.97:0/311892489 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f215009c650 con 0x55f5f005b9c0 2024-04-23T10:38:27.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.850+0000 7f213affd640 1 -- 172.21.15.97:0/311892489 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f21500a0300 con 0x55f5f005b9c0 2024-04-23T10:38:27.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.850+0000 7f21645c9740 10 client.17319 fetch_fsmap learned FSMap version 26 2024-04-23T10:38:27.863 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.850+0000 7f21645c9740 1 -- 172.21.15.97:0/311892489 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f5efc92670 con 0x55f5f005b9c0 2024-04-23T10:38:27.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.850+0000 7f213affd640 1 -- 172.21.15.97:0/311892489 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f215009c650 con 0x55f5f005b9c0 2024-04-23T10:38:27.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.850+0000 7f213affd640 1 -- 172.21.15.97:0/311892489 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f21500b5240 con 0x55f5f005b9c0 2024-04-23T10:38:27.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.851+0000 7f21645c9740 10 client.17319 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:38:27.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.851+0000 7f21645c9740 10 client.17319 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:38:27.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.851+0000 7f21645c9740 20 client.17319 populate_metadata read hostname 'smithi097' 2024-04-23T10:38:27.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.851+0000 7f21645c9740 1 --2- 172.21.15.97:0/311892489 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55f5f00761f0 0x55f5f00785e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:27.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.851+0000 7f21645c9740 4 client.17319 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:38:27.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.851+0000 7f21645c9740 1 -- 172.21.15.97:0/311892489 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f5effcdd90 con 0x55f5f00761f0 2024-04-23T10:38:27.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.851+0000 7f215affd640 1 --2- 172.21.15.97:0/311892489 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55f5f00761f0 0x55f5f00785e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:27.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.851+0000 7f215affd640 1 --2- 172.21.15.97:0/311892489 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55f5f00761f0 0x55f5f00785e0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb896a9d874d8d1 server_cookie=c768f4175941b0ae in_seq=0 out_seq=0 2024-04-23T10:38:27.864 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.851+0000 7f213affd640 10 client.17319 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:38:27.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.852+0000 7f213affd640 1 -- 172.21.15.97:0/311892489 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f5effcdd90 con 0x55f5f00761f0 2024-04-23T10:38:27.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.852+0000 7f213affd640 10 client.17319 handle_command_reply: tid=0 2024-04-23T10:38:27.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.852+0000 7f21645c9740 1 client.17319 shutdown 2024-04-23T10:38:27.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.852+0000 7f21645c9740 1 -- 172.21.15.97:0/311892489 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f5f005b9c0 msgr2=0x55f5f0063520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:27.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.852+0000 7f21645c9740 1 --2- 172.21.15.97:0/311892489 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f5f005b9c0 0x55f5f0063520 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f215009bee0 tx=0x7f21500047d0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.852+0000 7f21645c9740 1 -- 172.21.15.97:0/311892489 shutdown_connections 2024-04-23T10:38:27.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.852+0000 7f21645c9740 1 --2- 172.21.15.97:0/311892489 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55f5f00761f0 0x55f5f00785e0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.852+0000 7f21645c9740 1 --2- 172.21.15.97:0/311892489 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5f0066090 0x55f5f0063a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.852+0000 7f21645c9740 1 --2- 172.21.15.97:0/311892489 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f5f005b9c0 0x55f5f0063520 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.852+0000 7f213affd640 0 client.17319 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:38:27.865 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.852+0000 7f21645c9740 1 --2- 172.21.15.97:0/311892489 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f5f005b0a0 0x55f5f0062fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.852+0000 7f21645c9740 1 -- 172.21.15.97:0/311892489 >> 172.21.15.97:0/311892489 conn(0x55f5f00573c0 msgr2=0x55f5f005d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:38:27.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.852+0000 7f21645c9740 1 -- 172.21.15.97:0/311892489 shutdown_connections 2024-04-23T10:38:27.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.852+0000 7f21645c9740 1 -- 172.21.15.97:0/311892489 wait complete. 2024-04-23T10:38:27.866 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.852+0000 7f21645c9740 20 client.17319 trim_cache size 0 max 16384 2024-04-23T10:38:27.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.869+0000 7f21645c9740 1 Processor -- start 2024-04-23T10:38:27.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.869+0000 7f21645c9740 1 -- start start 2024-04-23T10:38:27.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.869+0000 7f21645c9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f5f005b0a0 0x55f5f00657b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:27.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.869+0000 7f21645c9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5f005b9c0 0x55f5f0056d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:27.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.869+0000 7f21645c9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f5f0066090 0x55f5f005d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:27.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.870+0000 7f21645c9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f215c1adb30 con 0x55f5f005b0a0 2024-04-23T10:38:27.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.870+0000 7f21645c9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f215c1af670 con 0x55f5f0066090 2024-04-23T10:38:27.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.870+0000 7f21645c9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f215c1ae200 con 0x55f5f005b9c0 2024-04-23T10:38:27.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.870+0000 7f215affd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f5f005b0a0 0x55f5f00657b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:27.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.870+0000 7f215affd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f5f005b0a0 0x55f5f00657b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52346/0 (socket says 172.21.15.97:52346) 2024-04-23T10:38:27.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.870+0000 7f215affd640 1 -- 172.21.15.97:0/230691121 learned_addr learned my addr 172.21.15.97:0/230691121 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:38:27.875 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.870+0000 7f215a7fc640 1 --2- 172.21.15.97:0/230691121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5f005b9c0 0x55f5f0056d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:27.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.870+0000 7f215affd640 1 -- 172.21.15.97:0/230691121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5f005b9c0 msgr2=0x55f5f0056d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:38:27.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.870+0000 7f215affd640 1 --2- 172.21.15.97:0/230691121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5f005b9c0 0x55f5f0056d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.870+0000 7f215affd640 1 -- 172.21.15.97:0/230691121 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f5f0066090 msgr2=0x55f5f005d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:38:27.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.870+0000 7f215affd640 1 --2- 172.21.15.97:0/230691121 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f5f0066090 0x55f5f005d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.870+0000 7f215affd640 1 -- 172.21.15.97:0/230691121 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5f005c560 con 0x55f5f005b0a0 2024-04-23T10:38:27.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.870+0000 7f215a7fc640 1 --2- 172.21.15.97:0/230691121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5f005b9c0 0x55f5f0056d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:38:27.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.870+0000 7f215affd640 1 --2- 172.21.15.97:0/230691121 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f5f005b0a0 0x55f5f00657b0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f2154005ab0 tx=0x7f2154005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a8e9c3f672e6838 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:38:27.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.870+0000 7f21417fa640 1 -- 172.21.15.97:0/230691121 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2154047070 con 0x55f5f005b0a0 2024-04-23T10:38:27.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.870+0000 7f21417fa640 1 -- 172.21.15.97:0/230691121 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2154043e10 con 0x55f5f005b0a0 2024-04-23T10:38:27.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.870+0000 7f21417fa640 1 -- 172.21.15.97:0/230691121 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2154035040 con 0x55f5f005b0a0 2024-04-23T10:38:27.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.871+0000 7f21645c9740 1 -- 172.21.15.97:0/230691121 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f5f005b0a0 msgr2=0x55f5f00657b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:27.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.871+0000 7f21645c9740 1 --2- 172.21.15.97:0/230691121 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f5f005b0a0 0x55f5f00657b0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f2154005ab0 tx=0x7f2154005a20 comp rx=0 tx=0).stop 2024-04-23T10:38:27.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.871+0000 7f21645c9740 1 -- 172.21.15.97:0/230691121 shutdown_connections 2024-04-23T10:38:27.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.871+0000 7f21645c9740 1 --2- 172.21.15.97:0/230691121 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f5f0066090 0x55f5f005d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.871+0000 7f21645c9740 1 --2- 172.21.15.97:0/230691121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5f005b9c0 0x55f5f0056d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.871+0000 7f21645c9740 1 --2- 172.21.15.97:0/230691121 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f5f005b0a0 0x55f5f00657b0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.871+0000 7f21645c9740 1 -- 172.21.15.97:0/230691121 >> 172.21.15.97:0/230691121 conn(0x55f5f00573c0 msgr2=0x55f5f00650a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:38:27.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.871+0000 7f21645c9740 1 -- 172.21.15.97:0/230691121 shutdown_connections 2024-04-23T10:38:27.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.871+0000 7f21645c9740 1 -- 172.21.15.97:0/230691121 wait complete. 2024-04-23T10:38:27.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.871+0000 7f21645c9740 1 -- start start 2024-04-23T10:38:27.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.871+0000 7f21645c9740 1 Processor -- start 2024-04-23T10:38:27.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.872+0000 7f21645c9740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f5f005b0a0 0x55f5f006e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:27.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.872+0000 7f21645c9740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5f005b9c0 0x55f5f006aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:27.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.872+0000 7f21645c9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f5f0066090 0x55f5f006b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:27.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.872+0000 7f21645c9740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f215c1aefa0 con 0x55f5f005b0a0 2024-04-23T10:38:27.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.872+0000 7f21645c9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f215c1ae8d0 con 0x55f5f0066090 2024-04-23T10:38:27.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.872+0000 7f21645c9740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f215c1abb30 con 0x55f5f005b9c0 2024-04-23T10:38:27.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.872+0000 7f21645c9740 10 client.0 _ll_register_callbacks cb 0x7f21500a4c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:38:27.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.872+0000 7f215affd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f5f005b0a0 0x55f5f006e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:27.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.872+0000 7f215affd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f5f005b0a0 0x55f5f006e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52362/0 (socket says 172.21.15.97:52362) 2024-04-23T10:38:27.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.872+0000 7f215affd640 1 -- 172.21.15.97:0/643230950 learned_addr learned my addr 172.21.15.97:0/643230950 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:38:27.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.872+0000 7f215a7fc640 1 --2- 172.21.15.97:0/643230950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5f005b9c0 0x55f5f006aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:27.878 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.872+0000 7f215b7fe640 1 --2- 172.21.15.97:0/643230950 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f5f0066090 0x55f5f006b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:27.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.872+0000 7f215affd640 1 -- 172.21.15.97:0/643230950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5f005b9c0 msgr2=0x55f5f006aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:38:27.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.872+0000 7f215affd640 1 --2- 172.21.15.97:0/643230950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5f005b9c0 0x55f5f006aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.872+0000 7f215affd640 1 -- 172.21.15.97:0/643230950 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f5f0066090 msgr2=0x55f5f006b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:38:27.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.872+0000 7f215affd640 1 --2- 172.21.15.97:0/643230950 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f5f0066090 0x55f5f006b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.872+0000 7f215affd640 1 -- 172.21.15.97:0/643230950 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5f00dc400 con 0x55f5f005b0a0 2024-04-23T10:38:27.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.872+0000 7f215a7fc640 1 --2- 172.21.15.97:0/643230950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5f005b9c0 0x55f5f006aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:38:27.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.872+0000 7f215b7fe640 1 --2- 172.21.15.97:0/643230950 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f5f0066090 0x55f5f006b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:38:27.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.872+0000 7f215affd640 1 --2- 172.21.15.97:0/643230950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f5f005b0a0 0x55f5f006e740 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f2154031440 tx=0x7f2154031ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74ce545458cd5dfc server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:38:27.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.872+0000 7f213affd640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:38:27.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.872+0000 7f213affd640 1 -- 172.21.15.97:0/643230950 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2154047070 con 0x55f5f005b0a0 2024-04-23T10:38:27.879 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.872+0000 7f213affd640 1 -- 172.21.15.97:0/643230950 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8484+0+0 (secure 0 0 0) 0x7f21540040c0 con 0x55f5f005b0a0 2024-04-23T10:38:27.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.872+0000 7f21645c9740 1 -- 172.21.15.97:0/643230950 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2154043e10 con 0x55f5f005b0a0 2024-04-23T10:38:27.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.872+0000 7f213affd640 1 -- 172.21.15.97:0/643230950 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f215405c050 con 0x55f5f005b0a0 2024-04-23T10:38:27.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.872+0000 7f21645c9740 10 client.17325 fetch_fsmap learned FSMap version 26 2024-04-23T10:38:27.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.872+0000 7f21645c9740 1 -- 172.21.15.97:0/643230950 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f5f00dd580 con 0x55f5f005b0a0 2024-04-23T10:38:27.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.873+0000 7f213affd640 1 -- 172.21.15.97:0/643230950 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2154043e10 con 0x55f5f005b0a0 2024-04-23T10:38:27.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.873+0000 7f213affd640 1 -- 172.21.15.97:0/643230950 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f21540048a0 con 0x55f5f005b0a0 2024-04-23T10:38:27.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.873+0000 7f21645c9740 10 client.17325 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:38:27.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.873+0000 7f21645c9740 10 client.17325 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:38:27.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.873+0000 7f21645c9740 20 client.17325 populate_metadata read hostname 'smithi097' 2024-04-23T10:38:27.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.873+0000 7f21645c9740 1 --2- 172.21.15.97:0/643230950 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55f5f00ddce0 0x55f5f00e0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:27.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.873+0000 7f21645c9740 4 client.17325 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 90.0, "await_for": 42.0}] 2024-04-23T10:38:27.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.873+0000 7f21645c9740 1 -- 172.21.15.97:0/643230950 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 90.0, "await_for": 42.0}) -- 0x55f5effc7c00 con 0x55f5f00ddce0 2024-04-23T10:38:27.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.873+0000 7f215a7fc640 1 --2- 172.21.15.97:0/643230950 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55f5f00ddce0 0x55f5f00e0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:27.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.873+0000 7f213affd640 10 client.17325 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:38:27.881 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.873+0000 7f215a7fc640 1 --2- 172.21.15.97:0/643230950 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55f5f00ddce0 0x55f5f00e0100 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4dd9be65ed6091f0 server_cookie=e2aa981f512b9099 in_seq=0 out_seq=0 2024-04-23T10:38:27.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.887+0000 7f213affd640 1 -- 172.21.15.97:0/643230950 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55f5effc7c00 con 0x55f5f00ddce0 2024-04-23T10:38:27.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.887+0000 7f213affd640 10 client.17325 handle_command_reply: tid=0 2024-04-23T10:38:27.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.887+0000 7f21645c9740 1 client.17325 shutdown 2024-04-23T10:38:27.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.887+0000 7f21645c9740 1 -- 172.21.15.97:0/643230950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f5f005b0a0 msgr2=0x55f5f006e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:27.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.887+0000 7f21645c9740 1 --2- 172.21.15.97:0/643230950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f5f005b0a0 0x55f5f006e740 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f2154031440 tx=0x7f2154031ce0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.887+0000 7f21645c9740 1 -- 172.21.15.97:0/643230950 shutdown_connections 2024-04-23T10:38:27.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.887+0000 7f21645c9740 1 --2- 172.21.15.97:0/643230950 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55f5f00ddce0 0x55f5f00e0100 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.887+0000 7f21645c9740 1 --2- 172.21.15.97:0/643230950 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f5f0066090 0x55f5f006b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.887+0000 7f21645c9740 1 --2- 172.21.15.97:0/643230950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f5f005b9c0 0x55f5f006aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.887+0000 7f213affd640 0 client.17325 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:38:27.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.887+0000 7f21645c9740 1 --2- 172.21.15.97:0/643230950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f5f005b0a0 0x55f5f006e740 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.887+0000 7f21645c9740 1 -- 172.21.15.97:0/643230950 >> 172.21.15.97:0/643230950 conn(0x55f5f00573c0 msgr2=0x55f5f0055d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:38:27.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.887+0000 7f21645c9740 1 -- 172.21.15.97:0/643230950 shutdown_connections 2024-04-23T10:38:27.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.887+0000 7f21645c9740 1 -- 172.21.15.97:0/643230950 wait complete. 2024-04-23T10:38:27.888 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.887+0000 7f21645c9740 20 client.17325 trim_cache size 0 max 16384 2024-04-23T10:38:27.888 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-23T10:38:27.888 INFO:teuthology.orchestra.run.smithi097.stdout: "epoch": 26, 2024-04-23T10:38:27.888 INFO:teuthology.orchestra.run.smithi097.stdout: "leader": 24488, 2024-04-23T10:38:27.889 INFO:teuthology.orchestra.run.smithi097.stdout: "set_version": 54, 2024-04-23T10:38:27.889 INFO:teuthology.orchestra.run.smithi097.stdout: "sets": { 2024-04-23T10:38:27.889 INFO:teuthology.orchestra.run.smithi097.stdout: "845978d3": { 2024-04-23T10:38:27.889 INFO:teuthology.orchestra.run.smithi097.stdout: "version": 54, 2024-04-23T10:38:27.889 INFO:teuthology.orchestra.run.smithi097.stdout: "age_ref": 0.0, 2024-04-23T10:38:27.889 INFO:teuthology.orchestra.run.smithi097.stdout: "state": { 2024-04-23T10:38:27.889 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "QUIESCED", 2024-04-23T10:38:27.889 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0 2024-04-23T10:38:27.889 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:38:27.889 INFO:teuthology.orchestra.run.smithi097.stdout: "timeout": 60.0, 2024-04-23T10:38:27.889 INFO:teuthology.orchestra.run.smithi097.stdout: "expiration": 90.0, 2024-04-23T10:38:27.890 INFO:teuthology.orchestra.run.smithi097.stdout: "members": { 2024-04-23T10:38:27.890 INFO:teuthology.orchestra.run.smithi097.stdout: "file:/": { 2024-04-23T10:38:27.890 INFO:teuthology.orchestra.run.smithi097.stdout: "excluded": false, 2024-04-23T10:38:27.890 INFO:teuthology.orchestra.run.smithi097.stdout: "state": { 2024-04-23T10:38:27.890 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "QUIESCED", 2024-04-23T10:38:27.890 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0 2024-04-23T10:38:27.890 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:38:27.890 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:38:27.890 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:38:27.890 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:38:27.890 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:38:27.891 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-23T10:38:27.891 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.888+0000 7f21618ae640 1 -- 172.21.15.97:0/942683846 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f21440a3d20 msgr2=0x7f21440a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:27.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.888+0000 7f21618ae640 1 --2- 172.21.15.97:0/942683846 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f21440a3d20 0x7f21440a61e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f214c00cfd0 tx=0x7f214c00a250 comp rx=0 tx=0).stop 2024-04-23T10:38:27.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.888+0000 7f21618ae640 1 -- 172.21.15.97:0/942683846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f215c103b50 msgr2=0x7f215c19e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:27.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.888+0000 7f21618ae640 1 --2- 172.21.15.97:0/942683846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f215c103b50 0x7f215c19e980 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f215000b4a0 tx=0x7f215000b960 comp rx=0 tx=0).stop 2024-04-23T10:38:27.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.888+0000 7f21618ae640 1 -- 172.21.15.97:0/942683846 shutdown_connections 2024-04-23T10:38:27.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.888+0000 7f21618ae640 1 --2- 172.21.15.97:0/942683846 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f21440a3d20 0x7f21440a61e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.888+0000 7f21618ae640 1 --2- 172.21.15.97:0/942683846 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f215c10c570 0x7f215c1985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.888+0000 7f21618ae640 1 --2- 172.21.15.97:0/942683846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f215c103b50 0x7f215c19e980 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.888+0000 7f21618ae640 1 --2- 172.21.15.97:0/942683846 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f215c103230 0x7f215c19e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:27.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.888+0000 7f21618ae640 1 -- 172.21.15.97:0/942683846 >> 172.21.15.97:0/942683846 conn(0x7f215c077fb0 msgr2=0x7f215c0fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:38:27.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.888+0000 7f21618ae640 1 -- 172.21.15.97:0/942683846 shutdown_connections 2024-04-23T10:38:27.892 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:27.888+0000 7f21618ae640 1 -- 172.21.15.97:0/942683846 wait complete. 2024-04-23T10:38:27.897 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '845978d3', quiesce took 0.5 seconds. Will release after: 30.0 2024-04-23T10:38:28.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:38:28 smithi183 ceph-mon[32317]: pgmap v383: 97 pgs: 97 active+clean; 1.4 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.4 KiB/s wr, 14 op/s 2024-04-23T10:38:28.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:38:28 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/3682236568' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:38:28.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:38:28 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:38:28.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:38:28 smithi196 ceph-mon[32919]: pgmap v383: 97 pgs: 97 active+clean; 1.4 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.4 KiB/s wr, 14 op/s 2024-04-23T10:38:28.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:38:28 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/3682236568' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:38:28.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:38:28 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:38:28.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:38:28 smithi097 ceph-mon[29197]: pgmap v383: 97 pgs: 97 active+clean; 1.4 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.4 KiB/s wr, 14 op/s 2024-04-23T10:38:28.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:38:28 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/3682236568' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:38:28.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:38:28 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:38:29.645 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.643+0000 7f178e3fe640 1 -- 172.21.15.97:0/307465823 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1788102b70 msgr2=0x7f1788106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:29.645 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.643+0000 7f178e3fe640 1 --2- 172.21.15.97:0/307465823 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1788102b70 0x7f1788106bb0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f177c009900 tx=0x7f177c02a410 comp rx=0 tx=0).stop 2024-04-23T10:38:29.645 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.643+0000 7f178e3fe640 1 -- 172.21.15.97:0/307465823 shutdown_connections 2024-04-23T10:38:29.645 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.643+0000 7f178e3fe640 1 --2- 172.21.15.97:0/307465823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1788107290 0x7f178810ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:29.645 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.643+0000 7f178e3fe640 1 --2- 172.21.15.97:0/307465823 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1788102b70 0x7f1788106bb0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:29.645 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.643+0000 7f178e3fe640 1 --2- 172.21.15.97:0/307465823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17881021c0 0x7f17881025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.643+0000 7f178e3fe640 1 -- 172.21.15.97:0/307465823 >> 172.21.15.97:0/307465823 conn(0x7f17880fda20 msgr2=0x7f17880ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:38:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.643+0000 7f178e3fe640 1 -- 172.21.15.97:0/307465823 shutdown_connections 2024-04-23T10:38:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.643+0000 7f178e3fe640 1 -- 172.21.15.97:0/307465823 wait complete. 2024-04-23T10:38:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.644+0000 7f178e3fe640 1 Processor -- start 2024-04-23T10:38:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.644+0000 7f178e3fe640 1 -- start start 2024-04-23T10:38:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.644+0000 7f178e3fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17881021c0 0x7f1788198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.644+0000 7f178e3fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1788107290 0x7f1788198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.644+0000 7f178e3fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f178819cf80 0x7f178819d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.644+0000 7f178e3fe640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f17881121d0 con 0x7f1788107290 2024-04-23T10:38:29.647 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.644+0000 7f178e3fe640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1788112050 con 0x7f17881021c0 2024-04-23T10:38:29.647 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.644+0000 7f178e3fe640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f1788112350 con 0x7f178819cf80 2024-04-23T10:38:29.647 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.644+0000 7f17877fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1788107290 0x7f1788198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:29.647 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.644+0000 7f17877fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1788107290 0x7f1788198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52382/0 (socket says 172.21.15.97:52382) 2024-04-23T10:38:29.647 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.644+0000 7f17877fe640 1 -- 172.21.15.97:0/3213216951 learned_addr learned my addr 172.21.15.97:0/3213216951 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:38:29.647 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.644+0000 7f1787fff640 1 --2- 172.21.15.97:0/3213216951 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17881021c0 0x7f1788198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:29.647 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.644+0000 7f17877fe640 1 -- 172.21.15.97:0/3213216951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f178819cf80 msgr2=0x7f178819d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:29.647 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.644+0000 7f178c99b640 1 --2- 172.21.15.97:0/3213216951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f178819cf80 0x7f178819d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:29.647 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.644+0000 7f17877fe640 1 --2- 172.21.15.97:0/3213216951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f178819cf80 0x7f178819d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:29.647 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.645+0000 7f17877fe640 1 -- 172.21.15.97:0/3213216951 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17881021c0 msgr2=0x7f1788198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:29.647 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.645+0000 7f17877fe640 1 --2- 172.21.15.97:0/3213216951 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17881021c0 0x7f1788198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:29.648 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.645+0000 7f17877fe640 1 -- 172.21.15.97:0/3213216951 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f178819da70 con 0x7f1788107290 2024-04-23T10:38:29.648 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.645+0000 7f17877fe640 1 --2- 172.21.15.97:0/3213216951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1788107290 0x7f1788198c50 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f177c032fd0 tx=0x7f177c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:38:29.648 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.645+0000 7f17857fa640 1 -- 172.21.15.97:0/3213216951 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f177c03f030 con 0x7f1788107290 2024-04-23T10:38:29.648 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.645+0000 7f178e3fe640 1 -- 172.21.15.97:0/3213216951 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f178819dd00 con 0x7f1788107290 2024-04-23T10:38:29.648 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.645+0000 7f17857fa640 1 -- 172.21.15.97:0/3213216951 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f177c004490 con 0x7f1788107290 2024-04-23T10:38:29.648 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.645+0000 7f178e3fe640 1 -- 172.21.15.97:0/3213216951 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17881a5520 con 0x7f1788107290 2024-04-23T10:38:29.648 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.645+0000 7f17857fa640 1 -- 172.21.15.97:0/3213216951 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f177c049060 con 0x7f1788107290 2024-04-23T10:38:29.656 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.647+0000 7f17857fa640 1 -- 172.21.15.97:0/3213216951 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f177c042030 con 0x7f1788107290 2024-04-23T10:38:29.656 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.647+0000 7f17911ec740 1 Processor -- start 2024-04-23T10:38:29.656 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.647+0000 7f17911ec740 1 -- start start 2024-04-23T10:38:29.656 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.647+0000 7f17911ec740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e32672c620 0x55e32672ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:29.656 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.647+0000 7f17857fa640 1 --2- 172.21.15.97:0/3213216951 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f17700a3dc0 0x7f17700a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:29.656 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.647+0000 7f17911ec740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e32672cf40 0x55e3267372c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:29.657 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.647+0000 7f17911ec740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e326737800 0x55e326739bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:29.657 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.647+0000 7f17911ec740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e326691970 con 0x55e32672c620 2024-04-23T10:38:29.657 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.647+0000 7f17911ec740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e32662e4a0 con 0x55e32672cf40 2024-04-23T10:38:29.657 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.647+0000 7f17911ec740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e32668d930 con 0x55e326737800 2024-04-23T10:38:29.657 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.648+0000 7f178c99b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e32672c620 0x55e32672ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:29.657 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.648+0000 7f1787fff640 1 --2- 172.21.15.97:0/3213216951 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f17700a3dc0 0x7f17700a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:29.657 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.648+0000 7f178c99b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e32672c620 0x55e32672ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52396/0 (socket says 172.21.15.97:52396) 2024-04-23T10:38:29.657 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.648+0000 7f178c99b640 1 -- 172.21.15.97:0/512744485 learned_addr learned my addr 172.21.15.97:0/512744485 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:38:29.657 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.648+0000 7f17857fa640 1 -- 172.21.15.97:0/3213216951 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8484+0+0 (secure 0 0 0) 0x7f177c0f1060 con 0x7f1788107290 2024-04-23T10:38:29.657 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.648+0000 7f17877fe640 1 --2- 172.21.15.97:0/512744485 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e32672cf40 0x55e3267372c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:29.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.648+0000 7f178c99b640 1 --2- 172.21.15.97:0/512744485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e326737800 0x55e326739bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:29.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.648+0000 7f17877fe640 1 -- 172.21.15.97:0/512744485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e326737800 msgr2=0x55e326739bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:38:29.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.648+0000 7f17877fe640 1 --2- 172.21.15.97:0/512744485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e326737800 0x55e326739bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:29.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.648+0000 7f17877fe640 1 -- 172.21.15.97:0/512744485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e32672c620 msgr2=0x55e32672ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:38:29.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.648+0000 7f17877fe640 1 --2- 172.21.15.97:0/512744485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e32672c620 0x55e32672ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:29.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.648+0000 7f17877fe640 1 -- 172.21.15.97:0/512744485 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3263a4210 con 0x55e32672cf40 2024-04-23T10:38:29.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.648+0000 7f178c99b640 1 --2- 172.21.15.97:0/512744485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e32672c620 0x55e32672ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T10:38:29.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.648+0000 7f1787fff640 1 --2- 172.21.15.97:0/3213216951 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f17700a3dc0 0x7f17700a6280 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1778005fd0 tx=0x7f177800b390 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:38:29.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.648+0000 7f17877fe640 1 --2- 172.21.15.97:0/512744485 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e32672cf40 0x55e3267372c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f177c0ee7d0 tx=0x7f177c04c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=744b043e80dcf519 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:38:29.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.648+0000 7f176e7fc640 1 -- 172.21.15.97:0/512744485 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f177c035040 con 0x55e32672cf40 2024-04-23T10:38:29.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.648+0000 7f176e7fc640 1 -- 172.21.15.97:0/512744485 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f177c0bd720 con 0x55e32672cf40 2024-04-23T10:38:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.648+0000 7f176e7fc640 1 -- 172.21.15.97:0/512744485 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f177c0bd020 con 0x55e32672cf40 2024-04-23T10:38:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.649+0000 7f17911ec740 1 -- 172.21.15.97:0/512744485 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e32672cf40 msgr2=0x55e3267372c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.649+0000 7f17911ec740 1 --2- 172.21.15.97:0/512744485 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e32672cf40 0x55e3267372c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f177c0ee7d0 tx=0x7f177c04c000 comp rx=0 tx=0).stop 2024-04-23T10:38:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.649+0000 7f17911ec740 1 -- 172.21.15.97:0/512744485 shutdown_connections 2024-04-23T10:38:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.649+0000 7f17911ec740 1 --2- 172.21.15.97:0/512744485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e326737800 0x55e326739bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.649+0000 7f17911ec740 1 --2- 172.21.15.97:0/512744485 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e32672cf40 0x55e3267372c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.649+0000 7f17911ec740 1 --2- 172.21.15.97:0/512744485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e32672c620 0x55e32672ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.649+0000 7f17911ec740 1 -- 172.21.15.97:0/512744485 >> 172.21.15.97:0/512744485 conn(0x55e326728b30 msgr2=0x55e326728f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:38:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.649+0000 7f17911ec740 1 -- 172.21.15.97:0/512744485 shutdown_connections 2024-04-23T10:38:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.649+0000 7f17911ec740 1 -- 172.21.15.97:0/512744485 wait complete. 2024-04-23T10:38:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.649+0000 7f17911ec740 1 -- start start 2024-04-23T10:38:29.660 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.649+0000 7f17911ec740 1 Processor -- start 2024-04-23T10:38:29.660 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.650+0000 7f17911ec740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e32672c620 0x55e3267371d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:29.660 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.650+0000 7f17911ec740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e32672cf40 0x55e3267344c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:29.660 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.650+0000 7f17911ec740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e326737800 0x55e326734ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:29.660 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.650+0000 7f17911ec740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e32673a1f0 con 0x55e32672c620 2024-04-23T10:38:29.660 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.650+0000 7f17911ec740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e32662c0b0 con 0x55e32672cf40 2024-04-23T10:38:29.660 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.650+0000 7f17911ec740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55e32673a370 con 0x55e326737800 2024-04-23T10:38:29.660 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.650+0000 7f17911ec740 10 client.0 _ll_register_callbacks cb 0x55e326619d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:38:29.660 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.650+0000 7f1787fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e32672c620 0x55e3267371d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:29.660 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.650+0000 7f1787fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e32672c620 0x55e3267371d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52406/0 (socket says 172.21.15.97:52406) 2024-04-23T10:38:29.660 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.650+0000 7f1787fff640 1 -- 172.21.15.97:0/458867601 learned_addr learned my addr 172.21.15.97:0/458867601 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:38:29.661 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.650+0000 7f178c99b640 1 --2- 172.21.15.97:0/458867601 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e326737800 0x55e326734ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:29.661 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.650+0000 7f17877fe640 1 --2- 172.21.15.97:0/458867601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e32672cf40 0x55e3267344c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:29.661 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.650+0000 7f1787fff640 1 -- 172.21.15.97:0/458867601 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e326737800 msgr2=0x55e326734ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:38:29.661 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.650+0000 7f1787fff640 1 --2- 172.21.15.97:0/458867601 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e326737800 0x55e326734ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:29.661 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.650+0000 7f1787fff640 1 -- 172.21.15.97:0/458867601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e32672cf40 msgr2=0x55e3267344c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:38:29.661 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.650+0000 7f1787fff640 1 --2- 172.21.15.97:0/458867601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e32672cf40 0x55e3267344c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:29.661 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.650+0000 7f1787fff640 1 -- 172.21.15.97:0/458867601 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e326470620 con 0x55e32672c620 2024-04-23T10:38:29.661 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.650+0000 7f1787fff640 1 --2- 172.21.15.97:0/458867601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e32672c620 0x55e3267371d0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f177800b620 tx=0x7f1778004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcbdeb483e4ff1d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:38:29.661 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.650+0000 7f17677fe640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:38:29.661 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.650+0000 7f17677fe640 1 -- 172.21.15.97:0/458867601 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1778005220 con 0x55e32672c620 2024-04-23T10:38:29.661 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.650+0000 7f17677fe640 1 -- 172.21.15.97:0/458867601 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8484+0+0 (secure 0 0 0) 0x7f17780053c0 con 0x55e32672c620 2024-04-23T10:38:29.662 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.650+0000 7f17911ec740 1 -- 172.21.15.97:0/458867601 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f177c0bd720 con 0x55e32672c620 2024-04-23T10:38:29.662 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.650+0000 7f17677fe640 1 -- 172.21.15.97:0/458867601 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f17780055c0 con 0x55e32672c620 2024-04-23T10:38:29.662 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.651+0000 7f17911ec740 10 client.17337 fetch_fsmap learned FSMap version 26 2024-04-23T10:38:29.662 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.651+0000 7f17911ec740 1 -- 172.21.15.97:0/458867601 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e326746150 con 0x55e32672c620 2024-04-23T10:38:29.662 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.651+0000 7f17677fe640 1 -- 172.21.15.97:0/458867601 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f177c0bd720 con 0x55e32672c620 2024-04-23T10:38:29.662 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.651+0000 7f17677fe640 1 -- 172.21.15.97:0/458867601 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f177802d020 con 0x55e32672c620 2024-04-23T10:38:29.662 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.651+0000 7f17911ec740 10 client.17337 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:38:29.662 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.651+0000 7f17911ec740 10 client.17337 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:38:29.662 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.651+0000 7f17911ec740 20 client.17337 populate_metadata read hostname 'smithi097' 2024-04-23T10:38:29.662 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.651+0000 7f17911ec740 1 --2- 172.21.15.97:0/458867601 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55e326747610 0x55e326749a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:29.663 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.651+0000 7f17911ec740 4 client.17337 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:38:29.663 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.651+0000 7f17911ec740 1 -- 172.21.15.97:0/458867601 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e32669f390 con 0x55e326747610 2024-04-23T10:38:29.663 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.651+0000 7f17877fe640 1 --2- 172.21.15.97:0/458867601 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55e326747610 0x55e326749a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:29.663 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.651+0000 7f17677fe640 10 client.17337 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:38:29.663 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.651+0000 7f17877fe640 1 --2- 172.21.15.97:0/458867601 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55e326747610 0x55e326749a40 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b374fff8bed2b83 server_cookie=9e30310363056351 in_seq=0 out_seq=0 2024-04-23T10:38:29.663 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.652+0000 7f17677fe640 1 -- 172.21.15.97:0/458867601 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e32669f390 con 0x55e326747610 2024-04-23T10:38:29.663 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.652+0000 7f17677fe640 10 client.17337 handle_command_reply: tid=0 2024-04-23T10:38:29.663 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.652+0000 7f17911ec740 1 client.17337 shutdown 2024-04-23T10:38:29.663 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.652+0000 7f17911ec740 1 -- 172.21.15.97:0/458867601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e32672c620 msgr2=0x55e3267371d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:29.663 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.652+0000 7f17911ec740 1 --2- 172.21.15.97:0/458867601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e32672c620 0x55e3267371d0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f177800b620 tx=0x7f1778004730 comp rx=0 tx=0).stop 2024-04-23T10:38:29.663 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.652+0000 7f17911ec740 1 -- 172.21.15.97:0/458867601 shutdown_connections 2024-04-23T10:38:29.664 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.652+0000 7f17911ec740 1 --2- 172.21.15.97:0/458867601 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55e326747610 0x55e326749a40 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:29.664 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.652+0000 7f17911ec740 1 --2- 172.21.15.97:0/458867601 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e326737800 0x55e326734ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:29.664 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.652+0000 7f17911ec740 1 --2- 172.21.15.97:0/458867601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e32672cf40 0x55e3267344c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:29.664 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.652+0000 7f17677fe640 0 client.17337 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:38:29.664 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.652+0000 7f17911ec740 1 --2- 172.21.15.97:0/458867601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e32672c620 0x55e3267371d0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:29.664 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.652+0000 7f17911ec740 1 -- 172.21.15.97:0/458867601 >> 172.21.15.97:0/458867601 conn(0x55e326728b30 msgr2=0x55e32672e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:38:29.664 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.652+0000 7f17911ec740 1 -- 172.21.15.97:0/458867601 shutdown_connections 2024-04-23T10:38:29.664 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.652+0000 7f17911ec740 1 -- 172.21.15.97:0/458867601 wait complete. 2024-04-23T10:38:29.664 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.652+0000 7f17911ec740 20 client.17337 trim_cache size 0 max 16384 2024-04-23T10:38:29.671 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.667+0000 7f17911ec740 1 Processor -- start 2024-04-23T10:38:29.671 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.667+0000 7f17911ec740 1 -- start start 2024-04-23T10:38:29.671 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.667+0000 7f17911ec740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e32672c620 0x55e326742f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:29.671 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.667+0000 7f17911ec740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e32672cf40 0x55e326736d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:29.671 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.667+0000 7f17911ec740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e326737800 0x55e32674f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:29.671 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.667+0000 7f17911ec740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f17881adaf0 con 0x55e32672cf40 2024-04-23T10:38:29.672 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.667+0000 7f17911ec740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f17881af630 con 0x55e32672c620 2024-04-23T10:38:29.672 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.667+0000 7f17911ec740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f17881ae1c0 con 0x55e326737800 2024-04-23T10:38:29.672 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.667+0000 7f17877fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e32672cf40 0x55e326736d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:29.672 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.667+0000 7f17877fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e32672cf40 0x55e326736d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52408/0 (socket says 172.21.15.97:52408) 2024-04-23T10:38:29.672 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.667+0000 7f17877fe640 1 -- 172.21.15.97:0/353018658 learned_addr learned my addr 172.21.15.97:0/353018658 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:38:29.672 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.667+0000 7f178c99b640 1 --2- 172.21.15.97:0/353018658 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e326737800 0x55e32674f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:29.672 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.667+0000 7f1787fff640 1 --2- 172.21.15.97:0/353018658 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e32672c620 0x55e326742f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:29.672 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.667+0000 7f17877fe640 1 -- 172.21.15.97:0/353018658 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e326737800 msgr2=0x55e32674f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:38:29.672 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.667+0000 7f17877fe640 1 --2- 172.21.15.97:0/353018658 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e326737800 0x55e32674f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:29.672 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.667+0000 7f17877fe640 1 -- 172.21.15.97:0/353018658 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e32672c620 msgr2=0x55e326742f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:38:29.672 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.667+0000 7f17877fe640 1 --2- 172.21.15.97:0/353018658 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e32672c620 0x55e326742f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:29.673 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.667+0000 7f17877fe640 1 -- 172.21.15.97:0/353018658 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e326733490 con 0x55e32672cf40 2024-04-23T10:38:29.673 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.668+0000 7f17877fe640 1 --2- 172.21.15.97:0/353018658 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e32672cf40 0x55e326736d00 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f177c0b9430 tx=0x7f177c0c1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=475edb9cabcde08b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:38:29.673 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.668+0000 7f176dffb640 1 -- 172.21.15.97:0/353018658 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f177c035040 con 0x55e32672cf40 2024-04-23T10:38:29.673 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.668+0000 7f176dffb640 1 -- 172.21.15.97:0/353018658 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f177c0bd720 con 0x55e32672cf40 2024-04-23T10:38:29.673 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.668+0000 7f176dffb640 1 -- 172.21.15.97:0/353018658 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f177c0bb320 con 0x55e32672cf40 2024-04-23T10:38:29.673 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.668+0000 7f17911ec740 1 -- 172.21.15.97:0/353018658 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e32672cf40 msgr2=0x55e326736d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:29.673 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.668+0000 7f17911ec740 1 --2- 172.21.15.97:0/353018658 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e32672cf40 0x55e326736d00 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f177c0b9430 tx=0x7f177c0c1040 comp rx=0 tx=0).stop 2024-04-23T10:38:29.673 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.668+0000 7f17911ec740 1 -- 172.21.15.97:0/353018658 shutdown_connections 2024-04-23T10:38:29.673 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.668+0000 7f17911ec740 1 --2- 172.21.15.97:0/353018658 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e326737800 0x55e32674f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:29.673 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.668+0000 7f17911ec740 1 --2- 172.21.15.97:0/353018658 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e32672cf40 0x55e326736d00 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:29.673 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.668+0000 7f17911ec740 1 --2- 172.21.15.97:0/353018658 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e32672c620 0x55e326742f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:29.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.668+0000 7f17911ec740 1 -- 172.21.15.97:0/353018658 >> 172.21.15.97:0/353018658 conn(0x55e326728b30 msgr2=0x55e326727cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:38:29.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.668+0000 7f17911ec740 1 -- 172.21.15.97:0/353018658 shutdown_connections 2024-04-23T10:38:29.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.668+0000 7f17911ec740 1 -- 172.21.15.97:0/353018658 wait complete. 2024-04-23T10:38:29.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.669+0000 7f17911ec740 1 -- start start 2024-04-23T10:38:29.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.669+0000 7f17911ec740 1 Processor -- start 2024-04-23T10:38:29.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.669+0000 7f17911ec740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e32672c620 0x55e326731590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:29.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.669+0000 7f17911ec740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e32672cf40 0x55e326731ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:29.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.669+0000 7f17911ec740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e326737800 0x55e3267acbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:29.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.669+0000 7f17911ec740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f17881aef60 con 0x55e32672c620 2024-04-23T10:38:29.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.669+0000 7f17911ec740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f17881ae890 con 0x55e32672cf40 2024-04-23T10:38:29.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.669+0000 7f17911ec740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f17881abaf0 con 0x55e326737800 2024-04-23T10:38:29.674 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.669+0000 7f17911ec740 10 client.0 _ll_register_callbacks cb 0x7f1778003b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:38:29.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.669+0000 7f1787fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e32672c620 0x55e326731590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:29.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.669+0000 7f1787fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e32672c620 0x55e326731590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52422/0 (socket says 172.21.15.97:52422) 2024-04-23T10:38:29.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.669+0000 7f1787fff640 1 -- 172.21.15.97:0/1825230104 learned_addr learned my addr 172.21.15.97:0/1825230104 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:38:29.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.669+0000 7f17877fe640 1 --2- 172.21.15.97:0/1825230104 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e32672cf40 0x55e326731ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:29.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.669+0000 7f178c99b640 1 --2- 172.21.15.97:0/1825230104 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e326737800 0x55e3267acbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:29.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.669+0000 7f1787fff640 1 -- 172.21.15.97:0/1825230104 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e326737800 msgr2=0x55e3267acbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:38:29.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.669+0000 7f1787fff640 1 --2- 172.21.15.97:0/1825230104 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e326737800 0x55e3267acbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:29.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.669+0000 7f1787fff640 1 -- 172.21.15.97:0/1825230104 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e32672cf40 msgr2=0x55e326731ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:38:29.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.669+0000 7f1787fff640 1 --2- 172.21.15.97:0/1825230104 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e32672cf40 0x55e326731ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:29.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.669+0000 7f1787fff640 1 -- 172.21.15.97:0/1825230104 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e326734e40 con 0x55e32672c620 2024-04-23T10:38:29.675 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.669+0000 7f17877fe640 1 --2- 172.21.15.97:0/1825230104 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e32672cf40 0x55e326731ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T10:38:29.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.670+0000 7f1787fff640 1 --2- 172.21.15.97:0/1825230104 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e32672c620 0x55e326731590 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f177802f980 tx=0x7f1778030e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1eb55495aae0770 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:38:29.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.670+0000 7f17677fe640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:38:29.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.670+0000 7f17677fe640 1 -- 172.21.15.97:0/1825230104 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1778035040 con 0x55e32672c620 2024-04-23T10:38:29.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.670+0000 7f17677fe640 1 -- 172.21.15.97:0/1825230104 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8484+0+0 (secure 0 0 0) 0x7f177801e420 con 0x55e32672c620 2024-04-23T10:38:29.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.670+0000 7f17911ec740 1 -- 172.21.15.97:0/1825230104 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f177c0bd720 con 0x55e32672c620 2024-04-23T10:38:29.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.670+0000 7f17677fe640 1 -- 172.21.15.97:0/1825230104 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f177802fb20 con 0x55e32672c620 2024-04-23T10:38:29.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.670+0000 7f17911ec740 10 client.17343 fetch_fsmap learned FSMap version 26 2024-04-23T10:38:29.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.670+0000 7f17911ec740 1 -- 172.21.15.97:0/1825230104 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e3267ae0e0 con 0x55e32672c620 2024-04-23T10:38:29.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.670+0000 7f17677fe640 1 -- 172.21.15.97:0/1825230104 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f177c0bd720 con 0x55e32672c620 2024-04-23T10:38:29.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.670+0000 7f17677fe640 1 -- 172.21.15.97:0/1825230104 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f177802d040 con 0x55e32672c620 2024-04-23T10:38:29.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.670+0000 7f17911ec740 10 client.17343 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:38:29.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.670+0000 7f17911ec740 10 client.17343 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:38:29.676 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.670+0000 7f17911ec740 20 client.17343 populate_metadata read hostname 'smithi097' 2024-04-23T10:38:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.670+0000 7f17911ec740 1 --2- 172.21.15.97:0/1825230104 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55e3267ae880 0x55e3267b0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.670+0000 7f17911ec740 4 client.17343 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:38:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.670+0000 7f17911ec740 1 -- 172.21.15.97:0/1825230104 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e326699200 con 0x55e3267ae880 2024-04-23T10:38:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.670+0000 7f17877fe640 1 --2- 172.21.15.97:0/1825230104 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55e3267ae880 0x55e3267b0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.671+0000 7f17677fe640 10 client.17343 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:38:29.677 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.671+0000 7f17877fe640 1 --2- 172.21.15.97:0/1825230104 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55e3267ae880 0x55e3267b0ca0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10d040666d276407 server_cookie=5b5266359020503c in_seq=0 out_seq=0 2024-04-23T10:38:29.680 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.680+0000 7f17677fe640 1 -- 172.21.15.97:0/1825230104 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e326699200 con 0x55e3267ae880 2024-04-23T10:38:29.680 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.680+0000 7f17677fe640 10 client.17343 handle_command_reply: tid=0 2024-04-23T10:38:29.680 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.681+0000 7f17911ec740 1 client.17343 shutdown 2024-04-23T10:38:29.680 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.681+0000 7f17911ec740 1 -- 172.21.15.97:0/1825230104 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e32672c620 msgr2=0x55e326731590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:29.681 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.681+0000 7f17911ec740 1 --2- 172.21.15.97:0/1825230104 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e32672c620 0x55e326731590 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f177802f980 tx=0x7f1778030e30 comp rx=0 tx=0).stop 2024-04-23T10:38:29.681 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.681+0000 7f17911ec740 1 -- 172.21.15.97:0/1825230104 shutdown_connections 2024-04-23T10:38:29.681 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.681+0000 7f17911ec740 1 --2- 172.21.15.97:0/1825230104 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55e3267ae880 0x55e3267b0ca0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:29.681 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.681+0000 7f17911ec740 1 --2- 172.21.15.97:0/1825230104 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e326737800 0x55e3267acbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:29.681 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.681+0000 7f17677fe640 0 client.17343 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:38:29.681 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.681+0000 7f17911ec740 1 --2- 172.21.15.97:0/1825230104 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e32672cf40 0x55e326731ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:29.681 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.681+0000 7f17911ec740 1 --2- 172.21.15.97:0/1825230104 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e32672c620 0x55e326731590 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:29.681 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.681+0000 7f17911ec740 1 -- 172.21.15.97:0/1825230104 >> 172.21.15.97:0/1825230104 conn(0x55e326728b30 msgr2=0x55e326727420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:38:29.681 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.681+0000 7f17911ec740 1 -- 172.21.15.97:0/1825230104 shutdown_connections 2024-04-23T10:38:29.681 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.681+0000 7f17911ec740 1 -- 172.21.15.97:0/1825230104 wait complete. 2024-04-23T10:38:29.681 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.681+0000 7f17911ec740 20 client.17343 trim_cache size 0 max 16384 2024-04-23T10:38:29.682 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:38:29.682 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:38:29.682 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:38:29.682 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:38:29.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.681+0000 7f178e3fe640 1 -- 172.21.15.97:0/3213216951 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f17700a3dc0 msgr2=0x7f17700a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:29.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.681+0000 7f178e3fe640 1 --2- 172.21.15.97:0/3213216951 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f17700a3dc0 0x7f17700a6280 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1778005fd0 tx=0x7f177800b390 comp rx=0 tx=0).stop 2024-04-23T10:38:29.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.681+0000 7f178e3fe640 1 -- 172.21.15.97:0/3213216951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1788107290 msgr2=0x7f1788198c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:29.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.682+0000 7f178e3fe640 1 --2- 172.21.15.97:0/3213216951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1788107290 0x7f1788198c50 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f177c032fd0 tx=0x7f177c004270 comp rx=0 tx=0).stop 2024-04-23T10:38:29.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.682+0000 7f178e3fe640 1 -- 172.21.15.97:0/3213216951 shutdown_connections 2024-04-23T10:38:29.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.682+0000 7f178e3fe640 1 --2- 172.21.15.97:0/3213216951 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f17700a3dc0 0x7f17700a6280 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:29.683 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.682+0000 7f178e3fe640 1 --2- 172.21.15.97:0/3213216951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f178819cf80 0x7f178819d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:29.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.682+0000 7f178e3fe640 1 --2- 172.21.15.97:0/3213216951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1788107290 0x7f1788198c50 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:29.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.682+0000 7f178e3fe640 1 --2- 172.21.15.97:0/3213216951 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17881021c0 0x7f1788198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:29.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.682+0000 7f178e3fe640 1 -- 172.21.15.97:0/3213216951 >> 172.21.15.97:0/3213216951 conn(0x7f17880fda20 msgr2=0x7f17880ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:38:29.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.682+0000 7f178e3fe640 1 -- 172.21.15.97:0/3213216951 shutdown_connections 2024-04-23T10:38:29.684 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:29.682+0000 7f178e3fe640 1 -- 172.21.15.97:0/3213216951 wait complete. 2024-04-23T10:38:29.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:38:29 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:38:29.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:38:29 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:38:29.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:38:29 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:38:29.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:38:29 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:38:29.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:38:29 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:38:29.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:38:29 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:38:29.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:38:29 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:38:30.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:38:29 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:38:30.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:38:29 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:38:30.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:38:29 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:38:30.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:38:29 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:38:30.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:38:29 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:38:30.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:38:29 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:38:30.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:38:29 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:38:30.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:38:29 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:38:30.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:38:29 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:38:30.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:38:29 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:38:30.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:38:29 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:38:30.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:38:29 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:38:30.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:38:29 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:38:30.111 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:38:29 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:38:30.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:38:30 smithi183 ceph-mon[32317]: pgmap v384: 97 pgs: 97 active+clean; 1.4 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1023 B/s wr, 8 op/s 2024-04-23T10:38:31.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:38:30 smithi196 ceph-mon[32919]: pgmap v384: 97 pgs: 97 active+clean; 1.4 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1023 B/s wr, 8 op/s 2024-04-23T10:38:31.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:38:30 smithi097 ceph-mon[29197]: pgmap v384: 97 pgs: 97 active+clean; 1.4 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1023 B/s wr, 8 op/s 2024-04-23T10:38:32.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:38:32 smithi183 ceph-mon[32317]: pgmap v385: 97 pgs: 97 active+clean; 1.4 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1023 B/s wr, 8 op/s 2024-04-23T10:38:33.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:38:32 smithi196 ceph-mon[32919]: pgmap v385: 97 pgs: 97 active+clean; 1.4 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1023 B/s wr, 8 op/s 2024-04-23T10:38:33.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:38:32 smithi097 ceph-mon[29197]: pgmap v385: 97 pgs: 97 active+clean; 1.4 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1023 B/s wr, 8 op/s 2024-04-23T10:38:34.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:38:34 smithi183 ceph-mon[32317]: pgmap v386: 97 pgs: 97 active+clean; 1.5 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-23T10:38:35.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:38:34 smithi196 ceph-mon[32919]: pgmap v386: 97 pgs: 97 active+clean; 1.5 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-23T10:38:35.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:38:34 smithi097 ceph-mon[29197]: pgmap v386: 97 pgs: 97 active+clean; 1.5 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-23T10:38:36.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:38:36 smithi183 ceph-mon[32317]: pgmap v387: 97 pgs: 97 active+clean; 1.5 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-23T10:38:37.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:38:36 smithi196 ceph-mon[32919]: pgmap v387: 97 pgs: 97 active+clean; 1.5 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-23T10:38:37.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:38:36 smithi097 ceph-mon[29197]: pgmap v387: 97 pgs: 97 active+clean; 1.5 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-23T10:38:38.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:38:38 smithi183 ceph-mon[32317]: pgmap v388: 97 pgs: 97 active+clean; 1.5 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-23T10:38:39.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:38:38 smithi196 ceph-mon[32919]: pgmap v388: 97 pgs: 97 active+clean; 1.5 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-23T10:38:39.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:38:38 smithi097 ceph-mon[29197]: pgmap v388: 97 pgs: 97 active+clean; 1.5 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-23T10:38:40.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:38:40 smithi183 ceph-mon[32317]: pgmap v389: 97 pgs: 97 active+clean; 1.5 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-04-23T10:38:40.913 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:38:40 smithi097 ceph-mon[29197]: pgmap v389: 97 pgs: 97 active+clean; 1.5 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-04-23T10:38:41.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:38:40 smithi196 ceph-mon[32919]: pgmap v389: 97 pgs: 97 active+clean; 1.5 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-04-23T10:38:42.779 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.777+0000 7f6dbb120640 1 -- 172.21.15.97:0/1619940314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6db4103b20 msgr2=0x7f6db410c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:42.779 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.777+0000 7f6dbb120640 1 --2- 172.21.15.97:0/1619940314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6db4103b20 0x7f6db410c000 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f6da0009900 tx=0x7f6da002a480 comp rx=0 tx=0).stop 2024-04-23T10:38:42.780 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.777+0000 7f6dbb120640 1 -- 172.21.15.97:0/1619940314 shutdown_connections 2024-04-23T10:38:42.780 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.777+0000 7f6dbb120640 1 --2- 172.21.15.97:0/1619940314 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6db410c5d0 0x7f6db410ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:42.780 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.777+0000 7f6dbb120640 1 --2- 172.21.15.97:0/1619940314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6db4103b20 0x7f6db410c000 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:42.780 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.777+0000 7f6dbb120640 1 --2- 172.21.15.97:0/1619940314 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6db4103200 0x7f6db41035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:42.780 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.777+0000 7f6dbb120640 1 -- 172.21.15.97:0/1619940314 >> 172.21.15.97:0/1619940314 conn(0x7f6db40771d0 msgr2=0x7f6db40775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:38:42.780 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.777+0000 7f6dbb120640 1 -- 172.21.15.97:0/1619940314 shutdown_connections 2024-04-23T10:38:42.780 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.777+0000 7f6dbb120640 1 -- 172.21.15.97:0/1619940314 wait complete. 2024-04-23T10:38:42.780 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.778+0000 7f6dbb120640 1 Processor -- start 2024-04-23T10:38:42.780 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.778+0000 7f6dbb120640 1 -- start start 2024-04-23T10:38:42.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.778+0000 7f6dbb120640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6db4103200 0x7f6db4100e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:42.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.778+0000 7f6dbb120640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6db4103b20 0x7f6db4101390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:42.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.778+0000 7f6dbb120640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6db410c5d0 0x7f6db41018d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:42.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.778+0000 7f6dbb120640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6db4112270 con 0x7f6db4103200 2024-04-23T10:38:42.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.778+0000 7f6dbb120640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6db41120f0 con 0x7f6db410c5d0 2024-04-23T10:38:42.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.778+0000 7f6dbb120640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6db41123f0 con 0x7f6db4103b20 2024-04-23T10:38:42.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.778+0000 7f6db8ebc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6db4103200 0x7f6db4100e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:42.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.778+0000 7f6db8ebc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6db4103200 0x7f6db4100e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52982/0 (socket says 172.21.15.97:52982) 2024-04-23T10:38:42.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.778+0000 7f6db8ebc640 1 -- 172.21.15.97:0/3355686871 learned_addr learned my addr 172.21.15.97:0/3355686871 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:38:42.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.778+0000 7f6db96bd640 1 --2- 172.21.15.97:0/3355686871 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6db410c5d0 0x7f6db41018d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:42.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.778+0000 7f6db3fff640 1 --2- 172.21.15.97:0/3355686871 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6db4103b20 0x7f6db4101390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:42.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.778+0000 7f6db8ebc640 1 -- 172.21.15.97:0/3355686871 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6db4103b20 msgr2=0x7f6db4101390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:42.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.778+0000 7f6db8ebc640 1 --2- 172.21.15.97:0/3355686871 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6db4103b20 0x7f6db4101390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:42.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.778+0000 7f6db8ebc640 1 -- 172.21.15.97:0/3355686871 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6db410c5d0 msgr2=0x7f6db41018d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:42.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.778+0000 7f6db8ebc640 1 --2- 172.21.15.97:0/3355686871 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6db410c5d0 0x7f6db41018d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:42.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.778+0000 7f6db8ebc640 1 -- 172.21.15.97:0/3355686871 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6db41a6b90 con 0x7f6db4103200 2024-04-23T10:38:42.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.778+0000 7f6db96bd640 1 --2- 172.21.15.97:0/3355686871 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6db410c5d0 0x7f6db41018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:38:42.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.779+0000 7f6db8ebc640 1 --2- 172.21.15.97:0/3355686871 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6db4103200 0x7f6db4100e50 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f6da800b4a0 tx=0x7f6da800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:38:42.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.779+0000 7f6db1ffb640 1 -- 172.21.15.97:0/3355686871 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da8014070 con 0x7f6db4103200 2024-04-23T10:38:42.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.779+0000 7f6db1ffb640 1 -- 172.21.15.97:0/3355686871 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6da8010ac0 con 0x7f6db4103200 2024-04-23T10:38:42.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.779+0000 7f6dbb120640 1 -- 172.21.15.97:0/3355686871 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6db41a6e80 con 0x7f6db4103200 2024-04-23T10:38:42.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.779+0000 7f6db1ffb640 1 -- 172.21.15.97:0/3355686871 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da8017c30 con 0x7f6db4103200 2024-04-23T10:38:42.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.779+0000 7f6dbb120640 1 -- 172.21.15.97:0/3355686871 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6db41a72f0 con 0x7f6db4103200 2024-04-23T10:38:42.791 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.781+0000 7f6db1ffb640 1 -- 172.21.15.97:0/3355686871 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6da8017070 con 0x7f6db4103200 2024-04-23T10:38:42.792 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.782+0000 7f6dbde36740 1 Processor -- start 2024-04-23T10:38:42.792 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.782+0000 7f6db1ffb640 1 --2- 172.21.15.97:0/3355686871 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f6da40a3df0 0x7f6da40a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:42.792 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.782+0000 7f6dbde36740 1 -- start start 2024-04-23T10:38:42.792 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.782+0000 7f6db1ffb640 1 -- 172.21.15.97:0/3355686871 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8484+0+0 (secure 0 0 0) 0x7f6da80cfa90 con 0x7f6db4103200 2024-04-23T10:38:42.792 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.782+0000 7f6db96bd640 1 --2- 172.21.15.97:0/3355686871 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f6da40a3df0 0x7f6da40a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:42.792 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.782+0000 7f6dbde36740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557226e36620 0x557226e36a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:42.792 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.782+0000 7f6dbde36740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557226e36f40 0x557226e412c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:42.792 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.782+0000 7f6dbde36740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557226e41800 0x557226e43bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:42.792 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.782+0000 7f6dbde36740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x557226d9b970 con 0x557226e41800 2024-04-23T10:38:42.792 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.782+0000 7f6dbde36740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557226d384a0 con 0x557226e36f40 2024-04-23T10:38:42.793 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.782+0000 7f6dbde36740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557226d97930 con 0x557226e36620 2024-04-23T10:38:42.793 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.782+0000 7f6db96bd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557226e41800 0x557226e43bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:42.793 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.782+0000 7f6db96bd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557226e41800 0x557226e43bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52986/0 (socket says 172.21.15.97:52986) 2024-04-23T10:38:42.793 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.782+0000 7f6db96bd640 1 -- 172.21.15.97:0/3055191397 learned_addr learned my addr 172.21.15.97:0/3055191397 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:38:42.793 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.782+0000 7f6db8ebc640 1 --2- 172.21.15.97:0/3055191397 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557226e36620 0x557226e36a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:42.793 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.782+0000 7f6db3fff640 1 --2- 172.21.15.97:0/3055191397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557226e36f40 0x557226e412c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:42.793 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.782+0000 7f6db96bd640 1 --2- 172.21.15.97:0/3355686871 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f6da40a3df0 0x7f6da40a62b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6dac012720 tx=0x7f6dac012b20 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:38:42.793 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.782+0000 7f6db96bd640 1 -- 172.21.15.97:0/3055191397 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557226e36620 msgr2=0x557226e36a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:38:42.793 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.782+0000 7f6db96bd640 1 --2- 172.21.15.97:0/3055191397 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557226e36620 0x557226e36a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:42.793 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.782+0000 7f6db96bd640 1 -- 172.21.15.97:0/3055191397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557226e36f40 msgr2=0x557226e412c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:38:42.793 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.782+0000 7f6db96bd640 1 --2- 172.21.15.97:0/3055191397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557226e36f40 0x557226e412c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:42.794 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.782+0000 7f6db96bd640 1 -- 172.21.15.97:0/3055191397 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557226aae210 con 0x557226e41800 2024-04-23T10:38:42.794 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.783+0000 7f6db96bd640 1 --2- 172.21.15.97:0/3055191397 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557226e41800 0x557226e43bf0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f6dac010740 tx=0x7f6dac01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=200880c09c585d4d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:38:42.794 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.783+0000 7f6d9effd640 1 -- 172.21.15.97:0/3055191397 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dac004d20 con 0x557226e41800 2024-04-23T10:38:42.794 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.783+0000 7f6d9effd640 1 -- 172.21.15.97:0/3055191397 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6dac003e20 con 0x557226e41800 2024-04-23T10:38:42.794 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.783+0000 7f6d9effd640 1 -- 172.21.15.97:0/3055191397 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dac004220 con 0x557226e41800 2024-04-23T10:38:42.794 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.783+0000 7f6dbde36740 1 -- 172.21.15.97:0/3055191397 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557226e41800 msgr2=0x557226e43bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:42.794 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.783+0000 7f6dbde36740 1 --2- 172.21.15.97:0/3055191397 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557226e41800 0x557226e43bf0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f6dac010740 tx=0x7f6dac01c000 comp rx=0 tx=0).stop 2024-04-23T10:38:42.794 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.783+0000 7f6dbde36740 1 -- 172.21.15.97:0/3055191397 shutdown_connections 2024-04-23T10:38:42.794 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.783+0000 7f6dbde36740 1 --2- 172.21.15.97:0/3055191397 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557226e41800 0x557226e43bf0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:42.794 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.783+0000 7f6dbde36740 1 --2- 172.21.15.97:0/3055191397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557226e36f40 0x557226e412c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.783+0000 7f6dbde36740 1 --2- 172.21.15.97:0/3055191397 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557226e36620 0x557226e36a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.783+0000 7f6dbde36740 1 -- 172.21.15.97:0/3055191397 >> 172.21.15.97:0/3055191397 conn(0x557226e32b30 msgr2=0x557226e32f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:38:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.783+0000 7f6dbde36740 1 -- 172.21.15.97:0/3055191397 shutdown_connections 2024-04-23T10:38:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.783+0000 7f6dbde36740 1 -- 172.21.15.97:0/3055191397 wait complete. 2024-04-23T10:38:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.784+0000 7f6dbde36740 1 -- start start 2024-04-23T10:38:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.784+0000 7f6dbde36740 1 Processor -- start 2024-04-23T10:38:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.784+0000 7f6dbde36740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557226e36620 0x557226e41180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.784+0000 7f6dbde36740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557226e36f40 0x557226e3e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.784+0000 7f6dbde36740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557226e41800 0x557226e3e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.784+0000 7f6dbde36740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x557226e443d0 con 0x557226e36f40 2024-04-23T10:38:42.796 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.784+0000 7f6dbde36740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557226d360b0 con 0x557226e41800 2024-04-23T10:38:42.796 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.784+0000 7f6dbde36740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557226e44550 con 0x557226e36620 2024-04-23T10:38:42.796 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.784+0000 7f6dbde36740 10 client.0 _ll_register_callbacks cb 0x557226d23d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:38:42.796 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.784+0000 7f6db3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557226e36f40 0x557226e3e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:42.796 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.784+0000 7f6db3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557226e36f40 0x557226e3e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52992/0 (socket says 172.21.15.97:52992) 2024-04-23T10:38:42.796 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.784+0000 7f6db3fff640 1 -- 172.21.15.97:0/1603771357 learned_addr learned my addr 172.21.15.97:0/1603771357 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:38:42.796 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.785+0000 7f6db96bd640 1 --2- 172.21.15.97:0/1603771357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557226e41800 0x557226e3e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:42.796 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.785+0000 7f6db8ebc640 1 --2- 172.21.15.97:0/1603771357 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557226e36620 0x557226e41180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:42.796 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.785+0000 7f6db3fff640 1 -- 172.21.15.97:0/1603771357 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557226e36620 msgr2=0x557226e41180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:38:42.796 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.785+0000 7f6db3fff640 1 --2- 172.21.15.97:0/1603771357 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557226e36620 0x557226e41180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:42.796 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.785+0000 7f6db3fff640 1 -- 172.21.15.97:0/1603771357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557226e41800 msgr2=0x557226e3e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:38:42.797 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.785+0000 7f6db3fff640 1 --2- 172.21.15.97:0/1603771357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557226e41800 0x557226e3e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:42.797 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.785+0000 7f6db3fff640 1 -- 172.21.15.97:0/1603771357 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557226b7a620 con 0x557226e36f40 2024-04-23T10:38:42.797 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.785+0000 7f6db3fff640 1 --2- 172.21.15.97:0/1603771357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557226e36f40 0x557226e3e470 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f6da002a9a0 tx=0x7f6da0031750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b273a7414407a1ac server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:38:42.797 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.785+0000 7f6d93fff640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:38:42.797 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.785+0000 7f6d93fff640 1 -- 172.21.15.97:0/1603771357 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da0035070 con 0x557226e36f40 2024-04-23T10:38:42.797 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.785+0000 7f6dbde36740 1 -- 172.21.15.97:0/1603771357 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6dac003e20 con 0x557226e36f40 2024-04-23T10:38:42.797 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.785+0000 7f6d93fff640 1 -- 172.21.15.97:0/1603771357 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8484+0+0 (secure 0 0 0) 0x7f6da0004240 con 0x557226e36f40 2024-04-23T10:38:42.797 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.785+0000 7f6d93fff640 1 -- 172.21.15.97:0/1603771357 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6da0004440 con 0x557226e36f40 2024-04-23T10:38:42.797 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.785+0000 7f6dbde36740 10 client.17355 fetch_fsmap learned FSMap version 26 2024-04-23T10:38:42.797 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.785+0000 7f6dbde36740 1 -- 172.21.15.97:0/1603771357 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557226e50320 con 0x557226e36f40 2024-04-23T10:38:42.797 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.785+0000 7f6d93fff640 1 -- 172.21.15.97:0/1603771357 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6dac003e20 con 0x557226e36f40 2024-04-23T10:38:42.798 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.786+0000 7f6d93fff640 1 -- 172.21.15.97:0/1603771357 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f6da0044640 con 0x557226e36f40 2024-04-23T10:38:42.798 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.786+0000 7f6dbde36740 10 client.17355 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:38:42.798 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.786+0000 7f6dbde36740 10 client.17355 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:38:42.798 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.786+0000 7f6dbde36740 20 client.17355 populate_metadata read hostname 'smithi097' 2024-04-23T10:38:42.798 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.786+0000 7f6dbde36740 1 --2- 172.21.15.97:0/1603771357 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x557226e517e0 0x557226e53bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:42.798 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.786+0000 7f6dbde36740 4 client.17355 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:38:42.798 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.786+0000 7f6dbde36740 1 -- 172.21.15.97:0/1603771357 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557226da9390 con 0x557226e517e0 2024-04-23T10:38:42.798 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.786+0000 7f6db8ebc640 1 --2- 172.21.15.97:0/1603771357 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x557226e517e0 0x557226e53bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:42.798 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.786+0000 7f6db8ebc640 1 --2- 172.21.15.97:0/1603771357 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x557226e517e0 0x557226e53bd0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2aec187d38c7ba75 server_cookie=ac3346d00411a5f3 in_seq=0 out_seq=0 2024-04-23T10:38:42.798 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.786+0000 7f6d93fff640 10 client.17355 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:38:42.798 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.787+0000 7f6d93fff640 1 -- 172.21.15.97:0/1603771357 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557226da9390 con 0x557226e517e0 2024-04-23T10:38:42.799 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.787+0000 7f6d93fff640 10 client.17355 handle_command_reply: tid=0 2024-04-23T10:38:42.799 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.787+0000 7f6dbde36740 1 client.17355 shutdown 2024-04-23T10:38:42.799 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.788+0000 7f6dbde36740 1 -- 172.21.15.97:0/1603771357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557226e36f40 msgr2=0x557226e3e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:42.799 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.788+0000 7f6dbde36740 1 --2- 172.21.15.97:0/1603771357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557226e36f40 0x557226e3e470 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f6da002a9a0 tx=0x7f6da0031750 comp rx=0 tx=0).stop 2024-04-23T10:38:42.799 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.788+0000 7f6dbde36740 1 -- 172.21.15.97:0/1603771357 shutdown_connections 2024-04-23T10:38:42.799 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.788+0000 7f6dbde36740 1 --2- 172.21.15.97:0/1603771357 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x557226e517e0 0x557226e53bd0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:42.799 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.788+0000 7f6dbde36740 1 --2- 172.21.15.97:0/1603771357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557226e41800 0x557226e3e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:42.799 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.788+0000 7f6dbde36740 1 --2- 172.21.15.97:0/1603771357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557226e36f40 0x557226e3e470 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:42.799 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.788+0000 7f6dbde36740 1 --2- 172.21.15.97:0/1603771357 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557226e36620 0x557226e41180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:42.799 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.788+0000 7f6d93fff640 0 client.17355 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:38:42.799 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.788+0000 7f6dbde36740 1 -- 172.21.15.97:0/1603771357 >> 172.21.15.97:0/1603771357 conn(0x557226e32b30 msgr2=0x557226e38860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:38:42.800 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.788+0000 7f6dbde36740 1 -- 172.21.15.97:0/1603771357 shutdown_connections 2024-04-23T10:38:42.800 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.788+0000 7f6dbde36740 1 -- 172.21.15.97:0/1603771357 wait complete. 2024-04-23T10:38:42.800 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.788+0000 7f6dbde36740 20 client.17355 trim_cache size 0 max 16384 2024-04-23T10:38:42.807 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.803+0000 7f6dbde36740 1 Processor -- start 2024-04-23T10:38:42.807 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.803+0000 7f6dbde36740 1 -- start start 2024-04-23T10:38:42.807 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.803+0000 7f6dbde36740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557226e36620 0x557226e32430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:42.808 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.803+0000 7f6dbde36740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557226e36f40 0x557226e40cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:42.808 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.803+0000 7f6dbde36740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557226e41800 0x557226e594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:42.808 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.803+0000 7f6dbde36740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6db41afbc0 con 0x557226e36620 2024-04-23T10:38:42.808 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.803+0000 7f6dbde36740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6db41b1700 con 0x557226e41800 2024-04-23T10:38:42.808 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.803+0000 7f6dbde36740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6db41b0290 con 0x557226e36f40 2024-04-23T10:38:42.808 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.803+0000 7f6db8ebc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557226e36620 0x557226e32430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:42.808 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.803+0000 7f6db8ebc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557226e36620 0x557226e32430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:52996/0 (socket says 172.21.15.97:52996) 2024-04-23T10:38:42.808 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.803+0000 7f6db8ebc640 1 -- 172.21.15.97:0/1527621276 learned_addr learned my addr 172.21.15.97:0/1527621276 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:38:42.808 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.803+0000 7f6db8ebc640 1 -- 172.21.15.97:0/1527621276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557226e36f40 msgr2=0x557226e40cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:38:42.808 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.803+0000 7f6db3fff640 1 --2- 172.21.15.97:0/1527621276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557226e36f40 0x557226e40cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:42.808 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.803+0000 7f6db8ebc640 1 --2- 172.21.15.97:0/1527621276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557226e36f40 0x557226e40cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.803+0000 7f6db8ebc640 1 -- 172.21.15.97:0/1527621276 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557226e41800 msgr2=0x557226e594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:38:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.803+0000 7f6db8ebc640 1 --2- 172.21.15.97:0/1527621276 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557226e41800 0x557226e594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.803+0000 7f6db8ebc640 1 -- 172.21.15.97:0/1527621276 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557226e54230 con 0x557226e36620 2024-04-23T10:38:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.804+0000 7f6db8ebc640 1 --2- 172.21.15.97:0/1527621276 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557226e36620 0x557226e32430 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f6da809b860 tx=0x7f6da80bc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79a8603653e9784a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:38:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.804+0000 7f6d9e7fc640 1 -- 172.21.15.97:0/1527621276 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da801e040 con 0x557226e36620 2024-04-23T10:38:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.804+0000 7f6d9e7fc640 1 -- 172.21.15.97:0/1527621276 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6da8099c00 con 0x557226e36620 2024-04-23T10:38:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.804+0000 7f6d9e7fc640 1 -- 172.21.15.97:0/1527621276 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da8010d80 con 0x557226e36620 2024-04-23T10:38:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.804+0000 7f6dbde36740 1 -- 172.21.15.97:0/1527621276 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557226e36620 msgr2=0x557226e32430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:42.810 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.804+0000 7f6dbde36740 1 --2- 172.21.15.97:0/1527621276 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557226e36620 0x557226e32430 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f6da809b860 tx=0x7f6da80bc040 comp rx=0 tx=0).stop 2024-04-23T10:38:42.810 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.804+0000 7f6dbde36740 1 -- 172.21.15.97:0/1527621276 shutdown_connections 2024-04-23T10:38:42.810 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.804+0000 7f6dbde36740 1 --2- 172.21.15.97:0/1527621276 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557226e41800 0x557226e594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:42.810 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.804+0000 7f6dbde36740 1 --2- 172.21.15.97:0/1527621276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557226e36f40 0x557226e40cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:42.810 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.804+0000 7f6dbde36740 1 --2- 172.21.15.97:0/1527621276 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557226e36620 0x557226e32430 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:42.810 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.804+0000 7f6dbde36740 1 -- 172.21.15.97:0/1527621276 >> 172.21.15.97:0/1527621276 conn(0x557226e32b30 msgr2=0x557226e39af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:38:42.810 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.804+0000 7f6dbde36740 1 -- 172.21.15.97:0/1527621276 shutdown_connections 2024-04-23T10:38:42.810 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.804+0000 7f6dbde36740 1 -- 172.21.15.97:0/1527621276 wait complete. 2024-04-23T10:38:42.810 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.805+0000 7f6dbde36740 1 -- start start 2024-04-23T10:38:42.810 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.805+0000 7f6dbde36740 1 Processor -- start 2024-04-23T10:38:42.810 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.805+0000 7f6dbde36740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557226e36620 0x557226e49c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:42.811 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.805+0000 7f6dbde36740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557226e36f40 0x557226e45040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:42.811 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.805+0000 7f6dbde36740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557226e41800 0x557226e45650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:42.811 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.805+0000 7f6dbde36740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6db41b1030 con 0x557226e41800 2024-04-23T10:38:42.811 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.805+0000 7f6dbde36740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6db41b0960 con 0x557226e36620 2024-04-23T10:38:42.811 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.805+0000 7f6dbde36740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6db41adbc0 con 0x557226e36f40 2024-04-23T10:38:42.811 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.805+0000 7f6dbde36740 10 client.0 _ll_register_callbacks cb 0x7f6da0005d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:38:42.811 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.805+0000 7f6db96bd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557226e41800 0x557226e45650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:42.811 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.805+0000 7f6db96bd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557226e41800 0x557226e45650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53004/0 (socket says 172.21.15.97:53004) 2024-04-23T10:38:42.811 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.805+0000 7f6db96bd640 1 -- 172.21.15.97:0/332281273 learned_addr learned my addr 172.21.15.97:0/332281273 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:38:42.811 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.805+0000 7f6db8ebc640 1 --2- 172.21.15.97:0/332281273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557226e36620 0x557226e49c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:42.811 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.806+0000 7f6db96bd640 1 -- 172.21.15.97:0/332281273 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557226e36f40 msgr2=0x557226e45040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:38:42.811 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.806+0000 7f6db96bd640 1 --2- 172.21.15.97:0/332281273 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557226e36f40 0x557226e45040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:42.812 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.806+0000 7f6db96bd640 1 -- 172.21.15.97:0/332281273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557226e36620 msgr2=0x557226e49c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:38:42.812 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.806+0000 7f6db96bd640 1 --2- 172.21.15.97:0/332281273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557226e36620 0x557226e49c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:42.812 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.806+0000 7f6db96bd640 1 -- 172.21.15.97:0/332281273 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557226e45e80 con 0x557226e41800 2024-04-23T10:38:42.812 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.806+0000 7f6db8ebc640 1 --2- 172.21.15.97:0/332281273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557226e36620 0x557226e49c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:38:42.812 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.806+0000 7f6db96bd640 1 --2- 172.21.15.97:0/332281273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557226e41800 0x557226e45650 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x557226e33cc0 tx=0x7f6dac00c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77e93fd1051628a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:38:42.812 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.806+0000 7f6d93fff640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:38:42.812 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.806+0000 7f6d93fff640 1 -- 172.21.15.97:0/332281273 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dac003970 con 0x557226e41800 2024-04-23T10:38:42.812 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.806+0000 7f6d93fff640 1 -- 172.21.15.97:0/332281273 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8484+0+0 (secure 0 0 0) 0x7f6dac003b10 con 0x557226e41800 2024-04-23T10:38:42.812 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.806+0000 7f6dbde36740 1 -- 172.21.15.97:0/332281273 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6da8099c00 con 0x557226e41800 2024-04-23T10:38:42.812 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.806+0000 7f6d93fff640 1 -- 172.21.15.97:0/332281273 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6dac003d10 con 0x557226e41800 2024-04-23T10:38:42.812 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.806+0000 7f6dbde36740 10 client.17361 fetch_fsmap learned FSMap version 26 2024-04-23T10:38:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.806+0000 7f6dbde36740 1 -- 172.21.15.97:0/332281273 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557226eb7e50 con 0x557226e41800 2024-04-23T10:38:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.806+0000 7f6d93fff640 1 -- 172.21.15.97:0/332281273 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6da8099c00 con 0x557226e41800 2024-04-23T10:38:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.806+0000 7f6d93fff640 1 -- 172.21.15.97:0/332281273 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f6dac02e070 con 0x557226e41800 2024-04-23T10:38:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.806+0000 7f6dbde36740 10 client.17361 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:38:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.806+0000 7f6dbde36740 10 client.17361 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:38:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.806+0000 7f6dbde36740 20 client.17361 populate_metadata read hostname 'smithi097' 2024-04-23T10:38:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.806+0000 7f6dbde36740 1 --2- 172.21.15.97:0/332281273 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x557226eb8450 0x557226eba870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.806+0000 7f6dbde36740 4 client.17361 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:38:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.806+0000 7f6dbde36740 1 -- 172.21.15.97:0/332281273 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557226da3200 con 0x557226eb8450 2024-04-23T10:38:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.807+0000 7f6db8ebc640 1 --2- 172.21.15.97:0/332281273 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x557226eb8450 0x557226eba870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.807+0000 7f6d93fff640 10 client.17361 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:38:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.807+0000 7f6db8ebc640 1 --2- 172.21.15.97:0/332281273 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x557226eb8450 0x557226eba870 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34034fee3adcdf18 server_cookie=eb708ab6a5894d2 in_seq=0 out_seq=0 2024-04-23T10:38:42.817 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.817+0000 7f6d93fff640 1 -- 172.21.15.97:0/332281273 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557226da3200 con 0x557226eb8450 2024-04-23T10:38:42.817 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.817+0000 7f6d93fff640 10 client.17361 handle_command_reply: tid=0 2024-04-23T10:38:42.817 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.817+0000 7f6dbde36740 1 client.17361 shutdown 2024-04-23T10:38:42.817 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.818+0000 7f6dbde36740 1 -- 172.21.15.97:0/332281273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557226e41800 msgr2=0x557226e45650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:42.817 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.818+0000 7f6dbde36740 1 --2- 172.21.15.97:0/332281273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557226e41800 0x557226e45650 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x557226e33cc0 tx=0x7f6dac00c9d0 comp rx=0 tx=0).stop 2024-04-23T10:38:42.817 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.818+0000 7f6dbde36740 1 -- 172.21.15.97:0/332281273 shutdown_connections 2024-04-23T10:38:42.818 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.818+0000 7f6dbde36740 1 --2- 172.21.15.97:0/332281273 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x557226eb8450 0x557226eba870 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:42.818 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.818+0000 7f6dbde36740 1 --2- 172.21.15.97:0/332281273 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557226e41800 0x557226e45650 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:42.818 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.818+0000 7f6dbde36740 1 --2- 172.21.15.97:0/332281273 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557226e36f40 0x557226e45040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:42.818 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.818+0000 7f6d93fff640 0 client.17361 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:38:42.818 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.818+0000 7f6dbde36740 1 --2- 172.21.15.97:0/332281273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557226e36620 0x557226e49c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:42.818 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.818+0000 7f6dbde36740 1 -- 172.21.15.97:0/332281273 >> 172.21.15.97:0/332281273 conn(0x557226e32b30 msgr2=0x557226e31420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:38:42.818 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.818+0000 7f6dbde36740 1 -- 172.21.15.97:0/332281273 shutdown_connections 2024-04-23T10:38:42.818 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.818+0000 7f6dbde36740 1 -- 172.21.15.97:0/332281273 wait complete. 2024-04-23T10:38:42.818 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.818+0000 7f6dbde36740 20 client.17361 trim_cache size 0 max 16384 2024-04-23T10:38:42.819 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:38:42.819 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:38:42.819 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:38:42.819 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:38:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.818+0000 7f6dbb120640 1 -- 172.21.15.97:0/3355686871 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f6da40a3df0 msgr2=0x7f6da40a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.818+0000 7f6dbb120640 1 --2- 172.21.15.97:0/3355686871 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f6da40a3df0 0x7f6da40a62b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6dac012720 tx=0x7f6dac012b20 comp rx=0 tx=0).stop 2024-04-23T10:38:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.818+0000 7f6dbb120640 1 -- 172.21.15.97:0/3355686871 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6db4103200 msgr2=0x7f6db4100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.818+0000 7f6dbb120640 1 --2- 172.21.15.97:0/3355686871 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6db4103200 0x7f6db4100e50 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f6da800b4a0 tx=0x7f6da800b960 comp rx=0 tx=0).stop 2024-04-23T10:38:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.818+0000 7f6dbb120640 1 -- 172.21.15.97:0/3355686871 shutdown_connections 2024-04-23T10:38:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.818+0000 7f6dbb120640 1 --2- 172.21.15.97:0/3355686871 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f6da40a3df0 0x7f6da40a62b0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.818+0000 7f6dbb120640 1 --2- 172.21.15.97:0/3355686871 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6db410c5d0 0x7f6db41018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.818+0000 7f6dbb120640 1 --2- 172.21.15.97:0/3355686871 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6db4103b20 0x7f6db4101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:42.821 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.818+0000 7f6dbb120640 1 --2- 172.21.15.97:0/3355686871 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6db4103200 0x7f6db4100e50 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:42.821 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.818+0000 7f6dbb120640 1 -- 172.21.15.97:0/3355686871 >> 172.21.15.97:0/3355686871 conn(0x7f6db40771d0 msgr2=0x7f6db40fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:38:42.821 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.819+0000 7f6dbb120640 1 -- 172.21.15.97:0/3355686871 shutdown_connections 2024-04-23T10:38:42.821 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:42.819+0000 7f6dbb120640 1 -- 172.21.15.97:0/3355686871 wait complete. 2024-04-23T10:38:42.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:38:42 smithi183 ceph-mon[32317]: pgmap v390: 97 pgs: 97 active+clean; 1.5 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-04-23T10:38:43.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:38:42 smithi196 ceph-mon[32919]: pgmap v390: 97 pgs: 97 active+clean; 1.5 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-04-23T10:38:43.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:38:42 smithi097 ceph-mon[29197]: pgmap v390: 97 pgs: 97 active+clean; 1.5 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-04-23T10:38:44.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:38:44 smithi183 ceph-mon[32317]: pgmap v391: 97 pgs: 97 active+clean; 1.5 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-04-23T10:38:45.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:38:44 smithi196 ceph-mon[32919]: pgmap v391: 97 pgs: 97 active+clean; 1.5 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-04-23T10:38:45.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:38:44 smithi097 ceph-mon[29197]: pgmap v391: 97 pgs: 97 active+clean; 1.5 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-04-23T10:38:46.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:38:46 smithi183 ceph-mon[32317]: pgmap v392: 97 pgs: 97 active+clean; 1.5 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T10:38:47.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:38:46 smithi196 ceph-mon[32919]: pgmap v392: 97 pgs: 97 active+clean; 1.5 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T10:38:47.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:38:46 smithi097 ceph-mon[29197]: pgmap v392: 97 pgs: 97 active+clean; 1.5 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T10:38:48.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:38:48 smithi183 ceph-mon[32317]: pgmap v393: 97 pgs: 97 active+clean; 1.5 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.6 KiB/s wr, 14 op/s 2024-04-23T10:38:49.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:38:48 smithi196 ceph-mon[32919]: pgmap v393: 97 pgs: 97 active+clean; 1.5 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.6 KiB/s wr, 14 op/s 2024-04-23T10:38:49.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:38:48 smithi097 ceph-mon[29197]: pgmap v393: 97 pgs: 97 active+clean; 1.5 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.6 KiB/s wr, 14 op/s 2024-04-23T10:38:50.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:38:50 smithi183 ceph-mon[32317]: pgmap v394: 97 pgs: 97 active+clean; 1.5 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.6 KiB/s wr, 10 op/s 2024-04-23T10:38:51.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:38:50 smithi196 ceph-mon[32919]: pgmap v394: 97 pgs: 97 active+clean; 1.5 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.6 KiB/s wr, 10 op/s 2024-04-23T10:38:51.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:38:50 smithi097 ceph-mon[29197]: pgmap v394: 97 pgs: 97 active+clean; 1.5 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.6 KiB/s wr, 10 op/s 2024-04-23T10:38:52.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:38:52 smithi183 ceph-mon[32317]: pgmap v395: 97 pgs: 97 active+clean; 1.5 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.6 KiB/s wr, 10 op/s 2024-04-23T10:38:53.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:38:52 smithi196 ceph-mon[32919]: pgmap v395: 97 pgs: 97 active+clean; 1.5 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.6 KiB/s wr, 10 op/s 2024-04-23T10:38:53.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:38:52 smithi097 ceph-mon[29197]: pgmap v395: 97 pgs: 97 active+clean; 1.5 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.6 KiB/s wr, 10 op/s 2024-04-23T10:38:55.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:38:54 smithi196 ceph-mon[32919]: pgmap v396: 97 pgs: 97 active+clean; 1.5 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.6 KiB/s wr, 14 op/s 2024-04-23T10:38:55.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:38:54 smithi097 ceph-mon[29197]: pgmap v396: 97 pgs: 97 active+clean; 1.5 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.6 KiB/s wr, 14 op/s 2024-04-23T10:38:55.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:38:54 smithi183 ceph-mon[32317]: pgmap v396: 97 pgs: 97 active+clean; 1.5 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.6 KiB/s wr, 14 op/s 2024-04-23T10:38:55.916 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.914+0000 7fe82f73d640 1 -- 172.21.15.97:0/1485639413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8281021d0 msgr2=0x7fe8281025b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:55.916 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.914+0000 7fe82f73d640 1 --2- 172.21.15.97:0/1485639413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8281021d0 0x7fe8281025b0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fe824009900 tx=0x7fe82402a410 comp rx=0 tx=0).stop 2024-04-23T10:38:55.917 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.915+0000 7fe82f73d640 1 -- 172.21.15.97:0/1485639413 shutdown_connections 2024-04-23T10:38:55.917 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.915+0000 7fe82f73d640 1 --2- 172.21.15.97:0/1485639413 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8281072a0 0x7fe82810ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:55.917 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.915+0000 7fe82f73d640 1 --2- 172.21.15.97:0/1485639413 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe828102b80 0x7fe828106bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:55.917 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.915+0000 7fe82f73d640 1 --2- 172.21.15.97:0/1485639413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8281021d0 0x7fe8281025b0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:55.917 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.915+0000 7fe82f73d640 1 -- 172.21.15.97:0/1485639413 >> 172.21.15.97:0/1485639413 conn(0x7fe8280793e0 msgr2=0x7fe8280797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:38:55.917 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.915+0000 7fe82f73d640 1 -- 172.21.15.97:0/1485639413 shutdown_connections 2024-04-23T10:38:55.917 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.915+0000 7fe82f73d640 1 -- 172.21.15.97:0/1485639413 wait complete. 2024-04-23T10:38:55.917 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.915+0000 7fe82f73d640 1 Processor -- start 2024-04-23T10:38:55.917 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.915+0000 7fe82f73d640 1 -- start start 2024-04-23T10:38:55.917 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.915+0000 7fe82f73d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8281021d0 0x7fe8281986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:55.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.915+0000 7fe82f73d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe8281072a0 0x7fe828198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:55.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.915+0000 7fe82f73d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe82819cf60 0x7fe82819d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:55.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.915+0000 7fe82f73d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe8281121e0 con 0x7fe82819cf60 2024-04-23T10:38:55.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.915+0000 7fe82f73d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe828112060 con 0x7fe8281072a0 2024-04-23T10:38:55.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.915+0000 7fe82f73d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe828112360 con 0x7fe8281021d0 2024-04-23T10:38:55.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.916+0000 7fe82dcda640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe82819cf60 0x7fe82819d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:55.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.916+0000 7fe82dcda640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe82819cf60 0x7fe82819d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37772/0 (socket says 172.21.15.97:37772) 2024-04-23T10:38:55.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.916+0000 7fe82dcda640 1 -- 172.21.15.97:0/3799868848 learned_addr learned my addr 172.21.15.97:0/3799868848 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:38:55.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.916+0000 7fe82d4d9640 1 --2- 172.21.15.97:0/3799868848 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8281021d0 0x7fe8281986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:55.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.916+0000 7fe82ccd8640 1 --2- 172.21.15.97:0/3799868848 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe8281072a0 0x7fe828198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:55.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.916+0000 7fe82dcda640 1 -- 172.21.15.97:0/3799868848 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8281021d0 msgr2=0x7fe8281986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:55.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.916+0000 7fe82dcda640 1 --2- 172.21.15.97:0/3799868848 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8281021d0 0x7fe8281986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:55.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.916+0000 7fe82dcda640 1 -- 172.21.15.97:0/3799868848 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe8281072a0 msgr2=0x7fe828198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:55.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.916+0000 7fe82dcda640 1 --2- 172.21.15.97:0/3799868848 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe8281072a0 0x7fe828198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:55.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.916+0000 7fe82dcda640 1 -- 172.21.15.97:0/3799868848 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe82819da50 con 0x7fe82819cf60 2024-04-23T10:38:55.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.916+0000 7fe82dcda640 1 --2- 172.21.15.97:0/3799868848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe82819cf60 0x7fe82819d410 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fe82000da40 tx=0x7fe82000df00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:38:55.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.916+0000 7fe81e7fc640 1 -- 172.21.15.97:0/3799868848 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe820017070 con 0x7fe82819cf60 2024-04-23T10:38:55.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.916+0000 7fe81e7fc640 1 -- 172.21.15.97:0/3799868848 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe820003e20 con 0x7fe82819cf60 2024-04-23T10:38:55.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.916+0000 7fe82f73d640 1 -- 172.21.15.97:0/3799868848 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe82819dd40 con 0x7fe82819cf60 2024-04-23T10:38:55.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.916+0000 7fe81e7fc640 1 -- 172.21.15.97:0/3799868848 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe820010460 con 0x7fe82819cf60 2024-04-23T10:38:55.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.916+0000 7fe82f73d640 1 -- 172.21.15.97:0/3799868848 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8281a54e0 con 0x7fe82819cf60 2024-04-23T10:38:55.927 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.919+0000 7fe81e7fc640 1 -- 172.21.15.97:0/3799868848 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe820004020 con 0x7fe82819cf60 2024-04-23T10:38:55.927 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.919+0000 7fe83244b740 1 Processor -- start 2024-04-23T10:38:55.927 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.919+0000 7fe83244b740 1 -- start start 2024-04-23T10:38:55.927 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.919+0000 7fe83244b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595d4587540 0x5595d4587920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:55.927 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.919+0000 7fe83244b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5595d4587e60 0x5595d45966d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:55.927 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.919+0000 7fe81e7fc640 1 --2- 172.21.15.97:0/3799868848 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fe8140a3dc0 0x7fe8140a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:55.927 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.919+0000 7fe83244b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595d4596c10 0x5595d4599000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:55.927 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.919+0000 7fe83244b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5595d44ec970 con 0x5595d4596c10 2024-04-23T10:38:55.928 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.919+0000 7fe83244b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5595d44894a0 con 0x5595d4587e60 2024-04-23T10:38:55.928 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.919+0000 7fe83244b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5595d44e8930 con 0x5595d4587540 2024-04-23T10:38:55.928 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.920+0000 7fe82dcda640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595d4596c10 0x5595d4599000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:55.928 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.920+0000 7fe82ccd8640 1 --2- 172.21.15.97:0/3799868848 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fe8140a3dc0 0x7fe8140a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:55.928 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.920+0000 7fe82dcda640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595d4596c10 0x5595d4599000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37776/0 (socket says 172.21.15.97:37776) 2024-04-23T10:38:55.928 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.920+0000 7fe81e7fc640 1 -- 172.21.15.97:0/3799868848 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8484+0+0 (secure 0 0 0) 0x7fe8200d0380 con 0x7fe82819cf60 2024-04-23T10:38:55.928 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.920+0000 7fe82dcda640 1 -- 172.21.15.97:0/2802809565 learned_addr learned my addr 172.21.15.97:0/2802809565 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:38:55.928 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.920+0000 7fe82ccd8640 1 --2- 172.21.15.97:0/2802809565 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595d4587540 0x5595d4587920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:55.928 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.920+0000 7fe82d4d9640 1 --2- 172.21.15.97:0/2802809565 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5595d4587e60 0x5595d45966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:55.928 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.920+0000 7fe82dcda640 1 -- 172.21.15.97:0/2802809565 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595d4587540 msgr2=0x5595d4587920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:38:55.928 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.920+0000 7fe82dcda640 1 --2- 172.21.15.97:0/2802809565 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595d4587540 0x5595d4587920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:55.928 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.920+0000 7fe82dcda640 1 -- 172.21.15.97:0/2802809565 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5595d4587e60 msgr2=0x5595d45966d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:38:55.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.920+0000 7fe82dcda640 1 --2- 172.21.15.97:0/2802809565 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5595d4587e60 0x5595d45966d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:55.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.920+0000 7fe82dcda640 1 -- 172.21.15.97:0/2802809565 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595d41ff210 con 0x5595d4596c10 2024-04-23T10:38:55.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.920+0000 7fe82ccd8640 1 --2- 172.21.15.97:0/3799868848 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fe8140a3dc0 0x7fe8140a6280 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe828101820 tx=0x7fe81800bad0 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:38:55.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.920+0000 7fe82dcda640 1 --2- 172.21.15.97:0/2802809565 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595d4596c10 0x5595d4599000 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fe820005c40 tx=0x7fe82001e790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acbc280536e0ded1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:38:55.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.920+0000 7fe8137fe640 1 -- 172.21.15.97:0/2802809565 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe820020070 con 0x5595d4596c10 2024-04-23T10:38:55.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.920+0000 7fe8137fe640 1 -- 172.21.15.97:0/2802809565 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe82009c3f0 con 0x5595d4596c10 2024-04-23T10:38:55.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.920+0000 7fe8137fe640 1 -- 172.21.15.97:0/2802809565 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8200a4d60 con 0x5595d4596c10 2024-04-23T10:38:55.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.921+0000 7fe83244b740 1 -- 172.21.15.97:0/2802809565 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595d4596c10 msgr2=0x5595d4599000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:55.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.921+0000 7fe83244b740 1 --2- 172.21.15.97:0/2802809565 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595d4596c10 0x5595d4599000 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fe820005c40 tx=0x7fe82001e790 comp rx=0 tx=0).stop 2024-04-23T10:38:55.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.921+0000 7fe83244b740 1 -- 172.21.15.97:0/2802809565 shutdown_connections 2024-04-23T10:38:55.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.921+0000 7fe83244b740 1 --2- 172.21.15.97:0/2802809565 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595d4596c10 0x5595d4599000 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:55.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.921+0000 7fe83244b740 1 --2- 172.21.15.97:0/2802809565 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5595d4587e60 0x5595d45966d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:55.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.921+0000 7fe83244b740 1 --2- 172.21.15.97:0/2802809565 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595d4587540 0x5595d4587920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:55.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.921+0000 7fe83244b740 1 -- 172.21.15.97:0/2802809565 >> 172.21.15.97:0/2802809565 conn(0x5595d4583b30 msgr2=0x5595d4583f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:38:55.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.921+0000 7fe83244b740 1 -- 172.21.15.97:0/2802809565 shutdown_connections 2024-04-23T10:38:55.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.921+0000 7fe83244b740 1 -- 172.21.15.97:0/2802809565 wait complete. 2024-04-23T10:38:55.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.921+0000 7fe83244b740 1 -- start start 2024-04-23T10:38:55.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.922+0000 7fe83244b740 1 Processor -- start 2024-04-23T10:38:55.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.922+0000 7fe83244b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595d4587540 0x5595d45965e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:55.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.922+0000 7fe83244b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595d4587e60 0x5595d45938d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:55.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.922+0000 7fe83244b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5595d4596c10 0x5595d4593ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:55.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.922+0000 7fe83244b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5595d4599840 con 0x5595d4587540 2024-04-23T10:38:55.931 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.922+0000 7fe83244b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5595d44870b0 con 0x5595d4596c10 2024-04-23T10:38:55.931 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.922+0000 7fe83244b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5595d45999c0 con 0x5595d4587e60 2024-04-23T10:38:55.931 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.922+0000 7fe83244b740 10 client.0 _ll_register_callbacks cb 0x5595d4474d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:38:55.931 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.922+0000 7fe82d4d9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595d4587540 0x5595d45965e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:55.931 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.922+0000 7fe82d4d9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595d4587540 0x5595d45965e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37784/0 (socket says 172.21.15.97:37784) 2024-04-23T10:38:55.931 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.922+0000 7fe82d4d9640 1 -- 172.21.15.97:0/3738447594 learned_addr learned my addr 172.21.15.97:0/3738447594 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:38:55.931 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.922+0000 7fe82dcda640 1 --2- 172.21.15.97:0/3738447594 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5595d4596c10 0x5595d4593ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:55.931 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.922+0000 7fe82ccd8640 1 --2- 172.21.15.97:0/3738447594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595d4587e60 0x5595d45938d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:55.931 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.922+0000 7fe82d4d9640 1 -- 172.21.15.97:0/3738447594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595d4587e60 msgr2=0x5595d45938d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:38:55.931 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.922+0000 7fe82d4d9640 1 --2- 172.21.15.97:0/3738447594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595d4587e60 0x5595d45938d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:55.931 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.923+0000 7fe82d4d9640 1 -- 172.21.15.97:0/3738447594 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5595d4596c10 msgr2=0x5595d4593ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:38:55.931 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.923+0000 7fe82d4d9640 1 --2- 172.21.15.97:0/3738447594 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5595d4596c10 0x5595d4593ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:55.932 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.923+0000 7fe82d4d9640 1 -- 172.21.15.97:0/3738447594 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595d42cb620 con 0x5595d4587540 2024-04-23T10:38:55.932 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.923+0000 7fe82d4d9640 1 --2- 172.21.15.97:0/3738447594 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595d4587540 0x5595d45965e0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fe824009900 tx=0x7fe8240314a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb4ff283ae38c58c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:38:55.932 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.923+0000 7fe810ff9640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:38:55.932 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.923+0000 7fe810ff9640 1 -- 172.21.15.97:0/3738447594 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe824035040 con 0x5595d4587540 2024-04-23T10:38:55.932 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.923+0000 7fe810ff9640 1 -- 172.21.15.97:0/3738447594 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8484+0+0 (secure 0 0 0) 0x7fe824004040 con 0x5595d4587540 2024-04-23T10:38:55.932 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.923+0000 7fe83244b740 1 -- 172.21.15.97:0/3738447594 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe82009c3f0 con 0x5595d4587540 2024-04-23T10:38:55.932 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.923+0000 7fe810ff9640 1 -- 172.21.15.97:0/3738447594 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe8240318d0 con 0x5595d4587540 2024-04-23T10:38:55.932 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.923+0000 7fe83244b740 10 client.17373 fetch_fsmap learned FSMap version 26 2024-04-23T10:38:55.932 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.923+0000 7fe83244b740 1 -- 172.21.15.97:0/3738447594 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5595d45a5820 con 0x5595d4587540 2024-04-23T10:38:55.932 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.923+0000 7fe810ff9640 1 -- 172.21.15.97:0/3738447594 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe82009c3f0 con 0x5595d4587540 2024-04-23T10:38:55.932 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.923+0000 7fe810ff9640 1 -- 172.21.15.97:0/3738447594 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe82404d020 con 0x5595d4587540 2024-04-23T10:38:55.932 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.923+0000 7fe83244b740 10 client.17373 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:38:55.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.923+0000 7fe83244b740 10 client.17373 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:38:55.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.923+0000 7fe83244b740 20 client.17373 populate_metadata read hostname 'smithi097' 2024-04-23T10:38:55.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.924+0000 7fe83244b740 1 --2- 172.21.15.97:0/3738447594 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5595d45a6ce0 0x5595d45a90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:55.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.924+0000 7fe83244b740 4 client.17373 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:38:55.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.924+0000 7fe83244b740 1 -- 172.21.15.97:0/3738447594 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5595d44fa390 con 0x5595d45a6ce0 2024-04-23T10:38:55.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.924+0000 7fe82ccd8640 1 --2- 172.21.15.97:0/3738447594 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5595d45a6ce0 0x5595d45a90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:55.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.924+0000 7fe810ff9640 10 client.17373 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:38:55.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.924+0000 7fe82ccd8640 1 --2- 172.21.15.97:0/3738447594 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5595d45a6ce0 0x5595d45a90d0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe6f47cdfded02ab server_cookie=710c4dd6f965415a in_seq=0 out_seq=0 2024-04-23T10:38:55.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.925+0000 7fe810ff9640 1 -- 172.21.15.97:0/3738447594 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5595d44fa390 con 0x5595d45a6ce0 2024-04-23T10:38:55.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.925+0000 7fe810ff9640 10 client.17373 handle_command_reply: tid=0 2024-04-23T10:38:55.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.925+0000 7fe83244b740 1 client.17373 shutdown 2024-04-23T10:38:55.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.925+0000 7fe83244b740 1 -- 172.21.15.97:0/3738447594 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595d4587540 msgr2=0x5595d45965e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:55.934 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.925+0000 7fe83244b740 1 --2- 172.21.15.97:0/3738447594 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595d4587540 0x5595d45965e0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fe824009900 tx=0x7fe8240314a0 comp rx=0 tx=0).stop 2024-04-23T10:38:55.934 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.925+0000 7fe83244b740 1 -- 172.21.15.97:0/3738447594 shutdown_connections 2024-04-23T10:38:55.934 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.925+0000 7fe83244b740 1 --2- 172.21.15.97:0/3738447594 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5595d45a6ce0 0x5595d45a90d0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:55.934 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.925+0000 7fe83244b740 1 --2- 172.21.15.97:0/3738447594 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5595d4596c10 0x5595d4593ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:55.934 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.925+0000 7fe810ff9640 0 client.17373 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:38:55.934 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.925+0000 7fe83244b740 1 --2- 172.21.15.97:0/3738447594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595d4587e60 0x5595d45938d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:55.934 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.925+0000 7fe83244b740 1 --2- 172.21.15.97:0/3738447594 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595d4587540 0x5595d45965e0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:55.934 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.925+0000 7fe83244b740 1 -- 172.21.15.97:0/3738447594 >> 172.21.15.97:0/3738447594 conn(0x5595d4583b30 msgr2=0x5595d45899a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:38:55.934 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.925+0000 7fe83244b740 1 -- 172.21.15.97:0/3738447594 shutdown_connections 2024-04-23T10:38:55.934 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.925+0000 7fe83244b740 1 -- 172.21.15.97:0/3738447594 wait complete. 2024-04-23T10:38:55.934 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.926+0000 7fe83244b740 20 client.17373 trim_cache size 0 max 16384 2024-04-23T10:38:55.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.940+0000 7fe83244b740 1 Processor -- start 2024-04-23T10:38:55.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.941+0000 7fe83244b740 1 -- start start 2024-04-23T10:38:55.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.941+0000 7fe83244b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595d4587540 0x5595d4582b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:55.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.941+0000 7fe83244b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5595d4587e60 0x5595d4596110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:55.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.941+0000 7fe83244b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595d4596c10 0x5595d45ae9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:55.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.941+0000 7fe83244b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe8281ab860 con 0x5595d4587540 2024-04-23T10:38:55.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.941+0000 7fe83244b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe8281ad3a0 con 0x5595d4587e60 2024-04-23T10:38:55.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.941+0000 7fe83244b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe8281abf30 con 0x5595d4596c10 2024-04-23T10:38:55.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.941+0000 7fe82d4d9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595d4587540 0x5595d4582b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:55.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.941+0000 7fe82d4d9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595d4587540 0x5595d4582b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37796/0 (socket says 172.21.15.97:37796) 2024-04-23T10:38:55.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.941+0000 7fe82d4d9640 1 -- 172.21.15.97:0/3664364965 learned_addr learned my addr 172.21.15.97:0/3664364965 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:38:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.941+0000 7fe82ccd8640 1 --2- 172.21.15.97:0/3664364965 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5595d4587e60 0x5595d4596110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.941+0000 7fe82dcda640 1 --2- 172.21.15.97:0/3664364965 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595d4596c10 0x5595d45ae9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.941+0000 7fe82d4d9640 1 -- 172.21.15.97:0/3664364965 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595d4596c10 msgr2=0x5595d45ae9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:38:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.941+0000 7fe82d4d9640 1 --2- 172.21.15.97:0/3664364965 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595d4596c10 0x5595d45ae9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.941+0000 7fe82d4d9640 1 -- 172.21.15.97:0/3664364965 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5595d4587e60 msgr2=0x5595d4596110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:38:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.941+0000 7fe82d4d9640 1 --2- 172.21.15.97:0/3664364965 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5595d4587e60 0x5595d4596110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.941+0000 7fe82d4d9640 1 -- 172.21.15.97:0/3664364965 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595d45944e0 con 0x5595d4587540 2024-04-23T10:38:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.941+0000 7fe82ccd8640 1 --2- 172.21.15.97:0/3664364965 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5595d4587e60 0x5595d4596110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:38:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.941+0000 7fe82dcda640 1 --2- 172.21.15.97:0/3664364965 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595d4596c10 0x5595d45ae9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:38:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.941+0000 7fe82d4d9640 1 --2- 172.21.15.97:0/3664364965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595d4587540 0x5595d4582b30 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fe82402a930 tx=0x7fe824000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1f973c5785c3aed server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:38:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.941+0000 7fe812ffd640 1 -- 172.21.15.97:0/3664364965 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe824035070 con 0x5595d4587540 2024-04-23T10:38:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.941+0000 7fe812ffd640 1 -- 172.21.15.97:0/3664364965 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe824054070 con 0x5595d4587540 2024-04-23T10:38:55.947 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.941+0000 7fe812ffd640 1 -- 172.21.15.97:0/3664364965 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe824005510 con 0x5595d4587540 2024-04-23T10:38:55.947 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.942+0000 7fe83244b740 1 -- 172.21.15.97:0/3664364965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595d4587540 msgr2=0x5595d4582b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:55.947 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.942+0000 7fe83244b740 1 --2- 172.21.15.97:0/3664364965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595d4587540 0x5595d4582b30 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fe82402a930 tx=0x7fe824000bc0 comp rx=0 tx=0).stop 2024-04-23T10:38:55.947 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.942+0000 7fe83244b740 1 -- 172.21.15.97:0/3664364965 shutdown_connections 2024-04-23T10:38:55.947 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.942+0000 7fe83244b740 1 --2- 172.21.15.97:0/3664364965 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595d4596c10 0x5595d45ae9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:55.947 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.942+0000 7fe83244b740 1 --2- 172.21.15.97:0/3664364965 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5595d4587e60 0x5595d4596110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:55.947 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.942+0000 7fe83244b740 1 --2- 172.21.15.97:0/3664364965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595d4587540 0x5595d4582b30 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:55.947 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.942+0000 7fe83244b740 1 -- 172.21.15.97:0/3664364965 >> 172.21.15.97:0/3664364965 conn(0x5595d4583b30 msgr2=0x5595d4582420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:38:55.947 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.942+0000 7fe83244b740 1 -- 172.21.15.97:0/3664364965 shutdown_connections 2024-04-23T10:38:55.947 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.942+0000 7fe83244b740 1 -- 172.21.15.97:0/3664364965 wait complete. 2024-04-23T10:38:55.947 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.942+0000 7fe83244b740 1 -- start start 2024-04-23T10:38:55.947 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.942+0000 7fe83244b740 1 Processor -- start 2024-04-23T10:38:55.948 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.943+0000 7fe83244b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5595d4587540 0x5595d4590cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:55.948 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.943+0000 7fe83244b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595d4587e60 0x5595d4591230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:55.948 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.943+0000 7fe83244b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595d4596c10 0x5595d460c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:55.948 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.943+0000 7fe83244b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe8281accd0 con 0x5595d4596c10 2024-04-23T10:38:55.948 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.943+0000 7fe83244b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe8281ac600 con 0x5595d4587540 2024-04-23T10:38:55.948 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.943+0000 7fe83244b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fe8281a9860 con 0x5595d4587e60 2024-04-23T10:38:55.948 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.943+0000 7fe83244b740 10 client.0 _ll_register_callbacks cb 0x7fe824032c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:38:55.948 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.943+0000 7fe82dcda640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595d4596c10 0x5595d460c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:55.948 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.943+0000 7fe82ccd8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595d4587e60 0x5595d4591230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:55.948 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.943+0000 7fe82dcda640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595d4596c10 0x5595d460c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37812/0 (socket says 172.21.15.97:37812) 2024-04-23T10:38:55.948 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.943+0000 7fe82dcda640 1 -- 172.21.15.97:0/2751960772 learned_addr learned my addr 172.21.15.97:0/2751960772 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:38:55.948 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.943+0000 7fe82d4d9640 1 --2- 172.21.15.97:0/2751960772 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5595d4587540 0x5595d4590cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:55.949 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.943+0000 7fe82dcda640 1 -- 172.21.15.97:0/2751960772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595d4587e60 msgr2=0x5595d4591230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:38:55.949 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.943+0000 7fe82dcda640 1 --2- 172.21.15.97:0/2751960772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595d4587e60 0x5595d4591230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:55.949 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.943+0000 7fe82dcda640 1 -- 172.21.15.97:0/2751960772 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5595d4587540 msgr2=0x5595d4590cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:38:55.949 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.943+0000 7fe82dcda640 1 --2- 172.21.15.97:0/2751960772 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5595d4587540 0x5595d4590cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:55.949 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.943+0000 7fe82dcda640 1 -- 172.21.15.97:0/2751960772 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595d460c790 con 0x5595d4596c10 2024-04-23T10:38:55.949 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.943+0000 7fe82ccd8640 1 --2- 172.21.15.97:0/2751960772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595d4587e60 0x5595d4591230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T10:38:55.949 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.943+0000 7fe82d4d9640 1 --2- 172.21.15.97:0/2751960772 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5595d4587540 0x5595d4590cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:38:55.949 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.943+0000 7fe82dcda640 1 --2- 172.21.15.97:0/2751960772 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595d4596c10 0x5595d460c250 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fe820098dc0 tx=0x7fe8200ce000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c79b93bc4fa00ad server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:38:55.949 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.943+0000 7fe810ff9640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:38:55.949 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.944+0000 7fe810ff9640 1 -- 172.21.15.97:0/2751960772 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe820020040 con 0x5595d4596c10 2024-04-23T10:38:55.949 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.944+0000 7fe810ff9640 1 -- 172.21.15.97:0/2751960772 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8484+0+0 (secure 0 0 0) 0x7fe8200a4d20 con 0x5595d4596c10 2024-04-23T10:38:55.949 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.944+0000 7fe83244b740 1 -- 172.21.15.97:0/2751960772 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe824054070 con 0x5595d4596c10 2024-04-23T10:38:55.950 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.944+0000 7fe810ff9640 1 -- 172.21.15.97:0/2751960772 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe8200b5050 con 0x5595d4596c10 2024-04-23T10:38:55.950 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.944+0000 7fe83244b740 10 client.17379 fetch_fsmap learned FSMap version 26 2024-04-23T10:38:55.950 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.944+0000 7fe83244b740 1 -- 172.21.15.97:0/2751960772 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5595d460da10 con 0x5595d4596c10 2024-04-23T10:38:55.950 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.944+0000 7fe810ff9640 1 -- 172.21.15.97:0/2751960772 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe824054070 con 0x5595d4596c10 2024-04-23T10:38:55.950 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.944+0000 7fe810ff9640 1 -- 172.21.15.97:0/2751960772 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe820003970 con 0x5595d4596c10 2024-04-23T10:38:55.950 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.944+0000 7fe83244b740 10 client.17379 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:38:55.950 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.944+0000 7fe83244b740 10 client.17379 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:38:55.950 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.944+0000 7fe83244b740 20 client.17379 populate_metadata read hostname 'smithi097' 2024-04-23T10:38:55.950 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.944+0000 7fe83244b740 1 --2- 172.21.15.97:0/2751960772 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5595d45a6ce0 0x5595d460fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:55.950 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.944+0000 7fe83244b740 4 client.17379 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:38:55.950 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.944+0000 7fe83244b740 1 -- 172.21.15.97:0/2751960772 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5595d44f4200 con 0x5595d45a6ce0 2024-04-23T10:38:55.950 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.944+0000 7fe82d4d9640 1 --2- 172.21.15.97:0/2751960772 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5595d45a6ce0 0x5595d460fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:55.951 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.945+0000 7fe810ff9640 10 client.17379 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:38:55.951 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.945+0000 7fe82d4d9640 1 --2- 172.21.15.97:0/2751960772 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5595d45a6ce0 0x5595d460fca0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=584ec609561d7c6d server_cookie=e3a211f9bc0f9a40 in_seq=0 out_seq=0 2024-04-23T10:38:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.953+0000 7fe810ff9640 1 -- 172.21.15.97:0/2751960772 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5595d44f4200 con 0x5595d45a6ce0 2024-04-23T10:38:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.953+0000 7fe810ff9640 10 client.17379 handle_command_reply: tid=0 2024-04-23T10:38:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.953+0000 7fe83244b740 1 client.17379 shutdown 2024-04-23T10:38:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.953+0000 7fe83244b740 1 -- 172.21.15.97:0/2751960772 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595d4596c10 msgr2=0x5595d460c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.953+0000 7fe83244b740 1 --2- 172.21.15.97:0/2751960772 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595d4596c10 0x5595d460c250 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fe820098dc0 tx=0x7fe8200ce000 comp rx=0 tx=0).stop 2024-04-23T10:38:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.953+0000 7fe83244b740 1 -- 172.21.15.97:0/2751960772 shutdown_connections 2024-04-23T10:38:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.953+0000 7fe83244b740 1 --2- 172.21.15.97:0/2751960772 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5595d45a6ce0 0x5595d460fca0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:55.953 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.953+0000 7fe83244b740 1 --2- 172.21.15.97:0/2751960772 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5595d4596c10 0x5595d460c250 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.953+0000 7fe83244b740 1 --2- 172.21.15.97:0/2751960772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5595d4587e60 0x5595d4591230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.953+0000 7fe83244b740 1 --2- 172.21.15.97:0/2751960772 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5595d4587540 0x5595d4590cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.953+0000 7fe810ff9640 0 client.17379 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:38:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.953+0000 7fe83244b740 1 -- 172.21.15.97:0/2751960772 >> 172.21.15.97:0/2751960772 conn(0x5595d4583b30 msgr2=0x5595d4582cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:38:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.954+0000 7fe83244b740 1 -- 172.21.15.97:0/2751960772 shutdown_connections 2024-04-23T10:38:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.954+0000 7fe83244b740 1 -- 172.21.15.97:0/2751960772 wait complete. 2024-04-23T10:38:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.954+0000 7fe83244b740 20 client.17379 trim_cache size 0 max 16384 2024-04-23T10:38:55.954 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:38:55.955 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:38:55.955 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:38:55.955 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:38:55.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.954+0000 7fe82f73d640 1 -- 172.21.15.97:0/3799868848 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fe8140a3dc0 msgr2=0x7fe8140a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:55.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.954+0000 7fe82f73d640 1 --2- 172.21.15.97:0/3799868848 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fe8140a3dc0 0x7fe8140a6280 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe828101820 tx=0x7fe81800bad0 comp rx=0 tx=0).stop 2024-04-23T10:38:55.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.954+0000 7fe82f73d640 1 -- 172.21.15.97:0/3799868848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe82819cf60 msgr2=0x7fe82819d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:55.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.954+0000 7fe82f73d640 1 --2- 172.21.15.97:0/3799868848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe82819cf60 0x7fe82819d410 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fe82000da40 tx=0x7fe82000df00 comp rx=0 tx=0).stop 2024-04-23T10:38:55.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.954+0000 7fe82f73d640 1 -- 172.21.15.97:0/3799868848 shutdown_connections 2024-04-23T10:38:55.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.954+0000 7fe82f73d640 1 --2- 172.21.15.97:0/3799868848 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fe8140a3dc0 0x7fe8140a6280 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:55.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.954+0000 7fe82f73d640 1 --2- 172.21.15.97:0/3799868848 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe82819cf60 0x7fe82819d410 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:55.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.954+0000 7fe82f73d640 1 --2- 172.21.15.97:0/3799868848 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe8281072a0 0x7fe828198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:55.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.954+0000 7fe82f73d640 1 --2- 172.21.15.97:0/3799868848 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8281021d0 0x7fe8281986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:55.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.954+0000 7fe82f73d640 1 -- 172.21.15.97:0/3799868848 >> 172.21.15.97:0/3799868848 conn(0x7fe8280793e0 msgr2=0x7fe8280ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:38:55.956 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.954+0000 7fe82f73d640 1 -- 172.21.15.97:0/3799868848 shutdown_connections 2024-04-23T10:38:55.957 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:38:55.954+0000 7fe82f73d640 1 -- 172.21.15.97:0/3799868848 wait complete. 2024-04-23T10:38:57.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:38:56 smithi196 ceph-mon[32919]: pgmap v397: 97 pgs: 97 active+clean; 1.5 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.6 KiB/s wr, 10 op/s 2024-04-23T10:38:57.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:38:56 smithi097 ceph-mon[29197]: pgmap v397: 97 pgs: 97 active+clean; 1.5 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.6 KiB/s wr, 10 op/s 2024-04-23T10:38:57.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:38:56 smithi183 ceph-mon[32317]: pgmap v397: 97 pgs: 97 active+clean; 1.5 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.6 KiB/s wr, 10 op/s 2024-04-23T10:38:57.899 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T10:38:58.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.011+0000 7fcd8d507640 1 -- 172.21.15.97:0/3413901348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd881072f0 msgr2=0x7fcd8810eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:58.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.011+0000 7fcd8d507640 1 --2- 172.21.15.97:0/3413901348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd881072f0 0x7fcd8810eec0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fcd80009900 tx=0x7fcd8002a410 comp rx=0 tx=0).stop 2024-04-23T10:38:58.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.011+0000 7fcd8d507640 1 -- 172.21.15.97:0/3413901348 shutdown_connections 2024-04-23T10:38:58.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.011+0000 7fcd8d507640 1 --2- 172.21.15.97:0/3413901348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd881072f0 0x7fcd8810eec0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:58.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.011+0000 7fcd8d507640 1 --2- 172.21.15.97:0/3413901348 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd88102bd0 0x7fcd88106c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:58.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.011+0000 7fcd8d507640 1 --2- 172.21.15.97:0/3413901348 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd88102220 0x7fcd88102600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:58.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.011+0000 7fcd8d507640 1 -- 172.21.15.97:0/3413901348 >> 172.21.15.97:0/3413901348 conn(0x7fcd880fda80 msgr2=0x7fcd880ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:38:58.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.011+0000 7fcd8d507640 1 -- 172.21.15.97:0/3413901348 shutdown_connections 2024-04-23T10:38:58.014 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.011+0000 7fcd8d507640 1 -- 172.21.15.97:0/3413901348 wait complete. 2024-04-23T10:38:58.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.012+0000 7fcd8d507640 1 Processor -- start 2024-04-23T10:38:58.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.012+0000 7fcd8d507640 1 -- start start 2024-04-23T10:38:58.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.012+0000 7fcd8d507640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd88102220 0x7fcd88198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:58.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.012+0000 7fcd8d507640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd881072f0 0x7fcd88198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:58.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.012+0000 7fcd8d507640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd8819cfd0 0x7fcd8819d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:58.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.012+0000 7fcd8d507640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcd88112240 con 0x7fcd8819cfd0 2024-04-23T10:38:58.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.012+0000 7fcd8d507640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcd881120c0 con 0x7fcd881072f0 2024-04-23T10:38:58.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.012+0000 7fcd8d507640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fcd881123c0 con 0x7fcd88102220 2024-04-23T10:38:58.015 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.012+0000 7fcd877fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd8819cfd0 0x7fcd8819d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:58.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.012+0000 7fcd877fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd8819cfd0 0x7fcd8819d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37832/0 (socket says 172.21.15.97:37832) 2024-04-23T10:38:58.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.012+0000 7fcd877fe640 1 -- 172.21.15.97:0/2104817394 learned_addr learned my addr 172.21.15.97:0/2104817394 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:38:58.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.013+0000 7fcd86ffd640 1 --2- 172.21.15.97:0/2104817394 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd88102220 0x7fcd88198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:58.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.013+0000 7fcd867fc640 1 --2- 172.21.15.97:0/2104817394 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd881072f0 0x7fcd88198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:58.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.013+0000 7fcd877fe640 1 -- 172.21.15.97:0/2104817394 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd88102220 msgr2=0x7fcd88198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:58.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.013+0000 7fcd877fe640 1 --2- 172.21.15.97:0/2104817394 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd88102220 0x7fcd88198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:58.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.013+0000 7fcd877fe640 1 -- 172.21.15.97:0/2104817394 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd881072f0 msgr2=0x7fcd88198ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:58.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.013+0000 7fcd877fe640 1 --2- 172.21.15.97:0/2104817394 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd881072f0 0x7fcd88198ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:58.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.013+0000 7fcd877fe640 1 -- 172.21.15.97:0/2104817394 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd8819dac0 con 0x7fcd8819cfd0 2024-04-23T10:38:58.016 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.013+0000 7fcd867fc640 1 --2- 172.21.15.97:0/2104817394 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd881072f0 0x7fcd88198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:38:58.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.013+0000 7fcd877fe640 1 --2- 172.21.15.97:0/2104817394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd8819cfd0 0x7fcd8819d480 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fcd80004770 tx=0x7fcd80004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:38:58.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.013+0000 7fcd6ffff640 1 -- 172.21.15.97:0/2104817394 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd8003f030 con 0x7fcd8819cfd0 2024-04-23T10:38:58.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.013+0000 7fcd8d507640 1 -- 172.21.15.97:0/2104817394 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd8819dd50 con 0x7fcd8819cfd0 2024-04-23T10:38:58.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.013+0000 7fcd6ffff640 1 -- 172.21.15.97:0/2104817394 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd80003aa0 con 0x7fcd8819cfd0 2024-04-23T10:38:58.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.013+0000 7fcd6ffff640 1 -- 172.21.15.97:0/2104817394 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd80035070 con 0x7fcd8819cfd0 2024-04-23T10:38:58.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.013+0000 7fcd8d507640 1 -- 172.21.15.97:0/2104817394 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd881a5570 con 0x7fcd8819cfd0 2024-04-23T10:38:58.017 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.014+0000 7fcd8d507640 1 -- 172.21.15.97:0/2104817394 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd88199a60 con 0x7fcd8819cfd0 2024-04-23T10:38:58.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.015+0000 7fcd6ffff640 1 -- 172.21.15.97:0/2104817394 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fcd80042030 con 0x7fcd8819cfd0 2024-04-23T10:38:58.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.016+0000 7fcd6ffff640 1 --2- 172.21.15.97:0/2104817394 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fcd700a3d20 0x7fcd700a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:58.018 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.016+0000 7fcd6ffff640 1 -- 172.21.15.97:0/2104817394 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8484+0+0 (secure 0 0 0) 0x7fcd800f10e0 con 0x7fcd8819cfd0 2024-04-23T10:38:58.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.016+0000 7fcd86ffd640 1 --2- 172.21.15.97:0/2104817394 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fcd700a3d20 0x7fcd700a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:58.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.016+0000 7fcd86ffd640 1 --2- 172.21.15.97:0/2104817394 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fcd700a3d20 0x7fcd700a61e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fcd78007ec0 tx=0x7fcd780075b0 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:38:58.019 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.020+0000 7fcd6ffff640 1 -- 172.21.15.97:0/2104817394 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcd800bd840 con 0x7fcd8819cfd0 2024-04-23T10:38:58.235 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.236+0000 7fcd8d507640 1 -- 172.21.15.97:0/2104817394 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcd8810a550 con 0x7fcd8819cfd0 2024-04-23T10:38:58.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.237+0000 7fcd6ffff640 1 -- 172.21.15.97:0/2104817394 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11931 (secure 0 0 0) 0x7fcd800bd140 con 0x7fcd8819cfd0 2024-04-23T10:38:58.236 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:38:58.236 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3946878762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3946878762},{"type":"v1","addr":"172.21.15.97:6835","nonce":3946878762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/209800861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":209800861},{"type":"v1","addr":"172.21.15.97:6837","nonce":209800861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T10:24:06.910134+0000","modified":"2024-04-23T10:26:09.820985+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6839/418013575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":418013575},{"type":"v1","addr":"172.21.15.97:6839","nonce":418013575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6841/747242680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":747242680},{"type":"v1","addr":"172.21.15.97:6841","nonce":747242680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6833/4292847121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4292847121},{"type":"v1","addr":"172.21.15.183:6833","nonce":4292847121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6833/3885816565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3885816565},{"type":"v1","addr":"172.21.15.196:6833","nonce":3885816565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1897093105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1897093105},{"type":"v1","addr":"172.21.15.183:6835","nonce":1897093105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1171477563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1171477563},{"type":"v1","addr":"172.21.15.196:6835","nonce":1171477563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1898341911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1898341911},{"type":"v1","addr":"172.21.15.183:6837","nonce":1898341911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.196:6837/3094946502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3094946502},{"type":"v1","addr":"172.21.15.196:6837","nonce":3094946502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.196:6839/83843556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":83843556},{"type":"v1","addr":"172.21.15.196:6839","nonce":83843556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6839/1763841130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1763841130},{"type":"v1","addr":"172.21.15.183:6839","nonce":1763841130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24487,24403,24398,24469,24488]},"id":1}]} 2024-04-23T10:38:58.236 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 26 2024-04-23T10:38:58.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.239+0000 7fcd8d507640 1 -- 172.21.15.97:0/2104817394 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fcd700a3d20 msgr2=0x7fcd700a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:58.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.239+0000 7fcd8d507640 1 --2- 172.21.15.97:0/2104817394 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fcd700a3d20 0x7fcd700a61e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fcd78007ec0 tx=0x7fcd780075b0 comp rx=0 tx=0).stop 2024-04-23T10:38:58.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.239+0000 7fcd8d507640 1 -- 172.21.15.97:0/2104817394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd8819cfd0 msgr2=0x7fcd8819d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:58.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.239+0000 7fcd8d507640 1 --2- 172.21.15.97:0/2104817394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd8819cfd0 0x7fcd8819d480 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fcd80004770 tx=0x7fcd80004270 comp rx=0 tx=0).stop 2024-04-23T10:38:58.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.239+0000 7fcd8d507640 1 -- 172.21.15.97:0/2104817394 shutdown_connections 2024-04-23T10:38:58.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.239+0000 7fcd8d507640 1 --2- 172.21.15.97:0/2104817394 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fcd700a3d20 0x7fcd700a61e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:58.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.239+0000 7fcd8d507640 1 --2- 172.21.15.97:0/2104817394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd8819cfd0 0x7fcd8819d480 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:58.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.239+0000 7fcd8d507640 1 --2- 172.21.15.97:0/2104817394 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd881072f0 0x7fcd88198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:58.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.239+0000 7fcd8d507640 1 --2- 172.21.15.97:0/2104817394 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcd88102220 0x7fcd88198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:58.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.239+0000 7fcd8d507640 1 -- 172.21.15.97:0/2104817394 >> 172.21.15.97:0/2104817394 conn(0x7fcd880fda80 msgr2=0x7fcd880ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:38:58.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.239+0000 7fcd8d507640 1 -- 172.21.15.97:0/2104817394 shutdown_connections 2024-04-23T10:38:58.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.239+0000 7fcd8d507640 1 -- 172.21.15.97:0/2104817394 wait complete. 2024-04-23T10:38:58.249 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --set-id 845978d3 --release --await' 2024-04-23T10:38:58.249 DEBUG:teuthology.orchestra.run.smithi097:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24488 quiesce db --set-id 845978d3 --release --await 2024-04-23T10:38:58.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.354+0000 7f6ed67cd640 1 -- 172.21.15.97:0/2914505885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ed0101a50 msgr2=0x7f6ed0105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:58.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.354+0000 7f6ed67cd640 1 --2- 172.21.15.97:0/2914505885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ed0101a50 0x7f6ed0105a90 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4009900 tx=0x7f6ec402a410 comp rx=0 tx=0).stop 2024-04-23T10:38:58.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.355+0000 7f6ed67cd640 1 -- 172.21.15.97:0/2914505885 shutdown_connections 2024-04-23T10:38:58.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.355+0000 7f6ed67cd640 1 --2- 172.21.15.97:0/2914505885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ed0106170 0x7f6ed010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:58.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.355+0000 7f6ed67cd640 1 --2- 172.21.15.97:0/2914505885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ed0101a50 0x7f6ed0105a90 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:58.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.355+0000 7f6ed67cd640 1 --2- 172.21.15.97:0/2914505885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ed01010a0 0x7f6ed0101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:58.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.355+0000 7f6ed67cd640 1 -- 172.21.15.97:0/2914505885 >> 172.21.15.97:0/2914505885 conn(0x7f6ed0077eb0 msgr2=0x7f6ed00782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:38:58.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.355+0000 7f6ed67cd640 1 -- 172.21.15.97:0/2914505885 shutdown_connections 2024-04-23T10:38:58.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.355+0000 7f6ed67cd640 1 -- 172.21.15.97:0/2914505885 wait complete. 2024-04-23T10:38:58.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.355+0000 7f6ed67cd640 1 Processor -- start 2024-04-23T10:38:58.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.355+0000 7f6ed67cd640 1 -- start start 2024-04-23T10:38:58.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.355+0000 7f6ed67cd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ed01010a0 0x7f6ed01994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:58.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.355+0000 7f6ed67cd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ed0106170 0x7f6ed0199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:58.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.355+0000 7f6ed67cd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ed0199fc0 0x7f6ed01a4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:58.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.356+0000 7f6ed67cd640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed0111090 con 0x7f6ed01010a0 2024-04-23T10:38:58.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.356+0000 7f6ed67cd640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed0110f10 con 0x7f6ed0199fc0 2024-04-23T10:38:58.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.356+0000 7f6ed67cd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed0111210 con 0x7f6ed0106170 2024-04-23T10:38:58.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.356+0000 7f6ecffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ed01010a0 0x7f6ed01994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:58.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.356+0000 7f6ecffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ed01010a0 0x7f6ed01994f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37854/0 (socket says 172.21.15.97:37854) 2024-04-23T10:38:58.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.356+0000 7f6ecffff640 1 -- 172.21.15.97:0/4215697054 learned_addr learned my addr 172.21.15.97:0/4215697054 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:38:58.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.356+0000 7f6ecf7fe640 1 --2- 172.21.15.97:0/4215697054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ed0106170 0x7f6ed0199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:58.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.356+0000 7f6ed4d6a640 1 --2- 172.21.15.97:0/4215697054 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ed0199fc0 0x7f6ed01a4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:58.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.356+0000 7f6ecffff640 1 -- 172.21.15.97:0/4215697054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ed0106170 msgr2=0x7f6ed0199a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:58.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.356+0000 7f6ecffff640 1 --2- 172.21.15.97:0/4215697054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ed0106170 0x7f6ed0199a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:58.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.356+0000 7f6ecffff640 1 -- 172.21.15.97:0/4215697054 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ed0199fc0 msgr2=0x7f6ed01a4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:58.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.356+0000 7f6ecffff640 1 --2- 172.21.15.97:0/4215697054 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ed0199fc0 0x7f6ed01a4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:58.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.356+0000 7f6ecffff640 1 -- 172.21.15.97:0/4215697054 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ed01a4d70 con 0x7f6ed01010a0 2024-04-23T10:38:58.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.356+0000 7f6ecffff640 1 --2- 172.21.15.97:0/4215697054 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ed01010a0 0x7f6ed01994f0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f6ec000d950 tx=0x7f6ec000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:38:58.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.356+0000 7f6ecd7fa640 1 -- 172.21.15.97:0/4215697054 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec0017030 con 0x7f6ed01010a0 2024-04-23T10:38:58.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.356+0000 7f6ed67cd640 1 -- 172.21.15.97:0/4215697054 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ed01a5060 con 0x7f6ed01010a0 2024-04-23T10:38:58.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.356+0000 7f6ecd7fa640 1 -- 172.21.15.97:0/4215697054 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ec0004020 con 0x7f6ed01010a0 2024-04-23T10:38:58.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.356+0000 7f6ecd7fa640 1 -- 172.21.15.97:0/4215697054 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec00047a0 con 0x7f6ed01010a0 2024-04-23T10:38:58.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.356+0000 7f6ed67cd640 1 -- 172.21.15.97:0/4215697054 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ed01a55d0 con 0x7f6ed01010a0 2024-04-23T10:38:58.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.358+0000 7f6ecd7fa640 1 -- 172.21.15.97:0/4215697054 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6ec0009a30 con 0x7f6ed01010a0 2024-04-23T10:38:58.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.359+0000 7f6ed95be740 1 Processor -- start 2024-04-23T10:38:58.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.359+0000 7f6ed95be740 1 -- start start 2024-04-23T10:38:58.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.359+0000 7f6ecd7fa640 1 --2- 172.21.15.97:0/4215697054 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f6eb80a3d20 0x7f6eb80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:58.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.359+0000 7f6ed95be740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e4e534390 0x557e4e534770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:58.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.359+0000 7f6ed95be740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e4e534cb0 0x557e4e543350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:58.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.359+0000 7f6ed95be740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e4e543890 0x557e4e545c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:58.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.359+0000 7f6ecd7fa640 1 -- 172.21.15.97:0/4215697054 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8484+0+0 (secure 0 0 0) 0x7f6ec00d01b0 con 0x7f6ed01010a0 2024-04-23T10:38:58.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.359+0000 7f6ed95be740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x557e4e4996d0 con 0x557e4e543890 2024-04-23T10:38:58.369 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.359+0000 7f6ed4d6a640 1 --2- 172.21.15.97:0/4215697054 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f6eb80a3d20 0x7f6eb80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:58.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.359+0000 7f6ed95be740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557e4e4361a0 con 0x557e4e534390 2024-04-23T10:38:58.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.359+0000 7f6ed95be740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557e4e495690 con 0x557e4e534cb0 2024-04-23T10:38:58.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.359+0000 7f6ed4d6a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e4e543890 0x557e4e545c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:58.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.359+0000 7f6ed4d6a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e4e543890 0x557e4e545c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37868/0 (socket says 172.21.15.97:37868) 2024-04-23T10:38:58.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.359+0000 7f6ed4d6a640 1 -- 172.21.15.97:0/4221236005 learned_addr learned my addr 172.21.15.97:0/4221236005 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:38:58.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.359+0000 7f6ecf7fe640 1 --2- 172.21.15.97:0/4221236005 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e4e534cb0 0x557e4e543350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:58.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.359+0000 7f6ecffff640 1 --2- 172.21.15.97:0/4221236005 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e4e534390 0x557e4e534770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:58.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.360+0000 7f6ed4d6a640 1 --2- 172.21.15.97:0/4215697054 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f6eb80a3d20 0x7f6eb80a61e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8013790 tx=0x7f6ec8013b90 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:38:58.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.360+0000 7f6ecf7fe640 1 -- 172.21.15.97:0/4221236005 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e4e534390 msgr2=0x557e4e534770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:38:58.370 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.360+0000 7f6ecf7fe640 1 --2- 172.21.15.97:0/4221236005 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e4e534390 0x557e4e534770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:58.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.360+0000 7f6ecf7fe640 1 -- 172.21.15.97:0/4221236005 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e4e543890 msgr2=0x557e4e545c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:38:58.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.360+0000 7f6ecf7fe640 1 --2- 172.21.15.97:0/4221236005 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e4e543890 0x557e4e545c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:58.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.360+0000 7f6ecf7fe640 1 -- 172.21.15.97:0/4221236005 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e4e4405b0 con 0x557e4e534cb0 2024-04-23T10:38:58.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.360+0000 7f6ed4d6a640 1 --2- 172.21.15.97:0/4221236005 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e4e543890 0x557e4e545c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T10:38:58.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.360+0000 7f6ecf7fe640 1 --2- 172.21.15.97:0/4221236005 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e4e534cb0 0x557e4e543350 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6ec402a930 tx=0x7f6ec4032df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef1d7e2ab619ff31 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:38:58.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.360+0000 7f6eb67fc640 1 -- 172.21.15.97:0/4221236005 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec4035040 con 0x557e4e534cb0 2024-04-23T10:38:58.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.360+0000 7f6eb67fc640 1 -- 172.21.15.97:0/4221236005 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ec40316a0 con 0x557e4e534cb0 2024-04-23T10:38:58.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.360+0000 7f6eb67fc640 1 -- 172.21.15.97:0/4221236005 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec4005590 con 0x557e4e534cb0 2024-04-23T10:38:58.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.361+0000 7f6ed95be740 1 -- 172.21.15.97:0/4221236005 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e4e534cb0 msgr2=0x557e4e543350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:58.371 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.361+0000 7f6ed95be740 1 --2- 172.21.15.97:0/4221236005 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e4e534cb0 0x557e4e543350 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6ec402a930 tx=0x7f6ec4032df0 comp rx=0 tx=0).stop 2024-04-23T10:38:58.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.361+0000 7f6ed95be740 1 -- 172.21.15.97:0/4221236005 shutdown_connections 2024-04-23T10:38:58.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.361+0000 7f6ed95be740 1 --2- 172.21.15.97:0/4221236005 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e4e543890 0x557e4e545c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:58.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.361+0000 7f6ed95be740 1 --2- 172.21.15.97:0/4221236005 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e4e534cb0 0x557e4e543350 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:58.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.361+0000 7f6ed95be740 1 --2- 172.21.15.97:0/4221236005 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e4e534390 0x557e4e534770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:58.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.361+0000 7f6ed95be740 1 -- 172.21.15.97:0/4221236005 >> 172.21.15.97:0/4221236005 conn(0x557e4e5307b0 msgr2=0x557e4e530b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:38:58.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.361+0000 7f6ed95be740 1 -- 172.21.15.97:0/4221236005 shutdown_connections 2024-04-23T10:38:58.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.361+0000 7f6ed95be740 1 -- 172.21.15.97:0/4221236005 wait complete. 2024-04-23T10:38:58.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.361+0000 7f6ed95be740 1 -- start start 2024-04-23T10:38:58.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.361+0000 7f6ed95be740 1 Processor -- start 2024-04-23T10:38:58.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.362+0000 7f6ed95be740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e4e534390 0x557e4e543350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:58.372 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.362+0000 7f6ed95be740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e4e534cb0 0x557e4e540640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:58.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.362+0000 7f6ed95be740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e4e543890 0x557e4e540d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:58.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.362+0000 7f6ed95be740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x557e4e546690 con 0x557e4e543890 2024-04-23T10:38:58.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.362+0000 7f6ed95be740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557e4e433b20 con 0x557e4e534cb0 2024-04-23T10:38:58.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.362+0000 7f6ed95be740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557e4e546810 con 0x557e4e534390 2024-04-23T10:38:58.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.362+0000 7f6ed95be740 10 client.0 _ll_register_callbacks cb 0x557e4e433530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:38:58.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.362+0000 7f6ed4d6a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e4e543890 0x557e4e540d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:58.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.362+0000 7f6ed4d6a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e4e543890 0x557e4e540d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37884/0 (socket says 172.21.15.97:37884) 2024-04-23T10:38:58.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.362+0000 7f6ecffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e4e534390 0x557e4e543350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:58.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.362+0000 7f6ed4d6a640 1 -- 172.21.15.97:0/3332971986 learned_addr learned my addr 172.21.15.97:0/3332971986 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:38:58.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.362+0000 7f6ecf7fe640 1 --2- 172.21.15.97:0/3332971986 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e4e534cb0 0x557e4e540640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:58.373 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.362+0000 7f6ed4d6a640 1 -- 172.21.15.97:0/3332971986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e4e534390 msgr2=0x557e4e543350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:38:58.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.362+0000 7f6ed4d6a640 1 --2- 172.21.15.97:0/3332971986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e4e534390 0x557e4e543350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:58.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.362+0000 7f6ed4d6a640 1 -- 172.21.15.97:0/3332971986 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e4e534cb0 msgr2=0x557e4e540640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:38:58.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.362+0000 7f6ed4d6a640 1 --2- 172.21.15.97:0/3332971986 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e4e534cb0 0x557e4e540640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:58.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.362+0000 7f6ed4d6a640 1 -- 172.21.15.97:0/3332971986 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e4e1ab920 con 0x557e4e543890 2024-04-23T10:38:58.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.362+0000 7f6ed4d6a640 1 --2- 172.21.15.97:0/3332971986 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e4e543890 0x557e4e540d80 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f6ec801f640 tx=0x7f6ec8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=679aee29baa9c703 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:38:58.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.362+0000 7f6eaf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:38:58.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.362+0000 7f6eaf7fe640 1 -- 172.21.15.97:0/3332971986 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec8020860 con 0x557e4e543890 2024-04-23T10:38:58.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.362+0000 7f6eaf7fe640 1 -- 172.21.15.97:0/3332971986 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8484+0+0 (secure 0 0 0) 0x7f6ec8020a00 con 0x557e4e543890 2024-04-23T10:38:58.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.362+0000 7f6ed95be740 1 -- 172.21.15.97:0/3332971986 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ec40316a0 con 0x557e4e543890 2024-04-23T10:38:58.374 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.362+0000 7f6eaf7fe640 1 -- 172.21.15.97:0/3332971986 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6ec8020c00 con 0x557e4e543890 2024-04-23T10:38:58.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.362+0000 7f6ed95be740 10 client.17397 fetch_fsmap learned FSMap version 26 2024-04-23T10:38:58.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.362+0000 7f6ed95be740 1 -- 172.21.15.97:0/3332971986 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557e4e277d30 con 0x557e4e543890 2024-04-23T10:38:58.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.363+0000 7f6eaf7fe640 1 -- 172.21.15.97:0/3332971986 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ec40316a0 con 0x557e4e543890 2024-04-23T10:38:58.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.363+0000 7f6eaf7fe640 1 -- 172.21.15.97:0/3332971986 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f6ec800e3b0 con 0x557e4e543890 2024-04-23T10:38:58.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.363+0000 7f6ed95be740 10 client.17397 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:38:58.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.363+0000 7f6ed95be740 10 client.17397 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:38:58.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.363+0000 7f6ed95be740 20 client.17397 populate_metadata read hostname 'smithi097' 2024-04-23T10:38:58.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.363+0000 7f6ed95be740 1 --2- 172.21.15.97:0/3332971986 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x557e4e5539c0 0x557e4e555df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:58.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.363+0000 7f6ed95be740 4 client.17397 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:38:58.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.363+0000 7f6ed95be740 1 -- 172.21.15.97:0/3332971986 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e4e457500 con 0x557e4e5539c0 2024-04-23T10:38:58.375 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.363+0000 7f6ecffff640 1 --2- 172.21.15.97:0/3332971986 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x557e4e5539c0 0x557e4e555df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:58.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.363+0000 7f6eaf7fe640 10 client.17397 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:38:58.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.363+0000 7f6ecffff640 1 --2- 172.21.15.97:0/3332971986 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x557e4e5539c0 0x557e4e555df0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f97486483c69bc79 server_cookie=81c1c7bcab3b82a1 in_seq=0 out_seq=0 2024-04-23T10:38:58.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.364+0000 7f6eaf7fe640 1 -- 172.21.15.97:0/3332971986 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557e4e457500 con 0x557e4e5539c0 2024-04-23T10:38:58.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.364+0000 7f6eaf7fe640 10 client.17397 handle_command_reply: tid=0 2024-04-23T10:38:58.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.364+0000 7f6ed95be740 1 client.17397 shutdown 2024-04-23T10:38:58.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.364+0000 7f6ed95be740 1 -- 172.21.15.97:0/3332971986 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e4e543890 msgr2=0x557e4e540d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:58.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.364+0000 7f6ed95be740 1 --2- 172.21.15.97:0/3332971986 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e4e543890 0x557e4e540d80 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f6ec801f640 tx=0x7f6ec8004ab0 comp rx=0 tx=0).stop 2024-04-23T10:38:58.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.364+0000 7f6ed95be740 1 -- 172.21.15.97:0/3332971986 shutdown_connections 2024-04-23T10:38:58.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.364+0000 7f6ed95be740 1 --2- 172.21.15.97:0/3332971986 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x557e4e5539c0 0x557e4e555df0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:58.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.364+0000 7f6ed95be740 1 --2- 172.21.15.97:0/3332971986 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e4e543890 0x557e4e540d80 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:58.376 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.364+0000 7f6ed95be740 1 --2- 172.21.15.97:0/3332971986 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e4e534cb0 0x557e4e540640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:58.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.364+0000 7f6eaf7fe640 0 client.17397 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:38:58.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.364+0000 7f6ed95be740 1 --2- 172.21.15.97:0/3332971986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e4e534390 0x557e4e543350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:58.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.364+0000 7f6ed95be740 1 -- 172.21.15.97:0/3332971986 >> 172.21.15.97:0/3332971986 conn(0x557e4e5307b0 msgr2=0x557e4e536620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:38:58.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.364+0000 7f6ed95be740 1 -- 172.21.15.97:0/3332971986 shutdown_connections 2024-04-23T10:38:58.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.364+0000 7f6ed95be740 1 -- 172.21.15.97:0/3332971986 wait complete. 2024-04-23T10:38:58.377 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.365+0000 7f6ed95be740 20 client.17397 trim_cache size 0 max 16384 2024-04-23T10:38:58.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.381+0000 7f6ed95be740 1 Processor -- start 2024-04-23T10:38:58.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.381+0000 7f6ed95be740 1 -- start start 2024-04-23T10:38:58.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.382+0000 7f6ed95be740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e4e534390 0x557e4e5364a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:58.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.382+0000 7f6ed95be740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e4e534cb0 0x557e4e542e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:58.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.382+0000 7f6ed95be740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e4e543890 0x557e4e55b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:58.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.382+0000 7f6ed95be740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed01ab8d0 con 0x557e4e534cb0 2024-04-23T10:38:58.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.382+0000 7f6ed95be740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed01ad410 con 0x557e4e543890 2024-04-23T10:38:58.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.382+0000 7f6ed95be740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed01abfa0 con 0x557e4e534390 2024-04-23T10:38:58.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.382+0000 7f6ecf7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e4e534cb0 0x557e4e542e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:58.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.382+0000 7f6ecf7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e4e534cb0 0x557e4e542e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37900/0 (socket says 172.21.15.97:37900) 2024-04-23T10:38:58.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.382+0000 7f6ecf7fe640 1 -- 172.21.15.97:0/477390336 learned_addr learned my addr 172.21.15.97:0/477390336 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:38:58.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.382+0000 7f6ed4d6a640 1 --2- 172.21.15.97:0/477390336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e4e543890 0x557e4e55b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:58.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.382+0000 7f6ecffff640 1 --2- 172.21.15.97:0/477390336 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e4e534390 0x557e4e5364a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:58.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.382+0000 7f6ecf7fe640 1 -- 172.21.15.97:0/477390336 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e4e534390 msgr2=0x557e4e5364a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:38:58.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.382+0000 7f6ecf7fe640 1 --2- 172.21.15.97:0/477390336 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e4e534390 0x557e4e5364a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:58.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.382+0000 7f6ecf7fe640 1 -- 172.21.15.97:0/477390336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e4e543890 msgr2=0x557e4e55b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:38:58.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.382+0000 7f6ecf7fe640 1 --2- 172.21.15.97:0/477390336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e4e543890 0x557e4e55b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:58.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.382+0000 7f6ecf7fe640 1 -- 172.21.15.97:0/477390336 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e4e53c280 con 0x557e4e534cb0 2024-04-23T10:38:58.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.382+0000 7f6ecffff640 1 --2- 172.21.15.97:0/477390336 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e4e534390 0x557e4e5364a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:38:58.388 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.382+0000 7f6ecf7fe640 1 --2- 172.21.15.97:0/477390336 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e4e534cb0 0x557e4e542e80 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4005240 tx=0x7f6ec4005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8e0f19aacab44b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:38:58.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.382+0000 7f6eb5ffb640 1 -- 172.21.15.97:0/477390336 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec4035040 con 0x557e4e534cb0 2024-04-23T10:38:58.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.382+0000 7f6eb5ffb640 1 -- 172.21.15.97:0/477390336 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ec40052e0 con 0x557e4e534cb0 2024-04-23T10:38:58.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.382+0000 7f6eb5ffb640 1 -- 172.21.15.97:0/477390336 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec4004290 con 0x557e4e534cb0 2024-04-23T10:38:58.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.383+0000 7f6ed95be740 1 -- 172.21.15.97:0/477390336 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e4e534cb0 msgr2=0x557e4e542e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:58.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.383+0000 7f6ed95be740 1 --2- 172.21.15.97:0/477390336 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e4e534cb0 0x557e4e542e80 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4005240 tx=0x7f6ec4005270 comp rx=0 tx=0).stop 2024-04-23T10:38:58.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.383+0000 7f6ed95be740 1 -- 172.21.15.97:0/477390336 shutdown_connections 2024-04-23T10:38:58.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.383+0000 7f6ed95be740 1 --2- 172.21.15.97:0/477390336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e4e543890 0x557e4e55b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:58.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.383+0000 7f6ed95be740 1 --2- 172.21.15.97:0/477390336 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e4e534cb0 0x557e4e542e80 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:58.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.383+0000 7f6ed95be740 1 --2- 172.21.15.97:0/477390336 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e4e534390 0x557e4e5364a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:58.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.383+0000 7f6ed95be740 1 -- 172.21.15.97:0/477390336 >> 172.21.15.97:0/477390336 conn(0x557e4e5307b0 msgr2=0x557e4e535d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:38:58.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.383+0000 7f6ed95be740 1 -- 172.21.15.97:0/477390336 shutdown_connections 2024-04-23T10:38:58.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.383+0000 7f6ed95be740 1 -- 172.21.15.97:0/477390336 wait complete. 2024-04-23T10:38:58.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.383+0000 7f6ed95be740 1 -- start start 2024-04-23T10:38:58.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.383+0000 7f6ed95be740 1 Processor -- start 2024-04-23T10:38:58.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.384+0000 7f6ed95be740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e4e534390 0x557e4e53da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:58.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.384+0000 7f6ed95be740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e4e534cb0 0x557e4e53dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:58.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.384+0000 7f6ed95be740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e4e543890 0x557e4e54a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:58.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.384+0000 7f6ed95be740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed01acd40 con 0x557e4e534cb0 2024-04-23T10:38:58.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.384+0000 7f6ed95be740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed01ac670 con 0x557e4e543890 2024-04-23T10:38:58.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.384+0000 7f6ed95be740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed01a98d0 con 0x557e4e534390 2024-04-23T10:38:58.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.384+0000 7f6ed95be740 10 client.0 _ll_register_callbacks cb 0x7f6ec8011d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:38:58.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.384+0000 7f6ecf7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e4e534cb0 0x557e4e53dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:58.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.384+0000 7f6ecf7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e4e534cb0 0x557e4e53dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37902/0 (socket says 172.21.15.97:37902) 2024-04-23T10:38:58.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.384+0000 7f6ecf7fe640 1 -- 172.21.15.97:0/2029844489 learned_addr learned my addr 172.21.15.97:0/2029844489 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:38:58.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.384+0000 7f6ed4d6a640 1 --2- 172.21.15.97:0/2029844489 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e4e543890 0x557e4e54a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:58.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.384+0000 7f6ecffff640 1 --2- 172.21.15.97:0/2029844489 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e4e534390 0x557e4e53da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:58.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.384+0000 7f6ecf7fe640 1 -- 172.21.15.97:0/2029844489 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e4e534390 msgr2=0x557e4e53da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:38:58.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.384+0000 7f6ecf7fe640 1 --2- 172.21.15.97:0/2029844489 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e4e534390 0x557e4e53da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:58.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.384+0000 7f6ecf7fe640 1 -- 172.21.15.97:0/2029844489 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e4e543890 msgr2=0x557e4e54a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:38:58.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.384+0000 7f6ecf7fe640 1 --2- 172.21.15.97:0/2029844489 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e4e543890 0x557e4e54a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:58.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.384+0000 7f6ecf7fe640 1 -- 172.21.15.97:0/2029844489 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e4e54a6d0 con 0x557e4e534cb0 2024-04-23T10:38:58.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.384+0000 7f6ecffff640 1 --2- 172.21.15.97:0/2029844489 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e4e534390 0x557e4e53da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:38:58.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.384+0000 7f6ed4d6a640 1 --2- 172.21.15.97:0/2029844489 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e4e543890 0x557e4e54a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:38:58.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.384+0000 7f6ecf7fe640 1 --2- 172.21.15.97:0/2029844489 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e4e534cb0 0x557e4e53dfb0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4009900 tx=0x7f6ec4005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45378e58efe7fe53 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:38:58.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.384+0000 7f6eaf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:38:58.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.384+0000 7f6eaf7fe640 1 -- 172.21.15.97:0/2029844489 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec4053070 con 0x557e4e534cb0 2024-04-23T10:38:58.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.384+0000 7f6eaf7fe640 1 -- 172.21.15.97:0/2029844489 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8484+0+0 (secure 0 0 0) 0x7f6ec40314a0 con 0x557e4e534cb0 2024-04-23T10:38:58.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.384+0000 7f6ed95be740 1 -- 172.21.15.97:0/2029844489 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ec40052e0 con 0x557e4e534cb0 2024-04-23T10:38:58.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.384+0000 7f6eaf7fe640 1 -- 172.21.15.97:0/2029844489 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6ec4035070 con 0x557e4e534cb0 2024-04-23T10:38:58.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.384+0000 7f6ed95be740 10 client.17403 fetch_fsmap learned FSMap version 26 2024-04-23T10:38:58.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.384+0000 7f6ed95be740 1 -- 172.21.15.97:0/2029844489 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557e4e5ba690 con 0x557e4e534cb0 2024-04-23T10:38:58.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.385+0000 7f6eaf7fe640 1 -- 172.21.15.97:0/2029844489 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ec40052e0 con 0x557e4e534cb0 2024-04-23T10:38:58.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.385+0000 7f6eaf7fe640 1 -- 172.21.15.97:0/2029844489 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f6ec4004cb0 con 0x557e4e534cb0 2024-04-23T10:38:58.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.385+0000 7f6ed95be740 10 client.17403 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:38:58.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.385+0000 7f6ed95be740 10 client.17403 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:38:58.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.385+0000 7f6ed95be740 20 client.17403 populate_metadata read hostname 'smithi097' 2024-04-23T10:38:58.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.385+0000 7f6ed95be740 1 --2- 172.21.15.97:0/2029844489 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x557e4e5ba960 0x557e4e5bcd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:38:58.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.385+0000 7f6ed95be740 4 client.17403 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "845978d3", "release": true, "await": true}] 2024-04-23T10:38:58.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.385+0000 7f6ed95be740 1 -- 172.21.15.97:0/2029844489 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "quiesce db", "set_id": "845978d3", "release": true, "await": true}) -- 0x557e4e4a70f0 con 0x557e4e5ba960 2024-04-23T10:38:58.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.385+0000 7f6ecffff640 1 --2- 172.21.15.97:0/2029844489 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x557e4e5ba960 0x557e4e5bcd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:38:58.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.385+0000 7f6ecffff640 1 --2- 172.21.15.97:0/2029844489 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x557e4e5ba960 0x557e4e5bcd50 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c419a4bfcec3f13 server_cookie=9bc2d9cfbbcad171 in_seq=0 out_seq=0 2024-04-23T10:38:58.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.385+0000 7f6eaf7fe640 10 client.17403 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:38:58.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.389+0000 7f6eaf7fe640 1 -- 172.21.15.97:0/2029844489 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x557e4e4a70f0 con 0x557e4e5ba960 2024-04-23T10:38:58.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.389+0000 7f6eaf7fe640 10 client.17403 handle_command_reply: tid=0 2024-04-23T10:38:58.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.389+0000 7f6ed95be740 1 client.17403 shutdown 2024-04-23T10:38:58.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.389+0000 7f6ed95be740 1 -- 172.21.15.97:0/2029844489 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e4e534cb0 msgr2=0x557e4e53dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:58.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.389+0000 7f6ed95be740 1 --2- 172.21.15.97:0/2029844489 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e4e534cb0 0x557e4e53dfb0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4009900 tx=0x7f6ec4005e80 comp rx=0 tx=0).stop 2024-04-23T10:38:58.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.389+0000 7f6ed95be740 1 -- 172.21.15.97:0/2029844489 shutdown_connections 2024-04-23T10:38:58.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.389+0000 7f6ed95be740 1 --2- 172.21.15.97:0/2029844489 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x557e4e5ba960 0x557e4e5bcd50 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:58.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.389+0000 7f6ed95be740 1 --2- 172.21.15.97:0/2029844489 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e4e543890 0x557e4e54a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:58.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.389+0000 7f6eaf7fe640 0 client.17403 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:38:58.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.389+0000 7f6ed95be740 1 --2- 172.21.15.97:0/2029844489 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557e4e534cb0 0x557e4e53dfb0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:58.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.389+0000 7f6ed95be740 1 --2- 172.21.15.97:0/2029844489 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557e4e534390 0x557e4e53da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:58.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.389+0000 7f6ed95be740 1 -- 172.21.15.97:0/2029844489 >> 172.21.15.97:0/2029844489 conn(0x557e4e5307b0 msgr2=0x557e4e52f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:38:58.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.389+0000 7f6ed95be740 1 -- 172.21.15.97:0/2029844489 shutdown_connections 2024-04-23T10:38:58.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.389+0000 7f6ed95be740 1 -- 172.21.15.97:0/2029844489 wait complete. 2024-04-23T10:38:58.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.389+0000 7f6ed95be740 20 client.17403 trim_cache size 0 max 16384 2024-04-23T10:38:58.396 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-23T10:38:58.396 INFO:teuthology.orchestra.run.smithi097.stdout: "epoch": 26, 2024-04-23T10:38:58.397 INFO:teuthology.orchestra.run.smithi097.stdout: "leader": 24488, 2024-04-23T10:38:58.397 INFO:teuthology.orchestra.run.smithi097.stdout: "set_version": 56, 2024-04-23T10:38:58.397 INFO:teuthology.orchestra.run.smithi097.stdout: "sets": { 2024-04-23T10:38:58.397 INFO:teuthology.orchestra.run.smithi097.stdout: "845978d3": { 2024-04-23T10:38:58.397 INFO:teuthology.orchestra.run.smithi097.stdout: "version": 56, 2024-04-23T10:38:58.397 INFO:teuthology.orchestra.run.smithi097.stdout: "age_ref": 0.0, 2024-04-23T10:38:58.397 INFO:teuthology.orchestra.run.smithi097.stdout: "state": { 2024-04-23T10:38:58.397 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RELEASED", 2024-04-23T10:38:58.398 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0 2024-04-23T10:38:58.398 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:38:58.398 INFO:teuthology.orchestra.run.smithi097.stdout: "timeout": 60.0, 2024-04-23T10:38:58.398 INFO:teuthology.orchestra.run.smithi097.stdout: "expiration": 90.0, 2024-04-23T10:38:58.398 INFO:teuthology.orchestra.run.smithi097.stdout: "members": { 2024-04-23T10:38:58.398 INFO:teuthology.orchestra.run.smithi097.stdout: "file:/": { 2024-04-23T10:38:58.398 INFO:teuthology.orchestra.run.smithi097.stdout: "excluded": false, 2024-04-23T10:38:58.398 INFO:teuthology.orchestra.run.smithi097.stdout: "state": { 2024-04-23T10:38:58.398 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RELEASED", 2024-04-23T10:38:58.398 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0 2024-04-23T10:38:58.399 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:38:58.399 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:38:58.399 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:38:58.399 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:38:58.399 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:38:58.399 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-23T10:38:58.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.390+0000 7f6ed67cd640 1 -- 172.21.15.97:0/4215697054 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f6eb80a3d20 msgr2=0x7f6eb80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:58.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.390+0000 7f6ed67cd640 1 --2- 172.21.15.97:0/4215697054 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f6eb80a3d20 0x7f6eb80a61e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8013790 tx=0x7f6ec8013b90 comp rx=0 tx=0).stop 2024-04-23T10:38:58.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.390+0000 7f6ed67cd640 1 -- 172.21.15.97:0/4215697054 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ed01010a0 msgr2=0x7f6ed01994f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:38:58.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.390+0000 7f6ed67cd640 1 --2- 172.21.15.97:0/4215697054 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ed01010a0 0x7f6ed01994f0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f6ec000d950 tx=0x7f6ec000de10 comp rx=0 tx=0).stop 2024-04-23T10:38:58.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.390+0000 7f6ed67cd640 1 -- 172.21.15.97:0/4215697054 shutdown_connections 2024-04-23T10:38:58.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.390+0000 7f6ed67cd640 1 --2- 172.21.15.97:0/4215697054 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f6eb80a3d20 0x7f6eb80a61e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:58.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.390+0000 7f6ed67cd640 1 --2- 172.21.15.97:0/4215697054 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ed0199fc0 0x7f6ed01a4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:58.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.390+0000 7f6ed67cd640 1 --2- 172.21.15.97:0/4215697054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ed0106170 0x7f6ed0199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:58.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.390+0000 7f6ed67cd640 1 --2- 172.21.15.97:0/4215697054 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ed01010a0 0x7f6ed01994f0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:38:58.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.390+0000 7f6ed67cd640 1 -- 172.21.15.97:0/4215697054 >> 172.21.15.97:0/4215697054 conn(0x7f6ed0077eb0 msgr2=0x7f6ed0107340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:38:58.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.390+0000 7f6ed67cd640 1 -- 172.21.15.97:0/4215697054 shutdown_connections 2024-04-23T10:38:58.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:38:58.390+0000 7f6ed67cd640 1 -- 172.21.15.97:0/4215697054 wait complete. 2024-04-23T10:38:58.402 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '845978d3', seconds elapsed: 31.0 2024-04-23T10:38:59.091 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:38:58 smithi196 ceph-mon[32919]: pgmap v398: 97 pgs: 97 active+clean; 1.5 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.2 KiB/s wr, 15 op/s 2024-04-23T10:38:59.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:38:58 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/2104817394' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:38:59.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:38:58 smithi097 ceph-mon[29197]: pgmap v398: 97 pgs: 97 active+clean; 1.5 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.2 KiB/s wr, 15 op/s 2024-04-23T10:38:59.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:38:58 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/2104817394' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:38:59.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:38:58 smithi183 ceph-mon[32317]: pgmap v398: 97 pgs: 97 active+clean; 1.5 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.2 KiB/s wr, 15 op/s 2024-04-23T10:38:59.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:38:58 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/2104817394' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:39:01.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:39:00 smithi196 ceph-mon[32919]: pgmap v399: 97 pgs: 97 active+clean; 1.5 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-23T10:39:01.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:39:00 smithi097 ceph-mon[29197]: pgmap v399: 97 pgs: 97 active+clean; 1.5 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-23T10:39:01.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:39:00 smithi183 ceph-mon[32317]: pgmap v399: 97 pgs: 97 active+clean; 1.5 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-23T10:39:03.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:39:02 smithi196 ceph-mon[32919]: pgmap v400: 97 pgs: 97 active+clean; 1.5 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-23T10:39:03.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:39:02 smithi097 ceph-mon[29197]: pgmap v400: 97 pgs: 97 active+clean; 1.5 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-23T10:39:03.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:39:02 smithi183 ceph-mon[32317]: pgmap v400: 97 pgs: 97 active+clean; 1.5 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-23T10:39:05.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:39:04 smithi196 ceph-mon[32919]: pgmap v401: 97 pgs: 97 active+clean; 1.7 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 61 KiB/s wr, 32 op/s 2024-04-23T10:39:05.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:39:04 smithi097 ceph-mon[29197]: pgmap v401: 97 pgs: 97 active+clean; 1.7 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 61 KiB/s wr, 32 op/s 2024-04-23T10:39:05.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:39:04 smithi183 ceph-mon[32317]: pgmap v401: 97 pgs: 97 active+clean; 1.7 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 61 KiB/s wr, 32 op/s 2024-04-23T10:39:07.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:39:06 smithi196 ceph-mon[32919]: pgmap v402: 97 pgs: 97 active+clean; 1.7 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 61 KiB/s wr, 28 op/s 2024-04-23T10:39:07.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:39:06 smithi097 ceph-mon[29197]: pgmap v402: 97 pgs: 97 active+clean; 1.7 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 61 KiB/s wr, 28 op/s 2024-04-23T10:39:07.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:39:06 smithi183 ceph-mon[32317]: pgmap v402: 97 pgs: 97 active+clean; 1.7 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 61 KiB/s wr, 28 op/s 2024-04-23T10:39:08.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:39:07 smithi196 ceph-mon[32919]: osdmap e102: 12 total, 12 up, 12 in 2024-04-23T10:39:08.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:39:07 smithi097 ceph-mon[29197]: osdmap e102: 12 total, 12 up, 12 in 2024-04-23T10:39:08.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:39:07 smithi183 ceph-mon[32317]: osdmap e102: 12 total, 12 up, 12 in 2024-04-23T10:39:09.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.050+0000 7fc3f96ef640 1 -- 172.21.15.97:0/217962583 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3f41021b0 msgr2=0x7fc3f4102590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:39:09.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.050+0000 7fc3f96ef640 1 --2- 172.21.15.97:0/217962583 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3f41021b0 0x7fc3f4102590 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4009ca0 tx=0x7fc3e402a8c0 comp rx=0 tx=0).stop 2024-04-23T10:39:09.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.050+0000 7fc3f96ef640 1 -- 172.21.15.97:0/217962583 shutdown_connections 2024-04-23T10:39:09.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.050+0000 7fc3f96ef640 1 --2- 172.21.15.97:0/217962583 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3f4107280 0x7fc3f410ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:09.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.050+0000 7fc3f96ef640 1 --2- 172.21.15.97:0/217962583 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3f4102b60 0x7fc3f4106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:09.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.050+0000 7fc3f96ef640 1 --2- 172.21.15.97:0/217962583 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3f41021b0 0x7fc3f4102590 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.050+0000 7fc3f96ef640 1 -- 172.21.15.97:0/217962583 >> 172.21.15.97:0/217962583 conn(0x7fc3f40fda10 msgr2=0x7fc3f40ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:39:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.050+0000 7fc3f96ef640 1 -- 172.21.15.97:0/217962583 shutdown_connections 2024-04-23T10:39:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.050+0000 7fc3f96ef640 1 -- 172.21.15.97:0/217962583 wait complete. 2024-04-23T10:39:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.050+0000 7fc3f96ef640 1 Processor -- start 2024-04-23T10:39:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.051+0000 7fc3f96ef640 1 -- start start 2024-04-23T10:39:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.051+0000 7fc3f96ef640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3f41021b0 0x7fc3f4198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.051+0000 7fc3f96ef640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3f4107280 0x7fc3f4198cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.051+0000 7fc3f96ef640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3f419cff0 0x7fc3f419d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.051+0000 7fc3f96ef640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f41121e0 con 0x7fc3f419cff0 2024-04-23T10:39:09.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.051+0000 7fc3f96ef640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f4112060 con 0x7fc3f4107280 2024-04-23T10:39:09.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.051+0000 7fc3f96ef640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f4112360 con 0x7fc3f41021b0 2024-04-23T10:39:09.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.051+0000 7fc3f37fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3f419cff0 0x7fc3f419d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:09.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.051+0000 7fc3f37fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3f419cff0 0x7fc3f419d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43954/0 (socket says 172.21.15.97:43954) 2024-04-23T10:39:09.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.051+0000 7fc3f37fe640 1 -- 172.21.15.97:0/473616222 learned_addr learned my addr 172.21.15.97:0/473616222 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:39:09.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.051+0000 7fc3f2ffd640 1 --2- 172.21.15.97:0/473616222 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3f41021b0 0x7fc3f4198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:09.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.051+0000 7fc3f27fc640 1 --2- 172.21.15.97:0/473616222 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3f4107280 0x7fc3f4198cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:09.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.051+0000 7fc3f37fe640 1 -- 172.21.15.97:0/473616222 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3f41021b0 msgr2=0x7fc3f4198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:39:09.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.051+0000 7fc3f37fe640 1 --2- 172.21.15.97:0/473616222 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3f41021b0 0x7fc3f4198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:09.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.051+0000 7fc3f37fe640 1 -- 172.21.15.97:0/473616222 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3f4107280 msgr2=0x7fc3f4198cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:39:09.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.051+0000 7fc3f37fe640 1 --2- 172.21.15.97:0/473616222 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3f4107280 0x7fc3f4198cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:09.055 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.051+0000 7fc3f37fe640 1 -- 172.21.15.97:0/473616222 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3f419dae0 con 0x7fc3f419cff0 2024-04-23T10:39:09.055 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.051+0000 7fc3f2ffd640 1 --2- 172.21.15.97:0/473616222 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3f41021b0 0x7fc3f4198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:39:09.055 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.051+0000 7fc3f37fe640 1 --2- 172.21.15.97:0/473616222 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3f419cff0 0x7fc3f419d4a0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec00d950 tx=0x7fc3ec00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:39:09.055 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.052+0000 7fc3dbfff640 1 -- 172.21.15.97:0/473616222 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec017030 con 0x7fc3f419cff0 2024-04-23T10:39:09.055 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.052+0000 7fc3dbfff640 1 -- 172.21.15.97:0/473616222 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3ec004020 con 0x7fc3f419cff0 2024-04-23T10:39:09.055 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.052+0000 7fc3f96ef640 1 -- 172.21.15.97:0/473616222 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3f419ddd0 con 0x7fc3f419cff0 2024-04-23T10:39:09.055 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.052+0000 7fc3dbfff640 1 -- 172.21.15.97:0/473616222 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec003b50 con 0x7fc3f419cff0 2024-04-23T10:39:09.055 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.052+0000 7fc3f96ef640 1 -- 172.21.15.97:0/473616222 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3f41a5570 con 0x7fc3f419cff0 2024-04-23T10:39:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.054+0000 7fc3dbfff640 1 -- 172.21.15.97:0/473616222 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc3ec01a070 con 0x7fc3f419cff0 2024-04-23T10:39:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.054+0000 7fc3fc409740 1 Processor -- start 2024-04-23T10:39:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.055+0000 7fc3fc409740 1 -- start start 2024-04-23T10:39:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.055+0000 7fc3dbfff640 1 --2- 172.21.15.97:0/473616222 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fc3dc0a3dc0 0x7fc3dc0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.055+0000 7fc3fc409740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55583682f620 0x55583682fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.055+0000 7fc3fc409740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55583682ff40 0x55583683a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.055+0000 7fc3fc409740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55583683a800 0x55583683cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.055+0000 7fc3fc409740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x555836794970 con 0x55583682f620 2024-04-23T10:39:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.055+0000 7fc3fc409740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5558367314a0 con 0x55583683a800 2024-04-23T10:39:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.055+0000 7fc3fc409740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x555836790930 con 0x55583682ff40 2024-04-23T10:39:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.055+0000 7fc3f27fc640 1 --2- 172.21.15.97:0/473616222 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fc3dc0a3dc0 0x7fc3dc0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.055+0000 7fc3f2ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55583682f620 0x55583682fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.055+0000 7fc3f2ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55583682f620 0x55583682fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43966/0 (socket says 172.21.15.97:43966) 2024-04-23T10:39:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.055+0000 7fc3f2ffd640 1 -- 172.21.15.97:0/766387499 learned_addr learned my addr 172.21.15.97:0/766387499 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:39:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.055+0000 7fc3dbfff640 1 -- 172.21.15.97:0/473616222 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8528+0+0 (secure 0 0 0) 0x7fc3ec0cff60 con 0x7fc3f419cff0 2024-04-23T10:39:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.055+0000 7fc3f27fc640 1 --2- 172.21.15.97:0/766387499 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55583682ff40 0x55583683a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.055+0000 7fc3f37fe640 1 --2- 172.21.15.97:0/766387499 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55583683a800 0x55583683cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.055+0000 7fc3f2ffd640 1 -- 172.21.15.97:0/766387499 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55583682ff40 msgr2=0x55583683a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:39:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.055+0000 7fc3f2ffd640 1 --2- 172.21.15.97:0/766387499 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55583682ff40 0x55583683a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.055+0000 7fc3f2ffd640 1 -- 172.21.15.97:0/766387499 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55583683a800 msgr2=0x55583683cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:39:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.055+0000 7fc3f2ffd640 1 --2- 172.21.15.97:0/766387499 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55583683a800 0x55583683cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.055+0000 7fc3f2ffd640 1 -- 172.21.15.97:0/766387499 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558364a7210 con 0x55583682f620 2024-04-23T10:39:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.055+0000 7fc3f27fc640 1 --2- 172.21.15.97:0/473616222 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fc3dc0a3dc0 0x7fc3dc0a6280 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc3f40fdf30 tx=0x7fc3e800be70 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:39:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.055+0000 7fc3f2ffd640 1 --2- 172.21.15.97:0/766387499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55583682f620 0x55583682fa00 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4035000 tx=0x7fc3e4009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8473e3e897222e1d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:39:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.056+0000 7fc3d97fa640 1 -- 172.21.15.97:0/766387499 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e4031cf0 con 0x55583682f620 2024-04-23T10:39:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.056+0000 7fc3d97fa640 1 -- 172.21.15.97:0/766387499 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3e4036040 con 0x55583682f620 2024-04-23T10:39:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.056+0000 7fc3d97fa640 1 -- 172.21.15.97:0/766387499 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e40042a0 con 0x55583682f620 2024-04-23T10:39:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.056+0000 7fc3fc409740 1 -- 172.21.15.97:0/766387499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55583682f620 msgr2=0x55583682fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:39:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.056+0000 7fc3fc409740 1 --2- 172.21.15.97:0/766387499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55583682f620 0x55583682fa00 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4035000 tx=0x7fc3e4009880 comp rx=0 tx=0).stop 2024-04-23T10:39:09.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.056+0000 7fc3fc409740 1 -- 172.21.15.97:0/766387499 shutdown_connections 2024-04-23T10:39:09.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.056+0000 7fc3fc409740 1 --2- 172.21.15.97:0/766387499 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55583683a800 0x55583683cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:09.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.056+0000 7fc3fc409740 1 --2- 172.21.15.97:0/766387499 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55583682ff40 0x55583683a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:09.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.056+0000 7fc3fc409740 1 --2- 172.21.15.97:0/766387499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55583682f620 0x55583682fa00 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:09.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.056+0000 7fc3fc409740 1 -- 172.21.15.97:0/766387499 >> 172.21.15.97:0/766387499 conn(0x55583682bb30 msgr2=0x55583682bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:39:09.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.056+0000 7fc3fc409740 1 -- 172.21.15.97:0/766387499 shutdown_connections 2024-04-23T10:39:09.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.056+0000 7fc3fc409740 1 -- 172.21.15.97:0/766387499 wait complete. 2024-04-23T10:39:09.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.057+0000 7fc3fc409740 1 -- start start 2024-04-23T10:39:09.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.057+0000 7fc3fc409740 1 Processor -- start 2024-04-23T10:39:09.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.057+0000 7fc3fc409740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55583682f620 0x55583683a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:09.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.057+0000 7fc3fc409740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55583682ff40 0x5558368373c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:09.069 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.057+0000 7fc3fc409740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55583683a800 0x555836837900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:09.069 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.057+0000 7fc3fc409740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55583683d3d0 con 0x55583682ff40 2024-04-23T10:39:09.069 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.057+0000 7fc3fc409740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55583672f0b0 con 0x55583682f620 2024-04-23T10:39:09.069 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.057+0000 7fc3fc409740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55583683d550 con 0x55583683a800 2024-04-23T10:39:09.069 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.057+0000 7fc3fc409740 10 client.0 _ll_register_callbacks cb 0x55583671cd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:39:09.069 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.057+0000 7fc3f27fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55583682ff40 0x5558368373c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:09.069 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.057+0000 7fc3f27fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55583682ff40 0x5558368373c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43976/0 (socket says 172.21.15.97:43976) 2024-04-23T10:39:09.069 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.057+0000 7fc3f27fc640 1 -- 172.21.15.97:0/1710432423 learned_addr learned my addr 172.21.15.97:0/1710432423 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:39:09.070 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.057+0000 7fc3f2ffd640 1 --2- 172.21.15.97:0/1710432423 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55583682f620 0x55583683a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:09.070 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.057+0000 7fc3f37fe640 1 --2- 172.21.15.97:0/1710432423 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55583683a800 0x555836837900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:09.070 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.057+0000 7fc3f27fc640 1 -- 172.21.15.97:0/1710432423 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55583683a800 msgr2=0x555836837900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:39:09.070 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.057+0000 7fc3f27fc640 1 --2- 172.21.15.97:0/1710432423 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55583683a800 0x555836837900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:09.070 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.057+0000 7fc3f27fc640 1 -- 172.21.15.97:0/1710432423 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55583682f620 msgr2=0x55583683a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:39:09.070 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.057+0000 7fc3f27fc640 1 --2- 172.21.15.97:0/1710432423 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55583682f620 0x55583683a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:09.070 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.057+0000 7fc3f27fc640 1 -- 172.21.15.97:0/1710432423 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555836573620 con 0x55583682ff40 2024-04-23T10:39:09.070 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.057+0000 7fc3f37fe640 1 --2- 172.21.15.97:0/1710432423 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55583683a800 0x555836837900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:39:09.070 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.058+0000 7fc3f27fc640 1 --2- 172.21.15.97:0/1710432423 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55583682ff40 0x5558368373c0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8012470 tx=0x7fc3e8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5d1dd8b930b86a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:39:09.070 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.058+0000 7fc3d6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:39:09.071 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.058+0000 7fc3d6ff5640 1 -- 172.21.15.97:0/1710432423 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e8005220 con 0x55583682ff40 2024-04-23T10:39:09.071 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.058+0000 7fc3d6ff5640 1 -- 172.21.15.97:0/1710432423 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8528+0+0 (secure 0 0 0) 0x7fc3e80053c0 con 0x55583682ff40 2024-04-23T10:39:09.071 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.058+0000 7fc3fc409740 1 -- 172.21.15.97:0/1710432423 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3e4036040 con 0x55583682ff40 2024-04-23T10:39:09.071 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.058+0000 7fc3d6ff5640 1 -- 172.21.15.97:0/1710432423 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc3e80055c0 con 0x55583682ff40 2024-04-23T10:39:09.071 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.058+0000 7fc3fc409740 10 client.17415 fetch_fsmap learned FSMap version 26 2024-04-23T10:39:09.071 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.058+0000 7fc3fc409740 1 -- 172.21.15.97:0/1710432423 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555836849270 con 0x55583682ff40 2024-04-23T10:39:09.071 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.058+0000 7fc3d6ff5640 1 -- 172.21.15.97:0/1710432423 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3e4036040 con 0x55583682ff40 2024-04-23T10:39:09.071 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.058+0000 7fc3d6ff5640 1 -- 172.21.15.97:0/1710432423 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc3e801fb80 con 0x55583682ff40 2024-04-23T10:39:09.071 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.058+0000 7fc3fc409740 10 client.17415 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:39:09.071 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.058+0000 7fc3fc409740 10 client.17415 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:39:09.071 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.058+0000 7fc3fc409740 20 client.17415 populate_metadata read hostname 'smithi097' 2024-04-23T10:39:09.072 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.058+0000 7fc3fc409740 1 --2- 172.21.15.97:0/1710432423 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55583684a730 0x55583684cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:09.072 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.058+0000 7fc3fc409740 4 client.17415 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:39:09.072 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.058+0000 7fc3fc409740 1 -- 172.21.15.97:0/1710432423 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5558367a2390 con 0x55583684a730 2024-04-23T10:39:09.072 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.059+0000 7fc3f2ffd640 1 --2- 172.21.15.97:0/1710432423 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55583684a730 0x55583684cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:09.072 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.059+0000 7fc3d6ff5640 10 client.17415 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:39:09.072 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.059+0000 7fc3f2ffd640 1 --2- 172.21.15.97:0/1710432423 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55583684a730 0x55583684cb40 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef297a075b72e420 server_cookie=ee66e48910b3f147 in_seq=0 out_seq=0 2024-04-23T10:39:09.072 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.060+0000 7fc3d6ff5640 1 -- 172.21.15.97:0/1710432423 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5558367a2390 con 0x55583684a730 2024-04-23T10:39:09.072 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.060+0000 7fc3d6ff5640 10 client.17415 handle_command_reply: tid=0 2024-04-23T10:39:09.072 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.060+0000 7fc3fc409740 1 client.17415 shutdown 2024-04-23T10:39:09.072 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.060+0000 7fc3fc409740 1 -- 172.21.15.97:0/1710432423 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55583682ff40 msgr2=0x5558368373c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:39:09.072 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.060+0000 7fc3fc409740 1 --2- 172.21.15.97:0/1710432423 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55583682ff40 0x5558368373c0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8012470 tx=0x7fc3e8004730 comp rx=0 tx=0).stop 2024-04-23T10:39:09.073 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.060+0000 7fc3fc409740 1 -- 172.21.15.97:0/1710432423 shutdown_connections 2024-04-23T10:39:09.073 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.060+0000 7fc3fc409740 1 --2- 172.21.15.97:0/1710432423 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55583684a730 0x55583684cb40 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:09.073 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.060+0000 7fc3fc409740 1 --2- 172.21.15.97:0/1710432423 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55583683a800 0x555836837900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:09.073 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.060+0000 7fc3d6ff5640 0 client.17415 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:39:09.073 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.060+0000 7fc3fc409740 1 --2- 172.21.15.97:0/1710432423 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55583682ff40 0x5558368373c0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:09.073 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.060+0000 7fc3fc409740 1 --2- 172.21.15.97:0/1710432423 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55583682f620 0x55583683a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:09.073 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.060+0000 7fc3fc409740 1 -- 172.21.15.97:0/1710432423 >> 172.21.15.97:0/1710432423 conn(0x55583682bb30 msgr2=0x555836831860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:39:09.073 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.060+0000 7fc3fc409740 1 -- 172.21.15.97:0/1710432423 shutdown_connections 2024-04-23T10:39:09.073 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.060+0000 7fc3fc409740 1 -- 172.21.15.97:0/1710432423 wait complete. 2024-04-23T10:39:09.073 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.060+0000 7fc3fc409740 20 client.17415 trim_cache size 0 max 16384 2024-04-23T10:39:09.079 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.075+0000 7fc3fc409740 1 Processor -- start 2024-04-23T10:39:09.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.075+0000 7fc3fc409740 1 -- start start 2024-04-23T10:39:09.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.075+0000 7fc3fc409740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55583682bb30 0x555836846290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:09.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.075+0000 7fc3fc409740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55583682f620 0x555836852450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:09.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.075+0000 7fc3fc409740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55583682ff40 0x555836852990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:09.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.076+0000 7fc3fc409740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f41adb40 con 0x55583682ff40 2024-04-23T10:39:09.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.076+0000 7fc3fc409740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f41af680 con 0x55583682f620 2024-04-23T10:39:09.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.076+0000 7fc3fc409740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f41ae210 con 0x55583682bb30 2024-04-23T10:39:09.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.076+0000 7fc3f37fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55583682ff40 0x555836852990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:09.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.076+0000 7fc3f37fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55583682ff40 0x555836852990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43988/0 (socket says 172.21.15.97:43988) 2024-04-23T10:39:09.081 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.076+0000 7fc3f37fe640 1 -- 172.21.15.97:0/1684720477 learned_addr learned my addr 172.21.15.97:0/1684720477 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:39:09.081 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.076+0000 7fc3f2ffd640 1 --2- 172.21.15.97:0/1684720477 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55583682bb30 0x555836846290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:09.081 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.076+0000 7fc3f27fc640 1 --2- 172.21.15.97:0/1684720477 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55583682f620 0x555836852450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:09.081 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.076+0000 7fc3f37fe640 1 -- 172.21.15.97:0/1684720477 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55583682bb30 msgr2=0x555836846290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:39:09.081 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.076+0000 7fc3f37fe640 1 --2- 172.21.15.97:0/1684720477 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55583682bb30 0x555836846290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:09.081 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.076+0000 7fc3f37fe640 1 -- 172.21.15.97:0/1684720477 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55583682f620 msgr2=0x555836852450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:39:09.081 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.076+0000 7fc3f37fe640 1 --2- 172.21.15.97:0/1684720477 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55583682f620 0x555836852450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:09.081 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.076+0000 7fc3f37fe640 1 -- 172.21.15.97:0/1684720477 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55583683cf10 con 0x55583682ff40 2024-04-23T10:39:09.081 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.076+0000 7fc3f2ffd640 1 --2- 172.21.15.97:0/1684720477 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55583682bb30 0x555836846290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:39:09.081 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.076+0000 7fc3f27fc640 1 --2- 172.21.15.97:0/1684720477 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55583682f620 0x555836852450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:39:09.082 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.076+0000 7fc3f37fe640 1 --2- 172.21.15.97:0/1684720477 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55583682ff40 0x555836852990 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec0cd710 tx=0x7fc3ec09c600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2a694c5424b8a91 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:39:09.082 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.076+0000 7fc3d8ff9640 1 -- 172.21.15.97:0/1684720477 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec09bf00 con 0x55583682ff40 2024-04-23T10:39:09.082 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.076+0000 7fc3d8ff9640 1 -- 172.21.15.97:0/1684720477 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3ec0a02b0 con 0x55583682ff40 2024-04-23T10:39:09.082 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.076+0000 7fc3d8ff9640 1 -- 172.21.15.97:0/1684720477 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec098570 con 0x55583682ff40 2024-04-23T10:39:09.082 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.077+0000 7fc3fc409740 1 -- 172.21.15.97:0/1684720477 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55583682ff40 msgr2=0x555836852990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:39:09.082 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.077+0000 7fc3fc409740 1 --2- 172.21.15.97:0/1684720477 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55583682ff40 0x555836852990 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec0cd710 tx=0x7fc3ec09c600 comp rx=0 tx=0).stop 2024-04-23T10:39:09.082 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.077+0000 7fc3fc409740 1 -- 172.21.15.97:0/1684720477 shutdown_connections 2024-04-23T10:39:09.082 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.077+0000 7fc3fc409740 1 --2- 172.21.15.97:0/1684720477 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55583682ff40 0x555836852990 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:09.082 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.077+0000 7fc3fc409740 1 --2- 172.21.15.97:0/1684720477 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55583682f620 0x555836852450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:09.082 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.077+0000 7fc3fc409740 1 --2- 172.21.15.97:0/1684720477 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55583682bb30 0x555836846290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:09.082 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.077+0000 7fc3fc409740 1 -- 172.21.15.97:0/1684720477 >> 172.21.15.97:0/1684720477 conn(0x555836837a60 msgr2=0x55583682a420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:39:09.083 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.077+0000 7fc3fc409740 1 -- 172.21.15.97:0/1684720477 shutdown_connections 2024-04-23T10:39:09.083 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.077+0000 7fc3fc409740 1 -- 172.21.15.97:0/1684720477 wait complete. 2024-04-23T10:39:09.083 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.077+0000 7fc3fc409740 1 -- start start 2024-04-23T10:39:09.083 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.077+0000 7fc3fc409740 1 Processor -- start 2024-04-23T10:39:09.083 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.077+0000 7fc3fc409740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55583682bb30 0x555836842b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:09.083 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.077+0000 7fc3fc409740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55583682f620 0x55583683feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:09.083 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.077+0000 7fc3fc409740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55583682ff40 0x5558368403f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:09.083 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.077+0000 7fc3fc409740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f41aefb0 con 0x55583682f620 2024-04-23T10:39:09.083 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.077+0000 7fc3fc409740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f41ae8e0 con 0x55583682ff40 2024-04-23T10:39:09.083 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.077+0000 7fc3fc409740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f41abb20 con 0x55583682bb30 2024-04-23T10:39:09.083 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.077+0000 7fc3fc409740 10 client.0 _ll_register_callbacks cb 0x7fc3e8003d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:39:09.083 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.078+0000 7fc3f27fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55583682f620 0x55583683feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:09.084 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.078+0000 7fc3f27fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55583682f620 0x55583683feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43990/0 (socket says 172.21.15.97:43990) 2024-04-23T10:39:09.084 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.078+0000 7fc3f27fc640 1 -- 172.21.15.97:0/164389323 learned_addr learned my addr 172.21.15.97:0/164389323 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:39:09.084 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.078+0000 7fc3f37fe640 1 --2- 172.21.15.97:0/164389323 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55583682ff40 0x5558368403f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:09.084 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.078+0000 7fc3f2ffd640 1 --2- 172.21.15.97:0/164389323 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55583682bb30 0x555836842b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:09.084 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.078+0000 7fc3f27fc640 1 -- 172.21.15.97:0/164389323 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55583682bb30 msgr2=0x555836842b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:39:09.084 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.078+0000 7fc3f27fc640 1 --2- 172.21.15.97:0/164389323 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55583682bb30 0x555836842b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:09.084 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.078+0000 7fc3f27fc640 1 -- 172.21.15.97:0/164389323 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55583682ff40 msgr2=0x5558368403f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:39:09.084 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.078+0000 7fc3f27fc640 1 --2- 172.21.15.97:0/164389323 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55583682ff40 0x5558368403f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:09.084 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.078+0000 7fc3f27fc640 1 -- 172.21.15.97:0/164389323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555836840bf0 con 0x55583682f620 2024-04-23T10:39:09.084 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.078+0000 7fc3f37fe640 1 --2- 172.21.15.97:0/164389323 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55583682ff40 0x5558368403f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:39:09.084 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.078+0000 7fc3f2ffd640 1 --2- 172.21.15.97:0/164389323 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55583682bb30 0x555836842b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:39:09.085 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.078+0000 7fc3f27fc640 1 --2- 172.21.15.97:0/164389323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55583682f620 0x55583683feb0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8006010 tx=0x7fc3e801f9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bf1ede900adb0ba server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:39:09.085 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.078+0000 7fc3d6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:39:09.085 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.078+0000 7fc3d6ff5640 1 -- 172.21.15.97:0/164389323 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e8012d00 con 0x55583682f620 2024-04-23T10:39:09.085 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.078+0000 7fc3d6ff5640 1 -- 172.21.15.97:0/164389323 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8528+0+0 (secure 0 0 0) 0x7fc3e8030480 con 0x55583682f620 2024-04-23T10:39:09.085 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.078+0000 7fc3fc409740 1 -- 172.21.15.97:0/164389323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3ec0a02b0 con 0x55583682f620 2024-04-23T10:39:09.085 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.078+0000 7fc3d6ff5640 1 -- 172.21.15.97:0/164389323 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc3e8003a10 con 0x55583682f620 2024-04-23T10:39:09.085 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.078+0000 7fc3fc409740 10 client.17421 fetch_fsmap learned FSMap version 26 2024-04-23T10:39:09.085 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.078+0000 7fc3fc409740 1 -- 172.21.15.97:0/164389323 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5558368b5030 con 0x55583682f620 2024-04-23T10:39:09.085 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.079+0000 7fc3d6ff5640 1 -- 172.21.15.97:0/164389323 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3ec0a02b0 con 0x55583682f620 2024-04-23T10:39:09.085 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.079+0000 7fc3d6ff5640 1 -- 172.21.15.97:0/164389323 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc3e8030b30 con 0x55583682f620 2024-04-23T10:39:09.085 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.079+0000 7fc3fc409740 10 client.17421 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:39:09.085 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.079+0000 7fc3fc409740 10 client.17421 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:39:09.085 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.079+0000 7fc3fc409740 20 client.17421 populate_metadata read hostname 'smithi097' 2024-04-23T10:39:09.086 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.079+0000 7fc3fc409740 1 --2- 172.21.15.97:0/164389323 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55583683a800 0x5558368b7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:09.086 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.079+0000 7fc3fc409740 4 client.17421 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:39:09.086 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.079+0000 7fc3fc409740 1 -- 172.21.15.97:0/164389323 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55583679c200 con 0x55583683a800 2024-04-23T10:39:09.086 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.079+0000 7fc3f2ffd640 1 --2- 172.21.15.97:0/164389323 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55583683a800 0x5558368b7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:09.086 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.079+0000 7fc3d6ff5640 10 client.17421 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:39:09.086 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.079+0000 7fc3f2ffd640 1 --2- 172.21.15.97:0/164389323 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55583683a800 0x5558368b7730 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c2a7cd08227765f server_cookie=7cc83f3c8a40053a in_seq=0 out_seq=0 2024-04-23T10:39:09.090 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.091+0000 7fc3d6ff5640 1 -- 172.21.15.97:0/164389323 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55583679c200 con 0x55583683a800 2024-04-23T10:39:09.091 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.091+0000 7fc3d6ff5640 10 client.17421 handle_command_reply: tid=0 2024-04-23T10:39:09.091 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.091+0000 7fc3fc409740 1 client.17421 shutdown 2024-04-23T10:39:09.091 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.091+0000 7fc3fc409740 1 -- 172.21.15.97:0/164389323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55583682f620 msgr2=0x55583683feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:39:09.091 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.091+0000 7fc3fc409740 1 --2- 172.21.15.97:0/164389323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55583682f620 0x55583683feb0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8006010 tx=0x7fc3e801f9f0 comp rx=0 tx=0).stop 2024-04-23T10:39:09.091 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.091+0000 7fc3fc409740 1 -- 172.21.15.97:0/164389323 shutdown_connections 2024-04-23T10:39:09.091 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.091+0000 7fc3fc409740 1 --2- 172.21.15.97:0/164389323 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55583683a800 0x5558368b7730 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:09.091 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.091+0000 7fc3fc409740 1 --2- 172.21.15.97:0/164389323 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55583682ff40 0x5558368403f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:09.091 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.091+0000 7fc3fc409740 1 --2- 172.21.15.97:0/164389323 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55583682f620 0x55583683feb0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:09.091 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.091+0000 7fc3d6ff5640 0 client.17421 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:39:09.091 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.091+0000 7fc3fc409740 1 --2- 172.21.15.97:0/164389323 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55583682bb30 0x555836842b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:09.092 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.091+0000 7fc3fc409740 1 -- 172.21.15.97:0/164389323 >> 172.21.15.97:0/164389323 conn(0x555836837a60 msgr2=0x55583682acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:39:09.092 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.091+0000 7fc3fc409740 1 -- 172.21.15.97:0/164389323 shutdown_connections 2024-04-23T10:39:09.092 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.091+0000 7fc3fc409740 1 -- 172.21.15.97:0/164389323 wait complete. 2024-04-23T10:39:09.092 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.091+0000 7fc3fc409740 20 client.17421 trim_cache size 0 max 16384 2024-04-23T10:39:09.092 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:39:09.092 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:39:09.092 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:39:09.092 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:39:09.093 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.092+0000 7fc3f96ef640 1 -- 172.21.15.97:0/473616222 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fc3dc0a3dc0 msgr2=0x7fc3dc0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:39:09.093 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.092+0000 7fc3f96ef640 1 --2- 172.21.15.97:0/473616222 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fc3dc0a3dc0 0x7fc3dc0a6280 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc3f40fdf30 tx=0x7fc3e800be70 comp rx=0 tx=0).stop 2024-04-23T10:39:09.093 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.092+0000 7fc3f96ef640 1 -- 172.21.15.97:0/473616222 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3f419cff0 msgr2=0x7fc3f419d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:39:09.094 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.092+0000 7fc3f96ef640 1 --2- 172.21.15.97:0/473616222 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3f419cff0 0x7fc3f419d4a0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec00d950 tx=0x7fc3ec00de10 comp rx=0 tx=0).stop 2024-04-23T10:39:09.094 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.092+0000 7fc3f96ef640 1 -- 172.21.15.97:0/473616222 shutdown_connections 2024-04-23T10:39:09.094 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.092+0000 7fc3f96ef640 1 --2- 172.21.15.97:0/473616222 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fc3dc0a3dc0 0x7fc3dc0a6280 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:09.094 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.092+0000 7fc3f96ef640 1 --2- 172.21.15.97:0/473616222 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3f419cff0 0x7fc3f419d4a0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:09.094 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.092+0000 7fc3f96ef640 1 --2- 172.21.15.97:0/473616222 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3f4107280 0x7fc3f4198cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:09.094 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.092+0000 7fc3f96ef640 1 --2- 172.21.15.97:0/473616222 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3f41021b0 0x7fc3f4198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:09.094 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.092+0000 7fc3f96ef640 1 -- 172.21.15.97:0/473616222 >> 172.21.15.97:0/473616222 conn(0x7fc3f40fda10 msgr2=0x7fc3f40feda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:39:09.094 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.092+0000 7fc3f96ef640 1 -- 172.21.15.97:0/473616222 shutdown_connections 2024-04-23T10:39:09.094 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:09.092+0000 7fc3f96ef640 1 -- 172.21.15.97:0/473616222 wait complete. 2024-04-23T10:39:09.095 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:39:08 smithi196 ceph-mon[32919]: pgmap v404: 97 pgs: 97 active+clean; 1.9 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 92 KiB/s wr, 35 op/s 2024-04-23T10:39:09.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:39:08 smithi097 ceph-mon[29197]: pgmap v404: 97 pgs: 97 active+clean; 1.9 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 92 KiB/s wr, 35 op/s 2024-04-23T10:39:09.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:39:08 smithi183 ceph-mon[32317]: pgmap v404: 97 pgs: 97 active+clean; 1.9 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 92 KiB/s wr, 35 op/s 2024-04-23T10:39:11.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:39:10 smithi196 ceph-mon[32919]: pgmap v405: 97 pgs: 97 active+clean; 1.9 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 92 KiB/s wr, 35 op/s 2024-04-23T10:39:11.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:39:10 smithi097 ceph-mon[29197]: pgmap v405: 97 pgs: 97 active+clean; 1.9 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 92 KiB/s wr, 35 op/s 2024-04-23T10:39:11.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:39:10 smithi183 ceph-mon[32317]: pgmap v405: 97 pgs: 97 active+clean; 1.9 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 92 KiB/s wr, 35 op/s 2024-04-23T10:39:12.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:39:12 smithi183 ceph-mon[32317]: osdmap e103: 12 total, 12 up, 12 in 2024-04-23T10:39:12.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:39:12 smithi196 ceph-mon[32919]: osdmap e103: 12 total, 12 up, 12 in 2024-04-23T10:39:12.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:39:12 smithi097 ceph-mon[29197]: osdmap e103: 12 total, 12 up, 12 in 2024-04-23T10:39:13.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:39:13 smithi196 ceph-mon[32919]: pgmap v407: 97 pgs: 97 active+clean; 1.9 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 26 KiB/s wr, 9 op/s 2024-04-23T10:39:13.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:39:13 smithi097 ceph-mon[29197]: pgmap v407: 97 pgs: 97 active+clean; 1.9 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 26 KiB/s wr, 9 op/s 2024-04-23T10:39:13.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:39:13 smithi183 ceph-mon[32317]: pgmap v407: 97 pgs: 97 active+clean; 1.9 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 26 KiB/s wr, 9 op/s 2024-04-23T10:39:15.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:39:15 smithi196 ceph-mon[32919]: pgmap v408: 97 pgs: 97 active+clean; 2.1 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-23T10:39:15.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:39:15 smithi097 ceph-mon[29197]: pgmap v408: 97 pgs: 97 active+clean; 2.1 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-23T10:39:15.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:39:15 smithi183 ceph-mon[32317]: pgmap v408: 97 pgs: 97 active+clean; 2.1 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-23T10:39:17.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:39:17 smithi196 ceph-mon[32919]: pgmap v409: 97 pgs: 97 active+clean; 2.1 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 54 KiB/s wr, 21 op/s 2024-04-23T10:39:17.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:39:17 smithi097 ceph-mon[29197]: pgmap v409: 97 pgs: 97 active+clean; 2.1 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 54 KiB/s wr, 21 op/s 2024-04-23T10:39:17.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:39:17 smithi183 ceph-mon[32317]: pgmap v409: 97 pgs: 97 active+clean; 2.1 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 54 KiB/s wr, 21 op/s 2024-04-23T10:39:19.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:39:19 smithi196 ceph-mon[32919]: pgmap v410: 97 pgs: 97 active+clean; 2.3 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 48 KiB/s wr, 18 op/s 2024-04-23T10:39:19.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:39:19 smithi097 ceph-mon[29197]: pgmap v410: 97 pgs: 97 active+clean; 2.3 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 48 KiB/s wr, 18 op/s 2024-04-23T10:39:19.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:39:19 smithi183 ceph-mon[32317]: pgmap v410: 97 pgs: 97 active+clean; 2.3 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 48 KiB/s wr, 18 op/s 2024-04-23T10:39:21.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:39:21 smithi196 ceph-mon[32919]: pgmap v411: 97 pgs: 97 active+clean; 2.3 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 48 KiB/s wr, 18 op/s 2024-04-23T10:39:21.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:39:21 smithi097 ceph-mon[29197]: pgmap v411: 97 pgs: 97 active+clean; 2.3 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 48 KiB/s wr, 18 op/s 2024-04-23T10:39:21.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:39:21 smithi183 ceph-mon[32317]: pgmap v411: 97 pgs: 97 active+clean; 2.3 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 48 KiB/s wr, 18 op/s 2024-04-23T10:39:22.187 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.188+0000 7fada0b44640 1 -- 172.21.15.97:0/1331080155 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad9c101a90 msgr2=0x7fad9c105ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:39:22.188 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.188+0000 7fada0b44640 1 --2- 172.21.15.97:0/1331080155 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad9c101a90 0x7fad9c105ad0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fad90009ca0 tx=0x7fad9002a8c0 comp rx=0 tx=0).stop 2024-04-23T10:39:22.188 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.188+0000 7fada0b44640 1 -- 172.21.15.97:0/1331080155 shutdown_connections 2024-04-23T10:39:22.188 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.188+0000 7fada0b44640 1 --2- 172.21.15.97:0/1331080155 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad9c1061b0 0x7fad9c10dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:22.188 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.188+0000 7fada0b44640 1 --2- 172.21.15.97:0/1331080155 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad9c101a90 0x7fad9c105ad0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:22.188 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.188+0000 7fada0b44640 1 --2- 172.21.15.97:0/1331080155 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad9c1010e0 0x7fad9c1014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:22.188 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.188+0000 7fada0b44640 1 -- 172.21.15.97:0/1331080155 >> 172.21.15.97:0/1331080155 conn(0x7fad9c0771d0 msgr2=0x7fad9c0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:39:22.188 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.188+0000 7fada0b44640 1 -- 172.21.15.97:0/1331080155 shutdown_connections 2024-04-23T10:39:22.188 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.188+0000 7fada0b44640 1 -- 172.21.15.97:0/1331080155 wait complete. 2024-04-23T10:39:22.190 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.189+0000 7fada0b44640 1 Processor -- start 2024-04-23T10:39:22.190 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.189+0000 7fada0b44640 1 -- start start 2024-04-23T10:39:22.190 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.189+0000 7fada0b44640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad9c1010e0 0x7fad9c198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:22.190 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.189+0000 7fada0b44640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad9c1061b0 0x7fad9c198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:22.190 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.189+0000 7fada0b44640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad9c19cf30 0x7fad9c19d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:22.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.189+0000 7fada0b44640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fad9c112230 con 0x7fad9c1010e0 2024-04-23T10:39:22.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.189+0000 7fada0b44640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fad9c1120b0 con 0x7fad9c19cf30 2024-04-23T10:39:22.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.189+0000 7fada0b44640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fad9c1123b0 con 0x7fad9c1061b0 2024-04-23T10:39:22.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.190+0000 7fad9a59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad9c1010e0 0x7fad9c198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:22.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.190+0000 7fad9a59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad9c1010e0 0x7fad9c198670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33634/0 (socket says 172.21.15.97:33634) 2024-04-23T10:39:22.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.190+0000 7fad9a59c640 1 -- 172.21.15.97:0/3882038961 learned_addr learned my addr 172.21.15.97:0/3882038961 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:39:22.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.190+0000 7fad99d9b640 1 --2- 172.21.15.97:0/3882038961 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad9c1061b0 0x7fad9c198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:22.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.190+0000 7fad9ad9d640 1 --2- 172.21.15.97:0/3882038961 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad9c19cf30 0x7fad9c19d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:22.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.190+0000 7fad9a59c640 1 -- 172.21.15.97:0/3882038961 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad9c1061b0 msgr2=0x7fad9c198bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:39:22.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.190+0000 7fad9a59c640 1 --2- 172.21.15.97:0/3882038961 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad9c1061b0 0x7fad9c198bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:22.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.190+0000 7fad9a59c640 1 -- 172.21.15.97:0/3882038961 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad9c19cf30 msgr2=0x7fad9c19d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:39:22.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.190+0000 7fad9a59c640 1 --2- 172.21.15.97:0/3882038961 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad9c19cf30 0x7fad9c19d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:22.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.190+0000 7fad9a59c640 1 -- 172.21.15.97:0/3882038961 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad9c19da20 con 0x7fad9c1010e0 2024-04-23T10:39:22.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.190+0000 7fad9ad9d640 1 --2- 172.21.15.97:0/3882038961 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad9c19cf30 0x7fad9c19d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:39:22.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.190+0000 7fad9a59c640 1 --2- 172.21.15.97:0/3882038961 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad9c1010e0 0x7fad9c198670 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fad8c00d950 tx=0x7fad8c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:39:22.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.190+0000 7fad8b7fe640 1 -- 172.21.15.97:0/3882038961 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad8c017070 con 0x7fad9c1010e0 2024-04-23T10:39:22.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.190+0000 7fada0b44640 1 -- 172.21.15.97:0/3882038961 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad9c19dd10 con 0x7fad9c1010e0 2024-04-23T10:39:22.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.190+0000 7fad8b7fe640 1 -- 172.21.15.97:0/3882038961 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad8c009c30 con 0x7fad9c1010e0 2024-04-23T10:39:22.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.190+0000 7fad8b7fe640 1 -- 172.21.15.97:0/3882038961 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad8c00f620 con 0x7fad9c1010e0 2024-04-23T10:39:22.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.190+0000 7fada0b44640 1 -- 172.21.15.97:0/3882038961 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad9c1a5550 con 0x7fad9c1010e0 2024-04-23T10:39:22.198 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.192+0000 7fad8b7fe640 1 -- 172.21.15.97:0/3882038961 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fad8c010460 con 0x7fad9c1010e0 2024-04-23T10:39:22.198 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.193+0000 7fada3869740 1 Processor -- start 2024-04-23T10:39:22.198 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.193+0000 7fada3869740 1 -- start start 2024-04-23T10:39:22.198 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.193+0000 7fada3869740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558c6a59a620 0x558c6a59aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:22.198 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.193+0000 7fada3869740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c6a59af40 0x558c6a5a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:22.198 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.193+0000 7fada3869740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c6a5a5800 0x558c6a5a7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:22.199 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.193+0000 7fada3869740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x558c6a4ff970 con 0x558c6a59af40 2024-04-23T10:39:22.199 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.193+0000 7fada3869740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558c6a49c4a0 con 0x558c6a59a620 2024-04-23T10:39:22.199 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.193+0000 7fada3869740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558c6a4fb930 con 0x558c6a5a5800 2024-04-23T10:39:22.199 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.194+0000 7fad8b7fe640 1 --2- 172.21.15.97:0/3882038961 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fad840a3dc0 0x7fad840a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:22.199 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.194+0000 7fad9a59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c6a59af40 0x558c6a5a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:22.199 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.194+0000 7fad9a59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c6a59af40 0x558c6a5a52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33640/0 (socket says 172.21.15.97:33640) 2024-04-23T10:39:22.199 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.194+0000 7fad9a59c640 1 -- 172.21.15.97:0/413620468 learned_addr learned my addr 172.21.15.97:0/413620468 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:39:22.199 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.194+0000 7fad9ad9d640 1 --2- 172.21.15.97:0/3882038961 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fad840a3dc0 0x7fad840a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:22.199 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.194+0000 7fad99d9b640 1 --2- 172.21.15.97:0/413620468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c6a5a5800 0x558c6a5a7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:22.199 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.194+0000 7fad9ad9d640 1 --2- 172.21.15.97:0/413620468 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558c6a59a620 0x558c6a59aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:22.199 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.194+0000 7fad8b7fe640 1 -- 172.21.15.97:0/3882038961 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8484+0+0 (secure 0 0 0) 0x7fad8c0d0530 con 0x7fad9c1010e0 2024-04-23T10:39:22.199 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.194+0000 7fad9a59c640 1 -- 172.21.15.97:0/413620468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c6a5a5800 msgr2=0x558c6a5a7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:39:22.199 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.194+0000 7fad9a59c640 1 --2- 172.21.15.97:0/413620468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c6a5a5800 0x558c6a5a7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:22.199 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.194+0000 7fad9a59c640 1 -- 172.21.15.97:0/413620468 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558c6a59a620 msgr2=0x558c6a59aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:39:22.199 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.194+0000 7fad9a59c640 1 --2- 172.21.15.97:0/413620468 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558c6a59a620 0x558c6a59aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:22.200 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.194+0000 7fad9a59c640 1 -- 172.21.15.97:0/413620468 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c6a212210 con 0x558c6a59af40 2024-04-23T10:39:22.200 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.194+0000 7fad99d9b640 1 --2- 172.21.15.97:0/413620468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c6a5a5800 0x558c6a5a7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T10:39:22.200 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.194+0000 7fad9ad9d640 1 --2- 172.21.15.97:0/3882038961 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fad840a3dc0 0x7fad840a6280 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fad94009cc0 tx=0x7fad940098d0 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:39:22.200 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.194+0000 7fad9a59c640 1 --2- 172.21.15.97:0/413620468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c6a59af40 0x558c6a5a52c0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fad8c000c00 tx=0x7fad8c01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53b4a4a2a2781aee server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:39:22.200 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.194+0000 7fad88ff9640 1 -- 172.21.15.97:0/413620468 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad8c020070 con 0x558c6a59af40 2024-04-23T10:39:22.200 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.194+0000 7fad88ff9640 1 -- 172.21.15.97:0/413620468 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad8c09c490 con 0x558c6a59af40 2024-04-23T10:39:22.200 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.195+0000 7fad88ff9640 1 -- 172.21.15.97:0/413620468 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad8c0981e0 con 0x558c6a59af40 2024-04-23T10:39:22.200 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.195+0000 7fada3869740 1 -- 172.21.15.97:0/413620468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c6a59af40 msgr2=0x558c6a5a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:39:22.200 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.195+0000 7fada3869740 1 --2- 172.21.15.97:0/413620468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c6a59af40 0x558c6a5a52c0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fad8c000c00 tx=0x7fad8c01e000 comp rx=0 tx=0).stop 2024-04-23T10:39:22.200 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.195+0000 7fada3869740 1 -- 172.21.15.97:0/413620468 shutdown_connections 2024-04-23T10:39:22.200 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.195+0000 7fada3869740 1 --2- 172.21.15.97:0/413620468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c6a5a5800 0x558c6a5a7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:22.200 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.195+0000 7fada3869740 1 --2- 172.21.15.97:0/413620468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c6a59af40 0x558c6a5a52c0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:22.200 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.195+0000 7fada3869740 1 --2- 172.21.15.97:0/413620468 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558c6a59a620 0x558c6a59aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:22.200 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.195+0000 7fada3869740 1 -- 172.21.15.97:0/413620468 >> 172.21.15.97:0/413620468 conn(0x558c6a596b30 msgr2=0x558c6a596f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:39:22.200 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.195+0000 7fada3869740 1 -- 172.21.15.97:0/413620468 shutdown_connections 2024-04-23T10:39:22.201 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.195+0000 7fada3869740 1 -- 172.21.15.97:0/413620468 wait complete. 2024-04-23T10:39:22.201 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.196+0000 7fada3869740 1 -- start start 2024-04-23T10:39:22.201 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.196+0000 7fada3869740 1 Processor -- start 2024-04-23T10:39:22.201 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.196+0000 7fada3869740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c6a59a620 0x558c6a5a5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:22.201 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.196+0000 7fada3869740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c6a59af40 0x558c6a5a2420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:22.201 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.196+0000 7fada3869740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558c6a5a5800 0x558c6a5a2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:22.201 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.196+0000 7fada3869740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x558c6a5a8370 con 0x558c6a59a620 2024-04-23T10:39:22.201 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.196+0000 7fada3869740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558c6a49a0b0 con 0x558c6a5a5800 2024-04-23T10:39:22.201 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.196+0000 7fada3869740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x558c6a5a84f0 con 0x558c6a59af40 2024-04-23T10:39:22.201 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.196+0000 7fad9a59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c6a59a620 0x558c6a5a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:22.201 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.196+0000 7fada3869740 10 client.0 _ll_register_callbacks cb 0x558c6a487d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:39:22.201 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.196+0000 7fad9a59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c6a59a620 0x558c6a5a5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33642/0 (socket says 172.21.15.97:33642) 2024-04-23T10:39:22.201 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.196+0000 7fad9a59c640 1 -- 172.21.15.97:0/17081243 learned_addr learned my addr 172.21.15.97:0/17081243 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:39:22.201 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.196+0000 7fad99d9b640 1 --2- 172.21.15.97:0/17081243 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c6a59af40 0x558c6a5a2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:22.201 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.196+0000 7fad9a59c640 1 -- 172.21.15.97:0/17081243 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c6a59af40 msgr2=0x558c6a5a2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:39:22.202 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.196+0000 7fad9a59c640 1 --2- 172.21.15.97:0/17081243 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c6a59af40 0x558c6a5a2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:22.202 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.196+0000 7fad9a59c640 1 -- 172.21.15.97:0/17081243 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558c6a5a5800 msgr2=0x558c6a5a2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:39:22.202 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.196+0000 7fad9a59c640 1 --2- 172.21.15.97:0/17081243 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558c6a5a5800 0x558c6a5a2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:22.202 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.196+0000 7fad9a59c640 1 -- 172.21.15.97:0/17081243 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c6a2de620 con 0x558c6a59a620 2024-04-23T10:39:22.202 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.197+0000 7fad9a59c640 1 --2- 172.21.15.97:0/17081243 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c6a59a620 0x558c6a5a5130 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fad8c0988f0 tx=0x7fad8c0ce160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2583b5113dbab3b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:39:22.202 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.197+0000 7fad81ffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:39:22.202 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.197+0000 7fad81ffb640 1 -- 172.21.15.97:0/17081243 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad8c020050 con 0x558c6a59a620 2024-04-23T10:39:22.202 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.197+0000 7fad81ffb640 1 -- 172.21.15.97:0/17081243 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8484+0+0 (secure 0 0 0) 0x7fad8c0d0040 con 0x558c6a59a620 2024-04-23T10:39:22.202 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.197+0000 7fada3869740 1 -- 172.21.15.97:0/17081243 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad8c09c490 con 0x558c6a59a620 2024-04-23T10:39:22.202 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.197+0000 7fad81ffb640 1 -- 172.21.15.97:0/17081243 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fad8c0a5100 con 0x558c6a59a620 2024-04-23T10:39:22.202 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.197+0000 7fada3869740 10 client.17433 fetch_fsmap learned FSMap version 26 2024-04-23T10:39:22.202 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.197+0000 7fada3869740 1 -- 172.21.15.97:0/17081243 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558c6a5b4280 con 0x558c6a59a620 2024-04-23T10:39:22.202 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.197+0000 7fad81ffb640 1 -- 172.21.15.97:0/17081243 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad8c09c490 con 0x558c6a59a620 2024-04-23T10:39:22.202 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.197+0000 7fad81ffb640 1 -- 172.21.15.97:0/17081243 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fad8c0981e0 con 0x558c6a59a620 2024-04-23T10:39:22.202 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.197+0000 7fada3869740 10 client.17433 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:39:22.203 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.197+0000 7fada3869740 10 client.17433 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:39:22.203 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.197+0000 7fada3869740 20 client.17433 populate_metadata read hostname 'smithi097' 2024-04-23T10:39:22.203 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.197+0000 7fada3869740 1 --2- 172.21.15.97:0/17081243 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x558c6a5b55d0 0x558c6a5b79c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:22.203 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.197+0000 7fada3869740 4 client.17433 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:39:22.203 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.197+0000 7fada3869740 1 -- 172.21.15.97:0/17081243 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c6a50d390 con 0x558c6a5b55d0 2024-04-23T10:39:22.203 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.198+0000 7fad99d9b640 1 --2- 172.21.15.97:0/17081243 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x558c6a5b55d0 0x558c6a5b79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:22.203 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.198+0000 7fad81ffb640 10 client.17433 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:39:22.203 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.198+0000 7fad99d9b640 1 --2- 172.21.15.97:0/17081243 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x558c6a5b55d0 0x558c6a5b79c0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bc2d04f76e9a614 server_cookie=5c700022887316e3 in_seq=0 out_seq=0 2024-04-23T10:39:22.203 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.199+0000 7fad81ffb640 1 -- 172.21.15.97:0/17081243 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558c6a50d390 con 0x558c6a5b55d0 2024-04-23T10:39:22.203 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.199+0000 7fad81ffb640 10 client.17433 handle_command_reply: tid=0 2024-04-23T10:39:22.203 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.199+0000 7fada3869740 1 client.17433 shutdown 2024-04-23T10:39:22.203 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.199+0000 7fada3869740 1 -- 172.21.15.97:0/17081243 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c6a59a620 msgr2=0x558c6a5a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:39:22.203 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.199+0000 7fada3869740 1 --2- 172.21.15.97:0/17081243 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c6a59a620 0x558c6a5a5130 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fad8c0988f0 tx=0x7fad8c0ce160 comp rx=0 tx=0).stop 2024-04-23T10:39:22.203 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.199+0000 7fada3869740 1 -- 172.21.15.97:0/17081243 shutdown_connections 2024-04-23T10:39:22.203 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.199+0000 7fada3869740 1 --2- 172.21.15.97:0/17081243 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x558c6a5b55d0 0x558c6a5b79c0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:22.203 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.199+0000 7fada3869740 1 --2- 172.21.15.97:0/17081243 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558c6a5a5800 0x558c6a5a2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:22.204 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.199+0000 7fada3869740 1 --2- 172.21.15.97:0/17081243 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c6a59af40 0x558c6a5a2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:22.204 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.199+0000 7fad81ffb640 0 client.17433 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:39:22.204 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.199+0000 7fada3869740 1 --2- 172.21.15.97:0/17081243 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c6a59a620 0x558c6a5a5130 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:22.204 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.199+0000 7fada3869740 1 -- 172.21.15.97:0/17081243 >> 172.21.15.97:0/17081243 conn(0x558c6a596b30 msgr2=0x558c6a59c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:39:22.204 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.199+0000 7fada3869740 1 -- 172.21.15.97:0/17081243 shutdown_connections 2024-04-23T10:39:22.204 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.199+0000 7fada3869740 1 -- 172.21.15.97:0/17081243 wait complete. 2024-04-23T10:39:22.204 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.199+0000 7fada3869740 20 client.17433 trim_cache size 0 max 16384 2024-04-23T10:39:22.218 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.215+0000 7fada3869740 1 Processor -- start 2024-04-23T10:39:22.218 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.215+0000 7fada3869740 1 -- start start 2024-04-23T10:39:22.218 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.215+0000 7fada3869740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c6a59a620 0x558c6a5b1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:22.218 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.215+0000 7fada3869740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558c6a59af40 0x558c6a59daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:22.219 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.215+0000 7fada3869740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c6a5a5800 0x558c6a5a4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:22.219 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.215+0000 7fada3869740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fad9c1adb10 con 0x558c6a59a620 2024-04-23T10:39:22.219 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.215+0000 7fada3869740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fad9c1af650 con 0x558c6a59af40 2024-04-23T10:39:22.219 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.215+0000 7fada3869740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fad9c1ae1e0 con 0x558c6a5a5800 2024-04-23T10:39:22.219 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.215+0000 7fad9a59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c6a59a620 0x558c6a5b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:22.219 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.215+0000 7fad9a59c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c6a59a620 0x558c6a5b1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33650/0 (socket says 172.21.15.97:33650) 2024-04-23T10:39:22.219 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.215+0000 7fad9a59c640 1 -- 172.21.15.97:0/3311488830 learned_addr learned my addr 172.21.15.97:0/3311488830 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:39:22.219 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.215+0000 7fad9ad9d640 1 --2- 172.21.15.97:0/3311488830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c6a5a5800 0x558c6a5a4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:22.219 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.215+0000 7fad99d9b640 1 --2- 172.21.15.97:0/3311488830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558c6a59af40 0x558c6a59daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:22.219 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.215+0000 7fad9a59c640 1 -- 172.21.15.97:0/3311488830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c6a5a5800 msgr2=0x558c6a5a4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:39:22.219 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.215+0000 7fad9a59c640 1 --2- 172.21.15.97:0/3311488830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c6a5a5800 0x558c6a5a4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:22.219 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.215+0000 7fad9a59c640 1 -- 172.21.15.97:0/3311488830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558c6a59af40 msgr2=0x558c6a59daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:39:22.219 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.215+0000 7fad9a59c640 1 --2- 172.21.15.97:0/3311488830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558c6a59af40 0x558c6a59daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:22.219 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.216+0000 7fad9a59c640 1 -- 172.21.15.97:0/3311488830 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c6a5a8490 con 0x558c6a59a620 2024-04-23T10:39:22.219 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.216+0000 7fad9a59c640 1 --2- 172.21.15.97:0/3311488830 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c6a59a620 0x558c6a5b1230 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fad8c098e60 tx=0x7fad8c010cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f89ce80c281d4a44 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:39:22.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.216+0000 7fad83fff640 1 -- 172.21.15.97:0/3311488830 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad8c020020 con 0x558c6a59a620 2024-04-23T10:39:22.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.216+0000 7fad83fff640 1 -- 172.21.15.97:0/3311488830 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad8c0b5070 con 0x558c6a59a620 2024-04-23T10:39:22.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.216+0000 7fad83fff640 1 -- 172.21.15.97:0/3311488830 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad8c0d0040 con 0x558c6a59a620 2024-04-23T10:39:22.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.216+0000 7fada3869740 1 -- 172.21.15.97:0/3311488830 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c6a59a620 msgr2=0x558c6a5b1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:39:22.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.216+0000 7fada3869740 1 --2- 172.21.15.97:0/3311488830 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c6a59a620 0x558c6a5b1230 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fad8c098e60 tx=0x7fad8c010cb0 comp rx=0 tx=0).stop 2024-04-23T10:39:22.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.217+0000 7fada3869740 1 -- 172.21.15.97:0/3311488830 shutdown_connections 2024-04-23T10:39:22.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.217+0000 7fada3869740 1 --2- 172.21.15.97:0/3311488830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c6a5a5800 0x558c6a5a4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:22.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.217+0000 7fada3869740 1 --2- 172.21.15.97:0/3311488830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558c6a59af40 0x558c6a59daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:22.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.217+0000 7fada3869740 1 --2- 172.21.15.97:0/3311488830 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c6a59a620 0x558c6a5b1230 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:22.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.217+0000 7fada3869740 1 -- 172.21.15.97:0/3311488830 >> 172.21.15.97:0/3311488830 conn(0x558c6a596b30 msgr2=0x558c6a5a1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:39:22.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.217+0000 7fada3869740 1 -- 172.21.15.97:0/3311488830 shutdown_connections 2024-04-23T10:39:22.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.217+0000 7fada3869740 1 -- 172.21.15.97:0/3311488830 wait complete. 2024-04-23T10:39:22.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.217+0000 7fada3869740 1 -- start start 2024-04-23T10:39:22.220 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.217+0000 7fada3869740 1 Processor -- start 2024-04-23T10:39:22.221 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.217+0000 7fada3869740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558c6a59a620 0x558c6a5ad810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:22.221 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.217+0000 7fada3869740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c6a59af40 0x558c6a5a8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:22.221 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.217+0000 7fada3869740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c6a5a5800 0x558c6a5a9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:22.221 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.217+0000 7fada3869740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fad9c1aef80 con 0x558c6a59af40 2024-04-23T10:39:22.221 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.217+0000 7fada3869740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fad9c1ae8b0 con 0x558c6a59a620 2024-04-23T10:39:22.221 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.217+0000 7fada3869740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fad9c1abb10 con 0x558c6a5a5800 2024-04-23T10:39:22.221 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.217+0000 7fada3869740 10 client.0 _ll_register_callbacks cb 0x7fad8c009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:39:22.221 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.217+0000 7fad99d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c6a59af40 0x558c6a5a8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:22.221 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.217+0000 7fad99d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c6a59af40 0x558c6a5a8d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33662/0 (socket says 172.21.15.97:33662) 2024-04-23T10:39:22.221 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.217+0000 7fad99d9b640 1 -- 172.21.15.97:0/7137778 learned_addr learned my addr 172.21.15.97:0/7137778 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:39:22.221 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.217+0000 7fad9a59c640 1 --2- 172.21.15.97:0/7137778 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558c6a59a620 0x558c6a5ad810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:22.221 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.217+0000 7fad9ad9d640 1 --2- 172.21.15.97:0/7137778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c6a5a5800 0x558c6a5a9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:22.221 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.218+0000 7fad99d9b640 1 -- 172.21.15.97:0/7137778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c6a5a5800 msgr2=0x558c6a5a9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:39:22.221 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.218+0000 7fad99d9b640 1 --2- 172.21.15.97:0/7137778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c6a5a5800 0x558c6a5a9240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:22.222 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.218+0000 7fad99d9b640 1 -- 172.21.15.97:0/7137778 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558c6a59a620 msgr2=0x558c6a5ad810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:39:22.222 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.218+0000 7fad99d9b640 1 --2- 172.21.15.97:0/7137778 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558c6a59a620 0x558c6a5ad810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:22.222 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.218+0000 7fad99d9b640 1 -- 172.21.15.97:0/7137778 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c6a5a9780 con 0x558c6a59af40 2024-04-23T10:39:22.222 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.218+0000 7fad9a59c640 1 --2- 172.21.15.97:0/7137778 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558c6a59a620 0x558c6a5ad810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:39:22.222 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.218+0000 7fad99d9b640 1 --2- 172.21.15.97:0/7137778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c6a59af40 0x558c6a5a8d00 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fad9002f550 tx=0x7fad90031b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bc79b7bac44ca96 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:39:22.222 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.218+0000 7fad81ffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:39:22.222 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.218+0000 7fad81ffb640 1 -- 172.21.15.97:0/7137778 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad90031650 con 0x558c6a59af40 2024-04-23T10:39:22.222 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.218+0000 7fad81ffb640 1 -- 172.21.15.97:0/7137778 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8484+0+0 (secure 0 0 0) 0x7fad900317f0 con 0x558c6a59af40 2024-04-23T10:39:22.222 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.218+0000 7fada3869740 1 -- 172.21.15.97:0/7137778 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad8c0b5070 con 0x558c6a59af40 2024-04-23T10:39:22.222 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.218+0000 7fad81ffb640 1 -- 172.21.15.97:0/7137778 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fad90003970 con 0x558c6a59af40 2024-04-23T10:39:22.222 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.218+0000 7fada3869740 10 client.17439 fetch_fsmap learned FSMap version 26 2024-04-23T10:39:22.222 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.218+0000 7fada3869740 1 -- 172.21.15.97:0/7137778 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558c6a61b580 con 0x558c6a59af40 2024-04-23T10:39:22.222 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.218+0000 7fad81ffb640 1 -- 172.21.15.97:0/7137778 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad8c0b5070 con 0x558c6a59af40 2024-04-23T10:39:22.223 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.218+0000 7fad81ffb640 1 -- 172.21.15.97:0/7137778 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fad90045030 con 0x558c6a59af40 2024-04-23T10:39:22.223 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.218+0000 7fada3869740 10 client.17439 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:39:22.223 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.218+0000 7fada3869740 10 client.17439 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:39:22.223 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.218+0000 7fada3869740 20 client.17439 populate_metadata read hostname 'smithi097' 2024-04-23T10:39:22.223 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.218+0000 7fada3869740 1 --2- 172.21.15.97:0/7137778 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x558c6a61bd30 0x558c6a61e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:22.223 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.218+0000 7fada3869740 4 client.17439 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:39:22.223 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.218+0000 7fada3869740 1 -- 172.21.15.97:0/7137778 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558c6a507200 con 0x558c6a61bd30 2024-04-23T10:39:22.223 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.219+0000 7fad9a59c640 1 --2- 172.21.15.97:0/7137778 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x558c6a61bd30 0x558c6a61e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:22.223 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.219+0000 7fad9a59c640 1 --2- 172.21.15.97:0/7137778 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x558c6a61bd30 0x558c6a61e150 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49a53d5ad82a5dcc server_cookie=6af8c9fcbb484e75 in_seq=0 out_seq=0 2024-04-23T10:39:22.223 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.219+0000 7fad81ffb640 10 client.17439 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:39:22.232 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.233+0000 7fad81ffb640 1 -- 172.21.15.97:0/7137778 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558c6a507200 con 0x558c6a61bd30 2024-04-23T10:39:22.232 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.233+0000 7fad81ffb640 10 client.17439 handle_command_reply: tid=0 2024-04-23T10:39:22.232 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.233+0000 7fada3869740 1 client.17439 shutdown 2024-04-23T10:39:22.232 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.233+0000 7fada3869740 1 -- 172.21.15.97:0/7137778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c6a59af40 msgr2=0x558c6a5a8d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:39:22.232 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.233+0000 7fada3869740 1 --2- 172.21.15.97:0/7137778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c6a59af40 0x558c6a5a8d00 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fad9002f550 tx=0x7fad90031b80 comp rx=0 tx=0).stop 2024-04-23T10:39:22.233 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.233+0000 7fada3869740 1 -- 172.21.15.97:0/7137778 shutdown_connections 2024-04-23T10:39:22.233 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.233+0000 7fada3869740 1 --2- 172.21.15.97:0/7137778 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x558c6a61bd30 0x558c6a61e150 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:22.233 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.233+0000 7fada3869740 1 --2- 172.21.15.97:0/7137778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c6a5a5800 0x558c6a5a9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:22.233 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.233+0000 7fada3869740 1 --2- 172.21.15.97:0/7137778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558c6a59af40 0x558c6a5a8d00 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:22.233 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.233+0000 7fad81ffb640 0 client.17439 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:39:22.233 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.233+0000 7fada3869740 1 --2- 172.21.15.97:0/7137778 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558c6a59a620 0x558c6a5ad810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:22.233 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.233+0000 7fada3869740 1 -- 172.21.15.97:0/7137778 >> 172.21.15.97:0/7137778 conn(0x558c6a596b30 msgr2=0x558c6a595420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:39:22.233 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.233+0000 7fada3869740 1 -- 172.21.15.97:0/7137778 shutdown_connections 2024-04-23T10:39:22.233 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.233+0000 7fada3869740 1 -- 172.21.15.97:0/7137778 wait complete. 2024-04-23T10:39:22.233 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.233+0000 7fada3869740 20 client.17439 trim_cache size 0 max 16384 2024-04-23T10:39:22.233 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:39:22.233 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:39:22.234 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:39:22.234 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:39:22.234 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.234+0000 7fada0b44640 1 -- 172.21.15.97:0/3882038961 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fad840a3dc0 msgr2=0x7fad840a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:39:22.234 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.234+0000 7fada0b44640 1 --2- 172.21.15.97:0/3882038961 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fad840a3dc0 0x7fad840a6280 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fad94009cc0 tx=0x7fad940098d0 comp rx=0 tx=0).stop 2024-04-23T10:39:22.235 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.234+0000 7fada0b44640 1 -- 172.21.15.97:0/3882038961 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad9c1010e0 msgr2=0x7fad9c198670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:39:22.235 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.234+0000 7fada0b44640 1 --2- 172.21.15.97:0/3882038961 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad9c1010e0 0x7fad9c198670 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fad8c00d950 tx=0x7fad8c00de10 comp rx=0 tx=0).stop 2024-04-23T10:39:22.235 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.234+0000 7fada0b44640 1 -- 172.21.15.97:0/3882038961 shutdown_connections 2024-04-23T10:39:22.235 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.234+0000 7fada0b44640 1 --2- 172.21.15.97:0/3882038961 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fad840a3dc0 0x7fad840a6280 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:22.235 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.234+0000 7fada0b44640 1 --2- 172.21.15.97:0/3882038961 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad9c19cf30 0x7fad9c19d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:22.235 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.234+0000 7fada0b44640 1 --2- 172.21.15.97:0/3882038961 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad9c1061b0 0x7fad9c198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:22.235 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.234+0000 7fada0b44640 1 --2- 172.21.15.97:0/3882038961 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad9c1010e0 0x7fad9c198670 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:22.235 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.234+0000 7fada0b44640 1 -- 172.21.15.97:0/3882038961 >> 172.21.15.97:0/3882038961 conn(0x7fad9c0771d0 msgr2=0x7fad9c0fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:39:22.235 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.234+0000 7fada0b44640 1 -- 172.21.15.97:0/3882038961 shutdown_connections 2024-04-23T10:39:22.235 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:22.234+0000 7fada0b44640 1 -- 172.21.15.97:0/3882038961 wait complete. 2024-04-23T10:39:23.452 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:39:23 smithi183 ceph-mon[32317]: pgmap v412: 97 pgs: 97 active+clean; 2.3 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 18 op/s 2024-04-23T10:39:23.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:39:23 smithi196 ceph-mon[32919]: pgmap v412: 97 pgs: 97 active+clean; 2.3 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 18 op/s 2024-04-23T10:39:23.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:39:23 smithi097 ceph-mon[29197]: pgmap v412: 97 pgs: 97 active+clean; 2.3 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 47 KiB/s wr, 18 op/s 2024-04-23T10:39:25.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:39:25 smithi196 ceph-mon[32919]: pgmap v413: 97 pgs: 97 active+clean; 2.5 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-23T10:39:25.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:39:25 smithi097 ceph-mon[29197]: pgmap v413: 97 pgs: 97 active+clean; 2.5 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-23T10:39:25.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:39:25 smithi183 ceph-mon[32317]: pgmap v413: 97 pgs: 97 active+clean; 2.5 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-23T10:39:27.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:39:27 smithi196 ceph-mon[32919]: pgmap v414: 97 pgs: 97 active+clean; 2.5 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-23T10:39:27.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:39:27 smithi097 ceph-mon[29197]: pgmap v414: 97 pgs: 97 active+clean; 2.5 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-23T10:39:27.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:39:27 smithi183 ceph-mon[32317]: pgmap v414: 97 pgs: 97 active+clean; 2.5 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-23T10:39:29.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:39:29 smithi196 ceph-mon[32919]: pgmap v415: 97 pgs: 97 active+clean; 2.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-23T10:39:29.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:39:29 smithi097 ceph-mon[29197]: pgmap v415: 97 pgs: 97 active+clean; 2.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-23T10:39:29.649 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:39:29 smithi183 ceph-mon[32317]: pgmap v415: 97 pgs: 97 active+clean; 2.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-23T10:39:30.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:39:30 smithi196 ceph-mon[32919]: pgmap v416: 97 pgs: 97 active+clean; 2.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-23T10:39:30.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:39:30 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:39:30.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:39:30 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:39:30.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:39:30 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:39:30.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:39:30 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:39:30.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:39:30 smithi097 ceph-mon[29197]: pgmap v416: 97 pgs: 97 active+clean; 2.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-23T10:39:30.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:39:30 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:39:30.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:39:30 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:39:30.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:39:30 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:39:30.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:39:30 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:39:30.649 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:39:30 smithi183 ceph-mon[32317]: pgmap v416: 97 pgs: 97 active+clean; 2.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-23T10:39:30.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:39:30 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:39:30.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:39:30 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:39:30.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:39:30 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:39:30.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:39:30 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:39:32.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:39:32 smithi183 ceph-mon[32317]: pgmap v417: 97 pgs: 97 active+clean; 2.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-23T10:39:32.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:39:32 smithi196 ceph-mon[32919]: pgmap v417: 97 pgs: 97 active+clean; 2.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-23T10:39:32.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:39:32 smithi097 ceph-mon[29197]: pgmap v417: 97 pgs: 97 active+clean; 2.7 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-23T10:39:34.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:39:34 smithi183 ceph-mon[32317]: pgmap v418: 97 pgs: 97 active+clean; 2.9 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-23T10:39:34.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:39:34 smithi196 ceph-mon[32919]: pgmap v418: 97 pgs: 97 active+clean; 2.9 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-23T10:39:34.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:39:34 smithi097 ceph-mon[29197]: pgmap v418: 97 pgs: 97 active+clean; 2.9 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-23T10:39:35.330 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.329+0000 7f6c85235640 1 -- 172.21.15.97:0/2872023647 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c80102b70 msgr2=0x7f6c80106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:39:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.329+0000 7f6c85235640 1 --2- 172.21.15.97:0/2872023647 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c80102b70 0x7f6c80106bb0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f6c68009900 tx=0x7f6c6802a410 comp rx=0 tx=0).stop 2024-04-23T10:39:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.329+0000 7f6c85235640 1 -- 172.21.15.97:0/2872023647 shutdown_connections 2024-04-23T10:39:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.329+0000 7f6c85235640 1 --2- 172.21.15.97:0/2872023647 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c80107290 0x7f6c8010ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.329+0000 7f6c85235640 1 --2- 172.21.15.97:0/2872023647 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c80102b70 0x7f6c80106bb0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.329+0000 7f6c85235640 1 --2- 172.21.15.97:0/2872023647 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c801021c0 0x7f6c801025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.329+0000 7f6c85235640 1 -- 172.21.15.97:0/2872023647 >> 172.21.15.97:0/2872023647 conn(0x7f6c800fda20 msgr2=0x7f6c800ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:39:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.329+0000 7f6c85235640 1 -- 172.21.15.97:0/2872023647 shutdown_connections 2024-04-23T10:39:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.329+0000 7f6c85235640 1 -- 172.21.15.97:0/2872023647 wait complete. 2024-04-23T10:39:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.330+0000 7f6c85235640 1 Processor -- start 2024-04-23T10:39:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.330+0000 7f6c85235640 1 -- start start 2024-04-23T10:39:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.330+0000 7f6c85235640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c801021c0 0x7f6c80198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.330+0000 7f6c85235640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c80107290 0x7f6c80198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.330+0000 7f6c85235640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c8019cf50 0x7f6c8019d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.330+0000 7f6c85235640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6c801121b0 con 0x7f6c801021c0 2024-04-23T10:39:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.330+0000 7f6c85235640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6c80112030 con 0x7f6c8019cf50 2024-04-23T10:39:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.330+0000 7f6c85235640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6c80112330 con 0x7f6c80107290 2024-04-23T10:39:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.331+0000 7f6c7ed9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c801021c0 0x7f6c80198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.331+0000 7f6c7ed9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c801021c0 0x7f6c80198690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53368/0 (socket says 172.21.15.97:53368) 2024-04-23T10:39:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.331+0000 7f6c7ed9d640 1 -- 172.21.15.97:0/1833848218 learned_addr learned my addr 172.21.15.97:0/1833848218 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:39:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.331+0000 7f6c7f59e640 1 --2- 172.21.15.97:0/1833848218 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c8019cf50 0x7f6c8019d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.331+0000 7f6c77fff640 1 --2- 172.21.15.97:0/1833848218 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c80107290 0x7f6c80198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.331+0000 7f6c7ed9d640 1 -- 172.21.15.97:0/1833848218 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c80107290 msgr2=0x7f6c80198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:39:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.331+0000 7f6c7ed9d640 1 --2- 172.21.15.97:0/1833848218 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c80107290 0x7f6c80198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.331+0000 7f6c7ed9d640 1 -- 172.21.15.97:0/1833848218 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c8019cf50 msgr2=0x7f6c8019d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:39:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.331+0000 7f6c7ed9d640 1 --2- 172.21.15.97:0/1833848218 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c8019cf50 0x7f6c8019d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.331+0000 7f6c7ed9d640 1 -- 172.21.15.97:0/1833848218 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c8019da40 con 0x7f6c801021c0 2024-04-23T10:39:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.331+0000 7f6c7ed9d640 1 --2- 172.21.15.97:0/1833848218 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c801021c0 0x7f6c80198690 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f6c7000d950 tx=0x7f6c7000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:39:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.331+0000 7f6c7cd99640 1 -- 172.21.15.97:0/1833848218 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c70017070 con 0x7f6c801021c0 2024-04-23T10:39:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.331+0000 7f6c7cd99640 1 -- 172.21.15.97:0/1833848218 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c70009c30 con 0x7f6c801021c0 2024-04-23T10:39:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.331+0000 7f6c85235640 1 -- 172.21.15.97:0/1833848218 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c8019dd30 con 0x7f6c801021c0 2024-04-23T10:39:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.331+0000 7f6c7cd99640 1 -- 172.21.15.97:0/1833848218 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c7000f620 con 0x7f6c801021c0 2024-04-23T10:39:35.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.331+0000 7f6c85235640 1 -- 172.21.15.97:0/1833848218 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c801a54f0 con 0x7f6c801021c0 2024-04-23T10:39:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.334+0000 7f6c7cd99640 1 -- 172.21.15.97:0/1833848218 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6c70010460 con 0x7f6c801021c0 2024-04-23T10:39:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.334+0000 7f6c8802e740 1 Processor -- start 2024-04-23T10:39:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.334+0000 7f6c8802e740 1 -- start start 2024-04-23T10:39:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.334+0000 7f6c8802e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563886fb5620 0x563886fb5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.334+0000 7f6c8802e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563886fb5f40 0x563886fc02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.334+0000 7f6c8802e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563886fc0800 0x563886fc2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.334+0000 7f6c8802e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x563886f1a970 con 0x563886fb5f40 2024-04-23T10:39:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.334+0000 7f6c8802e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563886eb74a0 con 0x563886fb5620 2024-04-23T10:39:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.334+0000 7f6c8802e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563886f16930 con 0x563886fc0800 2024-04-23T10:39:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.335+0000 7f6c7ed9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563886fb5f40 0x563886fc02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:35.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.335+0000 7f6c7ed9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563886fb5f40 0x563886fc02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53384/0 (socket says 172.21.15.97:53384) 2024-04-23T10:39:35.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.335+0000 7f6c7ed9d640 1 -- 172.21.15.97:0/2522437324 learned_addr learned my addr 172.21.15.97:0/2522437324 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:39:35.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.335+0000 7f6c7cd99640 1 --2- 172.21.15.97:0/1833848218 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f6c6c0a3dc0 0x7f6c6c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:35.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.335+0000 7f6c77fff640 1 --2- 172.21.15.97:0/2522437324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563886fc0800 0x563886fc2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:35.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.335+0000 7f6c7f59e640 1 --2- 172.21.15.97:0/2522437324 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563886fb5620 0x563886fb5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:35.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.335+0000 7f6c7ed9d640 1 -- 172.21.15.97:0/2522437324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563886fc0800 msgr2=0x563886fc2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:39:35.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.335+0000 7f6c7ed9d640 1 --2- 172.21.15.97:0/2522437324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563886fc0800 0x563886fc2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:35.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.335+0000 7f6c7ed9d640 1 -- 172.21.15.97:0/2522437324 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563886fb5620 msgr2=0x563886fb5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:39:35.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.335+0000 7f6c7ed9d640 1 --2- 172.21.15.97:0/2522437324 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563886fb5620 0x563886fb5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:35.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.335+0000 7f6c7ed9d640 1 -- 172.21.15.97:0/2522437324 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563886c2d210 con 0x563886fb5f40 2024-04-23T10:39:35.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.335+0000 7f6c7f59e640 1 --2- 172.21.15.97:0/2522437324 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563886fb5620 0x563886fb5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:39:35.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.335+0000 7f6c7f59e640 1 --2- 172.21.15.97:0/1833848218 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f6c6c0a3dc0 0x7f6c6c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:35.341 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.335+0000 7f6c7cd99640 1 -- 172.21.15.97:0/1833848218 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8484+0+0 (secure 0 0 0) 0x7f6c700d0370 con 0x7f6c801021c0 2024-04-23T10:39:35.342 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.335+0000 7f6c7ed9d640 1 --2- 172.21.15.97:0/2522437324 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563886fb5f40 0x563886fc02c0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f6c700cda50 tx=0x7f6c70022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bff69942adb86877 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:39:35.342 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.335+0000 7f6c757fa640 1 -- 172.21.15.97:0/2522437324 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c7009bf20 con 0x563886fb5f40 2024-04-23T10:39:35.342 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.335+0000 7f6c757fa640 1 -- 172.21.15.97:0/2522437324 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c70009de0 con 0x563886fb5f40 2024-04-23T10:39:35.342 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.335+0000 7f6c757fa640 1 -- 172.21.15.97:0/2522437324 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c7009bd40 con 0x563886fb5f40 2024-04-23T10:39:35.342 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.336+0000 7f6c8802e740 1 -- 172.21.15.97:0/2522437324 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563886fb5f40 msgr2=0x563886fc02c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:39:35.342 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.336+0000 7f6c7f59e640 1 --2- 172.21.15.97:0/1833848218 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f6c6c0a3dc0 0x7f6c6c0a6280 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6c78005fd0 tx=0x7f6c78004a40 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:39:35.342 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.336+0000 7f6c8802e740 1 --2- 172.21.15.97:0/2522437324 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563886fb5f40 0x563886fc02c0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f6c700cda50 tx=0x7f6c70022000 comp rx=0 tx=0).stop 2024-04-23T10:39:35.342 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.336+0000 7f6c8802e740 1 -- 172.21.15.97:0/2522437324 shutdown_connections 2024-04-23T10:39:35.342 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.336+0000 7f6c8802e740 1 --2- 172.21.15.97:0/2522437324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563886fc0800 0x563886fc2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:35.342 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.336+0000 7f6c8802e740 1 --2- 172.21.15.97:0/2522437324 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563886fb5f40 0x563886fc02c0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:35.342 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.336+0000 7f6c8802e740 1 --2- 172.21.15.97:0/2522437324 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563886fb5620 0x563886fb5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:35.342 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.336+0000 7f6c8802e740 1 -- 172.21.15.97:0/2522437324 >> 172.21.15.97:0/2522437324 conn(0x563886fb1b30 msgr2=0x563886fb1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:39:35.342 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.336+0000 7f6c8802e740 1 -- 172.21.15.97:0/2522437324 shutdown_connections 2024-04-23T10:39:35.342 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.336+0000 7f6c8802e740 1 -- 172.21.15.97:0/2522437324 wait complete. 2024-04-23T10:39:35.343 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.337+0000 7f6c8802e740 1 -- start start 2024-04-23T10:39:35.343 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.337+0000 7f6c8802e740 1 Processor -- start 2024-04-23T10:39:35.343 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.337+0000 7f6c8802e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563886fb5620 0x563886fc0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:35.343 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.337+0000 7f6c8802e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563886fb5f40 0x563886fbd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:35.343 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.337+0000 7f6c8802e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563886fc0800 0x563886fbd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:35.343 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.337+0000 7f6c8802e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x563886fc3370 con 0x563886fc0800 2024-04-23T10:39:35.343 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.337+0000 7f6c8802e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563886eb50b0 con 0x563886fb5f40 2024-04-23T10:39:35.343 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.337+0000 7f6c8802e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x563886fc34f0 con 0x563886fb5620 2024-04-23T10:39:35.343 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.337+0000 7f6c8802e740 10 client.0 _ll_register_callbacks cb 0x563886ea2d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:39:35.343 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.337+0000 7f6c7f59e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563886fc0800 0x563886fbd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:35.343 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.337+0000 7f6c7f59e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563886fc0800 0x563886fbd9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53388/0 (socket says 172.21.15.97:53388) 2024-04-23T10:39:35.343 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.337+0000 7f6c7f59e640 1 -- 172.21.15.97:0/2924323406 learned_addr learned my addr 172.21.15.97:0/2924323406 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:39:35.343 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.337+0000 7f6c77fff640 1 --2- 172.21.15.97:0/2924323406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563886fb5f40 0x563886fbd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:35.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.337+0000 7f6c7ed9d640 1 --2- 172.21.15.97:0/2924323406 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563886fb5620 0x563886fc0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:35.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.337+0000 7f6c7f59e640 1 -- 172.21.15.97:0/2924323406 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563886fb5620 msgr2=0x563886fc0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:39:35.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.337+0000 7f6c7f59e640 1 --2- 172.21.15.97:0/2924323406 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563886fb5620 0x563886fc0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:35.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.337+0000 7f6c7f59e640 1 -- 172.21.15.97:0/2924323406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563886fb5f40 msgr2=0x563886fbd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:39:35.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.337+0000 7f6c7f59e640 1 --2- 172.21.15.97:0/2924323406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563886fb5f40 0x563886fbd470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:35.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.337+0000 7f6c7f59e640 1 -- 172.21.15.97:0/2924323406 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563886cf9620 con 0x563886fc0800 2024-04-23T10:39:35.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.337+0000 7f6c7f59e640 1 --2- 172.21.15.97:0/2924323406 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563886fc0800 0x563886fbd9f0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f6c78009d80 tx=0x7f6c78009600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=876902c5e0c83407 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:39:35.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.337+0000 7f6c667fc640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:39:35.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.338+0000 7f6c667fc640 1 -- 172.21.15.97:0/2924323406 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c78023070 con 0x563886fc0800 2024-04-23T10:39:35.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.338+0000 7f6c667fc640 1 -- 172.21.15.97:0/2924323406 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8484+0+0 (secure 0 0 0) 0x7f6c7800ece0 con 0x563886fc0800 2024-04-23T10:39:35.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.338+0000 7f6c8802e740 1 -- 172.21.15.97:0/2924323406 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c70009de0 con 0x563886fc0800 2024-04-23T10:39:35.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.338+0000 7f6c667fc640 1 -- 172.21.15.97:0/2924323406 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6c7802f050 con 0x563886fc0800 2024-04-23T10:39:35.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.338+0000 7f6c8802e740 10 client.17451 fetch_fsmap learned FSMap version 26 2024-04-23T10:39:35.344 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.338+0000 7f6c8802e740 1 -- 172.21.15.97:0/2924323406 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563886fcf2e0 con 0x563886fc0800 2024-04-23T10:39:35.345 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.338+0000 7f6c667fc640 1 -- 172.21.15.97:0/2924323406 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c70009de0 con 0x563886fc0800 2024-04-23T10:39:35.345 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.338+0000 7f6c667fc640 1 -- 172.21.15.97:0/2924323406 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f6c7800f040 con 0x563886fc0800 2024-04-23T10:39:35.345 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.338+0000 7f6c8802e740 10 client.17451 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:39:35.345 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.338+0000 7f6c8802e740 10 client.17451 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:39:35.345 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.338+0000 7f6c8802e740 20 client.17451 populate_metadata read hostname 'smithi097' 2024-04-23T10:39:35.345 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.338+0000 7f6c8802e740 1 --2- 172.21.15.97:0/2924323406 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x563886fd0750 0x563886fd2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:35.345 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.338+0000 7f6c8802e740 4 client.17451 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:39:35.345 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.338+0000 7f6c8802e740 1 -- 172.21.15.97:0/2924323406 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563886f28390 con 0x563886fd0750 2024-04-23T10:39:35.345 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.339+0000 7f6c7ed9d640 1 --2- 172.21.15.97:0/2924323406 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x563886fd0750 0x563886fd2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:35.345 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.339+0000 7f6c7ed9d640 1 --2- 172.21.15.97:0/2924323406 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x563886fd0750 0x563886fd2b60 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29d109ec0e815069 server_cookie=eab7a4c45269af8d in_seq=0 out_seq=0 2024-04-23T10:39:35.345 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.339+0000 7f6c667fc640 10 client.17451 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:39:35.345 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.340+0000 7f6c667fc640 1 -- 172.21.15.97:0/2924323406 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563886f28390 con 0x563886fd0750 2024-04-23T10:39:35.345 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.340+0000 7f6c667fc640 10 client.17451 handle_command_reply: tid=0 2024-04-23T10:39:35.345 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.340+0000 7f6c8802e740 1 client.17451 shutdown 2024-04-23T10:39:35.346 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.340+0000 7f6c8802e740 1 -- 172.21.15.97:0/2924323406 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563886fc0800 msgr2=0x563886fbd9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:39:35.346 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.340+0000 7f6c8802e740 1 --2- 172.21.15.97:0/2924323406 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563886fc0800 0x563886fbd9f0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f6c78009d80 tx=0x7f6c78009600 comp rx=0 tx=0).stop 2024-04-23T10:39:35.346 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.340+0000 7f6c8802e740 1 -- 172.21.15.97:0/2924323406 shutdown_connections 2024-04-23T10:39:35.346 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.340+0000 7f6c8802e740 1 --2- 172.21.15.97:0/2924323406 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x563886fd0750 0x563886fd2b60 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:35.346 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.340+0000 7f6c8802e740 1 --2- 172.21.15.97:0/2924323406 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563886fc0800 0x563886fbd9f0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:35.346 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.340+0000 7f6c8802e740 1 --2- 172.21.15.97:0/2924323406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563886fb5f40 0x563886fbd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:35.346 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.340+0000 7f6c667fc640 0 client.17451 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:39:35.346 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.340+0000 7f6c8802e740 1 --2- 172.21.15.97:0/2924323406 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563886fb5620 0x563886fc0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:35.346 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.340+0000 7f6c8802e740 1 -- 172.21.15.97:0/2924323406 >> 172.21.15.97:0/2924323406 conn(0x563886fb1b30 msgr2=0x563886fb7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:39:35.346 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.340+0000 7f6c8802e740 1 -- 172.21.15.97:0/2924323406 shutdown_connections 2024-04-23T10:39:35.346 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.340+0000 7f6c8802e740 1 -- 172.21.15.97:0/2924323406 wait complete. 2024-04-23T10:39:35.346 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.340+0000 7f6c8802e740 20 client.17451 trim_cache size 0 max 16384 2024-04-23T10:39:35.358 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.355+0000 7f6c8802e740 1 Processor -- start 2024-04-23T10:39:35.359 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.355+0000 7f6c8802e740 1 -- start start 2024-04-23T10:39:35.359 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.355+0000 7f6c8802e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563886fb5620 0x563886fcc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:35.359 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.355+0000 7f6c8802e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563886fb5f40 0x563886fb8970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:35.359 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.355+0000 7f6c8802e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563886fc0800 0x563886fd84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:35.359 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.355+0000 7f6c8802e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6c801adaf0 con 0x563886fb5620 2024-04-23T10:39:35.359 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.355+0000 7f6c8802e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6c801af630 con 0x563886fb5f40 2024-04-23T10:39:35.359 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.355+0000 7f6c8802e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6c801ae1c0 con 0x563886fc0800 2024-04-23T10:39:35.359 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.355+0000 7f6c7ed9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563886fb5620 0x563886fcc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:35.359 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.355+0000 7f6c7ed9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563886fb5620 0x563886fcc230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53400/0 (socket says 172.21.15.97:53400) 2024-04-23T10:39:35.359 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.355+0000 7f6c7ed9d640 1 -- 172.21.15.97:0/3945042884 learned_addr learned my addr 172.21.15.97:0/3945042884 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:39:35.359 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.355+0000 7f6c7ed9d640 1 -- 172.21.15.97:0/3945042884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563886fc0800 msgr2=0x563886fd84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:39:35.359 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.355+0000 7f6c7f59e640 1 --2- 172.21.15.97:0/3945042884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563886fc0800 0x563886fd84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:35.359 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.355+0000 7f6c77fff640 1 --2- 172.21.15.97:0/3945042884 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563886fb5f40 0x563886fb8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:35.360 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.355+0000 7f6c7ed9d640 1 --2- 172.21.15.97:0/3945042884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563886fc0800 0x563886fd84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:35.360 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.355+0000 7f6c7ed9d640 1 -- 172.21.15.97:0/3945042884 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563886fb5f40 msgr2=0x563886fb8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:39:35.360 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.355+0000 7f6c7ed9d640 1 --2- 172.21.15.97:0/3945042884 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563886fb5f40 0x563886fb8970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:35.360 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.356+0000 7f6c7ed9d640 1 -- 172.21.15.97:0/3945042884 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563886fad2e0 con 0x563886fb5620 2024-04-23T10:39:35.360 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.356+0000 7f6c7ed9d640 1 --2- 172.21.15.97:0/3945042884 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563886fb5620 0x563886fcc230 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f6c70098840 tx=0x7f6c7009be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e798a95010ebc2b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:39:35.360 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.356+0000 7f6c74ff9640 1 -- 172.21.15.97:0/3945042884 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c70009de0 con 0x563886fb5620 2024-04-23T10:39:35.360 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.356+0000 7f6c74ff9640 1 -- 172.21.15.97:0/3945042884 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c700a4fa0 con 0x563886fb5620 2024-04-23T10:39:35.360 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.356+0000 7f6c74ff9640 1 -- 172.21.15.97:0/3945042884 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c700a06e0 con 0x563886fb5620 2024-04-23T10:39:35.360 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.356+0000 7f6c8802e740 1 -- 172.21.15.97:0/3945042884 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563886fb5620 msgr2=0x563886fcc230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:39:35.360 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.356+0000 7f6c8802e740 1 --2- 172.21.15.97:0/3945042884 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563886fb5620 0x563886fcc230 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f6c70098840 tx=0x7f6c7009be60 comp rx=0 tx=0).stop 2024-04-23T10:39:35.360 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.357+0000 7f6c8802e740 1 -- 172.21.15.97:0/3945042884 shutdown_connections 2024-04-23T10:39:35.360 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.357+0000 7f6c8802e740 1 --2- 172.21.15.97:0/3945042884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563886fc0800 0x563886fd84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:35.360 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.357+0000 7f6c8802e740 1 --2- 172.21.15.97:0/3945042884 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563886fb5f40 0x563886fb8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:35.360 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.357+0000 7f6c8802e740 1 --2- 172.21.15.97:0/3945042884 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563886fb5620 0x563886fcc230 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:35.360 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.357+0000 7f6c8802e740 1 -- 172.21.15.97:0/3945042884 >> 172.21.15.97:0/3945042884 conn(0x563886fb1b30 msgr2=0x563886fb8260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:39:35.361 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.357+0000 7f6c8802e740 1 -- 172.21.15.97:0/3945042884 shutdown_connections 2024-04-23T10:39:35.361 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.357+0000 7f6c8802e740 1 -- 172.21.15.97:0/3945042884 wait complete. 2024-04-23T10:39:35.361 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.357+0000 7f6c8802e740 1 -- start start 2024-04-23T10:39:35.361 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.357+0000 7f6c8802e740 1 Processor -- start 2024-04-23T10:39:35.361 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.357+0000 7f6c8802e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563886fb5620 0x563886fc89a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:35.361 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.357+0000 7f6c8802e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563886fb5f40 0x563886fb9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:35.361 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.357+0000 7f6c8802e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563886fc0800 0x563886fba220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:35.361 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.357+0000 7f6c8802e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6c801aef60 con 0x563886fc0800 2024-04-23T10:39:35.361 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.357+0000 7f6c8802e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6c801ae890 con 0x563886fb5f40 2024-04-23T10:39:35.361 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.357+0000 7f6c8802e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6c801abaf0 con 0x563886fb5620 2024-04-23T10:39:35.361 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.357+0000 7f6c8802e740 10 client.0 _ll_register_callbacks cb 0x7f6c780125b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:39:35.361 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.357+0000 7f6c7f59e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563886fc0800 0x563886fba220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:35.361 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.357+0000 7f6c7f59e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563886fc0800 0x563886fba220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53412/0 (socket says 172.21.15.97:53412) 2024-04-23T10:39:35.362 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.357+0000 7f6c7f59e640 1 -- 172.21.15.97:0/3596510378 learned_addr learned my addr 172.21.15.97:0/3596510378 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:39:35.362 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.357+0000 7f6c77fff640 1 --2- 172.21.15.97:0/3596510378 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563886fb5f40 0x563886fb9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:35.362 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.358+0000 7f6c7ed9d640 1 --2- 172.21.15.97:0/3596510378 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563886fb5620 0x563886fc89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:35.362 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.358+0000 7f6c7f59e640 1 -- 172.21.15.97:0/3596510378 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563886fb5620 msgr2=0x563886fc89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:39:35.362 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.358+0000 7f6c7f59e640 1 --2- 172.21.15.97:0/3596510378 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563886fb5620 0x563886fc89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:35.362 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.358+0000 7f6c7f59e640 1 -- 172.21.15.97:0/3596510378 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563886fb5f40 msgr2=0x563886fb9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:39:35.362 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.358+0000 7f6c7f59e640 1 --2- 172.21.15.97:0/3596510378 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563886fb5f40 0x563886fb9ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:35.362 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.358+0000 7f6c7f59e640 1 -- 172.21.15.97:0/3596510378 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563886fba890 con 0x563886fc0800 2024-04-23T10:39:35.362 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.358+0000 7f6c7ed9d640 1 --2- 172.21.15.97:0/3596510378 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563886fb5620 0x563886fc89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:39:35.362 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.358+0000 7f6c7f59e640 1 --2- 172.21.15.97:0/3596510378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563886fc0800 0x563886fba220 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f6c78005b00 tx=0x7f6c78009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b03aebdd17c2ab2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:39:35.362 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.358+0000 7f6c667fc640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:39:35.362 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.358+0000 7f6c667fc640 1 -- 172.21.15.97:0/3596510378 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c78037030 con 0x563886fc0800 2024-04-23T10:39:35.362 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.358+0000 7f6c667fc640 1 -- 172.21.15.97:0/3596510378 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8484+0+0 (secure 0 0 0) 0x7f6c7800ecf0 con 0x563886fc0800 2024-04-23T10:39:35.363 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.358+0000 7f6c8802e740 1 -- 172.21.15.97:0/3596510378 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c700a4fa0 con 0x563886fc0800 2024-04-23T10:39:35.363 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.358+0000 7f6c667fc640 1 -- 172.21.15.97:0/3596510378 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6c78041050 con 0x563886fc0800 2024-04-23T10:39:35.363 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.358+0000 7f6c8802e740 10 client.17457 fetch_fsmap learned FSMap version 26 2024-04-23T10:39:35.363 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.358+0000 7f6c8802e740 1 -- 172.21.15.97:0/3596510378 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563887036bd0 con 0x563886fc0800 2024-04-23T10:39:35.363 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.358+0000 7f6c667fc640 1 -- 172.21.15.97:0/3596510378 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c700a4fa0 con 0x563886fc0800 2024-04-23T10:39:35.363 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.358+0000 7f6c667fc640 1 -- 172.21.15.97:0/3596510378 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f6c78005020 con 0x563886fc0800 2024-04-23T10:39:35.363 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.359+0000 7f6c8802e740 10 client.17457 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:39:35.363 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.359+0000 7f6c8802e740 10 client.17457 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:39:35.363 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.359+0000 7f6c8802e740 20 client.17457 populate_metadata read hostname 'smithi097' 2024-04-23T10:39:35.363 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.359+0000 7f6c8802e740 1 --2- 172.21.15.97:0/3596510378 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5638870372e0 0x5638870396d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:35.363 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.359+0000 7f6c8802e740 4 client.17457 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:39:35.363 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.359+0000 7f6c8802e740 1 -- 172.21.15.97:0/3596510378 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563886f22200 con 0x5638870372e0 2024-04-23T10:39:35.363 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.359+0000 7f6c7ed9d640 1 --2- 172.21.15.97:0/3596510378 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5638870372e0 0x5638870396d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:35.363 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.359+0000 7f6c7ed9d640 1 --2- 172.21.15.97:0/3596510378 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5638870372e0 0x5638870396d0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8e700f4284735a0 server_cookie=25c71be74ed5efd2 in_seq=0 out_seq=0 2024-04-23T10:39:35.363 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.359+0000 7f6c667fc640 10 client.17457 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:39:35.372 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.373+0000 7f6c667fc640 1 -- 172.21.15.97:0/3596510378 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563886f22200 con 0x5638870372e0 2024-04-23T10:39:35.372 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.373+0000 7f6c667fc640 10 client.17457 handle_command_reply: tid=0 2024-04-23T10:39:35.373 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.373+0000 7f6c8802e740 1 client.17457 shutdown 2024-04-23T10:39:35.373 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.373+0000 7f6c8802e740 1 -- 172.21.15.97:0/3596510378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563886fc0800 msgr2=0x563886fba220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:39:35.373 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.373+0000 7f6c8802e740 1 --2- 172.21.15.97:0/3596510378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563886fc0800 0x563886fba220 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f6c78005b00 tx=0x7f6c78009e20 comp rx=0 tx=0).stop 2024-04-23T10:39:35.373 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.373+0000 7f6c8802e740 1 -- 172.21.15.97:0/3596510378 shutdown_connections 2024-04-23T10:39:35.373 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.373+0000 7f6c8802e740 1 --2- 172.21.15.97:0/3596510378 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5638870372e0 0x5638870396d0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:35.373 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.373+0000 7f6c8802e740 1 --2- 172.21.15.97:0/3596510378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563886fc0800 0x563886fba220 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:35.373 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.373+0000 7f6c8802e740 1 --2- 172.21.15.97:0/3596510378 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563886fb5f40 0x563886fb9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:35.373 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.373+0000 7f6c8802e740 1 --2- 172.21.15.97:0/3596510378 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563886fb5620 0x563886fc89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:35.373 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.373+0000 7f6c667fc640 0 client.17457 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:39:35.373 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.373+0000 7f6c8802e740 1 -- 172.21.15.97:0/3596510378 >> 172.21.15.97:0/3596510378 conn(0x563886fb1b30 msgr2=0x563886fb0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:39:35.373 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.373+0000 7f6c8802e740 1 -- 172.21.15.97:0/3596510378 shutdown_connections 2024-04-23T10:39:35.373 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.373+0000 7f6c8802e740 1 -- 172.21.15.97:0/3596510378 wait complete. 2024-04-23T10:39:35.373 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.373+0000 7f6c8802e740 20 client.17457 trim_cache size 0 max 16384 2024-04-23T10:39:35.374 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:39:35.374 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:39:35.374 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:39:35.374 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:39:35.375 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.374+0000 7f6c85235640 1 -- 172.21.15.97:0/1833848218 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f6c6c0a3dc0 msgr2=0x7f6c6c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:39:35.375 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.374+0000 7f6c85235640 1 --2- 172.21.15.97:0/1833848218 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f6c6c0a3dc0 0x7f6c6c0a6280 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6c78005fd0 tx=0x7f6c78004a40 comp rx=0 tx=0).stop 2024-04-23T10:39:35.375 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.374+0000 7f6c85235640 1 -- 172.21.15.97:0/1833848218 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c801021c0 msgr2=0x7f6c80198690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:39:35.375 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.374+0000 7f6c85235640 1 --2- 172.21.15.97:0/1833848218 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c801021c0 0x7f6c80198690 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f6c7000d950 tx=0x7f6c7000de10 comp rx=0 tx=0).stop 2024-04-23T10:39:35.375 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.374+0000 7f6c85235640 1 -- 172.21.15.97:0/1833848218 shutdown_connections 2024-04-23T10:39:35.375 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.374+0000 7f6c85235640 1 --2- 172.21.15.97:0/1833848218 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f6c6c0a3dc0 0x7f6c6c0a6280 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:35.375 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.374+0000 7f6c85235640 1 --2- 172.21.15.97:0/1833848218 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c8019cf50 0x7f6c8019d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:35.375 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.374+0000 7f6c85235640 1 --2- 172.21.15.97:0/1833848218 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c80107290 0x7f6c80198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:35.376 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.374+0000 7f6c85235640 1 --2- 172.21.15.97:0/1833848218 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c801021c0 0x7f6c80198690 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:35.376 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.374+0000 7f6c85235640 1 -- 172.21.15.97:0/1833848218 >> 172.21.15.97:0/1833848218 conn(0x7f6c800fda20 msgr2=0x7f6c800ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:39:35.376 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.374+0000 7f6c85235640 1 -- 172.21.15.97:0/1833848218 shutdown_connections 2024-04-23T10:39:35.376 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:35.374+0000 7f6c85235640 1 -- 172.21.15.97:0/1833848218 wait complete. 2024-04-23T10:39:36.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:39:36 smithi183 ceph-mon[32317]: pgmap v419: 97 pgs: 97 active+clean; 2.9 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:39:36.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:39:36 smithi196 ceph-mon[32919]: pgmap v419: 97 pgs: 97 active+clean; 2.9 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:39:36.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:39:36 smithi097 ceph-mon[29197]: pgmap v419: 97 pgs: 97 active+clean; 2.9 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:39:38.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:39:38 smithi196 ceph-mon[32919]: pgmap v420: 97 pgs: 97 active+clean; 3.1 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 65 KiB/s wr, 24 op/s 2024-04-23T10:39:38.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:39:38 smithi097 ceph-mon[29197]: pgmap v420: 97 pgs: 97 active+clean; 3.1 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 65 KiB/s wr, 24 op/s 2024-04-23T10:39:38.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:39:38 smithi183 ceph-mon[32317]: pgmap v420: 97 pgs: 97 active+clean; 3.1 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 65 KiB/s wr, 24 op/s 2024-04-23T10:39:40.662 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:39:40 smithi097 ceph-mon[29197]: pgmap v421: 97 pgs: 97 active+clean; 3.1 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:39:40.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:39:40 smithi196 ceph-mon[32919]: pgmap v421: 97 pgs: 97 active+clean; 3.1 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:39:40.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:39:40 smithi183 ceph-mon[32317]: pgmap v421: 97 pgs: 97 active+clean; 3.1 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:39:42.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:39:42 smithi196 ceph-mon[32919]: pgmap v422: 97 pgs: 97 active+clean; 3.1 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:39:42.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:39:42 smithi097 ceph-mon[29197]: pgmap v422: 97 pgs: 97 active+clean; 3.1 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:39:42.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:39:42 smithi183 ceph-mon[32317]: pgmap v422: 97 pgs: 97 active+clean; 3.1 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:39:44.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:39:44 smithi196 ceph-mon[32919]: pgmap v423: 97 pgs: 97 active+clean; 3.3 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-23T10:39:44.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:39:44 smithi097 ceph-mon[29197]: pgmap v423: 97 pgs: 97 active+clean; 3.3 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-23T10:39:44.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:39:44 smithi183 ceph-mon[32317]: pgmap v423: 97 pgs: 97 active+clean; 3.3 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-23T10:39:46.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:39:46 smithi196 ceph-mon[32919]: pgmap v424: 97 pgs: 97 active+clean; 3.3 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:39:46.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:39:46 smithi097 ceph-mon[29197]: pgmap v424: 97 pgs: 97 active+clean; 3.3 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:39:46.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:39:46 smithi183 ceph-mon[32317]: pgmap v424: 97 pgs: 97 active+clean; 3.3 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:39:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.472+0000 7f5f2decf640 1 -- 172.21.15.97:0/3985464397 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f281021c0 msgr2=0x7f5f281025a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:39:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.472+0000 7f5f2decf640 1 --2- 172.21.15.97:0/3985464397 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f281021c0 0x7f5f281025a0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f5f18009900 tx=0x7f5f1802a410 comp rx=0 tx=0).stop 2024-04-23T10:39:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.472+0000 7f5f2decf640 1 -- 172.21.15.97:0/3985464397 shutdown_connections 2024-04-23T10:39:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.472+0000 7f5f2decf640 1 --2- 172.21.15.97:0/3985464397 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f28107290 0x7f5f2810ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.472+0000 7f5f2decf640 1 --2- 172.21.15.97:0/3985464397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f28102b70 0x7f5f28106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.472+0000 7f5f2decf640 1 --2- 172.21.15.97:0/3985464397 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f281021c0 0x7f5f281025a0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.472+0000 7f5f2decf640 1 -- 172.21.15.97:0/3985464397 >> 172.21.15.97:0/3985464397 conn(0x7f5f280fda20 msgr2=0x7f5f280ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:39:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.472+0000 7f5f2decf640 1 -- 172.21.15.97:0/3985464397 shutdown_connections 2024-04-23T10:39:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.472+0000 7f5f2decf640 1 -- 172.21.15.97:0/3985464397 wait complete. 2024-04-23T10:39:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.473+0000 7f5f2decf640 1 Processor -- start 2024-04-23T10:39:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.473+0000 7f5f2decf640 1 -- start start 2024-04-23T10:39:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.473+0000 7f5f2decf640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f281021c0 0x7f5f28198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.473+0000 7f5f2decf640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f28107290 0x7f5f28198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.473+0000 7f5f2decf640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f2819cfa0 0x7f5f2819d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.473+0000 7f5f2decf640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5f281121d0 con 0x7f5f28107290 2024-04-23T10:39:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.473+0000 7f5f2decf640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5f28112050 con 0x7f5f2819cfa0 2024-04-23T10:39:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.473+0000 7f5f2decf640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5f28112350 con 0x7f5f281021c0 2024-04-23T10:39:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.473+0000 7f5f26ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f28107290 0x7f5f28198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.473+0000 7f5f26ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f28107290 0x7f5f28198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48670/0 (socket says 172.21.15.97:48670) 2024-04-23T10:39:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.473+0000 7f5f26ffd640 1 -- 172.21.15.97:0/1604548766 learned_addr learned my addr 172.21.15.97:0/1604548766 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:39:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.474+0000 7f5f27fff640 1 --2- 172.21.15.97:0/1604548766 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f2819cfa0 0x7f5f2819d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.474+0000 7f5f277fe640 1 --2- 172.21.15.97:0/1604548766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f281021c0 0x7f5f28198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.474+0000 7f5f26ffd640 1 -- 172.21.15.97:0/1604548766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f281021c0 msgr2=0x7f5f28198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:39:48.476 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.474+0000 7f5f26ffd640 1 --2- 172.21.15.97:0/1604548766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f281021c0 0x7f5f28198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:48.476 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.474+0000 7f5f26ffd640 1 -- 172.21.15.97:0/1604548766 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f2819cfa0 msgr2=0x7f5f2819d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:39:48.476 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.474+0000 7f5f26ffd640 1 --2- 172.21.15.97:0/1604548766 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f2819cfa0 0x7f5f2819d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:48.476 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.474+0000 7f5f26ffd640 1 -- 172.21.15.97:0/1604548766 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f2819da90 con 0x7f5f28107290 2024-04-23T10:39:48.476 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.474+0000 7f5f27fff640 1 --2- 172.21.15.97:0/1604548766 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f2819cfa0 0x7f5f2819d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:39:48.476 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.474+0000 7f5f26ffd640 1 --2- 172.21.15.97:0/1604548766 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f28107290 0x7f5f28198c70 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c005fd0 tx=0x7f5f1c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:39:48.476 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.474+0000 7f5f24ff9640 1 -- 172.21.15.97:0/1604548766 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c017070 con 0x7f5f28107290 2024-04-23T10:39:48.476 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.474+0000 7f5f2decf640 1 -- 172.21.15.97:0/1604548766 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f2819dd80 con 0x7f5f28107290 2024-04-23T10:39:48.476 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.474+0000 7f5f24ff9640 1 -- 172.21.15.97:0/1604548766 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f1c009c30 con 0x7f5f28107290 2024-04-23T10:39:48.476 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.474+0000 7f5f24ff9640 1 -- 172.21.15.97:0/1604548766 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c003d30 con 0x7f5f28107290 2024-04-23T10:39:48.476 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.474+0000 7f5f2decf640 1 -- 172.21.15.97:0/1604548766 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f281a5570 con 0x7f5f28107290 2024-04-23T10:39:48.483 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.476+0000 7f5f24ff9640 1 -- 172.21.15.97:0/1604548766 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5f1c003ed0 con 0x7f5f28107290 2024-04-23T10:39:48.483 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.476+0000 7f5f30be3740 1 Processor -- start 2024-04-23T10:39:48.483 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.477+0000 7f5f30be3740 1 -- start start 2024-04-23T10:39:48.483 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.477+0000 7f5f30be3740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623ade28620 0x5623ade28a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:48.483 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.477+0000 7f5f30be3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5623ade28f40 0x5623ade332c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:48.483 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.477+0000 7f5f30be3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623ade33800 0x5623ade35bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:48.483 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.477+0000 7f5f30be3740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5623add8d970 con 0x5623ade28620 2024-04-23T10:39:48.483 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.477+0000 7f5f30be3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5623add2a4a0 con 0x5623ade28f40 2024-04-23T10:39:48.483 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.477+0000 7f5f30be3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5623add89930 con 0x5623ade33800 2024-04-23T10:39:48.483 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.477+0000 7f5f24ff9640 1 --2- 172.21.15.97:0/1604548766 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f5f100a3dc0 0x7f5f100a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:48.483 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.477+0000 7f5f27fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623ade28620 0x5623ade28a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.477+0000 7f5f27fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623ade28620 0x5623ade28a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48674/0 (socket says 172.21.15.97:48674) 2024-04-23T10:39:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.477+0000 7f5f27fff640 1 -- 172.21.15.97:0/2020287249 learned_addr learned my addr 172.21.15.97:0/2020287249 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:39:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.477+0000 7f5f26ffd640 1 --2- 172.21.15.97:0/2020287249 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623ade33800 0x5623ade35bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.477+0000 7f5f27fff640 1 --2- 172.21.15.97:0/1604548766 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f5f100a3dc0 0x7f5f100a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.477+0000 7f5f277fe640 1 --2- 172.21.15.97:0/2020287249 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5623ade28f40 0x5623ade332c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.477+0000 7f5f24ff9640 1 -- 172.21.15.97:0/1604548766 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8484+0+0 (secure 0 0 0) 0x7f5f1c0d04e0 con 0x7f5f28107290 2024-04-23T10:39:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.477+0000 7f5f27fff640 1 -- 172.21.15.97:0/2020287249 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623ade33800 msgr2=0x5623ade35bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:39:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.477+0000 7f5f27fff640 1 --2- 172.21.15.97:0/2020287249 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623ade33800 0x5623ade35bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.477+0000 7f5f27fff640 1 -- 172.21.15.97:0/2020287249 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5623ade28f40 msgr2=0x5623ade332c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:39:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.477+0000 7f5f27fff640 1 --2- 172.21.15.97:0/2020287249 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5623ade28f40 0x5623ade332c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.477+0000 7f5f27fff640 1 -- 172.21.15.97:0/2020287249 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623adaa0210 con 0x5623ade28620 2024-04-23T10:39:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.478+0000 7f5f27fff640 1 --2- 172.21.15.97:0/2020287249 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623ade28620 0x5623ade28a00 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f5f20013e60 tx=0x7f5f200107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3903124d8d41b0ae server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:39:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.478+0000 7f5f27fff640 1 --2- 172.21.15.97:0/1604548766 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f5f100a3dc0 0x7f5f100a6280 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5f20010ca0 tx=0x7f5f20015000 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:39:48.485 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.478+0000 7f5f0dffb640 1 -- 172.21.15.97:0/2020287249 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2000ae10 con 0x5623ade28620 2024-04-23T10:39:48.485 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.478+0000 7f5f0dffb640 1 -- 172.21.15.97:0/2020287249 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f20007be0 con 0x5623ade28620 2024-04-23T10:39:48.485 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.478+0000 7f5f0dffb640 1 -- 172.21.15.97:0/2020287249 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2001f660 con 0x5623ade28620 2024-04-23T10:39:48.485 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.478+0000 7f5f30be3740 1 -- 172.21.15.97:0/2020287249 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623ade28620 msgr2=0x5623ade28a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:39:48.485 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.478+0000 7f5f30be3740 1 --2- 172.21.15.97:0/2020287249 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623ade28620 0x5623ade28a00 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f5f20013e60 tx=0x7f5f200107a0 comp rx=0 tx=0).stop 2024-04-23T10:39:48.485 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.478+0000 7f5f30be3740 1 -- 172.21.15.97:0/2020287249 shutdown_connections 2024-04-23T10:39:48.485 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.478+0000 7f5f30be3740 1 --2- 172.21.15.97:0/2020287249 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623ade33800 0x5623ade35bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:48.485 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.478+0000 7f5f30be3740 1 --2- 172.21.15.97:0/2020287249 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5623ade28f40 0x5623ade332c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:48.485 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.478+0000 7f5f30be3740 1 --2- 172.21.15.97:0/2020287249 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623ade28620 0x5623ade28a00 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:48.485 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.478+0000 7f5f30be3740 1 -- 172.21.15.97:0/2020287249 >> 172.21.15.97:0/2020287249 conn(0x5623ade24b30 msgr2=0x5623ade24f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:39:48.485 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.479+0000 7f5f30be3740 1 -- 172.21.15.97:0/2020287249 shutdown_connections 2024-04-23T10:39:48.485 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.479+0000 7f5f30be3740 1 -- 172.21.15.97:0/2020287249 wait complete. 2024-04-23T10:39:48.485 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.479+0000 7f5f30be3740 1 -- start start 2024-04-23T10:39:48.485 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.479+0000 7f5f30be3740 1 Processor -- start 2024-04-23T10:39:48.485 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.479+0000 7f5f30be3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5623ade28620 0x5623ade33190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:48.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.479+0000 7f5f30be3740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623ade28f40 0x5623ade30480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:48.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.479+0000 7f5f30be3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623ade33800 0x5623ade30a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:48.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.479+0000 7f5f30be3740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5623ade363d0 con 0x5623ade28f40 2024-04-23T10:39:48.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.479+0000 7f5f30be3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5623add280b0 con 0x5623ade28620 2024-04-23T10:39:48.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.479+0000 7f5f30be3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5623ade36550 con 0x5623ade33800 2024-04-23T10:39:48.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.479+0000 7f5f30be3740 10 client.0 _ll_register_callbacks cb 0x5623add15d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:39:48.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.480+0000 7f5f26ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623ade28f40 0x5623ade30480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:48.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.480+0000 7f5f26ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623ade28f40 0x5623ade30480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48684/0 (socket says 172.21.15.97:48684) 2024-04-23T10:39:48.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.480+0000 7f5f26ffd640 1 -- 172.21.15.97:0/324504330 learned_addr learned my addr 172.21.15.97:0/324504330 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:39:48.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.480+0000 7f5f277fe640 1 --2- 172.21.15.97:0/324504330 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5623ade28620 0x5623ade33190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:48.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.480+0000 7f5f27fff640 1 --2- 172.21.15.97:0/324504330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623ade33800 0x5623ade30a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:48.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.480+0000 7f5f26ffd640 1 -- 172.21.15.97:0/324504330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623ade33800 msgr2=0x5623ade30a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:39:48.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.480+0000 7f5f26ffd640 1 --2- 172.21.15.97:0/324504330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623ade33800 0x5623ade30a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:48.486 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.480+0000 7f5f26ffd640 1 -- 172.21.15.97:0/324504330 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5623ade28620 msgr2=0x5623ade33190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:39:48.487 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.480+0000 7f5f26ffd640 1 --2- 172.21.15.97:0/324504330 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5623ade28620 0x5623ade33190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:48.487 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.480+0000 7f5f26ffd640 1 -- 172.21.15.97:0/324504330 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623adb6c620 con 0x5623ade28f40 2024-04-23T10:39:48.487 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.480+0000 7f5f26ffd640 1 --2- 172.21.15.97:0/324504330 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623ade28f40 0x5623ade30480 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c098e70 tx=0x7f5f1c018460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=163d2967b9de8887 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:39:48.487 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.480+0000 7f5f06ffd640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:39:48.487 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.480+0000 7f5f06ffd640 1 -- 172.21.15.97:0/324504330 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c009de0 con 0x5623ade28f40 2024-04-23T10:39:48.487 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.480+0000 7f5f06ffd640 1 -- 172.21.15.97:0/324504330 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8484+0+0 (secure 0 0 0) 0x7f5f1c098b10 con 0x5623ade28f40 2024-04-23T10:39:48.487 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.480+0000 7f5f30be3740 1 -- 172.21.15.97:0/324504330 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f20007be0 con 0x5623ade28f40 2024-04-23T10:39:48.487 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.480+0000 7f5f06ffd640 1 -- 172.21.15.97:0/324504330 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5f1c09c4a0 con 0x5623ade28f40 2024-04-23T10:39:48.487 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.480+0000 7f5f30be3740 10 client.17469 fetch_fsmap learned FSMap version 26 2024-04-23T10:39:48.487 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.480+0000 7f5f30be3740 1 -- 172.21.15.97:0/324504330 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5623ade42320 con 0x5623ade28f40 2024-04-23T10:39:48.487 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.481+0000 7f5f06ffd640 1 -- 172.21.15.97:0/324504330 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f20007be0 con 0x5623ade28f40 2024-04-23T10:39:48.487 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.481+0000 7f5f06ffd640 1 -- 172.21.15.97:0/324504330 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5f1c0041c0 con 0x5623ade28f40 2024-04-23T10:39:48.487 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.481+0000 7f5f30be3740 10 client.17469 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:39:48.487 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.481+0000 7f5f30be3740 10 client.17469 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:39:48.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.481+0000 7f5f30be3740 20 client.17469 populate_metadata read hostname 'smithi097' 2024-04-23T10:39:48.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.481+0000 7f5f30be3740 1 --2- 172.21.15.97:0/324504330 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5623ade43800 0x5623ade45bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:48.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.481+0000 7f5f30be3740 4 client.17469 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:39:48.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.481+0000 7f5f30be3740 1 -- 172.21.15.97:0/324504330 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5623add9b390 con 0x5623ade43800 2024-04-23T10:39:48.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.481+0000 7f5f277fe640 1 --2- 172.21.15.97:0/324504330 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5623ade43800 0x5623ade45bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:48.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.481+0000 7f5f06ffd640 10 client.17469 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:39:48.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.481+0000 7f5f277fe640 1 --2- 172.21.15.97:0/324504330 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5623ade43800 0x5623ade45bf0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87742aa619699c78 server_cookie=1bea1242d901eea7 in_seq=0 out_seq=0 2024-04-23T10:39:48.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.482+0000 7f5f06ffd640 1 -- 172.21.15.97:0/324504330 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5623add9b390 con 0x5623ade43800 2024-04-23T10:39:48.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.482+0000 7f5f06ffd640 10 client.17469 handle_command_reply: tid=0 2024-04-23T10:39:48.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.482+0000 7f5f30be3740 1 client.17469 shutdown 2024-04-23T10:39:48.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.482+0000 7f5f30be3740 1 -- 172.21.15.97:0/324504330 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623ade28f40 msgr2=0x5623ade30480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:39:48.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.482+0000 7f5f30be3740 1 --2- 172.21.15.97:0/324504330 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623ade28f40 0x5623ade30480 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c098e70 tx=0x7f5f1c018460 comp rx=0 tx=0).stop 2024-04-23T10:39:48.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.482+0000 7f5f30be3740 1 -- 172.21.15.97:0/324504330 shutdown_connections 2024-04-23T10:39:48.488 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.482+0000 7f5f30be3740 1 --2- 172.21.15.97:0/324504330 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5623ade43800 0x5623ade45bf0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:48.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.482+0000 7f5f30be3740 1 --2- 172.21.15.97:0/324504330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623ade33800 0x5623ade30a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:48.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.482+0000 7f5f30be3740 1 --2- 172.21.15.97:0/324504330 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623ade28f40 0x5623ade30480 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:48.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.482+0000 7f5f06ffd640 0 client.17469 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:39:48.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.482+0000 7f5f30be3740 1 --2- 172.21.15.97:0/324504330 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5623ade28620 0x5623ade33190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:48.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.482+0000 7f5f30be3740 1 -- 172.21.15.97:0/324504330 >> 172.21.15.97:0/324504330 conn(0x5623ade24b30 msgr2=0x5623ade2a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:39:48.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.482+0000 7f5f30be3740 1 -- 172.21.15.97:0/324504330 shutdown_connections 2024-04-23T10:39:48.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.483+0000 7f5f30be3740 1 -- 172.21.15.97:0/324504330 wait complete. 2024-04-23T10:39:48.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.483+0000 7f5f30be3740 20 client.17469 trim_cache size 0 max 16384 2024-04-23T10:39:48.498 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.497+0000 7f5f30be3740 1 Processor -- start 2024-04-23T10:39:48.498 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.497+0000 7f5f30be3740 1 -- start start 2024-04-23T10:39:48.498 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.498+0000 7f5f30be3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5623ade28620 0x5623ade3bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:48.498 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.498+0000 7f5f30be3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623ade28f40 0x5623ade32cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:48.498 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.498+0000 7f5f30be3740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623ade33800 0x5623ade2a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:48.498 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.498+0000 7f5f30be3740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5f281adaf0 con 0x5623ade33800 2024-04-23T10:39:48.498 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.498+0000 7f5f30be3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5f281af630 con 0x5623ade28620 2024-04-23T10:39:48.498 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.498+0000 7f5f30be3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5f281ae1c0 con 0x5623ade28f40 2024-04-23T10:39:48.498 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.498+0000 7f5f27fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623ade33800 0x5623ade2a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:48.498 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.498+0000 7f5f27fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623ade33800 0x5623ade2a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48694/0 (socket says 172.21.15.97:48694) 2024-04-23T10:39:48.499 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.498+0000 7f5f27fff640 1 -- 172.21.15.97:0/996859 learned_addr learned my addr 172.21.15.97:0/996859 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:39:48.499 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.498+0000 7f5f27fff640 1 -- 172.21.15.97:0/996859 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623ade28f40 msgr2=0x5623ade32cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:39:48.499 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.498+0000 7f5f26ffd640 1 --2- 172.21.15.97:0/996859 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623ade28f40 0x5623ade32cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:48.499 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.498+0000 7f5f27fff640 1 --2- 172.21.15.97:0/996859 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623ade28f40 0x5623ade32cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:48.499 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.498+0000 7f5f27fff640 1 -- 172.21.15.97:0/996859 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5623ade28620 msgr2=0x5623ade3bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:39:48.499 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.498+0000 7f5f27fff640 1 --2- 172.21.15.97:0/996859 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5623ade28620 0x5623ade3bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:48.499 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.498+0000 7f5f27fff640 1 -- 172.21.15.97:0/996859 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623ade35f60 con 0x5623ade33800 2024-04-23T10:39:48.499 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.498+0000 7f5f26ffd640 1 --2- 172.21.15.97:0/996859 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623ade28f40 0x5623ade32cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:39:48.499 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.498+0000 7f5f27fff640 1 --2- 172.21.15.97:0/996859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623ade33800 0x5623ade2a6e0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f5f20007460 tx=0x7f5f200072a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b84fdfe6a6acb806 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:39:48.499 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.498+0000 7f5f0d7fa640 1 -- 172.21.15.97:0/996859 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f20007ca0 con 0x5623ade33800 2024-04-23T10:39:48.499 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.498+0000 7f5f0d7fa640 1 -- 172.21.15.97:0/996859 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f2000ac10 con 0x5623ade33800 2024-04-23T10:39:48.499 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.498+0000 7f5f0d7fa640 1 -- 172.21.15.97:0/996859 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f200052c0 con 0x5623ade33800 2024-04-23T10:39:48.499 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.499+0000 7f5f30be3740 1 -- 172.21.15.97:0/996859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623ade33800 msgr2=0x5623ade2a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:39:48.499 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.499+0000 7f5f30be3740 1 --2- 172.21.15.97:0/996859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623ade33800 0x5623ade2a6e0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f5f20007460 tx=0x7f5f200072a0 comp rx=0 tx=0).stop 2024-04-23T10:39:48.500 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.499+0000 7f5f30be3740 1 -- 172.21.15.97:0/996859 shutdown_connections 2024-04-23T10:39:48.500 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.499+0000 7f5f30be3740 1 --2- 172.21.15.97:0/996859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623ade33800 0x5623ade2a6e0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:48.500 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.499+0000 7f5f30be3740 1 --2- 172.21.15.97:0/996859 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623ade28f40 0x5623ade32cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:48.500 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.499+0000 7f5f30be3740 1 --2- 172.21.15.97:0/996859 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5623ade28620 0x5623ade3bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:48.500 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.499+0000 7f5f30be3740 1 -- 172.21.15.97:0/996859 >> 172.21.15.97:0/996859 conn(0x5623ade24b30 msgr2=0x5623ade29fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:39:48.500 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.499+0000 7f5f30be3740 1 -- 172.21.15.97:0/996859 shutdown_connections 2024-04-23T10:39:48.500 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.499+0000 7f5f30be3740 1 -- 172.21.15.97:0/996859 wait complete. 2024-04-23T10:39:48.500 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.499+0000 7f5f30be3740 1 -- start start 2024-04-23T10:39:48.503 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.500+0000 7f5f30be3740 1 Processor -- start 2024-04-23T10:39:48.503 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.500+0000 7f5f30be3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623ade28620 0x5623ade2db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:48.503 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.500+0000 7f5f30be3740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623ade28f40 0x5623ade2e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:48.503 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.500+0000 7f5f30be3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5623ade33800 0x5623ade2c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:48.503 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.500+0000 7f5f30be3740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5f281aef60 con 0x5623ade28f40 2024-04-23T10:39:48.503 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.500+0000 7f5f30be3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5f281ae890 con 0x5623ade33800 2024-04-23T10:39:48.503 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.500+0000 7f5f30be3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5f281abaf0 con 0x5623ade28620 2024-04-23T10:39:48.503 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.500+0000 7f5f30be3740 10 client.0 _ll_register_callbacks cb 0x7f5f1c09c270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:39:48.503 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.500+0000 7f5f26ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623ade28f40 0x5623ade2e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:48.503 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.500+0000 7f5f26ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623ade28f40 0x5623ade2e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48700/0 (socket says 172.21.15.97:48700) 2024-04-23T10:39:48.504 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.500+0000 7f5f26ffd640 1 -- 172.21.15.97:0/2599040839 learned_addr learned my addr 172.21.15.97:0/2599040839 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:39:48.504 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.500+0000 7f5f277fe640 1 --2- 172.21.15.97:0/2599040839 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623ade28620 0x5623ade2db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:48.504 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.500+0000 7f5f27fff640 1 --2- 172.21.15.97:0/2599040839 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5623ade33800 0x5623ade2c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:48.504 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.500+0000 7f5f26ffd640 1 -- 172.21.15.97:0/2599040839 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623ade28620 msgr2=0x5623ade2db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:39:48.504 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.500+0000 7f5f26ffd640 1 --2- 172.21.15.97:0/2599040839 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623ade28620 0x5623ade2db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:48.504 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.500+0000 7f5f26ffd640 1 -- 172.21.15.97:0/2599040839 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5623ade33800 msgr2=0x5623ade2c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:39:48.504 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.500+0000 7f5f26ffd640 1 --2- 172.21.15.97:0/2599040839 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5623ade33800 0x5623ade2c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:48.504 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.500+0000 7f5f26ffd640 1 -- 172.21.15.97:0/2599040839 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623ade2cf80 con 0x5623ade28f40 2024-04-23T10:39:48.504 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.500+0000 7f5f27fff640 1 --2- 172.21.15.97:0/2599040839 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5623ade33800 0x5623ade2c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:39:48.504 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.500+0000 7f5f26ffd640 1 --2- 172.21.15.97:0/2599040839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623ade28f40 0x5623ade2e050 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c09c430 tx=0x7f5f1c0ce100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1446720968b829b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:39:48.504 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.500+0000 7f5f06ffd640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:39:48.504 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.501+0000 7f5f06ffd640 1 -- 172.21.15.97:0/2599040839 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c009de0 con 0x5623ade28f40 2024-04-23T10:39:48.504 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.501+0000 7f5f06ffd640 1 -- 172.21.15.97:0/2599040839 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8484+0+0 (secure 0 0 0) 0x7f5f1c022bb0 con 0x5623ade28f40 2024-04-23T10:39:48.504 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.501+0000 7f5f30be3740 1 -- 172.21.15.97:0/2599040839 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f2000ac10 con 0x5623ade28f40 2024-04-23T10:39:48.505 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.501+0000 7f5f06ffd640 1 -- 172.21.15.97:0/2599040839 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5f1c098740 con 0x5623ade28f40 2024-04-23T10:39:48.505 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.501+0000 7f5f30be3740 10 client.17475 fetch_fsmap learned FSMap version 26 2024-04-23T10:39:48.505 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.501+0000 7f5f30be3740 1 -- 172.21.15.97:0/2599040839 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5623adeae740 con 0x5623ade28f40 2024-04-23T10:39:48.505 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.501+0000 7f5f06ffd640 1 -- 172.21.15.97:0/2599040839 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f2000ac10 con 0x5623ade28f40 2024-04-23T10:39:48.505 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.501+0000 7f5f06ffd640 1 -- 172.21.15.97:0/2599040839 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5f1c0bb2a0 con 0x5623ade28f40 2024-04-23T10:39:48.505 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.501+0000 7f5f30be3740 10 client.17475 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:39:48.505 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.501+0000 7f5f30be3740 10 client.17475 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:39:48.505 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.501+0000 7f5f30be3740 20 client.17475 populate_metadata read hostname 'smithi097' 2024-04-23T10:39:48.505 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.501+0000 7f5f30be3740 1 --2- 172.21.15.97:0/2599040839 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5623adeaed30 0x5623adeb1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:39:48.505 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.501+0000 7f5f30be3740 4 client.17475 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:39:48.505 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.501+0000 7f5f30be3740 1 -- 172.21.15.97:0/2599040839 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5623add95200 con 0x5623adeaed30 2024-04-23T10:39:48.505 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.502+0000 7f5f277fe640 1 --2- 172.21.15.97:0/2599040839 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5623adeaed30 0x5623adeb1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:39:48.505 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.502+0000 7f5f277fe640 1 --2- 172.21.15.97:0/2599040839 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5623adeaed30 0x5623adeb1150 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26b755b781dc588e server_cookie=6106f13f1976a4fb in_seq=0 out_seq=0 2024-04-23T10:39:48.505 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.502+0000 7f5f06ffd640 10 client.17475 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:39:48.515 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.516+0000 7f5f06ffd640 1 -- 172.21.15.97:0/2599040839 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5623add95200 con 0x5623adeaed30 2024-04-23T10:39:48.515 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.516+0000 7f5f06ffd640 10 client.17475 handle_command_reply: tid=0 2024-04-23T10:39:48.516 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.516+0000 7f5f30be3740 1 client.17475 shutdown 2024-04-23T10:39:48.516 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.516+0000 7f5f30be3740 1 -- 172.21.15.97:0/2599040839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623ade28f40 msgr2=0x5623ade2e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:39:48.516 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.516+0000 7f5f30be3740 1 --2- 172.21.15.97:0/2599040839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623ade28f40 0x5623ade2e050 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c09c430 tx=0x7f5f1c0ce100 comp rx=0 tx=0).stop 2024-04-23T10:39:48.516 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.516+0000 7f5f30be3740 1 -- 172.21.15.97:0/2599040839 shutdown_connections 2024-04-23T10:39:48.516 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.516+0000 7f5f30be3740 1 --2- 172.21.15.97:0/2599040839 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5623adeaed30 0x5623adeb1150 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:48.516 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.516+0000 7f5f30be3740 1 --2- 172.21.15.97:0/2599040839 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5623ade33800 0x5623ade2c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:48.516 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.516+0000 7f5f30be3740 1 --2- 172.21.15.97:0/2599040839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5623ade28f40 0x5623ade2e050 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:48.516 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.516+0000 7f5f06ffd640 0 client.17475 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:39:48.516 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.516+0000 7f5f30be3740 1 --2- 172.21.15.97:0/2599040839 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5623ade28620 0x5623ade2db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:48.516 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.516+0000 7f5f30be3740 1 -- 172.21.15.97:0/2599040839 >> 172.21.15.97:0/2599040839 conn(0x5623ade24b30 msgr2=0x5623ade23cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:39:48.516 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.516+0000 7f5f30be3740 1 -- 172.21.15.97:0/2599040839 shutdown_connections 2024-04-23T10:39:48.516 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.516+0000 7f5f30be3740 1 -- 172.21.15.97:0/2599040839 wait complete. 2024-04-23T10:39:48.516 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.517+0000 7f5f30be3740 20 client.17475 trim_cache size 0 max 16384 2024-04-23T10:39:48.517 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:39:48.517 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:39:48.517 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:39:48.517 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:39:48.518 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.517+0000 7f5f2decf640 1 -- 172.21.15.97:0/1604548766 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f5f100a3dc0 msgr2=0x7f5f100a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:39:48.518 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.517+0000 7f5f2decf640 1 --2- 172.21.15.97:0/1604548766 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f5f100a3dc0 0x7f5f100a6280 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5f20010ca0 tx=0x7f5f20015000 comp rx=0 tx=0).stop 2024-04-23T10:39:48.518 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.517+0000 7f5f2decf640 1 -- 172.21.15.97:0/1604548766 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f28107290 msgr2=0x7f5f28198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:39:48.518 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.517+0000 7f5f2decf640 1 --2- 172.21.15.97:0/1604548766 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f28107290 0x7f5f28198c70 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c005fd0 tx=0x7f5f1c00dde0 comp rx=0 tx=0).stop 2024-04-23T10:39:48.518 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.517+0000 7f5f2decf640 1 -- 172.21.15.97:0/1604548766 shutdown_connections 2024-04-23T10:39:48.518 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.517+0000 7f5f2decf640 1 --2- 172.21.15.97:0/1604548766 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f5f100a3dc0 0x7f5f100a6280 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:48.518 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.517+0000 7f5f2decf640 1 --2- 172.21.15.97:0/1604548766 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f2819cfa0 0x7f5f2819d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:48.518 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.517+0000 7f5f2decf640 1 --2- 172.21.15.97:0/1604548766 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f28107290 0x7f5f28198c70 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:48.518 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.517+0000 7f5f2decf640 1 --2- 172.21.15.97:0/1604548766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f281021c0 0x7f5f28198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:39:48.518 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.517+0000 7f5f2decf640 1 -- 172.21.15.97:0/1604548766 >> 172.21.15.97:0/1604548766 conn(0x7f5f280fda20 msgr2=0x7f5f280ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:39:48.518 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.517+0000 7f5f2decf640 1 -- 172.21.15.97:0/1604548766 shutdown_connections 2024-04-23T10:39:48.518 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:39:48.517+0000 7f5f2decf640 1 -- 172.21.15.97:0/1604548766 wait complete. 2024-04-23T10:39:48.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:39:48 smithi196 ceph-mon[32919]: pgmap v425: 97 pgs: 97 active+clean; 3.5 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-23T10:39:48.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:39:48 smithi097 ceph-mon[29197]: pgmap v425: 97 pgs: 97 active+clean; 3.5 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-23T10:39:48.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:39:48 smithi183 ceph-mon[32317]: pgmap v425: 97 pgs: 97 active+clean; 3.5 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-23T10:39:50.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:39:50 smithi196 ceph-mon[32919]: pgmap v426: 97 pgs: 97 active+clean; 3.5 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-23T10:39:50.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:39:50 smithi097 ceph-mon[29197]: pgmap v426: 97 pgs: 97 active+clean; 3.5 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-23T10:39:50.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:39:50 smithi183 ceph-mon[32317]: pgmap v426: 97 pgs: 97 active+clean; 3.5 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-23T10:39:52.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:39:52 smithi196 ceph-mon[32919]: pgmap v427: 97 pgs: 97 active+clean; 3.5 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-23T10:39:52.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:39:52 smithi097 ceph-mon[29197]: pgmap v427: 97 pgs: 97 active+clean; 3.5 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-23T10:39:52.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:39:52 smithi183 ceph-mon[32317]: pgmap v427: 97 pgs: 97 active+clean; 3.5 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-23T10:39:54.704 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:39:54 smithi183 ceph-mon[32317]: pgmap v428: 97 pgs: 97 active+clean; 3.7 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-23T10:39:54.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:39:54 smithi196 ceph-mon[32919]: pgmap v428: 97 pgs: 97 active+clean; 3.7 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-23T10:39:54.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:39:54 smithi097 ceph-mon[29197]: pgmap v428: 97 pgs: 97 active+clean; 3.7 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-23T10:39:55.865 INFO:tasks.workunit.client.0.smithi097.stdout:437/600: open fd = 3 2024-04-23T10:39:55.865 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.865 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.865 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:39:55.865 INFO:tasks.workunit.client.0.smithi097.stdout:438/600: open fd = 3 2024-04-23T10:39:55.865 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.865 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.865 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:39:55.865 INFO:tasks.workunit.client.0.smithi097.stdout:439/600: open fd = 3 2024-04-23T10:39:55.865 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.866 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.866 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:39:55.866 INFO:tasks.workunit.client.0.smithi097.stdout:440/600: open fd = 3 2024-04-23T10:39:55.866 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.866 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.866 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:39:55.866 INFO:tasks.workunit.client.0.smithi097.stdout:441/600: open fd = 3 2024-04-23T10:39:55.866 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.866 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.866 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:39:55.866 INFO:tasks.workunit.client.0.smithi097.stdout:442/600: open fd = 3 2024-04-23T10:39:55.866 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.866 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.866 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:39:55.866 INFO:tasks.workunit.client.0.smithi097.stdout:443/600: open fd = 3 2024-04-23T10:39:55.867 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.867 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.867 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:39:55.867 INFO:tasks.workunit.client.0.smithi097.stdout:444/600: open fd = 3 2024-04-23T10:39:55.867 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.867 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.867 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:39:55.867 INFO:tasks.workunit.client.0.smithi097.stdout:445/600: open fd = 3 2024-04-23T10:39:55.867 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.867 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.867 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:39:55.867 INFO:tasks.workunit.client.0.smithi097.stdout:446/600: open fd = 3 2024-04-23T10:39:55.867 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.867 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.867 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:39:55.867 INFO:tasks.workunit.client.0.smithi097.stdout:447/600: open fd = 3 2024-04-23T10:39:55.868 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.868 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.868 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:39:55.868 INFO:tasks.workunit.client.0.smithi097.stdout:448/600: open fd = 3 2024-04-23T10:39:55.868 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.868 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.868 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:39:55.868 INFO:tasks.workunit.client.0.smithi097.stdout:449/600: open fd = 3 2024-04-23T10:39:55.868 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.868 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.868 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:39:55.868 INFO:tasks.workunit.client.0.smithi097.stdout:450/600: open fd = 3 2024-04-23T10:39:55.868 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.868 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.868 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:39:55.869 INFO:tasks.workunit.client.0.smithi097.stdout:451/600: open fd = 3 2024-04-23T10:39:55.869 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.869 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.869 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:39:55.869 INFO:tasks.workunit.client.0.smithi097.stdout:452/600: open fd = 3 2024-04-23T10:39:55.869 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.869 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.869 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:39:55.869 INFO:tasks.workunit.client.0.smithi097.stdout:453/600: open fd = 3 2024-04-23T10:39:55.869 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.869 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.869 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:39:55.869 INFO:tasks.workunit.client.0.smithi097.stdout:454/600: open fd = 3 2024-04-23T10:39:55.869 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.869 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.870 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:39:55.870 INFO:tasks.workunit.client.0.smithi097.stdout:455/600: open fd = 3 2024-04-23T10:39:55.870 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.870 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.870 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:39:55.870 INFO:tasks.workunit.client.0.smithi097.stdout:456/600: open fd = 3 2024-04-23T10:39:55.870 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.870 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.870 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:39:55.870 INFO:tasks.workunit.client.0.smithi097.stdout:457/600: open fd = 3 2024-04-23T10:39:55.870 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.870 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.870 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:39:55.870 INFO:tasks.workunit.client.0.smithi097.stdout:458/600: open fd = 3 2024-04-23T10:39:55.870 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.871 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.871 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:39:55.871 INFO:tasks.workunit.client.0.smithi097.stdout:459/600: open fd = 3 2024-04-23T10:39:55.871 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.871 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.871 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:39:55.871 INFO:tasks.workunit.client.0.smithi097.stdout:460/600: open fd = 3 2024-04-23T10:39:55.871 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.871 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.871 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:39:55.871 INFO:tasks.workunit.client.0.smithi097.stdout:461/600: open fd = 3 2024-04-23T10:39:55.871 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.871 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.871 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:39:55.871 INFO:tasks.workunit.client.0.smithi097.stdout:462/600: open fd = 3 2024-04-23T10:39:55.872 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.872 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.872 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:39:55.872 INFO:tasks.workunit.client.0.smithi097.stdout:463/600: open fd = 3 2024-04-23T10:39:55.872 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.872 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.872 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:39:55.872 INFO:tasks.workunit.client.0.smithi097.stdout:464/600: open fd = 3 2024-04-23T10:39:55.872 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.872 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.872 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:39:55.872 INFO:tasks.workunit.client.0.smithi097.stdout:465/600: open fd = 3 2024-04-23T10:39:55.872 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.873 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.873 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:39:55.873 INFO:tasks.workunit.client.0.smithi097.stdout:466/600: open fd = 3 2024-04-23T10:39:55.873 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.873 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.873 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:39:55.873 INFO:tasks.workunit.client.0.smithi097.stdout:467/600: open fd = 3 2024-04-23T10:39:55.873 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.873 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.873 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:39:55.873 INFO:tasks.workunit.client.0.smithi097.stdout:468/600: open fd = 3 2024-04-23T10:39:55.873 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.873 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.874 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:39:55.874 INFO:tasks.workunit.client.0.smithi097.stdout:469/600: open fd = 3 2024-04-23T10:39:55.874 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.874 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.874 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:39:55.874 INFO:tasks.workunit.client.0.smithi097.stdout:470/600: open fd = 3 2024-04-23T10:39:55.874 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.874 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.874 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:39:55.874 INFO:tasks.workunit.client.0.smithi097.stdout:471/600: open fd = 3 2024-04-23T10:39:55.874 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.874 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.874 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:39:55.874 INFO:tasks.workunit.client.0.smithi097.stdout:472/600: open fd = 3 2024-04-23T10:39:55.875 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.875 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.875 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:39:55.875 INFO:tasks.workunit.client.0.smithi097.stdout:473/600: open fd = 3 2024-04-23T10:39:55.875 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.875 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.875 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:39:55.875 INFO:tasks.workunit.client.0.smithi097.stdout:474/600: open fd = 3 2024-04-23T10:39:55.875 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.875 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.875 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:39:55.875 INFO:tasks.workunit.client.0.smithi097.stdout:475/600: open fd = 3 2024-04-23T10:39:55.875 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.876 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.876 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:39:55.876 INFO:tasks.workunit.client.0.smithi097.stdout:476/600: open fd = 3 2024-04-23T10:39:55.876 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.876 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.876 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:39:55.876 INFO:tasks.workunit.client.0.smithi097.stdout:477/600: open fd = 3 2024-04-23T10:39:55.876 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.876 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.876 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:39:55.876 INFO:tasks.workunit.client.0.smithi097.stdout:478/600: open fd = 3 2024-04-23T10:39:55.876 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.876 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.876 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:39:55.876 INFO:tasks.workunit.client.0.smithi097.stdout:479/600: open fd = 3 2024-04-23T10:39:55.877 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.877 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.877 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:39:55.877 INFO:tasks.workunit.client.0.smithi097.stdout:480/600: open fd = 3 2024-04-23T10:39:55.877 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.877 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.877 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:39:55.877 INFO:tasks.workunit.client.0.smithi097.stdout:481/600: open fd = 3 2024-04-23T10:39:55.877 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.877 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.877 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:39:55.877 INFO:tasks.workunit.client.0.smithi097.stdout:482/600: open fd = 3 2024-04-23T10:39:55.877 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.877 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.878 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:39:55.878 INFO:tasks.workunit.client.0.smithi097.stdout:483/600: open fd = 3 2024-04-23T10:39:55.878 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.878 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.878 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:39:55.878 INFO:tasks.workunit.client.0.smithi097.stdout:484/600: open fd = 3 2024-04-23T10:39:55.878 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.878 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.878 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:39:55.878 INFO:tasks.workunit.client.0.smithi097.stdout:485/600: open fd = 3 2024-04-23T10:39:55.878 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.878 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.878 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:39:55.878 INFO:tasks.workunit.client.0.smithi097.stdout:486/600: open fd = 3 2024-04-23T10:39:55.879 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.879 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.879 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:39:55.879 INFO:tasks.workunit.client.0.smithi097.stdout:487/600: open fd = 3 2024-04-23T10:39:55.879 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.879 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.879 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:39:55.879 INFO:tasks.workunit.client.0.smithi097.stdout:488/600: open fd = 3 2024-04-23T10:39:55.879 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.879 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.879 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:39:55.879 INFO:tasks.workunit.client.0.smithi097.stdout:489/600: open fd = 3 2024-04-23T10:39:55.879 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.879 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.880 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:39:55.880 INFO:tasks.workunit.client.0.smithi097.stdout:490/600: open fd = 3 2024-04-23T10:39:55.880 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.880 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.880 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:39:55.880 INFO:tasks.workunit.client.0.smithi097.stdout:491/600: open fd = 3 2024-04-23T10:39:55.880 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.880 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.880 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:39:55.880 INFO:tasks.workunit.client.0.smithi097.stdout:492/600: open fd = 3 2024-04-23T10:39:55.880 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.880 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.881 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:39:55.881 INFO:tasks.workunit.client.0.smithi097.stdout:493/600: open fd = 3 2024-04-23T10:39:55.881 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.881 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.881 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:39:55.881 INFO:tasks.workunit.client.0.smithi097.stdout:494/600: open fd = 3 2024-04-23T10:39:55.881 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.881 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.881 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:39:55.881 INFO:tasks.workunit.client.0.smithi097.stdout:495/600: open fd = 3 2024-04-23T10:39:55.881 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.881 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.881 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:39:55.881 INFO:tasks.workunit.client.0.smithi097.stdout:496/600: open fd = 3 2024-04-23T10:39:55.882 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.882 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.882 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:39:55.882 INFO:tasks.workunit.client.0.smithi097.stdout:497/600: open fd = 3 2024-04-23T10:39:55.882 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.882 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.882 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:39:55.882 INFO:tasks.workunit.client.0.smithi097.stdout:498/600: open fd = 3 2024-04-23T10:39:55.882 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.882 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:39:55.882 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:39:56.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:39:56 smithi196 ceph-mon[32919]: pgmap v429: 97 pgs: 97 active+clean; 3.7 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:39:56.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:39:56 smithi097 ceph-mon[29197]: pgmap v429: 97 pgs: 97 active+clean; 3.7 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:39:56.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:39:56 smithi183 ceph-mon[32317]: pgmap v429: 97 pgs: 97 active+clean; 3.7 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:39:58.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:39:58 smithi196 ceph-mon[32919]: pgmap v430: 97 pgs: 97 active+clean; 3.9 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-23T10:39:58.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:39:58 smithi097 ceph-mon[29197]: pgmap v430: 97 pgs: 97 active+clean; 3.9 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-23T10:39:58.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:39:58 smithi183 ceph-mon[32317]: pgmap v430: 97 pgs: 97 active+clean; 3.9 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-23T10:40:00.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:40:00 smithi196 ceph-mon[32919]: pgmap v431: 97 pgs: 97 active+clean; 3.9 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:40:00.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:40:00 smithi196 ceph-mon[32919]: overall HEALTH_OK 2024-04-23T10:40:00.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:40:00 smithi097 ceph-mon[29197]: pgmap v431: 97 pgs: 97 active+clean; 3.9 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:40:00.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:40:00 smithi097 ceph-mon[29197]: overall HEALTH_OK 2024-04-23T10:40:00.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:40:00 smithi183 ceph-mon[32317]: pgmap v431: 97 pgs: 97 active+clean; 3.9 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:40:00.901 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:40:00 smithi183 ceph-mon[32317]: overall HEALTH_OK 2024-04-23T10:40:01.613 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.611+0000 7fbf3440b640 1 -- 172.21.15.97:0/3606123087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf2c106180 msgr2=0x7fbf2c10dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:40:01.613 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.611+0000 7fbf3440b640 1 --2- 172.21.15.97:0/3606123087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf2c106180 0x7fbf2c10dd50 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fbf24009ca0 tx=0x7fbf2402a8c0 comp rx=0 tx=0).stop 2024-04-23T10:40:01.613 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.612+0000 7fbf3440b640 1 -- 172.21.15.97:0/3606123087 shutdown_connections 2024-04-23T10:40:01.613 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.612+0000 7fbf3440b640 1 --2- 172.21.15.97:0/3606123087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf2c106180 0x7fbf2c10dd50 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:01.613 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.612+0000 7fbf3440b640 1 --2- 172.21.15.97:0/3606123087 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf2c101a60 0x7fbf2c105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:01.613 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.612+0000 7fbf3440b640 1 --2- 172.21.15.97:0/3606123087 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf2c1010b0 0x7fbf2c101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.612+0000 7fbf3440b640 1 -- 172.21.15.97:0/3606123087 >> 172.21.15.97:0/3606123087 conn(0x7fbf2c078000 msgr2=0x7fbf2c078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:40:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.612+0000 7fbf3440b640 1 -- 172.21.15.97:0/3606123087 shutdown_connections 2024-04-23T10:40:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.612+0000 7fbf3440b640 1 -- 172.21.15.97:0/3606123087 wait complete. 2024-04-23T10:40:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.612+0000 7fbf3440b640 1 Processor -- start 2024-04-23T10:40:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.612+0000 7fbf3440b640 1 -- start start 2024-04-23T10:40:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.612+0000 7fbf3440b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf2c1010b0 0x7fbf2c198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.612+0000 7fbf3440b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf2c106180 0x7fbf2c198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.612+0000 7fbf3440b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf2c19cf20 0x7fbf2c19d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.612+0000 7fbf3440b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbf2c1110f0 con 0x7fbf2c19cf20 2024-04-23T10:40:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.612+0000 7fbf3440b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbf2c110f70 con 0x7fbf2c1010b0 2024-04-23T10:40:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.612+0000 7fbf3440b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbf2c111270 con 0x7fbf2c106180 2024-04-23T10:40:01.615 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.613+0000 7fbf329a8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf2c19cf20 0x7fbf2c19d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:01.615 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.613+0000 7fbf329a8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf2c19cf20 0x7fbf2c19d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33290/0 (socket says 172.21.15.97:33290) 2024-04-23T10:40:01.615 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.613+0000 7fbf329a8640 1 -- 172.21.15.97:0/495922378 learned_addr learned my addr 172.21.15.97:0/495922378 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:40:01.615 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.613+0000 7fbf321a7640 1 --2- 172.21.15.97:0/495922378 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf2c1010b0 0x7fbf2c198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:01.615 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.613+0000 7fbf319a6640 1 --2- 172.21.15.97:0/495922378 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf2c106180 0x7fbf2c198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:01.615 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.613+0000 7fbf329a8640 1 -- 172.21.15.97:0/495922378 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf2c106180 msgr2=0x7fbf2c198ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:40:01.615 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.613+0000 7fbf329a8640 1 --2- 172.21.15.97:0/495922378 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf2c106180 0x7fbf2c198ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:01.615 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.613+0000 7fbf329a8640 1 -- 172.21.15.97:0/495922378 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf2c1010b0 msgr2=0x7fbf2c198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:40:01.615 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.613+0000 7fbf329a8640 1 --2- 172.21.15.97:0/495922378 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf2c1010b0 0x7fbf2c198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:01.615 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.613+0000 7fbf329a8640 1 -- 172.21.15.97:0/495922378 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf2c19da10 con 0x7fbf2c19cf20 2024-04-23T10:40:01.615 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.613+0000 7fbf321a7640 1 --2- 172.21.15.97:0/495922378 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf2c1010b0 0x7fbf2c198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T10:40:01.615 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.613+0000 7fbf329a8640 1 --2- 172.21.15.97:0/495922378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf2c19cf20 0x7fbf2c19d3d0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fbf24005cc0 tx=0x7fbf24031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:40:01.615 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.613+0000 7fbf237fe640 1 -- 172.21.15.97:0/495922378 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf24003b70 con 0x7fbf2c19cf20 2024-04-23T10:40:01.616 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.613+0000 7fbf3440b640 1 -- 172.21.15.97:0/495922378 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf2c19dca0 con 0x7fbf2c19cf20 2024-04-23T10:40:01.616 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.613+0000 7fbf237fe640 1 -- 172.21.15.97:0/495922378 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf240318f0 con 0x7fbf2c19cf20 2024-04-23T10:40:01.616 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.613+0000 7fbf237fe640 1 -- 172.21.15.97:0/495922378 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf2402f950 con 0x7fbf2c19cf20 2024-04-23T10:40:01.616 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.613+0000 7fbf3440b640 1 -- 172.21.15.97:0/495922378 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf2c1a54f0 con 0x7fbf2c19cf20 2024-04-23T10:40:01.625 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.615+0000 7fbf237fe640 1 -- 172.21.15.97:0/495922378 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbf2402fb50 con 0x7fbf2c19cf20 2024-04-23T10:40:01.625 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.616+0000 7fbf371f5740 1 Processor -- start 2024-04-23T10:40:01.625 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.616+0000 7fbf371f5740 1 -- start start 2024-04-23T10:40:01.625 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.616+0000 7fbf237fe640 1 --2- 172.21.15.97:0/495922378 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fbf180a3dc0 0x7fbf180a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:01.625 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.616+0000 7fbf371f5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612d1c58620 0x5612d1c58a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:01.625 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.616+0000 7fbf371f5740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612d1c58f40 0x5612d1c632c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:01.625 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.617+0000 7fbf371f5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612d1c63800 0x5612d1c65bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:01.625 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.617+0000 7fbf371f5740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5612d1bbd970 con 0x5612d1c58f40 2024-04-23T10:40:01.625 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.617+0000 7fbf371f5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5612d1b5a4a0 con 0x5612d1c63800 2024-04-23T10:40:01.626 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.617+0000 7fbf371f5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5612d1bb9930 con 0x5612d1c58620 2024-04-23T10:40:01.626 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.617+0000 7fbf319a6640 1 --2- 172.21.15.97:0/495922378 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fbf180a3dc0 0x7fbf180a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:01.626 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.617+0000 7fbf319a6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612d1c58f40 0x5612d1c632c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:01.626 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.617+0000 7fbf319a6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612d1c58f40 0x5612d1c632c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33304/0 (socket says 172.21.15.97:33304) 2024-04-23T10:40:01.626 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.617+0000 7fbf319a6640 1 -- 172.21.15.97:0/2709521104 learned_addr learned my addr 172.21.15.97:0/2709521104 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:40:01.626 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.617+0000 7fbf237fe640 1 -- 172.21.15.97:0/495922378 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8484+0+0 (secure 0 0 0) 0x7fbf240f1a40 con 0x7fbf2c19cf20 2024-04-23T10:40:01.626 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.617+0000 7fbf321a7640 1 --2- 172.21.15.97:0/2709521104 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612d1c58620 0x5612d1c58a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:01.626 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.617+0000 7fbf329a8640 1 --2- 172.21.15.97:0/2709521104 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612d1c63800 0x5612d1c65bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:01.626 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.617+0000 7fbf319a6640 1 -- 172.21.15.97:0/2709521104 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612d1c58620 msgr2=0x5612d1c58a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:40:01.626 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.617+0000 7fbf319a6640 1 --2- 172.21.15.97:0/2709521104 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612d1c58620 0x5612d1c58a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:01.626 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.617+0000 7fbf319a6640 1 -- 172.21.15.97:0/2709521104 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612d1c63800 msgr2=0x5612d1c65bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:40:01.626 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.617+0000 7fbf319a6640 1 --2- 172.21.15.97:0/2709521104 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612d1c63800 0x5612d1c65bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:01.627 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.617+0000 7fbf319a6640 1 -- 172.21.15.97:0/2709521104 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612d18d0210 con 0x5612d1c58f40 2024-04-23T10:40:01.627 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.617+0000 7fbf319a6640 1 --2- 172.21.15.97:0/2709521104 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612d1c58f40 0x5612d1c632c0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c010a30 tx=0x7fbf1c010ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aab29424b9a659 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:40:01.627 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.617+0000 7fbf319a6640 1 --2- 172.21.15.97:0/495922378 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fbf180a3dc0 0x7fbf180a6280 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c00f850 tx=0x7fbf1c018000 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:40:01.627 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.617+0000 7fbf20bfe640 1 -- 172.21.15.97:0/2709521104 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c022070 con 0x5612d1c58f40 2024-04-23T10:40:01.627 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.617+0000 7fbf20bfe640 1 -- 172.21.15.97:0/2709521104 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf1c010430 con 0x5612d1c58f40 2024-04-23T10:40:01.627 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.617+0000 7fbf20bfe640 1 -- 172.21.15.97:0/2709521104 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c019040 con 0x5612d1c58f40 2024-04-23T10:40:01.627 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.618+0000 7fbf371f5740 1 -- 172.21.15.97:0/2709521104 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612d1c58f40 msgr2=0x5612d1c632c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:40:01.627 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.618+0000 7fbf371f5740 1 --2- 172.21.15.97:0/2709521104 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612d1c58f40 0x5612d1c632c0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c010a30 tx=0x7fbf1c010ef0 comp rx=0 tx=0).stop 2024-04-23T10:40:01.627 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.618+0000 7fbf371f5740 1 -- 172.21.15.97:0/2709521104 shutdown_connections 2024-04-23T10:40:01.627 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.618+0000 7fbf371f5740 1 --2- 172.21.15.97:0/2709521104 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612d1c63800 0x5612d1c65bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:01.627 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.618+0000 7fbf371f5740 1 --2- 172.21.15.97:0/2709521104 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612d1c58f40 0x5612d1c632c0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:01.627 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.618+0000 7fbf371f5740 1 --2- 172.21.15.97:0/2709521104 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612d1c58620 0x5612d1c58a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:01.627 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.618+0000 7fbf371f5740 1 -- 172.21.15.97:0/2709521104 >> 172.21.15.97:0/2709521104 conn(0x5612d1c54b30 msgr2=0x5612d1c54f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:40:01.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.618+0000 7fbf371f5740 1 -- 172.21.15.97:0/2709521104 shutdown_connections 2024-04-23T10:40:01.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.618+0000 7fbf371f5740 1 -- 172.21.15.97:0/2709521104 wait complete. 2024-04-23T10:40:01.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.619+0000 7fbf371f5740 1 -- start start 2024-04-23T10:40:01.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.619+0000 7fbf371f5740 1 Processor -- start 2024-04-23T10:40:01.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.619+0000 7fbf371f5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612d1c58620 0x5612d1c63150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:01.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.619+0000 7fbf371f5740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612d1c58f40 0x5612d1c604b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:01.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.619+0000 7fbf371f5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612d1c63800 0x5612d1c60a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:01.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.619+0000 7fbf371f5740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5612d1c663d0 con 0x5612d1c58f40 2024-04-23T10:40:01.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.619+0000 7fbf371f5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5612d1b580b0 con 0x5612d1c63800 2024-04-23T10:40:01.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.619+0000 7fbf371f5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5612d1c66550 con 0x5612d1c58620 2024-04-23T10:40:01.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.619+0000 7fbf371f5740 10 client.0 _ll_register_callbacks cb 0x5612d1b45d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:40:01.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.619+0000 7fbf319a6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612d1c58f40 0x5612d1c604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:01.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.619+0000 7fbf319a6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612d1c58f40 0x5612d1c604b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33310/0 (socket says 172.21.15.97:33310) 2024-04-23T10:40:01.628 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.619+0000 7fbf319a6640 1 -- 172.21.15.97:0/3730865350 learned_addr learned my addr 172.21.15.97:0/3730865350 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:40:01.629 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.619+0000 7fbf321a7640 1 --2- 172.21.15.97:0/3730865350 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612d1c58620 0x5612d1c63150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:01.629 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.619+0000 7fbf329a8640 1 --2- 172.21.15.97:0/3730865350 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612d1c63800 0x5612d1c60a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:01.629 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.619+0000 7fbf319a6640 1 -- 172.21.15.97:0/3730865350 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612d1c58620 msgr2=0x5612d1c63150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:40:01.629 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.619+0000 7fbf319a6640 1 --2- 172.21.15.97:0/3730865350 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612d1c58620 0x5612d1c63150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:01.629 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.619+0000 7fbf319a6640 1 -- 172.21.15.97:0/3730865350 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612d1c63800 msgr2=0x5612d1c60a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:40:01.629 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.619+0000 7fbf319a6640 1 --2- 172.21.15.97:0/3730865350 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612d1c63800 0x5612d1c60a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:01.629 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.619+0000 7fbf319a6640 1 -- 172.21.15.97:0/3730865350 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612d199c620 con 0x5612d1c58f40 2024-04-23T10:40:01.629 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.619+0000 7fbf329a8640 1 --2- 172.21.15.97:0/3730865350 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612d1c63800 0x5612d1c60a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:40:01.629 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.620+0000 7fbf319a6640 1 --2- 172.21.15.97:0/3730865350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612d1c58f40 0x5612d1c604b0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c0105e0 tx=0x7fbf1c005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5906d497bac49ddb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:40:01.629 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.620+0000 7fbf15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:40:01.629 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.620+0000 7fbf15ffb640 1 -- 172.21.15.97:0/3730865350 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c022070 con 0x5612d1c58f40 2024-04-23T10:40:01.629 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.620+0000 7fbf15ffb640 1 -- 172.21.15.97:0/3730865350 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8484+0+0 (secure 0 0 0) 0x7fbf1c01e830 con 0x5612d1c58f40 2024-04-23T10:40:01.629 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.620+0000 7fbf371f5740 1 -- 172.21.15.97:0/3730865350 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf1c010430 con 0x5612d1c58f40 2024-04-23T10:40:01.629 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.620+0000 7fbf15ffb640 1 -- 172.21.15.97:0/3730865350 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbf1c037050 con 0x5612d1c58f40 2024-04-23T10:40:01.630 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.620+0000 7fbf371f5740 10 client.17487 fetch_fsmap learned FSMap version 26 2024-04-23T10:40:01.630 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.620+0000 7fbf371f5740 1 -- 172.21.15.97:0/3730865350 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5612d1c72360 con 0x5612d1c58f40 2024-04-23T10:40:01.630 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.620+0000 7fbf15ffb640 1 -- 172.21.15.97:0/3730865350 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf1c02b020 con 0x5612d1c58f40 2024-04-23T10:40:01.630 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.620+0000 7fbf15ffb640 1 -- 172.21.15.97:0/3730865350 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fbf1c0035f0 con 0x5612d1c58f40 2024-04-23T10:40:01.630 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.620+0000 7fbf371f5740 10 client.17487 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:40:01.630 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.620+0000 7fbf371f5740 10 client.17487 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:40:01.630 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.620+0000 7fbf371f5740 20 client.17487 populate_metadata read hostname 'smithi097' 2024-04-23T10:40:01.630 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.620+0000 7fbf371f5740 1 --2- 172.21.15.97:0/3730865350 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5612d1c73820 0x5612d1c75c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:01.630 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.620+0000 7fbf371f5740 4 client.17487 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:40:01.630 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.620+0000 7fbf371f5740 1 -- 172.21.15.97:0/3730865350 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612d1bcb390 con 0x5612d1c73820 2024-04-23T10:40:01.630 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.621+0000 7fbf321a7640 1 --2- 172.21.15.97:0/3730865350 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5612d1c73820 0x5612d1c75c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:01.630 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.621+0000 7fbf15ffb640 10 client.17487 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:40:01.630 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.621+0000 7fbf321a7640 1 --2- 172.21.15.97:0/3730865350 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5612d1c73820 0x5612d1c75c10 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2f79e6051a19c1a server_cookie=433735aeae6f1b7c in_seq=0 out_seq=0 2024-04-23T10:40:01.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.622+0000 7fbf15ffb640 1 -- 172.21.15.97:0/3730865350 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5612d1bcb390 con 0x5612d1c73820 2024-04-23T10:40:01.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.622+0000 7fbf15ffb640 10 client.17487 handle_command_reply: tid=0 2024-04-23T10:40:01.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.622+0000 7fbf371f5740 1 client.17487 shutdown 2024-04-23T10:40:01.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.622+0000 7fbf371f5740 1 -- 172.21.15.97:0/3730865350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612d1c58f40 msgr2=0x5612d1c604b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:40:01.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.622+0000 7fbf371f5740 1 --2- 172.21.15.97:0/3730865350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612d1c58f40 0x5612d1c604b0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c0105e0 tx=0x7fbf1c005270 comp rx=0 tx=0).stop 2024-04-23T10:40:01.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.622+0000 7fbf371f5740 1 -- 172.21.15.97:0/3730865350 shutdown_connections 2024-04-23T10:40:01.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.622+0000 7fbf371f5740 1 --2- 172.21.15.97:0/3730865350 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5612d1c73820 0x5612d1c75c10 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:01.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.622+0000 7fbf371f5740 1 --2- 172.21.15.97:0/3730865350 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612d1c63800 0x5612d1c60a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:01.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.622+0000 7fbf371f5740 1 --2- 172.21.15.97:0/3730865350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612d1c58f40 0x5612d1c604b0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:01.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.622+0000 7fbf15ffb640 0 client.17487 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:40:01.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.622+0000 7fbf371f5740 1 --2- 172.21.15.97:0/3730865350 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612d1c58620 0x5612d1c63150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:01.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.622+0000 7fbf371f5740 1 -- 172.21.15.97:0/3730865350 >> 172.21.15.97:0/3730865350 conn(0x5612d1c54b30 msgr2=0x5612d1c53cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:40:01.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.622+0000 7fbf371f5740 1 -- 172.21.15.97:0/3730865350 shutdown_connections 2024-04-23T10:40:01.632 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.622+0000 7fbf371f5740 1 -- 172.21.15.97:0/3730865350 wait complete. 2024-04-23T10:40:01.632 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.622+0000 7fbf371f5740 20 client.17487 trim_cache size 0 max 16384 2024-04-23T10:40:01.641 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.637+0000 7fbf371f5740 1 Processor -- start 2024-04-23T10:40:01.641 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.637+0000 7fbf371f5740 1 -- start start 2024-04-23T10:40:01.641 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.637+0000 7fbf371f5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612d1c58620 0x5612d1c6f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:01.641 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.637+0000 7fbf371f5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612d1c58f40 0x5612d1c53b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:01.641 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.637+0000 7fbf371f5740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612d1c63800 0x5612d1c62cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:01.641 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.637+0000 7fbf371f5740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbf2c1ad9e0 con 0x5612d1c63800 2024-04-23T10:40:01.641 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.637+0000 7fbf371f5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbf2c1af520 con 0x5612d1c58620 2024-04-23T10:40:01.641 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.637+0000 7fbf371f5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbf2c1ae0b0 con 0x5612d1c58f40 2024-04-23T10:40:01.641 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.637+0000 7fbf329a8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612d1c63800 0x5612d1c62cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:01.641 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.637+0000 7fbf329a8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612d1c63800 0x5612d1c62cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33324/0 (socket says 172.21.15.97:33324) 2024-04-23T10:40:01.641 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.637+0000 7fbf329a8640 1 -- 172.21.15.97:0/1470547960 learned_addr learned my addr 172.21.15.97:0/1470547960 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:40:01.642 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.637+0000 7fbf319a6640 1 --2- 172.21.15.97:0/1470547960 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612d1c58f40 0x5612d1c53b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:01.642 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.637+0000 7fbf321a7640 1 --2- 172.21.15.97:0/1470547960 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612d1c58620 0x5612d1c6f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:01.642 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.637+0000 7fbf329a8640 1 -- 172.21.15.97:0/1470547960 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612d1c58f40 msgr2=0x5612d1c53b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:40:01.642 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.637+0000 7fbf329a8640 1 --2- 172.21.15.97:0/1470547960 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612d1c58f40 0x5612d1c53b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:01.642 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.637+0000 7fbf329a8640 1 -- 172.21.15.97:0/1470547960 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612d1c58620 msgr2=0x5612d1c6f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:40:01.642 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.637+0000 7fbf329a8640 1 --2- 172.21.15.97:0/1470547960 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612d1c58620 0x5612d1c6f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:01.642 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.637+0000 7fbf329a8640 1 -- 172.21.15.97:0/1470547960 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612d1c5ea20 con 0x5612d1c63800 2024-04-23T10:40:01.642 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.637+0000 7fbf321a7640 1 --2- 172.21.15.97:0/1470547960 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612d1c58620 0x5612d1c6f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:40:01.642 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.637+0000 7fbf329a8640 1 --2- 172.21.15.97:0/1470547960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612d1c63800 0x5612d1c62cf0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fbf240bdaa0 tx=0x7fbf240de040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d4119c104fff6eb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:40:01.642 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.638+0000 7fbf17fff640 1 -- 172.21.15.97:0/1470547960 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf240be210 con 0x5612d1c63800 2024-04-23T10:40:01.642 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.638+0000 7fbf17fff640 1 -- 172.21.15.97:0/1470547960 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf24036040 con 0x5612d1c63800 2024-04-23T10:40:01.642 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.638+0000 7fbf17fff640 1 -- 172.21.15.97:0/1470547960 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf24003d10 con 0x5612d1c63800 2024-04-23T10:40:01.643 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.638+0000 7fbf371f5740 1 -- 172.21.15.97:0/1470547960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612d1c63800 msgr2=0x5612d1c62cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:40:01.643 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.638+0000 7fbf371f5740 1 --2- 172.21.15.97:0/1470547960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612d1c63800 0x5612d1c62cf0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fbf240bdaa0 tx=0x7fbf240de040 comp rx=0 tx=0).stop 2024-04-23T10:40:01.643 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.638+0000 7fbf371f5740 1 -- 172.21.15.97:0/1470547960 shutdown_connections 2024-04-23T10:40:01.643 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.638+0000 7fbf371f5740 1 --2- 172.21.15.97:0/1470547960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612d1c63800 0x5612d1c62cf0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:01.643 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.638+0000 7fbf371f5740 1 --2- 172.21.15.97:0/1470547960 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612d1c58f40 0x5612d1c53b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:01.643 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.638+0000 7fbf371f5740 1 --2- 172.21.15.97:0/1470547960 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612d1c58620 0x5612d1c6f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:01.643 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.638+0000 7fbf371f5740 1 -- 172.21.15.97:0/1470547960 >> 172.21.15.97:0/1470547960 conn(0x5612d1c54b30 msgr2=0x5612d1c53420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:40:01.643 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.638+0000 7fbf371f5740 1 -- 172.21.15.97:0/1470547960 shutdown_connections 2024-04-23T10:40:01.643 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.638+0000 7fbf371f5740 1 -- 172.21.15.97:0/1470547960 wait complete. 2024-04-23T10:40:01.643 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.638+0000 7fbf371f5740 1 -- start start 2024-04-23T10:40:01.643 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.639+0000 7fbf371f5740 1 Processor -- start 2024-04-23T10:40:01.643 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.639+0000 7fbf371f5740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612d1c58620 0x5612d1c5d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:01.643 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.639+0000 7fbf371f5740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612d1c58f40 0x5612d1c5da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:01.643 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.639+0000 7fbf371f5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612d1c63800 0x5612d1cd8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:01.643 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.639+0000 7fbf371f5740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbf2c1aee50 con 0x5612d1c58f40 2024-04-23T10:40:01.644 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.639+0000 7fbf371f5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbf2c1ae780 con 0x5612d1c63800 2024-04-23T10:40:01.644 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.639+0000 7fbf371f5740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbf2c1ab9e0 con 0x5612d1c58620 2024-04-23T10:40:01.644 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.639+0000 7fbf371f5740 10 client.0 _ll_register_callbacks cb 0x7fbf1c00ebd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:40:01.644 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.639+0000 7fbf319a6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612d1c58f40 0x5612d1c5da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:01.644 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.639+0000 7fbf319a6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612d1c58f40 0x5612d1c5da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:33326/0 (socket says 172.21.15.97:33326) 2024-04-23T10:40:01.644 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.639+0000 7fbf319a6640 1 -- 172.21.15.97:0/2831673956 learned_addr learned my addr 172.21.15.97:0/2831673956 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:40:01.644 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.639+0000 7fbf329a8640 1 --2- 172.21.15.97:0/2831673956 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612d1c63800 0x5612d1cd8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:01.644 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.639+0000 7fbf321a7640 1 --2- 172.21.15.97:0/2831673956 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612d1c58620 0x5612d1c5d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:01.644 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.639+0000 7fbf319a6640 1 -- 172.21.15.97:0/2831673956 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612d1c58620 msgr2=0x5612d1c5d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:40:01.644 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.639+0000 7fbf319a6640 1 --2- 172.21.15.97:0/2831673956 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612d1c58620 0x5612d1c5d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:01.644 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.639+0000 7fbf319a6640 1 -- 172.21.15.97:0/2831673956 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612d1c63800 msgr2=0x5612d1cd8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:40:01.644 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.639+0000 7fbf319a6640 1 --2- 172.21.15.97:0/2831673956 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612d1c63800 0x5612d1cd8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:01.644 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.639+0000 7fbf319a6640 1 -- 172.21.15.97:0/2831673956 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612d1c5e340 con 0x5612d1c58f40 2024-04-23T10:40:01.644 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.639+0000 7fbf329a8640 1 --2- 172.21.15.97:0/2831673956 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612d1c63800 0x5612d1cd8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:40:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.639+0000 7fbf319a6640 1 --2- 172.21.15.97:0/2831673956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612d1c58f40 0x5612d1c5da70 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c010010 tx=0x7fbf1c009c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=857b7f4a47d7fdda server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:40:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.639+0000 7fbf15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:40:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.639+0000 7fbf15ffb640 1 -- 172.21.15.97:0/2831673956 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c022040 con 0x5612d1c58f40 2024-04-23T10:40:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.639+0000 7fbf15ffb640 1 -- 172.21.15.97:0/2831673956 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8484+0+0 (secure 0 0 0) 0x7fbf1c00ec80 con 0x5612d1c58f40 2024-04-23T10:40:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.639+0000 7fbf371f5740 1 -- 172.21.15.97:0/2831673956 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf24036040 con 0x5612d1c58f40 2024-04-23T10:40:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.640+0000 7fbf15ffb640 1 -- 172.21.15.97:0/2831673956 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbf1c005a90 con 0x5612d1c58f40 2024-04-23T10:40:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.640+0000 7fbf371f5740 10 client.17493 fetch_fsmap learned FSMap version 26 2024-04-23T10:40:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.640+0000 7fbf371f5740 1 -- 172.21.15.97:0/2831673956 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5612d1cda070 con 0x5612d1c58f40 2024-04-23T10:40:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.640+0000 7fbf15ffb640 1 -- 172.21.15.97:0/2831673956 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf24036040 con 0x5612d1c58f40 2024-04-23T10:40:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.640+0000 7fbf15ffb640 1 -- 172.21.15.97:0/2831673956 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fbf1c042020 con 0x5612d1c58f40 2024-04-23T10:40:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.640+0000 7fbf371f5740 10 client.17493 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:40:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.640+0000 7fbf371f5740 10 client.17493 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:40:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.640+0000 7fbf371f5740 20 client.17493 populate_metadata read hostname 'smithi097' 2024-04-23T10:40:01.646 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.640+0000 7fbf371f5740 1 --2- 172.21.15.97:0/2831673956 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5612d1cda710 0x5612d1cdcb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:01.646 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.640+0000 7fbf371f5740 4 client.17493 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:40:01.646 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.640+0000 7fbf371f5740 1 -- 172.21.15.97:0/2831673956 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5612d1bc5200 con 0x5612d1cda710 2024-04-23T10:40:01.646 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.640+0000 7fbf321a7640 1 --2- 172.21.15.97:0/2831673956 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5612d1cda710 0x5612d1cdcb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:01.646 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.641+0000 7fbf15ffb640 10 client.17493 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:40:01.646 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.641+0000 7fbf321a7640 1 --2- 172.21.15.97:0/2831673956 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5612d1cda710 0x5612d1cdcb30 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36f003a3b33eb50f server_cookie=e6025f1fb3605bac in_seq=0 out_seq=0 2024-04-23T10:40:01.655 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.655+0000 7fbf15ffb640 1 -- 172.21.15.97:0/2831673956 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5612d1bc5200 con 0x5612d1cda710 2024-04-23T10:40:01.655 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.655+0000 7fbf15ffb640 10 client.17493 handle_command_reply: tid=0 2024-04-23T10:40:01.655 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.656+0000 7fbf371f5740 1 client.17493 shutdown 2024-04-23T10:40:01.655 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.656+0000 7fbf371f5740 1 -- 172.21.15.97:0/2831673956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612d1c58f40 msgr2=0x5612d1c5da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:40:01.655 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.656+0000 7fbf371f5740 1 --2- 172.21.15.97:0/2831673956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612d1c58f40 0x5612d1c5da70 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c010010 tx=0x7fbf1c009c50 comp rx=0 tx=0).stop 2024-04-23T10:40:01.655 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.656+0000 7fbf371f5740 1 -- 172.21.15.97:0/2831673956 shutdown_connections 2024-04-23T10:40:01.655 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.656+0000 7fbf371f5740 1 --2- 172.21.15.97:0/2831673956 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5612d1cda710 0x5612d1cdcb30 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:01.656 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.656+0000 7fbf371f5740 1 --2- 172.21.15.97:0/2831673956 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5612d1c63800 0x5612d1cd8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:01.656 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.656+0000 7fbf371f5740 1 --2- 172.21.15.97:0/2831673956 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612d1c58f40 0x5612d1c5da70 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:01.656 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.656+0000 7fbf15ffb640 0 client.17493 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:40:01.656 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.656+0000 7fbf371f5740 1 --2- 172.21.15.97:0/2831673956 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5612d1c58620 0x5612d1c5d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:01.656 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.656+0000 7fbf371f5740 1 -- 172.21.15.97:0/2831673956 >> 172.21.15.97:0/2831673956 conn(0x5612d1c54b30 msgr2=0x5612d1c53cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:40:01.656 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.656+0000 7fbf371f5740 1 -- 172.21.15.97:0/2831673956 shutdown_connections 2024-04-23T10:40:01.656 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.656+0000 7fbf371f5740 1 -- 172.21.15.97:0/2831673956 wait complete. 2024-04-23T10:40:01.656 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.656+0000 7fbf371f5740 20 client.17493 trim_cache size 0 max 16384 2024-04-23T10:40:01.656 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:40:01.656 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:40:01.656 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:40:01.657 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:40:01.657 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.656+0000 7fbf3440b640 1 -- 172.21.15.97:0/495922378 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fbf180a3dc0 msgr2=0x7fbf180a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:40:01.657 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.656+0000 7fbf3440b640 1 --2- 172.21.15.97:0/495922378 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fbf180a3dc0 0x7fbf180a6280 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c00f850 tx=0x7fbf1c018000 comp rx=0 tx=0).stop 2024-04-23T10:40:01.657 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.656+0000 7fbf3440b640 1 -- 172.21.15.97:0/495922378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf2c19cf20 msgr2=0x7fbf2c19d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:40:01.657 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.656+0000 7fbf3440b640 1 --2- 172.21.15.97:0/495922378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf2c19cf20 0x7fbf2c19d3d0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fbf24005cc0 tx=0x7fbf24031700 comp rx=0 tx=0).stop 2024-04-23T10:40:01.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.657+0000 7fbf3440b640 1 -- 172.21.15.97:0/495922378 shutdown_connections 2024-04-23T10:40:01.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.657+0000 7fbf3440b640 1 --2- 172.21.15.97:0/495922378 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fbf180a3dc0 0x7fbf180a6280 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:01.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.657+0000 7fbf3440b640 1 --2- 172.21.15.97:0/495922378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbf2c19cf20 0x7fbf2c19d3d0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:01.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.657+0000 7fbf3440b640 1 --2- 172.21.15.97:0/495922378 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf2c106180 0x7fbf2c198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:01.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.657+0000 7fbf3440b640 1 --2- 172.21.15.97:0/495922378 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf2c1010b0 0x7fbf2c198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:01.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.657+0000 7fbf3440b640 1 -- 172.21.15.97:0/495922378 >> 172.21.15.97:0/495922378 conn(0x7fbf2c078000 msgr2=0x7fbf2c06aa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:40:01.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.657+0000 7fbf3440b640 1 -- 172.21.15.97:0/495922378 shutdown_connections 2024-04-23T10:40:01.658 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:01.657+0000 7fbf3440b640 1 -- 172.21.15.97:0/495922378 wait complete. 2024-04-23T10:40:02.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:40:02 smithi196 ceph-mon[32919]: pgmap v432: 97 pgs: 97 active+clean; 3.9 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:40:02.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:40:02 smithi097 ceph-mon[29197]: pgmap v432: 97 pgs: 97 active+clean; 3.9 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:40:02.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:40:02 smithi183 ceph-mon[32317]: pgmap v432: 97 pgs: 97 active+clean; 3.9 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:40:04.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:40:04 smithi196 ceph-mon[32919]: pgmap v433: 97 pgs: 97 active+clean; 4.2 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 111 KiB/s wr, 41 op/s 2024-04-23T10:40:04.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:40:04 smithi097 ceph-mon[29197]: pgmap v433: 97 pgs: 97 active+clean; 4.2 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 111 KiB/s wr, 41 op/s 2024-04-23T10:40:04.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:40:04 smithi183 ceph-mon[32317]: pgmap v433: 97 pgs: 97 active+clean; 4.2 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 111 KiB/s wr, 41 op/s 2024-04-23T10:40:06.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:40:06 smithi196 ceph-mon[32919]: pgmap v434: 97 pgs: 97 active+clean; 4.2 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 87 KiB/s wr, 32 op/s 2024-04-23T10:40:06.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:40:06 smithi097 ceph-mon[29197]: pgmap v434: 97 pgs: 97 active+clean; 4.2 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 87 KiB/s wr, 32 op/s 2024-04-23T10:40:06.899 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:40:06 smithi183 ceph-mon[32317]: pgmap v434: 97 pgs: 97 active+clean; 4.2 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 87 KiB/s wr, 32 op/s 2024-04-23T10:40:07.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:40:07 smithi196 ceph-mon[32919]: osdmap e104: 12 total, 12 up, 12 in 2024-04-23T10:40:07.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:40:07 smithi097 ceph-mon[29197]: osdmap e104: 12 total, 12 up, 12 in 2024-04-23T10:40:07.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:40:07 smithi183 ceph-mon[32317]: osdmap e104: 12 total, 12 up, 12 in 2024-04-23T10:40:08.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:40:08 smithi196 ceph-mon[32919]: pgmap v436: 97 pgs: 97 active+clean; 4.4 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 103 KiB/s wr, 39 op/s 2024-04-23T10:40:08.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:40:08 smithi097 ceph-mon[29197]: pgmap v436: 97 pgs: 97 active+clean; 4.4 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 103 KiB/s wr, 39 op/s 2024-04-23T10:40:08.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:40:08 smithi183 ceph-mon[32317]: pgmap v436: 97 pgs: 97 active+clean; 4.4 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 103 KiB/s wr, 39 op/s 2024-04-23T10:40:10.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:40:10 smithi196 ceph-mon[32919]: pgmap v437: 97 pgs: 97 active+clean; 4.4 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 103 KiB/s wr, 39 op/s 2024-04-23T10:40:10.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:40:10 smithi097 ceph-mon[29197]: pgmap v437: 97 pgs: 97 active+clean; 4.4 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 103 KiB/s wr, 39 op/s 2024-04-23T10:40:10.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:40:10 smithi183 ceph-mon[32317]: pgmap v437: 97 pgs: 97 active+clean; 4.4 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 103 KiB/s wr, 39 op/s 2024-04-23T10:40:12.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:40:12 smithi196 ceph-mon[32919]: osdmap e105: 12 total, 12 up, 12 in 2024-04-23T10:40:12.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:40:12 smithi097 ceph-mon[29197]: osdmap e105: 12 total, 12 up, 12 in 2024-04-23T10:40:12.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:40:12 smithi183 ceph-mon[32317]: osdmap e105: 12 total, 12 up, 12 in 2024-04-23T10:40:13.593 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:40:13 smithi196 ceph-mon[32919]: pgmap v439: 97 pgs: 97 active+clean; 4.4 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 KiB/s wr, 9 op/s 2024-04-23T10:40:13.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:40:13 smithi097 ceph-mon[29197]: pgmap v439: 97 pgs: 97 active+clean; 4.4 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 KiB/s wr, 9 op/s 2024-04-23T10:40:13.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:40:13 smithi183 ceph-mon[32317]: pgmap v439: 97 pgs: 97 active+clean; 4.4 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 KiB/s wr, 9 op/s 2024-04-23T10:40:14.754 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.752+0000 7f54e0067640 1 -- 172.21.15.97:0/571858277 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54d81021b0 msgr2=0x7f54d8102590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:40:14.754 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.752+0000 7f54e0067640 1 --2- 172.21.15.97:0/571858277 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54d81021b0 0x7f54d8102590 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f54d0009900 tx=0x7f54d002a410 comp rx=0 tx=0).stop 2024-04-23T10:40:14.754 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.752+0000 7f54e0067640 1 -- 172.21.15.97:0/571858277 shutdown_connections 2024-04-23T10:40:14.755 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.752+0000 7f54e0067640 1 --2- 172.21.15.97:0/571858277 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54d8107280 0x7f54d810ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:14.755 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.752+0000 7f54e0067640 1 --2- 172.21.15.97:0/571858277 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54d8102b60 0x7f54d8106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:14.755 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.752+0000 7f54e0067640 1 --2- 172.21.15.97:0/571858277 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54d81021b0 0x7f54d8102590 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:14.755 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.752+0000 7f54e0067640 1 -- 172.21.15.97:0/571858277 >> 172.21.15.97:0/571858277 conn(0x7f54d80fda10 msgr2=0x7f54d80ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:40:14.755 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.753+0000 7f54e0067640 1 -- 172.21.15.97:0/571858277 shutdown_connections 2024-04-23T10:40:14.755 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.753+0000 7f54e0067640 1 -- 172.21.15.97:0/571858277 wait complete. 2024-04-23T10:40:14.755 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.753+0000 7f54e0067640 1 Processor -- start 2024-04-23T10:40:14.755 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.753+0000 7f54e0067640 1 -- start start 2024-04-23T10:40:14.755 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.753+0000 7f54e0067640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54d81021b0 0x7f54d81986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:14.755 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.753+0000 7f54e0067640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54d8107280 0x7f54d8198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:14.755 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.753+0000 7f54e0067640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54d819cf90 0x7f54d819d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:14.755 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.753+0000 7f54e0067640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f54d8112210 con 0x7f54d8107280 2024-04-23T10:40:14.755 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.753+0000 7f54e0067640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f54d8112090 con 0x7f54d819cf90 2024-04-23T10:40:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.753+0000 7f54e0067640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f54d8112390 con 0x7f54d81021b0 2024-04-23T10:40:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.754+0000 7f54dd602640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54d8107280 0x7f54d8198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.754+0000 7f54dd602640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54d8107280 0x7f54d8198c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54594/0 (socket says 172.21.15.97:54594) 2024-04-23T10:40:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.754+0000 7f54dd602640 1 -- 172.21.15.97:0/1907555183 learned_addr learned my addr 172.21.15.97:0/1907555183 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:40:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.754+0000 7f54de604640 1 --2- 172.21.15.97:0/1907555183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54d819cf90 0x7f54d819d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.754+0000 7f54dde03640 1 --2- 172.21.15.97:0/1907555183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54d81021b0 0x7f54d81986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.754+0000 7f54dd602640 1 -- 172.21.15.97:0/1907555183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54d81021b0 msgr2=0x7f54d81986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:40:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.754+0000 7f54dd602640 1 --2- 172.21.15.97:0/1907555183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54d81021b0 0x7f54d81986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.754+0000 7f54dd602640 1 -- 172.21.15.97:0/1907555183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54d819cf90 msgr2=0x7f54d819d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:40:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.754+0000 7f54dd602640 1 --2- 172.21.15.97:0/1907555183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54d819cf90 0x7f54d819d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.754+0000 7f54dd602640 1 -- 172.21.15.97:0/1907555183 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54d819da80 con 0x7f54d8107280 2024-04-23T10:40:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.754+0000 7f54de604640 1 --2- 172.21.15.97:0/1907555183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54d819cf90 0x7f54d819d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:40:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.754+0000 7f54dd602640 1 --2- 172.21.15.97:0/1907555183 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54d8107280 0x7f54d8198c10 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f54cc00d770 tx=0x7f54cc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:40:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.754+0000 7f54d6ffd640 1 -- 172.21.15.97:0/1907555183 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54cc017070 con 0x7f54d8107280 2024-04-23T10:40:14.757 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.754+0000 7f54d6ffd640 1 -- 172.21.15.97:0/1907555183 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54cc009c30 con 0x7f54d8107280 2024-04-23T10:40:14.757 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.754+0000 7f54e0067640 1 -- 172.21.15.97:0/1907555183 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54d819dd70 con 0x7f54d8107280 2024-04-23T10:40:14.757 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.754+0000 7f54d6ffd640 1 -- 172.21.15.97:0/1907555183 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54cc00f620 con 0x7f54d8107280 2024-04-23T10:40:14.757 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.754+0000 7f54e0067640 1 -- 172.21.15.97:0/1907555183 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54d81a5560 con 0x7f54d8107280 2024-04-23T10:40:14.763 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.757+0000 7f54d6ffd640 1 -- 172.21.15.97:0/1907555183 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f54cc01a070 con 0x7f54d8107280 2024-04-23T10:40:14.764 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.757+0000 7f54e2d78740 1 Processor -- start 2024-04-23T10:40:14.764 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.757+0000 7f54e2d78740 1 -- start start 2024-04-23T10:40:14.764 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.757+0000 7f54e2d78740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b74226620 0x564b74226a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:14.764 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.757+0000 7f54d6ffd640 1 --2- 172.21.15.97:0/1907555183 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f54c00a3dc0 0x7f54c00a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:14.764 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.757+0000 7f54e2d78740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b74226f40 0x564b742312c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:14.764 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.757+0000 7f54e2d78740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b74231800 0x564b74233bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:14.764 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.757+0000 7f54e2d78740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x564b7418b970 con 0x564b74226f40 2024-04-23T10:40:14.764 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.757+0000 7f54e2d78740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564b741284a0 con 0x564b74226620 2024-04-23T10:40:14.764 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.757+0000 7f54e2d78740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564b74187930 con 0x564b74231800 2024-04-23T10:40:14.764 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.758+0000 7f54dde03640 1 --2- 172.21.15.97:0/1907555183 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f54c00a3dc0 0x7f54c00a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:14.764 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.758+0000 7f54dd602640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b74226f40 0x564b742312c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:14.764 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.758+0000 7f54de604640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b74231800 0x564b74233bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:14.764 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.758+0000 7f54dd602640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b74226f40 0x564b742312c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54606/0 (socket says 172.21.15.97:54606) 2024-04-23T10:40:14.764 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.758+0000 7f54de604640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b74231800 0x564b74233bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.97:37092/0 (socket says 172.21.15.97:37092) 2024-04-23T10:40:14.765 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.758+0000 7f54dd602640 1 -- 172.21.15.97:0/3892890135 learned_addr learned my addr 172.21.15.97:0/3892890135 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:40:14.765 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.758+0000 7f54d6ffd640 1 -- 172.21.15.97:0/1907555183 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8484+0+0 (secure 0 0 0) 0x7f54cc0d0400 con 0x7f54d8107280 2024-04-23T10:40:14.765 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.758+0000 7f54de604640 1 --2- 172.21.15.97:0/3892890135 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b74226620 0x564b74226a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:14.765 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.758+0000 7f54de604640 1 -- 172.21.15.97:0/3892890135 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b74226620 msgr2=0x564b74226a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:40:14.765 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.758+0000 7f54de604640 1 --2- 172.21.15.97:0/3892890135 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b74226620 0x564b74226a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:14.765 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.758+0000 7f54de604640 1 -- 172.21.15.97:0/3892890135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b74226f40 msgr2=0x564b742312c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:40:14.765 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.758+0000 7f54de604640 1 --2- 172.21.15.97:0/3892890135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b74226f40 0x564b742312c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:14.765 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.758+0000 7f54de604640 1 -- 172.21.15.97:0/3892890135 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b73e9e210 con 0x564b74231800 2024-04-23T10:40:14.765 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.758+0000 7f54dd602640 1 --2- 172.21.15.97:0/3892890135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b74226f40 0x564b742312c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T10:40:14.765 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.758+0000 7f54dde03640 1 --2- 172.21.15.97:0/1907555183 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f54c00a3dc0 0x7f54c00a6280 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f54d0000c00 tx=0x7f54d0031400 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:40:14.765 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.758+0000 7f54de604640 1 --2- 172.21.15.97:0/3892890135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b74231800 0x564b74233bf0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f54c8005fd0 tx=0x7f54c8009710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=efb4ae8abf8f1b8c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:40:14.765 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.758+0000 7f54bffff640 1 -- 172.21.15.97:0/3892890135 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c8014310 con 0x564b74231800 2024-04-23T10:40:14.765 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.759+0000 7f54bffff640 1 -- 172.21.15.97:0/3892890135 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54c8014e20 con 0x564b74231800 2024-04-23T10:40:14.765 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.759+0000 7f54bffff640 1 -- 172.21.15.97:0/3892890135 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c8003db0 con 0x564b74231800 2024-04-23T10:40:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.759+0000 7f54e2d78740 1 -- 172.21.15.97:0/3892890135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b74231800 msgr2=0x564b74233bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:40:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.759+0000 7f54e2d78740 1 --2- 172.21.15.97:0/3892890135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b74231800 0x564b74233bf0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f54c8005fd0 tx=0x7f54c8009710 comp rx=0 tx=0).stop 2024-04-23T10:40:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.759+0000 7f54e2d78740 1 -- 172.21.15.97:0/3892890135 shutdown_connections 2024-04-23T10:40:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.759+0000 7f54e2d78740 1 --2- 172.21.15.97:0/3892890135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b74231800 0x564b74233bf0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.759+0000 7f54e2d78740 1 --2- 172.21.15.97:0/3892890135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b74226f40 0x564b742312c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.759+0000 7f54e2d78740 1 --2- 172.21.15.97:0/3892890135 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b74226620 0x564b74226a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.759+0000 7f54e2d78740 1 -- 172.21.15.97:0/3892890135 >> 172.21.15.97:0/3892890135 conn(0x564b74222b30 msgr2=0x564b74222f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:40:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.759+0000 7f54e2d78740 1 -- 172.21.15.97:0/3892890135 shutdown_connections 2024-04-23T10:40:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.759+0000 7f54e2d78740 1 -- 172.21.15.97:0/3892890135 wait complete. 2024-04-23T10:40:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.760+0000 7f54e2d78740 1 -- start start 2024-04-23T10:40:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.760+0000 7f54e2d78740 1 Processor -- start 2024-04-23T10:40:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.760+0000 7f54e2d78740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b74226620 0x564b74231100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.760+0000 7f54e2d78740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b74226f40 0x564b7422e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.760+0000 7f54e2d78740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b74231800 0x564b7422e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:14.767 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.760+0000 7f54e2d78740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x564b74234430 con 0x564b74226620 2024-04-23T10:40:14.767 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.760+0000 7f54e2d78740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564b741260b0 con 0x564b74226f40 2024-04-23T10:40:14.767 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.760+0000 7f54e2d78740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564b742345b0 con 0x564b74231800 2024-04-23T10:40:14.767 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.760+0000 7f54e2d78740 10 client.0 _ll_register_callbacks cb 0x564b74113d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:40:14.767 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.760+0000 7f54dde03640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b74226620 0x564b74231100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:14.767 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.760+0000 7f54dde03640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b74226620 0x564b74231100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54610/0 (socket says 172.21.15.97:54610) 2024-04-23T10:40:14.767 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.760+0000 7f54dde03640 1 -- 172.21.15.97:0/1512086418 learned_addr learned my addr 172.21.15.97:0/1512086418 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:40:14.767 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.760+0000 7f54de604640 1 --2- 172.21.15.97:0/1512086418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b74231800 0x564b7422e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:14.767 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.761+0000 7f54dd602640 1 --2- 172.21.15.97:0/1512086418 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b74226f40 0x564b7422e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:14.767 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.761+0000 7f54dde03640 1 -- 172.21.15.97:0/1512086418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b74231800 msgr2=0x564b7422e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:40:14.767 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.761+0000 7f54dde03640 1 --2- 172.21.15.97:0/1512086418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b74231800 0x564b7422e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:14.767 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.761+0000 7f54dde03640 1 -- 172.21.15.97:0/1512086418 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b74226f40 msgr2=0x564b7422e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:40:14.767 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.761+0000 7f54dde03640 1 --2- 172.21.15.97:0/1512086418 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b74226f40 0x564b7422e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:14.767 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.761+0000 7f54dde03640 1 -- 172.21.15.97:0/1512086418 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b73f6a620 con 0x564b74226620 2024-04-23T10:40:14.767 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.761+0000 7f54dde03640 1 --2- 172.21.15.97:0/1512086418 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b74226620 0x564b74231100 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f54d0033000 tx=0x7f54d0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cf892f99310c9a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:40:14.768 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.761+0000 7f54bd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:40:14.768 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.761+0000 7f54bd7fa640 1 -- 172.21.15.97:0/1512086418 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d004c020 con 0x564b74226620 2024-04-23T10:40:14.768 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.761+0000 7f54bd7fa640 1 -- 172.21.15.97:0/1512086418 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8484+0+0 (secure 0 0 0) 0x7f54d00048c0 con 0x564b74226620 2024-04-23T10:40:14.768 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.761+0000 7f54e2d78740 1 -- 172.21.15.97:0/1512086418 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54c8014e20 con 0x564b74226620 2024-04-23T10:40:14.768 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.761+0000 7f54bd7fa640 1 -- 172.21.15.97:0/1512086418 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f54d0057050 con 0x564b74226620 2024-04-23T10:40:14.768 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.761+0000 7f54e2d78740 10 client.17505 fetch_fsmap learned FSMap version 26 2024-04-23T10:40:14.768 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.761+0000 7f54e2d78740 1 -- 172.21.15.97:0/1512086418 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564b742403b0 con 0x564b74226620 2024-04-23T10:40:14.768 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.761+0000 7f54bd7fa640 1 -- 172.21.15.97:0/1512086418 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54c8014e20 con 0x564b74226620 2024-04-23T10:40:14.768 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.762+0000 7f54bd7fa640 1 -- 172.21.15.97:0/1512086418 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f54d0003d80 con 0x564b74226620 2024-04-23T10:40:14.768 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.762+0000 7f54e2d78740 10 client.17505 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:40:14.768 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.762+0000 7f54e2d78740 10 client.17505 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:40:14.768 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.762+0000 7f54e2d78740 20 client.17505 populate_metadata read hostname 'smithi097' 2024-04-23T10:40:14.768 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.762+0000 7f54e2d78740 1 --2- 172.21.15.97:0/1512086418 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x564b742418c0 0x564b74243cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:14.768 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.762+0000 7f54e2d78740 4 client.17505 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:40:14.768 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.762+0000 7f54e2d78740 1 -- 172.21.15.97:0/1512086418 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b74199390 con 0x564b742418c0 2024-04-23T10:40:14.769 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.762+0000 7f54dd602640 1 --2- 172.21.15.97:0/1512086418 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x564b742418c0 0x564b74243cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:14.769 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.762+0000 7f54bd7fa640 10 client.17505 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:40:14.769 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.762+0000 7f54dd602640 1 --2- 172.21.15.97:0/1512086418 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x564b742418c0 0x564b74243cf0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2a6dcde87457eda server_cookie=212e57b64e4821f1 in_seq=0 out_seq=0 2024-04-23T10:40:14.769 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.763+0000 7f54bd7fa640 1 -- 172.21.15.97:0/1512086418 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564b74199390 con 0x564b742418c0 2024-04-23T10:40:14.769 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.763+0000 7f54bd7fa640 10 client.17505 handle_command_reply: tid=0 2024-04-23T10:40:14.769 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.763+0000 7f54e2d78740 1 client.17505 shutdown 2024-04-23T10:40:14.769 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.763+0000 7f54e2d78740 1 -- 172.21.15.97:0/1512086418 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b74226620 msgr2=0x564b74231100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:40:14.769 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.763+0000 7f54e2d78740 1 --2- 172.21.15.97:0/1512086418 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b74226620 0x564b74231100 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f54d0033000 tx=0x7f54d0009260 comp rx=0 tx=0).stop 2024-04-23T10:40:14.769 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.763+0000 7f54e2d78740 1 -- 172.21.15.97:0/1512086418 shutdown_connections 2024-04-23T10:40:14.769 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.763+0000 7f54e2d78740 1 --2- 172.21.15.97:0/1512086418 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x564b742418c0 0x564b74243cf0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:14.769 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.763+0000 7f54e2d78740 1 --2- 172.21.15.97:0/1512086418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b74231800 0x564b7422e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:14.769 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.763+0000 7f54e2d78740 1 --2- 172.21.15.97:0/1512086418 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b74226f40 0x564b7422e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:14.769 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.763+0000 7f54e2d78740 1 --2- 172.21.15.97:0/1512086418 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b74226620 0x564b74231100 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:14.769 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.763+0000 7f54bd7fa640 0 client.17505 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:40:14.769 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.763+0000 7f54e2d78740 1 -- 172.21.15.97:0/1512086418 >> 172.21.15.97:0/1512086418 conn(0x564b74222b30 msgr2=0x564b742288b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:40:14.770 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.763+0000 7f54e2d78740 1 -- 172.21.15.97:0/1512086418 shutdown_connections 2024-04-23T10:40:14.770 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.763+0000 7f54e2d78740 1 -- 172.21.15.97:0/1512086418 wait complete. 2024-04-23T10:40:14.770 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.763+0000 7f54e2d78740 20 client.17505 trim_cache size 0 max 16384 2024-04-23T10:40:14.781 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.778+0000 7f54e2d78740 1 Processor -- start 2024-04-23T10:40:14.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.778+0000 7f54e2d78740 1 -- start start 2024-04-23T10:40:14.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.778+0000 7f54e2d78740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b74226620 0x564b7423d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:14.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.778+0000 7f54e2d78740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b74226f40 0x564b74230ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:14.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.778+0000 7f54e2d78740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b74231800 0x564b74249600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:14.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.778+0000 7f54e2d78740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f54d81adb10 con 0x564b74226f40 2024-04-23T10:40:14.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.778+0000 7f54e2d78740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f54d81af650 con 0x564b74231800 2024-04-23T10:40:14.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.778+0000 7f54e2d78740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f54d81ae1e0 con 0x564b74226620 2024-04-23T10:40:14.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.778+0000 7f54dd602640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b74226f40 0x564b74230ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:14.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.778+0000 7f54dd602640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b74226f40 0x564b74230ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54622/0 (socket says 172.21.15.97:54622) 2024-04-23T10:40:14.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.779+0000 7f54dd602640 1 -- 172.21.15.97:0/3325191705 learned_addr learned my addr 172.21.15.97:0/3325191705 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:40:14.782 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.779+0000 7f54de604640 1 --2- 172.21.15.97:0/3325191705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b74231800 0x564b74249600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:14.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.779+0000 7f54dd602640 1 -- 172.21.15.97:0/3325191705 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b74226620 msgr2=0x564b7423d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:40:14.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.779+0000 7f54dd602640 1 --2- 172.21.15.97:0/3325191705 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b74226620 0x564b7423d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:14.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.779+0000 7f54dd602640 1 -- 172.21.15.97:0/3325191705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b74231800 msgr2=0x564b74249600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:40:14.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.779+0000 7f54dd602640 1 --2- 172.21.15.97:0/3325191705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b74231800 0x564b74249600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:14.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.779+0000 7f54dd602640 1 -- 172.21.15.97:0/3325191705 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b7422a030 con 0x564b74226f40 2024-04-23T10:40:14.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.779+0000 7f54de604640 1 --2- 172.21.15.97:0/3325191705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b74231800 0x564b74249600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:40:14.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.779+0000 7f54dd602640 1 --2- 172.21.15.97:0/3325191705 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b74226f40 0x564b74230ca0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f54cc0ce170 tx=0x7f54cc010c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cf039008a9d840b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:40:14.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.779+0000 7f54bf7fe640 1 -- 172.21.15.97:0/3325191705 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54cc0b0e60 con 0x564b74226f40 2024-04-23T10:40:14.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.779+0000 7f54bf7fe640 1 -- 172.21.15.97:0/3325191705 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54cc09c3c0 con 0x564b74226f40 2024-04-23T10:40:14.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.779+0000 7f54bf7fe640 1 -- 172.21.15.97:0/3325191705 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54cc021040 con 0x564b74226f40 2024-04-23T10:40:14.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.780+0000 7f54e2d78740 1 -- 172.21.15.97:0/3325191705 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b74226f40 msgr2=0x564b74230ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:40:14.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.780+0000 7f54e2d78740 1 --2- 172.21.15.97:0/3325191705 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b74226f40 0x564b74230ca0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f54cc0ce170 tx=0x7f54cc010c00 comp rx=0 tx=0).stop 2024-04-23T10:40:14.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.780+0000 7f54e2d78740 1 -- 172.21.15.97:0/3325191705 shutdown_connections 2024-04-23T10:40:14.783 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.780+0000 7f54e2d78740 1 --2- 172.21.15.97:0/3325191705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b74231800 0x564b74249600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:14.784 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.780+0000 7f54e2d78740 1 --2- 172.21.15.97:0/3325191705 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b74226f40 0x564b74230ca0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:14.784 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.780+0000 7f54e2d78740 1 --2- 172.21.15.97:0/3325191705 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b74226620 0x564b7423d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:14.784 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.780+0000 7f54e2d78740 1 -- 172.21.15.97:0/3325191705 >> 172.21.15.97:0/3325191705 conn(0x564b74222b30 msgr2=0x564b74221cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:40:14.784 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.780+0000 7f54e2d78740 1 -- 172.21.15.97:0/3325191705 shutdown_connections 2024-04-23T10:40:14.784 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.780+0000 7f54e2d78740 1 -- 172.21.15.97:0/3325191705 wait complete. 2024-04-23T10:40:14.784 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.780+0000 7f54e2d78740 1 -- start start 2024-04-23T10:40:14.784 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.780+0000 7f54e2d78740 1 Processor -- start 2024-04-23T10:40:14.784 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.780+0000 7f54e2d78740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b74226620 0x564b742a0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:14.784 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.780+0000 7f54e2d78740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b74226f40 0x564b742a0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:14.784 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.780+0000 7f54e2d78740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b74231800 0x564b742a4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:14.784 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.780+0000 7f54e2d78740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f54d81aef80 con 0x564b74226620 2024-04-23T10:40:14.784 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.780+0000 7f54e2d78740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f54d81ae8b0 con 0x564b74231800 2024-04-23T10:40:14.784 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.780+0000 7f54e2d78740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f54d81abb10 con 0x564b74226f40 2024-04-23T10:40:14.784 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.781+0000 7f54e2d78740 10 client.0 _ll_register_callbacks cb 0x7f54d00050f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:40:14.784 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.781+0000 7f54dde03640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b74226620 0x564b742a0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:14.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.781+0000 7f54dde03640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b74226620 0x564b742a0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:54634/0 (socket says 172.21.15.97:54634) 2024-04-23T10:40:14.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.781+0000 7f54dde03640 1 -- 172.21.15.97:0/2802720286 learned_addr learned my addr 172.21.15.97:0/2802720286 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:40:14.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.781+0000 7f54de604640 1 --2- 172.21.15.97:0/2802720286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b74231800 0x564b742a4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:14.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.781+0000 7f54dde03640 1 -- 172.21.15.97:0/2802720286 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b74226f40 msgr2=0x564b742a0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:40:14.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.781+0000 7f54dd602640 1 --2- 172.21.15.97:0/2802720286 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b74226f40 0x564b742a0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:14.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.781+0000 7f54dde03640 1 --2- 172.21.15.97:0/2802720286 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b74226f40 0x564b742a0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:14.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.781+0000 7f54dde03640 1 -- 172.21.15.97:0/2802720286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b74231800 msgr2=0x564b742a4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:40:14.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.781+0000 7f54dde03640 1 --2- 172.21.15.97:0/2802720286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b74231800 0x564b742a4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:14.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.781+0000 7f54dde03640 1 -- 172.21.15.97:0/2802720286 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b742a55a0 con 0x564b74226620 2024-04-23T10:40:14.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.781+0000 7f54dd602640 1 --2- 172.21.15.97:0/2802720286 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b74226f40 0x564b742a0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:40:14.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.781+0000 7f54de604640 1 --2- 172.21.15.97:0/2802720286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b74231800 0x564b742a4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:40:14.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.781+0000 7f54dde03640 1 --2- 172.21.15.97:0/2802720286 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b74226620 0x564b742a0020 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f54d0055040 tx=0x7f54d00329d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad931200eebc7686 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:40:14.785 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.781+0000 7f54bd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:40:14.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.781+0000 7f54bd7fa640 1 -- 172.21.15.97:0/2802720286 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d004c040 con 0x564b74226620 2024-04-23T10:40:14.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.781+0000 7f54bd7fa640 1 -- 172.21.15.97:0/2802720286 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8484+0+0 (secure 0 0 0) 0x7f54d00046c0 con 0x564b74226620 2024-04-23T10:40:14.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.781+0000 7f54e2d78740 1 -- 172.21.15.97:0/2802720286 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54cc09c3c0 con 0x564b74226620 2024-04-23T10:40:14.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.781+0000 7f54bd7fa640 1 -- 172.21.15.97:0/2802720286 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f54d006b050 con 0x564b74226620 2024-04-23T10:40:14.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.781+0000 7f54e2d78740 10 client.17511 fetch_fsmap learned FSMap version 26 2024-04-23T10:40:14.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.781+0000 7f54e2d78740 1 -- 172.21.15.97:0/2802720286 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564b742ac890 con 0x564b74226620 2024-04-23T10:40:14.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.782+0000 7f54bd7fa640 1 -- 172.21.15.97:0/2802720286 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54cc09c3c0 con 0x564b74226620 2024-04-23T10:40:14.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.782+0000 7f54bd7fa640 1 -- 172.21.15.97:0/2802720286 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f54d0003940 con 0x564b74226620 2024-04-23T10:40:14.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.782+0000 7f54e2d78740 10 client.17511 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:40:14.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.782+0000 7f54e2d78740 10 client.17511 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:40:14.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.782+0000 7f54e2d78740 20 client.17511 populate_metadata read hostname 'smithi097' 2024-04-23T10:40:14.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.782+0000 7f54e2d78740 1 --2- 172.21.15.97:0/2802720286 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x564b742ad010 0x564b742af430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:14.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.782+0000 7f54e2d78740 4 client.17511 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:40:14.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.782+0000 7f54e2d78740 1 -- 172.21.15.97:0/2802720286 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564b74193200 con 0x564b742ad010 2024-04-23T10:40:14.786 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.782+0000 7f54dd602640 1 --2- 172.21.15.97:0/2802720286 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x564b742ad010 0x564b742af430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:14.787 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.782+0000 7f54bd7fa640 10 client.17511 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:40:14.787 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.782+0000 7f54dd602640 1 --2- 172.21.15.97:0/2802720286 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x564b742ad010 0x564b742af430 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df3f73fd00b5739e server_cookie=301aacf70e39190a in_seq=0 out_seq=0 2024-04-23T10:40:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.800+0000 7f54bd7fa640 1 -- 172.21.15.97:0/2802720286 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564b74193200 con 0x564b742ad010 2024-04-23T10:40:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.800+0000 7f54bd7fa640 10 client.17511 handle_command_reply: tid=0 2024-04-23T10:40:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.800+0000 7f54e2d78740 1 client.17511 shutdown 2024-04-23T10:40:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.800+0000 7f54e2d78740 1 -- 172.21.15.97:0/2802720286 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b74226620 msgr2=0x564b742a0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:40:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.800+0000 7f54e2d78740 1 --2- 172.21.15.97:0/2802720286 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b74226620 0x564b742a0020 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f54d0055040 tx=0x7f54d00329d0 comp rx=0 tx=0).stop 2024-04-23T10:40:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.800+0000 7f54e2d78740 1 -- 172.21.15.97:0/2802720286 shutdown_connections 2024-04-23T10:40:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.800+0000 7f54e2d78740 1 --2- 172.21.15.97:0/2802720286 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x564b742ad010 0x564b742af430 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.800+0000 7f54e2d78740 1 --2- 172.21.15.97:0/2802720286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564b74231800 0x564b742a4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.800+0000 7f54e2d78740 1 --2- 172.21.15.97:0/2802720286 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b74226f40 0x564b742a0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.800+0000 7f54e2d78740 1 --2- 172.21.15.97:0/2802720286 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564b74226620 0x564b742a0020 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.800+0000 7f54bd7fa640 0 client.17511 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:40:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.800+0000 7f54e2d78740 1 -- 172.21.15.97:0/2802720286 >> 172.21.15.97:0/2802720286 conn(0x564b74222b30 msgr2=0x564b7423c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:40:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.800+0000 7f54e2d78740 1 -- 172.21.15.97:0/2802720286 shutdown_connections 2024-04-23T10:40:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.800+0000 7f54e2d78740 1 -- 172.21.15.97:0/2802720286 wait complete. 2024-04-23T10:40:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.800+0000 7f54e2d78740 20 client.17511 trim_cache size 0 max 16384 2024-04-23T10:40:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:40:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:40:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:40:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:40:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.801+0000 7f54e0067640 1 -- 172.21.15.97:0/1907555183 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f54c00a3dc0 msgr2=0x7f54c00a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:40:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.801+0000 7f54e0067640 1 --2- 172.21.15.97:0/1907555183 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f54c00a3dc0 0x7f54c00a6280 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f54d0000c00 tx=0x7f54d0031400 comp rx=0 tx=0).stop 2024-04-23T10:40:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.801+0000 7f54e0067640 1 -- 172.21.15.97:0/1907555183 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54d8107280 msgr2=0x7f54d8198c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:40:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.801+0000 7f54e0067640 1 --2- 172.21.15.97:0/1907555183 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54d8107280 0x7f54d8198c10 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f54cc00d770 tx=0x7f54cc00dc30 comp rx=0 tx=0).stop 2024-04-23T10:40:14.802 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.801+0000 7f54e0067640 1 -- 172.21.15.97:0/1907555183 shutdown_connections 2024-04-23T10:40:14.802 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.801+0000 7f54e0067640 1 --2- 172.21.15.97:0/1907555183 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f54c00a3dc0 0x7f54c00a6280 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:14.802 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.801+0000 7f54e0067640 1 --2- 172.21.15.97:0/1907555183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54d819cf90 0x7f54d819d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:14.802 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.801+0000 7f54e0067640 1 --2- 172.21.15.97:0/1907555183 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f54d8107280 0x7f54d8198c10 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:14.802 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.801+0000 7f54e0067640 1 --2- 172.21.15.97:0/1907555183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54d81021b0 0x7f54d81986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:14.802 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.801+0000 7f54e0067640 1 -- 172.21.15.97:0/1907555183 >> 172.21.15.97:0/1907555183 conn(0x7f54d80fda10 msgr2=0x7f54d80fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:40:14.802 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.801+0000 7f54e0067640 1 -- 172.21.15.97:0/1907555183 shutdown_connections 2024-04-23T10:40:14.802 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:14.801+0000 7f54e0067640 1 -- 172.21.15.97:0/1907555183 wait complete. 2024-04-23T10:40:15.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:40:15 smithi196 ceph-mon[32919]: pgmap v440: 97 pgs: 97 active+clean; 4.6 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 52 KiB/s wr, 19 op/s 2024-04-23T10:40:15.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:40:15 smithi097 ceph-mon[29197]: pgmap v440: 97 pgs: 97 active+clean; 4.6 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 52 KiB/s wr, 19 op/s 2024-04-23T10:40:15.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:40:15 smithi183 ceph-mon[32317]: pgmap v440: 97 pgs: 97 active+clean; 4.6 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 52 KiB/s wr, 19 op/s 2024-04-23T10:40:17.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:40:17 smithi196 ceph-mon[32919]: pgmap v441: 97 pgs: 97 active+clean; 4.6 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 47 KiB/s wr, 17 op/s 2024-04-23T10:40:17.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:40:17 smithi097 ceph-mon[29197]: pgmap v441: 97 pgs: 97 active+clean; 4.6 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 47 KiB/s wr, 17 op/s 2024-04-23T10:40:17.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:40:17 smithi183 ceph-mon[32317]: pgmap v441: 97 pgs: 97 active+clean; 4.6 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 47 KiB/s wr, 17 op/s 2024-04-23T10:40:19.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:40:19 smithi196 ceph-mon[32919]: pgmap v442: 97 pgs: 97 active+clean; 4.8 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-23T10:40:19.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:40:19 smithi097 ceph-mon[29197]: pgmap v442: 97 pgs: 97 active+clean; 4.8 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-23T10:40:19.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:40:19 smithi183 ceph-mon[32317]: pgmap v442: 97 pgs: 97 active+clean; 4.8 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-23T10:40:21.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:40:21 smithi196 ceph-mon[32919]: pgmap v443: 97 pgs: 97 active+clean; 4.8 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-23T10:40:21.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:40:21 smithi097 ceph-mon[29197]: pgmap v443: 97 pgs: 97 active+clean; 4.8 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-23T10:40:21.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:40:21 smithi183 ceph-mon[32317]: pgmap v443: 97 pgs: 97 active+clean; 4.8 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-23T10:40:23.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:40:23 smithi196 ceph-mon[32919]: pgmap v444: 97 pgs: 97 active+clean; 4.8 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 49 KiB/s wr, 18 op/s 2024-04-23T10:40:23.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:40:23 smithi097 ceph-mon[29197]: pgmap v444: 97 pgs: 97 active+clean; 4.8 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 49 KiB/s wr, 18 op/s 2024-04-23T10:40:23.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:40:23 smithi183 ceph-mon[32317]: pgmap v444: 97 pgs: 97 active+clean; 4.8 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 49 KiB/s wr, 18 op/s 2024-04-23T10:40:25.452 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:40:25 smithi183 ceph-mon[32317]: pgmap v445: 97 pgs: 97 active+clean; 5.0 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-23T10:40:25.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:40:25 smithi196 ceph-mon[32919]: pgmap v445: 97 pgs: 97 active+clean; 5.0 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-23T10:40:25.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:40:25 smithi097 ceph-mon[29197]: pgmap v445: 97 pgs: 97 active+clean; 5.0 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-23T10:40:26.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:40:26 smithi196 ceph-mon[32919]: pgmap v446: 97 pgs: 97 active+clean; 5.0 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-23T10:40:26.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:40:26 smithi097 ceph-mon[29197]: pgmap v446: 97 pgs: 97 active+clean; 5.0 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-23T10:40:26.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:40:26 smithi183 ceph-mon[32317]: pgmap v446: 97 pgs: 97 active+clean; 5.0 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-23T10:40:27.899 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.896+0000 7ff0c3acb640 1 -- 172.21.15.97:0/3185555054 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0bc100f80 msgr2=0x7ff0bc10d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:40:27.899 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.896+0000 7ff0c3acb640 1 --2- 172.21.15.97:0/3185555054 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0bc100f80 0x7ff0bc10d490 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7ff0b400ade0 tx=0x7ff0b402b7e0 comp rx=0 tx=0).stop 2024-04-23T10:40:27.899 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.896+0000 7ff0c3acb640 1 -- 172.21.15.97:0/3185555054 shutdown_connections 2024-04-23T10:40:27.899 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.896+0000 7ff0c3acb640 1 --2- 172.21.15.97:0/3185555054 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0bc100f80 0x7ff0bc10d490 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:27.899 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.896+0000 7ff0c3acb640 1 --2- 172.21.15.97:0/3185555054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0bc1005c0 0x7ff0bc100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:27.899 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.896+0000 7ff0c3acb640 1 --2- 172.21.15.97:0/3185555054 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0bc106660 0x7ff0bc106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.896+0000 7ff0c3acb640 1 -- 172.21.15.97:0/3185555054 >> 172.21.15.97:0/3185555054 conn(0x7ff0bc077370 msgr2=0x7ff0bc0fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:40:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.896+0000 7ff0c3acb640 1 -- 172.21.15.97:0/3185555054 shutdown_connections 2024-04-23T10:40:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.896+0000 7ff0c3acb640 1 -- 172.21.15.97:0/3185555054 wait complete. 2024-04-23T10:40:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.897+0000 7ff0c3acb640 1 Processor -- start 2024-04-23T10:40:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.897+0000 7ff0c3acb640 1 -- start start 2024-04-23T10:40:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.897+0000 7ff0c3acb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0bc1005c0 0x7ff0bc102260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.897+0000 7ff0c3acb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0bc100f80 0x7ff0bc1027a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.897+0000 7ff0c3acb640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0bc106660 0x7ff0bc102ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.897+0000 7ff0c3acb640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff0bc110010 con 0x7ff0bc1005c0 2024-04-23T10:40:27.901 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.897+0000 7ff0c3acb640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff0bc10fe90 con 0x7ff0bc106660 2024-04-23T10:40:27.901 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.897+0000 7ff0c3acb640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff0bc110190 con 0x7ff0bc100f80 2024-04-23T10:40:27.901 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.897+0000 7ff0c1867640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0bc1005c0 0x7ff0bc102260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:27.901 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.897+0000 7ff0c1867640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0bc1005c0 0x7ff0bc102260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43484/0 (socket says 172.21.15.97:43484) 2024-04-23T10:40:27.901 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.897+0000 7ff0c1867640 1 -- 172.21.15.97:0/3561492584 learned_addr learned my addr 172.21.15.97:0/3561492584 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:40:27.901 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.898+0000 7ff0c1066640 1 --2- 172.21.15.97:0/3561492584 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0bc100f80 0x7ff0bc1027a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:27.901 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.898+0000 7ff0c2068640 1 --2- 172.21.15.97:0/3561492584 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0bc106660 0x7ff0bc102ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:27.901 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.898+0000 7ff0c1867640 1 -- 172.21.15.97:0/3561492584 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0bc100f80 msgr2=0x7ff0bc1027a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:40:27.901 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.898+0000 7ff0c1867640 1 --2- 172.21.15.97:0/3561492584 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0bc100f80 0x7ff0bc1027a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:27.902 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.898+0000 7ff0c1867640 1 -- 172.21.15.97:0/3561492584 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0bc106660 msgr2=0x7ff0bc102ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:40:27.902 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.898+0000 7ff0c1867640 1 --2- 172.21.15.97:0/3561492584 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0bc106660 0x7ff0bc102ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:27.902 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.898+0000 7ff0c1867640 1 -- 172.21.15.97:0/3561492584 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0bc1a6b20 con 0x7ff0bc1005c0 2024-04-23T10:40:27.902 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.898+0000 7ff0c2068640 1 --2- 172.21.15.97:0/3561492584 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0bc106660 0x7ff0bc102ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:40:27.902 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.898+0000 7ff0c1867640 1 --2- 172.21.15.97:0/3561492584 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0bc1005c0 0x7ff0bc102260 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7ff0b800c970 tx=0x7ff0b800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:40:27.902 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.898+0000 7ff0b2ffd640 1 -- 172.21.15.97:0/3561492584 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b8004220 con 0x7ff0bc1005c0 2024-04-23T10:40:27.902 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.898+0000 7ff0b2ffd640 1 -- 172.21.15.97:0/3561492584 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0b80043c0 con 0x7ff0bc1005c0 2024-04-23T10:40:27.902 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.898+0000 7ff0c3acb640 1 -- 172.21.15.97:0/3561492584 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0bc1a6e10 con 0x7ff0bc1005c0 2024-04-23T10:40:27.902 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.898+0000 7ff0b2ffd640 1 -- 172.21.15.97:0/3561492584 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b800f450 con 0x7ff0bc1005c0 2024-04-23T10:40:27.902 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.898+0000 7ff0c3acb640 1 -- 172.21.15.97:0/3561492584 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0bc1a72d0 con 0x7ff0bc1005c0 2024-04-23T10:40:27.911 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.900+0000 7ff0b2ffd640 1 -- 172.21.15.97:0/3561492584 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff0b8003610 con 0x7ff0bc1005c0 2024-04-23T10:40:27.911 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.901+0000 7ff0c67d7740 1 Processor -- start 2024-04-23T10:40:27.911 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.901+0000 7ff0c67d7740 1 -- start start 2024-04-23T10:40:27.911 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.901+0000 7ff0c67d7740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581cb05e620 0x5581cb05ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:27.911 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.901+0000 7ff0c67d7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581cb05ef40 0x5581cb0692c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:27.911 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.901+0000 7ff0c67d7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581cb069800 0x5581cb06bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:27.912 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.901+0000 7ff0c67d7740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5581cafc3970 con 0x5581cb05e620 2024-04-23T10:40:27.912 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.901+0000 7ff0c67d7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5581caf604a0 con 0x5581cb069800 2024-04-23T10:40:27.912 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.901+0000 7ff0c67d7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5581cafbf930 con 0x5581cb05ef40 2024-04-23T10:40:27.912 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.901+0000 7ff0b2ffd640 1 --2- 172.21.15.97:0/3561492584 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7ff0a80a3df0 0x7ff0a80a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:27.912 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.902+0000 7ff0c1867640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581cb05ef40 0x5581cb0692c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:27.912 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.902+0000 7ff0c2068640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581cb05e620 0x5581cb05ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:27.912 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.902+0000 7ff0c2068640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581cb05e620 0x5581cb05ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43486/0 (socket says 172.21.15.97:43486) 2024-04-23T10:40:27.912 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.902+0000 7ff0c2068640 1 -- 172.21.15.97:0/22709722 learned_addr learned my addr 172.21.15.97:0/22709722 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:40:27.912 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.902+0000 7ff0c2068640 1 --2- 172.21.15.97:0/3561492584 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7ff0a80a3df0 0x7ff0a80a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:27.912 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.902+0000 7ff0c1066640 1 --2- 172.21.15.97:0/22709722 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581cb069800 0x5581cb06bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.902+0000 7ff0b2ffd640 1 -- 172.21.15.97:0/3561492584 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8484+0+0 (secure 0 0 0) 0x7ff0b80d0520 con 0x7ff0bc1005c0 2024-04-23T10:40:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.902+0000 7ff0c1867640 1 -- 172.21.15.97:0/22709722 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581cb069800 msgr2=0x5581cb06bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:40:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.902+0000 7ff0c1867640 1 --2- 172.21.15.97:0/22709722 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581cb069800 0x5581cb06bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.902+0000 7ff0c1867640 1 -- 172.21.15.97:0/22709722 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581cb05e620 msgr2=0x5581cb05ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:40:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.902+0000 7ff0c1867640 1 --2- 172.21.15.97:0/22709722 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581cb05e620 0x5581cb05ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.902+0000 7ff0c1867640 1 -- 172.21.15.97:0/22709722 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581cacd6210 con 0x5581cb05ef40 2024-04-23T10:40:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.902+0000 7ff0c2068640 1 --2- 172.21.15.97:0/3561492584 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7ff0a80a3df0 0x7ff0a80a62b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff0b4033ff0 tx=0x7ff0b400a6d0 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:40:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.902+0000 7ff0c1867640 1 --2- 172.21.15.97:0/22709722 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581cb05ef40 0x5581cb0692c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff0b80cdc90 tx=0x7ff0b806efd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e54d0738b31b96d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:40:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.902+0000 7ff0a7fff640 1 -- 172.21.15.97:0/22709722 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b801d050 con 0x5581cb05ef40 2024-04-23T10:40:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.903+0000 7ff0a7fff640 1 -- 172.21.15.97:0/22709722 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0b809cbe0 con 0x5581cb05ef40 2024-04-23T10:40:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.903+0000 7ff0a7fff640 1 -- 172.21.15.97:0/22709722 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b80a4cc0 con 0x5581cb05ef40 2024-04-23T10:40:27.914 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.903+0000 7ff0c67d7740 1 -- 172.21.15.97:0/22709722 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581cb05ef40 msgr2=0x5581cb0692c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:40:27.914 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.903+0000 7ff0c67d7740 1 --2- 172.21.15.97:0/22709722 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581cb05ef40 0x5581cb0692c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff0b80cdc90 tx=0x7ff0b806efd0 comp rx=0 tx=0).stop 2024-04-23T10:40:27.914 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.903+0000 7ff0c67d7740 1 -- 172.21.15.97:0/22709722 shutdown_connections 2024-04-23T10:40:27.914 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.903+0000 7ff0c67d7740 1 --2- 172.21.15.97:0/22709722 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581cb069800 0x5581cb06bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:27.914 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.903+0000 7ff0c67d7740 1 --2- 172.21.15.97:0/22709722 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581cb05ef40 0x5581cb0692c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:27.914 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.903+0000 7ff0c67d7740 1 --2- 172.21.15.97:0/22709722 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581cb05e620 0x5581cb05ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:27.914 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.903+0000 7ff0c67d7740 1 -- 172.21.15.97:0/22709722 >> 172.21.15.97:0/22709722 conn(0x5581cb05ab30 msgr2=0x5581cb05af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:40:27.914 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.903+0000 7ff0c67d7740 1 -- 172.21.15.97:0/22709722 shutdown_connections 2024-04-23T10:40:27.914 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.903+0000 7ff0c67d7740 1 -- 172.21.15.97:0/22709722 wait complete. 2024-04-23T10:40:27.914 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.904+0000 7ff0c67d7740 1 -- start start 2024-04-23T10:40:27.915 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.904+0000 7ff0c67d7740 1 Processor -- start 2024-04-23T10:40:27.915 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.904+0000 7ff0c67d7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581cb05e620 0x5581cb069130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:27.915 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.904+0000 7ff0c67d7740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581cb05ef40 0x5581cb066420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:27.915 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.904+0000 7ff0c67d7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581cb069800 0x5581cb066960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:27.915 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.904+0000 7ff0c67d7740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5581cb06c430 con 0x5581cb05ef40 2024-04-23T10:40:27.915 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.904+0000 7ff0c67d7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5581caf5e0b0 con 0x5581cb05e620 2024-04-23T10:40:27.915 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.904+0000 7ff0c67d7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5581cb06c5b0 con 0x5581cb069800 2024-04-23T10:40:27.915 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.904+0000 7ff0c67d7740 10 client.0 _ll_register_callbacks cb 0x5581caf4bd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:40:27.915 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.904+0000 7ff0c1066640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581cb05ef40 0x5581cb066420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:27.915 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.904+0000 7ff0c1066640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581cb05ef40 0x5581cb066420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43490/0 (socket says 172.21.15.97:43490) 2024-04-23T10:40:27.915 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.904+0000 7ff0c1066640 1 -- 172.21.15.97:0/4264688744 learned_addr learned my addr 172.21.15.97:0/4264688744 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:40:27.916 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.905+0000 7ff0c2068640 1 --2- 172.21.15.97:0/4264688744 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581cb069800 0x5581cb066960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:27.916 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.905+0000 7ff0c1867640 1 --2- 172.21.15.97:0/4264688744 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581cb05e620 0x5581cb069130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:27.916 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.905+0000 7ff0c1066640 1 -- 172.21.15.97:0/4264688744 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581cb069800 msgr2=0x5581cb066960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:40:27.916 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.905+0000 7ff0c1066640 1 --2- 172.21.15.97:0/4264688744 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581cb069800 0x5581cb066960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:27.916 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.905+0000 7ff0c1066640 1 -- 172.21.15.97:0/4264688744 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581cb05e620 msgr2=0x5581cb069130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:40:27.916 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.905+0000 7ff0c1066640 1 --2- 172.21.15.97:0/4264688744 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581cb05e620 0x5581cb069130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:27.916 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.905+0000 7ff0c1066640 1 -- 172.21.15.97:0/4264688744 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581cada2620 con 0x5581cb05ef40 2024-04-23T10:40:27.916 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.905+0000 7ff0c1867640 1 --2- 172.21.15.97:0/4264688744 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581cb05e620 0x5581cb069130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:40:27.916 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.905+0000 7ff0c1066640 1 --2- 172.21.15.97:0/4264688744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581cb05ef40 0x5581cb066420 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac00b990 tx=0x7ff0ac00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b972d7b95f991389 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:40:27.916 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.905+0000 7ff0a57fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:40:27.916 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.905+0000 7ff0a57fa640 1 -- 172.21.15.97:0/4264688744 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0ac005020 con 0x5581cb05ef40 2024-04-23T10:40:27.917 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.905+0000 7ff0a57fa640 1 -- 172.21.15.97:0/4264688744 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8484+0+0 (secure 0 0 0) 0x7ff0ac0051c0 con 0x5581cb05ef40 2024-04-23T10:40:27.917 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.905+0000 7ff0c67d7740 1 -- 172.21.15.97:0/4264688744 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0b809cbe0 con 0x5581cb05ef40 2024-04-23T10:40:27.917 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.905+0000 7ff0a57fa640 1 -- 172.21.15.97:0/4264688744 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff0ac0053c0 con 0x5581cb05ef40 2024-04-23T10:40:27.917 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.905+0000 7ff0c67d7740 10 client.17523 fetch_fsmap learned FSMap version 26 2024-04-23T10:40:27.917 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.905+0000 7ff0c67d7740 1 -- 172.21.15.97:0/4264688744 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5581cb0783a0 con 0x5581cb05ef40 2024-04-23T10:40:27.917 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.905+0000 7ff0a57fa640 1 -- 172.21.15.97:0/4264688744 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0b809cbe0 con 0x5581cb05ef40 2024-04-23T10:40:27.917 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.906+0000 7ff0a57fa640 1 -- 172.21.15.97:0/4264688744 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff0ac022020 con 0x5581cb05ef40 2024-04-23T10:40:27.917 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.906+0000 7ff0c67d7740 10 client.17523 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:40:27.917 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.906+0000 7ff0c67d7740 10 client.17523 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:40:27.917 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.906+0000 7ff0c67d7740 20 client.17523 populate_metadata read hostname 'smithi097' 2024-04-23T10:40:27.917 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.906+0000 7ff0c67d7740 1 --2- 172.21.15.97:0/4264688744 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5581cb079810 0x5581cb07bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:27.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.906+0000 7ff0c67d7740 4 client.17523 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:40:27.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.906+0000 7ff0c67d7740 1 -- 172.21.15.97:0/4264688744 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581cafd1390 con 0x5581cb079810 2024-04-23T10:40:27.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.906+0000 7ff0c1867640 1 --2- 172.21.15.97:0/4264688744 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5581cb079810 0x5581cb07bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:27.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.906+0000 7ff0a57fa640 10 client.17523 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:40:27.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.906+0000 7ff0c1867640 1 --2- 172.21.15.97:0/4264688744 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5581cb079810 0x5581cb07bc00 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6b244ac08aface7 server_cookie=fc74470379f7269a in_seq=0 out_seq=0 2024-04-23T10:40:27.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.907+0000 7ff0a57fa640 1 -- 172.21.15.97:0/4264688744 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5581cafd1390 con 0x5581cb079810 2024-04-23T10:40:27.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.907+0000 7ff0a57fa640 10 client.17523 handle_command_reply: tid=0 2024-04-23T10:40:27.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.907+0000 7ff0c67d7740 1 client.17523 shutdown 2024-04-23T10:40:27.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.907+0000 7ff0c67d7740 1 -- 172.21.15.97:0/4264688744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581cb05ef40 msgr2=0x5581cb066420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:40:27.918 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.907+0000 7ff0c67d7740 1 --2- 172.21.15.97:0/4264688744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581cb05ef40 0x5581cb066420 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac00b990 tx=0x7ff0ac00be50 comp rx=0 tx=0).stop 2024-04-23T10:40:27.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.907+0000 7ff0c67d7740 1 -- 172.21.15.97:0/4264688744 shutdown_connections 2024-04-23T10:40:27.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.907+0000 7ff0c67d7740 1 --2- 172.21.15.97:0/4264688744 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5581cb079810 0x5581cb07bc00 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:27.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.907+0000 7ff0c67d7740 1 --2- 172.21.15.97:0/4264688744 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581cb069800 0x5581cb066960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:27.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.907+0000 7ff0c67d7740 1 --2- 172.21.15.97:0/4264688744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581cb05ef40 0x5581cb066420 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:27.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.907+0000 7ff0a57fa640 0 client.17523 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:40:27.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.907+0000 7ff0c67d7740 1 --2- 172.21.15.97:0/4264688744 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581cb05e620 0x5581cb069130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:27.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.907+0000 7ff0c67d7740 1 -- 172.21.15.97:0/4264688744 >> 172.21.15.97:0/4264688744 conn(0x5581cb05ab30 msgr2=0x5581cb0608b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:40:27.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.907+0000 7ff0c67d7740 1 -- 172.21.15.97:0/4264688744 shutdown_connections 2024-04-23T10:40:27.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.907+0000 7ff0c67d7740 1 -- 172.21.15.97:0/4264688744 wait complete. 2024-04-23T10:40:27.919 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.907+0000 7ff0c67d7740 20 client.17523 trim_cache size 0 max 16384 2024-04-23T10:40:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.922+0000 7ff0c67d7740 1 Processor -- start 2024-04-23T10:40:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.922+0000 7ff0c67d7740 1 -- start start 2024-04-23T10:40:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.922+0000 7ff0c67d7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581cb05ab30 0x5581cb0752f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.922+0000 7ff0c67d7740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581cb05e620 0x5581cb059b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.922+0000 7ff0c67d7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581cb05ef40 0x5581cb081510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.922+0000 7ff0c67d7740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff0bc1afd80 con 0x5581cb05e620 2024-04-23T10:40:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.922+0000 7ff0c67d7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff0bc1b18c0 con 0x5581cb05ef40 2024-04-23T10:40:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.922+0000 7ff0c67d7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff0bc1b0450 con 0x5581cb05ab30 2024-04-23T10:40:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.922+0000 7ff0c1066640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581cb05e620 0x5581cb059b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.923+0000 7ff0c1066640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581cb05e620 0x5581cb059b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43492/0 (socket says 172.21.15.97:43492) 2024-04-23T10:40:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.923+0000 7ff0c1066640 1 -- 172.21.15.97:0/868415996 learned_addr learned my addr 172.21.15.97:0/868415996 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:40:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.923+0000 7ff0c1867640 1 --2- 172.21.15.97:0/868415996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581cb05ab30 0x5581cb0752f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.923+0000 7ff0c1066640 1 -- 172.21.15.97:0/868415996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581cb05ab30 msgr2=0x5581cb0752f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:40:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.923+0000 7ff0c1066640 1 --2- 172.21.15.97:0/868415996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581cb05ab30 0x5581cb0752f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.923+0000 7ff0c1066640 1 -- 172.21.15.97:0/868415996 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581cb05ef40 msgr2=0x5581cb081510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:40:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.923+0000 7ff0c2068640 1 --2- 172.21.15.97:0/868415996 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581cb05ef40 0x5581cb081510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.923+0000 7ff0c1066640 1 --2- 172.21.15.97:0/868415996 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581cb05ef40 0x5581cb081510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.923+0000 7ff0c1066640 1 -- 172.21.15.97:0/868415996 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581cb05ed70 con 0x5581cb05e620 2024-04-23T10:40:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.923+0000 7ff0c2068640 1 --2- 172.21.15.97:0/868415996 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581cb05ef40 0x5581cb081510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:40:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.923+0000 7ff0c1066640 1 --2- 172.21.15.97:0/868415996 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581cb05e620 0x5581cb059b30 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac00b6b0 tx=0x7ff0ac005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e7bf898400c4681 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:40:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.923+0000 7ff0a77fe640 1 -- 172.21.15.97:0/868415996 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0ac005560 con 0x5581cb05e620 2024-04-23T10:40:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.923+0000 7ff0a77fe640 1 -- 172.21.15.97:0/868415996 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0ac01b7d0 con 0x5581cb05e620 2024-04-23T10:40:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.923+0000 7ff0a77fe640 1 -- 172.21.15.97:0/868415996 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0ac011440 con 0x5581cb05e620 2024-04-23T10:40:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.924+0000 7ff0c67d7740 1 -- 172.21.15.97:0/868415996 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581cb05e620 msgr2=0x5581cb059b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:40:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.924+0000 7ff0c67d7740 1 --2- 172.21.15.97:0/868415996 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581cb05e620 0x5581cb059b30 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac00b6b0 tx=0x7ff0ac005c50 comp rx=0 tx=0).stop 2024-04-23T10:40:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.924+0000 7ff0c67d7740 1 -- 172.21.15.97:0/868415996 shutdown_connections 2024-04-23T10:40:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.924+0000 7ff0c67d7740 1 --2- 172.21.15.97:0/868415996 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581cb05ef40 0x5581cb081510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.924+0000 7ff0c67d7740 1 --2- 172.21.15.97:0/868415996 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581cb05e620 0x5581cb059b30 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.924+0000 7ff0c67d7740 1 --2- 172.21.15.97:0/868415996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581cb05ab30 0x5581cb0752f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.924+0000 7ff0c67d7740 1 -- 172.21.15.97:0/868415996 >> 172.21.15.97:0/868415996 conn(0x5581cb066ac0 msgr2=0x5581cb059420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:40:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.924+0000 7ff0c67d7740 1 -- 172.21.15.97:0/868415996 shutdown_connections 2024-04-23T10:40:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.924+0000 7ff0c67d7740 1 -- 172.21.15.97:0/868415996 wait complete. 2024-04-23T10:40:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.924+0000 7ff0c67d7740 1 -- start start 2024-04-23T10:40:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.924+0000 7ff0c67d7740 1 Processor -- start 2024-04-23T10:40:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.924+0000 7ff0c67d7740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581cb05ab30 0x5581cb06f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.924+0000 7ff0c67d7740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581cb05e620 0x5581cb06f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.924+0000 7ff0c67d7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581cb05ef40 0x5581cb06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.925+0000 7ff0c67d7740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff0bc1b11f0 con 0x5581cb05e620 2024-04-23T10:40:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.925+0000 7ff0c67d7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff0bc1b0b20 con 0x5581cb05ef40 2024-04-23T10:40:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.925+0000 7ff0c67d7740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7ff0bc1add80 con 0x5581cb05ab30 2024-04-23T10:40:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.925+0000 7ff0c67d7740 10 client.0 _ll_register_callbacks cb 0x7ff0ac0045b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:40:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.925+0000 7ff0c1066640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581cb05e620 0x5581cb06f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.925+0000 7ff0c1066640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581cb05e620 0x5581cb06f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43496/0 (socket says 172.21.15.97:43496) 2024-04-23T10:40:27.932 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.925+0000 7ff0c1066640 1 -- 172.21.15.97:0/1506020328 learned_addr learned my addr 172.21.15.97:0/1506020328 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:40:27.932 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.925+0000 7ff0c1867640 1 --2- 172.21.15.97:0/1506020328 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581cb05ab30 0x5581cb06f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:27.932 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.925+0000 7ff0c2068640 1 --2- 172.21.15.97:0/1506020328 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581cb05ef40 0x5581cb06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:27.932 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.925+0000 7ff0c1066640 1 -- 172.21.15.97:0/1506020328 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581cb05ab30 msgr2=0x5581cb06f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:40:27.932 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.925+0000 7ff0c1066640 1 --2- 172.21.15.97:0/1506020328 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581cb05ab30 0x5581cb06f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:27.932 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.925+0000 7ff0c1066640 1 -- 172.21.15.97:0/1506020328 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581cb05ef40 msgr2=0x5581cb06fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:40:27.932 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.925+0000 7ff0c1066640 1 --2- 172.21.15.97:0/1506020328 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581cb05ef40 0x5581cb06fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:27.932 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.925+0000 7ff0c1066640 1 -- 172.21.15.97:0/1506020328 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581cb0ded90 con 0x5581cb05e620 2024-04-23T10:40:27.932 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.925+0000 7ff0c1066640 1 --2- 172.21.15.97:0/1506020328 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581cb05e620 0x5581cb06f750 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x5581cb0552d0 tx=0x7ff0ac00b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8cf41ce7a987bf3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:40:27.932 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.925+0000 7ff0a57fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:40:27.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.925+0000 7ff0a57fa640 1 -- 172.21.15.97:0/1506020328 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0ac024bd0 con 0x5581cb05e620 2024-04-23T10:40:27.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.925+0000 7ff0a57fa640 1 -- 172.21.15.97:0/1506020328 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8484+0+0 (secure 0 0 0) 0x7ff0ac024d70 con 0x5581cb05e620 2024-04-23T10:40:27.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.925+0000 7ff0c67d7740 1 -- 172.21.15.97:0/1506020328 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0ac01b7d0 con 0x5581cb05e620 2024-04-23T10:40:27.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.925+0000 7ff0a57fa640 1 -- 172.21.15.97:0/1506020328 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff0ac024460 con 0x5581cb05e620 2024-04-23T10:40:27.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.925+0000 7ff0c67d7740 10 client.17529 fetch_fsmap learned FSMap version 26 2024-04-23T10:40:27.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.925+0000 7ff0c67d7740 1 -- 172.21.15.97:0/1506020328 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5581cb0dffe0 con 0x5581cb05e620 2024-04-23T10:40:27.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.926+0000 7ff0a57fa640 1 -- 172.21.15.97:0/1506020328 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0ac01b7d0 con 0x5581cb05e620 2024-04-23T10:40:27.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.926+0000 7ff0a57fa640 1 -- 172.21.15.97:0/1506020328 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff0ac03f020 con 0x5581cb05e620 2024-04-23T10:40:27.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.926+0000 7ff0c67d7740 10 client.17529 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:40:27.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.926+0000 7ff0c67d7740 10 client.17529 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:40:27.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.926+0000 7ff0c67d7740 20 client.17529 populate_metadata read hostname 'smithi097' 2024-04-23T10:40:27.933 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.926+0000 7ff0c67d7740 1 --2- 172.21.15.97:0/1506020328 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5581cb069800 0x5581cb0e26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:27.934 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.926+0000 7ff0c67d7740 4 client.17529 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:40:27.934 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.926+0000 7ff0c67d7740 1 -- 172.21.15.97:0/1506020328 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5581cafcb200 con 0x5581cb069800 2024-04-23T10:40:27.934 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.926+0000 7ff0c1867640 1 --2- 172.21.15.97:0/1506020328 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5581cb069800 0x5581cb0e26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:27.934 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.926+0000 7ff0a57fa640 10 client.17529 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:40:27.934 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.926+0000 7ff0c1867640 1 --2- 172.21.15.97:0/1506020328 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5581cb069800 0x5581cb0e26d0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f206b71165dda6 server_cookie=447abb0c9932c3f4 in_seq=0 out_seq=0 2024-04-23T10:40:27.941 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.942+0000 7ff0a57fa640 1 -- 172.21.15.97:0/1506020328 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5581cafcb200 con 0x5581cb069800 2024-04-23T10:40:27.941 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.942+0000 7ff0a57fa640 10 client.17529 handle_command_reply: tid=0 2024-04-23T10:40:27.942 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.942+0000 7ff0c67d7740 1 client.17529 shutdown 2024-04-23T10:40:27.942 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.942+0000 7ff0c67d7740 1 -- 172.21.15.97:0/1506020328 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581cb05e620 msgr2=0x5581cb06f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:40:27.942 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.942+0000 7ff0c67d7740 1 --2- 172.21.15.97:0/1506020328 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581cb05e620 0x5581cb06f750 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x5581cb0552d0 tx=0x7ff0ac00b390 comp rx=0 tx=0).stop 2024-04-23T10:40:27.942 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.942+0000 7ff0c67d7740 1 -- 172.21.15.97:0/1506020328 shutdown_connections 2024-04-23T10:40:27.942 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.942+0000 7ff0c67d7740 1 --2- 172.21.15.97:0/1506020328 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5581cb069800 0x5581cb0e26d0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:27.942 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.942+0000 7ff0c67d7740 1 --2- 172.21.15.97:0/1506020328 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5581cb05ef40 0x5581cb06fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:27.942 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.942+0000 7ff0a57fa640 0 client.17529 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:40:27.942 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.942+0000 7ff0c67d7740 1 --2- 172.21.15.97:0/1506020328 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5581cb05e620 0x5581cb06f750 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:27.942 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.942+0000 7ff0c67d7740 1 --2- 172.21.15.97:0/1506020328 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5581cb05ab30 0x5581cb06f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:27.942 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.942+0000 7ff0c67d7740 1 -- 172.21.15.97:0/1506020328 >> 172.21.15.97:0/1506020328 conn(0x5581cb066ac0 msgr2=0x5581cb059cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:40:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.942+0000 7ff0c67d7740 1 -- 172.21.15.97:0/1506020328 shutdown_connections 2024-04-23T10:40:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.942+0000 7ff0c67d7740 1 -- 172.21.15.97:0/1506020328 wait complete. 2024-04-23T10:40:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.942+0000 7ff0c67d7740 20 client.17529 trim_cache size 0 max 16384 2024-04-23T10:40:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:40:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:40:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:40:27.943 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:40:27.944 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.943+0000 7ff0c3acb640 1 -- 172.21.15.97:0/3561492584 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7ff0a80a3df0 msgr2=0x7ff0a80a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:40:27.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.943+0000 7ff0c3acb640 1 --2- 172.21.15.97:0/3561492584 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7ff0a80a3df0 0x7ff0a80a62b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff0b4033ff0 tx=0x7ff0b400a6d0 comp rx=0 tx=0).stop 2024-04-23T10:40:27.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.943+0000 7ff0c3acb640 1 -- 172.21.15.97:0/3561492584 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0bc1005c0 msgr2=0x7ff0bc102260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:40:27.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.943+0000 7ff0c3acb640 1 --2- 172.21.15.97:0/3561492584 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0bc1005c0 0x7ff0bc102260 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7ff0b800c970 tx=0x7ff0b800ce30 comp rx=0 tx=0).stop 2024-04-23T10:40:27.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.943+0000 7ff0c3acb640 1 -- 172.21.15.97:0/3561492584 shutdown_connections 2024-04-23T10:40:27.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.943+0000 7ff0c3acb640 1 --2- 172.21.15.97:0/3561492584 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7ff0a80a3df0 0x7ff0a80a62b0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:27.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.943+0000 7ff0c3acb640 1 --2- 172.21.15.97:0/3561492584 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0bc106660 0x7ff0bc102ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:27.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.943+0000 7ff0c3acb640 1 --2- 172.21.15.97:0/3561492584 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0bc100f80 0x7ff0bc1027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:27.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.943+0000 7ff0c3acb640 1 --2- 172.21.15.97:0/3561492584 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0bc1005c0 0x7ff0bc102260 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:27.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.943+0000 7ff0c3acb640 1 -- 172.21.15.97:0/3561492584 >> 172.21.15.97:0/3561492584 conn(0x7ff0bc077370 msgr2=0x7ff0bc104940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:40:27.945 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.943+0000 7ff0c3acb640 1 -- 172.21.15.97:0/3561492584 shutdown_connections 2024-04-23T10:40:27.946 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:27.943+0000 7ff0c3acb640 1 -- 172.21.15.97:0/3561492584 wait complete. 2024-04-23T10:40:28.844 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:40:28 smithi196 ceph-mon[32919]: pgmap v447: 97 pgs: 97 active+clean; 5.2 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-23T10:40:28.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:40:28 smithi097 ceph-mon[29197]: pgmap v447: 97 pgs: 97 active+clean; 5.2 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-23T10:40:28.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:40:28 smithi183 ceph-mon[32317]: pgmap v447: 97 pgs: 97 active+clean; 5.2 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-23T10:40:30.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:40:30 smithi196 ceph-mon[32919]: pgmap v448: 97 pgs: 97 active+clean; 5.2 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 34 KiB/s wr, 12 op/s 2024-04-23T10:40:30.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:40:30 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:40:30.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:40:30 smithi097 ceph-mon[29197]: pgmap v448: 97 pgs: 97 active+clean; 5.2 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 34 KiB/s wr, 12 op/s 2024-04-23T10:40:30.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:40:30 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:40:30.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:40:30 smithi183 ceph-mon[32317]: pgmap v448: 97 pgs: 97 active+clean; 5.2 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 34 KiB/s wr, 12 op/s 2024-04-23T10:40:30.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:40:30 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:40:31.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:40:31 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:40:31.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:40:31 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:40:31.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:40:31 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:40:31.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:40:31 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:40:31.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:40:31 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:40:31.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:40:31 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:40:31.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:40:31 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:40:31.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:40:31 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:40:31.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:40:31 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:40:33.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:40:32 smithi196 ceph-mon[32919]: pgmap v449: 97 pgs: 97 active+clean; 5.2 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 34 KiB/s wr, 12 op/s 2024-04-23T10:40:33.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:40:32 smithi097 ceph-mon[29197]: pgmap v449: 97 pgs: 97 active+clean; 5.2 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 34 KiB/s wr, 12 op/s 2024-04-23T10:40:33.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:40:32 smithi183 ceph-mon[32317]: pgmap v449: 97 pgs: 97 active+clean; 5.2 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 34 KiB/s wr, 12 op/s 2024-04-23T10:40:35.092 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:40:34 smithi196 ceph-mon[32919]: pgmap v450: 97 pgs: 97 active+clean; 5.4 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 58 KiB/s wr, 22 op/s 2024-04-23T10:40:35.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:40:34 smithi097 ceph-mon[29197]: pgmap v450: 97 pgs: 97 active+clean; 5.4 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 58 KiB/s wr, 22 op/s 2024-04-23T10:40:35.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:40:34 smithi183 ceph-mon[32317]: pgmap v450: 97 pgs: 97 active+clean; 5.4 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 58 KiB/s wr, 22 op/s 2024-04-23T10:40:37.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:40:36 smithi097 ceph-mon[29197]: pgmap v451: 97 pgs: 97 active+clean; 5.4 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:40:37.149 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:40:36 smithi183 ceph-mon[32317]: pgmap v451: 97 pgs: 97 active+clean; 5.4 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:40:37.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:40:36 smithi196 ceph-mon[32919]: pgmap v451: 97 pgs: 97 active+clean; 5.4 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:40:39.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:40:38 smithi097 ceph-mon[29197]: pgmap v452: 97 pgs: 97 active+clean; 5.6 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 KiB/s wr, 22 op/s 2024-04-23T10:40:39.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:40:38 smithi183 ceph-mon[32317]: pgmap v452: 97 pgs: 97 active+clean; 5.6 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 KiB/s wr, 22 op/s 2024-04-23T10:40:39.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:40:38 smithi196 ceph-mon[32919]: pgmap v452: 97 pgs: 97 active+clean; 5.6 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 KiB/s wr, 22 op/s 2024-04-23T10:40:41.039 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.038+0000 7f08f6d64640 1 -- 172.21.15.97:0/2766885352 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08f0101af0 msgr2=0x7f08f010dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:40:41.040 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.038+0000 7f08f6d64640 1 --2- 172.21.15.97:0/2766885352 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08f0101af0 0x7f08f010dca0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f08e8009cd0 tx=0x7f08e802a8e0 comp rx=0 tx=0).stop 2024-04-23T10:40:41.040 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.038+0000 7f08f6d64640 1 -- 172.21.15.97:0/2766885352 shutdown_connections 2024-04-23T10:40:41.040 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.038+0000 7f08f6d64640 1 --2- 172.21.15.97:0/2766885352 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08f0101af0 0x7f08f010dca0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:41.040 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.038+0000 7f08f6d64640 1 --2- 172.21.15.97:0/2766885352 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08f0101130 0x7f08f01015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:41.040 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.038+0000 7f08f6d64640 1 --2- 172.21.15.97:0/2766885352 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f08f0106f90 0x7f08f0107370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:41.040 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.038+0000 7f08f6d64640 1 -- 172.21.15.97:0/2766885352 >> 172.21.15.97:0/2766885352 conn(0x7f08f0077c30 msgr2=0x7f08f0078040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:40:41.040 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.039+0000 7f08f6d64640 1 -- 172.21.15.97:0/2766885352 shutdown_connections 2024-04-23T10:40:41.040 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.039+0000 7f08f6d64640 1 -- 172.21.15.97:0/2766885352 wait complete. 2024-04-23T10:40:41.040 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.039+0000 7f08f6d64640 1 Processor -- start 2024-04-23T10:40:41.040 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.039+0000 7f08f6d64640 1 -- start start 2024-04-23T10:40:41.040 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.039+0000 7f08f6d64640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08f0101130 0x7f08f01974b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:41.040 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.039+0000 7f08f6d64640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f08f0101af0 0x7f08f01979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:41.040 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.039+0000 7f08f6d64640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08f0106f90 0x7f08f019bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:41.041 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.039+0000 7f08f6d64640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f08f0111090 con 0x7f08f0101130 2024-04-23T10:40:41.041 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.039+0000 7f08f6d64640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f08f0110f10 con 0x7f08f0106f90 2024-04-23T10:40:41.041 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.039+0000 7f08f6d64640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f08f0111210 con 0x7f08f0101af0 2024-04-23T10:40:41.041 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.039+0000 7f08f4b00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08f0101130 0x7f08f01974b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:41.041 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.040+0000 7f08f4b00640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08f0101130 0x7f08f01974b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57130/0 (socket says 172.21.15.97:57130) 2024-04-23T10:40:41.041 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.040+0000 7f08f4b00640 1 -- 172.21.15.97:0/334369497 learned_addr learned my addr 172.21.15.97:0/334369497 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:40:41.041 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.040+0000 7f08f5301640 1 --2- 172.21.15.97:0/334369497 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08f0106f90 0x7f08f019bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:41.041 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.040+0000 7f08e7fff640 1 --2- 172.21.15.97:0/334369497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f08f0101af0 0x7f08f01979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:41.041 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.040+0000 7f08f4b00640 1 -- 172.21.15.97:0/334369497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f08f0101af0 msgr2=0x7f08f01979f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:40:41.041 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.040+0000 7f08f4b00640 1 --2- 172.21.15.97:0/334369497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f08f0101af0 0x7f08f01979f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:41.041 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.040+0000 7f08f4b00640 1 -- 172.21.15.97:0/334369497 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08f0106f90 msgr2=0x7f08f019bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:40:41.041 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.040+0000 7f08f4b00640 1 --2- 172.21.15.97:0/334369497 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08f0106f90 0x7f08f019bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:41.041 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.040+0000 7f08f4b00640 1 -- 172.21.15.97:0/334369497 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08f019c4a0 con 0x7f08f0101130 2024-04-23T10:40:41.041 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.040+0000 7f08e7fff640 1 --2- 172.21.15.97:0/334369497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f08f0101af0 0x7f08f01979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:40:41.042 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.040+0000 7f08f4b00640 1 --2- 172.21.15.97:0/334369497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08f0101130 0x7f08f01974b0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f08ec00d770 tx=0x7f08ec00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:40:41.042 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.040+0000 7f08e5ffb640 1 -- 172.21.15.97:0/334369497 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ec017070 con 0x7f08f0101130 2024-04-23T10:40:41.042 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.040+0000 7f08e5ffb640 1 -- 172.21.15.97:0/334369497 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08ec0054d0 con 0x7f08f0101130 2024-04-23T10:40:41.042 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.040+0000 7f08f6d64640 1 -- 172.21.15.97:0/334369497 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08f019c790 con 0x7f08f0101130 2024-04-23T10:40:41.042 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.040+0000 7f08e5ffb640 1 -- 172.21.15.97:0/334369497 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ec004c30 con 0x7f08f0101130 2024-04-23T10:40:41.042 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.041+0000 7f08f6d64640 1 -- 172.21.15.97:0/334369497 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08f01a4020 con 0x7f08f0101130 2024-04-23T10:40:41.048 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.043+0000 7f08e5ffb640 1 -- 172.21.15.97:0/334369497 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f08ec004e10 con 0x7f08f0101130 2024-04-23T10:40:41.048 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.043+0000 7f08f9a7f740 1 Processor -- start 2024-04-23T10:40:41.048 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.043+0000 7f08f9a7f740 1 -- start start 2024-04-23T10:40:41.048 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.043+0000 7f08f9a7f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56510017b540 0x56510017b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:41.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.043+0000 7f08e5ffb640 1 --2- 172.21.15.97:0/334369497 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f08d80a3dc0 0x7f08d80a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:41.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.043+0000 7f08f9a7f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56510017be60 0x5651001884d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:41.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.043+0000 7f08f9a7f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x565100188a10 0x56510018ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:41.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.043+0000 7f08f9a7f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5651000e0970 con 0x565100188a10 2024-04-23T10:40:41.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.043+0000 7f08f9a7f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56510007d4a0 con 0x56510017be60 2024-04-23T10:40:41.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.043+0000 7f08f9a7f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5651000dc930 con 0x56510017b540 2024-04-23T10:40:41.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.043+0000 7f08f4b00640 1 --2- 172.21.15.97:0/334369497 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f08d80a3dc0 0x7f08d80a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:41.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.043+0000 7f08f5301640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x565100188a10 0x56510018ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:41.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.043+0000 7f08e5ffb640 1 -- 172.21.15.97:0/334369497 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8484+0+0 (secure 0 0 0) 0x7f08ec0d0240 con 0x7f08f0101130 2024-04-23T10:40:41.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.043+0000 7f08f5301640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x565100188a10 0x56510018ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57136/0 (socket says 172.21.15.97:57136) 2024-04-23T10:40:41.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.043+0000 7f08f5301640 1 -- 172.21.15.97:0/3914529246 learned_addr learned my addr 172.21.15.97:0/3914529246 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:40:41.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.044+0000 7f08f5301640 1 --2- 172.21.15.97:0/3914529246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56510017b540 0x56510017b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:41.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.044+0000 7f08e7fff640 1 --2- 172.21.15.97:0/3914529246 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56510017be60 0x5651001884d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:41.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.044+0000 7f08f5301640 1 -- 172.21.15.97:0/3914529246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56510017b540 msgr2=0x56510017b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:40:41.049 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.044+0000 7f08f5301640 1 --2- 172.21.15.97:0/3914529246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56510017b540 0x56510017b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:41.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.044+0000 7f08f5301640 1 -- 172.21.15.97:0/3914529246 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56510017be60 msgr2=0x5651001884d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:40:41.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.044+0000 7f08f5301640 1 --2- 172.21.15.97:0/3914529246 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56510017be60 0x5651001884d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:41.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.044+0000 7f08f5301640 1 -- 172.21.15.97:0/3914529246 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650ffdf3210 con 0x565100188a10 2024-04-23T10:40:41.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.044+0000 7f08e7fff640 1 --2- 172.21.15.97:0/3914529246 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56510017be60 0x5651001884d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:40:41.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.044+0000 7f08f4b00640 1 --2- 172.21.15.97:0/334369497 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f08d80a3dc0 0x7f08d80a6280 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f08ec00d740 tx=0x7f08ec061040 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:40:41.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.044+0000 7f08f5301640 1 --2- 172.21.15.97:0/3914529246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x565100188a10 0x56510018ae00 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f08e8009cd0 tx=0x7f08e8004a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b0bc0cf751099ef server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:40:41.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.044+0000 7f08d6ffd640 1 -- 172.21.15.97:0/3914529246 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e8031990 con 0x565100188a10 2024-04-23T10:40:41.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.044+0000 7f08d6ffd640 1 -- 172.21.15.97:0/3914529246 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08e8036050 con 0x565100188a10 2024-04-23T10:40:41.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.044+0000 7f08d6ffd640 1 -- 172.21.15.97:0/3914529246 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e80044c0 con 0x565100188a10 2024-04-23T10:40:41.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.045+0000 7f08f9a7f740 1 -- 172.21.15.97:0/3914529246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x565100188a10 msgr2=0x56510018ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:40:41.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.045+0000 7f08f9a7f740 1 --2- 172.21.15.97:0/3914529246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x565100188a10 0x56510018ae00 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f08e8009cd0 tx=0x7f08e8004a60 comp rx=0 tx=0).stop 2024-04-23T10:40:41.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.045+0000 7f08f9a7f740 1 -- 172.21.15.97:0/3914529246 shutdown_connections 2024-04-23T10:40:41.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.045+0000 7f08f9a7f740 1 --2- 172.21.15.97:0/3914529246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x565100188a10 0x56510018ae00 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:41.050 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.045+0000 7f08f9a7f740 1 --2- 172.21.15.97:0/3914529246 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56510017be60 0x5651001884d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:41.051 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.045+0000 7f08f9a7f740 1 --2- 172.21.15.97:0/3914529246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56510017b540 0x56510017b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:41.051 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.045+0000 7f08f9a7f740 1 -- 172.21.15.97:0/3914529246 >> 172.21.15.97:0/3914529246 conn(0x565100177b30 msgr2=0x565100177f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:40:41.051 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.045+0000 7f08f9a7f740 1 -- 172.21.15.97:0/3914529246 shutdown_connections 2024-04-23T10:40:41.051 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.045+0000 7f08f9a7f740 1 -- 172.21.15.97:0/3914529246 wait complete. 2024-04-23T10:40:41.051 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.045+0000 7f08f9a7f740 1 -- start start 2024-04-23T10:40:41.051 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.045+0000 7f08f9a7f740 1 Processor -- start 2024-04-23T10:40:41.051 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.046+0000 7f08f9a7f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56510017b540 0x5651001883f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:41.051 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.046+0000 7f08f9a7f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56510017be60 0x5651001856e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:41.051 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.046+0000 7f08f9a7f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565100188a10 0x565100185cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:41.051 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.046+0000 7f08f9a7f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56510018b5e0 con 0x56510017be60 2024-04-23T10:40:41.051 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.046+0000 7f08f9a7f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56510007b0b0 con 0x56510017b540 2024-04-23T10:40:41.051 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.046+0000 7f08f9a7f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56510018b760 con 0x565100188a10 2024-04-23T10:40:41.051 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.046+0000 7f08f9a7f740 10 client.0 _ll_register_callbacks cb 0x565100068d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:40:41.051 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.046+0000 7f08e7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56510017be60 0x5651001856e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:41.051 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.046+0000 7f08e7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56510017be60 0x5651001856e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57150/0 (socket says 172.21.15.97:57150) 2024-04-23T10:40:41.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.046+0000 7f08e7fff640 1 -- 172.21.15.97:0/4029695586 learned_addr learned my addr 172.21.15.97:0/4029695586 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:40:41.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.046+0000 7f08f5301640 1 --2- 172.21.15.97:0/4029695586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56510017b540 0x5651001883f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:41.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.046+0000 7f08f5301640 1 --2- 172.21.15.97:0/4029695586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565100188a10 0x565100185cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:41.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.046+0000 7f08e7fff640 1 -- 172.21.15.97:0/4029695586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565100188a10 msgr2=0x565100185cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:40:41.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.046+0000 7f08e7fff640 1 --2- 172.21.15.97:0/4029695586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565100188a10 0x565100185cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:41.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.046+0000 7f08e7fff640 1 -- 172.21.15.97:0/4029695586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56510017b540 msgr2=0x5651001883f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:40:41.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.046+0000 7f08e7fff640 1 --2- 172.21.15.97:0/4029695586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56510017b540 0x5651001883f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:41.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.046+0000 7f08e7fff640 1 -- 172.21.15.97:0/4029695586 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650ffebf620 con 0x56510017be60 2024-04-23T10:40:41.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.046+0000 7f08f5301640 1 --2- 172.21.15.97:0/4029695586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56510017b540 0x5651001883f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:40:41.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.046+0000 7f08e7fff640 1 --2- 172.21.15.97:0/4029695586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56510017be60 0x5651001856e0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f08e000bf40 tx=0x7f08e0011870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=466a1ab4265722db server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:40:41.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.046+0000 7f08cffff640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:40:41.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.047+0000 7f08cffff640 1 -- 172.21.15.97:0/4029695586 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e0012920 con 0x56510017be60 2024-04-23T10:40:41.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.047+0000 7f08cffff640 1 -- 172.21.15.97:0/4029695586 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8484+0+0 (secure 0 0 0) 0x7f08e0012ac0 con 0x56510017be60 2024-04-23T10:40:41.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.047+0000 7f08f9a7f740 1 -- 172.21.15.97:0/4029695586 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08e8036050 con 0x56510017be60 2024-04-23T10:40:41.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.047+0000 7f08cffff640 1 -- 172.21.15.97:0/4029695586 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f08e0012cc0 con 0x56510017be60 2024-04-23T10:40:41.052 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.047+0000 7f08f9a7f740 10 client.17541 fetch_fsmap learned FSMap version 26 2024-04-23T10:40:41.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.047+0000 7f08f9a7f740 1 -- 172.21.15.97:0/4029695586 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x565100197630 con 0x56510017be60 2024-04-23T10:40:41.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.047+0000 7f08cffff640 1 -- 172.21.15.97:0/4029695586 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08e8036050 con 0x56510017be60 2024-04-23T10:40:41.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.047+0000 7f08cffff640 1 -- 172.21.15.97:0/4029695586 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f08e0025490 con 0x56510017be60 2024-04-23T10:40:41.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.047+0000 7f08f9a7f740 10 client.17541 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:40:41.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.047+0000 7f08f9a7f740 10 client.17541 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:40:41.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.047+0000 7f08f9a7f740 20 client.17541 populate_metadata read hostname 'smithi097' 2024-04-23T10:40:41.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.047+0000 7f08f9a7f740 1 --2- 172.21.15.97:0/4029695586 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x565100198af0 0x56510019af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:41.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.047+0000 7f08f9a7f740 4 client.17541 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:40:41.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.047+0000 7f08f9a7f740 1 -- 172.21.15.97:0/4029695586 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651000ee390 con 0x565100198af0 2024-04-23T10:40:41.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.047+0000 7f08f5301640 1 --2- 172.21.15.97:0/4029695586 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x565100198af0 0x56510019af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:41.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.048+0000 7f08cffff640 10 client.17541 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:40:41.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.048+0000 7f08f5301640 1 --2- 172.21.15.97:0/4029695586 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x565100198af0 0x56510019af00 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8149c40e1edcb83 server_cookie=5cd21d8359d83852 in_seq=0 out_seq=0 2024-04-23T10:40:41.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.048+0000 7f08cffff640 1 -- 172.21.15.97:0/4029695586 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5651000ee390 con 0x565100198af0 2024-04-23T10:40:41.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.049+0000 7f08cffff640 10 client.17541 handle_command_reply: tid=0 2024-04-23T10:40:41.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.049+0000 7f08f9a7f740 1 client.17541 shutdown 2024-04-23T10:40:41.053 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.049+0000 7f08f9a7f740 1 -- 172.21.15.97:0/4029695586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56510017be60 msgr2=0x5651001856e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:40:41.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.049+0000 7f08f9a7f740 1 --2- 172.21.15.97:0/4029695586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56510017be60 0x5651001856e0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f08e000bf40 tx=0x7f08e0011870 comp rx=0 tx=0).stop 2024-04-23T10:40:41.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.049+0000 7f08f9a7f740 1 -- 172.21.15.97:0/4029695586 shutdown_connections 2024-04-23T10:40:41.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.049+0000 7f08f9a7f740 1 --2- 172.21.15.97:0/4029695586 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x565100198af0 0x56510019af00 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:41.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.049+0000 7f08f9a7f740 1 --2- 172.21.15.97:0/4029695586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565100188a10 0x565100185cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:41.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.049+0000 7f08f9a7f740 1 --2- 172.21.15.97:0/4029695586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56510017be60 0x5651001856e0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:41.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.049+0000 7f08cffff640 0 client.17541 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:40:41.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.049+0000 7f08f9a7f740 1 --2- 172.21.15.97:0/4029695586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56510017b540 0x5651001883f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:41.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.049+0000 7f08f9a7f740 1 -- 172.21.15.97:0/4029695586 >> 172.21.15.97:0/4029695586 conn(0x565100177b30 msgr2=0x56510017d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:40:41.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.049+0000 7f08f9a7f740 1 -- 172.21.15.97:0/4029695586 shutdown_connections 2024-04-23T10:40:41.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.049+0000 7f08f9a7f740 1 -- 172.21.15.97:0/4029695586 wait complete. 2024-04-23T10:40:41.054 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.049+0000 7f08f9a7f740 20 client.17541 trim_cache size 0 max 16384 2024-04-23T10:40:41.066 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.064+0000 7f08f9a7f740 1 Processor -- start 2024-04-23T10:40:41.066 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.064+0000 7f08f9a7f740 1 -- start start 2024-04-23T10:40:41.066 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.064+0000 7f08f9a7f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56510017b540 0x5651001944a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:41.067 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.064+0000 7f08f9a7f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56510017be60 0x565100180b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:41.067 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.064+0000 7f08f9a7f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565100188a10 0x565100187f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:41.067 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.064+0000 7f08f9a7f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f08f01ab7d0 con 0x56510017b540 2024-04-23T10:40:41.067 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.064+0000 7f08f9a7f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f08f01ad310 con 0x565100188a10 2024-04-23T10:40:41.067 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.064+0000 7f08f9a7f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f08f01abea0 con 0x56510017be60 2024-04-23T10:40:41.067 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.064+0000 7f08f5301640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56510017b540 0x5651001944a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:41.067 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.064+0000 7f08f5301640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56510017b540 0x5651001944a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57152/0 (socket says 172.21.15.97:57152) 2024-04-23T10:40:41.067 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.064+0000 7f08f5301640 1 -- 172.21.15.97:0/1087681247 learned_addr learned my addr 172.21.15.97:0/1087681247 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:40:41.067 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.064+0000 7f08f5301640 1 -- 172.21.15.97:0/1087681247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56510017be60 msgr2=0x565100180b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:40:41.067 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.064+0000 7f08f5301640 1 --2- 172.21.15.97:0/1087681247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56510017be60 0x565100180b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:41.067 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.064+0000 7f08f5301640 1 -- 172.21.15.97:0/1087681247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565100188a10 msgr2=0x565100187f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:40:41.067 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.064+0000 7f08f5301640 1 --2- 172.21.15.97:0/1087681247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565100188a10 0x565100187f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:41.067 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.064+0000 7f08f5301640 1 -- 172.21.15.97:0/1087681247 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565100186060 con 0x56510017b540 2024-04-23T10:40:41.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.064+0000 7f08f5301640 1 --2- 172.21.15.97:0/1087681247 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56510017b540 0x5651001944a0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f08e80314a0 tx=0x7f08e80098f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aa0ebd20b30fdab server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:40:41.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.064+0000 7f08d67fc640 1 -- 172.21.15.97:0/1087681247 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e8004230 con 0x56510017b540 2024-04-23T10:40:41.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.064+0000 7f08d67fc640 1 -- 172.21.15.97:0/1087681247 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08e8036050 con 0x56510017b540 2024-04-23T10:40:41.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.064+0000 7f08d67fc640 1 -- 172.21.15.97:0/1087681247 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e8003f20 con 0x56510017b540 2024-04-23T10:40:41.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.065+0000 7f08f9a7f740 1 -- 172.21.15.97:0/1087681247 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56510017b540 msgr2=0x5651001944a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:40:41.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.065+0000 7f08f9a7f740 1 --2- 172.21.15.97:0/1087681247 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56510017b540 0x5651001944a0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f08e80314a0 tx=0x7f08e80098f0 comp rx=0 tx=0).stop 2024-04-23T10:40:41.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.065+0000 7f08f9a7f740 1 -- 172.21.15.97:0/1087681247 shutdown_connections 2024-04-23T10:40:41.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.065+0000 7f08f9a7f740 1 --2- 172.21.15.97:0/1087681247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565100188a10 0x565100187f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:41.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.065+0000 7f08f9a7f740 1 --2- 172.21.15.97:0/1087681247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56510017be60 0x565100180b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:41.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.065+0000 7f08f9a7f740 1 --2- 172.21.15.97:0/1087681247 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56510017b540 0x5651001944a0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:41.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.065+0000 7f08f9a7f740 1 -- 172.21.15.97:0/1087681247 >> 172.21.15.97:0/1087681247 conn(0x565100177b30 msgr2=0x565100180470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:40:41.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.065+0000 7f08f9a7f740 1 -- 172.21.15.97:0/1087681247 shutdown_connections 2024-04-23T10:40:41.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.065+0000 7f08f9a7f740 1 -- 172.21.15.97:0/1087681247 wait complete. 2024-04-23T10:40:41.068 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.065+0000 7f08f9a7f740 1 -- start start 2024-04-23T10:40:41.069 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.065+0000 7f08f9a7f740 1 Processor -- start 2024-04-23T10:40:41.069 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.066+0000 7f08f9a7f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56510017b540 0x565100190d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:41.069 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.066+0000 7f08f9a7f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56510017be60 0x56510018c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:41.069 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.066+0000 7f08f9a7f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565100188a10 0x56510018c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:41.069 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.066+0000 7f08f9a7f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f08f01acc40 con 0x56510017b540 2024-04-23T10:40:41.069 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.066+0000 7f08f9a7f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f08f01ac570 con 0x56510017be60 2024-04-23T10:40:41.069 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.066+0000 7f08f9a7f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f08f01a97d0 con 0x565100188a10 2024-04-23T10:40:41.069 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.066+0000 7f08f9a7f740 10 client.0 _ll_register_callbacks cb 0x7f08e000bad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:40:41.069 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.066+0000 7f08f5301640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56510017b540 0x565100190d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:41.069 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.066+0000 7f08f5301640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56510017b540 0x565100190d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:57158/0 (socket says 172.21.15.97:57158) 2024-04-23T10:40:41.069 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.066+0000 7f08f5301640 1 -- 172.21.15.97:0/1183904698 learned_addr learned my addr 172.21.15.97:0/1183904698 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:40:41.069 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.066+0000 7f08e7fff640 1 --2- 172.21.15.97:0/1183904698 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56510017be60 0x56510018c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:41.069 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.066+0000 7f08f5301640 1 -- 172.21.15.97:0/1183904698 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565100188a10 msgr2=0x56510018c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:40:41.069 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.066+0000 7f08f5301640 1 --2- 172.21.15.97:0/1183904698 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565100188a10 0x56510018c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:41.069 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.066+0000 7f08f5301640 1 -- 172.21.15.97:0/1183904698 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56510017be60 msgr2=0x56510018c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:40:41.069 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.066+0000 7f08f5301640 1 --2- 172.21.15.97:0/1183904698 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56510017be60 0x56510018c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:41.070 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.066+0000 7f08f5301640 1 -- 172.21.15.97:0/1183904698 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56510018ce10 con 0x56510017b540 2024-04-23T10:40:41.070 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.066+0000 7f08f5301640 1 --2- 172.21.15.97:0/1183904698 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56510017b540 0x565100190d80 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f08e802ae00 tx=0x7f08e8009960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e94d1e1166b16701 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:40:41.070 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.066+0000 7f08cffff640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:40:41.070 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.066+0000 7f08cffff640 1 -- 172.21.15.97:0/1183904698 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e8004cc0 con 0x56510017b540 2024-04-23T10:40:41.070 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.066+0000 7f08f9a7f740 1 -- 172.21.15.97:0/1183904698 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08e8036050 con 0x56510017b540 2024-04-23T10:40:41.070 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.067+0000 7f08cffff640 1 -- 172.21.15.97:0/1183904698 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8484+0+0 (secure 0 0 0) 0x7f08e8004230 con 0x56510017b540 2024-04-23T10:40:41.070 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.067+0000 7f08cffff640 1 -- 172.21.15.97:0/1183904698 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f08e8064050 con 0x56510017b540 2024-04-23T10:40:41.070 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.067+0000 7f08f9a7f740 10 client.17547 fetch_fsmap learned FSMap version 26 2024-04-23T10:40:41.070 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.067+0000 7f08f9a7f740 1 -- 172.21.15.97:0/1183904698 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5651001ff0a0 con 0x56510017b540 2024-04-23T10:40:41.070 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.067+0000 7f08cffff640 1 -- 172.21.15.97:0/1183904698 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08e8036050 con 0x56510017b540 2024-04-23T10:40:41.070 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.067+0000 7f08cffff640 1 -- 172.21.15.97:0/1183904698 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f08e8048020 con 0x56510017b540 2024-04-23T10:40:41.070 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.067+0000 7f08f9a7f740 10 client.17547 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:40:41.070 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.067+0000 7f08f9a7f740 10 client.17547 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:40:41.070 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.067+0000 7f08f9a7f740 20 client.17547 populate_metadata read hostname 'smithi097' 2024-04-23T10:40:41.071 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.067+0000 7f08f9a7f740 1 --2- 172.21.15.97:0/1183904698 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5651001ff2d0 0x5651002016c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:41.071 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.067+0000 7f08f9a7f740 4 client.17547 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:40:41.071 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.067+0000 7f08f9a7f740 1 -- 172.21.15.97:0/1183904698 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5651000e8200 con 0x5651001ff2d0 2024-04-23T10:40:41.071 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.067+0000 7f08e7fff640 1 --2- 172.21.15.97:0/1183904698 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5651001ff2d0 0x5651002016c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:41.071 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.068+0000 7f08cffff640 10 client.17547 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:40:41.071 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.068+0000 7f08e7fff640 1 --2- 172.21.15.97:0/1183904698 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5651001ff2d0 0x5651002016c0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50d22fd6eb3e32a6 server_cookie=4a8200098f9a3245 in_seq=0 out_seq=0 2024-04-23T10:40:41.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.081+0000 7f08cffff640 1 -- 172.21.15.97:0/1183904698 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5651000e8200 con 0x5651001ff2d0 2024-04-23T10:40:41.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.081+0000 7f08cffff640 10 client.17547 handle_command_reply: tid=0 2024-04-23T10:40:41.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.081+0000 7f08f9a7f740 1 client.17547 shutdown 2024-04-23T10:40:41.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.081+0000 7f08f9a7f740 1 -- 172.21.15.97:0/1183904698 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56510017b540 msgr2=0x565100190d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:40:41.080 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.081+0000 7f08f9a7f740 1 --2- 172.21.15.97:0/1183904698 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56510017b540 0x565100190d80 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f08e802ae00 tx=0x7f08e8009960 comp rx=0 tx=0).stop 2024-04-23T10:40:41.081 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.081+0000 7f08f9a7f740 1 -- 172.21.15.97:0/1183904698 shutdown_connections 2024-04-23T10:40:41.081 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.081+0000 7f08f9a7f740 1 --2- 172.21.15.97:0/1183904698 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5651001ff2d0 0x5651002016c0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:41.081 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.081+0000 7f08f9a7f740 1 --2- 172.21.15.97:0/1183904698 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x565100188a10 0x56510018c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:41.081 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.081+0000 7f08f9a7f740 1 --2- 172.21.15.97:0/1183904698 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56510017be60 0x56510018c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:41.081 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.081+0000 7f08cffff640 0 client.17547 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:40:41.081 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.081+0000 7f08f9a7f740 1 --2- 172.21.15.97:0/1183904698 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56510017b540 0x565100190d80 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:41.081 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.081+0000 7f08f9a7f740 1 -- 172.21.15.97:0/1183904698 >> 172.21.15.97:0/1183904698 conn(0x565100177b30 msgr2=0x565100176cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:40:41.081 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.081+0000 7f08f9a7f740 1 -- 172.21.15.97:0/1183904698 shutdown_connections 2024-04-23T10:40:41.081 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.081+0000 7f08f9a7f740 1 -- 172.21.15.97:0/1183904698 wait complete. 2024-04-23T10:40:41.081 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.081+0000 7f08f9a7f740 20 client.17547 trim_cache size 0 max 16384 2024-04-23T10:40:41.082 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:40:41.082 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:40:41.082 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:40:41.082 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:40:41.082 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.082+0000 7f08f6d64640 1 -- 172.21.15.97:0/334369497 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f08d80a3dc0 msgr2=0x7f08d80a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:40:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.082+0000 7f08f6d64640 1 --2- 172.21.15.97:0/334369497 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f08d80a3dc0 0x7f08d80a6280 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f08ec00d740 tx=0x7f08ec061040 comp rx=0 tx=0).stop 2024-04-23T10:40:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.082+0000 7f08f6d64640 1 -- 172.21.15.97:0/334369497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08f0101130 msgr2=0x7f08f01974b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:40:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.082+0000 7f08f6d64640 1 --2- 172.21.15.97:0/334369497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08f0101130 0x7f08f01974b0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f08ec00d770 tx=0x7f08ec00dc30 comp rx=0 tx=0).stop 2024-04-23T10:40:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.082+0000 7f08f6d64640 1 -- 172.21.15.97:0/334369497 shutdown_connections 2024-04-23T10:40:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.082+0000 7f08f6d64640 1 --2- 172.21.15.97:0/334369497 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f08d80a3dc0 0x7f08d80a6280 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.082+0000 7f08f6d64640 1 --2- 172.21.15.97:0/334369497 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08f0106f90 0x7f08f019bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.082+0000 7f08f6d64640 1 --2- 172.21.15.97:0/334369497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f08f0101af0 0x7f08f01979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.082+0000 7f08f6d64640 1 --2- 172.21.15.97:0/334369497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08f0101130 0x7f08f01974b0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.082+0000 7f08f6d64640 1 -- 172.21.15.97:0/334369497 >> 172.21.15.97:0/334369497 conn(0x7f08f0077c30 msgr2=0x7f08f00ffd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:40:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.082+0000 7f08f6d64640 1 -- 172.21.15.97:0/334369497 shutdown_connections 2024-04-23T10:40:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:41.082+0000 7f08f6d64640 1 -- 172.21.15.97:0/334369497 wait complete. 2024-04-23T10:40:41.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:40:40 smithi097 ceph-mon[29197]: pgmap v453: 97 pgs: 97 active+clean; 5.6 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 39 KiB/s wr, 16 op/s 2024-04-23T10:40:41.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:40:40 smithi183 ceph-mon[32317]: pgmap v453: 97 pgs: 97 active+clean; 5.6 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 39 KiB/s wr, 16 op/s 2024-04-23T10:40:41.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:40:40 smithi196 ceph-mon[32919]: pgmap v453: 97 pgs: 97 active+clean; 5.6 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 39 KiB/s wr, 16 op/s 2024-04-23T10:40:43.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:40:42 smithi097 ceph-mon[29197]: pgmap v454: 97 pgs: 97 active+clean; 5.6 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 39 KiB/s wr, 16 op/s 2024-04-23T10:40:43.149 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:40:42 smithi183 ceph-mon[32317]: pgmap v454: 97 pgs: 97 active+clean; 5.6 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 39 KiB/s wr, 16 op/s 2024-04-23T10:40:43.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:40:42 smithi196 ceph-mon[32919]: pgmap v454: 97 pgs: 97 active+clean; 5.6 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 39 KiB/s wr, 16 op/s 2024-04-23T10:40:45.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:40:44 smithi097 ceph-mon[29197]: pgmap v455: 97 pgs: 97 active+clean; 1.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 63 KiB/s wr, 25 op/s 2024-04-23T10:40:45.149 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:40:44 smithi183 ceph-mon[32317]: pgmap v455: 97 pgs: 97 active+clean; 1.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 63 KiB/s wr, 25 op/s 2024-04-23T10:40:45.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:40:44 smithi196 ceph-mon[32919]: pgmap v455: 97 pgs: 97 active+clean; 1.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 63 KiB/s wr, 25 op/s 2024-04-23T10:40:47.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:40:46 smithi097 ceph-mon[29197]: pgmap v456: 97 pgs: 97 active+clean; 1.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-23T10:40:47.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:40:46 smithi183 ceph-mon[32317]: pgmap v456: 97 pgs: 97 active+clean; 1.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-23T10:40:47.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:40:46 smithi196 ceph-mon[32919]: pgmap v456: 97 pgs: 97 active+clean; 1.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-23T10:40:49.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:40:48 smithi097 ceph-mon[29197]: pgmap v457: 97 pgs: 97 active+clean; 2.0 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 KiB/s wr, 22 op/s 2024-04-23T10:40:49.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:40:48 smithi183 ceph-mon[32317]: pgmap v457: 97 pgs: 97 active+clean; 2.0 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 KiB/s wr, 22 op/s 2024-04-23T10:40:49.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:40:48 smithi196 ceph-mon[32919]: pgmap v457: 97 pgs: 97 active+clean; 2.0 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 KiB/s wr, 22 op/s 2024-04-23T10:40:49.831 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 10 2024-04-23T10:40:49.832 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T10:40:49.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:49.944+0000 7fbd6d0fd640 1 -- 172.21.15.97:0/4095439926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd68101a50 msgr2=0x7fbd68105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:40:49.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:49.944+0000 7fbd6d0fd640 1 --2- 172.21.15.97:0/4095439926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd68101a50 0x7fbd68105a90 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c009ca0 tx=0x7fbd5c02a8c0 comp rx=0 tx=0).stop 2024-04-23T10:40:49.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:49.944+0000 7fbd6d0fd640 1 -- 172.21.15.97:0/4095439926 shutdown_connections 2024-04-23T10:40:49.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:49.944+0000 7fbd6d0fd640 1 --2- 172.21.15.97:0/4095439926 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd68106170 0x7fbd6810dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:49.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:49.944+0000 7fbd6d0fd640 1 --2- 172.21.15.97:0/4095439926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd68101a50 0x7fbd68105a90 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:49.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:49.944+0000 7fbd6d0fd640 1 --2- 172.21.15.97:0/4095439926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd681010a0 0x7fbd68101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:49.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:49.944+0000 7fbd6d0fd640 1 -- 172.21.15.97:0/4095439926 >> 172.21.15.97:0/4095439926 conn(0x7fbd68077f50 msgr2=0x7fbd68078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:40:49.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:49.944+0000 7fbd6d0fd640 1 -- 172.21.15.97:0/4095439926 shutdown_connections 2024-04-23T10:40:49.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:49.945+0000 7fbd6d0fd640 1 -- 172.21.15.97:0/4095439926 wait complete. 2024-04-23T10:40:49.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:49.945+0000 7fbd6d0fd640 1 Processor -- start 2024-04-23T10:40:49.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:49.945+0000 7fbd6d0fd640 1 -- start start 2024-04-23T10:40:49.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:49.946+0000 7fbd6d0fd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd681010a0 0x7fbd68198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:49.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:49.946+0000 7fbd6d0fd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd68106170 0x7fbd68198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:49.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:49.946+0000 7fbd6d0fd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd6819cff0 0x7fbd6819d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:49.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:49.946+0000 7fbd6d0fd640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbd681121f0 con 0x7fbd681010a0 2024-04-23T10:40:49.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:49.946+0000 7fbd6d0fd640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbd68112070 con 0x7fbd6819cff0 2024-04-23T10:40:49.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:49.946+0000 7fbd6d0fd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fbd68112370 con 0x7fbd68106170 2024-04-23T10:40:49.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:49.946+0000 7fbd66d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd681010a0 0x7fbd68198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:49.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:49.946+0000 7fbd66d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd681010a0 0x7fbd68198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39116/0 (socket says 172.21.15.97:39116) 2024-04-23T10:40:49.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:49.946+0000 7fbd66d9d640 1 -- 172.21.15.97:0/3706444863 learned_addr learned my addr 172.21.15.97:0/3706444863 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:40:49.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:49.946+0000 7fbd6759e640 1 --2- 172.21.15.97:0/3706444863 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd6819cff0 0x7fbd6819d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:49.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:49.946+0000 7fbd6659c640 1 --2- 172.21.15.97:0/3706444863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd68106170 0x7fbd68198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:49.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:49.946+0000 7fbd66d9d640 1 -- 172.21.15.97:0/3706444863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd68106170 msgr2=0x7fbd68198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:40:49.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:49.946+0000 7fbd66d9d640 1 --2- 172.21.15.97:0/3706444863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd68106170 0x7fbd68198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:49.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:49.946+0000 7fbd66d9d640 1 -- 172.21.15.97:0/3706444863 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd6819cff0 msgr2=0x7fbd6819d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:40:49.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:49.946+0000 7fbd66d9d640 1 --2- 172.21.15.97:0/3706444863 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd6819cff0 0x7fbd6819d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:49.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:49.946+0000 7fbd66d9d640 1 -- 172.21.15.97:0/3706444863 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd6819dae0 con 0x7fbd681010a0 2024-04-23T10:40:49.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:49.946+0000 7fbd66d9d640 1 --2- 172.21.15.97:0/3706444863 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd681010a0 0x7fbd68198730 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fbd6000d950 tx=0x7fbd6000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:40:49.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:49.947+0000 7fbd4ffff640 1 -- 172.21.15.97:0/3706444863 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd60017070 con 0x7fbd681010a0 2024-04-23T10:40:49.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:49.947+0000 7fbd6d0fd640 1 -- 172.21.15.97:0/3706444863 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd6819ddd0 con 0x7fbd681010a0 2024-04-23T10:40:49.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:49.947+0000 7fbd4ffff640 1 -- 172.21.15.97:0/3706444863 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd60009c30 con 0x7fbd681010a0 2024-04-23T10:40:49.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:49.947+0000 7fbd4ffff640 1 -- 172.21.15.97:0/3706444863 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6000f620 con 0x7fbd681010a0 2024-04-23T10:40:49.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:49.947+0000 7fbd6d0fd640 1 -- 172.21.15.97:0/3706444863 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd681a55e0 con 0x7fbd681010a0 2024-04-23T10:40:49.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:49.948+0000 7fbd6d0fd640 1 -- 172.21.15.97:0/3706444863 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd68068d70 con 0x7fbd681010a0 2024-04-23T10:40:49.949 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:49.949+0000 7fbd4ffff640 1 -- 172.21.15.97:0/3706444863 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbd60010460 con 0x7fbd681010a0 2024-04-23T10:40:49.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:49.949+0000 7fbd4ffff640 1 --2- 172.21.15.97:0/3706444863 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fbd500a3d20 0x7fbd500a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:49.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:49.949+0000 7fbd6659c640 1 --2- 172.21.15.97:0/3706444863 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fbd500a3d20 0x7fbd500a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:49.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:49.950+0000 7fbd4ffff640 1 -- 172.21.15.97:0/3706444863 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8484+0+0 (secure 0 0 0) 0x7fbd600d0350 con 0x7fbd681010a0 2024-04-23T10:40:49.950 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:49.950+0000 7fbd6659c640 1 --2- 172.21.15.97:0/3706444863 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fbd500a3d20 0x7fbd500a61e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c005e30 tx=0x7fbd5c009360 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:40:49.952 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:49.953+0000 7fbd4ffff640 1 -- 172.21.15.97:0/3706444863 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbd60009de0 con 0x7fbd681010a0 2024-04-23T10:40:50.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.168+0000 7fbd6d0fd640 1 -- 172.21.15.97:0/3706444863 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbd681093d0 con 0x7fbd681010a0 2024-04-23T10:40:50.167 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.168+0000 7fbd4ffff640 1 -- 172.21.15.97:0/3706444863 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11931 (secure 0 0 0) 0x7fbd6009c3c0 con 0x7fbd681010a0 2024-04-23T10:40:50.167 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:40:50.167 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3946878762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3946878762},{"type":"v1","addr":"172.21.15.97:6835","nonce":3946878762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/209800861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":209800861},{"type":"v1","addr":"172.21.15.97:6837","nonce":209800861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T10:24:06.910134+0000","modified":"2024-04-23T10:26:09.820985+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6839/418013575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":418013575},{"type":"v1","addr":"172.21.15.97:6839","nonce":418013575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6841/747242680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":747242680},{"type":"v1","addr":"172.21.15.97:6841","nonce":747242680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6833/4292847121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4292847121},{"type":"v1","addr":"172.21.15.183:6833","nonce":4292847121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6833/3885816565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3885816565},{"type":"v1","addr":"172.21.15.196:6833","nonce":3885816565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1897093105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1897093105},{"type":"v1","addr":"172.21.15.183:6835","nonce":1897093105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1171477563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1171477563},{"type":"v1","addr":"172.21.15.196:6835","nonce":1171477563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1898341911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1898341911},{"type":"v1","addr":"172.21.15.183:6837","nonce":1898341911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.196:6837/3094946502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3094946502},{"type":"v1","addr":"172.21.15.196:6837","nonce":3094946502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.196:6839/83843556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":83843556},{"type":"v1","addr":"172.21.15.196:6839","nonce":83843556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6839/1763841130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1763841130},{"type":"v1","addr":"172.21.15.183:6839","nonce":1763841130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24487,24403,24398,24469,24488]},"id":1}]} 2024-04-23T10:40:50.168 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 26 2024-04-23T10:40:50.169 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.170+0000 7fbd6d0fd640 1 -- 172.21.15.97:0/3706444863 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fbd500a3d20 msgr2=0x7fbd500a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:40:50.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.170+0000 7fbd6d0fd640 1 --2- 172.21.15.97:0/3706444863 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fbd500a3d20 0x7fbd500a61e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c005e30 tx=0x7fbd5c009360 comp rx=0 tx=0).stop 2024-04-23T10:40:50.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.170+0000 7fbd6d0fd640 1 -- 172.21.15.97:0/3706444863 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd681010a0 msgr2=0x7fbd68198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:40:50.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.170+0000 7fbd6d0fd640 1 --2- 172.21.15.97:0/3706444863 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd681010a0 0x7fbd68198730 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fbd6000d950 tx=0x7fbd6000de10 comp rx=0 tx=0).stop 2024-04-23T10:40:50.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.170+0000 7fbd6d0fd640 1 -- 172.21.15.97:0/3706444863 shutdown_connections 2024-04-23T10:40:50.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.170+0000 7fbd6d0fd640 1 --2- 172.21.15.97:0/3706444863 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fbd500a3d20 0x7fbd500a61e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:50.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.170+0000 7fbd6d0fd640 1 --2- 172.21.15.97:0/3706444863 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbd6819cff0 0x7fbd6819d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:50.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.170+0000 7fbd6d0fd640 1 --2- 172.21.15.97:0/3706444863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd68106170 0x7fbd68198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:50.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.170+0000 7fbd6d0fd640 1 --2- 172.21.15.97:0/3706444863 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd681010a0 0x7fbd68198730 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:50.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.171+0000 7fbd6d0fd640 1 -- 172.21.15.97:0/3706444863 >> 172.21.15.97:0/3706444863 conn(0x7fbd68077f50 msgr2=0x7fbd680ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:40:50.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.171+0000 7fbd6d0fd640 1 -- 172.21.15.97:0/3706444863 shutdown_connections 2024-04-23T10:40:50.170 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.171+0000 7fbd6d0fd640 1 -- 172.21.15.97:0/3706444863 wait complete. 2024-04-23T10:40:50.180 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db / --timeout 60 --expiration 70 --await-for 42.0' 2024-04-23T10:40:50.180 DEBUG:teuthology.orchestra.run.smithi097:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24488 quiesce db / --timeout 60 --expiration 70 --await-for 42.0 2024-04-23T10:40:50.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.289+0000 7fef81e72640 1 -- 172.21.15.97:0/1491252665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef7c101aa0 msgr2=0x7fef7c105a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:40:50.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.289+0000 7fef81e72640 1 --2- 172.21.15.97:0/1491252665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef7c101aa0 0x7fef7c105a60 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fef70009900 tx=0x7fef7002a410 comp rx=0 tx=0).stop 2024-04-23T10:40:50.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.289+0000 7fef81e72640 1 -- 172.21.15.97:0/1491252665 shutdown_connections 2024-04-23T10:40:50.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.289+0000 7fef81e72640 1 --2- 172.21.15.97:0/1491252665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fef7c106140 0x7fef7c10dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:50.290 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.289+0000 7fef81e72640 1 --2- 172.21.15.97:0/1491252665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef7c101aa0 0x7fef7c105a60 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:50.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.289+0000 7fef81e72640 1 --2- 172.21.15.97:0/1491252665 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef7c1010f0 0x7fef7c1014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:50.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.289+0000 7fef81e72640 1 -- 172.21.15.97:0/1491252665 >> 172.21.15.97:0/1491252665 conn(0x7fef7c076f60 msgr2=0x7fef7c077370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:40:50.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.289+0000 7fef81e72640 1 -- 172.21.15.97:0/1491252665 shutdown_connections 2024-04-23T10:40:50.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.289+0000 7fef81e72640 1 -- 172.21.15.97:0/1491252665 wait complete. 2024-04-23T10:40:50.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.290+0000 7fef81e72640 1 Processor -- start 2024-04-23T10:40:50.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.290+0000 7fef81e72640 1 -- start start 2024-04-23T10:40:50.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.290+0000 7fef81e72640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fef7c1010f0 0x7fef7c1a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:50.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.290+0000 7fef81e72640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef7c106140 0x7fef7c1a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:50.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.290+0000 7fef81e72640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef7c19a790 0x7fef7c19ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:50.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.290+0000 7fef81e72640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c111080 con 0x7fef7c106140 2024-04-23T10:40:50.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.290+0000 7fef81e72640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c110f00 con 0x7fef7c1010f0 2024-04-23T10:40:50.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.290+0000 7fef81e72640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c111200 con 0x7fef7c19a790 2024-04-23T10:40:50.291 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.290+0000 7fef7affd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef7c106140 0x7fef7c1a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:50.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.290+0000 7fef7affd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef7c106140 0x7fef7c1a0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39144/0 (socket says 172.21.15.97:39144) 2024-04-23T10:40:50.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.290+0000 7fef7affd640 1 -- 172.21.15.97:0/618287785 learned_addr learned my addr 172.21.15.97:0/618287785 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:40:50.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.290+0000 7fef7bfff640 1 --2- 172.21.15.97:0/618287785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef7c19a790 0x7fef7c19ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:50.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.290+0000 7fef7b7fe640 1 --2- 172.21.15.97:0/618287785 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fef7c1010f0 0x7fef7c1a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:50.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.291+0000 7fef7affd640 1 -- 172.21.15.97:0/618287785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef7c19a790 msgr2=0x7fef7c19ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:40:50.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.291+0000 7fef7affd640 1 --2- 172.21.15.97:0/618287785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef7c19a790 0x7fef7c19ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:50.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.291+0000 7fef7affd640 1 -- 172.21.15.97:0/618287785 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fef7c1010f0 msgr2=0x7fef7c1a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:40:50.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.291+0000 7fef7affd640 1 --2- 172.21.15.97:0/618287785 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fef7c1010f0 0x7fef7c1a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:50.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.291+0000 7fef7affd640 1 -- 172.21.15.97:0/618287785 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef7c19b320 con 0x7fef7c106140 2024-04-23T10:40:50.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.291+0000 7fef7b7fe640 1 --2- 172.21.15.97:0/618287785 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fef7c1010f0 0x7fef7c1a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:40:50.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.291+0000 7fef7affd640 1 --2- 172.21.15.97:0/618287785 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef7c106140 0x7fef7c1a0b60 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fef70032fd0 tx=0x7fef70004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:40:50.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.291+0000 7fef78ff9640 1 -- 172.21.15.97:0/618287785 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7003f030 con 0x7fef7c106140 2024-04-23T10:40:50.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.291+0000 7fef78ff9640 1 -- 172.21.15.97:0/618287785 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef70004490 con 0x7fef7c106140 2024-04-23T10:40:50.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.291+0000 7fef81e72640 1 -- 172.21.15.97:0/618287785 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef7c19b5b0 con 0x7fef7c106140 2024-04-23T10:40:50.292 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.291+0000 7fef78ff9640 1 -- 172.21.15.97:0/618287785 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef70035070 con 0x7fef7c106140 2024-04-23T10:40:50.293 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.291+0000 7fef81e72640 1 -- 172.21.15.97:0/618287785 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef7c1a7760 con 0x7fef7c106140 2024-04-23T10:40:50.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.293+0000 7fef78ff9640 1 -- 172.21.15.97:0/618287785 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fef70042030 con 0x7fef7c106140 2024-04-23T10:40:50.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.293+0000 7fef84b88740 1 Processor -- start 2024-04-23T10:40:50.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.294+0000 7fef84b88740 1 -- start start 2024-04-23T10:40:50.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.294+0000 7fef78ff9640 1 --2- 172.21.15.97:0/618287785 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fef640a3d20 0x7fef640a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:50.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.294+0000 7fef84b88740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5584ecaff0a0 0x5584ecaff480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:50.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.294+0000 7fef84b88740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5584ecaff9c0 0x5584ecb09b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:50.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.294+0000 7fef84b88740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584ecb0a090 0x5584ecb0c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:50.299 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.294+0000 7fef84b88740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5584eca64370 con 0x5584ecaff0a0 2024-04-23T10:40:50.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.294+0000 7fef84b88740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5584eca012e0 con 0x5584ecb0a090 2024-04-23T10:40:50.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.294+0000 7fef84b88740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5584eca60330 con 0x5584ecaff9c0 2024-04-23T10:40:50.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.294+0000 7fef78ff9640 1 -- 172.21.15.97:0/618287785 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8484+0+0 (secure 0 0 0) 0x7fef700f0c90 con 0x7fef7c106140 2024-04-23T10:40:50.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.294+0000 7fef7bfff640 1 --2- 172.21.15.97:0/618287785 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fef640a3d20 0x7fef640a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:50.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.294+0000 7fef7b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5584ecaff0a0 0x5584ecaff480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:50.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.294+0000 7fef7b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5584ecaff0a0 0x5584ecaff480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39146/0 (socket says 172.21.15.97:39146) 2024-04-23T10:40:50.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.294+0000 7fef7b7fe640 1 -- 172.21.15.97:0/4070461010 learned_addr learned my addr 172.21.15.97:0/4070461010 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:40:50.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.294+0000 7fef7bfff640 1 --2- 172.21.15.97:0/4070461010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584ecb0a090 0x5584ecb0c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:50.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.294+0000 7fef7affd640 1 --2- 172.21.15.97:0/4070461010 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5584ecaff9c0 0x5584ecb09b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:50.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.294+0000 7fef7b7fe640 1 -- 172.21.15.97:0/4070461010 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5584ecaff9c0 msgr2=0x5584ecb09b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:40:50.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.294+0000 7fef7b7fe640 1 --2- 172.21.15.97:0/4070461010 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5584ecaff9c0 0x5584ecb09b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:50.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.294+0000 7fef7b7fe640 1 -- 172.21.15.97:0/4070461010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584ecb0a090 msgr2=0x5584ecb0c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:40:50.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.294+0000 7fef7b7fe640 1 --2- 172.21.15.97:0/4070461010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584ecb0a090 0x5584ecb0c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:50.300 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.294+0000 7fef7b7fe640 1 -- 172.21.15.97:0/4070461010 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584ec842d80 con 0x5584ecaff0a0 2024-04-23T10:40:50.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.294+0000 7fef7bfff640 1 --2- 172.21.15.97:0/4070461010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584ecb0a090 0x5584ecb0c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:40:50.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.294+0000 7fef7bfff640 1 --2- 172.21.15.97:0/618287785 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fef640a3d20 0x7fef640a61e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fef74005fd0 tx=0x7fef74012ab0 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:40:50.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.294+0000 7fef7b7fe640 1 --2- 172.21.15.97:0/4070461010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5584ecaff0a0 0x5584ecaff480 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fef6c00bd40 tx=0x7fef6c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f3fadcf3e59de99 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:40:50.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.295+0000 7fef61ffb640 1 -- 172.21.15.97:0/4070461010 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef6c015070 con 0x5584ecaff0a0 2024-04-23T10:40:50.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.295+0000 7fef61ffb640 1 -- 172.21.15.97:0/4070461010 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef6c00be30 con 0x5584ecaff0a0 2024-04-23T10:40:50.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.295+0000 7fef61ffb640 1 -- 172.21.15.97:0/4070461010 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef6c011980 con 0x5584ecaff0a0 2024-04-23T10:40:50.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.295+0000 7fef84b88740 1 -- 172.21.15.97:0/4070461010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5584ecaff0a0 msgr2=0x5584ecaff480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:40:50.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.295+0000 7fef84b88740 1 --2- 172.21.15.97:0/4070461010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5584ecaff0a0 0x5584ecaff480 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fef6c00bd40 tx=0x7fef6c00f730 comp rx=0 tx=0).stop 2024-04-23T10:40:50.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.295+0000 7fef84b88740 1 -- 172.21.15.97:0/4070461010 shutdown_connections 2024-04-23T10:40:50.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.295+0000 7fef84b88740 1 --2- 172.21.15.97:0/4070461010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584ecb0a090 0x5584ecb0c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:50.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.295+0000 7fef84b88740 1 --2- 172.21.15.97:0/4070461010 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5584ecaff9c0 0x5584ecb09b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:50.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.295+0000 7fef84b88740 1 --2- 172.21.15.97:0/4070461010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5584ecaff0a0 0x5584ecaff480 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:50.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.295+0000 7fef84b88740 1 -- 172.21.15.97:0/4070461010 >> 172.21.15.97:0/4070461010 conn(0x5584ecafb3c0 msgr2=0x5584ecafb7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:40:50.301 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.295+0000 7fef84b88740 1 -- 172.21.15.97:0/4070461010 shutdown_connections 2024-04-23T10:40:50.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.295+0000 7fef84b88740 1 -- 172.21.15.97:0/4070461010 wait complete. 2024-04-23T10:40:50.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.296+0000 7fef84b88740 1 -- start start 2024-04-23T10:40:50.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.296+0000 7fef84b88740 1 Processor -- start 2024-04-23T10:40:50.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.296+0000 7fef84b88740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5584ecaff0a0 0x5584ecb06fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:50.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.296+0000 7fef84b88740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5584ecaff9c0 0x5584ecb07520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:50.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.296+0000 7fef84b88740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584ecb0a090 0x5584ecb07a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:50.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.296+0000 7fef84b88740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5584ecb0ce80 con 0x5584ecaff0a0 2024-04-23T10:40:50.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.296+0000 7fef84b88740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5584ec9fefb0 con 0x5584ecb0a090 2024-04-23T10:40:50.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.296+0000 7fef84b88740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5584ecb0d000 con 0x5584ecaff9c0 2024-04-23T10:40:50.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.296+0000 7fef84b88740 10 client.0 _ll_register_callbacks cb 0x5584ec9fe9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:40:50.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.296+0000 7fef7b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5584ecaff0a0 0x5584ecb06fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:50.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.296+0000 7fef7b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5584ecaff0a0 0x5584ecb06fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39150/0 (socket says 172.21.15.97:39150) 2024-04-23T10:40:50.302 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.296+0000 7fef7b7fe640 1 -- 172.21.15.97:0/1977723095 learned_addr learned my addr 172.21.15.97:0/1977723095 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:40:50.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.296+0000 7fef7affd640 1 --2- 172.21.15.97:0/1977723095 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5584ecaff9c0 0x5584ecb07520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:50.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.296+0000 7fef7b7fe640 1 -- 172.21.15.97:0/1977723095 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5584ecaff9c0 msgr2=0x5584ecb07520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:40:50.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.296+0000 7fef7b7fe640 1 --2- 172.21.15.97:0/1977723095 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5584ecaff9c0 0x5584ecb07520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:50.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.296+0000 7fef7b7fe640 1 -- 172.21.15.97:0/1977723095 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584ecb0a090 msgr2=0x5584ecb07a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:40:50.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.296+0000 7fef7b7fe640 1 --2- 172.21.15.97:0/1977723095 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584ecb0a090 0x5584ecb07a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:50.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.296+0000 7fef7b7fe640 1 -- 172.21.15.97:0/1977723095 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584ec8a3510 con 0x5584ecaff0a0 2024-04-23T10:40:50.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.296+0000 7fef7b7fe640 1 --2- 172.21.15.97:0/1977723095 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5584ecaff0a0 0x5584ecb06fe0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fef6c011480 tx=0x7fef6c005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a484653b4e1bd4e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:40:50.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.297+0000 7fef5affd640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:40:50.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.297+0000 7fef5affd640 1 -- 172.21.15.97:0/1977723095 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef6c028060 con 0x5584ecaff0a0 2024-04-23T10:40:50.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.297+0000 7fef5affd640 1 -- 172.21.15.97:0/1977723095 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8484+0+0 (secure 0 0 0) 0x7fef6c005480 con 0x5584ecaff0a0 2024-04-23T10:40:50.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.297+0000 7fef84b88740 1 -- 172.21.15.97:0/1977723095 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef6c00be30 con 0x5584ecaff0a0 2024-04-23T10:40:50.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.297+0000 7fef5affd640 1 -- 172.21.15.97:0/1977723095 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fef6c015070 con 0x5584ecaff0a0 2024-04-23T10:40:50.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.297+0000 7fef84b88740 10 client.17565 fetch_fsmap learned FSMap version 26 2024-04-23T10:40:50.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.297+0000 7fef84b88740 1 -- 172.21.15.97:0/1977723095 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5584ec736670 con 0x5584ecaff0a0 2024-04-23T10:40:50.303 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.297+0000 7fef5affd640 1 -- 172.21.15.97:0/1977723095 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef6c00be30 con 0x5584ecaff0a0 2024-04-23T10:40:50.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.297+0000 7fef5affd640 1 -- 172.21.15.97:0/1977723095 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fef6c004990 con 0x5584ecaff0a0 2024-04-23T10:40:50.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.297+0000 7fef84b88740 10 client.17565 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:40:50.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.297+0000 7fef84b88740 10 client.17565 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:40:50.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.297+0000 7fef84b88740 20 client.17565 populate_metadata read hostname 'smithi097' 2024-04-23T10:40:50.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.297+0000 7fef84b88740 1 --2- 172.21.15.97:0/1977723095 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5584ecb1a360 0x5584ecb1c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:50.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.297+0000 7fef84b88740 4 client.17565 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:40:50.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.297+0000 7fef84b88740 1 -- 172.21.15.97:0/1977723095 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5584eca71d90 con 0x5584ecb1a360 2024-04-23T10:40:50.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.298+0000 7fef7affd640 1 --2- 172.21.15.97:0/1977723095 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5584ecb1a360 0x5584ecb1c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:50.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.298+0000 7fef7affd640 1 --2- 172.21.15.97:0/1977723095 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5584ecb1a360 0x5584ecb1c750 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2adf01f38276da29 server_cookie=90beae3cea49f4a6 in_seq=0 out_seq=0 2024-04-23T10:40:50.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.298+0000 7fef5affd640 10 client.17565 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:40:50.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.299+0000 7fef5affd640 1 -- 172.21.15.97:0/1977723095 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5584eca71d90 con 0x5584ecb1a360 2024-04-23T10:40:50.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.299+0000 7fef5affd640 10 client.17565 handle_command_reply: tid=0 2024-04-23T10:40:50.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.299+0000 7fef84b88740 1 client.17565 shutdown 2024-04-23T10:40:50.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.299+0000 7fef84b88740 1 -- 172.21.15.97:0/1977723095 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5584ecaff0a0 msgr2=0x5584ecb06fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:40:50.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.299+0000 7fef84b88740 1 --2- 172.21.15.97:0/1977723095 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5584ecaff0a0 0x5584ecb06fe0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fef6c011480 tx=0x7fef6c005040 comp rx=0 tx=0).stop 2024-04-23T10:40:50.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.299+0000 7fef84b88740 1 -- 172.21.15.97:0/1977723095 shutdown_connections 2024-04-23T10:40:50.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.299+0000 7fef84b88740 1 --2- 172.21.15.97:0/1977723095 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5584ecb1a360 0x5584ecb1c750 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:50.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.299+0000 7fef84b88740 1 --2- 172.21.15.97:0/1977723095 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584ecb0a090 0x5584ecb07a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:50.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.299+0000 7fef84b88740 1 --2- 172.21.15.97:0/1977723095 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5584ecaff9c0 0x5584ecb07520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:50.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.299+0000 7fef5affd640 0 client.17565 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:40:50.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.299+0000 7fef84b88740 1 --2- 172.21.15.97:0/1977723095 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5584ecaff0a0 0x5584ecb06fe0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:50.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.299+0000 7fef84b88740 1 -- 172.21.15.97:0/1977723095 >> 172.21.15.97:0/1977723095 conn(0x5584ecafb3c0 msgr2=0x5584ecb011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:40:50.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.299+0000 7fef84b88740 1 -- 172.21.15.97:0/1977723095 shutdown_connections 2024-04-23T10:40:50.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.299+0000 7fef84b88740 1 -- 172.21.15.97:0/1977723095 wait complete. 2024-04-23T10:40:50.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.299+0000 7fef84b88740 20 client.17565 trim_cache size 0 max 16384 2024-04-23T10:40:50.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.316+0000 7fef84b88740 1 Processor -- start 2024-04-23T10:40:50.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.316+0000 7fef84b88740 1 -- start start 2024-04-23T10:40:50.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.316+0000 7fef84b88740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584ecaff0a0 0x5584ecb01340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:50.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.316+0000 7fef84b88740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5584ecaff9c0 0x5584ecb076b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:50.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.316+0000 7fef84b88740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5584ecb0a090 0x5584ecb21d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:50.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.316+0000 7fef84b88740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c1afd20 con 0x5584ecb0a090 2024-04-23T10:40:50.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.316+0000 7fef84b88740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c1b1860 con 0x5584ecaff0a0 2024-04-23T10:40:50.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.316+0000 7fef84b88740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c1b03f0 con 0x5584ecaff9c0 2024-04-23T10:40:50.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.316+0000 7fef7bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5584ecb0a090 0x5584ecb21d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:50.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.316+0000 7fef7bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5584ecb0a090 0x5584ecb21d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39156/0 (socket says 172.21.15.97:39156) 2024-04-23T10:40:50.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.316+0000 7fef7bfff640 1 -- 172.21.15.97:0/3981391926 learned_addr learned my addr 172.21.15.97:0/3981391926 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:40:50.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.316+0000 7fef7b7fe640 1 --2- 172.21.15.97:0/3981391926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584ecaff0a0 0x5584ecb01340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:50.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.316+0000 7fef7bfff640 1 -- 172.21.15.97:0/3981391926 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5584ecaff9c0 msgr2=0x5584ecb076b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:40:50.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.316+0000 7fef7bfff640 1 --2- 172.21.15.97:0/3981391926 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5584ecaff9c0 0x5584ecb076b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:50.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.316+0000 7fef7bfff640 1 -- 172.21.15.97:0/3981391926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584ecaff0a0 msgr2=0x5584ecb01340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:40:50.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.316+0000 7fef7bfff640 1 --2- 172.21.15.97:0/3981391926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584ecaff0a0 0x5584ecb01340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:50.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.316+0000 7fef7bfff640 1 -- 172.21.15.97:0/3981391926 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584ecafc580 con 0x5584ecb0a090 2024-04-23T10:40:50.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.317+0000 7fef7bfff640 1 --2- 172.21.15.97:0/3981391926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5584ecb0a090 0x5584ecb21d80 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fef74003c00 tx=0x7fef7401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f814bce052a94b00 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:40:50.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.317+0000 7fef617fa640 1 -- 172.21.15.97:0/3981391926 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef74025070 con 0x5584ecb0a090 2024-04-23T10:40:50.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.317+0000 7fef617fa640 1 -- 172.21.15.97:0/3981391926 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef740044f0 con 0x5584ecb0a090 2024-04-23T10:40:50.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.317+0000 7fef617fa640 1 -- 172.21.15.97:0/3981391926 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7401fb00 con 0x5584ecb0a090 2024-04-23T10:40:50.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.317+0000 7fef84b88740 1 -- 172.21.15.97:0/3981391926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5584ecb0a090 msgr2=0x5584ecb21d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:40:50.320 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.317+0000 7fef84b88740 1 --2- 172.21.15.97:0/3981391926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5584ecb0a090 0x5584ecb21d80 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fef74003c00 tx=0x7fef7401a000 comp rx=0 tx=0).stop 2024-04-23T10:40:50.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.317+0000 7fef84b88740 1 -- 172.21.15.97:0/3981391926 shutdown_connections 2024-04-23T10:40:50.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.317+0000 7fef84b88740 1 --2- 172.21.15.97:0/3981391926 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5584ecb0a090 0x5584ecb21d80 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:50.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.317+0000 7fef84b88740 1 --2- 172.21.15.97:0/3981391926 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5584ecaff9c0 0x5584ecb076b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:50.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.317+0000 7fef84b88740 1 --2- 172.21.15.97:0/3981391926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584ecaff0a0 0x5584ecb01340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:50.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.317+0000 7fef84b88740 1 -- 172.21.15.97:0/3981391926 >> 172.21.15.97:0/3981391926 conn(0x5584ecafb3c0 msgr2=0x5584ecb06fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:40:50.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.317+0000 7fef84b88740 1 -- 172.21.15.97:0/3981391926 shutdown_connections 2024-04-23T10:40:50.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.317+0000 7fef84b88740 1 -- 172.21.15.97:0/3981391926 wait complete. 2024-04-23T10:40:50.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.318+0000 7fef84b88740 1 -- start start 2024-04-23T10:40:50.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.318+0000 7fef84b88740 1 Processor -- start 2024-04-23T10:40:50.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.318+0000 7fef84b88740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5584ecaff0a0 0x5584ecb12250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:50.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.318+0000 7fef84b88740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584ecaff9c0 0x5584ecb12790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:50.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.318+0000 7fef84b88740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5584ecb0a090 0x5584ecb0f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:50.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.318+0000 7fef84b88740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c1b1190 con 0x5584ecb0a090 2024-04-23T10:40:50.321 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.318+0000 7fef84b88740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c1b0ac0 con 0x5584ecaff9c0 2024-04-23T10:40:50.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.318+0000 7fef84b88740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c1add20 con 0x5584ecaff0a0 2024-04-23T10:40:50.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.318+0000 7fef84b88740 10 client.0 _ll_register_callbacks cb 0x7fef6c00ff90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:40:50.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.318+0000 7fef7bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5584ecb0a090 0x5584ecb0f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:50.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.318+0000 7fef7bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5584ecb0a090 0x5584ecb0f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39160/0 (socket says 172.21.15.97:39160) 2024-04-23T10:40:50.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.318+0000 7fef7bfff640 1 -- 172.21.15.97:0/3165134643 learned_addr learned my addr 172.21.15.97:0/3165134643 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:40:50.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.318+0000 7fef7affd640 1 --2- 172.21.15.97:0/3165134643 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584ecaff9c0 0x5584ecb12790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:50.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.319+0000 7fef7bfff640 1 -- 172.21.15.97:0/3165134643 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5584ecaff0a0 msgr2=0x5584ecb12250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:40:50.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.319+0000 7fef7bfff640 1 --2- 172.21.15.97:0/3165134643 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5584ecaff0a0 0x5584ecb12250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:50.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.319+0000 7fef7bfff640 1 -- 172.21.15.97:0/3165134643 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584ecaff9c0 msgr2=0x5584ecb12790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:40:50.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.319+0000 7fef7bfff640 1 --2- 172.21.15.97:0/3165134643 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584ecaff9c0 0x5584ecb12790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:50.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.319+0000 7fef7bfff640 1 -- 172.21.15.97:0/3165134643 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584ecb0f9f0 con 0x5584ecb0a090 2024-04-23T10:40:50.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.319+0000 7fef7bfff640 1 --2- 172.21.15.97:0/3165134643 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5584ecb0a090 0x5584ecb0f220 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fef74003ad0 tx=0x7fef7401f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44c5420051cc6123 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:40:50.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.319+0000 7fef5affd640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:40:50.322 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.319+0000 7fef5affd640 1 -- 172.21.15.97:0/3165134643 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7402e070 con 0x5584ecb0a090 2024-04-23T10:40:50.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.319+0000 7fef5affd640 1 -- 172.21.15.97:0/3165134643 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8484+0+0 (secure 0 0 0) 0x7fef74004240 con 0x5584ecb0a090 2024-04-23T10:40:50.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.319+0000 7fef84b88740 1 -- 172.21.15.97:0/3165134643 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef740044f0 con 0x5584ecb0a090 2024-04-23T10:40:50.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.319+0000 7fef5affd640 1 -- 172.21.15.97:0/3165134643 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fef7403a050 con 0x5584ecb0a090 2024-04-23T10:40:50.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.319+0000 7fef84b88740 10 client.17571 fetch_fsmap learned FSMap version 26 2024-04-23T10:40:50.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.319+0000 7fef84b88740 1 -- 172.21.15.97:0/3165134643 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5584ecb810a0 con 0x5584ecb0a090 2024-04-23T10:40:50.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.319+0000 7fef5affd640 1 -- 172.21.15.97:0/3165134643 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef740044f0 con 0x5584ecb0a090 2024-04-23T10:40:50.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.319+0000 7fef5affd640 1 -- 172.21.15.97:0/3165134643 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fef7401e070 con 0x5584ecb0a090 2024-04-23T10:40:50.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.319+0000 7fef84b88740 10 client.17571 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:40:50.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.319+0000 7fef84b88740 10 client.17571 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:40:50.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.319+0000 7fef84b88740 20 client.17571 populate_metadata read hostname 'smithi097' 2024-04-23T10:40:50.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.319+0000 7fef84b88740 1 --2- 172.21.15.97:0/3165134643 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5584ecb81890 0x5584ecb83cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:50.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.319+0000 7fef84b88740 4 client.17571 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 70.0, "await_for": 42.0}] 2024-04-23T10:40:50.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.319+0000 7fef84b88740 1 -- 172.21.15.97:0/3165134643 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 70.0, "await_for": 42.0}) -- 0x5584eca6bc00 con 0x5584ecb81890 2024-04-23T10:40:50.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.320+0000 7fef7b7fe640 1 --2- 172.21.15.97:0/3165134643 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5584ecb81890 0x5584ecb83cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:50.323 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.320+0000 7fef5affd640 10 client.17571 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:40:50.324 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.320+0000 7fef7b7fe640 1 --2- 172.21.15.97:0/3165134643 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5584ecb81890 0x5584ecb83cb0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cdec4cb971a08b5 server_cookie=18da9d558db4965c in_seq=0 out_seq=0 2024-04-23T10:40:50.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.340+0000 7fef5affd640 1 -- 172.21.15.97:0/3165134643 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5584eca6bc00 con 0x5584ecb81890 2024-04-23T10:40:50.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.340+0000 7fef5affd640 10 client.17571 handle_command_reply: tid=0 2024-04-23T10:40:50.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.340+0000 7fef84b88740 1 client.17571 shutdown 2024-04-23T10:40:50.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.340+0000 7fef84b88740 1 -- 172.21.15.97:0/3165134643 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5584ecb0a090 msgr2=0x5584ecb0f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:40:50.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.340+0000 7fef84b88740 1 --2- 172.21.15.97:0/3165134643 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5584ecb0a090 0x5584ecb0f220 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fef74003ad0 tx=0x7fef7401f4a0 comp rx=0 tx=0).stop 2024-04-23T10:40:50.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.340+0000 7fef84b88740 1 -- 172.21.15.97:0/3165134643 shutdown_connections 2024-04-23T10:40:50.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.340+0000 7fef84b88740 1 --2- 172.21.15.97:0/3165134643 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5584ecb81890 0x5584ecb83cb0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:50.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.340+0000 7fef84b88740 1 --2- 172.21.15.97:0/3165134643 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5584ecb0a090 0x5584ecb0f220 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:50.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.340+0000 7fef84b88740 1 --2- 172.21.15.97:0/3165134643 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5584ecaff9c0 0x5584ecb12790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:50.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.340+0000 7fef84b88740 1 --2- 172.21.15.97:0/3165134643 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5584ecaff0a0 0x5584ecb12250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:50.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.340+0000 7fef5affd640 0 client.17571 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:40:50.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.340+0000 7fef84b88740 1 -- 172.21.15.97:0/3165134643 >> 172.21.15.97:0/3165134643 conn(0x5584ecafb3c0 msgr2=0x5584ecb07810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:40:50.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.340+0000 7fef84b88740 1 -- 172.21.15.97:0/3165134643 shutdown_connections 2024-04-23T10:40:50.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.340+0000 7fef84b88740 1 -- 172.21.15.97:0/3165134643 wait complete. 2024-04-23T10:40:50.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.341+0000 7fef84b88740 20 client.17571 trim_cache size 0 max 16384 2024-04-23T10:40:50.341 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-23T10:40:50.341 INFO:teuthology.orchestra.run.smithi097.stdout: "epoch": 26, 2024-04-23T10:40:50.341 INFO:teuthology.orchestra.run.smithi097.stdout: "leader": 24488, 2024-04-23T10:40:50.341 INFO:teuthology.orchestra.run.smithi097.stdout: "set_version": 58, 2024-04-23T10:40:50.341 INFO:teuthology.orchestra.run.smithi097.stdout: "sets": { 2024-04-23T10:40:50.341 INFO:teuthology.orchestra.run.smithi097.stdout: "ca3de698": { 2024-04-23T10:40:50.341 INFO:teuthology.orchestra.run.smithi097.stdout: "version": 58, 2024-04-23T10:40:50.341 INFO:teuthology.orchestra.run.smithi097.stdout: "age_ref": 0.0, 2024-04-23T10:40:50.341 INFO:teuthology.orchestra.run.smithi097.stdout: "state": { 2024-04-23T10:40:50.341 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "QUIESCED", 2024-04-23T10:40:50.341 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0 2024-04-23T10:40:50.342 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:40:50.342 INFO:teuthology.orchestra.run.smithi097.stdout: "timeout": 60.0, 2024-04-23T10:40:50.342 INFO:teuthology.orchestra.run.smithi097.stdout: "expiration": 70.0, 2024-04-23T10:40:50.342 INFO:teuthology.orchestra.run.smithi097.stdout: "members": { 2024-04-23T10:40:50.342 INFO:teuthology.orchestra.run.smithi097.stdout: "file:/": { 2024-04-23T10:40:50.342 INFO:teuthology.orchestra.run.smithi097.stdout: "excluded": false, 2024-04-23T10:40:50.342 INFO:teuthology.orchestra.run.smithi097.stdout: "state": { 2024-04-23T10:40:50.342 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "QUIESCED", 2024-04-23T10:40:50.342 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0 2024-04-23T10:40:50.342 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:40:50.342 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:40:50.342 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:40:50.342 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:40:50.342 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:40:50.342 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-23T10:40:50.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.341+0000 7fef81e72640 1 -- 172.21.15.97:0/618287785 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fef640a3d20 msgr2=0x7fef640a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:40:50.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.341+0000 7fef81e72640 1 --2- 172.21.15.97:0/618287785 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fef640a3d20 0x7fef640a61e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fef74005fd0 tx=0x7fef74012ab0 comp rx=0 tx=0).stop 2024-04-23T10:40:50.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.341+0000 7fef81e72640 1 -- 172.21.15.97:0/618287785 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef7c106140 msgr2=0x7fef7c1a0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:40:50.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.341+0000 7fef81e72640 1 --2- 172.21.15.97:0/618287785 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef7c106140 0x7fef7c1a0b60 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fef70032fd0 tx=0x7fef70004270 comp rx=0 tx=0).stop 2024-04-23T10:40:50.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.341+0000 7fef81e72640 1 -- 172.21.15.97:0/618287785 shutdown_connections 2024-04-23T10:40:50.343 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.341+0000 7fef81e72640 1 --2- 172.21.15.97:0/618287785 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fef640a3d20 0x7fef640a61e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:50.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.341+0000 7fef81e72640 1 --2- 172.21.15.97:0/618287785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef7c19a790 0x7fef7c19ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:50.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.341+0000 7fef81e72640 1 --2- 172.21.15.97:0/618287785 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef7c106140 0x7fef7c1a0b60 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:50.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.341+0000 7fef81e72640 1 --2- 172.21.15.97:0/618287785 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fef7c1010f0 0x7fef7c1a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:50.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.341+0000 7fef81e72640 1 -- 172.21.15.97:0/618287785 >> 172.21.15.97:0/618287785 conn(0x7fef7c076f60 msgr2=0x7fef7c100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:40:50.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.341+0000 7fef81e72640 1 -- 172.21.15.97:0/618287785 shutdown_connections 2024-04-23T10:40:50.344 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:40:50.341+0000 7fef81e72640 1 -- 172.21.15.97:0/618287785 wait complete. 2024-04-23T10:40:50.350 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'ca3de698', quiesce took 0.5 seconds. Will release after: 10 2024-04-23T10:40:51.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:40:50 smithi183 ceph-mon[32317]: pgmap v458: 97 pgs: 97 active+clean; 2.0 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:40:51.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:40:50 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/3706444863' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:40:51.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:40:50 smithi196 ceph-mon[32919]: pgmap v458: 97 pgs: 97 active+clean; 2.0 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:40:51.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:40:50 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/3706444863' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:40:51.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:40:50 smithi097 ceph-mon[29197]: pgmap v458: 97 pgs: 97 active+clean; 2.0 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:40:51.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:40:50 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/3706444863' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:40:53.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:40:52 smithi183 ceph-mon[32317]: pgmap v459: 97 pgs: 97 active+clean; 2.0 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:40:53.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:40:52 smithi196 ceph-mon[32919]: pgmap v459: 97 pgs: 97 active+clean; 2.0 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:40:53.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:40:52 smithi097 ceph-mon[29197]: pgmap v459: 97 pgs: 97 active+clean; 2.0 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:40:54.180 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.179+0000 7fc7fffbf640 1 -- 172.21.15.97:0/2681172551 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7f81010f0 msgr2=0x7fc7f81014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:40:54.180 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.179+0000 7fc7fffbf640 1 --2- 172.21.15.97:0/2681172551 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7f81010f0 0x7fc7f81014d0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fc7f4009ca0 tx=0x7fc7f402a8c0 comp rx=0 tx=0).stop 2024-04-23T10:40:54.180 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.179+0000 7fc7fffbf640 1 -- 172.21.15.97:0/2681172551 shutdown_connections 2024-04-23T10:40:54.180 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.179+0000 7fc7fffbf640 1 --2- 172.21.15.97:0/2681172551 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7f8106170 0x7fc7f810dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:54.181 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.179+0000 7fc7fffbf640 1 --2- 172.21.15.97:0/2681172551 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7f8101aa0 0x7fc7f8105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:54.181 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.179+0000 7fc7fffbf640 1 --2- 172.21.15.97:0/2681172551 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7f81010f0 0x7fc7f81014d0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:54.181 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.179+0000 7fc7fffbf640 1 -- 172.21.15.97:0/2681172551 >> 172.21.15.97:0/2681172551 conn(0x7fc7f8077fd0 msgr2=0x7fc7f80783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:40:54.181 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.179+0000 7fc7fffbf640 1 -- 172.21.15.97:0/2681172551 shutdown_connections 2024-04-23T10:40:54.181 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.179+0000 7fc7fffbf640 1 -- 172.21.15.97:0/2681172551 wait complete. 2024-04-23T10:40:54.181 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.179+0000 7fc7fffbf640 1 Processor -- start 2024-04-23T10:40:54.181 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.179+0000 7fc7fffbf640 1 -- start start 2024-04-23T10:40:54.181 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.179+0000 7fc7fffbf640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7f81010f0 0x7fc7f81995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:54.181 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.180+0000 7fc7fffbf640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7f8106170 0x7fc7f8199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:54.181 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.180+0000 7fc7fffbf640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7f819a080 0x7fc7f81a4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:54.181 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.180+0000 7fc7fffbf640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f81110b0 con 0x7fc7f8106170 2024-04-23T10:40:54.181 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.180+0000 7fc7fffbf640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f8110f30 con 0x7fc7f819a080 2024-04-23T10:40:54.181 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.180+0000 7fc7fffbf640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f8111230 con 0x7fc7f81010f0 2024-04-23T10:40:54.182 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.180+0000 7fc7fd55a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7f8106170 0x7fc7f8199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:54.182 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.180+0000 7fc7fd55a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7f8106170 0x7fc7f8199af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39180/0 (socket says 172.21.15.97:39180) 2024-04-23T10:40:54.182 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.180+0000 7fc7fd55a640 1 -- 172.21.15.97:0/4181527799 learned_addr learned my addr 172.21.15.97:0/4181527799 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:40:54.182 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.180+0000 7fc7fdd5b640 1 --2- 172.21.15.97:0/4181527799 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7f81010f0 0x7fc7f81995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:54.182 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.180+0000 7fc7fe55c640 1 --2- 172.21.15.97:0/4181527799 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7f819a080 0x7fc7f81a4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:54.182 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.180+0000 7fc7fd55a640 1 -- 172.21.15.97:0/4181527799 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7f81010f0 msgr2=0x7fc7f81995b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:40:54.182 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.180+0000 7fc7fd55a640 1 --2- 172.21.15.97:0/4181527799 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7f81010f0 0x7fc7f81995b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:54.182 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.180+0000 7fc7fd55a640 1 -- 172.21.15.97:0/4181527799 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7f819a080 msgr2=0x7fc7f81a4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:40:54.182 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.180+0000 7fc7fd55a640 1 --2- 172.21.15.97:0/4181527799 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7f819a080 0x7fc7f81a4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:54.182 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.180+0000 7fc7fd55a640 1 -- 172.21.15.97:0/4181527799 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7f81a4e00 con 0x7fc7f8106170 2024-04-23T10:40:54.182 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.180+0000 7fc7fe55c640 1 --2- 172.21.15.97:0/4181527799 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7f819a080 0x7fc7f81a4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:40:54.182 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.180+0000 7fc7fd55a640 1 --2- 172.21.15.97:0/4181527799 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7f8106170 0x7fc7f8199af0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fc7e800c970 tx=0x7fc7e800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:40:54.182 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.180+0000 7fc7eeffd640 1 -- 172.21.15.97:0/4181527799 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e8007bb0 con 0x7fc7f8106170 2024-04-23T10:40:54.182 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.180+0000 7fc7eeffd640 1 -- 172.21.15.97:0/4181527799 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7e8007d50 con 0x7fc7f8106170 2024-04-23T10:40:54.183 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.180+0000 7fc7fffbf640 1 -- 172.21.15.97:0/4181527799 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7f81a50f0 con 0x7fc7f8106170 2024-04-23T10:40:54.183 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.180+0000 7fc7eeffd640 1 -- 172.21.15.97:0/4181527799 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e800f450 con 0x7fc7f8106170 2024-04-23T10:40:54.183 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.180+0000 7fc7fffbf640 1 -- 172.21.15.97:0/4181527799 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7f81a5690 con 0x7fc7f8106170 2024-04-23T10:40:54.189 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.183+0000 7fc7eeffd640 1 -- 172.21.15.97:0/4181527799 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc7e8004f20 con 0x7fc7f8106170 2024-04-23T10:40:54.189 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.183+0000 7fc802db2740 1 Processor -- start 2024-04-23T10:40:54.189 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.183+0000 7fc802db2740 1 -- start start 2024-04-23T10:40:54.189 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.183+0000 7fc802db2740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1d8f19620 0x55c1d8f19a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:54.189 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.183+0000 7fc802db2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c1d8f19f40 0x55c1d8f242c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:54.189 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.183+0000 7fc802db2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1d8f24800 0x55c1d8f26bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:54.189 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.183+0000 7fc802db2740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c1d8e7e970 con 0x55c1d8f19620 2024-04-23T10:40:54.189 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.183+0000 7fc802db2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c1d8e1b4a0 con 0x55c1d8f19f40 2024-04-23T10:40:54.190 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.183+0000 7fc802db2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c1d8e7a930 con 0x55c1d8f24800 2024-04-23T10:40:54.190 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.183+0000 7fc7eeffd640 1 --2- 172.21.15.97:0/4181527799 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fc7e40a3e40 0x7fc7e40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:54.190 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.184+0000 7fc7fe55c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1d8f19620 0x55c1d8f19a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:54.190 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.184+0000 7fc7fe55c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1d8f19620 0x55c1d8f19a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39186/0 (socket says 172.21.15.97:39186) 2024-04-23T10:40:54.190 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.184+0000 7fc7fe55c640 1 -- 172.21.15.97:0/1715910465 learned_addr learned my addr 172.21.15.97:0/1715910465 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:40:54.190 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.184+0000 7fc7eeffd640 1 -- 172.21.15.97:0/4181527799 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8484+0+0 (secure 0 0 0) 0x7fc7e80d0840 con 0x7fc7f8106170 2024-04-23T10:40:54.190 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.184+0000 7fc7fd55a640 1 --2- 172.21.15.97:0/1715910465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1d8f24800 0x55c1d8f26bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:54.190 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.184+0000 7fc7fe55c640 1 --2- 172.21.15.97:0/4181527799 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fc7e40a3e40 0x7fc7e40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:54.190 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.184+0000 7fc7fdd5b640 1 --2- 172.21.15.97:0/1715910465 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c1d8f19f40 0x55c1d8f242c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:54.190 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.184+0000 7fc7fe55c640 1 -- 172.21.15.97:0/1715910465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1d8f24800 msgr2=0x55c1d8f26bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:40:54.190 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.184+0000 7fc7fe55c640 1 --2- 172.21.15.97:0/1715910465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1d8f24800 0x55c1d8f26bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:54.190 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.184+0000 7fc7fe55c640 1 -- 172.21.15.97:0/1715910465 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c1d8f19f40 msgr2=0x55c1d8f242c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:40:54.190 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.184+0000 7fc7fe55c640 1 --2- 172.21.15.97:0/1715910465 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c1d8f19f40 0x55c1d8f242c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:54.190 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.184+0000 7fc7fe55c640 1 -- 172.21.15.97:0/1715910465 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1d8b91210 con 0x55c1d8f19620 2024-04-23T10:40:54.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.184+0000 7fc7fe55c640 1 --2- 172.21.15.97:0/1715910465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1d8f19620 0x55c1d8f19a00 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fc7f0013ab0 tx=0x7fc7f0013f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce121263d0a5e0c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:40:54.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.184+0000 7fc7fe55c640 1 --2- 172.21.15.97:0/4181527799 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fc7e40a3e40 0x7fc7e40a6300 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc7f0010800 tx=0x7fc7f001b000 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:40:54.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.184+0000 7fc7e3fff640 1 -- 172.21.15.97:0/1715910465 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f0021070 con 0x55c1d8f19620 2024-04-23T10:40:54.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.184+0000 7fc7e3fff640 1 -- 172.21.15.97:0/1715910465 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7f000b8b0 con 0x55c1d8f19620 2024-04-23T10:40:54.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.184+0000 7fc7e3fff640 1 -- 172.21.15.97:0/1715910465 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f001d650 con 0x55c1d8f19620 2024-04-23T10:40:54.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.185+0000 7fc802db2740 1 -- 172.21.15.97:0/1715910465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1d8f19620 msgr2=0x55c1d8f19a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:40:54.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.185+0000 7fc802db2740 1 --2- 172.21.15.97:0/1715910465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1d8f19620 0x55c1d8f19a00 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fc7f0013ab0 tx=0x7fc7f0013f70 comp rx=0 tx=0).stop 2024-04-23T10:40:54.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.185+0000 7fc802db2740 1 -- 172.21.15.97:0/1715910465 shutdown_connections 2024-04-23T10:40:54.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.185+0000 7fc802db2740 1 --2- 172.21.15.97:0/1715910465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1d8f24800 0x55c1d8f26bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:54.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.185+0000 7fc802db2740 1 --2- 172.21.15.97:0/1715910465 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c1d8f19f40 0x55c1d8f242c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:54.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.185+0000 7fc802db2740 1 --2- 172.21.15.97:0/1715910465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1d8f19620 0x55c1d8f19a00 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:54.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.185+0000 7fc802db2740 1 -- 172.21.15.97:0/1715910465 >> 172.21.15.97:0/1715910465 conn(0x55c1d8f15b30 msgr2=0x55c1d8f15f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:40:54.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.185+0000 7fc802db2740 1 -- 172.21.15.97:0/1715910465 shutdown_connections 2024-04-23T10:40:54.191 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.185+0000 7fc802db2740 1 -- 172.21.15.97:0/1715910465 wait complete. 2024-04-23T10:40:54.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.186+0000 7fc802db2740 1 -- start start 2024-04-23T10:40:54.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.186+0000 7fc802db2740 1 Processor -- start 2024-04-23T10:40:54.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.186+0000 7fc802db2740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1d8f19620 0x55c1d8f24210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:54.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.186+0000 7fc802db2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1d8f19f40 0x55c1d8f21500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:54.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.186+0000 7fc802db2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c1d8f24800 0x55c1d8f21b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:54.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.186+0000 7fc802db2740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c1d8f273d0 con 0x55c1d8f19620 2024-04-23T10:40:54.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.186+0000 7fc802db2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c1d8e190b0 con 0x55c1d8f24800 2024-04-23T10:40:54.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.186+0000 7fc802db2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c1d8f27550 con 0x55c1d8f19f40 2024-04-23T10:40:54.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.186+0000 7fc802db2740 10 client.0 _ll_register_callbacks cb 0x55c1d8e06d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:40:54.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.186+0000 7fc7fdd5b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1d8f19620 0x55c1d8f24210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:54.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.186+0000 7fc7fdd5b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1d8f19620 0x55c1d8f24210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39202/0 (socket says 172.21.15.97:39202) 2024-04-23T10:40:54.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.186+0000 7fc7fdd5b640 1 -- 172.21.15.97:0/2546227942 learned_addr learned my addr 172.21.15.97:0/2546227942 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:40:54.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.186+0000 7fc7fd55a640 1 --2- 172.21.15.97:0/2546227942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1d8f19f40 0x55c1d8f21500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:54.192 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.186+0000 7fc7fe55c640 1 --2- 172.21.15.97:0/2546227942 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c1d8f24800 0x55c1d8f21b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:54.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.187+0000 7fc7fdd5b640 1 -- 172.21.15.97:0/2546227942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1d8f19f40 msgr2=0x55c1d8f21500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:40:54.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.187+0000 7fc7fdd5b640 1 --2- 172.21.15.97:0/2546227942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1d8f19f40 0x55c1d8f21500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:54.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.187+0000 7fc7fdd5b640 1 -- 172.21.15.97:0/2546227942 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c1d8f24800 msgr2=0x55c1d8f21b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:40:54.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.187+0000 7fc7fdd5b640 1 --2- 172.21.15.97:0/2546227942 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c1d8f24800 0x55c1d8f21b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:54.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.187+0000 7fc7fdd5b640 1 -- 172.21.15.97:0/2546227942 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1d8c5d620 con 0x55c1d8f19620 2024-04-23T10:40:54.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.187+0000 7fc7fe55c640 1 --2- 172.21.15.97:0/2546227942 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c1d8f24800 0x55c1d8f21b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:40:54.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.187+0000 7fc7e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:40:54.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.187+0000 7fc7fdd5b640 1 --2- 172.21.15.97:0/2546227942 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1d8f19620 0x55c1d8f24210 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fc7f4009ca0 tx=0x7fc7f4031450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f88be6496f72762 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:40:54.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.187+0000 7fc7e17fa640 1 -- 172.21.15.97:0/2546227942 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f4031790 con 0x55c1d8f19620 2024-04-23T10:40:54.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.187+0000 7fc7e17fa640 1 -- 172.21.15.97:0/2546227942 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8484+0+0 (secure 0 0 0) 0x7fc7f4003970 con 0x55c1d8f19620 2024-04-23T10:40:54.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.187+0000 7fc802db2740 1 -- 172.21.15.97:0/2546227942 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c1d8e8c390 con 0x55c1d8f19620 2024-04-23T10:40:54.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.187+0000 7fc7e17fa640 1 -- 172.21.15.97:0/2546227942 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc7f4003b70 con 0x55c1d8f19620 2024-04-23T10:40:54.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.187+0000 7fc802db2740 10 client.17583 fetch_fsmap learned FSMap version 26 2024-04-23T10:40:54.193 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.187+0000 7fc802db2740 1 -- 172.21.15.97:0/2546227942 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c1d8f333c0 con 0x55c1d8f19620 2024-04-23T10:40:54.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.187+0000 7fc7e17fa640 1 -- 172.21.15.97:0/2546227942 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55c1d8e8c390 con 0x55c1d8f19620 2024-04-23T10:40:54.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.187+0000 7fc7e17fa640 1 -- 172.21.15.97:0/2546227942 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc7f404c030 con 0x55c1d8f19620 2024-04-23T10:40:54.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.188+0000 7fc802db2740 10 client.17583 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:40:54.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.188+0000 7fc802db2740 10 client.17583 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:40:54.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.188+0000 7fc802db2740 20 client.17583 populate_metadata read hostname 'smithi097' 2024-04-23T10:40:54.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.188+0000 7fc802db2740 1 --2- 172.21.15.97:0/2546227942 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55c1d8f348d0 0x55c1d8f36cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:54.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.188+0000 7fc802db2740 4 client.17583 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:40:54.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.188+0000 7fc802db2740 1 -- 172.21.15.97:0/2546227942 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c1d8e86200 con 0x55c1d8f348d0 2024-04-23T10:40:54.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.188+0000 7fc7fd55a640 1 --2- 172.21.15.97:0/2546227942 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55c1d8f348d0 0x55c1d8f36cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:54.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.188+0000 7fc7e17fa640 10 client.17583 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:40:54.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.188+0000 7fc7fd55a640 1 --2- 172.21.15.97:0/2546227942 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55c1d8f348d0 0x55c1d8f36cc0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=986c83bd16ca3386 server_cookie=7e46c5ffb7c905c6 in_seq=0 out_seq=0 2024-04-23T10:40:54.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.189+0000 7fc7e17fa640 1 -- 172.21.15.97:0/2546227942 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c1d8e86200 con 0x55c1d8f348d0 2024-04-23T10:40:54.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.189+0000 7fc7e17fa640 10 client.17583 handle_command_reply: tid=0 2024-04-23T10:40:54.194 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.189+0000 7fc802db2740 1 client.17583 shutdown 2024-04-23T10:40:54.195 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.189+0000 7fc802db2740 1 -- 172.21.15.97:0/2546227942 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1d8f19620 msgr2=0x55c1d8f24210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:40:54.195 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.189+0000 7fc802db2740 1 --2- 172.21.15.97:0/2546227942 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1d8f19620 0x55c1d8f24210 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fc7f4009ca0 tx=0x7fc7f4031450 comp rx=0 tx=0).stop 2024-04-23T10:40:54.195 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.189+0000 7fc802db2740 1 -- 172.21.15.97:0/2546227942 shutdown_connections 2024-04-23T10:40:54.195 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.189+0000 7fc802db2740 1 --2- 172.21.15.97:0/2546227942 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55c1d8f348d0 0x55c1d8f36cc0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:54.195 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.189+0000 7fc802db2740 1 --2- 172.21.15.97:0/2546227942 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c1d8f24800 0x55c1d8f21b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:54.195 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.189+0000 7fc802db2740 1 --2- 172.21.15.97:0/2546227942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1d8f19f40 0x55c1d8f21500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:54.195 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.189+0000 7fc802db2740 1 --2- 172.21.15.97:0/2546227942 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1d8f19620 0x55c1d8f24210 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:54.195 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.189+0000 7fc7e17fa640 0 client.17583 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:40:54.195 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.189+0000 7fc802db2740 1 -- 172.21.15.97:0/2546227942 >> 172.21.15.97:0/2546227942 conn(0x55c1d8f15b30 msgr2=0x55c1d8f1b860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:40:54.195 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.189+0000 7fc802db2740 1 -- 172.21.15.97:0/2546227942 shutdown_connections 2024-04-23T10:40:54.195 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.189+0000 7fc802db2740 1 -- 172.21.15.97:0/2546227942 wait complete. 2024-04-23T10:40:54.195 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.189+0000 7fc802db2740 20 client.17583 trim_cache size 0 max 16384 2024-04-23T10:40:54.209 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.204+0000 7fc802db2740 1 Processor -- start 2024-04-23T10:40:54.209 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.204+0000 7fc802db2740 1 -- start start 2024-04-23T10:40:54.209 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.204+0000 7fc802db2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1d8f19620 0x55c1d8f30290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:54.209 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.204+0000 7fc802db2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c1d8f19f40 0x55c1d8f14b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:54.209 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.204+0000 7fc802db2740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1d8f24800 0x55c1d8f23d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:54.209 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.204+0000 7fc802db2740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f81adb10 con 0x55c1d8f24800 2024-04-23T10:40:54.209 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.204+0000 7fc802db2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f81af650 con 0x55c1d8f19f40 2024-04-23T10:40:54.209 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.204+0000 7fc802db2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f81ae1e0 con 0x55c1d8f19620 2024-04-23T10:40:54.210 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.204+0000 7fc7fe55c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1d8f24800 0x55c1d8f23d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:54.210 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.204+0000 7fc7fe55c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1d8f24800 0x55c1d8f23d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39206/0 (socket says 172.21.15.97:39206) 2024-04-23T10:40:54.210 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.204+0000 7fc7fe55c640 1 -- 172.21.15.97:0/349667345 learned_addr learned my addr 172.21.15.97:0/349667345 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:40:54.210 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.205+0000 7fc7fdd5b640 1 --2- 172.21.15.97:0/349667345 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1d8f19620 0x55c1d8f30290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:54.210 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.205+0000 7fc7fe55c640 1 -- 172.21.15.97:0/349667345 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1d8f19620 msgr2=0x55c1d8f30290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:40:54.210 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.205+0000 7fc7fe55c640 1 --2- 172.21.15.97:0/349667345 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1d8f19620 0x55c1d8f30290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:54.210 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.205+0000 7fc7fe55c640 1 -- 172.21.15.97:0/349667345 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c1d8f19f40 msgr2=0x55c1d8f14b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:40:54.210 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.205+0000 7fc7fe55c640 1 --2- 172.21.15.97:0/349667345 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c1d8f19f40 0x55c1d8f14b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:54.210 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.205+0000 7fc7fe55c640 1 -- 172.21.15.97:0/349667345 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1d8f204b0 con 0x55c1d8f24800 2024-04-23T10:40:54.210 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.205+0000 7fc7fe55c640 1 --2- 172.21.15.97:0/349667345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1d8f24800 0x55c1d8f23d40 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fc7f0013950 tx=0x7fc7f000b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f960c9191ce5ffcf server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:40:54.210 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.205+0000 7fc7e37fe640 1 -- 172.21.15.97:0/349667345 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f0021040 con 0x55c1d8f24800 2024-04-23T10:40:54.210 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.205+0000 7fc7e37fe640 1 -- 172.21.15.97:0/349667345 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7f001d3f0 con 0x55c1d8f24800 2024-04-23T10:40:54.210 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.205+0000 7fc7e37fe640 1 -- 172.21.15.97:0/349667345 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f001de00 con 0x55c1d8f24800 2024-04-23T10:40:54.211 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.206+0000 7fc802db2740 1 -- 172.21.15.97:0/349667345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1d8f24800 msgr2=0x55c1d8f23d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:40:54.211 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.206+0000 7fc802db2740 1 --2- 172.21.15.97:0/349667345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1d8f24800 0x55c1d8f23d40 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fc7f0013950 tx=0x7fc7f000b3e0 comp rx=0 tx=0).stop 2024-04-23T10:40:54.211 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.206+0000 7fc802db2740 1 -- 172.21.15.97:0/349667345 shutdown_connections 2024-04-23T10:40:54.211 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.206+0000 7fc802db2740 1 --2- 172.21.15.97:0/349667345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1d8f24800 0x55c1d8f23d40 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:54.211 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.206+0000 7fc802db2740 1 --2- 172.21.15.97:0/349667345 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c1d8f19f40 0x55c1d8f14b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:54.211 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.206+0000 7fc802db2740 1 --2- 172.21.15.97:0/349667345 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1d8f19620 0x55c1d8f30290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:54.211 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.206+0000 7fc802db2740 1 -- 172.21.15.97:0/349667345 >> 172.21.15.97:0/349667345 conn(0x55c1d8f15b30 msgr2=0x55c1d8f14420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:40:54.211 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.206+0000 7fc802db2740 1 -- 172.21.15.97:0/349667345 shutdown_connections 2024-04-23T10:40:54.211 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.206+0000 7fc802db2740 1 -- 172.21.15.97:0/349667345 wait complete. 2024-04-23T10:40:54.211 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.206+0000 7fc802db2740 1 -- start start 2024-04-23T10:40:54.211 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.206+0000 7fc802db2740 1 Processor -- start 2024-04-23T10:40:54.211 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.206+0000 7fc802db2740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1d8f19620 0x55c1d8f1e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:54.211 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.206+0000 7fc802db2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c1d8f19f40 0x55c1d8f1e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:54.212 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.206+0000 7fc802db2740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1d8f24800 0x55c1d8f1ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:54.212 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.206+0000 7fc802db2740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f81aef80 con 0x55c1d8f19620 2024-04-23T10:40:54.212 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.206+0000 7fc802db2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f81ae8b0 con 0x55c1d8f19f40 2024-04-23T10:40:54.212 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.206+0000 7fc802db2740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f81abb30 con 0x55c1d8f24800 2024-04-23T10:40:54.212 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.206+0000 7fc802db2740 10 client.0 _ll_register_callbacks cb 0x7fc7f4031f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:40:54.212 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.206+0000 7fc7fdd5b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1d8f19620 0x55c1d8f1e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:54.212 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.207+0000 7fc7fdd5b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1d8f19620 0x55c1d8f1e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39208/0 (socket says 172.21.15.97:39208) 2024-04-23T10:40:54.212 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.207+0000 7fc7fdd5b640 1 -- 172.21.15.97:0/218571381 learned_addr learned my addr 172.21.15.97:0/218571381 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:40:54.212 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.207+0000 7fc7fe55c640 1 --2- 172.21.15.97:0/218571381 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1d8f24800 0x55c1d8f1ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:54.212 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.207+0000 7fc7fd55a640 1 --2- 172.21.15.97:0/218571381 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c1d8f19f40 0x55c1d8f1e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:54.212 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.207+0000 7fc7fdd5b640 1 -- 172.21.15.97:0/218571381 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1d8f24800 msgr2=0x55c1d8f1ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:40:54.213 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.207+0000 7fc7fdd5b640 1 --2- 172.21.15.97:0/218571381 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1d8f24800 0x55c1d8f1ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:54.213 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.207+0000 7fc7fdd5b640 1 -- 172.21.15.97:0/218571381 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c1d8f19f40 msgr2=0x55c1d8f1e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:40:54.213 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.207+0000 7fc7fdd5b640 1 --2- 172.21.15.97:0/218571381 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c1d8f19f40 0x55c1d8f1e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:54.213 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.207+0000 7fc7fdd5b640 1 -- 172.21.15.97:0/218571381 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1d8f99e30 con 0x55c1d8f19620 2024-04-23T10:40:54.213 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.207+0000 7fc7fe55c640 1 --2- 172.21.15.97:0/218571381 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1d8f24800 0x55c1d8f1ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:40:54.213 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.207+0000 7fc7fd55a640 1 --2- 172.21.15.97:0/218571381 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c1d8f19f40 0x55c1d8f1e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:40:54.213 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.207+0000 7fc7fdd5b640 1 --2- 172.21.15.97:0/218571381 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1d8f19620 0x55c1d8f1e200 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fc7f402a900 tx=0x7fc7f4003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3266f2f74053dc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:40:54.213 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.207+0000 7fc7e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:40:54.213 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.207+0000 7fc7e17fa640 1 -- 172.21.15.97:0/218571381 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f4004d50 con 0x55c1d8f19620 2024-04-23T10:40:54.213 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.207+0000 7fc7e17fa640 1 -- 172.21.15.97:0/218571381 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8484+0+0 (secure 0 0 0) 0x7fc7f402f900 con 0x55c1d8f19620 2024-04-23T10:40:54.214 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.207+0000 7fc802db2740 1 -- 172.21.15.97:0/218571381 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7f001d3f0 con 0x55c1d8f19620 2024-04-23T10:40:54.214 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.207+0000 7fc7e17fa640 1 -- 172.21.15.97:0/218571381 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc7f405e050 con 0x55c1d8f19620 2024-04-23T10:40:54.214 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.207+0000 7fc802db2740 10 client.17589 fetch_fsmap learned FSMap version 26 2024-04-23T10:40:54.214 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.207+0000 7fc802db2740 1 -- 172.21.15.97:0/218571381 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c1d8f9b140 con 0x55c1d8f19620 2024-04-23T10:40:54.214 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.208+0000 7fc7e17fa640 1 -- 172.21.15.97:0/218571381 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7f001d3f0 con 0x55c1d8f19620 2024-04-23T10:40:54.214 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.208+0000 7fc7e17fa640 1 -- 172.21.15.97:0/218571381 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc7f4055070 con 0x55c1d8f19620 2024-04-23T10:40:54.214 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.208+0000 7fc802db2740 10 client.17589 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:40:54.214 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.208+0000 7fc802db2740 10 client.17589 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:40:54.214 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.208+0000 7fc802db2740 20 client.17589 populate_metadata read hostname 'smithi097' 2024-04-23T10:40:54.214 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.208+0000 7fc802db2740 1 --2- 172.21.15.97:0/218571381 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55c1d8f9b8e0 0x55c1d8f9dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:40:54.214 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.208+0000 7fc802db2740 4 client.17589 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:40:54.214 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.208+0000 7fc802db2740 1 -- 172.21.15.97:0/218571381 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c1d8e025f0 con 0x55c1d8f9b8e0 2024-04-23T10:40:54.214 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.208+0000 7fc7fd55a640 1 --2- 172.21.15.97:0/218571381 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55c1d8f9b8e0 0x55c1d8f9dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:40:54.214 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.208+0000 7fc7e17fa640 10 client.17589 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:40:54.215 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.208+0000 7fc7fd55a640 1 --2- 172.21.15.97:0/218571381 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55c1d8f9b8e0 0x55c1d8f9dd00 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e489ff30ad94e9c server_cookie=f23b4e4dde3a5727 in_seq=0 out_seq=0 2024-04-23T10:40:54.221 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.221+0000 7fc7e17fa640 1 -- 172.21.15.97:0/218571381 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c1d8e025f0 con 0x55c1d8f9b8e0 2024-04-23T10:40:54.221 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.221+0000 7fc7e17fa640 10 client.17589 handle_command_reply: tid=0 2024-04-23T10:40:54.221 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.221+0000 7fc802db2740 1 client.17589 shutdown 2024-04-23T10:40:54.221 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.222+0000 7fc802db2740 1 -- 172.21.15.97:0/218571381 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1d8f19620 msgr2=0x55c1d8f1e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:40:54.221 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.222+0000 7fc802db2740 1 --2- 172.21.15.97:0/218571381 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1d8f19620 0x55c1d8f1e200 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fc7f402a900 tx=0x7fc7f4003970 comp rx=0 tx=0).stop 2024-04-23T10:40:54.221 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.222+0000 7fc802db2740 1 -- 172.21.15.97:0/218571381 shutdown_connections 2024-04-23T10:40:54.221 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.222+0000 7fc802db2740 1 --2- 172.21.15.97:0/218571381 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55c1d8f9b8e0 0x55c1d8f9dd00 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:54.221 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.222+0000 7fc802db2740 1 --2- 172.21.15.97:0/218571381 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1d8f24800 0x55c1d8f1ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:54.221 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.222+0000 7fc7e17fa640 0 client.17589 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:40:54.221 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.222+0000 7fc802db2740 1 --2- 172.21.15.97:0/218571381 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c1d8f19f40 0x55c1d8f1e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:54.222 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.222+0000 7fc802db2740 1 --2- 172.21.15.97:0/218571381 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c1d8f19620 0x55c1d8f1e200 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:54.222 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.222+0000 7fc802db2740 1 -- 172.21.15.97:0/218571381 >> 172.21.15.97:0/218571381 conn(0x55c1d8f15b30 msgr2=0x55c1d8f1c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:40:54.222 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.222+0000 7fc802db2740 1 -- 172.21.15.97:0/218571381 shutdown_connections 2024-04-23T10:40:54.222 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.222+0000 7fc802db2740 1 -- 172.21.15.97:0/218571381 wait complete. 2024-04-23T10:40:54.222 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.222+0000 7fc802db2740 20 client.17589 trim_cache size 0 max 16384 2024-04-23T10:40:54.222 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:40:54.222 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:40:54.222 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:40:54.222 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:40:54.223 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.222+0000 7fc7fffbf640 1 -- 172.21.15.97:0/4181527799 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fc7e40a3e40 msgr2=0x7fc7e40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:40:54.223 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.223+0000 7fc7fffbf640 1 --2- 172.21.15.97:0/4181527799 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fc7e40a3e40 0x7fc7e40a6300 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc7f0010800 tx=0x7fc7f001b000 comp rx=0 tx=0).stop 2024-04-23T10:40:54.223 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.223+0000 7fc7fffbf640 1 -- 172.21.15.97:0/4181527799 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7f8106170 msgr2=0x7fc7f8199af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:40:54.223 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.223+0000 7fc7fffbf640 1 --2- 172.21.15.97:0/4181527799 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7f8106170 0x7fc7f8199af0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fc7e800c970 tx=0x7fc7e800ce30 comp rx=0 tx=0).stop 2024-04-23T10:40:54.223 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.223+0000 7fc7fffbf640 1 -- 172.21.15.97:0/4181527799 shutdown_connections 2024-04-23T10:40:54.223 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.223+0000 7fc7fffbf640 1 --2- 172.21.15.97:0/4181527799 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fc7e40a3e40 0x7fc7e40a6300 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:54.223 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.223+0000 7fc7fffbf640 1 --2- 172.21.15.97:0/4181527799 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7f819a080 0x7fc7f81a4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:54.224 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.223+0000 7fc7fffbf640 1 --2- 172.21.15.97:0/4181527799 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7f8106170 0x7fc7f8199af0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:54.224 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.223+0000 7fc7fffbf640 1 --2- 172.21.15.97:0/4181527799 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7f81010f0 0x7fc7f81995b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:40:54.224 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.223+0000 7fc7fffbf640 1 -- 172.21.15.97:0/4181527799 >> 172.21.15.97:0/4181527799 conn(0x7fc7f8077fd0 msgr2=0x7fc7f80ff250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:40:54.224 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.223+0000 7fc7fffbf640 1 -- 172.21.15.97:0/4181527799 shutdown_connections 2024-04-23T10:40:54.224 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:40:54.223+0000 7fc7fffbf640 1 -- 172.21.15.97:0/4181527799 wait complete. 2024-04-23T10:40:55.144 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:40:54 smithi196 ceph-mon[32919]: pgmap v460: 97 pgs: 97 active+clean; 2.2 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-23T10:40:55.149 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:40:54 smithi183 ceph-mon[32317]: pgmap v460: 97 pgs: 97 active+clean; 2.2 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-23T10:40:55.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:40:54 smithi097 ceph-mon[29197]: pgmap v460: 97 pgs: 97 active+clean; 2.2 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-23T10:40:57.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:40:56 smithi183 ceph-mon[32317]: pgmap v461: 97 pgs: 97 active+clean; 2.2 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-04-23T10:40:57.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:40:56 smithi196 ceph-mon[32919]: pgmap v461: 97 pgs: 97 active+clean; 2.2 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-04-23T10:40:57.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:40:56 smithi097 ceph-mon[29197]: pgmap v461: 97 pgs: 97 active+clean; 2.2 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-04-23T10:40:59.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:40:58 smithi183 ceph-mon[32317]: pgmap v462: 97 pgs: 97 active+clean; 2.2 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 39 KiB/s wr, 19 op/s 2024-04-23T10:40:59.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:40:58 smithi196 ceph-mon[32919]: pgmap v462: 97 pgs: 97 active+clean; 2.2 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 39 KiB/s wr, 19 op/s 2024-04-23T10:40:59.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:40:58 smithi097 ceph-mon[29197]: pgmap v462: 97 pgs: 97 active+clean; 2.2 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 39 KiB/s wr, 19 op/s 2024-04-23T10:41:00.351 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T10:41:00.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.453+0000 7fb8b3ea7640 1 -- 172.21.15.97:0/329282200 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8ac1010a0 msgr2=0x7fb8ac101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:00.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.454+0000 7fb8b3ea7640 1 --2- 172.21.15.97:0/329282200 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8ac1010a0 0x7fb8ac101480 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fb89c009900 tx=0x7fb89c02a410 comp rx=0 tx=0).stop 2024-04-23T10:41:00.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.454+0000 7fb8b3ea7640 1 -- 172.21.15.97:0/329282200 shutdown_connections 2024-04-23T10:41:00.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.454+0000 7fb8b3ea7640 1 --2- 172.21.15.97:0/329282200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb8ac106170 0x7fb8ac10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:00.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.454+0000 7fb8b3ea7640 1 --2- 172.21.15.97:0/329282200 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8ac101a50 0x7fb8ac105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:00.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.454+0000 7fb8b3ea7640 1 --2- 172.21.15.97:0/329282200 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8ac1010a0 0x7fb8ac101480 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:00.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.454+0000 7fb8b3ea7640 1 -- 172.21.15.97:0/329282200 >> 172.21.15.97:0/329282200 conn(0x7fb8ac077f50 msgr2=0x7fb8ac078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:41:00.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.454+0000 7fb8b3ea7640 1 -- 172.21.15.97:0/329282200 shutdown_connections 2024-04-23T10:41:00.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.454+0000 7fb8b3ea7640 1 -- 172.21.15.97:0/329282200 wait complete. 2024-04-23T10:41:00.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.454+0000 7fb8b3ea7640 1 Processor -- start 2024-04-23T10:41:00.455 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.454+0000 7fb8b3ea7640 1 -- start start 2024-04-23T10:41:00.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.454+0000 7fb8b3ea7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8ac1010a0 0x7fb8ac19e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:00.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.454+0000 7fb8b3ea7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8ac106170 0x7fb8ac19e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:00.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.454+0000 7fb8b3ea7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb8ac1985e0 0x7fb8ac198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:00.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.454+0000 7fb8b3ea7640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ac111150 con 0x7fb8ac1010a0 2024-04-23T10:41:00.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.454+0000 7fb8b3ea7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ac110fd0 con 0x7fb8ac1985e0 2024-04-23T10:41:00.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.454+0000 7fb8b3ea7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb8ac1112d0 con 0x7fb8ac106170 2024-04-23T10:41:00.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.455+0000 7fb8b1c43640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8ac1010a0 0x7fb8ac19e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:00.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.455+0000 7fb8b1c43640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8ac1010a0 0x7fb8ac19e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41794/0 (socket says 172.21.15.97:41794) 2024-04-23T10:41:00.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.455+0000 7fb8b1c43640 1 -- 172.21.15.97:0/147068387 learned_addr learned my addr 172.21.15.97:0/147068387 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:41:00.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.455+0000 7fb8b1442640 1 --2- 172.21.15.97:0/147068387 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8ac106170 0x7fb8ac19e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:00.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.455+0000 7fb8b2444640 1 --2- 172.21.15.97:0/147068387 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb8ac1985e0 0x7fb8ac198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:00.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.455+0000 7fb8b1c43640 1 -- 172.21.15.97:0/147068387 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8ac106170 msgr2=0x7fb8ac19e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:00.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.455+0000 7fb8b1c43640 1 --2- 172.21.15.97:0/147068387 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8ac106170 0x7fb8ac19e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:00.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.455+0000 7fb8b1c43640 1 -- 172.21.15.97:0/147068387 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb8ac1985e0 msgr2=0x7fb8ac198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:00.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.455+0000 7fb8b1c43640 1 --2- 172.21.15.97:0/147068387 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb8ac1985e0 0x7fb8ac198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:00.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.455+0000 7fb8b1c43640 1 -- 172.21.15.97:0/147068387 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8ac199100 con 0x7fb8ac1010a0 2024-04-23T10:41:00.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.455+0000 7fb8b2444640 1 --2- 172.21.15.97:0/147068387 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb8ac1985e0 0x7fb8ac198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:41:00.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.455+0000 7fb8b1442640 1 --2- 172.21.15.97:0/147068387 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8ac106170 0x7fb8ac19e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:41:00.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.455+0000 7fb8b1c43640 1 --2- 172.21.15.97:0/147068387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8ac1010a0 0x7fb8ac19e430 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fb89c005940 tx=0x7fb89c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:41:00.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.455+0000 7fb8a2ffd640 1 -- 172.21.15.97:0/147068387 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89c03f030 con 0x7fb8ac1010a0 2024-04-23T10:41:00.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.455+0000 7fb8a2ffd640 1 -- 172.21.15.97:0/147068387 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb89c003cb0 con 0x7fb8ac1010a0 2024-04-23T10:41:00.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.455+0000 7fb8b3ea7640 1 -- 172.21.15.97:0/147068387 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8ac199390 con 0x7fb8ac1010a0 2024-04-23T10:41:00.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.455+0000 7fb8a2ffd640 1 -- 172.21.15.97:0/147068387 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89c035070 con 0x7fb8ac1010a0 2024-04-23T10:41:00.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.455+0000 7fb8b3ea7640 1 -- 172.21.15.97:0/147068387 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8ac1a5580 con 0x7fb8ac1010a0 2024-04-23T10:41:00.457 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.456+0000 7fb8b3ea7640 1 -- 172.21.15.97:0/147068387 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8ac068d70 con 0x7fb8ac1010a0 2024-04-23T10:41:00.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.457+0000 7fb8a2ffd640 1 -- 172.21.15.97:0/147068387 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb89c042030 con 0x7fb8ac1010a0 2024-04-23T10:41:00.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.458+0000 7fb8a2ffd640 1 --2- 172.21.15.97:0/147068387 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fb8980a3d20 0x7fb8980a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:00.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.458+0000 7fb8b1442640 1 --2- 172.21.15.97:0/147068387 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fb8980a3d20 0x7fb8980a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:00.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.458+0000 7fb8a2ffd640 1 -- 172.21.15.97:0/147068387 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8484+0+0 (secure 0 0 0) 0x7fb89c0f13f0 con 0x7fb8ac1010a0 2024-04-23T10:41:00.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.458+0000 7fb8b1442640 1 --2- 172.21.15.97:0/147068387 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fb8980a3d20 0x7fb8980a61e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb8a80058c0 tx=0x7fb8a8005850 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:41:00.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.461+0000 7fb8a2ffd640 1 -- 172.21.15.97:0/147068387 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb89c0bdab0 con 0x7fb8ac1010a0 2024-04-23T10:41:00.677 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.678+0000 7fb8b3ea7640 1 -- 172.21.15.97:0/147068387 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb8ac101c50 con 0x7fb8ac1010a0 2024-04-23T10:41:00.678 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.679+0000 7fb8a2ffd640 1 -- 172.21.15.97:0/147068387 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11931 (secure 0 0 0) 0x7fb89c0bd3b0 con 0x7fb8ac1010a0 2024-04-23T10:41:00.678 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:41:00.678 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3946878762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3946878762},{"type":"v1","addr":"172.21.15.97:6835","nonce":3946878762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/209800861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":209800861},{"type":"v1","addr":"172.21.15.97:6837","nonce":209800861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T10:24:06.910134+0000","modified":"2024-04-23T10:26:09.820985+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6839/418013575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":418013575},{"type":"v1","addr":"172.21.15.97:6839","nonce":418013575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6841/747242680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":747242680},{"type":"v1","addr":"172.21.15.97:6841","nonce":747242680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6833/4292847121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4292847121},{"type":"v1","addr":"172.21.15.183:6833","nonce":4292847121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6833/3885816565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3885816565},{"type":"v1","addr":"172.21.15.196:6833","nonce":3885816565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1897093105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1897093105},{"type":"v1","addr":"172.21.15.183:6835","nonce":1897093105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1171477563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1171477563},{"type":"v1","addr":"172.21.15.196:6835","nonce":1171477563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1898341911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1898341911},{"type":"v1","addr":"172.21.15.183:6837","nonce":1898341911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.196:6837/3094946502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3094946502},{"type":"v1","addr":"172.21.15.196:6837","nonce":3094946502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.196:6839/83843556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":83843556},{"type":"v1","addr":"172.21.15.196:6839","nonce":83843556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6839/1763841130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1763841130},{"type":"v1","addr":"172.21.15.183:6839","nonce":1763841130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24487,24403,24398,24469,24488]},"id":1}]} 2024-04-23T10:41:00.678 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 26 2024-04-23T10:41:00.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.681+0000 7fb8b3ea7640 1 -- 172.21.15.97:0/147068387 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fb8980a3d20 msgr2=0x7fb8980a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:00.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.681+0000 7fb8b3ea7640 1 --2- 172.21.15.97:0/147068387 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fb8980a3d20 0x7fb8980a61e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb8a80058c0 tx=0x7fb8a8005850 comp rx=0 tx=0).stop 2024-04-23T10:41:00.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.681+0000 7fb8b3ea7640 1 -- 172.21.15.97:0/147068387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8ac1010a0 msgr2=0x7fb8ac19e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:00.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.681+0000 7fb8b3ea7640 1 --2- 172.21.15.97:0/147068387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8ac1010a0 0x7fb8ac19e430 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fb89c005940 tx=0x7fb89c0042f0 comp rx=0 tx=0).stop 2024-04-23T10:41:00.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.681+0000 7fb8b3ea7640 1 -- 172.21.15.97:0/147068387 shutdown_connections 2024-04-23T10:41:00.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.681+0000 7fb8b3ea7640 1 --2- 172.21.15.97:0/147068387 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fb8980a3d20 0x7fb8980a61e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:00.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.681+0000 7fb8b3ea7640 1 --2- 172.21.15.97:0/147068387 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb8ac1985e0 0x7fb8ac198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:00.680 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.681+0000 7fb8b3ea7640 1 --2- 172.21.15.97:0/147068387 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8ac106170 0x7fb8ac19e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:00.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.681+0000 7fb8b3ea7640 1 --2- 172.21.15.97:0/147068387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8ac1010a0 0x7fb8ac19e430 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:00.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.681+0000 7fb8b3ea7640 1 -- 172.21.15.97:0/147068387 >> 172.21.15.97:0/147068387 conn(0x7fb8ac077f50 msgr2=0x7fb8ac104e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:41:00.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.681+0000 7fb8b3ea7640 1 -- 172.21.15.97:0/147068387 shutdown_connections 2024-04-23T10:41:00.681 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.681+0000 7fb8b3ea7640 1 -- 172.21.15.97:0/147068387 wait complete. 2024-04-23T10:41:00.690 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --set-id ca3de698 --release --await' 2024-04-23T10:41:00.691 DEBUG:teuthology.orchestra.run.smithi097:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24488 quiesce db --set-id ca3de698 --release --await 2024-04-23T10:41:00.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.796+0000 7f86864ef640 1 -- 172.21.15.97:0/976026563 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8680101ad0 msgr2=0x7f868010dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:00.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.796+0000 7f86864ef640 1 --2- 172.21.15.97:0/976026563 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8680101ad0 0x7f868010dcf0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f8678009ca0 tx=0x7f867802a8b0 comp rx=0 tx=0).stop 2024-04-23T10:41:00.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.796+0000 7f86864ef640 1 -- 172.21.15.97:0/976026563 shutdown_connections 2024-04-23T10:41:00.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.796+0000 7f86864ef640 1 --2- 172.21.15.97:0/976026563 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8680101ad0 0x7f868010dcf0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:00.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.796+0000 7f86864ef640 1 --2- 172.21.15.97:0/976026563 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8680101130 0x7f8680101590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:00.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.796+0000 7f86864ef640 1 --2- 172.21.15.97:0/976026563 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8680105ea0 0x7f8680106280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:00.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.796+0000 7f86864ef640 1 -- 172.21.15.97:0/976026563 >> 172.21.15.97:0/976026563 conn(0x7f8680077e30 msgr2=0x7f8680078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:41:00.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.796+0000 7f86864ef640 1 -- 172.21.15.97:0/976026563 shutdown_connections 2024-04-23T10:41:00.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.796+0000 7f86864ef640 1 -- 172.21.15.97:0/976026563 wait complete. 2024-04-23T10:41:00.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.797+0000 7f86864ef640 1 Processor -- start 2024-04-23T10:41:00.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.797+0000 7f86864ef640 1 -- start start 2024-04-23T10:41:00.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.797+0000 7f86864ef640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8680101130 0x7f868019e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:00.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.797+0000 7f86864ef640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8680101ad0 0x7f868019e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:00.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.797+0000 7f86864ef640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8680105ea0 0x7f8680198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:00.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.797+0000 7f86864ef640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8680112350 con 0x7f8680105ea0 2024-04-23T10:41:00.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.797+0000 7f86864ef640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f86801121d0 con 0x7f8680101ad0 2024-04-23T10:41:00.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.797+0000 7f86864ef640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f86801124d0 con 0x7f8680101130 2024-04-23T10:41:00.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.797+0000 7f8684a8c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8680105ea0 0x7f8680198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:00.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.798+0000 7f8684a8c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8680105ea0 0x7f8680198610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41822/0 (socket says 172.21.15.97:41822) 2024-04-23T10:41:00.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.798+0000 7f8684a8c640 1 -- 172.21.15.97:0/4039489497 learned_addr learned my addr 172.21.15.97:0/4039489497 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:41:00.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.798+0000 7f867ffff640 1 --2- 172.21.15.97:0/4039489497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8680101130 0x7f868019e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:00.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.798+0000 7f867f7fe640 1 --2- 172.21.15.97:0/4039489497 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8680101ad0 0x7f868019e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:00.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.798+0000 7f8684a8c640 1 -- 172.21.15.97:0/4039489497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8680101130 msgr2=0x7f868019e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:00.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.798+0000 7f8684a8c640 1 --2- 172.21.15.97:0/4039489497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8680101130 0x7f868019e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:00.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.798+0000 7f8684a8c640 1 -- 172.21.15.97:0/4039489497 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8680101ad0 msgr2=0x7f868019e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:00.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.798+0000 7f8684a8c640 1 --2- 172.21.15.97:0/4039489497 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8680101ad0 0x7f868019e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:00.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.798+0000 7f8684a8c640 1 -- 172.21.15.97:0/4039489497 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8680198d90 con 0x7f8680105ea0 2024-04-23T10:41:00.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.798+0000 7f867f7fe640 1 --2- 172.21.15.97:0/4039489497 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8680101ad0 0x7f868019e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:41:00.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.798+0000 7f8684a8c640 1 --2- 172.21.15.97:0/4039489497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8680105ea0 0x7f8680198610 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f8678006010 tx=0x7f8678003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:41:00.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.798+0000 7f867d7fa640 1 -- 172.21.15.97:0/4039489497 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8678004450 con 0x7f8680105ea0 2024-04-23T10:41:00.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.798+0000 7f867d7fa640 1 -- 172.21.15.97:0/4039489497 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8678040070 con 0x7f8680105ea0 2024-04-23T10:41:00.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.798+0000 7f867d7fa640 1 -- 172.21.15.97:0/4039489497 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8678045c10 con 0x7f8680105ea0 2024-04-23T10:41:00.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.798+0000 7f86864ef640 1 -- 172.21.15.97:0/4039489497 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8680199020 con 0x7f8680105ea0 2024-04-23T10:41:00.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.798+0000 7f86864ef640 1 -- 172.21.15.97:0/4039489497 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86801a5320 con 0x7f8680105ea0 2024-04-23T10:41:00.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.800+0000 7f867d7fa640 1 -- 172.21.15.97:0/4039489497 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8678045470 con 0x7f8680105ea0 2024-04-23T10:41:00.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.801+0000 7f8689206740 1 Processor -- start 2024-04-23T10:41:00.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.801+0000 7f8689206740 1 -- start start 2024-04-23T10:41:00.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.801+0000 7f8689206740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8a5c7d490 0x55d8a5c7d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:00.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.801+0000 7f867d7fa640 1 --2- 172.21.15.97:0/4039489497 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f86680a3d20 0x7f86680a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:00.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.801+0000 7f8689206740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8a5c7ddb0 0x55d8a5c87f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:00.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.801+0000 7f8689206740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8a5c88480 0x55d8a5c8a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:00.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.801+0000 7f8689206740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d8a5be26d0 con 0x55d8a5c7d490 2024-04-23T10:41:00.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.801+0000 7f8689206740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d8a5b7f1a0 con 0x55d8a5c7ddb0 2024-04-23T10:41:00.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.801+0000 7f8689206740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d8a5bde690 con 0x55d8a5c88480 2024-04-23T10:41:00.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.801+0000 7f867f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8a5c7d490 0x55d8a5c7d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:00.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.801+0000 7f867ffff640 1 --2- 172.21.15.97:0/4039489497 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f86680a3d20 0x7f86680a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:00.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.801+0000 7f867f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8a5c7d490 0x55d8a5c7d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41832/0 (socket says 172.21.15.97:41832) 2024-04-23T10:41:00.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.801+0000 7f867f7fe640 1 -- 172.21.15.97:0/546249825 learned_addr learned my addr 172.21.15.97:0/546249825 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:41:00.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.801+0000 7f867d7fa640 1 -- 172.21.15.97:0/4039489497 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8484+0+0 (secure 0 0 0) 0x7f86780f19f0 con 0x7f8680105ea0 2024-04-23T10:41:00.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.801+0000 7f867f7fe640 1 --2- 172.21.15.97:0/546249825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8a5c7ddb0 0x55d8a5c87f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:00.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.802+0000 7f8684a8c640 1 --2- 172.21.15.97:0/546249825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8a5c88480 0x55d8a5c8a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:00.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.802+0000 7f867f7fe640 1 -- 172.21.15.97:0/546249825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8a5c88480 msgr2=0x55d8a5c8a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:00.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.802+0000 7f867f7fe640 1 --2- 172.21.15.97:0/546249825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8a5c88480 0x55d8a5c8a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:00.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.802+0000 7f867f7fe640 1 -- 172.21.15.97:0/546249825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8a5c7ddb0 msgr2=0x55d8a5c87f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:00.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.802+0000 7f867f7fe640 1 --2- 172.21.15.97:0/546249825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8a5c7ddb0 0x55d8a5c87f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:00.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.802+0000 7f867f7fe640 1 -- 172.21.15.97:0/546249825 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8a5b895b0 con 0x55d8a5c7d490 2024-04-23T10:41:00.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.802+0000 7f867ffff640 1 --2- 172.21.15.97:0/4039489497 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f86680a3d20 0x7f86680a61e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f867000bbe0 tx=0x7f867000b390 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:41:00.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.802+0000 7f867f7fe640 1 --2- 172.21.15.97:0/546249825 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8a5c7d490 0x55d8a5c7d870 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f8674009fd0 tx=0x7f8674009440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2455ec8778e05e3d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:41:00.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.802+0000 7f86667fc640 1 -- 172.21.15.97:0/546249825 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8674013680 con 0x55d8a5c7d490 2024-04-23T10:41:00.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.802+0000 7f86667fc640 1 -- 172.21.15.97:0/546249825 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8674013cd0 con 0x55d8a5c7d490 2024-04-23T10:41:00.808 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.802+0000 7f86667fc640 1 -- 172.21.15.97:0/546249825 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f867400dbe0 con 0x55d8a5c7d490 2024-04-23T10:41:00.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.803+0000 7f8689206740 1 -- 172.21.15.97:0/546249825 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8a5c7d490 msgr2=0x55d8a5c7d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:00.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.803+0000 7f8689206740 1 --2- 172.21.15.97:0/546249825 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8a5c7d490 0x55d8a5c7d870 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f8674009fd0 tx=0x7f8674009440 comp rx=0 tx=0).stop 2024-04-23T10:41:00.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.803+0000 7f8689206740 1 -- 172.21.15.97:0/546249825 shutdown_connections 2024-04-23T10:41:00.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.803+0000 7f8689206740 1 --2- 172.21.15.97:0/546249825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8a5c88480 0x55d8a5c8a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:00.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.803+0000 7f8689206740 1 --2- 172.21.15.97:0/546249825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8a5c7ddb0 0x55d8a5c87f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:00.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.803+0000 7f8689206740 1 --2- 172.21.15.97:0/546249825 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8a5c7d490 0x55d8a5c7d870 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:00.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.803+0000 7f8689206740 1 -- 172.21.15.97:0/546249825 >> 172.21.15.97:0/546249825 conn(0x55d8a5c797b0 msgr2=0x55d8a5c79b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:41:00.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.803+0000 7f8689206740 1 -- 172.21.15.97:0/546249825 shutdown_connections 2024-04-23T10:41:00.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.803+0000 7f8689206740 1 -- 172.21.15.97:0/546249825 wait complete. 2024-04-23T10:41:00.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.803+0000 7f8689206740 1 -- start start 2024-04-23T10:41:00.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.803+0000 7f8689206740 1 Processor -- start 2024-04-23T10:41:00.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.804+0000 7f8689206740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8a5c7d490 0x55d8a5c87e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:00.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.804+0000 7f8689206740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8a5c7ddb0 0x55d8a5c85160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:00.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.804+0000 7f8689206740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8a5c88480 0x55d8a5c85790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:00.809 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.804+0000 7f8689206740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d8a5c8b2b0 con 0x55d8a5c7ddb0 2024-04-23T10:41:00.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.804+0000 7f8689206740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d8a5b7cb20 con 0x55d8a5c88480 2024-04-23T10:41:00.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.804+0000 7f8689206740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d8a5c8b430 con 0x55d8a5c7d490 2024-04-23T10:41:00.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.804+0000 7f8689206740 10 client.0 _ll_register_callbacks cb 0x55d8a5b7c530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:41:00.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.804+0000 7f867f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8a5c7ddb0 0x55d8a5c85160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:00.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.804+0000 7f867f7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8a5c7ddb0 0x55d8a5c85160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41840/0 (socket says 172.21.15.97:41840) 2024-04-23T10:41:00.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.804+0000 7f867f7fe640 1 -- 172.21.15.97:0/2159353791 learned_addr learned my addr 172.21.15.97:0/2159353791 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:41:00.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.804+0000 7f8684a8c640 1 --2- 172.21.15.97:0/2159353791 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8a5c88480 0x55d8a5c85790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:00.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.804+0000 7f867ffff640 1 --2- 172.21.15.97:0/2159353791 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8a5c7d490 0x55d8a5c87e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:00.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.804+0000 7f867f7fe640 1 -- 172.21.15.97:0/2159353791 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8a5c7d490 msgr2=0x55d8a5c87e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:00.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.804+0000 7f867f7fe640 1 --2- 172.21.15.97:0/2159353791 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8a5c7d490 0x55d8a5c87e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:00.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.804+0000 7f867f7fe640 1 -- 172.21.15.97:0/2159353791 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8a5c88480 msgr2=0x55d8a5c85790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:00.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.804+0000 7f867f7fe640 1 --2- 172.21.15.97:0/2159353791 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8a5c88480 0x55d8a5c85790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:00.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.804+0000 7f867f7fe640 1 -- 172.21.15.97:0/2159353791 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8a58f4920 con 0x55d8a5c7ddb0 2024-04-23T10:41:00.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.804+0000 7f867f7fe640 1 --2- 172.21.15.97:0/2159353791 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8a5c7ddb0 0x55d8a5c85160 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f867400d840 tx=0x7f867400d5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2272b8442974b64a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:41:00.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.804+0000 7f865f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:41:00.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.804+0000 7f865f7fe640 1 -- 172.21.15.97:0/2159353791 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8674005460 con 0x55d8a5c7ddb0 2024-04-23T10:41:00.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.804+0000 7f865f7fe640 1 -- 172.21.15.97:0/2159353791 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8484+0+0 (secure 0 0 0) 0x7f8674013480 con 0x55d8a5c7ddb0 2024-04-23T10:41:00.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.804+0000 7f8689206740 1 -- 172.21.15.97:0/2159353791 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8674013cd0 con 0x55d8a5c7ddb0 2024-04-23T10:41:00.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.805+0000 7f865f7fe640 1 -- 172.21.15.97:0/2159353791 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f867402e050 con 0x55d8a5c7ddb0 2024-04-23T10:41:00.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.805+0000 7f8689206740 10 client.17607 fetch_fsmap learned FSMap version 26 2024-04-23T10:41:00.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.805+0000 7f8689206740 1 -- 172.21.15.97:0/2159353791 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d8a59c0d30 con 0x55d8a5c7ddb0 2024-04-23T10:41:00.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.805+0000 7f865f7fe640 1 -- 172.21.15.97:0/2159353791 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8674013cd0 con 0x55d8a5c7ddb0 2024-04-23T10:41:00.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.805+0000 7f865f7fe640 1 -- 172.21.15.97:0/2159353791 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f867400c070 con 0x55d8a5c7ddb0 2024-04-23T10:41:00.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.805+0000 7f8689206740 10 client.17607 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:41:00.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.805+0000 7f8689206740 10 client.17607 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:41:00.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.805+0000 7f8689206740 20 client.17607 populate_metadata read hostname 'smithi097' 2024-04-23T10:41:00.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.805+0000 7f8689206740 1 --2- 172.21.15.97:0/2159353791 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55d8a5c98580 0x55d8a5c9a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:00.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.805+0000 7f8689206740 4 client.17607 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:41:00.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.805+0000 7f8689206740 1 -- 172.21.15.97:0/2159353791 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8a5ba0500 con 0x55d8a5c98580 2024-04-23T10:41:00.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.805+0000 7f867ffff640 1 --2- 172.21.15.97:0/2159353791 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55d8a5c98580 0x55d8a5c9a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:00.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.806+0000 7f867ffff640 1 --2- 172.21.15.97:0/2159353791 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55d8a5c98580 0x55d8a5c9a970 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e349eea308801f65 server_cookie=319e116a3a9b3e7b in_seq=0 out_seq=0 2024-04-23T10:41:00.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.806+0000 7f865f7fe640 10 client.17607 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:41:00.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.806+0000 7f865f7fe640 1 -- 172.21.15.97:0/2159353791 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d8a5ba0500 con 0x55d8a5c98580 2024-04-23T10:41:00.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.806+0000 7f865f7fe640 10 client.17607 handle_command_reply: tid=0 2024-04-23T10:41:00.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.806+0000 7f8689206740 1 client.17607 shutdown 2024-04-23T10:41:00.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.807+0000 7f8689206740 1 -- 172.21.15.97:0/2159353791 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8a5c7ddb0 msgr2=0x55d8a5c85160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:00.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.807+0000 7f8689206740 1 --2- 172.21.15.97:0/2159353791 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8a5c7ddb0 0x55d8a5c85160 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f867400d840 tx=0x7f867400d5e0 comp rx=0 tx=0).stop 2024-04-23T10:41:00.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.807+0000 7f8689206740 1 -- 172.21.15.97:0/2159353791 shutdown_connections 2024-04-23T10:41:00.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.807+0000 7f8689206740 1 --2- 172.21.15.97:0/2159353791 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55d8a5c98580 0x55d8a5c9a970 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:00.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.807+0000 7f8689206740 1 --2- 172.21.15.97:0/2159353791 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8a5c88480 0x55d8a5c85790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:00.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.807+0000 7f865f7fe640 0 client.17607 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:41:00.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.807+0000 7f8689206740 1 --2- 172.21.15.97:0/2159353791 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8a5c7ddb0 0x55d8a5c85160 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:00.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.807+0000 7f8689206740 1 --2- 172.21.15.97:0/2159353791 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8a5c7d490 0x55d8a5c87e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:00.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.807+0000 7f8689206740 1 -- 172.21.15.97:0/2159353791 >> 172.21.15.97:0/2159353791 conn(0x55d8a5c797b0 msgr2=0x55d8a5c789d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:41:00.812 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.807+0000 7f8689206740 1 -- 172.21.15.97:0/2159353791 shutdown_connections 2024-04-23T10:41:00.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.807+0000 7f8689206740 1 -- 172.21.15.97:0/2159353791 wait complete. 2024-04-23T10:41:00.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.807+0000 7f8689206740 20 client.17607 trim_cache size 0 max 16384 2024-04-23T10:41:00.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.823+0000 7f8689206740 1 Processor -- start 2024-04-23T10:41:00.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.823+0000 7f8689206740 1 -- start start 2024-04-23T10:41:00.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.824+0000 7f8689206740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8a5c7d490 0x55d8a5c79150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:00.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.824+0000 7f8689206740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8a5c7ddb0 0x55d8a5ca0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:00.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.824+0000 7f8689206740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8a5c88480 0x55d8a5ca0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:00.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.824+0000 7f8689206740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f86801ad980 con 0x55d8a5c88480 2024-04-23T10:41:00.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.824+0000 7f8689206740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f86801af4c0 con 0x55d8a5c7ddb0 2024-04-23T10:41:00.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.824+0000 7f8689206740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f86801ae050 con 0x55d8a5c7d490 2024-04-23T10:41:00.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.824+0000 7f8684a8c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8a5c88480 0x55d8a5ca0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:00.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.824+0000 7f8684a8c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8a5c88480 0x55d8a5ca0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41844/0 (socket says 172.21.15.97:41844) 2024-04-23T10:41:00.830 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.824+0000 7f8684a8c640 1 -- 172.21.15.97:0/490190909 learned_addr learned my addr 172.21.15.97:0/490190909 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:41:00.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.824+0000 7f867ffff640 1 --2- 172.21.15.97:0/490190909 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8a5c7d490 0x55d8a5c79150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:00.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.824+0000 7f8684a8c640 1 -- 172.21.15.97:0/490190909 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8a5c7d490 msgr2=0x55d8a5c79150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:00.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.824+0000 7f8684a8c640 1 --2- 172.21.15.97:0/490190909 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8a5c7d490 0x55d8a5c79150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:00.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.824+0000 7f8684a8c640 1 -- 172.21.15.97:0/490190909 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8a5c7ddb0 msgr2=0x55d8a5ca0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:00.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.824+0000 7f8684a8c640 1 --2- 172.21.15.97:0/490190909 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8a5c7ddb0 0x55d8a5ca0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:00.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.824+0000 7f8684a8c640 1 -- 172.21.15.97:0/490190909 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8a5c7e950 con 0x55d8a5c88480 2024-04-23T10:41:00.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.824+0000 7f8684a8c640 1 --2- 172.21.15.97:0/490190909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8a5c88480 0x55d8a5ca0540 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f86780097b0 tx=0x7f86780a8040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4288c92bba7cb3eb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:41:00.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.824+0000 7f8665ffb640 1 -- 172.21.15.97:0/490190909 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8678045760 con 0x55d8a5c88480 2024-04-23T10:41:00.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.824+0000 7f8665ffb640 1 -- 172.21.15.97:0/490190909 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8678036040 con 0x55d8a5c88480 2024-04-23T10:41:00.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.824+0000 7f8665ffb640 1 -- 172.21.15.97:0/490190909 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86780bda60 con 0x55d8a5c88480 2024-04-23T10:41:00.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.825+0000 7f8689206740 1 -- 172.21.15.97:0/490190909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8a5c88480 msgr2=0x55d8a5ca0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:00.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.825+0000 7f8689206740 1 --2- 172.21.15.97:0/490190909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8a5c88480 0x55d8a5ca0540 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f86780097b0 tx=0x7f86780a8040 comp rx=0 tx=0).stop 2024-04-23T10:41:00.831 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.825+0000 7f8689206740 1 -- 172.21.15.97:0/490190909 shutdown_connections 2024-04-23T10:41:00.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.825+0000 7f8689206740 1 --2- 172.21.15.97:0/490190909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8a5c88480 0x55d8a5ca0540 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:00.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.825+0000 7f8689206740 1 --2- 172.21.15.97:0/490190909 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8a5c7ddb0 0x55d8a5ca0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:00.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.825+0000 7f8689206740 1 --2- 172.21.15.97:0/490190909 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8a5c7d490 0x55d8a5c79150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:00.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.825+0000 7f8689206740 1 -- 172.21.15.97:0/490190909 >> 172.21.15.97:0/490190909 conn(0x55d8a5c797b0 msgr2=0x55d8a5c7f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:41:00.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.825+0000 7f8689206740 1 -- 172.21.15.97:0/490190909 shutdown_connections 2024-04-23T10:41:00.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.825+0000 7f8689206740 1 -- 172.21.15.97:0/490190909 wait complete. 2024-04-23T10:41:00.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.825+0000 7f8689206740 1 -- start start 2024-04-23T10:41:00.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.825+0000 7f8689206740 1 Processor -- start 2024-04-23T10:41:00.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.825+0000 7f8689206740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8a5c7d490 0x55d8a5c8c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:00.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.826+0000 7f8689206740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8a5c7ddb0 0x55d8a5c8c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:00.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.826+0000 7f8689206740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8a5c88480 0x55d8a5c8ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:00.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.826+0000 7f8689206740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f86801aedf0 con 0x55d8a5c7d490 2024-04-23T10:41:00.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.826+0000 7f8689206740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f86801ae720 con 0x55d8a5c7ddb0 2024-04-23T10:41:00.832 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.826+0000 7f8689206740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f86801ab980 con 0x55d8a5c88480 2024-04-23T10:41:00.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.826+0000 7f8689206740 10 client.0 _ll_register_callbacks cb 0x7f8674015850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:41:00.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.826+0000 7f867ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8a5c7d490 0x55d8a5c8c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:00.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.826+0000 7f867ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8a5c7d490 0x55d8a5c8c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:41860/0 (socket says 172.21.15.97:41860) 2024-04-23T10:41:00.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.826+0000 7f867ffff640 1 -- 172.21.15.97:0/1952389120 learned_addr learned my addr 172.21.15.97:0/1952389120 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:41:00.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.826+0000 7f8684a8c640 1 --2- 172.21.15.97:0/1952389120 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8a5c88480 0x55d8a5c8ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:00.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.826+0000 7f867f7fe640 1 --2- 172.21.15.97:0/1952389120 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8a5c7ddb0 0x55d8a5c8c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:00.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.826+0000 7f867ffff640 1 -- 172.21.15.97:0/1952389120 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8a5c88480 msgr2=0x55d8a5c8ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:00.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.826+0000 7f867ffff640 1 --2- 172.21.15.97:0/1952389120 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8a5c88480 0x55d8a5c8ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:00.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.826+0000 7f867ffff640 1 -- 172.21.15.97:0/1952389120 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8a5c7ddb0 msgr2=0x55d8a5c8c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:00.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.826+0000 7f867ffff640 1 --2- 172.21.15.97:0/1952389120 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8a5c7ddb0 0x55d8a5c8c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:00.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.826+0000 7f867ffff640 1 -- 172.21.15.97:0/1952389120 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8a5cfda40 con 0x55d8a5c7d490 2024-04-23T10:41:00.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.826+0000 7f867f7fe640 1 --2- 172.21.15.97:0/1952389120 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8a5c7ddb0 0x55d8a5c8c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:41:00.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.826+0000 7f867ffff640 1 --2- 172.21.15.97:0/1952389120 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8a5c7d490 0x55d8a5c8c240 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x55d8a5c9afa0 tx=0x7f8670004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feb548c5e4a832a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:41:00.833 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.826+0000 7f865f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:41:00.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.826+0000 7f865f7fe640 1 -- 172.21.15.97:0/1952389120 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86700273b0 con 0x55d8a5c7d490 2024-04-23T10:41:00.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.826+0000 7f865f7fe640 1 -- 172.21.15.97:0/1952389120 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8484+0+0 (secure 0 0 0) 0x7f8670027550 con 0x55d8a5c7d490 2024-04-23T10:41:00.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.826+0000 7f8689206740 1 -- 172.21.15.97:0/1952389120 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8678036040 con 0x55d8a5c7d490 2024-04-23T10:41:00.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.826+0000 7f865f7fe640 1 -- 172.21.15.97:0/1952389120 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8670027750 con 0x55d8a5c7d490 2024-04-23T10:41:00.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.826+0000 7f8689206740 10 client.17613 fetch_fsmap learned FSMap version 26 2024-04-23T10:41:00.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.826+0000 7f8689206740 1 -- 172.21.15.97:0/1952389120 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d8a5cfedd0 con 0x55d8a5c7d490 2024-04-23T10:41:00.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.826+0000 7f865f7fe640 1 -- 172.21.15.97:0/1952389120 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8678036040 con 0x55d8a5c7d490 2024-04-23T10:41:00.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.827+0000 7f865f7fe640 1 -- 172.21.15.97:0/1952389120 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8670036020 con 0x55d8a5c7d490 2024-04-23T10:41:00.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.827+0000 7f8689206740 10 client.17613 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:41:00.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.827+0000 7f8689206740 10 client.17613 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:41:00.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.827+0000 7f8689206740 20 client.17613 populate_metadata read hostname 'smithi097' 2024-04-23T10:41:00.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.827+0000 7f8689206740 1 --2- 172.21.15.97:0/1952389120 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55d8a5cff530 0x55d8a5d01950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:00.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.827+0000 7f8689206740 4 client.17613 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "ca3de698", "release": true, "await": true}] 2024-04-23T10:41:00.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.827+0000 7f8689206740 1 -- 172.21.15.97:0/1952389120 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "quiesce db", "set_id": "ca3de698", "release": true, "await": true}) -- 0x55d8a5bf00f0 con 0x55d8a5cff530 2024-04-23T10:41:00.834 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.827+0000 7f867f7fe640 1 --2- 172.21.15.97:0/1952389120 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55d8a5cff530 0x55d8a5d01950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:00.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.827+0000 7f867f7fe640 1 --2- 172.21.15.97:0/1952389120 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55d8a5cff530 0x55d8a5d01950 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a6707371a7a3bc0 server_cookie=6c073e2a6ee96ed7 in_seq=0 out_seq=0 2024-04-23T10:41:00.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.827+0000 7f865f7fe640 10 client.17613 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:41:00.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.830+0000 7f865f7fe640 1 -- 172.21.15.97:0/1952389120 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55d8a5bf00f0 con 0x55d8a5cff530 2024-04-23T10:41:00.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.830+0000 7f865f7fe640 10 client.17613 handle_command_reply: tid=0 2024-04-23T10:41:00.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.830+0000 7f8689206740 1 client.17613 shutdown 2024-04-23T10:41:00.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.830+0000 7f8689206740 1 -- 172.21.15.97:0/1952389120 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8a5c7d490 msgr2=0x55d8a5c8c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:00.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.830+0000 7f8689206740 1 --2- 172.21.15.97:0/1952389120 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8a5c7d490 0x55d8a5c8c240 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x55d8a5c9afa0 tx=0x7f8670004730 comp rx=0 tx=0).stop 2024-04-23T10:41:00.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.830+0000 7f8689206740 1 -- 172.21.15.97:0/1952389120 shutdown_connections 2024-04-23T10:41:00.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.830+0000 7f8689206740 1 --2- 172.21.15.97:0/1952389120 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55d8a5cff530 0x55d8a5d01950 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:00.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.830+0000 7f8689206740 1 --2- 172.21.15.97:0/1952389120 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d8a5c88480 0x55d8a5c8ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:00.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.830+0000 7f8689206740 1 --2- 172.21.15.97:0/1952389120 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8a5c7ddb0 0x55d8a5c8c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:00.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.830+0000 7f865f7fe640 0 client.17613 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:41:00.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.830+0000 7f8689206740 1 --2- 172.21.15.97:0/1952389120 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d8a5c7d490 0x55d8a5c8c240 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:00.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.830+0000 7f8689206740 1 -- 172.21.15.97:0/1952389120 >> 172.21.15.97:0/1952389120 conn(0x55d8a5c797b0 msgr2=0x55d8a5c939f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:41:00.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.830+0000 7f8689206740 1 -- 172.21.15.97:0/1952389120 shutdown_connections 2024-04-23T10:41:00.835 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.830+0000 7f8689206740 1 -- 172.21.15.97:0/1952389120 wait complete. 2024-04-23T10:41:00.836 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.830+0000 7f8689206740 20 client.17613 trim_cache size 0 max 16384 2024-04-23T10:41:00.836 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-23T10:41:00.836 INFO:teuthology.orchestra.run.smithi097.stdout: "epoch": 26, 2024-04-23T10:41:00.836 INFO:teuthology.orchestra.run.smithi097.stdout: "leader": 24488, 2024-04-23T10:41:00.836 INFO:teuthology.orchestra.run.smithi097.stdout: "set_version": 60, 2024-04-23T10:41:00.836 INFO:teuthology.orchestra.run.smithi097.stdout: "sets": { 2024-04-23T10:41:00.836 INFO:teuthology.orchestra.run.smithi097.stdout: "ca3de698": { 2024-04-23T10:41:00.836 INFO:teuthology.orchestra.run.smithi097.stdout: "version": 60, 2024-04-23T10:41:00.836 INFO:teuthology.orchestra.run.smithi097.stdout: "age_ref": 0.0, 2024-04-23T10:41:00.836 INFO:teuthology.orchestra.run.smithi097.stdout: "state": { 2024-04-23T10:41:00.836 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RELEASED", 2024-04-23T10:41:00.836 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0 2024-04-23T10:41:00.837 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:41:00.837 INFO:teuthology.orchestra.run.smithi097.stdout: "timeout": 60.0, 2024-04-23T10:41:00.837 INFO:teuthology.orchestra.run.smithi097.stdout: "expiration": 70.0, 2024-04-23T10:41:00.837 INFO:teuthology.orchestra.run.smithi097.stdout: "members": { 2024-04-23T10:41:00.837 INFO:teuthology.orchestra.run.smithi097.stdout: "file:/": { 2024-04-23T10:41:00.837 INFO:teuthology.orchestra.run.smithi097.stdout: "excluded": false, 2024-04-23T10:41:00.837 INFO:teuthology.orchestra.run.smithi097.stdout: "state": { 2024-04-23T10:41:00.837 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RELEASED", 2024-04-23T10:41:00.837 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0 2024-04-23T10:41:00.837 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:41:00.837 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:41:00.837 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:41:00.837 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:41:00.837 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:41:00.837 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-23T10:41:00.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.830+0000 7f86864ef640 1 -- 172.21.15.97:0/4039489497 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f86680a3d20 msgr2=0x7f86680a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:00.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.830+0000 7f86864ef640 1 --2- 172.21.15.97:0/4039489497 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f86680a3d20 0x7f86680a61e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f867000bbe0 tx=0x7f867000b390 comp rx=0 tx=0).stop 2024-04-23T10:41:00.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.830+0000 7f86864ef640 1 -- 172.21.15.97:0/4039489497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8680105ea0 msgr2=0x7f8680198610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:00.838 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.830+0000 7f86864ef640 1 --2- 172.21.15.97:0/4039489497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8680105ea0 0x7f8680198610 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f8678006010 tx=0x7f8678003c20 comp rx=0 tx=0).stop 2024-04-23T10:41:00.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.831+0000 7f86864ef640 1 -- 172.21.15.97:0/4039489497 shutdown_connections 2024-04-23T10:41:00.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.831+0000 7f86864ef640 1 --2- 172.21.15.97:0/4039489497 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f86680a3d20 0x7f86680a61e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:00.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.831+0000 7f86864ef640 1 --2- 172.21.15.97:0/4039489497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8680105ea0 0x7f8680198610 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:00.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.831+0000 7f86864ef640 1 --2- 172.21.15.97:0/4039489497 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8680101ad0 0x7f868019e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:00.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.831+0000 7f86864ef640 1 --2- 172.21.15.97:0/4039489497 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8680101130 0x7f868019e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:00.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.831+0000 7f86864ef640 1 -- 172.21.15.97:0/4039489497 >> 172.21.15.97:0/4039489497 conn(0x7f8680077e30 msgr2=0x7f86800fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:41:00.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.831+0000 7f86864ef640 1 -- 172.21.15.97:0/4039489497 shutdown_connections 2024-04-23T10:41:00.839 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:00.831+0000 7f86864ef640 1 -- 172.21.15.97:0/4039489497 wait complete. 2024-04-23T10:41:00.840 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'ca3de698', seconds elapsed: 11.0 2024-04-23T10:41:01.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:41:00 smithi097 ceph-mon[29197]: pgmap v463: 97 pgs: 97 active+clean; 2.2 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 KiB/s wr, 13 op/s 2024-04-23T10:41:01.110 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:41:00 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/147068387' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:41:01.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:41:00 smithi183 ceph-mon[32317]: pgmap v463: 97 pgs: 97 active+clean; 2.2 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 KiB/s wr, 13 op/s 2024-04-23T10:41:01.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:41:00 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/147068387' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:41:01.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:41:00 smithi196 ceph-mon[32919]: pgmap v463: 97 pgs: 97 active+clean; 2.2 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 KiB/s wr, 13 op/s 2024-04-23T10:41:01.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:41:00 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/147068387' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:41:03.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:41:02 smithi183 ceph-mon[32317]: pgmap v464: 97 pgs: 97 active+clean; 2.2 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 KiB/s wr, 13 op/s 2024-04-23T10:41:03.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:41:02 smithi183 ceph-mon[32317]: osdmap e106: 12 total, 12 up, 12 in 2024-04-23T10:41:03.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:41:02 smithi196 ceph-mon[32919]: pgmap v464: 97 pgs: 97 active+clean; 2.2 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 KiB/s wr, 13 op/s 2024-04-23T10:41:03.342 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:41:02 smithi196 ceph-mon[32919]: osdmap e106: 12 total, 12 up, 12 in 2024-04-23T10:41:03.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:41:02 smithi097 ceph-mon[29197]: pgmap v464: 97 pgs: 97 active+clean; 2.2 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 KiB/s wr, 13 op/s 2024-04-23T10:41:03.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:41:02 smithi097 ceph-mon[29197]: osdmap e106: 12 total, 12 up, 12 in 2024-04-23T10:41:05.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:41:04 smithi183 ceph-mon[32317]: pgmap v466: 97 pgs: 97 active+clean; 2.3 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 71 KiB/s wr, 35 op/s 2024-04-23T10:41:05.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:41:04 smithi196 ceph-mon[32919]: pgmap v466: 97 pgs: 97 active+clean; 2.3 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 71 KiB/s wr, 35 op/s 2024-04-23T10:41:05.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:41:04 smithi097 ceph-mon[29197]: pgmap v466: 97 pgs: 97 active+clean; 2.3 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 71 KiB/s wr, 35 op/s 2024-04-23T10:41:07.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:41:06 smithi183 ceph-mon[32317]: pgmap v467: 97 pgs: 97 active+clean; 2.3 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 71 KiB/s wr, 35 op/s 2024-04-23T10:41:07.319 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.316+0000 7f3f9b4fd640 1 -- 172.21.15.97:0/50135456 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f94101aa0 msgr2=0x7f3f94105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:07.319 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.316+0000 7f3f9b4fd640 1 --2- 172.21.15.97:0/50135456 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f94101aa0 0x7f3f94105a90 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f3f84009900 tx=0x7f3f8402a410 comp rx=0 tx=0).stop 2024-04-23T10:41:07.319 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.316+0000 7f3f9b4fd640 1 -- 172.21.15.97:0/50135456 shutdown_connections 2024-04-23T10:41:07.319 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.316+0000 7f3f9b4fd640 1 --2- 172.21.15.97:0/50135456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f94106170 0x7f3f9410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:07.319 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.316+0000 7f3f9b4fd640 1 --2- 172.21.15.97:0/50135456 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f94101aa0 0x7f3f94105a90 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:07.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.316+0000 7f3f9b4fd640 1 --2- 172.21.15.97:0/50135456 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f941010f0 0x7f3f941014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:07.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.316+0000 7f3f9b4fd640 1 -- 172.21.15.97:0/50135456 >> 172.21.15.97:0/50135456 conn(0x7f3f94077fd0 msgr2=0x7f3f940783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:41:07.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.316+0000 7f3f9b4fd640 1 -- 172.21.15.97:0/50135456 shutdown_connections 2024-04-23T10:41:07.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.316+0000 7f3f9b4fd640 1 -- 172.21.15.97:0/50135456 wait complete. 2024-04-23T10:41:07.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.316+0000 7f3f9b4fd640 1 Processor -- start 2024-04-23T10:41:07.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.317+0000 7f3f9b4fd640 1 -- start start 2024-04-23T10:41:07.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.317+0000 7f3f9b4fd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f941010f0 0x7f3f94198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:07.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.317+0000 7f3f9b4fd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f94106170 0x7f3f94198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:07.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.317+0000 7f3f9b4fd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f9419ce90 0x7f3f9419d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:07.320 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.317+0000 7f3f9b4fd640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3f941110b0 con 0x7f3f94106170 2024-04-23T10:41:07.321 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.317+0000 7f3f9b4fd640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3f94110f30 con 0x7f3f941010f0 2024-04-23T10:41:07.321 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.317+0000 7f3f9b4fd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3f94111230 con 0x7f3f9419ce90 2024-04-23T10:41:07.321 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.317+0000 7f3f98a98640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f94106170 0x7f3f94198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:07.321 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.317+0000 7f3f98a98640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f94106170 0x7f3f94198bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34560/0 (socket says 172.21.15.97:34560) 2024-04-23T10:41:07.321 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.317+0000 7f3f98a98640 1 -- 172.21.15.97:0/3974225544 learned_addr learned my addr 172.21.15.97:0/3974225544 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:41:07.321 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.317+0000 7f3f99299640 1 --2- 172.21.15.97:0/3974225544 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f941010f0 0x7f3f94198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:07.321 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.317+0000 7f3f99a9a640 1 --2- 172.21.15.97:0/3974225544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f9419ce90 0x7f3f9419d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:07.321 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.317+0000 7f3f98a98640 1 -- 172.21.15.97:0/3974225544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f9419ce90 msgr2=0x7f3f9419d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:07.321 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.317+0000 7f3f98a98640 1 --2- 172.21.15.97:0/3974225544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f9419ce90 0x7f3f9419d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:07.321 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.317+0000 7f3f98a98640 1 -- 172.21.15.97:0/3974225544 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f941010f0 msgr2=0x7f3f94198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:07.322 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.317+0000 7f3f98a98640 1 --2- 172.21.15.97:0/3974225544 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f941010f0 0x7f3f94198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:07.322 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.317+0000 7f3f98a98640 1 -- 172.21.15.97:0/3974225544 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f9419d980 con 0x7f3f94106170 2024-04-23T10:41:07.322 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.317+0000 7f3f98a98640 1 --2- 172.21.15.97:0/3974225544 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f94106170 0x7f3f94198bb0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f3f84032fd0 tx=0x7f3f84004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:41:07.322 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.318+0000 7f3f8a7fc640 1 -- 172.21.15.97:0/3974225544 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8403f030 con 0x7f3f94106170 2024-04-23T10:41:07.322 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.318+0000 7f3f8a7fc640 1 -- 172.21.15.97:0/3974225544 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f84004490 con 0x7f3f94106170 2024-04-23T10:41:07.322 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.318+0000 7f3f9b4fd640 1 -- 172.21.15.97:0/3974225544 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f9419dc10 con 0x7f3f94106170 2024-04-23T10:41:07.322 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.318+0000 7f3f8a7fc640 1 -- 172.21.15.97:0/3974225544 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f84035070 con 0x7f3f94106170 2024-04-23T10:41:07.322 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.318+0000 7f3f9b4fd640 1 -- 172.21.15.97:0/3974225544 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f941a5430 con 0x7f3f94106170 2024-04-23T10:41:07.330 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.320+0000 7f3f8a7fc640 1 -- 172.21.15.97:0/3974225544 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3f84042030 con 0x7f3f94106170 2024-04-23T10:41:07.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.320+0000 7f3f9e20f740 1 Processor -- start 2024-04-23T10:41:07.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.320+0000 7f3f9e20f740 1 -- start start 2024-04-23T10:41:07.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.321+0000 7f3f8a7fc640 1 --2- 172.21.15.97:0/3974225544 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f3f800a3e40 0x7f3f800a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:07.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.321+0000 7f3f9e20f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646c3641620 0x5646c3641a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:07.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.321+0000 7f3f9e20f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646c3641f40 0x5646c364c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:07.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.321+0000 7f3f9e20f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646c364c800 0x5646c364ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:07.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.321+0000 7f3f9e20f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5646c35a6970 con 0x5646c3641620 2024-04-23T10:41:07.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.321+0000 7f3f9e20f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5646c35434a0 con 0x5646c3641f40 2024-04-23T10:41:07.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.321+0000 7f3f9e20f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5646c35a2930 con 0x5646c364c800 2024-04-23T10:41:07.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.321+0000 7f3f99a9a640 1 --2- 172.21.15.97:0/3974225544 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f3f800a3e40 0x7f3f800a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:07.331 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.321+0000 7f3f99299640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646c3641620 0x5646c3641a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:07.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.321+0000 7f3f8a7fc640 1 -- 172.21.15.97:0/3974225544 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8528+0+0 (secure 0 0 0) 0x7f3f840f0bb0 con 0x7f3f94106170 2024-04-23T10:41:07.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.321+0000 7f3f99299640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646c3641620 0x5646c3641a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34568/0 (socket says 172.21.15.97:34568) 2024-04-23T10:41:07.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.321+0000 7f3f99299640 1 -- 172.21.15.97:0/351889744 learned_addr learned my addr 172.21.15.97:0/351889744 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:41:07.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.321+0000 7f3f98a98640 1 --2- 172.21.15.97:0/351889744 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646c3641f40 0x5646c364c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:07.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.321+0000 7f3f99a9a640 1 --2- 172.21.15.97:0/351889744 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646c364c800 0x5646c364ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:07.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.321+0000 7f3f99299640 1 -- 172.21.15.97:0/351889744 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646c364c800 msgr2=0x5646c364ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:07.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.321+0000 7f3f99299640 1 --2- 172.21.15.97:0/351889744 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646c364c800 0x5646c364ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:07.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.321+0000 7f3f99299640 1 -- 172.21.15.97:0/351889744 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646c3641f40 msgr2=0x5646c364c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:07.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.321+0000 7f3f99299640 1 --2- 172.21.15.97:0/351889744 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646c3641f40 0x5646c364c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:07.332 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.321+0000 7f3f99299640 1 -- 172.21.15.97:0/351889744 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646c32b9210 con 0x5646c3641620 2024-04-23T10:41:07.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.321+0000 7f3f99a9a640 1 --2- 172.21.15.97:0/3974225544 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f3f800a3e40 0x7f3f800a6300 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3f90005fd0 tx=0x7f3f90009ad0 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:41:07.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.321+0000 7f3f99299640 1 --2- 172.21.15.97:0/351889744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646c3641620 0x5646c3641a00 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c00ad00 tx=0x7f3f8c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=465a4ca78b552dae server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:41:07.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.321+0000 7f3f7f7fe640 1 -- 172.21.15.97:0/351889744 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c01d070 con 0x5646c3641620 2024-04-23T10:41:07.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.322+0000 7f3f7f7fe640 1 -- 172.21.15.97:0/351889744 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f8c018af0 con 0x5646c3641620 2024-04-23T10:41:07.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.322+0000 7f3f7f7fe640 1 -- 172.21.15.97:0/351889744 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c010a40 con 0x5646c3641620 2024-04-23T10:41:07.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.322+0000 7f3f9e20f740 1 -- 172.21.15.97:0/351889744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646c3641620 msgr2=0x5646c3641a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:07.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.322+0000 7f3f9e20f740 1 --2- 172.21.15.97:0/351889744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646c3641620 0x5646c3641a00 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c00ad00 tx=0x7f3f8c00f790 comp rx=0 tx=0).stop 2024-04-23T10:41:07.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.322+0000 7f3f9e20f740 1 -- 172.21.15.97:0/351889744 shutdown_connections 2024-04-23T10:41:07.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.322+0000 7f3f9e20f740 1 --2- 172.21.15.97:0/351889744 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646c364c800 0x5646c364ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:07.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.322+0000 7f3f9e20f740 1 --2- 172.21.15.97:0/351889744 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646c3641f40 0x5646c364c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:07.333 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.322+0000 7f3f9e20f740 1 --2- 172.21.15.97:0/351889744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646c3641620 0x5646c3641a00 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:07.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.322+0000 7f3f9e20f740 1 -- 172.21.15.97:0/351889744 >> 172.21.15.97:0/351889744 conn(0x5646c363db30 msgr2=0x5646c363df10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:41:07.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.322+0000 7f3f9e20f740 1 -- 172.21.15.97:0/351889744 shutdown_connections 2024-04-23T10:41:07.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.322+0000 7f3f9e20f740 1 -- 172.21.15.97:0/351889744 wait complete. 2024-04-23T10:41:07.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.323+0000 7f3f9e20f740 1 -- start start 2024-04-23T10:41:07.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.323+0000 7f3f9e20f740 1 Processor -- start 2024-04-23T10:41:07.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.323+0000 7f3f9e20f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646c3641620 0x5646c364c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:07.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.323+0000 7f3f9e20f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646c3641f40 0x5646c36493c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:07.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.323+0000 7f3f9e20f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646c364c800 0x5646c3649900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:07.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.323+0000 7f3f9e20f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5646c364f430 con 0x5646c3641620 2024-04-23T10:41:07.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.323+0000 7f3f9e20f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5646c35410b0 con 0x5646c364c800 2024-04-23T10:41:07.334 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.323+0000 7f3f9e20f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5646c364f5b0 con 0x5646c3641f40 2024-04-23T10:41:07.335 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.323+0000 7f3f9e20f740 10 client.0 _ll_register_callbacks cb 0x5646c352ed30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:41:07.335 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.323+0000 7f3f99299640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646c3641620 0x5646c364c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:07.335 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.323+0000 7f3f99299640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646c3641620 0x5646c364c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34576/0 (socket says 172.21.15.97:34576) 2024-04-23T10:41:07.335 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.323+0000 7f3f99299640 1 -- 172.21.15.97:0/1548487524 learned_addr learned my addr 172.21.15.97:0/1548487524 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:41:07.335 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.323+0000 7f3f98a98640 1 --2- 172.21.15.97:0/1548487524 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646c3641f40 0x5646c36493c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:07.335 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.323+0000 7f3f99a9a640 1 --2- 172.21.15.97:0/1548487524 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646c364c800 0x5646c3649900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:07.335 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.323+0000 7f3f99299640 1 -- 172.21.15.97:0/1548487524 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646c3641f40 msgr2=0x5646c36493c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:07.335 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.323+0000 7f3f99299640 1 --2- 172.21.15.97:0/1548487524 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646c3641f40 0x5646c36493c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:07.335 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.323+0000 7f3f99299640 1 -- 172.21.15.97:0/1548487524 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646c364c800 msgr2=0x5646c3649900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:07.335 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.323+0000 7f3f99299640 1 --2- 172.21.15.97:0/1548487524 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646c364c800 0x5646c3649900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:07.336 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.323+0000 7f3f99299640 1 -- 172.21.15.97:0/1548487524 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646c3385620 con 0x5646c3641620 2024-04-23T10:41:07.336 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.323+0000 7f3f99a9a640 1 --2- 172.21.15.97:0/1548487524 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646c364c800 0x5646c3649900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:41:07.336 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.324+0000 7f3f99299640 1 --2- 172.21.15.97:0/1548487524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646c3641620 0x5646c364c0d0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c00a5c0 tx=0x7f3f8c00a860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8134c0a857d7f8a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:41:07.336 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.324+0000 7f3f7cff9640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:41:07.336 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.324+0000 7f3f7cff9640 1 -- 172.21.15.97:0/1548487524 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c026020 con 0x5646c3641620 2024-04-23T10:41:07.336 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.324+0000 7f3f7cff9640 1 -- 172.21.15.97:0/1548487524 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8528+0+0 (secure 0 0 0) 0x7f3f8c005240 con 0x5646c3641620 2024-04-23T10:41:07.336 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.324+0000 7f3f9e20f740 1 -- 172.21.15.97:0/1548487524 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f8c018af0 con 0x5646c3641620 2024-04-23T10:41:07.336 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.324+0000 7f3f7cff9640 1 -- 172.21.15.97:0/1548487524 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3f8c031050 con 0x5646c3641620 2024-04-23T10:41:07.336 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.324+0000 7f3f9e20f740 10 client.17625 fetch_fsmap learned FSMap version 26 2024-04-23T10:41:07.336 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.324+0000 7f3f9e20f740 1 -- 172.21.15.97:0/1548487524 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5646c365b2d0 con 0x5646c3641620 2024-04-23T10:41:07.336 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.324+0000 7f3f7cff9640 1 -- 172.21.15.97:0/1548487524 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f8c018af0 con 0x5646c3641620 2024-04-23T10:41:07.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.324+0000 7f3f7cff9640 1 -- 172.21.15.97:0/1548487524 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3f8c0046c0 con 0x5646c3641620 2024-04-23T10:41:07.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.324+0000 7f3f9e20f740 10 client.17625 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:41:07.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.324+0000 7f3f9e20f740 10 client.17625 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:41:07.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.324+0000 7f3f9e20f740 20 client.17625 populate_metadata read hostname 'smithi097' 2024-04-23T10:41:07.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.324+0000 7f3f9e20f740 1 --2- 172.21.15.97:0/1548487524 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5646c365c6d0 0x5646c365eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:07.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.324+0000 7f3f9e20f740 4 client.17625 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:41:07.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.324+0000 7f3f9e20f740 1 -- 172.21.15.97:0/1548487524 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646c35b4390 con 0x5646c365c6d0 2024-04-23T10:41:07.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.325+0000 7f3f98a98640 1 --2- 172.21.15.97:0/1548487524 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5646c365c6d0 0x5646c365eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:07.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.325+0000 7f3f98a98640 1 --2- 172.21.15.97:0/1548487524 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5646c365c6d0 0x5646c365eac0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d42bc761bbc8404 server_cookie=d3b562fc84193c23 in_seq=0 out_seq=0 2024-04-23T10:41:07.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.325+0000 7f3f7cff9640 10 client.17625 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:41:07.337 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.326+0000 7f3f7cff9640 1 -- 172.21.15.97:0/1548487524 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5646c35b4390 con 0x5646c365c6d0 2024-04-23T10:41:07.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.326+0000 7f3f7cff9640 10 client.17625 handle_command_reply: tid=0 2024-04-23T10:41:07.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.326+0000 7f3f9e20f740 1 client.17625 shutdown 2024-04-23T10:41:07.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.326+0000 7f3f9e20f740 1 -- 172.21.15.97:0/1548487524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646c3641620 msgr2=0x5646c364c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:07.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.326+0000 7f3f9e20f740 1 --2- 172.21.15.97:0/1548487524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646c3641620 0x5646c364c0d0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c00a5c0 tx=0x7f3f8c00a860 comp rx=0 tx=0).stop 2024-04-23T10:41:07.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.326+0000 7f3f9e20f740 1 -- 172.21.15.97:0/1548487524 shutdown_connections 2024-04-23T10:41:07.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.326+0000 7f3f9e20f740 1 --2- 172.21.15.97:0/1548487524 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5646c365c6d0 0x5646c365eac0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:07.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.326+0000 7f3f9e20f740 1 --2- 172.21.15.97:0/1548487524 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646c364c800 0x5646c3649900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:07.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.326+0000 7f3f9e20f740 1 --2- 172.21.15.97:0/1548487524 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646c3641f40 0x5646c36493c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:07.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.326+0000 7f3f9e20f740 1 --2- 172.21.15.97:0/1548487524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646c3641620 0x5646c364c0d0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:07.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.326+0000 7f3f7cff9640 0 client.17625 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:41:07.338 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.326+0000 7f3f9e20f740 1 -- 172.21.15.97:0/1548487524 >> 172.21.15.97:0/1548487524 conn(0x5646c363db30 msgr2=0x5646c36438b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:41:07.339 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.326+0000 7f3f9e20f740 1 -- 172.21.15.97:0/1548487524 shutdown_connections 2024-04-23T10:41:07.339 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.326+0000 7f3f9e20f740 1 -- 172.21.15.97:0/1548487524 wait complete. 2024-04-23T10:41:07.339 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.326+0000 7f3f9e20f740 20 client.17625 trim_cache size 0 max 16384 2024-04-23T10:41:07.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.341+0000 7f3f9e20f740 1 Processor -- start 2024-04-23T10:41:07.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.341+0000 7f3f9e20f740 1 -- start start 2024-04-23T10:41:07.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.341+0000 7f3f9e20f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646c3641620 0x5646c3644c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:07.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.341+0000 7f3f9e20f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646c3641f40 0x5646c36582f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:07.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.341+0000 7f3f9e20f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646c364c800 0x5646c364bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:07.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.341+0000 7f3f9e20f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3f941adaa0 con 0x5646c3641620 2024-04-23T10:41:07.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.341+0000 7f3f9e20f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3f941af5e0 con 0x5646c364c800 2024-04-23T10:41:07.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.341+0000 7f3f9e20f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3f941ae170 con 0x5646c3641f40 2024-04-23T10:41:07.347 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.341+0000 7f3f99299640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646c3641620 0x5646c3644c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:07.348 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.341+0000 7f3f99299640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646c3641620 0x5646c3644c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34592/0 (socket says 172.21.15.97:34592) 2024-04-23T10:41:07.348 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.341+0000 7f3f99299640 1 -- 172.21.15.97:0/2572195767 learned_addr learned my addr 172.21.15.97:0/2572195767 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:41:07.348 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.341+0000 7f3f99299640 1 -- 172.21.15.97:0/2572195767 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646c3641f40 msgr2=0x5646c36582f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:07.348 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.341+0000 7f3f99299640 1 --2- 172.21.15.97:0/2572195767 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646c3641f40 0x5646c36582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:07.348 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.341+0000 7f3f99299640 1 -- 172.21.15.97:0/2572195767 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646c364c800 msgr2=0x5646c364bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:07.348 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.341+0000 7f3f99299640 1 --2- 172.21.15.97:0/2572195767 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646c364c800 0x5646c364bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:07.348 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.342+0000 7f3f99299640 1 -- 172.21.15.97:0/2572195767 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646c3647910 con 0x5646c3641620 2024-04-23T10:41:07.348 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.342+0000 7f3f99299640 1 --2- 172.21.15.97:0/2572195767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646c3641620 0x5646c3644c10 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c004c80 tx=0x7f3f8c00fc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f980670ad3e7432 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:41:07.348 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.342+0000 7f3f7effd640 1 -- 172.21.15.97:0/2572195767 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c03a070 con 0x5646c3641620 2024-04-23T10:41:07.348 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.342+0000 7f3f7effd640 1 -- 172.21.15.97:0/2572195767 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f8c01e040 con 0x5646c3641620 2024-04-23T10:41:07.349 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.342+0000 7f3f7effd640 1 -- 172.21.15.97:0/2572195767 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c026070 con 0x5646c3641620 2024-04-23T10:41:07.349 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.342+0000 7f3f9e20f740 1 -- 172.21.15.97:0/2572195767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646c3641620 msgr2=0x5646c3644c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:07.349 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.342+0000 7f3f9e20f740 1 --2- 172.21.15.97:0/2572195767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646c3641620 0x5646c3644c10 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c004c80 tx=0x7f3f8c00fc90 comp rx=0 tx=0).stop 2024-04-23T10:41:07.349 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.342+0000 7f3f9e20f740 1 -- 172.21.15.97:0/2572195767 shutdown_connections 2024-04-23T10:41:07.349 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.342+0000 7f3f9e20f740 1 --2- 172.21.15.97:0/2572195767 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646c364c800 0x5646c364bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:07.349 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.342+0000 7f3f9e20f740 1 --2- 172.21.15.97:0/2572195767 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646c3641f40 0x5646c36582f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:07.349 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.342+0000 7f3f9e20f740 1 --2- 172.21.15.97:0/2572195767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646c3641620 0x5646c3644c10 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:07.349 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.342+0000 7f3f9e20f740 1 -- 172.21.15.97:0/2572195767 >> 172.21.15.97:0/2572195767 conn(0x5646c363db30 msgr2=0x5646c3643860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:41:07.349 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.342+0000 7f3f9e20f740 1 -- 172.21.15.97:0/2572195767 shutdown_connections 2024-04-23T10:41:07.349 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.342+0000 7f3f9e20f740 1 -- 172.21.15.97:0/2572195767 wait complete. 2024-04-23T10:41:07.349 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.343+0000 7f3f9e20f740 1 -- start start 2024-04-23T10:41:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.343+0000 7f3f9e20f740 1 Processor -- start 2024-04-23T10:41:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.343+0000 7f3f9e20f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646c3641620 0x5646c3654b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.343+0000 7f3f9e20f740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646c3641f40 0x5646c364fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.343+0000 7f3f9e20f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646c364c800 0x5646c36503e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.343+0000 7f3f9e20f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3f941aef10 con 0x5646c3641620 2024-04-23T10:41:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.343+0000 7f3f9e20f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3f941ae840 con 0x5646c364c800 2024-04-23T10:41:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.343+0000 7f3f9e20f740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3f941abaa0 con 0x5646c3641f40 2024-04-23T10:41:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.343+0000 7f3f9e20f740 10 client.0 _ll_register_callbacks cb 0x7f3f8c00ab80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:41:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.343+0000 7f3f99299640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646c3641620 0x5646c3654b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.343+0000 7f3f99299640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646c3641620 0x5646c3654b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34596/0 (socket says 172.21.15.97:34596) 2024-04-23T10:41:07.351 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.343+0000 7f3f99299640 1 -- 172.21.15.97:0/1635362433 learned_addr learned my addr 172.21.15.97:0/1635362433 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:41:07.351 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.343+0000 7f3f98a98640 1 --2- 172.21.15.97:0/1635362433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646c3641f40 0x5646c364fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:07.351 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.343+0000 7f3f99a9a640 1 --2- 172.21.15.97:0/1635362433 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646c364c800 0x5646c36503e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:07.351 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.344+0000 7f3f99299640 1 -- 172.21.15.97:0/1635362433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646c3641f40 msgr2=0x5646c364fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:07.351 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.344+0000 7f3f99299640 1 --2- 172.21.15.97:0/1635362433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646c3641f40 0x5646c364fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:07.351 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.344+0000 7f3f99299640 1 -- 172.21.15.97:0/1635362433 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646c364c800 msgr2=0x5646c36503e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:07.351 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.344+0000 7f3f99299640 1 --2- 172.21.15.97:0/1635362433 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646c364c800 0x5646c36503e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:07.351 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.344+0000 7f3f99299640 1 -- 172.21.15.97:0/1635362433 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646c3650be0 con 0x5646c3641620 2024-04-23T10:41:07.351 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.344+0000 7f3f99a9a640 1 --2- 172.21.15.97:0/1635362433 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646c364c800 0x5646c36503e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:41:07.351 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.344+0000 7f3f99299640 1 --2- 172.21.15.97:0/1635362433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646c3641620 0x5646c3654b00 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c00aac0 tx=0x7f3f8c004890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90dac646d2b605a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:41:07.352 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.344+0000 7f3f7cff9640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:41:07.352 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.344+0000 7f3f7cff9640 1 -- 172.21.15.97:0/1635362433 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c03a030 con 0x5646c3641620 2024-04-23T10:41:07.352 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.344+0000 7f3f7cff9640 1 -- 172.21.15.97:0/1635362433 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8528+0+0 (secure 0 0 0) 0x7f3f8c018aa0 con 0x5646c3641620 2024-04-23T10:41:07.352 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.344+0000 7f3f9e20f740 1 -- 172.21.15.97:0/1635362433 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f8c01e040 con 0x5646c3641620 2024-04-23T10:41:07.352 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.344+0000 7f3f7cff9640 1 -- 172.21.15.97:0/1635362433 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3f8c026040 con 0x5646c3641620 2024-04-23T10:41:07.352 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.344+0000 7f3f9e20f740 10 client.17631 fetch_fsmap learned FSMap version 26 2024-04-23T10:41:07.352 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.344+0000 7f3f9e20f740 1 -- 172.21.15.97:0/1635362433 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5646c36c2880 con 0x5646c3641620 2024-04-23T10:41:07.352 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.344+0000 7f3f7cff9640 1 -- 172.21.15.97:0/1635362433 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f8c01e040 con 0x5646c3641620 2024-04-23T10:41:07.352 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.344+0000 7f3f7cff9640 1 -- 172.21.15.97:0/1635362433 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3f8c004fc0 con 0x5646c3641620 2024-04-23T10:41:07.352 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.344+0000 7f3f9e20f740 10 client.17631 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:41:07.352 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.344+0000 7f3f9e20f740 10 client.17631 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:41:07.353 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.344+0000 7f3f9e20f740 20 client.17631 populate_metadata read hostname 'smithi097' 2024-04-23T10:41:07.353 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.344+0000 7f3f9e20f740 1 --2- 172.21.15.97:0/1635362433 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5646c36c2f30 0x5646c36c5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:07.353 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.344+0000 7f3f9e20f740 4 client.17631 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:41:07.353 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.344+0000 7f3f9e20f740 1 -- 172.21.15.97:0/1635362433 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5646c35ae200 con 0x5646c36c2f30 2024-04-23T10:41:07.353 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.345+0000 7f3f98a98640 1 --2- 172.21.15.97:0/1635362433 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5646c36c2f30 0x5646c36c5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:07.353 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.345+0000 7f3f98a98640 1 --2- 172.21.15.97:0/1635362433 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5646c36c2f30 0x5646c36c5350 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14ce18d7fa4a0d6e server_cookie=895043b9c3ddd883 in_seq=0 out_seq=0 2024-04-23T10:41:07.353 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.345+0000 7f3f7cff9640 10 client.17631 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:41:07.354 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:41:06 smithi196 ceph-mon[32919]: pgmap v467: 97 pgs: 97 active+clean; 2.3 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 71 KiB/s wr, 35 op/s 2024-04-23T10:41:07.360 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.358+0000 7f3f7cff9640 1 -- 172.21.15.97:0/1635362433 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5646c35ae200 con 0x5646c36c2f30 2024-04-23T10:41:07.360 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.358+0000 7f3f7cff9640 10 client.17631 handle_command_reply: tid=0 2024-04-23T10:41:07.360 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.358+0000 7f3f9e20f740 1 client.17631 shutdown 2024-04-23T10:41:07.360 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.359+0000 7f3f9e20f740 1 -- 172.21.15.97:0/1635362433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646c3641620 msgr2=0x5646c3654b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:07.360 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.359+0000 7f3f9e20f740 1 --2- 172.21.15.97:0/1635362433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646c3641620 0x5646c3654b00 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c00aac0 tx=0x7f3f8c004890 comp rx=0 tx=0).stop 2024-04-23T10:41:07.360 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.359+0000 7f3f9e20f740 1 -- 172.21.15.97:0/1635362433 shutdown_connections 2024-04-23T10:41:07.360 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.359+0000 7f3f9e20f740 1 --2- 172.21.15.97:0/1635362433 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5646c36c2f30 0x5646c36c5350 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:07.360 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.359+0000 7f3f9e20f740 1 --2- 172.21.15.97:0/1635362433 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646c364c800 0x5646c36503e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:07.360 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.359+0000 7f3f7cff9640 0 client.17631 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:41:07.361 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.359+0000 7f3f9e20f740 1 --2- 172.21.15.97:0/1635362433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646c3641f40 0x5646c364fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:07.361 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.359+0000 7f3f9e20f740 1 --2- 172.21.15.97:0/1635362433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5646c3641620 0x5646c3654b00 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:07.361 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.359+0000 7f3f9e20f740 1 -- 172.21.15.97:0/1635362433 >> 172.21.15.97:0/1635362433 conn(0x5646c363db30 msgr2=0x5646c363c420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:41:07.361 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.359+0000 7f3f9e20f740 1 -- 172.21.15.97:0/1635362433 shutdown_connections 2024-04-23T10:41:07.361 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.359+0000 7f3f9e20f740 1 -- 172.21.15.97:0/1635362433 wait complete. 2024-04-23T10:41:07.361 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.359+0000 7f3f9e20f740 20 client.17631 trim_cache size 0 max 16384 2024-04-23T10:41:07.361 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:41:07.361 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:41:07.362 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:41:07.362 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:41:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.359+0000 7f3f9b4fd640 1 -- 172.21.15.97:0/3974225544 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f3f800a3e40 msgr2=0x7f3f800a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.359+0000 7f3f9b4fd640 1 --2- 172.21.15.97:0/3974225544 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f3f800a3e40 0x7f3f800a6300 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3f90005fd0 tx=0x7f3f90009ad0 comp rx=0 tx=0).stop 2024-04-23T10:41:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.359+0000 7f3f9b4fd640 1 -- 172.21.15.97:0/3974225544 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f94106170 msgr2=0x7f3f94198bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.359+0000 7f3f9b4fd640 1 --2- 172.21.15.97:0/3974225544 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f94106170 0x7f3f94198bb0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f3f84032fd0 tx=0x7f3f84004270 comp rx=0 tx=0).stop 2024-04-23T10:41:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.360+0000 7f3f9b4fd640 1 -- 172.21.15.97:0/3974225544 shutdown_connections 2024-04-23T10:41:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.360+0000 7f3f9b4fd640 1 --2- 172.21.15.97:0/3974225544 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f3f800a3e40 0x7f3f800a6300 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.360+0000 7f3f9b4fd640 1 --2- 172.21.15.97:0/3974225544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f9419ce90 0x7f3f9419d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.360+0000 7f3f9b4fd640 1 --2- 172.21.15.97:0/3974225544 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3f94106170 0x7f3f94198bb0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:07.363 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.360+0000 7f3f9b4fd640 1 --2- 172.21.15.97:0/3974225544 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f941010f0 0x7f3f94198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:07.364 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.360+0000 7f3f9b4fd640 1 -- 172.21.15.97:0/3974225544 >> 172.21.15.97:0/3974225544 conn(0x7f3f94077fd0 msgr2=0x7f3f940698e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:41:07.364 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.360+0000 7f3f9b4fd640 1 -- 172.21.15.97:0/3974225544 shutdown_connections 2024-04-23T10:41:07.364 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:07.360+0000 7f3f9b4fd640 1 -- 172.21.15.97:0/3974225544 wait complete. 2024-04-23T10:41:07.364 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:41:06 smithi097 ceph-mon[29197]: pgmap v467: 97 pgs: 97 active+clean; 2.3 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 71 KiB/s wr, 35 op/s 2024-04-23T10:41:07.896 INFO:tasks.workunit.client.0.smithi097.stdout:499/600: open fd = 3 2024-04-23T10:41:07.896 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.896 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.896 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:41:07.896 INFO:tasks.workunit.client.0.smithi097.stdout:500/600: open fd = 3 2024-04-23T10:41:07.897 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.897 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.897 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:41:07.897 INFO:tasks.workunit.client.0.smithi097.stdout:501/600: open fd = 3 2024-04-23T10:41:07.897 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.897 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.897 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:41:07.897 INFO:tasks.workunit.client.0.smithi097.stdout:502/600: open fd = 3 2024-04-23T10:41:07.897 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.897 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.898 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:41:07.898 INFO:tasks.workunit.client.0.smithi097.stdout:503/600: open fd = 3 2024-04-23T10:41:07.898 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.898 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.898 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:41:07.898 INFO:tasks.workunit.client.0.smithi097.stdout:504/600: open fd = 3 2024-04-23T10:41:07.898 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.898 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.898 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:41:07.898 INFO:tasks.workunit.client.0.smithi097.stdout:505/600: open fd = 3 2024-04-23T10:41:07.899 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.899 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.899 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:41:07.899 INFO:tasks.workunit.client.0.smithi097.stdout:506/600: open fd = 3 2024-04-23T10:41:07.899 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.899 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.899 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:41:07.899 INFO:tasks.workunit.client.0.smithi097.stdout:507/600: open fd = 3 2024-04-23T10:41:07.899 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.899 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.899 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:41:07.900 INFO:tasks.workunit.client.0.smithi097.stdout:508/600: open fd = 3 2024-04-23T10:41:07.900 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.900 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.900 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:41:07.900 INFO:tasks.workunit.client.0.smithi097.stdout:509/600: open fd = 3 2024-04-23T10:41:07.900 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.900 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.900 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:41:07.900 INFO:tasks.workunit.client.0.smithi097.stdout:510/600: open fd = 3 2024-04-23T10:41:07.900 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.900 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.901 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:41:07.901 INFO:tasks.workunit.client.0.smithi097.stdout:511/600: open fd = 3 2024-04-23T10:41:07.901 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.901 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.901 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:41:07.901 INFO:tasks.workunit.client.0.smithi097.stdout:512/600: open fd = 3 2024-04-23T10:41:07.901 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.901 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.901 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:41:07.901 INFO:tasks.workunit.client.0.smithi097.stdout:513/600: open fd = 3 2024-04-23T10:41:07.902 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.902 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.902 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:41:07.902 INFO:tasks.workunit.client.0.smithi097.stdout:514/600: open fd = 3 2024-04-23T10:41:07.902 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.902 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.902 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:41:07.902 INFO:tasks.workunit.client.0.smithi097.stdout:515/600: open fd = 3 2024-04-23T10:41:07.902 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.902 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.903 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:41:07.903 INFO:tasks.workunit.client.0.smithi097.stdout:516/600: open fd = 3 2024-04-23T10:41:07.903 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.903 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.903 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:41:07.903 INFO:tasks.workunit.client.0.smithi097.stdout:517/600: open fd = 3 2024-04-23T10:41:07.903 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.903 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.903 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:41:07.903 INFO:tasks.workunit.client.0.smithi097.stdout:518/600: open fd = 3 2024-04-23T10:41:07.903 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.904 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.904 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:41:07.904 INFO:tasks.workunit.client.0.smithi097.stdout:519/600: open fd = 3 2024-04-23T10:41:07.904 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.904 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.904 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:41:07.904 INFO:tasks.workunit.client.0.smithi097.stdout:520/600: open fd = 3 2024-04-23T10:41:07.904 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.904 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.904 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:41:07.904 INFO:tasks.workunit.client.0.smithi097.stdout:521/600: open fd = 3 2024-04-23T10:41:07.905 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.905 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.905 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:41:07.905 INFO:tasks.workunit.client.0.smithi097.stdout:522/600: open fd = 3 2024-04-23T10:41:07.905 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.905 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.905 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:41:07.905 INFO:tasks.workunit.client.0.smithi097.stdout:523/600: open fd = 3 2024-04-23T10:41:07.905 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.905 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.906 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:41:07.906 INFO:tasks.workunit.client.0.smithi097.stdout:524/600: open fd = 3 2024-04-23T10:41:07.906 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.906 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.906 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:41:07.906 INFO:tasks.workunit.client.0.smithi097.stdout:525/600: open fd = 3 2024-04-23T10:41:07.906 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.906 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.906 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:41:07.906 INFO:tasks.workunit.client.0.smithi097.stdout:526/600: open fd = 3 2024-04-23T10:41:07.907 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.907 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.907 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:41:07.907 INFO:tasks.workunit.client.0.smithi097.stdout:527/600: open fd = 3 2024-04-23T10:41:07.907 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.907 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.907 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:41:07.907 INFO:tasks.workunit.client.0.smithi097.stdout:528/600: open fd = 3 2024-04-23T10:41:07.907 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.907 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.907 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:41:07.908 INFO:tasks.workunit.client.0.smithi097.stdout:529/600: open fd = 3 2024-04-23T10:41:07.908 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.908 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.908 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:41:07.908 INFO:tasks.workunit.client.0.smithi097.stdout:530/600: open fd = 3 2024-04-23T10:41:07.908 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.908 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.908 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:41:07.908 INFO:tasks.workunit.client.0.smithi097.stdout:531/600: open fd = 3 2024-04-23T10:41:07.908 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.908 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.909 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:41:07.909 INFO:tasks.workunit.client.0.smithi097.stdout:532/600: open fd = 3 2024-04-23T10:41:07.909 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.909 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.909 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:41:07.909 INFO:tasks.workunit.client.0.smithi097.stdout:533/600: open fd = 3 2024-04-23T10:41:07.909 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.909 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.909 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:41:07.909 INFO:tasks.workunit.client.0.smithi097.stdout:534/600: open fd = 3 2024-04-23T10:41:07.909 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.909 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.910 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:41:07.910 INFO:tasks.workunit.client.0.smithi097.stdout:535/600: open fd = 3 2024-04-23T10:41:07.910 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.910 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.910 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:41:07.910 INFO:tasks.workunit.client.0.smithi097.stdout:536/600: open fd = 3 2024-04-23T10:41:07.910 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.910 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.910 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:41:07.910 INFO:tasks.workunit.client.0.smithi097.stdout:537/600: open fd = 3 2024-04-23T10:41:07.910 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.911 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.911 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:41:07.911 INFO:tasks.workunit.client.0.smithi097.stdout:538/600: open fd = 3 2024-04-23T10:41:07.911 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.911 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.911 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:41:07.911 INFO:tasks.workunit.client.0.smithi097.stdout:539/600: open fd = 3 2024-04-23T10:41:07.911 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.911 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.911 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:41:07.911 INFO:tasks.workunit.client.0.smithi097.stdout:540/600: open fd = 3 2024-04-23T10:41:07.912 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.912 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.912 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:41:07.912 INFO:tasks.workunit.client.0.smithi097.stdout:541/600: open fd = 3 2024-04-23T10:41:07.912 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.912 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.912 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:41:07.912 INFO:tasks.workunit.client.0.smithi097.stdout:542/600: open fd = 3 2024-04-23T10:41:07.912 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.912 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.912 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:41:07.913 INFO:tasks.workunit.client.0.smithi097.stdout:543/600: open fd = 3 2024-04-23T10:41:07.913 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.913 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.913 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:41:07.913 INFO:tasks.workunit.client.0.smithi097.stdout:544/600: open fd = 3 2024-04-23T10:41:07.913 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.913 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.913 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:41:07.913 INFO:tasks.workunit.client.0.smithi097.stdout:545/600: open fd = 3 2024-04-23T10:41:07.913 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.913 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.914 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:41:07.914 INFO:tasks.workunit.client.0.smithi097.stdout:546/600: open fd = 3 2024-04-23T10:41:07.914 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.914 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.914 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:41:07.914 INFO:tasks.workunit.client.0.smithi097.stdout:547/600: open fd = 3 2024-04-23T10:41:07.914 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.914 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.914 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:41:07.914 INFO:tasks.workunit.client.0.smithi097.stdout:548/600: open fd = 3 2024-04-23T10:41:07.914 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.915 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.915 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:41:07.915 INFO:tasks.workunit.client.0.smithi097.stdout:549/600: open fd = 3 2024-04-23T10:41:07.915 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.915 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.915 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:41:07.915 INFO:tasks.workunit.client.0.smithi097.stdout:550/600: open fd = 3 2024-04-23T10:41:07.915 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.915 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.915 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:41:07.915 INFO:tasks.workunit.client.0.smithi097.stdout:551/600: open fd = 3 2024-04-23T10:41:07.916 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.916 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.916 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:41:07.916 INFO:tasks.workunit.client.0.smithi097.stdout:552/600: open fd = 3 2024-04-23T10:41:07.916 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.916 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.916 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:41:07.916 INFO:tasks.workunit.client.0.smithi097.stdout:553/600: open fd = 3 2024-04-23T10:41:07.916 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.916 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.916 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:41:07.916 INFO:tasks.workunit.client.0.smithi097.stdout:554/600: open fd = 3 2024-04-23T10:41:07.917 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.917 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.917 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:41:07.917 INFO:tasks.workunit.client.0.smithi097.stdout:555/600: open fd = 3 2024-04-23T10:41:07.917 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.917 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.917 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:41:07.917 INFO:tasks.workunit.client.0.smithi097.stdout:556/600: open fd = 3 2024-04-23T10:41:07.917 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.917 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.917 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:41:07.917 INFO:tasks.workunit.client.0.smithi097.stdout:557/600: open fd = 3 2024-04-23T10:41:07.918 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.918 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.918 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:41:07.918 INFO:tasks.workunit.client.0.smithi097.stdout:558/600: open fd = 3 2024-04-23T10:41:07.918 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.918 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.918 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:41:07.918 INFO:tasks.workunit.client.0.smithi097.stdout:559/600: open fd = 3 2024-04-23T10:41:07.918 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.918 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.918 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:41:07.919 INFO:tasks.workunit.client.0.smithi097.stdout:560/600: open fd = 3 2024-04-23T10:41:07.919 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.919 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:41:07.919 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:41:09.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:41:08 smithi183 ceph-mon[32317]: pgmap v468: 97 pgs: 97 active+clean; 2.5 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 85 KiB/s wr, 34 op/s 2024-04-23T10:41:09.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:41:08 smithi196 ceph-mon[32919]: pgmap v468: 97 pgs: 97 active+clean; 2.5 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 85 KiB/s wr, 34 op/s 2024-04-23T10:41:09.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:41:08 smithi097 ceph-mon[29197]: pgmap v468: 97 pgs: 97 active+clean; 2.5 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 85 KiB/s wr, 34 op/s 2024-04-23T10:41:11.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:41:10 smithi183 ceph-mon[32317]: pgmap v469: 97 pgs: 97 active+clean; 2.5 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 85 KiB/s wr, 34 op/s 2024-04-23T10:41:11.163 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:41:10 smithi097 ceph-mon[29197]: pgmap v469: 97 pgs: 97 active+clean; 2.5 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 85 KiB/s wr, 34 op/s 2024-04-23T10:41:11.341 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:41:10 smithi196 ceph-mon[32919]: pgmap v469: 97 pgs: 97 active+clean; 2.5 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 85 KiB/s wr, 34 op/s 2024-04-23T10:41:12.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:41:12 smithi196 ceph-mon[32919]: osdmap e107: 12 total, 12 up, 12 in 2024-04-23T10:41:12.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:41:12 smithi097 ceph-mon[29197]: osdmap e107: 12 total, 12 up, 12 in 2024-04-23T10:41:12.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:41:12 smithi183 ceph-mon[32317]: osdmap e107: 12 total, 12 up, 12 in 2024-04-23T10:41:13.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:41:13 smithi196 ceph-mon[32919]: pgmap v471: 97 pgs: 97 active+clean; 2.5 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 90 KiB/s wr, 36 op/s 2024-04-23T10:41:13.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:41:13 smithi097 ceph-mon[29197]: pgmap v471: 97 pgs: 97 active+clean; 2.5 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 90 KiB/s wr, 36 op/s 2024-04-23T10:41:13.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:41:13 smithi183 ceph-mon[32317]: pgmap v471: 97 pgs: 97 active+clean; 2.5 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 90 KiB/s wr, 36 op/s 2024-04-23T10:41:15.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:41:15 smithi196 ceph-mon[32919]: pgmap v472: 97 pgs: 97 active+clean; 2.7 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 48 KiB/s wr, 18 op/s 2024-04-23T10:41:15.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:41:15 smithi097 ceph-mon[29197]: pgmap v472: 97 pgs: 97 active+clean; 2.7 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 48 KiB/s wr, 18 op/s 2024-04-23T10:41:15.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:41:15 smithi183 ceph-mon[32317]: pgmap v472: 97 pgs: 97 active+clean; 2.7 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 48 KiB/s wr, 18 op/s 2024-04-23T10:41:17.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:41:17 smithi196 ceph-mon[32919]: pgmap v473: 97 pgs: 97 active+clean; 2.7 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 48 KiB/s wr, 18 op/s 2024-04-23T10:41:17.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:41:17 smithi097 ceph-mon[29197]: pgmap v473: 97 pgs: 97 active+clean; 2.7 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 48 KiB/s wr, 18 op/s 2024-04-23T10:41:17.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:41:17 smithi183 ceph-mon[32317]: pgmap v473: 97 pgs: 97 active+clean; 2.7 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 48 KiB/s wr, 18 op/s 2024-04-23T10:41:19.412 INFO:tasks.quiescer.fs.[cephfs]:Total duration (49.3) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-04-23T10:41:19.413 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.65 2024-04-23T10:41:19.413 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T10:41:19.438 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:41:19 smithi097 ceph-mon[29197]: pgmap v474: 97 pgs: 97 active+clean; 2.9 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 49 KiB/s wr, 18 op/s 2024-04-23T10:41:19.525 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.524+0000 7f3375afa640 1 -- 172.21.15.97:0/342861140 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3370103230 msgr2=0x7f3370103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:19.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.524+0000 7f3375afa640 1 --2- 172.21.15.97:0/342861140 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3370103230 0x7f3370103610 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f3360009900 tx=0x7f336002a480 comp rx=0 tx=0).stop 2024-04-23T10:41:19.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.524+0000 7f3375afa640 1 -- 172.21.15.97:0/342861140 shutdown_connections 2024-04-23T10:41:19.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.524+0000 7f3375afa640 1 --2- 172.21.15.97:0/342861140 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f337010c570 0x7f337010e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:19.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.524+0000 7f3375afa640 1 --2- 172.21.15.97:0/342861140 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3370103b50 0x7f337010c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:19.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.524+0000 7f3375afa640 1 --2- 172.21.15.97:0/342861140 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3370103230 0x7f3370103610 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:19.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.524+0000 7f3375afa640 1 -- 172.21.15.97:0/342861140 >> 172.21.15.97:0/342861140 conn(0x7f3370077fb0 msgr2=0x7f33700783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:41:19.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.524+0000 7f3375afa640 1 -- 172.21.15.97:0/342861140 shutdown_connections 2024-04-23T10:41:19.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.524+0000 7f3375afa640 1 -- 172.21.15.97:0/342861140 wait complete. 2024-04-23T10:41:19.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.525+0000 7f3375afa640 1 Processor -- start 2024-04-23T10:41:19.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.525+0000 7f3375afa640 1 -- start start 2024-04-23T10:41:19.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.525+0000 7f3375afa640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3370103230 0x7f3370100e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:19.526 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.525+0000 7f3375afa640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3370103b50 0x7f33701013c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:19.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.525+0000 7f3375afa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f337010c570 0x7f3370101900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:19.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.525+0000 7f3375afa640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f33701121d0 con 0x7f337010c570 2024-04-23T10:41:19.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.525+0000 7f3375afa640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3370112050 con 0x7f3370103b50 2024-04-23T10:41:19.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.525+0000 7f3375afa640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3370112350 con 0x7f3370103230 2024-04-23T10:41:19.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.526+0000 7f336ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f337010c570 0x7f3370101900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:19.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.526+0000 7f336ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f337010c570 0x7f3370101900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37012/0 (socket says 172.21.15.97:37012) 2024-04-23T10:41:19.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.526+0000 7f336ffff640 1 -- 172.21.15.97:0/3244714904 learned_addr learned my addr 172.21.15.97:0/3244714904 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:41:19.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.526+0000 7f336effd640 1 --2- 172.21.15.97:0/3244714904 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3370103b50 0x7f33701013c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:19.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.526+0000 7f336f7fe640 1 --2- 172.21.15.97:0/3244714904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3370103230 0x7f3370100e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:19.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.526+0000 7f336ffff640 1 -- 172.21.15.97:0/3244714904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3370103230 msgr2=0x7f3370100e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:19.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.526+0000 7f336ffff640 1 --2- 172.21.15.97:0/3244714904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3370103230 0x7f3370100e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:19.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.526+0000 7f336ffff640 1 -- 172.21.15.97:0/3244714904 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3370103b50 msgr2=0x7f33701013c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:19.527 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.526+0000 7f336ffff640 1 --2- 172.21.15.97:0/3244714904 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3370103b50 0x7f33701013c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:19.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.526+0000 7f336ffff640 1 -- 172.21.15.97:0/3244714904 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33701a6cf0 con 0x7f337010c570 2024-04-23T10:41:19.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.526+0000 7f336f7fe640 1 --2- 172.21.15.97:0/3244714904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3370103230 0x7f3370100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:41:19.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.526+0000 7f336ffff640 1 --2- 172.21.15.97:0/3244714904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f337010c570 0x7f3370101900 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f336800e960 tx=0x7f336800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:41:19.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.526+0000 7f336cff9640 1 -- 172.21.15.97:0/3244714904 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3368004d20 con 0x7f337010c570 2024-04-23T10:41:19.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.526+0000 7f336cff9640 1 -- 172.21.15.97:0/3244714904 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3368005020 con 0x7f337010c570 2024-04-23T10:41:19.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.526+0000 7f3375afa640 1 -- 172.21.15.97:0/3244714904 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33701a6fe0 con 0x7f337010c570 2024-04-23T10:41:19.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.526+0000 7f336cff9640 1 -- 172.21.15.97:0/3244714904 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33680116d0 con 0x7f337010c570 2024-04-23T10:41:19.528 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.526+0000 7f3375afa640 1 -- 172.21.15.97:0/3244714904 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33701a7500 con 0x7f337010c570 2024-04-23T10:41:19.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.527+0000 7f3375afa640 1 -- 172.21.15.97:0/3244714904 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33700691c0 con 0x7f337010c570 2024-04-23T10:41:19.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.528+0000 7f336cff9640 1 -- 172.21.15.97:0/3244714904 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f33680051d0 con 0x7f337010c570 2024-04-23T10:41:19.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.529+0000 7f336cff9640 1 --2- 172.21.15.97:0/3244714904 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f33580a3d20 0x7f33580a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:19.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.529+0000 7f336f7fe640 1 --2- 172.21.15.97:0/3244714904 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f33580a3d20 0x7f33580a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:19.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.529+0000 7f336cff9640 1 -- 172.21.15.97:0/3244714904 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8484+0+0 (secure 0 0 0) 0x7f33680d0bc0 con 0x7f337010c570 2024-04-23T10:41:19.529 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.529+0000 7f336f7fe640 1 --2- 172.21.15.97:0/3244714904 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f33580a3d20 0x7f33580a61e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3360031440 tx=0x7f3360005a20 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:41:19.532 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.533+0000 7f336cff9640 1 -- 172.21.15.97:0/3244714904 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3368016030 con 0x7f337010c570 2024-04-23T10:41:19.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:41:19 smithi196 ceph-mon[32919]: pgmap v474: 97 pgs: 97 active+clean; 2.9 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 49 KiB/s wr, 18 op/s 2024-04-23T10:41:19.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:41:19 smithi183 ceph-mon[32317]: pgmap v474: 97 pgs: 97 active+clean; 2.9 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 49 KiB/s wr, 18 op/s 2024-04-23T10:41:19.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.747+0000 7f3375afa640 1 -- 172.21.15.97:0/3244714904 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f337002d5c0 con 0x7f337010c570 2024-04-23T10:41:19.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.748+0000 7f336cff9640 1 -- 172.21.15.97:0/3244714904 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11931 (secure 0 0 0) 0x7f336809d280 con 0x7f337010c570 2024-04-23T10:41:19.747 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:41:19.747 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3946878762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3946878762},{"type":"v1","addr":"172.21.15.97:6835","nonce":3946878762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/209800861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":209800861},{"type":"v1","addr":"172.21.15.97:6837","nonce":209800861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T10:24:06.910134+0000","modified":"2024-04-23T10:26:09.820985+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6839/418013575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":418013575},{"type":"v1","addr":"172.21.15.97:6839","nonce":418013575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6841/747242680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":747242680},{"type":"v1","addr":"172.21.15.97:6841","nonce":747242680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6833/4292847121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4292847121},{"type":"v1","addr":"172.21.15.183:6833","nonce":4292847121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6833/3885816565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3885816565},{"type":"v1","addr":"172.21.15.196:6833","nonce":3885816565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1897093105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1897093105},{"type":"v1","addr":"172.21.15.183:6835","nonce":1897093105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1171477563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1171477563},{"type":"v1","addr":"172.21.15.196:6835","nonce":1171477563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1898341911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1898341911},{"type":"v1","addr":"172.21.15.183:6837","nonce":1898341911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.196:6837/3094946502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3094946502},{"type":"v1","addr":"172.21.15.196:6837","nonce":3094946502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.196:6839/83843556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":83843556},{"type":"v1","addr":"172.21.15.196:6839","nonce":83843556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6839/1763841130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1763841130},{"type":"v1","addr":"172.21.15.183:6839","nonce":1763841130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24487,24403,24398,24469,24488]},"id":1}]} 2024-04-23T10:41:19.748 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 26 2024-04-23T10:41:19.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.750+0000 7f3375afa640 1 -- 172.21.15.97:0/3244714904 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f33580a3d20 msgr2=0x7f33580a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:19.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.750+0000 7f3375afa640 1 --2- 172.21.15.97:0/3244714904 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f33580a3d20 0x7f33580a61e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3360031440 tx=0x7f3360005a20 comp rx=0 tx=0).stop 2024-04-23T10:41:19.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.750+0000 7f3375afa640 1 -- 172.21.15.97:0/3244714904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f337010c570 msgr2=0x7f3370101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:19.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.750+0000 7f3375afa640 1 --2- 172.21.15.97:0/3244714904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f337010c570 0x7f3370101900 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f336800e960 tx=0x7f336800ee20 comp rx=0 tx=0).stop 2024-04-23T10:41:19.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.750+0000 7f3375afa640 1 -- 172.21.15.97:0/3244714904 shutdown_connections 2024-04-23T10:41:19.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.750+0000 7f3375afa640 1 --2- 172.21.15.97:0/3244714904 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f33580a3d20 0x7f33580a61e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:19.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.750+0000 7f3375afa640 1 --2- 172.21.15.97:0/3244714904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f337010c570 0x7f3370101900 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:19.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.750+0000 7f3375afa640 1 --2- 172.21.15.97:0/3244714904 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3370103b50 0x7f33701013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:19.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.750+0000 7f3375afa640 1 --2- 172.21.15.97:0/3244714904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3370103230 0x7f3370100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:19.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.750+0000 7f3375afa640 1 -- 172.21.15.97:0/3244714904 >> 172.21.15.97:0/3244714904 conn(0x7f3370077fb0 msgr2=0x7f33700ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:41:19.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.750+0000 7f3375afa640 1 -- 172.21.15.97:0/3244714904 shutdown_connections 2024-04-23T10:41:19.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.750+0000 7f3375afa640 1 -- 172.21.15.97:0/3244714904 wait complete. 2024-04-23T10:41:19.760 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db / --timeout 60 --expiration 84.65 --await-for 42.0' 2024-04-23T10:41:19.760 DEBUG:teuthology.orchestra.run.smithi097:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24488 quiesce db / --timeout 60 --expiration 84.65 --await-for 42.0 2024-04-23T10:41:19.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.910+0000 7f587ffcf640 1 -- 172.21.15.97:0/3190312465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5878106670 msgr2=0x7f5878106a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:19.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.910+0000 7f587ffcf640 1 --2- 172.21.15.97:0/3190312465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5878106670 0x7f5878106a50 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f5874009900 tx=0x7f587402a410 comp rx=0 tx=0).stop 2024-04-23T10:41:19.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.910+0000 7f587ffcf640 1 -- 172.21.15.97:0/3190312465 shutdown_connections 2024-04-23T10:41:19.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.910+0000 7f587ffcf640 1 --2- 172.21.15.97:0/3190312465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5878101240 0x7f587810d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:19.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.910+0000 7f587ffcf640 1 --2- 172.21.15.97:0/3190312465 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5878100880 0x7f5878100d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:19.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.910+0000 7f587ffcf640 1 --2- 172.21.15.97:0/3190312465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5878106670 0x7f5878106a50 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:19.911 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.910+0000 7f587ffcf640 1 -- 172.21.15.97:0/3190312465 >> 172.21.15.97:0/3190312465 conn(0x7f58780761d0 msgr2=0x7f58780765e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:41:19.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.910+0000 7f587ffcf640 1 -- 172.21.15.97:0/3190312465 shutdown_connections 2024-04-23T10:41:19.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.910+0000 7f587ffcf640 1 -- 172.21.15.97:0/3190312465 wait complete. 2024-04-23T10:41:19.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.911+0000 7f587ffcf640 1 Processor -- start 2024-04-23T10:41:19.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.911+0000 7f587ffcf640 1 -- start start 2024-04-23T10:41:19.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.911+0000 7f587ffcf640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5878100880 0x7f587810ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:19.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.911+0000 7f587ffcf640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5878101240 0x7f587810f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:19.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.911+0000 7f587ffcf640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5878106670 0x7f587810f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:19.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.911+0000 7f587ffcf640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f58781121e0 con 0x7f5878100880 2024-04-23T10:41:19.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.911+0000 7f587ffcf640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5878112060 con 0x7f5878101240 2024-04-23T10:41:19.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.911+0000 7f587ffcf640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5878112360 con 0x7f5878106670 2024-04-23T10:41:19.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.911+0000 7f587dd6b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5878100880 0x7f587810ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:19.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.911+0000 7f587dd6b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5878100880 0x7f587810ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37020/0 (socket says 172.21.15.97:37020) 2024-04-23T10:41:19.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.911+0000 7f587dd6b640 1 -- 172.21.15.97:0/3618672798 learned_addr learned my addr 172.21.15.97:0/3618672798 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:41:19.912 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.911+0000 7f587d56a640 1 --2- 172.21.15.97:0/3618672798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5878101240 0x7f587810f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:19.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.911+0000 7f587dd6b640 1 -- 172.21.15.97:0/3618672798 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5878106670 msgr2=0x7f587810f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:19.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.911+0000 7f587dd6b640 1 --2- 172.21.15.97:0/3618672798 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5878106670 0x7f587810f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:19.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.911+0000 7f587dd6b640 1 -- 172.21.15.97:0/3618672798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5878101240 msgr2=0x7f587810f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:19.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.911+0000 7f587dd6b640 1 --2- 172.21.15.97:0/3618672798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5878101240 0x7f587810f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:19.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.911+0000 7f587dd6b640 1 -- 172.21.15.97:0/3618672798 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58781a6c80 con 0x7f5878100880 2024-04-23T10:41:19.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.911+0000 7f587d56a640 1 --2- 172.21.15.97:0/3618672798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5878101240 0x7f587810f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:41:19.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.911+0000 7f587dd6b640 1 --2- 172.21.15.97:0/3618672798 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5878100880 0x7f587810ecd0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f5874006010 tx=0x7f58740042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:41:19.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.911+0000 7f586effd640 1 -- 172.21.15.97:0/3618672798 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f587403f030 con 0x7f5878100880 2024-04-23T10:41:19.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.912+0000 7f586effd640 1 -- 172.21.15.97:0/3618672798 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5874003800 con 0x7f5878100880 2024-04-23T10:41:19.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.912+0000 7f586effd640 1 -- 172.21.15.97:0/3618672798 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5874035070 con 0x7f5878100880 2024-04-23T10:41:19.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.912+0000 7f587ffcf640 1 -- 172.21.15.97:0/3618672798 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58781a6f10 con 0x7f5878100880 2024-04-23T10:41:19.913 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.912+0000 7f587ffcf640 1 -- 172.21.15.97:0/3618672798 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58781a7370 con 0x7f5878100880 2024-04-23T10:41:19.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.914+0000 7f586effd640 1 -- 172.21.15.97:0/3618672798 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5874003cb0 con 0x7f5878100880 2024-04-23T10:41:19.920 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.914+0000 7f5882dbd740 1 Processor -- start 2024-04-23T10:41:19.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.914+0000 7f5882dbd740 1 -- start start 2024-04-23T10:41:19.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.915+0000 7f5882dbd740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5571607340a0 0x557160734480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:19.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.915+0000 7f5882dbd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5571607349c0 0x55716073eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:19.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.915+0000 7f5882dbd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55716073f090 0x557160741480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:19.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.915+0000 7f5882dbd740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x557160699370 con 0x5571607340a0 2024-04-23T10:41:19.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.915+0000 7f5882dbd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5571606362e0 con 0x55716073f090 2024-04-23T10:41:19.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.915+0000 7f5882dbd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557160695330 con 0x5571607349c0 2024-04-23T10:41:19.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.915+0000 7f586effd640 1 --2- 172.21.15.97:0/3618672798 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f58640a3d20 0x7f58640a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:19.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.915+0000 7f587e56c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5571607340a0 0x557160734480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:19.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.915+0000 7f587e56c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5571607340a0 0x557160734480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37028/0 (socket says 172.21.15.97:37028) 2024-04-23T10:41:19.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.915+0000 7f587e56c640 1 -- 172.21.15.97:0/2155514625 learned_addr learned my addr 172.21.15.97:0/2155514625 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:41:19.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.915+0000 7f587e56c640 1 --2- 172.21.15.97:0/3618672798 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f58640a3d20 0x7f58640a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:19.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.915+0000 7f587d56a640 1 --2- 172.21.15.97:0/2155514625 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55716073f090 0x557160741480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:19.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.915+0000 7f586effd640 1 -- 172.21.15.97:0/3618672798 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8484+0+0 (secure 0 0 0) 0x7f58740f1280 con 0x7f5878100880 2024-04-23T10:41:19.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.915+0000 7f587dd6b640 1 --2- 172.21.15.97:0/2155514625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5571607349c0 0x55716073eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:19.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.915+0000 7f587e56c640 1 -- 172.21.15.97:0/2155514625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5571607349c0 msgr2=0x55716073eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:19.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.915+0000 7f587e56c640 1 --2- 172.21.15.97:0/2155514625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5571607349c0 0x55716073eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:19.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.915+0000 7f587e56c640 1 -- 172.21.15.97:0/2155514625 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55716073f090 msgr2=0x557160741480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:19.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.915+0000 7f587e56c640 1 --2- 172.21.15.97:0/2155514625 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55716073f090 0x557160741480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:19.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.915+0000 7f587e56c640 1 -- 172.21.15.97:0/2155514625 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557160477d80 con 0x5571607340a0 2024-04-23T10:41:19.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.915+0000 7f587e56c640 1 --2- 172.21.15.97:0/2155514625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5571607340a0 0x557160734480 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f5870013bd0 tx=0x7f5870011750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41e844ebe869b8c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:41:19.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.915+0000 7f587e56c640 1 --2- 172.21.15.97:0/3618672798 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f58640a3d20 0x7f58640a61e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5870011c50 tx=0x7f587001a000 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:41:19.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.916+0000 7f5863fff640 1 -- 172.21.15.97:0/2155514625 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5870022070 con 0x5571607340a0 2024-04-23T10:41:19.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.916+0000 7f5863fff640 1 -- 172.21.15.97:0/2155514625 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5870009990 con 0x5571607340a0 2024-04-23T10:41:19.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.916+0000 7f5863fff640 1 -- 172.21.15.97:0/2155514625 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5870004020 con 0x5571607340a0 2024-04-23T10:41:19.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.916+0000 7f5882dbd740 1 -- 172.21.15.97:0/2155514625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5571607340a0 msgr2=0x557160734480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:19.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.916+0000 7f5882dbd740 1 --2- 172.21.15.97:0/2155514625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5571607340a0 0x557160734480 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f5870013bd0 tx=0x7f5870011750 comp rx=0 tx=0).stop 2024-04-23T10:41:19.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.916+0000 7f5882dbd740 1 -- 172.21.15.97:0/2155514625 shutdown_connections 2024-04-23T10:41:19.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.916+0000 7f5882dbd740 1 --2- 172.21.15.97:0/2155514625 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55716073f090 0x557160741480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:19.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.916+0000 7f5882dbd740 1 --2- 172.21.15.97:0/2155514625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5571607349c0 0x55716073eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:19.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.916+0000 7f5882dbd740 1 --2- 172.21.15.97:0/2155514625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5571607340a0 0x557160734480 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:19.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.916+0000 7f5882dbd740 1 -- 172.21.15.97:0/2155514625 >> 172.21.15.97:0/2155514625 conn(0x5571607303c0 msgr2=0x5571607307a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:41:19.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.916+0000 7f5882dbd740 1 -- 172.21.15.97:0/2155514625 shutdown_connections 2024-04-23T10:41:19.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.916+0000 7f5882dbd740 1 -- 172.21.15.97:0/2155514625 wait complete. 2024-04-23T10:41:19.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.917+0000 7f5882dbd740 1 -- start start 2024-04-23T10:41:19.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.917+0000 7f5882dbd740 1 Processor -- start 2024-04-23T10:41:19.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.917+0000 7f5882dbd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571607340a0 0x55716073c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:19.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.917+0000 7f5882dbd740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5571607349c0 0x55716073c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:19.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.917+0000 7f5882dbd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55716073f090 0x55716073cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:19.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.917+0000 7f5882dbd740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x557160741e20 con 0x5571607349c0 2024-04-23T10:41:19.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.917+0000 7f5882dbd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557160633fb0 con 0x5571607340a0 2024-04-23T10:41:19.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.917+0000 7f5882dbd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557160741fa0 con 0x55716073f090 2024-04-23T10:41:19.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.917+0000 7f5882dbd740 10 client.0 _ll_register_callbacks cb 0x5571606339a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:41:19.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.917+0000 7f587d56a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5571607349c0 0x55716073c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:19.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.917+0000 7f587d56a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5571607349c0 0x55716073c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37034/0 (socket says 172.21.15.97:37034) 2024-04-23T10:41:19.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.917+0000 7f587d56a640 1 -- 172.21.15.97:0/2594204450 learned_addr learned my addr 172.21.15.97:0/2594204450 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:41:19.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.917+0000 7f587dd6b640 1 --2- 172.21.15.97:0/2594204450 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571607340a0 0x55716073c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:19.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.917+0000 7f587e56c640 1 --2- 172.21.15.97:0/2594204450 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55716073f090 0x55716073cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:19.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.918+0000 7f587d56a640 1 -- 172.21.15.97:0/2594204450 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55716073f090 msgr2=0x55716073cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:19.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.918+0000 7f587d56a640 1 --2- 172.21.15.97:0/2594204450 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55716073f090 0x55716073cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:19.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.918+0000 7f587d56a640 1 -- 172.21.15.97:0/2594204450 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571607340a0 msgr2=0x55716073c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:19.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.918+0000 7f587d56a640 1 --2- 172.21.15.97:0/2594204450 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571607340a0 0x55716073c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:19.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.918+0000 7f587d56a640 1 -- 172.21.15.97:0/2594204450 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571604d8510 con 0x5571607349c0 2024-04-23T10:41:19.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.918+0000 7f587d56a640 1 --2- 172.21.15.97:0/2594204450 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5571607349c0 0x55716073c5f0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f586800f670 tx=0x7f586800fb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e75b0f0679066ad4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:41:19.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.918+0000 7f58617fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:41:19.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.918+0000 7f58617fa640 1 -- 172.21.15.97:0/2594204450 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5868016070 con 0x5571607349c0 2024-04-23T10:41:19.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.918+0000 7f58617fa640 1 -- 172.21.15.97:0/2594204450 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8484+0+0 (secure 0 0 0) 0x7f586800bb00 con 0x5571607349c0 2024-04-23T10:41:19.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.918+0000 7f5882dbd740 1 -- 172.21.15.97:0/2594204450 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5870009990 con 0x5571607349c0 2024-04-23T10:41:19.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.918+0000 7f58617fa640 1 -- 172.21.15.97:0/2594204450 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f586800bd00 con 0x5571607349c0 2024-04-23T10:41:19.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.918+0000 7f5882dbd740 10 client.17649 fetch_fsmap learned FSMap version 26 2024-04-23T10:41:19.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.918+0000 7f5882dbd740 1 -- 172.21.15.97:0/2594204450 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55716036b670 con 0x5571607349c0 2024-04-23T10:41:19.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.918+0000 7f58617fa640 1 -- 172.21.15.97:0/2594204450 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5870009990 con 0x5571607349c0 2024-04-23T10:41:19.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.919+0000 7f58617fa640 1 -- 172.21.15.97:0/2594204450 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5868021020 con 0x5571607349c0 2024-04-23T10:41:19.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.919+0000 7f5882dbd740 10 client.17649 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:41:19.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.919+0000 7f5882dbd740 10 client.17649 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:41:19.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.919+0000 7f5882dbd740 20 client.17649 populate_metadata read hostname 'smithi097' 2024-04-23T10:41:19.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.919+0000 7f5882dbd740 1 --2- 172.21.15.97:0/2594204450 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55716074f4a0 0x557160751920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:19.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.919+0000 7f5882dbd740 4 client.17649 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:41:19.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.919+0000 7f5882dbd740 1 -- 172.21.15.97:0/2594204450 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5571606a6d90 con 0x55716074f4a0 2024-04-23T10:41:19.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.919+0000 7f587dd6b640 1 --2- 172.21.15.97:0/2594204450 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55716074f4a0 0x557160751920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:19.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.919+0000 7f58617fa640 10 client.17649 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:41:19.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.919+0000 7f587dd6b640 1 --2- 172.21.15.97:0/2594204450 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55716074f4a0 0x557160751920 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f42531159af4f75 server_cookie=2872b702e4d01ca8 in_seq=0 out_seq=0 2024-04-23T10:41:19.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.920+0000 7f58617fa640 1 -- 172.21.15.97:0/2594204450 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5571606a6d90 con 0x55716074f4a0 2024-04-23T10:41:19.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.920+0000 7f58617fa640 10 client.17649 handle_command_reply: tid=0 2024-04-23T10:41:19.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.920+0000 7f5882dbd740 1 client.17649 shutdown 2024-04-23T10:41:19.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.920+0000 7f5882dbd740 1 -- 172.21.15.97:0/2594204450 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5571607349c0 msgr2=0x55716073c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:19.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.920+0000 7f5882dbd740 1 --2- 172.21.15.97:0/2594204450 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5571607349c0 0x55716073c5f0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f586800f670 tx=0x7f586800fb30 comp rx=0 tx=0).stop 2024-04-23T10:41:19.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.920+0000 7f5882dbd740 1 -- 172.21.15.97:0/2594204450 shutdown_connections 2024-04-23T10:41:19.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.920+0000 7f5882dbd740 1 --2- 172.21.15.97:0/2594204450 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55716074f4a0 0x557160751920 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:19.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.920+0000 7f5882dbd740 1 --2- 172.21.15.97:0/2594204450 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55716073f090 0x55716073cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:19.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.920+0000 7f58617fa640 0 client.17649 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:41:19.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.920+0000 7f5882dbd740 1 --2- 172.21.15.97:0/2594204450 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5571607349c0 0x55716073c5f0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:19.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.920+0000 7f5882dbd740 1 --2- 172.21.15.97:0/2594204450 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571607340a0 0x55716073c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:19.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.920+0000 7f5882dbd740 1 -- 172.21.15.97:0/2594204450 >> 172.21.15.97:0/2594204450 conn(0x5571607303c0 msgr2=0x5571607361c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:41:19.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.921+0000 7f5882dbd740 1 -- 172.21.15.97:0/2594204450 shutdown_connections 2024-04-23T10:41:19.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.921+0000 7f5882dbd740 1 -- 172.21.15.97:0/2594204450 wait complete. 2024-04-23T10:41:19.927 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.921+0000 7f5882dbd740 20 client.17649 trim_cache size 0 max 16384 2024-04-23T10:41:19.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.938+0000 7f5882dbd740 1 Processor -- start 2024-04-23T10:41:19.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.938+0000 7f5882dbd740 1 -- start start 2024-04-23T10:41:19.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.938+0000 7f5882dbd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571607340a0 0x55716074ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:19.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.938+0000 7f5882dbd740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5571607349c0 0x5571607571f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:19.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.938+0000 7f5882dbd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55716073f090 0x557160757730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:19.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.938+0000 7f5882dbd740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f58781afd30 con 0x5571607349c0 2024-04-23T10:41:19.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.938+0000 7f5882dbd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f58781b1870 con 0x5571607340a0 2024-04-23T10:41:19.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.938+0000 7f5882dbd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f58781b0400 con 0x55716073f090 2024-04-23T10:41:19.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.938+0000 7f587d56a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5571607349c0 0x5571607571f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:19.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.938+0000 7f587d56a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5571607349c0 0x5571607571f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37040/0 (socket says 172.21.15.97:37040) 2024-04-23T10:41:19.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.938+0000 7f587d56a640 1 -- 172.21.15.97:0/533469875 learned_addr learned my addr 172.21.15.97:0/533469875 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:41:19.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.938+0000 7f587d56a640 1 -- 172.21.15.97:0/533469875 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55716073f090 msgr2=0x557160757730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:19.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.938+0000 7f587d56a640 1 --2- 172.21.15.97:0/533469875 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55716073f090 0x557160757730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:19.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.938+0000 7f587d56a640 1 -- 172.21.15.97:0/533469875 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571607340a0 msgr2=0x55716074ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:19.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.938+0000 7f587dd6b640 1 --2- 172.21.15.97:0/533469875 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571607340a0 0x55716074ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:19.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.938+0000 7f587d56a640 1 --2- 172.21.15.97:0/533469875 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571607340a0 0x55716074ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:19.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.938+0000 7f587d56a640 1 -- 172.21.15.97:0/533469875 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571607301f0 con 0x5571607349c0 2024-04-23T10:41:19.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.938+0000 7f587dd6b640 1 --2- 172.21.15.97:0/533469875 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571607340a0 0x55716074ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:41:19.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.938+0000 7f587d56a640 1 --2- 172.21.15.97:0/533469875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5571607349c0 0x5571607571f0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f586800f640 tx=0x7f586800d580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6e4485b0f0633af server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:41:19.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.938+0000 7f58637fe640 1 -- 172.21.15.97:0/533469875 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5868016040 con 0x5571607349c0 2024-04-23T10:41:19.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.939+0000 7f58637fe640 1 -- 172.21.15.97:0/533469875 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f586800b6f0 con 0x5571607349c0 2024-04-23T10:41:19.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.939+0000 7f58637fe640 1 -- 172.21.15.97:0/533469875 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5868023a20 con 0x5571607349c0 2024-04-23T10:41:19.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.939+0000 7f5882dbd740 1 -- 172.21.15.97:0/533469875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5571607349c0 msgr2=0x5571607571f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:19.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.939+0000 7f5882dbd740 1 --2- 172.21.15.97:0/533469875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5571607349c0 0x5571607571f0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f586800f640 tx=0x7f586800d580 comp rx=0 tx=0).stop 2024-04-23T10:41:19.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.939+0000 7f5882dbd740 1 -- 172.21.15.97:0/533469875 shutdown_connections 2024-04-23T10:41:19.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.939+0000 7f5882dbd740 1 --2- 172.21.15.97:0/533469875 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55716073f090 0x557160757730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:19.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.939+0000 7f5882dbd740 1 --2- 172.21.15.97:0/533469875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5571607349c0 0x5571607571f0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:19.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.939+0000 7f5882dbd740 1 --2- 172.21.15.97:0/533469875 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5571607340a0 0x55716074ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:19.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.939+0000 7f5882dbd740 1 -- 172.21.15.97:0/533469875 >> 172.21.15.97:0/533469875 conn(0x5571607303c0 msgr2=0x55716072f600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:41:19.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.939+0000 7f5882dbd740 1 -- 172.21.15.97:0/533469875 shutdown_connections 2024-04-23T10:41:19.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.939+0000 7f5882dbd740 1 -- 172.21.15.97:0/533469875 wait complete. 2024-04-23T10:41:19.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.939+0000 7f5882dbd740 1 -- start start 2024-04-23T10:41:19.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.940+0000 7f5882dbd740 1 Processor -- start 2024-04-23T10:41:19.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.940+0000 7f5882dbd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5571607340a0 0x5571607471f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:19.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.940+0000 7f5882dbd740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5571607349c0 0x557160747730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:19.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.940+0000 7f5882dbd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55716073f090 0x557160744240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:19.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.940+0000 7f5882dbd740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f58781b11a0 con 0x5571607349c0 2024-04-23T10:41:19.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.940+0000 7f5882dbd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f58781b0ad0 con 0x55716073f090 2024-04-23T10:41:19.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.940+0000 7f5882dbd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f58781add30 con 0x5571607340a0 2024-04-23T10:41:19.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.940+0000 7f5882dbd740 10 client.0 _ll_register_callbacks cb 0x7f586800de40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:41:19.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.940+0000 7f587d56a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5571607349c0 0x557160747730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:19.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.940+0000 7f587d56a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5571607349c0 0x557160747730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37046/0 (socket says 172.21.15.97:37046) 2024-04-23T10:41:19.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.940+0000 7f587d56a640 1 -- 172.21.15.97:0/2105116390 learned_addr learned my addr 172.21.15.97:0/2105116390 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:41:19.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.940+0000 7f587e56c640 1 --2- 172.21.15.97:0/2105116390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55716073f090 0x557160744240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:19.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.940+0000 7f587dd6b640 1 --2- 172.21.15.97:0/2105116390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5571607340a0 0x5571607471f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:19.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.940+0000 7f587d56a640 1 -- 172.21.15.97:0/2105116390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5571607340a0 msgr2=0x5571607471f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:19.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.940+0000 7f587d56a640 1 --2- 172.21.15.97:0/2105116390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5571607340a0 0x5571607471f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:19.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.940+0000 7f587d56a640 1 -- 172.21.15.97:0/2105116390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55716073f090 msgr2=0x557160744240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:19.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.940+0000 7f587d56a640 1 --2- 172.21.15.97:0/2105116390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55716073f090 0x557160744240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:19.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.940+0000 7f587d56a640 1 -- 172.21.15.97:0/2105116390 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557160744a70 con 0x5571607349c0 2024-04-23T10:41:19.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.940+0000 7f587dd6b640 1 --2- 172.21.15.97:0/2105116390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5571607340a0 0x5571607471f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:41:19.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.941+0000 7f587d56a640 1 --2- 172.21.15.97:0/2105116390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5571607349c0 0x557160747730 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f586800f670 tx=0x7f586800dfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82be4ffdadf4296e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:41:19.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.941+0000 7f58617fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:41:19.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.941+0000 7f58617fa640 1 -- 172.21.15.97:0/2105116390 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f586803d060 con 0x5571607349c0 2024-04-23T10:41:19.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.941+0000 7f58617fa640 1 -- 172.21.15.97:0/2105116390 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8484+0+0 (secure 0 0 0) 0x7f5868023890 con 0x5571607349c0 2024-04-23T10:41:19.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.941+0000 7f5882dbd740 1 -- 172.21.15.97:0/2105116390 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f586800b6f0 con 0x5571607349c0 2024-04-23T10:41:19.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.941+0000 7f58617fa640 1 -- 172.21.15.97:0/2105116390 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5868016040 con 0x5571607349c0 2024-04-23T10:41:19.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.941+0000 7f5882dbd740 10 client.17655 fetch_fsmap learned FSMap version 26 2024-04-23T10:41:19.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.941+0000 7f5882dbd740 1 -- 172.21.15.97:0/2105116390 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5571607b6560 con 0x5571607349c0 2024-04-23T10:41:19.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.941+0000 7f58617fa640 1 -- 172.21.15.97:0/2105116390 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f586800b6f0 con 0x5571607349c0 2024-04-23T10:41:19.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.941+0000 7f58617fa640 1 -- 172.21.15.97:0/2105116390 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f586802c070 con 0x5571607349c0 2024-04-23T10:41:19.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.941+0000 7f5882dbd740 10 client.17655 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:41:19.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.941+0000 7f5882dbd740 10 client.17655 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:41:19.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.941+0000 7f5882dbd740 20 client.17655 populate_metadata read hostname 'smithi097' 2024-04-23T10:41:19.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.941+0000 7f5882dbd740 1 --2- 172.21.15.97:0/2105116390 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5571607b6bb0 0x5571607b8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:19.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.941+0000 7f5882dbd740 4 client.17655 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 84.65, "await_for": 42.0}] 2024-04-23T10:41:19.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.941+0000 7f5882dbd740 1 -- 172.21.15.97:0/2105116390 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 84.65, "await_for": 42.0}) -- 0x5571606a0c00 con 0x5571607b6bb0 2024-04-23T10:41:19.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.942+0000 7f587dd6b640 1 --2- 172.21.15.97:0/2105116390 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5571607b6bb0 0x5571607b8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:19.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.942+0000 7f587dd6b640 1 --2- 172.21.15.97:0/2105116390 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5571607b6bb0 0x5571607b8fd0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ddd7fad9a5e6997 server_cookie=4e9fe6450d0f3631 in_seq=0 out_seq=0 2024-04-23T10:41:19.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.942+0000 7f58617fa640 10 client.17655 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:41:19.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.956+0000 7f58617fa640 1 -- 172.21.15.97:0/2105116390 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5571606a0c00 con 0x5571607b6bb0 2024-04-23T10:41:19.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.956+0000 7f58617fa640 10 client.17655 handle_command_reply: tid=0 2024-04-23T10:41:19.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.956+0000 7f5882dbd740 1 client.17655 shutdown 2024-04-23T10:41:19.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.956+0000 7f5882dbd740 1 -- 172.21.15.97:0/2105116390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5571607349c0 msgr2=0x557160747730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:19.955 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.956+0000 7f5882dbd740 1 --2- 172.21.15.97:0/2105116390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5571607349c0 0x557160747730 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f586800f670 tx=0x7f586800dfa0 comp rx=0 tx=0).stop 2024-04-23T10:41:19.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.956+0000 7f5882dbd740 1 -- 172.21.15.97:0/2105116390 shutdown_connections 2024-04-23T10:41:19.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.956+0000 7f5882dbd740 1 --2- 172.21.15.97:0/2105116390 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5571607b6bb0 0x5571607b8fd0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:19.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.956+0000 7f5882dbd740 1 --2- 172.21.15.97:0/2105116390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55716073f090 0x557160744240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:19.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.956+0000 7f58617fa640 0 client.17655 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:41:19.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.956+0000 7f5882dbd740 1 --2- 172.21.15.97:0/2105116390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5571607349c0 0x557160747730 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:19.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.956+0000 7f5882dbd740 1 --2- 172.21.15.97:0/2105116390 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5571607340a0 0x5571607471f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:19.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.956+0000 7f5882dbd740 1 -- 172.21.15.97:0/2105116390 >> 172.21.15.97:0/2105116390 conn(0x5571607303c0 msgr2=0x55716073c050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:41:19.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.956+0000 7f5882dbd740 1 -- 172.21.15.97:0/2105116390 shutdown_connections 2024-04-23T10:41:19.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.956+0000 7f5882dbd740 1 -- 172.21.15.97:0/2105116390 wait complete. 2024-04-23T10:41:19.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.956+0000 7f5882dbd740 20 client.17655 trim_cache size 0 max 16384 2024-04-23T10:41:19.956 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-23T10:41:19.957 INFO:teuthology.orchestra.run.smithi097.stdout: "epoch": 26, 2024-04-23T10:41:19.957 INFO:teuthology.orchestra.run.smithi097.stdout: "leader": 24488, 2024-04-23T10:41:19.957 INFO:teuthology.orchestra.run.smithi097.stdout: "set_version": 62, 2024-04-23T10:41:19.957 INFO:teuthology.orchestra.run.smithi097.stdout: "sets": { 2024-04-23T10:41:19.957 INFO:teuthology.orchestra.run.smithi097.stdout: "8d2d887a": { 2024-04-23T10:41:19.957 INFO:teuthology.orchestra.run.smithi097.stdout: "version": 62, 2024-04-23T10:41:19.957 INFO:teuthology.orchestra.run.smithi097.stdout: "age_ref": 0.0, 2024-04-23T10:41:19.957 INFO:teuthology.orchestra.run.smithi097.stdout: "state": { 2024-04-23T10:41:19.957 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "QUIESCED", 2024-04-23T10:41:19.957 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0 2024-04-23T10:41:19.957 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:41:19.957 INFO:teuthology.orchestra.run.smithi097.stdout: "timeout": 60.0, 2024-04-23T10:41:19.957 INFO:teuthology.orchestra.run.smithi097.stdout: "expiration": 84.7, 2024-04-23T10:41:19.957 INFO:teuthology.orchestra.run.smithi097.stdout: "members": { 2024-04-23T10:41:19.957 INFO:teuthology.orchestra.run.smithi097.stdout: "file:/": { 2024-04-23T10:41:19.957 INFO:teuthology.orchestra.run.smithi097.stdout: "excluded": false, 2024-04-23T10:41:19.958 INFO:teuthology.orchestra.run.smithi097.stdout: "state": { 2024-04-23T10:41:19.958 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "QUIESCED", 2024-04-23T10:41:19.958 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0 2024-04-23T10:41:19.958 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:41:19.958 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:41:19.958 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:41:19.958 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:41:19.958 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:41:19.958 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-23T10:41:19.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.957+0000 7f587ffcf640 1 -- 172.21.15.97:0/3618672798 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f58640a3d20 msgr2=0x7f58640a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:19.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.957+0000 7f587ffcf640 1 --2- 172.21.15.97:0/3618672798 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f58640a3d20 0x7f58640a61e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5870011c50 tx=0x7f587001a000 comp rx=0 tx=0).stop 2024-04-23T10:41:19.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.957+0000 7f587ffcf640 1 -- 172.21.15.97:0/3618672798 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5878100880 msgr2=0x7f587810ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:19.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.957+0000 7f587ffcf640 1 --2- 172.21.15.97:0/3618672798 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5878100880 0x7f587810ecd0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f5874006010 tx=0x7f58740042f0 comp rx=0 tx=0).stop 2024-04-23T10:41:19.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.957+0000 7f587ffcf640 1 -- 172.21.15.97:0/3618672798 shutdown_connections 2024-04-23T10:41:19.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.957+0000 7f587ffcf640 1 --2- 172.21.15.97:0/3618672798 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f58640a3d20 0x7f58640a61e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:19.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.957+0000 7f587ffcf640 1 --2- 172.21.15.97:0/3618672798 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5878106670 0x7f587810f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:19.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.957+0000 7f587ffcf640 1 --2- 172.21.15.97:0/3618672798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5878101240 0x7f587810f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:19.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.957+0000 7f587ffcf640 1 --2- 172.21.15.97:0/3618672798 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5878100880 0x7f587810ecd0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:19.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.957+0000 7f587ffcf640 1 -- 172.21.15.97:0/3618672798 >> 172.21.15.97:0/3618672798 conn(0x7f58780761d0 msgr2=0x7f58780ff730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:41:19.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.957+0000 7f587ffcf640 1 -- 172.21.15.97:0/3618672798 shutdown_connections 2024-04-23T10:41:19.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:19.957+0000 7f587ffcf640 1 -- 172.21.15.97:0/3618672798 wait complete. 2024-04-23T10:41:19.966 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '8d2d887a', quiesce took 0.6 seconds. Will release after: 24.65 2024-04-23T10:41:20.450 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.448+0000 7f974a7f0640 1 -- 172.21.15.97:0/25841840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9744102b80 msgr2=0x7f9744106bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:20.450 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.448+0000 7f974a7f0640 1 --2- 172.21.15.97:0/25841840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9744102b80 0x7f9744106bc0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f9738009900 tx=0x7f973802a410 comp rx=0 tx=0).stop 2024-04-23T10:41:20.450 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.448+0000 7f974a7f0640 1 -- 172.21.15.97:0/25841840 shutdown_connections 2024-04-23T10:41:20.450 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.448+0000 7f974a7f0640 1 --2- 172.21.15.97:0/25841840 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f97441072a0 0x7f974410ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:20.450 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.448+0000 7f974a7f0640 1 --2- 172.21.15.97:0/25841840 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9744102b80 0x7f9744106bc0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:20.451 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.448+0000 7f974a7f0640 1 --2- 172.21.15.97:0/25841840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97441021d0 0x7f97441025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:20.451 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.448+0000 7f974a7f0640 1 -- 172.21.15.97:0/25841840 >> 172.21.15.97:0/25841840 conn(0x7f97440793e0 msgr2=0x7f97440797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:41:20.451 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.448+0000 7f974a7f0640 1 -- 172.21.15.97:0/25841840 shutdown_connections 2024-04-23T10:41:20.451 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.448+0000 7f974a7f0640 1 -- 172.21.15.97:0/25841840 wait complete. 2024-04-23T10:41:20.451 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.448+0000 7f974a7f0640 1 Processor -- start 2024-04-23T10:41:20.451 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.448+0000 7f974a7f0640 1 -- start start 2024-04-23T10:41:20.451 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.449+0000 7f974a7f0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97441021d0 0x7f97441986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:20.451 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.449+0000 7f974a7f0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97441072a0 0x7f9744198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:20.451 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.449+0000 7f974a7f0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f974419cfa0 0x7f974419d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:20.451 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.449+0000 7f974a7f0640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f97441121e0 con 0x7f97441021d0 2024-04-23T10:41:20.451 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.449+0000 7f974a7f0640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9744112060 con 0x7f974419cfa0 2024-04-23T10:41:20.451 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.449+0000 7f974a7f0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9744112360 con 0x7f97441072a0 2024-04-23T10:41:20.452 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.449+0000 7f9743fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97441021d0 0x7f97441986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:20.452 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.449+0000 7f9743fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97441021d0 0x7f97441986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37062/0 (socket says 172.21.15.97:37062) 2024-04-23T10:41:20.452 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.449+0000 7f9743fff640 1 -- 172.21.15.97:0/656444023 learned_addr learned my addr 172.21.15.97:0/656444023 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:41:20.452 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.449+0000 7f9748d8d640 1 --2- 172.21.15.97:0/656444023 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f974419cfa0 0x7f974419d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:20.452 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.449+0000 7f97437fe640 1 --2- 172.21.15.97:0/656444023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97441072a0 0x7f9744198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:20.452 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.449+0000 7f9743fff640 1 -- 172.21.15.97:0/656444023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97441072a0 msgr2=0x7f9744198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:20.452 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.449+0000 7f9743fff640 1 --2- 172.21.15.97:0/656444023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97441072a0 0x7f9744198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:20.452 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.449+0000 7f9743fff640 1 -- 172.21.15.97:0/656444023 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f974419cfa0 msgr2=0x7f974419d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:20.452 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.449+0000 7f9743fff640 1 --2- 172.21.15.97:0/656444023 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f974419cfa0 0x7f974419d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:20.452 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.449+0000 7f9743fff640 1 -- 172.21.15.97:0/656444023 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f974419da90 con 0x7f97441021d0 2024-04-23T10:41:20.452 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.449+0000 7f97437fe640 1 --2- 172.21.15.97:0/656444023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97441072a0 0x7f9744198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:41:20.452 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.449+0000 7f9743fff640 1 --2- 172.21.15.97:0/656444023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97441021d0 0x7f97441986e0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f973400d770 tx=0x7f973400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:41:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.449+0000 7f97417fa640 1 -- 172.21.15.97:0/656444023 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9734017070 con 0x7f97441021d0 2024-04-23T10:41:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.449+0000 7f97417fa640 1 -- 172.21.15.97:0/656444023 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97340054d0 con 0x7f97441021d0 2024-04-23T10:41:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.449+0000 7f974a7f0640 1 -- 172.21.15.97:0/656444023 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f974419dd80 con 0x7f97441021d0 2024-04-23T10:41:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.449+0000 7f97417fa640 1 -- 172.21.15.97:0/656444023 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9734004c30 con 0x7f97441021d0 2024-04-23T10:41:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.450+0000 7f974a7f0640 1 -- 172.21.15.97:0/656444023 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97441a5520 con 0x7f97441021d0 2024-04-23T10:41:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.452+0000 7f97417fa640 1 -- 172.21.15.97:0/656444023 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9734004e10 con 0x7f97441021d0 2024-04-23T10:41:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.452+0000 7f974d5dd740 1 Processor -- start 2024-04-23T10:41:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.452+0000 7f974d5dd740 1 -- start start 2024-04-23T10:41:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.452+0000 7f97417fa640 1 --2- 172.21.15.97:0/656444023 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f972c0a3dc0 0x7f972c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.452+0000 7f974d5dd740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eccc044620 0x55eccc044a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.452+0000 7f974d5dd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eccc044f40 0x55eccc04f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.452+0000 7f974d5dd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eccc04f800 0x55eccc051bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.452+0000 7f974d5dd740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55eccbfa9970 con 0x55eccc044620 2024-04-23T10:41:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.452+0000 7f974d5dd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55eccbf464a0 con 0x55eccc044f40 2024-04-23T10:41:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.452+0000 7f974d5dd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55eccbfa5930 con 0x55eccc04f800 2024-04-23T10:41:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.452+0000 7f9743fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eccc044620 0x55eccc044a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.452+0000 7f9748d8d640 1 --2- 172.21.15.97:0/656444023 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f972c0a3dc0 0x7f972c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.452+0000 7f9743fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eccc044620 0x55eccc044a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37076/0 (socket says 172.21.15.97:37076) 2024-04-23T10:41:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.452+0000 7f9743fff640 1 -- 172.21.15.97:0/3720530951 learned_addr learned my addr 172.21.15.97:0/3720530951 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:41:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.452+0000 7f97417fa640 1 -- 172.21.15.97:0/656444023 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8484+0+0 (secure 0 0 0) 0x7f97340d0360 con 0x7f97441021d0 2024-04-23T10:41:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.452+0000 7f97437fe640 1 --2- 172.21.15.97:0/3720530951 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eccc044f40 0x55eccc04f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.453+0000 7f9748d8d640 1 --2- 172.21.15.97:0/3720530951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eccc04f800 0x55eccc051bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.453+0000 7f9743fff640 1 -- 172.21.15.97:0/3720530951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eccc04f800 msgr2=0x55eccc051bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.453+0000 7f9743fff640 1 --2- 172.21.15.97:0/3720530951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eccc04f800 0x55eccc051bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.453+0000 7f9743fff640 1 -- 172.21.15.97:0/3720530951 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eccc044f40 msgr2=0x55eccc04f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.453+0000 7f9743fff640 1 --2- 172.21.15.97:0/3720530951 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eccc044f40 0x55eccc04f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.453+0000 7f9743fff640 1 -- 172.21.15.97:0/3720530951 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eccbcbc210 con 0x55eccc044620 2024-04-23T10:41:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.453+0000 7f9748d8d640 1 --2- 172.21.15.97:0/656444023 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f972c0a3dc0 0x7f972c0a6280 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f973c0092f0 tx=0x7f973c014b10 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:41:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.453+0000 7f9743fff640 1 --2- 172.21.15.97:0/3720530951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eccc044620 0x55eccc044a00 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f97340a4a60 tx=0x7f973409c1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac28b98c1abb0440 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:41:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.453+0000 7f972a7fc640 1 -- 172.21.15.97:0/3720530951 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9734005680 con 0x55eccc044620 2024-04-23T10:41:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.453+0000 7f972a7fc640 1 -- 172.21.15.97:0/3720530951 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f973409ca20 con 0x55eccc044620 2024-04-23T10:41:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.453+0000 7f972a7fc640 1 -- 172.21.15.97:0/3720530951 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97340cb0b0 con 0x55eccc044620 2024-04-23T10:41:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.454+0000 7f974d5dd740 1 -- 172.21.15.97:0/3720530951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eccc044620 msgr2=0x55eccc044a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.454+0000 7f974d5dd740 1 --2- 172.21.15.97:0/3720530951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eccc044620 0x55eccc044a00 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f97340a4a60 tx=0x7f973409c1b0 comp rx=0 tx=0).stop 2024-04-23T10:41:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.454+0000 7f974d5dd740 1 -- 172.21.15.97:0/3720530951 shutdown_connections 2024-04-23T10:41:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.454+0000 7f974d5dd740 1 --2- 172.21.15.97:0/3720530951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eccc04f800 0x55eccc051bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.454+0000 7f974d5dd740 1 --2- 172.21.15.97:0/3720530951 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eccc044f40 0x55eccc04f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.454+0000 7f974d5dd740 1 --2- 172.21.15.97:0/3720530951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eccc044620 0x55eccc044a00 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.454+0000 7f974d5dd740 1 -- 172.21.15.97:0/3720530951 >> 172.21.15.97:0/3720530951 conn(0x55eccc040b30 msgr2=0x55eccc040f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:41:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.454+0000 7f974d5dd740 1 -- 172.21.15.97:0/3720530951 shutdown_connections 2024-04-23T10:41:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.454+0000 7f974d5dd740 1 -- 172.21.15.97:0/3720530951 wait complete. 2024-04-23T10:41:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.454+0000 7f974d5dd740 1 -- start start 2024-04-23T10:41:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.454+0000 7f974d5dd740 1 Processor -- start 2024-04-23T10:41:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.454+0000 7f974d5dd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eccc044620 0x55eccc04f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.454+0000 7f974d5dd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eccc044f40 0x55eccc04c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.455+0000 7f974d5dd740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eccc04f800 0x55eccc04ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.455+0000 7f974d5dd740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55eccc0523d0 con 0x55eccc04f800 2024-04-23T10:41:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.455+0000 7f974d5dd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55eccbf440b0 con 0x55eccc044620 2024-04-23T10:41:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.455+0000 7f974d5dd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55eccc052550 con 0x55eccc044f40 2024-04-23T10:41:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.455+0000 7f974d5dd740 10 client.0 _ll_register_callbacks cb 0x55eccbf31d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:41:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.455+0000 7f9748d8d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eccc04f800 0x55eccc04ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.455+0000 7f9748d8d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eccc04f800 0x55eccc04ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37080/0 (socket says 172.21.15.97:37080) 2024-04-23T10:41:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.455+0000 7f9748d8d640 1 -- 172.21.15.97:0/3176626902 learned_addr learned my addr 172.21.15.97:0/3176626902 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:41:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.455+0000 7f9743fff640 1 --2- 172.21.15.97:0/3176626902 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eccc044620 0x55eccc04f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.455+0000 7f97437fe640 1 --2- 172.21.15.97:0/3176626902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eccc044f40 0x55eccc04c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.455+0000 7f9748d8d640 1 -- 172.21.15.97:0/3176626902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eccc044f40 msgr2=0x55eccc04c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.455+0000 7f9748d8d640 1 --2- 172.21.15.97:0/3176626902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eccc044f40 0x55eccc04c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.455+0000 7f9748d8d640 1 -- 172.21.15.97:0/3176626902 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eccc044620 msgr2=0x55eccc04f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.455+0000 7f9748d8d640 1 --2- 172.21.15.97:0/3176626902 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eccc044620 0x55eccc04f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.455+0000 7f9748d8d640 1 -- 172.21.15.97:0/3176626902 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eccbd88620 con 0x55eccc04f800 2024-04-23T10:41:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.455+0000 7f9748d8d640 1 --2- 172.21.15.97:0/3176626902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eccc04f800 0x55eccc04ca40 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f973c012bf0 tx=0x7f973c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc9591c3ccd6e169 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:41:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.455+0000 7f97237fe640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:41:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.455+0000 7f97237fe640 1 -- 172.21.15.97:0/3176626902 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f973c026070 con 0x55eccc04f800 2024-04-23T10:41:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.455+0000 7f97237fe640 1 -- 172.21.15.97:0/3176626902 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8484+0+0 (secure 0 0 0) 0x7f973c004b20 con 0x55eccc04f800 2024-04-23T10:41:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.455+0000 7f974d5dd740 1 -- 172.21.15.97:0/3176626902 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f973409ca20 con 0x55eccc04f800 2024-04-23T10:41:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.455+0000 7f97237fe640 1 -- 172.21.15.97:0/3176626902 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f973c004d20 con 0x55eccc04f800 2024-04-23T10:41:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.455+0000 7f974d5dd740 10 client.17667 fetch_fsmap learned FSMap version 26 2024-04-23T10:41:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.455+0000 7f974d5dd740 1 -- 172.21.15.97:0/3176626902 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55eccc05e390 con 0x55eccc04f800 2024-04-23T10:41:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.456+0000 7f97237fe640 1 -- 172.21.15.97:0/3176626902 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f973409ca20 con 0x55eccc04f800 2024-04-23T10:41:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.456+0000 7f97237fe640 1 -- 172.21.15.97:0/3176626902 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f973c00f040 con 0x55eccc04f800 2024-04-23T10:41:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.456+0000 7f974d5dd740 10 client.17667 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:41:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.456+0000 7f974d5dd740 10 client.17667 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:41:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.456+0000 7f974d5dd740 20 client.17667 populate_metadata read hostname 'smithi097' 2024-04-23T10:41:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.456+0000 7f974d5dd740 1 --2- 172.21.15.97:0/3176626902 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55eccc05f850 0x55eccc061c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.456+0000 7f974d5dd740 4 client.17667 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:41:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.456+0000 7f974d5dd740 1 -- 172.21.15.97:0/3176626902 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eccbfb7390 con 0x55eccc05f850 2024-04-23T10:41:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.456+0000 7f9743fff640 1 --2- 172.21.15.97:0/3176626902 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55eccc05f850 0x55eccc061c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.456+0000 7f97237fe640 10 client.17667 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:41:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.456+0000 7f9743fff640 1 --2- 172.21.15.97:0/3176626902 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55eccc05f850 0x55eccc061c40 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ece7b93b8ec8c2d7 server_cookie=d44a2d4bd2a26637 in_seq=0 out_seq=0 2024-04-23T10:41:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.457+0000 7f97237fe640 1 -- 172.21.15.97:0/3176626902 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55eccbfb7390 con 0x55eccc05f850 2024-04-23T10:41:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.457+0000 7f97237fe640 10 client.17667 handle_command_reply: tid=0 2024-04-23T10:41:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.457+0000 7f974d5dd740 1 client.17667 shutdown 2024-04-23T10:41:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.457+0000 7f974d5dd740 1 -- 172.21.15.97:0/3176626902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eccc04f800 msgr2=0x55eccc04ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.457+0000 7f974d5dd740 1 --2- 172.21.15.97:0/3176626902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eccc04f800 0x55eccc04ca40 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f973c012bf0 tx=0x7f973c01a000 comp rx=0 tx=0).stop 2024-04-23T10:41:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.457+0000 7f974d5dd740 1 -- 172.21.15.97:0/3176626902 shutdown_connections 2024-04-23T10:41:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.458+0000 7f974d5dd740 1 --2- 172.21.15.97:0/3176626902 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55eccc05f850 0x55eccc061c40 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.458+0000 7f974d5dd740 1 --2- 172.21.15.97:0/3176626902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eccc04f800 0x55eccc04ca40 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.458+0000 7f974d5dd740 1 --2- 172.21.15.97:0/3176626902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eccc044f40 0x55eccc04c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.458+0000 7f97237fe640 0 client.17667 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:41:20.468 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.458+0000 7f974d5dd740 1 --2- 172.21.15.97:0/3176626902 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eccc044620 0x55eccc04f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:20.468 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.458+0000 7f974d5dd740 1 -- 172.21.15.97:0/3176626902 >> 172.21.15.97:0/3176626902 conn(0x55eccc040b30 msgr2=0x55eccc0468b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:41:20.468 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.458+0000 7f974d5dd740 1 -- 172.21.15.97:0/3176626902 shutdown_connections 2024-04-23T10:41:20.468 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.458+0000 7f974d5dd740 1 -- 172.21.15.97:0/3176626902 wait complete. 2024-04-23T10:41:20.468 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.458+0000 7f974d5dd740 20 client.17667 trim_cache size 0 max 16384 2024-04-23T10:41:20.477 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.472+0000 7f974d5dd740 1 Processor -- start 2024-04-23T10:41:20.477 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.472+0000 7f974d5dd740 1 -- start start 2024-04-23T10:41:20.477 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.472+0000 7f974d5dd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eccc044620 0x55eccc05b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:20.477 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.472+0000 7f974d5dd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eccc044f40 0x55eccc067530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:20.477 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.472+0000 7f974d5dd740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eccc04f800 0x55eccc067a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:20.477 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.472+0000 7f974d5dd740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f97441adb20 con 0x55eccc04f800 2024-04-23T10:41:20.478 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.472+0000 7f974d5dd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f97441af660 con 0x55eccc044f40 2024-04-23T10:41:20.478 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.472+0000 7f974d5dd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f97441ae1f0 con 0x55eccc044620 2024-04-23T10:41:20.478 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.473+0000 7f9748d8d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eccc04f800 0x55eccc067a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:20.478 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.473+0000 7f9748d8d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eccc04f800 0x55eccc067a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37084/0 (socket says 172.21.15.97:37084) 2024-04-23T10:41:20.478 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.473+0000 7f9748d8d640 1 -- 172.21.15.97:0/3151350007 learned_addr learned my addr 172.21.15.97:0/3151350007 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:41:20.478 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.473+0000 7f9743fff640 1 --2- 172.21.15.97:0/3151350007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eccc044620 0x55eccc05b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:20.478 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.473+0000 7f9748d8d640 1 -- 172.21.15.97:0/3151350007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eccc044620 msgr2=0x55eccc05b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:20.478 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.473+0000 7f9748d8d640 1 --2- 172.21.15.97:0/3151350007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eccc044620 0x55eccc05b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:20.478 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.473+0000 7f9748d8d640 1 -- 172.21.15.97:0/3151350007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eccc044f40 msgr2=0x55eccc067530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:20.479 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.473+0000 7f9748d8d640 1 --2- 172.21.15.97:0/3151350007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eccc044f40 0x55eccc067530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:20.479 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.473+0000 7f9748d8d640 1 -- 172.21.15.97:0/3151350007 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eccc061fb0 con 0x55eccc04f800 2024-04-23T10:41:20.479 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.473+0000 7f9743fff640 1 --2- 172.21.15.97:0/3151350007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eccc044620 0x55eccc05b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:41:20.479 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.473+0000 7f9748d8d640 1 --2- 172.21.15.97:0/3151350007 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eccc04f800 0x55eccc067a70 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f973c006010 tx=0x7f973c009d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca0196f034b5d112 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:41:20.479 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.473+0000 7f9729ffb640 1 -- 172.21.15.97:0/3151350007 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f973c03a070 con 0x55eccc04f800 2024-04-23T10:41:20.479 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.473+0000 7f9729ffb640 1 -- 172.21.15.97:0/3151350007 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f973c00f040 con 0x55eccc04f800 2024-04-23T10:41:20.479 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.473+0000 7f9729ffb640 1 -- 172.21.15.97:0/3151350007 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f973c026070 con 0x55eccc04f800 2024-04-23T10:41:20.479 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.474+0000 7f974d5dd740 1 -- 172.21.15.97:0/3151350007 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eccc04f800 msgr2=0x55eccc067a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:20.479 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.474+0000 7f974d5dd740 1 --2- 172.21.15.97:0/3151350007 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eccc04f800 0x55eccc067a70 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f973c006010 tx=0x7f973c009d30 comp rx=0 tx=0).stop 2024-04-23T10:41:20.479 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.474+0000 7f974d5dd740 1 -- 172.21.15.97:0/3151350007 shutdown_connections 2024-04-23T10:41:20.479 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.474+0000 7f974d5dd740 1 --2- 172.21.15.97:0/3151350007 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eccc04f800 0x55eccc067a70 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:20.479 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.474+0000 7f974d5dd740 1 --2- 172.21.15.97:0/3151350007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eccc044f40 0x55eccc067530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:20.480 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.474+0000 7f974d5dd740 1 --2- 172.21.15.97:0/3151350007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eccc044620 0x55eccc05b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:20.480 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.474+0000 7f974d5dd740 1 -- 172.21.15.97:0/3151350007 >> 172.21.15.97:0/3151350007 conn(0x55eccc040b30 msgr2=0x55eccc03f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:41:20.480 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.474+0000 7f974d5dd740 1 -- 172.21.15.97:0/3151350007 shutdown_connections 2024-04-23T10:41:20.480 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.474+0000 7f974d5dd740 1 -- 172.21.15.97:0/3151350007 wait complete. 2024-04-23T10:41:20.480 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.474+0000 7f974d5dd740 1 -- start start 2024-04-23T10:41:20.480 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.474+0000 7f974d5dd740 1 Processor -- start 2024-04-23T10:41:20.480 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.474+0000 7f974d5dd740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eccc044620 0x55eccc057a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:20.480 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.475+0000 7f974d5dd740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eccc044f40 0x55eccc054dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:20.480 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.475+0000 7f974d5dd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eccc04f800 0x55eccc055310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:20.480 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.475+0000 7f974d5dd740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f97441aef90 con 0x55eccc044620 2024-04-23T10:41:20.480 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.475+0000 7f974d5dd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f97441ae8c0 con 0x55eccc04f800 2024-04-23T10:41:20.480 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.475+0000 7f974d5dd740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f97441abb20 con 0x55eccc044f40 2024-04-23T10:41:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.475+0000 7f974d5dd740 10 client.0 _ll_register_callbacks cb 0x7f973c012ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:41:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.475+0000 7f9743fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eccc044620 0x55eccc057a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.475+0000 7f9743fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eccc044620 0x55eccc057a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37088/0 (socket says 172.21.15.97:37088) 2024-04-23T10:41:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.475+0000 7f9743fff640 1 -- 172.21.15.97:0/4198337590 learned_addr learned my addr 172.21.15.97:0/4198337590 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:41:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.475+0000 7f97437fe640 1 --2- 172.21.15.97:0/4198337590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eccc044f40 0x55eccc054dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.475+0000 7f9748d8d640 1 --2- 172.21.15.97:0/4198337590 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eccc04f800 0x55eccc055310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.475+0000 7f9743fff640 1 -- 172.21.15.97:0/4198337590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eccc044f40 msgr2=0x55eccc054dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.475+0000 7f9743fff640 1 --2- 172.21.15.97:0/4198337590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eccc044f40 0x55eccc054dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.475+0000 7f9743fff640 1 -- 172.21.15.97:0/4198337590 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eccc04f800 msgr2=0x55eccc055310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.475+0000 7f9743fff640 1 --2- 172.21.15.97:0/4198337590 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eccc04f800 0x55eccc055310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.475+0000 7f9743fff640 1 -- 172.21.15.97:0/4198337590 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eccc0559d0 con 0x55eccc044620 2024-04-23T10:41:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.475+0000 7f97437fe640 1 --2- 172.21.15.97:0/4198337590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eccc044f40 0x55eccc054dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:41:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.475+0000 7f9743fff640 1 --2- 172.21.15.97:0/4198337590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eccc044620 0x55eccc057a40 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f9734009f40 tx=0x7f9734009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d02f751176a33198 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:41:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.475+0000 7f97237fe640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:41:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.475+0000 7f97237fe640 1 -- 172.21.15.97:0/4198337590 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f973409ca20 con 0x55eccc044620 2024-04-23T10:41:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.475+0000 7f97237fe640 1 -- 172.21.15.97:0/4198337590 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8484+0+0 (secure 0 0 0) 0x7f9734098990 con 0x55eccc044620 2024-04-23T10:41:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.475+0000 7f974d5dd740 1 -- 172.21.15.97:0/4198337590 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f973c00f040 con 0x55eccc044620 2024-04-23T10:41:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.475+0000 7f97237fe640 1 -- 172.21.15.97:0/4198337590 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f97340ba050 con 0x55eccc044620 2024-04-23T10:41:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.475+0000 7f974d5dd740 10 client.17673 fetch_fsmap learned FSMap version 26 2024-04-23T10:41:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.475+0000 7f974d5dd740 1 -- 172.21.15.97:0/4198337590 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55eccc0c5c90 con 0x55eccc044620 2024-04-23T10:41:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.475+0000 7f97237fe640 1 -- 172.21.15.97:0/4198337590 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f973c00f040 con 0x55eccc044620 2024-04-23T10:41:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.476+0000 7f97237fe640 1 -- 172.21.15.97:0/4198337590 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f9734063080 con 0x55eccc044620 2024-04-23T10:41:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.476+0000 7f974d5dd740 10 client.17673 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:41:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.476+0000 7f974d5dd740 10 client.17673 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:41:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.476+0000 7f974d5dd740 20 client.17673 populate_metadata read hostname 'smithi097' 2024-04-23T10:41:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.476+0000 7f974d5dd740 1 --2- 172.21.15.97:0/4198337590 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55eccc0c6090 0x55eccc0c8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.476+0000 7f974d5dd740 4 client.17673 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:41:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.476+0000 7f974d5dd740 1 -- 172.21.15.97:0/4198337590 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55eccbfb1200 con 0x55eccc0c6090 2024-04-23T10:41:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.476+0000 7f97437fe640 1 --2- 172.21.15.97:0/4198337590 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55eccc0c6090 0x55eccc0c8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.476+0000 7f97237fe640 10 client.17673 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:41:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.476+0000 7f97437fe640 1 --2- 172.21.15.97:0/4198337590 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55eccc0c6090 0x55eccc0c8480 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53095e434bf325c9 server_cookie=75aac6f1672cd740 in_seq=0 out_seq=0 2024-04-23T10:41:20.489 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.490+0000 7f97237fe640 1 -- 172.21.15.97:0/4198337590 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55eccbfb1200 con 0x55eccc0c6090 2024-04-23T10:41:20.490 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.490+0000 7f97237fe640 10 client.17673 handle_command_reply: tid=0 2024-04-23T10:41:20.490 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.490+0000 7f974d5dd740 1 client.17673 shutdown 2024-04-23T10:41:20.490 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.490+0000 7f974d5dd740 1 -- 172.21.15.97:0/4198337590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eccc044620 msgr2=0x55eccc057a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:20.490 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.490+0000 7f974d5dd740 1 --2- 172.21.15.97:0/4198337590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eccc044620 0x55eccc057a40 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f9734009f40 tx=0x7f9734009f90 comp rx=0 tx=0).stop 2024-04-23T10:41:20.490 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.490+0000 7f974d5dd740 1 -- 172.21.15.97:0/4198337590 shutdown_connections 2024-04-23T10:41:20.490 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.490+0000 7f974d5dd740 1 --2- 172.21.15.97:0/4198337590 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55eccc0c6090 0x55eccc0c8480 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:20.490 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.490+0000 7f974d5dd740 1 --2- 172.21.15.97:0/4198337590 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eccc04f800 0x55eccc055310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:20.490 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.490+0000 7f974d5dd740 1 --2- 172.21.15.97:0/4198337590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55eccc044f40 0x55eccc054dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:20.490 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.490+0000 7f97237fe640 0 client.17673 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:41:20.490 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.490+0000 7f974d5dd740 1 --2- 172.21.15.97:0/4198337590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eccc044620 0x55eccc057a40 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:20.491 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.490+0000 7f974d5dd740 1 -- 172.21.15.97:0/4198337590 >> 172.21.15.97:0/4198337590 conn(0x55eccc040b30 msgr2=0x55eccc03fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:41:20.491 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.490+0000 7f974d5dd740 1 -- 172.21.15.97:0/4198337590 shutdown_connections 2024-04-23T10:41:20.491 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.490+0000 7f974d5dd740 1 -- 172.21.15.97:0/4198337590 wait complete. 2024-04-23T10:41:20.491 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.490+0000 7f974d5dd740 20 client.17673 trim_cache size 0 max 16384 2024-04-23T10:41:20.491 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:41:20.491 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:41:20.491 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:41:20.491 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:41:20.492 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.491+0000 7f974a7f0640 1 -- 172.21.15.97:0/656444023 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f972c0a3dc0 msgr2=0x7f972c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:20.493 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.491+0000 7f974a7f0640 1 --2- 172.21.15.97:0/656444023 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f972c0a3dc0 0x7f972c0a6280 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f973c0092f0 tx=0x7f973c014b10 comp rx=0 tx=0).stop 2024-04-23T10:41:20.493 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.491+0000 7f974a7f0640 1 -- 172.21.15.97:0/656444023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97441021d0 msgr2=0x7f97441986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:20.493 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.491+0000 7f974a7f0640 1 --2- 172.21.15.97:0/656444023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97441021d0 0x7f97441986e0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f973400d770 tx=0x7f973400dc30 comp rx=0 tx=0).stop 2024-04-23T10:41:20.493 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.491+0000 7f974a7f0640 1 -- 172.21.15.97:0/656444023 shutdown_connections 2024-04-23T10:41:20.493 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.491+0000 7f974a7f0640 1 --2- 172.21.15.97:0/656444023 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f972c0a3dc0 0x7f972c0a6280 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:20.493 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.491+0000 7f974a7f0640 1 --2- 172.21.15.97:0/656444023 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f974419cfa0 0x7f974419d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:20.493 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.491+0000 7f974a7f0640 1 --2- 172.21.15.97:0/656444023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97441072a0 0x7f9744198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:20.493 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.491+0000 7f974a7f0640 1 --2- 172.21.15.97:0/656444023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97441021d0 0x7f97441986e0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:20.493 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.491+0000 7f974a7f0640 1 -- 172.21.15.97:0/656444023 >> 172.21.15.97:0/656444023 conn(0x7f97440793e0 msgr2=0x7f97440ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:41:20.493 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.491+0000 7f974a7f0640 1 -- 172.21.15.97:0/656444023 shutdown_connections 2024-04-23T10:41:20.493 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:20.491+0000 7f974a7f0640 1 -- 172.21.15.97:0/656444023 wait complete. 2024-04-23T10:41:20.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:41:20 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/3244714904' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:41:20.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:41:20 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/3244714904' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:41:20.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:41:20 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/3244714904' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:41:21.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:41:21 smithi196 ceph-mon[32919]: pgmap v475: 97 pgs: 97 active+clean; 2.9 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 49 KiB/s wr, 18 op/s 2024-04-23T10:41:21.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:41:21 smithi097 ceph-mon[29197]: pgmap v475: 97 pgs: 97 active+clean; 2.9 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 49 KiB/s wr, 18 op/s 2024-04-23T10:41:21.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:41:21 smithi183 ceph-mon[32317]: pgmap v475: 97 pgs: 97 active+clean; 2.9 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 49 KiB/s wr, 18 op/s 2024-04-23T10:41:22.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:41:22 smithi196 ceph-mon[32919]: pgmap v476: 97 pgs: 97 active+clean; 2.9 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 48 KiB/s wr, 18 op/s 2024-04-23T10:41:22.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:41:22 smithi097 ceph-mon[29197]: pgmap v476: 97 pgs: 97 active+clean; 2.9 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 48 KiB/s wr, 18 op/s 2024-04-23T10:41:22.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:41:22 smithi183 ceph-mon[32317]: pgmap v476: 97 pgs: 97 active+clean; 2.9 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 48 KiB/s wr, 18 op/s 2024-04-23T10:41:24.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:41:24 smithi196 ceph-mon[32919]: pgmap v477: 97 pgs: 97 active+clean; 3.1 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 55 KiB/s wr, 23 op/s 2024-04-23T10:41:24.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:41:24 smithi097 ceph-mon[29197]: pgmap v477: 97 pgs: 97 active+clean; 3.1 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 55 KiB/s wr, 23 op/s 2024-04-23T10:41:24.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:41:24 smithi183 ceph-mon[32317]: pgmap v477: 97 pgs: 97 active+clean; 3.1 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 55 KiB/s wr, 23 op/s 2024-04-23T10:41:26.702 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:41:26 smithi183 ceph-mon[32317]: pgmap v478: 97 pgs: 97 active+clean; 3.1 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 32 KiB/s wr, 14 op/s 2024-04-23T10:41:26.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:41:26 smithi196 ceph-mon[32919]: pgmap v478: 97 pgs: 97 active+clean; 3.1 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 32 KiB/s wr, 14 op/s 2024-04-23T10:41:26.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:41:26 smithi097 ceph-mon[29197]: pgmap v478: 97 pgs: 97 active+clean; 3.1 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 32 KiB/s wr, 14 op/s 2024-04-23T10:41:28.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:41:28 smithi196 ceph-mon[32919]: pgmap v479: 97 pgs: 97 active+clean; 3.1 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 34 KiB/s wr, 18 op/s 2024-04-23T10:41:28.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:41:28 smithi097 ceph-mon[29197]: pgmap v479: 97 pgs: 97 active+clean; 3.1 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 34 KiB/s wr, 18 op/s 2024-04-23T10:41:28.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:41:28 smithi183 ceph-mon[32317]: pgmap v479: 97 pgs: 97 active+clean; 3.1 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 34 KiB/s wr, 18 op/s 2024-04-23T10:41:30.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:41:30 smithi196 ceph-mon[32919]: pgmap v480: 97 pgs: 97 active+clean; 3.1 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-04-23T10:41:30.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:41:30 smithi097 ceph-mon[29197]: pgmap v480: 97 pgs: 97 active+clean; 3.1 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-04-23T10:41:30.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:41:30 smithi183 ceph-mon[32317]: pgmap v480: 97 pgs: 97 active+clean; 3.1 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-04-23T10:41:31.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:41:31 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:41:31.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:41:31 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:41:31.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:41:31 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:41:32.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:41:32 smithi196 ceph-mon[32919]: pgmap v481: 97 pgs: 97 active+clean; 3.1 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-04-23T10:41:32.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:41:32 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:41:32.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:41:32 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:41:32.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:41:32 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:41:32.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:41:32 smithi097 ceph-mon[29197]: pgmap v481: 97 pgs: 97 active+clean; 3.1 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-04-23T10:41:32.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:41:32 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:41:32.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:41:32 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:41:32.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:41:32 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:41:32.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:41:32 smithi183 ceph-mon[32317]: pgmap v481: 97 pgs: 97 active+clean; 3.1 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-04-23T10:41:32.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:41:32 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:41:32.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:41:32 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:41:32.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:41:32 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:41:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.586+0000 7fb257ccf640 1 -- 172.21.15.97:0/988014963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb250074c20 msgr2=0x7fb25010e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.586+0000 7fb257ccf640 1 --2- 172.21.15.97:0/988014963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb250074c20 0x7fb25010e2d0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fb240009900 tx=0x7fb24002a480 comp rx=0 tx=0).stop 2024-04-23T10:41:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.587+0000 7fb257ccf640 1 -- 172.21.15.97:0/988014963 shutdown_connections 2024-04-23T10:41:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.587+0000 7fb257ccf640 1 --2- 172.21.15.97:0/988014963 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb25010e810 0x7fb250110c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.587+0000 7fb257ccf640 1 --2- 172.21.15.97:0/988014963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb250074c20 0x7fb25010e2d0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.587+0000 7fb257ccf640 1 --2- 172.21.15.97:0/988014963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb250074300 0x7fb2500746e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.587+0000 7fb257ccf640 1 -- 172.21.15.97:0/988014963 >> 172.21.15.97:0/988014963 conn(0x7fb2500fda20 msgr2=0x7fb2500ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:41:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.587+0000 7fb257ccf640 1 -- 172.21.15.97:0/988014963 shutdown_connections 2024-04-23T10:41:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.587+0000 7fb257ccf640 1 -- 172.21.15.97:0/988014963 wait complete. 2024-04-23T10:41:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.587+0000 7fb257ccf640 1 Processor -- start 2024-04-23T10:41:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.587+0000 7fb257ccf640 1 -- start start 2024-04-23T10:41:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.587+0000 7fb257ccf640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb250074300 0x7fb2501a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.587+0000 7fb257ccf640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb250074c20 0x7fb2501a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.587+0000 7fb257ccf640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb25010e810 0x7fb25019a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.587+0000 7fb257ccf640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb2501143d0 con 0x7fb25010e810 2024-04-23T10:41:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.587+0000 7fb257ccf640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb250114250 con 0x7fb250074300 2024-04-23T10:41:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.587+0000 7fb257ccf640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb250114550 con 0x7fb250074c20 2024-04-23T10:41:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.588+0000 7fb25626c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb25010e810 0x7fb25019a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.588+0000 7fb25626c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb25010e810 0x7fb25019a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50696/0 (socket says 172.21.15.97:50696) 2024-04-23T10:41:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.588+0000 7fb25626c640 1 -- 172.21.15.97:0/3496660969 learned_addr learned my addr 172.21.15.97:0/3496660969 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:41:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.588+0000 7fb25526a640 1 --2- 172.21.15.97:0/3496660969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb250074c20 0x7fb2501a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.588+0000 7fb255a6b640 1 --2- 172.21.15.97:0/3496660969 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb250074300 0x7fb2501a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.588+0000 7fb25626c640 1 -- 172.21.15.97:0/3496660969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb250074c20 msgr2=0x7fb2501a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.588+0000 7fb25626c640 1 --2- 172.21.15.97:0/3496660969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb250074c20 0x7fb2501a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.588+0000 7fb25626c640 1 -- 172.21.15.97:0/3496660969 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb250074300 msgr2=0x7fb2501a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.588+0000 7fb25626c640 1 --2- 172.21.15.97:0/3496660969 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb250074300 0x7fb2501a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.588+0000 7fb25626c640 1 -- 172.21.15.97:0/3496660969 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb25019afd0 con 0x7fb25010e810 2024-04-23T10:41:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.588+0000 7fb25626c640 1 --2- 172.21.15.97:0/3496660969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb25010e810 0x7fb25019a710 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fb24800d750 tx=0x7fb24800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:41:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.588+0000 7fb246ffd640 1 -- 172.21.15.97:0/3496660969 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb248017070 con 0x7fb25010e810 2024-04-23T10:41:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.588+0000 7fb246ffd640 1 -- 172.21.15.97:0/3496660969 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2480054d0 con 0x7fb25010e810 2024-04-23T10:41:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.588+0000 7fb257ccf640 1 -- 172.21.15.97:0/3496660969 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb25019b2c0 con 0x7fb25010e810 2024-04-23T10:41:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.588+0000 7fb246ffd640 1 -- 172.21.15.97:0/3496660969 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb248004c60 con 0x7fb25010e810 2024-04-23T10:41:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.588+0000 7fb257ccf640 1 -- 172.21.15.97:0/3496660969 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2501a75d0 con 0x7fb25010e810 2024-04-23T10:41:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.591+0000 7fb246ffd640 1 -- 172.21.15.97:0/3496660969 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb248009bd0 con 0x7fb25010e810 2024-04-23T10:41:33.602 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.591+0000 7fb25a9e4740 1 Processor -- start 2024-04-23T10:41:33.602 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.592+0000 7fb25a9e4740 1 -- start start 2024-04-23T10:41:33.602 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.592+0000 7fb25a9e4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cbc243c620 0x55cbc243ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:33.602 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.592+0000 7fb25a9e4740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbc243cf40 0x55cbc24472c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:33.602 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.592+0000 7fb25a9e4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbc2447800 0x55cbc2449bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:33.602 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.592+0000 7fb25a9e4740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55cbc23a1970 con 0x55cbc243cf40 2024-04-23T10:41:33.603 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.592+0000 7fb25a9e4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55cbc233e4a0 con 0x55cbc243c620 2024-04-23T10:41:33.603 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.592+0000 7fb25a9e4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55cbc239d930 con 0x55cbc2447800 2024-04-23T10:41:33.603 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.592+0000 7fb246ffd640 1 --2- 172.21.15.97:0/3496660969 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fb23c0a3e40 0x7fb23c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:33.603 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.592+0000 7fb255a6b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbc243cf40 0x55cbc24472c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:33.603 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.592+0000 7fb255a6b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbc243cf40 0x55cbc24472c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50700/0 (socket says 172.21.15.97:50700) 2024-04-23T10:41:33.603 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.592+0000 7fb255a6b640 1 -- 172.21.15.97:0/3186632563 learned_addr learned my addr 172.21.15.97:0/3186632563 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:41:33.603 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.592+0000 7fb25626c640 1 --2- 172.21.15.97:0/3496660969 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fb23c0a3e40 0x7fb23c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:33.603 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.592+0000 7fb25526a640 1 --2- 172.21.15.97:0/3186632563 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbc2447800 0x55cbc2449bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:33.603 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.592+0000 7fb246ffd640 1 -- 172.21.15.97:0/3496660969 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8484+0+0 (secure 0 0 0) 0x7fb2480d0070 con 0x7fb25010e810 2024-04-23T10:41:33.603 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.592+0000 7fb25526a640 1 --2- 172.21.15.97:0/3186632563 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cbc243c620 0x55cbc243ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.593+0000 7fb255a6b640 1 -- 172.21.15.97:0/3186632563 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbc2447800 msgr2=0x55cbc2449bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.593+0000 7fb255a6b640 1 --2- 172.21.15.97:0/3186632563 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbc2447800 0x55cbc2449bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.593+0000 7fb255a6b640 1 -- 172.21.15.97:0/3186632563 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cbc243c620 msgr2=0x55cbc243ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.593+0000 7fb255a6b640 1 --2- 172.21.15.97:0/3186632563 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cbc243c620 0x55cbc243ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.593+0000 7fb255a6b640 1 -- 172.21.15.97:0/3186632563 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbc20b4210 con 0x55cbc243cf40 2024-04-23T10:41:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.593+0000 7fb25526a640 1 --2- 172.21.15.97:0/3186632563 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cbc243c620 0x55cbc243ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:41:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.593+0000 7fb255a6b640 1 --2- 172.21.15.97:0/3186632563 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbc243cf40 0x55cbc24472c0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fb24c0114f0 tx=0x7fb24c0119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52e9092e3710ce96 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:41:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.593+0000 7fb25626c640 1 --2- 172.21.15.97:0/3496660969 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fb23c0a3e40 0x7fb23c0a6300 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb24809bd80 tx=0x7fb248009f00 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:41:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.593+0000 7fb23bfff640 1 -- 172.21.15.97:0/3186632563 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb24c004d20 con 0x55cbc243cf40 2024-04-23T10:41:33.605 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.593+0000 7fb23bfff640 1 -- 172.21.15.97:0/3186632563 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb24c004050 con 0x55cbc243cf40 2024-04-23T10:41:33.605 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.593+0000 7fb23bfff640 1 -- 172.21.15.97:0/3186632563 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb24c00f760 con 0x55cbc243cf40 2024-04-23T10:41:33.605 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.594+0000 7fb25a9e4740 1 -- 172.21.15.97:0/3186632563 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbc243cf40 msgr2=0x55cbc24472c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:33.605 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.594+0000 7fb25a9e4740 1 --2- 172.21.15.97:0/3186632563 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbc243cf40 0x55cbc24472c0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fb24c0114f0 tx=0x7fb24c0119b0 comp rx=0 tx=0).stop 2024-04-23T10:41:33.605 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.594+0000 7fb25a9e4740 1 -- 172.21.15.97:0/3186632563 shutdown_connections 2024-04-23T10:41:33.605 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.594+0000 7fb25a9e4740 1 --2- 172.21.15.97:0/3186632563 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbc2447800 0x55cbc2449bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:33.605 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.594+0000 7fb25a9e4740 1 --2- 172.21.15.97:0/3186632563 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbc243cf40 0x55cbc24472c0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:33.605 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.594+0000 7fb25a9e4740 1 --2- 172.21.15.97:0/3186632563 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cbc243c620 0x55cbc243ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:33.605 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.594+0000 7fb25a9e4740 1 -- 172.21.15.97:0/3186632563 >> 172.21.15.97:0/3186632563 conn(0x55cbc2438b30 msgr2=0x55cbc2438f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:41:33.605 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.594+0000 7fb25a9e4740 1 -- 172.21.15.97:0/3186632563 shutdown_connections 2024-04-23T10:41:33.606 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.594+0000 7fb25a9e4740 1 -- 172.21.15.97:0/3186632563 wait complete. 2024-04-23T10:41:33.606 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.594+0000 7fb25a9e4740 1 -- start start 2024-04-23T10:41:33.606 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.594+0000 7fb25a9e4740 1 Processor -- start 2024-04-23T10:41:33.606 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.595+0000 7fb25a9e4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cbc243c620 0x55cbc24470b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:33.606 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.595+0000 7fb25a9e4740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbc243cf40 0x55cbc24443a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:33.606 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.595+0000 7fb25a9e4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbc2447800 0x55cbc24448e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:33.606 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.595+0000 7fb25a9e4740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55cbc244a430 con 0x55cbc243cf40 2024-04-23T10:41:33.606 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.595+0000 7fb25a9e4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55cbc233c0b0 con 0x55cbc243c620 2024-04-23T10:41:33.606 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.595+0000 7fb25a9e4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55cbc244a5b0 con 0x55cbc2447800 2024-04-23T10:41:33.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.595+0000 7fb25a9e4740 10 client.0 _ll_register_callbacks cb 0x55cbc2329d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:41:33.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.595+0000 7fb255a6b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbc243cf40 0x55cbc24443a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:33.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.595+0000 7fb255a6b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbc243cf40 0x55cbc24443a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50706/0 (socket says 172.21.15.97:50706) 2024-04-23T10:41:33.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.595+0000 7fb255a6b640 1 -- 172.21.15.97:0/3364167048 learned_addr learned my addr 172.21.15.97:0/3364167048 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:41:33.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.595+0000 7fb25526a640 1 --2- 172.21.15.97:0/3364167048 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cbc243c620 0x55cbc24470b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:33.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.595+0000 7fb255a6b640 1 -- 172.21.15.97:0/3364167048 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbc2447800 msgr2=0x55cbc24448e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:33.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.595+0000 7fb25526a640 1 --2- 172.21.15.97:0/3364167048 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbc2447800 0x55cbc24448e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:33.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.595+0000 7fb255a6b640 1 --2- 172.21.15.97:0/3364167048 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbc2447800 0x55cbc24448e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:33.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.595+0000 7fb255a6b640 1 -- 172.21.15.97:0/3364167048 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cbc243c620 msgr2=0x55cbc24470b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:33.607 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.595+0000 7fb255a6b640 1 --2- 172.21.15.97:0/3364167048 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cbc243c620 0x55cbc24470b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:33.608 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.595+0000 7fb255a6b640 1 -- 172.21.15.97:0/3364167048 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbc2180620 con 0x55cbc243cf40 2024-04-23T10:41:33.608 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.595+0000 7fb255a6b640 1 --2- 172.21.15.97:0/3364167048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbc243cf40 0x55cbc24443a0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fb24c01b040 tx=0x7fb24c004280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2ce588dc6fc5d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:41:33.608 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.595+0000 7fb2397fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:41:33.608 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.595+0000 7fb2397fa640 1 -- 172.21.15.97:0/3364167048 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb24c028020 con 0x55cbc243cf40 2024-04-23T10:41:33.608 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.596+0000 7fb25a9e4740 1 -- 172.21.15.97:0/3364167048 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb24c004050 con 0x55cbc243cf40 2024-04-23T10:41:33.608 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.596+0000 7fb2397fa640 1 -- 172.21.15.97:0/3364167048 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8484+0+0 (secure 0 0 0) 0x7fb24c004a40 con 0x55cbc243cf40 2024-04-23T10:41:33.608 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.596+0000 7fb2397fa640 1 -- 172.21.15.97:0/3364167048 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb24c033050 con 0x55cbc243cf40 2024-04-23T10:41:33.608 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.596+0000 7fb25a9e4740 10 client.17685 fetch_fsmap learned FSMap version 26 2024-04-23T10:41:33.608 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.596+0000 7fb25a9e4740 1 -- 172.21.15.97:0/3364167048 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cbc2456210 con 0x55cbc243cf40 2024-04-23T10:41:33.608 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.596+0000 7fb2397fa640 1 -- 172.21.15.97:0/3364167048 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb24c004050 con 0x55cbc243cf40 2024-04-23T10:41:33.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.596+0000 7fb2397fa640 1 -- 172.21.15.97:0/3364167048 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb24c005040 con 0x55cbc243cf40 2024-04-23T10:41:33.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.596+0000 7fb25a9e4740 10 client.17685 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:41:33.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.596+0000 7fb25a9e4740 10 client.17685 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:41:33.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.596+0000 7fb25a9e4740 20 client.17685 populate_metadata read hostname 'smithi097' 2024-04-23T10:41:33.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.596+0000 7fb25a9e4740 1 --2- 172.21.15.97:0/3364167048 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55cbc24576d0 0x55cbc2459ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:33.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.596+0000 7fb25a9e4740 4 client.17685 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:41:33.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.596+0000 7fb25a9e4740 1 -- 172.21.15.97:0/3364167048 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cbc23af390 con 0x55cbc24576d0 2024-04-23T10:41:33.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.596+0000 7fb25526a640 1 --2- 172.21.15.97:0/3364167048 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55cbc24576d0 0x55cbc2459ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:33.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.597+0000 7fb2397fa640 10 client.17685 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:41:33.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.597+0000 7fb25526a640 1 --2- 172.21.15.97:0/3364167048 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55cbc24576d0 0x55cbc2459ac0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=275ccaf05cac250c server_cookie=4cc38316bc99bd57 in_seq=0 out_seq=0 2024-04-23T10:41:33.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.597+0000 7fb2397fa640 1 -- 172.21.15.97:0/3364167048 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cbc23af390 con 0x55cbc24576d0 2024-04-23T10:41:33.609 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.597+0000 7fb2397fa640 10 client.17685 handle_command_reply: tid=0 2024-04-23T10:41:33.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.597+0000 7fb25a9e4740 1 client.17685 shutdown 2024-04-23T10:41:33.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.598+0000 7fb25a9e4740 1 -- 172.21.15.97:0/3364167048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbc243cf40 msgr2=0x55cbc24443a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:33.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.598+0000 7fb25a9e4740 1 --2- 172.21.15.97:0/3364167048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbc243cf40 0x55cbc24443a0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fb24c01b040 tx=0x7fb24c004280 comp rx=0 tx=0).stop 2024-04-23T10:41:33.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.598+0000 7fb25a9e4740 1 -- 172.21.15.97:0/3364167048 shutdown_connections 2024-04-23T10:41:33.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.598+0000 7fb25a9e4740 1 --2- 172.21.15.97:0/3364167048 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55cbc24576d0 0x55cbc2459ac0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:33.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.598+0000 7fb25a9e4740 1 --2- 172.21.15.97:0/3364167048 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbc2447800 0x55cbc24448e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:33.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.598+0000 7fb25a9e4740 1 --2- 172.21.15.97:0/3364167048 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbc243cf40 0x55cbc24443a0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:33.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.598+0000 7fb2397fa640 0 client.17685 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:41:33.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.598+0000 7fb25a9e4740 1 --2- 172.21.15.97:0/3364167048 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cbc243c620 0x55cbc24470b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:33.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.598+0000 7fb25a9e4740 1 -- 172.21.15.97:0/3364167048 >> 172.21.15.97:0/3364167048 conn(0x55cbc2438b30 msgr2=0x55cbc243e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:41:33.610 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.598+0000 7fb25a9e4740 1 -- 172.21.15.97:0/3364167048 shutdown_connections 2024-04-23T10:41:33.611 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.598+0000 7fb25a9e4740 1 -- 172.21.15.97:0/3364167048 wait complete. 2024-04-23T10:41:33.611 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.598+0000 7fb25a9e4740 20 client.17685 trim_cache size 0 max 16384 2024-04-23T10:41:33.617 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.612+0000 7fb25a9e4740 1 Processor -- start 2024-04-23T10:41:33.618 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.613+0000 7fb25a9e4740 1 -- start start 2024-04-23T10:41:33.618 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.613+0000 7fb25a9e4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbc2459e30 0x55cbc24532f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:33.618 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.613+0000 7fb25a9e4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cbc2438b30 0x55cbc243f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:33.618 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.613+0000 7fb25a9e4740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbc243c620 0x55cbc2446be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:33.618 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.613+0000 7fb25a9e4740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb2501afbf0 con 0x55cbc243c620 2024-04-23T10:41:33.618 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.613+0000 7fb25a9e4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb2501b1730 con 0x55cbc2438b30 2024-04-23T10:41:33.618 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.613+0000 7fb25a9e4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb2501b02c0 con 0x55cbc2459e30 2024-04-23T10:41:33.618 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.613+0000 7fb25526a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbc243c620 0x55cbc2446be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:33.618 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.613+0000 7fb25526a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbc243c620 0x55cbc2446be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50722/0 (socket says 172.21.15.97:50722) 2024-04-23T10:41:33.618 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.613+0000 7fb25526a640 1 -- 172.21.15.97:0/284044780 learned_addr learned my addr 172.21.15.97:0/284044780 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:41:33.619 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.613+0000 7fb255a6b640 1 --2- 172.21.15.97:0/284044780 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cbc2438b30 0x55cbc243f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:33.619 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.613+0000 7fb25526a640 1 --2- 172.21.15.97:0/284044780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbc2459e30 0x55cbc24532f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:33.619 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.613+0000 7fb25526a640 1 -- 172.21.15.97:0/284044780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbc2459e30 msgr2=0x55cbc24532f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:33.619 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.613+0000 7fb25526a640 1 --2- 172.21.15.97:0/284044780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbc2459e30 0x55cbc24532f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:33.619 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.613+0000 7fb25526a640 1 -- 172.21.15.97:0/284044780 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cbc2438b30 msgr2=0x55cbc243f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:33.619 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.613+0000 7fb25526a640 1 --2- 172.21.15.97:0/284044780 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cbc2438b30 0x55cbc243f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:33.619 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.613+0000 7fb25526a640 1 -- 172.21.15.97:0/284044780 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbc2439700 con 0x55cbc243c620 2024-04-23T10:41:33.619 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.613+0000 7fb255a6b640 1 --2- 172.21.15.97:0/284044780 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cbc2438b30 0x55cbc243f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:41:33.619 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.613+0000 7fb25526a640 1 --2- 172.21.15.97:0/284044780 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbc243c620 0x55cbc2446be0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fb2400096c0 tx=0x7fb24002a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b942e4c6c87379d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:41:33.619 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.614+0000 7fb23b7fe640 1 -- 172.21.15.97:0/284044780 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb24004f070 con 0x55cbc243c620 2024-04-23T10:41:33.619 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.614+0000 7fb23b7fe640 1 -- 172.21.15.97:0/284044780 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2400043f0 con 0x55cbc243c620 2024-04-23T10:41:33.619 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.614+0000 7fb23b7fe640 1 -- 172.21.15.97:0/284044780 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb240035040 con 0x55cbc243c620 2024-04-23T10:41:33.620 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.614+0000 7fb25a9e4740 1 -- 172.21.15.97:0/284044780 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbc243c620 msgr2=0x55cbc2446be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:33.620 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.614+0000 7fb25a9e4740 1 --2- 172.21.15.97:0/284044780 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbc243c620 0x55cbc2446be0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fb2400096c0 tx=0x7fb24002a480 comp rx=0 tx=0).stop 2024-04-23T10:41:33.620 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.614+0000 7fb25a9e4740 1 -- 172.21.15.97:0/284044780 shutdown_connections 2024-04-23T10:41:33.620 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.614+0000 7fb25a9e4740 1 --2- 172.21.15.97:0/284044780 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbc243c620 0x55cbc2446be0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:33.620 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.614+0000 7fb25a9e4740 1 --2- 172.21.15.97:0/284044780 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cbc2438b30 0x55cbc243f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:33.620 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.614+0000 7fb25a9e4740 1 --2- 172.21.15.97:0/284044780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbc2459e30 0x55cbc24532f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:33.620 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.614+0000 7fb25a9e4740 1 -- 172.21.15.97:0/284044780 >> 172.21.15.97:0/284044780 conn(0x55cbc2444a40 msgr2=0x55cbc243f140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:41:33.620 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.614+0000 7fb25a9e4740 1 -- 172.21.15.97:0/284044780 shutdown_connections 2024-04-23T10:41:33.620 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.614+0000 7fb25a9e4740 1 -- 172.21.15.97:0/284044780 wait complete. 2024-04-23T10:41:33.620 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.614+0000 7fb25a9e4740 1 -- start start 2024-04-23T10:41:33.620 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.615+0000 7fb25a9e4740 1 Processor -- start 2024-04-23T10:41:33.620 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.615+0000 7fb25a9e4740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbc2438b30 0x55cbc244fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:33.621 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.615+0000 7fb25a9e4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cbc243c620 0x55cbc244aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:33.621 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.615+0000 7fb25a9e4740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbc2459e30 0x55cbc244b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:33.621 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.615+0000 7fb25a9e4740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb2501b1060 con 0x55cbc2438b30 2024-04-23T10:41:33.621 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.615+0000 7fb25a9e4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb2501b0990 con 0x55cbc243c620 2024-04-23T10:41:33.621 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.615+0000 7fb25a9e4740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb2501adbf0 con 0x55cbc2459e30 2024-04-23T10:41:33.621 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.615+0000 7fb25a9e4740 10 client.0 _ll_register_callbacks cb 0x7fb24c0047f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:41:33.621 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.615+0000 7fb25526a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbc2438b30 0x55cbc244fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:33.621 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.615+0000 7fb25526a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbc2438b30 0x55cbc244fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50728/0 (socket says 172.21.15.97:50728) 2024-04-23T10:41:33.621 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.615+0000 7fb25526a640 1 -- 172.21.15.97:0/4291760434 learned_addr learned my addr 172.21.15.97:0/4291760434 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:41:33.621 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.615+0000 7fb25526a640 1 -- 172.21.15.97:0/4291760434 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbc2459e30 msgr2=0x55cbc244b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:33.621 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.615+0000 7fb25526a640 1 --2- 172.21.15.97:0/4291760434 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbc2459e30 0x55cbc244b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:33.621 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.615+0000 7fb25526a640 1 -- 172.21.15.97:0/4291760434 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cbc243c620 msgr2=0x55cbc244aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:33.622 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.615+0000 7fb255a6b640 1 --2- 172.21.15.97:0/4291760434 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cbc243c620 0x55cbc244aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:33.622 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.615+0000 7fb25526a640 1 --2- 172.21.15.97:0/4291760434 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cbc243c620 0x55cbc244aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:33.622 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.615+0000 7fb25526a640 1 -- 172.21.15.97:0/4291760434 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbc244bad0 con 0x55cbc2438b30 2024-04-23T10:41:33.622 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.616+0000 7fb25526a640 1 --2- 172.21.15.97:0/4291760434 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbc2438b30 0x55cbc244fab0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fb24004d040 tx=0x7fb240047c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=548deb1910551d8e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:41:33.622 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.616+0000 7fb2397fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:41:33.622 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.616+0000 7fb2397fa640 1 -- 172.21.15.97:0/4291760434 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb240035070 con 0x55cbc2438b30 2024-04-23T10:41:33.622 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.616+0000 7fb2397fa640 1 -- 172.21.15.97:0/4291760434 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8484+0+0 (secure 0 0 0) 0x7fb2400318b0 con 0x55cbc2438b30 2024-04-23T10:41:33.622 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.616+0000 7fb25a9e4740 1 -- 172.21.15.97:0/4291760434 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2400043f0 con 0x55cbc2438b30 2024-04-23T10:41:33.622 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.616+0000 7fb2397fa640 1 -- 172.21.15.97:0/4291760434 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb24004f070 con 0x55cbc2438b30 2024-04-23T10:41:33.622 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.616+0000 7fb25a9e4740 10 client.17691 fetch_fsmap learned FSMap version 26 2024-04-23T10:41:33.622 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.616+0000 7fb25a9e4740 1 -- 172.21.15.97:0/4291760434 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cbc24bd9f0 con 0x55cbc2438b30 2024-04-23T10:41:33.623 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.616+0000 7fb2397fa640 1 -- 172.21.15.97:0/4291760434 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2400043f0 con 0x55cbc2438b30 2024-04-23T10:41:33.623 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.616+0000 7fb2397fa640 1 -- 172.21.15.97:0/4291760434 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb24005a020 con 0x55cbc2438b30 2024-04-23T10:41:33.623 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.616+0000 7fb25a9e4740 10 client.17691 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:41:33.623 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.616+0000 7fb25a9e4740 10 client.17691 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:41:33.623 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.616+0000 7fb25a9e4740 20 client.17691 populate_metadata read hostname 'smithi097' 2024-04-23T10:41:33.623 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.616+0000 7fb25a9e4740 1 --2- 172.21.15.97:0/4291760434 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55cbc243cf40 0x55cbc24c0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:33.623 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.616+0000 7fb25a9e4740 4 client.17691 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:41:33.623 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.616+0000 7fb25a9e4740 1 -- 172.21.15.97:0/4291760434 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cbc23a9200 con 0x55cbc243cf40 2024-04-23T10:41:33.623 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.617+0000 7fb255a6b640 1 --2- 172.21.15.97:0/4291760434 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55cbc243cf40 0x55cbc24c0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:33.623 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.617+0000 7fb2397fa640 10 client.17691 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:41:33.623 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.617+0000 7fb255a6b640 1 --2- 172.21.15.97:0/4291760434 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55cbc243cf40 0x55cbc24c0020 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f808e65b40773982 server_cookie=809544778d31407b in_seq=0 out_seq=0 2024-04-23T10:41:33.629 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.630+0000 7fb2397fa640 1 -- 172.21.15.97:0/4291760434 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cbc23a9200 con 0x55cbc243cf40 2024-04-23T10:41:33.629 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.630+0000 7fb2397fa640 10 client.17691 handle_command_reply: tid=0 2024-04-23T10:41:33.630 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.630+0000 7fb25a9e4740 1 client.17691 shutdown 2024-04-23T10:41:33.630 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.630+0000 7fb25a9e4740 1 -- 172.21.15.97:0/4291760434 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbc2438b30 msgr2=0x55cbc244fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:33.630 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.630+0000 7fb25a9e4740 1 --2- 172.21.15.97:0/4291760434 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbc2438b30 0x55cbc244fab0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fb24004d040 tx=0x7fb240047c60 comp rx=0 tx=0).stop 2024-04-23T10:41:33.630 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.630+0000 7fb25a9e4740 1 -- 172.21.15.97:0/4291760434 shutdown_connections 2024-04-23T10:41:33.630 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.630+0000 7fb25a9e4740 1 --2- 172.21.15.97:0/4291760434 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55cbc243cf40 0x55cbc24c0020 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:33.630 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.630+0000 7fb25a9e4740 1 --2- 172.21.15.97:0/4291760434 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbc2459e30 0x55cbc244b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:33.630 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.630+0000 7fb25a9e4740 1 --2- 172.21.15.97:0/4291760434 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cbc243c620 0x55cbc244aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:33.630 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.630+0000 7fb25a9e4740 1 --2- 172.21.15.97:0/4291760434 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cbc2438b30 0x55cbc244fab0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:33.630 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.630+0000 7fb2397fa640 0 client.17691 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:41:33.630 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.630+0000 7fb25a9e4740 1 -- 172.21.15.97:0/4291760434 >> 172.21.15.97:0/4291760434 conn(0x55cbc2444a40 msgr2=0x55cbc243f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:41:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.630+0000 7fb25a9e4740 1 -- 172.21.15.97:0/4291760434 shutdown_connections 2024-04-23T10:41:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.630+0000 7fb25a9e4740 1 -- 172.21.15.97:0/4291760434 wait complete. 2024-04-23T10:41:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.630+0000 7fb25a9e4740 20 client.17691 trim_cache size 0 max 16384 2024-04-23T10:41:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:41:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:41:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:41:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:41:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.631+0000 7fb257ccf640 1 -- 172.21.15.97:0/3496660969 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fb23c0a3e40 msgr2=0x7fb23c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.631+0000 7fb257ccf640 1 --2- 172.21.15.97:0/3496660969 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fb23c0a3e40 0x7fb23c0a6300 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb24809bd80 tx=0x7fb248009f00 comp rx=0 tx=0).stop 2024-04-23T10:41:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.631+0000 7fb257ccf640 1 -- 172.21.15.97:0/3496660969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb25010e810 msgr2=0x7fb25019a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.631+0000 7fb257ccf640 1 --2- 172.21.15.97:0/3496660969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb25010e810 0x7fb25019a710 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fb24800d750 tx=0x7fb24800dc10 comp rx=0 tx=0).stop 2024-04-23T10:41:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.631+0000 7fb257ccf640 1 -- 172.21.15.97:0/3496660969 shutdown_connections 2024-04-23T10:41:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.631+0000 7fb257ccf640 1 --2- 172.21.15.97:0/3496660969 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fb23c0a3e40 0x7fb23c0a6300 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.631+0000 7fb257ccf640 1 --2- 172.21.15.97:0/3496660969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb25010e810 0x7fb25019a710 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.631+0000 7fb257ccf640 1 --2- 172.21.15.97:0/3496660969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb250074c20 0x7fb2501a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.631+0000 7fb257ccf640 1 --2- 172.21.15.97:0/3496660969 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb250074300 0x7fb2501a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.631+0000 7fb257ccf640 1 -- 172.21.15.97:0/3496660969 >> 172.21.15.97:0/3496660969 conn(0x7fb2500fda20 msgr2=0x7fb2500fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:41:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.631+0000 7fb257ccf640 1 -- 172.21.15.97:0/3496660969 shutdown_connections 2024-04-23T10:41:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:33.631+0000 7fb257ccf640 1 -- 172.21.15.97:0/3496660969 wait complete. 2024-04-23T10:41:34.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:41:34 smithi196 ceph-mon[32919]: pgmap v482: 97 pgs: 97 active+clean; 3.1 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-04-23T10:41:34.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:41:34 smithi097 ceph-mon[29197]: pgmap v482: 97 pgs: 97 active+clean; 3.1 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-04-23T10:41:34.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:41:34 smithi183 ceph-mon[32317]: pgmap v482: 97 pgs: 97 active+clean; 3.1 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-04-23T10:41:36.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:41:36 smithi196 ceph-mon[32919]: pgmap v483: 97 pgs: 97 active+clean; 3.1 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-04-23T10:41:36.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:41:36 smithi097 ceph-mon[29197]: pgmap v483: 97 pgs: 97 active+clean; 3.1 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-04-23T10:41:36.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:41:36 smithi183 ceph-mon[32317]: pgmap v483: 97 pgs: 97 active+clean; 3.1 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-04-23T10:41:38.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:41:38 smithi196 ceph-mon[32919]: pgmap v484: 97 pgs: 97 active+clean; 3.1 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.6 KiB/s wr, 14 op/s 2024-04-23T10:41:38.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:41:38 smithi097 ceph-mon[29197]: pgmap v484: 97 pgs: 97 active+clean; 3.1 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.6 KiB/s wr, 14 op/s 2024-04-23T10:41:38.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:41:38 smithi183 ceph-mon[32317]: pgmap v484: 97 pgs: 97 active+clean; 3.1 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.6 KiB/s wr, 14 op/s 2024-04-23T10:41:40.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:41:40 smithi196 ceph-mon[32919]: pgmap v485: 97 pgs: 97 active+clean; 3.1 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.7 KiB/s wr, 10 op/s 2024-04-23T10:41:40.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:41:40 smithi097 ceph-mon[29197]: pgmap v485: 97 pgs: 97 active+clean; 3.1 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.7 KiB/s wr, 10 op/s 2024-04-23T10:41:40.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:41:40 smithi183 ceph-mon[32317]: pgmap v485: 97 pgs: 97 active+clean; 3.1 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.7 KiB/s wr, 10 op/s 2024-04-23T10:41:42.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:41:42 smithi196 ceph-mon[32919]: pgmap v486: 97 pgs: 97 active+clean; 3.1 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.7 KiB/s wr, 10 op/s 2024-04-23T10:41:42.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:41:42 smithi097 ceph-mon[29197]: pgmap v486: 97 pgs: 97 active+clean; 3.1 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.7 KiB/s wr, 10 op/s 2024-04-23T10:41:42.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:41:42 smithi183 ceph-mon[32317]: pgmap v486: 97 pgs: 97 active+clean; 3.1 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.7 KiB/s wr, 10 op/s 2024-04-23T10:41:44.617 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T10:41:44.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:44.729+0000 7f7d82b43640 1 -- 172.21.15.97:0/3466232885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d7c1031b0 msgr2=0x7f7d7c103590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:44.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:44.729+0000 7f7d82b43640 1 --2- 172.21.15.97:0/3466232885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d7c1031b0 0x7f7d7c103590 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c009900 tx=0x7f7d6c02a480 comp rx=0 tx=0).stop 2024-04-23T10:41:44.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:44.729+0000 7f7d82b43640 1 -- 172.21.15.97:0/3466232885 shutdown_connections 2024-04-23T10:41:44.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:44.729+0000 7f7d82b43640 1 --2- 172.21.15.97:0/3466232885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d7c10c580 0x7f7d7c10e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:44.731 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:44.729+0000 7f7d82b43640 1 --2- 172.21.15.97:0/3466232885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d7c103ad0 0x7f7d7c10bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:44.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:44.729+0000 7f7d82b43640 1 --2- 172.21.15.97:0/3466232885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d7c1031b0 0x7f7d7c103590 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:44.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:44.729+0000 7f7d82b43640 1 -- 172.21.15.97:0/3466232885 >> 172.21.15.97:0/3466232885 conn(0x7f7d7c0771d0 msgr2=0x7f7d7c0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:41:44.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:44.729+0000 7f7d82b43640 1 -- 172.21.15.97:0/3466232885 shutdown_connections 2024-04-23T10:41:44.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:44.729+0000 7f7d82b43640 1 -- 172.21.15.97:0/3466232885 wait complete. 2024-04-23T10:41:44.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:44.730+0000 7f7d82b43640 1 Processor -- start 2024-04-23T10:41:44.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:44.730+0000 7f7d82b43640 1 -- start start 2024-04-23T10:41:44.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:44.730+0000 7f7d82b43640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d7c1031b0 0x7f7d7c19e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:44.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:44.730+0000 7f7d82b43640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d7c103ad0 0x7f7d7c19e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:44.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:44.730+0000 7f7d82b43640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d7c10c580 0x7f7d7c1985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:44.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:44.730+0000 7f7d82b43640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7c1121d0 con 0x7f7d7c10c580 2024-04-23T10:41:44.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:44.730+0000 7f7d82b43640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7c112050 con 0x7f7d7c1031b0 2024-04-23T10:41:44.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:44.730+0000 7f7d82b43640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7c112350 con 0x7f7d7c103ad0 2024-04-23T10:41:44.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:44.730+0000 7f7d810e0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d7c10c580 0x7f7d7c1985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:44.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:44.730+0000 7f7d810e0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d7c10c580 0x7f7d7c1985e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37904/0 (socket says 172.21.15.97:37904) 2024-04-23T10:41:44.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:44.730+0000 7f7d810e0640 1 -- 172.21.15.97:0/2780927010 learned_addr learned my addr 172.21.15.97:0/2780927010 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:41:44.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:44.730+0000 7f7d808df640 1 --2- 172.21.15.97:0/2780927010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d7c1031b0 0x7f7d7c19e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:44.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:44.730+0000 7f7d7bfff640 1 --2- 172.21.15.97:0/2780927010 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d7c103ad0 0x7f7d7c19e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:44.733 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:44.730+0000 7f7d810e0640 1 -- 172.21.15.97:0/2780927010 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d7c103ad0 msgr2=0x7f7d7c19e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:44.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:44.730+0000 7f7d810e0640 1 --2- 172.21.15.97:0/2780927010 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d7c103ad0 0x7f7d7c19e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:44.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:44.730+0000 7f7d810e0640 1 -- 172.21.15.97:0/2780927010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d7c1031b0 msgr2=0x7f7d7c19e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:44.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:44.730+0000 7f7d810e0640 1 --2- 172.21.15.97:0/2780927010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d7c1031b0 0x7f7d7c19e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:44.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:44.730+0000 7f7d810e0640 1 -- 172.21.15.97:0/2780927010 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d7c198d60 con 0x7f7d7c10c580 2024-04-23T10:41:44.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:44.731+0000 7f7d810e0640 1 --2- 172.21.15.97:0/2780927010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d7c10c580 0x7f7d7c1985e0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f7d7400e9d0 tx=0x7f7d7400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:41:44.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:44.731+0000 7f7d79ffb640 1 -- 172.21.15.97:0/2780927010 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7400cd80 con 0x7f7d7c10c580 2024-04-23T10:41:44.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:44.731+0000 7f7d79ffb640 1 -- 172.21.15.97:0/2780927010 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d74005020 con 0x7f7d7c10c580 2024-04-23T10:41:44.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:44.731+0000 7f7d82b43640 1 -- 172.21.15.97:0/2780927010 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d7c199050 con 0x7f7d7c10c580 2024-04-23T10:41:44.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:44.731+0000 7f7d79ffb640 1 -- 172.21.15.97:0/2780927010 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d740116d0 con 0x7f7d7c10c580 2024-04-23T10:41:44.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:44.731+0000 7f7d82b43640 1 -- 172.21.15.97:0/2780927010 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d7c1a52f0 con 0x7f7d7c10c580 2024-04-23T10:41:44.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:44.732+0000 7f7d82b43640 1 -- 172.21.15.97:0/2780927010 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d7c06add0 con 0x7f7d7c10c580 2024-04-23T10:41:44.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:44.733+0000 7f7d79ffb640 1 -- 172.21.15.97:0/2780927010 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7d740051d0 con 0x7f7d7c10c580 2024-04-23T10:41:44.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:44.734+0000 7f7d79ffb640 1 --2- 172.21.15.97:0/2780927010 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f7d680a3d20 0x7f7d680a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:44.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:44.734+0000 7f7d79ffb640 1 -- 172.21.15.97:0/2780927010 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8484+0+0 (secure 0 0 0) 0x7f7d740d08c0 con 0x7f7d7c10c580 2024-04-23T10:41:44.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:44.734+0000 7f7d808df640 1 --2- 172.21.15.97:0/2780927010 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f7d680a3d20 0x7f7d680a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:44.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:44.734+0000 7f7d808df640 1 --2- 172.21.15.97:0/2780927010 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f7d680a3d20 0x7f7d680a61e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c031440 tx=0x7f7d6c005b50 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:41:44.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:44.738+0000 7f7d79ffb640 1 -- 172.21.15.97:0/2780927010 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7d74016030 con 0x7f7d7c10c580 2024-04-23T10:41:44.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:41:44 smithi196 ceph-mon[32919]: pgmap v487: 97 pgs: 97 active+clean; 3.1 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 5.7 KiB/s wr, 14 op/s 2024-04-23T10:41:44.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:41:44 smithi097 ceph-mon[29197]: pgmap v487: 97 pgs: 97 active+clean; 3.1 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 5.7 KiB/s wr, 14 op/s 2024-04-23T10:41:44.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:41:44 smithi183 ceph-mon[32317]: pgmap v487: 97 pgs: 97 active+clean; 3.1 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 5.7 KiB/s wr, 14 op/s 2024-04-23T10:41:44.953 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:44.954+0000 7f7d82b43640 1 -- 172.21.15.97:0/2780927010 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7d7c0691c0 con 0x7f7d7c10c580 2024-04-23T10:41:44.954 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:44.955+0000 7f7d79ffb640 1 -- 172.21.15.97:0/2780927010 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11931 (secure 0 0 0) 0x7f7d7c0691c0 con 0x7f7d7c10c580 2024-04-23T10:41:44.954 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:41:44.954 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3946878762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3946878762},{"type":"v1","addr":"172.21.15.97:6835","nonce":3946878762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/209800861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":209800861},{"type":"v1","addr":"172.21.15.97:6837","nonce":209800861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T10:24:06.910134+0000","modified":"2024-04-23T10:26:09.820985+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6839/418013575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":418013575},{"type":"v1","addr":"172.21.15.97:6839","nonce":418013575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6841/747242680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":747242680},{"type":"v1","addr":"172.21.15.97:6841","nonce":747242680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6833/4292847121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4292847121},{"type":"v1","addr":"172.21.15.183:6833","nonce":4292847121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6833/3885816565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3885816565},{"type":"v1","addr":"172.21.15.196:6833","nonce":3885816565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1897093105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1897093105},{"type":"v1","addr":"172.21.15.183:6835","nonce":1897093105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1171477563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1171477563},{"type":"v1","addr":"172.21.15.196:6835","nonce":1171477563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1898341911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1898341911},{"type":"v1","addr":"172.21.15.183:6837","nonce":1898341911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.196:6837/3094946502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3094946502},{"type":"v1","addr":"172.21.15.196:6837","nonce":3094946502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.196:6839/83843556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":83843556},{"type":"v1","addr":"172.21.15.196:6839","nonce":83843556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6839/1763841130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1763841130},{"type":"v1","addr":"172.21.15.183:6839","nonce":1763841130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24487,24403,24398,24469,24488]},"id":1}]} 2024-04-23T10:41:44.955 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 26 2024-04-23T10:41:44.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:44.957+0000 7f7d82b43640 1 -- 172.21.15.97:0/2780927010 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f7d680a3d20 msgr2=0x7f7d680a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:44.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:44.957+0000 7f7d82b43640 1 --2- 172.21.15.97:0/2780927010 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f7d680a3d20 0x7f7d680a61e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c031440 tx=0x7f7d6c005b50 comp rx=0 tx=0).stop 2024-04-23T10:41:44.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:44.957+0000 7f7d82b43640 1 -- 172.21.15.97:0/2780927010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d7c10c580 msgr2=0x7f7d7c1985e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:44.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:44.957+0000 7f7d82b43640 1 --2- 172.21.15.97:0/2780927010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d7c10c580 0x7f7d7c1985e0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f7d7400e9d0 tx=0x7f7d7400ee90 comp rx=0 tx=0).stop 2024-04-23T10:41:44.956 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:44.957+0000 7f7d82b43640 1 -- 172.21.15.97:0/2780927010 shutdown_connections 2024-04-23T10:41:44.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:44.957+0000 7f7d82b43640 1 --2- 172.21.15.97:0/2780927010 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f7d680a3d20 0x7f7d680a61e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:44.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:44.957+0000 7f7d82b43640 1 --2- 172.21.15.97:0/2780927010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d7c10c580 0x7f7d7c1985e0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:44.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:44.957+0000 7f7d82b43640 1 --2- 172.21.15.97:0/2780927010 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d7c103ad0 0x7f7d7c19e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:44.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:44.957+0000 7f7d82b43640 1 --2- 172.21.15.97:0/2780927010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d7c1031b0 0x7f7d7c19e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:44.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:44.957+0000 7f7d82b43640 1 -- 172.21.15.97:0/2780927010 >> 172.21.15.97:0/2780927010 conn(0x7f7d7c0771d0 msgr2=0x7f7d7c0fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:41:44.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:44.957+0000 7f7d82b43640 1 -- 172.21.15.97:0/2780927010 shutdown_connections 2024-04-23T10:41:44.957 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:44.957+0000 7f7d82b43640 1 -- 172.21.15.97:0/2780927010 wait complete. 2024-04-23T10:41:44.967 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --set-id 8d2d887a --release --await' 2024-04-23T10:41:44.967 DEBUG:teuthology.orchestra.run.smithi097:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24488 quiesce db --set-id 8d2d887a --release --await 2024-04-23T10:41:45.074 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.073+0000 7fa87dafd640 1 -- 172.21.15.97:0/1984285013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa878101a40 msgr2=0x7fa878105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:45.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.073+0000 7fa87dafd640 1 --2- 172.21.15.97:0/1984285013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa878101a40 0x7fa878105a80 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fa86c009900 tx=0x7fa86c02a410 comp rx=0 tx=0).stop 2024-04-23T10:41:45.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.073+0000 7fa87dafd640 1 -- 172.21.15.97:0/1984285013 shutdown_connections 2024-04-23T10:41:45.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.073+0000 7fa87dafd640 1 --2- 172.21.15.97:0/1984285013 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa878106160 0x7fa87810dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.073+0000 7fa87dafd640 1 --2- 172.21.15.97:0/1984285013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa878101a40 0x7fa878105a80 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.073+0000 7fa87dafd640 1 --2- 172.21.15.97:0/1984285013 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa878101090 0x7fa878101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.073+0000 7fa87dafd640 1 -- 172.21.15.97:0/1984285013 >> 172.21.15.97:0/1984285013 conn(0x7fa878077f50 msgr2=0x7fa878078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:41:45.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.074+0000 7fa87dafd640 1 -- 172.21.15.97:0/1984285013 shutdown_connections 2024-04-23T10:41:45.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.074+0000 7fa87dafd640 1 -- 172.21.15.97:0/1984285013 wait complete. 2024-04-23T10:41:45.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.074+0000 7fa87dafd640 1 Processor -- start 2024-04-23T10:41:45.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.074+0000 7fa87dafd640 1 -- start start 2024-04-23T10:41:45.075 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.074+0000 7fa87dafd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa878101090 0x7fa878196580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:45.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.074+0000 7fa87dafd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa878106160 0x7fa878196ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:45.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.074+0000 7fa87dafd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa87819ae40 0x7fa87819b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:45.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.074+0000 7fa87dafd640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa87810ffc0 con 0x7fa878106160 2024-04-23T10:41:45.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.074+0000 7fa87dafd640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa87810fe40 con 0x7fa87819ae40 2024-04-23T10:41:45.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.074+0000 7fa87dafd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa878110140 con 0x7fa878101090 2024-04-23T10:41:45.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.075+0000 7fa876ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa878106160 0x7fa878196ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:45.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.075+0000 7fa876ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa878106160 0x7fa878196ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37930/0 (socket says 172.21.15.97:37930) 2024-04-23T10:41:45.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.075+0000 7fa876ffd640 1 -- 172.21.15.97:0/897168985 learned_addr learned my addr 172.21.15.97:0/897168985 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:41:45.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.075+0000 7fa8777fe640 1 --2- 172.21.15.97:0/897168985 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa878101090 0x7fa878196580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:45.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.075+0000 7fa877fff640 1 --2- 172.21.15.97:0/897168985 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa87819ae40 0x7fa87819b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:45.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.075+0000 7fa876ffd640 1 -- 172.21.15.97:0/897168985 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa878101090 msgr2=0x7fa878196580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:45.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.075+0000 7fa876ffd640 1 --2- 172.21.15.97:0/897168985 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa878101090 0x7fa878196580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.076 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.075+0000 7fa876ffd640 1 -- 172.21.15.97:0/897168985 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa87819ae40 msgr2=0x7fa87819b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:45.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.075+0000 7fa876ffd640 1 --2- 172.21.15.97:0/897168985 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa87819ae40 0x7fa87819b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.075+0000 7fa876ffd640 1 -- 172.21.15.97:0/897168985 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa87819b930 con 0x7fa878106160 2024-04-23T10:41:45.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.075+0000 7fa877fff640 1 --2- 172.21.15.97:0/897168985 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa87819ae40 0x7fa87819b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:41:45.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.075+0000 7fa876ffd640 1 --2- 172.21.15.97:0/897168985 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa878106160 0x7fa878196ac0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fa86c032fd0 tx=0x7fa86c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:41:45.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.075+0000 7fa874ff9640 1 -- 172.21.15.97:0/897168985 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86c03f030 con 0x7fa878106160 2024-04-23T10:41:45.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.075+0000 7fa874ff9640 1 -- 172.21.15.97:0/897168985 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa86c004490 con 0x7fa878106160 2024-04-23T10:41:45.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.075+0000 7fa874ff9640 1 -- 172.21.15.97:0/897168985 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86c035070 con 0x7fa878106160 2024-04-23T10:41:45.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.075+0000 7fa87dafd640 1 -- 172.21.15.97:0/897168985 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa87819bbc0 con 0x7fa878106160 2024-04-23T10:41:45.077 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.075+0000 7fa87dafd640 1 -- 172.21.15.97:0/897168985 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8781a3380 con 0x7fa878106160 2024-04-23T10:41:45.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.078+0000 7fa874ff9640 1 -- 172.21.15.97:0/897168985 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa86c042030 con 0x7fa878106160 2024-04-23T10:41:45.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.078+0000 7fa88080e740 1 Processor -- start 2024-04-23T10:41:45.084 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.078+0000 7fa88080e740 1 -- start start 2024-04-23T10:41:45.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.078+0000 7fa88080e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56089c6ee490 0x56089c6ee870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:45.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.078+0000 7fa88080e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56089c6eedb0 0x56089c6f8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:45.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.078+0000 7fa88080e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56089c6f9480 0x56089c6fb870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:45.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.078+0000 7fa88080e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56089c6536d0 con 0x56089c6eedb0 2024-04-23T10:41:45.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.078+0000 7fa88080e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56089c5f01a0 con 0x56089c6ee490 2024-04-23T10:41:45.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.078+0000 7fa88080e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56089c64f690 con 0x56089c6f9480 2024-04-23T10:41:45.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.078+0000 7fa874ff9640 1 --2- 172.21.15.97:0/897168985 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fa8600a5f30 0x7fa8600a83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:45.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.078+0000 7fa8777fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56089c6eedb0 0x56089c6f8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:45.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.078+0000 7fa8777fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56089c6eedb0 0x56089c6f8f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37946/0 (socket says 172.21.15.97:37946) 2024-04-23T10:41:45.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.079+0000 7fa8777fe640 1 -- 172.21.15.97:0/4157560892 learned_addr learned my addr 172.21.15.97:0/4157560892 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:41:45.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.079+0000 7fa877fff640 1 --2- 172.21.15.97:0/897168985 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fa8600a5f30 0x7fa8600a83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:45.085 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.079+0000 7fa877fff640 1 --2- 172.21.15.97:0/4157560892 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56089c6ee490 0x56089c6ee870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:45.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.079+0000 7fa8777fe640 1 -- 172.21.15.97:0/4157560892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56089c6f9480 msgr2=0x56089c6fb870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:45.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.079+0000 7fa8777fe640 1 --2- 172.21.15.97:0/4157560892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56089c6f9480 0x56089c6fb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.079+0000 7fa874ff9640 1 -- 172.21.15.97:0/897168985 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8484+0+0 (secure 0 0 0) 0x7fa86c0f0cd0 con 0x7fa878106160 2024-04-23T10:41:45.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.079+0000 7fa8777fe640 1 -- 172.21.15.97:0/4157560892 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56089c6ee490 msgr2=0x56089c6ee870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:45.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.079+0000 7fa8777fe640 1 --2- 172.21.15.97:0/4157560892 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56089c6ee490 0x56089c6ee870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.079+0000 7fa8777fe640 1 -- 172.21.15.97:0/4157560892 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56089c5fa5b0 con 0x56089c6eedb0 2024-04-23T10:41:45.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.079+0000 7fa877fff640 1 --2- 172.21.15.97:0/4157560892 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56089c6ee490 0x56089c6ee870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:41:45.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.079+0000 7fa8777fe640 1 --2- 172.21.15.97:0/4157560892 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56089c6eedb0 0x56089c6f8f40 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fa86800bcd0 tx=0x7fa8680107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=561d61da6d927797 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:41:45.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.079+0000 7fa85dffb640 1 -- 172.21.15.97:0/4157560892 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa868005220 con 0x56089c6eedb0 2024-04-23T10:41:45.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.079+0000 7fa877fff640 1 --2- 172.21.15.97:0/897168985 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fa8600a5f30 0x7fa8600a83f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa8700092f0 tx=0x7fa870009280 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:41:45.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.079+0000 7fa85dffb640 1 -- 172.21.15.97:0/4157560892 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa868004c50 con 0x56089c6eedb0 2024-04-23T10:41:45.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.079+0000 7fa85dffb640 1 -- 172.21.15.97:0/4157560892 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8680116a0 con 0x56089c6eedb0 2024-04-23T10:41:45.086 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.080+0000 7fa88080e740 1 -- 172.21.15.97:0/4157560892 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56089c6eedb0 msgr2=0x56089c6f8f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:45.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.080+0000 7fa88080e740 1 --2- 172.21.15.97:0/4157560892 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56089c6eedb0 0x56089c6f8f40 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fa86800bcd0 tx=0x7fa8680107a0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.080+0000 7fa88080e740 1 -- 172.21.15.97:0/4157560892 shutdown_connections 2024-04-23T10:41:45.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.080+0000 7fa88080e740 1 --2- 172.21.15.97:0/4157560892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56089c6f9480 0x56089c6fb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.080+0000 7fa88080e740 1 --2- 172.21.15.97:0/4157560892 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56089c6eedb0 0x56089c6f8f40 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.080+0000 7fa88080e740 1 --2- 172.21.15.97:0/4157560892 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56089c6ee490 0x56089c6ee870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.080+0000 7fa88080e740 1 -- 172.21.15.97:0/4157560892 >> 172.21.15.97:0/4157560892 conn(0x56089c6ea7b0 msgr2=0x56089c6eab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:41:45.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.080+0000 7fa88080e740 1 -- 172.21.15.97:0/4157560892 shutdown_connections 2024-04-23T10:41:45.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.080+0000 7fa88080e740 1 -- 172.21.15.97:0/4157560892 wait complete. 2024-04-23T10:41:45.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.080+0000 7fa88080e740 1 -- start start 2024-04-23T10:41:45.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.080+0000 7fa88080e740 1 Processor -- start 2024-04-23T10:41:45.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.081+0000 7fa88080e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56089c6ee490 0x56089c6f8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:45.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.081+0000 7fa88080e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56089c6eedb0 0x56089c6f6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:45.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.081+0000 7fa88080e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56089c6f9480 0x56089c6f6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:45.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.081+0000 7fa88080e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56089c6fc310 con 0x56089c6f9480 2024-04-23T10:41:45.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.081+0000 7fa88080e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56089c5edb20 con 0x56089c6ee490 2024-04-23T10:41:45.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.081+0000 7fa88080e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x56089c6fc490 con 0x56089c6eedb0 2024-04-23T10:41:45.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.081+0000 7fa88080e740 10 client.0 _ll_register_callbacks cb 0x56089c5ed530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:41:45.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.081+0000 7fa877fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56089c6f9480 0x56089c6f6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:45.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.081+0000 7fa877fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56089c6f9480 0x56089c6f6900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37956/0 (socket says 172.21.15.97:37956) 2024-04-23T10:41:45.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.081+0000 7fa877fff640 1 -- 172.21.15.97:0/159651187 learned_addr learned my addr 172.21.15.97:0/159651187 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:41:45.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.081+0000 7fa876ffd640 1 --2- 172.21.15.97:0/159651187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56089c6eedb0 0x56089c6f6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:45.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.081+0000 7fa8777fe640 1 --2- 172.21.15.97:0/159651187 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56089c6ee490 0x56089c6f8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:45.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.081+0000 7fa877fff640 1 -- 172.21.15.97:0/159651187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56089c6eedb0 msgr2=0x56089c6f6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:45.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.081+0000 7fa877fff640 1 --2- 172.21.15.97:0/159651187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56089c6eedb0 0x56089c6f6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.088 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.081+0000 7fa877fff640 1 -- 172.21.15.97:0/159651187 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56089c6ee490 msgr2=0x56089c6f8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:45.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.081+0000 7fa877fff640 1 --2- 172.21.15.97:0/159651187 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56089c6ee490 0x56089c6f8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.081+0000 7fa877fff640 1 -- 172.21.15.97:0/159651187 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56089c365920 con 0x56089c6f9480 2024-04-23T10:41:45.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.081+0000 7fa8777fe640 1 --2- 172.21.15.97:0/159651187 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56089c6ee490 0x56089c6f8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:41:45.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.081+0000 7fa877fff640 1 --2- 172.21.15.97:0/159651187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56089c6f9480 0x56089c6f6900 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fa870013a20 tx=0x7fa870004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=338390f63736c2e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:41:45.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.081+0000 7fa856ffd640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:41:45.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.081+0000 7fa856ffd640 1 -- 172.21.15.97:0/159651187 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87000fd50 con 0x56089c6f9480 2024-04-23T10:41:45.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.081+0000 7fa856ffd640 1 -- 172.21.15.97:0/159651187 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8484+0+0 (secure 0 0 0) 0x7fa87001c420 con 0x56089c6f9480 2024-04-23T10:41:45.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.081+0000 7fa88080e740 1 -- 172.21.15.97:0/159651187 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa868004c50 con 0x56089c6f9480 2024-04-23T10:41:45.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.082+0000 7fa856ffd640 1 -- 172.21.15.97:0/159651187 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa87001c620 con 0x56089c6f9480 2024-04-23T10:41:45.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.082+0000 7fa88080e740 10 client.17709 fetch_fsmap learned FSMap version 26 2024-04-23T10:41:45.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.082+0000 7fa88080e740 1 -- 172.21.15.97:0/159651187 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56089c431d30 con 0x56089c6f9480 2024-04-23T10:41:45.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.082+0000 7fa856ffd640 1 -- 172.21.15.97:0/159651187 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa868004c50 con 0x56089c6f9480 2024-04-23T10:41:45.089 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.082+0000 7fa856ffd640 1 -- 172.21.15.97:0/159651187 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa87002d020 con 0x56089c6f9480 2024-04-23T10:41:45.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.082+0000 7fa88080e740 10 client.17709 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:41:45.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.082+0000 7fa88080e740 10 client.17709 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:41:45.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.082+0000 7fa88080e740 20 client.17709 populate_metadata read hostname 'smithi097' 2024-04-23T10:41:45.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.082+0000 7fa88080e740 1 --2- 172.21.15.97:0/159651187 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x56089c7095e0 0x56089c70b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:45.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.082+0000 7fa88080e740 4 client.17709 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:41:45.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.082+0000 7fa88080e740 1 -- 172.21.15.97:0/159651187 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56089c611500 con 0x56089c7095e0 2024-04-23T10:41:45.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.082+0000 7fa8777fe640 1 --2- 172.21.15.97:0/159651187 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x56089c7095e0 0x56089c70b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:45.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.083+0000 7fa856ffd640 10 client.17709 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:41:45.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.083+0000 7fa8777fe640 1 --2- 172.21.15.97:0/159651187 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x56089c7095e0 0x56089c70b9f0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a259359494372e25 server_cookie=bd9f26a4c0ca6a64 in_seq=0 out_seq=0 2024-04-23T10:41:45.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.083+0000 7fa856ffd640 1 -- 172.21.15.97:0/159651187 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56089c611500 con 0x56089c7095e0 2024-04-23T10:41:45.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.083+0000 7fa856ffd640 10 client.17709 handle_command_reply: tid=0 2024-04-23T10:41:45.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.083+0000 7fa88080e740 1 client.17709 shutdown 2024-04-23T10:41:45.090 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.083+0000 7fa88080e740 1 -- 172.21.15.97:0/159651187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56089c6f9480 msgr2=0x56089c6f6900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:45.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.083+0000 7fa88080e740 1 --2- 172.21.15.97:0/159651187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56089c6f9480 0x56089c6f6900 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fa870013a20 tx=0x7fa870004a40 comp rx=0 tx=0).stop 2024-04-23T10:41:45.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.084+0000 7fa88080e740 1 -- 172.21.15.97:0/159651187 shutdown_connections 2024-04-23T10:41:45.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.084+0000 7fa88080e740 1 --2- 172.21.15.97:0/159651187 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x56089c7095e0 0x56089c70b9f0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.084+0000 7fa88080e740 1 --2- 172.21.15.97:0/159651187 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56089c6f9480 0x56089c6f6900 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.084+0000 7fa88080e740 1 --2- 172.21.15.97:0/159651187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56089c6eedb0 0x56089c6f6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.084+0000 7fa856ffd640 0 client.17709 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:41:45.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.084+0000 7fa88080e740 1 --2- 172.21.15.97:0/159651187 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56089c6ee490 0x56089c6f8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.084+0000 7fa88080e740 1 -- 172.21.15.97:0/159651187 >> 172.21.15.97:0/159651187 conn(0x56089c6ea7b0 msgr2=0x56089c6e99d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:41:45.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.084+0000 7fa88080e740 1 -- 172.21.15.97:0/159651187 shutdown_connections 2024-04-23T10:41:45.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.084+0000 7fa88080e740 1 -- 172.21.15.97:0/159651187 wait complete. 2024-04-23T10:41:45.091 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.084+0000 7fa88080e740 20 client.17709 trim_cache size 0 max 16384 2024-04-23T10:41:45.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.100+0000 7fa88080e740 1 Processor -- start 2024-04-23T10:41:45.104 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.100+0000 7fa88080e740 1 -- start start 2024-04-23T10:41:45.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.101+0000 7fa88080e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56089c6ee490 0x56089c705200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:45.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.101+0000 7fa88080e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56089c6eedb0 0x56089c6f8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:45.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.101+0000 7fa88080e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56089c6f9480 0x56089c7112c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:45.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.101+0000 7fa88080e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa8781ab980 con 0x56089c6f9480 2024-04-23T10:41:45.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.101+0000 7fa88080e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa8781ad4c0 con 0x56089c6eedb0 2024-04-23T10:41:45.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.101+0000 7fa88080e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa8781ac050 con 0x56089c6ee490 2024-04-23T10:41:45.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.101+0000 7fa877fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56089c6f9480 0x56089c7112c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:45.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.101+0000 7fa877fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56089c6f9480 0x56089c7112c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37964/0 (socket says 172.21.15.97:37964) 2024-04-23T10:41:45.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.101+0000 7fa877fff640 1 -- 172.21.15.97:0/125246765 learned_addr learned my addr 172.21.15.97:0/125246765 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:41:45.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.101+0000 7fa8777fe640 1 --2- 172.21.15.97:0/125246765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56089c6ee490 0x56089c705200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:45.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.101+0000 7fa876ffd640 1 --2- 172.21.15.97:0/125246765 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56089c6eedb0 0x56089c6f8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:45.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.101+0000 7fa877fff640 1 -- 172.21.15.97:0/125246765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56089c6ee490 msgr2=0x56089c705200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:45.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.101+0000 7fa877fff640 1 --2- 172.21.15.97:0/125246765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56089c6ee490 0x56089c705200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.101+0000 7fa877fff640 1 -- 172.21.15.97:0/125246765 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56089c6eedb0 msgr2=0x56089c6f8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:45.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.101+0000 7fa877fff640 1 --2- 172.21.15.97:0/125246765 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56089c6eedb0 0x56089c6f8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.101+0000 7fa877fff640 1 -- 172.21.15.97:0/125246765 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56089c6f0ce0 con 0x56089c6f9480 2024-04-23T10:41:45.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.101+0000 7fa876ffd640 1 --2- 172.21.15.97:0/125246765 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56089c6eedb0 0x56089c6f8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:41:45.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.101+0000 7fa877fff640 1 --2- 172.21.15.97:0/125246765 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56089c6f9480 0x56089c7112c0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fa870012010 tx=0x7fa870026f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72009bdac3efdb7d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:41:45.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.101+0000 7fa85d7fa640 1 -- 172.21.15.97:0/125246765 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa870026b50 con 0x56089c6f9480 2024-04-23T10:41:45.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.101+0000 7fa85d7fa640 1 -- 172.21.15.97:0/125246765 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa870021070 con 0x56089c6f9480 2024-04-23T10:41:45.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.101+0000 7fa85d7fa640 1 -- 172.21.15.97:0/125246765 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87001cab0 con 0x56089c6f9480 2024-04-23T10:41:45.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.102+0000 7fa88080e740 1 -- 172.21.15.97:0/125246765 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56089c6f9480 msgr2=0x56089c7112c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:45.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.102+0000 7fa88080e740 1 --2- 172.21.15.97:0/125246765 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56089c6f9480 0x56089c7112c0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fa870012010 tx=0x7fa870026f20 comp rx=0 tx=0).stop 2024-04-23T10:41:45.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.102+0000 7fa88080e740 1 -- 172.21.15.97:0/125246765 shutdown_connections 2024-04-23T10:41:45.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.102+0000 7fa88080e740 1 --2- 172.21.15.97:0/125246765 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56089c6f9480 0x56089c7112c0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.102+0000 7fa88080e740 1 --2- 172.21.15.97:0/125246765 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56089c6eedb0 0x56089c6f8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.106 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.102+0000 7fa88080e740 1 --2- 172.21.15.97:0/125246765 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56089c6ee490 0x56089c705200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.102+0000 7fa88080e740 1 -- 172.21.15.97:0/125246765 >> 172.21.15.97:0/125246765 conn(0x56089c6ea7b0 msgr2=0x56089c6f04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:41:45.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.102+0000 7fa88080e740 1 -- 172.21.15.97:0/125246765 shutdown_connections 2024-04-23T10:41:45.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.102+0000 7fa88080e740 1 -- 172.21.15.97:0/125246765 wait complete. 2024-04-23T10:41:45.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.102+0000 7fa88080e740 1 -- start start 2024-04-23T10:41:45.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.102+0000 7fa88080e740 1 Processor -- start 2024-04-23T10:41:45.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.103+0000 7fa88080e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56089c6ee490 0x56089c701720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:45.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.103+0000 7fa88080e740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56089c6eedb0 0x56089c701c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:45.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.103+0000 7fa88080e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56089c6f9480 0x56089c6fc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:45.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.103+0000 7fa88080e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa8781acdf0 con 0x56089c6f9480 2024-04-23T10:41:45.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.103+0000 7fa88080e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa8781ac720 con 0x56089c6ee490 2024-04-23T10:41:45.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.103+0000 7fa88080e740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa8781a9980 con 0x56089c6eedb0 2024-04-23T10:41:45.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.103+0000 7fa88080e740 10 client.0 _ll_register_callbacks cb 0x7fa8700136a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:41:45.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.103+0000 7fa877fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56089c6f9480 0x56089c6fc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:45.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.103+0000 7fa877fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56089c6f9480 0x56089c6fc7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37968/0 (socket says 172.21.15.97:37968) 2024-04-23T10:41:45.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.103+0000 7fa877fff640 1 -- 172.21.15.97:0/3625121366 learned_addr learned my addr 172.21.15.97:0/3625121366 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:41:45.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.103+0000 7fa8777fe640 1 --2- 172.21.15.97:0/3625121366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56089c6ee490 0x56089c701720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:45.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.103+0000 7fa876ffd640 1 --2- 172.21.15.97:0/3625121366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56089c6eedb0 0x56089c701c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:45.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.103+0000 7fa877fff640 1 -- 172.21.15.97:0/3625121366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56089c6eedb0 msgr2=0x56089c701c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:45.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.103+0000 7fa877fff640 1 --2- 172.21.15.97:0/3625121366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56089c6eedb0 0x56089c701c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.103+0000 7fa877fff640 1 -- 172.21.15.97:0/3625121366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56089c6ee490 msgr2=0x56089c701720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:45.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.103+0000 7fa877fff640 1 --2- 172.21.15.97:0/3625121366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56089c6ee490 0x56089c701720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.103+0000 7fa877fff640 1 -- 172.21.15.97:0/3625121366 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56089c6fcf90 con 0x56089c6f9480 2024-04-23T10:41:45.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.103+0000 7fa877fff640 1 --2- 172.21.15.97:0/3625121366 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56089c6f9480 0x56089c6fc7c0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fa870011d90 tx=0x7fa87000fd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59d55fb227a92f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:41:45.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.103+0000 7fa856ffd640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:41:45.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.103+0000 7fa856ffd640 1 -- 172.21.15.97:0/3625121366 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa870026680 con 0x56089c6f9480 2024-04-23T10:41:45.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.103+0000 7fa856ffd640 1 -- 172.21.15.97:0/3625121366 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8484+0+0 (secure 0 0 0) 0x7fa87001c7c0 con 0x56089c6f9480 2024-04-23T10:41:45.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.103+0000 7fa88080e740 1 -- 172.21.15.97:0/3625121366 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa870021070 con 0x56089c6f9480 2024-04-23T10:41:45.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.103+0000 7fa856ffd640 1 -- 172.21.15.97:0/3625121366 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa87001c9c0 con 0x56089c6f9480 2024-04-23T10:41:45.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.104+0000 7fa88080e740 10 client.17715 fetch_fsmap learned FSMap version 26 2024-04-23T10:41:45.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.104+0000 7fa88080e740 1 -- 172.21.15.97:0/3625121366 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56089c770660 con 0x56089c6f9480 2024-04-23T10:41:45.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.104+0000 7fa856ffd640 1 -- 172.21.15.97:0/3625121366 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa870021070 con 0x56089c6f9480 2024-04-23T10:41:45.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.104+0000 7fa856ffd640 1 -- 172.21.15.97:0/3625121366 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa870048020 con 0x56089c6f9480 2024-04-23T10:41:45.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.104+0000 7fa88080e740 10 client.17715 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:41:45.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.104+0000 7fa88080e740 10 client.17715 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:41:45.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.104+0000 7fa88080e740 20 client.17715 populate_metadata read hostname 'smithi097' 2024-04-23T10:41:45.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.104+0000 7fa88080e740 1 --2- 172.21.15.97:0/3625121366 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x56089c770c20 0x56089c773040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:45.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.104+0000 7fa88080e740 4 client.17715 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "8d2d887a", "release": true, "await": true}] 2024-04-23T10:41:45.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.104+0000 7fa88080e740 1 -- 172.21.15.97:0/3625121366 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "quiesce db", "set_id": "8d2d887a", "release": true, "await": true}) -- 0x56089c6610f0 con 0x56089c770c20 2024-04-23T10:41:45.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.104+0000 7fa8777fe640 1 --2- 172.21.15.97:0/3625121366 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x56089c770c20 0x56089c773040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:45.109 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.104+0000 7fa856ffd640 10 client.17715 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:41:45.110 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.104+0000 7fa8777fe640 1 --2- 172.21.15.97:0/3625121366 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x56089c770c20 0x56089c773040 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26a82defbc1f80ae server_cookie=3bf7c7d7a0e40d40 in_seq=0 out_seq=0 2024-04-23T10:41:45.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.108+0000 7fa856ffd640 1 -- 172.21.15.97:0/3625121366 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x56089c6610f0 con 0x56089c770c20 2024-04-23T10:41:45.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.108+0000 7fa856ffd640 10 client.17715 handle_command_reply: tid=0 2024-04-23T10:41:45.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.108+0000 7fa88080e740 1 client.17715 shutdown 2024-04-23T10:41:45.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.108+0000 7fa88080e740 1 -- 172.21.15.97:0/3625121366 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56089c6f9480 msgr2=0x56089c6fc7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:45.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.108+0000 7fa88080e740 1 --2- 172.21.15.97:0/3625121366 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56089c6f9480 0x56089c6fc7c0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fa870011d90 tx=0x7fa87000fd80 comp rx=0 tx=0).stop 2024-04-23T10:41:45.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.109+0000 7fa88080e740 1 -- 172.21.15.97:0/3625121366 shutdown_connections 2024-04-23T10:41:45.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.109+0000 7fa88080e740 1 --2- 172.21.15.97:0/3625121366 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x56089c770c20 0x56089c773040 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.109+0000 7fa856ffd640 0 client.17715 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:41:45.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.109+0000 7fa88080e740 1 --2- 172.21.15.97:0/3625121366 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56089c6f9480 0x56089c6fc7c0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.109+0000 7fa88080e740 1 --2- 172.21.15.97:0/3625121366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56089c6eedb0 0x56089c701c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.111 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.109+0000 7fa88080e740 1 --2- 172.21.15.97:0/3625121366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56089c6ee490 0x56089c701720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.109+0000 7fa88080e740 1 -- 172.21.15.97:0/3625121366 >> 172.21.15.97:0/3625121366 conn(0x56089c6ea7b0 msgr2=0x56089c7041f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:41:45.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.109+0000 7fa88080e740 1 -- 172.21.15.97:0/3625121366 shutdown_connections 2024-04-23T10:41:45.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.109+0000 7fa88080e740 1 -- 172.21.15.97:0/3625121366 wait complete. 2024-04-23T10:41:45.112 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.109+0000 7fa88080e740 20 client.17715 trim_cache size 0 max 16384 2024-04-23T10:41:45.112 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-23T10:41:45.112 INFO:teuthology.orchestra.run.smithi097.stdout: "epoch": 26, 2024-04-23T10:41:45.112 INFO:teuthology.orchestra.run.smithi097.stdout: "leader": 24488, 2024-04-23T10:41:45.112 INFO:teuthology.orchestra.run.smithi097.stdout: "set_version": 64, 2024-04-23T10:41:45.112 INFO:teuthology.orchestra.run.smithi097.stdout: "sets": { 2024-04-23T10:41:45.112 INFO:teuthology.orchestra.run.smithi097.stdout: "8d2d887a": { 2024-04-23T10:41:45.112 INFO:teuthology.orchestra.run.smithi097.stdout: "version": 64, 2024-04-23T10:41:45.113 INFO:teuthology.orchestra.run.smithi097.stdout: "age_ref": 0.0, 2024-04-23T10:41:45.113 INFO:teuthology.orchestra.run.smithi097.stdout: "state": { 2024-04-23T10:41:45.113 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RELEASED", 2024-04-23T10:41:45.113 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0 2024-04-23T10:41:45.113 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:41:45.113 INFO:teuthology.orchestra.run.smithi097.stdout: "timeout": 60.0, 2024-04-23T10:41:45.113 INFO:teuthology.orchestra.run.smithi097.stdout: "expiration": 84.7, 2024-04-23T10:41:45.113 INFO:teuthology.orchestra.run.smithi097.stdout: "members": { 2024-04-23T10:41:45.113 INFO:teuthology.orchestra.run.smithi097.stdout: "file:/": { 2024-04-23T10:41:45.113 INFO:teuthology.orchestra.run.smithi097.stdout: "excluded": false, 2024-04-23T10:41:45.113 INFO:teuthology.orchestra.run.smithi097.stdout: "state": { 2024-04-23T10:41:45.113 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RELEASED", 2024-04-23T10:41:45.113 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0 2024-04-23T10:41:45.113 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:41:45.114 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:41:45.114 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:41:45.114 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:41:45.114 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:41:45.114 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-23T10:41:45.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.109+0000 7fa87dafd640 1 -- 172.21.15.97:0/897168985 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fa8600a5f30 msgr2=0x7fa8600a83f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:45.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.109+0000 7fa87dafd640 1 --2- 172.21.15.97:0/897168985 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fa8600a5f30 0x7fa8600a83f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa8700092f0 tx=0x7fa870009280 comp rx=0 tx=0).stop 2024-04-23T10:41:45.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.109+0000 7fa87dafd640 1 -- 172.21.15.97:0/897168985 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa878106160 msgr2=0x7fa878196ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:45.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.109+0000 7fa87dafd640 1 --2- 172.21.15.97:0/897168985 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa878106160 0x7fa878196ac0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fa86c032fd0 tx=0x7fa86c004270 comp rx=0 tx=0).stop 2024-04-23T10:41:45.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.109+0000 7fa87dafd640 1 -- 172.21.15.97:0/897168985 shutdown_connections 2024-04-23T10:41:45.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.109+0000 7fa87dafd640 1 --2- 172.21.15.97:0/897168985 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fa8600a5f30 0x7fa8600a83f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.109+0000 7fa87dafd640 1 --2- 172.21.15.97:0/897168985 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa87819ae40 0x7fa87819b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.109+0000 7fa87dafd640 1 --2- 172.21.15.97:0/897168985 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa878106160 0x7fa878196ac0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.109+0000 7fa87dafd640 1 --2- 172.21.15.97:0/897168985 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa878101090 0x7fa878196580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.115 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.109+0000 7fa87dafd640 1 -- 172.21.15.97:0/897168985 >> 172.21.15.97:0/897168985 conn(0x7fa878077f50 msgr2=0x7fa8780ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:41:45.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.109+0000 7fa87dafd640 1 -- 172.21.15.97:0/897168985 shutdown_connections 2024-04-23T10:41:45.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.109+0000 7fa87dafd640 1 -- 172.21.15.97:0/897168985 wait complete. 2024-04-23T10:41:45.119 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '8d2d887a', seconds elapsed: 25.7 2024-04-23T10:41:45.120 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.65 2024-04-23T10:41:45.120 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T10:41:45.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.219+0000 7f9ca16cb640 1 -- 172.21.15.97:0/2290584199 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c9c1010a0 msgr2=0x7f9c9c101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:45.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.219+0000 7f9ca16cb640 1 --2- 172.21.15.97:0/2290584199 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c9c1010a0 0x7f9c9c101480 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c009ca0 tx=0x7f9c8c02a8c0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.219+0000 7f9ca16cb640 1 -- 172.21.15.97:0/2290584199 shutdown_connections 2024-04-23T10:41:45.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.219+0000 7f9ca16cb640 1 --2- 172.21.15.97:0/2290584199 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c9c106170 0x7f9c9c10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.219+0000 7f9ca16cb640 1 --2- 172.21.15.97:0/2290584199 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9c9c101a50 0x7f9c9c105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.219+0000 7f9ca16cb640 1 --2- 172.21.15.97:0/2290584199 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c9c1010a0 0x7f9c9c101480 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.219+0000 7f9ca16cb640 1 -- 172.21.15.97:0/2290584199 >> 172.21.15.97:0/2290584199 conn(0x7f9c9c077f50 msgr2=0x7f9c9c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:41:45.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.219+0000 7f9ca16cb640 1 -- 172.21.15.97:0/2290584199 shutdown_connections 2024-04-23T10:41:45.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.219+0000 7f9ca16cb640 1 -- 172.21.15.97:0/2290584199 wait complete. 2024-04-23T10:41:45.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.220+0000 7f9ca16cb640 1 Processor -- start 2024-04-23T10:41:45.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.220+0000 7f9ca16cb640 1 -- start start 2024-04-23T10:41:45.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.220+0000 7f9ca16cb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c9c1010a0 0x7f9c9c10e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:45.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.220+0000 7f9ca16cb640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9c9c106170 0x7f9c9c10ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:45.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.220+0000 7f9ca16cb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c9c10f150 0x7f9c9c1a6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:45.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.220+0000 7f9ca16cb640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9c9c1121f0 con 0x7f9c9c1010a0 2024-04-23T10:41:45.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.220+0000 7f9ca16cb640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9c9c112070 con 0x7f9c9c106170 2024-04-23T10:41:45.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.220+0000 7f9ca16cb640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f9c9c112370 con 0x7f9c9c10f150 2024-04-23T10:41:45.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.220+0000 7f9c9affd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c9c1010a0 0x7f9c9c10e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:45.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.220+0000 7f9c9affd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c9c1010a0 0x7f9c9c10e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:37980/0 (socket says 172.21.15.97:37980) 2024-04-23T10:41:45.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.220+0000 7f9c9affd640 1 -- 172.21.15.97:0/1652892200 learned_addr learned my addr 172.21.15.97:0/1652892200 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:41:45.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.220+0000 7f9c9a7fc640 1 --2- 172.21.15.97:0/1652892200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9c9c106170 0x7f9c9c10ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:45.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.221+0000 7f9c9b7fe640 1 --2- 172.21.15.97:0/1652892200 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c9c10f150 0x7f9c9c1a6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:45.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.221+0000 7f9c9affd640 1 -- 172.21.15.97:0/1652892200 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c9c10f150 msgr2=0x7f9c9c1a6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:45.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.221+0000 7f9c9affd640 1 --2- 172.21.15.97:0/1652892200 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c9c10f150 0x7f9c9c1a6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.221+0000 7f9c9affd640 1 -- 172.21.15.97:0/1652892200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9c9c106170 msgr2=0x7f9c9c10ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:45.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.221+0000 7f9c9affd640 1 --2- 172.21.15.97:0/1652892200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9c9c106170 0x7f9c9c10ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.221+0000 7f9c9affd640 1 -- 172.21.15.97:0/1652892200 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c9c1a7020 con 0x7f9c9c1010a0 2024-04-23T10:41:45.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.221+0000 7f9c9b7fe640 1 --2- 172.21.15.97:0/1652892200 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c9c10f150 0x7f9c9c1a6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:41:45.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.221+0000 7f9c9a7fc640 1 --2- 172.21.15.97:0/1652892200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9c9c106170 0x7f9c9c10ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:41:45.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.221+0000 7f9c9affd640 1 --2- 172.21.15.97:0/1652892200 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c9c1010a0 0x7f9c9c10e6d0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c02ade0 tx=0x7f9c8c031770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:41:45.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.221+0000 7f9c83fff640 1 -- 172.21.15.97:0/1652892200 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c003b70 con 0x7f9c9c1010a0 2024-04-23T10:41:45.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.221+0000 7f9c83fff640 1 -- 172.21.15.97:0/1652892200 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c8c003d10 con 0x7f9c9c1010a0 2024-04-23T10:41:45.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.221+0000 7f9ca16cb640 1 -- 172.21.15.97:0/1652892200 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c9c1a72b0 con 0x7f9c9c1010a0 2024-04-23T10:41:45.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.221+0000 7f9c83fff640 1 -- 172.21.15.97:0/1652892200 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c02fa50 con 0x7f9c9c1010a0 2024-04-23T10:41:45.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.221+0000 7f9ca16cb640 1 -- 172.21.15.97:0/1652892200 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c9c1a7800 con 0x7f9c9c1010a0 2024-04-23T10:41:45.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.222+0000 7f9ca16cb640 1 -- 172.21.15.97:0/1652892200 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c9c068d70 con 0x7f9c9c1010a0 2024-04-23T10:41:45.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.223+0000 7f9c83fff640 1 -- 172.21.15.97:0/1652892200 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9c8c004580 con 0x7f9c9c1010a0 2024-04-23T10:41:45.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.224+0000 7f9c83fff640 1 --2- 172.21.15.97:0/1652892200 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f9c840a3d50 0x7f9c840a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:45.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.224+0000 7f9c9a7fc640 1 --2- 172.21.15.97:0/1652892200 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f9c840a3d50 0x7f9c840a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:45.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.224+0000 7f9c83fff640 1 -- 172.21.15.97:0/1652892200 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8484+0+0 (secure 0 0 0) 0x7f9c8c0f1d50 con 0x7f9c9c1010a0 2024-04-23T10:41:45.225 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.224+0000 7f9c9a7fc640 1 --2- 172.21.15.97:0/1652892200 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f9c840a3d50 0x7f9c840a6210 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9c900058c0 tx=0x7f9c90005850 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:41:45.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.227+0000 7f9c83fff640 1 -- 172.21.15.97:0/1652892200 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9c8c0be520 con 0x7f9c9c1010a0 2024-04-23T10:41:45.443 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.444+0000 7f9ca16cb640 1 -- 172.21.15.97:0/1652892200 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9c9c1093a0 con 0x7f9c9c1010a0 2024-04-23T10:41:45.444 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.445+0000 7f9c83fff640 1 -- 172.21.15.97:0/1652892200 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11931 (secure 0 0 0) 0x7f9c8c0bde20 con 0x7f9c9c1010a0 2024-04-23T10:41:45.444 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:41:45.444 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3946878762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3946878762},{"type":"v1","addr":"172.21.15.97:6835","nonce":3946878762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/209800861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":209800861},{"type":"v1","addr":"172.21.15.97:6837","nonce":209800861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T10:24:06.910134+0000","modified":"2024-04-23T10:26:09.820985+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6839/418013575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":418013575},{"type":"v1","addr":"172.21.15.97:6839","nonce":418013575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6841/747242680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":747242680},{"type":"v1","addr":"172.21.15.97:6841","nonce":747242680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6833/4292847121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4292847121},{"type":"v1","addr":"172.21.15.183:6833","nonce":4292847121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6833/3885816565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3885816565},{"type":"v1","addr":"172.21.15.196:6833","nonce":3885816565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1897093105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1897093105},{"type":"v1","addr":"172.21.15.183:6835","nonce":1897093105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1171477563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1171477563},{"type":"v1","addr":"172.21.15.196:6835","nonce":1171477563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1898341911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1898341911},{"type":"v1","addr":"172.21.15.183:6837","nonce":1898341911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.196:6837/3094946502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3094946502},{"type":"v1","addr":"172.21.15.196:6837","nonce":3094946502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.196:6839/83843556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":83843556},{"type":"v1","addr":"172.21.15.196:6839","nonce":83843556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6839/1763841130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1763841130},{"type":"v1","addr":"172.21.15.183:6839","nonce":1763841130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24487,24403,24398,24469,24488]},"id":1}]} 2024-04-23T10:41:45.445 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 26 2024-04-23T10:41:45.446 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.447+0000 7f9ca16cb640 1 -- 172.21.15.97:0/1652892200 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f9c840a3d50 msgr2=0x7f9c840a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:45.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.447+0000 7f9ca16cb640 1 --2- 172.21.15.97:0/1652892200 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f9c840a3d50 0x7f9c840a6210 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9c900058c0 tx=0x7f9c90005850 comp rx=0 tx=0).stop 2024-04-23T10:41:45.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.447+0000 7f9ca16cb640 1 -- 172.21.15.97:0/1652892200 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c9c1010a0 msgr2=0x7f9c9c10e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:45.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.447+0000 7f9ca16cb640 1 --2- 172.21.15.97:0/1652892200 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c9c1010a0 0x7f9c9c10e6d0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c02ade0 tx=0x7f9c8c031770 comp rx=0 tx=0).stop 2024-04-23T10:41:45.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.447+0000 7f9ca16cb640 1 -- 172.21.15.97:0/1652892200 shutdown_connections 2024-04-23T10:41:45.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.447+0000 7f9ca16cb640 1 --2- 172.21.15.97:0/1652892200 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f9c840a3d50 0x7f9c840a6210 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.447+0000 7f9ca16cb640 1 --2- 172.21.15.97:0/1652892200 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c9c10f150 0x7f9c9c1a6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.447+0000 7f9ca16cb640 1 --2- 172.21.15.97:0/1652892200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9c9c106170 0x7f9c9c10ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.447+0000 7f9ca16cb640 1 --2- 172.21.15.97:0/1652892200 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c9c1010a0 0x7f9c9c10e6d0 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.447+0000 7f9ca16cb640 1 -- 172.21.15.97:0/1652892200 >> 172.21.15.97:0/1652892200 conn(0x7f9c9c077f50 msgr2=0x7f9c9c0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:41:45.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.447+0000 7f9ca16cb640 1 -- 172.21.15.97:0/1652892200 shutdown_connections 2024-04-23T10:41:45.447 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.447+0000 7f9ca16cb640 1 -- 172.21.15.97:0/1652892200 wait complete. 2024-04-23T10:41:45.458 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db / --timeout 60 --expiration 84.65 --await-for 42.0' 2024-04-23T10:41:45.458 DEBUG:teuthology.orchestra.run.smithi097:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24488 quiesce db / --timeout 60 --expiration 84.65 --await-for 42.0 2024-04-23T10:41:45.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.567+0000 7f3ef8919640 1 -- 172.21.15.97:0/4072893783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ef4101a40 msgr2=0x7f3ef4105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:45.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.567+0000 7f3ef8919640 1 --2- 172.21.15.97:0/4072893783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ef4101a40 0x7f3ef4105a80 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f3edc009900 tx=0x7f3edc02a410 comp rx=0 tx=0).stop 2024-04-23T10:41:45.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.567+0000 7f3ef8919640 1 -- 172.21.15.97:0/4072893783 shutdown_connections 2024-04-23T10:41:45.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.567+0000 7f3ef8919640 1 --2- 172.21.15.97:0/4072893783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ef4106160 0x7f3ef410dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.567+0000 7f3ef8919640 1 --2- 172.21.15.97:0/4072893783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ef4101a40 0x7f3ef4105a80 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.567+0000 7f3ef8919640 1 --2- 172.21.15.97:0/4072893783 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ef4101090 0x7f3ef4101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.567+0000 7f3ef8919640 1 -- 172.21.15.97:0/4072893783 >> 172.21.15.97:0/4072893783 conn(0x7f3ef40771d0 msgr2=0x7f3ef40775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:41:45.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.567+0000 7f3ef8919640 1 -- 172.21.15.97:0/4072893783 shutdown_connections 2024-04-23T10:41:45.569 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.567+0000 7f3ef8919640 1 -- 172.21.15.97:0/4072893783 wait complete. 2024-04-23T10:41:45.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.568+0000 7f3ef8919640 1 Processor -- start 2024-04-23T10:41:45.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.568+0000 7f3ef8919640 1 -- start start 2024-04-23T10:41:45.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.568+0000 7f3ef8919640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ef4101090 0x7f3ef4199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:45.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.568+0000 7f3ef8919640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ef4106160 0x7f3ef4199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:45.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.568+0000 7f3ef8919640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ef4199ff0 0x7f3ef41a45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:45.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.568+0000 7f3ef8919640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef4111080 con 0x7f3ef4199ff0 2024-04-23T10:41:45.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.568+0000 7f3ef8919640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef4110f00 con 0x7f3ef4106160 2024-04-23T10:41:45.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.568+0000 7f3ef8919640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef4111200 con 0x7f3ef4101090 2024-04-23T10:41:45.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.568+0000 7f3ef2d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ef4199ff0 0x7f3ef41a45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:45.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.568+0000 7f3ef2d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ef4199ff0 0x7f3ef41a45a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34616/0 (socket says 172.21.15.97:34616) 2024-04-23T10:41:45.570 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.568+0000 7f3ef2d9d640 1 -- 172.21.15.97:0/268478777 learned_addr learned my addr 172.21.15.97:0/268478777 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:41:45.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.568+0000 7f3ef259c640 1 --2- 172.21.15.97:0/268478777 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ef4101090 0x7f3ef4199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:45.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.568+0000 7f3eebfff640 1 --2- 172.21.15.97:0/268478777 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ef4106160 0x7f3ef4199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:45.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.568+0000 7f3ef2d9d640 1 -- 172.21.15.97:0/268478777 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ef4101090 msgr2=0x7f3ef4199520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:45.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.568+0000 7f3ef2d9d640 1 --2- 172.21.15.97:0/268478777 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ef4101090 0x7f3ef4199520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.568+0000 7f3ef2d9d640 1 -- 172.21.15.97:0/268478777 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ef4106160 msgr2=0x7f3ef4199a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:45.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.568+0000 7f3ef2d9d640 1 --2- 172.21.15.97:0/268478777 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ef4106160 0x7f3ef4199a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.568+0000 7f3ef2d9d640 1 -- 172.21.15.97:0/268478777 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ef41a4da0 con 0x7f3ef4199ff0 2024-04-23T10:41:45.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.569+0000 7f3ef2d9d640 1 --2- 172.21.15.97:0/268478777 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ef4199ff0 0x7f3ef41a45a0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f3eec00cad0 tx=0x7f3eec00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:41:45.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.569+0000 7f3eeb7fe640 1 -- 172.21.15.97:0/268478777 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eec00ae20 con 0x7f3ef4199ff0 2024-04-23T10:41:45.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.569+0000 7f3eeb7fe640 1 -- 172.21.15.97:0/268478777 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3eec005020 con 0x7f3ef4199ff0 2024-04-23T10:41:45.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.569+0000 7f3ef8919640 1 -- 172.21.15.97:0/268478777 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ef41a5090 con 0x7f3ef4199ff0 2024-04-23T10:41:45.571 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.569+0000 7f3eeb7fe640 1 -- 172.21.15.97:0/268478777 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eec00f6b0 con 0x7f3ef4199ff0 2024-04-23T10:41:45.572 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.569+0000 7f3ef8919640 1 -- 172.21.15.97:0/268478777 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ef41a5590 con 0x7f3ef4199ff0 2024-04-23T10:41:45.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.571+0000 7f3eeb7fe640 1 -- 172.21.15.97:0/268478777 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3eec0051d0 con 0x7f3ef4199ff0 2024-04-23T10:41:45.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.571+0000 7f3efb62c740 1 Processor -- start 2024-04-23T10:41:45.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.572+0000 7f3efb62c740 1 -- start start 2024-04-23T10:41:45.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.572+0000 7f3efb62c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564c513760a0 0x564c51376480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:45.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.572+0000 7f3efb62c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564c513769c0 0x564c51380b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:45.579 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.572+0000 7f3efb62c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564c51381090 0x564c51383480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:45.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.572+0000 7f3efb62c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x564c512db370 con 0x564c513769c0 2024-04-23T10:41:45.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.572+0000 7f3efb62c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564c512782e0 con 0x564c513760a0 2024-04-23T10:41:45.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.572+0000 7f3efb62c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564c512d7330 con 0x564c51381090 2024-04-23T10:41:45.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.572+0000 7f3ef259c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564c513769c0 0x564c51380b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:45.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.572+0000 7f3ef259c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564c513769c0 0x564c51380b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34622/0 (socket says 172.21.15.97:34622) 2024-04-23T10:41:45.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.572+0000 7f3ef259c640 1 -- 172.21.15.97:0/404871885 learned_addr learned my addr 172.21.15.97:0/404871885 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:41:45.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.572+0000 7f3eeb7fe640 1 --2- 172.21.15.97:0/268478777 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f3ee00a3d20 0x7f3ee00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:45.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.572+0000 7f3eebfff640 1 --2- 172.21.15.97:0/404871885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564c513760a0 0x564c51376480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:45.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.572+0000 7f3eebfff640 1 --2- 172.21.15.97:0/404871885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564c51381090 0x564c51383480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:45.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.572+0000 7f3ef259c640 1 -- 172.21.15.97:0/404871885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564c51381090 msgr2=0x564c51383480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:45.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.572+0000 7f3ef2d9d640 1 --2- 172.21.15.97:0/268478777 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f3ee00a3d20 0x7f3ee00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:45.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.572+0000 7f3ef259c640 1 --2- 172.21.15.97:0/404871885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564c51381090 0x564c51383480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.572+0000 7f3ef259c640 1 -- 172.21.15.97:0/404871885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564c513760a0 msgr2=0x564c51376480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:45.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.572+0000 7f3ef259c640 1 --2- 172.21.15.97:0/404871885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564c513760a0 0x564c51376480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.572+0000 7f3ef259c640 1 -- 172.21.15.97:0/404871885 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c510b9d80 con 0x564c513769c0 2024-04-23T10:41:45.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.572+0000 7f3eebfff640 1 --2- 172.21.15.97:0/404871885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564c51381090 0x564c51383480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:41:45.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.572+0000 7f3eeb7fe640 1 -- 172.21.15.97:0/268478777 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8484+0+0 (secure 0 0 0) 0x7f3eec0d0910 con 0x7f3ef4199ff0 2024-04-23T10:41:45.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.573+0000 7f3ef259c640 1 --2- 172.21.15.97:0/404871885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564c513769c0 0x564c51380b50 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f3ee400cc60 tx=0x7f3ee400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9031c06b23db846 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:41:45.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.573+0000 7f3ee8ff9640 1 -- 172.21.15.97:0/404871885 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee4014070 con 0x564c513769c0 2024-04-23T10:41:45.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.573+0000 7f3ef2d9d640 1 --2- 172.21.15.97:0/268478777 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f3ee00a3d20 0x7f3ee00a61e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3eec09c5b0 tx=0x7f3eec087040 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:41:45.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.573+0000 7f3ee8ff9640 1 -- 172.21.15.97:0/404871885 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ee40047a0 con 0x564c513769c0 2024-04-23T10:41:45.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.573+0000 7f3ee8ff9640 1 -- 172.21.15.97:0/404871885 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee4010690 con 0x564c513769c0 2024-04-23T10:41:45.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.573+0000 7f3efb62c740 1 -- 172.21.15.97:0/404871885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564c513769c0 msgr2=0x564c51380b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:45.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.573+0000 7f3efb62c740 1 --2- 172.21.15.97:0/404871885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564c513769c0 0x564c51380b50 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f3ee400cc60 tx=0x7f3ee400e720 comp rx=0 tx=0).stop 2024-04-23T10:41:45.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.574+0000 7f3efb62c740 1 -- 172.21.15.97:0/404871885 shutdown_connections 2024-04-23T10:41:45.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.574+0000 7f3efb62c740 1 --2- 172.21.15.97:0/404871885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564c51381090 0x564c51383480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.574+0000 7f3efb62c740 1 --2- 172.21.15.97:0/404871885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564c513769c0 0x564c51380b50 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.574+0000 7f3efb62c740 1 --2- 172.21.15.97:0/404871885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564c513760a0 0x564c51376480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.574+0000 7f3efb62c740 1 -- 172.21.15.97:0/404871885 >> 172.21.15.97:0/404871885 conn(0x564c513723c0 msgr2=0x564c513727a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:41:45.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.574+0000 7f3efb62c740 1 -- 172.21.15.97:0/404871885 shutdown_connections 2024-04-23T10:41:45.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.574+0000 7f3efb62c740 1 -- 172.21.15.97:0/404871885 wait complete. 2024-04-23T10:41:45.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.574+0000 7f3efb62c740 1 -- start start 2024-04-23T10:41:45.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.574+0000 7f3efb62c740 1 Processor -- start 2024-04-23T10:41:45.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.574+0000 7f3efb62c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564c513760a0 0x564c5137e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:45.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.574+0000 7f3efb62c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564c513769c0 0x564c5137e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:45.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.574+0000 7f3efb62c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564c51381090 0x564c5137eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:45.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.574+0000 7f3efb62c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x564c51383e80 con 0x564c513769c0 2024-04-23T10:41:45.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.574+0000 7f3efb62c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564c51275fb0 con 0x564c51381090 2024-04-23T10:41:45.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.574+0000 7f3efb62c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x564c51384000 con 0x564c513760a0 2024-04-23T10:41:45.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.574+0000 7f3efb62c740 10 client.0 _ll_register_callbacks cb 0x564c512759a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:41:45.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.574+0000 7f3ef259c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564c513769c0 0x564c5137e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:45.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.574+0000 7f3ef259c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564c513769c0 0x564c5137e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34636/0 (socket says 172.21.15.97:34636) 2024-04-23T10:41:45.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.574+0000 7f3ef259c640 1 -- 172.21.15.97:0/1292623657 learned_addr learned my addr 172.21.15.97:0/1292623657 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:41:45.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.575+0000 7f3eebfff640 1 --2- 172.21.15.97:0/1292623657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564c513760a0 0x564c5137e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:45.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.575+0000 7f3ef259c640 1 -- 172.21.15.97:0/1292623657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564c513760a0 msgr2=0x564c5137e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:45.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.575+0000 7f3ef259c640 1 --2- 172.21.15.97:0/1292623657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564c513760a0 0x564c5137e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.575+0000 7f3eebfff640 1 --2- 172.21.15.97:0/1292623657 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564c51381090 0x564c5137eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:45.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.575+0000 7f3ef259c640 1 -- 172.21.15.97:0/1292623657 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564c51381090 msgr2=0x564c5137eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:45.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.575+0000 7f3ef259c640 1 --2- 172.21.15.97:0/1292623657 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564c51381090 0x564c5137eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.575+0000 7f3ef259c640 1 -- 172.21.15.97:0/1292623657 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c5111a510 con 0x564c513769c0 2024-04-23T10:41:45.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.575+0000 7f3ef259c640 1 --2- 172.21.15.97:0/1292623657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564c513769c0 0x564c5137e570 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4010be0 tx=0x7f3ee4010c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e6f05251bba0e36 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:41:45.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.575+0000 7f3ed9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:41:45.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.575+0000 7f3ed9ffb640 1 -- 172.21.15.97:0/1292623657 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee4014070 con 0x564c513769c0 2024-04-23T10:41:45.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.575+0000 7f3efb62c740 1 -- 172.21.15.97:0/1292623657 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ee40047a0 con 0x564c513769c0 2024-04-23T10:41:45.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.575+0000 7f3ed9ffb640 1 -- 172.21.15.97:0/1292623657 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8484+0+0 (secure 0 0 0) 0x7f3ee4004020 con 0x564c513769c0 2024-04-23T10:41:45.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.575+0000 7f3ed9ffb640 1 -- 172.21.15.97:0/1292623657 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3ee4004220 con 0x564c513769c0 2024-04-23T10:41:45.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.575+0000 7f3efb62c740 10 client.17733 fetch_fsmap learned FSMap version 26 2024-04-23T10:41:45.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.575+0000 7f3efb62c740 1 -- 172.21.15.97:0/1292623657 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564c50fad670 con 0x564c513769c0 2024-04-23T10:41:45.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.575+0000 7f3ed9ffb640 1 -- 172.21.15.97:0/1292623657 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ee40047a0 con 0x564c513769c0 2024-04-23T10:41:45.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.575+0000 7f3ed9ffb640 1 -- 172.21.15.97:0/1292623657 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3ee4029030 con 0x564c513769c0 2024-04-23T10:41:45.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.575+0000 7f3efb62c740 10 client.17733 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:41:45.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.575+0000 7f3efb62c740 10 client.17733 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:41:45.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.576+0000 7f3efb62c740 20 client.17733 populate_metadata read hostname 'smithi097' 2024-04-23T10:41:45.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.576+0000 7f3efb62c740 1 --2- 172.21.15.97:0/1292623657 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x564c51391340 0x564c51393730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:45.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.576+0000 7f3efb62c740 4 client.17733 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:41:45.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.576+0000 7f3efb62c740 1 -- 172.21.15.97:0/1292623657 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564c512e8d90 con 0x564c51391340 2024-04-23T10:41:45.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.576+0000 7f3eebfff640 1 --2- 172.21.15.97:0/1292623657 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x564c51391340 0x564c51393730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:45.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.576+0000 7f3eebfff640 1 --2- 172.21.15.97:0/1292623657 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x564c51391340 0x564c51393730 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e432dc91fdbc6a1 server_cookie=182b8830434689f7 in_seq=0 out_seq=0 2024-04-23T10:41:45.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.576+0000 7f3ed9ffb640 10 client.17733 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:41:45.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.577+0000 7f3ed9ffb640 1 -- 172.21.15.97:0/1292623657 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564c512e8d90 con 0x564c51391340 2024-04-23T10:41:45.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.577+0000 7f3ed9ffb640 10 client.17733 handle_command_reply: tid=0 2024-04-23T10:41:45.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.577+0000 7f3efb62c740 1 client.17733 shutdown 2024-04-23T10:41:45.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.577+0000 7f3efb62c740 1 -- 172.21.15.97:0/1292623657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564c513769c0 msgr2=0x564c5137e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:45.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.577+0000 7f3efb62c740 1 --2- 172.21.15.97:0/1292623657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564c513769c0 0x564c5137e570 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4010be0 tx=0x7f3ee4010c10 comp rx=0 tx=0).stop 2024-04-23T10:41:45.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.577+0000 7f3efb62c740 1 -- 172.21.15.97:0/1292623657 shutdown_connections 2024-04-23T10:41:45.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.577+0000 7f3efb62c740 1 --2- 172.21.15.97:0/1292623657 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x564c51391340 0x564c51393730 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.577+0000 7f3efb62c740 1 --2- 172.21.15.97:0/1292623657 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564c51381090 0x564c5137eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.577+0000 7f3ed9ffb640 0 client.17733 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:41:45.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.577+0000 7f3efb62c740 1 --2- 172.21.15.97:0/1292623657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564c513769c0 0x564c5137e570 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.577+0000 7f3efb62c740 1 --2- 172.21.15.97:0/1292623657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564c513760a0 0x564c5137e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.577+0000 7f3efb62c740 1 -- 172.21.15.97:0/1292623657 >> 172.21.15.97:0/1292623657 conn(0x564c513723c0 msgr2=0x564c51371600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:41:45.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.577+0000 7f3efb62c740 1 -- 172.21.15.97:0/1292623657 shutdown_connections 2024-04-23T10:41:45.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.577+0000 7f3efb62c740 1 -- 172.21.15.97:0/1292623657 wait complete. 2024-04-23T10:41:45.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.577+0000 7f3efb62c740 20 client.17733 trim_cache size 0 max 16384 2024-04-23T10:41:45.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.594+0000 7f3efb62c740 1 Processor -- start 2024-04-23T10:41:45.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.594+0000 7f3efb62c740 1 -- start start 2024-04-23T10:41:45.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.594+0000 7f3efb62c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564c513760a0 0x564c5138cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:45.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.594+0000 7f3efb62c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564c513769c0 0x564c5137ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:45.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.594+0000 7f3efb62c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564c51381090 0x564c51398dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:45.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.594+0000 7f3efb62c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef41adb70 con 0x564c51381090 2024-04-23T10:41:45.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.594+0000 7f3efb62c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef41af6b0 con 0x564c513769c0 2024-04-23T10:41:45.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.594+0000 7f3efb62c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef41ae240 con 0x564c513760a0 2024-04-23T10:41:45.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.595+0000 7f3eebfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564c51381090 0x564c51398dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:45.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.595+0000 7f3eebfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564c51381090 0x564c51398dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34650/0 (socket says 172.21.15.97:34650) 2024-04-23T10:41:45.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.595+0000 7f3eebfff640 1 -- 172.21.15.97:0/2621515244 learned_addr learned my addr 172.21.15.97:0/2621515244 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:41:45.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.595+0000 7f3eebfff640 1 --2- 172.21.15.97:0/2621515244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564c513760a0 0x564c5138cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:45.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.595+0000 7f3eebfff640 1 -- 172.21.15.97:0/2621515244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564c513760a0 msgr2=0x564c5138cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:45.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.595+0000 7f3eebfff640 1 --2- 172.21.15.97:0/2621515244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564c513760a0 0x564c5138cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.595+0000 7f3eebfff640 1 -- 172.21.15.97:0/2621515244 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564c513769c0 msgr2=0x564c5137ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:45.599 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.595+0000 7f3ef259c640 1 --2- 172.21.15.97:0/2621515244 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564c513769c0 0x564c5137ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:45.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.595+0000 7f3eebfff640 1 --2- 172.21.15.97:0/2621515244 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564c513769c0 0x564c5137ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.595+0000 7f3eebfff640 1 -- 172.21.15.97:0/2621515244 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c51393730 con 0x564c51381090 2024-04-23T10:41:45.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.595+0000 7f3ef259c640 1 --2- 172.21.15.97:0/2621515244 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564c513769c0 0x564c5137ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:41:45.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.595+0000 7f3eebfff640 1 --2- 172.21.15.97:0/2621515244 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564c51381090 0x564c51398dc0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f3edc009900 tx=0x7f3edc031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faf185cd214ca972 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:41:45.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.595+0000 7f3edbfff640 1 -- 172.21.15.97:0/2621515244 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3edc035040 con 0x564c51381090 2024-04-23T10:41:45.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.595+0000 7f3edbfff640 1 -- 172.21.15.97:0/2621515244 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3edc005030 con 0x564c51381090 2024-04-23T10:41:45.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.595+0000 7f3edbfff640 1 -- 172.21.15.97:0/2621515244 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3edc044670 con 0x564c51381090 2024-04-23T10:41:45.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.596+0000 7f3efb62c740 1 -- 172.21.15.97:0/2621515244 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564c51381090 msgr2=0x564c51398dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:45.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.596+0000 7f3efb62c740 1 --2- 172.21.15.97:0/2621515244 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564c51381090 0x564c51398dc0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f3edc009900 tx=0x7f3edc031400 comp rx=0 tx=0).stop 2024-04-23T10:41:45.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.596+0000 7f3efb62c740 1 -- 172.21.15.97:0/2621515244 shutdown_connections 2024-04-23T10:41:45.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.596+0000 7f3efb62c740 1 --2- 172.21.15.97:0/2621515244 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564c51381090 0x564c51398dc0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.596+0000 7f3efb62c740 1 --2- 172.21.15.97:0/2621515244 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564c513769c0 0x564c5137ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.600 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.596+0000 7f3efb62c740 1 --2- 172.21.15.97:0/2621515244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564c513760a0 0x564c5138cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.596+0000 7f3efb62c740 1 -- 172.21.15.97:0/2621515244 >> 172.21.15.97:0/2621515244 conn(0x564c513723c0 msgr2=0x564c5137f860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:41:45.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.596+0000 7f3efb62c740 1 -- 172.21.15.97:0/2621515244 shutdown_connections 2024-04-23T10:41:45.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.596+0000 7f3efb62c740 1 -- 172.21.15.97:0/2621515244 wait complete. 2024-04-23T10:41:45.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.596+0000 7f3efb62c740 1 -- start start 2024-04-23T10:41:45.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.596+0000 7f3efb62c740 1 Processor -- start 2024-04-23T10:41:45.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.596+0000 7f3efb62c740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564c513760a0 0x564c51389250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:45.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.596+0000 7f3efb62c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564c513769c0 0x564c51389790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:45.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.596+0000 7f3efb62c740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564c51381090 0x564c51386220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:45.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.596+0000 7f3efb62c740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef41aefe0 con 0x564c51381090 2024-04-23T10:41:45.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.596+0000 7f3efb62c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef41ae910 con 0x564c513769c0 2024-04-23T10:41:45.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.596+0000 7f3efb62c740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef41abb70 con 0x564c513760a0 2024-04-23T10:41:45.601 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.596+0000 7f3efb62c740 10 client.0 _ll_register_callbacks cb 0x7f3ee4010840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:41:45.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.596+0000 7f3eebfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564c51381090 0x564c51386220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:45.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.596+0000 7f3eebfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564c51381090 0x564c51386220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34656/0 (socket says 172.21.15.97:34656) 2024-04-23T10:41:45.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.596+0000 7f3eebfff640 1 -- 172.21.15.97:0/2679482603 learned_addr learned my addr 172.21.15.97:0/2679482603 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:41:45.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.597+0000 7f3ef259c640 1 --2- 172.21.15.97:0/2679482603 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564c513769c0 0x564c51389790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:45.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.597+0000 7f3eebfff640 1 --2- 172.21.15.97:0/2679482603 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564c513760a0 0x564c51389250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:45.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.597+0000 7f3eebfff640 1 -- 172.21.15.97:0/2679482603 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564c513760a0 msgr2=0x564c51389250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:45.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.597+0000 7f3eebfff640 1 --2- 172.21.15.97:0/2679482603 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564c513760a0 0x564c51389250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.597+0000 7f3eebfff640 1 -- 172.21.15.97:0/2679482603 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564c513769c0 msgr2=0x564c51389790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:45.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.597+0000 7f3eebfff640 1 --2- 172.21.15.97:0/2679482603 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564c513769c0 0x564c51389790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.597+0000 7f3eebfff640 1 -- 172.21.15.97:0/2679482603 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c51386a30 con 0x564c51381090 2024-04-23T10:41:45.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.597+0000 7f3eebfff640 1 --2- 172.21.15.97:0/2679482603 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564c51381090 0x564c51386220 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f3edc02a930 tx=0x7f3edc041040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=268a805b03b85623 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:41:45.602 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.597+0000 7f3ed9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:41:45.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.597+0000 7f3ed9ffb640 1 -- 172.21.15.97:0/2679482603 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3edc035040 con 0x564c51381090 2024-04-23T10:41:45.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.597+0000 7f3ed9ffb640 1 -- 172.21.15.97:0/2679482603 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8484+0+0 (secure 0 0 0) 0x7f3edc0318b0 con 0x564c51381090 2024-04-23T10:41:45.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.597+0000 7f3efb62c740 1 -- 172.21.15.97:0/2679482603 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3edc005030 con 0x564c51381090 2024-04-23T10:41:45.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.597+0000 7f3ed9ffb640 1 -- 172.21.15.97:0/2679482603 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3edc0443f0 con 0x564c51381090 2024-04-23T10:41:45.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.597+0000 7f3efb62c740 10 client.17739 fetch_fsmap learned FSMap version 26 2024-04-23T10:41:45.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.597+0000 7f3efb62c740 1 -- 172.21.15.97:0/2679482603 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564c513fc420 con 0x564c51381090 2024-04-23T10:41:45.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.597+0000 7f3ed9ffb640 1 -- 172.21.15.97:0/2679482603 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3edc005030 con 0x564c51381090 2024-04-23T10:41:45.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.597+0000 7f3ed9ffb640 1 -- 172.21.15.97:0/2679482603 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3edc059070 con 0x564c51381090 2024-04-23T10:41:45.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.598+0000 7f3efb62c740 10 client.17739 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:41:45.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.598+0000 7f3efb62c740 10 client.17739 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:41:45.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.598+0000 7f3efb62c740 20 client.17739 populate_metadata read hostname 'smithi097' 2024-04-23T10:41:45.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.598+0000 7f3efb62c740 1 --2- 172.21.15.97:0/2679482603 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x564c513fca80 0x564c513feea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:45.603 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.598+0000 7f3efb62c740 4 client.17739 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 84.65, "await_for": 42.0}] 2024-04-23T10:41:45.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.598+0000 7f3efb62c740 1 -- 172.21.15.97:0/2679482603 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 84.65, "await_for": 42.0}) -- 0x564c512e2c00 con 0x564c513fca80 2024-04-23T10:41:45.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.598+0000 7f3ef259c640 1 --2- 172.21.15.97:0/2679482603 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x564c513fca80 0x564c513feea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:45.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.598+0000 7f3ed9ffb640 10 client.17739 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:41:45.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.598+0000 7f3ef259c640 1 --2- 172.21.15.97:0/2679482603 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x564c513fca80 0x564c513feea0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfc7e28758b802e server_cookie=9b1b28dbbbc50ff in_seq=0 out_seq=0 2024-04-23T10:41:45.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.612+0000 7f3ed9ffb640 1 -- 172.21.15.97:0/2679482603 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x564c512e2c00 con 0x564c513fca80 2024-04-23T10:41:45.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.612+0000 7f3ed9ffb640 10 client.17739 handle_command_reply: tid=0 2024-04-23T10:41:45.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.612+0000 7f3efb62c740 1 client.17739 shutdown 2024-04-23T10:41:45.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.612+0000 7f3efb62c740 1 -- 172.21.15.97:0/2679482603 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564c51381090 msgr2=0x564c51386220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:45.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.612+0000 7f3efb62c740 1 --2- 172.21.15.97:0/2679482603 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564c51381090 0x564c51386220 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f3edc02a930 tx=0x7f3edc041040 comp rx=0 tx=0).stop 2024-04-23T10:41:45.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.612+0000 7f3efb62c740 1 -- 172.21.15.97:0/2679482603 shutdown_connections 2024-04-23T10:41:45.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.612+0000 7f3efb62c740 1 --2- 172.21.15.97:0/2679482603 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x564c513fca80 0x564c513feea0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.612+0000 7f3efb62c740 1 --2- 172.21.15.97:0/2679482603 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564c51381090 0x564c51386220 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.612+0000 7f3efb62c740 1 --2- 172.21.15.97:0/2679482603 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564c513769c0 0x564c51389790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.612+0000 7f3ed9ffb640 0 client.17739 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:41:45.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.612+0000 7f3efb62c740 1 --2- 172.21.15.97:0/2679482603 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564c513760a0 0x564c51389250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.612+0000 7f3efb62c740 1 -- 172.21.15.97:0/2679482603 >> 172.21.15.97:0/2679482603 conn(0x564c513723c0 msgr2=0x564c513800f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:41:45.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.612+0000 7f3efb62c740 1 -- 172.21.15.97:0/2679482603 shutdown_connections 2024-04-23T10:41:45.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.612+0000 7f3efb62c740 1 -- 172.21.15.97:0/2679482603 wait complete. 2024-04-23T10:41:45.612 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.612+0000 7f3efb62c740 20 client.17739 trim_cache size 0 max 16384 2024-04-23T10:41:45.613 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-23T10:41:45.613 INFO:teuthology.orchestra.run.smithi097.stdout: "epoch": 26, 2024-04-23T10:41:45.613 INFO:teuthology.orchestra.run.smithi097.stdout: "leader": 24488, 2024-04-23T10:41:45.613 INFO:teuthology.orchestra.run.smithi097.stdout: "set_version": 66, 2024-04-23T10:41:45.613 INFO:teuthology.orchestra.run.smithi097.stdout: "sets": { 2024-04-23T10:41:45.613 INFO:teuthology.orchestra.run.smithi097.stdout: "26d43730": { 2024-04-23T10:41:45.613 INFO:teuthology.orchestra.run.smithi097.stdout: "version": 66, 2024-04-23T10:41:45.613 INFO:teuthology.orchestra.run.smithi097.stdout: "age_ref": 0.0, 2024-04-23T10:41:45.613 INFO:teuthology.orchestra.run.smithi097.stdout: "state": { 2024-04-23T10:41:45.613 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "QUIESCED", 2024-04-23T10:41:45.613 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0 2024-04-23T10:41:45.614 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:41:45.614 INFO:teuthology.orchestra.run.smithi097.stdout: "timeout": 60.0, 2024-04-23T10:41:45.614 INFO:teuthology.orchestra.run.smithi097.stdout: "expiration": 84.7, 2024-04-23T10:41:45.614 INFO:teuthology.orchestra.run.smithi097.stdout: "members": { 2024-04-23T10:41:45.614 INFO:teuthology.orchestra.run.smithi097.stdout: "file:/": { 2024-04-23T10:41:45.614 INFO:teuthology.orchestra.run.smithi097.stdout: "excluded": false, 2024-04-23T10:41:45.614 INFO:teuthology.orchestra.run.smithi097.stdout: "state": { 2024-04-23T10:41:45.614 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "QUIESCED", 2024-04-23T10:41:45.614 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0 2024-04-23T10:41:45.614 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:41:45.614 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:41:45.614 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:41:45.614 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:41:45.615 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:41:45.615 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-23T10:41:45.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.612+0000 7f3ef8919640 1 -- 172.21.15.97:0/268478777 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f3ee00a3d20 msgr2=0x7f3ee00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:45.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.612+0000 7f3ef8919640 1 --2- 172.21.15.97:0/268478777 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f3ee00a3d20 0x7f3ee00a61e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3eec09c5b0 tx=0x7f3eec087040 comp rx=0 tx=0).stop 2024-04-23T10:41:45.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.612+0000 7f3ef8919640 1 -- 172.21.15.97:0/268478777 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ef4199ff0 msgr2=0x7f3ef41a45a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:45.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.612+0000 7f3ef8919640 1 --2- 172.21.15.97:0/268478777 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ef4199ff0 0x7f3ef41a45a0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f3eec00cad0 tx=0x7f3eec00cf90 comp rx=0 tx=0).stop 2024-04-23T10:41:45.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.613+0000 7f3ef8919640 1 -- 172.21.15.97:0/268478777 shutdown_connections 2024-04-23T10:41:45.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.613+0000 7f3ef8919640 1 --2- 172.21.15.97:0/268478777 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f3ee00a3d20 0x7f3ee00a61e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.613+0000 7f3ef8919640 1 --2- 172.21.15.97:0/268478777 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3ef4199ff0 0x7f3ef41a45a0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.613+0000 7f3ef8919640 1 --2- 172.21.15.97:0/268478777 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ef4106160 0x7f3ef4199a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.613+0000 7f3ef8919640 1 --2- 172.21.15.97:0/268478777 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ef4101090 0x7f3ef4199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:45.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.613+0000 7f3ef8919640 1 -- 172.21.15.97:0/268478777 >> 172.21.15.97:0/268478777 conn(0x7f3ef40771d0 msgr2=0x7f3ef40feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:41:45.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.613+0000 7f3ef8919640 1 -- 172.21.15.97:0/268478777 shutdown_connections 2024-04-23T10:41:45.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:41:45.613+0000 7f3ef8919640 1 -- 172.21.15.97:0/268478777 wait complete. 2024-04-23T10:41:45.622 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '26d43730', quiesce took 0.5 seconds. Will release after: 24.65 2024-04-23T10:41:45.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:41:45 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/2780927010' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:41:45.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:41:45 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/1652892200' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:41:45.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:41:45 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/2780927010' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:41:45.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:41:45 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/1652892200' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:41:45.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:41:45 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/2780927010' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:41:45.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:41:45 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/1652892200' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:41:46.730 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.727+0000 7f7197e10640 1 -- 172.21.15.97:0/2879420499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7190102b80 msgr2=0x7f7190106bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:46.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.727+0000 7f7197e10640 1 --2- 172.21.15.97:0/2879420499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7190102b80 0x7f7190106bc0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f7180009900 tx=0x7f718002a410 comp rx=0 tx=0).stop 2024-04-23T10:41:46.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.727+0000 7f7197e10640 1 -- 172.21.15.97:0/2879420499 shutdown_connections 2024-04-23T10:41:46.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.727+0000 7f7197e10640 1 --2- 172.21.15.97:0/2879420499 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71901072a0 0x7f719010ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:46.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.727+0000 7f7197e10640 1 --2- 172.21.15.97:0/2879420499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7190102b80 0x7f7190106bc0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:46.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.727+0000 7f7197e10640 1 --2- 172.21.15.97:0/2879420499 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71901021d0 0x7f71901025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:46.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.727+0000 7f7197e10640 1 -- 172.21.15.97:0/2879420499 >> 172.21.15.97:0/2879420499 conn(0x7f71900793e0 msgr2=0x7f71900797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:41:46.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.727+0000 7f7197e10640 1 -- 172.21.15.97:0/2879420499 shutdown_connections 2024-04-23T10:41:46.732 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.727+0000 7f7197e10640 1 -- 172.21.15.97:0/2879420499 wait complete. 2024-04-23T10:41:46.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.728+0000 7f7197e10640 1 Processor -- start 2024-04-23T10:41:46.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.728+0000 7f7197e10640 1 -- start start 2024-04-23T10:41:46.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.728+0000 7f7197e10640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71901021d0 0x7f71901986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:46.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.728+0000 7f7197e10640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71901072a0 0x7f7190198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:46.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.728+0000 7f7197e10640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f719019cf80 0x7f719019d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:46.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.728+0000 7f7197e10640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f71901121c0 con 0x7f71901072a0 2024-04-23T10:41:46.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.728+0000 7f7197e10640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7190112040 con 0x7f719019cf80 2024-04-23T10:41:46.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.728+0000 7f7197e10640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7190112340 con 0x7f71901021d0 2024-04-23T10:41:46.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.728+0000 7f71953ab640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71901072a0 0x7f7190198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:46.733 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.728+0000 7f71953ab640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71901072a0 0x7f7190198c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34674/0 (socket says 172.21.15.97:34674) 2024-04-23T10:41:46.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.728+0000 7f71953ab640 1 -- 172.21.15.97:0/4076044 learned_addr learned my addr 172.21.15.97:0/4076044 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:41:46.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.728+0000 7f71963ad640 1 --2- 172.21.15.97:0/4076044 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f719019cf80 0x7f719019d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:46.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.728+0000 7f7195bac640 1 --2- 172.21.15.97:0/4076044 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71901021d0 0x7f71901986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:46.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.729+0000 7f71953ab640 1 -- 172.21.15.97:0/4076044 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71901021d0 msgr2=0x7f71901986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:46.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.729+0000 7f71953ab640 1 --2- 172.21.15.97:0/4076044 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71901021d0 0x7f71901986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:46.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.729+0000 7f71953ab640 1 -- 172.21.15.97:0/4076044 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f719019cf80 msgr2=0x7f719019d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:46.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.729+0000 7f71953ab640 1 --2- 172.21.15.97:0/4076044 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f719019cf80 0x7f719019d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:46.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.729+0000 7f71953ab640 1 -- 172.21.15.97:0/4076044 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f719019da70 con 0x7f71901072a0 2024-04-23T10:41:46.734 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.729+0000 7f7195bac640 1 --2- 172.21.15.97:0/4076044 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71901021d0 0x7f71901986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:41:46.735 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.729+0000 7f71953ab640 1 --2- 172.21.15.97:0/4076044 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71901072a0 0x7f7190198c00 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f7180032fd0 tx=0x7f7180004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:41:46.735 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.729+0000 7f7186ffd640 1 -- 172.21.15.97:0/4076044 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f718003f030 con 0x7f71901072a0 2024-04-23T10:41:46.735 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.729+0000 7f7197e10640 1 -- 172.21.15.97:0/4076044 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f719019dd00 con 0x7f71901072a0 2024-04-23T10:41:46.735 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.729+0000 7f7186ffd640 1 -- 172.21.15.97:0/4076044 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7180004490 con 0x7f71901072a0 2024-04-23T10:41:46.735 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.729+0000 7f7186ffd640 1 -- 172.21.15.97:0/4076044 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7180035070 con 0x7f71901072a0 2024-04-23T10:41:46.735 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.729+0000 7f7197e10640 1 -- 172.21.15.97:0/4076044 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71901a54d0 con 0x7f71901072a0 2024-04-23T10:41:46.735 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.731+0000 7f7186ffd640 1 -- 172.21.15.97:0/4076044 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7180042030 con 0x7f71901072a0 2024-04-23T10:41:46.735 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.732+0000 7f719ac02740 1 Processor -- start 2024-04-23T10:41:46.735 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.732+0000 7f719ac02740 1 -- start start 2024-04-23T10:41:46.743 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.732+0000 7f719ac02740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b9c308500 0x557b9c3088e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:46.744 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.732+0000 7f719ac02740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b9c308e20 0x557b9c3176d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:46.744 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.732+0000 7f719ac02740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b9c317c10 0x557b9c31a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:46.744 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.732+0000 7f719ac02740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x557b9c26d970 con 0x557b9c308e20 2024-04-23T10:41:46.744 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.732+0000 7f719ac02740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557b9c20a4a0 con 0x557b9c308500 2024-04-23T10:41:46.744 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.732+0000 7f719ac02740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557b9c269930 con 0x557b9c317c10 2024-04-23T10:41:46.744 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.732+0000 7f7186ffd640 1 --2- 172.21.15.97:0/4076044 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f717c0a3dc0 0x7f717c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:46.744 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.732+0000 7f7195bac640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b9c308e20 0x557b9c3176d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:46.744 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.732+0000 7f7195bac640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b9c308e20 0x557b9c3176d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34688/0 (socket says 172.21.15.97:34688) 2024-04-23T10:41:46.744 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.732+0000 7f71963ad640 1 --2- 172.21.15.97:0/4076044 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f717c0a3dc0 0x7f717c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:46.745 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.732+0000 7f7195bac640 1 -- 172.21.15.97:0/1126960666 learned_addr learned my addr 172.21.15.97:0/1126960666 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:41:46.745 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.732+0000 7f7186ffd640 1 -- 172.21.15.97:0/4076044 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8484+0+0 (secure 0 0 0) 0x7f71800f0950 con 0x7f71901072a0 2024-04-23T10:41:46.745 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.733+0000 7f71953ab640 1 --2- 172.21.15.97:0/1126960666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b9c317c10 0x557b9c31a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:46.745 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.733+0000 7f71963ad640 1 --2- 172.21.15.97:0/1126960666 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b9c308500 0x557b9c3088e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:46.745 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.733+0000 7f7195bac640 1 -- 172.21.15.97:0/1126960666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b9c317c10 msgr2=0x557b9c31a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:46.745 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.733+0000 7f7195bac640 1 --2- 172.21.15.97:0/1126960666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b9c317c10 0x557b9c31a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:46.745 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.733+0000 7f7195bac640 1 -- 172.21.15.97:0/1126960666 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b9c308500 msgr2=0x557b9c3088e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:46.745 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.733+0000 7f7195bac640 1 --2- 172.21.15.97:0/1126960666 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b9c308500 0x557b9c3088e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:46.745 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.733+0000 7f7195bac640 1 -- 172.21.15.97:0/1126960666 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b9bf80210 con 0x557b9c308e20 2024-04-23T10:41:46.745 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.733+0000 7f71963ad640 1 --2- 172.21.15.97:0/4076044 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f717c0a3dc0 0x7f717c0a6280 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7188005fd0 tx=0x7f7188012a50 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:41:46.746 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.733+0000 7f7195bac640 1 --2- 172.21.15.97:0/1126960666 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b9c308e20 0x557b9c3176d0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f718c009c80 tx=0x7f718c00e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39678d6e154ee9fa server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:41:46.746 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.733+0000 7f717bfff640 1 -- 172.21.15.97:0/1126960666 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f718c016070 con 0x557b9c308e20 2024-04-23T10:41:46.746 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.733+0000 7f717bfff640 1 -- 172.21.15.97:0/1126960666 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f718c0054d0 con 0x557b9c308e20 2024-04-23T10:41:46.746 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.733+0000 7f717bfff640 1 -- 172.21.15.97:0/1126960666 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f718c011470 con 0x557b9c308e20 2024-04-23T10:41:46.746 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.734+0000 7f719ac02740 1 -- 172.21.15.97:0/1126960666 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b9c308e20 msgr2=0x557b9c3176d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:46.746 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.734+0000 7f719ac02740 1 --2- 172.21.15.97:0/1126960666 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b9c308e20 0x557b9c3176d0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f718c009c80 tx=0x7f718c00e6c0 comp rx=0 tx=0).stop 2024-04-23T10:41:46.746 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.734+0000 7f719ac02740 1 -- 172.21.15.97:0/1126960666 shutdown_connections 2024-04-23T10:41:46.746 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.734+0000 7f719ac02740 1 --2- 172.21.15.97:0/1126960666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b9c317c10 0x557b9c31a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:46.746 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.734+0000 7f719ac02740 1 --2- 172.21.15.97:0/1126960666 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b9c308e20 0x557b9c3176d0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:46.746 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.734+0000 7f719ac02740 1 --2- 172.21.15.97:0/1126960666 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b9c308500 0x557b9c3088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:46.747 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.734+0000 7f719ac02740 1 -- 172.21.15.97:0/1126960666 >> 172.21.15.97:0/1126960666 conn(0x557b9c304b30 msgr2=0x557b9c304f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:41:46.747 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.734+0000 7f719ac02740 1 -- 172.21.15.97:0/1126960666 shutdown_connections 2024-04-23T10:41:46.747 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.734+0000 7f719ac02740 1 -- 172.21.15.97:0/1126960666 wait complete. 2024-04-23T10:41:46.747 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.734+0000 7f719ac02740 1 -- start start 2024-04-23T10:41:46.747 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.734+0000 7f719ac02740 1 Processor -- start 2024-04-23T10:41:46.747 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.735+0000 7f719ac02740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b9c308500 0x557b9c3174d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:46.747 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.735+0000 7f719ac02740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b9c308e20 0x557b9c3147c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:46.747 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.735+0000 7f719ac02740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b9c317c10 0x557b9c314d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:46.747 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.735+0000 7f719ac02740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x557b9c31a840 con 0x557b9c308e20 2024-04-23T10:41:46.747 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.735+0000 7f719ac02740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557b9c2080b0 con 0x557b9c308500 2024-04-23T10:41:46.748 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.735+0000 7f719ac02740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x557b9c31a9c0 con 0x557b9c317c10 2024-04-23T10:41:46.748 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.735+0000 7f719ac02740 10 client.0 _ll_register_callbacks cb 0x557b9c1f5d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:41:46.748 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.735+0000 7f71953ab640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b9c308e20 0x557b9c3147c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:46.748 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.735+0000 7f71953ab640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b9c308e20 0x557b9c3147c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34692/0 (socket says 172.21.15.97:34692) 2024-04-23T10:41:46.748 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.735+0000 7f71953ab640 1 -- 172.21.15.97:0/3436891454 learned_addr learned my addr 172.21.15.97:0/3436891454 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:41:46.748 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.735+0000 7f71963ad640 1 --2- 172.21.15.97:0/3436891454 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b9c317c10 0x557b9c314d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:46.748 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.735+0000 7f7195bac640 1 --2- 172.21.15.97:0/3436891454 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b9c308500 0x557b9c3174d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:46.748 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.735+0000 7f71953ab640 1 -- 172.21.15.97:0/3436891454 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b9c317c10 msgr2=0x557b9c314d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:46.748 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.735+0000 7f71953ab640 1 --2- 172.21.15.97:0/3436891454 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b9c317c10 0x557b9c314d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:46.748 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.735+0000 7f71953ab640 1 -- 172.21.15.97:0/3436891454 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b9c308500 msgr2=0x557b9c3174d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:46.749 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.735+0000 7f71953ab640 1 --2- 172.21.15.97:0/3436891454 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b9c308500 0x557b9c3174d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:46.749 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.735+0000 7f71953ab640 1 -- 172.21.15.97:0/3436891454 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b9c04c620 con 0x557b9c308e20 2024-04-23T10:41:46.749 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.735+0000 7f71963ad640 1 --2- 172.21.15.97:0/3436891454 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b9c317c10 0x557b9c314d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:41:46.749 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.735+0000 7f71953ab640 1 --2- 172.21.15.97:0/3436891454 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b9c308e20 0x557b9c3147c0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f71800b8d70 tx=0x7f718002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6fd25e24f9251ce server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:41:46.749 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.735+0000 7f71797fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:41:46.749 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.735+0000 7f71797fa640 1 -- 172.21.15.97:0/3436891454 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7180049070 con 0x557b9c308e20 2024-04-23T10:41:46.749 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.735+0000 7f71797fa640 1 -- 172.21.15.97:0/3436891454 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8484+0+0 (secure 0 0 0) 0x7f71800b8f80 con 0x557b9c308e20 2024-04-23T10:41:46.749 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.735+0000 7f719ac02740 1 -- 172.21.15.97:0/3436891454 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f718c0054d0 con 0x557b9c308e20 2024-04-23T10:41:46.749 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.736+0000 7f71797fa640 1 -- 172.21.15.97:0/3436891454 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f71800bc910 con 0x557b9c308e20 2024-04-23T10:41:46.749 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.736+0000 7f719ac02740 10 client.17751 fetch_fsmap learned FSMap version 26 2024-04-23T10:41:46.749 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.736+0000 7f719ac02740 1 -- 172.21.15.97:0/3436891454 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557b9c326750 con 0x557b9c308e20 2024-04-23T10:41:46.750 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.736+0000 7f71797fa640 1 -- 172.21.15.97:0/3436891454 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f718c0054d0 con 0x557b9c308e20 2024-04-23T10:41:46.750 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.736+0000 7f71797fa640 1 -- 172.21.15.97:0/3436891454 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f718003bc20 con 0x557b9c308e20 2024-04-23T10:41:46.750 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.736+0000 7f719ac02740 10 client.17751 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:41:46.750 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.736+0000 7f719ac02740 10 client.17751 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:41:46.750 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.736+0000 7f719ac02740 20 client.17751 populate_metadata read hostname 'smithi097' 2024-04-23T10:41:46.750 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.736+0000 7f719ac02740 1 --2- 172.21.15.97:0/3436891454 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x557b9c327b60 0x557b9c329fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:46.750 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.736+0000 7f719ac02740 4 client.17751 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:41:46.750 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.736+0000 7f719ac02740 1 -- 172.21.15.97:0/3436891454 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b9c27b390 con 0x557b9c327b60 2024-04-23T10:41:46.750 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.736+0000 7f7195bac640 1 --2- 172.21.15.97:0/3436891454 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x557b9c327b60 0x557b9c329fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:46.750 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.737+0000 7f71797fa640 10 client.17751 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:41:46.751 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.737+0000 7f7195bac640 1 --2- 172.21.15.97:0/3436891454 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x557b9c327b60 0x557b9c329fc0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e18680b404c10980 server_cookie=66cf16c9b08971b in_seq=0 out_seq=0 2024-04-23T10:41:46.751 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.737+0000 7f71797fa640 1 -- 172.21.15.97:0/3436891454 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557b9c27b390 con 0x557b9c327b60 2024-04-23T10:41:46.751 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.737+0000 7f71797fa640 10 client.17751 handle_command_reply: tid=0 2024-04-23T10:41:46.751 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.738+0000 7f719ac02740 1 client.17751 shutdown 2024-04-23T10:41:46.751 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.738+0000 7f719ac02740 1 -- 172.21.15.97:0/3436891454 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b9c308e20 msgr2=0x557b9c3147c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:46.751 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.738+0000 7f719ac02740 1 --2- 172.21.15.97:0/3436891454 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b9c308e20 0x557b9c3147c0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f71800b8d70 tx=0x7f718002a410 comp rx=0 tx=0).stop 2024-04-23T10:41:46.751 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.738+0000 7f719ac02740 1 -- 172.21.15.97:0/3436891454 shutdown_connections 2024-04-23T10:41:46.751 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.738+0000 7f719ac02740 1 --2- 172.21.15.97:0/3436891454 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x557b9c327b60 0x557b9c329fc0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:46.751 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.738+0000 7f719ac02740 1 --2- 172.21.15.97:0/3436891454 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b9c317c10 0x557b9c314d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:46.751 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.738+0000 7f719ac02740 1 --2- 172.21.15.97:0/3436891454 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b9c308e20 0x557b9c3147c0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.738+0000 7f71797fa640 0 client.17751 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:41:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.738+0000 7f719ac02740 1 --2- 172.21.15.97:0/3436891454 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b9c308500 0x557b9c3174d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.738+0000 7f719ac02740 1 -- 172.21.15.97:0/3436891454 >> 172.21.15.97:0/3436891454 conn(0x557b9c304b30 msgr2=0x557b9c30a940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:41:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.738+0000 7f719ac02740 1 -- 172.21.15.97:0/3436891454 shutdown_connections 2024-04-23T10:41:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.738+0000 7f719ac02740 1 -- 172.21.15.97:0/3436891454 wait complete. 2024-04-23T10:41:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.738+0000 7f719ac02740 20 client.17751 trim_cache size 0 max 16384 2024-04-23T10:41:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.753+0000 7f719ac02740 1 Processor -- start 2024-04-23T10:41:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.753+0000 7f719ac02740 1 -- start start 2024-04-23T10:41:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.753+0000 7f719ac02740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b9c308500 0x557b9c30a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.753+0000 7f719ac02740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b9c308e20 0x557b9c317000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.753+0000 7f719ac02740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b9c317c10 0x557b9c32f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.753+0000 7f719ac02740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f71901ab880 con 0x557b9c317c10 2024-04-23T10:41:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.753+0000 7f719ac02740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f71901ad3c0 con 0x557b9c308500 2024-04-23T10:41:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.753+0000 7f719ac02740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f71901abf50 con 0x557b9c308e20 2024-04-23T10:41:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.753+0000 7f71963ad640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b9c317c10 0x557b9c32f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.753+0000 7f71963ad640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b9c317c10 0x557b9c32f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34696/0 (socket says 172.21.15.97:34696) 2024-04-23T10:41:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.753+0000 7f71963ad640 1 -- 172.21.15.97:0/4094792665 learned_addr learned my addr 172.21.15.97:0/4094792665 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:41:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.753+0000 7f71953ab640 1 --2- 172.21.15.97:0/4094792665 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b9c308e20 0x557b9c317000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.753+0000 7f71963ad640 1 -- 172.21.15.97:0/4094792665 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b9c308e20 msgr2=0x557b9c317000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.753+0000 7f71963ad640 1 --2- 172.21.15.97:0/4094792665 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b9c308e20 0x557b9c317000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.753+0000 7f7195bac640 1 --2- 172.21.15.97:0/4094792665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b9c308500 0x557b9c30a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.753+0000 7f71963ad640 1 -- 172.21.15.97:0/4094792665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b9c308500 msgr2=0x557b9c30a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.753+0000 7f71963ad640 1 --2- 172.21.15.97:0/4094792665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b9c308500 0x557b9c30a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.754+0000 7f71963ad640 1 -- 172.21.15.97:0/4094792665 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b9c304960 con 0x557b9c317c10 2024-04-23T10:41:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.754+0000 7f71963ad640 1 --2- 172.21.15.97:0/4094792665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b9c317c10 0x557b9c32f8d0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f71880109f0 tx=0x7f7188004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17d15fca868aaa1e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:41:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.754+0000 7f717b7fe640 1 -- 172.21.15.97:0/4094792665 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7188024070 con 0x557b9c317c10 2024-04-23T10:41:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.754+0000 7f717b7fe640 1 -- 172.21.15.97:0/4094792665 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f718801cbe0 con 0x557b9c317c10 2024-04-23T10:41:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.754+0000 7f717b7fe640 1 -- 172.21.15.97:0/4094792665 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7188020400 con 0x557b9c317c10 2024-04-23T10:41:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.754+0000 7f719ac02740 1 -- 172.21.15.97:0/4094792665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b9c317c10 msgr2=0x557b9c32f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.754+0000 7f719ac02740 1 --2- 172.21.15.97:0/4094792665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b9c317c10 0x557b9c32f8d0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f71880109f0 tx=0x7f7188004ab0 comp rx=0 tx=0).stop 2024-04-23T10:41:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.754+0000 7f719ac02740 1 -- 172.21.15.97:0/4094792665 shutdown_connections 2024-04-23T10:41:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.754+0000 7f719ac02740 1 --2- 172.21.15.97:0/4094792665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b9c317c10 0x557b9c32f8d0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.754+0000 7f719ac02740 1 --2- 172.21.15.97:0/4094792665 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b9c308e20 0x557b9c317000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.754+0000 7f719ac02740 1 --2- 172.21.15.97:0/4094792665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b9c308500 0x557b9c30a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.754+0000 7f719ac02740 1 -- 172.21.15.97:0/4094792665 >> 172.21.15.97:0/4094792665 conn(0x557b9c304b30 msgr2=0x557b9c30a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:41:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.754+0000 7f719ac02740 1 -- 172.21.15.97:0/4094792665 shutdown_connections 2024-04-23T10:41:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.754+0000 7f719ac02740 1 -- 172.21.15.97:0/4094792665 wait complete. 2024-04-23T10:41:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.755+0000 7f719ac02740 1 -- start start 2024-04-23T10:41:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.755+0000 7f719ac02740 1 Processor -- start 2024-04-23T10:41:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.755+0000 7f719ac02740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b9c308500 0x557b9c30d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.755+0000 7f719ac02740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b9c308e20 0x557b9c30d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.755+0000 7f719ac02740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b9c317c10 0x557b9c31ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.755+0000 7f719ac02740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f71901accf0 con 0x557b9c308500 2024-04-23T10:41:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.755+0000 7f719ac02740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f71901ac620 con 0x557b9c308e20 2024-04-23T10:41:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.755+0000 7f719ac02740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f71901a9880 con 0x557b9c317c10 2024-04-23T10:41:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.755+0000 7f719ac02740 10 client.0 _ll_register_callbacks cb 0x7f71800b8c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:41:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.755+0000 7f7195bac640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b9c308500 0x557b9c30d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.755+0000 7f7195bac640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b9c308500 0x557b9c30d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:34712/0 (socket says 172.21.15.97:34712) 2024-04-23T10:41:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.755+0000 7f7195bac640 1 -- 172.21.15.97:0/1104788552 learned_addr learned my addr 172.21.15.97:0/1104788552 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:41:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.756+0000 7f71953ab640 1 --2- 172.21.15.97:0/1104788552 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b9c308e20 0x557b9c30d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.756+0000 7f71963ad640 1 --2- 172.21.15.97:0/1104788552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b9c317c10 0x557b9c31ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.756+0000 7f7195bac640 1 -- 172.21.15.97:0/1104788552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b9c317c10 msgr2=0x557b9c31ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.756+0000 7f7195bac640 1 --2- 172.21.15.97:0/1104788552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b9c317c10 0x557b9c31ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.756+0000 7f7195bac640 1 -- 172.21.15.97:0/1104788552 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b9c308e20 msgr2=0x557b9c30d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.756+0000 7f7195bac640 1 --2- 172.21.15.97:0/1104788552 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b9c308e20 0x557b9c30d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.756+0000 7f7195bac640 1 -- 172.21.15.97:0/1104788552 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b9c31f200 con 0x557b9c308500 2024-04-23T10:41:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.756+0000 7f71953ab640 1 --2- 172.21.15.97:0/1104788552 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b9c308e20 0x557b9c30d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:41:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.756+0000 7f7195bac640 1 --2- 172.21.15.97:0/1104788552 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b9c308500 0x557b9c30d3b0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f718c005e90 tx=0x7f718c0058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a6d07d8ae8a158d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:41:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.756+0000 7f71797fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:41:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.756+0000 7f71797fa640 1 -- 172.21.15.97:0/1104788552 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f718c02e060 con 0x557b9c308500 2024-04-23T10:41:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.756+0000 7f71797fa640 1 -- 172.21.15.97:0/1104788552 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8484+0+0 (secure 0 0 0) 0x7f718c029110 con 0x557b9c308500 2024-04-23T10:41:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.756+0000 7f719ac02740 1 -- 172.21.15.97:0/1104788552 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f718801cbe0 con 0x557b9c308500 2024-04-23T10:41:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.756+0000 7f71797fa640 1 -- 172.21.15.97:0/1104788552 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f718c016040 con 0x557b9c308500 2024-04-23T10:41:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.756+0000 7f719ac02740 10 client.17757 fetch_fsmap learned FSMap version 26 2024-04-23T10:41:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.756+0000 7f719ac02740 1 -- 172.21.15.97:0/1104788552 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557b9c320530 con 0x557b9c308500 2024-04-23T10:41:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.756+0000 7f71797fa640 1 -- 172.21.15.97:0/1104788552 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f718801cbe0 con 0x557b9c308500 2024-04-23T10:41:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.756+0000 7f71797fa640 1 -- 172.21.15.97:0/1104788552 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f718c00f040 con 0x557b9c308500 2024-04-23T10:41:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.757+0000 7f719ac02740 10 client.17757 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:41:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.757+0000 7f719ac02740 10 client.17757 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:41:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.757+0000 7f719ac02740 20 client.17757 populate_metadata read hostname 'smithi097' 2024-04-23T10:41:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.757+0000 7f719ac02740 1 --2- 172.21.15.97:0/1104788552 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x557b9c392d80 0x557b9c3951a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.757+0000 7f719ac02740 4 client.17757 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:41:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.757+0000 7f719ac02740 1 -- 172.21.15.97:0/1104788552 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557b9c315090 con 0x557b9c392d80 2024-04-23T10:41:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.757+0000 7f71953ab640 1 --2- 172.21.15.97:0/1104788552 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x557b9c392d80 0x557b9c3951a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.757+0000 7f71953ab640 1 --2- 172.21.15.97:0/1104788552 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x557b9c392d80 0x557b9c3951a0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7a323c7907b3e51 server_cookie=9e53cee027191c58 in_seq=0 out_seq=0 2024-04-23T10:41:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.757+0000 7f71797fa640 10 client.17757 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:41:46.770 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.770+0000 7f71797fa640 1 -- 172.21.15.97:0/1104788552 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557b9c315090 con 0x557b9c392d80 2024-04-23T10:41:46.770 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.770+0000 7f71797fa640 10 client.17757 handle_command_reply: tid=0 2024-04-23T10:41:46.770 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.770+0000 7f719ac02740 1 client.17757 shutdown 2024-04-23T10:41:46.770 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.770+0000 7f719ac02740 1 -- 172.21.15.97:0/1104788552 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b9c308500 msgr2=0x557b9c30d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:46.770 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.770+0000 7f719ac02740 1 --2- 172.21.15.97:0/1104788552 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b9c308500 0x557b9c30d3b0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f718c005e90 tx=0x7f718c0058e0 comp rx=0 tx=0).stop 2024-04-23T10:41:46.770 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.770+0000 7f719ac02740 1 -- 172.21.15.97:0/1104788552 shutdown_connections 2024-04-23T10:41:46.770 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.770+0000 7f719ac02740 1 --2- 172.21.15.97:0/1104788552 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x557b9c392d80 0x557b9c3951a0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:46.770 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.770+0000 7f719ac02740 1 --2- 172.21.15.97:0/1104788552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x557b9c317c10 0x557b9c31ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:46.770 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.770+0000 7f719ac02740 1 --2- 172.21.15.97:0/1104788552 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557b9c308e20 0x557b9c30d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:46.771 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.770+0000 7f719ac02740 1 --2- 172.21.15.97:0/1104788552 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557b9c308500 0x557b9c30d3b0 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:46.771 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.770+0000 7f719ac02740 1 -- 172.21.15.97:0/1104788552 >> 172.21.15.97:0/1104788552 conn(0x557b9c304b30 msgr2=0x557b9c303cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:41:46.771 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.770+0000 7f71797fa640 0 client.17757 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:41:46.771 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.770+0000 7f719ac02740 1 -- 172.21.15.97:0/1104788552 shutdown_connections 2024-04-23T10:41:46.771 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.770+0000 7f719ac02740 1 -- 172.21.15.97:0/1104788552 wait complete. 2024-04-23T10:41:46.771 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.770+0000 7f719ac02740 20 client.17757 trim_cache size 0 max 16384 2024-04-23T10:41:46.771 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:41:46.771 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:41:46.772 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:41:46.772 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:41:46.773 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.771+0000 7f7197e10640 1 -- 172.21.15.97:0/4076044 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f717c0a3dc0 msgr2=0x7f717c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:46.773 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.771+0000 7f7197e10640 1 --2- 172.21.15.97:0/4076044 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f717c0a3dc0 0x7f717c0a6280 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7188005fd0 tx=0x7f7188012a50 comp rx=0 tx=0).stop 2024-04-23T10:41:46.773 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.771+0000 7f7197e10640 1 -- 172.21.15.97:0/4076044 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71901072a0 msgr2=0x7f7190198c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:46.773 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.771+0000 7f7197e10640 1 --2- 172.21.15.97:0/4076044 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71901072a0 0x7f7190198c00 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f7180032fd0 tx=0x7f7180004270 comp rx=0 tx=0).stop 2024-04-23T10:41:46.773 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.771+0000 7f7197e10640 1 -- 172.21.15.97:0/4076044 shutdown_connections 2024-04-23T10:41:46.773 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.771+0000 7f7197e10640 1 --2- 172.21.15.97:0/4076044 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f717c0a3dc0 0x7f717c0a6280 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:46.773 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.771+0000 7f7197e10640 1 --2- 172.21.15.97:0/4076044 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f719019cf80 0x7f719019d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:46.773 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.771+0000 7f7197e10640 1 --2- 172.21.15.97:0/4076044 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71901072a0 0x7f7190198c00 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:46.773 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.771+0000 7f7197e10640 1 --2- 172.21.15.97:0/4076044 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71901021d0 0x7f71901986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:46.773 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.771+0000 7f7197e10640 1 -- 172.21.15.97:0/4076044 >> 172.21.15.97:0/4076044 conn(0x7f71900793e0 msgr2=0x7f71900ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:41:46.773 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.771+0000 7f7197e10640 1 -- 172.21.15.97:0/4076044 shutdown_connections 2024-04-23T10:41:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:46.771+0000 7f7197e10640 1 -- 172.21.15.97:0/4076044 wait complete. 2024-04-23T10:41:46.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:41:46 smithi196 ceph-mon[32919]: pgmap v488: 97 pgs: 97 active+clean; 3.1 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.7 KiB/s wr, 10 op/s 2024-04-23T10:41:46.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:41:46 smithi097 ceph-mon[29197]: pgmap v488: 97 pgs: 97 active+clean; 3.1 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.7 KiB/s wr, 10 op/s 2024-04-23T10:41:46.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:41:46 smithi183 ceph-mon[32317]: pgmap v488: 97 pgs: 97 active+clean; 3.1 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.7 KiB/s wr, 10 op/s 2024-04-23T10:41:48.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:41:48 smithi196 ceph-mon[32919]: pgmap v489: 97 pgs: 97 active+clean; 3.2 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 17 op/s 2024-04-23T10:41:48.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:41:48 smithi097 ceph-mon[29197]: pgmap v489: 97 pgs: 97 active+clean; 3.2 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 17 op/s 2024-04-23T10:41:48.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:41:48 smithi183 ceph-mon[32317]: pgmap v489: 97 pgs: 97 active+clean; 3.2 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 17 op/s 2024-04-23T10:41:50.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:41:50 smithi196 ceph-mon[32919]: pgmap v490: 97 pgs: 97 active+clean; 3.2 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-04-23T10:41:50.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:41:50 smithi097 ceph-mon[29197]: pgmap v490: 97 pgs: 97 active+clean; 3.2 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-04-23T10:41:50.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:41:50 smithi183 ceph-mon[32317]: pgmap v490: 97 pgs: 97 active+clean; 3.2 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-04-23T10:41:52.843 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:41:52 smithi196 ceph-mon[32919]: pgmap v491: 97 pgs: 97 active+clean; 3.2 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-04-23T10:41:52.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:41:52 smithi097 ceph-mon[29197]: pgmap v491: 97 pgs: 97 active+clean; 3.2 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-04-23T10:41:52.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:41:52 smithi183 ceph-mon[32317]: pgmap v491: 97 pgs: 97 active+clean; 3.2 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-04-23T10:41:54.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:41:54 smithi196 ceph-mon[32919]: pgmap v492: 97 pgs: 97 active+clean; 3.2 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-04-23T10:41:54.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:41:54 smithi097 ceph-mon[29197]: pgmap v492: 97 pgs: 97 active+clean; 3.2 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-04-23T10:41:54.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:41:54 smithi183 ceph-mon[32317]: pgmap v492: 97 pgs: 97 active+clean; 3.2 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-04-23T10:41:56.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:41:56 smithi196 ceph-mon[32919]: pgmap v493: 97 pgs: 97 active+clean; 3.2 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-04-23T10:41:56.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:41:56 smithi097 ceph-mon[29197]: pgmap v493: 97 pgs: 97 active+clean; 3.2 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-04-23T10:41:56.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:41:56 smithi183 ceph-mon[32317]: pgmap v493: 97 pgs: 97 active+clean; 3.2 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-04-23T10:41:58.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:41:58 smithi196 ceph-mon[32919]: pgmap v494: 97 pgs: 97 active+clean; 3.2 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-04-23T10:41:58.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:41:58 smithi097 ceph-mon[29197]: pgmap v494: 97 pgs: 97 active+clean; 3.2 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-04-23T10:41:58.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:41:58 smithi183 ceph-mon[32317]: pgmap v494: 97 pgs: 97 active+clean; 3.2 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 15 op/s 2024-04-23T10:41:59.868 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.867+0000 7f4f6ba4e640 1 -- 172.21.15.97:0/3266474856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f64107290 msgr2=0x7f4f6410ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:59.869 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.867+0000 7f4f6ba4e640 1 --2- 172.21.15.97:0/3266474856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f64107290 0x7f4f6410ee60 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c009ca0 tx=0x7f4f5c02a8c0 comp rx=0 tx=0).stop 2024-04-23T10:41:59.869 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.867+0000 7f4f6ba4e640 1 -- 172.21.15.97:0/3266474856 shutdown_connections 2024-04-23T10:41:59.869 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.867+0000 7f4f6ba4e640 1 --2- 172.21.15.97:0/3266474856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f64107290 0x7f4f6410ee60 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:59.869 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.867+0000 7f4f6ba4e640 1 --2- 172.21.15.97:0/3266474856 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f64102b70 0x7f4f64106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:59.869 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.867+0000 7f4f6ba4e640 1 --2- 172.21.15.97:0/3266474856 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f641021c0 0x7f4f641025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:59.869 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.867+0000 7f4f6ba4e640 1 -- 172.21.15.97:0/3266474856 >> 172.21.15.97:0/3266474856 conn(0x7f4f640fda20 msgr2=0x7f4f640ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:41:59.869 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.867+0000 7f4f6ba4e640 1 -- 172.21.15.97:0/3266474856 shutdown_connections 2024-04-23T10:41:59.869 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.867+0000 7f4f6ba4e640 1 -- 172.21.15.97:0/3266474856 wait complete. 2024-04-23T10:41:59.869 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.868+0000 7f4f6ba4e640 1 Processor -- start 2024-04-23T10:41:59.870 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.868+0000 7f4f6ba4e640 1 -- start start 2024-04-23T10:41:59.870 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.868+0000 7f4f6ba4e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f641021c0 0x7f4f64198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:59.870 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.868+0000 7f4f6ba4e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f64107290 0x7f4f64198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:59.870 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.868+0000 7f4f6ba4e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f6419cfb0 0x7f4f6419d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:59.870 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.868+0000 7f4f6ba4e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4f64112200 con 0x7f4f64107290 2024-04-23T10:41:59.870 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.868+0000 7f4f6ba4e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4f64112080 con 0x7f4f6419cfb0 2024-04-23T10:41:59.870 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.868+0000 7f4f6ba4e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4f64112380 con 0x7f4f641021c0 2024-04-23T10:41:59.870 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.868+0000 7f4f68fe9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f64107290 0x7f4f64198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:59.870 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.868+0000 7f4f68fe9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f64107290 0x7f4f64198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60678/0 (socket says 172.21.15.97:60678) 2024-04-23T10:41:59.870 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.868+0000 7f4f68fe9640 1 -- 172.21.15.97:0/3240835351 learned_addr learned my addr 172.21.15.97:0/3240835351 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:41:59.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.869+0000 7f4f697ea640 1 --2- 172.21.15.97:0/3240835351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f641021c0 0x7f4f64198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:59.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.869+0000 7f4f69feb640 1 --2- 172.21.15.97:0/3240835351 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f6419cfb0 0x7f4f6419d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:59.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.869+0000 7f4f68fe9640 1 -- 172.21.15.97:0/3240835351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f641021c0 msgr2=0x7f4f64198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:59.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.869+0000 7f4f68fe9640 1 --2- 172.21.15.97:0/3240835351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f641021c0 0x7f4f64198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:59.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.869+0000 7f4f68fe9640 1 -- 172.21.15.97:0/3240835351 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f6419cfb0 msgr2=0x7f4f6419d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:59.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.869+0000 7f4f68fe9640 1 --2- 172.21.15.97:0/3240835351 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f6419cfb0 0x7f4f6419d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:59.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.869+0000 7f4f68fe9640 1 -- 172.21.15.97:0/3240835351 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f6419daa0 con 0x7f4f64107290 2024-04-23T10:41:59.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.869+0000 7f4f68fe9640 1 --2- 172.21.15.97:0/3240835351 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f64107290 0x7f4f64198c80 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f4f5400c970 tx=0x7f4f5400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:41:59.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.869+0000 7f4f5a7fc640 1 -- 172.21.15.97:0/3240835351 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f54007bb0 con 0x7f4f64107290 2024-04-23T10:41:59.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.869+0000 7f4f6ba4e640 1 -- 172.21.15.97:0/3240835351 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f6419dd90 con 0x7f4f64107290 2024-04-23T10:41:59.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.869+0000 7f4f5a7fc640 1 -- 172.21.15.97:0/3240835351 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f54007d50 con 0x7f4f64107290 2024-04-23T10:41:59.871 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.869+0000 7f4f5a7fc640 1 -- 172.21.15.97:0/3240835351 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f54018730 con 0x7f4f64107290 2024-04-23T10:41:59.872 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.869+0000 7f4f6ba4e640 1 -- 172.21.15.97:0/3240835351 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f641a5580 con 0x7f4f64107290 2024-04-23T10:41:59.879 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.871+0000 7f4f5a7fc640 1 -- 172.21.15.97:0/3240835351 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4f5400f990 con 0x7f4f64107290 2024-04-23T10:41:59.879 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.872+0000 7f4f6e764740 1 Processor -- start 2024-04-23T10:41:59.879 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.872+0000 7f4f6e764740 1 -- start start 2024-04-23T10:41:59.879 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.872+0000 7f4f6e764740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c437f6a620 0x55c437f6aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:59.879 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.872+0000 7f4f6e764740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c437f6af40 0x55c437f752c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:59.879 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.872+0000 7f4f5a7fc640 1 --2- 172.21.15.97:0/3240835351 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f4f500a3dc0 0x7f4f500a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:59.879 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.872+0000 7f4f6e764740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c437f75800 0x55c437f77bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:59.879 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.872+0000 7f4f6e764740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c437ecf970 con 0x55c437f75800 2024-04-23T10:41:59.879 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.872+0000 7f4f6e764740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c437e6c4a0 con 0x55c437f6af40 2024-04-23T10:41:59.880 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.872+0000 7f4f6e764740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c437ecb930 con 0x55c437f6a620 2024-04-23T10:41:59.880 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.872+0000 7f4f69feb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c437f75800 0x55c437f77bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:59.880 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.872+0000 7f4f5a7fc640 1 -- 172.21.15.97:0/3240835351 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8484+0+0 (secure 0 0 0) 0x7f4f540d0840 con 0x7f4f64107290 2024-04-23T10:41:59.880 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.872+0000 7f4f68fe9640 1 --2- 172.21.15.97:0/3240835351 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f4f500a3dc0 0x7f4f500a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:59.880 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.872+0000 7f4f69feb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c437f75800 0x55c437f77bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60682/0 (socket says 172.21.15.97:60682) 2024-04-23T10:41:59.880 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.872+0000 7f4f69feb640 1 -- 172.21.15.97:0/3569108442 learned_addr learned my addr 172.21.15.97:0/3569108442 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:41:59.880 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.872+0000 7f4f69feb640 1 --2- 172.21.15.97:0/3569108442 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c437f6a620 0x55c437f6aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:59.880 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.873+0000 7f4f697ea640 1 --2- 172.21.15.97:0/3569108442 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c437f6af40 0x55c437f752c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:59.880 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.873+0000 7f4f69feb640 1 -- 172.21.15.97:0/3569108442 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c437f6a620 msgr2=0x55c437f6aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:59.880 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.873+0000 7f4f69feb640 1 --2- 172.21.15.97:0/3569108442 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c437f6a620 0x55c437f6aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:59.880 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.873+0000 7f4f69feb640 1 -- 172.21.15.97:0/3569108442 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c437f6af40 msgr2=0x55c437f752c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:59.880 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.873+0000 7f4f69feb640 1 --2- 172.21.15.97:0/3569108442 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c437f6af40 0x55c437f752c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:59.881 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.873+0000 7f4f69feb640 1 -- 172.21.15.97:0/3569108442 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c437be2210 con 0x55c437f75800 2024-04-23T10:41:59.881 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.873+0000 7f4f68fe9640 1 --2- 172.21.15.97:0/3240835351 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f4f500a3dc0 0x7f4f500a6280 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4f540ce010 tx=0x7f4f54087040 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:41:59.881 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.873+0000 7f4f69feb640 1 --2- 172.21.15.97:0/3569108442 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c437f75800 0x55c437f77bf0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c009590 tx=0x7f4f5c02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a2528315250b858 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:41:59.881 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.873+0000 7f4f4f7fe640 1 -- 172.21.15.97:0/3569108442 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c032ea0 con 0x55c437f75800 2024-04-23T10:41:59.881 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.873+0000 7f4f4f7fe640 1 -- 172.21.15.97:0/3569108442 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f5c036050 con 0x55c437f75800 2024-04-23T10:41:59.881 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.873+0000 7f4f4f7fe640 1 -- 172.21.15.97:0/3569108442 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c004280 con 0x55c437f75800 2024-04-23T10:41:59.881 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.874+0000 7f4f6e764740 1 -- 172.21.15.97:0/3569108442 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c437f75800 msgr2=0x55c437f77bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:59.881 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.874+0000 7f4f6e764740 1 --2- 172.21.15.97:0/3569108442 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c437f75800 0x55c437f77bf0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c009590 tx=0x7f4f5c02a8c0 comp rx=0 tx=0).stop 2024-04-23T10:41:59.881 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.874+0000 7f4f6e764740 1 -- 172.21.15.97:0/3569108442 shutdown_connections 2024-04-23T10:41:59.881 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.874+0000 7f4f6e764740 1 --2- 172.21.15.97:0/3569108442 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c437f75800 0x55c437f77bf0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:59.881 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.874+0000 7f4f6e764740 1 --2- 172.21.15.97:0/3569108442 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c437f6af40 0x55c437f752c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:59.881 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.874+0000 7f4f6e764740 1 --2- 172.21.15.97:0/3569108442 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c437f6a620 0x55c437f6aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:59.882 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.874+0000 7f4f6e764740 1 -- 172.21.15.97:0/3569108442 >> 172.21.15.97:0/3569108442 conn(0x55c437f66b30 msgr2=0x55c437f66f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:41:59.882 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.874+0000 7f4f6e764740 1 -- 172.21.15.97:0/3569108442 shutdown_connections 2024-04-23T10:41:59.882 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.874+0000 7f4f6e764740 1 -- 172.21.15.97:0/3569108442 wait complete. 2024-04-23T10:41:59.882 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.874+0000 7f4f6e764740 1 -- start start 2024-04-23T10:41:59.882 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.874+0000 7f4f6e764740 1 Processor -- start 2024-04-23T10:41:59.882 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.875+0000 7f4f6e764740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c437f6a620 0x55c437f751c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:59.882 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.875+0000 7f4f6e764740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c437f6af40 0x55c437f724b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:59.882 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.875+0000 7f4f6e764740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c437f75800 0x55c437f72ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:59.882 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.875+0000 7f4f6e764740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c437f78250 con 0x55c437f6a620 2024-04-23T10:41:59.882 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.875+0000 7f4f6e764740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c437e6a0b0 con 0x55c437f6af40 2024-04-23T10:41:59.882 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.875+0000 7f4f6e764740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55c437f783d0 con 0x55c437f75800 2024-04-23T10:41:59.883 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.875+0000 7f4f6e764740 10 client.0 _ll_register_callbacks cb 0x55c437e57d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:41:59.883 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.875+0000 7f4f69feb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c437f6a620 0x55c437f751c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:59.883 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.875+0000 7f4f69feb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c437f6a620 0x55c437f751c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60690/0 (socket says 172.21.15.97:60690) 2024-04-23T10:41:59.883 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.875+0000 7f4f69feb640 1 -- 172.21.15.97:0/2050341191 learned_addr learned my addr 172.21.15.97:0/2050341191 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:41:59.883 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.875+0000 7f4f697ea640 1 --2- 172.21.15.97:0/2050341191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c437f6af40 0x55c437f724b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:59.883 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.875+0000 7f4f69feb640 1 -- 172.21.15.97:0/2050341191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c437f75800 msgr2=0x55c437f72ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:59.883 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.875+0000 7f4f69feb640 1 --2- 172.21.15.97:0/2050341191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c437f75800 0x55c437f72ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:59.883 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.875+0000 7f4f69feb640 1 -- 172.21.15.97:0/2050341191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c437f6af40 msgr2=0x55c437f724b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:59.883 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.875+0000 7f4f69feb640 1 --2- 172.21.15.97:0/2050341191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c437f6af40 0x55c437f724b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:59.883 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.875+0000 7f4f69feb640 1 -- 172.21.15.97:0/2050341191 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c437cae620 con 0x55c437f6a620 2024-04-23T10:41:59.883 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.875+0000 7f4f69feb640 1 --2- 172.21.15.97:0/2050341191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c437f6a620 0x55c437f751c0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c009ca0 tx=0x7f4f5c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d3a5beb2fea2f56 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:41:59.883 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.875+0000 7f4f4cff9640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:41:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.875+0000 7f4f4cff9640 1 -- 172.21.15.97:0/2050341191 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c04e070 con 0x55c437f6a620 2024-04-23T10:41:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.875+0000 7f4f4cff9640 1 -- 172.21.15.97:0/2050341191 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8484+0+0 (secure 0 0 0) 0x7f4f5c031660 con 0x55c437f6a620 2024-04-23T10:41:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.876+0000 7f4f6e764740 1 -- 172.21.15.97:0/2050341191 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f5c036050 con 0x55c437f6a620 2024-04-23T10:41:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.876+0000 7f4f4cff9640 1 -- 172.21.15.97:0/2050341191 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4f5c05a050 con 0x55c437f6a620 2024-04-23T10:41:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.876+0000 7f4f6e764740 10 client.17769 fetch_fsmap learned FSMap version 26 2024-04-23T10:41:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.876+0000 7f4f6e764740 1 -- 172.21.15.97:0/2050341191 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c437f78250 con 0x55c437f6a620 2024-04-23T10:41:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.876+0000 7f4f4cff9640 1 -- 172.21.15.97:0/2050341191 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f5c036050 con 0x55c437f6a620 2024-04-23T10:41:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.876+0000 7f4f4cff9640 1 -- 172.21.15.97:0/2050341191 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4f5c0305c0 con 0x55c437f6a620 2024-04-23T10:41:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.876+0000 7f4f6e764740 10 client.17769 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:41:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.876+0000 7f4f6e764740 10 client.17769 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:41:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.876+0000 7f4f6e764740 20 client.17769 populate_metadata read hostname 'smithi097' 2024-04-23T10:41:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.876+0000 7f4f6e764740 1 --2- 172.21.15.97:0/2050341191 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55c437f853a0 0x55c437f87790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.876+0000 7f4f6e764740 4 client.17769 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:41:59.885 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.876+0000 7f4f6e764740 1 -- 172.21.15.97:0/2050341191 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c437edd390 con 0x55c437f853a0 2024-04-23T10:41:59.885 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.876+0000 7f4f697ea640 1 --2- 172.21.15.97:0/2050341191 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55c437f853a0 0x55c437f87790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:59.885 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.877+0000 7f4f4cff9640 10 client.17769 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:41:59.885 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.877+0000 7f4f697ea640 1 --2- 172.21.15.97:0/2050341191 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55c437f853a0 0x55c437f87790 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=966f86318dea84a9 server_cookie=60a99c0bbbe2a938 in_seq=0 out_seq=0 2024-04-23T10:41:59.885 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.878+0000 7f4f4cff9640 1 -- 172.21.15.97:0/2050341191 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c437edd390 con 0x55c437f853a0 2024-04-23T10:41:59.885 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.878+0000 7f4f4cff9640 10 client.17769 handle_command_reply: tid=0 2024-04-23T10:41:59.885 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.878+0000 7f4f6e764740 1 client.17769 shutdown 2024-04-23T10:41:59.885 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.878+0000 7f4f6e764740 1 -- 172.21.15.97:0/2050341191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c437f6a620 msgr2=0x55c437f751c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:59.885 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.878+0000 7f4f6e764740 1 --2- 172.21.15.97:0/2050341191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c437f6a620 0x55c437f751c0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c009ca0 tx=0x7f4f5c003ba0 comp rx=0 tx=0).stop 2024-04-23T10:41:59.885 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.878+0000 7f4f6e764740 1 -- 172.21.15.97:0/2050341191 shutdown_connections 2024-04-23T10:41:59.885 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.878+0000 7f4f6e764740 1 --2- 172.21.15.97:0/2050341191 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55c437f853a0 0x55c437f87790 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:59.885 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.878+0000 7f4f6e764740 1 --2- 172.21.15.97:0/2050341191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c437f75800 0x55c437f72ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:59.886 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.878+0000 7f4f6e764740 1 --2- 172.21.15.97:0/2050341191 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c437f6af40 0x55c437f724b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:59.886 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.878+0000 7f4f4cff9640 0 client.17769 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:41:59.886 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.878+0000 7f4f6e764740 1 --2- 172.21.15.97:0/2050341191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c437f6a620 0x55c437f751c0 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:59.886 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.878+0000 7f4f6e764740 1 -- 172.21.15.97:0/2050341191 >> 172.21.15.97:0/2050341191 conn(0x55c437f66b30 msgr2=0x55c437f6c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:41:59.886 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.878+0000 7f4f6e764740 1 -- 172.21.15.97:0/2050341191 shutdown_connections 2024-04-23T10:41:59.886 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.878+0000 7f4f6e764740 1 -- 172.21.15.97:0/2050341191 wait complete. 2024-04-23T10:41:59.886 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.878+0000 7f4f6e764740 20 client.17769 trim_cache size 0 max 16384 2024-04-23T10:41:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.893+0000 7f4f6e764740 1 Processor -- start 2024-04-23T10:41:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.893+0000 7f4f6e764740 1 -- start start 2024-04-23T10:41:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.893+0000 7f4f6e764740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c437f6a620 0x55c437f81110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.893+0000 7f4f6e764740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c437f6af40 0x55c437f71630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.893+0000 7f4f6e764740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c437f75800 0x55c437f6c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.893+0000 7f4f6e764740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4f641ada70 con 0x55c437f6af40 2024-04-23T10:41:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.893+0000 7f4f6e764740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4f641af5b0 con 0x55c437f75800 2024-04-23T10:41:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.893+0000 7f4f6e764740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4f641ae140 con 0x55c437f6a620 2024-04-23T10:41:59.898 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.893+0000 7f4f697ea640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c437f6af40 0x55c437f71630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:59.898 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.893+0000 7f4f697ea640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c437f6af40 0x55c437f71630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60704/0 (socket says 172.21.15.97:60704) 2024-04-23T10:41:59.898 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.893+0000 7f4f697ea640 1 -- 172.21.15.97:0/2079810427 learned_addr learned my addr 172.21.15.97:0/2079810427 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:41:59.898 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.894+0000 7f4f69feb640 1 --2- 172.21.15.97:0/2079810427 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c437f6a620 0x55c437f81110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:59.898 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.894+0000 7f4f697ea640 1 -- 172.21.15.97:0/2079810427 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c437f6a620 msgr2=0x55c437f81110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:59.898 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.894+0000 7f4f697ea640 1 --2- 172.21.15.97:0/2079810427 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c437f6a620 0x55c437f81110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:59.898 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.894+0000 7f4f697ea640 1 -- 172.21.15.97:0/2079810427 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c437f75800 msgr2=0x55c437f6c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:59.898 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.894+0000 7f4f697ea640 1 --2- 172.21.15.97:0/2079810427 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c437f75800 0x55c437f6c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:59.898 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.894+0000 7f4f697ea640 1 -- 172.21.15.97:0/2079810427 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c437f70a20 con 0x55c437f6af40 2024-04-23T10:41:59.898 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.894+0000 7f4f697ea640 1 --2- 172.21.15.97:0/2079810427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c437f6af40 0x55c437f71630 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x55c437f6bbc0 tx=0x7f4f6000e3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efe822ffa306a493 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:41:59.898 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.894+0000 7f4f4effd640 1 -- 172.21.15.97:0/2079810427 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f6001e070 con 0x55c437f6af40 2024-04-23T10:41:59.898 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.894+0000 7f4f4effd640 1 -- 172.21.15.97:0/2079810427 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f6001adc0 con 0x55c437f6af40 2024-04-23T10:41:59.899 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.894+0000 7f4f4effd640 1 -- 172.21.15.97:0/2079810427 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f60010a70 con 0x55c437f6af40 2024-04-23T10:41:59.899 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.894+0000 7f4f6e764740 1 -- 172.21.15.97:0/2079810427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c437f6af40 msgr2=0x55c437f71630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:59.899 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.894+0000 7f4f6e764740 1 --2- 172.21.15.97:0/2079810427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c437f6af40 0x55c437f71630 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x55c437f6bbc0 tx=0x7f4f6000e3b0 comp rx=0 tx=0).stop 2024-04-23T10:41:59.899 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.895+0000 7f4f6e764740 1 -- 172.21.15.97:0/2079810427 shutdown_connections 2024-04-23T10:41:59.899 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.895+0000 7f4f6e764740 1 --2- 172.21.15.97:0/2079810427 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c437f75800 0x55c437f6c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:59.899 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.895+0000 7f4f6e764740 1 --2- 172.21.15.97:0/2079810427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c437f6af40 0x55c437f71630 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:59.899 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.895+0000 7f4f6e764740 1 --2- 172.21.15.97:0/2079810427 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c437f6a620 0x55c437f81110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:59.899 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.895+0000 7f4f6e764740 1 -- 172.21.15.97:0/2079810427 >> 172.21.15.97:0/2079810427 conn(0x55c437f66b30 msgr2=0x55c437f6bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:41:59.899 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.895+0000 7f4f6e764740 1 -- 172.21.15.97:0/2079810427 shutdown_connections 2024-04-23T10:41:59.899 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.895+0000 7f4f6e764740 1 -- 172.21.15.97:0/2079810427 wait complete. 2024-04-23T10:41:59.899 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.895+0000 7f4f6e764740 1 -- start start 2024-04-23T10:41:59.899 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.895+0000 7f4f6e764740 1 Processor -- start 2024-04-23T10:41:59.899 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.895+0000 7f4f6e764740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c437f6a620 0x55c437f7d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:59.900 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.895+0000 7f4f6e764740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c437f6af40 0x55c437f6ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:59.900 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.895+0000 7f4f6e764740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c437f75800 0x55c437f6f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:59.900 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.895+0000 7f4f6e764740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4f641aeee0 con 0x55c437f6af40 2024-04-23T10:41:59.900 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.895+0000 7f4f6e764740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4f641ae810 con 0x55c437f75800 2024-04-23T10:41:59.900 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.895+0000 7f4f6e764740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4f641aba70 con 0x55c437f6a620 2024-04-23T10:41:59.900 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.895+0000 7f4f6e764740 10 client.0 _ll_register_callbacks cb 0x7f4f5c02f310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:41:59.900 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.895+0000 7f4f697ea640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c437f6af40 0x55c437f6ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:59.900 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.895+0000 7f4f697ea640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c437f6af40 0x55c437f6ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60712/0 (socket says 172.21.15.97:60712) 2024-04-23T10:41:59.900 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.895+0000 7f4f697ea640 1 -- 172.21.15.97:0/2618749208 learned_addr learned my addr 172.21.15.97:0/2618749208 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:41:59.900 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.896+0000 7f4f69feb640 1 --2- 172.21.15.97:0/2618749208 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c437f6a620 0x55c437f7d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:59.900 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.896+0000 7f4f697ea640 1 -- 172.21.15.97:0/2618749208 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c437f6a620 msgr2=0x55c437f7d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:59.900 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.896+0000 7f4f697ea640 1 --2- 172.21.15.97:0/2618749208 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c437f6a620 0x55c437f7d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:59.900 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.896+0000 7f4f69feb640 1 --2- 172.21.15.97:0/2618749208 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c437f75800 0x55c437f6f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:59.901 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.896+0000 7f4f697ea640 1 -- 172.21.15.97:0/2618749208 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c437f75800 msgr2=0x55c437f6f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:41:59.901 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.896+0000 7f4f697ea640 1 --2- 172.21.15.97:0/2618749208 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c437f75800 0x55c437f6f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:59.901 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.896+0000 7f4f697ea640 1 -- 172.21.15.97:0/2618749208 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c437f6f8c0 con 0x55c437f6af40 2024-04-23T10:41:59.901 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.896+0000 7f4f697ea640 1 --2- 172.21.15.97:0/2618749208 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c437f6af40 0x55c437f6ed40 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f4f600107f0 tx=0x7f4f60010820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dd33e7fd80c434e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:41:59.901 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.896+0000 7f4f4cff9640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:41:59.901 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.896+0000 7f4f4cff9640 1 -- 172.21.15.97:0/2618749208 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f6001e050 con 0x55c437f6af40 2024-04-23T10:41:59.901 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.896+0000 7f4f4cff9640 1 -- 172.21.15.97:0/2618749208 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8484+0+0 (secure 0 0 0) 0x7f4f60010bc0 con 0x55c437f6af40 2024-04-23T10:41:59.901 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.896+0000 7f4f6e764740 1 -- 172.21.15.97:0/2618749208 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f6001adc0 con 0x55c437f6af40 2024-04-23T10:41:59.901 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.896+0000 7f4f4cff9640 1 -- 172.21.15.97:0/2618749208 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4f60010dc0 con 0x55c437f6af40 2024-04-23T10:41:59.901 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.896+0000 7f4f6e764740 10 client.17775 fetch_fsmap learned FSMap version 26 2024-04-23T10:41:59.901 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.896+0000 7f4f6e764740 1 -- 172.21.15.97:0/2618749208 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c437feb6c0 con 0x55c437f6af40 2024-04-23T10:41:59.901 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.896+0000 7f4f4cff9640 1 -- 172.21.15.97:0/2618749208 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f6001adc0 con 0x55c437f6af40 2024-04-23T10:41:59.902 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.897+0000 7f4f4cff9640 1 -- 172.21.15.97:0/2618749208 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4f60033070 con 0x55c437f6af40 2024-04-23T10:41:59.902 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.897+0000 7f4f6e764740 10 client.17775 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:41:59.902 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.897+0000 7f4f6e764740 10 client.17775 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:41:59.902 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.897+0000 7f4f6e764740 20 client.17775 populate_metadata read hostname 'smithi097' 2024-04-23T10:41:59.902 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.897+0000 7f4f6e764740 1 --2- 172.21.15.97:0/2618749208 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55c437febe60 0x55c437fee280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:41:59.902 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.897+0000 7f4f6e764740 4 client.17775 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:41:59.902 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.897+0000 7f4f6e764740 1 -- 172.21.15.97:0/2618749208 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c437ed7200 con 0x55c437febe60 2024-04-23T10:41:59.902 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.897+0000 7f4f69feb640 1 --2- 172.21.15.97:0/2618749208 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55c437febe60 0x55c437fee280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:41:59.902 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.897+0000 7f4f4cff9640 10 client.17775 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:41:59.902 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.897+0000 7f4f69feb640 1 --2- 172.21.15.97:0/2618749208 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55c437febe60 0x55c437fee280 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e701841e8d18c6c server_cookie=7baa99ea8c154461 in_seq=0 out_seq=0 2024-04-23T10:41:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.908+0000 7f4f4cff9640 1 -- 172.21.15.97:0/2618749208 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c437ed7200 con 0x55c437febe60 2024-04-23T10:41:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.908+0000 7f4f4cff9640 10 client.17775 handle_command_reply: tid=0 2024-04-23T10:41:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.908+0000 7f4f6e764740 1 client.17775 shutdown 2024-04-23T10:41:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.908+0000 7f4f6e764740 1 -- 172.21.15.97:0/2618749208 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c437f6af40 msgr2=0x55c437f6ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.908+0000 7f4f6e764740 1 --2- 172.21.15.97:0/2618749208 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c437f6af40 0x55c437f6ed40 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f4f600107f0 tx=0x7f4f60010820 comp rx=0 tx=0).stop 2024-04-23T10:41:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.908+0000 7f4f6e764740 1 -- 172.21.15.97:0/2618749208 shutdown_connections 2024-04-23T10:41:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.908+0000 7f4f6e764740 1 --2- 172.21.15.97:0/2618749208 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55c437febe60 0x55c437fee280 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.908+0000 7f4f6e764740 1 --2- 172.21.15.97:0/2618749208 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c437f75800 0x55c437f6f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.908+0000 7f4f4cff9640 0 client.17775 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:41:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.908+0000 7f4f6e764740 1 --2- 172.21.15.97:0/2618749208 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c437f6af40 0x55c437f6ed40 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.908+0000 7f4f6e764740 1 --2- 172.21.15.97:0/2618749208 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c437f6a620 0x55c437f7d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:59.909 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.908+0000 7f4f6e764740 1 -- 172.21.15.97:0/2618749208 >> 172.21.15.97:0/2618749208 conn(0x55c437f66b30 msgr2=0x55c437f65cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:41:59.909 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.909+0000 7f4f6e764740 1 -- 172.21.15.97:0/2618749208 shutdown_connections 2024-04-23T10:41:59.909 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.909+0000 7f4f6e764740 1 -- 172.21.15.97:0/2618749208 wait complete. 2024-04-23T10:41:59.909 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.909+0000 7f4f6e764740 20 client.17775 trim_cache size 0 max 16384 2024-04-23T10:41:59.909 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:41:59.909 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:41:59.909 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:41:59.909 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:41:59.910 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.909+0000 7f4f6ba4e640 1 -- 172.21.15.97:0/3240835351 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f4f500a3dc0 msgr2=0x7f4f500a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:59.910 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.909+0000 7f4f6ba4e640 1 --2- 172.21.15.97:0/3240835351 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f4f500a3dc0 0x7f4f500a6280 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4f540ce010 tx=0x7f4f54087040 comp rx=0 tx=0).stop 2024-04-23T10:41:59.910 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.909+0000 7f4f6ba4e640 1 -- 172.21.15.97:0/3240835351 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f64107290 msgr2=0x7f4f64198c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:41:59.910 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.909+0000 7f4f6ba4e640 1 --2- 172.21.15.97:0/3240835351 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f64107290 0x7f4f64198c80 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f4f5400c970 tx=0x7f4f5400ce30 comp rx=0 tx=0).stop 2024-04-23T10:41:59.910 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.909+0000 7f4f6ba4e640 1 -- 172.21.15.97:0/3240835351 shutdown_connections 2024-04-23T10:41:59.911 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.909+0000 7f4f6ba4e640 1 --2- 172.21.15.97:0/3240835351 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f4f500a3dc0 0x7f4f500a6280 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:59.911 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.909+0000 7f4f6ba4e640 1 --2- 172.21.15.97:0/3240835351 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f6419cfb0 0x7f4f6419d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:59.911 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.909+0000 7f4f6ba4e640 1 --2- 172.21.15.97:0/3240835351 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f64107290 0x7f4f64198c80 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:59.911 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.909+0000 7f4f6ba4e640 1 --2- 172.21.15.97:0/3240835351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f641021c0 0x7f4f64198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:41:59.911 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.909+0000 7f4f6ba4e640 1 -- 172.21.15.97:0/3240835351 >> 172.21.15.97:0/3240835351 conn(0x7f4f640fda20 msgr2=0x7f4f640ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:41:59.911 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.909+0000 7f4f6ba4e640 1 -- 172.21.15.97:0/3240835351 shutdown_connections 2024-04-23T10:41:59.911 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:41:59.909+0000 7f4f6ba4e640 1 -- 172.21.15.97:0/3240835351 wait complete. 2024-04-23T10:42:00.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:42:00 smithi196 ceph-mon[32919]: pgmap v495: 97 pgs: 97 active+clean; 3.2 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 8 op/s 2024-04-23T10:42:00.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:42:00 smithi097 ceph-mon[29197]: pgmap v495: 97 pgs: 97 active+clean; 3.2 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 8 op/s 2024-04-23T10:42:00.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:42:00 smithi183 ceph-mon[32317]: pgmap v495: 97 pgs: 97 active+clean; 3.2 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 8 op/s 2024-04-23T10:42:02.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:42:02 smithi196 ceph-mon[32919]: pgmap v496: 97 pgs: 97 active+clean; 3.2 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 8 op/s 2024-04-23T10:42:02.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:42:02 smithi097 ceph-mon[29197]: pgmap v496: 97 pgs: 97 active+clean; 3.2 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 8 op/s 2024-04-23T10:42:02.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:42:02 smithi183 ceph-mon[32317]: pgmap v496: 97 pgs: 97 active+clean; 3.2 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 8 op/s 2024-04-23T10:42:04.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:42:04 smithi196 ceph-mon[32919]: pgmap v497: 97 pgs: 97 active+clean; 3.2 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 49 KiB/s wr, 31 op/s 2024-04-23T10:42:04.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:42:04 smithi097 ceph-mon[29197]: pgmap v497: 97 pgs: 97 active+clean; 3.2 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 49 KiB/s wr, 31 op/s 2024-04-23T10:42:04.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:42:04 smithi183 ceph-mon[32317]: pgmap v497: 97 pgs: 97 active+clean; 3.2 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 49 KiB/s wr, 31 op/s 2024-04-23T10:42:06.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:42:06 smithi196 ceph-mon[32919]: pgmap v498: 97 pgs: 97 active+clean; 3.2 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 49 KiB/s wr, 27 op/s 2024-04-23T10:42:06.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:42:06 smithi097 ceph-mon[29197]: pgmap v498: 97 pgs: 97 active+clean; 3.2 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 49 KiB/s wr, 27 op/s 2024-04-23T10:42:06.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:42:06 smithi183 ceph-mon[32317]: pgmap v498: 97 pgs: 97 active+clean; 3.2 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 49 KiB/s wr, 27 op/s 2024-04-23T10:42:07.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:42:07 smithi196 ceph-mon[32919]: osdmap e108: 12 total, 12 up, 12 in 2024-04-23T10:42:07.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:42:07 smithi097 ceph-mon[29197]: osdmap e108: 12 total, 12 up, 12 in 2024-04-23T10:42:07.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:42:07 smithi183 ceph-mon[32317]: osdmap e108: 12 total, 12 up, 12 in 2024-04-23T10:42:08.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:42:08 smithi196 ceph-mon[32919]: pgmap v500: 97 pgs: 97 active+clean; 3.2 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 59 KiB/s wr, 33 op/s 2024-04-23T10:42:08.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:42:08 smithi097 ceph-mon[29197]: pgmap v500: 97 pgs: 97 active+clean; 3.2 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 59 KiB/s wr, 33 op/s 2024-04-23T10:42:08.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:42:08 smithi183 ceph-mon[32317]: pgmap v500: 97 pgs: 97 active+clean; 3.2 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 59 KiB/s wr, 33 op/s 2024-04-23T10:42:10.274 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T10:42:10.389 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.387+0000 7f7d2b04e640 1 -- 172.21.15.97:0/1191742842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d24106340 msgr2=0x7f7d241067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:10.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.387+0000 7f7d2b04e640 1 --2- 172.21.15.97:0/1191742842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d24106340 0x7f7d241067d0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c009ca0 tx=0x7f7d1c02a950 comp rx=0 tx=0).stop 2024-04-23T10:42:10.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.387+0000 7f7d2b04e640 1 -- 172.21.15.97:0/1191742842 shutdown_connections 2024-04-23T10:42:10.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.387+0000 7f7d2b04e640 1 --2- 172.21.15.97:0/1191742842 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d24106340 0x7f7d241067d0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:10.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.387+0000 7f7d2b04e640 1 --2- 172.21.15.97:0/1191742842 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d24101940 0x7f7d24105e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:10.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.387+0000 7f7d2b04e640 1 --2- 172.21.15.97:0/1191742842 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d24101020 0x7f7d24101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:10.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.387+0000 7f7d2b04e640 1 -- 172.21.15.97:0/1191742842 >> 172.21.15.97:0/1191742842 conn(0x7f7d24077fb0 msgr2=0x7f7d240783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:42:10.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.387+0000 7f7d2b04e640 1 -- 172.21.15.97:0/1191742842 shutdown_connections 2024-04-23T10:42:10.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.387+0000 7f7d2b04e640 1 -- 172.21.15.97:0/1191742842 wait complete. 2024-04-23T10:42:10.390 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.388+0000 7f7d2b04e640 1 Processor -- start 2024-04-23T10:42:10.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.388+0000 7f7d2b04e640 1 -- start start 2024-04-23T10:42:10.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.388+0000 7f7d2b04e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d24101020 0x7f7d2419ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:10.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.388+0000 7f7d2b04e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d24101940 0x7f7d2419f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:10.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.388+0000 7f7d2b04e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d24106340 0x7f7d241a35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:10.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.388+0000 7f7d2b04e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7d241187f0 con 0x7f7d24101020 2024-04-23T10:42:10.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.388+0000 7f7d2b04e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7d24118670 con 0x7f7d24101940 2024-04-23T10:42:10.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.388+0000 7f7d2b04e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7d24118970 con 0x7f7d24106340 2024-04-23T10:42:10.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.388+0000 7f7d28dea640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d24101020 0x7f7d2419ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:10.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.388+0000 7f7d28dea640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d24101020 0x7f7d2419ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50558/0 (socket says 172.21.15.97:50558) 2024-04-23T10:42:10.391 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.388+0000 7f7d28dea640 1 -- 172.21.15.97:0/1651317742 learned_addr learned my addr 172.21.15.97:0/1651317742 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:42:10.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.389+0000 7f7d1bfff640 1 --2- 172.21.15.97:0/1651317742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d24101940 0x7f7d2419f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:10.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.389+0000 7f7d295eb640 1 --2- 172.21.15.97:0/1651317742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d24106340 0x7f7d241a35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:10.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.389+0000 7f7d28dea640 1 -- 172.21.15.97:0/1651317742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d24106340 msgr2=0x7f7d241a35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:10.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.389+0000 7f7d28dea640 1 --2- 172.21.15.97:0/1651317742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d24106340 0x7f7d241a35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:10.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.389+0000 7f7d28dea640 1 -- 172.21.15.97:0/1651317742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d24101940 msgr2=0x7f7d2419f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:10.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.389+0000 7f7d28dea640 1 --2- 172.21.15.97:0/1651317742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d24101940 0x7f7d2419f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:10.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.389+0000 7f7d28dea640 1 -- 172.21.15.97:0/1651317742 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d241a3d20 con 0x7f7d24101020 2024-04-23T10:42:10.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.389+0000 7f7d1bfff640 1 --2- 172.21.15.97:0/1651317742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d24101940 0x7f7d2419f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:42:10.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.389+0000 7f7d295eb640 1 --2- 172.21.15.97:0/1651317742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d24106340 0x7f7d241a35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:42:10.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.389+0000 7f7d28dea640 1 --2- 172.21.15.97:0/1651317742 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d24101020 0x7f7d2419ed30 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f7d2000d950 tx=0x7f7d2000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:42:10.392 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.389+0000 7f7d19ffb640 1 -- 172.21.15.97:0/1651317742 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d20017070 con 0x7f7d24101020 2024-04-23T10:42:10.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.389+0000 7f7d19ffb640 1 -- 172.21.15.97:0/1651317742 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d20009c30 con 0x7f7d24101020 2024-04-23T10:42:10.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.389+0000 7f7d19ffb640 1 -- 172.21.15.97:0/1651317742 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2000f620 con 0x7f7d24101020 2024-04-23T10:42:10.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.389+0000 7f7d2b04e640 1 -- 172.21.15.97:0/1651317742 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d241a4010 con 0x7f7d24101020 2024-04-23T10:42:10.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.389+0000 7f7d2b04e640 1 -- 172.21.15.97:0/1651317742 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d241ab8a0 con 0x7f7d24101020 2024-04-23T10:42:10.393 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.390+0000 7f7d2b04e640 1 -- 172.21.15.97:0/1651317742 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d240691c0 con 0x7f7d24101020 2024-04-23T10:42:10.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.391+0000 7f7d19ffb640 1 -- 172.21.15.97:0/1651317742 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7d20010460 con 0x7f7d24101020 2024-04-23T10:42:10.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.392+0000 7f7d19ffb640 1 --2- 172.21.15.97:0/1651317742 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f7d0c0a3d20 0x7f7d0c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:10.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.392+0000 7f7d19ffb640 1 -- 172.21.15.97:0/1651317742 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8528+0+0 (secure 0 0 0) 0x7f7d200d0510 con 0x7f7d24101020 2024-04-23T10:42:10.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.392+0000 7f7d1bfff640 1 --2- 172.21.15.97:0/1651317742 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f7d0c0a3d20 0x7f7d0c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:10.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.392+0000 7f7d1bfff640 1 --2- 172.21.15.97:0/1651317742 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f7d0c0a3d20 0x7f7d0c0a61e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7d14005fd0 tx=0x7f7d140094d0 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:42:10.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.396+0000 7f7d19ffb640 1 -- 172.21.15.97:0/1651317742 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7d2009cbb0 con 0x7f7d24101020 2024-04-23T10:42:10.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:42:10 smithi097 ceph-mon[29197]: pgmap v501: 97 pgs: 97 active+clean; 3.2 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 59 KiB/s wr, 33 op/s 2024-04-23T10:42:10.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.614+0000 7f7d2b04e640 1 -- 172.21.15.97:0/1651317742 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7d241abad0 con 0x7f7d24101020 2024-04-23T10:42:10.613 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.615+0000 7f7d19ffb640 1 -- 172.21.15.97:0/1651317742 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11931 (secure 0 0 0) 0x7f7d20009de0 con 0x7f7d24101020 2024-04-23T10:42:10.614 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:42:10.614 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3946878762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3946878762},{"type":"v1","addr":"172.21.15.97:6835","nonce":3946878762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/209800861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":209800861},{"type":"v1","addr":"172.21.15.97:6837","nonce":209800861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T10:24:06.910134+0000","modified":"2024-04-23T10:26:09.820985+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6839/418013575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":418013575},{"type":"v1","addr":"172.21.15.97:6839","nonce":418013575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6841/747242680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":747242680},{"type":"v1","addr":"172.21.15.97:6841","nonce":747242680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6833/4292847121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4292847121},{"type":"v1","addr":"172.21.15.183:6833","nonce":4292847121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6833/3885816565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3885816565},{"type":"v1","addr":"172.21.15.196:6833","nonce":3885816565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1897093105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1897093105},{"type":"v1","addr":"172.21.15.183:6835","nonce":1897093105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1171477563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1171477563},{"type":"v1","addr":"172.21.15.196:6835","nonce":1171477563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1898341911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1898341911},{"type":"v1","addr":"172.21.15.183:6837","nonce":1898341911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.196:6837/3094946502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3094946502},{"type":"v1","addr":"172.21.15.196:6837","nonce":3094946502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.196:6839/83843556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":83843556},{"type":"v1","addr":"172.21.15.196:6839","nonce":83843556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6839/1763841130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1763841130},{"type":"v1","addr":"172.21.15.183:6839","nonce":1763841130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24487,24403,24398,24469,24488]},"id":1}]} 2024-04-23T10:42:10.614 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 26 2024-04-23T10:42:10.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.617+0000 7f7d2b04e640 1 -- 172.21.15.97:0/1651317742 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f7d0c0a3d20 msgr2=0x7f7d0c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:10.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.617+0000 7f7d2b04e640 1 --2- 172.21.15.97:0/1651317742 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f7d0c0a3d20 0x7f7d0c0a61e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7d14005fd0 tx=0x7f7d140094d0 comp rx=0 tx=0).stop 2024-04-23T10:42:10.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.617+0000 7f7d2b04e640 1 -- 172.21.15.97:0/1651317742 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d24101020 msgr2=0x7f7d2419ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:10.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.617+0000 7f7d2b04e640 1 --2- 172.21.15.97:0/1651317742 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d24101020 0x7f7d2419ed30 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f7d2000d950 tx=0x7f7d2000de10 comp rx=0 tx=0).stop 2024-04-23T10:42:10.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.617+0000 7f7d2b04e640 1 -- 172.21.15.97:0/1651317742 shutdown_connections 2024-04-23T10:42:10.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.617+0000 7f7d2b04e640 1 --2- 172.21.15.97:0/1651317742 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f7d0c0a3d20 0x7f7d0c0a61e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:10.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.617+0000 7f7d2b04e640 1 --2- 172.21.15.97:0/1651317742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d24106340 0x7f7d241a35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:10.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.617+0000 7f7d2b04e640 1 --2- 172.21.15.97:0/1651317742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d24101940 0x7f7d2419f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:10.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.617+0000 7f7d2b04e640 1 --2- 172.21.15.97:0/1651317742 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7d24101020 0x7f7d2419ed30 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:10.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.617+0000 7f7d2b04e640 1 -- 172.21.15.97:0/1651317742 >> 172.21.15.97:0/1651317742 conn(0x7f7d24077fb0 msgr2=0x7f7d24103dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:42:10.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.617+0000 7f7d2b04e640 1 -- 172.21.15.97:0/1651317742 shutdown_connections 2024-04-23T10:42:10.617 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.617+0000 7f7d2b04e640 1 -- 172.21.15.97:0/1651317742 wait complete. 2024-04-23T10:42:10.627 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --set-id 26d43730 --release --await' 2024-04-23T10:42:10.627 DEBUG:teuthology.orchestra.run.smithi097:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24488 quiesce db --set-id 26d43730 --release --await 2024-04-23T10:42:10.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.733+0000 7f5a9b19b640 1 -- 172.21.15.97:0/1825603436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a94102190 msgr2=0x7f5a94102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:10.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.733+0000 7f5a9b19b640 1 --2- 172.21.15.97:0/1825603436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a94102190 0x7f5a94102570 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f5a84009900 tx=0x7f5a8402a410 comp rx=0 tx=0).stop 2024-04-23T10:42:10.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.733+0000 7f5a9b19b640 1 -- 172.21.15.97:0/1825603436 shutdown_connections 2024-04-23T10:42:10.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.733+0000 7f5a9b19b640 1 --2- 172.21.15.97:0/1825603436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a94107260 0x7f5a9410ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:10.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.733+0000 7f5a9b19b640 1 --2- 172.21.15.97:0/1825603436 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a94102b40 0x7f5a94106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:10.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.733+0000 7f5a9b19b640 1 --2- 172.21.15.97:0/1825603436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a94102190 0x7f5a94102570 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:10.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.733+0000 7f5a9b19b640 1 -- 172.21.15.97:0/1825603436 >> 172.21.15.97:0/1825603436 conn(0x7f5a940fd990 msgr2=0x7f5a940ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:42:10.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.733+0000 7f5a9b19b640 1 -- 172.21.15.97:0/1825603436 shutdown_connections 2024-04-23T10:42:10.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.733+0000 7f5a9b19b640 1 -- 172.21.15.97:0/1825603436 wait complete. 2024-04-23T10:42:10.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.734+0000 7f5a9b19b640 1 Processor -- start 2024-04-23T10:42:10.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.734+0000 7f5a9b19b640 1 -- start start 2024-04-23T10:42:10.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.734+0000 7f5a9b19b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a94102190 0x7f5a94198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:10.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.734+0000 7f5a9b19b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a94107260 0x7f5a94198cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:10.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.734+0000 7f5a9b19b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a9419d000 0x7f5a9419d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:10.735 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.734+0000 7f5a9b19b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5a941121d0 con 0x7f5a9419d000 2024-04-23T10:42:10.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.734+0000 7f5a9b19b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5a94112050 con 0x7f5a94102190 2024-04-23T10:42:10.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.734+0000 7f5a9b19b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5a94112350 con 0x7f5a94107260 2024-04-23T10:42:10.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.734+0000 7f5a99738640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a9419d000 0x7f5a9419d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:10.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.734+0000 7f5a99738640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a9419d000 0x7f5a9419d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50576/0 (socket says 172.21.15.97:50576) 2024-04-23T10:42:10.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.734+0000 7f5a99738640 1 -- 172.21.15.97:0/3789891137 learned_addr learned my addr 172.21.15.97:0/3789891137 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:42:10.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.734+0000 7f5a98f37640 1 --2- 172.21.15.97:0/3789891137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a94102190 0x7f5a94198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:10.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.734+0000 7f5a93fff640 1 --2- 172.21.15.97:0/3789891137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a94107260 0x7f5a94198cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:10.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.734+0000 7f5a99738640 1 -- 172.21.15.97:0/3789891137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a94107260 msgr2=0x7f5a94198cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:10.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.734+0000 7f5a99738640 1 --2- 172.21.15.97:0/3789891137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a94107260 0x7f5a94198cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:10.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.734+0000 7f5a99738640 1 -- 172.21.15.97:0/3789891137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a94102190 msgr2=0x7f5a94198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:10.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.734+0000 7f5a99738640 1 --2- 172.21.15.97:0/3789891137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a94102190 0x7f5a94198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:10.736 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.734+0000 7f5a99738640 1 -- 172.21.15.97:0/3789891137 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a9419daf0 con 0x7f5a9419d000 2024-04-23T10:42:10.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.734+0000 7f5a98f37640 1 --2- 172.21.15.97:0/3789891137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a94102190 0x7f5a94198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:42:10.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.735+0000 7f5a99738640 1 --2- 172.21.15.97:0/3789891137 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a9419d000 0x7f5a9419d4b0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c00eaa0 tx=0x7f5a8c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:42:10.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.735+0000 7f5a91ffb640 1 -- 172.21.15.97:0/3789891137 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c00cdb0 con 0x7f5a9419d000 2024-04-23T10:42:10.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.735+0000 7f5a9b19b640 1 -- 172.21.15.97:0/3789891137 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a9419dde0 con 0x7f5a9419d000 2024-04-23T10:42:10.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.735+0000 7f5a91ffb640 1 -- 172.21.15.97:0/3789891137 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a8c005020 con 0x7f5a9419d000 2024-04-23T10:42:10.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.735+0000 7f5a91ffb640 1 -- 172.21.15.97:0/3789891137 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c0116d0 con 0x7f5a9419d000 2024-04-23T10:42:10.737 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.735+0000 7f5a9b19b640 1 -- 172.21.15.97:0/3789891137 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a941a55d0 con 0x7f5a9419d000 2024-04-23T10:42:10.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.738+0000 7f5a91ffb640 1 -- 172.21.15.97:0/3789891137 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5a8c0054a0 con 0x7f5a9419d000 2024-04-23T10:42:10.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.738+0000 7f5a9df93740 1 Processor -- start 2024-04-23T10:42:10.744 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.738+0000 7f5a9df93740 1 -- start start 2024-04-23T10:42:10.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.738+0000 7f5a9df93740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556660426490 0x556660426870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:10.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.738+0000 7f5a9df93740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556660426db0 0x556660430f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:10.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.738+0000 7f5a9df93740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556660431480 0x556660433870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:10.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.738+0000 7f5a9df93740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55666038b6d0 con 0x556660431480 2024-04-23T10:42:10.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.738+0000 7f5a9df93740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5566603281a0 con 0x556660426490 2024-04-23T10:42:10.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.738+0000 7f5a91ffb640 1 --2- 172.21.15.97:0/3789891137 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f5a7c0a3d20 0x7f5a7c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:10.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.738+0000 7f5a9df93740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556660387690 con 0x556660426db0 2024-04-23T10:42:10.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.738+0000 7f5a93fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556660431480 0x556660433870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:10.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.738+0000 7f5a99738640 1 --2- 172.21.15.97:0/3789891137 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f5a7c0a3d20 0x7f5a7c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:10.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.738+0000 7f5a93fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556660431480 0x556660433870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50588/0 (socket says 172.21.15.97:50588) 2024-04-23T10:42:10.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.738+0000 7f5a93fff640 1 -- 172.21.15.97:0/474481241 learned_addr learned my addr 172.21.15.97:0/474481241 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:42:10.745 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.738+0000 7f5a93fff640 1 --2- 172.21.15.97:0/474481241 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556660426490 0x556660426870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:10.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.738+0000 7f5a91ffb640 1 -- 172.21.15.97:0/3789891137 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8528+0+0 (secure 0 0 0) 0x7f5a8c0d09e0 con 0x7f5a9419d000 2024-04-23T10:42:10.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.739+0000 7f5a98f37640 1 --2- 172.21.15.97:0/474481241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556660426db0 0x556660430f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:10.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.739+0000 7f5a93fff640 1 -- 172.21.15.97:0/474481241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556660426db0 msgr2=0x556660430f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:42:10.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.739+0000 7f5a93fff640 1 --2- 172.21.15.97:0/474481241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556660426db0 0x556660430f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:10.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.739+0000 7f5a93fff640 1 -- 172.21.15.97:0/474481241 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556660426490 msgr2=0x556660426870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:42:10.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.739+0000 7f5a93fff640 1 --2- 172.21.15.97:0/474481241 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556660426490 0x556660426870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:10.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.739+0000 7f5a93fff640 1 -- 172.21.15.97:0/474481241 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566603325b0 con 0x556660431480 2024-04-23T10:42:10.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.739+0000 7f5a99738640 1 --2- 172.21.15.97:0/3789891137 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f5a7c0a3d20 0x7f5a7c0a61e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c09c680 tx=0x7f5a8c06e3c0 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:42:10.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.739+0000 7f5a93fff640 1 --2- 172.21.15.97:0/474481241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556660431480 0x556660433870 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f5a88013830 tx=0x7f5a8800ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89cc6aecefd08866 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:42:10.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.739+0000 7f5a7affd640 1 -- 172.21.15.97:0/474481241 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a88005220 con 0x556660431480 2024-04-23T10:42:10.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.739+0000 7f5a7affd640 1 -- 172.21.15.97:0/474481241 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a88004c50 con 0x556660431480 2024-04-23T10:42:10.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.739+0000 7f5a7affd640 1 -- 172.21.15.97:0/474481241 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a8800b8a0 con 0x556660431480 2024-04-23T10:42:10.746 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.740+0000 7f5a9df93740 1 -- 172.21.15.97:0/474481241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556660431480 msgr2=0x556660433870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:10.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.740+0000 7f5a9df93740 1 --2- 172.21.15.97:0/474481241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556660431480 0x556660433870 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f5a88013830 tx=0x7f5a8800ef90 comp rx=0 tx=0).stop 2024-04-23T10:42:10.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.740+0000 7f5a9df93740 1 -- 172.21.15.97:0/474481241 shutdown_connections 2024-04-23T10:42:10.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.740+0000 7f5a9df93740 1 --2- 172.21.15.97:0/474481241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556660431480 0x556660433870 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:10.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.740+0000 7f5a9df93740 1 --2- 172.21.15.97:0/474481241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556660426db0 0x556660430f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:10.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.740+0000 7f5a9df93740 1 --2- 172.21.15.97:0/474481241 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556660426490 0x556660426870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:10.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.740+0000 7f5a9df93740 1 -- 172.21.15.97:0/474481241 >> 172.21.15.97:0/474481241 conn(0x5566604227b0 msgr2=0x556660422b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:42:10.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.740+0000 7f5a9df93740 1 -- 172.21.15.97:0/474481241 shutdown_connections 2024-04-23T10:42:10.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.740+0000 7f5a9df93740 1 -- 172.21.15.97:0/474481241 wait complete. 2024-04-23T10:42:10.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.740+0000 7f5a9df93740 1 -- start start 2024-04-23T10:42:10.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.740+0000 7f5a9df93740 1 Processor -- start 2024-04-23T10:42:10.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.740+0000 7f5a9df93740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556660426490 0x556660430ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:10.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.740+0000 7f5a9df93740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556660426db0 0x55666042e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:10.747 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.740+0000 7f5a9df93740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556660431480 0x55666042e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:10.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.740+0000 7f5a9df93740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x556660434310 con 0x556660426db0 2024-04-23T10:42:10.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.740+0000 7f5a9df93740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556660325b20 con 0x556660431480 2024-04-23T10:42:10.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.740+0000 7f5a9df93740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556660434490 con 0x556660426490 2024-04-23T10:42:10.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.741+0000 7f5a9df93740 10 client.0 _ll_register_callbacks cb 0x556660325530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:42:10.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.741+0000 7f5a98f37640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556660426db0 0x55666042e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:10.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.741+0000 7f5a98f37640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556660426db0 0x55666042e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50598/0 (socket says 172.21.15.97:50598) 2024-04-23T10:42:10.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.741+0000 7f5a98f37640 1 -- 172.21.15.97:0/3060703203 learned_addr learned my addr 172.21.15.97:0/3060703203 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:42:10.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.741+0000 7f5a93fff640 1 --2- 172.21.15.97:0/3060703203 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556660431480 0x55666042e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:10.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.741+0000 7f5a93fff640 1 --2- 172.21.15.97:0/3060703203 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556660426490 0x556660430ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:10.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.741+0000 7f5a98f37640 1 -- 172.21.15.97:0/3060703203 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556660426490 msgr2=0x556660430ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:42:10.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.741+0000 7f5a98f37640 1 --2- 172.21.15.97:0/3060703203 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556660426490 0x556660430ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:10.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.741+0000 7f5a98f37640 1 -- 172.21.15.97:0/3060703203 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556660431480 msgr2=0x55666042e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:42:10.748 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.741+0000 7f5a98f37640 1 --2- 172.21.15.97:0/3060703203 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556660431480 0x55666042e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:10.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.741+0000 7f5a98f37640 1 -- 172.21.15.97:0/3060703203 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55666009d920 con 0x556660426db0 2024-04-23T10:42:10.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.741+0000 7f5a93fff640 1 --2- 172.21.15.97:0/3060703203 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556660431480 0x55666042e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:42:10.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.741+0000 7f5a98f37640 1 --2- 172.21.15.97:0/3060703203 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556660426db0 0x55666042e190 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f5a84009900 tx=0x7f5a84031740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcfc6e3bcd63607 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:42:10.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.741+0000 7f5a73fff640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:42:10.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.741+0000 7f5a73fff640 1 -- 172.21.15.97:0/3060703203 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a84035070 con 0x556660426db0 2024-04-23T10:42:10.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.741+0000 7f5a73fff640 1 -- 172.21.15.97:0/3060703203 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8528+0+0 (secure 0 0 0) 0x7f5a84004240 con 0x556660426db0 2024-04-23T10:42:10.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.741+0000 7f5a9df93740 1 -- 172.21.15.97:0/3060703203 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a88004c50 con 0x556660426db0 2024-04-23T10:42:10.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.741+0000 7f5a73fff640 1 -- 172.21.15.97:0/3060703203 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5a84004440 con 0x556660426db0 2024-04-23T10:42:10.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.741+0000 7f5a9df93740 10 client.17793 fetch_fsmap learned FSMap version 26 2024-04-23T10:42:10.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.741+0000 7f5a9df93740 1 -- 172.21.15.97:0/3060703203 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556660169d30 con 0x556660426db0 2024-04-23T10:42:10.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.742+0000 7f5a73fff640 1 -- 172.21.15.97:0/3060703203 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a88004c50 con 0x556660426db0 2024-04-23T10:42:10.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.742+0000 7f5a73fff640 1 -- 172.21.15.97:0/3060703203 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5a84004ad0 con 0x556660426db0 2024-04-23T10:42:10.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.742+0000 7f5a9df93740 10 client.17793 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:42:10.749 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.742+0000 7f5a9df93740 10 client.17793 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:42:10.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.742+0000 7f5a9df93740 20 client.17793 populate_metadata read hostname 'smithi097' 2024-04-23T10:42:10.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.742+0000 7f5a9df93740 1 --2- 172.21.15.97:0/3060703203 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x556660441680 0x556660443aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:10.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.742+0000 7f5a9df93740 4 client.17793 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:42:10.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.742+0000 7f5a9df93740 1 -- 172.21.15.97:0/3060703203 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556660349500 con 0x556660441680 2024-04-23T10:42:10.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.742+0000 7f5a93fff640 1 --2- 172.21.15.97:0/3060703203 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x556660441680 0x556660443aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:10.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.742+0000 7f5a93fff640 1 --2- 172.21.15.97:0/3060703203 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x556660441680 0x556660443aa0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=302ac86550ea928c server_cookie=b8e41e38ad6ab252 in_seq=0 out_seq=0 2024-04-23T10:42:10.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.742+0000 7f5a73fff640 10 client.17793 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:42:10.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.743+0000 7f5a73fff640 1 -- 172.21.15.97:0/3060703203 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556660349500 con 0x556660441680 2024-04-23T10:42:10.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.743+0000 7f5a73fff640 10 client.17793 handle_command_reply: tid=0 2024-04-23T10:42:10.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.743+0000 7f5a9df93740 1 client.17793 shutdown 2024-04-23T10:42:10.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.743+0000 7f5a9df93740 1 -- 172.21.15.97:0/3060703203 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556660426db0 msgr2=0x55666042e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:10.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.743+0000 7f5a9df93740 1 --2- 172.21.15.97:0/3060703203 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556660426db0 0x55666042e190 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f5a84009900 tx=0x7f5a84031740 comp rx=0 tx=0).stop 2024-04-23T10:42:10.750 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.743+0000 7f5a9df93740 1 -- 172.21.15.97:0/3060703203 shutdown_connections 2024-04-23T10:42:10.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.743+0000 7f5a9df93740 1 --2- 172.21.15.97:0/3060703203 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x556660441680 0x556660443aa0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:10.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.743+0000 7f5a9df93740 1 --2- 172.21.15.97:0/3060703203 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556660431480 0x55666042e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:10.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.743+0000 7f5a73fff640 0 client.17793 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:42:10.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.743+0000 7f5a9df93740 1 --2- 172.21.15.97:0/3060703203 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556660426db0 0x55666042e190 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:10.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.743+0000 7f5a9df93740 1 --2- 172.21.15.97:0/3060703203 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556660426490 0x556660430ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:10.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.743+0000 7f5a9df93740 1 -- 172.21.15.97:0/3060703203 >> 172.21.15.97:0/3060703203 conn(0x5566604227b0 msgr2=0x5566604285b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:42:10.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.743+0000 7f5a9df93740 1 -- 172.21.15.97:0/3060703203 shutdown_connections 2024-04-23T10:42:10.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.743+0000 7f5a9df93740 1 -- 172.21.15.97:0/3060703203 wait complete. 2024-04-23T10:42:10.751 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.744+0000 7f5a9df93740 20 client.17793 trim_cache size 0 max 16384 2024-04-23T10:42:10.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.760+0000 7f5a9df93740 1 Processor -- start 2024-04-23T10:42:10.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.760+0000 7f5a9df93740 1 -- start start 2024-04-23T10:42:10.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.760+0000 7f5a9df93740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556660426490 0x5566604309d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:10.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.761+0000 7f5a9df93740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556660426db0 0x556660449370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:10.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.761+0000 7f5a9df93740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556660431480 0x5566604498b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:10.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.761+0000 7f5a9df93740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5a941adb70 con 0x556660426490 2024-04-23T10:42:10.765 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.761+0000 7f5a9df93740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5a941af6b0 con 0x556660426db0 2024-04-23T10:42:10.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.761+0000 7f5a9df93740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5a941ae240 con 0x556660431480 2024-04-23T10:42:10.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.761+0000 7f5a93fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556660426490 0x5566604309d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:10.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.761+0000 7f5a93fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556660426490 0x5566604309d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50610/0 (socket says 172.21.15.97:50610) 2024-04-23T10:42:10.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.761+0000 7f5a93fff640 1 -- 172.21.15.97:0/574904309 learned_addr learned my addr 172.21.15.97:0/574904309 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:42:10.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.761+0000 7f5a93fff640 1 -- 172.21.15.97:0/574904309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556660431480 msgr2=0x5566604498b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:42:10.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.761+0000 7f5a98f37640 1 --2- 172.21.15.97:0/574904309 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556660426db0 0x556660449370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:10.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.761+0000 7f5a93fff640 1 --2- 172.21.15.97:0/574904309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556660431480 0x5566604498b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:10.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.761+0000 7f5a93fff640 1 -- 172.21.15.97:0/574904309 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556660426db0 msgr2=0x556660449370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:42:10.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.761+0000 7f5a93fff640 1 --2- 172.21.15.97:0/574904309 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556660426db0 0x556660449370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:10.766 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.761+0000 7f5a93fff640 1 -- 172.21.15.97:0/574904309 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556660428cd0 con 0x556660426490 2024-04-23T10:42:10.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.761+0000 7f5a98f37640 1 --2- 172.21.15.97:0/574904309 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556660426db0 0x556660449370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:42:10.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.761+0000 7f5a93fff640 1 --2- 172.21.15.97:0/574904309 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556660426490 0x5566604309d0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f5a880137d0 tx=0x7f5a8800eab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dee1df1e827a8f6d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:42:10.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.761+0000 7f5a7a7fc640 1 -- 172.21.15.97:0/574904309 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a88030070 con 0x556660426490 2024-04-23T10:42:10.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.761+0000 7f5a7a7fc640 1 -- 172.21.15.97:0/574904309 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a88005850 con 0x556660426490 2024-04-23T10:42:10.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.761+0000 7f5a7a7fc640 1 -- 172.21.15.97:0/574904309 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a8802b650 con 0x556660426490 2024-04-23T10:42:10.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.762+0000 7f5a9df93740 1 -- 172.21.15.97:0/574904309 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556660426490 msgr2=0x5566604309d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:10.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.762+0000 7f5a9df93740 1 --2- 172.21.15.97:0/574904309 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556660426490 0x5566604309d0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f5a880137d0 tx=0x7f5a8800eab0 comp rx=0 tx=0).stop 2024-04-23T10:42:10.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.762+0000 7f5a9df93740 1 -- 172.21.15.97:0/574904309 shutdown_connections 2024-04-23T10:42:10.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.762+0000 7f5a9df93740 1 --2- 172.21.15.97:0/574904309 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556660431480 0x5566604498b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:10.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.762+0000 7f5a9df93740 1 --2- 172.21.15.97:0/574904309 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556660426db0 0x556660449370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:10.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.762+0000 7f5a9df93740 1 --2- 172.21.15.97:0/574904309 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556660426490 0x5566604309d0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:10.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.762+0000 7f5a9df93740 1 -- 172.21.15.97:0/574904309 >> 172.21.15.97:0/574904309 conn(0x5566604227b0 msgr2=0x5566604219d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:42:10.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.762+0000 7f5a9df93740 1 -- 172.21.15.97:0/574904309 shutdown_connections 2024-04-23T10:42:10.767 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.762+0000 7f5a9df93740 1 -- 172.21.15.97:0/574904309 wait complete. 2024-04-23T10:42:10.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.762+0000 7f5a9df93740 1 -- start start 2024-04-23T10:42:10.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.762+0000 7f5a9df93740 1 Processor -- start 2024-04-23T10:42:10.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.763+0000 7f5a9df93740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556660426490 0x5566604346a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:10.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.763+0000 7f5a9df93740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556660426db0 0x556660434be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:10.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.763+0000 7f5a9df93740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556660431480 0x556660435120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:10.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.763+0000 7f5a9df93740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5a941aefe0 con 0x556660426490 2024-04-23T10:42:10.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.763+0000 7f5a9df93740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5a941ae910 con 0x556660431480 2024-04-23T10:42:10.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.763+0000 7f5a9df93740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5a941abb50 con 0x556660426db0 2024-04-23T10:42:10.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.763+0000 7f5a9df93740 10 client.0 _ll_register_callbacks cb 0x7f5a84005de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:42:10.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.763+0000 7f5a93fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556660426490 0x5566604346a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:10.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.763+0000 7f5a93fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556660426490 0x5566604346a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50622/0 (socket says 172.21.15.97:50622) 2024-04-23T10:42:10.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.763+0000 7f5a93fff640 1 -- 172.21.15.97:0/3984078946 learned_addr learned my addr 172.21.15.97:0/3984078946 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:42:10.768 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.763+0000 7f5a93fff640 1 --2- 172.21.15.97:0/3984078946 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556660431480 0x556660435120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:10.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.763+0000 7f5a98f37640 1 --2- 172.21.15.97:0/3984078946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556660426db0 0x556660434be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:10.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.763+0000 7f5a93fff640 1 -- 172.21.15.97:0/3984078946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556660426db0 msgr2=0x556660434be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:42:10.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.763+0000 7f5a93fff640 1 --2- 172.21.15.97:0/3984078946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556660426db0 0x556660434be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:10.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.763+0000 7f5a93fff640 1 -- 172.21.15.97:0/3984078946 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556660431480 msgr2=0x556660435120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:42:10.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.763+0000 7f5a93fff640 1 --2- 172.21.15.97:0/3984078946 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556660431480 0x556660435120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:10.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.763+0000 7f5a93fff640 1 -- 172.21.15.97:0/3984078946 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566604a7be0 con 0x556660426490 2024-04-23T10:42:10.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.763+0000 7f5a93fff640 1 --2- 172.21.15.97:0/3984078946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556660426490 0x5566604346a0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f5a8800eb60 tx=0x7f5a8800ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7403b821f04e5f13 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:42:10.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.763+0000 7f5a73fff640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:42:10.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.763+0000 7f5a73fff640 1 -- 172.21.15.97:0/3984078946 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a88030070 con 0x556660426490 2024-04-23T10:42:10.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.763+0000 7f5a73fff640 1 -- 172.21.15.97:0/3984078946 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8528+0+0 (secure 0 0 0) 0x7f5a880047c0 con 0x556660426490 2024-04-23T10:42:10.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.763+0000 7f5a9df93740 1 -- 172.21.15.97:0/3984078946 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a88005850 con 0x556660426490 2024-04-23T10:42:10.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.764+0000 7f5a73fff640 1 -- 172.21.15.97:0/3984078946 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5a880049c0 con 0x556660426490 2024-04-23T10:42:10.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.764+0000 7f5a9df93740 10 client.17799 fetch_fsmap learned FSMap version 26 2024-04-23T10:42:10.769 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.764+0000 7f5a9df93740 1 -- 172.21.15.97:0/3984078946 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5566604a8d70 con 0x556660426490 2024-04-23T10:42:10.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.764+0000 7f5a73fff640 1 -- 172.21.15.97:0/3984078946 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a88005850 con 0x556660426490 2024-04-23T10:42:10.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.764+0000 7f5a73fff640 1 -- 172.21.15.97:0/3984078946 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5a8800f040 con 0x556660426490 2024-04-23T10:42:10.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.764+0000 7f5a9df93740 10 client.17799 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:42:10.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.764+0000 7f5a9df93740 10 client.17799 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:42:10.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.764+0000 7f5a9df93740 20 client.17799 populate_metadata read hostname 'smithi097' 2024-04-23T10:42:10.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.764+0000 7f5a9df93740 1 --2- 172.21.15.97:0/3984078946 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5566604a91c0 0x5566604ab5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:10.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.764+0000 7f5a9df93740 4 client.17799 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "26d43730", "release": true, "await": true}] 2024-04-23T10:42:10.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.764+0000 7f5a9df93740 1 -- 172.21.15.97:0/3984078946 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "quiesce db", "set_id": "26d43730", "release": true, "await": true}) -- 0x5566603990f0 con 0x5566604a91c0 2024-04-23T10:42:10.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.764+0000 7f5a98f37640 1 --2- 172.21.15.97:0/3984078946 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5566604a91c0 0x5566604ab5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:10.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.764+0000 7f5a73fff640 10 client.17799 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:42:10.770 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.764+0000 7f5a98f37640 1 --2- 172.21.15.97:0/3984078946 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5566604a91c0 0x5566604ab5e0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32a488fd55aa0906 server_cookie=c6fe042c819356f8 in_seq=0 out_seq=0 2024-04-23T10:42:10.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.767+0000 7f5a73fff640 1 -- 172.21.15.97:0/3984078946 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5566603990f0 con 0x5566604a91c0 2024-04-23T10:42:10.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.767+0000 7f5a73fff640 10 client.17799 handle_command_reply: tid=0 2024-04-23T10:42:10.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.767+0000 7f5a9df93740 1 client.17799 shutdown 2024-04-23T10:42:10.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.767+0000 7f5a9df93740 1 -- 172.21.15.97:0/3984078946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556660426490 msgr2=0x5566604346a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:10.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.767+0000 7f5a9df93740 1 --2- 172.21.15.97:0/3984078946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556660426490 0x5566604346a0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f5a8800eb60 tx=0x7f5a8800ef90 comp rx=0 tx=0).stop 2024-04-23T10:42:10.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.767+0000 7f5a9df93740 1 -- 172.21.15.97:0/3984078946 shutdown_connections 2024-04-23T10:42:10.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.767+0000 7f5a9df93740 1 --2- 172.21.15.97:0/3984078946 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5566604a91c0 0x5566604ab5e0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:10.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.767+0000 7f5a9df93740 1 --2- 172.21.15.97:0/3984078946 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556660431480 0x556660435120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:10.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.767+0000 7f5a73fff640 0 client.17799 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:42:10.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.767+0000 7f5a9df93740 1 --2- 172.21.15.97:0/3984078946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556660426db0 0x556660434be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:10.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.767+0000 7f5a9df93740 1 --2- 172.21.15.97:0/3984078946 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556660426490 0x5566604346a0 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:10.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.767+0000 7f5a9df93740 1 -- 172.21.15.97:0/3984078946 >> 172.21.15.97:0/3984078946 conn(0x5566604227b0 msgr2=0x556660427cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:42:10.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.767+0000 7f5a9df93740 1 -- 172.21.15.97:0/3984078946 shutdown_connections 2024-04-23T10:42:10.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.767+0000 7f5a9df93740 1 -- 172.21.15.97:0/3984078946 wait complete. 2024-04-23T10:42:10.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.767+0000 7f5a9df93740 20 client.17799 trim_cache size 0 max 16384 2024-04-23T10:42:10.773 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-23T10:42:10.773 INFO:teuthology.orchestra.run.smithi097.stdout: "epoch": 26, 2024-04-23T10:42:10.773 INFO:teuthology.orchestra.run.smithi097.stdout: "leader": 24488, 2024-04-23T10:42:10.773 INFO:teuthology.orchestra.run.smithi097.stdout: "set_version": 68, 2024-04-23T10:42:10.773 INFO:teuthology.orchestra.run.smithi097.stdout: "sets": { 2024-04-23T10:42:10.773 INFO:teuthology.orchestra.run.smithi097.stdout: "26d43730": { 2024-04-23T10:42:10.773 INFO:teuthology.orchestra.run.smithi097.stdout: "version": 68, 2024-04-23T10:42:10.773 INFO:teuthology.orchestra.run.smithi097.stdout: "age_ref": 0.0, 2024-04-23T10:42:10.773 INFO:teuthology.orchestra.run.smithi097.stdout: "state": { 2024-04-23T10:42:10.774 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RELEASED", 2024-04-23T10:42:10.774 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0 2024-04-23T10:42:10.774 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:10.774 INFO:teuthology.orchestra.run.smithi097.stdout: "timeout": 60.0, 2024-04-23T10:42:10.774 INFO:teuthology.orchestra.run.smithi097.stdout: "expiration": 84.7, 2024-04-23T10:42:10.774 INFO:teuthology.orchestra.run.smithi097.stdout: "members": { 2024-04-23T10:42:10.774 INFO:teuthology.orchestra.run.smithi097.stdout: "file:/": { 2024-04-23T10:42:10.774 INFO:teuthology.orchestra.run.smithi097.stdout: "excluded": false, 2024-04-23T10:42:10.774 INFO:teuthology.orchestra.run.smithi097.stdout: "state": { 2024-04-23T10:42:10.774 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RELEASED", 2024-04-23T10:42:10.774 INFO:teuthology.orchestra.run.smithi097.stdout: "age": 0.0 2024-04-23T10:42:10.774 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:10.774 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:10.774 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:10.775 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:10.775 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:10.775 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-23T10:42:10.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.768+0000 7f5a9b19b640 1 -- 172.21.15.97:0/3789891137 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f5a7c0a3d20 msgr2=0x7f5a7c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:10.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.768+0000 7f5a9b19b640 1 --2- 172.21.15.97:0/3789891137 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f5a7c0a3d20 0x7f5a7c0a61e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c09c680 tx=0x7f5a8c06e3c0 comp rx=0 tx=0).stop 2024-04-23T10:42:10.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.768+0000 7f5a9b19b640 1 -- 172.21.15.97:0/3789891137 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a9419d000 msgr2=0x7f5a9419d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:10.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.768+0000 7f5a9b19b640 1 --2- 172.21.15.97:0/3789891137 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a9419d000 0x7f5a9419d4b0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c00eaa0 tx=0x7f5a8c00ef60 comp rx=0 tx=0).stop 2024-04-23T10:42:10.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.768+0000 7f5a9b19b640 1 -- 172.21.15.97:0/3789891137 shutdown_connections 2024-04-23T10:42:10.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.768+0000 7f5a9b19b640 1 --2- 172.21.15.97:0/3789891137 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f5a7c0a3d20 0x7f5a7c0a61e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:10.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.768+0000 7f5a9b19b640 1 --2- 172.21.15.97:0/3789891137 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a9419d000 0x7f5a9419d4b0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:10.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.768+0000 7f5a9b19b640 1 --2- 172.21.15.97:0/3789891137 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5a94107260 0x7f5a94198cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:10.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.768+0000 7f5a9b19b640 1 --2- 172.21.15.97:0/3789891137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a94102190 0x7f5a94198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:10.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.768+0000 7f5a9b19b640 1 -- 172.21.15.97:0/3789891137 >> 172.21.15.97:0/3789891137 conn(0x7f5a940fd990 msgr2=0x7f5a940fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:42:10.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.768+0000 7f5a9b19b640 1 -- 172.21.15.97:0/3789891137 shutdown_connections 2024-04-23T10:42:10.776 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:10.768+0000 7f5a9b19b640 1 -- 172.21.15.97:0/3789891137 wait complete. 2024-04-23T10:42:10.778 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '26d43730', seconds elapsed: 25.7 2024-04-23T10:42:10.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:42:10 smithi196 ceph-mon[32919]: pgmap v501: 97 pgs: 97 active+clean; 3.2 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 59 KiB/s wr, 33 op/s 2024-04-23T10:42:10.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:42:10 smithi183 ceph-mon[32317]: pgmap v501: 97 pgs: 97 active+clean; 3.2 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 59 KiB/s wr, 33 op/s 2024-04-23T10:42:11.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:42:11 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/1651317742' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:42:11.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:42:11 smithi196 ceph-mon[32919]: osdmap e109: 12 total, 12 up, 12 in 2024-04-23T10:42:11.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:42:11 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/1651317742' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:42:11.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:42:11 smithi097 ceph-mon[29197]: osdmap e109: 12 total, 12 up, 12 in 2024-04-23T10:42:11.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:42:11 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/1651317742' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:42:11.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:42:11 smithi183 ceph-mon[32317]: osdmap e109: 12 total, 12 up, 12 in 2024-04-23T10:42:13.005 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.004+0000 7f7e57fa1640 1 -- 172.21.15.97:0/582595841 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e501010f0 msgr2=0x7f7e501014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:13.005 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.004+0000 7f7e57fa1640 1 --2- 172.21.15.97:0/582595841 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e501010f0 0x7f7e501014d0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f7e40009900 tx=0x7f7e4002a410 comp rx=0 tx=0).stop 2024-04-23T10:42:13.005 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.004+0000 7f7e57fa1640 1 -- 172.21.15.97:0/582595841 shutdown_connections 2024-04-23T10:42:13.005 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.004+0000 7f7e57fa1640 1 --2- 172.21.15.97:0/582595841 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e50106170 0x7f7e5010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:13.006 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.004+0000 7f7e57fa1640 1 --2- 172.21.15.97:0/582595841 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e50101aa0 0x7f7e50105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:13.006 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.004+0000 7f7e57fa1640 1 --2- 172.21.15.97:0/582595841 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e501010f0 0x7f7e501014d0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:13.006 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.004+0000 7f7e57fa1640 1 -- 172.21.15.97:0/582595841 >> 172.21.15.97:0/582595841 conn(0x7f7e50077fd0 msgr2=0x7f7e500783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:42:13.006 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.004+0000 7f7e57fa1640 1 -- 172.21.15.97:0/582595841 shutdown_connections 2024-04-23T10:42:13.006 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.004+0000 7f7e57fa1640 1 -- 172.21.15.97:0/582595841 wait complete. 2024-04-23T10:42:13.006 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.005+0000 7f7e57fa1640 1 Processor -- start 2024-04-23T10:42:13.006 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.005+0000 7f7e57fa1640 1 -- start start 2024-04-23T10:42:13.006 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.005+0000 7f7e57fa1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e501010f0 0x7f7e50198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:13.006 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.005+0000 7f7e57fa1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e50106170 0x7f7e50198cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:13.006 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.005+0000 7f7e57fa1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e5019cff0 0x7f7e5019d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:13.006 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.005+0000 7f7e57fa1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7e501110d0 con 0x7f7e5019cff0 2024-04-23T10:42:13.006 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.005+0000 7f7e57fa1640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7e50110f50 con 0x7f7e50106170 2024-04-23T10:42:13.006 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.005+0000 7f7e57fa1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7e50111250 con 0x7f7e501010f0 2024-04-23T10:42:13.006 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.005+0000 7f7e5653e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e5019cff0 0x7f7e5019d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:13.006 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.005+0000 7f7e5653e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e5019cff0 0x7f7e5019d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50634/0 (socket says 172.21.15.97:50634) 2024-04-23T10:42:13.007 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.005+0000 7f7e5653e640 1 -- 172.21.15.97:0/3371155995 learned_addr learned my addr 172.21.15.97:0/3371155995 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:42:13.007 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.005+0000 7f7e55d3d640 1 --2- 172.21.15.97:0/3371155995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e501010f0 0x7f7e50198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:13.007 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.005+0000 7f7e5553c640 1 --2- 172.21.15.97:0/3371155995 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e50106170 0x7f7e50198cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:13.007 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.005+0000 7f7e5653e640 1 -- 172.21.15.97:0/3371155995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e501010f0 msgr2=0x7f7e50198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:13.007 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.005+0000 7f7e5653e640 1 --2- 172.21.15.97:0/3371155995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e501010f0 0x7f7e50198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:13.007 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.005+0000 7f7e5653e640 1 -- 172.21.15.97:0/3371155995 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e50106170 msgr2=0x7f7e50198cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:13.007 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.005+0000 7f7e5653e640 1 --2- 172.21.15.97:0/3371155995 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e50106170 0x7f7e50198cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:13.007 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.005+0000 7f7e5653e640 1 -- 172.21.15.97:0/3371155995 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e5019dae0 con 0x7f7e5019cff0 2024-04-23T10:42:13.007 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.005+0000 7f7e5553c640 1 --2- 172.21.15.97:0/3371155995 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e50106170 0x7f7e50198cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:42:13.007 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.006+0000 7f7e5653e640 1 --2- 172.21.15.97:0/3371155995 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e5019cff0 0x7f7e5019d4a0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f7e4800eaa0 tx=0x7f7e4800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:42:13.007 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.006+0000 7f7e46ffd640 1 -- 172.21.15.97:0/3371155995 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e4800cdb0 con 0x7f7e5019cff0 2024-04-23T10:42:13.007 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.006+0000 7f7e57fa1640 1 -- 172.21.15.97:0/3371155995 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e5019ddd0 con 0x7f7e5019cff0 2024-04-23T10:42:13.007 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.006+0000 7f7e46ffd640 1 -- 172.21.15.97:0/3371155995 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e48005020 con 0x7f7e5019cff0 2024-04-23T10:42:13.007 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.006+0000 7f7e46ffd640 1 -- 172.21.15.97:0/3371155995 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e480116d0 con 0x7f7e5019cff0 2024-04-23T10:42:13.007 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.006+0000 7f7e57fa1640 1 -- 172.21.15.97:0/3371155995 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e501a5570 con 0x7f7e5019cff0 2024-04-23T10:42:13.014 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.009+0000 7f7e46ffd640 1 -- 172.21.15.97:0/3371155995 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7e480051d0 con 0x7f7e5019cff0 2024-04-23T10:42:13.014 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.009+0000 7f7e5ad8b740 1 Processor -- start 2024-04-23T10:42:13.014 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.009+0000 7f7e5ad8b740 1 -- start start 2024-04-23T10:42:13.014 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.009+0000 7f7e5ad8b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55677409c520 0x55677409c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:13.015 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.009+0000 7f7e5ad8b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55677409ce40 0x5567740ab6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:13.015 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.009+0000 7f7e5ad8b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5567740abc10 0x5567740ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:13.015 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.009+0000 7f7e5ad8b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x556774001970 con 0x55677409c520 2024-04-23T10:42:13.015 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.009+0000 7f7e5ad8b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556773f9e4a0 con 0x55677409ce40 2024-04-23T10:42:13.015 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.009+0000 7f7e5ad8b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x556773ffd930 con 0x5567740abc10 2024-04-23T10:42:13.015 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.009+0000 7f7e5553c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55677409c520 0x55677409c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:13.015 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.009+0000 7f7e5553c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55677409c520 0x55677409c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50646/0 (socket says 172.21.15.97:50646) 2024-04-23T10:42:13.015 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.009+0000 7f7e5553c640 1 -- 172.21.15.97:0/625995947 learned_addr learned my addr 172.21.15.97:0/625995947 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:42:13.015 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.010+0000 7f7e46ffd640 1 --2- 172.21.15.97:0/3371155995 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f7e3c0a3dc0 0x7f7e3c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:13.015 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.010+0000 7f7e55d3d640 1 --2- 172.21.15.97:0/625995947 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55677409ce40 0x5567740ab6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:13.015 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.010+0000 7f7e5553c640 1 --2- 172.21.15.97:0/625995947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5567740abc10 0x5567740ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:13.015 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.010+0000 7f7e5553c640 1 -- 172.21.15.97:0/625995947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5567740abc10 msgr2=0x5567740ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:42:13.015 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.010+0000 7f7e5553c640 1 --2- 172.21.15.97:0/625995947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5567740abc10 0x5567740ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:13.015 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.010+0000 7f7e5553c640 1 -- 172.21.15.97:0/625995947 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55677409ce40 msgr2=0x5567740ab6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:42:13.015 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.010+0000 7f7e5553c640 1 --2- 172.21.15.97:0/625995947 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55677409ce40 0x5567740ab6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:13.016 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.010+0000 7f7e5553c640 1 -- 172.21.15.97:0/625995947 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556773d14210 con 0x55677409c520 2024-04-23T10:42:13.016 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.010+0000 7f7e5653e640 1 --2- 172.21.15.97:0/3371155995 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f7e3c0a3dc0 0x7f7e3c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:13.016 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.010+0000 7f7e46ffd640 1 -- 172.21.15.97:0/3371155995 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8484+0+0 (secure 0 0 0) 0x7f7e480d0a30 con 0x7f7e5019cff0 2024-04-23T10:42:13.016 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.010+0000 7f7e5553c640 1 --2- 172.21.15.97:0/625995947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55677409c520 0x55677409c900 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c013570 tx=0x7f7e4c0097a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b88544a408a88dde server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:42:13.016 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.010+0000 7f7e3bfff640 1 -- 172.21.15.97:0/625995947 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c011470 con 0x55677409c520 2024-04-23T10:42:13.016 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.010+0000 7f7e3bfff640 1 -- 172.21.15.97:0/625995947 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e4c011ac0 con 0x55677409c520 2024-04-23T10:42:13.016 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.010+0000 7f7e3bfff640 1 -- 172.21.15.97:0/625995947 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c00f880 con 0x55677409c520 2024-04-23T10:42:13.016 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.011+0000 7f7e5ad8b740 1 -- 172.21.15.97:0/625995947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55677409c520 msgr2=0x55677409c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:13.016 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.011+0000 7f7e5653e640 1 --2- 172.21.15.97:0/3371155995 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f7e3c0a3dc0 0x7f7e3c0a6280 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7e480092f0 tx=0x7f7e4806b3a0 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:42:13.016 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.011+0000 7f7e5ad8b740 1 --2- 172.21.15.97:0/625995947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55677409c520 0x55677409c900 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c013570 tx=0x7f7e4c0097a0 comp rx=0 tx=0).stop 2024-04-23T10:42:13.016 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.011+0000 7f7e5ad8b740 1 -- 172.21.15.97:0/625995947 shutdown_connections 2024-04-23T10:42:13.016 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.011+0000 7f7e5ad8b740 1 --2- 172.21.15.97:0/625995947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5567740abc10 0x5567740ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:13.016 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.011+0000 7f7e5ad8b740 1 --2- 172.21.15.97:0/625995947 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55677409ce40 0x5567740ab6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:13.016 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.011+0000 7f7e5ad8b740 1 --2- 172.21.15.97:0/625995947 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55677409c520 0x55677409c900 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:13.016 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.011+0000 7f7e5ad8b740 1 -- 172.21.15.97:0/625995947 >> 172.21.15.97:0/625995947 conn(0x556774098b30 msgr2=0x556774098f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:42:13.017 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.011+0000 7f7e5ad8b740 1 -- 172.21.15.97:0/625995947 shutdown_connections 2024-04-23T10:42:13.017 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.011+0000 7f7e5ad8b740 1 -- 172.21.15.97:0/625995947 wait complete. 2024-04-23T10:42:13.017 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.011+0000 7f7e5ad8b740 1 -- start start 2024-04-23T10:42:13.017 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.011+0000 7f7e5ad8b740 1 Processor -- start 2024-04-23T10:42:13.017 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.012+0000 7f7e5ad8b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55677409c520 0x5567740ab490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:13.017 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.012+0000 7f7e5ad8b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55677409ce40 0x5567740a8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:13.017 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.012+0000 7f7e5ad8b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5567740abc10 0x5567740a8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:13.017 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.012+0000 7f7e5ad8b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5567740ae840 con 0x55677409c520 2024-04-23T10:42:13.017 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.012+0000 7f7e5ad8b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556773f9c0b0 con 0x55677409ce40 2024-04-23T10:42:13.017 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.012+0000 7f7e5ad8b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x5567740ae9c0 con 0x5567740abc10 2024-04-23T10:42:13.017 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.012+0000 7f7e5ad8b740 10 client.0 _ll_register_callbacks cb 0x556773f89d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:42:13.017 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.012+0000 7f7e5553c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55677409c520 0x5567740ab490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:13.017 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.012+0000 7f7e5553c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55677409c520 0x5567740ab490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50662/0 (socket says 172.21.15.97:50662) 2024-04-23T10:42:13.017 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.012+0000 7f7e5553c640 1 -- 172.21.15.97:0/1534397037 learned_addr learned my addr 172.21.15.97:0/1534397037 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:42:13.017 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.012+0000 7f7e55d3d640 1 --2- 172.21.15.97:0/1534397037 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55677409ce40 0x5567740a8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:13.018 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.012+0000 7f7e5553c640 1 --2- 172.21.15.97:0/1534397037 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5567740abc10 0x5567740a8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:13.018 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.012+0000 7f7e5553c640 1 -- 172.21.15.97:0/1534397037 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5567740abc10 msgr2=0x5567740a8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:42:13.018 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.012+0000 7f7e5553c640 1 --2- 172.21.15.97:0/1534397037 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5567740abc10 0x5567740a8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:13.018 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.012+0000 7f7e5553c640 1 -- 172.21.15.97:0/1534397037 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55677409ce40 msgr2=0x5567740a8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:42:13.018 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.012+0000 7f7e5553c640 1 --2- 172.21.15.97:0/1534397037 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55677409ce40 0x5567740a8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:13.018 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.012+0000 7f7e5553c640 1 -- 172.21.15.97:0/1534397037 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556773de0620 con 0x55677409c520 2024-04-23T10:42:13.018 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.012+0000 7f7e55d3d640 1 --2- 172.21.15.97:0/1534397037 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55677409ce40 0x5567740a8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:42:13.018 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.012+0000 7f7e5553c640 1 --2- 172.21.15.97:0/1534397037 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55677409c520 0x5567740ab490 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c009ec0 tx=0x7f7e4c00f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88ebbbe3354bd09b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:42:13.018 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.012+0000 7f7e397fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:42:13.018 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.013+0000 7f7e397fa640 1 -- 172.21.15.97:0/1534397037 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c005240 con 0x55677409c520 2024-04-23T10:42:13.018 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.013+0000 7f7e397fa640 1 -- 172.21.15.97:0/1534397037 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8484+0+0 (secure 0 0 0) 0x7f7e4c0053e0 con 0x55677409c520 2024-04-23T10:42:13.018 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.013+0000 7f7e5ad8b740 1 -- 172.21.15.97:0/1534397037 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e4c011ac0 con 0x55677409c520 2024-04-23T10:42:13.018 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.013+0000 7f7e397fa640 1 -- 172.21.15.97:0/1534397037 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7e4c0098d0 con 0x55677409c520 2024-04-23T10:42:13.018 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.013+0000 7f7e5ad8b740 10 client.17811 fetch_fsmap learned FSMap version 26 2024-04-23T10:42:13.019 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.013+0000 7f7e5ad8b740 1 -- 172.21.15.97:0/1534397037 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5567740ba710 con 0x55677409c520 2024-04-23T10:42:13.019 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.013+0000 7f7e397fa640 1 -- 172.21.15.97:0/1534397037 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e4c011ac0 con 0x55677409c520 2024-04-23T10:42:13.019 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.013+0000 7f7e397fa640 1 -- 172.21.15.97:0/1534397037 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f7e4c035020 con 0x55677409c520 2024-04-23T10:42:13.019 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.013+0000 7f7e5ad8b740 10 client.17811 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:42:13.019 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.013+0000 7f7e5ad8b740 10 client.17811 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:42:13.019 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.013+0000 7f7e5ad8b740 20 client.17811 populate_metadata read hostname 'smithi097' 2024-04-23T10:42:13.019 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.013+0000 7f7e5ad8b740 1 --2- 172.21.15.97:0/1534397037 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5567740bbb20 0x5567740bdf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:13.019 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.013+0000 7f7e5ad8b740 4 client.17811 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:42:13.019 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.013+0000 7f7e5ad8b740 1 -- 172.21.15.97:0/1534397037 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55677400f390 con 0x5567740bbb20 2024-04-23T10:42:13.019 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.014+0000 7f7e55d3d640 1 --2- 172.21.15.97:0/1534397037 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5567740bbb20 0x5567740bdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:13.019 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.014+0000 7f7e397fa640 10 client.17811 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:42:13.019 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.014+0000 7f7e55d3d640 1 --2- 172.21.15.97:0/1534397037 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5567740bbb20 0x5567740bdf10 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5eb29190c01f3f3 server_cookie=7de15ef8b6c0a858 in_seq=0 out_seq=0 2024-04-23T10:42:13.019 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.015+0000 7f7e397fa640 1 -- 172.21.15.97:0/1534397037 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55677400f390 con 0x5567740bbb20 2024-04-23T10:42:13.019 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.015+0000 7f7e397fa640 10 client.17811 handle_command_reply: tid=0 2024-04-23T10:42:13.020 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.015+0000 7f7e5ad8b740 1 client.17811 shutdown 2024-04-23T10:42:13.020 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.015+0000 7f7e5ad8b740 1 -- 172.21.15.97:0/1534397037 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55677409c520 msgr2=0x5567740ab490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:13.020 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.015+0000 7f7e5ad8b740 1 --2- 172.21.15.97:0/1534397037 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55677409c520 0x5567740ab490 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c009ec0 tx=0x7f7e4c00f4a0 comp rx=0 tx=0).stop 2024-04-23T10:42:13.020 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.015+0000 7f7e5ad8b740 1 -- 172.21.15.97:0/1534397037 shutdown_connections 2024-04-23T10:42:13.020 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.015+0000 7f7e5ad8b740 1 --2- 172.21.15.97:0/1534397037 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5567740bbb20 0x5567740bdf10 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:13.020 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.015+0000 7f7e5ad8b740 1 --2- 172.21.15.97:0/1534397037 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5567740abc10 0x5567740a8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:13.020 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.015+0000 7f7e5ad8b740 1 --2- 172.21.15.97:0/1534397037 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55677409ce40 0x5567740a8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:13.020 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.015+0000 7f7e5ad8b740 1 --2- 172.21.15.97:0/1534397037 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55677409c520 0x5567740ab490 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:13.020 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.015+0000 7f7e397fa640 0 client.17811 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:42:13.020 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.015+0000 7f7e5ad8b740 1 -- 172.21.15.97:0/1534397037 >> 172.21.15.97:0/1534397037 conn(0x556774098b30 msgr2=0x55677409e970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:42:13.020 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.015+0000 7f7e5ad8b740 1 -- 172.21.15.97:0/1534397037 shutdown_connections 2024-04-23T10:42:13.020 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.015+0000 7f7e5ad8b740 1 -- 172.21.15.97:0/1534397037 wait complete. 2024-04-23T10:42:13.020 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.015+0000 7f7e5ad8b740 20 client.17811 trim_cache size 0 max 16384 2024-04-23T10:42:13.034 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.030+0000 7f7e5ad8b740 1 Processor -- start 2024-04-23T10:42:13.034 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.030+0000 7f7e5ad8b740 1 -- start start 2024-04-23T10:42:13.034 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.030+0000 7f7e5ad8b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556774098b30 0x5567740a7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:13.035 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.030+0000 7f7e5ad8b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55677409c520 0x5567740aafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:13.035 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.030+0000 7f7e5ad8b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55677409ce40 0x5567740c3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:13.035 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.030+0000 7f7e5ad8b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7e501ab850 con 0x55677409ce40 2024-04-23T10:42:13.035 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.030+0000 7f7e5ad8b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7e501ad390 con 0x55677409c520 2024-04-23T10:42:13.035 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.030+0000 7f7e5ad8b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7e501abf20 con 0x556774098b30 2024-04-23T10:42:13.035 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.031+0000 7f7e5553c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55677409ce40 0x5567740c3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:13.035 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.031+0000 7f7e5553c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55677409ce40 0x5567740c3650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50672/0 (socket says 172.21.15.97:50672) 2024-04-23T10:42:13.035 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.031+0000 7f7e5553c640 1 -- 172.21.15.97:0/3783617084 learned_addr learned my addr 172.21.15.97:0/3783617084 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:42:13.035 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.031+0000 7f7e5553c640 1 --2- 172.21.15.97:0/3783617084 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556774098b30 0x5567740a7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:13.035 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.031+0000 7f7e55d3d640 1 --2- 172.21.15.97:0/3783617084 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55677409c520 0x5567740aafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:13.035 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.031+0000 7f7e5553c640 1 -- 172.21.15.97:0/3783617084 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556774098b30 msgr2=0x5567740a7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:42:13.035 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.031+0000 7f7e5553c640 1 --2- 172.21.15.97:0/3783617084 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556774098b30 0x5567740a7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:13.035 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.031+0000 7f7e5553c640 1 -- 172.21.15.97:0/3783617084 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55677409c520 msgr2=0x5567740aafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:42:13.035 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.031+0000 7f7e5553c640 1 --2- 172.21.15.97:0/3783617084 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55677409c520 0x5567740aafc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:13.036 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.031+0000 7f7e5553c640 1 -- 172.21.15.97:0/3783617084 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55677409d380 con 0x55677409ce40 2024-04-23T10:42:13.036 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.031+0000 7f7e5553c640 1 --2- 172.21.15.97:0/3783617084 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55677409ce40 0x5567740c3650 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c00fd90 tx=0x7f7e4c016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=318efb417de7db91 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:42:13.036 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.031+0000 7f7e3b7fe640 1 -- 172.21.15.97:0/3783617084 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c00fe10 con 0x55677409ce40 2024-04-23T10:42:13.036 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.031+0000 7f7e3b7fe640 1 -- 172.21.15.97:0/3783617084 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e4c029040 con 0x55677409ce40 2024-04-23T10:42:13.036 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.031+0000 7f7e3b7fe640 1 -- 172.21.15.97:0/3783617084 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c0052a0 con 0x55677409ce40 2024-04-23T10:42:13.036 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.032+0000 7f7e5ad8b740 1 -- 172.21.15.97:0/3783617084 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55677409ce40 msgr2=0x5567740c3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:13.036 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.032+0000 7f7e5ad8b740 1 --2- 172.21.15.97:0/3783617084 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55677409ce40 0x5567740c3650 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c00fd90 tx=0x7f7e4c016040 comp rx=0 tx=0).stop 2024-04-23T10:42:13.036 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.032+0000 7f7e5ad8b740 1 -- 172.21.15.97:0/3783617084 shutdown_connections 2024-04-23T10:42:13.036 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.032+0000 7f7e5ad8b740 1 --2- 172.21.15.97:0/3783617084 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55677409ce40 0x5567740c3650 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:13.036 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.032+0000 7f7e5ad8b740 1 --2- 172.21.15.97:0/3783617084 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55677409c520 0x5567740aafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:13.036 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.032+0000 7f7e5ad8b740 1 --2- 172.21.15.97:0/3783617084 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556774098b30 0x5567740a7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:13.036 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.032+0000 7f7e5ad8b740 1 -- 172.21.15.97:0/3783617084 >> 172.21.15.97:0/3783617084 conn(0x5567740a8e20 msgr2=0x556774097cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:42:13.036 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.032+0000 7f7e5ad8b740 1 -- 172.21.15.97:0/3783617084 shutdown_connections 2024-04-23T10:42:13.036 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.032+0000 7f7e5ad8b740 1 -- 172.21.15.97:0/3783617084 wait complete. 2024-04-23T10:42:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.032+0000 7f7e5ad8b740 1 -- start start 2024-04-23T10:42:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.032+0000 7f7e5ad8b740 1 Processor -- start 2024-04-23T10:42:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.032+0000 7f7e5ad8b740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556774098b30 0x5567740b3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.033+0000 7f7e5ad8b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55677409c520 0x5567740a5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.033+0000 7f7e5ad8b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55677409ce40 0x5567740a5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.033+0000 7f7e5ad8b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7e501accc0 con 0x55677409c520 2024-04-23T10:42:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.033+0000 7f7e5ad8b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7e501ac5f0 con 0x55677409ce40 2024-04-23T10:42:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.033+0000 7f7e5ad8b740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7e501a9850 con 0x556774098b30 2024-04-23T10:42:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.033+0000 7f7e5ad8b740 10 client.0 _ll_register_callbacks cb 0x7f7e4c011990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:42:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.033+0000 7f7e55d3d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55677409c520 0x5567740a5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.033+0000 7f7e55d3d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55677409c520 0x5567740a5010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:50686/0 (socket says 172.21.15.97:50686) 2024-04-23T10:42:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.033+0000 7f7e55d3d640 1 -- 172.21.15.97:0/3235814529 learned_addr learned my addr 172.21.15.97:0/3235814529 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:42:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.033+0000 7f7e5553c640 1 --2- 172.21.15.97:0/3235814529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556774098b30 0x5567740b3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.033+0000 7f7e5553c640 1 --2- 172.21.15.97:0/3235814529 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55677409ce40 0x5567740a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.033+0000 7f7e55d3d640 1 -- 172.21.15.97:0/3235814529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556774098b30 msgr2=0x5567740b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:42:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.033+0000 7f7e55d3d640 1 --2- 172.21.15.97:0/3235814529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556774098b30 0x5567740b3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.033+0000 7f7e55d3d640 1 -- 172.21.15.97:0/3235814529 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55677409ce40 msgr2=0x5567740a5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:42:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.033+0000 7f7e55d3d640 1 --2- 172.21.15.97:0/3235814529 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55677409ce40 0x5567740a5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.033+0000 7f7e55d3d640 1 -- 172.21.15.97:0/3235814529 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556774098960 con 0x55677409c520 2024-04-23T10:42:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.033+0000 7f7e5553c640 1 --2- 172.21.15.97:0/3235814529 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55677409ce40 0x5567740a5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:42:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.033+0000 7f7e55d3d640 1 --2- 172.21.15.97:0/3235814529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55677409c520 0x5567740a5010 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f7e4002a450 tx=0x7f7e400095b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=229c950c2188e045 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:42:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.033+0000 7f7e397fa640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:42:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.033+0000 7f7e397fa640 1 -- 172.21.15.97:0/3235814529 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e40035050 con 0x55677409c520 2024-04-23T10:42:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.033+0000 7f7e397fa640 1 -- 172.21.15.97:0/3235814529 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8484+0+0 (secure 0 0 0) 0x7f7e40043bc0 con 0x55677409c520 2024-04-23T10:42:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.033+0000 7f7e5ad8b740 1 -- 172.21.15.97:0/3235814529 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e4c029040 con 0x55677409c520 2024-04-23T10:42:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.033+0000 7f7e397fa640 1 -- 172.21.15.97:0/3235814529 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7e40043dc0 con 0x55677409c520 2024-04-23T10:42:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.033+0000 7f7e5ad8b740 10 client.17817 fetch_fsmap learned FSMap version 26 2024-04-23T10:42:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.033+0000 7f7e5ad8b740 1 -- 172.21.15.97:0/3235814529 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556774121c50 con 0x55677409c520 2024-04-23T10:42:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.034+0000 7f7e397fa640 1 -- 172.21.15.97:0/3235814529 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e4c029040 con 0x55677409c520 2024-04-23T10:42:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.034+0000 7f7e397fa640 1 -- 172.21.15.97:0/3235814529 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f7e40005420 con 0x55677409c520 2024-04-23T10:42:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.034+0000 7f7e5ad8b740 10 client.17817 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:42:13.039 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.034+0000 7f7e5ad8b740 10 client.17817 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:42:13.039 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.034+0000 7f7e5ad8b740 20 client.17817 populate_metadata read hostname 'smithi097' 2024-04-23T10:42:13.039 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.034+0000 7f7e5ad8b740 1 --2- 172.21.15.97:0/3235814529 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5567740abc10 0x5567741243f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:13.039 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.034+0000 7f7e5ad8b740 4 client.17817 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:42:13.039 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.034+0000 7f7e5ad8b740 1 -- 172.21.15.97:0/3235814529 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556774009200 con 0x5567740abc10 2024-04-23T10:42:13.039 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.034+0000 7f7e5553c640 1 --2- 172.21.15.97:0/3235814529 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5567740abc10 0x5567741243f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:13.039 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.034+0000 7f7e5553c640 1 --2- 172.21.15.97:0/3235814529 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5567740abc10 0x5567741243f0 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ff8327b1f966b2 server_cookie=468c23691aa24650 in_seq=0 out_seq=0 2024-04-23T10:42:13.039 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.034+0000 7f7e397fa640 10 client.17817 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:42:13.045 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.045+0000 7f7e397fa640 1 -- 172.21.15.97:0/3235814529 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556774009200 con 0x5567740abc10 2024-04-23T10:42:13.045 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.045+0000 7f7e397fa640 10 client.17817 handle_command_reply: tid=0 2024-04-23T10:42:13.045 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.045+0000 7f7e5ad8b740 1 client.17817 shutdown 2024-04-23T10:42:13.045 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.046+0000 7f7e5ad8b740 1 -- 172.21.15.97:0/3235814529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55677409c520 msgr2=0x5567740a5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:13.045 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.046+0000 7f7e5ad8b740 1 --2- 172.21.15.97:0/3235814529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55677409c520 0x5567740a5010 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f7e4002a450 tx=0x7f7e400095b0 comp rx=0 tx=0).stop 2024-04-23T10:42:13.045 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.046+0000 7f7e5ad8b740 1 -- 172.21.15.97:0/3235814529 shutdown_connections 2024-04-23T10:42:13.045 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.046+0000 7f7e5ad8b740 1 --2- 172.21.15.97:0/3235814529 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x5567740abc10 0x5567741243f0 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:13.045 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.046+0000 7f7e5ad8b740 1 --2- 172.21.15.97:0/3235814529 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55677409ce40 0x5567740a5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:13.045 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.046+0000 7f7e5ad8b740 1 --2- 172.21.15.97:0/3235814529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55677409c520 0x5567740a5010 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:13.045 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.046+0000 7f7e397fa640 0 client.17817 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:42:13.045 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.046+0000 7f7e5ad8b740 1 --2- 172.21.15.97:0/3235814529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556774098b30 0x5567740b3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:13.045 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.046+0000 7f7e5ad8b740 1 -- 172.21.15.97:0/3235814529 >> 172.21.15.97:0/3235814529 conn(0x5567740a8e20 msgr2=0x55677409e070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:42:13.045 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.046+0000 7f7e5ad8b740 1 -- 172.21.15.97:0/3235814529 shutdown_connections 2024-04-23T10:42:13.046 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.046+0000 7f7e5ad8b740 1 -- 172.21.15.97:0/3235814529 wait complete. 2024-04-23T10:42:13.046 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.046+0000 7f7e5ad8b740 20 client.17817 trim_cache size 0 max 16384 2024-04-23T10:42:13.046 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:42:13.046 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:42:13.046 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:42:13.046 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:42:13.047 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.046+0000 7f7e57fa1640 1 -- 172.21.15.97:0/3371155995 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f7e3c0a3dc0 msgr2=0x7f7e3c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:13.047 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.046+0000 7f7e57fa1640 1 --2- 172.21.15.97:0/3371155995 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f7e3c0a3dc0 0x7f7e3c0a6280 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7e480092f0 tx=0x7f7e4806b3a0 comp rx=0 tx=0).stop 2024-04-23T10:42:13.047 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.046+0000 7f7e57fa1640 1 -- 172.21.15.97:0/3371155995 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e5019cff0 msgr2=0x7f7e5019d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:13.047 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.046+0000 7f7e57fa1640 1 --2- 172.21.15.97:0/3371155995 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e5019cff0 0x7f7e5019d4a0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f7e4800eaa0 tx=0x7f7e4800ef60 comp rx=0 tx=0).stop 2024-04-23T10:42:13.047 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.046+0000 7f7e57fa1640 1 -- 172.21.15.97:0/3371155995 shutdown_connections 2024-04-23T10:42:13.047 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.046+0000 7f7e57fa1640 1 --2- 172.21.15.97:0/3371155995 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f7e3c0a3dc0 0x7f7e3c0a6280 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:13.047 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.046+0000 7f7e57fa1640 1 --2- 172.21.15.97:0/3371155995 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7e5019cff0 0x7f7e5019d4a0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:13.047 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.046+0000 7f7e57fa1640 1 --2- 172.21.15.97:0/3371155995 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e50106170 0x7f7e50198cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:13.047 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.046+0000 7f7e57fa1640 1 --2- 172.21.15.97:0/3371155995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e501010f0 0x7f7e50198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:13.048 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.046+0000 7f7e57fa1640 1 -- 172.21.15.97:0/3371155995 >> 172.21.15.97:0/3371155995 conn(0x7f7e50077fd0 msgr2=0x7f7e50069a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:42:13.048 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.047+0000 7f7e57fa1640 1 -- 172.21.15.97:0/3371155995 shutdown_connections 2024-04-23T10:42:13.048 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:13.047+0000 7f7e57fa1640 1 -- 172.21.15.97:0/3371155995 wait complete. 2024-04-23T10:42:13.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:42:13 smithi196 ceph-mon[32919]: pgmap v503: 97 pgs: 97 active+clean; 3.2 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-23T10:42:13.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:42:13 smithi097 ceph-mon[29197]: pgmap v503: 97 pgs: 97 active+clean; 3.2 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-23T10:42:13.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:42:13 smithi183 ceph-mon[32317]: pgmap v503: 97 pgs: 97 active+clean; 3.2 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 127 B/s wr, 6 op/s 2024-04-23T10:42:15.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:42:15 smithi196 ceph-mon[32919]: pgmap v504: 97 pgs: 97 active+clean; 3.3 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-04-23T10:42:15.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:42:15 smithi097 ceph-mon[29197]: pgmap v504: 97 pgs: 97 active+clean; 3.3 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-04-23T10:42:15.649 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:42:15 smithi183 ceph-mon[32317]: pgmap v504: 97 pgs: 97 active+clean; 3.3 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-04-23T10:42:17.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:42:17 smithi196 ceph-mon[32919]: pgmap v505: 97 pgs: 97 active+clean; 3.3 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-04-23T10:42:17.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:42:17 smithi097 ceph-mon[29197]: pgmap v505: 97 pgs: 97 active+clean; 3.3 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-04-23T10:42:17.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:42:17 smithi183 ceph-mon[32317]: pgmap v505: 97 pgs: 97 active+clean; 3.3 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-04-23T10:42:19.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:42:19 smithi196 ceph-mon[32919]: pgmap v506: 97 pgs: 97 active+clean; 3.5 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 35 KiB/s wr, 16 op/s 2024-04-23T10:42:19.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:42:19 smithi097 ceph-mon[29197]: pgmap v506: 97 pgs: 97 active+clean; 3.5 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 35 KiB/s wr, 16 op/s 2024-04-23T10:42:19.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:42:19 smithi183 ceph-mon[32317]: pgmap v506: 97 pgs: 97 active+clean; 3.5 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 35 KiB/s wr, 16 op/s 2024-04-23T10:42:21.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:42:21 smithi196 ceph-mon[32919]: pgmap v507: 97 pgs: 97 active+clean; 3.5 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 35 KiB/s wr, 16 op/s 2024-04-23T10:42:21.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:42:21 smithi097 ceph-mon[29197]: pgmap v507: 97 pgs: 97 active+clean; 3.5 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 35 KiB/s wr, 16 op/s 2024-04-23T10:42:21.649 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:42:21 smithi183 ceph-mon[32317]: pgmap v507: 97 pgs: 97 active+clean; 3.5 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 35 KiB/s wr, 16 op/s 2024-04-23T10:42:23.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:42:23 smithi196 ceph-mon[32919]: pgmap v508: 97 pgs: 97 active+clean; 3.5 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 35 KiB/s wr, 16 op/s 2024-04-23T10:42:23.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:42:23 smithi097 ceph-mon[29197]: pgmap v508: 97 pgs: 97 active+clean; 3.5 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 35 KiB/s wr, 16 op/s 2024-04-23T10:42:23.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:42:23 smithi183 ceph-mon[32317]: pgmap v508: 97 pgs: 97 active+clean; 3.5 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 35 KiB/s wr, 16 op/s 2024-04-23T10:42:25.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:42:25 smithi196 ceph-mon[32919]: pgmap v509: 97 pgs: 97 active+clean; 3.7 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 47 KiB/s wr, 20 op/s 2024-04-23T10:42:25.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:42:25 smithi097 ceph-mon[29197]: pgmap v509: 97 pgs: 97 active+clean; 3.7 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 47 KiB/s wr, 20 op/s 2024-04-23T10:42:25.649 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:42:25 smithi183 ceph-mon[32317]: pgmap v509: 97 pgs: 97 active+clean; 3.7 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 47 KiB/s wr, 20 op/s 2024-04-23T10:42:26.144 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.143+0000 7f6b98fcd640 1 -- 172.21.15.97:0/653859608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b941010e0 msgr2=0x7f6b941014c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:26.144 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.143+0000 7f6b98fcd640 1 --2- 172.21.15.97:0/653859608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b941010e0 0x7f6b941014c0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c009900 tx=0x7f6b7c02a410 comp rx=0 tx=0).stop 2024-04-23T10:42:26.145 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.143+0000 7f6b98fcd640 1 -- 172.21.15.97:0/653859608 shutdown_connections 2024-04-23T10:42:26.145 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.143+0000 7f6b98fcd640 1 --2- 172.21.15.97:0/653859608 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b941061b0 0x7f6b9410dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:26.145 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.143+0000 7f6b98fcd640 1 --2- 172.21.15.97:0/653859608 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b94101a90 0x7f6b94105ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:26.145 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.143+0000 7f6b98fcd640 1 --2- 172.21.15.97:0/653859608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b941010e0 0x7f6b941014c0 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:26.145 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.143+0000 7f6b98fcd640 1 -- 172.21.15.97:0/653859608 >> 172.21.15.97:0/653859608 conn(0x7f6b940771d0 msgr2=0x7f6b940775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:42:26.145 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.143+0000 7f6b98fcd640 1 -- 172.21.15.97:0/653859608 shutdown_connections 2024-04-23T10:42:26.145 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.143+0000 7f6b98fcd640 1 -- 172.21.15.97:0/653859608 wait complete. 2024-04-23T10:42:26.145 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.144+0000 7f6b98fcd640 1 Processor -- start 2024-04-23T10:42:26.145 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.144+0000 7f6b98fcd640 1 -- start start 2024-04-23T10:42:26.145 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.144+0000 7f6b98fcd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b941010e0 0x7f6b941986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:26.145 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.144+0000 7f6b98fcd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b941061b0 0x7f6b94198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:26.145 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.144+0000 7f6b98fcd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b9419cfa0 0x7f6b9419d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:26.145 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.144+0000 7f6b98fcd640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6b941110f0 con 0x7f6b941010e0 2024-04-23T10:42:26.146 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.144+0000 7f6b98fcd640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6b94110f70 con 0x7f6b9419cfa0 2024-04-23T10:42:26.146 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.144+0000 7f6b98fcd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6b94111270 con 0x7f6b941061b0 2024-04-23T10:42:26.146 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.144+0000 7f6b9259c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b941010e0 0x7f6b941986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:26.146 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.145+0000 7f6b9259c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b941010e0 0x7f6b941986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43018/0 (socket says 172.21.15.97:43018) 2024-04-23T10:42:26.146 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.145+0000 7f6b9259c640 1 -- 172.21.15.97:0/942913466 learned_addr learned my addr 172.21.15.97:0/942913466 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:42:26.146 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.145+0000 7f6b91d9b640 1 --2- 172.21.15.97:0/942913466 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b941061b0 0x7f6b94198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:26.146 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.145+0000 7f6b92d9d640 1 --2- 172.21.15.97:0/942913466 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b9419cfa0 0x7f6b9419d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:26.146 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.145+0000 7f6b9259c640 1 -- 172.21.15.97:0/942913466 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b941061b0 msgr2=0x7f6b94198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:26.146 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.145+0000 7f6b9259c640 1 --2- 172.21.15.97:0/942913466 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b941061b0 0x7f6b94198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:26.146 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.145+0000 7f6b9259c640 1 -- 172.21.15.97:0/942913466 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b9419cfa0 msgr2=0x7f6b9419d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:26.147 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.145+0000 7f6b9259c640 1 --2- 172.21.15.97:0/942913466 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b9419cfa0 0x7f6b9419d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:26.147 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.145+0000 7f6b9259c640 1 -- 172.21.15.97:0/942913466 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b9419da90 con 0x7f6b941010e0 2024-04-23T10:42:26.147 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.145+0000 7f6b92d9d640 1 --2- 172.21.15.97:0/942913466 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b9419cfa0 0x7f6b9419d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:42:26.147 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.145+0000 7f6b9259c640 1 --2- 172.21.15.97:0/942913466 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b941010e0 0x7f6b941986e0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c005940 tx=0x7f6b7c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:42:26.147 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.145+0000 7f6b8b7fe640 1 -- 172.21.15.97:0/942913466 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c03f030 con 0x7f6b941010e0 2024-04-23T10:42:26.147 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.145+0000 7f6b8b7fe640 1 -- 172.21.15.97:0/942913466 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b7c003aa0 con 0x7f6b941010e0 2024-04-23T10:42:26.147 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.145+0000 7f6b98fcd640 1 -- 172.21.15.97:0/942913466 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b9419dd20 con 0x7f6b941010e0 2024-04-23T10:42:26.147 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.145+0000 7f6b8b7fe640 1 -- 172.21.15.97:0/942913466 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c035070 con 0x7f6b941010e0 2024-04-23T10:42:26.147 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.145+0000 7f6b98fcd640 1 -- 172.21.15.97:0/942913466 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b941a54e0 con 0x7f6b941010e0 2024-04-23T10:42:26.154 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.147+0000 7f6b8b7fe640 1 -- 172.21.15.97:0/942913466 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6b7c042030 con 0x7f6b941010e0 2024-04-23T10:42:26.154 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.148+0000 7f6b9bdc3740 1 Processor -- start 2024-04-23T10:42:26.154 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.148+0000 7f6b9bdc3740 1 -- start start 2024-04-23T10:42:26.154 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.148+0000 7f6b9bdc3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55968a4fd620 0x55968a4fda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:26.154 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.148+0000 7f6b9bdc3740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55968a4fdf40 0x55968a5082c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:26.154 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.148+0000 7f6b9bdc3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55968a508800 0x55968a50abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:26.154 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.148+0000 7f6b9bdc3740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55968a462970 con 0x55968a4fdf40 2024-04-23T10:42:26.154 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.148+0000 7f6b9bdc3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55968a3ff4a0 con 0x55968a508800 2024-04-23T10:42:26.154 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.148+0000 7f6b9bdc3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55968a45e930 con 0x55968a4fd620 2024-04-23T10:42:26.154 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.148+0000 7f6b8b7fe640 1 --2- 172.21.15.97:0/942913466 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f6b800a3dc0 0x7f6b800a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:26.154 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.148+0000 7f6b9259c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55968a4fdf40 0x55968a5082c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:26.154 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.148+0000 7f6b9259c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55968a4fdf40 0x55968a5082c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43026/0 (socket says 172.21.15.97:43026) 2024-04-23T10:42:26.155 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.148+0000 7f6b9259c640 1 -- 172.21.15.97:0/1402092079 learned_addr learned my addr 172.21.15.97:0/1402092079 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:42:26.155 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.148+0000 7f6b92d9d640 1 --2- 172.21.15.97:0/942913466 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f6b800a3dc0 0x7f6b800a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:26.155 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.149+0000 7f6b92d9d640 1 --2- 172.21.15.97:0/1402092079 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55968a4fd620 0x55968a4fda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:26.155 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.149+0000 7f6b91d9b640 1 --2- 172.21.15.97:0/1402092079 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55968a508800 0x55968a50abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:26.155 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.149+0000 7f6b8b7fe640 1 -- 172.21.15.97:0/942913466 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8484+0+0 (secure 0 0 0) 0x7f6b7c0f0ed0 con 0x7f6b941010e0 2024-04-23T10:42:26.155 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.149+0000 7f6b9259c640 1 -- 172.21.15.97:0/1402092079 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55968a4fd620 msgr2=0x55968a4fda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:42:26.155 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.149+0000 7f6b9259c640 1 --2- 172.21.15.97:0/1402092079 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55968a4fd620 0x55968a4fda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:26.155 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.149+0000 7f6b9259c640 1 -- 172.21.15.97:0/1402092079 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55968a508800 msgr2=0x55968a50abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:42:26.155 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.149+0000 7f6b9259c640 1 --2- 172.21.15.97:0/1402092079 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55968a508800 0x55968a50abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:26.155 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.149+0000 7f6b9259c640 1 -- 172.21.15.97:0/1402092079 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55968a175210 con 0x55968a4fdf40 2024-04-23T10:42:26.155 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.149+0000 7f6b91d9b640 1 --2- 172.21.15.97:0/1402092079 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55968a508800 0x55968a50abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:42:26.155 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.149+0000 7f6b92d9d640 1 --2- 172.21.15.97:0/942913466 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f6b800a3dc0 0x7f6b800a6280 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c005fd0 tx=0x7f6b8c0129e0 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:42:26.155 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.149+0000 7f6b9259c640 1 --2- 172.21.15.97:0/1402092079 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55968a4fdf40 0x55968a5082c0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c046000 tx=0x7f6b7c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0ad5e87497f6c16 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:42:26.156 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.149+0000 7f6b88ff9640 1 -- 172.21.15.97:0/1402092079 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c049070 con 0x55968a4fdf40 2024-04-23T10:42:26.156 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.149+0000 7f6b88ff9640 1 -- 172.21.15.97:0/1402092079 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b7c0bd590 con 0x55968a4fdf40 2024-04-23T10:42:26.156 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.149+0000 7f6b88ff9640 1 -- 172.21.15.97:0/1402092079 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c0b9500 con 0x55968a4fdf40 2024-04-23T10:42:26.156 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.150+0000 7f6b9bdc3740 1 -- 172.21.15.97:0/1402092079 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55968a4fdf40 msgr2=0x55968a5082c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:26.156 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.150+0000 7f6b9bdc3740 1 --2- 172.21.15.97:0/1402092079 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55968a4fdf40 0x55968a5082c0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c046000 tx=0x7f6b7c02a410 comp rx=0 tx=0).stop 2024-04-23T10:42:26.156 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.150+0000 7f6b9bdc3740 1 -- 172.21.15.97:0/1402092079 shutdown_connections 2024-04-23T10:42:26.156 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.150+0000 7f6b9bdc3740 1 --2- 172.21.15.97:0/1402092079 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55968a508800 0x55968a50abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:26.156 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.150+0000 7f6b9bdc3740 1 --2- 172.21.15.97:0/1402092079 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55968a4fdf40 0x55968a5082c0 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:26.156 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.150+0000 7f6b9bdc3740 1 --2- 172.21.15.97:0/1402092079 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55968a4fd620 0x55968a4fda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:26.156 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.150+0000 7f6b9bdc3740 1 -- 172.21.15.97:0/1402092079 >> 172.21.15.97:0/1402092079 conn(0x55968a4f9b30 msgr2=0x55968a4f9f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:42:26.156 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.150+0000 7f6b9bdc3740 1 -- 172.21.15.97:0/1402092079 shutdown_connections 2024-04-23T10:42:26.156 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.150+0000 7f6b9bdc3740 1 -- 172.21.15.97:0/1402092079 wait complete. 2024-04-23T10:42:26.156 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.150+0000 7f6b9bdc3740 1 -- start start 2024-04-23T10:42:26.156 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.151+0000 7f6b9bdc3740 1 Processor -- start 2024-04-23T10:42:26.156 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.151+0000 7f6b9bdc3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55968a4fd620 0x55968a508190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.151+0000 7f6b9bdc3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55968a4fdf40 0x55968a505480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.151+0000 7f6b9bdc3740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55968a508800 0x55968a505a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.151+0000 7f6b9bdc3740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55968a462970 con 0x55968a508800 2024-04-23T10:42:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.151+0000 7f6b9bdc3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55968a3fd0b0 con 0x55968a4fd620 2024-04-23T10:42:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.151+0000 7f6b9bdc3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55968a50b1f0 con 0x55968a4fdf40 2024-04-23T10:42:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.151+0000 7f6b9bdc3740 10 client.0 _ll_register_callbacks cb 0x55968a3ead30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:42:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.151+0000 7f6b92d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55968a508800 0x55968a505a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.151+0000 7f6b9259c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55968a4fd620 0x55968a508190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.151+0000 7f6b92d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55968a508800 0x55968a505a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43042/0 (socket says 172.21.15.97:43042) 2024-04-23T10:42:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.151+0000 7f6b92d9d640 1 -- 172.21.15.97:0/1264198586 learned_addr learned my addr 172.21.15.97:0/1264198586 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:42:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.151+0000 7f6b91d9b640 1 --2- 172.21.15.97:0/1264198586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55968a4fdf40 0x55968a505480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.151+0000 7f6b92d9d640 1 -- 172.21.15.97:0/1264198586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55968a4fdf40 msgr2=0x55968a505480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:42:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.151+0000 7f6b92d9d640 1 --2- 172.21.15.97:0/1264198586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55968a4fdf40 0x55968a505480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.151+0000 7f6b92d9d640 1 -- 172.21.15.97:0/1264198586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55968a4fd620 msgr2=0x55968a508190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:42:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.151+0000 7f6b92d9d640 1 --2- 172.21.15.97:0/1264198586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55968a4fd620 0x55968a508190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.151+0000 7f6b92d9d640 1 -- 172.21.15.97:0/1264198586 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55968a241620 con 0x55968a508800 2024-04-23T10:42:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.151+0000 7f6b91d9b640 1 --2- 172.21.15.97:0/1264198586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55968a4fdf40 0x55968a505480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:42:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.151+0000 7f6b92d9d640 1 --2- 172.21.15.97:0/1264198586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55968a508800 0x55968a505a00 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c010860 tx=0x7f6b8c009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b60a3c8e5bf5349c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:42:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.151+0000 7f6b79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:42:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.152+0000 7f6b79ffb640 1 -- 172.21.15.97:0/1264198586 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c003b70 con 0x55968a508800 2024-04-23T10:42:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.152+0000 7f6b79ffb640 1 -- 172.21.15.97:0/1264198586 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8484+0+0 (secure 0 0 0) 0x7f6b8c003d10 con 0x55968a508800 2024-04-23T10:42:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.152+0000 7f6b9bdc3740 1 -- 172.21.15.97:0/1264198586 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b7c0bd590 con 0x55968a508800 2024-04-23T10:42:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.152+0000 7f6b79ffb640 1 -- 172.21.15.97:0/1264198586 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6b8c02e050 con 0x55968a508800 2024-04-23T10:42:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.152+0000 7f6b9bdc3740 10 client.17829 fetch_fsmap learned FSMap version 26 2024-04-23T10:42:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.152+0000 7f6b9bdc3740 1 -- 172.21.15.97:0/1264198586 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55968a517000 con 0x55968a508800 2024-04-23T10:42:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.152+0000 7f6b79ffb640 1 -- 172.21.15.97:0/1264198586 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b7c0bd590 con 0x55968a508800 2024-04-23T10:42:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.152+0000 7f6b79ffb640 1 -- 172.21.15.97:0/1264198586 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f6b8c01b050 con 0x55968a508800 2024-04-23T10:42:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.152+0000 7f6b9bdc3740 10 client.17829 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:42:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.152+0000 7f6b9bdc3740 10 client.17829 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:42:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.152+0000 7f6b9bdc3740 20 client.17829 populate_metadata read hostname 'smithi097' 2024-04-23T10:42:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.152+0000 7f6b9bdc3740 1 --2- 172.21.15.97:0/1264198586 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55968a518560 0x55968a51a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.152+0000 7f6b9bdc3740 4 client.17829 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:42:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.152+0000 7f6b9bdc3740 1 -- 172.21.15.97:0/1264198586 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55968a470390 con 0x55968a518560 2024-04-23T10:42:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.152+0000 7f6b9259c640 1 --2- 172.21.15.97:0/1264198586 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55968a518560 0x55968a51a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.153+0000 7f6b9259c640 1 --2- 172.21.15.97:0/1264198586 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55968a518560 0x55968a51a990 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9e90ac01de0bb19 server_cookie=598c52edb1012540 in_seq=0 out_seq=0 2024-04-23T10:42:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.153+0000 7f6b79ffb640 10 client.17829 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:42:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.154+0000 7f6b79ffb640 1 -- 172.21.15.97:0/1264198586 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55968a470390 con 0x55968a518560 2024-04-23T10:42:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.154+0000 7f6b79ffb640 10 client.17829 handle_command_reply: tid=0 2024-04-23T10:42:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.154+0000 7f6b9bdc3740 1 client.17829 shutdown 2024-04-23T10:42:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.154+0000 7f6b9bdc3740 1 -- 172.21.15.97:0/1264198586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55968a508800 msgr2=0x55968a505a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.154+0000 7f6b9bdc3740 1 --2- 172.21.15.97:0/1264198586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55968a508800 0x55968a505a00 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c010860 tx=0x7f6b8c009240 comp rx=0 tx=0).stop 2024-04-23T10:42:26.160 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.154+0000 7f6b9bdc3740 1 -- 172.21.15.97:0/1264198586 shutdown_connections 2024-04-23T10:42:26.160 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.154+0000 7f6b9bdc3740 1 --2- 172.21.15.97:0/1264198586 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55968a518560 0x55968a51a990 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:26.160 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.154+0000 7f6b9bdc3740 1 --2- 172.21.15.97:0/1264198586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55968a508800 0x55968a505a00 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:26.160 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.154+0000 7f6b9bdc3740 1 --2- 172.21.15.97:0/1264198586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55968a4fdf40 0x55968a505480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:26.160 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.154+0000 7f6b79ffb640 0 client.17829 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:42:26.160 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.154+0000 7f6b9bdc3740 1 --2- 172.21.15.97:0/1264198586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55968a4fd620 0x55968a508190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:26.160 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.154+0000 7f6b9bdc3740 1 -- 172.21.15.97:0/1264198586 >> 172.21.15.97:0/1264198586 conn(0x55968a4f9b30 msgr2=0x55968a4ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:42:26.160 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.154+0000 7f6b9bdc3740 1 -- 172.21.15.97:0/1264198586 shutdown_connections 2024-04-23T10:42:26.160 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.154+0000 7f6b9bdc3740 1 -- 172.21.15.97:0/1264198586 wait complete. 2024-04-23T10:42:26.160 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.154+0000 7f6b9bdc3740 20 client.17829 trim_cache size 0 max 16384 2024-04-23T10:42:26.172 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.169+0000 7f6b9bdc3740 1 Processor -- start 2024-04-23T10:42:26.172 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.169+0000 7f6b9bdc3740 1 -- start start 2024-04-23T10:42:26.172 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.169+0000 7f6b9bdc3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55968a4fd620 0x55968a513f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:26.172 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.169+0000 7f6b9bdc3740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55968a4fdf40 0x55968a5202a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:26.173 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.169+0000 7f6b9bdc3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55968a508800 0x55968a5207e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:26.173 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.169+0000 7f6b9bdc3740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6b941ada60 con 0x55968a4fdf40 2024-04-23T10:42:26.173 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.169+0000 7f6b9bdc3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6b941af5a0 con 0x55968a508800 2024-04-23T10:42:26.173 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.169+0000 7f6b9bdc3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6b941ae130 con 0x55968a4fd620 2024-04-23T10:42:26.173 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.169+0000 7f6b91d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55968a4fdf40 0x55968a5202a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:26.173 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.169+0000 7f6b91d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55968a4fdf40 0x55968a5202a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43056/0 (socket says 172.21.15.97:43056) 2024-04-23T10:42:26.173 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.169+0000 7f6b91d9b640 1 -- 172.21.15.97:0/2871147415 learned_addr learned my addr 172.21.15.97:0/2871147415 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:42:26.173 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.169+0000 7f6b9259c640 1 --2- 172.21.15.97:0/2871147415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55968a4fd620 0x55968a513f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:26.173 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.169+0000 7f6b91d9b640 1 -- 172.21.15.97:0/2871147415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55968a4fd620 msgr2=0x55968a513f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:42:26.173 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.169+0000 7f6b92d9d640 1 --2- 172.21.15.97:0/2871147415 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55968a508800 0x55968a5207e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:26.173 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.169+0000 7f6b91d9b640 1 --2- 172.21.15.97:0/2871147415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55968a4fd620 0x55968a513f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:26.173 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.169+0000 7f6b91d9b640 1 -- 172.21.15.97:0/2871147415 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55968a508800 msgr2=0x55968a5207e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:42:26.173 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.169+0000 7f6b91d9b640 1 --2- 172.21.15.97:0/2871147415 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55968a508800 0x55968a5207e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:26.173 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.169+0000 7f6b91d9b640 1 -- 172.21.15.97:0/2871147415 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55968a504430 con 0x55968a4fdf40 2024-04-23T10:42:26.173 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.170+0000 7f6b91d9b640 1 --2- 172.21.15.97:0/2871147415 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55968a4fdf40 0x55968a5202a0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f6b9406a570 tx=0x7f6b84009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e9a860e8b45208 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:42:26.174 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.170+0000 7f6b7bfff640 1 -- 172.21.15.97:0/2871147415 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b84020070 con 0x55968a4fdf40 2024-04-23T10:42:26.174 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.170+0000 7f6b7bfff640 1 -- 172.21.15.97:0/2871147415 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b84010e50 con 0x55968a4fdf40 2024-04-23T10:42:26.174 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.170+0000 7f6b7bfff640 1 -- 172.21.15.97:0/2871147415 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8401b400 con 0x55968a4fdf40 2024-04-23T10:42:26.174 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.170+0000 7f6b9bdc3740 1 -- 172.21.15.97:0/2871147415 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55968a4fdf40 msgr2=0x55968a5202a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:26.174 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.170+0000 7f6b9bdc3740 1 --2- 172.21.15.97:0/2871147415 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55968a4fdf40 0x55968a5202a0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f6b9406a570 tx=0x7f6b84009f70 comp rx=0 tx=0).stop 2024-04-23T10:42:26.174 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.170+0000 7f6b9bdc3740 1 -- 172.21.15.97:0/2871147415 shutdown_connections 2024-04-23T10:42:26.174 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.170+0000 7f6b9bdc3740 1 --2- 172.21.15.97:0/2871147415 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55968a508800 0x55968a5207e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:26.174 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.170+0000 7f6b9bdc3740 1 --2- 172.21.15.97:0/2871147415 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55968a4fdf40 0x55968a5202a0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:26.174 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.170+0000 7f6b9bdc3740 1 --2- 172.21.15.97:0/2871147415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55968a4fd620 0x55968a513f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:26.174 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.170+0000 7f6b9bdc3740 1 -- 172.21.15.97:0/2871147415 >> 172.21.15.97:0/2871147415 conn(0x55968a4f9b30 msgr2=0x55968a4f9f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:42:26.174 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.170+0000 7f6b9bdc3740 1 -- 172.21.15.97:0/2871147415 shutdown_connections 2024-04-23T10:42:26.174 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.170+0000 7f6b9bdc3740 1 -- 172.21.15.97:0/2871147415 wait complete. 2024-04-23T10:42:26.174 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.171+0000 7f6b9bdc3740 1 -- start start 2024-04-23T10:42:26.174 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.171+0000 7f6b9bdc3740 1 Processor -- start 2024-04-23T10:42:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.171+0000 7f6b9bdc3740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55968a4fd620 0x55968a50de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.171+0000 7f6b9bdc3740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55968a4fdf40 0x55968a50e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.171+0000 7f6b9bdc3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55968a508800 0x55968a50e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.171+0000 7f6b9bdc3740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6b941aeed0 con 0x55968a4fdf40 2024-04-23T10:42:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.171+0000 7f6b9bdc3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6b941ae800 con 0x55968a508800 2024-04-23T10:42:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.171+0000 7f6b9bdc3740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6b941aba60 con 0x55968a4fd620 2024-04-23T10:42:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.171+0000 7f6b9bdc3740 10 client.0 _ll_register_callbacks cb 0x7f6b8c010740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:42:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.171+0000 7f6b91d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55968a4fdf40 0x55968a50e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.171+0000 7f6b91d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55968a4fdf40 0x55968a50e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:43066/0 (socket says 172.21.15.97:43066) 2024-04-23T10:42:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.171+0000 7f6b91d9b640 1 -- 172.21.15.97:0/1198490209 learned_addr learned my addr 172.21.15.97:0/1198490209 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:42:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.171+0000 7f6b92d9d640 1 --2- 172.21.15.97:0/1198490209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55968a508800 0x55968a50e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.171+0000 7f6b9259c640 1 --2- 172.21.15.97:0/1198490209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55968a4fd620 0x55968a50de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.171+0000 7f6b91d9b640 1 -- 172.21.15.97:0/1198490209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55968a4fd620 msgr2=0x55968a50de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:42:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.171+0000 7f6b91d9b640 1 --2- 172.21.15.97:0/1198490209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55968a4fd620 0x55968a50de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.171+0000 7f6b91d9b640 1 -- 172.21.15.97:0/1198490209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55968a508800 msgr2=0x55968a50e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:42:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.171+0000 7f6b91d9b640 1 --2- 172.21.15.97:0/1198490209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55968a508800 0x55968a50e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.171+0000 7f6b91d9b640 1 -- 172.21.15.97:0/1198490209 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55968a57db20 con 0x55968a4fdf40 2024-04-23T10:42:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.171+0000 7f6b9259c640 1 --2- 172.21.15.97:0/1198490209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55968a4fd620 0x55968a50de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:42:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.172+0000 7f6b91d9b640 1 --2- 172.21.15.97:0/1198490209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55968a4fdf40 0x55968a50e3b0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f6b8401d000 tx=0x7f6b84003af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ec61f2c073a92a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:42:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.172+0000 7f6b79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:42:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.172+0000 7f6b79ffb640 1 -- 172.21.15.97:0/1198490209 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b84029070 con 0x55968a4fdf40 2024-04-23T10:42:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.172+0000 7f6b79ffb640 1 -- 172.21.15.97:0/1198490209 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8484+0+0 (secure 0 0 0) 0x7f6b84004050 con 0x55968a4fdf40 2024-04-23T10:42:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.172+0000 7f6b9bdc3740 1 -- 172.21.15.97:0/1198490209 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b84010e50 con 0x55968a4fdf40 2024-04-23T10:42:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.172+0000 7f6b79ffb640 1 -- 172.21.15.97:0/1198490209 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6b84035050 con 0x55968a4fdf40 2024-04-23T10:42:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.172+0000 7f6b9bdc3740 10 client.17835 fetch_fsmap learned FSMap version 26 2024-04-23T10:42:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.172+0000 7f6b9bdc3740 1 -- 172.21.15.97:0/1198490209 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55968a57ee10 con 0x55968a4fdf40 2024-04-23T10:42:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.172+0000 7f6b79ffb640 1 -- 172.21.15.97:0/1198490209 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b84010e50 con 0x55968a4fdf40 2024-04-23T10:42:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.172+0000 7f6b79ffb640 1 -- 172.21.15.97:0/1198490209 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f6b84025410 con 0x55968a4fdf40 2024-04-23T10:42:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.172+0000 7f6b9bdc3740 10 client.17835 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:42:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.172+0000 7f6b9bdc3740 10 client.17835 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T10:42:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.172+0000 7f6b9bdc3740 20 client.17835 populate_metadata read hostname 'smithi097' 2024-04-23T10:42:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.172+0000 7f6b9bdc3740 1 --2- 172.21.15.97:0/1198490209 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55968a57f4b0 0x55968a5818d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.172+0000 7f6b9bdc3740 4 client.17835 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T10:42:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.172+0000 7f6b9bdc3740 1 -- 172.21.15.97:0/1198490209 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55968a46a200 con 0x55968a57f4b0 2024-04-23T10:42:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.172+0000 7f6b9259c640 1 --2- 172.21.15.97:0/1198490209 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55968a57f4b0 0x55968a5818d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.173+0000 7f6b79ffb640 10 client.17835 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:42:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.173+0000 7f6b9259c640 1 --2- 172.21.15.97:0/1198490209 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55968a57f4b0 0x55968a5818d0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=252a00e5bc9a5c42 server_cookie=d831020292354a78 in_seq=0 out_seq=0 2024-04-23T10:42:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.186+0000 7f6b79ffb640 1 -- 172.21.15.97:0/1198490209 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55968a46a200 con 0x55968a57f4b0 2024-04-23T10:42:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.186+0000 7f6b79ffb640 10 client.17835 handle_command_reply: tid=0 2024-04-23T10:42:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.186+0000 7f6b9bdc3740 1 client.17835 shutdown 2024-04-23T10:42:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.187+0000 7f6b9bdc3740 1 -- 172.21.15.97:0/1198490209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55968a4fdf40 msgr2=0x55968a50e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.187+0000 7f6b9bdc3740 1 --2- 172.21.15.97:0/1198490209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55968a4fdf40 0x55968a50e3b0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f6b8401d000 tx=0x7f6b84003af0 comp rx=0 tx=0).stop 2024-04-23T10:42:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.187+0000 7f6b9bdc3740 1 -- 172.21.15.97:0/1198490209 shutdown_connections 2024-04-23T10:42:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.187+0000 7f6b9bdc3740 1 --2- 172.21.15.97:0/1198490209 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55968a57f4b0 0x55968a5818d0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.187+0000 7f6b9bdc3740 1 --2- 172.21.15.97:0/1198490209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55968a508800 0x55968a50e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.187+0000 7f6b9bdc3740 1 --2- 172.21.15.97:0/1198490209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55968a4fdf40 0x55968a50e3b0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.187+0000 7f6b79ffb640 0 client.17835 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:42:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.187+0000 7f6b9bdc3740 1 --2- 172.21.15.97:0/1198490209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55968a4fd620 0x55968a50de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.187+0000 7f6b9bdc3740 1 -- 172.21.15.97:0/1198490209 >> 172.21.15.97:0/1198490209 conn(0x55968a4f9b30 msgr2=0x55968a4f8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:42:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.187+0000 7f6b9bdc3740 1 -- 172.21.15.97:0/1198490209 shutdown_connections 2024-04-23T10:42:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.187+0000 7f6b9bdc3740 1 -- 172.21.15.97:0/1198490209 wait complete. 2024-04-23T10:42:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.187+0000 7f6b9bdc3740 20 client.17835 trim_cache size 0 max 16384 2024-04-23T10:42:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:{ 2024-04-23T10:42:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "message": "", 2024-04-23T10:42:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout: "return_code": 0 2024-04-23T10:42:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stdout:} 2024-04-23T10:42:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.187+0000 7f6b98fcd640 1 -- 172.21.15.97:0/942913466 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f6b800a3dc0 msgr2=0x7f6b800a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.187+0000 7f6b98fcd640 1 --2- 172.21.15.97:0/942913466 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f6b800a3dc0 0x7f6b800a6280 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c005fd0 tx=0x7f6b8c0129e0 comp rx=0 tx=0).stop 2024-04-23T10:42:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.187+0000 7f6b98fcd640 1 -- 172.21.15.97:0/942913466 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b941010e0 msgr2=0x7f6b941986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.187+0000 7f6b98fcd640 1 --2- 172.21.15.97:0/942913466 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b941010e0 0x7f6b941986e0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c005940 tx=0x7f6b7c004270 comp rx=0 tx=0).stop 2024-04-23T10:42:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.187+0000 7f6b98fcd640 1 -- 172.21.15.97:0/942913466 shutdown_connections 2024-04-23T10:42:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.187+0000 7f6b98fcd640 1 --2- 172.21.15.97:0/942913466 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f6b800a3dc0 0x7f6b800a6280 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.187+0000 7f6b98fcd640 1 --2- 172.21.15.97:0/942913466 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b9419cfa0 0x7f6b9419d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.187+0000 7f6b98fcd640 1 --2- 172.21.15.97:0/942913466 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b941061b0 0x7f6b94198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.187+0000 7f6b98fcd640 1 --2- 172.21.15.97:0/942913466 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b941010e0 0x7f6b941986e0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.187+0000 7f6b98fcd640 1 -- 172.21.15.97:0/942913466 >> 172.21.15.97:0/942913466 conn(0x7f6b940771d0 msgr2=0x7f6b94104e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:42:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.188+0000 7f6b98fcd640 1 -- 172.21.15.97:0/942913466 shutdown_connections 2024-04-23T10:42:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi097.front.sepia.ceph.com.smithi097.stderr:2024-04-23T10:42:26.188+0000 7f6b98fcd640 1 -- 172.21.15.97:0/942913466 wait complete. 2024-04-23T10:42:26.551 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:42:26 smithi183 ceph-mon[32317]: pgmap v510: 97 pgs: 97 active+clean; 3.7 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-23T10:42:26.593 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:42:26 smithi196 ceph-mon[32919]: pgmap v510: 97 pgs: 97 active+clean; 3.7 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-23T10:42:26.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:42:26 smithi097 ceph-mon[29197]: pgmap v510: 97 pgs: 97 active+clean; 3.7 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-23T10:42:28.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:42:28 smithi196 ceph-mon[32919]: pgmap v511: 97 pgs: 97 active+clean; 4.0 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 63 KiB/s wr, 24 op/s 2024-04-23T10:42:28.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:42:28 smithi097 ceph-mon[29197]: pgmap v511: 97 pgs: 97 active+clean; 4.0 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 63 KiB/s wr, 24 op/s 2024-04-23T10:42:28.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:42:28 smithi183 ceph-mon[32317]: pgmap v511: 97 pgs: 97 active+clean; 4.0 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 63 KiB/s wr, 24 op/s 2024-04-23T10:42:30.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:42:30 smithi196 ceph-mon[32919]: pgmap v512: 97 pgs: 97 active+clean; 4.0 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:42:30.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:42:30 smithi097 ceph-mon[29197]: pgmap v512: 97 pgs: 97 active+clean; 4.0 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:42:30.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:42:30 smithi183 ceph-mon[32317]: pgmap v512: 97 pgs: 97 active+clean; 4.0 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:42:32.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:42:32 smithi196 ceph-mon[32919]: pgmap v513: 97 pgs: 97 active+clean; 4.0 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:42:32.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:42:32 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:42:32.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:42:32 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:42:32.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:42:32 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:42:32.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:42:32 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:42:32.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:42:32 smithi097 ceph-mon[29197]: pgmap v513: 97 pgs: 97 active+clean; 4.0 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:42:32.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:42:32 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:42:32.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:42:32 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:42:32.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:42:32 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:42:32.861 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:42:32 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:42:32.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:42:32 smithi183 ceph-mon[32317]: pgmap v513: 97 pgs: 97 active+clean; 4.0 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:42:32.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:42:32 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T10:42:32.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:42:32 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T10:42:32.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:42:32 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T10:42:32.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:42:32 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' 2024-04-23T10:42:34.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:42:34 smithi196 ceph-mon[32919]: pgmap v514: 97 pgs: 97 active+clean; 4.2 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-23T10:42:34.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:42:34 smithi097 ceph-mon[29197]: pgmap v514: 97 pgs: 97 active+clean; 4.2 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-23T10:42:34.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:42:34 smithi183 ceph-mon[32317]: pgmap v514: 97 pgs: 97 active+clean; 4.2 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-23T10:42:36.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:42:36 smithi196 ceph-mon[32919]: pgmap v515: 97 pgs: 97 active+clean; 4.2 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:42:36.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:42:36 smithi097 ceph-mon[29197]: pgmap v515: 97 pgs: 97 active+clean; 4.2 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:42:36.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:42:36 smithi183 ceph-mon[32317]: pgmap v515: 97 pgs: 97 active+clean; 4.2 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-23T10:42:38.009 INFO:tasks.workunit.client.0.smithi097.stdout:561/600: open fd = 3 2024-04-23T10:42:38.010 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.010 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.010 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:42:38.010 INFO:tasks.workunit.client.0.smithi097.stdout:562/600: open fd = 3 2024-04-23T10:42:38.010 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.010 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.010 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:42:38.010 INFO:tasks.workunit.client.0.smithi097.stdout:563/600: open fd = 3 2024-04-23T10:42:38.010 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.011 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.011 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:42:38.011 INFO:tasks.workunit.client.0.smithi097.stdout:564/600: open fd = 3 2024-04-23T10:42:38.011 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.011 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.011 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:42:38.011 INFO:tasks.workunit.client.0.smithi097.stdout:565/600: open fd = 3 2024-04-23T10:42:38.011 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.011 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.011 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:42:38.011 INFO:tasks.workunit.client.0.smithi097.stdout:566/600: open fd = 3 2024-04-23T10:42:38.012 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.012 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.012 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:42:38.012 INFO:tasks.workunit.client.0.smithi097.stdout:567/600: open fd = 3 2024-04-23T10:42:38.012 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.012 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.012 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:42:38.012 INFO:tasks.workunit.client.0.smithi097.stdout:568/600: open fd = 3 2024-04-23T10:42:38.012 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.012 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.012 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:42:38.013 INFO:tasks.workunit.client.0.smithi097.stdout:569/600: open fd = 3 2024-04-23T10:42:38.013 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.013 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.013 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:42:38.013 INFO:tasks.workunit.client.0.smithi097.stdout:570/600: open fd = 3 2024-04-23T10:42:38.013 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.013 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.013 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:42:38.013 INFO:tasks.workunit.client.0.smithi097.stdout:571/600: open fd = 3 2024-04-23T10:42:38.013 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.013 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.014 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:42:38.014 INFO:tasks.workunit.client.0.smithi097.stdout:572/600: open fd = 3 2024-04-23T10:42:38.014 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.014 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.014 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:42:38.014 INFO:tasks.workunit.client.0.smithi097.stdout:573/600: open fd = 3 2024-04-23T10:42:38.014 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.014 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.014 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:42:38.014 INFO:tasks.workunit.client.0.smithi097.stdout:574/600: open fd = 3 2024-04-23T10:42:38.015 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.015 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.015 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:42:38.015 INFO:tasks.workunit.client.0.smithi097.stdout:575/600: open fd = 3 2024-04-23T10:42:38.015 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.015 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.015 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:42:38.015 INFO:tasks.workunit.client.0.smithi097.stdout:576/600: open fd = 3 2024-04-23T10:42:38.015 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.015 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.015 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:42:38.016 INFO:tasks.workunit.client.0.smithi097.stdout:577/600: open fd = 3 2024-04-23T10:42:38.016 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.016 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.016 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:42:38.016 INFO:tasks.workunit.client.0.smithi097.stdout:578/600: open fd = 3 2024-04-23T10:42:38.016 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.016 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.016 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:42:38.016 INFO:tasks.workunit.client.0.smithi097.stdout:579/600: open fd = 3 2024-04-23T10:42:38.016 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.016 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.017 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:42:38.017 INFO:tasks.workunit.client.0.smithi097.stdout:580/600: open fd = 3 2024-04-23T10:42:38.017 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.017 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.017 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:42:38.017 INFO:tasks.workunit.client.0.smithi097.stdout:581/600: open fd = 3 2024-04-23T10:42:38.017 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.017 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.017 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:42:38.017 INFO:tasks.workunit.client.0.smithi097.stdout:582/600: open fd = 3 2024-04-23T10:42:38.018 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.018 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.018 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:42:38.018 INFO:tasks.workunit.client.0.smithi097.stdout:583/600: open fd = 3 2024-04-23T10:42:38.018 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.018 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.018 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:42:38.018 INFO:tasks.workunit.client.0.smithi097.stdout:584/600: open fd = 3 2024-04-23T10:42:38.018 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.018 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.018 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:42:38.019 INFO:tasks.workunit.client.0.smithi097.stdout:585/600: open fd = 3 2024-04-23T10:42:38.019 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.019 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.019 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:42:38.019 INFO:tasks.workunit.client.0.smithi097.stdout:586/600: open fd = 3 2024-04-23T10:42:38.019 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.019 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.019 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:42:38.019 INFO:tasks.workunit.client.0.smithi097.stdout:587/600: open fd = 3 2024-04-23T10:42:38.019 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.019 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.020 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:42:38.020 INFO:tasks.workunit.client.0.smithi097.stdout:588/600: open fd = 3 2024-04-23T10:42:38.020 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.020 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.020 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:42:38.020 INFO:tasks.workunit.client.0.smithi097.stdout:589/600: open fd = 3 2024-04-23T10:42:38.020 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.020 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.020 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:42:38.020 INFO:tasks.workunit.client.0.smithi097.stdout:590/600: open fd = 3 2024-04-23T10:42:38.021 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.021 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.021 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:42:38.021 INFO:tasks.workunit.client.0.smithi097.stdout:591/600: open fd = 3 2024-04-23T10:42:38.021 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.021 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.021 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:42:38.021 INFO:tasks.workunit.client.0.smithi097.stdout:592/600: open fd = 3 2024-04-23T10:42:38.021 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.021 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.021 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:42:38.022 INFO:tasks.workunit.client.0.smithi097.stdout:593/600: open fd = 3 2024-04-23T10:42:38.022 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.022 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.022 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:42:38.022 INFO:tasks.workunit.client.0.smithi097.stdout:594/600: open fd = 3 2024-04-23T10:42:38.022 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.022 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.022 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:42:38.022 INFO:tasks.workunit.client.0.smithi097.stdout:595/600: open fd = 3 2024-04-23T10:42:38.022 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.023 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.023 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:42:38.023 INFO:tasks.workunit.client.0.smithi097.stdout:596/600: open fd = 3 2024-04-23T10:42:38.023 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.023 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.023 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:42:38.023 INFO:tasks.workunit.client.0.smithi097.stdout:597/600: open fd = 3 2024-04-23T10:42:38.023 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.023 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.023 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:42:38.023 INFO:tasks.workunit.client.0.smithi097.stdout:598/600: open fd = 3 2024-04-23T10:42:38.023 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.024 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.024 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:42:38.024 INFO:tasks.workunit.client.0.smithi097.stdout:599/600: open fd = 3 2024-04-23T10:42:38.024 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.024 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.024 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:42:38.024 INFO:tasks.workunit.client.0.smithi097.stdout:600/600: open fd = 3 2024-04-23T10:42:38.024 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.024 INFO:tasks.workunit.client.0.smithi097.stdout:write ret = 32 2024-04-23T10:42:38.024 INFO:tasks.workunit.client.0.smithi097.stdout:pread ret = 64 2024-04-23T10:42:38.025 INFO:tasks.workunit.client.0.smithi097.stderr:+ echo OK 2024-04-23T10:42:38.026 INFO:tasks.workunit.client.0.smithi097.stdout:OK 2024-04-23T10:42:38.026 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-23T10:42:38.026 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-23T10:42:38.069 INFO:tasks.workunit:Stopping ['fs/test_o_trunc.sh'] on client.0... 2024-04-23T10:42:38.069 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-23T10:42:38.404 DEBUG:teuthology.parallel:result is None 2024-04-23T10:42:38.404 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-23T10:42:38.439 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-23T10:42:38.439 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-04-23T10:42:38.450 INFO:tasks.quiescer:joining Quiescers 2024-04-23T10:42:38.450 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-04-23T10:42:38.451 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T10:42:38.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.579+0000 7f3a6c835640 1 -- 172.21.15.97:0/1978875005 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a68101a40 msgr2=0x7f3a68105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:38.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.579+0000 7f3a6c835640 1 --2- 172.21.15.97:0/1978875005 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a68101a40 0x7f3a68105a80 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f3a50009900 tx=0x7f3a5002a410 comp rx=0 tx=0).stop 2024-04-23T10:42:38.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.579+0000 7f3a6c835640 1 -- 172.21.15.97:0/1978875005 shutdown_connections 2024-04-23T10:42:38.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.579+0000 7f3a6c835640 1 --2- 172.21.15.97:0/1978875005 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a68106160 0x7f3a6810dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:38.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.579+0000 7f3a6c835640 1 --2- 172.21.15.97:0/1978875005 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a68101a40 0x7f3a68105a80 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:38.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.579+0000 7f3a6c835640 1 --2- 172.21.15.97:0/1978875005 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a68101090 0x7f3a68101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:38.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.579+0000 7f3a6c835640 1 -- 172.21.15.97:0/1978875005 >> 172.21.15.97:0/1978875005 conn(0x7f3a680771d0 msgr2=0x7f3a680775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:42:38.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.579+0000 7f3a6c835640 1 -- 172.21.15.97:0/1978875005 shutdown_connections 2024-04-23T10:42:38.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.579+0000 7f3a6c835640 1 -- 172.21.15.97:0/1978875005 wait complete. 2024-04-23T10:42:38.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.580+0000 7f3a6c835640 1 Processor -- start 2024-04-23T10:42:38.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.580+0000 7f3a6c835640 1 -- start start 2024-04-23T10:42:38.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.580+0000 7f3a6c835640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a68101090 0x7f3a6819e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:38.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.580+0000 7f3a6c835640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a68106160 0x7f3a6819e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:38.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.580+0000 7f3a6c835640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a681985b0 0x7f3a68198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:38.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.580+0000 7f3a6c835640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3a681110a0 con 0x7f3a681985b0 2024-04-23T10:42:38.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.580+0000 7f3a6c835640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3a68110f20 con 0x7f3a68106160 2024-04-23T10:42:38.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.580+0000 7f3a6c835640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3a68111220 con 0x7f3a68101090 2024-04-23T10:42:38.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.580+0000 7f3a66d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a681985b0 0x7f3a68198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:38.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.580+0000 7f3a66d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a681985b0 0x7f3a68198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48144/0 (socket says 172.21.15.97:48144) 2024-04-23T10:42:38.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.580+0000 7f3a66d9d640 1 -- 172.21.15.97:0/2535469030 learned_addr learned my addr 172.21.15.97:0/2535469030 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:42:38.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.580+0000 7f3a6659c640 1 --2- 172.21.15.97:0/2535469030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a68101090 0x7f3a6819e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:38.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.580+0000 7f3a5ffff640 1 --2- 172.21.15.97:0/2535469030 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a68106160 0x7f3a6819e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:38.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.580+0000 7f3a66d9d640 1 -- 172.21.15.97:0/2535469030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a68101090 msgr2=0x7f3a6819e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:38.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.580+0000 7f3a66d9d640 1 --2- 172.21.15.97:0/2535469030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a68101090 0x7f3a6819e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:38.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.580+0000 7f3a66d9d640 1 -- 172.21.15.97:0/2535469030 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a68106160 msgr2=0x7f3a6819e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:38.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.580+0000 7f3a66d9d640 1 --2- 172.21.15.97:0/2535469030 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a68106160 0x7f3a6819e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:38.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.580+0000 7f3a66d9d640 1 -- 172.21.15.97:0/2535469030 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a681990d0 con 0x7f3a681985b0 2024-04-23T10:42:38.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.581+0000 7f3a66d9d640 1 --2- 172.21.15.97:0/2535469030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a681985b0 0x7f3a68198a60 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f3a6000eaa0 tx=0x7f3a6000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:42:38.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.581+0000 7f3a5f7fe640 1 -- 172.21.15.97:0/2535469030 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a6000cdb0 con 0x7f3a681985b0 2024-04-23T10:42:38.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.581+0000 7f3a5f7fe640 1 -- 172.21.15.97:0/2535469030 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a60005020 con 0x7f3a681985b0 2024-04-23T10:42:38.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.581+0000 7f3a6c835640 1 -- 172.21.15.97:0/2535469030 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a681993c0 con 0x7f3a681985b0 2024-04-23T10:42:38.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.581+0000 7f3a5f7fe640 1 -- 172.21.15.97:0/2535469030 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a600116d0 con 0x7f3a681985b0 2024-04-23T10:42:38.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.581+0000 7f3a6c835640 1 -- 172.21.15.97:0/2535469030 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a681a55c0 con 0x7f3a681985b0 2024-04-23T10:42:38.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.582+0000 7f3a6c835640 1 -- 172.21.15.97:0/2535469030 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a68102560 con 0x7f3a681985b0 2024-04-23T10:42:38.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.583+0000 7f3a5f7fe640 1 -- 172.21.15.97:0/2535469030 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3a600051d0 con 0x7f3a681985b0 2024-04-23T10:42:38.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.583+0000 7f3a5f7fe640 1 --2- 172.21.15.97:0/2535469030 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f3a540a3d20 0x7f3a540a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:38.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.584+0000 7f3a6659c640 1 --2- 172.21.15.97:0/2535469030 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f3a540a3d20 0x7f3a540a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:38.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.584+0000 7f3a5f7fe640 1 -- 172.21.15.97:0/2535469030 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8484+0+0 (secure 0 0 0) 0x7f3a600d0d50 con 0x7f3a681985b0 2024-04-23T10:42:38.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.584+0000 7f3a6659c640 1 --2- 172.21.15.97:0/2535469030 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f3a540a3d20 0x7f3a540a61e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3a58005fd0 tx=0x7f3a5800b390 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:42:38.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.587+0000 7f3a5f7fe640 1 -- 172.21.15.97:0/2535469030 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3a60016030 con 0x7f3a681985b0 2024-04-23T10:42:38.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.808+0000 7f3a6c835640 1 -- 172.21.15.97:0/2535469030 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3a680691c0 con 0x7f3a681985b0 2024-04-23T10:42:38.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.809+0000 7f3a5f7fe640 1 -- 172.21.15.97:0/2535469030 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11931 (secure 0 0 0) 0x7f3a680691c0 con 0x7f3a681985b0 2024-04-23T10:42:38.808 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:42:38.808 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3946878762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3946878762},{"type":"v1","addr":"172.21.15.97:6835","nonce":3946878762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/209800861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":209800861},{"type":"v1","addr":"172.21.15.97:6837","nonce":209800861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T10:24:06.910134+0000","modified":"2024-04-23T10:26:09.820985+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6839/418013575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":418013575},{"type":"v1","addr":"172.21.15.97:6839","nonce":418013575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6841/747242680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":747242680},{"type":"v1","addr":"172.21.15.97:6841","nonce":747242680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6833/4292847121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4292847121},{"type":"v1","addr":"172.21.15.183:6833","nonce":4292847121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6833/3885816565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3885816565},{"type":"v1","addr":"172.21.15.196:6833","nonce":3885816565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1897093105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1897093105},{"type":"v1","addr":"172.21.15.183:6835","nonce":1897093105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1171477563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1171477563},{"type":"v1","addr":"172.21.15.196:6835","nonce":1171477563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1898341911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1898341911},{"type":"v1","addr":"172.21.15.183:6837","nonce":1898341911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.196:6837/3094946502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3094946502},{"type":"v1","addr":"172.21.15.196:6837","nonce":3094946502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.196:6839/83843556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":83843556},{"type":"v1","addr":"172.21.15.196:6839","nonce":83843556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6839/1763841130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1763841130},{"type":"v1","addr":"172.21.15.183:6839","nonce":1763841130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24487,24403,24398,24469,24488]},"id":1}]} 2024-04-23T10:42:38.808 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 26 2024-04-23T10:42:38.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.811+0000 7f3a6c835640 1 -- 172.21.15.97:0/2535469030 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f3a540a3d20 msgr2=0x7f3a540a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:38.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.811+0000 7f3a6c835640 1 --2- 172.21.15.97:0/2535469030 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f3a540a3d20 0x7f3a540a61e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3a58005fd0 tx=0x7f3a5800b390 comp rx=0 tx=0).stop 2024-04-23T10:42:38.810 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.811+0000 7f3a6c835640 1 -- 172.21.15.97:0/2535469030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a681985b0 msgr2=0x7f3a68198a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:38.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.811+0000 7f3a6c835640 1 --2- 172.21.15.97:0/2535469030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a681985b0 0x7f3a68198a60 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f3a6000eaa0 tx=0x7f3a6000ef60 comp rx=0 tx=0).stop 2024-04-23T10:42:38.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.811+0000 7f3a6c835640 1 -- 172.21.15.97:0/2535469030 shutdown_connections 2024-04-23T10:42:38.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.811+0000 7f3a6c835640 1 --2- 172.21.15.97:0/2535469030 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f3a540a3d20 0x7f3a540a61e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:38.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.811+0000 7f3a6c835640 1 --2- 172.21.15.97:0/2535469030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a681985b0 0x7f3a68198a60 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:38.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.811+0000 7f3a6c835640 1 --2- 172.21.15.97:0/2535469030 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a68106160 0x7f3a6819e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:38.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.811+0000 7f3a6c835640 1 --2- 172.21.15.97:0/2535469030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3a68101090 0x7f3a6819e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:38.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.811+0000 7f3a6c835640 1 -- 172.21.15.97:0/2535469030 >> 172.21.15.97:0/2535469030 conn(0x7f3a680771d0 msgr2=0x7f3a6806a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:42:38.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.811+0000 7f3a6c835640 1 -- 172.21.15.97:0/2535469030 shutdown_connections 2024-04-23T10:42:38.811 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.811+0000 7f3a6c835640 1 -- 172.21.15.97:0/2535469030 wait complete. 2024-04-23T10:42:38.820 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:42:38 smithi097 ceph-mon[29197]: pgmap v516: 97 pgs: 97 active+clean; 4.3 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-23T10:42:38.821 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --cancel --all' 2024-04-23T10:42:38.822 DEBUG:teuthology.orchestra.run.smithi097:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24488 quiesce db --cancel --all 2024-04-23T10:42:38.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:42:38 smithi196 ceph-mon[32919]: pgmap v516: 97 pgs: 97 active+clean; 4.3 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-23T10:42:38.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:42:38 smithi183 ceph-mon[32317]: pgmap v516: 97 pgs: 97 active+clean; 4.3 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-23T10:42:38.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.926+0000 7f6c7d810640 1 -- 172.21.15.97:0/682754356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c781016d0 msgr2=0x7f6c7810be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:38.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.926+0000 7f6c7d810640 1 --2- 172.21.15.97:0/682754356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c781016d0 0x7f6c7810be30 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c009900 tx=0x7f6c6c02a410 comp rx=0 tx=0).stop 2024-04-23T10:42:38.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.926+0000 7f6c7d810640 1 -- 172.21.15.97:0/682754356 shutdown_connections 2024-04-23T10:42:38.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.926+0000 7f6c7d810640 1 --2- 172.21.15.97:0/682754356 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c7810c4f0 0x7f6c7810e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:38.928 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.926+0000 7f6c7d810640 1 --2- 172.21.15.97:0/682754356 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c781016d0 0x7f6c7810be30 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:38.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.926+0000 7f6c7d810640 1 --2- 172.21.15.97:0/682754356 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c78100d20 0x7f6c78101100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:38.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.926+0000 7f6c7d810640 1 -- 172.21.15.97:0/682754356 >> 172.21.15.97:0/682754356 conn(0x7f6c78076150 msgr2=0x7f6c78076560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:42:38.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.926+0000 7f6c7d810640 1 -- 172.21.15.97:0/682754356 shutdown_connections 2024-04-23T10:42:38.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.926+0000 7f6c7d810640 1 -- 172.21.15.97:0/682754356 wait complete. 2024-04-23T10:42:38.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.927+0000 7f6c7d810640 1 Processor -- start 2024-04-23T10:42:38.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.927+0000 7f6c7d810640 1 -- start start 2024-04-23T10:42:38.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.927+0000 7f6c7d810640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c78100d20 0x7f6c7819e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:38.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.927+0000 7f6c7d810640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c7810c4f0 0x7f6c7819eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:38.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.927+0000 7f6c7d810640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c781986d0 0x7f6c78198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:38.929 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.927+0000 7f6c7d810640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6c781121a0 con 0x7f6c7810c4f0 2024-04-23T10:42:38.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.927+0000 7f6c7d810640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6c78112020 con 0x7f6c781986d0 2024-04-23T10:42:38.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.927+0000 7f6c7d810640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6c78112320 con 0x7f6c78100d20 2024-04-23T10:42:38.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.927+0000 7f6c767fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c7810c4f0 0x7f6c7819eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:38.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.927+0000 7f6c767fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c7810c4f0 0x7f6c7819eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48158/0 (socket says 172.21.15.97:48158) 2024-04-23T10:42:38.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.927+0000 7f6c767fc640 1 -- 172.21.15.97:0/1313853024 learned_addr learned my addr 172.21.15.97:0/1313853024 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:42:38.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.927+0000 7f6c76ffd640 1 --2- 172.21.15.97:0/1313853024 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c78100d20 0x7f6c7819e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:38.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.927+0000 7f6c777fe640 1 --2- 172.21.15.97:0/1313853024 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c781986d0 0x7f6c78198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:38.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.927+0000 7f6c767fc640 1 -- 172.21.15.97:0/1313853024 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c78100d20 msgr2=0x7f6c7819e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:38.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.927+0000 7f6c767fc640 1 --2- 172.21.15.97:0/1313853024 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c78100d20 0x7f6c7819e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:38.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.927+0000 7f6c767fc640 1 -- 172.21.15.97:0/1313853024 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c781986d0 msgr2=0x7f6c78198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:38.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.927+0000 7f6c767fc640 1 --2- 172.21.15.97:0/1313853024 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c781986d0 0x7f6c78198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:38.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.927+0000 7f6c767fc640 1 -- 172.21.15.97:0/1313853024 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c781992d0 con 0x7f6c7810c4f0 2024-04-23T10:42:38.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.927+0000 7f6c777fe640 1 --2- 172.21.15.97:0/1313853024 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c781986d0 0x7f6c78198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:42:38.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.928+0000 7f6c767fc640 1 --2- 172.21.15.97:0/1313853024 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c7810c4f0 0x7f6c7819eb00 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c02a930 tx=0x7f6c6c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:42:38.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.928+0000 7f6c7c80e640 1 -- 172.21.15.97:0/1313853024 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c03f030 con 0x7f6c7810c4f0 2024-04-23T10:42:38.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.928+0000 7f6c7c80e640 1 -- 172.21.15.97:0/1313853024 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c6c003aa0 con 0x7f6c7810c4f0 2024-04-23T10:42:38.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.928+0000 7f6c7d810640 1 -- 172.21.15.97:0/1313853024 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c78199560 con 0x7f6c7810c4f0 2024-04-23T10:42:38.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.928+0000 7f6c7c80e640 1 -- 172.21.15.97:0/1313853024 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c035070 con 0x7f6c7810c4f0 2024-04-23T10:42:38.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.928+0000 7f6c7d810640 1 -- 172.21.15.97:0/1313853024 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c781a5650 con 0x7f6c7810c4f0 2024-04-23T10:42:38.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.930+0000 7f6c7c80e640 1 -- 172.21.15.97:0/1313853024 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6c6c042030 con 0x7f6c7810c4f0 2024-04-23T10:42:38.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.931+0000 7f6c80601740 1 Processor -- start 2024-04-23T10:42:38.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.931+0000 7f6c80601740 1 -- start start 2024-04-23T10:42:38.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.931+0000 7f6c80601740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2d0a85200 0x55d2d0a855e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:38.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.931+0000 7f6c80601740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2d0a85bb0 0x55d2d0a901a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:38.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.931+0000 7f6c80601740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2d0a907c0 0x55d2d0a90c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:38.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.931+0000 7f6c80601740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d2d09eb130 con 0x55d2d0a85bb0 2024-04-23T10:42:38.940 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.931+0000 7f6c80601740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d2d0987c50 con 0x55d2d0a85200 2024-04-23T10:42:38.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.931+0000 7f6c80601740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d2d09e70f0 con 0x55d2d0a907c0 2024-04-23T10:42:38.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.931+0000 7f6c76ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2d0a85bb0 0x55d2d0a901a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:38.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.931+0000 7f6c76ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2d0a85bb0 0x55d2d0a901a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48172/0 (socket says 172.21.15.97:48172) 2024-04-23T10:42:38.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.931+0000 7f6c76ffd640 1 -- 172.21.15.97:0/2130044281 learned_addr learned my addr 172.21.15.97:0/2130044281 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:42:38.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.931+0000 7f6c7c80e640 1 --2- 172.21.15.97:0/1313853024 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f6c600a3d20 0x7f6c600a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:38.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.931+0000 7f6c767fc640 1 --2- 172.21.15.97:0/2130044281 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2d0a907c0 0x55d2d0a90c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:38.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.931+0000 7f6c777fe640 1 --2- 172.21.15.97:0/2130044281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2d0a85200 0x55d2d0a855e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:38.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.931+0000 7f6c777fe640 1 --2- 172.21.15.97:0/1313853024 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f6c600a3d20 0x7f6c600a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:38.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.931+0000 7f6c76ffd640 1 -- 172.21.15.97:0/2130044281 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2d0a907c0 msgr2=0x55d2d0a90c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:42:38.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.931+0000 7f6c76ffd640 1 --2- 172.21.15.97:0/2130044281 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2d0a907c0 0x55d2d0a90c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:38.941 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.931+0000 7f6c76ffd640 1 -- 172.21.15.97:0/2130044281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2d0a85200 msgr2=0x55d2d0a855e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:42:38.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.931+0000 7f6c76ffd640 1 --2- 172.21.15.97:0/2130044281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2d0a85200 0x55d2d0a855e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:38.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.931+0000 7f6c76ffd640 1 -- 172.21.15.97:0/2130044281 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2d09a1fd0 con 0x55d2d0a85bb0 2024-04-23T10:42:38.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.931+0000 7f6c7c80e640 1 -- 172.21.15.97:0/1313853024 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8484+0+0 (secure 0 0 0) 0x7f6c6c0f0e60 con 0x7f6c7810c4f0 2024-04-23T10:42:38.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.932+0000 7f6c76ffd640 1 --2- 172.21.15.97:0/2130044281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2d0a85bb0 0x55d2d0a901a0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f6c6800ac10 tx=0x7f6c6800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a5e756855648189 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:42:38.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.932+0000 7f6c777fe640 1 --2- 172.21.15.97:0/1313853024 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f6c600a3d20 0x7f6c600a61e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6c70005c40 tx=0x7f6c70009280 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:42:38.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.932+0000 7f6c5dffb640 1 -- 172.21.15.97:0/2130044281 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6801d070 con 0x55d2d0a85bb0 2024-04-23T10:42:38.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.932+0000 7f6c5dffb640 1 -- 172.21.15.97:0/2130044281 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c68018af0 con 0x55d2d0a85bb0 2024-04-23T10:42:38.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.932+0000 7f6c5dffb640 1 -- 172.21.15.97:0/2130044281 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c68010a60 con 0x55d2d0a85bb0 2024-04-23T10:42:38.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.932+0000 7f6c80601740 1 -- 172.21.15.97:0/2130044281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2d0a85bb0 msgr2=0x55d2d0a901a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:38.942 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.932+0000 7f6c80601740 1 --2- 172.21.15.97:0/2130044281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2d0a85bb0 0x55d2d0a901a0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f6c6800ac10 tx=0x7f6c6800f790 comp rx=0 tx=0).stop 2024-04-23T10:42:38.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.932+0000 7f6c80601740 1 -- 172.21.15.97:0/2130044281 shutdown_connections 2024-04-23T10:42:38.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.932+0000 7f6c80601740 1 --2- 172.21.15.97:0/2130044281 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2d0a907c0 0x55d2d0a90c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:38.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.932+0000 7f6c80601740 1 --2- 172.21.15.97:0/2130044281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2d0a85bb0 0x55d2d0a901a0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:38.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.932+0000 7f6c80601740 1 --2- 172.21.15.97:0/2130044281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2d0a85200 0x55d2d0a855e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:38.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.932+0000 7f6c80601740 1 -- 172.21.15.97:0/2130044281 >> 172.21.15.97:0/2130044281 conn(0x55d2d0a81620 msgr2=0x55d2d0a81a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:42:38.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.932+0000 7f6c80601740 1 -- 172.21.15.97:0/2130044281 shutdown_connections 2024-04-23T10:42:38.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.932+0000 7f6c80601740 1 -- 172.21.15.97:0/2130044281 wait complete. 2024-04-23T10:42:38.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.933+0000 7f6c80601740 1 -- start start 2024-04-23T10:42:38.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.933+0000 7f6c80601740 1 Processor -- start 2024-04-23T10:42:38.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.933+0000 7f6c80601740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2d0a85200 0x55d2d0a940e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:38.943 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.933+0000 7f6c80601740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2d0a85bb0 0x55d2d0a94620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:38.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.933+0000 7f6c80601740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2d0a907c0 0x55d2d0a94b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:38.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.933+0000 7f6c80601740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d2d0a97740 con 0x55d2d0a85200 2024-04-23T10:42:38.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.933+0000 7f6c80601740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d2d0985640 con 0x55d2d0a85bb0 2024-04-23T10:42:38.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.933+0000 7f6c80601740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x55d2d0a978c0 con 0x55d2d0a907c0 2024-04-23T10:42:38.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.933+0000 7f6c80601740 10 client.0 _ll_register_callbacks cb 0x55d2d0985030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:42:38.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.933+0000 7f6c76ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2d0a85200 0x55d2d0a940e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:38.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.933+0000 7f6c76ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2d0a85200 0x55d2d0a940e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48182/0 (socket says 172.21.15.97:48182) 2024-04-23T10:42:38.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.933+0000 7f6c76ffd640 1 -- 172.21.15.97:0/3340751355 learned_addr learned my addr 172.21.15.97:0/3340751355 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:42:38.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.933+0000 7f6c767fc640 1 --2- 172.21.15.97:0/3340751355 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2d0a85bb0 0x55d2d0a94620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:38.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.934+0000 7f6c76ffd640 1 -- 172.21.15.97:0/3340751355 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2d0a907c0 msgr2=0x55d2d0a94b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:42:38.944 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.934+0000 7f6c76ffd640 1 --2- 172.21.15.97:0/3340751355 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2d0a907c0 0x55d2d0a94b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:38.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.934+0000 7f6c76ffd640 1 -- 172.21.15.97:0/3340751355 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2d0a85bb0 msgr2=0x55d2d0a94620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:42:38.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.934+0000 7f6c76ffd640 1 --2- 172.21.15.97:0/3340751355 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2d0a85bb0 0x55d2d0a94620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:38.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.934+0000 7f6c76ffd640 1 -- 172.21.15.97:0/3340751355 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2d07c9a20 con 0x55d2d0a85200 2024-04-23T10:42:38.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.934+0000 7f6c767fc640 1 --2- 172.21.15.97:0/3340751355 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2d0a85bb0 0x55d2d0a94620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:42:38.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.934+0000 7f6c76ffd640 1 --2- 172.21.15.97:0/3340751355 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2d0a85200 0x55d2d0a940e0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f6c68018670 tx=0x7f6c680186a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa9a59954c77e041 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:42:38.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.934+0000 7f6c56ffd640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:42:38.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.934+0000 7f6c56ffd640 1 -- 172.21.15.97:0/3340751355 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c68027020 con 0x55d2d0a85200 2024-04-23T10:42:38.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.934+0000 7f6c56ffd640 1 -- 172.21.15.97:0/3340751355 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8484+0+0 (secure 0 0 0) 0x7f6c68005040 con 0x55d2d0a85200 2024-04-23T10:42:38.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.934+0000 7f6c80601740 1 -- 172.21.15.97:0/3340751355 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c68018af0 con 0x55d2d0a85200 2024-04-23T10:42:38.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.934+0000 7f6c56ffd640 1 -- 172.21.15.97:0/3340751355 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6c68032050 con 0x55d2d0a85200 2024-04-23T10:42:38.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.934+0000 7f6c80601740 10 client.17853 fetch_fsmap learned FSMap version 26 2024-04-23T10:42:38.945 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.934+0000 7f6c80601740 1 -- 172.21.15.97:0/3340751355 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d2d082a1b0 con 0x55d2d0a85200 2024-04-23T10:42:38.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.934+0000 7f6c56ffd640 1 -- 172.21.15.97:0/3340751355 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c68018af0 con 0x55d2d0a85200 2024-04-23T10:42:38.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.934+0000 7f6c56ffd640 1 -- 172.21.15.97:0/3340751355 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f6c680046c0 con 0x55d2d0a85200 2024-04-23T10:42:38.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.934+0000 7f6c80601740 10 client.17853 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:42:38.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.934+0000 7f6c80601740 10 client.17853 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:42:38.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.934+0000 7f6c80601740 20 client.17853 populate_metadata read hostname 'smithi097' 2024-04-23T10:42:38.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.934+0000 7f6c80601740 1 --2- 172.21.15.97:0/3340751355 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55d2d0aa4b50 0x55d2d0aa6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:38.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.934+0000 7f6c80601740 4 client.17853 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T10:42:38.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.934+0000 7f6c80601740 1 -- 172.21.15.97:0/3340751355 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2d09f8b50 con 0x55d2d0aa4b50 2024-04-23T10:42:38.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.935+0000 7f6c767fc640 1 --2- 172.21.15.97:0/3340751355 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55d2d0aa4b50 0x55d2d0aa6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:38.946 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.935+0000 7f6c56ffd640 10 client.17853 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:42:38.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.935+0000 7f6c767fc640 1 --2- 172.21.15.97:0/3340751355 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55d2d0aa4b50 0x55d2d0aa6f40 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=750d7433385bb1b3 server_cookie=d59f3ddcbcb3647f in_seq=0 out_seq=0 2024-04-23T10:42:38.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.936+0000 7f6c56ffd640 1 -- 172.21.15.97:0/3340751355 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d2d09f8b50 con 0x55d2d0aa4b50 2024-04-23T10:42:38.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.936+0000 7f6c56ffd640 10 client.17853 handle_command_reply: tid=0 2024-04-23T10:42:38.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.936+0000 7f6c80601740 1 client.17853 shutdown 2024-04-23T10:42:38.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.936+0000 7f6c80601740 1 -- 172.21.15.97:0/3340751355 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2d0a85200 msgr2=0x55d2d0a940e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:38.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.936+0000 7f6c80601740 1 --2- 172.21.15.97:0/3340751355 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2d0a85200 0x55d2d0a940e0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f6c68018670 tx=0x7f6c680186a0 comp rx=0 tx=0).stop 2024-04-23T10:42:38.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.936+0000 7f6c80601740 1 -- 172.21.15.97:0/3340751355 shutdown_connections 2024-04-23T10:42:38.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.936+0000 7f6c80601740 1 --2- 172.21.15.97:0/3340751355 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55d2d0aa4b50 0x55d2d0aa6f40 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:38.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.936+0000 7f6c80601740 1 --2- 172.21.15.97:0/3340751355 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2d0a907c0 0x55d2d0a94b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:38.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.936+0000 7f6c80601740 1 --2- 172.21.15.97:0/3340751355 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2d0a85bb0 0x55d2d0a94620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:38.947 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.936+0000 7f6c56ffd640 0 client.17853 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:42:38.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.936+0000 7f6c80601740 1 --2- 172.21.15.97:0/3340751355 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2d0a85200 0x55d2d0a940e0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:38.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.936+0000 7f6c80601740 1 -- 172.21.15.97:0/3340751355 >> 172.21.15.97:0/3340751355 conn(0x55d2d0a81620 msgr2=0x55d2d0a87470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:42:38.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.936+0000 7f6c80601740 1 -- 172.21.15.97:0/3340751355 shutdown_connections 2024-04-23T10:42:38.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.936+0000 7f6c80601740 1 -- 172.21.15.97:0/3340751355 wait complete. 2024-04-23T10:42:38.948 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.936+0000 7f6c80601740 20 client.17853 trim_cache size 0 max 16384 2024-04-23T10:42:38.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.953+0000 7f6c80601740 1 Processor -- start 2024-04-23T10:42:38.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.954+0000 7f6c80601740 1 -- start start 2024-04-23T10:42:38.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.954+0000 7f6c80601740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2d0a81620 0x55d2d0aa0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:38.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.954+0000 7f6c80601740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2d0a85200 0x55d2d0a87250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:38.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.954+0000 7f6c80601740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2d0a85bb0 0x55d2d0a91bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:38.959 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.954+0000 7f6c80601740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6c781ab9f0 con 0x55d2d0a85bb0 2024-04-23T10:42:38.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.954+0000 7f6c80601740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6c781ad530 con 0x55d2d0a85200 2024-04-23T10:42:38.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.954+0000 7f6c80601740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6c781ac0c0 con 0x55d2d0a81620 2024-04-23T10:42:38.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.954+0000 7f6c777fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2d0a85bb0 0x55d2d0a91bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:38.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.954+0000 7f6c777fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2d0a85bb0 0x55d2d0a91bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48194/0 (socket says 172.21.15.97:48194) 2024-04-23T10:42:38.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.954+0000 7f6c777fe640 1 -- 172.21.15.97:0/3620677419 learned_addr learned my addr 172.21.15.97:0/3620677419 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:42:38.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.954+0000 7f6c76ffd640 1 --2- 172.21.15.97:0/3620677419 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2d0a81620 0x55d2d0aa0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:38.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.954+0000 7f6c777fe640 1 -- 172.21.15.97:0/3620677419 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2d0a81620 msgr2=0x55d2d0aa0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:42:38.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.954+0000 7f6c777fe640 1 --2- 172.21.15.97:0/3620677419 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2d0a81620 0x55d2d0aa0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:38.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.954+0000 7f6c777fe640 1 -- 172.21.15.97:0/3620677419 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2d0a85200 msgr2=0x55d2d0a87250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:42:38.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.954+0000 7f6c767fc640 1 --2- 172.21.15.97:0/3620677419 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2d0a85200 0x55d2d0a87250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:38.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.954+0000 7f6c777fe640 1 --2- 172.21.15.97:0/3620677419 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2d0a85200 0x55d2d0a87250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:38.960 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.954+0000 7f6c777fe640 1 -- 172.21.15.97:0/3620677419 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2d06bd310 con 0x55d2d0a85bb0 2024-04-23T10:42:38.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.954+0000 7f6c76ffd640 1 --2- 172.21.15.97:0/3620677419 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2d0a81620 0x55d2d0aa0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:42:38.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.954+0000 7f6c777fe640 1 --2- 172.21.15.97:0/3620677419 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2d0a85bb0 0x55d2d0a91bb0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f6c70010810 tx=0x7f6c70004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=520889a714e99a04 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:42:38.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.954+0000 7f6c5d7fa640 1 -- 172.21.15.97:0/3620677419 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c70010cd0 con 0x55d2d0a85bb0 2024-04-23T10:42:38.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.955+0000 7f6c5d7fa640 1 -- 172.21.15.97:0/3620677419 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c70023070 con 0x55d2d0a85bb0 2024-04-23T10:42:38.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.955+0000 7f6c5d7fa640 1 -- 172.21.15.97:0/3620677419 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c7001fae0 con 0x55d2d0a85bb0 2024-04-23T10:42:38.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.955+0000 7f6c80601740 1 -- 172.21.15.97:0/3620677419 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2d0a85bb0 msgr2=0x55d2d0a91bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:38.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.955+0000 7f6c80601740 1 --2- 172.21.15.97:0/3620677419 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2d0a85bb0 0x55d2d0a91bb0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f6c70010810 tx=0x7f6c70004ab0 comp rx=0 tx=0).stop 2024-04-23T10:42:38.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.955+0000 7f6c80601740 1 -- 172.21.15.97:0/3620677419 shutdown_connections 2024-04-23T10:42:38.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.955+0000 7f6c80601740 1 --2- 172.21.15.97:0/3620677419 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2d0a85bb0 0x55d2d0a91bb0 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:38.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.955+0000 7f6c80601740 1 --2- 172.21.15.97:0/3620677419 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2d0a85200 0x55d2d0a87250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:38.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.955+0000 7f6c80601740 1 --2- 172.21.15.97:0/3620677419 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2d0a81620 0x55d2d0aa0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:38.961 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.955+0000 7f6c80601740 1 -- 172.21.15.97:0/3620677419 >> 172.21.15.97:0/3620677419 conn(0x55d2d0a8f560 msgr2=0x55d2d0a80940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:42:38.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.955+0000 7f6c80601740 1 -- 172.21.15.97:0/3620677419 shutdown_connections 2024-04-23T10:42:38.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.955+0000 7f6c80601740 1 -- 172.21.15.97:0/3620677419 wait complete. 2024-04-23T10:42:38.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.955+0000 7f6c80601740 1 -- start start 2024-04-23T10:42:38.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.956+0000 7f6c80601740 1 Processor -- start 2024-04-23T10:42:38.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.956+0000 7f6c80601740 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2d0a81620 0x55d2d0a9cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:38.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.956+0000 7f6c80601740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2d0a85200 0x55d2d0a9d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:38.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.956+0000 7f6c80601740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2d0a85bb0 0x55d2d0a99b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:38.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.956+0000 7f6c80601740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6c781ace60 con 0x55d2d0a85bb0 2024-04-23T10:42:38.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.956+0000 7f6c80601740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6c781ac790 con 0x55d2d0a85200 2024-04-23T10:42:38.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.956+0000 7f6c80601740 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6c781a99f0 con 0x55d2d0a81620 2024-04-23T10:42:38.962 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.956+0000 7f6c80601740 10 client.0 _ll_register_callbacks cb 0x7f6c68005da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T10:42:38.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.956+0000 7f6c777fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2d0a85bb0 0x55d2d0a99b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:38.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.956+0000 7f6c777fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2d0a85bb0 0x55d2d0a99b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48204/0 (socket says 172.21.15.97:48204) 2024-04-23T10:42:38.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.956+0000 7f6c777fe640 1 -- 172.21.15.97:0/1509538730 learned_addr learned my addr 172.21.15.97:0/1509538730 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:42:38.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.956+0000 7f6c76ffd640 1 --2- 172.21.15.97:0/1509538730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2d0a81620 0x55d2d0a9cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:38.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.956+0000 7f6c777fe640 1 -- 172.21.15.97:0/1509538730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2d0a81620 msgr2=0x55d2d0a9cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:42:38.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.956+0000 7f6c767fc640 1 --2- 172.21.15.97:0/1509538730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2d0a85200 0x55d2d0a9d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:38.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.956+0000 7f6c777fe640 1 --2- 172.21.15.97:0/1509538730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2d0a81620 0x55d2d0a9cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:38.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.956+0000 7f6c777fe640 1 -- 172.21.15.97:0/1509538730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2d0a85200 msgr2=0x55d2d0a9d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T10:42:38.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.956+0000 7f6c777fe640 1 --2- 172.21.15.97:0/1509538730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2d0a85200 0x55d2d0a9d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:38.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.956+0000 7f6c777fe640 1 -- 172.21.15.97:0/1509538730 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2d0a9a390 con 0x55d2d0a85bb0 2024-04-23T10:42:38.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.956+0000 7f6c76ffd640 1 --2- 172.21.15.97:0/1509538730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2d0a81620 0x55d2d0a9cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:42:38.963 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.956+0000 7f6c777fe640 1 --2- 172.21.15.97:0/1509538730 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2d0a85bb0 0x55d2d0a99b60 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f6c70006010 tx=0x7f6c70010f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3d1b8bb2c2ad2e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:42:38.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.956+0000 7f6c56ffd640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-23T10:42:38.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.957+0000 7f6c56ffd640 1 -- 172.21.15.97:0/1509538730 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c7000fb90 con 0x55d2d0a85bb0 2024-04-23T10:42:38.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.957+0000 7f6c56ffd640 1 -- 172.21.15.97:0/1509538730 <== mon.0 v2:172.21.15.97:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8484+0+0 (secure 0 0 0) 0x7f6c7000fd30 con 0x55d2d0a85bb0 2024-04-23T10:42:38.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.957+0000 7f6c80601740 1 -- 172.21.15.97:0/1509538730 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c70023070 con 0x55d2d0a85bb0 2024-04-23T10:42:38.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.957+0000 7f6c56ffd640 1 -- 172.21.15.97:0/1509538730 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6c7001f720 con 0x55d2d0a85bb0 2024-04-23T10:42:38.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.957+0000 7f6c80601740 10 client.17859 fetch_fsmap learned FSMap version 26 2024-04-23T10:42:38.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.957+0000 7f6c80601740 1 -- 172.21.15.97:0/1509538730 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d2d0b0b930 con 0x55d2d0a85bb0 2024-04-23T10:42:38.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.957+0000 7f6c56ffd640 1 -- 172.21.15.97:0/1509538730 <== mon.0 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c70023070 con 0x55d2d0a85bb0 2024-04-23T10:42:38.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.957+0000 7f6c56ffd640 1 -- 172.21.15.97:0/1509538730 <== mon.0 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f6c70038020 con 0x55d2d0a85bb0 2024-04-23T10:42:38.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.957+0000 7f6c80601740 10 client.17859 fetch_fsmap finished waiting for FSMap version 26 2024-04-23T10:42:38.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.957+0000 7f6c80601740 10 client.17859 resolve_mds: validated gid 24488 aka daemon mds.b 2024-04-23T10:42:38.964 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.957+0000 7f6c80601740 20 client.17859 populate_metadata read hostname 'smithi097' 2024-04-23T10:42:38.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.957+0000 7f6c80601740 1 --2- 172.21.15.97:0/1509538730 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55d2d0a907c0 0x55d2d0b0e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:38.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.957+0000 7f6c80601740 4 client.17859 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-04-23T10:42:38.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.957+0000 7f6c80601740 1 -- 172.21.15.97:0/1509538730 --> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x55d2d09f29c0 con 0x55d2d0a907c0 2024-04-23T10:42:38.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.957+0000 7f6c76ffd640 1 --2- 172.21.15.97:0/1509538730 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55d2d0a907c0 0x55d2d0b0e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:38.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.958+0000 7f6c76ffd640 1 --2- 172.21.15.97:0/1509538730 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55d2d0a907c0 0x55d2d0b0e0a0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be5050eb9ca2f8a0 server_cookie=af874519f57908c1 in_seq=0 out_seq=0 2024-04-23T10:42:38.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.958+0000 7f6c56ffd640 10 client.17859 ms_handle_connect on v2:172.21.15.183:6838/1763841130 2024-04-23T10:42:38.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.958+0000 7f6c56ffd640 1 -- 172.21.15.97:0/1509538730 <== mds.0 v2:172.21.15.183:6838/1763841130 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x55d2d09f29c0 con 0x55d2d0a907c0 2024-04-23T10:42:38.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.958+0000 7f6c56ffd640 10 client.17859 handle_command_reply: tid=0 2024-04-23T10:42:38.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.958+0000 7f6c80601740 1 client.17859 shutdown 2024-04-23T10:42:38.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.958+0000 7f6c80601740 1 -- 172.21.15.97:0/1509538730 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2d0a85bb0 msgr2=0x55d2d0a99b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:38.965 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.958+0000 7f6c80601740 1 --2- 172.21.15.97:0/1509538730 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2d0a85bb0 0x55d2d0a99b60 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f6c70006010 tx=0x7f6c70010f00 comp rx=0 tx=0).stop 2024-04-23T10:42:38.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.958+0000 7f6c80601740 1 -- 172.21.15.97:0/1509538730 shutdown_connections 2024-04-23T10:42:38.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.958+0000 7f6c80601740 1 --2- 172.21.15.97:0/1509538730 >> [v2:172.21.15.183:6838/1763841130,v1:172.21.15.183:6839/1763841130] conn(0x55d2d0a907c0 0x55d2d0b0e0a0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:38.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.958+0000 7f6c80601740 1 --2- 172.21.15.97:0/1509538730 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d2d0a85bb0 0x55d2d0a99b60 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:38.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.958+0000 7f6c80601740 1 --2- 172.21.15.97:0/1509538730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d2d0a85200 0x55d2d0a9d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:38.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.958+0000 7f6c56ffd640 0 client.17859 ms_handle_reset on v2:172.21.15.183:6838/1763841130 2024-04-23T10:42:38.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.958+0000 7f6c80601740 1 --2- 172.21.15.97:0/1509538730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2d0a81620 0x55d2d0a9cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:38.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.958+0000 7f6c80601740 1 -- 172.21.15.97:0/1509538730 >> 172.21.15.97:0/1509538730 conn(0x55d2d0a8f560 msgr2=0x55d2d0a800b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:42:38.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.958+0000 7f6c80601740 1 -- 172.21.15.97:0/1509538730 shutdown_connections 2024-04-23T10:42:38.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.958+0000 7f6c80601740 1 -- 172.21.15.97:0/1509538730 wait complete. 2024-04-23T10:42:38.966 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.958+0000 7f6c80601740 20 client.17859 trim_cache size 0 max 16384 2024-04-23T10:42:38.967 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-23T10:42:38.967 INFO:teuthology.orchestra.run.smithi097.stdout: "epoch": 26, 2024-04-23T10:42:38.967 INFO:teuthology.orchestra.run.smithi097.stdout: "leader": 24488, 2024-04-23T10:42:38.967 INFO:teuthology.orchestra.run.smithi097.stdout: "set_version": 68, 2024-04-23T10:42:38.967 INFO:teuthology.orchestra.run.smithi097.stdout: "sets": {} 2024-04-23T10:42:38.967 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-23T10:42:38.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.959+0000 7f6c7d810640 1 -- 172.21.15.97:0/1313853024 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f6c600a3d20 msgr2=0x7f6c600a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:38.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.959+0000 7f6c7d810640 1 --2- 172.21.15.97:0/1313853024 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f6c600a3d20 0x7f6c600a61e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6c70005c40 tx=0x7f6c70009280 comp rx=0 tx=0).stop 2024-04-23T10:42:38.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.959+0000 7f6c7d810640 1 -- 172.21.15.97:0/1313853024 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c7810c4f0 msgr2=0x7f6c7819eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:38.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.959+0000 7f6c7d810640 1 --2- 172.21.15.97:0/1313853024 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c7810c4f0 0x7f6c7819eb00 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c02a930 tx=0x7f6c6c004270 comp rx=0 tx=0).stop 2024-04-23T10:42:38.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.959+0000 7f6c7d810640 1 -- 172.21.15.97:0/1313853024 shutdown_connections 2024-04-23T10:42:38.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.959+0000 7f6c7d810640 1 --2- 172.21.15.97:0/1313853024 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f6c600a3d20 0x7f6c600a61e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:38.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.959+0000 7f6c7d810640 1 --2- 172.21.15.97:0/1313853024 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6c781986d0 0x7f6c78198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:38.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.959+0000 7f6c7d810640 1 --2- 172.21.15.97:0/1313853024 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c7810c4f0 0x7f6c7819eb00 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:38.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.959+0000 7f6c7d810640 1 --2- 172.21.15.97:0/1313853024 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c78100d20 0x7f6c7819e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:38.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.959+0000 7f6c7d810640 1 -- 172.21.15.97:0/1313853024 >> 172.21.15.97:0/1313853024 conn(0x7f6c78076150 msgr2=0x7f6c780ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:42:38.969 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.959+0000 7f6c7d810640 1 -- 172.21.15.97:0/1313853024 shutdown_connections 2024-04-23T10:42:38.970 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:38.959+0000 7f6c7d810640 1 -- 172.21.15.97:0/1313853024 wait complete. 2024-04-23T10:42:38.970 INFO:tasks.quiescer:done joining Quiescers 2024-04-23T10:42:38.970 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-04-23T10:42:38.980 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi097.front.sepia.ceph.com 2024-04-23T10:42:38.989 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-04-23T10:42:39.098 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-04-23T10:42:39.099 INFO:teuthology.task.exec:Executing custom commands... 2024-04-23T10:42:39.099 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi097.front.sepia.ceph.com 2024-04-23T10:42:39.099 DEBUG:teuthology.orchestra.run.smithi097:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-04-23T10:42:39.125 DEBUG:teuthology.orchestra.run.smithi097:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-04-23T10:42:39.191 DEBUG:teuthology.orchestra.run.smithi097:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-23T10:42:39.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.332+0000 7f3da6493640 1 -- 172.21.15.97:0/2420754217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3da0106190 msgr2=0x7f3da010dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:39.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.332+0000 7f3da6493640 1 --2- 172.21.15.97:0/2420754217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3da0106190 0x7f3da010dd60 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f3d98009ca0 tx=0x7f3d9802a8c0 comp rx=0 tx=0).stop 2024-04-23T10:42:39.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.332+0000 7f3da6493640 1 -- 172.21.15.97:0/2420754217 shutdown_connections 2024-04-23T10:42:39.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.332+0000 7f3da6493640 1 --2- 172.21.15.97:0/2420754217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3da0106190 0x7f3da010dd60 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:39.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.332+0000 7f3da6493640 1 --2- 172.21.15.97:0/2420754217 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3da0101ac0 0x7f3da0105ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:39.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.332+0000 7f3da6493640 1 --2- 172.21.15.97:0/2420754217 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3da0101110 0x7f3da01014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:39.334 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.332+0000 7f3da6493640 1 -- 172.21.15.97:0/2420754217 >> 172.21.15.97:0/2420754217 conn(0x7f3da0077fd0 msgr2=0x7f3da00783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:42:39.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.332+0000 7f3da6493640 1 -- 172.21.15.97:0/2420754217 shutdown_connections 2024-04-23T10:42:39.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.332+0000 7f3da6493640 1 -- 172.21.15.97:0/2420754217 wait complete. 2024-04-23T10:42:39.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.333+0000 7f3da6493640 1 Processor -- start 2024-04-23T10:42:39.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.333+0000 7f3da6493640 1 -- start start 2024-04-23T10:42:39.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.333+0000 7f3da6493640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3da0101110 0x7f3da01972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:39.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.333+0000 7f3da6493640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3da0106190 0x7f3da0197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:39.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.333+0000 7f3da6493640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3da0197d40 0x7f3da01a6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:39.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.333+0000 7f3da6493640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3da0111170 con 0x7f3da0101110 2024-04-23T10:42:39.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.333+0000 7f3da6493640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3da0110ff0 con 0x7f3da0197d40 2024-04-23T10:42:39.335 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.333+0000 7f3da6493640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f3da01112f0 con 0x7f3da0106190 2024-04-23T10:42:39.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.333+0000 7f3d9ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3da0101110 0x7f3da01972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:39.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.333+0000 7f3d9ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3da0101110 0x7f3da01972c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48220/0 (socket says 172.21.15.97:48220) 2024-04-23T10:42:39.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.333+0000 7f3d9ffff640 1 -- 172.21.15.97:0/3296853751 learned_addr learned my addr 172.21.15.97:0/3296853751 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:42:39.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.333+0000 7f3da4a30640 1 --2- 172.21.15.97:0/3296853751 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3da0197d40 0x7f3da01a6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:39.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.333+0000 7f3d9f7fe640 1 --2- 172.21.15.97:0/3296853751 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3da0106190 0x7f3da0197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:39.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.333+0000 7f3d9ffff640 1 -- 172.21.15.97:0/3296853751 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3da0106190 msgr2=0x7f3da0197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:39.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.333+0000 7f3d9ffff640 1 --2- 172.21.15.97:0/3296853751 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3da0106190 0x7f3da0197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:39.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.333+0000 7f3d9ffff640 1 -- 172.21.15.97:0/3296853751 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3da0197d40 msgr2=0x7f3da01a6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:39.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.333+0000 7f3d9ffff640 1 --2- 172.21.15.97:0/3296853751 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3da0197d40 0x7f3da01a6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:39.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.333+0000 7f3d9ffff640 1 -- 172.21.15.97:0/3296853751 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3da01a6f00 con 0x7f3da0101110 2024-04-23T10:42:39.336 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.334+0000 7f3d9ffff640 1 --2- 172.21.15.97:0/3296853751 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3da0101110 0x7f3da01972c0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f3d9000c9a0 tx=0x7f3d9000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:42:39.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.334+0000 7f3d9d7fa640 1 -- 172.21.15.97:0/3296853751 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d90007b50 con 0x7f3da0101110 2024-04-23T10:42:39.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.334+0000 7f3d9d7fa640 1 -- 172.21.15.97:0/3296853751 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d90007cf0 con 0x7f3da0101110 2024-04-23T10:42:39.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.334+0000 7f3da6493640 1 -- 172.21.15.97:0/3296853751 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3da01a71f0 con 0x7f3da0101110 2024-04-23T10:42:39.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.334+0000 7f3d9d7fa640 1 -- 172.21.15.97:0/3296853751 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d9000f450 con 0x7f3da0101110 2024-04-23T10:42:39.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.334+0000 7f3da6493640 1 -- 172.21.15.97:0/3296853751 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3da01a76f0 con 0x7f3da0101110 2024-04-23T10:42:39.337 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.335+0000 7f3da6493640 1 -- 172.21.15.97:0/3296853751 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3da004fce0 con 0x7f3da0101110 2024-04-23T10:42:39.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.336+0000 7f3d9d7fa640 1 -- 172.21.15.97:0/3296853751 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3d90004eb0 con 0x7f3da0101110 2024-04-23T10:42:39.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.337+0000 7f3d9d7fa640 1 --2- 172.21.15.97:0/3296853751 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f3d880a3dc0 0x7f3d880a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:39.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.337+0000 7f3d9f7fe640 1 --2- 172.21.15.97:0/3296853751 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f3d880a3dc0 0x7f3d880a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:39.338 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.337+0000 7f3d9d7fa640 1 -- 172.21.15.97:0/3296853751 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8484+0+0 (secure 0 0 0) 0x7f3d900d0630 con 0x7f3da0101110 2024-04-23T10:42:39.339 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.337+0000 7f3d9f7fe640 1 --2- 172.21.15.97:0/3296853751 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f3d880a3dc0 0x7f3d880a6280 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3da0199420 tx=0x7f3d94008040 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:42:39.340 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.341+0000 7f3d9d7fa640 1 -- 172.21.15.97:0/3296853751 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3d90014030 con 0x7f3da0101110 2024-04-23T10:42:39.541 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.542+0000 7f3da6493640 1 -- 172.21.15.97:0/3296853751 --> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f3da0101b50 con 0x7f3d880a3dc0 2024-04-23T10:42:39.549 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.550+0000 7f3d9d7fa640 1 -- 172.21.15.97:0/3296853751 <== mgr.15378 v2:172.21.15.97:6800/3567268502 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+351 (secure 0 0 0) 0x7f3da0101b50 con 0x7f3d880a3dc0 2024-04-23T10:42:39.549 INFO:teuthology.orchestra.run.smithi097.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-23T00:00:00", "created": "2024-04-23T10:26:12", "first": "2024-04-23T10:27:00", "last": "2024-04-23T10:42:00", "last_pruned": "2024-04-23T10:42:00", "created_count": 16, "pruned_count": 10, "active": true} 2024-04-23T10:42:39.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.552+0000 7f3da6493640 1 -- 172.21.15.97:0/3296853751 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f3d880a3dc0 msgr2=0x7f3d880a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:39.551 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.552+0000 7f3da6493640 1 --2- 172.21.15.97:0/3296853751 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f3d880a3dc0 0x7f3d880a6280 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3da0199420 tx=0x7f3d94008040 comp rx=0 tx=0).stop 2024-04-23T10:42:39.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.552+0000 7f3da6493640 1 -- 172.21.15.97:0/3296853751 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3da0101110 msgr2=0x7f3da01972c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:39.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.552+0000 7f3da6493640 1 --2- 172.21.15.97:0/3296853751 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3da0101110 0x7f3da01972c0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f3d9000c9a0 tx=0x7f3d9000ce60 comp rx=0 tx=0).stop 2024-04-23T10:42:39.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.552+0000 7f3da6493640 1 -- 172.21.15.97:0/3296853751 shutdown_connections 2024-04-23T10:42:39.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.552+0000 7f3da6493640 1 --2- 172.21.15.97:0/3296853751 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f3d880a3dc0 0x7f3d880a6280 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:39.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.552+0000 7f3da6493640 1 --2- 172.21.15.97:0/3296853751 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3da0197d40 0x7f3da01a6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:39.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.552+0000 7f3da6493640 1 --2- 172.21.15.97:0/3296853751 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3da0106190 0x7f3da0197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:39.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.552+0000 7f3da6493640 1 --2- 172.21.15.97:0/3296853751 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3da0101110 0x7f3da01972c0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:39.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.552+0000 7f3da6493640 1 -- 172.21.15.97:0/3296853751 >> 172.21.15.97:0/3296853751 conn(0x7f3da0077fd0 msgr2=0x7f3da00fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:42:39.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.552+0000 7f3da6493640 1 -- 172.21.15.97:0/3296853751 shutdown_connections 2024-04-23T10:42:39.552 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.552+0000 7f3da6493640 1 -- 172.21.15.97:0/3296853751 wait complete. 2024-04-23T10:42:39.562 DEBUG:teuthology.orchestra.run.smithi097:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-23T10:42:39.666 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.664+0000 7fd508efd640 1 -- 172.21.15.97:0/1232515839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5041006c0 msgr2=0x7fd504100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:39.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.664+0000 7fd508efd640 1 --2- 172.21.15.97:0/1232515839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5041006c0 0x7fd504100aa0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fd4f4009900 tx=0x7fd4f402a410 comp rx=0 tx=0).stop 2024-04-23T10:42:39.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.664+0000 7fd508efd640 1 -- 172.21.15.97:0/1232515839 shutdown_connections 2024-04-23T10:42:39.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.664+0000 7fd508efd640 1 --2- 172.21.15.97:0/1232515839 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd504107280 0x7fd50410ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:39.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.664+0000 7fd508efd640 1 --2- 172.21.15.97:0/1232515839 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd504101070 0x7fd504106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:39.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.664+0000 7fd508efd640 1 --2- 172.21.15.97:0/1232515839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5041006c0 0x7fd504100aa0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:39.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.664+0000 7fd508efd640 1 -- 172.21.15.97:0/1232515839 >> 172.21.15.97:0/1232515839 conn(0x7fd504077390 msgr2=0x7fd5040fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:42:39.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.664+0000 7fd508efd640 1 -- 172.21.15.97:0/1232515839 shutdown_connections 2024-04-23T10:42:39.667 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.664+0000 7fd508efd640 1 -- 172.21.15.97:0/1232515839 wait complete. 2024-04-23T10:42:39.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.665+0000 7fd508efd640 1 Processor -- start 2024-04-23T10:42:39.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.665+0000 7fd508efd640 1 -- start start 2024-04-23T10:42:39.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.665+0000 7fd508efd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5041006c0 0x7fd50419e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:39.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.665+0000 7fd508efd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd504107280 0x7fd50419e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:39.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.665+0000 7fd508efd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5041985a0 0x7fd504198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:39.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.665+0000 7fd508efd640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd5041121c0 con 0x7fd504107280 2024-04-23T10:42:39.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.665+0000 7fd508efd640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd504112040 con 0x7fd5041006c0 2024-04-23T10:42:39.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.665+0000 7fd508efd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fd504112340 con 0x7fd5041985a0 2024-04-23T10:42:39.668 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.665+0000 7fd4f9d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd504107280 0x7fd50419e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:39.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.665+0000 7fd4f9d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd504107280 0x7fd50419e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48238/0 (socket says 172.21.15.97:48238) 2024-04-23T10:42:39.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.665+0000 7fd4f9d9b640 1 -- 172.21.15.97:0/797276354 learned_addr learned my addr 172.21.15.97:0/797276354 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:42:39.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.665+0000 7fd502d9d640 1 --2- 172.21.15.97:0/797276354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5041985a0 0x7fd504198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:39.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.665+0000 7fd50259c640 1 --2- 172.21.15.97:0/797276354 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5041006c0 0x7fd50419e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:39.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.665+0000 7fd4f9d9b640 1 -- 172.21.15.97:0/797276354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5041985a0 msgr2=0x7fd504198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:39.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.665+0000 7fd4f9d9b640 1 --2- 172.21.15.97:0/797276354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5041985a0 0x7fd504198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:39.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.665+0000 7fd4f9d9b640 1 -- 172.21.15.97:0/797276354 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5041006c0 msgr2=0x7fd50419e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:39.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.665+0000 7fd4f9d9b640 1 --2- 172.21.15.97:0/797276354 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5041006c0 0x7fd50419e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:39.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.665+0000 7fd4f9d9b640 1 -- 172.21.15.97:0/797276354 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5041990c0 con 0x7fd504107280 2024-04-23T10:42:39.669 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.665+0000 7fd50259c640 1 --2- 172.21.15.97:0/797276354 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5041006c0 0x7fd50419e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:42:39.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.666+0000 7fd4f9d9b640 1 --2- 172.21.15.97:0/797276354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd504107280 0x7fd50419e930 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec00d770 tx=0x7fd4ec00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:42:39.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.666+0000 7fd4fbfff640 1 -- 172.21.15.97:0/797276354 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4ec017070 con 0x7fd504107280 2024-04-23T10:42:39.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.666+0000 7fd4fbfff640 1 -- 172.21.15.97:0/797276354 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4ec009c30 con 0x7fd504107280 2024-04-23T10:42:39.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.666+0000 7fd508efd640 1 -- 172.21.15.97:0/797276354 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5041993b0 con 0x7fd504107280 2024-04-23T10:42:39.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.666+0000 7fd4fbfff640 1 -- 172.21.15.97:0/797276354 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4ec00f620 con 0x7fd504107280 2024-04-23T10:42:39.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.666+0000 7fd508efd640 1 -- 172.21.15.97:0/797276354 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5041a5600 con 0x7fd504107280 2024-04-23T10:42:39.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.667+0000 7fd508efd640 1 -- 172.21.15.97:0/797276354 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd50404fce0 con 0x7fd504107280 2024-04-23T10:42:39.670 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.668+0000 7fd4fbfff640 1 -- 172.21.15.97:0/797276354 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd4ec01a070 con 0x7fd504107280 2024-04-23T10:42:39.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.668+0000 7fd4fbfff640 1 --2- 172.21.15.97:0/797276354 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fd4f00a3dc0 0x7fd4f00a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:39.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.669+0000 7fd50259c640 1 --2- 172.21.15.97:0/797276354 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fd4f00a3dc0 0x7fd4f00a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:39.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.669+0000 7fd4fbfff640 1 -- 172.21.15.97:0/797276354 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8484+0+0 (secure 0 0 0) 0x7fd4ec0d0350 con 0x7fd504107280 2024-04-23T10:42:39.671 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.669+0000 7fd50259c640 1 --2- 172.21.15.97:0/797276354 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fd4f00a3dc0 0x7fd4f00a6280 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd4f4000c00 tx=0x7fd4f4031400 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:42:39.672 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.673+0000 7fd4fbfff640 1 -- 172.21.15.97:0/797276354 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd4ec009de0 con 0x7fd504107280 2024-04-23T10:42:39.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:42:39 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/2535469030' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:42:39.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:42:39 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/2535469030' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:42:39.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.874+0000 7fd508efd640 1 -- 172.21.15.97:0/797276354 --> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fd504101070 con 0x7fd4f00a3dc0 2024-04-23T10:42:39.880 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.881+0000 7fd4fbfff640 1 -- 172.21.15.97:0/797276354 <== mgr.15378 v2:172.21.15.97:6800/3567268502 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fd504101070 con 0x7fd4f00a3dc0 2024-04-23T10:42:39.880 INFO:teuthology.orchestra.run.smithi097.stdout:/ 1m 6m3h 2024-04-23T10:42:39.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.883+0000 7fd508efd640 1 -- 172.21.15.97:0/797276354 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fd4f00a3dc0 msgr2=0x7fd4f00a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:39.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.883+0000 7fd508efd640 1 --2- 172.21.15.97:0/797276354 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fd4f00a3dc0 0x7fd4f00a6280 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd4f4000c00 tx=0x7fd4f4031400 comp rx=0 tx=0).stop 2024-04-23T10:42:39.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.883+0000 7fd508efd640 1 -- 172.21.15.97:0/797276354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd504107280 msgr2=0x7fd50419e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:39.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.883+0000 7fd508efd640 1 --2- 172.21.15.97:0/797276354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd504107280 0x7fd50419e930 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec00d770 tx=0x7fd4ec00dc30 comp rx=0 tx=0).stop 2024-04-23T10:42:39.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.883+0000 7fd508efd640 1 -- 172.21.15.97:0/797276354 shutdown_connections 2024-04-23T10:42:39.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.883+0000 7fd508efd640 1 --2- 172.21.15.97:0/797276354 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fd4f00a3dc0 0x7fd4f00a6280 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:39.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.883+0000 7fd508efd640 1 --2- 172.21.15.97:0/797276354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5041985a0 0x7fd504198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:39.883 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.883+0000 7fd508efd640 1 --2- 172.21.15.97:0/797276354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd504107280 0x7fd50419e930 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:39.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.883+0000 7fd508efd640 1 --2- 172.21.15.97:0/797276354 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5041006c0 0x7fd50419e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:39.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.883+0000 7fd508efd640 1 -- 172.21.15.97:0/797276354 >> 172.21.15.97:0/797276354 conn(0x7fd504077390 msgr2=0x7fd5041054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:42:39.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.884+0000 7fd508efd640 1 -- 172.21.15.97:0/797276354 shutdown_connections 2024-04-23T10:42:39.884 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:39.884+0000 7fd508efd640 1 -- 172.21.15.97:0/797276354 wait complete. 2024-04-23T10:42:39.894 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-23T10:42:39.905 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T10:42:39.906 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:42:39 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/2535469030' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:42:40.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.005+0000 7f6d65c79640 1 -- 172.21.15.97:0/496523439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d60101a50 msgr2=0x7f6d60105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:40.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.005+0000 7f6d65c79640 1 --2- 172.21.15.97:0/496523439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d60101a50 0x7f6d60105a90 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f6d50009900 tx=0x7f6d5002a410 comp rx=0 tx=0).stop 2024-04-23T10:42:40.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.005+0000 7f6d65c79640 1 -- 172.21.15.97:0/496523439 shutdown_connections 2024-04-23T10:42:40.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.005+0000 7f6d65c79640 1 --2- 172.21.15.97:0/496523439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d60106170 0x7f6d6010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:40.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.005+0000 7f6d65c79640 1 --2- 172.21.15.97:0/496523439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d60101a50 0x7f6d60105a90 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:40.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.005+0000 7f6d65c79640 1 --2- 172.21.15.97:0/496523439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d601010a0 0x7f6d60101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:40.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.005+0000 7f6d65c79640 1 -- 172.21.15.97:0/496523439 >> 172.21.15.97:0/496523439 conn(0x7f6d60077f50 msgr2=0x7f6d60078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:42:40.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.005+0000 7f6d65c79640 1 -- 172.21.15.97:0/496523439 shutdown_connections 2024-04-23T10:42:40.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.006+0000 7f6d65c79640 1 -- 172.21.15.97:0/496523439 wait complete. 2024-04-23T10:42:40.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.006+0000 7f6d65c79640 1 Processor -- start 2024-04-23T10:42:40.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.006+0000 7f6d65c79640 1 -- start start 2024-04-23T10:42:40.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.006+0000 7f6d65c79640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d601010a0 0x7f6d60198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:40.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.006+0000 7f6d65c79640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d60106170 0x7f6d60198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:40.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.006+0000 7f6d65c79640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d6019cfd0 0x7f6d6019d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:40.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.006+0000 7f6d65c79640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6d601121b0 con 0x7f6d60106170 2024-04-23T10:42:40.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.006+0000 7f6d65c79640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6d60112030 con 0x7f6d6019cfd0 2024-04-23T10:42:40.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.006+0000 7f6d65c79640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f6d60112330 con 0x7f6d601010a0 2024-04-23T10:42:40.009 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.007+0000 7f6d5effd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d60106170 0x7f6d60198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:40.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.007+0000 7f6d5effd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d60106170 0x7f6d60198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48260/0 (socket says 172.21.15.97:48260) 2024-04-23T10:42:40.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.007+0000 7f6d5effd640 1 -- 172.21.15.97:0/1747501229 learned_addr learned my addr 172.21.15.97:0/1747501229 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:42:40.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.007+0000 7f6d5ffff640 1 --2- 172.21.15.97:0/1747501229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d6019cfd0 0x7f6d6019d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:40.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.007+0000 7f6d5f7fe640 1 --2- 172.21.15.97:0/1747501229 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d601010a0 0x7f6d60198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:40.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.007+0000 7f6d5effd640 1 -- 172.21.15.97:0/1747501229 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d601010a0 msgr2=0x7f6d60198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:40.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.007+0000 7f6d5effd640 1 --2- 172.21.15.97:0/1747501229 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d601010a0 0x7f6d60198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:40.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.007+0000 7f6d5effd640 1 -- 172.21.15.97:0/1747501229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d6019cfd0 msgr2=0x7f6d6019d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:40.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.007+0000 7f6d5effd640 1 --2- 172.21.15.97:0/1747501229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d6019cfd0 0x7f6d6019d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:40.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.007+0000 7f6d5effd640 1 -- 172.21.15.97:0/1747501229 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d6019dac0 con 0x7f6d60106170 2024-04-23T10:42:40.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.007+0000 7f6d5ffff640 1 --2- 172.21.15.97:0/1747501229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d6019cfd0 0x7f6d6019d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:42:40.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.007+0000 7f6d5effd640 1 --2- 172.21.15.97:0/1747501229 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d60106170 0x7f6d60198c50 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f6d50032fd0 tx=0x7f6d50004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:42:40.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.007+0000 7f6d5cff9640 1 -- 172.21.15.97:0/1747501229 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5003f030 con 0x7f6d60106170 2024-04-23T10:42:40.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.007+0000 7f6d5cff9640 1 -- 172.21.15.97:0/1747501229 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d50004490 con 0x7f6d60106170 2024-04-23T10:42:40.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.007+0000 7f6d65c79640 1 -- 172.21.15.97:0/1747501229 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d6019dd50 con 0x7f6d60106170 2024-04-23T10:42:40.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.007+0000 7f6d5cff9640 1 -- 172.21.15.97:0/1747501229 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d50035070 con 0x7f6d60106170 2024-04-23T10:42:40.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.007+0000 7f6d65c79640 1 -- 172.21.15.97:0/1747501229 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d601a5510 con 0x7f6d60106170 2024-04-23T10:42:40.011 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.008+0000 7f6d65c79640 1 -- 172.21.15.97:0/1747501229 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d60068d70 con 0x7f6d60106170 2024-04-23T10:42:40.012 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.010+0000 7f6d5cff9640 1 -- 172.21.15.97:0/1747501229 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6d50042030 con 0x7f6d60106170 2024-04-23T10:42:40.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.010+0000 7f6d5cff9640 1 --2- 172.21.15.97:0/1747501229 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f6d480a3d20 0x7f6d480a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:40.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.011+0000 7f6d5f7fe640 1 --2- 172.21.15.97:0/1747501229 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f6d480a3d20 0x7f6d480a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:40.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.011+0000 7f6d5cff9640 1 -- 172.21.15.97:0/1747501229 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8484+0+0 (secure 0 0 0) 0x7f6d500f10a0 con 0x7f6d60106170 2024-04-23T10:42:40.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.011+0000 7f6d5f7fe640 1 --2- 172.21.15.97:0/1747501229 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f6d480a3d20 0x7f6d480a61e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6d5400abe0 tx=0x7f6d5400a2d0 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:42:40.013 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.014+0000 7f6d5cff9640 1 -- 172.21.15.97:0/1747501229 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6d500bd760 con 0x7f6d60106170 2024-04-23T10:42:40.192 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.193+0000 7f6d65c79640 1 -- 172.21.15.97:0/1747501229 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6d60101a50 con 0x7f6d60106170 2024-04-23T10:42:40.194 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.195+0000 7f6d5cff9640 1 -- 172.21.15.97:0/1747501229 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) ==== 74+0+23075 (secure 0 0 0) 0x7f6d500bd060 con 0x7f6d60106170 2024-04-23T10:42:40.194 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:42:40.194 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":109,"fsid":"9b22af88-0159-11ef-bc93-c7b262605968","created":"2024-04-23T10:16:05.983305+0000","modified":"2024-04-23T10:42:11.181740+0000","last_up_change":"2024-04-23T10:23:13.114761+0000","last_in_change":"2024-04-23T10:22:52.804880+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T10:19:25.559138+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T10:24:04.570049+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T10:24:05.861631+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":108,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"58cb995d-de35-4522-ac81-4982401b260d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6803","nonce":3031221802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6805","nonce":3031221802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6809","nonce":3031221802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6807","nonce":3031221802}]},"public_addr":"172.21.15.97:6803/3031221802","cluster_addr":"172.21.15.97:6805/3031221802","heartbeat_back_addr":"172.21.15.97:6809/3031221802","heartbeat_front_addr":"172.21.15.97:6807/3031221802","state":["exists","up"]},{"osd":1,"uuid":"b47bd8ee-1f21-4834-acf5-bb40f97ab3ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6801","nonce":1108870440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6803","nonce":1108870440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6807","nonce":1108870440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6805","nonce":1108870440}]},"public_addr":"172.21.15.183:6801/1108870440","cluster_addr":"172.21.15.183:6803/1108870440","heartbeat_back_addr":"172.21.15.183:6807/1108870440","heartbeat_front_addr":"172.21.15.183:6805/1108870440","state":["exists","up"]},{"osd":2,"uuid":"17c4eec3-6c47-4334-9b63-1d5ad9ec5988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6801","nonce":2877039731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6803","nonce":2877039731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6807","nonce":2877039731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6805","nonce":2877039731}]},"public_addr":"172.21.15.196:6801/2877039731","cluster_addr":"172.21.15.196:6803/2877039731","heartbeat_back_addr":"172.21.15.196:6807/2877039731","heartbeat_front_addr":"172.21.15.196:6805/2877039731","state":["exists","up"]},{"osd":3,"uuid":"3f731c38-2e45-4ca1-8dce-efc4a1b2166b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6811","nonce":4276311194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6813","nonce":4276311194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6817","nonce":4276311194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6815","nonce":4276311194}]},"public_addr":"172.21.15.97:6811/4276311194","cluster_addr":"172.21.15.97:6813/4276311194","heartbeat_back_addr":"172.21.15.97:6817/4276311194","heartbeat_front_addr":"172.21.15.97:6815/4276311194","state":["exists","up"]},{"osd":4,"uuid":"fdcafb14-9b82-4134-bd5b-79c0516f6cbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6809","nonce":63913916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6811","nonce":63913916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6815","nonce":63913916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6813","nonce":63913916}]},"public_addr":"172.21.15.183:6809/63913916","cluster_addr":"172.21.15.183:6811/63913916","heartbeat_back_addr":"172.21.15.183:6815/63913916","heartbeat_front_addr":"172.21.15.183:6813/63913916","state":["exists","up"]},{"osd":5,"uuid":"cdcc18d8-da3c-4012-bf12-876bb39e66eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6809","nonce":2804446896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6811","nonce":2804446896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6815","nonce":2804446896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6813","nonce":2804446896}]},"public_addr":"172.21.15.196:6809/2804446896","cluster_addr":"172.21.15.196:6811/2804446896","heartbeat_back_addr":"172.21.15.196:6815/2804446896","heartbeat_front_addr":"172.21.15.196:6813/2804446896","state":["exists","up"]},{"osd":6,"uuid":"071abb5c-2edf-4832-b8aa-3b80d20096fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6819","nonce":2080916705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6821","nonce":2080916705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6825","nonce":2080916705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6823","nonce":2080916705}]},"public_addr":"172.21.15.97:6819/2080916705","cluster_addr":"172.21.15.97:6821/2080916705","heartbeat_back_addr":"172.21.15.97:6825/2080916705","heartbeat_front_addr":"172.21.15.97:6823/2080916705","state":["exists","up"]},{"osd":7,"uuid":"9d54760b-66df-486b-8abb-017f3b64b4d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6817","nonce":1865873033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6819","nonce":1865873033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6823","nonce":1865873033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6821","nonce":1865873033}]},"public_addr":"172.21.15.183:6817/1865873033","cluster_addr":"172.21.15.183:6819/1865873033","heartbeat_back_addr":"172.21.15.183:6823/1865873033","heartbeat_front_addr":"172.21.15.183:6821/1865873033","state":["exists","up"]},{"osd":8,"uuid":"5cd3100a-bd6a-4be5-b5e9-0d0afc325193","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6817","nonce":4063932139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6819","nonce":4063932139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6823","nonce":4063932139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6821","nonce":4063932139}]},"public_addr":"172.21.15.196:6817/4063932139","cluster_addr":"172.21.15.196:6819/4063932139","heartbeat_back_addr":"172.21.15.196:6823/4063932139","heartbeat_front_addr":"172.21.15.196:6821/4063932139","state":["exists","up"]},{"osd":9,"uuid":"b6a5f4a1-4ecc-4f73-a2df-07551d9298ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6827","nonce":1887432474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6829","nonce":1887432474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6833","nonce":1887432474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6831","nonce":1887432474}]},"public_addr":"172.21.15.97:6827/1887432474","cluster_addr":"172.21.15.97:6829/1887432474","heartbeat_back_addr":"172.21.15.97:6833/1887432474","heartbeat_front_addr":"172.21.15.97:6831/1887432474","state":["exists","up"]},{"osd":10,"uuid":"86f59816-1ea6-4ae9-8796-cf5fc4831064","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6825","nonce":799434647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6827","nonce":799434647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6831","nonce":799434647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6829","nonce":799434647}]},"public_addr":"172.21.15.183:6825/799434647","cluster_addr":"172.21.15.183:6827/799434647","heartbeat_back_addr":"172.21.15.183:6831/799434647","heartbeat_front_addr":"172.21.15.183:6829/799434647","state":["exists","up"]},{"osd":11,"uuid":"95d64a89-a9f8-4c7a-8c0d-da7fb26cfff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6825","nonce":1873969467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6827","nonce":1873969467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6831","nonce":1873969467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6829","nonce":1873969467}]},"public_addr":"172.21.15.196:6825/1873969467","cluster_addr":"172.21.15.196:6827/1873969467","heartbeat_back_addr":"172.21.15.196:6831/1873969467","heartbeat_front_addr":"172.21.15.196:6829/1873969467","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:18:33.721156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:18:57.586709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:19:22.069458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:19:46.451597+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:20:11.561019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:20:36.298672+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:21:01.661804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:21:27.324128+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:21:52.898197+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:22:19.614326+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:22:45.154917+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:23:11.222612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/233335470":"2024-04-24T10:26:11.109535+0000","172.21.15.97:6800/3149043749":"2024-04-24T10:26:11.109535+0000","172.21.15.97:0/338277142":"2024-04-24T10:26:11.109535+0000","172.21.15.97:0/3866016040":"2024-04-24T10:16:59.468237+0000","172.21.15.97:6801/3297166858":"2024-04-24T10:16:59.468237+0000","172.21.15.97:0/3804743548":"2024-04-24T10:16:59.468237+0000","172.21.15.97:6800/3706463017":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/3715079438":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/597405418":"2024-04-24T10:26:11.109535+0000","172.21.15.97:0/786806834":"2024-04-24T10:16:59.468237+0000","172.21.15.97:6801/3706463017":"2024-04-24T10:16:33.128917+0000","172.21.15.97:6801/3149043749":"2024-04-24T10:26:11.109535+0000","172.21.15.97:0/3294175554":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/1449136760":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/3198144747":"2024-04-24T10:26:11.109535+0000","172.21.15.97:6800/3297166858":"2024-04-24T10:16:59.468237+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T10:42:40.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.197+0000 7f6d65c79640 1 -- 172.21.15.97:0/1747501229 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f6d480a3d20 msgr2=0x7f6d480a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:40.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.197+0000 7f6d65c79640 1 --2- 172.21.15.97:0/1747501229 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f6d480a3d20 0x7f6d480a61e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6d5400abe0 tx=0x7f6d5400a2d0 comp rx=0 tx=0).stop 2024-04-23T10:42:40.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.197+0000 7f6d65c79640 1 -- 172.21.15.97:0/1747501229 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d60106170 msgr2=0x7f6d60198c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:40.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.197+0000 7f6d65c79640 1 --2- 172.21.15.97:0/1747501229 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d60106170 0x7f6d60198c50 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f6d50032fd0 tx=0x7f6d50004270 comp rx=0 tx=0).stop 2024-04-23T10:42:40.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.197+0000 7f6d65c79640 1 -- 172.21.15.97:0/1747501229 shutdown_connections 2024-04-23T10:42:40.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.197+0000 7f6d65c79640 1 --2- 172.21.15.97:0/1747501229 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f6d480a3d20 0x7f6d480a61e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:40.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.197+0000 7f6d65c79640 1 --2- 172.21.15.97:0/1747501229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d6019cfd0 0x7f6d6019d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:40.196 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.197+0000 7f6d65c79640 1 --2- 172.21.15.97:0/1747501229 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d60106170 0x7f6d60198c50 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:40.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.197+0000 7f6d65c79640 1 --2- 172.21.15.97:0/1747501229 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d601010a0 0x7f6d60198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:40.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.197+0000 7f6d65c79640 1 -- 172.21.15.97:0/1747501229 >> 172.21.15.97:0/1747501229 conn(0x7f6d60077f50 msgr2=0x7f6d600ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:42:40.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.197+0000 7f6d65c79640 1 -- 172.21.15.97:0/1747501229 shutdown_connections 2024-04-23T10:42:40.197 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.197+0000 7f6d65c79640 1 -- 172.21.15.97:0/1747501229 wait complete. 2024-04-23T10:42:40.208 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T10:19:25.559138+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T10:24:04.570049+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T10:24:05.861631+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 27, 'snap_epoch': 108, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T10:42:40.208 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T10:42:40.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.311+0000 7f981404e640 1 -- 172.21.15.97:0/2112899908 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f980c107260 msgr2=0x7f980c10ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:40.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.311+0000 7f981404e640 1 --2- 172.21.15.97:0/2112899908 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f980c107260 0x7f980c10ee30 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f9808009900 tx=0x7f980802a410 comp rx=0 tx=0).stop 2024-04-23T10:42:40.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.311+0000 7f981404e640 1 -- 172.21.15.97:0/2112899908 shutdown_connections 2024-04-23T10:42:40.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.311+0000 7f981404e640 1 --2- 172.21.15.97:0/2112899908 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f980c107260 0x7f980c10ee30 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:40.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.311+0000 7f981404e640 1 --2- 172.21.15.97:0/2112899908 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f980c102b40 0x7f980c106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:40.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.311+0000 7f981404e640 1 --2- 172.21.15.97:0/2112899908 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f980c102190 0x7f980c102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:40.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.311+0000 7f981404e640 1 -- 172.21.15.97:0/2112899908 >> 172.21.15.97:0/2112899908 conn(0x7f980c0fd990 msgr2=0x7f980c0ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:42:40.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.311+0000 7f981404e640 1 -- 172.21.15.97:0/2112899908 shutdown_connections 2024-04-23T10:42:40.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.311+0000 7f981404e640 1 -- 172.21.15.97:0/2112899908 wait complete. 2024-04-23T10:42:40.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.312+0000 7f981404e640 1 Processor -- start 2024-04-23T10:42:40.314 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.312+0000 7f981404e640 1 -- start start 2024-04-23T10:42:40.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.312+0000 7f981404e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f980c102190 0x7f980c1986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:40.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.312+0000 7f981404e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f980c107260 0x7f980c198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:40.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.312+0000 7f981404e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f980c19cf70 0x7f980c19d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:40.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.312+0000 7f981404e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f980c1121b0 con 0x7f980c19cf70 2024-04-23T10:42:40.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.312+0000 7f981404e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f980c112030 con 0x7f980c107260 2024-04-23T10:42:40.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.312+0000 7f981404e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f980c112330 con 0x7f980c102190 2024-04-23T10:42:40.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.312+0000 7f98125eb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f980c19cf70 0x7f980c19d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:40.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.312+0000 7f98125eb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f980c19cf70 0x7f980c19d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48284/0 (socket says 172.21.15.97:48284) 2024-04-23T10:42:40.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.312+0000 7f98125eb640 1 -- 172.21.15.97:0/465067942 learned_addr learned my addr 172.21.15.97:0/465067942 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:42:40.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.312+0000 7f98115e9640 1 --2- 172.21.15.97:0/465067942 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f980c107260 0x7f980c198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:40.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.312+0000 7f9811dea640 1 --2- 172.21.15.97:0/465067942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f980c102190 0x7f980c1986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:40.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.312+0000 7f98125eb640 1 -- 172.21.15.97:0/465067942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f980c102190 msgr2=0x7f980c1986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:40.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.313+0000 7f98125eb640 1 --2- 172.21.15.97:0/465067942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f980c102190 0x7f980c1986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:40.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.313+0000 7f98125eb640 1 -- 172.21.15.97:0/465067942 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f980c107260 msgr2=0x7f980c198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:40.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.313+0000 7f98125eb640 1 --2- 172.21.15.97:0/465067942 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f980c107260 0x7f980c198bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:40.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.313+0000 7f98125eb640 1 -- 172.21.15.97:0/465067942 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f980c19da60 con 0x7f980c19cf70 2024-04-23T10:42:40.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.313+0000 7f9811dea640 1 --2- 172.21.15.97:0/465067942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f980c102190 0x7f980c1986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:42:40.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.313+0000 7f98125eb640 1 --2- 172.21.15.97:0/465067942 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f980c19cf70 0x7f980c19d420 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f9808009a20 tx=0x7f9808004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:42:40.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.313+0000 7f9802ffd640 1 -- 172.21.15.97:0/465067942 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f980803f030 con 0x7f980c19cf70 2024-04-23T10:42:40.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.313+0000 7f9802ffd640 1 -- 172.21.15.97:0/465067942 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9808003aa0 con 0x7f980c19cf70 2024-04-23T10:42:40.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.313+0000 7f981404e640 1 -- 172.21.15.97:0/465067942 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f980c19dcf0 con 0x7f980c19cf70 2024-04-23T10:42:40.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.313+0000 7f9802ffd640 1 -- 172.21.15.97:0/465067942 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9808035070 con 0x7f980c19cf70 2024-04-23T10:42:40.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.313+0000 7f981404e640 1 -- 172.21.15.97:0/465067942 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f980c1a54b0 con 0x7f980c19cf70 2024-04-23T10:42:40.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.314+0000 7f981404e640 1 -- 172.21.15.97:0/465067942 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f980c0691c0 con 0x7f980c19cf70 2024-04-23T10:42:40.317 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.315+0000 7f9802ffd640 1 -- 172.21.15.97:0/465067942 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9808042030 con 0x7f980c19cf70 2024-04-23T10:42:40.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.315+0000 7f9802ffd640 1 --2- 172.21.15.97:0/465067942 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f97f80a3d20 0x7f97f80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:40.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.315+0000 7f9811dea640 1 --2- 172.21.15.97:0/465067942 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f97f80a3d20 0x7f97f80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:40.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.316+0000 7f9802ffd640 1 -- 172.21.15.97:0/465067942 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8484+0+0 (secure 0 0 0) 0x7f98080f1070 con 0x7f980c19cf70 2024-04-23T10:42:40.318 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.316+0000 7f9811dea640 1 --2- 172.21.15.97:0/465067942 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f97f80a3d20 0x7f97f80a61e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9804005c40 tx=0x7f98040082f0 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:42:40.319 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.320+0000 7f9802ffd640 1 -- 172.21.15.97:0/465067942 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f980803be20 con 0x7f980c19cf70 2024-04-23T10:42:40.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.502+0000 7f981404e640 1 -- 172.21.15.97:0/465067942 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f980c102b40 con 0x7f980c19cf70 2024-04-23T10:42:40.501 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.502+0000 7f9802ffd640 1 -- 172.21.15.97:0/465067942 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v109) ==== 93+0+10 (secure 0 0 0) 0x7f98080bd730 con 0x7f980c19cf70 2024-04-23T10:42:40.501 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-23T10:42:40.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.504+0000 7f981404e640 1 -- 172.21.15.97:0/465067942 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f97f80a3d20 msgr2=0x7f97f80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:40.503 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.504+0000 7f981404e640 1 --2- 172.21.15.97:0/465067942 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f97f80a3d20 0x7f97f80a61e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9804005c40 tx=0x7f98040082f0 comp rx=0 tx=0).stop 2024-04-23T10:42:40.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.504+0000 7f981404e640 1 -- 172.21.15.97:0/465067942 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f980c19cf70 msgr2=0x7f980c19d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:40.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.504+0000 7f981404e640 1 --2- 172.21.15.97:0/465067942 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f980c19cf70 0x7f980c19d420 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f9808009a20 tx=0x7f9808004270 comp rx=0 tx=0).stop 2024-04-23T10:42:40.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.504+0000 7f981404e640 1 -- 172.21.15.97:0/465067942 shutdown_connections 2024-04-23T10:42:40.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.504+0000 7f981404e640 1 --2- 172.21.15.97:0/465067942 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f97f80a3d20 0x7f97f80a61e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:40.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.504+0000 7f981404e640 1 --2- 172.21.15.97:0/465067942 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f980c19cf70 0x7f980c19d420 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:40.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.504+0000 7f981404e640 1 --2- 172.21.15.97:0/465067942 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f980c107260 0x7f980c198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:40.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.504+0000 7f981404e640 1 --2- 172.21.15.97:0/465067942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f980c102190 0x7f980c1986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:40.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.504+0000 7f981404e640 1 -- 172.21.15.97:0/465067942 >> 172.21.15.97:0/465067942 conn(0x7f980c0fd990 msgr2=0x7f980c0fe4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:42:40.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.504+0000 7f981404e640 1 -- 172.21.15.97:0/465067942 shutdown_connections 2024-04-23T10:42:40.504 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.504+0000 7f981404e640 1 -- 172.21.15.97:0/465067942 wait complete. 2024-04-23T10:42:40.514 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T10:42:40.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.618+0000 7fc1a235c640 1 -- 172.21.15.97:0/156006486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc19c101a10 msgr2=0x7fc19c105a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:40.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.618+0000 7fc1a235c640 1 --2- 172.21.15.97:0/156006486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc19c101a10 0x7fc19c105a50 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fc190009900 tx=0x7fc19002a410 comp rx=0 tx=0).stop 2024-04-23T10:42:40.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.618+0000 7fc1a235c640 1 -- 172.21.15.97:0/156006486 shutdown_connections 2024-04-23T10:42:40.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.618+0000 7fc1a235c640 1 --2- 172.21.15.97:0/156006486 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc19c106130 0x7fc19c10dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:40.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.618+0000 7fc1a235c640 1 --2- 172.21.15.97:0/156006486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc19c101a10 0x7fc19c105a50 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:40.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.618+0000 7fc1a235c640 1 --2- 172.21.15.97:0/156006486 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc19c101060 0x7fc19c101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:40.620 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.618+0000 7fc1a235c640 1 -- 172.21.15.97:0/156006486 >> 172.21.15.97:0/156006486 conn(0x7fc19c0771d0 msgr2=0x7fc19c0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:42:40.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.618+0000 7fc1a235c640 1 -- 172.21.15.97:0/156006486 shutdown_connections 2024-04-23T10:42:40.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.618+0000 7fc1a235c640 1 -- 172.21.15.97:0/156006486 wait complete. 2024-04-23T10:42:40.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.618+0000 7fc1a235c640 1 Processor -- start 2024-04-23T10:42:40.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.619+0000 7fc1a235c640 1 -- start start 2024-04-23T10:42:40.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.619+0000 7fc1a235c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc19c101060 0x7fc19c197630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:40.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.619+0000 7fc1a235c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc19c106130 0x7fc19c197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:40.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.619+0000 7fc1a235c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc19c19bef0 0x7fc19c19c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:40.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.619+0000 7fc1a235c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc19c111070 con 0x7fc19c19bef0 2024-04-23T10:42:40.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.619+0000 7fc1a235c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc19c110ef0 con 0x7fc19c106130 2024-04-23T10:42:40.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.619+0000 7fc1a235c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc19c1111f0 con 0x7fc19c101060 2024-04-23T10:42:40.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.619+0000 7fc1a08f9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc19c19bef0 0x7fc19c19c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:40.621 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.619+0000 7fc1a08f9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc19c19bef0 0x7fc19c19c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48290/0 (socket says 172.21.15.97:48290) 2024-04-23T10:42:40.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.619+0000 7fc1a08f9640 1 -- 172.21.15.97:0/1256091128 learned_addr learned my addr 172.21.15.97:0/1256091128 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:42:40.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.619+0000 7fc19b7fe640 1 --2- 172.21.15.97:0/1256091128 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc19c106130 0x7fc19c197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:40.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.619+0000 7fc19bfff640 1 --2- 172.21.15.97:0/1256091128 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc19c101060 0x7fc19c197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:40.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.619+0000 7fc1a08f9640 1 -- 172.21.15.97:0/1256091128 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc19c101060 msgr2=0x7fc19c197630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:40.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.619+0000 7fc1a08f9640 1 --2- 172.21.15.97:0/1256091128 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc19c101060 0x7fc19c197630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:40.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.619+0000 7fc1a08f9640 1 -- 172.21.15.97:0/1256091128 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc19c106130 msgr2=0x7fc19c197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:40.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.619+0000 7fc1a08f9640 1 --2- 172.21.15.97:0/1256091128 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc19c106130 0x7fc19c197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:40.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.619+0000 7fc1a08f9640 1 -- 172.21.15.97:0/1256091128 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc19c19c9e0 con 0x7fc19c19bef0 2024-04-23T10:42:40.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.619+0000 7fc19bfff640 1 --2- 172.21.15.97:0/1256091128 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc19c101060 0x7fc19c197630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:42:40.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.619+0000 7fc1a08f9640 1 --2- 172.21.15.97:0/1256091128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc19c19bef0 0x7fc19c19c3a0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fc19400cad0 tx=0x7fc19400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:42:40.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.620+0000 7fc1997fa640 1 -- 172.21.15.97:0/1256091128 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19400ae20 con 0x7fc19c19bef0 2024-04-23T10:42:40.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.620+0000 7fc1997fa640 1 -- 172.21.15.97:0/1256091128 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc194005020 con 0x7fc19c19bef0 2024-04-23T10:42:40.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.620+0000 7fc1a235c640 1 -- 172.21.15.97:0/1256091128 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc19c19ccd0 con 0x7fc19c19bef0 2024-04-23T10:42:40.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.620+0000 7fc1997fa640 1 -- 172.21.15.97:0/1256091128 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19400f6b0 con 0x7fc19c19bef0 2024-04-23T10:42:40.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.620+0000 7fc1a235c640 1 -- 172.21.15.97:0/1256091128 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc19c1a4440 con 0x7fc19c19bef0 2024-04-23T10:42:40.623 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.621+0000 7fc1a235c640 1 -- 172.21.15.97:0/1256091128 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc19c102530 con 0x7fc19c19bef0 2024-04-23T10:42:40.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.622+0000 7fc1997fa640 1 -- 172.21.15.97:0/1256091128 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc1940051d0 con 0x7fc19c19bef0 2024-04-23T10:42:40.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.622+0000 7fc1997fa640 1 --2- 172.21.15.97:0/1256091128 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fc1840a3d20 0x7fc1840a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:40.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.623+0000 7fc19bfff640 1 --2- 172.21.15.97:0/1256091128 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fc1840a3d20 0x7fc1840a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:40.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.623+0000 7fc1997fa640 1 -- 172.21.15.97:0/1256091128 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8484+0+0 (secure 0 0 0) 0x7fc1940d08f0 con 0x7fc19c19bef0 2024-04-23T10:42:40.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.623+0000 7fc19bfff640 1 --2- 172.21.15.97:0/1256091128 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fc1840a3d20 0x7fc1840a61e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc18c009b40 tx=0x7fc18c005e50 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:42:40.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.627+0000 7fc1997fa640 1 -- 172.21.15.97:0/1256091128 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc194014030 con 0x7fc19c19bef0 2024-04-23T10:42:40.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.803+0000 7fc1a235c640 1 -- 172.21.15.97:0/1256091128 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc19c198980 con 0x7fc19c19bef0 2024-04-23T10:42:40.803 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.803+0000 7fc1997fa640 1 -- 172.21.15.97:0/1256091128 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v109) ==== 104+0+11 (secure 0 0 0) 0x7fc19409cfb0 con 0x7fc19c19bef0 2024-04-23T10:42:40.803 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-23T10:42:40.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.805+0000 7fc1a235c640 1 -- 172.21.15.97:0/1256091128 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fc1840a3d20 msgr2=0x7fc1840a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:40.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.805+0000 7fc1a235c640 1 --2- 172.21.15.97:0/1256091128 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fc1840a3d20 0x7fc1840a61e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc18c009b40 tx=0x7fc18c005e50 comp rx=0 tx=0).stop 2024-04-23T10:42:40.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.805+0000 7fc1a235c640 1 -- 172.21.15.97:0/1256091128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc19c19bef0 msgr2=0x7fc19c19c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:40.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.805+0000 7fc1a235c640 1 --2- 172.21.15.97:0/1256091128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc19c19bef0 0x7fc19c19c3a0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fc19400cad0 tx=0x7fc19400cf90 comp rx=0 tx=0).stop 2024-04-23T10:42:40.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.805+0000 7fc1a235c640 1 -- 172.21.15.97:0/1256091128 shutdown_connections 2024-04-23T10:42:40.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.805+0000 7fc1a235c640 1 --2- 172.21.15.97:0/1256091128 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fc1840a3d20 0x7fc1840a61e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:40.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.805+0000 7fc1a235c640 1 --2- 172.21.15.97:0/1256091128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc19c19bef0 0x7fc19c19c3a0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:40.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.805+0000 7fc1a235c640 1 --2- 172.21.15.97:0/1256091128 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc19c106130 0x7fc19c197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:40.805 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.805+0000 7fc1a235c640 1 --2- 172.21.15.97:0/1256091128 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc19c101060 0x7fc19c197630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:40.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.806+0000 7fc1a235c640 1 -- 172.21.15.97:0/1256091128 >> 172.21.15.97:0/1256091128 conn(0x7fc19c0771d0 msgr2=0x7fc19c0fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:42:40.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.806+0000 7fc1a235c640 1 -- 172.21.15.97:0/1256091128 shutdown_connections 2024-04-23T10:42:40.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.806+0000 7fc1a235c640 1 -- 172.21.15.97:0/1256091128 wait complete. 2024-04-23T10:42:40.814 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:42:40 smithi097 ceph-mon[29197]: pgmap v517: 97 pgs: 97 active+clean; 4.3 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-04-23T10:42:40.815 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:42:40 smithi097 ceph-mon[29197]: from='client.17865 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:42:40.815 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:42:40 smithi097 ceph-mon[29197]: from='client.17871 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:42:40.815 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:42:40 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/1747501229' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T10:42:40.816 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T10:42:40.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:42:40 smithi196 ceph-mon[32919]: pgmap v517: 97 pgs: 97 active+clean; 4.3 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-04-23T10:42:40.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:42:40 smithi196 ceph-mon[32919]: from='client.17865 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:42:40.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:42:40 smithi196 ceph-mon[32919]: from='client.17871 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:42:40.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:42:40 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/1747501229' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T10:42:40.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:42:40 smithi183 ceph-mon[32317]: pgmap v517: 97 pgs: 97 active+clean; 4.3 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-04-23T10:42:40.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:42:40 smithi183 ceph-mon[32317]: from='client.17865 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:42:40.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:42:40 smithi183 ceph-mon[32317]: from='client.17871 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T10:42:40.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:42:40 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/1747501229' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T10:42:40.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.919+0000 7f0befff7640 1 -- 172.21.15.97:0/1725178489 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0be8106170 msgr2=0x7f0be810dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:40.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.919+0000 7f0befff7640 1 --2- 172.21.15.97:0/1725178489 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0be8106170 0x7f0be810dd40 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f0be4009900 tx=0x7f0be402a410 comp rx=0 tx=0).stop 2024-04-23T10:42:40.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.919+0000 7f0befff7640 1 -- 172.21.15.97:0/1725178489 shutdown_connections 2024-04-23T10:42:40.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.919+0000 7f0befff7640 1 --2- 172.21.15.97:0/1725178489 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0be8106170 0x7f0be810dd40 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:40.921 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.919+0000 7f0befff7640 1 --2- 172.21.15.97:0/1725178489 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0be8101a50 0x7f0be8105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:40.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.919+0000 7f0befff7640 1 --2- 172.21.15.97:0/1725178489 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0be81010a0 0x7f0be8101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:40.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.919+0000 7f0befff7640 1 -- 172.21.15.97:0/1725178489 >> 172.21.15.97:0/1725178489 conn(0x7f0be8077eb0 msgr2=0x7f0be80782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:42:40.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.919+0000 7f0befff7640 1 -- 172.21.15.97:0/1725178489 shutdown_connections 2024-04-23T10:42:40.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.919+0000 7f0befff7640 1 -- 172.21.15.97:0/1725178489 wait complete. 2024-04-23T10:42:40.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.920+0000 7f0befff7640 1 Processor -- start 2024-04-23T10:42:40.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.920+0000 7f0befff7640 1 -- start start 2024-04-23T10:42:40.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.920+0000 7f0befff7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0be81010a0 0x7f0be81986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:40.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.920+0000 7f0befff7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0be8106170 0x7f0be8198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:40.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.920+0000 7f0befff7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0be819cf70 0x7f0be819d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:40.922 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.920+0000 7f0befff7640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0be8111110 con 0x7f0be819cf70 2024-04-23T10:42:40.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.920+0000 7f0befff7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0be8110f90 con 0x7f0be8106170 2024-04-23T10:42:40.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.920+0000 7f0befff7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0be8111290 con 0x7f0be81010a0 2024-04-23T10:42:40.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.920+0000 7f0bee594640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0be819cf70 0x7f0be819d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:40.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.920+0000 7f0bee594640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0be819cf70 0x7f0be819d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48314/0 (socket says 172.21.15.97:48314) 2024-04-23T10:42:40.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.920+0000 7f0bee594640 1 -- 172.21.15.97:0/3727061661 learned_addr learned my addr 172.21.15.97:0/3727061661 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:42:40.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.920+0000 7f0bed592640 1 --2- 172.21.15.97:0/3727061661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0be8106170 0x7f0be8198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:40.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.920+0000 7f0bedd93640 1 --2- 172.21.15.97:0/3727061661 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0be81010a0 0x7f0be81986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:40.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.920+0000 7f0bee594640 1 -- 172.21.15.97:0/3727061661 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0be81010a0 msgr2=0x7f0be81986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:40.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.920+0000 7f0bee594640 1 --2- 172.21.15.97:0/3727061661 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0be81010a0 0x7f0be81986b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:40.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.920+0000 7f0bee594640 1 -- 172.21.15.97:0/3727061661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0be8106170 msgr2=0x7f0be8198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:40.923 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.920+0000 7f0bee594640 1 --2- 172.21.15.97:0/3727061661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0be8106170 0x7f0be8198bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:40.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.920+0000 7f0bee594640 1 -- 172.21.15.97:0/3727061661 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0be819da60 con 0x7f0be819cf70 2024-04-23T10:42:40.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.920+0000 7f0bed592640 1 --2- 172.21.15.97:0/3727061661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0be8106170 0x7f0be8198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:42:40.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.920+0000 7f0bee594640 1 --2- 172.21.15.97:0/3727061661 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0be819cf70 0x7f0be819d420 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f0be402a930 tx=0x7f0be4003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:42:40.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.920+0000 7f0bdeffd640 1 -- 172.21.15.97:0/3727061661 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be403f070 con 0x7f0be819cf70 2024-04-23T10:42:40.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.920+0000 7f0bdeffd640 1 -- 172.21.15.97:0/3727061661 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0be4031d60 con 0x7f0be819cf70 2024-04-23T10:42:40.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.920+0000 7f0befff7640 1 -- 172.21.15.97:0/3727061661 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0be819dcf0 con 0x7f0be819cf70 2024-04-23T10:42:40.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.920+0000 7f0bdeffd640 1 -- 172.21.15.97:0/3727061661 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be4035030 con 0x7f0be819cf70 2024-04-23T10:42:40.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.921+0000 7f0befff7640 1 -- 172.21.15.97:0/3727061661 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0be81a54b0 con 0x7f0be819cf70 2024-04-23T10:42:40.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.921+0000 7f0befff7640 1 -- 172.21.15.97:0/3727061661 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0be80691c0 con 0x7f0be819cf70 2024-04-23T10:42:40.924 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.922+0000 7f0bdeffd640 1 -- 172.21.15.97:0/3727061661 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0be4042070 con 0x7f0be819cf70 2024-04-23T10:42:40.925 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.923+0000 7f0bdeffd640 1 --2- 172.21.15.97:0/3727061661 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f0bd40a3d20 0x7f0bd40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:40.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.923+0000 7f0bedd93640 1 --2- 172.21.15.97:0/3727061661 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f0bd40a3d20 0x7f0bd40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:40.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.923+0000 7f0bdeffd640 1 -- 172.21.15.97:0/3727061661 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8484+0+0 (secure 0 0 0) 0x7f0be40f1780 con 0x7f0be819cf70 2024-04-23T10:42:40.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.924+0000 7f0bedd93640 1 --2- 172.21.15.97:0/3727061661 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f0bd40a3d20 0x7f0bd40a61e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0be0005fd0 tx=0x7f0be0009810 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:42:40.926 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:40.927+0000 7f0bdeffd640 1 -- 172.21.15.97:0/3727061661 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0be40bde40 con 0x7f0be819cf70 2024-04-23T10:42:41.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:41.106+0000 7f0befff7640 1 -- 172.21.15.97:0/3727061661 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0be8101c00 con 0x7f0be819cf70 2024-04-23T10:42:41.105 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:41.106+0000 7f0bdeffd640 1 -- 172.21.15.97:0/3727061661 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v109) ==== 100+0+11 (secure 0 0 0) 0x7f0be40bd740 con 0x7f0be819cf70 2024-04-23T10:42:41.105 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-04-23T10:42:41.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:41.108+0000 7f0befff7640 1 -- 172.21.15.97:0/3727061661 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f0bd40a3d20 msgr2=0x7f0bd40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:41.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:41.108+0000 7f0befff7640 1 --2- 172.21.15.97:0/3727061661 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f0bd40a3d20 0x7f0bd40a61e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0be0005fd0 tx=0x7f0be0009810 comp rx=0 tx=0).stop 2024-04-23T10:42:41.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:41.108+0000 7f0befff7640 1 -- 172.21.15.97:0/3727061661 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0be819cf70 msgr2=0x7f0be819d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:41.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:41.108+0000 7f0befff7640 1 --2- 172.21.15.97:0/3727061661 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0be819cf70 0x7f0be819d420 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f0be402a930 tx=0x7f0be4003cd0 comp rx=0 tx=0).stop 2024-04-23T10:42:41.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:41.108+0000 7f0befff7640 1 -- 172.21.15.97:0/3727061661 shutdown_connections 2024-04-23T10:42:41.107 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:41.108+0000 7f0befff7640 1 --2- 172.21.15.97:0/3727061661 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f0bd40a3d20 0x7f0bd40a61e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:41.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:41.108+0000 7f0befff7640 1 --2- 172.21.15.97:0/3727061661 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0be819cf70 0x7f0be819d420 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:41.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:41.108+0000 7f0befff7640 1 --2- 172.21.15.97:0/3727061661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0be8106170 0x7f0be8198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:41.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:41.108+0000 7f0befff7640 1 --2- 172.21.15.97:0/3727061661 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0be81010a0 0x7f0be81986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:41.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:41.108+0000 7f0befff7640 1 -- 172.21.15.97:0/3727061661 >> 172.21.15.97:0/3727061661 conn(0x7f0be8077eb0 msgr2=0x7f0be8104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:42:41.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:41.108+0000 7f0befff7640 1 -- 172.21.15.97:0/3727061661 shutdown_connections 2024-04-23T10:42:41.108 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:41.108+0000 7f0befff7640 1 -- 172.21.15.97:0/3727061661 wait complete. 2024-04-23T10:42:41.118 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-23T10:42:41.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:41.219+0000 7fdea218e640 1 -- 172.21.15.97:0/1739010365 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde9c103b00 msgr2=0x7fde9c10bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:41.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:41.219+0000 7fdea218e640 1 --2- 172.21.15.97:0/1739010365 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde9c103b00 0x7fde9c10bfe0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fde84009900 tx=0x7fde8402a510 comp rx=0 tx=0).stop 2024-04-23T10:42:41.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:41.219+0000 7fdea218e640 1 -- 172.21.15.97:0/1739010365 shutdown_connections 2024-04-23T10:42:41.220 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:41.219+0000 7fdea218e640 1 --2- 172.21.15.97:0/1739010365 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fde9c10c520 0x7fde9c10e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:41.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:41.219+0000 7fdea218e640 1 --2- 172.21.15.97:0/1739010365 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde9c103b00 0x7fde9c10bfe0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:41.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:41.219+0000 7fdea218e640 1 --2- 172.21.15.97:0/1739010365 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde9c1031e0 0x7fde9c1035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:41.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:41.219+0000 7fdea218e640 1 -- 172.21.15.97:0/1739010365 >> 172.21.15.97:0/1739010365 conn(0x7fde9c077f50 msgr2=0x7fde9c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:42:41.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:41.219+0000 7fdea218e640 1 -- 172.21.15.97:0/1739010365 shutdown_connections 2024-04-23T10:42:41.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:41.219+0000 7fdea218e640 1 -- 172.21.15.97:0/1739010365 wait complete. 2024-04-23T10:42:41.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:41.219+0000 7fdea218e640 1 Processor -- start 2024-04-23T10:42:41.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:41.219+0000 7fdea218e640 1 -- start start 2024-04-23T10:42:41.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:41.219+0000 7fdea218e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde9c1031e0 0x7fde9c1a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:41.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:41.219+0000 7fdea218e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fde9c103b00 0x7fde9c1a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:41.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:41.219+0000 7fdea218e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde9c10c520 0x7fde9c19a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:41.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:41.219+0000 7fdea218e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c1121b0 con 0x7fde9c10c520 2024-04-23T10:42:41.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:41.219+0000 7fdea218e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c112030 con 0x7fde9c103b00 2024-04-23T10:42:41.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:41.219+0000 7fdea218e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c112330 con 0x7fde9c1031e0 2024-04-23T10:42:41.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:41.220+0000 7fde9bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde9c10c520 0x7fde9c19a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:41.221 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:41.220+0000 7fde9bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde9c10c520 0x7fde9c19a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:48330/0 (socket says 172.21.15.97:48330) 2024-04-23T10:42:41.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:41.220+0000 7fde9bfff640 1 -- 172.21.15.97:0/2289655586 learned_addr learned my addr 172.21.15.97:0/2289655586 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:42:41.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:41.220+0000 7fde9b7fe640 1 --2- 172.21.15.97:0/2289655586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde9c1031e0 0x7fde9c1a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:41.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:41.220+0000 7fde93fff640 1 --2- 172.21.15.97:0/2289655586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fde9c103b00 0x7fde9c1a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:41.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:41.220+0000 7fde9bfff640 1 -- 172.21.15.97:0/2289655586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde9c1031e0 msgr2=0x7fde9c1a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:41.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:41.220+0000 7fde9bfff640 1 --2- 172.21.15.97:0/2289655586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde9c1031e0 0x7fde9c1a0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:41.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:41.220+0000 7fde9bfff640 1 -- 172.21.15.97:0/2289655586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fde9c103b00 msgr2=0x7fde9c1a0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:41.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:41.220+0000 7fde9bfff640 1 --2- 172.21.15.97:0/2289655586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fde9c103b00 0x7fde9c1a0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:41.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:41.220+0000 7fde9bfff640 1 -- 172.21.15.97:0/2289655586 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde9c19af80 con 0x7fde9c10c520 2024-04-23T10:42:41.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:41.220+0000 7fde93fff640 1 --2- 172.21.15.97:0/2289655586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fde9c103b00 0x7fde9c1a0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:42:41.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:41.220+0000 7fde9bfff640 1 --2- 172.21.15.97:0/2289655586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde9c10c520 0x7fde9c19a6c0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fde9400cce0 tx=0x7fde9400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:42:41.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:41.220+0000 7fde997fa640 1 -- 172.21.15.97:0/2289655586 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde9400f450 con 0x7fde9c10c520 2024-04-23T10:42:41.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:41.220+0000 7fde997fa640 1 -- 172.21.15.97:0/2289655586 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde9400faa0 con 0x7fde9c10c520 2024-04-23T10:42:41.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:41.220+0000 7fdea218e640 1 -- 172.21.15.97:0/2289655586 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde9c19b270 con 0x7fde9c10c520 2024-04-23T10:42:41.222 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:41.220+0000 7fde997fa640 1 -- 172.21.15.97:0/2289655586 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde94018b70 con 0x7fde9c10c520 2024-04-23T10:42:41.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:41.220+0000 7fdea218e640 1 -- 172.21.15.97:0/2289655586 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde9c1a7580 con 0x7fde9c10c520 2024-04-23T10:42:41.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:41.221+0000 7fdea218e640 1 -- 172.21.15.97:0/2289655586 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde9c0698e0 con 0x7fde9c10c520 2024-04-23T10:42:41.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:41.222+0000 7fde997fa640 1 -- 172.21.15.97:0/2289655586 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fde94018d10 con 0x7fde9c10c520 2024-04-23T10:42:41.223 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:41.223+0000 7fde997fa640 1 --2- 172.21.15.97:0/2289655586 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fde880a3d20 0x7fde880a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:42:41.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:41.223+0000 7fde9b7fe640 1 --2- 172.21.15.97:0/2289655586 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fde880a3d20 0x7fde880a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:42:41.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:41.223+0000 7fde997fa640 1 -- 172.21.15.97:0/2289655586 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8484+0+0 (secure 0 0 0) 0x7fde940d0bf0 con 0x7fde9c10c520 2024-04-23T10:42:41.224 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:41.223+0000 7fde9b7fe640 1 --2- 172.21.15.97:0/2289655586 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fde880a3d20 0x7fde880a61e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fde8c005fd0 tx=0x7fde8c00b390 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:42:41.226 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:41.228+0000 7fde997fa640 1 -- 172.21.15.97:0/2289655586 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fde94014030 con 0x7fde9c10c520 2024-04-23T10:42:41.454 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:41.455+0000 7fdea218e640 1 -- 172.21.15.97:0/2289655586 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fde9c19b5a0 con 0x7fde9c10c520 2024-04-23T10:42:41.462 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:41.463+0000 7fde997fa640 1 -- 172.21.15.97:0/2289655586 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+746059 (secure 0 0 0) 0x7fde9c19b5a0 con 0x7fde9c10c520 2024-04-23T10:42:41.474 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:42:41.474 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-23T10:42:41.474 INFO:teuthology.orchestra.run.smithi097.stdout: "epoch": 26, 2024-04-23T10:42:41.474 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.474 INFO:teuthology.orchestra.run.smithi097.stdout: "active_gid": 15378, 2024-04-23T10:42:41.474 INFO:teuthology.orchestra.run.smithi097.stdout: "active_name": "x", 2024-04-23T10:42:41.475 INFO:teuthology.orchestra.run.smithi097.stdout: "active_addrs": { 2024-04-23T10:42:41.475 INFO:teuthology.orchestra.run.smithi097.stdout: "addrvec": [ 2024-04-23T10:42:41.475 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:41.475 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v2", 2024-04-23T10:42:41.475 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:6800", 2024-04-23T10:42:41.475 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 3567268502 2024-04-23T10:42:41.475 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.475 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:41.475 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v1", 2024-04-23T10:42:41.475 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:6801", 2024-04-23T10:42:41.475 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 3567268502 2024-04-23T10:42:41.475 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.476 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-04-23T10:42:41.476 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.476 INFO:teuthology.orchestra.run.smithi097.stdout: "active_addr": "172.21.15.97:6801/3567268502", 2024-04-23T10:42:41.476 INFO:teuthology.orchestra.run.smithi097.stdout: "active_change": "2024-04-23T10:26:11.109745+0000", 2024-04-23T10:42:41.476 INFO:teuthology.orchestra.run.smithi097.stdout: "active_mgr_features": 4540701547738038271, 2024-04-23T10:42:41.476 INFO:teuthology.orchestra.run.smithi097.stdout: "available": true, 2024-04-23T10:42:41.476 INFO:teuthology.orchestra.run.smithi097.stdout: "standbys": [ 2024-04-23T10:42:41.476 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:41.476 INFO:teuthology.orchestra.run.smithi097.stdout: "gid": 24916, 2024-04-23T10:42:41.476 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "z", 2024-04-23T10:42:41.476 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr_features": 4540701547738038271, 2024-04-23T10:42:41.477 INFO:teuthology.orchestra.run.smithi097.stdout: "available_modules": [ 2024-04-23T10:42:41.477 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:41.477 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "alerts", 2024-04-23T10:42:41.477 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:41.477 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:41.477 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:41.477 INFO:teuthology.orchestra.run.smithi097.stdout: "interval": { 2024-04-23T10:42:41.477 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "interval", 2024-04-23T10:42:41.477 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-04-23T10:42:41.477 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.477 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.477 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "60", 2024-04-23T10:42:41.478 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.478 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.478 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.478 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "How frequently to reexamine health status", 2024-04-23T10:42:41.478 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.478 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.478 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.478 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.478 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:41.478 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:41.478 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.479 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.479 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.479 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.479 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.479 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.479 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.479 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.479 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.479 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.479 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.479 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.479 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.480 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.480 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.480 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.480 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.480 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.480 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.480 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:41.480 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:41.480 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.480 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.480 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.481 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.481 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.481 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.481 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.481 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.481 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.481 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.481 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.481 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.481 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:41.481 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:41.482 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.482 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.482 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.482 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:41.482 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.482 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.482 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.482 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.482 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.482 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.482 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.483 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.483 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.483 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.483 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.483 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.483 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.483 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.483 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.483 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:41.483 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:41.483 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.483 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.484 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.484 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.484 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.484 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.484 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.484 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.484 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.484 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.484 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.484 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.484 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_destination": { 2024-04-23T10:42:41.484 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_destination", 2024-04-23T10:42:41.485 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.485 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.485 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.485 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.485 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.485 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.485 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.485 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Email address to send alerts to", 2024-04-23T10:42:41.485 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.485 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.485 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.485 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.486 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_from_name": { 2024-04-23T10:42:41.486 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_from_name", 2024-04-23T10:42:41.486 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.486 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.486 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.486 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "Ceph", 2024-04-23T10:42:41.486 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.486 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.486 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.486 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Email From: name", 2024-04-23T10:42:41.486 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.486 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.487 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.487 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.487 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_host": { 2024-04-23T10:42:41.487 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_host", 2024-04-23T10:42:41.487 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.487 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.487 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.487 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.487 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.487 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.487 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.487 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "SMTP server", 2024-04-23T10:42:41.488 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.488 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.488 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.488 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.488 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_password": { 2024-04-23T10:42:41.488 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_password", 2024-04-23T10:42:41.488 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.488 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.488 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.488 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.488 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.488 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.488 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.489 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Password to authenticate with", 2024-04-23T10:42:41.489 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.489 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.489 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.489 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.489 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_port": { 2024-04-23T10:42:41.489 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_port", 2024-04-23T10:42:41.489 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:41.489 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.489 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.489 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "465", 2024-04-23T10:42:41.489 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.490 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.490 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.490 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "SMTP port", 2024-04-23T10:42:41.490 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.490 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.490 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.490 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.490 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_sender": { 2024-04-23T10:42:41.490 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_sender", 2024-04-23T10:42:41.490 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.490 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.490 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.491 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.491 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.491 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.491 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.491 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "SMTP envelope sender", 2024-04-23T10:42:41.491 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.491 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.491 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.491 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.491 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_ssl": { 2024-04-23T10:42:41.491 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_ssl", 2024-04-23T10:42:41.492 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.492 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.492 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.492 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:41.492 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.492 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.492 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.492 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-23T10:42:41.492 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.492 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.492 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.492 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.493 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_user": { 2024-04-23T10:42:41.493 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_user", 2024-04-23T10:42:41.493 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.493 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.493 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.493 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.493 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.493 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.493 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.493 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "User to authenticate as", 2024-04-23T10:42:41.493 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.493 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.493 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.494 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.494 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.494 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.494 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:41.494 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "balancer", 2024-04-23T10:42:41.494 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:41.494 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:41.494 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:41.494 INFO:teuthology.orchestra.run.smithi097.stdout: "active": { 2024-04-23T10:42:41.494 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "active", 2024-04-23T10:42:41.494 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.494 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.495 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.495 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:41.495 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.495 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.495 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.495 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "automatically balance PGs across cluster", 2024-04-23T10:42:41.495 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.495 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.495 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.495 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.495 INFO:teuthology.orchestra.run.smithi097.stdout: "begin_time": { 2024-04-23T10:42:41.495 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "begin_time", 2024-04-23T10:42:41.495 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.496 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.496 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.496 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0000", 2024-04-23T10:42:41.496 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.496 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.496 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.496 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "beginning time of day to automatically balance", 2024-04-23T10:42:41.496 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-23T10:42:41.496 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.496 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.496 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.496 INFO:teuthology.orchestra.run.smithi097.stdout: "begin_weekday": { 2024-04-23T10:42:41.497 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "begin_weekday", 2024-04-23T10:42:41.497 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "uint", 2024-04-23T10:42:41.497 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.497 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.497 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0", 2024-04-23T10:42:41.497 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "0", 2024-04-23T10:42:41.497 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "6", 2024-04-23T10:42:41.497 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.497 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-23T10:42:41.497 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-23T10:42:41.497 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.497 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.497 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.498 INFO:teuthology.orchestra.run.smithi097.stdout: "crush_compat_max_iterations": { 2024-04-23T10:42:41.498 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "crush_compat_max_iterations", 2024-04-23T10:42:41.498 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "uint", 2024-04-23T10:42:41.498 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.498 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.498 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "25", 2024-04-23T10:42:41.498 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "1", 2024-04-23T10:42:41.498 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "250", 2024-04-23T10:42:41.498 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.498 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-23T10:42:41.498 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.498 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.499 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.499 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.499 INFO:teuthology.orchestra.run.smithi097.stdout: "crush_compat_metrics": { 2024-04-23T10:42:41.499 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "crush_compat_metrics", 2024-04-23T10:42:41.499 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.499 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.499 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.499 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "pgs,objects,bytes", 2024-04-23T10:42:41.499 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.499 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.499 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.499 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-23T10:42:41.499 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-23T10:42:41.500 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.500 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.500 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.500 INFO:teuthology.orchestra.run.smithi097.stdout: "crush_compat_step": { 2024-04-23T10:42:41.500 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "crush_compat_step", 2024-04-23T10:42:41.500 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2024-04-23T10:42:41.500 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.500 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.500 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0.5", 2024-04-23T10:42:41.500 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "0.001", 2024-04-23T10:42:41.500 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "0.999", 2024-04-23T10:42:41.500 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.501 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "aggressiveness of optimization", 2024-04-23T10:42:41.501 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-23T10:42:41.501 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.501 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.501 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.501 INFO:teuthology.orchestra.run.smithi097.stdout: "end_time": { 2024-04-23T10:42:41.501 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "end_time", 2024-04-23T10:42:41.501 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.501 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.501 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.501 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "2359", 2024-04-23T10:42:41.501 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.501 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.502 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.502 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "ending time of day to automatically balance", 2024-04-23T10:42:41.502 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-23T10:42:41.502 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.502 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.502 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.502 INFO:teuthology.orchestra.run.smithi097.stdout: "end_weekday": { 2024-04-23T10:42:41.502 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "end_weekday", 2024-04-23T10:42:41.502 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "uint", 2024-04-23T10:42:41.502 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.502 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.502 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0", 2024-04-23T10:42:41.503 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "0", 2024-04-23T10:42:41.503 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "6", 2024-04-23T10:42:41.503 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.503 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-23T10:42:41.503 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-23T10:42:41.503 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.503 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.503 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.503 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:41.503 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:41.503 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.503 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.503 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.503 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.504 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.504 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.504 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.504 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.504 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.504 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.504 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.504 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.504 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.504 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.504 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.504 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.505 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.505 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.505 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.505 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:41.505 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:41.505 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.505 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.505 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.505 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.505 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.505 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.505 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.506 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.506 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.506 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.506 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.506 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.506 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:41.506 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:41.506 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.506 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.506 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.506 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:41.506 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.507 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.507 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.507 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.507 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.507 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.507 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.507 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.507 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.507 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.507 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.507 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.507 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.508 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.508 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.508 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:41.508 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:41.508 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.508 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.508 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.508 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.508 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.508 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.508 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.508 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.508 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.509 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.509 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.509 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.509 INFO:teuthology.orchestra.run.smithi097.stdout: "min_score": { 2024-04-23T10:42:41.509 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "min_score", 2024-04-23T10:42:41.509 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2024-04-23T10:42:41.509 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.509 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.509 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0", 2024-04-23T10:42:41.509 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.509 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.509 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.509 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-23T10:42:41.510 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.510 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.510 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.510 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.510 INFO:teuthology.orchestra.run.smithi097.stdout: "mode": { 2024-04-23T10:42:41.510 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mode", 2024-04-23T10:42:41.510 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.510 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.510 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.510 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "upmap", 2024-04-23T10:42:41.510 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.510 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.511 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.511 INFO:teuthology.orchestra.run.smithi097.stdout: "crush-compat", 2024-04-23T10:42:41.511 INFO:teuthology.orchestra.run.smithi097.stdout: "none", 2024-04-23T10:42:41.511 INFO:teuthology.orchestra.run.smithi097.stdout: "read", 2024-04-23T10:42:41.511 INFO:teuthology.orchestra.run.smithi097.stdout: "upmap", 2024-04-23T10:42:41.511 INFO:teuthology.orchestra.run.smithi097.stdout: "upmap-read" 2024-04-23T10:42:41.511 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.511 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Balancer mode", 2024-04-23T10:42:41.511 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.511 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.511 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.511 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.512 INFO:teuthology.orchestra.run.smithi097.stdout: "pool_ids": { 2024-04-23T10:42:41.512 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "pool_ids", 2024-04-23T10:42:41.512 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.512 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.512 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.512 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.512 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.512 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.512 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.512 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-23T10:42:41.512 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.512 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.512 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.512 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.513 INFO:teuthology.orchestra.run.smithi097.stdout: "sleep_interval": { 2024-04-23T10:42:41.513 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "sleep_interval", 2024-04-23T10:42:41.513 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-04-23T10:42:41.513 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.513 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.513 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "60", 2024-04-23T10:42:41.513 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.513 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.513 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.513 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-23T10:42:41.513 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.513 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.513 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.514 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.514 INFO:teuthology.orchestra.run.smithi097.stdout: "upmap_max_deviation": { 2024-04-23T10:42:41.514 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "upmap_max_deviation", 2024-04-23T10:42:41.514 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:41.514 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.514 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.514 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "5", 2024-04-23T10:42:41.514 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "1", 2024-04-23T10:42:41.514 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.514 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.514 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-23T10:42:41.514 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-23T10:42:41.514 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.515 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.515 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.515 INFO:teuthology.orchestra.run.smithi097.stdout: "upmap_max_optimizations": { 2024-04-23T10:42:41.515 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "upmap_max_optimizations", 2024-04-23T10:42:41.515 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "uint", 2024-04-23T10:42:41.515 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.515 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.515 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-04-23T10:42:41.515 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.515 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.515 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.515 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-23T10:42:41.515 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.516 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.516 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.516 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.516 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.516 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.516 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:41.516 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "cephadm", 2024-04-23T10:42:41.516 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:41.516 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:41.516 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:41.516 INFO:teuthology.orchestra.run.smithi097.stdout: "agent_down_multiplier": { 2024-04-23T10:42:41.516 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "agent_down_multiplier", 2024-04-23T10:42:41.516 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2024-04-23T10:42:41.516 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.517 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.517 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "3.0", 2024-04-23T10:42:41.517 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.517 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.517 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.517 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-23T10:42:41.517 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.517 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.517 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.517 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.517 INFO:teuthology.orchestra.run.smithi097.stdout: "agent_refresh_rate": { 2024-04-23T10:42:41.517 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "agent_refresh_rate", 2024-04-23T10:42:41.517 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-04-23T10:42:41.518 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.518 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.518 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "20", 2024-04-23T10:42:41.518 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.518 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.518 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.518 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-23T10:42:41.518 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.518 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.518 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.518 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.518 INFO:teuthology.orchestra.run.smithi097.stdout: "agent_starting_port": { 2024-04-23T10:42:41.518 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "agent_starting_port", 2024-04-23T10:42:41.519 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:41.519 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.519 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.519 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "4721", 2024-04-23T10:42:41.519 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.519 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.519 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.519 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-23T10:42:41.519 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.519 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.519 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.519 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.520 INFO:teuthology.orchestra.run.smithi097.stdout: "allow_ptrace": { 2024-04-23T10:42:41.520 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "allow_ptrace", 2024-04-23T10:42:41.520 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.520 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.520 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.520 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.520 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.520 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.520 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.520 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-23T10:42:41.520 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-23T10:42:41.520 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.520 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.520 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.521 INFO:teuthology.orchestra.run.smithi097.stdout: "autotune_interval": { 2024-04-23T10:42:41.521 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "autotune_interval", 2024-04-23T10:42:41.521 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-04-23T10:42:41.521 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.521 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.521 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "600", 2024-04-23T10:42:41.521 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.521 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.521 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.521 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-23T10:42:41.521 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.521 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.522 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.522 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.522 INFO:teuthology.orchestra.run.smithi097.stdout: "autotune_memory_target_ratio": { 2024-04-23T10:42:41.522 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "autotune_memory_target_ratio", 2024-04-23T10:42:41.522 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2024-04-23T10:42:41.522 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.522 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.522 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0.7", 2024-04-23T10:42:41.522 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.522 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.522 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.522 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-23T10:42:41.522 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.523 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.523 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.523 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.523 INFO:teuthology.orchestra.run.smithi097.stdout: "cephadm_log_destination": { 2024-04-23T10:42:41.523 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "cephadm_log_destination", 2024-04-23T10:42:41.523 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.523 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.523 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.523 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.523 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.523 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.523 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.523 INFO:teuthology.orchestra.run.smithi097.stdout: "file", 2024-04-23T10:42:41.523 INFO:teuthology.orchestra.run.smithi097.stdout: "file,syslog", 2024-04-23T10:42:41.524 INFO:teuthology.orchestra.run.smithi097.stdout: "syslog" 2024-04-23T10:42:41.524 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.524 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-23T10:42:41.524 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.524 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.524 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.524 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.524 INFO:teuthology.orchestra.run.smithi097.stdout: "cgroups_split": { 2024-04-23T10:42:41.524 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "cgroups_split", 2024-04-23T10:42:41.524 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.524 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.524 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.524 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:41.525 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.525 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.525 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.525 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-23T10:42:41.525 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.525 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.525 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.525 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.525 INFO:teuthology.orchestra.run.smithi097.stdout: "config_checks_enabled": { 2024-04-23T10:42:41.525 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "config_checks_enabled", 2024-04-23T10:42:41.525 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.525 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.525 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.525 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.526 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.526 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.526 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.526 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-23T10:42:41.526 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.526 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.526 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.526 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.526 INFO:teuthology.orchestra.run.smithi097.stdout: "config_dashboard": { 2024-04-23T10:42:41.526 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "config_dashboard", 2024-04-23T10:42:41.526 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.526 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.526 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.527 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:41.527 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.527 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.527 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.527 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-23T10:42:41.527 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.527 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.527 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.527 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.527 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_alertmanager": { 2024-04-23T10:42:41.527 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_alertmanager", 2024-04-23T10:42:41.527 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.527 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.528 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.528 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-23T10:42:41.528 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.528 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.528 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.528 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Prometheus container image", 2024-04-23T10:42:41.528 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.528 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.528 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.528 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.528 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_base": { 2024-04-23T10:42:41.528 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_base", 2024-04-23T10:42:41.528 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.528 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.529 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.529 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-23T10:42:41.529 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.529 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.529 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.529 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Container image name, without the tag", 2024-04-23T10:42:41.529 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.529 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.529 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.529 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.529 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_elasticsearch": { 2024-04-23T10:42:41.529 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_elasticsearch", 2024-04-23T10:42:41.529 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.529 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.530 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.530 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-23T10:42:41.530 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.530 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.530 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.530 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "elasticsearch container image", 2024-04-23T10:42:41.530 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.530 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.530 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.530 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.530 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_grafana": { 2024-04-23T10:42:41.530 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_grafana", 2024-04-23T10:42:41.530 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.531 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.531 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.531 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-23T10:42:41.531 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.531 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.531 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.531 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Prometheus container image", 2024-04-23T10:42:41.531 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.531 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.531 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.531 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.531 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_haproxy": { 2024-04-23T10:42:41.532 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_haproxy", 2024-04-23T10:42:41.532 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.532 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.532 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.532 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-23T10:42:41.532 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.532 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.532 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.532 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "HAproxy container image", 2024-04-23T10:42:41.532 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.532 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.532 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.532 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.533 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_jaeger_agent": { 2024-04-23T10:42:41.533 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_jaeger_agent", 2024-04-23T10:42:41.533 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.533 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.533 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.533 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-23T10:42:41.533 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.533 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.533 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.533 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Jaeger agent container image", 2024-04-23T10:42:41.533 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.533 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.533 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.533 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.534 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_jaeger_collector": { 2024-04-23T10:42:41.534 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_jaeger_collector", 2024-04-23T10:42:41.534 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.534 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.534 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.534 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-23T10:42:41.534 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.534 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.534 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.534 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Jaeger collector container image", 2024-04-23T10:42:41.534 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.534 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.534 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.534 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.535 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_jaeger_query": { 2024-04-23T10:42:41.535 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_jaeger_query", 2024-04-23T10:42:41.535 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.535 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.535 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.535 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-23T10:42:41.535 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.535 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.535 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.535 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Jaeger query container image", 2024-04-23T10:42:41.535 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.535 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.535 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.536 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.536 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_keepalived": { 2024-04-23T10:42:41.536 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_keepalived", 2024-04-23T10:42:41.536 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.536 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.536 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.536 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-23T10:42:41.536 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.536 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.536 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.536 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Keepalived container image", 2024-04-23T10:42:41.536 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.536 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.536 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.537 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.537 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_loki": { 2024-04-23T10:42:41.537 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_loki", 2024-04-23T10:42:41.537 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.537 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.537 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.537 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-23T10:42:41.537 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.537 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.537 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.537 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Loki container image", 2024-04-23T10:42:41.537 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.537 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.537 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.538 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.538 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_node_exporter": { 2024-04-23T10:42:41.538 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_node_exporter", 2024-04-23T10:42:41.538 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.538 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.538 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.538 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-23T10:42:41.538 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.538 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.538 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.538 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Prometheus container image", 2024-04-23T10:42:41.538 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.538 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.539 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.539 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.539 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_nvmeof": { 2024-04-23T10:42:41.539 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_nvmeof", 2024-04-23T10:42:41.539 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.539 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.539 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.539 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-23T10:42:41.539 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.539 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.539 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.539 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Nvme-of container image", 2024-04-23T10:42:41.539 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.539 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.540 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.540 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.540 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_prometheus": { 2024-04-23T10:42:41.540 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_prometheus", 2024-04-23T10:42:41.540 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.540 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.540 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.540 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-23T10:42:41.540 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.540 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.540 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.540 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Prometheus container image", 2024-04-23T10:42:41.540 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.540 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.541 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.541 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.541 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_promtail": { 2024-04-23T10:42:41.541 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_promtail", 2024-04-23T10:42:41.541 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.541 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.541 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.541 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-23T10:42:41.541 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.541 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.541 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.541 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Promtail container image", 2024-04-23T10:42:41.541 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.541 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.542 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.542 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.542 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_samba": { 2024-04-23T10:42:41.542 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_samba", 2024-04-23T10:42:41.542 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.542 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.542 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.542 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-23T10:42:41.542 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.542 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.542 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.542 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Samba/SMB container image", 2024-04-23T10:42:41.542 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.542 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.543 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.543 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.543 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_snmp_gateway": { 2024-04-23T10:42:41.543 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_snmp_gateway", 2024-04-23T10:42:41.543 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.543 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.543 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.543 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-23T10:42:41.543 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.543 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.543 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.543 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "SNMP Gateway container image", 2024-04-23T10:42:41.543 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.544 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.544 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.544 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.544 INFO:teuthology.orchestra.run.smithi097.stdout: "container_init": { 2024-04-23T10:42:41.544 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_init", 2024-04-23T10:42:41.544 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.544 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.544 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.544 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:41.544 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.544 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.544 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.544 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Run podman/docker with `--init`", 2024-04-23T10:42:41.545 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.545 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.545 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.545 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.545 INFO:teuthology.orchestra.run.smithi097.stdout: "daemon_cache_timeout": { 2024-04-23T10:42:41.545 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "daemon_cache_timeout", 2024-04-23T10:42:41.545 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-04-23T10:42:41.545 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.545 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.545 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "600", 2024-04-23T10:42:41.545 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.545 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.545 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.545 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-23T10:42:41.545 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.546 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.546 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.546 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.546 INFO:teuthology.orchestra.run.smithi097.stdout: "default_cephadm_command_timeout": { 2024-04-23T10:42:41.546 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "default_cephadm_command_timeout", 2024-04-23T10:42:41.546 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:41.546 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.546 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.546 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "900", 2024-04-23T10:42:41.546 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.546 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.546 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.546 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-23T10:42:41.547 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.547 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.547 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.547 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.547 INFO:teuthology.orchestra.run.smithi097.stdout: "default_registry": { 2024-04-23T10:42:41.547 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "default_registry", 2024-04-23T10:42:41.547 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.547 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.547 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.547 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "docker.io", 2024-04-23T10:42:41.547 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.547 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.547 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.547 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-23T10:42:41.548 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.548 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.548 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.548 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.548 INFO:teuthology.orchestra.run.smithi097.stdout: "device_cache_timeout": { 2024-04-23T10:42:41.548 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "device_cache_timeout", 2024-04-23T10:42:41.548 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-04-23T10:42:41.548 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.548 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.548 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "1800", 2024-04-23T10:42:41.548 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.548 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.548 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.548 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "seconds to cache device inventory", 2024-04-23T10:42:41.548 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.549 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.549 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.549 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.549 INFO:teuthology.orchestra.run.smithi097.stdout: "device_enhanced_scan": { 2024-04-23T10:42:41.549 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "device_enhanced_scan", 2024-04-23T10:42:41.549 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.549 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.549 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.549 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.549 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.549 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.549 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.549 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-23T10:42:41.549 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.550 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.550 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.550 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.550 INFO:teuthology.orchestra.run.smithi097.stdout: "facts_cache_timeout": { 2024-04-23T10:42:41.550 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "facts_cache_timeout", 2024-04-23T10:42:41.550 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-04-23T10:42:41.550 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.550 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.550 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "60", 2024-04-23T10:42:41.550 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.550 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.550 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.550 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "seconds to cache host facts data", 2024-04-23T10:42:41.550 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.551 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.551 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.551 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.551 INFO:teuthology.orchestra.run.smithi097.stdout: "grafana_dashboards_path": { 2024-04-23T10:42:41.551 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "grafana_dashboards_path", 2024-04-23T10:42:41.551 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.551 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.551 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.551 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-23T10:42:41.551 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.551 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.551 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.551 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-23T10:42:41.551 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.552 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.552 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.552 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.552 INFO:teuthology.orchestra.run.smithi097.stdout: "host_check_interval": { 2024-04-23T10:42:41.552 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "host_check_interval", 2024-04-23T10:42:41.552 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-04-23T10:42:41.552 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.552 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.552 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "600", 2024-04-23T10:42:41.552 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.552 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.552 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.552 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how frequently to perform a host check", 2024-04-23T10:42:41.552 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.552 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.553 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.553 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.553 INFO:teuthology.orchestra.run.smithi097.stdout: "hw_monitoring": { 2024-04-23T10:42:41.553 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "hw_monitoring", 2024-04-23T10:42:41.553 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.553 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.553 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.553 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.553 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.553 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.553 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.553 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-23T10:42:41.553 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.553 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.553 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.554 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.554 INFO:teuthology.orchestra.run.smithi097.stdout: "inventory_list_all": { 2024-04-23T10:42:41.554 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "inventory_list_all", 2024-04-23T10:42:41.554 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.554 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.554 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.554 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.554 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.554 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.554 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.554 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-23T10:42:41.554 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.554 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.555 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.555 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.555 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:41.555 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:41.555 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.555 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.555 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.555 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.555 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.555 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.555 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.555 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.555 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.555 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.555 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.556 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.556 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.556 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.556 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.556 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.556 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.556 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.556 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.556 INFO:teuthology.orchestra.run.smithi097.stdout: "log_refresh_metadata": { 2024-04-23T10:42:41.556 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_refresh_metadata", 2024-04-23T10:42:41.556 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.556 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.556 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.556 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.557 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.557 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.557 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.557 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-23T10:42:41.557 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.557 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.557 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.557 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.557 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:41.557 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:41.557 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.557 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.557 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.557 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:41.558 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.558 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.558 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.558 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-23T10:42:41.558 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.558 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.558 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.558 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.558 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:41.558 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:41.558 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.558 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.558 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.558 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:41.559 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.559 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.559 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.559 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.559 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.559 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.559 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.559 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.559 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.559 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.559 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.559 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.559 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.559 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.560 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.560 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:41.560 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:41.560 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.560 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.560 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.560 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.560 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.560 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.560 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.560 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.560 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.560 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.561 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.561 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.561 INFO:teuthology.orchestra.run.smithi097.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-23T10:42:41.561 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-23T10:42:41.561 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.561 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.561 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.561 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.561 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.561 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.561 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.561 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-23T10:42:41.561 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.561 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.561 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.562 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.562 INFO:teuthology.orchestra.run.smithi097.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-23T10:42:41.562 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-23T10:42:41.562 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.562 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.562 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.562 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "*", 2024-04-23T10:42:41.562 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.562 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.562 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.562 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-23T10:42:41.562 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.562 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.562 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.563 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.563 INFO:teuthology.orchestra.run.smithi097.stdout: "max_count_per_host": { 2024-04-23T10:42:41.563 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "max_count_per_host", 2024-04-23T10:42:41.563 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:41.563 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.563 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.563 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-04-23T10:42:41.563 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.563 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.563 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.563 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "max number of daemons per service per host", 2024-04-23T10:42:41.563 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.563 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.563 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.563 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.564 INFO:teuthology.orchestra.run.smithi097.stdout: "max_osd_draining_count": { 2024-04-23T10:42:41.564 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "max_osd_draining_count", 2024-04-23T10:42:41.564 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:41.564 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.564 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.564 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-04-23T10:42:41.564 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.564 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.564 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.564 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-23T10:42:41.564 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.564 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.564 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.564 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.564 INFO:teuthology.orchestra.run.smithi097.stdout: "migration_current": { 2024-04-23T10:42:41.565 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "migration_current", 2024-04-23T10:42:41.565 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:41.565 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.565 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.565 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.565 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.565 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.565 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.565 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "internal - do not modify", 2024-04-23T10:42:41.565 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.565 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.565 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.565 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.565 INFO:teuthology.orchestra.run.smithi097.stdout: "mode": { 2024-04-23T10:42:41.565 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mode", 2024-04-23T10:42:41.566 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.566 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.566 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.566 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "root", 2024-04-23T10:42:41.566 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.566 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.566 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.566 INFO:teuthology.orchestra.run.smithi097.stdout: "cephadm-package", 2024-04-23T10:42:41.566 INFO:teuthology.orchestra.run.smithi097.stdout: "root" 2024-04-23T10:42:41.566 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.566 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "mode for remote execution of cephadm", 2024-04-23T10:42:41.566 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.566 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.567 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.567 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.567 INFO:teuthology.orchestra.run.smithi097.stdout: "oob_default_addr": { 2024-04-23T10:42:41.567 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "oob_default_addr", 2024-04-23T10:42:41.567 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.567 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.567 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.567 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "169.254.1.1", 2024-04-23T10:42:41.567 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.567 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.567 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.567 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-23T10:42:41.567 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.567 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.567 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.568 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.568 INFO:teuthology.orchestra.run.smithi097.stdout: "prometheus_alerts_path": { 2024-04-23T10:42:41.568 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "prometheus_alerts_path", 2024-04-23T10:42:41.568 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.568 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.568 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.568 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-23T10:42:41.568 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.568 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.568 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.568 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-23T10:42:41.568 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.568 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.568 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.568 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.569 INFO:teuthology.orchestra.run.smithi097.stdout: "registry_insecure": { 2024-04-23T10:42:41.569 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "registry_insecure", 2024-04-23T10:42:41.569 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.569 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.569 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.569 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.569 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.569 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.569 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.569 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-23T10:42:41.569 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.569 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.569 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.569 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.570 INFO:teuthology.orchestra.run.smithi097.stdout: "registry_password": { 2024-04-23T10:42:41.570 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "registry_password", 2024-04-23T10:42:41.570 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.570 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.570 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.570 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.570 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.570 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.570 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.570 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-23T10:42:41.570 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.570 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.570 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.570 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.571 INFO:teuthology.orchestra.run.smithi097.stdout: "registry_url": { 2024-04-23T10:42:41.571 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "registry_url", 2024-04-23T10:42:41.571 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.571 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.571 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.571 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.571 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.571 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.571 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.571 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-23T10:42:41.571 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.571 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.571 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.572 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.572 INFO:teuthology.orchestra.run.smithi097.stdout: "registry_username": { 2024-04-23T10:42:41.572 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "registry_username", 2024-04-23T10:42:41.572 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.572 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.572 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.572 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.572 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.572 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.572 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.572 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-23T10:42:41.572 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.572 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.572 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.573 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.573 INFO:teuthology.orchestra.run.smithi097.stdout: "secure_monitoring_stack": { 2024-04-23T10:42:41.573 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "secure_monitoring_stack", 2024-04-23T10:42:41.573 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.573 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.573 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.573 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.573 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.573 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.573 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.573 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-23T10:42:41.573 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.573 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.574 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.574 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.574 INFO:teuthology.orchestra.run.smithi097.stdout: "service_discovery_port": { 2024-04-23T10:42:41.574 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "service_discovery_port", 2024-04-23T10:42:41.574 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:41.574 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.574 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.574 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "8765", 2024-04-23T10:42:41.574 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.574 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.574 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.574 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "cephadm service discovery port", 2024-04-23T10:42:41.574 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.575 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.575 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.575 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.575 INFO:teuthology.orchestra.run.smithi097.stdout: "ssh_config_file": { 2024-04-23T10:42:41.575 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ssh_config_file", 2024-04-23T10:42:41.575 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.575 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.575 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.575 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.575 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.575 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.575 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.575 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-23T10:42:41.575 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.576 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.576 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.576 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.576 INFO:teuthology.orchestra.run.smithi097.stdout: "use_agent": { 2024-04-23T10:42:41.576 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "use_agent", 2024-04-23T10:42:41.576 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.576 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.576 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.576 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.576 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.576 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.576 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.576 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-23T10:42:41.576 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.577 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.577 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.577 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.577 INFO:teuthology.orchestra.run.smithi097.stdout: "use_repo_digest": { 2024-04-23T10:42:41.577 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "use_repo_digest", 2024-04-23T10:42:41.577 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.577 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.577 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.577 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:41.577 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.577 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.577 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.577 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-23T10:42:41.577 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.578 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.578 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.578 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.578 INFO:teuthology.orchestra.run.smithi097.stdout: "warn_on_failed_host_check": { 2024-04-23T10:42:41.578 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "warn_on_failed_host_check", 2024-04-23T10:42:41.578 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.578 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.578 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.578 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:41.578 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.578 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.578 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.579 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "raise a health warning if the host check fails", 2024-04-23T10:42:41.579 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.579 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.579 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.579 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.579 INFO:teuthology.orchestra.run.smithi097.stdout: "warn_on_stray_daemons": { 2024-04-23T10:42:41.579 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "warn_on_stray_daemons", 2024-04-23T10:42:41.579 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.579 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.579 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.579 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:41.579 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.579 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.580 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.580 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-23T10:42:41.580 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.580 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.580 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.580 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.580 INFO:teuthology.orchestra.run.smithi097.stdout: "warn_on_stray_hosts": { 2024-04-23T10:42:41.580 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "warn_on_stray_hosts", 2024-04-23T10:42:41.580 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.580 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.580 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.580 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:41.580 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.580 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.581 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.581 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-23T10:42:41.581 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.581 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.581 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.581 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.581 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.581 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.581 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:41.581 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "crash", 2024-04-23T10:42:41.581 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:41.581 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:41.581 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:41.582 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:41.582 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:41.582 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.582 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.582 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.582 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.582 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.582 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.582 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.582 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.582 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.582 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.583 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.583 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.583 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.583 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.583 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.583 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.583 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.583 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.583 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.583 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:41.583 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:41.583 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.583 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.584 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.584 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.584 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.584 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.584 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.584 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.584 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.584 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.584 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.584 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.584 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:41.584 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:41.584 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.584 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.585 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.585 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:41.585 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.585 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.585 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.585 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.585 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.585 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.585 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.585 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.585 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.585 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.585 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.585 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.586 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.586 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.586 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.586 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:41.586 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:41.586 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.586 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.586 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.586 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.586 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.586 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.586 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.586 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.587 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.587 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.587 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.587 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.587 INFO:teuthology.orchestra.run.smithi097.stdout: "retain_interval": { 2024-04-23T10:42:41.587 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "retain_interval", 2024-04-23T10:42:41.587 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-04-23T10:42:41.587 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.587 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.587 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "31536000", 2024-04-23T10:42:41.587 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.587 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.587 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.587 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-23T10:42:41.588 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.588 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.588 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.588 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.588 INFO:teuthology.orchestra.run.smithi097.stdout: "warn_recent_interval": { 2024-04-23T10:42:41.588 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "warn_recent_interval", 2024-04-23T10:42:41.588 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-04-23T10:42:41.588 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.588 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.588 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "1209600", 2024-04-23T10:42:41.588 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.588 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.588 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.588 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-23T10:42:41.588 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.589 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.589 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.589 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.589 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.589 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.589 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:41.589 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "dashboard", 2024-04-23T10:42:41.589 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:41.589 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:41.589 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:41.589 INFO:teuthology.orchestra.run.smithi097.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-23T10:42:41.589 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-23T10:42:41.589 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:41.589 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.590 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.590 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-04-23T10:42:41.590 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.590 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.590 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.590 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.590 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.590 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.590 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.590 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.590 INFO:teuthology.orchestra.run.smithi097.stdout: "ALERTMANAGER_API_HOST": { 2024-04-23T10:42:41.590 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-23T10:42:41.590 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.590 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.591 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.591 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.591 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.591 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.591 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.591 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.591 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.591 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.591 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.591 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.591 INFO:teuthology.orchestra.run.smithi097.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-23T10:42:41.591 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-23T10:42:41.591 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.591 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.591 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.592 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:41.592 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.592 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.592 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.592 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.592 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.592 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.592 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.592 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.592 INFO:teuthology.orchestra.run.smithi097.stdout: "AUDIT_API_ENABLED": { 2024-04-23T10:42:41.592 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "AUDIT_API_ENABLED", 2024-04-23T10:42:41.592 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.592 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.592 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.592 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.593 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.593 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.593 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.593 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.593 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.593 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.593 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.593 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.593 INFO:teuthology.orchestra.run.smithi097.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-23T10:42:41.593 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-23T10:42:41.593 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.593 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.593 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.593 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:41.593 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.594 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.594 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.594 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.594 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.594 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.594 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.594 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.594 INFO:teuthology.orchestra.run.smithi097.stdout: "ENABLE_BROWSABLE_API": { 2024-04-23T10:42:41.594 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-23T10:42:41.594 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.594 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.594 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.594 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:41.594 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.595 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.595 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.595 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.595 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.595 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.595 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.595 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.595 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-23T10:42:41.595 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-23T10:42:41.595 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.595 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.595 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.595 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:41.595 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.595 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.596 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.596 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.596 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.596 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.596 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.596 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.596 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-23T10:42:41.596 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-23T10:42:41.596 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.596 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.596 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.596 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:41.596 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.596 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.596 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.597 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.597 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.597 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.597 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.597 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.597 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-23T10:42:41.597 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-23T10:42:41.597 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.597 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.597 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.597 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:41.597 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.597 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.597 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.597 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.598 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.598 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.598 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.598 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.598 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-23T10:42:41.598 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-23T10:42:41.598 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.598 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.598 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.598 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:41.598 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.598 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.598 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.598 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.598 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.599 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.599 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.599 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.599 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-23T10:42:41.599 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-23T10:42:41.599 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.599 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.599 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.599 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:41.599 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.599 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.599 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.599 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.599 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.599 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.600 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.600 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.600 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-23T10:42:41.600 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-23T10:42:41.600 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.600 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.600 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.600 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:41.600 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.600 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.600 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.600 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.600 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.600 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.600 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.600 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.601 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-23T10:42:41.601 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-23T10:42:41.601 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.601 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.601 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.601 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:41.601 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.601 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.601 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.601 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.601 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.601 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.601 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.601 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.601 INFO:teuthology.orchestra.run.smithi097.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-23T10:42:41.602 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-23T10:42:41.602 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.602 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.602 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.602 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.602 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.602 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.602 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.602 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.602 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.602 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.602 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.602 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.602 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_API_PASSWORD": { 2024-04-23T10:42:41.602 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-23T10:42:41.603 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.603 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.603 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.603 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "admin", 2024-04-23T10:42:41.603 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.603 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.603 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.603 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.603 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.603 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.603 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.603 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.603 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-23T10:42:41.603 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-23T10:42:41.603 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.603 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.604 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.604 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:41.604 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.604 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.604 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.604 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.604 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.604 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.604 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.604 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.604 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_API_URL": { 2024-04-23T10:42:41.604 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_API_URL", 2024-04-23T10:42:41.604 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.604 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.604 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.605 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.605 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.605 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.605 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.605 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.605 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.605 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.605 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.605 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.605 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_API_USERNAME": { 2024-04-23T10:42:41.605 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-23T10:42:41.605 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.605 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.605 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.605 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "admin", 2024-04-23T10:42:41.605 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.606 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.606 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.606 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.606 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.606 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.606 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.606 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.606 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-23T10:42:41.606 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-23T10:42:41.606 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.606 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.606 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.606 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.606 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.606 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.607 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.607 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.607 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.607 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.607 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.607 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.607 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-23T10:42:41.607 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-23T10:42:41.607 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.607 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.607 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.607 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.607 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.607 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.607 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.608 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.608 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.608 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.608 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.608 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.608 INFO:teuthology.orchestra.run.smithi097.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-23T10:42:41.608 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-23T10:42:41.608 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.608 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.608 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.608 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:41.608 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.608 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.608 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.608 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.609 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.609 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.609 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.609 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.609 INFO:teuthology.orchestra.run.smithi097.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-23T10:42:41.609 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-23T10:42:41.609 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.609 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.609 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.609 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.609 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.609 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.609 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.609 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.609 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.609 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.610 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.610 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.610 INFO:teuthology.orchestra.run.smithi097.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-23T10:42:41.610 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-23T10:42:41.610 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.610 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.610 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.610 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "[]", 2024-04-23T10:42:41.610 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.610 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.610 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.610 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.610 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.610 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.610 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.611 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.611 INFO:teuthology.orchestra.run.smithi097.stdout: "MULTICLUSTER_CONFIG": { 2024-04-23T10:42:41.611 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-23T10:42:41.611 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.611 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.611 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.611 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "{}", 2024-04-23T10:42:41.611 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.611 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.611 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.611 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.611 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.611 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.611 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.611 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.612 INFO:teuthology.orchestra.run.smithi097.stdout: "PROMETHEUS_API_HOST": { 2024-04-23T10:42:41.612 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-23T10:42:41.612 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.612 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.612 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.612 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.612 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.612 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.612 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.612 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.612 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.612 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.612 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.612 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.612 INFO:teuthology.orchestra.run.smithi097.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-23T10:42:41.613 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-23T10:42:41.613 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.613 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.613 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.613 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:41.613 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.613 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.613 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.613 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.613 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.613 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.613 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.613 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.613 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-23T10:42:41.613 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-23T10:42:41.613 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.614 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.614 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.614 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.614 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.614 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.614 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.614 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.614 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.614 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.614 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.614 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.614 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-23T10:42:41.614 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-23T10:42:41.614 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.614 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.615 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.615 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.615 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.615 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.615 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.615 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.615 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.615 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.615 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.615 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.615 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-23T10:42:41.615 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-23T10:42:41.615 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.615 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.615 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.615 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:41.616 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.616 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.616 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.616 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.616 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.616 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.616 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.616 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.616 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-23T10:42:41.616 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-23T10:42:41.616 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.616 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.616 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.616 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:41.616 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.616 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.617 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.617 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.617 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.617 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.617 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.617 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.617 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-23T10:42:41.617 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-23T10:42:41.617 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.617 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.617 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.617 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.617 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.617 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.617 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.617 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.618 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.618 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.618 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.618 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.618 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-23T10:42:41.618 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-23T10:42:41.618 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.618 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.618 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.618 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.618 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.618 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.618 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.618 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.618 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.619 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.619 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.619 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.619 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-23T10:42:41.619 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-23T10:42:41.619 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.619 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.619 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.619 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.619 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.619 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.619 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.619 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.619 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.619 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.620 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.620 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.620 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_ENABLED": { 2024-04-23T10:42:41.620 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-23T10:42:41.620 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.620 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.620 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.620 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:41.620 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.620 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.620 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.620 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.620 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.620 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.620 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.620 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.621 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-23T10:42:41.621 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-23T10:42:41.621 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.621 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.621 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.621 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-23T10:42:41.621 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.621 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.621 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.621 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.621 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.621 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.621 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.621 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.621 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-23T10:42:41.622 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-23T10:42:41.622 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:41.622 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.622 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.622 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-04-23T10:42:41.622 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.622 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.622 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.622 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.622 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.622 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.622 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.622 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.622 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-23T10:42:41.623 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-23T10:42:41.623 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:41.623 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.623 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.623 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "8", 2024-04-23T10:42:41.623 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.623 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.623 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.623 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.623 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.623 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.623 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.623 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.623 INFO:teuthology.orchestra.run.smithi097.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-23T10:42:41.623 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-23T10:42:41.623 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:41.623 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.624 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.624 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "45", 2024-04-23T10:42:41.624 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.624 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.624 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.624 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.624 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.624 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.624 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.624 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.624 INFO:teuthology.orchestra.run.smithi097.stdout: "RGW_API_ACCESS_KEY": { 2024-04-23T10:42:41.624 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-23T10:42:41.624 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.624 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.624 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.624 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.625 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.625 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.625 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.625 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.625 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.625 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.625 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.625 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.625 INFO:teuthology.orchestra.run.smithi097.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-23T10:42:41.625 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-23T10:42:41.625 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.625 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.625 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.625 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "admin", 2024-04-23T10:42:41.625 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.625 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.626 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.626 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.626 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.626 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.626 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.626 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.626 INFO:teuthology.orchestra.run.smithi097.stdout: "RGW_API_SECRET_KEY": { 2024-04-23T10:42:41.626 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-23T10:42:41.626 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.626 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.626 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.626 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.626 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.626 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.626 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.627 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.627 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.627 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.627 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.627 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.627 INFO:teuthology.orchestra.run.smithi097.stdout: "RGW_API_SSL_VERIFY": { 2024-04-23T10:42:41.627 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-23T10:42:41.627 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.627 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.627 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.627 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:41.627 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.627 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.627 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.627 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.627 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.628 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.628 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.628 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.628 INFO:teuthology.orchestra.run.smithi097.stdout: "UNSAFE_TLS_v1_2": { 2024-04-23T10:42:41.628 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-23T10:42:41.628 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.628 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.628 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.628 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.628 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.628 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.628 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.628 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.628 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.628 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.628 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.628 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.629 INFO:teuthology.orchestra.run.smithi097.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-23T10:42:41.629 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-23T10:42:41.629 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:41.629 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.629 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.629 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0", 2024-04-23T10:42:41.629 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.629 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.629 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.629 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.629 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.629 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.629 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.629 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.629 INFO:teuthology.orchestra.run.smithi097.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-23T10:42:41.629 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-23T10:42:41.630 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:41.630 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.630 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.630 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-04-23T10:42:41.630 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.630 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.630 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.630 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.630 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.630 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.630 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.630 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.630 INFO:teuthology.orchestra.run.smithi097.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-23T10:42:41.631 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-23T10:42:41.631 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:41.631 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.631 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.631 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "5", 2024-04-23T10:42:41.631 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.631 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.631 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.631 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.631 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.631 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.631 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.631 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.631 INFO:teuthology.orchestra.run.smithi097.stdout: "cross_origin_url": { 2024-04-23T10:42:41.632 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "cross_origin_url", 2024-04-23T10:42:41.632 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.632 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.632 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.632 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.632 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.632 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.632 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.632 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.632 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.632 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.632 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.632 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.632 INFO:teuthology.orchestra.run.smithi097.stdout: "crt_file": { 2024-04-23T10:42:41.632 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "crt_file", 2024-04-23T10:42:41.632 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.633 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.633 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.633 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.633 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.633 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.633 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.633 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.633 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.633 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.633 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.633 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.633 INFO:teuthology.orchestra.run.smithi097.stdout: "debug": { 2024-04-23T10:42:41.633 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "debug", 2024-04-23T10:42:41.633 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.633 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.633 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.633 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.634 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.634 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.634 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.634 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Enable/disable debug options", 2024-04-23T10:42:41.634 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.634 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.634 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.634 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.634 INFO:teuthology.orchestra.run.smithi097.stdout: "jwt_token_ttl": { 2024-04-23T10:42:41.634 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "jwt_token_ttl", 2024-04-23T10:42:41.634 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:41.634 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.634 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.634 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "28800", 2024-04-23T10:42:41.634 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.635 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.635 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.635 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.635 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.635 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.635 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.635 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.635 INFO:teuthology.orchestra.run.smithi097.stdout: "key_file": { 2024-04-23T10:42:41.635 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "key_file", 2024-04-23T10:42:41.635 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.635 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.635 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.635 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.635 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.635 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.635 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.636 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.636 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.636 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.636 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.636 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.636 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:41.636 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:41.636 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.636 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.636 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.636 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.636 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.636 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.636 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.636 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.636 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.637 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.637 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.637 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.637 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.637 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.637 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.637 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.637 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.637 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.637 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.637 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:41.637 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:41.637 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.637 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.637 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.637 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.638 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.638 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.638 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.638 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.638 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.638 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.638 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.638 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.638 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:41.638 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:41.638 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.638 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.638 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.638 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:41.638 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.639 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.639 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.639 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.639 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.639 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.639 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.639 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.639 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.639 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.639 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.639 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.639 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.639 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.639 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.639 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:41.639 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:41.639 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.640 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.640 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.640 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.640 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.640 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.640 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.640 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.640 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.640 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.640 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.640 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.640 INFO:teuthology.orchestra.run.smithi097.stdout: "motd": { 2024-04-23T10:42:41.640 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "motd", 2024-04-23T10:42:41.640 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.640 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.640 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.640 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.641 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.641 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.641 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.641 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "The message of the day", 2024-04-23T10:42:41.641 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.641 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.641 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.641 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.641 INFO:teuthology.orchestra.run.smithi097.stdout: "redirect_resolve_ip_addr": { 2024-04-23T10:42:41.641 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "redirect_resolve_ip_addr", 2024-04-23T10:42:41.641 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.641 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.641 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.641 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.641 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.641 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.641 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.642 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.642 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.642 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.642 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.642 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.642 INFO:teuthology.orchestra.run.smithi097.stdout: "server_addr": { 2024-04-23T10:42:41.642 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_addr", 2024-04-23T10:42:41.642 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.642 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.642 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.642 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "::", 2024-04-23T10:42:41.642 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.642 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.642 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.643 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.643 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.643 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.643 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.643 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.643 INFO:teuthology.orchestra.run.smithi097.stdout: "server_port": { 2024-04-23T10:42:41.643 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_port", 2024-04-23T10:42:41.643 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:41.643 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.643 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.643 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "8080", 2024-04-23T10:42:41.643 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.643 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.643 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.643 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.643 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.644 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.644 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.644 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.644 INFO:teuthology.orchestra.run.smithi097.stdout: "ssl": { 2024-04-23T10:42:41.644 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ssl", 2024-04-23T10:42:41.644 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.644 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.644 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.644 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:41.644 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.644 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.644 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.644 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.644 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.644 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.644 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.645 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.645 INFO:teuthology.orchestra.run.smithi097.stdout: "ssl_server_port": { 2024-04-23T10:42:41.645 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ssl_server_port", 2024-04-23T10:42:41.645 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:41.645 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.645 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.645 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "8443", 2024-04-23T10:42:41.645 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.645 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.645 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.645 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.645 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.645 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.645 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.645 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.645 INFO:teuthology.orchestra.run.smithi097.stdout: "standby_behaviour": { 2024-04-23T10:42:41.645 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "standby_behaviour", 2024-04-23T10:42:41.646 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.646 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.646 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.646 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "redirect", 2024-04-23T10:42:41.646 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.646 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.646 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.646 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.646 INFO:teuthology.orchestra.run.smithi097.stdout: "redirect" 2024-04-23T10:42:41.646 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.646 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.646 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.646 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.646 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.646 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.647 INFO:teuthology.orchestra.run.smithi097.stdout: "standby_error_status_code": { 2024-04-23T10:42:41.647 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "standby_error_status_code", 2024-04-23T10:42:41.647 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:41.647 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.647 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.647 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "500", 2024-04-23T10:42:41.647 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "400", 2024-04-23T10:42:41.647 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "599", 2024-04-23T10:42:41.647 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.647 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.647 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.647 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.647 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.647 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.647 INFO:teuthology.orchestra.run.smithi097.stdout: "url_prefix": { 2024-04-23T10:42:41.647 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "url_prefix", 2024-04-23T10:42:41.648 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.648 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.648 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.648 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.648 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.648 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.648 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.648 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.648 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.648 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.648 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.648 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.648 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.648 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.648 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:41.648 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "devicehealth", 2024-04-23T10:42:41.649 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:41.649 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:41.649 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:41.649 INFO:teuthology.orchestra.run.smithi097.stdout: "enable_monitoring": { 2024-04-23T10:42:41.649 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "enable_monitoring", 2024-04-23T10:42:41.649 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.649 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.649 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.649 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:41.649 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.649 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.649 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.649 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "monitor device health metrics", 2024-04-23T10:42:41.649 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.649 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.649 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.650 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.650 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:41.650 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:41.650 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.650 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.650 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.650 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.650 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.650 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.650 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.650 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.650 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.650 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.650 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.650 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.651 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.651 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.651 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.651 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.651 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.651 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.651 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.651 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:41.651 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:41.651 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.651 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.651 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.651 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.651 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.651 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.651 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.652 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.652 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.652 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.652 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.652 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.652 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:41.652 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:41.652 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.652 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.652 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.652 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:41.652 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.652 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.652 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.652 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.652 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.653 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.653 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.653 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.653 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.653 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.653 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.653 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.653 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.653 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.653 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.653 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:41.653 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:41.653 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.653 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.653 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.653 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.653 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.654 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.654 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.654 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.654 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.654 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.654 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.654 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.654 INFO:teuthology.orchestra.run.smithi097.stdout: "mark_out_threshold": { 2024-04-23T10:42:41.654 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mark_out_threshold", 2024-04-23T10:42:41.654 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-04-23T10:42:41.654 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.654 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.654 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "2419200", 2024-04-23T10:42:41.654 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.654 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.655 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.655 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-23T10:42:41.655 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.655 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.655 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.655 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.655 INFO:teuthology.orchestra.run.smithi097.stdout: "pool_name": { 2024-04-23T10:42:41.655 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "pool_name", 2024-04-23T10:42:41.655 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.655 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.655 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.655 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "device_health_metrics", 2024-04-23T10:42:41.655 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.655 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.655 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.656 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-23T10:42:41.656 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.656 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.656 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.656 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.656 INFO:teuthology.orchestra.run.smithi097.stdout: "retention_period": { 2024-04-23T10:42:41.656 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "retention_period", 2024-04-23T10:42:41.656 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-04-23T10:42:41.656 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.656 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.656 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "15552000", 2024-04-23T10:42:41.656 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.656 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.656 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.656 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how long to retain device health metrics", 2024-04-23T10:42:41.656 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.657 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.657 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.657 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.657 INFO:teuthology.orchestra.run.smithi097.stdout: "scrape_frequency": { 2024-04-23T10:42:41.657 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "scrape_frequency", 2024-04-23T10:42:41.657 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-04-23T10:42:41.657 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.657 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.657 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "86400", 2024-04-23T10:42:41.657 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.657 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.657 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.657 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-23T10:42:41.657 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.657 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.657 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.658 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.658 INFO:teuthology.orchestra.run.smithi097.stdout: "self_heal": { 2024-04-23T10:42:41.658 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "self_heal", 2024-04-23T10:42:41.658 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.658 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.658 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.658 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:41.658 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.658 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.658 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.658 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-23T10:42:41.658 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.658 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.658 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.658 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.659 INFO:teuthology.orchestra.run.smithi097.stdout: "sleep_interval": { 2024-04-23T10:42:41.659 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "sleep_interval", 2024-04-23T10:42:41.659 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-04-23T10:42:41.659 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.659 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.659 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "600", 2024-04-23T10:42:41.659 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.659 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.659 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.659 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how frequently to wake up and check device health", 2024-04-23T10:42:41.659 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.659 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.659 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.659 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.659 INFO:teuthology.orchestra.run.smithi097.stdout: "warn_threshold": { 2024-04-23T10:42:41.659 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "warn_threshold", 2024-04-23T10:42:41.660 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-04-23T10:42:41.660 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.660 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.660 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "7257600", 2024-04-23T10:42:41.660 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.660 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.660 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.660 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-23T10:42:41.660 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.660 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.660 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.660 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.660 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.660 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.660 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:41.660 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "diskprediction_local", 2024-04-23T10:42:41.661 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:41.661 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:41.661 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:41.661 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:41.661 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:41.661 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.661 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.661 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.661 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.661 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.661 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.661 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.661 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.661 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.661 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.661 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.662 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.662 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.662 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.662 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.662 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.662 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.662 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.662 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.662 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:41.662 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:41.662 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.662 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.662 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.662 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.662 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.663 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.663 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.663 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.663 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.663 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.663 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.663 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.663 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:41.663 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:41.663 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.663 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.663 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.663 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:41.663 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.663 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.663 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.664 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.664 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.664 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.664 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.664 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.664 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.664 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.664 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.664 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.664 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.664 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.664 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.664 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:41.664 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:41.664 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.665 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.665 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.665 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.665 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.665 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.665 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.665 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.665 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.665 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.665 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.665 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.665 INFO:teuthology.orchestra.run.smithi097.stdout: "predict_interval": { 2024-04-23T10:42:41.665 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "predict_interval", 2024-04-23T10:42:41.665 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.665 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.665 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.666 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "86400", 2024-04-23T10:42:41.666 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.666 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.666 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.666 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.666 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.666 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.666 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.666 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.666 INFO:teuthology.orchestra.run.smithi097.stdout: "predictor_model": { 2024-04-23T10:42:41.666 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "predictor_model", 2024-04-23T10:42:41.666 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.666 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.666 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.666 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "prophetstor", 2024-04-23T10:42:41.667 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.667 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.667 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.667 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.667 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.667 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.667 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.667 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.667 INFO:teuthology.orchestra.run.smithi097.stdout: "sleep_interval": { 2024-04-23T10:42:41.667 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "sleep_interval", 2024-04-23T10:42:41.667 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.667 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.667 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.667 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "600", 2024-04-23T10:42:41.667 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.668 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.668 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.668 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.668 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.668 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.668 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.668 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.668 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.668 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.668 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:41.668 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "influx", 2024-04-23T10:42:41.668 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": false, 2024-04-23T10:42:41.668 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "influxdb python module not found", 2024-04-23T10:42:41.668 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:41.668 INFO:teuthology.orchestra.run.smithi097.stdout: "batch_size": { 2024-04-23T10:42:41.668 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "batch_size", 2024-04-23T10:42:41.669 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:41.669 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.669 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.669 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "5000", 2024-04-23T10:42:41.669 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.669 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.669 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.669 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-23T10:42:41.669 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.669 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.669 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.669 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.669 INFO:teuthology.orchestra.run.smithi097.stdout: "database": { 2024-04-23T10:42:41.669 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "database", 2024-04-23T10:42:41.669 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.669 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.670 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.670 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "ceph", 2024-04-23T10:42:41.670 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.670 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.670 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.670 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-23T10:42:41.670 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.670 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.670 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.670 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.670 INFO:teuthology.orchestra.run.smithi097.stdout: "hostname": { 2024-04-23T10:42:41.670 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "hostname", 2024-04-23T10:42:41.670 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.670 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.670 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.671 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.671 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.671 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.671 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.671 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "InfluxDB server hostname", 2024-04-23T10:42:41.671 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.671 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.671 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.671 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.671 INFO:teuthology.orchestra.run.smithi097.stdout: "interval": { 2024-04-23T10:42:41.671 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "interval", 2024-04-23T10:42:41.671 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-04-23T10:42:41.671 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.671 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.671 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "30", 2024-04-23T10:42:41.672 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "5", 2024-04-23T10:42:41.672 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.672 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.672 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-23T10:42:41.672 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.672 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.672 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.672 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.672 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:41.672 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:41.672 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.672 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.672 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.672 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.672 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.672 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.673 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.673 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.673 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.673 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.673 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.673 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.673 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.673 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.673 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.673 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.673 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.673 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.673 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.673 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:41.673 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:41.674 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.674 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.674 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.674 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.674 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.674 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.674 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.674 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.674 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.674 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.674 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.674 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.674 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:41.674 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:41.675 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.675 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.675 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.675 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:41.675 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.675 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.675 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.675 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.675 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.675 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.675 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.675 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.675 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.675 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.675 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.675 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.676 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.676 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.676 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.676 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:41.676 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:41.676 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.676 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.676 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.676 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.676 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.676 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.676 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.676 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.676 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.676 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.676 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.677 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.677 INFO:teuthology.orchestra.run.smithi097.stdout: "password": { 2024-04-23T10:42:41.677 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "password", 2024-04-23T10:42:41.677 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.677 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.677 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.677 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.677 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.677 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.677 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.677 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "password of InfluxDB server user", 2024-04-23T10:42:41.677 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.677 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.677 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.677 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.677 INFO:teuthology.orchestra.run.smithi097.stdout: "port": { 2024-04-23T10:42:41.677 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "port", 2024-04-23T10:42:41.678 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:41.678 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.678 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.678 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "8086", 2024-04-23T10:42:41.678 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.678 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.678 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.678 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "InfluxDB server port", 2024-04-23T10:42:41.678 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.678 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.678 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.678 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.678 INFO:teuthology.orchestra.run.smithi097.stdout: "ssl": { 2024-04-23T10:42:41.678 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ssl", 2024-04-23T10:42:41.678 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.679 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.679 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.679 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "false", 2024-04-23T10:42:41.679 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.679 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.679 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.679 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-23T10:42:41.679 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.679 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.679 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.679 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.679 INFO:teuthology.orchestra.run.smithi097.stdout: "threads": { 2024-04-23T10:42:41.679 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "threads", 2024-04-23T10:42:41.679 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:41.679 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.680 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.680 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "5", 2024-04-23T10:42:41.680 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "1", 2024-04-23T10:42:41.680 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "32", 2024-04-23T10:42:41.680 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.680 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-23T10:42:41.680 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.680 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.680 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.680 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.680 INFO:teuthology.orchestra.run.smithi097.stdout: "username": { 2024-04-23T10:42:41.680 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "username", 2024-04-23T10:42:41.680 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.680 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.680 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.680 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.681 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.681 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.681 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.681 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "username of InfluxDB server user", 2024-04-23T10:42:41.681 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.681 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.681 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.681 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.681 INFO:teuthology.orchestra.run.smithi097.stdout: "verify_ssl": { 2024-04-23T10:42:41.681 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "verify_ssl", 2024-04-23T10:42:41.681 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.681 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.681 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.681 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "true", 2024-04-23T10:42:41.681 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.682 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.682 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.682 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-23T10:42:41.682 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.682 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.682 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.682 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.682 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.682 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.682 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:41.682 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "insights", 2024-04-23T10:42:41.682 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:41.682 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:41.682 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:41.683 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:41.683 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:41.683 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.683 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.683 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.683 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.683 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.683 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.683 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.683 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.683 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.683 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.683 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.683 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.683 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.683 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.684 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.684 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.684 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.684 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.684 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.684 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:41.684 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:41.684 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.684 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.684 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.684 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.684 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.684 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.684 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.684 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.684 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.685 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.685 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.685 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.685 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:41.685 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:41.685 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.685 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.685 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.685 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:41.685 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.685 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.685 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.685 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.685 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.685 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.686 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.686 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.686 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.686 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.686 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.686 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.686 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.686 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.686 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.686 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:41.686 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:41.686 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.686 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.686 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.686 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.686 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.687 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.687 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.687 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.687 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.687 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.687 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.687 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.687 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.687 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.687 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:41.687 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "iostat", 2024-04-23T10:42:41.687 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:41.687 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:41.687 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:41.687 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:41.688 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:41.688 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.688 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.688 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.688 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.688 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.688 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.688 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.688 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.688 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.688 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.688 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.688 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.688 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.688 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.688 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.689 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.689 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.689 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.689 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.689 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:41.689 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:41.689 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.689 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.689 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.689 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.689 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.689 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.689 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.689 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.689 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.689 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.689 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.690 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.690 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:41.690 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:41.690 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.690 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.690 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.690 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:41.690 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.690 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.690 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.690 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.690 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.690 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.690 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.690 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.691 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.691 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.691 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.691 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.691 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.691 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.691 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.691 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:41.691 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:41.691 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.691 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.691 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.691 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.691 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.691 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.692 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.692 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.692 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.692 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.692 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.692 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.692 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.692 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.692 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:41.692 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "k8sevents", 2024-04-23T10:42:41.692 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:41.692 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:41.692 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:41.692 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_event_retention_days": { 2024-04-23T10:42:41.692 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ceph_event_retention_days", 2024-04-23T10:42:41.692 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:41.693 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.693 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.693 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "7", 2024-04-23T10:42:41.693 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.693 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.693 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.693 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-23T10:42:41.693 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.693 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.693 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.693 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.693 INFO:teuthology.orchestra.run.smithi097.stdout: "config_check_secs": { 2024-04-23T10:42:41.693 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "config_check_secs", 2024-04-23T10:42:41.693 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:41.693 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.693 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.693 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-04-23T10:42:41.694 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "10", 2024-04-23T10:42:41.694 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.694 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.694 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-23T10:42:41.694 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.694 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.694 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.694 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.694 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:41.694 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:41.694 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.694 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.694 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.694 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.694 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.694 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.695 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.695 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.695 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.695 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.695 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.695 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.695 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.695 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.695 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.695 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.695 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.695 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.695 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.695 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:41.695 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:41.695 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.696 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.696 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.696 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.696 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.696 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.696 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.696 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.696 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.696 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.696 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.696 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.696 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:41.696 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:41.696 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.696 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.696 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.697 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:41.697 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.697 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.697 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.697 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.697 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.697 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.697 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.697 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.697 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.697 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.697 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.697 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.697 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.697 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.697 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.698 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:41.698 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:41.698 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.698 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.698 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.698 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.698 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.698 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.698 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.698 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.698 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.698 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.698 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.698 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.698 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.698 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.699 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:41.699 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "localpool", 2024-04-23T10:42:41.699 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:41.699 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:41.699 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:41.699 INFO:teuthology.orchestra.run.smithi097.stdout: "failure_domain": { 2024-04-23T10:42:41.699 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "failure_domain", 2024-04-23T10:42:41.699 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.699 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.699 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.699 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "host", 2024-04-23T10:42:41.699 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.699 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.699 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.699 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "failure domain for any created local pool", 2024-04-23T10:42:41.699 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-23T10:42:41.700 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.700 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.700 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.700 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:41.700 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:41.700 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.700 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.700 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.700 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.700 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.700 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.700 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.700 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.700 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.700 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.700 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.700 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.701 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.701 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.701 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.701 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.701 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.701 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.701 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.701 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:41.701 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:41.701 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.701 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.701 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.701 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.701 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.701 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.702 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.702 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.702 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.702 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.702 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.702 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.702 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:41.702 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:41.702 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.702 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.702 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.702 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:41.702 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.702 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.702 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.702 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.702 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.703 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.703 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.703 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.703 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.703 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.703 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.703 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.703 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.703 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.703 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.703 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:41.703 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:41.703 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.703 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.703 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.704 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.704 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.704 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.704 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.704 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.704 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.704 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.704 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.704 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.704 INFO:teuthology.orchestra.run.smithi097.stdout: "min_size": { 2024-04-23T10:42:41.704 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "min_size", 2024-04-23T10:42:41.704 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:41.704 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.704 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.704 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.704 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.704 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.705 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.705 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "default min_size for any created local pool", 2024-04-23T10:42:41.705 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-23T10:42:41.705 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.705 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.705 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.705 INFO:teuthology.orchestra.run.smithi097.stdout: "num_rep": { 2024-04-23T10:42:41.705 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "num_rep", 2024-04-23T10:42:41.705 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:41.705 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.705 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.705 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "3", 2024-04-23T10:42:41.705 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.705 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.705 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.706 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "default replica count for any created local pool", 2024-04-23T10:42:41.706 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.706 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.706 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.706 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.706 INFO:teuthology.orchestra.run.smithi097.stdout: "pg_num": { 2024-04-23T10:42:41.706 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "pg_num", 2024-04-23T10:42:41.706 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:41.706 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.706 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.706 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "128", 2024-04-23T10:42:41.706 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.706 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.706 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.706 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "default pg_num for any created local pool", 2024-04-23T10:42:41.706 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.707 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.707 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.707 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.707 INFO:teuthology.orchestra.run.smithi097.stdout: "prefix": { 2024-04-23T10:42:41.707 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "prefix", 2024-04-23T10:42:41.707 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.707 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.707 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.707 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.707 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.707 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.707 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.707 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "name prefix for any created local pool", 2024-04-23T10:42:41.707 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.707 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.707 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.708 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.708 INFO:teuthology.orchestra.run.smithi097.stdout: "subtree": { 2024-04-23T10:42:41.708 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "subtree", 2024-04-23T10:42:41.708 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.708 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.708 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.708 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "rack", 2024-04-23T10:42:41.708 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.708 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.708 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.708 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-23T10:42:41.708 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-23T10:42:41.708 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.708 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.708 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.708 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.709 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.709 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:41.709 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mds_autoscaler", 2024-04-23T10:42:41.709 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:41.709 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:41.709 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:41.709 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:41.709 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:41.709 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.709 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.709 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.709 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.709 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.709 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.709 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.709 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.709 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.710 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.710 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.710 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.710 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.710 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.710 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.710 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.710 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.710 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.710 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.710 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:41.710 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:41.710 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.710 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.710 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.710 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.711 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.711 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.711 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.711 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.711 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.711 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.711 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.711 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.711 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:41.711 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:41.711 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.711 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.711 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.711 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:41.711 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.711 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.712 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.712 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.712 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.712 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.712 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.712 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.712 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.712 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.712 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.712 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.712 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.712 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.712 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.712 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:41.712 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:41.712 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.712 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.713 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.713 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.713 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.713 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.713 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.713 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.713 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.713 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.713 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.713 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.713 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.713 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.713 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:41.713 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mirroring", 2024-04-23T10:42:41.713 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:41.713 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:41.714 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:41.714 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:41.714 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:41.714 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.714 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.714 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.714 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.714 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.714 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.714 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.714 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.714 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.714 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.714 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.714 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.714 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.715 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.715 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.715 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.715 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.715 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.715 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.715 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:41.715 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:41.715 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.715 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.715 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.715 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.715 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.715 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.715 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.715 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.716 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.716 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.716 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.716 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.716 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:41.716 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:41.716 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.716 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.716 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.716 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:41.716 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.716 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.716 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.716 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.716 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.716 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.717 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.717 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.717 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.717 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.717 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.717 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.717 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.717 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.717 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.717 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:41.717 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:41.717 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.717 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.717 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.717 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.717 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.718 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.718 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.718 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.718 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.718 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.718 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.718 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.718 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.718 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.718 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:41.718 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "nfs", 2024-04-23T10:42:41.718 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:41.718 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:41.718 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:41.718 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:41.718 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:41.719 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.719 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.719 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.719 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.719 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.719 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.719 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.719 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.719 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.719 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.719 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.719 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.719 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.719 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.719 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.719 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.719 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.720 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.720 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:41.720 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:41.720 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.720 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.720 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.720 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.720 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.720 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.720 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.720 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.720 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.720 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.720 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.721 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:41.721 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:41.721 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.721 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.721 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.721 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:41.721 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.721 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.721 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.721 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.721 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.721 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.721 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.721 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.721 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.721 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.721 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.722 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.722 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.722 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.722 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:41.722 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:41.722 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.722 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.722 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.722 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.722 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.722 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.722 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.722 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.722 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.723 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.723 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.723 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.723 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.723 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.723 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:41.723 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "orchestrator", 2024-04-23T10:42:41.723 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:41.723 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:41.723 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:41.723 INFO:teuthology.orchestra.run.smithi097.stdout: "fail_fs": { 2024-04-23T10:42:41.723 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "fail_fs", 2024-04-23T10:42:41.723 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.723 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.723 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.723 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.723 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.724 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.724 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.724 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-23T10:42:41.724 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.724 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.724 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.724 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.724 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:41.724 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:41.724 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.724 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.724 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.724 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.724 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.724 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.724 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.724 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.725 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.725 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.725 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.725 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.725 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.725 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.725 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.725 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.725 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.725 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.725 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:41.725 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:41.725 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.725 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.726 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.726 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.726 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.726 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.726 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.726 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.726 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.726 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.726 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.726 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.726 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:41.726 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:41.726 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.726 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.726 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.726 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:41.727 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.727 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.727 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.727 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.727 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.727 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.727 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.727 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.727 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.727 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.727 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.727 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.727 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.727 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.727 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.728 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:41.728 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:41.728 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.728 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.728 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.728 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.728 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.728 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.728 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.728 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.728 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.728 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.728 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.728 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.728 INFO:teuthology.orchestra.run.smithi097.stdout: "orchestrator": { 2024-04-23T10:42:41.728 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "orchestrator", 2024-04-23T10:42:41.728 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.729 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.729 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.729 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.729 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.729 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.729 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.729 INFO:teuthology.orchestra.run.smithi097.stdout: "cephadm", 2024-04-23T10:42:41.729 INFO:teuthology.orchestra.run.smithi097.stdout: "rook", 2024-04-23T10:42:41.729 INFO:teuthology.orchestra.run.smithi097.stdout: "test_orchestrator" 2024-04-23T10:42:41.729 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.729 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Orchestrator backend", 2024-04-23T10:42:41.729 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.729 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.729 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.729 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.730 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.730 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.730 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:41.730 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "osd_perf_query", 2024-04-23T10:42:41.730 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:41.730 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:41.730 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:41.730 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:41.730 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:41.730 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.730 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.730 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.730 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.730 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.730 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.730 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.731 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.731 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.731 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.731 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.731 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.731 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.731 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.731 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.731 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.731 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.731 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.731 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.731 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:41.731 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:41.731 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.732 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.732 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.732 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.732 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.732 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.732 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.732 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.732 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.732 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.732 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.732 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.732 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:41.732 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:41.732 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.732 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.733 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.733 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:41.733 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.733 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.733 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.733 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.733 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.733 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.733 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.733 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.733 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.733 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.733 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.733 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.733 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.733 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.733 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.734 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:41.734 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:41.734 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.734 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.734 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.734 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.734 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.734 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.734 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.734 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.734 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.734 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.734 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.734 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.734 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.735 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.735 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:41.735 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "osd_support", 2024-04-23T10:42:41.735 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:41.735 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:41.735 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:41.735 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:41.735 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:41.735 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.735 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.735 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.735 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.735 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.735 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.735 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.735 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.735 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.736 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.736 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.736 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.736 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.736 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.736 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.736 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.736 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.736 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.736 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.736 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:41.736 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:41.736 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.736 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.736 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.737 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.737 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.737 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.737 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.737 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.737 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.737 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.737 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.737 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.737 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:41.737 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:41.737 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.737 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.737 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.737 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:41.737 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.738 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.738 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.738 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.738 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.738 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.738 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.738 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.738 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.738 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.738 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.738 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.738 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.738 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.738 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.738 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:41.738 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:41.739 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.739 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.739 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.739 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.739 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.739 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.739 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.739 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.739 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.739 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.739 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.739 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.739 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.739 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.739 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:41.739 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "pg_autoscaler", 2024-04-23T10:42:41.740 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:41.740 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:41.740 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:41.740 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:41.740 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:41.740 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.740 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.740 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.740 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.740 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.740 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.740 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.740 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.740 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.740 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.740 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.741 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.741 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.741 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.741 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.741 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.741 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.741 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.741 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.741 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:41.741 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:41.741 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.741 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.741 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.741 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.741 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.741 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.741 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.742 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.742 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.742 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.742 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.742 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.742 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:41.742 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:41.742 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.742 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.742 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.742 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:41.742 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.742 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.742 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.742 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.743 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.743 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.743 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.743 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.743 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.743 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.743 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.743 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.743 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.743 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.743 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.743 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:41.743 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:41.743 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.743 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.743 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.744 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.744 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.744 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.744 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.744 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.744 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.744 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.744 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.744 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.744 INFO:teuthology.orchestra.run.smithi097.stdout: "sleep_interval": { 2024-04-23T10:42:41.744 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "sleep_interval", 2024-04-23T10:42:41.744 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-04-23T10:42:41.744 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.744 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.744 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "60", 2024-04-23T10:42:41.744 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.745 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.745 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.745 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.745 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.745 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.745 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.745 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.745 INFO:teuthology.orchestra.run.smithi097.stdout: "threshold": { 2024-04-23T10:42:41.745 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "threshold", 2024-04-23T10:42:41.745 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2024-04-23T10:42:41.745 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.745 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.745 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "3.0", 2024-04-23T10:42:41.745 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "1.0", 2024-04-23T10:42:41.745 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.745 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.745 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "scaling threshold", 2024-04-23T10:42:41.746 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-23T10:42:41.746 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.746 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.746 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.746 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.746 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.746 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:41.746 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "progress", 2024-04-23T10:42:41.746 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:41.746 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:41.746 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:41.746 INFO:teuthology.orchestra.run.smithi097.stdout: "allow_pg_recovery_event": { 2024-04-23T10:42:41.746 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "allow_pg_recovery_event", 2024-04-23T10:42:41.746 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.746 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.746 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.747 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.747 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.747 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.747 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.747 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-23T10:42:41.747 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.747 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.747 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.747 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.747 INFO:teuthology.orchestra.run.smithi097.stdout: "enabled": { 2024-04-23T10:42:41.747 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "enabled", 2024-04-23T10:42:41.747 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.747 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.747 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.747 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:41.747 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.748 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.748 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.748 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.748 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.748 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.748 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.748 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.748 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:41.748 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:41.748 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.748 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.748 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.748 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.748 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.748 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.748 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.748 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.749 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.749 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.749 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.749 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.749 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.749 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.749 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.749 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.749 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.749 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.749 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.749 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:41.749 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:41.749 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.749 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.749 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.750 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.750 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.750 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.750 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.750 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.750 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.750 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.750 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.750 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.750 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:41.750 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:41.750 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.750 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.750 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.750 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:41.750 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.751 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.751 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.751 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.751 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.751 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.751 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.751 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.751 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.751 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.751 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.751 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.751 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.751 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.751 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.751 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:41.752 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:41.752 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.752 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.752 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.752 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.752 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.752 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.752 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.752 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.752 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.752 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.752 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.752 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.752 INFO:teuthology.orchestra.run.smithi097.stdout: "max_completed_events": { 2024-04-23T10:42:41.752 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "max_completed_events", 2024-04-23T10:42:41.752 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:41.753 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.753 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.753 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "50", 2024-04-23T10:42:41.753 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.753 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.753 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.753 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "number of past completed events to remember", 2024-04-23T10:42:41.753 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.753 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.753 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.753 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.753 INFO:teuthology.orchestra.run.smithi097.stdout: "sleep_interval": { 2024-04-23T10:42:41.753 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "sleep_interval", 2024-04-23T10:42:41.753 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-04-23T10:42:41.753 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.753 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.754 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "5", 2024-04-23T10:42:41.754 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.754 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.754 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.754 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how long the module is going to sleep", 2024-04-23T10:42:41.754 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.754 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.754 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.754 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.754 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.754 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.754 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:41.754 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "prometheus", 2024-04-23T10:42:41.754 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:41.754 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:41.754 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:41.755 INFO:teuthology.orchestra.run.smithi097.stdout: "cache": { 2024-04-23T10:42:41.755 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "cache", 2024-04-23T10:42:41.755 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.755 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.755 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.755 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:41.755 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.755 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.755 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.755 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.755 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.755 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.755 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.755 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.755 INFO:teuthology.orchestra.run.smithi097.stdout: "exclude_perf_counters": { 2024-04-23T10:42:41.755 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "exclude_perf_counters", 2024-04-23T10:42:41.756 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.756 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.756 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.756 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:41.756 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.756 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.756 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.756 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-23T10:42:41.756 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-23T10:42:41.756 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.756 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.756 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.756 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:41.756 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:41.756 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.756 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.757 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.757 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.757 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.757 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.757 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.757 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.757 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.757 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.757 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.757 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.757 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.757 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.757 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.757 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.757 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.757 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.758 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.758 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:41.758 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:41.758 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.758 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.758 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.758 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.758 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.758 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.758 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.758 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.758 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.758 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.758 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.758 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.758 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:41.759 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:41.759 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.759 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.759 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.759 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:41.759 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.759 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.759 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.759 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.759 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.759 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.759 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.759 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.759 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.759 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.759 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.759 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.760 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.760 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.760 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.760 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:41.760 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:41.760 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.760 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.760 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.760 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.760 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.760 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.760 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.760 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.760 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.760 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.760 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.761 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.761 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_stats_pools": { 2024-04-23T10:42:41.761 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rbd_stats_pools", 2024-04-23T10:42:41.761 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.761 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.761 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.761 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.761 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.761 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.761 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.761 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.761 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.761 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.761 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.761 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.761 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-23T10:42:41.762 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-23T10:42:41.762 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:41.762 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.762 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.762 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "300", 2024-04-23T10:42:41.762 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.762 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.762 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.762 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.762 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.762 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.762 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.762 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.762 INFO:teuthology.orchestra.run.smithi097.stdout: "scrape_interval": { 2024-04-23T10:42:41.762 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "scrape_interval", 2024-04-23T10:42:41.762 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2024-04-23T10:42:41.762 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.763 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.763 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "15.0", 2024-04-23T10:42:41.763 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.763 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.763 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.763 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.763 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.763 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.763 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.763 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.763 INFO:teuthology.orchestra.run.smithi097.stdout: "server_addr": { 2024-04-23T10:42:41.763 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_addr", 2024-04-23T10:42:41.763 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.763 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.763 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.764 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "::", 2024-04-23T10:42:41.764 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.764 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.764 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.764 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-23T10:42:41.764 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.764 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.764 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.764 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.764 INFO:teuthology.orchestra.run.smithi097.stdout: "server_port": { 2024-04-23T10:42:41.764 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_port", 2024-04-23T10:42:41.764 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:41.764 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.764 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.764 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "9283", 2024-04-23T10:42:41.764 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.765 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.765 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.765 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-23T10:42:41.765 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.765 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.765 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.765 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.765 INFO:teuthology.orchestra.run.smithi097.stdout: "stale_cache_strategy": { 2024-04-23T10:42:41.765 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "stale_cache_strategy", 2024-04-23T10:42:41.765 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.765 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.765 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.765 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "log", 2024-04-23T10:42:41.765 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.765 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.766 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.766 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.766 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.766 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.766 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.766 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.766 INFO:teuthology.orchestra.run.smithi097.stdout: "standby_behaviour": { 2024-04-23T10:42:41.766 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "standby_behaviour", 2024-04-23T10:42:41.766 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.766 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.766 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.766 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "default", 2024-04-23T10:42:41.766 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.766 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.766 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.766 INFO:teuthology.orchestra.run.smithi097.stdout: "default", 2024-04-23T10:42:41.767 INFO:teuthology.orchestra.run.smithi097.stdout: "error" 2024-04-23T10:42:41.767 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.767 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.767 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.767 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.767 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.767 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.767 INFO:teuthology.orchestra.run.smithi097.stdout: "standby_error_status_code": { 2024-04-23T10:42:41.767 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "standby_error_status_code", 2024-04-23T10:42:41.767 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:41.767 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.767 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.767 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "500", 2024-04-23T10:42:41.767 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "400", 2024-04-23T10:42:41.767 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "599", 2024-04-23T10:42:41.767 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.768 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.768 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.768 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.768 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.768 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.768 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.768 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.768 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:41.768 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rbd_support", 2024-04-23T10:42:41.768 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:41.768 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:41.768 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:41.768 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:41.768 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:41.768 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.768 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.769 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.769 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.769 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.769 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.769 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.769 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.769 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.769 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.769 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.769 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.769 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.769 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.769 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.769 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.769 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.769 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.769 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.770 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:41.770 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:41.770 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.770 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.770 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.770 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.770 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.770 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.770 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.770 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.770 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.770 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.770 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.770 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.770 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:41.770 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:41.771 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.771 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.771 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.771 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:41.771 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.771 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.771 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.771 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.771 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.771 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.771 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.771 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.771 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.771 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.771 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.771 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.771 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.772 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.772 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.772 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:41.772 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:41.772 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.772 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.772 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.772 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.772 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.772 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.772 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.772 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.772 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.772 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.772 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.772 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.773 INFO:teuthology.orchestra.run.smithi097.stdout: "max_concurrent_snap_create": { 2024-04-23T10:42:41.773 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "max_concurrent_snap_create", 2024-04-23T10:42:41.773 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:41.773 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.773 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.773 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-04-23T10:42:41.773 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.773 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.773 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.773 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.773 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.773 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.773 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.773 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.773 INFO:teuthology.orchestra.run.smithi097.stdout: "mirror_snapshot_schedule": { 2024-04-23T10:42:41.773 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mirror_snapshot_schedule", 2024-04-23T10:42:41.774 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.774 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.774 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.774 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.774 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.774 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.774 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.774 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.774 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.774 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.774 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.774 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.774 INFO:teuthology.orchestra.run.smithi097.stdout: "trash_purge_schedule": { 2024-04-23T10:42:41.775 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "trash_purge_schedule", 2024-04-23T10:42:41.775 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.775 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.775 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.775 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.775 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.775 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.775 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.775 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.775 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.775 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.775 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.775 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.775 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.775 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.776 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:41.776 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "restful", 2024-04-23T10:42:41.776 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:41.776 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:41.776 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:41.776 INFO:teuthology.orchestra.run.smithi097.stdout: "enable_auth": { 2024-04-23T10:42:41.776 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "enable_auth", 2024-04-23T10:42:41.776 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.776 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.776 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.776 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:41.776 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.776 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.776 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.776 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.776 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.777 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.777 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.777 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.777 INFO:teuthology.orchestra.run.smithi097.stdout: "key_file": { 2024-04-23T10:42:41.777 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "key_file", 2024-04-23T10:42:41.777 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.777 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.777 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.777 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.777 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.777 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.777 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.777 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.777 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.777 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.777 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.778 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.778 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:41.778 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:41.778 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.778 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.778 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.778 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.778 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.778 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.778 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.778 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.778 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.778 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.778 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.779 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.779 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.779 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.779 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.779 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.779 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.779 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.779 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.779 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:41.779 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:41.779 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.779 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.779 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.779 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.779 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.779 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.779 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.780 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.780 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.780 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.780 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.780 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.780 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:41.780 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:41.780 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.780 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.780 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.780 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:41.780 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.780 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.780 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.780 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.780 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.781 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.781 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.781 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.781 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.781 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.781 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.781 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.781 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.781 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.781 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.781 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:41.781 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:41.781 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.781 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.781 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.781 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.782 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.782 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.782 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.782 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.782 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.782 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.782 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.782 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.782 INFO:teuthology.orchestra.run.smithi097.stdout: "server_addr": { 2024-04-23T10:42:41.782 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_addr", 2024-04-23T10:42:41.782 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.782 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.782 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.782 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.782 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.782 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.783 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.783 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.783 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.783 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.783 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.783 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.783 INFO:teuthology.orchestra.run.smithi097.stdout: "server_port": { 2024-04-23T10:42:41.783 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_port", 2024-04-23T10:42:41.783 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.783 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.783 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.783 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.783 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.783 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.783 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.783 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.783 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.784 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.784 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.784 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.784 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.784 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.784 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:41.784 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rgw", 2024-04-23T10:42:41.784 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:41.784 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:41.784 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:41.784 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:41.784 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:41.784 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.784 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.784 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.784 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.785 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.785 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.785 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.785 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.785 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.785 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.785 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.785 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.785 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.785 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.785 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.785 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.785 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.785 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.785 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.785 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:41.786 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:41.786 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.786 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.786 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.786 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.786 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.786 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.786 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.786 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.786 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.786 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.786 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.786 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.786 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:41.786 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:41.786 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.786 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.787 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.787 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:41.787 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.787 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.787 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.787 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.787 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.787 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.787 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.787 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.787 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.787 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.787 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.787 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.787 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.788 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.788 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.788 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:41.788 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:41.788 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.788 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.788 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.788 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.788 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.788 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.788 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.788 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.788 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.788 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.788 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.788 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.788 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.789 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.789 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:41.789 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rook", 2024-04-23T10:42:41.789 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:41.789 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:41.789 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:41.789 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:41.789 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:41.789 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.789 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.789 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.789 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.789 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.789 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.789 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.789 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.790 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.790 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.790 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.790 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.790 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.790 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.790 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.790 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.790 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.790 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.790 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.790 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:41.790 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:41.790 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.790 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.790 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.791 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.791 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.791 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.791 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.791 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.791 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.791 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.791 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.791 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.791 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:41.791 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:41.791 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.791 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.791 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.791 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:41.791 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.792 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.792 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.792 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.792 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.792 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.792 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.792 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.792 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.792 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.792 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.792 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.792 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.792 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.792 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.792 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:41.792 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:41.792 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.793 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.793 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.793 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.793 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.793 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.793 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.793 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.793 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.793 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.793 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.793 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.793 INFO:teuthology.orchestra.run.smithi097.stdout: "storage_class": { 2024-04-23T10:42:41.793 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "storage_class", 2024-04-23T10:42:41.793 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.793 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.793 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.794 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "local", 2024-04-23T10:42:41.794 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.794 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.794 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.794 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-23T10:42:41.794 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.794 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.794 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.794 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.794 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.794 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.794 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:41.794 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "selftest", 2024-04-23T10:42:41.794 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:41.794 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:41.794 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:41.794 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:41.795 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:41.795 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.795 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.795 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.795 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.795 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.795 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.795 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.795 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.795 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.795 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.795 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.795 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.795 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.795 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.796 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.796 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.796 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.796 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.796 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.796 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:41.796 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:41.796 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.796 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.796 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.796 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.796 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.796 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.796 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.796 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.796 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.796 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.797 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.797 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.797 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:41.797 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:41.797 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.797 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.797 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.797 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:41.797 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.797 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.797 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.797 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.797 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.797 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.797 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.797 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.798 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.798 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.798 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.798 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.798 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.798 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.798 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.798 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:41.798 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:41.798 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.798 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.798 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.798 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.798 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.798 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.798 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.799 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.799 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.799 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.799 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.799 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.799 INFO:teuthology.orchestra.run.smithi097.stdout: "roption1": { 2024-04-23T10:42:41.799 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "roption1", 2024-04-23T10:42:41.799 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.799 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.799 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.799 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.799 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.799 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.799 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.799 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.799 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.800 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.800 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.800 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.800 INFO:teuthology.orchestra.run.smithi097.stdout: "roption2": { 2024-04-23T10:42:41.800 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "roption2", 2024-04-23T10:42:41.800 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.800 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.800 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.800 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "xyz", 2024-04-23T10:42:41.800 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.800 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.800 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.800 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.800 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.800 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.800 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.801 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.801 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption1": { 2024-04-23T10:42:41.801 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption1", 2024-04-23T10:42:41.801 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.801 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.801 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.801 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.801 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.801 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.801 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.801 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.801 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.801 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.801 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.801 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.801 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption2": { 2024-04-23T10:42:41.802 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption2", 2024-04-23T10:42:41.802 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:41.802 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.802 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.802 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.802 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.802 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.802 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.802 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.802 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.802 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.802 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.802 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.802 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption3": { 2024-04-23T10:42:41.802 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption3", 2024-04-23T10:42:41.802 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2024-04-23T10:42:41.803 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.803 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.803 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.803 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.803 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.803 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.803 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.803 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.803 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.803 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.803 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.803 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption4": { 2024-04-23T10:42:41.803 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption4", 2024-04-23T10:42:41.803 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.803 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.803 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.804 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.804 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.804 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.804 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.804 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.804 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.804 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.804 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.804 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.804 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption5": { 2024-04-23T10:42:41.804 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption5", 2024-04-23T10:42:41.804 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.804 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.804 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.804 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.804 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.805 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.805 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.805 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.805 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.805 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.805 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.805 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.805 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption6": { 2024-04-23T10:42:41.805 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption6", 2024-04-23T10:42:41.805 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.805 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.805 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.805 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:41.805 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.805 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.805 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.805 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.806 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.806 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.806 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.806 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.806 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption7": { 2024-04-23T10:42:41.806 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption7", 2024-04-23T10:42:41.806 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:41.806 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.806 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.806 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.806 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "1", 2024-04-23T10:42:41.806 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "42", 2024-04-23T10:42:41.806 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.806 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.806 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.806 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.807 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.807 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.807 INFO:teuthology.orchestra.run.smithi097.stdout: "testkey": { 2024-04-23T10:42:41.807 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "testkey", 2024-04-23T10:42:41.807 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.807 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.807 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.807 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.807 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.807 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.807 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.807 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.807 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.807 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.807 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.807 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.808 INFO:teuthology.orchestra.run.smithi097.stdout: "testlkey": { 2024-04-23T10:42:41.808 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "testlkey", 2024-04-23T10:42:41.808 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.808 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.808 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.808 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.808 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.808 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.808 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.808 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.808 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.808 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.808 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.808 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.808 INFO:teuthology.orchestra.run.smithi097.stdout: "testnewline": { 2024-04-23T10:42:41.808 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "testnewline", 2024-04-23T10:42:41.808 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.809 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.809 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.809 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.809 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.809 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.809 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.809 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.809 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.809 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.809 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.809 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.809 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.809 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.809 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:41.809 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "snap_schedule", 2024-04-23T10:42:41.809 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:41.810 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:41.810 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:41.810 INFO:teuthology.orchestra.run.smithi097.stdout: "allow_m_granularity": { 2024-04-23T10:42:41.810 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "allow_m_granularity", 2024-04-23T10:42:41.810 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.810 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.810 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.810 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.810 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.810 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.810 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.810 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "allow minute scheduled snapshots", 2024-04-23T10:42:41.810 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.810 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.810 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.810 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.811 INFO:teuthology.orchestra.run.smithi097.stdout: "dump_on_update": { 2024-04-23T10:42:41.811 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "dump_on_update", 2024-04-23T10:42:41.811 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.811 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.811 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.811 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.811 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.811 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.811 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.811 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "dump database to debug log on update", 2024-04-23T10:42:41.811 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.811 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.811 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.811 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.811 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:41.811 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:41.812 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.812 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.812 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.812 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.812 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.812 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.812 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.812 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.812 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.812 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.812 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.812 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.812 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.812 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.812 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.812 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.813 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.813 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.813 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.813 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:41.813 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:41.813 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.813 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.813 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.813 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.813 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.813 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.813 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.813 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.813 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.813 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.813 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.814 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.814 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:41.814 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:41.814 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.814 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.814 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.814 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:41.814 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.814 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.814 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.814 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.814 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.814 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.814 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.814 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.814 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.815 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.815 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.815 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.815 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.815 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.815 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.815 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:41.815 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:41.815 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.815 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.815 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.815 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.815 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.815 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.815 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.815 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.816 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.816 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.816 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.816 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.816 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.816 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.816 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:41.816 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "stats", 2024-04-23T10:42:41.816 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:41.816 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:41.816 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:41.816 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:41.816 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:41.816 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.816 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.816 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.816 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.817 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.817 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.817 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.817 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.817 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.817 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.817 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.817 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.817 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.817 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.817 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.817 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.817 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.817 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.817 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.817 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:41.818 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:41.818 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.818 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.818 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.818 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.818 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.818 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.818 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.818 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.818 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.818 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.818 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.818 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.818 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:41.818 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:41.818 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.819 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.819 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.819 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:41.819 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.819 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.819 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.819 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.819 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.819 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.819 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.819 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.819 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.819 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.819 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.819 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.819 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.820 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.820 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.820 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:41.820 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:41.820 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.820 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.820 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.820 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.820 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.820 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.820 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.820 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.820 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.820 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.820 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.820 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.821 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.821 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.821 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:41.821 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "status", 2024-04-23T10:42:41.821 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:41.821 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:41.821 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:41.821 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:41.821 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:41.821 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.821 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.821 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.821 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.821 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.821 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.821 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.821 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.822 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.822 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.822 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.822 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.822 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.822 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.822 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.822 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.822 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.822 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.822 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.822 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:41.822 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:41.822 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.822 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.823 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.823 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.823 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.823 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.823 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.823 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.823 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.823 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.823 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.823 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.823 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:41.823 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:41.823 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.823 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.823 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.823 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:41.824 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.824 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.824 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.824 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.824 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.824 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.824 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.824 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.824 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.824 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.824 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.824 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.824 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.824 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.824 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.824 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:41.825 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:41.825 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.825 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.825 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.825 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.825 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.825 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.825 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.825 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.825 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.825 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.825 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.825 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.825 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.825 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.825 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:41.826 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "telegraf", 2024-04-23T10:42:41.826 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:41.826 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:41.826 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:41.826 INFO:teuthology.orchestra.run.smithi097.stdout: "address": { 2024-04-23T10:42:41.826 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "address", 2024-04-23T10:42:41.826 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.826 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.826 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.826 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-23T10:42:41.826 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.826 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.826 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.826 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.826 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.826 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.827 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.827 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.827 INFO:teuthology.orchestra.run.smithi097.stdout: "interval": { 2024-04-23T10:42:41.827 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "interval", 2024-04-23T10:42:41.827 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-04-23T10:42:41.827 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.827 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.827 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "15", 2024-04-23T10:42:41.827 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.827 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.827 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.827 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.827 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.827 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.827 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.827 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.827 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:41.828 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:41.828 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.828 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.828 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.828 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.828 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.828 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.828 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.828 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.828 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.828 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.828 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.828 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.828 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.828 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.828 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.829 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.829 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.829 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.829 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.829 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:41.829 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:41.829 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.829 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.829 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.829 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.829 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.829 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.829 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.829 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.829 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.829 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.829 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.830 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.830 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:41.830 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:41.830 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.830 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.830 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.830 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:41.830 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.830 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.830 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.830 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.830 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.830 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.830 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.830 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.830 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.831 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.831 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.831 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.831 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.831 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.831 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.831 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:41.831 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:41.831 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.831 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.831 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.831 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.831 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.831 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.832 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.832 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.832 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.832 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.832 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.832 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.832 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.832 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.832 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:41.832 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "telemetry", 2024-04-23T10:42:41.832 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:41.832 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:41.832 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:41.832 INFO:teuthology.orchestra.run.smithi097.stdout: "channel_basic": { 2024-04-23T10:42:41.832 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "channel_basic", 2024-04-23T10:42:41.832 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.832 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.833 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.833 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:41.833 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.833 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.833 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.833 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-23T10:42:41.833 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.833 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.833 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.833 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.833 INFO:teuthology.orchestra.run.smithi097.stdout: "channel_crash": { 2024-04-23T10:42:41.833 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "channel_crash", 2024-04-23T10:42:41.833 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.833 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.833 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.833 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:41.834 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.834 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.834 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.834 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-23T10:42:41.834 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.834 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.834 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.834 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.834 INFO:teuthology.orchestra.run.smithi097.stdout: "channel_device": { 2024-04-23T10:42:41.834 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "channel_device", 2024-04-23T10:42:41.834 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.834 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.834 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.834 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:41.834 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.834 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.835 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.835 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-23T10:42:41.835 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.835 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.835 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.835 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.835 INFO:teuthology.orchestra.run.smithi097.stdout: "channel_ident": { 2024-04-23T10:42:41.835 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "channel_ident", 2024-04-23T10:42:41.835 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.835 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.835 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.835 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.835 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.835 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.835 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.836 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-23T10:42:41.836 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.836 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.836 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.836 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.836 INFO:teuthology.orchestra.run.smithi097.stdout: "channel_perf": { 2024-04-23T10:42:41.836 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "channel_perf", 2024-04-23T10:42:41.836 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.836 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.836 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.836 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.836 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.836 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.836 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.836 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-23T10:42:41.836 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.836 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.837 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.837 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.837 INFO:teuthology.orchestra.run.smithi097.stdout: "contact": { 2024-04-23T10:42:41.837 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "contact", 2024-04-23T10:42:41.837 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.837 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.837 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.837 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.837 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.837 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.837 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.837 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.837 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.837 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.837 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.837 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.838 INFO:teuthology.orchestra.run.smithi097.stdout: "description": { 2024-04-23T10:42:41.838 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "description", 2024-04-23T10:42:41.838 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.838 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.838 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.838 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.838 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.838 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.838 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.838 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.838 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.838 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.838 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.838 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.838 INFO:teuthology.orchestra.run.smithi097.stdout: "device_url": { 2024-04-23T10:42:41.838 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "device_url", 2024-04-23T10:42:41.839 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.839 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.839 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.839 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-23T10:42:41.839 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.839 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.839 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.839 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.839 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.839 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.839 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.839 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.839 INFO:teuthology.orchestra.run.smithi097.stdout: "enabled": { 2024-04-23T10:42:41.839 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "enabled", 2024-04-23T10:42:41.839 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.839 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.840 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.840 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.840 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.840 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.840 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.840 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.840 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.840 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.840 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.840 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.840 INFO:teuthology.orchestra.run.smithi097.stdout: "interval": { 2024-04-23T10:42:41.840 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "interval", 2024-04-23T10:42:41.840 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:41.840 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.840 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.840 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "24", 2024-04-23T10:42:41.841 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "8", 2024-04-23T10:42:41.841 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.841 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.841 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.841 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.841 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.841 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.841 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.841 INFO:teuthology.orchestra.run.smithi097.stdout: "last_opt_revision": { 2024-04-23T10:42:41.841 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "last_opt_revision", 2024-04-23T10:42:41.841 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:41.841 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.841 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.841 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "1", 2024-04-23T10:42:41.841 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.841 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.841 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.842 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.842 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.842 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.842 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.842 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.842 INFO:teuthology.orchestra.run.smithi097.stdout: "leaderboard": { 2024-04-23T10:42:41.842 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "leaderboard", 2024-04-23T10:42:41.842 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.842 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.842 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.842 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.842 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.842 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.842 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.842 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.843 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.843 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.843 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.843 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.843 INFO:teuthology.orchestra.run.smithi097.stdout: "leaderboard_description": { 2024-04-23T10:42:41.843 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "leaderboard_description", 2024-04-23T10:42:41.843 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.843 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.843 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.843 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.843 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.843 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.843 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.843 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.843 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.843 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.844 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.844 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.844 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:41.844 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:41.844 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.844 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.844 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.844 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.844 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.844 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.844 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.844 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.844 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.844 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.844 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.844 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.844 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.845 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.845 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.845 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.845 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.845 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.845 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.845 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:41.845 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:41.845 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.845 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.845 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.845 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.845 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.845 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.845 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.845 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.846 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.846 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.846 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.846 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.846 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:41.846 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:41.846 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.846 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.846 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.846 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:41.846 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.846 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.846 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.846 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.846 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.846 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.847 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.847 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.847 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.847 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.847 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.847 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.847 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.847 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.847 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.847 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:41.847 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:41.847 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.847 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.847 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.847 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.847 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.848 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.848 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.848 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.848 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.848 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.848 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.848 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.848 INFO:teuthology.orchestra.run.smithi097.stdout: "organization": { 2024-04-23T10:42:41.848 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "organization", 2024-04-23T10:42:41.848 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.848 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.848 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.848 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.848 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.848 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.849 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.849 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.849 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.849 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.849 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.849 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.849 INFO:teuthology.orchestra.run.smithi097.stdout: "proxy": { 2024-04-23T10:42:41.849 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "proxy", 2024-04-23T10:42:41.849 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.849 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.849 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.849 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.849 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.849 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.849 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.849 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.850 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.850 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.850 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.850 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.850 INFO:teuthology.orchestra.run.smithi097.stdout: "url": { 2024-04-23T10:42:41.850 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "url", 2024-04-23T10:42:41.850 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.850 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.850 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.850 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-23T10:42:41.850 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.850 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.850 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.850 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.850 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.850 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.851 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.851 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.851 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.851 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.851 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:41.851 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "test_orchestrator", 2024-04-23T10:42:41.851 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:41.851 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:41.851 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:41.851 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:41.851 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:41.851 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.851 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.851 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.851 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.851 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.851 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.852 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.852 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.852 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.852 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.852 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.852 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.852 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.852 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.852 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.852 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.852 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.852 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.852 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.852 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:41.852 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:41.852 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.853 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.853 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.853 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.853 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.853 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.853 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.853 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.853 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.853 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.853 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.853 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.853 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:41.853 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:41.853 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.853 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.853 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.854 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:41.854 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.854 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.854 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.854 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.854 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.854 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.854 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.854 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.854 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.854 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.854 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.854 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.854 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.854 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.854 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.855 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:41.855 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:41.855 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.855 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.855 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.855 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.855 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.855 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.855 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.855 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.855 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.855 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.855 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.855 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.855 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.855 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.855 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:41.856 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "volumes", 2024-04-23T10:42:41.856 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:41.856 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:41.856 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:41.856 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:41.856 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:41.856 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.856 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.856 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.856 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.856 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.856 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.856 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.856 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.856 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.856 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.857 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.857 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.857 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.857 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.857 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.857 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.857 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.857 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.857 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.857 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:41.857 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:41.857 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.857 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.857 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.857 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.857 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.858 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.858 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.858 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.858 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.858 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.858 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.858 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.858 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:41.858 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:41.858 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.858 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.858 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.858 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:41.858 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.858 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.859 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.859 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.859 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.859 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.859 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.859 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.859 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.859 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.859 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.859 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.859 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.859 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.859 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.859 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:41.859 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:41.859 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.860 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.860 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.860 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.860 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.860 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.860 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.860 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.860 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.860 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.860 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.860 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.860 INFO:teuthology.orchestra.run.smithi097.stdout: "max_concurrent_clones": { 2024-04-23T10:42:41.860 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "max_concurrent_clones", 2024-04-23T10:42:41.860 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:41.860 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.860 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.861 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "4", 2024-04-23T10:42:41.861 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.861 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.861 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.861 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-23T10:42:41.861 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.861 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.861 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.861 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.861 INFO:teuthology.orchestra.run.smithi097.stdout: "periodic_async_work": { 2024-04-23T10:42:41.861 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "periodic_async_work", 2024-04-23T10:42:41.861 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.861 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.861 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.861 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.861 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.862 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.862 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.862 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Periodically check for async work", 2024-04-23T10:42:41.862 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.862 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.862 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.862 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.862 INFO:teuthology.orchestra.run.smithi097.stdout: "snapshot_clone_delay": { 2024-04-23T10:42:41.862 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "snapshot_clone_delay", 2024-04-23T10:42:41.862 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:41.862 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.862 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.862 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0", 2024-04-23T10:42:41.862 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.862 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.863 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.863 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-23T10:42:41.863 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.863 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.863 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.863 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.863 INFO:teuthology.orchestra.run.smithi097.stdout: "snapshot_clone_no_wait": { 2024-04-23T10:42:41.863 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "snapshot_clone_no_wait", 2024-04-23T10:42:41.863 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.863 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.863 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.863 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:41.863 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.863 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.863 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.864 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-23T10:42:41.864 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.864 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.864 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.864 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.864 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.864 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.864 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:41.864 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "zabbix", 2024-04-23T10:42:41.864 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:41.864 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:41.864 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:41.864 INFO:teuthology.orchestra.run.smithi097.stdout: "discovery_interval": { 2024-04-23T10:42:41.864 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "discovery_interval", 2024-04-23T10:42:41.864 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "uint", 2024-04-23T10:42:41.864 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.865 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.865 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "100", 2024-04-23T10:42:41.865 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.865 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.865 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.865 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.865 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.865 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.865 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.865 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.865 INFO:teuthology.orchestra.run.smithi097.stdout: "identifier": { 2024-04-23T10:42:41.865 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "identifier", 2024-04-23T10:42:41.865 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.865 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.865 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.866 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.866 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.866 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.866 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.866 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.866 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.866 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.866 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.866 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.866 INFO:teuthology.orchestra.run.smithi097.stdout: "interval": { 2024-04-23T10:42:41.866 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "interval", 2024-04-23T10:42:41.866 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-04-23T10:42:41.866 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.866 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.866 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "60", 2024-04-23T10:42:41.867 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.867 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.867 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.867 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.867 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.867 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.867 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.867 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.867 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:41.867 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:41.867 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.867 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.867 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.867 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.867 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.867 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.868 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.868 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.868 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.868 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.868 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.868 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.868 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.868 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.868 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.868 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.868 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.868 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.868 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.868 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:41.868 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:41.868 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.869 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.869 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.869 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.869 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.869 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.869 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.869 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.869 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.869 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.869 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.869 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.869 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:41.869 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:41.869 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.869 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.870 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.870 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:41.870 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.870 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.870 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.870 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.870 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.870 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.870 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.870 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.870 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.870 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.870 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.870 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.871 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.871 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.871 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.871 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:41.871 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:41.871 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.871 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.871 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.871 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.871 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.871 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.871 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.871 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.871 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.872 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.872 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.872 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.872 INFO:teuthology.orchestra.run.smithi097.stdout: "zabbix_host": { 2024-04-23T10:42:41.872 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "zabbix_host", 2024-04-23T10:42:41.872 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.872 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.872 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.872 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.872 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.872 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.872 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.872 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.872 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.872 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.873 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.873 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.873 INFO:teuthology.orchestra.run.smithi097.stdout: "zabbix_port": { 2024-04-23T10:42:41.873 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "zabbix_port", 2024-04-23T10:42:41.873 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:41.873 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.873 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.873 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10051", 2024-04-23T10:42:41.873 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.873 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.873 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.873 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.873 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.873 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.873 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.874 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.874 INFO:teuthology.orchestra.run.smithi097.stdout: "zabbix_sender": { 2024-04-23T10:42:41.874 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "zabbix_sender", 2024-04-23T10:42:41.874 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.874 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.874 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.874 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-23T10:42:41.874 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.874 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.874 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.874 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.874 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.874 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.874 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.875 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.875 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.875 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.875 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-04-23T10:42:41.875 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.875 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:41.875 INFO:teuthology.orchestra.run.smithi097.stdout: "gid": 24941, 2024-04-23T10:42:41.875 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "y", 2024-04-23T10:42:41.875 INFO:teuthology.orchestra.run.smithi097.stdout: "mgr_features": 4540701547738038271, 2024-04-23T10:42:41.875 INFO:teuthology.orchestra.run.smithi097.stdout: "available_modules": [ 2024-04-23T10:42:41.875 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:41.875 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "alerts", 2024-04-23T10:42:41.875 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:41.875 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:41.875 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:41.876 INFO:teuthology.orchestra.run.smithi097.stdout: "interval": { 2024-04-23T10:42:41.876 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "interval", 2024-04-23T10:42:41.876 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-04-23T10:42:41.876 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.876 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.876 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "60", 2024-04-23T10:42:41.876 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.876 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.876 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.876 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "How frequently to reexamine health status", 2024-04-23T10:42:41.876 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.876 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.876 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.876 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.876 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:41.877 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:41.877 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.877 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.877 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.877 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.877 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.877 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.877 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.877 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.877 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.877 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.877 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.877 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.877 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.877 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.878 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.878 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.878 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.878 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.878 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.878 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:41.878 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:41.878 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.878 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.878 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.878 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.878 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.878 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.878 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.878 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.878 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.878 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.879 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.879 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.879 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:41.879 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:41.879 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.879 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.879 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.879 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:41.879 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.879 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.879 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.879 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.879 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.879 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.879 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.879 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.880 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.880 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.880 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.880 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.880 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.880 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.880 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.880 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:41.880 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:41.880 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.880 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.880 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.880 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.880 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.880 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.880 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.880 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.881 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.881 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.881 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.881 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.881 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_destination": { 2024-04-23T10:42:41.881 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_destination", 2024-04-23T10:42:41.881 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.881 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.881 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.881 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.881 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.881 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.881 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.881 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Email address to send alerts to", 2024-04-23T10:42:41.881 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.881 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.882 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.882 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.882 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_from_name": { 2024-04-23T10:42:41.882 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_from_name", 2024-04-23T10:42:41.882 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.882 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.882 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.882 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "Ceph", 2024-04-23T10:42:41.882 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.882 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.882 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.882 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Email From: name", 2024-04-23T10:42:41.882 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.882 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.882 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.883 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.883 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_host": { 2024-04-23T10:42:41.883 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_host", 2024-04-23T10:42:41.883 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.883 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.883 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.883 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.883 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.883 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.883 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.883 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "SMTP server", 2024-04-23T10:42:41.883 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.883 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.883 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.883 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.883 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_password": { 2024-04-23T10:42:41.883 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_password", 2024-04-23T10:42:41.884 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.884 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.884 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.884 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.884 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.884 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.884 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.884 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Password to authenticate with", 2024-04-23T10:42:41.884 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.884 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.884 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.884 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.884 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_port": { 2024-04-23T10:42:41.884 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_port", 2024-04-23T10:42:41.884 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:41.885 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.885 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.885 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "465", 2024-04-23T10:42:41.885 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.885 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.885 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.885 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "SMTP port", 2024-04-23T10:42:41.885 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.885 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.885 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.885 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.885 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_sender": { 2024-04-23T10:42:41.885 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_sender", 2024-04-23T10:42:41.885 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.885 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.885 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.885 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.886 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.886 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.886 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.886 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "SMTP envelope sender", 2024-04-23T10:42:41.886 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.886 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.886 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.886 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.886 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_ssl": { 2024-04-23T10:42:41.886 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_ssl", 2024-04-23T10:42:41.886 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.886 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.886 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.886 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:41.886 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.887 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.887 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.887 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-23T10:42:41.887 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.887 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.887 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.887 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.887 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_user": { 2024-04-23T10:42:41.887 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_user", 2024-04-23T10:42:41.887 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.887 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.887 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.887 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.887 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.887 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.887 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.887 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "User to authenticate as", 2024-04-23T10:42:41.888 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.888 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.888 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.888 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.888 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.888 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.888 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:41.888 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "balancer", 2024-04-23T10:42:41.888 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:41.888 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:41.888 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:41.888 INFO:teuthology.orchestra.run.smithi097.stdout: "active": { 2024-04-23T10:42:41.888 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "active", 2024-04-23T10:42:41.888 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.888 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.888 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.889 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:41.889 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.889 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.889 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.889 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "automatically balance PGs across cluster", 2024-04-23T10:42:41.889 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.889 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.889 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.889 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.889 INFO:teuthology.orchestra.run.smithi097.stdout: "begin_time": { 2024-04-23T10:42:41.889 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "begin_time", 2024-04-23T10:42:41.889 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.889 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.889 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.889 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0000", 2024-04-23T10:42:41.889 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.890 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.890 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.890 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "beginning time of day to automatically balance", 2024-04-23T10:42:41.890 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-23T10:42:41.890 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.890 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.890 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.890 INFO:teuthology.orchestra.run.smithi097.stdout: "begin_weekday": { 2024-04-23T10:42:41.890 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "begin_weekday", 2024-04-23T10:42:41.890 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "uint", 2024-04-23T10:42:41.890 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.890 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.890 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0", 2024-04-23T10:42:41.890 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "0", 2024-04-23T10:42:41.890 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "6", 2024-04-23T10:42:41.891 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.891 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-23T10:42:41.891 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-23T10:42:41.891 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.891 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.891 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.891 INFO:teuthology.orchestra.run.smithi097.stdout: "crush_compat_max_iterations": { 2024-04-23T10:42:41.891 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "crush_compat_max_iterations", 2024-04-23T10:42:41.891 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "uint", 2024-04-23T10:42:41.891 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.891 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.891 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "25", 2024-04-23T10:42:41.891 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "1", 2024-04-23T10:42:41.891 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "250", 2024-04-23T10:42:41.891 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.891 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-23T10:42:41.891 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.892 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.892 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.892 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.892 INFO:teuthology.orchestra.run.smithi097.stdout: "crush_compat_metrics": { 2024-04-23T10:42:41.892 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "crush_compat_metrics", 2024-04-23T10:42:41.892 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.892 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.892 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.892 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "pgs,objects,bytes", 2024-04-23T10:42:41.892 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.892 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.892 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.892 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-23T10:42:41.892 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-23T10:42:41.892 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.892 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.893 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.893 INFO:teuthology.orchestra.run.smithi097.stdout: "crush_compat_step": { 2024-04-23T10:42:41.893 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "crush_compat_step", 2024-04-23T10:42:41.893 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2024-04-23T10:42:41.893 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.893 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.893 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0.5", 2024-04-23T10:42:41.893 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "0.001", 2024-04-23T10:42:41.893 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "0.999", 2024-04-23T10:42:41.893 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.893 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "aggressiveness of optimization", 2024-04-23T10:42:41.893 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-23T10:42:41.893 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.893 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.893 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.893 INFO:teuthology.orchestra.run.smithi097.stdout: "end_time": { 2024-04-23T10:42:41.893 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "end_time", 2024-04-23T10:42:41.894 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.894 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.894 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.894 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "2359", 2024-04-23T10:42:41.894 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.894 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.894 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.894 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "ending time of day to automatically balance", 2024-04-23T10:42:41.894 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-23T10:42:41.894 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.894 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.894 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.894 INFO:teuthology.orchestra.run.smithi097.stdout: "end_weekday": { 2024-04-23T10:42:41.894 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "end_weekday", 2024-04-23T10:42:41.894 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "uint", 2024-04-23T10:42:41.894 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.895 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.895 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0", 2024-04-23T10:42:41.895 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "0", 2024-04-23T10:42:41.895 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "6", 2024-04-23T10:42:41.895 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.895 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-23T10:42:41.895 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-23T10:42:41.895 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.895 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.895 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.895 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:41.895 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:41.895 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.895 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.895 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.895 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.896 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.896 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.896 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.896 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.896 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.896 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.896 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.896 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.896 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.896 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.896 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.896 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.896 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.896 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.896 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.896 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:41.897 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:41.897 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.897 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.897 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.897 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.897 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.897 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.897 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.897 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.897 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.897 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.897 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.897 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.897 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:41.897 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:41.897 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.898 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.898 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.898 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:41.898 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.898 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.898 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.898 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.898 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.898 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.898 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.898 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.898 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.898 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.898 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.898 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.899 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.899 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.899 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.899 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:41.899 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:41.899 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.899 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.899 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.899 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.899 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.899 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.899 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.899 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.899 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.899 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.899 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.900 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.900 INFO:teuthology.orchestra.run.smithi097.stdout: "min_score": { 2024-04-23T10:42:41.900 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "min_score", 2024-04-23T10:42:41.900 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2024-04-23T10:42:41.900 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.900 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.900 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0", 2024-04-23T10:42:41.900 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.900 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.900 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.900 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-23T10:42:41.900 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.900 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.900 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.900 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.900 INFO:teuthology.orchestra.run.smithi097.stdout: "mode": { 2024-04-23T10:42:41.900 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mode", 2024-04-23T10:42:41.901 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.901 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.901 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.901 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "upmap", 2024-04-23T10:42:41.901 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.901 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.901 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.901 INFO:teuthology.orchestra.run.smithi097.stdout: "crush-compat", 2024-04-23T10:42:41.901 INFO:teuthology.orchestra.run.smithi097.stdout: "none", 2024-04-23T10:42:41.901 INFO:teuthology.orchestra.run.smithi097.stdout: "read", 2024-04-23T10:42:41.901 INFO:teuthology.orchestra.run.smithi097.stdout: "upmap", 2024-04-23T10:42:41.901 INFO:teuthology.orchestra.run.smithi097.stdout: "upmap-read" 2024-04-23T10:42:41.901 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.901 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Balancer mode", 2024-04-23T10:42:41.901 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.901 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.902 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.902 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.902 INFO:teuthology.orchestra.run.smithi097.stdout: "pool_ids": { 2024-04-23T10:42:41.902 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "pool_ids", 2024-04-23T10:42:41.902 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.902 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.902 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.902 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.902 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.902 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.902 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.902 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-23T10:42:41.902 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.902 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.902 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.902 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.903 INFO:teuthology.orchestra.run.smithi097.stdout: "sleep_interval": { 2024-04-23T10:42:41.903 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "sleep_interval", 2024-04-23T10:42:41.903 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-04-23T10:42:41.903 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.903 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.903 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "60", 2024-04-23T10:42:41.903 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.903 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.903 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.903 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-23T10:42:41.903 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.903 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.903 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.903 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.903 INFO:teuthology.orchestra.run.smithi097.stdout: "upmap_max_deviation": { 2024-04-23T10:42:41.903 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "upmap_max_deviation", 2024-04-23T10:42:41.904 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:41.904 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.904 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.904 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "5", 2024-04-23T10:42:41.904 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "1", 2024-04-23T10:42:41.904 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.904 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.904 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-23T10:42:41.904 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-23T10:42:41.904 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.904 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.904 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.904 INFO:teuthology.orchestra.run.smithi097.stdout: "upmap_max_optimizations": { 2024-04-23T10:42:41.904 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "upmap_max_optimizations", 2024-04-23T10:42:41.904 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "uint", 2024-04-23T10:42:41.904 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.905 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.905 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-04-23T10:42:41.905 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.905 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.905 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.905 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-23T10:42:41.905 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.905 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.905 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.905 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.905 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.905 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.905 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:41.905 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "cephadm", 2024-04-23T10:42:41.905 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:41.905 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:41.905 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:41.906 INFO:teuthology.orchestra.run.smithi097.stdout: "agent_down_multiplier": { 2024-04-23T10:42:41.906 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "agent_down_multiplier", 2024-04-23T10:42:41.906 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2024-04-23T10:42:41.906 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.906 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.906 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "3.0", 2024-04-23T10:42:41.906 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.906 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.906 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.906 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-23T10:42:41.906 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.906 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.906 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.906 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.906 INFO:teuthology.orchestra.run.smithi097.stdout: "agent_refresh_rate": { 2024-04-23T10:42:41.906 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "agent_refresh_rate", 2024-04-23T10:42:41.907 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-04-23T10:42:41.907 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.907 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.907 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "20", 2024-04-23T10:42:41.907 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.907 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.907 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.907 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-23T10:42:41.907 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.907 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.907 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.907 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.907 INFO:teuthology.orchestra.run.smithi097.stdout: "agent_starting_port": { 2024-04-23T10:42:41.907 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "agent_starting_port", 2024-04-23T10:42:41.907 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:41.907 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.908 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.908 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "4721", 2024-04-23T10:42:41.908 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.908 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.908 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.908 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-23T10:42:41.908 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.908 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.908 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.908 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.908 INFO:teuthology.orchestra.run.smithi097.stdout: "allow_ptrace": { 2024-04-23T10:42:41.908 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "allow_ptrace", 2024-04-23T10:42:41.908 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.908 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.908 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.909 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.909 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.909 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.909 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.909 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-23T10:42:41.909 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-23T10:42:41.909 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.909 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.909 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.909 INFO:teuthology.orchestra.run.smithi097.stdout: "autotune_interval": { 2024-04-23T10:42:41.909 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "autotune_interval", 2024-04-23T10:42:41.909 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-04-23T10:42:41.909 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.909 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.909 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "600", 2024-04-23T10:42:41.909 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.910 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.910 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.910 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-23T10:42:41.910 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.910 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.910 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.910 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.910 INFO:teuthology.orchestra.run.smithi097.stdout: "autotune_memory_target_ratio": { 2024-04-23T10:42:41.910 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "autotune_memory_target_ratio", 2024-04-23T10:42:41.910 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2024-04-23T10:42:41.910 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.910 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.910 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0.7", 2024-04-23T10:42:41.910 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.910 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.910 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.911 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-23T10:42:41.911 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.911 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.911 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.911 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.911 INFO:teuthology.orchestra.run.smithi097.stdout: "cephadm_log_destination": { 2024-04-23T10:42:41.911 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "cephadm_log_destination", 2024-04-23T10:42:41.911 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.911 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.911 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.911 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.911 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.911 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.911 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.911 INFO:teuthology.orchestra.run.smithi097.stdout: "file", 2024-04-23T10:42:41.911 INFO:teuthology.orchestra.run.smithi097.stdout: "file,syslog", 2024-04-23T10:42:41.911 INFO:teuthology.orchestra.run.smithi097.stdout: "syslog" 2024-04-23T10:42:41.912 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.912 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-23T10:42:41.912 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.912 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.912 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.912 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.912 INFO:teuthology.orchestra.run.smithi097.stdout: "cgroups_split": { 2024-04-23T10:42:41.912 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "cgroups_split", 2024-04-23T10:42:41.912 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.912 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.912 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.912 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:41.912 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.912 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.912 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.912 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-23T10:42:41.913 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.913 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.913 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.913 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.913 INFO:teuthology.orchestra.run.smithi097.stdout: "config_checks_enabled": { 2024-04-23T10:42:41.913 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "config_checks_enabled", 2024-04-23T10:42:41.913 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.913 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.913 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.913 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.913 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.913 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.913 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.913 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-23T10:42:41.913 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.913 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.913 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.914 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.914 INFO:teuthology.orchestra.run.smithi097.stdout: "config_dashboard": { 2024-04-23T10:42:41.914 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "config_dashboard", 2024-04-23T10:42:41.914 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.914 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.914 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.914 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:41.914 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.914 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.914 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.914 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-23T10:42:41.914 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.914 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.914 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.914 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.914 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_alertmanager": { 2024-04-23T10:42:41.915 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_alertmanager", 2024-04-23T10:42:41.915 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.915 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.915 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.915 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-23T10:42:41.915 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.915 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.915 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.915 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Prometheus container image", 2024-04-23T10:42:41.915 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.915 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.915 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.915 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.915 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_base": { 2024-04-23T10:42:41.915 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_base", 2024-04-23T10:42:41.915 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.916 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.916 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.916 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-23T10:42:41.916 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.916 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.916 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.916 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Container image name, without the tag", 2024-04-23T10:42:41.916 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.916 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.916 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.916 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.916 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_elasticsearch": { 2024-04-23T10:42:41.916 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_elasticsearch", 2024-04-23T10:42:41.916 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.916 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.916 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.916 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-23T10:42:41.917 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.917 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.917 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.917 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "elasticsearch container image", 2024-04-23T10:42:41.917 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.917 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.917 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.917 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.917 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_grafana": { 2024-04-23T10:42:41.917 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_grafana", 2024-04-23T10:42:41.917 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.917 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.917 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.917 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-23T10:42:41.917 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.918 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.918 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.918 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Prometheus container image", 2024-04-23T10:42:41.918 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.918 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.918 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.918 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.918 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_haproxy": { 2024-04-23T10:42:41.918 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_haproxy", 2024-04-23T10:42:41.918 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.918 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.918 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.918 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-23T10:42:41.918 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.918 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.919 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.919 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "HAproxy container image", 2024-04-23T10:42:41.919 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.919 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.919 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.919 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.919 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_jaeger_agent": { 2024-04-23T10:42:41.919 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_jaeger_agent", 2024-04-23T10:42:41.919 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.919 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.919 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.919 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-23T10:42:41.919 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.919 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.919 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.919 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Jaeger agent container image", 2024-04-23T10:42:41.919 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.920 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.920 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.920 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.920 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_jaeger_collector": { 2024-04-23T10:42:41.920 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_jaeger_collector", 2024-04-23T10:42:41.920 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.920 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.920 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.920 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-23T10:42:41.920 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.920 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.920 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.920 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Jaeger collector container image", 2024-04-23T10:42:41.920 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.920 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.920 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.921 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.921 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_jaeger_query": { 2024-04-23T10:42:41.921 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_jaeger_query", 2024-04-23T10:42:41.921 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.921 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.921 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.921 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-23T10:42:41.921 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.921 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.921 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.921 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Jaeger query container image", 2024-04-23T10:42:41.921 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.921 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.921 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.921 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.921 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_keepalived": { 2024-04-23T10:42:41.922 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_keepalived", 2024-04-23T10:42:41.922 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.922 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.922 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.922 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-23T10:42:41.922 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.922 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.922 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.922 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Keepalived container image", 2024-04-23T10:42:41.922 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.922 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.922 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.922 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.922 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_loki": { 2024-04-23T10:42:41.922 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_loki", 2024-04-23T10:42:41.922 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.922 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.923 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.923 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-23T10:42:41.923 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.923 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.923 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.923 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Loki container image", 2024-04-23T10:42:41.923 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.923 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.923 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.923 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.923 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_node_exporter": { 2024-04-23T10:42:41.923 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_node_exporter", 2024-04-23T10:42:41.923 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.923 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.923 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.924 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-23T10:42:41.924 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.924 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.924 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.924 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Prometheus container image", 2024-04-23T10:42:41.924 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.924 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.924 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.924 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.924 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_nvmeof": { 2024-04-23T10:42:41.924 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_nvmeof", 2024-04-23T10:42:41.924 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.924 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.924 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.924 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-23T10:42:41.924 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.924 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.925 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.925 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Nvme-of container image", 2024-04-23T10:42:41.925 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.925 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.925 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.925 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.925 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_prometheus": { 2024-04-23T10:42:41.925 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_prometheus", 2024-04-23T10:42:41.925 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.925 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.925 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.925 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-23T10:42:41.925 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.925 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.925 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.925 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Prometheus container image", 2024-04-23T10:42:41.926 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.926 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.926 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.926 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.926 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_promtail": { 2024-04-23T10:42:41.926 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_promtail", 2024-04-23T10:42:41.926 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.926 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.926 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.926 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-23T10:42:41.926 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.926 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.926 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.926 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Promtail container image", 2024-04-23T10:42:41.926 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.926 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.927 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.927 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.927 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_samba": { 2024-04-23T10:42:41.927 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_samba", 2024-04-23T10:42:41.927 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.927 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.927 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.927 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-23T10:42:41.927 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.927 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.927 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.927 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Samba/SMB container image", 2024-04-23T10:42:41.927 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.927 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.927 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.927 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.928 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_snmp_gateway": { 2024-04-23T10:42:41.928 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_snmp_gateway", 2024-04-23T10:42:41.928 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.928 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.928 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.928 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-23T10:42:41.928 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.928 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.928 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.928 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "SNMP Gateway container image", 2024-04-23T10:42:41.928 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.928 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.928 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.928 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.928 INFO:teuthology.orchestra.run.smithi097.stdout: "container_init": { 2024-04-23T10:42:41.929 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_init", 2024-04-23T10:42:41.929 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.929 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.929 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.929 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:41.929 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.929 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.929 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.929 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Run podman/docker with `--init`", 2024-04-23T10:42:41.929 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.929 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.929 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.929 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.929 INFO:teuthology.orchestra.run.smithi097.stdout: "daemon_cache_timeout": { 2024-04-23T10:42:41.929 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "daemon_cache_timeout", 2024-04-23T10:42:41.929 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-04-23T10:42:41.929 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.930 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.930 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "600", 2024-04-23T10:42:41.930 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.930 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.930 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.930 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-23T10:42:41.930 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.930 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.930 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.930 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.930 INFO:teuthology.orchestra.run.smithi097.stdout: "default_cephadm_command_timeout": { 2024-04-23T10:42:41.930 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "default_cephadm_command_timeout", 2024-04-23T10:42:41.930 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:41.930 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.930 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.930 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "900", 2024-04-23T10:42:41.931 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.931 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.931 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.931 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-23T10:42:41.931 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.931 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.931 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.931 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.931 INFO:teuthology.orchestra.run.smithi097.stdout: "default_registry": { 2024-04-23T10:42:41.931 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "default_registry", 2024-04-23T10:42:41.931 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.931 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.931 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.931 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "docker.io", 2024-04-23T10:42:41.932 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.932 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.932 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.932 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-23T10:42:41.932 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.932 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.932 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.932 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.932 INFO:teuthology.orchestra.run.smithi097.stdout: "device_cache_timeout": { 2024-04-23T10:42:41.932 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "device_cache_timeout", 2024-04-23T10:42:41.932 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-04-23T10:42:41.932 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.932 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.932 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "1800", 2024-04-23T10:42:41.932 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.932 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.933 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.933 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "seconds to cache device inventory", 2024-04-23T10:42:41.933 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.933 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.933 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.933 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.933 INFO:teuthology.orchestra.run.smithi097.stdout: "device_enhanced_scan": { 2024-04-23T10:42:41.933 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "device_enhanced_scan", 2024-04-23T10:42:41.933 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.933 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.933 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.933 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.933 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.933 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.933 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.933 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-23T10:42:41.933 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.934 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.934 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.934 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.934 INFO:teuthology.orchestra.run.smithi097.stdout: "facts_cache_timeout": { 2024-04-23T10:42:41.934 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "facts_cache_timeout", 2024-04-23T10:42:41.934 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-04-23T10:42:41.934 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.934 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.934 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "60", 2024-04-23T10:42:41.934 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.934 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.934 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.934 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "seconds to cache host facts data", 2024-04-23T10:42:41.934 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.934 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.934 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.935 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.935 INFO:teuthology.orchestra.run.smithi097.stdout: "grafana_dashboards_path": { 2024-04-23T10:42:41.935 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "grafana_dashboards_path", 2024-04-23T10:42:41.935 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.935 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.935 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.935 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-23T10:42:41.935 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.935 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.935 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.935 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-23T10:42:41.935 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.935 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.935 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.935 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.935 INFO:teuthology.orchestra.run.smithi097.stdout: "host_check_interval": { 2024-04-23T10:42:41.936 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "host_check_interval", 2024-04-23T10:42:41.936 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-04-23T10:42:41.936 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.936 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.936 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "600", 2024-04-23T10:42:41.936 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.936 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.936 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.936 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how frequently to perform a host check", 2024-04-23T10:42:41.936 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.936 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.936 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.936 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.936 INFO:teuthology.orchestra.run.smithi097.stdout: "hw_monitoring": { 2024-04-23T10:42:41.936 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "hw_monitoring", 2024-04-23T10:42:41.936 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.936 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.937 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.937 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.937 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.937 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.937 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.937 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-23T10:42:41.937 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.937 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.937 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.937 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.937 INFO:teuthology.orchestra.run.smithi097.stdout: "inventory_list_all": { 2024-04-23T10:42:41.937 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "inventory_list_all", 2024-04-23T10:42:41.937 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.937 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.937 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.937 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.938 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.938 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.938 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.938 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-23T10:42:41.938 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.938 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.938 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.938 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.938 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:41.938 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:41.938 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.938 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.938 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.938 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.938 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.939 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.939 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.939 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.939 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.939 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.939 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.939 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.939 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.939 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.939 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.939 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.939 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.939 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.939 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.939 INFO:teuthology.orchestra.run.smithi097.stdout: "log_refresh_metadata": { 2024-04-23T10:42:41.939 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_refresh_metadata", 2024-04-23T10:42:41.939 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.940 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.940 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.940 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.940 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.940 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.940 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.940 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-23T10:42:41.940 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.940 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.940 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.940 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.940 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:41.940 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:41.940 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.941 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.941 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.941 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:41.941 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.941 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.941 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.941 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-23T10:42:41.941 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.941 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.941 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.941 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.941 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:41.941 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:41.941 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.941 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.941 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.941 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:41.942 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.942 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.942 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.942 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.942 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.942 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.942 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.942 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.942 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.942 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.942 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.942 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.942 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.942 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.942 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.942 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:41.943 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:41.943 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.943 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.943 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.943 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.943 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.943 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.943 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.943 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.943 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.943 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.943 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.943 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.943 INFO:teuthology.orchestra.run.smithi097.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-23T10:42:41.943 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-23T10:42:41.943 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.944 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.944 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.944 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.944 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.944 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.944 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.944 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-23T10:42:41.944 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.944 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.944 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.944 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.944 INFO:teuthology.orchestra.run.smithi097.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-23T10:42:41.944 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-23T10:42:41.944 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.944 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.944 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.944 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "*", 2024-04-23T10:42:41.945 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.945 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.945 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.945 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-23T10:42:41.945 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.945 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.945 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.945 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.945 INFO:teuthology.orchestra.run.smithi097.stdout: "max_count_per_host": { 2024-04-23T10:42:41.945 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "max_count_per_host", 2024-04-23T10:42:41.945 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:41.945 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.945 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.945 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-04-23T10:42:41.945 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.945 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.946 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.946 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "max number of daemons per service per host", 2024-04-23T10:42:41.946 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.946 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.946 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.946 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.946 INFO:teuthology.orchestra.run.smithi097.stdout: "max_osd_draining_count": { 2024-04-23T10:42:41.946 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "max_osd_draining_count", 2024-04-23T10:42:41.946 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:41.946 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.946 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.946 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-04-23T10:42:41.946 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.946 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.946 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.946 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-23T10:42:41.947 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.947 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.947 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.947 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.947 INFO:teuthology.orchestra.run.smithi097.stdout: "migration_current": { 2024-04-23T10:42:41.947 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "migration_current", 2024-04-23T10:42:41.947 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:41.947 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.947 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.947 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.947 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.947 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.947 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.947 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "internal - do not modify", 2024-04-23T10:42:41.947 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.947 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.947 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.948 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.948 INFO:teuthology.orchestra.run.smithi097.stdout: "mode": { 2024-04-23T10:42:41.948 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mode", 2024-04-23T10:42:41.948 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.948 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.948 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.948 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "root", 2024-04-23T10:42:41.948 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.948 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.948 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.948 INFO:teuthology.orchestra.run.smithi097.stdout: "cephadm-package", 2024-04-23T10:42:41.948 INFO:teuthology.orchestra.run.smithi097.stdout: "root" 2024-04-23T10:42:41.948 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.948 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "mode for remote execution of cephadm", 2024-04-23T10:42:41.948 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.949 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.949 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.949 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.949 INFO:teuthology.orchestra.run.smithi097.stdout: "oob_default_addr": { 2024-04-23T10:42:41.949 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "oob_default_addr", 2024-04-23T10:42:41.949 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.949 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.949 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.949 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "169.254.1.1", 2024-04-23T10:42:41.949 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.949 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.949 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.949 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-23T10:42:41.949 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.949 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.949 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.950 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.950 INFO:teuthology.orchestra.run.smithi097.stdout: "prometheus_alerts_path": { 2024-04-23T10:42:41.950 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "prometheus_alerts_path", 2024-04-23T10:42:41.950 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.950 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.950 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.950 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-23T10:42:41.950 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.950 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.950 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.950 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-23T10:42:41.950 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.950 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.950 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.950 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.951 INFO:teuthology.orchestra.run.smithi097.stdout: "registry_insecure": { 2024-04-23T10:42:41.951 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "registry_insecure", 2024-04-23T10:42:41.951 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.951 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.951 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.951 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.951 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.951 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.951 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.951 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-23T10:42:41.951 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.951 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.951 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.951 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.951 INFO:teuthology.orchestra.run.smithi097.stdout: "registry_password": { 2024-04-23T10:42:41.951 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "registry_password", 2024-04-23T10:42:41.952 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.952 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.952 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.952 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.952 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.952 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.952 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.952 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-23T10:42:41.952 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.952 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.952 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.952 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.952 INFO:teuthology.orchestra.run.smithi097.stdout: "registry_url": { 2024-04-23T10:42:41.952 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "registry_url", 2024-04-23T10:42:41.952 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.952 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.952 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.953 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.953 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.953 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.953 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.953 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-23T10:42:41.953 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.953 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.953 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.953 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.953 INFO:teuthology.orchestra.run.smithi097.stdout: "registry_username": { 2024-04-23T10:42:41.953 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "registry_username", 2024-04-23T10:42:41.953 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.953 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.953 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.953 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.953 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.954 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.954 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.954 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-23T10:42:41.954 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.954 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.954 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.954 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.954 INFO:teuthology.orchestra.run.smithi097.stdout: "secure_monitoring_stack": { 2024-04-23T10:42:41.954 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "secure_monitoring_stack", 2024-04-23T10:42:41.954 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.954 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.954 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.954 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.954 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.954 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.955 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.955 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-23T10:42:41.955 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.955 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.955 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.955 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.955 INFO:teuthology.orchestra.run.smithi097.stdout: "service_discovery_port": { 2024-04-23T10:42:41.955 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "service_discovery_port", 2024-04-23T10:42:41.955 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:41.955 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.955 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.955 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "8765", 2024-04-23T10:42:41.955 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.955 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.955 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.955 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "cephadm service discovery port", 2024-04-23T10:42:41.956 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.956 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.956 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.956 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.956 INFO:teuthology.orchestra.run.smithi097.stdout: "ssh_config_file": { 2024-04-23T10:42:41.956 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ssh_config_file", 2024-04-23T10:42:41.956 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.956 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.956 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.956 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.956 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.956 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.956 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.956 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-23T10:42:41.956 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.956 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.956 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.957 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.957 INFO:teuthology.orchestra.run.smithi097.stdout: "use_agent": { 2024-04-23T10:42:41.957 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "use_agent", 2024-04-23T10:42:41.957 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.957 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.957 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.957 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.957 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.957 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.957 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.957 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-23T10:42:41.957 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.957 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.957 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.957 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.957 INFO:teuthology.orchestra.run.smithi097.stdout: "use_repo_digest": { 2024-04-23T10:42:41.958 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "use_repo_digest", 2024-04-23T10:42:41.958 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.958 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.958 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.958 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:41.958 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.958 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.958 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.958 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-23T10:42:41.958 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.958 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.958 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.958 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.958 INFO:teuthology.orchestra.run.smithi097.stdout: "warn_on_failed_host_check": { 2024-04-23T10:42:41.958 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "warn_on_failed_host_check", 2024-04-23T10:42:41.958 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.959 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.959 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.959 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:41.959 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.959 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.959 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.959 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "raise a health warning if the host check fails", 2024-04-23T10:42:41.959 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.959 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.959 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.959 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.959 INFO:teuthology.orchestra.run.smithi097.stdout: "warn_on_stray_daemons": { 2024-04-23T10:42:41.959 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "warn_on_stray_daemons", 2024-04-23T10:42:41.959 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.959 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.959 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.960 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:41.960 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.960 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.960 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.960 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-23T10:42:41.960 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.960 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.960 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.960 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.960 INFO:teuthology.orchestra.run.smithi097.stdout: "warn_on_stray_hosts": { 2024-04-23T10:42:41.960 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "warn_on_stray_hosts", 2024-04-23T10:42:41.960 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.960 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.960 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.960 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:41.960 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.960 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.961 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.961 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-23T10:42:41.961 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.961 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.961 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.961 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.961 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.961 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.961 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:41.961 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "crash", 2024-04-23T10:42:41.961 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:41.961 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:41.961 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:41.961 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:41.961 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:41.961 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.962 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.962 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.962 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.962 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.962 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.962 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.962 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.962 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.962 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.962 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.962 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.962 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.962 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.962 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.962 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.962 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.963 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.963 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.963 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:41.963 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:41.963 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.963 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.963 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.963 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.963 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.963 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.963 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.963 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.963 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.963 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.963 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.963 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.964 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:41.964 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:41.964 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.964 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.964 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.964 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:41.964 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.964 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.964 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:41.964 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:41.964 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:41.964 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:41.964 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:41.964 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:41.964 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:41.964 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:41.965 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.965 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.965 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.965 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.965 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.965 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:41.965 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:41.965 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.965 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.965 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.965 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.965 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.965 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.965 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.965 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.965 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.965 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.966 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.966 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.966 INFO:teuthology.orchestra.run.smithi097.stdout: "retain_interval": { 2024-04-23T10:42:41.966 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "retain_interval", 2024-04-23T10:42:41.966 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-04-23T10:42:41.966 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.966 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.966 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "31536000", 2024-04-23T10:42:41.966 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.966 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.966 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.966 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-23T10:42:41.966 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.966 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.966 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.967 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.967 INFO:teuthology.orchestra.run.smithi097.stdout: "warn_recent_interval": { 2024-04-23T10:42:41.967 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "warn_recent_interval", 2024-04-23T10:42:41.967 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-04-23T10:42:41.967 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.967 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:41.967 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "1209600", 2024-04-23T10:42:41.967 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.967 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.967 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.967 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-23T10:42:41.967 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.967 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.967 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.967 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.967 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:41.967 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.968 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:41.968 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "dashboard", 2024-04-23T10:42:41.968 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:41.968 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:41.968 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:41.968 INFO:teuthology.orchestra.run.smithi097.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-23T10:42:41.968 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-23T10:42:41.968 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:41.968 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.968 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.968 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-04-23T10:42:41.968 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.968 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.968 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.968 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.968 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.969 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.969 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.969 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.969 INFO:teuthology.orchestra.run.smithi097.stdout: "ALERTMANAGER_API_HOST": { 2024-04-23T10:42:41.969 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-23T10:42:41.969 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.969 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.969 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.969 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.969 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.969 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.969 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.969 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.969 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.969 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.969 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.970 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.970 INFO:teuthology.orchestra.run.smithi097.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-23T10:42:41.970 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-23T10:42:41.970 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.970 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.970 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.970 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:41.970 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.970 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.970 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.970 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.970 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.970 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.970 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.970 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.970 INFO:teuthology.orchestra.run.smithi097.stdout: "AUDIT_API_ENABLED": { 2024-04-23T10:42:41.971 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "AUDIT_API_ENABLED", 2024-04-23T10:42:41.971 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.971 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.971 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.971 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.971 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.971 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.971 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.971 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.971 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.971 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.971 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.971 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.971 INFO:teuthology.orchestra.run.smithi097.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-23T10:42:41.971 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-23T10:42:41.971 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.972 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.972 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.972 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:41.972 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.972 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.972 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.972 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.972 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.972 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.972 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.972 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.972 INFO:teuthology.orchestra.run.smithi097.stdout: "ENABLE_BROWSABLE_API": { 2024-04-23T10:42:41.972 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-23T10:42:41.972 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.972 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.972 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.973 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:41.973 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.973 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.973 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.973 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.973 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.973 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.973 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.973 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.973 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-23T10:42:41.973 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-23T10:42:41.973 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.973 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.973 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.973 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:41.973 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.974 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.974 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.974 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.974 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.974 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.974 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.974 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.974 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-23T10:42:41.974 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-23T10:42:41.974 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.974 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.974 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.974 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:41.974 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.975 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.975 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.975 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.975 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.975 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.975 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.975 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.975 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-23T10:42:41.975 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-23T10:42:41.975 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.975 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.975 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.975 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:41.975 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.975 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.975 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.975 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.976 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.976 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.976 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.976 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.976 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-23T10:42:41.976 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-23T10:42:41.976 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.976 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.976 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.976 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:41.976 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.976 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.976 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.976 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.976 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.976 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.977 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.977 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.977 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-23T10:42:41.977 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-23T10:42:41.977 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.977 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.977 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.977 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:41.977 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.977 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.977 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.977 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.977 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.977 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.977 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.977 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.978 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-23T10:42:41.978 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-23T10:42:41.978 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.978 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.978 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.978 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:41.978 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.978 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.978 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.978 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.978 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.978 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.978 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.978 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.978 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-23T10:42:41.978 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-23T10:42:41.979 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.979 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.979 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.979 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:41.979 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.979 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.979 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.979 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.979 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.979 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.979 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.979 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.979 INFO:teuthology.orchestra.run.smithi097.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-23T10:42:41.979 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-23T10:42:41.979 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.979 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.979 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.980 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.980 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.980 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.980 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.980 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.980 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.980 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.980 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.980 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.980 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_API_PASSWORD": { 2024-04-23T10:42:41.980 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-23T10:42:41.980 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.980 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.980 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.980 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "admin", 2024-04-23T10:42:41.980 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.980 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.981 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.981 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.981 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.981 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.981 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.981 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.981 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-23T10:42:41.981 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-23T10:42:41.981 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.981 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.981 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.981 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:41.981 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.981 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.981 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.981 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.982 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.982 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.982 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.982 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.982 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_API_URL": { 2024-04-23T10:42:41.982 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_API_URL", 2024-04-23T10:42:41.982 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.982 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.982 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.982 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.982 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.982 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.982 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.982 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.982 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.982 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.982 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.983 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.983 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_API_USERNAME": { 2024-04-23T10:42:41.983 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-23T10:42:41.983 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.983 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.983 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.983 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "admin", 2024-04-23T10:42:41.983 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.983 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.983 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.983 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.983 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.983 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.983 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.983 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.983 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-23T10:42:41.984 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-23T10:42:41.984 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.984 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.984 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.984 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.984 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.984 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.984 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.984 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.984 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.984 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.984 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.984 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.984 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-23T10:42:41.984 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-23T10:42:41.985 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.985 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.985 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.985 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.985 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.985 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.985 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.985 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.985 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.985 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.985 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.985 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.985 INFO:teuthology.orchestra.run.smithi097.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-23T10:42:41.985 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-23T10:42:41.986 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.986 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.986 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.986 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:41.986 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.986 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.986 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.986 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.986 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.986 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.987 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.987 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.987 INFO:teuthology.orchestra.run.smithi097.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-23T10:42:41.987 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-23T10:42:41.987 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.987 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.987 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.987 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.987 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.988 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.988 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.988 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.988 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.988 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.988 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.988 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.988 INFO:teuthology.orchestra.run.smithi097.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-23T10:42:41.988 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-23T10:42:41.988 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.989 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.989 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.989 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "[]", 2024-04-23T10:42:41.989 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.989 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.989 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.989 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.989 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.990 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.990 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.990 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.990 INFO:teuthology.orchestra.run.smithi097.stdout: "MULTICLUSTER_CONFIG": { 2024-04-23T10:42:41.990 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-23T10:42:41.990 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.990 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.990 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.990 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "{}", 2024-04-23T10:42:41.991 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.991 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.991 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.991 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.991 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.991 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.991 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.992 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.992 INFO:teuthology.orchestra.run.smithi097.stdout: "PROMETHEUS_API_HOST": { 2024-04-23T10:42:41.992 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-23T10:42:41.992 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:41.992 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.992 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.992 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:41.992 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.992 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.993 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.993 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.993 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.993 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.993 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.993 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.993 INFO:teuthology.orchestra.run.smithi097.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-23T10:42:41.993 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-23T10:42:41.993 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.993 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.993 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.993 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:41.994 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.994 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.994 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.994 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.994 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.994 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.994 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.994 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.994 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-23T10:42:41.994 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-23T10:42:41.994 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.994 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.994 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.994 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.994 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.994 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.995 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.995 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.995 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.995 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.995 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.995 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.995 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-23T10:42:41.995 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-23T10:42:41.995 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.995 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.995 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.995 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.995 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.995 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.995 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.995 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.996 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.996 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.996 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.996 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.996 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-23T10:42:41.996 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-23T10:42:41.996 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.996 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.996 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.996 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:41.996 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.996 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.996 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.996 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.996 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.996 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.997 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.997 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.997 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-23T10:42:41.997 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-23T10:42:41.997 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.997 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.997 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.997 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:41.997 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.997 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.997 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.997 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.997 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.997 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.997 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.997 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.998 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-23T10:42:41.998 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-23T10:42:41.998 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.998 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.998 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.998 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.998 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.998 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.998 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.998 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.998 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.998 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.998 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.998 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.998 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-23T10:42:41.999 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-23T10:42:41.999 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:41.999 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:41.999 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:41.999 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:41.999 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:41.999 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:41.999 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:41.999 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:41.999 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:41.999 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:41.999 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:41.999 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:41.999 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-23T10:42:41.999 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-23T10:42:41.999 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.000 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.000 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.000 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.000 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.000 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.000 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.000 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.000 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.000 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.000 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.000 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.000 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_ENABLED": { 2024-04-23T10:42:42.000 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-23T10:42:42.000 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.000 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.000 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.000 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:42.001 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.001 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.001 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.001 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.001 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.001 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.001 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.001 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.001 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-23T10:42:42.001 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-23T10:42:42.001 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.001 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.001 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.001 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-23T10:42:42.001 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.001 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.002 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.002 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.002 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.002 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.002 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.002 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.002 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-23T10:42:42.002 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-23T10:42:42.002 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:42.002 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.002 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.002 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-04-23T10:42:42.002 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.002 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.002 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.002 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.003 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.003 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.003 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.003 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.003 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-23T10:42:42.003 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-23T10:42:42.003 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:42.003 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.003 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.003 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "8", 2024-04-23T10:42:42.003 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.003 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.003 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.003 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.003 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.003 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.004 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.004 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.004 INFO:teuthology.orchestra.run.smithi097.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-23T10:42:42.004 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-23T10:42:42.004 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:42.004 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.004 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.004 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "45", 2024-04-23T10:42:42.004 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.004 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.004 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.004 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.004 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.004 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.004 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.004 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.005 INFO:teuthology.orchestra.run.smithi097.stdout: "RGW_API_ACCESS_KEY": { 2024-04-23T10:42:42.005 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-23T10:42:42.005 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.005 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.005 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.005 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.005 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.005 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.005 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.005 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.005 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.005 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.005 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.005 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.005 INFO:teuthology.orchestra.run.smithi097.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-23T10:42:42.005 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-23T10:42:42.006 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.006 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.006 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.006 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "admin", 2024-04-23T10:42:42.006 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.006 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.006 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.006 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.006 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.006 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.006 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.006 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.006 INFO:teuthology.orchestra.run.smithi097.stdout: "RGW_API_SECRET_KEY": { 2024-04-23T10:42:42.006 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-23T10:42:42.006 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.006 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.006 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.007 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.007 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.007 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.007 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.007 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.007 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.007 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.007 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.007 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.007 INFO:teuthology.orchestra.run.smithi097.stdout: "RGW_API_SSL_VERIFY": { 2024-04-23T10:42:42.007 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-23T10:42:42.007 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.007 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.007 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.007 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:42.007 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.008 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.008 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.008 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.008 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.008 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.008 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.008 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.008 INFO:teuthology.orchestra.run.smithi097.stdout: "UNSAFE_TLS_v1_2": { 2024-04-23T10:42:42.008 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-23T10:42:42.008 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.008 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.008 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.008 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.008 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.008 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.008 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.009 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.009 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.009 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.009 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.009 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.009 INFO:teuthology.orchestra.run.smithi097.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-23T10:42:42.009 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-23T10:42:42.009 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:42.009 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.009 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.009 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0", 2024-04-23T10:42:42.009 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.009 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.009 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.009 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.010 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.010 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.010 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.010 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.010 INFO:teuthology.orchestra.run.smithi097.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-23T10:42:42.010 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-23T10:42:42.010 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:42.010 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.010 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.010 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-04-23T10:42:42.010 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.010 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.010 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.010 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.011 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.011 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.011 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.011 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.011 INFO:teuthology.orchestra.run.smithi097.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-23T10:42:42.011 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-23T10:42:42.011 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:42.011 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.011 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.011 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "5", 2024-04-23T10:42:42.011 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.011 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.011 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.011 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.011 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.012 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.012 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.012 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.012 INFO:teuthology.orchestra.run.smithi097.stdout: "cross_origin_url": { 2024-04-23T10:42:42.012 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "cross_origin_url", 2024-04-23T10:42:42.012 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.012 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.012 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.012 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.012 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.012 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.012 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.012 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.012 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.012 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.012 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.013 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.013 INFO:teuthology.orchestra.run.smithi097.stdout: "crt_file": { 2024-04-23T10:42:42.013 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "crt_file", 2024-04-23T10:42:42.013 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.013 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.013 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.013 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.013 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.013 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.013 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.013 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.013 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.013 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.013 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.013 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.013 INFO:teuthology.orchestra.run.smithi097.stdout: "debug": { 2024-04-23T10:42:42.013 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "debug", 2024-04-23T10:42:42.014 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.014 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.014 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.014 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.014 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.014 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.014 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.014 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Enable/disable debug options", 2024-04-23T10:42:42.014 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.014 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.014 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.014 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.014 INFO:teuthology.orchestra.run.smithi097.stdout: "jwt_token_ttl": { 2024-04-23T10:42:42.014 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "jwt_token_ttl", 2024-04-23T10:42:42.014 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:42.015 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.015 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.015 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "28800", 2024-04-23T10:42:42.015 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.015 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.015 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.015 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.015 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.015 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.015 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.015 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.015 INFO:teuthology.orchestra.run.smithi097.stdout: "key_file": { 2024-04-23T10:42:42.015 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "key_file", 2024-04-23T10:42:42.015 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.015 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.015 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.016 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.016 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.016 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.016 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.016 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.016 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.016 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.016 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.016 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.016 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:42.016 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:42.016 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.016 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.016 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.016 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.016 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.017 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.017 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.017 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.017 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.017 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.017 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.017 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.017 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.017 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.017 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.017 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.017 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.017 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.017 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.017 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:42.017 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:42.017 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.018 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.018 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.018 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.018 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.018 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.018 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.018 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.018 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.018 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.018 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.018 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.018 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:42.018 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:42.018 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.018 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.019 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.019 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:42.019 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.019 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.019 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.019 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.019 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.019 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.019 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.019 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.019 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.019 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.019 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.019 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.019 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.019 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.020 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.020 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:42.020 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:42.020 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.020 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.020 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.020 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.020 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.020 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.020 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.020 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.020 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.020 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.020 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.020 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.020 INFO:teuthology.orchestra.run.smithi097.stdout: "motd": { 2024-04-23T10:42:42.020 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "motd", 2024-04-23T10:42:42.021 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.021 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.021 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.021 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.021 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.021 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.021 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.021 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "The message of the day", 2024-04-23T10:42:42.021 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.021 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.021 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.021 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.021 INFO:teuthology.orchestra.run.smithi097.stdout: "redirect_resolve_ip_addr": { 2024-04-23T10:42:42.021 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "redirect_resolve_ip_addr", 2024-04-23T10:42:42.021 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.022 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.022 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.022 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.022 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.022 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.022 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.022 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.022 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.022 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.022 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.022 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.022 INFO:teuthology.orchestra.run.smithi097.stdout: "server_addr": { 2024-04-23T10:42:42.022 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_addr", 2024-04-23T10:42:42.022 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.022 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.022 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.023 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "::", 2024-04-23T10:42:42.023 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.023 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.023 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.023 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.023 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.023 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.023 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.023 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.023 INFO:teuthology.orchestra.run.smithi097.stdout: "server_port": { 2024-04-23T10:42:42.023 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_port", 2024-04-23T10:42:42.023 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:42.023 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.023 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.023 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "8080", 2024-04-23T10:42:42.023 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.024 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.024 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.024 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.024 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.024 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.024 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.024 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.024 INFO:teuthology.orchestra.run.smithi097.stdout: "ssl": { 2024-04-23T10:42:42.024 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ssl", 2024-04-23T10:42:42.024 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.024 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.024 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.024 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:42.024 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.024 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.024 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.025 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.025 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.025 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.025 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.025 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.025 INFO:teuthology.orchestra.run.smithi097.stdout: "ssl_server_port": { 2024-04-23T10:42:42.025 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ssl_server_port", 2024-04-23T10:42:42.025 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:42.025 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.025 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.025 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "8443", 2024-04-23T10:42:42.025 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.025 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.025 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.025 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.025 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.025 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.026 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.026 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.026 INFO:teuthology.orchestra.run.smithi097.stdout: "standby_behaviour": { 2024-04-23T10:42:42.026 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "standby_behaviour", 2024-04-23T10:42:42.026 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.026 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.026 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.026 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "redirect", 2024-04-23T10:42:42.026 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.026 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.026 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.026 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.026 INFO:teuthology.orchestra.run.smithi097.stdout: "redirect" 2024-04-23T10:42:42.026 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.026 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.027 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.027 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.027 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.027 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.027 INFO:teuthology.orchestra.run.smithi097.stdout: "standby_error_status_code": { 2024-04-23T10:42:42.027 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "standby_error_status_code", 2024-04-23T10:42:42.027 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:42.027 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.027 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.027 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "500", 2024-04-23T10:42:42.027 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "400", 2024-04-23T10:42:42.027 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "599", 2024-04-23T10:42:42.027 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.027 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.027 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.027 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.027 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.028 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.028 INFO:teuthology.orchestra.run.smithi097.stdout: "url_prefix": { 2024-04-23T10:42:42.028 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "url_prefix", 2024-04-23T10:42:42.028 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.028 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.028 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.028 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.028 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.028 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.028 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.028 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.028 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.028 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.028 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.028 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.028 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.029 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.029 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.029 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "devicehealth", 2024-04-23T10:42:42.029 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:42.029 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:42.029 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:42.029 INFO:teuthology.orchestra.run.smithi097.stdout: "enable_monitoring": { 2024-04-23T10:42:42.029 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "enable_monitoring", 2024-04-23T10:42:42.029 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.029 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.029 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.029 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:42.029 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.029 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.029 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.029 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "monitor device health metrics", 2024-04-23T10:42:42.030 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.030 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.030 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.030 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.030 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:42.030 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:42.030 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.030 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.030 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.030 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.030 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.030 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.030 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.030 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.030 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.030 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.031 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.031 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.031 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.031 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.031 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.031 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.031 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.031 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.031 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.031 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:42.031 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:42.031 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.031 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.031 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.031 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.032 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.032 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.032 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.032 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.032 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.032 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.032 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.032 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.032 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:42.032 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:42.032 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.032 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.032 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.032 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:42.032 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.033 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.033 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.033 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.033 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.033 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.033 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.033 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.033 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.033 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.033 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.033 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.033 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.033 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.033 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.033 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:42.033 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:42.034 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.034 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.034 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.034 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.034 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.034 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.034 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.034 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.034 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.034 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.034 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.034 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.034 INFO:teuthology.orchestra.run.smithi097.stdout: "mark_out_threshold": { 2024-04-23T10:42:42.034 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mark_out_threshold", 2024-04-23T10:42:42.034 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-04-23T10:42:42.034 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.035 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.035 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "2419200", 2024-04-23T10:42:42.035 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.035 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.035 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.035 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-23T10:42:42.035 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.035 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.035 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.035 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.035 INFO:teuthology.orchestra.run.smithi097.stdout: "pool_name": { 2024-04-23T10:42:42.035 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "pool_name", 2024-04-23T10:42:42.035 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.035 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.035 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.035 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "device_health_metrics", 2024-04-23T10:42:42.036 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.036 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.036 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.036 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-23T10:42:42.036 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.036 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.036 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.036 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.036 INFO:teuthology.orchestra.run.smithi097.stdout: "retention_period": { 2024-04-23T10:42:42.036 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "retention_period", 2024-04-23T10:42:42.036 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-04-23T10:42:42.036 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.036 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.036 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "15552000", 2024-04-23T10:42:42.036 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.036 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.036 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.037 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how long to retain device health metrics", 2024-04-23T10:42:42.037 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.037 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.037 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.037 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.037 INFO:teuthology.orchestra.run.smithi097.stdout: "scrape_frequency": { 2024-04-23T10:42:42.037 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "scrape_frequency", 2024-04-23T10:42:42.037 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-04-23T10:42:42.037 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.037 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.037 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "86400", 2024-04-23T10:42:42.037 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.037 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.037 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.037 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-23T10:42:42.038 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.038 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.038 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.038 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.038 INFO:teuthology.orchestra.run.smithi097.stdout: "self_heal": { 2024-04-23T10:42:42.038 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "self_heal", 2024-04-23T10:42:42.038 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.038 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.038 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.038 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:42.038 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.038 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.038 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.038 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-23T10:42:42.038 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.038 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.039 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.039 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.039 INFO:teuthology.orchestra.run.smithi097.stdout: "sleep_interval": { 2024-04-23T10:42:42.039 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "sleep_interval", 2024-04-23T10:42:42.039 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-04-23T10:42:42.039 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.039 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.039 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "600", 2024-04-23T10:42:42.039 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.039 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.039 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.039 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how frequently to wake up and check device health", 2024-04-23T10:42:42.039 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.039 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.039 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.039 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.039 INFO:teuthology.orchestra.run.smithi097.stdout: "warn_threshold": { 2024-04-23T10:42:42.040 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "warn_threshold", 2024-04-23T10:42:42.040 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-04-23T10:42:42.040 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.040 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.040 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "7257600", 2024-04-23T10:42:42.040 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.040 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.040 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.040 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-23T10:42:42.040 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.040 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.040 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.040 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.040 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.040 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.040 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.041 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "diskprediction_local", 2024-04-23T10:42:42.041 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:42.041 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:42.041 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:42.041 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:42.041 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:42.041 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.041 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.041 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.041 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.041 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.041 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.041 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.041 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.041 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.041 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.042 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.042 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.042 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.042 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.042 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.042 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.042 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.042 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.042 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.042 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:42.042 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:42.042 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.042 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.042 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.042 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.042 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.043 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.043 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.043 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.043 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.043 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.043 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.043 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.043 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:42.043 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:42.043 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.043 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.043 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.043 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:42.043 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.043 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.043 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.043 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.044 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.044 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.044 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.044 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.044 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.044 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.044 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.044 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.044 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.044 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.044 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.044 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:42.044 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:42.044 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.044 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.044 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.045 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.045 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.045 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.045 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.045 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.045 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.045 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.045 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.045 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.045 INFO:teuthology.orchestra.run.smithi097.stdout: "predict_interval": { 2024-04-23T10:42:42.045 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "predict_interval", 2024-04-23T10:42:42.045 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.045 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.045 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.045 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "86400", 2024-04-23T10:42:42.045 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.046 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.046 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.046 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.046 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.046 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.046 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.046 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.046 INFO:teuthology.orchestra.run.smithi097.stdout: "predictor_model": { 2024-04-23T10:42:42.046 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "predictor_model", 2024-04-23T10:42:42.046 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.046 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.046 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.046 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "prophetstor", 2024-04-23T10:42:42.046 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.046 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.047 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.047 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.047 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.047 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.047 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.047 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.047 INFO:teuthology.orchestra.run.smithi097.stdout: "sleep_interval": { 2024-04-23T10:42:42.047 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "sleep_interval", 2024-04-23T10:42:42.047 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.047 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.047 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.047 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "600", 2024-04-23T10:42:42.047 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.047 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.047 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.047 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.047 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.048 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.048 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.048 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.048 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.048 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.048 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.048 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "influx", 2024-04-23T10:42:42.048 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": false, 2024-04-23T10:42:42.048 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "influxdb python module not found", 2024-04-23T10:42:42.048 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:42.048 INFO:teuthology.orchestra.run.smithi097.stdout: "batch_size": { 2024-04-23T10:42:42.048 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "batch_size", 2024-04-23T10:42:42.048 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:42.048 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.048 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.048 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "5000", 2024-04-23T10:42:42.049 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.049 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.049 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.049 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-23T10:42:42.049 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.049 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.049 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.049 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.049 INFO:teuthology.orchestra.run.smithi097.stdout: "database": { 2024-04-23T10:42:42.049 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "database", 2024-04-23T10:42:42.049 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.049 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.049 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.049 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "ceph", 2024-04-23T10:42:42.049 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.050 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.050 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.050 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-23T10:42:42.050 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.050 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.050 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.050 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.050 INFO:teuthology.orchestra.run.smithi097.stdout: "hostname": { 2024-04-23T10:42:42.050 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "hostname", 2024-04-23T10:42:42.050 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.050 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.050 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.050 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.050 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.050 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.051 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.051 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "InfluxDB server hostname", 2024-04-23T10:42:42.051 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.051 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.051 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.051 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.051 INFO:teuthology.orchestra.run.smithi097.stdout: "interval": { 2024-04-23T10:42:42.051 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "interval", 2024-04-23T10:42:42.051 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-04-23T10:42:42.051 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.051 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.051 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "30", 2024-04-23T10:42:42.051 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "5", 2024-04-23T10:42:42.051 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.051 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.051 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-23T10:42:42.052 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.052 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.052 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.052 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.052 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:42.052 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:42.052 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.052 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.052 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.052 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.052 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.052 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.052 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.052 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.052 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.052 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.052 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.053 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.053 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.053 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.053 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.053 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.053 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.053 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.053 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.053 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:42.053 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:42.053 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.053 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.053 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.053 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.053 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.053 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.054 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.054 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.054 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.054 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.054 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.054 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.054 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:42.054 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:42.054 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.054 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.054 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.054 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:42.054 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.054 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.054 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.054 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.055 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.055 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.055 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.055 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.055 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.055 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.055 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.055 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.055 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.055 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.055 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.055 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:42.055 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:42.055 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.055 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.055 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.055 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.056 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.056 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.056 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.056 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.056 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.056 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.056 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.056 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.056 INFO:teuthology.orchestra.run.smithi097.stdout: "password": { 2024-04-23T10:42:42.056 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "password", 2024-04-23T10:42:42.056 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.056 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.056 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.056 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.056 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.057 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.057 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.057 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "password of InfluxDB server user", 2024-04-23T10:42:42.057 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.057 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.057 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.057 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.057 INFO:teuthology.orchestra.run.smithi097.stdout: "port": { 2024-04-23T10:42:42.057 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "port", 2024-04-23T10:42:42.057 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:42.057 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.057 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.057 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "8086", 2024-04-23T10:42:42.057 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.057 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.057 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.057 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "InfluxDB server port", 2024-04-23T10:42:42.058 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.058 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.058 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.058 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.058 INFO:teuthology.orchestra.run.smithi097.stdout: "ssl": { 2024-04-23T10:42:42.058 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ssl", 2024-04-23T10:42:42.058 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.058 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.058 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.058 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "false", 2024-04-23T10:42:42.058 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.058 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.058 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.058 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-23T10:42:42.058 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.058 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.059 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.059 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.059 INFO:teuthology.orchestra.run.smithi097.stdout: "threads": { 2024-04-23T10:42:42.059 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "threads", 2024-04-23T10:42:42.059 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:42.059 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.059 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.059 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "5", 2024-04-23T10:42:42.059 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "1", 2024-04-23T10:42:42.059 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "32", 2024-04-23T10:42:42.059 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.059 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-23T10:42:42.059 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.059 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.059 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.059 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.060 INFO:teuthology.orchestra.run.smithi097.stdout: "username": { 2024-04-23T10:42:42.060 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "username", 2024-04-23T10:42:42.060 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.060 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.060 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.060 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.060 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.060 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.060 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.060 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "username of InfluxDB server user", 2024-04-23T10:42:42.060 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.060 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.060 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.060 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.060 INFO:teuthology.orchestra.run.smithi097.stdout: "verify_ssl": { 2024-04-23T10:42:42.060 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "verify_ssl", 2024-04-23T10:42:42.061 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.061 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.061 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.061 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "true", 2024-04-23T10:42:42.061 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.061 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.061 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.061 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-23T10:42:42.061 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.061 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.061 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.061 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.061 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.061 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.061 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.061 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "insights", 2024-04-23T10:42:42.062 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:42.062 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:42.062 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:42.062 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:42.062 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:42.062 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.062 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.062 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.062 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.062 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.062 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.062 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.062 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.062 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.062 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.062 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.063 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.063 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.063 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.063 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.063 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.063 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.063 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.063 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.063 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:42.063 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:42.063 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.063 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.063 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.063 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.063 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.063 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.064 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.064 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.064 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.064 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.064 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.064 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.064 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:42.064 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:42.064 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.064 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.064 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.064 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:42.064 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.064 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.064 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.064 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.064 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.065 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.065 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.065 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.065 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.065 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.065 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.065 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.065 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.065 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.065 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.065 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:42.065 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:42.065 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.065 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.065 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.065 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.066 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.066 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.066 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.066 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.066 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.066 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.066 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.066 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.066 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.066 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.066 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.066 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "iostat", 2024-04-23T10:42:42.066 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:42.066 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:42.066 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:42.067 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:42.067 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:42.067 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.067 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.067 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.067 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.067 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.067 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.067 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.067 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.067 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.067 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.067 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.067 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.067 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.067 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.068 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.068 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.068 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.068 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.068 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.068 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:42.068 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:42.068 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.068 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.068 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.068 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.068 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.068 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.068 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.068 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.068 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.069 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.069 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.069 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.069 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:42.069 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:42.069 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.069 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.069 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.069 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:42.069 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.069 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.069 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.069 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.069 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.069 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.069 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.070 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.070 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.070 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.070 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.070 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.070 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.070 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.070 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.070 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:42.070 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:42.070 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.070 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.070 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.070 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.070 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.070 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.071 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.071 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.071 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.071 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.071 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.071 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.071 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.071 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.071 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.071 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "k8sevents", 2024-04-23T10:42:42.071 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:42.071 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:42.071 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:42.071 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_event_retention_days": { 2024-04-23T10:42:42.071 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ceph_event_retention_days", 2024-04-23T10:42:42.071 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:42.072 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.072 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.072 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "7", 2024-04-23T10:42:42.072 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.072 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.072 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.072 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-23T10:42:42.072 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.072 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.072 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.072 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.072 INFO:teuthology.orchestra.run.smithi097.stdout: "config_check_secs": { 2024-04-23T10:42:42.072 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "config_check_secs", 2024-04-23T10:42:42.072 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:42.072 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.073 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.073 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-04-23T10:42:42.073 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "10", 2024-04-23T10:42:42.073 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.073 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.073 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-23T10:42:42.073 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.073 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.073 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.073 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.073 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:42.073 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:42.073 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.073 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.073 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.073 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.073 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.074 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.074 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.074 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.074 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.074 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.074 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.074 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.074 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.074 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.074 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.074 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.074 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.074 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.074 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.075 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:42.075 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:42.075 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.075 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.075 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.075 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.075 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.075 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.075 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.075 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.075 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.075 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.075 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.075 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.075 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:42.075 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:42.076 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.076 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.076 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.076 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:42.076 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.076 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.076 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.076 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.076 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.076 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.076 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.076 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.076 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.076 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.076 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.076 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.076 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.077 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.077 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.077 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:42.077 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:42.077 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.077 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.077 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.077 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.077 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.077 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.077 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.077 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.077 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.077 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.077 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.077 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.078 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.078 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.078 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.078 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "localpool", 2024-04-23T10:42:42.078 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:42.078 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:42.078 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:42.078 INFO:teuthology.orchestra.run.smithi097.stdout: "failure_domain": { 2024-04-23T10:42:42.078 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "failure_domain", 2024-04-23T10:42:42.078 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.078 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.078 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.078 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "host", 2024-04-23T10:42:42.078 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.078 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.078 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.079 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "failure domain for any created local pool", 2024-04-23T10:42:42.079 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-23T10:42:42.079 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.079 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.079 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.079 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:42.079 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:42.079 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.079 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.079 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.079 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.079 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.079 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.079 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.079 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.079 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.080 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.080 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.080 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.080 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.080 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.080 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.080 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.080 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.080 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.080 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.080 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:42.080 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:42.080 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.080 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.081 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.081 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.081 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.081 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.081 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.081 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.081 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.081 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.081 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.081 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.081 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:42.081 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:42.081 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.081 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.081 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.081 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:42.082 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.082 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.082 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.082 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.082 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.082 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.082 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.082 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.082 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.082 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.082 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.082 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.082 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.082 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.082 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.082 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:42.083 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:42.083 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.083 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.083 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.083 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.083 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.083 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.083 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.083 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.083 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.083 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.083 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.083 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.083 INFO:teuthology.orchestra.run.smithi097.stdout: "min_size": { 2024-04-23T10:42:42.083 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "min_size", 2024-04-23T10:42:42.083 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:42.084 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.084 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.084 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.084 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.084 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.084 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.084 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "default min_size for any created local pool", 2024-04-23T10:42:42.084 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-23T10:42:42.084 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.084 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.084 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.084 INFO:teuthology.orchestra.run.smithi097.stdout: "num_rep": { 2024-04-23T10:42:42.084 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "num_rep", 2024-04-23T10:42:42.084 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:42.084 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.084 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.085 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "3", 2024-04-23T10:42:42.085 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.085 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.085 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.085 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "default replica count for any created local pool", 2024-04-23T10:42:42.085 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.085 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.085 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.085 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.085 INFO:teuthology.orchestra.run.smithi097.stdout: "pg_num": { 2024-04-23T10:42:42.085 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "pg_num", 2024-04-23T10:42:42.085 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:42.085 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.085 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.085 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "128", 2024-04-23T10:42:42.085 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.085 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.086 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.086 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "default pg_num for any created local pool", 2024-04-23T10:42:42.086 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.086 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.086 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.086 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.086 INFO:teuthology.orchestra.run.smithi097.stdout: "prefix": { 2024-04-23T10:42:42.086 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "prefix", 2024-04-23T10:42:42.086 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.086 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.086 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.086 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.086 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.086 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.087 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.087 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "name prefix for any created local pool", 2024-04-23T10:42:42.087 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.087 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.087 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.087 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.087 INFO:teuthology.orchestra.run.smithi097.stdout: "subtree": { 2024-04-23T10:42:42.087 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "subtree", 2024-04-23T10:42:42.087 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.087 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.087 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.087 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "rack", 2024-04-23T10:42:42.087 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.087 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.087 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.087 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-23T10:42:42.088 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-23T10:42:42.088 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.088 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.088 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.088 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.088 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.088 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.088 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mds_autoscaler", 2024-04-23T10:42:42.088 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:42.088 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:42.088 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:42.088 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:42.088 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:42.088 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.088 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.088 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.089 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.089 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.089 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.089 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.089 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.089 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.089 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.089 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.089 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.089 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.089 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.089 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.089 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.089 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.089 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.089 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.090 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:42.090 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:42.090 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.090 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.090 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.090 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.090 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.090 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.090 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.090 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.090 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.090 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.090 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.090 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.090 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:42.090 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:42.091 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.091 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.091 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.091 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:42.091 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.091 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.091 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.091 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.091 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.091 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.091 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.091 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.091 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.091 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.091 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.091 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.092 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.092 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.092 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.092 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:42.092 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:42.092 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.092 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.092 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.092 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.092 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.092 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.092 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.092 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.092 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.092 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.092 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.093 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.093 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.093 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.093 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.093 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mirroring", 2024-04-23T10:42:42.093 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:42.093 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:42.093 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:42.093 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:42.093 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:42.093 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.093 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.093 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.093 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.093 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.093 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.094 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.094 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.094 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.094 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.094 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.094 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.094 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.094 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.094 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.094 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.094 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.094 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.094 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.094 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:42.094 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:42.095 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.095 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.095 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.095 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.095 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.095 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.095 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.095 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.095 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.095 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.095 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.095 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.095 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:42.095 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:42.095 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.095 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.096 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.096 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:42.096 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.096 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.096 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.096 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.096 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.096 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.096 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.096 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.096 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.096 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.096 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.096 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.096 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.096 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.096 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.097 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:42.097 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:42.097 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.097 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.097 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.097 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.097 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.097 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.097 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.097 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.097 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.097 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.097 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.097 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.097 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.097 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.098 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.098 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "nfs", 2024-04-23T10:42:42.098 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:42.098 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:42.098 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:42.098 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:42.098 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:42.098 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.098 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.098 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.098 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.098 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.098 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.098 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.098 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.098 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.099 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.099 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.099 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.099 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.099 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.099 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.099 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.099 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.099 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.099 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.099 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:42.099 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:42.099 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.099 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.099 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.099 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.100 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.100 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.100 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.100 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.100 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.100 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.100 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.100 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.100 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:42.100 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:42.100 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.100 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.100 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.100 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:42.100 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.100 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.101 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.101 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.101 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.101 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.101 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.101 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.101 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.101 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.101 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.101 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.101 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.101 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.101 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.101 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:42.101 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:42.101 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.101 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.102 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.102 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.102 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.102 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.102 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.102 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.102 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.102 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.102 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.102 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.102 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.102 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.102 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.102 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "orchestrator", 2024-04-23T10:42:42.102 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:42.102 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:42.103 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:42.103 INFO:teuthology.orchestra.run.smithi097.stdout: "fail_fs": { 2024-04-23T10:42:42.103 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "fail_fs", 2024-04-23T10:42:42.103 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.103 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.103 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.103 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.103 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.103 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.103 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.103 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-23T10:42:42.103 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.103 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.103 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.103 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.103 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:42.103 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:42.104 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.104 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.104 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.104 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.104 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.104 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.104 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.104 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.104 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.104 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.104 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.104 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.104 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.104 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.104 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.104 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.105 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.105 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.105 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.105 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:42.105 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:42.105 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.105 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.105 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.105 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.105 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.105 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.105 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.105 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.105 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.105 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.105 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.106 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.106 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:42.106 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:42.106 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.106 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.106 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.106 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:42.106 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.106 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.106 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.106 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.106 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.106 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.106 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.106 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.106 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.107 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.107 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.107 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.107 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.107 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.107 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.107 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:42.107 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:42.107 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.107 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.107 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.107 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.107 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.107 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.107 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.107 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.108 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.108 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.108 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.108 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.108 INFO:teuthology.orchestra.run.smithi097.stdout: "orchestrator": { 2024-04-23T10:42:42.108 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "orchestrator", 2024-04-23T10:42:42.108 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.108 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.108 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.108 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.108 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.108 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.108 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.108 INFO:teuthology.orchestra.run.smithi097.stdout: "cephadm", 2024-04-23T10:42:42.108 INFO:teuthology.orchestra.run.smithi097.stdout: "rook", 2024-04-23T10:42:42.108 INFO:teuthology.orchestra.run.smithi097.stdout: "test_orchestrator" 2024-04-23T10:42:42.109 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.109 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Orchestrator backend", 2024-04-23T10:42:42.109 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.109 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.109 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.109 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.109 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.109 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.109 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.109 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "osd_perf_query", 2024-04-23T10:42:42.109 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:42.109 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:42.109 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:42.109 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:42.109 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:42.109 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.109 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.110 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.110 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.110 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.110 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.110 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.110 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.110 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.110 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.110 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.110 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.110 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.110 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.110 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.110 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.110 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.110 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.111 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.111 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:42.111 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:42.111 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.111 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.111 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.111 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.111 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.111 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.111 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.111 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.111 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.111 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.111 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.111 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.112 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:42.112 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:42.112 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.112 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.112 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.112 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:42.112 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.112 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.112 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.112 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.112 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.112 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.112 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.112 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.112 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.112 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.112 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.113 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.113 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.113 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.113 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.113 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:42.113 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:42.113 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.113 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.113 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.113 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.113 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.113 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.113 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.113 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.113 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.113 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.114 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.114 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.114 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.114 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.114 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.114 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "osd_support", 2024-04-23T10:42:42.114 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:42.114 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:42.114 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:42.114 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:42.114 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:42.114 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.114 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.114 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.114 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.114 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.115 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.115 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.115 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.115 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.115 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.115 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.115 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.115 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.115 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.115 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.115 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.115 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.115 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.115 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.115 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:42.115 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:42.115 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.116 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.116 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.116 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.116 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.116 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.116 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.116 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.116 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.116 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.116 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.116 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.116 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:42.116 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:42.116 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.116 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.116 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.117 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:42.117 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.117 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.117 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.117 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.117 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.117 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.117 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.117 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.117 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.117 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.117 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.117 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.117 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.117 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.117 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.118 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:42.118 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:42.118 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.118 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.118 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.118 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.118 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.118 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.118 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.118 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.118 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.118 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.118 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.118 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.118 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.118 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.119 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.119 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "pg_autoscaler", 2024-04-23T10:42:42.119 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:42.119 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:42.119 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:42.119 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:42.119 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:42.119 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.119 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.119 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.119 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.119 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.119 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.119 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.119 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.119 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.119 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.120 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.120 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.120 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.120 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.120 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.120 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.120 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.120 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.120 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.120 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:42.120 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:42.120 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.120 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.120 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.120 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.120 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.121 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.121 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.121 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.121 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.121 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.121 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.121 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.121 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:42.121 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:42.121 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.121 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.121 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.121 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:42.121 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.121 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.121 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.122 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.122 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.122 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.122 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.122 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.122 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.122 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.122 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.122 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.122 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.122 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.122 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.122 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:42.122 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:42.122 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.122 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.123 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.123 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.123 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.123 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.123 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.123 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.123 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.123 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.123 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.123 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.123 INFO:teuthology.orchestra.run.smithi097.stdout: "sleep_interval": { 2024-04-23T10:42:42.123 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "sleep_interval", 2024-04-23T10:42:42.123 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-04-23T10:42:42.123 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.123 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.123 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "60", 2024-04-23T10:42:42.123 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.124 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.124 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.124 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.124 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.124 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.124 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.124 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.124 INFO:teuthology.orchestra.run.smithi097.stdout: "threshold": { 2024-04-23T10:42:42.124 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "threshold", 2024-04-23T10:42:42.124 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2024-04-23T10:42:42.124 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.124 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.124 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "3.0", 2024-04-23T10:42:42.124 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "1.0", 2024-04-23T10:42:42.124 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.124 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.125 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "scaling threshold", 2024-04-23T10:42:42.125 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-23T10:42:42.125 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.125 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.125 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.125 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.125 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.125 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.125 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "progress", 2024-04-23T10:42:42.125 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:42.125 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:42.125 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:42.125 INFO:teuthology.orchestra.run.smithi097.stdout: "allow_pg_recovery_event": { 2024-04-23T10:42:42.125 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "allow_pg_recovery_event", 2024-04-23T10:42:42.125 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.125 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.125 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.126 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.126 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.126 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.126 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.126 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-23T10:42:42.126 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.126 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.126 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.126 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.126 INFO:teuthology.orchestra.run.smithi097.stdout: "enabled": { 2024-04-23T10:42:42.126 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "enabled", 2024-04-23T10:42:42.126 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.126 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.126 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.126 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:42.126 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.127 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.127 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.127 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.127 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.127 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.127 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.127 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.127 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:42.127 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:42.127 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.127 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.127 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.127 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.127 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.127 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.127 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.127 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.128 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.128 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.128 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.128 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.128 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.128 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.128 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.128 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.128 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.128 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.128 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.128 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:42.128 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:42.128 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.128 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.128 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.129 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.129 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.129 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.129 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.129 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.129 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.129 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.129 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.129 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.129 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:42.129 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:42.129 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.129 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.129 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.129 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:42.129 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.130 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.130 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.130 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.130 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.130 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.130 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.130 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.130 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.130 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.130 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.130 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.130 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.130 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.130 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.130 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:42.131 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:42.131 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.131 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.131 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.131 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.131 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.131 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.131 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.131 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.131 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.131 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.131 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.131 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.131 INFO:teuthology.orchestra.run.smithi097.stdout: "max_completed_events": { 2024-04-23T10:42:42.131 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "max_completed_events", 2024-04-23T10:42:42.131 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:42.132 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.132 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.132 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "50", 2024-04-23T10:42:42.132 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.132 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.132 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.132 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "number of past completed events to remember", 2024-04-23T10:42:42.132 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.132 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.132 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.132 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.132 INFO:teuthology.orchestra.run.smithi097.stdout: "sleep_interval": { 2024-04-23T10:42:42.132 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "sleep_interval", 2024-04-23T10:42:42.132 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-04-23T10:42:42.132 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.133 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.133 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "5", 2024-04-23T10:42:42.133 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.133 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.133 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.133 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how long the module is going to sleep", 2024-04-23T10:42:42.133 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.133 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.133 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.133 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.133 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.133 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.133 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.133 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "prometheus", 2024-04-23T10:42:42.133 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:42.133 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:42.133 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:42.134 INFO:teuthology.orchestra.run.smithi097.stdout: "cache": { 2024-04-23T10:42:42.134 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "cache", 2024-04-23T10:42:42.134 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.134 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.134 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.134 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:42.134 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.134 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.134 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.134 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.134 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.134 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.134 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.134 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.134 INFO:teuthology.orchestra.run.smithi097.stdout: "exclude_perf_counters": { 2024-04-23T10:42:42.135 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "exclude_perf_counters", 2024-04-23T10:42:42.135 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.135 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.135 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.135 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:42.135 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.135 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.135 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.135 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-23T10:42:42.135 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-23T10:42:42.135 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.135 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.135 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.135 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:42.135 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:42.135 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.136 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.136 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.136 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.136 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.136 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.136 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.136 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.136 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.136 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.136 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.136 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.136 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.136 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.136 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.136 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.136 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.136 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.137 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.137 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:42.137 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:42.137 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.137 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.137 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.137 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.137 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.137 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.137 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.137 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.137 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.137 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.137 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.137 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.137 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:42.137 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:42.138 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.138 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.138 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.138 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:42.138 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.138 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.138 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.138 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.138 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.138 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.138 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.138 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.138 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.138 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.138 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.138 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.139 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.139 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.139 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.139 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:42.139 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:42.139 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.139 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.139 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.139 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.139 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.139 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.139 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.139 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.139 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.139 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.139 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.139 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.140 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_stats_pools": { 2024-04-23T10:42:42.140 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rbd_stats_pools", 2024-04-23T10:42:42.140 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.140 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.140 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.140 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.140 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.140 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.140 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.140 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.140 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.140 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.140 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.140 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.140 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-23T10:42:42.140 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-23T10:42:42.141 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:42.141 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.141 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.141 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "300", 2024-04-23T10:42:42.141 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.141 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.141 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.141 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.141 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.141 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.141 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.141 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.141 INFO:teuthology.orchestra.run.smithi097.stdout: "scrape_interval": { 2024-04-23T10:42:42.141 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "scrape_interval", 2024-04-23T10:42:42.141 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2024-04-23T10:42:42.141 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.142 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.142 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "15.0", 2024-04-23T10:42:42.142 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.142 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.142 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.142 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.142 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.142 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.142 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.142 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.142 INFO:teuthology.orchestra.run.smithi097.stdout: "server_addr": { 2024-04-23T10:42:42.142 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_addr", 2024-04-23T10:42:42.142 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.142 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.142 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.143 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "::", 2024-04-23T10:42:42.143 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.143 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.143 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.143 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-23T10:42:42.143 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.143 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.143 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.143 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.143 INFO:teuthology.orchestra.run.smithi097.stdout: "server_port": { 2024-04-23T10:42:42.143 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_port", 2024-04-23T10:42:42.143 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:42.143 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.143 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.143 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "9283", 2024-04-23T10:42:42.143 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.144 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.144 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.144 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-23T10:42:42.144 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.144 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.144 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.144 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.144 INFO:teuthology.orchestra.run.smithi097.stdout: "stale_cache_strategy": { 2024-04-23T10:42:42.144 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "stale_cache_strategy", 2024-04-23T10:42:42.144 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.144 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.144 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.144 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "log", 2024-04-23T10:42:42.144 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.144 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.144 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.145 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.145 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.145 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.145 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.145 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.145 INFO:teuthology.orchestra.run.smithi097.stdout: "standby_behaviour": { 2024-04-23T10:42:42.145 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "standby_behaviour", 2024-04-23T10:42:42.145 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.145 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.145 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.145 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "default", 2024-04-23T10:42:42.145 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.145 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.145 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.145 INFO:teuthology.orchestra.run.smithi097.stdout: "default", 2024-04-23T10:42:42.145 INFO:teuthology.orchestra.run.smithi097.stdout: "error" 2024-04-23T10:42:42.145 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.146 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.146 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.146 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.146 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.146 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.146 INFO:teuthology.orchestra.run.smithi097.stdout: "standby_error_status_code": { 2024-04-23T10:42:42.146 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "standby_error_status_code", 2024-04-23T10:42:42.146 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:42.146 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.146 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.146 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "500", 2024-04-23T10:42:42.146 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "400", 2024-04-23T10:42:42.146 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "599", 2024-04-23T10:42:42.146 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.146 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.146 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.147 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.147 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.147 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.147 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.147 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.147 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.147 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rbd_support", 2024-04-23T10:42:42.147 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:42.147 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:42.147 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:42.147 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:42.147 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:42.147 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.147 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.147 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.147 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.147 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.148 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.148 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.148 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.148 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.148 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.148 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.148 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.148 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.148 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.148 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.148 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.148 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.148 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.148 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.148 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:42.148 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:42.149 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.149 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.149 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.149 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.149 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.149 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.149 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.149 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.149 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.149 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.149 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.149 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.149 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:42.149 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:42.149 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.149 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.149 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.150 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:42.150 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.150 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.150 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.150 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.150 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.150 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.150 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.150 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.150 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.150 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.150 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.150 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.150 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.150 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.151 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.151 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:42.151 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:42.151 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.151 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.151 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.151 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.151 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.151 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.151 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.151 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.151 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.151 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.151 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.151 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.151 INFO:teuthology.orchestra.run.smithi097.stdout: "max_concurrent_snap_create": { 2024-04-23T10:42:42.152 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "max_concurrent_snap_create", 2024-04-23T10:42:42.152 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:42.152 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.152 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.152 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-04-23T10:42:42.152 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.152 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.152 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.152 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.152 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.152 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.152 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.152 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.152 INFO:teuthology.orchestra.run.smithi097.stdout: "mirror_snapshot_schedule": { 2024-04-23T10:42:42.152 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mirror_snapshot_schedule", 2024-04-23T10:42:42.152 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.153 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.153 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.153 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.153 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.153 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.153 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.153 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.153 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.153 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.153 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.153 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.153 INFO:teuthology.orchestra.run.smithi097.stdout: "trash_purge_schedule": { 2024-04-23T10:42:42.153 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "trash_purge_schedule", 2024-04-23T10:42:42.153 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.153 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.153 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.154 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.154 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.154 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.154 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.154 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.154 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.154 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.154 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.154 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.154 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.154 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.154 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.155 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "restful", 2024-04-23T10:42:42.155 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:42.155 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:42.155 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:42.155 INFO:teuthology.orchestra.run.smithi097.stdout: "enable_auth": { 2024-04-23T10:42:42.155 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "enable_auth", 2024-04-23T10:42:42.155 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.155 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.155 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.155 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:42.155 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.155 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.155 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.155 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.156 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.156 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.156 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.156 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.156 INFO:teuthology.orchestra.run.smithi097.stdout: "key_file": { 2024-04-23T10:42:42.156 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "key_file", 2024-04-23T10:42:42.156 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.156 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.156 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.156 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.156 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.156 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.156 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.156 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.156 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.156 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.157 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.157 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.157 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:42.157 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:42.157 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.157 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.157 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.157 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.157 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.157 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.157 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.157 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.157 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.157 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.157 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.157 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.158 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.158 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.158 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.158 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.158 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.158 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.158 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.158 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:42.158 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:42.158 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.158 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.158 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.158 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.158 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.158 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.158 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.159 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.159 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.159 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.159 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.159 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.159 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:42.159 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:42.159 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.159 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.159 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.159 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:42.159 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.159 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.159 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.159 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.159 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.160 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.160 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.160 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.160 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.160 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.160 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.160 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.160 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.160 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.160 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.160 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:42.160 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:42.160 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.160 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.160 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.160 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.160 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.161 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.161 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.161 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.161 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.161 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.161 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.161 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.161 INFO:teuthology.orchestra.run.smithi097.stdout: "server_addr": { 2024-04-23T10:42:42.161 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_addr", 2024-04-23T10:42:42.161 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.161 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.161 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.161 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.161 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.161 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.161 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.162 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.162 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.162 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.162 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.162 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.162 INFO:teuthology.orchestra.run.smithi097.stdout: "server_port": { 2024-04-23T10:42:42.162 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_port", 2024-04-23T10:42:42.162 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.162 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.162 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.162 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.162 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.162 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.162 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.162 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.162 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.163 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.163 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.163 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.163 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.163 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.163 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.163 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rgw", 2024-04-23T10:42:42.163 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:42.163 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:42.163 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:42.163 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:42.163 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:42.163 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.163 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.163 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.163 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.164 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.164 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.164 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.164 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.164 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.164 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.164 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.164 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.164 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.164 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.164 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.164 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.164 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.164 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.164 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.164 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:42.165 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:42.165 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.165 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.165 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.165 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.165 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.165 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.165 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.165 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.165 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.165 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.165 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.165 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.165 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:42.165 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:42.165 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.166 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.166 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.166 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:42.166 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.166 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.166 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.166 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.166 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.166 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.166 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.166 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.166 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.166 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.166 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.166 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.166 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.167 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.167 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.167 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:42.167 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:42.167 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.167 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.167 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.167 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.167 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.167 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.167 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.167 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.167 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.167 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.167 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.168 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.168 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.168 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.168 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.168 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rook", 2024-04-23T10:42:42.168 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:42.168 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:42.168 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:42.168 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:42.168 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:42.168 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.168 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.168 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.168 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.168 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.168 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.169 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.169 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.169 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.169 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.169 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.169 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.169 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.169 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.169 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.169 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.169 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.169 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.169 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.169 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:42.169 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:42.170 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.170 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.170 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.170 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.170 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.170 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.170 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.170 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.170 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.170 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.170 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.170 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.170 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:42.170 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:42.170 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.171 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.171 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.171 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:42.171 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.171 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.171 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.171 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.171 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.171 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.171 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.171 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.171 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.171 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.171 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.171 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.171 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.172 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.172 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.172 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:42.172 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:42.172 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.172 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.172 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.172 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.172 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.172 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.172 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.172 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.172 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.172 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.172 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.172 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.172 INFO:teuthology.orchestra.run.smithi097.stdout: "storage_class": { 2024-04-23T10:42:42.173 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "storage_class", 2024-04-23T10:42:42.173 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.173 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.173 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.173 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "local", 2024-04-23T10:42:42.173 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.173 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.173 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.173 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-23T10:42:42.173 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.173 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.173 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.173 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.173 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.173 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.173 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.174 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "selftest", 2024-04-23T10:42:42.174 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:42.174 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:42.174 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:42.174 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:42.174 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:42.174 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.174 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.174 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.174 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.174 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.174 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.174 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.174 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.175 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.175 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.175 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.175 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.175 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.175 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.175 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.175 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.175 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.175 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.175 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.175 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:42.175 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:42.175 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.175 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.175 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.175 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.176 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.176 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.176 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.176 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.176 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.176 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.176 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.176 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.176 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:42.176 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:42.176 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.176 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.176 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.176 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:42.176 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.177 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.177 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.177 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.177 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.177 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.177 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.177 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.177 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.177 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.177 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.177 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.177 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.177 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.177 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.177 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:42.177 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:42.178 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.178 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.178 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.178 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.178 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.178 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.178 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.178 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.178 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.178 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.178 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.178 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.178 INFO:teuthology.orchestra.run.smithi097.stdout: "roption1": { 2024-04-23T10:42:42.178 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "roption1", 2024-04-23T10:42:42.178 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.179 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.179 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.179 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.179 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.179 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.179 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.179 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.179 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.179 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.179 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.179 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.179 INFO:teuthology.orchestra.run.smithi097.stdout: "roption2": { 2024-04-23T10:42:42.179 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "roption2", 2024-04-23T10:42:42.179 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.179 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.179 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.180 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "xyz", 2024-04-23T10:42:42.180 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.180 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.180 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.180 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.180 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.180 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.180 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.180 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.180 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption1": { 2024-04-23T10:42:42.180 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption1", 2024-04-23T10:42:42.180 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.180 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.180 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.180 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.180 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.180 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.181 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.181 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.181 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.181 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.181 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.181 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.181 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption2": { 2024-04-23T10:42:42.181 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption2", 2024-04-23T10:42:42.181 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:42.181 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.181 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.181 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.181 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.181 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.181 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.182 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.182 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.182 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.182 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.182 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.182 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption3": { 2024-04-23T10:42:42.182 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption3", 2024-04-23T10:42:42.182 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2024-04-23T10:42:42.182 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.182 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.182 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.182 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.182 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.182 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.183 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.183 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.183 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.183 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.183 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.183 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption4": { 2024-04-23T10:42:42.183 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption4", 2024-04-23T10:42:42.183 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.183 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.183 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.183 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.183 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.183 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.183 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.183 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.184 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.184 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.184 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.184 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.184 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption5": { 2024-04-23T10:42:42.184 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption5", 2024-04-23T10:42:42.184 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.184 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.184 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.184 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.184 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.184 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.184 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.184 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.184 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.184 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.185 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.185 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.185 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption6": { 2024-04-23T10:42:42.185 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption6", 2024-04-23T10:42:42.185 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.185 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.185 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.185 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:42.185 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.185 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.185 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.185 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.185 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.185 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.185 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.185 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.186 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption7": { 2024-04-23T10:42:42.186 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption7", 2024-04-23T10:42:42.186 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:42.186 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.186 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.186 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.186 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "1", 2024-04-23T10:42:42.186 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "42", 2024-04-23T10:42:42.186 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.186 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.186 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.186 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.186 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.186 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.187 INFO:teuthology.orchestra.run.smithi097.stdout: "testkey": { 2024-04-23T10:42:42.187 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "testkey", 2024-04-23T10:42:42.187 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.187 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.187 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.187 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.187 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.187 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.187 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.187 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.187 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.187 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.187 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.187 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.187 INFO:teuthology.orchestra.run.smithi097.stdout: "testlkey": { 2024-04-23T10:42:42.187 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "testlkey", 2024-04-23T10:42:42.187 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.188 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.188 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.188 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.188 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.188 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.188 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.188 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.188 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.188 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.188 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.188 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.188 INFO:teuthology.orchestra.run.smithi097.stdout: "testnewline": { 2024-04-23T10:42:42.188 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "testnewline", 2024-04-23T10:42:42.188 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.188 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.188 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.189 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.189 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.189 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.189 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.189 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.189 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.189 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.189 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.189 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.189 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.189 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.189 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.189 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "snap_schedule", 2024-04-23T10:42:42.189 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:42.189 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:42.190 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:42.190 INFO:teuthology.orchestra.run.smithi097.stdout: "allow_m_granularity": { 2024-04-23T10:42:42.190 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "allow_m_granularity", 2024-04-23T10:42:42.190 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.190 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.190 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.190 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.190 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.190 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.190 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.190 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "allow minute scheduled snapshots", 2024-04-23T10:42:42.190 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.190 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.190 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.190 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.190 INFO:teuthology.orchestra.run.smithi097.stdout: "dump_on_update": { 2024-04-23T10:42:42.191 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "dump_on_update", 2024-04-23T10:42:42.191 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.191 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.191 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.191 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.191 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.191 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.191 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.191 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "dump database to debug log on update", 2024-04-23T10:42:42.191 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.191 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.191 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.191 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.191 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:42.191 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:42.192 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.192 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.192 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.192 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.192 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.192 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.192 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.192 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.192 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.192 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.192 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.192 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.192 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.192 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.192 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.192 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.193 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.193 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.193 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.193 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:42.193 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:42.193 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.193 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.193 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.193 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.193 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.193 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.193 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.193 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.193 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.193 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.193 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.194 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.194 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:42.194 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:42.194 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.194 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.194 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.194 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:42.194 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.194 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.194 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.194 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.194 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.194 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.194 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.194 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.194 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.195 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.195 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.195 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.195 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.195 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.195 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.195 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:42.195 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:42.195 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.195 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.195 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.195 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.195 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.195 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.195 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.195 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.195 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.196 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.196 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.196 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.196 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.196 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.196 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.196 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "stats", 2024-04-23T10:42:42.196 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:42.196 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:42.196 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:42.196 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:42.196 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:42.196 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.196 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.196 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.196 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.197 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.197 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.197 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.197 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.197 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.197 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.197 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.197 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.197 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.197 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.197 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.197 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.197 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.197 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.197 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.197 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:42.198 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:42.198 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.198 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.198 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.198 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.198 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.198 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.198 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.198 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.198 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.198 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.198 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.198 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.198 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:42.198 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:42.199 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.199 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.199 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.199 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:42.199 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.199 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.199 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.199 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.199 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.199 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.199 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.199 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.199 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.199 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.199 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.199 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.199 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.200 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.200 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.200 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:42.200 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:42.200 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.200 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.200 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.200 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.200 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.200 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.200 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.200 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.200 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.200 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.200 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.200 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.201 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.201 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.201 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.201 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "status", 2024-04-23T10:42:42.201 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:42.201 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:42.201 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:42.201 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:42.201 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:42.201 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.201 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.201 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.201 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.201 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.201 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.202 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.202 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.202 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.202 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.202 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.202 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.202 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.202 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.202 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.202 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.202 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.202 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.202 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.202 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:42.202 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:42.202 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.203 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.203 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.203 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.203 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.203 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.203 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.203 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.203 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.203 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.203 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.203 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.203 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:42.203 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:42.203 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.203 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.203 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.204 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:42.204 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.204 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.204 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.204 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.204 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.204 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.204 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.204 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.204 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.204 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.204 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.204 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.204 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.204 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.205 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.205 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:42.205 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:42.205 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.205 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.205 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.205 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.205 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.205 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.205 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.205 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.205 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.205 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.205 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.205 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.206 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.206 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.206 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.206 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "telegraf", 2024-04-23T10:42:42.206 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:42.206 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:42.206 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:42.206 INFO:teuthology.orchestra.run.smithi097.stdout: "address": { 2024-04-23T10:42:42.206 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "address", 2024-04-23T10:42:42.206 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.206 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.206 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.206 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-23T10:42:42.206 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.206 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.206 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.207 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.207 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.207 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.207 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.207 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.207 INFO:teuthology.orchestra.run.smithi097.stdout: "interval": { 2024-04-23T10:42:42.207 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "interval", 2024-04-23T10:42:42.207 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-04-23T10:42:42.207 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.207 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.207 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "15", 2024-04-23T10:42:42.207 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.207 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.207 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.207 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.207 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.208 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.208 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.208 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.208 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:42.208 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:42.208 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.208 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.208 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.208 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.208 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.208 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.208 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.208 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.208 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.208 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.208 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.209 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.209 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.209 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.209 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.209 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.209 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.209 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.209 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.209 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:42.209 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:42.209 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.209 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.209 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.209 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.209 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.209 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.210 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.210 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.210 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.210 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.210 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.210 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.210 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:42.210 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:42.210 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.210 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.210 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.210 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:42.210 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.210 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.210 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.210 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.211 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.211 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.211 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.211 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.211 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.211 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.211 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.211 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.211 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.211 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.211 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.211 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:42.211 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:42.211 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.211 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.211 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.212 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.212 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.212 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.212 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.212 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.212 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.212 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.212 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.212 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.212 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.212 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.212 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.212 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "telemetry", 2024-04-23T10:42:42.212 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:42.212 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:42.213 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:42.213 INFO:teuthology.orchestra.run.smithi097.stdout: "channel_basic": { 2024-04-23T10:42:42.213 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "channel_basic", 2024-04-23T10:42:42.213 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.213 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.213 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.213 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:42.213 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.213 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.213 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.213 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-23T10:42:42.213 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.213 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.213 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.213 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.213 INFO:teuthology.orchestra.run.smithi097.stdout: "channel_crash": { 2024-04-23T10:42:42.213 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "channel_crash", 2024-04-23T10:42:42.214 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.214 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.214 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.214 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:42.214 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.214 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.214 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.214 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-23T10:42:42.214 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.214 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.214 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.214 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.214 INFO:teuthology.orchestra.run.smithi097.stdout: "channel_device": { 2024-04-23T10:42:42.214 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "channel_device", 2024-04-23T10:42:42.214 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.215 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.215 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.215 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:42.215 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.215 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.215 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.215 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-23T10:42:42.215 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.215 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.215 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.215 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.215 INFO:teuthology.orchestra.run.smithi097.stdout: "channel_ident": { 2024-04-23T10:42:42.215 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "channel_ident", 2024-04-23T10:42:42.215 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.215 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.216 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.216 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.216 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.216 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.216 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.216 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-23T10:42:42.216 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.216 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.216 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.216 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.216 INFO:teuthology.orchestra.run.smithi097.stdout: "channel_perf": { 2024-04-23T10:42:42.216 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "channel_perf", 2024-04-23T10:42:42.216 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.216 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.216 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.217 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.217 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.217 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.217 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.217 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-23T10:42:42.217 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.217 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.217 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.217 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.217 INFO:teuthology.orchestra.run.smithi097.stdout: "contact": { 2024-04-23T10:42:42.217 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "contact", 2024-04-23T10:42:42.217 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.217 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.217 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.217 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.217 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.217 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.218 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.218 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.218 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.218 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.218 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.218 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.218 INFO:teuthology.orchestra.run.smithi097.stdout: "description": { 2024-04-23T10:42:42.218 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "description", 2024-04-23T10:42:42.218 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.218 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.218 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.218 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.218 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.218 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.218 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.218 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.219 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.219 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.219 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.219 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.219 INFO:teuthology.orchestra.run.smithi097.stdout: "device_url": { 2024-04-23T10:42:42.219 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "device_url", 2024-04-23T10:42:42.219 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.219 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.219 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.219 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-23T10:42:42.219 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.219 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.219 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.219 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.219 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.219 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.220 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.220 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.220 INFO:teuthology.orchestra.run.smithi097.stdout: "enabled": { 2024-04-23T10:42:42.220 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "enabled", 2024-04-23T10:42:42.220 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.220 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.220 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.220 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.220 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.220 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.220 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.220 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.220 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.220 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.220 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.220 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.221 INFO:teuthology.orchestra.run.smithi097.stdout: "interval": { 2024-04-23T10:42:42.221 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "interval", 2024-04-23T10:42:42.221 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:42.221 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.221 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.221 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "24", 2024-04-23T10:42:42.221 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "8", 2024-04-23T10:42:42.221 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.221 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.221 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.221 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.221 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.221 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.221 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.221 INFO:teuthology.orchestra.run.smithi097.stdout: "last_opt_revision": { 2024-04-23T10:42:42.221 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "last_opt_revision", 2024-04-23T10:42:42.222 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:42.222 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.222 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.222 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "1", 2024-04-23T10:42:42.222 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.222 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.222 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.222 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.222 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.222 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.222 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.222 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.222 INFO:teuthology.orchestra.run.smithi097.stdout: "leaderboard": { 2024-04-23T10:42:42.222 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "leaderboard", 2024-04-23T10:42:42.222 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.223 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.223 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.223 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.223 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.223 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.223 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.223 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.223 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.223 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.223 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.223 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.223 INFO:teuthology.orchestra.run.smithi097.stdout: "leaderboard_description": { 2024-04-23T10:42:42.223 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "leaderboard_description", 2024-04-23T10:42:42.223 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.223 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.223 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.224 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.224 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.224 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.224 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.224 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.224 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.224 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.224 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.224 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.224 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:42.224 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:42.224 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.224 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.224 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.224 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.224 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.225 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.225 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.225 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.225 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.225 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.225 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.225 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.225 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.225 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.225 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.225 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.225 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.225 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.225 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.225 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:42.225 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:42.226 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.226 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.226 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.226 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.226 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.226 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.226 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.226 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.226 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.226 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.226 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.226 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.226 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:42.226 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:42.226 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.227 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.227 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.227 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:42.227 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.227 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.227 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.227 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.227 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.227 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.227 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.227 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.227 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.227 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.227 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.227 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.228 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.228 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.228 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.228 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:42.228 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:42.228 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.228 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.228 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.228 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.228 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.228 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.228 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.228 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.228 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.228 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.228 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.229 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.229 INFO:teuthology.orchestra.run.smithi097.stdout: "organization": { 2024-04-23T10:42:42.229 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "organization", 2024-04-23T10:42:42.229 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.229 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.229 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.229 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.229 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.229 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.229 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.229 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.229 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.229 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.229 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.229 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.229 INFO:teuthology.orchestra.run.smithi097.stdout: "proxy": { 2024-04-23T10:42:42.230 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "proxy", 2024-04-23T10:42:42.230 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.230 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.230 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.230 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.230 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.230 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.230 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.230 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.230 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.230 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.230 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.230 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.230 INFO:teuthology.orchestra.run.smithi097.stdout: "url": { 2024-04-23T10:42:42.230 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "url", 2024-04-23T10:42:42.231 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.231 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.231 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.231 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-23T10:42:42.231 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.231 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.231 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.231 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.231 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.231 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.231 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.231 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.231 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.231 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.231 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.232 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "test_orchestrator", 2024-04-23T10:42:42.232 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:42.232 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:42.232 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:42.232 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:42.232 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:42.232 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.232 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.232 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.232 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.232 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.232 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.232 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.232 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.232 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.232 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.233 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.233 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.233 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.233 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.233 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.233 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.233 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.233 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.233 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.233 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:42.233 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:42.233 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.233 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.233 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.233 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.233 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.234 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.234 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.234 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.234 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.234 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.234 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.234 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.234 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:42.234 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:42.234 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.234 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.234 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.234 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:42.234 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.235 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.235 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.235 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.235 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.235 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.235 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.235 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.235 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.235 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.235 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.235 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.235 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.235 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.235 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.235 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:42.235 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:42.235 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.236 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.236 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.236 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.236 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.236 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.236 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.236 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.236 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.236 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.236 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.236 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.236 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.236 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.236 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.236 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "volumes", 2024-04-23T10:42:42.236 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:42.237 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:42.237 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:42.237 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:42.237 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:42.237 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.237 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.237 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.237 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.237 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.237 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.237 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.237 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.237 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.237 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.237 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.237 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.237 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.238 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.238 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.238 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.238 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.238 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.238 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.238 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:42.238 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:42.238 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.238 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.238 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.238 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.238 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.238 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.239 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.239 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.239 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.239 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.239 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.239 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.239 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:42.239 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:42.239 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.239 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.239 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.239 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:42.239 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.239 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.239 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.239 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.240 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.240 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.240 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.240 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.240 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.240 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.240 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.240 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.240 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.240 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.240 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.240 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:42.240 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:42.240 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.240 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.240 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.241 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.241 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.241 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.241 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.241 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.241 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.241 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.241 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.241 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.241 INFO:teuthology.orchestra.run.smithi097.stdout: "max_concurrent_clones": { 2024-04-23T10:42:42.241 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "max_concurrent_clones", 2024-04-23T10:42:42.241 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:42.241 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.241 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.241 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "4", 2024-04-23T10:42:42.241 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.242 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.242 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.242 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-23T10:42:42.242 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.242 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.242 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.242 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.242 INFO:teuthology.orchestra.run.smithi097.stdout: "periodic_async_work": { 2024-04-23T10:42:42.242 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "periodic_async_work", 2024-04-23T10:42:42.242 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.242 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.242 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.242 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.242 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.242 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.242 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.243 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Periodically check for async work", 2024-04-23T10:42:42.243 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.243 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.243 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.243 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.243 INFO:teuthology.orchestra.run.smithi097.stdout: "snapshot_clone_delay": { 2024-04-23T10:42:42.243 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "snapshot_clone_delay", 2024-04-23T10:42:42.243 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:42.243 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.243 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.243 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0", 2024-04-23T10:42:42.243 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.243 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.243 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.243 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-23T10:42:42.244 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.244 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.244 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.244 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.244 INFO:teuthology.orchestra.run.smithi097.stdout: "snapshot_clone_no_wait": { 2024-04-23T10:42:42.244 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "snapshot_clone_no_wait", 2024-04-23T10:42:42.244 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.244 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.244 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.244 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:42.244 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.244 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.244 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.244 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-23T10:42:42.244 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.244 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.245 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.245 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.245 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.245 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.245 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.245 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "zabbix", 2024-04-23T10:42:42.245 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:42.245 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:42.245 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:42.245 INFO:teuthology.orchestra.run.smithi097.stdout: "discovery_interval": { 2024-04-23T10:42:42.245 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "discovery_interval", 2024-04-23T10:42:42.245 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "uint", 2024-04-23T10:42:42.245 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.245 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.245 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "100", 2024-04-23T10:42:42.245 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.246 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.246 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.246 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.246 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.246 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.246 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.246 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.246 INFO:teuthology.orchestra.run.smithi097.stdout: "identifier": { 2024-04-23T10:42:42.246 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "identifier", 2024-04-23T10:42:42.246 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.246 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.246 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.246 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.246 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.246 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.246 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.247 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.247 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.247 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.247 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.247 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.247 INFO:teuthology.orchestra.run.smithi097.stdout: "interval": { 2024-04-23T10:42:42.247 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "interval", 2024-04-23T10:42:42.247 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-04-23T10:42:42.247 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.247 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.247 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "60", 2024-04-23T10:42:42.247 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.247 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.247 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.247 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.247 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.248 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.248 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.248 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.248 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:42.248 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:42.248 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.248 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.248 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.248 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.248 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.248 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.248 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.248 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.248 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.248 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.248 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.249 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.249 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.249 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.249 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.249 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.249 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.249 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.249 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.249 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:42.249 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:42.249 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.249 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.249 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.249 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.249 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.249 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.250 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.250 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.250 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.250 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.250 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.250 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.250 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:42.250 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:42.250 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.250 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.250 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.250 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:42.250 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.250 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.250 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.251 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.251 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.251 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.251 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.251 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.251 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.251 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.251 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.251 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.251 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.251 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.251 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.251 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:42.251 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:42.251 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.252 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.252 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.252 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.252 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.252 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.252 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.252 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.252 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.252 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.252 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.252 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.252 INFO:teuthology.orchestra.run.smithi097.stdout: "zabbix_host": { 2024-04-23T10:42:42.252 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "zabbix_host", 2024-04-23T10:42:42.252 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.252 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.253 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.253 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.253 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.253 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.253 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.253 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.253 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.253 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.253 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.253 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.253 INFO:teuthology.orchestra.run.smithi097.stdout: "zabbix_port": { 2024-04-23T10:42:42.253 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "zabbix_port", 2024-04-23T10:42:42.253 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:42.253 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.253 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.253 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10051", 2024-04-23T10:42:42.254 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.254 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.254 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.254 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.254 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.254 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.254 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.254 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.254 INFO:teuthology.orchestra.run.smithi097.stdout: "zabbix_sender": { 2024-04-23T10:42:42.254 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "zabbix_sender", 2024-04-23T10:42:42.254 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.254 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.254 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.254 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-23T10:42:42.254 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.255 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.255 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.255 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.255 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.255 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.255 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.255 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.255 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.255 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.255 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-04-23T10:42:42.255 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.255 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.255 INFO:teuthology.orchestra.run.smithi097.stdout: "modules": [ 2024-04-23T10:42:42.255 INFO:teuthology.orchestra.run.smithi097.stdout: "cephadm", 2024-04-23T10:42:42.255 INFO:teuthology.orchestra.run.smithi097.stdout: "dashboard", 2024-04-23T10:42:42.255 INFO:teuthology.orchestra.run.smithi097.stdout: "iostat", 2024-04-23T10:42:42.256 INFO:teuthology.orchestra.run.smithi097.stdout: "nfs", 2024-04-23T10:42:42.256 INFO:teuthology.orchestra.run.smithi097.stdout: "restful", 2024-04-23T10:42:42.256 INFO:teuthology.orchestra.run.smithi097.stdout: "snap_schedule" 2024-04-23T10:42:42.256 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.256 INFO:teuthology.orchestra.run.smithi097.stdout: "available_modules": [ 2024-04-23T10:42:42.256 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.256 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "alerts", 2024-04-23T10:42:42.256 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:42.256 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:42.256 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:42.256 INFO:teuthology.orchestra.run.smithi097.stdout: "interval": { 2024-04-23T10:42:42.256 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "interval", 2024-04-23T10:42:42.256 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-04-23T10:42:42.256 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.256 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.256 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "60", 2024-04-23T10:42:42.257 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.257 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.257 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.257 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "How frequently to reexamine health status", 2024-04-23T10:42:42.257 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.257 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.257 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.257 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.257 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:42.257 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:42.257 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.257 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.257 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.257 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.257 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.257 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.258 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.258 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.258 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.258 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.258 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.258 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.258 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.258 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.258 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.258 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.258 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.258 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.258 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.258 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:42.258 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:42.259 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.259 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.259 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.259 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.259 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.259 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.259 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.259 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.259 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.259 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.259 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.259 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.259 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:42.259 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:42.259 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.259 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.259 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.260 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:42.260 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.260 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.260 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.260 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.260 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.260 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.260 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.260 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.260 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.260 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.260 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.260 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.260 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.260 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.260 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.261 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:42.261 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:42.261 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.261 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.261 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.261 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.261 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.261 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.261 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.261 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.261 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.261 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.261 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.261 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.261 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_destination": { 2024-04-23T10:42:42.261 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_destination", 2024-04-23T10:42:42.262 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.262 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.262 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.262 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.262 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.262 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.262 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.262 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Email address to send alerts to", 2024-04-23T10:42:42.262 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.262 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.262 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.262 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.262 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_from_name": { 2024-04-23T10:42:42.262 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_from_name", 2024-04-23T10:42:42.262 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.262 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.263 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.263 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "Ceph", 2024-04-23T10:42:42.263 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.263 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.263 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.263 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Email From: name", 2024-04-23T10:42:42.263 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.263 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.263 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.263 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.263 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_host": { 2024-04-23T10:42:42.263 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_host", 2024-04-23T10:42:42.263 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.263 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.263 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.263 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.264 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.264 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.264 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.264 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "SMTP server", 2024-04-23T10:42:42.264 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.264 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.264 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.264 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.264 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_password": { 2024-04-23T10:42:42.264 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_password", 2024-04-23T10:42:42.264 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.264 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.264 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.264 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.264 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.264 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.265 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.265 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Password to authenticate with", 2024-04-23T10:42:42.265 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.265 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.265 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.265 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.265 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_port": { 2024-04-23T10:42:42.265 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_port", 2024-04-23T10:42:42.265 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:42.265 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.265 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.265 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "465", 2024-04-23T10:42:42.265 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.265 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.265 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.266 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "SMTP port", 2024-04-23T10:42:42.266 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.266 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.266 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.266 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.266 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_sender": { 2024-04-23T10:42:42.266 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_sender", 2024-04-23T10:42:42.266 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.266 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.266 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.266 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.266 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.266 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.266 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.266 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "SMTP envelope sender", 2024-04-23T10:42:42.267 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.267 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.267 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.267 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.267 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_ssl": { 2024-04-23T10:42:42.267 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_ssl", 2024-04-23T10:42:42.267 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.267 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.267 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.267 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:42.267 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.267 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.267 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.267 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-23T10:42:42.267 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.267 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.268 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.268 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.268 INFO:teuthology.orchestra.run.smithi097.stdout: "smtp_user": { 2024-04-23T10:42:42.268 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "smtp_user", 2024-04-23T10:42:42.268 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.268 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.268 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.268 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.268 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.268 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.268 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.268 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "User to authenticate as", 2024-04-23T10:42:42.268 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.268 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.268 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.269 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.269 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.269 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.269 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.269 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "balancer", 2024-04-23T10:42:42.269 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:42.269 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:42.269 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:42.269 INFO:teuthology.orchestra.run.smithi097.stdout: "active": { 2024-04-23T10:42:42.269 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "active", 2024-04-23T10:42:42.269 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.269 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.269 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.269 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:42.269 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.269 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.270 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.270 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "automatically balance PGs across cluster", 2024-04-23T10:42:42.270 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.270 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.270 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.270 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.270 INFO:teuthology.orchestra.run.smithi097.stdout: "begin_time": { 2024-04-23T10:42:42.270 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "begin_time", 2024-04-23T10:42:42.270 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.270 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.270 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.270 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0000", 2024-04-23T10:42:42.270 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.270 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.270 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.271 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "beginning time of day to automatically balance", 2024-04-23T10:42:42.271 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-23T10:42:42.271 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.271 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.271 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.271 INFO:teuthology.orchestra.run.smithi097.stdout: "begin_weekday": { 2024-04-23T10:42:42.271 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "begin_weekday", 2024-04-23T10:42:42.271 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "uint", 2024-04-23T10:42:42.271 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.271 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.271 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0", 2024-04-23T10:42:42.271 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "0", 2024-04-23T10:42:42.271 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "6", 2024-04-23T10:42:42.271 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.271 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-23T10:42:42.272 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-23T10:42:42.272 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.272 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.272 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.272 INFO:teuthology.orchestra.run.smithi097.stdout: "crush_compat_max_iterations": { 2024-04-23T10:42:42.272 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "crush_compat_max_iterations", 2024-04-23T10:42:42.272 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "uint", 2024-04-23T10:42:42.272 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.272 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.272 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "25", 2024-04-23T10:42:42.272 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "1", 2024-04-23T10:42:42.272 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "250", 2024-04-23T10:42:42.272 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.272 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-23T10:42:42.272 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.273 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.273 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.273 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.273 INFO:teuthology.orchestra.run.smithi097.stdout: "crush_compat_metrics": { 2024-04-23T10:42:42.273 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "crush_compat_metrics", 2024-04-23T10:42:42.273 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.273 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.273 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.273 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "pgs,objects,bytes", 2024-04-23T10:42:42.273 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.273 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.273 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.273 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-23T10:42:42.273 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-23T10:42:42.273 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.273 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.274 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.274 INFO:teuthology.orchestra.run.smithi097.stdout: "crush_compat_step": { 2024-04-23T10:42:42.274 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "crush_compat_step", 2024-04-23T10:42:42.274 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2024-04-23T10:42:42.274 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.274 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.274 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0.5", 2024-04-23T10:42:42.274 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "0.001", 2024-04-23T10:42:42.274 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "0.999", 2024-04-23T10:42:42.274 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.274 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "aggressiveness of optimization", 2024-04-23T10:42:42.274 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-23T10:42:42.274 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.274 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.275 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.275 INFO:teuthology.orchestra.run.smithi097.stdout: "end_time": { 2024-04-23T10:42:42.275 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "end_time", 2024-04-23T10:42:42.275 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.275 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.275 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.275 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "2359", 2024-04-23T10:42:42.275 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.275 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.275 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.275 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "ending time of day to automatically balance", 2024-04-23T10:42:42.275 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-23T10:42:42.275 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.275 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.275 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.275 INFO:teuthology.orchestra.run.smithi097.stdout: "end_weekday": { 2024-04-23T10:42:42.276 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "end_weekday", 2024-04-23T10:42:42.276 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "uint", 2024-04-23T10:42:42.276 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.276 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.276 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0", 2024-04-23T10:42:42.276 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "0", 2024-04-23T10:42:42.276 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "6", 2024-04-23T10:42:42.276 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.276 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-23T10:42:42.276 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-23T10:42:42.276 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.276 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.276 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.276 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:42.276 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:42.277 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.277 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.277 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.277 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.277 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.277 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.277 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.277 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.277 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.277 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.277 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.277 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.277 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.277 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.277 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.277 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.277 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.278 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.278 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.278 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:42.278 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:42.278 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.278 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.278 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.278 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.278 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.278 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.278 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.278 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.278 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.278 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.278 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.278 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.279 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:42.279 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:42.279 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.279 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.279 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.279 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:42.279 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.279 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.279 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.279 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.279 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.279 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.279 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.279 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.279 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.280 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.280 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.280 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.280 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.280 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.280 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.280 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:42.280 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:42.280 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.280 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.280 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.280 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.280 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.280 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.280 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.281 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.281 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.281 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.281 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.281 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.281 INFO:teuthology.orchestra.run.smithi097.stdout: "min_score": { 2024-04-23T10:42:42.281 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "min_score", 2024-04-23T10:42:42.281 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2024-04-23T10:42:42.281 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.281 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.281 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0", 2024-04-23T10:42:42.281 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.281 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.281 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.281 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-23T10:42:42.281 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.282 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.282 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.282 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.282 INFO:teuthology.orchestra.run.smithi097.stdout: "mode": { 2024-04-23T10:42:42.282 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mode", 2024-04-23T10:42:42.282 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.282 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.282 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.282 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "upmap", 2024-04-23T10:42:42.282 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.282 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.282 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.282 INFO:teuthology.orchestra.run.smithi097.stdout: "crush-compat", 2024-04-23T10:42:42.282 INFO:teuthology.orchestra.run.smithi097.stdout: "none", 2024-04-23T10:42:42.282 INFO:teuthology.orchestra.run.smithi097.stdout: "read", 2024-04-23T10:42:42.283 INFO:teuthology.orchestra.run.smithi097.stdout: "upmap", 2024-04-23T10:42:42.283 INFO:teuthology.orchestra.run.smithi097.stdout: "upmap-read" 2024-04-23T10:42:42.283 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.283 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Balancer mode", 2024-04-23T10:42:42.283 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.283 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.283 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.283 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.283 INFO:teuthology.orchestra.run.smithi097.stdout: "pool_ids": { 2024-04-23T10:42:42.283 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "pool_ids", 2024-04-23T10:42:42.283 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.283 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.283 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.283 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.283 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.283 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.284 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.284 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-23T10:42:42.284 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.284 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.284 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.284 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.284 INFO:teuthology.orchestra.run.smithi097.stdout: "sleep_interval": { 2024-04-23T10:42:42.284 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "sleep_interval", 2024-04-23T10:42:42.284 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-04-23T10:42:42.284 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.284 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.284 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "60", 2024-04-23T10:42:42.284 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.284 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.284 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.285 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-23T10:42:42.285 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.285 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.285 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.285 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.285 INFO:teuthology.orchestra.run.smithi097.stdout: "upmap_max_deviation": { 2024-04-23T10:42:42.285 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "upmap_max_deviation", 2024-04-23T10:42:42.285 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:42.285 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.285 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.285 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "5", 2024-04-23T10:42:42.285 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "1", 2024-04-23T10:42:42.285 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.285 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.285 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-23T10:42:42.285 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-23T10:42:42.286 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.286 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.286 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.286 INFO:teuthology.orchestra.run.smithi097.stdout: "upmap_max_optimizations": { 2024-04-23T10:42:42.286 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "upmap_max_optimizations", 2024-04-23T10:42:42.286 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "uint", 2024-04-23T10:42:42.286 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.286 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.286 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-04-23T10:42:42.286 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.286 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.286 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.286 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-23T10:42:42.286 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.286 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.286 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.287 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.287 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.287 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.287 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.287 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "cephadm", 2024-04-23T10:42:42.287 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:42.287 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:42.287 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:42.287 INFO:teuthology.orchestra.run.smithi097.stdout: "agent_down_multiplier": { 2024-04-23T10:42:42.287 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "agent_down_multiplier", 2024-04-23T10:42:42.287 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2024-04-23T10:42:42.287 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.287 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.287 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "3.0", 2024-04-23T10:42:42.287 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.287 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.288 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.288 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-23T10:42:42.288 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.288 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.288 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.288 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.288 INFO:teuthology.orchestra.run.smithi097.stdout: "agent_refresh_rate": { 2024-04-23T10:42:42.288 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "agent_refresh_rate", 2024-04-23T10:42:42.288 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-04-23T10:42:42.288 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.288 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.288 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "20", 2024-04-23T10:42:42.288 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.288 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.288 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.288 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-23T10:42:42.289 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.289 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.289 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.289 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.289 INFO:teuthology.orchestra.run.smithi097.stdout: "agent_starting_port": { 2024-04-23T10:42:42.289 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "agent_starting_port", 2024-04-23T10:42:42.289 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:42.289 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.289 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.289 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "4721", 2024-04-23T10:42:42.289 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.289 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.289 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.289 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-23T10:42:42.289 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.289 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.289 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.290 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.290 INFO:teuthology.orchestra.run.smithi097.stdout: "allow_ptrace": { 2024-04-23T10:42:42.290 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "allow_ptrace", 2024-04-23T10:42:42.290 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.290 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.290 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.290 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.290 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.290 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.290 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.290 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-23T10:42:42.290 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-23T10:42:42.290 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.290 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.290 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.290 INFO:teuthology.orchestra.run.smithi097.stdout: "autotune_interval": { 2024-04-23T10:42:42.291 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "autotune_interval", 2024-04-23T10:42:42.291 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-04-23T10:42:42.291 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.291 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.291 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "600", 2024-04-23T10:42:42.291 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.291 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.291 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.291 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-23T10:42:42.291 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.291 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.291 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.291 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.291 INFO:teuthology.orchestra.run.smithi097.stdout: "autotune_memory_target_ratio": { 2024-04-23T10:42:42.291 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "autotune_memory_target_ratio", 2024-04-23T10:42:42.291 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2024-04-23T10:42:42.292 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.292 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.292 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0.7", 2024-04-23T10:42:42.292 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.292 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.292 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.292 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-23T10:42:42.292 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.292 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.292 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.292 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.292 INFO:teuthology.orchestra.run.smithi097.stdout: "cephadm_log_destination": { 2024-04-23T10:42:42.292 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "cephadm_log_destination", 2024-04-23T10:42:42.292 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.292 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.293 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.293 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.293 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.293 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.293 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.293 INFO:teuthology.orchestra.run.smithi097.stdout: "file", 2024-04-23T10:42:42.293 INFO:teuthology.orchestra.run.smithi097.stdout: "file,syslog", 2024-04-23T10:42:42.293 INFO:teuthology.orchestra.run.smithi097.stdout: "syslog" 2024-04-23T10:42:42.293 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.293 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-23T10:42:42.293 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.293 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.293 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.293 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.293 INFO:teuthology.orchestra.run.smithi097.stdout: "cgroups_split": { 2024-04-23T10:42:42.293 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "cgroups_split", 2024-04-23T10:42:42.294 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.294 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.294 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.294 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:42.294 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.294 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.294 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.294 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-23T10:42:42.294 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.294 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.294 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.294 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.294 INFO:teuthology.orchestra.run.smithi097.stdout: "config_checks_enabled": { 2024-04-23T10:42:42.294 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "config_checks_enabled", 2024-04-23T10:42:42.294 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.295 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.295 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.295 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.295 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.295 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.295 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.295 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-23T10:42:42.295 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.295 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.295 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.295 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.295 INFO:teuthology.orchestra.run.smithi097.stdout: "config_dashboard": { 2024-04-23T10:42:42.295 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "config_dashboard", 2024-04-23T10:42:42.295 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.295 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.295 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.296 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:42.296 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.296 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.296 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.296 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-23T10:42:42.296 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.296 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.296 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.296 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.296 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_alertmanager": { 2024-04-23T10:42:42.296 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_alertmanager", 2024-04-23T10:42:42.296 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.296 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.296 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.296 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-23T10:42:42.296 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.297 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.297 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.297 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Prometheus container image", 2024-04-23T10:42:42.297 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.297 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.297 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.297 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.297 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_base": { 2024-04-23T10:42:42.297 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_base", 2024-04-23T10:42:42.297 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.297 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.297 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.297 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-23T10:42:42.297 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.297 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.297 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.297 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Container image name, without the tag", 2024-04-23T10:42:42.298 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.298 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.298 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.298 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.298 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_elasticsearch": { 2024-04-23T10:42:42.298 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_elasticsearch", 2024-04-23T10:42:42.298 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.298 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.298 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.298 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-23T10:42:42.298 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.298 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.298 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.298 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "elasticsearch container image", 2024-04-23T10:42:42.298 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.298 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.299 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.299 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.299 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_grafana": { 2024-04-23T10:42:42.299 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_grafana", 2024-04-23T10:42:42.299 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.299 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.299 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.299 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-23T10:42:42.299 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.299 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.299 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.299 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Prometheus container image", 2024-04-23T10:42:42.299 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.299 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.299 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.299 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.300 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_haproxy": { 2024-04-23T10:42:42.300 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_haproxy", 2024-04-23T10:42:42.300 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.300 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.300 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.300 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-23T10:42:42.300 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.300 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.300 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.300 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "HAproxy container image", 2024-04-23T10:42:42.300 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.300 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.300 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.300 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.301 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_jaeger_agent": { 2024-04-23T10:42:42.301 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_jaeger_agent", 2024-04-23T10:42:42.301 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.301 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.301 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.301 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-23T10:42:42.301 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.301 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.301 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.301 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Jaeger agent container image", 2024-04-23T10:42:42.301 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.301 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.301 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.301 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.301 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_jaeger_collector": { 2024-04-23T10:42:42.301 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_jaeger_collector", 2024-04-23T10:42:42.302 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.302 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.302 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.302 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-23T10:42:42.302 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.302 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.302 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.302 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Jaeger collector container image", 2024-04-23T10:42:42.302 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.302 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.302 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.302 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.302 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_jaeger_query": { 2024-04-23T10:42:42.302 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_jaeger_query", 2024-04-23T10:42:42.302 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.303 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.303 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.303 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-23T10:42:42.303 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.303 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.303 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.303 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Jaeger query container image", 2024-04-23T10:42:42.303 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.303 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.303 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.303 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.303 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_keepalived": { 2024-04-23T10:42:42.303 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_keepalived", 2024-04-23T10:42:42.303 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.303 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.303 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.304 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-23T10:42:42.304 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.304 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.304 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.304 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Keepalived container image", 2024-04-23T10:42:42.304 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.304 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.304 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.304 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.304 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_loki": { 2024-04-23T10:42:42.304 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_loki", 2024-04-23T10:42:42.304 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.304 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.304 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.304 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-23T10:42:42.304 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.305 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.305 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.305 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Loki container image", 2024-04-23T10:42:42.305 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.305 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.305 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.305 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.305 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_node_exporter": { 2024-04-23T10:42:42.305 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_node_exporter", 2024-04-23T10:42:42.305 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.305 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.305 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.305 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-23T10:42:42.305 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.306 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.306 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.306 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Prometheus container image", 2024-04-23T10:42:42.306 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.306 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.306 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.306 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.306 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_nvmeof": { 2024-04-23T10:42:42.306 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_nvmeof", 2024-04-23T10:42:42.306 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.306 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.306 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.306 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-23T10:42:42.306 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.306 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.307 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.307 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Nvme-of container image", 2024-04-23T10:42:42.307 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.307 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.307 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.307 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.307 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_prometheus": { 2024-04-23T10:42:42.307 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_prometheus", 2024-04-23T10:42:42.307 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.307 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.307 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.307 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-23T10:42:42.307 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.307 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.307 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.307 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Prometheus container image", 2024-04-23T10:42:42.308 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.308 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.308 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.308 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.308 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_promtail": { 2024-04-23T10:42:42.308 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_promtail", 2024-04-23T10:42:42.308 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.308 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.308 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.308 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-23T10:42:42.308 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.308 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.308 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.308 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Promtail container image", 2024-04-23T10:42:42.308 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.309 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.309 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.309 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.309 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_samba": { 2024-04-23T10:42:42.309 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_samba", 2024-04-23T10:42:42.309 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.309 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.309 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.309 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-23T10:42:42.309 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.309 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.309 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.309 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Samba/SMB container image", 2024-04-23T10:42:42.309 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.309 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.309 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.310 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.310 INFO:teuthology.orchestra.run.smithi097.stdout: "container_image_snmp_gateway": { 2024-04-23T10:42:42.310 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_image_snmp_gateway", 2024-04-23T10:42:42.310 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.310 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.310 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.310 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-23T10:42:42.310 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.310 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.310 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.310 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "SNMP Gateway container image", 2024-04-23T10:42:42.310 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.310 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.310 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.310 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.310 INFO:teuthology.orchestra.run.smithi097.stdout: "container_init": { 2024-04-23T10:42:42.311 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "container_init", 2024-04-23T10:42:42.311 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.311 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.311 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.311 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:42.311 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.311 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.311 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.311 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Run podman/docker with `--init`", 2024-04-23T10:42:42.311 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.311 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.311 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.311 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.311 INFO:teuthology.orchestra.run.smithi097.stdout: "daemon_cache_timeout": { 2024-04-23T10:42:42.311 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "daemon_cache_timeout", 2024-04-23T10:42:42.311 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-04-23T10:42:42.312 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.312 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.312 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "600", 2024-04-23T10:42:42.312 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.312 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.312 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.312 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-23T10:42:42.312 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.312 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.312 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.312 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.312 INFO:teuthology.orchestra.run.smithi097.stdout: "default_cephadm_command_timeout": { 2024-04-23T10:42:42.312 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "default_cephadm_command_timeout", 2024-04-23T10:42:42.312 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:42.312 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.313 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.313 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "900", 2024-04-23T10:42:42.313 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.313 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.313 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.313 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-23T10:42:42.313 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.313 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.313 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.313 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.313 INFO:teuthology.orchestra.run.smithi097.stdout: "default_registry": { 2024-04-23T10:42:42.313 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "default_registry", 2024-04-23T10:42:42.313 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.313 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.313 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.313 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "docker.io", 2024-04-23T10:42:42.313 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.314 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.314 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.314 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-23T10:42:42.314 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.314 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.314 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.314 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.314 INFO:teuthology.orchestra.run.smithi097.stdout: "device_cache_timeout": { 2024-04-23T10:42:42.314 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "device_cache_timeout", 2024-04-23T10:42:42.314 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-04-23T10:42:42.314 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.314 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.314 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "1800", 2024-04-23T10:42:42.314 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.314 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.315 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.315 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "seconds to cache device inventory", 2024-04-23T10:42:42.315 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.315 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.315 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.315 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.315 INFO:teuthology.orchestra.run.smithi097.stdout: "device_enhanced_scan": { 2024-04-23T10:42:42.315 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "device_enhanced_scan", 2024-04-23T10:42:42.315 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.315 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.315 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.315 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.315 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.315 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.315 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.315 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-23T10:42:42.315 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.316 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.316 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.316 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.316 INFO:teuthology.orchestra.run.smithi097.stdout: "facts_cache_timeout": { 2024-04-23T10:42:42.316 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "facts_cache_timeout", 2024-04-23T10:42:42.316 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-04-23T10:42:42.316 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.316 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.316 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "60", 2024-04-23T10:42:42.316 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.316 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.316 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.316 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "seconds to cache host facts data", 2024-04-23T10:42:42.316 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.316 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.316 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.317 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.317 INFO:teuthology.orchestra.run.smithi097.stdout: "grafana_dashboards_path": { 2024-04-23T10:42:42.317 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "grafana_dashboards_path", 2024-04-23T10:42:42.317 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.317 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.317 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.317 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-23T10:42:42.317 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.317 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.317 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.317 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-23T10:42:42.317 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.317 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.317 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.317 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.317 INFO:teuthology.orchestra.run.smithi097.stdout: "host_check_interval": { 2024-04-23T10:42:42.317 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "host_check_interval", 2024-04-23T10:42:42.318 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-04-23T10:42:42.318 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.318 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.318 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "600", 2024-04-23T10:42:42.318 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.318 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.318 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.318 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how frequently to perform a host check", 2024-04-23T10:42:42.318 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.318 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.318 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.318 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.318 INFO:teuthology.orchestra.run.smithi097.stdout: "hw_monitoring": { 2024-04-23T10:42:42.318 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "hw_monitoring", 2024-04-23T10:42:42.318 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.319 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.319 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.319 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.319 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.319 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.319 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.319 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-23T10:42:42.319 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.319 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.319 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.319 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.319 INFO:teuthology.orchestra.run.smithi097.stdout: "inventory_list_all": { 2024-04-23T10:42:42.319 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "inventory_list_all", 2024-04-23T10:42:42.319 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.319 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.319 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.320 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.320 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.320 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.320 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.320 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-23T10:42:42.320 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.320 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.320 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.320 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.320 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:42.320 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:42.320 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.320 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.320 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.320 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.320 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.320 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.321 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.321 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.321 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.321 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.321 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.321 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.321 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.321 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.321 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.321 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.321 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.321 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.321 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.321 INFO:teuthology.orchestra.run.smithi097.stdout: "log_refresh_metadata": { 2024-04-23T10:42:42.321 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_refresh_metadata", 2024-04-23T10:42:42.321 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.322 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.322 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.322 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.322 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.322 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.322 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.322 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-23T10:42:42.322 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.322 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.322 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.322 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.322 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:42.322 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:42.322 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.323 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.323 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.323 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:42.323 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.323 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.323 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.323 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-23T10:42:42.323 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.323 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.323 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.323 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:42.323 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:42.323 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.323 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.323 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.324 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:42.324 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.324 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.324 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.324 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.324 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.324 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.324 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.324 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.324 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.325 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.325 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.325 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.325 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.325 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.325 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.325 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:42.325 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:42.325 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.325 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.325 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.325 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.325 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.326 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.326 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.326 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.326 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.326 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.326 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.326 INFO:teuthology.orchestra.run.smithi097.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-23T10:42:42.326 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-23T10:42:42.326 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.326 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.326 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.326 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.326 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.326 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.326 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.327 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-23T10:42:42.327 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.327 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.327 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.327 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.327 INFO:teuthology.orchestra.run.smithi097.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-23T10:42:42.327 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-23T10:42:42.327 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.327 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.327 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.327 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "*", 2024-04-23T10:42:42.327 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.327 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.327 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.327 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-23T10:42:42.327 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.328 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.328 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.328 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.328 INFO:teuthology.orchestra.run.smithi097.stdout: "max_count_per_host": { 2024-04-23T10:42:42.328 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "max_count_per_host", 2024-04-23T10:42:42.328 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:42.328 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.328 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.328 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-04-23T10:42:42.328 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.328 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.328 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.328 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "max number of daemons per service per host", 2024-04-23T10:42:42.328 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.328 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.328 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.329 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.329 INFO:teuthology.orchestra.run.smithi097.stdout: "max_osd_draining_count": { 2024-04-23T10:42:42.329 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "max_osd_draining_count", 2024-04-23T10:42:42.329 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:42.329 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.329 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.329 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-04-23T10:42:42.329 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.329 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.329 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.329 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-23T10:42:42.329 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.329 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.329 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.329 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.329 INFO:teuthology.orchestra.run.smithi097.stdout: "migration_current": { 2024-04-23T10:42:42.330 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "migration_current", 2024-04-23T10:42:42.330 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:42.330 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.330 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.330 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.330 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.330 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.330 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.330 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "internal - do not modify", 2024-04-23T10:42:42.330 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.330 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.330 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.330 INFO:teuthology.orchestra.run.smithi097.stdout: "mode": { 2024-04-23T10:42:42.330 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mode", 2024-04-23T10:42:42.331 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.331 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.331 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.331 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "root", 2024-04-23T10:42:42.331 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.331 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.331 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.331 INFO:teuthology.orchestra.run.smithi097.stdout: "cephadm-package", 2024-04-23T10:42:42.331 INFO:teuthology.orchestra.run.smithi097.stdout: "root" 2024-04-23T10:42:42.331 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.331 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "mode for remote execution of cephadm", 2024-04-23T10:42:42.331 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.331 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.331 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.331 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.332 INFO:teuthology.orchestra.run.smithi097.stdout: "oob_default_addr": { 2024-04-23T10:42:42.332 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "oob_default_addr", 2024-04-23T10:42:42.332 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.332 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.332 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.332 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "169.254.1.1", 2024-04-23T10:42:42.332 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.332 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.332 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.332 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-23T10:42:42.332 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.332 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.332 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.332 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.332 INFO:teuthology.orchestra.run.smithi097.stdout: "prometheus_alerts_path": { 2024-04-23T10:42:42.333 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "prometheus_alerts_path", 2024-04-23T10:42:42.333 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.333 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.333 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.333 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-23T10:42:42.333 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.333 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.333 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.333 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-23T10:42:42.333 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.333 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.333 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.333 INFO:teuthology.orchestra.run.smithi097.stdout: "registry_insecure": { 2024-04-23T10:42:42.333 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "registry_insecure", 2024-04-23T10:42:42.333 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.334 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.334 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.334 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.334 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.334 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.334 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.334 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-23T10:42:42.334 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.334 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.334 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.334 INFO:teuthology.orchestra.run.smithi097.stdout: "registry_password": { 2024-04-23T10:42:42.334 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "registry_password", 2024-04-23T10:42:42.334 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.334 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.335 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.335 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.335 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.335 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.335 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.335 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-23T10:42:42.335 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.335 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.335 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.335 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.335 INFO:teuthology.orchestra.run.smithi097.stdout: "registry_url": { 2024-04-23T10:42:42.335 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "registry_url", 2024-04-23T10:42:42.335 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.335 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.335 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.335 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.336 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.336 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.336 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.336 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-23T10:42:42.336 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.336 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.336 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.336 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.336 INFO:teuthology.orchestra.run.smithi097.stdout: "registry_username": { 2024-04-23T10:42:42.336 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "registry_username", 2024-04-23T10:42:42.336 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.336 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.336 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.336 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.336 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.336 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.336 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.337 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-23T10:42:42.337 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.337 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.337 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.337 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.337 INFO:teuthology.orchestra.run.smithi097.stdout: "secure_monitoring_stack": { 2024-04-23T10:42:42.337 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "secure_monitoring_stack", 2024-04-23T10:42:42.337 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.337 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.337 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.337 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.337 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.337 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.337 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.337 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-23T10:42:42.338 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.338 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.338 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.338 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.338 INFO:teuthology.orchestra.run.smithi097.stdout: "service_discovery_port": { 2024-04-23T10:42:42.338 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "service_discovery_port", 2024-04-23T10:42:42.338 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:42.338 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.338 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.338 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "8765", 2024-04-23T10:42:42.338 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.338 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.338 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.338 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "cephadm service discovery port", 2024-04-23T10:42:42.338 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.339 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.339 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.339 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.339 INFO:teuthology.orchestra.run.smithi097.stdout: "ssh_config_file": { 2024-04-23T10:42:42.339 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ssh_config_file", 2024-04-23T10:42:42.339 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.339 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.339 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.339 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.339 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.339 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.339 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.339 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-23T10:42:42.339 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.339 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.339 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.339 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.340 INFO:teuthology.orchestra.run.smithi097.stdout: "use_agent": { 2024-04-23T10:42:42.340 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "use_agent", 2024-04-23T10:42:42.340 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.340 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.340 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.340 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.340 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.340 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.340 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.340 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-23T10:42:42.340 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.340 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.340 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.340 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.340 INFO:teuthology.orchestra.run.smithi097.stdout: "use_repo_digest": { 2024-04-23T10:42:42.340 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "use_repo_digest", 2024-04-23T10:42:42.341 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.341 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.341 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.341 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:42.341 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.341 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.341 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.341 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-23T10:42:42.341 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.341 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.341 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.341 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.341 INFO:teuthology.orchestra.run.smithi097.stdout: "warn_on_failed_host_check": { 2024-04-23T10:42:42.341 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "warn_on_failed_host_check", 2024-04-23T10:42:42.341 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.341 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.342 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.342 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:42.342 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.342 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.342 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.342 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "raise a health warning if the host check fails", 2024-04-23T10:42:42.342 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.342 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.342 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.342 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.342 INFO:teuthology.orchestra.run.smithi097.stdout: "warn_on_stray_daemons": { 2024-04-23T10:42:42.342 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "warn_on_stray_daemons", 2024-04-23T10:42:42.342 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.342 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.342 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.342 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:42.343 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.343 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.343 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.343 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-23T10:42:42.343 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.343 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.343 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.343 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.343 INFO:teuthology.orchestra.run.smithi097.stdout: "warn_on_stray_hosts": { 2024-04-23T10:42:42.343 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "warn_on_stray_hosts", 2024-04-23T10:42:42.343 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.343 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.343 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.343 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:42.343 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.343 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.344 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.344 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-23T10:42:42.344 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.344 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.344 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.344 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.344 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.344 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.344 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.344 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "crash", 2024-04-23T10:42:42.344 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:42.344 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:42.344 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:42.344 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:42.344 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:42.344 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.345 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.345 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.345 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.345 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.345 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.345 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.345 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.345 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.345 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.345 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.345 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.345 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.345 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.345 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.345 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.345 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.346 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.346 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:42.346 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:42.346 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.346 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.346 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.346 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.346 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.346 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.346 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.346 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.346 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.346 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.346 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.346 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.347 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:42.347 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:42.347 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.347 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.347 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.347 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:42.347 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.347 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.347 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.347 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.347 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.347 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.347 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.347 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.347 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.347 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.348 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.348 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.348 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.348 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.348 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.348 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:42.348 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:42.348 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.348 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.348 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.348 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.348 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.348 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.348 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.348 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.348 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.349 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.349 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.349 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.349 INFO:teuthology.orchestra.run.smithi097.stdout: "retain_interval": { 2024-04-23T10:42:42.349 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "retain_interval", 2024-04-23T10:42:42.349 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-04-23T10:42:42.349 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.349 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.349 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "31536000", 2024-04-23T10:42:42.349 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.349 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.349 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.349 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-23T10:42:42.349 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.349 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.349 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.350 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.350 INFO:teuthology.orchestra.run.smithi097.stdout: "warn_recent_interval": { 2024-04-23T10:42:42.350 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "warn_recent_interval", 2024-04-23T10:42:42.350 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-04-23T10:42:42.350 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.350 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.350 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "1209600", 2024-04-23T10:42:42.350 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.350 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.350 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.350 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-23T10:42:42.350 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.350 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.350 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.350 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.350 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.351 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.351 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.351 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "dashboard", 2024-04-23T10:42:42.351 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:42.351 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:42.351 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:42.351 INFO:teuthology.orchestra.run.smithi097.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-23T10:42:42.351 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-23T10:42:42.351 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:42.351 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.351 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.351 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-04-23T10:42:42.351 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.351 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.351 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.351 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.352 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.352 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.352 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.352 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.352 INFO:teuthology.orchestra.run.smithi097.stdout: "ALERTMANAGER_API_HOST": { 2024-04-23T10:42:42.352 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-23T10:42:42.352 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.352 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.352 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.352 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.352 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.352 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.352 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.352 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.352 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.353 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.353 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.353 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.353 INFO:teuthology.orchestra.run.smithi097.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-23T10:42:42.353 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-23T10:42:42.353 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.353 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.353 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.353 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:42.353 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.353 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.353 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.353 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.353 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.353 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.353 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.353 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.354 INFO:teuthology.orchestra.run.smithi097.stdout: "AUDIT_API_ENABLED": { 2024-04-23T10:42:42.354 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "AUDIT_API_ENABLED", 2024-04-23T10:42:42.354 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.354 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.354 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.354 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.354 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.354 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.354 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.354 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.354 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.354 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.354 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.354 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.354 INFO:teuthology.orchestra.run.smithi097.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-23T10:42:42.354 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-23T10:42:42.355 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.355 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.355 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.355 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:42.355 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.355 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.355 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.355 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.355 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.355 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.355 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.355 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.355 INFO:teuthology.orchestra.run.smithi097.stdout: "ENABLE_BROWSABLE_API": { 2024-04-23T10:42:42.355 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-23T10:42:42.355 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.355 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.356 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.356 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:42.356 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.356 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.356 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.356 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.356 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.356 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.356 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.356 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.356 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-23T10:42:42.356 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-23T10:42:42.356 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.356 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.356 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.356 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:42.357 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.357 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.357 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.357 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.357 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.357 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.357 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.357 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.357 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-23T10:42:42.357 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-23T10:42:42.357 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.357 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.357 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.357 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:42.357 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.357 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.357 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.358 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.358 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.358 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.358 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.358 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.358 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-23T10:42:42.358 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-23T10:42:42.358 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.358 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.358 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.358 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:42.358 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.358 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.358 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.358 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.359 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.359 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.359 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.359 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.359 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-23T10:42:42.359 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-23T10:42:42.359 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.359 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.359 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.359 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:42.359 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.359 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.359 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.359 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.359 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.360 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.360 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.360 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.360 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-23T10:42:42.360 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-23T10:42:42.360 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.360 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.360 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.360 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:42.360 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.360 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.360 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.360 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.360 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.360 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.361 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.361 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.361 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-23T10:42:42.361 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-23T10:42:42.361 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.361 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.361 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.361 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:42.361 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.361 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.361 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.361 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.361 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.361 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.361 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.362 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.362 INFO:teuthology.orchestra.run.smithi097.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-23T10:42:42.362 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-23T10:42:42.362 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.362 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.362 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.362 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:42.362 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.362 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.362 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.362 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.362 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.362 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.362 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.362 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.363 INFO:teuthology.orchestra.run.smithi097.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-23T10:42:42.363 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-23T10:42:42.363 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.363 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.363 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.363 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.363 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.363 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.363 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.363 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.363 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.363 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.363 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.363 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.363 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_API_PASSWORD": { 2024-04-23T10:42:42.363 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-23T10:42:42.364 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.364 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.364 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.364 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "admin", 2024-04-23T10:42:42.364 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.364 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.364 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.364 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.364 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.364 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.364 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.364 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.364 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-23T10:42:42.364 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-23T10:42:42.364 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.365 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.365 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.365 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:42.365 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.365 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.365 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.365 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.365 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.365 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.365 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.365 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.365 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_API_URL": { 2024-04-23T10:42:42.365 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_API_URL", 2024-04-23T10:42:42.365 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.365 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.366 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.366 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.366 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.366 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.366 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.366 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.366 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.366 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.366 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.366 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.366 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_API_USERNAME": { 2024-04-23T10:42:42.366 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-23T10:42:42.366 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.366 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.366 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.366 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "admin", 2024-04-23T10:42:42.367 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.367 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.367 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.367 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.367 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.367 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.367 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.367 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.367 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-23T10:42:42.367 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-23T10:42:42.367 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.367 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.367 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.367 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.367 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.367 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.368 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.368 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.368 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.368 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.368 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.368 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.368 INFO:teuthology.orchestra.run.smithi097.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-23T10:42:42.368 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-23T10:42:42.368 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.368 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.368 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.368 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.368 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.368 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.368 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.369 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.369 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.369 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.369 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.369 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.369 INFO:teuthology.orchestra.run.smithi097.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-23T10:42:42.369 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-23T10:42:42.369 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.369 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.369 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.369 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:42.369 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.369 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.369 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.369 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.369 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.370 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.370 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.370 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.370 INFO:teuthology.orchestra.run.smithi097.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-23T10:42:42.370 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-23T10:42:42.370 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.370 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.370 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.370 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.370 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.370 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.370 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.370 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.370 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.370 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.371 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.371 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.371 INFO:teuthology.orchestra.run.smithi097.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-23T10:42:42.371 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-23T10:42:42.371 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.371 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.371 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.371 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "[]", 2024-04-23T10:42:42.371 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.371 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.371 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.371 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.371 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.371 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.372 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.372 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.372 INFO:teuthology.orchestra.run.smithi097.stdout: "MULTICLUSTER_CONFIG": { 2024-04-23T10:42:42.372 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-23T10:42:42.372 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.372 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.372 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.372 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "{}", 2024-04-23T10:42:42.372 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.372 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.372 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.372 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.372 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.372 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.372 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.373 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.373 INFO:teuthology.orchestra.run.smithi097.stdout: "PROMETHEUS_API_HOST": { 2024-04-23T10:42:42.373 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-23T10:42:42.373 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.373 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.373 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.373 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.373 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.373 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.373 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.373 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.373 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.373 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.373 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.373 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.374 INFO:teuthology.orchestra.run.smithi097.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-23T10:42:42.374 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-23T10:42:42.374 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.374 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.374 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.374 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:42.374 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.374 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.374 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.374 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.374 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.374 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.374 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.374 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.375 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-23T10:42:42.375 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-23T10:42:42.375 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.375 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.375 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.375 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.375 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.375 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.375 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.375 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.375 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.375 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.375 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.375 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.376 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-23T10:42:42.376 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-23T10:42:42.376 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.376 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.376 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.376 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.376 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.376 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.376 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.376 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.376 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.376 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.376 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.376 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.377 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-23T10:42:42.377 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-23T10:42:42.377 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.377 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.377 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.377 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:42.377 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.377 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.377 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.377 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.377 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.377 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.377 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.377 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.377 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-23T10:42:42.378 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-23T10:42:42.378 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.378 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.378 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.378 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:42.378 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.378 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.378 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.378 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.378 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.378 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.378 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.378 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.378 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-23T10:42:42.379 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-23T10:42:42.379 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.379 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.379 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.379 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.379 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.379 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.379 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.379 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.379 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.379 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.379 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.379 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.379 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-23T10:42:42.380 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-23T10:42:42.380 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.380 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.380 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.380 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.380 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.380 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.380 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.380 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.380 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.380 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.380 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.380 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.380 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-23T10:42:42.380 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-23T10:42:42.381 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.381 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.381 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.381 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.381 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.381 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.381 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.381 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.381 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.381 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.381 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.381 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.381 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_ENABLED": { 2024-04-23T10:42:42.381 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-23T10:42:42.381 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.382 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.382 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.382 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:42.382 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.382 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.382 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.382 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.382 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.382 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.382 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.382 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.382 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-23T10:42:42.382 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-23T10:42:42.382 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.382 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.383 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.383 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-23T10:42:42.383 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.383 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.383 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.383 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.383 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.383 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.383 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.383 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.383 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-23T10:42:42.383 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-23T10:42:42.383 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:42.383 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.383 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.384 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-04-23T10:42:42.384 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.384 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.384 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.384 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.384 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.384 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.384 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.384 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.384 INFO:teuthology.orchestra.run.smithi097.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-23T10:42:42.384 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-23T10:42:42.384 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:42.384 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.384 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.384 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "8", 2024-04-23T10:42:42.384 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.385 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.385 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.385 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.385 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.385 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.385 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.385 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.385 INFO:teuthology.orchestra.run.smithi097.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-23T10:42:42.385 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-23T10:42:42.385 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:42.385 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.385 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.385 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "45", 2024-04-23T10:42:42.385 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.385 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.386 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.386 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.386 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.386 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.386 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.386 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.386 INFO:teuthology.orchestra.run.smithi097.stdout: "RGW_API_ACCESS_KEY": { 2024-04-23T10:42:42.386 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-23T10:42:42.386 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.386 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.386 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.386 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.386 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.386 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.386 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.387 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.387 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.387 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.387 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.387 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.387 INFO:teuthology.orchestra.run.smithi097.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-23T10:42:42.387 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-23T10:42:42.387 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.387 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.387 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.387 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "admin", 2024-04-23T10:42:42.387 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.387 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.387 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.387 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.387 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.387 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.388 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.388 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.388 INFO:teuthology.orchestra.run.smithi097.stdout: "RGW_API_SECRET_KEY": { 2024-04-23T10:42:42.388 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-23T10:42:42.388 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.388 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.388 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.388 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.388 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.388 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.388 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.388 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.388 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.388 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.388 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.388 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.389 INFO:teuthology.orchestra.run.smithi097.stdout: "RGW_API_SSL_VERIFY": { 2024-04-23T10:42:42.389 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-23T10:42:42.389 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.389 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.389 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.389 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:42.389 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.389 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.389 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.389 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.389 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.389 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.389 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.389 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.389 INFO:teuthology.orchestra.run.smithi097.stdout: "UNSAFE_TLS_v1_2": { 2024-04-23T10:42:42.389 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-23T10:42:42.390 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.390 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.390 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.390 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.390 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.390 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.390 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.390 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.390 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.390 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.390 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.390 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.390 INFO:teuthology.orchestra.run.smithi097.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-23T10:42:42.390 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-23T10:42:42.390 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:42.390 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.391 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.391 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0", 2024-04-23T10:42:42.391 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.391 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.391 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.391 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.391 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.391 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.391 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.391 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.391 INFO:teuthology.orchestra.run.smithi097.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-23T10:42:42.391 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-23T10:42:42.391 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:42.391 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.391 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.391 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-04-23T10:42:42.392 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.392 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.392 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.392 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.392 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.392 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.392 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.392 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.392 INFO:teuthology.orchestra.run.smithi097.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-23T10:42:42.392 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-23T10:42:42.392 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:42.392 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.392 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.392 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "5", 2024-04-23T10:42:42.392 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.392 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.393 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.393 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.393 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.393 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.393 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.393 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.393 INFO:teuthology.orchestra.run.smithi097.stdout: "cross_origin_url": { 2024-04-23T10:42:42.393 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "cross_origin_url", 2024-04-23T10:42:42.393 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.393 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.393 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.393 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.393 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.393 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.393 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.393 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.394 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.394 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.394 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.394 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.394 INFO:teuthology.orchestra.run.smithi097.stdout: "crt_file": { 2024-04-23T10:42:42.394 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "crt_file", 2024-04-23T10:42:42.394 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.394 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.394 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.394 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.394 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.394 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.394 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.394 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.394 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.394 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.395 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.395 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.395 INFO:teuthology.orchestra.run.smithi097.stdout: "debug": { 2024-04-23T10:42:42.395 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "debug", 2024-04-23T10:42:42.395 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.395 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.395 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.395 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.395 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.395 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.395 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.395 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Enable/disable debug options", 2024-04-23T10:42:42.395 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.395 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.395 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.395 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.396 INFO:teuthology.orchestra.run.smithi097.stdout: "jwt_token_ttl": { 2024-04-23T10:42:42.396 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "jwt_token_ttl", 2024-04-23T10:42:42.396 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:42.396 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.396 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.396 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "28800", 2024-04-23T10:42:42.396 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.396 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.396 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.396 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.396 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.396 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.396 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.396 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.396 INFO:teuthology.orchestra.run.smithi097.stdout: "key_file": { 2024-04-23T10:42:42.396 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "key_file", 2024-04-23T10:42:42.396 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.397 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.397 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.397 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.397 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.397 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.397 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.397 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.397 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.397 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.397 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.397 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.397 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:42.397 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:42.397 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.397 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.398 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.398 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.398 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.398 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.398 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.398 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.398 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.398 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.398 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.398 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.398 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.398 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.398 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.398 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.398 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.399 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.399 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.399 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:42.399 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:42.399 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.399 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.399 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.399 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.399 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.399 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.399 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.399 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.399 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.399 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.399 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.399 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.399 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:42.400 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:42.400 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.400 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.400 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.400 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:42.400 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.400 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.400 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.400 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.400 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.400 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.400 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.400 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.400 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.400 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.400 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.401 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.401 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.401 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.401 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.401 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:42.401 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:42.401 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.401 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.401 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.401 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.401 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.401 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.401 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.401 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.401 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.402 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.402 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.402 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.402 INFO:teuthology.orchestra.run.smithi097.stdout: "motd": { 2024-04-23T10:42:42.402 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "motd", 2024-04-23T10:42:42.402 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.402 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.402 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.402 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.402 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.402 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.402 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.402 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "The message of the day", 2024-04-23T10:42:42.402 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.402 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.402 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.403 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.403 INFO:teuthology.orchestra.run.smithi097.stdout: "redirect_resolve_ip_addr": { 2024-04-23T10:42:42.403 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "redirect_resolve_ip_addr", 2024-04-23T10:42:42.403 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.403 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.403 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.403 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.403 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.403 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.403 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.403 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.403 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.403 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.403 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.403 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.403 INFO:teuthology.orchestra.run.smithi097.stdout: "server_addr": { 2024-04-23T10:42:42.404 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_addr", 2024-04-23T10:42:42.404 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.404 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.404 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.404 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "::", 2024-04-23T10:42:42.404 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.404 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.404 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.404 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.404 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.404 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.404 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.404 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.404 INFO:teuthology.orchestra.run.smithi097.stdout: "server_port": { 2024-04-23T10:42:42.404 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_port", 2024-04-23T10:42:42.405 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:42.405 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.405 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.405 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "8080", 2024-04-23T10:42:42.405 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.405 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.405 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.405 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.405 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.405 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.405 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.405 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.405 INFO:teuthology.orchestra.run.smithi097.stdout: "ssl": { 2024-04-23T10:42:42.405 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ssl", 2024-04-23T10:42:42.406 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.406 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.406 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.406 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:42.406 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.406 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.406 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.406 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.406 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.406 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.406 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.406 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.406 INFO:teuthology.orchestra.run.smithi097.stdout: "ssl_server_port": { 2024-04-23T10:42:42.406 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ssl_server_port", 2024-04-23T10:42:42.406 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:42.406 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.407 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.407 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "8443", 2024-04-23T10:42:42.407 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.407 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.407 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.407 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.407 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.407 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.407 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.407 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.407 INFO:teuthology.orchestra.run.smithi097.stdout: "standby_behaviour": { 2024-04-23T10:42:42.407 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "standby_behaviour", 2024-04-23T10:42:42.407 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.407 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.407 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.407 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "redirect", 2024-04-23T10:42:42.408 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.408 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.408 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.408 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.408 INFO:teuthology.orchestra.run.smithi097.stdout: "redirect" 2024-04-23T10:42:42.408 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.408 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.408 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.408 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.408 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.408 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.408 INFO:teuthology.orchestra.run.smithi097.stdout: "standby_error_status_code": { 2024-04-23T10:42:42.408 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "standby_error_status_code", 2024-04-23T10:42:42.408 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:42.408 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.408 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.409 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "500", 2024-04-23T10:42:42.409 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "400", 2024-04-23T10:42:42.409 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "599", 2024-04-23T10:42:42.409 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.409 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.409 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.409 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.409 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.409 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.409 INFO:teuthology.orchestra.run.smithi097.stdout: "url_prefix": { 2024-04-23T10:42:42.409 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "url_prefix", 2024-04-23T10:42:42.409 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.409 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.409 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.409 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.409 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.410 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.410 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.410 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.410 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.410 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.410 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.410 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.410 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.410 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.410 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.410 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "devicehealth", 2024-04-23T10:42:42.410 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:42.410 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:42.410 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:42.410 INFO:teuthology.orchestra.run.smithi097.stdout: "enable_monitoring": { 2024-04-23T10:42:42.411 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "enable_monitoring", 2024-04-23T10:42:42.411 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.411 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.411 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.411 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:42.411 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.411 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.411 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.411 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "monitor device health metrics", 2024-04-23T10:42:42.411 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.411 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.411 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.411 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.411 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:42.411 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:42.411 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.412 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.412 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.412 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.412 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.412 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.412 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.412 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.412 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.412 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.412 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.412 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.412 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.412 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.412 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.412 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.413 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.413 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.413 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.413 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:42.413 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:42.413 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.413 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.413 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.413 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.413 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.413 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.413 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.413 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.413 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.413 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.413 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.413 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.414 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:42.414 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:42.414 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.414 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.414 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.414 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:42.414 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.414 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.414 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.414 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.414 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.414 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.414 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.414 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.414 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.415 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.415 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.415 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.415 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.415 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.415 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.415 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:42.415 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:42.415 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.415 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.415 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.415 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.415 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.415 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.415 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.415 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.416 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.416 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.416 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.416 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.416 INFO:teuthology.orchestra.run.smithi097.stdout: "mark_out_threshold": { 2024-04-23T10:42:42.416 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mark_out_threshold", 2024-04-23T10:42:42.416 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-04-23T10:42:42.416 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.416 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.416 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "2419200", 2024-04-23T10:42:42.416 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.416 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.416 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.416 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-23T10:42:42.416 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.416 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.416 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.417 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.417 INFO:teuthology.orchestra.run.smithi097.stdout: "pool_name": { 2024-04-23T10:42:42.417 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "pool_name", 2024-04-23T10:42:42.417 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.417 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.417 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.417 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "device_health_metrics", 2024-04-23T10:42:42.417 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.417 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.417 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.417 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-23T10:42:42.417 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.417 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.417 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.417 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.417 INFO:teuthology.orchestra.run.smithi097.stdout: "retention_period": { 2024-04-23T10:42:42.417 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "retention_period", 2024-04-23T10:42:42.418 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-04-23T10:42:42.418 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.418 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.418 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "15552000", 2024-04-23T10:42:42.418 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.418 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.418 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.418 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how long to retain device health metrics", 2024-04-23T10:42:42.418 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.418 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.418 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.418 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.418 INFO:teuthology.orchestra.run.smithi097.stdout: "scrape_frequency": { 2024-04-23T10:42:42.419 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "scrape_frequency", 2024-04-23T10:42:42.419 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-04-23T10:42:42.419 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.419 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.419 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "86400", 2024-04-23T10:42:42.419 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.419 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.419 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.419 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-23T10:42:42.419 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.419 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.419 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.419 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.419 INFO:teuthology.orchestra.run.smithi097.stdout: "self_heal": { 2024-04-23T10:42:42.419 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "self_heal", 2024-04-23T10:42:42.419 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.420 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.420 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.420 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:42.420 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.420 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.420 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.420 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-23T10:42:42.420 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.420 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.420 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.420 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.420 INFO:teuthology.orchestra.run.smithi097.stdout: "sleep_interval": { 2024-04-23T10:42:42.420 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "sleep_interval", 2024-04-23T10:42:42.420 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-04-23T10:42:42.420 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.420 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.421 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "600", 2024-04-23T10:42:42.421 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.421 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.421 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.421 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how frequently to wake up and check device health", 2024-04-23T10:42:42.421 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.421 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.421 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.421 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.421 INFO:teuthology.orchestra.run.smithi097.stdout: "warn_threshold": { 2024-04-23T10:42:42.421 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "warn_threshold", 2024-04-23T10:42:42.421 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-04-23T10:42:42.421 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.421 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.421 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "7257600", 2024-04-23T10:42:42.422 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.422 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.422 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.422 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-23T10:42:42.422 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.422 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.422 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.422 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.422 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.422 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.422 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.422 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "diskprediction_local", 2024-04-23T10:42:42.422 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:42.422 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:42.422 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:42.423 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:42.423 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:42.423 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.423 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.423 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.423 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.423 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.423 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.423 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.423 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.423 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.423 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.423 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.423 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.423 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.423 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.424 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.424 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.424 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.424 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.424 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.424 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:42.424 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:42.424 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.424 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.424 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.424 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.424 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.424 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.424 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.424 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.424 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.425 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.425 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.425 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.425 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:42.425 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:42.425 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.425 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.425 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.425 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:42.425 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.425 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.425 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.425 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.425 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.425 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.426 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.426 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.426 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.426 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.426 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.426 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.426 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.426 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.426 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.426 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:42.426 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:42.426 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.426 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.426 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.427 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.427 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.427 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.427 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.427 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.427 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.427 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.427 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.427 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.427 INFO:teuthology.orchestra.run.smithi097.stdout: "predict_interval": { 2024-04-23T10:42:42.427 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "predict_interval", 2024-04-23T10:42:42.427 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.427 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.427 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.427 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "86400", 2024-04-23T10:42:42.427 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.428 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.428 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.428 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.428 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.428 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.428 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.428 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.428 INFO:teuthology.orchestra.run.smithi097.stdout: "predictor_model": { 2024-04-23T10:42:42.428 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "predictor_model", 2024-04-23T10:42:42.428 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.428 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.428 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.428 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "prophetstor", 2024-04-23T10:42:42.428 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.428 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.428 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.429 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.429 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.429 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.429 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.429 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.429 INFO:teuthology.orchestra.run.smithi097.stdout: "sleep_interval": { 2024-04-23T10:42:42.429 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "sleep_interval", 2024-04-23T10:42:42.429 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.429 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.429 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.429 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "600", 2024-04-23T10:42:42.429 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.429 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.429 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.429 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.430 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.430 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.430 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.430 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.430 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.430 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.430 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.430 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "influx", 2024-04-23T10:42:42.430 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": false, 2024-04-23T10:42:42.430 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "influxdb python module not found", 2024-04-23T10:42:42.430 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:42.430 INFO:teuthology.orchestra.run.smithi097.stdout: "batch_size": { 2024-04-23T10:42:42.430 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "batch_size", 2024-04-23T10:42:42.430 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:42.430 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.431 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.431 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "5000", 2024-04-23T10:42:42.431 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.431 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.431 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.431 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-23T10:42:42.431 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.431 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.431 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.431 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.431 INFO:teuthology.orchestra.run.smithi097.stdout: "database": { 2024-04-23T10:42:42.431 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "database", 2024-04-23T10:42:42.431 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.431 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.432 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.432 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "ceph", 2024-04-23T10:42:42.432 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.432 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.432 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.432 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-23T10:42:42.432 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.432 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.432 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.432 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.432 INFO:teuthology.orchestra.run.smithi097.stdout: "hostname": { 2024-04-23T10:42:42.432 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "hostname", 2024-04-23T10:42:42.432 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.432 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.432 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.432 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.433 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.433 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.433 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.433 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "InfluxDB server hostname", 2024-04-23T10:42:42.433 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.433 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.433 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.433 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.433 INFO:teuthology.orchestra.run.smithi097.stdout: "interval": { 2024-04-23T10:42:42.433 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "interval", 2024-04-23T10:42:42.433 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-04-23T10:42:42.433 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.433 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.433 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "30", 2024-04-23T10:42:42.434 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "5", 2024-04-23T10:42:42.434 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.434 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.434 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-23T10:42:42.434 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.434 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.434 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.434 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.434 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:42.434 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:42.434 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.434 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.434 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.434 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.434 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.435 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.435 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.435 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.435 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.435 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.435 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.435 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.435 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.435 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.435 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.435 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.435 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.435 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.435 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.435 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:42.435 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:42.436 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.436 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.436 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.436 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.436 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.436 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.436 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.436 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.436 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.436 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.436 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.436 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.436 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:42.436 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:42.436 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.436 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.437 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.437 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:42.437 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.437 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.437 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.437 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.437 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.437 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.437 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.437 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.437 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.437 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.437 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.437 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.437 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.437 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.438 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.438 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:42.438 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:42.438 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.438 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.438 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.438 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.438 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.438 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.438 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.438 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.438 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.438 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.438 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.438 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.439 INFO:teuthology.orchestra.run.smithi097.stdout: "password": { 2024-04-23T10:42:42.439 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "password", 2024-04-23T10:42:42.439 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.439 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.439 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.439 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.439 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.439 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.439 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.439 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "password of InfluxDB server user", 2024-04-23T10:42:42.439 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.439 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.439 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.439 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.439 INFO:teuthology.orchestra.run.smithi097.stdout: "port": { 2024-04-23T10:42:42.440 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "port", 2024-04-23T10:42:42.440 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:42.440 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.440 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.440 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "8086", 2024-04-23T10:42:42.440 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.440 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.440 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.440 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "InfluxDB server port", 2024-04-23T10:42:42.440 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.440 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.440 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.440 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.440 INFO:teuthology.orchestra.run.smithi097.stdout: "ssl": { 2024-04-23T10:42:42.440 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ssl", 2024-04-23T10:42:42.440 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.440 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.441 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.441 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "false", 2024-04-23T10:42:42.441 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.441 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.441 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.441 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-23T10:42:42.441 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.441 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.441 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.441 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.441 INFO:teuthology.orchestra.run.smithi097.stdout: "threads": { 2024-04-23T10:42:42.441 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "threads", 2024-04-23T10:42:42.441 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:42.441 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.441 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.442 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "5", 2024-04-23T10:42:42.442 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "1", 2024-04-23T10:42:42.442 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "32", 2024-04-23T10:42:42.442 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.442 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-23T10:42:42.442 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.442 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.442 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.442 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.442 INFO:teuthology.orchestra.run.smithi097.stdout: "username": { 2024-04-23T10:42:42.442 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "username", 2024-04-23T10:42:42.442 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.442 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.442 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.442 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.442 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.443 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.443 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.443 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "username of InfluxDB server user", 2024-04-23T10:42:42.443 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.443 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.443 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.443 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.443 INFO:teuthology.orchestra.run.smithi097.stdout: "verify_ssl": { 2024-04-23T10:42:42.443 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "verify_ssl", 2024-04-23T10:42:42.443 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.443 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.443 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.443 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "true", 2024-04-23T10:42:42.443 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.443 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.443 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.444 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-23T10:42:42.444 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.444 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.444 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.444 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.444 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.444 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.444 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.444 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "insights", 2024-04-23T10:42:42.444 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:42.444 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:42.444 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:42.444 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:42.444 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:42.444 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.445 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.445 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.445 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.445 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.445 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.445 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.445 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.445 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.445 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.445 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.445 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.445 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.445 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.445 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.445 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.445 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.445 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.446 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.446 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:42.446 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:42.446 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.446 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.446 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.446 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.446 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.446 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.446 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.446 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.446 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.446 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.446 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.446 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.447 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:42.447 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:42.447 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.447 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.447 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.447 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:42.447 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.447 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.447 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.447 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.447 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.447 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.447 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.447 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.447 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.447 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.447 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.448 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.448 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.448 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.448 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.448 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:42.448 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:42.448 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.448 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.448 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.448 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.448 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.448 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.448 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.448 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.448 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.448 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.449 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.449 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.449 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.449 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.449 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.449 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "iostat", 2024-04-23T10:42:42.449 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:42.449 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:42.449 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:42.449 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:42.449 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:42.449 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.449 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.449 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.449 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.449 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.450 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.450 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.450 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.450 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.450 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.450 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.450 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.450 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.450 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.450 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.450 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.450 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.450 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.450 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.450 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:42.450 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:42.451 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.451 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.451 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.451 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.451 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.451 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.451 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.451 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.451 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.451 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.451 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.451 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.451 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:42.451 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:42.452 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.452 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.452 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.452 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:42.452 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.452 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.452 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.452 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.452 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.452 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.452 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.452 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.452 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.452 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.452 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.452 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.453 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.453 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.453 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.453 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:42.453 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:42.453 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.453 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.453 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.453 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.453 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.453 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.453 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.453 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.453 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.453 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.453 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.454 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.454 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.454 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.454 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.454 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "k8sevents", 2024-04-23T10:42:42.454 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:42.454 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:42.454 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:42.454 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_event_retention_days": { 2024-04-23T10:42:42.454 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "ceph_event_retention_days", 2024-04-23T10:42:42.454 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:42.454 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.454 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.454 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "7", 2024-04-23T10:42:42.454 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.454 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.455 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.455 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-23T10:42:42.455 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.455 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.455 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.455 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.455 INFO:teuthology.orchestra.run.smithi097.stdout: "config_check_secs": { 2024-04-23T10:42:42.455 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "config_check_secs", 2024-04-23T10:42:42.455 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:42.455 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.455 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.455 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-04-23T10:42:42.455 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "10", 2024-04-23T10:42:42.455 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.455 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.455 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-23T10:42:42.455 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.456 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.456 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.456 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.456 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:42.456 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:42.456 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.456 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.456 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.456 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.456 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.456 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.456 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.456 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.456 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.456 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.456 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.457 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.457 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.457 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.457 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.457 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.457 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.457 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.457 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.457 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:42.457 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:42.457 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.457 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.457 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.457 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.457 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.457 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.457 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.458 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.458 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.458 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.458 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.458 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.458 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:42.458 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:42.458 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.458 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.458 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.458 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:42.458 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.458 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.458 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.458 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.459 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.459 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.459 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.459 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.459 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.459 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.459 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.459 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.459 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.459 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.459 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.459 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:42.459 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:42.459 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.459 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.459 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.459 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.460 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.460 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.460 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.460 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.460 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.460 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.460 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.460 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.460 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.460 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.460 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.460 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "localpool", 2024-04-23T10:42:42.460 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:42.460 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:42.460 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:42.460 INFO:teuthology.orchestra.run.smithi097.stdout: "failure_domain": { 2024-04-23T10:42:42.461 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "failure_domain", 2024-04-23T10:42:42.461 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.461 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.461 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.461 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "host", 2024-04-23T10:42:42.461 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.461 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.461 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.461 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "failure domain for any created local pool", 2024-04-23T10:42:42.461 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-23T10:42:42.461 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.461 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.461 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.461 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:42.461 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:42.461 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.462 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.462 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.462 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.462 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.462 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.462 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.462 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.462 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.462 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.462 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.462 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.462 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.462 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.462 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.462 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.462 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.463 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.463 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.463 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:42.463 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:42.463 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.463 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.463 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.463 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.463 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.463 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.463 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.463 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.463 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.463 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.463 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.463 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.464 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:42.464 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:42.464 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.464 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.464 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.464 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:42.464 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.464 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.464 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.464 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.464 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.464 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.464 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.464 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.464 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.464 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.465 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.465 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.465 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.465 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.465 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.465 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:42.465 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:42.465 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.465 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.465 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.465 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.465 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.465 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.465 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.465 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.465 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.466 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.466 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.466 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.466 INFO:teuthology.orchestra.run.smithi097.stdout: "min_size": { 2024-04-23T10:42:42.466 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "min_size", 2024-04-23T10:42:42.466 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:42.466 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.466 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.466 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.466 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.466 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.466 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.466 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "default min_size for any created local pool", 2024-04-23T10:42:42.466 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-23T10:42:42.466 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.467 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.467 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.467 INFO:teuthology.orchestra.run.smithi097.stdout: "num_rep": { 2024-04-23T10:42:42.467 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "num_rep", 2024-04-23T10:42:42.467 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:42.467 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.467 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.467 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "3", 2024-04-23T10:42:42.467 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.467 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.467 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.467 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "default replica count for any created local pool", 2024-04-23T10:42:42.467 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.467 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.467 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.467 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.468 INFO:teuthology.orchestra.run.smithi097.stdout: "pg_num": { 2024-04-23T10:42:42.468 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "pg_num", 2024-04-23T10:42:42.468 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:42.468 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.468 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.468 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "128", 2024-04-23T10:42:42.468 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.468 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.468 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.468 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "default pg_num for any created local pool", 2024-04-23T10:42:42.468 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.468 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.468 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.468 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.468 INFO:teuthology.orchestra.run.smithi097.stdout: "prefix": { 2024-04-23T10:42:42.468 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "prefix", 2024-04-23T10:42:42.469 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.469 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.469 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.469 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.469 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.469 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.469 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.469 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "name prefix for any created local pool", 2024-04-23T10:42:42.469 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.469 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.469 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.469 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.469 INFO:teuthology.orchestra.run.smithi097.stdout: "subtree": { 2024-04-23T10:42:42.469 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "subtree", 2024-04-23T10:42:42.469 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.469 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.470 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.470 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "rack", 2024-04-23T10:42:42.470 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.470 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.470 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.470 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-23T10:42:42.470 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-23T10:42:42.470 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.470 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.470 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.470 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.470 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.470 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.470 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mds_autoscaler", 2024-04-23T10:42:42.470 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:42.470 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:42.471 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:42.471 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:42.471 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:42.471 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.471 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.471 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.471 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.471 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.471 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.471 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.471 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.471 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.471 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.471 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.471 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.472 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.472 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.472 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.472 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.472 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.472 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.472 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.472 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:42.472 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:42.472 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.472 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.472 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.472 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.472 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.472 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.473 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.473 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.473 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.473 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.473 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.473 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.473 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:42.473 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:42.473 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.473 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.473 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.473 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:42.473 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.473 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.473 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.473 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.474 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.474 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.474 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.474 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.474 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.474 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.474 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.474 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.474 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.474 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.474 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.474 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:42.474 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:42.474 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.474 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.474 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.475 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.475 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.475 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.475 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.475 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.475 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.475 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.475 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.475 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.475 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.475 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.475 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.475 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mirroring", 2024-04-23T10:42:42.475 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:42.475 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:42.475 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:42.476 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:42.476 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:42.476 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.476 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.476 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.476 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.476 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.476 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.476 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.476 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.476 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.476 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.476 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.476 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.477 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.477 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.477 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.477 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.477 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.477 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.477 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.477 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:42.477 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:42.477 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.477 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.477 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.477 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.477 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.477 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.477 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.477 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.478 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.478 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.478 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.478 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.478 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:42.478 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:42.478 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.478 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.478 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.478 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:42.478 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.478 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.478 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.478 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.478 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.479 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.479 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.479 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.479 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.479 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.479 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.479 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.479 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.479 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.479 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.479 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:42.479 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:42.479 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.479 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.479 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.480 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.480 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.480 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.480 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.480 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.480 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.480 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.480 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.480 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.480 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.480 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.480 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.480 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "nfs", 2024-04-23T10:42:42.480 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:42.480 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:42.480 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:42.481 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:42.481 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:42.481 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.481 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.481 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.481 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.481 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.481 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.481 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.481 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.481 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.481 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.481 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.481 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.481 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.481 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.482 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.482 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.482 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.482 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.482 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.482 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:42.482 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:42.482 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.482 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.482 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.482 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.482 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.482 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.482 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.482 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.482 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.483 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.483 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.483 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.483 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:42.483 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:42.483 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.483 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.483 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.483 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:42.483 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.483 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.483 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.483 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.483 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.483 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.483 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.484 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.484 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.484 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.484 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.484 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.484 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.484 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.484 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.484 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:42.484 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:42.484 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.484 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.484 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.484 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.484 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.484 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.485 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.485 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.485 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.485 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.485 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.485 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.485 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.485 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.485 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.485 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "orchestrator", 2024-04-23T10:42:42.485 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:42.485 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:42.485 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:42.485 INFO:teuthology.orchestra.run.smithi097.stdout: "fail_fs": { 2024-04-23T10:42:42.485 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "fail_fs", 2024-04-23T10:42:42.485 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.486 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.486 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.486 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.486 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.486 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.486 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.486 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-23T10:42:42.486 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.486 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.486 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.486 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.486 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:42.486 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:42.486 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.486 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.487 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.487 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.487 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.487 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.487 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.487 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.487 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.487 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.487 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.487 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.487 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.487 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.487 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.487 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.487 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.487 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.488 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.488 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:42.488 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:42.488 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.488 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.488 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.488 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.488 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.488 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.488 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.488 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.488 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.488 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.488 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.488 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.488 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:42.489 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:42.489 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.489 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.489 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.489 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:42.489 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.489 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.489 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.489 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.489 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.489 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.489 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.489 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.489 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.489 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.489 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.489 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.490 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.490 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.490 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.490 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:42.490 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:42.490 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.490 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.490 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.490 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.490 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.490 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.490 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.490 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.490 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.490 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.491 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.491 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.491 INFO:teuthology.orchestra.run.smithi097.stdout: "orchestrator": { 2024-04-23T10:42:42.491 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "orchestrator", 2024-04-23T10:42:42.491 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.491 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.491 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.491 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.491 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.491 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.491 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.491 INFO:teuthology.orchestra.run.smithi097.stdout: "cephadm", 2024-04-23T10:42:42.491 INFO:teuthology.orchestra.run.smithi097.stdout: "rook", 2024-04-23T10:42:42.491 INFO:teuthology.orchestra.run.smithi097.stdout: "test_orchestrator" 2024-04-23T10:42:42.491 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.491 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Orchestrator backend", 2024-04-23T10:42:42.492 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.492 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.492 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.492 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.492 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.492 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.492 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.492 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "osd_perf_query", 2024-04-23T10:42:42.492 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:42.492 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:42.492 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:42.492 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:42.492 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:42.492 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.492 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.492 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.493 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.493 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.493 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.493 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.493 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.493 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.493 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.493 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.493 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.493 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.493 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.493 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.493 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.493 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.493 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.493 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.494 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:42.494 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:42.494 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.494 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.494 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.494 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.494 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.494 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.494 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.494 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.494 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.494 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.494 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.494 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.494 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:42.494 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:42.495 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.495 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.495 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.495 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:42.495 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.495 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.495 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.495 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.495 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.495 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.495 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.495 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.495 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.495 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.495 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.495 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.496 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.496 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.496 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.496 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:42.496 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:42.496 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.496 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.496 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.496 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.496 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.496 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.496 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.496 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.496 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.496 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.496 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.496 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.497 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.497 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.497 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.497 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "osd_support", 2024-04-23T10:42:42.497 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:42.497 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:42.497 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:42.497 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:42.497 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:42.497 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.497 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.497 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.497 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.497 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.497 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.497 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.498 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.498 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.498 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.498 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.498 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.498 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.498 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.498 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.498 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.498 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.498 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.498 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.498 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:42.498 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:42.498 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.498 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.499 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.499 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.499 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.499 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.499 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.499 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.499 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.499 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.499 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.499 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.499 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:42.499 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:42.499 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.499 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.499 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.499 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:42.500 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.500 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.500 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.500 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.500 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.500 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.500 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.500 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.500 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.500 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.500 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.500 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.500 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.500 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.500 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.500 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:42.501 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:42.501 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.501 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.501 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.501 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.501 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.501 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.501 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.501 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.501 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.501 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.501 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.501 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.501 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.501 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.502 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.502 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "pg_autoscaler", 2024-04-23T10:42:42.502 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:42.502 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:42.502 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:42.502 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:42.502 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:42.502 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.502 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.502 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.502 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.502 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.502 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.502 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.503 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.503 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.503 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.503 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.503 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.503 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.503 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.503 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.503 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.503 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.503 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.503 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.503 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:42.503 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:42.503 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.503 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.504 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.504 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.504 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.504 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.504 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.504 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.504 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.504 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.504 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.504 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.504 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:42.504 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:42.504 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.504 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.504 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.504 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:42.505 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.505 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.505 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.505 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.505 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.505 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.505 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.505 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.505 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.505 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.505 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.505 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.505 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.505 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.505 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.506 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:42.506 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:42.506 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.506 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.506 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.506 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.506 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.506 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.506 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.506 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.506 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.506 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.506 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.506 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.507 INFO:teuthology.orchestra.run.smithi097.stdout: "sleep_interval": { 2024-04-23T10:42:42.507 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "sleep_interval", 2024-04-23T10:42:42.507 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-04-23T10:42:42.507 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.507 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.507 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "60", 2024-04-23T10:42:42.507 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.507 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.507 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.507 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.507 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.507 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.507 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.507 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.507 INFO:teuthology.orchestra.run.smithi097.stdout: "threshold": { 2024-04-23T10:42:42.507 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "threshold", 2024-04-23T10:42:42.507 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2024-04-23T10:42:42.508 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.508 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.508 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "3.0", 2024-04-23T10:42:42.508 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "1.0", 2024-04-23T10:42:42.508 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.508 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.508 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "scaling threshold", 2024-04-23T10:42:42.508 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-23T10:42:42.508 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.508 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.508 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.508 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.508 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.508 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.508 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "progress", 2024-04-23T10:42:42.509 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:42.509 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:42.509 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:42.509 INFO:teuthology.orchestra.run.smithi097.stdout: "allow_pg_recovery_event": { 2024-04-23T10:42:42.509 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "allow_pg_recovery_event", 2024-04-23T10:42:42.509 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.509 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.509 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.509 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.509 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.509 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.509 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.509 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-23T10:42:42.509 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.509 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.509 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.510 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.510 INFO:teuthology.orchestra.run.smithi097.stdout: "enabled": { 2024-04-23T10:42:42.510 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "enabled", 2024-04-23T10:42:42.510 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.510 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.510 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.510 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:42.510 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.510 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.510 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.510 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.510 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.510 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.510 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.510 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.511 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:42.511 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:42.511 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.511 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.511 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.511 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.511 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.511 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.511 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.511 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.511 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.511 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.511 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.511 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.511 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.511 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.512 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.512 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.512 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.512 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.512 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.512 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:42.512 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:42.512 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.512 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.512 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.512 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.512 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.512 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.512 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.512 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.512 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.513 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.513 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.513 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.513 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:42.513 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:42.513 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.513 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.513 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.513 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:42.513 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.513 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.513 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.513 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.513 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.513 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.514 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.514 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.514 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.514 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.514 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.514 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.514 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.514 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.514 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.514 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:42.514 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:42.514 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.514 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.514 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.514 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.515 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.515 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.515 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.515 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.515 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.515 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.515 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.515 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.515 INFO:teuthology.orchestra.run.smithi097.stdout: "max_completed_events": { 2024-04-23T10:42:42.515 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "max_completed_events", 2024-04-23T10:42:42.515 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:42.515 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.515 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.515 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "50", 2024-04-23T10:42:42.515 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.515 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.516 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.516 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "number of past completed events to remember", 2024-04-23T10:42:42.516 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.516 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.516 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.516 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.516 INFO:teuthology.orchestra.run.smithi097.stdout: "sleep_interval": { 2024-04-23T10:42:42.516 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "sleep_interval", 2024-04-23T10:42:42.516 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-04-23T10:42:42.516 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.516 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.516 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "5", 2024-04-23T10:42:42.516 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.516 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.516 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.516 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "how long the module is going to sleep", 2024-04-23T10:42:42.517 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.517 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.517 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.517 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.517 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.517 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.517 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.517 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "prometheus", 2024-04-23T10:42:42.517 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:42.517 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:42.517 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:42.517 INFO:teuthology.orchestra.run.smithi097.stdout: "cache": { 2024-04-23T10:42:42.517 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "cache", 2024-04-23T10:42:42.517 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.517 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.517 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.517 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:42.518 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.518 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.518 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.518 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.518 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.518 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.518 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.518 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.518 INFO:teuthology.orchestra.run.smithi097.stdout: "exclude_perf_counters": { 2024-04-23T10:42:42.518 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "exclude_perf_counters", 2024-04-23T10:42:42.518 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.518 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.518 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.518 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:42.518 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.519 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.519 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.519 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-23T10:42:42.519 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-23T10:42:42.519 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.519 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.519 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.519 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:42.519 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:42.519 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.519 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.519 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.519 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.519 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.519 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.519 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.520 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.520 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.520 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.520 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.520 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.520 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.520 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.520 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.520 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.520 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.520 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.520 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.520 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:42.520 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:42.520 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.520 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.520 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.521 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.521 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.521 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.521 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.521 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.521 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.521 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.521 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.521 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.521 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:42.521 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:42.521 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.521 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.521 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.521 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:42.521 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.522 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.522 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.522 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.522 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.522 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.522 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.522 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.522 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.522 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.522 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.522 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.522 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.522 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.522 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.522 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:42.522 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:42.523 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.523 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.523 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.523 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.523 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.523 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.523 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.523 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.523 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.523 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.523 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.523 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.523 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_stats_pools": { 2024-04-23T10:42:42.523 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rbd_stats_pools", 2024-04-23T10:42:42.523 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.523 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.524 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.524 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.524 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.524 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.524 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.524 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.524 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.524 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.524 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.524 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.524 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-23T10:42:42.524 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-23T10:42:42.524 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:42.524 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.524 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.524 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "300", 2024-04-23T10:42:42.525 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.525 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.525 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.525 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.525 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.525 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.525 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.525 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.525 INFO:teuthology.orchestra.run.smithi097.stdout: "scrape_interval": { 2024-04-23T10:42:42.525 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "scrape_interval", 2024-04-23T10:42:42.525 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2024-04-23T10:42:42.525 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.525 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.525 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "15.0", 2024-04-23T10:42:42.525 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.525 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.526 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.526 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.526 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.526 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.526 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.526 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.526 INFO:teuthology.orchestra.run.smithi097.stdout: "server_addr": { 2024-04-23T10:42:42.526 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_addr", 2024-04-23T10:42:42.526 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.526 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.526 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.526 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "::", 2024-04-23T10:42:42.526 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.526 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.526 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.526 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-23T10:42:42.527 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.527 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.527 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.527 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.527 INFO:teuthology.orchestra.run.smithi097.stdout: "server_port": { 2024-04-23T10:42:42.527 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_port", 2024-04-23T10:42:42.527 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:42.527 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.527 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.527 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "9283", 2024-04-23T10:42:42.527 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.527 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.527 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.527 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-23T10:42:42.527 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.527 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.528 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.528 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.528 INFO:teuthology.orchestra.run.smithi097.stdout: "stale_cache_strategy": { 2024-04-23T10:42:42.528 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "stale_cache_strategy", 2024-04-23T10:42:42.528 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.528 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.528 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.528 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "log", 2024-04-23T10:42:42.528 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.528 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.528 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.528 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.528 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.528 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.528 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.529 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.529 INFO:teuthology.orchestra.run.smithi097.stdout: "standby_behaviour": { 2024-04-23T10:42:42.529 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "standby_behaviour", 2024-04-23T10:42:42.529 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.529 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.529 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.529 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "default", 2024-04-23T10:42:42.529 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.529 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.529 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.529 INFO:teuthology.orchestra.run.smithi097.stdout: "default", 2024-04-23T10:42:42.529 INFO:teuthology.orchestra.run.smithi097.stdout: "error" 2024-04-23T10:42:42.529 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.529 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.529 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.529 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.530 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.530 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.530 INFO:teuthology.orchestra.run.smithi097.stdout: "standby_error_status_code": { 2024-04-23T10:42:42.530 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "standby_error_status_code", 2024-04-23T10:42:42.530 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:42.530 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.530 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.530 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "500", 2024-04-23T10:42:42.530 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "400", 2024-04-23T10:42:42.530 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "599", 2024-04-23T10:42:42.530 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.530 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.530 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.530 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.530 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.531 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.531 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.531 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.531 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.531 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rbd_support", 2024-04-23T10:42:42.531 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:42.531 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:42.531 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:42.531 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:42.531 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:42.531 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.531 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.531 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.531 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.531 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.532 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.532 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.532 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.532 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.532 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.532 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.532 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.532 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.532 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.532 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.532 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.532 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.532 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.532 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.532 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:42.532 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:42.533 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.533 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.533 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.533 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.533 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.533 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.533 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.533 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.533 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.533 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.533 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.533 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.533 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:42.533 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:42.533 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.533 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.534 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.534 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:42.534 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.534 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.534 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.534 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.534 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.534 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.534 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.534 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.534 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.534 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.534 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.534 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.534 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.534 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.535 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.535 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:42.535 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:42.535 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.535 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.535 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.535 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.535 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.535 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.535 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.535 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.535 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.535 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.535 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.535 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.535 INFO:teuthology.orchestra.run.smithi097.stdout: "max_concurrent_snap_create": { 2024-04-23T10:42:42.536 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "max_concurrent_snap_create", 2024-04-23T10:42:42.536 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:42.536 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.536 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.536 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10", 2024-04-23T10:42:42.536 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.536 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.536 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.536 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.536 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.536 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.536 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.536 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.536 INFO:teuthology.orchestra.run.smithi097.stdout: "mirror_snapshot_schedule": { 2024-04-23T10:42:42.536 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "mirror_snapshot_schedule", 2024-04-23T10:42:42.536 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.537 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.537 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.537 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.537 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.537 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.537 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.537 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.537 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.537 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.537 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.537 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.537 INFO:teuthology.orchestra.run.smithi097.stdout: "trash_purge_schedule": { 2024-04-23T10:42:42.537 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "trash_purge_schedule", 2024-04-23T10:42:42.537 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.537 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.537 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.538 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.538 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.538 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.538 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.538 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.538 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.538 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.538 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.538 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.538 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.538 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.538 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.538 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "restful", 2024-04-23T10:42:42.538 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:42.538 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:42.539 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:42.539 INFO:teuthology.orchestra.run.smithi097.stdout: "enable_auth": { 2024-04-23T10:42:42.539 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "enable_auth", 2024-04-23T10:42:42.539 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.539 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.539 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.539 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:42.539 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.539 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.539 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.539 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.539 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.539 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.539 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.539 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.539 INFO:teuthology.orchestra.run.smithi097.stdout: "key_file": { 2024-04-23T10:42:42.540 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "key_file", 2024-04-23T10:42:42.540 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.540 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.540 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.540 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.540 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.540 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.540 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.540 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.540 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.540 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.540 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.540 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.540 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:42.540 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:42.540 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.540 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.541 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.541 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.541 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.541 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.541 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.541 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.541 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.541 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.541 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.541 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.541 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.541 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.541 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.541 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.541 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.541 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.542 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.542 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:42.542 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:42.542 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.542 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.542 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.542 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.542 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.542 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.542 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.542 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.542 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.542 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.542 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.542 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.543 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:42.543 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:42.543 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.543 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.543 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.543 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:42.543 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.543 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.543 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.543 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.543 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.543 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.543 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.543 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.543 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.544 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.544 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.544 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.544 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.544 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.544 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.544 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:42.544 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:42.544 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.544 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.544 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.544 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.544 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.544 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.544 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.545 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.545 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.545 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.545 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.545 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.545 INFO:teuthology.orchestra.run.smithi097.stdout: "server_addr": { 2024-04-23T10:42:42.545 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_addr", 2024-04-23T10:42:42.545 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.545 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.545 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.545 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.545 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.545 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.545 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.545 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.545 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.545 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.546 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.546 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.546 INFO:teuthology.orchestra.run.smithi097.stdout: "server_port": { 2024-04-23T10:42:42.546 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "server_port", 2024-04-23T10:42:42.546 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.546 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.546 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.546 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.546 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.546 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.546 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.546 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.546 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.546 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.546 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.547 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.547 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.547 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.547 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.547 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rgw", 2024-04-23T10:42:42.547 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:42.547 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:42.547 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:42.547 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:42.547 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:42.547 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.547 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.547 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.547 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.547 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.547 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.548 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.548 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.548 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.548 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.548 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.548 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.548 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.548 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.548 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.548 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.548 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.548 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.548 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.548 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:42.548 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:42.548 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.548 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.549 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.549 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.549 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.549 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.549 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.549 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.549 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.549 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.549 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.549 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.549 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:42.549 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:42.549 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.549 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.549 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.549 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:42.550 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.550 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.550 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.550 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.550 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.550 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.550 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.550 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.550 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.550 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.550 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.550 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.550 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.550 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.550 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.550 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:42.551 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:42.551 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.551 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.551 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.551 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.551 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.551 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.551 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.551 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.551 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.551 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.551 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.551 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.551 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.551 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.551 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.552 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rook", 2024-04-23T10:42:42.552 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:42.552 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:42.552 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:42.552 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:42.552 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:42.552 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.552 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.552 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.552 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.552 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.552 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.552 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.552 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.552 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.552 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.553 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.553 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.553 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.553 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.553 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.553 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.553 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.553 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.553 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.553 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:42.553 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:42.553 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.553 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.553 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.553 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.553 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.554 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.554 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.554 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.554 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.554 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.554 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.554 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.554 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:42.554 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:42.554 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.554 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.554 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.554 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:42.554 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.554 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.554 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.555 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.555 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.555 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.555 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.555 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.555 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.555 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.555 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.555 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.555 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.555 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.555 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.555 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:42.555 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:42.555 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.555 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.556 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.556 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.556 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.556 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.556 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.556 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.556 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.556 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.556 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.556 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.556 INFO:teuthology.orchestra.run.smithi097.stdout: "storage_class": { 2024-04-23T10:42:42.556 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "storage_class", 2024-04-23T10:42:42.556 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.556 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.556 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.556 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "local", 2024-04-23T10:42:42.557 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.557 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.557 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.557 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-23T10:42:42.557 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.557 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.557 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.557 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.557 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.557 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.557 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.557 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "selftest", 2024-04-23T10:42:42.557 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:42.557 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:42.557 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:42.557 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:42.558 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:42.558 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.558 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.558 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.558 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.558 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.558 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.558 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.558 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.558 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.558 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.558 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.558 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.558 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.558 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.559 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.559 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.559 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.559 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.559 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.559 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:42.559 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:42.559 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.559 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.559 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.559 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.559 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.559 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.559 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.559 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.559 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.560 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.560 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.560 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.560 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:42.560 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:42.560 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.560 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.560 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.560 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:42.560 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.560 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.560 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.560 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.560 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.560 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.560 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.560 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.561 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.561 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.561 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.561 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.561 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.561 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.561 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.561 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:42.561 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:42.561 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.561 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.561 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.561 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.561 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.561 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.561 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.562 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.562 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.562 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.562 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.562 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.562 INFO:teuthology.orchestra.run.smithi097.stdout: "roption1": { 2024-04-23T10:42:42.562 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "roption1", 2024-04-23T10:42:42.562 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.562 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.562 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.562 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.562 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.562 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.562 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.562 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.562 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.563 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.563 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.563 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.563 INFO:teuthology.orchestra.run.smithi097.stdout: "roption2": { 2024-04-23T10:42:42.563 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "roption2", 2024-04-23T10:42:42.563 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.563 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.563 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.563 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "xyz", 2024-04-23T10:42:42.563 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.563 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.563 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.563 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.563 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.563 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.563 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.564 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.564 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption1": { 2024-04-23T10:42:42.564 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption1", 2024-04-23T10:42:42.564 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.564 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.564 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.564 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.564 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.564 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.564 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.564 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.564 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.564 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.564 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.564 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.564 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption2": { 2024-04-23T10:42:42.564 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption2", 2024-04-23T10:42:42.565 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:42.565 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.565 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.565 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.565 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.565 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.565 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.565 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.565 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.565 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.565 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.565 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.565 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption3": { 2024-04-23T10:42:42.565 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption3", 2024-04-23T10:42:42.565 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "float", 2024-04-23T10:42:42.566 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.566 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.566 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.566 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.566 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.566 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.566 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.566 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.566 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.566 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.566 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.566 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption4": { 2024-04-23T10:42:42.566 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption4", 2024-04-23T10:42:42.566 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.566 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.566 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.567 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.567 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.567 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.567 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.567 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.567 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.567 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.567 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.567 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.567 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption5": { 2024-04-23T10:42:42.567 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption5", 2024-04-23T10:42:42.567 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.567 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.567 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.567 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.567 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.567 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.568 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.568 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.568 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.568 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.568 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.568 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.568 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption6": { 2024-04-23T10:42:42.568 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption6", 2024-04-23T10:42:42.568 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.568 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.568 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.568 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:42.568 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.568 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.568 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.568 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.569 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.569 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.569 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.569 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.569 INFO:teuthology.orchestra.run.smithi097.stdout: "rwoption7": { 2024-04-23T10:42:42.569 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rwoption7", 2024-04-23T10:42:42.569 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:42.569 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.569 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.569 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.569 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "1", 2024-04-23T10:42:42.569 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "42", 2024-04-23T10:42:42.569 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.569 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.569 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.570 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.570 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.570 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.570 INFO:teuthology.orchestra.run.smithi097.stdout: "testkey": { 2024-04-23T10:42:42.570 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "testkey", 2024-04-23T10:42:42.570 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.570 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.570 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.570 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.570 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.570 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.570 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.570 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.570 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.570 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.571 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.571 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.571 INFO:teuthology.orchestra.run.smithi097.stdout: "testlkey": { 2024-04-23T10:42:42.571 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "testlkey", 2024-04-23T10:42:42.571 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.571 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.571 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.571 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.571 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.571 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.571 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.571 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.571 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.571 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.571 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.572 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.572 INFO:teuthology.orchestra.run.smithi097.stdout: "testnewline": { 2024-04-23T10:42:42.572 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "testnewline", 2024-04-23T10:42:42.572 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.572 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.572 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.572 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.572 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.572 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.572 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.572 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.572 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.572 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.572 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.572 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.572 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.573 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.573 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.573 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "snap_schedule", 2024-04-23T10:42:42.573 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:42.573 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:42.573 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:42.573 INFO:teuthology.orchestra.run.smithi097.stdout: "allow_m_granularity": { 2024-04-23T10:42:42.573 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "allow_m_granularity", 2024-04-23T10:42:42.573 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.573 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.573 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.573 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.573 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.573 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.573 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.574 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "allow minute scheduled snapshots", 2024-04-23T10:42:42.574 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.574 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.574 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.574 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.574 INFO:teuthology.orchestra.run.smithi097.stdout: "dump_on_update": { 2024-04-23T10:42:42.574 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "dump_on_update", 2024-04-23T10:42:42.574 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.574 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.574 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.574 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.574 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.574 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.574 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.574 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "dump database to debug log on update", 2024-04-23T10:42:42.574 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.575 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.575 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.575 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.575 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:42.575 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:42.575 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.575 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.575 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.575 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.575 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.575 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.575 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.575 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.575 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.576 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.576 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.576 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.576 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.576 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.576 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.576 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.576 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.576 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.576 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.576 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:42.576 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:42.576 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.576 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.576 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.576 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.577 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.577 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.577 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.577 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.577 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.577 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.577 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.577 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.577 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:42.577 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:42.577 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.577 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.577 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.577 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:42.578 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.578 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.578 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.578 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.578 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.578 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.578 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.578 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.578 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.578 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.578 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.578 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.578 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.578 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.578 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.579 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:42.579 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:42.579 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.579 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.579 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.579 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.579 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.579 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.579 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.579 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.579 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.579 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.579 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.579 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.579 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.580 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.580 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.580 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "stats", 2024-04-23T10:42:42.580 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:42.580 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:42.580 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:42.580 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:42.580 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:42.580 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.580 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.580 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.580 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.580 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.580 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.581 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.581 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.581 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.581 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.581 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.581 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.581 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.581 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.581 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.581 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.581 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.581 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.581 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.581 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:42.582 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:42.582 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.582 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.582 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.582 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.582 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.582 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.582 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.582 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.582 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.582 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.582 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.582 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.582 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:42.583 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:42.583 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.583 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.583 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.583 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:42.583 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.583 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.583 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.583 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.583 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.583 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.583 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.583 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.583 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.583 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.584 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.584 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.584 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.584 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.584 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.584 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:42.584 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:42.584 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.584 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.584 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.584 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.584 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.584 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.584 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.584 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.584 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.585 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.585 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.585 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.585 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.585 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.585 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.585 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "status", 2024-04-23T10:42:42.585 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:42.585 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:42.585 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:42.585 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:42.585 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:42.585 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.585 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.586 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.586 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.586 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.586 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.586 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.586 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.586 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.586 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.586 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.586 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.586 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.586 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.586 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.586 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.586 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.587 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.587 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.587 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:42.587 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:42.587 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.587 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.587 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.587 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.587 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.587 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.587 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.587 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.587 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.587 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.587 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.588 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.588 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:42.588 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:42.588 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.588 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.588 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.588 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:42.588 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.588 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.588 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.588 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.588 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.588 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.588 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.588 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.588 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.589 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.589 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.589 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.589 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.589 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.589 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.589 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:42.589 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:42.589 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.589 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.589 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.589 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.589 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.589 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.589 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.589 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.589 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.590 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.590 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.590 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.590 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.590 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.590 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.590 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "telegraf", 2024-04-23T10:42:42.590 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:42.590 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:42.590 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:42.590 INFO:teuthology.orchestra.run.smithi097.stdout: "address": { 2024-04-23T10:42:42.590 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "address", 2024-04-23T10:42:42.590 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.590 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.591 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.591 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-23T10:42:42.591 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.591 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.591 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.591 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.591 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.591 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.591 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.591 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.591 INFO:teuthology.orchestra.run.smithi097.stdout: "interval": { 2024-04-23T10:42:42.591 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "interval", 2024-04-23T10:42:42.591 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-04-23T10:42:42.591 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.591 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.592 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "15", 2024-04-23T10:42:42.592 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.592 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.592 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.592 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.592 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.592 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.592 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.592 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.592 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:42.592 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:42.592 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.592 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.592 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.592 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.593 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.593 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.593 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.593 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.593 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.593 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.593 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.593 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.593 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.593 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.593 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.593 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.593 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.593 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.593 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.593 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:42.594 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:42.594 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.594 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.594 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.594 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.594 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.594 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.594 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.594 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.594 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.594 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.594 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.594 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.594 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:42.595 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:42.595 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.595 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.595 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.595 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:42.595 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.595 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.595 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.595 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.595 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.595 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.595 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.595 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.595 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.595 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.595 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.596 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.596 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.596 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.596 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.596 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:42.596 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:42.596 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.596 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.596 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.596 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.596 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.596 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.596 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.596 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.596 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.597 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.597 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.597 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.597 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.597 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.597 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.597 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "telemetry", 2024-04-23T10:42:42.597 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:42.597 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:42.597 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:42.597 INFO:teuthology.orchestra.run.smithi097.stdout: "channel_basic": { 2024-04-23T10:42:42.597 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "channel_basic", 2024-04-23T10:42:42.597 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.597 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.597 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.598 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:42.598 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.598 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.598 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.598 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-23T10:42:42.598 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.598 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.598 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.598 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.598 INFO:teuthology.orchestra.run.smithi097.stdout: "channel_crash": { 2024-04-23T10:42:42.598 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "channel_crash", 2024-04-23T10:42:42.598 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.598 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.598 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.599 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:42.599 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.599 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.599 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.599 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-23T10:42:42.599 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.599 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.599 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.599 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.599 INFO:teuthology.orchestra.run.smithi097.stdout: "channel_device": { 2024-04-23T10:42:42.599 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "channel_device", 2024-04-23T10:42:42.599 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.599 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.599 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.599 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:42.599 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.599 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.600 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.600 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-23T10:42:42.600 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.600 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.600 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.600 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.600 INFO:teuthology.orchestra.run.smithi097.stdout: "channel_ident": { 2024-04-23T10:42:42.600 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "channel_ident", 2024-04-23T10:42:42.600 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.600 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.600 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.600 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.600 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.600 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.600 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.600 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-23T10:42:42.601 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.601 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.601 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.601 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.601 INFO:teuthology.orchestra.run.smithi097.stdout: "channel_perf": { 2024-04-23T10:42:42.601 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "channel_perf", 2024-04-23T10:42:42.601 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.601 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.601 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.601 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.601 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.601 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.601 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.601 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-23T10:42:42.601 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.601 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.602 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.602 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.602 INFO:teuthology.orchestra.run.smithi097.stdout: "contact": { 2024-04-23T10:42:42.602 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "contact", 2024-04-23T10:42:42.602 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.602 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.602 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.602 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.602 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.602 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.602 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.602 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.602 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.602 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.602 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.603 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.603 INFO:teuthology.orchestra.run.smithi097.stdout: "description": { 2024-04-23T10:42:42.603 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "description", 2024-04-23T10:42:42.603 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.603 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.603 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.603 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.603 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.603 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.603 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.603 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.603 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.603 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.603 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.603 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.603 INFO:teuthology.orchestra.run.smithi097.stdout: "device_url": { 2024-04-23T10:42:42.604 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "device_url", 2024-04-23T10:42:42.604 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.604 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.604 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.604 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-23T10:42:42.604 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.604 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.604 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.604 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.604 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.604 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.604 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.604 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.604 INFO:teuthology.orchestra.run.smithi097.stdout: "enabled": { 2024-04-23T10:42:42.604 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "enabled", 2024-04-23T10:42:42.605 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.605 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.605 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.605 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.605 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.605 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.605 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.605 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.605 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.605 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.605 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.605 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.605 INFO:teuthology.orchestra.run.smithi097.stdout: "interval": { 2024-04-23T10:42:42.605 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "interval", 2024-04-23T10:42:42.605 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:42.606 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.606 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.606 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "24", 2024-04-23T10:42:42.606 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "8", 2024-04-23T10:42:42.606 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.606 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.606 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.606 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.606 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.606 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.606 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.606 INFO:teuthology.orchestra.run.smithi097.stdout: "last_opt_revision": { 2024-04-23T10:42:42.606 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "last_opt_revision", 2024-04-23T10:42:42.606 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:42.606 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.607 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.607 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "1", 2024-04-23T10:42:42.607 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.607 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.607 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.607 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.607 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.607 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.607 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.607 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.607 INFO:teuthology.orchestra.run.smithi097.stdout: "leaderboard": { 2024-04-23T10:42:42.607 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "leaderboard", 2024-04-23T10:42:42.607 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.607 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.607 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.607 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.608 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.608 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.608 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.608 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.608 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.608 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.608 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.608 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.608 INFO:teuthology.orchestra.run.smithi097.stdout: "leaderboard_description": { 2024-04-23T10:42:42.608 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "leaderboard_description", 2024-04-23T10:42:42.608 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.608 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.608 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.608 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.608 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.608 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.609 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.609 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.609 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.609 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.609 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.609 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.609 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:42.609 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:42.609 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.609 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.609 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.609 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.609 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.609 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.609 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.610 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.610 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.610 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.610 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.610 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.610 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.610 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.610 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.610 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.610 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.610 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.610 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.610 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:42.610 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:42.610 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.611 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.611 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.611 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.611 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.611 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.611 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.611 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.611 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.611 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.611 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.611 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.611 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:42.611 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:42.611 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.611 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.611 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.612 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:42.612 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.612 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.612 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.612 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.612 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.612 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.612 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.612 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.612 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.612 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.612 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.612 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.612 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.612 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.612 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.613 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:42.613 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:42.613 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.613 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.613 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.613 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.613 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.613 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.613 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.613 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.613 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.613 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.613 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.613 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.613 INFO:teuthology.orchestra.run.smithi097.stdout: "organization": { 2024-04-23T10:42:42.613 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "organization", 2024-04-23T10:42:42.613 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.614 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.614 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.614 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.614 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.614 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.614 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.614 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.614 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.614 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.614 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.614 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.614 INFO:teuthology.orchestra.run.smithi097.stdout: "proxy": { 2024-04-23T10:42:42.614 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "proxy", 2024-04-23T10:42:42.614 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.615 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.615 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.615 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.615 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.615 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.615 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.615 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.615 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.615 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.615 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.615 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.615 INFO:teuthology.orchestra.run.smithi097.stdout: "url": { 2024-04-23T10:42:42.615 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "url", 2024-04-23T10:42:42.615 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.616 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.616 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.616 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-23T10:42:42.616 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.616 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.616 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.616 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.616 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.616 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.616 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.616 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.616 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.616 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.616 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.616 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "test_orchestrator", 2024-04-23T10:42:42.617 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:42.617 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:42.617 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:42.617 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:42.617 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:42.617 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.617 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.617 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.617 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.617 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.617 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.617 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.617 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.617 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.617 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.617 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.618 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.618 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.618 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.618 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.618 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.618 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.618 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.618 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.618 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:42.618 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:42.618 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.618 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.618 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.618 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.618 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.619 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.619 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.619 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.619 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.619 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.619 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.619 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.619 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:42.619 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:42.619 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.619 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.619 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.619 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:42.619 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.619 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.619 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.620 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.620 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.620 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.620 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.620 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.620 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.620 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.620 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.620 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.620 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.620 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.620 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.620 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:42.620 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:42.620 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.620 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.620 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.621 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.621 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.621 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.621 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.621 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.621 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.621 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.621 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.621 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.621 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.621 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.621 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.621 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "volumes", 2024-04-23T10:42:42.621 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:42.621 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:42.621 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:42.622 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:42.622 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:42.622 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.622 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.622 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.622 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.622 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.622 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.622 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.622 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.622 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.622 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.622 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.622 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.622 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.623 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.623 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.623 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.623 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.623 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.623 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.623 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:42.623 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:42.623 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.623 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.623 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.623 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.623 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.623 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.623 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.623 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.624 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.624 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.624 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.624 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.624 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:42.624 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:42.624 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.624 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.624 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.624 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:42.624 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.624 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.624 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.624 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.624 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.625 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.625 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.625 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.625 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.625 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.625 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.625 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.625 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.625 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.625 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.625 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:42.625 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:42.625 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.625 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.625 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.625 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.625 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.626 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.626 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.626 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.626 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.626 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.626 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.626 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.626 INFO:teuthology.orchestra.run.smithi097.stdout: "max_concurrent_clones": { 2024-04-23T10:42:42.626 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "max_concurrent_clones", 2024-04-23T10:42:42.626 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:42.626 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.626 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.626 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "4", 2024-04-23T10:42:42.626 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.626 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.627 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.627 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-23T10:42:42.627 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.627 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.627 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.627 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.627 INFO:teuthology.orchestra.run.smithi097.stdout: "periodic_async_work": { 2024-04-23T10:42:42.627 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "periodic_async_work", 2024-04-23T10:42:42.627 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.627 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.627 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.627 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.627 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.627 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.627 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.627 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Periodically check for async work", 2024-04-23T10:42:42.628 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.628 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.628 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.628 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.628 INFO:teuthology.orchestra.run.smithi097.stdout: "snapshot_clone_delay": { 2024-04-23T10:42:42.628 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "snapshot_clone_delay", 2024-04-23T10:42:42.628 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:42.628 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.628 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.628 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "0", 2024-04-23T10:42:42.628 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.628 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.628 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.628 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-23T10:42:42.628 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.628 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.629 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.629 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.629 INFO:teuthology.orchestra.run.smithi097.stdout: "snapshot_clone_no_wait": { 2024-04-23T10:42:42.629 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "snapshot_clone_no_wait", 2024-04-23T10:42:42.629 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.629 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.629 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.629 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "True", 2024-04-23T10:42:42.629 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.629 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.629 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.629 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-23T10:42:42.629 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.629 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.629 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.630 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.630 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.630 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.630 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.630 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "zabbix", 2024-04-23T10:42:42.630 INFO:teuthology.orchestra.run.smithi097.stdout: "can_run": true, 2024-04-23T10:42:42.630 INFO:teuthology.orchestra.run.smithi097.stdout: "error_string": "", 2024-04-23T10:42:42.630 INFO:teuthology.orchestra.run.smithi097.stdout: "module_options": { 2024-04-23T10:42:42.630 INFO:teuthology.orchestra.run.smithi097.stdout: "discovery_interval": { 2024-04-23T10:42:42.630 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "discovery_interval", 2024-04-23T10:42:42.630 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "uint", 2024-04-23T10:42:42.630 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.630 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.630 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "100", 2024-04-23T10:42:42.630 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.631 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.631 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.631 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.631 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.631 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.631 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.631 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.631 INFO:teuthology.orchestra.run.smithi097.stdout: "identifier": { 2024-04-23T10:42:42.631 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "identifier", 2024-04-23T10:42:42.631 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.631 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.631 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.631 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.631 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.631 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.632 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.632 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.632 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.632 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.632 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.632 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.632 INFO:teuthology.orchestra.run.smithi097.stdout: "interval": { 2024-04-23T10:42:42.632 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "interval", 2024-04-23T10:42:42.632 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "secs", 2024-04-23T10:42:42.632 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.632 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.632 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "60", 2024-04-23T10:42:42.632 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.632 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.632 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.633 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.633 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.633 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.633 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.633 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.633 INFO:teuthology.orchestra.run.smithi097.stdout: "log_level": { 2024-04-23T10:42:42.633 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_level", 2024-04-23T10:42:42.633 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.633 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.633 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.633 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.633 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.633 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.633 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.633 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.633 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.634 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.634 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.634 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.634 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.634 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.634 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.634 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.634 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.634 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.634 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.634 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster": { 2024-04-23T10:42:42.634 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster", 2024-04-23T10:42:42.634 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.634 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.634 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.635 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.635 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.635 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.635 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.635 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.635 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.635 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.635 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.635 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.635 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_cluster_level": { 2024-04-23T10:42:42.635 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_cluster_level", 2024-04-23T10:42:42.635 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.635 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.635 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.635 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "info", 2024-04-23T10:42:42.635 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.636 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.636 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [ 2024-04-23T10:42:42.636 INFO:teuthology.orchestra.run.smithi097.stdout: "", 2024-04-23T10:42:42.636 INFO:teuthology.orchestra.run.smithi097.stdout: "critical", 2024-04-23T10:42:42.636 INFO:teuthology.orchestra.run.smithi097.stdout: "debug", 2024-04-23T10:42:42.636 INFO:teuthology.orchestra.run.smithi097.stdout: "error", 2024-04-23T10:42:42.636 INFO:teuthology.orchestra.run.smithi097.stdout: "info", 2024-04-23T10:42:42.636 INFO:teuthology.orchestra.run.smithi097.stdout: "warning" 2024-04-23T10:42:42.636 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.636 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.636 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.636 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.636 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.636 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.636 INFO:teuthology.orchestra.run.smithi097.stdout: "log_to_file": { 2024-04-23T10:42:42.636 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "log_to_file", 2024-04-23T10:42:42.637 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "bool", 2024-04-23T10:42:42.637 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.637 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 1, 2024-04-23T10:42:42.637 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "False", 2024-04-23T10:42:42.637 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.637 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.637 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.637 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.637 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.637 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.637 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.637 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.637 INFO:teuthology.orchestra.run.smithi097.stdout: "zabbix_host": { 2024-04-23T10:42:42.637 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "zabbix_host", 2024-04-23T10:42:42.637 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.638 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.638 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.638 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "", 2024-04-23T10:42:42.638 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.638 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.638 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.638 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.638 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.638 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.638 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.638 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.638 INFO:teuthology.orchestra.run.smithi097.stdout: "zabbix_port": { 2024-04-23T10:42:42.638 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "zabbix_port", 2024-04-23T10:42:42.638 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "int", 2024-04-23T10:42:42.639 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.639 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.639 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "10051", 2024-04-23T10:42:42.639 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.639 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.639 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.639 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.639 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.639 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.639 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.639 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.639 INFO:teuthology.orchestra.run.smithi097.stdout: "zabbix_sender": { 2024-04-23T10:42:42.639 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "zabbix_sender", 2024-04-23T10:42:42.639 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "str", 2024-04-23T10:42:42.639 INFO:teuthology.orchestra.run.smithi097.stdout: "level": "advanced", 2024-04-23T10:42:42.639 INFO:teuthology.orchestra.run.smithi097.stdout: "flags": 0, 2024-04-23T10:42:42.640 INFO:teuthology.orchestra.run.smithi097.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-23T10:42:42.640 INFO:teuthology.orchestra.run.smithi097.stdout: "min": "", 2024-04-23T10:42:42.640 INFO:teuthology.orchestra.run.smithi097.stdout: "max": "", 2024-04-23T10:42:42.640 INFO:teuthology.orchestra.run.smithi097.stdout: "enum_allowed": [], 2024-04-23T10:42:42.640 INFO:teuthology.orchestra.run.smithi097.stdout: "desc": "", 2024-04-23T10:42:42.640 INFO:teuthology.orchestra.run.smithi097.stdout: "long_desc": "", 2024-04-23T10:42:42.640 INFO:teuthology.orchestra.run.smithi097.stdout: "tags": [], 2024-04-23T10:42:42.640 INFO:teuthology.orchestra.run.smithi097.stdout: "see_also": [] 2024-04-23T10:42:42.640 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.640 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.640 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.640 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.640 INFO:teuthology.orchestra.run.smithi097.stdout: "services": { 2024-04-23T10:42:42.640 INFO:teuthology.orchestra.run.smithi097.stdout: "dashboard": "https://172.21.15.97:8443/" 2024-04-23T10:42:42.640 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.641 INFO:teuthology.orchestra.run.smithi097.stdout: "always_on_modules": { 2024-04-23T10:42:42.641 INFO:teuthology.orchestra.run.smithi097.stdout: "octopus": [ 2024-04-23T10:42:42.641 INFO:teuthology.orchestra.run.smithi097.stdout: "balancer", 2024-04-23T10:42:42.641 INFO:teuthology.orchestra.run.smithi097.stdout: "crash", 2024-04-23T10:42:42.641 INFO:teuthology.orchestra.run.smithi097.stdout: "devicehealth", 2024-04-23T10:42:42.641 INFO:teuthology.orchestra.run.smithi097.stdout: "orchestrator", 2024-04-23T10:42:42.641 INFO:teuthology.orchestra.run.smithi097.stdout: "pg_autoscaler", 2024-04-23T10:42:42.641 INFO:teuthology.orchestra.run.smithi097.stdout: "progress", 2024-04-23T10:42:42.641 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_support", 2024-04-23T10:42:42.641 INFO:teuthology.orchestra.run.smithi097.stdout: "status", 2024-04-23T10:42:42.641 INFO:teuthology.orchestra.run.smithi097.stdout: "telemetry", 2024-04-23T10:42:42.641 INFO:teuthology.orchestra.run.smithi097.stdout: "volumes" 2024-04-23T10:42:42.641 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.641 INFO:teuthology.orchestra.run.smithi097.stdout: "pacific": [ 2024-04-23T10:42:42.641 INFO:teuthology.orchestra.run.smithi097.stdout: "balancer", 2024-04-23T10:42:42.641 INFO:teuthology.orchestra.run.smithi097.stdout: "crash", 2024-04-23T10:42:42.642 INFO:teuthology.orchestra.run.smithi097.stdout: "devicehealth", 2024-04-23T10:42:42.642 INFO:teuthology.orchestra.run.smithi097.stdout: "orchestrator", 2024-04-23T10:42:42.642 INFO:teuthology.orchestra.run.smithi097.stdout: "pg_autoscaler", 2024-04-23T10:42:42.642 INFO:teuthology.orchestra.run.smithi097.stdout: "progress", 2024-04-23T10:42:42.642 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_support", 2024-04-23T10:42:42.642 INFO:teuthology.orchestra.run.smithi097.stdout: "status", 2024-04-23T10:42:42.642 INFO:teuthology.orchestra.run.smithi097.stdout: "telemetry", 2024-04-23T10:42:42.642 INFO:teuthology.orchestra.run.smithi097.stdout: "volumes" 2024-04-23T10:42:42.642 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.642 INFO:teuthology.orchestra.run.smithi097.stdout: "quincy": [ 2024-04-23T10:42:42.642 INFO:teuthology.orchestra.run.smithi097.stdout: "balancer", 2024-04-23T10:42:42.642 INFO:teuthology.orchestra.run.smithi097.stdout: "crash", 2024-04-23T10:42:42.642 INFO:teuthology.orchestra.run.smithi097.stdout: "devicehealth", 2024-04-23T10:42:42.642 INFO:teuthology.orchestra.run.smithi097.stdout: "orchestrator", 2024-04-23T10:42:42.643 INFO:teuthology.orchestra.run.smithi097.stdout: "pg_autoscaler", 2024-04-23T10:42:42.643 INFO:teuthology.orchestra.run.smithi097.stdout: "progress", 2024-04-23T10:42:42.643 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_support", 2024-04-23T10:42:42.643 INFO:teuthology.orchestra.run.smithi097.stdout: "status", 2024-04-23T10:42:42.643 INFO:teuthology.orchestra.run.smithi097.stdout: "telemetry", 2024-04-23T10:42:42.643 INFO:teuthology.orchestra.run.smithi097.stdout: "volumes" 2024-04-23T10:42:42.643 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.643 INFO:teuthology.orchestra.run.smithi097.stdout: "reef": [ 2024-04-23T10:42:42.643 INFO:teuthology.orchestra.run.smithi097.stdout: "balancer", 2024-04-23T10:42:42.643 INFO:teuthology.orchestra.run.smithi097.stdout: "crash", 2024-04-23T10:42:42.643 INFO:teuthology.orchestra.run.smithi097.stdout: "devicehealth", 2024-04-23T10:42:42.643 INFO:teuthology.orchestra.run.smithi097.stdout: "orchestrator", 2024-04-23T10:42:42.643 INFO:teuthology.orchestra.run.smithi097.stdout: "pg_autoscaler", 2024-04-23T10:42:42.643 INFO:teuthology.orchestra.run.smithi097.stdout: "progress", 2024-04-23T10:42:42.643 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_support", 2024-04-23T10:42:42.643 INFO:teuthology.orchestra.run.smithi097.stdout: "status", 2024-04-23T10:42:42.643 INFO:teuthology.orchestra.run.smithi097.stdout: "telemetry", 2024-04-23T10:42:42.644 INFO:teuthology.orchestra.run.smithi097.stdout: "volumes" 2024-04-23T10:42:42.644 INFO:teuthology.orchestra.run.smithi097.stdout: ], 2024-04-23T10:42:42.644 INFO:teuthology.orchestra.run.smithi097.stdout: "squid": [ 2024-04-23T10:42:42.644 INFO:teuthology.orchestra.run.smithi097.stdout: "balancer", 2024-04-23T10:42:42.644 INFO:teuthology.orchestra.run.smithi097.stdout: "crash", 2024-04-23T10:42:42.644 INFO:teuthology.orchestra.run.smithi097.stdout: "devicehealth", 2024-04-23T10:42:42.644 INFO:teuthology.orchestra.run.smithi097.stdout: "orchestrator", 2024-04-23T10:42:42.644 INFO:teuthology.orchestra.run.smithi097.stdout: "pg_autoscaler", 2024-04-23T10:42:42.644 INFO:teuthology.orchestra.run.smithi097.stdout: "progress", 2024-04-23T10:42:42.644 INFO:teuthology.orchestra.run.smithi097.stdout: "rbd_support", 2024-04-23T10:42:42.644 INFO:teuthology.orchestra.run.smithi097.stdout: "status", 2024-04-23T10:42:42.644 INFO:teuthology.orchestra.run.smithi097.stdout: "telemetry", 2024-04-23T10:42:42.644 INFO:teuthology.orchestra.run.smithi097.stdout: "volumes" 2024-04-23T10:42:42.644 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-04-23T10:42:42.644 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.645 INFO:teuthology.orchestra.run.smithi097.stdout: "last_failure_osd_epoch": 89, 2024-04-23T10:42:42.645 INFO:teuthology.orchestra.run.smithi097.stdout: "active_clients": [ 2024-04-23T10:42:42.645 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.645 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "devicehealth", 2024-04-23T10:42:42.645 INFO:teuthology.orchestra.run.smithi097.stdout: "addrvec": [ 2024-04-23T10:42:42.645 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.645 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v2", 2024-04-23T10:42:42.645 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:0", 2024-04-23T10:42:42.645 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 2146640166 2024-04-23T10:42:42.645 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.645 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-04-23T10:42:42.645 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.645 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.645 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "libcephsqlite", 2024-04-23T10:42:42.645 INFO:teuthology.orchestra.run.smithi097.stdout: "addrvec": [ 2024-04-23T10:42:42.645 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.646 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v2", 2024-04-23T10:42:42.646 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:0", 2024-04-23T10:42:42.646 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 232784572 2024-04-23T10:42:42.646 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.646 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-04-23T10:42:42.646 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.646 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.646 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "rbd_support", 2024-04-23T10:42:42.646 INFO:teuthology.orchestra.run.smithi097.stdout: "addrvec": [ 2024-04-23T10:42:42.646 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.646 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v2", 2024-04-23T10:42:42.646 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:0", 2024-04-23T10:42:42.646 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 814158414 2024-04-23T10:42:42.646 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.646 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-04-23T10:42:42.647 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.647 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.647 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "snap_schedule", 2024-04-23T10:42:42.647 INFO:teuthology.orchestra.run.smithi097.stdout: "addrvec": [ 2024-04-23T10:42:42.647 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.647 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v2", 2024-04-23T10:42:42.647 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:0", 2024-04-23T10:42:42.647 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 931342509 2024-04-23T10:42:42.647 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.647 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-04-23T10:42:42.647 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.647 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.647 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "snap_schedule", 2024-04-23T10:42:42.647 INFO:teuthology.orchestra.run.smithi097.stdout: "addrvec": [ 2024-04-23T10:42:42.647 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.647 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v2", 2024-04-23T10:42:42.648 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:0", 2024-04-23T10:42:42.648 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 1522860733 2024-04-23T10:42:42.648 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.648 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-04-23T10:42:42.648 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:42.648 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.648 INFO:teuthology.orchestra.run.smithi097.stdout: "name": "volumes", 2024-04-23T10:42:42.648 INFO:teuthology.orchestra.run.smithi097.stdout: "addrvec": [ 2024-04-23T10:42:42.648 INFO:teuthology.orchestra.run.smithi097.stdout: { 2024-04-23T10:42:42.648 INFO:teuthology.orchestra.run.smithi097.stdout: "type": "v2", 2024-04-23T10:42:42.648 INFO:teuthology.orchestra.run.smithi097.stdout: "addr": "172.21.15.97:0", 2024-04-23T10:42:42.648 INFO:teuthology.orchestra.run.smithi097.stdout: "nonce": 519366653 2024-04-23T10:42:42.648 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.648 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-04-23T10:42:42.648 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:42.648 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-04-23T10:42:42.649 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-23T10:42:42.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:42:41 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/465067942' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T10:42:42.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:42:41 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/1256091128' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T10:42:42.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:42:41 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/3727061661' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T10:42:42.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:42:41 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/2289655586' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-23T10:42:42.650 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:42:41 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/465067942' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T10:42:42.650 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:42:41 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/1256091128' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T10:42:42.650 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:42:41 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/3727061661' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T10:42:42.650 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:42:41 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/2289655586' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-23T10:42:42.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:41.468+0000 7fdea218e640 1 -- 172.21.15.97:0/2289655586 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fde880a3d20 msgr2=0x7fde880a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:42.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:41.468+0000 7fdea218e640 1 --2- 172.21.15.97:0/2289655586 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fde880a3d20 0x7fde880a61e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fde8c005fd0 tx=0x7fde8c00b390 comp rx=0 tx=0).stop 2024-04-23T10:42:42.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:41.468+0000 7fdea218e640 1 -- 172.21.15.97:0/2289655586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde9c10c520 msgr2=0x7fde9c19a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:42:42.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:41.468+0000 7fdea218e640 1 --2- 172.21.15.97:0/2289655586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde9c10c520 0x7fde9c19a6c0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fde9400cce0 tx=0x7fde9400a620 comp rx=0 tx=0).stop 2024-04-23T10:42:42.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:41.468+0000 7fdea218e640 1 -- 172.21.15.97:0/2289655586 shutdown_connections 2024-04-23T10:42:42.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:41.468+0000 7fdea218e640 1 --2- 172.21.15.97:0/2289655586 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fde880a3d20 0x7fde880a61e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:42.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:41.468+0000 7fdea218e640 1 --2- 172.21.15.97:0/2289655586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde9c10c520 0x7fde9c19a6c0 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:42.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:41.468+0000 7fdea218e640 1 --2- 172.21.15.97:0/2289655586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fde9c103b00 0x7fde9c1a0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:42.651 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:41.468+0000 7fdea218e640 1 --2- 172.21.15.97:0/2289655586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde9c1031e0 0x7fde9c1a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:42:42.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:41.468+0000 7fdea218e640 1 -- 172.21.15.97:0/2289655586 >> 172.21.15.97:0/2289655586 conn(0x7fde9c077f50 msgr2=0x7fde9c108490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:42:42.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:41.469+0000 7fdea218e640 1 -- 172.21.15.97:0/2289655586 shutdown_connections 2024-04-23T10:42:42.652 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:42:41.469+0000 7fdea218e640 1 -- 172.21.15.97:0/2289655586 wait complete. 2024-04-23T10:42:42.656 DEBUG:tasks.check_counter:Getting stats from a 2024-04-23T10:42:42.657 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-23T10:42:42.657 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:42:41 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/465067942' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T10:42:42.657 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:42:41 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/1256091128' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T10:42:42.657 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:42:41 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/3727061661' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T10:42:42.657 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:42:41 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/2289655586' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-23T10:42:42.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:42:42 smithi196 ceph-mon[32919]: pgmap v518: 97 pgs: 97 active+clean; 4.3 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-04-23T10:42:42.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:42:42 smithi097 ceph-mon[29197]: pgmap v518: 97 pgs: 97 active+clean; 4.3 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-04-23T10:42:42.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:42:42 smithi183 ceph-mon[32317]: pgmap v518: 97 pgs: 97 active+clean; 4.3 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-04-23T10:42:42.908 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T10:42:44.715 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:42:44 smithi097 ceph-mon[29197]: pgmap v519: 97 pgs: 97 active+clean; 4.4 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 38 KiB/s wr, 20 op/s 2024-04-23T10:42:44.717 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-23T10:42:44.717 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-0": { 2024-04-23T10:42:44.718 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 1451, 2024-04-23T10:42:44.718 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 1439, 2024-04-23T10:42:44.718 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 2025952, 2024-04-23T10:42:44.718 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 543380, 2024-04-23T10:42:44.718 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 5, 2024-04-23T10:42:44.718 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 1, 2024-04-23T10:42:44.718 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.196397462, 2024-04-23T10:42:44.718 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.066409795, 2024-04-23T10:42:44.719 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.072592878, 2024-04-23T10:42:44.719 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.051627796, 2024-04-23T10:42:44.719 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T10:42:44.719 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1438, 2024-04-23T10:42:44.719 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.074801009, 2024-04-23T10:42:44.719 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000052017 2024-04-23T10:42:44.719 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:44.719 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-23T10:42:44.719 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:44.719 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:44.719 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:44.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:44.720 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 1652992, 2024-04-23T10:42:44.720 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 190400 2024-04-23T10:42:44.720 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:44.720 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-1": { 2024-04-23T10:42:44.720 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 1042, 2024-04-23T10:42:44.720 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 1268, 2024-04-23T10:42:44.720 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 258868, 2024-04-23T10:42:44.721 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 903964, 2024-04-23T10:42:44.721 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 8, 2024-04-23T10:42:44.721 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 5, 2024-04-23T10:42:44.721 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.133184214, 2024-04-23T10:42:44.721 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.043514552, 2024-04-23T10:42:44.721 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.031177335, 2024-04-23T10:42:44.721 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.046303319, 2024-04-23T10:42:44.721 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T10:42:44.721 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1266, 2024-04-23T10:42:44.721 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.057123138, 2024-04-23T10:42:44.721 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000045120 2024-04-23T10:42:44.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:44.722 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-23T10:42:44.722 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:44.722 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:44.722 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:44.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:44.722 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 3360, 2024-04-23T10:42:44.722 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 574448 2024-04-23T10:42:44.722 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:44.723 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-2": { 2024-04-23T10:42:44.723 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 1117, 2024-04-23T10:42:44.723 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 1119, 2024-04-23T10:42:44.723 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 295174, 2024-04-23T10:42:44.723 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 355207, 2024-04-23T10:42:44.723 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 5, 2024-04-23T10:42:44.723 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 3, 2024-04-23T10:42:44.723 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.112617622, 2024-04-23T10:42:44.723 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.036836969, 2024-04-23T10:42:44.723 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.030648162, 2024-04-23T10:42:44.724 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.034873541, 2024-04-23T10:42:44.724 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T10:42:44.724 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1117, 2024-04-23T10:42:44.724 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.043352149, 2024-04-23T10:42:44.724 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000038811 2024-04-23T10:42:44.724 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:44.724 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-23T10:42:44.724 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:44.724 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:44.724 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:44.724 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:44.725 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-23T10:42:44.725 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 0 2024-04-23T10:42:44.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:44.725 INFO:teuthology.orchestra.run.smithi097.stdout: "cct": { 2024-04-23T10:42:44.725 INFO:teuthology.orchestra.run.smithi097.stdout: "total_workers": 1, 2024-04-23T10:42:44.725 INFO:teuthology.orchestra.run.smithi097.stdout: "unhealthy_workers": 0 2024-04-23T10:42:44.725 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:44.725 INFO:teuthology.orchestra.run.smithi097.stdout: "finisher-MDSRank": { 2024-04-23T10:42:44.725 INFO:teuthology.orchestra.run.smithi097.stdout: "queue_len": 0, 2024-04-23T10:42:44.725 INFO:teuthology.orchestra.run.smithi097.stdout: "complete_latency": { 2024-04-23T10:42:44.726 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 5140, 2024-04-23T10:42:44.726 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.167719560, 2024-04-23T10:42:44.726 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000032630 2024-04-23T10:42:44.726 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:44.726 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:44.726 INFO:teuthology.orchestra.run.smithi097.stdout: "finisher-PurgeQueue": { 2024-04-23T10:42:44.726 INFO:teuthology.orchestra.run.smithi097.stdout: "queue_len": 0, 2024-04-23T10:42:44.726 INFO:teuthology.orchestra.run.smithi097.stdout: "complete_latency": { 2024-04-23T10:42:44.726 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:44.726 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:44.726 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:44.727 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:44.727 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:44.727 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": { 2024-04-23T10:42:44.727 INFO:teuthology.orchestra.run.smithi097.stdout: "request": 0, 2024-04-23T10:42:44.727 INFO:teuthology.orchestra.run.smithi097.stdout: "reply": 0, 2024-04-23T10:42:44.727 INFO:teuthology.orchestra.run.smithi097.stdout: "reply_latency": { 2024-04-23T10:42:44.727 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:44.727 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:44.727 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:44.727 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:44.728 INFO:teuthology.orchestra.run.smithi097.stdout: "slow_reply": 0, 2024-04-23T10:42:44.728 INFO:teuthology.orchestra.run.smithi097.stdout: "forward": 0, 2024-04-23T10:42:44.728 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_fetch_complete": 0, 2024-04-23T10:42:44.728 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_fetch_keys": 0, 2024-04-23T10:42:44.728 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_commit": 0, 2024-04-23T10:42:44.728 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_split": 0, 2024-04-23T10:42:44.728 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_merge": 0, 2024-04-23T10:42:44.728 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes": 0, 2024-04-23T10:42:44.728 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_top": 0, 2024-04-23T10:42:44.728 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_bottom": 0, 2024-04-23T10:42:44.729 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_pin_tail": 0, 2024-04-23T10:42:44.729 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_pinned": 0, 2024-04-23T10:42:44.729 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_expired": 0, 2024-04-23T10:42:44.729 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_with_caps": 0, 2024-04-23T10:42:44.729 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": 0, 2024-04-23T10:42:44.729 INFO:teuthology.orchestra.run.smithi097.stdout: "subtrees": 1, 2024-04-23T10:42:44.729 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse": 0, 2024-04-23T10:42:44.729 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_hit": 0, 2024-04-23T10:42:44.729 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_forward": 0, 2024-04-23T10:42:44.729 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_discover": 0, 2024-04-23T10:42:44.729 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_dir_fetch": 0, 2024-04-23T10:42:44.730 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_remote_ino": 0, 2024-04-23T10:42:44.730 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_lock": 0, 2024-04-23T10:42:44.730 INFO:teuthology.orchestra.run.smithi097.stdout: "load_cent": 0, 2024-04-23T10:42:44.730 INFO:teuthology.orchestra.run.smithi097.stdout: "q": 0, 2024-04-23T10:42:44.730 INFO:teuthology.orchestra.run.smithi097.stdout: "exported": 0, 2024-04-23T10:42:44.730 INFO:teuthology.orchestra.run.smithi097.stdout: "exported_inodes": 0, 2024-04-23T10:42:44.730 INFO:teuthology.orchestra.run.smithi097.stdout: "imported": 0, 2024-04-23T10:42:44.730 INFO:teuthology.orchestra.run.smithi097.stdout: "imported_inodes": 0, 2024-04-23T10:42:44.730 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_dir_fetch": 0, 2024-04-23T10:42:44.730 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_backtrace_fetch": 0, 2024-04-23T10:42:44.730 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_peer_discover": 0, 2024-04-23T10:42:44.731 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rfiles": 0, 2024-04-23T10:42:44.731 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rbytes": 0, 2024-04-23T10:42:44.731 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rsnaps": 0, 2024-04-23T10:42:44.731 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_backtrace_fetch": 0, 2024-04-23T10:42:44.731 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_set_tag": 0, 2024-04-23T10:42:44.731 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_backtrace_repaired": 0, 2024-04-23T10:42:44.731 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_inotable_repaired": 0, 2024-04-23T10:42:44.731 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dir_inodes": 0, 2024-04-23T10:42:44.731 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dir_base_inodes": 0, 2024-04-23T10:42:44.731 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dirfrag_rstats": 0, 2024-04-23T10:42:44.731 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_file_inodes": 0, 2024-04-23T10:42:44.732 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_inode_file_caps": 0, 2024-04-23T10:42:44.732 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_revoke": 0, 2024-04-23T10:42:44.732 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_grant": 0, 2024-04-23T10:42:44.732 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_trunc": 0, 2024-04-23T10:42:44.732 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-23T10:42:44.732 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-23T10:42:44.732 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_caps": 0, 2024-04-23T10:42:44.732 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_caps_dirty": 0, 2024-04-23T10:42:44.732 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_cap_release": 0, 2024-04-23T10:42:44.732 INFO:teuthology.orchestra.run.smithi097.stdout: "process_request_cap_release": 0 2024-04-23T10:42:44.733 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:44.733 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_cache": { 2024-04-23T10:42:44.733 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_update": 0, 2024-04-23T10:42:44.733 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_update_receipt": 0, 2024-04-23T10:42:44.733 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_try_discover": 0, 2024-04-23T10:42:44.733 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_send_discover": 0, 2024-04-23T10:42:44.733 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_handle_discover": 0, 2024-04-23T10:42:44.733 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays": 0, 2024-04-23T10:42:44.733 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays_delayed": 0, 2024-04-23T10:42:44.733 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays_enqueuing": 0, 2024-04-23T10:42:44.733 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_created": 0, 2024-04-23T10:42:44.734 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_enqueued": 0, 2024-04-23T10:42:44.734 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_reintegrated": 0, 2024-04-23T10:42:44.734 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_migrated": 0, 2024-04-23T10:42:44.734 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_processing": 0, 2024-04-23T10:42:44.734 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_enqueued": 0, 2024-04-23T10:42:44.734 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_prioritized": 0, 2024-04-23T10:42:44.734 INFO:teuthology.orchestra.run.smithi097.stdout: "recovery_started": 0, 2024-04-23T10:42:44.734 INFO:teuthology.orchestra.run.smithi097.stdout: "recovery_completed": 0, 2024-04-23T10:42:44.734 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_quiesce_path": 0, 2024-04-23T10:42:44.734 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_quiesce_inode": 0, 2024-04-23T10:42:44.735 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_enqueue_scrub": 0, 2024-04-23T10:42:44.735 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_exportdir": 0, 2024-04-23T10:42:44.735 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_flush": 0, 2024-04-23T10:42:44.735 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_fragmentdir": 0, 2024-04-23T10:42:44.735 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_fragstats": 0, 2024-04-23T10:42:44.735 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_inodestats": 0 2024-04-23T10:42:44.735 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:44.735 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_log": { 2024-04-23T10:42:44.735 INFO:teuthology.orchestra.run.smithi097.stdout: "evlrg": 0, 2024-04-23T10:42:44.735 INFO:teuthology.orchestra.run.smithi097.stdout: "evadd": 0, 2024-04-23T10:42:44.735 INFO:teuthology.orchestra.run.smithi097.stdout: "evex": 0, 2024-04-23T10:42:44.736 INFO:teuthology.orchestra.run.smithi097.stdout: "evtrm": 0, 2024-04-23T10:42:44.736 INFO:teuthology.orchestra.run.smithi097.stdout: "ev": 9, 2024-04-23T10:42:44.736 INFO:teuthology.orchestra.run.smithi097.stdout: "evexg": 0, 2024-04-23T10:42:44.736 INFO:teuthology.orchestra.run.smithi097.stdout: "evexd": 0, 2024-04-23T10:42:44.736 INFO:teuthology.orchestra.run.smithi097.stdout: "segadd": 0, 2024-04-23T10:42:44.736 INFO:teuthology.orchestra.run.smithi097.stdout: "segex": 0, 2024-04-23T10:42:44.736 INFO:teuthology.orchestra.run.smithi097.stdout: "segtrm": 0, 2024-04-23T10:42:44.736 INFO:teuthology.orchestra.run.smithi097.stdout: "seg": 2, 2024-04-23T10:42:44.736 INFO:teuthology.orchestra.run.smithi097.stdout: "segmjr": 2, 2024-04-23T10:42:44.736 INFO:teuthology.orchestra.run.smithi097.stdout: "segexg": 0, 2024-04-23T10:42:44.736 INFO:teuthology.orchestra.run.smithi097.stdout: "segexd": 0, 2024-04-23T10:42:44.736 INFO:teuthology.orchestra.run.smithi097.stdout: "expos": 4194304, 2024-04-23T10:42:44.737 INFO:teuthology.orchestra.run.smithi097.stdout: "wrpos": 4196879, 2024-04-23T10:42:44.737 INFO:teuthology.orchestra.run.smithi097.stdout: "rdpos": 4196738, 2024-04-23T10:42:44.737 INFO:teuthology.orchestra.run.smithi097.stdout: "jlat": { 2024-04-23T10:42:44.737 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:44.737 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:44.737 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:44.737 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:44.737 INFO:teuthology.orchestra.run.smithi097.stdout: "replayed": 9 2024-04-23T10:42:44.737 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:44.737 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_mem": { 2024-04-23T10:42:44.737 INFO:teuthology.orchestra.run.smithi097.stdout: "ino": 3, 2024-04-23T10:42:44.737 INFO:teuthology.orchestra.run.smithi097.stdout: "ino+": 3, 2024-04-23T10:42:44.738 INFO:teuthology.orchestra.run.smithi097.stdout: "ino-": 0, 2024-04-23T10:42:44.738 INFO:teuthology.orchestra.run.smithi097.stdout: "dir": 1, 2024-04-23T10:42:44.738 INFO:teuthology.orchestra.run.smithi097.stdout: "dir+": 1, 2024-04-23T10:42:44.738 INFO:teuthology.orchestra.run.smithi097.stdout: "dir-": 0, 2024-04-23T10:42:44.738 INFO:teuthology.orchestra.run.smithi097.stdout: "dn": 0, 2024-04-23T10:42:44.738 INFO:teuthology.orchestra.run.smithi097.stdout: "dn+": 0, 2024-04-23T10:42:44.738 INFO:teuthology.orchestra.run.smithi097.stdout: "dn-": 0, 2024-04-23T10:42:44.738 INFO:teuthology.orchestra.run.smithi097.stdout: "cap": 0, 2024-04-23T10:42:44.738 INFO:teuthology.orchestra.run.smithi097.stdout: "cap+": 0, 2024-04-23T10:42:44.738 INFO:teuthology.orchestra.run.smithi097.stdout: "cap-": 0, 2024-04-23T10:42:44.739 INFO:teuthology.orchestra.run.smithi097.stdout: "rss": 51004, 2024-04-23T10:42:44.739 INFO:teuthology.orchestra.run.smithi097.stdout: "heap": 215416 2024-04-23T10:42:44.739 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:44.739 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_server": { 2024-04-23T10:42:44.739 INFO:teuthology.orchestra.run.smithi097.stdout: "dispatch_client_request": 0, 2024-04-23T10:42:44.739 INFO:teuthology.orchestra.run.smithi097.stdout: "dispatch_server_request": 0, 2024-04-23T10:42:44.739 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_request": 0, 2024-04-23T10:42:44.739 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_session": 0, 2024-04-23T10:42:44.739 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_peer_request": 0, 2024-04-23T10:42:44.739 INFO:teuthology.orchestra.run.smithi097.stdout: "req_create_latency": { 2024-04-23T10:42:44.739 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:44.739 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:44.740 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:44.740 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:44.740 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getattr_latency": { 2024-04-23T10:42:44.740 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:44.740 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:44.740 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:44.740 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:44.740 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getfilelock_latency": { 2024-04-23T10:42:44.740 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:44.740 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:44.740 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:44.740 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:44.741 INFO:teuthology.orchestra.run.smithi097.stdout: "req_link_latency": { 2024-04-23T10:42:44.741 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:44.741 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:44.741 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:44.741 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:44.741 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookup_latency": { 2024-04-23T10:42:44.741 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:44.741 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:44.741 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:44.741 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:44.741 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookuphash_latency": { 2024-04-23T10:42:44.741 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:44.742 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:44.742 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:44.742 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:44.742 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupino_latency": { 2024-04-23T10:42:44.742 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:44.742 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:44.742 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:44.742 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:44.742 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupname_latency": { 2024-04-23T10:42:44.742 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:44.743 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:44.743 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:44.743 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:44.743 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupparent_latency": { 2024-04-23T10:42:44.743 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:44.743 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:44.743 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:44.743 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:44.743 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupsnap_latency": { 2024-04-23T10:42:44.743 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:44.743 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:44.743 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:44.744 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:44.744 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lssnap_latency": { 2024-04-23T10:42:44.744 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:44.744 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:44.744 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:44.744 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:44.744 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mkdir_latency": { 2024-04-23T10:42:44.744 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:44.745 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:44.745 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:44.745 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:44.745 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mknod_latency": { 2024-04-23T10:42:44.745 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:44.745 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:44.745 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:44.745 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:44.746 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mksnap_latency": { 2024-04-23T10:42:44.746 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:44.746 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:44.746 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:44.746 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:44.746 INFO:teuthology.orchestra.run.smithi097.stdout: "req_open_latency": { 2024-04-23T10:42:44.746 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:44.746 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:44.746 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:44.746 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:44.746 INFO:teuthology.orchestra.run.smithi097.stdout: "req_readdir_latency": { 2024-04-23T10:42:44.747 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:44.747 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:44.747 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:44.747 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:44.747 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rename_latency": { 2024-04-23T10:42:44.747 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:44.747 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:44.747 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:44.747 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:44.747 INFO:teuthology.orchestra.run.smithi097.stdout: "req_renamesnap_latency": { 2024-04-23T10:42:44.747 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:44.748 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:44.748 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:44.748 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:44.748 INFO:teuthology.orchestra.run.smithi097.stdout: "req_snapdiff_latency": { 2024-04-23T10:42:44.748 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:44.748 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:44.748 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:44.748 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:44.748 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmdir_latency": { 2024-04-23T10:42:44.748 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:44.748 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:44.748 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:44.749 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:44.749 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmsnap_latency": { 2024-04-23T10:42:44.749 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:44.749 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:44.749 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:44.749 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:44.749 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmxattr_latency": { 2024-04-23T10:42:44.749 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:44.749 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:44.749 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:44.749 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:44.749 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setattr_latency": { 2024-04-23T10:42:44.750 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:44.750 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:44.750 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:44.750 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:44.750 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setdirlayout_latency": { 2024-04-23T10:42:44.750 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:44.750 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:44.750 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:44.750 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:44.750 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setfilelock_latency": { 2024-04-23T10:42:44.750 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:44.751 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:44.751 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:44.751 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:44.751 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setlayout_latency": { 2024-04-23T10:42:44.751 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:44.751 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:44.751 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:44.751 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:44.751 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setxattr_latency": { 2024-04-23T10:42:44.751 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:44.751 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:44.751 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:44.752 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:44.752 INFO:teuthology.orchestra.run.smithi097.stdout: "req_symlink_latency": { 2024-04-23T10:42:44.752 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:44.752 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:44.752 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:44.752 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:44.752 INFO:teuthology.orchestra.run.smithi097.stdout: "req_unlink_latency": { 2024-04-23T10:42:44.752 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:44.752 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:44.752 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:44.752 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:44.752 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_revoke_eviction": 0, 2024-04-23T10:42:44.753 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_acquisition_throttle": 0, 2024-04-23T10:42:44.753 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getvxattr_latency": { 2024-04-23T10:42:44.753 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:44.753 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:44.753 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:44.753 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:44.753 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:44.753 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_sessions": { 2024-04-23T10:42:44.753 INFO:teuthology.orchestra.run.smithi097.stdout: "session_count": 1, 2024-04-23T10:42:44.753 INFO:teuthology.orchestra.run.smithi097.stdout: "session_add": 4, 2024-04-23T10:42:44.753 INFO:teuthology.orchestra.run.smithi097.stdout: "session_remove": 3, 2024-04-23T10:42:44.754 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions_open": 3, 2024-04-23T10:42:44.754 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions_stale": 0, 2024-04-23T10:42:44.754 INFO:teuthology.orchestra.run.smithi097.stdout: "total_load": 0, 2024-04-23T10:42:44.754 INFO:teuthology.orchestra.run.smithi097.stdout: "average_load": 0, 2024-04-23T10:42:44.754 INFO:teuthology.orchestra.run.smithi097.stdout: "avg_session_uptime": 5969, 2024-04-23T10:42:44.754 INFO:teuthology.orchestra.run.smithi097.stdout: "mdthresh_evicted": 0 2024-04-23T10:42:44.754 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:44.754 INFO:teuthology.orchestra.run.smithi097.stdout: "mempool": { 2024-04-23T10:42:44.754 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_bytes": 0, 2024-04-23T10:42:44.754 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_items": 0, 2024-04-23T10:42:44.754 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_bytes": 0, 2024-04-23T10:42:44.755 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_items": 0, 2024-04-23T10:42:44.755 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_bytes": 0, 2024-04-23T10:42:44.755 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_items": 0, 2024-04-23T10:42:44.755 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-23T10:42:44.755 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_items": 0, 2024-04-23T10:42:44.755 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-23T10:42:44.755 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_items": 0, 2024-04-23T10:42:44.755 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_bytes": 0, 2024-04-23T10:42:44.755 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_items": 0, 2024-04-23T10:42:44.755 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-23T10:42:44.755 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_items": 0, 2024-04-23T10:42:44.755 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_bytes": 0, 2024-04-23T10:42:44.756 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_items": 0, 2024-04-23T10:42:44.756 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_bytes": 0, 2024-04-23T10:42:44.756 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_items": 0, 2024-04-23T10:42:44.756 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-23T10:42:44.756 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_items": 0, 2024-04-23T10:42:44.756 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-23T10:42:44.756 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_items": 0, 2024-04-23T10:42:44.756 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_bytes": 0, 2024-04-23T10:42:44.756 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_items": 0, 2024-04-23T10:42:44.756 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_bytes": 0, 2024-04-23T10:42:44.756 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_items": 0, 2024-04-23T10:42:44.756 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-23T10:42:44.757 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_items": 0, 2024-04-23T10:42:44.757 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_bytes": 0, 2024-04-23T10:42:44.757 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_items": 0, 2024-04-23T10:42:44.757 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_bytes": 0, 2024-04-23T10:42:44.757 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_items": 0, 2024-04-23T10:42:44.757 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_bytes": 0, 2024-04-23T10:42:44.757 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_items": 0, 2024-04-23T10:42:44.757 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_bytes": 0, 2024-04-23T10:42:44.757 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_items": 0, 2024-04-23T10:42:44.757 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_bytes": 190234, 2024-04-23T10:42:44.757 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_items": 55, 2024-04-23T10:42:44.757 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_bytes": 0, 2024-04-23T10:42:44.758 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_items": 0, 2024-04-23T10:42:44.758 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_bytes": 0, 2024-04-23T10:42:44.758 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_items": 0, 2024-04-23T10:42:44.758 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_bytes": 0, 2024-04-23T10:42:44.758 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_items": 0, 2024-04-23T10:42:44.758 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_bytes": 0, 2024-04-23T10:42:44.758 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_items": 0, 2024-04-23T10:42:44.758 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_bytes": 7936, 2024-04-23T10:42:44.758 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_items": 165, 2024-04-23T10:42:44.758 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_bytes": 0, 2024-04-23T10:42:44.758 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_items": 0, 2024-04-23T10:42:44.758 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_bytes": 0, 2024-04-23T10:42:44.759 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_items": 0, 2024-04-23T10:42:44.759 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_bytes": 8512, 2024-04-23T10:42:44.759 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_items": 13, 2024-04-23T10:42:44.759 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_bytes": 0, 2024-04-23T10:42:44.759 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_items": 0, 2024-04-23T10:42:44.759 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_bytes": 0, 2024-04-23T10:42:44.759 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_items": 0 2024-04-23T10:42:44.759 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:44.759 INFO:teuthology.orchestra.run.smithi097.stdout: "objecter": { 2024-04-23T10:42:44.759 INFO:teuthology.orchestra.run.smithi097.stdout: "op_active": 0, 2024-04-23T10:42:44.759 INFO:teuthology.orchestra.run.smithi097.stdout: "op_laggy": 0, 2024-04-23T10:42:44.759 INFO:teuthology.orchestra.run.smithi097.stdout: "op_send": 3263, 2024-04-23T10:42:44.759 INFO:teuthology.orchestra.run.smithi097.stdout: "op_send_bytes": 21, 2024-04-23T10:42:44.760 INFO:teuthology.orchestra.run.smithi097.stdout: "op_resend": 3, 2024-04-23T10:42:44.760 INFO:teuthology.orchestra.run.smithi097.stdout: "op_reply": 3263, 2024-04-23T10:42:44.760 INFO:teuthology.orchestra.run.smithi097.stdout: "op_latency": { 2024-04-23T10:42:44.760 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 3263, 2024-04-23T10:42:44.760 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 2.047962528, 2024-04-23T10:42:44.760 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000627631 2024-04-23T10:42:44.760 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:44.760 INFO:teuthology.orchestra.run.smithi097.stdout: "op_inflight": 0, 2024-04-23T10:42:44.760 INFO:teuthology.orchestra.run.smithi097.stdout: "oplen_avg": { 2024-04-23T10:42:44.760 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 3263, 2024-04-23T10:42:44.760 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 3264 2024-04-23T10:42:44.760 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:44.761 INFO:teuthology.orchestra.run.smithi097.stdout: "op": 3263, 2024-04-23T10:42:44.761 INFO:teuthology.orchestra.run.smithi097.stdout: "op_r": 3263, 2024-04-23T10:42:44.761 INFO:teuthology.orchestra.run.smithi097.stdout: "op_w": 0, 2024-04-23T10:42:44.761 INFO:teuthology.orchestra.run.smithi097.stdout: "op_rmw": 0, 2024-04-23T10:42:44.761 INFO:teuthology.orchestra.run.smithi097.stdout: "op_pg": 0, 2024-04-23T10:42:44.761 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_stat": 2143, 2024-04-23T10:42:44.761 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_create": 0, 2024-04-23T10:42:44.761 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_read": 1118, 2024-04-23T10:42:44.761 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_write": 0, 2024-04-23T10:42:44.761 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_writefull": 0, 2024-04-23T10:42:44.761 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_writesame": 0, 2024-04-23T10:42:44.761 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_append": 0, 2024-04-23T10:42:44.761 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_zero": 0, 2024-04-23T10:42:44.762 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_truncate": 0, 2024-04-23T10:42:44.762 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_delete": 0, 2024-04-23T10:42:44.762 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_mapext": 0, 2024-04-23T10:42:44.762 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_sparse_read": 0, 2024-04-23T10:42:44.762 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_clonerange": 0, 2024-04-23T10:42:44.762 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_getxattr": 1, 2024-04-23T10:42:44.762 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_setxattr": 0, 2024-04-23T10:42:44.762 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_cmpxattr": 0, 2024-04-23T10:42:44.762 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_rmxattr": 0, 2024-04-23T10:42:44.762 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_resetxattrs": 0, 2024-04-23T10:42:44.762 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_call": 0, 2024-04-23T10:42:44.763 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_watch": 0, 2024-04-23T10:42:44.763 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_notify": 0, 2024-04-23T10:42:44.763 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_src_cmpxattr": 0, 2024-04-23T10:42:44.763 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_pgls": 0, 2024-04-23T10:42:44.763 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_pgls_filter": 0, 2024-04-23T10:42:44.763 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_other": 0, 2024-04-23T10:42:44.763 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_active": 0, 2024-04-23T10:42:44.763 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_send": 0, 2024-04-23T10:42:44.763 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_resend": 0, 2024-04-23T10:42:44.763 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_ping": 0, 2024-04-23T10:42:44.763 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_active": 0, 2024-04-23T10:42:44.763 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_send": 0, 2024-04-23T10:42:44.763 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_resend": 0, 2024-04-23T10:42:44.764 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_active": 0, 2024-04-23T10:42:44.764 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_send": 0, 2024-04-23T10:42:44.764 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_resend": 0, 2024-04-23T10:42:44.764 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_active": 0, 2024-04-23T10:42:44.764 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_send": 0, 2024-04-23T10:42:44.764 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_resend": 0, 2024-04-23T10:42:44.764 INFO:teuthology.orchestra.run.smithi097.stdout: "command_active": 0, 2024-04-23T10:42:44.764 INFO:teuthology.orchestra.run.smithi097.stdout: "command_send": 0, 2024-04-23T10:42:44.764 INFO:teuthology.orchestra.run.smithi097.stdout: "command_resend": 0, 2024-04-23T10:42:44.764 INFO:teuthology.orchestra.run.smithi097.stdout: "map_epoch": 109, 2024-04-23T10:42:44.764 INFO:teuthology.orchestra.run.smithi097.stdout: "map_full": 0, 2024-04-23T10:42:44.764 INFO:teuthology.orchestra.run.smithi097.stdout: "map_inc": 23, 2024-04-23T10:42:44.764 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_sessions": 5, 2024-04-23T10:42:44.765 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_session_open": 7, 2024-04-23T10:42:44.765 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_session_close": 2, 2024-04-23T10:42:44.765 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_laggy": 0, 2024-04-23T10:42:44.765 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_wr": 0, 2024-04-23T10:42:44.765 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_rd": 2, 2024-04-23T10:42:44.765 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_del": 0 2024-04-23T10:42:44.765 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:44.765 INFO:teuthology.orchestra.run.smithi097.stdout: "oft": { 2024-04-23T10:42:44.765 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_objs": 0, 2024-04-23T10:42:44.765 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_kv_pairs": 0, 2024-04-23T10:42:44.765 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_updates": 0, 2024-04-23T10:42:44.765 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_removes": 0 2024-04-23T10:42:44.766 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:44.766 INFO:teuthology.orchestra.run.smithi097.stdout: "osd-slow-ops": { 2024-04-23T10:42:44.766 INFO:teuthology.orchestra.run.smithi097.stdout: "slow_ops_count": 0 2024-04-23T10:42:44.766 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:44.766 INFO:teuthology.orchestra.run.smithi097.stdout: "purge_queue": { 2024-04-23T10:42:44.766 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_ops": 0, 2024-04-23T10:42:44.766 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_ops_high_water": 0, 2024-04-23T10:42:44.766 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing": 0, 2024-04-23T10:42:44.766 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_high_water": 0, 2024-04-23T10:42:44.766 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executed_ops": 0, 2024-04-23T10:42:44.766 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executed": 0, 2024-04-23T10:42:44.766 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_item_in_journal": 0 2024-04-23T10:42:44.766 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:44.767 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-23T10:42:44.767 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-23T10:42:44.767 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2024-04-23T10:42:44.767 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-23T10:42:44.767 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 3606, 2024-04-23T10:42:44.767 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 2234940, 2024-04-23T10:42:44.767 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-23T10:42:44.767 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 3606, 2024-04-23T10:42:44.767 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-04-23T10:42:44.767 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-04-23T10:42:44.767 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 3606, 2024-04-23T10:42:44.767 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 2234940, 2024-04-23T10:42:44.767 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-23T10:42:44.768 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:44.768 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:44.768 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:44.768 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:44.768 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:44.768 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-objecter_bytes": { 2024-04-23T10:42:44.768 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-23T10:42:44.768 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2024-04-23T10:42:44.768 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-23T10:42:44.768 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-04-23T10:42:44.768 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-04-23T10:42:44.768 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-23T10:42:44.768 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-04-23T10:42:44.769 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 3263, 2024-04-23T10:42:44.769 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 2580, 2024-04-23T10:42:44.769 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 3263, 2024-04-23T10:42:44.769 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 2580, 2024-04-23T10:42:44.769 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-23T10:42:44.769 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:44.769 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:44.769 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:44.769 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:44.769 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:44.769 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-objecter_ops": { 2024-04-23T10:42:44.769 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-23T10:42:44.769 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 1024, 2024-04-23T10:42:44.770 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-23T10:42:44.770 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-04-23T10:42:44.770 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-04-23T10:42:44.770 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-23T10:42:44.770 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-04-23T10:42:44.770 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 3263, 2024-04-23T10:42:44.770 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 3263, 2024-04-23T10:42:44.770 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 3263, 2024-04-23T10:42:44.770 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 3263, 2024-04-23T10:42:44.770 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-23T10:42:44.770 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:44.770 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:44.771 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:44.771 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:44.771 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:44.771 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-write_buf_throttle": { 2024-04-23T10:42:44.771 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-23T10:42:44.771 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 3758096384, 2024-04-23T10:42:44.771 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-23T10:42:44.771 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-04-23T10:42:44.771 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-04-23T10:42:44.771 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-23T10:42:44.771 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-04-23T10:42:44.771 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-04-23T10:42:44.771 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-04-23T10:42:44.771 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 0, 2024-04-23T10:42:44.772 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 0, 2024-04-23T10:42:44.772 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-23T10:42:44.772 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:44.772 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:44.772 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:44.772 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:44.772 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:44.772 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-write_buf_throttle-0x5595338606e0": { 2024-04-23T10:42:44.772 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-23T10:42:44.772 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 3758096384, 2024-04-23T10:42:44.772 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-23T10:42:44.772 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-04-23T10:42:44.773 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-04-23T10:42:44.773 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-23T10:42:44.773 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-04-23T10:42:44.773 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-04-23T10:42:44.773 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-04-23T10:42:44.773 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 0, 2024-04-23T10:42:44.773 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 0, 2024-04-23T10:42:44.773 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-23T10:42:44.773 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:44.773 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:44.773 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:44.773 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:44.773 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:44.774 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-23T10:42:44.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:42:44 smithi196 ceph-mon[32919]: pgmap v519: 97 pgs: 97 active+clean; 4.4 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 38 KiB/s wr, 20 op/s 2024-04-23T10:42:44.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:42:44 smithi183 ceph-mon[32317]: pgmap v519: 97 pgs: 97 active+clean; 4.4 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 38 KiB/s wr, 20 op/s 2024-04-23T10:42:45.476 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-04-23T10:42:45.476 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-04-23T10:42:45.476 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2024-04-23T10:42:45.476 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=0 2024-04-23T10:42:45.476 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-04-23T10:42:45.476 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-04-23T10:42:45.476 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-04-23T10:42:45.476 DEBUG:tasks.check_counter:Getting stats from d 2024-04-23T10:42:45.477 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-23T10:42:45.732 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T10:42:46.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:42:46 smithi196 ceph-mon[32919]: pgmap v520: 97 pgs: 97 active+clean; 4.4 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 21 KiB/s wr, 14 op/s 2024-04-23T10:42:46.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:42:46 smithi097 ceph-mon[29197]: pgmap v520: 97 pgs: 97 active+clean; 4.4 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 21 KiB/s wr, 14 op/s 2024-04-23T10:42:46.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:42:46 smithi183 ceph-mon[32317]: pgmap v520: 97 pgs: 97 active+clean; 4.4 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 21 KiB/s wr, 14 op/s 2024-04-23T10:42:47.562 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-23T10:42:47.563 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-0": { 2024-04-23T10:42:47.563 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 4, 2024-04-23T10:42:47.563 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 4, 2024-04-23T10:42:47.563 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 3040, 2024-04-23T10:42:47.563 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 448, 2024-04-23T10:42:47.563 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 4, 2024-04-23T10:42:47.563 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 0, 2024-04-23T10:42:47.564 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.001355615, 2024-04-23T10:42:47.564 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.000041373, 2024-04-23T10:42:47.564 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.000453901, 2024-04-23T10:42:47.564 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-23T10:42:47.564 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T10:42:47.564 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 3, 2024-04-23T10:42:47.564 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000695071, 2024-04-23T10:42:47.564 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000231690 2024-04-23T10:42:47.564 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:47.564 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-23T10:42:47.564 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:47.565 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:47.565 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:47.565 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:47.565 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-04-23T10:42:47.565 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 448 2024-04-23T10:42:47.565 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:47.565 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-1": { 2024-04-23T10:42:47.565 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 3, 2024-04-23T10:42:47.565 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 236, 2024-04-23T10:42:47.565 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 480, 2024-04-23T10:42:47.566 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 276112, 2024-04-23T10:42:47.566 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 6, 2024-04-23T10:42:47.566 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 3, 2024-04-23T10:42:47.566 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.020222268, 2024-04-23T10:42:47.566 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.014913101, 2024-04-23T10:42:47.566 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.002153670, 2024-04-23T10:42:47.566 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-23T10:42:47.566 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T10:42:47.566 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 233, 2024-04-23T10:42:47.566 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.021708111, 2024-04-23T10:42:47.567 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000093167 2024-04-23T10:42:47.567 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:47.567 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-23T10:42:47.567 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:47.567 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:47.567 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:47.567 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:47.567 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 480, 2024-04-23T10:42:47.567 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 276112 2024-04-23T10:42:47.567 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:47.568 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-2": { 2024-04-23T10:42:47.568 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 318, 2024-04-23T10:42:47.568 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 307, 2024-04-23T10:42:47.568 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 1617504, 2024-04-23T10:42:47.568 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 189360, 2024-04-23T10:42:47.568 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 2, 2024-04-23T10:42:47.568 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 1, 2024-04-23T10:42:47.568 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.057499496, 2024-04-23T10:42:47.568 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.020907194, 2024-04-23T10:42:47.568 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.037594745, 2024-04-23T10:42:47.569 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.006327854, 2024-04-23T10:42:47.569 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T10:42:47.569 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 306, 2024-04-23T10:42:47.569 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.015116678, 2024-04-23T10:42:47.569 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000049400 2024-04-23T10:42:47.569 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:47.569 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-23T10:42:47.569 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:47.569 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:47.569 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:47.569 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:47.569 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 1617504, 2024-04-23T10:42:47.570 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 189360 2024-04-23T10:42:47.570 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:47.570 INFO:teuthology.orchestra.run.smithi097.stdout: "cct": { 2024-04-23T10:42:47.570 INFO:teuthology.orchestra.run.smithi097.stdout: "total_workers": 0, 2024-04-23T10:42:47.570 INFO:teuthology.orchestra.run.smithi097.stdout: "unhealthy_workers": 0 2024-04-23T10:42:47.570 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:47.570 INFO:teuthology.orchestra.run.smithi097.stdout: "mempool": { 2024-04-23T10:42:47.570 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_bytes": 0, 2024-04-23T10:42:47.570 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_items": 0, 2024-04-23T10:42:47.570 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_bytes": 0, 2024-04-23T10:42:47.571 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_items": 0, 2024-04-23T10:42:47.571 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_bytes": 0, 2024-04-23T10:42:47.571 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_items": 0, 2024-04-23T10:42:47.571 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-23T10:42:47.571 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_items": 0, 2024-04-23T10:42:47.571 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-23T10:42:47.571 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_items": 0, 2024-04-23T10:42:47.571 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_bytes": 0, 2024-04-23T10:42:47.571 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_items": 0, 2024-04-23T10:42:47.571 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-23T10:42:47.571 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_items": 0, 2024-04-23T10:42:47.571 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_bytes": 0, 2024-04-23T10:42:47.572 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_items": 0, 2024-04-23T10:42:47.572 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_bytes": 0, 2024-04-23T10:42:47.572 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_items": 0, 2024-04-23T10:42:47.572 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-23T10:42:47.572 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_items": 0, 2024-04-23T10:42:47.572 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-23T10:42:47.572 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_items": 0, 2024-04-23T10:42:47.572 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_bytes": 0, 2024-04-23T10:42:47.572 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_items": 0, 2024-04-23T10:42:47.572 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_bytes": 0, 2024-04-23T10:42:47.572 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_items": 0, 2024-04-23T10:42:47.573 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-23T10:42:47.573 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_items": 0, 2024-04-23T10:42:47.573 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_bytes": 0, 2024-04-23T10:42:47.573 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_items": 0, 2024-04-23T10:42:47.573 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_bytes": 0, 2024-04-23T10:42:47.573 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_items": 0, 2024-04-23T10:42:47.573 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_bytes": 0, 2024-04-23T10:42:47.573 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_items": 0, 2024-04-23T10:42:47.573 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_bytes": 0, 2024-04-23T10:42:47.573 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_items": 0, 2024-04-23T10:42:47.573 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_bytes": 148692, 2024-04-23T10:42:47.574 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_items": 30, 2024-04-23T10:42:47.574 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_bytes": 0, 2024-04-23T10:42:47.574 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_items": 0, 2024-04-23T10:42:47.574 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_bytes": 0, 2024-04-23T10:42:47.574 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_items": 0, 2024-04-23T10:42:47.574 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_bytes": 0, 2024-04-23T10:42:47.574 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_items": 0, 2024-04-23T10:42:47.574 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_bytes": 0, 2024-04-23T10:42:47.574 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_items": 0, 2024-04-23T10:42:47.574 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_bytes": 0, 2024-04-23T10:42:47.574 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_items": 0, 2024-04-23T10:42:47.575 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_bytes": 0, 2024-04-23T10:42:47.575 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_items": 0, 2024-04-23T10:42:47.575 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_bytes": 0, 2024-04-23T10:42:47.575 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_items": 0, 2024-04-23T10:42:47.575 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_bytes": 928, 2024-04-23T10:42:47.575 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_items": 2, 2024-04-23T10:42:47.575 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_bytes": 0, 2024-04-23T10:42:47.575 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_items": 0, 2024-04-23T10:42:47.575 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_bytes": 0, 2024-04-23T10:42:47.575 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_items": 0 2024-04-23T10:42:47.575 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:47.576 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-23T10:42:47.576 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-23T10:42:47.576 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2024-04-23T10:42:47.576 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-23T10:42:47.576 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 321, 2024-04-23T10:42:47.576 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 1572615, 2024-04-23T10:42:47.576 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-23T10:42:47.576 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 321, 2024-04-23T10:42:47.576 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-04-23T10:42:47.576 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-04-23T10:42:47.576 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 321, 2024-04-23T10:42:47.576 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 1572615, 2024-04-23T10:42:47.577 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-23T10:42:47.577 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:47.577 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:47.577 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:47.577 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:47.577 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:47.577 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-23T10:42:48.320 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-04-23T10:42:48.320 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-04-23T10:42:48.320 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-04-23T10:42:48.320 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-04-23T10:42:48.320 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-04-23T10:42:48.320 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-04-23T10:42:48.320 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-04-23T10:42:48.320 DEBUG:tasks.check_counter:Getting stats from g 2024-04-23T10:42:48.321 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-23T10:42:48.564 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T10:42:48.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:42:48 smithi097 ceph-mon[29197]: pgmap v521: 97 pgs: 97 active+clean; 4.4 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 24 KiB/s wr, 18 op/s 2024-04-23T10:42:48.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:42:48 smithi196 ceph-mon[32919]: pgmap v521: 97 pgs: 97 active+clean; 4.4 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 24 KiB/s wr, 18 op/s 2024-04-23T10:42:48.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:42:48 smithi183 ceph-mon[32317]: pgmap v521: 97 pgs: 97 active+clean; 4.4 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 24 KiB/s wr, 18 op/s 2024-04-23T10:42:50.247 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-23T10:42:50.248 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-0": { 2024-04-23T10:42:50.248 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 1117, 2024-04-23T10:42:50.248 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 1118, 2024-04-23T10:42:50.248 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 375339, 2024-04-23T10:42:50.248 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 354889, 2024-04-23T10:42:50.248 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 7, 2024-04-23T10:42:50.248 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 2, 2024-04-23T10:42:50.248 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.128088147, 2024-04-23T10:42:50.249 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.055585045, 2024-04-23T10:42:50.249 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.026096837, 2024-04-23T10:42:50.249 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.032700776, 2024-04-23T10:42:50.249 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T10:42:50.249 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1116, 2024-04-23T10:42:50.249 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.064680039, 2024-04-23T10:42:50.249 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000057957 2024-04-23T10:42:50.249 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:50.250 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-23T10:42:50.250 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:50.250 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:50.250 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:50.250 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:50.250 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-23T10:42:50.250 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 0 2024-04-23T10:42:50.250 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:50.250 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-1": { 2024-04-23T10:42:50.250 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 2463, 2024-04-23T10:42:50.251 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 2449, 2024-04-23T10:42:50.251 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 2196668, 2024-04-23T10:42:50.251 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 865073, 2024-04-23T10:42:50.251 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 4, 2024-04-23T10:42:50.251 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 3, 2024-04-23T10:42:50.251 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.195510875, 2024-04-23T10:42:50.251 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.058936814, 2024-04-23T10:42:50.251 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.075200465, 2024-04-23T10:42:50.251 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.060135366, 2024-04-23T10:42:50.251 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T10:42:50.251 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 2449, 2024-04-23T10:42:50.251 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.080447599, 2024-04-23T10:42:50.252 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000032849 2024-04-23T10:42:50.252 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:50.252 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-23T10:42:50.252 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:50.252 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:50.252 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:50.252 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:50.252 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 1656576, 2024-04-23T10:42:50.252 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 192128 2024-04-23T10:42:50.252 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:50.253 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-2": { 2024-04-23T10:42:50.253 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 6, 2024-04-23T10:42:50.253 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 235, 2024-04-23T10:42:50.253 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 1828, 2024-04-23T10:42:50.253 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 662965, 2024-04-23T10:42:50.253 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 9, 2024-04-23T10:42:50.253 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 5, 2024-04-23T10:42:50.253 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.018425212, 2024-04-23T10:42:50.253 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.012215080, 2024-04-23T10:42:50.253 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.003052301, 2024-04-23T10:42:50.253 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.000049531, 2024-04-23T10:42:50.254 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T10:42:50.254 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 232, 2024-04-23T10:42:50.254 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.015975955, 2024-04-23T10:42:50.254 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000068861 2024-04-23T10:42:50.254 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:50.254 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-23T10:42:50.254 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:50.254 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:50.254 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:50.254 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:50.255 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 480, 2024-04-23T10:42:50.255 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 662000 2024-04-23T10:42:50.255 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:50.255 INFO:teuthology.orchestra.run.smithi097.stdout: "cct": { 2024-04-23T10:42:50.255 INFO:teuthology.orchestra.run.smithi097.stdout: "total_workers": 1, 2024-04-23T10:42:50.255 INFO:teuthology.orchestra.run.smithi097.stdout: "unhealthy_workers": 0 2024-04-23T10:42:50.255 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:50.255 INFO:teuthology.orchestra.run.smithi097.stdout: "finisher-MDSRank": { 2024-04-23T10:42:50.255 INFO:teuthology.orchestra.run.smithi097.stdout: "queue_len": 0, 2024-04-23T10:42:50.255 INFO:teuthology.orchestra.run.smithi097.stdout: "complete_latency": { 2024-04-23T10:42:50.255 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 5441, 2024-04-23T10:42:50.256 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.132855994, 2024-04-23T10:42:50.256 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000024417 2024-04-23T10:42:50.256 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:50.256 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:50.256 INFO:teuthology.orchestra.run.smithi097.stdout: "finisher-PurgeQueue": { 2024-04-23T10:42:50.256 INFO:teuthology.orchestra.run.smithi097.stdout: "queue_len": 0, 2024-04-23T10:42:50.256 INFO:teuthology.orchestra.run.smithi097.stdout: "complete_latency": { 2024-04-23T10:42:50.256 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:50.256 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:50.256 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:50.257 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:50.257 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:50.257 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": { 2024-04-23T10:42:50.257 INFO:teuthology.orchestra.run.smithi097.stdout: "request": 0, 2024-04-23T10:42:50.257 INFO:teuthology.orchestra.run.smithi097.stdout: "reply": 0, 2024-04-23T10:42:50.257 INFO:teuthology.orchestra.run.smithi097.stdout: "reply_latency": { 2024-04-23T10:42:50.257 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:50.257 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:50.257 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:50.257 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:50.258 INFO:teuthology.orchestra.run.smithi097.stdout: "slow_reply": 0, 2024-04-23T10:42:50.258 INFO:teuthology.orchestra.run.smithi097.stdout: "forward": 0, 2024-04-23T10:42:50.258 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_fetch_complete": 0, 2024-04-23T10:42:50.258 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_fetch_keys": 0, 2024-04-23T10:42:50.258 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_commit": 0, 2024-04-23T10:42:50.258 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_split": 0, 2024-04-23T10:42:50.258 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_merge": 0, 2024-04-23T10:42:50.258 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes": 0, 2024-04-23T10:42:50.258 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_top": 0, 2024-04-23T10:42:50.258 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_bottom": 0, 2024-04-23T10:42:50.259 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_pin_tail": 0, 2024-04-23T10:42:50.259 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_pinned": 0, 2024-04-23T10:42:50.259 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_expired": 0, 2024-04-23T10:42:50.259 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_with_caps": 0, 2024-04-23T10:42:50.259 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": 0, 2024-04-23T10:42:50.259 INFO:teuthology.orchestra.run.smithi097.stdout: "subtrees": 1, 2024-04-23T10:42:50.259 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse": 0, 2024-04-23T10:42:50.259 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_hit": 0, 2024-04-23T10:42:50.259 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_forward": 0, 2024-04-23T10:42:50.259 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_discover": 0, 2024-04-23T10:42:50.259 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_dir_fetch": 0, 2024-04-23T10:42:50.260 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_remote_ino": 0, 2024-04-23T10:42:50.260 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_lock": 0, 2024-04-23T10:42:50.260 INFO:teuthology.orchestra.run.smithi097.stdout: "load_cent": 0, 2024-04-23T10:42:50.260 INFO:teuthology.orchestra.run.smithi097.stdout: "q": 0, 2024-04-23T10:42:50.260 INFO:teuthology.orchestra.run.smithi097.stdout: "exported": 0, 2024-04-23T10:42:50.260 INFO:teuthology.orchestra.run.smithi097.stdout: "exported_inodes": 0, 2024-04-23T10:42:50.260 INFO:teuthology.orchestra.run.smithi097.stdout: "imported": 0, 2024-04-23T10:42:50.260 INFO:teuthology.orchestra.run.smithi097.stdout: "imported_inodes": 0, 2024-04-23T10:42:50.260 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_dir_fetch": 0, 2024-04-23T10:42:50.260 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_backtrace_fetch": 0, 2024-04-23T10:42:50.260 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_peer_discover": 0, 2024-04-23T10:42:50.260 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rfiles": 0, 2024-04-23T10:42:50.261 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rbytes": 0, 2024-04-23T10:42:50.261 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rsnaps": 0, 2024-04-23T10:42:50.261 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_backtrace_fetch": 0, 2024-04-23T10:42:50.261 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_set_tag": 0, 2024-04-23T10:42:50.261 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_backtrace_repaired": 0, 2024-04-23T10:42:50.261 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_inotable_repaired": 0, 2024-04-23T10:42:50.261 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dir_inodes": 0, 2024-04-23T10:42:50.261 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dir_base_inodes": 0, 2024-04-23T10:42:50.261 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dirfrag_rstats": 0, 2024-04-23T10:42:50.261 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_file_inodes": 0, 2024-04-23T10:42:50.261 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_inode_file_caps": 0, 2024-04-23T10:42:50.261 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_revoke": 0, 2024-04-23T10:42:50.262 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_grant": 0, 2024-04-23T10:42:50.262 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_trunc": 0, 2024-04-23T10:42:50.262 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-23T10:42:50.262 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-23T10:42:50.262 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_caps": 0, 2024-04-23T10:42:50.262 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_caps_dirty": 0, 2024-04-23T10:42:50.262 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_cap_release": 0, 2024-04-23T10:42:50.262 INFO:teuthology.orchestra.run.smithi097.stdout: "process_request_cap_release": 0 2024-04-23T10:42:50.262 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:50.262 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_cache": { 2024-04-23T10:42:50.262 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_update": 0, 2024-04-23T10:42:50.263 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_update_receipt": 0, 2024-04-23T10:42:50.263 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_try_discover": 0, 2024-04-23T10:42:50.263 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_send_discover": 0, 2024-04-23T10:42:50.263 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_handle_discover": 0, 2024-04-23T10:42:50.263 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays": 0, 2024-04-23T10:42:50.263 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays_delayed": 0, 2024-04-23T10:42:50.263 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays_enqueuing": 0, 2024-04-23T10:42:50.263 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_created": 0, 2024-04-23T10:42:50.263 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_enqueued": 0, 2024-04-23T10:42:50.263 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_reintegrated": 0, 2024-04-23T10:42:50.263 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_migrated": 0, 2024-04-23T10:42:50.263 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_processing": 0, 2024-04-23T10:42:50.264 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_enqueued": 0, 2024-04-23T10:42:50.264 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_prioritized": 0, 2024-04-23T10:42:50.264 INFO:teuthology.orchestra.run.smithi097.stdout: "recovery_started": 0, 2024-04-23T10:42:50.264 INFO:teuthology.orchestra.run.smithi097.stdout: "recovery_completed": 0, 2024-04-23T10:42:50.264 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_quiesce_path": 0, 2024-04-23T10:42:50.264 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_quiesce_inode": 0, 2024-04-23T10:42:50.264 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_enqueue_scrub": 0, 2024-04-23T10:42:50.264 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_exportdir": 0, 2024-04-23T10:42:50.264 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_flush": 0, 2024-04-23T10:42:50.264 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_fragmentdir": 0, 2024-04-23T10:42:50.264 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_fragstats": 0, 2024-04-23T10:42:50.265 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_inodestats": 0 2024-04-23T10:42:50.265 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:50.265 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_log": { 2024-04-23T10:42:50.265 INFO:teuthology.orchestra.run.smithi097.stdout: "evlrg": 0, 2024-04-23T10:42:50.265 INFO:teuthology.orchestra.run.smithi097.stdout: "evadd": 0, 2024-04-23T10:42:50.265 INFO:teuthology.orchestra.run.smithi097.stdout: "evex": 0, 2024-04-23T10:42:50.265 INFO:teuthology.orchestra.run.smithi097.stdout: "evtrm": 0, 2024-04-23T10:42:50.265 INFO:teuthology.orchestra.run.smithi097.stdout: "ev": 6, 2024-04-23T10:42:50.265 INFO:teuthology.orchestra.run.smithi097.stdout: "evexg": 0, 2024-04-23T10:42:50.265 INFO:teuthology.orchestra.run.smithi097.stdout: "evexd": 0, 2024-04-23T10:42:50.265 INFO:teuthology.orchestra.run.smithi097.stdout: "segadd": 0, 2024-04-23T10:42:50.265 INFO:teuthology.orchestra.run.smithi097.stdout: "segex": 0, 2024-04-23T10:42:50.266 INFO:teuthology.orchestra.run.smithi097.stdout: "segtrm": 0, 2024-04-23T10:42:50.266 INFO:teuthology.orchestra.run.smithi097.stdout: "seg": 2, 2024-04-23T10:42:50.266 INFO:teuthology.orchestra.run.smithi097.stdout: "segmjr": 2, 2024-04-23T10:42:50.266 INFO:teuthology.orchestra.run.smithi097.stdout: "segexg": 0, 2024-04-23T10:42:50.266 INFO:teuthology.orchestra.run.smithi097.stdout: "segexd": 0, 2024-04-23T10:42:50.266 INFO:teuthology.orchestra.run.smithi097.stdout: "expos": 4194304, 2024-04-23T10:42:50.266 INFO:teuthology.orchestra.run.smithi097.stdout: "wrpos": 4195956, 2024-04-23T10:42:50.266 INFO:teuthology.orchestra.run.smithi097.stdout: "rdpos": 4195815, 2024-04-23T10:42:50.266 INFO:teuthology.orchestra.run.smithi097.stdout: "jlat": { 2024-04-23T10:42:50.266 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:50.266 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:50.267 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:50.267 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:50.267 INFO:teuthology.orchestra.run.smithi097.stdout: "replayed": 6 2024-04-23T10:42:50.267 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:50.267 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_mem": { 2024-04-23T10:42:50.267 INFO:teuthology.orchestra.run.smithi097.stdout: "ino": 3, 2024-04-23T10:42:50.267 INFO:teuthology.orchestra.run.smithi097.stdout: "ino+": 3, 2024-04-23T10:42:50.267 INFO:teuthology.orchestra.run.smithi097.stdout: "ino-": 0, 2024-04-23T10:42:50.267 INFO:teuthology.orchestra.run.smithi097.stdout: "dir": 1, 2024-04-23T10:42:50.267 INFO:teuthology.orchestra.run.smithi097.stdout: "dir+": 1, 2024-04-23T10:42:50.267 INFO:teuthology.orchestra.run.smithi097.stdout: "dir-": 0, 2024-04-23T10:42:50.267 INFO:teuthology.orchestra.run.smithi097.stdout: "dn": 0, 2024-04-23T10:42:50.268 INFO:teuthology.orchestra.run.smithi097.stdout: "dn+": 0, 2024-04-23T10:42:50.268 INFO:teuthology.orchestra.run.smithi097.stdout: "dn-": 0, 2024-04-23T10:42:50.268 INFO:teuthology.orchestra.run.smithi097.stdout: "cap": 0, 2024-04-23T10:42:50.268 INFO:teuthology.orchestra.run.smithi097.stdout: "cap+": 0, 2024-04-23T10:42:50.268 INFO:teuthology.orchestra.run.smithi097.stdout: "cap-": 0, 2024-04-23T10:42:50.268 INFO:teuthology.orchestra.run.smithi097.stdout: "rss": 50984, 2024-04-23T10:42:50.268 INFO:teuthology.orchestra.run.smithi097.stdout: "heap": 215416 2024-04-23T10:42:50.268 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:50.268 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_server": { 2024-04-23T10:42:50.268 INFO:teuthology.orchestra.run.smithi097.stdout: "dispatch_client_request": 0, 2024-04-23T10:42:50.268 INFO:teuthology.orchestra.run.smithi097.stdout: "dispatch_server_request": 0, 2024-04-23T10:42:50.269 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_request": 0, 2024-04-23T10:42:50.269 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_session": 0, 2024-04-23T10:42:50.269 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_peer_request": 0, 2024-04-23T10:42:50.269 INFO:teuthology.orchestra.run.smithi097.stdout: "req_create_latency": { 2024-04-23T10:42:50.269 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:50.269 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:50.269 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:50.269 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:50.269 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getattr_latency": { 2024-04-23T10:42:50.269 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:50.269 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:50.270 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:50.270 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:50.270 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getfilelock_latency": { 2024-04-23T10:42:50.270 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:50.270 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:50.270 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:50.270 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:50.270 INFO:teuthology.orchestra.run.smithi097.stdout: "req_link_latency": { 2024-04-23T10:42:50.270 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:50.270 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:50.271 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:50.271 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:50.271 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookup_latency": { 2024-04-23T10:42:50.271 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:50.271 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:50.271 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:50.271 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:50.271 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookuphash_latency": { 2024-04-23T10:42:50.271 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:50.271 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:50.271 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:50.271 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:50.272 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupino_latency": { 2024-04-23T10:42:50.272 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:50.272 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:50.272 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:50.272 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:50.272 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupname_latency": { 2024-04-23T10:42:50.272 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:50.272 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:50.272 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:50.272 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:50.272 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupparent_latency": { 2024-04-23T10:42:50.273 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:50.273 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:50.273 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:50.273 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:50.273 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupsnap_latency": { 2024-04-23T10:42:50.273 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:50.273 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:50.273 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:50.273 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:50.273 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lssnap_latency": { 2024-04-23T10:42:50.273 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:50.273 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:50.274 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:50.274 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:50.274 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mkdir_latency": { 2024-04-23T10:42:50.274 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:50.275 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:50.275 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:50.275 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:50.275 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mknod_latency": { 2024-04-23T10:42:50.275 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:50.275 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:50.275 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:50.275 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:50.275 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mksnap_latency": { 2024-04-23T10:42:50.275 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:50.276 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:50.276 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:50.276 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:50.276 INFO:teuthology.orchestra.run.smithi097.stdout: "req_open_latency": { 2024-04-23T10:42:50.276 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:50.276 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:50.276 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:50.276 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:50.276 INFO:teuthology.orchestra.run.smithi097.stdout: "req_readdir_latency": { 2024-04-23T10:42:50.276 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:50.276 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:50.276 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:50.277 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:50.277 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rename_latency": { 2024-04-23T10:42:50.277 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:50.277 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:50.277 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:50.277 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:50.277 INFO:teuthology.orchestra.run.smithi097.stdout: "req_renamesnap_latency": { 2024-04-23T10:42:50.277 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:50.277 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:50.277 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:50.277 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:50.278 INFO:teuthology.orchestra.run.smithi097.stdout: "req_snapdiff_latency": { 2024-04-23T10:42:50.278 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:50.278 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:50.278 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:50.278 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:50.278 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmdir_latency": { 2024-04-23T10:42:50.278 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:50.278 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:50.278 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:50.278 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:50.278 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmsnap_latency": { 2024-04-23T10:42:50.278 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:50.279 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:50.279 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:50.279 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:50.279 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmxattr_latency": { 2024-04-23T10:42:50.279 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:50.279 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:50.279 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:50.279 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:50.279 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setattr_latency": { 2024-04-23T10:42:50.279 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:50.279 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:50.279 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:50.280 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:50.280 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setdirlayout_latency": { 2024-04-23T10:42:50.280 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:50.280 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:50.280 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:50.280 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:50.280 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setfilelock_latency": { 2024-04-23T10:42:50.280 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:50.280 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:50.280 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:50.280 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:50.280 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setlayout_latency": { 2024-04-23T10:42:50.280 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:50.281 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:50.281 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:50.281 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:50.281 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setxattr_latency": { 2024-04-23T10:42:50.281 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:50.281 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:50.281 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:50.281 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:50.281 INFO:teuthology.orchestra.run.smithi097.stdout: "req_symlink_latency": { 2024-04-23T10:42:50.281 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:50.281 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:50.281 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:50.282 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:50.282 INFO:teuthology.orchestra.run.smithi097.stdout: "req_unlink_latency": { 2024-04-23T10:42:50.282 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:50.282 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:50.282 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:50.282 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:50.282 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_revoke_eviction": 0, 2024-04-23T10:42:50.282 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_acquisition_throttle": 0, 2024-04-23T10:42:50.282 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getvxattr_latency": { 2024-04-23T10:42:50.282 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:50.282 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:50.283 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:50.283 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:50.283 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:50.283 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_sessions": { 2024-04-23T10:42:50.283 INFO:teuthology.orchestra.run.smithi097.stdout: "session_count": 0, 2024-04-23T10:42:50.283 INFO:teuthology.orchestra.run.smithi097.stdout: "session_add": 2, 2024-04-23T10:42:50.283 INFO:teuthology.orchestra.run.smithi097.stdout: "session_remove": 2, 2024-04-23T10:42:50.283 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions_open": 2, 2024-04-23T10:42:50.283 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions_stale": 0, 2024-04-23T10:42:50.283 INFO:teuthology.orchestra.run.smithi097.stdout: "total_load": 0, 2024-04-23T10:42:50.283 INFO:teuthology.orchestra.run.smithi097.stdout: "average_load": 0, 2024-04-23T10:42:50.284 INFO:teuthology.orchestra.run.smithi097.stdout: "avg_session_uptime": 1747, 2024-04-23T10:42:50.284 INFO:teuthology.orchestra.run.smithi097.stdout: "mdthresh_evicted": 0 2024-04-23T10:42:50.284 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:50.284 INFO:teuthology.orchestra.run.smithi097.stdout: "mempool": { 2024-04-23T10:42:50.284 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_bytes": 0, 2024-04-23T10:42:50.284 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_items": 0, 2024-04-23T10:42:50.284 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_bytes": 0, 2024-04-23T10:42:50.284 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_items": 0, 2024-04-23T10:42:50.284 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_bytes": 0, 2024-04-23T10:42:50.284 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_items": 0, 2024-04-23T10:42:50.284 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-23T10:42:50.284 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_items": 0, 2024-04-23T10:42:50.285 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-23T10:42:50.285 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_items": 0, 2024-04-23T10:42:50.285 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_bytes": 0, 2024-04-23T10:42:50.285 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_items": 0, 2024-04-23T10:42:50.285 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-23T10:42:50.285 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_items": 0, 2024-04-23T10:42:50.285 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_bytes": 0, 2024-04-23T10:42:50.285 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_items": 0, 2024-04-23T10:42:50.285 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_bytes": 0, 2024-04-23T10:42:50.285 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_items": 0, 2024-04-23T10:42:50.285 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-23T10:42:50.285 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_items": 0, 2024-04-23T10:42:50.285 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-23T10:42:50.286 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_items": 0, 2024-04-23T10:42:50.286 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_bytes": 0, 2024-04-23T10:42:50.286 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_items": 0, 2024-04-23T10:42:50.286 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_bytes": 0, 2024-04-23T10:42:50.286 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_items": 0, 2024-04-23T10:42:50.286 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-23T10:42:50.286 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_items": 0, 2024-04-23T10:42:50.286 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_bytes": 0, 2024-04-23T10:42:50.286 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_items": 0, 2024-04-23T10:42:50.286 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_bytes": 0, 2024-04-23T10:42:50.286 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_items": 0, 2024-04-23T10:42:50.286 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_bytes": 0, 2024-04-23T10:42:50.287 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_items": 0, 2024-04-23T10:42:50.287 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_bytes": 0, 2024-04-23T10:42:50.287 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_items": 0, 2024-04-23T10:42:50.287 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_bytes": 198297, 2024-04-23T10:42:50.287 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_items": 58, 2024-04-23T10:42:50.287 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_bytes": 0, 2024-04-23T10:42:50.287 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_items": 0, 2024-04-23T10:42:50.287 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_bytes": 0, 2024-04-23T10:42:50.287 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_items": 0, 2024-04-23T10:42:50.287 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_bytes": 0, 2024-04-23T10:42:50.287 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_items": 0, 2024-04-23T10:42:50.287 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_bytes": 0, 2024-04-23T10:42:50.288 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_items": 0, 2024-04-23T10:42:50.288 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_bytes": 7936, 2024-04-23T10:42:50.288 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_items": 165, 2024-04-23T10:42:50.288 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_bytes": 0, 2024-04-23T10:42:50.288 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_items": 0, 2024-04-23T10:42:50.288 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_bytes": 0, 2024-04-23T10:42:50.288 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_items": 0, 2024-04-23T10:42:50.288 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_bytes": 8512, 2024-04-23T10:42:50.288 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_items": 13, 2024-04-23T10:42:50.288 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_bytes": 0, 2024-04-23T10:42:50.288 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_items": 0, 2024-04-23T10:42:50.288 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_bytes": 0, 2024-04-23T10:42:50.289 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_items": 0 2024-04-23T10:42:50.289 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:50.289 INFO:teuthology.orchestra.run.smithi097.stdout: "objecter": { 2024-04-23T10:42:50.289 INFO:teuthology.orchestra.run.smithi097.stdout: "op_active": 0, 2024-04-23T10:42:50.289 INFO:teuthology.orchestra.run.smithi097.stdout: "op_laggy": 0, 2024-04-23T10:42:50.289 INFO:teuthology.orchestra.run.smithi097.stdout: "op_send": 3235, 2024-04-23T10:42:50.289 INFO:teuthology.orchestra.run.smithi097.stdout: "op_send_bytes": 21, 2024-04-23T10:42:50.289 INFO:teuthology.orchestra.run.smithi097.stdout: "op_resend": 3, 2024-04-23T10:42:50.289 INFO:teuthology.orchestra.run.smithi097.stdout: "op_reply": 3235, 2024-04-23T10:42:50.289 INFO:teuthology.orchestra.run.smithi097.stdout: "op_latency": { 2024-04-23T10:42:50.289 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 3235, 2024-04-23T10:42:50.289 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 2.408955927, 2024-04-23T10:42:50.290 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000744654 2024-04-23T10:42:50.290 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:50.290 INFO:teuthology.orchestra.run.smithi097.stdout: "op_inflight": 0, 2024-04-23T10:42:50.290 INFO:teuthology.orchestra.run.smithi097.stdout: "oplen_avg": { 2024-04-23T10:42:50.290 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 3235, 2024-04-23T10:42:50.290 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 3236 2024-04-23T10:42:50.290 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:50.290 INFO:teuthology.orchestra.run.smithi097.stdout: "op": 3235, 2024-04-23T10:42:50.290 INFO:teuthology.orchestra.run.smithi097.stdout: "op_r": 3235, 2024-04-23T10:42:50.290 INFO:teuthology.orchestra.run.smithi097.stdout: "op_w": 0, 2024-04-23T10:42:50.290 INFO:teuthology.orchestra.run.smithi097.stdout: "op_rmw": 0, 2024-04-23T10:42:50.290 INFO:teuthology.orchestra.run.smithi097.stdout: "op_pg": 0, 2024-04-23T10:42:50.290 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_stat": 2112, 2024-04-23T10:42:50.291 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_create": 0, 2024-04-23T10:42:50.291 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_read": 1121, 2024-04-23T10:42:50.291 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_write": 0, 2024-04-23T10:42:50.291 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_writefull": 0, 2024-04-23T10:42:50.291 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_writesame": 0, 2024-04-23T10:42:50.291 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_append": 0, 2024-04-23T10:42:50.291 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_zero": 0, 2024-04-23T10:42:50.291 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_truncate": 0, 2024-04-23T10:42:50.291 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_delete": 0, 2024-04-23T10:42:50.291 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_mapext": 0, 2024-04-23T10:42:50.291 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_sparse_read": 0, 2024-04-23T10:42:50.291 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_clonerange": 0, 2024-04-23T10:42:50.291 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_getxattr": 1, 2024-04-23T10:42:50.292 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_setxattr": 0, 2024-04-23T10:42:50.292 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_cmpxattr": 0, 2024-04-23T10:42:50.292 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_rmxattr": 0, 2024-04-23T10:42:50.292 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_resetxattrs": 0, 2024-04-23T10:42:50.292 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_call": 0, 2024-04-23T10:42:50.292 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_watch": 0, 2024-04-23T10:42:50.292 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_notify": 0, 2024-04-23T10:42:50.292 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_src_cmpxattr": 0, 2024-04-23T10:42:50.292 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_pgls": 0, 2024-04-23T10:42:50.292 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_pgls_filter": 0, 2024-04-23T10:42:50.292 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_other": 0, 2024-04-23T10:42:50.292 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_active": 0, 2024-04-23T10:42:50.293 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_send": 0, 2024-04-23T10:42:50.293 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_resend": 0, 2024-04-23T10:42:50.293 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_ping": 0, 2024-04-23T10:42:50.293 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_active": 0, 2024-04-23T10:42:50.293 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_send": 0, 2024-04-23T10:42:50.293 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_resend": 0, 2024-04-23T10:42:50.293 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_active": 0, 2024-04-23T10:42:50.293 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_send": 0, 2024-04-23T10:42:50.293 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_resend": 0, 2024-04-23T10:42:50.293 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_active": 0, 2024-04-23T10:42:50.293 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_send": 0, 2024-04-23T10:42:50.293 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_resend": 0, 2024-04-23T10:42:50.293 INFO:teuthology.orchestra.run.smithi097.stdout: "command_active": 0, 2024-04-23T10:42:50.294 INFO:teuthology.orchestra.run.smithi097.stdout: "command_send": 0, 2024-04-23T10:42:50.294 INFO:teuthology.orchestra.run.smithi097.stdout: "command_resend": 0, 2024-04-23T10:42:50.294 INFO:teuthology.orchestra.run.smithi097.stdout: "map_epoch": 109, 2024-04-23T10:42:50.294 INFO:teuthology.orchestra.run.smithi097.stdout: "map_full": 0, 2024-04-23T10:42:50.294 INFO:teuthology.orchestra.run.smithi097.stdout: "map_inc": 23, 2024-04-23T10:42:50.294 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_sessions": 6, 2024-04-23T10:42:50.294 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_session_open": 8, 2024-04-23T10:42:50.294 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_session_close": 2, 2024-04-23T10:42:50.294 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_laggy": 0, 2024-04-23T10:42:50.294 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_wr": 0, 2024-04-23T10:42:50.294 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_rd": 2, 2024-04-23T10:42:50.294 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_del": 0 2024-04-23T10:42:50.294 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:50.295 INFO:teuthology.orchestra.run.smithi097.stdout: "oft": { 2024-04-23T10:42:50.295 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_objs": 0, 2024-04-23T10:42:50.295 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_kv_pairs": 0, 2024-04-23T10:42:50.295 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_updates": 0, 2024-04-23T10:42:50.295 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_removes": 0 2024-04-23T10:42:50.295 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:50.295 INFO:teuthology.orchestra.run.smithi097.stdout: "osd-slow-ops": { 2024-04-23T10:42:50.295 INFO:teuthology.orchestra.run.smithi097.stdout: "slow_ops_count": 0 2024-04-23T10:42:50.295 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:50.295 INFO:teuthology.orchestra.run.smithi097.stdout: "purge_queue": { 2024-04-23T10:42:50.295 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_ops": 0, 2024-04-23T10:42:50.295 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_ops_high_water": 0, 2024-04-23T10:42:50.295 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing": 0, 2024-04-23T10:42:50.296 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_high_water": 0, 2024-04-23T10:42:50.296 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executed_ops": 0, 2024-04-23T10:42:50.296 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executed": 0, 2024-04-23T10:42:50.296 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_item_in_journal": 0 2024-04-23T10:42:50.296 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:50.296 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-23T10:42:50.296 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-23T10:42:50.296 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2024-04-23T10:42:50.296 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-23T10:42:50.296 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 3582, 2024-04-23T10:42:50.296 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 2230794, 2024-04-23T10:42:50.296 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-23T10:42:50.296 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 3582, 2024-04-23T10:42:50.297 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-04-23T10:42:50.297 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-04-23T10:42:50.297 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 3582, 2024-04-23T10:42:50.297 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 2230794, 2024-04-23T10:42:50.297 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-23T10:42:50.297 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:50.297 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:50.297 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:50.297 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:50.297 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:50.297 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-objecter_bytes": { 2024-04-23T10:42:50.297 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-23T10:42:50.297 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2024-04-23T10:42:50.298 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-23T10:42:50.298 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-04-23T10:42:50.298 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-04-23T10:42:50.298 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-23T10:42:50.298 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-04-23T10:42:50.298 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 3235, 2024-04-23T10:42:50.298 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 1657, 2024-04-23T10:42:50.298 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 3235, 2024-04-23T10:42:50.298 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 1657, 2024-04-23T10:42:50.298 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-23T10:42:50.298 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:50.298 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:50.299 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:50.299 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:50.299 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:50.299 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-objecter_ops": { 2024-04-23T10:42:50.299 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-23T10:42:50.299 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 1024, 2024-04-23T10:42:50.299 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-23T10:42:50.299 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-04-23T10:42:50.299 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-04-23T10:42:50.299 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-23T10:42:50.299 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-04-23T10:42:50.299 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 3235, 2024-04-23T10:42:50.300 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 3235, 2024-04-23T10:42:50.300 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 3235, 2024-04-23T10:42:50.300 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 3235, 2024-04-23T10:42:50.300 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-23T10:42:50.300 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:50.300 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:50.300 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:50.300 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:50.300 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:50.300 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-write_buf_throttle": { 2024-04-23T10:42:50.300 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-23T10:42:50.300 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 3758096384, 2024-04-23T10:42:50.300 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-23T10:42:50.300 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-04-23T10:42:50.301 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-04-23T10:42:50.301 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-23T10:42:50.301 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-04-23T10:42:50.301 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-04-23T10:42:50.301 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-04-23T10:42:50.301 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 0, 2024-04-23T10:42:50.301 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 0, 2024-04-23T10:42:50.301 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-23T10:42:50.301 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:50.301 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:50.301 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:50.302 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:50.302 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:50.302 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-write_buf_throttle-0x560c212826e0": { 2024-04-23T10:42:50.302 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-23T10:42:50.302 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 3758096384, 2024-04-23T10:42:50.302 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-23T10:42:50.302 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-04-23T10:42:50.302 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-04-23T10:42:50.302 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-23T10:42:50.302 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-04-23T10:42:50.302 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-04-23T10:42:50.302 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-04-23T10:42:50.303 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 0, 2024-04-23T10:42:50.303 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 0, 2024-04-23T10:42:50.303 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-23T10:42:50.303 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:50.303 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:50.303 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:50.303 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:50.303 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:50.303 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-23T10:42:50.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:42:50 smithi097 ceph-mon[29197]: pgmap v522: 97 pgs: 97 active+clean; 4.4 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.7 KiB/s wr, 12 op/s 2024-04-23T10:42:50.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:42:50 smithi196 ceph-mon[32919]: pgmap v522: 97 pgs: 97 active+clean; 4.4 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.7 KiB/s wr, 12 op/s 2024-04-23T10:42:50.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:42:50 smithi183 ceph-mon[32317]: pgmap v522: 97 pgs: 97 active+clean; 4.4 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.7 KiB/s wr, 12 op/s 2024-04-23T10:42:51.063 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-04-23T10:42:51.064 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-04-23T10:42:51.064 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2024-04-23T10:42:51.064 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=0 2024-04-23T10:42:51.064 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-04-23T10:42:51.064 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-04-23T10:42:51.064 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-04-23T10:42:51.064 DEBUG:tasks.check_counter:Getting stats from j 2024-04-23T10:42:51.064 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-23T10:42:51.307 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T10:42:53.266 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:42:53 smithi097 ceph-mon[29197]: pgmap v523: 97 pgs: 97 active+clean; 4.4 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.7 KiB/s wr, 12 op/s 2024-04-23T10:42:53.293 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-23T10:42:53.293 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-0": { 2024-04-23T10:42:53.293 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 0, 2024-04-23T10:42:53.293 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 2, 2024-04-23T10:42:53.293 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 0, 2024-04-23T10:42:53.293 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 0, 2024-04-23T10:42:53.294 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 4, 2024-04-23T10:42:53.294 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 0, 2024-04-23T10:42:53.294 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.000958985, 2024-04-23T10:42:53.294 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.000000000, 2024-04-23T10:42:53.294 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.000105881, 2024-04-23T10:42:53.294 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-23T10:42:53.294 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T10:42:53.294 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:53.294 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:53.295 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:53.295 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:53.295 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-23T10:42:53.295 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:53.295 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:53.295 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:53.295 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:53.295 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-23T10:42:53.295 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 0 2024-04-23T10:42:53.295 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:53.295 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-1": { 2024-04-23T10:42:53.296 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 7, 2024-04-23T10:42:53.296 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 239, 2024-04-23T10:42:53.296 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 3520, 2024-04-23T10:42:53.296 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 276736, 2024-04-23T10:42:53.296 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 6, 2024-04-23T10:42:53.296 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 3, 2024-04-23T10:42:53.296 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.020838771, 2024-04-23T10:42:53.296 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.015274381, 2024-04-23T10:42:53.296 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.002480790, 2024-04-23T10:42:53.296 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-23T10:42:53.297 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T10:42:53.297 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 237, 2024-04-23T10:42:53.297 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.020926305, 2024-04-23T10:42:53.297 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000088296 2024-04-23T10:42:53.297 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:53.297 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-23T10:42:53.297 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:53.297 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:53.297 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:53.297 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:53.297 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 3520, 2024-04-23T10:42:53.298 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 276736 2024-04-23T10:42:53.298 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:53.298 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-2": { 2024-04-23T10:42:53.298 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 320, 2024-04-23T10:42:53.298 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 309, 2024-04-23T10:42:53.298 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 1618048, 2024-04-23T10:42:53.298 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 190640, 2024-04-23T10:42:53.298 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 2, 2024-04-23T10:42:53.298 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 1, 2024-04-23T10:42:53.298 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.061683776, 2024-04-23T10:42:53.299 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.021671809, 2024-04-23T10:42:53.299 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.041038830, 2024-04-23T10:42:53.299 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.006436607, 2024-04-23T10:42:53.299 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T10:42:53.299 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 308, 2024-04-23T10:42:53.299 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.014796786, 2024-04-23T10:42:53.299 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000048041 2024-04-23T10:42:53.299 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:53.299 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-23T10:42:53.299 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:53.300 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:53.300 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:53.300 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:53.300 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 1618048, 2024-04-23T10:42:53.300 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 190640 2024-04-23T10:42:53.300 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:53.300 INFO:teuthology.orchestra.run.smithi097.stdout: "cct": { 2024-04-23T10:42:53.300 INFO:teuthology.orchestra.run.smithi097.stdout: "total_workers": 0, 2024-04-23T10:42:53.300 INFO:teuthology.orchestra.run.smithi097.stdout: "unhealthy_workers": 0 2024-04-23T10:42:53.300 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:53.301 INFO:teuthology.orchestra.run.smithi097.stdout: "mempool": { 2024-04-23T10:42:53.301 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_bytes": 0, 2024-04-23T10:42:53.301 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_items": 0, 2024-04-23T10:42:53.301 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_bytes": 0, 2024-04-23T10:42:53.301 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_items": 0, 2024-04-23T10:42:53.301 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_bytes": 0, 2024-04-23T10:42:53.301 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_items": 0, 2024-04-23T10:42:53.301 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-23T10:42:53.301 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_items": 0, 2024-04-23T10:42:53.301 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-23T10:42:53.302 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_items": 0, 2024-04-23T10:42:53.302 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_bytes": 0, 2024-04-23T10:42:53.302 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_items": 0, 2024-04-23T10:42:53.302 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-23T10:42:53.302 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_items": 0, 2024-04-23T10:42:53.302 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_bytes": 0, 2024-04-23T10:42:53.302 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_items": 0, 2024-04-23T10:42:53.302 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_bytes": 0, 2024-04-23T10:42:53.302 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_items": 0, 2024-04-23T10:42:53.302 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-23T10:42:53.302 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_items": 0, 2024-04-23T10:42:53.303 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-23T10:42:53.303 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_items": 0, 2024-04-23T10:42:53.303 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_bytes": 0, 2024-04-23T10:42:53.303 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_items": 0, 2024-04-23T10:42:53.303 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_bytes": 0, 2024-04-23T10:42:53.303 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_items": 0, 2024-04-23T10:42:53.303 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-23T10:42:53.303 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_items": 0, 2024-04-23T10:42:53.303 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_bytes": 0, 2024-04-23T10:42:53.303 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_items": 0, 2024-04-23T10:42:53.304 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_bytes": 0, 2024-04-23T10:42:53.304 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_items": 0, 2024-04-23T10:42:53.304 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_bytes": 0, 2024-04-23T10:42:53.304 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_items": 0, 2024-04-23T10:42:53.304 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_bytes": 0, 2024-04-23T10:42:53.304 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_items": 0, 2024-04-23T10:42:53.304 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_bytes": 148709, 2024-04-23T10:42:53.304 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_items": 31, 2024-04-23T10:42:53.304 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_bytes": 0, 2024-04-23T10:42:53.304 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_items": 0, 2024-04-23T10:42:53.304 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_bytes": 0, 2024-04-23T10:42:53.305 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_items": 0, 2024-04-23T10:42:53.305 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_bytes": 0, 2024-04-23T10:42:53.305 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_items": 0, 2024-04-23T10:42:53.305 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_bytes": 0, 2024-04-23T10:42:53.305 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_items": 0, 2024-04-23T10:42:53.305 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_bytes": 0, 2024-04-23T10:42:53.305 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_items": 0, 2024-04-23T10:42:53.305 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_bytes": 0, 2024-04-23T10:42:53.305 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_items": 0, 2024-04-23T10:42:53.305 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_bytes": 0, 2024-04-23T10:42:53.306 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_items": 0, 2024-04-23T10:42:53.306 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_bytes": 928, 2024-04-23T10:42:53.306 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_items": 2, 2024-04-23T10:42:53.306 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_bytes": 0, 2024-04-23T10:42:53.306 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_items": 0, 2024-04-23T10:42:53.306 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_bytes": 0, 2024-04-23T10:42:53.306 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_items": 0 2024-04-23T10:42:53.306 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-23T10:42:53.306 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-23T10:42:53.306 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-23T10:42:53.307 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2024-04-23T10:42:53.307 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-23T10:42:53.307 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 323, 2024-04-23T10:42:53.307 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 1572875, 2024-04-23T10:42:53.307 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-23T10:42:53.307 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 323, 2024-04-23T10:42:53.307 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-04-23T10:42:53.307 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-04-23T10:42:53.307 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 323, 2024-04-23T10:42:53.307 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 1572875, 2024-04-23T10:42:53.308 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-23T10:42:53.308 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-23T10:42:53.308 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-23T10:42:53.308 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-23T10:42:53.308 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:53.308 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-23T10:42:53.308 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-23T10:42:53.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:42:53 smithi196 ceph-mon[32919]: pgmap v523: 97 pgs: 97 active+clean; 4.4 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.7 KiB/s wr, 12 op/s 2024-04-23T10:42:53.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:42:53 smithi183 ceph-mon[32317]: pgmap v523: 97 pgs: 97 active+clean; 4.4 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.7 KiB/s wr, 12 op/s 2024-04-23T10:42:53.999 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-04-23T10:42:53.999 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-04-23T10:42:53.999 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-04-23T10:42:53.999 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-04-23T10:42:53.999 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-04-23T10:42:54.000 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-04-23T10:42:54.000 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-04-23T10:42:54.000 DEBUG:tasks.check_counter:Getting stats from b 2024-04-23T10:42:54.000 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-23T10:42:54.252 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.b/config 2024-04-23T10:42:55.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:42:55 smithi183 ceph-mon[32317]: pgmap v524: 97 pgs: 97 active+clean; 4.4 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.3 KiB/s wr, 17 op/s 2024-04-23T10:42:55.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:42:55 smithi196 ceph-mon[32919]: pgmap v524: 97 pgs: 97 active+clean; 4.4 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.3 KiB/s wr, 17 op/s 2024-04-23T10:42:55.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:42:55 smithi097 ceph-mon[29197]: pgmap v524: 97 pgs: 97 active+clean; 4.4 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.3 KiB/s wr, 17 op/s 2024-04-23T10:42:56.185 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-04-23T10:42:56.185 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-0": { 2024-04-23T10:42:56.185 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 4237, 2024-04-23T10:42:56.186 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 5169, 2024-04-23T10:42:56.186 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 1161759, 2024-04-23T10:42:56.186 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 15135194, 2024-04-23T10:42:56.186 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 20, 2024-04-23T10:42:56.186 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 7, 2024-04-23T10:42:56.186 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.406539349, 2024-04-23T10:42:56.186 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.152543446, 2024-04-23T10:42:56.186 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.129910756, 2024-04-23T10:42:56.186 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.092654754, 2024-04-23T10:42:56.186 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T10:42:56.186 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 5164, 2024-04-23T10:42:56.186 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.448757921, 2024-04-23T10:42:56.186 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000086901 2024-04-23T10:42:56.186 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:56.187 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-04-23T10:42:56.187 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 3698, 2024-04-23T10:42:56.187 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.001011571, 2024-04-23T10:42:56.187 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000273 2024-04-23T10:42:56.187 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:56.187 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-23T10:42:56.187 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 451248 2024-04-23T10:42:56.187 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:56.187 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-1": { 2024-04-23T10:42:56.187 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 4730, 2024-04-23T10:42:56.187 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 6024, 2024-04-23T10:42:56.187 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 2857337, 2024-04-23T10:42:56.187 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 9809639, 2024-04-23T10:42:56.187 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 10, 2024-04-23T10:42:56.188 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 5, 2024-04-23T10:42:56.188 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.462790050, 2024-04-23T10:42:56.188 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.181230631, 2024-04-23T10:42:56.188 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.196291294, 2024-04-23T10:42:56.188 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.091947952, 2024-04-23T10:42:56.188 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T10:42:56.188 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 6026, 2024-04-23T10:42:56.188 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.284401099, 2024-04-23T10:42:56.188 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000047195 2024-04-23T10:42:56.188 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:56.188 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-04-23T10:42:56.188 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 7261, 2024-04-23T10:42:56.188 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.001553981, 2024-04-23T10:42:56.188 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000214 2024-04-23T10:42:56.189 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:56.189 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 1740848, 2024-04-23T10:42:56.189 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 203792 2024-04-23T10:42:56.189 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:56.189 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-2": { 2024-04-23T10:42:56.189 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 6468, 2024-04-23T10:42:56.189 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 7635, 2024-04-23T10:42:56.189 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 2212589, 2024-04-23T10:42:56.189 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 10206968, 2024-04-23T10:42:56.189 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 230, 2024-04-23T10:42:56.189 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 9, 2024-04-23T10:42:56.189 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.815845512, 2024-04-23T10:42:56.189 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.259271002, 2024-04-23T10:42:56.189 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.370822435, 2024-04-23T10:42:56.189 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.181848426, 2024-04-23T10:42:56.189 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T10:42:56.190 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 7633, 2024-04-23T10:42:56.190 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.474399377, 2024-04-23T10:42:56.190 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000062151 2024-04-23T10:42:56.190 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:56.190 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-04-23T10:42:56.190 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 10776, 2024-04-23T10:42:56.190 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.002453699, 2024-04-23T10:42:56.190 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000227 2024-04-23T10:42:56.190 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:56.190 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 1612592, 2024-04-23T10:42:56.190 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 1890080 2024-04-23T10:42:56.190 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:56.190 INFO:teuthology.orchestra.run.smithi183.stdout: "cct": { 2024-04-23T10:42:56.191 INFO:teuthology.orchestra.run.smithi183.stdout: "total_workers": 1, 2024-04-23T10:42:56.191 INFO:teuthology.orchestra.run.smithi183.stdout: "unhealthy_workers": 0 2024-04-23T10:42:56.191 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:56.191 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-MDSRank": { 2024-04-23T10:42:56.191 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-04-23T10:42:56.191 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-04-23T10:42:56.191 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 6507, 2024-04-23T10:42:56.191 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 2.031359136, 2024-04-23T10:42:56.191 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000312180 2024-04-23T10:42:56.191 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-23T10:42:56.191 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:56.191 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-PurgeQueue": { 2024-04-23T10:42:56.191 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-04-23T10:42:56.192 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-04-23T10:42:56.192 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 16, 2024-04-23T10:42:56.192 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000348221, 2024-04-23T10:42:56.192 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000021763 2024-04-23T10:42:56.192 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-23T10:42:56.192 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:56.192 INFO:teuthology.orchestra.run.smithi183.stdout: "mds": { 2024-04-23T10:42:56.192 INFO:teuthology.orchestra.run.smithi183.stdout: "request": 856, 2024-04-23T10:42:56.192 INFO:teuthology.orchestra.run.smithi183.stdout: "reply": 856, 2024-04-23T10:42:56.192 INFO:teuthology.orchestra.run.smithi183.stdout: "reply_latency": { 2024-04-23T10:42:56.192 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 856, 2024-04-23T10:42:56.192 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 20.319875655, 2024-04-23T10:42:56.192 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.023738172 2024-04-23T10:42:56.192 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:56.193 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_reply": 0, 2024-04-23T10:42:56.193 INFO:teuthology.orchestra.run.smithi183.stdout: "forward": 0, 2024-04-23T10:42:56.193 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_complete": 0, 2024-04-23T10:42:56.193 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_keys": 0, 2024-04-23T10:42:56.193 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_commit": 484, 2024-04-23T10:42:56.193 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_split": 0, 2024-04-23T10:42:56.193 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_merge": 0, 2024-04-23T10:42:56.193 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes": 48, 2024-04-23T10:42:56.193 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_top": 17, 2024-04-23T10:42:56.193 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_bottom": 31, 2024-04-23T10:42:56.193 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pin_tail": 0, 2024-04-23T10:42:56.193 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pinned": 23, 2024-04-23T10:42:56.193 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_expired": 0, 2024-04-23T10:42:56.193 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_with_caps": 2, 2024-04-23T10:42:56.193 INFO:teuthology.orchestra.run.smithi183.stdout: "caps": 3, 2024-04-23T10:42:56.194 INFO:teuthology.orchestra.run.smithi183.stdout: "subtrees": 2, 2024-04-23T10:42:56.194 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse": 943, 2024-04-23T10:42:56.194 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_hit": 931, 2024-04-23T10:42:56.194 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_forward": 0, 2024-04-23T10:42:56.194 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_discover": 0, 2024-04-23T10:42:56.194 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_dir_fetch": 0, 2024-04-23T10:42:56.194 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_remote_ino": 0, 2024-04-23T10:42:56.194 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_lock": 0, 2024-04-23T10:42:56.194 INFO:teuthology.orchestra.run.smithi183.stdout: "load_cent": 0, 2024-04-23T10:42:56.194 INFO:teuthology.orchestra.run.smithi183.stdout: "q": 0, 2024-04-23T10:42:56.194 INFO:teuthology.orchestra.run.smithi183.stdout: "exported": 0, 2024-04-23T10:42:56.194 INFO:teuthology.orchestra.run.smithi183.stdout: "exported_inodes": 0, 2024-04-23T10:42:56.194 INFO:teuthology.orchestra.run.smithi183.stdout: "imported": 0, 2024-04-23T10:42:56.194 INFO:teuthology.orchestra.run.smithi183.stdout: "imported_inodes": 0, 2024-04-23T10:42:56.194 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_dir_fetch": 0, 2024-04-23T10:42:56.195 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_backtrace_fetch": 0, 2024-04-23T10:42:56.195 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_peer_discover": 0, 2024-04-23T10:42:56.195 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rfiles": 4, 2024-04-23T10:42:56.195 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rbytes": 234, 2024-04-23T10:42:56.195 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rsnaps": 6, 2024-04-23T10:42:56.195 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_fetch": 0, 2024-04-23T10:42:56.195 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_set_tag": 0, 2024-04-23T10:42:56.195 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_repaired": 0, 2024-04-23T10:42:56.195 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_inotable_repaired": 0, 2024-04-23T10:42:56.195 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_inodes": 0, 2024-04-23T10:42:56.195 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_base_inodes": 0, 2024-04-23T10:42:56.195 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dirfrag_rstats": 0, 2024-04-23T10:42:56.195 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_file_inodes": 0, 2024-04-23T10:42:56.195 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_inode_file_caps": 0, 2024-04-23T10:42:56.195 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_revoke": 675, 2024-04-23T10:42:56.196 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_grant": 674, 2024-04-23T10:42:56.196 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_trunc": 599, 2024-04-23T10:42:56.196 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-23T10:42:56.196 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-23T10:42:56.196 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps": 753, 2024-04-23T10:42:56.196 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps_dirty": 688, 2024-04-23T10:42:56.196 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_cap_release": 3, 2024-04-23T10:42:56.196 INFO:teuthology.orchestra.run.smithi183.stdout: "process_request_cap_release": 627 2024-04-23T10:42:56.196 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:56.196 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_cache": { 2024-04-23T10:42:56.196 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update": 21, 2024-04-23T10:42:56.196 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update_receipt": 0, 2024-04-23T10:42:56.196 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_try_discover": 0, 2024-04-23T10:42:56.196 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_send_discover": 0, 2024-04-23T10:42:56.196 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_handle_discover": 42, 2024-04-23T10:42:56.197 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays": 2, 2024-04-23T10:42:56.197 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_delayed": 0, 2024-04-23T10:42:56.197 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_enqueuing": 0, 2024-04-23T10:42:56.197 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_created": 3, 2024-04-23T10:42:56.197 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_enqueued": 1, 2024-04-23T10:42:56.197 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_reintegrated": 0, 2024-04-23T10:42:56.197 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_migrated": 0, 2024-04-23T10:42:56.197 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_processing": 0, 2024-04-23T10:42:56.197 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_enqueued": 0, 2024-04-23T10:42:56.197 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_prioritized": 0, 2024-04-23T10:42:56.197 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_started": 0, 2024-04-23T10:42:56.197 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_completed": 0, 2024-04-23T10:42:56.197 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_path": 17, 2024-04-23T10:42:56.197 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_inode": 234, 2024-04-23T10:42:56.197 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_enqueue_scrub": 0, 2024-04-23T10:42:56.198 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_exportdir": 0, 2024-04-23T10:42:56.198 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_flush": 0, 2024-04-23T10:42:56.198 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragmentdir": 0, 2024-04-23T10:42:56.198 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragstats": 0, 2024-04-23T10:42:56.198 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_inodestats": 0 2024-04-23T10:42:56.198 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:56.198 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_log": { 2024-04-23T10:42:56.198 INFO:teuthology.orchestra.run.smithi183.stdout: "evlrg": 0, 2024-04-23T10:42:56.198 INFO:teuthology.orchestra.run.smithi183.stdout: "evadd": 2649, 2024-04-23T10:42:56.198 INFO:teuthology.orchestra.run.smithi183.stdout: "evex": 2594, 2024-04-23T10:42:56.198 INFO:teuthology.orchestra.run.smithi183.stdout: "evtrm": 2594, 2024-04-23T10:42:56.198 INFO:teuthology.orchestra.run.smithi183.stdout: "ev": 55, 2024-04-23T10:42:56.198 INFO:teuthology.orchestra.run.smithi183.stdout: "evexg": 0, 2024-04-23T10:42:56.198 INFO:teuthology.orchestra.run.smithi183.stdout: "evexd": 47, 2024-04-23T10:42:56.198 INFO:teuthology.orchestra.run.smithi183.stdout: "segadd": 81, 2024-04-23T10:42:56.199 INFO:teuthology.orchestra.run.smithi183.stdout: "segex": 80, 2024-04-23T10:42:56.199 INFO:teuthology.orchestra.run.smithi183.stdout: "segtrm": 80, 2024-04-23T10:42:56.199 INFO:teuthology.orchestra.run.smithi183.stdout: "seg": 1, 2024-04-23T10:42:56.199 INFO:teuthology.orchestra.run.smithi183.stdout: "segmjr": 1, 2024-04-23T10:42:56.199 INFO:teuthology.orchestra.run.smithi183.stdout: "segexg": 0, 2024-04-23T10:42:56.199 INFO:teuthology.orchestra.run.smithi183.stdout: "segexd": 1, 2024-04-23T10:42:56.199 INFO:teuthology.orchestra.run.smithi183.stdout: "expos": 27911604, 2024-04-23T10:42:56.199 INFO:teuthology.orchestra.run.smithi183.stdout: "wrpos": 28498493, 2024-04-23T10:42:56.199 INFO:teuthology.orchestra.run.smithi183.stdout: "rdpos": 0, 2024-04-23T10:42:56.199 INFO:teuthology.orchestra.run.smithi183.stdout: "jlat": { 2024-04-23T10:42:56.199 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 2568, 2024-04-23T10:42:56.199 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 5.440528053, 2024-04-23T10:42:56.199 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.002118585 2024-04-23T10:42:56.199 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:56.199 INFO:teuthology.orchestra.run.smithi183.stdout: "replayed": 0 2024-04-23T10:42:56.199 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:56.200 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_mem": { 2024-04-23T10:42:56.200 INFO:teuthology.orchestra.run.smithi183.stdout: "ino": 40, 2024-04-23T10:42:56.200 INFO:teuthology.orchestra.run.smithi183.stdout: "ino+": 41, 2024-04-23T10:42:56.200 INFO:teuthology.orchestra.run.smithi183.stdout: "ino-": 1, 2024-04-23T10:42:56.200 INFO:teuthology.orchestra.run.smithi183.stdout: "dir": 20, 2024-04-23T10:42:56.200 INFO:teuthology.orchestra.run.smithi183.stdout: "dir+": 20, 2024-04-23T10:42:56.200 INFO:teuthology.orchestra.run.smithi183.stdout: "dir-": 0, 2024-04-23T10:42:56.200 INFO:teuthology.orchestra.run.smithi183.stdout: "dn": 47, 2024-04-23T10:42:56.200 INFO:teuthology.orchestra.run.smithi183.stdout: "dn+": 48, 2024-04-23T10:42:56.200 INFO:teuthology.orchestra.run.smithi183.stdout: "dn-": 1, 2024-04-23T10:42:56.200 INFO:teuthology.orchestra.run.smithi183.stdout: "cap": 3, 2024-04-23T10:42:56.200 INFO:teuthology.orchestra.run.smithi183.stdout: "cap+": 21, 2024-04-23T10:42:56.200 INFO:teuthology.orchestra.run.smithi183.stdout: "cap-": 18, 2024-04-23T10:42:56.200 INFO:teuthology.orchestra.run.smithi183.stdout: "rss": 61460, 2024-04-23T10:42:56.200 INFO:teuthology.orchestra.run.smithi183.stdout: "heap": 231800 2024-04-23T10:42:56.200 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:56.201 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_server": { 2024-04-23T10:42:56.201 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_client_request": 2131, 2024-04-23T10:42:56.201 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_server_request": 388, 2024-04-23T10:42:56.201 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_request": 856, 2024-04-23T10:42:56.201 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_session": 44, 2024-04-23T10:42:56.201 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_peer_request": 776, 2024-04-23T10:42:56.201 INFO:teuthology.orchestra.run.smithi183.stdout: "req_create_latency": { 2024-04-23T10:42:56.201 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 20, 2024-04-23T10:42:56.201 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.015010347, 2024-04-23T10:42:56.201 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000750517 2024-04-23T10:42:56.201 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:56.201 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getattr_latency": { 2024-04-23T10:42:56.201 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 164, 2024-04-23T10:42:56.201 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.032437577, 2024-04-23T10:42:56.201 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000197790 2024-04-23T10:42:56.201 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:56.202 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getfilelock_latency": { 2024-04-23T10:42:56.202 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:42:56.202 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:42:56.202 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:42:56.202 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:56.202 INFO:teuthology.orchestra.run.smithi183.stdout: "req_link_latency": { 2024-04-23T10:42:56.202 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:42:56.202 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:42:56.202 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:42:56.202 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:56.202 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookup_latency": { 2024-04-23T10:42:56.202 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 5, 2024-04-23T10:42:56.202 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.001796817, 2024-04-23T10:42:56.202 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000359363 2024-04-23T10:42:56.202 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:56.203 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookuphash_latency": { 2024-04-23T10:42:56.203 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:42:56.203 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:42:56.203 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:42:56.203 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:56.203 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupino_latency": { 2024-04-23T10:42:56.203 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1, 2024-04-23T10:42:56.203 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000241629, 2024-04-23T10:42:56.203 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000241629 2024-04-23T10:42:56.203 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:56.203 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupname_latency": { 2024-04-23T10:42:56.203 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:42:56.203 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:42:56.203 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:42:56.203 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:56.203 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupparent_latency": { 2024-04-23T10:42:56.204 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:42:56.204 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:42:56.204 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:42:56.204 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:56.204 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupsnap_latency": { 2024-04-23T10:42:56.204 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:42:56.204 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:42:56.204 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:42:56.204 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:56.204 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lssnap_latency": { 2024-04-23T10:42:56.204 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 16, 2024-04-23T10:42:56.204 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.003040550, 2024-04-23T10:42:56.205 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000190034 2024-04-23T10:42:56.205 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:56.205 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mkdir_latency": { 2024-04-23T10:42:56.205 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 8, 2024-04-23T10:42:56.205 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 16.710958816, 2024-04-23T10:42:56.205 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 2.088869852 2024-04-23T10:42:56.205 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:56.205 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mknod_latency": { 2024-04-23T10:42:56.205 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:42:56.205 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:42:56.205 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:42:56.206 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:56.206 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mksnap_latency": { 2024-04-23T10:42:56.206 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 16, 2024-04-23T10:42:56.206 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.078951798, 2024-04-23T10:42:56.206 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.004934487 2024-04-23T10:42:56.206 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:56.206 INFO:teuthology.orchestra.run.smithi183.stdout: "req_open_latency": { 2024-04-23T10:42:56.206 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1, 2024-04-23T10:42:56.206 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000329864, 2024-04-23T10:42:56.206 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000329864 2024-04-23T10:42:56.206 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:56.206 INFO:teuthology.orchestra.run.smithi183.stdout: "req_readdir_latency": { 2024-04-23T10:42:56.206 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 2, 2024-04-23T10:42:56.206 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.004785872, 2024-04-23T10:42:56.206 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.002392936 2024-04-23T10:42:56.207 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:56.207 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rename_latency": { 2024-04-23T10:42:56.207 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 2, 2024-04-23T10:42:56.207 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.002064513, 2024-04-23T10:42:56.207 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.001032256 2024-04-23T10:42:56.207 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:56.207 INFO:teuthology.orchestra.run.smithi183.stdout: "req_renamesnap_latency": { 2024-04-23T10:42:56.207 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:42:56.207 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:42:56.207 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:42:56.207 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:56.207 INFO:teuthology.orchestra.run.smithi183.stdout: "req_snapdiff_latency": { 2024-04-23T10:42:56.207 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:42:56.207 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:42:56.207 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:42:56.208 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:56.208 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmdir_latency": { 2024-04-23T10:42:56.208 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 2, 2024-04-23T10:42:56.208 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.013218529, 2024-04-23T10:42:56.208 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.006609264 2024-04-23T10:42:56.208 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:56.208 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmsnap_latency": { 2024-04-23T10:42:56.208 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 10, 2024-04-23T10:42:56.208 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.045057080, 2024-04-23T10:42:56.208 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.004505708 2024-04-23T10:42:56.208 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:56.208 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmxattr_latency": { 2024-04-23T10:42:56.208 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:42:56.208 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:42:56.208 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:42:56.209 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:56.209 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setattr_latency": { 2024-04-23T10:42:56.209 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 600, 2024-04-23T10:42:56.209 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 3.392882168, 2024-04-23T10:42:56.209 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.005654803 2024-04-23T10:42:56.209 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:56.209 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setdirlayout_latency": { 2024-04-23T10:42:56.209 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:42:56.209 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:42:56.209 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:42:56.209 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:56.209 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setfilelock_latency": { 2024-04-23T10:42:56.209 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:42:56.209 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:42:56.209 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:42:56.210 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:56.210 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setlayout_latency": { 2024-04-23T10:42:56.210 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:42:56.210 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:42:56.210 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:42:56.210 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:56.210 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setxattr_latency": { 2024-04-23T10:42:56.210 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 8, 2024-04-23T10:42:56.210 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.016912345, 2024-04-23T10:42:56.210 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.002114043 2024-04-23T10:42:56.210 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:56.210 INFO:teuthology.orchestra.run.smithi183.stdout: "req_symlink_latency": { 2024-04-23T10:42:56.210 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:42:56.210 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:42:56.211 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:42:56.211 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:56.211 INFO:teuthology.orchestra.run.smithi183.stdout: "req_unlink_latency": { 2024-04-23T10:42:56.211 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1, 2024-04-23T10:42:56.211 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.002187750, 2024-04-23T10:42:56.211 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.002187750 2024-04-23T10:42:56.211 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:56.211 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_revoke_eviction": 0, 2024-04-23T10:42:56.211 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_acquisition_throttle": 0, 2024-04-23T10:42:56.211 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getvxattr_latency": { 2024-04-23T10:42:56.211 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:42:56.211 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:42:56.211 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:42:56.211 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-23T10:42:56.211 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:56.211 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_sessions": { 2024-04-23T10:42:56.212 INFO:teuthology.orchestra.run.smithi183.stdout: "session_count": 2, 2024-04-23T10:42:56.212 INFO:teuthology.orchestra.run.smithi183.stdout: "session_add": 6, 2024-04-23T10:42:56.212 INFO:teuthology.orchestra.run.smithi183.stdout: "session_remove": 4, 2024-04-23T10:42:56.212 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_open": 2, 2024-04-23T10:42:56.212 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_stale": 0, 2024-04-23T10:42:56.212 INFO:teuthology.orchestra.run.smithi183.stdout: "total_load": 77, 2024-04-23T10:42:56.212 INFO:teuthology.orchestra.run.smithi183.stdout: "average_load": 38, 2024-04-23T10:42:56.212 INFO:teuthology.orchestra.run.smithi183.stdout: "avg_session_uptime": 980, 2024-04-23T10:42:56.212 INFO:teuthology.orchestra.run.smithi183.stdout: "mdthresh_evicted": 0 2024-04-23T10:42:56.212 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:56.212 INFO:teuthology.orchestra.run.smithi183.stdout: "mempool": { 2024-04-23T10:42:56.212 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_bytes": 0, 2024-04-23T10:42:56.212 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_items": 0, 2024-04-23T10:42:56.212 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_bytes": 0, 2024-04-23T10:42:56.213 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_items": 0, 2024-04-23T10:42:56.213 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_bytes": 0, 2024-04-23T10:42:56.213 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_items": 0, 2024-04-23T10:42:56.213 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-23T10:42:56.213 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_items": 0, 2024-04-23T10:42:56.213 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-23T10:42:56.213 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_items": 0, 2024-04-23T10:42:56.213 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_bytes": 0, 2024-04-23T10:42:56.213 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_items": 0, 2024-04-23T10:42:56.213 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-23T10:42:56.213 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_items": 0, 2024-04-23T10:42:56.213 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_bytes": 0, 2024-04-23T10:42:56.213 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_items": 0, 2024-04-23T10:42:56.213 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_bytes": 0, 2024-04-23T10:42:56.214 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_items": 0, 2024-04-23T10:42:56.214 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-23T10:42:56.214 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_items": 0, 2024-04-23T10:42:56.214 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-23T10:42:56.214 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_items": 0, 2024-04-23T10:42:56.214 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_bytes": 0, 2024-04-23T10:42:56.214 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_items": 0, 2024-04-23T10:42:56.214 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_bytes": 0, 2024-04-23T10:42:56.214 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_items": 0, 2024-04-23T10:42:56.214 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-23T10:42:56.214 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_items": 0, 2024-04-23T10:42:56.214 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_bytes": 0, 2024-04-23T10:42:56.214 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_items": 0, 2024-04-23T10:42:56.214 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_bytes": 0, 2024-04-23T10:42:56.214 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_items": 0, 2024-04-23T10:42:56.215 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_bytes": 0, 2024-04-23T10:42:56.215 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_items": 0, 2024-04-23T10:42:56.215 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_bytes": 0, 2024-04-23T10:42:56.215 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_items": 0, 2024-04-23T10:42:56.215 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_bytes": 347550, 2024-04-23T10:42:56.215 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_items": 127, 2024-04-23T10:42:56.215 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_bytes": 0, 2024-04-23T10:42:56.215 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_items": 0, 2024-04-23T10:42:56.215 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_bytes": 0, 2024-04-23T10:42:56.215 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_items": 0, 2024-04-23T10:42:56.215 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_bytes": 0, 2024-04-23T10:42:56.215 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_items": 0, 2024-04-23T10:42:56.215 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_bytes": 0, 2024-04-23T10:42:56.215 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_items": 0, 2024-04-23T10:42:56.215 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_bytes": 7936, 2024-04-23T10:42:56.215 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_items": 165, 2024-04-23T10:42:56.216 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_bytes": 0, 2024-04-23T10:42:56.216 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_items": 0, 2024-04-23T10:42:56.216 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_bytes": 0, 2024-04-23T10:42:56.216 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_items": 0, 2024-04-23T10:42:56.216 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_bytes": 193176, 2024-04-23T10:42:56.216 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_items": 1557, 2024-04-23T10:42:56.216 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_bytes": 0, 2024-04-23T10:42:56.216 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_items": 0, 2024-04-23T10:42:56.216 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_bytes": 0, 2024-04-23T10:42:56.216 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_items": 0 2024-04-23T10:42:56.216 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:56.216 INFO:teuthology.orchestra.run.smithi183.stdout: "objecter": { 2024-04-23T10:42:56.216 INFO:teuthology.orchestra.run.smithi183.stdout: "op_active": 0, 2024-04-23T10:42:56.216 INFO:teuthology.orchestra.run.smithi183.stdout: "op_laggy": 0, 2024-04-23T10:42:56.216 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send": 4623, 2024-04-23T10:42:56.217 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send_bytes": 25999593, 2024-04-23T10:42:56.217 INFO:teuthology.orchestra.run.smithi183.stdout: "op_resend": 37, 2024-04-23T10:42:56.217 INFO:teuthology.orchestra.run.smithi183.stdout: "op_reply": 4623, 2024-04-23T10:42:56.217 INFO:teuthology.orchestra.run.smithi183.stdout: "op_latency": { 2024-04-23T10:42:56.217 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 4623, 2024-04-23T10:42:56.217 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 27.947233494, 2024-04-23T10:42:56.217 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.006045259 2024-04-23T10:42:56.217 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:56.217 INFO:teuthology.orchestra.run.smithi183.stdout: "op_inflight": 0, 2024-04-23T10:42:56.217 INFO:teuthology.orchestra.run.smithi183.stdout: "oplen_avg": { 2024-04-23T10:42:56.217 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 4623, 2024-04-23T10:42:56.217 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 5626 2024-04-23T10:42:56.217 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:56.217 INFO:teuthology.orchestra.run.smithi183.stdout: "op": 4623, 2024-04-23T10:42:56.217 INFO:teuthology.orchestra.run.smithi183.stdout: "op_r": 1, 2024-04-23T10:42:56.217 INFO:teuthology.orchestra.run.smithi183.stdout: "op_w": 4622, 2024-04-23T10:42:56.218 INFO:teuthology.orchestra.run.smithi183.stdout: "op_rmw": 0, 2024-04-23T10:42:56.218 INFO:teuthology.orchestra.run.smithi183.stdout: "op_pg": 0, 2024-04-23T10:42:56.218 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_stat": 464, 2024-04-23T10:42:56.218 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_create": 25, 2024-04-23T10:42:56.218 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_read": 1, 2024-04-23T10:42:56.218 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_write": 2569, 2024-04-23T10:42:56.218 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writefull": 257, 2024-04-23T10:42:56.218 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writesame": 0, 2024-04-23T10:42:56.218 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_append": 0, 2024-04-23T10:42:56.218 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_zero": 0, 2024-04-23T10:42:56.218 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_truncate": 0, 2024-04-23T10:42:56.218 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_delete": 523, 2024-04-23T10:42:56.218 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_mapext": 0, 2024-04-23T10:42:56.218 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_sparse_read": 0, 2024-04-23T10:42:56.218 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_clonerange": 0, 2024-04-23T10:42:56.219 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_getxattr": 0, 2024-04-23T10:42:56.219 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_setxattr": 50, 2024-04-23T10:42:56.219 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_cmpxattr": 0, 2024-04-23T10:42:56.219 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_rmxattr": 0, 2024-04-23T10:42:56.219 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_resetxattrs": 0, 2024-04-23T10:42:56.219 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_call": 0, 2024-04-23T10:42:56.219 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_watch": 0, 2024-04-23T10:42:56.219 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_notify": 0, 2024-04-23T10:42:56.219 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_src_cmpxattr": 0, 2024-04-23T10:42:56.219 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls": 0, 2024-04-23T10:42:56.219 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls_filter": 0, 2024-04-23T10:42:56.219 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_other": 599, 2024-04-23T10:42:56.219 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_active": 0, 2024-04-23T10:42:56.219 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_send": 0, 2024-04-23T10:42:56.219 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_resend": 0, 2024-04-23T10:42:56.219 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_ping": 0, 2024-04-23T10:42:56.220 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_active": 0, 2024-04-23T10:42:56.220 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_send": 0, 2024-04-23T10:42:56.220 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_resend": 0, 2024-04-23T10:42:56.220 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_active": 0, 2024-04-23T10:42:56.220 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_send": 0, 2024-04-23T10:42:56.220 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_resend": 0, 2024-04-23T10:42:56.220 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_active": 0, 2024-04-23T10:42:56.220 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_send": 0, 2024-04-23T10:42:56.220 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_resend": 0, 2024-04-23T10:42:56.220 INFO:teuthology.orchestra.run.smithi183.stdout: "command_active": 0, 2024-04-23T10:42:56.220 INFO:teuthology.orchestra.run.smithi183.stdout: "command_send": 0, 2024-04-23T10:42:56.220 INFO:teuthology.orchestra.run.smithi183.stdout: "command_resend": 0, 2024-04-23T10:42:56.220 INFO:teuthology.orchestra.run.smithi183.stdout: "map_epoch": 109, 2024-04-23T10:42:56.220 INFO:teuthology.orchestra.run.smithi183.stdout: "map_full": 0, 2024-04-23T10:42:56.220 INFO:teuthology.orchestra.run.smithi183.stdout: "map_inc": 24, 2024-04-23T10:42:56.220 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_sessions": 12, 2024-04-23T10:42:56.221 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_open": 12, 2024-04-23T10:42:56.221 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_close": 0, 2024-04-23T10:42:56.221 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_laggy": 0, 2024-04-23T10:42:56.221 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_wr": 1128, 2024-04-23T10:42:56.221 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_rd": 0, 2024-04-23T10:42:56.221 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_del": 10 2024-04-23T10:42:56.221 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:56.221 INFO:teuthology.orchestra.run.smithi183.stdout: "oft": { 2024-04-23T10:42:56.221 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_objs": 1, 2024-04-23T10:42:56.221 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_kv_pairs": 0, 2024-04-23T10:42:56.221 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_updates": 0, 2024-04-23T10:42:56.221 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_removes": 0 2024-04-23T10:42:56.221 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:56.221 INFO:teuthology.orchestra.run.smithi183.stdout: "osd-slow-ops": { 2024-04-23T10:42:56.221 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_ops_count": 639 2024-04-23T10:42:56.222 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:56.222 INFO:teuthology.orchestra.run.smithi183.stdout: "purge_queue": { 2024-04-23T10:42:56.222 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops": 0, 2024-04-23T10:42:56.222 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops_high_water": 1, 2024-04-23T10:42:56.222 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing": 0, 2024-04-23T10:42:56.222 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_high_water": 1, 2024-04-23T10:42:56.222 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed_ops": 1, 2024-04-23T10:42:56.222 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed": 1, 2024-04-23T10:42:56.222 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_item_in_journal": 0 2024-04-23T10:42:56.222 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:56.222 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-23T10:42:56.222 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-23T10:42:56.222 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-04-23T10:42:56.222 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-23T10:42:56.222 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 15431, 2024-04-23T10:42:56.223 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 4637798, 2024-04-23T10:42:56.223 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-23T10:42:56.223 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 15431, 2024-04-23T10:42:56.223 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-04-23T10:42:56.223 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-04-23T10:42:56.223 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 15431, 2024-04-23T10:42:56.223 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 4637798, 2024-04-23T10:42:56.223 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-23T10:42:56.223 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:42:56.223 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:42:56.223 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:42:56.223 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-23T10:42:56.223 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:56.223 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_bytes": { 2024-04-23T10:42:56.223 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-23T10:42:56.223 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-04-23T10:42:56.223 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-23T10:42:56.224 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-04-23T10:42:56.224 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-04-23T10:42:56.224 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-23T10:42:56.224 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-04-23T10:42:56.224 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 4623, 2024-04-23T10:42:56.224 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 25999742, 2024-04-23T10:42:56.224 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 4623, 2024-04-23T10:42:56.224 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 25999742, 2024-04-23T10:42:56.224 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-23T10:42:56.224 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:42:56.224 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:42:56.224 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:42:56.224 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-23T10:42:56.224 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:56.224 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_ops": { 2024-04-23T10:42:56.225 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-23T10:42:56.225 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 1024, 2024-04-23T10:42:56.225 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-23T10:42:56.225 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-04-23T10:42:56.225 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-04-23T10:42:56.225 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-23T10:42:56.225 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-04-23T10:42:56.225 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 4623, 2024-04-23T10:42:56.225 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 4623, 2024-04-23T10:42:56.225 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 4623, 2024-04-23T10:42:56.225 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 4623, 2024-04-23T10:42:56.225 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-23T10:42:56.225 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:42:56.225 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:42:56.225 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:42:56.225 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-23T10:42:56.226 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:56.226 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle": { 2024-04-23T10:42:56.226 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-23T10:42:56.226 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-04-23T10:42:56.226 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-23T10:42:56.226 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 1, 2024-04-23T10:42:56.226 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 149, 2024-04-23T10:42:56.226 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-23T10:42:56.226 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 1, 2024-04-23T10:42:56.226 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-04-23T10:42:56.226 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-04-23T10:42:56.226 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 1, 2024-04-23T10:42:56.226 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 149, 2024-04-23T10:42:56.226 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-23T10:42:56.226 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:42:56.227 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:42:56.227 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:42:56.227 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-23T10:42:56.227 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:56.227 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle-0x56225e35c500": { 2024-04-23T10:42:56.227 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-23T10:42:56.227 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-04-23T10:42:56.227 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-23T10:42:56.227 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 2649, 2024-04-23T10:42:56.227 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 24304189, 2024-04-23T10:42:56.227 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-23T10:42:56.227 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 2649, 2024-04-23T10:42:56.227 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-04-23T10:42:56.227 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-04-23T10:42:56.227 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 2568, 2024-04-23T10:42:56.227 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 24304189, 2024-04-23T10:42:56.227 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-23T10:42:56.228 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:42:56.228 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:42:56.228 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:42:56.228 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-23T10:42:56.228 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-23T10:42:56.228 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-04-23T10:42:57.086 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-04-23T10:42:57.087 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-04-23T10:42:57.087 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=21 2024-04-23T10:42:57.087 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2024-04-23T10:42:57.087 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-04-23T10:42:57.087 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=16 2024-04-23T10:42:57.087 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=10 2024-04-23T10:42:57.087 DEBUG:tasks.check_counter:Getting stats from e 2024-04-23T10:42:57.087 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-23T10:42:57.335 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.b/config 2024-04-23T10:42:57.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:42:57 smithi183 ceph-mon[32317]: pgmap v525: 97 pgs: 97 active+clean; 4.4 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.7 KiB/s wr, 9 op/s 2024-04-23T10:42:57.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:42:57 smithi196 ceph-mon[32919]: pgmap v525: 97 pgs: 97 active+clean; 4.4 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.7 KiB/s wr, 9 op/s 2024-04-23T10:42:57.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:42:57 smithi097 ceph-mon[29197]: pgmap v525: 97 pgs: 97 active+clean; 4.4 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.7 KiB/s wr, 9 op/s 2024-04-23T10:42:59.120 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-04-23T10:42:59.120 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-0": { 2024-04-23T10:42:59.121 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 1140, 2024-04-23T10:42:59.121 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 1140, 2024-04-23T10:42:59.121 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 303399, 2024-04-23T10:42:59.121 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 361760, 2024-04-23T10:42:59.121 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 7, 2024-04-23T10:42:59.121 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 2, 2024-04-23T10:42:59.121 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.109601813, 2024-04-23T10:42:59.121 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.045998895, 2024-04-23T10:42:59.121 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.024875298, 2024-04-23T10:42:59.122 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.027188968, 2024-04-23T10:42:59.122 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T10:42:59.122 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1139, 2024-04-23T10:42:59.122 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.047187631, 2024-04-23T10:42:59.122 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000041429 2024-04-23T10:42:59.122 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:59.122 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-04-23T10:42:59.123 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:42:59.123 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:42:59.123 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:42:59.123 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:59.123 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-04-23T10:42:59.123 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 448 2024-04-23T10:42:59.123 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:59.123 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-1": { 2024-04-23T10:42:59.124 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 1404, 2024-04-23T10:42:59.124 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 1387, 2024-04-23T10:42:59.124 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 1938882, 2024-04-23T10:42:59.124 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 529425, 2024-04-23T10:42:59.124 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 4, 2024-04-23T10:42:59.124 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 3, 2024-04-23T10:42:59.124 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.160686299, 2024-04-23T10:42:59.124 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.053007645, 2024-04-23T10:42:59.124 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.062455803, 2024-04-23T10:42:59.124 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.043355432, 2024-04-23T10:42:59.125 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T10:42:59.125 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1386, 2024-04-23T10:42:59.125 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.054451130, 2024-04-23T10:42:59.125 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000039286 2024-04-23T10:42:59.125 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:59.125 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-04-23T10:42:59.125 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:42:59.125 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:42:59.125 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:42:59.125 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:59.126 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 1680560, 2024-04-23T10:42:59.126 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 195520 2024-04-23T10:42:59.126 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:59.126 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-2": { 2024-04-23T10:42:59.126 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 1129, 2024-04-23T10:42:59.126 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 1363, 2024-04-23T10:42:59.126 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 378992, 2024-04-23T10:42:59.126 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 938290, 2024-04-23T10:42:59.126 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 6, 2024-04-23T10:42:59.127 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 4, 2024-04-23T10:42:59.127 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.155475911, 2024-04-23T10:42:59.127 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.051129071, 2024-04-23T10:42:59.127 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.039533903, 2024-04-23T10:42:59.127 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.048095853, 2024-04-23T10:42:59.127 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T10:42:59.127 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1360, 2024-04-23T10:42:59.127 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.085158440, 2024-04-23T10:42:59.127 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000062616 2024-04-23T10:42:59.128 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:59.128 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-04-23T10:42:59.128 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:42:59.128 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:42:59.128 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:42:59.128 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:59.128 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-23T10:42:59.128 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 579904 2024-04-23T10:42:59.128 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:59.128 INFO:teuthology.orchestra.run.smithi183.stdout: "cct": { 2024-04-23T10:42:59.129 INFO:teuthology.orchestra.run.smithi183.stdout: "total_workers": 1, 2024-04-23T10:42:59.129 INFO:teuthology.orchestra.run.smithi183.stdout: "unhealthy_workers": 0 2024-04-23T10:42:59.129 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:59.129 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-MDSRank": { 2024-04-23T10:42:59.129 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-04-23T10:42:59.129 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-04-23T10:42:59.129 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 5448, 2024-04-23T10:42:59.129 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.168322596, 2024-04-23T10:42:59.129 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000030896 2024-04-23T10:42:59.129 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-23T10:42:59.130 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:59.130 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-PurgeQueue": { 2024-04-23T10:42:59.130 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-04-23T10:42:59.130 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-04-23T10:42:59.130 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:42:59.130 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:42:59.130 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:42:59.130 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-23T10:42:59.130 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:59.131 INFO:teuthology.orchestra.run.smithi183.stdout: "mds": { 2024-04-23T10:42:59.131 INFO:teuthology.orchestra.run.smithi183.stdout: "request": 0, 2024-04-23T10:42:59.131 INFO:teuthology.orchestra.run.smithi183.stdout: "reply": 0, 2024-04-23T10:42:59.131 INFO:teuthology.orchestra.run.smithi183.stdout: "reply_latency": { 2024-04-23T10:42:59.131 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:42:59.131 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:42:59.131 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:42:59.131 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:59.131 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_reply": 0, 2024-04-23T10:42:59.131 INFO:teuthology.orchestra.run.smithi183.stdout: "forward": 0, 2024-04-23T10:42:59.132 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_complete": 0, 2024-04-23T10:42:59.132 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_keys": 0, 2024-04-23T10:42:59.132 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_commit": 0, 2024-04-23T10:42:59.132 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_split": 0, 2024-04-23T10:42:59.132 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_merge": 0, 2024-04-23T10:42:59.132 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes": 0, 2024-04-23T10:42:59.132 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_top": 0, 2024-04-23T10:42:59.132 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_bottom": 0, 2024-04-23T10:42:59.133 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pin_tail": 0, 2024-04-23T10:42:59.133 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pinned": 0, 2024-04-23T10:42:59.133 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_expired": 0, 2024-04-23T10:42:59.133 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_with_caps": 0, 2024-04-23T10:42:59.133 INFO:teuthology.orchestra.run.smithi183.stdout: "caps": 0, 2024-04-23T10:42:59.133 INFO:teuthology.orchestra.run.smithi183.stdout: "subtrees": 1, 2024-04-23T10:42:59.133 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse": 0, 2024-04-23T10:42:59.133 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_hit": 0, 2024-04-23T10:42:59.133 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_forward": 0, 2024-04-23T10:42:59.133 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_discover": 0, 2024-04-23T10:42:59.133 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_dir_fetch": 0, 2024-04-23T10:42:59.134 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_remote_ino": 0, 2024-04-23T10:42:59.134 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_lock": 0, 2024-04-23T10:42:59.134 INFO:teuthology.orchestra.run.smithi183.stdout: "load_cent": 0, 2024-04-23T10:42:59.134 INFO:teuthology.orchestra.run.smithi183.stdout: "q": 0, 2024-04-23T10:42:59.134 INFO:teuthology.orchestra.run.smithi183.stdout: "exported": 0, 2024-04-23T10:42:59.134 INFO:teuthology.orchestra.run.smithi183.stdout: "exported_inodes": 0, 2024-04-23T10:42:59.134 INFO:teuthology.orchestra.run.smithi183.stdout: "imported": 0, 2024-04-23T10:42:59.134 INFO:teuthology.orchestra.run.smithi183.stdout: "imported_inodes": 0, 2024-04-23T10:42:59.134 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_dir_fetch": 0, 2024-04-23T10:42:59.134 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_backtrace_fetch": 0, 2024-04-23T10:42:59.135 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_peer_discover": 0, 2024-04-23T10:42:59.135 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rfiles": 0, 2024-04-23T10:42:59.135 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rbytes": 0, 2024-04-23T10:42:59.135 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rsnaps": 0, 2024-04-23T10:42:59.135 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_fetch": 0, 2024-04-23T10:42:59.135 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_set_tag": 0, 2024-04-23T10:42:59.135 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_repaired": 0, 2024-04-23T10:42:59.135 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_inotable_repaired": 0, 2024-04-23T10:42:59.135 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_inodes": 0, 2024-04-23T10:42:59.135 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_base_inodes": 0, 2024-04-23T10:42:59.136 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dirfrag_rstats": 0, 2024-04-23T10:42:59.136 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_file_inodes": 0, 2024-04-23T10:42:59.136 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_inode_file_caps": 0, 2024-04-23T10:42:59.136 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_revoke": 0, 2024-04-23T10:42:59.136 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_grant": 0, 2024-04-23T10:42:59.136 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_trunc": 0, 2024-04-23T10:42:59.136 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-23T10:42:59.136 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-23T10:42:59.136 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps": 0, 2024-04-23T10:42:59.136 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps_dirty": 0, 2024-04-23T10:42:59.136 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_cap_release": 0, 2024-04-23T10:42:59.137 INFO:teuthology.orchestra.run.smithi183.stdout: "process_request_cap_release": 0 2024-04-23T10:42:59.137 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:59.137 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_cache": { 2024-04-23T10:42:59.137 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update": 0, 2024-04-23T10:42:59.137 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update_receipt": 0, 2024-04-23T10:42:59.137 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_try_discover": 0, 2024-04-23T10:42:59.137 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_send_discover": 0, 2024-04-23T10:42:59.137 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_handle_discover": 0, 2024-04-23T10:42:59.137 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays": 0, 2024-04-23T10:42:59.137 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_delayed": 0, 2024-04-23T10:42:59.138 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_enqueuing": 0, 2024-04-23T10:42:59.138 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_created": 0, 2024-04-23T10:42:59.138 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_enqueued": 0, 2024-04-23T10:42:59.138 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_reintegrated": 0, 2024-04-23T10:42:59.138 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_migrated": 0, 2024-04-23T10:42:59.138 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_processing": 0, 2024-04-23T10:42:59.138 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_enqueued": 0, 2024-04-23T10:42:59.138 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_prioritized": 0, 2024-04-23T10:42:59.138 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_started": 0, 2024-04-23T10:42:59.138 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_completed": 0, 2024-04-23T10:42:59.139 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_path": 0, 2024-04-23T10:42:59.139 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_inode": 0, 2024-04-23T10:42:59.139 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_enqueue_scrub": 0, 2024-04-23T10:42:59.139 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_exportdir": 0, 2024-04-23T10:42:59.139 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_flush": 0, 2024-04-23T10:42:59.139 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragmentdir": 0, 2024-04-23T10:42:59.139 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragstats": 0, 2024-04-23T10:42:59.139 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_inodestats": 0 2024-04-23T10:42:59.139 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:59.139 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_log": { 2024-04-23T10:42:59.139 INFO:teuthology.orchestra.run.smithi183.stdout: "evlrg": 0, 2024-04-23T10:42:59.140 INFO:teuthology.orchestra.run.smithi183.stdout: "evadd": 0, 2024-04-23T10:42:59.140 INFO:teuthology.orchestra.run.smithi183.stdout: "evex": 0, 2024-04-23T10:42:59.140 INFO:teuthology.orchestra.run.smithi183.stdout: "evtrm": 0, 2024-04-23T10:42:59.140 INFO:teuthology.orchestra.run.smithi183.stdout: "ev": 8, 2024-04-23T10:42:59.140 INFO:teuthology.orchestra.run.smithi183.stdout: "evexg": 0, 2024-04-23T10:42:59.140 INFO:teuthology.orchestra.run.smithi183.stdout: "evexd": 0, 2024-04-23T10:42:59.140 INFO:teuthology.orchestra.run.smithi183.stdout: "segadd": 0, 2024-04-23T10:42:59.140 INFO:teuthology.orchestra.run.smithi183.stdout: "segex": 0, 2024-04-23T10:42:59.140 INFO:teuthology.orchestra.run.smithi183.stdout: "segtrm": 0, 2024-04-23T10:42:59.140 INFO:teuthology.orchestra.run.smithi183.stdout: "seg": 2, 2024-04-23T10:42:59.141 INFO:teuthology.orchestra.run.smithi183.stdout: "segmjr": 2, 2024-04-23T10:42:59.141 INFO:teuthology.orchestra.run.smithi183.stdout: "segexg": 0, 2024-04-23T10:42:59.141 INFO:teuthology.orchestra.run.smithi183.stdout: "segexd": 0, 2024-04-23T10:42:59.141 INFO:teuthology.orchestra.run.smithi183.stdout: "expos": 4194304, 2024-04-23T10:42:59.141 INFO:teuthology.orchestra.run.smithi183.stdout: "wrpos": 4196488, 2024-04-23T10:42:59.141 INFO:teuthology.orchestra.run.smithi183.stdout: "rdpos": 4196347, 2024-04-23T10:42:59.141 INFO:teuthology.orchestra.run.smithi183.stdout: "jlat": { 2024-04-23T10:42:59.141 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:42:59.141 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:42:59.141 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:42:59.142 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:59.142 INFO:teuthology.orchestra.run.smithi183.stdout: "replayed": 8 2024-04-23T10:42:59.142 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:59.142 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_mem": { 2024-04-23T10:42:59.142 INFO:teuthology.orchestra.run.smithi183.stdout: "ino": 3, 2024-04-23T10:42:59.142 INFO:teuthology.orchestra.run.smithi183.stdout: "ino+": 3, 2024-04-23T10:42:59.142 INFO:teuthology.orchestra.run.smithi183.stdout: "ino-": 0, 2024-04-23T10:42:59.142 INFO:teuthology.orchestra.run.smithi183.stdout: "dir": 1, 2024-04-23T10:42:59.142 INFO:teuthology.orchestra.run.smithi183.stdout: "dir+": 1, 2024-04-23T10:42:59.142 INFO:teuthology.orchestra.run.smithi183.stdout: "dir-": 0, 2024-04-23T10:42:59.143 INFO:teuthology.orchestra.run.smithi183.stdout: "dn": 0, 2024-04-23T10:42:59.143 INFO:teuthology.orchestra.run.smithi183.stdout: "dn+": 0, 2024-04-23T10:42:59.143 INFO:teuthology.orchestra.run.smithi183.stdout: "dn-": 0, 2024-04-23T10:42:59.143 INFO:teuthology.orchestra.run.smithi183.stdout: "cap": 0, 2024-04-23T10:42:59.143 INFO:teuthology.orchestra.run.smithi183.stdout: "cap+": 0, 2024-04-23T10:42:59.143 INFO:teuthology.orchestra.run.smithi183.stdout: "cap-": 0, 2024-04-23T10:42:59.143 INFO:teuthology.orchestra.run.smithi183.stdout: "rss": 49900, 2024-04-23T10:42:59.143 INFO:teuthology.orchestra.run.smithi183.stdout: "heap": 215416 2024-04-23T10:42:59.143 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:59.143 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_server": { 2024-04-23T10:42:59.143 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_client_request": 0, 2024-04-23T10:42:59.144 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_server_request": 0, 2024-04-23T10:42:59.144 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_request": 0, 2024-04-23T10:42:59.144 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_session": 0, 2024-04-23T10:42:59.144 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_peer_request": 0, 2024-04-23T10:42:59.144 INFO:teuthology.orchestra.run.smithi183.stdout: "req_create_latency": { 2024-04-23T10:42:59.144 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:42:59.144 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:42:59.144 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:42:59.144 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:59.144 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getattr_latency": { 2024-04-23T10:42:59.144 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:42:59.145 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:42:59.145 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:42:59.145 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:59.145 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getfilelock_latency": { 2024-04-23T10:42:59.145 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:42:59.145 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:42:59.145 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:42:59.145 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:59.145 INFO:teuthology.orchestra.run.smithi183.stdout: "req_link_latency": { 2024-04-23T10:42:59.145 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:42:59.145 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:42:59.146 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:42:59.146 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:59.146 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookup_latency": { 2024-04-23T10:42:59.146 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:42:59.146 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:42:59.146 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:42:59.146 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:59.146 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookuphash_latency": { 2024-04-23T10:42:59.146 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:42:59.146 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:42:59.146 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:42:59.147 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:59.147 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupino_latency": { 2024-04-23T10:42:59.147 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:42:59.147 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:42:59.147 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:42:59.147 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:59.147 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupname_latency": { 2024-04-23T10:42:59.147 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:42:59.147 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:42:59.147 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:42:59.147 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:59.148 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupparent_latency": { 2024-04-23T10:42:59.148 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:42:59.148 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:42:59.148 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:42:59.148 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:59.148 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupsnap_latency": { 2024-04-23T10:42:59.148 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:42:59.148 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:42:59.148 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:42:59.148 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:59.148 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lssnap_latency": { 2024-04-23T10:42:59.149 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:42:59.149 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:42:59.149 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:42:59.149 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:59.149 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mkdir_latency": { 2024-04-23T10:42:59.149 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:42:59.150 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:42:59.150 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:42:59.150 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:59.150 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mknod_latency": { 2024-04-23T10:42:59.150 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:42:59.150 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:42:59.150 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:42:59.150 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:59.151 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mksnap_latency": { 2024-04-23T10:42:59.151 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:42:59.151 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:42:59.151 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:42:59.151 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:59.151 INFO:teuthology.orchestra.run.smithi183.stdout: "req_open_latency": { 2024-04-23T10:42:59.151 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:42:59.151 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:42:59.151 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:42:59.151 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:59.151 INFO:teuthology.orchestra.run.smithi183.stdout: "req_readdir_latency": { 2024-04-23T10:42:59.152 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:42:59.152 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:42:59.152 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:42:59.152 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:59.152 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rename_latency": { 2024-04-23T10:42:59.152 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:42:59.152 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:42:59.152 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:42:59.152 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:59.152 INFO:teuthology.orchestra.run.smithi183.stdout: "req_renamesnap_latency": { 2024-04-23T10:42:59.153 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:42:59.153 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:42:59.153 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:42:59.153 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:59.153 INFO:teuthology.orchestra.run.smithi183.stdout: "req_snapdiff_latency": { 2024-04-23T10:42:59.153 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:42:59.153 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:42:59.153 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:42:59.153 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:59.153 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmdir_latency": { 2024-04-23T10:42:59.153 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:42:59.154 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:42:59.154 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:42:59.154 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:59.154 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmsnap_latency": { 2024-04-23T10:42:59.154 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:42:59.154 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:42:59.154 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:42:59.154 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:59.154 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmxattr_latency": { 2024-04-23T10:42:59.154 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:42:59.154 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:42:59.155 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:42:59.155 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:59.155 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setattr_latency": { 2024-04-23T10:42:59.155 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:42:59.155 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:42:59.155 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:42:59.155 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:59.155 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setdirlayout_latency": { 2024-04-23T10:42:59.155 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:42:59.155 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:42:59.155 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:42:59.156 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:59.156 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setfilelock_latency": { 2024-04-23T10:42:59.156 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:42:59.156 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:42:59.156 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:42:59.156 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:59.156 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setlayout_latency": { 2024-04-23T10:42:59.156 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:42:59.156 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:42:59.156 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:42:59.156 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:59.156 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setxattr_latency": { 2024-04-23T10:42:59.157 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:42:59.157 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:42:59.157 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:42:59.157 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:59.157 INFO:teuthology.orchestra.run.smithi183.stdout: "req_symlink_latency": { 2024-04-23T10:42:59.157 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:42:59.157 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:42:59.157 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:42:59.157 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:59.157 INFO:teuthology.orchestra.run.smithi183.stdout: "req_unlink_latency": { 2024-04-23T10:42:59.158 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:42:59.158 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:42:59.158 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:42:59.158 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:59.158 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_revoke_eviction": 0, 2024-04-23T10:42:59.158 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_acquisition_throttle": 0, 2024-04-23T10:42:59.158 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getvxattr_latency": { 2024-04-23T10:42:59.158 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:42:59.158 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:42:59.158 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:42:59.158 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-23T10:42:59.159 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:59.159 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_sessions": { 2024-04-23T10:42:59.159 INFO:teuthology.orchestra.run.smithi183.stdout: "session_count": 0, 2024-04-23T10:42:59.159 INFO:teuthology.orchestra.run.smithi183.stdout: "session_add": 3, 2024-04-23T10:42:59.159 INFO:teuthology.orchestra.run.smithi183.stdout: "session_remove": 3, 2024-04-23T10:42:59.159 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_open": 2, 2024-04-23T10:42:59.159 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_stale": 0, 2024-04-23T10:42:59.159 INFO:teuthology.orchestra.run.smithi183.stdout: "total_load": 0, 2024-04-23T10:42:59.159 INFO:teuthology.orchestra.run.smithi183.stdout: "average_load": 0, 2024-04-23T10:42:59.159 INFO:teuthology.orchestra.run.smithi183.stdout: "avg_session_uptime": 1750, 2024-04-23T10:42:59.159 INFO:teuthology.orchestra.run.smithi183.stdout: "mdthresh_evicted": 0 2024-04-23T10:42:59.160 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:59.160 INFO:teuthology.orchestra.run.smithi183.stdout: "mempool": { 2024-04-23T10:42:59.160 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_bytes": 0, 2024-04-23T10:42:59.160 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_items": 0, 2024-04-23T10:42:59.160 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_bytes": 0, 2024-04-23T10:42:59.160 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_items": 0, 2024-04-23T10:42:59.160 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_bytes": 0, 2024-04-23T10:42:59.160 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_items": 0, 2024-04-23T10:42:59.160 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-23T10:42:59.160 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_items": 0, 2024-04-23T10:42:59.160 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-23T10:42:59.161 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_items": 0, 2024-04-23T10:42:59.161 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_bytes": 0, 2024-04-23T10:42:59.161 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_items": 0, 2024-04-23T10:42:59.161 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-23T10:42:59.161 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_items": 0, 2024-04-23T10:42:59.161 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_bytes": 0, 2024-04-23T10:42:59.161 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_items": 0, 2024-04-23T10:42:59.161 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_bytes": 0, 2024-04-23T10:42:59.161 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_items": 0, 2024-04-23T10:42:59.161 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-23T10:42:59.161 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_items": 0, 2024-04-23T10:42:59.162 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-23T10:42:59.162 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_items": 0, 2024-04-23T10:42:59.162 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_bytes": 0, 2024-04-23T10:42:59.162 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_items": 0, 2024-04-23T10:42:59.162 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_bytes": 0, 2024-04-23T10:42:59.162 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_items": 0, 2024-04-23T10:42:59.162 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-23T10:42:59.162 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_items": 0, 2024-04-23T10:42:59.162 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_bytes": 0, 2024-04-23T10:42:59.162 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_items": 0, 2024-04-23T10:42:59.163 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_bytes": 0, 2024-04-23T10:42:59.163 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_items": 0, 2024-04-23T10:42:59.163 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_bytes": 0, 2024-04-23T10:42:59.163 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_items": 0, 2024-04-23T10:42:59.163 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_bytes": 0, 2024-04-23T10:42:59.163 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_items": 0, 2024-04-23T10:42:59.163 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_bytes": 190194, 2024-04-23T10:42:59.163 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_items": 54, 2024-04-23T10:42:59.163 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_bytes": 0, 2024-04-23T10:42:59.163 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_items": 0, 2024-04-23T10:42:59.163 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_bytes": 0, 2024-04-23T10:42:59.164 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_items": 0, 2024-04-23T10:42:59.164 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_bytes": 0, 2024-04-23T10:42:59.164 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_items": 0, 2024-04-23T10:42:59.164 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_bytes": 0, 2024-04-23T10:42:59.164 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_items": 0, 2024-04-23T10:42:59.164 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_bytes": 7936, 2024-04-23T10:42:59.164 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_items": 165, 2024-04-23T10:42:59.164 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_bytes": 0, 2024-04-23T10:42:59.164 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_items": 0, 2024-04-23T10:42:59.164 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_bytes": 0, 2024-04-23T10:42:59.164 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_items": 0, 2024-04-23T10:42:59.164 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_bytes": 8512, 2024-04-23T10:42:59.165 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_items": 13, 2024-04-23T10:42:59.165 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_bytes": 0, 2024-04-23T10:42:59.165 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_items": 0, 2024-04-23T10:42:59.165 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_bytes": 0, 2024-04-23T10:42:59.165 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_items": 0 2024-04-23T10:42:59.165 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:59.165 INFO:teuthology.orchestra.run.smithi183.stdout: "objecter": { 2024-04-23T10:42:59.165 INFO:teuthology.orchestra.run.smithi183.stdout: "op_active": 0, 2024-04-23T10:42:59.165 INFO:teuthology.orchestra.run.smithi183.stdout: "op_laggy": 0, 2024-04-23T10:42:59.165 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send": 3313, 2024-04-23T10:42:59.165 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send_bytes": 21, 2024-04-23T10:42:59.166 INFO:teuthology.orchestra.run.smithi183.stdout: "op_resend": 3, 2024-04-23T10:42:59.166 INFO:teuthology.orchestra.run.smithi183.stdout: "op_reply": 3313, 2024-04-23T10:42:59.166 INFO:teuthology.orchestra.run.smithi183.stdout: "op_latency": { 2024-04-23T10:42:59.166 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 3313, 2024-04-23T10:42:59.166 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 1.677953912, 2024-04-23T10:42:59.166 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000506475 2024-04-23T10:42:59.166 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:59.166 INFO:teuthology.orchestra.run.smithi183.stdout: "op_inflight": 0, 2024-04-23T10:42:59.166 INFO:teuthology.orchestra.run.smithi183.stdout: "oplen_avg": { 2024-04-23T10:42:59.166 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 3313, 2024-04-23T10:42:59.166 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 3314 2024-04-23T10:42:59.167 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:59.167 INFO:teuthology.orchestra.run.smithi183.stdout: "op": 3313, 2024-04-23T10:42:59.167 INFO:teuthology.orchestra.run.smithi183.stdout: "op_r": 3313, 2024-04-23T10:42:59.167 INFO:teuthology.orchestra.run.smithi183.stdout: "op_w": 0, 2024-04-23T10:42:59.167 INFO:teuthology.orchestra.run.smithi183.stdout: "op_rmw": 0, 2024-04-23T10:42:59.167 INFO:teuthology.orchestra.run.smithi183.stdout: "op_pg": 0, 2024-04-23T10:42:59.167 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_stat": 2175, 2024-04-23T10:42:59.167 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_create": 0, 2024-04-23T10:42:59.167 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_read": 1136, 2024-04-23T10:42:59.167 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_write": 0, 2024-04-23T10:42:59.167 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writefull": 0, 2024-04-23T10:42:59.168 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writesame": 0, 2024-04-23T10:42:59.168 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_append": 0, 2024-04-23T10:42:59.168 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_zero": 0, 2024-04-23T10:42:59.168 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_truncate": 0, 2024-04-23T10:42:59.168 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_delete": 0, 2024-04-23T10:42:59.168 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_mapext": 0, 2024-04-23T10:42:59.168 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_sparse_read": 0, 2024-04-23T10:42:59.168 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_clonerange": 0, 2024-04-23T10:42:59.168 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_getxattr": 1, 2024-04-23T10:42:59.168 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_setxattr": 0, 2024-04-23T10:42:59.169 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_cmpxattr": 0, 2024-04-23T10:42:59.169 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_rmxattr": 0, 2024-04-23T10:42:59.169 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_resetxattrs": 0, 2024-04-23T10:42:59.169 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_call": 0, 2024-04-23T10:42:59.169 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_watch": 0, 2024-04-23T10:42:59.169 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_notify": 0, 2024-04-23T10:42:59.169 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_src_cmpxattr": 0, 2024-04-23T10:42:59.169 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls": 0, 2024-04-23T10:42:59.169 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls_filter": 0, 2024-04-23T10:42:59.169 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_other": 0, 2024-04-23T10:42:59.169 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_active": 0, 2024-04-23T10:42:59.169 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_send": 0, 2024-04-23T10:42:59.169 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_resend": 0, 2024-04-23T10:42:59.170 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_ping": 0, 2024-04-23T10:42:59.170 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_active": 0, 2024-04-23T10:42:59.170 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_send": 0, 2024-04-23T10:42:59.170 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_resend": 0, 2024-04-23T10:42:59.170 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_active": 0, 2024-04-23T10:42:59.170 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_send": 0, 2024-04-23T10:42:59.170 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_resend": 0, 2024-04-23T10:42:59.170 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_active": 0, 2024-04-23T10:42:59.170 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_send": 0, 2024-04-23T10:42:59.170 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_resend": 0, 2024-04-23T10:42:59.171 INFO:teuthology.orchestra.run.smithi183.stdout: "command_active": 0, 2024-04-23T10:42:59.171 INFO:teuthology.orchestra.run.smithi183.stdout: "command_send": 0, 2024-04-23T10:42:59.171 INFO:teuthology.orchestra.run.smithi183.stdout: "command_resend": 0, 2024-04-23T10:42:59.171 INFO:teuthology.orchestra.run.smithi183.stdout: "map_epoch": 109, 2024-04-23T10:42:59.171 INFO:teuthology.orchestra.run.smithi183.stdout: "map_full": 0, 2024-04-23T10:42:59.171 INFO:teuthology.orchestra.run.smithi183.stdout: "map_inc": 23, 2024-04-23T10:42:59.171 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_sessions": 5, 2024-04-23T10:42:59.171 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_open": 6, 2024-04-23T10:42:59.171 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_close": 1, 2024-04-23T10:42:59.171 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_laggy": 0, 2024-04-23T10:42:59.171 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_wr": 0, 2024-04-23T10:42:59.172 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_rd": 2, 2024-04-23T10:42:59.172 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_del": 0 2024-04-23T10:42:59.172 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:59.172 INFO:teuthology.orchestra.run.smithi183.stdout: "oft": { 2024-04-23T10:42:59.172 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_objs": 0, 2024-04-23T10:42:59.172 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_kv_pairs": 0, 2024-04-23T10:42:59.172 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_updates": 0, 2024-04-23T10:42:59.172 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_removes": 0 2024-04-23T10:42:59.172 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:59.172 INFO:teuthology.orchestra.run.smithi183.stdout: "osd-slow-ops": { 2024-04-23T10:42:59.172 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_ops_count": 0 2024-04-23T10:42:59.173 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:59.173 INFO:teuthology.orchestra.run.smithi183.stdout: "purge_queue": { 2024-04-23T10:42:59.173 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops": 0, 2024-04-23T10:42:59.173 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops_high_water": 0, 2024-04-23T10:42:59.173 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing": 0, 2024-04-23T10:42:59.173 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_high_water": 0, 2024-04-23T10:42:59.173 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed_ops": 0, 2024-04-23T10:42:59.173 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed": 0, 2024-04-23T10:42:59.173 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_item_in_journal": 0 2024-04-23T10:42:59.173 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:59.173 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-23T10:42:59.173 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-23T10:42:59.173 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-04-23T10:42:59.174 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-23T10:42:59.174 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 3669, 2024-04-23T10:42:59.174 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 2269898, 2024-04-23T10:42:59.174 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-23T10:42:59.174 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 3669, 2024-04-23T10:42:59.174 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-04-23T10:42:59.174 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-04-23T10:42:59.174 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 3669, 2024-04-23T10:42:59.174 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 2269898, 2024-04-23T10:42:59.174 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-23T10:42:59.174 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:42:59.175 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:42:59.175 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:42:59.175 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-23T10:42:59.175 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:59.175 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_bytes": { 2024-04-23T10:42:59.175 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-23T10:42:59.175 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-04-23T10:42:59.175 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-23T10:42:59.175 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-04-23T10:42:59.175 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-04-23T10:42:59.175 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-23T10:42:59.175 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-04-23T10:42:59.176 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 3313, 2024-04-23T10:42:59.176 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 2189, 2024-04-23T10:42:59.176 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 3313, 2024-04-23T10:42:59.176 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 2189, 2024-04-23T10:42:59.176 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-23T10:42:59.176 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:42:59.176 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:42:59.176 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:42:59.176 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-23T10:42:59.176 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:59.176 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_ops": { 2024-04-23T10:42:59.176 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-23T10:42:59.176 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 1024, 2024-04-23T10:42:59.177 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-23T10:42:59.177 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-04-23T10:42:59.177 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-04-23T10:42:59.177 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-23T10:42:59.177 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-04-23T10:42:59.177 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 3313, 2024-04-23T10:42:59.177 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 3313, 2024-04-23T10:42:59.177 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 3313, 2024-04-23T10:42:59.177 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 3313, 2024-04-23T10:42:59.177 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-23T10:42:59.177 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:42:59.177 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:42:59.178 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:42:59.178 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-23T10:42:59.178 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:59.178 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle": { 2024-04-23T10:42:59.178 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-23T10:42:59.178 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-04-23T10:42:59.178 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-23T10:42:59.178 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-04-23T10:42:59.178 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-04-23T10:42:59.178 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-23T10:42:59.178 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-04-23T10:42:59.178 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-04-23T10:42:59.179 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-04-23T10:42:59.179 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 0, 2024-04-23T10:42:59.179 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 0, 2024-04-23T10:42:59.179 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-23T10:42:59.179 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:42:59.179 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:42:59.179 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:42:59.179 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-23T10:42:59.179 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:42:59.179 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle-0x55dc3c4666e0": { 2024-04-23T10:42:59.179 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-23T10:42:59.179 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-04-23T10:42:59.179 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-23T10:42:59.180 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-04-23T10:42:59.180 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-04-23T10:42:59.180 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-23T10:42:59.180 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-04-23T10:42:59.180 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-04-23T10:42:59.180 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-04-23T10:42:59.180 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 0, 2024-04-23T10:42:59.180 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 0, 2024-04-23T10:42:59.180 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-23T10:42:59.180 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:42:59.180 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:42:59.180 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:42:59.181 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-23T10:42:59.181 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-23T10:42:59.181 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-04-23T10:42:59.400 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:42:59 smithi183 ceph-mon[32317]: pgmap v526: 97 pgs: 97 active+clean; 4.4 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.3 KiB/s wr, 14 op/s 2024-04-23T10:42:59.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:42:59 smithi196 ceph-mon[32919]: pgmap v526: 97 pgs: 97 active+clean; 4.4 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.3 KiB/s wr, 14 op/s 2024-04-23T10:42:59.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:42:59 smithi097 ceph-mon[29197]: pgmap v526: 97 pgs: 97 active+clean; 4.4 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.3 KiB/s wr, 14 op/s 2024-04-23T10:43:00.012 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-04-23T10:43:00.012 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-04-23T10:43:00.012 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-04-23T10:43:00.012 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-04-23T10:43:00.012 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-04-23T10:43:00.013 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-04-23T10:43:00.013 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-04-23T10:43:00.013 DEBUG:tasks.check_counter:Getting stats from h 2024-04-23T10:43:00.013 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-23T10:43:00.256 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.b/config 2024-04-23T10:43:01.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:43:01 smithi196 ceph-mon[32919]: pgmap v527: 97 pgs: 97 active+clean; 4.4 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.3 KiB/s wr, 9 op/s 2024-04-23T10:43:01.608 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:43:01 smithi183 ceph-mon[32317]: pgmap v527: 97 pgs: 97 active+clean; 4.4 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.3 KiB/s wr, 9 op/s 2024-04-23T10:43:01.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:43:01 smithi097 ceph-mon[29197]: pgmap v527: 97 pgs: 97 active+clean; 4.4 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.3 KiB/s wr, 9 op/s 2024-04-23T10:43:02.014 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-04-23T10:43:02.014 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-0": { 2024-04-23T10:43:02.015 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 2495, 2024-04-23T10:43:02.015 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 2478, 2024-04-23T10:43:02.015 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 10707346, 2024-04-23T10:43:02.015 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 875882, 2024-04-23T10:43:02.015 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 5, 2024-04-23T10:43:02.015 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 1, 2024-04-23T10:43:02.015 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.262041300, 2024-04-23T10:43:02.015 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.099295634, 2024-04-23T10:43:02.015 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.089080326, 2024-04-23T10:43:02.016 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.062621171, 2024-04-23T10:43:02.016 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T10:43:02.016 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 2477, 2024-04-23T10:43:02.016 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.103545681, 2024-04-23T10:43:02.016 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000041802 2024-04-23T10:43:02.016 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:02.016 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-04-23T10:43:02.016 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:02.016 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:02.016 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:02.017 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:02.017 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 1680448, 2024-04-23T10:43:02.017 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 195040 2024-04-23T10:43:02.017 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:02.017 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-1": { 2024-04-23T10:43:02.017 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 899, 2024-04-23T10:43:02.017 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 1131, 2024-04-23T10:43:02.017 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 8614902, 2024-04-23T10:43:02.017 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 863248, 2024-04-23T10:43:02.017 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 7, 2024-04-23T10:43:02.018 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 5, 2024-04-23T10:43:02.018 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.093068009, 2024-04-23T10:43:02.018 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.027125903, 2024-04-23T10:43:02.018 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.031081024, 2024-04-23T10:43:02.018 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.027053221, 2024-04-23T10:43:02.018 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T10:43:02.018 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1128, 2024-04-23T10:43:02.018 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.040679605, 2024-04-23T10:43:02.018 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000036063 2024-04-23T10:43:02.019 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:02.019 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-04-23T10:43:02.019 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:02.019 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:02.019 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:02.019 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:02.019 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-23T10:43:02.019 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 577936 2024-04-23T10:43:02.019 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:02.019 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-2": { 2024-04-23T10:43:02.020 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 1037, 2024-04-23T10:43:02.020 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 1037, 2024-04-23T10:43:02.020 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 7797823, 2024-04-23T10:43:02.020 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 328949, 2024-04-23T10:43:02.020 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 5, 2024-04-23T10:43:02.020 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 4, 2024-04-23T10:43:02.020 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.095884749, 2024-04-23T10:43:02.020 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.024310757, 2024-04-23T10:43:02.020 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.032526144, 2024-04-23T10:43:02.020 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.031487068, 2024-04-23T10:43:02.021 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T10:43:02.021 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1036, 2024-04-23T10:43:02.021 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.030365062, 2024-04-23T10:43:02.021 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000029309 2024-04-23T10:43:02.021 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:02.021 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-04-23T10:43:02.021 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:02.021 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:02.021 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:02.021 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:02.022 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-04-23T10:43:02.022 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 448 2024-04-23T10:43:02.022 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:02.022 INFO:teuthology.orchestra.run.smithi183.stdout: "cct": { 2024-04-23T10:43:02.022 INFO:teuthology.orchestra.run.smithi183.stdout: "total_workers": 1, 2024-04-23T10:43:02.022 INFO:teuthology.orchestra.run.smithi183.stdout: "unhealthy_workers": 0 2024-04-23T10:43:02.022 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:02.022 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-MDSRank": { 2024-04-23T10:43:02.023 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-04-23T10:43:02.023 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-04-23T10:43:02.023 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 6955, 2024-04-23T10:43:02.023 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.167114832, 2024-04-23T10:43:02.023 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000024028 2024-04-23T10:43:02.023 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-23T10:43:02.023 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:02.023 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-PurgeQueue": { 2024-04-23T10:43:02.023 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-04-23T10:43:02.023 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-04-23T10:43:02.024 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:02.024 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:02.024 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:02.024 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-23T10:43:02.024 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:02.024 INFO:teuthology.orchestra.run.smithi183.stdout: "mds": { 2024-04-23T10:43:02.024 INFO:teuthology.orchestra.run.smithi183.stdout: "request": 0, 2024-04-23T10:43:02.024 INFO:teuthology.orchestra.run.smithi183.stdout: "reply": 0, 2024-04-23T10:43:02.024 INFO:teuthology.orchestra.run.smithi183.stdout: "reply_latency": { 2024-04-23T10:43:02.024 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:02.025 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:02.025 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:02.025 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:02.025 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_reply": 0, 2024-04-23T10:43:02.025 INFO:teuthology.orchestra.run.smithi183.stdout: "forward": 0, 2024-04-23T10:43:02.025 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_complete": 0, 2024-04-23T10:43:02.025 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_keys": 0, 2024-04-23T10:43:02.025 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_commit": 0, 2024-04-23T10:43:02.025 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_split": 0, 2024-04-23T10:43:02.025 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_merge": 0, 2024-04-23T10:43:02.026 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes": 36, 2024-04-23T10:43:02.026 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_top": 16, 2024-04-23T10:43:02.026 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_bottom": 20, 2024-04-23T10:43:02.026 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pin_tail": 0, 2024-04-23T10:43:02.026 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pinned": 14, 2024-04-23T10:43:02.026 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_expired": 0, 2024-04-23T10:43:02.026 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_with_caps": 0, 2024-04-23T10:43:02.026 INFO:teuthology.orchestra.run.smithi183.stdout: "caps": 0, 2024-04-23T10:43:02.026 INFO:teuthology.orchestra.run.smithi183.stdout: "subtrees": 2, 2024-04-23T10:43:02.026 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse": 0, 2024-04-23T10:43:02.027 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_hit": 0, 2024-04-23T10:43:02.027 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_forward": 0, 2024-04-23T10:43:02.027 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_discover": 0, 2024-04-23T10:43:02.027 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_dir_fetch": 0, 2024-04-23T10:43:02.027 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_remote_ino": 0, 2024-04-23T10:43:02.027 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_lock": 0, 2024-04-23T10:43:02.027 INFO:teuthology.orchestra.run.smithi183.stdout: "load_cent": 0, 2024-04-23T10:43:02.027 INFO:teuthology.orchestra.run.smithi183.stdout: "q": 0, 2024-04-23T10:43:02.027 INFO:teuthology.orchestra.run.smithi183.stdout: "exported": 0, 2024-04-23T10:43:02.027 INFO:teuthology.orchestra.run.smithi183.stdout: "exported_inodes": 0, 2024-04-23T10:43:02.027 INFO:teuthology.orchestra.run.smithi183.stdout: "imported": 0, 2024-04-23T10:43:02.028 INFO:teuthology.orchestra.run.smithi183.stdout: "imported_inodes": 0, 2024-04-23T10:43:02.028 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_dir_fetch": 0, 2024-04-23T10:43:02.028 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_backtrace_fetch": 0, 2024-04-23T10:43:02.028 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_peer_discover": 0, 2024-04-23T10:43:02.028 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rfiles": 4, 2024-04-23T10:43:02.028 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rbytes": 234, 2024-04-23T10:43:02.028 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rsnaps": 6, 2024-04-23T10:43:02.028 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_fetch": 0, 2024-04-23T10:43:02.028 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_set_tag": 0, 2024-04-23T10:43:02.028 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_repaired": 0, 2024-04-23T10:43:02.029 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_inotable_repaired": 0, 2024-04-23T10:43:02.029 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_inodes": 0, 2024-04-23T10:43:02.029 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_base_inodes": 0, 2024-04-23T10:43:02.029 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dirfrag_rstats": 0, 2024-04-23T10:43:02.029 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_file_inodes": 0, 2024-04-23T10:43:02.029 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_inode_file_caps": 0, 2024-04-23T10:43:02.029 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_revoke": 0, 2024-04-23T10:43:02.029 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_grant": 0, 2024-04-23T10:43:02.029 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_trunc": 0, 2024-04-23T10:43:02.029 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-23T10:43:02.029 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-23T10:43:02.030 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps": 0, 2024-04-23T10:43:02.030 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps_dirty": 0, 2024-04-23T10:43:02.030 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_cap_release": 0, 2024-04-23T10:43:02.030 INFO:teuthology.orchestra.run.smithi183.stdout: "process_request_cap_release": 0 2024-04-23T10:43:02.030 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:02.030 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_cache": { 2024-04-23T10:43:02.030 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update": 0, 2024-04-23T10:43:02.030 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update_receipt": 0, 2024-04-23T10:43:02.030 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_try_discover": 0, 2024-04-23T10:43:02.031 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_send_discover": 0, 2024-04-23T10:43:02.031 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_handle_discover": 0, 2024-04-23T10:43:02.031 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays": 0, 2024-04-23T10:43:02.031 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_delayed": 0, 2024-04-23T10:43:02.031 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_enqueuing": 0, 2024-04-23T10:43:02.031 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_created": 0, 2024-04-23T10:43:02.031 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_enqueued": 0, 2024-04-23T10:43:02.031 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_reintegrated": 0, 2024-04-23T10:43:02.031 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_migrated": 0, 2024-04-23T10:43:02.031 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_processing": 0, 2024-04-23T10:43:02.031 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_enqueued": 0, 2024-04-23T10:43:02.032 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_prioritized": 0, 2024-04-23T10:43:02.032 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_started": 0, 2024-04-23T10:43:02.032 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_completed": 0, 2024-04-23T10:43:02.032 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_path": 0, 2024-04-23T10:43:02.032 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_inode": 0, 2024-04-23T10:43:02.032 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_enqueue_scrub": 0, 2024-04-23T10:43:02.032 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_exportdir": 0, 2024-04-23T10:43:02.032 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_flush": 0, 2024-04-23T10:43:02.032 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragmentdir": 0, 2024-04-23T10:43:02.032 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragstats": 0, 2024-04-23T10:43:02.033 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_inodestats": 0 2024-04-23T10:43:02.033 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:02.033 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_log": { 2024-04-23T10:43:02.033 INFO:teuthology.orchestra.run.smithi183.stdout: "evlrg": 0, 2024-04-23T10:43:02.033 INFO:teuthology.orchestra.run.smithi183.stdout: "evadd": 0, 2024-04-23T10:43:02.033 INFO:teuthology.orchestra.run.smithi183.stdout: "evex": 0, 2024-04-23T10:43:02.033 INFO:teuthology.orchestra.run.smithi183.stdout: "evtrm": 0, 2024-04-23T10:43:02.033 INFO:teuthology.orchestra.run.smithi183.stdout: "ev": 2656, 2024-04-23T10:43:02.033 INFO:teuthology.orchestra.run.smithi183.stdout: "evexg": 0, 2024-04-23T10:43:02.033 INFO:teuthology.orchestra.run.smithi183.stdout: "evexd": 0, 2024-04-23T10:43:02.033 INFO:teuthology.orchestra.run.smithi183.stdout: "segadd": 0, 2024-04-23T10:43:02.034 INFO:teuthology.orchestra.run.smithi183.stdout: "segex": 0, 2024-04-23T10:43:02.034 INFO:teuthology.orchestra.run.smithi183.stdout: "segtrm": 0, 2024-04-23T10:43:02.034 INFO:teuthology.orchestra.run.smithi183.stdout: "seg": 2, 2024-04-23T10:43:02.034 INFO:teuthology.orchestra.run.smithi183.stdout: "segmjr": 76, 2024-04-23T10:43:02.034 INFO:teuthology.orchestra.run.smithi183.stdout: "segexg": 0, 2024-04-23T10:43:02.034 INFO:teuthology.orchestra.run.smithi183.stdout: "segexd": 0, 2024-04-23T10:43:02.034 INFO:teuthology.orchestra.run.smithi183.stdout: "expos": 27911604, 2024-04-23T10:43:02.034 INFO:teuthology.orchestra.run.smithi183.stdout: "wrpos": 28507375, 2024-04-23T10:43:02.034 INFO:teuthology.orchestra.run.smithi183.stdout: "rdpos": 28507291, 2024-04-23T10:43:02.034 INFO:teuthology.orchestra.run.smithi183.stdout: "jlat": { 2024-04-23T10:43:02.035 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:02.035 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:02.035 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:02.035 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:02.035 INFO:teuthology.orchestra.run.smithi183.stdout: "replayed": 2656 2024-04-23T10:43:02.035 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:02.035 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_mem": { 2024-04-23T10:43:02.035 INFO:teuthology.orchestra.run.smithi183.stdout: "ino": 31, 2024-04-23T10:43:02.035 INFO:teuthology.orchestra.run.smithi183.stdout: "ino+": 32, 2024-04-23T10:43:02.035 INFO:teuthology.orchestra.run.smithi183.stdout: "ino-": 1, 2024-04-23T10:43:02.035 INFO:teuthology.orchestra.run.smithi183.stdout: "dir": 11, 2024-04-23T10:43:02.036 INFO:teuthology.orchestra.run.smithi183.stdout: "dir+": 11, 2024-04-23T10:43:02.036 INFO:teuthology.orchestra.run.smithi183.stdout: "dir-": 0, 2024-04-23T10:43:02.036 INFO:teuthology.orchestra.run.smithi183.stdout: "dn": 36, 2024-04-23T10:43:02.036 INFO:teuthology.orchestra.run.smithi183.stdout: "dn+": 36, 2024-04-23T10:43:02.036 INFO:teuthology.orchestra.run.smithi183.stdout: "dn-": 0, 2024-04-23T10:43:02.036 INFO:teuthology.orchestra.run.smithi183.stdout: "cap": 0, 2024-04-23T10:43:02.036 INFO:teuthology.orchestra.run.smithi183.stdout: "cap+": 0, 2024-04-23T10:43:02.036 INFO:teuthology.orchestra.run.smithi183.stdout: "cap-": 0, 2024-04-23T10:43:02.036 INFO:teuthology.orchestra.run.smithi183.stdout: "rss": 52048, 2024-04-23T10:43:02.036 INFO:teuthology.orchestra.run.smithi183.stdout: "heap": 215416 2024-04-23T10:43:02.036 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:02.037 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_server": { 2024-04-23T10:43:02.037 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_client_request": 0, 2024-04-23T10:43:02.037 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_server_request": 0, 2024-04-23T10:43:02.037 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_request": 0, 2024-04-23T10:43:02.037 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_session": 0, 2024-04-23T10:43:02.037 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_peer_request": 0, 2024-04-23T10:43:02.037 INFO:teuthology.orchestra.run.smithi183.stdout: "req_create_latency": { 2024-04-23T10:43:02.037 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:02.037 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:02.037 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:02.037 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:02.037 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getattr_latency": { 2024-04-23T10:43:02.038 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:02.038 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:02.038 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:02.038 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:02.038 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getfilelock_latency": { 2024-04-23T10:43:02.038 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:02.038 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:02.038 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:02.038 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:02.039 INFO:teuthology.orchestra.run.smithi183.stdout: "req_link_latency": { 2024-04-23T10:43:02.039 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:02.039 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:02.039 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:02.039 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:02.039 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookup_latency": { 2024-04-23T10:43:02.039 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:02.039 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:02.039 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:02.039 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:02.039 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookuphash_latency": { 2024-04-23T10:43:02.039 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:02.040 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:02.040 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:02.040 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:02.040 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupino_latency": { 2024-04-23T10:43:02.040 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:02.040 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:02.040 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:02.040 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:02.040 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupname_latency": { 2024-04-23T10:43:02.040 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:02.040 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:02.041 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:02.041 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:02.041 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupparent_latency": { 2024-04-23T10:43:02.041 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:02.041 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:02.041 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:02.041 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:02.041 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupsnap_latency": { 2024-04-23T10:43:02.041 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:02.041 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:02.041 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:02.041 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:02.042 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lssnap_latency": { 2024-04-23T10:43:02.042 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:02.042 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:02.042 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:02.042 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:02.042 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mkdir_latency": { 2024-04-23T10:43:02.042 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:02.042 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:02.042 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:02.042 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:02.043 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mknod_latency": { 2024-04-23T10:43:02.043 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:02.043 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:02.043 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:02.043 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:02.043 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mksnap_latency": { 2024-04-23T10:43:02.043 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:02.043 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:02.043 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:02.043 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:02.043 INFO:teuthology.orchestra.run.smithi183.stdout: "req_open_latency": { 2024-04-23T10:43:02.044 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:02.044 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:02.044 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:02.044 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:02.044 INFO:teuthology.orchestra.run.smithi183.stdout: "req_readdir_latency": { 2024-04-23T10:43:02.044 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:02.044 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:02.044 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:02.044 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:02.044 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rename_latency": { 2024-04-23T10:43:02.044 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:02.045 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:02.045 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:02.045 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:02.045 INFO:teuthology.orchestra.run.smithi183.stdout: "req_renamesnap_latency": { 2024-04-23T10:43:02.045 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:02.045 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:02.045 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:02.045 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:02.045 INFO:teuthology.orchestra.run.smithi183.stdout: "req_snapdiff_latency": { 2024-04-23T10:43:02.045 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:02.045 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:02.045 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:02.046 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:02.046 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmdir_latency": { 2024-04-23T10:43:02.046 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:02.046 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:02.046 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:02.046 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:02.046 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmsnap_latency": { 2024-04-23T10:43:02.046 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:02.046 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:02.047 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:02.047 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:02.047 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmxattr_latency": { 2024-04-23T10:43:02.047 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:02.047 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:02.047 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:02.047 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:02.047 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setattr_latency": { 2024-04-23T10:43:02.047 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:02.047 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:02.047 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:02.047 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:02.048 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setdirlayout_latency": { 2024-04-23T10:43:02.048 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:02.048 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:02.048 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:02.048 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:02.048 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setfilelock_latency": { 2024-04-23T10:43:02.048 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:02.048 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:02.048 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:02.048 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:02.048 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setlayout_latency": { 2024-04-23T10:43:02.048 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:02.049 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:02.049 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:02.049 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:02.049 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setxattr_latency": { 2024-04-23T10:43:02.049 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:02.049 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:02.049 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:02.049 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:02.049 INFO:teuthology.orchestra.run.smithi183.stdout: "req_symlink_latency": { 2024-04-23T10:43:02.049 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:02.049 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:02.049 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:02.050 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:02.050 INFO:teuthology.orchestra.run.smithi183.stdout: "req_unlink_latency": { 2024-04-23T10:43:02.050 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:02.050 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:02.050 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:02.050 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:02.050 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_revoke_eviction": 0, 2024-04-23T10:43:02.050 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_acquisition_throttle": 0, 2024-04-23T10:43:02.050 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getvxattr_latency": { 2024-04-23T10:43:02.050 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:02.050 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:02.051 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:02.051 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-23T10:43:02.051 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:02.051 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_sessions": { 2024-04-23T10:43:02.051 INFO:teuthology.orchestra.run.smithi183.stdout: "session_count": 2, 2024-04-23T10:43:02.051 INFO:teuthology.orchestra.run.smithi183.stdout: "session_add": 6, 2024-04-23T10:43:02.051 INFO:teuthology.orchestra.run.smithi183.stdout: "session_remove": 4, 2024-04-23T10:43:02.051 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_open": 5, 2024-04-23T10:43:02.051 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_stale": 0, 2024-04-23T10:43:02.051 INFO:teuthology.orchestra.run.smithi183.stdout: "total_load": 0, 2024-04-23T10:43:02.051 INFO:teuthology.orchestra.run.smithi183.stdout: "average_load": 0, 2024-04-23T10:43:02.052 INFO:teuthology.orchestra.run.smithi183.stdout: "avg_session_uptime": 5161, 2024-04-23T10:43:02.052 INFO:teuthology.orchestra.run.smithi183.stdout: "mdthresh_evicted": 0 2024-04-23T10:43:02.052 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:02.052 INFO:teuthology.orchestra.run.smithi183.stdout: "mempool": { 2024-04-23T10:43:02.052 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_bytes": 0, 2024-04-23T10:43:02.052 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_items": 0, 2024-04-23T10:43:02.052 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_bytes": 0, 2024-04-23T10:43:02.052 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_items": 0, 2024-04-23T10:43:02.052 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_bytes": 0, 2024-04-23T10:43:02.052 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_items": 0, 2024-04-23T10:43:02.052 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-23T10:43:02.052 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_items": 0, 2024-04-23T10:43:02.053 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-23T10:43:02.053 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_items": 0, 2024-04-23T10:43:02.053 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_bytes": 0, 2024-04-23T10:43:02.053 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_items": 0, 2024-04-23T10:43:02.053 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-23T10:43:02.053 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_items": 0, 2024-04-23T10:43:02.053 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_bytes": 0, 2024-04-23T10:43:02.053 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_items": 0, 2024-04-23T10:43:02.053 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_bytes": 0, 2024-04-23T10:43:02.053 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_items": 0, 2024-04-23T10:43:02.053 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-23T10:43:02.053 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_items": 0, 2024-04-23T10:43:02.054 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-23T10:43:02.054 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_items": 0, 2024-04-23T10:43:02.054 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_bytes": 0, 2024-04-23T10:43:02.054 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_items": 0, 2024-04-23T10:43:02.054 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_bytes": 0, 2024-04-23T10:43:02.054 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_items": 0, 2024-04-23T10:43:02.054 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-23T10:43:02.054 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_items": 0, 2024-04-23T10:43:02.054 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_bytes": 0, 2024-04-23T10:43:02.054 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_items": 0, 2024-04-23T10:43:02.054 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_bytes": 0, 2024-04-23T10:43:02.054 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_items": 0, 2024-04-23T10:43:02.055 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_bytes": 0, 2024-04-23T10:43:02.055 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_items": 0, 2024-04-23T10:43:02.055 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_bytes": 0, 2024-04-23T10:43:02.055 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_items": 0, 2024-04-23T10:43:02.055 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_bytes": 199347, 2024-04-23T10:43:02.055 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_items": 87, 2024-04-23T10:43:02.055 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_bytes": 0, 2024-04-23T10:43:02.055 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_items": 0, 2024-04-23T10:43:02.055 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_bytes": 0, 2024-04-23T10:43:02.055 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_items": 0, 2024-04-23T10:43:02.055 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_bytes": 0, 2024-04-23T10:43:02.055 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_items": 0, 2024-04-23T10:43:02.056 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_bytes": 0, 2024-04-23T10:43:02.056 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_items": 0, 2024-04-23T10:43:02.056 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_bytes": 7936, 2024-04-23T10:43:02.056 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_items": 165, 2024-04-23T10:43:02.056 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_bytes": 0, 2024-04-23T10:43:02.056 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_items": 0, 2024-04-23T10:43:02.056 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_bytes": 0, 2024-04-23T10:43:02.056 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_items": 0, 2024-04-23T10:43:02.056 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_bytes": 145417, 2024-04-23T10:43:02.056 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_items": 1129, 2024-04-23T10:43:02.056 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_bytes": 0, 2024-04-23T10:43:02.056 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_items": 0, 2024-04-23T10:43:02.057 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_bytes": 0, 2024-04-23T10:43:02.057 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_items": 0 2024-04-23T10:43:02.057 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:02.057 INFO:teuthology.orchestra.run.smithi183.stdout: "objecter": { 2024-04-23T10:43:02.057 INFO:teuthology.orchestra.run.smithi183.stdout: "op_active": 0, 2024-04-23T10:43:02.057 INFO:teuthology.orchestra.run.smithi183.stdout: "op_laggy": 0, 2024-04-23T10:43:02.057 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send": 4071, 2024-04-23T10:43:02.057 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send_bytes": 26, 2024-04-23T10:43:02.057 INFO:teuthology.orchestra.run.smithi183.stdout: "op_resend": 4, 2024-04-23T10:43:02.057 INFO:teuthology.orchestra.run.smithi183.stdout: "op_reply": 4071, 2024-04-23T10:43:02.057 INFO:teuthology.orchestra.run.smithi183.stdout: "op_latency": { 2024-04-23T10:43:02.057 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 4071, 2024-04-23T10:43:02.057 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 1.816950092, 2024-04-23T10:43:02.058 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000446315 2024-04-23T10:43:02.058 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:02.058 INFO:teuthology.orchestra.run.smithi183.stdout: "op_inflight": 0, 2024-04-23T10:43:02.058 INFO:teuthology.orchestra.run.smithi183.stdout: "oplen_avg": { 2024-04-23T10:43:02.058 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 4071, 2024-04-23T10:43:02.058 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 4072 2024-04-23T10:43:02.058 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:02.058 INFO:teuthology.orchestra.run.smithi183.stdout: "op": 4071, 2024-04-23T10:43:02.058 INFO:teuthology.orchestra.run.smithi183.stdout: "op_r": 4071, 2024-04-23T10:43:02.058 INFO:teuthology.orchestra.run.smithi183.stdout: "op_w": 0, 2024-04-23T10:43:02.058 INFO:teuthology.orchestra.run.smithi183.stdout: "op_rmw": 0, 2024-04-23T10:43:02.058 INFO:teuthology.orchestra.run.smithi183.stdout: "op_pg": 0, 2024-04-23T10:43:02.059 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_stat": 2237, 2024-04-23T10:43:02.059 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_create": 0, 2024-04-23T10:43:02.059 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_read": 1831, 2024-04-23T10:43:02.059 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_write": 0, 2024-04-23T10:43:02.059 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writefull": 0, 2024-04-23T10:43:02.059 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writesame": 0, 2024-04-23T10:43:02.059 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_append": 0, 2024-04-23T10:43:02.059 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_zero": 0, 2024-04-23T10:43:02.059 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_truncate": 0, 2024-04-23T10:43:02.059 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_delete": 0, 2024-04-23T10:43:02.059 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_mapext": 0, 2024-04-23T10:43:02.059 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_sparse_read": 0, 2024-04-23T10:43:02.060 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_clonerange": 0, 2024-04-23T10:43:02.060 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_getxattr": 2, 2024-04-23T10:43:02.060 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_setxattr": 0, 2024-04-23T10:43:02.060 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_cmpxattr": 0, 2024-04-23T10:43:02.060 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_rmxattr": 0, 2024-04-23T10:43:02.060 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_resetxattrs": 0, 2024-04-23T10:43:02.060 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_call": 0, 2024-04-23T10:43:02.060 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_watch": 0, 2024-04-23T10:43:02.060 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_notify": 0, 2024-04-23T10:43:02.060 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_src_cmpxattr": 0, 2024-04-23T10:43:02.060 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls": 0, 2024-04-23T10:43:02.060 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls_filter": 0, 2024-04-23T10:43:02.061 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_other": 0, 2024-04-23T10:43:02.061 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_active": 0, 2024-04-23T10:43:02.061 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_send": 0, 2024-04-23T10:43:02.061 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_resend": 0, 2024-04-23T10:43:02.061 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_ping": 0, 2024-04-23T10:43:02.061 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_active": 0, 2024-04-23T10:43:02.061 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_send": 0, 2024-04-23T10:43:02.061 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_resend": 0, 2024-04-23T10:43:02.061 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_active": 0, 2024-04-23T10:43:02.061 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_send": 0, 2024-04-23T10:43:02.061 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_resend": 0, 2024-04-23T10:43:02.061 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_active": 0, 2024-04-23T10:43:02.061 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_send": 0, 2024-04-23T10:43:02.062 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_resend": 0, 2024-04-23T10:43:02.062 INFO:teuthology.orchestra.run.smithi183.stdout: "command_active": 0, 2024-04-23T10:43:02.062 INFO:teuthology.orchestra.run.smithi183.stdout: "command_send": 0, 2024-04-23T10:43:02.062 INFO:teuthology.orchestra.run.smithi183.stdout: "command_resend": 0, 2024-04-23T10:43:02.062 INFO:teuthology.orchestra.run.smithi183.stdout: "map_epoch": 109, 2024-04-23T10:43:02.062 INFO:teuthology.orchestra.run.smithi183.stdout: "map_full": 0, 2024-04-23T10:43:02.062 INFO:teuthology.orchestra.run.smithi183.stdout: "map_inc": 23, 2024-04-23T10:43:02.062 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_sessions": 6, 2024-04-23T10:43:02.062 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_open": 6, 2024-04-23T10:43:02.062 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_close": 0, 2024-04-23T10:43:02.063 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_laggy": 0, 2024-04-23T10:43:02.063 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_wr": 0, 2024-04-23T10:43:02.063 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_rd": 2, 2024-04-23T10:43:02.063 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_del": 0 2024-04-23T10:43:02.063 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:02.063 INFO:teuthology.orchestra.run.smithi183.stdout: "oft": { 2024-04-23T10:43:02.063 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_objs": 0, 2024-04-23T10:43:02.063 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_kv_pairs": 0, 2024-04-23T10:43:02.063 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_updates": 0, 2024-04-23T10:43:02.063 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_removes": 0 2024-04-23T10:43:02.063 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:02.063 INFO:teuthology.orchestra.run.smithi183.stdout: "osd-slow-ops": { 2024-04-23T10:43:02.064 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_ops_count": 0 2024-04-23T10:43:02.064 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:02.064 INFO:teuthology.orchestra.run.smithi183.stdout: "purge_queue": { 2024-04-23T10:43:02.064 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops": 0, 2024-04-23T10:43:02.064 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops_high_water": 0, 2024-04-23T10:43:02.064 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing": 0, 2024-04-23T10:43:02.064 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_high_water": 0, 2024-04-23T10:43:02.064 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed_ops": 0, 2024-04-23T10:43:02.064 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed": 0, 2024-04-23T10:43:02.064 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_item_in_journal": 0 2024-04-23T10:43:02.064 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:02.064 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-23T10:43:02.065 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-23T10:43:02.065 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-04-23T10:43:02.065 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-23T10:43:02.065 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 4427, 2024-04-23T10:43:02.065 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 26700482, 2024-04-23T10:43:02.065 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-23T10:43:02.065 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 4427, 2024-04-23T10:43:02.065 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-04-23T10:43:02.065 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-04-23T10:43:02.065 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 4427, 2024-04-23T10:43:02.065 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 26700482, 2024-04-23T10:43:02.065 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-23T10:43:02.065 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:02.066 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:02.066 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:02.066 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-23T10:43:02.066 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:02.066 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_bytes": { 2024-04-23T10:43:02.066 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-23T10:43:02.066 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-04-23T10:43:02.066 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-23T10:43:02.066 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-04-23T10:43:02.066 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-04-23T10:43:02.066 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-23T10:43:02.066 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-04-23T10:43:02.067 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 4071, 2024-04-23T10:43:02.067 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 24313081, 2024-04-23T10:43:02.067 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 4071, 2024-04-23T10:43:02.067 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 24313081, 2024-04-23T10:43:02.067 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-23T10:43:02.067 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:02.067 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:02.067 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:02.068 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-23T10:43:02.068 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:02.068 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_ops": { 2024-04-23T10:43:02.068 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-23T10:43:02.068 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 1024, 2024-04-23T10:43:02.068 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-23T10:43:02.068 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-04-23T10:43:02.068 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-04-23T10:43:02.068 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-23T10:43:02.068 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-04-23T10:43:02.068 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 4071, 2024-04-23T10:43:02.068 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 4071, 2024-04-23T10:43:02.068 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 4071, 2024-04-23T10:43:02.069 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 4071, 2024-04-23T10:43:02.069 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-23T10:43:02.069 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:02.069 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:02.069 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:02.069 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-23T10:43:02.069 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:02.069 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle": { 2024-04-23T10:43:02.069 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-23T10:43:02.069 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-04-23T10:43:02.069 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-23T10:43:02.069 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-04-23T10:43:02.069 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-04-23T10:43:02.070 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-23T10:43:02.070 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-04-23T10:43:02.070 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-04-23T10:43:02.070 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-04-23T10:43:02.070 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 0, 2024-04-23T10:43:02.070 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 0, 2024-04-23T10:43:02.070 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-23T10:43:02.070 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:02.070 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:02.070 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:02.070 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-23T10:43:02.071 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:02.071 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle-0x559a5cbb06e0": { 2024-04-23T10:43:02.071 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-23T10:43:02.071 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-04-23T10:43:02.071 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-23T10:43:02.071 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-04-23T10:43:02.071 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-04-23T10:43:02.071 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-23T10:43:02.071 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-04-23T10:43:02.071 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-04-23T10:43:02.071 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-04-23T10:43:02.071 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 0, 2024-04-23T10:43:02.071 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 0, 2024-04-23T10:43:02.072 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-23T10:43:02.072 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:02.072 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:02.072 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:02.072 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-23T10:43:02.072 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-23T10:43:02.072 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-04-23T10:43:02.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:43:02 smithi196 ceph-mon[32919]: pgmap v528: 97 pgs: 97 active+clean; 4.4 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.3 KiB/s wr, 9 op/s 2024-04-23T10:43:02.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:43:02 smithi097 ceph-mon[29197]: pgmap v528: 97 pgs: 97 active+clean; 4.4 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.3 KiB/s wr, 9 op/s 2024-04-23T10:43:02.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:43:02 smithi183 ceph-mon[32317]: pgmap v528: 97 pgs: 97 active+clean; 4.4 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.3 KiB/s wr, 9 op/s 2024-04-23T10:43:02.868 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-04-23T10:43:02.868 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-04-23T10:43:02.868 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-04-23T10:43:02.868 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=0 2024-04-23T10:43:02.869 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-04-23T10:43:02.869 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-04-23T10:43:02.869 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-04-23T10:43:02.869 DEBUG:tasks.check_counter:Getting stats from k 2024-04-23T10:43:02.869 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-23T10:43:03.109 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.b/config 2024-04-23T10:43:04.527 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:43:04 smithi183 ceph-mon[32317]: pgmap v529: 97 pgs: 97 active+clean; 4.5 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 47 KiB/s wr, 30 op/s 2024-04-23T10:43:04.802 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-04-23T10:43:04.802 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-0": { 2024-04-23T10:43:04.802 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 2546, 2024-04-23T10:43:04.802 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 1903, 2024-04-23T10:43:04.803 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 2245936, 2024-04-23T10:43:04.803 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 546520, 2024-04-23T10:43:04.803 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 10, 2024-04-23T10:43:04.803 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 3, 2024-04-23T10:43:04.803 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.238913689, 2024-04-23T10:43:04.803 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.105806783, 2024-04-23T10:43:04.803 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.146430971, 2024-04-23T10:43:04.803 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.014159560, 2024-04-23T10:43:04.803 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T10:43:04.803 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1902, 2024-04-23T10:43:04.804 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.088003079, 2024-04-23T10:43:04.804 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000046268 2024-04-23T10:43:04.804 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:04.804 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-04-23T10:43:04.804 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 3521, 2024-04-23T10:43:04.804 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.001022813, 2024-04-23T10:43:04.804 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000290 2024-04-23T10:43:04.804 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:04.804 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 1672192, 2024-04-23T10:43:04.804 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 202880 2024-04-23T10:43:04.805 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:04.805 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-1": { 2024-04-23T10:43:04.805 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 32, 2024-04-23T10:43:04.805 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 67, 2024-04-23T10:43:04.805 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 10153, 2024-04-23T10:43:04.805 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 193016, 2024-04-23T10:43:04.805 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 17, 2024-04-23T10:43:04.805 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 5, 2024-04-23T10:43:04.805 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.010896348, 2024-04-23T10:43:04.805 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.002113311, 2024-04-23T10:43:04.806 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.007317291, 2024-04-23T10:43:04.806 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.000334190, 2024-04-23T10:43:04.806 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T10:43:04.806 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 65, 2024-04-23T10:43:04.806 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.015059523, 2024-04-23T10:43:04.806 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000231684 2024-04-23T10:43:04.806 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:04.806 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-04-23T10:43:04.806 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 32, 2024-04-23T10:43:04.806 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000004143, 2024-04-23T10:43:04.807 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000129 2024-04-23T10:43:04.807 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:04.807 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 7072, 2024-04-23T10:43:04.807 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 135696 2024-04-23T10:43:04.807 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:04.807 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-2": { 2024-04-23T10:43:04.807 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 23, 2024-04-23T10:43:04.807 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 228, 2024-04-23T10:43:04.807 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 5969, 2024-04-23T10:43:04.807 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 549196, 2024-04-23T10:43:04.807 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 9, 2024-04-23T10:43:04.808 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 4, 2024-04-23T10:43:04.808 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.013500512, 2024-04-23T10:43:04.808 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.008571853, 2024-04-23T10:43:04.808 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.002494612, 2024-04-23T10:43:04.808 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.000401906, 2024-04-23T10:43:04.808 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T10:43:04.808 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 226, 2024-04-23T10:43:04.808 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.030421913, 2024-04-23T10:43:04.808 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000134610 2024-04-23T10:43:04.808 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:04.809 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-04-23T10:43:04.809 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:04.809 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:04.809 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:04.809 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:04.809 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-23T10:43:04.809 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 539424 2024-04-23T10:43:04.809 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:04.809 INFO:teuthology.orchestra.run.smithi183.stdout: "cct": { 2024-04-23T10:43:04.809 INFO:teuthology.orchestra.run.smithi183.stdout: "total_workers": 1, 2024-04-23T10:43:04.810 INFO:teuthology.orchestra.run.smithi183.stdout: "unhealthy_workers": 0 2024-04-23T10:43:04.810 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:04.810 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-MDSRank": { 2024-04-23T10:43:04.810 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-04-23T10:43:04.810 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-04-23T10:43:04.810 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 50, 2024-04-23T10:43:04.810 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.001451735, 2024-04-23T10:43:04.810 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000029034 2024-04-23T10:43:04.810 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-23T10:43:04.810 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:04.811 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-PurgeQueue": { 2024-04-23T10:43:04.811 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-04-23T10:43:04.811 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-04-23T10:43:04.811 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 2, 2024-04-23T10:43:04.811 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000007803, 2024-04-23T10:43:04.811 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000003901 2024-04-23T10:43:04.811 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-23T10:43:04.811 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:04.811 INFO:teuthology.orchestra.run.smithi183.stdout: "mds": { 2024-04-23T10:43:04.811 INFO:teuthology.orchestra.run.smithi183.stdout: "request": 4, 2024-04-23T10:43:04.812 INFO:teuthology.orchestra.run.smithi183.stdout: "reply": 3, 2024-04-23T10:43:04.812 INFO:teuthology.orchestra.run.smithi183.stdout: "reply_latency": { 2024-04-23T10:43:04.812 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 3, 2024-04-23T10:43:04.812 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.002578139, 2024-04-23T10:43:04.812 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000859379 2024-04-23T10:43:04.812 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:04.812 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_reply": 0, 2024-04-23T10:43:04.812 INFO:teuthology.orchestra.run.smithi183.stdout: "forward": 1, 2024-04-23T10:43:04.812 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_complete": 0, 2024-04-23T10:43:04.812 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_keys": 0, 2024-04-23T10:43:04.813 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_commit": 11, 2024-04-23T10:43:04.813 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_split": 0, 2024-04-23T10:43:04.813 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_merge": 0, 2024-04-23T10:43:04.813 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes": 15, 2024-04-23T10:43:04.813 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_top": 0, 2024-04-23T10:43:04.813 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_bottom": 15, 2024-04-23T10:43:04.813 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pin_tail": 0, 2024-04-23T10:43:04.813 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pinned": 13, 2024-04-23T10:43:04.813 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_expired": 5, 2024-04-23T10:43:04.813 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_with_caps": 0, 2024-04-23T10:43:04.814 INFO:teuthology.orchestra.run.smithi183.stdout: "caps": 0, 2024-04-23T10:43:04.814 INFO:teuthology.orchestra.run.smithi183.stdout: "subtrees": 3, 2024-04-23T10:43:04.814 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse": 51, 2024-04-23T10:43:04.814 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_hit": 42, 2024-04-23T10:43:04.814 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_forward": 1, 2024-04-23T10:43:04.814 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_discover": 6, 2024-04-23T10:43:04.814 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_dir_fetch": 0, 2024-04-23T10:43:04.814 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_remote_ino": 0, 2024-04-23T10:43:04.814 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_lock": 0, 2024-04-23T10:43:04.814 INFO:teuthology.orchestra.run.smithi183.stdout: "load_cent": 0, 2024-04-23T10:43:04.814 INFO:teuthology.orchestra.run.smithi183.stdout: "q": 0, 2024-04-23T10:43:04.815 INFO:teuthology.orchestra.run.smithi183.stdout: "exported": 0, 2024-04-23T10:43:04.815 INFO:teuthology.orchestra.run.smithi183.stdout: "exported_inodes": 0, 2024-04-23T10:43:04.815 INFO:teuthology.orchestra.run.smithi183.stdout: "imported": 0, 2024-04-23T10:43:04.815 INFO:teuthology.orchestra.run.smithi183.stdout: "imported_inodes": 0, 2024-04-23T10:43:04.815 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_dir_fetch": 0, 2024-04-23T10:43:04.815 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_backtrace_fetch": 0, 2024-04-23T10:43:04.815 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_peer_discover": 0, 2024-04-23T10:43:04.815 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rfiles": 4, 2024-04-23T10:43:04.815 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rbytes": 234, 2024-04-23T10:43:04.815 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rsnaps": 6, 2024-04-23T10:43:04.815 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_fetch": 0, 2024-04-23T10:43:04.816 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_set_tag": 0, 2024-04-23T10:43:04.816 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_repaired": 0, 2024-04-23T10:43:04.816 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_inotable_repaired": 0, 2024-04-23T10:43:04.816 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_inodes": 0, 2024-04-23T10:43:04.816 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_base_inodes": 0, 2024-04-23T10:43:04.816 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dirfrag_rstats": 0, 2024-04-23T10:43:04.816 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_file_inodes": 0, 2024-04-23T10:43:04.816 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_inode_file_caps": 0, 2024-04-23T10:43:04.816 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_revoke": 2, 2024-04-23T10:43:04.816 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_grant": 0, 2024-04-23T10:43:04.816 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_trunc": 0, 2024-04-23T10:43:04.817 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-23T10:43:04.817 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-23T10:43:04.817 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps": 2, 2024-04-23T10:43:04.817 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps_dirty": 0, 2024-04-23T10:43:04.817 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_cap_release": 0, 2024-04-23T10:43:04.817 INFO:teuthology.orchestra.run.smithi183.stdout: "process_request_cap_release": 0 2024-04-23T10:43:04.817 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:04.817 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_cache": { 2024-04-23T10:43:04.817 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update": 0, 2024-04-23T10:43:04.817 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update_receipt": 13, 2024-04-23T10:43:04.817 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_try_discover": 9, 2024-04-23T10:43:04.818 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_send_discover": 10, 2024-04-23T10:43:04.818 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_handle_discover": 0, 2024-04-23T10:43:04.818 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays": 0, 2024-04-23T10:43:04.818 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_delayed": 0, 2024-04-23T10:43:04.818 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_enqueuing": 0, 2024-04-23T10:43:04.818 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_created": 0, 2024-04-23T10:43:04.818 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_enqueued": 0, 2024-04-23T10:43:04.818 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_reintegrated": 0, 2024-04-23T10:43:04.818 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_migrated": 0, 2024-04-23T10:43:04.818 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_processing": 0, 2024-04-23T10:43:04.819 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_enqueued": 0, 2024-04-23T10:43:04.819 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_prioritized": 0, 2024-04-23T10:43:04.819 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_started": 0, 2024-04-23T10:43:04.819 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_completed": 0, 2024-04-23T10:43:04.819 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_path": 17, 2024-04-23T10:43:04.819 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_inode": 84, 2024-04-23T10:43:04.819 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_enqueue_scrub": 0, 2024-04-23T10:43:04.819 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_exportdir": 0, 2024-04-23T10:43:04.819 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_flush": 0, 2024-04-23T10:43:04.819 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragmentdir": 0, 2024-04-23T10:43:04.819 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragstats": 0, 2024-04-23T10:43:04.819 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_inodestats": 0 2024-04-23T10:43:04.820 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:04.820 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_log": { 2024-04-23T10:43:04.820 INFO:teuthology.orchestra.run.smithi183.stdout: "evlrg": 0, 2024-04-23T10:43:04.820 INFO:teuthology.orchestra.run.smithi183.stdout: "evadd": 6, 2024-04-23T10:43:04.820 INFO:teuthology.orchestra.run.smithi183.stdout: "evex": 0, 2024-04-23T10:43:04.820 INFO:teuthology.orchestra.run.smithi183.stdout: "evtrm": 0, 2024-04-23T10:43:04.820 INFO:teuthology.orchestra.run.smithi183.stdout: "ev": 6, 2024-04-23T10:43:04.820 INFO:teuthology.orchestra.run.smithi183.stdout: "evexg": 0, 2024-04-23T10:43:04.820 INFO:teuthology.orchestra.run.smithi183.stdout: "evexd": 0, 2024-04-23T10:43:04.820 INFO:teuthology.orchestra.run.smithi183.stdout: "segadd": 2, 2024-04-23T10:43:04.821 INFO:teuthology.orchestra.run.smithi183.stdout: "segex": 0, 2024-04-23T10:43:04.821 INFO:teuthology.orchestra.run.smithi183.stdout: "segtrm": 0, 2024-04-23T10:43:04.821 INFO:teuthology.orchestra.run.smithi183.stdout: "seg": 2, 2024-04-23T10:43:04.821 INFO:teuthology.orchestra.run.smithi183.stdout: "segmjr": 2, 2024-04-23T10:43:04.821 INFO:teuthology.orchestra.run.smithi183.stdout: "segexg": 0, 2024-04-23T10:43:04.821 INFO:teuthology.orchestra.run.smithi183.stdout: "segexd": 0, 2024-04-23T10:43:04.821 INFO:teuthology.orchestra.run.smithi183.stdout: "expos": 4194304, 2024-04-23T10:43:04.821 INFO:teuthology.orchestra.run.smithi183.stdout: "wrpos": 4195956, 2024-04-23T10:43:04.821 INFO:teuthology.orchestra.run.smithi183.stdout: "rdpos": 0, 2024-04-23T10:43:04.821 INFO:teuthology.orchestra.run.smithi183.stdout: "jlat": { 2024-04-23T10:43:04.821 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 5, 2024-04-23T10:43:04.822 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.012210886, 2024-04-23T10:43:04.822 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.002442177 2024-04-23T10:43:04.822 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:04.822 INFO:teuthology.orchestra.run.smithi183.stdout: "replayed": 0 2024-04-23T10:43:04.822 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:04.822 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_mem": { 2024-04-23T10:43:04.822 INFO:teuthology.orchestra.run.smithi183.stdout: "ino": 19, 2024-04-23T10:43:04.822 INFO:teuthology.orchestra.run.smithi183.stdout: "ino+": 21, 2024-04-23T10:43:04.822 INFO:teuthology.orchestra.run.smithi183.stdout: "ino-": 2, 2024-04-23T10:43:04.822 INFO:teuthology.orchestra.run.smithi183.stdout: "dir": 18, 2024-04-23T10:43:04.822 INFO:teuthology.orchestra.run.smithi183.stdout: "dir+": 21, 2024-04-23T10:43:04.823 INFO:teuthology.orchestra.run.smithi183.stdout: "dir-": 3, 2024-04-23T10:43:04.823 INFO:teuthology.orchestra.run.smithi183.stdout: "dn": 15, 2024-04-23T10:43:04.823 INFO:teuthology.orchestra.run.smithi183.stdout: "dn+": 20, 2024-04-23T10:43:04.823 INFO:teuthology.orchestra.run.smithi183.stdout: "dn-": 5, 2024-04-23T10:43:04.823 INFO:teuthology.orchestra.run.smithi183.stdout: "cap": 0, 2024-04-23T10:43:04.823 INFO:teuthology.orchestra.run.smithi183.stdout: "cap+": 2, 2024-04-23T10:43:04.823 INFO:teuthology.orchestra.run.smithi183.stdout: "cap-": 2, 2024-04-23T10:43:04.823 INFO:teuthology.orchestra.run.smithi183.stdout: "rss": 53328, 2024-04-23T10:43:04.823 INFO:teuthology.orchestra.run.smithi183.stdout: "heap": 223608 2024-04-23T10:43:04.823 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:04.823 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_server": { 2024-04-23T10:43:04.824 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_client_request": 8, 2024-04-23T10:43:04.824 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_server_request": 0, 2024-04-23T10:43:04.824 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_request": 4, 2024-04-23T10:43:04.824 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_session": 8, 2024-04-23T10:43:04.824 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_peer_request": 85, 2024-04-23T10:43:04.824 INFO:teuthology.orchestra.run.smithi183.stdout: "req_create_latency": { 2024-04-23T10:43:04.824 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:04.824 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:04.824 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:04.824 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:04.824 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getattr_latency": { 2024-04-23T10:43:04.825 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1, 2024-04-23T10:43:04.825 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.001685000, 2024-04-23T10:43:04.825 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.001685000 2024-04-23T10:43:04.825 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:04.825 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getfilelock_latency": { 2024-04-23T10:43:04.825 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:04.825 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:04.825 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:04.825 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:04.825 INFO:teuthology.orchestra.run.smithi183.stdout: "req_link_latency": { 2024-04-23T10:43:04.825 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:04.826 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:04.826 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:04.826 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:04.826 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookup_latency": { 2024-04-23T10:43:04.826 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 2, 2024-04-23T10:43:04.826 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000893139, 2024-04-23T10:43:04.826 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000446569 2024-04-23T10:43:04.826 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:04.826 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookuphash_latency": { 2024-04-23T10:43:04.826 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:04.826 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:04.826 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:04.827 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:04.827 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupino_latency": { 2024-04-23T10:43:04.827 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:04.827 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:04.827 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:04.827 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:04.827 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupname_latency": { 2024-04-23T10:43:04.827 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:04.827 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:04.827 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:04.827 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:04.828 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupparent_latency": { 2024-04-23T10:43:04.828 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:04.828 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:04.828 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:04.828 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:04.828 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupsnap_latency": { 2024-04-23T10:43:04.828 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:04.828 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:04.828 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:04.828 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:04.828 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lssnap_latency": { 2024-04-23T10:43:04.828 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:04.829 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:04.829 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:04.829 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:04.829 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mkdir_latency": { 2024-04-23T10:43:04.829 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:04.830 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:04.830 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:04.830 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:04.830 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mknod_latency": { 2024-04-23T10:43:04.830 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:04.830 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:04.830 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:04.830 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:04.830 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mksnap_latency": { 2024-04-23T10:43:04.831 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:04.831 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:04.831 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:04.831 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:04.831 INFO:teuthology.orchestra.run.smithi183.stdout: "req_open_latency": { 2024-04-23T10:43:04.831 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:04.831 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:04.831 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:04.831 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:04.831 INFO:teuthology.orchestra.run.smithi183.stdout: "req_readdir_latency": { 2024-04-23T10:43:04.832 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:04.832 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:04.832 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:04.832 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:04.832 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rename_latency": { 2024-04-23T10:43:04.832 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:04.832 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:04.832 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:04.832 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:04.832 INFO:teuthology.orchestra.run.smithi183.stdout: "req_renamesnap_latency": { 2024-04-23T10:43:04.832 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:04.833 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:04.833 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:04.833 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:04.833 INFO:teuthology.orchestra.run.smithi183.stdout: "req_snapdiff_latency": { 2024-04-23T10:43:04.833 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:04.833 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:04.833 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:04.833 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:04.833 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmdir_latency": { 2024-04-23T10:43:04.833 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:04.833 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:04.833 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:04.834 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:04.834 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmsnap_latency": { 2024-04-23T10:43:04.834 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:04.834 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:04.834 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:04.834 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:04.834 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmxattr_latency": { 2024-04-23T10:43:04.834 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:04.834 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:04.834 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:04.835 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:04.835 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setattr_latency": { 2024-04-23T10:43:04.835 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:04.835 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:04.835 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:04.835 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:04.835 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setdirlayout_latency": { 2024-04-23T10:43:04.835 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:04.835 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:04.835 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:04.835 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:04.836 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setfilelock_latency": { 2024-04-23T10:43:04.836 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:04.836 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:04.836 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:04.836 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:04.836 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setlayout_latency": { 2024-04-23T10:43:04.836 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:04.836 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:04.836 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:04.836 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:04.836 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setxattr_latency": { 2024-04-23T10:43:04.836 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:04.837 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:04.837 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:04.837 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:04.837 INFO:teuthology.orchestra.run.smithi183.stdout: "req_symlink_latency": { 2024-04-23T10:43:04.837 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:04.837 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:04.837 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:04.837 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:04.837 INFO:teuthology.orchestra.run.smithi183.stdout: "req_unlink_latency": { 2024-04-23T10:43:04.837 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:04.837 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:04.837 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:04.838 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:04.838 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_revoke_eviction": 0, 2024-04-23T10:43:04.838 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_acquisition_throttle": 0, 2024-04-23T10:43:04.838 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getvxattr_latency": { 2024-04-23T10:43:04.838 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:04.838 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:04.838 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:04.838 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-23T10:43:04.838 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:04.838 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_sessions": { 2024-04-23T10:43:04.838 INFO:teuthology.orchestra.run.smithi183.stdout: "session_count": 0, 2024-04-23T10:43:04.839 INFO:teuthology.orchestra.run.smithi183.stdout: "session_add": 2, 2024-04-23T10:43:04.839 INFO:teuthology.orchestra.run.smithi183.stdout: "session_remove": 2, 2024-04-23T10:43:04.839 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_open": 0, 2024-04-23T10:43:04.839 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_stale": 0, 2024-04-23T10:43:04.839 INFO:teuthology.orchestra.run.smithi183.stdout: "total_load": 2, 2024-04-23T10:43:04.839 INFO:teuthology.orchestra.run.smithi183.stdout: "average_load": 1, 2024-04-23T10:43:04.839 INFO:teuthology.orchestra.run.smithi183.stdout: "avg_session_uptime": 86, 2024-04-23T10:43:04.839 INFO:teuthology.orchestra.run.smithi183.stdout: "mdthresh_evicted": 0 2024-04-23T10:43:04.839 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:04.839 INFO:teuthology.orchestra.run.smithi183.stdout: "mempool": { 2024-04-23T10:43:04.839 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_bytes": 0, 2024-04-23T10:43:04.839 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_items": 0, 2024-04-23T10:43:04.840 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_bytes": 0, 2024-04-23T10:43:04.840 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_items": 0, 2024-04-23T10:43:04.840 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_bytes": 0, 2024-04-23T10:43:04.840 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_items": 0, 2024-04-23T10:43:04.840 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-23T10:43:04.840 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_items": 0, 2024-04-23T10:43:04.840 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-23T10:43:04.840 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_items": 0, 2024-04-23T10:43:04.840 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_bytes": 0, 2024-04-23T10:43:04.840 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_items": 0, 2024-04-23T10:43:04.840 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-23T10:43:04.841 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_items": 0, 2024-04-23T10:43:04.841 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_bytes": 0, 2024-04-23T10:43:04.841 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_items": 0, 2024-04-23T10:43:04.841 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_bytes": 0, 2024-04-23T10:43:04.841 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_items": 0, 2024-04-23T10:43:04.841 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-23T10:43:04.841 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_items": 0, 2024-04-23T10:43:04.841 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-23T10:43:04.841 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_items": 0, 2024-04-23T10:43:04.841 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_bytes": 0, 2024-04-23T10:43:04.841 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_items": 0, 2024-04-23T10:43:04.841 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_bytes": 0, 2024-04-23T10:43:04.841 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_items": 0, 2024-04-23T10:43:04.842 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-23T10:43:04.842 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_items": 0, 2024-04-23T10:43:04.842 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_bytes": 0, 2024-04-23T10:43:04.842 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_items": 0, 2024-04-23T10:43:04.842 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_bytes": 0, 2024-04-23T10:43:04.842 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_items": 0, 2024-04-23T10:43:04.842 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_bytes": 0, 2024-04-23T10:43:04.842 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_items": 0, 2024-04-23T10:43:04.842 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_bytes": 0, 2024-04-23T10:43:04.842 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_items": 0, 2024-04-23T10:43:04.842 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_bytes": 254983, 2024-04-23T10:43:04.843 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_items": 81, 2024-04-23T10:43:04.843 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_bytes": 0, 2024-04-23T10:43:04.843 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_items": 0, 2024-04-23T10:43:04.843 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_bytes": 0, 2024-04-23T10:43:04.843 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_items": 0, 2024-04-23T10:43:04.843 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_bytes": 0, 2024-04-23T10:43:04.843 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_items": 0, 2024-04-23T10:43:04.843 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_bytes": 0, 2024-04-23T10:43:04.843 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_items": 0, 2024-04-23T10:43:04.843 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_bytes": 7936, 2024-04-23T10:43:04.843 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_items": 165, 2024-04-23T10:43:04.843 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_bytes": 0, 2024-04-23T10:43:04.844 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_items": 0, 2024-04-23T10:43:04.844 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_bytes": 0, 2024-04-23T10:43:04.844 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_items": 0, 2024-04-23T10:43:04.844 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_bytes": 77725, 2024-04-23T10:43:04.844 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_items": 330, 2024-04-23T10:43:04.844 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_bytes": 0, 2024-04-23T10:43:04.844 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_items": 0, 2024-04-23T10:43:04.844 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_bytes": 0, 2024-04-23T10:43:04.844 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_items": 0 2024-04-23T10:43:04.844 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:04.844 INFO:teuthology.orchestra.run.smithi183.stdout: "objecter": { 2024-04-23T10:43:04.844 INFO:teuthology.orchestra.run.smithi183.stdout: "op_active": 0, 2024-04-23T10:43:04.845 INFO:teuthology.orchestra.run.smithi183.stdout: "op_laggy": 0, 2024-04-23T10:43:04.845 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send": 43, 2024-04-23T10:43:04.845 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send_bytes": 12099, 2024-04-23T10:43:04.845 INFO:teuthology.orchestra.run.smithi183.stdout: "op_resend": 33, 2024-04-23T10:43:04.845 INFO:teuthology.orchestra.run.smithi183.stdout: "op_reply": 43, 2024-04-23T10:43:04.845 INFO:teuthology.orchestra.run.smithi183.stdout: "op_latency": { 2024-04-23T10:43:04.845 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 43, 2024-04-23T10:43:04.845 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.223993862, 2024-04-23T10:43:04.845 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.005209159 2024-04-23T10:43:04.845 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:04.845 INFO:teuthology.orchestra.run.smithi183.stdout: "op_inflight": 0, 2024-04-23T10:43:04.845 INFO:teuthology.orchestra.run.smithi183.stdout: "oplen_avg": { 2024-04-23T10:43:04.846 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 43, 2024-04-23T10:43:04.846 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 67 2024-04-23T10:43:04.846 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:04.846 INFO:teuthology.orchestra.run.smithi183.stdout: "op": 43, 2024-04-23T10:43:04.846 INFO:teuthology.orchestra.run.smithi183.stdout: "op_r": 0, 2024-04-23T10:43:04.846 INFO:teuthology.orchestra.run.smithi183.stdout: "op_w": 43, 2024-04-23T10:43:04.846 INFO:teuthology.orchestra.run.smithi183.stdout: "op_rmw": 0, 2024-04-23T10:43:04.846 INFO:teuthology.orchestra.run.smithi183.stdout: "op_pg": 0, 2024-04-23T10:43:04.846 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_stat": 0, 2024-04-23T10:43:04.846 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_create": 10, 2024-04-23T10:43:04.846 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_read": 0, 2024-04-23T10:43:04.846 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_write": 5, 2024-04-23T10:43:04.847 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writefull": 7, 2024-04-23T10:43:04.847 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writesame": 0, 2024-04-23T10:43:04.847 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_append": 0, 2024-04-23T10:43:04.847 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_zero": 0, 2024-04-23T10:43:04.847 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_truncate": 0, 2024-04-23T10:43:04.847 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_delete": 6, 2024-04-23T10:43:04.847 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_mapext": 0, 2024-04-23T10:43:04.847 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_sparse_read": 0, 2024-04-23T10:43:04.847 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_clonerange": 0, 2024-04-23T10:43:04.847 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_getxattr": 0, 2024-04-23T10:43:04.847 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_setxattr": 20, 2024-04-23T10:43:04.847 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_cmpxattr": 0, 2024-04-23T10:43:04.848 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_rmxattr": 0, 2024-04-23T10:43:04.848 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_resetxattrs": 0, 2024-04-23T10:43:04.848 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_call": 0, 2024-04-23T10:43:04.848 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_watch": 0, 2024-04-23T10:43:04.848 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_notify": 0, 2024-04-23T10:43:04.848 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_src_cmpxattr": 0, 2024-04-23T10:43:04.848 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls": 0, 2024-04-23T10:43:04.848 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls_filter": 0, 2024-04-23T10:43:04.848 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_other": 0, 2024-04-23T10:43:04.848 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_active": 0, 2024-04-23T10:43:04.848 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_send": 0, 2024-04-23T10:43:04.848 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_resend": 0, 2024-04-23T10:43:04.848 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_ping": 0, 2024-04-23T10:43:04.849 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_active": 0, 2024-04-23T10:43:04.849 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_send": 0, 2024-04-23T10:43:04.849 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_resend": 0, 2024-04-23T10:43:04.849 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_active": 0, 2024-04-23T10:43:04.849 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_send": 0, 2024-04-23T10:43:04.849 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_resend": 0, 2024-04-23T10:43:04.849 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_active": 0, 2024-04-23T10:43:04.849 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_send": 0, 2024-04-23T10:43:04.849 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_resend": 0, 2024-04-23T10:43:04.849 INFO:teuthology.orchestra.run.smithi183.stdout: "command_active": 0, 2024-04-23T10:43:04.849 INFO:teuthology.orchestra.run.smithi183.stdout: "command_send": 0, 2024-04-23T10:43:04.849 INFO:teuthology.orchestra.run.smithi183.stdout: "command_resend": 0, 2024-04-23T10:43:04.849 INFO:teuthology.orchestra.run.smithi183.stdout: "map_epoch": 109, 2024-04-23T10:43:04.850 INFO:teuthology.orchestra.run.smithi183.stdout: "map_full": 0, 2024-04-23T10:43:04.850 INFO:teuthology.orchestra.run.smithi183.stdout: "map_inc": 23, 2024-04-23T10:43:04.850 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_sessions": 9, 2024-04-23T10:43:04.850 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_open": 14, 2024-04-23T10:43:04.850 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_close": 5, 2024-04-23T10:43:04.850 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_laggy": 0, 2024-04-23T10:43:04.850 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_wr": 17, 2024-04-23T10:43:04.850 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_rd": 0, 2024-04-23T10:43:04.850 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_del": 2 2024-04-23T10:43:04.850 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:04.850 INFO:teuthology.orchestra.run.smithi183.stdout: "oft": { 2024-04-23T10:43:04.850 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_objs": 1, 2024-04-23T10:43:04.851 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_kv_pairs": 0, 2024-04-23T10:43:04.851 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_updates": 0, 2024-04-23T10:43:04.851 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_removes": 0 2024-04-23T10:43:04.851 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:04.851 INFO:teuthology.orchestra.run.smithi183.stdout: "osd-slow-ops": { 2024-04-23T10:43:04.851 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_ops_count": 101 2024-04-23T10:43:04.851 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:04.851 INFO:teuthology.orchestra.run.smithi183.stdout: "purge_queue": { 2024-04-23T10:43:04.851 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops": 0, 2024-04-23T10:43:04.851 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops_high_water": 0, 2024-04-23T10:43:04.851 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing": 0, 2024-04-23T10:43:04.851 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_high_water": 0, 2024-04-23T10:43:04.852 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed_ops": 0, 2024-04-23T10:43:04.852 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed": 0, 2024-04-23T10:43:04.852 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_item_in_journal": 0 2024-04-23T10:43:04.852 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:04.852 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-23T10:43:04.852 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-23T10:43:04.852 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-04-23T10:43:04.852 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-23T10:43:04.852 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 2597, 2024-04-23T10:43:04.852 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 2005064, 2024-04-23T10:43:04.852 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-23T10:43:04.852 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 2597, 2024-04-23T10:43:04.853 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-04-23T10:43:04.853 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-04-23T10:43:04.853 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 2597, 2024-04-23T10:43:04.853 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 2005064, 2024-04-23T10:43:04.853 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-23T10:43:04.853 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:04.853 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:04.853 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:04.853 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-23T10:43:04.853 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:04.853 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_bytes": { 2024-04-23T10:43:04.853 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-23T10:43:04.853 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-04-23T10:43:04.854 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-23T10:43:04.854 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-04-23T10:43:04.854 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-04-23T10:43:04.854 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-23T10:43:04.854 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-04-23T10:43:04.854 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 43, 2024-04-23T10:43:04.854 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 12099, 2024-04-23T10:43:04.854 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 43, 2024-04-23T10:43:04.854 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 12099, 2024-04-23T10:43:04.854 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-23T10:43:04.854 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:04.854 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:04.855 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:04.855 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-23T10:43:04.855 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:04.855 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_ops": { 2024-04-23T10:43:04.855 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-23T10:43:04.855 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 1024, 2024-04-23T10:43:04.855 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-23T10:43:04.855 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-04-23T10:43:04.855 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-04-23T10:43:04.855 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-23T10:43:04.855 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-04-23T10:43:04.855 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 43, 2024-04-23T10:43:04.856 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 43, 2024-04-23T10:43:04.856 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 43, 2024-04-23T10:43:04.856 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 43, 2024-04-23T10:43:04.856 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-23T10:43:04.856 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:04.856 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:04.856 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:04.856 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-23T10:43:04.856 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:04.856 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle": { 2024-04-23T10:43:04.856 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-23T10:43:04.856 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-04-23T10:43:04.856 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-23T10:43:04.857 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-04-23T10:43:04.857 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-04-23T10:43:04.857 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-23T10:43:04.857 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-04-23T10:43:04.857 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-04-23T10:43:04.857 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-04-23T10:43:04.857 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 0, 2024-04-23T10:43:04.857 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 0, 2024-04-23T10:43:04.857 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-23T10:43:04.857 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:04.857 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:04.857 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:04.857 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-23T10:43:04.858 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-04-23T10:43:04.858 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle-0x5636e568e500": { 2024-04-23T10:43:04.858 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-04-23T10:43:04.858 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-04-23T10:43:04.858 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-04-23T10:43:04.858 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 6, 2024-04-23T10:43:04.858 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 1652, 2024-04-23T10:43:04.858 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-04-23T10:43:04.858 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 6, 2024-04-23T10:43:04.858 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-04-23T10:43:04.858 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-04-23T10:43:04.858 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 5, 2024-04-23T10:43:04.858 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 1652, 2024-04-23T10:43:04.859 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-04-23T10:43:04.859 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-04-23T10:43:04.859 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-04-23T10:43:04.859 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-04-23T10:43:04.859 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-23T10:43:04.859 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-04-23T10:43:04.859 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-04-23T10:43:04.859 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:43:04 smithi196 ceph-mon[32919]: pgmap v529: 97 pgs: 97 active+clean; 4.5 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 47 KiB/s wr, 30 op/s 2024-04-23T10:43:04.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:43:04 smithi097 ceph-mon[29197]: pgmap v529: 97 pgs: 97 active+clean; 4.5 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 47 KiB/s wr, 30 op/s 2024-04-23T10:43:05.652 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-04-23T10:43:05.652 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-04-23T10:43:05.652 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-04-23T10:43:05.652 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=13 2024-04-23T10:43:05.652 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=6 2024-04-23T10:43:05.652 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-04-23T10:43:05.653 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-04-23T10:43:05.653 DEBUG:tasks.check_counter:Getting stats from c 2024-04-23T10:43:05.653 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-23T10:43:05.903 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T10:43:06.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:43:06 smithi196 ceph-mon[32919]: pgmap v530: 97 pgs: 97 active+clean; 4.5 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 46 KiB/s wr, 25 op/s 2024-04-23T10:43:06.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:43:06 smithi097 ceph-mon[29197]: pgmap v530: 97 pgs: 97 active+clean; 4.5 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 46 KiB/s wr, 25 op/s 2024-04-23T10:43:06.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:43:06 smithi183 ceph-mon[32317]: pgmap v530: 97 pgs: 97 active+clean; 4.5 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 46 KiB/s wr, 25 op/s 2024-04-23T10:43:07.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:43:07 smithi196 ceph-mon[32919]: osdmap e110: 12 total, 12 up, 12 in 2024-04-23T10:43:07.721 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-23T10:43:07.721 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-0": { 2024-04-23T10:43:07.721 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 2362, 2024-04-23T10:43:07.721 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 1605, 2024-04-23T10:43:07.721 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 632499, 2024-04-23T10:43:07.721 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 360329, 2024-04-23T10:43:07.721 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 8, 2024-04-23T10:43:07.722 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 1, 2024-04-23T10:43:07.722 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.166721413, 2024-04-23T10:43:07.722 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.066113227, 2024-04-23T10:43:07.722 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.108528786, 2024-04-23T10:43:07.722 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.008426649, 2024-04-23T10:43:07.722 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T10:43:07.722 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 1607, 2024-04-23T10:43:07.722 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.080642717, 2024-04-23T10:43:07.722 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000050182 2024-04-23T10:43:07.722 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:07.722 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-04-23T10:43:07.722 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 3754, 2024-04-23T10:43:07.722 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.001055869, 2024-04-23T10:43:07.722 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000281 2024-04-23T10:43:07.723 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:07.723 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-23T10:43:07.723 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 0 2024-04-23T10:43:07.723 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:07.723 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-1": { 2024-04-23T10:43:07.723 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 27, 2024-04-23T10:43:07.723 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 58, 2024-04-23T10:43:07.723 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 8846, 2024-04-23T10:43:07.723 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 144424, 2024-04-23T10:43:07.723 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 11, 2024-04-23T10:43:07.723 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 4, 2024-04-23T10:43:07.723 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.006928414, 2024-04-23T10:43:07.723 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.001592573, 2024-04-23T10:43:07.723 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.004296467, 2024-04-23T10:43:07.723 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.000332864, 2024-04-23T10:43:07.724 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T10:43:07.724 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 56, 2024-04-23T10:43:07.724 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.010752331, 2024-04-23T10:43:07.724 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000192005 2024-04-23T10:43:07.724 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:07.724 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-04-23T10:43:07.724 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 16, 2024-04-23T10:43:07.724 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000002396, 2024-04-23T10:43:07.724 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000149 2024-04-23T10:43:07.724 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:07.724 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 5056, 2024-04-23T10:43:07.724 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 132320 2024-04-23T10:43:07.724 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:07.724 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-2": { 2024-04-23T10:43:07.724 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 403, 2024-04-23T10:43:07.724 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 593, 2024-04-23T10:43:07.725 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 1692432, 2024-04-23T10:43:07.725 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 729069, 2024-04-23T10:43:07.725 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 15, 2024-04-23T10:43:07.725 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 5, 2024-04-23T10:43:07.725 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.078638239, 2024-04-23T10:43:07.725 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.029853617, 2024-04-23T10:43:07.725 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.046994936, 2024-04-23T10:43:07.725 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.008349597, 2024-04-23T10:43:07.725 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T10:43:07.725 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 588, 2024-04-23T10:43:07.725 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.035684028, 2024-04-23T10:43:07.725 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000060687 2024-04-23T10:43:07.725 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:07.725 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-04-23T10:43:07.725 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 13, 2024-04-23T10:43:07.725 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000001547, 2024-04-23T10:43:07.726 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000119 2024-04-23T10:43:07.726 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:07.726 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 1690176, 2024-04-23T10:43:07.726 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 655264 2024-04-23T10:43:07.726 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:07.726 INFO:teuthology.orchestra.run.smithi196.stdout: "cct": { 2024-04-23T10:43:07.726 INFO:teuthology.orchestra.run.smithi196.stdout: "total_workers": 1, 2024-04-23T10:43:07.726 INFO:teuthology.orchestra.run.smithi196.stdout: "unhealthy_workers": 0 2024-04-23T10:43:07.726 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:07.726 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-MDSRank": { 2024-04-23T10:43:07.726 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-04-23T10:43:07.726 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-04-23T10:43:07.726 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 57, 2024-04-23T10:43:07.726 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.001747296, 2024-04-23T10:43:07.727 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000030654 2024-04-23T10:43:07.727 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-23T10:43:07.727 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:07.727 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-PurgeQueue": { 2024-04-23T10:43:07.727 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-04-23T10:43:07.727 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-04-23T10:43:07.727 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 2, 2024-04-23T10:43:07.727 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000007588, 2024-04-23T10:43:07.727 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000003794 2024-04-23T10:43:07.727 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-23T10:43:07.727 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:07.727 INFO:teuthology.orchestra.run.smithi196.stdout: "mds": { 2024-04-23T10:43:07.727 INFO:teuthology.orchestra.run.smithi196.stdout: "request": 0, 2024-04-23T10:43:07.727 INFO:teuthology.orchestra.run.smithi196.stdout: "reply": 0, 2024-04-23T10:43:07.727 INFO:teuthology.orchestra.run.smithi196.stdout: "reply_latency": { 2024-04-23T10:43:07.727 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:07.728 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:07.728 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:07.728 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:07.728 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_reply": 0, 2024-04-23T10:43:07.728 INFO:teuthology.orchestra.run.smithi196.stdout: "forward": 0, 2024-04-23T10:43:07.728 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_complete": 0, 2024-04-23T10:43:07.728 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_keys": 0, 2024-04-23T10:43:07.728 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_commit": 11, 2024-04-23T10:43:07.728 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_split": 0, 2024-04-23T10:43:07.728 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_merge": 0, 2024-04-23T10:43:07.728 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes": 16, 2024-04-23T10:43:07.728 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_top": 0, 2024-04-23T10:43:07.728 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_bottom": 16, 2024-04-23T10:43:07.728 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pin_tail": 0, 2024-04-23T10:43:07.728 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pinned": 13, 2024-04-23T10:43:07.728 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_expired": 4, 2024-04-23T10:43:07.729 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_with_caps": 0, 2024-04-23T10:43:07.729 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": 0, 2024-04-23T10:43:07.729 INFO:teuthology.orchestra.run.smithi196.stdout: "subtrees": 3, 2024-04-23T10:43:07.729 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse": 48, 2024-04-23T10:43:07.729 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_hit": 39, 2024-04-23T10:43:07.729 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_forward": 0, 2024-04-23T10:43:07.729 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_discover": 7, 2024-04-23T10:43:07.729 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_dir_fetch": 0, 2024-04-23T10:43:07.729 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_remote_ino": 0, 2024-04-23T10:43:07.729 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_lock": 1, 2024-04-23T10:43:07.729 INFO:teuthology.orchestra.run.smithi196.stdout: "load_cent": 0, 2024-04-23T10:43:07.729 INFO:teuthology.orchestra.run.smithi196.stdout: "q": 0, 2024-04-23T10:43:07.729 INFO:teuthology.orchestra.run.smithi196.stdout: "exported": 0, 2024-04-23T10:43:07.729 INFO:teuthology.orchestra.run.smithi196.stdout: "exported_inodes": 0, 2024-04-23T10:43:07.729 INFO:teuthology.orchestra.run.smithi196.stdout: "imported": 0, 2024-04-23T10:43:07.729 INFO:teuthology.orchestra.run.smithi196.stdout: "imported_inodes": 0, 2024-04-23T10:43:07.730 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_dir_fetch": 0, 2024-04-23T10:43:07.730 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_backtrace_fetch": 0, 2024-04-23T10:43:07.730 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_peer_discover": 0, 2024-04-23T10:43:07.730 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rfiles": 4, 2024-04-23T10:43:07.730 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rbytes": 234, 2024-04-23T10:43:07.730 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rsnaps": 6, 2024-04-23T10:43:07.730 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_fetch": 0, 2024-04-23T10:43:07.730 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_set_tag": 0, 2024-04-23T10:43:07.730 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_repaired": 0, 2024-04-23T10:43:07.730 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_inotable_repaired": 0, 2024-04-23T10:43:07.730 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_inodes": 0, 2024-04-23T10:43:07.730 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_base_inodes": 0, 2024-04-23T10:43:07.730 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dirfrag_rstats": 0, 2024-04-23T10:43:07.730 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_file_inodes": 0, 2024-04-23T10:43:07.730 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_inode_file_caps": 0, 2024-04-23T10:43:07.731 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_revoke": 0, 2024-04-23T10:43:07.731 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_grant": 0, 2024-04-23T10:43:07.731 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_trunc": 0, 2024-04-23T10:43:07.731 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-23T10:43:07.731 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-23T10:43:07.731 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps": 0, 2024-04-23T10:43:07.731 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps_dirty": 0, 2024-04-23T10:43:07.731 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_cap_release": 0, 2024-04-23T10:43:07.731 INFO:teuthology.orchestra.run.smithi196.stdout: "process_request_cap_release": 0 2024-04-23T10:43:07.731 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:07.731 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_cache": { 2024-04-23T10:43:07.731 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update": 0, 2024-04-23T10:43:07.731 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update_receipt": 19, 2024-04-23T10:43:07.731 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_try_discover": 14, 2024-04-23T10:43:07.731 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_send_discover": 11, 2024-04-23T10:43:07.732 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_handle_discover": 0, 2024-04-23T10:43:07.732 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays": 0, 2024-04-23T10:43:07.732 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_delayed": 0, 2024-04-23T10:43:07.732 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_enqueuing": 0, 2024-04-23T10:43:07.732 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_created": 0, 2024-04-23T10:43:07.732 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_enqueued": 0, 2024-04-23T10:43:07.732 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_reintegrated": 0, 2024-04-23T10:43:07.732 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_migrated": 0, 2024-04-23T10:43:07.732 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_processing": 0, 2024-04-23T10:43:07.732 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_enqueued": 0, 2024-04-23T10:43:07.732 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_prioritized": 0, 2024-04-23T10:43:07.732 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_started": 0, 2024-04-23T10:43:07.732 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_completed": 0, 2024-04-23T10:43:07.732 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_path": 17, 2024-04-23T10:43:07.732 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_inode": 101, 2024-04-23T10:43:07.732 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_enqueue_scrub": 0, 2024-04-23T10:43:07.733 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_exportdir": 0, 2024-04-23T10:43:07.733 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_flush": 0, 2024-04-23T10:43:07.733 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragmentdir": 0, 2024-04-23T10:43:07.733 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragstats": 0, 2024-04-23T10:43:07.733 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_inodestats": 0 2024-04-23T10:43:07.733 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:07.733 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_log": { 2024-04-23T10:43:07.733 INFO:teuthology.orchestra.run.smithi196.stdout: "evlrg": 0, 2024-04-23T10:43:07.733 INFO:teuthology.orchestra.run.smithi196.stdout: "evadd": 8, 2024-04-23T10:43:07.733 INFO:teuthology.orchestra.run.smithi196.stdout: "evex": 0, 2024-04-23T10:43:07.733 INFO:teuthology.orchestra.run.smithi196.stdout: "evtrm": 0, 2024-04-23T10:43:07.733 INFO:teuthology.orchestra.run.smithi196.stdout: "ev": 8, 2024-04-23T10:43:07.733 INFO:teuthology.orchestra.run.smithi196.stdout: "evexg": 0, 2024-04-23T10:43:07.733 INFO:teuthology.orchestra.run.smithi196.stdout: "evexd": 0, 2024-04-23T10:43:07.733 INFO:teuthology.orchestra.run.smithi196.stdout: "segadd": 2, 2024-04-23T10:43:07.733 INFO:teuthology.orchestra.run.smithi196.stdout: "segex": 0, 2024-04-23T10:43:07.734 INFO:teuthology.orchestra.run.smithi196.stdout: "segtrm": 0, 2024-04-23T10:43:07.734 INFO:teuthology.orchestra.run.smithi196.stdout: "seg": 2, 2024-04-23T10:43:07.734 INFO:teuthology.orchestra.run.smithi196.stdout: "segmjr": 2, 2024-04-23T10:43:07.734 INFO:teuthology.orchestra.run.smithi196.stdout: "segexg": 0, 2024-04-23T10:43:07.734 INFO:teuthology.orchestra.run.smithi196.stdout: "segexd": 0, 2024-04-23T10:43:07.734 INFO:teuthology.orchestra.run.smithi196.stdout: "expos": 4194304, 2024-04-23T10:43:07.734 INFO:teuthology.orchestra.run.smithi196.stdout: "wrpos": 4196488, 2024-04-23T10:43:07.734 INFO:teuthology.orchestra.run.smithi196.stdout: "rdpos": 0, 2024-04-23T10:43:07.734 INFO:teuthology.orchestra.run.smithi196.stdout: "jlat": { 2024-04-23T10:43:07.734 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 7, 2024-04-23T10:43:07.734 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.014935153, 2024-04-23T10:43:07.734 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.002133593 2024-04-23T10:43:07.734 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:07.734 INFO:teuthology.orchestra.run.smithi196.stdout: "replayed": 0 2024-04-23T10:43:07.735 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:07.735 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_mem": { 2024-04-23T10:43:07.735 INFO:teuthology.orchestra.run.smithi196.stdout: "ino": 20, 2024-04-23T10:43:07.735 INFO:teuthology.orchestra.run.smithi196.stdout: "ino+": 22, 2024-04-23T10:43:07.735 INFO:teuthology.orchestra.run.smithi196.stdout: "ino-": 2, 2024-04-23T10:43:07.735 INFO:teuthology.orchestra.run.smithi196.stdout: "dir": 19, 2024-04-23T10:43:07.735 INFO:teuthology.orchestra.run.smithi196.stdout: "dir+": 21, 2024-04-23T10:43:07.735 INFO:teuthology.orchestra.run.smithi196.stdout: "dir-": 2, 2024-04-23T10:43:07.735 INFO:teuthology.orchestra.run.smithi196.stdout: "dn": 16, 2024-04-23T10:43:07.735 INFO:teuthology.orchestra.run.smithi196.stdout: "dn+": 20, 2024-04-23T10:43:07.735 INFO:teuthology.orchestra.run.smithi196.stdout: "dn-": 4, 2024-04-23T10:43:07.735 INFO:teuthology.orchestra.run.smithi196.stdout: "cap": 0, 2024-04-23T10:43:07.735 INFO:teuthology.orchestra.run.smithi196.stdout: "cap+": 0, 2024-04-23T10:43:07.735 INFO:teuthology.orchestra.run.smithi196.stdout: "cap-": 0, 2024-04-23T10:43:07.735 INFO:teuthology.orchestra.run.smithi196.stdout: "rss": 56136, 2024-04-23T10:43:07.735 INFO:teuthology.orchestra.run.smithi196.stdout: "heap": 223608 2024-04-23T10:43:07.736 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:07.736 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_server": { 2024-04-23T10:43:07.736 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_client_request": 0, 2024-04-23T10:43:07.736 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_server_request": 0, 2024-04-23T10:43:07.736 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_request": 0, 2024-04-23T10:43:07.736 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_session": 12, 2024-04-23T10:43:07.736 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_peer_request": 101, 2024-04-23T10:43:07.736 INFO:teuthology.orchestra.run.smithi196.stdout: "req_create_latency": { 2024-04-23T10:43:07.736 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:07.736 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:07.736 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:07.736 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:07.736 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getattr_latency": { 2024-04-23T10:43:07.736 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:07.736 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:07.736 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:07.737 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:07.737 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getfilelock_latency": { 2024-04-23T10:43:07.737 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:07.737 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:07.737 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:07.737 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:07.737 INFO:teuthology.orchestra.run.smithi196.stdout: "req_link_latency": { 2024-04-23T10:43:07.737 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:07.737 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:07.737 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:07.737 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:07.737 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookup_latency": { 2024-04-23T10:43:07.737 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:07.737 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:07.737 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:07.737 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:07.738 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookuphash_latency": { 2024-04-23T10:43:07.738 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:07.738 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:07.738 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:07.738 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:07.738 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupino_latency": { 2024-04-23T10:43:07.738 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:07.738 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:07.738 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:07.738 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:07.738 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupname_latency": { 2024-04-23T10:43:07.738 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:07.738 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:07.738 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:07.738 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:07.739 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupparent_latency": { 2024-04-23T10:43:07.739 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:07.739 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:07.739 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:07.739 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:07.739 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupsnap_latency": { 2024-04-23T10:43:07.739 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:07.739 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:07.739 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:07.739 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:07.739 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lssnap_latency": { 2024-04-23T10:43:07.739 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:07.739 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:07.739 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:07.739 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:07.739 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mkdir_latency": { 2024-04-23T10:43:07.739 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:07.740 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:07.740 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:07.740 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:07.740 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mknod_latency": { 2024-04-23T10:43:07.740 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:07.740 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:07.740 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:07.740 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:07.741 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mksnap_latency": { 2024-04-23T10:43:07.741 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:07.741 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:07.741 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:07.741 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:07.741 INFO:teuthology.orchestra.run.smithi196.stdout: "req_open_latency": { 2024-04-23T10:43:07.741 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:07.741 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:07.741 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:07.741 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:07.741 INFO:teuthology.orchestra.run.smithi196.stdout: "req_readdir_latency": { 2024-04-23T10:43:07.741 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:07.741 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:07.741 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:07.742 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:07.742 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rename_latency": { 2024-04-23T10:43:07.742 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:07.742 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:07.742 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:07.742 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:07.742 INFO:teuthology.orchestra.run.smithi196.stdout: "req_renamesnap_latency": { 2024-04-23T10:43:07.742 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:07.742 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:07.742 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:07.742 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:07.742 INFO:teuthology.orchestra.run.smithi196.stdout: "req_snapdiff_latency": { 2024-04-23T10:43:07.742 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:07.742 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:07.742 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:07.743 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:07.743 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmdir_latency": { 2024-04-23T10:43:07.743 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:07.743 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:07.743 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:07.743 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:07.743 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmsnap_latency": { 2024-04-23T10:43:07.743 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:07.743 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:07.743 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:07.743 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:07.743 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmxattr_latency": { 2024-04-23T10:43:07.743 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:07.743 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:07.743 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:07.744 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:07.744 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setattr_latency": { 2024-04-23T10:43:07.744 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:07.744 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:07.744 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:07.744 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:07.744 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setdirlayout_latency": { 2024-04-23T10:43:07.744 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:07.744 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:07.744 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:07.744 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:07.744 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setfilelock_latency": { 2024-04-23T10:43:07.744 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:07.744 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:07.744 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:07.745 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:07.745 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setlayout_latency": { 2024-04-23T10:43:07.745 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:07.745 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:07.745 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:07.745 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:07.745 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setxattr_latency": { 2024-04-23T10:43:07.745 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:07.745 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:07.745 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:07.745 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:07.745 INFO:teuthology.orchestra.run.smithi196.stdout: "req_symlink_latency": { 2024-04-23T10:43:07.745 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:07.745 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:07.745 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:07.745 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:07.745 INFO:teuthology.orchestra.run.smithi196.stdout: "req_unlink_latency": { 2024-04-23T10:43:07.746 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:07.746 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:07.746 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:07.746 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:07.746 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_revoke_eviction": 0, 2024-04-23T10:43:07.746 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_acquisition_throttle": 0, 2024-04-23T10:43:07.746 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getvxattr_latency": { 2024-04-23T10:43:07.746 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:07.746 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:07.746 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:07.746 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-23T10:43:07.746 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:07.746 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_sessions": { 2024-04-23T10:43:07.746 INFO:teuthology.orchestra.run.smithi196.stdout: "session_count": 0, 2024-04-23T10:43:07.747 INFO:teuthology.orchestra.run.smithi196.stdout: "session_add": 3, 2024-04-23T10:43:07.747 INFO:teuthology.orchestra.run.smithi196.stdout: "session_remove": 3, 2024-04-23T10:43:07.747 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_open": 0, 2024-04-23T10:43:07.747 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_stale": 0, 2024-04-23T10:43:07.747 INFO:teuthology.orchestra.run.smithi196.stdout: "total_load": 0, 2024-04-23T10:43:07.747 INFO:teuthology.orchestra.run.smithi196.stdout: "average_load": 0, 2024-04-23T10:43:07.747 INFO:teuthology.orchestra.run.smithi196.stdout: "avg_session_uptime": 84, 2024-04-23T10:43:07.747 INFO:teuthology.orchestra.run.smithi196.stdout: "mdthresh_evicted": 0 2024-04-23T10:43:07.747 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:07.747 INFO:teuthology.orchestra.run.smithi196.stdout: "mempool": { 2024-04-23T10:43:07.747 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_bytes": 0, 2024-04-23T10:43:07.747 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_items": 0, 2024-04-23T10:43:07.747 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_bytes": 0, 2024-04-23T10:43:07.747 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_items": 0, 2024-04-23T10:43:07.747 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_bytes": 0, 2024-04-23T10:43:07.747 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_items": 0, 2024-04-23T10:43:07.748 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-23T10:43:07.748 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_items": 0, 2024-04-23T10:43:07.748 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-23T10:43:07.748 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_items": 0, 2024-04-23T10:43:07.748 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_bytes": 0, 2024-04-23T10:43:07.748 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_items": 0, 2024-04-23T10:43:07.748 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-23T10:43:07.748 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_items": 0, 2024-04-23T10:43:07.748 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_bytes": 0, 2024-04-23T10:43:07.748 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_items": 0, 2024-04-23T10:43:07.748 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_bytes": 0, 2024-04-23T10:43:07.748 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_items": 0, 2024-04-23T10:43:07.748 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-23T10:43:07.748 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_items": 0, 2024-04-23T10:43:07.748 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-23T10:43:07.748 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_items": 0, 2024-04-23T10:43:07.749 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_bytes": 0, 2024-04-23T10:43:07.749 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_items": 0, 2024-04-23T10:43:07.749 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_bytes": 0, 2024-04-23T10:43:07.749 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_items": 0, 2024-04-23T10:43:07.749 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-23T10:43:07.749 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_items": 0, 2024-04-23T10:43:07.749 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_bytes": 0, 2024-04-23T10:43:07.749 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_items": 0, 2024-04-23T10:43:07.749 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_bytes": 0, 2024-04-23T10:43:07.749 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_items": 0, 2024-04-23T10:43:07.749 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_bytes": 0, 2024-04-23T10:43:07.749 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_items": 0, 2024-04-23T10:43:07.749 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_bytes": 0, 2024-04-23T10:43:07.749 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_items": 0, 2024-04-23T10:43:07.749 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_bytes": 230407, 2024-04-23T10:43:07.749 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_items": 67, 2024-04-23T10:43:07.750 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_bytes": 0, 2024-04-23T10:43:07.750 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_items": 0, 2024-04-23T10:43:07.750 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_bytes": 0, 2024-04-23T10:43:07.750 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_items": 0, 2024-04-23T10:43:07.750 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_bytes": 0, 2024-04-23T10:43:07.750 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_items": 0, 2024-04-23T10:43:07.750 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_bytes": 0, 2024-04-23T10:43:07.750 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_items": 0, 2024-04-23T10:43:07.750 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_bytes": 8056, 2024-04-23T10:43:07.750 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_items": 168, 2024-04-23T10:43:07.750 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_bytes": 0, 2024-04-23T10:43:07.750 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_items": 0, 2024-04-23T10:43:07.750 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_bytes": 0, 2024-04-23T10:43:07.750 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_items": 0, 2024-04-23T10:43:07.750 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_bytes": 81861, 2024-04-23T10:43:07.751 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_items": 336, 2024-04-23T10:43:07.751 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_bytes": 0, 2024-04-23T10:43:07.751 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_items": 0, 2024-04-23T10:43:07.751 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_bytes": 0, 2024-04-23T10:43:07.751 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_items": 0 2024-04-23T10:43:07.751 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:07.751 INFO:teuthology.orchestra.run.smithi196.stdout: "objecter": { 2024-04-23T10:43:07.751 INFO:teuthology.orchestra.run.smithi196.stdout: "op_active": 0, 2024-04-23T10:43:07.751 INFO:teuthology.orchestra.run.smithi196.stdout: "op_laggy": 0, 2024-04-23T10:43:07.751 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send": 44, 2024-04-23T10:43:07.751 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send_bytes": 12576, 2024-04-23T10:43:07.751 INFO:teuthology.orchestra.run.smithi196.stdout: "op_resend": 33, 2024-04-23T10:43:07.751 INFO:teuthology.orchestra.run.smithi196.stdout: "op_reply": 44, 2024-04-23T10:43:07.751 INFO:teuthology.orchestra.run.smithi196.stdout: "op_latency": { 2024-04-23T10:43:07.751 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 44, 2024-04-23T10:43:07.752 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.213995138, 2024-04-23T10:43:07.752 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.004863525 2024-04-23T10:43:07.752 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:07.752 INFO:teuthology.orchestra.run.smithi196.stdout: "op_inflight": 0, 2024-04-23T10:43:07.752 INFO:teuthology.orchestra.run.smithi196.stdout: "oplen_avg": { 2024-04-23T10:43:07.752 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 44, 2024-04-23T10:43:07.752 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 66 2024-04-23T10:43:07.752 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:07.752 INFO:teuthology.orchestra.run.smithi196.stdout: "op": 44, 2024-04-23T10:43:07.752 INFO:teuthology.orchestra.run.smithi196.stdout: "op_r": 0, 2024-04-23T10:43:07.752 INFO:teuthology.orchestra.run.smithi196.stdout: "op_w": 44, 2024-04-23T10:43:07.752 INFO:teuthology.orchestra.run.smithi196.stdout: "op_rmw": 0, 2024-04-23T10:43:07.752 INFO:teuthology.orchestra.run.smithi196.stdout: "op_pg": 0, 2024-04-23T10:43:07.752 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_stat": 0, 2024-04-23T10:43:07.752 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_create": 10, 2024-04-23T10:43:07.752 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_read": 0, 2024-04-23T10:43:07.753 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_write": 7, 2024-04-23T10:43:07.753 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writefull": 8, 2024-04-23T10:43:07.753 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writesame": 0, 2024-04-23T10:43:07.753 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_append": 0, 2024-04-23T10:43:07.753 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_zero": 0, 2024-04-23T10:43:07.753 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_truncate": 0, 2024-04-23T10:43:07.753 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_delete": 6, 2024-04-23T10:43:07.753 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_mapext": 0, 2024-04-23T10:43:07.753 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_sparse_read": 0, 2024-04-23T10:43:07.753 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_clonerange": 0, 2024-04-23T10:43:07.753 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_getxattr": 0, 2024-04-23T10:43:07.753 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_setxattr": 20, 2024-04-23T10:43:07.753 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_cmpxattr": 0, 2024-04-23T10:43:07.753 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_rmxattr": 0, 2024-04-23T10:43:07.753 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_resetxattrs": 0, 2024-04-23T10:43:07.753 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_call": 0, 2024-04-23T10:43:07.753 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_watch": 0, 2024-04-23T10:43:07.754 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_notify": 0, 2024-04-23T10:43:07.754 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_src_cmpxattr": 0, 2024-04-23T10:43:07.754 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls": 0, 2024-04-23T10:43:07.754 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls_filter": 0, 2024-04-23T10:43:07.754 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_other": 0, 2024-04-23T10:43:07.754 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_active": 0, 2024-04-23T10:43:07.754 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_send": 0, 2024-04-23T10:43:07.754 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_resend": 0, 2024-04-23T10:43:07.754 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_ping": 0, 2024-04-23T10:43:07.754 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_active": 0, 2024-04-23T10:43:07.754 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_send": 0, 2024-04-23T10:43:07.754 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_resend": 0, 2024-04-23T10:43:07.754 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_active": 0, 2024-04-23T10:43:07.754 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_send": 0, 2024-04-23T10:43:07.755 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_resend": 0, 2024-04-23T10:43:07.755 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_active": 0, 2024-04-23T10:43:07.755 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_send": 0, 2024-04-23T10:43:07.755 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_resend": 0, 2024-04-23T10:43:07.755 INFO:teuthology.orchestra.run.smithi196.stdout: "command_active": 0, 2024-04-23T10:43:07.755 INFO:teuthology.orchestra.run.smithi196.stdout: "command_send": 0, 2024-04-23T10:43:07.755 INFO:teuthology.orchestra.run.smithi196.stdout: "command_resend": 0, 2024-04-23T10:43:07.755 INFO:teuthology.orchestra.run.smithi196.stdout: "map_epoch": 110, 2024-04-23T10:43:07.755 INFO:teuthology.orchestra.run.smithi196.stdout: "map_full": 0, 2024-04-23T10:43:07.755 INFO:teuthology.orchestra.run.smithi196.stdout: "map_inc": 24, 2024-04-23T10:43:07.755 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_sessions": 8, 2024-04-23T10:43:07.755 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_open": 12, 2024-04-23T10:43:07.755 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_close": 4, 2024-04-23T10:43:07.755 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_laggy": 0, 2024-04-23T10:43:07.755 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_wr": 14, 2024-04-23T10:43:07.755 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_rd": 0, 2024-04-23T10:43:07.755 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_del": 1 2024-04-23T10:43:07.756 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:07.756 INFO:teuthology.orchestra.run.smithi196.stdout: "oft": { 2024-04-23T10:43:07.756 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_objs": 1, 2024-04-23T10:43:07.756 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_kv_pairs": 0, 2024-04-23T10:43:07.756 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_updates": 0, 2024-04-23T10:43:07.756 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_removes": 0 2024-04-23T10:43:07.756 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:07.756 INFO:teuthology.orchestra.run.smithi196.stdout: "osd-slow-ops": { 2024-04-23T10:43:07.756 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_ops_count": 118 2024-04-23T10:43:07.756 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:07.756 INFO:teuthology.orchestra.run.smithi196.stdout: "purge_queue": { 2024-04-23T10:43:07.756 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops": 0, 2024-04-23T10:43:07.756 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops_high_water": 0, 2024-04-23T10:43:07.756 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing": 0, 2024-04-23T10:43:07.756 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_high_water": 0, 2024-04-23T10:43:07.756 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed_ops": 0, 2024-04-23T10:43:07.757 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed": 0, 2024-04-23T10:43:07.757 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_item_in_journal": 0 2024-04-23T10:43:07.757 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:07.757 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-23T10:43:07.757 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-04-23T10:43:07.757 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-04-23T10:43:07.757 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-04-23T10:43:07.757 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 2788, 2024-04-23T10:43:07.757 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 2059649, 2024-04-23T10:43:07.757 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-04-23T10:43:07.757 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 2788, 2024-04-23T10:43:07.757 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-04-23T10:43:07.757 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-04-23T10:43:07.757 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 2788, 2024-04-23T10:43:07.757 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 2059649, 2024-04-23T10:43:07.757 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-04-23T10:43:07.758 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:07.758 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:07.758 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:07.758 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-23T10:43:07.758 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:07.758 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_bytes": { 2024-04-23T10:43:07.758 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-04-23T10:43:07.758 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-04-23T10:43:07.758 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-04-23T10:43:07.758 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-04-23T10:43:07.758 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-04-23T10:43:07.758 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-04-23T10:43:07.758 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-04-23T10:43:07.758 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 44, 2024-04-23T10:43:07.758 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 12576, 2024-04-23T10:43:07.759 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 44, 2024-04-23T10:43:07.759 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 12576, 2024-04-23T10:43:07.759 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-04-23T10:43:07.759 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:07.759 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:07.759 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:07.759 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-23T10:43:07.759 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:07.759 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_ops": { 2024-04-23T10:43:07.759 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-04-23T10:43:07.759 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 1024, 2024-04-23T10:43:07.759 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-04-23T10:43:07.759 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-04-23T10:43:07.759 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-04-23T10:43:07.759 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-04-23T10:43:07.759 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-04-23T10:43:07.760 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 44, 2024-04-23T10:43:07.760 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 44, 2024-04-23T10:43:07.760 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 44, 2024-04-23T10:43:07.760 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 44, 2024-04-23T10:43:07.760 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-04-23T10:43:07.760 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:07.760 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:07.760 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:07.760 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-23T10:43:07.760 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:07.760 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle": { 2024-04-23T10:43:07.760 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-04-23T10:43:07.760 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-04-23T10:43:07.760 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-04-23T10:43:07.760 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-04-23T10:43:07.761 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-04-23T10:43:07.761 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-04-23T10:43:07.761 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-04-23T10:43:07.761 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-04-23T10:43:07.761 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-04-23T10:43:07.761 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 0, 2024-04-23T10:43:07.761 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 0, 2024-04-23T10:43:07.761 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-04-23T10:43:07.761 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:07.761 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:07.761 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:07.761 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-23T10:43:07.761 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:07.761 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle-0x55bd6590e500": { 2024-04-23T10:43:07.761 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-04-23T10:43:07.762 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-04-23T10:43:07.762 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-04-23T10:43:07.762 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 8, 2024-04-23T10:43:07.762 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 2184, 2024-04-23T10:43:07.762 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-04-23T10:43:07.762 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 8, 2024-04-23T10:43:07.762 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-04-23T10:43:07.762 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-04-23T10:43:07.762 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 7, 2024-04-23T10:43:07.762 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 2184, 2024-04-23T10:43:07.762 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-04-23T10:43:07.762 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:07.762 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:07.762 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:07.762 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-23T10:43:07.762 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-23T10:43:07.763 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-23T10:43:07.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:43:07 smithi097 ceph-mon[29197]: osdmap e110: 12 total, 12 up, 12 in 2024-04-23T10:43:07.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:43:07 smithi183 ceph-mon[32317]: osdmap e110: 12 total, 12 up, 12 in 2024-04-23T10:43:08.601 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-04-23T10:43:08.601 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-04-23T10:43:08.601 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-04-23T10:43:08.601 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=19 2024-04-23T10:43:08.601 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-04-23T10:43:08.601 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-04-23T10:43:08.601 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-04-23T10:43:08.602 DEBUG:tasks.check_counter:Getting stats from f 2024-04-23T10:43:08.602 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-23T10:43:08.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:43:08 smithi196 ceph-mon[32919]: pgmap v532: 97 pgs: 97 active+clean; 4.5 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-23T10:43:08.851 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T10:43:08.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:43:08 smithi097 ceph-mon[29197]: pgmap v532: 97 pgs: 97 active+clean; 4.5 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-23T10:43:08.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:43:08 smithi183 ceph-mon[32317]: pgmap v532: 97 pgs: 97 active+clean; 4.5 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-23T10:43:10.556 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-23T10:43:10.557 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-0": { 2024-04-23T10:43:10.557 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 1499, 2024-04-23T10:43:10.557 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 1480, 2024-04-23T10:43:10.557 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 2067376, 2024-04-23T10:43:10.557 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 559308, 2024-04-23T10:43:10.557 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 5, 2024-04-23T10:43:10.557 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 1, 2024-04-23T10:43:10.558 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.208540037, 2024-04-23T10:43:10.558 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.065332097, 2024-04-23T10:43:10.558 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.080955263, 2024-04-23T10:43:10.558 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.056177655, 2024-04-23T10:43:10.558 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T10:43:10.558 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 1480, 2024-04-23T10:43:10.558 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.089481856, 2024-04-23T10:43:10.558 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000060460 2024-04-23T10:43:10.559 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:10.559 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-04-23T10:43:10.559 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:10.559 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:10.559 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:10.559 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:10.559 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 1685008, 2024-04-23T10:43:10.559 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 197424 2024-04-23T10:43:10.559 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:10.560 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-1": { 2024-04-23T10:43:10.560 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 5, 2024-04-23T10:43:10.560 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 240, 2024-04-23T10:43:10.560 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 1036, 2024-04-23T10:43:10.560 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 670218, 2024-04-23T10:43:10.560 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 8, 2024-04-23T10:43:10.560 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 4, 2024-04-23T10:43:10.560 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.014953347, 2024-04-23T10:43:10.560 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.009651693, 2024-04-23T10:43:10.561 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.002499536, 2024-04-23T10:43:10.561 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.000063434, 2024-04-23T10:43:10.561 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T10:43:10.561 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 237, 2024-04-23T10:43:10.561 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.014964173, 2024-04-23T10:43:10.561 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000063139 2024-04-23T10:43:10.561 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:10.561 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-04-23T10:43:10.561 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:10.562 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:10.562 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:10.562 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:10.562 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 480, 2024-04-23T10:43:10.562 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 669520 2024-04-23T10:43:10.562 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:10.562 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-2": { 2024-04-23T10:43:10.562 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 2206, 2024-04-23T10:43:10.563 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 2208, 2024-04-23T10:43:10.563 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 563489, 2024-04-23T10:43:10.563 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 701515, 2024-04-23T10:43:10.563 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 5, 2024-04-23T10:43:10.563 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 3, 2024-04-23T10:43:10.563 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.153240074, 2024-04-23T10:43:10.563 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.049831476, 2024-04-23T10:43:10.563 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.037049535, 2024-04-23T10:43:10.563 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.055081358, 2024-04-23T10:43:10.563 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T10:43:10.564 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 2206, 2024-04-23T10:43:10.564 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.085206065, 2024-04-23T10:43:10.564 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000038624 2024-04-23T10:43:10.564 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:10.564 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-04-23T10:43:10.564 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:10.564 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:10.564 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:10.564 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:10.565 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-23T10:43:10.565 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 0 2024-04-23T10:43:10.565 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:10.565 INFO:teuthology.orchestra.run.smithi196.stdout: "cct": { 2024-04-23T10:43:10.565 INFO:teuthology.orchestra.run.smithi196.stdout: "total_workers": 1, 2024-04-23T10:43:10.565 INFO:teuthology.orchestra.run.smithi196.stdout: "unhealthy_workers": 0 2024-04-23T10:43:10.565 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:10.565 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-MDSRank": { 2024-04-23T10:43:10.565 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-04-23T10:43:10.565 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-04-23T10:43:10.566 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 5624, 2024-04-23T10:43:10.566 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.164769575, 2024-04-23T10:43:10.566 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000029297 2024-04-23T10:43:10.566 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-23T10:43:10.566 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:10.566 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-PurgeQueue": { 2024-04-23T10:43:10.566 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-04-23T10:43:10.566 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-04-23T10:43:10.566 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:10.567 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:10.567 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:10.567 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-23T10:43:10.567 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:10.567 INFO:teuthology.orchestra.run.smithi196.stdout: "mds": { 2024-04-23T10:43:10.567 INFO:teuthology.orchestra.run.smithi196.stdout: "request": 0, 2024-04-23T10:43:10.567 INFO:teuthology.orchestra.run.smithi196.stdout: "reply": 0, 2024-04-23T10:43:10.567 INFO:teuthology.orchestra.run.smithi196.stdout: "reply_latency": { 2024-04-23T10:43:10.567 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:10.567 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:10.568 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:10.568 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:10.568 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_reply": 0, 2024-04-23T10:43:10.568 INFO:teuthology.orchestra.run.smithi196.stdout: "forward": 0, 2024-04-23T10:43:10.568 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_complete": 0, 2024-04-23T10:43:10.568 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_keys": 0, 2024-04-23T10:43:10.568 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_commit": 0, 2024-04-23T10:43:10.568 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_split": 0, 2024-04-23T10:43:10.568 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_merge": 0, 2024-04-23T10:43:10.568 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes": 0, 2024-04-23T10:43:10.569 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_top": 0, 2024-04-23T10:43:10.569 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_bottom": 0, 2024-04-23T10:43:10.569 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pin_tail": 0, 2024-04-23T10:43:10.569 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pinned": 0, 2024-04-23T10:43:10.569 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_expired": 0, 2024-04-23T10:43:10.569 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_with_caps": 0, 2024-04-23T10:43:10.569 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": 0, 2024-04-23T10:43:10.569 INFO:teuthology.orchestra.run.smithi196.stdout: "subtrees": 1, 2024-04-23T10:43:10.569 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse": 0, 2024-04-23T10:43:10.569 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_hit": 0, 2024-04-23T10:43:10.570 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_forward": 0, 2024-04-23T10:43:10.570 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_discover": 0, 2024-04-23T10:43:10.570 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_dir_fetch": 0, 2024-04-23T10:43:10.570 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_remote_ino": 0, 2024-04-23T10:43:10.570 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_lock": 0, 2024-04-23T10:43:10.570 INFO:teuthology.orchestra.run.smithi196.stdout: "load_cent": 0, 2024-04-23T10:43:10.570 INFO:teuthology.orchestra.run.smithi196.stdout: "q": 0, 2024-04-23T10:43:10.570 INFO:teuthology.orchestra.run.smithi196.stdout: "exported": 0, 2024-04-23T10:43:10.570 INFO:teuthology.orchestra.run.smithi196.stdout: "exported_inodes": 0, 2024-04-23T10:43:10.571 INFO:teuthology.orchestra.run.smithi196.stdout: "imported": 0, 2024-04-23T10:43:10.571 INFO:teuthology.orchestra.run.smithi196.stdout: "imported_inodes": 0, 2024-04-23T10:43:10.571 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_dir_fetch": 0, 2024-04-23T10:43:10.571 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_backtrace_fetch": 0, 2024-04-23T10:43:10.571 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_peer_discover": 0, 2024-04-23T10:43:10.571 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rfiles": 0, 2024-04-23T10:43:10.571 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rbytes": 0, 2024-04-23T10:43:10.571 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rsnaps": 0, 2024-04-23T10:43:10.571 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_fetch": 0, 2024-04-23T10:43:10.572 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_set_tag": 0, 2024-04-23T10:43:10.572 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_repaired": 0, 2024-04-23T10:43:10.572 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_inotable_repaired": 0, 2024-04-23T10:43:10.572 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_inodes": 0, 2024-04-23T10:43:10.572 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_base_inodes": 0, 2024-04-23T10:43:10.572 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dirfrag_rstats": 0, 2024-04-23T10:43:10.572 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_file_inodes": 0, 2024-04-23T10:43:10.572 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_inode_file_caps": 0, 2024-04-23T10:43:10.572 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_revoke": 0, 2024-04-23T10:43:10.572 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_grant": 0, 2024-04-23T10:43:10.573 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_trunc": 0, 2024-04-23T10:43:10.573 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-23T10:43:10.573 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-23T10:43:10.573 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps": 0, 2024-04-23T10:43:10.573 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps_dirty": 0, 2024-04-23T10:43:10.573 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_cap_release": 0, 2024-04-23T10:43:10.573 INFO:teuthology.orchestra.run.smithi196.stdout: "process_request_cap_release": 0 2024-04-23T10:43:10.573 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:10.573 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_cache": { 2024-04-23T10:43:10.573 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update": 0, 2024-04-23T10:43:10.574 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update_receipt": 0, 2024-04-23T10:43:10.574 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_try_discover": 0, 2024-04-23T10:43:10.574 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_send_discover": 0, 2024-04-23T10:43:10.574 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_handle_discover": 0, 2024-04-23T10:43:10.574 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays": 0, 2024-04-23T10:43:10.574 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_delayed": 0, 2024-04-23T10:43:10.574 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_enqueuing": 0, 2024-04-23T10:43:10.574 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_created": 0, 2024-04-23T10:43:10.574 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_enqueued": 0, 2024-04-23T10:43:10.574 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_reintegrated": 0, 2024-04-23T10:43:10.575 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_migrated": 0, 2024-04-23T10:43:10.575 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_processing": 0, 2024-04-23T10:43:10.575 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_enqueued": 0, 2024-04-23T10:43:10.575 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_prioritized": 0, 2024-04-23T10:43:10.575 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_started": 0, 2024-04-23T10:43:10.575 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_completed": 0, 2024-04-23T10:43:10.575 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_path": 0, 2024-04-23T10:43:10.575 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_inode": 0, 2024-04-23T10:43:10.575 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_enqueue_scrub": 0, 2024-04-23T10:43:10.575 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_exportdir": 0, 2024-04-23T10:43:10.576 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_flush": 0, 2024-04-23T10:43:10.576 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragmentdir": 0, 2024-04-23T10:43:10.576 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragstats": 0, 2024-04-23T10:43:10.576 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_inodestats": 0 2024-04-23T10:43:10.576 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:10.576 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_log": { 2024-04-23T10:43:10.576 INFO:teuthology.orchestra.run.smithi196.stdout: "evlrg": 0, 2024-04-23T10:43:10.576 INFO:teuthology.orchestra.run.smithi196.stdout: "evadd": 0, 2024-04-23T10:43:10.576 INFO:teuthology.orchestra.run.smithi196.stdout: "evex": 0, 2024-04-23T10:43:10.576 INFO:teuthology.orchestra.run.smithi196.stdout: "evtrm": 0, 2024-04-23T10:43:10.577 INFO:teuthology.orchestra.run.smithi196.stdout: "ev": 7, 2024-04-23T10:43:10.577 INFO:teuthology.orchestra.run.smithi196.stdout: "evexg": 0, 2024-04-23T10:43:10.577 INFO:teuthology.orchestra.run.smithi196.stdout: "evexd": 0, 2024-04-23T10:43:10.577 INFO:teuthology.orchestra.run.smithi196.stdout: "segadd": 0, 2024-04-23T10:43:10.577 INFO:teuthology.orchestra.run.smithi196.stdout: "segex": 0, 2024-04-23T10:43:10.577 INFO:teuthology.orchestra.run.smithi196.stdout: "segtrm": 0, 2024-04-23T10:43:10.577 INFO:teuthology.orchestra.run.smithi196.stdout: "seg": 2, 2024-04-23T10:43:10.577 INFO:teuthology.orchestra.run.smithi196.stdout: "segmjr": 2, 2024-04-23T10:43:10.577 INFO:teuthology.orchestra.run.smithi196.stdout: "segexg": 0, 2024-04-23T10:43:10.578 INFO:teuthology.orchestra.run.smithi196.stdout: "segexd": 0, 2024-04-23T10:43:10.578 INFO:teuthology.orchestra.run.smithi196.stdout: "expos": 4194304, 2024-04-23T10:43:10.578 INFO:teuthology.orchestra.run.smithi196.stdout: "wrpos": 4196264, 2024-04-23T10:43:10.578 INFO:teuthology.orchestra.run.smithi196.stdout: "rdpos": 4196123, 2024-04-23T10:43:10.578 INFO:teuthology.orchestra.run.smithi196.stdout: "jlat": { 2024-04-23T10:43:10.578 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:10.578 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:10.578 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:10.578 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:10.578 INFO:teuthology.orchestra.run.smithi196.stdout: "replayed": 7 2024-04-23T10:43:10.579 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:10.579 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_mem": { 2024-04-23T10:43:10.579 INFO:teuthology.orchestra.run.smithi196.stdout: "ino": 3, 2024-04-23T10:43:10.579 INFO:teuthology.orchestra.run.smithi196.stdout: "ino+": 3, 2024-04-23T10:43:10.579 INFO:teuthology.orchestra.run.smithi196.stdout: "ino-": 0, 2024-04-23T10:43:10.579 INFO:teuthology.orchestra.run.smithi196.stdout: "dir": 1, 2024-04-23T10:43:10.579 INFO:teuthology.orchestra.run.smithi196.stdout: "dir+": 1, 2024-04-23T10:43:10.579 INFO:teuthology.orchestra.run.smithi196.stdout: "dir-": 0, 2024-04-23T10:43:10.579 INFO:teuthology.orchestra.run.smithi196.stdout: "dn": 0, 2024-04-23T10:43:10.579 INFO:teuthology.orchestra.run.smithi196.stdout: "dn+": 0, 2024-04-23T10:43:10.580 INFO:teuthology.orchestra.run.smithi196.stdout: "dn-": 0, 2024-04-23T10:43:10.580 INFO:teuthology.orchestra.run.smithi196.stdout: "cap": 0, 2024-04-23T10:43:10.580 INFO:teuthology.orchestra.run.smithi196.stdout: "cap+": 0, 2024-04-23T10:43:10.580 INFO:teuthology.orchestra.run.smithi196.stdout: "cap-": 0, 2024-04-23T10:43:10.580 INFO:teuthology.orchestra.run.smithi196.stdout: "rss": 50944, 2024-04-23T10:43:10.580 INFO:teuthology.orchestra.run.smithi196.stdout: "heap": 215416 2024-04-23T10:43:10.580 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:10.580 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_server": { 2024-04-23T10:43:10.580 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_client_request": 0, 2024-04-23T10:43:10.580 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_server_request": 0, 2024-04-23T10:43:10.581 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_request": 0, 2024-04-23T10:43:10.581 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_session": 0, 2024-04-23T10:43:10.581 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_peer_request": 0, 2024-04-23T10:43:10.581 INFO:teuthology.orchestra.run.smithi196.stdout: "req_create_latency": { 2024-04-23T10:43:10.581 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:10.581 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:10.581 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:10.581 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:10.581 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getattr_latency": { 2024-04-23T10:43:10.581 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:10.582 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:10.582 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:10.582 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:10.582 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getfilelock_latency": { 2024-04-23T10:43:10.582 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:10.582 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:10.582 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:10.582 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:10.582 INFO:teuthology.orchestra.run.smithi196.stdout: "req_link_latency": { 2024-04-23T10:43:10.582 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:10.583 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:10.583 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:10.583 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:10.583 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookup_latency": { 2024-04-23T10:43:10.583 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:10.583 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:10.583 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:10.583 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:10.583 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookuphash_latency": { 2024-04-23T10:43:10.583 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:10.583 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:10.584 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:10.584 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:10.584 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupino_latency": { 2024-04-23T10:43:10.584 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:10.584 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:10.584 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:10.584 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:10.584 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupname_latency": { 2024-04-23T10:43:10.584 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:10.584 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:10.584 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:10.585 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:10.585 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupparent_latency": { 2024-04-23T10:43:10.585 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:10.585 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:10.585 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:10.585 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:10.585 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupsnap_latency": { 2024-04-23T10:43:10.585 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:10.586 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:10.586 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:10.586 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:10.586 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lssnap_latency": { 2024-04-23T10:43:10.586 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:10.586 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:10.586 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:10.586 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:10.586 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mkdir_latency": { 2024-04-23T10:43:10.587 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:10.587 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:10.587 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:10.587 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:10.587 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mknod_latency": { 2024-04-23T10:43:10.587 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:10.587 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:10.587 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:10.587 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:10.587 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mksnap_latency": { 2024-04-23T10:43:10.588 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:10.588 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:10.588 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:10.588 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:10.588 INFO:teuthology.orchestra.run.smithi196.stdout: "req_open_latency": { 2024-04-23T10:43:10.588 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:10.588 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:10.588 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:10.588 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:10.588 INFO:teuthology.orchestra.run.smithi196.stdout: "req_readdir_latency": { 2024-04-23T10:43:10.588 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:10.589 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:10.589 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:10.589 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:10.589 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rename_latency": { 2024-04-23T10:43:10.589 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:10.589 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:10.589 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:10.589 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:10.589 INFO:teuthology.orchestra.run.smithi196.stdout: "req_renamesnap_latency": { 2024-04-23T10:43:10.589 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:10.590 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:10.590 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:10.590 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:10.590 INFO:teuthology.orchestra.run.smithi196.stdout: "req_snapdiff_latency": { 2024-04-23T10:43:10.590 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:10.590 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:10.590 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:10.590 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:10.590 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmdir_latency": { 2024-04-23T10:43:10.590 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:10.591 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:10.591 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:10.591 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:10.591 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmsnap_latency": { 2024-04-23T10:43:10.591 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:10.591 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:10.591 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:10.591 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:10.591 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmxattr_latency": { 2024-04-23T10:43:10.591 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:10.592 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:10.592 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:10.592 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:10.592 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setattr_latency": { 2024-04-23T10:43:10.592 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:10.592 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:10.592 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:10.592 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:10.592 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setdirlayout_latency": { 2024-04-23T10:43:10.592 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:10.593 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:10.593 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:10.593 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:10.593 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setfilelock_latency": { 2024-04-23T10:43:10.593 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:10.593 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:10.593 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:10.593 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:10.593 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setlayout_latency": { 2024-04-23T10:43:10.593 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:10.594 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:10.594 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:10.594 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:10.594 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setxattr_latency": { 2024-04-23T10:43:10.594 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:10.594 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:10.594 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:10.594 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:10.594 INFO:teuthology.orchestra.run.smithi196.stdout: "req_symlink_latency": { 2024-04-23T10:43:10.594 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:10.594 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:10.595 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:10.595 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:10.595 INFO:teuthology.orchestra.run.smithi196.stdout: "req_unlink_latency": { 2024-04-23T10:43:10.595 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:10.595 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:10.595 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:10.595 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:10.595 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_revoke_eviction": 0, 2024-04-23T10:43:10.595 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_acquisition_throttle": 0, 2024-04-23T10:43:10.595 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getvxattr_latency": { 2024-04-23T10:43:10.595 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:10.596 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:10.596 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:10.596 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-23T10:43:10.596 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:10.596 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_sessions": { 2024-04-23T10:43:10.596 INFO:teuthology.orchestra.run.smithi196.stdout: "session_count": 1, 2024-04-23T10:43:10.596 INFO:teuthology.orchestra.run.smithi196.stdout: "session_add": 3, 2024-04-23T10:43:10.596 INFO:teuthology.orchestra.run.smithi196.stdout: "session_remove": 2, 2024-04-23T10:43:10.596 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_open": 2, 2024-04-23T10:43:10.596 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_stale": 0, 2024-04-23T10:43:10.596 INFO:teuthology.orchestra.run.smithi196.stdout: "total_load": 0, 2024-04-23T10:43:10.597 INFO:teuthology.orchestra.run.smithi196.stdout: "average_load": 0, 2024-04-23T10:43:10.597 INFO:teuthology.orchestra.run.smithi196.stdout: "avg_session_uptime": 4416, 2024-04-23T10:43:10.597 INFO:teuthology.orchestra.run.smithi196.stdout: "mdthresh_evicted": 0 2024-04-23T10:43:10.597 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:10.597 INFO:teuthology.orchestra.run.smithi196.stdout: "mempool": { 2024-04-23T10:43:10.597 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_bytes": 0, 2024-04-23T10:43:10.597 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_items": 0, 2024-04-23T10:43:10.597 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_bytes": 0, 2024-04-23T10:43:10.597 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_items": 0, 2024-04-23T10:43:10.597 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_bytes": 0, 2024-04-23T10:43:10.597 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_items": 0, 2024-04-23T10:43:10.598 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-23T10:43:10.598 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_items": 0, 2024-04-23T10:43:10.598 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-23T10:43:10.598 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_items": 0, 2024-04-23T10:43:10.598 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_bytes": 0, 2024-04-23T10:43:10.598 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_items": 0, 2024-04-23T10:43:10.598 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-23T10:43:10.598 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_items": 0, 2024-04-23T10:43:10.598 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_bytes": 0, 2024-04-23T10:43:10.598 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_items": 0, 2024-04-23T10:43:10.599 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_bytes": 0, 2024-04-23T10:43:10.599 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_items": 0, 2024-04-23T10:43:10.599 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-23T10:43:10.599 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_items": 0, 2024-04-23T10:43:10.599 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-23T10:43:10.599 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_items": 0, 2024-04-23T10:43:10.599 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_bytes": 0, 2024-04-23T10:43:10.599 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_items": 0, 2024-04-23T10:43:10.599 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_bytes": 0, 2024-04-23T10:43:10.599 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_items": 0, 2024-04-23T10:43:10.600 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-23T10:43:10.600 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_items": 0, 2024-04-23T10:43:10.600 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_bytes": 0, 2024-04-23T10:43:10.600 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_items": 0, 2024-04-23T10:43:10.600 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_bytes": 0, 2024-04-23T10:43:10.600 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_items": 0, 2024-04-23T10:43:10.600 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_bytes": 0, 2024-04-23T10:43:10.600 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_items": 0, 2024-04-23T10:43:10.600 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_bytes": 0, 2024-04-23T10:43:10.600 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_items": 0, 2024-04-23T10:43:10.600 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_bytes": 182087, 2024-04-23T10:43:10.601 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_items": 48, 2024-04-23T10:43:10.601 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_bytes": 0, 2024-04-23T10:43:10.601 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_items": 0, 2024-04-23T10:43:10.601 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_bytes": 0, 2024-04-23T10:43:10.601 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_items": 0, 2024-04-23T10:43:10.601 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_bytes": 0, 2024-04-23T10:43:10.601 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_items": 0, 2024-04-23T10:43:10.601 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_bytes": 0, 2024-04-23T10:43:10.601 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_items": 0, 2024-04-23T10:43:10.601 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_bytes": 8056, 2024-04-23T10:43:10.602 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_items": 168, 2024-04-23T10:43:10.602 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_bytes": 0, 2024-04-23T10:43:10.602 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_items": 0, 2024-04-23T10:43:10.602 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_bytes": 0, 2024-04-23T10:43:10.602 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_items": 0, 2024-04-23T10:43:10.602 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_bytes": 8512, 2024-04-23T10:43:10.602 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_items": 13, 2024-04-23T10:43:10.602 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_bytes": 0, 2024-04-23T10:43:10.602 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_items": 0, 2024-04-23T10:43:10.603 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_bytes": 0, 2024-04-23T10:43:10.603 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_items": 0 2024-04-23T10:43:10.603 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:10.603 INFO:teuthology.orchestra.run.smithi196.stdout: "objecter": { 2024-04-23T10:43:10.603 INFO:teuthology.orchestra.run.smithi196.stdout: "op_active": 0, 2024-04-23T10:43:10.603 INFO:teuthology.orchestra.run.smithi196.stdout: "op_laggy": 0, 2024-04-23T10:43:10.603 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send": 3346, 2024-04-23T10:43:10.603 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send_bytes": 21, 2024-04-23T10:43:10.603 INFO:teuthology.orchestra.run.smithi196.stdout: "op_resend": 3, 2024-04-23T10:43:10.603 INFO:teuthology.orchestra.run.smithi196.stdout: "op_reply": 3346, 2024-04-23T10:43:10.603 INFO:teuthology.orchestra.run.smithi196.stdout: "op_latency": { 2024-04-23T10:43:10.604 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 3346, 2024-04-23T10:43:10.604 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 1.620963180, 2024-04-23T10:43:10.604 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000484448 2024-04-23T10:43:10.604 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:10.604 INFO:teuthology.orchestra.run.smithi196.stdout: "op_inflight": 0, 2024-04-23T10:43:10.604 INFO:teuthology.orchestra.run.smithi196.stdout: "oplen_avg": { 2024-04-23T10:43:10.604 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 3346, 2024-04-23T10:43:10.604 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 3347 2024-04-23T10:43:10.604 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:10.604 INFO:teuthology.orchestra.run.smithi196.stdout: "op": 3346, 2024-04-23T10:43:10.604 INFO:teuthology.orchestra.run.smithi196.stdout: "op_r": 3346, 2024-04-23T10:43:10.605 INFO:teuthology.orchestra.run.smithi196.stdout: "op_w": 0, 2024-04-23T10:43:10.605 INFO:teuthology.orchestra.run.smithi196.stdout: "op_rmw": 0, 2024-04-23T10:43:10.605 INFO:teuthology.orchestra.run.smithi196.stdout: "op_pg": 0, 2024-04-23T10:43:10.605 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_stat": 2197, 2024-04-23T10:43:10.605 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_create": 0, 2024-04-23T10:43:10.605 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_read": 1147, 2024-04-23T10:43:10.605 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_write": 0, 2024-04-23T10:43:10.605 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writefull": 0, 2024-04-23T10:43:10.605 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writesame": 0, 2024-04-23T10:43:10.605 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_append": 0, 2024-04-23T10:43:10.605 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_zero": 0, 2024-04-23T10:43:10.606 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_truncate": 0, 2024-04-23T10:43:10.606 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_delete": 0, 2024-04-23T10:43:10.606 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_mapext": 0, 2024-04-23T10:43:10.606 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_sparse_read": 0, 2024-04-23T10:43:10.606 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_clonerange": 0, 2024-04-23T10:43:10.606 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_getxattr": 1, 2024-04-23T10:43:10.606 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_setxattr": 0, 2024-04-23T10:43:10.606 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_cmpxattr": 0, 2024-04-23T10:43:10.606 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_rmxattr": 0, 2024-04-23T10:43:10.606 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_resetxattrs": 0, 2024-04-23T10:43:10.606 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_call": 0, 2024-04-23T10:43:10.607 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_watch": 0, 2024-04-23T10:43:10.607 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_notify": 0, 2024-04-23T10:43:10.607 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_src_cmpxattr": 0, 2024-04-23T10:43:10.607 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls": 0, 2024-04-23T10:43:10.607 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls_filter": 0, 2024-04-23T10:43:10.607 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_other": 0, 2024-04-23T10:43:10.607 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_active": 0, 2024-04-23T10:43:10.607 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_send": 0, 2024-04-23T10:43:10.607 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_resend": 0, 2024-04-23T10:43:10.607 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_ping": 0, 2024-04-23T10:43:10.607 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_active": 0, 2024-04-23T10:43:10.608 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_send": 0, 2024-04-23T10:43:10.608 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_resend": 0, 2024-04-23T10:43:10.608 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_active": 0, 2024-04-23T10:43:10.608 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_send": 0, 2024-04-23T10:43:10.608 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_resend": 0, 2024-04-23T10:43:10.608 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_active": 0, 2024-04-23T10:43:10.608 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_send": 0, 2024-04-23T10:43:10.608 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_resend": 0, 2024-04-23T10:43:10.608 INFO:teuthology.orchestra.run.smithi196.stdout: "command_active": 0, 2024-04-23T10:43:10.608 INFO:teuthology.orchestra.run.smithi196.stdout: "command_send": 0, 2024-04-23T10:43:10.609 INFO:teuthology.orchestra.run.smithi196.stdout: "command_resend": 0, 2024-04-23T10:43:10.609 INFO:teuthology.orchestra.run.smithi196.stdout: "map_epoch": 110, 2024-04-23T10:43:10.609 INFO:teuthology.orchestra.run.smithi196.stdout: "map_full": 0, 2024-04-23T10:43:10.609 INFO:teuthology.orchestra.run.smithi196.stdout: "map_inc": 24, 2024-04-23T10:43:10.609 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_sessions": 4, 2024-04-23T10:43:10.609 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_open": 6, 2024-04-23T10:43:10.609 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_close": 2, 2024-04-23T10:43:10.609 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_laggy": 0, 2024-04-23T10:43:10.609 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_wr": 0, 2024-04-23T10:43:10.609 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_rd": 2, 2024-04-23T10:43:10.609 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_del": 0 2024-04-23T10:43:10.610 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:10.610 INFO:teuthology.orchestra.run.smithi196.stdout: "oft": { 2024-04-23T10:43:10.610 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_objs": 0, 2024-04-23T10:43:10.610 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_kv_pairs": 0, 2024-04-23T10:43:10.610 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_updates": 0, 2024-04-23T10:43:10.610 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_removes": 0 2024-04-23T10:43:10.610 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:10.610 INFO:teuthology.orchestra.run.smithi196.stdout: "osd-slow-ops": { 2024-04-23T10:43:10.610 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_ops_count": 0 2024-04-23T10:43:10.610 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:10.611 INFO:teuthology.orchestra.run.smithi196.stdout: "purge_queue": { 2024-04-23T10:43:10.611 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops": 0, 2024-04-23T10:43:10.611 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops_high_water": 0, 2024-04-23T10:43:10.611 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing": 0, 2024-04-23T10:43:10.611 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_high_water": 0, 2024-04-23T10:43:10.611 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed_ops": 0, 2024-04-23T10:43:10.611 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed": 0, 2024-04-23T10:43:10.611 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_item_in_journal": 0 2024-04-23T10:43:10.611 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:10.611 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-23T10:43:10.611 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-04-23T10:43:10.611 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-04-23T10:43:10.612 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-04-23T10:43:10.612 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 3706, 2024-04-23T10:43:10.612 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 2276998, 2024-04-23T10:43:10.612 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-04-23T10:43:10.612 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 3706, 2024-04-23T10:43:10.612 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-04-23T10:43:10.612 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-04-23T10:43:10.612 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 3706, 2024-04-23T10:43:10.612 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 2276998, 2024-04-23T10:43:10.612 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-04-23T10:43:10.613 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:10.613 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:10.613 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:10.613 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-23T10:43:10.613 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:10.613 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_bytes": { 2024-04-23T10:43:10.613 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-04-23T10:43:10.613 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-04-23T10:43:10.613 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-04-23T10:43:10.613 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-04-23T10:43:10.613 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-04-23T10:43:10.613 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-04-23T10:43:10.614 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-04-23T10:43:10.614 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 3346, 2024-04-23T10:43:10.614 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 1965, 2024-04-23T10:43:10.614 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 3346, 2024-04-23T10:43:10.614 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 1965, 2024-04-23T10:43:10.614 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-04-23T10:43:10.614 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:10.615 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:10.615 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:10.615 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-23T10:43:10.615 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:10.615 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_ops": { 2024-04-23T10:43:10.615 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-04-23T10:43:10.615 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 1024, 2024-04-23T10:43:10.616 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-04-23T10:43:10.616 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-04-23T10:43:10.616 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-04-23T10:43:10.616 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-04-23T10:43:10.616 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-04-23T10:43:10.616 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 3346, 2024-04-23T10:43:10.616 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 3346, 2024-04-23T10:43:10.616 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 3346, 2024-04-23T10:43:10.616 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 3346, 2024-04-23T10:43:10.616 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-04-23T10:43:10.616 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:10.616 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:10.617 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:10.617 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-23T10:43:10.617 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:10.617 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle": { 2024-04-23T10:43:10.617 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-04-23T10:43:10.617 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-04-23T10:43:10.617 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-04-23T10:43:10.617 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-04-23T10:43:10.617 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-04-23T10:43:10.617 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-04-23T10:43:10.617 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-04-23T10:43:10.617 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-04-23T10:43:10.618 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-04-23T10:43:10.618 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 0, 2024-04-23T10:43:10.618 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 0, 2024-04-23T10:43:10.618 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-04-23T10:43:10.618 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:10.618 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:10.618 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:10.618 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-23T10:43:10.618 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:10.618 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle-0x561a2f0786e0": { 2024-04-23T10:43:10.618 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-04-23T10:43:10.619 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-04-23T10:43:10.619 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-04-23T10:43:10.619 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-04-23T10:43:10.619 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-04-23T10:43:10.619 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-04-23T10:43:10.619 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-04-23T10:43:10.619 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-04-23T10:43:10.619 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-04-23T10:43:10.619 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 0, 2024-04-23T10:43:10.619 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 0, 2024-04-23T10:43:10.619 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-04-23T10:43:10.619 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:10.620 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:10.620 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:10.620 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-23T10:43:10.620 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-23T10:43:10.620 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-23T10:43:10.620 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:43:10 smithi196 ceph-mon[32919]: pgmap v533: 97 pgs: 97 active+clean; 4.5 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-23T10:43:10.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:43:10 smithi097 ceph-mon[29197]: pgmap v533: 97 pgs: 97 active+clean; 4.5 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-23T10:43:10.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:43:10 smithi183 ceph-mon[32317]: pgmap v533: 97 pgs: 97 active+clean; 4.5 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-23T10:43:11.450 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-04-23T10:43:11.454 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-04-23T10:43:11.454 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-04-23T10:43:11.454 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-04-23T10:43:11.454 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-04-23T10:43:11.454 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-04-23T10:43:11.454 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-04-23T10:43:11.454 DEBUG:tasks.check_counter:Getting stats from i 2024-04-23T10:43:11.455 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-23T10:43:11.703 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T10:43:12.557 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:43:12 smithi196 ceph-mon[32919]: osdmap e111: 12 total, 12 up, 12 in 2024-04-23T10:43:12.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:43:12 smithi097 ceph-mon[29197]: osdmap e111: 12 total, 12 up, 12 in 2024-04-23T10:43:12.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:43:12 smithi183 ceph-mon[32317]: osdmap e111: 12 total, 12 up, 12 in 2024-04-23T10:43:13.447 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:43:13 smithi196 ceph-mon[32919]: pgmap v535: 97 pgs: 97 active+clean; 4.5 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-23T10:43:13.448 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-23T10:43:13.448 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-0": { 2024-04-23T10:43:13.448 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 413, 2024-04-23T10:43:13.448 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 603, 2024-04-23T10:43:13.448 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 1699023, 2024-04-23T10:43:13.449 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 736731, 2024-04-23T10:43:13.449 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 17, 2024-04-23T10:43:13.449 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 4, 2024-04-23T10:43:13.449 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.080361513, 2024-04-23T10:43:13.449 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.030153078, 2024-04-23T10:43:13.449 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.045272650, 2024-04-23T10:43:13.449 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.008538617, 2024-04-23T10:43:13.449 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T10:43:13.449 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 597, 2024-04-23T10:43:13.449 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.043845523, 2024-04-23T10:43:13.449 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000073443 2024-04-23T10:43:13.449 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:13.449 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-04-23T10:43:13.449 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 13, 2024-04-23T10:43:13.449 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000001649, 2024-04-23T10:43:13.450 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000126 2024-04-23T10:43:13.450 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:13.450 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 1693808, 2024-04-23T10:43:13.450 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 656432 2024-04-23T10:43:13.450 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:13.450 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-1": { 2024-04-23T10:43:13.450 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 2332, 2024-04-23T10:43:13.450 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 1632, 2024-04-23T10:43:13.450 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 624771, 2024-04-23T10:43:13.450 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 634798, 2024-04-23T10:43:13.450 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 12, 2024-04-23T10:43:13.450 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 4, 2024-04-23T10:43:13.450 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.170675968, 2024-04-23T10:43:13.451 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.068247747, 2024-04-23T10:43:13.451 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.113386793, 2024-04-23T10:43:13.451 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.007592996, 2024-04-23T10:43:13.451 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T10:43:13.451 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 1635, 2024-04-23T10:43:13.451 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.077996249, 2024-04-23T10:43:13.451 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000047704 2024-04-23T10:43:13.451 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:13.451 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-04-23T10:43:13.451 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 3728, 2024-04-23T10:43:13.451 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.001131999, 2024-04-23T10:43:13.451 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000303 2024-04-23T10:43:13.451 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:13.451 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 4128, 2024-04-23T10:43:13.451 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 131168 2024-04-23T10:43:13.452 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:13.452 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-2": { 2024-04-23T10:43:13.452 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 33, 2024-04-23T10:43:13.452 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 91, 2024-04-23T10:43:13.452 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 7727, 2024-04-23T10:43:13.452 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 30818, 2024-04-23T10:43:13.452 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 7, 2024-04-23T10:43:13.452 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 4, 2024-04-23T10:43:13.452 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.017931485, 2024-04-23T10:43:13.452 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.005553935, 2024-04-23T10:43:13.452 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.010667834, 2024-04-23T10:43:13.452 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.000472551, 2024-04-23T10:43:13.453 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T10:43:13.453 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 89, 2024-04-23T10:43:13.453 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.017466873, 2024-04-23T10:43:13.453 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000196257 2024-04-23T10:43:13.453 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:13.453 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-04-23T10:43:13.453 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 83, 2024-04-23T10:43:13.453 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000031556, 2024-04-23T10:43:13.453 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000380 2024-04-23T10:43:13.453 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:13.453 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-04-23T10:43:13.453 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 17376 2024-04-23T10:43:13.453 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:13.454 INFO:teuthology.orchestra.run.smithi196.stdout: "cct": { 2024-04-23T10:43:13.454 INFO:teuthology.orchestra.run.smithi196.stdout: "total_workers": 1, 2024-04-23T10:43:13.454 INFO:teuthology.orchestra.run.smithi196.stdout: "unhealthy_workers": 0 2024-04-23T10:43:13.454 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:13.454 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-MDSRank": { 2024-04-23T10:43:13.454 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-04-23T10:43:13.454 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-04-23T10:43:13.454 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 47, 2024-04-23T10:43:13.454 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.002359585, 2024-04-23T10:43:13.454 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000050203 2024-04-23T10:43:13.454 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-23T10:43:13.454 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:13.454 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-PurgeQueue": { 2024-04-23T10:43:13.454 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-04-23T10:43:13.454 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-04-23T10:43:13.455 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 2, 2024-04-23T10:43:13.455 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000009042, 2024-04-23T10:43:13.455 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000004521 2024-04-23T10:43:13.455 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-23T10:43:13.455 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:13.455 INFO:teuthology.orchestra.run.smithi196.stdout: "mds": { 2024-04-23T10:43:13.455 INFO:teuthology.orchestra.run.smithi196.stdout: "request": 1, 2024-04-23T10:43:13.455 INFO:teuthology.orchestra.run.smithi196.stdout: "reply": 1, 2024-04-23T10:43:13.455 INFO:teuthology.orchestra.run.smithi196.stdout: "reply_latency": { 2024-04-23T10:43:13.455 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 1, 2024-04-23T10:43:13.455 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000125502, 2024-04-23T10:43:13.455 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000125502 2024-04-23T10:43:13.455 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:13.455 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_reply": 0, 2024-04-23T10:43:13.455 INFO:teuthology.orchestra.run.smithi196.stdout: "forward": 0, 2024-04-23T10:43:13.456 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_complete": 0, 2024-04-23T10:43:13.456 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_keys": 0, 2024-04-23T10:43:13.456 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_commit": 11, 2024-04-23T10:43:13.456 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_split": 0, 2024-04-23T10:43:13.456 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_merge": 0, 2024-04-23T10:43:13.456 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes": 16, 2024-04-23T10:43:13.456 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_top": 0, 2024-04-23T10:43:13.456 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_bottom": 16, 2024-04-23T10:43:13.456 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pin_tail": 0, 2024-04-23T10:43:13.456 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pinned": 13, 2024-04-23T10:43:13.456 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_expired": 4, 2024-04-23T10:43:13.456 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_with_caps": 1, 2024-04-23T10:43:13.456 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": 1, 2024-04-23T10:43:13.456 INFO:teuthology.orchestra.run.smithi196.stdout: "subtrees": 3, 2024-04-23T10:43:13.456 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse": 48, 2024-04-23T10:43:13.457 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_hit": 39, 2024-04-23T10:43:13.457 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_forward": 0, 2024-04-23T10:43:13.457 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_discover": 7, 2024-04-23T10:43:13.457 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_dir_fetch": 0, 2024-04-23T10:43:13.457 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_remote_ino": 0, 2024-04-23T10:43:13.457 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_lock": 1, 2024-04-23T10:43:13.457 INFO:teuthology.orchestra.run.smithi196.stdout: "load_cent": 0, 2024-04-23T10:43:13.457 INFO:teuthology.orchestra.run.smithi196.stdout: "q": 0, 2024-04-23T10:43:13.457 INFO:teuthology.orchestra.run.smithi196.stdout: "exported": 0, 2024-04-23T10:43:13.457 INFO:teuthology.orchestra.run.smithi196.stdout: "exported_inodes": 0, 2024-04-23T10:43:13.457 INFO:teuthology.orchestra.run.smithi196.stdout: "imported": 0, 2024-04-23T10:43:13.457 INFO:teuthology.orchestra.run.smithi196.stdout: "imported_inodes": 0, 2024-04-23T10:43:13.457 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_dir_fetch": 0, 2024-04-23T10:43:13.457 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_backtrace_fetch": 0, 2024-04-23T10:43:13.457 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_peer_discover": 0, 2024-04-23T10:43:13.457 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rfiles": 4, 2024-04-23T10:43:13.458 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rbytes": 234, 2024-04-23T10:43:13.458 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rsnaps": 6, 2024-04-23T10:43:13.458 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_fetch": 0, 2024-04-23T10:43:13.458 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_set_tag": 0, 2024-04-23T10:43:13.458 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_repaired": 0, 2024-04-23T10:43:13.458 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_inotable_repaired": 0, 2024-04-23T10:43:13.458 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_inodes": 0, 2024-04-23T10:43:13.458 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_base_inodes": 0, 2024-04-23T10:43:13.458 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dirfrag_rstats": 0, 2024-04-23T10:43:13.458 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_file_inodes": 0, 2024-04-23T10:43:13.458 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_inode_file_caps": 0, 2024-04-23T10:43:13.458 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_revoke": 1, 2024-04-23T10:43:13.458 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_grant": 0, 2024-04-23T10:43:13.458 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_trunc": 0, 2024-04-23T10:43:13.458 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-23T10:43:13.459 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-23T10:43:13.459 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps": 1, 2024-04-23T10:43:13.459 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps_dirty": 0, 2024-04-23T10:43:13.459 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_cap_release": 0, 2024-04-23T10:43:13.459 INFO:teuthology.orchestra.run.smithi196.stdout: "process_request_cap_release": 0 2024-04-23T10:43:13.459 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:13.459 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_cache": { 2024-04-23T10:43:13.459 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update": 0, 2024-04-23T10:43:13.459 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update_receipt": 19, 2024-04-23T10:43:13.459 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_try_discover": 14, 2024-04-23T10:43:13.459 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_send_discover": 11, 2024-04-23T10:43:13.459 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_handle_discover": 0, 2024-04-23T10:43:13.459 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays": 0, 2024-04-23T10:43:13.459 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_delayed": 0, 2024-04-23T10:43:13.459 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_enqueuing": 0, 2024-04-23T10:43:13.459 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_created": 0, 2024-04-23T10:43:13.460 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_enqueued": 0, 2024-04-23T10:43:13.460 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_reintegrated": 0, 2024-04-23T10:43:13.460 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_migrated": 0, 2024-04-23T10:43:13.460 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_processing": 0, 2024-04-23T10:43:13.460 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_enqueued": 0, 2024-04-23T10:43:13.460 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_prioritized": 0, 2024-04-23T10:43:13.460 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_started": 0, 2024-04-23T10:43:13.460 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_completed": 0, 2024-04-23T10:43:13.460 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_path": 17, 2024-04-23T10:43:13.460 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_inode": 101, 2024-04-23T10:43:13.460 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_enqueue_scrub": 0, 2024-04-23T10:43:13.460 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_exportdir": 0, 2024-04-23T10:43:13.460 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_flush": 0, 2024-04-23T10:43:13.460 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragmentdir": 0, 2024-04-23T10:43:13.460 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragstats": 0, 2024-04-23T10:43:13.461 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_inodestats": 0 2024-04-23T10:43:13.461 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:13.461 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_log": { 2024-04-23T10:43:13.461 INFO:teuthology.orchestra.run.smithi196.stdout: "evlrg": 0, 2024-04-23T10:43:13.461 INFO:teuthology.orchestra.run.smithi196.stdout: "evadd": 7, 2024-04-23T10:43:13.461 INFO:teuthology.orchestra.run.smithi196.stdout: "evex": 0, 2024-04-23T10:43:13.461 INFO:teuthology.orchestra.run.smithi196.stdout: "evtrm": 0, 2024-04-23T10:43:13.461 INFO:teuthology.orchestra.run.smithi196.stdout: "ev": 7, 2024-04-23T10:43:13.461 INFO:teuthology.orchestra.run.smithi196.stdout: "evexg": 0, 2024-04-23T10:43:13.461 INFO:teuthology.orchestra.run.smithi196.stdout: "evexd": 0, 2024-04-23T10:43:13.461 INFO:teuthology.orchestra.run.smithi196.stdout: "segadd": 2, 2024-04-23T10:43:13.461 INFO:teuthology.orchestra.run.smithi196.stdout: "segex": 0, 2024-04-23T10:43:13.461 INFO:teuthology.orchestra.run.smithi196.stdout: "segtrm": 0, 2024-04-23T10:43:13.461 INFO:teuthology.orchestra.run.smithi196.stdout: "seg": 2, 2024-04-23T10:43:13.461 INFO:teuthology.orchestra.run.smithi196.stdout: "segmjr": 2, 2024-04-23T10:43:13.461 INFO:teuthology.orchestra.run.smithi196.stdout: "segexg": 0, 2024-04-23T10:43:13.462 INFO:teuthology.orchestra.run.smithi196.stdout: "segexd": 0, 2024-04-23T10:43:13.462 INFO:teuthology.orchestra.run.smithi196.stdout: "expos": 4194304, 2024-04-23T10:43:13.462 INFO:teuthology.orchestra.run.smithi196.stdout: "wrpos": 4196264, 2024-04-23T10:43:13.462 INFO:teuthology.orchestra.run.smithi196.stdout: "rdpos": 0, 2024-04-23T10:43:13.462 INFO:teuthology.orchestra.run.smithi196.stdout: "jlat": { 2024-04-23T10:43:13.462 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 6, 2024-04-23T10:43:13.462 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.010843568, 2024-04-23T10:43:13.462 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.001807261 2024-04-23T10:43:13.462 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:13.462 INFO:teuthology.orchestra.run.smithi196.stdout: "replayed": 0 2024-04-23T10:43:13.462 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:13.462 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_mem": { 2024-04-23T10:43:13.462 INFO:teuthology.orchestra.run.smithi196.stdout: "ino": 20, 2024-04-23T10:43:13.462 INFO:teuthology.orchestra.run.smithi196.stdout: "ino+": 22, 2024-04-23T10:43:13.462 INFO:teuthology.orchestra.run.smithi196.stdout: "ino-": 2, 2024-04-23T10:43:13.463 INFO:teuthology.orchestra.run.smithi196.stdout: "dir": 19, 2024-04-23T10:43:13.463 INFO:teuthology.orchestra.run.smithi196.stdout: "dir+": 21, 2024-04-23T10:43:13.463 INFO:teuthology.orchestra.run.smithi196.stdout: "dir-": 2, 2024-04-23T10:43:13.463 INFO:teuthology.orchestra.run.smithi196.stdout: "dn": 16, 2024-04-23T10:43:13.463 INFO:teuthology.orchestra.run.smithi196.stdout: "dn+": 20, 2024-04-23T10:43:13.463 INFO:teuthology.orchestra.run.smithi196.stdout: "dn-": 4, 2024-04-23T10:43:13.463 INFO:teuthology.orchestra.run.smithi196.stdout: "cap": 1, 2024-04-23T10:43:13.463 INFO:teuthology.orchestra.run.smithi196.stdout: "cap+": 1, 2024-04-23T10:43:13.463 INFO:teuthology.orchestra.run.smithi196.stdout: "cap-": 0, 2024-04-23T10:43:13.463 INFO:teuthology.orchestra.run.smithi196.stdout: "rss": 52624, 2024-04-23T10:43:13.463 INFO:teuthology.orchestra.run.smithi196.stdout: "heap": 223608 2024-04-23T10:43:13.463 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:13.463 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_server": { 2024-04-23T10:43:13.463 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_client_request": 1, 2024-04-23T10:43:13.463 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_server_request": 0, 2024-04-23T10:43:13.463 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_request": 1, 2024-04-23T10:43:13.464 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_session": 22, 2024-04-23T10:43:13.464 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_peer_request": 101, 2024-04-23T10:43:13.464 INFO:teuthology.orchestra.run.smithi196.stdout: "req_create_latency": { 2024-04-23T10:43:13.464 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:13.464 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:13.464 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:13.464 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:13.464 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getattr_latency": { 2024-04-23T10:43:13.464 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:13.464 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:13.464 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:13.464 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:13.464 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getfilelock_latency": { 2024-04-23T10:43:13.464 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:13.464 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:13.464 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:13.465 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:13.465 INFO:teuthology.orchestra.run.smithi196.stdout: "req_link_latency": { 2024-04-23T10:43:13.465 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:13.465 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:13.465 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:13.465 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:13.465 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookup_latency": { 2024-04-23T10:43:13.465 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:13.465 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:13.465 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:13.465 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:13.465 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookuphash_latency": { 2024-04-23T10:43:13.465 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:13.465 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:13.465 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:13.465 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:13.466 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupino_latency": { 2024-04-23T10:43:13.466 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 1, 2024-04-23T10:43:13.466 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000125502, 2024-04-23T10:43:13.466 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000125502 2024-04-23T10:43:13.466 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:13.466 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupname_latency": { 2024-04-23T10:43:13.466 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:13.466 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:13.466 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:13.466 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:13.466 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupparent_latency": { 2024-04-23T10:43:13.466 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:13.466 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:13.466 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:13.466 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:13.467 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupsnap_latency": { 2024-04-23T10:43:13.467 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:13.467 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:13.467 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:13.467 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:13.467 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lssnap_latency": { 2024-04-23T10:43:13.467 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:13.467 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:13.467 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:13.467 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:13.467 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mkdir_latency": { 2024-04-23T10:43:13.468 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:13.468 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:13.468 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:13.468 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:13.468 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mknod_latency": { 2024-04-23T10:43:13.468 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:13.468 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:13.468 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:13.468 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:13.468 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mksnap_latency": { 2024-04-23T10:43:13.469 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:13.469 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:13.469 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:13.469 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:13.469 INFO:teuthology.orchestra.run.smithi196.stdout: "req_open_latency": { 2024-04-23T10:43:13.469 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:13.469 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:13.469 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:13.469 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:13.469 INFO:teuthology.orchestra.run.smithi196.stdout: "req_readdir_latency": { 2024-04-23T10:43:13.469 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:13.469 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:13.469 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:13.469 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:13.469 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rename_latency": { 2024-04-23T10:43:13.469 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:13.470 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:13.470 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:13.470 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:13.470 INFO:teuthology.orchestra.run.smithi196.stdout: "req_renamesnap_latency": { 2024-04-23T10:43:13.470 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:13.470 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:13.470 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:13.470 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:13.470 INFO:teuthology.orchestra.run.smithi196.stdout: "req_snapdiff_latency": { 2024-04-23T10:43:13.470 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:13.470 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:13.470 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:13.470 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:13.470 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmdir_latency": { 2024-04-23T10:43:13.471 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:13.471 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:13.471 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:13.471 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:13.471 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmsnap_latency": { 2024-04-23T10:43:13.471 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:13.471 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:13.471 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:13.471 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:13.471 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmxattr_latency": { 2024-04-23T10:43:13.471 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:13.471 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:13.471 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:13.471 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:13.471 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setattr_latency": { 2024-04-23T10:43:13.471 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:13.472 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:13.472 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:13.472 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:13.472 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setdirlayout_latency": { 2024-04-23T10:43:13.472 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:13.472 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:13.472 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:13.472 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:13.472 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setfilelock_latency": { 2024-04-23T10:43:13.472 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:13.472 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:13.472 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:13.472 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:13.472 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setlayout_latency": { 2024-04-23T10:43:13.473 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:13.473 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:13.473 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:13.473 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:13.473 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setxattr_latency": { 2024-04-23T10:43:13.473 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:13.473 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:13.473 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:13.473 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:13.473 INFO:teuthology.orchestra.run.smithi196.stdout: "req_symlink_latency": { 2024-04-23T10:43:13.473 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:13.473 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:13.473 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:13.473 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:13.473 INFO:teuthology.orchestra.run.smithi196.stdout: "req_unlink_latency": { 2024-04-23T10:43:13.473 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:13.474 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:13.474 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:13.474 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:13.474 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_revoke_eviction": 0, 2024-04-23T10:43:13.474 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_acquisition_throttle": 0, 2024-04-23T10:43:13.474 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getvxattr_latency": { 2024-04-23T10:43:13.474 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:13.474 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:13.474 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:13.474 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-23T10:43:13.474 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:13.474 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_sessions": { 2024-04-23T10:43:13.474 INFO:teuthology.orchestra.run.smithi196.stdout: "session_count": 1, 2024-04-23T10:43:13.474 INFO:teuthology.orchestra.run.smithi196.stdout: "session_add": 3, 2024-04-23T10:43:13.474 INFO:teuthology.orchestra.run.smithi196.stdout: "session_remove": 2, 2024-04-23T10:43:13.475 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_open": 1, 2024-04-23T10:43:13.475 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_stale": 0, 2024-04-23T10:43:13.475 INFO:teuthology.orchestra.run.smithi196.stdout: "total_load": 1, 2024-04-23T10:43:13.475 INFO:teuthology.orchestra.run.smithi196.stdout: "average_load": 0, 2024-04-23T10:43:13.475 INFO:teuthology.orchestra.run.smithi196.stdout: "avg_session_uptime": 1012, 2024-04-23T10:43:13.475 INFO:teuthology.orchestra.run.smithi196.stdout: "mdthresh_evicted": 0 2024-04-23T10:43:13.475 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:13.475 INFO:teuthology.orchestra.run.smithi196.stdout: "mempool": { 2024-04-23T10:43:13.475 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_bytes": 0, 2024-04-23T10:43:13.475 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_items": 0, 2024-04-23T10:43:13.475 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_bytes": 0, 2024-04-23T10:43:13.475 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_items": 0, 2024-04-23T10:43:13.475 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_bytes": 0, 2024-04-23T10:43:13.475 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_items": 0, 2024-04-23T10:43:13.475 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-23T10:43:13.475 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_items": 0, 2024-04-23T10:43:13.476 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-23T10:43:13.476 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_items": 0, 2024-04-23T10:43:13.476 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_bytes": 0, 2024-04-23T10:43:13.476 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_items": 0, 2024-04-23T10:43:13.476 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-23T10:43:13.476 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_items": 0, 2024-04-23T10:43:13.476 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_bytes": 0, 2024-04-23T10:43:13.476 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_items": 0, 2024-04-23T10:43:13.476 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_bytes": 0, 2024-04-23T10:43:13.476 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_items": 0, 2024-04-23T10:43:13.476 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-23T10:43:13.476 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_items": 0, 2024-04-23T10:43:13.476 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-23T10:43:13.476 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_items": 0, 2024-04-23T10:43:13.476 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_bytes": 0, 2024-04-23T10:43:13.476 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_items": 0, 2024-04-23T10:43:13.477 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_bytes": 0, 2024-04-23T10:43:13.477 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_items": 0, 2024-04-23T10:43:13.477 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-23T10:43:13.477 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_items": 0, 2024-04-23T10:43:13.477 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_bytes": 0, 2024-04-23T10:43:13.477 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_items": 0, 2024-04-23T10:43:13.477 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_bytes": 0, 2024-04-23T10:43:13.477 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_items": 0, 2024-04-23T10:43:13.477 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_bytes": 0, 2024-04-23T10:43:13.477 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_items": 0, 2024-04-23T10:43:13.477 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_bytes": 0, 2024-04-23T10:43:13.477 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_items": 0, 2024-04-23T10:43:13.477 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_bytes": 262759, 2024-04-23T10:43:13.477 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_items": 80, 2024-04-23T10:43:13.477 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_bytes": 0, 2024-04-23T10:43:13.477 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_items": 0, 2024-04-23T10:43:13.478 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_bytes": 0, 2024-04-23T10:43:13.478 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_items": 0, 2024-04-23T10:43:13.478 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_bytes": 0, 2024-04-23T10:43:13.478 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_items": 0, 2024-04-23T10:43:13.478 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_bytes": 0, 2024-04-23T10:43:13.478 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_items": 0, 2024-04-23T10:43:13.478 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_bytes": 7936, 2024-04-23T10:43:13.478 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_items": 165, 2024-04-23T10:43:13.478 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_bytes": 0, 2024-04-23T10:43:13.478 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_items": 0, 2024-04-23T10:43:13.478 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_bytes": 0, 2024-04-23T10:43:13.478 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_items": 0, 2024-04-23T10:43:13.478 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_bytes": 82221, 2024-04-23T10:43:13.478 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_items": 337, 2024-04-23T10:43:13.478 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_bytes": 0, 2024-04-23T10:43:13.479 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_items": 0, 2024-04-23T10:43:13.479 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_bytes": 0, 2024-04-23T10:43:13.479 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_items": 0 2024-04-23T10:43:13.479 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:13.479 INFO:teuthology.orchestra.run.smithi196.stdout: "objecter": { 2024-04-23T10:43:13.479 INFO:teuthology.orchestra.run.smithi196.stdout: "op_active": 0, 2024-04-23T10:43:13.479 INFO:teuthology.orchestra.run.smithi196.stdout: "op_laggy": 0, 2024-04-23T10:43:13.479 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send": 44, 2024-04-23T10:43:13.479 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send_bytes": 12563, 2024-04-23T10:43:13.479 INFO:teuthology.orchestra.run.smithi196.stdout: "op_resend": 28, 2024-04-23T10:43:13.479 INFO:teuthology.orchestra.run.smithi196.stdout: "op_reply": 44, 2024-04-23T10:43:13.479 INFO:teuthology.orchestra.run.smithi196.stdout: "op_latency": { 2024-04-23T10:43:13.479 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 44, 2024-04-23T10:43:13.479 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.240994534, 2024-04-23T10:43:13.479 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.005477148 2024-04-23T10:43:13.479 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:13.480 INFO:teuthology.orchestra.run.smithi196.stdout: "op_inflight": 0, 2024-04-23T10:43:13.480 INFO:teuthology.orchestra.run.smithi196.stdout: "oplen_avg": { 2024-04-23T10:43:13.480 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 44, 2024-04-23T10:43:13.480 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 67 2024-04-23T10:43:13.480 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:13.480 INFO:teuthology.orchestra.run.smithi196.stdout: "op": 44, 2024-04-23T10:43:13.480 INFO:teuthology.orchestra.run.smithi196.stdout: "op_r": 0, 2024-04-23T10:43:13.480 INFO:teuthology.orchestra.run.smithi196.stdout: "op_w": 44, 2024-04-23T10:43:13.480 INFO:teuthology.orchestra.run.smithi196.stdout: "op_rmw": 0, 2024-04-23T10:43:13.480 INFO:teuthology.orchestra.run.smithi196.stdout: "op_pg": 0, 2024-04-23T10:43:13.480 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_stat": 0, 2024-04-23T10:43:13.480 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_create": 10, 2024-04-23T10:43:13.480 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_read": 0, 2024-04-23T10:43:13.480 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_write": 6, 2024-04-23T10:43:13.480 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writefull": 8, 2024-04-23T10:43:13.480 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writesame": 0, 2024-04-23T10:43:13.481 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_append": 0, 2024-04-23T10:43:13.481 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_zero": 0, 2024-04-23T10:43:13.481 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_truncate": 0, 2024-04-23T10:43:13.481 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_delete": 6, 2024-04-23T10:43:13.481 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_mapext": 0, 2024-04-23T10:43:13.481 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_sparse_read": 0, 2024-04-23T10:43:13.481 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_clonerange": 0, 2024-04-23T10:43:13.481 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_getxattr": 0, 2024-04-23T10:43:13.481 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_setxattr": 20, 2024-04-23T10:43:13.481 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_cmpxattr": 0, 2024-04-23T10:43:13.481 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_rmxattr": 0, 2024-04-23T10:43:13.481 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_resetxattrs": 0, 2024-04-23T10:43:13.481 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_call": 0, 2024-04-23T10:43:13.481 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_watch": 0, 2024-04-23T10:43:13.481 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_notify": 0, 2024-04-23T10:43:13.481 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_src_cmpxattr": 0, 2024-04-23T10:43:13.482 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls": 0, 2024-04-23T10:43:13.482 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls_filter": 0, 2024-04-23T10:43:13.482 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_other": 0, 2024-04-23T10:43:13.482 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_active": 0, 2024-04-23T10:43:13.482 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_send": 0, 2024-04-23T10:43:13.482 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_resend": 0, 2024-04-23T10:43:13.482 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_ping": 0, 2024-04-23T10:43:13.482 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_active": 0, 2024-04-23T10:43:13.482 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_send": 0, 2024-04-23T10:43:13.482 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_resend": 0, 2024-04-23T10:43:13.482 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_active": 0, 2024-04-23T10:43:13.482 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_send": 0, 2024-04-23T10:43:13.482 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_resend": 0, 2024-04-23T10:43:13.482 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_active": 0, 2024-04-23T10:43:13.482 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_send": 0, 2024-04-23T10:43:13.483 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_resend": 0, 2024-04-23T10:43:13.483 INFO:teuthology.orchestra.run.smithi196.stdout: "command_active": 0, 2024-04-23T10:43:13.483 INFO:teuthology.orchestra.run.smithi196.stdout: "command_send": 0, 2024-04-23T10:43:13.483 INFO:teuthology.orchestra.run.smithi196.stdout: "command_resend": 0, 2024-04-23T10:43:13.483 INFO:teuthology.orchestra.run.smithi196.stdout: "map_epoch": 111, 2024-04-23T10:43:13.483 INFO:teuthology.orchestra.run.smithi196.stdout: "map_full": 0, 2024-04-23T10:43:13.483 INFO:teuthology.orchestra.run.smithi196.stdout: "map_inc": 25, 2024-04-23T10:43:13.483 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_sessions": 9, 2024-04-23T10:43:13.483 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_open": 14, 2024-04-23T10:43:13.483 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_close": 5, 2024-04-23T10:43:13.483 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_laggy": 0, 2024-04-23T10:43:13.483 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_wr": 16, 2024-04-23T10:43:13.483 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_rd": 0, 2024-04-23T10:43:13.483 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_del": 1 2024-04-23T10:43:13.483 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:13.483 INFO:teuthology.orchestra.run.smithi196.stdout: "oft": { 2024-04-23T10:43:13.484 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_objs": 1, 2024-04-23T10:43:13.484 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_kv_pairs": 0, 2024-04-23T10:43:13.484 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_updates": 0, 2024-04-23T10:43:13.484 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_removes": 0 2024-04-23T10:43:13.484 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:13.484 INFO:teuthology.orchestra.run.smithi196.stdout: "osd-slow-ops": { 2024-04-23T10:43:13.484 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_ops_count": 118 2024-04-23T10:43:13.484 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:13.484 INFO:teuthology.orchestra.run.smithi196.stdout: "purge_queue": { 2024-04-23T10:43:13.484 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops": 0, 2024-04-23T10:43:13.484 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops_high_water": 0, 2024-04-23T10:43:13.484 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing": 0, 2024-04-23T10:43:13.484 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_high_water": 0, 2024-04-23T10:43:13.484 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed_ops": 0, 2024-04-23T10:43:13.484 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed": 0, 2024-04-23T10:43:13.484 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_item_in_journal": 0 2024-04-23T10:43:13.485 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:13.485 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-23T10:43:13.485 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-04-23T10:43:13.485 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-04-23T10:43:13.485 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-04-23T10:43:13.485 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 2774, 2024-04-23T10:43:13.485 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 2058248, 2024-04-23T10:43:13.485 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-04-23T10:43:13.485 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 2774, 2024-04-23T10:43:13.485 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-04-23T10:43:13.485 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-04-23T10:43:13.485 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 2774, 2024-04-23T10:43:13.485 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 2058248, 2024-04-23T10:43:13.485 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-04-23T10:43:13.485 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:13.485 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:13.486 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:13.486 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-23T10:43:13.486 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:13.486 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_bytes": { 2024-04-23T10:43:13.486 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-04-23T10:43:13.486 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-04-23T10:43:13.486 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-04-23T10:43:13.486 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-04-23T10:43:13.486 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-04-23T10:43:13.486 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-04-23T10:43:13.486 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-04-23T10:43:13.486 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 44, 2024-04-23T10:43:13.486 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 12563, 2024-04-23T10:43:13.486 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 44, 2024-04-23T10:43:13.486 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 12563, 2024-04-23T10:43:13.486 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-04-23T10:43:13.487 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:13.487 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:13.487 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:13.487 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-23T10:43:13.487 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:13.487 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_ops": { 2024-04-23T10:43:13.487 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-04-23T10:43:13.487 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 1024, 2024-04-23T10:43:13.487 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-04-23T10:43:13.487 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-04-23T10:43:13.487 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-04-23T10:43:13.487 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-04-23T10:43:13.487 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-04-23T10:43:13.487 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 44, 2024-04-23T10:43:13.487 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 44, 2024-04-23T10:43:13.487 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 44, 2024-04-23T10:43:13.488 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 44, 2024-04-23T10:43:13.488 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-04-23T10:43:13.488 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:13.488 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:13.488 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:13.488 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-23T10:43:13.488 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:13.488 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle": { 2024-04-23T10:43:13.488 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-04-23T10:43:13.488 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-04-23T10:43:13.488 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-04-23T10:43:13.488 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-04-23T10:43:13.488 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-04-23T10:43:13.488 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-04-23T10:43:13.488 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-04-23T10:43:13.488 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-04-23T10:43:13.489 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-04-23T10:43:13.489 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 0, 2024-04-23T10:43:13.489 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 0, 2024-04-23T10:43:13.489 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-04-23T10:43:13.489 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:13.489 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:13.489 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:13.489 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-23T10:43:13.489 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:13.489 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle-0x5557a304c500": { 2024-04-23T10:43:13.489 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-04-23T10:43:13.489 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-04-23T10:43:13.489 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-04-23T10:43:13.489 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 7, 2024-04-23T10:43:13.489 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 1960, 2024-04-23T10:43:13.489 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-04-23T10:43:13.489 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 7, 2024-04-23T10:43:13.490 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-04-23T10:43:13.490 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-04-23T10:43:13.490 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 6, 2024-04-23T10:43:13.490 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 1960, 2024-04-23T10:43:13.490 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-04-23T10:43:13.490 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:13.490 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:13.490 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:13.490 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-23T10:43:13.490 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-23T10:43:13.490 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-23T10:43:13.512 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:43:13 smithi097 ceph-mon[29197]: pgmap v535: 97 pgs: 97 active+clean; 4.5 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-23T10:43:13.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:43:13 smithi183 ceph-mon[32317]: pgmap v535: 97 pgs: 97 active+clean; 4.5 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-23T10:43:14.280 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-04-23T10:43:14.280 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-04-23T10:43:14.280 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2024-04-23T10:43:14.280 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=19 2024-04-23T10:43:14.281 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-04-23T10:43:14.281 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-04-23T10:43:14.281 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-04-23T10:43:14.281 DEBUG:tasks.check_counter:Getting stats from l 2024-04-23T10:43:14.281 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 9b22af88-0159-11ef-bc93-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-23T10:43:14.521 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T10:43:15.592 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:43:15 smithi196 ceph-mon[32919]: pgmap v536: 97 pgs: 97 active+clean; 4.5 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 383 B/s wr, 13 op/s 2024-04-23T10:43:15.610 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:43:15 smithi097 ceph-mon[29197]: pgmap v536: 97 pgs: 97 active+clean; 4.5 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 383 B/s wr, 13 op/s 2024-04-23T10:43:15.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:43:15 smithi183 ceph-mon[32317]: pgmap v536: 97 pgs: 97 active+clean; 4.5 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 383 B/s wr, 13 op/s 2024-04-23T10:43:16.278 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-23T10:43:16.278 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-0": { 2024-04-23T10:43:16.278 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 43, 2024-04-23T10:43:16.278 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 250, 2024-04-23T10:43:16.278 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 10767, 2024-04-23T10:43:16.279 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 607393, 2024-04-23T10:43:16.279 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 15, 2024-04-23T10:43:16.279 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 3, 2024-04-23T10:43:16.279 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.017564778, 2024-04-23T10:43:16.279 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.009250340, 2024-04-23T10:43:16.279 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.004056980, 2024-04-23T10:43:16.279 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.000773819, 2024-04-23T10:43:16.279 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T10:43:16.279 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 248, 2024-04-23T10:43:16.279 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.029958077, 2024-04-23T10:43:16.280 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000120798 2024-04-23T10:43:16.280 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:16.280 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-04-23T10:43:16.280 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 17, 2024-04-23T10:43:16.280 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000001850, 2024-04-23T10:43:16.280 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000108 2024-04-23T10:43:16.280 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:16.280 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 2256, 2024-04-23T10:43:16.280 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 544592 2024-04-23T10:43:16.280 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:16.281 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-1": { 2024-04-23T10:43:16.281 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 420, 2024-04-23T10:43:16.281 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 404, 2024-04-23T10:43:16.281 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 1694219, 2024-04-23T10:43:16.281 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 216919, 2024-04-23T10:43:16.281 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 7, 2024-04-23T10:43:16.281 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 5, 2024-04-23T10:43:16.281 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.073273999, 2024-04-23T10:43:16.281 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.023302469, 2024-04-23T10:43:16.281 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.050348261, 2024-04-23T10:43:16.282 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.008524951, 2024-04-23T10:43:16.282 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T10:43:16.282 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 404, 2024-04-23T10:43:16.282 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.045084078, 2024-04-23T10:43:16.282 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000111594 2024-04-23T10:43:16.282 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:16.282 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-04-23T10:43:16.282 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 23, 2024-04-23T10:43:16.282 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000003972, 2024-04-23T10:43:16.282 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000172 2024-04-23T10:43:16.283 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:16.283 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 1690928, 2024-04-23T10:43:16.283 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 211248 2024-04-23T10:43:16.283 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:16.283 INFO:teuthology.orchestra.run.smithi196.stdout: "AsyncMessenger::Worker-2": { 2024-04-23T10:43:16.283 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_messages": 2377, 2024-04-23T10:43:16.283 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages": 1701, 2024-04-23T10:43:16.283 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_bytes": 626271, 2024-04-23T10:43:16.283 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_bytes": 685669, 2024-04-23T10:43:16.283 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_created_connections": 13, 2024-04-23T10:43:16.284 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_active_connections": 5, 2024-04-23T10:43:16.284 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_total_time": 0.179624135, 2024-04-23T10:43:16.284 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_send_time": 0.071276509, 2024-04-23T10:43:16.284 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_recv_time": 0.115385603, 2024-04-23T10:43:16.284 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_running_fast_dispatch_time": 0.007310449, 2024-04-23T10:43:16.284 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T10:43:16.284 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 1698, 2024-04-23T10:43:16.284 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.075773417, 2024-04-23T10:43:16.284 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000044625 2024-04-23T10:43:16.284 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:16.284 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_handle_ack_lat": { 2024-04-23T10:43:16.285 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 3857, 2024-04-23T10:43:16.285 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.001035517, 2024-04-23T10:43:16.285 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000268 2024-04-23T10:43:16.285 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:16.285 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_recv_encrypted_bytes": 5616, 2024-04-23T10:43:16.285 INFO:teuthology.orchestra.run.smithi196.stdout: "msgr_send_encrypted_bytes": 149744 2024-04-23T10:43:16.285 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:16.285 INFO:teuthology.orchestra.run.smithi196.stdout: "cct": { 2024-04-23T10:43:16.285 INFO:teuthology.orchestra.run.smithi196.stdout: "total_workers": 1, 2024-04-23T10:43:16.285 INFO:teuthology.orchestra.run.smithi196.stdout: "unhealthy_workers": 0 2024-04-23T10:43:16.286 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:16.286 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-MDSRank": { 2024-04-23T10:43:16.286 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-04-23T10:43:16.286 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-04-23T10:43:16.286 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 68, 2024-04-23T10:43:16.286 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.001636263, 2024-04-23T10:43:16.286 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000024062 2024-04-23T10:43:16.286 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-23T10:43:16.286 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:16.286 INFO:teuthology.orchestra.run.smithi196.stdout: "finisher-PurgeQueue": { 2024-04-23T10:43:16.287 INFO:teuthology.orchestra.run.smithi196.stdout: "queue_len": 0, 2024-04-23T10:43:16.287 INFO:teuthology.orchestra.run.smithi196.stdout: "complete_latency": { 2024-04-23T10:43:16.287 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 2, 2024-04-23T10:43:16.287 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000014996, 2024-04-23T10:43:16.287 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000007498 2024-04-23T10:43:16.287 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-23T10:43:16.287 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:16.287 INFO:teuthology.orchestra.run.smithi196.stdout: "mds": { 2024-04-23T10:43:16.287 INFO:teuthology.orchestra.run.smithi196.stdout: "request": 13, 2024-04-23T10:43:16.287 INFO:teuthology.orchestra.run.smithi196.stdout: "reply": 7, 2024-04-23T10:43:16.288 INFO:teuthology.orchestra.run.smithi196.stdout: "reply_latency": { 2024-04-23T10:43:16.288 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 7, 2024-04-23T10:43:16.288 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.001721756, 2024-04-23T10:43:16.288 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000245965 2024-04-23T10:43:16.288 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:16.288 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_reply": 0, 2024-04-23T10:43:16.288 INFO:teuthology.orchestra.run.smithi196.stdout: "forward": 6, 2024-04-23T10:43:16.288 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_complete": 0, 2024-04-23T10:43:16.288 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_fetch_keys": 0, 2024-04-23T10:43:16.289 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_commit": 11, 2024-04-23T10:43:16.289 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_split": 0, 2024-04-23T10:43:16.289 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_merge": 0, 2024-04-23T10:43:16.289 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes": 16, 2024-04-23T10:43:16.290 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_top": 0, 2024-04-23T10:43:16.290 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_bottom": 16, 2024-04-23T10:43:16.290 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pin_tail": 0, 2024-04-23T10:43:16.290 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_pinned": 13, 2024-04-23T10:43:16.290 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_expired": 4, 2024-04-23T10:43:16.290 INFO:teuthology.orchestra.run.smithi196.stdout: "inodes_with_caps": 1, 2024-04-23T10:43:16.290 INFO:teuthology.orchestra.run.smithi196.stdout: "caps": 1, 2024-04-23T10:43:16.290 INFO:teuthology.orchestra.run.smithi196.stdout: "subtrees": 3, 2024-04-23T10:43:16.290 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse": 64, 2024-04-23T10:43:16.290 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_hit": 52, 2024-04-23T10:43:16.291 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_forward": 0, 2024-04-23T10:43:16.291 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_discover": 7, 2024-04-23T10:43:16.291 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_dir_fetch": 0, 2024-04-23T10:43:16.291 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_remote_ino": 0, 2024-04-23T10:43:16.291 INFO:teuthology.orchestra.run.smithi196.stdout: "traverse_lock": 1, 2024-04-23T10:43:16.291 INFO:teuthology.orchestra.run.smithi196.stdout: "load_cent": 0, 2024-04-23T10:43:16.291 INFO:teuthology.orchestra.run.smithi196.stdout: "q": 0, 2024-04-23T10:43:16.291 INFO:teuthology.orchestra.run.smithi196.stdout: "exported": 0, 2024-04-23T10:43:16.291 INFO:teuthology.orchestra.run.smithi196.stdout: "exported_inodes": 0, 2024-04-23T10:43:16.291 INFO:teuthology.orchestra.run.smithi196.stdout: "imported": 0, 2024-04-23T10:43:16.291 INFO:teuthology.orchestra.run.smithi196.stdout: "imported_inodes": 0, 2024-04-23T10:43:16.292 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_dir_fetch": 0, 2024-04-23T10:43:16.292 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_backtrace_fetch": 0, 2024-04-23T10:43:16.292 INFO:teuthology.orchestra.run.smithi196.stdout: "openino_peer_discover": 0, 2024-04-23T10:43:16.292 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rfiles": 4, 2024-04-23T10:43:16.292 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rbytes": 234, 2024-04-23T10:43:16.292 INFO:teuthology.orchestra.run.smithi196.stdout: "root_rsnaps": 6, 2024-04-23T10:43:16.292 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_fetch": 0, 2024-04-23T10:43:16.292 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_set_tag": 0, 2024-04-23T10:43:16.292 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_backtrace_repaired": 0, 2024-04-23T10:43:16.292 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_inotable_repaired": 0, 2024-04-23T10:43:16.292 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_inodes": 0, 2024-04-23T10:43:16.293 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dir_base_inodes": 0, 2024-04-23T10:43:16.293 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_dirfrag_rstats": 0, 2024-04-23T10:43:16.293 INFO:teuthology.orchestra.run.smithi196.stdout: "scrub_file_inodes": 0, 2024-04-23T10:43:16.293 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_inode_file_caps": 0, 2024-04-23T10:43:16.293 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_revoke": 4, 2024-04-23T10:43:16.293 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_grant": 0, 2024-04-23T10:43:16.293 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_trunc": 0, 2024-04-23T10:43:16.293 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-23T10:43:16.293 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-23T10:43:16.293 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps": 4, 2024-04-23T10:43:16.294 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_caps_dirty": 0, 2024-04-23T10:43:16.294 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_cap_release": 0, 2024-04-23T10:43:16.294 INFO:teuthology.orchestra.run.smithi196.stdout: "process_request_cap_release": 2 2024-04-23T10:43:16.294 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:16.294 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_cache": { 2024-04-23T10:43:16.294 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update": 0, 2024-04-23T10:43:16.294 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_update_receipt": 17, 2024-04-23T10:43:16.294 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_try_discover": 13, 2024-04-23T10:43:16.294 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_send_discover": 10, 2024-04-23T10:43:16.294 INFO:teuthology.orchestra.run.smithi196.stdout: "dir_handle_discover": 0, 2024-04-23T10:43:16.294 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays": 0, 2024-04-23T10:43:16.295 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_delayed": 0, 2024-04-23T10:43:16.295 INFO:teuthology.orchestra.run.smithi196.stdout: "num_strays_enqueuing": 0, 2024-04-23T10:43:16.295 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_created": 0, 2024-04-23T10:43:16.295 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_enqueued": 0, 2024-04-23T10:43:16.295 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_reintegrated": 0, 2024-04-23T10:43:16.295 INFO:teuthology.orchestra.run.smithi196.stdout: "strays_migrated": 0, 2024-04-23T10:43:16.295 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_processing": 0, 2024-04-23T10:43:16.295 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_enqueued": 0, 2024-04-23T10:43:16.295 INFO:teuthology.orchestra.run.smithi196.stdout: "num_recovering_prioritized": 0, 2024-04-23T10:43:16.295 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_started": 0, 2024-04-23T10:43:16.295 INFO:teuthology.orchestra.run.smithi196.stdout: "recovery_completed": 0, 2024-04-23T10:43:16.296 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_path": 17, 2024-04-23T10:43:16.296 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_quiesce_inode": 101, 2024-04-23T10:43:16.296 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_enqueue_scrub": 0, 2024-04-23T10:43:16.296 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_exportdir": 0, 2024-04-23T10:43:16.296 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_flush": 0, 2024-04-23T10:43:16.296 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragmentdir": 0, 2024-04-23T10:43:16.296 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_fragstats": 0, 2024-04-23T10:43:16.296 INFO:teuthology.orchestra.run.smithi196.stdout: "ireq_inodestats": 0 2024-04-23T10:43:16.296 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:16.296 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_log": { 2024-04-23T10:43:16.296 INFO:teuthology.orchestra.run.smithi196.stdout: "evlrg": 0, 2024-04-23T10:43:16.297 INFO:teuthology.orchestra.run.smithi196.stdout: "evadd": 9, 2024-04-23T10:43:16.297 INFO:teuthology.orchestra.run.smithi196.stdout: "evex": 0, 2024-04-23T10:43:16.297 INFO:teuthology.orchestra.run.smithi196.stdout: "evtrm": 0, 2024-04-23T10:43:16.297 INFO:teuthology.orchestra.run.smithi196.stdout: "ev": 9, 2024-04-23T10:43:16.297 INFO:teuthology.orchestra.run.smithi196.stdout: "evexg": 0, 2024-04-23T10:43:16.297 INFO:teuthology.orchestra.run.smithi196.stdout: "evexd": 0, 2024-04-23T10:43:16.297 INFO:teuthology.orchestra.run.smithi196.stdout: "segadd": 2, 2024-04-23T10:43:16.297 INFO:teuthology.orchestra.run.smithi196.stdout: "segex": 0, 2024-04-23T10:43:16.297 INFO:teuthology.orchestra.run.smithi196.stdout: "segtrm": 0, 2024-04-23T10:43:16.297 INFO:teuthology.orchestra.run.smithi196.stdout: "seg": 2, 2024-04-23T10:43:16.297 INFO:teuthology.orchestra.run.smithi196.stdout: "segmjr": 2, 2024-04-23T10:43:16.298 INFO:teuthology.orchestra.run.smithi196.stdout: "segexg": 0, 2024-04-23T10:43:16.298 INFO:teuthology.orchestra.run.smithi196.stdout: "segexd": 0, 2024-04-23T10:43:16.298 INFO:teuthology.orchestra.run.smithi196.stdout: "expos": 4194304, 2024-04-23T10:43:16.298 INFO:teuthology.orchestra.run.smithi196.stdout: "wrpos": 4196879, 2024-04-23T10:43:16.298 INFO:teuthology.orchestra.run.smithi196.stdout: "rdpos": 0, 2024-04-23T10:43:16.298 INFO:teuthology.orchestra.run.smithi196.stdout: "jlat": { 2024-04-23T10:43:16.298 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 8, 2024-04-23T10:43:16.298 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.013974698, 2024-04-23T10:43:16.298 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.001746837 2024-04-23T10:43:16.298 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:16.299 INFO:teuthology.orchestra.run.smithi196.stdout: "replayed": 0 2024-04-23T10:43:16.299 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:16.299 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_mem": { 2024-04-23T10:43:16.299 INFO:teuthology.orchestra.run.smithi196.stdout: "ino": 20, 2024-04-23T10:43:16.299 INFO:teuthology.orchestra.run.smithi196.stdout: "ino+": 22, 2024-04-23T10:43:16.299 INFO:teuthology.orchestra.run.smithi196.stdout: "ino-": 2, 2024-04-23T10:43:16.299 INFO:teuthology.orchestra.run.smithi196.stdout: "dir": 19, 2024-04-23T10:43:16.299 INFO:teuthology.orchestra.run.smithi196.stdout: "dir+": 20, 2024-04-23T10:43:16.299 INFO:teuthology.orchestra.run.smithi196.stdout: "dir-": 1, 2024-04-23T10:43:16.299 INFO:teuthology.orchestra.run.smithi196.stdout: "dn": 16, 2024-04-23T10:43:16.300 INFO:teuthology.orchestra.run.smithi196.stdout: "dn+": 20, 2024-04-23T10:43:16.300 INFO:teuthology.orchestra.run.smithi196.stdout: "dn-": 4, 2024-04-23T10:43:16.300 INFO:teuthology.orchestra.run.smithi196.stdout: "cap": 1, 2024-04-23T10:43:16.300 INFO:teuthology.orchestra.run.smithi196.stdout: "cap+": 5, 2024-04-23T10:43:16.300 INFO:teuthology.orchestra.run.smithi196.stdout: "cap-": 4, 2024-04-23T10:43:16.300 INFO:teuthology.orchestra.run.smithi196.stdout: "rss": 53436, 2024-04-23T10:43:16.300 INFO:teuthology.orchestra.run.smithi196.stdout: "heap": 223608 2024-04-23T10:43:16.300 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:16.300 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_server": { 2024-04-23T10:43:16.300 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_client_request": 13, 2024-04-23T10:43:16.300 INFO:teuthology.orchestra.run.smithi196.stdout: "dispatch_server_request": 0, 2024-04-23T10:43:16.300 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_request": 13, 2024-04-23T10:43:16.301 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_client_session": 24, 2024-04-23T10:43:16.301 INFO:teuthology.orchestra.run.smithi196.stdout: "handle_peer_request": 101, 2024-04-23T10:43:16.301 INFO:teuthology.orchestra.run.smithi196.stdout: "req_create_latency": { 2024-04-23T10:43:16.301 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:16.301 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:16.301 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:16.301 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:16.301 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getattr_latency": { 2024-04-23T10:43:16.301 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 5, 2024-04-23T10:43:16.302 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.001356196, 2024-04-23T10:43:16.302 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000271239 2024-04-23T10:43:16.302 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:16.302 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getfilelock_latency": { 2024-04-23T10:43:16.302 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:16.302 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:16.302 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:16.302 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:16.302 INFO:teuthology.orchestra.run.smithi196.stdout: "req_link_latency": { 2024-04-23T10:43:16.302 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:16.302 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:16.303 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:16.303 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:16.303 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookup_latency": { 2024-04-23T10:43:16.303 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 2, 2024-04-23T10:43:16.303 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000365560, 2024-04-23T10:43:16.303 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000182780 2024-04-23T10:43:16.303 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:16.303 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookuphash_latency": { 2024-04-23T10:43:16.303 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:16.303 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:16.303 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:16.303 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:16.304 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupino_latency": { 2024-04-23T10:43:16.304 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:16.304 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:16.304 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:16.304 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:16.304 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupname_latency": { 2024-04-23T10:43:16.304 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:16.304 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:16.304 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:16.304 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:16.304 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupparent_latency": { 2024-04-23T10:43:16.304 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:16.305 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:16.305 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:16.305 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:16.305 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lookupsnap_latency": { 2024-04-23T10:43:16.305 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:16.305 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:16.305 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:16.305 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:16.305 INFO:teuthology.orchestra.run.smithi196.stdout: "req_lssnap_latency": { 2024-04-23T10:43:16.305 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:16.305 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:16.305 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:16.306 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:16.306 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mkdir_latency": { 2024-04-23T10:43:16.306 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:16.306 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:16.307 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:16.307 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:16.307 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mknod_latency": { 2024-04-23T10:43:16.307 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:16.307 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:16.307 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:16.307 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:16.307 INFO:teuthology.orchestra.run.smithi196.stdout: "req_mksnap_latency": { 2024-04-23T10:43:16.307 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:16.307 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:16.307 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:16.308 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:16.308 INFO:teuthology.orchestra.run.smithi196.stdout: "req_open_latency": { 2024-04-23T10:43:16.308 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:16.308 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:16.308 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:16.308 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:16.308 INFO:teuthology.orchestra.run.smithi196.stdout: "req_readdir_latency": { 2024-04-23T10:43:16.308 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:16.308 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:16.308 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:16.308 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:16.308 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rename_latency": { 2024-04-23T10:43:16.309 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:16.309 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:16.309 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:16.309 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:16.309 INFO:teuthology.orchestra.run.smithi196.stdout: "req_renamesnap_latency": { 2024-04-23T10:43:16.309 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:16.309 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:16.309 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:16.309 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:16.309 INFO:teuthology.orchestra.run.smithi196.stdout: "req_snapdiff_latency": { 2024-04-23T10:43:16.309 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:16.310 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:16.310 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:16.310 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:16.310 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmdir_latency": { 2024-04-23T10:43:16.310 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:16.310 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:16.310 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:16.310 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:16.310 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmsnap_latency": { 2024-04-23T10:43:16.310 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:16.310 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:16.310 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:16.311 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:16.311 INFO:teuthology.orchestra.run.smithi196.stdout: "req_rmxattr_latency": { 2024-04-23T10:43:16.311 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:16.311 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:16.311 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:16.311 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:16.311 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setattr_latency": { 2024-04-23T10:43:16.311 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:16.311 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:16.311 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:16.311 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:16.311 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setdirlayout_latency": { 2024-04-23T10:43:16.312 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:16.312 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:16.312 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:16.312 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:16.312 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setfilelock_latency": { 2024-04-23T10:43:16.312 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:16.312 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:16.312 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:16.312 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:16.312 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setlayout_latency": { 2024-04-23T10:43:16.312 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:16.312 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:16.313 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:16.313 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:16.313 INFO:teuthology.orchestra.run.smithi196.stdout: "req_setxattr_latency": { 2024-04-23T10:43:16.313 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:16.313 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:16.313 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:16.313 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:16.313 INFO:teuthology.orchestra.run.smithi196.stdout: "req_symlink_latency": { 2024-04-23T10:43:16.313 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:16.313 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:16.313 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:16.313 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:16.314 INFO:teuthology.orchestra.run.smithi196.stdout: "req_unlink_latency": { 2024-04-23T10:43:16.314 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:16.314 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:16.314 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:16.314 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:16.314 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_revoke_eviction": 0, 2024-04-23T10:43:16.314 INFO:teuthology.orchestra.run.smithi196.stdout: "cap_acquisition_throttle": 0, 2024-04-23T10:43:16.314 INFO:teuthology.orchestra.run.smithi196.stdout: "req_getvxattr_latency": { 2024-04-23T10:43:16.314 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:16.314 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:16.314 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:16.314 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-23T10:43:16.315 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:16.315 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_sessions": { 2024-04-23T10:43:16.315 INFO:teuthology.orchestra.run.smithi196.stdout: "session_count": 1, 2024-04-23T10:43:16.315 INFO:teuthology.orchestra.run.smithi196.stdout: "session_add": 4, 2024-04-23T10:43:16.315 INFO:teuthology.orchestra.run.smithi196.stdout: "session_remove": 3, 2024-04-23T10:43:16.315 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_open": 1, 2024-04-23T10:43:16.315 INFO:teuthology.orchestra.run.smithi196.stdout: "sessions_stale": 0, 2024-04-23T10:43:16.315 INFO:teuthology.orchestra.run.smithi196.stdout: "total_load": 4, 2024-04-23T10:43:16.315 INFO:teuthology.orchestra.run.smithi196.stdout: "average_load": 1, 2024-04-23T10:43:16.315 INFO:teuthology.orchestra.run.smithi196.stdout: "avg_session_uptime": 973, 2024-04-23T10:43:16.315 INFO:teuthology.orchestra.run.smithi196.stdout: "mdthresh_evicted": 0 2024-04-23T10:43:16.315 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:16.316 INFO:teuthology.orchestra.run.smithi196.stdout: "mempool": { 2024-04-23T10:43:16.316 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_bytes": 0, 2024-04-23T10:43:16.316 INFO:teuthology.orchestra.run.smithi196.stdout: "bloom_filter_items": 0, 2024-04-23T10:43:16.316 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_bytes": 0, 2024-04-23T10:43:16.316 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_alloc_items": 0, 2024-04-23T10:43:16.316 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_bytes": 0, 2024-04-23T10:43:16.316 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_data_items": 0, 2024-04-23T10:43:16.316 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-23T10:43:16.316 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_onode_items": 0, 2024-04-23T10:43:16.316 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-23T10:43:16.316 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_meta_items": 0, 2024-04-23T10:43:16.316 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_bytes": 0, 2024-04-23T10:43:16.317 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_other_items": 0, 2024-04-23T10:43:16.317 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-23T10:43:16.317 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_cache_buffer_items": 0, 2024-04-23T10:43:16.317 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_bytes": 0, 2024-04-23T10:43:16.317 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_extent_items": 0, 2024-04-23T10:43:16.317 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_bytes": 0, 2024-04-23T10:43:16.317 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_blob_items": 0, 2024-04-23T10:43:16.317 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-23T10:43:16.317 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_shared_blob_items": 0, 2024-04-23T10:43:16.317 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-23T10:43:16.317 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_inline_bl_items": 0, 2024-04-23T10:43:16.317 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_bytes": 0, 2024-04-23T10:43:16.318 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_fsck_items": 0, 2024-04-23T10:43:16.318 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_bytes": 0, 2024-04-23T10:43:16.318 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_txc_items": 0, 2024-04-23T10:43:16.318 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-23T10:43:16.318 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_deferred_items": 0, 2024-04-23T10:43:16.318 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_bytes": 0, 2024-04-23T10:43:16.318 INFO:teuthology.orchestra.run.smithi196.stdout: "bluestore_writing_items": 0, 2024-04-23T10:43:16.318 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_bytes": 0, 2024-04-23T10:43:16.318 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_items": 0, 2024-04-23T10:43:16.318 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_bytes": 0, 2024-04-23T10:43:16.318 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_reader_items": 0, 2024-04-23T10:43:16.318 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_bytes": 0, 2024-04-23T10:43:16.319 INFO:teuthology.orchestra.run.smithi196.stdout: "bluefs_file_writer_items": 0, 2024-04-23T10:43:16.319 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_bytes": 247127, 2024-04-23T10:43:16.319 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_anon_items": 80, 2024-04-23T10:43:16.319 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_bytes": 0, 2024-04-23T10:43:16.319 INFO:teuthology.orchestra.run.smithi196.stdout: "buffer_meta_items": 0, 2024-04-23T10:43:16.319 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_bytes": 0, 2024-04-23T10:43:16.319 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_items": 0, 2024-04-23T10:43:16.319 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_bytes": 0, 2024-04-23T10:43:16.319 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_mapbl_items": 0, 2024-04-23T10:43:16.319 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_bytes": 0, 2024-04-23T10:43:16.319 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_pglog_items": 0, 2024-04-23T10:43:16.319 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_bytes": 7936, 2024-04-23T10:43:16.320 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_items": 165, 2024-04-23T10:43:16.320 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_bytes": 0, 2024-04-23T10:43:16.320 INFO:teuthology.orchestra.run.smithi196.stdout: "osdmap_mapping_items": 0, 2024-04-23T10:43:16.320 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_bytes": 0, 2024-04-23T10:43:16.320 INFO:teuthology.orchestra.run.smithi196.stdout: "pgmap_items": 0, 2024-04-23T10:43:16.320 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_bytes": 82277, 2024-04-23T10:43:16.320 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_co_items": 344, 2024-04-23T10:43:16.320 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_bytes": 0, 2024-04-23T10:43:16.320 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_1_items": 0, 2024-04-23T10:43:16.320 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_bytes": 0, 2024-04-23T10:43:16.320 INFO:teuthology.orchestra.run.smithi196.stdout: "unittest_2_items": 0 2024-04-23T10:43:16.320 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:16.320 INFO:teuthology.orchestra.run.smithi196.stdout: "objecter": { 2024-04-23T10:43:16.321 INFO:teuthology.orchestra.run.smithi196.stdout: "op_active": 0, 2024-04-23T10:43:16.321 INFO:teuthology.orchestra.run.smithi196.stdout: "op_laggy": 0, 2024-04-23T10:43:16.321 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send": 48, 2024-04-23T10:43:16.321 INFO:teuthology.orchestra.run.smithi196.stdout: "op_send_bytes": 13282, 2024-04-23T10:43:16.321 INFO:teuthology.orchestra.run.smithi196.stdout: "op_resend": 33, 2024-04-23T10:43:16.321 INFO:teuthology.orchestra.run.smithi196.stdout: "op_reply": 48, 2024-04-23T10:43:16.321 INFO:teuthology.orchestra.run.smithi196.stdout: "op_latency": { 2024-04-23T10:43:16.321 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 48, 2024-04-23T10:43:16.321 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.237994598, 2024-04-23T10:43:16.321 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.004958220 2024-04-23T10:43:16.321 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:16.321 INFO:teuthology.orchestra.run.smithi196.stdout: "op_inflight": 0, 2024-04-23T10:43:16.322 INFO:teuthology.orchestra.run.smithi196.stdout: "oplen_avg": { 2024-04-23T10:43:16.322 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 48, 2024-04-23T10:43:16.322 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 72 2024-04-23T10:43:16.322 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:16.322 INFO:teuthology.orchestra.run.smithi196.stdout: "op": 48, 2024-04-23T10:43:16.322 INFO:teuthology.orchestra.run.smithi196.stdout: "op_r": 0, 2024-04-23T10:43:16.322 INFO:teuthology.orchestra.run.smithi196.stdout: "op_w": 48, 2024-04-23T10:43:16.322 INFO:teuthology.orchestra.run.smithi196.stdout: "op_rmw": 0, 2024-04-23T10:43:16.322 INFO:teuthology.orchestra.run.smithi196.stdout: "op_pg": 0, 2024-04-23T10:43:16.322 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_stat": 0, 2024-04-23T10:43:16.322 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_create": 10, 2024-04-23T10:43:16.322 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_read": 0, 2024-04-23T10:43:16.323 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_write": 8, 2024-04-23T10:43:16.323 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writefull": 9, 2024-04-23T10:43:16.323 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_writesame": 0, 2024-04-23T10:43:16.323 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_append": 0, 2024-04-23T10:43:16.323 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_zero": 0, 2024-04-23T10:43:16.323 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_truncate": 0, 2024-04-23T10:43:16.323 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_delete": 6, 2024-04-23T10:43:16.323 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_mapext": 0, 2024-04-23T10:43:16.323 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_sparse_read": 0, 2024-04-23T10:43:16.323 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_clonerange": 0, 2024-04-23T10:43:16.323 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_getxattr": 0, 2024-04-23T10:43:16.323 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_setxattr": 20, 2024-04-23T10:43:16.323 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_cmpxattr": 0, 2024-04-23T10:43:16.324 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_rmxattr": 0, 2024-04-23T10:43:16.324 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_resetxattrs": 0, 2024-04-23T10:43:16.324 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_call": 0, 2024-04-23T10:43:16.324 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_watch": 0, 2024-04-23T10:43:16.324 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_notify": 0, 2024-04-23T10:43:16.324 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_src_cmpxattr": 0, 2024-04-23T10:43:16.324 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls": 0, 2024-04-23T10:43:16.324 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_pgls_filter": 0, 2024-04-23T10:43:16.324 INFO:teuthology.orchestra.run.smithi196.stdout: "osdop_other": 0, 2024-04-23T10:43:16.324 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_active": 0, 2024-04-23T10:43:16.324 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_send": 0, 2024-04-23T10:43:16.324 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_resend": 0, 2024-04-23T10:43:16.325 INFO:teuthology.orchestra.run.smithi196.stdout: "linger_ping": 0, 2024-04-23T10:43:16.325 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_active": 0, 2024-04-23T10:43:16.325 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_send": 0, 2024-04-23T10:43:16.325 INFO:teuthology.orchestra.run.smithi196.stdout: "poolop_resend": 0, 2024-04-23T10:43:16.325 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_active": 0, 2024-04-23T10:43:16.325 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_send": 0, 2024-04-23T10:43:16.325 INFO:teuthology.orchestra.run.smithi196.stdout: "poolstat_resend": 0, 2024-04-23T10:43:16.325 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_active": 0, 2024-04-23T10:43:16.325 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_send": 0, 2024-04-23T10:43:16.325 INFO:teuthology.orchestra.run.smithi196.stdout: "statfs_resend": 0, 2024-04-23T10:43:16.325 INFO:teuthology.orchestra.run.smithi196.stdout: "command_active": 0, 2024-04-23T10:43:16.325 INFO:teuthology.orchestra.run.smithi196.stdout: "command_send": 0, 2024-04-23T10:43:16.325 INFO:teuthology.orchestra.run.smithi196.stdout: "command_resend": 0, 2024-04-23T10:43:16.325 INFO:teuthology.orchestra.run.smithi196.stdout: "map_epoch": 111, 2024-04-23T10:43:16.326 INFO:teuthology.orchestra.run.smithi196.stdout: "map_full": 0, 2024-04-23T10:43:16.326 INFO:teuthology.orchestra.run.smithi196.stdout: "map_inc": 25, 2024-04-23T10:43:16.326 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_sessions": 8, 2024-04-23T10:43:16.326 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_open": 12, 2024-04-23T10:43:16.326 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_session_close": 4, 2024-04-23T10:43:16.326 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_laggy": 0, 2024-04-23T10:43:16.326 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_wr": 17, 2024-04-23T10:43:16.326 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_rd": 0, 2024-04-23T10:43:16.326 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_del": 2 2024-04-23T10:43:16.326 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:16.326 INFO:teuthology.orchestra.run.smithi196.stdout: "oft": { 2024-04-23T10:43:16.326 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_objs": 1, 2024-04-23T10:43:16.327 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_kv_pairs": 0, 2024-04-23T10:43:16.327 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_updates": 0, 2024-04-23T10:43:16.327 INFO:teuthology.orchestra.run.smithi196.stdout: "omap_total_removes": 0 2024-04-23T10:43:16.327 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:16.327 INFO:teuthology.orchestra.run.smithi196.stdout: "osd-slow-ops": { 2024-04-23T10:43:16.327 INFO:teuthology.orchestra.run.smithi196.stdout: "slow_ops_count": 118 2024-04-23T10:43:16.327 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:16.327 INFO:teuthology.orchestra.run.smithi196.stdout: "purge_queue": { 2024-04-23T10:43:16.327 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops": 0, 2024-04-23T10:43:16.327 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_ops_high_water": 0, 2024-04-23T10:43:16.327 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing": 0, 2024-04-23T10:43:16.327 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executing_high_water": 0, 2024-04-23T10:43:16.328 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed_ops": 0, 2024-04-23T10:43:16.328 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_executed": 0, 2024-04-23T10:43:16.328 INFO:teuthology.orchestra.run.smithi196.stdout: "pq_item_in_journal": 0 2024-04-23T10:43:16.328 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:16.328 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-23T10:43:16.328 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-04-23T10:43:16.328 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-04-23T10:43:16.328 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-04-23T10:43:16.328 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 2836, 2024-04-23T10:43:16.328 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 2051148, 2024-04-23T10:43:16.328 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-04-23T10:43:16.328 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 2836, 2024-04-23T10:43:16.329 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-04-23T10:43:16.329 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-04-23T10:43:16.329 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 2836, 2024-04-23T10:43:16.329 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 2051148, 2024-04-23T10:43:16.329 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-04-23T10:43:16.329 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:16.329 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:16.329 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:16.329 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-23T10:43:16.329 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:16.329 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_bytes": { 2024-04-23T10:43:16.329 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-04-23T10:43:16.329 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 104857600, 2024-04-23T10:43:16.330 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-04-23T10:43:16.330 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-04-23T10:43:16.330 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-04-23T10:43:16.330 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-04-23T10:43:16.330 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-04-23T10:43:16.330 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 48, 2024-04-23T10:43:16.330 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 13282, 2024-04-23T10:43:16.330 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 48, 2024-04-23T10:43:16.330 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 13282, 2024-04-23T10:43:16.330 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-04-23T10:43:16.330 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:16.330 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:16.331 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:16.331 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-23T10:43:16.331 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:16.331 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-objecter_ops": { 2024-04-23T10:43:16.331 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-04-23T10:43:16.331 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 1024, 2024-04-23T10:43:16.331 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-04-23T10:43:16.331 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-04-23T10:43:16.331 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-04-23T10:43:16.331 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-04-23T10:43:16.331 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-04-23T10:43:16.331 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 48, 2024-04-23T10:43:16.332 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 48, 2024-04-23T10:43:16.332 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 48, 2024-04-23T10:43:16.332 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 48, 2024-04-23T10:43:16.332 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-04-23T10:43:16.332 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:16.332 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:16.332 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:16.332 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-23T10:43:16.332 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:16.332 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle": { 2024-04-23T10:43:16.332 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-04-23T10:43:16.332 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-04-23T10:43:16.332 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-04-23T10:43:16.333 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 0, 2024-04-23T10:43:16.333 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 0, 2024-04-23T10:43:16.333 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-04-23T10:43:16.333 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 0, 2024-04-23T10:43:16.333 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-04-23T10:43:16.333 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-04-23T10:43:16.333 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 0, 2024-04-23T10:43:16.333 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 0, 2024-04-23T10:43:16.333 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-04-23T10:43:16.333 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:16.333 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:16.333 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:16.333 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-23T10:43:16.333 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-23T10:43:16.334 INFO:teuthology.orchestra.run.smithi196.stdout: "throttle-write_buf_throttle-0x55d8d6590500": { 2024-04-23T10:43:16.334 INFO:teuthology.orchestra.run.smithi196.stdout: "val": 0, 2024-04-23T10:43:16.334 INFO:teuthology.orchestra.run.smithi196.stdout: "max": 3758096384, 2024-04-23T10:43:16.334 INFO:teuthology.orchestra.run.smithi196.stdout: "get_started": 0, 2024-04-23T10:43:16.334 INFO:teuthology.orchestra.run.smithi196.stdout: "get": 9, 2024-04-23T10:43:16.334 INFO:teuthology.orchestra.run.smithi196.stdout: "get_sum": 2575, 2024-04-23T10:43:16.334 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_fail": 0, 2024-04-23T10:43:16.334 INFO:teuthology.orchestra.run.smithi196.stdout: "get_or_fail_success": 9, 2024-04-23T10:43:16.334 INFO:teuthology.orchestra.run.smithi196.stdout: "take": 0, 2024-04-23T10:43:16.334 INFO:teuthology.orchestra.run.smithi196.stdout: "take_sum": 0, 2024-04-23T10:43:16.334 INFO:teuthology.orchestra.run.smithi196.stdout: "put": 8, 2024-04-23T10:43:16.334 INFO:teuthology.orchestra.run.smithi196.stdout: "put_sum": 2575, 2024-04-23T10:43:16.334 INFO:teuthology.orchestra.run.smithi196.stdout: "wait": { 2024-04-23T10:43:16.335 INFO:teuthology.orchestra.run.smithi196.stdout: "avgcount": 0, 2024-04-23T10:43:16.335 INFO:teuthology.orchestra.run.smithi196.stdout: "sum": 0.000000000, 2024-04-23T10:43:16.335 INFO:teuthology.orchestra.run.smithi196.stdout: "avgtime": 0.000000000 2024-04-23T10:43:16.335 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-23T10:43:16.335 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-23T10:43:16.335 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-23T10:43:17.144 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-04-23T10:43:17.144 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-04-23T10:43:17.145 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-04-23T10:43:17.145 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=17 2024-04-23T10:43:17.145 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=6 2024-04-23T10:43:17.145 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-04-23T10:43:17.145 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-04-23T10:43:17.145 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-23T10:43:17.158 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-23T10:43:17.158 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-23T10:43:17.158 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T10:43:17.180 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-23T10:43:17.180 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T10:43:17.241 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T10:43:17.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.392+0000 7f8d133f7640 1 -- 172.21.15.97:0/2353327545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d0c1031b0 msgr2=0x7f8d0c103590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:43:17.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.392+0000 7f8d133f7640 1 --2- 172.21.15.97:0/2353327545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d0c1031b0 0x7f8d0c103590 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f8d04009900 tx=0x7f8d0402a480 comp rx=0 tx=0).stop 2024-04-23T10:43:17.394 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.392+0000 7f8d133f7640 1 -- 172.21.15.97:0/2353327545 shutdown_connections 2024-04-23T10:43:17.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.392+0000 7f8d133f7640 1 --2- 172.21.15.97:0/2353327545 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8d0c10c580 0x7f8d0c10e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:17.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.392+0000 7f8d133f7640 1 --2- 172.21.15.97:0/2353327545 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d0c103ad0 0x7f8d0c10bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:17.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.392+0000 7f8d133f7640 1 --2- 172.21.15.97:0/2353327545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d0c1031b0 0x7f8d0c103590 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:17.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.392+0000 7f8d133f7640 1 -- 172.21.15.97:0/2353327545 >> 172.21.15.97:0/2353327545 conn(0x7f8d0c0771d0 msgr2=0x7f8d0c0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:43:17.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.393+0000 7f8d133f7640 1 -- 172.21.15.97:0/2353327545 shutdown_connections 2024-04-23T10:43:17.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.393+0000 7f8d133f7640 1 -- 172.21.15.97:0/2353327545 wait complete. 2024-04-23T10:43:17.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.393+0000 7f8d133f7640 1 Processor -- start 2024-04-23T10:43:17.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.393+0000 7f8d133f7640 1 -- start start 2024-04-23T10:43:17.395 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.393+0000 7f8d133f7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8d0c1031b0 0x7f8d0c19e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:43:17.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.393+0000 7f8d133f7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d0c103ad0 0x7f8d0c19e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:43:17.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.393+0000 7f8d133f7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d0c10c580 0x7f8d0c1985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:43:17.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.393+0000 7f8d133f7640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8d0c1121d0 con 0x7f8d0c103ad0 2024-04-23T10:43:17.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.393+0000 7f8d133f7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8d0c112050 con 0x7f8d0c1031b0 2024-04-23T10:43:17.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.393+0000 7f8d133f7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f8d0c112350 con 0x7f8d0c10c580 2024-04-23T10:43:17.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.394+0000 7f8d10992640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d0c103ad0 0x7f8d0c19e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:43:17.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.394+0000 7f8d10992640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d0c103ad0 0x7f8d0c19e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40586/0 (socket says 172.21.15.97:40586) 2024-04-23T10:43:17.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.394+0000 7f8d10992640 1 -- 172.21.15.97:0/1151248210 learned_addr learned my addr 172.21.15.97:0/1151248210 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:43:17.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.394+0000 7f8d11193640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8d0c1031b0 0x7f8d0c19e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:43:17.396 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.394+0000 7f8d11994640 1 --2- 172.21.15.97:0/1151248210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d0c10c580 0x7f8d0c1985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:43:17.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.394+0000 7f8d10992640 1 -- 172.21.15.97:0/1151248210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d0c10c580 msgr2=0x7f8d0c1985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:43:17.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.394+0000 7f8d10992640 1 --2- 172.21.15.97:0/1151248210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d0c10c580 0x7f8d0c1985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:17.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.394+0000 7f8d10992640 1 -- 172.21.15.97:0/1151248210 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8d0c1031b0 msgr2=0x7f8d0c19e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:43:17.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.394+0000 7f8d10992640 1 --2- 172.21.15.97:0/1151248210 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8d0c1031b0 0x7f8d0c19e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:17.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.394+0000 7f8d10992640 1 -- 172.21.15.97:0/1151248210 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d0c198d60 con 0x7f8d0c103ad0 2024-04-23T10:43:17.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.394+0000 7f8d10992640 1 --2- 172.21.15.97:0/1151248210 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d0c103ad0 0x7f8d0c19e970 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f8cf400ea80 tx=0x7f8cf400ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:43:17.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.394+0000 7f8d0a7fc640 1 -- 172.21.15.97:0/1151248210 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf400ce40 con 0x7f8d0c103ad0 2024-04-23T10:43:17.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.394+0000 7f8d133f7640 1 -- 172.21.15.97:0/1151248210 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d0c199050 con 0x7f8d0c103ad0 2024-04-23T10:43:17.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.394+0000 7f8d0a7fc640 1 -- 172.21.15.97:0/1151248210 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cf4005020 con 0x7f8d0c103ad0 2024-04-23T10:43:17.397 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.394+0000 7f8d0a7fc640 1 -- 172.21.15.97:0/1151248210 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf40116d0 con 0x7f8d0c103ad0 2024-04-23T10:43:17.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.394+0000 7f8d133f7640 1 -- 172.21.15.97:0/1151248210 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d0c1a52f0 con 0x7f8d0c103ad0 2024-04-23T10:43:17.398 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.395+0000 7f8d133f7640 1 -- 172.21.15.97:0/1151248210 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d0c06add0 con 0x7f8d0c103ad0 2024-04-23T10:43:17.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.396+0000 7f8d0a7fc640 1 -- 172.21.15.97:0/1151248210 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8cf40054d0 con 0x7f8d0c103ad0 2024-04-23T10:43:17.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.397+0000 7f8d0a7fc640 1 --2- 172.21.15.97:0/1151248210 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f8cf80a3d20 0x7f8cf80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:43:17.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.397+0000 7f8d11193640 1 --2- 172.21.15.97:0/1151248210 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f8cf80a3d20 0x7f8cf80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:43:17.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.397+0000 7f8d0a7fc640 1 -- 172.21.15.97:0/1151248210 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8484+0+0 (secure 0 0 0) 0x7f8cf40d0a30 con 0x7f8d0c103ad0 2024-04-23T10:43:17.399 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.397+0000 7f8d11193640 1 --2- 172.21.15.97:0/1151248210 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f8cf80a3d20 0x7f8cf80a61e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8d04031440 tx=0x7f8d04005b50 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:43:17.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.402+0000 7f8d0a7fc640 1 -- 172.21.15.97:0/1151248210 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8cf4016030 con 0x7f8d0c103ad0 2024-04-23T10:43:17.580 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.581+0000 7f8d133f7640 1 -- 172.21.15.97:0/1151248210 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8d0c1076d0 con 0x7f8d0c103ad0 2024-04-23T10:43:17.581 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.582+0000 7f8d0a7fc640 1 -- 172.21.15.97:0/1151248210 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) ==== 74+0+23075 (secure 0 0 0) 0x7f8cf409caa0 con 0x7f8d0c103ad0 2024-04-23T10:43:17.582 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:43:17.582 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":111,"fsid":"9b22af88-0159-11ef-bc93-c7b262605968","created":"2024-04-23T10:16:05.983305+0000","modified":"2024-04-23T10:43:11.195576+0000","last_up_change":"2024-04-23T10:23:13.114761+0000","last_in_change":"2024-04-23T10:22:52.804880+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T10:19:25.559138+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T10:24:04.570049+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T10:24:05.861631+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":29,"snap_epoch":110,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"58cb995d-de35-4522-ac81-4982401b260d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6803","nonce":3031221802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6805","nonce":3031221802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6809","nonce":3031221802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6807","nonce":3031221802}]},"public_addr":"172.21.15.97:6803/3031221802","cluster_addr":"172.21.15.97:6805/3031221802","heartbeat_back_addr":"172.21.15.97:6809/3031221802","heartbeat_front_addr":"172.21.15.97:6807/3031221802","state":["exists","up"]},{"osd":1,"uuid":"b47bd8ee-1f21-4834-acf5-bb40f97ab3ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6801","nonce":1108870440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6803","nonce":1108870440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6807","nonce":1108870440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6805","nonce":1108870440}]},"public_addr":"172.21.15.183:6801/1108870440","cluster_addr":"172.21.15.183:6803/1108870440","heartbeat_back_addr":"172.21.15.183:6807/1108870440","heartbeat_front_addr":"172.21.15.183:6805/1108870440","state":["exists","up"]},{"osd":2,"uuid":"17c4eec3-6c47-4334-9b63-1d5ad9ec5988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6801","nonce":2877039731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6803","nonce":2877039731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6807","nonce":2877039731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6805","nonce":2877039731}]},"public_addr":"172.21.15.196:6801/2877039731","cluster_addr":"172.21.15.196:6803/2877039731","heartbeat_back_addr":"172.21.15.196:6807/2877039731","heartbeat_front_addr":"172.21.15.196:6805/2877039731","state":["exists","up"]},{"osd":3,"uuid":"3f731c38-2e45-4ca1-8dce-efc4a1b2166b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6811","nonce":4276311194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6813","nonce":4276311194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6817","nonce":4276311194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6815","nonce":4276311194}]},"public_addr":"172.21.15.97:6811/4276311194","cluster_addr":"172.21.15.97:6813/4276311194","heartbeat_back_addr":"172.21.15.97:6817/4276311194","heartbeat_front_addr":"172.21.15.97:6815/4276311194","state":["exists","up"]},{"osd":4,"uuid":"fdcafb14-9b82-4134-bd5b-79c0516f6cbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6809","nonce":63913916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6811","nonce":63913916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6815","nonce":63913916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6813","nonce":63913916}]},"public_addr":"172.21.15.183:6809/63913916","cluster_addr":"172.21.15.183:6811/63913916","heartbeat_back_addr":"172.21.15.183:6815/63913916","heartbeat_front_addr":"172.21.15.183:6813/63913916","state":["exists","up"]},{"osd":5,"uuid":"cdcc18d8-da3c-4012-bf12-876bb39e66eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6809","nonce":2804446896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6811","nonce":2804446896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6815","nonce":2804446896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6813","nonce":2804446896}]},"public_addr":"172.21.15.196:6809/2804446896","cluster_addr":"172.21.15.196:6811/2804446896","heartbeat_back_addr":"172.21.15.196:6815/2804446896","heartbeat_front_addr":"172.21.15.196:6813/2804446896","state":["exists","up"]},{"osd":6,"uuid":"071abb5c-2edf-4832-b8aa-3b80d20096fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6819","nonce":2080916705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6821","nonce":2080916705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6825","nonce":2080916705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6823","nonce":2080916705}]},"public_addr":"172.21.15.97:6819/2080916705","cluster_addr":"172.21.15.97:6821/2080916705","heartbeat_back_addr":"172.21.15.97:6825/2080916705","heartbeat_front_addr":"172.21.15.97:6823/2080916705","state":["exists","up"]},{"osd":7,"uuid":"9d54760b-66df-486b-8abb-017f3b64b4d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6817","nonce":1865873033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6819","nonce":1865873033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6823","nonce":1865873033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6821","nonce":1865873033}]},"public_addr":"172.21.15.183:6817/1865873033","cluster_addr":"172.21.15.183:6819/1865873033","heartbeat_back_addr":"172.21.15.183:6823/1865873033","heartbeat_front_addr":"172.21.15.183:6821/1865873033","state":["exists","up"]},{"osd":8,"uuid":"5cd3100a-bd6a-4be5-b5e9-0d0afc325193","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6817","nonce":4063932139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6819","nonce":4063932139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6823","nonce":4063932139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6821","nonce":4063932139}]},"public_addr":"172.21.15.196:6817/4063932139","cluster_addr":"172.21.15.196:6819/4063932139","heartbeat_back_addr":"172.21.15.196:6823/4063932139","heartbeat_front_addr":"172.21.15.196:6821/4063932139","state":["exists","up"]},{"osd":9,"uuid":"b6a5f4a1-4ecc-4f73-a2df-07551d9298ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6827","nonce":1887432474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6829","nonce":1887432474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6833","nonce":1887432474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6831","nonce":1887432474}]},"public_addr":"172.21.15.97:6827/1887432474","cluster_addr":"172.21.15.97:6829/1887432474","heartbeat_back_addr":"172.21.15.97:6833/1887432474","heartbeat_front_addr":"172.21.15.97:6831/1887432474","state":["exists","up"]},{"osd":10,"uuid":"86f59816-1ea6-4ae9-8796-cf5fc4831064","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6825","nonce":799434647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6827","nonce":799434647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6831","nonce":799434647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6829","nonce":799434647}]},"public_addr":"172.21.15.183:6825/799434647","cluster_addr":"172.21.15.183:6827/799434647","heartbeat_back_addr":"172.21.15.183:6831/799434647","heartbeat_front_addr":"172.21.15.183:6829/799434647","state":["exists","up"]},{"osd":11,"uuid":"95d64a89-a9f8-4c7a-8c0d-da7fb26cfff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6825","nonce":1873969467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6827","nonce":1873969467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6831","nonce":1873969467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6829","nonce":1873969467}]},"public_addr":"172.21.15.196:6825/1873969467","cluster_addr":"172.21.15.196:6827/1873969467","heartbeat_back_addr":"172.21.15.196:6831/1873969467","heartbeat_front_addr":"172.21.15.196:6829/1873969467","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:18:33.721156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:18:57.586709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:19:22.069458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:19:46.451597+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:20:11.561019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:20:36.298672+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:21:01.661804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:21:27.324128+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:21:52.898197+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:22:19.614326+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:22:45.154917+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:23:11.222612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/233335470":"2024-04-24T10:26:11.109535+0000","172.21.15.97:6800/3149043749":"2024-04-24T10:26:11.109535+0000","172.21.15.97:0/338277142":"2024-04-24T10:26:11.109535+0000","172.21.15.97:0/3866016040":"2024-04-24T10:16:59.468237+0000","172.21.15.97:6801/3297166858":"2024-04-24T10:16:59.468237+0000","172.21.15.97:0/3804743548":"2024-04-24T10:16:59.468237+0000","172.21.15.97:6800/3706463017":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/3715079438":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/597405418":"2024-04-24T10:26:11.109535+0000","172.21.15.97:0/786806834":"2024-04-24T10:16:59.468237+0000","172.21.15.97:6801/3706463017":"2024-04-24T10:16:33.128917+0000","172.21.15.97:6801/3149043749":"2024-04-24T10:26:11.109535+0000","172.21.15.97:0/3294175554":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/1449136760":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/3198144747":"2024-04-24T10:26:11.109535+0000","172.21.15.97:6800/3297166858":"2024-04-24T10:16:59.468237+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":16,"length":1},{"begin":29,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T10:43:17.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.584+0000 7f8d133f7640 1 -- 172.21.15.97:0/1151248210 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f8cf80a3d20 msgr2=0x7f8cf80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:43:17.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.584+0000 7f8d133f7640 1 --2- 172.21.15.97:0/1151248210 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f8cf80a3d20 0x7f8cf80a61e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8d04031440 tx=0x7f8d04005b50 comp rx=0 tx=0).stop 2024-04-23T10:43:17.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.584+0000 7f8d133f7640 1 -- 172.21.15.97:0/1151248210 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d0c103ad0 msgr2=0x7f8d0c19e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:43:17.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.584+0000 7f8d133f7640 1 --2- 172.21.15.97:0/1151248210 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d0c103ad0 0x7f8d0c19e970 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f8cf400ea80 tx=0x7f8cf400ef40 comp rx=0 tx=0).stop 2024-04-23T10:43:17.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.584+0000 7f8d133f7640 1 -- 172.21.15.97:0/1151248210 shutdown_connections 2024-04-23T10:43:17.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.584+0000 7f8d133f7640 1 --2- 172.21.15.97:0/1151248210 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f8cf80a3d20 0x7f8cf80a61e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:17.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.584+0000 7f8d133f7640 1 --2- 172.21.15.97:0/1151248210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d0c10c580 0x7f8d0c1985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:17.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.584+0000 7f8d133f7640 1 --2- 172.21.15.97:0/1151248210 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d0c103ad0 0x7f8d0c19e970 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:17.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.584+0000 7f8d133f7640 1 --2- 172.21.15.97:0/1151248210 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8d0c1031b0 0x7f8d0c19e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:17.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.584+0000 7f8d133f7640 1 -- 172.21.15.97:0/1151248210 >> 172.21.15.97:0/1151248210 conn(0x7f8d0c0771d0 msgr2=0x7f8d0c0fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:43:17.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.584+0000 7f8d133f7640 1 -- 172.21.15.97:0/1151248210 shutdown_connections 2024-04-23T10:43:17.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.584+0000 7f8d133f7640 1 -- 172.21.15.97:0/1151248210 wait complete. 2024-04-23T10:43:17.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:43:17 smithi196 ceph-mon[32919]: pgmap v537: 97 pgs: 97 active+clean; 4.5 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 343 B/s wr, 11 op/s 2024-04-23T10:43:17.593 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:43:17 smithi097 ceph-mon[29197]: pgmap v537: 97 pgs: 97 active+clean; 4.5 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 343 B/s wr, 11 op/s 2024-04-23T10:43:17.594 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T10:19:25.559138+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T10:24:04.570049+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T10:24:05.861631+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 29, 'snap_epoch': 110, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T10:43:17.595 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T10:43:17.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:43:17 smithi183 ceph-mon[32317]: pgmap v537: 97 pgs: 97 active+clean; 4.5 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 343 B/s wr, 11 op/s 2024-04-23T10:43:17.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.698+0000 7fb062d76640 1 -- 172.21.15.97:0/1494989771 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb05c101090 msgr2=0x7fb05c101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:43:17.700 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.698+0000 7fb062d76640 1 --2- 172.21.15.97:0/1494989771 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb05c101090 0x7fb05c101470 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fb058009900 tx=0x7fb05802a410 comp rx=0 tx=0).stop 2024-04-23T10:43:17.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.699+0000 7fb062d76640 1 -- 172.21.15.97:0/1494989771 shutdown_connections 2024-04-23T10:43:17.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.699+0000 7fb062d76640 1 --2- 172.21.15.97:0/1494989771 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb05c106160 0x7fb05c10dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:17.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.699+0000 7fb062d76640 1 --2- 172.21.15.97:0/1494989771 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb05c101a40 0x7fb05c105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:17.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.699+0000 7fb062d76640 1 --2- 172.21.15.97:0/1494989771 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb05c101090 0x7fb05c101470 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:17.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.699+0000 7fb062d76640 1 -- 172.21.15.97:0/1494989771 >> 172.21.15.97:0/1494989771 conn(0x7fb05c077f50 msgr2=0x7fb05c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:43:17.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.699+0000 7fb062d76640 1 -- 172.21.15.97:0/1494989771 shutdown_connections 2024-04-23T10:43:17.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.699+0000 7fb062d76640 1 -- 172.21.15.97:0/1494989771 wait complete. 2024-04-23T10:43:17.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.699+0000 7fb062d76640 1 Processor -- start 2024-04-23T10:43:17.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.699+0000 7fb062d76640 1 -- start start 2024-04-23T10:43:17.701 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.699+0000 7fb062d76640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb05c101090 0x7fb05c198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:43:17.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.699+0000 7fb062d76640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb05c106160 0x7fb05c198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:43:17.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.699+0000 7fb062d76640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb05c19cff0 0x7fb05c19d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:43:17.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.699+0000 7fb062d76640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb05c1110a0 con 0x7fb05c106160 2024-04-23T10:43:17.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.699+0000 7fb062d76640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb05c110f20 con 0x7fb05c101090 2024-04-23T10:43:17.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.699+0000 7fb062d76640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fb05c111220 con 0x7fb05c19cff0 2024-04-23T10:43:17.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.699+0000 7fb053fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb05c106160 0x7fb05c198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:43:17.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.699+0000 7fb053fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb05c106160 0x7fb05c198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40598/0 (socket says 172.21.15.97:40598) 2024-04-23T10:43:17.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.699+0000 7fb053fff640 1 -- 172.21.15.97:0/3853199937 learned_addr learned my addr 172.21.15.97:0/3853199937 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:43:17.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.699+0000 7fb060b12640 1 --2- 172.21.15.97:0/3853199937 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb05c101090 0x7fb05c198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:43:17.702 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.700+0000 7fb061313640 1 --2- 172.21.15.97:0/3853199937 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb05c19cff0 0x7fb05c19d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:43:17.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.700+0000 7fb053fff640 1 -- 172.21.15.97:0/3853199937 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb05c19cff0 msgr2=0x7fb05c19d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:43:17.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.700+0000 7fb053fff640 1 --2- 172.21.15.97:0/3853199937 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb05c19cff0 0x7fb05c19d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:17.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.700+0000 7fb053fff640 1 -- 172.21.15.97:0/3853199937 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb05c101090 msgr2=0x7fb05c198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:43:17.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.700+0000 7fb053fff640 1 --2- 172.21.15.97:0/3853199937 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb05c101090 0x7fb05c198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:17.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.700+0000 7fb053fff640 1 -- 172.21.15.97:0/3853199937 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb05c19dae0 con 0x7fb05c106160 2024-04-23T10:43:17.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.700+0000 7fb053fff640 1 --2- 172.21.15.97:0/3853199937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb05c106160 0x7fb05c198c70 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fb04c00d770 tx=0x7fb04c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:43:17.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.700+0000 7fb051ffb640 1 -- 172.21.15.97:0/3853199937 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04c017070 con 0x7fb05c106160 2024-04-23T10:43:17.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.700+0000 7fb051ffb640 1 -- 172.21.15.97:0/3853199937 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb04c009c30 con 0x7fb05c106160 2024-04-23T10:43:17.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.700+0000 7fb051ffb640 1 -- 172.21.15.97:0/3853199937 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04c00f620 con 0x7fb05c106160 2024-04-23T10:43:17.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.700+0000 7fb062d76640 1 -- 172.21.15.97:0/3853199937 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb05c19ddd0 con 0x7fb05c106160 2024-04-23T10:43:17.703 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.700+0000 7fb062d76640 1 -- 172.21.15.97:0/3853199937 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb05c1a55c0 con 0x7fb05c106160 2024-04-23T10:43:17.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.701+0000 7fb062d76640 1 -- 172.21.15.97:0/3853199937 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb05c068d70 con 0x7fb05c106160 2024-04-23T10:43:17.704 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.702+0000 7fb051ffb640 1 -- 172.21.15.97:0/3853199937 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb04c01a070 con 0x7fb05c106160 2024-04-23T10:43:17.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.703+0000 7fb051ffb640 1 --2- 172.21.15.97:0/3853199937 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fb0440a3d20 0x7fb0440a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:43:17.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.703+0000 7fb060b12640 1 --2- 172.21.15.97:0/3853199937 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fb0440a3d20 0x7fb0440a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:43:17.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.703+0000 7fb051ffb640 1 -- 172.21.15.97:0/3853199937 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8484+0+0 (secure 0 0 0) 0x7fb04c0d02a0 con 0x7fb05c106160 2024-04-23T10:43:17.705 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.704+0000 7fb060b12640 1 --2- 172.21.15.97:0/3853199937 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fb0440a3d20 0x7fb0440a61e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb058031440 tx=0x7fb058005b50 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:43:17.706 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.708+0000 7fb051ffb640 1 -- 172.21.15.97:0/3853199937 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb04c009de0 con 0x7fb05c106160 2024-04-23T10:43:17.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.887+0000 7fb062d76640 1 -- 172.21.15.97:0/3853199937 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb05c101c20 con 0x7fb05c106160 2024-04-23T10:43:17.887 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.888+0000 7fb051ffb640 1 -- 172.21.15.97:0/3853199937 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v111) ==== 93+0+10 (secure 0 0 0) 0x7fb04c09c310 con 0x7fb05c106160 2024-04-23T10:43:17.887 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 1 2024-04-23T10:43:17.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.890+0000 7fb062d76640 1 -- 172.21.15.97:0/3853199937 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fb0440a3d20 msgr2=0x7fb0440a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:43:17.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.890+0000 7fb062d76640 1 --2- 172.21.15.97:0/3853199937 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fb0440a3d20 0x7fb0440a61e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb058031440 tx=0x7fb058005b50 comp rx=0 tx=0).stop 2024-04-23T10:43:17.889 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.890+0000 7fb062d76640 1 -- 172.21.15.97:0/3853199937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb05c106160 msgr2=0x7fb05c198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:43:17.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.890+0000 7fb062d76640 1 --2- 172.21.15.97:0/3853199937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb05c106160 0x7fb05c198c70 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fb04c00d770 tx=0x7fb04c00dc30 comp rx=0 tx=0).stop 2024-04-23T10:43:17.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.890+0000 7fb062d76640 1 -- 172.21.15.97:0/3853199937 shutdown_connections 2024-04-23T10:43:17.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.890+0000 7fb062d76640 1 --2- 172.21.15.97:0/3853199937 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fb0440a3d20 0x7fb0440a61e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:17.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.890+0000 7fb062d76640 1 --2- 172.21.15.97:0/3853199937 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb05c19cff0 0x7fb05c19d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:17.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.890+0000 7fb062d76640 1 --2- 172.21.15.97:0/3853199937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb05c106160 0x7fb05c198c70 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:17.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.890+0000 7fb062d76640 1 --2- 172.21.15.97:0/3853199937 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb05c101090 0x7fb05c198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:17.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.890+0000 7fb062d76640 1 -- 172.21.15.97:0/3853199937 >> 172.21.15.97:0/3853199937 conn(0x7fb05c077f50 msgr2=0x7fb05c104ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:43:17.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.890+0000 7fb062d76640 1 -- 172.21.15.97:0/3853199937 shutdown_connections 2024-04-23T10:43:17.890 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:17.890+0000 7fb062d76640 1 -- 172.21.15.97:0/3853199937 wait complete. 2024-04-23T10:43:17.900 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T10:43:18.003 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.002+0000 7f2888b36640 1 -- 172.21.15.97:0/1057670929 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f288410c530 msgr2=0x7f288410e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:43:18.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.002+0000 7f2888b36640 1 --2- 172.21.15.97:0/1057670929 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f288410c530 0x7f288410e9d0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f287c009ca0 tx=0x7f287c02a9e0 comp rx=0 tx=0).stop 2024-04-23T10:43:18.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.002+0000 7f2888b36640 1 -- 172.21.15.97:0/1057670929 shutdown_connections 2024-04-23T10:43:18.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.002+0000 7f2888b36640 1 --2- 172.21.15.97:0/1057670929 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f288410c530 0x7f288410e9d0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:18.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.002+0000 7f2888b36640 1 --2- 172.21.15.97:0/1057670929 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2884103b10 0x7f288410bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:18.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.002+0000 7f2888b36640 1 --2- 172.21.15.97:0/1057670929 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f28841031f0 0x7f28841035d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:18.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.002+0000 7f2888b36640 1 -- 172.21.15.97:0/1057670929 >> 172.21.15.97:0/1057670929 conn(0x7f28840771d0 msgr2=0x7f28840775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:43:18.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.002+0000 7f2888b36640 1 -- 172.21.15.97:0/1057670929 shutdown_connections 2024-04-23T10:43:18.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.002+0000 7f2888b36640 1 -- 172.21.15.97:0/1057670929 wait complete. 2024-04-23T10:43:18.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.003+0000 7f2888b36640 1 Processor -- start 2024-04-23T10:43:18.004 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.003+0000 7f2888b36640 1 -- start start 2024-04-23T10:43:18.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.003+0000 7f2888b36640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f28841031f0 0x7f288419e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:43:18.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.003+0000 7f2888b36640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2884103b10 0x7f288419e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:43:18.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.003+0000 7f2888b36640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f288410c530 0x7f2884198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:43:18.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.003+0000 7f2888b36640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2884112240 con 0x7f288410c530 2024-04-23T10:43:18.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.003+0000 7f2888b36640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f28841120c0 con 0x7f28841031f0 2024-04-23T10:43:18.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.003+0000 7f2888b36640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f28841123c0 con 0x7f2884103b10 2024-04-23T10:43:18.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.003+0000 7f2882d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f288410c530 0x7f2884198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:43:18.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.003+0000 7f2882d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f288410c530 0x7f2884198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40616/0 (socket says 172.21.15.97:40616) 2024-04-23T10:43:18.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.003+0000 7f2882d9d640 1 -- 172.21.15.97:0/1705376590 learned_addr learned my addr 172.21.15.97:0/1705376590 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:43:18.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.003+0000 7f288259c640 1 --2- 172.21.15.97:0/1705376590 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f28841031f0 0x7f288419e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:43:18.005 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.003+0000 7f287bfff640 1 --2- 172.21.15.97:0/1705376590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2884103b10 0x7f288419e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:43:18.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.003+0000 7f2882d9d640 1 -- 172.21.15.97:0/1705376590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2884103b10 msgr2=0x7f288419e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:43:18.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.003+0000 7f2882d9d640 1 --2- 172.21.15.97:0/1705376590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2884103b10 0x7f288419e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:18.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.003+0000 7f2882d9d640 1 -- 172.21.15.97:0/1705376590 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f28841031f0 msgr2=0x7f288419e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:43:18.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.003+0000 7f2882d9d640 1 --2- 172.21.15.97:0/1705376590 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f28841031f0 0x7f288419e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:18.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.003+0000 7f2882d9d640 1 -- 172.21.15.97:0/1705376590 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2884198d80 con 0x7f288410c530 2024-04-23T10:43:18.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.003+0000 7f287bfff640 1 --2- 172.21.15.97:0/1705376590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2884103b10 0x7f288419e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:43:18.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.004+0000 7f2882d9d640 1 --2- 172.21.15.97:0/1705376590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f288410c530 0x7f2884198600 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f287c009dc0 tx=0x7f287c02e5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:43:18.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.004+0000 7f287b7fe640 1 -- 172.21.15.97:0/1705376590 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f287c033040 con 0x7f288410c530 2024-04-23T10:43:18.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.004+0000 7f2888b36640 1 -- 172.21.15.97:0/1705376590 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2884199010 con 0x7f288410c530 2024-04-23T10:43:18.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.004+0000 7f287b7fe640 1 -- 172.21.15.97:0/1705376590 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f287c038b00 con 0x7f288410c530 2024-04-23T10:43:18.006 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.004+0000 7f287b7fe640 1 -- 172.21.15.97:0/1705376590 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f287c043a00 con 0x7f288410c530 2024-04-23T10:43:18.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.004+0000 7f2888b36640 1 -- 172.21.15.97:0/1705376590 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28841a5310 con 0x7f288410c530 2024-04-23T10:43:18.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.005+0000 7f2888b36640 1 -- 172.21.15.97:0/1705376590 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28840691c0 con 0x7f288410c530 2024-04-23T10:43:18.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.006+0000 7f287b7fe640 1 -- 172.21.15.97:0/1705376590 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f287c036070 con 0x7f288410c530 2024-04-23T10:43:18.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.006+0000 7f287b7fe640 1 --2- 172.21.15.97:0/1705376590 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f28700a3d20 0x7f28700a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:43:18.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.007+0000 7f288259c640 1 --2- 172.21.15.97:0/1705376590 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f28700a3d20 0x7f28700a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:43:18.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.007+0000 7f287b7fe640 1 -- 172.21.15.97:0/1705376590 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8484+0+0 (secure 0 0 0) 0x7f287c0f1930 con 0x7f288410c530 2024-04-23T10:43:18.008 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.007+0000 7f288259c640 1 --2- 172.21.15.97:0/1705376590 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f28700a3d20 0x7f28700a61e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2874009890 tx=0x7f2874008040 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:43:18.010 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.011+0000 7f287b7fe640 1 -- 172.21.15.97:0/1705376590 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f287c03e070 con 0x7f288410c530 2024-04-23T10:43:18.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.187+0000 7f2888b36640 1 -- 172.21.15.97:0/1705376590 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f288402d5c0 con 0x7f288410c530 2024-04-23T10:43:18.186 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.187+0000 7f287b7fe640 1 -- 172.21.15.97:0/1705376590 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v111) ==== 104+0+11 (secure 0 0 0) 0x7f287c0bdff0 con 0x7f288410c530 2024-04-23T10:43:18.186 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 32 2024-04-23T10:43:18.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.189+0000 7f2888b36640 1 -- 172.21.15.97:0/1705376590 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f28700a3d20 msgr2=0x7f28700a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:43:18.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.189+0000 7f2888b36640 1 --2- 172.21.15.97:0/1705376590 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f28700a3d20 0x7f28700a61e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2874009890 tx=0x7f2874008040 comp rx=0 tx=0).stop 2024-04-23T10:43:18.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.189+0000 7f2888b36640 1 -- 172.21.15.97:0/1705376590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f288410c530 msgr2=0x7f2884198600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:43:18.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.189+0000 7f2888b36640 1 --2- 172.21.15.97:0/1705376590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f288410c530 0x7f2884198600 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f287c009dc0 tx=0x7f287c02e5c0 comp rx=0 tx=0).stop 2024-04-23T10:43:18.188 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.189+0000 7f2888b36640 1 -- 172.21.15.97:0/1705376590 shutdown_connections 2024-04-23T10:43:18.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.189+0000 7f2888b36640 1 --2- 172.21.15.97:0/1705376590 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f28700a3d20 0x7f28700a61e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:18.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.189+0000 7f2888b36640 1 --2- 172.21.15.97:0/1705376590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f288410c530 0x7f2884198600 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:18.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.189+0000 7f2888b36640 1 --2- 172.21.15.97:0/1705376590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2884103b10 0x7f288419e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:18.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.189+0000 7f2888b36640 1 --2- 172.21.15.97:0/1705376590 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f28841031f0 0x7f288419e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:18.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.189+0000 7f2888b36640 1 -- 172.21.15.97:0/1705376590 >> 172.21.15.97:0/1705376590 conn(0x7f28840771d0 msgr2=0x7f28840ff090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:43:18.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.189+0000 7f2888b36640 1 -- 172.21.15.97:0/1705376590 shutdown_connections 2024-04-23T10:43:18.189 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.189+0000 7f2888b36640 1 -- 172.21.15.97:0/1705376590 wait complete. 2024-04-23T10:43:18.199 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T10:43:18.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.302+0000 7f4890a31640 1 -- 172.21.15.97:0/3206473894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f488c105e20 msgr2=0x7f488c106200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:43:18.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.302+0000 7f4890a31640 1 --2- 172.21.15.97:0/3206473894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f488c105e20 0x7f488c106200 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f487c009930 tx=0x7f487c02a420 comp rx=0 tx=0).stop 2024-04-23T10:43:18.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.302+0000 7f4890a31640 1 -- 172.21.15.97:0/3206473894 shutdown_connections 2024-04-23T10:43:18.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.302+0000 7f4890a31640 1 --2- 172.21.15.97:0/3206473894 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f488c102350 0x7f488c10dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:18.304 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.302+0000 7f4890a31640 1 --2- 172.21.15.97:0/3206473894 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f488c101990 0x7f488c101e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:18.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.302+0000 7f4890a31640 1 --2- 172.21.15.97:0/3206473894 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f488c105e20 0x7f488c106200 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:18.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.302+0000 7f4890a31640 1 -- 172.21.15.97:0/3206473894 >> 172.21.15.97:0/3206473894 conn(0x7f488c0760e0 msgr2=0x7f488c0764f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:43:18.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.302+0000 7f4890a31640 1 -- 172.21.15.97:0/3206473894 shutdown_connections 2024-04-23T10:43:18.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.302+0000 7f4890a31640 1 -- 172.21.15.97:0/3206473894 wait complete. 2024-04-23T10:43:18.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.303+0000 7f4890a31640 1 Processor -- start 2024-04-23T10:43:18.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.303+0000 7f4890a31640 1 -- start start 2024-04-23T10:43:18.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.303+0000 7f4890a31640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f488c101990 0x7f488c198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:43:18.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.303+0000 7f4890a31640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f488c102350 0x7f488c198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:43:18.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.303+0000 7f4890a31640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f488c105e20 0x7f488c19d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:43:18.305 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.303+0000 7f4890a31640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f488c112200 con 0x7f488c105e20 2024-04-23T10:43:18.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.303+0000 7f4890a31640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f488c112080 con 0x7f488c101990 2024-04-23T10:43:18.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.303+0000 7f4890a31640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f488c112380 con 0x7f488c102350 2024-04-23T10:43:18.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.303+0000 7f488ad9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f488c105e20 0x7f488c19d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:43:18.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.303+0000 7f488ad9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f488c105e20 0x7f488c19d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40636/0 (socket says 172.21.15.97:40636) 2024-04-23T10:43:18.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.303+0000 7f488ad9d640 1 -- 172.21.15.97:0/3178664736 learned_addr learned my addr 172.21.15.97:0/3178664736 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:43:18.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.304+0000 7f488a59c640 1 --2- 172.21.15.97:0/3178664736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f488c101990 0x7f488c198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:43:18.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.304+0000 7f4889d9b640 1 --2- 172.21.15.97:0/3178664736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f488c102350 0x7f488c198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:43:18.306 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.304+0000 7f488ad9d640 1 -- 172.21.15.97:0/3178664736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f488c102350 msgr2=0x7f488c198ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:43:18.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.304+0000 7f488ad9d640 1 --2- 172.21.15.97:0/3178664736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f488c102350 0x7f488c198ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:18.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.304+0000 7f488ad9d640 1 -- 172.21.15.97:0/3178664736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f488c101990 msgr2=0x7f488c198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:43:18.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.304+0000 7f488ad9d640 1 --2- 172.21.15.97:0/3178664736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f488c101990 0x7f488c198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:18.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.304+0000 7f488ad9d640 1 -- 172.21.15.97:0/3178664736 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f488c19d750 con 0x7f488c105e20 2024-04-23T10:43:18.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.304+0000 7f488a59c640 1 --2- 172.21.15.97:0/3178664736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f488c101990 0x7f488c198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:43:18.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.304+0000 7f488ad9d640 1 --2- 172.21.15.97:0/3178664736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f488c105e20 0x7f488c19d020 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f488400b970 tx=0x7f488400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:43:18.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.304+0000 7f487b7fe640 1 -- 172.21.15.97:0/3178664736 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4884019070 con 0x7f488c105e20 2024-04-23T10:43:18.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.304+0000 7f4890a31640 1 -- 172.21.15.97:0/3178664736 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f488c19da40 con 0x7f488c105e20 2024-04-23T10:43:18.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.304+0000 7f487b7fe640 1 -- 172.21.15.97:0/3178664736 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4884014440 con 0x7f488c105e20 2024-04-23T10:43:18.307 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.304+0000 7f487b7fe640 1 -- 172.21.15.97:0/3178664736 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48840075d0 con 0x7f488c105e20 2024-04-23T10:43:18.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.304+0000 7f4890a31640 1 -- 172.21.15.97:0/3178664736 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f488c1a52d0 con 0x7f488c105e20 2024-04-23T10:43:18.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.305+0000 7f4890a31640 1 -- 172.21.15.97:0/3178664736 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f488c0691c0 con 0x7f488c105e20 2024-04-23T10:43:18.308 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.306+0000 7f487b7fe640 1 -- 172.21.15.97:0/3178664736 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f488401c070 con 0x7f488c105e20 2024-04-23T10:43:18.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.307+0000 7f487b7fe640 1 --2- 172.21.15.97:0/3178664736 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f48740a3d20 0x7f48740a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:43:18.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.307+0000 7f488a59c640 1 --2- 172.21.15.97:0/3178664736 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f48740a3d20 0x7f48740a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:43:18.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.307+0000 7f487b7fe640 1 -- 172.21.15.97:0/3178664736 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8484+0+0 (secure 0 0 0) 0x7f48840cfd30 con 0x7f488c105e20 2024-04-23T10:43:18.309 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.307+0000 7f488a59c640 1 --2- 172.21.15.97:0/3178664736 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f48740a3d20 0x7f48740a61e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f487c006010 tx=0x7f487c005a20 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:43:18.310 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.311+0000 7f487b7fe640 1 -- 172.21.15.97:0/3178664736 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f488409c500 con 0x7f488c105e20 2024-04-23T10:43:18.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.491+0000 7f4890a31640 1 -- 172.21.15.97:0/3178664736 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f488c108da0 con 0x7f488c105e20 2024-04-23T10:43:18.490 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.491+0000 7f487b7fe640 1 -- 172.21.15.97:0/3178664736 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v111) ==== 100+0+11 (secure 0 0 0) 0x7f488409be00 con 0x7f488c105e20 2024-04-23T10:43:18.490 INFO:teuthology.orchestra.run.smithi097.stdout:pg_num: 64 2024-04-23T10:43:18.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.493+0000 7f4890a31640 1 -- 172.21.15.97:0/3178664736 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f48740a3d20 msgr2=0x7f48740a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:43:18.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.493+0000 7f4890a31640 1 --2- 172.21.15.97:0/3178664736 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f48740a3d20 0x7f48740a61e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f487c006010 tx=0x7f487c005a20 comp rx=0 tx=0).stop 2024-04-23T10:43:18.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.493+0000 7f4890a31640 1 -- 172.21.15.97:0/3178664736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f488c105e20 msgr2=0x7f488c19d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:43:18.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.493+0000 7f4890a31640 1 --2- 172.21.15.97:0/3178664736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f488c105e20 0x7f488c19d020 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f488400b970 tx=0x7f488400be30 comp rx=0 tx=0).stop 2024-04-23T10:43:18.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.493+0000 7f4890a31640 1 -- 172.21.15.97:0/3178664736 shutdown_connections 2024-04-23T10:43:18.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.493+0000 7f4890a31640 1 --2- 172.21.15.97:0/3178664736 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f48740a3d20 0x7f48740a61e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:18.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.493+0000 7f4890a31640 1 --2- 172.21.15.97:0/3178664736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f488c105e20 0x7f488c19d020 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:18.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.493+0000 7f4890a31640 1 --2- 172.21.15.97:0/3178664736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f488c102350 0x7f488c198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:18.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.493+0000 7f4890a31640 1 --2- 172.21.15.97:0/3178664736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f488c101990 0x7f488c198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:18.493 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.494+0000 7f4890a31640 1 -- 172.21.15.97:0/3178664736 >> 172.21.15.97:0/3178664736 conn(0x7f488c0760e0 msgr2=0x7f488c0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:43:18.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.494+0000 7f4890a31640 1 -- 172.21.15.97:0/3178664736 shutdown_connections 2024-04-23T10:43:18.494 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.494+0000 7f4890a31640 1 -- 172.21.15.97:0/3178664736 wait complete. 2024-04-23T10:43:18.502 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:43:18 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/1151248210' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T10:43:18.502 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:43:18 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/3853199937' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T10:43:18.503 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:43:18 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/1705376590' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T10:43:18.504 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-23T10:43:18.593 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:43:18 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/1151248210' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T10:43:18.594 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:43:18 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/3853199937' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T10:43:18.594 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:43:18 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/1705376590' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T10:43:18.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.604+0000 7f0efc35d640 1 -- 172.21.15.97:0/911881177 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ef4101a30 msgr2=0x7f0ef4105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:43:18.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.604+0000 7f0efc35d640 1 --2- 172.21.15.97:0/911881177 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ef4101a30 0x7f0ef4105a70 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4009900 tx=0x7f0ee402a410 comp rx=0 tx=0).stop 2024-04-23T10:43:18.606 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.604+0000 7f0efc35d640 1 -- 172.21.15.97:0/911881177 shutdown_connections 2024-04-23T10:43:18.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.604+0000 7f0efc35d640 1 --2- 172.21.15.97:0/911881177 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ef4106150 0x7f0ef410dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:18.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.604+0000 7f0efc35d640 1 --2- 172.21.15.97:0/911881177 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ef4101a30 0x7f0ef4105a70 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:18.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.604+0000 7f0efc35d640 1 --2- 172.21.15.97:0/911881177 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ef4101080 0x7f0ef4101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:18.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.604+0000 7f0efc35d640 1 -- 172.21.15.97:0/911881177 >> 172.21.15.97:0/911881177 conn(0x7f0ef40771d0 msgr2=0x7f0ef40775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:43:18.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.604+0000 7f0efc35d640 1 -- 172.21.15.97:0/911881177 shutdown_connections 2024-04-23T10:43:18.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.604+0000 7f0efc35d640 1 -- 172.21.15.97:0/911881177 wait complete. 2024-04-23T10:43:18.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.604+0000 7f0efc35d640 1 Processor -- start 2024-04-23T10:43:18.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.604+0000 7f0efc35d640 1 -- start start 2024-04-23T10:43:18.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.605+0000 7f0efc35d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ef4101080 0x7f0ef4198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:43:18.607 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.605+0000 7f0efc35d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ef4106150 0x7f0ef4198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:43:18.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.605+0000 7f0efc35d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ef419cfd0 0x7f0ef419d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:43:18.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.605+0000 7f0efc35d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef41121d0 con 0x7f0ef4106150 2024-04-23T10:43:18.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.605+0000 7f0efc35d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef4112050 con 0x7f0ef419cfd0 2024-04-23T10:43:18.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.605+0000 7f0efc35d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef4112350 con 0x7f0ef4101080 2024-04-23T10:43:18.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.605+0000 7f0ef98f8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ef4106150 0x7f0ef4198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:43:18.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.605+0000 7f0ef98f8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ef4106150 0x7f0ef4198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40662/0 (socket says 172.21.15.97:40662) 2024-04-23T10:43:18.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.605+0000 7f0ef98f8640 1 -- 172.21.15.97:0/2232224382 learned_addr learned my addr 172.21.15.97:0/2232224382 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:43:18.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.605+0000 7f0efa0f9640 1 --2- 172.21.15.97:0/2232224382 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ef4101080 0x7f0ef4198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:43:18.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.605+0000 7f0efa8fa640 1 --2- 172.21.15.97:0/2232224382 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ef419cfd0 0x7f0ef419d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:43:18.608 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.605+0000 7f0ef98f8640 1 -- 172.21.15.97:0/2232224382 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ef4101080 msgr2=0x7f0ef4198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:43:18.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.605+0000 7f0ef98f8640 1 --2- 172.21.15.97:0/2232224382 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ef4101080 0x7f0ef4198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:18.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.605+0000 7f0ef98f8640 1 -- 172.21.15.97:0/2232224382 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ef419cfd0 msgr2=0x7f0ef419d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:43:18.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.605+0000 7f0ef98f8640 1 --2- 172.21.15.97:0/2232224382 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ef419cfd0 0x7f0ef419d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:18.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.605+0000 7f0ef98f8640 1 -- 172.21.15.97:0/2232224382 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ef419dac0 con 0x7f0ef4106150 2024-04-23T10:43:18.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.605+0000 7f0ef98f8640 1 --2- 172.21.15.97:0/2232224382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ef4106150 0x7f0ef4198c50 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4032fd0 tx=0x7f0ee4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:43:18.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.605+0000 7f0eeb7fe640 1 -- 172.21.15.97:0/2232224382 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee403f030 con 0x7f0ef4106150 2024-04-23T10:43:18.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.605+0000 7f0eeb7fe640 1 -- 172.21.15.97:0/2232224382 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ee4004490 con 0x7f0ef4106150 2024-04-23T10:43:18.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.605+0000 7f0efc35d640 1 -- 172.21.15.97:0/2232224382 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ef419dd50 con 0x7f0ef4106150 2024-04-23T10:43:18.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.605+0000 7f0eeb7fe640 1 -- 172.21.15.97:0/2232224382 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee4035070 con 0x7f0ef4106150 2024-04-23T10:43:18.609 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.605+0000 7f0efc35d640 1 -- 172.21.15.97:0/2232224382 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ef41a5510 con 0x7f0ef4106150 2024-04-23T10:43:18.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.606+0000 7f0efc35d640 1 -- 172.21.15.97:0/2232224382 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ef40691c0 con 0x7f0ef4106150 2024-04-23T10:43:18.610 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.607+0000 7f0eeb7fe640 1 -- 172.21.15.97:0/2232224382 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0ee4042030 con 0x7f0ef4106150 2024-04-23T10:43:18.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.608+0000 7f0eeb7fe640 1 --2- 172.21.15.97:0/2232224382 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f0ee00a3d20 0x7f0ee00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:43:18.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.608+0000 7f0efa0f9640 1 --2- 172.21.15.97:0/2232224382 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f0ee00a3d20 0x7f0ee00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:43:18.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.608+0000 7f0eeb7fe640 1 -- 172.21.15.97:0/2232224382 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8484+0+0 (secure 0 0 0) 0x7f0ee40f11c0 con 0x7f0ef4106150 2024-04-23T10:43:18.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.609+0000 7f0efa0f9640 1 --2- 172.21.15.97:0/2232224382 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f0ee00a3d20 0x7f0ee00a61e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0005fd0 tx=0x7f0ef000b590 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:43:18.611 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.612+0000 7f0eeb7fe640 1 -- 172.21.15.97:0/2232224382 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0ee40bd880 con 0x7f0ef4106150 2024-04-23T10:43:18.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:43:18 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/1151248210' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T10:43:18.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:43:18 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/3853199937' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T10:43:18.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:43:18 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/1705376590' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T10:43:18.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.803+0000 7f0efc35d640 1 -- 172.21.15.97:0/2232224382 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0ef4069dc0 con 0x7f0ef4106150 2024-04-23T10:43:18.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.804+0000 7f0eeb7fe640 1 -- 172.21.15.97:0/2232224382 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v111) ==== 81+0+942 (secure 0 0 0) 0x7f0ee40bd180 con 0x7f0ef4106150 2024-04-23T10:43:18.803 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:0/233335470 2024-04-24T10:26:11.109535+0000 2024-04-23T10:43:18.803 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6800/3149043749 2024-04-24T10:26:11.109535+0000 2024-04-23T10:43:18.803 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:0/338277142 2024-04-24T10:26:11.109535+0000 2024-04-23T10:43:18.803 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:0/3866016040 2024-04-24T10:16:59.468237+0000 2024-04-23T10:43:18.803 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6801/3297166858 2024-04-24T10:16:59.468237+0000 2024-04-23T10:43:18.803 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:0/3804743548 2024-04-24T10:16:59.468237+0000 2024-04-23T10:43:18.803 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6800/3706463017 2024-04-24T10:16:33.128917+0000 2024-04-23T10:43:18.804 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:0/3715079438 2024-04-24T10:16:33.128917+0000 2024-04-23T10:43:18.804 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:0/597405418 2024-04-24T10:26:11.109535+0000 2024-04-23T10:43:18.804 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:0/786806834 2024-04-24T10:16:59.468237+0000 2024-04-23T10:43:18.804 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6801/3706463017 2024-04-24T10:16:33.128917+0000 2024-04-23T10:43:18.804 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6801/3149043749 2024-04-24T10:26:11.109535+0000 2024-04-23T10:43:18.804 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:0/3294175554 2024-04-24T10:16:33.128917+0000 2024-04-23T10:43:18.804 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:0/1449136760 2024-04-24T10:16:33.128917+0000 2024-04-23T10:43:18.804 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:0/3198144747 2024-04-24T10:26:11.109535+0000 2024-04-23T10:43:18.805 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.15.97:6800/3297166858 2024-04-24T10:16:59.468237+0000 2024-04-23T10:43:18.806 INFO:teuthology.orchestra.run.smithi097.stderr:listed 16 entries 2024-04-23T10:43:18.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.805+0000 7f0efc35d640 1 -- 172.21.15.97:0/2232224382 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f0ee00a3d20 msgr2=0x7f0ee00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:43:18.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.805+0000 7f0efc35d640 1 --2- 172.21.15.97:0/2232224382 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f0ee00a3d20 0x7f0ee00a61e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0005fd0 tx=0x7f0ef000b590 comp rx=0 tx=0).stop 2024-04-23T10:43:18.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.805+0000 7f0efc35d640 1 -- 172.21.15.97:0/2232224382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ef4106150 msgr2=0x7f0ef4198c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:43:18.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.805+0000 7f0efc35d640 1 --2- 172.21.15.97:0/2232224382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ef4106150 0x7f0ef4198c50 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4032fd0 tx=0x7f0ee4004270 comp rx=0 tx=0).stop 2024-04-23T10:43:18.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.805+0000 7f0efc35d640 1 -- 172.21.15.97:0/2232224382 shutdown_connections 2024-04-23T10:43:18.806 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.805+0000 7f0efc35d640 1 --2- 172.21.15.97:0/2232224382 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f0ee00a3d20 0x7f0ee00a61e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:18.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.805+0000 7f0efc35d640 1 --2- 172.21.15.97:0/2232224382 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ef419cfd0 0x7f0ef419d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:18.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.805+0000 7f0efc35d640 1 --2- 172.21.15.97:0/2232224382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ef4106150 0x7f0ef4198c50 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:18.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.805+0000 7f0efc35d640 1 --2- 172.21.15.97:0/2232224382 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ef4101080 0x7f0ef4198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:18.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.805+0000 7f0efc35d640 1 -- 172.21.15.97:0/2232224382 >> 172.21.15.97:0/2232224382 conn(0x7f0ef40771d0 msgr2=0x7f0ef40fee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:43:18.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.806+0000 7f0efc35d640 1 -- 172.21.15.97:0/2232224382 shutdown_connections 2024-04-23T10:43:18.807 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:18.806+0000 7f0efc35d640 1 -- 172.21.15.97:0/2232224382 wait complete. 2024-04-23T10:43:18.816 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-23T10:43:18.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T10:43:18.816 DEBUG:teuthology.orchestra.run.smithi097:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-23T10:43:18.854 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-04-23T10:43:18.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T10:43:18.855 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-23T10:43:18.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T10:43:18.915 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-04-23T10:43:18.970 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-23T10:43:18.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T10:43:18.971 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T10:43:19.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T10:43:19.036 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link delete ceph-brx 2024-04-23T10:43:19.132 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-23T10:43:19.146 INFO:tasks.cephadm:Teardown begin 2024-04-23T10:43:19.146 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-23T10:43:19.146 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T10:43:19.250 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.248+0000 7f7c26ea7640 1 -- 172.21.15.97:0/88879992 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c20101a50 msgr2=0x7f7c20105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:43:19.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.248+0000 7f7c26ea7640 1 --2- 172.21.15.97:0/88879992 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c20101a50 0x7f7c20105a90 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f7c10009900 tx=0x7f7c1002a410 comp rx=0 tx=0).stop 2024-04-23T10:43:19.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.248+0000 7f7c26ea7640 1 -- 172.21.15.97:0/88879992 shutdown_connections 2024-04-23T10:43:19.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.248+0000 7f7c26ea7640 1 --2- 172.21.15.97:0/88879992 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c20106170 0x7f7c2010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:19.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.248+0000 7f7c26ea7640 1 --2- 172.21.15.97:0/88879992 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c20101a50 0x7f7c20105a90 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:19.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.248+0000 7f7c26ea7640 1 --2- 172.21.15.97:0/88879992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c201010a0 0x7f7c20101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:19.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.248+0000 7f7c26ea7640 1 -- 172.21.15.97:0/88879992 >> 172.21.15.97:0/88879992 conn(0x7f7c20077f50 msgr2=0x7f7c20078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:43:19.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.248+0000 7f7c26ea7640 1 -- 172.21.15.97:0/88879992 shutdown_connections 2024-04-23T10:43:19.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.248+0000 7f7c26ea7640 1 -- 172.21.15.97:0/88879992 wait complete. 2024-04-23T10:43:19.251 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.248+0000 7f7c26ea7640 1 Processor -- start 2024-04-23T10:43:19.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.248+0000 7f7c26ea7640 1 -- start start 2024-04-23T10:43:19.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.248+0000 7f7c26ea7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c201010a0 0x7f7c20199530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:43:19.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.248+0000 7f7c26ea7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c20106170 0x7f7c20199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:43:19.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.248+0000 7f7c26ea7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c2019a050 0x7f7c201a45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:43:19.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.248+0000 7f7c26ea7640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7c201110b0 con 0x7f7c20106170 2024-04-23T10:43:19.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.248+0000 7f7c26ea7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7c20110f30 con 0x7f7c2019a050 2024-04-23T10:43:19.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.248+0000 7f7c26ea7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f7c20111230 con 0x7f7c201010a0 2024-04-23T10:43:19.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.249+0000 7f7c1ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c20106170 0x7f7c20199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:43:19.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.249+0000 7f7c1ffff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c20106170 0x7f7c20199a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40690/0 (socket says 172.21.15.97:40690) 2024-04-23T10:43:19.252 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.249+0000 7f7c1ffff640 1 -- 172.21.15.97:0/507335830 learned_addr learned my addr 172.21.15.97:0/507335830 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:43:19.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.249+0000 7f7c24c43640 1 --2- 172.21.15.97:0/507335830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c201010a0 0x7f7c20199530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:43:19.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.249+0000 7f7c25444640 1 --2- 172.21.15.97:0/507335830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c2019a050 0x7f7c201a45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:43:19.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.249+0000 7f7c1ffff640 1 -- 172.21.15.97:0/507335830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c201010a0 msgr2=0x7f7c20199530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:43:19.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.249+0000 7f7c1ffff640 1 --2- 172.21.15.97:0/507335830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c201010a0 0x7f7c20199530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:19.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.249+0000 7f7c1ffff640 1 -- 172.21.15.97:0/507335830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c2019a050 msgr2=0x7f7c201a45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:43:19.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.249+0000 7f7c1ffff640 1 --2- 172.21.15.97:0/507335830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c2019a050 0x7f7c201a45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:19.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.249+0000 7f7c1ffff640 1 -- 172.21.15.97:0/507335830 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c201a4dd0 con 0x7f7c20106170 2024-04-23T10:43:19.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.249+0000 7f7c25444640 1 --2- 172.21.15.97:0/507335830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c2019a050 0x7f7c201a45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:43:19.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.249+0000 7f7c1ffff640 1 --2- 172.21.15.97:0/507335830 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c20106170 0x7f7c20199a70 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f7c10032fd0 tx=0x7f7c10004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:43:19.253 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.249+0000 7f7c1dffb640 1 -- 172.21.15.97:0/507335830 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1003f030 con 0x7f7c20106170 2024-04-23T10:43:19.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.249+0000 7f7c1dffb640 1 -- 172.21.15.97:0/507335830 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c10004490 con 0x7f7c20106170 2024-04-23T10:43:19.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.249+0000 7f7c1dffb640 1 -- 172.21.15.97:0/507335830 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c10035070 con 0x7f7c20106170 2024-04-23T10:43:19.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.249+0000 7f7c26ea7640 1 -- 172.21.15.97:0/507335830 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c201a5060 con 0x7f7c20106170 2024-04-23T10:43:19.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.249+0000 7f7c26ea7640 1 -- 172.21.15.97:0/507335830 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c201a5550 con 0x7f7c20106170 2024-04-23T10:43:19.254 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.250+0000 7f7c26ea7640 1 -- 172.21.15.97:0/507335830 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c20068d70 con 0x7f7c20106170 2024-04-23T10:43:19.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.251+0000 7f7c1dffb640 1 -- 172.21.15.97:0/507335830 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7c10042030 con 0x7f7c20106170 2024-04-23T10:43:19.255 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.252+0000 7f7c1dffb640 1 --2- 172.21.15.97:0/507335830 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f7c0c0a3d20 0x7f7c0c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:43:19.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.252+0000 7f7c24c43640 1 --2- 172.21.15.97:0/507335830 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f7c0c0a3d20 0x7f7c0c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:43:19.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.252+0000 7f7c1dffb640 1 -- 172.21.15.97:0/507335830 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8484+0+0 (secure 0 0 0) 0x7f7c100f0cd0 con 0x7f7c20106170 2024-04-23T10:43:19.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.252+0000 7f7c24c43640 1 --2- 172.21.15.97:0/507335830 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f7c0c0a3d20 0x7f7c0c0a61e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7c18005fd0 tx=0x7f7c18009680 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:43:19.256 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.256+0000 7f7c1dffb640 1 -- 172.21.15.97:0/507335830 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7c100bd390 con 0x7f7c20106170 2024-04-23T10:43:19.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:43:19 smithi097 ceph-mon[29197]: pgmap v538: 97 pgs: 97 active+clean; 4.5 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T10:43:19.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:43:19 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/3178664736' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T10:43:19.360 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:43:19 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/2232224382' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-23T10:43:19.475 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.476+0000 7f7c26ea7640 1 -- 172.21.15.97:0/507335830 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7c201093a0 con 0x7f7c20106170 2024-04-23T10:43:19.477 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.477+0000 7f7c1dffb640 1 -- 172.21.15.97:0/507335830 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11931 (secure 0 0 0) 0x7f7c100bcc90 con 0x7f7c20106170 2024-04-23T10:43:19.477 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:43:19.477 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3946878762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3946878762},{"type":"v1","addr":"172.21.15.97:6835","nonce":3946878762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/209800861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":209800861},{"type":"v1","addr":"172.21.15.97:6837","nonce":209800861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T10:24:06.910134+0000","modified":"2024-04-23T10:26:09.820985+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6839/418013575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":418013575},{"type":"v1","addr":"172.21.15.97:6839","nonce":418013575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6841/747242680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":747242680},{"type":"v1","addr":"172.21.15.97:6841","nonce":747242680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6833/4292847121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4292847121},{"type":"v1","addr":"172.21.15.183:6833","nonce":4292847121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6833/3885816565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3885816565},{"type":"v1","addr":"172.21.15.196:6833","nonce":3885816565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1897093105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1897093105},{"type":"v1","addr":"172.21.15.183:6835","nonce":1897093105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1171477563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1171477563},{"type":"v1","addr":"172.21.15.196:6835","nonce":1171477563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1898341911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1898341911},{"type":"v1","addr":"172.21.15.183:6837","nonce":1898341911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.196:6837/3094946502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3094946502},{"type":"v1","addr":"172.21.15.196:6837","nonce":3094946502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.196:6839/83843556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":83843556},{"type":"v1","addr":"172.21.15.196:6839","nonce":83843556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6839/1763841130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1763841130},{"type":"v1","addr":"172.21.15.183:6839","nonce":1763841130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24487,24403,24398,24469,24488]},"id":1}]} 2024-04-23T10:43:19.478 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 26 2024-04-23T10:43:19.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.479+0000 7f7c26ea7640 1 -- 172.21.15.97:0/507335830 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f7c0c0a3d20 msgr2=0x7f7c0c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:43:19.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.479+0000 7f7c26ea7640 1 --2- 172.21.15.97:0/507335830 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f7c0c0a3d20 0x7f7c0c0a61e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7c18005fd0 tx=0x7f7c18009680 comp rx=0 tx=0).stop 2024-04-23T10:43:19.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.479+0000 7f7c26ea7640 1 -- 172.21.15.97:0/507335830 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c20106170 msgr2=0x7f7c20199a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:43:19.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.479+0000 7f7c26ea7640 1 --2- 172.21.15.97:0/507335830 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c20106170 0x7f7c20199a70 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f7c10032fd0 tx=0x7f7c10004270 comp rx=0 tx=0).stop 2024-04-23T10:43:19.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.479+0000 7f7c26ea7640 1 -- 172.21.15.97:0/507335830 shutdown_connections 2024-04-23T10:43:19.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.479+0000 7f7c26ea7640 1 --2- 172.21.15.97:0/507335830 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f7c0c0a3d20 0x7f7c0c0a61e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:19.479 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.479+0000 7f7c26ea7640 1 --2- 172.21.15.97:0/507335830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c2019a050 0x7f7c201a45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:19.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.479+0000 7f7c26ea7640 1 --2- 172.21.15.97:0/507335830 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c20106170 0x7f7c20199a70 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:19.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.479+0000 7f7c26ea7640 1 --2- 172.21.15.97:0/507335830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c201010a0 0x7f7c20199530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:19.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.479+0000 7f7c26ea7640 1 -- 172.21.15.97:0/507335830 >> 172.21.15.97:0/507335830 conn(0x7f7c20077f50 msgr2=0x7f7c200ff090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:43:19.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.480+0000 7f7c26ea7640 1 -- 172.21.15.97:0/507335830 shutdown_connections 2024-04-23T10:43:19.480 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.480+0000 7f7c26ea7640 1 -- 172.21.15.97:0/507335830 wait complete. 2024-04-23T10:43:19.490 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T10:43:19.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.590+0000 7f4b6967c640 1 -- 172.21.15.97:0/2780637197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b64101a50 msgr2=0x7f4b64105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:43:19.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.590+0000 7f4b6967c640 1 --2- 172.21.15.97:0/2780637197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b64101a50 0x7f4b64105a90 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f4b58009900 tx=0x7f4b5802a410 comp rx=0 tx=0).stop 2024-04-23T10:43:19.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.590+0000 7f4b6967c640 1 -- 172.21.15.97:0/2780637197 shutdown_connections 2024-04-23T10:43:19.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.590+0000 7f4b6967c640 1 --2- 172.21.15.97:0/2780637197 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b64106170 0x7f4b6410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:19.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.590+0000 7f4b6967c640 1 --2- 172.21.15.97:0/2780637197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b64101a50 0x7f4b64105a90 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:19.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.590+0000 7f4b6967c640 1 --2- 172.21.15.97:0/2780637197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b641010a0 0x7f4b64101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:19.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.590+0000 7f4b6967c640 1 -- 172.21.15.97:0/2780637197 >> 172.21.15.97:0/2780637197 conn(0x7f4b64077f50 msgr2=0x7f4b64078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:43:19.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.590+0000 7f4b6967c640 1 -- 172.21.15.97:0/2780637197 shutdown_connections 2024-04-23T10:43:19.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.590+0000 7f4b6967c640 1 -- 172.21.15.97:0/2780637197 wait complete. 2024-04-23T10:43:19.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.590+0000 7f4b6967c640 1 Processor -- start 2024-04-23T10:43:19.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.591+0000 7f4b6967c640 1 -- start start 2024-04-23T10:43:19.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.591+0000 7f4b6967c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b641010a0 0x7f4b64198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:43:19.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.591+0000 7f4b6967c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b64106170 0x7f4b64198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:43:19.593 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.591+0000 7f4b6967c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b6419cfe0 0x7f4b6419d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:43:19.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.591+0000 7f4b6967c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4b64111090 con 0x7f4b64106170 2024-04-23T10:43:19.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.591+0000 7f4b6967c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4b64110f10 con 0x7f4b641010a0 2024-04-23T10:43:19.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.591+0000 7f4b6967c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f4b64111210 con 0x7f4b6419cfe0 2024-04-23T10:43:19.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.591+0000 7f4b627fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b64106170 0x7f4b64198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:43:19.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.591+0000 7f4b627fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b64106170 0x7f4b64198c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40706/0 (socket says 172.21.15.97:40706) 2024-04-23T10:43:19.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.591+0000 7f4b627fc640 1 -- 172.21.15.97:0/4036975303 learned_addr learned my addr 172.21.15.97:0/4036975303 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:43:19.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.591+0000 7f4b62ffd640 1 --2- 172.21.15.97:0/4036975303 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b641010a0 0x7f4b64198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:43:19.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.591+0000 7f4b637fe640 1 --2- 172.21.15.97:0/4036975303 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b6419cfe0 0x7f4b6419d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:43:19.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.591+0000 7f4b627fc640 1 -- 172.21.15.97:0/4036975303 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b6419cfe0 msgr2=0x7f4b6419d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:43:19.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.591+0000 7f4b627fc640 1 --2- 172.21.15.97:0/4036975303 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b6419cfe0 0x7f4b6419d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:19.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.591+0000 7f4b627fc640 1 -- 172.21.15.97:0/4036975303 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b641010a0 msgr2=0x7f4b64198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:43:19.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.591+0000 7f4b627fc640 1 --2- 172.21.15.97:0/4036975303 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b641010a0 0x7f4b64198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:19.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.591+0000 7f4b627fc640 1 -- 172.21.15.97:0/4036975303 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b6419dad0 con 0x7f4b64106170 2024-04-23T10:43:19.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.591+0000 7f4b627fc640 1 --2- 172.21.15.97:0/4036975303 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b64106170 0x7f4b64198c60 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f4b58032fd0 tx=0x7f4b58031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:43:19.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.591+0000 7f4b4bfff640 1 -- 172.21.15.97:0/4036975303 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b5803f030 con 0x7f4b64106170 2024-04-23T10:43:19.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.591+0000 7f4b4bfff640 1 -- 172.21.15.97:0/4036975303 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b58004490 con 0x7f4b64106170 2024-04-23T10:43:19.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.591+0000 7f4b6967c640 1 -- 172.21.15.97:0/4036975303 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b6419dd60 con 0x7f4b64106170 2024-04-23T10:43:19.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.591+0000 7f4b4bfff640 1 -- 172.21.15.97:0/4036975303 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b58035070 con 0x7f4b64106170 2024-04-23T10:43:19.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.592+0000 7f4b6967c640 1 -- 172.21.15.97:0/4036975303 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b641a5520 con 0x7f4b64106170 2024-04-23T10:43:19.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.592+0000 7f4b6967c640 1 -- 172.21.15.97:0/4036975303 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b64068d70 con 0x7f4b64106170 2024-04-23T10:43:19.595 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.593+0000 7f4b4bfff640 1 -- 172.21.15.97:0/4036975303 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4b58042030 con 0x7f4b64106170 2024-04-23T10:43:19.596 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:43:19 smithi196 ceph-mon[32919]: pgmap v538: 97 pgs: 97 active+clean; 4.5 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T10:43:19.596 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:43:19 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/3178664736' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T10:43:19.597 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:43:19 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/2232224382' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-23T10:43:19.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.594+0000 7f4b4bfff640 1 --2- 172.21.15.97:0/4036975303 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f4b4c0a3d20 0x7f4b4c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:43:19.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.594+0000 7f4b62ffd640 1 --2- 172.21.15.97:0/4036975303 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f4b4c0a3d20 0x7f4b4c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:43:19.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.594+0000 7f4b4bfff640 1 -- 172.21.15.97:0/4036975303 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8484+0+0 (secure 0 0 0) 0x7f4b580f0c40 con 0x7f4b64106170 2024-04-23T10:43:19.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.594+0000 7f4b62ffd640 1 --2- 172.21.15.97:0/4036975303 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f4b4c0a3d20 0x7f4b4c0a61e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4b54009b60 tx=0x7f4b54005f90 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:43:19.598 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.598+0000 7f4b4bfff640 1 -- 172.21.15.97:0/4036975303 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4b580bd300 con 0x7f4b64106170 2024-04-23T10:43:19.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:43:19 smithi183 ceph-mon[32317]: pgmap v538: 97 pgs: 97 active+clean; 4.5 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T10:43:19.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:43:19 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/3178664736' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T10:43:19.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:43:19 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/2232224382' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-23T10:43:19.813 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.815+0000 7f4b6967c640 1 -- 172.21.15.97:0/4036975303 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4b64101a50 con 0x7f4b64106170 2024-04-23T10:43:19.814 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.815+0000 7f4b4bfff640 1 -- 172.21.15.97:0/4036975303 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11931 (secure 0 0 0) 0x7f4b580bcc00 con 0x7f4b64106170 2024-04-23T10:43:19.814 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:43:19.815 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14559,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/3946878762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":3946878762},{"type":"v1","addr":"172.21.15.97:6835","nonce":3946878762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14571,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/209800861","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":209800861},{"type":"v1","addr":"172.21.15.97:6837","nonce":209800861}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T10:24:06.910134+0000","modified":"2024-04-23T10:26:09.820985+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469,"mds_3":24403,"mds_4":24398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6839/418013575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":418013575},{"type":"v1","addr":"172.21.15.97:6839","nonce":418013575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6841/747242680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6840","nonce":747242680},{"type":"v1","addr":"172.21.15.97:6841","nonce":747242680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":4,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.183:6833/4292847121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4292847121},{"type":"v1","addr":"172.21.15.183:6833","nonce":4292847121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"l","rank":3,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.196:6833/3885816565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3885816565},{"type":"v1","addr":"172.21.15.196:6833","nonce":3885816565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/1897093105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":1897093105},{"type":"v1","addr":"172.21.15.183:6835","nonce":1897093105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.196:6835/1171477563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1171477563},{"type":"v1","addr":"172.21.15.196:6835","nonce":1171477563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24461":{"gid":24461,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6837/1898341911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":1898341911},{"type":"v1","addr":"172.21.15.183:6837","nonce":1898341911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.196:6837/3094946502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6836","nonce":3094946502},{"type":"v1","addr":"172.21.15.196:6837","nonce":3094946502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":6,"addr":"172.21.15.196:6839/83843556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6838","nonce":83843556},{"type":"v1","addr":"172.21.15.196:6839","nonce":83843556}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.183:6839/1763841130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":1763841130},{"type":"v1","addr":"172.21.15.183:6839","nonce":1763841130}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24487,24403,24398,24469,24488]},"id":1}]} 2024-04-23T10:43:19.815 INFO:teuthology.orchestra.run.smithi097.stderr:dumped fsmap epoch 26 2024-04-23T10:43:19.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.817+0000 7f4b6967c640 1 -- 172.21.15.97:0/4036975303 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f4b4c0a3d20 msgr2=0x7f4b4c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:43:19.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.817+0000 7f4b6967c640 1 --2- 172.21.15.97:0/4036975303 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f4b4c0a3d20 0x7f4b4c0a61e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4b54009b60 tx=0x7f4b54005f90 comp rx=0 tx=0).stop 2024-04-23T10:43:19.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.817+0000 7f4b6967c640 1 -- 172.21.15.97:0/4036975303 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b64106170 msgr2=0x7f4b64198c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:43:19.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.817+0000 7f4b6967c640 1 --2- 172.21.15.97:0/4036975303 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b64106170 0x7f4b64198c60 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f4b58032fd0 tx=0x7f4b58031a20 comp rx=0 tx=0).stop 2024-04-23T10:43:19.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.817+0000 7f4b6967c640 1 -- 172.21.15.97:0/4036975303 shutdown_connections 2024-04-23T10:43:19.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.817+0000 7f4b6967c640 1 --2- 172.21.15.97:0/4036975303 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f4b4c0a3d20 0x7f4b4c0a61e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:19.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.817+0000 7f4b6967c640 1 --2- 172.21.15.97:0/4036975303 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b6419cfe0 0x7f4b6419d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:19.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.817+0000 7f4b6967c640 1 --2- 172.21.15.97:0/4036975303 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b64106170 0x7f4b64198c60 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:19.817 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.817+0000 7f4b6967c640 1 --2- 172.21.15.97:0/4036975303 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b641010a0 0x7f4b64198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:19.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.817+0000 7f4b6967c640 1 -- 172.21.15.97:0/4036975303 >> 172.21.15.97:0/4036975303 conn(0x7f4b64077f50 msgr2=0x7f4b64069480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:43:19.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.818+0000 7f4b6967c640 1 -- 172.21.15.97:0/4036975303 shutdown_connections 2024-04-23T10:43:19.818 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.818+0000 7f4b6967c640 1 -- 172.21.15.97:0/4036975303 wait complete. 2024-04-23T10:43:19.828 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T10:43:19.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.927+0000 7f05fa4a7640 1 -- 172.21.15.97:0/3079898399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05f41031e0 msgr2=0x7f05f41035c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:43:19.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.927+0000 7f05fa4a7640 1 --2- 172.21.15.97:0/3079898399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05f41031e0 0x7f05f41035c0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f05e8009900 tx=0x7f05e802a510 comp rx=0 tx=0).stop 2024-04-23T10:43:19.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.927+0000 7f05fa4a7640 1 -- 172.21.15.97:0/3079898399 shutdown_connections 2024-04-23T10:43:19.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.927+0000 7f05fa4a7640 1 --2- 172.21.15.97:0/3079898399 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05f410c520 0x7f05f410e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:19.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.928+0000 7f05fa4a7640 1 --2- 172.21.15.97:0/3079898399 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f05f4103b00 0x7f05f410bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:19.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.928+0000 7f05fa4a7640 1 --2- 172.21.15.97:0/3079898399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05f41031e0 0x7f05f41035c0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:19.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.928+0000 7f05fa4a7640 1 -- 172.21.15.97:0/3079898399 >> 172.21.15.97:0/3079898399 conn(0x7f05f4077f50 msgr2=0x7f05f4078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:43:19.930 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.928+0000 7f05fa4a7640 1 -- 172.21.15.97:0/3079898399 shutdown_connections 2024-04-23T10:43:19.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.928+0000 7f05fa4a7640 1 -- 172.21.15.97:0/3079898399 wait complete. 2024-04-23T10:43:19.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.928+0000 7f05fa4a7640 1 Processor -- start 2024-04-23T10:43:19.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.928+0000 7f05fa4a7640 1 -- start start 2024-04-23T10:43:19.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.928+0000 7f05fa4a7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05f41031e0 0x7f05f4198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:43:19.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.928+0000 7f05fa4a7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f05f4103b00 0x7f05f4198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:43:19.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.928+0000 7f05fa4a7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05f410c520 0x7f05f419d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:43:19.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.928+0000 7f05fa4a7640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f05f4112190 con 0x7f05f410c520 2024-04-23T10:43:19.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.928+0000 7f05fa4a7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f05f4112010 con 0x7f05f4103b00 2024-04-23T10:43:19.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.928+0000 7f05fa4a7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f05f4112310 con 0x7f05f41031e0 2024-04-23T10:43:19.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.929+0000 7f05f8a44640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05f410c520 0x7f05f419d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:43:19.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.929+0000 7f05f8a44640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05f410c520 0x7f05f419d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40732/0 (socket says 172.21.15.97:40732) 2024-04-23T10:43:19.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.929+0000 7f05f8a44640 1 -- 172.21.15.97:0/31518302 learned_addr learned my addr 172.21.15.97:0/31518302 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:43:19.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.929+0000 7f05f3fff640 1 --2- 172.21.15.97:0/31518302 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05f41031e0 0x7f05f4198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:43:19.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.929+0000 7f05f37fe640 1 --2- 172.21.15.97:0/31518302 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f05f4103b00 0x7f05f4198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:43:19.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.929+0000 7f05f8a44640 1 -- 172.21.15.97:0/31518302 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05f41031e0 msgr2=0x7f05f4198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:43:19.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.929+0000 7f05f8a44640 1 --2- 172.21.15.97:0/31518302 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05f41031e0 0x7f05f4198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:19.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.929+0000 7f05f8a44640 1 -- 172.21.15.97:0/31518302 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f05f4103b00 msgr2=0x7f05f4198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:43:19.932 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.929+0000 7f05f8a44640 1 --2- 172.21.15.97:0/31518302 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f05f4103b00 0x7f05f4198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:19.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.929+0000 7f05f8a44640 1 -- 172.21.15.97:0/31518302 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05f419d740 con 0x7f05f410c520 2024-04-23T10:43:19.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.929+0000 7f05f37fe640 1 --2- 172.21.15.97:0/31518302 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f05f4103b00 0x7f05f4198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:43:19.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.929+0000 7f05f8a44640 1 --2- 172.21.15.97:0/31518302 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05f410c520 0x7f05f419d010 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f05ec00eaa0 tx=0x7f05ec00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:43:19.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.929+0000 7f05f17fa640 1 -- 172.21.15.97:0/31518302 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ec00cdb0 con 0x7f05f410c520 2024-04-23T10:43:19.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.929+0000 7f05f17fa640 1 -- 172.21.15.97:0/31518302 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05ec005020 con 0x7f05f410c520 2024-04-23T10:43:19.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.929+0000 7f05fa4a7640 1 -- 172.21.15.97:0/31518302 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05f419da30 con 0x7f05f410c520 2024-04-23T10:43:19.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.929+0000 7f05f17fa640 1 -- 172.21.15.97:0/31518302 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ec0116d0 con 0x7f05f410c520 2024-04-23T10:43:19.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.929+0000 7f05fa4a7640 1 -- 172.21.15.97:0/31518302 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05f41a52c0 con 0x7f05f410c520 2024-04-23T10:43:19.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.930+0000 7f05fa4a7640 1 -- 172.21.15.97:0/31518302 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05f4068d70 con 0x7f05f410c520 2024-04-23T10:43:19.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.931+0000 7f05f17fa640 1 -- 172.21.15.97:0/31518302 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f05ec0051d0 con 0x7f05f410c520 2024-04-23T10:43:19.934 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.932+0000 7f05f17fa640 1 --2- 172.21.15.97:0/31518302 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f05dc0a3d20 0x7f05dc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:43:19.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.932+0000 7f05f3fff640 1 --2- 172.21.15.97:0/31518302 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f05dc0a3d20 0x7f05dc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:43:19.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.932+0000 7f05f17fa640 1 -- 172.21.15.97:0/31518302 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8484+0+0 (secure 0 0 0) 0x7f05ec0d0940 con 0x7f05f410c520 2024-04-23T10:43:19.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.932+0000 7f05f3fff640 1 --2- 172.21.15.97:0/31518302 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f05dc0a3d20 0x7f05dc0a61e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f05e8031440 tx=0x7f05e8005b50 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:43:19.935 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:19.936+0000 7f05f17fa640 1 -- 172.21.15.97:0/31518302 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f05ec016030 con 0x7f05f410c520 2024-04-23T10:43:20.114 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.115+0000 7f05fa4a7640 1 -- 172.21.15.97:0/31518302 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f05f4107110 con 0x7f05f410c520 2024-04-23T10:43:20.116 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.117+0000 7f05f17fa640 1 -- 172.21.15.97:0/31518302 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) ==== 74+0+23075 (secure 0 0 0) 0x7f05ec09d000 con 0x7f05f410c520 2024-04-23T10:43:20.116 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-23T10:43:20.116 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":111,"fsid":"9b22af88-0159-11ef-bc93-c7b262605968","created":"2024-04-23T10:16:05.983305+0000","modified":"2024-04-23T10:43:11.195576+0000","last_up_change":"2024-04-23T10:23:13.114761+0000","last_in_change":"2024-04-23T10:22:52.804880+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T10:19:25.559138+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T10:24:04.570049+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T10:24:05.861631+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":29,"snap_epoch":110,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"58cb995d-de35-4522-ac81-4982401b260d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6803","nonce":3031221802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6805","nonce":3031221802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6809","nonce":3031221802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3031221802},{"type":"v1","addr":"172.21.15.97:6807","nonce":3031221802}]},"public_addr":"172.21.15.97:6803/3031221802","cluster_addr":"172.21.15.97:6805/3031221802","heartbeat_back_addr":"172.21.15.97:6809/3031221802","heartbeat_front_addr":"172.21.15.97:6807/3031221802","state":["exists","up"]},{"osd":1,"uuid":"b47bd8ee-1f21-4834-acf5-bb40f97ab3ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6801","nonce":1108870440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6803","nonce":1108870440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6807","nonce":1108870440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":1108870440},{"type":"v1","addr":"172.21.15.183:6805","nonce":1108870440}]},"public_addr":"172.21.15.183:6801/1108870440","cluster_addr":"172.21.15.183:6803/1108870440","heartbeat_back_addr":"172.21.15.183:6807/1108870440","heartbeat_front_addr":"172.21.15.183:6805/1108870440","state":["exists","up"]},{"osd":2,"uuid":"17c4eec3-6c47-4334-9b63-1d5ad9ec5988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6801","nonce":2877039731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6803","nonce":2877039731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6807","nonce":2877039731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":2877039731},{"type":"v1","addr":"172.21.15.196:6805","nonce":2877039731}]},"public_addr":"172.21.15.196:6801/2877039731","cluster_addr":"172.21.15.196:6803/2877039731","heartbeat_back_addr":"172.21.15.196:6807/2877039731","heartbeat_front_addr":"172.21.15.196:6805/2877039731","state":["exists","up"]},{"osd":3,"uuid":"3f731c38-2e45-4ca1-8dce-efc4a1b2166b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6811","nonce":4276311194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6813","nonce":4276311194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6817","nonce":4276311194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":4276311194},{"type":"v1","addr":"172.21.15.97:6815","nonce":4276311194}]},"public_addr":"172.21.15.97:6811/4276311194","cluster_addr":"172.21.15.97:6813/4276311194","heartbeat_back_addr":"172.21.15.97:6817/4276311194","heartbeat_front_addr":"172.21.15.97:6815/4276311194","state":["exists","up"]},{"osd":4,"uuid":"fdcafb14-9b82-4134-bd5b-79c0516f6cbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6809","nonce":63913916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6811","nonce":63913916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6815","nonce":63913916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":63913916},{"type":"v1","addr":"172.21.15.183:6813","nonce":63913916}]},"public_addr":"172.21.15.183:6809/63913916","cluster_addr":"172.21.15.183:6811/63913916","heartbeat_back_addr":"172.21.15.183:6815/63913916","heartbeat_front_addr":"172.21.15.183:6813/63913916","state":["exists","up"]},{"osd":5,"uuid":"cdcc18d8-da3c-4012-bf12-876bb39e66eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6809","nonce":2804446896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6811","nonce":2804446896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6815","nonce":2804446896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":2804446896},{"type":"v1","addr":"172.21.15.196:6813","nonce":2804446896}]},"public_addr":"172.21.15.196:6809/2804446896","cluster_addr":"172.21.15.196:6811/2804446896","heartbeat_back_addr":"172.21.15.196:6815/2804446896","heartbeat_front_addr":"172.21.15.196:6813/2804446896","state":["exists","up"]},{"osd":6,"uuid":"071abb5c-2edf-4832-b8aa-3b80d20096fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6819","nonce":2080916705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6821","nonce":2080916705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6825","nonce":2080916705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2080916705},{"type":"v1","addr":"172.21.15.97:6823","nonce":2080916705}]},"public_addr":"172.21.15.97:6819/2080916705","cluster_addr":"172.21.15.97:6821/2080916705","heartbeat_back_addr":"172.21.15.97:6825/2080916705","heartbeat_front_addr":"172.21.15.97:6823/2080916705","state":["exists","up"]},{"osd":7,"uuid":"9d54760b-66df-486b-8abb-017f3b64b4d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6817","nonce":1865873033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6819","nonce":1865873033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6823","nonce":1865873033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1865873033},{"type":"v1","addr":"172.21.15.183:6821","nonce":1865873033}]},"public_addr":"172.21.15.183:6817/1865873033","cluster_addr":"172.21.15.183:6819/1865873033","heartbeat_back_addr":"172.21.15.183:6823/1865873033","heartbeat_front_addr":"172.21.15.183:6821/1865873033","state":["exists","up"]},{"osd":8,"uuid":"5cd3100a-bd6a-4be5-b5e9-0d0afc325193","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6817","nonce":4063932139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6819","nonce":4063932139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6823","nonce":4063932139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":4063932139},{"type":"v1","addr":"172.21.15.196:6821","nonce":4063932139}]},"public_addr":"172.21.15.196:6817/4063932139","cluster_addr":"172.21.15.196:6819/4063932139","heartbeat_back_addr":"172.21.15.196:6823/4063932139","heartbeat_front_addr":"172.21.15.196:6821/4063932139","state":["exists","up"]},{"osd":9,"uuid":"b6a5f4a1-4ecc-4f73-a2df-07551d9298ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6827","nonce":1887432474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6829","nonce":1887432474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6833","nonce":1887432474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":1887432474},{"type":"v1","addr":"172.21.15.97:6831","nonce":1887432474}]},"public_addr":"172.21.15.97:6827/1887432474","cluster_addr":"172.21.15.97:6829/1887432474","heartbeat_back_addr":"172.21.15.97:6833/1887432474","heartbeat_front_addr":"172.21.15.97:6831/1887432474","state":["exists","up"]},{"osd":10,"uuid":"86f59816-1ea6-4ae9-8796-cf5fc4831064","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6825","nonce":799434647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6827","nonce":799434647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6831","nonce":799434647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":799434647},{"type":"v1","addr":"172.21.15.183:6829","nonce":799434647}]},"public_addr":"172.21.15.183:6825/799434647","cluster_addr":"172.21.15.183:6827/799434647","heartbeat_back_addr":"172.21.15.183:6831/799434647","heartbeat_front_addr":"172.21.15.183:6829/799434647","state":["exists","up"]},{"osd":11,"uuid":"95d64a89-a9f8-4c7a-8c0d-da7fb26cfff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6825","nonce":1873969467}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6827","nonce":1873969467}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6831","nonce":1873969467}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":1873969467},{"type":"v1","addr":"172.21.15.196:6829","nonce":1873969467}]},"public_addr":"172.21.15.196:6825/1873969467","cluster_addr":"172.21.15.196:6827/1873969467","heartbeat_back_addr":"172.21.15.196:6831/1873969467","heartbeat_front_addr":"172.21.15.196:6829/1873969467","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:18:33.721156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:18:57.586709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:19:22.069458+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:19:46.451597+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:20:11.561019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:20:36.298672+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:21:01.661804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:21:27.324128+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:21:52.898197+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:22:19.614326+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:22:45.154917+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T10:23:11.222612+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.97:0/233335470":"2024-04-24T10:26:11.109535+0000","172.21.15.97:6800/3149043749":"2024-04-24T10:26:11.109535+0000","172.21.15.97:0/338277142":"2024-04-24T10:26:11.109535+0000","172.21.15.97:0/3866016040":"2024-04-24T10:16:59.468237+0000","172.21.15.97:6801/3297166858":"2024-04-24T10:16:59.468237+0000","172.21.15.97:0/3804743548":"2024-04-24T10:16:59.468237+0000","172.21.15.97:6800/3706463017":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/3715079438":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/597405418":"2024-04-24T10:26:11.109535+0000","172.21.15.97:0/786806834":"2024-04-24T10:16:59.468237+0000","172.21.15.97:6801/3706463017":"2024-04-24T10:16:33.128917+0000","172.21.15.97:6801/3149043749":"2024-04-24T10:26:11.109535+0000","172.21.15.97:0/3294175554":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/1449136760":"2024-04-24T10:16:33.128917+0000","172.21.15.97:0/3198144747":"2024-04-24T10:26:11.109535+0000","172.21.15.97:6800/3297166858":"2024-04-24T10:16:59.468237+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":16,"length":1},{"begin":29,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T10:43:20.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.119+0000 7f05fa4a7640 1 -- 172.21.15.97:0/31518302 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f05dc0a3d20 msgr2=0x7f05dc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:43:20.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.119+0000 7f05fa4a7640 1 --2- 172.21.15.97:0/31518302 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f05dc0a3d20 0x7f05dc0a61e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f05e8031440 tx=0x7f05e8005b50 comp rx=0 tx=0).stop 2024-04-23T10:43:20.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.119+0000 7f05fa4a7640 1 -- 172.21.15.97:0/31518302 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05f410c520 msgr2=0x7f05f419d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:43:20.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.119+0000 7f05fa4a7640 1 --2- 172.21.15.97:0/31518302 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05f410c520 0x7f05f419d010 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f05ec00eaa0 tx=0x7f05ec00ef60 comp rx=0 tx=0).stop 2024-04-23T10:43:20.118 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.119+0000 7f05fa4a7640 1 -- 172.21.15.97:0/31518302 shutdown_connections 2024-04-23T10:43:20.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.119+0000 7f05fa4a7640 1 --2- 172.21.15.97:0/31518302 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f05dc0a3d20 0x7f05dc0a61e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:20.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.119+0000 7f05fa4a7640 1 --2- 172.21.15.97:0/31518302 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f05f410c520 0x7f05f419d010 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:20.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.119+0000 7f05fa4a7640 1 --2- 172.21.15.97:0/31518302 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f05f4103b00 0x7f05f4198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:20.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.119+0000 7f05fa4a7640 1 --2- 172.21.15.97:0/31518302 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05f41031e0 0x7f05f4198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:20.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.119+0000 7f05fa4a7640 1 -- 172.21.15.97:0/31518302 >> 172.21.15.97:0/31518302 conn(0x7f05f4077f50 msgr2=0x7f05f40fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:43:20.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.119+0000 7f05fa4a7640 1 -- 172.21.15.97:0/31518302 shutdown_connections 2024-04-23T10:43:20.119 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.119+0000 7f05fa4a7640 1 -- 172.21.15.97:0/31518302 wait complete. 2024-04-23T10:43:20.129 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-23T10:43:20.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.231+0000 7fa5c88bb640 1 -- 172.21.15.97:0/1287926155 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5c41010c0 msgr2=0x7fa5c41014a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:43:20.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.231+0000 7fa5c88bb640 1 --2- 172.21.15.97:0/1287926155 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5c41010c0 0x7fa5c41014a0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4009900 tx=0x7fa5b402a410 comp rx=0 tx=0).stop 2024-04-23T10:43:20.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.231+0000 7fa5c88bb640 1 -- 172.21.15.97:0/1287926155 shutdown_connections 2024-04-23T10:43:20.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.231+0000 7fa5c88bb640 1 --2- 172.21.15.97:0/1287926155 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5c4106190 0x7fa5c410dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:20.236 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.231+0000 7fa5c88bb640 1 --2- 172.21.15.97:0/1287926155 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5c4101a70 0x7fa5c4105ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:20.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.231+0000 7fa5c88bb640 1 --2- 172.21.15.97:0/1287926155 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5c41010c0 0x7fa5c41014a0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:20.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.231+0000 7fa5c88bb640 1 -- 172.21.15.97:0/1287926155 >> 172.21.15.97:0/1287926155 conn(0x7fa5c40771d0 msgr2=0x7fa5c40775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:43:20.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.231+0000 7fa5c88bb640 1 -- 172.21.15.97:0/1287926155 shutdown_connections 2024-04-23T10:43:20.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.231+0000 7fa5c88bb640 1 -- 172.21.15.97:0/1287926155 wait complete. 2024-04-23T10:43:20.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.232+0000 7fa5c88bb640 1 Processor -- start 2024-04-23T10:43:20.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.232+0000 7fa5c88bb640 1 -- start start 2024-04-23T10:43:20.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.232+0000 7fa5c88bb640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5c41010c0 0x7fa5c41994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:43:20.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.232+0000 7fa5c88bb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5c4106190 0x7fa5c4199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:43:20.237 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.232+0000 7fa5c88bb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5c4199fc0 0x7fa5c41a4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:43:20.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.232+0000 7fa5c88bb640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c41110d0 con 0x7fa5c4106190 2024-04-23T10:43:20.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.232+0000 7fa5c88bb640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c4110f50 con 0x7fa5c41010c0 2024-04-23T10:43:20.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.232+0000 7fa5c88bb640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c4111250 con 0x7fa5c4199fc0 2024-04-23T10:43:20.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.232+0000 7fa5bbfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5c4106190 0x7fa5c4199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:43:20.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.232+0000 7fa5bbfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5c4106190 0x7fa5c4199a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40750/0 (socket says 172.21.15.97:40750) 2024-04-23T10:43:20.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.232+0000 7fa5bbfff640 1 -- 172.21.15.97:0/563832314 learned_addr learned my addr 172.21.15.97:0/563832314 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:43:20.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.232+0000 7fa5c259c640 1 --2- 172.21.15.97:0/563832314 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5c41010c0 0x7fa5c41994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:43:20.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.232+0000 7fa5c2d9d640 1 --2- 172.21.15.97:0/563832314 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5c4199fc0 0x7fa5c41a4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:43:20.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.232+0000 7fa5bbfff640 1 -- 172.21.15.97:0/563832314 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5c4199fc0 msgr2=0x7fa5c41a4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:43:20.238 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.233+0000 7fa5bbfff640 1 --2- 172.21.15.97:0/563832314 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5c4199fc0 0x7fa5c41a4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:20.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.233+0000 7fa5bbfff640 1 -- 172.21.15.97:0/563832314 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5c41010c0 msgr2=0x7fa5c41994f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:43:20.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.233+0000 7fa5bbfff640 1 --2- 172.21.15.97:0/563832314 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5c41010c0 0x7fa5c41994f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:20.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.233+0000 7fa5bbfff640 1 -- 172.21.15.97:0/563832314 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5c41a4d40 con 0x7fa5c4106190 2024-04-23T10:43:20.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.233+0000 7fa5bbfff640 1 --2- 172.21.15.97:0/563832314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5c4106190 0x7fa5c4199a30 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac005fd0 tx=0x7fa5ac00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:43:20.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.233+0000 7fa5bb7fe640 1 -- 172.21.15.97:0/563832314 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac017070 con 0x7fa5c4106190 2024-04-23T10:43:20.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.233+0000 7fa5c88bb640 1 -- 172.21.15.97:0/563832314 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5c41a4fd0 con 0x7fa5c4106190 2024-04-23T10:43:20.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.233+0000 7fa5bb7fe640 1 -- 172.21.15.97:0/563832314 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5ac009c30 con 0x7fa5c4106190 2024-04-23T10:43:20.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.233+0000 7fa5bb7fe640 1 -- 172.21.15.97:0/563832314 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac003d30 con 0x7fa5c4106190 2024-04-23T10:43:20.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.233+0000 7fa5c88bb640 1 -- 172.21.15.97:0/563832314 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5c41a54f0 con 0x7fa5c4106190 2024-04-23T10:43:20.239 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.234+0000 7fa5c88bb640 1 -- 172.21.15.97:0/563832314 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5c40691c0 con 0x7fa5c4106190 2024-04-23T10:43:20.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.235+0000 7fa5bb7fe640 1 -- 172.21.15.97:0/563832314 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa5ac003ed0 con 0x7fa5c4106190 2024-04-23T10:43:20.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.235+0000 7fa5bb7fe640 1 --2- 172.21.15.97:0/563832314 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fa5b00a3d20 0x7fa5b00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:43:20.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.236+0000 7fa5c259c640 1 --2- 172.21.15.97:0/563832314 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fa5b00a3d20 0x7fa5b00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:43:20.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.236+0000 7fa5bb7fe640 1 -- 172.21.15.97:0/563832314 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8484+0+0 (secure 0 0 0) 0x7fa5ac0d04d0 con 0x7fa5c4106190 2024-04-23T10:43:20.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.236+0000 7fa5c259c640 1 --2- 172.21.15.97:0/563832314 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fa5b00a3d20 0x7fa5b00a61e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4031440 tx=0x7fa5b40095f0 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:43:20.240 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.240+0000 7fa5bb7fe640 1 -- 172.21.15.97:0/563832314 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa5ac020030 con 0x7fa5c4106190 2024-04-23T10:43:20.453 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.454+0000 7fa5c88bb640 1 -- 172.21.15.97:0/563832314 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fa5c4101a70 con 0x7fa5c4106190 2024-04-23T10:43:20.455 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:43:20 smithi097 ceph-mon[29197]: pgmap v539: 97 pgs: 97 active+clean; 4.5 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T10:43:20.456 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:43:20 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/507335830' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:43:20.456 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:43:20 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/4036975303' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:43:20.456 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:43:20 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/31518302' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T10:43:20.456 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.457+0000 7fa5bb7fe640 1 -- 172.21.15.97:0/563832314 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v27) ==== 161+0+0 (secure 0 0 0) 0x7fa5ac009de0 con 0x7fa5c4106190 2024-04-23T10:43:20.456 INFO:teuthology.orchestra.run.smithi097.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-23T10:43:20.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.460+0000 7fa5c88bb640 1 -- 172.21.15.97:0/563832314 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fa5b00a3d20 msgr2=0x7fa5b00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:43:20.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.460+0000 7fa5c88bb640 1 --2- 172.21.15.97:0/563832314 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fa5b00a3d20 0x7fa5b00a61e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4031440 tx=0x7fa5b40095f0 comp rx=0 tx=0).stop 2024-04-23T10:43:20.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.460+0000 7fa5c88bb640 1 -- 172.21.15.97:0/563832314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5c4106190 msgr2=0x7fa5c4199a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:43:20.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.460+0000 7fa5c88bb640 1 --2- 172.21.15.97:0/563832314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5c4106190 0x7fa5c4199a30 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac005fd0 tx=0x7fa5ac00dde0 comp rx=0 tx=0).stop 2024-04-23T10:43:20.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.460+0000 7fa5c88bb640 1 -- 172.21.15.97:0/563832314 shutdown_connections 2024-04-23T10:43:20.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.460+0000 7fa5c88bb640 1 --2- 172.21.15.97:0/563832314 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fa5b00a3d20 0x7fa5b00a61e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:20.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.460+0000 7fa5c88bb640 1 --2- 172.21.15.97:0/563832314 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa5c4199fc0 0x7fa5c41a4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:20.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.460+0000 7fa5c88bb640 1 --2- 172.21.15.97:0/563832314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa5c4106190 0x7fa5c4199a30 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:20.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.460+0000 7fa5c88bb640 1 --2- 172.21.15.97:0/563832314 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5c41010c0 0x7fa5c41994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:20.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.460+0000 7fa5c88bb640 1 -- 172.21.15.97:0/563832314 >> 172.21.15.97:0/563832314 conn(0x7fa5c40771d0 msgr2=0x7fa5c40fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:43:20.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.460+0000 7fa5c88bb640 1 -- 172.21.15.97:0/563832314 shutdown_connections 2024-04-23T10:43:20.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.460+0000 7fa5c88bb640 1 -- 172.21.15.97:0/563832314 wait complete. 2024-04-23T10:43:20.472 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-23T10:43:20.501 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:43:20 smithi196 ceph-mon[32919]: pgmap v539: 97 pgs: 97 active+clean; 4.5 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T10:43:20.501 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:43:20 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/507335830' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:43:20.501 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:43:20 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/4036975303' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:43:20.501 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:43:20 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/31518302' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T10:43:20.502 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:43:20 smithi183 ceph-mon[32317]: pgmap v539: 97 pgs: 97 active+clean; 4.5 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T10:43:20.502 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:43:20 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/507335830' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:43:20.502 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:43:20 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/4036975303' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T10:43:20.502 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:43:20 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/31518302' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T10:43:20.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.579+0000 7f5cfee91640 1 -- 172.21.15.97:0/3342584083 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cf8101090 msgr2=0x7f5cf8101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:43:20.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.579+0000 7f5cfee91640 1 --2- 172.21.15.97:0/3342584083 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cf8101090 0x7f5cf8101470 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8009900 tx=0x7f5ce802a410 comp rx=0 tx=0).stop 2024-04-23T10:43:20.582 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.579+0000 7f5cfee91640 1 -- 172.21.15.97:0/3342584083 shutdown_connections 2024-04-23T10:43:20.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.579+0000 7f5cfee91640 1 --2- 172.21.15.97:0/3342584083 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5cf8106160 0x7f5cf810dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:20.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.579+0000 7f5cfee91640 1 --2- 172.21.15.97:0/3342584083 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5cf8101a40 0x7f5cf8105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:20.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.579+0000 7f5cfee91640 1 --2- 172.21.15.97:0/3342584083 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cf8101090 0x7f5cf8101470 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:20.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.579+0000 7f5cfee91640 1 -- 172.21.15.97:0/3342584083 >> 172.21.15.97:0/3342584083 conn(0x7f5cf8077f50 msgr2=0x7f5cf8078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:43:20.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.579+0000 7f5cfee91640 1 -- 172.21.15.97:0/3342584083 shutdown_connections 2024-04-23T10:43:20.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.579+0000 7f5cfee91640 1 -- 172.21.15.97:0/3342584083 wait complete. 2024-04-23T10:43:20.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.580+0000 7f5cfee91640 1 Processor -- start 2024-04-23T10:43:20.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.580+0000 7f5cfee91640 1 -- start start 2024-04-23T10:43:20.583 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.580+0000 7f5cfee91640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5cf8101090 0x7f5cf8198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:43:20.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.580+0000 7f5cfee91640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5cf8106160 0x7f5cf8198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:43:20.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.580+0000 7f5cfee91640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cf819cfc0 0x7f5cf819d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:43:20.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.580+0000 7f5cfee91640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf81121e0 con 0x7f5cf819cfc0 2024-04-23T10:43:20.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.580+0000 7f5cfee91640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf8112060 con 0x7f5cf8101090 2024-04-23T10:43:20.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.580+0000 7f5cfee91640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf8112360 con 0x7f5cf8106160 2024-04-23T10:43:20.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.580+0000 7f5cfd42e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cf819cfc0 0x7f5cf819d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:43:20.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.580+0000 7f5cfd42e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cf819cfc0 0x7f5cf819d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40804/0 (socket says 172.21.15.97:40804) 2024-04-23T10:43:20.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.580+0000 7f5cfd42e640 1 -- 172.21.15.97:0/3147293525 learned_addr learned my addr 172.21.15.97:0/3147293525 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:43:20.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.580+0000 7f5cfcc2d640 1 --2- 172.21.15.97:0/3147293525 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5cf8101090 0x7f5cf8198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:43:20.584 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.580+0000 7f5cf7fff640 1 --2- 172.21.15.97:0/3147293525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5cf8106160 0x7f5cf8198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:43:20.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.580+0000 7f5cfd42e640 1 -- 172.21.15.97:0/3147293525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5cf8106160 msgr2=0x7f5cf8198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:43:20.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.580+0000 7f5cfd42e640 1 --2- 172.21.15.97:0/3147293525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5cf8106160 0x7f5cf8198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:20.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.580+0000 7f5cfd42e640 1 -- 172.21.15.97:0/3147293525 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5cf8101090 msgr2=0x7f5cf8198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:43:20.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.580+0000 7f5cfd42e640 1 --2- 172.21.15.97:0/3147293525 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5cf8101090 0x7f5cf8198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:20.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.581+0000 7f5cfd42e640 1 -- 172.21.15.97:0/3147293525 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cf819dab0 con 0x7f5cf819cfc0 2024-04-23T10:43:20.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.581+0000 7f5cfcc2d640 1 --2- 172.21.15.97:0/3147293525 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5cf8101090 0x7f5cf8198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T10:43:20.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.581+0000 7f5cfd42e640 1 --2- 172.21.15.97:0/3147293525 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cf819cfc0 0x7f5cf819d470 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f5cf000cce0 tx=0x7f5cf000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:43:20.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.581+0000 7f5cf5ffb640 1 -- 172.21.15.97:0/3147293525 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf000f450 con 0x7f5cf819cfc0 2024-04-23T10:43:20.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.581+0000 7f5cf5ffb640 1 -- 172.21.15.97:0/3147293525 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cf000faa0 con 0x7f5cf819cfc0 2024-04-23T10:43:20.585 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.581+0000 7f5cfee91640 1 -- 172.21.15.97:0/3147293525 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cf819dda0 con 0x7f5cf819cfc0 2024-04-23T10:43:20.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.581+0000 7f5cf5ffb640 1 -- 172.21.15.97:0/3147293525 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf0018b70 con 0x7f5cf819cfc0 2024-04-23T10:43:20.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.581+0000 7f5cfee91640 1 -- 172.21.15.97:0/3147293525 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cf81a5540 con 0x7f5cf819cfc0 2024-04-23T10:43:20.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.582+0000 7f5cfee91640 1 -- 172.21.15.97:0/3147293525 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cf8068d70 con 0x7f5cf819cfc0 2024-04-23T10:43:20.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.583+0000 7f5cf5ffb640 1 -- 172.21.15.97:0/3147293525 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5cf0018d10 con 0x7f5cf819cfc0 2024-04-23T10:43:20.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.583+0000 7f5cf5ffb640 1 --2- 172.21.15.97:0/3147293525 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f5ce00a3d20 0x7f5ce00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:43:20.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.584+0000 7f5cfcc2d640 1 --2- 172.21.15.97:0/3147293525 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f5ce00a3d20 0x7f5ce00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:43:20.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.584+0000 7f5cf5ffb640 1 -- 172.21.15.97:0/3147293525 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 8870+0+0 (secure 0 0 0) 0x7f5cf00d0e10 con 0x7f5cf819cfc0 2024-04-23T10:43:20.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.584+0000 7f5cfcc2d640 1 --2- 172.21.15.97:0/3147293525 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f5ce00a3d20 0x7f5ce00a61e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8031440 tx=0x7f5ce8005a20 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:43:20.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.588+0000 7f5cf5ffb640 1 -- 172.21.15.97:0/3147293525 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5cf0014030 con 0x7f5cf819cfc0 2024-04-23T10:43:20.804 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:20.805+0000 7f5cfee91640 1 -- 172.21.15.97:0/3147293525 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f5cf8101a40 con 0x7f5cf819cfc0 2024-04-23T10:43:20.841 INFO:journalctl@ceph.mds.f.smithi196.stdout:Apr 23 10:43:20 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-f[63731]: ignoring --setuser ceph since I am not root 2024-04-23T10:43:20.842 INFO:journalctl@ceph.mds.f.smithi196.stdout:Apr 23 10:43:20 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-f[63731]: ignoring --setgroup ceph since I am not root 2024-04-23T10:43:20.842 INFO:journalctl@ceph.mds.f.smithi196.stdout:Apr 23 10:43:20 smithi196 ceph-mds[63735]: -- 172.21.15.196:0/3674713054 <== mon.0 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561bcb515a40 con 0x561bcb4e8800 2024-04-23T10:43:20.842 INFO:journalctl@ceph.mds.f.smithi196.stdout:Apr 23 10:43:20 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-f[63731]: starting mds.f at 2024-04-23T10:43:20.845 INFO:journalctl@ceph.mds.c.smithi196.stdout:Apr 23 10:43:20 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-c[64046]: ignoring --setuser ceph since I am not root 2024-04-23T10:43:20.845 INFO:journalctl@ceph.mds.c.smithi196.stdout:Apr 23 10:43:20 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-c[64046]: ignoring --setgroup ceph since I am not root 2024-04-23T10:43:20.845 INFO:journalctl@ceph.mds.c.smithi196.stdout:Apr 23 10:43:20 smithi196 ceph-mds[64050]: -- 172.21.15.196:0/3442743530 <== mon.0 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e9a7d1ba40 con 0x55e9a7cef000 2024-04-23T10:43:20.845 INFO:journalctl@ceph.mds.c.smithi196.stdout:Apr 23 10:43:20 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-c[64046]: starting mds.c at 2024-04-23T10:43:20.846 INFO:journalctl@ceph.mds.i.smithi196.stdout:Apr 23 10:43:20 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-i[64351]: ignoring --setuser ceph since I am not root 2024-04-23T10:43:20.846 INFO:journalctl@ceph.mds.i.smithi196.stdout:Apr 23 10:43:20 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-i[64351]: ignoring --setgroup ceph since I am not root 2024-04-23T10:43:20.846 INFO:journalctl@ceph.mds.i.smithi196.stdout:Apr 23 10:43:20 smithi196 ceph-mds[64355]: -- 172.21.15.196:0/2885192670 <== mon.0 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55865510fa40 con 0x5586550e2800 2024-04-23T10:43:20.846 INFO:journalctl@ceph.mds.i.smithi196.stdout:Apr 23 10:43:20 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-i[64351]: starting mds.i at 2024-04-23T10:43:20.847 INFO:journalctl@ceph.mds.l.smithi196.stdout:Apr 23 10:43:20 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-l[63409]: ignoring --setuser ceph since I am not root 2024-04-23T10:43:20.847 INFO:journalctl@ceph.mds.l.smithi196.stdout:Apr 23 10:43:20 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-l[63409]: ignoring --setgroup ceph since I am not root 2024-04-23T10:43:20.847 INFO:journalctl@ceph.mds.l.smithi196.stdout:Apr 23 10:43:20 smithi196 ceph-mds[63413]: -- 172.21.15.196:0/1036135757 <== mon.2 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c5c7355a40 con 0x55c5c7328800 2024-04-23T10:43:20.847 INFO:journalctl@ceph.mds.l.smithi196.stdout:Apr 23 10:43:20 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-l[63409]: starting mds.l at 2024-04-23T10:43:20.860 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:43:20 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mon-a[29193]: 2024-04-23T10:43:20.455+0000 7f8b9171d700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-23T10:43:20.861 INFO:journalctl@ceph.mds.a.smithi097.stdout:Apr 23 10:43:20 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-a[68280]: ignoring --setuser ceph since I am not root 2024-04-23T10:43:20.861 INFO:journalctl@ceph.mds.a.smithi097.stdout:Apr 23 10:43:20 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-a[68280]: ignoring --setgroup ceph since I am not root 2024-04-23T10:43:20.861 INFO:journalctl@ceph.mds.a.smithi097.stdout:Apr 23 10:43:20 smithi097 ceph-mds[68284]: -- 172.21.15.97:0/712287251 <== mon.0 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562c13d21a40 con 0x562c13cf5000 2024-04-23T10:43:20.861 INFO:journalctl@ceph.mds.a.smithi097.stdout:Apr 23 10:43:20 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-a[68280]: starting mds.a at 2024-04-23T10:43:20.861 INFO:journalctl@ceph.mds.g.smithi097.stdout:Apr 23 10:43:20 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-g[67920]: ignoring --setuser ceph since I am not root 2024-04-23T10:43:20.861 INFO:journalctl@ceph.mds.g.smithi097.stdout:Apr 23 10:43:20 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-g[67920]: ignoring --setgroup ceph since I am not root 2024-04-23T10:43:20.862 INFO:journalctl@ceph.mds.g.smithi097.stdout:Apr 23 10:43:20 smithi097 ceph-mds[67924]: -- 172.21.15.97:0/2334735592 <== mon.0 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d926d0da40 con 0x55d926ce0c00 2024-04-23T10:43:20.862 INFO:journalctl@ceph.mds.g.smithi097.stdout:Apr 23 10:43:20 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-g[67920]: starting mds.g at 2024-04-23T10:43:20.900 INFO:journalctl@ceph.mds.b.smithi183.stdout:Apr 23 10:43:20 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-b[62562]: ignoring --setuser ceph since I am not root 2024-04-23T10:43:20.900 INFO:journalctl@ceph.mds.b.smithi183.stdout:Apr 23 10:43:20 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-b[62562]: ignoring --setgroup ceph since I am not root 2024-04-23T10:43:20.900 INFO:journalctl@ceph.mds.b.smithi183.stdout:Apr 23 10:43:20 smithi183 ceph-mds[62566]: -- 172.21.15.183:0/833398438 <== mon.2 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555a1c7eba40 con 0x555a1c7bec00 2024-04-23T10:43:20.900 INFO:journalctl@ceph.mds.b.smithi183.stdout:Apr 23 10:43:20 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-b[62562]: starting mds.b at 2024-04-23T10:43:20.901 INFO:journalctl@ceph.mds.e.smithi183.stdout:Apr 23 10:43:20 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-e[61950]: ignoring --setuser ceph since I am not root 2024-04-23T10:43:20.901 INFO:journalctl@ceph.mds.e.smithi183.stdout:Apr 23 10:43:20 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-e[61950]: ignoring --setgroup ceph since I am not root 2024-04-23T10:43:20.901 INFO:journalctl@ceph.mds.e.smithi183.stdout:Apr 23 10:43:20 smithi183 ceph-mds[61954]: -- 172.21.15.183:0/1114450052 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563c7db59a40 con 0x563c7db2c800 2024-04-23T10:43:20.901 INFO:journalctl@ceph.mds.e.smithi183.stdout:Apr 23 10:43:20 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-e[61950]: starting mds.e at 2024-04-23T10:43:20.902 INFO:journalctl@ceph.mds.k.smithi183.stdout:Apr 23 10:43:20 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-k[61625]: ignoring --setuser ceph since I am not root 2024-04-23T10:43:20.902 INFO:journalctl@ceph.mds.k.smithi183.stdout:Apr 23 10:43:20 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-k[61625]: ignoring --setgroup ceph since I am not root 2024-04-23T10:43:20.902 INFO:journalctl@ceph.mds.k.smithi183.stdout:Apr 23 10:43:20 smithi183 ceph-mds[61629]: -- 172.21.15.183:0/419590507 <== mon.2 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5608a5cd3a40 con 0x5608a5ca7000 2024-04-23T10:43:20.902 INFO:journalctl@ceph.mds.k.smithi183.stdout:Apr 23 10:43:20 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-k[61625]: starting mds.k at 2024-04-23T10:43:20.902 INFO:journalctl@ceph.mds.h.smithi183.stdout:Apr 23 10:43:20 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-h[62254]: ignoring --setuser ceph since I am not root 2024-04-23T10:43:20.902 INFO:journalctl@ceph.mds.h.smithi183.stdout:Apr 23 10:43:20 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-h[62254]: ignoring --setgroup ceph since I am not root 2024-04-23T10:43:20.902 INFO:journalctl@ceph.mds.h.smithi183.stdout:Apr 23 10:43:20 smithi183 ceph-mds[62258]: -- 172.21.15.183:0/2669691867 <== mon.0 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563fde345a40 con 0x563fde318c00 2024-04-23T10:43:20.903 INFO:journalctl@ceph.mds.h.smithi183.stdout:Apr 23 10:43:20 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-h[62254]: starting mds.h at 2024-04-23T10:43:21.458 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:21.458+0000 7f5cf5ffb640 1 -- 172.21.15.97:0/3147293525 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v28) ==== 104+0+0 (secure 0 0 0) 0x7f5cf009d350 con 0x7f5cf819cfc0 2024-04-23T10:43:21.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:21.461+0000 7f5cfee91640 1 -- 172.21.15.97:0/3147293525 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f5ce00a3d20 msgr2=0x7f5ce00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:43:21.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:21.461+0000 7f5cfee91640 1 --2- 172.21.15.97:0/3147293525 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f5ce00a3d20 0x7f5ce00a61e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8031440 tx=0x7f5ce8005a20 comp rx=0 tx=0).stop 2024-04-23T10:43:21.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:21.461+0000 7f5cfee91640 1 -- 172.21.15.97:0/3147293525 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cf819cfc0 msgr2=0x7f5cf819d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:43:21.460 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:21.461+0000 7f5cfee91640 1 --2- 172.21.15.97:0/3147293525 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cf819cfc0 0x7f5cf819d470 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f5cf000cce0 tx=0x7f5cf000a620 comp rx=0 tx=0).stop 2024-04-23T10:43:21.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:21.461+0000 7f5cfee91640 1 -- 172.21.15.97:0/3147293525 shutdown_connections 2024-04-23T10:43:21.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:21.461+0000 7f5cfee91640 1 --2- 172.21.15.97:0/3147293525 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f5ce00a3d20 0x7f5ce00a61e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:21.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:21.461+0000 7f5cfee91640 1 --2- 172.21.15.97:0/3147293525 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5cf819cfc0 0x7f5cf819d470 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:21.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:21.461+0000 7f5cfee91640 1 --2- 172.21.15.97:0/3147293525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5cf8106160 0x7f5cf8198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:21.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:21.461+0000 7f5cfee91640 1 --2- 172.21.15.97:0/3147293525 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5cf8101090 0x7f5cf8198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:21.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:21.461+0000 7f5cfee91640 1 -- 172.21.15.97:0/3147293525 >> 172.21.15.97:0/3147293525 conn(0x7f5cf8077f50 msgr2=0x7f5cf80ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:43:21.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:21.461+0000 7f5cfee91640 1 -- 172.21.15.97:0/3147293525 shutdown_connections 2024-04-23T10:43:21.461 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:21.461+0000 7f5cfee91640 1 -- 172.21.15.97:0/3147293525 wait complete. 2024-04-23T10:43:21.475 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:43:21 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/563832314' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-23T10:43:21.475 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:43:21 smithi097 ceph-mon[29197]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-23T10:43:21.475 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:43:21 smithi097 ceph-mon[29197]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-23T10:43:21.475 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:43:21 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/563832314' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-23T10:43:21.475 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:43:21 smithi097 ceph-mon[29197]: osdmap e112: 12 total, 12 up, 12 in 2024-04-23T10:43:21.475 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:43:21 smithi097 ceph-mon[29197]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-23T10:43:21.475 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:43:21 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/3147293525' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-23T10:43:21.477 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-23T10:43:21.586 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:21.584+0000 7fc3d55d8640 1 -- 172.21.15.97:0/3751195660 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3d010c510 msgr2=0x7fc3d010e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:43:21.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:21.584+0000 7fc3d55d8640 1 --2- 172.21.15.97:0/3751195660 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3d010c510 0x7fc3d010e9b0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8009900 tx=0x7fc3c802a510 comp rx=0 tx=0).stop 2024-04-23T10:43:21.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:21.584+0000 7fc3d55d8640 1 -- 172.21.15.97:0/3751195660 shutdown_connections 2024-04-23T10:43:21.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:21.584+0000 7fc3d55d8640 1 --2- 172.21.15.97:0/3751195660 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3d010c510 0x7fc3d010e9b0 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:21.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:21.584+0000 7fc3d55d8640 1 --2- 172.21.15.97:0/3751195660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3d0103af0 0x7fc3d010bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:21.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:21.584+0000 7fc3d55d8640 1 --2- 172.21.15.97:0/3751195660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3d01031d0 0x7fc3d01035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:21.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:21.584+0000 7fc3d55d8640 1 -- 172.21.15.97:0/3751195660 >> 172.21.15.97:0/3751195660 conn(0x7fc3d0077f50 msgr2=0x7fc3d0078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:43:21.587 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:21.584+0000 7fc3d55d8640 1 -- 172.21.15.97:0/3751195660 shutdown_connections 2024-04-23T10:43:21.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:21.584+0000 7fc3d55d8640 1 -- 172.21.15.97:0/3751195660 wait complete. 2024-04-23T10:43:21.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:21.585+0000 7fc3d55d8640 1 Processor -- start 2024-04-23T10:43:21.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:21.585+0000 7fc3d55d8640 1 -- start start 2024-04-23T10:43:21.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:21.585+0000 7fc3d55d8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3d01031d0 0x7fc3d0198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:43:21.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:21.585+0000 7fc3d55d8640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3d0103af0 0x7fc3d0198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:43:21.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:21.585+0000 7fc3d55d8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3d010c510 0x7fc3d019cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:43:21.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:21.585+0000 7fc3d55d8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d0112200 con 0x7fc3d010c510 2024-04-23T10:43:21.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:21.585+0000 7fc3d55d8640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d0112080 con 0x7fc3d0103af0 2024-04-23T10:43:21.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:21.585+0000 7fc3d55d8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d0112380 con 0x7fc3d01031d0 2024-04-23T10:43:21.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:21.585+0000 7fc3cf7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3d010c510 0x7fc3d019cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:43:21.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:21.585+0000 7fc3cf7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3d010c510 0x7fc3d019cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40818/0 (socket says 172.21.15.97:40818) 2024-04-23T10:43:21.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:21.585+0000 7fc3cf7fe640 1 -- 172.21.15.97:0/2102927168 learned_addr learned my addr 172.21.15.97:0/2102927168 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:43:21.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:21.585+0000 7fc3ceffd640 1 --2- 172.21.15.97:0/2102927168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3d01031d0 0x7fc3d0198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:43:21.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:21.586+0000 7fc3ce7fc640 1 --2- 172.21.15.97:0/2102927168 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3d0103af0 0x7fc3d0198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:43:21.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:21.586+0000 7fc3cf7fe640 1 -- 172.21.15.97:0/2102927168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3d01031d0 msgr2=0x7fc3d0198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:43:21.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:21.586+0000 7fc3cf7fe640 1 --2- 172.21.15.97:0/2102927168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3d01031d0 0x7fc3d0198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:21.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:21.586+0000 7fc3cf7fe640 1 -- 172.21.15.97:0/2102927168 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3d0103af0 msgr2=0x7fc3d0198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:43:21.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:21.586+0000 7fc3cf7fe640 1 --2- 172.21.15.97:0/2102927168 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3d0103af0 0x7fc3d0198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:21.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:21.586+0000 7fc3cf7fe640 1 -- 172.21.15.97:0/2102927168 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3d019d6e0 con 0x7fc3d010c510 2024-04-23T10:43:21.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:21.586+0000 7fc3ce7fc640 1 --2- 172.21.15.97:0/2102927168 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3d0103af0 0x7fc3d0198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T10:43:21.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:21.586+0000 7fc3cf7fe640 1 --2- 172.21.15.97:0/2102927168 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3d010c510 0x7fc3d019cfb0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8009a20 tx=0x7fc3c8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:43:21.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:21.586+0000 7fc3b7fff640 1 -- 172.21.15.97:0/2102927168 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c803f030 con 0x7fc3d010c510 2024-04-23T10:43:21.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:21.586+0000 7fc3d55d8640 1 -- 172.21.15.97:0/2102927168 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3d019d970 con 0x7fc3d010c510 2024-04-23T10:43:21.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:21.586+0000 7fc3d55d8640 1 -- 172.21.15.97:0/2102927168 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3d01a5260 con 0x7fc3d010c510 2024-04-23T10:43:21.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:21.586+0000 7fc3b7fff640 1 -- 172.21.15.97:0/2102927168 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3c8003aa0 con 0x7fc3d010c510 2024-04-23T10:43:21.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:21.586+0000 7fc3b7fff640 1 -- 172.21.15.97:0/2102927168 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c8049060 con 0x7fc3d010c510 2024-04-23T10:43:21.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:21.587+0000 7fc3d55d8640 1 -- 172.21.15.97:0/2102927168 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3d0068d70 con 0x7fc3d010c510 2024-04-23T10:43:21.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:21.588+0000 7fc3b7fff640 1 -- 172.21.15.97:0/2102927168 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc3c8042030 con 0x7fc3d010c510 2024-04-23T10:43:21.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:21.589+0000 7fc3b7fff640 1 --2- 172.21.15.97:0/2102927168 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fc3b80a3d20 0x7fc3b80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:43:21.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:21.589+0000 7fc3ceffd640 1 --2- 172.21.15.97:0/2102927168 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fc3b80a3d20 0x7fc3b80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:43:21.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:21.589+0000 7fc3b7fff640 1 -- 172.21.15.97:0/2102927168 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 8870+0+0 (secure 0 0 0) 0x7fc3c80f0e60 con 0x7fc3d010c510 2024-04-23T10:43:21.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:21.589+0000 7fc3ceffd640 1 --2- 172.21.15.97:0/2102927168 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fc3b80a3d20 0x7fc3b80a61e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc3c0007350 tx=0x7fc3c00072e0 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:43:21.593 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:43:21 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/563832314' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-23T10:43:21.593 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:43:21 smithi196 ceph-mon[32919]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-23T10:43:21.593 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:43:21 smithi196 ceph-mon[32919]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-23T10:43:21.593 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:43:21 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/563832314' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-23T10:43:21.593 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:43:21 smithi196 ceph-mon[32919]: osdmap e112: 12 total, 12 up, 12 in 2024-04-23T10:43:21.593 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:43:21 smithi196 ceph-mon[32919]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-23T10:43:21.593 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:43:21 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/3147293525' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-23T10:43:21.594 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:21.594+0000 7fc3b7fff640 1 -- 172.21.15.97:0/2102927168 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc3c80bd3a0 con 0x7fc3d010c510 2024-04-23T10:43:21.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:43:21 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/563832314' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-23T10:43:21.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:43:21 smithi183 ceph-mon[32317]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-23T10:43:21.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:43:21 smithi183 ceph-mon[32317]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-23T10:43:21.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:43:21 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/563832314' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-23T10:43:21.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:43:21 smithi183 ceph-mon[32317]: osdmap e112: 12 total, 12 up, 12 in 2024-04-23T10:43:21.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:43:21 smithi183 ceph-mon[32317]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-23T10:43:21.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:43:21 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/3147293525' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-23T10:43:21.773 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:21.774+0000 7fc3d55d8640 1 -- 172.21.15.97:0/2102927168 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc3d01076f0 con 0x7fc3d010c510 2024-04-23T10:43:22.459 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:22.459+0000 7fc3b7fff640 1 -- 172.21.15.97:0/2102927168 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v113) ==== 181+0+0 (secure 0 0 0) 0x7fc3c80bcca0 con 0x7fc3d010c510 2024-04-23T10:43:22.459 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_metadata' removed 2024-04-23T10:43:22.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:22.463+0000 7fc3d55d8640 1 -- 172.21.15.97:0/2102927168 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fc3b80a3d20 msgr2=0x7fc3b80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:43:22.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:22.463+0000 7fc3d55d8640 1 --2- 172.21.15.97:0/2102927168 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fc3b80a3d20 0x7fc3b80a61e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc3c0007350 tx=0x7fc3c00072e0 comp rx=0 tx=0).stop 2024-04-23T10:43:22.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:22.463+0000 7fc3d55d8640 1 -- 172.21.15.97:0/2102927168 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3d010c510 msgr2=0x7fc3d019cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:43:22.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:22.463+0000 7fc3d55d8640 1 --2- 172.21.15.97:0/2102927168 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3d010c510 0x7fc3d019cfb0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8009a20 tx=0x7fc3c8004270 comp rx=0 tx=0).stop 2024-04-23T10:43:22.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:22.464+0000 7fc3d55d8640 1 -- 172.21.15.97:0/2102927168 shutdown_connections 2024-04-23T10:43:22.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:22.464+0000 7fc3d55d8640 1 --2- 172.21.15.97:0/2102927168 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7fc3b80a3d20 0x7fc3b80a61e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:22.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:22.464+0000 7fc3d55d8640 1 --2- 172.21.15.97:0/2102927168 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3d010c510 0x7fc3d019cfb0 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:22.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:22.464+0000 7fc3d55d8640 1 --2- 172.21.15.97:0/2102927168 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3d0103af0 0x7fc3d0198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:22.463 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:22.464+0000 7fc3d55d8640 1 --2- 172.21.15.97:0/2102927168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3d01031d0 0x7fc3d0198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:22.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:22.464+0000 7fc3d55d8640 1 -- 172.21.15.97:0/2102927168 >> 172.21.15.97:0/2102927168 conn(0x7fc3d0077f50 msgr2=0x7fc3d00fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:43:22.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:22.464+0000 7fc3d55d8640 1 -- 172.21.15.97:0/2102927168 shutdown_connections 2024-04-23T10:43:22.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:22.464+0000 7fc3d55d8640 1 -- 172.21.15.97:0/2102927168 wait complete. 2024-04-23T10:43:22.478 DEBUG:teuthology.orchestra.run.smithi097:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-23T10:43:22.501 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:43:22 smithi097 ceph-mon[29197]: pgmap v541: 97 pgs: 97 active+clean; 4.5 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T10:43:22.501 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:43:22 smithi097 ceph-mon[29197]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-23T10:43:22.501 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:43:22 smithi097 ceph-mon[29197]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-23T10:43:22.501 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:43:22 smithi097 ceph-mon[29197]: Cluster is now healthy 2024-04-23T10:43:22.501 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:43:22 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/3147293525' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-23T10:43:22.501 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:43:22 smithi097 ceph-mon[29197]: mds.? [v2:172.21.15.97:6838/4218517567,v1:172.21.15.97:6840/4218517567] up:boot 2024-04-23T10:43:22.501 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:43:22 smithi097 ceph-mon[29197]: mds.? [v2:172.21.15.97:6839/4126847460,v1:172.21.15.97:6841/4126847460] up:boot 2024-04-23T10:43:22.501 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:43:22 smithi097 ceph-mon[29197]: mds.? [v2:172.21.15.196:6832/2476590192,v1:172.21.15.196:6835/2476590192] up:boot 2024-04-23T10:43:22.501 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:43:22 smithi097 ceph-mon[29197]: mds.? [v2:172.21.15.196:6833/2441655,v1:172.21.15.196:6834/2441655] up:boot 2024-04-23T10:43:22.502 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:43:22 smithi097 ceph-mon[29197]: mds.? [v2:172.21.15.183:6832/268488257,v1:172.21.15.183:6834/268488257] up:boot 2024-04-23T10:43:22.502 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:43:22 smithi097 ceph-mon[29197]: mds.? [v2:172.21.15.183:6833/3614972857,v1:172.21.15.183:6835/3614972857] up:boot 2024-04-23T10:43:22.502 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:43:22 smithi097 ceph-mon[29197]: mds.? [v2:172.21.15.183:6837/215866972,v1:172.21.15.183:6839/215866972] up:boot 2024-04-23T10:43:22.502 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:43:22 smithi097 ceph-mon[29197]: mds.? [v2:172.21.15.183:6836/877984005,v1:172.21.15.183:6838/877984005] up:boot 2024-04-23T10:43:22.502 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:43:22 smithi097 ceph-mon[29197]: mds.? [v2:172.21.15.196:6836/3812475403,v1:172.21.15.196:6838/3812475403] up:boot 2024-04-23T10:43:22.502 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:43:22 smithi097 ceph-mon[29197]: mds.? [v2:172.21.15.196:6837/3502017320,v1:172.21.15.196:6839/3502017320] up:boot 2024-04-23T10:43:22.502 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:43:22 smithi097 ceph-mon[29197]: fsmap 12 up:standby 2024-04-23T10:43:22.502 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:43:22 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-23T10:43:22.502 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:43:22 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-23T10:43:22.502 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:43:22 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-23T10:43:22.502 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:43:22 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-23T10:43:22.502 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:43:22 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-23T10:43:22.503 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:43:22 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-23T10:43:22.503 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:43:22 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-23T10:43:22.503 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:43:22 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-23T10:43:22.503 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:43:22 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-23T10:43:22.503 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:43:22 smithi097 ceph-mon[29197]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-23T10:43:22.503 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:43:22 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/2102927168' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-23T10:43:22.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:22.586+0000 7f48eb795640 1 -- 172.21.15.97:0/960906841 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48e4103af0 msgr2=0x7f48e410bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:43:22.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:22.586+0000 7f48eb795640 1 --2- 172.21.15.97:0/960906841 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48e4103af0 0x7f48e410bfd0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f48d4009900 tx=0x7f48d402a510 comp rx=0 tx=0).stop 2024-04-23T10:43:22.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:22.586+0000 7f48eb795640 1 -- 172.21.15.97:0/960906841 shutdown_connections 2024-04-23T10:43:22.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:22.586+0000 7f48eb795640 1 --2- 172.21.15.97:0/960906841 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48e410c510 0x7f48e410e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:22.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:22.586+0000 7f48eb795640 1 --2- 172.21.15.97:0/960906841 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48e4103af0 0x7f48e410bfd0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:22.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:22.586+0000 7f48eb795640 1 --2- 172.21.15.97:0/960906841 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f48e41031d0 0x7f48e41035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:22.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:22.586+0000 7f48eb795640 1 -- 172.21.15.97:0/960906841 >> 172.21.15.97:0/960906841 conn(0x7f48e4077f50 msgr2=0x7f48e4078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:43:22.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:22.587+0000 7f48eb795640 1 -- 172.21.15.97:0/960906841 shutdown_connections 2024-04-23T10:43:22.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:22.587+0000 7f48eb795640 1 -- 172.21.15.97:0/960906841 wait complete. 2024-04-23T10:43:22.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:22.587+0000 7f48eb795640 1 Processor -- start 2024-04-23T10:43:22.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:22.587+0000 7f48eb795640 1 -- start start 2024-04-23T10:43:22.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:22.588+0000 7f48eb795640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f48e41031d0 0x7f48e41a3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:43:22.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:22.588+0000 7f48eb795640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48e4103af0 0x7f48e41a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:43:22.589 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:22.588+0000 7f48eb795640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48e410c510 0x7f48e410b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:43:22.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:22.588+0000 7f48eb795640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f48e4112200 con 0x7f48e410c510 2024-04-23T10:43:22.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:22.588+0000 7f48eb795640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f48e4112080 con 0x7f48e41031d0 2024-04-23T10:43:22.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:22.588+0000 7f48eb795640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 -- 0x7f48e4112380 con 0x7f48e4103af0 2024-04-23T10:43:22.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:22.588+0000 7f48e9d32640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48e410c510 0x7f48e410b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:43:22.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:22.588+0000 7f48e9d32640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48e410c510 0x7f48e410b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:40846/0 (socket says 172.21.15.97:40846) 2024-04-23T10:43:22.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:22.588+0000 7f48e9d32640 1 -- 172.21.15.97:0/3039097284 learned_addr learned my addr 172.21.15.97:0/3039097284 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-23T10:43:22.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:22.588+0000 7f48e8d30640 1 --2- 172.21.15.97:0/3039097284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48e4103af0 0x7f48e41a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:43:22.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:22.588+0000 7f48e9531640 1 --2- 172.21.15.97:0/3039097284 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f48e41031d0 0x7f48e41a3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:43:22.590 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:22.588+0000 7f48e9d32640 1 -- 172.21.15.97:0/3039097284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48e4103af0 msgr2=0x7f48e41a3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:43:22.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:22.588+0000 7f48e9d32640 1 --2- 172.21.15.97:0/3039097284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48e4103af0 0x7f48e41a3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:22.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:22.588+0000 7f48e9d32640 1 -- 172.21.15.97:0/3039097284 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f48e41031d0 msgr2=0x7f48e41a3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:43:22.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:22.588+0000 7f48e9d32640 1 --2- 172.21.15.97:0/3039097284 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f48e41031d0 0x7f48e41a3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:22.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:22.588+0000 7f48e9d32640 1 -- 172.21.15.97:0/3039097284 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48e410ba90 con 0x7f48e410c510 2024-04-23T10:43:22.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:22.588+0000 7f48e9d32640 1 --2- 172.21.15.97:0/3039097284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48e410c510 0x7f48e410b310 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f48dc00c970 tx=0x7f48dc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:43:22.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:22.589+0000 7f48da7fc640 1 -- 172.21.15.97:0/3039097284 <== mon.0 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48dc007bb0 con 0x7f48e410c510 2024-04-23T10:43:22.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:22.589+0000 7f48da7fc640 1 -- 172.21.15.97:0/3039097284 <== mon.0 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48dc007d50 con 0x7f48e410c510 2024-04-23T10:43:22.591 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:22.589+0000 7f48eb795640 1 -- 172.21.15.97:0/3039097284 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48e410bd80 con 0x7f48e410c510 2024-04-23T10:43:22.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:22.589+0000 7f48da7fc640 1 -- 172.21.15.97:0/3039097284 <== mon.0 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48dc004e60 con 0x7f48e410c510 2024-04-23T10:43:22.592 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:22.589+0000 7f48eb795640 1 -- 172.21.15.97:0/3039097284 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48e41ab9a0 con 0x7f48e410c510 2024-04-23T10:43:22.593 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:43:22 smithi196 ceph-mon[32919]: pgmap v541: 97 pgs: 97 active+clean; 4.5 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T10:43:22.593 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:43:22 smithi196 ceph-mon[32919]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-23T10:43:22.593 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:43:22 smithi196 ceph-mon[32919]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-23T10:43:22.593 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:43:22 smithi196 ceph-mon[32919]: Cluster is now healthy 2024-04-23T10:43:22.593 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:43:22 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/3147293525' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-23T10:43:22.593 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:43:22 smithi196 ceph-mon[32919]: mds.? [v2:172.21.15.97:6838/4218517567,v1:172.21.15.97:6840/4218517567] up:boot 2024-04-23T10:43:22.593 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:43:22 smithi196 ceph-mon[32919]: mds.? [v2:172.21.15.97:6839/4126847460,v1:172.21.15.97:6841/4126847460] up:boot 2024-04-23T10:43:22.593 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:43:22 smithi196 ceph-mon[32919]: mds.? [v2:172.21.15.196:6832/2476590192,v1:172.21.15.196:6835/2476590192] up:boot 2024-04-23T10:43:22.593 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:43:22 smithi196 ceph-mon[32919]: mds.? [v2:172.21.15.196:6833/2441655,v1:172.21.15.196:6834/2441655] up:boot 2024-04-23T10:43:22.594 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:43:22 smithi196 ceph-mon[32919]: mds.? [v2:172.21.15.183:6832/268488257,v1:172.21.15.183:6834/268488257] up:boot 2024-04-23T10:43:22.594 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:43:22 smithi196 ceph-mon[32919]: mds.? [v2:172.21.15.183:6833/3614972857,v1:172.21.15.183:6835/3614972857] up:boot 2024-04-23T10:43:22.594 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:43:22 smithi196 ceph-mon[32919]: mds.? [v2:172.21.15.183:6837/215866972,v1:172.21.15.183:6839/215866972] up:boot 2024-04-23T10:43:22.594 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:43:22 smithi196 ceph-mon[32919]: mds.? [v2:172.21.15.183:6836/877984005,v1:172.21.15.183:6838/877984005] up:boot 2024-04-23T10:43:22.594 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:43:22 smithi196 ceph-mon[32919]: mds.? [v2:172.21.15.196:6836/3812475403,v1:172.21.15.196:6838/3812475403] up:boot 2024-04-23T10:43:22.594 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:43:22 smithi196 ceph-mon[32919]: mds.? [v2:172.21.15.196:6837/3502017320,v1:172.21.15.196:6839/3502017320] up:boot 2024-04-23T10:43:22.594 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:43:22 smithi196 ceph-mon[32919]: fsmap 12 up:standby 2024-04-23T10:43:22.594 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:43:22 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-23T10:43:22.594 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:43:22 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-23T10:43:22.594 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:43:22 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-23T10:43:22.595 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:43:22 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-23T10:43:22.595 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:43:22 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-23T10:43:22.595 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:43:22 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-23T10:43:22.595 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:43:22 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-23T10:43:22.595 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:43:22 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-23T10:43:22.595 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:43:22 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-23T10:43:22.595 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:43:22 smithi196 ceph-mon[32919]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-23T10:43:22.595 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:43:22 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/2102927168' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-23T10:43:22.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:22.591+0000 7f48eb795640 1 -- 172.21.15.97:0/3039097284 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48e4068d70 con 0x7f48e410c510 2024-04-23T10:43:22.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:22.592+0000 7f48da7fc640 1 -- 172.21.15.97:0/3039097284 <== mon.0 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f48dc005020 con 0x7f48e410c510 2024-04-23T10:43:22.596 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:22.593+0000 7f48da7fc640 1 --2- 172.21.15.97:0/3039097284 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f48d00a3e70 0x7f48d00a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T10:43:22.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:22.593+0000 7f48e9531640 1 --2- 172.21.15.97:0/3039097284 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f48d00a3e70 0x7f48d00a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T10:43:22.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:22.593+0000 7f48da7fc640 1 -- 172.21.15.97:0/3039097284 <== mon.0 v2:172.21.15.97:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8316+0+0 (secure 0 0 0) 0x7f48dc0d0250 con 0x7f48e410c510 2024-04-23T10:43:22.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:22.594+0000 7f48e9531640 1 --2- 172.21.15.97:0/3039097284 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f48d00a3e70 0x7f48d00a6330 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f48e41a04c0 tx=0x7f48e000d040 comp rx=0 tx=0).ready entity=mgr.15378 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T10:43:22.597 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:22.598+0000 7f48da7fc640 1 -- 172.21.15.97:0/3039097284 <== mon.0 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f48dc014030 con 0x7f48e410c510 2024-04-23T10:43:22.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:43:22 smithi183 ceph-mon[32317]: pgmap v541: 97 pgs: 97 active+clean; 4.5 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T10:43:22.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:43:22 smithi183 ceph-mon[32317]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-23T10:43:22.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:43:22 smithi183 ceph-mon[32317]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-23T10:43:22.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:43:22 smithi183 ceph-mon[32317]: Cluster is now healthy 2024-04-23T10:43:22.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:43:22 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/3147293525' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-23T10:43:22.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:43:22 smithi183 ceph-mon[32317]: mds.? [v2:172.21.15.97:6838/4218517567,v1:172.21.15.97:6840/4218517567] up:boot 2024-04-23T10:43:22.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:43:22 smithi183 ceph-mon[32317]: mds.? [v2:172.21.15.97:6839/4126847460,v1:172.21.15.97:6841/4126847460] up:boot 2024-04-23T10:43:22.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:43:22 smithi183 ceph-mon[32317]: mds.? [v2:172.21.15.196:6832/2476590192,v1:172.21.15.196:6835/2476590192] up:boot 2024-04-23T10:43:22.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:43:22 smithi183 ceph-mon[32317]: mds.? [v2:172.21.15.196:6833/2441655,v1:172.21.15.196:6834/2441655] up:boot 2024-04-23T10:43:22.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:43:22 smithi183 ceph-mon[32317]: mds.? [v2:172.21.15.183:6832/268488257,v1:172.21.15.183:6834/268488257] up:boot 2024-04-23T10:43:22.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:43:22 smithi183 ceph-mon[32317]: mds.? [v2:172.21.15.183:6833/3614972857,v1:172.21.15.183:6835/3614972857] up:boot 2024-04-23T10:43:22.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:43:22 smithi183 ceph-mon[32317]: mds.? [v2:172.21.15.183:6837/215866972,v1:172.21.15.183:6839/215866972] up:boot 2024-04-23T10:43:22.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:43:22 smithi183 ceph-mon[32317]: mds.? [v2:172.21.15.183:6836/877984005,v1:172.21.15.183:6838/877984005] up:boot 2024-04-23T10:43:22.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:43:22 smithi183 ceph-mon[32317]: mds.? [v2:172.21.15.196:6836/3812475403,v1:172.21.15.196:6838/3812475403] up:boot 2024-04-23T10:43:22.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:43:22 smithi183 ceph-mon[32317]: mds.? [v2:172.21.15.196:6837/3502017320,v1:172.21.15.196:6839/3502017320] up:boot 2024-04-23T10:43:22.651 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:43:22 smithi183 ceph-mon[32317]: fsmap 12 up:standby 2024-04-23T10:43:22.652 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:43:22 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-23T10:43:22.652 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:43:22 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-23T10:43:22.652 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:43:22 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-23T10:43:22.652 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:43:22 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-23T10:43:22.652 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:43:22 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-23T10:43:22.652 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:43:22 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-23T10:43:22.652 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:43:22 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-23T10:43:22.652 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:43:22 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-23T10:43:22.652 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:43:22 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-23T10:43:22.652 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:43:22 smithi183 ceph-mon[32317]: from='mgr.15378 172.21.15.97:0/3632722328' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-23T10:43:22.652 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:43:22 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/2102927168' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-23T10:43:22.772 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:22.774+0000 7f48eb795640 1 -- 172.21.15.97:0/3039097284 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f48e41abc30 con 0x7f48e410c510 2024-04-23T10:43:23.464 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:23.465+0000 7f48da7fc640 1 -- 172.21.15.97:0/3039097284 <== mon.0 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v114) ==== 169+0+0 (secure 0 0 0) 0x7f48dc09c370 con 0x7f48e410c510 2024-04-23T10:43:23.464 INFO:teuthology.orchestra.run.smithi097.stderr:pool 'cephfs_data' removed 2024-04-23T10:43:23.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:23.468+0000 7f48eb795640 1 -- 172.21.15.97:0/3039097284 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f48d00a3e70 msgr2=0x7f48d00a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:43:23.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:23.468+0000 7f48eb795640 1 --2- 172.21.15.97:0/3039097284 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f48d00a3e70 0x7f48d00a6330 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f48e41a04c0 tx=0x7f48e000d040 comp rx=0 tx=0).stop 2024-04-23T10:43:23.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:23.468+0000 7f48eb795640 1 -- 172.21.15.97:0/3039097284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48e410c510 msgr2=0x7f48e410b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T10:43:23.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:23.468+0000 7f48eb795640 1 --2- 172.21.15.97:0/3039097284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48e410c510 0x7f48e410b310 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f48dc00c970 tx=0x7f48dc00ce30 comp rx=0 tx=0).stop 2024-04-23T10:43:23.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:23.468+0000 7f48eb795640 1 -- 172.21.15.97:0/3039097284 shutdown_connections 2024-04-23T10:43:23.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:23.468+0000 7f48eb795640 1 --2- 172.21.15.97:0/3039097284 >> [v2:172.21.15.97:6800/3567268502,v1:172.21.15.97:6801/3567268502] conn(0x7f48d00a3e70 0x7f48d00a6330 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:23.468 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:23.468+0000 7f48eb795640 1 --2- 172.21.15.97:0/3039097284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48e410c510 0x7f48e410b310 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:23.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:23.468+0000 7f48eb795640 1 --2- 172.21.15.97:0/3039097284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48e4103af0 0x7f48e41a3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:23.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:23.468+0000 7f48eb795640 1 --2- 172.21.15.97:0/3039097284 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f48e41031d0 0x7f48e41a3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T10:43:23.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:23.468+0000 7f48eb795640 1 -- 172.21.15.97:0/3039097284 >> 172.21.15.97:0/3039097284 conn(0x7f48e4077f50 msgr2=0x7f48e40ff180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T10:43:23.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:23.469+0000 7f48eb795640 1 -- 172.21.15.97:0/3039097284 shutdown_connections 2024-04-23T10:43:23.469 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-23T10:43:23.469+0000 7f48eb795640 1 -- 172.21.15.97:0/3039097284 wait complete. 2024-04-23T10:43:23.484 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-23T10:43:23.554 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-23T10:43:23.584 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-23T10:43:23.614 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-23T10:43:23.614 DEBUG:teuthology.orchestra.run.smithi097:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-23T10:43:23.631 DEBUG:teuthology.orchestra.run.smithi183:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-23T10:43:23.647 DEBUG:teuthology.orchestra.run.smithi196:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-23T10:43:23.670 INFO:tasks.cephadm:Stopping all daemons... 2024-04-23T10:43:23.670 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-23T10:43:23.671 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-9b22af88-0159-11ef-bc93-c7b262605968@mon.a 2024-04-23T10:43:23.716 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:43:23 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/2102927168' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-23T10:43:23.716 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:43:23 smithi097 ceph-mon[29197]: osdmap e113: 12 total, 12 up, 12 in 2024-04-23T10:43:23.716 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:43:23 smithi097 ceph-mon[29197]: from='client.? 172.21.15.97:0/3039097284' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-23T10:43:23.841 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:43:23 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/2102927168' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-23T10:43:23.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:43:23 smithi196 ceph-mon[32919]: osdmap e113: 12 total, 12 up, 12 in 2024-04-23T10:43:23.842 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:43:23 smithi196 ceph-mon[32919]: from='client.? 172.21.15.97:0/3039097284' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-23T10:43:23.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:43:23 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/2102927168' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-23T10:43:23.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:43:23 smithi183 ceph-mon[32317]: osdmap e113: 12 total, 12 up, 12 in 2024-04-23T10:43:23.900 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:43:23 smithi183 ceph-mon[32317]: from='client.? 172.21.15.97:0/3039097284' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-23T10:43:23.984 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:43:23 smithi097 systemd[1]: Stopping Ceph mon.a for 9b22af88-0159-11ef-bc93-c7b262605968... 2024-04-23T10:43:24.269 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:43:23 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mon-a[29193]: 2024-04-23T10:43:23.985+0000 7f8b96f28700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T10:43:24.269 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:43:23 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mon-a[29193]: 2024-04-23T10:43:23.985+0000 7f8b96f28700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-23T10:43:24.269 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:43:24 smithi097 podman[93780]: 2024-04-23 10:43:24.007490196 +0000 UTC m=+0.141904876 container died f2859bdee5d0a82e12711228b6fbee80a2e446ba32e6b3a320d9737e3d6e2b4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mon-a, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-23T10:43:24.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:43:24 smithi097 podman[93780]: 2024-04-23 10:43:24.387068768 +0000 UTC m=+0.521483453 container cleanup f2859bdee5d0a82e12711228b6fbee80a2e446ba32e6b3a320d9737e3d6e2b4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T10:43:24.611 INFO:journalctl@ceph.mon.a.smithi097.stdout:Apr 23 10:43:24 smithi097 bash[93780]: ceph-9b22af88-0159-11ef-bc93-c7b262605968-mon-a 2024-04-23T10:43:24.787 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b22af88-0159-11ef-bc93-c7b262605968@mon.a.service' 2024-04-23T10:43:24.860 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T10:43:24.860 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-23T10:43:24.860 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-23T10:43:24.860 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-9b22af88-0159-11ef-bc93-c7b262605968@mon.b 2024-04-23T10:43:25.150 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:43:24 smithi183 systemd[1]: Stopping Ceph mon.b for 9b22af88-0159-11ef-bc93-c7b262605968... 2024-04-23T10:43:25.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:43:25 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mon-b[32313]: 2024-04-23T10:43:25.186+0000 7fe31dcbc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T10:43:25.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:43:25 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mon-b[32313]: 2024-04-23T10:43:25.186+0000 7fe31dcbc700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-23T10:43:25.650 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:43:25 smithi183 podman[69495]: 2024-04-23 10:43:25.381690689 +0000 UTC m=+0.321898575 container died 7e8f30ab14b837cdf77364aa38b0cdd8f0e6540c6774d8d257d326a93fbef244 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mon-b, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2) 2024-04-23T10:43:26.070 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:43:25 smithi183 podman[69495]: 2024-04-23 10:43:25.770915687 +0000 UTC m=+0.711123567 container cleanup 7e8f30ab14b837cdf77364aa38b0cdd8f0e6540c6774d8d257d326a93fbef244 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mon-b, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T10:43:26.070 INFO:journalctl@ceph.mon.b.smithi183.stdout:Apr 23 10:43:25 smithi183 bash[69495]: ceph-9b22af88-0159-11ef-bc93-c7b262605968-mon-b 2024-04-23T10:43:26.241 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b22af88-0159-11ef-bc93-c7b262605968@mon.b.service' 2024-04-23T10:43:26.282 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T10:43:26.282 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-23T10:43:26.282 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-23T10:43:26.282 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-9b22af88-0159-11ef-bc93-c7b262605968@mon.c 2024-04-23T10:43:26.591 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:43:26 smithi196 systemd[1]: Stopping Ceph mon.c for 9b22af88-0159-11ef-bc93-c7b262605968... 2024-04-23T10:43:26.953 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:43:26 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mon-c[32902]: 2024-04-23T10:43:26.612+0000 7f37f5fa4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T10:43:26.954 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:43:26 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mon-c[32902]: 2024-04-23T10:43:26.612+0000 7f37f5fa4700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-23T10:43:26.954 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:43:26 smithi196 podman[70603]: 2024-04-23 10:43:26.633124262 +0000 UTC m=+0.150237416 container died 2505a701babd24022931b21bf99653cc140802cf9bdd18f01a30bf8572b99de7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mon-c, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T10:43:27.296 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:43:27 smithi196 podman[70603]: 2024-04-23 10:43:27.054984839 +0000 UTC m=+0.572097985 container cleanup 2505a701babd24022931b21bf99653cc140802cf9bdd18f01a30bf8572b99de7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mon-c, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.license=GPLv2) 2024-04-23T10:43:27.297 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:43:27 smithi196 bash[70603]: ceph-9b22af88-0159-11ef-bc93-c7b262605968-mon-c 2024-04-23T10:43:27.534 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b22af88-0159-11ef-bc93-c7b262605968@mon.c.service' 2024-04-23T10:43:27.555 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:43:27 smithi196 podman[70618]: 2024-04-23 10:43:27.298545514 +0000 UTC m=+0.669388997 container remove 2505a701babd24022931b21bf99653cc140802cf9bdd18f01a30bf8572b99de7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mon-c, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-23T10:43:27.555 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:43:27 smithi196 systemd[1]: ceph-9b22af88-0159-11ef-bc93-c7b262605968@mon.c.service: Deactivated successfully. 2024-04-23T10:43:27.556 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:43:27 smithi196 systemd[1]: Stopped Ceph mon.c for 9b22af88-0159-11ef-bc93-c7b262605968. 2024-04-23T10:43:27.556 INFO:journalctl@ceph.mon.c.smithi196.stdout:Apr 23 10:43:27 smithi196 systemd[1]: ceph-9b22af88-0159-11ef-bc93-c7b262605968@mon.c.service: Consumed 14.935s CPU time. 2024-04-23T10:43:27.576 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T10:43:27.576 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-23T10:43:27.576 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-23T10:43:27.576 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-9b22af88-0159-11ef-bc93-c7b262605968@mgr.x 2024-04-23T10:43:27.860 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:43:27 smithi097 systemd[1]: Stopping Ceph mgr.x for 9b22af88-0159-11ef-bc93-c7b262605968... 2024-04-23T10:43:28.191 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:43:27 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x[29430]: 2024-04-23T10:43:27.882+0000 7ff76e261700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T10:43:28.191 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:43:27 smithi097 podman[93902]: 2024-04-23 10:43:27.91506163 +0000 UTC m=+0.145722533 container died 1f0c379183dda898c9277a28b5be4d44f9b6def4496432ea2a358dfe4af57094 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-23T10:43:28.534 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:43:28 smithi097 podman[93902]: 2024-04-23 10:43:28.301651454 +0000 UTC m=+0.532312374 container cleanup 1f0c379183dda898c9277a28b5be4d44f9b6def4496432ea2a358dfe4af57094 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2) 2024-04-23T10:43:28.534 INFO:journalctl@ceph.mgr.x.smithi097.stdout:Apr 23 10:43:28 smithi097 bash[93902]: ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-x 2024-04-23T10:43:28.658 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b22af88-0159-11ef-bc93-c7b262605968@mgr.x.service' 2024-04-23T10:43:28.692 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T10:43:28.692 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-23T10:43:28.692 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-23T10:43:28.692 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-9b22af88-0159-11ef-bc93-c7b262605968@mgr.y 2024-04-23T10:43:28.900 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 23 10:43:28 smithi183 systemd[1]: Stopping Ceph mgr.y for 9b22af88-0159-11ef-bc93-c7b262605968... 2024-04-23T10:43:29.333 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 23 10:43:29 smithi183 podman[69628]: 2024-04-23 10:43:29.047217613 +0000 UTC m=+0.172935991 container died 6ba1874f12255e0c28bd2da29e055498c4ac5618c6a7e86ce53125b66325ac91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-y, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.license=GPLv2) 2024-04-23T10:43:29.650 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 23 10:43:29 smithi183 podman[69628]: 2024-04-23 10:43:29.417909404 +0000 UTC m=+0.543627762 container cleanup 6ba1874f12255e0c28bd2da29e055498c4ac5618c6a7e86ce53125b66325ac91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416) 2024-04-23T10:43:29.650 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Apr 23 10:43:29 smithi183 bash[69628]: ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-y 2024-04-23T10:43:29.833 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b22af88-0159-11ef-bc93-c7b262605968@mgr.y.service' 2024-04-23T10:43:29.868 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T10:43:29.868 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-23T10:43:29.868 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-23T10:43:29.869 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-9b22af88-0159-11ef-bc93-c7b262605968@mgr.z 2024-04-23T10:43:30.145 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 23 10:43:29 smithi196 systemd[1]: Stopping Ceph mgr.z for 9b22af88-0159-11ef-bc93-c7b262605968... 2024-04-23T10:43:30.519 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 23 10:43:30 smithi196 podman[70735]: 2024-04-23 10:43:30.222688226 +0000 UTC m=+0.141114275 container died 488569418dfaa1f1067b1d681631ea167d2f6c1a66784d8a5825ced715c1f656 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-23T10:43:30.842 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 23 10:43:30 smithi196 podman[70735]: 2024-04-23 10:43:30.619550721 +0000 UTC m=+0.537976772 container cleanup 488569418dfaa1f1067b1d681631ea167d2f6c1a66784d8a5825ced715c1f656 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-z, ceph=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-23T10:43:30.842 INFO:journalctl@ceph.mgr.z.smithi196.stdout:Apr 23 10:43:30 smithi196 bash[70735]: ceph-9b22af88-0159-11ef-bc93-c7b262605968-mgr-z 2024-04-23T10:43:31.031 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b22af88-0159-11ef-bc93-c7b262605968@mgr.z.service' 2024-04-23T10:43:31.068 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T10:43:31.068 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-23T10:43:31.068 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-23T10:43:31.069 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-9b22af88-0159-11ef-bc93-c7b262605968@osd.0 2024-04-23T10:43:31.360 INFO:journalctl@ceph.osd.0.smithi097.stdout:Apr 23 10:43:31 smithi097 systemd[1]: Stopping Ceph osd.0 for 9b22af88-0159-11ef-bc93-c7b262605968... 2024-04-23T10:43:31.860 INFO:journalctl@ceph.osd.0.smithi097.stdout:Apr 23 10:43:31 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-0[41281]: 2024-04-23T10:43:31.380+0000 7f0e68c7a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T10:43:31.860 INFO:journalctl@ceph.osd.0.smithi097.stdout:Apr 23 10:43:31 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-0[41281]: 2024-04-23T10:43:31.380+0000 7f0e68c7a700 -1 osd.0 114 *** Got signal Terminated *** 2024-04-23T10:43:31.860 INFO:journalctl@ceph.osd.0.smithi097.stdout:Apr 23 10:43:31 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-0[41281]: 2024-04-23T10:43:31.380+0000 7f0e68c7a700 -1 osd.0 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T10:43:37.098 INFO:journalctl@ceph.osd.0.smithi097.stdout:Apr 23 10:43:36 smithi097 podman[94026]: 2024-04-23 10:43:36.808665301 +0000 UTC m=+5.546317901 container died 5a27b7127abee203120b49a5d731f6563f2a55ddac2ae664ed7cfae1db1d3207 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416) 2024-04-23T10:43:37.361 INFO:journalctl@ceph.osd.0.smithi097.stdout:Apr 23 10:43:37 smithi097 podman[94026]: 2024-04-23 10:43:37.181425325 +0000 UTC m=+5.919077927 container cleanup 5a27b7127abee203120b49a5d731f6563f2a55ddac2ae664ed7cfae1db1d3207 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-0, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True) 2024-04-23T10:43:37.361 INFO:journalctl@ceph.osd.0.smithi097.stdout:Apr 23 10:43:37 smithi097 bash[94026]: ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-0 2024-04-23T10:43:37.790 INFO:journalctl@ceph.osd.0.smithi097.stdout:Apr 23 10:43:37 smithi097 podman[94043]: 2024-04-23 10:43:37.458023532 +0000 UTC m=+0.648296182 container remove 5a27b7127abee203120b49a5d731f6563f2a55ddac2ae664ed7cfae1db1d3207 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True) 2024-04-23T10:43:38.068 INFO:journalctl@ceph.osd.0.smithi097.stdout:Apr 23 10:43:37 smithi097 podman[94135]: 2024-04-23 10:43:37.695014171 +0000 UTC m=+0.017888892 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T10:43:38.069 INFO:journalctl@ceph.osd.0.smithi097.stdout:Apr 23 10:43:37 smithi097 podman[94135]: 2024-04-23 10:43:37.809137692 +0000 UTC m=+0.132012393 container create 8d4133618ea8c19bf50a9204ebc2d3bbe8abf4435bb1efeaeca619e768161a7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-0-deactivate, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-23T10:43:38.069 INFO:journalctl@ceph.osd.0.smithi097.stdout:Apr 23 10:43:38 smithi097 podman[94135]: 2024-04-23 10:43:38.070265948 +0000 UTC m=+0.393140648 container init 8d4133618ea8c19bf50a9204ebc2d3bbe8abf4435bb1efeaeca619e768161a7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T10:43:38.360 INFO:journalctl@ceph.osd.0.smithi097.stdout:Apr 23 10:43:38 smithi097 podman[94135]: 2024-04-23 10:43:38.074034905 +0000 UTC m=+0.396909606 container start 8d4133618ea8c19bf50a9204ebc2d3bbe8abf4435bb1efeaeca619e768161a7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-23T10:43:38.361 INFO:journalctl@ceph.osd.0.smithi097.stdout:Apr 23 10:43:38 smithi097 podman[94135]: 2024-04-23 10:43:38.161993152 +0000 UTC m=+0.484868031 container attach 8d4133618ea8c19bf50a9204ebc2d3bbe8abf4435bb1efeaeca619e768161a7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS) 2024-04-23T10:43:38.738 INFO:journalctl@ceph.osd.0.smithi097.stdout:Apr 23 10:43:38 smithi097 podman[94135]: 2024-04-23 10:43:38.366315858 +0000 UTC m=+0.689190567 container died 8d4133618ea8c19bf50a9204ebc2d3bbe8abf4435bb1efeaeca619e768161a7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T10:43:39.084 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b22af88-0159-11ef-bc93-c7b262605968@osd.0.service' 2024-04-23T10:43:39.124 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T10:43:39.124 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-23T10:43:39.124 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-23T10:43:39.125 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-9b22af88-0159-11ef-bc93-c7b262605968@osd.1 2024-04-23T10:43:39.437 INFO:journalctl@ceph.osd.1.smithi183.stdout:Apr 23 10:43:39 smithi183 systemd[1]: Stopping Ceph osd.1 for 9b22af88-0159-11ef-bc93-c7b262605968... 2024-04-23T10:43:39.900 INFO:journalctl@ceph.osd.1.smithi183.stdout:Apr 23 10:43:39 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-1[36908]: 2024-04-23T10:43:39.437+0000 7f6748206700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T10:43:39.900 INFO:journalctl@ceph.osd.1.smithi183.stdout:Apr 23 10:43:39 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-1[36908]: 2024-04-23T10:43:39.437+0000 7f6748206700 -1 osd.1 114 *** Got signal Terminated *** 2024-04-23T10:43:39.900 INFO:journalctl@ceph.osd.1.smithi183.stdout:Apr 23 10:43:39 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-1[36908]: 2024-04-23T10:43:39.437+0000 7f6748206700 -1 osd.1 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T10:43:45.281 INFO:journalctl@ceph.osd.1.smithi183.stdout:Apr 23 10:43:44 smithi183 podman[69749]: 2024-04-23 10:43:44.965007637 +0000 UTC m=+5.646980581 container died 0f8de5477269003580108177274cc044a2fc07d3c99cb1662604bec80ea0070a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-1, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2) 2024-04-23T10:43:45.649 INFO:journalctl@ceph.osd.1.smithi183.stdout:Apr 23 10:43:45 smithi183 podman[69749]: 2024-04-23 10:43:45.423540114 +0000 UTC m=+6.105513057 container cleanup 0f8de5477269003580108177274cc044a2fc07d3c99cb1662604bec80ea0070a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-1, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416) 2024-04-23T10:43:45.650 INFO:journalctl@ceph.osd.1.smithi183.stdout:Apr 23 10:43:45 smithi183 bash[69749]: ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-1 2024-04-23T10:43:46.037 INFO:journalctl@ceph.osd.1.smithi183.stdout:Apr 23 10:43:45 smithi183 podman[69767]: 2024-04-23 10:43:45.708454888 +0000 UTC m=+0.746619983 container remove 0f8de5477269003580108177274cc044a2fc07d3c99cb1662604bec80ea0070a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-1, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T10:43:46.326 INFO:journalctl@ceph.osd.1.smithi183.stdout:Apr 23 10:43:46 smithi183 podman[69858]: 2024-04-23 10:43:45.942524967 +0000 UTC m=+0.018371387 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T10:43:46.326 INFO:journalctl@ceph.osd.1.smithi183.stdout:Apr 23 10:43:46 smithi183 podman[69858]: 2024-04-23 10:43:46.067517312 +0000 UTC m=+0.143363724 container create 2deb643c4b5bcd6412e8381f47899dd9c5fa4dfcdaa24e0ec5a1463fc697c029 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=) 2024-04-23T10:43:46.326 INFO:journalctl@ceph.osd.1.smithi183.stdout:Apr 23 10:43:46 smithi183 podman[69858]: 2024-04-23 10:43:46.328873598 +0000 UTC m=+0.404720003 container init 2deb643c4b5bcd6412e8381f47899dd9c5fa4dfcdaa24e0ec5a1463fc697c029 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T10:43:46.619 INFO:journalctl@ceph.osd.1.smithi183.stdout:Apr 23 10:43:46 smithi183 podman[69858]: 2024-04-23 10:43:46.33204937 +0000 UTC m=+0.407895788 container start 2deb643c4b5bcd6412e8381f47899dd9c5fa4dfcdaa24e0ec5a1463fc697c029 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T10:43:46.619 INFO:journalctl@ceph.osd.1.smithi183.stdout:Apr 23 10:43:46 smithi183 podman[69858]: 2024-04-23 10:43:46.462268325 +0000 UTC m=+0.538114738 container attach 2deb643c4b5bcd6412e8381f47899dd9c5fa4dfcdaa24e0ec5a1463fc697c029 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-23T10:43:46.900 INFO:journalctl@ceph.osd.1.smithi183.stdout:Apr 23 10:43:46 smithi183 podman[69858]: 2024-04-23 10:43:46.621759027 +0000 UTC m=+0.697605443 container died 2deb643c4b5bcd6412e8381f47899dd9c5fa4dfcdaa24e0ec5a1463fc697c029 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-23T10:43:47.304 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b22af88-0159-11ef-bc93-c7b262605968@osd.1.service' 2024-04-23T10:43:47.342 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T10:43:47.342 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-23T10:43:47.342 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-23T10:43:47.342 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-9b22af88-0159-11ef-bc93-c7b262605968@osd.2 2024-04-23T10:43:47.788 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 23 10:43:47 smithi196 systemd[1]: Stopping Ceph osd.2 for 9b22af88-0159-11ef-bc93-c7b262605968... 2024-04-23T10:43:48.091 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 23 10:43:47 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-2[38073]: 2024-04-23T10:43:47.789+0000 7f10822bb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T10:43:48.091 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 23 10:43:47 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-2[38073]: 2024-04-23T10:43:47.789+0000 7f10822bb700 -1 osd.2 114 *** Got signal Terminated *** 2024-04-23T10:43:48.092 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 23 10:43:47 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-2[38073]: 2024-04-23T10:43:47.789+0000 7f10822bb700 -1 osd.2 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T10:43:53.548 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 23 10:43:53 smithi196 podman[70864]: 2024-04-23 10:43:53.258289817 +0000 UTC m=+5.719000426 container died f1b76491119e951c32fa4a7585d64f9b6c3a959d910084ea6257f197dc251601 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T10:43:53.841 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 23 10:43:53 smithi196 podman[70864]: 2024-04-23 10:43:53.640512458 +0000 UTC m=+6.101223066 container cleanup f1b76491119e951c32fa4a7585d64f9b6c3a959d910084ea6257f197dc251601 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-2, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-23T10:43:53.842 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 23 10:43:53 smithi196 bash[70864]: ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-2 2024-04-23T10:43:54.267 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 23 10:43:53 smithi196 podman[70879]: 2024-04-23 10:43:53.942132629 +0000 UTC m=+0.684828642 container remove f1b76491119e951c32fa4a7585d64f9b6c3a959d910084ea6257f197dc251601 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T10:43:54.544 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 23 10:43:54 smithi196 podman[70968]: 2024-04-23 10:43:54.171811903 +0000 UTC m=+0.018064358 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T10:43:54.544 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 23 10:43:54 smithi196 podman[70968]: 2024-04-23 10:43:54.301824391 +0000 UTC m=+0.148076879 container create f9041773de04cb49aade685b2938489650fa8c3e2a5ccf76fb682cf44354fffe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-23T10:43:54.841 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 23 10:43:54 smithi196 podman[70968]: 2024-04-23 10:43:54.546288364 +0000 UTC m=+0.392540810 container init f9041773de04cb49aade685b2938489650fa8c3e2a5ccf76fb682cf44354fffe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T10:43:54.841 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 23 10:43:54 smithi196 podman[70968]: 2024-04-23 10:43:54.549550392 +0000 UTC m=+0.395802847 container start f9041773de04cb49aade685b2938489650fa8c3e2a5ccf76fb682cf44354fffe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T10:43:54.842 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 23 10:43:54 smithi196 podman[70968]: 2024-04-23 10:43:54.646361331 +0000 UTC m=+0.492613781 container attach f9041773de04cb49aade685b2938489650fa8c3e2a5ccf76fb682cf44354fffe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416) 2024-04-23T10:43:55.148 INFO:journalctl@ceph.osd.2.smithi196.stdout:Apr 23 10:43:54 smithi196 podman[70968]: 2024-04-23 10:43:54.851593214 +0000 UTC m=+0.697845661 container died f9041773de04cb49aade685b2938489650fa8c3e2a5ccf76fb682cf44354fffe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True) 2024-04-23T10:43:55.477 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b22af88-0159-11ef-bc93-c7b262605968@osd.2.service' 2024-04-23T10:43:55.516 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T10:43:55.516 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-23T10:43:55.516 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-23T10:43:55.516 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-9b22af88-0159-11ef-bc93-c7b262605968@osd.3 2024-04-23T10:43:55.834 INFO:journalctl@ceph.osd.3.smithi097.stdout:Apr 23 10:43:55 smithi097 systemd[1]: Stopping Ceph osd.3 for 9b22af88-0159-11ef-bc93-c7b262605968... 2024-04-23T10:43:56.110 INFO:journalctl@ceph.osd.3.smithi097.stdout:Apr 23 10:43:55 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-3[47966]: 2024-04-23T10:43:55.834+0000 7f4c24c7d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T10:43:56.110 INFO:journalctl@ceph.osd.3.smithi097.stdout:Apr 23 10:43:55 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-3[47966]: 2024-04-23T10:43:55.834+0000 7f4c24c7d700 -1 osd.3 114 *** Got signal Terminated *** 2024-04-23T10:43:56.111 INFO:journalctl@ceph.osd.3.smithi097.stdout:Apr 23 10:43:55 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-3[47966]: 2024-04-23T10:43:55.835+0000 7f4c24c7d700 -1 osd.3 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T10:43:58.111 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:43:57 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6[54668]: 2024-04-23T10:43:57.703+0000 7f85a54bb700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.200851+0000 front 2024-04-23T10:43:33.200795+0000 (oldest deadline 2024-04-23T10:43:57.300941+0000) 2024-04-23T10:43:58.728 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:43:58 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:43:58.456+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.896652+0000 front 2024-04-23T10:43:34.896698+0000 (oldest deadline 2024-04-23T10:43:58.396793+0000) 2024-04-23T10:43:58.841 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:43:58 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:43:58.351+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.335834+0000 front 2024-04-23T10:43:36.335867+0000 (oldest deadline 2024-04-23T10:43:57.435822+0000) 2024-04-23T10:43:59.111 INFO:journalctl@ceph.osd.3.smithi097.stdout:Apr 23 10:43:58 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-3[47966]: 2024-04-23T10:43:58.838+0000 7f4c1d269700 -1 osd.3 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.530632+0000 front 2024-04-23T10:43:34.530700+0000 (oldest deadline 2024-04-23T10:43:58.630855+0000) 2024-04-23T10:43:59.111 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:43:58 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6[54668]: 2024-04-23T10:43:58.728+0000 7f85a54bb700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.200851+0000 front 2024-04-23T10:43:33.200795+0000 (oldest deadline 2024-04-23T10:43:57.300941+0000) 2024-04-23T10:43:59.150 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:43:59 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:43:59.037+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.211952+0000 front 2024-04-23T10:43:33.211903+0000 (oldest deadline 2024-04-23T10:43:59.012678+0000) 2024-04-23T10:43:59.592 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 23 10:43:59 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-5[44718]: 2024-04-23T10:43:59.194+0000 7f33ec49b700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.371482+0000 front 2024-04-23T10:43:34.371454+0000 (oldest deadline 2024-04-23T10:43:58.471359+0000) 2024-04-23T10:43:59.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:43:59 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:43:59.321+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.335834+0000 front 2024-04-23T10:43:36.335867+0000 (oldest deadline 2024-04-23T10:43:57.435822+0000) 2024-04-23T10:43:59.849 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:43:59 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6[54668]: 2024-04-23T10:43:59.703+0000 7f85a54bb700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.200851+0000 front 2024-04-23T10:43:33.200795+0000 (oldest deadline 2024-04-23T10:43:57.300941+0000) 2024-04-23T10:43:59.849 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:43:59 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:43:59.455+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.896652+0000 front 2024-04-23T10:43:34.896698+0000 (oldest deadline 2024-04-23T10:43:58.396793+0000) 2024-04-23T10:44:00.091 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:43:59 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:43:59.841+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.311585+0000 front 2024-04-23T10:43:36.311514+0000 (oldest deadline 2024-04-23T10:43:59.211672+0000) 2024-04-23T10:44:00.110 INFO:journalctl@ceph.osd.3.smithi097.stdout:Apr 23 10:43:59 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-3[47966]: 2024-04-23T10:43:59.850+0000 7f4c1d269700 -1 osd.3 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.530632+0000 front 2024-04-23T10:43:34.530700+0000 (oldest deadline 2024-04-23T10:43:58.630855+0000) 2024-04-23T10:44:00.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:00 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:00.002+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.211952+0000 front 2024-04-23T10:43:33.211903+0000 (oldest deadline 2024-04-23T10:43:59.012678+0000) 2024-04-23T10:44:00.591 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:00 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:00.331+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.335834+0000 front 2024-04-23T10:43:36.335867+0000 (oldest deadline 2024-04-23T10:43:57.435822+0000) 2024-04-23T10:44:00.592 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 23 10:44:00 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-5[44718]: 2024-04-23T10:44:00.233+0000 7f33ec49b700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.371482+0000 front 2024-04-23T10:43:34.371454+0000 (oldest deadline 2024-04-23T10:43:58.471359+0000) 2024-04-23T10:44:00.802 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:00 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6[54668]: 2024-04-23T10:44:00.655+0000 7f85a54bb700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.200851+0000 front 2024-04-23T10:43:33.200795+0000 (oldest deadline 2024-04-23T10:43:57.300941+0000) 2024-04-23T10:44:00.802 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:00 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:00.474+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.896652+0000 front 2024-04-23T10:43:34.896698+0000 (oldest deadline 2024-04-23T10:43:58.396793+0000) 2024-04-23T10:44:01.110 INFO:journalctl@ceph.osd.3.smithi097.stdout:Apr 23 10:44:00 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-3[47966]: 2024-04-23T10:44:00.803+0000 7f4c1d269700 -1 osd.3 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.530632+0000 front 2024-04-23T10:43:34.530700+0000 (oldest deadline 2024-04-23T10:43:58.630855+0000) 2024-04-23T10:44:01.189 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:00 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:00.863+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.311585+0000 front 2024-04-23T10:43:36.311514+0000 (oldest deadline 2024-04-23T10:43:59.211672+0000) 2024-04-23T10:44:01.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:00 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:00.964+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.211952+0000 front 2024-04-23T10:43:33.211903+0000 (oldest deadline 2024-04-23T10:43:59.012678+0000) 2024-04-23T10:44:01.591 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:01 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:01.346+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.335834+0000 front 2024-04-23T10:43:36.335867+0000 (oldest deadline 2024-04-23T10:43:57.435822+0000) 2024-04-23T10:44:01.591 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 23 10:44:01 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-5[44718]: 2024-04-23T10:44:01.191+0000 7f33ec49b700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.371482+0000 front 2024-04-23T10:43:34.371454+0000 (oldest deadline 2024-04-23T10:43:58.471359+0000) 2024-04-23T10:44:01.676 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:01 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:01.457+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.896652+0000 front 2024-04-23T10:43:34.896698+0000 (oldest deadline 2024-04-23T10:43:58.396793+0000) 2024-04-23T10:44:01.676 INFO:journalctl@ceph.osd.3.smithi097.stdout:Apr 23 10:44:01 smithi097 podman[94243]: 2024-04-23 10:44:01.387870822 +0000 UTC m=+5.670322779 container died bcf103c5245d8ab17eef1582b12162eba7912d44628357aa6fd779e23c67da80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-3, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T10:44:02.035 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:01 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6[54668]: 2024-04-23T10:44:01.705+0000 7f85a54bb700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.200851+0000 front 2024-04-23T10:43:33.200795+0000 (oldest deadline 2024-04-23T10:43:57.300941+0000) 2024-04-23T10:44:02.035 INFO:journalctl@ceph.osd.3.smithi097.stdout:Apr 23 10:44:01 smithi097 podman[94243]: 2024-04-23 10:44:01.777716043 +0000 UTC m=+6.060168012 container cleanup bcf103c5245d8ab17eef1582b12162eba7912d44628357aa6fd779e23c67da80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-3, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T10:44:02.035 INFO:journalctl@ceph.osd.3.smithi097.stdout:Apr 23 10:44:01 smithi097 bash[94243]: ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-3 2024-04-23T10:44:02.091 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:01 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:01.841+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.311585+0000 front 2024-04-23T10:43:36.311514+0000 (oldest deadline 2024-04-23T10:43:59.211672+0000) 2024-04-23T10:44:02.360 INFO:journalctl@ceph.osd.3.smithi097.stdout:Apr 23 10:44:02 smithi097 podman[94271]: 2024-04-23 10:44:02.03732445 +0000 UTC m=+0.646263222 container remove bcf103c5245d8ab17eef1582b12162eba7912d44628357aa6fd779e23c67da80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-3, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416) 2024-04-23T10:44:02.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:01 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:01.917+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.211952+0000 front 2024-04-23T10:43:33.211903+0000 (oldest deadline 2024-04-23T10:43:59.012678+0000) 2024-04-23T10:44:02.591 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 23 10:44:02 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-5[44718]: 2024-04-23T10:44:02.216+0000 7f33ec49b700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.371482+0000 front 2024-04-23T10:43:34.371454+0000 (oldest deadline 2024-04-23T10:43:58.471359+0000) 2024-04-23T10:44:02.591 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:02 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:02.362+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.335834+0000 front 2024-04-23T10:43:36.335867+0000 (oldest deadline 2024-04-23T10:43:57.435822+0000) 2024-04-23T10:44:02.680 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:02 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:02.477+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.896652+0000 front 2024-04-23T10:43:34.896698+0000 (oldest deadline 2024-04-23T10:43:58.396793+0000) 2024-04-23T10:44:02.680 INFO:journalctl@ceph.osd.3.smithi097.stdout:Apr 23 10:44:02 smithi097 podman[94364]: 2024-04-23 10:44:02.297776142 +0000 UTC m=+0.018552888 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T10:44:02.681 INFO:journalctl@ceph.osd.3.smithi097.stdout:Apr 23 10:44:02 smithi097 podman[94364]: 2024-04-23 10:44:02.430049161 +0000 UTC m=+0.150825894 container create 9f5c509c0ad513df763919ee68f565c7eba4ad7261315f19c01a1c7d11f99151 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD) 2024-04-23T10:44:02.974 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:02 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6[54668]: 2024-04-23T10:44:02.718+0000 7f85a54bb700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.200851+0000 front 2024-04-23T10:43:33.200795+0000 (oldest deadline 2024-04-23T10:43:57.300941+0000) 2024-04-23T10:44:02.975 INFO:journalctl@ceph.osd.3.smithi097.stdout:Apr 23 10:44:02 smithi097 podman[94364]: 2024-04-23 10:44:02.682835329 +0000 UTC m=+0.403612062 container init 9f5c509c0ad513df763919ee68f565c7eba4ad7261315f19c01a1c7d11f99151 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS) 2024-04-23T10:44:02.975 INFO:journalctl@ceph.osd.3.smithi097.stdout:Apr 23 10:44:02 smithi097 podman[94364]: 2024-04-23 10:44:02.686105323 +0000 UTC m=+0.406882057 container start 9f5c509c0ad513df763919ee68f565c7eba4ad7261315f19c01a1c7d11f99151 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-23T10:44:02.975 INFO:journalctl@ceph.osd.3.smithi097.stdout:Apr 23 10:44:02 smithi097 podman[94364]: 2024-04-23 10:44:02.749592261 +0000 UTC m=+0.470369007 container attach 9f5c509c0ad513df763919ee68f565c7eba4ad7261315f19c01a1c7d11f99151 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T10:44:03.172 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:02 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:02.863+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.311585+0000 front 2024-04-23T10:43:36.311514+0000 (oldest deadline 2024-04-23T10:43:59.211672+0000) 2024-04-23T10:44:03.276 INFO:journalctl@ceph.osd.3.smithi097.stdout:Apr 23 10:44:02 smithi097 podman[94364]: 2024-04-23 10:44:02.97687822 +0000 UTC m=+0.697654963 container died 9f5c509c0ad513df763919ee68f565c7eba4ad7261315f19c01a1c7d11f99151 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True) 2024-04-23T10:44:03.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:02 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:02.946+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.211952+0000 front 2024-04-23T10:43:33.211903+0000 (oldest deadline 2024-04-23T10:43:59.012678+0000) 2024-04-23T10:44:03.585 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:03 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:03.487+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.896652+0000 front 2024-04-23T10:43:34.896698+0000 (oldest deadline 2024-04-23T10:43:58.396793+0000) 2024-04-23T10:44:03.591 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:03 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:03.363+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.335834+0000 front 2024-04-23T10:43:36.335867+0000 (oldest deadline 2024-04-23T10:43:57.435822+0000) 2024-04-23T10:44:03.592 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 23 10:44:03 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-5[44718]: 2024-04-23T10:44:03.173+0000 7f33ec49b700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.371482+0000 front 2024-04-23T10:43:34.371454+0000 (oldest deadline 2024-04-23T10:43:58.471359+0000) 2024-04-23T10:44:03.605 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b22af88-0159-11ef-bc93-c7b262605968@osd.3.service' 2024-04-23T10:44:03.645 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T10:44:03.645 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-23T10:44:03.646 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-23T10:44:03.646 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-9b22af88-0159-11ef-bc93-c7b262605968@osd.4 2024-04-23T10:44:03.860 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:03 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6[54668]: 2024-04-23T10:44:03.670+0000 7f85a54bb700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.200851+0000 front 2024-04-23T10:43:33.200795+0000 (oldest deadline 2024-04-23T10:43:57.300941+0000) 2024-04-23T10:44:03.963 INFO:journalctl@ceph.osd.4.smithi183.stdout:Apr 23 10:44:03 smithi183 systemd[1]: Stopping Ceph osd.4 for 9b22af88-0159-11ef-bc93-c7b262605968... 2024-04-23T10:44:04.150 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:03 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:03.897+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.311585+0000 front 2024-04-23T10:43:36.311514+0000 (oldest deadline 2024-04-23T10:43:59.211672+0000) 2024-04-23T10:44:04.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:03 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:03.964+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.211952+0000 front 2024-04-23T10:43:33.211903+0000 (oldest deadline 2024-04-23T10:43:59.012678+0000) 2024-04-23T10:44:04.400 INFO:journalctl@ceph.osd.4.smithi183.stdout:Apr 23 10:44:03 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-4[43863]: 2024-04-23T10:44:03.976+0000 7f6d703e0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T10:44:04.401 INFO:journalctl@ceph.osd.4.smithi183.stdout:Apr 23 10:44:03 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-4[43863]: 2024-04-23T10:44:03.976+0000 7f6d703e0700 -1 osd.4 114 *** Got signal Terminated *** 2024-04-23T10:44:04.401 INFO:journalctl@ceph.osd.4.smithi183.stdout:Apr 23 10:44:03 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-4[43863]: 2024-04-23T10:44:03.976+0000 7f6d703e0700 -1 osd.4 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T10:44:04.591 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 23 10:44:04 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-5[44718]: 2024-04-23T10:44:04.152+0000 7f33ec49b700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.371482+0000 front 2024-04-23T10:43:34.371454+0000 (oldest deadline 2024-04-23T10:43:58.471359+0000) 2024-04-23T10:44:04.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:04 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:04.382+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.335834+0000 front 2024-04-23T10:43:36.335867+0000 (oldest deadline 2024-04-23T10:43:57.435822+0000) 2024-04-23T10:44:04.860 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:04 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6[54668]: 2024-04-23T10:44:04.694+0000 7f85a54bb700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.200851+0000 front 2024-04-23T10:43:33.200795+0000 (oldest deadline 2024-04-23T10:43:57.300941+0000) 2024-04-23T10:44:04.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:04 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:04.492+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.896652+0000 front 2024-04-23T10:43:34.896698+0000 (oldest deadline 2024-04-23T10:43:58.396793+0000) 2024-04-23T10:44:05.193 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:04 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:04.906+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.311585+0000 front 2024-04-23T10:43:36.311514+0000 (oldest deadline 2024-04-23T10:43:59.211672+0000) 2024-04-23T10:44:05.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:05 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:05.010+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.211952+0000 front 2024-04-23T10:43:33.211903+0000 (oldest deadline 2024-04-23T10:43:59.012678+0000) 2024-04-23T10:44:05.591 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 23 10:44:05 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-5[44718]: 2024-04-23T10:44:05.195+0000 7f33ec49b700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.371482+0000 front 2024-04-23T10:43:34.371454+0000 (oldest deadline 2024-04-23T10:43:58.471359+0000) 2024-04-23T10:44:05.591 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:05 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:05.352+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.335834+0000 front 2024-04-23T10:43:36.335867+0000 (oldest deadline 2024-04-23T10:43:57.435822+0000) 2024-04-23T10:44:05.860 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:05 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6[54668]: 2024-04-23T10:44:05.669+0000 7f85a54bb700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.200851+0000 front 2024-04-23T10:43:33.200795+0000 (oldest deadline 2024-04-23T10:43:57.300941+0000) 2024-04-23T10:44:05.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:05 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:05.453+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.896652+0000 front 2024-04-23T10:43:34.896698+0000 (oldest deadline 2024-04-23T10:43:58.396793+0000) 2024-04-23T10:44:05.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:05 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:05.453+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.998590+0000 front 2024-04-23T10:43:42.998576+0000 (oldest deadline 2024-04-23T10:44:05.298692+0000) 2024-04-23T10:44:06.219 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:05 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:05.866+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.311585+0000 front 2024-04-23T10:43:36.311514+0000 (oldest deadline 2024-04-23T10:43:59.211672+0000) 2024-04-23T10:44:06.400 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:06 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7[50726]: 2024-04-23T10:44:06.166+0000 7f5dbd523700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:41.728600+0000 front 2024-04-23T10:43:41.728546+0000 (oldest deadline 2024-04-23T10:44:05.228700+0000) 2024-04-23T10:44:06.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:05 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:05.994+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.211952+0000 front 2024-04-23T10:43:33.211903+0000 (oldest deadline 2024-04-23T10:43:59.012678+0000) 2024-04-23T10:44:06.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:05 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:05.994+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:43.515012+0000 front 2024-04-23T10:43:43.515041+0000 (oldest deadline 2024-04-23T10:44:05.815169+0000) 2024-04-23T10:44:06.591 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 23 10:44:06 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-5[44718]: 2024-04-23T10:44:06.220+0000 7f33ec49b700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.371482+0000 front 2024-04-23T10:43:34.371454+0000 (oldest deadline 2024-04-23T10:43:58.471359+0000) 2024-04-23T10:44:06.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:06 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:06.354+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.335834+0000 front 2024-04-23T10:43:36.335867+0000 (oldest deadline 2024-04-23T10:43:57.435822+0000) 2024-04-23T10:44:06.861 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:06 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6[54668]: 2024-04-23T10:44:06.684+0000 7f85a54bb700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.200851+0000 front 2024-04-23T10:43:33.200795+0000 (oldest deadline 2024-04-23T10:43:57.300941+0000) 2024-04-23T10:44:06.861 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:06 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6[54668]: 2024-04-23T10:44:06.684+0000 7f85a54bb700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:39.002107+0000 front 2024-04-23T10:43:39.001911+0000 (oldest deadline 2024-04-23T10:44:06.002356+0000) 2024-04-23T10:44:06.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:06 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:06.500+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.896652+0000 front 2024-04-23T10:43:34.896698+0000 (oldest deadline 2024-04-23T10:43:58.396793+0000) 2024-04-23T10:44:06.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:06 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:06.500+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.998590+0000 front 2024-04-23T10:43:42.998576+0000 (oldest deadline 2024-04-23T10:44:05.298692+0000) 2024-04-23T10:44:07.249 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:06 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:06.894+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.311585+0000 front 2024-04-23T10:43:36.311514+0000 (oldest deadline 2024-04-23T10:43:59.211672+0000) 2024-04-23T10:44:07.400 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:07 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7[50726]: 2024-04-23T10:44:07.216+0000 7f5dbd523700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:41.728600+0000 front 2024-04-23T10:43:41.728546+0000 (oldest deadline 2024-04-23T10:44:05.228700+0000) 2024-04-23T10:44:07.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:06 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:06.998+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.211952+0000 front 2024-04-23T10:43:33.211903+0000 (oldest deadline 2024-04-23T10:43:59.012678+0000) 2024-04-23T10:44:07.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:06 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:06.998+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:43.515012+0000 front 2024-04-23T10:43:43.515041+0000 (oldest deadline 2024-04-23T10:44:05.815169+0000) 2024-04-23T10:44:07.591 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 23 10:44:07 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-5[44718]: 2024-04-23T10:44:07.251+0000 7f33ec49b700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.371482+0000 front 2024-04-23T10:43:34.371454+0000 (oldest deadline 2024-04-23T10:43:58.471359+0000) 2024-04-23T10:44:07.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:07 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:07.366+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.335834+0000 front 2024-04-23T10:43:36.335867+0000 (oldest deadline 2024-04-23T10:43:57.435822+0000) 2024-04-23T10:44:07.860 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:07 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6[54668]: 2024-04-23T10:44:07.644+0000 7f85a54bb700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.200851+0000 front 2024-04-23T10:43:33.200795+0000 (oldest deadline 2024-04-23T10:43:57.300941+0000) 2024-04-23T10:44:07.861 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:07 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6[54668]: 2024-04-23T10:44:07.644+0000 7f85a54bb700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:39.002107+0000 front 2024-04-23T10:43:39.001911+0000 (oldest deadline 2024-04-23T10:44:06.002356+0000) 2024-04-23T10:44:07.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:07 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:07.474+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.896652+0000 front 2024-04-23T10:43:34.896698+0000 (oldest deadline 2024-04-23T10:43:58.396793+0000) 2024-04-23T10:44:07.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:07 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:07.474+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.998590+0000 front 2024-04-23T10:43:42.998576+0000 (oldest deadline 2024-04-23T10:44:05.298692+0000) 2024-04-23T10:44:08.243 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:08 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7[50726]: 2024-04-23T10:44:08.199+0000 7f5dbd523700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:41.728600+0000 front 2024-04-23T10:43:41.728546+0000 (oldest deadline 2024-04-23T10:44:05.228700+0000) 2024-04-23T10:44:08.243 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:07 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:07.968+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.211952+0000 front 2024-04-23T10:43:33.211903+0000 (oldest deadline 2024-04-23T10:43:59.012678+0000) 2024-04-23T10:44:08.243 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:07 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:07.968+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:43.515012+0000 front 2024-04-23T10:43:43.515041+0000 (oldest deadline 2024-04-23T10:44:05.815169+0000) 2024-04-23T10:44:08.263 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:07 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:07.915+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.311585+0000 front 2024-04-23T10:43:36.311514+0000 (oldest deadline 2024-04-23T10:43:59.211672+0000) 2024-04-23T10:44:08.591 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 23 10:44:08 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-5[44718]: 2024-04-23T10:44:08.264+0000 7f33ec49b700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.371482+0000 front 2024-04-23T10:43:34.371454+0000 (oldest deadline 2024-04-23T10:43:58.471359+0000) 2024-04-23T10:44:08.592 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 23 10:44:08 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-5[44718]: 2024-04-23T10:44:08.264+0000 7f33ec49b700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:43.772548+0000 front 2024-04-23T10:43:43.772512+0000 (oldest deadline 2024-04-23T10:44:07.872586+0000) 2024-04-23T10:44:08.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:08 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:08.341+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.335834+0000 front 2024-04-23T10:43:36.335867+0000 (oldest deadline 2024-04-23T10:43:57.435822+0000) 2024-04-23T10:44:08.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:08 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:08.341+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.637810+0000 front 2024-04-23T10:43:42.637720+0000 (oldest deadline 2024-04-23T10:44:07.937634+0000) 2024-04-23T10:44:08.650 INFO:journalctl@ceph.osd.4.smithi183.stdout:Apr 23 10:44:08 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-4[43863]: 2024-04-23T10:44:08.244+0000 7f6d689cc700 -1 osd.4 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.892272+0000 front 2024-04-23T10:43:42.892296+0000 (oldest deadline 2024-04-23T10:44:07.592414+0000) 2024-04-23T10:44:08.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:08 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:08.428+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.896652+0000 front 2024-04-23T10:43:34.896698+0000 (oldest deadline 2024-04-23T10:43:58.396793+0000) 2024-04-23T10:44:08.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:08 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:08.428+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.998590+0000 front 2024-04-23T10:43:42.998576+0000 (oldest deadline 2024-04-23T10:44:05.298692+0000) 2024-04-23T10:44:08.861 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:08 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6[54668]: 2024-04-23T10:44:08.633+0000 7f85a54bb700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.200851+0000 front 2024-04-23T10:43:33.200795+0000 (oldest deadline 2024-04-23T10:43:57.300941+0000) 2024-04-23T10:44:08.861 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:08 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6[54668]: 2024-04-23T10:44:08.633+0000 7f85a54bb700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:39.002107+0000 front 2024-04-23T10:43:39.001911+0000 (oldest deadline 2024-04-23T10:44:06.002356+0000) 2024-04-23T10:44:09.177 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:08 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:08.921+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.211952+0000 front 2024-04-23T10:43:33.211903+0000 (oldest deadline 2024-04-23T10:43:59.012678+0000) 2024-04-23T10:44:09.177 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:08 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:08.921+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:43.515012+0000 front 2024-04-23T10:43:43.515041+0000 (oldest deadline 2024-04-23T10:44:05.815169+0000) 2024-04-23T10:44:09.217 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:08 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:08.925+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.311585+0000 front 2024-04-23T10:43:36.311514+0000 (oldest deadline 2024-04-23T10:43:59.211672+0000) 2024-04-23T10:44:09.488 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:09 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7[50726]: 2024-04-23T10:44:09.178+0000 7f5dbd523700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:41.728600+0000 front 2024-04-23T10:43:41.728546+0000 (oldest deadline 2024-04-23T10:44:05.228700+0000) 2024-04-23T10:44:09.591 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:09 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:09.376+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.335834+0000 front 2024-04-23T10:43:36.335867+0000 (oldest deadline 2024-04-23T10:43:57.435822+0000) 2024-04-23T10:44:09.591 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:09 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:09.376+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.637810+0000 front 2024-04-23T10:43:42.637720+0000 (oldest deadline 2024-04-23T10:44:07.937634+0000) 2024-04-23T10:44:09.592 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 23 10:44:09 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-5[44718]: 2024-04-23T10:44:09.219+0000 7f33ec49b700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.371482+0000 front 2024-04-23T10:43:34.371454+0000 (oldest deadline 2024-04-23T10:43:58.471359+0000) 2024-04-23T10:44:09.592 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 23 10:44:09 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-5[44718]: 2024-04-23T10:44:09.219+0000 7f33ec49b700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:43.772548+0000 front 2024-04-23T10:43:43.772512+0000 (oldest deadline 2024-04-23T10:44:07.872586+0000) 2024-04-23T10:44:09.809 INFO:journalctl@ceph.osd.4.smithi183.stdout:Apr 23 10:44:09 smithi183 podman[69988]: 2024-04-23 10:44:09.49028205 +0000 UTC m=+5.646435844 container died cfbf02a3871cd9fc86e593961180a6fa3d67f086f149549efb900205bcaacd59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-4, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T10:44:09.860 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:09 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6[54668]: 2024-04-23T10:44:09.631+0000 7f85a54bb700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.200851+0000 front 2024-04-23T10:43:33.200795+0000 (oldest deadline 2024-04-23T10:43:57.300941+0000) 2024-04-23T10:44:09.861 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:09 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6[54668]: 2024-04-23T10:44:09.631+0000 7f85a54bb700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:39.002107+0000 front 2024-04-23T10:43:39.001911+0000 (oldest deadline 2024-04-23T10:44:06.002356+0000) 2024-04-23T10:44:09.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:09 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:09.408+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.896652+0000 front 2024-04-23T10:43:34.896698+0000 (oldest deadline 2024-04-23T10:43:58.396793+0000) 2024-04-23T10:44:09.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:09 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:09.408+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.998590+0000 front 2024-04-23T10:43:42.998576+0000 (oldest deadline 2024-04-23T10:44:05.298692+0000) 2024-04-23T10:44:10.133 INFO:journalctl@ceph.osd.4.smithi183.stdout:Apr 23 10:44:09 smithi183 podman[69988]: 2024-04-23 10:44:09.910185574 +0000 UTC m=+6.066339357 container cleanup cfbf02a3871cd9fc86e593961180a6fa3d67f086f149549efb900205bcaacd59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-4, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-23T10:44:10.133 INFO:journalctl@ceph.osd.4.smithi183.stdout:Apr 23 10:44:09 smithi183 bash[69988]: ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-4 2024-04-23T10:44:10.133 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:09 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:09.962+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.211952+0000 front 2024-04-23T10:43:33.211903+0000 (oldest deadline 2024-04-23T10:43:59.012678+0000) 2024-04-23T10:44:10.133 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:09 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:09.962+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:43.515012+0000 front 2024-04-23T10:43:43.515041+0000 (oldest deadline 2024-04-23T10:44:05.815169+0000) 2024-04-23T10:44:10.221 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:09 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:09.912+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.311585+0000 front 2024-04-23T10:43:36.311514+0000 (oldest deadline 2024-04-23T10:43:59.211672+0000) 2024-04-23T10:44:10.400 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:10 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7[50726]: 2024-04-23T10:44:10.133+0000 7f5dbd523700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:41.728600+0000 front 2024-04-23T10:43:41.728546+0000 (oldest deadline 2024-04-23T10:44:05.228700+0000) 2024-04-23T10:44:10.400 INFO:journalctl@ceph.osd.4.smithi183.stdout:Apr 23 10:44:10 smithi183 podman[70005]: 2024-04-23 10:44:10.158213974 +0000 UTC m=+0.669140723 container remove cfbf02a3871cd9fc86e593961180a6fa3d67f086f149549efb900205bcaacd59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-4, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-23T10:44:10.591 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:10 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:10.362+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.335834+0000 front 2024-04-23T10:43:36.335867+0000 (oldest deadline 2024-04-23T10:43:57.435822+0000) 2024-04-23T10:44:10.591 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:10 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:10.362+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.637810+0000 front 2024-04-23T10:43:42.637720+0000 (oldest deadline 2024-04-23T10:44:07.937634+0000) 2024-04-23T10:44:10.592 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 23 10:44:10 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-5[44718]: 2024-04-23T10:44:10.223+0000 7f33ec49b700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.371482+0000 front 2024-04-23T10:43:34.371454+0000 (oldest deadline 2024-04-23T10:43:58.471359+0000) 2024-04-23T10:44:10.592 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 23 10:44:10 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-5[44718]: 2024-04-23T10:44:10.223+0000 7f33ec49b700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:43.772548+0000 front 2024-04-23T10:43:43.772512+0000 (oldest deadline 2024-04-23T10:44:07.872586+0000) 2024-04-23T10:44:10.675 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:10 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:10.367+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.896652+0000 front 2024-04-23T10:43:34.896698+0000 (oldest deadline 2024-04-23T10:43:58.396793+0000) 2024-04-23T10:44:10.675 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:10 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:10.367+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.998590+0000 front 2024-04-23T10:43:42.998576+0000 (oldest deadline 2024-04-23T10:44:05.298692+0000) 2024-04-23T10:44:10.863 INFO:journalctl@ceph.osd.4.smithi183.stdout:Apr 23 10:44:10 smithi183 podman[70094]: 2024-04-23 10:44:10.483107627 +0000 UTC m=+0.018012343 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T10:44:10.863 INFO:journalctl@ceph.osd.4.smithi183.stdout:Apr 23 10:44:10 smithi183 podman[70094]: 2024-04-23 10:44:10.605023173 +0000 UTC m=+0.139927877 container create 56fbfb7a4a332ea400fbe814b86228ff6f8f9dd9f8589a199707ae4af02bea18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5) 2024-04-23T10:44:10.863 INFO:journalctl@ceph.osd.4.smithi183.stdout:Apr 23 10:44:10 smithi183 podman[70094]: 2024-04-23 10:44:10.865676799 +0000 UTC m=+0.400581503 container init 56fbfb7a4a332ea400fbe814b86228ff6f8f9dd9f8589a199707ae4af02bea18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-23T10:44:11.110 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:10 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6[54668]: 2024-04-23T10:44:10.676+0000 7f85a54bb700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.200851+0000 front 2024-04-23T10:43:33.200795+0000 (oldest deadline 2024-04-23T10:43:57.300941+0000) 2024-04-23T10:44:11.111 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:10 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6[54668]: 2024-04-23T10:44:10.676+0000 7f85a54bb700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:39.002107+0000 front 2024-04-23T10:43:39.001911+0000 (oldest deadline 2024-04-23T10:44:06.002356+0000) 2024-04-23T10:44:11.150 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:10 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:10.918+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.211952+0000 front 2024-04-23T10:43:33.211903+0000 (oldest deadline 2024-04-23T10:43:59.012678+0000) 2024-04-23T10:44:11.150 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:10 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:10.918+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:43.515012+0000 front 2024-04-23T10:43:43.515041+0000 (oldest deadline 2024-04-23T10:44:05.815169+0000) 2024-04-23T10:44:11.150 INFO:journalctl@ceph.osd.4.smithi183.stdout:Apr 23 10:44:10 smithi183 podman[70094]: 2024-04-23 10:44:10.868879093 +0000 UTC m=+0.403783801 container start 56fbfb7a4a332ea400fbe814b86228ff6f8f9dd9f8589a199707ae4af02bea18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True) 2024-04-23T10:44:11.151 INFO:journalctl@ceph.osd.4.smithi183.stdout:Apr 23 10:44:10 smithi183 podman[70094]: 2024-04-23 10:44:10.974312168 +0000 UTC m=+0.509216884 container attach 56fbfb7a4a332ea400fbe814b86228ff6f8f9dd9f8589a199707ae4af02bea18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-23T10:44:11.201 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:10 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:10.869+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.311585+0000 front 2024-04-23T10:43:36.311514+0000 (oldest deadline 2024-04-23T10:43:59.211672+0000) 2024-04-23T10:44:11.517 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:11 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7[50726]: 2024-04-23T10:44:11.164+0000 7f5dbd523700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:41.728600+0000 front 2024-04-23T10:43:41.728546+0000 (oldest deadline 2024-04-23T10:44:05.228700+0000) 2024-04-23T10:44:11.517 INFO:journalctl@ceph.osd.4.smithi183.stdout:Apr 23 10:44:11 smithi183 podman[70094]: 2024-04-23 10:44:11.16301699 +0000 UTC m=+0.697921697 container died 56fbfb7a4a332ea400fbe814b86228ff6f8f9dd9f8589a199707ae4af02bea18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T10:44:11.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:11 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:11.323+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.335834+0000 front 2024-04-23T10:43:36.335867+0000 (oldest deadline 2024-04-23T10:43:57.435822+0000) 2024-04-23T10:44:11.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:11 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:11.323+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.637810+0000 front 2024-04-23T10:43:42.637720+0000 (oldest deadline 2024-04-23T10:44:07.937634+0000) 2024-04-23T10:44:11.592 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 23 10:44:11 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-5[44718]: 2024-04-23T10:44:11.203+0000 7f33ec49b700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.371482+0000 front 2024-04-23T10:43:34.371454+0000 (oldest deadline 2024-04-23T10:43:58.471359+0000) 2024-04-23T10:44:11.592 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 23 10:44:11 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-5[44718]: 2024-04-23T10:44:11.203+0000 7f33ec49b700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:43.772548+0000 front 2024-04-23T10:43:43.772512+0000 (oldest deadline 2024-04-23T10:44:07.872586+0000) 2024-04-23T10:44:11.656 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:11 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:11.385+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.896652+0000 front 2024-04-23T10:43:34.896698+0000 (oldest deadline 2024-04-23T10:43:58.396793+0000) 2024-04-23T10:44:11.656 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:11 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:11.385+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.998590+0000 front 2024-04-23T10:43:42.998576+0000 (oldest deadline 2024-04-23T10:44:05.298692+0000) 2024-04-23T10:44:11.822 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b22af88-0159-11ef-bc93-c7b262605968@osd.4.service' 2024-04-23T10:44:11.858 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T10:44:11.859 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-23T10:44:11.859 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-23T10:44:11.859 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-9b22af88-0159-11ef-bc93-c7b262605968@osd.5 2024-04-23T10:44:12.091 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 23 10:44:11 smithi196 systemd[1]: Stopping Ceph osd.5 for 9b22af88-0159-11ef-bc93-c7b262605968... 2024-04-23T10:44:12.092 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:11 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:11.842+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.311585+0000 front 2024-04-23T10:43:36.311514+0000 (oldest deadline 2024-04-23T10:43:59.211672+0000) 2024-04-23T10:44:12.110 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:11 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6[54668]: 2024-04-23T10:44:11.657+0000 7f85a54bb700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.200851+0000 front 2024-04-23T10:43:33.200795+0000 (oldest deadline 2024-04-23T10:43:57.300941+0000) 2024-04-23T10:44:12.110 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:11 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6[54668]: 2024-04-23T10:44:11.657+0000 7f85a54bb700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:39.002107+0000 front 2024-04-23T10:43:39.001911+0000 (oldest deadline 2024-04-23T10:44:06.002356+0000) 2024-04-23T10:44:12.113 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:11 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:11.951+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.211952+0000 front 2024-04-23T10:43:33.211903+0000 (oldest deadline 2024-04-23T10:43:59.012678+0000) 2024-04-23T10:44:12.113 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:11 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:11.951+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:43.515012+0000 front 2024-04-23T10:43:43.515041+0000 (oldest deadline 2024-04-23T10:44:05.815169+0000) 2024-04-23T10:44:12.400 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:12 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7[50726]: 2024-04-23T10:44:12.115+0000 7f5dbd523700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:41.728600+0000 front 2024-04-23T10:43:41.728546+0000 (oldest deadline 2024-04-23T10:44:05.228700+0000) 2024-04-23T10:44:12.591 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:12 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:12.326+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.335834+0000 front 2024-04-23T10:43:36.335867+0000 (oldest deadline 2024-04-23T10:43:57.435822+0000) 2024-04-23T10:44:12.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:12 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:12.326+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.637810+0000 front 2024-04-23T10:43:42.637720+0000 (oldest deadline 2024-04-23T10:44:07.937634+0000) 2024-04-23T10:44:12.592 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 23 10:44:12 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-5[44718]: 2024-04-23T10:44:12.186+0000 7f33f3eaf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T10:44:12.592 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 23 10:44:12 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-5[44718]: 2024-04-23T10:44:12.186+0000 7f33f3eaf700 -1 osd.5 114 *** Got signal Terminated *** 2024-04-23T10:44:12.592 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 23 10:44:12 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-5[44718]: 2024-04-23T10:44:12.186+0000 7f33f3eaf700 -1 osd.5 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T10:44:12.592 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 23 10:44:12 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-5[44718]: 2024-04-23T10:44:12.225+0000 7f33ec49b700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.371482+0000 front 2024-04-23T10:43:34.371454+0000 (oldest deadline 2024-04-23T10:43:58.471359+0000) 2024-04-23T10:44:12.592 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 23 10:44:12 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-5[44718]: 2024-04-23T10:44:12.225+0000 7f33ec49b700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:43.772548+0000 front 2024-04-23T10:43:43.772512+0000 (oldest deadline 2024-04-23T10:44:07.872586+0000) 2024-04-23T10:44:12.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:12 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:12.401+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.896652+0000 front 2024-04-23T10:43:34.896698+0000 (oldest deadline 2024-04-23T10:43:58.396793+0000) 2024-04-23T10:44:12.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:12 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:12.401+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.998590+0000 front 2024-04-23T10:43:42.998576+0000 (oldest deadline 2024-04-23T10:44:05.298692+0000) 2024-04-23T10:44:12.861 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:12 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6[54668]: 2024-04-23T10:44:12.623+0000 7f85a54bb700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.200851+0000 front 2024-04-23T10:43:33.200795+0000 (oldest deadline 2024-04-23T10:43:57.300941+0000) 2024-04-23T10:44:12.861 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:12 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6[54668]: 2024-04-23T10:44:12.623+0000 7f85a54bb700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:39.002107+0000 front 2024-04-23T10:43:39.001911+0000 (oldest deadline 2024-04-23T10:44:06.002356+0000) 2024-04-23T10:44:13.193 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:12 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:12.865+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.311585+0000 front 2024-04-23T10:43:36.311514+0000 (oldest deadline 2024-04-23T10:43:59.211672+0000) 2024-04-23T10:44:13.400 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:13 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7[50726]: 2024-04-23T10:44:13.069+0000 7f5dbd523700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:41.728600+0000 front 2024-04-23T10:43:41.728546+0000 (oldest deadline 2024-04-23T10:44:05.228700+0000) 2024-04-23T10:44:13.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:12 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:12.944+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.211952+0000 front 2024-04-23T10:43:33.211903+0000 (oldest deadline 2024-04-23T10:43:59.012678+0000) 2024-04-23T10:44:13.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:12 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:12.944+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:43.515012+0000 front 2024-04-23T10:43:43.515041+0000 (oldest deadline 2024-04-23T10:44:05.815169+0000) 2024-04-23T10:44:13.591 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 23 10:44:13 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-5[44718]: 2024-04-23T10:44:13.195+0000 7f33ec49b700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.371482+0000 front 2024-04-23T10:43:34.371454+0000 (oldest deadline 2024-04-23T10:43:58.471359+0000) 2024-04-23T10:44:13.591 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 23 10:44:13 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-5[44718]: 2024-04-23T10:44:13.195+0000 7f33ec49b700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:43.772548+0000 front 2024-04-23T10:43:43.772512+0000 (oldest deadline 2024-04-23T10:44:07.872586+0000) 2024-04-23T10:44:13.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:13 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:13.368+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.335834+0000 front 2024-04-23T10:43:36.335867+0000 (oldest deadline 2024-04-23T10:43:57.435822+0000) 2024-04-23T10:44:13.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:13 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:13.368+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.637810+0000 front 2024-04-23T10:43:42.637720+0000 (oldest deadline 2024-04-23T10:44:07.937634+0000) 2024-04-23T10:44:13.653 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:13 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:13.393+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.896652+0000 front 2024-04-23T10:43:34.896698+0000 (oldest deadline 2024-04-23T10:43:58.396793+0000) 2024-04-23T10:44:13.653 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:13 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:13.393+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.998590+0000 front 2024-04-23T10:43:42.998576+0000 (oldest deadline 2024-04-23T10:44:05.298692+0000) 2024-04-23T10:44:13.653 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:13 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6[54668]: 2024-04-23T10:44:13.597+0000 7f85a54bb700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.200851+0000 front 2024-04-23T10:43:33.200795+0000 (oldest deadline 2024-04-23T10:43:57.300941+0000) 2024-04-23T10:44:13.654 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:13 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6[54668]: 2024-04-23T10:44:13.597+0000 7f85a54bb700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:39.002107+0000 front 2024-04-23T10:43:39.001911+0000 (oldest deadline 2024-04-23T10:44:06.002356+0000) 2024-04-23T10:44:13.654 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:13 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6[54668]: 2024-04-23T10:44:13.597+0000 7f85a54bb700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:51.203968+0000 front 2024-04-23T10:43:51.204236+0000 (oldest deadline 2024-04-23T10:44:13.503949+0000) 2024-04-23T10:44:14.209 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:13 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:13.864+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.311585+0000 front 2024-04-23T10:43:36.311514+0000 (oldest deadline 2024-04-23T10:43:59.211672+0000) 2024-04-23T10:44:14.400 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:14 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7[50726]: 2024-04-23T10:44:14.068+0000 7f5dbd523700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:41.728600+0000 front 2024-04-23T10:43:41.728546+0000 (oldest deadline 2024-04-23T10:44:05.228700+0000) 2024-04-23T10:44:14.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:13 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:13.954+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.211952+0000 front 2024-04-23T10:43:33.211903+0000 (oldest deadline 2024-04-23T10:43:59.012678+0000) 2024-04-23T10:44:14.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:13 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:13.954+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:43.515012+0000 front 2024-04-23T10:43:43.515041+0000 (oldest deadline 2024-04-23T10:44:05.815169+0000) 2024-04-23T10:44:14.592 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 23 10:44:14 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-5[44718]: 2024-04-23T10:44:14.211+0000 7f33ec49b700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.371482+0000 front 2024-04-23T10:43:34.371454+0000 (oldest deadline 2024-04-23T10:43:58.471359+0000) 2024-04-23T10:44:14.592 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 23 10:44:14 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-5[44718]: 2024-04-23T10:44:14.211+0000 7f33ec49b700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:43.772548+0000 front 2024-04-23T10:43:43.772512+0000 (oldest deadline 2024-04-23T10:44:07.872586+0000) 2024-04-23T10:44:14.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:14 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:14.324+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.335834+0000 front 2024-04-23T10:43:36.335867+0000 (oldest deadline 2024-04-23T10:43:57.435822+0000) 2024-04-23T10:44:14.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:14 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:14.324+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.637810+0000 front 2024-04-23T10:43:42.637720+0000 (oldest deadline 2024-04-23T10:44:07.937634+0000) 2024-04-23T10:44:14.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:14 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:14.324+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:52.538910+0000 front 2024-04-23T10:43:52.538984+0000 (oldest deadline 2024-04-23T10:44:14.238911+0000) 2024-04-23T10:44:14.610 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:14 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6[54668]: 2024-04-23T10:44:14.599+0000 7f85a54bb700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.200851+0000 front 2024-04-23T10:43:33.200795+0000 (oldest deadline 2024-04-23T10:43:57.300941+0000) 2024-04-23T10:44:14.611 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:14 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6[54668]: 2024-04-23T10:44:14.599+0000 7f85a54bb700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:39.002107+0000 front 2024-04-23T10:43:39.001911+0000 (oldest deadline 2024-04-23T10:44:06.002356+0000) 2024-04-23T10:44:14.611 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:14 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6[54668]: 2024-04-23T10:44:14.599+0000 7f85a54bb700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:51.203968+0000 front 2024-04-23T10:43:51.204236+0000 (oldest deadline 2024-04-23T10:44:13.503949+0000) 2024-04-23T10:44:14.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:14 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:14.354+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.896652+0000 front 2024-04-23T10:43:34.896698+0000 (oldest deadline 2024-04-23T10:43:58.396793+0000) 2024-04-23T10:44:14.611 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:14 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:14.354+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.998590+0000 front 2024-04-23T10:43:42.998576+0000 (oldest deadline 2024-04-23T10:44:05.298692+0000) 2024-04-23T10:44:15.208 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:14 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:14.896+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.311585+0000 front 2024-04-23T10:43:36.311514+0000 (oldest deadline 2024-04-23T10:43:59.211672+0000) 2024-04-23T10:44:15.209 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:14 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:14.896+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:48.314772+0000 front 2024-04-23T10:43:48.314791+0000 (oldest deadline 2024-04-23T10:44:14.214936+0000) 2024-04-23T10:44:15.400 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:15 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7[50726]: 2024-04-23T10:44:15.029+0000 7f5dbd523700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:41.728600+0000 front 2024-04-23T10:43:41.728546+0000 (oldest deadline 2024-04-23T10:44:05.228700+0000) 2024-04-23T10:44:15.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:14 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:14.943+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.211952+0000 front 2024-04-23T10:43:33.211903+0000 (oldest deadline 2024-04-23T10:43:59.012678+0000) 2024-04-23T10:44:15.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:14 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:14.943+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:43.515012+0000 front 2024-04-23T10:43:43.515041+0000 (oldest deadline 2024-04-23T10:44:05.815169+0000) 2024-04-23T10:44:15.592 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 23 10:44:15 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-5[44718]: 2024-04-23T10:44:15.210+0000 7f33ec49b700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.371482+0000 front 2024-04-23T10:43:34.371454+0000 (oldest deadline 2024-04-23T10:43:58.471359+0000) 2024-04-23T10:44:15.592 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 23 10:44:15 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-5[44718]: 2024-04-23T10:44:15.210+0000 7f33ec49b700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:43.772548+0000 front 2024-04-23T10:43:43.772512+0000 (oldest deadline 2024-04-23T10:44:07.872586+0000) 2024-04-23T10:44:15.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:15 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:15.305+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.335834+0000 front 2024-04-23T10:43:36.335867+0000 (oldest deadline 2024-04-23T10:43:57.435822+0000) 2024-04-23T10:44:15.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:15 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:15.305+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.637810+0000 front 2024-04-23T10:43:42.637720+0000 (oldest deadline 2024-04-23T10:44:07.937634+0000) 2024-04-23T10:44:15.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:15 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:15.305+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:52.538910+0000 front 2024-04-23T10:43:52.538984+0000 (oldest deadline 2024-04-23T10:44:14.238911+0000) 2024-04-23T10:44:15.860 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:15 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6[54668]: 2024-04-23T10:44:15.568+0000 7f85a54bb700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.200851+0000 front 2024-04-23T10:43:33.200795+0000 (oldest deadline 2024-04-23T10:43:57.300941+0000) 2024-04-23T10:44:15.860 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:15 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6[54668]: 2024-04-23T10:44:15.568+0000 7f85a54bb700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:39.002107+0000 front 2024-04-23T10:43:39.001911+0000 (oldest deadline 2024-04-23T10:44:06.002356+0000) 2024-04-23T10:44:15.861 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:15 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6[54668]: 2024-04-23T10:44:15.568+0000 7f85a54bb700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:51.203968+0000 front 2024-04-23T10:43:51.204236+0000 (oldest deadline 2024-04-23T10:44:13.503949+0000) 2024-04-23T10:44:15.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:15 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:15.380+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.896652+0000 front 2024-04-23T10:43:34.896698+0000 (oldest deadline 2024-04-23T10:43:58.396793+0000) 2024-04-23T10:44:15.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:15 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:15.380+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.998590+0000 front 2024-04-23T10:43:42.998576+0000 (oldest deadline 2024-04-23T10:44:05.298692+0000) 2024-04-23T10:44:15.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:15 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:15.380+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:49.900586+0000 front 2024-04-23T10:43:49.900497+0000 (oldest deadline 2024-04-23T10:44:15.200545+0000) 2024-04-23T10:44:16.172 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:15 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:15.914+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.311585+0000 front 2024-04-23T10:43:36.311514+0000 (oldest deadline 2024-04-23T10:43:59.211672+0000) 2024-04-23T10:44:16.172 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:15 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:15.915+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:48.314772+0000 front 2024-04-23T10:43:48.314791+0000 (oldest deadline 2024-04-23T10:44:14.214936+0000) 2024-04-23T10:44:16.400 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:16 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7[50726]: 2024-04-23T10:44:16.030+0000 7f5dbd523700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:41.728600+0000 front 2024-04-23T10:43:41.728546+0000 (oldest deadline 2024-04-23T10:44:05.228700+0000) 2024-04-23T10:44:16.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:15 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:15.983+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.211952+0000 front 2024-04-23T10:43:33.211903+0000 (oldest deadline 2024-04-23T10:43:59.012678+0000) 2024-04-23T10:44:16.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:15 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:15.983+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:43.515012+0000 front 2024-04-23T10:43:43.515041+0000 (oldest deadline 2024-04-23T10:44:05.815169+0000) 2024-04-23T10:44:16.591 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:16 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:16.293+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.335834+0000 front 2024-04-23T10:43:36.335867+0000 (oldest deadline 2024-04-23T10:43:57.435822+0000) 2024-04-23T10:44:16.591 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:16 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:16.293+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.637810+0000 front 2024-04-23T10:43:42.637720+0000 (oldest deadline 2024-04-23T10:44:07.937634+0000) 2024-04-23T10:44:16.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:16 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:16.293+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:52.538910+0000 front 2024-04-23T10:43:52.538984+0000 (oldest deadline 2024-04-23T10:44:14.238911+0000) 2024-04-23T10:44:16.592 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 23 10:44:16 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-5[44718]: 2024-04-23T10:44:16.173+0000 7f33ec49b700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.371482+0000 front 2024-04-23T10:43:34.371454+0000 (oldest deadline 2024-04-23T10:43:58.471359+0000) 2024-04-23T10:44:16.592 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 23 10:44:16 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-5[44718]: 2024-04-23T10:44:16.173+0000 7f33ec49b700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:43.772548+0000 front 2024-04-23T10:43:43.772512+0000 (oldest deadline 2024-04-23T10:44:07.872586+0000) 2024-04-23T10:44:16.592 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 23 10:44:16 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-5[44718]: 2024-04-23T10:44:16.173+0000 7f33ec49b700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:51.373720+0000 front 2024-04-23T10:43:51.373761+0000 (oldest deadline 2024-04-23T10:44:15.473905+0000) 2024-04-23T10:44:16.860 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:16 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6[54668]: 2024-04-23T10:44:16.566+0000 7f85a54bb700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.200851+0000 front 2024-04-23T10:43:33.200795+0000 (oldest deadline 2024-04-23T10:43:57.300941+0000) 2024-04-23T10:44:16.860 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:16 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6[54668]: 2024-04-23T10:44:16.566+0000 7f85a54bb700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:39.002107+0000 front 2024-04-23T10:43:39.001911+0000 (oldest deadline 2024-04-23T10:44:06.002356+0000) 2024-04-23T10:44:16.860 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:16 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6[54668]: 2024-04-23T10:44:16.566+0000 7f85a54bb700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:51.203968+0000 front 2024-04-23T10:43:51.204236+0000 (oldest deadline 2024-04-23T10:44:13.503949+0000) 2024-04-23T10:44:16.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:16 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:16.427+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.896652+0000 front 2024-04-23T10:43:34.896698+0000 (oldest deadline 2024-04-23T10:43:58.396793+0000) 2024-04-23T10:44:16.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:16 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:16.427+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.998590+0000 front 2024-04-23T10:43:42.998576+0000 (oldest deadline 2024-04-23T10:44:05.298692+0000) 2024-04-23T10:44:16.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:16 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:16.427+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:49.900586+0000 front 2024-04-23T10:43:49.900497+0000 (oldest deadline 2024-04-23T10:44:15.200545+0000) 2024-04-23T10:44:17.339 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:16 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:16.885+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.311585+0000 front 2024-04-23T10:43:36.311514+0000 (oldest deadline 2024-04-23T10:43:59.211672+0000) 2024-04-23T10:44:17.339 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:16 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:16.885+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:48.314772+0000 front 2024-04-23T10:43:48.314791+0000 (oldest deadline 2024-04-23T10:44:14.214936+0000) 2024-04-23T10:44:17.340 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 23 10:44:17 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-5[44718]: 2024-04-23T10:44:17.134+0000 7f33ec49b700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.371482+0000 front 2024-04-23T10:43:34.371454+0000 (oldest deadline 2024-04-23T10:43:58.471359+0000) 2024-04-23T10:44:17.340 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 23 10:44:17 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-5[44718]: 2024-04-23T10:44:17.134+0000 7f33ec49b700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:43.772548+0000 front 2024-04-23T10:43:43.772512+0000 (oldest deadline 2024-04-23T10:44:07.872586+0000) 2024-04-23T10:44:17.340 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 23 10:44:17 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-5[44718]: 2024-04-23T10:44:17.134+0000 7f33ec49b700 -1 osd.5 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:51.373720+0000 front 2024-04-23T10:43:51.373761+0000 (oldest deadline 2024-04-23T10:44:15.473905+0000) 2024-04-23T10:44:17.400 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:17 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7[50726]: 2024-04-23T10:44:17.014+0000 7f5dbd523700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:41.728600+0000 front 2024-04-23T10:43:41.728546+0000 (oldest deadline 2024-04-23T10:44:05.228700+0000) 2024-04-23T10:44:17.400 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:17 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7[50726]: 2024-04-23T10:44:17.014+0000 7f5dbd523700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:52.130820+0000 front 2024-04-23T10:43:52.130687+0000 (oldest deadline 2024-04-23T10:44:16.230820+0000) 2024-04-23T10:44:17.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:17 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:17.028+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.211952+0000 front 2024-04-23T10:43:33.211903+0000 (oldest deadline 2024-04-23T10:43:59.012678+0000) 2024-04-23T10:44:17.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:17 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:17.028+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:43.515012+0000 front 2024-04-23T10:43:43.515041+0000 (oldest deadline 2024-04-23T10:44:05.815169+0000) 2024-04-23T10:44:17.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:17 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:17.028+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:51.716168+0000 front 2024-04-23T10:43:51.716308+0000 (oldest deadline 2024-04-23T10:44:17.016216+0000) 2024-04-23T10:44:17.592 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 23 10:44:17 smithi196 podman[71086]: 2024-04-23 10:44:17.542383009 +0000 UTC m=+5.477806592 container died aa998b5674603b1f21e9aa77917be45c6f6edb6379a072e899530868f98c4272 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T10:44:17.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:17 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:17.341+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.335834+0000 front 2024-04-23T10:43:36.335867+0000 (oldest deadline 2024-04-23T10:43:57.435822+0000) 2024-04-23T10:44:17.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:17 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:17.341+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.637810+0000 front 2024-04-23T10:43:42.637720+0000 (oldest deadline 2024-04-23T10:44:07.937634+0000) 2024-04-23T10:44:17.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:17 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:17.341+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:52.538910+0000 front 2024-04-23T10:43:52.538984+0000 (oldest deadline 2024-04-23T10:44:14.238911+0000) 2024-04-23T10:44:17.860 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:17 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6[54668]: 2024-04-23T10:44:17.558+0000 7f85a54bb700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.200851+0000 front 2024-04-23T10:43:33.200795+0000 (oldest deadline 2024-04-23T10:43:57.300941+0000) 2024-04-23T10:44:17.860 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:17 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6[54668]: 2024-04-23T10:44:17.558+0000 7f85a54bb700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:39.002107+0000 front 2024-04-23T10:43:39.001911+0000 (oldest deadline 2024-04-23T10:44:06.002356+0000) 2024-04-23T10:44:17.860 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:17 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6[54668]: 2024-04-23T10:44:17.558+0000 7f85a54bb700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:51.203968+0000 front 2024-04-23T10:43:51.204236+0000 (oldest deadline 2024-04-23T10:44:13.503949+0000) 2024-04-23T10:44:17.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:17 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:17.469+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.896652+0000 front 2024-04-23T10:43:34.896698+0000 (oldest deadline 2024-04-23T10:43:58.396793+0000) 2024-04-23T10:44:17.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:17 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:17.469+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.998590+0000 front 2024-04-23T10:43:42.998576+0000 (oldest deadline 2024-04-23T10:44:05.298692+0000) 2024-04-23T10:44:17.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:17 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:17.469+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:49.900586+0000 front 2024-04-23T10:43:49.900497+0000 (oldest deadline 2024-04-23T10:44:15.200545+0000) 2024-04-23T10:44:18.091 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:17 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:17.866+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.311585+0000 front 2024-04-23T10:43:36.311514+0000 (oldest deadline 2024-04-23T10:43:59.211672+0000) 2024-04-23T10:44:18.091 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:17 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:17.866+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:48.314772+0000 front 2024-04-23T10:43:48.314791+0000 (oldest deadline 2024-04-23T10:44:14.214936+0000) 2024-04-23T10:44:18.092 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 23 10:44:17 smithi196 podman[71086]: 2024-04-23 10:44:17.937339438 +0000 UTC m=+5.872763021 container cleanup aa998b5674603b1f21e9aa77917be45c6f6edb6379a072e899530868f98c4272 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-5, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, ceph=True) 2024-04-23T10:44:18.092 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 23 10:44:17 smithi196 bash[71086]: ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-5 2024-04-23T10:44:18.400 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:18 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7[50726]: 2024-04-23T10:44:18.037+0000 7f5dbd523700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:41.728600+0000 front 2024-04-23T10:43:41.728546+0000 (oldest deadline 2024-04-23T10:44:05.228700+0000) 2024-04-23T10:44:18.400 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:18 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7[50726]: 2024-04-23T10:44:18.037+0000 7f5dbd523700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:52.130820+0000 front 2024-04-23T10:43:52.130687+0000 (oldest deadline 2024-04-23T10:44:16.230820+0000) 2024-04-23T10:44:18.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:17 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:17.991+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.211952+0000 front 2024-04-23T10:43:33.211903+0000 (oldest deadline 2024-04-23T10:43:59.012678+0000) 2024-04-23T10:44:18.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:17 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:17.991+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:43.515012+0000 front 2024-04-23T10:43:43.515041+0000 (oldest deadline 2024-04-23T10:44:05.815169+0000) 2024-04-23T10:44:18.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:17 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:17.991+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:51.716168+0000 front 2024-04-23T10:43:51.716308+0000 (oldest deadline 2024-04-23T10:44:17.016216+0000) 2024-04-23T10:44:18.591 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 23 10:44:18 smithi196 podman[71103]: 2024-04-23 10:44:18.189851863 +0000 UTC m=+0.645498160 container remove aa998b5674603b1f21e9aa77917be45c6f6edb6379a072e899530868f98c4272 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS) 2024-04-23T10:44:18.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:18 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:18.340+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.335834+0000 front 2024-04-23T10:43:36.335867+0000 (oldest deadline 2024-04-23T10:43:57.435822+0000) 2024-04-23T10:44:18.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:18 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:18.340+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.637810+0000 front 2024-04-23T10:43:42.637720+0000 (oldest deadline 2024-04-23T10:44:07.937634+0000) 2024-04-23T10:44:18.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:18 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:18.340+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:52.538910+0000 front 2024-04-23T10:43:52.538984+0000 (oldest deadline 2024-04-23T10:44:14.238911+0000) 2024-04-23T10:44:18.861 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:18 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6[54668]: 2024-04-23T10:44:18.525+0000 7f85a54bb700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.200851+0000 front 2024-04-23T10:43:33.200795+0000 (oldest deadline 2024-04-23T10:43:57.300941+0000) 2024-04-23T10:44:18.861 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:18 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6[54668]: 2024-04-23T10:44:18.525+0000 7f85a54bb700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:39.002107+0000 front 2024-04-23T10:43:39.001911+0000 (oldest deadline 2024-04-23T10:44:06.002356+0000) 2024-04-23T10:44:18.861 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:18 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6[54668]: 2024-04-23T10:44:18.525+0000 7f85a54bb700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:51.203968+0000 front 2024-04-23T10:43:51.204236+0000 (oldest deadline 2024-04-23T10:44:13.503949+0000) 2024-04-23T10:44:18.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:18 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:18.477+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.896652+0000 front 2024-04-23T10:43:34.896698+0000 (oldest deadline 2024-04-23T10:43:58.396793+0000) 2024-04-23T10:44:18.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:18 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:18.477+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.998590+0000 front 2024-04-23T10:43:42.998576+0000 (oldest deadline 2024-04-23T10:44:05.298692+0000) 2024-04-23T10:44:18.862 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:18 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:18.477+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:49.900586+0000 front 2024-04-23T10:43:49.900497+0000 (oldest deadline 2024-04-23T10:44:15.200545+0000) 2024-04-23T10:44:18.925 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:18 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:18.831+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.311585+0000 front 2024-04-23T10:43:36.311514+0000 (oldest deadline 2024-04-23T10:43:59.211672+0000) 2024-04-23T10:44:18.925 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:18 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:18.831+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:48.314772+0000 front 2024-04-23T10:43:48.314791+0000 (oldest deadline 2024-04-23T10:44:14.214936+0000) 2024-04-23T10:44:18.925 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 23 10:44:18 smithi196 podman[71196]: 2024-04-23 10:44:18.511106923 +0000 UTC m=+0.018072594 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T10:44:18.925 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 23 10:44:18 smithi196 podman[71196]: 2024-04-23 10:44:18.624088312 +0000 UTC m=+0.131053970 container create 096e33a387221d7285af6084543fe41a5e5c8c1187974421740dbcc5e76a8dd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240416, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T10:44:18.925 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 23 10:44:18 smithi196 podman[71196]: 2024-04-23 10:44:18.835546395 +0000 UTC m=+0.342512053 container init 096e33a387221d7285af6084543fe41a5e5c8c1187974421740dbcc5e76a8dd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-5-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-23T10:44:18.925 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 23 10:44:18 smithi196 podman[71196]: 2024-04-23 10:44:18.838679185 +0000 UTC m=+0.345644844 container start 096e33a387221d7285af6084543fe41a5e5c8c1187974421740dbcc5e76a8dd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-23T10:44:19.291 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 23 10:44:18 smithi196 podman[71196]: 2024-04-23 10:44:18.927328087 +0000 UTC m=+0.434293745 container attach 096e33a387221d7285af6084543fe41a5e5c8c1187974421740dbcc5e76a8dd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-23T10:44:19.291 INFO:journalctl@ceph.osd.5.smithi196.stdout:Apr 23 10:44:19 smithi196 podman[71196]: 2024-04-23 10:44:19.129666764 +0000 UTC m=+0.636632414 container died 096e33a387221d7285af6084543fe41a5e5c8c1187974421740dbcc5e76a8dd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True) 2024-04-23T10:44:19.400 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:19 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7[50726]: 2024-04-23T10:44:19.077+0000 7f5dbd523700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:41.728600+0000 front 2024-04-23T10:43:41.728546+0000 (oldest deadline 2024-04-23T10:44:05.228700+0000) 2024-04-23T10:44:19.400 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:19 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7[50726]: 2024-04-23T10:44:19.077+0000 7f5dbd523700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:52.130820+0000 front 2024-04-23T10:43:52.130687+0000 (oldest deadline 2024-04-23T10:44:16.230820+0000) 2024-04-23T10:44:19.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:18 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:18.944+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.211952+0000 front 2024-04-23T10:43:33.211903+0000 (oldest deadline 2024-04-23T10:43:59.012678+0000) 2024-04-23T10:44:19.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:18 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:18.944+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:43.515012+0000 front 2024-04-23T10:43:43.515041+0000 (oldest deadline 2024-04-23T10:44:05.815169+0000) 2024-04-23T10:44:19.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:18 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:18.944+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:51.716168+0000 front 2024-04-23T10:43:51.716308+0000 (oldest deadline 2024-04-23T10:44:17.016216+0000) 2024-04-23T10:44:19.591 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:19 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:19.292+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.335834+0000 front 2024-04-23T10:43:36.335867+0000 (oldest deadline 2024-04-23T10:43:57.435822+0000) 2024-04-23T10:44:19.591 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:19 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:19.292+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.637810+0000 front 2024-04-23T10:43:42.637720+0000 (oldest deadline 2024-04-23T10:44:07.937634+0000) 2024-04-23T10:44:19.591 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:19 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:19.292+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:52.538910+0000 front 2024-04-23T10:43:52.538984+0000 (oldest deadline 2024-04-23T10:44:14.238911+0000) 2024-04-23T10:44:19.845 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b22af88-0159-11ef-bc93-c7b262605968@osd.5.service' 2024-04-23T10:44:19.860 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:19 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6[54668]: 2024-04-23T10:44:19.495+0000 7f85a54bb700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.200851+0000 front 2024-04-23T10:43:33.200795+0000 (oldest deadline 2024-04-23T10:43:57.300941+0000) 2024-04-23T10:44:19.860 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:19 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6[54668]: 2024-04-23T10:44:19.495+0000 7f85a54bb700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:39.002107+0000 front 2024-04-23T10:43:39.001911+0000 (oldest deadline 2024-04-23T10:44:06.002356+0000) 2024-04-23T10:44:19.860 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:19 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6[54668]: 2024-04-23T10:44:19.495+0000 7f85a54bb700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:51.203968+0000 front 2024-04-23T10:43:51.204236+0000 (oldest deadline 2024-04-23T10:44:13.503949+0000) 2024-04-23T10:44:19.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:19 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:19.492+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.896652+0000 front 2024-04-23T10:43:34.896698+0000 (oldest deadline 2024-04-23T10:43:58.396793+0000) 2024-04-23T10:44:19.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:19 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:19.492+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.998590+0000 front 2024-04-23T10:43:42.998576+0000 (oldest deadline 2024-04-23T10:44:05.298692+0000) 2024-04-23T10:44:19.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:19 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:19.492+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:49.900586+0000 front 2024-04-23T10:43:49.900497+0000 (oldest deadline 2024-04-23T10:44:15.200545+0000) 2024-04-23T10:44:19.883 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T10:44:19.883 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-23T10:44:19.884 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-23T10:44:19.884 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-9b22af88-0159-11ef-bc93-c7b262605968@osd.6 2024-04-23T10:44:20.091 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:19 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:19.807+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.311585+0000 front 2024-04-23T10:43:36.311514+0000 (oldest deadline 2024-04-23T10:43:59.211672+0000) 2024-04-23T10:44:20.091 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:19 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:19.807+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:48.314772+0000 front 2024-04-23T10:43:48.314791+0000 (oldest deadline 2024-04-23T10:44:14.214936+0000) 2024-04-23T10:44:20.196 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:19 smithi097 systemd[1]: Stopping Ceph osd.6 for 9b22af88-0159-11ef-bc93-c7b262605968... 2024-04-23T10:44:20.400 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:20 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7[50726]: 2024-04-23T10:44:20.040+0000 7f5dbd523700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:41.728600+0000 front 2024-04-23T10:43:41.728546+0000 (oldest deadline 2024-04-23T10:44:05.228700+0000) 2024-04-23T10:44:20.400 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:20 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7[50726]: 2024-04-23T10:44:20.040+0000 7f5dbd523700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:52.130820+0000 front 2024-04-23T10:43:52.130687+0000 (oldest deadline 2024-04-23T10:44:16.230820+0000) 2024-04-23T10:44:20.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:19 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:19.992+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.211952+0000 front 2024-04-23T10:43:33.211903+0000 (oldest deadline 2024-04-23T10:43:59.012678+0000) 2024-04-23T10:44:20.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:19 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:19.992+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:43.515012+0000 front 2024-04-23T10:43:43.515041+0000 (oldest deadline 2024-04-23T10:44:05.815169+0000) 2024-04-23T10:44:20.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:19 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:19.992+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:51.716168+0000 front 2024-04-23T10:43:51.716308+0000 (oldest deadline 2024-04-23T10:44:17.016216+0000) 2024-04-23T10:44:20.517 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:20 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6[54668]: 2024-04-23T10:44:20.198+0000 7f85acecf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T10:44:20.517 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:20 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6[54668]: 2024-04-23T10:44:20.198+0000 7f85acecf700 -1 osd.6 114 *** Got signal Terminated *** 2024-04-23T10:44:20.517 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:20 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6[54668]: 2024-04-23T10:44:20.198+0000 7f85acecf700 -1 osd.6 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T10:44:20.518 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:20 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6[54668]: 2024-04-23T10:44:20.519+0000 7f85a54bb700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.200851+0000 front 2024-04-23T10:43:33.200795+0000 (oldest deadline 2024-04-23T10:43:57.300941+0000) 2024-04-23T10:44:20.518 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:20 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6[54668]: 2024-04-23T10:44:20.519+0000 7f85a54bb700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:39.002107+0000 front 2024-04-23T10:43:39.001911+0000 (oldest deadline 2024-04-23T10:44:06.002356+0000) 2024-04-23T10:44:20.518 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:20 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6[54668]: 2024-04-23T10:44:20.519+0000 7f85a54bb700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:51.203968+0000 front 2024-04-23T10:43:51.204236+0000 (oldest deadline 2024-04-23T10:44:13.503949+0000) 2024-04-23T10:44:20.591 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:20 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:20.286+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.335834+0000 front 2024-04-23T10:43:36.335867+0000 (oldest deadline 2024-04-23T10:43:57.435822+0000) 2024-04-23T10:44:20.591 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:20 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:20.286+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.637810+0000 front 2024-04-23T10:43:42.637720+0000 (oldest deadline 2024-04-23T10:44:07.937634+0000) 2024-04-23T10:44:20.591 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:20 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:20.286+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:52.538910+0000 front 2024-04-23T10:43:52.538984+0000 (oldest deadline 2024-04-23T10:44:14.238911+0000) 2024-04-23T10:44:20.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:20 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:20.527+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.896652+0000 front 2024-04-23T10:43:34.896698+0000 (oldest deadline 2024-04-23T10:43:58.396793+0000) 2024-04-23T10:44:20.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:20 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:20.527+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.998590+0000 front 2024-04-23T10:43:42.998576+0000 (oldest deadline 2024-04-23T10:44:05.298692+0000) 2024-04-23T10:44:20.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:20 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:20.527+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:49.900586+0000 front 2024-04-23T10:43:49.900497+0000 (oldest deadline 2024-04-23T10:44:15.200545+0000) 2024-04-23T10:44:21.091 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:20 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:20.776+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.311585+0000 front 2024-04-23T10:43:36.311514+0000 (oldest deadline 2024-04-23T10:43:59.211672+0000) 2024-04-23T10:44:21.091 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:20 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:20.776+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:48.314772+0000 front 2024-04-23T10:43:48.314791+0000 (oldest deadline 2024-04-23T10:44:14.214936+0000) 2024-04-23T10:44:21.400 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:20 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7[50726]: 2024-04-23T10:44:20.993+0000 7f5dbd523700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:41.728600+0000 front 2024-04-23T10:43:41.728546+0000 (oldest deadline 2024-04-23T10:44:05.228700+0000) 2024-04-23T10:44:21.400 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:20 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7[50726]: 2024-04-23T10:44:20.993+0000 7f5dbd523700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:52.130820+0000 front 2024-04-23T10:43:52.130687+0000 (oldest deadline 2024-04-23T10:44:16.230820+0000) 2024-04-23T10:44:21.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:21 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:21.000+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.211952+0000 front 2024-04-23T10:43:33.211903+0000 (oldest deadline 2024-04-23T10:43:59.012678+0000) 2024-04-23T10:44:21.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:21 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:21.000+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:43.515012+0000 front 2024-04-23T10:43:43.515041+0000 (oldest deadline 2024-04-23T10:44:05.815169+0000) 2024-04-23T10:44:21.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:21 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:21.000+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:51.716168+0000 front 2024-04-23T10:43:51.716308+0000 (oldest deadline 2024-04-23T10:44:17.016216+0000) 2024-04-23T10:44:21.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:21 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:21.332+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.335834+0000 front 2024-04-23T10:43:36.335867+0000 (oldest deadline 2024-04-23T10:43:57.435822+0000) 2024-04-23T10:44:21.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:21 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:21.332+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.637810+0000 front 2024-04-23T10:43:42.637720+0000 (oldest deadline 2024-04-23T10:44:07.937634+0000) 2024-04-23T10:44:21.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:21 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:21.332+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:52.538910+0000 front 2024-04-23T10:43:52.538984+0000 (oldest deadline 2024-04-23T10:44:14.238911+0000) 2024-04-23T10:44:21.861 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:21 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6[54668]: 2024-04-23T10:44:21.566+0000 7f85a54bb700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.200851+0000 front 2024-04-23T10:43:33.200795+0000 (oldest deadline 2024-04-23T10:43:57.300941+0000) 2024-04-23T10:44:21.861 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:21 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6[54668]: 2024-04-23T10:44:21.566+0000 7f85a54bb700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:39.002107+0000 front 2024-04-23T10:43:39.001911+0000 (oldest deadline 2024-04-23T10:44:06.002356+0000) 2024-04-23T10:44:21.861 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:21 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6[54668]: 2024-04-23T10:44:21.566+0000 7f85a54bb700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:51.203968+0000 front 2024-04-23T10:43:51.204236+0000 (oldest deadline 2024-04-23T10:44:13.503949+0000) 2024-04-23T10:44:21.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:21 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:21.566+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.896652+0000 front 2024-04-23T10:43:34.896698+0000 (oldest deadline 2024-04-23T10:43:58.396793+0000) 2024-04-23T10:44:21.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:21 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:21.566+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.998590+0000 front 2024-04-23T10:43:42.998576+0000 (oldest deadline 2024-04-23T10:44:05.298692+0000) 2024-04-23T10:44:21.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:21 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:21.566+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:49.900586+0000 front 2024-04-23T10:43:49.900497+0000 (oldest deadline 2024-04-23T10:44:15.200545+0000) 2024-04-23T10:44:22.092 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:21 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:21.746+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.311585+0000 front 2024-04-23T10:43:36.311514+0000 (oldest deadline 2024-04-23T10:43:59.211672+0000) 2024-04-23T10:44:22.092 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:21 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:21.746+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:48.314772+0000 front 2024-04-23T10:43:48.314791+0000 (oldest deadline 2024-04-23T10:44:14.214936+0000) 2024-04-23T10:44:22.400 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:22 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7[50726]: 2024-04-23T10:44:22.026+0000 7f5dbd523700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:41.728600+0000 front 2024-04-23T10:43:41.728546+0000 (oldest deadline 2024-04-23T10:44:05.228700+0000) 2024-04-23T10:44:22.400 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:22 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7[50726]: 2024-04-23T10:44:22.026+0000 7f5dbd523700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:52.130820+0000 front 2024-04-23T10:43:52.130687+0000 (oldest deadline 2024-04-23T10:44:16.230820+0000) 2024-04-23T10:44:22.400 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:22 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7[50726]: 2024-04-23T10:44:22.026+0000 7f5dbd523700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:44:00.331949+0000 front 2024-04-23T10:44:00.332016+0000 (oldest deadline 2024-04-23T10:44:21.431895+0000) 2024-04-23T10:44:22.401 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:21 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:21.979+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.211952+0000 front 2024-04-23T10:43:33.211903+0000 (oldest deadline 2024-04-23T10:43:59.012678+0000) 2024-04-23T10:44:22.401 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:21 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:21.979+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:43.515012+0000 front 2024-04-23T10:43:43.515041+0000 (oldest deadline 2024-04-23T10:44:05.815169+0000) 2024-04-23T10:44:22.401 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:21 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:21.979+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:51.716168+0000 front 2024-04-23T10:43:51.716308+0000 (oldest deadline 2024-04-23T10:44:17.016216+0000) 2024-04-23T10:44:22.591 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:22 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:22.284+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.335834+0000 front 2024-04-23T10:43:36.335867+0000 (oldest deadline 2024-04-23T10:43:57.435822+0000) 2024-04-23T10:44:22.591 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:22 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:22.284+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.637810+0000 front 2024-04-23T10:43:42.637720+0000 (oldest deadline 2024-04-23T10:44:07.937634+0000) 2024-04-23T10:44:22.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:22 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:22.284+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:52.538910+0000 front 2024-04-23T10:43:52.538984+0000 (oldest deadline 2024-04-23T10:44:14.238911+0000) 2024-04-23T10:44:22.860 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:22 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6[54668]: 2024-04-23T10:44:22.563+0000 7f85a54bb700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.200851+0000 front 2024-04-23T10:43:33.200795+0000 (oldest deadline 2024-04-23T10:43:57.300941+0000) 2024-04-23T10:44:22.860 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:22 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6[54668]: 2024-04-23T10:44:22.563+0000 7f85a54bb700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:39.002107+0000 front 2024-04-23T10:43:39.001911+0000 (oldest deadline 2024-04-23T10:44:06.002356+0000) 2024-04-23T10:44:22.861 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:22 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6[54668]: 2024-04-23T10:44:22.563+0000 7f85a54bb700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:51.203968+0000 front 2024-04-23T10:43:51.204236+0000 (oldest deadline 2024-04-23T10:44:13.503949+0000) 2024-04-23T10:44:22.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:22 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:22.549+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.896652+0000 front 2024-04-23T10:43:34.896698+0000 (oldest deadline 2024-04-23T10:43:58.396793+0000) 2024-04-23T10:44:22.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:22 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:22.549+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.998590+0000 front 2024-04-23T10:43:42.998576+0000 (oldest deadline 2024-04-23T10:44:05.298692+0000) 2024-04-23T10:44:22.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:22 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:22.549+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:49.900586+0000 front 2024-04-23T10:43:49.900497+0000 (oldest deadline 2024-04-23T10:44:15.200545+0000) 2024-04-23T10:44:23.091 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:22 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:22.775+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.311585+0000 front 2024-04-23T10:43:36.311514+0000 (oldest deadline 2024-04-23T10:43:59.211672+0000) 2024-04-23T10:44:23.091 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:22 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:22.775+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:48.314772+0000 front 2024-04-23T10:43:48.314791+0000 (oldest deadline 2024-04-23T10:44:14.214936+0000) 2024-04-23T10:44:23.400 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:23 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7[50726]: 2024-04-23T10:44:23.015+0000 7f5dbd523700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:41.728600+0000 front 2024-04-23T10:43:41.728546+0000 (oldest deadline 2024-04-23T10:44:05.228700+0000) 2024-04-23T10:44:23.400 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:23 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7[50726]: 2024-04-23T10:44:23.015+0000 7f5dbd523700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:52.130820+0000 front 2024-04-23T10:43:52.130687+0000 (oldest deadline 2024-04-23T10:44:16.230820+0000) 2024-04-23T10:44:23.400 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:23 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7[50726]: 2024-04-23T10:44:23.015+0000 7f5dbd523700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:44:00.331949+0000 front 2024-04-23T10:44:00.332016+0000 (oldest deadline 2024-04-23T10:44:21.431895+0000) 2024-04-23T10:44:23.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:23 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:23.008+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.211952+0000 front 2024-04-23T10:43:33.211903+0000 (oldest deadline 2024-04-23T10:43:59.012678+0000) 2024-04-23T10:44:23.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:23 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:23.008+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:43.515012+0000 front 2024-04-23T10:43:43.515041+0000 (oldest deadline 2024-04-23T10:44:05.815169+0000) 2024-04-23T10:44:23.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:23 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:23.008+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:51.716168+0000 front 2024-04-23T10:43:51.716308+0000 (oldest deadline 2024-04-23T10:44:17.016216+0000) 2024-04-23T10:44:23.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:23 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:23.008+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:57.517193+0000 front 2024-04-23T10:43:57.517281+0000 (oldest deadline 2024-04-23T10:44:22.817359+0000) 2024-04-23T10:44:23.591 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:23 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:23.327+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.335834+0000 front 2024-04-23T10:43:36.335867+0000 (oldest deadline 2024-04-23T10:43:57.435822+0000) 2024-04-23T10:44:23.591 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:23 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:23.327+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.637810+0000 front 2024-04-23T10:43:42.637720+0000 (oldest deadline 2024-04-23T10:44:07.937634+0000) 2024-04-23T10:44:23.591 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:23 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:23.327+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:52.538910+0000 front 2024-04-23T10:43:52.538984+0000 (oldest deadline 2024-04-23T10:44:14.238911+0000) 2024-04-23T10:44:23.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:23 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:23.595+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.896652+0000 front 2024-04-23T10:43:34.896698+0000 (oldest deadline 2024-04-23T10:43:58.396793+0000) 2024-04-23T10:44:23.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:23 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:23.595+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.998590+0000 front 2024-04-23T10:43:42.998576+0000 (oldest deadline 2024-04-23T10:44:05.298692+0000) 2024-04-23T10:44:23.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:23 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:23.595+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:49.900586+0000 front 2024-04-23T10:43:49.900497+0000 (oldest deadline 2024-04-23T10:44:15.200545+0000) 2024-04-23T10:44:23.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:23 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:23.595+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:59.802141+0000 front 2024-04-23T10:43:59.802168+0000 (oldest deadline 2024-04-23T10:44:23.302165+0000) 2024-04-23T10:44:23.860 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:23 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6[54668]: 2024-04-23T10:44:23.593+0000 7f85a54bb700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.200851+0000 front 2024-04-23T10:43:33.200795+0000 (oldest deadline 2024-04-23T10:43:57.300941+0000) 2024-04-23T10:44:23.861 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:23 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6[54668]: 2024-04-23T10:44:23.593+0000 7f85a54bb700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:39.002107+0000 front 2024-04-23T10:43:39.001911+0000 (oldest deadline 2024-04-23T10:44:06.002356+0000) 2024-04-23T10:44:23.861 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:23 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6[54668]: 2024-04-23T10:44:23.593+0000 7f85a54bb700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:51.203968+0000 front 2024-04-23T10:43:51.204236+0000 (oldest deadline 2024-04-23T10:44:13.503949+0000) 2024-04-23T10:44:23.861 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:23 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6[54668]: 2024-04-23T10:44:23.593+0000 7f85a54bb700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:59.805771+0000 front 2024-04-23T10:43:59.805734+0000 (oldest deadline 2024-04-23T10:44:22.705847+0000) 2024-04-23T10:44:24.091 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:23 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:23.783+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.311585+0000 front 2024-04-23T10:43:36.311514+0000 (oldest deadline 2024-04-23T10:43:59.211672+0000) 2024-04-23T10:44:24.091 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:23 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:23.783+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:48.314772+0000 front 2024-04-23T10:43:48.314791+0000 (oldest deadline 2024-04-23T10:44:14.214936+0000) 2024-04-23T10:44:24.400 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:23 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7[50726]: 2024-04-23T10:44:23.975+0000 7f5dbd523700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:41.728600+0000 front 2024-04-23T10:43:41.728546+0000 (oldest deadline 2024-04-23T10:44:05.228700+0000) 2024-04-23T10:44:24.400 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:23 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7[50726]: 2024-04-23T10:44:23.975+0000 7f5dbd523700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:52.130820+0000 front 2024-04-23T10:43:52.130687+0000 (oldest deadline 2024-04-23T10:44:16.230820+0000) 2024-04-23T10:44:24.400 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:23 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7[50726]: 2024-04-23T10:44:23.975+0000 7f5dbd523700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:44:00.331949+0000 front 2024-04-23T10:44:00.332016+0000 (oldest deadline 2024-04-23T10:44:21.431895+0000) 2024-04-23T10:44:24.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:23 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:23.967+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.211952+0000 front 2024-04-23T10:43:33.211903+0000 (oldest deadline 2024-04-23T10:43:59.012678+0000) 2024-04-23T10:44:24.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:23 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:23.967+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:43.515012+0000 front 2024-04-23T10:43:43.515041+0000 (oldest deadline 2024-04-23T10:44:05.815169+0000) 2024-04-23T10:44:24.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:23 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:23.967+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:51.716168+0000 front 2024-04-23T10:43:51.716308+0000 (oldest deadline 2024-04-23T10:44:17.016216+0000) 2024-04-23T10:44:24.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:23 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:23.967+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:57.517193+0000 front 2024-04-23T10:43:57.517281+0000 (oldest deadline 2024-04-23T10:44:22.817359+0000) 2024-04-23T10:44:24.763 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:24 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:24.373+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.335834+0000 front 2024-04-23T10:43:36.335867+0000 (oldest deadline 2024-04-23T10:43:57.435822+0000) 2024-04-23T10:44:24.763 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:24 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:24.373+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.637810+0000 front 2024-04-23T10:43:42.637720+0000 (oldest deadline 2024-04-23T10:44:07.937634+0000) 2024-04-23T10:44:24.763 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:24 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:24.373+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:52.538910+0000 front 2024-04-23T10:43:52.538984+0000 (oldest deadline 2024-04-23T10:44:14.238911+0000) 2024-04-23T10:44:24.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:24 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:24.640+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.896652+0000 front 2024-04-23T10:43:34.896698+0000 (oldest deadline 2024-04-23T10:43:58.396793+0000) 2024-04-23T10:44:24.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:24 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:24.640+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.998590+0000 front 2024-04-23T10:43:42.998576+0000 (oldest deadline 2024-04-23T10:44:05.298692+0000) 2024-04-23T10:44:24.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:24 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:24.640+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:49.900586+0000 front 2024-04-23T10:43:49.900497+0000 (oldest deadline 2024-04-23T10:44:15.200545+0000) 2024-04-23T10:44:24.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:24 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:24.640+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:59.802141+0000 front 2024-04-23T10:43:59.802168+0000 (oldest deadline 2024-04-23T10:44:23.302165+0000) 2024-04-23T10:44:24.861 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:24 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6[54668]: 2024-04-23T10:44:24.605+0000 7f85a54bb700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.200851+0000 front 2024-04-23T10:43:33.200795+0000 (oldest deadline 2024-04-23T10:43:57.300941+0000) 2024-04-23T10:44:24.861 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:24 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6[54668]: 2024-04-23T10:44:24.605+0000 7f85a54bb700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:39.002107+0000 front 2024-04-23T10:43:39.001911+0000 (oldest deadline 2024-04-23T10:44:06.002356+0000) 2024-04-23T10:44:24.861 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:24 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6[54668]: 2024-04-23T10:44:24.605+0000 7f85a54bb700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:51.203968+0000 front 2024-04-23T10:43:51.204236+0000 (oldest deadline 2024-04-23T10:44:13.503949+0000) 2024-04-23T10:44:24.861 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:24 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6[54668]: 2024-04-23T10:44:24.605+0000 7f85a54bb700 -1 osd.6 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:59.805771+0000 front 2024-04-23T10:43:59.805734+0000 (oldest deadline 2024-04-23T10:44:22.705847+0000) 2024-04-23T10:44:25.091 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:24 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:24.764+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.311585+0000 front 2024-04-23T10:43:36.311514+0000 (oldest deadline 2024-04-23T10:43:59.211672+0000) 2024-04-23T10:44:25.091 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:24 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:24.764+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:48.314772+0000 front 2024-04-23T10:43:48.314791+0000 (oldest deadline 2024-04-23T10:44:14.214936+0000) 2024-04-23T10:44:25.091 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:24 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:24.764+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:58.316108+0000 front 2024-04-23T10:43:58.315933+0000 (oldest deadline 2024-04-23T10:44:24.215991+0000) 2024-04-23T10:44:25.400 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:24 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7[50726]: 2024-04-23T10:44:24.942+0000 7f5dbd523700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:41.728600+0000 front 2024-04-23T10:43:41.728546+0000 (oldest deadline 2024-04-23T10:44:05.228700+0000) 2024-04-23T10:44:25.400 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:24 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7[50726]: 2024-04-23T10:44:24.942+0000 7f5dbd523700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:52.130820+0000 front 2024-04-23T10:43:52.130687+0000 (oldest deadline 2024-04-23T10:44:16.230820+0000) 2024-04-23T10:44:25.400 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:24 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7[50726]: 2024-04-23T10:44:24.942+0000 7f5dbd523700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:44:00.331949+0000 front 2024-04-23T10:44:00.332016+0000 (oldest deadline 2024-04-23T10:44:21.431895+0000) 2024-04-23T10:44:25.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:24 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:24.988+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.211952+0000 front 2024-04-23T10:43:33.211903+0000 (oldest deadline 2024-04-23T10:43:59.012678+0000) 2024-04-23T10:44:25.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:24 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:24.988+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:43.515012+0000 front 2024-04-23T10:43:43.515041+0000 (oldest deadline 2024-04-23T10:44:05.815169+0000) 2024-04-23T10:44:25.401 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:24 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:24.988+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:51.716168+0000 front 2024-04-23T10:43:51.716308+0000 (oldest deadline 2024-04-23T10:44:17.016216+0000) 2024-04-23T10:44:25.401 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:24 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:24.988+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:57.517193+0000 front 2024-04-23T10:43:57.517281+0000 (oldest deadline 2024-04-23T10:44:22.817359+0000) 2024-04-23T10:44:25.771 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:25 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:25.417+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.335834+0000 front 2024-04-23T10:43:36.335867+0000 (oldest deadline 2024-04-23T10:43:57.435822+0000) 2024-04-23T10:44:25.771 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:25 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:25.417+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.637810+0000 front 2024-04-23T10:43:42.637720+0000 (oldest deadline 2024-04-23T10:44:07.937634+0000) 2024-04-23T10:44:25.771 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:25 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:25.417+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:52.538910+0000 front 2024-04-23T10:43:52.538984+0000 (oldest deadline 2024-04-23T10:44:14.238911+0000) 2024-04-23T10:44:25.860 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:25 smithi097 podman[94471]: 2024-04-23 10:44:25.647710401 +0000 UTC m=+5.570596032 container died 8c5f7ac7861f7f08ff3fd7ed634544f29f0167e621c6f524c78b966a00ea7378 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T10:44:25.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:25 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:25.606+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.896652+0000 front 2024-04-23T10:43:34.896698+0000 (oldest deadline 2024-04-23T10:43:58.396793+0000) 2024-04-23T10:44:25.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:25 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:25.606+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.998590+0000 front 2024-04-23T10:43:42.998576+0000 (oldest deadline 2024-04-23T10:44:05.298692+0000) 2024-04-23T10:44:25.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:25 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:25.606+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:49.900586+0000 front 2024-04-23T10:43:49.900497+0000 (oldest deadline 2024-04-23T10:44:15.200545+0000) 2024-04-23T10:44:25.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:25 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:25.606+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:59.802141+0000 front 2024-04-23T10:43:59.802168+0000 (oldest deadline 2024-04-23T10:44:23.302165+0000) 2024-04-23T10:44:26.091 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:25 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:25.772+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.311585+0000 front 2024-04-23T10:43:36.311514+0000 (oldest deadline 2024-04-23T10:43:59.211672+0000) 2024-04-23T10:44:26.091 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:25 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:25.772+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:48.314772+0000 front 2024-04-23T10:43:48.314791+0000 (oldest deadline 2024-04-23T10:44:14.214936+0000) 2024-04-23T10:44:26.092 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:25 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:25.772+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:58.316108+0000 front 2024-04-23T10:43:58.315933+0000 (oldest deadline 2024-04-23T10:44:24.215991+0000) 2024-04-23T10:44:26.297 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:26 smithi097 podman[94471]: 2024-04-23 10:44:26.039927351 +0000 UTC m=+5.962812982 container cleanup 8c5f7ac7861f7f08ff3fd7ed634544f29f0167e621c6f524c78b966a00ea7378 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416) 2024-04-23T10:44:26.297 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:26 smithi097 bash[94471]: ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6 2024-04-23T10:44:26.400 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:25 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7[50726]: 2024-04-23T10:44:25.985+0000 7f5dbd523700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:41.728600+0000 front 2024-04-23T10:43:41.728546+0000 (oldest deadline 2024-04-23T10:44:05.228700+0000) 2024-04-23T10:44:26.400 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:25 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7[50726]: 2024-04-23T10:44:25.986+0000 7f5dbd523700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:52.130820+0000 front 2024-04-23T10:43:52.130687+0000 (oldest deadline 2024-04-23T10:44:16.230820+0000) 2024-04-23T10:44:26.400 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:25 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7[50726]: 2024-04-23T10:44:25.986+0000 7f5dbd523700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:44:00.331949+0000 front 2024-04-23T10:44:00.332016+0000 (oldest deadline 2024-04-23T10:44:21.431895+0000) 2024-04-23T10:44:26.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:25 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:25.993+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.211952+0000 front 2024-04-23T10:43:33.211903+0000 (oldest deadline 2024-04-23T10:43:59.012678+0000) 2024-04-23T10:44:26.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:25 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:25.993+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:43.515012+0000 front 2024-04-23T10:43:43.515041+0000 (oldest deadline 2024-04-23T10:44:05.815169+0000) 2024-04-23T10:44:26.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:25 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:25.993+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:51.716168+0000 front 2024-04-23T10:43:51.716308+0000 (oldest deadline 2024-04-23T10:44:17.016216+0000) 2024-04-23T10:44:26.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:25 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:25.993+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:57.517193+0000 front 2024-04-23T10:43:57.517281+0000 (oldest deadline 2024-04-23T10:44:22.817359+0000) 2024-04-23T10:44:26.560 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:26 smithi097 podman[94488]: 2024-04-23 10:44:26.299501528 +0000 UTC m=+0.649267418 container remove 8c5f7ac7861f7f08ff3fd7ed634544f29f0167e621c6f524c78b966a00ea7378 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True) 2024-04-23T10:44:26.797 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:26 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:26.390+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.335834+0000 front 2024-04-23T10:43:36.335867+0000 (oldest deadline 2024-04-23T10:43:57.435822+0000) 2024-04-23T10:44:26.798 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:26 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:26.390+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.637810+0000 front 2024-04-23T10:43:42.637720+0000 (oldest deadline 2024-04-23T10:44:07.937634+0000) 2024-04-23T10:44:26.798 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:26 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:26.390+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:52.538910+0000 front 2024-04-23T10:43:52.538984+0000 (oldest deadline 2024-04-23T10:44:14.238911+0000) 2024-04-23T10:44:26.811 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:26 smithi097 podman[94579]: 2024-04-23 10:44:26.561921311 +0000 UTC m=+0.018228439 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T10:44:26.811 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:26 smithi097 podman[94579]: 2024-04-23 10:44:26.683743441 +0000 UTC m=+0.140050597 container create b02a232903195fc7aba806d5f0f911dc570fc7d2b7477ecc54e26c60e3baf79b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T10:44:26.811 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:26 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:26.561+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.896652+0000 front 2024-04-23T10:43:34.896698+0000 (oldest deadline 2024-04-23T10:43:58.396793+0000) 2024-04-23T10:44:26.811 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:26 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:26.561+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.998590+0000 front 2024-04-23T10:43:42.998576+0000 (oldest deadline 2024-04-23T10:44:05.298692+0000) 2024-04-23T10:44:26.811 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:26 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:26.561+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:49.900586+0000 front 2024-04-23T10:43:49.900497+0000 (oldest deadline 2024-04-23T10:44:15.200545+0000) 2024-04-23T10:44:26.811 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:26 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:26.561+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:59.802141+0000 front 2024-04-23T10:43:59.802168+0000 (oldest deadline 2024-04-23T10:44:23.302165+0000) 2024-04-23T10:44:27.091 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:26 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:26.799+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.311585+0000 front 2024-04-23T10:43:36.311514+0000 (oldest deadline 2024-04-23T10:43:59.211672+0000) 2024-04-23T10:44:27.091 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:26 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:26.799+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:48.314772+0000 front 2024-04-23T10:43:48.314791+0000 (oldest deadline 2024-04-23T10:44:14.214936+0000) 2024-04-23T10:44:27.091 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:26 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:26.799+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:58.316108+0000 front 2024-04-23T10:43:58.315933+0000 (oldest deadline 2024-04-23T10:44:24.215991+0000) 2024-04-23T10:44:27.197 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:26 smithi097 podman[94579]: 2024-04-23 10:44:26.902887323 +0000 UTC m=+0.359194423 container init b02a232903195fc7aba806d5f0f911dc570fc7d2b7477ecc54e26c60e3baf79b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-23T10:44:27.197 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:26 smithi097 podman[94579]: 2024-04-23 10:44:26.90590691 +0000 UTC m=+0.362214011 container start b02a232903195fc7aba806d5f0f911dc570fc7d2b7477ecc54e26c60e3baf79b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD) 2024-04-23T10:44:27.197 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:26 smithi097 podman[94579]: 2024-04-23 10:44:26.960984093 +0000 UTC m=+0.417291209 container attach b02a232903195fc7aba806d5f0f911dc570fc7d2b7477ecc54e26c60e3baf79b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T10:44:27.400 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:26 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7[50726]: 2024-04-23T10:44:26.946+0000 7f5dbd523700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:41.728600+0000 front 2024-04-23T10:43:41.728546+0000 (oldest deadline 2024-04-23T10:44:05.228700+0000) 2024-04-23T10:44:27.400 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:26 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7[50726]: 2024-04-23T10:44:26.946+0000 7f5dbd523700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:52.130820+0000 front 2024-04-23T10:43:52.130687+0000 (oldest deadline 2024-04-23T10:44:16.230820+0000) 2024-04-23T10:44:27.400 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:26 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7[50726]: 2024-04-23T10:44:26.946+0000 7f5dbd523700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:44:00.331949+0000 front 2024-04-23T10:44:00.332016+0000 (oldest deadline 2024-04-23T10:44:21.431895+0000) 2024-04-23T10:44:27.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:27 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:27.019+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.211952+0000 front 2024-04-23T10:43:33.211903+0000 (oldest deadline 2024-04-23T10:43:59.012678+0000) 2024-04-23T10:44:27.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:27 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:27.019+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:43.515012+0000 front 2024-04-23T10:43:43.515041+0000 (oldest deadline 2024-04-23T10:44:05.815169+0000) 2024-04-23T10:44:27.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:27 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:27.019+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:51.716168+0000 front 2024-04-23T10:43:51.716308+0000 (oldest deadline 2024-04-23T10:44:17.016216+0000) 2024-04-23T10:44:27.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:27 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:27.019+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:57.517193+0000 front 2024-04-23T10:43:57.517281+0000 (oldest deadline 2024-04-23T10:44:22.817359+0000) 2024-04-23T10:44:27.478 INFO:journalctl@ceph.osd.6.smithi097.stdout:Apr 23 10:44:27 smithi097 podman[94579]: 2024-04-23 10:44:27.199607472 +0000 UTC m=+0.655914591 container died b02a232903195fc7aba806d5f0f911dc570fc7d2b7477ecc54e26c60e3baf79b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416) 2024-04-23T10:44:27.754 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:27 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:27.422+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.335834+0000 front 2024-04-23T10:43:36.335867+0000 (oldest deadline 2024-04-23T10:43:57.435822+0000) 2024-04-23T10:44:27.754 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:27 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:27.422+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.637810+0000 front 2024-04-23T10:43:42.637720+0000 (oldest deadline 2024-04-23T10:44:07.937634+0000) 2024-04-23T10:44:27.754 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:27 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:27.422+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:52.538910+0000 front 2024-04-23T10:43:52.538984+0000 (oldest deadline 2024-04-23T10:44:14.238911+0000) 2024-04-23T10:44:27.779 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:27 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:27.585+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.896652+0000 front 2024-04-23T10:43:34.896698+0000 (oldest deadline 2024-04-23T10:43:58.396793+0000) 2024-04-23T10:44:27.779 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:27 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:27.585+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.998590+0000 front 2024-04-23T10:43:42.998576+0000 (oldest deadline 2024-04-23T10:44:05.298692+0000) 2024-04-23T10:44:27.779 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:27 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:27.585+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:49.900586+0000 front 2024-04-23T10:43:49.900497+0000 (oldest deadline 2024-04-23T10:44:15.200545+0000) 2024-04-23T10:44:27.779 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:27 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:27.585+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:59.802141+0000 front 2024-04-23T10:43:59.802168+0000 (oldest deadline 2024-04-23T10:44:23.302165+0000) 2024-04-23T10:44:27.794 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b22af88-0159-11ef-bc93-c7b262605968@osd.6.service' 2024-04-23T10:44:27.832 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T10:44:27.832 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-23T10:44:27.832 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-23T10:44:27.832 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-9b22af88-0159-11ef-bc93-c7b262605968@osd.7 2024-04-23T10:44:28.091 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:27 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:27.755+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.311585+0000 front 2024-04-23T10:43:36.311514+0000 (oldest deadline 2024-04-23T10:43:59.211672+0000) 2024-04-23T10:44:28.091 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:27 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:27.755+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:48.314772+0000 front 2024-04-23T10:43:48.314791+0000 (oldest deadline 2024-04-23T10:44:14.214936+0000) 2024-04-23T10:44:28.091 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:27 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:27.755+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:58.316108+0000 front 2024-04-23T10:43:58.315933+0000 (oldest deadline 2024-04-23T10:44:24.215991+0000) 2024-04-23T10:44:28.136 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:28 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:28.058+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.211952+0000 front 2024-04-23T10:43:33.211903+0000 (oldest deadline 2024-04-23T10:43:59.012678+0000) 2024-04-23T10:44:28.136 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:28 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:28.058+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:43.515012+0000 front 2024-04-23T10:43:43.515041+0000 (oldest deadline 2024-04-23T10:44:05.815169+0000) 2024-04-23T10:44:28.136 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:28 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:28.058+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:51.716168+0000 front 2024-04-23T10:43:51.716308+0000 (oldest deadline 2024-04-23T10:44:17.016216+0000) 2024-04-23T10:44:28.136 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:28 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:28.058+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:57.517193+0000 front 2024-04-23T10:43:57.517281+0000 (oldest deadline 2024-04-23T10:44:22.817359+0000) 2024-04-23T10:44:28.137 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:27 smithi183 systemd[1]: Stopping Ceph osd.7 for 9b22af88-0159-11ef-bc93-c7b262605968... 2024-04-23T10:44:28.137 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:27 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7[50726]: 2024-04-23T10:44:27.928+0000 7f5dbd523700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:41.728600+0000 front 2024-04-23T10:43:41.728546+0000 (oldest deadline 2024-04-23T10:44:05.228700+0000) 2024-04-23T10:44:28.137 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:27 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7[50726]: 2024-04-23T10:44:27.929+0000 7f5dbd523700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:52.130820+0000 front 2024-04-23T10:43:52.130687+0000 (oldest deadline 2024-04-23T10:44:16.230820+0000) 2024-04-23T10:44:28.137 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:27 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7[50726]: 2024-04-23T10:44:27.929+0000 7f5dbd523700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:44:00.331949+0000 front 2024-04-23T10:44:00.332016+0000 (oldest deadline 2024-04-23T10:44:21.431895+0000) 2024-04-23T10:44:28.137 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:28 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7[50726]: 2024-04-23T10:44:28.138+0000 7f5dc4f37700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T10:44:28.137 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:28 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7[50726]: 2024-04-23T10:44:28.138+0000 7f5dc4f37700 -1 osd.7 114 *** Got signal Terminated *** 2024-04-23T10:44:28.137 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:28 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7[50726]: 2024-04-23T10:44:28.138+0000 7f5dc4f37700 -1 osd.7 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T10:44:28.713 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:28 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:28.375+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.335834+0000 front 2024-04-23T10:43:36.335867+0000 (oldest deadline 2024-04-23T10:43:57.435822+0000) 2024-04-23T10:44:28.713 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:28 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:28.375+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.637810+0000 front 2024-04-23T10:43:42.637720+0000 (oldest deadline 2024-04-23T10:44:07.937634+0000) 2024-04-23T10:44:28.713 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:28 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:28.375+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:52.538910+0000 front 2024-04-23T10:43:52.538984+0000 (oldest deadline 2024-04-23T10:44:14.238911+0000) 2024-04-23T10:44:28.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:28 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:28.584+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.896652+0000 front 2024-04-23T10:43:34.896698+0000 (oldest deadline 2024-04-23T10:43:58.396793+0000) 2024-04-23T10:44:28.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:28 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:28.584+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.998590+0000 front 2024-04-23T10:43:42.998576+0000 (oldest deadline 2024-04-23T10:44:05.298692+0000) 2024-04-23T10:44:28.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:28 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:28.584+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:49.900586+0000 front 2024-04-23T10:43:49.900497+0000 (oldest deadline 2024-04-23T10:44:15.200545+0000) 2024-04-23T10:44:28.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:28 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:28.584+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:59.802141+0000 front 2024-04-23T10:43:59.802168+0000 (oldest deadline 2024-04-23T10:44:23.302165+0000) 2024-04-23T10:44:28.989 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:28 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:28.714+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.311585+0000 front 2024-04-23T10:43:36.311514+0000 (oldest deadline 2024-04-23T10:43:59.211672+0000) 2024-04-23T10:44:28.989 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:28 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:28.714+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:48.314772+0000 front 2024-04-23T10:43:48.314791+0000 (oldest deadline 2024-04-23T10:44:14.214936+0000) 2024-04-23T10:44:28.989 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:28 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:28.714+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:58.316108+0000 front 2024-04-23T10:43:58.315933+0000 (oldest deadline 2024-04-23T10:44:24.215991+0000) 2024-04-23T10:44:29.309 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:28 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7[50726]: 2024-04-23T10:44:28.963+0000 7f5dbd523700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:41.728600+0000 front 2024-04-23T10:43:41.728546+0000 (oldest deadline 2024-04-23T10:44:05.228700+0000) 2024-04-23T10:44:29.310 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:28 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7[50726]: 2024-04-23T10:44:28.963+0000 7f5dbd523700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:52.130820+0000 front 2024-04-23T10:43:52.130687+0000 (oldest deadline 2024-04-23T10:44:16.230820+0000) 2024-04-23T10:44:29.310 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:28 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7[50726]: 2024-04-23T10:44:28.963+0000 7f5dbd523700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:44:00.331949+0000 front 2024-04-23T10:44:00.332016+0000 (oldest deadline 2024-04-23T10:44:21.431895+0000) 2024-04-23T10:44:29.310 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:29 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:29.018+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.211952+0000 front 2024-04-23T10:43:33.211903+0000 (oldest deadline 2024-04-23T10:43:59.012678+0000) 2024-04-23T10:44:29.310 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:29 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:29.018+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:43.515012+0000 front 2024-04-23T10:43:43.515041+0000 (oldest deadline 2024-04-23T10:44:05.815169+0000) 2024-04-23T10:44:29.310 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:29 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:29.018+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:51.716168+0000 front 2024-04-23T10:43:51.716308+0000 (oldest deadline 2024-04-23T10:44:17.016216+0000) 2024-04-23T10:44:29.310 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:29 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:29.018+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:57.517193+0000 front 2024-04-23T10:43:57.517281+0000 (oldest deadline 2024-04-23T10:44:22.817359+0000) 2024-04-23T10:44:29.743 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:29 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:29.352+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.335834+0000 front 2024-04-23T10:43:36.335867+0000 (oldest deadline 2024-04-23T10:43:57.435822+0000) 2024-04-23T10:44:29.743 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:29 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:29.352+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.637810+0000 front 2024-04-23T10:43:42.637720+0000 (oldest deadline 2024-04-23T10:44:07.937634+0000) 2024-04-23T10:44:29.743 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:29 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:29.352+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:52.538910+0000 front 2024-04-23T10:43:52.538984+0000 (oldest deadline 2024-04-23T10:44:14.238911+0000) 2024-04-23T10:44:29.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:29 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:29.544+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.896652+0000 front 2024-04-23T10:43:34.896698+0000 (oldest deadline 2024-04-23T10:43:58.396793+0000) 2024-04-23T10:44:29.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:29 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:29.544+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.998590+0000 front 2024-04-23T10:43:42.998576+0000 (oldest deadline 2024-04-23T10:44:05.298692+0000) 2024-04-23T10:44:29.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:29 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:29.544+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:49.900586+0000 front 2024-04-23T10:43:49.900497+0000 (oldest deadline 2024-04-23T10:44:15.200545+0000) 2024-04-23T10:44:29.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:29 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:29.544+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:59.802141+0000 front 2024-04-23T10:43:59.802168+0000 (oldest deadline 2024-04-23T10:44:23.302165+0000) 2024-04-23T10:44:30.091 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:29 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:29.744+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.311585+0000 front 2024-04-23T10:43:36.311514+0000 (oldest deadline 2024-04-23T10:43:59.211672+0000) 2024-04-23T10:44:30.092 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:29 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:29.744+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:48.314772+0000 front 2024-04-23T10:43:48.314791+0000 (oldest deadline 2024-04-23T10:44:14.214936+0000) 2024-04-23T10:44:30.092 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:29 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:29.744+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:58.316108+0000 front 2024-04-23T10:43:58.315933+0000 (oldest deadline 2024-04-23T10:44:24.215991+0000) 2024-04-23T10:44:30.400 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:29 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7[50726]: 2024-04-23T10:44:29.915+0000 7f5dbd523700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:41.728600+0000 front 2024-04-23T10:43:41.728546+0000 (oldest deadline 2024-04-23T10:44:05.228700+0000) 2024-04-23T10:44:30.400 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:29 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7[50726]: 2024-04-23T10:44:29.916+0000 7f5dbd523700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:52.130820+0000 front 2024-04-23T10:43:52.130687+0000 (oldest deadline 2024-04-23T10:44:16.230820+0000) 2024-04-23T10:44:30.400 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:29 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7[50726]: 2024-04-23T10:44:29.916+0000 7f5dbd523700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:44:00.331949+0000 front 2024-04-23T10:44:00.332016+0000 (oldest deadline 2024-04-23T10:44:21.431895+0000) 2024-04-23T10:44:30.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:30 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:30.052+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.211952+0000 front 2024-04-23T10:43:33.211903+0000 (oldest deadline 2024-04-23T10:43:59.012678+0000) 2024-04-23T10:44:30.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:30 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:30.052+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:43.515012+0000 front 2024-04-23T10:43:43.515041+0000 (oldest deadline 2024-04-23T10:44:05.815169+0000) 2024-04-23T10:44:30.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:30 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:30.052+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:51.716168+0000 front 2024-04-23T10:43:51.716308+0000 (oldest deadline 2024-04-23T10:44:17.016216+0000) 2024-04-23T10:44:30.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:30 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:30.052+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:57.517193+0000 front 2024-04-23T10:43:57.517281+0000 (oldest deadline 2024-04-23T10:44:22.817359+0000) 2024-04-23T10:44:30.738 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:30 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:30.395+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.335834+0000 front 2024-04-23T10:43:36.335867+0000 (oldest deadline 2024-04-23T10:43:57.435822+0000) 2024-04-23T10:44:30.738 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:30 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:30.395+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.637810+0000 front 2024-04-23T10:43:42.637720+0000 (oldest deadline 2024-04-23T10:44:07.937634+0000) 2024-04-23T10:44:30.738 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:30 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:30.395+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:52.538910+0000 front 2024-04-23T10:43:52.538984+0000 (oldest deadline 2024-04-23T10:44:14.238911+0000) 2024-04-23T10:44:30.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:30 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:30.551+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.896652+0000 front 2024-04-23T10:43:34.896698+0000 (oldest deadline 2024-04-23T10:43:58.396793+0000) 2024-04-23T10:44:30.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:30 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:30.551+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.998590+0000 front 2024-04-23T10:43:42.998576+0000 (oldest deadline 2024-04-23T10:44:05.298692+0000) 2024-04-23T10:44:30.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:30 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:30.551+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:49.900586+0000 front 2024-04-23T10:43:49.900497+0000 (oldest deadline 2024-04-23T10:44:15.200545+0000) 2024-04-23T10:44:30.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:30 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:30.551+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:59.802141+0000 front 2024-04-23T10:43:59.802168+0000 (oldest deadline 2024-04-23T10:44:23.302165+0000) 2024-04-23T10:44:31.091 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:30 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:30.740+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.311585+0000 front 2024-04-23T10:43:36.311514+0000 (oldest deadline 2024-04-23T10:43:59.211672+0000) 2024-04-23T10:44:31.091 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:30 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:30.740+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:48.314772+0000 front 2024-04-23T10:43:48.314791+0000 (oldest deadline 2024-04-23T10:44:14.214936+0000) 2024-04-23T10:44:31.091 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:30 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:30.740+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:58.316108+0000 front 2024-04-23T10:43:58.315933+0000 (oldest deadline 2024-04-23T10:44:24.215991+0000) 2024-04-23T10:44:31.092 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:30 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:30.740+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:04.216587+0000 front 2024-04-23T10:44:04.216528+0000 (oldest deadline 2024-04-23T10:44:30.116473+0000) 2024-04-23T10:44:31.400 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:30 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7[50726]: 2024-04-23T10:44:30.902+0000 7f5dbd523700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:41.728600+0000 front 2024-04-23T10:43:41.728546+0000 (oldest deadline 2024-04-23T10:44:05.228700+0000) 2024-04-23T10:44:31.400 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:30 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7[50726]: 2024-04-23T10:44:30.902+0000 7f5dbd523700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:52.130820+0000 front 2024-04-23T10:43:52.130687+0000 (oldest deadline 2024-04-23T10:44:16.230820+0000) 2024-04-23T10:44:31.400 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:30 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7[50726]: 2024-04-23T10:44:30.902+0000 7f5dbd523700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:44:00.331949+0000 front 2024-04-23T10:44:00.332016+0000 (oldest deadline 2024-04-23T10:44:21.431895+0000) 2024-04-23T10:44:31.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:31 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:31.040+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.211952+0000 front 2024-04-23T10:43:33.211903+0000 (oldest deadline 2024-04-23T10:43:59.012678+0000) 2024-04-23T10:44:31.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:31 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:31.040+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:43.515012+0000 front 2024-04-23T10:43:43.515041+0000 (oldest deadline 2024-04-23T10:44:05.815169+0000) 2024-04-23T10:44:31.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:31 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:31.040+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:51.716168+0000 front 2024-04-23T10:43:51.716308+0000 (oldest deadline 2024-04-23T10:44:17.016216+0000) 2024-04-23T10:44:31.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:31 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:31.040+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:57.517193+0000 front 2024-04-23T10:43:57.517281+0000 (oldest deadline 2024-04-23T10:44:22.817359+0000) 2024-04-23T10:44:31.731 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:31 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:31.367+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.335834+0000 front 2024-04-23T10:43:36.335867+0000 (oldest deadline 2024-04-23T10:43:57.435822+0000) 2024-04-23T10:44:31.731 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:31 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:31.367+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.637810+0000 front 2024-04-23T10:43:42.637720+0000 (oldest deadline 2024-04-23T10:44:07.937634+0000) 2024-04-23T10:44:31.731 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:31 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:31.367+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:52.538910+0000 front 2024-04-23T10:43:52.538984+0000 (oldest deadline 2024-04-23T10:44:14.238911+0000) 2024-04-23T10:44:31.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:31 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:31.592+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.896652+0000 front 2024-04-23T10:43:34.896698+0000 (oldest deadline 2024-04-23T10:43:58.396793+0000) 2024-04-23T10:44:31.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:31 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:31.592+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.998590+0000 front 2024-04-23T10:43:42.998576+0000 (oldest deadline 2024-04-23T10:44:05.298692+0000) 2024-04-23T10:44:31.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:31 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:31.592+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:49.900586+0000 front 2024-04-23T10:43:49.900497+0000 (oldest deadline 2024-04-23T10:44:15.200545+0000) 2024-04-23T10:44:31.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:31 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:31.592+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:59.802141+0000 front 2024-04-23T10:43:59.802168+0000 (oldest deadline 2024-04-23T10:44:23.302165+0000) 2024-04-23T10:44:32.091 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:31 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:31.732+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.311585+0000 front 2024-04-23T10:43:36.311514+0000 (oldest deadline 2024-04-23T10:43:59.211672+0000) 2024-04-23T10:44:32.092 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:31 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:31.732+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:48.314772+0000 front 2024-04-23T10:43:48.314791+0000 (oldest deadline 2024-04-23T10:44:14.214936+0000) 2024-04-23T10:44:32.092 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:31 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:31.732+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:58.316108+0000 front 2024-04-23T10:43:58.315933+0000 (oldest deadline 2024-04-23T10:44:24.215991+0000) 2024-04-23T10:44:32.092 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:31 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:31.732+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:04.216587+0000 front 2024-04-23T10:44:04.216528+0000 (oldest deadline 2024-04-23T10:44:30.116473+0000) 2024-04-23T10:44:32.400 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:31 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7[50726]: 2024-04-23T10:44:31.913+0000 7f5dbd523700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:41.728600+0000 front 2024-04-23T10:43:41.728546+0000 (oldest deadline 2024-04-23T10:44:05.228700+0000) 2024-04-23T10:44:32.400 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:31 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7[50726]: 2024-04-23T10:44:31.913+0000 7f5dbd523700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:52.130820+0000 front 2024-04-23T10:43:52.130687+0000 (oldest deadline 2024-04-23T10:44:16.230820+0000) 2024-04-23T10:44:32.400 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:31 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7[50726]: 2024-04-23T10:44:31.913+0000 7f5dbd523700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:44:00.331949+0000 front 2024-04-23T10:44:00.332016+0000 (oldest deadline 2024-04-23T10:44:21.431895+0000) 2024-04-23T10:44:32.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:32 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:32.063+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.211952+0000 front 2024-04-23T10:43:33.211903+0000 (oldest deadline 2024-04-23T10:43:59.012678+0000) 2024-04-23T10:44:32.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:32 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:32.063+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:43.515012+0000 front 2024-04-23T10:43:43.515041+0000 (oldest deadline 2024-04-23T10:44:05.815169+0000) 2024-04-23T10:44:32.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:32 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:32.063+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:51.716168+0000 front 2024-04-23T10:43:51.716308+0000 (oldest deadline 2024-04-23T10:44:17.016216+0000) 2024-04-23T10:44:32.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:32 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:32.063+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:57.517193+0000 front 2024-04-23T10:43:57.517281+0000 (oldest deadline 2024-04-23T10:44:22.817359+0000) 2024-04-23T10:44:32.591 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:32 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:32.332+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.335834+0000 front 2024-04-23T10:43:36.335867+0000 (oldest deadline 2024-04-23T10:43:57.435822+0000) 2024-04-23T10:44:32.591 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:32 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:32.332+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.637810+0000 front 2024-04-23T10:43:42.637720+0000 (oldest deadline 2024-04-23T10:44:07.937634+0000) 2024-04-23T10:44:32.591 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:32 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:32.332+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:52.538910+0000 front 2024-04-23T10:43:52.538984+0000 (oldest deadline 2024-04-23T10:44:14.238911+0000) 2024-04-23T10:44:32.591 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:32 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:32.332+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:08.241835+0000 front 2024-04-23T10:44:08.241852+0000 (oldest deadline 2024-04-23T10:44:31.741843+0000) 2024-04-23T10:44:32.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:32 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:32.561+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.896652+0000 front 2024-04-23T10:43:34.896698+0000 (oldest deadline 2024-04-23T10:43:58.396793+0000) 2024-04-23T10:44:32.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:32 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:32.561+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.998590+0000 front 2024-04-23T10:43:42.998576+0000 (oldest deadline 2024-04-23T10:44:05.298692+0000) 2024-04-23T10:44:32.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:32 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:32.561+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:49.900586+0000 front 2024-04-23T10:43:49.900497+0000 (oldest deadline 2024-04-23T10:44:15.200545+0000) 2024-04-23T10:44:32.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:32 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:32.561+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:59.802141+0000 front 2024-04-23T10:43:59.802168+0000 (oldest deadline 2024-04-23T10:44:23.302165+0000) 2024-04-23T10:44:33.091 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:32 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:32.711+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.311585+0000 front 2024-04-23T10:43:36.311514+0000 (oldest deadline 2024-04-23T10:43:59.211672+0000) 2024-04-23T10:44:33.091 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:32 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:32.711+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:48.314772+0000 front 2024-04-23T10:43:48.314791+0000 (oldest deadline 2024-04-23T10:44:14.214936+0000) 2024-04-23T10:44:33.091 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:32 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:32.711+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:58.316108+0000 front 2024-04-23T10:43:58.315933+0000 (oldest deadline 2024-04-23T10:44:24.215991+0000) 2024-04-23T10:44:33.092 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:32 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:32.711+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:04.216587+0000 front 2024-04-23T10:44:04.216528+0000 (oldest deadline 2024-04-23T10:44:30.116473+0000) 2024-04-23T10:44:33.400 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:32 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7[50726]: 2024-04-23T10:44:32.920+0000 7f5dbd523700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:41.728600+0000 front 2024-04-23T10:43:41.728546+0000 (oldest deadline 2024-04-23T10:44:05.228700+0000) 2024-04-23T10:44:33.400 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:32 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7[50726]: 2024-04-23T10:44:32.920+0000 7f5dbd523700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:52.130820+0000 front 2024-04-23T10:43:52.130687+0000 (oldest deadline 2024-04-23T10:44:16.230820+0000) 2024-04-23T10:44:33.400 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:32 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7[50726]: 2024-04-23T10:44:32.920+0000 7f5dbd523700 -1 osd.7 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:44:00.331949+0000 front 2024-04-23T10:44:00.332016+0000 (oldest deadline 2024-04-23T10:44:21.431895+0000) 2024-04-23T10:44:33.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:33 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:33.033+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.211952+0000 front 2024-04-23T10:43:33.211903+0000 (oldest deadline 2024-04-23T10:43:59.012678+0000) 2024-04-23T10:44:33.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:33 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:33.033+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:43.515012+0000 front 2024-04-23T10:43:43.515041+0000 (oldest deadline 2024-04-23T10:44:05.815169+0000) 2024-04-23T10:44:33.401 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:33 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:33.033+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:51.716168+0000 front 2024-04-23T10:43:51.716308+0000 (oldest deadline 2024-04-23T10:44:17.016216+0000) 2024-04-23T10:44:33.401 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:33 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:33.033+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:57.517193+0000 front 2024-04-23T10:43:57.517281+0000 (oldest deadline 2024-04-23T10:44:22.817359+0000) 2024-04-23T10:44:33.401 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:33 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:33.033+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:06.918122+0000 front 2024-04-23T10:44:06.918039+0000 (oldest deadline 2024-04-23T10:44:32.218202+0000) 2024-04-23T10:44:33.591 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:33 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:33.289+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.335834+0000 front 2024-04-23T10:43:36.335867+0000 (oldest deadline 2024-04-23T10:43:57.435822+0000) 2024-04-23T10:44:33.591 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:33 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:33.289+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.637810+0000 front 2024-04-23T10:43:42.637720+0000 (oldest deadline 2024-04-23T10:44:07.937634+0000) 2024-04-23T10:44:33.591 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:33 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:33.289+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:52.538910+0000 front 2024-04-23T10:43:52.538984+0000 (oldest deadline 2024-04-23T10:44:14.238911+0000) 2024-04-23T10:44:33.591 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:33 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:33.289+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:08.241835+0000 front 2024-04-23T10:44:08.241852+0000 (oldest deadline 2024-04-23T10:44:31.741843+0000) 2024-04-23T10:44:33.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:33 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:33.588+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.896652+0000 front 2024-04-23T10:43:34.896698+0000 (oldest deadline 2024-04-23T10:43:58.396793+0000) 2024-04-23T10:44:33.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:33 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:33.588+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.998590+0000 front 2024-04-23T10:43:42.998576+0000 (oldest deadline 2024-04-23T10:44:05.298692+0000) 2024-04-23T10:44:33.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:33 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:33.588+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:49.900586+0000 front 2024-04-23T10:43:49.900497+0000 (oldest deadline 2024-04-23T10:44:15.200545+0000) 2024-04-23T10:44:33.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:33 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:33.588+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:59.802141+0000 front 2024-04-23T10:43:59.802168+0000 (oldest deadline 2024-04-23T10:44:23.302165+0000) 2024-04-23T10:44:33.900 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:33 smithi183 podman[70200]: 2024-04-23 10:44:33.650154122 +0000 UTC m=+5.628602246 container died b8fd49bcf7cb6c7f3e29663c7432ff0c465b42c7e824d836f1f9fe6a54c8c2b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T10:44:34.091 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:33 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:33.734+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.311585+0000 front 2024-04-23T10:43:36.311514+0000 (oldest deadline 2024-04-23T10:43:59.211672+0000) 2024-04-23T10:44:34.091 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:33 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:33.734+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:48.314772+0000 front 2024-04-23T10:43:48.314791+0000 (oldest deadline 2024-04-23T10:44:14.214936+0000) 2024-04-23T10:44:34.092 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:33 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:33.734+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:58.316108+0000 front 2024-04-23T10:43:58.315933+0000 (oldest deadline 2024-04-23T10:44:24.215991+0000) 2024-04-23T10:44:34.092 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:33 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:33.734+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:04.216587+0000 front 2024-04-23T10:44:04.216528+0000 (oldest deadline 2024-04-23T10:44:30.116473+0000) 2024-04-23T10:44:34.304 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:34 smithi183 podman[70200]: 2024-04-23 10:44:34.046437459 +0000 UTC m=+6.024885633 container cleanup b8fd49bcf7cb6c7f3e29663c7432ff0c465b42c7e824d836f1f9fe6a54c8c2b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T10:44:34.304 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:34 smithi183 bash[70200]: ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7 2024-04-23T10:44:34.304 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:34 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:34.072+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.211952+0000 front 2024-04-23T10:43:33.211903+0000 (oldest deadline 2024-04-23T10:43:59.012678+0000) 2024-04-23T10:44:34.304 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:34 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:34.072+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:43.515012+0000 front 2024-04-23T10:43:43.515041+0000 (oldest deadline 2024-04-23T10:44:05.815169+0000) 2024-04-23T10:44:34.304 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:34 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:34.072+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:51.716168+0000 front 2024-04-23T10:43:51.716308+0000 (oldest deadline 2024-04-23T10:44:17.016216+0000) 2024-04-23T10:44:34.304 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:34 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:34.073+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:57.517193+0000 front 2024-04-23T10:43:57.517281+0000 (oldest deadline 2024-04-23T10:44:22.817359+0000) 2024-04-23T10:44:34.305 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:34 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:34.073+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:06.918122+0000 front 2024-04-23T10:44:06.918039+0000 (oldest deadline 2024-04-23T10:44:32.218202+0000) 2024-04-23T10:44:34.591 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:34 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:34.312+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.335834+0000 front 2024-04-23T10:43:36.335867+0000 (oldest deadline 2024-04-23T10:43:57.435822+0000) 2024-04-23T10:44:34.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:34 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:34.312+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.637810+0000 front 2024-04-23T10:43:42.637720+0000 (oldest deadline 2024-04-23T10:44:07.937634+0000) 2024-04-23T10:44:34.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:34 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:34.312+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:52.538910+0000 front 2024-04-23T10:43:52.538984+0000 (oldest deadline 2024-04-23T10:44:14.238911+0000) 2024-04-23T10:44:34.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:34 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:34.312+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:08.241835+0000 front 2024-04-23T10:44:08.241852+0000 (oldest deadline 2024-04-23T10:44:31.741843+0000) 2024-04-23T10:44:34.649 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:34 smithi183 podman[70227]: 2024-04-23 10:44:34.306157285 +0000 UTC m=+0.654357594 container remove b8fd49bcf7cb6c7f3e29663c7432ff0c465b42c7e824d836f1f9fe6a54c8c2b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T10:44:34.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:34 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:34.581+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.896652+0000 front 2024-04-23T10:43:34.896698+0000 (oldest deadline 2024-04-23T10:43:58.396793+0000) 2024-04-23T10:44:34.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:34 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:34.581+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.998590+0000 front 2024-04-23T10:43:42.998576+0000 (oldest deadline 2024-04-23T10:44:05.298692+0000) 2024-04-23T10:44:34.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:34 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:34.581+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:49.900586+0000 front 2024-04-23T10:43:49.900497+0000 (oldest deadline 2024-04-23T10:44:15.200545+0000) 2024-04-23T10:44:34.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:34 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:34.581+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:59.802141+0000 front 2024-04-23T10:43:59.802168+0000 (oldest deadline 2024-04-23T10:44:23.302165+0000) 2024-04-23T10:44:34.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:34 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:34.581+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:06.203107+0000 front 2024-04-23T10:44:06.203078+0000 (oldest deadline 2024-04-23T10:44:34.403475+0000) 2024-04-23T10:44:34.941 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:34 smithi183 podman[70320]: 2024-04-23 10:44:34.571186113 +0000 UTC m=+0.018043992 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T10:44:34.941 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:34 smithi183 podman[70320]: 2024-04-23 10:44:34.690803511 +0000 UTC m=+0.137661431 container create 7cb064b3f08c9ae46054cb5041228fd0310fb71d24cf656ab79614916e30081f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_CLEAN=True) 2024-04-23T10:44:34.941 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:34 smithi183 podman[70320]: 2024-04-23 10:44:34.943162762 +0000 UTC m=+0.390020641 container init 7cb064b3f08c9ae46054cb5041228fd0310fb71d24cf656ab79614916e30081f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240416) 2024-04-23T10:44:35.091 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:34 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:34.721+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.311585+0000 front 2024-04-23T10:43:36.311514+0000 (oldest deadline 2024-04-23T10:43:59.211672+0000) 2024-04-23T10:44:35.091 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:34 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:34.721+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:48.314772+0000 front 2024-04-23T10:43:48.314791+0000 (oldest deadline 2024-04-23T10:44:14.214936+0000) 2024-04-23T10:44:35.092 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:34 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:34.721+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:58.316108+0000 front 2024-04-23T10:43:58.315933+0000 (oldest deadline 2024-04-23T10:44:24.215991+0000) 2024-04-23T10:44:35.092 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:34 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:34.721+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:04.216587+0000 front 2024-04-23T10:44:04.216528+0000 (oldest deadline 2024-04-23T10:44:30.116473+0000) 2024-04-23T10:44:35.237 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:34 smithi183 podman[70320]: 2024-04-23 10:44:34.946557161 +0000 UTC m=+0.393415043 container start 7cb064b3f08c9ae46054cb5041228fd0310fb71d24cf656ab79614916e30081f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T10:44:35.237 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:35 smithi183 podman[70320]: 2024-04-23 10:44:35.035008858 +0000 UTC m=+0.481866770 container attach 7cb064b3f08c9ae46054cb5041228fd0310fb71d24cf656ab79614916e30081f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T10:44:35.237 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:35 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:35.046+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.211952+0000 front 2024-04-23T10:43:33.211903+0000 (oldest deadline 2024-04-23T10:43:59.012678+0000) 2024-04-23T10:44:35.237 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:35 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:35.046+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:43.515012+0000 front 2024-04-23T10:43:43.515041+0000 (oldest deadline 2024-04-23T10:44:05.815169+0000) 2024-04-23T10:44:35.237 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:35 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:35.046+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:51.716168+0000 front 2024-04-23T10:43:51.716308+0000 (oldest deadline 2024-04-23T10:44:17.016216+0000) 2024-04-23T10:44:35.237 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:35 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:35.046+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:57.517193+0000 front 2024-04-23T10:43:57.517281+0000 (oldest deadline 2024-04-23T10:44:22.817359+0000) 2024-04-23T10:44:35.237 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:35 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:35.046+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:06.918122+0000 front 2024-04-23T10:44:06.918039+0000 (oldest deadline 2024-04-23T10:44:32.218202+0000) 2024-04-23T10:44:35.603 INFO:journalctl@ceph.osd.7.smithi183.stdout:Apr 23 10:44:35 smithi183 podman[70320]: 2024-04-23 10:44:35.239213955 +0000 UTC m=+0.686071835 container died 7cb064b3f08c9ae46054cb5041228fd0310fb71d24cf656ab79614916e30081f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-7-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T10:44:35.738 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:35 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:35.345+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.335834+0000 front 2024-04-23T10:43:36.335867+0000 (oldest deadline 2024-04-23T10:43:57.435822+0000) 2024-04-23T10:44:35.739 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:35 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:35.345+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.637810+0000 front 2024-04-23T10:43:42.637720+0000 (oldest deadline 2024-04-23T10:44:07.937634+0000) 2024-04-23T10:44:35.739 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:35 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:35.345+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:52.538910+0000 front 2024-04-23T10:43:52.538984+0000 (oldest deadline 2024-04-23T10:44:14.238911+0000) 2024-04-23T10:44:35.739 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:35 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:35.345+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:08.241835+0000 front 2024-04-23T10:44:08.241852+0000 (oldest deadline 2024-04-23T10:44:31.741843+0000) 2024-04-23T10:44:35.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:35 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:35.570+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.896652+0000 front 2024-04-23T10:43:34.896698+0000 (oldest deadline 2024-04-23T10:43:58.396793+0000) 2024-04-23T10:44:35.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:35 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:35.570+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.998590+0000 front 2024-04-23T10:43:42.998576+0000 (oldest deadline 2024-04-23T10:44:05.298692+0000) 2024-04-23T10:44:35.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:35 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:35.570+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:49.900586+0000 front 2024-04-23T10:43:49.900497+0000 (oldest deadline 2024-04-23T10:44:15.200545+0000) 2024-04-23T10:44:35.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:35 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:35.570+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:59.802141+0000 front 2024-04-23T10:43:59.802168+0000 (oldest deadline 2024-04-23T10:44:23.302165+0000) 2024-04-23T10:44:35.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:35 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:35.570+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:06.203107+0000 front 2024-04-23T10:44:06.203078+0000 (oldest deadline 2024-04-23T10:44:34.403475+0000) 2024-04-23T10:44:35.918 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b22af88-0159-11ef-bc93-c7b262605968@osd.7.service' 2024-04-23T10:44:35.955 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T10:44:35.955 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-23T10:44:35.955 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-23T10:44:35.955 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-9b22af88-0159-11ef-bc93-c7b262605968@osd.8 2024-04-23T10:44:35.998 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:35 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:35.740+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.311585+0000 front 2024-04-23T10:43:36.311514+0000 (oldest deadline 2024-04-23T10:43:59.211672+0000) 2024-04-23T10:44:35.999 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:35 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:35.740+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:48.314772+0000 front 2024-04-23T10:43:48.314791+0000 (oldest deadline 2024-04-23T10:44:14.214936+0000) 2024-04-23T10:44:35.999 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:35 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:35.740+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:58.316108+0000 front 2024-04-23T10:43:58.315933+0000 (oldest deadline 2024-04-23T10:44:24.215991+0000) 2024-04-23T10:44:35.999 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:35 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:35.740+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:04.216587+0000 front 2024-04-23T10:44:04.216528+0000 (oldest deadline 2024-04-23T10:44:30.116473+0000) 2024-04-23T10:44:36.150 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:36 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:36.066+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.211952+0000 front 2024-04-23T10:43:33.211903+0000 (oldest deadline 2024-04-23T10:43:59.012678+0000) 2024-04-23T10:44:36.150 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:36 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:36.066+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:43.515012+0000 front 2024-04-23T10:43:43.515041+0000 (oldest deadline 2024-04-23T10:44:05.815169+0000) 2024-04-23T10:44:36.150 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:36 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:36.066+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:51.716168+0000 front 2024-04-23T10:43:51.716308+0000 (oldest deadline 2024-04-23T10:44:17.016216+0000) 2024-04-23T10:44:36.150 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:36 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:36.066+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:57.517193+0000 front 2024-04-23T10:43:57.517281+0000 (oldest deadline 2024-04-23T10:44:22.817359+0000) 2024-04-23T10:44:36.150 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:36 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:36.066+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:06.918122+0000 front 2024-04-23T10:44:06.918039+0000 (oldest deadline 2024-04-23T10:44:32.218202+0000) 2024-04-23T10:44:36.275 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:36 smithi196 systemd[1]: Stopping Ceph osd.8 for 9b22af88-0159-11ef-bc93-c7b262605968... 2024-04-23T10:44:36.591 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:36 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:36.276+0000 7f31a174e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T10:44:36.591 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:36 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:36.276+0000 7f31a174e700 -1 osd.8 114 *** Got signal Terminated *** 2024-04-23T10:44:36.591 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:36 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:36.276+0000 7f31a174e700 -1 osd.8 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T10:44:36.591 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:36 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:36.365+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.335834+0000 front 2024-04-23T10:43:36.335867+0000 (oldest deadline 2024-04-23T10:43:57.435822+0000) 2024-04-23T10:44:36.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:36 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:36.365+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.637810+0000 front 2024-04-23T10:43:42.637720+0000 (oldest deadline 2024-04-23T10:44:07.937634+0000) 2024-04-23T10:44:36.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:36 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:36.365+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:52.538910+0000 front 2024-04-23T10:43:52.538984+0000 (oldest deadline 2024-04-23T10:44:14.238911+0000) 2024-04-23T10:44:36.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:36 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:36.365+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:08.241835+0000 front 2024-04-23T10:44:08.241852+0000 (oldest deadline 2024-04-23T10:44:31.741843+0000) 2024-04-23T10:44:36.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:36 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:36.520+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.896652+0000 front 2024-04-23T10:43:34.896698+0000 (oldest deadline 2024-04-23T10:43:58.396793+0000) 2024-04-23T10:44:36.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:36 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:36.520+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.998590+0000 front 2024-04-23T10:43:42.998576+0000 (oldest deadline 2024-04-23T10:44:05.298692+0000) 2024-04-23T10:44:36.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:36 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:36.520+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:49.900586+0000 front 2024-04-23T10:43:49.900497+0000 (oldest deadline 2024-04-23T10:44:15.200545+0000) 2024-04-23T10:44:36.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:36 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:36.520+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:59.802141+0000 front 2024-04-23T10:43:59.802168+0000 (oldest deadline 2024-04-23T10:44:23.302165+0000) 2024-04-23T10:44:36.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:36 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:36.520+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:06.203107+0000 front 2024-04-23T10:44:06.203078+0000 (oldest deadline 2024-04-23T10:44:34.403475+0000) 2024-04-23T10:44:37.091 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:36 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:36.762+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.311585+0000 front 2024-04-23T10:43:36.311514+0000 (oldest deadline 2024-04-23T10:43:59.211672+0000) 2024-04-23T10:44:37.091 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:36 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:36.762+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:48.314772+0000 front 2024-04-23T10:43:48.314791+0000 (oldest deadline 2024-04-23T10:44:14.214936+0000) 2024-04-23T10:44:37.092 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:36 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:36.762+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:58.316108+0000 front 2024-04-23T10:43:58.315933+0000 (oldest deadline 2024-04-23T10:44:24.215991+0000) 2024-04-23T10:44:37.092 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:36 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:36.762+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:04.216587+0000 front 2024-04-23T10:44:04.216528+0000 (oldest deadline 2024-04-23T10:44:30.116473+0000) 2024-04-23T10:44:37.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:37 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:37.025+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.211952+0000 front 2024-04-23T10:43:33.211903+0000 (oldest deadline 2024-04-23T10:43:59.012678+0000) 2024-04-23T10:44:37.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:37 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:37.025+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:43.515012+0000 front 2024-04-23T10:43:43.515041+0000 (oldest deadline 2024-04-23T10:44:05.815169+0000) 2024-04-23T10:44:37.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:37 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:37.025+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:51.716168+0000 front 2024-04-23T10:43:51.716308+0000 (oldest deadline 2024-04-23T10:44:17.016216+0000) 2024-04-23T10:44:37.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:37 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:37.025+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:57.517193+0000 front 2024-04-23T10:43:57.517281+0000 (oldest deadline 2024-04-23T10:44:22.817359+0000) 2024-04-23T10:44:37.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:37 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:37.025+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:06.918122+0000 front 2024-04-23T10:44:06.918039+0000 (oldest deadline 2024-04-23T10:44:32.218202+0000) 2024-04-23T10:44:37.718 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:37 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:37.406+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.335834+0000 front 2024-04-23T10:43:36.335867+0000 (oldest deadline 2024-04-23T10:43:57.435822+0000) 2024-04-23T10:44:37.718 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:37 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:37.406+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.637810+0000 front 2024-04-23T10:43:42.637720+0000 (oldest deadline 2024-04-23T10:44:07.937634+0000) 2024-04-23T10:44:37.718 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:37 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:37.406+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:52.538910+0000 front 2024-04-23T10:43:52.538984+0000 (oldest deadline 2024-04-23T10:44:14.238911+0000) 2024-04-23T10:44:37.718 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:37 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:37.406+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:08.241835+0000 front 2024-04-23T10:44:08.241852+0000 (oldest deadline 2024-04-23T10:44:31.741843+0000) 2024-04-23T10:44:37.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:37 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:37.507+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.896652+0000 front 2024-04-23T10:43:34.896698+0000 (oldest deadline 2024-04-23T10:43:58.396793+0000) 2024-04-23T10:44:37.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:37 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:37.507+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.998590+0000 front 2024-04-23T10:43:42.998576+0000 (oldest deadline 2024-04-23T10:44:05.298692+0000) 2024-04-23T10:44:37.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:37 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:37.507+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:49.900586+0000 front 2024-04-23T10:43:49.900497+0000 (oldest deadline 2024-04-23T10:44:15.200545+0000) 2024-04-23T10:44:37.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:37 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:37.507+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:59.802141+0000 front 2024-04-23T10:43:59.802168+0000 (oldest deadline 2024-04-23T10:44:23.302165+0000) 2024-04-23T10:44:37.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:37 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:37.507+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:06.203107+0000 front 2024-04-23T10:44:06.203078+0000 (oldest deadline 2024-04-23T10:44:34.403475+0000) 2024-04-23T10:44:38.092 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:37 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:37.719+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.311585+0000 front 2024-04-23T10:43:36.311514+0000 (oldest deadline 2024-04-23T10:43:59.211672+0000) 2024-04-23T10:44:38.092 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:37 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:37.719+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:48.314772+0000 front 2024-04-23T10:43:48.314791+0000 (oldest deadline 2024-04-23T10:44:14.214936+0000) 2024-04-23T10:44:38.092 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:37 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:37.719+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:58.316108+0000 front 2024-04-23T10:43:58.315933+0000 (oldest deadline 2024-04-23T10:44:24.215991+0000) 2024-04-23T10:44:38.092 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:37 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:37.719+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:04.216587+0000 front 2024-04-23T10:44:04.216528+0000 (oldest deadline 2024-04-23T10:44:30.116473+0000) 2024-04-23T10:44:38.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:38 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:38.002+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.211952+0000 front 2024-04-23T10:43:33.211903+0000 (oldest deadline 2024-04-23T10:43:59.012678+0000) 2024-04-23T10:44:38.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:38 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:38.002+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:43.515012+0000 front 2024-04-23T10:43:43.515041+0000 (oldest deadline 2024-04-23T10:44:05.815169+0000) 2024-04-23T10:44:38.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:38 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:38.002+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:51.716168+0000 front 2024-04-23T10:43:51.716308+0000 (oldest deadline 2024-04-23T10:44:17.016216+0000) 2024-04-23T10:44:38.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:38 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:38.002+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:57.517193+0000 front 2024-04-23T10:43:57.517281+0000 (oldest deadline 2024-04-23T10:44:22.817359+0000) 2024-04-23T10:44:38.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:38 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:38.002+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:06.918122+0000 front 2024-04-23T10:44:06.918039+0000 (oldest deadline 2024-04-23T10:44:32.218202+0000) 2024-04-23T10:44:38.735 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:38 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:38.377+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.335834+0000 front 2024-04-23T10:43:36.335867+0000 (oldest deadline 2024-04-23T10:43:57.435822+0000) 2024-04-23T10:44:38.735 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:38 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:38.377+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.637810+0000 front 2024-04-23T10:43:42.637720+0000 (oldest deadline 2024-04-23T10:44:07.937634+0000) 2024-04-23T10:44:38.735 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:38 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:38.377+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:52.538910+0000 front 2024-04-23T10:43:52.538984+0000 (oldest deadline 2024-04-23T10:44:14.238911+0000) 2024-04-23T10:44:38.735 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:38 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:38.377+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:08.241835+0000 front 2024-04-23T10:44:08.241852+0000 (oldest deadline 2024-04-23T10:44:31.741843+0000) 2024-04-23T10:44:38.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:38 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:38.552+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.896652+0000 front 2024-04-23T10:43:34.896698+0000 (oldest deadline 2024-04-23T10:43:58.396793+0000) 2024-04-23T10:44:38.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:38 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:38.552+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.998590+0000 front 2024-04-23T10:43:42.998576+0000 (oldest deadline 2024-04-23T10:44:05.298692+0000) 2024-04-23T10:44:38.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:38 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:38.552+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:49.900586+0000 front 2024-04-23T10:43:49.900497+0000 (oldest deadline 2024-04-23T10:44:15.200545+0000) 2024-04-23T10:44:38.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:38 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:38.552+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:59.802141+0000 front 2024-04-23T10:43:59.802168+0000 (oldest deadline 2024-04-23T10:44:23.302165+0000) 2024-04-23T10:44:38.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:38 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:38.552+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:06.203107+0000 front 2024-04-23T10:44:06.203078+0000 (oldest deadline 2024-04-23T10:44:34.403475+0000) 2024-04-23T10:44:38.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:38 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:38.552+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-23T10:44:14.404453+0000 front 2024-04-23T10:44:14.404350+0000 (oldest deadline 2024-04-23T10:44:38.404808+0000) 2024-04-23T10:44:39.091 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:38 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:38.735+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.311585+0000 front 2024-04-23T10:43:36.311514+0000 (oldest deadline 2024-04-23T10:43:59.211672+0000) 2024-04-23T10:44:39.092 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:38 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:38.735+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:48.314772+0000 front 2024-04-23T10:43:48.314791+0000 (oldest deadline 2024-04-23T10:44:14.214936+0000) 2024-04-23T10:44:39.092 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:38 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:38.735+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:58.316108+0000 front 2024-04-23T10:43:58.315933+0000 (oldest deadline 2024-04-23T10:44:24.215991+0000) 2024-04-23T10:44:39.092 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:38 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:38.735+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:04.216587+0000 front 2024-04-23T10:44:04.216528+0000 (oldest deadline 2024-04-23T10:44:30.116473+0000) 2024-04-23T10:44:39.150 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:38 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:38.953+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.211952+0000 front 2024-04-23T10:43:33.211903+0000 (oldest deadline 2024-04-23T10:43:59.012678+0000) 2024-04-23T10:44:39.150 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:38 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:38.953+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:43.515012+0000 front 2024-04-23T10:43:43.515041+0000 (oldest deadline 2024-04-23T10:44:05.815169+0000) 2024-04-23T10:44:39.150 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:38 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:38.953+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:51.716168+0000 front 2024-04-23T10:43:51.716308+0000 (oldest deadline 2024-04-23T10:44:17.016216+0000) 2024-04-23T10:44:39.150 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:38 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:38.953+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:57.517193+0000 front 2024-04-23T10:43:57.517281+0000 (oldest deadline 2024-04-23T10:44:22.817359+0000) 2024-04-23T10:44:39.150 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:38 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:38.953+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:06.918122+0000 front 2024-04-23T10:44:06.918039+0000 (oldest deadline 2024-04-23T10:44:32.218202+0000) 2024-04-23T10:44:39.591 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:39 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:39.329+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.335834+0000 front 2024-04-23T10:43:36.335867+0000 (oldest deadline 2024-04-23T10:43:57.435822+0000) 2024-04-23T10:44:39.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:39 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:39.329+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.637810+0000 front 2024-04-23T10:43:42.637720+0000 (oldest deadline 2024-04-23T10:44:07.937634+0000) 2024-04-23T10:44:39.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:39 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:39.329+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:52.538910+0000 front 2024-04-23T10:43:52.538984+0000 (oldest deadline 2024-04-23T10:44:14.238911+0000) 2024-04-23T10:44:39.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:39 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:39.329+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:08.241835+0000 front 2024-04-23T10:44:08.241852+0000 (oldest deadline 2024-04-23T10:44:31.741843+0000) 2024-04-23T10:44:39.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:39 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:39.551+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.896652+0000 front 2024-04-23T10:43:34.896698+0000 (oldest deadline 2024-04-23T10:43:58.396793+0000) 2024-04-23T10:44:39.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:39 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:39.551+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.998590+0000 front 2024-04-23T10:43:42.998576+0000 (oldest deadline 2024-04-23T10:44:05.298692+0000) 2024-04-23T10:44:39.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:39 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:39.551+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:49.900586+0000 front 2024-04-23T10:43:49.900497+0000 (oldest deadline 2024-04-23T10:44:15.200545+0000) 2024-04-23T10:44:39.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:39 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:39.551+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:59.802141+0000 front 2024-04-23T10:43:59.802168+0000 (oldest deadline 2024-04-23T10:44:23.302165+0000) 2024-04-23T10:44:39.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:39 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:39.551+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:06.203107+0000 front 2024-04-23T10:44:06.203078+0000 (oldest deadline 2024-04-23T10:44:34.403475+0000) 2024-04-23T10:44:39.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:39 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:39.551+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-23T10:44:14.404453+0000 front 2024-04-23T10:44:14.404350+0000 (oldest deadline 2024-04-23T10:44:38.404808+0000) 2024-04-23T10:44:39.975 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:39 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:39.693+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.311585+0000 front 2024-04-23T10:43:36.311514+0000 (oldest deadline 2024-04-23T10:43:59.211672+0000) 2024-04-23T10:44:39.975 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:39 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:39.693+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:48.314772+0000 front 2024-04-23T10:43:48.314791+0000 (oldest deadline 2024-04-23T10:44:14.214936+0000) 2024-04-23T10:44:39.975 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:39 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:39.693+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:58.316108+0000 front 2024-04-23T10:43:58.315933+0000 (oldest deadline 2024-04-23T10:44:24.215991+0000) 2024-04-23T10:44:39.975 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:39 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:39.693+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:04.216587+0000 front 2024-04-23T10:44:04.216528+0000 (oldest deadline 2024-04-23T10:44:30.116473+0000) 2024-04-23T10:44:39.975 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:39 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:39.693+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-23T10:44:13.517948+0000 front 2024-04-23T10:44:13.517996+0000 (oldest deadline 2024-04-23T10:44:39.418141+0000) 2024-04-23T10:44:40.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:39 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:39.971+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.211952+0000 front 2024-04-23T10:43:33.211903+0000 (oldest deadline 2024-04-23T10:43:59.012678+0000) 2024-04-23T10:44:40.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:39 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:39.971+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:43.515012+0000 front 2024-04-23T10:43:43.515041+0000 (oldest deadline 2024-04-23T10:44:05.815169+0000) 2024-04-23T10:44:40.401 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:39 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:39.971+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:51.716168+0000 front 2024-04-23T10:43:51.716308+0000 (oldest deadline 2024-04-23T10:44:17.016216+0000) 2024-04-23T10:44:40.401 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:39 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:39.971+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:57.517193+0000 front 2024-04-23T10:43:57.517281+0000 (oldest deadline 2024-04-23T10:44:22.817359+0000) 2024-04-23T10:44:40.401 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:39 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:39.971+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:06.918122+0000 front 2024-04-23T10:44:06.918039+0000 (oldest deadline 2024-04-23T10:44:32.218202+0000) 2024-04-23T10:44:40.401 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:39 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:39.971+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-23T10:44:14.420042+0000 front 2024-04-23T10:44:14.419849+0000 (oldest deadline 2024-04-23T10:44:39.719765+0000) 2024-04-23T10:44:40.643 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:40 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:40.359+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.335834+0000 front 2024-04-23T10:43:36.335867+0000 (oldest deadline 2024-04-23T10:43:57.435822+0000) 2024-04-23T10:44:40.643 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:40 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:40.359+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.637810+0000 front 2024-04-23T10:43:42.637720+0000 (oldest deadline 2024-04-23T10:44:07.937634+0000) 2024-04-23T10:44:40.643 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:40 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:40.359+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:52.538910+0000 front 2024-04-23T10:43:52.538984+0000 (oldest deadline 2024-04-23T10:44:14.238911+0000) 2024-04-23T10:44:40.643 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:40 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:40.359+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:08.241835+0000 front 2024-04-23T10:44:08.241852+0000 (oldest deadline 2024-04-23T10:44:31.741843+0000) 2024-04-23T10:44:40.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:40 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:40.577+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.896652+0000 front 2024-04-23T10:43:34.896698+0000 (oldest deadline 2024-04-23T10:43:58.396793+0000) 2024-04-23T10:44:40.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:40 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:40.577+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.998590+0000 front 2024-04-23T10:43:42.998576+0000 (oldest deadline 2024-04-23T10:44:05.298692+0000) 2024-04-23T10:44:40.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:40 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:40.577+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:49.900586+0000 front 2024-04-23T10:43:49.900497+0000 (oldest deadline 2024-04-23T10:44:15.200545+0000) 2024-04-23T10:44:40.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:40 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:40.577+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:59.802141+0000 front 2024-04-23T10:43:59.802168+0000 (oldest deadline 2024-04-23T10:44:23.302165+0000) 2024-04-23T10:44:40.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:40 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:40.577+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:06.203107+0000 front 2024-04-23T10:44:06.203078+0000 (oldest deadline 2024-04-23T10:44:34.403475+0000) 2024-04-23T10:44:40.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:40 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:40.577+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-23T10:44:14.404453+0000 front 2024-04-23T10:44:14.404350+0000 (oldest deadline 2024-04-23T10:44:38.404808+0000) 2024-04-23T10:44:41.092 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:40 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:40.644+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.311585+0000 front 2024-04-23T10:43:36.311514+0000 (oldest deadline 2024-04-23T10:43:59.211672+0000) 2024-04-23T10:44:41.092 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:40 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:40.644+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:48.314772+0000 front 2024-04-23T10:43:48.314791+0000 (oldest deadline 2024-04-23T10:44:14.214936+0000) 2024-04-23T10:44:41.092 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:40 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:40.644+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:58.316108+0000 front 2024-04-23T10:43:58.315933+0000 (oldest deadline 2024-04-23T10:44:24.215991+0000) 2024-04-23T10:44:41.092 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:40 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:40.644+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:04.216587+0000 front 2024-04-23T10:44:04.216528+0000 (oldest deadline 2024-04-23T10:44:30.116473+0000) 2024-04-23T10:44:41.092 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:40 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8[51720]: 2024-04-23T10:44:40.644+0000 7f319dd42700 -1 osd.8 114 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-23T10:44:13.517948+0000 front 2024-04-23T10:44:13.517996+0000 (oldest deadline 2024-04-23T10:44:39.418141+0000) 2024-04-23T10:44:41.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:40 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:40.996+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.211952+0000 front 2024-04-23T10:43:33.211903+0000 (oldest deadline 2024-04-23T10:43:59.012678+0000) 2024-04-23T10:44:41.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:40 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:40.996+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:43.515012+0000 front 2024-04-23T10:43:43.515041+0000 (oldest deadline 2024-04-23T10:44:05.815169+0000) 2024-04-23T10:44:41.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:40 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:40.996+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:51.716168+0000 front 2024-04-23T10:43:51.716308+0000 (oldest deadline 2024-04-23T10:44:17.016216+0000) 2024-04-23T10:44:41.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:40 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:40.996+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:57.517193+0000 front 2024-04-23T10:43:57.517281+0000 (oldest deadline 2024-04-23T10:44:22.817359+0000) 2024-04-23T10:44:41.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:40 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:40.996+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:06.918122+0000 front 2024-04-23T10:44:06.918039+0000 (oldest deadline 2024-04-23T10:44:32.218202+0000) 2024-04-23T10:44:41.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:40 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:40.996+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-23T10:44:14.420042+0000 front 2024-04-23T10:44:14.419849+0000 (oldest deadline 2024-04-23T10:44:39.719765+0000) 2024-04-23T10:44:41.842 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:41 smithi196 podman[71312]: 2024-04-23 10:44:41.598459792 +0000 UTC m=+5.451886154 container died 854ca2fb9aa1ea07225795892e2ccca52ab044b6e41fb1e50066d6cffc95609c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T10:44:41.842 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:41 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:41.392+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.335834+0000 front 2024-04-23T10:43:36.335867+0000 (oldest deadline 2024-04-23T10:43:57.435822+0000) 2024-04-23T10:44:41.842 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:41 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:41.392+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.637810+0000 front 2024-04-23T10:43:42.637720+0000 (oldest deadline 2024-04-23T10:44:07.937634+0000) 2024-04-23T10:44:41.842 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:41 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:41.392+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:52.538910+0000 front 2024-04-23T10:43:52.538984+0000 (oldest deadline 2024-04-23T10:44:14.238911+0000) 2024-04-23T10:44:41.842 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:41 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:41.392+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:08.241835+0000 front 2024-04-23T10:44:08.241852+0000 (oldest deadline 2024-04-23T10:44:31.741843+0000) 2024-04-23T10:44:41.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:41 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:41.551+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.896652+0000 front 2024-04-23T10:43:34.896698+0000 (oldest deadline 2024-04-23T10:43:58.396793+0000) 2024-04-23T10:44:41.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:41 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:41.551+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.998590+0000 front 2024-04-23T10:43:42.998576+0000 (oldest deadline 2024-04-23T10:44:05.298692+0000) 2024-04-23T10:44:41.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:41 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:41.551+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:49.900586+0000 front 2024-04-23T10:43:49.900497+0000 (oldest deadline 2024-04-23T10:44:15.200545+0000) 2024-04-23T10:44:41.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:41 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:41.551+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:59.802141+0000 front 2024-04-23T10:43:59.802168+0000 (oldest deadline 2024-04-23T10:44:23.302165+0000) 2024-04-23T10:44:41.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:41 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:41.551+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:06.203107+0000 front 2024-04-23T10:44:06.203078+0000 (oldest deadline 2024-04-23T10:44:34.403475+0000) 2024-04-23T10:44:41.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:41 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:41.551+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-23T10:44:14.404453+0000 front 2024-04-23T10:44:14.404350+0000 (oldest deadline 2024-04-23T10:44:38.404808+0000) 2024-04-23T10:44:42.259 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:42 smithi196 podman[71312]: 2024-04-23 10:44:42.009527461 +0000 UTC m=+5.862953821 container cleanup 854ca2fb9aa1ea07225795892e2ccca52ab044b6e41fb1e50066d6cffc95609c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-23T10:44:42.259 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:42 smithi196 bash[71312]: ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8 2024-04-23T10:44:42.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:42 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:42.026+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.211952+0000 front 2024-04-23T10:43:33.211903+0000 (oldest deadline 2024-04-23T10:43:59.012678+0000) 2024-04-23T10:44:42.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:42 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:42.026+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:43.515012+0000 front 2024-04-23T10:43:43.515041+0000 (oldest deadline 2024-04-23T10:44:05.815169+0000) 2024-04-23T10:44:42.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:42 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:42.026+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:51.716168+0000 front 2024-04-23T10:43:51.716308+0000 (oldest deadline 2024-04-23T10:44:17.016216+0000) 2024-04-23T10:44:42.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:42 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:42.026+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:57.517193+0000 front 2024-04-23T10:43:57.517281+0000 (oldest deadline 2024-04-23T10:44:22.817359+0000) 2024-04-23T10:44:42.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:42 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:42.026+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:06.918122+0000 front 2024-04-23T10:44:06.918039+0000 (oldest deadline 2024-04-23T10:44:32.218202+0000) 2024-04-23T10:44:42.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:42 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:42.026+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-23T10:44:14.420042+0000 front 2024-04-23T10:44:14.419849+0000 (oldest deadline 2024-04-23T10:44:39.719765+0000) 2024-04-23T10:44:42.560 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:42 smithi196 podman[71330]: 2024-04-23 10:44:42.261058553 +0000 UTC m=+0.659816045 container remove 854ca2fb9aa1ea07225795892e2ccca52ab044b6e41fb1e50066d6cffc95609c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-23T10:44:42.560 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:42 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:42.425+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.335834+0000 front 2024-04-23T10:43:36.335867+0000 (oldest deadline 2024-04-23T10:43:57.435822+0000) 2024-04-23T10:44:42.560 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:42 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:42.425+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.637810+0000 front 2024-04-23T10:43:42.637720+0000 (oldest deadline 2024-04-23T10:44:07.937634+0000) 2024-04-23T10:44:42.560 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:42 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:42.425+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:52.538910+0000 front 2024-04-23T10:43:52.538984+0000 (oldest deadline 2024-04-23T10:44:14.238911+0000) 2024-04-23T10:44:42.560 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:42 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:42.425+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:08.241835+0000 front 2024-04-23T10:44:08.241852+0000 (oldest deadline 2024-04-23T10:44:31.741843+0000) 2024-04-23T10:44:42.841 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:42 smithi196 podman[71421]: 2024-04-23 10:44:42.561803295 +0000 UTC m=+0.113383537 container create dfd6c3bc4eda86cdb185a4c6a4bca995852e564f4aa9f7fc14fd0dd2cb68fd3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-23T10:44:42.842 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:42 smithi196 podman[71421]: 2024-04-23 10:44:42.466509874 +0000 UTC m=+0.018090141 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T10:44:42.842 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:42 smithi196 podman[71421]: 2024-04-23 10:44:42.806089385 +0000 UTC m=+0.357669632 container init dfd6c3bc4eda86cdb185a4c6a4bca995852e564f4aa9f7fc14fd0dd2cb68fd3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-23T10:44:42.842 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:42 smithi196 podman[71421]: 2024-04-23 10:44:42.809089342 +0000 UTC m=+0.360669588 container start dfd6c3bc4eda86cdb185a4c6a4bca995852e564f4aa9f7fc14fd0dd2cb68fd3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8-deactivate, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0) 2024-04-23T10:44:42.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:42 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:42.540+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.896652+0000 front 2024-04-23T10:43:34.896698+0000 (oldest deadline 2024-04-23T10:43:58.396793+0000) 2024-04-23T10:44:42.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:42 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:42.540+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.998590+0000 front 2024-04-23T10:43:42.998576+0000 (oldest deadline 2024-04-23T10:44:05.298692+0000) 2024-04-23T10:44:42.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:42 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:42.540+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:49.900586+0000 front 2024-04-23T10:43:49.900497+0000 (oldest deadline 2024-04-23T10:44:15.200545+0000) 2024-04-23T10:44:42.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:42 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:42.540+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:59.802141+0000 front 2024-04-23T10:43:59.802168+0000 (oldest deadline 2024-04-23T10:44:23.302165+0000) 2024-04-23T10:44:42.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:42 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:42.540+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:06.203107+0000 front 2024-04-23T10:44:06.203078+0000 (oldest deadline 2024-04-23T10:44:34.403475+0000) 2024-04-23T10:44:42.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:42 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:42.540+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-23T10:44:14.404453+0000 front 2024-04-23T10:44:14.404350+0000 (oldest deadline 2024-04-23T10:44:38.404808+0000) 2024-04-23T10:44:43.107 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:42 smithi196 podman[71421]: 2024-04-23 10:44:42.906163893 +0000 UTC m=+0.457744137 container attach dfd6c3bc4eda86cdb185a4c6a4bca995852e564f4aa9f7fc14fd0dd2cb68fd3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T10:44:43.376 INFO:journalctl@ceph.osd.8.smithi196.stdout:Apr 23 10:44:43 smithi196 podman[71421]: 2024-04-23 10:44:43.110495029 +0000 UTC m=+0.662075278 container died dfd6c3bc4eda86cdb185a4c6a4bca995852e564f4aa9f7fc14fd0dd2cb68fd3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-8-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T10:44:43.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:43 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:43.030+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.211952+0000 front 2024-04-23T10:43:33.211903+0000 (oldest deadline 2024-04-23T10:43:59.012678+0000) 2024-04-23T10:44:43.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:43 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:43.030+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:43.515012+0000 front 2024-04-23T10:43:43.515041+0000 (oldest deadline 2024-04-23T10:44:05.815169+0000) 2024-04-23T10:44:43.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:43 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:43.030+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:51.716168+0000 front 2024-04-23T10:43:51.716308+0000 (oldest deadline 2024-04-23T10:44:17.016216+0000) 2024-04-23T10:44:43.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:43 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:43.030+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:57.517193+0000 front 2024-04-23T10:43:57.517281+0000 (oldest deadline 2024-04-23T10:44:22.817359+0000) 2024-04-23T10:44:43.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:43 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:43.030+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:06.918122+0000 front 2024-04-23T10:44:06.918039+0000 (oldest deadline 2024-04-23T10:44:32.218202+0000) 2024-04-23T10:44:43.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:43 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:43.030+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-23T10:44:14.420042+0000 front 2024-04-23T10:44:14.419849+0000 (oldest deadline 2024-04-23T10:44:39.719765+0000) 2024-04-23T10:44:43.758 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:43 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:43.425+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.335834+0000 front 2024-04-23T10:43:36.335867+0000 (oldest deadline 2024-04-23T10:43:57.435822+0000) 2024-04-23T10:44:43.758 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:43 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:43.425+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.637810+0000 front 2024-04-23T10:43:42.637720+0000 (oldest deadline 2024-04-23T10:44:07.937634+0000) 2024-04-23T10:44:43.758 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:43 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:43.425+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:52.538910+0000 front 2024-04-23T10:43:52.538984+0000 (oldest deadline 2024-04-23T10:44:14.238911+0000) 2024-04-23T10:44:43.758 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:43 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:43.425+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:08.241835+0000 front 2024-04-23T10:44:08.241852+0000 (oldest deadline 2024-04-23T10:44:31.741843+0000) 2024-04-23T10:44:43.758 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:43 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:43.425+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-23T10:44:17.042719+0000 front 2024-04-23T10:44:17.042744+0000 (oldest deadline 2024-04-23T10:44:42.942817+0000) 2024-04-23T10:44:43.775 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b22af88-0159-11ef-bc93-c7b262605968@osd.8.service' 2024-04-23T10:44:43.853 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T10:44:43.853 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-23T10:44:43.853 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-23T10:44:43.853 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-9b22af88-0159-11ef-bc93-c7b262605968@osd.9 2024-04-23T10:44:43.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:43 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:43.524+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.896652+0000 front 2024-04-23T10:43:34.896698+0000 (oldest deadline 2024-04-23T10:43:58.396793+0000) 2024-04-23T10:44:43.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:43 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:43.524+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.998590+0000 front 2024-04-23T10:43:42.998576+0000 (oldest deadline 2024-04-23T10:44:05.298692+0000) 2024-04-23T10:44:43.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:43 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:43.524+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:49.900586+0000 front 2024-04-23T10:43:49.900497+0000 (oldest deadline 2024-04-23T10:44:15.200545+0000) 2024-04-23T10:44:43.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:43 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:43.524+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:59.802141+0000 front 2024-04-23T10:43:59.802168+0000 (oldest deadline 2024-04-23T10:44:23.302165+0000) 2024-04-23T10:44:43.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:43 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:43.524+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:06.203107+0000 front 2024-04-23T10:44:06.203078+0000 (oldest deadline 2024-04-23T10:44:34.403475+0000) 2024-04-23T10:44:43.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:43 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:43.524+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-23T10:44:14.404453+0000 front 2024-04-23T10:44:14.404350+0000 (oldest deadline 2024-04-23T10:44:38.404808+0000) 2024-04-23T10:44:44.140 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:43 smithi097 systemd[1]: Stopping Ceph osd.9 for 9b22af88-0159-11ef-bc93-c7b262605968... 2024-04-23T10:44:44.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:44 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:44.043+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.211952+0000 front 2024-04-23T10:43:33.211903+0000 (oldest deadline 2024-04-23T10:43:59.012678+0000) 2024-04-23T10:44:44.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:44 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:44.043+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:43.515012+0000 front 2024-04-23T10:43:43.515041+0000 (oldest deadline 2024-04-23T10:44:05.815169+0000) 2024-04-23T10:44:44.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:44 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:44.043+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:51.716168+0000 front 2024-04-23T10:43:51.716308+0000 (oldest deadline 2024-04-23T10:44:17.016216+0000) 2024-04-23T10:44:44.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:44 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:44.043+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:57.517193+0000 front 2024-04-23T10:43:57.517281+0000 (oldest deadline 2024-04-23T10:44:22.817359+0000) 2024-04-23T10:44:44.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:44 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:44.043+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:06.918122+0000 front 2024-04-23T10:44:06.918039+0000 (oldest deadline 2024-04-23T10:44:32.218202+0000) 2024-04-23T10:44:44.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:44 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:44.043+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-23T10:44:14.420042+0000 front 2024-04-23T10:44:14.419849+0000 (oldest deadline 2024-04-23T10:44:39.719765+0000) 2024-04-23T10:44:44.563 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:44 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:44.140+0000 7f96b4e12700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T10:44:44.563 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:44 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:44.140+0000 7f96b4e12700 -1 osd.9 114 *** Got signal Terminated *** 2024-04-23T10:44:44.564 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:44 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:44.140+0000 7f96b4e12700 -1 osd.9 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T10:44:44.564 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:44 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:44.564+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.896652+0000 front 2024-04-23T10:43:34.896698+0000 (oldest deadline 2024-04-23T10:43:58.396793+0000) 2024-04-23T10:44:44.564 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:44 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:44.564+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.998590+0000 front 2024-04-23T10:43:42.998576+0000 (oldest deadline 2024-04-23T10:44:05.298692+0000) 2024-04-23T10:44:44.564 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:44 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:44.564+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:49.900586+0000 front 2024-04-23T10:43:49.900497+0000 (oldest deadline 2024-04-23T10:44:15.200545+0000) 2024-04-23T10:44:44.564 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:44 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:44.564+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:59.802141+0000 front 2024-04-23T10:43:59.802168+0000 (oldest deadline 2024-04-23T10:44:23.302165+0000) 2024-04-23T10:44:44.564 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:44 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:44.564+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:06.203107+0000 front 2024-04-23T10:44:06.203078+0000 (oldest deadline 2024-04-23T10:44:34.403475+0000) 2024-04-23T10:44:44.564 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:44 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:44.564+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-23T10:44:14.404453+0000 front 2024-04-23T10:44:14.404350+0000 (oldest deadline 2024-04-23T10:44:38.404808+0000) 2024-04-23T10:44:44.591 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:44 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:44.459+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.335834+0000 front 2024-04-23T10:43:36.335867+0000 (oldest deadline 2024-04-23T10:43:57.435822+0000) 2024-04-23T10:44:44.591 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:44 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:44.459+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.637810+0000 front 2024-04-23T10:43:42.637720+0000 (oldest deadline 2024-04-23T10:44:07.937634+0000) 2024-04-23T10:44:44.591 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:44 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:44.459+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:52.538910+0000 front 2024-04-23T10:43:52.538984+0000 (oldest deadline 2024-04-23T10:44:14.238911+0000) 2024-04-23T10:44:44.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:44 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:44.459+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:08.241835+0000 front 2024-04-23T10:44:08.241852+0000 (oldest deadline 2024-04-23T10:44:31.741843+0000) 2024-04-23T10:44:44.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:44 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:44.459+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-23T10:44:17.042719+0000 front 2024-04-23T10:44:17.042744+0000 (oldest deadline 2024-04-23T10:44:42.942817+0000) 2024-04-23T10:44:45.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:45 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:45.035+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.211952+0000 front 2024-04-23T10:43:33.211903+0000 (oldest deadline 2024-04-23T10:43:59.012678+0000) 2024-04-23T10:44:45.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:45 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:45.035+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:43.515012+0000 front 2024-04-23T10:43:43.515041+0000 (oldest deadline 2024-04-23T10:44:05.815169+0000) 2024-04-23T10:44:45.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:45 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:45.035+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:51.716168+0000 front 2024-04-23T10:43:51.716308+0000 (oldest deadline 2024-04-23T10:44:17.016216+0000) 2024-04-23T10:44:45.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:45 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:45.035+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:57.517193+0000 front 2024-04-23T10:43:57.517281+0000 (oldest deadline 2024-04-23T10:44:22.817359+0000) 2024-04-23T10:44:45.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:45 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:45.035+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:06.918122+0000 front 2024-04-23T10:44:06.918039+0000 (oldest deadline 2024-04-23T10:44:32.218202+0000) 2024-04-23T10:44:45.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:45 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:45.035+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-23T10:44:14.420042+0000 front 2024-04-23T10:44:14.419849+0000 (oldest deadline 2024-04-23T10:44:39.719765+0000) 2024-04-23T10:44:45.842 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:45 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:45.440+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.335834+0000 front 2024-04-23T10:43:36.335867+0000 (oldest deadline 2024-04-23T10:43:57.435822+0000) 2024-04-23T10:44:45.842 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:45 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:45.440+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.637810+0000 front 2024-04-23T10:43:42.637720+0000 (oldest deadline 2024-04-23T10:44:07.937634+0000) 2024-04-23T10:44:45.842 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:45 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:45.440+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:52.538910+0000 front 2024-04-23T10:43:52.538984+0000 (oldest deadline 2024-04-23T10:44:14.238911+0000) 2024-04-23T10:44:45.842 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:45 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:45.440+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:08.241835+0000 front 2024-04-23T10:44:08.241852+0000 (oldest deadline 2024-04-23T10:44:31.741843+0000) 2024-04-23T10:44:45.842 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:45 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:45.440+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-23T10:44:17.042719+0000 front 2024-04-23T10:44:17.042744+0000 (oldest deadline 2024-04-23T10:44:42.942817+0000) 2024-04-23T10:44:45.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:45 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:45.541+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.896652+0000 front 2024-04-23T10:43:34.896698+0000 (oldest deadline 2024-04-23T10:43:58.396793+0000) 2024-04-23T10:44:45.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:45 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:45.541+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.998590+0000 front 2024-04-23T10:43:42.998576+0000 (oldest deadline 2024-04-23T10:44:05.298692+0000) 2024-04-23T10:44:45.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:45 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:45.541+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:49.900586+0000 front 2024-04-23T10:43:49.900497+0000 (oldest deadline 2024-04-23T10:44:15.200545+0000) 2024-04-23T10:44:45.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:45 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:45.541+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:59.802141+0000 front 2024-04-23T10:43:59.802168+0000 (oldest deadline 2024-04-23T10:44:23.302165+0000) 2024-04-23T10:44:45.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:45 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:45.541+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:06.203107+0000 front 2024-04-23T10:44:06.203078+0000 (oldest deadline 2024-04-23T10:44:34.403475+0000) 2024-04-23T10:44:45.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:45 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:45.541+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-23T10:44:14.404453+0000 front 2024-04-23T10:44:14.404350+0000 (oldest deadline 2024-04-23T10:44:38.404808+0000) 2024-04-23T10:44:46.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:46 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:46.070+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.211952+0000 front 2024-04-23T10:43:33.211903+0000 (oldest deadline 2024-04-23T10:43:59.012678+0000) 2024-04-23T10:44:46.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:46 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:46.070+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:43.515012+0000 front 2024-04-23T10:43:43.515041+0000 (oldest deadline 2024-04-23T10:44:05.815169+0000) 2024-04-23T10:44:46.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:46 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:46.070+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:51.716168+0000 front 2024-04-23T10:43:51.716308+0000 (oldest deadline 2024-04-23T10:44:17.016216+0000) 2024-04-23T10:44:46.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:46 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:46.070+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:57.517193+0000 front 2024-04-23T10:43:57.517281+0000 (oldest deadline 2024-04-23T10:44:22.817359+0000) 2024-04-23T10:44:46.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:46 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:46.070+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:06.918122+0000 front 2024-04-23T10:44:06.918039+0000 (oldest deadline 2024-04-23T10:44:32.218202+0000) 2024-04-23T10:44:46.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:46 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:46.070+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-23T10:44:14.420042+0000 front 2024-04-23T10:44:14.419849+0000 (oldest deadline 2024-04-23T10:44:39.719765+0000) 2024-04-23T10:44:46.842 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:46 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:46.437+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.335834+0000 front 2024-04-23T10:43:36.335867+0000 (oldest deadline 2024-04-23T10:43:57.435822+0000) 2024-04-23T10:44:46.842 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:46 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:46.437+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.637810+0000 front 2024-04-23T10:43:42.637720+0000 (oldest deadline 2024-04-23T10:44:07.937634+0000) 2024-04-23T10:44:46.842 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:46 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:46.437+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:52.538910+0000 front 2024-04-23T10:43:52.538984+0000 (oldest deadline 2024-04-23T10:44:14.238911+0000) 2024-04-23T10:44:46.842 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:46 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:46.437+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:08.241835+0000 front 2024-04-23T10:44:08.241852+0000 (oldest deadline 2024-04-23T10:44:31.741843+0000) 2024-04-23T10:44:46.842 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:46 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:46.437+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-23T10:44:17.042719+0000 front 2024-04-23T10:44:17.042744+0000 (oldest deadline 2024-04-23T10:44:42.942817+0000) 2024-04-23T10:44:46.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:46 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:46.515+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.896652+0000 front 2024-04-23T10:43:34.896698+0000 (oldest deadline 2024-04-23T10:43:58.396793+0000) 2024-04-23T10:44:46.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:46 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:46.515+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.998590+0000 front 2024-04-23T10:43:42.998576+0000 (oldest deadline 2024-04-23T10:44:05.298692+0000) 2024-04-23T10:44:46.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:46 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:46.515+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:49.900586+0000 front 2024-04-23T10:43:49.900497+0000 (oldest deadline 2024-04-23T10:44:15.200545+0000) 2024-04-23T10:44:46.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:46 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:46.515+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:59.802141+0000 front 2024-04-23T10:43:59.802168+0000 (oldest deadline 2024-04-23T10:44:23.302165+0000) 2024-04-23T10:44:46.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:46 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:46.515+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:06.203107+0000 front 2024-04-23T10:44:06.203078+0000 (oldest deadline 2024-04-23T10:44:34.403475+0000) 2024-04-23T10:44:46.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:46 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:46.515+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-23T10:44:14.404453+0000 front 2024-04-23T10:44:14.404350+0000 (oldest deadline 2024-04-23T10:44:38.404808+0000) 2024-04-23T10:44:47.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:47 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:47.027+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.211952+0000 front 2024-04-23T10:43:33.211903+0000 (oldest deadline 2024-04-23T10:43:59.012678+0000) 2024-04-23T10:44:47.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:47 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:47.027+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:43.515012+0000 front 2024-04-23T10:43:43.515041+0000 (oldest deadline 2024-04-23T10:44:05.815169+0000) 2024-04-23T10:44:47.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:47 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:47.027+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:51.716168+0000 front 2024-04-23T10:43:51.716308+0000 (oldest deadline 2024-04-23T10:44:17.016216+0000) 2024-04-23T10:44:47.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:47 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:47.027+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:57.517193+0000 front 2024-04-23T10:43:57.517281+0000 (oldest deadline 2024-04-23T10:44:22.817359+0000) 2024-04-23T10:44:47.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:47 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:47.027+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:06.918122+0000 front 2024-04-23T10:44:06.918039+0000 (oldest deadline 2024-04-23T10:44:32.218202+0000) 2024-04-23T10:44:47.401 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:47 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:47.027+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-23T10:44:14.420042+0000 front 2024-04-23T10:44:14.419849+0000 (oldest deadline 2024-04-23T10:44:39.719765+0000) 2024-04-23T10:44:47.842 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:47 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:47.404+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.335834+0000 front 2024-04-23T10:43:36.335867+0000 (oldest deadline 2024-04-23T10:43:57.435822+0000) 2024-04-23T10:44:47.842 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:47 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:47.404+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.637810+0000 front 2024-04-23T10:43:42.637720+0000 (oldest deadline 2024-04-23T10:44:07.937634+0000) 2024-04-23T10:44:47.842 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:47 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:47.404+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:52.538910+0000 front 2024-04-23T10:43:52.538984+0000 (oldest deadline 2024-04-23T10:44:14.238911+0000) 2024-04-23T10:44:47.842 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:47 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:47.404+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:08.241835+0000 front 2024-04-23T10:44:08.241852+0000 (oldest deadline 2024-04-23T10:44:31.741843+0000) 2024-04-23T10:44:47.842 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:47 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:47.404+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-23T10:44:17.042719+0000 front 2024-04-23T10:44:17.042744+0000 (oldest deadline 2024-04-23T10:44:42.942817+0000) 2024-04-23T10:44:47.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:47 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:47.556+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.896652+0000 front 2024-04-23T10:43:34.896698+0000 (oldest deadline 2024-04-23T10:43:58.396793+0000) 2024-04-23T10:44:47.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:47 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:47.556+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.998590+0000 front 2024-04-23T10:43:42.998576+0000 (oldest deadline 2024-04-23T10:44:05.298692+0000) 2024-04-23T10:44:47.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:47 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:47.556+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:49.900586+0000 front 2024-04-23T10:43:49.900497+0000 (oldest deadline 2024-04-23T10:44:15.200545+0000) 2024-04-23T10:44:47.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:47 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:47.556+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:59.802141+0000 front 2024-04-23T10:43:59.802168+0000 (oldest deadline 2024-04-23T10:44:23.302165+0000) 2024-04-23T10:44:47.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:47 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:47.556+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:06.203107+0000 front 2024-04-23T10:44:06.203078+0000 (oldest deadline 2024-04-23T10:44:34.403475+0000) 2024-04-23T10:44:47.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:47 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:47.556+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-23T10:44:14.404453+0000 front 2024-04-23T10:44:14.404350+0000 (oldest deadline 2024-04-23T10:44:38.404808+0000) 2024-04-23T10:44:47.861 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:47 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:47.556+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6822 osd.6 since back 2024-04-23T10:44:23.506203+0000 front 2024-04-23T10:44:23.506251+0000 (oldest deadline 2024-04-23T10:44:47.006236+0000) 2024-04-23T10:44:48.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:48 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:48.009+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.211952+0000 front 2024-04-23T10:43:33.211903+0000 (oldest deadline 2024-04-23T10:43:59.012678+0000) 2024-04-23T10:44:48.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:48 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:48.009+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:43.515012+0000 front 2024-04-23T10:43:43.515041+0000 (oldest deadline 2024-04-23T10:44:05.815169+0000) 2024-04-23T10:44:48.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:48 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:48.009+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:51.716168+0000 front 2024-04-23T10:43:51.716308+0000 (oldest deadline 2024-04-23T10:44:17.016216+0000) 2024-04-23T10:44:48.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:48 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:48.009+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:57.517193+0000 front 2024-04-23T10:43:57.517281+0000 (oldest deadline 2024-04-23T10:44:22.817359+0000) 2024-04-23T10:44:48.401 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:48 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:48.009+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:06.918122+0000 front 2024-04-23T10:44:06.918039+0000 (oldest deadline 2024-04-23T10:44:32.218202+0000) 2024-04-23T10:44:48.401 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:48 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:48.009+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-23T10:44:14.420042+0000 front 2024-04-23T10:44:14.419849+0000 (oldest deadline 2024-04-23T10:44:39.719765+0000) 2024-04-23T10:44:48.401 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:48 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:48.009+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6822 osd.6 since back 2024-04-23T10:44:23.721209+0000 front 2024-04-23T10:44:23.721338+0000 (oldest deadline 2024-04-23T10:44:47.821149+0000) 2024-04-23T10:44:48.842 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:48 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:48.407+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.335834+0000 front 2024-04-23T10:43:36.335867+0000 (oldest deadline 2024-04-23T10:43:57.435822+0000) 2024-04-23T10:44:48.842 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:48 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:48.407+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.637810+0000 front 2024-04-23T10:43:42.637720+0000 (oldest deadline 2024-04-23T10:44:07.937634+0000) 2024-04-23T10:44:48.842 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:48 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:48.407+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:52.538910+0000 front 2024-04-23T10:43:52.538984+0000 (oldest deadline 2024-04-23T10:44:14.238911+0000) 2024-04-23T10:44:48.842 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:48 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:48.407+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:08.241835+0000 front 2024-04-23T10:44:08.241852+0000 (oldest deadline 2024-04-23T10:44:31.741843+0000) 2024-04-23T10:44:48.842 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:48 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:48.407+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-23T10:44:17.042719+0000 front 2024-04-23T10:44:17.042744+0000 (oldest deadline 2024-04-23T10:44:42.942817+0000) 2024-04-23T10:44:48.862 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:48 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:48.578+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:34.896652+0000 front 2024-04-23T10:43:34.896698+0000 (oldest deadline 2024-04-23T10:43:58.396793+0000) 2024-04-23T10:44:48.862 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:48 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:48.578+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.998590+0000 front 2024-04-23T10:43:42.998576+0000 (oldest deadline 2024-04-23T10:44:05.298692+0000) 2024-04-23T10:44:48.862 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:48 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:48.578+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:49.900586+0000 front 2024-04-23T10:43:49.900497+0000 (oldest deadline 2024-04-23T10:44:15.200545+0000) 2024-04-23T10:44:48.862 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:48 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:48.579+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:59.802141+0000 front 2024-04-23T10:43:59.802168+0000 (oldest deadline 2024-04-23T10:44:23.302165+0000) 2024-04-23T10:44:48.862 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:48 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:48.579+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:06.203107+0000 front 2024-04-23T10:44:06.203078+0000 (oldest deadline 2024-04-23T10:44:34.403475+0000) 2024-04-23T10:44:48.862 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:48 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:48.579+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-23T10:44:14.404453+0000 front 2024-04-23T10:44:14.404350+0000 (oldest deadline 2024-04-23T10:44:38.404808+0000) 2024-04-23T10:44:48.863 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:48 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9[61276]: 2024-04-23T10:44:48.579+0000 7f96b1406700 -1 osd.9 114 heartbeat_check: no reply from 172.21.15.97:6822 osd.6 since back 2024-04-23T10:44:23.506203+0000 front 2024-04-23T10:44:23.506251+0000 (oldest deadline 2024-04-23T10:44:47.006236+0000) 2024-04-23T10:44:49.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:49 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:49.013+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.211952+0000 front 2024-04-23T10:43:33.211903+0000 (oldest deadline 2024-04-23T10:43:59.012678+0000) 2024-04-23T10:44:49.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:49 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:49.013+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:43.515012+0000 front 2024-04-23T10:43:43.515041+0000 (oldest deadline 2024-04-23T10:44:05.815169+0000) 2024-04-23T10:44:49.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:49 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:49.013+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:51.716168+0000 front 2024-04-23T10:43:51.716308+0000 (oldest deadline 2024-04-23T10:44:17.016216+0000) 2024-04-23T10:44:49.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:49 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:49.013+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:57.517193+0000 front 2024-04-23T10:43:57.517281+0000 (oldest deadline 2024-04-23T10:44:22.817359+0000) 2024-04-23T10:44:49.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:49 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:49.013+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:06.918122+0000 front 2024-04-23T10:44:06.918039+0000 (oldest deadline 2024-04-23T10:44:32.218202+0000) 2024-04-23T10:44:49.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:49 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:49.013+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-23T10:44:14.420042+0000 front 2024-04-23T10:44:14.419849+0000 (oldest deadline 2024-04-23T10:44:39.719765+0000) 2024-04-23T10:44:49.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:49 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:49.013+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6822 osd.6 since back 2024-04-23T10:44:23.721209+0000 front 2024-04-23T10:44:23.721338+0000 (oldest deadline 2024-04-23T10:44:47.821149+0000) 2024-04-23T10:44:49.842 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:49 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:49.435+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.335834+0000 front 2024-04-23T10:43:36.335867+0000 (oldest deadline 2024-04-23T10:43:57.435822+0000) 2024-04-23T10:44:49.842 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:49 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:49.435+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.637810+0000 front 2024-04-23T10:43:42.637720+0000 (oldest deadline 2024-04-23T10:44:07.937634+0000) 2024-04-23T10:44:49.842 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:49 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:49.435+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:52.538910+0000 front 2024-04-23T10:43:52.538984+0000 (oldest deadline 2024-04-23T10:44:14.238911+0000) 2024-04-23T10:44:49.842 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:49 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:49.435+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:08.241835+0000 front 2024-04-23T10:44:08.241852+0000 (oldest deadline 2024-04-23T10:44:31.741843+0000) 2024-04-23T10:44:49.842 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:49 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:49.435+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-23T10:44:17.042719+0000 front 2024-04-23T10:44:17.042744+0000 (oldest deadline 2024-04-23T10:44:42.942817+0000) 2024-04-23T10:44:49.842 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:49 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:49.435+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6822 osd.6 since back 2024-04-23T10:44:23.944475+0000 front 2024-04-23T10:44:23.944535+0000 (oldest deadline 2024-04-23T10:44:48.644425+0000) 2024-04-23T10:44:49.860 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:49 smithi097 podman[94684]: 2024-04-23 10:44:49.607908847 +0000 UTC m=+5.559371880 container died a3ffebf4ff7e8facd3c63636bb4eb4a462e7e135a4475f92d09529c16dde00c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-23T10:44:50.240 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:49 smithi097 podman[94684]: 2024-04-23 10:44:49.966425912 +0000 UTC m=+5.917888942 container cleanup a3ffebf4ff7e8facd3c63636bb4eb4a462e7e135a4475f92d09529c16dde00c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-23T10:44:50.241 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:49 smithi097 bash[94684]: ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9 2024-04-23T10:44:50.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:50 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:50.022+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.211952+0000 front 2024-04-23T10:43:33.211903+0000 (oldest deadline 2024-04-23T10:43:59.012678+0000) 2024-04-23T10:44:50.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:50 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:50.022+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:43.515012+0000 front 2024-04-23T10:43:43.515041+0000 (oldest deadline 2024-04-23T10:44:05.815169+0000) 2024-04-23T10:44:50.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:50 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:50.022+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:51.716168+0000 front 2024-04-23T10:43:51.716308+0000 (oldest deadline 2024-04-23T10:44:17.016216+0000) 2024-04-23T10:44:50.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:50 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:50.022+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:57.517193+0000 front 2024-04-23T10:43:57.517281+0000 (oldest deadline 2024-04-23T10:44:22.817359+0000) 2024-04-23T10:44:50.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:50 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:50.022+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:06.918122+0000 front 2024-04-23T10:44:06.918039+0000 (oldest deadline 2024-04-23T10:44:32.218202+0000) 2024-04-23T10:44:50.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:50 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:50.022+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-23T10:44:14.420042+0000 front 2024-04-23T10:44:14.419849+0000 (oldest deadline 2024-04-23T10:44:39.719765+0000) 2024-04-23T10:44:50.401 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:50 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:50.022+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6822 osd.6 since back 2024-04-23T10:44:23.721209+0000 front 2024-04-23T10:44:23.721338+0000 (oldest deadline 2024-04-23T10:44:47.821149+0000) 2024-04-23T10:44:50.585 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:50 smithi097 podman[94701]: 2024-04-23 10:44:50.242542107 +0000 UTC m=+0.633252479 container remove a3ffebf4ff7e8facd3c63636bb4eb4a462e7e135a4475f92d09529c16dde00c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-23T10:44:50.835 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:50 smithi097 podman[94794]: 2024-04-23 10:44:50.489962706 +0000 UTC m=+0.017608439 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T10:44:50.835 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:50 smithi097 podman[94794]: 2024-04-23 10:44:50.610129034 +0000 UTC m=+0.137774804 container create 4fb39dbe22bf137ce8f3332030238c66c117f88011275df2b6cebdafe097644b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS) 2024-04-23T10:44:50.841 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:50 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:50.401+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.335834+0000 front 2024-04-23T10:43:36.335867+0000 (oldest deadline 2024-04-23T10:43:57.435822+0000) 2024-04-23T10:44:50.841 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:50 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:50.401+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.637810+0000 front 2024-04-23T10:43:42.637720+0000 (oldest deadline 2024-04-23T10:44:07.937634+0000) 2024-04-23T10:44:50.842 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:50 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:50.401+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:52.538910+0000 front 2024-04-23T10:43:52.538984+0000 (oldest deadline 2024-04-23T10:44:14.238911+0000) 2024-04-23T10:44:50.842 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:50 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:50.401+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:08.241835+0000 front 2024-04-23T10:44:08.241852+0000 (oldest deadline 2024-04-23T10:44:31.741843+0000) 2024-04-23T10:44:50.842 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:50 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:50.401+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-23T10:44:17.042719+0000 front 2024-04-23T10:44:17.042744+0000 (oldest deadline 2024-04-23T10:44:42.942817+0000) 2024-04-23T10:44:50.842 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:50 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:50.401+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6822 osd.6 since back 2024-04-23T10:44:23.944475+0000 front 2024-04-23T10:44:23.944535+0000 (oldest deadline 2024-04-23T10:44:48.644425+0000) 2024-04-23T10:44:51.110 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:50 smithi097 podman[94794]: 2024-04-23 10:44:50.837474929 +0000 UTC m=+0.365120634 container init 4fb39dbe22bf137ce8f3332030238c66c117f88011275df2b6cebdafe097644b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T10:44:51.111 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:50 smithi097 podman[94794]: 2024-04-23 10:44:50.841345848 +0000 UTC m=+0.368991554 container start 4fb39dbe22bf137ce8f3332030238c66c117f88011275df2b6cebdafe097644b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416) 2024-04-23T10:44:51.111 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:50 smithi097 podman[94794]: 2024-04-23 10:44:50.929024017 +0000 UTC m=+0.456669741 container attach 4fb39dbe22bf137ce8f3332030238c66c117f88011275df2b6cebdafe097644b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-23T10:44:51.396 INFO:journalctl@ceph.osd.9.smithi097.stdout:Apr 23 10:44:51 smithi097 podman[94794]: 2024-04-23 10:44:51.127481173 +0000 UTC m=+0.655126889 container died 4fb39dbe22bf137ce8f3332030238c66c117f88011275df2b6cebdafe097644b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-23T10:44:51.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:51 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:51.013+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.211952+0000 front 2024-04-23T10:43:33.211903+0000 (oldest deadline 2024-04-23T10:43:59.012678+0000) 2024-04-23T10:44:51.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:51 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:51.013+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:43.515012+0000 front 2024-04-23T10:43:43.515041+0000 (oldest deadline 2024-04-23T10:44:05.815169+0000) 2024-04-23T10:44:51.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:51 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:51.013+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:51.716168+0000 front 2024-04-23T10:43:51.716308+0000 (oldest deadline 2024-04-23T10:44:17.016216+0000) 2024-04-23T10:44:51.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:51 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:51.013+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:57.517193+0000 front 2024-04-23T10:43:57.517281+0000 (oldest deadline 2024-04-23T10:44:22.817359+0000) 2024-04-23T10:44:51.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:51 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:51.013+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:06.918122+0000 front 2024-04-23T10:44:06.918039+0000 (oldest deadline 2024-04-23T10:44:32.218202+0000) 2024-04-23T10:44:51.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:51 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:51.013+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-23T10:44:14.420042+0000 front 2024-04-23T10:44:14.419849+0000 (oldest deadline 2024-04-23T10:44:39.719765+0000) 2024-04-23T10:44:51.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:51 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:51.013+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6822 osd.6 since back 2024-04-23T10:44:23.721209+0000 front 2024-04-23T10:44:23.721338+0000 (oldest deadline 2024-04-23T10:44:47.821149+0000) 2024-04-23T10:44:51.721 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b22af88-0159-11ef-bc93-c7b262605968@osd.9.service' 2024-04-23T10:44:51.800 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T10:44:51.800 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-23T10:44:51.800 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-23T10:44:51.800 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-9b22af88-0159-11ef-bc93-c7b262605968@osd.10 2024-04-23T10:44:51.841 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:51 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:51.370+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.335834+0000 front 2024-04-23T10:43:36.335867+0000 (oldest deadline 2024-04-23T10:43:57.435822+0000) 2024-04-23T10:44:51.842 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:51 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:51.370+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.637810+0000 front 2024-04-23T10:43:42.637720+0000 (oldest deadline 2024-04-23T10:44:07.937634+0000) 2024-04-23T10:44:51.842 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:51 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:51.370+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:52.538910+0000 front 2024-04-23T10:43:52.538984+0000 (oldest deadline 2024-04-23T10:44:14.238911+0000) 2024-04-23T10:44:51.842 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:51 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:51.370+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:08.241835+0000 front 2024-04-23T10:44:08.241852+0000 (oldest deadline 2024-04-23T10:44:31.741843+0000) 2024-04-23T10:44:51.842 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:51 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:51.370+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-23T10:44:17.042719+0000 front 2024-04-23T10:44:17.042744+0000 (oldest deadline 2024-04-23T10:44:42.942817+0000) 2024-04-23T10:44:51.842 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:51 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:51.370+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6822 osd.6 since back 2024-04-23T10:44:23.944475+0000 front 2024-04-23T10:44:23.944535+0000 (oldest deadline 2024-04-23T10:44:48.644425+0000) 2024-04-23T10:44:52.130 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:51 smithi183 systemd[1]: Stopping Ceph osd.10 for 9b22af88-0159-11ef-bc93-c7b262605968... 2024-04-23T10:44:52.130 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:51 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:51.966+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.211952+0000 front 2024-04-23T10:43:33.211903+0000 (oldest deadline 2024-04-23T10:43:59.012678+0000) 2024-04-23T10:44:52.131 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:51 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:51.966+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:43.515012+0000 front 2024-04-23T10:43:43.515041+0000 (oldest deadline 2024-04-23T10:44:05.815169+0000) 2024-04-23T10:44:52.131 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:51 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:51.966+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:51.716168+0000 front 2024-04-23T10:43:51.716308+0000 (oldest deadline 2024-04-23T10:44:17.016216+0000) 2024-04-23T10:44:52.131 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:51 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:51.966+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:57.517193+0000 front 2024-04-23T10:43:57.517281+0000 (oldest deadline 2024-04-23T10:44:22.817359+0000) 2024-04-23T10:44:52.131 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:51 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:51.966+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:06.918122+0000 front 2024-04-23T10:44:06.918039+0000 (oldest deadline 2024-04-23T10:44:32.218202+0000) 2024-04-23T10:44:52.131 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:51 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:51.966+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-23T10:44:14.420042+0000 front 2024-04-23T10:44:14.419849+0000 (oldest deadline 2024-04-23T10:44:39.719765+0000) 2024-04-23T10:44:52.131 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:51 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:51.966+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6822 osd.6 since back 2024-04-23T10:44:23.721209+0000 front 2024-04-23T10:44:23.721338+0000 (oldest deadline 2024-04-23T10:44:47.821149+0000) 2024-04-23T10:44:52.131 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:52 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:52.130+0000 7fe4cb391700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T10:44:52.131 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:52 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:52.130+0000 7fe4cb391700 -1 osd.10 114 *** Got signal Terminated *** 2024-04-23T10:44:52.131 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:52 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:52.130+0000 7fe4cb391700 -1 osd.10 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T10:44:52.591 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:52 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:52.326+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.335834+0000 front 2024-04-23T10:43:36.335867+0000 (oldest deadline 2024-04-23T10:43:57.435822+0000) 2024-04-23T10:44:52.591 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:52 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:52.326+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.637810+0000 front 2024-04-23T10:43:42.637720+0000 (oldest deadline 2024-04-23T10:44:07.937634+0000) 2024-04-23T10:44:52.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:52 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:52.326+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:52.538910+0000 front 2024-04-23T10:43:52.538984+0000 (oldest deadline 2024-04-23T10:44:14.238911+0000) 2024-04-23T10:44:52.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:52 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:52.326+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:08.241835+0000 front 2024-04-23T10:44:08.241852+0000 (oldest deadline 2024-04-23T10:44:31.741843+0000) 2024-04-23T10:44:52.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:52 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:52.326+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-23T10:44:17.042719+0000 front 2024-04-23T10:44:17.042744+0000 (oldest deadline 2024-04-23T10:44:42.942817+0000) 2024-04-23T10:44:52.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:52 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:52.326+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6822 osd.6 since back 2024-04-23T10:44:23.944475+0000 front 2024-04-23T10:44:23.944535+0000 (oldest deadline 2024-04-23T10:44:48.644425+0000) 2024-04-23T10:44:53.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:52 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:52.956+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.211952+0000 front 2024-04-23T10:43:33.211903+0000 (oldest deadline 2024-04-23T10:43:59.012678+0000) 2024-04-23T10:44:53.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:52 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:52.956+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:43.515012+0000 front 2024-04-23T10:43:43.515041+0000 (oldest deadline 2024-04-23T10:44:05.815169+0000) 2024-04-23T10:44:53.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:52 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:52.956+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:51.716168+0000 front 2024-04-23T10:43:51.716308+0000 (oldest deadline 2024-04-23T10:44:17.016216+0000) 2024-04-23T10:44:53.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:52 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:52.956+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:57.517193+0000 front 2024-04-23T10:43:57.517281+0000 (oldest deadline 2024-04-23T10:44:22.817359+0000) 2024-04-23T10:44:53.401 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:52 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:52.956+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:06.918122+0000 front 2024-04-23T10:44:06.918039+0000 (oldest deadline 2024-04-23T10:44:32.218202+0000) 2024-04-23T10:44:53.401 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:52 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:52.956+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-23T10:44:14.420042+0000 front 2024-04-23T10:44:14.419849+0000 (oldest deadline 2024-04-23T10:44:39.719765+0000) 2024-04-23T10:44:53.401 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:52 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:52.956+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6822 osd.6 since back 2024-04-23T10:44:23.721209+0000 front 2024-04-23T10:44:23.721338+0000 (oldest deadline 2024-04-23T10:44:47.821149+0000) 2024-04-23T10:44:53.591 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:53 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:53.300+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.335834+0000 front 2024-04-23T10:43:36.335867+0000 (oldest deadline 2024-04-23T10:43:57.435822+0000) 2024-04-23T10:44:53.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:53 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:53.300+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.637810+0000 front 2024-04-23T10:43:42.637720+0000 (oldest deadline 2024-04-23T10:44:07.937634+0000) 2024-04-23T10:44:53.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:53 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:53.300+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:52.538910+0000 front 2024-04-23T10:43:52.538984+0000 (oldest deadline 2024-04-23T10:44:14.238911+0000) 2024-04-23T10:44:53.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:53 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:53.300+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:08.241835+0000 front 2024-04-23T10:44:08.241852+0000 (oldest deadline 2024-04-23T10:44:31.741843+0000) 2024-04-23T10:44:53.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:53 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:53.300+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-23T10:44:17.042719+0000 front 2024-04-23T10:44:17.042744+0000 (oldest deadline 2024-04-23T10:44:42.942817+0000) 2024-04-23T10:44:53.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:53 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:53.300+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6822 osd.6 since back 2024-04-23T10:44:23.944475+0000 front 2024-04-23T10:44:23.944535+0000 (oldest deadline 2024-04-23T10:44:48.644425+0000) 2024-04-23T10:44:54.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:53 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:53.939+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.211952+0000 front 2024-04-23T10:43:33.211903+0000 (oldest deadline 2024-04-23T10:43:59.012678+0000) 2024-04-23T10:44:54.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:53 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:53.939+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:43.515012+0000 front 2024-04-23T10:43:43.515041+0000 (oldest deadline 2024-04-23T10:44:05.815169+0000) 2024-04-23T10:44:54.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:53 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:53.939+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:51.716168+0000 front 2024-04-23T10:43:51.716308+0000 (oldest deadline 2024-04-23T10:44:17.016216+0000) 2024-04-23T10:44:54.401 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:53 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:53.939+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:57.517193+0000 front 2024-04-23T10:43:57.517281+0000 (oldest deadline 2024-04-23T10:44:22.817359+0000) 2024-04-23T10:44:54.401 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:53 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:53.939+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:06.918122+0000 front 2024-04-23T10:44:06.918039+0000 (oldest deadline 2024-04-23T10:44:32.218202+0000) 2024-04-23T10:44:54.401 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:53 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:53.939+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-23T10:44:14.420042+0000 front 2024-04-23T10:44:14.419849+0000 (oldest deadline 2024-04-23T10:44:39.719765+0000) 2024-04-23T10:44:54.401 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:53 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:53.939+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6822 osd.6 since back 2024-04-23T10:44:23.721209+0000 front 2024-04-23T10:44:23.721338+0000 (oldest deadline 2024-04-23T10:44:47.821149+0000) 2024-04-23T10:44:54.591 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:54 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:54.328+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.335834+0000 front 2024-04-23T10:43:36.335867+0000 (oldest deadline 2024-04-23T10:43:57.435822+0000) 2024-04-23T10:44:54.591 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:54 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:54.328+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.637810+0000 front 2024-04-23T10:43:42.637720+0000 (oldest deadline 2024-04-23T10:44:07.937634+0000) 2024-04-23T10:44:54.591 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:54 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:54.328+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:52.538910+0000 front 2024-04-23T10:43:52.538984+0000 (oldest deadline 2024-04-23T10:44:14.238911+0000) 2024-04-23T10:44:54.591 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:54 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:54.328+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:08.241835+0000 front 2024-04-23T10:44:08.241852+0000 (oldest deadline 2024-04-23T10:44:31.741843+0000) 2024-04-23T10:44:54.591 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:54 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:54.328+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-23T10:44:17.042719+0000 front 2024-04-23T10:44:17.042744+0000 (oldest deadline 2024-04-23T10:44:42.942817+0000) 2024-04-23T10:44:54.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:54 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:54.328+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6822 osd.6 since back 2024-04-23T10:44:23.944475+0000 front 2024-04-23T10:44:23.944535+0000 (oldest deadline 2024-04-23T10:44:48.644425+0000) 2024-04-23T10:44:55.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:54 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:54.988+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.211952+0000 front 2024-04-23T10:43:33.211903+0000 (oldest deadline 2024-04-23T10:43:59.012678+0000) 2024-04-23T10:44:55.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:54 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:54.988+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:43.515012+0000 front 2024-04-23T10:43:43.515041+0000 (oldest deadline 2024-04-23T10:44:05.815169+0000) 2024-04-23T10:44:55.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:54 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:54.988+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:51.716168+0000 front 2024-04-23T10:43:51.716308+0000 (oldest deadline 2024-04-23T10:44:17.016216+0000) 2024-04-23T10:44:55.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:54 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:54.988+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:57.517193+0000 front 2024-04-23T10:43:57.517281+0000 (oldest deadline 2024-04-23T10:44:22.817359+0000) 2024-04-23T10:44:55.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:54 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:54.988+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:06.918122+0000 front 2024-04-23T10:44:06.918039+0000 (oldest deadline 2024-04-23T10:44:32.218202+0000) 2024-04-23T10:44:55.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:54 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:54.988+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-23T10:44:14.420042+0000 front 2024-04-23T10:44:14.419849+0000 (oldest deadline 2024-04-23T10:44:39.719765+0000) 2024-04-23T10:44:55.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:54 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:54.988+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6822 osd.6 since back 2024-04-23T10:44:23.721209+0000 front 2024-04-23T10:44:23.721338+0000 (oldest deadline 2024-04-23T10:44:47.821149+0000) 2024-04-23T10:44:55.591 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:55 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:55.314+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.335834+0000 front 2024-04-23T10:43:36.335867+0000 (oldest deadline 2024-04-23T10:43:57.435822+0000) 2024-04-23T10:44:55.591 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:55 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:55.314+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.637810+0000 front 2024-04-23T10:43:42.637720+0000 (oldest deadline 2024-04-23T10:44:07.937634+0000) 2024-04-23T10:44:55.591 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:55 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:55.314+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:52.538910+0000 front 2024-04-23T10:43:52.538984+0000 (oldest deadline 2024-04-23T10:44:14.238911+0000) 2024-04-23T10:44:55.591 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:55 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:55.314+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:08.241835+0000 front 2024-04-23T10:44:08.241852+0000 (oldest deadline 2024-04-23T10:44:31.741843+0000) 2024-04-23T10:44:55.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:55 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:55.314+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-23T10:44:17.042719+0000 front 2024-04-23T10:44:17.042744+0000 (oldest deadline 2024-04-23T10:44:42.942817+0000) 2024-04-23T10:44:55.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:55 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:55.314+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6822 osd.6 since back 2024-04-23T10:44:23.944475+0000 front 2024-04-23T10:44:23.944535+0000 (oldest deadline 2024-04-23T10:44:48.644425+0000) 2024-04-23T10:44:56.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:55 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:55.988+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.211952+0000 front 2024-04-23T10:43:33.211903+0000 (oldest deadline 2024-04-23T10:43:59.012678+0000) 2024-04-23T10:44:56.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:55 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:55.988+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:43.515012+0000 front 2024-04-23T10:43:43.515041+0000 (oldest deadline 2024-04-23T10:44:05.815169+0000) 2024-04-23T10:44:56.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:55 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:55.988+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:51.716168+0000 front 2024-04-23T10:43:51.716308+0000 (oldest deadline 2024-04-23T10:44:17.016216+0000) 2024-04-23T10:44:56.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:55 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:55.988+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:57.517193+0000 front 2024-04-23T10:43:57.517281+0000 (oldest deadline 2024-04-23T10:44:22.817359+0000) 2024-04-23T10:44:56.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:55 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:55.988+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:06.918122+0000 front 2024-04-23T10:44:06.918039+0000 (oldest deadline 2024-04-23T10:44:32.218202+0000) 2024-04-23T10:44:56.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:55 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:55.988+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-23T10:44:14.420042+0000 front 2024-04-23T10:44:14.419849+0000 (oldest deadline 2024-04-23T10:44:39.719765+0000) 2024-04-23T10:44:56.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:55 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:55.988+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6822 osd.6 since back 2024-04-23T10:44:23.721209+0000 front 2024-04-23T10:44:23.721338+0000 (oldest deadline 2024-04-23T10:44:47.821149+0000) 2024-04-23T10:44:56.591 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:56 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:56.286+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.335834+0000 front 2024-04-23T10:43:36.335867+0000 (oldest deadline 2024-04-23T10:43:57.435822+0000) 2024-04-23T10:44:56.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:56 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:56.286+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.637810+0000 front 2024-04-23T10:43:42.637720+0000 (oldest deadline 2024-04-23T10:44:07.937634+0000) 2024-04-23T10:44:56.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:56 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:56.286+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:52.538910+0000 front 2024-04-23T10:43:52.538984+0000 (oldest deadline 2024-04-23T10:44:14.238911+0000) 2024-04-23T10:44:56.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:56 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:56.286+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:08.241835+0000 front 2024-04-23T10:44:08.241852+0000 (oldest deadline 2024-04-23T10:44:31.741843+0000) 2024-04-23T10:44:56.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:56 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:56.286+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-23T10:44:17.042719+0000 front 2024-04-23T10:44:17.042744+0000 (oldest deadline 2024-04-23T10:44:42.942817+0000) 2024-04-23T10:44:56.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:56 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:56.286+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6822 osd.6 since back 2024-04-23T10:44:23.944475+0000 front 2024-04-23T10:44:23.944535+0000 (oldest deadline 2024-04-23T10:44:48.644425+0000) 2024-04-23T10:44:56.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:56 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:56.286+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6820 osd.7 since back 2024-04-23T10:44:31.445753+0000 front 2024-04-23T10:44:31.445782+0000 (oldest deadline 2024-04-23T10:44:56.145728+0000) 2024-04-23T10:44:57.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:57 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:57.023+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:33.211952+0000 front 2024-04-23T10:43:33.211903+0000 (oldest deadline 2024-04-23T10:43:59.012678+0000) 2024-04-23T10:44:57.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:57 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:57.023+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:43.515012+0000 front 2024-04-23T10:43:43.515041+0000 (oldest deadline 2024-04-23T10:44:05.815169+0000) 2024-04-23T10:44:57.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:57 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:57.023+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:51.716168+0000 front 2024-04-23T10:43:51.716308+0000 (oldest deadline 2024-04-23T10:44:17.016216+0000) 2024-04-23T10:44:57.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:57 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:57.023+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6814 osd.3 since back 2024-04-23T10:43:57.517193+0000 front 2024-04-23T10:43:57.517281+0000 (oldest deadline 2024-04-23T10:44:22.817359+0000) 2024-04-23T10:44:57.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:57 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:57.023+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:06.918122+0000 front 2024-04-23T10:44:06.918039+0000 (oldest deadline 2024-04-23T10:44:32.218202+0000) 2024-04-23T10:44:57.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:57 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:57.023+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-23T10:44:14.420042+0000 front 2024-04-23T10:44:14.419849+0000 (oldest deadline 2024-04-23T10:44:39.719765+0000) 2024-04-23T10:44:57.401 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:57 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10[57311]: 2024-04-23T10:44:57.023+0000 7fe4c7985700 -1 osd.10 114 heartbeat_check: no reply from 172.21.15.97:6822 osd.6 since back 2024-04-23T10:44:23.721209+0000 front 2024-04-23T10:44:23.721338+0000 (oldest deadline 2024-04-23T10:44:47.821149+0000) 2024-04-23T10:44:57.591 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:57 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:57.266+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.335834+0000 front 2024-04-23T10:43:36.335867+0000 (oldest deadline 2024-04-23T10:43:57.435822+0000) 2024-04-23T10:44:57.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:57 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:57.266+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.637810+0000 front 2024-04-23T10:43:42.637720+0000 (oldest deadline 2024-04-23T10:44:07.937634+0000) 2024-04-23T10:44:57.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:57 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:57.266+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:52.538910+0000 front 2024-04-23T10:43:52.538984+0000 (oldest deadline 2024-04-23T10:44:14.238911+0000) 2024-04-23T10:44:57.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:57 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:57.266+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:08.241835+0000 front 2024-04-23T10:44:08.241852+0000 (oldest deadline 2024-04-23T10:44:31.741843+0000) 2024-04-23T10:44:57.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:57 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:57.266+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-23T10:44:17.042719+0000 front 2024-04-23T10:44:17.042744+0000 (oldest deadline 2024-04-23T10:44:42.942817+0000) 2024-04-23T10:44:57.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:57 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:57.266+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6822 osd.6 since back 2024-04-23T10:44:23.944475+0000 front 2024-04-23T10:44:23.944535+0000 (oldest deadline 2024-04-23T10:44:48.644425+0000) 2024-04-23T10:44:57.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:57 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:57.266+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6820 osd.7 since back 2024-04-23T10:44:31.445753+0000 front 2024-04-23T10:44:31.445782+0000 (oldest deadline 2024-04-23T10:44:56.145728+0000) 2024-04-23T10:44:57.763 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:57 smithi183 podman[70426]: 2024-04-23 10:44:57.491991822 +0000 UTC m=+5.488022625 container died 5ff072c98c744b5ddf2e67a08e6d0a0a3a3f682b6dcad373732101e4851ef2fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-23T10:44:58.138 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:57 smithi183 podman[70426]: 2024-04-23 10:44:57.855951344 +0000 UTC m=+5.851982136 container cleanup 5ff072c98c744b5ddf2e67a08e6d0a0a3a3f682b6dcad373732101e4851ef2fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T10:44:58.138 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:57 smithi183 bash[70426]: ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10 2024-04-23T10:44:58.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:58 smithi183 podman[70445]: 2024-04-23 10:44:58.140298589 +0000 UTC m=+0.639929264 container remove 5ff072c98c744b5ddf2e67a08e6d0a0a3a3f682b6dcad373732101e4851ef2fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T10:44:58.507 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:58 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:58.250+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.335834+0000 front 2024-04-23T10:43:36.335867+0000 (oldest deadline 2024-04-23T10:43:57.435822+0000) 2024-04-23T10:44:58.507 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:58 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:58.250+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.637810+0000 front 2024-04-23T10:43:42.637720+0000 (oldest deadline 2024-04-23T10:44:07.937634+0000) 2024-04-23T10:44:58.507 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:58 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:58.250+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:52.538910+0000 front 2024-04-23T10:43:52.538984+0000 (oldest deadline 2024-04-23T10:44:14.238911+0000) 2024-04-23T10:44:58.507 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:58 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:58.250+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:08.241835+0000 front 2024-04-23T10:44:08.241852+0000 (oldest deadline 2024-04-23T10:44:31.741843+0000) 2024-04-23T10:44:58.507 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:58 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:58.250+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-23T10:44:17.042719+0000 front 2024-04-23T10:44:17.042744+0000 (oldest deadline 2024-04-23T10:44:42.942817+0000) 2024-04-23T10:44:58.507 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:58 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:58.250+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6822 osd.6 since back 2024-04-23T10:44:23.944475+0000 front 2024-04-23T10:44:23.944535+0000 (oldest deadline 2024-04-23T10:44:48.644425+0000) 2024-04-23T10:44:58.508 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:58 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:58.250+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6820 osd.7 since back 2024-04-23T10:44:31.445753+0000 front 2024-04-23T10:44:31.445782+0000 (oldest deadline 2024-04-23T10:44:56.145728+0000) 2024-04-23T10:44:58.734 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:58 smithi183 podman[70534]: 2024-04-23 10:44:58.37724786 +0000 UTC m=+0.017706530 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T10:44:58.734 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:58 smithi183 podman[70534]: 2024-04-23 10:44:58.516385721 +0000 UTC m=+0.156844420 container create e269173fbf72934ee4efda404384a4105c93c2e03f53e8e172da96ff7fb2bfc1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-23T10:44:58.734 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:58 smithi183 podman[70534]: 2024-04-23 10:44:58.73550205 +0000 UTC m=+0.375960717 container init e269173fbf72934ee4efda404384a4105c93c2e03f53e8e172da96ff7fb2bfc1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-23T10:44:59.020 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:58 smithi183 podman[70534]: 2024-04-23 10:44:58.739197745 +0000 UTC m=+0.379656403 container start e269173fbf72934ee4efda404384a4105c93c2e03f53e8e172da96ff7fb2bfc1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T10:44:59.020 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:58 smithi183 podman[70534]: 2024-04-23 10:44:58.801732959 +0000 UTC m=+0.442191633 container attach e269173fbf72934ee4efda404384a4105c93c2e03f53e8e172da96ff7fb2bfc1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T10:44:59.293 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:59 smithi183 podman[70534]: 2024-04-23 10:44:59.02227536 +0000 UTC m=+0.662734025 container died e269173fbf72934ee4efda404384a4105c93c2e03f53e8e172da96ff7fb2bfc1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-23T10:44:59.530 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b22af88-0159-11ef-bc93-c7b262605968@osd.10.service' 2024-04-23T10:44:59.552 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:59 smithi183 podman[70560]: 2024-04-23 10:44:59.504970134 +0000 UTC m=+0.477304357 container remove e269173fbf72934ee4efda404384a4105c93c2e03f53e8e172da96ff7fb2bfc1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T10:44:59.583 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:59 smithi183 systemd[1]: ceph-9b22af88-0159-11ef-bc93-c7b262605968@osd.10.service: Deactivated successfully. 2024-04-23T10:44:59.583 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:59 smithi183 systemd[1]: Stopped Ceph osd.10 for 9b22af88-0159-11ef-bc93-c7b262605968. 2024-04-23T10:44:59.583 INFO:journalctl@ceph.osd.10.smithi183.stdout:Apr 23 10:44:59 smithi183 systemd[1]: ceph-9b22af88-0159-11ef-bc93-c7b262605968@osd.10.service: Consumed 16.889s CPU time. 2024-04-23T10:44:59.585 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T10:44:59.585 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-23T10:44:59.585 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-23T10:44:59.586 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-9b22af88-0159-11ef-bc93-c7b262605968@osd.11 2024-04-23T10:44:59.591 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:59 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:59.283+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.335834+0000 front 2024-04-23T10:43:36.335867+0000 (oldest deadline 2024-04-23T10:43:57.435822+0000) 2024-04-23T10:44:59.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:59 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:59.283+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.637810+0000 front 2024-04-23T10:43:42.637720+0000 (oldest deadline 2024-04-23T10:44:07.937634+0000) 2024-04-23T10:44:59.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:59 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:59.283+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:52.538910+0000 front 2024-04-23T10:43:52.538984+0000 (oldest deadline 2024-04-23T10:44:14.238911+0000) 2024-04-23T10:44:59.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:59 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:59.283+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:08.241835+0000 front 2024-04-23T10:44:08.241852+0000 (oldest deadline 2024-04-23T10:44:31.741843+0000) 2024-04-23T10:44:59.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:59 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:59.283+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-23T10:44:17.042719+0000 front 2024-04-23T10:44:17.042744+0000 (oldest deadline 2024-04-23T10:44:42.942817+0000) 2024-04-23T10:44:59.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:59 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:59.283+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6822 osd.6 since back 2024-04-23T10:44:23.944475+0000 front 2024-04-23T10:44:23.944535+0000 (oldest deadline 2024-04-23T10:44:48.644425+0000) 2024-04-23T10:44:59.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:59 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:59.283+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6820 osd.7 since back 2024-04-23T10:44:31.445753+0000 front 2024-04-23T10:44:31.445782+0000 (oldest deadline 2024-04-23T10:44:56.145728+0000) 2024-04-23T10:44:59.885 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:59 smithi196 systemd[1]: Stopping Ceph osd.11 for 9b22af88-0159-11ef-bc93-c7b262605968... 2024-04-23T10:45:00.309 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:59 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:59.886+0000 7fa4fb477700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T10:45:00.309 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:59 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:59.886+0000 7fa4fb477700 -1 osd.11 114 *** Got signal Terminated *** 2024-04-23T10:45:00.309 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:44:59 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:44:59.886+0000 7fa4fb477700 -1 osd.11 114 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T10:45:00.591 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:45:00 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:45:00.311+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.335834+0000 front 2024-04-23T10:43:36.335867+0000 (oldest deadline 2024-04-23T10:43:57.435822+0000) 2024-04-23T10:45:00.591 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:45:00 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:45:00.311+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.637810+0000 front 2024-04-23T10:43:42.637720+0000 (oldest deadline 2024-04-23T10:44:07.937634+0000) 2024-04-23T10:45:00.591 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:45:00 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:45:00.311+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:52.538910+0000 front 2024-04-23T10:43:52.538984+0000 (oldest deadline 2024-04-23T10:44:14.238911+0000) 2024-04-23T10:45:00.591 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:45:00 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:45:00.311+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:08.241835+0000 front 2024-04-23T10:44:08.241852+0000 (oldest deadline 2024-04-23T10:44:31.741843+0000) 2024-04-23T10:45:00.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:45:00 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:45:00.311+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-23T10:44:17.042719+0000 front 2024-04-23T10:44:17.042744+0000 (oldest deadline 2024-04-23T10:44:42.942817+0000) 2024-04-23T10:45:00.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:45:00 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:45:00.311+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6822 osd.6 since back 2024-04-23T10:44:23.944475+0000 front 2024-04-23T10:44:23.944535+0000 (oldest deadline 2024-04-23T10:44:48.644425+0000) 2024-04-23T10:45:00.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:45:00 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:45:00.311+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6820 osd.7 since back 2024-04-23T10:44:31.445753+0000 front 2024-04-23T10:44:31.445782+0000 (oldest deadline 2024-04-23T10:44:56.145728+0000) 2024-04-23T10:45:01.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:45:01 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:45:01.334+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.335834+0000 front 2024-04-23T10:43:36.335867+0000 (oldest deadline 2024-04-23T10:43:57.435822+0000) 2024-04-23T10:45:01.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:45:01 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:45:01.334+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.637810+0000 front 2024-04-23T10:43:42.637720+0000 (oldest deadline 2024-04-23T10:44:07.937634+0000) 2024-04-23T10:45:01.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:45:01 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:45:01.334+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:52.538910+0000 front 2024-04-23T10:43:52.538984+0000 (oldest deadline 2024-04-23T10:44:14.238911+0000) 2024-04-23T10:45:01.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:45:01 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:45:01.334+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:08.241835+0000 front 2024-04-23T10:44:08.241852+0000 (oldest deadline 2024-04-23T10:44:31.741843+0000) 2024-04-23T10:45:01.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:45:01 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:45:01.334+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-23T10:44:17.042719+0000 front 2024-04-23T10:44:17.042744+0000 (oldest deadline 2024-04-23T10:44:42.942817+0000) 2024-04-23T10:45:01.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:45:01 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:45:01.334+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6822 osd.6 since back 2024-04-23T10:44:23.944475+0000 front 2024-04-23T10:44:23.944535+0000 (oldest deadline 2024-04-23T10:44:48.644425+0000) 2024-04-23T10:45:01.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:45:01 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:45:01.334+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6820 osd.7 since back 2024-04-23T10:44:31.445753+0000 front 2024-04-23T10:44:31.445782+0000 (oldest deadline 2024-04-23T10:44:56.145728+0000) 2024-04-23T10:45:02.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:45:02 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:45:02.329+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.335834+0000 front 2024-04-23T10:43:36.335867+0000 (oldest deadline 2024-04-23T10:43:57.435822+0000) 2024-04-23T10:45:02.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:45:02 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:45:02.329+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.637810+0000 front 2024-04-23T10:43:42.637720+0000 (oldest deadline 2024-04-23T10:44:07.937634+0000) 2024-04-23T10:45:02.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:45:02 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:45:02.329+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:52.538910+0000 front 2024-04-23T10:43:52.538984+0000 (oldest deadline 2024-04-23T10:44:14.238911+0000) 2024-04-23T10:45:02.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:45:02 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:45:02.329+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:08.241835+0000 front 2024-04-23T10:44:08.241852+0000 (oldest deadline 2024-04-23T10:44:31.741843+0000) 2024-04-23T10:45:02.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:45:02 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:45:02.329+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-23T10:44:17.042719+0000 front 2024-04-23T10:44:17.042744+0000 (oldest deadline 2024-04-23T10:44:42.942817+0000) 2024-04-23T10:45:02.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:45:02 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:45:02.329+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6822 osd.6 since back 2024-04-23T10:44:23.944475+0000 front 2024-04-23T10:44:23.944535+0000 (oldest deadline 2024-04-23T10:44:48.644425+0000) 2024-04-23T10:45:02.592 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:45:02 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:45:02.329+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6820 osd.7 since back 2024-04-23T10:44:31.445753+0000 front 2024-04-23T10:44:31.445782+0000 (oldest deadline 2024-04-23T10:44:56.145728+0000) 2024-04-23T10:45:03.841 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:45:03 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:45:03.349+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.335834+0000 front 2024-04-23T10:43:36.335867+0000 (oldest deadline 2024-04-23T10:43:57.435822+0000) 2024-04-23T10:45:03.842 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:45:03 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:45:03.349+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.637810+0000 front 2024-04-23T10:43:42.637720+0000 (oldest deadline 2024-04-23T10:44:07.937634+0000) 2024-04-23T10:45:03.842 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:45:03 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:45:03.349+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:52.538910+0000 front 2024-04-23T10:43:52.538984+0000 (oldest deadline 2024-04-23T10:44:14.238911+0000) 2024-04-23T10:45:03.842 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:45:03 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:45:03.349+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:08.241835+0000 front 2024-04-23T10:44:08.241852+0000 (oldest deadline 2024-04-23T10:44:31.741843+0000) 2024-04-23T10:45:03.842 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:45:03 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:45:03.349+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-23T10:44:17.042719+0000 front 2024-04-23T10:44:17.042744+0000 (oldest deadline 2024-04-23T10:44:42.942817+0000) 2024-04-23T10:45:03.842 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:45:03 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:45:03.349+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6822 osd.6 since back 2024-04-23T10:44:23.944475+0000 front 2024-04-23T10:44:23.944535+0000 (oldest deadline 2024-04-23T10:44:48.644425+0000) 2024-04-23T10:45:03.842 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:45:03 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:45:03.349+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6820 osd.7 since back 2024-04-23T10:44:31.445753+0000 front 2024-04-23T10:44:31.445782+0000 (oldest deadline 2024-04-23T10:44:56.145728+0000) 2024-04-23T10:45:04.841 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:45:04 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:45:04.343+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6806 osd.0 since back 2024-04-23T10:43:36.335834+0000 front 2024-04-23T10:43:36.335867+0000 (oldest deadline 2024-04-23T10:43:57.435822+0000) 2024-04-23T10:45:04.842 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:45:04 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:45:04.343+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-04-23T10:43:42.637810+0000 front 2024-04-23T10:43:42.637720+0000 (oldest deadline 2024-04-23T10:44:07.937634+0000) 2024-04-23T10:45:04.842 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:45:04 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:45:04.343+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6804 osd.2 since back 2024-04-23T10:43:52.538910+0000 front 2024-04-23T10:43:52.538984+0000 (oldest deadline 2024-04-23T10:44:14.238911+0000) 2024-04-23T10:45:04.842 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:45:04 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:45:04.343+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-04-23T10:44:08.241835+0000 front 2024-04-23T10:44:08.241852+0000 (oldest deadline 2024-04-23T10:44:31.741843+0000) 2024-04-23T10:45:04.842 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:45:04 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:45:04.343+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6812 osd.5 since back 2024-04-23T10:44:17.042719+0000 front 2024-04-23T10:44:17.042744+0000 (oldest deadline 2024-04-23T10:44:42.942817+0000) 2024-04-23T10:45:04.842 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:45:04 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:45:04.343+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.97:6822 osd.6 since back 2024-04-23T10:44:23.944475+0000 front 2024-04-23T10:44:23.944535+0000 (oldest deadline 2024-04-23T10:44:48.644425+0000) 2024-04-23T10:45:04.842 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:45:04 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:45:04.343+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.183:6820 osd.7 since back 2024-04-23T10:44:31.445753+0000 front 2024-04-23T10:44:31.445782+0000 (oldest deadline 2024-04-23T10:44:56.145728+0000) 2024-04-23T10:45:04.842 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:45:04 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11[58595]: 2024-04-23T10:45:04.343+0000 7fa4f7a6b700 -1 osd.11 114 heartbeat_check: no reply from 172.21.15.196:6820 osd.8 since back 2024-04-23T10:44:38.446495+0000 front 2024-04-23T10:44:38.446543+0000 (oldest deadline 2024-04-23T10:45:03.746541+0000) 2024-04-23T10:45:05.477 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:45:05 smithi196 podman[71549]: 2024-04-23 10:45:05.208530976 +0000 UTC m=+5.438464441 container died dd014384992fe71a9f170c8b5cb7b95604350a66eeb829f6d5094e92c719c1f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-23T10:45:05.841 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:45:05 smithi196 podman[71549]: 2024-04-23 10:45:05.578094917 +0000 UTC m=+5.808028391 container cleanup dd014384992fe71a9f170c8b5cb7b95604350a66eeb829f6d5094e92c719c1f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0) 2024-04-23T10:45:05.841 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:45:05 smithi196 bash[71549]: ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11 2024-04-23T10:45:06.189 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:45:05 smithi196 podman[71565]: 2024-04-23 10:45:05.845649656 +0000 UTC m=+0.635298012 container remove dd014384992fe71a9f170c8b5cb7b95604350a66eeb829f6d5094e92c719c1f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-23T10:45:06.480 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:45:06 smithi196 podman[71656]: 2024-04-23 10:45:06.094746152 +0000 UTC m=+0.018382038 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T10:45:06.480 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:45:06 smithi196 podman[71656]: 2024-04-23 10:45:06.204646628 +0000 UTC m=+0.128282507 container create d155ca48fe0323bf5ee8ac754a400daf66747ae76f53188f48c54b237e5093d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-23T10:45:06.480 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:45:06 smithi196 podman[71656]: 2024-04-23 10:45:06.415574184 +0000 UTC m=+0.339210063 container init d155ca48fe0323bf5ee8ac754a400daf66747ae76f53188f48c54b237e5093d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True) 2024-04-23T10:45:06.480 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:45:06 smithi196 podman[71656]: 2024-04-23 10:45:06.418598356 +0000 UTC m=+0.342234235 container start d155ca48fe0323bf5ee8ac754a400daf66747ae76f53188f48c54b237e5093d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-23T10:45:06.480 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:45:06 smithi196 podman[71656]: 2024-04-23 10:45:06.48221447 +0000 UTC m=+0.405850367 container attach d155ca48fe0323bf5ee8ac754a400daf66747ae76f53188f48c54b237e5093d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-23T10:45:06.841 INFO:journalctl@ceph.osd.11.smithi196.stdout:Apr 23 10:45:06 smithi196 podman[71656]: 2024-04-23 10:45:06.702907619 +0000 UTC m=+0.626543504 container died d155ca48fe0323bf5ee8ac754a400daf66747ae76f53188f48c54b237e5093d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-23T10:45:07.285 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b22af88-0159-11ef-bc93-c7b262605968@osd.11.service' 2024-04-23T10:45:07.363 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T10:45:07.364 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-23T10:45:07.364 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-23T10:45:07.364 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-9b22af88-0159-11ef-bc93-c7b262605968@mds.a 2024-04-23T10:45:07.666 INFO:journalctl@ceph.mds.a.smithi097.stdout:Apr 23 10:45:07 smithi097 systemd[1]: Stopping Ceph mds.a for 9b22af88-0159-11ef-bc93-c7b262605968... 2024-04-23T10:45:08.110 INFO:journalctl@ceph.mds.a.smithi097.stdout:Apr 23 10:45:07 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-a[68280]: 2024-04-23T10:45:07.667+0000 7fe37689e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T10:45:08.110 INFO:journalctl@ceph.mds.a.smithi097.stdout:Apr 23 10:45:07 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-a[68280]: 2024-04-23T10:45:07.667+0000 7fe37689e700 -1 mds.a *** got signal Terminated *** 2024-04-23T10:45:12.025 INFO:journalctl@ceph.mds.a.smithi097.stdout:Apr 23 10:45:11 smithi097 podman[94925]: 2024-04-23 10:45:11.674127611 +0000 UTC m=+4.112493463 container died 1ac5fb4846ea1de32c2e0ce6bb29d77d76f7bcedc3b04f48db995270ba742ec4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-a, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T10:45:12.334 INFO:journalctl@ceph.mds.a.smithi097.stdout:Apr 23 10:45:12 smithi097 podman[94925]: 2024-04-23 10:45:12.109644232 +0000 UTC m=+4.548010074 container cleanup 1ac5fb4846ea1de32c2e0ce6bb29d77d76f7bcedc3b04f48db995270ba742ec4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-a, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-23T10:45:12.334 INFO:journalctl@ceph.mds.a.smithi097.stdout:Apr 23 10:45:12 smithi097 bash[94925]: ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-a 2024-04-23T10:45:12.584 INFO:journalctl@ceph.mds.a.smithi097.stdout:Apr 23 10:45:12 smithi097 podman[94939]: 2024-04-23 10:45:12.335573105 +0000 UTC m=+0.655920714 container remove 1ac5fb4846ea1de32c2e0ce6bb29d77d76f7bcedc3b04f48db995270ba742ec4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-a, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-23T10:45:12.588 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b22af88-0159-11ef-bc93-c7b262605968@mds.a.service' 2024-04-23T10:45:12.623 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T10:45:12.623 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-23T10:45:12.623 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-23T10:45:12.623 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-9b22af88-0159-11ef-bc93-c7b262605968@mds.d 2024-04-23T10:45:12.860 INFO:journalctl@ceph.mds.d.smithi097.stdout:Apr 23 10:45:12 smithi097 systemd[1]: Stopping Ceph mds.d for 9b22af88-0159-11ef-bc93-c7b262605968... 2024-04-23T10:45:13.361 INFO:journalctl@ceph.mds.d.smithi097.stdout:Apr 23 10:45:12 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-d[67323]: 2024-04-23T10:45:12.996+0000 7fb409b73700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T10:45:13.361 INFO:journalctl@ceph.mds.d.smithi097.stdout:Apr 23 10:45:12 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-d[67323]: 2024-04-23T10:45:12.996+0000 7fb409b73700 -1 mds.d *** got signal Terminated *** 2024-04-23T10:45:14.578 INFO:journalctl@ceph.mds.d.smithi097.stdout:Apr 23 10:45:14 smithi097 podman[95048]: 2024-04-23 10:45:14.305658803 +0000 UTC m=+1.457716484 container died ed03f3b8a268626149c3c2a399acd09c3b7e7a13d040c80e1065199c19ed3cfc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, ceph=True) 2024-04-23T10:45:14.860 INFO:journalctl@ceph.mds.d.smithi097.stdout:Apr 23 10:45:14 smithi097 podman[95048]: 2024-04-23 10:45:14.679776818 +0000 UTC m=+1.831834535 container cleanup ed03f3b8a268626149c3c2a399acd09c3b7e7a13d040c80e1065199c19ed3cfc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-d, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-23T10:45:14.861 INFO:journalctl@ceph.mds.d.smithi097.stdout:Apr 23 10:45:14 smithi097 bash[95048]: ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-d 2024-04-23T10:45:15.113 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b22af88-0159-11ef-bc93-c7b262605968@mds.d.service' 2024-04-23T10:45:15.145 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T10:45:15.145 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-23T10:45:15.145 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-23T10:45:15.146 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-9b22af88-0159-11ef-bc93-c7b262605968@mds.g 2024-04-23T10:45:15.610 INFO:journalctl@ceph.mds.g.smithi097.stdout:Apr 23 10:45:15 smithi097 systemd[1]: Stopping Ceph mds.g for 9b22af88-0159-11ef-bc93-c7b262605968... 2024-04-23T10:45:15.611 INFO:journalctl@ceph.mds.g.smithi097.stdout:Apr 23 10:45:15 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-g[67920]: 2024-04-23T10:45:15.450+0000 7f24d7e36700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T10:45:15.611 INFO:journalctl@ceph.mds.g.smithi097.stdout:Apr 23 10:45:15 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-g[67920]: 2024-04-23T10:45:15.450+0000 7f24d7e36700 -1 mds.g *** got signal Terminated *** 2024-04-23T10:45:16.800 INFO:journalctl@ceph.mds.g.smithi097.stdout:Apr 23 10:45:16 smithi097 podman[95169]: 2024-04-23 10:45:16.515623431 +0000 UTC m=+1.157248613 container died ac5e05633d0d7c18661e0c519582c7846d4220f6208d0d7c5a91212aa6425e29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-g, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-23T10:45:17.110 INFO:journalctl@ceph.mds.g.smithi097.stdout:Apr 23 10:45:16 smithi097 podman[95169]: 2024-04-23 10:45:16.900601428 +0000 UTC m=+1.542226682 container cleanup ac5e05633d0d7c18661e0c519582c7846d4220f6208d0d7c5a91212aa6425e29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-g, org.label-schema.build-date=20240416, GIT_CLEAN=True, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T10:45:17.110 INFO:journalctl@ceph.mds.g.smithi097.stdout:Apr 23 10:45:16 smithi097 bash[95169]: ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-g 2024-04-23T10:45:17.376 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b22af88-0159-11ef-bc93-c7b262605968@mds.g.service' 2024-04-23T10:45:17.407 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T10:45:17.407 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-23T10:45:17.407 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-23T10:45:17.408 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-9b22af88-0159-11ef-bc93-c7b262605968@mds.j 2024-04-23T10:45:17.735 INFO:journalctl@ceph.mds.j.smithi097.stdout:Apr 23 10:45:17 smithi097 systemd[1]: Stopping Ceph mds.j for 9b22af88-0159-11ef-bc93-c7b262605968... 2024-04-23T10:45:18.110 INFO:journalctl@ceph.mds.j.smithi097.stdout:Apr 23 10:45:17 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-j[66977]: 2024-04-23T10:45:17.736+0000 7f9f01a82700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T10:45:18.110 INFO:journalctl@ceph.mds.j.smithi097.stdout:Apr 23 10:45:17 smithi097 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-j[66977]: 2024-04-23T10:45:17.736+0000 7f9f01a82700 -1 mds.j *** got signal Terminated *** 2024-04-23T10:45:19.544 INFO:journalctl@ceph.mds.j.smithi097.stdout:Apr 23 10:45:19 smithi097 podman[95290]: 2024-04-23 10:45:19.274564322 +0000 UTC m=+1.642642888 container died 38fb77b0074693f124f67ae53d2e2400e839ae9f5f6660339270fc33697fed63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-j, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-23T10:45:19.860 INFO:journalctl@ceph.mds.j.smithi097.stdout:Apr 23 10:45:19 smithi097 podman[95290]: 2024-04-23 10:45:19.637615907 +0000 UTC m=+2.005694483 container cleanup 38fb77b0074693f124f67ae53d2e2400e839ae9f5f6660339270fc33697fed63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-j, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-23T10:45:19.860 INFO:journalctl@ceph.mds.j.smithi097.stdout:Apr 23 10:45:19 smithi097 bash[95290]: ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-j 2024-04-23T10:45:20.017 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b22af88-0159-11ef-bc93-c7b262605968@mds.j.service' 2024-04-23T10:45:20.048 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T10:45:20.049 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-23T10:45:20.049 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-23T10:45:20.049 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-9b22af88-0159-11ef-bc93-c7b262605968@mds.b 2024-04-23T10:45:20.345 INFO:journalctl@ceph.mds.b.smithi183.stdout:Apr 23 10:45:20 smithi183 systemd[1]: Stopping Ceph mds.b for 9b22af88-0159-11ef-bc93-c7b262605968... 2024-04-23T10:45:20.650 INFO:journalctl@ceph.mds.b.smithi183.stdout:Apr 23 10:45:20 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-b[62562]: 2024-04-23T10:45:20.346+0000 7f2c910dd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T10:45:20.650 INFO:journalctl@ceph.mds.b.smithi183.stdout:Apr 23 10:45:20 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-b[62562]: 2024-04-23T10:45:20.346+0000 7f2c910dd700 -1 mds.b *** got signal Terminated *** 2024-04-23T10:45:24.650 INFO:journalctl@ceph.mds.b.smithi183.stdout:Apr 23 10:45:24 smithi183 podman[70651]: 2024-04-23 10:45:24.351802993 +0000 UTC m=+4.107881609 container died fddaf1fb6bb473828f67bd8156924c51ab64bcddff9dca253bb81a695a386833 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-b, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2) 2024-04-23T10:45:24.994 INFO:journalctl@ceph.mds.b.smithi183.stdout:Apr 23 10:45:24 smithi183 podman[70651]: 2024-04-23 10:45:24.754402332 +0000 UTC m=+4.510480990 container cleanup fddaf1fb6bb473828f67bd8156924c51ab64bcddff9dca253bb81a695a386833 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-b, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-23T10:45:24.995 INFO:journalctl@ceph.mds.b.smithi183.stdout:Apr 23 10:45:24 smithi183 bash[70651]: ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-b 2024-04-23T10:45:25.119 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b22af88-0159-11ef-bc93-c7b262605968@mds.b.service' 2024-04-23T10:45:25.153 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T10:45:25.153 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-23T10:45:25.153 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-23T10:45:25.153 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-9b22af88-0159-11ef-bc93-c7b262605968@mds.e 2024-04-23T10:45:25.400 INFO:journalctl@ceph.mds.e.smithi183.stdout:Apr 23 10:45:25 smithi183 systemd[1]: Stopping Ceph mds.e for 9b22af88-0159-11ef-bc93-c7b262605968... 2024-04-23T10:45:25.900 INFO:journalctl@ceph.mds.e.smithi183.stdout:Apr 23 10:45:25 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-e[61950]: 2024-04-23T10:45:25.482+0000 7fd2b7fd3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T10:45:25.900 INFO:journalctl@ceph.mds.e.smithi183.stdout:Apr 23 10:45:25 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-e[61950]: 2024-04-23T10:45:25.482+0000 7fd2b7fd3700 -1 mds.e *** got signal Terminated *** 2024-04-23T10:45:28.800 INFO:journalctl@ceph.mds.e.smithi183.stdout:Apr 23 10:45:28 smithi183 podman[70772]: 2024-04-23 10:45:28.538369929 +0000 UTC m=+3.171065259 container died bb92e4a07cf02d4aa12286edcb23a645d2053ab8f73de3af08b7c426cbb882a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-e, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T10:45:29.150 INFO:journalctl@ceph.mds.e.smithi183.stdout:Apr 23 10:45:28 smithi183 podman[70772]: 2024-04-23 10:45:28.902004245 +0000 UTC m=+3.534699577 container cleanup bb92e4a07cf02d4aa12286edcb23a645d2053ab8f73de3af08b7c426cbb882a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-e, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-23T10:45:29.150 INFO:journalctl@ceph.mds.e.smithi183.stdout:Apr 23 10:45:28 smithi183 bash[70772]: ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-e 2024-04-23T10:45:29.330 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b22af88-0159-11ef-bc93-c7b262605968@mds.e.service' 2024-04-23T10:45:29.364 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T10:45:29.364 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-23T10:45:29.364 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-23T10:45:29.364 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-9b22af88-0159-11ef-bc93-c7b262605968@mds.h 2024-04-23T10:45:29.696 INFO:journalctl@ceph.mds.h.smithi183.stdout:Apr 23 10:45:29 smithi183 systemd[1]: Stopping Ceph mds.h for 9b22af88-0159-11ef-bc93-c7b262605968... 2024-04-23T10:45:30.150 INFO:journalctl@ceph.mds.h.smithi183.stdout:Apr 23 10:45:29 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-h[62254]: 2024-04-23T10:45:29.696+0000 7fe89beac700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T10:45:30.150 INFO:journalctl@ceph.mds.h.smithi183.stdout:Apr 23 10:45:29 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-h[62254]: 2024-04-23T10:45:29.696+0000 7fe89beac700 -1 mds.h *** got signal Terminated *** 2024-04-23T10:45:32.829 INFO:journalctl@ceph.mds.h.smithi183.stdout:Apr 23 10:45:32 smithi183 podman[70905]: 2024-04-23 10:45:32.521722072 +0000 UTC m=+2.929074281 container died 0340b54e43161253590f48b368098f34c291b5f95c53c6e74e294a95f40db9f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-h, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-23T10:45:33.121 INFO:journalctl@ceph.mds.h.smithi183.stdout:Apr 23 10:45:32 smithi183 podman[70905]: 2024-04-23 10:45:32.930387515 +0000 UTC m=+3.337739793 container cleanup 0340b54e43161253590f48b368098f34c291b5f95c53c6e74e294a95f40db9f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-h, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-23T10:45:33.121 INFO:journalctl@ceph.mds.h.smithi183.stdout:Apr 23 10:45:32 smithi183 bash[70905]: ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-h 2024-04-23T10:45:33.230 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b22af88-0159-11ef-bc93-c7b262605968@mds.h.service' 2024-04-23T10:45:33.263 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T10:45:33.264 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-23T10:45:33.264 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-23T10:45:33.264 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-9b22af88-0159-11ef-bc93-c7b262605968@mds.k 2024-04-23T10:45:33.399 INFO:journalctl@ceph.mds.k.smithi183.stdout:Apr 23 10:45:33 smithi183 systemd[1]: Stopping Ceph mds.k for 9b22af88-0159-11ef-bc93-c7b262605968... 2024-04-23T10:45:33.900 INFO:journalctl@ceph.mds.k.smithi183.stdout:Apr 23 10:45:33 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-k[61625]: 2024-04-23T10:45:33.566+0000 7fd340d47700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T10:45:33.900 INFO:journalctl@ceph.mds.k.smithi183.stdout:Apr 23 10:45:33 smithi183 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-k[61625]: 2024-04-23T10:45:33.566+0000 7fd340d47700 -1 mds.k *** got signal Terminated *** 2024-04-23T10:45:36.817 INFO:journalctl@ceph.mds.k.smithi183.stdout:Apr 23 10:45:36 smithi183 podman[71025]: 2024-04-23 10:45:36.539107815 +0000 UTC m=+3.057067018 container died 6d907df4143ee3df4254116a7906808d37864c786dc21f630e8cd8c00d860332 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-k, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-23T10:45:37.149 INFO:journalctl@ceph.mds.k.smithi183.stdout:Apr 23 10:45:36 smithi183 podman[71025]: 2024-04-23 10:45:36.892173627 +0000 UTC m=+3.410132822 container cleanup 6d907df4143ee3df4254116a7906808d37864c786dc21f630e8cd8c00d860332 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-23T10:45:37.149 INFO:journalctl@ceph.mds.k.smithi183.stdout:Apr 23 10:45:36 smithi183 bash[71025]: ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-k 2024-04-23T10:45:37.312 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b22af88-0159-11ef-bc93-c7b262605968@mds.k.service' 2024-04-23T10:45:37.344 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T10:45:37.345 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-23T10:45:37.345 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-23T10:45:37.345 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-9b22af88-0159-11ef-bc93-c7b262605968@mds.c 2024-04-23T10:45:37.650 INFO:journalctl@ceph.mds.c.smithi196.stdout:Apr 23 10:45:37 smithi196 systemd[1]: Stopping Ceph mds.c for 9b22af88-0159-11ef-bc93-c7b262605968... 2024-04-23T10:45:37.905 INFO:journalctl@ceph.mds.c.smithi196.stdout:Apr 23 10:45:37 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-c[64046]: 2024-04-23T10:45:37.650+0000 7fa2bf1b1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T10:45:37.905 INFO:journalctl@ceph.mds.c.smithi196.stdout:Apr 23 10:45:37 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-c[64046]: 2024-04-23T10:45:37.650+0000 7fa2bf1b1700 -1 mds.c *** got signal Terminated *** 2024-04-23T10:45:40.808 INFO:journalctl@ceph.mds.c.smithi196.stdout:Apr 23 10:45:40 smithi196 podman[71762]: 2024-04-23 10:45:40.5218182 +0000 UTC m=+2.980489448 container died 5ceaaab165ca8b257190b3f4680b8d86c9109dc7426895dfe4e1ec7a278fbcd2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-c, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T10:45:41.091 INFO:journalctl@ceph.mds.c.smithi196.stdout:Apr 23 10:45:40 smithi196 podman[71762]: 2024-04-23 10:45:40.900538773 +0000 UTC m=+3.359210012 container cleanup 5ceaaab165ca8b257190b3f4680b8d86c9109dc7426895dfe4e1ec7a278fbcd2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-c, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T10:45:41.091 INFO:journalctl@ceph.mds.c.smithi196.stdout:Apr 23 10:45:40 smithi196 bash[71762]: ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-c 2024-04-23T10:45:41.316 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b22af88-0159-11ef-bc93-c7b262605968@mds.c.service' 2024-04-23T10:45:41.351 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T10:45:41.351 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-23T10:45:41.351 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-23T10:45:41.351 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-9b22af88-0159-11ef-bc93-c7b262605968@mds.f 2024-04-23T10:45:41.842 INFO:journalctl@ceph.mds.f.smithi196.stdout:Apr 23 10:45:41 smithi196 systemd[1]: Stopping Ceph mds.f for 9b22af88-0159-11ef-bc93-c7b262605968... 2024-04-23T10:45:41.842 INFO:journalctl@ceph.mds.f.smithi196.stdout:Apr 23 10:45:41 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-f[63731]: 2024-04-23T10:45:41.670+0000 7f0025397700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T10:45:41.842 INFO:journalctl@ceph.mds.f.smithi196.stdout:Apr 23 10:45:41 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-f[63731]: 2024-04-23T10:45:41.670+0000 7f0025397700 -1 mds.f *** got signal Terminated *** 2024-04-23T10:45:44.820 INFO:journalctl@ceph.mds.f.smithi196.stdout:Apr 23 10:45:44 smithi196 podman[71897]: 2024-04-23 10:45:44.548859427 +0000 UTC m=+2.973248058 container died 5fa315c4ec002895b34791b34d289501b9c4aa95171e7ba48c6931bc9b2a3520 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-f, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416) 2024-04-23T10:45:45.091 INFO:journalctl@ceph.mds.f.smithi196.stdout:Apr 23 10:45:44 smithi196 podman[71897]: 2024-04-23 10:45:44.929004651 +0000 UTC m=+3.353393269 container cleanup 5fa315c4ec002895b34791b34d289501b9c4aa95171e7ba48c6931bc9b2a3520 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-f, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T10:45:45.092 INFO:journalctl@ceph.mds.f.smithi196.stdout:Apr 23 10:45:44 smithi196 bash[71897]: ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-f 2024-04-23T10:45:45.299 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b22af88-0159-11ef-bc93-c7b262605968@mds.f.service' 2024-04-23T10:45:45.331 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T10:45:45.331 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-23T10:45:45.332 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-23T10:45:45.332 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-9b22af88-0159-11ef-bc93-c7b262605968@mds.i 2024-04-23T10:45:45.591 INFO:journalctl@ceph.mds.i.smithi196.stdout:Apr 23 10:45:45 smithi196 systemd[1]: Stopping Ceph mds.i for 9b22af88-0159-11ef-bc93-c7b262605968... 2024-04-23T10:45:46.091 INFO:journalctl@ceph.mds.i.smithi196.stdout:Apr 23 10:45:45 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-i[64351]: 2024-04-23T10:45:45.789+0000 7f313e950700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T10:45:46.091 INFO:journalctl@ceph.mds.i.smithi196.stdout:Apr 23 10:45:45 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-i[64351]: 2024-04-23T10:45:45.789+0000 7f313e950700 -1 mds.i *** got signal Terminated *** 2024-04-23T10:45:48.591 INFO:journalctl@ceph.mds.i.smithi196.stdout:Apr 23 10:45:48 smithi196 podman[72019]: 2024-04-23 10:45:48.527819238 +0000 UTC m=+2.983893128 container died 13dc06b92bdf2afba3fbfd562d3120c9f9cf5973118fd94879aa914d0fcc4d80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-i, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-23T10:45:49.092 INFO:journalctl@ceph.mds.i.smithi196.stdout:Apr 23 10:45:48 smithi196 podman[72019]: 2024-04-23 10:45:48.890929381 +0000 UTC m=+3.347003284 container cleanup 13dc06b92bdf2afba3fbfd562d3120c9f9cf5973118fd94879aa914d0fcc4d80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-i, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240416) 2024-04-23T10:45:49.092 INFO:journalctl@ceph.mds.i.smithi196.stdout:Apr 23 10:45:48 smithi196 bash[72019]: ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-i 2024-04-23T10:45:49.269 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b22af88-0159-11ef-bc93-c7b262605968@mds.i.service' 2024-04-23T10:45:49.299 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T10:45:49.299 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-23T10:45:49.300 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-23T10:45:49.300 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-9b22af88-0159-11ef-bc93-c7b262605968@mds.l 2024-04-23T10:45:49.591 INFO:journalctl@ceph.mds.l.smithi196.stdout:Apr 23 10:45:49 smithi196 systemd[1]: Stopping Ceph mds.l for 9b22af88-0159-11ef-bc93-c7b262605968... 2024-04-23T10:45:50.091 INFO:journalctl@ceph.mds.l.smithi196.stdout:Apr 23 10:45:49 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-l[63409]: 2024-04-23T10:45:49.619+0000 7ff110fa8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T10:45:50.091 INFO:journalctl@ceph.mds.l.smithi196.stdout:Apr 23 10:45:49 smithi196 ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-l[63409]: 2024-04-23T10:45:49.619+0000 7ff110fa8700 -1 mds.l *** got signal Terminated *** 2024-04-23T10:45:52.841 INFO:journalctl@ceph.mds.l.smithi196.stdout:Apr 23 10:45:52 smithi196 podman[72141]: 2024-04-23 10:45:52.547421187 +0000 UTC m=+3.030187376 container died d00e288d04d9a2cced0ac98ab87f98eabca234e016a96df4a5fb410029905f08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-l, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-23T10:45:53.168 INFO:journalctl@ceph.mds.l.smithi196.stdout:Apr 23 10:45:52 smithi196 podman[72141]: 2024-04-23 10:45:52.936340545 +0000 UTC m=+3.419106730 container cleanup d00e288d04d9a2cced0ac98ab87f98eabca234e016a96df4a5fb410029905f08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-l, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T10:45:53.168 INFO:journalctl@ceph.mds.l.smithi196.stdout:Apr 23 10:45:52 smithi196 bash[72141]: ceph-9b22af88-0159-11ef-bc93-c7b262605968-mds-l 2024-04-23T10:45:53.306 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b22af88-0159-11ef-bc93-c7b262605968@mds.l.service' 2024-04-23T10:45:53.337 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T10:45:53.337 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-23T10:45:53.337 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9b22af88-0159-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-23T10:45:53.527 INFO:teuthology.orchestra.run.smithi097.stdout:Deleting cluster with fsid: 9b22af88-0159-11ef-bc93-c7b262605968 2024-04-23T10:45:55.992 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9b22af88-0159-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-23T10:45:56.182 INFO:teuthology.orchestra.run.smithi183.stdout:Deleting cluster with fsid: 9b22af88-0159-11ef-bc93-c7b262605968 2024-04-23T10:45:58.520 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9b22af88-0159-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-23T10:45:58.711 INFO:teuthology.orchestra.run.smithi196.stdout:Deleting cluster with fsid: 9b22af88-0159-11ef-bc93-c7b262605968 2024-04-23T10:46:01.070 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-23T10:46:01.102 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-23T10:46:01.129 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-23T10:46:01.153 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-23T10:46:01.155 DEBUG:teuthology.misc:Transferring archived files from smithi097:/var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-23_09:49:14-fs-wip-lusov-quiescer-distro-default-smithi/7669527/remote/smithi097/crash 2024-04-23T10:46:01.156 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/crash -- . 2024-04-23T10:46:01.198 INFO:teuthology.orchestra.run.smithi097.stderr:tar: /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-23T10:46:01.198 INFO:teuthology.orchestra.run.smithi097.stderr:tar: Error is not recoverable: exiting now 2024-04-23T10:46:01.200 DEBUG:teuthology.misc:Transferring archived files from smithi183:/var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-23_09:49:14-fs-wip-lusov-quiescer-distro-default-smithi/7669527/remote/smithi183/crash 2024-04-23T10:46:01.201 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/crash -- . 2024-04-23T10:46:01.324 INFO:teuthology.orchestra.run.smithi183.stderr:tar: /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-23T10:46:01.325 INFO:teuthology.orchestra.run.smithi183.stderr:tar: Error is not recoverable: exiting now 2024-04-23T10:46:01.327 DEBUG:teuthology.misc:Transferring archived files from smithi196:/var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-23_09:49:14-fs-wip-lusov-quiescer-distro-default-smithi/7669527/remote/smithi196/crash 2024-04-23T10:46:01.328 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/crash -- . 2024-04-23T10:46:01.362 INFO:teuthology.orchestra.run.smithi196.stderr:tar: /var/lib/ceph/9b22af88-0159-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-23T10:46:01.362 INFO:teuthology.orchestra.run.smithi196.stderr:tar: Error is not recoverable: exiting now 2024-04-23T10:46:01.363 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-23T10:46:01.363 DEBUG:teuthology.orchestra.run.smithi097:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-04-23T10:46:01.398 INFO:tasks.cephadm:Compressing logs... 2024-04-23T10:46:01.398 DEBUG:teuthology.orchestra.run.smithi097:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-23T10:46:01.440 DEBUG:teuthology.orchestra.run.smithi183:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-23T10:46:01.443 DEBUG:teuthology.orchestra.run.smithi196:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-23T10:46:01.466 INFO:teuthology.orchestra.run.smithi183.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-23T10:46:01.466 INFO:teuthology.orchestra.run.smithi183.stderr:: No such file or directory 2024-04-23T10:46:01.467 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph.log 2024-04-23T10:46:01.467 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph.audit.log/var/log/ceph/cephadm.log: 2024-04-23T10:46:01.467 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-osd.7.log 2024-04-23T10:46:01.467 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-mds.b.log 2024-04-23T10:46:01.467 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-23T10:46:01.467 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-osd.7.log: /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-23T10:46:01.467 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-osd.4.log: 92.1%gzip -5 --verbose -- /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-mgr.y.log 2024-04-23T10:46:01.467 INFO:teuthology.orchestra.run.smithi183.stderr: -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-23T10:46:01.469 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-osd.1.log: 90.4% -- replaced with /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-23T10:46:01.469 INFO:teuthology.orchestra.run.smithi183.stderr: 87.4% -- replaced with /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-23T10:46:01.486 INFO:teuthology.orchestra.run.smithi097.stderr:find: ‘/var/log/rbd-target-api’gzip -5: No such file or directory --verbose 2024-04-23T10:46:01.486 INFO:teuthology.orchestra.run.smithi097.stderr: -- /var/log/ceph/cephadm.log 2024-04-23T10:46:01.486 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph.log 2024-04-23T10:46:01.487 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-23T10:46:01.487 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-mds.g.log 2024-04-23T10:46:01.487 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-mds.j.log 2024-04-23T10:46:01.487 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph.audit.log: /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-23T10:46:01.487 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-23T10:46:01.487 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-mgr.x.log 2024-04-23T10:46:01.487 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-mon.a.log 2024-04-23T10:46:01.487 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-mgr.x.log: 87.5% -- replaced with /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-23T10:46:01.487 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-osd.6.log 2024-04-23T10:46:01.487 INFO:teuthology.orchestra.run.smithi097.stderr: 90.3% -- replaced with /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-23T10:46:01.487 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-mon.a.log: 94.6% -- replaced with /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-mds.j.log.gz 2024-04-23T10:46:01.487 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- 93.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-23T10:46:01.488 INFO:teuthology.orchestra.run.smithi097.stderr: /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-23T10:46:01.488 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-mds.a.log 2024-04-23T10:46:01.488 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-23T10:46:01.488 INFO:teuthology.orchestra.run.smithi097.stderr: 84.3% -- replaced with /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-23T10:46:01.488 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-mds.d.log 2024-04-23T10:46:01.488 INFO:teuthology.orchestra.run.smithi196.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-23T10:46:01.488 INFO:teuthology.orchestra.run.smithi196.stderr:: No such file or directory 2024-04-23T10:46:01.489 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph.log 2024-04-23T10:46:01.489 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-23T10:46:01.489 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-mds.f.log 2024-04-23T10:46:01.489 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-23T10:46:01.489 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-23T10:46:01.489 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-mds.f.log: /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph.audit.log: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-23T10:46:01.489 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-mds.c.log 2024-04-23T10:46:01.489 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-mon.c.log 2024-04-23T10:46:01.490 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-osd.5.log: /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-mgr.z.log 2024-04-23T10:46:01.490 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-mds.i.log 2024-04-23T10:46:01.490 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-mgr.z.log: 87.5% -- replaced with /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-23T10:46:01.490 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-23T10:46:01.490 INFO:teuthology.orchestra.run.smithi196.stderr: 90.5% -- replaced with /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-23T10:46:01.490 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-osd.11.log 2024-04-23T10:46:01.490 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph.cephadm.log: 84.0% -- replaced with /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-23T10:46:01.490 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-osd.8.log 2024-04-23T10:46:01.490 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-osd.11.log: 92.3% -- replaced with /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-23T10:46:01.490 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-23T10:46:01.490 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-mds.h.log 2024-04-23T10:46:01.491 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-osd.10.log 2024-04-23T10:46:01.491 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-23T10:46:01.492 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-osd.10.log: 92.2% -- replaced with /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-23T10:46:01.493 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-volume.log: 91.1% -- replaced with /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-23T10:46:01.493 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-osd.9.log 2024-04-23T10:46:01.495 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-mds.l.log 2024-04-23T10:46:01.496 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-mds.e.log 2024-04-23T10:46:01.496 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph.cephadm.log: 84.0% -- replaced with /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-23T10:46:01.496 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-mds.d.log: 94.5% -- replaced with /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-23T10:46:01.505 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-23T10:46:01.521 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-volume.log: /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-mds.l.log: 91.0% -- replaced with /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-23T10:46:01.524 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-mon.b.log 2024-04-23T10:46:01.538 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-mds.k.log 2024-04-23T10:46:01.550 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-mon.b.log: /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-mds.k.log: 91.0% -- replaced with /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-23T10:46:01.553 INFO:teuthology.orchestra.run.smithi196.stderr: 94.8% -- replaced with /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-23T10:46:01.554 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-osd.9.log: 94.6% -- replaced with /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-23T10:46:01.563 INFO:teuthology.orchestra.run.smithi097.stderr: 94.6% -- replaced with /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-23T10:46:01.594 INFO:teuthology.orchestra.run.smithi196.stderr: 93.3% -- replaced with /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-23T10:46:01.600 INFO:teuthology.orchestra.run.smithi196.stderr: 93.1% -- replaced with /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-23T10:46:01.638 INFO:teuthology.orchestra.run.smithi196.stderr: 93.1% -- replaced with /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-23T10:46:01.659 INFO:teuthology.orchestra.run.smithi183.stderr: 94.6% -- replaced with /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-23T10:46:01.723 INFO:teuthology.orchestra.run.smithi183.stderr: 93.3% -- replaced with /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-23T10:46:01.805 INFO:teuthology.orchestra.run.smithi183.stderr: 93.9% -- replaced with /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-23T10:46:01.868 INFO:teuthology.orchestra.run.smithi196.stderr: 92.1% -- replaced with /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-23T10:46:01.924 INFO:teuthology.orchestra.run.smithi097.stderr: 90.3% -- replaced with /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-23T10:46:02.045 INFO:teuthology.orchestra.run.smithi183.stderr: 92.1% -- replaced with /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-23T10:46:02.209 INFO:teuthology.orchestra.run.smithi196.stderr: 92.8% -- replaced with /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-23T10:46:02.623 INFO:teuthology.orchestra.run.smithi097.stderr: 90.3% -- replaced with /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-23T10:46:02.782 INFO:teuthology.orchestra.run.smithi183.stderr: 91.8% -- replaced with /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-23T10:46:02.840 INFO:teuthology.orchestra.run.smithi196.stderr: 92.0% -- replaced with /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-23T10:46:02.881 INFO:teuthology.orchestra.run.smithi097.stderr: 92.6% -- replaced with /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-23T10:46:03.008 INFO:teuthology.orchestra.run.smithi196.stderr: 91.7% -- replaced with /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-23T10:46:03.160 INFO:teuthology.orchestra.run.smithi097.stderr: 92.7% -- replaced with /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-23T10:46:03.160 INFO:teuthology.orchestra.run.smithi097.stderr: 93.0% -- replaced with /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-23T10:46:03.281 INFO:teuthology.orchestra.run.smithi183.stderr: 92.2% -- replaced with /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-23T10:46:03.294 INFO:teuthology.orchestra.run.smithi097.stderr: 91.8% -- replaced with /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-23T10:46:03.296 INFO:teuthology.orchestra.run.smithi097.stderr: 2024-04-23T10:46:03.296 INFO:teuthology.orchestra.run.smithi097.stderr:real 0m1.842s 2024-04-23T10:46:03.296 INFO:teuthology.orchestra.run.smithi097.stderr:user 0m8.146s 2024-04-23T10:46:03.296 INFO:teuthology.orchestra.run.smithi097.stderr:sys 0m0.290s 2024-04-23T10:46:03.600 INFO:teuthology.orchestra.run.smithi183.stderr: 92.5% -- replaced with /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-23T10:46:03.720 INFO:teuthology.orchestra.run.smithi183.stderr: 92.9% -- replaced with /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-23T10:46:03.813 INFO:teuthology.orchestra.run.smithi183.stderr: 91.9% -- replaced with /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-23T10:46:03.815 INFO:teuthology.orchestra.run.smithi183.stderr: 2024-04-23T10:46:03.815 INFO:teuthology.orchestra.run.smithi183.stderr:real 0m2.360s 2024-04-23T10:46:03.815 INFO:teuthology.orchestra.run.smithi183.stderr:user 0m10.392s 2024-04-23T10:46:03.815 INFO:teuthology.orchestra.run.smithi183.stderr:sys 0m0.349s 2024-04-23T10:46:03.858 INFO:teuthology.orchestra.run.smithi196.stderr: 92.1% -- replaced with /var/log/ceph/9b22af88-0159-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-23T10:46:03.860 INFO:teuthology.orchestra.run.smithi196.stderr: 2024-04-23T10:46:03.860 INFO:teuthology.orchestra.run.smithi196.stderr:real 0m2.403s 2024-04-23T10:46:03.860 INFO:teuthology.orchestra.run.smithi196.stderr:user 0m6.645s 2024-04-23T10:46:03.860 INFO:teuthology.orchestra.run.smithi196.stderr:sys 0m0.240s 2024-04-23T10:46:03.861 INFO:tasks.cephadm:Archiving logs... 2024-04-23T10:46:03.861 DEBUG:teuthology.misc:Transferring archived files from smithi097:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-23_09:49:14-fs-wip-lusov-quiescer-distro-default-smithi/7669527/remote/smithi097/log 2024-04-23T10:46:03.862 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-23T10:46:05.068 DEBUG:teuthology.misc:Transferring archived files from smithi183:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-23_09:49:14-fs-wip-lusov-quiescer-distro-default-smithi/7669527/remote/smithi183/log 2024-04-23T10:46:05.069 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-23T10:46:06.555 DEBUG:teuthology.misc:Transferring archived files from smithi196:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-23_09:49:14-fs-wip-lusov-quiescer-distro-default-smithi/7669527/remote/smithi196/log 2024-04-23T10:46:06.556 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-23T10:46:07.574 INFO:tasks.cephadm:Removing cluster... 2024-04-23T10:46:07.574 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9b22af88-0159-11ef-bc93-c7b262605968 --force 2024-04-23T10:46:07.765 INFO:teuthology.orchestra.run.smithi097.stdout:Deleting cluster with fsid: 9b22af88-0159-11ef-bc93-c7b262605968 2024-04-23T10:46:08.038 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9b22af88-0159-11ef-bc93-c7b262605968 --force 2024-04-23T10:46:08.228 INFO:teuthology.orchestra.run.smithi183.stdout:Deleting cluster with fsid: 9b22af88-0159-11ef-bc93-c7b262605968 2024-04-23T10:46:08.550 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9b22af88-0159-11ef-bc93-c7b262605968 --force 2024-04-23T10:46:08.733 INFO:teuthology.orchestra.run.smithi196.stdout:Deleting cluster with fsid: 9b22af88-0159-11ef-bc93-c7b262605968 2024-04-23T10:46:09.009 INFO:tasks.cephadm:Removing cephadm ... 2024-04-23T10:46:09.009 DEBUG:teuthology.orchestra.run.smithi097:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-23T10:46:09.027 DEBUG:teuthology.orchestra.run.smithi183:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-23T10:46:09.043 DEBUG:teuthology.orchestra.run.smithi196:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-23T10:46:09.059 INFO:tasks.cephadm:Teardown complete 2024-04-23T10:46:09.059 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-23T10:46:09.069 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-23T10:46:09.070 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-23T10:46:09.072 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-23T10:46:09.086 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-23T10:46:09.192 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-23T10:46:09.202 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-23T10:46:09.203 DEBUG:teuthology.orchestra.run.smithi097:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-23T10:46:09.205 DEBUG:teuthology.orchestra.run.smithi183:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-23T10:46:09.207 DEBUG:teuthology.orchestra.run.smithi196:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-23T10:46:09.219 INFO:teuthology.orchestra.run.smithi183.stderr:bash: line 1: ntpq: command not found 2024-04-23T10:46:09.219 INFO:teuthology.orchestra.run.smithi097.stderr:bash: line 1: ntpq: command not found 2024-04-23T10:46:09.221 INFO:teuthology.orchestra.run.smithi196.stderr:bash: line 1: ntpq: command not found 2024-04-23T10:46:09.223 INFO:teuthology.orchestra.run.smithi183.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-23T10:46:09.223 INFO:teuthology.orchestra.run.smithi183.stdout:=============================================================================== 2024-04-23T10:46:09.223 INFO:teuthology.orchestra.run.smithi183.stdout:^- hv01.front.sepia.ceph.com 2 6 377 41 +1721us[+1721us] +/- 594ms 2024-04-23T10:46:09.223 INFO:teuthology.orchestra.run.smithi183.stdout:^* hv02.front.sepia.ceph.com 3 6 377 43 +147us[ +153us] +/- 59ms 2024-04-23T10:46:09.224 INFO:teuthology.orchestra.run.smithi183.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 41 -203us[ -203us] +/- 74ms 2024-04-23T10:46:09.224 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T10:46:09.224 INFO:teuthology.orchestra.run.smithi097.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-23T10:46:09.225 INFO:teuthology.orchestra.run.smithi097.stdout:=============================================================================== 2024-04-23T10:46:09.225 INFO:teuthology.orchestra.run.smithi097.stdout:^- hv01.front.sepia.ceph.com 2 6 377 40 +1728us[+1731us] +/- 594ms 2024-04-23T10:46:09.225 INFO:teuthology.orchestra.run.smithi097.stdout:^* hv02.front.sepia.ceph.com 3 6 377 39 +151us[ +153us] +/- 59ms 2024-04-23T10:46:09.225 INFO:teuthology.orchestra.run.smithi097.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 45 -195us[ -193us] +/- 74ms 2024-04-23T10:46:09.225 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T10:46:09.226 INFO:teuthology.orchestra.run.smithi196.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-23T10:46:09.226 INFO:teuthology.orchestra.run.smithi196.stdout:=============================================================================== 2024-04-23T10:46:09.226 INFO:teuthology.orchestra.run.smithi196.stdout:^- hv01.front.sepia.ceph.com 2 6 377 40 +1734us[+1734us] +/- 594ms 2024-04-23T10:46:09.226 INFO:teuthology.orchestra.run.smithi196.stdout:^* hv02.front.sepia.ceph.com 3 7 377 46 +144us[ +147us] +/- 59ms 2024-04-23T10:46:09.226 INFO:teuthology.orchestra.run.smithi196.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 45 -208us[ -208us] +/- 74ms 2024-04-23T10:46:09.226 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T10:46:09.226 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-23T10:46:09.253 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-23T10:46:09.253 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-23T10:46:09.264 DEBUG:teuthology.orchestra.run.smithi097:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-23T10:46:09.343 DEBUG:teuthology.orchestra.run.smithi183:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-23T10:46:09.408 DEBUG:teuthology.orchestra.run.smithi196:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-23T10:46:09.472 DEBUG:teuthology.orchestra.run.smithi097:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-23T10:46:09.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T10:46:09.501 DEBUG:teuthology.orchestra.run.smithi183:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-23T10:46:09.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T10:46:09.528 DEBUG:teuthology.orchestra.run.smithi196:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-23T10:46:09.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T10:46:09.554 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-23T10:46:09.564 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-23T10:46:09.589 INFO:teuthology.task.internal:Duration was 2768.701962 seconds 2024-04-23T10:46:09.589 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-23T10:46:09.599 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-23T10:46:09.599 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-23T10:46:09.601 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-23T10:46:09.604 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-23T10:46:09.637 INFO:teuthology.orchestra.run.smithi097.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-23T10:46:09.638 INFO:teuthology.orchestra.run.smithi183.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-23T10:46:09.640 INFO:teuthology.orchestra.run.smithi196.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-23T10:46:09.941 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-23T10:46:09.941 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi097.front.sepia.ceph.com 2024-04-23T10:46:09.942 DEBUG:teuthology.orchestra.run.smithi097:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-23T10:46:09.962 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi183.front.sepia.ceph.com 2024-04-23T10:46:09.962 DEBUG:teuthology.orchestra.run.smithi183:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-23T10:46:10.007 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi196.front.sepia.ceph.com 2024-04-23T10:46:10.007 DEBUG:teuthology.orchestra.run.smithi196:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-23T10:46:10.026 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-23T10:46:10.027 DEBUG:teuthology.orchestra.run.smithi097:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-23T10:46:10.029 DEBUG:teuthology.orchestra.run.smithi183:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-23T10:46:10.050 DEBUG:teuthology.orchestra.run.smithi196:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-23T10:46:10.090 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-23T10:46:10.091 DEBUG:teuthology.orchestra.run.smithi097:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-23T10:46:10.095 DEBUG:teuthology.orchestra.run.smithi183:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-23T10:46:10.115 DEBUG:teuthology.orchestra.run.smithi196:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-23T10:46:10.489 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-23T10:46:10.500 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-23T10:46:10.501 DEBUG:teuthology.orchestra.run.smithi097:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-23T10:46:10.523 DEBUG:teuthology.orchestra.run.smithi183:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-23T10:46:10.547 DEBUG:teuthology.orchestra.run.smithi196:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-23T10:46:10.571 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-23T10:46:10.582 DEBUG:teuthology.orchestra.run.smithi097:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-23T10:46:10.584 DEBUG:teuthology.orchestra.run.smithi183:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-23T10:46:10.590 DEBUG:teuthology.orchestra.run.smithi196:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-23T10:46:10.603 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern = core 2024-04-23T10:46:10.609 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern = core 2024-04-23T10:46:10.633 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern = core 2024-04-23T10:46:10.655 DEBUG:teuthology.orchestra.run.smithi097:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-23T10:46:10.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T10:46:10.669 DEBUG:teuthology.orchestra.run.smithi183:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-23T10:46:10.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T10:46:10.685 DEBUG:teuthology.orchestra.run.smithi196:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-23T10:46:10.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T10:46:10.709 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-23T10:46:10.720 INFO:teuthology.task.internal:Transferring archived files... 2024-04-23T10:46:10.721 DEBUG:teuthology.misc:Transferring archived files from smithi097:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-23_09:49:14-fs-wip-lusov-quiescer-distro-default-smithi/7669527/remote/smithi097 2024-04-23T10:46:10.721 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-23T10:46:10.761 DEBUG:teuthology.misc:Transferring archived files from smithi183:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-23_09:49:14-fs-wip-lusov-quiescer-distro-default-smithi/7669527/remote/smithi183 2024-04-23T10:46:10.761 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-23T10:46:10.796 DEBUG:teuthology.misc:Transferring archived files from smithi196:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-23_09:49:14-fs-wip-lusov-quiescer-distro-default-smithi/7669527/remote/smithi196 2024-04-23T10:46:10.796 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-23T10:46:10.831 INFO:teuthology.task.internal:Removing archive directory... 2024-04-23T10:46:10.831 DEBUG:teuthology.orchestra.run.smithi097:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-23T10:46:10.833 DEBUG:teuthology.orchestra.run.smithi183:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-23T10:46:10.835 DEBUG:teuthology.orchestra.run.smithi196:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-23T10:46:10.879 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-23T10:46:10.892 INFO:teuthology.task.internal:Not uploading archives. 2024-04-23T10:46:10.892 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-23T10:46:10.903 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-23T10:46:10.904 DEBUG:teuthology.orchestra.run.smithi097:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-23T10:46:10.906 DEBUG:teuthology.orchestra.run.smithi183:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-23T10:46:10.908 DEBUG:teuthology.orchestra.run.smithi196:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-23T10:46:10.918 INFO:teuthology.orchestra.run.smithi097.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 23 10:46 /home/ubuntu/cephtest 2024-04-23T10:46:10.920 INFO:teuthology.orchestra.run.smithi183.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 23 10:46 /home/ubuntu/cephtest 2024-04-23T10:46:10.932 INFO:teuthology.orchestra.run.smithi196.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 23 10:46 /home/ubuntu/cephtest 2024-04-23T10:46:10.933 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-23T10:46:10.945 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-23T10:46:10.971 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/fs/test_o_trunc}} duration: 2768.701961517334 flavor: default owner: scheduled_leonidus@teuthology success: true 2024-04-23T10:46:10.972 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-23T10:46:11.051 INFO:teuthology.run:pass